2023-12-07T09:50:09.448 INFO:root:teuthology version: 0.0.1.dev214+gf2de108 2023-12-07T09:50:09.448 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} --verbose --name vshankar-2023-12-07_09:28:49-fs-wip-vshankar-testing-20231127.102654-r2-distro-default-smithi --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2023-12-07_09:28:49-fs-wip-vshankar-testing-20231127.102654-r2-distro-default-smithi/7481878 -- /home/teuthworker/archive/vshankar-2023-12-07_09:28:49-fs-wip-vshankar-testing-20231127.102654-r2-distro-default-smithi/7481878/orig.config.yaml 2023-12-07T09:50:09.507 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-07T09:50:09.605 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2023-12-07_09:28:49-fs-wip-vshankar-testing-20231127.102654-r2-distro-default-smithi/7481878 branch: wip-vshankar-testing-20231127.102654-r2 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} email: vshankar@redhat.com first_in_suite: false job_id: '7481878' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2023-12-07_09:28:49-fs-wip-vshankar-testing-20231127.102654-r2-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-vshankar-testing-20231127.102654-r2 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20231127.102654-r2 sha1: 3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 owner: scheduled_vshankar@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1098 sha1: 3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 sleep_before_teardown: 0 subset: 854/1024 suite: fs suite_branch: wip-vshankar-testing-20231127.102654-r2 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi119.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi153.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi159.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: f2de108beb0a6dde7f3e6c96039071828a1f6269 timestamp: 2023-12-07_09:28:49 tube: smithi use_shaman: true user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-12-07T09:50:09.606 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/qa; will attempt to use it 2023-12-07T09:50:09.606 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/qa/tasks 2023-12-07T09:50:09.607 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-12-07T09:50:09.608 INFO:teuthology.task.internal:Checking packages... 2023-12-07T09:50:09.632 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '3bd0bfa458b4399e7266c56d8215a2cf3a6091a0' 2023-12-07T09:50:09.632 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-12-07T09:50:09.632 INFO:teuthology.packaging:ref: None 2023-12-07T09:50:09.632 INFO:teuthology.packaging:tag: None 2023-12-07T09:50:09.632 INFO:teuthology.packaging:branch: wip-vshankar-testing-20231127.102654-r2 2023-12-07T09:50:09.632 INFO:teuthology.packaging:sha1: 3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 2023-12-07T09:50:09.633 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-vshankar-testing-20231127.102654-r2 2023-12-07T09:50:09.855 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-18.g3bd0bfa4 2023-12-07T09:50:09.855 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-12-07T09:50:09.862 INFO:teuthology.task.internal:no buildpackages task found 2023-12-07T09:50:09.862 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-12-07T09:50:09.871 INFO:teuthology.task.internal:Saving configuration 2023-12-07T09:50:09.896 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-12-07T09:50:09.902 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-07T09:50:09.928 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-07_09:28:49-fs-wip-vshankar-testing-20231127.102654-r2-distro-default-smithi/7481878', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-07 09:44:39.115502', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-07T09:50:09.949 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi153.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-07_09:28:49-fs-wip-vshankar-testing-20231127.102654-r2-distro-default-smithi/7481878', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-07 09:44:39.117617', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-07T09:50:09.967 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-07_09:28:49-fs-wip-vshankar-testing-20231127.102654-r2-distro-default-smithi/7481878', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-07 09:44:39.116665', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-07T09:50:09.968 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-12-07T09:50:09.973 INFO:teuthology.task.internal:roles: ubuntu@smithi119.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-12-07T09:50:09.973 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-12-07T09:50:09.973 INFO:teuthology.task.internal:roles: ubuntu@smithi159.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-12-07T09:50:09.973 INFO:teuthology.run_tasks:Running task console_log... 2023-12-07T09:50:10.104 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb0b1cafa60>, signals=[15]) 2023-12-07T09:50:10.104 INFO:teuthology.run_tasks:Running task internal.connect... 2023-12-07T09:50:10.111 INFO:teuthology.task.internal:Opening connections... 2023-12-07T09:50:10.111 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi119.front.sepia.ceph.com 2023-12-07T09:50:10.112 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-07T09:50:10.180 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi153.front.sepia.ceph.com 2023-12-07T09:50:10.181 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-07T09:50:10.248 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi159.front.sepia.ceph.com 2023-12-07T09:50:10.249 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-07T09:50:10.364 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-12-07T09:50:10.370 DEBUG:teuthology.orchestra.run.smithi119:> uname -m 2023-12-07T09:50:10.390 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2023-12-07T09:50:10.391 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2023-12-07T09:50:10.449 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="Red Hat Enterprise Linux" 2023-12-07T09:50:10.450 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="8.6 (Ootpa)" 2023-12-07T09:50:10.450 INFO:teuthology.orchestra.run.smithi119.stdout:ID="rhel" 2023-12-07T09:50:10.450 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="fedora" 2023-12-07T09:50:10.450 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="8.6" 2023-12-07T09:50:10.450 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el8" 2023-12-07T09:50:10.450 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-07T09:50:10.450 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31" 2023-12-07T09:50:10.450 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-07T09:50:10.450 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://www.redhat.com/" 2023-12-07T09:50:10.450 INFO:teuthology.orchestra.run.smithi119.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-07T09:50:10.450 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-07T09:50:10.450 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T09:50:10.450 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-07T09:50:10.451 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-07T09:50:10.451 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-07T09:50:10.451 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-07T09:50:10.451 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2023-12-07T09:50:10.478 DEBUG:teuthology.orchestra.run.smithi153:> uname -m 2023-12-07T09:50:10.499 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64 2023-12-07T09:50:10.499 DEBUG:teuthology.orchestra.run.smithi153:> cat /etc/os-release 2023-12-07T09:50:10.557 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="Red Hat Enterprise Linux" 2023-12-07T09:50:10.558 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="8.6 (Ootpa)" 2023-12-07T09:50:10.558 INFO:teuthology.orchestra.run.smithi153.stdout:ID="rhel" 2023-12-07T09:50:10.558 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="fedora" 2023-12-07T09:50:10.558 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="8.6" 2023-12-07T09:50:10.558 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el8" 2023-12-07T09:50:10.558 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-07T09:50:10.558 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31" 2023-12-07T09:50:10.558 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-07T09:50:10.558 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://www.redhat.com/" 2023-12-07T09:50:10.558 INFO:teuthology.orchestra.run.smithi153.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-07T09:50:10.558 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-07T09:50:10.558 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-12-07T09:50:10.559 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-07T09:50:10.559 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-07T09:50:10.559 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-07T09:50:10.559 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-07T09:50:10.559 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2023-12-07T09:50:10.578 DEBUG:teuthology.orchestra.run.smithi159:> uname -m 2023-12-07T09:50:10.598 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64 2023-12-07T09:50:10.599 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release 2023-12-07T09:50:10.662 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="Red Hat Enterprise Linux" 2023-12-07T09:50:10.662 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="8.6 (Ootpa)" 2023-12-07T09:50:10.662 INFO:teuthology.orchestra.run.smithi159.stdout:ID="rhel" 2023-12-07T09:50:10.662 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="fedora" 2023-12-07T09:50:10.662 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="8.6" 2023-12-07T09:50:10.662 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el8" 2023-12-07T09:50:10.663 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-07T09:50:10.663 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31" 2023-12-07T09:50:10.663 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-07T09:50:10.663 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://www.redhat.com/" 2023-12-07T09:50:10.663 INFO:teuthology.orchestra.run.smithi159.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-07T09:50:10.663 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-07T09:50:10.663 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-12-07T09:50:10.663 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-07T09:50:10.663 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-07T09:50:10.663 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-07T09:50:10.663 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-07T09:50:10.664 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2023-12-07T09:50:10.687 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-12-07T09:50:10.696 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-12-07T09:50:10.701 INFO:teuthology.task.internal:Checking for old test directory... 2023-12-07T09:50:10.701 DEBUG:teuthology.orchestra.run.smithi119:> test '!' -e /home/ubuntu/cephtest 2023-12-07T09:50:10.704 DEBUG:teuthology.orchestra.run.smithi153:> test '!' -e /home/ubuntu/cephtest 2023-12-07T09:50:10.706 DEBUG:teuthology.orchestra.run.smithi159:> test '!' -e /home/ubuntu/cephtest 2023-12-07T09:50:10.725 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-12-07T09:50:10.730 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-12-07T09:50:10.730 DEBUG:teuthology.orchestra.run.smithi119:> test -z $(ls -A /var/lib/ceph) 2023-12-07T09:50:10.763 DEBUG:teuthology.orchestra.run.smithi153:> test -z $(ls -A /var/lib/ceph) 2023-12-07T09:50:10.765 DEBUG:teuthology.orchestra.run.smithi159:> test -z $(ls -A /var/lib/ceph) 2023-12-07T09:50:10.805 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-12-07T09:50:10.881 INFO:teuthology.run_tasks:Running task kernel... 2023-12-07T09:50:10.896 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-12-07T09:50:10.896 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-12-07T09:50:10.897 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-12-07T09:50:10.897 DEBUG:teuthology.orchestra.run.smithi119:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-07T09:50:10.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-07T09:50:10.916 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-12-07T09:50:11.184 DEBUG:teuthology.task.kernel:sha1 for client.0 is 9471594ae3a4c6992c4327df9a789df8f2b6aeae 2023-12-07T09:50:11.185 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "9471594ae3a4c6992c4327df9a789df8f2b6aeae"... 2023-12-07T09:50:11.185 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2023-12-07T09:50:11.204 INFO:teuthology.orchestra.run.smithi119.stdout:4.18.0-372.9.1.el8.x86_64 2023-12-07T09:50:11.205 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs 9471594ae3a4c6992c4327df9a789df8f2b6aeae 2023-12-07T09:50:11.205 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-12-07T09:50:11.205 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum upgrade -y linux-firmware 2023-12-07T09:50:12.374 INFO:teuthology.orchestra.run.smithi119.stdout:Updating Subscription Management repositories. 2023-12-07T09:50:12.375 INFO:teuthology.orchestra.run.smithi119.stdout:Unable to read consumer identity 2023-12-07T09:50:13.777 INFO:teuthology.orchestra.run.smithi119.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-12-07T09:50:14.444 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 25 MB/s | 16 MB 00:00 2023-12-07T09:50:17.978 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 414 kB/s | 24 kB 00:00 2023-12-07T09:50:18.757 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-12-07T09:50:18.758 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2023-12-07T09:50:18.758 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-12-07T09:50:18.878 INFO:teuthology.task.kernel:Downloading kernel 9471594ae3a4c6992c4327df9a789df8f2b6aeae on client.0... 2023-12-07T09:50:18.879 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=9471594ae3a4c6992c4327df9a789df8f2b6aeae 2023-12-07T09:50:19.028 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/9471594ae3a4c6992c4327df9a789df8f2b6aeae/centos/8/flavors/default/x86_64/ 2023-12-07T09:50:19.028 DEBUG:teuthology.orchestra.run.smithi119:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_g9471594ae3a4-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/9471594ae3a4c6992c4327df9a789df8f2b6aeae/centos/8/flavors/default/x86_64/ --input-file=- 2023-12-07T09:50:19.033 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi119.front.sepia.ceph.com to complete... 2023-12-07T09:50:32.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07 09:50:32 URL:https://2.chacra.ceph.com/r/kernel/testing/9471594ae3a4c6992c4327df9a789df8f2b6aeae/centos/8/flavors/default/x86_64/kernel-6.6.0_g9471594ae3a4-1.x86_64.rpm [383178600/383178600] -> "/tmp/kernel.x86_64.rpm" [1] 2023-12-07T09:50:32.652 INFO:teuthology.orchestra.run.smithi119.stderr:FINISHED --2023-12-07 09:50:32-- 2023-12-07T09:50:32.653 INFO:teuthology.orchestra.run.smithi119.stderr:Total wall clock time: 14s 2023-12-07T09:50:32.653 INFO:teuthology.orchestra.run.smithi119.stderr:Downloaded: 1 files, 365M in 13s (27.5 MB/s) 2023-12-07T09:50:32.654 INFO:teuthology.task.kernel:Installing kernel 9471594ae3a4c6992c4327df9a789df8f2b6aeae on client.0... 2023-12-07T09:50:32.654 DEBUG:teuthology.orchestra.run.smithi119:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-12-07T09:50:34.284 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying... ######################################## 2023-12-07T09:50:34.361 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing... ######################################## 2023-12-07T09:50:34.445 INFO:teuthology.orchestra.run.smithi119.stdout:Updating / installing... 2023-12-07T09:51:12.893 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-6.6.0_g9471594ae3a4-1 ######################################## 2023-12-07T09:51:14.343 INFO:teuthology.orchestra.run.smithi119.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-12-07T09:51:39.654 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi119.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-12-07T09:51:39.654 DEBUG:teuthology.orchestra.run.smithi119:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-12-07T09:51:39.714 INFO:teuthology.orchestra.run.smithi119.stdout:/boot/System.map-6.6.0-g9471594ae3a4 2023-12-07T09:51:39.715 INFO:teuthology.orchestra.run.smithi119.stdout:/boot/config-6.6.0-g9471594ae3a4 2023-12-07T09:51:39.715 INFO:teuthology.orchestra.run.smithi119.stdout:/boot/vmlinuz-6.6.0-g9471594ae3a4 2023-12-07T09:51:39.715 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4 2023-12-07T09:51:39.715 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel 2023-12-07T09:51:39.715 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch 2023-12-07T09:51:39.715 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86 2023-12-07T09:51:39.715 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/crypto 2023-12-07T09:51:39.715 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-12-07T09:51:39.715 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-12-07T09:51:39.715 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-12-07T09:51:39.715 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-12-07T09:51:39.715 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-12-07T09:51:39.716 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-12-07T09:51:39.716 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-12-07T09:51:39.716 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/crypto/crc32c-intel.ko 2023-12-07T09:51:39.716 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-12-07T09:51:39.716 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-12-07T09:51:39.716 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-12-07T09:51:39.716 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-12-07T09:51:39.716 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/crypto/serpent-avx2.ko 2023-12-07T09:51:39.716 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-12-07T09:51:39.716 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-12-07T09:51:39.716 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-12-07T09:51:39.716 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-12-07T09:51:39.716 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-12-07T09:51:39.716 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-12-07T09:51:39.717 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-12-07T09:51:39.717 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/kernel 2023-12-07T09:51:39.717 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/kernel/cpu 2023-12-07T09:51:39.717 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/kernel/cpu/mce 2023-12-07T09:51:39.717 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-12-07T09:51:39.717 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/kvm 2023-12-07T09:51:39.717 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/kvm/kvm-amd.ko 2023-12-07T09:51:39.717 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/kvm/kvm-intel.ko 2023-12-07T09:51:39.717 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/arch/x86/kvm/kvm.ko 2023-12-07T09:51:39.717 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto 2023-12-07T09:51:39.717 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/ansi_cprng.ko 2023-12-07T09:51:39.717 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/anubis.ko 2023-12-07T09:51:39.717 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/arc4.ko 2023-12-07T09:51:39.717 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/async_tx 2023-12-07T09:51:39.717 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/async_tx/async_memcpy.ko 2023-12-07T09:51:39.718 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/async_tx/async_pq.ko 2023-12-07T09:51:39.718 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/async_tx/async_raid6_recov.ko 2023-12-07T09:51:39.718 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/async_tx/async_tx.ko 2023-12-07T09:51:39.718 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/async_tx/async_xor.ko 2023-12-07T09:51:39.718 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/async_tx/raid6test.ko 2023-12-07T09:51:39.718 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/authenc.ko 2023-12-07T09:51:39.718 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/authencesn.ko 2023-12-07T09:51:39.718 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/blake2b_generic.ko 2023-12-07T09:51:39.718 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/blowfish_common.ko 2023-12-07T09:51:39.718 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/blowfish_generic.ko 2023-12-07T09:51:39.718 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/camellia_generic.ko 2023-12-07T09:51:39.718 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/cast5_generic.ko 2023-12-07T09:51:39.718 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/cast6_generic.ko 2023-12-07T09:51:39.718 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/cast_common.ko 2023-12-07T09:51:39.719 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/ccm.ko 2023-12-07T09:51:39.719 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/cmac.ko 2023-12-07T09:51:39.719 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/crc32_generic.ko 2023-12-07T09:51:39.719 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/crypto_engine.ko 2023-12-07T09:51:39.719 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/crypto_user.ko 2023-12-07T09:51:39.719 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/cts.ko 2023-12-07T09:51:39.719 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/des_generic.ko 2023-12-07T09:51:39.719 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/dh_generic.ko 2023-12-07T09:51:39.719 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/echainiv.ko 2023-12-07T09:51:39.719 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/essiv.ko 2023-12-07T09:51:39.719 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/fcrypt.ko 2023-12-07T09:51:39.719 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/khazad.ko 2023-12-07T09:51:39.719 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/lz4.ko 2023-12-07T09:51:39.719 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/lz4hc.ko 2023-12-07T09:51:39.720 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/md4.ko 2023-12-07T09:51:39.720 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/michael_mic.ko 2023-12-07T09:51:39.720 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/pcbc.ko 2023-12-07T09:51:39.720 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/pcrypt.ko 2023-12-07T09:51:39.720 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/rmd160.ko 2023-12-07T09:51:39.720 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/seed.ko 2023-12-07T09:51:39.720 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/serpent_generic.ko 2023-12-07T09:51:39.720 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/tcrypt.ko 2023-12-07T09:51:39.720 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/tea.ko 2023-12-07T09:51:39.720 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/twofish_common.ko 2023-12-07T09:51:39.720 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/twofish_generic.ko 2023-12-07T09:51:39.720 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/vmac.ko 2023-12-07T09:51:39.720 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/wp512.ko 2023-12-07T09:51:39.720 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/xcbc.ko 2023-12-07T09:51:39.720 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/xor.ko 2023-12-07T09:51:39.721 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/crypto/xxhash_generic.ko 2023-12-07T09:51:39.721 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers 2023-12-07T09:51:39.721 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/acpi 2023-12-07T09:51:39.721 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/acpi/acpi_ipmi.ko 2023-12-07T09:51:39.721 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/acpi/acpi_pad.ko 2023-12-07T09:51:39.721 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/acpi/custom_method.ko 2023-12-07T09:51:39.721 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/acpi/ec_sys.ko 2023-12-07T09:51:39.721 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/acpi/platform_profile.ko 2023-12-07T09:51:39.721 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/acpi/sbs.ko 2023-12-07T09:51:39.721 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/acpi/sbshc.ko 2023-12-07T09:51:39.721 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/acpi/video.ko 2023-12-07T09:51:39.721 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata 2023-12-07T09:51:39.721 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/acard-ahci.ko 2023-12-07T09:51:39.721 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/ahci_platform.ko 2023-12-07T09:51:39.721 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/ata_generic.ko 2023-12-07T09:51:39.722 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/libahci_platform.ko 2023-12-07T09:51:39.722 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_acpi.ko 2023-12-07T09:51:39.722 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_ali.ko 2023-12-07T09:51:39.722 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_amd.ko 2023-12-07T09:51:39.722 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_artop.ko 2023-12-07T09:51:39.722 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_atiixp.ko 2023-12-07T09:51:39.722 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_atp867x.ko 2023-12-07T09:51:39.722 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_cmd640.ko 2023-12-07T09:51:39.722 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_cmd64x.ko 2023-12-07T09:51:39.722 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_cypress.ko 2023-12-07T09:51:39.722 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_efar.ko 2023-12-07T09:51:39.725 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_hpt366.ko 2023-12-07T09:51:39.725 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_hpt37x.ko 2023-12-07T09:51:39.725 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_hpt3x2n.ko 2023-12-07T09:51:39.725 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_hpt3x3.ko 2023-12-07T09:51:39.725 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_it8213.ko 2023-12-07T09:51:39.725 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_it821x.ko 2023-12-07T09:51:39.725 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_jmicron.ko 2023-12-07T09:51:39.725 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_marvell.ko 2023-12-07T09:51:39.725 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_mpiix.ko 2023-12-07T09:51:39.725 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_netcell.ko 2023-12-07T09:51:39.725 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_ninja32.ko 2023-12-07T09:51:39.726 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_ns87410.ko 2023-12-07T09:51:39.726 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_ns87415.ko 2023-12-07T09:51:39.726 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_oldpiix.ko 2023-12-07T09:51:39.726 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_opti.ko 2023-12-07T09:51:39.726 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_optidma.ko 2023-12-07T09:51:39.726 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_pcmcia.ko 2023-12-07T09:51:39.726 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_pdc2027x.ko 2023-12-07T09:51:39.726 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-12-07T09:51:39.726 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_piccolo.ko 2023-12-07T09:51:39.726 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_rdc.ko 2023-12-07T09:51:39.726 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_sch.ko 2023-12-07T09:51:39.726 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_serverworks.ko 2023-12-07T09:51:39.726 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_sil680.ko 2023-12-07T09:51:39.727 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_sis.ko 2023-12-07T09:51:39.727 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_sl82c105.ko 2023-12-07T09:51:39.727 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_triflex.ko 2023-12-07T09:51:39.727 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pata_via.ko 2023-12-07T09:51:39.727 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/pdc_adma.ko 2023-12-07T09:51:39.727 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/sata_inic162x.ko 2023-12-07T09:51:39.727 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/sata_mv.ko 2023-12-07T09:51:39.727 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/sata_nv.ko 2023-12-07T09:51:39.727 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/sata_promise.ko 2023-12-07T09:51:39.727 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/sata_qstor.ko 2023-12-07T09:51:39.727 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/sata_sil.ko 2023-12-07T09:51:39.727 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/sata_sil24.ko 2023-12-07T09:51:39.727 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/sata_sis.ko 2023-12-07T09:51:39.727 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/sata_svw.ko 2023-12-07T09:51:39.728 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/sata_sx4.ko 2023-12-07T09:51:39.728 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/sata_uli.ko 2023-12-07T09:51:39.728 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/sata_via.ko 2023-12-07T09:51:39.728 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ata/sata_vsc.ko 2023-12-07T09:51:39.728 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/auxdisplay 2023-12-07T09:51:39.728 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/auxdisplay/ks0108.ko 2023-12-07T09:51:39.728 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/base 2023-12-07T09:51:39.728 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/base/regmap 2023-12-07T09:51:39.728 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/base/regmap/regmap-i2c.ko 2023-12-07T09:51:39.728 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/base/regmap/regmap-mmio.ko 2023-12-07T09:51:39.728 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/bcma 2023-12-07T09:51:39.728 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/bcma/bcma.ko 2023-12-07T09:51:39.728 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/block 2023-12-07T09:51:39.728 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/block/aoe 2023-12-07T09:51:39.728 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/block/aoe/aoe.ko 2023-12-07T09:51:39.729 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/block/brd.ko 2023-12-07T09:51:39.729 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/block/drbd 2023-12-07T09:51:39.729 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/block/drbd/drbd.ko 2023-12-07T09:51:39.729 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/block/floppy.ko 2023-12-07T09:51:39.729 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/block/loop.ko 2023-12-07T09:51:39.729 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/block/mtip32xx 2023-12-07T09:51:39.729 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-12-07T09:51:39.729 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/block/nbd.ko 2023-12-07T09:51:39.729 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/block/null_blk 2023-12-07T09:51:39.729 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/block/null_blk/null_blk.ko 2023-12-07T09:51:39.729 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/block/pktcdvd.ko 2023-12-07T09:51:39.729 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/block/rbd.ko 2023-12-07T09:51:39.729 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/block/virtio_blk.ko 2023-12-07T09:51:39.729 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/block/xen-blkback 2023-12-07T09:51:39.729 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-12-07T09:51:39.730 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/block/xen-blkfront.ko 2023-12-07T09:51:39.730 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/block/zram 2023-12-07T09:51:39.730 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/block/zram/zram.ko 2023-12-07T09:51:39.730 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char 2023-12-07T09:51:39.730 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/hangcheck-timer.ko 2023-12-07T09:51:39.730 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/hw_random 2023-12-07T09:51:39.730 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/hw_random/amd-rng.ko 2023-12-07T09:51:39.730 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/hw_random/intel-rng.ko 2023-12-07T09:51:39.730 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-12-07T09:51:39.730 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/hw_random/via-rng.ko 2023-12-07T09:51:39.730 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/hw_random/virtio-rng.ko 2023-12-07T09:51:39.730 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/ipmi 2023-12-07T09:51:39.730 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-12-07T09:51:39.730 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-12-07T09:51:39.731 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-12-07T09:51:39.731 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/ipmi/ipmi_si.ko 2023-12-07T09:51:39.731 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-12-07T09:51:39.731 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-12-07T09:51:39.731 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/lp.ko 2023-12-07T09:51:39.731 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/mwave 2023-12-07T09:51:39.731 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/mwave/mwave.ko 2023-12-07T09:51:39.731 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/ppdev.ko 2023-12-07T09:51:39.731 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/tlclk.ko 2023-12-07T09:51:39.731 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/tpm 2023-12-07T09:51:39.731 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/tpm/tpm.ko 2023-12-07T09:51:39.731 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/tpm/tpm_atmel.ko 2023-12-07T09:51:39.731 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/tpm/tpm_infineon.ko 2023-12-07T09:51:39.731 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/tpm/tpm_nsc.ko 2023-12-07T09:51:39.731 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/tpm/tpm_tis.ko 2023-12-07T09:51:39.732 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-12-07T09:51:39.732 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/uv_mmtimer.ko 2023-12-07T09:51:39.732 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/char/virtio_console.ko 2023-12-07T09:51:39.732 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/cpufreq 2023-12-07T09:51:39.732 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-12-07T09:51:39.732 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-12-07T09:51:39.732 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/cpufreq/p4-clockmod.ko 2023-12-07T09:51:39.732 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-12-07T09:51:39.732 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/cpufreq/powernow-k8.ko 2023-12-07T09:51:39.732 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/cpufreq/speedstep-lib.ko 2023-12-07T09:51:39.732 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/crypto 2023-12-07T09:51:39.732 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/crypto/ccp 2023-12-07T09:51:39.732 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-12-07T09:51:39.732 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/crypto/ccp/ccp.ko 2023-12-07T09:51:39.732 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/crypto/intel 2023-12-07T09:51:39.733 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/crypto/intel/qat 2023-12-07T09:51:39.733 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/crypto/intel/qat/qat_common 2023-12-07T09:51:39.733 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-12-07T09:51:39.733 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-12-07T09:51:39.733 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-12-07T09:51:39.733 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/crypto/padlock-aes.ko 2023-12-07T09:51:39.733 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/crypto/padlock-sha.ko 2023-12-07T09:51:39.733 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/crypto/virtio 2023-12-07T09:51:39.733 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-12-07T09:51:39.733 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/dca 2023-12-07T09:51:39.733 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/dca/dca.ko 2023-12-07T09:51:39.733 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/devfreq 2023-12-07T09:51:39.733 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-12-07T09:51:39.733 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/dma 2023-12-07T09:51:39.734 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/dma/dw 2023-12-07T09:51:39.734 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/dma/dw/dw_dmac.ko 2023-12-07T09:51:39.734 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/dma/ioat 2023-12-07T09:51:39.734 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/dma/ioat/ioatdma.ko 2023-12-07T09:51:39.734 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/edac 2023-12-07T09:51:39.734 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/edac/amd64_edac.ko 2023-12-07T09:51:39.734 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/edac/e752x_edac.ko 2023-12-07T09:51:39.734 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/edac/edac_mce_amd.ko 2023-12-07T09:51:39.734 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/edac/i3000_edac.ko 2023-12-07T09:51:39.734 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/edac/i3200_edac.ko 2023-12-07T09:51:39.734 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/edac/i5100_edac.ko 2023-12-07T09:51:39.734 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/edac/i5400_edac.ko 2023-12-07T09:51:39.734 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/edac/i7300_edac.ko 2023-12-07T09:51:39.734 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/edac/i7core_edac.ko 2023-12-07T09:51:39.735 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/edac/i82975x_edac.ko 2023-12-07T09:51:39.735 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/edac/ie31200_edac.ko 2023-12-07T09:51:39.735 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/edac/sb_edac.ko 2023-12-07T09:51:39.735 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/edac/x38_edac.ko 2023-12-07T09:51:39.735 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/firmware 2023-12-07T09:51:39.735 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/firmware/edd.ko 2023-12-07T09:51:39.735 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/firmware/iscsi_ibft.ko 2023-12-07T09:51:39.735 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/gpio 2023-12-07T09:51:39.735 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/gpio/gpio-ich.ko 2023-12-07T09:51:39.735 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/gpio/gpio-viperboard.ko 2023-12-07T09:51:39.735 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid 2023-12-07T09:51:39.735 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-appleir.ko 2023-12-07T09:51:39.735 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-aureal.ko 2023-12-07T09:51:39.735 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-axff.ko 2023-12-07T09:51:39.736 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-dr.ko 2023-12-07T09:51:39.736 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-elecom.ko 2023-12-07T09:51:39.736 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-elo.ko 2023-12-07T09:51:39.736 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-emsff.ko 2023-12-07T09:51:39.736 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-gaff.ko 2023-12-07T09:51:39.736 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-gt683r.ko 2023-12-07T09:51:39.736 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-gyration.ko 2023-12-07T09:51:39.736 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-holtek-kbd.ko 2023-12-07T09:51:39.736 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-holtek-mouse.ko 2023-12-07T09:51:39.736 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-holtekff.ko 2023-12-07T09:51:39.736 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-hyperv.ko 2023-12-07T09:51:39.736 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-icade.ko 2023-12-07T09:51:39.736 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-keytouch.ko 2023-12-07T09:51:39.736 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-kye.ko 2023-12-07T09:51:39.736 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-lcpower.ko 2023-12-07T09:51:39.737 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-led.ko 2023-12-07T09:51:39.737 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-lenovo.ko 2023-12-07T09:51:39.737 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-logitech-dj.ko 2023-12-07T09:51:39.737 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-12-07T09:51:39.737 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-multitouch.ko 2023-12-07T09:51:39.737 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-ortek.ko 2023-12-07T09:51:39.737 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-penmount.ko 2023-12-07T09:51:39.737 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-petalynx.ko 2023-12-07T09:51:39.737 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-picolcd.ko 2023-12-07T09:51:39.737 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-pl.ko 2023-12-07T09:51:39.737 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-plantronics.ko 2023-12-07T09:51:39.737 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-primax.ko 2023-12-07T09:51:39.737 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-rmi.ko 2023-12-07T09:51:39.737 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-roccat-arvo.ko 2023-12-07T09:51:39.737 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-roccat-common.ko 2023-12-07T09:51:39.738 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-roccat-isku.ko 2023-12-07T09:51:39.738 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-roccat-kone.ko 2023-12-07T09:51:39.738 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-12-07T09:51:39.738 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-roccat-konepure.ko 2023-12-07T09:51:39.738 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-12-07T09:51:39.738 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-roccat-lua.ko 2023-12-07T09:51:39.738 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-roccat-pyra.ko 2023-12-07T09:51:39.738 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-roccat-ryos.ko 2023-12-07T09:51:39.738 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-roccat-savu.ko 2023-12-07T09:51:39.738 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-roccat.ko 2023-12-07T09:51:39.738 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-saitek.ko 2023-12-07T09:51:39.738 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-samsung.ko 2023-12-07T09:51:39.738 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-sensor-hub.ko 2023-12-07T09:51:39.738 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-sjoy.ko 2023-12-07T09:51:39.738 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-sony.ko 2023-12-07T09:51:39.739 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-speedlink.ko 2023-12-07T09:51:39.739 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-steelseries.ko 2023-12-07T09:51:39.739 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-sunplus.ko 2023-12-07T09:51:39.739 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-thrustmaster.ko 2023-12-07T09:51:39.739 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-tivo.ko 2023-12-07T09:51:39.739 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-tmff.ko 2023-12-07T09:51:39.739 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-topseed.ko 2023-12-07T09:51:39.739 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-twinhan.ko 2023-12-07T09:51:39.739 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-uclogic.ko 2023-12-07T09:51:39.739 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-waltop.ko 2023-12-07T09:51:39.739 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-wiimote.ko 2023-12-07T09:51:39.739 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-xinmo.ko 2023-12-07T09:51:39.739 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-zpff.ko 2023-12-07T09:51:39.739 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/hid-zydacron.ko 2023-12-07T09:51:39.739 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/uhid.ko 2023-12-07T09:51:39.740 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hid/wacom.ko 2023-12-07T09:51:39.740 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hv 2023-12-07T09:51:39.740 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hv/hv_balloon.ko 2023-12-07T09:51:39.740 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hv/hv_utils.ko 2023-12-07T09:51:39.740 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hv/hv_vmbus.ko 2023-12-07T09:51:39.740 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon 2023-12-07T09:51:39.740 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/abituguru.ko 2023-12-07T09:51:39.740 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/abituguru3.ko 2023-12-07T09:51:39.740 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/acpi_power_meter.ko 2023-12-07T09:51:39.740 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/ad7414.ko 2023-12-07T09:51:39.740 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/ad7418.ko 2023-12-07T09:51:39.740 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/adc128d818.ko 2023-12-07T09:51:39.740 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/adm1025.ko 2023-12-07T09:51:39.740 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/adm1026.ko 2023-12-07T09:51:39.740 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/adm1029.ko 2023-12-07T09:51:39.741 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/adm1031.ko 2023-12-07T09:51:39.741 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/adm9240.ko 2023-12-07T09:51:39.741 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/ads7828.ko 2023-12-07T09:51:39.741 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/adt7410.ko 2023-12-07T09:51:39.741 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/adt7411.ko 2023-12-07T09:51:39.741 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/adt7462.ko 2023-12-07T09:51:39.741 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/adt7470.ko 2023-12-07T09:51:39.741 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/adt7475.ko 2023-12-07T09:51:39.741 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/adt7x10.ko 2023-12-07T09:51:39.741 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/amc6821.ko 2023-12-07T09:51:39.741 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/applesmc.ko 2023-12-07T09:51:39.741 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/asb100.ko 2023-12-07T09:51:39.741 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/asc7621.ko 2023-12-07T09:51:39.741 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/asus_atk0110.ko 2023-12-07T09:51:39.741 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/atxp1.ko 2023-12-07T09:51:39.742 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/coretemp.ko 2023-12-07T09:51:39.742 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-12-07T09:51:39.742 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/dme1737.ko 2023-12-07T09:51:39.742 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/ds1621.ko 2023-12-07T09:51:39.742 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/ds620.ko 2023-12-07T09:51:39.742 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/emc1403.ko 2023-12-07T09:51:39.742 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/emc6w201.ko 2023-12-07T09:51:39.742 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/f71805f.ko 2023-12-07T09:51:39.742 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/f71882fg.ko 2023-12-07T09:51:39.742 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/f75375s.ko 2023-12-07T09:51:39.742 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/fam15h_power.ko 2023-12-07T09:51:39.742 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/fschmd.ko 2023-12-07T09:51:39.742 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/g760a.ko 2023-12-07T09:51:39.742 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/g762.ko 2023-12-07T09:51:39.742 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/gl518sm.ko 2023-12-07T09:51:39.743 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/gl520sm.ko 2023-12-07T09:51:39.743 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/hwmon-vid.ko 2023-12-07T09:51:39.743 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/i5500_temp.ko 2023-12-07T09:51:39.743 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/i5k_amb.ko 2023-12-07T09:51:39.743 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/ibmaem.ko 2023-12-07T09:51:39.743 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/ibmpex.ko 2023-12-07T09:51:39.743 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/ina209.ko 2023-12-07T09:51:39.743 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/ina2xx.ko 2023-12-07T09:51:39.743 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/it87.ko 2023-12-07T09:51:39.743 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/k10temp.ko 2023-12-07T09:51:39.743 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/k8temp.ko 2023-12-07T09:51:39.743 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/lineage-pem.ko 2023-12-07T09:51:39.743 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/lm63.ko 2023-12-07T09:51:39.744 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/lm73.ko 2023-12-07T09:51:39.744 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/lm75.ko 2023-12-07T09:51:39.744 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/lm77.ko 2023-12-07T09:51:39.744 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/lm78.ko 2023-12-07T09:51:39.744 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/lm80.ko 2023-12-07T09:51:39.744 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/lm83.ko 2023-12-07T09:51:39.744 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/lm85.ko 2023-12-07T09:51:39.744 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/lm87.ko 2023-12-07T09:51:39.744 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/lm90.ko 2023-12-07T09:51:39.744 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/lm92.ko 2023-12-07T09:51:39.744 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/lm93.ko 2023-12-07T09:51:39.744 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/lm95234.ko 2023-12-07T09:51:39.744 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/lm95241.ko 2023-12-07T09:51:39.744 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/lm95245.ko 2023-12-07T09:51:39.744 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/ltc2945.ko 2023-12-07T09:51:39.745 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/ltc4151.ko 2023-12-07T09:51:39.745 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/ltc4215.ko 2023-12-07T09:51:39.745 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/ltc4222.ko 2023-12-07T09:51:39.745 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/ltc4245.ko 2023-12-07T09:51:39.745 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/ltc4260.ko 2023-12-07T09:51:39.745 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/ltc4261.ko 2023-12-07T09:51:39.745 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/max16065.ko 2023-12-07T09:51:39.745 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/max1619.ko 2023-12-07T09:51:39.745 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/max1668.ko 2023-12-07T09:51:39.745 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/max197.ko 2023-12-07T09:51:39.745 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/max6639.ko 2023-12-07T09:51:39.745 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/max6650.ko 2023-12-07T09:51:39.745 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/max6697.ko 2023-12-07T09:51:39.745 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/mcp3021.ko 2023-12-07T09:51:39.745 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/nct6683.ko 2023-12-07T09:51:39.746 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/nct6775-core.ko 2023-12-07T09:51:39.746 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/nct6775.ko 2023-12-07T09:51:39.746 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/nct7802.ko 2023-12-07T09:51:39.746 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/pc87360.ko 2023-12-07T09:51:39.746 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/pc87427.ko 2023-12-07T09:51:39.746 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/pcf8591.ko 2023-12-07T09:51:39.746 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/pmbus 2023-12-07T09:51:39.746 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-12-07T09:51:39.746 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-12-07T09:51:39.746 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-12-07T09:51:39.746 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/pmbus/max16064.ko 2023-12-07T09:51:39.746 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/pmbus/max34440.ko 2023-12-07T09:51:39.746 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/pmbus/max8688.ko 2023-12-07T09:51:39.746 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-12-07T09:51:39.747 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-12-07T09:51:39.747 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-12-07T09:51:39.747 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-12-07T09:51:39.747 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-12-07T09:51:39.747 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-12-07T09:51:39.747 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/powr1220.ko 2023-12-07T09:51:39.747 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/sch5627.ko 2023-12-07T09:51:39.747 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/sch5636.ko 2023-12-07T09:51:39.747 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/sch56xx-common.ko 2023-12-07T09:51:39.747 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/sht15.ko 2023-12-07T09:51:39.747 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/sht21.ko 2023-12-07T09:51:39.747 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/shtc1.ko 2023-12-07T09:51:39.747 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/sis5595.ko 2023-12-07T09:51:39.747 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/smsc47b397.ko 2023-12-07T09:51:39.747 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/smsc47m1.ko 2023-12-07T09:51:39.748 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/smsc47m192.ko 2023-12-07T09:51:39.748 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/thmc50.ko 2023-12-07T09:51:39.748 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/tmp102.ko 2023-12-07T09:51:39.748 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/tmp103.ko 2023-12-07T09:51:39.748 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/tmp401.ko 2023-12-07T09:51:39.748 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/tmp421.ko 2023-12-07T09:51:39.748 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/via-cputemp.ko 2023-12-07T09:51:39.748 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/via686a.ko 2023-12-07T09:51:39.748 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/vt1211.ko 2023-12-07T09:51:39.748 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/vt8231.ko 2023-12-07T09:51:39.748 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/w83627ehf.ko 2023-12-07T09:51:39.748 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/w83627hf.ko 2023-12-07T09:51:39.748 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/w83781d.ko 2023-12-07T09:51:39.748 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/w83791d.ko 2023-12-07T09:51:39.748 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/w83792d.ko 2023-12-07T09:51:39.749 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/w83793.ko 2023-12-07T09:51:39.749 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/w83795.ko 2023-12-07T09:51:39.749 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/w83l785ts.ko 2023-12-07T09:51:39.749 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/hwmon/w83l786ng.ko 2023-12-07T09:51:39.749 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c 2023-12-07T09:51:39.749 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/algos 2023-12-07T09:51:39.749 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-12-07T09:51:39.749 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-12-07T09:51:39.749 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/busses 2023-12-07T09:51:39.749 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-12-07T09:51:39.749 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-12-07T09:51:39.749 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-12-07T09:51:39.749 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-12-07T09:51:39.750 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-12-07T09:51:39.750 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-12-07T09:51:39.750 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-12-07T09:51:39.750 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-12-07T09:51:39.750 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/busses/i2c-i801.ko 2023-12-07T09:51:39.750 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/busses/i2c-isch.ko 2023-12-07T09:51:39.750 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-12-07T09:51:39.750 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-12-07T09:51:39.750 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-12-07T09:51:39.750 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/busses/i2c-parport.ko 2023-12-07T09:51:39.750 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-12-07T09:51:39.750 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-12-07T09:51:39.750 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-12-07T09:51:39.750 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-12-07T09:51:39.751 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-12-07T09:51:39.751 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-12-07T09:51:39.751 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/busses/i2c-via.ko 2023-12-07T09:51:39.751 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-12-07T09:51:39.751 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-12-07T09:51:39.751 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/i2c-dev.ko 2023-12-07T09:51:39.751 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/i2c-mux.ko 2023-12-07T09:51:39.751 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-12-07T09:51:39.751 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/i2c-smbus.ko 2023-12-07T09:51:39.751 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/i2c/i2c-stub.ko 2023-12-07T09:51:39.751 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband 2023-12-07T09:51:39.751 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/core 2023-12-07T09:51:39.751 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/core/ib_cm.ko 2023-12-07T09:51:39.751 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/core/ib_core.ko 2023-12-07T09:51:39.752 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/core/ib_umad.ko 2023-12-07T09:51:39.752 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-12-07T09:51:39.752 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/core/iw_cm.ko 2023-12-07T09:51:39.752 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/core/rdma_cm.ko 2023-12-07T09:51:39.752 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-12-07T09:51:39.752 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/hw 2023-12-07T09:51:39.752 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/hw/cxgb4 2023-12-07T09:51:39.752 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-12-07T09:51:39.752 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/hw/mlx4 2023-12-07T09:51:39.752 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-12-07T09:51:39.752 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/hw/mlx5 2023-12-07T09:51:39.752 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-12-07T09:51:39.752 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/hw/mthca 2023-12-07T09:51:39.752 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-12-07T09:51:39.752 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/ulp 2023-12-07T09:51:39.753 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/ulp/ipoib 2023-12-07T09:51:39.753 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-12-07T09:51:39.753 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/ulp/iser 2023-12-07T09:51:39.753 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-12-07T09:51:39.753 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/ulp/isert 2023-12-07T09:51:39.753 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-12-07T09:51:39.753 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/ulp/srp 2023-12-07T09:51:39.753 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-12-07T09:51:39.753 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/ulp/srpt 2023-12-07T09:51:39.753 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-12-07T09:51:39.753 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/input 2023-12-07T09:51:39.753 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/input/gameport 2023-12-07T09:51:39.753 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-12-07T09:51:39.753 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/input/gameport/fm801-gp.ko 2023-12-07T09:51:39.753 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/input/gameport/gameport.ko 2023-12-07T09:51:39.754 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/input/joydev.ko 2023-12-07T09:51:39.754 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/input/keyboard 2023-12-07T09:51:39.754 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/input/keyboard/gpio_keys.ko 2023-12-07T09:51:39.754 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/input/mouse 2023-12-07T09:51:39.754 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/input/mouse/appletouch.ko 2023-12-07T09:51:39.754 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/input/mouse/bcm5974.ko 2023-12-07T09:51:39.754 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/input/mouse/cyapatp.ko 2023-12-07T09:51:39.754 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/input/mouse/elan_i2c.ko 2023-12-07T09:51:39.754 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/input/mouse/sermouse.ko 2023-12-07T09:51:39.754 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-12-07T09:51:39.754 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/input/mouse/synaptics_usb.ko 2023-12-07T09:51:39.754 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/input/mouse/vsxxxaa.ko 2023-12-07T09:51:39.754 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/input/rmi4 2023-12-07T09:51:39.754 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/input/rmi4/rmi_core.ko 2023-12-07T09:51:39.755 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/input/serio 2023-12-07T09:51:39.755 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/input/serio/altera_ps2.ko 2023-12-07T09:51:39.755 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/input/serio/arc_ps2.ko 2023-12-07T09:51:39.755 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-12-07T09:51:39.755 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/input/serio/serio_raw.ko 2023-12-07T09:51:39.755 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/input/sparse-keymap.ko 2023-12-07T09:51:39.755 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/iommu 2023-12-07T09:51:39.755 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/iommu/amd 2023-12-07T09:51:39.755 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/iommu/amd/iommu_v2.ko 2023-12-07T09:51:39.755 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/leds 2023-12-07T09:51:39.755 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/leds/leds-blinkm.ko 2023-12-07T09:51:39.755 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/leds/leds-lm3530.ko 2023-12-07T09:51:39.755 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/leds/leds-lp3944.ko 2023-12-07T09:51:39.755 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/leds/leds-ss4200.ko 2023-12-07T09:51:39.756 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/leds/trigger 2023-12-07T09:51:39.756 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-12-07T09:51:39.756 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-12-07T09:51:39.756 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-12-07T09:51:39.756 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-12-07T09:51:39.756 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-12-07T09:51:39.756 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-12-07T09:51:39.756 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-12-07T09:51:39.756 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-12-07T09:51:39.756 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md 2023-12-07T09:51:39.756 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md/bcache 2023-12-07T09:51:39.756 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md/bcache/bcache.ko 2023-12-07T09:51:39.756 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md/dm-bio-prison.ko 2023-12-07T09:51:39.756 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md/dm-cache-smq.ko 2023-12-07T09:51:39.756 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md/dm-cache.ko 2023-12-07T09:51:39.757 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md/dm-crypt.ko 2023-12-07T09:51:39.757 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md/dm-delay.ko 2023-12-07T09:51:39.757 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md/dm-flakey.ko 2023-12-07T09:51:39.757 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md/dm-log-userspace.ko 2023-12-07T09:51:39.757 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md/dm-multipath.ko 2023-12-07T09:51:39.757 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md/dm-queue-length.ko 2023-12-07T09:51:39.757 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md/dm-raid.ko 2023-12-07T09:51:39.757 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md/dm-round-robin.ko 2023-12-07T09:51:39.757 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md/dm-service-time.ko 2023-12-07T09:51:39.757 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md/dm-switch.ko 2023-12-07T09:51:39.757 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md/dm-thin-pool.ko 2023-12-07T09:51:39.757 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md/dm-verity.ko 2023-12-07T09:51:39.757 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md/faulty.ko 2023-12-07T09:51:39.757 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md/linear.ko 2023-12-07T09:51:39.757 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md/multipath.ko 2023-12-07T09:51:39.758 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md/persistent-data 2023-12-07T09:51:39.758 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-12-07T09:51:39.758 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md/raid0.ko 2023-12-07T09:51:39.758 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md/raid1.ko 2023-12-07T09:51:39.758 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md/raid10.ko 2023-12-07T09:51:39.758 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/md/raid456.ko 2023-12-07T09:51:39.758 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/message 2023-12-07T09:51:39.758 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/message/fusion 2023-12-07T09:51:39.758 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/message/fusion/mptbase.ko 2023-12-07T09:51:39.758 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/message/fusion/mptctl.ko 2023-12-07T09:51:39.758 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/message/fusion/mptfc.ko 2023-12-07T09:51:39.758 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/message/fusion/mptlan.ko 2023-12-07T09:51:39.758 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/message/fusion/mptsas.ko 2023-12-07T09:51:39.758 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/message/fusion/mptscsih.ko 2023-12-07T09:51:39.759 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/message/fusion/mptspi.ko 2023-12-07T09:51:39.759 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/mfd 2023-12-07T09:51:39.759 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/mfd/lpc_ich.ko 2023-12-07T09:51:39.759 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/mfd/lpc_sch.ko 2023-12-07T09:51:39.759 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/mfd/mfd-core.ko 2023-12-07T09:51:39.759 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/mfd/sm501.ko 2023-12-07T09:51:39.759 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/mfd/viperboard.ko 2023-12-07T09:51:39.759 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/mfd/vx855.ko 2023-12-07T09:51:39.759 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/mfd/wl1273-core.ko 2023-12-07T09:51:39.759 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc 2023-12-07T09:51:39.759 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/altera-stapl 2023-12-07T09:51:39.759 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-12-07T09:51:39.759 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/apds9802als.ko 2023-12-07T09:51:39.759 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/apds990x.ko 2023-12-07T09:51:39.759 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/bh1770glc.ko 2023-12-07T09:51:39.760 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/cb710 2023-12-07T09:51:39.760 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/cb710/cb710.ko 2023-12-07T09:51:39.760 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/eeprom 2023-12-07T09:51:39.760 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/eeprom/at24.ko 2023-12-07T09:51:39.760 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/eeprom/eeprom.ko 2023-12-07T09:51:39.760 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-12-07T09:51:39.760 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/eeprom/max6875.ko 2023-12-07T09:51:39.760 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/enclosure.ko 2023-12-07T09:51:39.760 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/hpilo.ko 2023-12-07T09:51:39.760 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/isl29003.ko 2023-12-07T09:51:39.760 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/isl29020.ko 2023-12-07T09:51:39.760 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/lis3lv02d 2023-12-07T09:51:39.760 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-12-07T09:51:39.760 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-12-07T09:51:39.760 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/mei 2023-12-07T09:51:39.761 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/mei/mei-me.ko 2023-12-07T09:51:39.761 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/mei/mei-txe.ko 2023-12-07T09:51:39.761 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/mei/mei.ko 2023-12-07T09:51:39.761 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/sgi-gru 2023-12-07T09:51:39.761 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/sgi-gru/gru.ko 2023-12-07T09:51:39.761 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/sgi-xp 2023-12-07T09:51:39.761 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/sgi-xp/xp.ko 2023-12-07T09:51:39.761 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/sgi-xp/xpc.ko 2023-12-07T09:51:39.761 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-12-07T09:51:39.761 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/tifm_7xx1.ko 2023-12-07T09:51:39.761 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/tifm_core.ko 2023-12-07T09:51:39.761 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/tsl2550.ko 2023-12-07T09:51:39.761 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/vmw_balloon.ko 2023-12-07T09:51:39.761 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/vmw_vmci 2023-12-07T09:51:39.762 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-12-07T09:51:39.762 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net 2023-12-07T09:51:39.762 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/bonding 2023-12-07T09:51:39.762 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/bonding/bonding.ko 2023-12-07T09:51:39.762 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/dsa 2023-12-07T09:51:39.762 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/dsa/b53 2023-12-07T09:51:39.762 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/dsa/b53/b53_common.ko 2023-12-07T09:51:39.762 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/dsa/bcm-sf2.ko 2023-12-07T09:51:39.762 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/dsa/mv88e6060.ko 2023-12-07T09:51:39.762 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/dsa/mv88e6xxx 2023-12-07T09:51:39.762 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-12-07T09:51:39.762 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/dummy.ko 2023-12-07T09:51:39.762 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/eql.ko 2023-12-07T09:51:39.762 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet 2023-12-07T09:51:39.762 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/3com 2023-12-07T09:51:39.763 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-12-07T09:51:39.763 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-12-07T09:51:39.763 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-12-07T09:51:39.763 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-12-07T09:51:39.763 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/8390 2023-12-07T09:51:39.763 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/8390/8390.ko 2023-12-07T09:51:39.763 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-12-07T09:51:39.763 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-12-07T09:51:39.763 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-12-07T09:51:39.763 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/adaptec 2023-12-07T09:51:39.763 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-12-07T09:51:39.763 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/agere 2023-12-07T09:51:39.763 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/agere/et131x.ko 2023-12-07T09:51:39.763 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/alteon 2023-12-07T09:51:39.763 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-12-07T09:51:39.764 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/altera 2023-12-07T09:51:39.764 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-12-07T09:51:39.764 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/amd 2023-12-07T09:51:39.764 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-12-07T09:51:39.764 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-12-07T09:51:39.764 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-12-07T09:51:39.764 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/atheros 2023-12-07T09:51:39.764 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/atheros/alx 2023-12-07T09:51:39.764 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-12-07T09:51:39.764 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/atheros/atl1c 2023-12-07T09:51:39.764 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-12-07T09:51:39.764 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/atheros/atl1e 2023-12-07T09:51:39.764 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-12-07T09:51:39.764 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/atheros/atlx 2023-12-07T09:51:39.764 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-12-07T09:51:39.765 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-12-07T09:51:39.765 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/broadcom 2023-12-07T09:51:39.765 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-12-07T09:51:39.765 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-12-07T09:51:39.765 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-12-07T09:51:39.765 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-12-07T09:51:39.765 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-12-07T09:51:39.765 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/broadcom/genet 2023-12-07T09:51:39.765 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-12-07T09:51:39.765 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-12-07T09:51:39.765 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/brocade 2023-12-07T09:51:39.765 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/brocade/bna 2023-12-07T09:51:39.765 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-12-07T09:51:39.765 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/chelsio 2023-12-07T09:51:39.765 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/chelsio/cxgb 2023-12-07T09:51:39.766 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-12-07T09:51:39.766 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-12-07T09:51:39.766 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-12-07T09:51:39.766 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-12-07T09:51:39.766 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-12-07T09:51:39.766 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-12-07T09:51:39.766 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-12-07T09:51:39.766 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-12-07T09:51:39.766 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-12-07T09:51:39.766 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/cisco 2023-12-07T09:51:39.766 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/cisco/enic 2023-12-07T09:51:39.766 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-12-07T09:51:39.766 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/dec 2023-12-07T09:51:39.766 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/dec/tulip 2023-12-07T09:51:39.767 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-12-07T09:51:39.767 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-12-07T09:51:39.767 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-12-07T09:51:39.767 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-12-07T09:51:39.767 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-12-07T09:51:39.767 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-12-07T09:51:39.767 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/dlink 2023-12-07T09:51:39.767 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-12-07T09:51:39.767 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-12-07T09:51:39.767 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/dnet.ko 2023-12-07T09:51:39.767 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/emulex 2023-12-07T09:51:39.767 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/emulex/benet 2023-12-07T09:51:39.767 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-12-07T09:51:39.767 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/ethoc.ko 2023-12-07T09:51:39.767 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/fealnx.ko 2023-12-07T09:51:39.768 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/intel 2023-12-07T09:51:39.768 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/intel/e100.ko 2023-12-07T09:51:39.768 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/intel/e1000 2023-12-07T09:51:39.768 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-12-07T09:51:39.768 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/intel/e1000e 2023-12-07T09:51:39.768 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-12-07T09:51:39.768 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/intel/fm10k 2023-12-07T09:51:39.768 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-12-07T09:51:39.768 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/intel/i40e 2023-12-07T09:51:39.768 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-12-07T09:51:39.768 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/intel/iavf 2023-12-07T09:51:39.768 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-12-07T09:51:39.768 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/intel/igb 2023-12-07T09:51:39.768 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-12-07T09:51:39.768 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/intel/igbvf 2023-12-07T09:51:39.769 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-12-07T09:51:39.769 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/intel/ixgbe 2023-12-07T09:51:39.769 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-12-07T09:51:39.769 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/intel/ixgbevf 2023-12-07T09:51:39.769 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-12-07T09:51:39.769 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/jme.ko 2023-12-07T09:51:39.769 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/marvell 2023-12-07T09:51:39.769 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-12-07T09:51:39.769 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/marvell/skge.ko 2023-12-07T09:51:39.769 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-12-07T09:51:39.769 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/mellanox 2023-12-07T09:51:39.769 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/mellanox/mlx4 2023-12-07T09:51:39.769 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-12-07T09:51:39.769 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-12-07T09:51:39.769 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/mellanox/mlx5 2023-12-07T09:51:39.770 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-12-07T09:51:39.770 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-12-07T09:51:39.770 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/micrel 2023-12-07T09:51:39.770 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-12-07T09:51:39.770 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/myricom 2023-12-07T09:51:39.770 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/myricom/myri10ge 2023-12-07T09:51:39.770 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-12-07T09:51:39.770 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/natsemi 2023-12-07T09:51:39.770 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-12-07T09:51:39.770 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-12-07T09:51:39.770 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/neterion 2023-12-07T09:51:39.770 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-12-07T09:51:39.770 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/nvidia 2023-12-07T09:51:39.770 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-12-07T09:51:39.771 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/packetengines 2023-12-07T09:51:39.771 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-12-07T09:51:39.771 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-12-07T09:51:39.771 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/qlogic 2023-12-07T09:51:39.771 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/qlogic/netxen 2023-12-07T09:51:39.771 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-12-07T09:51:39.771 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-12-07T09:51:39.771 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-12-07T09:51:39.771 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-12-07T09:51:39.771 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/rdc 2023-12-07T09:51:39.771 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-12-07T09:51:39.771 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/realtek 2023-12-07T09:51:39.771 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-12-07T09:51:39.771 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-12-07T09:51:39.771 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/realtek/atp.ko 2023-12-07T09:51:39.772 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-12-07T09:51:39.772 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/rocker 2023-12-07T09:51:39.772 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-12-07T09:51:39.772 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/sfc 2023-12-07T09:51:39.772 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-12-07T09:51:39.772 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/silan 2023-12-07T09:51:39.772 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-12-07T09:51:39.772 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/sis 2023-12-07T09:51:39.772 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/sis/sis190.ko 2023-12-07T09:51:39.772 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/sis/sis900.ko 2023-12-07T09:51:39.772 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/smsc 2023-12-07T09:51:39.772 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-12-07T09:51:39.772 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-12-07T09:51:39.772 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-12-07T09:51:39.772 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-12-07T09:51:39.773 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/stmicro 2023-12-07T09:51:39.773 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/stmicro/stmmac 2023-12-07T09:51:39.773 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-12-07T09:51:39.773 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-12-07T09:51:39.773 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/sun 2023-12-07T09:51:39.773 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/sun/cassini.ko 2023-12-07T09:51:39.773 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/sun/niu.ko 2023-12-07T09:51:39.773 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/sun/sungem.ko 2023-12-07T09:51:39.773 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-12-07T09:51:39.773 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/tehuti 2023-12-07T09:51:39.773 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-12-07T09:51:39.773 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/ti 2023-12-07T09:51:39.773 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/ti/tlan.ko 2023-12-07T09:51:39.773 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/via 2023-12-07T09:51:39.774 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-12-07T09:51:39.774 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-12-07T09:51:39.774 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/wiznet 2023-12-07T09:51:39.774 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-12-07T09:51:39.774 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-12-07T09:51:39.774 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/xircom 2023-12-07T09:51:39.774 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-12-07T09:51:39.774 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/geneve.ko 2023-12-07T09:51:39.774 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/hyperv 2023-12-07T09:51:39.774 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-12-07T09:51:39.774 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ieee802154 2023-12-07T09:51:39.774 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ieee802154/fakelb.ko 2023-12-07T09:51:39.774 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ifb.ko 2023-12-07T09:51:39.774 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ipvlan 2023-12-07T09:51:39.774 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ipvlan/ipvlan.ko 2023-12-07T09:51:39.775 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/macvlan.ko 2023-12-07T09:51:39.775 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/macvtap.ko 2023-12-07T09:51:39.775 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/mdio 2023-12-07T09:51:39.775 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/mdio.ko 2023-12-07T09:51:39.775 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-12-07T09:51:39.775 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-12-07T09:51:39.775 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/mdio/mdio-regmap.ko 2023-12-07T09:51:39.775 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/mii.ko 2023-12-07T09:51:39.775 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/net_failover.ko 2023-12-07T09:51:39.775 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/netconsole.ko 2023-12-07T09:51:39.775 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/nlmon.ko 2023-12-07T09:51:39.775 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/pcs 2023-12-07T09:51:39.775 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/pcs/pcs-lynx.ko 2023-12-07T09:51:39.775 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-12-07T09:51:39.776 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/phy 2023-12-07T09:51:39.776 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/phy/amd.ko 2023-12-07T09:51:39.776 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-12-07T09:51:39.776 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-12-07T09:51:39.776 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/phy/bcm7xxx.ko 2023-12-07T09:51:39.776 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/phy/bcm87xx.ko 2023-12-07T09:51:39.776 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/phy/broadcom.ko 2023-12-07T09:51:39.776 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/phy/cicada.ko 2023-12-07T09:51:39.776 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/phy/davicom.ko 2023-12-07T09:51:39.776 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/phy/dp83640.ko 2023-12-07T09:51:39.776 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/phy/et1011c.ko 2023-12-07T09:51:39.776 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/phy/icplus.ko 2023-12-07T09:51:39.776 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/phy/lxt.ko 2023-12-07T09:51:39.776 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/phy/marvell.ko 2023-12-07T09:51:39.776 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/phy/micrel.ko 2023-12-07T09:51:39.776 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/phy/national.ko 2023-12-07T09:51:39.777 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/phy/phylink.ko 2023-12-07T09:51:39.777 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/phy/qsemi.ko 2023-12-07T09:51:39.777 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/phy/realtek.ko 2023-12-07T09:51:39.777 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/phy/smsc.ko 2023-12-07T09:51:39.777 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/phy/ste10Xp.ko 2023-12-07T09:51:39.777 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/phy/vitesse.ko 2023-12-07T09:51:39.777 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ppp 2023-12-07T09:51:39.777 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ppp/bsd_comp.ko 2023-12-07T09:51:39.777 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ppp/ppp_async.ko 2023-12-07T09:51:39.777 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ppp/ppp_deflate.ko 2023-12-07T09:51:39.777 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ppp/ppp_generic.ko 2023-12-07T09:51:39.777 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ppp/ppp_mppe.ko 2023-12-07T09:51:39.777 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ppp/ppp_synctty.ko 2023-12-07T09:51:39.777 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ppp/pppoe.ko 2023-12-07T09:51:39.778 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ppp/pppox.ko 2023-12-07T09:51:39.778 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/ppp/pptp.ko 2023-12-07T09:51:39.778 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/slip 2023-12-07T09:51:39.778 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/slip/slhc.ko 2023-12-07T09:51:39.778 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/slip/slip.ko 2023-12-07T09:51:39.778 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/sungem_phy.ko 2023-12-07T09:51:39.778 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/tap.ko 2023-12-07T09:51:39.778 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/team 2023-12-07T09:51:39.778 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/team/team.ko 2023-12-07T09:51:39.778 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/team/team_mode_activebackup.ko 2023-12-07T09:51:39.778 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/team/team_mode_broadcast.ko 2023-12-07T09:51:39.778 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-12-07T09:51:39.778 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/team/team_mode_random.ko 2023-12-07T09:51:39.778 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-12-07T09:51:39.778 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/tun.ko 2023-12-07T09:51:39.779 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/veth.ko 2023-12-07T09:51:39.779 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/virtio_net.ko 2023-12-07T09:51:39.779 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/vmxnet3 2023-12-07T09:51:39.779 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-12-07T09:51:39.779 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/vxlan 2023-12-07T09:51:39.779 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/vxlan/vxlan.ko 2023-12-07T09:51:39.779 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/xen-netback 2023-12-07T09:51:39.779 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/xen-netback/xen-netback.ko 2023-12-07T09:51:39.779 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/net/xen-netfront.ko 2023-12-07T09:51:39.779 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ntb 2023-12-07T09:51:39.779 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ntb/ntb.ko 2023-12-07T09:51:39.779 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/nvme 2023-12-07T09:51:39.779 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/nvme/host 2023-12-07T09:51:39.779 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/nvme/host/nvme-core.ko 2023-12-07T09:51:39.780 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/nvme/host/nvme.ko 2023-12-07T09:51:39.780 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/parport 2023-12-07T09:51:39.780 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/parport/parport.ko 2023-12-07T09:51:39.780 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/parport/parport_cs.ko 2023-12-07T09:51:39.780 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/parport/parport_pc.ko 2023-12-07T09:51:39.780 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/parport/parport_serial.ko 2023-12-07T09:51:39.780 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/pci 2023-12-07T09:51:39.780 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/pci/hotplug 2023-12-07T09:51:39.780 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-12-07T09:51:39.780 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/pci/pcie 2023-12-07T09:51:39.780 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/pci/pcie/aer_inject.ko 2023-12-07T09:51:39.780 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/pci/xen-pcifront.ko 2023-12-07T09:51:39.780 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/pcmcia 2023-12-07T09:51:39.780 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/pcmcia/i82092.ko 2023-12-07T09:51:39.780 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/pcmcia/pd6729.ko 2023-12-07T09:51:39.781 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/pcmcia/yenta_socket.ko 2023-12-07T09:51:39.781 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/pinctrl 2023-12-07T09:51:39.781 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/pinctrl/intel 2023-12-07T09:51:39.781 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-12-07T09:51:39.781 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform 2023-12-07T09:51:39.781 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform/x86 2023-12-07T09:51:39.781 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform/x86/acer-wmi.ko 2023-12-07T09:51:39.781 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform/x86/acerhdf.ko 2023-12-07T09:51:39.781 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform/x86/apple-gmux.ko 2023-12-07T09:51:39.781 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform/x86/asus-laptop.ko 2023-12-07T09:51:39.781 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-12-07T09:51:39.781 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform/x86/asus-wmi.ko 2023-12-07T09:51:39.781 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform/x86/classmate-laptop.ko 2023-12-07T09:51:39.781 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-12-07T09:51:39.781 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-12-07T09:51:39.782 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-12-07T09:51:39.782 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-12-07T09:51:39.782 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform/x86/hdaps.ko 2023-12-07T09:51:39.782 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform/x86/intel 2023-12-07T09:51:39.782 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-12-07T09:51:39.782 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform/x86/intel_ips.ko 2023-12-07T09:51:39.782 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform/x86/msi-wmi.ko 2023-12-07T09:51:39.782 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform/x86/mxm-wmi.ko 2023-12-07T09:51:39.782 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-12-07T09:51:39.782 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform/x86/samsung-laptop.ko 2023-12-07T09:51:39.782 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform/x86/samsung-q10.ko 2023-12-07T09:51:39.782 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform/x86/topstar-laptop.ko 2023-12-07T09:51:39.782 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-12-07T09:51:39.782 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform/x86/toshiba_haps.ko 2023-12-07T09:51:39.782 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform/x86/wmi-bmof.ko 2023-12-07T09:51:39.783 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/platform/x86/wmi.ko 2023-12-07T09:51:39.783 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/powercap 2023-12-07T09:51:39.783 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/powercap/intel_rapl_common.ko 2023-12-07T09:51:39.783 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/powercap/intel_rapl_msr.ko 2023-12-07T09:51:39.783 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/pps 2023-12-07T09:51:39.783 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/pps/clients 2023-12-07T09:51:39.783 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/pps/clients/pps-gpio.ko 2023-12-07T09:51:39.783 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/pps/clients/pps-ldisc.ko 2023-12-07T09:51:39.783 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/pps/clients/pps_parport.ko 2023-12-07T09:51:39.783 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/pps/pps_core.ko 2023-12-07T09:51:39.783 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ptp 2023-12-07T09:51:39.783 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ptp/ptp.ko 2023-12-07T09:51:39.783 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ptp/ptp_kvm.ko 2023-12-07T09:51:39.783 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc 2023-12-07T09:51:39.784 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-bq32k.ko 2023-12-07T09:51:39.784 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-ds1286.ko 2023-12-07T09:51:39.784 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-ds1307.ko 2023-12-07T09:51:39.784 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-ds1374.ko 2023-12-07T09:51:39.784 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-ds1511.ko 2023-12-07T09:51:39.784 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-ds1553.ko 2023-12-07T09:51:39.784 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-ds1672.ko 2023-12-07T09:51:39.784 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-ds1742.ko 2023-12-07T09:51:39.784 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-ds2404.ko 2023-12-07T09:51:39.784 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-ds3232.ko 2023-12-07T09:51:39.784 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-em3027.ko 2023-12-07T09:51:39.784 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-fm3130.ko 2023-12-07T09:51:39.784 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-isl12022.ko 2023-12-07T09:51:39.784 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-isl1208.ko 2023-12-07T09:51:39.784 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-m41t80.ko 2023-12-07T09:51:39.785 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-m48t35.ko 2023-12-07T09:51:39.785 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-m48t59.ko 2023-12-07T09:51:39.785 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-max6900.ko 2023-12-07T09:51:39.785 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-msm6242.ko 2023-12-07T09:51:39.785 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-pcf2127.ko 2023-12-07T09:51:39.785 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-pcf85063.ko 2023-12-07T09:51:39.785 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-pcf8523.ko 2023-12-07T09:51:39.785 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-pcf8563.ko 2023-12-07T09:51:39.785 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-pcf8583.ko 2023-12-07T09:51:39.785 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-rp5c01.ko 2023-12-07T09:51:39.785 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-rs5c372.ko 2023-12-07T09:51:39.785 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-12-07T09:51:39.785 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-rx8025.ko 2023-12-07T09:51:39.785 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-rx8581.ko 2023-12-07T09:51:39.786 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-12-07T09:51:39.786 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/rtc/rtc-x1205.ko 2023-12-07T09:51:39.786 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi 2023-12-07T09:51:39.786 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/3w-9xxx.ko 2023-12-07T09:51:39.786 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/3w-sas.ko 2023-12-07T09:51:39.786 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/3w-xxxx.ko 2023-12-07T09:51:39.786 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/BusLogic.ko 2023-12-07T09:51:39.786 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/a100u2w.ko 2023-12-07T09:51:39.786 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/aacraid 2023-12-07T09:51:39.786 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/aacraid/aacraid.ko 2023-12-07T09:51:39.786 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/advansys.ko 2023-12-07T09:51:39.786 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/aic7xxx 2023-12-07T09:51:39.786 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-12-07T09:51:39.786 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-12-07T09:51:39.787 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/am53c974.ko 2023-12-07T09:51:39.787 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/arcmsr 2023-12-07T09:51:39.787 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-12-07T09:51:39.787 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/atp870u.ko 2023-12-07T09:51:39.787 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/be2iscsi 2023-12-07T09:51:39.787 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-12-07T09:51:39.787 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/bfa 2023-12-07T09:51:39.787 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/bfa/bfa.ko 2023-12-07T09:51:39.787 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/bnx2fc 2023-12-07T09:51:39.787 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-12-07T09:51:39.787 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/bnx2i 2023-12-07T09:51:39.787 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-12-07T09:51:39.787 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/ch.ko 2023-12-07T09:51:39.788 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/csiostor 2023-12-07T09:51:39.788 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/csiostor/csiostor.ko 2023-12-07T09:51:39.788 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/cxgbi 2023-12-07T09:51:39.788 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/cxgbi/cxgb3i 2023-12-07T09:51:39.788 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-12-07T09:51:39.788 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/cxgbi/cxgb4i 2023-12-07T09:51:39.788 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-12-07T09:51:39.788 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-12-07T09:51:39.788 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/dc395x.ko 2023-12-07T09:51:39.788 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/device_handler 2023-12-07T09:51:39.788 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-12-07T09:51:39.788 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-12-07T09:51:39.788 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-12-07T09:51:39.788 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-12-07T09:51:39.788 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/dmx3191d.ko 2023-12-07T09:51:39.788 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/esas2r 2023-12-07T09:51:39.789 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/esas2r/esas2r.ko 2023-12-07T09:51:39.789 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/esp_scsi.ko 2023-12-07T09:51:39.789 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/fcoe 2023-12-07T09:51:39.789 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/fcoe/fcoe.ko 2023-12-07T09:51:39.789 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-12-07T09:51:39.789 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/fnic 2023-12-07T09:51:39.789 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/fnic/fnic.ko 2023-12-07T09:51:39.789 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/hpsa.ko 2023-12-07T09:51:39.789 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/hptiop.ko 2023-12-07T09:51:39.789 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/hv_storvsc.ko 2023-12-07T09:51:39.789 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/initio.ko 2023-12-07T09:51:39.789 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/ipr.ko 2023-12-07T09:51:39.789 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/ips.ko 2023-12-07T09:51:39.789 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/isci 2023-12-07T09:51:39.789 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/isci/isci.ko 2023-12-07T09:51:39.790 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-12-07T09:51:39.790 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/iscsi_tcp.ko 2023-12-07T09:51:39.790 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/libfc 2023-12-07T09:51:39.790 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/libfc/libfc.ko 2023-12-07T09:51:39.790 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/libiscsi.ko 2023-12-07T09:51:39.790 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/libiscsi_tcp.ko 2023-12-07T09:51:39.790 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/libsas 2023-12-07T09:51:39.790 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/libsas/libsas.ko 2023-12-07T09:51:39.790 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/lpfc 2023-12-07T09:51:39.790 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/lpfc/lpfc.ko 2023-12-07T09:51:39.790 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/megaraid 2023-12-07T09:51:39.790 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/megaraid.ko 2023-12-07T09:51:39.790 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-12-07T09:51:39.790 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-12-07T09:51:39.790 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-12-07T09:51:39.791 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/mpt3sas 2023-12-07T09:51:39.791 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-12-07T09:51:39.791 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/mvsas 2023-12-07T09:51:39.791 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/mvsas/mvsas.ko 2023-12-07T09:51:39.791 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/mvumi.ko 2023-12-07T09:51:39.791 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/pm8001 2023-12-07T09:51:39.791 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-12-07T09:51:39.791 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/pmcraid.ko 2023-12-07T09:51:39.791 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/qla1280.ko 2023-12-07T09:51:39.791 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/qla2xxx 2023-12-07T09:51:39.791 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-12-07T09:51:39.791 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-12-07T09:51:39.791 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/qla4xxx 2023-12-07T09:51:39.791 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-12-07T09:51:39.792 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/raid_class.ko 2023-12-07T09:51:39.792 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/scsi_debug.ko 2023-12-07T09:51:39.792 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/scsi_transport_fc.ko 2023-12-07T09:51:39.792 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-12-07T09:51:39.792 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/scsi_transport_sas.ko 2023-12-07T09:51:39.792 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/scsi_transport_spi.ko 2023-12-07T09:51:39.792 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/scsi_transport_srp.ko 2023-12-07T09:51:39.792 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/ses.ko 2023-12-07T09:51:39.792 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/st.ko 2023-12-07T09:51:39.792 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/stex.ko 2023-12-07T09:51:39.792 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/sym53c8xx_2 2023-12-07T09:51:39.792 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-12-07T09:51:39.792 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/virtio_scsi.ko 2023-12-07T09:51:39.792 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/vmw_pvscsi.ko 2023-12-07T09:51:39.792 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/scsi/wd719x.ko 2023-12-07T09:51:39.793 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ssb 2023-12-07T09:51:39.793 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ssb/ssb.ko 2023-12-07T09:51:39.793 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/target 2023-12-07T09:51:39.793 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/target/iscsi 2023-12-07T09:51:39.793 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-12-07T09:51:39.793 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/target/loopback 2023-12-07T09:51:39.793 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/target/loopback/tcm_loop.ko 2023-12-07T09:51:39.793 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/target/target_core_file.ko 2023-12-07T09:51:39.793 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/target/target_core_iblock.ko 2023-12-07T09:51:39.793 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/target/target_core_mod.ko 2023-12-07T09:51:39.793 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/target/target_core_pscsi.ko 2023-12-07T09:51:39.793 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/target/target_core_user.ko 2023-12-07T09:51:39.793 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/target/tcm_fc 2023-12-07T09:51:39.793 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-12-07T09:51:39.793 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/thermal 2023-12-07T09:51:39.793 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/thermal/intel 2023-12-07T09:51:39.794 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/thermal/intel/int340x_thermal 2023-12-07T09:51:39.794 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-12-07T09:51:39.794 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-12-07T09:51:39.794 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-12-07T09:51:39.794 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-12-07T09:51:39.794 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-12-07T09:51:39.794 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-12-07T09:51:39.794 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-12-07T09:51:39.794 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-12-07T09:51:39.794 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-12-07T09:51:39.794 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-12-07T09:51:39.794 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-12-07T09:51:39.794 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-12-07T09:51:39.794 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-12-07T09:51:39.795 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-12-07T09:51:39.795 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-12-07T09:51:39.795 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/tty 2023-12-07T09:51:39.795 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/tty/ipwireless 2023-12-07T09:51:39.795 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-12-07T09:51:39.795 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/tty/n_gsm.ko 2023-12-07T09:51:39.795 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/tty/n_hdlc.ko 2023-12-07T09:51:39.795 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/tty/nozomi.ko 2023-12-07T09:51:39.795 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/tty/serial 2023-12-07T09:51:39.795 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/tty/serial/8250 2023-12-07T09:51:39.795 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-12-07T09:51:39.795 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/tty/serial/arc_uart.ko 2023-12-07T09:51:39.795 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/tty/serial/jsm 2023-12-07T09:51:39.795 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/tty/serial/jsm/jsm.ko 2023-12-07T09:51:39.795 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ufs 2023-12-07T09:51:39.795 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ufs/core 2023-12-07T09:51:39.796 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ufs/core/ufshcd-core.ko 2023-12-07T09:51:39.796 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ufs/host 2023-12-07T09:51:39.796 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-12-07T09:51:39.796 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/uio 2023-12-07T09:51:39.796 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/uio/uio.ko 2023-12-07T09:51:39.796 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/uio/uio_aec.ko 2023-12-07T09:51:39.796 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/uio/uio_cif.ko 2023-12-07T09:51:39.796 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/uio/uio_pci_generic.ko 2023-12-07T09:51:39.796 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/uio/uio_sercos3.ko 2023-12-07T09:51:39.796 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb 2023-12-07T09:51:39.796 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/class 2023-12-07T09:51:39.796 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/class/cdc-acm.ko 2023-12-07T09:51:39.796 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/class/cdc-wdm.ko 2023-12-07T09:51:39.796 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/class/usblp.ko 2023-12-07T09:51:39.796 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/class/usbtmc.ko 2023-12-07T09:51:39.797 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/host 2023-12-07T09:51:39.797 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/host/sl811-hcd.ko 2023-12-07T09:51:39.797 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/image 2023-12-07T09:51:39.797 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/image/mdc800.ko 2023-12-07T09:51:39.797 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/image/microtek.ko 2023-12-07T09:51:39.797 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/misc 2023-12-07T09:51:39.797 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/misc/adutux.ko 2023-12-07T09:51:39.797 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/misc/appledisplay.ko 2023-12-07T09:51:39.797 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/misc/emi26.ko 2023-12-07T09:51:39.797 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/misc/emi62.ko 2023-12-07T09:51:39.797 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/misc/ezusb.ko 2023-12-07T09:51:39.797 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/misc/idmouse.ko 2023-12-07T09:51:39.797 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/misc/iowarrior.ko 2023-12-07T09:51:39.797 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/misc/isight_firmware.ko 2023-12-07T09:51:39.797 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/misc/ldusb.ko 2023-12-07T09:51:39.798 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/misc/legousbtower.ko 2023-12-07T09:51:39.798 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/misc/sisusbvga 2023-12-07T09:51:39.798 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-12-07T09:51:39.798 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/misc/trancevibrator.ko 2023-12-07T09:51:39.798 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/misc/usb3503.ko 2023-12-07T09:51:39.798 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/misc/usblcd.ko 2023-12-07T09:51:39.798 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/misc/usbsevseg.ko 2023-12-07T09:51:39.798 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/misc/uss720.ko 2023-12-07T09:51:39.798 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/misc/yurex.ko 2023-12-07T09:51:39.798 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/phy 2023-12-07T09:51:39.798 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/phy/phy-generic.ko 2023-12-07T09:51:39.798 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial 2023-12-07T09:51:39.798 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/aircable.ko 2023-12-07T09:51:39.798 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/ark3116.ko 2023-12-07T09:51:39.798 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/belkin_sa.ko 2023-12-07T09:51:39.799 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/ch341.ko 2023-12-07T09:51:39.799 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/cp210x.ko 2023-12-07T09:51:39.799 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/cyberjack.ko 2023-12-07T09:51:39.799 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/cypress_m8.ko 2023-12-07T09:51:39.799 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/digi_acceleport.ko 2023-12-07T09:51:39.799 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/empeg.ko 2023-12-07T09:51:39.799 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/ftdi_sio.ko 2023-12-07T09:51:39.799 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/garmin_gps.ko 2023-12-07T09:51:39.799 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/io_edgeport.ko 2023-12-07T09:51:39.799 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/io_ti.ko 2023-12-07T09:51:39.799 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/ipaq.ko 2023-12-07T09:51:39.799 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/ipw.ko 2023-12-07T09:51:39.799 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/ir-usb.ko 2023-12-07T09:51:39.799 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-12-07T09:51:39.799 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/keyspan.ko 2023-12-07T09:51:39.800 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/keyspan_pda.ko 2023-12-07T09:51:39.800 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/kl5kusb105.ko 2023-12-07T09:51:39.800 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/kobil_sct.ko 2023-12-07T09:51:39.800 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/mct_u232.ko 2023-12-07T09:51:39.800 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/mos7720.ko 2023-12-07T09:51:39.800 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/mos7840.ko 2023-12-07T09:51:39.800 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/navman.ko 2023-12-07T09:51:39.800 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/omninet.ko 2023-12-07T09:51:39.800 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/opticon.ko 2023-12-07T09:51:39.800 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/option.ko 2023-12-07T09:51:39.800 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/oti6858.ko 2023-12-07T09:51:39.800 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/pl2303.ko 2023-12-07T09:51:39.800 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/qcaux.ko 2023-12-07T09:51:39.800 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/qcserial.ko 2023-12-07T09:51:39.801 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/quatech2.ko 2023-12-07T09:51:39.801 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/safe_serial.ko 2023-12-07T09:51:39.801 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/sierra.ko 2023-12-07T09:51:39.801 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/spcp8x5.ko 2023-12-07T09:51:39.801 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/ssu100.ko 2023-12-07T09:51:39.801 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/symbolserial.ko 2023-12-07T09:51:39.801 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-12-07T09:51:39.801 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-12-07T09:51:39.801 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/usb_debug.ko 2023-12-07T09:51:39.801 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/usb_wwan.ko 2023-12-07T09:51:39.801 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/visor.ko 2023-12-07T09:51:39.801 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/whiteheat.ko 2023-12-07T09:51:39.801 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/serial/xsens_mt.ko 2023-12-07T09:51:39.801 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/storage 2023-12-07T09:51:39.802 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/storage/uas.ko 2023-12-07T09:51:39.802 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/storage/ums-alauda.ko 2023-12-07T09:51:39.802 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/storage/ums-cypress.ko 2023-12-07T09:51:39.802 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/storage/ums-datafab.ko 2023-12-07T09:51:39.802 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-12-07T09:51:39.802 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/storage/ums-freecom.ko 2023-12-07T09:51:39.802 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/storage/ums-isd200.ko 2023-12-07T09:51:39.802 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-12-07T09:51:39.802 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/storage/ums-karma.ko 2023-12-07T09:51:39.802 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/storage/ums-onetouch.ko 2023-12-07T09:51:39.802 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/storage/ums-realtek.ko 2023-12-07T09:51:39.802 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/storage/ums-sddr09.ko 2023-12-07T09:51:39.802 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/storage/ums-sddr55.ko 2023-12-07T09:51:39.802 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/storage/ums-usbat.ko 2023-12-07T09:51:39.802 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/storage/usb-storage.ko 2023-12-07T09:51:39.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/usbip 2023-12-07T09:51:39.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/usbip/usbip-core.ko 2023-12-07T09:51:39.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/usbip/usbip-host.ko 2023-12-07T09:51:39.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-12-07T09:51:39.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/vfio 2023-12-07T09:51:39.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/vfio/pci 2023-12-07T09:51:39.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-12-07T09:51:39.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/vfio/pci/vfio-pci.ko 2023-12-07T09:51:39.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/vfio/vfio.ko 2023-12-07T09:51:39.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-12-07T09:51:39.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/vhost 2023-12-07T09:51:39.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/vhost/vhost.ko 2023-12-07T09:51:39.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/vhost/vhost_iotlb.ko 2023-12-07T09:51:39.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/vhost/vhost_net.ko 2023-12-07T09:51:39.803 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/vhost/vhost_scsi.ko 2023-12-07T09:51:39.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/video 2023-12-07T09:51:39.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/video/backlight 2023-12-07T09:51:39.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/video/backlight/apple_bl.ko 2023-12-07T09:51:39.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/video/backlight/lcd.ko 2023-12-07T09:51:39.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/video/backlight/platform_lcd.ko 2023-12-07T09:51:39.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/virtio 2023-12-07T09:51:39.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/virtio/virtio.ko 2023-12-07T09:51:39.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/virtio/virtio_balloon.ko 2023-12-07T09:51:39.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/virtio/virtio_mmio.ko 2023-12-07T09:51:39.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/virtio/virtio_pci.ko 2023-12-07T09:51:39.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-12-07T09:51:39.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-12-07T09:51:39.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/virtio/virtio_ring.ko 2023-12-07T09:51:39.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/w1 2023-12-07T09:51:39.804 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/w1/masters 2023-12-07T09:51:39.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/w1/masters/ds2482.ko 2023-12-07T09:51:39.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/w1/masters/ds2490.ko 2023-12-07T09:51:39.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/w1/slaves 2023-12-07T09:51:39.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-12-07T09:51:39.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-12-07T09:51:39.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-12-07T09:51:39.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-12-07T09:51:39.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-12-07T09:51:39.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-12-07T09:51:39.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-12-07T09:51:39.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-12-07T09:51:39.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-12-07T09:51:39.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/w1/slaves/w1_smem.ko 2023-12-07T09:51:39.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/w1/slaves/w1_therm.ko 2023-12-07T09:51:39.805 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/w1/wire.ko 2023-12-07T09:51:39.806 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog 2023-12-07T09:51:39.806 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog/alim1535_wdt.ko 2023-12-07T09:51:39.806 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog/alim7101_wdt.ko 2023-12-07T09:51:39.806 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog/f71808e_wdt.ko 2023-12-07T09:51:39.806 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog/hpwdt.ko 2023-12-07T09:51:39.806 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog/i6300esb.ko 2023-12-07T09:51:39.806 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-12-07T09:51:39.806 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog/iTCO_wdt.ko 2023-12-07T09:51:39.806 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog/ib700wdt.ko 2023-12-07T09:51:39.806 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog/ibmasr.ko 2023-12-07T09:51:39.806 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-12-07T09:51:39.806 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog/it8712f_wdt.ko 2023-12-07T09:51:39.806 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog/it87_wdt.ko 2023-12-07T09:51:39.806 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog/machzwd.ko 2023-12-07T09:51:39.806 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog/nv_tco.ko 2023-12-07T09:51:39.807 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog/pcwd_pci.ko 2023-12-07T09:51:39.807 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog/pcwd_usb.ko 2023-12-07T09:51:39.807 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-12-07T09:51:39.807 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog/sch311x_wdt.ko 2023-12-07T09:51:39.807 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog/softdog.ko 2023-12-07T09:51:39.807 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog/sp5100_tco.ko 2023-12-07T09:51:39.807 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog/via_wdt.ko 2023-12-07T09:51:39.807 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-12-07T09:51:39.807 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog/w83877f_wdt.ko 2023-12-07T09:51:39.807 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog/w83977f_wdt.ko 2023-12-07T09:51:39.807 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog/wdt_pci.ko 2023-12-07T09:51:39.807 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/watchdog/xen_wdt.ko 2023-12-07T09:51:39.807 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/xen 2023-12-07T09:51:39.807 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/xen/xen-acpi-processor.ko 2023-12-07T09:51:39.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/xen/xen-evtchn.ko 2023-12-07T09:51:39.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/xen/xen-gntalloc.ko 2023-12-07T09:51:39.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/xen/xen-gntdev.ko 2023-12-07T09:51:39.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/xen/xen-pciback 2023-12-07T09:51:39.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-12-07T09:51:39.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/xen/xen-privcmd.ko 2023-12-07T09:51:39.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/xen/xenfs 2023-12-07T09:51:39.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/drivers/xen/xenfs/xenfs.ko 2023-12-07T09:51:39.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs 2023-12-07T09:51:39.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/9p 2023-12-07T09:51:39.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/9p/9p.ko 2023-12-07T09:51:39.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/affs 2023-12-07T09:51:39.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/affs/affs.ko 2023-12-07T09:51:39.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/befs 2023-12-07T09:51:39.808 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/befs/befs.ko 2023-12-07T09:51:39.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/binfmt_misc.ko 2023-12-07T09:51:39.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/btrfs 2023-12-07T09:51:39.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/btrfs/btrfs.ko 2023-12-07T09:51:39.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/cachefiles 2023-12-07T09:51:39.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/cachefiles/cachefiles.ko 2023-12-07T09:51:39.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/ceph 2023-12-07T09:51:39.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/ceph/ceph.ko 2023-12-07T09:51:39.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/coda 2023-12-07T09:51:39.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/coda/coda.ko 2023-12-07T09:51:39.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/cramfs 2023-12-07T09:51:39.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/cramfs/cramfs.ko 2023-12-07T09:51:39.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/dlm 2023-12-07T09:51:39.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/dlm/dlm.ko 2023-12-07T09:51:39.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/ecryptfs 2023-12-07T09:51:39.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/ecryptfs/ecryptfs.ko 2023-12-07T09:51:39.809 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/f2fs 2023-12-07T09:51:39.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/f2fs/f2fs.ko 2023-12-07T09:51:39.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/fat 2023-12-07T09:51:39.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/fat/fat.ko 2023-12-07T09:51:39.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/fat/msdos.ko 2023-12-07T09:51:39.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/fat/vfat.ko 2023-12-07T09:51:39.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/fscache 2023-12-07T09:51:39.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/fscache/fscache.ko 2023-12-07T09:51:39.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/fuse 2023-12-07T09:51:39.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/fuse/cuse.ko 2023-12-07T09:51:39.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/fuse/fuse.ko 2023-12-07T09:51:39.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/gfs2 2023-12-07T09:51:39.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/gfs2/gfs2.ko 2023-12-07T09:51:39.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/hfs 2023-12-07T09:51:39.810 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/hfs/hfs.ko 2023-12-07T09:51:39.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/hfsplus 2023-12-07T09:51:39.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/hfsplus/hfsplus.ko 2023-12-07T09:51:39.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/isofs 2023-12-07T09:51:39.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/isofs/isofs.ko 2023-12-07T09:51:39.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/jfs 2023-12-07T09:51:39.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/jfs/jfs.ko 2023-12-07T09:51:39.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/lockd 2023-12-07T09:51:39.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/lockd/lockd.ko 2023-12-07T09:51:39.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/minix 2023-12-07T09:51:39.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/minix/minix.ko 2023-12-07T09:51:39.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/netfs 2023-12-07T09:51:39.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/netfs/netfs.ko 2023-12-07T09:51:39.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nfs 2023-12-07T09:51:39.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nfs/blocklayout 2023-12-07T09:51:39.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-12-07T09:51:39.811 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nfs/filelayout 2023-12-07T09:51:39.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-12-07T09:51:39.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nfs/flexfilelayout 2023-12-07T09:51:39.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-12-07T09:51:39.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nfs/nfs.ko 2023-12-07T09:51:39.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nfs/nfsv3.ko 2023-12-07T09:51:39.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nfs/nfsv4.ko 2023-12-07T09:51:39.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nfs_common 2023-12-07T09:51:39.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nfs_common/grace.ko 2023-12-07T09:51:39.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nfs_common/nfs_acl.ko 2023-12-07T09:51:39.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nfsd 2023-12-07T09:51:39.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nfsd/nfsd.ko 2023-12-07T09:51:39.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nilfs2 2023-12-07T09:51:39.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nilfs2/nilfs2.ko 2023-12-07T09:51:39.812 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls 2023-12-07T09:51:39.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/mac-celtic.ko 2023-12-07T09:51:39.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/mac-centeuro.ko 2023-12-07T09:51:39.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/mac-croatian.ko 2023-12-07T09:51:39.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/mac-cyrillic.ko 2023-12-07T09:51:39.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/mac-gaelic.ko 2023-12-07T09:51:39.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/mac-greek.ko 2023-12-07T09:51:39.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/mac-iceland.ko 2023-12-07T09:51:39.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/mac-inuit.ko 2023-12-07T09:51:39.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/mac-roman.ko 2023-12-07T09:51:39.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/mac-romanian.ko 2023-12-07T09:51:39.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/mac-turkish.ko 2023-12-07T09:51:39.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_cp1250.ko 2023-12-07T09:51:39.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_cp1251.ko 2023-12-07T09:51:39.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_cp1255.ko 2023-12-07T09:51:39.813 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_cp737.ko 2023-12-07T09:51:39.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_cp775.ko 2023-12-07T09:51:39.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_cp850.ko 2023-12-07T09:51:39.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_cp852.ko 2023-12-07T09:51:39.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_cp855.ko 2023-12-07T09:51:39.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_cp857.ko 2023-12-07T09:51:39.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_cp860.ko 2023-12-07T09:51:39.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_cp861.ko 2023-12-07T09:51:39.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_cp862.ko 2023-12-07T09:51:39.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_cp863.ko 2023-12-07T09:51:39.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_cp864.ko 2023-12-07T09:51:39.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_cp865.ko 2023-12-07T09:51:39.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_cp866.ko 2023-12-07T09:51:39.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_cp869.ko 2023-12-07T09:51:39.814 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_cp874.ko 2023-12-07T09:51:39.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_cp932.ko 2023-12-07T09:51:39.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_cp936.ko 2023-12-07T09:51:39.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_cp949.ko 2023-12-07T09:51:39.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_cp950.ko 2023-12-07T09:51:39.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_euc-jp.ko 2023-12-07T09:51:39.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_iso8859-1.ko 2023-12-07T09:51:39.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_iso8859-13.ko 2023-12-07T09:51:39.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_iso8859-14.ko 2023-12-07T09:51:39.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_iso8859-15.ko 2023-12-07T09:51:39.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_iso8859-2.ko 2023-12-07T09:51:39.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_iso8859-3.ko 2023-12-07T09:51:39.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_iso8859-4.ko 2023-12-07T09:51:39.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_iso8859-5.ko 2023-12-07T09:51:39.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_iso8859-6.ko 2023-12-07T09:51:39.815 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_iso8859-7.ko 2023-12-07T09:51:39.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_iso8859-9.ko 2023-12-07T09:51:39.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_koi8-r.ko 2023-12-07T09:51:39.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_koi8-ru.ko 2023-12-07T09:51:39.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_koi8-u.ko 2023-12-07T09:51:39.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_ucs2_utils.ko 2023-12-07T09:51:39.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/nls/nls_utf8.ko 2023-12-07T09:51:39.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/ocfs2 2023-12-07T09:51:39.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/ocfs2/cluster 2023-12-07T09:51:39.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-12-07T09:51:39.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/ocfs2/dlm 2023-12-07T09:51:39.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-12-07T09:51:39.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/ocfs2/dlmfs 2023-12-07T09:51:39.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-12-07T09:51:39.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/ocfs2/ocfs2.ko 2023-12-07T09:51:39.816 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-12-07T09:51:39.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-12-07T09:51:39.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-12-07T09:51:39.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/overlayfs 2023-12-07T09:51:39.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/overlayfs/overlay.ko 2023-12-07T09:51:39.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/pstore 2023-12-07T09:51:39.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/pstore/ramoops.ko 2023-12-07T09:51:39.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/reiserfs 2023-12-07T09:51:39.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/reiserfs/reiserfs.ko 2023-12-07T09:51:39.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/romfs 2023-12-07T09:51:39.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/romfs/romfs.ko 2023-12-07T09:51:39.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/smb 2023-12-07T09:51:39.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/smb/client 2023-12-07T09:51:39.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/smb/client/cifs.ko 2023-12-07T09:51:39.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/smb/common 2023-12-07T09:51:39.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/smb/common/cifs_arc4.ko 2023-12-07T09:51:39.817 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/smb/common/cifs_md4.ko 2023-12-07T09:51:39.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/squashfs 2023-12-07T09:51:39.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/squashfs/squashfs.ko 2023-12-07T09:51:39.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/sysv 2023-12-07T09:51:39.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/sysv/sysv.ko 2023-12-07T09:51:39.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/udf 2023-12-07T09:51:39.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/udf/udf.ko 2023-12-07T09:51:39.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/ufs 2023-12-07T09:51:39.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/ufs/ufs.ko 2023-12-07T09:51:39.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/xfs 2023-12-07T09:51:39.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/fs/xfs/xfs.ko 2023-12-07T09:51:39.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/kernel 2023-12-07T09:51:39.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/kernel/rcu 2023-12-07T09:51:39.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/kernel/rcu/rcutorture.ko 2023-12-07T09:51:39.818 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/kernel/torture.ko 2023-12-07T09:51:39.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/kernel/trace 2023-12-07T09:51:39.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-12-07T09:51:39.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/lib 2023-12-07T09:51:39.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/lib/asn1_encoder.ko 2023-12-07T09:51:39.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/lib/crc-itu-t.ko 2023-12-07T09:51:39.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/lib/crc8.ko 2023-12-07T09:51:39.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/lib/crypto 2023-12-07T09:51:39.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/lib/crypto/libarc4.ko 2023-12-07T09:51:39.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/lib/crypto/libdes.ko 2023-12-07T09:51:39.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/lib/libcrc32c.ko 2023-12-07T09:51:39.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/lib/lru_cache.ko 2023-12-07T09:51:39.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/lib/lz4 2023-12-07T09:51:39.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/lib/lz4/lz4_compress.ko 2023-12-07T09:51:39.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/lib/lz4/lz4hc_compress.ko 2023-12-07T09:51:39.819 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/lib/math 2023-12-07T09:51:39.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/lib/math/cordic.ko 2023-12-07T09:51:39.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/lib/raid6 2023-12-07T09:51:39.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/lib/raid6/raid6_pq.ko 2023-12-07T09:51:39.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/lib/reed_solomon 2023-12-07T09:51:39.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/lib/reed_solomon/reed_solomon.ko 2023-12-07T09:51:39.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/lib/ts_bm.ko 2023-12-07T09:51:39.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/lib/ts_fsm.ko 2023-12-07T09:51:39.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/lib/ts_kmp.ko 2023-12-07T09:51:39.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/lib/zstd 2023-12-07T09:51:39.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/lib/zstd/zstd_compress.ko 2023-12-07T09:51:39.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/mm 2023-12-07T09:51:39.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/mm/hwpoison-inject.ko 2023-12-07T09:51:39.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net 2023-12-07T09:51:39.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/6lowpan 2023-12-07T09:51:39.820 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/6lowpan/6lowpan.ko 2023-12-07T09:51:39.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/6lowpan/nhc_dest.ko 2023-12-07T09:51:39.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/6lowpan/nhc_fragment.ko 2023-12-07T09:51:39.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/6lowpan/nhc_hop.ko 2023-12-07T09:51:39.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/6lowpan/nhc_ipv6.ko 2023-12-07T09:51:39.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/6lowpan/nhc_mobility.ko 2023-12-07T09:51:39.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/6lowpan/nhc_routing.ko 2023-12-07T09:51:39.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/6lowpan/nhc_udp.ko 2023-12-07T09:51:39.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/802 2023-12-07T09:51:39.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/802/garp.ko 2023-12-07T09:51:39.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/802/mrp.ko 2023-12-07T09:51:39.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/802/p8022.ko 2023-12-07T09:51:39.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/802/psnap.ko 2023-12-07T09:51:39.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/802/stp.ko 2023-12-07T09:51:39.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/8021q 2023-12-07T09:51:39.821 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/8021q/8021q.ko 2023-12-07T09:51:39.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/9p 2023-12-07T09:51:39.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/9p/9pnet.ko 2023-12-07T09:51:39.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/9p/9pnet_fd.ko 2023-12-07T09:51:39.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/9p/9pnet_rdma.ko 2023-12-07T09:51:39.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/9p/9pnet_virtio.ko 2023-12-07T09:51:39.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/batman-adv 2023-12-07T09:51:39.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/batman-adv/batman-adv.ko 2023-12-07T09:51:39.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge 2023-12-07T09:51:39.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/br_netfilter.ko 2023-12-07T09:51:39.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/bridge.ko 2023-12-07T09:51:39.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/netfilter 2023-12-07T09:51:39.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-12-07T09:51:39.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/netfilter/ebt_among.ko 2023-12-07T09:51:39.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/netfilter/ebt_arp.ko 2023-12-07T09:51:39.822 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-12-07T09:51:39.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-12-07T09:51:39.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/netfilter/ebt_ip.ko 2023-12-07T09:51:39.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-12-07T09:51:39.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/netfilter/ebt_limit.ko 2023-12-07T09:51:39.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/netfilter/ebt_log.ko 2023-12-07T09:51:39.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/netfilter/ebt_mark.ko 2023-12-07T09:51:39.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-12-07T09:51:39.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-12-07T09:51:39.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-12-07T09:51:39.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-12-07T09:51:39.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/netfilter/ebt_snat.ko 2023-12-07T09:51:39.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/netfilter/ebt_stp.ko 2023-12-07T09:51:39.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-12-07T09:51:39.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-12-07T09:51:39.823 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-12-07T09:51:39.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-12-07T09:51:39.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/netfilter/ebtables.ko 2023-12-07T09:51:39.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-12-07T09:51:39.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-12-07T09:51:39.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-12-07T09:51:39.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ceph 2023-12-07T09:51:39.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ceph/libceph.ko 2023-12-07T09:51:39.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/core 2023-12-07T09:51:39.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/core/failover.ko 2023-12-07T09:51:39.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/core/pktgen.ko 2023-12-07T09:51:39.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/dccp 2023-12-07T09:51:39.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/dccp/dccp.ko 2023-12-07T09:51:39.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/dccp/dccp_diag.ko 2023-12-07T09:51:39.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/dccp/dccp_ipv4.ko 2023-12-07T09:51:39.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/dccp/dccp_ipv6.ko 2023-12-07T09:51:39.824 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/dns_resolver 2023-12-07T09:51:39.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/dns_resolver/dns_resolver.ko 2023-12-07T09:51:39.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/dsa 2023-12-07T09:51:39.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/dsa/dsa_core.ko 2023-12-07T09:51:39.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/dsa/tag_brcm.ko 2023-12-07T09:51:39.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/dsa/tag_dsa.ko 2023-12-07T09:51:39.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/dsa/tag_none.ko 2023-12-07T09:51:39.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/dsa/tag_trailer.ko 2023-12-07T09:51:39.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ieee802154 2023-12-07T09:51:39.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ieee802154/6lowpan 2023-12-07T09:51:39.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-12-07T09:51:39.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ieee802154/ieee802154.ko 2023-12-07T09:51:39.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ieee802154/ieee802154_socket.ko 2023-12-07T09:51:39.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4 2023-12-07T09:51:39.825 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/ah4.ko 2023-12-07T09:51:39.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/esp4.ko 2023-12-07T09:51:39.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/fou.ko 2023-12-07T09:51:39.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/gre.ko 2023-12-07T09:51:39.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/inet_diag.ko 2023-12-07T09:51:39.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/ip_gre.ko 2023-12-07T09:51:39.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/ip_tunnel.ko 2023-12-07T09:51:39.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/ip_vti.ko 2023-12-07T09:51:39.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/ipcomp.ko 2023-12-07T09:51:39.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/ipip.ko 2023-12-07T09:51:39.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/netfilter 2023-12-07T09:51:39.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/netfilter/arp_tables.ko 2023-12-07T09:51:39.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-12-07T09:51:39.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-12-07T09:51:39.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-12-07T09:51:39.826 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-12-07T09:51:39.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-12-07T09:51:39.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-12-07T09:51:39.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-12-07T09:51:39.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-12-07T09:51:39.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-12-07T09:51:39.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/netfilter/iptable_security.ko 2023-12-07T09:51:39.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-12-07T09:51:39.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-12-07T09:51:39.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-12-07T09:51:39.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-12-07T09:51:39.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-12-07T09:51:39.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-12-07T09:51:39.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-12-07T09:51:39.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-12-07T09:51:39.827 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-12-07T09:51:39.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-12-07T09:51:39.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/tcp_bic.ko 2023-12-07T09:51:39.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/tcp_dctcp.ko 2023-12-07T09:51:39.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/tcp_diag.ko 2023-12-07T09:51:39.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/tcp_highspeed.ko 2023-12-07T09:51:39.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/tcp_htcp.ko 2023-12-07T09:51:39.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/tcp_hybla.ko 2023-12-07T09:51:39.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/tcp_illinois.ko 2023-12-07T09:51:39.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/tcp_lp.ko 2023-12-07T09:51:39.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/tcp_scalable.ko 2023-12-07T09:51:39.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/tcp_vegas.ko 2023-12-07T09:51:39.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/tcp_veno.ko 2023-12-07T09:51:39.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/tcp_westwood.ko 2023-12-07T09:51:39.828 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/tcp_yeah.ko 2023-12-07T09:51:39.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/tunnel4.ko 2023-12-07T09:51:39.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/udp_diag.ko 2023-12-07T09:51:39.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/udp_tunnel.ko 2023-12-07T09:51:39.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv4/xfrm4_tunnel.ko 2023-12-07T09:51:39.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6 2023-12-07T09:51:39.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/ah6.ko 2023-12-07T09:51:39.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/esp6.ko 2023-12-07T09:51:39.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/fou6.ko 2023-12-07T09:51:39.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/ip6_tunnel.ko 2023-12-07T09:51:39.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-12-07T09:51:39.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/ip6_vti.ko 2023-12-07T09:51:39.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/ipcomp6.ko 2023-12-07T09:51:39.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/netfilter 2023-12-07T09:51:39.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-12-07T09:51:39.829 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-12-07T09:51:39.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-12-07T09:51:39.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-12-07T09:51:39.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-12-07T09:51:39.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-12-07T09:51:39.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-12-07T09:51:39.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-12-07T09:51:39.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-12-07T09:51:39.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-12-07T09:51:39.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-12-07T09:51:39.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-12-07T09:51:39.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-12-07T09:51:39.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-12-07T09:51:39.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-12-07T09:51:39.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-12-07T09:51:39.830 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-12-07T09:51:39.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-12-07T09:51:39.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-12-07T09:51:39.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-12-07T09:51:39.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-12-07T09:51:39.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-12-07T09:51:39.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-12-07T09:51:39.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-12-07T09:51:39.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/sit.ko 2023-12-07T09:51:39.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/tunnel6.ko 2023-12-07T09:51:39.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/ipv6/xfrm6_tunnel.ko 2023-12-07T09:51:39.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/key 2023-12-07T09:51:39.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/key/af_key.ko 2023-12-07T09:51:39.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/l2tp 2023-12-07T09:51:39.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/l2tp/l2tp_core.ko 2023-12-07T09:51:39.831 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/l2tp/l2tp_debugfs.ko 2023-12-07T09:51:39.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/l2tp/l2tp_eth.ko 2023-12-07T09:51:39.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/l2tp/l2tp_ip.ko 2023-12-07T09:51:39.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/l2tp/l2tp_ip6.ko 2023-12-07T09:51:39.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/l2tp/l2tp_netlink.ko 2023-12-07T09:51:39.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/l2tp/l2tp_ppp.ko 2023-12-07T09:51:39.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/llc 2023-12-07T09:51:39.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/llc/llc.ko 2023-12-07T09:51:39.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/mac802154 2023-12-07T09:51:39.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/mac802154/mac802154.ko 2023-12-07T09:51:39.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/mpls 2023-12-07T09:51:39.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/mpls/mpls_gso.ko 2023-12-07T09:51:39.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter 2023-12-07T09:51:39.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipset 2023-12-07T09:51:39.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipset/ip_set.ko 2023-12-07T09:51:39.832 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-12-07T09:51:39.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-12-07T09:51:39.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-12-07T09:51:39.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-12-07T09:51:39.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-12-07T09:51:39.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-12-07T09:51:39.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-12-07T09:51:39.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-12-07T09:51:39.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-12-07T09:51:39.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-12-07T09:51:39.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-12-07T09:51:39.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-12-07T09:51:39.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-12-07T09:51:39.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-12-07T09:51:39.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-12-07T09:51:39.833 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipvs 2023-12-07T09:51:39.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipvs/ip_vs.ko 2023-12-07T09:51:39.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-12-07T09:51:39.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-12-07T09:51:39.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-12-07T09:51:39.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-12-07T09:51:39.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-12-07T09:51:39.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-12-07T09:51:39.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-12-07T09:51:39.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-12-07T09:51:39.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-12-07T09:51:39.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-12-07T09:51:39.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-12-07T09:51:39.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-12-07T09:51:39.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-12-07T09:51:39.834 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nf_conncount.ko 2023-12-07T09:51:39.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nf_conntrack.ko 2023-12-07T09:51:39.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-12-07T09:51:39.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-12-07T09:51:39.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-12-07T09:51:39.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nf_conntrack_h323.ko 2023-12-07T09:51:39.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nf_conntrack_irc.ko 2023-12-07T09:51:39.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-12-07T09:51:39.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-12-07T09:51:39.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-12-07T09:51:39.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nf_conntrack_sane.ko 2023-12-07T09:51:39.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nf_conntrack_sip.ko 2023-12-07T09:51:39.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-12-07T09:51:39.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-12-07T09:51:39.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nf_dup_netdev.ko 2023-12-07T09:51:39.835 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nf_flow_table.ko 2023-12-07T09:51:39.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nf_flow_table_inet.ko 2023-12-07T09:51:39.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nf_log_syslog.ko 2023-12-07T09:51:39.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nf_nat.ko 2023-12-07T09:51:39.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nf_nat_amanda.ko 2023-12-07T09:51:39.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nf_nat_ftp.ko 2023-12-07T09:51:39.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nf_nat_irc.ko 2023-12-07T09:51:39.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nf_nat_sip.ko 2023-12-07T09:51:39.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nf_nat_tftp.ko 2023-12-07T09:51:39.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nf_synproxy_core.ko 2023-12-07T09:51:39.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nf_tables.ko 2023-12-07T09:51:39.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nfnetlink.ko 2023-12-07T09:51:39.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nfnetlink_acct.ko 2023-12-07T09:51:39.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nfnetlink_log.ko 2023-12-07T09:51:39.836 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nfnetlink_osf.ko 2023-12-07T09:51:39.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nfnetlink_queue.ko 2023-12-07T09:51:39.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nft_chain_nat.ko 2023-12-07T09:51:39.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nft_compat.ko 2023-12-07T09:51:39.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nft_ct.ko 2023-12-07T09:51:39.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nft_dup_netdev.ko 2023-12-07T09:51:39.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nft_fib.ko 2023-12-07T09:51:39.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nft_fwd_netdev.ko 2023-12-07T09:51:39.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nft_hash.ko 2023-12-07T09:51:39.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nft_limit.ko 2023-12-07T09:51:39.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nft_log.ko 2023-12-07T09:51:39.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nft_masq.ko 2023-12-07T09:51:39.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nft_nat.ko 2023-12-07T09:51:39.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nft_numgen.ko 2023-12-07T09:51:39.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nft_queue.ko 2023-12-07T09:51:39.837 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nft_quota.ko 2023-12-07T09:51:39.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nft_redir.ko 2023-12-07T09:51:39.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nft_reject.ko 2023-12-07T09:51:39.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nft_reject_inet.ko 2023-12-07T09:51:39.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nft_socket.ko 2023-12-07T09:51:39.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/nft_tunnel.ko 2023-12-07T09:51:39.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_AUDIT.ko 2023-12-07T09:51:39.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_CHECKSUM.ko 2023-12-07T09:51:39.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_CLASSIFY.ko 2023-12-07T09:51:39.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-12-07T09:51:39.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_CT.ko 2023-12-07T09:51:39.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_DSCP.ko 2023-12-07T09:51:39.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_HL.ko 2023-12-07T09:51:39.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_HMARK.ko 2023-12-07T09:51:39.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_IDLETIMER.ko 2023-12-07T09:51:39.838 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_LED.ko 2023-12-07T09:51:39.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_LOG.ko 2023-12-07T09:51:39.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_MASQUERADE.ko 2023-12-07T09:51:39.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_NETMAP.ko 2023-12-07T09:51:39.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_NFLOG.ko 2023-12-07T09:51:39.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_NFQUEUE.ko 2023-12-07T09:51:39.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_RATEEST.ko 2023-12-07T09:51:39.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_REDIRECT.ko 2023-12-07T09:51:39.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_SECMARK.ko 2023-12-07T09:51:39.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_TCPMSS.ko 2023-12-07T09:51:39.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-12-07T09:51:39.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_TEE.ko 2023-12-07T09:51:39.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_TPROXY.ko 2023-12-07T09:51:39.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_TRACE.ko 2023-12-07T09:51:39.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_addrtype.ko 2023-12-07T09:51:39.839 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_bpf.ko 2023-12-07T09:51:39.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_cgroup.ko 2023-12-07T09:51:39.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_cluster.ko 2023-12-07T09:51:39.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_comment.ko 2023-12-07T09:51:39.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_connbytes.ko 2023-12-07T09:51:39.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_connlabel.ko 2023-12-07T09:51:39.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_connlimit.ko 2023-12-07T09:51:39.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_connmark.ko 2023-12-07T09:51:39.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_conntrack.ko 2023-12-07T09:51:39.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_cpu.ko 2023-12-07T09:51:39.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_dccp.ko 2023-12-07T09:51:39.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_devgroup.ko 2023-12-07T09:51:39.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_dscp.ko 2023-12-07T09:51:39.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_ecn.ko 2023-12-07T09:51:39.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_esp.ko 2023-12-07T09:51:39.840 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_hashlimit.ko 2023-12-07T09:51:39.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_helper.ko 2023-12-07T09:51:39.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_hl.ko 2023-12-07T09:51:39.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_ipcomp.ko 2023-12-07T09:51:39.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_iprange.ko 2023-12-07T09:51:39.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_ipvs.ko 2023-12-07T09:51:39.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_l2tp.ko 2023-12-07T09:51:39.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_length.ko 2023-12-07T09:51:39.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_limit.ko 2023-12-07T09:51:39.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_mac.ko 2023-12-07T09:51:39.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_mark.ko 2023-12-07T09:51:39.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_multiport.ko 2023-12-07T09:51:39.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_nat.ko 2023-12-07T09:51:39.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_nfacct.ko 2023-12-07T09:51:39.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_osf.ko 2023-12-07T09:51:39.841 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_owner.ko 2023-12-07T09:51:39.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_physdev.ko 2023-12-07T09:51:39.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_pkttype.ko 2023-12-07T09:51:39.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_policy.ko 2023-12-07T09:51:39.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_quota.ko 2023-12-07T09:51:39.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_rateest.ko 2023-12-07T09:51:39.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_realm.ko 2023-12-07T09:51:39.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_recent.ko 2023-12-07T09:51:39.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_sctp.ko 2023-12-07T09:51:39.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_set.ko 2023-12-07T09:51:39.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_socket.ko 2023-12-07T09:51:39.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_state.ko 2023-12-07T09:51:39.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_statistic.ko 2023-12-07T09:51:39.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_string.ko 2023-12-07T09:51:39.842 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_tcpmss.ko 2023-12-07T09:51:39.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_time.ko 2023-12-07T09:51:39.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netfilter/xt_u32.ko 2023-12-07T09:51:39.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netlink 2023-12-07T09:51:39.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/netlink/netlink_diag.ko 2023-12-07T09:51:39.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/nsh 2023-12-07T09:51:39.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/nsh/nsh.ko 2023-12-07T09:51:39.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/openvswitch 2023-12-07T09:51:39.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/openvswitch/openvswitch.ko 2023-12-07T09:51:39.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/openvswitch/vport-geneve.ko 2023-12-07T09:51:39.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/openvswitch/vport-gre.ko 2023-12-07T09:51:39.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/openvswitch/vport-vxlan.ko 2023-12-07T09:51:39.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/packet 2023-12-07T09:51:39.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/packet/af_packet_diag.ko 2023-12-07T09:51:39.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/rds 2023-12-07T09:51:39.843 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/rds/rds.ko 2023-12-07T09:51:39.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/rds/rds_rdma.ko 2023-12-07T09:51:39.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/rds/rds_tcp.ko 2023-12-07T09:51:39.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched 2023-12-07T09:51:39.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/act_csum.ko 2023-12-07T09:51:39.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/act_gact.ko 2023-12-07T09:51:39.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/act_ipt.ko 2023-12-07T09:51:39.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/act_mirred.ko 2023-12-07T09:51:39.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/act_nat.ko 2023-12-07T09:51:39.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/act_pedit.ko 2023-12-07T09:51:39.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/act_police.ko 2023-12-07T09:51:39.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/act_simple.ko 2023-12-07T09:51:39.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/act_skbedit.ko 2023-12-07T09:51:39.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/act_vlan.ko 2023-12-07T09:51:39.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/cls_basic.ko 2023-12-07T09:51:39.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/cls_bpf.ko 2023-12-07T09:51:39.844 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/cls_flow.ko 2023-12-07T09:51:39.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/cls_fw.ko 2023-12-07T09:51:39.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/cls_route.ko 2023-12-07T09:51:39.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/cls_u32.ko 2023-12-07T09:51:39.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/em_cmp.ko 2023-12-07T09:51:39.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/em_ipset.ko 2023-12-07T09:51:39.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/em_meta.ko 2023-12-07T09:51:39.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/em_nbyte.ko 2023-12-07T09:51:39.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/em_text.ko 2023-12-07T09:51:39.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/em_u32.ko 2023-12-07T09:51:39.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/sch_choke.ko 2023-12-07T09:51:39.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/sch_codel.ko 2023-12-07T09:51:39.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/sch_drr.ko 2023-12-07T09:51:39.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/sch_fq.ko 2023-12-07T09:51:39.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/sch_gred.ko 2023-12-07T09:51:39.845 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/sch_hfsc.ko 2023-12-07T09:51:39.846 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/sch_hhf.ko 2023-12-07T09:51:39.846 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/sch_htb.ko 2023-12-07T09:51:39.846 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/sch_ingress.ko 2023-12-07T09:51:39.846 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/sch_mqprio.ko 2023-12-07T09:51:39.846 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/sch_mqprio_lib.ko 2023-12-07T09:51:39.846 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/sch_multiq.ko 2023-12-07T09:51:39.846 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/sch_netem.ko 2023-12-07T09:51:39.846 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/sch_pie.ko 2023-12-07T09:51:39.846 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/sch_plug.ko 2023-12-07T09:51:39.846 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/sch_prio.ko 2023-12-07T09:51:39.846 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/sch_qfq.ko 2023-12-07T09:51:39.846 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/sch_red.ko 2023-12-07T09:51:39.846 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/sch_sfb.ko 2023-12-07T09:51:39.846 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/sch_sfq.ko 2023-12-07T09:51:39.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/sch_tbf.ko 2023-12-07T09:51:39.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sched/sch_teql.ko 2023-12-07T09:51:39.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sctp 2023-12-07T09:51:39.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sctp/sctp.ko 2023-12-07T09:51:39.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sctp/sctp_diag.ko 2023-12-07T09:51:39.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sunrpc 2023-12-07T09:51:39.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sunrpc/auth_gss 2023-12-07T09:51:39.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-12-07T09:51:39.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-12-07T09:51:39.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sunrpc/sunrpc.ko 2023-12-07T09:51:39.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sunrpc/xprtrdma 2023-12-07T09:51:39.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-12-07T09:51:39.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/tipc 2023-12-07T09:51:39.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/tipc/diag.ko 2023-12-07T09:51:39.847 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/tipc/tipc.ko 2023-12-07T09:51:39.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/unix 2023-12-07T09:51:39.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/unix/unix_diag.ko 2023-12-07T09:51:39.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/vmw_vsock 2023-12-07T09:51:39.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-12-07T09:51:39.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-12-07T09:51:39.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/vmw_vsock/vsock.ko 2023-12-07T09:51:39.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/vmw_vsock/vsock_diag.ko 2023-12-07T09:51:39.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/vmw_vsock/vsock_loopback.ko 2023-12-07T09:51:39.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/xfrm 2023-12-07T09:51:39.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/net/xfrm/xfrm_ipcomp.ko 2023-12-07T09:51:39.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/security 2023-12-07T09:51:39.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/security/keys 2023-12-07T09:51:39.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/security/keys/encrypted-keys 2023-12-07T09:51:39.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-12-07T09:51:39.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/security/keys/trusted-keys 2023-12-07T09:51:39.848 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/security/keys/trusted-keys/trusted.ko 2023-12-07T09:51:39.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/virt 2023-12-07T09:51:39.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/virt/lib 2023-12-07T09:51:39.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/kernel/virt/lib/irqbypass.ko 2023-12-07T09:51:39.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/modules.alias 2023-12-07T09:51:39.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/modules.alias.bin 2023-12-07T09:51:39.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/modules.builtin 2023-12-07T09:51:39.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/modules.builtin.bin 2023-12-07T09:51:39.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/modules.builtin.modinfo 2023-12-07T09:51:39.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/modules.dep 2023-12-07T09:51:39.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/modules.dep.bin 2023-12-07T09:51:39.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/modules.devname 2023-12-07T09:51:39.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/modules.order 2023-12-07T09:51:39.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/modules.softdep 2023-12-07T09:51:39.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/modules.symbols 2023-12-07T09:51:39.849 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-g9471594ae3a4/modules.symbols.bin 2023-12-07T09:51:39.850 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g9471594ae3a4 2023-12-07T09:51:39.850 DEBUG:teuthology.orchestra.run.smithi119:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-12-07T09:51:39.877 INFO:teuthology.orchestra.run.smithi119.stdout:postinstall scriptlet (using /bin/sh): 2023-12-07T09:51:39.877 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-g9471594ae3a4 -a -r /boot/System.map-6.6.0-g9471594ae3a4 ]; then 2023-12-07T09:51:39.877 INFO:teuthology.orchestra.run.smithi119.stdout:cp /boot/vmlinuz-6.6.0-g9471594ae3a4 /boot/.vmlinuz-6.6.0-g9471594ae3a4-rpm 2023-12-07T09:51:39.877 INFO:teuthology.orchestra.run.smithi119.stdout:cp /boot/System.map-6.6.0-g9471594ae3a4 /boot/.System.map-6.6.0-g9471594ae3a4-rpm 2023-12-07T09:51:39.877 INFO:teuthology.orchestra.run.smithi119.stdout:rm -f /boot/vmlinuz-6.6.0-g9471594ae3a4 /boot/System.map-6.6.0-g9471594ae3a4 2023-12-07T09:51:39.877 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/installkernel 6.6.0-g9471594ae3a4 /boot/.vmlinuz-6.6.0-g9471594ae3a4-rpm /boot/.System.map-6.6.0-g9471594ae3a4-rpm 2023-12-07T09:51:39.877 INFO:teuthology.orchestra.run.smithi119.stdout:rm -f /boot/.vmlinuz-6.6.0-g9471594ae3a4-rpm /boot/.System.map-6.6.0-g9471594ae3a4-rpm 2023-12-07T09:51:39.877 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2023-12-07T09:51:39.878 INFO:teuthology.orchestra.run.smithi119.stdout:preuninstall scriptlet (using /bin/sh): 2023-12-07T09:51:39.878 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-12-07T09:51:39.878 INFO:teuthology.orchestra.run.smithi119.stdout:new-kernel-pkg --remove 6.6.0-g9471594ae3a4 --rminitrd --initrdfile=/boot/initramfs-6.6.0-g9471594ae3a4.img 2023-12-07T09:51:39.878 INFO:teuthology.orchestra.run.smithi119.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-12-07T09:51:39.878 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-install remove 6.6.0-g9471594ae3a4 2023-12-07T09:51:39.878 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2023-12-07T09:51:39.878 INFO:teuthology.orchestra.run.smithi119.stdout:postuninstall scriptlet (using /bin/sh): 2023-12-07T09:51:39.878 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -x /sbin/update-bootloader ]; then 2023-12-07T09:51:39.878 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/update-bootloader --remove 6.6.0-g9471594ae3a4 2023-12-07T09:51:39.878 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2023-12-07T09:51:39.879 DEBUG:teuthology.orchestra.run.smithi119:> sudo rpm -qi grub2-tools 2023-12-07T09:51:39.958 INFO:teuthology.orchestra.run.smithi119.stdout:Name : grub2-tools 2023-12-07T09:51:39.958 INFO:teuthology.orchestra.run.smithi119.stdout:Epoch : 1 2023-12-07T09:51:39.958 INFO:teuthology.orchestra.run.smithi119.stdout:Version : 2.02 2023-12-07T09:51:39.958 INFO:teuthology.orchestra.run.smithi119.stdout:Release : 123.el8 2023-12-07T09:51:39.958 INFO:teuthology.orchestra.run.smithi119.stdout:Architecture: x86_64 2023-12-07T09:51:39.959 INFO:teuthology.orchestra.run.smithi119.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-12-07T09:51:39.959 INFO:teuthology.orchestra.run.smithi119.stdout:Group : System Environment/Base 2023-12-07T09:51:39.959 INFO:teuthology.orchestra.run.smithi119.stdout:Size : 9265450 2023-12-07T09:51:39.959 INFO:teuthology.orchestra.run.smithi119.stdout:License : GPLv3+ 2023-12-07T09:51:39.959 INFO:teuthology.orchestra.run.smithi119.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-12-07T09:51:39.959 INFO:teuthology.orchestra.run.smithi119.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-12-07T09:51:39.959 INFO:teuthology.orchestra.run.smithi119.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-12-07T09:51:39.959 INFO:teuthology.orchestra.run.smithi119.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-12-07T09:51:39.959 INFO:teuthology.orchestra.run.smithi119.stdout:Relocations : (not relocatable) 2023-12-07T09:51:39.959 INFO:teuthology.orchestra.run.smithi119.stdout:Packager : Red Hat, Inc. 2023-12-07T09:51:39.959 INFO:teuthology.orchestra.run.smithi119.stdout:Vendor : Red Hat, Inc. 2023-12-07T09:51:39.959 INFO:teuthology.orchestra.run.smithi119.stdout:URL : http://www.gnu.org/software/grub/ 2023-12-07T09:51:39.959 INFO:teuthology.orchestra.run.smithi119.stdout:Summary : Support tools for GRUB. 2023-12-07T09:51:39.959 INFO:teuthology.orchestra.run.smithi119.stdout:Description : 2023-12-07T09:51:39.960 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T09:51:39.960 INFO:teuthology.orchestra.run.smithi119.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-12-07T09:51:39.960 INFO:teuthology.orchestra.run.smithi119.stdout:customizable bootloader with modular architecture. It supports a rich 2023-12-07T09:51:39.960 INFO:teuthology.orchestra.run.smithi119.stdout:variety of kernel formats, file systems, computer architectures and 2023-12-07T09:51:39.960 INFO:teuthology.orchestra.run.smithi119.stdout:hardware devices. 2023-12-07T09:51:39.960 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T09:51:39.960 INFO:teuthology.orchestra.run.smithi119.stdout:This subpackage provides tools for support of all platforms. 2023-12-07T09:51:39.961 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-12-07T09:51:39.961 INFO:teuthology.task.kernel:Updating grub on smithi119 to boot 6.6.0-g9471594ae3a4 2023-12-07T09:51:39.961 DEBUG:teuthology.orchestra.run.smithi119:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-12-07T09:51:42.144 INFO:teuthology.orchestra.run.smithi119.stderr:Generating grub configuration file ... 2023-12-07T09:51:44.214 INFO:teuthology.orchestra.run.smithi119.stderr:done 2023-12-07T09:51:44.216 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2023-12-07T09:51:44.233 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.2jMvnuuke6 2023-12-07T09:51:44.233 DEBUG:teuthology.orchestra.run.smithi119:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.2jMvnuuke6 2023-12-07T09:51:44.304 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.2jMvnuuke6 2023-12-07T09:51:44.454 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.2jMvnuuke6 is 6KB 2023-12-07T09:51:44.504 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.2jMvnuuke6 2023-12-07T09:51:44.520 DEBUG:teuthology.orchestra.run.smithi119:> sudo /bin/ls /boot/loader/entries || true 2023-12-07T09:51:44.587 INFO:teuthology.orchestra.run.smithi119.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-12-07T09:51:44.587 INFO:teuthology.orchestra.run.smithi119.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-12-07T09:51:44.587 INFO:teuthology.orchestra.run.smithi119.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.6.0-g9471594ae3a4.conf 2023-12-07T09:51:44.589 DEBUG:teuthology.orchestra.run.smithi119:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.6.0-g9471594ae3a4 2023-12-07T09:51:44.735 DEBUG:teuthology.orchestra.run.smithi119:> sudo shutdown -r now 2023-12-07T09:52:14.738 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-12-07T09:52:14.738 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-12-07T09:52:14.739 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-07T09:52:49.254 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2023-12-07T09:52:58.256 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-12-07T09:52:58.257 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-07T09:53:16.642 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2023-12-07T09:53:28.643 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-12-07T09:53:28.644 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-07T09:53:31.319 DEBUG:teuthology.orchestra.run.smithi119:> true 2023-12-07T09:53:31.765 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-12-07T09:53:31.765 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-g9471594ae3a4"... 2023-12-07T09:53:31.766 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2023-12-07T09:53:31.810 INFO:teuthology.orchestra.run.smithi119.stdout:6.6.0-g9471594ae3a4 2023-12-07T09:53:31.810 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g9471594ae3a4 vs 6.6.0-g9471594ae3a4 2023-12-07T09:53:31.810 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-07T09:53:31.811 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-12-07T09:53:32.811 DEBUG:teuthology.parallel:result is None 2023-12-07T09:53:32.811 INFO:teuthology.run_tasks:Running task internal.base... 2023-12-07T09:53:32.821 INFO:teuthology.task.internal:Creating test directory... 2023-12-07T09:53:32.821 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-07T09:53:32.827 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-07T09:53:32.829 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-07T09:53:32.870 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-12-07T09:53:32.877 INFO:teuthology.run_tasks:Running task internal.archive... 2023-12-07T09:53:32.884 INFO:teuthology.task.internal:Creating archive directory... 2023-12-07T09:53:32.884 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-07T09:53:32.888 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-07T09:53:32.890 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-07T09:53:32.942 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-12-07T09:53:32.949 INFO:teuthology.task.internal:Enabling coredump saving... 2023-12-07T09:53:32.949 DEBUG:teuthology.orchestra.run.smithi153:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-07T09:53:32.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-07T09:53:32.980 DEBUG:teuthology.orchestra.run.smithi159:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-07T09:53:32.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-07T09:53:33.000 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-07T09:53:33.004 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-07T09:53:33.022 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-07T09:53:33.080 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-07T09:53:33.081 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-07T09:53:33.096 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-07T09:53:33.101 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-07T09:53:33.264 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-07T09:53:33.299 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-07T09:53:33.303 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-12-07T09:53:33.310 INFO:teuthology.task.internal:Configuring sudo... 2023-12-07T09:53:33.310 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-07T09:53:33.314 DEBUG:teuthology.orchestra.run.smithi153:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-07T09:53:33.316 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-07T09:53:33.379 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-12-07T09:53:33.388 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-12-07T09:53:33.389 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-07T09:53:33.393 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-07T09:53:33.395 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-07T09:53:33.432 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-07T09:53:33.545 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-07T09:53:33.635 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-07T09:53:33.708 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-07T09:53:33.776 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-12-07T09:53:33.776 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-07T09:53:33.846 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-07T09:53:33.893 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-07T09:53:33.955 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-07T09:53:34.002 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-07T09:53:34.045 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-12-07T09:53:34.045 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-07T09:53:34.115 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-07T09:53:34.161 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-07T09:53:34.219 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-07T09:53:34.308 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-07T09:53:34.348 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-12-07T09:53:34.348 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-07T09:53:34.446 DEBUG:teuthology.orchestra.run.smithi119:> sudo service rsyslog restart 2023-12-07T09:53:34.451 DEBUG:teuthology.orchestra.run.smithi153:> sudo service rsyslog restart 2023-12-07T09:53:34.453 DEBUG:teuthology.orchestra.run.smithi159:> sudo service rsyslog restart 2023-12-07T09:53:34.514 INFO:teuthology.orchestra.run.smithi153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-07T09:53:34.554 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-07T09:53:34.575 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-07T09:53:35.032 INFO:teuthology.run_tasks:Running task internal.timer... 2023-12-07T09:53:35.041 INFO:teuthology.task.internal:Starting timer... 2023-12-07T09:53:35.041 INFO:teuthology.run_tasks:Running task pcp... 2023-12-07T09:53:35.129 INFO:teuthology.run_tasks:Running task selinux... 2023-12-07T09:53:35.137 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-12-07T09:53:35.138 DEBUG:teuthology.orchestra.run.smithi119:> sudo service auditd rotate 2023-12-07T09:53:35.264 INFO:teuthology.orchestra.run.smithi119.stdout:Rotating logs: 2023-12-07T09:53:35.266 DEBUG:teuthology.orchestra.run.smithi153:> sudo service auditd rotate 2023-12-07T09:53:35.320 INFO:teuthology.orchestra.run.smithi153.stdout:Rotating logs: 2023-12-07T09:53:35.322 DEBUG:teuthology.orchestra.run.smithi159:> sudo service auditd rotate 2023-12-07T09:53:35.386 INFO:teuthology.orchestra.run.smithi159.stdout:Rotating logs: 2023-12-07T09:53:35.388 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-12-07T09:53:35.388 DEBUG:teuthology.orchestra.run.smithi119:> /usr/sbin/getenforce 2023-12-07T09:53:35.436 INFO:teuthology.orchestra.run.smithi119.stdout:Permissive 2023-12-07T09:53:35.437 DEBUG:teuthology.orchestra.run.smithi153:> /usr/sbin/getenforce 2023-12-07T09:53:35.461 INFO:teuthology.orchestra.run.smithi153.stdout:Permissive 2023-12-07T09:53:35.462 DEBUG:teuthology.orchestra.run.smithi159:> /usr/sbin/getenforce 2023-12-07T09:53:35.483 INFO:teuthology.orchestra.run.smithi159.stdout:Permissive 2023-12-07T09:53:35.483 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi119.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi153.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi159.front.sepia.ceph.com': 'permissive'} 2023-12-07T09:53:35.483 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-07T09:53:35.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-07T09:53:35.554 DEBUG:teuthology.orchestra.run.smithi153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-07T09:53:35.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-07T09:53:35.581 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-07T09:53:35.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-07T09:53:35.609 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-12-07T09:53:35.609 DEBUG:teuthology.orchestra.run.smithi119:> sudo /usr/sbin/setenforce permissive 2023-12-07T09:53:35.679 DEBUG:teuthology.orchestra.run.smithi153:> sudo /usr/sbin/setenforce permissive 2023-12-07T09:53:35.706 DEBUG:teuthology.orchestra.run.smithi159:> sudo /usr/sbin/setenforce permissive 2023-12-07T09:53:35.743 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-12-07T09:53:35.753 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-12-07T09:53:35.759 INFO:teuthology.repo_utils:Fetching main from origin 2023-12-07T09:53:35.891 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-12-07T09:53:35.908 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-12-07T09:53:35.909 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi119.front.sepia.ceph.com,smithi153.front.sepia.ceph.com,smithi159.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-12-07T10:06:03.285 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi119.front.sepia.ceph.com'), Remote(name='ubuntu@smithi153.front.sepia.ceph.com'), Remote(name='ubuntu@smithi159.front.sepia.ceph.com')] 2023-12-07T10:06:03.286 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-12-07T10:06:03.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-07T10:06:03.375 DEBUG:teuthology.orchestra.run.smithi119:> true 2023-12-07T10:06:03.496 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-12-07T10:06:03.497 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2023-12-07T10:06:03.497 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-07T10:06:03.566 DEBUG:teuthology.orchestra.run.smithi153:> true 2023-12-07T10:06:03.638 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi153.front.sepia.ceph.com' 2023-12-07T10:06:03.638 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2023-12-07T10:06:03.639 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-07T10:06:03.707 DEBUG:teuthology.orchestra.run.smithi159:> true 2023-12-07T10:06:03.781 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com' 2023-12-07T10:06:03.782 INFO:teuthology.run_tasks:Running task clock... 2023-12-07T10:06:03.791 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-12-07T10:06:03.792 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-07T10:06:03.792 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-07T10:06:03.797 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-07T10:06:03.797 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-07T10:06:03.799 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-07T10:06:03.799 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-07T10:06:03.833 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-07T10:06:03.849 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-07T10:06:03.855 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-07T10:06:03.867 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-07T10:06:03.871 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-07T10:06:03.901 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-07T10:06:03.917 INFO:teuthology.orchestra.run.smithi153.stderr:sudo: ntpd: command not found 2023-12-07T10:06:03.930 INFO:teuthology.orchestra.run.smithi153.stdout:506 Cannot talk to daemon 2023-12-07T10:06:03.935 INFO:teuthology.orchestra.run.smithi159.stderr:sudo: ntpd: command not found 2023-12-07T10:06:03.945 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-07T10:06:03.947 INFO:teuthology.orchestra.run.smithi159.stdout:506 Cannot talk to daemon 2023-12-07T10:06:03.961 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-07T10:06:03.962 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-07T10:06:03.976 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-07T10:06:03.994 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: ntpd: command not found 2023-12-07T10:06:04.023 INFO:teuthology.orchestra.run.smithi119.stdout:506 Cannot talk to daemon 2023-12-07T10:06:04.043 INFO:teuthology.orchestra.run.smithi159.stderr:bash: ntpq: command not found 2023-12-07T10:06:04.045 INFO:teuthology.orchestra.run.smithi153.stderr:bash: ntpq: command not found 2023-12-07T10:06:04.046 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-07T10:06:04.046 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2023-12-07T10:06:04.046 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-07T10:06:04.046 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-07T10:06:04.046 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-07T10:06:04.046 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-07T10:06:04.049 INFO:teuthology.orchestra.run.smithi153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-07T10:06:04.049 INFO:teuthology.orchestra.run.smithi153.stdout:=============================================================================== 2023-12-07T10:06:04.049 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-07T10:06:04.049 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-07T10:06:04.049 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-07T10:06:04.049 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-07T10:06:04.057 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-07T10:06:04.091 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-07T10:06:04.178 INFO:teuthology.orchestra.run.smithi119.stderr:bash: ntpq: command not found 2023-12-07T10:06:04.186 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-07T10:06:04.186 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2023-12-07T10:06:04.186 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-07T10:06:04.186 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-07T10:06:04.186 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-07T10:06:04.186 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-07T10:06:04.187 INFO:teuthology.run_tasks:Running task pexec... 2023-12-07T10:06:04.200 INFO:teuthology.task.pexec:Executing custom commands... 2023-12-07T10:06:04.200 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi119.front.sepia.ceph.com 2023-12-07T10:06:04.200 DEBUG:teuthology.orchestra.run.smithi119:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-07T10:06:04.201 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi153.front.sepia.ceph.com 2023-12-07T10:06:04.201 DEBUG:teuthology.orchestra.run.smithi153:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-07T10:06:04.201 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi159.front.sepia.ceph.com 2023-12-07T10:06:04.201 DEBUG:teuthology.orchestra.run.smithi159:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-07T10:06:04.574 INFO:teuthology.orchestra.run.smithi159.stdout:Updating Subscription Management repositories. 2023-12-07T10:06:04.574 INFO:teuthology.orchestra.run.smithi153.stdout:Updating Subscription Management repositories. 2023-12-07T10:06:04.682 INFO:teuthology.orchestra.run.smithi119.stdout:Updating Subscription Management repositories. 2023-12-07T10:06:05.296 INFO:teuthology.orchestra.run.smithi159.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 115 kB/s | 4.1 kB 00:00 2023-12-07T10:06:05.314 INFO:teuthology.orchestra.run.smithi153.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 140 kB/s | 4.1 kB 00:00 2023-12-07T10:06:05.435 INFO:teuthology.orchestra.run.smithi119.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 163 kB/s | 4.5 kB 00:00 2023-12-07T10:06:05.496 INFO:teuthology.orchestra.run.smithi159.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 170 kB/s | 4.5 kB 00:00 2023-12-07T10:06:05.526 INFO:teuthology.orchestra.run.smithi153.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 138 kB/s | 4.5 kB 00:00 2023-12-07T10:06:05.775 INFO:teuthology.orchestra.run.smithi119.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 121 kB/s | 4.1 kB 00:00 2023-12-07T10:06:07.558 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-12-07T10:06:07.559 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-12-07T10:06:07.559 INFO:teuthology.orchestra.run.smithi159.stdout: Package Architecture Version Repository Size 2023-12-07T10:06:07.559 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-12-07T10:06:07.560 INFO:teuthology.orchestra.run.smithi159.stdout:Resetting modules: 2023-12-07T10:06:07.560 INFO:teuthology.orchestra.run.smithi159.stdout: container-tools 2023-12-07T10:06:07.560 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-12-07T10:06:07.560 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2023-12-07T10:06:07.560 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2023-12-07T10:06:07.560 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-12-07T10:06:07.596 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2023-12-07T10:06:07.597 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2023-12-07T10:06:07.598 INFO:teuthology.orchestra.run.smithi153.stdout: Package Architecture Version Repository Size 2023-12-07T10:06:07.598 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2023-12-07T10:06:07.598 INFO:teuthology.orchestra.run.smithi153.stdout:Resetting modules: 2023-12-07T10:06:07.598 INFO:teuthology.orchestra.run.smithi153.stdout: container-tools 2023-12-07T10:06:07.598 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-12-07T10:06:07.598 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2023-12-07T10:06:07.598 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2023-12-07T10:06:07.598 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-12-07T10:06:07.756 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-12-07T10:06:07.757 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-12-07T10:06:07.757 INFO:teuthology.orchestra.run.smithi119.stdout: Package Architecture Version Repository Size 2023-12-07T10:06:07.757 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-12-07T10:06:07.757 INFO:teuthology.orchestra.run.smithi119.stdout:Resetting modules: 2023-12-07T10:06:07.757 INFO:teuthology.orchestra.run.smithi119.stdout: container-tools 2023-12-07T10:06:07.757 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:06:07.757 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-12-07T10:06:07.757 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-12-07T10:06:07.757 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:06:11.503 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-12-07T10:06:11.611 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2023-12-07T10:06:11.821 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-12-07T10:06:11.981 INFO:teuthology.orchestra.run.smithi159.stdout:Updating Subscription Management repositories. 2023-12-07T10:06:12.082 INFO:teuthology.orchestra.run.smithi153.stdout:Updating Subscription Management repositories. 2023-12-07T10:06:12.406 INFO:teuthology.orchestra.run.smithi119.stdout:Updating Subscription Management repositories. 2023-12-07T10:06:12.632 INFO:teuthology.orchestra.run.smithi159.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 146 kB/s | 4.1 kB 00:00 2023-12-07T10:06:12.747 INFO:teuthology.orchestra.run.smithi153.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 151 kB/s | 4.1 kB 00:00 2023-12-07T10:06:12.840 INFO:teuthology.orchestra.run.smithi159.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 168 kB/s | 4.5 kB 00:00 2023-12-07T10:06:12.953 INFO:teuthology.orchestra.run.smithi153.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 139 kB/s | 4.5 kB 00:00 2023-12-07T10:06:13.136 INFO:teuthology.orchestra.run.smithi119.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 167 kB/s | 4.5 kB 00:00 2023-12-07T10:06:13.480 INFO:teuthology.orchestra.run.smithi119.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 111 kB/s | 4.1 kB 00:00 2023-12-07T10:06:14.893 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-12-07T10:06:14.987 INFO:teuthology.orchestra.run.smithi159.stdout:============================================================================================================ 2023-12-07T10:06:14.987 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2023-12-07T10:06:14.987 INFO:teuthology.orchestra.run.smithi159.stdout:============================================================================================================ 2023-12-07T10:06:14.987 INFO:teuthology.orchestra.run.smithi159.stdout:Installing group/module packages: 2023-12-07T10:06:14.987 INFO:teuthology.orchestra.run.smithi159.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-12-07T10:06:14.987 INFO:teuthology.orchestra.run.smithi159.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-12-07T10:06:14.988 INFO:teuthology.orchestra.run.smithi159.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-12-07T10:06:14.988 INFO:teuthology.orchestra.run.smithi159.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-12-07T10:06:14.988 INFO:teuthology.orchestra.run.smithi159.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-12-07T10:06:14.988 INFO:teuthology.orchestra.run.smithi159.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-12-07T10:06:14.988 INFO:teuthology.orchestra.run.smithi159.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-12-07T10:06:14.988 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2023-12-07T10:06:14.988 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-12-07T10:06:14.988 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-12-07T10:06:14.988 INFO:teuthology.orchestra.run.smithi159.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-12-07T10:06:14.988 INFO:teuthology.orchestra.run.smithi159.stdout:Installing module profiles: 2023-12-07T10:06:14.988 INFO:teuthology.orchestra.run.smithi159.stdout: container-tools/common 2023-12-07T10:06:14.988 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-12-07T10:06:14.988 INFO:teuthology.orchestra.run.smithi159.stdout:Enabling module streams: 2023-12-07T10:06:14.989 INFO:teuthology.orchestra.run.smithi159.stdout: container-tools rhel8 2023-12-07T10:06:14.989 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-12-07T10:06:14.989 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2023-12-07T10:06:14.989 INFO:teuthology.orchestra.run.smithi159.stdout:============================================================================================================ 2023-12-07T10:06:14.989 INFO:teuthology.orchestra.run.smithi159.stdout:Install 10 Packages 2023-12-07T10:06:14.989 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-12-07T10:06:14.989 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 18 M 2023-12-07T10:06:14.989 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 65 M 2023-12-07T10:06:14.990 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2023-12-07T10:06:15.026 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2023-12-07T10:06:15.054 INFO:teuthology.orchestra.run.smithi159.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.4 MB/s | 94 kB 00:00 2023-12-07T10:06:15.079 INFO:teuthology.orchestra.run.smithi159.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 277 kB/s | 25 kB 00:00 2023-12-07T10:06:15.105 INFO:teuthology.orchestra.run.smithi159.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 354 kB/s | 41 kB 00:00 2023-12-07T10:06:15.118 INFO:teuthology.orchestra.run.smithi153.stdout:============================================================================================================ 2023-12-07T10:06:15.119 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2023-12-07T10:06:15.119 INFO:teuthology.orchestra.run.smithi153.stdout:============================================================================================================ 2023-12-07T10:06:15.119 INFO:teuthology.orchestra.run.smithi153.stdout:Installing group/module packages: 2023-12-07T10:06:15.119 INFO:teuthology.orchestra.run.smithi153.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-12-07T10:06:15.119 INFO:teuthology.orchestra.run.smithi153.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-12-07T10:06:15.119 INFO:teuthology.orchestra.run.smithi153.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-12-07T10:06:15.119 INFO:teuthology.orchestra.run.smithi153.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-12-07T10:06:15.119 INFO:teuthology.orchestra.run.smithi153.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-12-07T10:06:15.119 INFO:teuthology.orchestra.run.smithi153.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-12-07T10:06:15.119 INFO:teuthology.orchestra.run.smithi153.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-12-07T10:06:15.120 INFO:teuthology.orchestra.run.smithi153.stdout:Installing dependencies: 2023-12-07T10:06:15.120 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-12-07T10:06:15.120 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-12-07T10:06:15.120 INFO:teuthology.orchestra.run.smithi153.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-12-07T10:06:15.120 INFO:teuthology.orchestra.run.smithi153.stdout:Installing module profiles: 2023-12-07T10:06:15.120 INFO:teuthology.orchestra.run.smithi153.stdout: container-tools/common 2023-12-07T10:06:15.120 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-12-07T10:06:15.120 INFO:teuthology.orchestra.run.smithi153.stdout:Enabling module streams: 2023-12-07T10:06:15.120 INFO:teuthology.orchestra.run.smithi153.stdout: container-tools rhel8 2023-12-07T10:06:15.120 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-12-07T10:06:15.120 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2023-12-07T10:06:15.121 INFO:teuthology.orchestra.run.smithi153.stdout:============================================================================================================ 2023-12-07T10:06:15.121 INFO:teuthology.orchestra.run.smithi153.stdout:Install 10 Packages 2023-12-07T10:06:15.121 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-12-07T10:06:15.121 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 18 M 2023-12-07T10:06:15.121 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 65 M 2023-12-07T10:06:15.122 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2023-12-07T10:06:15.130 INFO:teuthology.orchestra.run.smithi159.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 653 kB/s | 49 kB 00:00 2023-12-07T10:06:15.164 INFO:teuthology.orchestra.run.smithi159.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-12-07T10:06:15.180 INFO:teuthology.orchestra.run.smithi159.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.9 MB/s | 149 kB 00:00 2023-12-07T10:06:15.195 INFO:teuthology.orchestra.run.smithi153.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.2 MB/s | 94 kB 00:00 2023-12-07T10:06:15.214 INFO:teuthology.orchestra.run.smithi159.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-12-07T10:06:15.220 INFO:teuthology.orchestra.run.smithi153.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 250 kB/s | 25 kB 00:00 2023-12-07T10:06:15.246 INFO:teuthology.orchestra.run.smithi153.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 328 kB/s | 41 kB 00:00 2023-12-07T10:06:15.271 INFO:teuthology.orchestra.run.smithi153.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 654 kB/s | 49 kB 00:00 2023-12-07T10:06:15.304 INFO:teuthology.orchestra.run.smithi153.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-12-07T10:06:15.322 INFO:teuthology.orchestra.run.smithi153.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.9 MB/s | 149 kB 00:00 2023-12-07T10:06:15.323 INFO:teuthology.orchestra.run.smithi159.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 20 MB/s | 2.2 MB 00:00 2023-12-07T10:06:15.355 INFO:teuthology.orchestra.run.smithi153.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-12-07T10:06:15.433 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-12-07T10:06:15.439 INFO:teuthology.orchestra.run.smithi159.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 27 MB/s | 7.0 MB 00:00 2023-12-07T10:06:15.464 INFO:teuthology.orchestra.run.smithi153.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 20 MB/s | 2.2 MB 00:00 2023-12-07T10:06:15.523 INFO:teuthology.orchestra.run.smithi159.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 19 MB/s | 8.0 MB 00:00 2023-12-07T10:06:15.524 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2023-12-07T10:06:15.524 INFO:teuthology.orchestra.run.smithi159.stdout:Total 34 MB/s | 18 MB 00:00 2023-12-07T10:06:15.532 INFO:teuthology.orchestra.run.smithi119.stdout:============================================================================================================ 2023-12-07T10:06:15.532 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2023-12-07T10:06:15.532 INFO:teuthology.orchestra.run.smithi119.stdout:============================================================================================================ 2023-12-07T10:06:15.532 INFO:teuthology.orchestra.run.smithi119.stdout:Installing group/module packages: 2023-12-07T10:06:15.532 INFO:teuthology.orchestra.run.smithi119.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-12-07T10:06:15.532 INFO:teuthology.orchestra.run.smithi119.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-12-07T10:06:15.532 INFO:teuthology.orchestra.run.smithi119.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-12-07T10:06:15.532 INFO:teuthology.orchestra.run.smithi119.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-12-07T10:06:15.532 INFO:teuthology.orchestra.run.smithi119.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-12-07T10:06:15.532 INFO:teuthology.orchestra.run.smithi119.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-12-07T10:06:15.532 INFO:teuthology.orchestra.run.smithi119.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-12-07T10:06:15.532 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2023-12-07T10:06:15.532 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-12-07T10:06:15.533 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-12-07T10:06:15.533 INFO:teuthology.orchestra.run.smithi119.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-12-07T10:06:15.533 INFO:teuthology.orchestra.run.smithi119.stdout:Installing module profiles: 2023-12-07T10:06:15.533 INFO:teuthology.orchestra.run.smithi119.stdout: container-tools/common 2023-12-07T10:06:15.533 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:06:15.533 INFO:teuthology.orchestra.run.smithi119.stdout:Enabling module streams: 2023-12-07T10:06:15.533 INFO:teuthology.orchestra.run.smithi119.stdout: container-tools rhel8 2023-12-07T10:06:15.533 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:06:15.533 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-12-07T10:06:15.533 INFO:teuthology.orchestra.run.smithi119.stdout:============================================================================================================ 2023-12-07T10:06:15.533 INFO:teuthology.orchestra.run.smithi119.stdout:Install 10 Packages 2023-12-07T10:06:15.533 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:06:15.534 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 18 M 2023-12-07T10:06:15.534 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 65 M 2023-12-07T10:06:15.534 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-12-07T10:06:15.572 INFO:teuthology.orchestra.run.smithi153.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 28 MB/s | 7.0 MB 00:00 2023-12-07T10:06:15.609 INFO:teuthology.orchestra.run.smithi119.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.2 MB/s | 94 kB 00:00 2023-12-07T10:06:15.660 INFO:teuthology.orchestra.run.smithi119.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 199 kB/s | 25 kB 00:00 2023-12-07T10:06:15.681 INFO:teuthology.orchestra.run.smithi153.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 18 MB/s | 8.0 MB 00:00 2023-12-07T10:06:15.681 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2023-12-07T10:06:15.681 INFO:teuthology.orchestra.run.smithi153.stdout:Total 33 MB/s | 18 MB 00:00 2023-12-07T10:06:15.693 INFO:teuthology.orchestra.run.smithi119.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 258 kB/s | 41 kB 00:00 2023-12-07T10:06:15.718 INFO:teuthology.orchestra.run.smithi119.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 454 kB/s | 49 kB 00:00 2023-12-07T10:06:15.725 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2023-12-07T10:06:15.744 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2023-12-07T10:06:15.744 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2023-12-07T10:06:15.752 INFO:teuthology.orchestra.run.smithi119.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.6 MB/s | 524 kB 00:00 2023-12-07T10:06:15.777 INFO:teuthology.orchestra.run.smithi119.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-12-07T10:06:15.803 INFO:teuthology.orchestra.run.smithi119.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-12-07T10:06:15.883 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2023-12-07T10:06:15.902 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2023-12-07T10:06:15.902 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2023-12-07T10:06:15.903 INFO:teuthology.orchestra.run.smithi119.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 22 MB/s | 2.2 MB 00:00 2023-12-07T10:06:15.925 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2023-12-07T10:06:15.929 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2023-12-07T10:06:16.056 INFO:teuthology.orchestra.run.smithi119.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 25 MB/s | 7.0 MB 00:00 2023-12-07T10:06:16.083 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2023-12-07T10:06:16.087 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2023-12-07T10:06:16.153 INFO:teuthology.orchestra.run.smithi119.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 17 MB/s | 8.0 MB 00:00 2023-12-07T10:06:16.154 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-12-07T10:06:16.154 INFO:teuthology.orchestra.run.smithi119.stdout:Total 29 MB/s | 18 MB 00:00 2023-12-07T10:06:16.458 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2023-12-07T10:06:16.489 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-12-07T10:06:16.512 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-12-07T10:06:16.512 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-12-07T10:06:16.606 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2023-12-07T10:06:16.643 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-12-07T10:06:16.799 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-07T10:06:16.820 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-12-07T10:06:16.980 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-07T10:06:16.984 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-12-07T10:06:17.020 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-12-07T10:06:17.025 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-12-07T10:06:17.166 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-12-07T10:06:17.205 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-12-07T10:06:17.392 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-12-07T10:06:17.552 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-12-07T10:06:17.598 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-12-07T10:06:17.724 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-12-07T10:06:17.793 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-12-07T10:06:17.978 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-07T10:06:18.237 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-12-07T10:06:18.332 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-12-07T10:06:18.462 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-12-07T10:06:18.497 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-12-07T10:06:18.889 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-12-07T10:06:19.250 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-12-07T10:06:19.361 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-12-07T10:06:19.485 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-12-07T10:06:19.587 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-12-07T10:06:19.714 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-12-07T10:06:19.756 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-12-07T10:06:19.795 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-12-07T10:06:19.949 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-07T10:06:19.963 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-07T10:06:20.223 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-07T10:06:20.223 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-12-07T10:06:20.223 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-07T10:06:20.223 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-12-07T10:06:20.223 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-12-07T10:06:20.223 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-12-07T10:06:20.223 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-12-07T10:06:20.223 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-12-07T10:06:20.224 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-12-07T10:06:20.224 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-12-07T10:06:20.233 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-07T10:06:20.233 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-12-07T10:06:20.233 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-07T10:06:20.233 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-12-07T10:06:20.233 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-12-07T10:06:20.233 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-12-07T10:06:20.233 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-12-07T10:06:20.234 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-12-07T10:06:20.234 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-12-07T10:06:20.234 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-12-07T10:06:20.640 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-12-07T10:06:20.942 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-12-07T10:06:21.158 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-12-07T10:06:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-07T10:06:21.738 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-07T10:06:21.738 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-12-07T10:06:21.739 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-07T10:06:21.739 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-12-07T10:06:21.739 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-12-07T10:06:21.739 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-12-07T10:06:21.739 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-12-07T10:06:21.739 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-12-07T10:06:21.739 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-12-07T10:06:21.739 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-12-07T10:06:24.456 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-12-07T10:06:24.456 INFO:teuthology.orchestra.run.smithi159.stdout:Installed products updated. 2023-12-07T10:06:24.656 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-12-07T10:06:24.656 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2023-12-07T10:06:24.656 INFO:teuthology.orchestra.run.smithi159.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-07T10:06:24.656 INFO:teuthology.orchestra.run.smithi159.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-12-07T10:06:24.656 INFO:teuthology.orchestra.run.smithi159.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-07T10:06:24.657 INFO:teuthology.orchestra.run.smithi159.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-12-07T10:06:24.657 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-12-07T10:06:24.657 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-12-07T10:06:24.657 INFO:teuthology.orchestra.run.smithi159.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-07T10:06:24.657 INFO:teuthology.orchestra.run.smithi159.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-12-07T10:06:24.657 INFO:teuthology.orchestra.run.smithi159.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-12-07T10:06:24.657 INFO:teuthology.orchestra.run.smithi159.stdout: yajl-2.1.0-10.el8.x86_64 2023-12-07T10:06:24.657 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-12-07T10:06:24.657 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-12-07T10:06:24.807 DEBUG:teuthology.parallel:result is None 2023-12-07T10:06:24.867 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-12-07T10:06:24.867 INFO:teuthology.orchestra.run.smithi153.stdout:Installed products updated. 2023-12-07T10:06:25.105 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-12-07T10:06:25.105 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2023-12-07T10:06:25.105 INFO:teuthology.orchestra.run.smithi153.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-07T10:06:25.105 INFO:teuthology.orchestra.run.smithi153.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-12-07T10:06:25.105 INFO:teuthology.orchestra.run.smithi153.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-07T10:06:25.105 INFO:teuthology.orchestra.run.smithi153.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-12-07T10:06:25.105 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-12-07T10:06:25.106 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-12-07T10:06:25.106 INFO:teuthology.orchestra.run.smithi153.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-07T10:06:25.106 INFO:teuthology.orchestra.run.smithi153.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-12-07T10:06:25.106 INFO:teuthology.orchestra.run.smithi153.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-12-07T10:06:25.106 INFO:teuthology.orchestra.run.smithi153.stdout: yajl-2.1.0-10.el8.x86_64 2023-12-07T10:06:25.106 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-12-07T10:06:25.106 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2023-12-07T10:06:25.259 DEBUG:teuthology.parallel:result is None 2023-12-07T10:06:27.407 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-12-07T10:06:27.407 INFO:teuthology.orchestra.run.smithi119.stdout:Installed products updated. 2023-12-07T10:06:27.607 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:06:27.607 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2023-12-07T10:06:27.607 INFO:teuthology.orchestra.run.smithi119.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-07T10:06:27.607 INFO:teuthology.orchestra.run.smithi119.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-12-07T10:06:27.607 INFO:teuthology.orchestra.run.smithi119.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-07T10:06:27.607 INFO:teuthology.orchestra.run.smithi119.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-12-07T10:06:27.607 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-12-07T10:06:27.607 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-12-07T10:06:27.607 INFO:teuthology.orchestra.run.smithi119.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-07T10:06:27.607 INFO:teuthology.orchestra.run.smithi119.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-12-07T10:06:27.607 INFO:teuthology.orchestra.run.smithi119.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-12-07T10:06:27.607 INFO:teuthology.orchestra.run.smithi119.stdout: yajl-2.1.0-10.el8.x86_64 2023-12-07T10:06:27.608 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:06:27.608 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-12-07T10:06:27.824 DEBUG:teuthology.parallel:result is None 2023-12-07T10:06:27.824 INFO:teuthology.run_tasks:Running task install... 2023-12-07T10:06:27.833 DEBUG:teuthology.task.install:project ceph 2023-12-07T10:06:27.833 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3bd0bfa458b4399e7266c56d8215a2cf3a6091a0'}} 2023-12-07T10:06:27.833 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3bd0bfa458b4399e7266c56d8215a2cf3a6091a0'} 2023-12-07T10:06:27.833 INFO:teuthology.task.install:Using flavor: default 2023-12-07T10:06:27.839 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-12-07T10:06:27.840 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-12-07T10:06:27.840 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3bd0bfa458b4399e7266c56d8215a2cf3a6091a0', 'tag': None, 'wait_for_package': False} 2023-12-07T10:06:27.840 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 2023-12-07T10:06:27.843 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3bd0bfa458b4399e7266c56d8215a2cf3a6091a0', 'tag': None, 'wait_for_package': False} 2023-12-07T10:06:27.843 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 2023-12-07T10:06:27.844 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3bd0bfa458b4399e7266c56d8215a2cf3a6091a0', 'tag': None, 'wait_for_package': False} 2023-12-07T10:06:27.844 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 2023-12-07T10:06:28.001 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654-r2/3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/centos/8/flavors/default/ 2023-12-07T10:06:28.001 INFO:teuthology.task.install.rpm:Package version is 19.0.0-18.g3bd0bfa4 2023-12-07T10:06:28.062 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654-r2/3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/centos/8/flavors/default/ 2023-12-07T10:06:28.062 INFO:teuthology.task.install.rpm:Package version is 19.0.0-18.g3bd0bfa4 2023-12-07T10:06:28.067 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654-r2/3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/centos/8/flavors/default/ 2023-12-07T10:06:28.068 INFO:teuthology.task.install.rpm:Package version is 19.0.0-18.g3bd0bfa4 2023-12-07T10:06:28.116 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654-r2/3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654-r2/3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654-r2/3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-07T10:06:28.116 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-12-07T10:06:28.116 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-07T10:06:28.142 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-07T10:06:28.142 DEBUG:teuthology.orchestra.run.smithi159:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-07T10:06:28.174 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654-r2/3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654-r2/3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654-r2/3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-07T10:06:28.174 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-12-07T10:06:28.174 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-07T10:06:28.200 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-07T10:06:28.200 DEBUG:teuthology.orchestra.run.smithi153:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-07T10:06:28.209 DEBUG:teuthology.orchestra.run.smithi159:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-07T10:06:28.233 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654-r2/3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654-r2/3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654-r2/3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-07T10:06:28.233 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-12-07T10:06:28.233 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-07T10:06:28.267 DEBUG:teuthology.orchestra.run.smithi153:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-07T10:06:28.286 DEBUG:teuthology.orchestra.run.smithi159:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-07T10:06:28.302 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-07T10:06:28.302 DEBUG:teuthology.orchestra.run.smithi119:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-07T10:06:28.346 DEBUG:teuthology.orchestra.run.smithi153:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-07T10:06:28.353 INFO:teuthology.orchestra.run.smithi159.stdout:check_obsoletes = 1 2023-12-07T10:06:28.354 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum clean all 2023-12-07T10:06:28.412 INFO:teuthology.orchestra.run.smithi153.stdout:check_obsoletes = 1 2023-12-07T10:06:28.413 DEBUG:teuthology.orchestra.run.smithi119:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-07T10:06:28.415 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum clean all 2023-12-07T10:06:28.509 DEBUG:teuthology.orchestra.run.smithi119:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-07T10:06:28.576 INFO:teuthology.orchestra.run.smithi119.stdout:check_obsoletes = 1 2023-12-07T10:06:28.580 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum clean all 2023-12-07T10:06:28.759 INFO:teuthology.orchestra.run.smithi159.stdout:Updating Subscription Management repositories. 2023-12-07T10:06:28.807 INFO:teuthology.orchestra.run.smithi153.stdout:Updating Subscription Management repositories. 2023-12-07T10:06:29.023 INFO:teuthology.orchestra.run.smithi119.stdout:Updating Subscription Management repositories. 2023-12-07T10:06:30.097 INFO:teuthology.orchestra.run.smithi159.stdout:44 files removed 2023-12-07T10:06:30.153 INFO:teuthology.orchestra.run.smithi153.stdout:44 files removed 2023-12-07T10:06:30.157 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-07T10:06:30.216 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-07T10:06:30.458 INFO:teuthology.orchestra.run.smithi119.stdout:44 files removed 2023-12-07T10:06:30.513 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-07T10:06:30.561 INFO:teuthology.orchestra.run.smithi159.stdout:Updating Subscription Management repositories. 2023-12-07T10:06:30.581 INFO:teuthology.orchestra.run.smithi153.stdout:Updating Subscription Management repositories. 2023-12-07T10:06:30.958 INFO:teuthology.orchestra.run.smithi119.stdout:Updating Subscription Management repositories. 2023-12-07T10:06:32.162 INFO:teuthology.orchestra.run.smithi159.stdout:ceph packages for x86_64 267 kB/s | 81 kB 00:00 2023-12-07T10:06:32.208 INFO:teuthology.orchestra.run.smithi153.stdout:ceph packages for x86_64 276 kB/s | 81 kB 00:00 2023-12-07T10:06:32.497 INFO:teuthology.orchestra.run.smithi159.stdout:ceph noarch packages 31 kB/s | 9.5 kB 00:00 2023-12-07T10:06:32.509 INFO:teuthology.orchestra.run.smithi153.stdout:ceph noarch packages 36 kB/s | 9.5 kB 00:00 2023-12-07T10:06:32.665 INFO:teuthology.orchestra.run.smithi119.stdout:ceph packages for x86_64 253 kB/s | 81 kB 00:00 2023-12-07T10:06:32.799 INFO:teuthology.orchestra.run.smithi159.stdout:ceph source packages 5.3 kB/s | 1.5 kB 00:00 2023-12-07T10:06:32.894 INFO:teuthology.orchestra.run.smithi153.stdout:ceph source packages 4.2 kB/s | 1.5 kB 00:00 2023-12-07T10:06:33.001 INFO:teuthology.orchestra.run.smithi159.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2023-12-07T10:06:33.032 INFO:teuthology.orchestra.run.smithi119.stdout:ceph noarch packages 28 kB/s | 9.5 kB 00:00 2023-12-07T10:06:33.103 INFO:teuthology.orchestra.run.smithi153.stdout:Copr repo for python3-asyncssh owned by ceph 18 kB/s | 3.5 kB 00:00 2023-12-07T10:06:33.350 INFO:teuthology.orchestra.run.smithi119.stdout:ceph source packages 5.1 kB/s | 1.5 kB 00:00 2023-12-07T10:06:33.525 INFO:teuthology.orchestra.run.smithi119.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-12-07T10:06:33.578 INFO:teuthology.orchestra.run.smithi159.stdout:Extra Packages for Enterprise Linux 29 MB/s | 16 MB 00:00 2023-12-07T10:06:34.024 INFO:teuthology.orchestra.run.smithi153.stdout:Extra Packages for Enterprise Linux 18 MB/s | 16 MB 00:00 2023-12-07T10:06:36.471 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 5.5 MB/s | 16 MB 00:02 2023-12-07T10:06:37.222 INFO:teuthology.orchestra.run.smithi159.stdout:lab-extras 456 kB/s | 24 kB 00:00 2023-12-07T10:06:37.626 INFO:teuthology.orchestra.run.smithi153.stdout:lab-extras 390 kB/s | 24 kB 00:00 2023-12-07T10:06:38.148 INFO:teuthology.orchestra.run.smithi159.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 62 MB/s | 53 MB 00:00 2023-12-07T10:06:38.569 INFO:teuthology.orchestra.run.smithi153.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 61 MB/s | 53 MB 00:00 2023-12-07T10:06:40.593 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 500 kB/s | 24 kB 00:00 2023-12-07T10:06:41.629 INFO:teuthology.orchestra.run.smithi119.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 49 MB/s | 47 MB 00:00 2023-12-07T10:06:46.191 INFO:teuthology.orchestra.run.smithi159.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 61 MB/s | 47 MB 00:00 2023-12-07T10:06:46.778 INFO:teuthology.orchestra.run.smithi153.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 62 MB/s | 47 MB 00:00 2023-12-07T10:06:50.237 INFO:teuthology.orchestra.run.smithi119.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 50 MB/s | 53 MB 00:01 2023-12-07T10:06:57.082 INFO:teuthology.orchestra.run.smithi159.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-07T10:06:57.083 INFO:teuthology.orchestra.run.smithi159.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-07T10:06:57.098 INFO:teuthology.orchestra.run.smithi159.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-12-07T10:06:57.099 INFO:teuthology.orchestra.run.smithi159.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-12-07T10:06:57.100 INFO:teuthology.orchestra.run.smithi159.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-12-07T10:06:57.101 INFO:teuthology.orchestra.run.smithi159.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-12-07T10:06:57.103 INFO:teuthology.orchestra.run.smithi159.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-12-07T10:06:57.107 INFO:teuthology.orchestra.run.smithi159.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-12-07T10:06:57.108 INFO:teuthology.orchestra.run.smithi159.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-12-07T10:06:57.244 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2023-12-07T10:06:57.258 INFO:teuthology.orchestra.run.smithi159.stdout:=========================================================================================================================== 2023-12-07T10:06:57.258 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2023-12-07T10:06:57.258 INFO:teuthology.orchestra.run.smithi159.stdout:=========================================================================================================================== 2023-12-07T10:06:57.258 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2023-12-07T10:06:57.258 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-12-07T10:06:57.258 INFO:teuthology.orchestra.run.smithi159.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-12-07T10:06:57.258 INFO:teuthology.orchestra.run.smithi159.stdout: ceph x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 6.4 k 2023-12-07T10:06:57.258 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 5.4 M 2023-12-07T10:06:57.259 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 920 k 2023-12-07T10:06:57.259 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 171 k 2023-12-07T10:06:57.259 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 1.6 M 2023-12-07T10:06:57.259 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-cephadm noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 133 k 2023-12-07T10:06:57.259 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-dashboard noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 1.8 M 2023-12-07T10:06:57.259 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 7.4 M 2023-12-07T10:06:57.259 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-rook noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 50 k 2023-12-07T10:06:57.259 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 14 M 2023-12-07T10:06:57.259 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 49 M 2023-12-07T10:06:57.259 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-volume noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 261 k 2023-12-07T10:06:57.259 INFO:teuthology.orchestra.run.smithi159.stdout: cephadm noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 414 k 2023-12-07T10:06:57.259 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-mirror x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 263 k 2023-12-07T10:06:57.259 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-top noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 18 k 2023-12-07T10:06:57.259 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-12-07T10:06:57.260 INFO:teuthology.orchestra.run.smithi159.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-12-07T10:06:57.260 INFO:teuthology.orchestra.run.smithi159.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-12-07T10:06:57.260 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 31 k 2023-12-07T10:06:57.260 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2 x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 838 k 2023-12-07T10:06:57.260 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 139 k 2023-12-07T10:06:57.260 INFO:teuthology.orchestra.run.smithi159.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-12-07T10:06:57.260 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-12-07T10:06:57.260 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-12-07T10:06:57.260 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-12-07T10:06:57.260 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-12-07T10:06:57.260 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-12-07T10:06:57.260 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 197 k 2023-12-07T10:06:57.260 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 371 k 2023-12-07T10:06:57.261 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 364 k 2023-12-07T10:06:57.261 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 106 k 2023-12-07T10:06:57.261 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 93 k 2023-12-07T10:06:57.261 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 3.6 M 2023-12-07T10:06:57.261 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 181 k 2023-12-07T10:06:57.261 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-12-07T10:06:57.261 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrading: 2023-12-07T10:06:57.261 INFO:teuthology.orchestra.run.smithi159.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-12-07T10:06:57.261 INFO:teuthology.orchestra.run.smithi159.stdout: librados2 x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 3.8 M 2023-12-07T10:06:57.261 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1 x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 3.7 M 2023-12-07T10:06:57.261 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2023-12-07T10:06:57.261 INFO:teuthology.orchestra.run.smithi159.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-12-07T10:06:57.261 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 23 M 2023-12-07T10:06:57.261 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-grafana-dashboards noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 28 k 2023-12-07T10:06:57.262 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 2.3 M 2023-12-07T10:06:57.262 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 243 k 2023-12-07T10:06:57.262 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 4.6 M 2023-12-07T10:06:57.262 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 18 M 2023-12-07T10:06:57.262 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-prometheus-alerts noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 15 k 2023-12-07T10:06:57.262 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 24 k 2023-12-07T10:06:57.262 INFO:teuthology.orchestra.run.smithi159.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-12-07T10:06:57.262 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-07T10:06:57.262 INFO:teuthology.orchestra.run.smithi159.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-12-07T10:06:57.262 INFO:teuthology.orchestra.run.smithi159.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-12-07T10:06:57.262 INFO:teuthology.orchestra.run.smithi159.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-12-07T10:06:57.263 INFO:teuthology.orchestra.run.smithi159.stdout: libcephsqlite x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 183 k 2023-12-07T10:06:57.263 INFO:teuthology.orchestra.run.smithi159.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-12-07T10:06:57.263 INFO:teuthology.orchestra.run.smithi159.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-12-07T10:06:57.263 INFO:teuthology.orchestra.run.smithi159.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-07T10:06:57.263 INFO:teuthology.orchestra.run.smithi159.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-12-07T10:06:57.263 INFO:teuthology.orchestra.run.smithi159.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-12-07T10:06:57.263 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-12-07T10:06:57.263 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1 x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 518 k 2023-12-07T10:06:57.263 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-12-07T10:06:57.263 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2 x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 6.9 M 2023-12-07T10:06:57.263 INFO:teuthology.orchestra.run.smithi159.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-12-07T10:06:57.263 INFO:teuthology.orchestra.run.smithi159.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-12-07T10:06:57.263 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-07T10:06:57.264 INFO:teuthology.orchestra.run.smithi159.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-12-07T10:06:57.264 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-12-07T10:06:57.264 INFO:teuthology.orchestra.run.smithi159.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-12-07T10:06:57.264 INFO:teuthology.orchestra.run.smithi159.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-12-07T10:06:57.264 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-12-07T10:06:57.264 INFO:teuthology.orchestra.run.smithi159.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-12-07T10:06:57.264 INFO:teuthology.orchestra.run.smithi159.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-12-07T10:06:57.264 INFO:teuthology.orchestra.run.smithi159.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-07T10:06:57.264 INFO:teuthology.orchestra.run.smithi159.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-12-07T10:06:57.264 INFO:teuthology.orchestra.run.smithi159.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-12-07T10:06:57.264 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-07T10:06:57.264 INFO:teuthology.orchestra.run.smithi159.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-07T10:06:57.265 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-07T10:06:57.265 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 45 k 2023-12-07T10:06:57.265 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 128 k 2023-12-07T10:06:57.265 INFO:teuthology.orchestra.run.smithi159.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-07T10:06:57.265 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-12-07T10:06:57.265 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-07T10:06:57.265 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-07T10:06:57.265 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-12-07T10:06:57.265 INFO:teuthology.orchestra.run.smithi159.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-07T10:06:57.265 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-07T10:06:57.266 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-07T10:06:57.266 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-12-07T10:06:57.266 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-12-07T10:06:57.266 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-12-07T10:06:57.266 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-07T10:06:57.266 INFO:teuthology.orchestra.run.smithi159.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-07T10:06:57.266 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-07T10:06:57.266 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-12-07T10:06:57.266 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-07T10:06:57.266 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-07T10:06:57.266 INFO:teuthology.orchestra.run.smithi159.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-12-07T10:06:57.266 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-07T10:06:57.266 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-12-07T10:06:57.267 INFO:teuthology.orchestra.run.smithi159.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-12-07T10:06:57.267 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-07T10:06:57.267 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-07T10:06:57.267 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-12-07T10:06:57.267 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-12-07T10:06:57.267 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-12-07T10:06:57.267 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-12-07T10:06:57.267 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-12-07T10:06:57.267 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-12-07T10:06:57.267 INFO:teuthology.orchestra.run.smithi159.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-07T10:06:57.267 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-07T10:06:57.267 INFO:teuthology.orchestra.run.smithi159.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-07T10:06:57.267 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-07T10:06:57.268 INFO:teuthology.orchestra.run.smithi159.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-12-07T10:06:57.268 INFO:teuthology.orchestra.run.smithi159.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-07T10:06:57.268 INFO:teuthology.orchestra.run.smithi159.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-07T10:06:57.268 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-07T10:06:57.268 INFO:teuthology.orchestra.run.smithi159.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-07T10:06:57.268 INFO:teuthology.orchestra.run.smithi159.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-07T10:06:57.268 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-07T10:06:57.268 INFO:teuthology.orchestra.run.smithi159.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-07T10:06:57.268 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-07T10:06:57.268 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-12-07T10:06:57.268 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-07T10:06:57.268 INFO:teuthology.orchestra.run.smithi159.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-12-07T10:06:57.269 INFO:teuthology.orchestra.run.smithi159.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-12-07T10:06:57.269 INFO:teuthology.orchestra.run.smithi159.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-12-07T10:06:57.269 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-12-07T10:06:57.269 INFO:teuthology.orchestra.run.smithi159.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-12-07T10:06:57.269 INFO:teuthology.orchestra.run.smithi159.stdout:Enabling module streams: 2023-12-07T10:06:57.269 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql 10 2023-12-07T10:06:57.269 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-12-07T10:06:57.269 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2023-12-07T10:06:57.269 INFO:teuthology.orchestra.run.smithi159.stdout:=========================================================================================================================== 2023-12-07T10:06:57.269 INFO:teuthology.orchestra.run.smithi159.stdout:Install 128 Packages 2023-12-07T10:06:57.269 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrade 3 Packages 2023-12-07T10:06:57.269 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-12-07T10:06:57.270 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 209 M 2023-12-07T10:06:57.270 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2023-12-07T10:06:57.421 INFO:teuthology.orchestra.run.smithi159.stdout:(1/131): ceph-19.0.0-18.g3bd0bfa4.el8.x86_64.rp 53 kB/s | 6.4 kB 00:00 2023-12-07T10:06:57.751 INFO:teuthology.orchestra.run.smithi159.stdout:(2/131): ceph-fuse-19.0.0-18.g3bd0bfa4.el8.x86_ 2.7 MB/s | 920 kB 00:00 2023-12-07T10:06:57.790 INFO:teuthology.orchestra.run.smithi153.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-07T10:06:57.791 INFO:teuthology.orchestra.run.smithi153.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-07T10:06:57.805 INFO:teuthology.orchestra.run.smithi153.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-12-07T10:06:57.806 INFO:teuthology.orchestra.run.smithi153.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-12-07T10:06:57.807 INFO:teuthology.orchestra.run.smithi153.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-12-07T10:06:57.808 INFO:teuthology.orchestra.run.smithi153.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-12-07T10:06:57.810 INFO:teuthology.orchestra.run.smithi153.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-12-07T10:06:57.814 INFO:teuthology.orchestra.run.smithi153.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-12-07T10:06:57.815 INFO:teuthology.orchestra.run.smithi153.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-12-07T10:06:57.843 INFO:teuthology.orchestra.run.smithi159.stdout:(3/131): ceph-immutable-object-cache-19.0.0-18. 1.8 MB/s | 171 kB 00:00 2023-12-07T10:06:57.943 INFO:teuthology.orchestra.run.smithi159.stdout:(4/131): ceph-base-19.0.0-18.g3bd0bfa4.el8.x86_ 8.3 MB/s | 5.4 MB 00:00 2023-12-07T10:06:57.946 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2023-12-07T10:06:57.959 INFO:teuthology.orchestra.run.smithi153.stdout:=========================================================================================================================== 2023-12-07T10:06:57.959 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2023-12-07T10:06:57.959 INFO:teuthology.orchestra.run.smithi153.stdout:=========================================================================================================================== 2023-12-07T10:06:57.959 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2023-12-07T10:06:57.959 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-12-07T10:06:57.959 INFO:teuthology.orchestra.run.smithi153.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-12-07T10:06:57.959 INFO:teuthology.orchestra.run.smithi153.stdout: ceph x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 6.4 k 2023-12-07T10:06:57.959 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 5.4 M 2023-12-07T10:06:57.959 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 920 k 2023-12-07T10:06:57.959 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 171 k 2023-12-07T10:06:57.959 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 1.6 M 2023-12-07T10:06:57.960 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-cephadm noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 133 k 2023-12-07T10:06:57.960 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 1.8 M 2023-12-07T10:06:57.960 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 7.4 M 2023-12-07T10:06:57.960 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 50 k 2023-12-07T10:06:57.960 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 14 M 2023-12-07T10:06:57.960 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 49 M 2023-12-07T10:06:57.960 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-volume noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 261 k 2023-12-07T10:06:57.960 INFO:teuthology.orchestra.run.smithi153.stdout: cephadm noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 414 k 2023-12-07T10:06:57.960 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-mirror x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 263 k 2023-12-07T10:06:57.960 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-top noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 18 k 2023-12-07T10:06:57.961 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-12-07T10:06:57.961 INFO:teuthology.orchestra.run.smithi153.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-12-07T10:06:57.961 INFO:teuthology.orchestra.run.smithi153.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-12-07T10:06:57.961 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 31 k 2023-12-07T10:06:57.961 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2 x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 838 k 2023-12-07T10:06:57.961 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 139 k 2023-12-07T10:06:57.961 INFO:teuthology.orchestra.run.smithi153.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-12-07T10:06:57.961 INFO:teuthology.orchestra.run.smithi153.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-12-07T10:06:57.961 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-12-07T10:06:57.961 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-12-07T10:06:57.962 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-12-07T10:06:57.962 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-12-07T10:06:57.962 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cephfs x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 197 k 2023-12-07T10:06:57.962 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rados x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 371 k 2023-12-07T10:06:57.962 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rbd x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 364 k 2023-12-07T10:06:57.962 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rgw x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 106 k 2023-12-07T10:06:57.962 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 93 k 2023-12-07T10:06:57.962 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-mirror x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 3.6 M 2023-12-07T10:06:57.962 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-nbd x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 181 k 2023-12-07T10:06:57.962 INFO:teuthology.orchestra.run.smithi153.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-12-07T10:06:57.962 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrading: 2023-12-07T10:06:57.962 INFO:teuthology.orchestra.run.smithi153.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-12-07T10:06:57.962 INFO:teuthology.orchestra.run.smithi153.stdout: librados2 x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 3.8 M 2023-12-07T10:06:57.963 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1 x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 3.7 M 2023-12-07T10:06:57.963 INFO:teuthology.orchestra.run.smithi153.stdout:Installing dependencies: 2023-12-07T10:06:57.963 INFO:teuthology.orchestra.run.smithi153.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-12-07T10:06:57.963 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 23 M 2023-12-07T10:06:57.963 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 28 k 2023-12-07T10:06:57.963 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 2.3 M 2023-12-07T10:06:57.963 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-modules-core noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 243 k 2023-12-07T10:06:57.963 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 4.6 M 2023-12-07T10:06:57.963 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 18 M 2023-12-07T10:06:57.963 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-prometheus-alerts noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 15 k 2023-12-07T10:06:57.963 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 24 k 2023-12-07T10:06:57.964 INFO:teuthology.orchestra.run.smithi153.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-12-07T10:06:57.964 INFO:teuthology.orchestra.run.smithi153.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-07T10:06:57.964 INFO:teuthology.orchestra.run.smithi153.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-12-07T10:06:57.964 INFO:teuthology.orchestra.run.smithi153.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-12-07T10:06:57.964 INFO:teuthology.orchestra.run.smithi153.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-12-07T10:06:57.964 INFO:teuthology.orchestra.run.smithi153.stdout: libcephsqlite x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 183 k 2023-12-07T10:06:57.964 INFO:teuthology.orchestra.run.smithi153.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-12-07T10:06:57.964 INFO:teuthology.orchestra.run.smithi153.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-12-07T10:06:57.964 INFO:teuthology.orchestra.run.smithi153.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-07T10:06:57.964 INFO:teuthology.orchestra.run.smithi153.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-12-07T10:06:57.964 INFO:teuthology.orchestra.run.smithi153.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-12-07T10:06:57.964 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-12-07T10:06:57.964 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1 x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 518 k 2023-12-07T10:06:57.965 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-12-07T10:06:57.965 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2 x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 6.9 M 2023-12-07T10:06:57.965 INFO:teuthology.orchestra.run.smithi153.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-12-07T10:06:57.965 INFO:teuthology.orchestra.run.smithi153.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-12-07T10:06:57.965 INFO:teuthology.orchestra.run.smithi153.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-07T10:06:57.965 INFO:teuthology.orchestra.run.smithi153.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-12-07T10:06:57.965 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-12-07T10:06:57.965 INFO:teuthology.orchestra.run.smithi153.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-12-07T10:06:57.965 INFO:teuthology.orchestra.run.smithi153.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-12-07T10:06:57.965 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-12-07T10:06:57.965 INFO:teuthology.orchestra.run.smithi153.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-12-07T10:06:57.965 INFO:teuthology.orchestra.run.smithi153.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-12-07T10:06:57.965 INFO:teuthology.orchestra.run.smithi153.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-07T10:06:57.965 INFO:teuthology.orchestra.run.smithi153.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-12-07T10:06:57.966 INFO:teuthology.orchestra.run.smithi153.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-12-07T10:06:57.966 INFO:teuthology.orchestra.run.smithi153.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-07T10:06:57.966 INFO:teuthology.orchestra.run.smithi153.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-07T10:06:57.966 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-07T10:06:57.966 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-argparse x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 45 k 2023-12-07T10:06:57.966 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-common x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 128 k 2023-12-07T10:06:57.966 INFO:teuthology.orchestra.run.smithi153.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-07T10:06:57.966 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-12-07T10:06:57.966 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-07T10:06:57.966 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-07T10:06:57.966 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-12-07T10:06:57.966 INFO:teuthology.orchestra.run.smithi153.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-07T10:06:57.966 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-07T10:06:57.966 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-07T10:06:57.967 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-12-07T10:06:57.967 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-12-07T10:06:57.967 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-12-07T10:06:57.967 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-07T10:06:57.967 INFO:teuthology.orchestra.run.smithi153.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-07T10:06:57.967 INFO:teuthology.orchestra.run.smithi153.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-07T10:06:57.967 INFO:teuthology.orchestra.run.smithi153.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-12-07T10:06:57.967 INFO:teuthology.orchestra.run.smithi153.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-07T10:06:57.967 INFO:teuthology.orchestra.run.smithi153.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-07T10:06:57.967 INFO:teuthology.orchestra.run.smithi153.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-12-07T10:06:57.967 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-07T10:06:57.967 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-12-07T10:06:57.967 INFO:teuthology.orchestra.run.smithi153.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-12-07T10:06:57.968 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-07T10:06:57.968 INFO:teuthology.orchestra.run.smithi153.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-07T10:06:57.968 INFO:teuthology.orchestra.run.smithi153.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-12-07T10:06:57.968 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-12-07T10:06:57.968 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-12-07T10:06:57.968 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-12-07T10:06:57.968 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-12-07T10:06:57.968 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-12-07T10:06:57.968 INFO:teuthology.orchestra.run.smithi153.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-07T10:06:57.968 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-07T10:06:57.968 INFO:teuthology.orchestra.run.smithi153.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-07T10:06:57.968 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-07T10:06:57.968 INFO:teuthology.orchestra.run.smithi153.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-12-07T10:06:57.968 INFO:teuthology.orchestra.run.smithi153.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-07T10:06:57.969 INFO:teuthology.orchestra.run.smithi153.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-07T10:06:57.969 INFO:teuthology.orchestra.run.smithi153.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-07T10:06:57.969 INFO:teuthology.orchestra.run.smithi153.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-07T10:06:57.969 INFO:teuthology.orchestra.run.smithi153.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-07T10:06:57.969 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-07T10:06:57.969 INFO:teuthology.orchestra.run.smithi153.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-07T10:06:57.969 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-07T10:06:57.969 INFO:teuthology.orchestra.run.smithi153.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-12-07T10:06:57.969 INFO:teuthology.orchestra.run.smithi153.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-07T10:06:57.969 INFO:teuthology.orchestra.run.smithi153.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-12-07T10:06:57.969 INFO:teuthology.orchestra.run.smithi153.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-12-07T10:06:57.969 INFO:teuthology.orchestra.run.smithi153.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-12-07T10:06:57.969 INFO:teuthology.orchestra.run.smithi153.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-12-07T10:06:57.969 INFO:teuthology.orchestra.run.smithi153.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-12-07T10:06:57.970 INFO:teuthology.orchestra.run.smithi153.stdout:Enabling module streams: 2023-12-07T10:06:57.970 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql 10 2023-12-07T10:06:57.970 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-12-07T10:06:57.970 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2023-12-07T10:06:57.970 INFO:teuthology.orchestra.run.smithi153.stdout:=========================================================================================================================== 2023-12-07T10:06:57.970 INFO:teuthology.orchestra.run.smithi153.stdout:Install 128 Packages 2023-12-07T10:06:57.970 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrade 3 Packages 2023-12-07T10:06:57.970 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-12-07T10:06:57.971 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 209 M 2023-12-07T10:06:57.971 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2023-12-07T10:06:58.152 INFO:teuthology.orchestra.run.smithi159.stdout:(5/131): ceph-mgr-19.0.0-18.g3bd0bfa4.el8.x86_6 7.8 MB/s | 1.6 MB 00:00 2023-12-07T10:06:58.204 INFO:teuthology.orchestra.run.smithi153.stdout:(1/131): ceph-19.0.0-18.g3bd0bfa4.el8.x86_64.rp 31 kB/s | 6.4 kB 00:00 2023-12-07T10:06:58.452 INFO:teuthology.orchestra.run.smithi159.stdout:(6/131): ceph-mds-19.0.0-18.g3bd0bfa4.el8.x86_6 3.8 MB/s | 2.3 MB 00:00 2023-12-07T10:06:58.702 INFO:teuthology.orchestra.run.smithi159.stdout:(7/131): ceph-mon-19.0.0-18.g3bd0bfa4.el8.x86_6 8.4 MB/s | 4.6 MB 00:00 2023-12-07T10:06:59.088 INFO:teuthology.orchestra.run.smithi153.stdout:(2/131): ceph-fuse-19.0.0-18.g3bd0bfa4.el8.x86_ 1.0 MB/s | 920 kB 00:00 2023-12-07T10:06:59.155 INFO:teuthology.orchestra.run.smithi153.stdout:(3/131): ceph-immutable-object-cache-19.0.0-18. 2.5 MB/s | 171 kB 00:00 2023-12-07T10:06:59.203 INFO:teuthology.orchestra.run.smithi159.stdout:(8/131): ceph-common-19.0.0-18.g3bd0bfa4.el8.x8 12 MB/s | 23 MB 00:01 2023-12-07T10:06:59.270 INFO:teuthology.orchestra.run.smithi159.stdout:(9/131): ceph-selinux-19.0.0-18.g3bd0bfa4.el8.x 366 kB/s | 24 kB 00:00 2023-12-07T10:07:00.088 INFO:teuthology.orchestra.run.smithi153.stdout:(4/131): ceph-mds-19.0.0-18.g3bd0bfa4.el8.x86_6 2.5 MB/s | 2.3 MB 00:00 2023-12-07T10:07:00.153 INFO:teuthology.orchestra.run.smithi159.stdout:(10/131): ceph-radosgw-19.0.0-18.g3bd0bfa4.el8. 9.5 MB/s | 14 MB 00:01 2023-12-07T10:07:00.228 INFO:teuthology.orchestra.run.smithi159.stdout:(11/131): cephfs-mirror-19.0.0-18.g3bd0bfa4.el8 3.4 MB/s | 263 kB 00:00 2023-12-07T10:07:00.279 INFO:teuthology.orchestra.run.smithi159.stdout:(12/131): libcephfs-devel-19.0.0-18.g3bd0bfa4.e 621 kB/s | 31 kB 00:00 2023-12-07T10:07:00.420 INFO:teuthology.orchestra.run.smithi159.stdout:(13/131): libcephfs2-19.0.0-18.g3bd0bfa4.el8.x8 5.8 MB/s | 838 kB 00:00 2023-12-07T10:07:00.496 INFO:teuthology.orchestra.run.smithi159.stdout:(14/131): libcephsqlite-19.0.0-18.g3bd0bfa4.el8 2.4 MB/s | 183 kB 00:00 2023-12-07T10:07:00.563 INFO:teuthology.orchestra.run.smithi159.stdout:(15/131): librados-devel-19.0.0-18.g3bd0bfa4.el 2.0 MB/s | 139 kB 00:00 2023-12-07T10:07:00.563 INFO:teuthology.orchestra.run.smithi153.stdout:(5/131): ceph-mgr-19.0.0-18.g3bd0bfa4.el8.x86_6 3.4 MB/s | 1.6 MB 00:00 2023-12-07T10:07:00.655 INFO:teuthology.orchestra.run.smithi159.stdout:(16/131): libradosstriper1-19.0.0-18.g3bd0bfa4. 5.5 MB/s | 518 kB 00:00 2023-12-07T10:07:01.072 INFO:teuthology.orchestra.run.smithi153.stdout:(6/131): ceph-base-19.0.0-18.g3bd0bfa4.el8.x86_ 1.7 MB/s | 5.4 MB 00:03 2023-12-07T10:07:01.563 INFO:teuthology.orchestra.run.smithi159.stdout:(17/131): librgw2-19.0.0-18.g3bd0bfa4.el8.x86_6 7.6 MB/s | 6.9 MB 00:00 2023-12-07T10:07:01.647 INFO:teuthology.orchestra.run.smithi159.stdout:(18/131): python3-ceph-argparse-19.0.0-18.g3bd0 537 kB/s | 45 kB 00:00 2023-12-07T10:07:01.731 INFO:teuthology.orchestra.run.smithi159.stdout:(19/131): python3-ceph-common-19.0.0-18.g3bd0bf 1.5 MB/s | 128 kB 00:00 2023-12-07T10:07:01.814 INFO:teuthology.orchestra.run.smithi159.stdout:(20/131): python3-cephfs-19.0.0-18.g3bd0bfa4.el 2.3 MB/s | 197 kB 00:00 2023-12-07T10:07:01.847 INFO:teuthology.orchestra.run.smithi119.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-07T10:07:01.848 INFO:teuthology.orchestra.run.smithi119.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-07T10:07:01.863 INFO:teuthology.orchestra.run.smithi119.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-12-07T10:07:01.864 INFO:teuthology.orchestra.run.smithi119.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-12-07T10:07:01.866 INFO:teuthology.orchestra.run.smithi119.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-12-07T10:07:01.867 INFO:teuthology.orchestra.run.smithi119.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-12-07T10:07:01.869 INFO:teuthology.orchestra.run.smithi119.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-12-07T10:07:01.872 INFO:teuthology.orchestra.run.smithi119.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-12-07T10:07:01.874 INFO:teuthology.orchestra.run.smithi119.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-12-07T10:07:01.906 INFO:teuthology.orchestra.run.smithi159.stdout:(21/131): python3-rados-19.0.0-18.g3bd0bfa4.el8 3.9 MB/s | 371 kB 00:00 2023-12-07T10:07:02.007 INFO:teuthology.orchestra.run.smithi159.stdout:(22/131): python3-rbd-19.0.0-18.g3bd0bfa4.el8.x 3.6 MB/s | 364 kB 00:00 2023-12-07T10:07:02.011 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-12-07T10:07:02.024 INFO:teuthology.orchestra.run.smithi119.stdout:=========================================================================================================================== 2023-12-07T10:07:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2023-12-07T10:07:02.024 INFO:teuthology.orchestra.run.smithi119.stdout:=========================================================================================================================== 2023-12-07T10:07:02.024 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2023-12-07T10:07:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-12-07T10:07:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-12-07T10:07:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: ceph x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 6.4 k 2023-12-07T10:07:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 5.4 M 2023-12-07T10:07:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 920 k 2023-12-07T10:07:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 171 k 2023-12-07T10:07:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 1.6 M 2023-12-07T10:07:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 133 k 2023-12-07T10:07:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 1.8 M 2023-12-07T10:07:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 7.4 M 2023-12-07T10:07:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 50 k 2023-12-07T10:07:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 14 M 2023-12-07T10:07:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 49 M 2023-12-07T10:07:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 261 k 2023-12-07T10:07:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 414 k 2023-12-07T10:07:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-mirror x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 263 k 2023-12-07T10:07:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-top noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 18 k 2023-12-07T10:07:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-12-07T10:07:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-12-07T10:07:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-12-07T10:07:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 31 k 2023-12-07T10:07:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2 x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 838 k 2023-12-07T10:07:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 139 k 2023-12-07T10:07:02.025 INFO:teuthology.orchestra.run.smithi119.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-12-07T10:07:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-12-07T10:07:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-12-07T10:07:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-12-07T10:07:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-12-07T10:07:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-12-07T10:07:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 197 k 2023-12-07T10:07:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 371 k 2023-12-07T10:07:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 364 k 2023-12-07T10:07:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 106 k 2023-12-07T10:07:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 93 k 2023-12-07T10:07:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 3.6 M 2023-12-07T10:07:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 181 k 2023-12-07T10:07:02.026 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-12-07T10:07:02.027 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrading: 2023-12-07T10:07:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-12-07T10:07:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: librados2 x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 3.8 M 2023-12-07T10:07:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1 x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 3.7 M 2023-12-07T10:07:02.027 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2023-12-07T10:07:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-12-07T10:07:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 23 M 2023-12-07T10:07:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 28 k 2023-12-07T10:07:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 2.3 M 2023-12-07T10:07:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 243 k 2023-12-07T10:07:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 4.6 M 2023-12-07T10:07:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 18 M 2023-12-07T10:07:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts noarch 2:19.0.0-18.g3bd0bfa4.el8 ceph-noarch 15 k 2023-12-07T10:07:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 24 k 2023-12-07T10:07:02.027 INFO:teuthology.orchestra.run.smithi119.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-12-07T10:07:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-07T10:07:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-12-07T10:07:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-12-07T10:07:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-12-07T10:07:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 183 k 2023-12-07T10:07:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-12-07T10:07:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-12-07T10:07:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-07T10:07:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-12-07T10:07:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-12-07T10:07:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-12-07T10:07:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1 x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 518 k 2023-12-07T10:07:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-12-07T10:07:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2 x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 6.9 M 2023-12-07T10:07:02.028 INFO:teuthology.orchestra.run.smithi119.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-12-07T10:07:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-12-07T10:07:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-07T10:07:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-12-07T10:07:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-12-07T10:07:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-12-07T10:07:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-12-07T10:07:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-12-07T10:07:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-12-07T10:07:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-12-07T10:07:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-07T10:07:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-12-07T10:07:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-12-07T10:07:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-07T10:07:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-07T10:07:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-07T10:07:02.029 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 45 k 2023-12-07T10:07:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common x86_64 2:19.0.0-18.g3bd0bfa4.el8 ceph 128 k 2023-12-07T10:07:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-07T10:07:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-12-07T10:07:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-07T10:07:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-07T10:07:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-12-07T10:07:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-07T10:07:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-07T10:07:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-07T10:07:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-12-07T10:07:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-12-07T10:07:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-12-07T10:07:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-07T10:07:02.030 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-07T10:07:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-07T10:07:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-12-07T10:07:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-07T10:07:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-07T10:07:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-12-07T10:07:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-07T10:07:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-12-07T10:07:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-12-07T10:07:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-07T10:07:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-07T10:07:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-12-07T10:07:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-12-07T10:07:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-12-07T10:07:02.031 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-12-07T10:07:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-12-07T10:07:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-12-07T10:07:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-07T10:07:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-07T10:07:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-07T10:07:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-07T10:07:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-12-07T10:07:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-07T10:07:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-07T10:07:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-07T10:07:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-07T10:07:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-07T10:07:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-07T10:07:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-07T10:07:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-07T10:07:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-12-07T10:07:02.032 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-07T10:07:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-12-07T10:07:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-12-07T10:07:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-12-07T10:07:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-12-07T10:07:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-12-07T10:07:02.033 INFO:teuthology.orchestra.run.smithi119.stdout:Enabling module streams: 2023-12-07T10:07:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql 10 2023-12-07T10:07:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:07:02.033 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-12-07T10:07:02.034 INFO:teuthology.orchestra.run.smithi119.stdout:=========================================================================================================================== 2023-12-07T10:07:02.034 INFO:teuthology.orchestra.run.smithi119.stdout:Install 128 Packages 2023-12-07T10:07:02.034 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrade 3 Packages 2023-12-07T10:07:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:07:02.034 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 209 M 2023-12-07T10:07:02.034 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-12-07T10:07:02.090 INFO:teuthology.orchestra.run.smithi159.stdout:(23/131): python3-rgw-19.0.0-18.g3bd0bfa4.el8.x 1.3 MB/s | 106 kB 00:00 2023-12-07T10:07:02.174 INFO:teuthology.orchestra.run.smithi159.stdout:(24/131): rbd-fuse-19.0.0-18.g3bd0bfa4.el8.x86_ 1.1 MB/s | 93 kB 00:00 2023-12-07T10:07:02.272 INFO:teuthology.orchestra.run.smithi153.stdout:(7/131): ceph-mon-19.0.0-18.g3bd0bfa4.el8.x86_6 2.7 MB/s | 4.6 MB 00:01 2023-12-07T10:07:02.538 INFO:teuthology.orchestra.run.smithi119.stdout:(1/131): ceph-19.0.0-18.g3bd0bfa4.el8.x86_64.rp 18 kB/s | 6.4 kB 00:00 2023-12-07T10:07:02.807 INFO:teuthology.orchestra.run.smithi159.stdout:(25/131): rbd-mirror-19.0.0-18.g3bd0bfa4.el8.x8 5.7 MB/s | 3.6 MB 00:00 2023-12-07T10:07:03.013 INFO:teuthology.orchestra.run.smithi119.stdout:(2/131): ceph-fuse-19.0.0-18.g3bd0bfa4.el8.x86_ 1.9 MB/s | 920 kB 00:00 2023-12-07T10:07:03.105 INFO:teuthology.orchestra.run.smithi119.stdout:(3/131): ceph-immutable-object-cache-19.0.0-18. 1.8 MB/s | 171 kB 00:00 2023-12-07T10:07:03.258 INFO:teuthology.orchestra.run.smithi159.stdout:(26/131): rbd-nbd-19.0.0-18.g3bd0bfa4.el8.x86_6 402 kB/s | 181 kB 00:00 2023-12-07T10:07:03.375 INFO:teuthology.orchestra.run.smithi159.stdout:(27/131): ceph-grafana-dashboards-19.0.0-18.g3b 240 kB/s | 28 kB 00:00 2023-12-07T10:07:03.475 INFO:teuthology.orchestra.run.smithi159.stdout:(28/131): ceph-mgr-cephadm-19.0.0-18.g3bd0bfa4. 1.3 MB/s | 133 kB 00:00 2023-12-07T10:07:03.692 INFO:teuthology.orchestra.run.smithi119.stdout:(4/131): ceph-mds-19.0.0-18.g3bd0bfa4.el8.x86_6 3.9 MB/s | 2.3 MB 00:00 2023-12-07T10:07:03.900 INFO:teuthology.orchestra.run.smithi159.stdout:(29/131): ceph-mgr-dashboard-19.0.0-18.g3bd0bfa 4.2 MB/s | 1.8 MB 00:00 2023-12-07T10:07:04.098 INFO:teuthology.orchestra.run.smithi119.stdout:(5/131): ceph-mgr-19.0.0-18.g3bd0bfa4.el8.x86_6 4.0 MB/s | 1.6 MB 00:00 2023-12-07T10:07:04.309 INFO:teuthology.orchestra.run.smithi159.stdout:(30/131): ceph-osd-19.0.0-18.g3bd0bfa4.el8.x86_ 3.1 MB/s | 18 MB 00:05 2023-12-07T10:07:04.476 INFO:teuthology.orchestra.run.smithi159.stdout:(31/131): ceph-mgr-modules-core-19.0.0-18.g3bd0 1.4 MB/s | 243 kB 00:00 2023-12-07T10:07:04.540 INFO:teuthology.orchestra.run.smithi119.stdout:(6/131): ceph-base-19.0.0-18.g3bd0bfa4.el8.x86_ 2.3 MB/s | 5.4 MB 00:02 2023-12-07T10:07:04.580 INFO:teuthology.orchestra.run.smithi159.stdout:(32/131): ceph-mgr-rook-19.0.0-18.g3bd0bfa4.el8 482 kB/s | 50 kB 00:00 2023-12-07T10:07:04.950 INFO:teuthology.orchestra.run.smithi153.stdout:(8/131): ceph-osd-19.0.0-18.g3bd0bfa4.el8.x86_6 4.6 MB/s | 18 MB 00:03 2023-12-07T10:07:04.956 INFO:teuthology.orchestra.run.smithi119.stdout:(7/131): ceph-mon-19.0.0-18.g3bd0bfa4.el8.x86_6 5.4 MB/s | 4.6 MB 00:00 2023-12-07T10:07:05.014 INFO:teuthology.orchestra.run.smithi153.stdout:(9/131): ceph-selinux-19.0.0-18.g3bd0bfa4.el8.x 380 kB/s | 24 kB 00:00 2023-12-07T10:07:05.052 INFO:teuthology.orchestra.run.smithi159.stdout:(33/131): ceph-test-19.0.0-18.g3bd0bfa4.el8.x86 8.5 MB/s | 49 MB 00:05 2023-12-07T10:07:05.077 INFO:teuthology.orchestra.run.smithi159.stdout:(34/131): ceph-prometheus-alerts-19.0.0-18.g3bd 30 kB/s | 15 kB 00:00 2023-12-07T10:07:05.244 INFO:teuthology.orchestra.run.smithi159.stdout:(35/131): cephadm-19.0.0-18.g3bd0bfa4.el8.noarc 2.4 MB/s | 414 kB 00:00 2023-12-07T10:07:05.277 INFO:teuthology.orchestra.run.smithi159.stdout:(36/131): ceph-volume-19.0.0-18.g3bd0bfa4.el8.n 1.1 MB/s | 261 kB 00:00 2023-12-07T10:07:05.328 INFO:teuthology.orchestra.run.smithi159.stdout:(37/131): cephfs-top-19.0.0-18.g3bd0bfa4.el8.no 217 kB/s | 18 kB 00:00 2023-12-07T10:07:05.620 INFO:teuthology.orchestra.run.smithi159.stdout:(38/131): ceph-mgr-diskprediction-local-19.0.0- 4.3 MB/s | 7.4 MB 00:01 2023-12-07T10:07:05.645 INFO:teuthology.orchestra.run.smithi159.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 188 kB/s | 59 kB 00:00 2023-12-07T10:07:05.737 INFO:teuthology.orchestra.run.smithi159.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 667 kB/s | 306 kB 00:00 2023-12-07T10:07:05.829 INFO:teuthology.orchestra.run.smithi159.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 479 kB/s | 44 kB 00:00 2023-12-07T10:07:05.913 INFO:teuthology.orchestra.run.smithi159.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.7 MB/s | 476 kB 00:00 2023-12-07T10:07:05.930 INFO:teuthology.orchestra.run.smithi159.stdout:(43/131): libunwind-1.3.1-3.el8.x86_64.rpm 242 kB/s | 75 kB 00:00 2023-12-07T10:07:05.955 INFO:teuthology.orchestra.run.smithi159.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.4 MB/s | 185 kB 00:00 2023-12-07T10:07:05.982 INFO:teuthology.orchestra.run.smithi159.stdout:(45/131): python3-cachetools-3.1.1-4.el8.noarch 490 kB/s | 33 kB 00:00 2023-12-07T10:07:06.006 INFO:teuthology.orchestra.run.smithi159.stdout:(46/131): python3-certifi-2018.10.15-7.el8.noar 216 kB/s | 16 kB 00:00 2023-12-07T10:07:06.056 INFO:teuthology.orchestra.run.smithi159.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.7 MB/s | 173 kB 00:00 2023-12-07T10:07:06.090 INFO:teuthology.orchestra.run.smithi159.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.5 MB/s | 384 kB 00:00 2023-12-07T10:07:06.106 INFO:teuthology.orchestra.run.smithi159.stdout:(49/131): python3-google-auth-1.1.1-10.el8.noar 809 kB/s | 81 kB 00:00 2023-12-07T10:07:06.124 INFO:teuthology.orchestra.run.smithi159.stdout:(50/131): python3-jaraco-6.2-6.el8.noarch.rpm 165 kB/s | 11 kB 00:00 2023-12-07T10:07:06.157 INFO:teuthology.orchestra.run.smithi159.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 263 kB/s | 18 kB 00:00 2023-12-07T10:07:06.208 INFO:teuthology.orchestra.run.smithi159.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 583 kB/s | 49 kB 00:00 2023-12-07T10:07:06.224 INFO:teuthology.orchestra.run.smithi159.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 877 kB/s | 59 kB 00:00 2023-12-07T10:07:06.308 INFO:teuthology.orchestra.run.smithi159.stdout:(54/131): python3-kubernetes-11.0.0-6.el8.noarc 5.8 MB/s | 1.2 MB 00:00 2023-12-07T10:07:06.325 INFO:teuthology.orchestra.run.smithi159.stdout:(55/131): python3-natsort-7.1.1-2.el8.noarch.rp 515 kB/s | 60 kB 00:00 2023-12-07T10:07:06.350 INFO:teuthology.orchestra.run.smithi159.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.2 MB/s | 283 kB 00:00 2023-12-07T10:07:06.384 INFO:teuthology.orchestra.run.smithi159.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 206 kB/s | 16 kB 00:00 2023-12-07T10:07:06.409 INFO:teuthology.orchestra.run.smithi159.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 401 kB/s | 34 kB 00:00 2023-12-07T10:07:06.426 INFO:teuthology.orchestra.run.smithi159.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 2.5 MB/s | 196 kB 00:00 2023-12-07T10:07:06.451 INFO:teuthology.orchestra.run.smithi159.stdout:(60/131): python3-rsa-4.9-2.el8.noarch.rpm 956 kB/s | 64 kB 00:00 2023-12-07T10:07:06.485 INFO:teuthology.orchestra.run.smithi159.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 247 kB/s | 19 kB 00:00 2023-12-07T10:07:06.510 INFO:teuthology.orchestra.run.smithi159.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 285 kB/s | 24 kB 00:00 2023-12-07T10:07:06.535 INFO:teuthology.orchestra.run.smithi159.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 348 kB/s | 29 kB 00:00 2023-12-07T10:07:06.569 INFO:teuthology.orchestra.run.smithi159.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 329 kB/s | 27 kB 00:00 2023-12-07T10:07:06.638 INFO:teuthology.orchestra.run.smithi159.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 2.0 MB/s | 255 kB 00:00 2023-12-07T10:07:06.811 INFO:teuthology.orchestra.run.smithi159.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 253 kB/s | 61 kB 00:00 2023-12-07T10:07:06.907 INFO:teuthology.orchestra.run.smithi119.stdout:(8/131): ceph-radosgw-19.0.0-18.g3bd0bfa4.el8.x 7.1 MB/s | 14 MB 00:01 2023-12-07T10:07:06.963 INFO:teuthology.orchestra.run.smithi159.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 263 kB/s | 85 kB 00:00 2023-12-07T10:07:06.964 INFO:teuthology.orchestra.run.smithi153.stdout:(10/131): ceph-radosgw-19.0.0-18.g3bd0bfa4.el8. 3.0 MB/s | 14 MB 00:04 2023-12-07T10:07:06.965 INFO:teuthology.orchestra.run.smithi119.stdout:(9/131): ceph-selinux-19.0.0-18.g3bd0bfa4.el8.x 418 kB/s | 24 kB 00:00 2023-12-07T10:07:06.987 INFO:teuthology.orchestra.run.smithi159.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 131 kB/s | 23 kB 00:00 2023-12-07T10:07:07.037 INFO:teuthology.orchestra.run.smithi159.stdout:(69/131): python3-webob-1.8.5-1.el8.1.noarch.rp 501 kB/s | 251 kB 00:00 2023-12-07T10:07:07.057 INFO:teuthology.orchestra.run.smithi153.stdout:(11/131): cephfs-mirror-19.0.0-18.g3bd0bfa4.el8 2.8 MB/s | 263 kB 00:00 2023-12-07T10:07:07.063 INFO:teuthology.orchestra.run.smithi159.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 922 kB/s | 70 kB 00:00 2023-12-07T10:07:07.113 INFO:teuthology.orchestra.run.smithi159.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 622 kB/s | 47 kB 00:00 2023-12-07T10:07:07.115 INFO:teuthology.orchestra.run.smithi153.stdout:(12/131): libcephfs-devel-19.0.0-18.g3bd0bfa4.e 533 kB/s | 31 kB 00:00 2023-12-07T10:07:07.139 INFO:teuthology.orchestra.run.smithi159.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.8 MB/s | 200 kB 00:00 2023-12-07T10:07:07.164 INFO:teuthology.orchestra.run.smithi159.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 848 kB/s | 43 kB 00:00 2023-12-07T10:07:07.197 INFO:teuthology.orchestra.run.smithi159.stdout:(74/131): thrift-0.13.0-2.el8.x86_64.rpm 7.4 MB/s | 1.7 MB 00:00 2023-12-07T10:07:07.223 INFO:teuthology.orchestra.run.smithi159.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 823 kB/s | 48 kB 00:00 2023-12-07T10:07:07.224 INFO:teuthology.orchestra.run.smithi119.stdout:(10/131): ceph-common-19.0.0-18.g3bd0bfa4.el8.x 4.5 MB/s | 23 MB 00:05 2023-12-07T10:07:07.248 INFO:teuthology.orchestra.run.smithi159.stdout:(76/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.0 MB/s | 220 kB 00:00 2023-12-07T10:07:07.273 INFO:teuthology.orchestra.run.smithi159.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 954 kB/s | 43 kB 00:00 2023-12-07T10:07:07.274 INFO:teuthology.orchestra.run.smithi153.stdout:(13/131): libcephfs2-19.0.0-18.g3bd0bfa4.el8.x8 5.2 MB/s | 838 kB 00:00 2023-12-07T10:07:07.290 INFO:teuthology.orchestra.run.smithi159.stdout:(78/131): libsepol-devel-2.9-3.el8.x86_64.rpm 942 kB/s | 87 kB 00:00 2023-12-07T10:07:07.315 INFO:teuthology.orchestra.run.smithi159.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 1.6 MB/s | 109 kB 00:00 2023-12-07T10:07:07.316 INFO:teuthology.orchestra.run.smithi119.stdout:(11/131): cephfs-mirror-19.0.0-18.g3bd0bfa4.el8 2.8 MB/s | 263 kB 00:00 2023-12-07T10:07:07.332 INFO:teuthology.orchestra.run.smithi153.stdout:(14/131): libcephsqlite-19.0.0-18.g3bd0bfa4.el8 3.1 MB/s | 183 kB 00:00 2023-12-07T10:07:07.341 INFO:teuthology.orchestra.run.smithi159.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.3 MB/s | 229 kB 00:00 2023-12-07T10:07:07.366 INFO:teuthology.orchestra.run.smithi159.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 373 kB/s | 19 kB 00:00 2023-12-07T10:07:07.374 INFO:teuthology.orchestra.run.smithi119.stdout:(12/131): libcephfs-devel-19.0.0-18.g3bd0bfa4.e 533 kB/s | 31 kB 00:00 2023-12-07T10:07:07.382 INFO:teuthology.orchestra.run.smithi153.stdout:(15/131): librados-devel-19.0.0-18.g3bd0bfa4.el 2.7 MB/s | 139 kB 00:00 2023-12-07T10:07:07.391 INFO:teuthology.orchestra.run.smithi159.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-12-07T10:07:07.425 INFO:teuthology.orchestra.run.smithi159.stdout:(83/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.1 MB/s | 560 kB 00:00 2023-12-07T10:07:07.450 INFO:teuthology.orchestra.run.smithi159.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 305 kB/s | 18 kB 00:00 2023-12-07T10:07:07.492 INFO:teuthology.orchestra.run.smithi159.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 4.3 MB/s | 559 kB 00:00 2023-12-07T10:07:07.517 INFO:teuthology.orchestra.run.smithi159.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-12-07T10:07:07.543 INFO:teuthology.orchestra.run.smithi159.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.0 MB/s | 605 kB 00:00 2023-12-07T10:07:07.566 INFO:teuthology.orchestra.run.smithi153.stdout:(16/131): libradosstriper1-19.0.0-18.g3bd0bfa4. 2.8 MB/s | 518 kB 00:00 2023-12-07T10:07:07.568 INFO:teuthology.orchestra.run.smithi159.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-12-07T10:07:07.586 INFO:teuthology.orchestra.run.smithi119.stdout:(13/131): libcephfs2-19.0.0-18.g3bd0bfa4.el8.x8 3.9 MB/s | 838 kB 00:00 2023-12-07T10:07:07.593 INFO:teuthology.orchestra.run.smithi159.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-12-07T10:07:07.618 INFO:teuthology.orchestra.run.smithi159.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-12-07T10:07:07.644 INFO:teuthology.orchestra.run.smithi159.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-12-07T10:07:07.704 INFO:teuthology.orchestra.run.smithi119.stdout:(14/131): libcephsqlite-19.0.0-18.g3bd0bfa4.el8 2.2 MB/s | 183 kB 00:00 2023-12-07T10:07:07.705 INFO:teuthology.orchestra.run.smithi159.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-12-07T10:07:07.705 INFO:teuthology.orchestra.run.smithi159.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 768 kB/s | 39 kB 00:00 2023-12-07T10:07:07.728 INFO:teuthology.orchestra.run.smithi159.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-12-07T10:07:07.734 INFO:teuthology.orchestra.run.smithi119.stdout:(15/131): librados-devel-19.0.0-18.g3bd0bfa4.el 2.0 MB/s | 139 kB 00:00 2023-12-07T10:07:07.745 INFO:teuthology.orchestra.run.smithi159.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 3.3 MB/s | 171 kB 00:00 2023-12-07T10:07:07.770 INFO:teuthology.orchestra.run.smithi159.stdout:(96/131): NetworkManager-initscripts-updown-1.3 3.2 MB/s | 138 kB 00:00 2023-12-07T10:07:07.804 INFO:teuthology.orchestra.run.smithi159.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 3.2 MB/s | 110 kB 00:00 2023-12-07T10:07:07.854 INFO:teuthology.orchestra.run.smithi159.stdout:(98/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 6.4 MB/s | 2.3 MB 00:00 2023-12-07T10:07:07.876 INFO:teuthology.orchestra.run.smithi119.stdout:(16/131): libradosstriper1-19.0.0-18.g3bd0bfa4. 3.6 MB/s | 518 kB 00:00 2023-12-07T10:07:07.879 INFO:teuthology.orchestra.run.smithi159.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 718 kB/s | 54 kB 00:00 2023-12-07T10:07:07.905 INFO:teuthology.orchestra.run.smithi159.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.0 MB/s | 259 kB 00:00 2023-12-07T10:07:07.939 INFO:teuthology.orchestra.run.smithi159.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-12-07T10:07:07.964 INFO:teuthology.orchestra.run.smithi159.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 662 kB/s | 39 kB 00:00 2023-12-07T10:07:07.998 INFO:teuthology.orchestra.run.smithi159.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 11 MB/s | 688 kB 00:00 2023-12-07T10:07:08.040 INFO:teuthology.orchestra.run.smithi159.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.0 MB/s | 538 kB 00:00 2023-12-07T10:07:08.065 INFO:teuthology.orchestra.run.smithi159.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-12-07T10:07:08.090 INFO:teuthology.orchestra.run.smithi159.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-12-07T10:07:08.132 INFO:teuthology.orchestra.run.smithi159.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-12-07T10:07:08.157 INFO:teuthology.orchestra.run.smithi159.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 1.0 MB/s | 27 kB 00:00 2023-12-07T10:07:08.199 INFO:teuthology.orchestra.run.smithi159.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-12-07T10:07:08.258 INFO:teuthology.orchestra.run.smithi159.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 756 kB/s | 44 kB 00:00 2023-12-07T10:07:08.350 INFO:teuthology.orchestra.run.smithi159.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2023-12-07T10:07:08.392 INFO:teuthology.orchestra.run.smithi159.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 11 MB/s | 457 kB 00:00 2023-12-07T10:07:08.426 INFO:teuthology.orchestra.run.smithi159.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 6.6 MB/s | 225 kB 00:00 2023-12-07T10:07:08.476 INFO:teuthology.orchestra.run.smithi159.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 17 MB/s | 3.7 MB 00:00 2023-12-07T10:07:08.501 INFO:teuthology.orchestra.run.smithi159.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 849 kB/s | 64 kB 00:00 2023-12-07T10:07:08.526 INFO:teuthology.orchestra.run.smithi159.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.4 MB/s | 126 kB 00:00 2023-12-07T10:07:08.543 INFO:teuthology.orchestra.run.smithi159.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 477 kB/s | 20 kB 00:00 2023-12-07T10:07:08.568 INFO:teuthology.orchestra.run.smithi159.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2023-12-07T10:07:08.602 INFO:teuthology.orchestra.run.smithi159.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-12-07T10:07:08.644 INFO:teuthology.orchestra.run.smithi159.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-12-07T10:07:08.786 INFO:teuthology.orchestra.run.smithi159.stdout:(121/131): python3-scipy-1.0.0-21.module+el8.5. 13 MB/s | 14 MB 00:01 2023-12-07T10:07:08.828 INFO:teuthology.orchestra.run.smithi159.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-12-07T10:07:08.870 INFO:teuthology.orchestra.run.smithi159.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-12-07T10:07:08.954 INFO:teuthology.orchestra.run.smithi159.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 13 MB/s | 4.9 MB 00:00 2023-12-07T10:07:09.029 INFO:teuthology.orchestra.run.smithi159.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 12 MB/s | 4.7 MB 00:00 2023-12-07T10:07:09.042 INFO:teuthology.orchestra.run.smithi119.stdout:(17/131): ceph-osd-19.0.0-18.g3bd0bfa4.el8.x86_ 4.0 MB/s | 18 MB 00:04 2023-12-07T10:07:09.063 INFO:teuthology.orchestra.run.smithi159.stdout:(126/131): postgresql-contrib-10.21-2.module+el 7.3 MB/s | 808 kB 00:00 2023-12-07T10:07:09.121 INFO:teuthology.orchestra.run.smithi159.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 16 MB/s | 1.5 MB 00:00 2023-12-07T10:07:09.201 INFO:teuthology.orchestra.run.smithi119.stdout:(18/131): librgw2-19.0.0-18.g3bd0bfa4.el8.x86_6 5.2 MB/s | 6.9 MB 00:01 2023-12-07T10:07:09.213 INFO:teuthology.orchestra.run.smithi159.stdout:(128/131): postgresql-server-10.21-2.module+el8 15 MB/s | 5.1 MB 00:00 2023-12-07T10:07:09.226 INFO:teuthology.orchestra.run.smithi119.stdout:(19/131): python3-ceph-argparse-19.0.0-18.g3bd0 243 kB/s | 45 kB 00:00 2023-12-07T10:07:09.251 INFO:teuthology.orchestra.run.smithi119.stdout:(20/131): python3-ceph-common-19.0.0-18.g3bd0bf 2.5 MB/s | 128 kB 00:00 2023-12-07T10:07:09.263 INFO:teuthology.orchestra.run.smithi159.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-12-07T10:07:09.291 INFO:teuthology.orchestra.run.smithi153.stdout:(17/131): librgw2-19.0.0-18.g3bd0bfa4.el8.x86_6 4.0 MB/s | 6.9 MB 00:01 2023-12-07T10:07:09.311 INFO:teuthology.orchestra.run.smithi119.stdout:(21/131): python3-cephfs-19.0.0-18.g3bd0bfa4.el 2.3 MB/s | 197 kB 00:00 2023-12-07T10:07:09.344 INFO:teuthology.orchestra.run.smithi119.stdout:(22/131): python3-rados-19.0.0-18.g3bd0bfa4.el8 3.9 MB/s | 371 kB 00:00 2023-12-07T10:07:09.358 INFO:teuthology.orchestra.run.smithi153.stdout:(18/131): python3-ceph-argparse-19.0.0-18.g3bd0 672 kB/s | 45 kB 00:00 2023-12-07T10:07:09.412 INFO:teuthology.orchestra.run.smithi119.stdout:(23/131): python3-rgw-19.0.0-18.g3bd0bfa4.el8.x 1.6 MB/s | 106 kB 00:00 2023-12-07T10:07:09.458 INFO:teuthology.orchestra.run.smithi153.stdout:(19/131): python3-ceph-common-19.0.0-18.g3bd0bf 1.3 MB/s | 128 kB 00:00 2023-12-07T10:07:09.469 INFO:teuthology.orchestra.run.smithi119.stdout:(24/131): python3-rbd-19.0.0-18.g3bd0bfa4.el8.x 2.2 MB/s | 364 kB 00:00 2023-12-07T10:07:09.495 INFO:teuthology.orchestra.run.smithi119.stdout:(25/131): rbd-fuse-19.0.0-18.g3bd0bfa4.el8.x86_ 1.1 MB/s | 93 kB 00:00 2023-12-07T10:07:09.603 INFO:teuthology.orchestra.run.smithi119.stdout:(26/131): rbd-nbd-19.0.0-18.g3bd0bfa4.el8.x86_6 1.6 MB/s | 181 kB 00:00 2023-12-07T10:07:09.688 INFO:teuthology.orchestra.run.smithi119.stdout:(27/131): ceph-grafana-dashboards-19.0.0-18.g3b 335 kB/s | 28 kB 00:00 2023-12-07T10:07:09.734 INFO:teuthology.orchestra.run.smithi153.stdout:(20/131): python3-cephfs-19.0.0-18.g3bd0bfa4.el 715 kB/s | 197 kB 00:00 2023-12-07T10:07:09.779 INFO:teuthology.orchestra.run.smithi119.stdout:(28/131): ceph-mgr-cephadm-19.0.0-18.g3bd0bfa4. 1.4 MB/s | 133 kB 00:00 2023-12-07T10:07:09.959 INFO:teuthology.orchestra.run.smithi153.stdout:(21/131): python3-rados-19.0.0-18.g3bd0bfa4.el8 1.6 MB/s | 371 kB 00:00 2023-12-07T10:07:09.963 INFO:teuthology.orchestra.run.smithi159.stdout:(130/131): librbd1-19.0.0-18.g3bd0bfa4.el8.x86_ 4.4 MB/s | 3.7 MB 00:00 2023-12-07T10:07:10.101 INFO:teuthology.orchestra.run.smithi153.stdout:(22/131): python3-rbd-19.0.0-18.g3bd0bfa4.el8.x 2.5 MB/s | 364 kB 00:00 2023-12-07T10:07:10.176 INFO:teuthology.orchestra.run.smithi153.stdout:(23/131): python3-rgw-19.0.0-18.g3bd0bfa4.el8.x 1.4 MB/s | 106 kB 00:00 2023-12-07T10:07:10.230 INFO:teuthology.orchestra.run.smithi119.stdout:(29/131): ceph-mgr-dashboard-19.0.0-18.g3bd0bfa 3.9 MB/s | 1.8 MB 00:00 2023-12-07T10:07:10.251 INFO:teuthology.orchestra.run.smithi153.stdout:(24/131): rbd-fuse-19.0.0-18.g3bd0bfa4.el8.x86_ 1.2 MB/s | 93 kB 00:00 2023-12-07T10:07:10.498 INFO:teuthology.orchestra.run.smithi159.stdout:(131/131): librados2-19.0.0-18.g3bd0bfa4.el8.x8 2.6 MB/s | 3.8 MB 00:01 2023-12-07T10:07:10.500 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2023-12-07T10:07:10.500 INFO:teuthology.orchestra.run.smithi159.stdout:Total 16 MB/s | 209 MB 00:13 2023-12-07T10:07:10.597 INFO:teuthology.orchestra.run.smithi119.stdout:(30/131): rbd-mirror-19.0.0-18.g3bd0bfa4.el8.x8 3.2 MB/s | 3.6 MB 00:01 2023-12-07T10:07:10.694 INFO:teuthology.orchestra.run.smithi153.stdout:(25/131): ceph-common-19.0.0-18.g3bd0bfa4.el8.x 1.8 MB/s | 23 MB 00:12 2023-12-07T10:07:10.714 INFO:teuthology.orchestra.run.smithi119.stdout:(31/131): ceph-mgr-modules-core-19.0.0-18.g3bd0 2.0 MB/s | 243 kB 00:00 2023-12-07T10:07:10.819 INFO:teuthology.orchestra.run.smithi153.stdout:(26/131): rbd-nbd-19.0.0-18.g3bd0bfa4.el8.x86_6 1.4 MB/s | 181 kB 00:00 2023-12-07T10:07:10.886 INFO:teuthology.orchestra.run.smithi153.stdout:(27/131): ceph-grafana-dashboards-19.0.0-18.g3b 421 kB/s | 28 kB 00:00 2023-12-07T10:07:10.961 INFO:teuthology.orchestra.run.smithi153.stdout:(28/131): ceph-mgr-cephadm-19.0.0-18.g3bd0bfa4. 1.7 MB/s | 133 kB 00:00 2023-12-07T10:07:10.981 INFO:teuthology.orchestra.run.smithi119.stdout:(32/131): ceph-mgr-rook-19.0.0-18.g3bd0bfa4.el8 188 kB/s | 50 kB 00:00 2023-12-07T10:07:11.044 INFO:teuthology.orchestra.run.smithi153.stdout:(29/131): rbd-mirror-19.0.0-18.g3bd0bfa4.el8.x8 4.5 MB/s | 3.6 MB 00:00 2023-12-07T10:07:11.098 INFO:teuthology.orchestra.run.smithi119.stdout:(33/131): ceph-prometheus-alerts-19.0.0-18.g3bd 128 kB/s | 15 kB 00:00 2023-12-07T10:07:11.265 INFO:teuthology.orchestra.run.smithi119.stdout:(34/131): ceph-volume-19.0.0-18.g3bd0bfa4.el8.n 1.5 MB/s | 261 kB 00:00 2023-12-07T10:07:11.324 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2023-12-07T10:07:11.328 INFO:teuthology.orchestra.run.smithi153.stdout:(30/131): ceph-mgr-dashboard-19.0.0-18.g3bd0bfa 4.8 MB/s | 1.8 MB 00:00 2023-12-07T10:07:11.437 INFO:teuthology.orchestra.run.smithi153.stdout:(31/131): ceph-mgr-modules-core-19.0.0-18.g3bd0 2.2 MB/s | 243 kB 00:00 2023-12-07T10:07:11.444 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2023-12-07T10:07:11.444 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2023-12-07T10:07:11.451 INFO:teuthology.orchestra.run.smithi119.stdout:(35/131): cephadm-19.0.0-18.g3bd0bfa4.el8.noarc 2.2 MB/s | 414 kB 00:00 2023-12-07T10:07:11.495 INFO:teuthology.orchestra.run.smithi153.stdout:(32/131): ceph-mgr-rook-19.0.0-18.g3bd0bfa4.el8 861 kB/s | 50 kB 00:00 2023-12-07T10:07:11.541 INFO:teuthology.orchestra.run.smithi119.stdout:(36/131): cephfs-top-19.0.0-18.g3bd0bfa4.el8.no 203 kB/s | 18 kB 00:00 2023-12-07T10:07:11.579 INFO:teuthology.orchestra.run.smithi153.stdout:(33/131): ceph-prometheus-alerts-19.0.0-18.g3bd 180 kB/s | 15 kB 00:00 2023-12-07T10:07:11.662 INFO:teuthology.orchestra.run.smithi153.stdout:(34/131): ceph-volume-19.0.0-18.g3bd0bfa4.el8.n 3.1 MB/s | 261 kB 00:00 2023-12-07T10:07:11.682 INFO:teuthology.orchestra.run.smithi119.stdout:(37/131): ceph-mgr-diskprediction-local-19.0.0- 5.1 MB/s | 7.4 MB 00:01 2023-12-07T10:07:11.708 INFO:teuthology.orchestra.run.smithi119.stdout:(38/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.8 MB/s | 306 kB 00:00 2023-12-07T10:07:11.742 INFO:teuthology.orchestra.run.smithi119.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 2.2 MB/s | 75 kB 00:00 2023-12-07T10:07:11.767 INFO:teuthology.orchestra.run.smithi119.stdout:(40/131): liboath-2.6.2-3.el8.x86_64.rpm 701 kB/s | 59 kB 00:00 2023-12-07T10:07:11.780 INFO:teuthology.orchestra.run.smithi153.stdout:(35/131): cephadm-19.0.0-18.g3bd0bfa4.el8.noarc 3.5 MB/s | 414 kB 00:00 2023-12-07T10:07:11.800 INFO:teuthology.orchestra.run.smithi119.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 7.9 MB/s | 476 kB 00:00 2023-12-07T10:07:11.838 INFO:teuthology.orchestra.run.smithi153.stdout:(36/131): cephfs-top-19.0.0-18.g3bd0bfa4.el8.no 310 kB/s | 18 kB 00:00 2023-12-07T10:07:11.846 INFO:teuthology.orchestra.run.smithi119.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 555 kB/s | 44 kB 00:00 2023-12-07T10:07:11.876 INFO:teuthology.orchestra.run.smithi119.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.4 MB/s | 185 kB 00:00 2023-12-07T10:07:11.901 INFO:teuthology.orchestra.run.smithi119.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 599 kB/s | 33 kB 00:00 2023-12-07T10:07:11.926 INFO:teuthology.orchestra.run.smithi119.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 325 kB/s | 16 kB 00:00 2023-12-07T10:07:11.960 INFO:teuthology.orchestra.run.smithi119.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.9 MB/s | 173 kB 00:00 2023-12-07T10:07:11.994 INFO:teuthology.orchestra.run.smithi119.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 5.6 MB/s | 384 kB 00:00 2023-12-07T10:07:12.019 INFO:teuthology.orchestra.run.smithi119.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2023-12-07T10:07:12.030 INFO:teuthology.orchestra.run.smithi153.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.6 MB/s | 306 kB 00:00 2023-12-07T10:07:12.044 INFO:teuthology.orchestra.run.smithi119.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 220 kB/s | 11 kB 00:00 2023-12-07T10:07:12.061 INFO:teuthology.orchestra.run.smithi119.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 419 kB/s | 18 kB 00:00 2023-12-07T10:07:12.064 INFO:teuthology.orchestra.run.smithi153.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 1.7 MB/s | 59 kB 00:00 2023-12-07T10:07:12.087 INFO:teuthology.orchestra.run.smithi119.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 1.9 MB/s | 49 kB 00:00 2023-12-07T10:07:12.106 INFO:teuthology.orchestra.run.smithi153.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.8 MB/s | 75 kB 00:00 2023-12-07T10:07:12.129 INFO:teuthology.orchestra.run.smithi119.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 14 MB/s | 1.2 MB 00:00 2023-12-07T10:07:12.146 INFO:teuthology.orchestra.run.smithi119.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 1.0 MB/s | 59 kB 00:00 2023-12-07T10:07:12.162 INFO:teuthology.orchestra.run.smithi119.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.8 MB/s | 60 kB 00:00 2023-12-07T10:07:12.190 INFO:teuthology.orchestra.run.smithi119.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 6.3 MB/s | 283 kB 00:00 2023-12-07T10:07:12.245 INFO:teuthology.orchestra.run.smithi119.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 190 kB/s | 16 kB 00:00 2023-12-07T10:07:12.274 INFO:teuthology.orchestra.run.smithi119.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 402 kB/s | 34 kB 00:00 2023-12-07T10:07:12.290 INFO:teuthology.orchestra.run.smithi153.stdout:(40/131): ceph-mgr-diskprediction-local-19.0.0- 6.0 MB/s | 7.4 MB 00:01 2023-12-07T10:07:12.300 INFO:teuthology.orchestra.run.smithi119.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 3.6 MB/s | 196 kB 00:00 2023-12-07T10:07:12.315 INFO:teuthology.orchestra.run.smithi153.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.2 MB/s | 476 kB 00:00 2023-12-07T10:07:12.325 INFO:teuthology.orchestra.run.smithi119.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.3 MB/s | 64 kB 00:00 2023-12-07T10:07:12.350 INFO:teuthology.orchestra.run.smithi119.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 373 kB/s | 19 kB 00:00 2023-12-07T10:07:12.375 INFO:teuthology.orchestra.run.smithi119.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 474 kB/s | 24 kB 00:00 2023-12-07T10:07:12.390 INFO:teuthology.orchestra.run.smithi153.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.6 MB/s | 185 kB 00:00 2023-12-07T10:07:12.401 INFO:teuthology.orchestra.run.smithi119.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 580 kB/s | 29 kB 00:00 2023-12-07T10:07:12.426 INFO:teuthology.orchestra.run.smithi119.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 549 kB/s | 27 kB 00:00 2023-12-07T10:07:12.432 INFO:teuthology.orchestra.run.smithi153.stdout:(43/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 307 kB/s | 44 kB 00:00 2023-12-07T10:07:12.451 INFO:teuthology.orchestra.run.smithi119.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 5.0 MB/s | 255 kB 00:00 2023-12-07T10:07:12.458 INFO:teuthology.orchestra.run.smithi153.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 492 kB/s | 33 kB 00:00 2023-12-07T10:07:12.477 INFO:teuthology.orchestra.run.smithi119.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 4.9 MB/s | 251 kB 00:00 2023-12-07T10:07:12.483 INFO:teuthology.orchestra.run.smithi153.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 322 kB/s | 16 kB 00:00 2023-12-07T10:07:12.502 INFO:teuthology.orchestra.run.smithi119.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-12-07T10:07:12.527 INFO:teuthology.orchestra.run.smithi119.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-12-07T10:07:12.541 INFO:teuthology.orchestra.run.smithi153.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.0 MB/s | 173 kB 00:00 2023-12-07T10:07:12.552 INFO:teuthology.orchestra.run.smithi119.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 457 kB/s | 23 kB 00:00 2023-12-07T10:07:12.569 INFO:teuthology.orchestra.run.smithi119.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 4.1 MB/s | 70 kB 00:00 2023-12-07T10:07:12.582 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2023-12-07T10:07:12.586 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2023-12-07T10:07:12.591 INFO:teuthology.orchestra.run.smithi153.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.6 MB/s | 81 kB 00:00 2023-12-07T10:07:12.634 INFO:teuthology.orchestra.run.smithi153.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.5 MB/s | 384 kB 00:00 2023-12-07T10:07:12.644 INFO:teuthology.orchestra.run.smithi119.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 15 MB/s | 1.7 MB 00:00 2023-12-07T10:07:12.659 INFO:teuthology.orchestra.run.smithi153.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 166 kB/s | 11 kB 00:00 2023-12-07T10:07:12.684 INFO:teuthology.orchestra.run.smithi153.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 351 kB/s | 18 kB 00:00 2023-12-07T10:07:12.751 INFO:teuthology.orchestra.run.smithi153.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 731 kB/s | 49 kB 00:00 2023-12-07T10:07:12.793 INFO:teuthology.orchestra.run.smithi153.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 1.4 MB/s | 59 kB 00:00 2023-12-07T10:07:12.835 INFO:teuthology.orchestra.run.smithi153.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.4 MB/s | 60 kB 00:00 2023-12-07T10:07:12.927 INFO:teuthology.orchestra.run.smithi153.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.0 MB/s | 283 kB 00:00 2023-12-07T10:07:12.969 INFO:teuthology.orchestra.run.smithi153.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 375 kB/s | 16 kB 00:00 2023-12-07T10:07:13.011 INFO:teuthology.orchestra.run.smithi153.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 810 kB/s | 34 kB 00:00 2023-12-07T10:07:13.086 INFO:teuthology.orchestra.run.smithi153.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 2.6 MB/s | 196 kB 00:00 2023-12-07T10:07:13.120 INFO:teuthology.orchestra.run.smithi153.stdout:(58/131): python3-kubernetes-11.0.0-6.el8.noarc 2.5 MB/s | 1.2 MB 00:00 2023-12-07T10:07:13.128 INFO:teuthology.orchestra.run.smithi119.stdout:(71/131): ceph-test-19.0.0-18.g3bd0bfa4.el8.x86 8.0 MB/s | 49 MB 00:06 2023-12-07T10:07:13.145 INFO:teuthology.orchestra.run.smithi153.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.1 MB/s | 64 kB 00:00 2023-12-07T10:07:13.153 INFO:teuthology.orchestra.run.smithi119.stdout:(72/131): python3-pyasn1-modules-0.3.7-6.el8.no 216 kB/s | 110 kB 00:00 2023-12-07T10:07:13.170 INFO:teuthology.orchestra.run.smithi153.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 373 kB/s | 19 kB 00:00 2023-12-07T10:07:13.195 INFO:teuthology.orchestra.run.smithi153.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 477 kB/s | 24 kB 00:00 2023-12-07T10:07:13.204 INFO:teuthology.orchestra.run.smithi119.stdout:(73/131): python3-pytz-2017.2-9.el8.noarch.rpm 725 kB/s | 54 kB 00:00 2023-12-07T10:07:13.229 INFO:teuthology.orchestra.run.smithi119.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.6 MB/s | 259 kB 00:00 2023-12-07T10:07:13.244 INFO:teuthology.orchestra.run.smithi153.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 399 kB/s | 29 kB 00:00 2023-12-07T10:07:13.263 INFO:teuthology.orchestra.run.smithi119.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-12-07T10:07:13.271 INFO:teuthology.orchestra.run.smithi153.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 366 kB/s | 27 kB 00:00 2023-12-07T10:07:13.288 INFO:teuthology.orchestra.run.smithi119.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 664 kB/s | 39 kB 00:00 2023-12-07T10:07:13.330 INFO:teuthology.orchestra.run.smithi119.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-12-07T10:07:13.346 INFO:teuthology.orchestra.run.smithi153.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.4 MB/s | 255 kB 00:00 2023-12-07T10:07:13.372 INFO:teuthology.orchestra.run.smithi153.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.4 MB/s | 251 kB 00:00 2023-12-07T10:07:13.372 INFO:teuthology.orchestra.run.smithi119.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 6.3 MB/s | 538 kB 00:00 2023-12-07T10:07:13.397 INFO:teuthology.orchestra.run.smithi153.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-12-07T10:07:13.422 INFO:teuthology.orchestra.run.smithi153.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-12-07T10:07:13.431 INFO:teuthology.orchestra.run.smithi119.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.5 MB/s | 157 kB 00:00 2023-12-07T10:07:13.448 INFO:teuthology.orchestra.run.smithi153.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 459 kB/s | 23 kB 00:00 2023-12-07T10:07:13.464 INFO:teuthology.orchestra.run.smithi119.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 2.0 MB/s | 187 kB 00:00 2023-12-07T10:07:13.473 INFO:teuthology.orchestra.run.smithi153.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-12-07T10:07:13.506 INFO:teuthology.orchestra.run.smithi119.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 9.3 MB/s | 713 kB 00:00 2023-12-07T10:07:13.523 INFO:teuthology.orchestra.run.smithi153.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 942 kB/s | 47 kB 00:00 2023-12-07T10:07:13.549 INFO:teuthology.orchestra.run.smithi119.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 635 kB/s | 27 kB 00:00 2023-12-07T10:07:13.565 INFO:teuthology.orchestra.run.smithi153.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 4.7 MB/s | 200 kB 00:00 2023-12-07T10:07:13.591 INFO:teuthology.orchestra.run.smithi119.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-12-07T10:07:13.598 INFO:teuthology.orchestra.run.smithi153.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.3 MB/s | 43 kB 00:00 2023-12-07T10:07:13.632 INFO:teuthology.orchestra.run.smithi119.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.0 MB/s | 44 kB 00:00 2023-12-07T10:07:13.640 INFO:teuthology.orchestra.run.smithi153.stdout:(73/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2023-12-07T10:07:13.683 INFO:teuthology.orchestra.run.smithi153.stdout:(74/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.1 MB/s | 48 kB 00:00 2023-12-07T10:07:13.741 INFO:teuthology.orchestra.run.smithi119.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2023-12-07T10:07:13.818 INFO:teuthology.orchestra.run.smithi119.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 5.8 MB/s | 457 kB 00:00 2023-12-07T10:07:13.900 INFO:teuthology.orchestra.run.smithi119.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 14 MB/s | 3.7 MB 00:00 2023-12-07T10:07:13.925 INFO:teuthology.orchestra.run.smithi119.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.1 MB/s | 225 kB 00:00 2023-12-07T10:07:13.950 INFO:teuthology.orchestra.run.smithi119.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2023-12-07T10:07:13.976 INFO:teuthology.orchestra.run.smithi119.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-12-07T10:07:14.001 INFO:teuthology.orchestra.run.smithi119.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 397 kB/s | 20 kB 00:00 2023-12-07T10:07:14.027 INFO:teuthology.orchestra.run.smithi119.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-12-07T10:07:14.068 INFO:teuthology.orchestra.run.smithi119.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-12-07T10:07:14.191 INFO:teuthology.orchestra.run.smithi153.stdout:(75/131): ceph-test-19.0.0-18.g3bd0bfa4.el8.x86 5.4 MB/s | 49 MB 00:09 2023-12-07T10:07:14.217 INFO:teuthology.orchestra.run.smithi153.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 163 kB/s | 87 kB 00:00 2023-12-07T10:07:14.252 INFO:teuthology.orchestra.run.smithi119.stdout:(94/131): python3-scipy-1.0.0-21.module+el8.5.0 8.1 MB/s | 14 MB 00:01 2023-12-07T10:07:14.275 INFO:teuthology.orchestra.run.smithi153.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 513 kB/s | 43 kB 00:00 2023-12-07T10:07:14.277 INFO:teuthology.orchestra.run.smithi119.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 948 kB/s | 198 kB 00:00 2023-12-07T10:07:14.292 INFO:teuthology.orchestra.run.smithi153.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 1.5 MB/s | 109 kB 00:00 2023-12-07T10:07:14.319 INFO:teuthology.orchestra.run.smithi119.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-12-07T10:07:14.334 INFO:teuthology.orchestra.run.smithi153.stdout:(79/131): thrift-0.13.0-2.el8.x86_64.rpm 1.9 MB/s | 1.7 MB 00:00 2023-12-07T10:07:14.360 INFO:teuthology.orchestra.run.smithi153.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.7 MB/s | 229 kB 00:00 2023-12-07T10:07:14.361 INFO:teuthology.orchestra.run.smithi119.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-12-07T10:07:14.402 INFO:teuthology.orchestra.run.smithi153.stdout:(81/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 5.0 MB/s | 560 kB 00:00 2023-12-07T10:07:14.427 INFO:teuthology.orchestra.run.smithi153.stdout:(82/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 203 kB/s | 19 kB 00:00 2023-12-07T10:07:14.452 INFO:teuthology.orchestra.run.smithi153.stdout:(83/131): python3-oauthlib-2.1.0-1.el8.noarch.r 1.7 MB/s | 155 kB 00:00 2023-12-07T10:07:14.453 INFO:teuthology.orchestra.run.smithi119.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 11 MB/s | 4.9 MB 00:00 2023-12-07T10:07:14.477 INFO:teuthology.orchestra.run.smithi153.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 357 kB/s | 18 kB 00:00 2023-12-07T10:07:14.519 INFO:teuthology.orchestra.run.smithi153.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 4.7 MB/s | 559 kB 00:00 2023-12-07T10:07:14.520 INFO:teuthology.orchestra.run.smithi119.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 12 MB/s | 808 kB 00:00 2023-12-07T10:07:14.553 INFO:teuthology.orchestra.run.smithi153.stdout:(86/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.9 MB/s | 605 kB 00:00 2023-12-07T10:07:14.578 INFO:teuthology.orchestra.run.smithi153.stdout:(87/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.3 MB/s | 238 kB 00:00 2023-12-07T10:07:14.587 INFO:teuthology.orchestra.run.smithi119.stdout:(100/131): openblas-0.3.15-3.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-12-07T10:07:14.603 INFO:teuthology.orchestra.run.smithi153.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-12-07T10:07:14.628 INFO:teuthology.orchestra.run.smithi153.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-12-07T10:07:14.646 INFO:teuthology.orchestra.run.smithi119.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 804 kB/s | 47 kB 00:00 2023-12-07T10:07:14.654 INFO:teuthology.orchestra.run.smithi153.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-12-07T10:07:14.679 INFO:teuthology.orchestra.run.smithi153.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-12-07T10:07:14.696 INFO:teuthology.orchestra.run.smithi119.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 8.6 MB/s | 1.5 MB 00:00 2023-12-07T10:07:14.704 INFO:teuthology.orchestra.run.smithi153.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-12-07T10:07:14.721 INFO:teuthology.orchestra.run.smithi119.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.6 MB/s | 200 kB 00:00 2023-12-07T10:07:14.729 INFO:teuthology.orchestra.run.smithi153.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 769 kB/s | 39 kB 00:00 2023-12-07T10:07:14.747 INFO:teuthology.orchestra.run.smithi119.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 854 kB/s | 43 kB 00:00 2023-12-07T10:07:14.772 INFO:teuthology.orchestra.run.smithi119.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-12-07T10:07:14.796 INFO:teuthology.orchestra.run.smithi153.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 6.9 MB/s | 644 kB 00:00 2023-12-07T10:07:14.797 INFO:teuthology.orchestra.run.smithi119.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 958 kB/s | 48 kB 00:00 2023-12-07T10:07:14.823 INFO:teuthology.orchestra.run.smithi119.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-12-07T10:07:14.830 INFO:teuthology.orchestra.run.smithi153.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 1.7 MB/s | 171 kB 00:00 2023-12-07T10:07:14.847 INFO:teuthology.orchestra.run.smithi153.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.6 MB/s | 138 kB 00:00 2023-12-07T10:07:14.847 INFO:teuthology.orchestra.run.smithi119.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 850 kB/s | 43 kB 00:00 2023-12-07T10:07:14.873 INFO:teuthology.orchestra.run.smithi119.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-12-07T10:07:14.889 INFO:teuthology.orchestra.run.smithi153.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-12-07T10:07:14.898 INFO:teuthology.orchestra.run.smithi119.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-12-07T10:07:14.939 INFO:teuthology.orchestra.run.smithi153.stdout:(98/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 5.5 MB/s | 2.3 MB 00:00 2023-12-07T10:07:14.964 INFO:teuthology.orchestra.run.smithi153.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 721 kB/s | 54 kB 00:00 2023-12-07T10:07:14.974 INFO:teuthology.orchestra.run.smithi119.stdout:(111/131): postgresql-server-10.21-2.module+el8 8.3 MB/s | 5.1 MB 00:00 2023-12-07T10:07:14.990 INFO:teuthology.orchestra.run.smithi153.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.0 MB/s | 259 kB 00:00 2023-12-07T10:07:15.021 INFO:teuthology.orchestra.run.smithi119.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 153 kB/s | 19 kB 00:00 2023-12-07T10:07:15.023 INFO:teuthology.orchestra.run.smithi153.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-12-07T10:07:15.049 INFO:teuthology.orchestra.run.smithi153.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 664 kB/s | 39 kB 00:00 2023-12-07T10:07:15.058 INFO:teuthology.orchestra.run.smithi119.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 1.8 MB/s | 155 kB 00:00 2023-12-07T10:07:15.099 INFO:teuthology.orchestra.run.smithi153.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 9.0 MB/s | 688 kB 00:00 2023-12-07T10:07:15.100 INFO:teuthology.orchestra.run.smithi119.stdout:(114/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 2.4 MB/s | 560 kB 00:00 2023-12-07T10:07:15.125 INFO:teuthology.orchestra.run.smithi119.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 267 kB/s | 18 kB 00:00 2023-12-07T10:07:15.135 INFO:teuthology.orchestra.run.smithi153.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 6.1 MB/s | 538 kB 00:00 2023-12-07T10:07:15.167 INFO:teuthology.orchestra.run.smithi119.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 3.7 MB/s | 559 kB 00:00 2023-12-07T10:07:15.201 INFO:teuthology.orchestra.run.smithi119.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.1 MB/s | 238 kB 00:00 2023-12-07T10:07:15.226 INFO:teuthology.orchestra.run.smithi119.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.7 MB/s | 605 kB 00:00 2023-12-07T10:07:15.260 INFO:teuthology.orchestra.run.smithi119.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.2 MB/s | 72 kB 00:00 2023-12-07T10:07:15.277 INFO:teuthology.orchestra.run.smithi119.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-12-07T10:07:15.285 INFO:teuthology.orchestra.run.smithi153.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 845 kB/s | 157 kB 00:00 2023-12-07T10:07:15.302 INFO:teuthology.orchestra.run.smithi119.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.4 MB/s | 60 kB 00:00 2023-12-07T10:07:15.327 INFO:teuthology.orchestra.run.smithi119.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-12-07T10:07:15.352 INFO:teuthology.orchestra.run.smithi119.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-12-07T10:07:15.378 INFO:teuthology.orchestra.run.smithi119.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 768 kB/s | 39 kB 00:00 2023-12-07T10:07:15.436 INFO:teuthology.orchestra.run.smithi119.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 7.5 MB/s | 644 kB 00:00 2023-12-07T10:07:15.462 INFO:teuthology.orchestra.run.smithi119.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.0 MB/s | 171 kB 00:00 2023-12-07T10:07:15.487 INFO:teuthology.orchestra.run.smithi119.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-12-07T10:07:15.537 INFO:teuthology.orchestra.run.smithi119.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.3 MB/s | 2.3 MB 00:00 2023-12-07T10:07:15.552 INFO:teuthology.orchestra.run.smithi153.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 450 kB/s | 187 kB 00:00 2023-12-07T10:07:15.596 INFO:teuthology.orchestra.run.smithi119.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-12-07T10:07:15.602 INFO:teuthology.orchestra.run.smithi153.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 2.2 MB/s | 713 kB 00:00 2023-12-07T10:07:15.636 INFO:teuthology.orchestra.run.smithi153.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 800 kB/s | 27 kB 00:00 2023-12-07T10:07:15.678 INFO:teuthology.orchestra.run.smithi153.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-12-07T10:07:15.703 INFO:teuthology.orchestra.run.smithi153.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.7 MB/s | 44 kB 00:00 2023-12-07T10:07:15.795 INFO:teuthology.orchestra.run.smithi153.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2023-12-07T10:07:15.846 INFO:teuthology.orchestra.run.smithi119.stdout:(130/131): librados2-19.0.0-18.g3bd0bfa4.el8.x8 9.8 MB/s | 3.8 MB 00:00 2023-12-07T10:07:15.862 INFO:teuthology.orchestra.run.smithi153.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 6.7 MB/s | 457 kB 00:00 2023-12-07T10:07:15.904 INFO:teuthology.orchestra.run.smithi153.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-12-07T10:07:15.979 INFO:teuthology.orchestra.run.smithi153.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 14 MB/s | 3.7 MB 00:00 2023-12-07T10:07:15.996 INFO:teuthology.orchestra.run.smithi153.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 697 kB/s | 64 kB 00:00 2023-12-07T10:07:16.021 INFO:teuthology.orchestra.run.smithi153.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 3.0 MB/s | 126 kB 00:00 2023-12-07T10:07:16.038 INFO:teuthology.orchestra.run.smithi153.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 476 kB/s | 20 kB 00:00 2023-12-07T10:07:16.063 INFO:teuthology.orchestra.run.smithi153.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2023-12-07T10:07:16.105 INFO:teuthology.orchestra.run.smithi153.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-12-07T10:07:16.147 INFO:teuthology.orchestra.run.smithi153.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-12-07T10:07:16.281 INFO:teuthology.orchestra.run.smithi153.stdout:(121/131): python3-scipy-1.0.0-21.module+el8.5. 9.4 MB/s | 14 MB 00:01 2023-12-07T10:07:16.296 INFO:teuthology.orchestra.run.smithi119.stdout:(131/131): librbd1-19.0.0-18.g3bd0bfa4.el8.x86_ 4.6 MB/s | 3.7 MB 00:00 2023-12-07T10:07:16.299 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-12-07T10:07:16.299 INFO:teuthology.orchestra.run.smithi119.stdout:Total 15 MB/s | 209 MB 00:14 2023-12-07T10:07:16.314 INFO:teuthology.orchestra.run.smithi153.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 6.0 MB/s | 202 kB 00:00 2023-12-07T10:07:16.373 INFO:teuthology.orchestra.run.smithi153.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 5.9 MB/s | 354 kB 00:00 2023-12-07T10:07:16.457 INFO:teuthology.orchestra.run.smithi153.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 12 MB/s | 4.9 MB 00:00 2023-12-07T10:07:16.532 INFO:teuthology.orchestra.run.smithi153.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 12 MB/s | 4.7 MB 00:00 2023-12-07T10:07:16.565 INFO:teuthology.orchestra.run.smithi153.stdout:(126/131): postgresql-contrib-10.21-2.module+el 7.3 MB/s | 808 kB 00:00 2023-12-07T10:07:16.625 INFO:teuthology.orchestra.run.smithi153.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 16 MB/s | 1.5 MB 00:00 2023-12-07T10:07:16.708 INFO:teuthology.orchestra.run.smithi153.stdout:(128/131): postgresql-server-10.21-2.module+el8 15 MB/s | 5.1 MB 00:00 2023-12-07T10:07:16.766 INFO:teuthology.orchestra.run.smithi153.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-12-07T10:07:17.091 INFO:teuthology.orchestra.run.smithi153.stdout:(130/131): librados2-19.0.0-18.g3bd0bfa4.el8.x8 7.2 MB/s | 3.8 MB 00:00 2023-12-07T10:07:17.258 INFO:teuthology.orchestra.run.smithi153.stdout:(131/131): librbd1-19.0.0-18.g3bd0bfa4.el8.x86_ 5.8 MB/s | 3.7 MB 00:00 2023-12-07T10:07:17.262 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2023-12-07T10:07:17.262 INFO:teuthology.orchestra.run.smithi153.stdout:Total 11 MB/s | 209 MB 00:19 2023-12-07T10:07:17.342 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2023-12-07T10:07:17.767 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-12-07T10:07:17.930 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-12-07T10:07:17.951 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-12-07T10:07:18.063 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-12-07T10:07:18.063 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-12-07T10:07:18.127 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2023-12-07T10:07:18.143 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-12-07T10:07:18.248 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2023-12-07T10:07:18.248 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2023-12-07T10:07:18.260 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-12-07T10:07:18.477 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-12-07T10:07:18.869 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-12-07T10:07:19.093 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librados2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 5/134 2023-12-07T10:07:19.380 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2023-12-07T10:07:19.384 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2023-12-07T10:07:19.542 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 5/134 2023-12-07T10:07:19.632 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-12-07T10:07:19.636 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-12-07T10:07:19.726 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librbd1-2:19.0.0-18.g3bd0bfa4.el8.x86_64 6/134 2023-12-07T10:07:19.859 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-2:19.0.0-18.g3bd0bfa4.el8.x86_64 6/134 2023-12-07T10:07:20.043 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 7/134 2023-12-07T10:07:20.136 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephfs2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 7/134 2023-12-07T10:07:20.281 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rados-2:19.0.0-18.g3bd0bfa4.el8.x86_64 8/134 2023-12-07T10:07:21.058 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-12-07T10:07:21.201 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librgw2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 10/134 2023-12-07T10:07:21.292 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librgw2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 10/134 2023-12-07T10:07:21.538 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libpq-13.5-1.el8.x86_64 11/134 2023-12-07T10:07:21.804 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/134 2023-12-07T10:07:22.209 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/134 2023-12-07T10:07:22.442 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/134 2023-12-07T10:07:22.584 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephsqlite-2:19.0.0-18.g3bd0bfa4.el8.x86_64 15/134 2023-12-07T10:07:22.687 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephsqlite-2:19.0.0-18.g3bd0bfa4.el8.x86_64 15/134 2023-12-07T10:07:22.851 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libradosstriper1-2:19.0.0-18.g3bd0bfa4.el8.x86_6 16/134 2023-12-07T10:07:22.994 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libradosstriper1-2:19.0.0-18.g3bd0bfa4.el8.x86_6 16/134 2023-12-07T10:07:23.415 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/134 2023-12-07T10:07:23.609 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 18/134 2023-12-07T10:07:23.782 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 19/134 2023-12-07T10:07:23.942 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 20/134 2023-12-07T10:07:24.077 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 20/134 2023-12-07T10:07:24.251 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 21/134 2023-12-07T10:07:24.451 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2023-12-07T10:07:24.637 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-12-07T10:07:24.829 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-12-07T10:07:24.861 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 21/134 2023-12-07T10:07:24.997 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-12-07T10:07:25.037 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-12-07T10:07:25.051 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 22/134 2023-12-07T10:07:25.149 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-12-07T10:07:25.246 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-12-07T10:07:25.347 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-12-07T10:07:25.438 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-12-07T10:07:25.651 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 22/134 2023-12-07T10:07:25.669 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-12-07T10:07:25.761 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-12-07T10:07:25.817 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openblas-0.3.15-3.el8.x86_64 23/134 2023-12-07T10:07:25.849 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-12-07T10:07:25.983 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-12-07T10:07:26.239 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : librados2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 5/134 2023-12-07T10:07:26.299 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-12-07T10:07:26.512 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 23/134 2023-12-07T10:07:26.650 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 5/134 2023-12-07T10:07:26.757 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 24/134 2023-12-07T10:07:26.762 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-12-07T10:07:26.839 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : librbd1-2:19.0.0-18.g3bd0bfa4.el8.x86_64 6/134 2023-12-07T10:07:26.954 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librbd1-2:19.0.0-18.g3bd0bfa4.el8.x86_64 6/134 2023-12-07T10:07:26.998 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 25/134 2023-12-07T10:07:27.032 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librados2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 6/134 2023-12-07T10:07:27.138 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephfs2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 7/134 2023-12-07T10:07:27.221 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephfs2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 7/134 2023-12-07T10:07:27.245 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/134 2023-12-07T10:07:27.377 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rados-2:19.0.0-18.g3bd0bfa4.el8.x86_64 8/134 2023-12-07T10:07:27.398 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-common-2:19.0.0-18.g3bd0bfa4.el8.x8 27/134 2023-12-07T10:07:27.490 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 6/134 2023-12-07T10:07:27.528 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-argparse-2:19.0.0-18.g3bd0bfa4.el8. 28/134 2023-12-07T10:07:27.690 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librbd1-2:19.0.0-18.g3bd0bfa4.el8.x86_64 7/134 2023-12-07T10:07:27.779 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cephfs-2:19.0.0-18.g3bd0bfa4.el8.x86_64 29/134 2023-12-07T10:07:27.843 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:19.0.0-18.g3bd0bfa4.el8.x86_64 7/134 2023-12-07T10:07:27.992 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/134 2023-12-07T10:07:28.082 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 8/134 2023-12-07T10:07:28.130 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-12-07T10:07:28.199 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephfs2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 8/134 2023-12-07T10:07:28.305 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librgw2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 10/134 2023-12-07T10:07:28.396 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librgw2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 10/134 2023-12-07T10:07:28.617 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libpq-13.5-1.el8.x86_64 11/134 2023-12-07T10:07:28.841 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/134 2023-12-07T10:07:29.087 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rados-2:19.0.0-18.g3bd0bfa4.el8.x86_64 9/134 2023-12-07T10:07:29.223 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/134 2023-12-07T10:07:29.265 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librgw2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 10/134 2023-12-07T10:07:29.375 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librgw2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 10/134 2023-12-07T10:07:29.481 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/134 2023-12-07T10:07:29.621 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephsqlite-2:19.0.0-18.g3bd0bfa4.el8.x86_64 15/134 2023-12-07T10:07:29.664 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libpq-13.5-1.el8.x86_64 11/134 2023-12-07T10:07:29.722 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephsqlite-2:19.0.0-18.g3bd0bfa4.el8.x86_64 15/134 2023-12-07T10:07:29.758 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/134 2023-12-07T10:07:29.903 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/134 2023-12-07T10:07:29.913 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libradosstriper1-2:19.0.0-18.g3bd0bfa4.el8.x86_6 16/134 2023-12-07T10:07:30.061 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libradosstriper1-2:19.0.0-18.g3bd0bfa4.el8.x86_6 16/134 2023-12-07T10:07:30.275 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/134 2023-12-07T10:07:30.346 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/134 2023-12-07T10:07:30.469 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 18/134 2023-12-07T10:07:30.607 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/134 2023-12-07T10:07:30.640 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 19/134 2023-12-07T10:07:30.755 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 31/134 2023-12-07T10:07:30.779 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 20/134 2023-12-07T10:07:30.815 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephsqlite-2:19.0.0-18.g3bd0bfa4.el8.x86_64 15/134 2023-12-07T10:07:30.941 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 20/134 2023-12-07T10:07:30.943 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephsqlite-2:19.0.0-18.g3bd0bfa4.el8.x86_64 15/134 2023-12-07T10:07:31.056 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 32/134 2023-12-07T10:07:31.105 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 21/134 2023-12-07T10:07:31.165 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libradosstriper1-2:19.0.0-18.g3bd0bfa4.el8.x86_6 16/134 2023-12-07T10:07:31.319 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 33/134 2023-12-07T10:07:31.333 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libradosstriper1-2:19.0.0-18.g3bd0bfa4.el8.x86_6 16/134 2023-12-07T10:07:31.570 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 34/134 2023-12-07T10:07:31.631 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 17/134 2023-12-07T10:07:31.721 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 21/134 2023-12-07T10:07:31.778 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rsa-4.9-2.el8.noarch 35/134 2023-12-07T10:07:31.807 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 17/134 2023-12-07T10:07:31.921 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 22/134 2023-12-07T10:07:31.944 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/134 2023-12-07T10:07:31.998 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 18/134 2023-12-07T10:07:32.134 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rgw-2:19.0.0-18.g3bd0bfa4.el8.x86_64 37/134 2023-12-07T10:07:32.297 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rbd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 38/134 2023-12-07T10:07:32.473 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librados-devel-2:19.0.0-18.g3bd0bfa4.el8.x86_64 39/134 2023-12-07T10:07:32.519 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 22/134 2023-12-07T10:07:32.665 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 40/134 2023-12-07T10:07:32.684 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 18/134 2023-12-07T10:07:32.687 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openblas-0.3.15-3.el8.x86_64 23/134 2023-12-07T10:07:32.834 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 41/134 2023-12-07T10:07:32.864 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 19/134 2023-12-07T10:07:33.025 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : uuid-1.6.2-43.el8.x86_64 42/134 2023-12-07T10:07:33.094 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 42/134 2023-12-07T10:07:33.329 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 43/134 2023-12-07T10:07:33.395 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 23/134 2023-12-07T10:07:33.541 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 19/134 2023-12-07T10:07:33.544 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : automake-1.16.1-7.el8.noarch 44/134 2023-12-07T10:07:33.636 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 24/134 2023-12-07T10:07:33.724 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/134 2023-12-07T10:07:33.739 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-0.3.15-3.el8.x86_64 20/134 2023-12-07T10:07:33.810 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 25/134 2023-12-07T10:07:33.839 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/134 2023-12-07T10:07:34.037 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : jq-1.6-3.el8.x86_64 46/134 2023-12-07T10:07:34.073 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/134 2023-12-07T10:07:34.186 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 47/134 2023-12-07T10:07:34.235 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ceph-common-2:19.0.0-18.g3bd0bfa4.el8.x8 27/134 2023-12-07T10:07:34.356 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 48/134 2023-12-07T10:07:34.365 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ceph-argparse-2:19.0.0-18.g3bd0bfa4.el8. 28/134 2023-12-07T10:07:34.550 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 49/134 2023-12-07T10:07:34.599 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 20/134 2023-12-07T10:07:34.600 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cephfs-2:19.0.0-18.g3bd0bfa4.el8.x86_64 29/134 2023-12-07T10:07:34.743 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : fuse-2.9.7-15.el8.x86_64 50/134 2023-12-07T10:07:34.812 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/134 2023-12-07T10:07:34.855 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 21/134 2023-12-07T10:07:34.903 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 51/134 2023-12-07T10:07:35.083 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 22/134 2023-12-07T10:07:35.149 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 52/134 2023-12-07T10:07:35.312 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/134 2023-12-07T10:07:35.372 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 53/134 2023-12-07T10:07:35.523 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/134 2023-12-07T10:07:35.765 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 25/134 2023-12-07T10:07:35.804 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 54/134 2023-12-07T10:07:36.060 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/134 2023-12-07T10:07:36.094 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 55/134 2023-12-07T10:07:36.254 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-common-2:19.0.0-18.g3bd0bfa4.el8.x8 27/134 2023-12-07T10:07:36.329 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 56/134 2023-12-07T10:07:36.436 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-argparse-2:19.0.0-18.g3bd0bfa4.el8. 28/134 2023-12-07T10:07:36.536 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 57/134 2023-12-07T10:07:36.581 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/134 2023-12-07T10:07:36.753 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 58/134 2023-12-07T10:07:37.073 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 59/134 2023-12-07T10:07:37.274 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 60/134 2023-12-07T10:07:37.483 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cephfs-2:19.0.0-18.g3bd0bfa4.el8.x86_64 29/134 2023-12-07T10:07:37.508 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 61/134 2023-12-07T10:07:37.620 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 31/134 2023-12-07T10:07:37.752 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 62/134 2023-12-07T10:07:37.852 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 30/134 2023-12-07T10:07:37.919 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 32/134 2023-12-07T10:07:37.927 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 63/134 2023-12-07T10:07:38.084 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 64/134 2023-12-07T10:07:38.139 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 31/134 2023-12-07T10:07:38.189 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 33/134 2023-12-07T10:07:38.363 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 65/134 2023-12-07T10:07:38.400 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 34/134 2023-12-07T10:07:38.447 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 32/134 2023-12-07T10:07:38.607 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rsa-4.9-2.el8.noarch 35/134 2023-12-07T10:07:38.644 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 66/134 2023-12-07T10:07:38.680 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/134 2023-12-07T10:07:38.756 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/134 2023-12-07T10:07:38.880 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 67/134 2023-12-07T10:07:38.963 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rgw-2:19.0.0-18.g3bd0bfa4.el8.x86_64 37/134 2023-12-07T10:07:39.186 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rbd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 38/134 2023-12-07T10:07:39.343 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librados-devel-2:19.0.0-18.g3bd0bfa4.el8.x86_64 39/134 2023-12-07T10:07:39.349 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 68/134 2023-12-07T10:07:39.485 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 40/134 2023-12-07T10:07:39.514 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 69/134 2023-12-07T10:07:39.654 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 41/134 2023-12-07T10:07:39.692 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 70/134 2023-12-07T10:07:39.836 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : uuid-1.6.2-43.el8.x86_64 42/134 2023-12-07T10:07:39.894 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 42/134 2023-12-07T10:07:39.950 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 71/134 2023-12-07T10:07:40.105 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 43/134 2023-12-07T10:07:40.180 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 72/134 2023-12-07T10:07:40.306 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : automake-1.16.1-7.el8.noarch 44/134 2023-12-07T10:07:40.335 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 73/134 2023-12-07T10:07:40.445 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/134 2023-12-07T10:07:40.492 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 74/134 2023-12-07T10:07:40.514 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/134 2023-12-07T10:07:40.681 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 75/134 2023-12-07T10:07:40.732 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/134 2023-12-07T10:07:40.740 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : jq-1.6-3.el8.x86_64 46/134 2023-12-07T10:07:40.883 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 76/134 2023-12-07T10:07:40.906 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 47/134 2023-12-07T10:07:41.076 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 48/134 2023-12-07T10:07:41.096 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 34/134 2023-12-07T10:07:41.160 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 77/134 2023-12-07T10:07:41.271 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 49/134 2023-12-07T10:07:41.326 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rsa-4.9-2.el8.noarch 35/134 2023-12-07T10:07:41.326 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 78/134 2023-12-07T10:07:41.455 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : fuse-2.9.7-15.el8.x86_64 50/134 2023-12-07T10:07:41.496 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 79/134 2023-12-07T10:07:41.631 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 51/134 2023-12-07T10:07:41.696 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 80/134 2023-12-07T10:07:41.815 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 52/134 2023-12-07T10:07:41.846 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/134 2023-12-07T10:07:41.902 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 81/134 2023-12-07T10:07:41.988 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 53/134 2023-12-07T10:07:42.087 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rgw-2:19.0.0-18.g3bd0bfa4.el8.x86_64 37/134 2023-12-07T10:07:42.087 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 82/134 2023-12-07T10:07:42.291 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rbd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 38/134 2023-12-07T10:07:42.329 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-portend-2.6-1.el8.noarch 83/134 2023-12-07T10:07:42.411 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 54/134 2023-12-07T10:07:42.469 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librados-devel-2:19.0.0-18.g3bd0bfa4.el8.x86_64 39/134 2023-12-07T10:07:42.547 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 84/134 2023-12-07T10:07:42.665 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 40/134 2023-12-07T10:07:42.722 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 85/134 2023-12-07T10:07:42.729 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 55/134 2023-12-07T10:07:42.867 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 41/134 2023-12-07T10:07:42.939 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 56/134 2023-12-07T10:07:42.973 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 86/134 2023-12-07T10:07:43.082 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : fuse-2.9.7-15.el8.x86_64 42/134 2023-12-07T10:07:43.148 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 57/134 2023-12-07T10:07:43.244 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 43/134 2023-12-07T10:07:43.365 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 58/134 2023-12-07T10:07:43.473 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 44/134 2023-12-07T10:07:43.651 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 59/134 2023-12-07T10:07:43.720 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 45/134 2023-12-07T10:07:43.869 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 60/134 2023-12-07T10:07:44.062 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 61/134 2023-12-07T10:07:44.075 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 87/134 2023-12-07T10:07:44.174 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 46/134 2023-12-07T10:07:44.324 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 62/134 2023-12-07T10:07:44.351 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 88/134 2023-12-07T10:07:44.482 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 63/134 2023-12-07T10:07:44.495 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 47/134 2023-12-07T10:07:44.638 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 64/134 2023-12-07T10:07:44.647 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 89/134 2023-12-07T10:07:44.736 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 48/134 2023-12-07T10:07:44.972 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 90/134 2023-12-07T10:07:44.976 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 65/134 2023-12-07T10:07:44.990 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 49/134 2023-12-07T10:07:45.226 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 91/134 2023-12-07T10:07:45.259 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 66/134 2023-12-07T10:07:45.273 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 50/134 2023-12-07T10:07:45.433 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 92/134 2023-12-07T10:07:45.474 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 67/134 2023-12-07T10:07:45.623 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 93/134 2023-12-07T10:07:45.654 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 51/134 2023-12-07T10:07:45.703 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 68/134 2023-12-07T10:07:45.885 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 69/134 2023-12-07T10:07:45.923 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 52/134 2023-12-07T10:07:46.079 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 70/134 2023-12-07T10:07:46.123 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 53/134 2023-12-07T10:07:46.344 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 71/134 2023-12-07T10:07:46.409 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 54/134 2023-12-07T10:07:46.542 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 72/134 2023-12-07T10:07:46.630 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 55/134 2023-12-07T10:07:46.689 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 73/134 2023-12-07T10:07:46.860 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 56/134 2023-12-07T10:07:46.862 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 74/134 2023-12-07T10:07:47.236 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 57/134 2023-12-07T10:07:47.412 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 75/134 2023-12-07T10:07:47.559 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 58/134 2023-12-07T10:07:47.608 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 76/134 2023-12-07T10:07:47.788 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 77/134 2023-12-07T10:07:47.852 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 59/134 2023-12-07T10:07:47.980 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 78/134 2023-12-07T10:07:48.079 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 60/134 2023-12-07T10:07:48.133 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 79/134 2023-12-07T10:07:48.329 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 61/134 2023-12-07T10:07:48.333 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 80/134 2023-12-07T10:07:48.499 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 62/134 2023-12-07T10:07:48.573 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 81/134 2023-12-07T10:07:48.720 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : uuid-1.6.2-43.el8.x86_64 63/134 2023-12-07T10:07:48.758 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 82/134 2023-12-07T10:07:48.819 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 63/134 2023-12-07T10:07:49.033 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-portend-2.6-1.el8.noarch 83/134 2023-12-07T10:07:49.116 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 64/134 2023-12-07T10:07:49.243 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 84/134 2023-12-07T10:07:49.346 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : automake-1.16.1-7.el8.noarch 65/134 2023-12-07T10:07:49.391 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:19.0.0-18.g3bd0bfa4.el8.x86_64 94/134 2023-12-07T10:07:49.435 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 85/134 2023-12-07T10:07:49.503 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 66/134 2023-12-07T10:07:49.605 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 66/134 2023-12-07T10:07:49.662 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 86/134 2023-12-07T10:07:49.857 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : jq-1.6-3.el8.x86_64 67/134 2023-12-07T10:07:50.031 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 68/134 2023-12-07T10:07:50.046 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-common-2:19.0.0-18.g3bd0bfa4.el8.x86_64 94/134 2023-12-07T10:07:50.230 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 69/134 2023-12-07T10:07:50.429 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 70/134 2023-12-07T10:07:50.641 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:19.0.0-18.g3bd0bfa4.el8.x86_64 94/134 2023-12-07T10:07:50.738 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 71/134 2023-12-07T10:07:50.745 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 87/134 2023-12-07T10:07:50.924 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-base-2:19.0.0-18.g3bd0bfa4.el8.x86_64 95/134 2023-12-07T10:07:51.020 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 72/134 2023-12-07T10:07:51.032 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-base-2:19.0.0-18.g3bd0bfa4.el8.x86_64 95/134 2023-12-07T10:07:51.038 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 88/134 2023-12-07T10:07:51.163 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-selinux-2:19.0.0-18.g3bd0bfa4.el8.x86_64 96/134 2023-12-07T10:07:51.191 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 73/134 2023-12-07T10:07:51.259 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 89/134 2023-12-07T10:07:51.391 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 74/134 2023-12-07T10:07:51.577 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 90/134 2023-12-07T10:07:51.593 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 75/134 2023-12-07T10:07:51.828 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 76/134 2023-12-07T10:07:51.852 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 91/134 2023-12-07T10:07:52.035 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 77/134 2023-12-07T10:07:52.082 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 92/134 2023-12-07T10:07:52.240 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 78/134 2023-12-07T10:07:52.268 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 93/134 2023-12-07T10:07:52.410 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 79/134 2023-12-07T10:07:52.667 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 80/134 2023-12-07T10:07:52.884 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 81/134 2023-12-07T10:07:53.127 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 82/134 2023-12-07T10:07:53.443 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-portend-2.6-1.el8.noarch 83/134 2023-12-07T10:07:53.671 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 84/134 2023-12-07T10:07:53.882 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 85/134 2023-12-07T10:07:54.156 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 86/134 2023-12-07T10:07:55.489 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 87/134 2023-12-07T10:07:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 88/134 2023-12-07T10:07:55.981 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-common-2:19.0.0-18.g3bd0bfa4.el8.x86_64 94/134 2023-12-07T10:07:56.141 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 89/134 2023-12-07T10:07:56.243 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-common-2:19.0.0-18.g3bd0bfa4.el8.x86_64 94/134 2023-12-07T10:07:56.518 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 90/134 2023-12-07T10:07:56.814 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 91/134 2023-12-07T10:07:56.824 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-common-2:19.0.0-18.g3bd0bfa4.el8.x86_64 94/134 2023-12-07T10:07:57.130 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-base-2:19.0.0-18.g3bd0bfa4.el8.x86_64 95/134 2023-12-07T10:07:57.201 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-base-2:19.0.0-18.g3bd0bfa4.el8.x86_64 95/134 2023-12-07T10:07:57.258 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 92/134 2023-12-07T10:07:57.333 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-selinux-2:19.0.0-18.g3bd0bfa4.el8.x86_64 96/134 2023-12-07T10:07:57.453 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 93/134 2023-12-07T10:08:02.374 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:19.0.0-18.g3bd0bfa4.el8.x86_64 94/134 2023-12-07T10:08:02.753 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-common-2:19.0.0-18.g3bd0bfa4.el8.x86_64 94/134 2023-12-07T10:08:03.431 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:19.0.0-18.g3bd0bfa4.el8.x86_64 94/134 2023-12-07T10:08:03.725 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-base-2:19.0.0-18.g3bd0bfa4.el8.x86_64 95/134 2023-12-07T10:08:03.850 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-base-2:19.0.0-18.g3bd0bfa4.el8.x86_64 95/134 2023-12-07T10:08:04.051 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-selinux-2:19.0.0-18.g3bd0bfa4.el8.x86_64 96/134 2023-12-07T10:08:09.989 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-selinux-2:19.0.0-18.g3bd0bfa4.el8.x86_64 96/134 2023-12-07T10:08:09.989 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /sys 2023-12-07T10:08:09.989 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /proc 2023-12-07T10:08:09.989 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /dev 2023-12-07T10:08:09.989 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /run 2023-12-07T10:08:09.989 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /mnt 2023-12-07T10:08:09.989 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /var/tmp 2023-12-07T10:08:09.989 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /home 2023-12-07T10:08:09.989 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /tmp 2023-12-07T10:08:09.989 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /dev 2023-12-07T10:08:09.990 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-12-07T10:08:10.345 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-volume-2:19.0.0-18.g3bd0bfa4.el8.noarch 97/134 2023-12-07T10:08:11.960 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-volume-2:19.0.0-18.g3bd0bfa4.el8.noarch 97/134 2023-12-07T10:08:12.129 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-osd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 98/134 2023-12-07T10:08:12.350 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-osd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 98/134 2023-12-07T10:08:12.518 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mds-2:19.0.0-18.g3bd0bfa4.el8.x86_64 99/134 2023-12-07T10:08:12.983 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mds-2:19.0.0-18.g3bd0bfa4.el8.x86_64 99/134 2023-12-07T10:08:13.151 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mon-2:19.0.0-18.g3bd0bfa4.el8.x86_64 100/134 2023-12-07T10:08:13.168 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mon-2:19.0.0-18.g3bd0bfa4.el8.x86_64 100/134 2023-12-07T10:08:15.046 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: cephadm-2:19.0.0-18.g3bd0bfa4.el8.noarch 101/134 2023-12-07T10:08:15.281 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephadm-2:19.0.0-18.g3bd0bfa4.el8.noarch 101/134 2023-12-07T10:08:15.417 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-18.g3bd0bfa4.el8 102/134 2023-12-07T10:08:15.609 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-18.g3bd0bfa4.el 103/134 2023-12-07T10:08:15.793 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-18.g3bd0bfa4.el8.noarc 104/134 2023-12-07T10:08:16.008 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-selinux-2:19.0.0-18.g3bd0bfa4.el8.x86_64 96/134 2023-12-07T10:08:16.008 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /sys 2023-12-07T10:08:16.008 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /proc 2023-12-07T10:08:16.008 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /dev 2023-12-07T10:08:16.008 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /run 2023-12-07T10:08:16.008 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /mnt 2023-12-07T10:08:16.008 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /var/tmp 2023-12-07T10:08:16.008 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /home 2023-12-07T10:08:16.008 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /tmp 2023-12-07T10:08:16.009 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /dev 2023-12-07T10:08:16.009 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-12-07T10:08:16.016 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-18.g3bd0bfa4.el8.noarc 104/134 2023-12-07T10:08:16.303 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-18.g3bd0bfa4.el8.noa 105/134 2023-12-07T10:08:16.348 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-volume-2:19.0.0-18.g3bd0bfa4.el8.noarch 97/134 2023-12-07T10:08:17.266 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-18.g3bd0bfa4.el8.noa 105/134 2023-12-07T10:08:17.434 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-18.g3bd0b 106/134 2023-12-07T10:08:17.524 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-18.g3bd0b 106/134 2023-12-07T10:08:17.914 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-18.g3bd0bfa4.el8. 107/134 2023-12-07T10:08:17.968 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-volume-2:19.0.0-18.g3bd0bfa4.el8.noarch 97/134 2023-12-07T10:08:18.135 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-osd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 98/134 2023-12-07T10:08:18.136 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-2:19.0.0-18.g3bd0bfa4.el8.x86_64 108/134 2023-12-07T10:08:18.172 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-2:19.0.0-18.g3bd0bfa4.el8.x86_64 108/134 2023-12-07T10:08:18.351 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-rook-2:19.0.0-18.g3bd0bfa4.el8.noarch 109/134 2023-12-07T10:08:18.355 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-osd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 98/134 2023-12-07T10:08:18.362 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-18.g3bd0bfa4.el8.noarch 109/134 2023-12-07T10:08:18.522 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mds-2:19.0.0-18.g3bd0bfa4.el8.x86_64 99/134 2023-12-07T10:08:18.990 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mds-2:19.0.0-18.g3bd0bfa4.el8.x86_64 99/134 2023-12-07T10:08:19.139 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mon-2:19.0.0-18.g3bd0bfa4.el8.x86_64 100/134 2023-12-07T10:08:19.156 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mon-2:19.0.0-18.g3bd0bfa4.el8.x86_64 100/134 2023-12-07T10:08:19.694 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-2:19.0.0-18.g3bd0bfa4.el8.x86_64 110/134 2023-12-07T10:08:19.880 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-radosgw-2:19.0.0-18.g3bd0bfa4.el8.x86_64 111/134 2023-12-07T10:08:19.915 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-18.g3bd0bfa4.el8.x86_64 111/134 2023-12-07T10:08:20.109 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-18.g3bd0bfa 112/134 2023-12-07T10:08:20.151 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-18.g3bd0bfa 112/134 2023-12-07T10:08:20.326 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-mirror-2:19.0.0-18.g3bd0bfa4.el8.x86_64 113/134 2023-12-07T10:08:20.684 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-18.g3bd0bfa4.el8.x86_64 113/134 2023-12-07T10:08:20.851 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-mirror-2:19.0.0-18.g3bd0bfa4.el8.x86_64 114/134 2023-12-07T10:08:21.166 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: cephadm-2:19.0.0-18.g3bd0bfa4.el8.noarch 101/134 2023-12-07T10:08:21.370 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephadm-2:19.0.0-18.g3bd0bfa4.el8.noarch 101/134 2023-12-07T10:08:21.487 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-18.g3bd0bfa4.el8 102/134 2023-12-07T10:08:21.687 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-18.g3bd0bfa4.el 103/134 2023-12-07T10:08:21.913 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-18.g3bd0bfa4.el8.noarc 104/134 2023-12-07T10:08:22.137 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-18.g3bd0bfa4.el8.noarc 104/134 2023-12-07T10:08:22.342 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-18.g3bd0bfa4.el8.noa 105/134 2023-12-07T10:08:23.305 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-18.g3bd0bfa4.el8.noa 105/134 2023-12-07T10:08:23.562 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-18.g3bd0b 106/134 2023-12-07T10:08:23.657 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-18.g3bd0b 106/134 2023-12-07T10:08:24.014 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-18.g3bd0bfa4.el8. 107/134 2023-12-07T10:08:24.209 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-2:19.0.0-18.g3bd0bfa4.el8.x86_64 108/134 2023-12-07T10:08:24.212 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-selinux-2:19.0.0-18.g3bd0bfa4.el8.x86_64 96/134 2023-12-07T10:08:24.212 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /sys 2023-12-07T10:08:24.212 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /proc 2023-12-07T10:08:24.212 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /dev 2023-12-07T10:08:24.212 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /run 2023-12-07T10:08:24.212 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /mnt 2023-12-07T10:08:24.212 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /var/tmp 2023-12-07T10:08:24.213 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /home 2023-12-07T10:08:24.213 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /tmp 2023-12-07T10:08:24.213 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /dev 2023-12-07T10:08:24.213 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:08:24.247 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-2:19.0.0-18.g3bd0bfa4.el8.x86_64 108/134 2023-12-07T10:08:24.438 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-rook-2:19.0.0-18.g3bd0bfa4.el8.noarch 109/134 2023-12-07T10:08:24.450 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-18.g3bd0bfa4.el8.noarch 109/134 2023-12-07T10:08:24.665 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-volume-2:19.0.0-18.g3bd0bfa4.el8.noarch 97/134 2023-12-07T10:08:25.518 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: rbd-mirror-2:19.0.0-18.g3bd0bfa4.el8.x86_64 114/134 2023-12-07T10:08:25.773 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-2:19.0.0-18.g3bd0bfa4.el8.x86_64 110/134 2023-12-07T10:08:26.002 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-radosgw-2:19.0.0-18.g3bd0bfa4.el8.x86_64 111/134 2023-12-07T10:08:26.037 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-18.g3bd0bfa4.el8.x86_64 111/134 2023-12-07T10:08:26.187 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-18.g3bd0bfa 112/134 2023-12-07T10:08:26.230 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-18.g3bd0bfa 112/134 2023-12-07T10:08:26.388 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-mirror-2:19.0.0-18.g3bd0bfa4.el8.x86_64 113/134 2023-12-07T10:08:26.396 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-test-2:19.0.0-18.g3bd0bfa4.el8.x86_64 115/134 2023-12-07T10:08:26.406 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-volume-2:19.0.0-18.g3bd0bfa4.el8.noarch 97/134 2023-12-07T10:08:26.604 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-osd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 98/134 2023-12-07T10:08:26.736 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-12-07T10:08:26.748 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-18.g3bd0bfa4.el8.x86_64 113/134 2023-12-07T10:08:26.870 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-osd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 98/134 2023-12-07T10:08:26.896 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-mirror-2:19.0.0-18.g3bd0bfa4.el8.x86_64 114/134 2023-12-07T10:08:26.927 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-fuse-2:19.0.0-18.g3bd0bfa4.el8.x86_64 117/134 2023-12-07T10:08:27.047 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mds-2:19.0.0-18.g3bd0bfa4.el8.x86_64 99/134 2023-12-07T10:08:27.191 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-12-07T10:08:27.382 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-12-07T10:08:27.575 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-12-07T10:08:27.598 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mds-2:19.0.0-18.g3bd0bfa4.el8.x86_64 99/134 2023-12-07T10:08:27.812 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-12-07T10:08:27.822 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mon-2:19.0.0-18.g3bd0bfa4.el8.x86_64 100/134 2023-12-07T10:08:27.864 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mon-2:19.0.0-18.g3bd0bfa4.el8.x86_64 100/134 2023-12-07T10:08:27.995 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs-devel-2:19.0.0-18.g3bd0bfa4.el8.x86_64 121/134 2023-12-07T10:08:28.159 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-12-07T10:08:30.434 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-12-07T10:08:30.682 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-12-07T10:08:30.700 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-12-07T10:08:30.784 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephadm-2:19.0.0-18.g3bd0bfa4.el8.noarch 101/134 2023-12-07T10:08:30.891 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-top-2:19.0.0-18.g3bd0bfa4.el8.noarch 124/134 2023-12-07T10:08:30.956 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephadm-2:19.0.0-18.g3bd0bfa4.el8.noarch 101/134 2023-12-07T10:08:31.106 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-18.g3bd0bfa4.el8 102/134 2023-12-07T10:08:31.210 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-fuse-2:19.0.0-18.g3bd0bfa4.el8.x86_64 125/134 2023-12-07T10:08:31.312 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-18.g3bd0bfa4.el 103/134 2023-12-07T10:08:31.527 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-nbd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 126/134 2023-12-07T10:08:31.555 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-18.g3bd0bfa4.el8.noarc 104/134 2023-12-07T10:08:31.571 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: rbd-mirror-2:19.0.0-18.g3bd0bfa4.el8.x86_64 114/134 2023-12-07T10:08:31.825 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-18.g3bd0bfa4.el8.noarc 104/134 2023-12-07T10:08:31.886 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-12-07T10:08:32.069 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-18.g3bd0bfa4.el8.noa 105/134 2023-12-07T10:08:32.073 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-12-07T10:08:32.156 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-12-07T10:08:32.390 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-12-07T10:08:32.402 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-12-07T10:08:32.464 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-test-2:19.0.0-18.g3bd0bfa4.el8.x86_64 115/134 2023-12-07T10:08:32.557 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-12-07T10:08:32.565 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-12-07T10:08:32.692 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-12-07T10:08:32.692 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-12-07T10:08:32.798 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-12-07T10:08:32.902 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-12-07T10:08:32.902 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-12-07T10:08:32.972 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-fuse-2:19.0.0-18.g3bd0bfa4.el8.x86_64 117/134 2023-12-07T10:08:33.145 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-12-07T10:08:33.153 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-18.g3bd0bfa4.el8.noa 105/134 2023-12-07T10:08:33.252 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-12-07T10:08:33.347 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-12-07T10:08:33.363 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-18.g3bd0b 106/134 2023-12-07T10:08:33.394 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-12-07T10:08:33.493 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-18.g3bd0b 106/134 2023-12-07T10:08:33.585 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-12-07T10:08:33.857 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-12-07T10:08:33.869 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-18.g3bd0bfa4.el8. 107/134 2023-12-07T10:08:33.922 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-12-07T10:08:33.923 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-2:19.0.0-18.g3bd0bfa4.el8.x86_64 1/134 2023-12-07T10:08:33.923 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-base-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2/134 2023-12-07T10:08:33.923 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-common-2:19.0.0-18.g3bd0bfa4.el8.x86_64 3/134 2023-12-07T10:08:33.923 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-fuse-2:19.0.0-18.g3bd0bfa4.el8.x86_64 4/134 2023-12-07T10:08:33.923 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-18.g3bd0bfa 5/134 2023-12-07T10:08:33.923 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mds-2:19.0.0-18.g3bd0bfa4.el8.x86_64 6/134 2023-12-07T10:08:33.923 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-2:19.0.0-18.g3bd0bfa4.el8.x86_64 7/134 2023-12-07T10:08:33.923 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mon-2:19.0.0-18.g3bd0bfa4.el8.x86_64 8/134 2023-12-07T10:08:33.923 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-osd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 9/134 2023-12-07T10:08:33.923 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-radosgw-2:19.0.0-18.g3bd0bfa4.el8.x86_64 10/134 2023-12-07T10:08:33.923 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-selinux-2:19.0.0-18.g3bd0bfa4.el8.x86_64 11/134 2023-12-07T10:08:33.923 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-test-2:19.0.0-18.g3bd0bfa4.el8.x86_64 12/134 2023-12-07T10:08:33.923 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-mirror-2:19.0.0-18.g3bd0bfa4.el8.x86_64 13/134 2023-12-07T10:08:33.924 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs-devel-2:19.0.0-18.g3bd0bfa4.el8.x86_64 14/134 2023-12-07T10:08:33.924 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 15/134 2023-12-07T10:08:33.924 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephsqlite-2:19.0.0-18.g3bd0bfa4.el8.x86_64 16/134 2023-12-07T10:08:33.924 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados-devel-2:19.0.0-18.g3bd0bfa4.el8.x86_64 17/134 2023-12-07T10:08:33.924 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libradosstriper1-2:19.0.0-18.g3bd0bfa4.el8.x86_6 18/134 2023-12-07T10:08:33.924 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librgw2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 19/134 2023-12-07T10:08:33.924 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-argparse-2:19.0.0-18.g3bd0bfa4.el8. 20/134 2023-12-07T10:08:33.924 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-common-2:19.0.0-18.g3bd0bfa4.el8.x8 21/134 2023-12-07T10:08:33.924 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cephfs-2:19.0.0-18.g3bd0bfa4.el8.x86_64 22/134 2023-12-07T10:08:33.924 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rados-2:19.0.0-18.g3bd0bfa4.el8.x86_64 23/134 2023-12-07T10:08:33.924 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rbd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 24/134 2023-12-07T10:08:33.925 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rgw-2:19.0.0-18.g3bd0bfa4.el8.x86_64 25/134 2023-12-07T10:08:33.925 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-fuse-2:19.0.0-18.g3bd0bfa4.el8.x86_64 26/134 2023-12-07T10:08:33.925 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-mirror-2:19.0.0-18.g3bd0bfa4.el8.x86_64 27/134 2023-12-07T10:08:33.925 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-nbd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 28/134 2023-12-07T10:08:33.925 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-18.g3bd0bfa4.el 29/134 2023-12-07T10:08:33.925 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-18.g3bd0bfa4.el8.noarc 30/134 2023-12-07T10:08:33.925 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-18.g3bd0bfa4.el8.noa 31/134 2023-12-07T10:08:33.925 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-18.g3bd0b 32/134 2023-12-07T10:08:33.925 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-18.g3bd0bfa4.el8. 33/134 2023-12-07T10:08:33.925 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-rook-2:19.0.0-18.g3bd0bfa4.el8.noarch 34/134 2023-12-07T10:08:33.926 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-18.g3bd0bfa4.el8 35/134 2023-12-07T10:08:33.926 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-volume-2:19.0.0-18.g3bd0bfa4.el8.noarch 36/134 2023-12-07T10:08:33.926 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephadm-2:19.0.0-18.g3bd0bfa4.el8.noarch 37/134 2023-12-07T10:08:33.926 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-top-2:19.0.0-18.g3bd0bfa4.el8.noarch 38/134 2023-12-07T10:08:33.926 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-12-07T10:08:33.926 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-12-07T10:08:33.926 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-12-07T10:08:33.926 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-12-07T10:08:33.926 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-12-07T10:08:33.926 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-12-07T10:08:33.926 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-12-07T10:08:33.926 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-12-07T10:08:33.927 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-12-07T10:08:33.927 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-12-07T10:08:33.927 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-12-07T10:08:33.927 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-12-07T10:08:33.927 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-12-07T10:08:33.927 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-12-07T10:08:33.927 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-12-07T10:08:33.927 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-12-07T10:08:33.927 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-12-07T10:08:33.927 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-12-07T10:08:33.927 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-12-07T10:08:33.928 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-12-07T10:08:33.928 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-12-07T10:08:33.928 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-12-07T10:08:33.928 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-12-07T10:08:33.928 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-12-07T10:08:33.928 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-12-07T10:08:33.928 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-12-07T10:08:33.928 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-12-07T10:08:33.928 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-12-07T10:08:33.928 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-12-07T10:08:33.928 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-12-07T10:08:33.928 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-12-07T10:08:33.928 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-12-07T10:08:33.928 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-12-07T10:08:33.928 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-12-07T10:08:33.929 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-12-07T10:08:33.929 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-12-07T10:08:33.929 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-12-07T10:08:33.929 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-12-07T10:08:33.929 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-12-07T10:08:33.929 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-12-07T10:08:33.929 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-12-07T10:08:33.929 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-12-07T10:08:33.929 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-12-07T10:08:33.929 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-12-07T10:08:33.929 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-12-07T10:08:33.929 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-12-07T10:08:33.929 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-12-07T10:08:33.929 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-12-07T10:08:33.930 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-12-07T10:08:33.930 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-12-07T10:08:33.930 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-12-07T10:08:33.930 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-12-07T10:08:33.930 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-12-07T10:08:33.930 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-12-07T10:08:33.930 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-12-07T10:08:33.930 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-12-07T10:08:33.930 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-12-07T10:08:33.930 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-12-07T10:08:33.930 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-12-07T10:08:33.930 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-12-07T10:08:33.930 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-12-07T10:08:33.930 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-12-07T10:08:33.931 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-12-07T10:08:33.931 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-12-07T10:08:33.931 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-12-07T10:08:33.931 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-12-07T10:08:33.931 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-12-07T10:08:33.931 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-12-07T10:08:33.931 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-12-07T10:08:33.931 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-12-07T10:08:33.931 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-12-07T10:08:33.931 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-12-07T10:08:33.931 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-12-07T10:08:33.931 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-12-07T10:08:33.931 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-12-07T10:08:33.932 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-12-07T10:08:33.932 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-12-07T10:08:33.932 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-12-07T10:08:33.932 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-12-07T10:08:33.932 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-12-07T10:08:33.932 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-12-07T10:08:33.932 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-12-07T10:08:33.932 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-12-07T10:08:33.932 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-12-07T10:08:33.932 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-12-07T10:08:33.932 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-12-07T10:08:33.932 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-12-07T10:08:33.932 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-12-07T10:08:33.932 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-12-07T10:08:33.932 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-12-07T10:08:33.933 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 129/134 2023-12-07T10:08:33.933 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-12-07T10:08:33.933 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-2:19.0.0-18.g3bd0bfa4.el8.x86_64 131/134 2023-12-07T10:08:33.933 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-12-07T10:08:33.933 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-12-07T10:08:34.015 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephfs-devel-2:19.0.0-18.g3bd0bfa4.el8.x86_64 121/134 2023-12-07T10:08:34.104 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-2:19.0.0-18.g3bd0bfa4.el8.x86_64 108/134 2023-12-07T10:08:34.145 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-12-07T10:08:34.171 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-2:19.0.0-18.g3bd0bfa4.el8.x86_64 108/134 2023-12-07T10:08:34.404 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-rook-2:19.0.0-18.g3bd0bfa4.el8.noarch 109/134 2023-12-07T10:08:34.433 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-18.g3bd0bfa4.el8.noarch 109/134 2023-12-07T10:08:35.849 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-2:19.0.0-18.g3bd0bfa4.el8.x86_64 110/134 2023-12-07T10:08:36.063 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-radosgw-2:19.0.0-18.g3bd0bfa4.el8.x86_64 111/134 2023-12-07T10:08:36.126 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-18.g3bd0bfa4.el8.x86_64 111/134 2023-12-07T10:08:36.296 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-18.g3bd0bfa 112/134 2023-12-07T10:08:36.369 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-18.g3bd0bfa 112/134 2023-12-07T10:08:36.394 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-12-07T10:08:36.571 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-mirror-2:19.0.0-18.g3bd0bfa4.el8.x86_64 113/134 2023-12-07T10:08:36.661 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-12-07T10:08:36.679 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-12-07T10:08:36.947 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-top-2:19.0.0-18.g3bd0bfa4.el8.noarch 124/134 2023-12-07T10:08:36.989 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-18.g3bd0bfa4.el8.x86_64 113/134 2023-12-07T10:08:37.162 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-mirror-2:19.0.0-18.g3bd0bfa4.el8.x86_64 114/134 2023-12-07T10:08:37.194 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-fuse-2:19.0.0-18.g3bd0bfa4.el8.x86_64 125/134 2023-12-07T10:08:37.533 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-nbd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 126/134 2023-12-07T10:08:37.882 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-12-07T10:08:38.060 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-12-07T10:08:38.139 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-12-07T10:08:38.385 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-12-07T10:08:38.398 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-12-07T10:08:38.552 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-12-07T10:08:38.560 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-12-07T10:08:38.704 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-12-07T10:08:38.704 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-12-07T10:08:38.875 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-12-07T10:08:38.875 INFO:teuthology.orchestra.run.smithi159.stdout:Installed products updated. 2023-12-07T10:08:38.896 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-12-07T10:08:38.896 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-12-07T10:08:39.091 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-12-07T10:08:39.091 INFO:teuthology.orchestra.run.smithi159.stdout:Upgraded: 2023-12-07T10:08:39.091 INFO:teuthology.orchestra.run.smithi159.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-12-07T10:08:39.091 INFO:teuthology.orchestra.run.smithi159.stdout: librados2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.092 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.092 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2023-12-07T10:08:39.092 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-12-07T10:08:39.092 INFO:teuthology.orchestra.run.smithi159.stdout: automake-1.16.1-7.el8.noarch 2023-12-07T10:08:39.092 INFO:teuthology.orchestra.run.smithi159.stdout: bison-3.0.4-10.el8.x86_64 2023-12-07T10:08:39.092 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.092 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.092 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.092 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.092 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-grafana-dashboards-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:39.092 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.092 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.092 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.092 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-cephadm-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:39.093 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-dashboard-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:39.093 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-diskprediction-local-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:39.093 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:39.093 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-rook-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:39.093 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.093 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.093 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-prometheus-alerts-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:39.093 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.093 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.093 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.093 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-volume-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:39.093 INFO:teuthology.orchestra.run.smithi159.stdout: cephadm-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:39.093 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-mirror-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.094 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-top-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:39.094 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-12-07T10:08:39.094 INFO:teuthology.orchestra.run.smithi159.stdout: flex-2.6.1-9.el8.x86_64 2023-12-07T10:08:39.094 INFO:teuthology.orchestra.run.smithi159.stdout: fuse-2.9.7-15.el8.x86_64 2023-12-07T10:08:39.094 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-07T10:08:39.094 INFO:teuthology.orchestra.run.smithi159.stdout: jq-1.6-3.el8.x86_64 2023-12-07T10:08:39.094 INFO:teuthology.orchestra.run.smithi159.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-12-07T10:08:39.094 INFO:teuthology.orchestra.run.smithi159.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-12-07T10:08:39.094 INFO:teuthology.orchestra.run.smithi159.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-12-07T10:08:39.094 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.094 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.094 INFO:teuthology.orchestra.run.smithi159.stdout: libcephsqlite-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.094 INFO:teuthology.orchestra.run.smithi159.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-12-07T10:08:39.094 INFO:teuthology.orchestra.run.smithi159.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-12-07T10:08:39.095 INFO:teuthology.orchestra.run.smithi159.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-07T10:08:39.095 INFO:teuthology.orchestra.run.smithi159.stdout: libpq-13.5-1.el8.x86_64 2023-12-07T10:08:39.095 INFO:teuthology.orchestra.run.smithi159.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-12-07T10:08:39.095 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-12-07T10:08:39.095 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.095 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.095 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-07T10:08:39.095 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.095 INFO:teuthology.orchestra.run.smithi159.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-12-07T10:08:39.095 INFO:teuthology.orchestra.run.smithi159.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-12-07T10:08:39.095 INFO:teuthology.orchestra.run.smithi159.stdout: libtool-2.4.6-25.el8.x86_64 2023-12-07T10:08:39.095 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-07T10:08:39.095 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-12-07T10:08:39.096 INFO:teuthology.orchestra.run.smithi159.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-12-07T10:08:39.096 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-07T10:08:39.096 INFO:teuthology.orchestra.run.smithi159.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-12-07T10:08:39.096 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-0.3.15-3.el8.x86_64 2023-12-07T10:08:39.096 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-12-07T10:08:39.096 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-12-07T10:08:39.096 INFO:teuthology.orchestra.run.smithi159.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-12-07T10:08:39.096 INFO:teuthology.orchestra.run.smithi159.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-12-07T10:08:39.096 INFO:teuthology.orchestra.run.smithi159.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-12-07T10:08:39.096 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-07T10:08:39.096 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-07T10:08:39.096 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-07T10:08:39.096 INFO:teuthology.orchestra.run.smithi159.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-12-07T10:08:39.096 INFO:teuthology.orchestra.run.smithi159.stdout: python3-babel-2.5.1-7.el8.noarch 2023-12-07T10:08:39.097 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-07T10:08:39.097 INFO:teuthology.orchestra.run.smithi159.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-07T10:08:39.097 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-07T10:08:39.097 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.097 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.097 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.097 INFO:teuthology.orchestra.run.smithi159.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-07T10:08:39.097 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-12-07T10:08:39.097 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-07T10:08:39.097 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-07T10:08:39.097 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-12-07T10:08:39.097 INFO:teuthology.orchestra.run.smithi159.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-07T10:08:39.097 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-07T10:08:39.097 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-07T10:08:39.098 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-12-07T10:08:39.098 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-07T10:08:39.098 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-07T10:08:39.098 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-07T10:08:39.098 INFO:teuthology.orchestra.run.smithi159.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-07T10:08:39.098 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-07T10:08:39.098 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako-1.0.6-13.el8.noarch 2023-12-07T10:08:39.098 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-07T10:08:39.098 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-07T10:08:39.098 INFO:teuthology.orchestra.run.smithi159.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-12-07T10:08:39.098 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-07T10:08:39.098 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-07T10:08:39.098 INFO:teuthology.orchestra.run.smithi159.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-07T10:08:39.098 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-07T10:08:39.099 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend-2.6-1.el8.noarch 2023-12-07T10:08:39.099 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-07T10:08:39.099 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-07T10:08:39.099 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-07T10:08:39.099 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-07T10:08:39.099 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-07T10:08:39.099 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz-2017.2-9.el8.noarch 2023-12-07T10:08:39.099 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.099 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.099 INFO:teuthology.orchestra.run.smithi159.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-07T10:08:39.099 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-07T10:08:39.099 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.100 INFO:teuthology.orchestra.run.smithi159.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-07T10:08:39.100 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-07T10:08:39.100 INFO:teuthology.orchestra.run.smithi159.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-12-07T10:08:39.100 INFO:teuthology.orchestra.run.smithi159.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-07T10:08:39.100 INFO:teuthology.orchestra.run.smithi159.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-07T10:08:39.100 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-07T10:08:39.100 INFO:teuthology.orchestra.run.smithi159.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-07T10:08:39.100 INFO:teuthology.orchestra.run.smithi159.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-07T10:08:39.100 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-07T10:08:39.100 INFO:teuthology.orchestra.run.smithi159.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-07T10:08:39.100 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-07T10:08:39.100 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-07T10:08:39.100 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-07T10:08:39.101 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.101 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.101 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:39.101 INFO:teuthology.orchestra.run.smithi159.stdout: socat-1.7.4.1-1.el8.x86_64 2023-12-07T10:08:39.101 INFO:teuthology.orchestra.run.smithi159.stdout: thrift-0.13.0-2.el8.x86_64 2023-12-07T10:08:39.101 INFO:teuthology.orchestra.run.smithi159.stdout: uuid-1.6.2-43.el8.x86_64 2023-12-07T10:08:39.101 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-12-07T10:08:39.101 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-12-07T10:08:39.101 INFO:teuthology.orchestra.run.smithi159.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-12-07T10:08:39.102 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-12-07T10:08:39.102 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2023-12-07T10:08:39.131 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-12-07T10:08:39.303 DEBUG:teuthology.parallel:result is None 2023-12-07T10:08:39.359 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-12-07T10:08:39.855 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-12-07T10:08:39.855 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-2:19.0.0-18.g3bd0bfa4.el8.x86_64 1/134 2023-12-07T10:08:39.856 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-base-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2/134 2023-12-07T10:08:39.856 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-common-2:19.0.0-18.g3bd0bfa4.el8.x86_64 3/134 2023-12-07T10:08:39.856 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-fuse-2:19.0.0-18.g3bd0bfa4.el8.x86_64 4/134 2023-12-07T10:08:39.856 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-18.g3bd0bfa 5/134 2023-12-07T10:08:39.856 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mds-2:19.0.0-18.g3bd0bfa4.el8.x86_64 6/134 2023-12-07T10:08:39.856 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-2:19.0.0-18.g3bd0bfa4.el8.x86_64 7/134 2023-12-07T10:08:39.856 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mon-2:19.0.0-18.g3bd0bfa4.el8.x86_64 8/134 2023-12-07T10:08:39.856 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-osd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 9/134 2023-12-07T10:08:39.856 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-radosgw-2:19.0.0-18.g3bd0bfa4.el8.x86_64 10/134 2023-12-07T10:08:39.856 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-selinux-2:19.0.0-18.g3bd0bfa4.el8.x86_64 11/134 2023-12-07T10:08:39.856 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-test-2:19.0.0-18.g3bd0bfa4.el8.x86_64 12/134 2023-12-07T10:08:39.856 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-mirror-2:19.0.0-18.g3bd0bfa4.el8.x86_64 13/134 2023-12-07T10:08:39.856 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephfs-devel-2:19.0.0-18.g3bd0bfa4.el8.x86_64 14/134 2023-12-07T10:08:39.857 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephfs2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 15/134 2023-12-07T10:08:39.857 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephsqlite-2:19.0.0-18.g3bd0bfa4.el8.x86_64 16/134 2023-12-07T10:08:39.857 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados-devel-2:19.0.0-18.g3bd0bfa4.el8.x86_64 17/134 2023-12-07T10:08:39.857 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libradosstriper1-2:19.0.0-18.g3bd0bfa4.el8.x86_6 18/134 2023-12-07T10:08:39.857 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librgw2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 19/134 2023-12-07T10:08:39.857 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ceph-argparse-2:19.0.0-18.g3bd0bfa4.el8. 20/134 2023-12-07T10:08:39.857 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ceph-common-2:19.0.0-18.g3bd0bfa4.el8.x8 21/134 2023-12-07T10:08:39.857 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cephfs-2:19.0.0-18.g3bd0bfa4.el8.x86_64 22/134 2023-12-07T10:08:39.857 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rados-2:19.0.0-18.g3bd0bfa4.el8.x86_64 23/134 2023-12-07T10:08:39.857 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rbd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 24/134 2023-12-07T10:08:39.857 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rgw-2:19.0.0-18.g3bd0bfa4.el8.x86_64 25/134 2023-12-07T10:08:39.857 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-fuse-2:19.0.0-18.g3bd0bfa4.el8.x86_64 26/134 2023-12-07T10:08:39.857 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-mirror-2:19.0.0-18.g3bd0bfa4.el8.x86_64 27/134 2023-12-07T10:08:39.857 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-nbd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 28/134 2023-12-07T10:08:39.858 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-18.g3bd0bfa4.el 29/134 2023-12-07T10:08:39.858 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-18.g3bd0bfa4.el8.noarc 30/134 2023-12-07T10:08:39.858 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-18.g3bd0bfa4.el8.noa 31/134 2023-12-07T10:08:39.858 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-18.g3bd0b 32/134 2023-12-07T10:08:39.866 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-18.g3bd0bfa4.el8. 33/134 2023-12-07T10:08:39.866 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-rook-2:19.0.0-18.g3bd0bfa4.el8.noarch 34/134 2023-12-07T10:08:39.866 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-18.g3bd0bfa4.el8 35/134 2023-12-07T10:08:39.866 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-volume-2:19.0.0-18.g3bd0bfa4.el8.noarch 36/134 2023-12-07T10:08:39.867 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephadm-2:19.0.0-18.g3bd0bfa4.el8.noarch 37/134 2023-12-07T10:08:39.867 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-top-2:19.0.0-18.g3bd0bfa4.el8.noarch 38/134 2023-12-07T10:08:39.867 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-12-07T10:08:39.867 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-12-07T10:08:39.867 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-12-07T10:08:39.867 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-12-07T10:08:39.867 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-12-07T10:08:39.867 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-12-07T10:08:39.867 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-12-07T10:08:39.867 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-12-07T10:08:39.867 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-12-07T10:08:39.867 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-12-07T10:08:39.867 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-12-07T10:08:39.868 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-12-07T10:08:39.868 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-12-07T10:08:39.868 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-12-07T10:08:39.868 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-12-07T10:08:39.868 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-12-07T10:08:39.868 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-12-07T10:08:39.868 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-12-07T10:08:39.868 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-12-07T10:08:39.868 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-12-07T10:08:39.868 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-12-07T10:08:39.868 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-12-07T10:08:39.868 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-12-07T10:08:39.868 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-12-07T10:08:39.868 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-12-07T10:08:39.869 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-12-07T10:08:39.869 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-12-07T10:08:39.869 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-12-07T10:08:39.869 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-12-07T10:08:39.869 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-12-07T10:08:39.869 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-12-07T10:08:39.869 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-12-07T10:08:39.869 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-12-07T10:08:39.869 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-12-07T10:08:39.869 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-12-07T10:08:39.869 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-12-07T10:08:39.869 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-12-07T10:08:39.869 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-12-07T10:08:39.870 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-12-07T10:08:39.870 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-12-07T10:08:39.870 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-12-07T10:08:39.870 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-12-07T10:08:39.870 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-12-07T10:08:39.870 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-12-07T10:08:39.870 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-12-07T10:08:39.870 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-12-07T10:08:39.870 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-12-07T10:08:39.870 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-12-07T10:08:39.870 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-12-07T10:08:39.870 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-12-07T10:08:39.870 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-12-07T10:08:39.870 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-12-07T10:08:39.871 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-12-07T10:08:39.871 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-12-07T10:08:39.871 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-12-07T10:08:39.871 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-12-07T10:08:39.871 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-12-07T10:08:39.871 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-12-07T10:08:39.871 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-12-07T10:08:39.871 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-12-07T10:08:39.871 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-12-07T10:08:39.871 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-12-07T10:08:39.871 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-12-07T10:08:39.871 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-12-07T10:08:39.871 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-12-07T10:08:39.871 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-12-07T10:08:39.872 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-12-07T10:08:39.872 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-12-07T10:08:39.872 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-12-07T10:08:39.872 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-12-07T10:08:39.872 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-12-07T10:08:39.872 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-12-07T10:08:39.872 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-12-07T10:08:39.872 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-12-07T10:08:39.872 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-12-07T10:08:39.872 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-12-07T10:08:39.872 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-12-07T10:08:39.872 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-12-07T10:08:39.872 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-12-07T10:08:39.872 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-12-07T10:08:39.872 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-12-07T10:08:39.873 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-12-07T10:08:39.873 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-12-07T10:08:39.873 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-12-07T10:08:39.873 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-12-07T10:08:39.873 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-12-07T10:08:39.873 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-12-07T10:08:39.873 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-12-07T10:08:39.873 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-12-07T10:08:39.873 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-12-07T10:08:39.873 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 129/134 2023-12-07T10:08:39.873 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-12-07T10:08:39.873 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librbd1-2:19.0.0-18.g3bd0bfa4.el8.x86_64 131/134 2023-12-07T10:08:39.873 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-12-07T10:08:39.873 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-12-07T10:08:42.230 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: rbd-mirror-2:19.0.0-18.g3bd0bfa4.el8.x86_64 114/134 2023-12-07T10:08:42.575 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-test-2:19.0.0-18.g3bd0bfa4.el8.x86_64 115/134 2023-12-07T10:08:42.778 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-12-07T10:08:43.030 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-12-07T10:08:44.314 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-12-07T10:08:44.679 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-12-07T10:08:44.884 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-fuse-2:19.0.0-18.g3bd0bfa4.el8.x86_64 119/134 2023-12-07T10:08:44.936 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-12-07T10:08:44.936 INFO:teuthology.orchestra.run.smithi153.stdout:Installed products updated. 2023-12-07T10:08:45.133 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-12-07T10:08:45.148 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-12-07T10:08:45.148 INFO:teuthology.orchestra.run.smithi153.stdout:Upgraded: 2023-12-07T10:08:45.148 INFO:teuthology.orchestra.run.smithi153.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-12-07T10:08:45.148 INFO:teuthology.orchestra.run.smithi153.stdout: librados2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.148 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.148 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2023-12-07T10:08:45.148 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-12-07T10:08:45.149 INFO:teuthology.orchestra.run.smithi153.stdout: automake-1.16.1-7.el8.noarch 2023-12-07T10:08:45.149 INFO:teuthology.orchestra.run.smithi153.stdout: bison-3.0.4-10.el8.x86_64 2023-12-07T10:08:45.149 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.149 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.149 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.149 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.149 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:45.149 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-immutable-object-cache-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.149 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.149 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.149 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-cephadm-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:45.149 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:45.149 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:45.149 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-modules-core-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:45.150 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:45.150 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.150 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.150 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-prometheus-alerts-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:45.150 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.150 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.150 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.150 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-volume-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:45.150 INFO:teuthology.orchestra.run.smithi153.stdout: cephadm-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:45.150 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-mirror-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.150 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-top-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:45.150 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-12-07T10:08:45.150 INFO:teuthology.orchestra.run.smithi153.stdout: flex-2.6.1-9.el8.x86_64 2023-12-07T10:08:45.150 INFO:teuthology.orchestra.run.smithi153.stdout: fuse-2.9.7-15.el8.x86_64 2023-12-07T10:08:45.151 INFO:teuthology.orchestra.run.smithi153.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-07T10:08:45.151 INFO:teuthology.orchestra.run.smithi153.stdout: jq-1.6-3.el8.x86_64 2023-12-07T10:08:45.151 INFO:teuthology.orchestra.run.smithi153.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-12-07T10:08:45.151 INFO:teuthology.orchestra.run.smithi153.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-12-07T10:08:45.151 INFO:teuthology.orchestra.run.smithi153.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-12-07T10:08:45.151 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.151 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.151 INFO:teuthology.orchestra.run.smithi153.stdout: libcephsqlite-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.151 INFO:teuthology.orchestra.run.smithi153.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-12-07T10:08:45.151 INFO:teuthology.orchestra.run.smithi153.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-12-07T10:08:45.151 INFO:teuthology.orchestra.run.smithi153.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-07T10:08:45.151 INFO:teuthology.orchestra.run.smithi153.stdout: libpq-13.5-1.el8.x86_64 2023-12-07T10:08:45.151 INFO:teuthology.orchestra.run.smithi153.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-12-07T10:08:45.152 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-12-07T10:08:45.152 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.152 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.152 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-07T10:08:45.152 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.152 INFO:teuthology.orchestra.run.smithi153.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-12-07T10:08:45.152 INFO:teuthology.orchestra.run.smithi153.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-12-07T10:08:45.152 INFO:teuthology.orchestra.run.smithi153.stdout: libtool-2.4.6-25.el8.x86_64 2023-12-07T10:08:45.152 INFO:teuthology.orchestra.run.smithi153.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-07T10:08:45.152 INFO:teuthology.orchestra.run.smithi153.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-12-07T10:08:45.152 INFO:teuthology.orchestra.run.smithi153.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-12-07T10:08:45.152 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-07T10:08:45.152 INFO:teuthology.orchestra.run.smithi153.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-12-07T10:08:45.152 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-0.3.15-3.el8.x86_64 2023-12-07T10:08:45.153 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-12-07T10:08:45.153 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-12-07T10:08:45.153 INFO:teuthology.orchestra.run.smithi153.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-12-07T10:08:45.153 INFO:teuthology.orchestra.run.smithi153.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-12-07T10:08:45.153 INFO:teuthology.orchestra.run.smithi153.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-12-07T10:08:45.153 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-07T10:08:45.153 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-07T10:08:45.153 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-07T10:08:45.153 INFO:teuthology.orchestra.run.smithi153.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-12-07T10:08:45.153 INFO:teuthology.orchestra.run.smithi153.stdout: python3-babel-2.5.1-7.el8.noarch 2023-12-07T10:08:45.153 INFO:teuthology.orchestra.run.smithi153.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-07T10:08:45.153 INFO:teuthology.orchestra.run.smithi153.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-07T10:08:45.153 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-07T10:08:45.153 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-argparse-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.154 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-common-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.154 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cephfs-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.154 INFO:teuthology.orchestra.run.smithi153.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-07T10:08:45.154 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-12-07T10:08:45.154 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-07T10:08:45.154 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-07T10:08:45.154 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-12-07T10:08:45.154 INFO:teuthology.orchestra.run.smithi153.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-07T10:08:45.154 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-07T10:08:45.154 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-07T10:08:45.154 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-12-07T10:08:45.154 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-07T10:08:45.154 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-07T10:08:45.154 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-07T10:08:45.155 INFO:teuthology.orchestra.run.smithi153.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-07T10:08:45.155 INFO:teuthology.orchestra.run.smithi153.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-07T10:08:45.155 INFO:teuthology.orchestra.run.smithi153.stdout: python3-mako-1.0.6-13.el8.noarch 2023-12-07T10:08:45.155 INFO:teuthology.orchestra.run.smithi153.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-07T10:08:45.155 INFO:teuthology.orchestra.run.smithi153.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-07T10:08:45.155 INFO:teuthology.orchestra.run.smithi153.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-12-07T10:08:45.155 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-07T10:08:45.155 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-07T10:08:45.155 INFO:teuthology.orchestra.run.smithi153.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-07T10:08:45.155 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-07T10:08:45.155 INFO:teuthology.orchestra.run.smithi153.stdout: python3-portend-2.6-1.el8.noarch 2023-12-07T10:08:45.155 INFO:teuthology.orchestra.run.smithi153.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-07T10:08:45.155 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-07T10:08:45.155 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-07T10:08:45.156 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-07T10:08:45.156 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-07T10:08:45.156 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytz-2017.2-9.el8.noarch 2023-12-07T10:08:45.156 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rados-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.156 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rbd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.156 INFO:teuthology.orchestra.run.smithi153.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-07T10:08:45.156 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-07T10:08:45.156 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rgw-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.156 INFO:teuthology.orchestra.run.smithi153.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-07T10:08:45.156 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-07T10:08:45.156 INFO:teuthology.orchestra.run.smithi153.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-12-07T10:08:45.156 INFO:teuthology.orchestra.run.smithi153.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-07T10:08:45.156 INFO:teuthology.orchestra.run.smithi153.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-07T10:08:45.156 INFO:teuthology.orchestra.run.smithi153.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-07T10:08:45.157 INFO:teuthology.orchestra.run.smithi153.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-07T10:08:45.157 INFO:teuthology.orchestra.run.smithi153.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-07T10:08:45.157 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-07T10:08:45.157 INFO:teuthology.orchestra.run.smithi153.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-07T10:08:45.157 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-07T10:08:45.157 INFO:teuthology.orchestra.run.smithi153.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-07T10:08:45.157 INFO:teuthology.orchestra.run.smithi153.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-07T10:08:45.157 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.157 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-mirror-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.157 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-nbd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:45.157 INFO:teuthology.orchestra.run.smithi153.stdout: socat-1.7.4.1-1.el8.x86_64 2023-12-07T10:08:45.157 INFO:teuthology.orchestra.run.smithi153.stdout: thrift-0.13.0-2.el8.x86_64 2023-12-07T10:08:45.157 INFO:teuthology.orchestra.run.smithi153.stdout: uuid-1.6.2-43.el8.x86_64 2023-12-07T10:08:45.157 INFO:teuthology.orchestra.run.smithi153.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-12-07T10:08:45.158 INFO:teuthology.orchestra.run.smithi153.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-12-07T10:08:45.158 INFO:teuthology.orchestra.run.smithi153.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-12-07T10:08:45.158 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-12-07T10:08:45.158 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2023-12-07T10:08:45.333 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs-devel-2:19.0.0-18.g3bd0bfa4.el8.x86_64 121/134 2023-12-07T10:08:45.358 DEBUG:teuthology.parallel:result is None 2023-12-07T10:08:45.512 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-12-07T10:08:48.915 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-12-07T10:08:49.202 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-12-07T10:08:49.240 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-12-07T10:08:49.519 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-top-2:19.0.0-18.g3bd0bfa4.el8.noarch 124/134 2023-12-07T10:08:49.778 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-fuse-2:19.0.0-18.g3bd0bfa4.el8.x86_64 125/134 2023-12-07T10:08:50.122 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-nbd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 126/134 2023-12-07T10:08:50.504 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-12-07T10:08:50.693 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-12-07T10:08:50.717 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-12-07T10:08:50.988 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-12-07T10:08:51.168 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-12-07T10:08:51.285 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-12-07T10:08:51.501 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-12-07T10:08:51.531 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-12-07T10:08:51.531 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-12-07T10:08:51.783 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-12-07T10:08:51.783 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-12-07T10:08:52.091 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-12-07T10:08:52.318 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-12-07T10:08:52.594 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-12-07T10:08:52.594 INFO:teuthology.orchestra.run.smithi119.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-12-07T10:08:52.594 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:08:53.221 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-2:19.0.0-18.g3bd0bfa4.el8.x86_64 1/134 2023-12-07T10:08:53.221 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-base-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2/134 2023-12-07T10:08:53.221 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-common-2:19.0.0-18.g3bd0bfa4.el8.x86_64 3/134 2023-12-07T10:08:53.221 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-fuse-2:19.0.0-18.g3bd0bfa4.el8.x86_64 4/134 2023-12-07T10:08:53.221 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-18.g3bd0bfa 5/134 2023-12-07T10:08:53.222 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mds-2:19.0.0-18.g3bd0bfa4.el8.x86_64 6/134 2023-12-07T10:08:53.222 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-2:19.0.0-18.g3bd0bfa4.el8.x86_64 7/134 2023-12-07T10:08:53.222 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mon-2:19.0.0-18.g3bd0bfa4.el8.x86_64 8/134 2023-12-07T10:08:53.222 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-osd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 9/134 2023-12-07T10:08:53.222 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-radosgw-2:19.0.0-18.g3bd0bfa4.el8.x86_64 10/134 2023-12-07T10:08:53.222 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-selinux-2:19.0.0-18.g3bd0bfa4.el8.x86_64 11/134 2023-12-07T10:08:53.222 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-test-2:19.0.0-18.g3bd0bfa4.el8.x86_64 12/134 2023-12-07T10:08:53.222 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-mirror-2:19.0.0-18.g3bd0bfa4.el8.x86_64 13/134 2023-12-07T10:08:53.222 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs-devel-2:19.0.0-18.g3bd0bfa4.el8.x86_64 14/134 2023-12-07T10:08:53.222 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 15/134 2023-12-07T10:08:53.222 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephsqlite-2:19.0.0-18.g3bd0bfa4.el8.x86_64 16/134 2023-12-07T10:08:53.222 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados-devel-2:19.0.0-18.g3bd0bfa4.el8.x86_64 17/134 2023-12-07T10:08:53.222 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libradosstriper1-2:19.0.0-18.g3bd0bfa4.el8.x86_6 18/134 2023-12-07T10:08:53.222 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librgw2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 19/134 2023-12-07T10:08:53.222 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-argparse-2:19.0.0-18.g3bd0bfa4.el8. 20/134 2023-12-07T10:08:53.223 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-common-2:19.0.0-18.g3bd0bfa4.el8.x8 21/134 2023-12-07T10:08:53.223 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cephfs-2:19.0.0-18.g3bd0bfa4.el8.x86_64 22/134 2023-12-07T10:08:53.223 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rados-2:19.0.0-18.g3bd0bfa4.el8.x86_64 23/134 2023-12-07T10:08:53.223 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rbd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 24/134 2023-12-07T10:08:53.223 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rgw-2:19.0.0-18.g3bd0bfa4.el8.x86_64 25/134 2023-12-07T10:08:53.223 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-fuse-2:19.0.0-18.g3bd0bfa4.el8.x86_64 26/134 2023-12-07T10:08:53.223 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-mirror-2:19.0.0-18.g3bd0bfa4.el8.x86_64 27/134 2023-12-07T10:08:53.232 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-nbd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 28/134 2023-12-07T10:08:53.232 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-18.g3bd0bfa4.el 29/134 2023-12-07T10:08:53.232 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-18.g3bd0bfa4.el8.noarc 30/134 2023-12-07T10:08:53.232 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-18.g3bd0bfa4.el8.noa 31/134 2023-12-07T10:08:53.233 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-18.g3bd0b 32/134 2023-12-07T10:08:53.233 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-18.g3bd0bfa4.el8. 33/134 2023-12-07T10:08:53.233 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-rook-2:19.0.0-18.g3bd0bfa4.el8.noarch 34/134 2023-12-07T10:08:53.233 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-18.g3bd0bfa4.el8 35/134 2023-12-07T10:08:53.233 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-volume-2:19.0.0-18.g3bd0bfa4.el8.noarch 36/134 2023-12-07T10:08:53.233 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephadm-2:19.0.0-18.g3bd0bfa4.el8.noarch 37/134 2023-12-07T10:08:53.233 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-top-2:19.0.0-18.g3bd0bfa4.el8.noarch 38/134 2023-12-07T10:08:53.233 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-12-07T10:08:53.233 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-12-07T10:08:53.233 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-12-07T10:08:53.233 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-12-07T10:08:53.233 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-12-07T10:08:53.233 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-12-07T10:08:53.233 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-12-07T10:08:53.233 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-12-07T10:08:53.234 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-12-07T10:08:53.234 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-12-07T10:08:53.234 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-12-07T10:08:53.234 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-12-07T10:08:53.234 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-12-07T10:08:53.234 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-12-07T10:08:53.234 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-12-07T10:08:53.234 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-12-07T10:08:53.234 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-12-07T10:08:53.234 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-12-07T10:08:53.234 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-12-07T10:08:53.234 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-12-07T10:08:53.234 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-12-07T10:08:53.234 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-12-07T10:08:53.235 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-12-07T10:08:53.235 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-12-07T10:08:53.235 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-12-07T10:08:53.235 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-12-07T10:08:53.235 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-12-07T10:08:53.235 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-12-07T10:08:53.235 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-12-07T10:08:53.235 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-12-07T10:08:53.235 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-12-07T10:08:53.235 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-12-07T10:08:53.235 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-12-07T10:08:53.235 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-12-07T10:08:53.235 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-12-07T10:08:53.235 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-12-07T10:08:53.235 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-12-07T10:08:53.236 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-12-07T10:08:53.236 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-12-07T10:08:53.236 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-12-07T10:08:53.236 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-12-07T10:08:53.236 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-12-07T10:08:53.236 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-12-07T10:08:53.236 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-12-07T10:08:53.236 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-12-07T10:08:53.236 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-12-07T10:08:53.236 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-12-07T10:08:53.236 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-12-07T10:08:53.236 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-12-07T10:08:53.236 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-12-07T10:08:53.236 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-12-07T10:08:53.237 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-12-07T10:08:53.237 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-12-07T10:08:53.237 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-12-07T10:08:53.237 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-12-07T10:08:53.237 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-12-07T10:08:53.237 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-12-07T10:08:53.237 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-12-07T10:08:53.237 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-12-07T10:08:53.237 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-12-07T10:08:53.237 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-12-07T10:08:53.237 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-12-07T10:08:53.237 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-12-07T10:08:53.237 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-12-07T10:08:53.237 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-12-07T10:08:53.237 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-12-07T10:08:53.237 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-12-07T10:08:53.238 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-12-07T10:08:53.238 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-12-07T10:08:53.238 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-12-07T10:08:53.238 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-12-07T10:08:53.238 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-12-07T10:08:53.238 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-12-07T10:08:53.238 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-12-07T10:08:53.238 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-12-07T10:08:53.238 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-12-07T10:08:53.238 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-12-07T10:08:53.238 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-12-07T10:08:53.238 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-12-07T10:08:53.238 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-12-07T10:08:53.238 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-12-07T10:08:53.238 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-12-07T10:08:53.239 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-12-07T10:08:53.239 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-12-07T10:08:53.239 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-12-07T10:08:53.239 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-12-07T10:08:53.239 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-12-07T10:08:53.239 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-12-07T10:08:53.239 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-12-07T10:08:53.239 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-12-07T10:08:53.239 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 129/134 2023-12-07T10:08:53.239 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-12-07T10:08:53.239 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:19.0.0-18.g3bd0bfa4.el8.x86_64 131/134 2023-12-07T10:08:53.239 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-12-07T10:08:53.239 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-12-07T10:08:58.735 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-12-07T10:08:58.736 INFO:teuthology.orchestra.run.smithi119.stdout:Installed products updated. 2023-12-07T10:08:58.974 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:08:58.974 INFO:teuthology.orchestra.run.smithi119.stdout:Upgraded: 2023-12-07T10:08:58.974 INFO:teuthology.orchestra.run.smithi119.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-12-07T10:08:58.974 INFO:teuthology.orchestra.run.smithi119.stdout: librados2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.974 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.974 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2023-12-07T10:08:58.974 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-12-07T10:08:58.974 INFO:teuthology.orchestra.run.smithi119.stdout: automake-1.16.1-7.el8.noarch 2023-12-07T10:08:58.974 INFO:teuthology.orchestra.run.smithi119.stdout: bison-3.0.4-10.el8.x86_64 2023-12-07T10:08:58.974 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.974 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.975 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.975 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.975 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:58.975 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.975 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.975 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.975 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:58.975 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:58.975 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:58.975 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:58.975 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:58.975 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.975 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.976 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:58.976 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.976 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.976 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.976 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:58.976 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:58.976 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-mirror-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.976 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-top-2:19.0.0-18.g3bd0bfa4.el8.noarch 2023-12-07T10:08:58.976 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-12-07T10:08:58.976 INFO:teuthology.orchestra.run.smithi119.stdout: flex-2.6.1-9.el8.x86_64 2023-12-07T10:08:58.976 INFO:teuthology.orchestra.run.smithi119.stdout: fuse-2.9.7-15.el8.x86_64 2023-12-07T10:08:58.976 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-07T10:08:58.976 INFO:teuthology.orchestra.run.smithi119.stdout: jq-1.6-3.el8.x86_64 2023-12-07T10:08:58.976 INFO:teuthology.orchestra.run.smithi119.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-12-07T10:08:58.977 INFO:teuthology.orchestra.run.smithi119.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-12-07T10:08:58.977 INFO:teuthology.orchestra.run.smithi119.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-12-07T10:08:58.977 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.977 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.977 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.977 INFO:teuthology.orchestra.run.smithi119.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-12-07T10:08:58.977 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-12-07T10:08:58.977 INFO:teuthology.orchestra.run.smithi119.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-07T10:08:58.977 INFO:teuthology.orchestra.run.smithi119.stdout: libpq-13.5-1.el8.x86_64 2023-12-07T10:08:58.977 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-12-07T10:08:58.977 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-12-07T10:08:58.977 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.977 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.977 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-07T10:08:58.977 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.978 INFO:teuthology.orchestra.run.smithi119.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-12-07T10:08:58.978 INFO:teuthology.orchestra.run.smithi119.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-12-07T10:08:58.978 INFO:teuthology.orchestra.run.smithi119.stdout: libtool-2.4.6-25.el8.x86_64 2023-12-07T10:08:58.978 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-07T10:08:58.978 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-12-07T10:08:58.978 INFO:teuthology.orchestra.run.smithi119.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-12-07T10:08:58.978 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-07T10:08:58.978 INFO:teuthology.orchestra.run.smithi119.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-12-07T10:08:58.978 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-0.3.15-3.el8.x86_64 2023-12-07T10:08:58.978 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-12-07T10:08:58.978 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-12-07T10:08:58.978 INFO:teuthology.orchestra.run.smithi119.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-12-07T10:08:58.978 INFO:teuthology.orchestra.run.smithi119.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-12-07T10:08:58.979 INFO:teuthology.orchestra.run.smithi119.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-12-07T10:08:58.979 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-07T10:08:58.979 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-07T10:08:58.979 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-07T10:08:58.979 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-12-07T10:08:58.979 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel-2.5.1-7.el8.noarch 2023-12-07T10:08:58.979 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-07T10:08:58.979 INFO:teuthology.orchestra.run.smithi119.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-07T10:08:58.979 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-07T10:08:58.979 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.979 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.979 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.979 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-07T10:08:58.979 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-12-07T10:08:58.979 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-07T10:08:58.980 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-07T10:08:58.980 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-12-07T10:08:58.980 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-07T10:08:58.980 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-07T10:08:58.980 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-07T10:08:58.980 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-12-07T10:08:58.980 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-07T10:08:58.980 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-07T10:08:58.980 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-07T10:08:58.980 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-07T10:08:58.980 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-07T10:08:58.980 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako-1.0.6-13.el8.noarch 2023-12-07T10:08:58.980 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-07T10:08:58.980 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-07T10:08:58.981 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-12-07T10:08:58.981 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-07T10:08:58.981 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-07T10:08:58.981 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-07T10:08:58.981 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-07T10:08:58.981 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend-2.6-1.el8.noarch 2023-12-07T10:08:58.981 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-07T10:08:58.981 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-07T10:08:58.981 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-07T10:08:58.981 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-07T10:08:58.981 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-07T10:08:58.981 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz-2017.2-9.el8.noarch 2023-12-07T10:08:58.981 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.982 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.982 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-07T10:08:58.982 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-07T10:08:58.982 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.982 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-07T10:08:58.982 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-07T10:08:58.982 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-12-07T10:08:58.982 INFO:teuthology.orchestra.run.smithi119.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-07T10:08:58.982 INFO:teuthology.orchestra.run.smithi119.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-07T10:08:58.982 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-07T10:08:58.982 INFO:teuthology.orchestra.run.smithi119.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-07T10:08:58.982 INFO:teuthology.orchestra.run.smithi119.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-07T10:08:58.982 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-07T10:08:58.982 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-07T10:08:58.983 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-07T10:08:58.983 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-07T10:08:58.983 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-07T10:08:58.983 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.983 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.983 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd-2:19.0.0-18.g3bd0bfa4.el8.x86_64 2023-12-07T10:08:58.983 INFO:teuthology.orchestra.run.smithi119.stdout: socat-1.7.4.1-1.el8.x86_64 2023-12-07T10:08:58.983 INFO:teuthology.orchestra.run.smithi119.stdout: thrift-0.13.0-2.el8.x86_64 2023-12-07T10:08:58.983 INFO:teuthology.orchestra.run.smithi119.stdout: uuid-1.6.2-43.el8.x86_64 2023-12-07T10:08:58.983 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-12-07T10:08:58.983 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-12-07T10:08:58.984 INFO:teuthology.orchestra.run.smithi119.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-12-07T10:08:58.984 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:08:58.984 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-12-07T10:08:59.332 DEBUG:teuthology.parallel:result is None 2023-12-07T10:08:59.332 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 2023-12-07T10:08:59.476 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-07T10:08:59.533 INFO:teuthology.orchestra.run.smithi119.stdout:19.0.0-18.g3bd0bfa4.el8 2023-12-07T10:08:59.534 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-18.g3bd0bfa4.el8 2023-12-07T10:08:59.534 INFO:teuthology.task.install:The correct ceph version 19.0.0-18.g3bd0bfa4 is installed. 2023-12-07T10:08:59.534 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 2023-12-07T10:08:59.670 DEBUG:teuthology.orchestra.run.smithi153:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-07T10:08:59.699 INFO:teuthology.orchestra.run.smithi153.stdout:19.0.0-18.g3bd0bfa4.el8 2023-12-07T10:08:59.699 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-18.g3bd0bfa4.el8 2023-12-07T10:08:59.699 INFO:teuthology.task.install:The correct ceph version 19.0.0-18.g3bd0bfa4 is installed. 2023-12-07T10:08:59.699 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 2023-12-07T10:08:59.835 DEBUG:teuthology.orchestra.run.smithi159:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-07T10:08:59.863 INFO:teuthology.orchestra.run.smithi159.stdout:19.0.0-18.g3bd0bfa4.el8 2023-12-07T10:08:59.863 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-18.g3bd0bfa4.el8 2023-12-07T10:08:59.863 INFO:teuthology.task.install:The correct ceph version 19.0.0-18.g3bd0bfa4 is installed. 2023-12-07T10:08:59.864 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-12-07T10:08:59.864 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-12-07T10:08:59.864 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-07T10:08:59.953 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-12-07T10:08:59.954 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-07T10:09:00.002 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-12-07T10:09:00.002 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-07T10:09:00.049 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-12-07T10:09:00.049 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-12-07T10:09:00.050 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/daemon-helper 2023-12-07T10:09:00.119 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-07T10:09:00.227 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-12-07T10:09:00.228 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/daemon-helper 2023-12-07T10:09:00.259 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-07T10:09:00.329 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-12-07T10:09:00.329 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/daemon-helper 2023-12-07T10:09:00.362 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-07T10:09:00.433 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-12-07T10:09:00.434 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-12-07T10:09:00.434 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-07T10:09:00.500 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-07T10:09:00.607 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-12-07T10:09:00.607 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-07T10:09:00.638 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-07T10:09:00.707 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-12-07T10:09:00.707 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-07T10:09:00.739 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-07T10:09:00.809 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-12-07T10:09:00.810 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-12-07T10:09:00.810 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/stdin-killer 2023-12-07T10:09:00.881 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-07T10:09:00.990 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-12-07T10:09:00.991 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/stdin-killer 2023-12-07T10:09:01.021 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-07T10:09:01.091 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-12-07T10:09:01.092 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/stdin-killer 2023-12-07T10:09:01.123 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-07T10:09:01.194 INFO:teuthology.run_tasks:Running task cephadm... 2023-12-07T10:09:01.283 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '3bd0bfa458b4399e7266c56d8215a2cf3a6091a0', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2023-12-07T10:09:01.284 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 2023-12-07T10:09:01.284 INFO:tasks.cephadm:Cluster fsid is a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:09:01.284 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-12-07T10:09:01.284 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.119', 'mon.b': '172.21.15.153', 'mon.c': '172.21.15.159'} 2023-12-07T10:09:01.284 INFO:tasks.cephadm:First mon is mon.a on smithi119 2023-12-07T10:09:01.285 INFO:tasks.cephadm:First mgr is x 2023-12-07T10:09:01.285 INFO:tasks.cephadm:Normalizing hostnames... 2023-12-07T10:09:01.285 DEBUG:teuthology.orchestra.run.smithi119:> sudo hostname $(hostname -s) 2023-12-07T10:09:01.357 DEBUG:teuthology.orchestra.run.smithi153:> sudo hostname $(hostname -s) 2023-12-07T10:09:01.387 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname $(hostname -s) 2023-12-07T10:09:01.420 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-12-07T10:09:01.420 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 2023-12-07T10:09:01.561 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3bd0bfa458b4399e7266c56d8215a2cf3a6091a0', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/74762/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-18-g3bd0bfa4', 'node_name': '172.21.2.6+braggi06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-18.g3bd0bfa4'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102654-r2/3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-12-07 08:11:49.657466', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20231127.102654-r2', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20231127.102654-r2/3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-12-07T10:09:01.697 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-vshankar-testing-20231127.102654-r2, sha1 3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 2023-12-07T10:09:01.697 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20231127.102654-r2/3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/centos/8/x86_64/flavors/default/cephadm 2023-12-07T10:09:01.697 DEBUG:teuthology.orchestra.run.smithi119:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20231127.102654-r2/3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-07T10:09:02.823 INFO:teuthology.orchestra.run.smithi119.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2270970 Dec 7 10:09 /home/ubuntu/cephtest/cephadm 2023-12-07T10:09:02.823 DEBUG:teuthology.orchestra.run.smithi153:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20231127.102654-r2/3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-07T10:09:03.347 INFO:teuthology.orchestra.run.smithi153.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2270970 Dec 7 10:09 /home/ubuntu/cephtest/cephadm 2023-12-07T10:09:03.347 DEBUG:teuthology.orchestra.run.smithi159:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20231127.102654-r2/3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-07T10:09:03.914 INFO:teuthology.orchestra.run.smithi159.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2270970 Dec 7 10:09 /home/ubuntu/cephtest/cephadm 2023-12-07T10:09:03.914 DEBUG:teuthology.orchestra.run.smithi119:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-07T10:09:03.965 DEBUG:teuthology.orchestra.run.smithi153:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-07T10:09:03.987 DEBUG:teuthology.orchestra.run.smithi159:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-07T10:09:04.022 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 on all hosts... 2023-12-07T10:09:04.022 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 pull 2023-12-07T10:09:04.027 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 pull 2023-12-07T10:09:04.030 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 pull 2023-12-07T10:09:05.599 INFO:teuthology.orchestra.run.smithi159.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0... 2023-12-07T10:09:06.741 INFO:teuthology.orchestra.run.smithi153.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0... 2023-12-07T10:09:08.742 INFO:teuthology.orchestra.run.smithi119.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0... 2023-12-07T10:09:43.001 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2023-12-07T10:09:43.001 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_version": "ceph version 19.0.0-18-g3bd0bfa4 (3bd0bfa458b4399e7266c56d8215a2cf3a6091a0) reef (dev)", 2023-12-07T10:09:43.002 INFO:teuthology.orchestra.run.smithi159.stdout: "image_id": "29737aea1903b84c85aede01d2e9a690df3ab37eec1e6067044d04efa9bd073e", 2023-12-07T10:09:43.002 INFO:teuthology.orchestra.run.smithi159.stdout: "repo_digests": [ 2023-12-07T10:09:43.002 INFO:teuthology.orchestra.run.smithi159.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:585a1e0544a730f9e80c1c8eda49c0530b0d6321429d152fe52210b86c8055f9" 2023-12-07T10:09:43.002 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2023-12-07T10:09:43.002 INFO:teuthology.orchestra.run.smithi159.stdout:} 2023-12-07T10:09:43.532 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-12-07T10:09:43.533 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_version": "ceph version 19.0.0-18-g3bd0bfa4 (3bd0bfa458b4399e7266c56d8215a2cf3a6091a0) reef (dev)", 2023-12-07T10:09:43.533 INFO:teuthology.orchestra.run.smithi153.stdout: "image_id": "29737aea1903b84c85aede01d2e9a690df3ab37eec1e6067044d04efa9bd073e", 2023-12-07T10:09:43.533 INFO:teuthology.orchestra.run.smithi153.stdout: "repo_digests": [ 2023-12-07T10:09:43.533 INFO:teuthology.orchestra.run.smithi153.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:585a1e0544a730f9e80c1c8eda49c0530b0d6321429d152fe52210b86c8055f9" 2023-12-07T10:09:43.533 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2023-12-07T10:09:43.533 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-12-07T10:09:48.150 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:09:48.151 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_version": "ceph version 19.0.0-18-g3bd0bfa4 (3bd0bfa458b4399e7266c56d8215a2cf3a6091a0) reef (dev)", 2023-12-07T10:09:48.151 INFO:teuthology.orchestra.run.smithi119.stdout: "image_id": "29737aea1903b84c85aede01d2e9a690df3ab37eec1e6067044d04efa9bd073e", 2023-12-07T10:09:48.151 INFO:teuthology.orchestra.run.smithi119.stdout: "repo_digests": [ 2023-12-07T10:09:48.151 INFO:teuthology.orchestra.run.smithi119.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:585a1e0544a730f9e80c1c8eda49c0530b0d6321429d152fe52210b86c8055f9" 2023-12-07T10:09:48.151 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-12-07T10:09:48.151 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:09:48.183 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph 2023-12-07T10:09:48.254 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /etc/ceph 2023-12-07T10:09:48.291 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /etc/ceph 2023-12-07T10:09:48.331 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 777 /etc/ceph 2023-12-07T10:09:48.398 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 777 /etc/ceph 2023-12-07T10:09:48.429 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 777 /etc/ceph 2023-12-07T10:09:48.461 INFO:tasks.cephadm:Writing seed config... 2023-12-07T10:09:48.462 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-12-07T10:09:48.462 INFO:tasks.cephadm: override: [client] debug client = 20 2023-12-07T10:09:48.462 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-12-07T10:09:48.462 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-12-07T10:09:48.462 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-12-07T10:09:48.463 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-12-07T10:09:48.463 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-12-07T10:09:48.463 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-12-07T10:09:48.463 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-12-07T10:09:48.463 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-12-07T10:09:48.463 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-12-07T10:09:48.463 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-12-07T10:09:48.463 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-12-07T10:09:48.463 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-12-07T10:09:48.463 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-12-07T10:09:48.463 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-12-07T10:09:48.463 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-12-07T10:09:48.463 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-12-07T10:09:48.463 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-12-07T10:09:48.463 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-12-07T10:09:48.464 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-12-07T10:09:48.464 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-12-07T10:09:48.464 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-12-07T10:09:48.464 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-12-07T10:09:48.464 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-12-07T10:09:48.464 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-12-07T10:09:48.464 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-12-07T10:09:48.464 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-12-07T10:09:48.464 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-12-07T10:09:48.464 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-12-07T10:09:48.464 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-12-07T10:09:48.464 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-12-07T10:09:48.464 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-12-07T10:09:48.464 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-12-07T10:09:48.464 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-12-07T10:09:48.464 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-12-07T10:09:48.465 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-12-07T10:09:48.465 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-12-07T10:09:48.465 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-12-07T10:09:48.465 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-12-07T10:09:48.465 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-12-07T10:09:48.465 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-12-07T10:09:48.465 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2023-12-07T10:09:48.466 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-12-07T10:09:48.466 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-12-07T10:09:48.508 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = a4c642c6-94e8-11ee-95a2-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-12-07T10:09:48.509 DEBUG:teuthology.orchestra.run.smithi119:mon.a> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mon.a.service 2023-12-07T10:09:48.553 DEBUG:teuthology.orchestra.run.smithi119:mgr.x> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mgr.x.service 2023-12-07T10:09:48.598 INFO:tasks.cephadm:Bootstrapping... 2023-12-07T10:09:48.599 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 -v bootstrap --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.119 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-12-07T10:09:48.620 INFO:journalctl@ceph.mon.a.smithi119.stdout:-- Logs begin at Thu 2023-12-07 09:53:13 UTC. -- 2023-12-07T10:09:48.667 INFO:journalctl@ceph.mgr.x.smithi119.stdout:-- Logs begin at Thu 2023-12-07 09:53:13 UTC. -- 2023-12-07T10:09:48.865 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-12-07T10:09:48.865 INFO:teuthology.orchestra.run.smithi119.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0', '-v', 'bootstrap', '--fsid', 'a4c642c6-94e8-11ee-95a2-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.119', '--skip-admin-label'] 2023-12-07T10:09:49.039 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stdout 4.1.1 2023-12-07T10:09:49.040 INFO:teuthology.orchestra.run.smithi119.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-12-07T10:09:49.040 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying podman|docker is present... 2023-12-07T10:09:49.214 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stdout 4.1.1 2023-12-07T10:09:49.214 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying lvm2 is present... 2023-12-07T10:09:49.214 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying time synchronization is in place... 2023-12-07T10:09:49.228 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-12-07T10:09:49.228 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-12-07T10:09:49.241 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-12-07T10:09:49.241 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout inactive 2023-12-07T10:09:49.254 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout enabled 2023-12-07T10:09:49.267 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout active 2023-12-07T10:09:49.267 INFO:teuthology.orchestra.run.smithi119.stdout:Unit chronyd.service is enabled and running 2023-12-07T10:09:49.267 INFO:teuthology.orchestra.run.smithi119.stdout:Repeating the final host check... 2023-12-07T10:09:49.438 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stdout 4.1.1 2023-12-07T10:09:49.438 INFO:teuthology.orchestra.run.smithi119.stdout:podman (/bin/podman) version 4.1.1 is present 2023-12-07T10:09:49.439 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl is present 2023-12-07T10:09:49.439 INFO:teuthology.orchestra.run.smithi119.stdout:lvcreate is present 2023-12-07T10:09:49.451 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-12-07T10:09:49.451 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-12-07T10:09:49.464 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-12-07T10:09:49.464 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout inactive 2023-12-07T10:09:49.477 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout enabled 2023-12-07T10:09:49.490 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout active 2023-12-07T10:09:49.490 INFO:teuthology.orchestra.run.smithi119.stdout:Unit chronyd.service is enabled and running 2023-12-07T10:09:49.490 INFO:teuthology.orchestra.run.smithi119.stdout:Host looks OK 2023-12-07T10:09:49.490 INFO:teuthology.orchestra.run.smithi119.stdout:Cluster fsid: a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:09:49.491 INFO:teuthology.orchestra.run.smithi119.stdout:Acquiring lock 139722646300040 on /run/cephadm/a4c642c6-94e8-11ee-95a2-87774f69a715.lock 2023-12-07T10:09:49.491 INFO:teuthology.orchestra.run.smithi119.stdout:Lock 139722646300040 acquired on /run/cephadm/a4c642c6-94e8-11ee-95a2-87774f69a715.lock 2023-12-07T10:09:49.491 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying IP 172.21.15.119 port 3300 ... 2023-12-07T10:09:49.492 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying IP 172.21.15.119 port 6789 ... 2023-12-07T10:09:49.492 INFO:teuthology.orchestra.run.smithi119.stdout:Base mon IP(s) is [172.21.15.119:3300, 172.21.15.119:6789], mon addrv is [v2:172.21.15.119:3300,v1:172.21.15.119:6789] 2023-12-07T10:09:49.500 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-12-07T10:09:49.500 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.119 metric 100 2023-12-07T10:09:49.506 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-12-07T10:09:49.506 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2023-12-07T10:09:49.506 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2023-12-07T10:09:49.513 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-12-07T10:09:49.513 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-12-07T10:09:49.513 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-12-07T10:09:49.513 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-12-07T10:09:49.513 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout inet6 fe80::9e03:31a5:7c84:6136/64 scope link noprefixroute 2023-12-07T10:09:49.513 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-12-07T10:09:49.514 INFO:teuthology.orchestra.run.smithi119.stdout:Mon IP `172.21.15.119` is in CIDR network `172.21.0.0/20` 2023-12-07T10:09:49.514 INFO:teuthology.orchestra.run.smithi119.stdout:Mon IP `172.21.15.119` is in CIDR network `172.21.0.0/20` 2023-12-07T10:09:49.514 INFO:teuthology.orchestra.run.smithi119.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-12-07T10:09:49.515 INFO:teuthology.orchestra.run.smithi119.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-12-07T10:09:49.516 INFO:teuthology.orchestra.run.smithi119.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0... 2023-12-07T10:09:49.989 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stdout 29737aea1903b84c85aede01d2e9a690df3ab37eec1e6067044d04efa9bd073e 2023-12-07T10:09:49.990 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0... 2023-12-07T10:09:49.990 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Getting image source signatures 2023-12-07T10:09:49.990 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Copying blob sha256:83219eef0df4f603d576f714f56d70a26ac110020266364735be852452d96c10 2023-12-07T10:09:49.990 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Copying blob sha256:d3e436ac93e4b5a6d94c7ac8024df331e66e67275836276615dc044fd3cbf83a 2023-12-07T10:09:49.990 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Copying config sha256:29737aea1903b84c85aede01d2e9a690df3ab37eec1e6067044d04efa9bd073e 2023-12-07T10:09:49.990 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Writing manifest to image destination 2023-12-07T10:09:49.990 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Storing signatures 2023-12-07T10:09:50.966 INFO:teuthology.orchestra.run.smithi119.stdout:ceph: stdout ceph version 19.0.0-18-g3bd0bfa4 (3bd0bfa458b4399e7266c56d8215a2cf3a6091a0) reef (dev) 2023-12-07T10:09:50.966 INFO:teuthology.orchestra.run.smithi119.stdout:Ceph version: ceph version 19.0.0-18-g3bd0bfa4 (3bd0bfa458b4399e7266c56d8215a2cf3a6091a0) reef (dev) 2023-12-07T10:09:50.966 INFO:teuthology.orchestra.run.smithi119.stdout:Extracting ceph user uid/gid from container image... 2023-12-07T10:09:53.547 INFO:teuthology.orchestra.run.smithi119.stdout:stat: stdout 167 167 2023-12-07T10:09:53.547 INFO:teuthology.orchestra.run.smithi119.stdout:Creating initial keys... 2023-12-07T10:09:54.423 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph-authtool: stdout AQDxmXFlXkF8OhAAcUljw2cKu4u7IgMLK3Olow== 2023-12-07T10:09:55.300 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph-authtool: stdout AQDymXFlWOByMRAATXzYUQ89EtQJfy6GhOlIQQ== 2023-12-07T10:09:56.176 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph-authtool: stdout AQDzmXFlfjYrKxAAYZ8cpTu+xPLVW0//bejxbA== 2023-12-07T10:09:56.177 INFO:teuthology.orchestra.run.smithi119.stdout:Creating initial monmap... 2023-12-07T10:09:56.953 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-12-07T10:09:56.953 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-12-07T10:09:56.953 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:09:56.953 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-12-07T10:09:56.953 INFO:teuthology.orchestra.run.smithi119.stdout:monmaptool for a [v2:172.21.15.119:3300,v1:172.21.15.119:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-12-07T10:09:56.953 INFO:teuthology.orchestra.run.smithi119.stdout:setting min_mon_release = pacific 2023-12-07T10:09:56.953 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/monmaptool: set fsid to a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:09:56.954 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-12-07T10:09:56.954 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:09:56.954 INFO:teuthology.orchestra.run.smithi119.stdout:Creating mon... 2023-12-07T10:09:57.930 INFO:teuthology.orchestra.run.smithi119.stdout:create mon.a on 2023-12-07T10:09:58.164 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-12-07T10:09:58.367 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-12-07T10:09:58.629 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-a4c642c6-94e8-11ee-95a2-87774f69a715.target → /etc/systemd/system/ceph-a4c642c6-94e8-11ee-95a2-87774f69a715.target. 2023-12-07T10:09:58.630 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-a4c642c6-94e8-11ee-95a2-87774f69a715.target → /etc/systemd/system/ceph-a4c642c6-94e8-11ee-95a2-87774f69a715.target. 2023-12-07T10:09:58.980 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mon.a 2023-12-07T10:09:58.981 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mon.a.service: Unit ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mon.a.service not loaded. 2023-12-07T10:09:59.182 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a4c642c6-94e8-11ee-95a2-87774f69a715.target.wants/ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mon.a.service → /etc/systemd/system/ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@.service. 2023-12-07T10:09:59.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:09:59 smithi119 systemd[1]: Starting Ceph mon.a for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:09:59.951 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:09:59 smithi119 podman[97353]: 2023-12-07T10:09:59.951 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:09:59 smithi119 bash[97353]: 7d5fdba28272bfb0bb833809f1764fd239cf15c0c932c5625fb3f786bdc1a5c9 2023-12-07T10:09:59.991 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-07T10:09:59.991 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout disabled 2023-12-07T10:10:00.007 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-07T10:10:00.007 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout inactive 2023-12-07T10:10:00.007 INFO:teuthology.orchestra.run.smithi119.stdout:firewalld.service is not enabled 2023-12-07T10:10:00.007 INFO:teuthology.orchestra.run.smithi119.stdout:Not possible to enable service . firewalld.service is not available 2023-12-07T10:10:00.007 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for mon to start... 2023-12-07T10:10:00.007 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for mon... 2023-12-07T10:10:00.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:09:59 smithi119 ceph-mon[97393]: mkfs a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:10:00.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:09:59 smithi119 ceph-mon[97393]: mon.a is new leader, mons a in quorum (ranks 0) 2023-12-07T10:10:00.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:09:59 smithi119 systemd[1]: Started Ceph mon.a for a4c642c6-94e8-11ee-95a2-87774f69a715. 2023-12-07T10:10:01.190 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout cluster: 2023-12-07T10:10:01.196 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout id: a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:10:01.196 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-12-07T10:10:01.196 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2023-12-07T10:10:01.196 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout services: 2023-12-07T10:10:01.197 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.705474s) 2023-12-07T10:10:01.197 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-12-07T10:10:01.197 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-12-07T10:10:01.197 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2023-12-07T10:10:01.197 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout data: 2023-12-07T10:10:01.197 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-12-07T10:10:01.197 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-12-07T10:10:01.197 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-12-07T10:10:01.197 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout pgs: 2023-12-07T10:10:01.197 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2023-12-07T10:10:01.197 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.566+0000 7f668575b700 1 Processor -- start 2023-12-07T10:10:01.197 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.566+0000 7f668575b700 1 -- start start 2023-12-07T10:10:01.197 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.567+0000 7f668575b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f668007ce70 0x7f668007b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:01.197 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.567+0000 7f668575b700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668007b8a0 con 0x7f668007ce70 2023-12-07T10:10:01.197 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.568+0000 7f667effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f668007ce70 0x7f668007b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:01.198 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.568+0000 7f667effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f668007ce70 0x7f668007b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47576/0 (socket says 172.21.15.119:47576) 2023-12-07T10:10:01.198 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.568+0000 7f667effd700 1 -- 172.21.15.119:0/2716553314 learned_addr learned my addr 172.21.15.119:0/2716553314 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:01.198 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.568+0000 7f667effd700 1 -- 172.21.15.119:0/2716553314 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668007b9e0 con 0x7f668007ce70 2023-12-07T10:10:01.198 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.568+0000 7f667effd700 1 --2- 172.21.15.119:0/2716553314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f668007ce70 0x7f668007b2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f66700098a0 tx=0x7f6670009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b45b7fbaf4d2633f server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:01.198 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.568+0000 7f667dffb700 1 -- 172.21.15.119:0/2716553314 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6670004160 con 0x7f668007ce70 2023-12-07T10:10:01.198 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.568+0000 7f667dffb700 1 -- 172.21.15.119:0/2716553314 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66700042c0 con 0x7f668007ce70 2023-12-07T10:10:01.198 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.568+0000 7f667dffb700 1 -- 172.21.15.119:0/2716553314 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66700045a0 con 0x7f668007ce70 2023-12-07T10:10:01.198 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.569+0000 7f668575b700 1 -- 172.21.15.119:0/2716553314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f668007ce70 msgr2=0x7f668007b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:01.198 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.569+0000 7f668575b700 1 --2- 172.21.15.119:0/2716553314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f668007ce70 0x7f668007b2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f66700098a0 tx=0x7f6670009bb0 comp rx=0 tx=0).stop 2023-12-07T10:10:01.199 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.569+0000 7f668575b700 1 -- 172.21.15.119:0/2716553314 shutdown_connections 2023-12-07T10:10:01.199 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.569+0000 7f668575b700 1 --2- 172.21.15.119:0/2716553314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f668007ce70 0x7f668007b2d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:01.199 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.569+0000 7f668575b700 1 -- 172.21.15.119:0/2716553314 >> 172.21.15.119:0/2716553314 conn(0x7f6680102220 msgr2=0x7f6680104660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:01.199 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.569+0000 7f668575b700 1 -- 172.21.15.119:0/2716553314 shutdown_connections 2023-12-07T10:10:01.199 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.569+0000 7f668575b700 1 -- 172.21.15.119:0/2716553314 wait complete. 2023-12-07T10:10:01.199 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.570+0000 7f668575b700 1 Processor -- start 2023-12-07T10:10:01.199 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.570+0000 7f668575b700 1 -- start start 2023-12-07T10:10:01.199 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.571+0000 7f668575b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f668007ce70 0x7f668019c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:01.199 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.571+0000 7f668575b700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668019c890 con 0x7f668007ce70 2023-12-07T10:10:01.199 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.571+0000 7f667effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f668007ce70 0x7f668019c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:01.199 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.571+0000 7f667effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f668007ce70 0x7f668019c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47582/0 (socket says 172.21.15.119:47582) 2023-12-07T10:10:01.199 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.571+0000 7f667effd700 1 -- 172.21.15.119:0/3059510926 learned_addr learned my addr 172.21.15.119:0/3059510926 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:01.199 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.571+0000 7f667effd700 1 -- 172.21.15.119:0/3059510926 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6670009550 con 0x7f668007ce70 2023-12-07T10:10:01.199 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.571+0000 7f667effd700 1 --2- 172.21.15.119:0/3059510926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f668007ce70 0x7f668019c350 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f668007c340 tx=0x7f6670004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:01.200 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.572+0000 7f6667fff700 1 -- 172.21.15.119:0/3059510926 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6670011630 con 0x7f668007ce70 2023-12-07T10:10:01.200 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.572+0000 7f6667fff700 1 -- 172.21.15.119:0/3059510926 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6670011790 con 0x7f668007ce70 2023-12-07T10:10:01.200 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.572+0000 7f668575b700 1 -- 172.21.15.119:0/3059510926 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f668019ca90 con 0x7f668007ce70 2023-12-07T10:10:01.200 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.572+0000 7f6667fff700 1 -- 172.21.15.119:0/3059510926 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6670011a30 con 0x7f668007ce70 2023-12-07T10:10:01.200 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.572+0000 7f668575b700 1 -- 172.21.15.119:0/3059510926 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f668019cf30 con 0x7f668007ce70 2023-12-07T10:10:01.200 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.572+0000 7f6667fff700 1 -- 172.21.15.119:0/3059510926 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6670022070 con 0x7f668007ce70 2023-12-07T10:10:01.200 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.572+0000 7f6667fff700 1 -- 172.21.15.119:0/3059510926 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f667001c450 con 0x7f668007ce70 2023-12-07T10:10:01.200 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.573+0000 7f668575b700 1 -- 172.21.15.119:0/3059510926 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6680196260 con 0x7f668007ce70 2023-12-07T10:10:01.200 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.576+0000 7f6667fff700 1 -- 172.21.15.119:0/3059510926 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f667001b450 con 0x7f668007ce70 2023-12-07T10:10:01.200 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.649+0000 7f668575b700 1 -- 172.21.15.119:0/3059510926 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f668007bf80 con 0x7f668007ce70 2023-12-07T10:10:01.200 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.649+0000 7f6667fff700 1 -- 172.21.15.119:0/3059510926 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f667001b5f0 con 0x7f668007ce70 2023-12-07T10:10:01.200 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.650+0000 7f668575b700 1 -- 172.21.15.119:0/3059510926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f668007ce70 msgr2=0x7f668019c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:01.200 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.650+0000 7f668575b700 1 --2- 172.21.15.119:0/3059510926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f668007ce70 0x7f668019c350 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f668007c340 tx=0x7f6670004750 comp rx=0 tx=0).stop 2023-12-07T10:10:01.200 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.650+0000 7f668575b700 1 -- 172.21.15.119:0/3059510926 shutdown_connections 2023-12-07T10:10:01.201 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.650+0000 7f668575b700 1 --2- 172.21.15.119:0/3059510926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f668007ce70 0x7f668019c350 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:01.201 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.650+0000 7f668575b700 1 -- 172.21.15.119:0/3059510926 >> 172.21.15.119:0/3059510926 conn(0x7f6680102220 msgr2=0x7f6680104550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:01.201 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.650+0000 7f668575b700 1 -- 172.21.15.119:0/3059510926 shutdown_connections 2023-12-07T10:10:01.201 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:00.650+0000 7f668575b700 1 -- 172.21.15.119:0/3059510926 wait complete. 2023-12-07T10:10:01.201 INFO:teuthology.orchestra.run.smithi119.stdout:mon is available 2023-12-07T10:10:01.201 INFO:teuthology.orchestra.run.smithi119.stdout:Assimilating anything we can from ceph.conf... 2023-12-07T10:10:01.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:00 smithi119 ceph-mon[97393]: mon.a is new leader, mons a in quorum (ranks 0) 2023-12-07T10:10:01.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:00 smithi119 ceph-mon[97393]: monmap e1: 1 mons at {a=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-07T10:10:01.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:00 smithi119 ceph-mon[97393]: fsmap 2023-12-07T10:10:01.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:00 smithi119 ceph-mon[97393]: osdmap e1: 0 total, 0 up, 0 in 2023-12-07T10:10:01.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:00 smithi119 ceph-mon[97393]: mgrmap e1: no daemons active 2023-12-07T10:10:01.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:00 smithi119 ceph-mon[97393]: from='client.? 172.21.15.119:0/3059510926' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-12-07T10:10:02.184 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:01 smithi119 ceph-mon[97393]: from='client.? 172.21.15.119:0/1567776862' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-12-07T10:10:02.184 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:01 smithi119 ceph-mon[97393]: from='client.? 172.21.15.119:0/1567776862' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-12-07T10:10:02.282 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2023-12-07T10:10:02.282 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout [global] 2023-12-07T10:10:02.282 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout fsid = a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:10:02.283 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.119:3300,v1:172.21.15.119:6789] 2023-12-07T10:10:02.283 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-12-07T10:10:02.283 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-12-07T10:10:02.283 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-12-07T10:10:02.283 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-12-07T10:10:02.283 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2023-12-07T10:10:02.283 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout [mgr] 2023-12-07T10:10:02.283 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-12-07T10:10:02.284 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2023-12-07T10:10:02.284 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout [osd] 2023-12-07T10:10:02.284 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-12-07T10:10:02.284 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-12-07T10:10:02.284 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-12-07T10:10:02.284 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-12-07T10:10:02.284 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-12-07T10:10:02.284 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.716+0000 7f20e30d5700 1 Processor -- start 2023-12-07T10:10:02.285 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.717+0000 7f20e30d5700 1 -- start start 2023-12-07T10:10:02.285 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.717+0000 7f20e30d5700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20dc106c60 0x7f20dc107080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:02.285 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.717+0000 7f20e30d5700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc107650 con 0x7f20dc106c60 2023-12-07T10:10:02.285 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.717+0000 7f20e0e71700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20dc106c60 0x7f20dc107080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:02.285 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.717+0000 7f20e0e71700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20dc106c60 0x7f20dc107080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47592/0 (socket says 172.21.15.119:47592) 2023-12-07T10:10:02.285 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.717+0000 7f20e0e71700 1 -- 172.21.15.119:0/4109704356 learned_addr learned my addr 172.21.15.119:0/4109704356 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:02.285 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.718+0000 7f20e0e71700 1 -- 172.21.15.119:0/4109704356 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20dc107e60 con 0x7f20dc106c60 2023-12-07T10:10:02.285 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.718+0000 7f20e0e71700 1 --2- 172.21.15.119:0/4109704356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20dc106c60 0x7f20dc107080 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f20d00098a0 tx=0x7f20d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9bd6e75804ac9c4 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:02.285 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.718+0000 7f20db7fe700 1 -- 172.21.15.119:0/4109704356 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20d0004160 con 0x7f20dc106c60 2023-12-07T10:10:02.285 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.718+0000 7f20db7fe700 1 -- 172.21.15.119:0/4109704356 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d00042c0 con 0x7f20dc106c60 2023-12-07T10:10:02.286 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.718+0000 7f20db7fe700 1 -- 172.21.15.119:0/4109704356 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20d00045a0 con 0x7f20dc106c60 2023-12-07T10:10:02.286 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.719+0000 7f20e30d5700 1 -- 172.21.15.119:0/4109704356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20dc106c60 msgr2=0x7f20dc107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:02.286 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.719+0000 7f20e30d5700 1 --2- 172.21.15.119:0/4109704356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20dc106c60 0x7f20dc107080 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f20d00098a0 tx=0x7f20d0009bb0 comp rx=0 tx=0).stop 2023-12-07T10:10:02.286 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.719+0000 7f20e30d5700 1 -- 172.21.15.119:0/4109704356 shutdown_connections 2023-12-07T10:10:02.286 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.719+0000 7f20e30d5700 1 --2- 172.21.15.119:0/4109704356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20dc106c60 0x7f20dc107080 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:02.286 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.719+0000 7f20e30d5700 1 -- 172.21.15.119:0/4109704356 >> 172.21.15.119:0/4109704356 conn(0x7f20dc102220 msgr2=0x7f20dc104640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:02.286 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.719+0000 7f20e30d5700 1 -- 172.21.15.119:0/4109704356 shutdown_connections 2023-12-07T10:10:02.286 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.719+0000 7f20e30d5700 1 -- 172.21.15.119:0/4109704356 wait complete. 2023-12-07T10:10:02.286 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.720+0000 7f20e30d5700 1 Processor -- start 2023-12-07T10:10:02.286 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.720+0000 7f20e30d5700 1 -- start start 2023-12-07T10:10:02.286 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.720+0000 7f20e30d5700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20dc106c60 0x7f20dc19c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:02.287 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.721+0000 7f20e30d5700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc19c940 con 0x7f20dc106c60 2023-12-07T10:10:02.287 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.721+0000 7f20e0e71700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20dc106c60 0x7f20dc19c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:02.287 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.721+0000 7f20e0e71700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20dc106c60 0x7f20dc19c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47602/0 (socket says 172.21.15.119:47602) 2023-12-07T10:10:02.287 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.721+0000 7f20e0e71700 1 -- 172.21.15.119:0/1567776862 learned_addr learned my addr 172.21.15.119:0/1567776862 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:02.287 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.721+0000 7f20e0e71700 1 -- 172.21.15.119:0/1567776862 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d0009550 con 0x7f20dc106c60 2023-12-07T10:10:02.287 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.721+0000 7f20e0e71700 1 --2- 172.21.15.119:0/1567776862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20dc106c60 0x7f20dc19c400 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f20d0004000 tx=0x7f20d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:02.287 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.722+0000 7f20d9ffb700 1 -- 172.21.15.119:0/1567776862 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20d0011630 con 0x7f20dc106c60 2023-12-07T10:10:02.287 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.722+0000 7f20d9ffb700 1 -- 172.21.15.119:0/1567776862 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d0011790 con 0x7f20dc106c60 2023-12-07T10:10:02.287 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.722+0000 7f20e30d5700 1 -- 172.21.15.119:0/1567776862 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20dc19cb40 con 0x7f20dc106c60 2023-12-07T10:10:02.287 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.722+0000 7f20d9ffb700 1 -- 172.21.15.119:0/1567776862 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20d0011a30 con 0x7f20dc106c60 2023-12-07T10:10:02.288 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.722+0000 7f20e30d5700 1 -- 172.21.15.119:0/1567776862 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20dc19cfe0 con 0x7f20dc106c60 2023-12-07T10:10:02.288 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.722+0000 7f20d9ffb700 1 -- 172.21.15.119:0/1567776862 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f20d0022070 con 0x7f20dc106c60 2023-12-07T10:10:02.288 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.722+0000 7f20d9ffb700 1 -- 172.21.15.119:0/1567776862 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f20d001c450 con 0x7f20dc106c60 2023-12-07T10:10:02.288 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.723+0000 7f20e30d5700 1 -- 172.21.15.119:0/1567776862 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20dc1081d0 con 0x7f20dc106c60 2023-12-07T10:10:02.288 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.725+0000 7f20d9ffb700 1 -- 172.21.15.119:0/1567776862 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f20d001b450 con 0x7f20dc106c60 2023-12-07T10:10:02.288 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.797+0000 7f20e30d5700 1 -- 172.21.15.119:0/1567776862 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f20dc108360 con 0x7f20dc106c60 2023-12-07T10:10:02.288 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.801+0000 7f20d9ffb700 1 -- 172.21.15.119:0/1567776862 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7f20d001b5f0 con 0x7f20dc106c60 2023-12-07T10:10:02.288 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.801+0000 7f20d9ffb700 1 -- 172.21.15.119:0/1567776862 <== mon.0 v2:172.21.15.119:3300/0 8 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f20d001b790 con 0x7f20dc106c60 2023-12-07T10:10:02.289 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.802+0000 7f20e30d5700 1 -- 172.21.15.119:0/1567776862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20dc106c60 msgr2=0x7f20dc19c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:02.289 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.802+0000 7f20e30d5700 1 --2- 172.21.15.119:0/1567776862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20dc106c60 0x7f20dc19c400 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f20d0004000 tx=0x7f20d0004750 comp rx=0 tx=0).stop 2023-12-07T10:10:02.289 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.802+0000 7f20e30d5700 1 -- 172.21.15.119:0/1567776862 shutdown_connections 2023-12-07T10:10:02.289 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.802+0000 7f20e30d5700 1 --2- 172.21.15.119:0/1567776862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20dc106c60 0x7f20dc19c400 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:02.289 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.802+0000 7f20e30d5700 1 -- 172.21.15.119:0/1567776862 >> 172.21.15.119:0/1567776862 conn(0x7f20dc102220 msgr2=0x7f20dc07c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:02.289 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.802+0000 7f20e30d5700 1 -- 172.21.15.119:0/1567776862 shutdown_connections 2023-12-07T10:10:02.289 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:01.803+0000 7f20e30d5700 1 -- 172.21.15.119:0/1567776862 wait complete. 2023-12-07T10:10:02.289 INFO:teuthology.orchestra.run.smithi119.stdout:Generating new minimal ceph.conf... 2023-12-07T10:10:03.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:02 smithi119 ceph-mon[97393]: from='client.? 172.21.15.119:0/2909779657' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:10:03.356 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.808+0000 7f59da06c700 1 Processor -- start 2023-12-07T10:10:03.356 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.808+0000 7f59da06c700 1 -- start start 2023-12-07T10:10:03.356 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.808+0000 7f59da06c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d4105b60 0x7f59d4105f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:03.356 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.808+0000 7f59da06c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d4106550 con 0x7f59d4105b60 2023-12-07T10:10:03.356 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.809+0000 7f59d37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d4105b60 0x7f59d4105f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:03.357 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.809+0000 7f59d37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d4105b60 0x7f59d4105f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39066/0 (socket says 172.21.15.119:39066) 2023-12-07T10:10:03.357 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.809+0000 7f59d37fe700 1 -- 172.21.15.119:0/949000736 learned_addr learned my addr 172.21.15.119:0/949000736 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:03.357 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.809+0000 7f59d37fe700 1 -- 172.21.15.119:0/949000736 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d4106d60 con 0x7f59d4105b60 2023-12-07T10:10:03.357 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.810+0000 7f59d37fe700 1 --2- 172.21.15.119:0/949000736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d4105b60 0x7f59d4105f80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f59c40098a0 tx=0x7f59c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4179cea84231763 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:03.357 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.810+0000 7f59d27fc700 1 -- 172.21.15.119:0/949000736 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59c4004160 con 0x7f59d4105b60 2023-12-07T10:10:03.357 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.810+0000 7f59d27fc700 1 -- 172.21.15.119:0/949000736 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f59c40042c0 con 0x7f59d4105b60 2023-12-07T10:10:03.357 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.810+0000 7f59d27fc700 1 -- 172.21.15.119:0/949000736 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59c4011a00 con 0x7f59d4105b60 2023-12-07T10:10:03.357 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.810+0000 7f59da06c700 1 -- 172.21.15.119:0/949000736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d4105b60 msgr2=0x7f59d4105f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:03.357 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.810+0000 7f59da06c700 1 --2- 172.21.15.119:0/949000736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d4105b60 0x7f59d4105f80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f59c40098a0 tx=0x7f59c4009bb0 comp rx=0 tx=0).stop 2023-12-07T10:10:03.357 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.810+0000 7f59da06c700 1 -- 172.21.15.119:0/949000736 shutdown_connections 2023-12-07T10:10:03.357 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.810+0000 7f59da06c700 1 --2- 172.21.15.119:0/949000736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d4105b60 0x7f59d4105f80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:03.357 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.810+0000 7f59da06c700 1 -- 172.21.15.119:0/949000736 >> 172.21.15.119:0/949000736 conn(0x7f59d4101140 msgr2=0x7f59d4103560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:03.357 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.811+0000 7f59da06c700 1 -- 172.21.15.119:0/949000736 shutdown_connections 2023-12-07T10:10:03.357 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.811+0000 7f59da06c700 1 -- 172.21.15.119:0/949000736 wait complete. 2023-12-07T10:10:03.358 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.812+0000 7f59da06c700 1 Processor -- start 2023-12-07T10:10:03.358 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.812+0000 7f59da06c700 1 -- start start 2023-12-07T10:10:03.358 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.812+0000 7f59da06c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d4105b60 0x7f59d419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:03.358 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.812+0000 7f59da06c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d419af90 con 0x7f59d4105b60 2023-12-07T10:10:03.358 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.812+0000 7f59d37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d4105b60 0x7f59d419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:03.358 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.812+0000 7f59d37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d4105b60 0x7f59d419aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39076/0 (socket says 172.21.15.119:39076) 2023-12-07T10:10:03.358 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.812+0000 7f59d37fe700 1 -- 172.21.15.119:0/2909779657 learned_addr learned my addr 172.21.15.119:0/2909779657 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:03.358 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.813+0000 7f59d37fe700 1 -- 172.21.15.119:0/2909779657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c4009550 con 0x7f59d4105b60 2023-12-07T10:10:03.358 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.813+0000 7f59d37fe700 1 --2- 172.21.15.119:0/2909779657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d4105b60 0x7f59d419aa50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009130 tx=0x7f59c4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:03.358 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.813+0000 7f59d0ff9700 1 -- 172.21.15.119:0/2909779657 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59c4003ab0 con 0x7f59d4105b60 2023-12-07T10:10:03.358 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.813+0000 7f59d0ff9700 1 -- 172.21.15.119:0/2909779657 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f59c4003c10 con 0x7f59d4105b60 2023-12-07T10:10:03.358 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.813+0000 7f59da06c700 1 -- 172.21.15.119:0/2909779657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d419b190 con 0x7f59d4105b60 2023-12-07T10:10:03.358 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.813+0000 7f59d0ff9700 1 -- 172.21.15.119:0/2909779657 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59c40247d0 con 0x7f59d4105b60 2023-12-07T10:10:03.358 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.813+0000 7f59da06c700 1 -- 172.21.15.119:0/2909779657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d419b630 con 0x7f59d4105b60 2023-12-07T10:10:03.359 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.813+0000 7f59d0ff9700 1 -- 172.21.15.119:0/2909779657 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f59c4024d50 con 0x7f59d4105b60 2023-12-07T10:10:03.359 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.814+0000 7f59d0ff9700 1 -- 172.21.15.119:0/2909779657 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f59c40193c0 con 0x7f59d4105b60 2023-12-07T10:10:03.359 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.814+0000 7f59da06c700 1 -- 172.21.15.119:0/2909779657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d4194b40 con 0x7f59d4105b60 2023-12-07T10:10:03.359 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.817+0000 7f59d0ff9700 1 -- 172.21.15.119:0/2909779657 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f59c401abb0 con 0x7f59d4105b60 2023-12-07T10:10:03.359 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.888+0000 7f59da06c700 1 -- 172.21.15.119:0/2909779657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f59d4109f90 con 0x7f59d4105b60 2023-12-07T10:10:03.359 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.889+0000 7f59d0ff9700 1 -- 172.21.15.119:0/2909779657 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7f59c401a430 con 0x7f59d4105b60 2023-12-07T10:10:03.359 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.890+0000 7f59da06c700 1 -- 172.21.15.119:0/2909779657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d4105b60 msgr2=0x7f59d419aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:03.359 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.890+0000 7f59da06c700 1 --2- 172.21.15.119:0/2909779657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d4105b60 0x7f59d419aa50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009130 tx=0x7f59c4011ad0 comp rx=0 tx=0).stop 2023-12-07T10:10:03.359 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.890+0000 7f59da06c700 1 -- 172.21.15.119:0/2909779657 shutdown_connections 2023-12-07T10:10:03.359 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.890+0000 7f59da06c700 1 --2- 172.21.15.119:0/2909779657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59d4105b60 0x7f59d419aa50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:03.359 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.890+0000 7f59da06c700 1 -- 172.21.15.119:0/2909779657 >> 172.21.15.119:0/2909779657 conn(0x7f59d4101140 msgr2=0x7f59d4102070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:03.359 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.890+0000 7f59da06c700 1 -- 172.21.15.119:0/2909779657 shutdown_connections 2023-12-07T10:10:03.359 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:02.890+0000 7f59da06c700 1 -- 172.21.15.119:0/2909779657 wait complete. 2023-12-07T10:10:03.359 INFO:teuthology.orchestra.run.smithi119.stdout:Restarting the monitor... 2023-12-07T10:10:03.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:03 smithi119 systemd[1]: Stopping Ceph mon.a for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:10:04.105 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:03 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mon-a[97369]: 2023-12-07T10:10:03.820+0000 7f4f7588c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:10:04.105 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:03 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mon-a[97369]: 2023-12-07T10:10:03.820+0000 7f4f7588c700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-12-07T10:10:04.458 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:04 smithi119 bash[97738]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mon-a 2023-12-07T10:10:04.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:04 smithi119 systemd[1]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mon.a.service: Succeeded. 2023-12-07T10:10:04.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:04 smithi119 systemd[1]: Stopped Ceph mon.a for a4c642c6-94e8-11ee-95a2-87774f69a715. 2023-12-07T10:10:04.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:04 smithi119 systemd[1]: Starting Ceph mon.a for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:10:05.345 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 podman[97868]: 2023-12-07T10:10:05.345 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 bash[97868]: f719e80f5e13089d789cfe2fc081c94ed85dc04490ac28ac3d3f3143acff774d 2023-12-07T10:10:05.345 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: set uid:gid to 167:167 (ceph:ceph) 2023-12-07T10:10:05.361 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: ceph version 19.0.0-18-g3bd0bfa4 (3bd0bfa458b4399e7266c56d8215a2cf3a6091a0) reef (dev), process ceph-mon, pid 7 2023-12-07T10:10:05.361 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: pidfile_write: ignore empty --pid-file 2023-12-07T10:10:05.361 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: load: jerasure load: lrc 2023-12-07T10:10:05.361 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: RocksDB version: 7.9.2 2023-12-07T10:10:05.361 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Git sha 0 2023-12-07T10:10:05.361 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Compile date 2023-12-07 06:56:28 2023-12-07T10:10:05.362 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: DB SUMMARY 2023-12-07T10:10:05.362 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: DB Session ID: NEOBVM5VGKUFV2HR2PMX 2023-12-07T10:10:05.362 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: CURRENT file: CURRENT 2023-12-07T10:10:05.362 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: IDENTITY file: IDENTITY 2023-12-07T10:10:05.362 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-12-07T10:10:05.362 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-12-07T10:10:05.362 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98319 ; 2023-12-07T10:10:05.362 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.error_if_exists: 0 2023-12-07T10:10:05.362 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.create_if_missing: 0 2023-12-07T10:10:05.362 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.paranoid_checks: 1 2023-12-07T10:10:05.362 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.flush_verify_memtable_count: 1 2023-12-07T10:10:05.363 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-12-07T10:10:05.363 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-12-07T10:10:05.363 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.env: 0x556bb2a1aa60 2023-12-07T10:10:05.363 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.fs: PosixFileSystem 2023-12-07T10:10:05.363 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.info_log: 0x556bb3f5b780 2023-12-07T10:10:05.363 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.max_file_opening_threads: 16 2023-12-07T10:10:05.363 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.statistics: (nil) 2023-12-07T10:10:05.363 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.use_fsync: 0 2023-12-07T10:10:05.363 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.max_log_file_size: 0 2023-12-07T10:10:05.363 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-07T10:10:05.363 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.log_file_time_to_roll: 0 2023-12-07T10:10:05.363 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.keep_log_file_num: 1000 2023-12-07T10:10:05.363 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.recycle_log_file_num: 0 2023-12-07T10:10:05.363 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.allow_fallocate: 1 2023-12-07T10:10:05.363 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.allow_mmap_reads: 0 2023-12-07T10:10:05.364 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.allow_mmap_writes: 0 2023-12-07T10:10:05.364 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.use_direct_reads: 0 2023-12-07T10:10:05.364 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-07T10:10:05.364 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.create_missing_column_families: 0 2023-12-07T10:10:05.364 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.db_log_dir: 2023-12-07T10:10:05.364 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.wal_dir: 2023-12-07T10:10:05.364 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.table_cache_numshardbits: 6 2023-12-07T10:10:05.364 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.WAL_ttl_seconds: 0 2023-12-07T10:10:05.364 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.WAL_size_limit_MB: 0 2023-12-07T10:10:05.364 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-07T10:10:05.364 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-07T10:10:05.364 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.is_fd_close_on_exec: 1 2023-12-07T10:10:05.364 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.advise_random_on_open: 1 2023-12-07T10:10:05.364 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.db_write_buffer_size: 0 2023-12-07T10:10:05.364 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.write_buffer_manager: 0x556bb3f11360 2023-12-07T10:10:05.364 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-07T10:10:05.365 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-07T10:10:05.365 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.use_adaptive_mutex: 0 2023-12-07T10:10:05.365 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.rate_limiter: (nil) 2023-12-07T10:10:05.365 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-07T10:10:05.365 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.wal_recovery_mode: 2 2023-12-07T10:10:05.365 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.enable_thread_tracking: 0 2023-12-07T10:10:05.365 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.enable_pipelined_write: 0 2023-12-07T10:10:05.365 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.unordered_write: 0 2023-12-07T10:10:05.365 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-07T10:10:05.365 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-07T10:10:05.365 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-07T10:10:05.365 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-07T10:10:05.365 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.row_cache: None 2023-12-07T10:10:05.365 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.wal_filter: None 2023-12-07T10:10:05.365 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-07T10:10:05.366 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.allow_ingest_behind: 0 2023-12-07T10:10:05.366 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.two_write_queues: 0 2023-12-07T10:10:05.366 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.manual_wal_flush: 0 2023-12-07T10:10:05.366 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.wal_compression: 0 2023-12-07T10:10:05.366 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.atomic_flush: 0 2023-12-07T10:10:05.366 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-07T10:10:05.366 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.persist_stats_to_disk: 0 2023-12-07T10:10:05.366 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.write_dbid_to_manifest: 0 2023-12-07T10:10:05.366 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.log_readahead_size: 0 2023-12-07T10:10:05.366 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-12-07T10:10:05.366 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.best_efforts_recovery: 0 2023-12-07T10:10:05.366 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-12-07T10:10:05.366 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-12-07T10:10:05.366 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.allow_data_in_errors: 0 2023-12-07T10:10:05.366 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.db_host_id: __hostname__ 2023-12-07T10:10:05.367 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.enforce_single_del_contracts: true 2023-12-07T10:10:05.367 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.max_background_jobs: 2 2023-12-07T10:10:05.367 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.max_background_compactions: -1 2023-12-07T10:10:05.367 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.max_subcompactions: 1 2023-12-07T10:10:05.367 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-07T10:10:05.367 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-07T10:10:05.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.delayed_write_rate : 16777216 2023-12-07T10:10:05.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.max_total_wal_size: 0 2023-12-07T10:10:05.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-07T10:10:05.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.stats_dump_period_sec: 600 2023-12-07T10:10:05.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.stats_persist_period_sec: 600 2023-12-07T10:10:05.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-07T10:10:05.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.max_open_files: -1 2023-12-07T10:10:05.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.bytes_per_sync: 0 2023-12-07T10:10:05.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.wal_bytes_per_sync: 0 2023-12-07T10:10:05.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.strict_bytes_per_sync: 0 2023-12-07T10:10:05.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.compaction_readahead_size: 0 2023-12-07T10:10:05.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.max_background_flushes: -1 2023-12-07T10:10:05.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Compression algorithms supported: 2023-12-07T10:10:05.368 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-07T10:10:05.369 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: kZSTD supported: 0 2023-12-07T10:10:05.369 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: kXpressCompression supported: 0 2023-12-07T10:10:05.369 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: kLZ4HCCompression supported: 1 2023-12-07T10:10:05.369 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: kZlibCompression supported: 1 2023-12-07T10:10:05.369 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: kSnappyCompression supported: 1 2023-12-07T10:10:05.369 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: kLZ4Compression supported: 1 2023-12-07T10:10:05.369 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: kBZip2Compression supported: 0 2023-12-07T10:10:05.369 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Fast CRC32 supported: Supported on x86 2023-12-07T10:10:05.369 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: DMutex implementation: pthread_mutex_t 2023-12-07T10:10:05.369 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-12-07T10:10:05.369 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-12-07T10:10:05.369 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-07T10:10:05.369 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.merge_operator: 2023-12-07T10:10:05.369 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.compaction_filter: None 2023-12-07T10:10:05.369 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.compaction_filter_factory: None 2023-12-07T10:10:05.370 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.sst_partitioner_factory: None 2023-12-07T10:10:05.370 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.memtable_factory: SkipListFactory 2023-12-07T10:10:05.370 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.table_factory: BlockBasedTable 2023-12-07T10:10:05.370 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556bb3f5b8a0) 2023-12-07T10:10:05.370 INFO:journalctl@ceph.mon.a.smithi119.stdout: cache_index_and_filter_blocks: 1 2023-12-07T10:10:05.370 INFO:journalctl@ceph.mon.a.smithi119.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-12-07T10:10:05.370 INFO:journalctl@ceph.mon.a.smithi119.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-07T10:10:05.370 INFO:journalctl@ceph.mon.a.smithi119.stdout: pin_top_level_index_and_filter: 1 2023-12-07T10:10:05.370 INFO:journalctl@ceph.mon.a.smithi119.stdout: index_type: 0 2023-12-07T10:10:05.370 INFO:journalctl@ceph.mon.a.smithi119.stdout: data_block_index_type: 0 2023-12-07T10:10:05.370 INFO:journalctl@ceph.mon.a.smithi119.stdout: index_shortening: 1 2023-12-07T10:10:05.370 INFO:journalctl@ceph.mon.a.smithi119.stdout: data_block_hash_table_util_ratio: 0.750000 2023-12-07T10:10:05.370 INFO:journalctl@ceph.mon.a.smithi119.stdout: checksum: 4 2023-12-07T10:10:05.370 INFO:journalctl@ceph.mon.a.smithi119.stdout: no_block_cache: 0 2023-12-07T10:10:05.370 INFO:journalctl@ceph.mon.a.smithi119.stdout: block_cache: 0x556bb3fb1090 2023-12-07T10:10:05.371 INFO:journalctl@ceph.mon.a.smithi119.stdout: block_cache_name: BinnedLRUCache 2023-12-07T10:10:05.371 INFO:journalctl@ceph.mon.a.smithi119.stdout: block_cache_options: 2023-12-07T10:10:05.371 INFO:journalctl@ceph.mon.a.smithi119.stdout: capacity : 536870912 2023-12-07T10:10:05.371 INFO:journalctl@ceph.mon.a.smithi119.stdout: num_shard_bits : 4 2023-12-07T10:10:05.371 INFO:journalctl@ceph.mon.a.smithi119.stdout: strict_capacity_limit : 0 2023-12-07T10:10:05.371 INFO:journalctl@ceph.mon.a.smithi119.stdout: high_pri_pool_ratio: 0.000 2023-12-07T10:10:05.371 INFO:journalctl@ceph.mon.a.smithi119.stdout: block_cache_compressed: (nil) 2023-12-07T10:10:05.371 INFO:journalctl@ceph.mon.a.smithi119.stdout: persistent_cache: (nil) 2023-12-07T10:10:05.371 INFO:journalctl@ceph.mon.a.smithi119.stdout: block_size: 4096 2023-12-07T10:10:05.371 INFO:journalctl@ceph.mon.a.smithi119.stdout: block_size_deviation: 10 2023-12-07T10:10:05.371 INFO:journalctl@ceph.mon.a.smithi119.stdout: block_restart_interval: 16 2023-12-07T10:10:05.371 INFO:journalctl@ceph.mon.a.smithi119.stdout: index_block_restart_interval: 1 2023-12-07T10:10:05.371 INFO:journalctl@ceph.mon.a.smithi119.stdout: metadata_block_size: 4096 2023-12-07T10:10:05.371 INFO:journalctl@ceph.mon.a.smithi119.stdout: partition_filters: 0 2023-12-07T10:10:05.371 INFO:journalctl@ceph.mon.a.smithi119.stdout: use_delta_encoding: 1 2023-12-07T10:10:05.372 INFO:journalctl@ceph.mon.a.smithi119.stdout: filter_policy: bloomfilter 2023-12-07T10:10:05.372 INFO:journalctl@ceph.mon.a.smithi119.stdout: whole_key_filtering: 1 2023-12-07T10:10:05.372 INFO:journalctl@ceph.mon.a.smithi119.stdout: verify_compression: 0 2023-12-07T10:10:05.372 INFO:journalctl@ceph.mon.a.smithi119.stdout: read_amp_bytes_per_bit: 0 2023-12-07T10:10:05.372 INFO:journalctl@ceph.mon.a.smithi119.stdout: format_version: 5 2023-12-07T10:10:05.372 INFO:journalctl@ceph.mon.a.smithi119.stdout: enable_index_compression: 1 2023-12-07T10:10:05.372 INFO:journalctl@ceph.mon.a.smithi119.stdout: block_align: 0 2023-12-07T10:10:05.372 INFO:journalctl@ceph.mon.a.smithi119.stdout: max_auto_readahead_size: 262144 2023-12-07T10:10:05.372 INFO:journalctl@ceph.mon.a.smithi119.stdout: prepopulate_block_cache: 0 2023-12-07T10:10:05.372 INFO:journalctl@ceph.mon.a.smithi119.stdout: initial_auto_readahead_size: 8192 2023-12-07T10:10:05.372 INFO:journalctl@ceph.mon.a.smithi119.stdout: num_file_reads_for_auto_readahead: 2 2023-12-07T10:10:05.372 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.write_buffer_size: 33554432 2023-12-07T10:10:05.372 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.max_write_buffer_number: 2 2023-12-07T10:10:05.372 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.compression: NoCompression 2023-12-07T10:10:05.372 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.bottommost_compression: Disabled 2023-12-07T10:10:05.372 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.prefix_extractor: nullptr 2023-12-07T10:10:05.373 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-07T10:10:05.373 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.num_levels: 7 2023-12-07T10:10:05.373 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-07T10:10:05.373 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-07T10:10:05.373 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-07T10:10:05.373 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-07T10:10:05.373 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-07T10:10:05.379 INFO:teuthology.orchestra.run.smithi119.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-12-07T10:10:05.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-07T10:10:05.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-07T10:10:05.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-07T10:10:05.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-12-07T10:10:05.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-07T10:10:05.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-12-07T10:10:05.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-12-07T10:10:05.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.compression_opts.window_bits: -14 2023-12-07T10:10:05.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.compression_opts.level: 32767 2023-12-07T10:10:05.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.compression_opts.strategy: 0 2023-12-07T10:10:05.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-07T10:10:05.646 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-07T10:10:05.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-12-07T10:10:05.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-12-07T10:10:05.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.compression_opts.enabled: false 2023-12-07T10:10:05.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-12-07T10:10:05.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-07T10:10:05.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-07T10:10:05.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-07T10:10:05.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.target_file_size_base: 67108864 2023-12-07T10:10:05.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.target_file_size_multiplier: 1 2023-12-07T10:10:05.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-07T10:10:05.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-07T10:10:05.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-07T10:10:05.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-07T10:10:05.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-07T10:10:05.647 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-07T10:10:05.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-07T10:10:05.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-07T10:10:05.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-07T10:10:05.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-07T10:10:05.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-07T10:10:05.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-07T10:10:05.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-12-07T10:10:05.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.arena_block_size: 1048576 2023-12-07T10:10:05.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-07T10:10:05.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-07T10:10:05.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.disable_auto_compactions: 0 2023-12-07T10:10:05.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-07T10:10:05.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-07T10:10:05.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-07T10:10:05.648 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-07T10:10:05.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-07T10:10:05.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-07T10:10:05.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-07T10:10:05.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-07T10:10:05.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-07T10:10:05.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-07T10:10:05.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.table_properties_collectors: 2023-12-07T10:10:05.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.inplace_update_support: 0 2023-12-07T10:10:05.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.inplace_update_num_locks: 10000 2023-12-07T10:10:05.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-07T10:10:05.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-07T10:10:05.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.memtable_huge_page_size: 0 2023-12-07T10:10:05.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.bloom_locality: 0 2023-12-07T10:10:05.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.max_successive_merges: 0 2023-12-07T10:10:05.649 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.optimize_filters_for_hits: 0 2023-12-07T10:10:05.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.paranoid_file_checks: 0 2023-12-07T10:10:05.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.force_consistency_checks: 1 2023-12-07T10:10:05.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.report_bg_io_stats: 0 2023-12-07T10:10:05.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.ttl: 2592000 2023-12-07T10:10:05.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.periodic_compaction_seconds: 0 2023-12-07T10:10:05.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-12-07T10:10:05.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-12-07T10:10:05.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.enable_blob_files: false 2023-12-07T10:10:05.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.min_blob_size: 0 2023-12-07T10:10:05.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.blob_file_size: 268435456 2023-12-07T10:10:05.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.blob_compression_type: NoCompression 2023-12-07T10:10:05.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.enable_blob_garbage_collection: false 2023-12-07T10:10:05.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-12-07T10:10:05.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-12-07T10:10:05.650 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-12-07T10:10:05.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.blob_file_starting_level: 0 2023-12-07T10:10:05.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-12-07T10:10:05.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-12-07T10:10:05.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-12-07T10:10:05.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 812a8668-326e-4125-a0c9-831904ab6512 2023-12-07T10:10:05.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701943805352006, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-12-07T10:10:05.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-12-07T10:10:05.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701943805353630, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 312, "table_properties": {"data_size": 91622, "index_size": 282, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13509, "raw_average_key_size": 49, "raw_value_size": 84251, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 271, "num_filter_entries": 271, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1701943805, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "812a8668-326e-4125-a0c9-831904ab6512", "db_session_id": "NEOBVM5VGKUFV2HR2PMX", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-12-07T10:10:05.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701943805353761, "job": 1, "event": "recovery_finished"} 2023-12-07T10:10:05.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-12-07T10:10:05.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-12-07T10:10:05.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556bb404c000 2023-12-07T10:10:05.651 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: DB pointer 0x556bb4038000 2023-12-07T10:10:05.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-12-07T10:10:05.652 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-12-07T10:10:05.652 INFO:journalctl@ceph.mon.a.smithi119.stdout: ** DB Stats ** 2023-12-07T10:10:05.652 INFO:journalctl@ceph.mon.a.smithi119.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-07T10:10:05.652 INFO:journalctl@ceph.mon.a.smithi119.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-07T10:10:05.652 INFO:journalctl@ceph.mon.a.smithi119.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-07T10:10:05.652 INFO:journalctl@ceph.mon.a.smithi119.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-07T10:10:05.652 INFO:journalctl@ceph.mon.a.smithi119.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-07T10:10:05.653 INFO:journalctl@ceph.mon.a.smithi119.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-07T10:10:05.653 INFO:journalctl@ceph.mon.a.smithi119.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-07T10:10:05.653 INFO:journalctl@ceph.mon.a.smithi119.stdout: 2023-12-07T10:10:05.653 INFO:journalctl@ceph.mon.a.smithi119.stdout: ** Compaction Stats [default] ** 2023-12-07T10:10:05.653 INFO:journalctl@ceph.mon.a.smithi119.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-07T10:10:05.653 INFO:journalctl@ceph.mon.a.smithi119.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-12-07T10:10:05.653 INFO:journalctl@ceph.mon.a.smithi119.stdout: L0 2/0 93.26 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 85.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-07T10:10:05.653 INFO:journalctl@ceph.mon.a.smithi119.stdout: Sum 2/0 93.26 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 85.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-07T10:10:05.653 INFO:journalctl@ceph.mon.a.smithi119.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 85.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-07T10:10:05.653 INFO:journalctl@ceph.mon.a.smithi119.stdout: 2023-12-07T10:10:05.653 INFO:journalctl@ceph.mon.a.smithi119.stdout: ** Compaction Stats [default] ** 2023-12-07T10:10:05.654 INFO:journalctl@ceph.mon.a.smithi119.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-07T10:10:05.654 INFO:journalctl@ceph.mon.a.smithi119.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-07T10:10:05.654 INFO:journalctl@ceph.mon.a.smithi119.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 85.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-07T10:10:05.654 INFO:journalctl@ceph.mon.a.smithi119.stdout: 2023-12-07T10:10:05.654 INFO:journalctl@ceph.mon.a.smithi119.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-12-07T10:10:05.654 INFO:journalctl@ceph.mon.a.smithi119.stdout: 2023-12-07T10:10:05.654 INFO:journalctl@ceph.mon.a.smithi119.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-07T10:10:05.654 INFO:journalctl@ceph.mon.a.smithi119.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-12-07T10:10:05.654 INFO:journalctl@ceph.mon.a.smithi119.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-07T10:10:05.654 INFO:journalctl@ceph.mon.a.smithi119.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-12-07T10:10:05.654 INFO:journalctl@ceph.mon.a.smithi119.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-12-07T10:10:05.654 INFO:journalctl@ceph.mon.a.smithi119.stdout: AddFile(Keys): cumulative 0, interval 0 2023-12-07T10:10:05.654 INFO:journalctl@ceph.mon.a.smithi119.stdout: Cumulative compaction: 0.00 GB write, 11.52 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-07T10:10:05.655 INFO:journalctl@ceph.mon.a.smithi119.stdout: Interval compaction: 0.00 GB write, 11.52 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-07T10:10:05.655 INFO:journalctl@ceph.mon.a.smithi119.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-07T10:10:05.655 INFO:journalctl@ceph.mon.a.smithi119.stdout: Block cache BinnedLRUCache@0x556bb3fb1090#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2023-12-07T10:10:05.655 INFO:journalctl@ceph.mon.a.smithi119.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-12-07T10:10:05.655 INFO:journalctl@ceph.mon.a.smithi119.stdout: 2023-12-07T10:10:05.655 INFO:journalctl@ceph.mon.a.smithi119.stdout: ** File Read Latency Histogram By Level [default] ** 2023-12-07T10:10:05.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: starting mon.a rank 0 at public addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] at bind addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:10:05.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: mon.a@-1(???) e1 preinit fsid a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:10:05.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: mon.a@-1(???).mds e1 new map 2023-12-07T10:10:05.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: mon.a@-1(???).mds e1 print_map 2023-12-07T10:10:05.655 INFO:journalctl@ceph.mon.a.smithi119.stdout: e1 2023-12-07T10:10:05.655 INFO:journalctl@ceph.mon.a.smithi119.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-12-07T10:10:05.656 INFO:journalctl@ceph.mon.a.smithi119.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-12-07T10:10:05.656 INFO:journalctl@ceph.mon.a.smithi119.stdout: legacy client fscid: -1 2023-12-07T10:10:05.656 INFO:journalctl@ceph.mon.a.smithi119.stdout: 2023-12-07T10:10:05.656 INFO:journalctl@ceph.mon.a.smithi119.stdout: No filesystems configured 2023-12-07T10:10:05.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-12-07T10:10:05.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-07T10:10:05.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-07T10:10:05.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-07T10:10:05.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-12-07T10:10:05.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 systemd[1]: Started Ceph mon.a for a4c642c6-94e8-11ee-95a2-87774f69a715. 2023-12-07T10:10:05.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: mon.a is new leader, mons a in quorum (ranks 0) 2023-12-07T10:10:05.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: monmap e1: 1 mons at {a=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-07T10:10:05.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: fsmap 2023-12-07T10:10:05.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: osdmap e1: 0 total, 0 up, 0 in 2023-12-07T10:10:05.656 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:05 smithi119 ceph-mon[97908]: mgrmap e1: no daemons active 2023-12-07T10:10:06.465 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.906+0000 7fba696ce700 1 Processor -- start 2023-12-07T10:10:06.465 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.906+0000 7fba696ce700 1 -- start start 2023-12-07T10:10:06.465 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.907+0000 7fba696ce700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba64108e70 0x7fba64109290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:06.465 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.907+0000 7fba696ce700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba641097d0 con 0x7fba64108e70 2023-12-07T10:10:06.465 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.907+0000 7fba62ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba64108e70 0x7fba64109290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:06.466 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.908+0000 7fba62ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba64108e70 0x7fba64109290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39080/0 (socket says 172.21.15.119:39080) 2023-12-07T10:10:06.466 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.908+0000 7fba62ffd700 1 -- 172.21.15.119:0/689728643 learned_addr learned my addr 172.21.15.119:0/689728643 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:06.466 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.908+0000 7fba62ffd700 1 -- 172.21.15.119:0/689728643 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba64109910 con 0x7fba64108e70 2023-12-07T10:10:06.466 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.908+0000 7fba62ffd700 1 --2- 172.21.15.119:0/689728643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba64108e70 0x7fba64109290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fba540098a0 tx=0x7fba54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34bad438d8c651d server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:06.466 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.908+0000 7fba61ffb700 1 -- 172.21.15.119:0/689728643 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba54004160 con 0x7fba64108e70 2023-12-07T10:10:06.466 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.908+0000 7fba61ffb700 1 -- 172.21.15.119:0/689728643 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fba540042c0 con 0x7fba64108e70 2023-12-07T10:10:06.466 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.908+0000 7fba61ffb700 1 -- 172.21.15.119:0/689728643 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba54011a00 con 0x7fba64108e70 2023-12-07T10:10:06.466 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.909+0000 7fba696ce700 1 -- 172.21.15.119:0/689728643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba64108e70 msgr2=0x7fba64109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:06.466 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.909+0000 7fba696ce700 1 --2- 172.21.15.119:0/689728643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba64108e70 0x7fba64109290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fba540098a0 tx=0x7fba54009bb0 comp rx=0 tx=0).stop 2023-12-07T10:10:06.466 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.909+0000 7fba696ce700 1 -- 172.21.15.119:0/689728643 shutdown_connections 2023-12-07T10:10:06.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.909+0000 7fba696ce700 1 --2- 172.21.15.119:0/689728643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba64108e70 0x7fba64109290 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:06.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.909+0000 7fba696ce700 1 -- 172.21.15.119:0/689728643 >> 172.21.15.119:0/689728643 conn(0x7fba6407c240 msgr2=0x7fba6407c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:06.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.909+0000 7fba696ce700 1 -- 172.21.15.119:0/689728643 shutdown_connections 2023-12-07T10:10:06.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.909+0000 7fba696ce700 1 -- 172.21.15.119:0/689728643 wait complete. 2023-12-07T10:10:06.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.911+0000 7fba696ce700 1 Processor -- start 2023-12-07T10:10:06.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.911+0000 7fba696ce700 1 -- start start 2023-12-07T10:10:06.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.911+0000 7fba696ce700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba64108e70 0x7fba6419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:06.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.911+0000 7fba696ce700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6419d100 con 0x7fba64108e70 2023-12-07T10:10:06.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.912+0000 7fba62ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba64108e70 0x7fba6419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:06.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.912+0000 7fba62ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba64108e70 0x7fba6419cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39086/0 (socket says 172.21.15.119:39086) 2023-12-07T10:10:06.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.912+0000 7fba62ffd700 1 -- 172.21.15.119:0/1644011964 learned_addr learned my addr 172.21.15.119:0/1644011964 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:06.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.912+0000 7fba62ffd700 1 -- 172.21.15.119:0/1644011964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba54009550 con 0x7fba64108e70 2023-12-07T10:10:06.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.912+0000 7fba62ffd700 1 --2- 172.21.15.119:0/1644011964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba64108e70 0x7fba6419cbc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fba54009130 tx=0x7fba54011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:06.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.912+0000 7fba4bfff700 1 -- 172.21.15.119:0/1644011964 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba54003b70 con 0x7fba64108e70 2023-12-07T10:10:06.468 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.912+0000 7fba4bfff700 1 -- 172.21.15.119:0/1644011964 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fba54003cd0 con 0x7fba64108e70 2023-12-07T10:10:06.468 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.912+0000 7fba696ce700 1 -- 172.21.15.119:0/1644011964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba6419d300 con 0x7fba64108e70 2023-12-07T10:10:06.468 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.912+0000 7fba4bfff700 1 -- 172.21.15.119:0/1644011964 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba540197d0 con 0x7fba64108e70 2023-12-07T10:10:06.468 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.913+0000 7fba696ce700 1 -- 172.21.15.119:0/1644011964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba6419d7a0 con 0x7fba64108e70 2023-12-07T10:10:06.468 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.913+0000 7fba4bfff700 1 -- 172.21.15.119:0/1644011964 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fba540199c0 con 0x7fba64108e70 2023-12-07T10:10:06.468 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.913+0000 7fba4bfff700 1 -- 172.21.15.119:0/1644011964 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fba5400b720 con 0x7fba64108e70 2023-12-07T10:10:06.468 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.914+0000 7fba696ce700 1 -- 172.21.15.119:0/1644011964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba64062c40 con 0x7fba64108e70 2023-12-07T10:10:06.468 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.917+0000 7fba4bfff700 1 -- 172.21.15.119:0/1644011964 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fba5402a060 con 0x7fba64108e70 2023-12-07T10:10:06.468 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.987+0000 7fba696ce700 1 -- 172.21.15.119:0/1644011964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fba6410ac50 con 0x7fba64108e70 2023-12-07T10:10:06.468 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.989+0000 7fba4bfff700 1 -- 172.21.15.119:0/1644011964 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fba54019e00 con 0x7fba64108e70 2023-12-07T10:10:06.468 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.990+0000 7fba696ce700 1 -- 172.21.15.119:0/1644011964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba64108e70 msgr2=0x7fba6419cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:06.468 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.990+0000 7fba696ce700 1 --2- 172.21.15.119:0/1644011964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba64108e70 0x7fba6419cbc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fba54009130 tx=0x7fba54011ad0 comp rx=0 tx=0).stop 2023-12-07T10:10:06.468 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.991+0000 7fba696ce700 1 -- 172.21.15.119:0/1644011964 shutdown_connections 2023-12-07T10:10:06.468 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.991+0000 7fba696ce700 1 --2- 172.21.15.119:0/1644011964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba64108e70 0x7fba6419cbc0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:06.469 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.991+0000 7fba696ce700 1 -- 172.21.15.119:0/1644011964 >> 172.21.15.119:0/1644011964 conn(0x7fba6407c240 msgr2=0x7fba64106550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:06.469 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.991+0000 7fba696ce700 1 -- 172.21.15.119:0/1644011964 shutdown_connections 2023-12-07T10:10:06.469 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:05.991+0000 7fba696ce700 1 -- 172.21.15.119:0/1644011964 wait complete. 2023-12-07T10:10:06.469 INFO:teuthology.orchestra.run.smithi119.stdout:Wrote config to /etc/ceph/ceph.conf 2023-12-07T10:10:06.489 INFO:teuthology.orchestra.run.smithi119.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-12-07T10:10:06.489 INFO:teuthology.orchestra.run.smithi119.stdout:Creating mgr... 2023-12-07T10:10:06.490 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying port 0.0.0.0:9283 ... 2023-12-07T10:10:06.490 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying port 0.0.0.0:8765 ... 2023-12-07T10:10:06.842 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mgr.x 2023-12-07T10:10:06.843 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mgr.x.service: Unit ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mgr.x.service not loaded. 2023-12-07T10:10:07.045 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a4c642c6-94e8-11ee-95a2-87774f69a715.target.wants/ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@.service. 2023-12-07T10:10:07.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:06 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1644011964' entity='client.admin' 2023-12-07T10:10:07.230 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:07 smithi119 systemd[1]: Starting Ceph mgr.x for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:10:07.818 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:07 smithi119 podman[98143]: 2023-12-07T10:10:07.855 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-07T10:10:07.855 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout disabled 2023-12-07T10:10:07.871 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-07T10:10:07.871 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout inactive 2023-12-07T10:10:07.871 INFO:teuthology.orchestra.run.smithi119.stdout:firewalld.service is not enabled 2023-12-07T10:10:07.871 INFO:teuthology.orchestra.run.smithi119.stdout:Not possible to enable service . firewalld.service is not available 2023-12-07T10:10:07.889 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-07T10:10:07.889 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout disabled 2023-12-07T10:10:07.904 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-07T10:10:07.905 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout inactive 2023-12-07T10:10:07.905 INFO:teuthology.orchestra.run.smithi119.stdout:firewalld.service is not enabled 2023-12-07T10:10:07.905 INFO:teuthology.orchestra.run.smithi119.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-12-07T10:10:07.905 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for mgr to start... 2023-12-07T10:10:07.905 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for mgr... 2023-12-07T10:10:08.078 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:07 smithi119 bash[98143]: 9b1f99d0259769e0e256039caeaba0b992217b7e66030b56c6bbca9ea3c0469e 2023-12-07T10:10:08.078 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:07 smithi119 systemd[1]: Started Ceph mgr.x for a4c642c6-94e8-11ee-95a2-87774f69a715. 2023-12-07T10:10:08.078 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:07 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:07.968+0000 7f8cd0821200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-07T10:10:08.078 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:08 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:08.074+0000 7f8cd0821200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-07T10:10:08.342 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:08 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:08.191+0000 7f8cd0821200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-07T10:10:08.343 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:08 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:08.319+0000 7f8cd0821200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-07T10:10:08.723 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:08 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:08.559+0000 7f8cd0821200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-07T10:10:08.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:08 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/4153975978' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-07T10:10:09.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2023-12-07T10:10:09.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2023-12-07T10:10:09.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsid": "a4c642c6-94e8-11ee-95a2-87774f69a715", 2023-12-07T10:10:09.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "health": { 2023-12-07T10:10:09.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-07T10:10:09.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-07T10:10:09.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-07T10:10:09.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-12-07T10:10:09.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-07T10:10:09.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-07T10:10:09.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 0 2023-12-07T10:10:09.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2023-12-07T10:10:09.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-07T10:10:09.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "a" 2023-12-07T10:10:09.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2023-12-07T10:10:09.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-12-07T10:10:09.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-07T10:10:09.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T10:10:09.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-07T10:10:09.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-07T10:10:09.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-12-07T10:10:09.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-07T10:10:09.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T10:10:09.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-07T10:10:09.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-07T10:10:09.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-07T10:10:09.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-07T10:10:09.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-07T10:10:09.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-07T10:10:09.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-12-07T10:10:09.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-07T10:10:09.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-07T10:10:09.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-07T10:10:09.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-07T10:10:09.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-07T10:10:09.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-07T10:10:09.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-07T10:10:09.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-07T10:10:09.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-07T10:10:09.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-12-07T10:10:09.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-07T10:10:09.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T10:10:09.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-07T10:10:09.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-07T10:10:09.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-12-07T10:10:09.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-07T10:10:09.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "available": false, 2023-12-07T10:10:09.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-07T10:10:09.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-07T10:10:09.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-07T10:10:09.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-07T10:10:09.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "restful" 2023-12-07T10:10:09.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2023-12-07T10:10:09.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-07T10:10:09.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-12-07T10:10:09.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-07T10:10:09.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T10:10:09.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modified": "2023-12-07T10:09:59.946493+0000", 2023-12-07T10:10:09.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-07T10:10:09.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-12-07T10:10:09.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-07T10:10:09.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2023-12-07T10:10:09.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.611+0000 7f4bc5c78700 1 Processor -- start 2023-12-07T10:10:09.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.611+0000 7f4bc5c78700 1 -- start start 2023-12-07T10:10:09.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.611+0000 7f4bc5c78700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bc007ce70 0x7f4bc007b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:09.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.612+0000 7f4bc5c78700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc007b8a0 con 0x7f4bc007ce70 2023-12-07T10:10:09.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.612+0000 7f4bbf7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bc007ce70 0x7f4bc007b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:09.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.612+0000 7f4bbf7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bc007ce70 0x7f4bc007b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39112/0 (socket says 172.21.15.119:39112) 2023-12-07T10:10:09.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.612+0000 7f4bbf7fe700 1 -- 172.21.15.119:0/4242850746 learned_addr learned my addr 172.21.15.119:0/4242850746 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:09.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.612+0000 7f4bbf7fe700 1 -- 172.21.15.119:0/4242850746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc007b9e0 con 0x7f4bc007ce70 2023-12-07T10:10:09.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.613+0000 7f4bbf7fe700 1 --2- 172.21.15.119:0/4242850746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bc007ce70 0x7f4bc007b2d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00098a0 tx=0x7f4bb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3058c76567ea6b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:09.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.613+0000 7f4bbe7fc700 1 -- 172.21.15.119:0/4242850746 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bb0004160 con 0x7f4bc007ce70 2023-12-07T10:10:09.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.613+0000 7f4bbe7fc700 1 -- 172.21.15.119:0/4242850746 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4bb00042c0 con 0x7f4bc007ce70 2023-12-07T10:10:09.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.613+0000 7f4bbe7fc700 1 -- 172.21.15.119:0/4242850746 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bb0011a00 con 0x7f4bc007ce70 2023-12-07T10:10:09.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.614+0000 7f4bc5c78700 1 -- 172.21.15.119:0/4242850746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bc007ce70 msgr2=0x7f4bc007b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:09.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.614+0000 7f4bc5c78700 1 --2- 172.21.15.119:0/4242850746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bc007ce70 0x7f4bc007b2d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00098a0 tx=0x7f4bb0009bb0 comp rx=0 tx=0).stop 2023-12-07T10:10:09.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.614+0000 7f4bc5c78700 1 -- 172.21.15.119:0/4242850746 shutdown_connections 2023-12-07T10:10:09.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.614+0000 7f4bc5c78700 1 --2- 172.21.15.119:0/4242850746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bc007ce70 0x7f4bc007b2d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:09.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.614+0000 7f4bc5c78700 1 -- 172.21.15.119:0/4242850746 >> 172.21.15.119:0/4242850746 conn(0x7f4bc0102220 msgr2=0x7f4bc0104660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:09.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.614+0000 7f4bc5c78700 1 -- 172.21.15.119:0/4242850746 shutdown_connections 2023-12-07T10:10:09.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.614+0000 7f4bc5c78700 1 -- 172.21.15.119:0/4242850746 wait complete. 2023-12-07T10:10:09.210 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.615+0000 7f4bc5c78700 1 Processor -- start 2023-12-07T10:10:09.210 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.615+0000 7f4bc5c78700 1 -- start start 2023-12-07T10:10:09.210 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.615+0000 7f4bc5c78700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bc007ce70 0x7f4bc019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:09.210 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.615+0000 7f4bc5c78700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc019d100 con 0x7f4bc007ce70 2023-12-07T10:10:09.210 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.616+0000 7f4bbf7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bc007ce70 0x7f4bc019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:09.210 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.616+0000 7f4bbf7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bc007ce70 0x7f4bc019cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39114/0 (socket says 172.21.15.119:39114) 2023-12-07T10:10:09.210 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.616+0000 7f4bbf7fe700 1 -- 172.21.15.119:0/4153975978 learned_addr learned my addr 172.21.15.119:0/4153975978 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:09.210 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.616+0000 7f4bbf7fe700 1 -- 172.21.15.119:0/4153975978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb0009550 con 0x7f4bc007ce70 2023-12-07T10:10:09.210 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.616+0000 7f4bbf7fe700 1 --2- 172.21.15.119:0/4153975978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bc007ce70 0x7f4bc019cbc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0009130 tx=0x7f4bb0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:09.210 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.616+0000 7f4bbcff9700 1 -- 172.21.15.119:0/4153975978 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bb0003b70 con 0x7f4bc007ce70 2023-12-07T10:10:09.210 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.616+0000 7f4bc5c78700 1 -- 172.21.15.119:0/4153975978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc019d300 con 0x7f4bc007ce70 2023-12-07T10:10:09.210 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.617+0000 7f4bbcff9700 1 -- 172.21.15.119:0/4153975978 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4bb0003cd0 con 0x7f4bc007ce70 2023-12-07T10:10:09.210 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.617+0000 7f4bbcff9700 1 -- 172.21.15.119:0/4153975978 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bb00197d0 con 0x7f4bc007ce70 2023-12-07T10:10:09.211 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.617+0000 7f4bc5c78700 1 -- 172.21.15.119:0/4153975978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc019d7a0 con 0x7f4bc007ce70 2023-12-07T10:10:09.211 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.617+0000 7f4bbcff9700 1 -- 172.21.15.119:0/4153975978 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4bb0019a10 con 0x7f4bc007ce70 2023-12-07T10:10:09.211 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.617+0000 7f4bbcff9700 1 -- 172.21.15.119:0/4153975978 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4bb000b720 con 0x7f4bc007ce70 2023-12-07T10:10:09.211 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.618+0000 7f4bc5c78700 1 -- 172.21.15.119:0/4153975978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc010d070 con 0x7f4bc007ce70 2023-12-07T10:10:09.211 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.620+0000 7f4bbcff9700 1 -- 172.21.15.119:0/4153975978 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f4bb002a060 con 0x7f4bc007ce70 2023-12-07T10:10:09.211 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.697+0000 7f4bc5c78700 1 -- 172.21.15.119:0/4153975978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f4bc010d220 con 0x7f4bc007ce70 2023-12-07T10:10:09.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.697+0000 7f4bbcff9700 1 -- 172.21.15.119:0/4153975978 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f4bb0019e50 con 0x7f4bc007ce70 2023-12-07T10:10:09.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.698+0000 7f4bc5c78700 1 -- 172.21.15.119:0/4153975978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bc007ce70 msgr2=0x7f4bc019cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:09.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.698+0000 7f4bc5c78700 1 --2- 172.21.15.119:0/4153975978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bc007ce70 0x7f4bc019cbc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0009130 tx=0x7f4bb0011ad0 comp rx=0 tx=0).stop 2023-12-07T10:10:09.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.698+0000 7f4bc5c78700 1 -- 172.21.15.119:0/4153975978 shutdown_connections 2023-12-07T10:10:09.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.698+0000 7f4bc5c78700 1 --2- 172.21.15.119:0/4153975978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bc007ce70 0x7f4bc019cbc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:09.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.698+0000 7f4bc5c78700 1 -- 172.21.15.119:0/4153975978 >> 172.21.15.119:0/4153975978 conn(0x7f4bc0102220 msgr2=0x7f4bc0102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:09.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.699+0000 7f4bc5c78700 1 -- 172.21.15.119:0/4153975978 shutdown_connections 2023-12-07T10:10:09.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:08.699+0000 7f4bc5c78700 1 -- 172.21.15.119:0/4153975978 wait complete. 2023-12-07T10:10:09.213 INFO:teuthology.orchestra.run.smithi119.stdout:mgr not available, waiting (1/15)... 2023-12-07T10:10:09.335 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:09 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:09.049+0000 7f8cd0821200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-07T10:10:09.335 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:09 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:09.216+0000 7f8cd0821200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-07T10:10:09.335 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:09 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:09.332+0000 7f8cd0821200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-07T10:10:09.730 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:09 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:09.455+0000 7f8cd0821200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-07T10:10:10.480 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:10 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:09.999+0000 7f8cd0821200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-07T10:10:11.230 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:10 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:10.893+0000 7f8cd0821200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-07T10:10:11.230 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:11 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:10.999+0000 7f8cd0821200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-07T10:10:12.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:11 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/704264842' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-07T10:10:12.230 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:12 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:12.050+0000 7f8cd0821200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-07T10:10:12.231 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:12 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:12.182+0000 7f8cd0821200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-07T10:10:12.385 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2023-12-07T10:10:12.385 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2023-12-07T10:10:12.385 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsid": "a4c642c6-94e8-11ee-95a2-87774f69a715", 2023-12-07T10:10:12.386 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "health": { 2023-12-07T10:10:12.386 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-07T10:10:12.386 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-07T10:10:12.386 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-07T10:10:12.386 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-12-07T10:10:12.386 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-07T10:10:12.386 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-07T10:10:12.386 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 0 2023-12-07T10:10:12.386 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2023-12-07T10:10:12.386 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-07T10:10:12.386 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "a" 2023-12-07T10:10:12.386 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2023-12-07T10:10:12.386 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-12-07T10:10:12.386 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-07T10:10:12.386 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T10:10:12.387 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-07T10:10:12.387 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-07T10:10:12.387 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-12-07T10:10:12.387 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-07T10:10:12.387 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T10:10:12.387 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-07T10:10:12.387 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-07T10:10:12.398 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-07T10:10:12.398 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-07T10:10:12.398 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-07T10:10:12.399 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-07T10:10:12.399 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-12-07T10:10:12.399 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-07T10:10:12.399 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-07T10:10:12.399 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-07T10:10:12.399 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-07T10:10:12.399 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-07T10:10:12.399 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-07T10:10:12.399 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-07T10:10:12.399 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-07T10:10:12.399 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-07T10:10:12.399 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-12-07T10:10:12.399 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-07T10:10:12.399 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T10:10:12.399 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-07T10:10:12.400 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-07T10:10:12.400 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-12-07T10:10:12.400 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-07T10:10:12.400 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "available": false, 2023-12-07T10:10:12.400 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-07T10:10:12.400 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-07T10:10:12.400 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-07T10:10:12.400 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-07T10:10:12.400 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "restful" 2023-12-07T10:10:12.400 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2023-12-07T10:10:12.400 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-07T10:10:12.400 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-12-07T10:10:12.400 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-07T10:10:12.400 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T10:10:12.400 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modified": "2023-12-07T10:09:59.946493+0000", 2023-12-07T10:10:12.401 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-07T10:10:12.401 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-12-07T10:10:12.401 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-07T10:10:12.401 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2023-12-07T10:10:12.401 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.782+0000 7f134ee3c700 1 Processor -- start 2023-12-07T10:10:12.401 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.782+0000 7f134ee3c700 1 -- start start 2023-12-07T10:10:12.401 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.782+0000 7f134ee3c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f134807ce70 0x7f134807b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:12.401 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.782+0000 7f134ee3c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134807b8a0 con 0x7f134807ce70 2023-12-07T10:10:12.401 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.783+0000 7f134cbd8700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f134807ce70 0x7f134807b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:12.401 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.783+0000 7f134cbd8700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f134807ce70 0x7f134807b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39126/0 (socket says 172.21.15.119:39126) 2023-12-07T10:10:12.401 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.783+0000 7f134cbd8700 1 -- 172.21.15.119:0/3346231746 learned_addr learned my addr 172.21.15.119:0/3346231746 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:12.401 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.783+0000 7f134cbd8700 1 -- 172.21.15.119:0/3346231746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134807b9e0 con 0x7f134807ce70 2023-12-07T10:10:12.401 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.783+0000 7f134cbd8700 1 --2- 172.21.15.119:0/3346231746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f134807ce70 0x7f134807b2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f13440098a0 tx=0x7f1344009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416989fcea1f622e server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:12.401 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.784+0000 7f133f7fe700 1 -- 172.21.15.119:0/3346231746 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1344004160 con 0x7f134807ce70 2023-12-07T10:10:12.401 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.784+0000 7f133f7fe700 1 -- 172.21.15.119:0/3346231746 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f13440042c0 con 0x7f134807ce70 2023-12-07T10:10:12.402 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.784+0000 7f133f7fe700 1 -- 172.21.15.119:0/3346231746 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1344011a00 con 0x7f134807ce70 2023-12-07T10:10:12.402 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.784+0000 7f134ee3c700 1 -- 172.21.15.119:0/3346231746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f134807ce70 msgr2=0x7f134807b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:12.402 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.784+0000 7f134ee3c700 1 --2- 172.21.15.119:0/3346231746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f134807ce70 0x7f134807b2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f13440098a0 tx=0x7f1344009bb0 comp rx=0 tx=0).stop 2023-12-07T10:10:12.402 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.784+0000 7f134ee3c700 1 -- 172.21.15.119:0/3346231746 shutdown_connections 2023-12-07T10:10:12.402 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.784+0000 7f134ee3c700 1 --2- 172.21.15.119:0/3346231746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f134807ce70 0x7f134807b2d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:12.402 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.784+0000 7f134ee3c700 1 -- 172.21.15.119:0/3346231746 >> 172.21.15.119:0/3346231746 conn(0x7f1348102220 msgr2=0x7f1348104660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:12.402 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.784+0000 7f134ee3c700 1 -- 172.21.15.119:0/3346231746 shutdown_connections 2023-12-07T10:10:12.402 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.784+0000 7f134ee3c700 1 -- 172.21.15.119:0/3346231746 wait complete. 2023-12-07T10:10:12.402 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.785+0000 7f134ee3c700 1 Processor -- start 2023-12-07T10:10:12.402 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.786+0000 7f134ee3c700 1 -- start start 2023-12-07T10:10:12.402 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.786+0000 7f134ee3c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f134807ce70 0x7f134819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:12.402 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.786+0000 7f134ee3c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134819d100 con 0x7f134807ce70 2023-12-07T10:10:12.402 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.786+0000 7f134cbd8700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f134807ce70 0x7f134819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:12.402 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.786+0000 7f134cbd8700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f134807ce70 0x7f134819cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39136/0 (socket says 172.21.15.119:39136) 2023-12-07T10:10:12.403 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.786+0000 7f134cbd8700 1 -- 172.21.15.119:0/704264842 learned_addr learned my addr 172.21.15.119:0/704264842 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:12.403 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.787+0000 7f134cbd8700 1 -- 172.21.15.119:0/704264842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1344009550 con 0x7f134807ce70 2023-12-07T10:10:12.403 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.787+0000 7f134cbd8700 1 --2- 172.21.15.119:0/704264842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f134807ce70 0x7f134819cbc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1344009130 tx=0x7f1344011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:12.403 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.787+0000 7f133dffb700 1 -- 172.21.15.119:0/704264842 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1344003b70 con 0x7f134807ce70 2023-12-07T10:10:12.403 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.787+0000 7f133dffb700 1 -- 172.21.15.119:0/704264842 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1344003cd0 con 0x7f134807ce70 2023-12-07T10:10:12.403 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.787+0000 7f134ee3c700 1 -- 172.21.15.119:0/704264842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f134819d300 con 0x7f134807ce70 2023-12-07T10:10:12.403 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.787+0000 7f133dffb700 1 -- 172.21.15.119:0/704264842 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13440197d0 con 0x7f134807ce70 2023-12-07T10:10:12.403 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.787+0000 7f134ee3c700 1 -- 172.21.15.119:0/704264842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f134819d7a0 con 0x7f134807ce70 2023-12-07T10:10:12.403 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.788+0000 7f133dffb700 1 -- 172.21.15.119:0/704264842 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f13440199c0 con 0x7f134807ce70 2023-12-07T10:10:12.403 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.788+0000 7f133dffb700 1 -- 172.21.15.119:0/704264842 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f134400b720 con 0x7f134807ce70 2023-12-07T10:10:12.403 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.788+0000 7f134ee3c700 1 -- 172.21.15.119:0/704264842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f134810d070 con 0x7f134807ce70 2023-12-07T10:10:12.403 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.791+0000 7f133dffb700 1 -- 172.21.15.119:0/704264842 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f134402a060 con 0x7f134807ce70 2023-12-07T10:10:12.403 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.911+0000 7f134ee3c700 1 -- 172.21.15.119:0/704264842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f134810d310 con 0x7f134807ce70 2023-12-07T10:10:12.403 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.912+0000 7f133dffb700 1 -- 172.21.15.119:0/704264842 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f134810d310 con 0x7f134807ce70 2023-12-07T10:10:12.404 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.913+0000 7f134ee3c700 1 -- 172.21.15.119:0/704264842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f134807ce70 msgr2=0x7f134819cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:12.404 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.913+0000 7f134ee3c700 1 --2- 172.21.15.119:0/704264842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f134807ce70 0x7f134819cbc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1344009130 tx=0x7f1344011ad0 comp rx=0 tx=0).stop 2023-12-07T10:10:12.404 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.913+0000 7f134ee3c700 1 -- 172.21.15.119:0/704264842 shutdown_connections 2023-12-07T10:10:12.404 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.913+0000 7f134ee3c700 1 --2- 172.21.15.119:0/704264842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f134807ce70 0x7f134819cbc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:12.404 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.913+0000 7f134ee3c700 1 -- 172.21.15.119:0/704264842 >> 172.21.15.119:0/704264842 conn(0x7f1348102220 msgr2=0x7f1348102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:12.404 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.913+0000 7f134ee3c700 1 -- 172.21.15.119:0/704264842 shutdown_connections 2023-12-07T10:10:12.404 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:11.913+0000 7f134ee3c700 1 -- 172.21.15.119:0/704264842 wait complete. 2023-12-07T10:10:12.404 INFO:teuthology.orchestra.run.smithi119.stdout:mgr not available, waiting (2/15)... 2023-12-07T10:10:12.730 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:12 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:12.414+0000 7f8cd0821200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-07T10:10:13.230 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:12 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:12.934+0000 7f8cd0821200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-07T10:10:13.231 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:13 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:13.048+0000 7f8cd0821200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-07T10:10:13.592 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:13 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:13.242+0000 7f8cd0821200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-07T10:10:13.980 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:13 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:13.589+0000 7f8cd0821200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-07T10:10:14.903 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:14 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:14.544+0000 7f8cd0821200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-07T10:10:15.230 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:14 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:14.719+0000 7f8cd0821200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-07T10:10:15.230 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:14 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:14.968+0000 7f8cd0821200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-07T10:10:15.230 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:15 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:15.112+0000 7f8cd0821200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-07T10:10:15.730 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:15 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:15.399+0000 7f8cd0821200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-07T10:10:16.980 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:16 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:16.510+0000 7f8cd0821200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-07T10:10:16.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:16 smithi119 ceph-mon[97908]: Activating manager daemon x 2023-12-07T10:10:16.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:16 smithi119 ceph-mon[97908]: mgrmap e2: x(active, starting, since 0.00403154s) 2023-12-07T10:10:16.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:16 smithi119 ceph-mon[97908]: from='mgr.14100 172.21.15.119:0/2409277063' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-07T10:10:16.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:16 smithi119 ceph-mon[97908]: from='mgr.14100 172.21.15.119:0/2409277063' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-07T10:10:16.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:16 smithi119 ceph-mon[97908]: from='mgr.14100 172.21.15.119:0/2409277063' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-07T10:10:16.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:16 smithi119 ceph-mon[97908]: from='mgr.14100 172.21.15.119:0/2409277063' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T10:10:16.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:16 smithi119 ceph-mon[97908]: from='mgr.14100 172.21.15.119:0/2409277063' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-07T10:10:16.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:16 smithi119 ceph-mon[97908]: Manager daemon x is now available 2023-12-07T10:10:16.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:16 smithi119 ceph-mon[97908]: from='mgr.14100 172.21.15.119:0/2409277063' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-07T10:10:16.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:16 smithi119 ceph-mon[97908]: from='mgr.14100 172.21.15.119:0/2409277063' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-07T10:10:16.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:16 smithi119 ceph-mon[97908]: from='mgr.14100 172.21.15.119:0/2409277063' entity='mgr.x' 2023-12-07T10:10:16.982 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:16 smithi119 ceph-mon[97908]: from='mgr.14100 172.21.15.119:0/2409277063' entity='mgr.x' 2023-12-07T10:10:17.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:17 smithi119 ceph-mon[97908]: from='mgr.14100 172.21.15.119:0/2409277063' entity='mgr.x' 2023-12-07T10:10:17.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:17 smithi119 ceph-mon[97908]: mgrmap e3: x(active, since 1.00993s) 2023-12-07T10:10:19.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:19 smithi119 ceph-mon[97908]: mgrmap e4: x(active, since 2s) 2023-12-07T10:10:21.500 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2023-12-07T10:10:21.500 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2023-12-07T10:10:21.500 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsid": "a4c642c6-94e8-11ee-95a2-87774f69a715", 2023-12-07T10:10:21.500 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "health": { 2023-12-07T10:10:21.500 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-07T10:10:21.501 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-07T10:10:21.501 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-07T10:10:21.501 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-12-07T10:10:21.501 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-07T10:10:21.501 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-07T10:10:21.501 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 0 2023-12-07T10:10:21.501 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2023-12-07T10:10:21.501 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-07T10:10:21.501 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "a" 2023-12-07T10:10:21.501 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2023-12-07T10:10:21.501 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2023-12-07T10:10:21.501 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-07T10:10:21.501 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T10:10:21.502 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-07T10:10:21.502 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-07T10:10:21.502 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-12-07T10:10:21.502 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-07T10:10:21.502 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T10:10:21.502 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-07T10:10:21.503 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-07T10:10:21.503 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-07T10:10:21.511 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-07T10:10:21.511 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-07T10:10:21.511 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-07T10:10:21.511 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-12-07T10:10:21.511 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-07T10:10:21.511 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-07T10:10:21.511 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-07T10:10:21.511 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-07T10:10:21.511 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-07T10:10:21.511 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-07T10:10:21.511 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-07T10:10:21.511 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-07T10:10:21.511 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-07T10:10:21.512 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-12-07T10:10:21.512 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-07T10:10:21.512 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T10:10:21.512 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-07T10:10:21.512 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-07T10:10:21.512 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-12-07T10:10:21.512 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-07T10:10:21.512 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-07T10:10:21.512 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-07T10:10:21.512 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-07T10:10:21.512 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-07T10:10:21.512 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-07T10:10:21.512 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "restful" 2023-12-07T10:10:21.512 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2023-12-07T10:10:21.513 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-07T10:10:21.513 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-12-07T10:10:21.513 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-07T10:10:21.513 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-07T10:10:21.513 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modified": "2023-12-07T10:09:59.946493+0000", 2023-12-07T10:10:21.513 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-07T10:10:21.513 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-12-07T10:10:21.513 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-07T10:10:21.513 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2023-12-07T10:10:21.513 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.720+0000 7f1d68008700 1 Processor -- start 2023-12-07T10:10:21.513 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.720+0000 7f1d68008700 1 -- start start 2023-12-07T10:10:21.513 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.721+0000 7f1d68008700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d60106c60 0x7f1d60107080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:21.513 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.721+0000 7f1d68008700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d60107650 con 0x7f1d60106c60 2023-12-07T10:10:21.513 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.721+0000 7f1d65da4700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d60106c60 0x7f1d60107080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:21.514 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.721+0000 7f1d65da4700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d60106c60 0x7f1d60107080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41278/0 (socket says 172.21.15.119:41278) 2023-12-07T10:10:21.514 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.721+0000 7f1d65da4700 1 -- 172.21.15.119:0/877156583 learned_addr learned my addr 172.21.15.119:0/877156583 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:21.514 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.721+0000 7f1d65da4700 1 -- 172.21.15.119:0/877156583 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d60107e60 con 0x7f1d60106c60 2023-12-07T10:10:21.514 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.722+0000 7f1d65da4700 1 --2- 172.21.15.119:0/877156583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d60106c60 0x7f1d60107080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0098a0 tx=0x7f1d5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2c194fe1b40018 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:21.514 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.722+0000 7f1d64da2700 1 -- 172.21.15.119:0/877156583 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d5c004160 con 0x7f1d60106c60 2023-12-07T10:10:21.514 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.722+0000 7f1d64da2700 1 -- 172.21.15.119:0/877156583 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1d5c0042c0 con 0x7f1d60106c60 2023-12-07T10:10:21.514 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.722+0000 7f1d64da2700 1 -- 172.21.15.119:0/877156583 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d5c011a00 con 0x7f1d60106c60 2023-12-07T10:10:21.514 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.722+0000 7f1d68008700 1 -- 172.21.15.119:0/877156583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d60106c60 msgr2=0x7f1d60107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:21.514 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.722+0000 7f1d68008700 1 --2- 172.21.15.119:0/877156583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d60106c60 0x7f1d60107080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0098a0 tx=0x7f1d5c009bb0 comp rx=0 tx=0).stop 2023-12-07T10:10:21.514 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.723+0000 7f1d68008700 1 -- 172.21.15.119:0/877156583 shutdown_connections 2023-12-07T10:10:21.514 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.723+0000 7f1d68008700 1 --2- 172.21.15.119:0/877156583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d60106c60 0x7f1d60107080 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:21.514 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.723+0000 7f1d68008700 1 -- 172.21.15.119:0/877156583 >> 172.21.15.119:0/877156583 conn(0x7f1d60102220 msgr2=0x7f1d60104640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:21.514 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.723+0000 7f1d68008700 1 -- 172.21.15.119:0/877156583 shutdown_connections 2023-12-07T10:10:21.514 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.723+0000 7f1d68008700 1 -- 172.21.15.119:0/877156583 wait complete. 2023-12-07T10:10:21.515 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.724+0000 7f1d68008700 1 Processor -- start 2023-12-07T10:10:21.515 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.724+0000 7f1d68008700 1 -- start start 2023-12-07T10:10:21.515 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.724+0000 7f1d68008700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d60106c60 0x7f1d6019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:21.515 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.724+0000 7f1d68008700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6019d140 con 0x7f1d60106c60 2023-12-07T10:10:21.515 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.724+0000 7f1d65da4700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d60106c60 0x7f1d6019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:21.515 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.724+0000 7f1d65da4700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d60106c60 0x7f1d6019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41282/0 (socket says 172.21.15.119:41282) 2023-12-07T10:10:21.515 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.724+0000 7f1d65da4700 1 -- 172.21.15.119:0/376101148 learned_addr learned my addr 172.21.15.119:0/376101148 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:21.515 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.725+0000 7f1d65da4700 1 -- 172.21.15.119:0/376101148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5c009550 con 0x7f1d60106c60 2023-12-07T10:10:21.515 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.725+0000 7f1d65da4700 1 --2- 172.21.15.119:0/376101148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d60106c60 0x7f1d6019cc00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009130 tx=0x7f1d5c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:21.515 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.725+0000 7f1d56ffd700 1 -- 172.21.15.119:0/376101148 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d5c003ab0 con 0x7f1d60106c60 2023-12-07T10:10:21.515 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.725+0000 7f1d56ffd700 1 -- 172.21.15.119:0/376101148 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1d5c003c10 con 0x7f1d60106c60 2023-12-07T10:10:21.515 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.725+0000 7f1d68008700 1 -- 172.21.15.119:0/376101148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d6019d340 con 0x7f1d60106c60 2023-12-07T10:10:21.515 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.725+0000 7f1d56ffd700 1 -- 172.21.15.119:0/376101148 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d5c0247d0 con 0x7f1d60106c60 2023-12-07T10:10:21.516 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.725+0000 7f1d68008700 1 -- 172.21.15.119:0/376101148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d6019d7e0 con 0x7f1d60106c60 2023-12-07T10:10:21.516 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.726+0000 7f1d56ffd700 1 -- 172.21.15.119:0/376101148 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 4) v1 ==== 45728+0+0 (secure 0 0 0) 0x7f1d5c01abb0 con 0x7f1d60106c60 2023-12-07T10:10:21.516 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.726+0000 7f1d68008700 1 -- 172.21.15.119:0/376101148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d6010b090 con 0x7f1d60106c60 2023-12-07T10:10:21.516 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.727+0000 7f1d56ffd700 1 --2- 172.21.15.119:0/376101148 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7f1d4c038a00 0x7f1d4c03aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:21.516 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.727+0000 7f1d56ffd700 1 -- 172.21.15.119:0/376101148 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1d5c04c880 con 0x7f1d60106c60 2023-12-07T10:10:21.516 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.727+0000 7f1d655a3700 1 --2- 172.21.15.119:0/376101148 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7f1d4c038a00 0x7f1d4c03aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:21.516 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.727+0000 7f1d655a3700 1 --2- 172.21.15.119:0/376101148 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7f1d4c038a00 0x7f1d4c03aec0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1d50006fd0 tx=0x7f1d50006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:21.516 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:20.732+0000 7f1d56ffd700 1 -- 172.21.15.119:0/376101148 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1d5c018530 con 0x7f1d60106c60 2023-12-07T10:10:21.516 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:21.006+0000 7f1d68008700 1 -- 172.21.15.119:0/376101148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f1d601a0070 con 0x7f1d60106c60 2023-12-07T10:10:21.516 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:21.006+0000 7f1d56ffd700 1 -- 172.21.15.119:0/376101148 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f1d5c018350 con 0x7f1d60106c60 2023-12-07T10:10:21.516 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:21.008+0000 7f1d68008700 1 -- 172.21.15.119:0/376101148 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7f1d4c038a00 msgr2=0x7f1d4c03aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:21.516 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:21.008+0000 7f1d68008700 1 --2- 172.21.15.119:0/376101148 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7f1d4c038a00 0x7f1d4c03aec0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1d50006fd0 tx=0x7f1d50006e40 comp rx=0 tx=0).stop 2023-12-07T10:10:21.516 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:21.008+0000 7f1d68008700 1 -- 172.21.15.119:0/376101148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d60106c60 msgr2=0x7f1d6019cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:21.516 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:21.008+0000 7f1d68008700 1 --2- 172.21.15.119:0/376101148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d60106c60 0x7f1d6019cc00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009130 tx=0x7f1d5c011ad0 comp rx=0 tx=0).stop 2023-12-07T10:10:21.516 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:21.009+0000 7f1d68008700 1 -- 172.21.15.119:0/376101148 shutdown_connections 2023-12-07T10:10:21.517 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:21.009+0000 7f1d68008700 1 --2- 172.21.15.119:0/376101148 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7f1d4c038a00 0x7f1d4c03aec0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:21.517 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:21.009+0000 7f1d68008700 1 --2- 172.21.15.119:0/376101148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d60106c60 0x7f1d6019cc00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:21.517 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:21.009+0000 7f1d68008700 1 -- 172.21.15.119:0/376101148 >> 172.21.15.119:0/376101148 conn(0x7f1d60102220 msgr2=0x7f1d60102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:21.517 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:21.009+0000 7f1d68008700 1 -- 172.21.15.119:0/376101148 shutdown_connections 2023-12-07T10:10:21.517 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:21.009+0000 7f1d68008700 1 -- 172.21.15.119:0/376101148 wait complete. 2023-12-07T10:10:21.517 INFO:teuthology.orchestra.run.smithi119.stdout:mgr is available 2023-12-07T10:10:21.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:21 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/376101148' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-07T10:10:22.681 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2023-12-07T10:10:22.688 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout [global] 2023-12-07T10:10:22.688 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout fsid = a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:10:22.688 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-12-07T10:10:22.688 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-12-07T10:10:22.688 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-12-07T10:10:22.688 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-12-07T10:10:22.688 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2023-12-07T10:10:22.688 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout [mgr] 2023-12-07T10:10:22.688 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-12-07T10:10:22.688 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2023-12-07T10:10:22.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout [osd] 2023-12-07T10:10:22.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-12-07T10:10:22.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-12-07T10:10:22.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-12-07T10:10:22.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-12-07T10:10:22.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-12-07T10:10:22.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.030+0000 7fea7e293700 1 Processor -- start 2023-12-07T10:10:22.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.030+0000 7fea7e293700 1 -- start start 2023-12-07T10:10:22.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.030+0000 7fea7e293700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea7807ce50 0x7fea7807b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:22.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.030+0000 7fea7e293700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7807b880 con 0x7fea7807ce50 2023-12-07T10:10:22.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.030+0000 7fea77fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea7807ce50 0x7fea7807b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:22.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.031+0000 7fea77fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea7807ce50 0x7fea7807b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41290/0 (socket says 172.21.15.119:41290) 2023-12-07T10:10:22.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.031+0000 7fea77fff700 1 -- 172.21.15.119:0/3429006153 learned_addr learned my addr 172.21.15.119:0/3429006153 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:22.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.031+0000 7fea77fff700 1 -- 172.21.15.119:0/3429006153 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7807b9c0 con 0x7fea7807ce50 2023-12-07T10:10:22.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.031+0000 7fea77fff700 1 --2- 172.21.15.119:0/3429006153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea7807ce50 0x7fea7807b2b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fea680098a0 tx=0x7fea68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3296181d6df2f14 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:22.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.031+0000 7fea76ffd700 1 -- 172.21.15.119:0/3429006153 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea68004160 con 0x7fea7807ce50 2023-12-07T10:10:22.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.031+0000 7fea76ffd700 1 -- 172.21.15.119:0/3429006153 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fea680042c0 con 0x7fea7807ce50 2023-12-07T10:10:22.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.031+0000 7fea76ffd700 1 -- 172.21.15.119:0/3429006153 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea68011a00 con 0x7fea7807ce50 2023-12-07T10:10:22.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.032+0000 7fea7e293700 1 -- 172.21.15.119:0/3429006153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea7807ce50 msgr2=0x7fea7807b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:22.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.032+0000 7fea7e293700 1 --2- 172.21.15.119:0/3429006153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea7807ce50 0x7fea7807b2b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fea680098a0 tx=0x7fea68009bb0 comp rx=0 tx=0).stop 2023-12-07T10:10:22.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.032+0000 7fea7e293700 1 -- 172.21.15.119:0/3429006153 shutdown_connections 2023-12-07T10:10:22.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.032+0000 7fea7e293700 1 --2- 172.21.15.119:0/3429006153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea7807ce50 0x7fea7807b2b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:22.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.032+0000 7fea7e293700 1 -- 172.21.15.119:0/3429006153 >> 172.21.15.119:0/3429006153 conn(0x7fea78102220 msgr2=0x7fea78104640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:22.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.032+0000 7fea7e293700 1 -- 172.21.15.119:0/3429006153 shutdown_connections 2023-12-07T10:10:22.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.032+0000 7fea7e293700 1 -- 172.21.15.119:0/3429006153 wait complete. 2023-12-07T10:10:22.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.033+0000 7fea7e293700 1 Processor -- start 2023-12-07T10:10:22.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.033+0000 7fea7e293700 1 -- start start 2023-12-07T10:10:22.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.033+0000 7fea7e293700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea7807ce50 0x7fea7819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:22.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.034+0000 7fea7e293700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7819d0c0 con 0x7fea7807ce50 2023-12-07T10:10:22.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.034+0000 7fea77fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea7807ce50 0x7fea7819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:22.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.034+0000 7fea77fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea7807ce50 0x7fea7819cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41292/0 (socket says 172.21.15.119:41292) 2023-12-07T10:10:22.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.034+0000 7fea77fff700 1 -- 172.21.15.119:0/2102499312 learned_addr learned my addr 172.21.15.119:0/2102499312 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:22.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.034+0000 7fea77fff700 1 -- 172.21.15.119:0/2102499312 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea68009550 con 0x7fea7807ce50 2023-12-07T10:10:22.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.034+0000 7fea77fff700 1 --2- 172.21.15.119:0/2102499312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea7807ce50 0x7fea7819cb80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fea68009fb0 tx=0x7fea68011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:22.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.034+0000 7fea757fa700 1 -- 172.21.15.119:0/2102499312 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea68003a40 con 0x7fea7807ce50 2023-12-07T10:10:22.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.035+0000 7fea757fa700 1 -- 172.21.15.119:0/2102499312 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fea68003ba0 con 0x7fea7807ce50 2023-12-07T10:10:22.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.035+0000 7fea757fa700 1 -- 172.21.15.119:0/2102499312 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea6801a750 con 0x7fea7807ce50 2023-12-07T10:10:22.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.035+0000 7fea7e293700 1 -- 172.21.15.119:0/2102499312 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea7819d2c0 con 0x7fea7807ce50 2023-12-07T10:10:22.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.035+0000 7fea7e293700 1 -- 172.21.15.119:0/2102499312 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea7819d760 con 0x7fea7807ce50 2023-12-07T10:10:22.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.036+0000 7fea757fa700 1 -- 172.21.15.119:0/2102499312 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 4) v1 ==== 45728+0+0 (secure 0 0 0) 0x7fea68024430 con 0x7fea7807ce50 2023-12-07T10:10:22.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.036+0000 7fea7e293700 1 -- 172.21.15.119:0/2102499312 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea7810d010 con 0x7fea7807ce50 2023-12-07T10:10:22.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.036+0000 7fea757fa700 1 --2- 172.21.15.119:0/2102499312 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7fea60038a00 0x7fea6003aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:22.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.036+0000 7fea757fa700 1 -- 172.21.15.119:0/2102499312 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fea6804c6f0 con 0x7fea7807ce50 2023-12-07T10:10:22.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.036+0000 7fea777fe700 1 --2- 172.21.15.119:0/2102499312 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7fea60038a00 0x7fea6003aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:22.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.037+0000 7fea777fe700 1 --2- 172.21.15.119:0/2102499312 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7fea60038a00 0x7fea6003aec0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fea6c006fd0 tx=0x7fea6c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:22.692 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.041+0000 7fea757fa700 1 -- 172.21.15.119:0/2102499312 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fea68024b60 con 0x7fea7807ce50 2023-12-07T10:10:22.692 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.239+0000 7fea7e293700 1 -- 172.21.15.119:0/2102499312 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fea7810d250 con 0x7fea7807ce50 2023-12-07T10:10:22.692 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.240+0000 7fea757fa700 1 -- 172.21.15.119:0/2102499312 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fea68018350 con 0x7fea7807ce50 2023-12-07T10:10:22.692 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.243+0000 7fea7e293700 1 -- 172.21.15.119:0/2102499312 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7fea60038a00 msgr2=0x7fea6003aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:22.692 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.243+0000 7fea7e293700 1 --2- 172.21.15.119:0/2102499312 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7fea60038a00 0x7fea6003aec0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fea6c006fd0 tx=0x7fea6c006e40 comp rx=0 tx=0).stop 2023-12-07T10:10:22.692 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.243+0000 7fea7e293700 1 -- 172.21.15.119:0/2102499312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea7807ce50 msgr2=0x7fea7819cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:22.692 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.243+0000 7fea7e293700 1 --2- 172.21.15.119:0/2102499312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea7807ce50 0x7fea7819cb80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fea68009fb0 tx=0x7fea68011ad0 comp rx=0 tx=0).stop 2023-12-07T10:10:22.692 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.243+0000 7fea7e293700 1 -- 172.21.15.119:0/2102499312 shutdown_connections 2023-12-07T10:10:22.692 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.243+0000 7fea7e293700 1 --2- 172.21.15.119:0/2102499312 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7fea60038a00 0x7fea6003aec0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:22.692 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.243+0000 7fea7e293700 1 --2- 172.21.15.119:0/2102499312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea7807ce50 0x7fea7819cb80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:22.692 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.243+0000 7fea7e293700 1 -- 172.21.15.119:0/2102499312 >> 172.21.15.119:0/2102499312 conn(0x7fea78102220 msgr2=0x7fea78102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:22.692 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.243+0000 7fea7e293700 1 -- 172.21.15.119:0/2102499312 shutdown_connections 2023-12-07T10:10:22.692 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:22.243+0000 7fea7e293700 1 -- 172.21.15.119:0/2102499312 wait complete. 2023-12-07T10:10:22.692 INFO:teuthology.orchestra.run.smithi119.stdout:Enabling cephadm module... 2023-12-07T10:10:22.722 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:22 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2102499312' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-12-07T10:10:23.795 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:23 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/883142978' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-12-07T10:10:23.796 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:23 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: ignoring --setuser ceph since I am not root 2023-12-07T10:10:23.796 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:23 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: ignoring --setgroup ceph since I am not root 2023-12-07T10:10:23.796 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:23 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:23.791+0000 7fa8a5769200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-07T10:10:23.978 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.212+0000 7f56b00c0700 1 Processor -- start 2023-12-07T10:10:23.978 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.213+0000 7f56b00c0700 1 -- start start 2023-12-07T10:10:23.978 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.213+0000 7f56b00c0700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56a8106c60 0x7f56a8107080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:23.978 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.213+0000 7f56b00c0700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a8107650 con 0x7f56a8106c60 2023-12-07T10:10:23.978 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.213+0000 7f56ade5c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56a8106c60 0x7f56a8107080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.213+0000 7f56ade5c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56a8106c60 0x7f56a8107080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43948/0 (socket says 172.21.15.119:43948) 2023-12-07T10:10:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.213+0000 7f56ade5c700 1 -- 172.21.15.119:0/1609161643 learned_addr learned my addr 172.21.15.119:0/1609161643 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.214+0000 7f56ade5c700 1 -- 172.21.15.119:0/1609161643 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a8107e60 con 0x7f56a8106c60 2023-12-07T10:10:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.214+0000 7f56ade5c700 1 --2- 172.21.15.119:0/1609161643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56a8106c60 0x7f56a8107080 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f56a40098a0 tx=0x7f56a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ffa74d4c86d673 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.214+0000 7f56ace5a700 1 -- 172.21.15.119:0/1609161643 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56a4004160 con 0x7f56a8106c60 2023-12-07T10:10:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.214+0000 7f56ace5a700 1 -- 172.21.15.119:0/1609161643 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f56a40042c0 con 0x7f56a8106c60 2023-12-07T10:10:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.214+0000 7f56ace5a700 1 -- 172.21.15.119:0/1609161643 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56a4011a00 con 0x7f56a8106c60 2023-12-07T10:10:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.215+0000 7f56b00c0700 1 -- 172.21.15.119:0/1609161643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56a8106c60 msgr2=0x7f56a8107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.215+0000 7f56b00c0700 1 --2- 172.21.15.119:0/1609161643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56a8106c60 0x7f56a8107080 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f56a40098a0 tx=0x7f56a4009bb0 comp rx=0 tx=0).stop 2023-12-07T10:10:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.215+0000 7f56b00c0700 1 -- 172.21.15.119:0/1609161643 shutdown_connections 2023-12-07T10:10:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.215+0000 7f56b00c0700 1 --2- 172.21.15.119:0/1609161643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56a8106c60 0x7f56a8107080 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.215+0000 7f56b00c0700 1 -- 172.21.15.119:0/1609161643 >> 172.21.15.119:0/1609161643 conn(0x7f56a8102220 msgr2=0x7f56a8104640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.215+0000 7f56b00c0700 1 -- 172.21.15.119:0/1609161643 shutdown_connections 2023-12-07T10:10:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.215+0000 7f56b00c0700 1 -- 172.21.15.119:0/1609161643 wait complete. 2023-12-07T10:10:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.216+0000 7f56b00c0700 1 Processor -- start 2023-12-07T10:10:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.216+0000 7f56b00c0700 1 -- start start 2023-12-07T10:10:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.216+0000 7f56b00c0700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56a8106c60 0x7f56a819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.216+0000 7f56b00c0700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a819af10 con 0x7f56a8106c60 2023-12-07T10:10:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.217+0000 7f56ade5c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56a8106c60 0x7f56a819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.217+0000 7f56ade5c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56a8106c60 0x7f56a819a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43954/0 (socket says 172.21.15.119:43954) 2023-12-07T10:10:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.217+0000 7f56ade5c700 1 -- 172.21.15.119:0/883142978 learned_addr learned my addr 172.21.15.119:0/883142978 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.217+0000 7f56ade5c700 1 -- 172.21.15.119:0/883142978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a4009550 con 0x7f56a8106c60 2023-12-07T10:10:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.217+0000 7f56ade5c700 1 --2- 172.21.15.119:0/883142978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56a8106c60 0x7f56a819a9d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f56a4009130 tx=0x7f56a4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.217+0000 7f569effd700 1 -- 172.21.15.119:0/883142978 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56a4003ab0 con 0x7f56a8106c60 2023-12-07T10:10:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.218+0000 7f569effd700 1 -- 172.21.15.119:0/883142978 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f56a4003c10 con 0x7f56a8106c60 2023-12-07T10:10:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.218+0000 7f56b00c0700 1 -- 172.21.15.119:0/883142978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a819b110 con 0x7f56a8106c60 2023-12-07T10:10:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.218+0000 7f569effd700 1 -- 172.21.15.119:0/883142978 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56a40247d0 con 0x7f56a8106c60 2023-12-07T10:10:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.218+0000 7f56b00c0700 1 -- 172.21.15.119:0/883142978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a819b5b0 con 0x7f56a8106c60 2023-12-07T10:10:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.219+0000 7f569effd700 1 -- 172.21.15.119:0/883142978 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 4) v1 ==== 45728+0+0 (secure 0 0 0) 0x7f56a401a430 con 0x7f56a8106c60 2023-12-07T10:10:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.219+0000 7f56b00c0700 1 -- 172.21.15.119:0/883142978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56a810b090 con 0x7f56a8106c60 2023-12-07T10:10:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.219+0000 7f569effd700 1 --2- 172.21.15.119:0/883142978 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7f5694038a00 0x7f569403aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.219+0000 7f569effd700 1 -- 172.21.15.119:0/883142978 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f56a404c870 con 0x7f56a8106c60 2023-12-07T10:10:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.219+0000 7f56ad65b700 1 --2- 172.21.15.119:0/883142978 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7f5694038a00 0x7f569403aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.220+0000 7f56ad65b700 1 --2- 172.21.15.119:0/883142978 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7f5694038a00 0x7f569403aec0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5698006fd0 tx=0x7f5698006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.224+0000 7f569effd700 1 -- 172.21.15.119:0/883142978 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f56a4018530 con 0x7f56a8106c60 2023-12-07T10:10:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.474+0000 7f56b00c0700 1 -- 172.21.15.119:0/883142978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f56a819de40 con 0x7f56a8106c60 2023-12-07T10:10:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.526+0000 7f569effd700 1 -- 172.21.15.119:0/883142978 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f56a4018350 con 0x7f56a8106c60 2023-12-07T10:10:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.528+0000 7f569effd700 1 -- 172.21.15.119:0/883142978 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mgrmap(e 5) v1 ==== 45739+0+0 (secure 0 0 0) 0x7f56a404ecf0 con 0x7f56a8106c60 2023-12-07T10:10:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.530+0000 7f56b00c0700 1 -- 172.21.15.119:0/883142978 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7f5694038a00 msgr2=0x7f569403aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.530+0000 7f56b00c0700 1 --2- 172.21.15.119:0/883142978 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7f5694038a00 0x7f569403aec0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5698006fd0 tx=0x7f5698006e40 comp rx=0 tx=0).stop 2023-12-07T10:10:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.530+0000 7f56b00c0700 1 -- 172.21.15.119:0/883142978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56a8106c60 msgr2=0x7f56a819a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:23.982 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.530+0000 7f56b00c0700 1 --2- 172.21.15.119:0/883142978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56a8106c60 0x7f56a819a9d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f56a4009130 tx=0x7f56a4011ad0 comp rx=0 tx=0).stop 2023-12-07T10:10:23.982 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.530+0000 7f56b00c0700 1 -- 172.21.15.119:0/883142978 shutdown_connections 2023-12-07T10:10:23.982 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.530+0000 7f56b00c0700 1 --2- 172.21.15.119:0/883142978 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7f5694038a00 0x7f569403aec0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:23.982 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.530+0000 7f56b00c0700 1 --2- 172.21.15.119:0/883142978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56a8106c60 0x7f56a819a9d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:23.982 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.530+0000 7f56b00c0700 1 -- 172.21.15.119:0/883142978 >> 172.21.15.119:0/883142978 conn(0x7f56a8102220 msgr2=0x7f56a8102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:23.982 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.530+0000 7f56b00c0700 1 -- 172.21.15.119:0/883142978 shutdown_connections 2023-12-07T10:10:23.982 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:23.530+0000 7f56b00c0700 1 -- 172.21.15.119:0/883142978 wait complete. 2023-12-07T10:10:24.159 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:23 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:23.896+0000 7fa8a5769200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-07T10:10:24.159 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:24 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:24.025+0000 7fa8a5769200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-07T10:10:24.480 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:24 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:24.155+0000 7fa8a5769200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-07T10:10:24.480 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:24 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:24.390+0000 7fa8a5769200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-07T10:10:24.940 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:24 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/883142978' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-12-07T10:10:24.940 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:24 smithi119 ceph-mon[97908]: mgrmap e5: x(active, since 7s) 2023-12-07T10:10:25.230 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:24 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:24.935+0000 7fa8a5769200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-07T10:10:25.230 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:25 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:25.109+0000 7fa8a5769200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-07T10:10:25.536 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:25 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:25.229+0000 7fa8a5769200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-07T10:10:25.536 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:25 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:25.355+0000 7fa8a5769200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-07T10:10:25.675 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2023-12-07T10:10:25.675 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-12-07T10:10:25.675 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-07T10:10:25.675 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-12-07T10:10:25.675 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-12-07T10:10:25.676 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2023-12-07T10:10:25.676 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.586+0000 7fdd73922700 1 Processor -- start 2023-12-07T10:10:25.676 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.586+0000 7fdd73922700 1 -- start start 2023-12-07T10:10:25.676 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.586+0000 7fdd73922700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd6c07ce70 0x7fdd6c07b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:25.676 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.586+0000 7fdd73922700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c07b8a0 con 0x7fdd6c07ce70 2023-12-07T10:10:25.676 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.586+0000 7fdd716be700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd6c07ce70 0x7fdd6c07b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:25.676 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.586+0000 7fdd716be700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd6c07ce70 0x7fdd6c07b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43974/0 (socket says 172.21.15.119:43974) 2023-12-07T10:10:25.676 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.586+0000 7fdd716be700 1 -- 172.21.15.119:0/3925610449 learned_addr learned my addr 172.21.15.119:0/3925610449 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:25.676 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.587+0000 7fdd716be700 1 -- 172.21.15.119:0/3925610449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c07b9e0 con 0x7fdd6c07ce70 2023-12-07T10:10:25.676 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.587+0000 7fdd716be700 1 --2- 172.21.15.119:0/3925610449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd6c07ce70 0x7fdd6c07b2d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdd680098a0 tx=0x7fdd68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56150434233b31ed server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:25.677 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.587+0000 7fdd63fff700 1 -- 172.21.15.119:0/3925610449 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd68004160 con 0x7fdd6c07ce70 2023-12-07T10:10:25.677 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.587+0000 7fdd63fff700 1 -- 172.21.15.119:0/3925610449 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdd680042c0 con 0x7fdd6c07ce70 2023-12-07T10:10:25.677 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.587+0000 7fdd63fff700 1 -- 172.21.15.119:0/3925610449 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd68011a00 con 0x7fdd6c07ce70 2023-12-07T10:10:25.677 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.588+0000 7fdd73922700 1 -- 172.21.15.119:0/3925610449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd6c07ce70 msgr2=0x7fdd6c07b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:25.677 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.588+0000 7fdd73922700 1 --2- 172.21.15.119:0/3925610449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd6c07ce70 0x7fdd6c07b2d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdd680098a0 tx=0x7fdd68009bb0 comp rx=0 tx=0).stop 2023-12-07T10:10:25.677 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.588+0000 7fdd73922700 1 -- 172.21.15.119:0/3925610449 shutdown_connections 2023-12-07T10:10:25.677 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.588+0000 7fdd73922700 1 --2- 172.21.15.119:0/3925610449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd6c07ce70 0x7fdd6c07b2d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:25.677 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.588+0000 7fdd73922700 1 -- 172.21.15.119:0/3925610449 >> 172.21.15.119:0/3925610449 conn(0x7fdd6c102220 msgr2=0x7fdd6c104640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:25.677 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.588+0000 7fdd73922700 1 -- 172.21.15.119:0/3925610449 shutdown_connections 2023-12-07T10:10:25.677 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.588+0000 7fdd73922700 1 -- 172.21.15.119:0/3925610449 wait complete. 2023-12-07T10:10:25.677 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.590+0000 7fdd73922700 1 Processor -- start 2023-12-07T10:10:25.677 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.590+0000 7fdd73922700 1 -- start start 2023-12-07T10:10:25.677 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.590+0000 7fdd73922700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd6c07ce70 0x7fdd6c19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:25.678 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.590+0000 7fdd73922700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c19d0b0 con 0x7fdd6c07ce70 2023-12-07T10:10:25.678 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.591+0000 7fdd716be700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd6c07ce70 0x7fdd6c19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:25.678 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.591+0000 7fdd716be700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd6c07ce70 0x7fdd6c19cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43978/0 (socket says 172.21.15.119:43978) 2023-12-07T10:10:25.678 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.591+0000 7fdd716be700 1 -- 172.21.15.119:0/3445475674 learned_addr learned my addr 172.21.15.119:0/3445475674 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:25.678 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.591+0000 7fdd716be700 1 -- 172.21.15.119:0/3445475674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd68009550 con 0x7fdd6c07ce70 2023-12-07T10:10:25.678 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.591+0000 7fdd716be700 1 --2- 172.21.15.119:0/3445475674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd6c07ce70 0x7fdd6c19cb70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdd68009130 tx=0x7fdd68011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:25.678 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.591+0000 7fdd627fc700 1 -- 172.21.15.119:0/3445475674 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd68003b70 con 0x7fdd6c07ce70 2023-12-07T10:10:25.678 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.592+0000 7fdd627fc700 1 -- 172.21.15.119:0/3445475674 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdd68003cd0 con 0x7fdd6c07ce70 2023-12-07T10:10:25.678 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.592+0000 7fdd73922700 1 -- 172.21.15.119:0/3445475674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6c19d2b0 con 0x7fdd6c07ce70 2023-12-07T10:10:25.678 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.592+0000 7fdd627fc700 1 -- 172.21.15.119:0/3445475674 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd680197d0 con 0x7fdd6c07ce70 2023-12-07T10:10:25.678 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.592+0000 7fdd73922700 1 -- 172.21.15.119:0/3445475674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6c19d750 con 0x7fdd6c07ce70 2023-12-07T10:10:25.678 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.593+0000 7fdd627fc700 1 -- 172.21.15.119:0/3445475674 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 45739+0+0 (secure 0 0 0) 0x7fdd6801abb0 con 0x7fdd6c07ce70 2023-12-07T10:10:25.679 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.593+0000 7fdd73922700 1 -- 172.21.15.119:0/3445475674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd6c10d050 con 0x7fdd6c07ce70 2023-12-07T10:10:25.679 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.593+0000 7fdd627fc700 1 --2- 172.21.15.119:0/3445475674 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7fdd58038aa0 0x7fdd5803af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:25.679 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.593+0000 7fdd627fc700 1 -- 172.21.15.119:0/3445475674 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdd6804cd30 con 0x7fdd6c07ce70 2023-12-07T10:10:25.679 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.593+0000 7fdd70ebd700 1 -- 172.21.15.119:0/3445475674 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7fdd58038aa0 msgr2=0x7fdd5803af60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/3515858087 2023-12-07T10:10:25.679 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.593+0000 7fdd70ebd700 1 --2- 172.21.15.119:0/3445475674 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7fdd58038aa0 0x7fdd5803af60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-07T10:10:25.679 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.599+0000 7fdd627fc700 1 -- 172.21.15.119:0/3445475674 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fdd6802db40 con 0x7fdd6c07ce70 2023-12-07T10:10:25.679 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.794+0000 7fdd70ebd700 1 -- 172.21.15.119:0/3445475674 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7fdd58038aa0 msgr2=0x7fdd5803af60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/3515858087 2023-12-07T10:10:25.679 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.794+0000 7fdd70ebd700 1 --2- 172.21.15.119:0/3445475674 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7fdd58038aa0 0x7fdd5803af60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-07T10:10:25.679 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.948+0000 7fdd73922700 1 -- 172.21.15.119:0/3445475674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fdd6c19ffe0 con 0x7fdd6c07ce70 2023-12-07T10:10:25.679 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.948+0000 7fdd627fc700 1 -- 172.21.15.119:0/3445475674 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fdd6802db40 con 0x7fdd6c07ce70 2023-12-07T10:10:25.680 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.951+0000 7fdd73922700 1 -- 172.21.15.119:0/3445475674 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7fdd58038aa0 msgr2=0x7fdd5803af60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-07T10:10:25.680 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.951+0000 7fdd73922700 1 --2- 172.21.15.119:0/3445475674 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7fdd58038aa0 0x7fdd5803af60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:25.680 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.951+0000 7fdd73922700 1 -- 172.21.15.119:0/3445475674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd6c07ce70 msgr2=0x7fdd6c19cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:25.680 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.951+0000 7fdd73922700 1 --2- 172.21.15.119:0/3445475674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd6c07ce70 0x7fdd6c19cb70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdd68009130 tx=0x7fdd68011ad0 comp rx=0 tx=0).stop 2023-12-07T10:10:25.680 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.951+0000 7fdd73922700 1 -- 172.21.15.119:0/3445475674 shutdown_connections 2023-12-07T10:10:25.680 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.951+0000 7fdd73922700 1 --2- 172.21.15.119:0/3445475674 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7fdd58038aa0 0x7fdd5803af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:25.680 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.951+0000 7fdd73922700 1 --2- 172.21.15.119:0/3445475674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd6c07ce70 0x7fdd6c19cb70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:25.680 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.951+0000 7fdd73922700 1 -- 172.21.15.119:0/3445475674 >> 172.21.15.119:0/3445475674 conn(0x7fdd6c102220 msgr2=0x7fdd6c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:25.680 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.951+0000 7fdd73922700 1 -- 172.21.15.119:0/3445475674 shutdown_connections 2023-12-07T10:10:25.681 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:24.951+0000 7fdd73922700 1 -- 172.21.15.119:0/3445475674 wait complete. 2023-12-07T10:10:25.681 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for the mgr to restart... 2023-12-07T10:10:25.681 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for mgr epoch 5... 2023-12-07T10:10:25.915 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:25 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3445475674' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-12-07T10:10:26.230 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:25 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:25.911+0000 7fa8a5769200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-07T10:10:27.230 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:26 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:26.783+0000 7fa8a5769200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-07T10:10:27.230 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:26 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:26.889+0000 7fa8a5769200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-07T10:10:28.183 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:27 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:27.837+0000 7fa8a5769200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-07T10:10:28.183 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:27 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:27.957+0000 7fa8a5769200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-07T10:10:28.183 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:28 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:28.179+0000 7fa8a5769200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-07T10:10:28.980 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:28 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:28.703+0000 7fa8a5769200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-07T10:10:28.980 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:28 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:28.818+0000 7fa8a5769200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-07T10:10:29.354 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:29 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:29.009+0000 7fa8a5769200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-07T10:10:29.730 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:29 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:29.351+0000 7fa8a5769200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-07T10:10:30.697 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:30 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:30.271+0000 7fa8a5769200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-07T10:10:30.697 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:30 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:30.444+0000 7fa8a5769200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-07T10:10:30.697 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:30 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:30.694+0000 7fa8a5769200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-07T10:10:30.980 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:30 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:30.838+0000 7fa8a5769200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-07T10:10:31.480 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:31 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:31.113+0000 7fa8a5769200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-07T10:10:32.480 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:32 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:32.175+0000 7fa8a5769200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-07T10:10:32.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:32 smithi119 ceph-mon[97908]: Active manager daemon x restarted 2023-12-07T10:10:32.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:32 smithi119 ceph-mon[97908]: Activating manager daemon x 2023-12-07T10:10:32.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:32 smithi119 ceph-mon[97908]: osdmap e2: 0 total, 0 up, 0 in 2023-12-07T10:10:32.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:32 smithi119 ceph-mon[97908]: mgrmap e6: x(active, starting, since 0.00352969s) 2023-12-07T10:10:32.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:32 smithi119 ceph-mon[97908]: from='mgr.14118 172.21.15.119:0/725582431' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T10:10:32.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:32 smithi119 ceph-mon[97908]: from='mgr.14118 172.21.15.119:0/725582431' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-07T10:10:32.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:32 smithi119 ceph-mon[97908]: from='mgr.14118 172.21.15.119:0/725582431' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-07T10:10:32.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:32 smithi119 ceph-mon[97908]: from='mgr.14118 172.21.15.119:0/725582431' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-07T10:10:32.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:32 smithi119 ceph-mon[97908]: from='mgr.14118 172.21.15.119:0/725582431' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-07T10:10:32.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:32 smithi119 ceph-mon[97908]: Manager daemon x is now available 2023-12-07T10:10:32.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:32 smithi119 ceph-mon[97908]: from='mgr.14118 172.21.15.119:0/725582431' entity='mgr.x' 2023-12-07T10:10:32.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:32 smithi119 ceph-mon[97908]: from='mgr.14118 172.21.15.119:0/725582431' entity='mgr.x' 2023-12-07T10:10:32.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:32 smithi119 ceph-mon[97908]: from='mgr.14118 172.21.15.119:0/725582431' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:10:32.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:32 smithi119 ceph-mon[97908]: from='mgr.14118 172.21.15.119:0/725582431' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:10:33.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:33 smithi119 ceph-mon[97908]: Found migration_current of "None". Setting to last migration. 2023-12-07T10:10:33.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:33 smithi119 ceph-mon[97908]: from='mgr.14118 172.21.15.119:0/725582431' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:10:33.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:33 smithi119 ceph-mon[97908]: from='mgr.14118 172.21.15.119:0/725582431' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-07T10:10:33.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:33 smithi119 ceph-mon[97908]: from='mgr.14118 172.21.15.119:0/725582431' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-07T10:10:33.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:33 smithi119 ceph-mon[97908]: from='mgr.14118 172.21.15.119:0/725582431' entity='mgr.x' 2023-12-07T10:10:33.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:33 smithi119 ceph-mon[97908]: from='mgr.14118 172.21.15.119:0/725582431' entity='mgr.x' 2023-12-07T10:10:33.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:33 smithi119 ceph-mon[97908]: mgrmap e7: x(active, since 1.00662s) 2023-12-07T10:10:33.677 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2023-12-07T10:10:33.677 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-12-07T10:10:33.677 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "initialized": true 2023-12-07T10:10:33.677 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2023-12-07T10:10:33.677 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.253+0000 7f017779a700 1 Processor -- start 2023-12-07T10:10:33.677 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.253+0000 7f017779a700 1 -- start start 2023-12-07T10:10:33.677 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.254+0000 7f017779a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f017007ce50 0x7f017007b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:33.677 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.254+0000 7f017779a700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017007b880 con 0x7f017007ce50 2023-12-07T10:10:33.678 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.254+0000 7f0175536700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f017007ce50 0x7f017007b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:33.678 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.254+0000 7f0175536700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f017007ce50 0x7f017007b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43984/0 (socket says 172.21.15.119:43984) 2023-12-07T10:10:33.678 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.254+0000 7f0175536700 1 -- 172.21.15.119:0/1328580425 learned_addr learned my addr 172.21.15.119:0/1328580425 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:33.678 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.254+0000 7f0175536700 1 -- 172.21.15.119:0/1328580425 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f017007b9c0 con 0x7f017007ce50 2023-12-07T10:10:33.678 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.254+0000 7f0175536700 1 --2- 172.21.15.119:0/1328580425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f017007ce50 0x7f017007b2b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f016c0098a0 tx=0x7f016c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fefc1abeb696221 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:33.678 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.255+0000 7f0167fff700 1 -- 172.21.15.119:0/1328580425 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f016c004160 con 0x7f017007ce50 2023-12-07T10:10:33.678 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.255+0000 7f0167fff700 1 -- 172.21.15.119:0/1328580425 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f016c0042c0 con 0x7f017007ce50 2023-12-07T10:10:33.678 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.255+0000 7f0167fff700 1 -- 172.21.15.119:0/1328580425 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f016c011a00 con 0x7f017007ce50 2023-12-07T10:10:33.678 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.255+0000 7f017779a700 1 -- 172.21.15.119:0/1328580425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f017007ce50 msgr2=0x7f017007b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:33.678 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.255+0000 7f017779a700 1 --2- 172.21.15.119:0/1328580425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f017007ce50 0x7f017007b2b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f016c0098a0 tx=0x7f016c009bb0 comp rx=0 tx=0).stop 2023-12-07T10:10:33.678 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.255+0000 7f017779a700 1 -- 172.21.15.119:0/1328580425 shutdown_connections 2023-12-07T10:10:33.678 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.255+0000 7f017779a700 1 --2- 172.21.15.119:0/1328580425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f017007ce50 0x7f017007b2b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:33.678 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.255+0000 7f017779a700 1 -- 172.21.15.119:0/1328580425 >> 172.21.15.119:0/1328580425 conn(0x7f0170102220 msgr2=0x7f0170104660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:33.679 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.255+0000 7f017779a700 1 -- 172.21.15.119:0/1328580425 shutdown_connections 2023-12-07T10:10:33.679 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.256+0000 7f017779a700 1 -- 172.21.15.119:0/1328580425 wait complete. 2023-12-07T10:10:33.679 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.256+0000 7f017779a700 1 Processor -- start 2023-12-07T10:10:33.679 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.257+0000 7f017779a700 1 -- start start 2023-12-07T10:10:33.679 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.257+0000 7f017779a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f017007ce50 0x7f017019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:33.679 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.257+0000 7f017779a700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017019d100 con 0x7f017007ce50 2023-12-07T10:10:33.679 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.257+0000 7f0175536700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f017007ce50 0x7f017019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:33.679 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.257+0000 7f0175536700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f017007ce50 0x7f017019cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43988/0 (socket says 172.21.15.119:43988) 2023-12-07T10:10:33.679 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.257+0000 7f0175536700 1 -- 172.21.15.119:0/2826330230 learned_addr learned my addr 172.21.15.119:0/2826330230 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:33.679 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.257+0000 7f0175536700 1 -- 172.21.15.119:0/2826330230 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c009550 con 0x7f017007ce50 2023-12-07T10:10:33.679 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.258+0000 7f0175536700 1 --2- 172.21.15.119:0/2826330230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f017007ce50 0x7f017019cbc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f016c009130 tx=0x7f016c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:33.679 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.258+0000 7f01667fc700 1 -- 172.21.15.119:0/2826330230 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f016c003b70 con 0x7f017007ce50 2023-12-07T10:10:33.679 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.258+0000 7f017779a700 1 -- 172.21.15.119:0/2826330230 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f017019d300 con 0x7f017007ce50 2023-12-07T10:10:33.679 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.258+0000 7f01667fc700 1 -- 172.21.15.119:0/2826330230 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f016c003cd0 con 0x7f017007ce50 2023-12-07T10:10:33.679 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.258+0000 7f01667fc700 1 -- 172.21.15.119:0/2826330230 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f016c0197d0 con 0x7f017007ce50 2023-12-07T10:10:33.680 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.258+0000 7f017779a700 1 -- 172.21.15.119:0/2826330230 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f017019d7a0 con 0x7f017007ce50 2023-12-07T10:10:33.680 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.259+0000 7f01667fc700 1 -- 172.21.15.119:0/2826330230 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 45739+0+0 (secure 0 0 0) 0x7f016c01a430 con 0x7f017007ce50 2023-12-07T10:10:33.680 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.260+0000 7f01667fc700 1 --2- 172.21.15.119:0/2826330230 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7f015c038a50 0x7f015c03af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:33.680 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.260+0000 7f01667fc700 1 -- 172.21.15.119:0/2826330230 --> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f015c03b620 con 0x7f015c038a50 2023-12-07T10:10:33.680 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.260+0000 7f01667fc700 1 -- 172.21.15.119:0/2826330230 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f016c04cd40 con 0x7f017007ce50 2023-12-07T10:10:33.680 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.260+0000 7f0174d35700 1 -- 172.21.15.119:0/2826330230 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7f015c038a50 msgr2=0x7f015c03af10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/3515858087 2023-12-07T10:10:33.680 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.260+0000 7f0174d35700 1 --2- 172.21.15.119:0/2826330230 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7f015c038a50 0x7f015c03af10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-07T10:10:33.680 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.461+0000 7f0174d35700 1 -- 172.21.15.119:0/2826330230 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7f015c038a50 msgr2=0x7f015c03af10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/3515858087 2023-12-07T10:10:33.680 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.461+0000 7f0174d35700 1 --2- 172.21.15.119:0/2826330230 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7f015c038a50 0x7f015c03af10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-07T10:10:33.680 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.862+0000 7f0174d35700 1 -- 172.21.15.119:0/2826330230 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7f015c038a50 msgr2=0x7f015c03af10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/3515858087 2023-12-07T10:10:33.680 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:26.862+0000 7f0174d35700 1 --2- 172.21.15.119:0/2826330230 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7f015c038a50 0x7f015c03af10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-07T10:10:33.680 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:27.663+0000 7f0174d35700 1 -- 172.21.15.119:0/2826330230 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7f015c038a50 msgr2=0x7f015c03af10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/3515858087 2023-12-07T10:10:33.680 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:27.663+0000 7f0174d35700 1 --2- 172.21.15.119:0/2826330230 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7f015c038a50 0x7f015c03af10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-07T10:10:33.680 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:29.265+0000 7f0174d35700 1 -- 172.21.15.119:0/2826330230 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7f015c038a50 msgr2=0x7f015c03af10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/3515858087 2023-12-07T10:10:33.681 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:29.265+0000 7f0174d35700 1 --2- 172.21.15.119:0/2826330230 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7f015c038a50 0x7f015c03af10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-07T10:10:33.681 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:32.179+0000 7f01667fc700 1 -- 172.21.15.119:0/2826330230 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mgrmap(e 6) v1 ==== 45506+0+0 (secure 0 0 0) 0x7f016c01a730 con 0x7f017007ce50 2023-12-07T10:10:33.681 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:32.179+0000 7f01667fc700 1 -- 172.21.15.119:0/2826330230 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7f015c038a50 msgr2=0x7f015c03af10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-07T10:10:33.681 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:32.179+0000 7f01667fc700 1 --2- 172.21.15.119:0/2826330230 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7f015c038a50 0x7f015c03af10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:33.681 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:33.181+0000 7f01667fc700 1 -- 172.21.15.119:0/2826330230 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 7) v1 ==== 45633+0+0 (secure 0 0 0) 0x7f016c04eb60 con 0x7f017007ce50 2023-12-07T10:10:33.681 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:33.181+0000 7f01667fc700 1 --2- 172.21.15.119:0/2826330230 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f015c03c360 0x7f015c03e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:33.681 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:33.182+0000 7f01667fc700 1 -- 172.21.15.119:0/2826330230 --> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f015c03b620 con 0x7f015c03c360 2023-12-07T10:10:33.681 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:33.182+0000 7f0174d35700 1 --2- 172.21.15.119:0/2826330230 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f015c03c360 0x7f015c03e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:33.681 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:33.183+0000 7f0174d35700 1 --2- 172.21.15.119:0/2826330230 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f015c03c360 0x7f015c03e750 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0160003a10 tx=0x7f0160009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:33.681 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:33.183+0000 7f01667fc700 1 -- 172.21.15.119:0/2826330230 <== mgr.14118 v2:172.21.15.119:6800/1355617847 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f015c03b620 con 0x7f015c03c360 2023-12-07T10:10:33.681 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:33.190+0000 7f017779a700 1 -- 172.21.15.119:0/2826330230 --> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f017010d030 con 0x7f015c03c360 2023-12-07T10:10:33.681 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:33.191+0000 7f01667fc700 1 -- 172.21.15.119:0/2826330230 <== mgr.14118 v2:172.21.15.119:6800/1355617847 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f017010d030 con 0x7f015c03c360 2023-12-07T10:10:33.681 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:33.191+0000 7f017779a700 1 -- 172.21.15.119:0/2826330230 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f015c03c360 msgr2=0x7f015c03e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:33.681 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:33.191+0000 7f017779a700 1 --2- 172.21.15.119:0/2826330230 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f015c03c360 0x7f015c03e750 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0160003a10 tx=0x7f0160009290 comp rx=0 tx=0).stop 2023-12-07T10:10:33.682 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:33.191+0000 7f017779a700 1 -- 172.21.15.119:0/2826330230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f017007ce50 msgr2=0x7f017019cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:33.682 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:33.191+0000 7f017779a700 1 --2- 172.21.15.119:0/2826330230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f017007ce50 0x7f017019cbc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f016c009130 tx=0x7f016c011ad0 comp rx=0 tx=0).stop 2023-12-07T10:10:33.682 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:33.191+0000 7f017779a700 1 -- 172.21.15.119:0/2826330230 shutdown_connections 2023-12-07T10:10:33.682 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:33.191+0000 7f017779a700 1 --2- 172.21.15.119:0/2826330230 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f015c03c360 0x7f015c03e750 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:33.682 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:33.191+0000 7f017779a700 1 --2- 172.21.15.119:0/2826330230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f017007ce50 0x7f017019cbc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:33.682 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:33.191+0000 7f017779a700 1 --2- 172.21.15.119:0/2826330230 >> [v2:172.21.15.119:6800/3515858087,v1:172.21.15.119:6801/3515858087] conn(0x7f015c038a50 0x7f015c03af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:33.682 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:33.191+0000 7f017779a700 1 -- 172.21.15.119:0/2826330230 >> 172.21.15.119:0/2826330230 conn(0x7f0170102220 msgr2=0x7f0170102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:33.682 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:33.191+0000 7f017779a700 1 -- 172.21.15.119:0/2826330230 shutdown_connections 2023-12-07T10:10:33.682 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:33.191+0000 7f017779a700 1 -- 172.21.15.119:0/2826330230 wait complete. 2023-12-07T10:10:33.682 INFO:teuthology.orchestra.run.smithi119.stdout:mgr epoch 5 is available 2023-12-07T10:10:33.682 INFO:teuthology.orchestra.run.smithi119.stdout:Setting orchestrator backend to cephadm... 2023-12-07T10:10:34.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:34 smithi119 ceph-mon[97908]: [07/Dec/2023:10:10:33] ENGINE Bus STARTING 2023-12-07T10:10:34.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:34 smithi119 ceph-mon[97908]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-12-07T10:10:34.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:34 smithi119 ceph-mon[97908]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-12-07T10:10:34.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:34 smithi119 ceph-mon[97908]: [07/Dec/2023:10:10:33] ENGINE Serving on https://172.21.15.119:7150 2023-12-07T10:10:34.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:34 smithi119 ceph-mon[97908]: [07/Dec/2023:10:10:33] ENGINE Serving on http://172.21.15.119:8765 2023-12-07T10:10:34.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:34 smithi119 ceph-mon[97908]: [07/Dec/2023:10:10:33] ENGINE Bus STARTED 2023-12-07T10:10:34.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:34 smithi119 ceph-mon[97908]: from='mgr.14118 172.21.15.119:0/725582431' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:10:34.959 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.263+0000 7fc1237e8700 1 Processor -- start 2023-12-07T10:10:34.959 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.263+0000 7fc1237e8700 1 -- start start 2023-12-07T10:10:34.959 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.264+0000 7fc1237e8700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11c106c80 0x7fc11c1070a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:34.959 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.264+0000 7fc1237e8700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c107670 con 0x7fc11c106c80 2023-12-07T10:10:34.959 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.264+0000 7fc121584700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11c106c80 0x7fc11c1070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:34.960 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.264+0000 7fc121584700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11c106c80 0x7fc11c1070a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34026/0 (socket says 172.21.15.119:34026) 2023-12-07T10:10:34.960 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.264+0000 7fc121584700 1 -- 172.21.15.119:0/1218052479 learned_addr learned my addr 172.21.15.119:0/1218052479 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:34.960 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.264+0000 7fc121584700 1 -- 172.21.15.119:0/1218052479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c107e80 con 0x7fc11c106c80 2023-12-07T10:10:34.960 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.265+0000 7fc121584700 1 --2- 172.21.15.119:0/1218052479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11c106c80 0x7fc11c1070a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc1180098a0 tx=0x7fc118009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff75d41c239dbdea server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:34.960 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.265+0000 7fc113fff700 1 -- 172.21.15.119:0/1218052479 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc118004160 con 0x7fc11c106c80 2023-12-07T10:10:34.960 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.265+0000 7fc113fff700 1 -- 172.21.15.119:0/1218052479 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc1180042c0 con 0x7fc11c106c80 2023-12-07T10:10:34.960 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.265+0000 7fc113fff700 1 -- 172.21.15.119:0/1218052479 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc118011a00 con 0x7fc11c106c80 2023-12-07T10:10:34.960 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.265+0000 7fc1237e8700 1 -- 172.21.15.119:0/1218052479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11c106c80 msgr2=0x7fc11c1070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:34.960 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.265+0000 7fc1237e8700 1 --2- 172.21.15.119:0/1218052479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11c106c80 0x7fc11c1070a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc1180098a0 tx=0x7fc118009bb0 comp rx=0 tx=0).stop 2023-12-07T10:10:34.960 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.266+0000 7fc1237e8700 1 -- 172.21.15.119:0/1218052479 shutdown_connections 2023-12-07T10:10:34.960 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.266+0000 7fc1237e8700 1 --2- 172.21.15.119:0/1218052479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11c106c80 0x7fc11c1070a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:34.960 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.266+0000 7fc1237e8700 1 -- 172.21.15.119:0/1218052479 >> 172.21.15.119:0/1218052479 conn(0x7fc11c102220 msgr2=0x7fc11c104660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:34.960 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.266+0000 7fc1237e8700 1 -- 172.21.15.119:0/1218052479 shutdown_connections 2023-12-07T10:10:34.960 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.266+0000 7fc1237e8700 1 -- 172.21.15.119:0/1218052479 wait complete. 2023-12-07T10:10:34.960 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.267+0000 7fc1237e8700 1 Processor -- start 2023-12-07T10:10:34.960 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.267+0000 7fc1237e8700 1 -- start start 2023-12-07T10:10:34.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.267+0000 7fc1237e8700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11c106c80 0x7fc11c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:34.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.267+0000 7fc1237e8700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c19af30 con 0x7fc11c106c80 2023-12-07T10:10:34.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.267+0000 7fc121584700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11c106c80 0x7fc11c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:34.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.267+0000 7fc121584700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11c106c80 0x7fc11c19a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34040/0 (socket says 172.21.15.119:34040) 2023-12-07T10:10:34.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.267+0000 7fc121584700 1 -- 172.21.15.119:0/717352526 learned_addr learned my addr 172.21.15.119:0/717352526 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:34.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.268+0000 7fc121584700 1 -- 172.21.15.119:0/717352526 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc118009550 con 0x7fc11c106c80 2023-12-07T10:10:34.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.268+0000 7fc121584700 1 --2- 172.21.15.119:0/717352526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11c106c80 0x7fc11c19a9f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc118009130 tx=0x7fc118011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:34.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.268+0000 7fc1127fc700 1 -- 172.21.15.119:0/717352526 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc118003ab0 con 0x7fc11c106c80 2023-12-07T10:10:34.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.268+0000 7fc1127fc700 1 -- 172.21.15.119:0/717352526 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc118003c10 con 0x7fc11c106c80 2023-12-07T10:10:34.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.268+0000 7fc1237e8700 1 -- 172.21.15.119:0/717352526 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11c19b130 con 0x7fc11c106c80 2023-12-07T10:10:34.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.268+0000 7fc1127fc700 1 -- 172.21.15.119:0/717352526 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1180247d0 con 0x7fc11c106c80 2023-12-07T10:10:34.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.268+0000 7fc1237e8700 1 -- 172.21.15.119:0/717352526 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c19b5d0 con 0x7fc11c106c80 2023-12-07T10:10:34.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.269+0000 7fc1237e8700 1 -- 172.21.15.119:0/717352526 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11c10b0b0 con 0x7fc11c106c80 2023-12-07T10:10:34.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.270+0000 7fc1127fc700 1 -- 172.21.15.119:0/717352526 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 45739+0+0 (secure 0 0 0) 0x7fc11801abb0 con 0x7fc11c106c80 2023-12-07T10:10:34.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.270+0000 7fc1127fc700 1 --2- 172.21.15.119:0/717352526 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7fc108038aa0 0x7fc10803af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:34.961 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.270+0000 7fc1127fc700 1 -- 172.21.15.119:0/717352526 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc11804cc70 con 0x7fc11c106c80 2023-12-07T10:10:34.962 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.271+0000 7fc120d83700 1 --2- 172.21.15.119:0/717352526 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7fc108038aa0 0x7fc10803af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:34.962 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.271+0000 7fc120d83700 1 --2- 172.21.15.119:0/717352526 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7fc108038aa0 0x7fc10803af60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc10c006fd0 tx=0x7fc10c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:34.962 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.276+0000 7fc1127fc700 1 -- 172.21.15.119:0/717352526 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc11802d3f0 con 0x7fc11c106c80 2023-12-07T10:10:34.962 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.491+0000 7fc1237e8700 1 -- 172.21.15.119:0/717352526 --> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fc11c103530 con 0x7fc108038aa0 2023-12-07T10:10:34.962 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.498+0000 7fc1127fc700 1 -- 172.21.15.119:0/717352526 <== mgr.14118 v2:172.21.15.119:6800/1355617847 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc11c103530 con 0x7fc108038aa0 2023-12-07T10:10:34.962 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.500+0000 7fc1237e8700 1 -- 172.21.15.119:0/717352526 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7fc108038aa0 msgr2=0x7fc10803af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:34.962 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.500+0000 7fc1237e8700 1 --2- 172.21.15.119:0/717352526 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7fc108038aa0 0x7fc10803af60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc10c006fd0 tx=0x7fc10c006e40 comp rx=0 tx=0).stop 2023-12-07T10:10:34.962 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.500+0000 7fc1237e8700 1 -- 172.21.15.119:0/717352526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11c106c80 msgr2=0x7fc11c19a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:34.962 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.500+0000 7fc1237e8700 1 --2- 172.21.15.119:0/717352526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11c106c80 0x7fc11c19a9f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc118009130 tx=0x7fc118011ad0 comp rx=0 tx=0).stop 2023-12-07T10:10:34.962 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.500+0000 7fc1237e8700 1 -- 172.21.15.119:0/717352526 shutdown_connections 2023-12-07T10:10:34.962 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.500+0000 7fc1237e8700 1 --2- 172.21.15.119:0/717352526 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7fc108038aa0 0x7fc10803af60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:34.962 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.500+0000 7fc1237e8700 1 --2- 172.21.15.119:0/717352526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11c106c80 0x7fc11c19a9f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:34.962 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.500+0000 7fc1237e8700 1 -- 172.21.15.119:0/717352526 >> 172.21.15.119:0/717352526 conn(0x7fc11c102220 msgr2=0x7fc11c102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:34.962 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.501+0000 7fc1237e8700 1 -- 172.21.15.119:0/717352526 shutdown_connections 2023-12-07T10:10:34.963 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:34.501+0000 7fc1237e8700 1 -- 172.21.15.119:0/717352526 wait complete. 2023-12-07T10:10:35.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:35 smithi119 ceph-mon[97908]: mgrmap e8: x(active, since 2s) 2023-12-07T10:10:35.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:35 smithi119 ceph-mon[97908]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:10:35.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:35 smithi119 ceph-mon[97908]: from='mgr.14118 172.21.15.119:0/725582431' entity='mgr.x' 2023-12-07T10:10:35.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:35 smithi119 ceph-mon[97908]: from='mgr.14118 172.21.15.119:0/725582431' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:10:36.239 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout value unchanged 2023-12-07T10:10:36.239 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.488+0000 7fadf78bf700 1 Processor -- start 2023-12-07T10:10:36.239 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.488+0000 7fadf78bf700 1 -- start start 2023-12-07T10:10:36.239 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.488+0000 7fadf78bf700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadf007ce70 0x7fadf007b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:36.239 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.488+0000 7fadf78bf700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf007b8a0 con 0x7fadf007ce70 2023-12-07T10:10:36.239 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.488+0000 7fadf565b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadf007ce70 0x7fadf007b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:36.239 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.488+0000 7fadf565b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadf007ce70 0x7fadf007b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34048/0 (socket says 172.21.15.119:34048) 2023-12-07T10:10:36.240 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.488+0000 7fadf565b700 1 -- 172.21.15.119:0/2010226264 learned_addr learned my addr 172.21.15.119:0/2010226264 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:36.240 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.489+0000 7fadf565b700 1 -- 172.21.15.119:0/2010226264 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf007b9e0 con 0x7fadf007ce70 2023-12-07T10:10:36.240 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.489+0000 7fadf565b700 1 --2- 172.21.15.119:0/2010226264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadf007ce70 0x7fadf007b2d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fadec0098a0 tx=0x7fadec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332bea6b43a77d5d server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:36.240 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.489+0000 7fade7fff700 1 -- 172.21.15.119:0/2010226264 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fadec004160 con 0x7fadf007ce70 2023-12-07T10:10:36.240 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.489+0000 7fade7fff700 1 -- 172.21.15.119:0/2010226264 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fadec0042c0 con 0x7fadf007ce70 2023-12-07T10:10:36.240 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.489+0000 7fade7fff700 1 -- 172.21.15.119:0/2010226264 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fadec011a00 con 0x7fadf007ce70 2023-12-07T10:10:36.240 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.490+0000 7fadf78bf700 1 -- 172.21.15.119:0/2010226264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadf007ce70 msgr2=0x7fadf007b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:36.240 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.490+0000 7fadf78bf700 1 --2- 172.21.15.119:0/2010226264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadf007ce70 0x7fadf007b2d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fadec0098a0 tx=0x7fadec009bb0 comp rx=0 tx=0).stop 2023-12-07T10:10:36.240 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.490+0000 7fadf78bf700 1 -- 172.21.15.119:0/2010226264 shutdown_connections 2023-12-07T10:10:36.240 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.490+0000 7fadf78bf700 1 --2- 172.21.15.119:0/2010226264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadf007ce70 0x7fadf007b2d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:36.240 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.490+0000 7fadf78bf700 1 -- 172.21.15.119:0/2010226264 >> 172.21.15.119:0/2010226264 conn(0x7fadf0102220 msgr2=0x7fadf0104660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:36.240 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.490+0000 7fadf78bf700 1 -- 172.21.15.119:0/2010226264 shutdown_connections 2023-12-07T10:10:36.240 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.490+0000 7fadf78bf700 1 -- 172.21.15.119:0/2010226264 wait complete. 2023-12-07T10:10:36.240 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.491+0000 7fadf78bf700 1 Processor -- start 2023-12-07T10:10:36.240 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.491+0000 7fadf78bf700 1 -- start start 2023-12-07T10:10:36.241 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.491+0000 7fadf78bf700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadf007ce70 0x7fadf019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:36.241 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.491+0000 7fadf78bf700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf019d100 con 0x7fadf007ce70 2023-12-07T10:10:36.241 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.492+0000 7fadf565b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadf007ce70 0x7fadf019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:36.241 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.492+0000 7fadf565b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadf007ce70 0x7fadf019cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34054/0 (socket says 172.21.15.119:34054) 2023-12-07T10:10:36.241 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.492+0000 7fadf565b700 1 -- 172.21.15.119:0/775254471 learned_addr learned my addr 172.21.15.119:0/775254471 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:36.241 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.492+0000 7fadf565b700 1 -- 172.21.15.119:0/775254471 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadec009550 con 0x7fadf007ce70 2023-12-07T10:10:36.241 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.492+0000 7fadf565b700 1 --2- 172.21.15.119:0/775254471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadf007ce70 0x7fadf019cbc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fadec009130 tx=0x7fadec011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:36.241 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.492+0000 7fade67fc700 1 -- 172.21.15.119:0/775254471 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fadec003b70 con 0x7fadf007ce70 2023-12-07T10:10:36.241 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.492+0000 7fade67fc700 1 -- 172.21.15.119:0/775254471 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fadec003cd0 con 0x7fadf007ce70 2023-12-07T10:10:36.241 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.492+0000 7fade67fc700 1 -- 172.21.15.119:0/775254471 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fadec0197d0 con 0x7fadf007ce70 2023-12-07T10:10:36.241 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.493+0000 7fadf78bf700 1 -- 172.21.15.119:0/775254471 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf019d300 con 0x7fadf007ce70 2023-12-07T10:10:36.241 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.493+0000 7fadf78bf700 1 -- 172.21.15.119:0/775254471 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf019d7a0 con 0x7fadf007ce70 2023-12-07T10:10:36.241 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.494+0000 7fade67fc700 1 -- 172.21.15.119:0/775254471 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 45739+0+0 (secure 0 0 0) 0x7fadec01a430 con 0x7fadf007ce70 2023-12-07T10:10:36.241 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.494+0000 7fadf78bf700 1 -- 172.21.15.119:0/775254471 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf010d070 con 0x7fadf007ce70 2023-12-07T10:10:36.241 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.494+0000 7fade67fc700 1 --2- 172.21.15.119:0/775254471 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7faddc038aa0 0x7faddc03af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:36.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.494+0000 7fade67fc700 1 -- 172.21.15.119:0/775254471 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fadec04cdb0 con 0x7fadf007ce70 2023-12-07T10:10:36.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.494+0000 7fadf4e5a700 1 --2- 172.21.15.119:0/775254471 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7faddc038aa0 0x7faddc03af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:36.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.495+0000 7fadf4e5a700 1 --2- 172.21.15.119:0/775254471 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7faddc038aa0 0x7faddc03af60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fade0006fd0 tx=0x7fade0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:36.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.500+0000 7fade67fc700 1 -- 172.21.15.119:0/775254471 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fadec02d960 con 0x7fadf007ce70 2023-12-07T10:10:36.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.703+0000 7fadf78bf700 1 -- 172.21.15.119:0/775254471 --> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fadf01034c0 con 0x7faddc038aa0 2023-12-07T10:10:36.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.703+0000 7fade67fc700 1 -- 172.21.15.119:0/775254471 <== mgr.14118 v2:172.21.15.119:6800/1355617847 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fadf01034c0 con 0x7faddc038aa0 2023-12-07T10:10:36.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.706+0000 7fadf78bf700 1 -- 172.21.15.119:0/775254471 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7faddc038aa0 msgr2=0x7faddc03af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:36.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.706+0000 7fadf78bf700 1 --2- 172.21.15.119:0/775254471 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7faddc038aa0 0x7faddc03af60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fade0006fd0 tx=0x7fade0006e40 comp rx=0 tx=0).stop 2023-12-07T10:10:36.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.706+0000 7fadf78bf700 1 -- 172.21.15.119:0/775254471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadf007ce70 msgr2=0x7fadf019cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:36.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.706+0000 7fadf78bf700 1 --2- 172.21.15.119:0/775254471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadf007ce70 0x7fadf019cbc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fadec009130 tx=0x7fadec011ad0 comp rx=0 tx=0).stop 2023-12-07T10:10:36.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.706+0000 7fadf78bf700 1 -- 172.21.15.119:0/775254471 shutdown_connections 2023-12-07T10:10:36.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.706+0000 7fadf78bf700 1 --2- 172.21.15.119:0/775254471 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7faddc038aa0 0x7faddc03af60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:36.242 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.706+0000 7fadf78bf700 1 --2- 172.21.15.119:0/775254471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadf007ce70 0x7fadf019cbc0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:36.243 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.706+0000 7fadf78bf700 1 -- 172.21.15.119:0/775254471 >> 172.21.15.119:0/775254471 conn(0x7fadf0102220 msgr2=0x7fadf0102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:36.243 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.706+0000 7fadf78bf700 1 -- 172.21.15.119:0/775254471 shutdown_connections 2023-12-07T10:10:36.243 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:35.706+0000 7fadf78bf700 1 -- 172.21.15.119:0/775254471 wait complete. 2023-12-07T10:10:36.243 INFO:teuthology.orchestra.run.smithi119.stdout:Generating ssh key... 2023-12-07T10:10:36.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:36 smithi119 ceph-mon[97908]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:10:37.643 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:37 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: Generating public/private rsa key pair. 2023-12-07T10:10:37.643 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:37 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: Your identification has been saved in /tmp/tmpimymelhd/key. 2023-12-07T10:10:37.644 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:37 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: Your public key has been saved in /tmp/tmpimymelhd/key.pub. 2023-12-07T10:10:37.644 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:37 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: The key fingerprint is: 2023-12-07T10:10:37.644 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:37 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: SHA256:iZI7b3He2J6EwKIfUfVqsmu+x9r+d/N5v+n8wOfcW+g ceph-a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:10:37.644 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:37 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: The key's randomart image is: 2023-12-07T10:10:37.644 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:37 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: +---[RSA 3072]----+ 2023-12-07T10:10:37.644 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:37 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: | . | 2023-12-07T10:10:37.644 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:37 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: | . . | 2023-12-07T10:10:37.644 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:37 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: | . . | 2023-12-07T10:10:37.644 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:37 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: | + . o | 2023-12-07T10:10:37.644 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:37 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: | = = S | 2023-12-07T10:10:37.644 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:37 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: | . =.=.. . . | 2023-12-07T10:10:37.644 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:37 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: | . + .=.+. + o| 2023-12-07T10:10:37.644 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:37 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: | . +oo=.o...+=*| 2023-12-07T10:10:37.644 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:37 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: | .+*=oo+. .EX&| 2023-12-07T10:10:37.645 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:37 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: +----[SHA256]-----+ 2023-12-07T10:10:37.716 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.777+0000 7fe27ac28700 1 Processor -- start 2023-12-07T10:10:37.716 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.777+0000 7fe27ac28700 1 -- start start 2023-12-07T10:10:37.716 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.778+0000 7fe27ac28700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe27407ce50 0x7fe27407b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:37.721 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.778+0000 7fe27ac28700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27407b880 con 0x7fe27407ce50 2023-12-07T10:10:37.721 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.778+0000 7fe2789c4700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe27407ce50 0x7fe27407b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:37.721 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.778+0000 7fe2789c4700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe27407ce50 0x7fe27407b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34068/0 (socket says 172.21.15.119:34068) 2023-12-07T10:10:37.721 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.778+0000 7fe2789c4700 1 -- 172.21.15.119:0/2560177343 learned_addr learned my addr 172.21.15.119:0/2560177343 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:37.721 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.778+0000 7fe2789c4700 1 -- 172.21.15.119:0/2560177343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27407b9c0 con 0x7fe27407ce50 2023-12-07T10:10:37.721 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.779+0000 7fe2789c4700 1 --2- 172.21.15.119:0/2560177343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe27407ce50 0x7fe27407b2b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe2680098a0 tx=0x7fe268009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308579d4d3edeba4 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:37.721 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.779+0000 7fe2737fe700 1 -- 172.21.15.119:0/2560177343 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe268004160 con 0x7fe27407ce50 2023-12-07T10:10:37.722 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.779+0000 7fe2737fe700 1 -- 172.21.15.119:0/2560177343 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe2680042c0 con 0x7fe27407ce50 2023-12-07T10:10:37.722 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.779+0000 7fe2737fe700 1 -- 172.21.15.119:0/2560177343 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe268011a00 con 0x7fe27407ce50 2023-12-07T10:10:37.722 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.779+0000 7fe27ac28700 1 -- 172.21.15.119:0/2560177343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe27407ce50 msgr2=0x7fe27407b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:37.722 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.779+0000 7fe27ac28700 1 --2- 172.21.15.119:0/2560177343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe27407ce50 0x7fe27407b2b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe2680098a0 tx=0x7fe268009bb0 comp rx=0 tx=0).stop 2023-12-07T10:10:37.722 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.780+0000 7fe27ac28700 1 -- 172.21.15.119:0/2560177343 shutdown_connections 2023-12-07T10:10:37.722 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.780+0000 7fe27ac28700 1 --2- 172.21.15.119:0/2560177343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe27407ce50 0x7fe27407b2b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:37.722 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.780+0000 7fe27ac28700 1 -- 172.21.15.119:0/2560177343 >> 172.21.15.119:0/2560177343 conn(0x7fe274102220 msgr2=0x7fe274104640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:37.722 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.780+0000 7fe27ac28700 1 -- 172.21.15.119:0/2560177343 shutdown_connections 2023-12-07T10:10:37.722 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.780+0000 7fe27ac28700 1 -- 172.21.15.119:0/2560177343 wait complete. 2023-12-07T10:10:37.722 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.781+0000 7fe27ac28700 1 Processor -- start 2023-12-07T10:10:37.722 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.781+0000 7fe27ac28700 1 -- start start 2023-12-07T10:10:37.722 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.781+0000 7fe27ac28700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe27407ce50 0x7fe27419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:37.722 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.781+0000 7fe27ac28700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27419d0c0 con 0x7fe27407ce50 2023-12-07T10:10:37.722 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.781+0000 7fe2789c4700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe27407ce50 0x7fe27419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:37.723 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.781+0000 7fe2789c4700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe27407ce50 0x7fe27419cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34074/0 (socket says 172.21.15.119:34074) 2023-12-07T10:10:37.723 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.781+0000 7fe2789c4700 1 -- 172.21.15.119:0/3913317696 learned_addr learned my addr 172.21.15.119:0/3913317696 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:37.723 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.782+0000 7fe2789c4700 1 -- 172.21.15.119:0/3913317696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe268009550 con 0x7fe27407ce50 2023-12-07T10:10:37.723 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.782+0000 7fe2789c4700 1 --2- 172.21.15.119:0/3913317696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe27407ce50 0x7fe27419cb80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe268009130 tx=0x7fe268011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:37.723 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.782+0000 7fe271ffb700 1 -- 172.21.15.119:0/3913317696 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe268003b70 con 0x7fe27407ce50 2023-12-07T10:10:37.723 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.782+0000 7fe271ffb700 1 -- 172.21.15.119:0/3913317696 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe268003cd0 con 0x7fe27407ce50 2023-12-07T10:10:37.723 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.782+0000 7fe271ffb700 1 -- 172.21.15.119:0/3913317696 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2680197d0 con 0x7fe27407ce50 2023-12-07T10:10:37.723 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.782+0000 7fe27ac28700 1 -- 172.21.15.119:0/3913317696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe27419d2c0 con 0x7fe27407ce50 2023-12-07T10:10:37.723 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.782+0000 7fe27ac28700 1 -- 172.21.15.119:0/3913317696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe27419d760 con 0x7fe27407ce50 2023-12-07T10:10:37.723 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.783+0000 7fe27ac28700 1 -- 172.21.15.119:0/3913317696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe27410d010 con 0x7fe27407ce50 2023-12-07T10:10:37.723 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.784+0000 7fe271ffb700 1 -- 172.21.15.119:0/3913317696 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 45739+0+0 (secure 0 0 0) 0x7fe26801abb0 con 0x7fe27407ce50 2023-12-07T10:10:37.723 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.784+0000 7fe271ffb700 1 --2- 172.21.15.119:0/3913317696 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7fe264038aa0 0x7fe26403af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:37.723 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.784+0000 7fe271ffb700 1 -- 172.21.15.119:0/3913317696 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe26804cd30 con 0x7fe27407ce50 2023-12-07T10:10:37.723 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.784+0000 7fe273fff700 1 --2- 172.21.15.119:0/3913317696 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7fe264038aa0 0x7fe26403af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:37.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.785+0000 7fe273fff700 1 --2- 172.21.15.119:0/3913317696 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7fe264038aa0 0x7fe26403af60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe260006fd0 tx=0x7fe260006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:37.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.789+0000 7fe271ffb700 1 -- 172.21.15.119:0/3913317696 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe26801a430 con 0x7fe27407ce50 2023-12-07T10:10:37.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:36.990+0000 7fe27ac28700 1 -- 172.21.15.119:0/3913317696 --> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2741034e0 con 0x7fe264038aa0 2023-12-07T10:10:37.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:37.244+0000 7fe271ffb700 1 -- 172.21.15.119:0/3913317696 <== mgr.14118 v2:172.21.15.119:6800/1355617847 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe2741034e0 con 0x7fe264038aa0 2023-12-07T10:10:37.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:37.247+0000 7fe27ac28700 1 -- 172.21.15.119:0/3913317696 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7fe264038aa0 msgr2=0x7fe26403af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:37.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:37.247+0000 7fe27ac28700 1 --2- 172.21.15.119:0/3913317696 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7fe264038aa0 0x7fe26403af60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe260006fd0 tx=0x7fe260006e40 comp rx=0 tx=0).stop 2023-12-07T10:10:37.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:37.247+0000 7fe27ac28700 1 -- 172.21.15.119:0/3913317696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe27407ce50 msgr2=0x7fe27419cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:37.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:37.247+0000 7fe27ac28700 1 --2- 172.21.15.119:0/3913317696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe27407ce50 0x7fe27419cb80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe268009130 tx=0x7fe268011ad0 comp rx=0 tx=0).stop 2023-12-07T10:10:37.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:37.247+0000 7fe27ac28700 1 -- 172.21.15.119:0/3913317696 shutdown_connections 2023-12-07T10:10:37.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:37.247+0000 7fe27ac28700 1 --2- 172.21.15.119:0/3913317696 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7fe264038aa0 0x7fe26403af60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:37.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:37.247+0000 7fe27ac28700 1 --2- 172.21.15.119:0/3913317696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe27407ce50 0x7fe27419cb80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:37.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:37.247+0000 7fe27ac28700 1 -- 172.21.15.119:0/3913317696 >> 172.21.15.119:0/3913317696 conn(0x7fe274102220 msgr2=0x7fe274102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:37.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:37.247+0000 7fe27ac28700 1 -- 172.21.15.119:0/3913317696 shutdown_connections 2023-12-07T10:10:37.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:37.247+0000 7fe27ac28700 1 -- 172.21.15.119:0/3913317696 wait complete. 2023-12-07T10:10:38.631 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:38 smithi119 ceph-mon[97908]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:10:38.631 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:38 smithi119 ceph-mon[97908]: Generating ssh key... 2023-12-07T10:10:38.631 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:38 smithi119 ceph-mon[97908]: from='mgr.14118 172.21.15.119:0/725582431' entity='mgr.x' 2023-12-07T10:10:38.632 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:38 smithi119 ceph-mon[97908]: from='mgr.14118 172.21.15.119:0/725582431' entity='mgr.x' 2023-12-07T10:10:38.901 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+j/G7Kagnmwi5u9jUXZynEt7Ze+IxYbWrDeLNakvmCiKMqyj4BuMYFNI9SBpMg6XAYeOhcGKGQuONJyqQEXcemcPoZYyOcpFURxoOCxAcnHsUCLTFGZ3ZpceBVERR3nqgf1UA54+9LYttoDERldFZVD1kYCe+AQbkEBMNhMH6gZ/n0BSWx05V4A7HGxV++vCndUKLIM2ZfJddvpc82cy6ASrGAy7jSE8HtdQQ3Yq/h6yXFyZSIIwT0XSueDMThFtRSa9D3EQ1fbZsYRN4+LAKsJgcveNGksb0qmRLItpK2hpQHsv1uXtDOgvvI5pZhWic7Lv6O+toncRhOyEc1oEbQBRYYJI6ZXWCh3zH7UvjKycSNu7nVM+LV68Td49/Fk04pso7V9HNt9ipjpPcO1t859H0lD6HcUKVOwOmSIa6E3tSqHR+B22CjYgjWTmS2i8Y870DEnFuSXPY1QcdFSYkTSrkf2RicfXQ8WH36ID1IPRG92EPWr7R7Ld5F1SAQoM= ceph-a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:10:38.901 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.226+0000 7f952893c700 1 Processor -- start 2023-12-07T10:10:38.901 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.227+0000 7f952893c700 1 -- start start 2023-12-07T10:10:38.901 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.227+0000 7f952893c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f952407ce50 0x7f952407b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:38.901 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.227+0000 7f952893c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952407b880 con 0x7f952407ce50 2023-12-07T10:10:38.901 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.227+0000 7f952259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f952407ce50 0x7f952407b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:38.901 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.227+0000 7f952259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f952407ce50 0x7f952407b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34082/0 (socket says 172.21.15.119:34082) 2023-12-07T10:10:38.902 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.227+0000 7f952259c700 1 -- 172.21.15.119:0/1437116324 learned_addr learned my addr 172.21.15.119:0/1437116324 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:38.902 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.228+0000 7f952259c700 1 -- 172.21.15.119:0/1437116324 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952407b9c0 con 0x7f952407ce50 2023-12-07T10:10:38.902 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.228+0000 7f952259c700 1 --2- 172.21.15.119:0/1437116324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f952407ce50 0x7f952407b2b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f95140098a0 tx=0x7f9514009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1704144fc170c2 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:38.902 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.228+0000 7f952159a700 1 -- 172.21.15.119:0/1437116324 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9514004160 con 0x7f952407ce50 2023-12-07T10:10:38.902 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.228+0000 7f952159a700 1 -- 172.21.15.119:0/1437116324 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f95140042c0 con 0x7f952407ce50 2023-12-07T10:10:38.902 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.228+0000 7f952159a700 1 -- 172.21.15.119:0/1437116324 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9514011a00 con 0x7f952407ce50 2023-12-07T10:10:38.902 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.229+0000 7f952893c700 1 -- 172.21.15.119:0/1437116324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f952407ce50 msgr2=0x7f952407b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:38.902 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.229+0000 7f952893c700 1 --2- 172.21.15.119:0/1437116324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f952407ce50 0x7f952407b2b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f95140098a0 tx=0x7f9514009bb0 comp rx=0 tx=0).stop 2023-12-07T10:10:38.902 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.229+0000 7f952893c700 1 -- 172.21.15.119:0/1437116324 shutdown_connections 2023-12-07T10:10:38.902 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.229+0000 7f952893c700 1 --2- 172.21.15.119:0/1437116324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f952407ce50 0x7f952407b2b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:38.902 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.229+0000 7f952893c700 1 -- 172.21.15.119:0/1437116324 >> 172.21.15.119:0/1437116324 conn(0x7f9524102220 msgr2=0x7f9524104660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:38.902 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.229+0000 7f952893c700 1 -- 172.21.15.119:0/1437116324 shutdown_connections 2023-12-07T10:10:38.902 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.229+0000 7f952893c700 1 -- 172.21.15.119:0/1437116324 wait complete. 2023-12-07T10:10:38.903 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.230+0000 7f952893c700 1 Processor -- start 2023-12-07T10:10:38.903 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.230+0000 7f952893c700 1 -- start start 2023-12-07T10:10:38.903 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.230+0000 7f952893c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f952407ce50 0x7f952419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:38.903 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.230+0000 7f952893c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952419d100 con 0x7f952407ce50 2023-12-07T10:10:38.903 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.230+0000 7f952259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f952407ce50 0x7f952419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:38.903 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.231+0000 7f952259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f952407ce50 0x7f952419cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34090/0 (socket says 172.21.15.119:34090) 2023-12-07T10:10:38.903 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.231+0000 7f952259c700 1 -- 172.21.15.119:0/243901994 learned_addr learned my addr 172.21.15.119:0/243901994 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:38.903 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.231+0000 7f952259c700 1 -- 172.21.15.119:0/243901994 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9514009550 con 0x7f952407ce50 2023-12-07T10:10:38.903 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.231+0000 7f952259c700 1 --2- 172.21.15.119:0/243901994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f952407ce50 0x7f952419cbc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9514009130 tx=0x7f9514011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:38.903 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.231+0000 7f95137fe700 1 -- 172.21.15.119:0/243901994 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9514003b70 con 0x7f952407ce50 2023-12-07T10:10:38.903 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.231+0000 7f95137fe700 1 -- 172.21.15.119:0/243901994 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9514003cd0 con 0x7f952407ce50 2023-12-07T10:10:38.903 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.231+0000 7f95137fe700 1 -- 172.21.15.119:0/243901994 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95140197d0 con 0x7f952407ce50 2023-12-07T10:10:38.903 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.231+0000 7f952893c700 1 -- 172.21.15.119:0/243901994 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952419d300 con 0x7f952407ce50 2023-12-07T10:10:38.903 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.231+0000 7f952893c700 1 -- 172.21.15.119:0/243901994 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952419d7a0 con 0x7f952407ce50 2023-12-07T10:10:38.904 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.233+0000 7f952893c700 1 -- 172.21.15.119:0/243901994 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952410d030 con 0x7f952407ce50 2023-12-07T10:10:38.904 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.233+0000 7f95137fe700 1 -- 172.21.15.119:0/243901994 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 45739+0+0 (secure 0 0 0) 0x7f951401abb0 con 0x7f952407ce50 2023-12-07T10:10:38.904 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.233+0000 7f95137fe700 1 --2- 172.21.15.119:0/243901994 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f950c038a50 0x7f950c03af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:38.904 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.233+0000 7f95137fe700 1 -- 172.21.15.119:0/243901994 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f951404cde0 con 0x7f952407ce50 2023-12-07T10:10:38.904 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.233+0000 7f9521d9b700 1 --2- 172.21.15.119:0/243901994 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f950c038a50 0x7f950c03af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:38.904 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.234+0000 7f9521d9b700 1 --2- 172.21.15.119:0/243901994 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f950c038a50 0x7f950c03af10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9518006fd0 tx=0x7f9518006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:38.904 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.238+0000 7f95137fe700 1 -- 172.21.15.119:0/243901994 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f951401a430 con 0x7f952407ce50 2023-12-07T10:10:38.904 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.437+0000 7f952893c700 1 -- 172.21.15.119:0/243901994 --> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f95241034c0 con 0x7f950c038a50 2023-12-07T10:10:38.904 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.438+0000 7f95137fe700 1 -- 172.21.15.119:0/243901994 <== mgr.14118 v2:172.21.15.119:6800/1355617847 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f95241034c0 con 0x7f950c038a50 2023-12-07T10:10:38.904 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.440+0000 7f952893c700 1 -- 172.21.15.119:0/243901994 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f950c038a50 msgr2=0x7f950c03af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:38.904 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.440+0000 7f952893c700 1 --2- 172.21.15.119:0/243901994 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f950c038a50 0x7f950c03af10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9518006fd0 tx=0x7f9518006e40 comp rx=0 tx=0).stop 2023-12-07T10:10:38.904 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.440+0000 7f952893c700 1 -- 172.21.15.119:0/243901994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f952407ce50 msgr2=0x7f952419cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:38.904 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.440+0000 7f952893c700 1 --2- 172.21.15.119:0/243901994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f952407ce50 0x7f952419cbc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9514009130 tx=0x7f9514011ad0 comp rx=0 tx=0).stop 2023-12-07T10:10:38.904 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.440+0000 7f952893c700 1 -- 172.21.15.119:0/243901994 shutdown_connections 2023-12-07T10:10:38.905 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.440+0000 7f952893c700 1 --2- 172.21.15.119:0/243901994 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f950c038a50 0x7f950c03af10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:38.905 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.440+0000 7f952893c700 1 --2- 172.21.15.119:0/243901994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f952407ce50 0x7f952419cbc0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:38.905 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.440+0000 7f952893c700 1 -- 172.21.15.119:0/243901994 >> 172.21.15.119:0/243901994 conn(0x7f9524102220 msgr2=0x7f9524102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:38.905 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.440+0000 7f952893c700 1 -- 172.21.15.119:0/243901994 shutdown_connections 2023-12-07T10:10:38.905 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:38.440+0000 7f952893c700 1 -- 172.21.15.119:0/243901994 wait complete. 2023-12-07T10:10:38.905 INFO:teuthology.orchestra.run.smithi119.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-12-07T10:10:38.905 INFO:teuthology.orchestra.run.smithi119.stdout:Adding key to root@localhost authorized_keys... 2023-12-07T10:10:38.905 INFO:teuthology.orchestra.run.smithi119.stdout:Adding host smithi119... 2023-12-07T10:10:39.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:39 smithi119 ceph-mon[97908]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:10:41.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:41 smithi119 ceph-mon[97908]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi119", "addr": "172.21.15.119", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:10:41.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:41 smithi119 ceph-mon[97908]: Deploying cephadm binary to smithi119 2023-12-07T10:10:44.898 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout Added host 'smithi119' with addr '172.21.15.119' 2023-12-07T10:10:44.898 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.461+0000 7fd6f3dd3700 1 Processor -- start 2023-12-07T10:10:44.898 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.461+0000 7fd6f3dd3700 1 -- start start 2023-12-07T10:10:44.899 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.461+0000 7fd6f3dd3700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6ec108e70 0x7fd6ec109290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:44.899 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.461+0000 7fd6f3dd3700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec1097d0 con 0x7fd6ec108e70 2023-12-07T10:10:44.899 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.462+0000 7fd6f1b6f700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6ec108e70 0x7fd6ec109290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:44.899 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.462+0000 7fd6f1b6f700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6ec108e70 0x7fd6ec109290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34092/0 (socket says 172.21.15.119:34092) 2023-12-07T10:10:44.899 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.462+0000 7fd6f1b6f700 1 -- 172.21.15.119:0/3997804793 learned_addr learned my addr 172.21.15.119:0/3997804793 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:44.899 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.462+0000 7fd6f1b6f700 1 -- 172.21.15.119:0/3997804793 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec109910 con 0x7fd6ec108e70 2023-12-07T10:10:44.899 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.462+0000 7fd6f1b6f700 1 --2- 172.21.15.119:0/3997804793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6ec108e70 0x7fd6ec109290 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd6e80098a0 tx=0x7fd6e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137745fa29497de2 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:44.899 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.462+0000 7fd6f0b6d700 1 -- 172.21.15.119:0/3997804793 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6e8004160 con 0x7fd6ec108e70 2023-12-07T10:10:44.899 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.463+0000 7fd6f0b6d700 1 -- 172.21.15.119:0/3997804793 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd6e80042c0 con 0x7fd6ec108e70 2023-12-07T10:10:44.899 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.463+0000 7fd6f0b6d700 1 -- 172.21.15.119:0/3997804793 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6e8011a00 con 0x7fd6ec108e70 2023-12-07T10:10:44.899 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.463+0000 7fd6f3dd3700 1 -- 172.21.15.119:0/3997804793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6ec108e70 msgr2=0x7fd6ec109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:44.899 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.463+0000 7fd6f3dd3700 1 --2- 172.21.15.119:0/3997804793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6ec108e70 0x7fd6ec109290 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd6e80098a0 tx=0x7fd6e8009bb0 comp rx=0 tx=0).stop 2023-12-07T10:10:44.900 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.463+0000 7fd6f3dd3700 1 -- 172.21.15.119:0/3997804793 shutdown_connections 2023-12-07T10:10:44.900 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.463+0000 7fd6f3dd3700 1 --2- 172.21.15.119:0/3997804793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6ec108e70 0x7fd6ec109290 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:44.900 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.463+0000 7fd6f3dd3700 1 -- 172.21.15.119:0/3997804793 >> 172.21.15.119:0/3997804793 conn(0x7fd6ec07c240 msgr2=0x7fd6ec07c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:44.900 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.463+0000 7fd6f3dd3700 1 -- 172.21.15.119:0/3997804793 shutdown_connections 2023-12-07T10:10:44.900 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.463+0000 7fd6f3dd3700 1 -- 172.21.15.119:0/3997804793 wait complete. 2023-12-07T10:10:44.900 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.464+0000 7fd6f3dd3700 1 Processor -- start 2023-12-07T10:10:44.900 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.464+0000 7fd6f3dd3700 1 -- start start 2023-12-07T10:10:44.900 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.465+0000 7fd6f3dd3700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6ec108e70 0x7fd6ec19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:44.900 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.465+0000 7fd6f3dd3700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec19d100 con 0x7fd6ec108e70 2023-12-07T10:10:44.900 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.465+0000 7fd6f1b6f700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6ec108e70 0x7fd6ec19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:44.900 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.465+0000 7fd6f1b6f700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6ec108e70 0x7fd6ec19cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34102/0 (socket says 172.21.15.119:34102) 2023-12-07T10:10:44.900 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.465+0000 7fd6f1b6f700 1 -- 172.21.15.119:0/179083196 learned_addr learned my addr 172.21.15.119:0/179083196 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:44.900 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.465+0000 7fd6f1b6f700 1 -- 172.21.15.119:0/179083196 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e8009550 con 0x7fd6ec108e70 2023-12-07T10:10:44.901 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.465+0000 7fd6f1b6f700 1 --2- 172.21.15.119:0/179083196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6ec108e70 0x7fd6ec19cbc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8009fb0 tx=0x7fd6e8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:44.901 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.466+0000 7fd6e2ffd700 1 -- 172.21.15.119:0/179083196 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6e8003a40 con 0x7fd6ec108e70 2023-12-07T10:10:44.901 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.466+0000 7fd6e2ffd700 1 -- 172.21.15.119:0/179083196 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd6e8003ba0 con 0x7fd6ec108e70 2023-12-07T10:10:44.901 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.466+0000 7fd6e2ffd700 1 -- 172.21.15.119:0/179083196 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6e801a750 con 0x7fd6ec108e70 2023-12-07T10:10:44.901 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.466+0000 7fd6f3dd3700 1 -- 172.21.15.119:0/179083196 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ec19d300 con 0x7fd6ec108e70 2023-12-07T10:10:44.901 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.466+0000 7fd6f3dd3700 1 -- 172.21.15.119:0/179083196 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ec19d7a0 con 0x7fd6ec108e70 2023-12-07T10:10:44.901 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.467+0000 7fd6f3dd3700 1 -- 172.21.15.119:0/179083196 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ec10ac50 con 0x7fd6ec108e70 2023-12-07T10:10:44.901 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.467+0000 7fd6e2ffd700 1 -- 172.21.15.119:0/179083196 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 45739+0+0 (secure 0 0 0) 0x7fd6e8024b60 con 0x7fd6ec108e70 2023-12-07T10:10:44.901 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.467+0000 7fd6e2ffd700 1 --2- 172.21.15.119:0/179083196 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7fd6d8038a50 0x7fd6d803af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:44.901 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.467+0000 7fd6e2ffd700 1 -- 172.21.15.119:0/179083196 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd6e804cb60 con 0x7fd6ec108e70 2023-12-07T10:10:44.901 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.468+0000 7fd6f136e700 1 --2- 172.21.15.119:0/179083196 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7fd6d8038a50 0x7fd6d803af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:44.901 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.468+0000 7fd6f136e700 1 --2- 172.21.15.119:0/179083196 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7fd6d8038a50 0x7fd6d803af10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc006fd0 tx=0x7fd6dc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:44.901 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.472+0000 7fd6e2ffd700 1 -- 172.21.15.119:0/179083196 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd6e8024430 con 0x7fd6ec108e70 2023-12-07T10:10:44.901 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:39.675+0000 7fd6f3dd3700 1 -- 172.21.15.119:0/179083196 --> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi119", "addr": "172.21.15.119", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6ec1a0030 con 0x7fd6d8038a50 2023-12-07T10:10:44.902 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:44.441+0000 7fd6e2ffd700 1 -- 172.21.15.119:0/179083196 <== mgr.14118 v2:172.21.15.119:6800/1355617847 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fd6ec1a0030 con 0x7fd6d8038a50 2023-12-07T10:10:44.902 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:44.444+0000 7fd6f3dd3700 1 -- 172.21.15.119:0/179083196 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7fd6d8038a50 msgr2=0x7fd6d803af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:44.902 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:44.444+0000 7fd6f3dd3700 1 --2- 172.21.15.119:0/179083196 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7fd6d8038a50 0x7fd6d803af10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc006fd0 tx=0x7fd6dc006e40 comp rx=0 tx=0).stop 2023-12-07T10:10:44.902 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:44.444+0000 7fd6f3dd3700 1 -- 172.21.15.119:0/179083196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6ec108e70 msgr2=0x7fd6ec19cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:44.902 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:44.444+0000 7fd6f3dd3700 1 --2- 172.21.15.119:0/179083196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6ec108e70 0x7fd6ec19cbc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8009fb0 tx=0x7fd6e8011ad0 comp rx=0 tx=0).stop 2023-12-07T10:10:44.902 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:44.444+0000 7fd6f3dd3700 1 -- 172.21.15.119:0/179083196 shutdown_connections 2023-12-07T10:10:44.902 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:44.444+0000 7fd6f3dd3700 1 --2- 172.21.15.119:0/179083196 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7fd6d8038a50 0x7fd6d803af10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:44.902 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:44.444+0000 7fd6f3dd3700 1 --2- 172.21.15.119:0/179083196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6ec108e70 0x7fd6ec19cbc0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:44.902 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:44.444+0000 7fd6f3dd3700 1 -- 172.21.15.119:0/179083196 >> 172.21.15.119:0/179083196 conn(0x7fd6ec07c240 msgr2=0x7fd6ec106550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:44.902 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:44.444+0000 7fd6f3dd3700 1 -- 172.21.15.119:0/179083196 shutdown_connections 2023-12-07T10:10:44.902 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:44.444+0000 7fd6f3dd3700 1 -- 172.21.15.119:0/179083196 wait complete. 2023-12-07T10:10:44.902 INFO:teuthology.orchestra.run.smithi119.stdout:Deploying unmanaged mon service... 2023-12-07T10:10:45.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:45 smithi119 ceph-mon[97908]: from='mgr.14118 172.21.15.119:0/725582431' entity='mgr.x' 2023-12-07T10:10:45.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:45 smithi119 ceph-mon[97908]: Added host smithi119 2023-12-07T10:10:45.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:45 smithi119 ceph-mon[97908]: from='mgr.14118 172.21.15.119:0/725582431' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:10:46.177 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-12-07T10:10:46.177 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.472+0000 7f956e7b7700 1 Processor -- start 2023-12-07T10:10:46.177 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.472+0000 7f956e7b7700 1 -- start start 2023-12-07T10:10:46.177 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.472+0000 7f956e7b7700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f956807ce90 0x7f956807b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:46.177 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.472+0000 7f956e7b7700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956807b8c0 con 0x7f956807ce90 2023-12-07T10:10:46.177 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.473+0000 7f9567fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f956807ce90 0x7f956807b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:46.177 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.473+0000 7f9567fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f956807ce90 0x7f956807b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56608/0 (socket says 172.21.15.119:56608) 2023-12-07T10:10:46.177 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.473+0000 7f9567fff700 1 -- 172.21.15.119:0/2292482513 learned_addr learned my addr 172.21.15.119:0/2292482513 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:46.177 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.473+0000 7f9567fff700 1 -- 172.21.15.119:0/2292482513 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956807ba00 con 0x7f956807ce90 2023-12-07T10:10:46.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.473+0000 7f9567fff700 1 --2- 172.21.15.119:0/2292482513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f956807ce90 0x7f956807b2f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f95580098a0 tx=0x7f9558009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1d004b2d8915ca server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:46.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.474+0000 7f9566ffd700 1 -- 172.21.15.119:0/2292482513 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9558004160 con 0x7f956807ce90 2023-12-07T10:10:46.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.474+0000 7f9566ffd700 1 -- 172.21.15.119:0/2292482513 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f95580042c0 con 0x7f956807ce90 2023-12-07T10:10:46.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.474+0000 7f9566ffd700 1 -- 172.21.15.119:0/2292482513 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9558011a00 con 0x7f956807ce90 2023-12-07T10:10:46.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.474+0000 7f956e7b7700 1 -- 172.21.15.119:0/2292482513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f956807ce90 msgr2=0x7f956807b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:46.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.474+0000 7f956e7b7700 1 --2- 172.21.15.119:0/2292482513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f956807ce90 0x7f956807b2f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f95580098a0 tx=0x7f9558009bb0 comp rx=0 tx=0).stop 2023-12-07T10:10:46.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.474+0000 7f956e7b7700 1 -- 172.21.15.119:0/2292482513 shutdown_connections 2023-12-07T10:10:46.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.474+0000 7f956e7b7700 1 --2- 172.21.15.119:0/2292482513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f956807ce90 0x7f956807b2f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:46.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.474+0000 7f956e7b7700 1 -- 172.21.15.119:0/2292482513 >> 172.21.15.119:0/2292482513 conn(0x7f9568102220 msgr2=0x7f9568104660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:46.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.474+0000 7f956e7b7700 1 -- 172.21.15.119:0/2292482513 shutdown_connections 2023-12-07T10:10:46.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.474+0000 7f956e7b7700 1 -- 172.21.15.119:0/2292482513 wait complete. 2023-12-07T10:10:46.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.475+0000 7f956e7b7700 1 Processor -- start 2023-12-07T10:10:46.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.476+0000 7f956e7b7700 1 -- start start 2023-12-07T10:10:46.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.476+0000 7f956e7b7700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f956807ce90 0x7f956819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:46.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.476+0000 7f956e7b7700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956819d110 con 0x7f956807ce90 2023-12-07T10:10:46.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.476+0000 7f9567fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f956807ce90 0x7f956819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:46.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.476+0000 7f9567fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f956807ce90 0x7f956819cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56610/0 (socket says 172.21.15.119:56610) 2023-12-07T10:10:46.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.476+0000 7f9567fff700 1 -- 172.21.15.119:0/1229664586 learned_addr learned my addr 172.21.15.119:0/1229664586 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:46.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.476+0000 7f9567fff700 1 -- 172.21.15.119:0/1229664586 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9558009550 con 0x7f956807ce90 2023-12-07T10:10:46.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.477+0000 7f9567fff700 1 --2- 172.21.15.119:0/1229664586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f956807ce90 0x7f956819cbd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9558009130 tx=0x7f9558011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:46.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.477+0000 7f95657fa700 1 -- 172.21.15.119:0/1229664586 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9558003ab0 con 0x7f956807ce90 2023-12-07T10:10:46.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.477+0000 7f95657fa700 1 -- 172.21.15.119:0/1229664586 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9558003c10 con 0x7f956807ce90 2023-12-07T10:10:46.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.477+0000 7f956e7b7700 1 -- 172.21.15.119:0/1229664586 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f956819d310 con 0x7f956807ce90 2023-12-07T10:10:46.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.477+0000 7f95657fa700 1 -- 172.21.15.119:0/1229664586 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95580247d0 con 0x7f956807ce90 2023-12-07T10:10:46.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.477+0000 7f956e7b7700 1 -- 172.21.15.119:0/1229664586 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f956819d7b0 con 0x7f956807ce90 2023-12-07T10:10:46.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.478+0000 7f95657fa700 1 -- 172.21.15.119:0/1229664586 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 45739+0+0 (secure 0 0 0) 0x7f955801a430 con 0x7f956807ce90 2023-12-07T10:10:46.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.478+0000 7f956e7b7700 1 -- 172.21.15.119:0/1229664586 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f956810d050 con 0x7f956807ce90 2023-12-07T10:10:46.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.479+0000 7f95657fa700 1 --2- 172.21.15.119:0/1229664586 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f9550038aa0 0x7f955003af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:46.180 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.479+0000 7f95657fa700 1 -- 172.21.15.119:0/1229664586 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f955804cc90 con 0x7f956807ce90 2023-12-07T10:10:46.180 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.479+0000 7f95677fe700 1 --2- 172.21.15.119:0/1229664586 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f9550038aa0 0x7f955003af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:46.180 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.480+0000 7f95677fe700 1 --2- 172.21.15.119:0/1229664586 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f9550038aa0 0x7f955003af60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f955c006fd0 tx=0x7f955c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:46.180 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.484+0000 7f95657fa700 1 -- 172.21.15.119:0/1229664586 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9558018b40 con 0x7f956807ce90 2023-12-07T10:10:46.180 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.702+0000 7f956e7b7700 1 -- 172.21.15.119:0/1229664586 --> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f9568103510 con 0x7f9550038aa0 2023-12-07T10:10:46.180 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.710+0000 7f95657fa700 1 -- 172.21.15.119:0/1229664586 <== mgr.14118 v2:172.21.15.119:6800/1355617847 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f9568103510 con 0x7f9550038aa0 2023-12-07T10:10:46.180 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.712+0000 7f956e7b7700 1 -- 172.21.15.119:0/1229664586 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f9550038aa0 msgr2=0x7f955003af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:46.180 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.712+0000 7f956e7b7700 1 --2- 172.21.15.119:0/1229664586 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f9550038aa0 0x7f955003af60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f955c006fd0 tx=0x7f955c006e40 comp rx=0 tx=0).stop 2023-12-07T10:10:46.180 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.712+0000 7f956e7b7700 1 -- 172.21.15.119:0/1229664586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f956807ce90 msgr2=0x7f956819cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:46.180 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.712+0000 7f956e7b7700 1 --2- 172.21.15.119:0/1229664586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f956807ce90 0x7f956819cbd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9558009130 tx=0x7f9558011ad0 comp rx=0 tx=0).stop 2023-12-07T10:10:46.180 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.712+0000 7f956e7b7700 1 -- 172.21.15.119:0/1229664586 shutdown_connections 2023-12-07T10:10:46.180 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.712+0000 7f956e7b7700 1 --2- 172.21.15.119:0/1229664586 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f9550038aa0 0x7f955003af60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:46.180 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.712+0000 7f956e7b7700 1 --2- 172.21.15.119:0/1229664586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f956807ce90 0x7f956819cbd0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:46.180 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.712+0000 7f956e7b7700 1 -- 172.21.15.119:0/1229664586 >> 172.21.15.119:0/1229664586 conn(0x7f9568102220 msgr2=0x7f9568102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:46.181 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.712+0000 7f956e7b7700 1 -- 172.21.15.119:0/1229664586 shutdown_connections 2023-12-07T10:10:46.181 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:45.712+0000 7f956e7b7700 1 -- 172.21.15.119:0/1229664586 wait complete. 2023-12-07T10:10:46.181 INFO:teuthology.orchestra.run.smithi119.stdout:Deploying unmanaged mgr service... 2023-12-07T10:10:46.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:46 smithi119 ceph-mon[97908]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:10:46.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:46 smithi119 ceph-mon[97908]: Saving service mon spec with placement count:5 2023-12-07T10:10:46.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:46 smithi119 ceph-mon[97908]: from='mgr.14118 172.21.15.119:0/725582431' entity='mgr.x' 2023-12-07T10:10:47.558 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-12-07T10:10:47.558 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.769+0000 7f294dd01700 1 Processor -- start 2023-12-07T10:10:47.558 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.769+0000 7f294dd01700 1 -- start start 2023-12-07T10:10:47.558 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.769+0000 7f294dd01700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2948106c20 0x7f2948107040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:47.558 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.769+0000 7f294dd01700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2948107610 con 0x7f2948106c20 2023-12-07T10:10:47.558 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.770+0000 7f29477fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2948106c20 0x7f2948107040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:47.558 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.770+0000 7f29477fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2948106c20 0x7f2948107040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56618/0 (socket says 172.21.15.119:56618) 2023-12-07T10:10:47.558 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.770+0000 7f29477fe700 1 -- 172.21.15.119:0/3651100255 learned_addr learned my addr 172.21.15.119:0/3651100255 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:47.558 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.770+0000 7f29477fe700 1 -- 172.21.15.119:0/3651100255 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2948107e20 con 0x7f2948106c20 2023-12-07T10:10:47.558 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.771+0000 7f29477fe700 1 --2- 172.21.15.119:0/3651100255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2948106c20 0x7f2948107040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f29380098a0 tx=0x7f2938009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5068fbd4fabf52 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:47.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.771+0000 7f29467fc700 1 -- 172.21.15.119:0/3651100255 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2938004160 con 0x7f2948106c20 2023-12-07T10:10:47.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.771+0000 7f29467fc700 1 -- 172.21.15.119:0/3651100255 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f29380042c0 con 0x7f2948106c20 2023-12-07T10:10:47.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.771+0000 7f29467fc700 1 -- 172.21.15.119:0/3651100255 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2938011a00 con 0x7f2948106c20 2023-12-07T10:10:47.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.771+0000 7f294dd01700 1 -- 172.21.15.119:0/3651100255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2948106c20 msgr2=0x7f2948107040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:47.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.771+0000 7f294dd01700 1 --2- 172.21.15.119:0/3651100255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2948106c20 0x7f2948107040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f29380098a0 tx=0x7f2938009bb0 comp rx=0 tx=0).stop 2023-12-07T10:10:47.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.772+0000 7f294dd01700 1 -- 172.21.15.119:0/3651100255 shutdown_connections 2023-12-07T10:10:47.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.772+0000 7f294dd01700 1 --2- 172.21.15.119:0/3651100255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2948106c20 0x7f2948107040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:47.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.772+0000 7f294dd01700 1 -- 172.21.15.119:0/3651100255 >> 172.21.15.119:0/3651100255 conn(0x7f2948102200 msgr2=0x7f2948104620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:47.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.772+0000 7f294dd01700 1 -- 172.21.15.119:0/3651100255 shutdown_connections 2023-12-07T10:10:47.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.772+0000 7f294dd01700 1 -- 172.21.15.119:0/3651100255 wait complete. 2023-12-07T10:10:47.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.773+0000 7f294dd01700 1 Processor -- start 2023-12-07T10:10:47.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.773+0000 7f294dd01700 1 -- start start 2023-12-07T10:10:47.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.773+0000 7f294dd01700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2948106c20 0x7f294807cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:47.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.773+0000 7f294dd01700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294807d050 con 0x7f2948106c20 2023-12-07T10:10:47.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.773+0000 7f29477fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2948106c20 0x7f294807cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:47.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.773+0000 7f29477fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2948106c20 0x7f294807cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56626/0 (socket says 172.21.15.119:56626) 2023-12-07T10:10:47.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.773+0000 7f29477fe700 1 -- 172.21.15.119:0/2396492080 learned_addr learned my addr 172.21.15.119:0/2396492080 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:47.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.774+0000 7f29477fe700 1 -- 172.21.15.119:0/2396492080 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2938009550 con 0x7f2948106c20 2023-12-07T10:10:47.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.774+0000 7f29477fe700 1 --2- 172.21.15.119:0/2396492080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2948106c20 0x7f294807cb10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2938009130 tx=0x7f2938011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:47.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.774+0000 7f2944ff9700 1 -- 172.21.15.119:0/2396492080 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2938003ab0 con 0x7f2948106c20 2023-12-07T10:10:47.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.774+0000 7f2944ff9700 1 -- 172.21.15.119:0/2396492080 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2938003c10 con 0x7f2948106c20 2023-12-07T10:10:47.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.774+0000 7f294dd01700 1 -- 172.21.15.119:0/2396492080 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294807b290 con 0x7f2948106c20 2023-12-07T10:10:47.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.774+0000 7f2944ff9700 1 -- 172.21.15.119:0/2396492080 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29380247d0 con 0x7f2948106c20 2023-12-07T10:10:47.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.774+0000 7f294dd01700 1 -- 172.21.15.119:0/2396492080 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f294807b730 con 0x7f2948106c20 2023-12-07T10:10:47.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.776+0000 7f2944ff9700 1 -- 172.21.15.119:0/2396492080 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 45739+0+0 (secure 0 0 0) 0x7f293802d3f0 con 0x7f2948106c20 2023-12-07T10:10:47.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.776+0000 7f294dd01700 1 -- 172.21.15.119:0/2396492080 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2948062c40 con 0x7f2948106c20 2023-12-07T10:10:47.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.776+0000 7f2944ff9700 1 --2- 172.21.15.119:0/2396492080 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f2930038aa0 0x7f293003af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:47.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.776+0000 7f2944ff9700 1 -- 172.21.15.119:0/2396492080 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f293804cc40 con 0x7f2948106c20 2023-12-07T10:10:47.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.777+0000 7f2946ffd700 1 --2- 172.21.15.119:0/2396492080 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f2930038aa0 0x7f293003af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:47.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.777+0000 7f2946ffd700 1 --2- 172.21.15.119:0/2396492080 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f2930038aa0 0x7f293003af60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f293c006fd0 tx=0x7f293c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:47.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:46.781+0000 7f2944ff9700 1 -- 172.21.15.119:0/2396492080 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2938010970 con 0x7f2948106c20 2023-12-07T10:10:47.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:47.013+0000 7f294dd01700 1 -- 172.21.15.119:0/2396492080 --> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f294807bd60 con 0x7f2930038aa0 2023-12-07T10:10:47.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:47.021+0000 7f2944ff9700 1 -- 172.21.15.119:0/2396492080 <== mgr.14118 v2:172.21.15.119:6800/1355617847 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f294807bd60 con 0x7f2930038aa0 2023-12-07T10:10:47.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:47.023+0000 7f294dd01700 1 -- 172.21.15.119:0/2396492080 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f2930038aa0 msgr2=0x7f293003af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:47.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:47.023+0000 7f294dd01700 1 --2- 172.21.15.119:0/2396492080 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f2930038aa0 0x7f293003af60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f293c006fd0 tx=0x7f293c006e40 comp rx=0 tx=0).stop 2023-12-07T10:10:47.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:47.023+0000 7f294dd01700 1 -- 172.21.15.119:0/2396492080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2948106c20 msgr2=0x7f294807cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:47.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:47.023+0000 7f294dd01700 1 --2- 172.21.15.119:0/2396492080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2948106c20 0x7f294807cb10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2938009130 tx=0x7f2938011ad0 comp rx=0 tx=0).stop 2023-12-07T10:10:47.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:47.023+0000 7f294dd01700 1 -- 172.21.15.119:0/2396492080 shutdown_connections 2023-12-07T10:10:47.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:47.023+0000 7f294dd01700 1 --2- 172.21.15.119:0/2396492080 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f2930038aa0 0x7f293003af60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:47.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:47.023+0000 7f294dd01700 1 --2- 172.21.15.119:0/2396492080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2948106c20 0x7f294807cb10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:47.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:47.023+0000 7f294dd01700 1 -- 172.21.15.119:0/2396492080 >> 172.21.15.119:0/2396492080 conn(0x7f2948102200 msgr2=0x7f2948102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:47.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:47.023+0000 7f294dd01700 1 -- 172.21.15.119:0/2396492080 shutdown_connections 2023-12-07T10:10:47.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:47.023+0000 7f294dd01700 1 -- 172.21.15.119:0/2396492080 wait complete. 2023-12-07T10:10:48.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:48 smithi119 ceph-mon[97908]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:10:48.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:48 smithi119 ceph-mon[97908]: Saving service mgr spec with placement count:2 2023-12-07T10:10:48.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:48 smithi119 ceph-mon[97908]: from='mgr.14118 172.21.15.119:0/725582431' entity='mgr.x' 2023-12-07T10:10:48.943 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.325+0000 7f6968d2c700 1 Processor -- start 2023-12-07T10:10:48.944 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.326+0000 7f6968d2c700 1 -- start start 2023-12-07T10:10:48.944 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.326+0000 7f6968d2c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6964108e60 0x7f6964109280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:48.944 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.326+0000 7f6968d2c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69641097c0 con 0x7f6964108e60 2023-12-07T10:10:48.944 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.326+0000 7f696259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6964108e60 0x7f6964109280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:48.944 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.326+0000 7f696259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6964108e60 0x7f6964109280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56630/0 (socket says 172.21.15.119:56630) 2023-12-07T10:10:48.944 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.326+0000 7f696259c700 1 -- 172.21.15.119:0/1539514646 learned_addr learned my addr 172.21.15.119:0/1539514646 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:48.944 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.327+0000 7f696259c700 1 -- 172.21.15.119:0/1539514646 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6964109900 con 0x7f6964108e60 2023-12-07T10:10:48.944 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.327+0000 7f696259c700 1 --2- 172.21.15.119:0/1539514646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6964108e60 0x7f6964109280 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f69540098a0 tx=0x7f6954009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f29462320d3d8f9 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:48.945 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.327+0000 7f696159a700 1 -- 172.21.15.119:0/1539514646 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6954004160 con 0x7f6964108e60 2023-12-07T10:10:48.945 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.327+0000 7f696159a700 1 -- 172.21.15.119:0/1539514646 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69540042c0 con 0x7f6964108e60 2023-12-07T10:10:48.945 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.327+0000 7f696159a700 1 -- 172.21.15.119:0/1539514646 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69540119b0 con 0x7f6964108e60 2023-12-07T10:10:48.945 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.328+0000 7f6968d2c700 1 -- 172.21.15.119:0/1539514646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6964108e60 msgr2=0x7f6964109280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:48.945 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.328+0000 7f6968d2c700 1 --2- 172.21.15.119:0/1539514646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6964108e60 0x7f6964109280 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f69540098a0 tx=0x7f6954009bb0 comp rx=0 tx=0).stop 2023-12-07T10:10:48.945 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.328+0000 7f6968d2c700 1 -- 172.21.15.119:0/1539514646 shutdown_connections 2023-12-07T10:10:48.945 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.328+0000 7f6968d2c700 1 --2- 172.21.15.119:0/1539514646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6964108e60 0x7f6964109280 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:48.946 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.328+0000 7f6968d2c700 1 -- 172.21.15.119:0/1539514646 >> 172.21.15.119:0/1539514646 conn(0x7f696407c240 msgr2=0x7f696407c650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:48.946 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.328+0000 7f6968d2c700 1 -- 172.21.15.119:0/1539514646 shutdown_connections 2023-12-07T10:10:48.946 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.328+0000 7f6968d2c700 1 -- 172.21.15.119:0/1539514646 wait complete. 2023-12-07T10:10:48.946 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.329+0000 7f6968d2c700 1 Processor -- start 2023-12-07T10:10:48.946 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.329+0000 7f6968d2c700 1 -- start start 2023-12-07T10:10:48.946 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.329+0000 7f6968d2c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6964108e60 0x7f696419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:48.946 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.329+0000 7f6968d2c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696419d100 con 0x7f6964108e60 2023-12-07T10:10:48.946 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.330+0000 7f696259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6964108e60 0x7f696419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:48.947 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.330+0000 7f696259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6964108e60 0x7f696419cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56634/0 (socket says 172.21.15.119:56634) 2023-12-07T10:10:48.947 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.330+0000 7f696259c700 1 -- 172.21.15.119:0/3387271664 learned_addr learned my addr 172.21.15.119:0/3387271664 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:48.947 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.330+0000 7f696259c700 1 -- 172.21.15.119:0/3387271664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6954009550 con 0x7f6964108e60 2023-12-07T10:10:48.947 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.330+0000 7f696259c700 1 --2- 172.21.15.119:0/3387271664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6964108e60 0x7f696419cbc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6954011670 tx=0x7f6954011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:48.947 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.330+0000 7f69537fe700 1 -- 172.21.15.119:0/3387271664 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6954003b20 con 0x7f6964108e60 2023-12-07T10:10:48.947 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.330+0000 7f69537fe700 1 -- 172.21.15.119:0/3387271664 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6954003c80 con 0x7f6964108e60 2023-12-07T10:10:48.947 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.330+0000 7f69537fe700 1 -- 172.21.15.119:0/3387271664 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6954019770 con 0x7f6964108e60 2023-12-07T10:10:48.947 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.330+0000 7f6968d2c700 1 -- 172.21.15.119:0/3387271664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f696419d300 con 0x7f6964108e60 2023-12-07T10:10:48.948 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.331+0000 7f6968d2c700 1 -- 172.21.15.119:0/3387271664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f696419d7a0 con 0x7f6964108e60 2023-12-07T10:10:48.948 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.332+0000 7f69537fe700 1 -- 172.21.15.119:0/3387271664 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 45739+0+0 (secure 0 0 0) 0x7f695401ab60 con 0x7f6964108e60 2023-12-07T10:10:48.948 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.332+0000 7f6968d2c700 1 -- 172.21.15.119:0/3387271664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f696410acd0 con 0x7f6964108e60 2023-12-07T10:10:48.948 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.332+0000 7f69537fe700 1 --2- 172.21.15.119:0/3387271664 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f694c038aa0 0x7f694c03af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:48.948 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.332+0000 7f69537fe700 1 -- 172.21.15.119:0/3387271664 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f695404cd70 con 0x7f6964108e60 2023-12-07T10:10:48.949 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.332+0000 7f6961d9b700 1 --2- 172.21.15.119:0/3387271664 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f694c038aa0 0x7f694c03af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:48.949 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.333+0000 7f6961d9b700 1 --2- 172.21.15.119:0/3387271664 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f694c038aa0 0x7f694c03af60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6958006fd0 tx=0x7f6958006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:48.949 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.337+0000 7f69537fe700 1 -- 172.21.15.119:0/3387271664 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f6954051070 con 0x7f6964108e60 2023-12-07T10:10:48.949 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.543+0000 7f6968d2c700 1 -- 172.21.15.119:0/3387271664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f69641a0090 con 0x7f6964108e60 2023-12-07T10:10:48.949 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.546+0000 7f69537fe700 1 -- 172.21.15.119:0/3387271664 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f695401a430 con 0x7f6964108e60 2023-12-07T10:10:48.949 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.549+0000 7f6968d2c700 1 -- 172.21.15.119:0/3387271664 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f694c038aa0 msgr2=0x7f694c03af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:48.949 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.549+0000 7f6968d2c700 1 --2- 172.21.15.119:0/3387271664 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f694c038aa0 0x7f694c03af60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6958006fd0 tx=0x7f6958006e40 comp rx=0 tx=0).stop 2023-12-07T10:10:48.949 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.549+0000 7f6968d2c700 1 -- 172.21.15.119:0/3387271664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6964108e60 msgr2=0x7f696419cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:48.950 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.549+0000 7f6968d2c700 1 --2- 172.21.15.119:0/3387271664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6964108e60 0x7f696419cbc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6954011670 tx=0x7f6954011a30 comp rx=0 tx=0).stop 2023-12-07T10:10:48.950 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.549+0000 7f6968d2c700 1 -- 172.21.15.119:0/3387271664 shutdown_connections 2023-12-07T10:10:48.950 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.549+0000 7f6968d2c700 1 --2- 172.21.15.119:0/3387271664 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f694c038aa0 0x7f694c03af60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:48.950 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.549+0000 7f6968d2c700 1 --2- 172.21.15.119:0/3387271664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6964108e60 0x7f696419cbc0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:48.950 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.549+0000 7f6968d2c700 1 -- 172.21.15.119:0/3387271664 >> 172.21.15.119:0/3387271664 conn(0x7f696407c240 msgr2=0x7f69641064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:48.950 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.549+0000 7f6968d2c700 1 -- 172.21.15.119:0/3387271664 shutdown_connections 2023-12-07T10:10:48.950 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:48.549+0000 7f6968d2c700 1 -- 172.21.15.119:0/3387271664 wait complete. 2023-12-07T10:10:49.275 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:49 smithi119 ceph-mon[97908]: from='mgr.14118 172.21.15.119:0/725582431' entity='mgr.x' 2023-12-07T10:10:49.276 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:49 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3387271664' entity='client.admin' 2023-12-07T10:10:49.276 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:49 smithi119 ceph-mon[97908]: from='mgr.14118 172.21.15.119:0/725582431' entity='mgr.x' 2023-12-07T10:10:50.151 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.513+0000 7f148142d700 1 Processor -- start 2023-12-07T10:10:50.151 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.513+0000 7f148142d700 1 -- start start 2023-12-07T10:10:50.151 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.513+0000 7f148142d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f147c07ce70 0x7f147c07b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:50.151 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.513+0000 7f148142d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147c07b8a0 con 0x7f147c07ce70 2023-12-07T10:10:50.151 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.514+0000 7f147affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f147c07ce70 0x7f147c07b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:50.151 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.514+0000 7f147affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f147c07ce70 0x7f147c07b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56642/0 (socket says 172.21.15.119:56642) 2023-12-07T10:10:50.151 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.514+0000 7f147affd700 1 -- 172.21.15.119:0/2015411449 learned_addr learned my addr 172.21.15.119:0/2015411449 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:50.151 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.514+0000 7f147affd700 1 -- 172.21.15.119:0/2015411449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f147c07b9e0 con 0x7f147c07ce70 2023-12-07T10:10:50.151 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.515+0000 7f147affd700 1 --2- 172.21.15.119:0/2015411449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f147c07ce70 0x7f147c07b2d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f146c0098a0 tx=0x7f146c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da8a8a0e10b819d server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:50.151 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.515+0000 7f1479ffb700 1 -- 172.21.15.119:0/2015411449 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f146c004160 con 0x7f147c07ce70 2023-12-07T10:10:50.151 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.515+0000 7f1479ffb700 1 -- 172.21.15.119:0/2015411449 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f146c0042c0 con 0x7f147c07ce70 2023-12-07T10:10:50.152 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.515+0000 7f1479ffb700 1 -- 172.21.15.119:0/2015411449 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f146c0119b0 con 0x7f147c07ce70 2023-12-07T10:10:50.152 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.515+0000 7f148142d700 1 -- 172.21.15.119:0/2015411449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f147c07ce70 msgr2=0x7f147c07b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:50.152 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.515+0000 7f148142d700 1 --2- 172.21.15.119:0/2015411449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f147c07ce70 0x7f147c07b2d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f146c0098a0 tx=0x7f146c009bb0 comp rx=0 tx=0).stop 2023-12-07T10:10:50.152 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.516+0000 7f148142d700 1 -- 172.21.15.119:0/2015411449 shutdown_connections 2023-12-07T10:10:50.152 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.516+0000 7f148142d700 1 --2- 172.21.15.119:0/2015411449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f147c07ce70 0x7f147c07b2d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:50.152 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.516+0000 7f148142d700 1 -- 172.21.15.119:0/2015411449 >> 172.21.15.119:0/2015411449 conn(0x7f147c102220 msgr2=0x7f147c104660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:50.152 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.516+0000 7f148142d700 1 -- 172.21.15.119:0/2015411449 shutdown_connections 2023-12-07T10:10:50.152 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.516+0000 7f148142d700 1 -- 172.21.15.119:0/2015411449 wait complete. 2023-12-07T10:10:50.152 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.517+0000 7f148142d700 1 Processor -- start 2023-12-07T10:10:50.152 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.517+0000 7f148142d700 1 -- start start 2023-12-07T10:10:50.152 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.517+0000 7f148142d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f147c07ce70 0x7f147c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:50.152 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.517+0000 7f148142d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147c19d170 con 0x7f147c07ce70 2023-12-07T10:10:50.153 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.517+0000 7f147affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f147c07ce70 0x7f147c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:50.153 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.518+0000 7f147affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f147c07ce70 0x7f147c19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56656/0 (socket says 172.21.15.119:56656) 2023-12-07T10:10:50.153 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.518+0000 7f147affd700 1 -- 172.21.15.119:0/3744292033 learned_addr learned my addr 172.21.15.119:0/3744292033 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:50.153 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.518+0000 7f147affd700 1 -- 172.21.15.119:0/3744292033 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c009550 con 0x7f147c07ce70 2023-12-07T10:10:50.153 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.518+0000 7f147affd700 1 --2- 172.21.15.119:0/3744292033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f147c07ce70 0x7f147c19cc30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f146c011670 tx=0x7f146c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:50.153 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.518+0000 7f1463fff700 1 -- 172.21.15.119:0/3744292033 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f146c003b20 con 0x7f147c07ce70 2023-12-07T10:10:50.153 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.518+0000 7f1463fff700 1 -- 172.21.15.119:0/3744292033 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f146c003c80 con 0x7f147c07ce70 2023-12-07T10:10:50.153 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.518+0000 7f1463fff700 1 -- 172.21.15.119:0/3744292033 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f146c019770 con 0x7f147c07ce70 2023-12-07T10:10:50.153 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.518+0000 7f148142d700 1 -- 172.21.15.119:0/3744292033 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f147c19d370 con 0x7f147c07ce70 2023-12-07T10:10:50.153 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.519+0000 7f148142d700 1 -- 172.21.15.119:0/3744292033 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f147c19d810 con 0x7f147c07ce70 2023-12-07T10:10:50.153 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.520+0000 7f1463fff700 1 -- 172.21.15.119:0/3744292033 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 45739+0+0 (secure 0 0 0) 0x7f146c0199e0 con 0x7f147c07ce70 2023-12-07T10:10:50.153 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.520+0000 7f148142d700 1 -- 172.21.15.119:0/3744292033 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f147c10d0c0 con 0x7f147c07ce70 2023-12-07T10:10:50.153 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.520+0000 7f1463fff700 1 --2- 172.21.15.119:0/3744292033 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f1464038aa0 0x7f146403af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:50.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.520+0000 7f1463fff700 1 -- 172.21.15.119:0/3744292033 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f146c04cf60 con 0x7f147c07ce70 2023-12-07T10:10:50.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.520+0000 7f147a7fc700 1 --2- 172.21.15.119:0/3744292033 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f1464038aa0 0x7f146403af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:50.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.521+0000 7f147a7fc700 1 --2- 172.21.15.119:0/3744292033 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f1464038aa0 0x7f146403af60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1470006fd0 tx=0x7f1470006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:50.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.525+0000 7f1463fff700 1 -- 172.21.15.119:0/3744292033 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f146c018ba0 con 0x7f147c07ce70 2023-12-07T10:10:50.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.733+0000 7f148142d700 1 -- 172.21.15.119:0/3744292033 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f147c1a0050 con 0x7f147c07ce70 2023-12-07T10:10:50.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.735+0000 7f1463fff700 1 -- 172.21.15.119:0/3744292033 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f146c023e60 con 0x7f147c07ce70 2023-12-07T10:10:50.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.738+0000 7f148142d700 1 -- 172.21.15.119:0/3744292033 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f1464038aa0 msgr2=0x7f146403af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:50.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.738+0000 7f148142d700 1 --2- 172.21.15.119:0/3744292033 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f1464038aa0 0x7f146403af60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1470006fd0 tx=0x7f1470006e40 comp rx=0 tx=0).stop 2023-12-07T10:10:50.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.738+0000 7f148142d700 1 -- 172.21.15.119:0/3744292033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f147c07ce70 msgr2=0x7f147c19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:50.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.738+0000 7f148142d700 1 --2- 172.21.15.119:0/3744292033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f147c07ce70 0x7f147c19cc30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f146c011670 tx=0x7f146c011a30 comp rx=0 tx=0).stop 2023-12-07T10:10:50.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.738+0000 7f148142d700 1 -- 172.21.15.119:0/3744292033 shutdown_connections 2023-12-07T10:10:50.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.738+0000 7f148142d700 1 --2- 172.21.15.119:0/3744292033 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f1464038aa0 0x7f146403af60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:50.155 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.738+0000 7f148142d700 1 --2- 172.21.15.119:0/3744292033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f147c07ce70 0x7f147c19cc30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:50.155 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.738+0000 7f148142d700 1 -- 172.21.15.119:0/3744292033 >> 172.21.15.119:0/3744292033 conn(0x7f147c102220 msgr2=0x7f147c102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:50.155 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.739+0000 7f148142d700 1 -- 172.21.15.119:0/3744292033 shutdown_connections 2023-12-07T10:10:50.155 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:49.739+0000 7f148142d700 1 -- 172.21.15.119:0/3744292033 wait complete. 2023-12-07T10:10:50.155 INFO:teuthology.orchestra.run.smithi119.stdout:Enabling the dashboard module... 2023-12-07T10:10:50.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:50 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3744292033' entity='client.admin' 2023-12-07T10:10:52.284 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:51 smithi119 ceph-mon[97908]: from='mgr.14118 172.21.15.119:0/725582431' entity='mgr.x' 2023-12-07T10:10:52.284 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:51 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1346762240' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-12-07T10:10:52.285 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:52 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: ignoring --setuser ceph since I am not root 2023-12-07T10:10:52.285 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:52 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: ignoring --setgroup ceph since I am not root 2023-12-07T10:10:52.436 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.882+0000 7fb0d1ea3700 1 Processor -- start 2023-12-07T10:10:52.436 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.882+0000 7fb0d1ea3700 1 -- start start 2023-12-07T10:10:52.436 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.883+0000 7fb0d1ea3700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0cc07ce90 0x7fb0cc07b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:52.436 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.883+0000 7fb0d1ea3700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc07b8c0 con 0x7fb0cc07ce90 2023-12-07T10:10:52.436 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.883+0000 7fb0cb7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0cc07ce90 0x7fb0cc07b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:52.436 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.883+0000 7fb0cb7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0cc07ce90 0x7fb0cc07b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56664/0 (socket says 172.21.15.119:56664) 2023-12-07T10:10:52.436 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.883+0000 7fb0cb7fe700 1 -- 172.21.15.119:0/296463767 learned_addr learned my addr 172.21.15.119:0/296463767 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:52.436 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.884+0000 7fb0cb7fe700 1 -- 172.21.15.119:0/296463767 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0cc07ba00 con 0x7fb0cc07ce90 2023-12-07T10:10:52.436 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.884+0000 7fb0cb7fe700 1 --2- 172.21.15.119:0/296463767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0cc07ce90 0x7fb0cc07b2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0098a0 tx=0x7fb0bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8aeebda257b0a9 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:52.436 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.884+0000 7fb0ca7fc700 1 -- 172.21.15.119:0/296463767 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0bc004160 con 0x7fb0cc07ce90 2023-12-07T10:10:52.436 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.884+0000 7fb0ca7fc700 1 -- 172.21.15.119:0/296463767 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0bc0042c0 con 0x7fb0cc07ce90 2023-12-07T10:10:52.436 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.884+0000 7fb0ca7fc700 1 -- 172.21.15.119:0/296463767 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0bc0119b0 con 0x7fb0cc07ce90 2023-12-07T10:10:52.436 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.885+0000 7fb0d1ea3700 1 -- 172.21.15.119:0/296463767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0cc07ce90 msgr2=0x7fb0cc07b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:52.437 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.885+0000 7fb0d1ea3700 1 --2- 172.21.15.119:0/296463767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0cc07ce90 0x7fb0cc07b2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0098a0 tx=0x7fb0bc009bb0 comp rx=0 tx=0).stop 2023-12-07T10:10:52.437 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.885+0000 7fb0d1ea3700 1 -- 172.21.15.119:0/296463767 shutdown_connections 2023-12-07T10:10:52.437 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.885+0000 7fb0d1ea3700 1 --2- 172.21.15.119:0/296463767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0cc07ce90 0x7fb0cc07b2f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:52.437 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.885+0000 7fb0d1ea3700 1 -- 172.21.15.119:0/296463767 >> 172.21.15.119:0/296463767 conn(0x7fb0cc102220 msgr2=0x7fb0cc104640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:52.437 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.885+0000 7fb0d1ea3700 1 -- 172.21.15.119:0/296463767 shutdown_connections 2023-12-07T10:10:52.437 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.885+0000 7fb0d1ea3700 1 -- 172.21.15.119:0/296463767 wait complete. 2023-12-07T10:10:52.437 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.886+0000 7fb0d1ea3700 1 Processor -- start 2023-12-07T10:10:52.437 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.886+0000 7fb0d1ea3700 1 -- start start 2023-12-07T10:10:52.437 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.886+0000 7fb0d1ea3700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0cc07ce90 0x7fb0cc19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:52.437 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.886+0000 7fb0d1ea3700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc19d140 con 0x7fb0cc07ce90 2023-12-07T10:10:52.437 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.887+0000 7fb0cb7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0cc07ce90 0x7fb0cc19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:52.437 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.887+0000 7fb0cb7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0cc07ce90 0x7fb0cc19cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56680/0 (socket says 172.21.15.119:56680) 2023-12-07T10:10:52.437 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.887+0000 7fb0cb7fe700 1 -- 172.21.15.119:0/1346762240 learned_addr learned my addr 172.21.15.119:0/1346762240 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:52.437 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.887+0000 7fb0cb7fe700 1 -- 172.21.15.119:0/1346762240 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0bc009550 con 0x7fb0cc07ce90 2023-12-07T10:10:52.438 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.887+0000 7fb0cb7fe700 1 --2- 172.21.15.119:0/1346762240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0cc07ce90 0x7fb0cc19cc00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc003e60 tx=0x7fb0bc011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:52.438 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.887+0000 7fb0c8ff9700 1 -- 172.21.15.119:0/1346762240 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0bc003a60 con 0x7fb0cc07ce90 2023-12-07T10:10:52.438 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.887+0000 7fb0c8ff9700 1 -- 172.21.15.119:0/1346762240 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0bc003bc0 con 0x7fb0cc07ce90 2023-12-07T10:10:52.438 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.887+0000 7fb0d1ea3700 1 -- 172.21.15.119:0/1346762240 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0cc19d340 con 0x7fb0cc07ce90 2023-12-07T10:10:52.438 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.887+0000 7fb0c8ff9700 1 -- 172.21.15.119:0/1346762240 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0bc0196c0 con 0x7fb0cc07ce90 2023-12-07T10:10:52.438 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.888+0000 7fb0d1ea3700 1 -- 172.21.15.119:0/1346762240 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0cc19d7e0 con 0x7fb0cc07ce90 2023-12-07T10:10:52.438 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.889+0000 7fb0d1ea3700 1 -- 172.21.15.119:0/1346762240 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0cc10d080 con 0x7fb0cc07ce90 2023-12-07T10:10:52.438 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.889+0000 7fb0c8ff9700 1 -- 172.21.15.119:0/1346762240 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 45739+0+0 (secure 0 0 0) 0x7fb0bc01ab60 con 0x7fb0cc07ce90 2023-12-07T10:10:52.438 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.889+0000 7fb0c8ff9700 1 --2- 172.21.15.119:0/1346762240 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7fb0b4038a80 0x7fb0b403af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:52.438 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.889+0000 7fb0c8ff9700 1 -- 172.21.15.119:0/1346762240 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb0bc04cd80 con 0x7fb0cc07ce90 2023-12-07T10:10:52.438 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.890+0000 7fb0caffd700 1 --2- 172.21.15.119:0/1346762240 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7fb0b4038a80 0x7fb0b403af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:52.438 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.890+0000 7fb0caffd700 1 --2- 172.21.15.119:0/1346762240 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7fb0b4038a80 0x7fb0b403af40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0006fd0 tx=0x7fb0c0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:52.438 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:50.895+0000 7fb0c8ff9700 1 -- 172.21.15.119:0/1346762240 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fb0bc018b00 con 0x7fb0cc07ce90 2023-12-07T10:10:52.439 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:51.167+0000 7fb0d1ea3700 1 -- 172.21.15.119:0/1346762240 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fb0cc10d290 con 0x7fb0cc07ce90 2023-12-07T10:10:52.439 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:51.985+0000 7fb0c8ff9700 1 -- 172.21.15.119:0/1346762240 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fb0bc018b00 con 0x7fb0cc07ce90 2023-12-07T10:10:52.439 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:51.987+0000 7fb0c8ff9700 1 -- 172.21.15.119:0/1346762240 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mgrmap(e 9) v1 ==== 45752+0+0 (secure 0 0 0) 0x7fb0bc01a7b0 con 0x7fb0cc07ce90 2023-12-07T10:10:52.439 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:51.988+0000 7fb0d1ea3700 1 -- 172.21.15.119:0/1346762240 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7fb0b4038a80 msgr2=0x7fb0b403af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:52.439 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:51.988+0000 7fb0d1ea3700 1 --2- 172.21.15.119:0/1346762240 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7fb0b4038a80 0x7fb0b403af40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0006fd0 tx=0x7fb0c0006e40 comp rx=0 tx=0).stop 2023-12-07T10:10:52.439 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:51.988+0000 7fb0d1ea3700 1 -- 172.21.15.119:0/1346762240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0cc07ce90 msgr2=0x7fb0cc19cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:52.439 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:51.988+0000 7fb0d1ea3700 1 --2- 172.21.15.119:0/1346762240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0cc07ce90 0x7fb0cc19cc00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc003e60 tx=0x7fb0bc011a30 comp rx=0 tx=0).stop 2023-12-07T10:10:52.439 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:51.989+0000 7fb0d1ea3700 1 -- 172.21.15.119:0/1346762240 shutdown_connections 2023-12-07T10:10:52.439 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:51.989+0000 7fb0d1ea3700 1 --2- 172.21.15.119:0/1346762240 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7fb0b4038a80 0x7fb0b403af40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:52.439 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:51.989+0000 7fb0d1ea3700 1 --2- 172.21.15.119:0/1346762240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0cc07ce90 0x7fb0cc19cc00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:52.439 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:51.989+0000 7fb0d1ea3700 1 -- 172.21.15.119:0/1346762240 >> 172.21.15.119:0/1346762240 conn(0x7fb0cc102220 msgr2=0x7fb0cc102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:52.439 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:51.989+0000 7fb0d1ea3700 1 -- 172.21.15.119:0/1346762240 shutdown_connections 2023-12-07T10:10:52.439 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:51.989+0000 7fb0d1ea3700 1 -- 172.21.15.119:0/1346762240 wait complete. 2023-12-07T10:10:52.636 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:52 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:52.281+0000 7f8af4ba5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-07T10:10:52.636 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:52 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:52.387+0000 7f8af4ba5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-07T10:10:52.636 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:52 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:52.509+0000 7f8af4ba5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-07T10:10:52.636 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:52 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:52.632+0000 7f8af4ba5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-07T10:10:52.980 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:52 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:52.867+0000 7f8af4ba5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-07T10:10:53.298 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:52 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1346762240' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-12-07T10:10:53.298 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:52 smithi119 ceph-mon[97908]: mgrmap e9: x(active, since 19s) 2023-12-07T10:10:53.638 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:53 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:53.340+0000 7f8af4ba5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-07T10:10:53.638 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:53 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:53.517+0000 7f8af4ba5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-07T10:10:53.638 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:53 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:53.634+0000 7f8af4ba5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-07T10:10:53.718 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2023-12-07T10:10:53.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-12-07T10:10:53.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-07T10:10:53.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-12-07T10:10:53.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-12-07T10:10:53.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2023-12-07T10:10:53.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:52.997+0000 7f0b6c00d700 1 Processor -- start 2023-12-07T10:10:53.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:52.997+0000 7f0b6c00d700 1 -- start start 2023-12-07T10:10:53.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:52.997+0000 7f0b6c00d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b6407ce50 0x7f0b6407b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:53.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:52.997+0000 7f0b6c00d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6407b880 con 0x7f0b6407ce50 2023-12-07T10:10:53.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:52.998+0000 7f0b69da9700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b6407ce50 0x7f0b6407b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:53.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:52.998+0000 7f0b69da9700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b6407ce50 0x7f0b6407b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36080/0 (socket says 172.21.15.119:36080) 2023-12-07T10:10:53.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:52.998+0000 7f0b69da9700 1 -- 172.21.15.119:0/1996994958 learned_addr learned my addr 172.21.15.119:0/1996994958 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:53.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:52.998+0000 7f0b69da9700 1 -- 172.21.15.119:0/1996994958 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6407b9c0 con 0x7f0b6407ce50 2023-12-07T10:10:53.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:52.999+0000 7f0b69da9700 1 --2- 172.21.15.119:0/1996994958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b6407ce50 0x7f0b6407b2b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0b600098a0 tx=0x7f0b60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526a992e5f346e14 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:53.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:52.999+0000 7f0b68da7700 1 -- 172.21.15.119:0/1996994958 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b60004160 con 0x7f0b6407ce50 2023-12-07T10:10:53.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:52.999+0000 7f0b68da7700 1 -- 172.21.15.119:0/1996994958 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b600042c0 con 0x7f0b6407ce50 2023-12-07T10:10:53.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:52.999+0000 7f0b68da7700 1 -- 172.21.15.119:0/1996994958 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b600119b0 con 0x7f0b6407ce50 2023-12-07T10:10:53.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:52.999+0000 7f0b6c00d700 1 -- 172.21.15.119:0/1996994958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b6407ce50 msgr2=0x7f0b6407b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:53.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:52.999+0000 7f0b6c00d700 1 --2- 172.21.15.119:0/1996994958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b6407ce50 0x7f0b6407b2b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0b600098a0 tx=0x7f0b60009bb0 comp rx=0 tx=0).stop 2023-12-07T10:10:53.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.000+0000 7f0b6c00d700 1 -- 172.21.15.119:0/1996994958 shutdown_connections 2023-12-07T10:10:53.727 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.000+0000 7f0b6c00d700 1 --2- 172.21.15.119:0/1996994958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b6407ce50 0x7f0b6407b2b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:53.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.000+0000 7f0b6c00d700 1 -- 172.21.15.119:0/1996994958 >> 172.21.15.119:0/1996994958 conn(0x7f0b64102220 msgr2=0x7f0b64104640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:53.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.000+0000 7f0b6c00d700 1 -- 172.21.15.119:0/1996994958 shutdown_connections 2023-12-07T10:10:53.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.000+0000 7f0b6c00d700 1 -- 172.21.15.119:0/1996994958 wait complete. 2023-12-07T10:10:53.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.001+0000 7f0b6c00d700 1 Processor -- start 2023-12-07T10:10:53.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.001+0000 7f0b6c00d700 1 -- start start 2023-12-07T10:10:53.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.001+0000 7f0b6c00d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b6407ce50 0x7f0b6419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:53.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.001+0000 7f0b6c00d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6419d0f0 con 0x7f0b6407ce50 2023-12-07T10:10:53.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.001+0000 7f0b69da9700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b6407ce50 0x7f0b6419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:10:53.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.001+0000 7f0b69da9700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b6407ce50 0x7f0b6419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36094/0 (socket says 172.21.15.119:36094) 2023-12-07T10:10:53.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.001+0000 7f0b69da9700 1 -- 172.21.15.119:0/322629799 learned_addr learned my addr 172.21.15.119:0/322629799 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:10:53.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.002+0000 7f0b69da9700 1 -- 172.21.15.119:0/322629799 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b60009550 con 0x7f0b6407ce50 2023-12-07T10:10:53.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.002+0000 7f0b69da9700 1 --2- 172.21.15.119:0/322629799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b6407ce50 0x7f0b6419cbb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0b60011670 tx=0x7f0b60011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:10:53.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.002+0000 7f0b5affd700 1 -- 172.21.15.119:0/322629799 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b60003b20 con 0x7f0b6407ce50 2023-12-07T10:10:53.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.002+0000 7f0b5affd700 1 -- 172.21.15.119:0/322629799 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b60003c80 con 0x7f0b6407ce50 2023-12-07T10:10:53.728 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.002+0000 7f0b5affd700 1 -- 172.21.15.119:0/322629799 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b60019770 con 0x7f0b6407ce50 2023-12-07T10:10:53.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.002+0000 7f0b6c00d700 1 -- 172.21.15.119:0/322629799 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b6419d2f0 con 0x7f0b6407ce50 2023-12-07T10:10:53.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.002+0000 7f0b6c00d700 1 -- 172.21.15.119:0/322629799 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b6419d790 con 0x7f0b6407ce50 2023-12-07T10:10:53.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.004+0000 7f0b6c00d700 1 -- 172.21.15.119:0/322629799 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b6410d060 con 0x7f0b6407ce50 2023-12-07T10:10:53.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.004+0000 7f0b5affd700 1 -- 172.21.15.119:0/322629799 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 9) v1 ==== 45752+0+0 (secure 0 0 0) 0x7f0b600199e0 con 0x7f0b6407ce50 2023-12-07T10:10:53.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.004+0000 7f0b5affd700 1 --2- 172.21.15.119:0/322629799 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f0b50038af0 0x7f0b5003afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:10:53.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.004+0000 7f0b5affd700 1 -- 172.21.15.119:0/322629799 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0b6004cfb0 con 0x7f0b6407ce50 2023-12-07T10:10:53.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.004+0000 7f0b695a8700 1 -- 172.21.15.119:0/322629799 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f0b50038af0 msgr2=0x7f0b5003afb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1355617847 2023-12-07T10:10:53.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.004+0000 7f0b695a8700 1 --2- 172.21.15.119:0/322629799 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f0b50038af0 0x7f0b5003afb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-07T10:10:53.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.009+0000 7f0b5affd700 1 -- 172.21.15.119:0/322629799 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0b60018350 con 0x7f0b6407ce50 2023-12-07T10:10:53.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.205+0000 7f0b695a8700 1 -- 172.21.15.119:0/322629799 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f0b50038af0 msgr2=0x7f0b5003afb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1355617847 2023-12-07T10:10:53.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.205+0000 7f0b695a8700 1 --2- 172.21.15.119:0/322629799 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f0b50038af0 0x7f0b5003afb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-07T10:10:53.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.270+0000 7f0b6c00d700 1 -- 172.21.15.119:0/322629799 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f0b641a0080 con 0x7f0b6407ce50 2023-12-07T10:10:53.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.271+0000 7f0b5affd700 1 -- 172.21.15.119:0/322629799 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f0b60018350 con 0x7f0b6407ce50 2023-12-07T10:10:53.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.273+0000 7f0b6c00d700 1 -- 172.21.15.119:0/322629799 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f0b50038af0 msgr2=0x7f0b5003afb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-07T10:10:53.729 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.273+0000 7f0b6c00d700 1 --2- 172.21.15.119:0/322629799 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f0b50038af0 0x7f0b5003afb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:53.730 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.273+0000 7f0b6c00d700 1 -- 172.21.15.119:0/322629799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b6407ce50 msgr2=0x7f0b6419cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:10:53.730 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.273+0000 7f0b6c00d700 1 --2- 172.21.15.119:0/322629799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b6407ce50 0x7f0b6419cbb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0b60011670 tx=0x7f0b60011a30 comp rx=0 tx=0).stop 2023-12-07T10:10:53.730 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.273+0000 7f0b6c00d700 1 -- 172.21.15.119:0/322629799 shutdown_connections 2023-12-07T10:10:53.730 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.273+0000 7f0b6c00d700 1 --2- 172.21.15.119:0/322629799 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7f0b50038af0 0x7f0b5003afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:53.730 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.273+0000 7f0b6c00d700 1 --2- 172.21.15.119:0/322629799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b6407ce50 0x7f0b6419cbb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:10:53.730 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.273+0000 7f0b6c00d700 1 -- 172.21.15.119:0/322629799 >> 172.21.15.119:0/322629799 conn(0x7f0b64102220 msgr2=0x7f0b64102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:10:53.730 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.274+0000 7f0b6c00d700 1 -- 172.21.15.119:0/322629799 shutdown_connections 2023-12-07T10:10:53.730 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:53.274+0000 7f0b6c00d700 1 -- 172.21.15.119:0/322629799 wait complete. 2023-12-07T10:10:53.730 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for the mgr to restart... 2023-12-07T10:10:53.730 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for mgr epoch 9... 2023-12-07T10:10:53.980 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:53 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:53.766+0000 7f8af4ba5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-07T10:10:54.340 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:10:53 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/322629799' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-12-07T10:10:54.730 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:54 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:54.337+0000 7f8af4ba5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-07T10:10:55.480 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:55 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:55.184+0000 7f8af4ba5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-07T10:10:55.480 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:55 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:55.290+0000 7f8af4ba5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-07T10:10:56.586 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:56 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:56.241+0000 7f8af4ba5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-07T10:10:56.586 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:56 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:56.359+0000 7f8af4ba5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-07T10:10:56.980 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:56 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:56.582+0000 7f8af4ba5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-07T10:10:57.416 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:57 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:57.105+0000 7f8af4ba5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-07T10:10:57.417 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:57 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:57.222+0000 7f8af4ba5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-07T10:10:57.730 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:57 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:57.413+0000 7f8af4ba5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-07T10:10:58.230 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:57 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:57.754+0000 7f8af4ba5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-07T10:10:58.980 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:58 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:58.676+0000 7f8af4ba5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-07T10:10:58.980 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:58 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:58.849+0000 7f8af4ba5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-07T10:10:59.480 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:59 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:59.100+0000 7f8af4ba5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-07T10:10:59.480 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:59 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:59.246+0000 7f8af4ba5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-07T10:10:59.980 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:10:59 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:10:59.522+0000 7f8af4ba5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-07T10:11:00.980 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:11:00 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:11:00.582+0000 7f8af4ba5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-07T10:11:01.627 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:01 smithi119 ceph-mon[97908]: Active manager daemon x restarted 2023-12-07T10:11:01.628 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:01 smithi119 ceph-mon[97908]: Activating manager daemon x 2023-12-07T10:11:01.628 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:01 smithi119 ceph-mon[97908]: osdmap e3: 0 total, 0 up, 0 in 2023-12-07T10:11:01.628 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:01 smithi119 ceph-mon[97908]: mgrmap e10: x(active, starting, since 0.00592628s) 2023-12-07T10:11:01.628 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T10:11:01.628 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-07T10:11:01.628 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-07T10:11:01.628 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-07T10:11:01.628 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-07T10:11:01.628 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:01 smithi119 ceph-mon[97908]: Manager daemon x is now available 2023-12-07T10:11:01.628 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:11:01.628 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:11:01.628 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-07T10:11:01.628 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-07T10:11:01.628 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:01 smithi119 ceph-mon[97908]: [07/Dec/2023:10:11:00] ENGINE Bus STARTING 2023-12-07T10:11:01.629 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:01 smithi119 ceph-mon[97908]: [07/Dec/2023:10:11:00] ENGINE Serving on http://172.21.15.119:8765 2023-12-07T10:11:01.629 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:01 smithi119 ceph-mon[97908]: [07/Dec/2023:10:11:01] ENGINE Serving on https://172.21.15.119:7150 2023-12-07T10:11:01.629 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:01 smithi119 ceph-mon[97908]: [07/Dec/2023:10:11:01] ENGINE Bus STARTED 2023-12-07T10:11:02.130 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2023-12-07T10:11:02.130 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-12-07T10:11:02.130 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "initialized": true 2023-12-07T10:11:02.131 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2023-12-07T10:11:02.131 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.274+0000 7ff8ccf25700 1 Processor -- start 2023-12-07T10:11:02.131 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.275+0000 7ff8ccf25700 1 -- start start 2023-12-07T10:11:02.131 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.275+0000 7ff8ccf25700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8c8106c60 0x7ff8c8107080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:02.131 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.275+0000 7ff8ccf25700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c8107650 con 0x7ff8c8106c60 2023-12-07T10:11:02.131 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.275+0000 7ff8c659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8c8106c60 0x7ff8c8107080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:02.131 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.275+0000 7ff8c659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8c8106c60 0x7ff8c8107080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36110/0 (socket says 172.21.15.119:36110) 2023-12-07T10:11:02.131 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.275+0000 7ff8c659c700 1 -- 172.21.15.119:0/2947556675 learned_addr learned my addr 172.21.15.119:0/2947556675 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:11:02.131 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.276+0000 7ff8c659c700 1 -- 172.21.15.119:0/2947556675 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c8107e60 con 0x7ff8c8106c60 2023-12-07T10:11:02.131 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.276+0000 7ff8c659c700 1 --2- 172.21.15.119:0/2947556675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8c8106c60 0x7ff8c8107080 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80098a0 tx=0x7ff8b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ff1f5992cc3263 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:02.131 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.276+0000 7ff8c559a700 1 -- 172.21.15.119:0/2947556675 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8b8004160 con 0x7ff8c8106c60 2023-12-07T10:11:02.131 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.276+0000 7ff8c559a700 1 -- 172.21.15.119:0/2947556675 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8b80042c0 con 0x7ff8c8106c60 2023-12-07T10:11:02.131 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.276+0000 7ff8c559a700 1 -- 172.21.15.119:0/2947556675 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8b80119b0 con 0x7ff8c8106c60 2023-12-07T10:11:02.132 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.277+0000 7ff8ccf25700 1 -- 172.21.15.119:0/2947556675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8c8106c60 msgr2=0x7ff8c8107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:02.132 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.277+0000 7ff8ccf25700 1 --2- 172.21.15.119:0/2947556675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8c8106c60 0x7ff8c8107080 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80098a0 tx=0x7ff8b8009bb0 comp rx=0 tx=0).stop 2023-12-07T10:11:02.132 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.277+0000 7ff8ccf25700 1 -- 172.21.15.119:0/2947556675 shutdown_connections 2023-12-07T10:11:02.132 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.277+0000 7ff8ccf25700 1 --2- 172.21.15.119:0/2947556675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8c8106c60 0x7ff8c8107080 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:02.132 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.277+0000 7ff8ccf25700 1 -- 172.21.15.119:0/2947556675 >> 172.21.15.119:0/2947556675 conn(0x7ff8c8102220 msgr2=0x7ff8c8104640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:02.132 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.277+0000 7ff8ccf25700 1 -- 172.21.15.119:0/2947556675 shutdown_connections 2023-12-07T10:11:02.132 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.277+0000 7ff8ccf25700 1 -- 172.21.15.119:0/2947556675 wait complete. 2023-12-07T10:11:02.132 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.278+0000 7ff8ccf25700 1 Processor -- start 2023-12-07T10:11:02.132 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.278+0000 7ff8ccf25700 1 -- start start 2023-12-07T10:11:02.132 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.279+0000 7ff8ccf25700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8c8106c60 0x7ff8c807cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:02.132 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.279+0000 7ff8ccf25700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c807d050 con 0x7ff8c8106c60 2023-12-07T10:11:02.132 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.279+0000 7ff8c659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8c8106c60 0x7ff8c807cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:02.132 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.279+0000 7ff8c659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8c8106c60 0x7ff8c807cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36118/0 (socket says 172.21.15.119:36118) 2023-12-07T10:11:02.132 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.279+0000 7ff8c659c700 1 -- 172.21.15.119:0/3939945030 learned_addr learned my addr 172.21.15.119:0/3939945030 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:11:02.133 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.279+0000 7ff8c659c700 1 -- 172.21.15.119:0/3939945030 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b8009550 con 0x7ff8c8106c60 2023-12-07T10:11:02.133 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.279+0000 7ff8c659c700 1 --2- 172.21.15.119:0/3939945030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8c8106c60 0x7ff8c807cb10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8003e60 tx=0x7ff8b8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:02.133 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.280+0000 7ff8b77fe700 1 -- 172.21.15.119:0/3939945030 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8b8003a60 con 0x7ff8c8106c60 2023-12-07T10:11:02.133 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.280+0000 7ff8b77fe700 1 -- 172.21.15.119:0/3939945030 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8b8003bc0 con 0x7ff8c8106c60 2023-12-07T10:11:02.133 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.280+0000 7ff8ccf25700 1 -- 172.21.15.119:0/3939945030 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c807b290 con 0x7ff8c8106c60 2023-12-07T10:11:02.133 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.280+0000 7ff8b77fe700 1 -- 172.21.15.119:0/3939945030 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8b80196c0 con 0x7ff8c8106c60 2023-12-07T10:11:02.133 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.280+0000 7ff8ccf25700 1 -- 172.21.15.119:0/3939945030 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c807b730 con 0x7ff8c8106c60 2023-12-07T10:11:02.133 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.281+0000 7ff8b77fe700 1 -- 172.21.15.119:0/3939945030 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 9) v1 ==== 45752+0+0 (secure 0 0 0) 0x7ff8b801ab60 con 0x7ff8c8106c60 2023-12-07T10:11:02.133 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.281+0000 7ff8b77fe700 1 --2- 172.21.15.119:0/3939945030 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7ff8b0038ad0 0x7ff8b003af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:02.133 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.281+0000 7ff8b77fe700 1 -- 172.21.15.119:0/3939945030 --> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8b003b6a0 con 0x7ff8b0038ad0 2023-12-07T10:11:02.133 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.281+0000 7ff8b77fe700 1 -- 172.21.15.119:0/3939945030 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff8b804ce60 con 0x7ff8c8106c60 2023-12-07T10:11:02.133 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.281+0000 7ff8c5d9b700 1 -- 172.21.15.119:0/3939945030 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7ff8b0038ad0 msgr2=0x7ff8b003af90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1355617847 2023-12-07T10:11:02.133 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.282+0000 7ff8c5d9b700 1 --2- 172.21.15.119:0/3939945030 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7ff8b0038ad0 0x7ff8b003af90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-07T10:11:02.133 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.483+0000 7ff8c5d9b700 1 -- 172.21.15.119:0/3939945030 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7ff8b0038ad0 msgr2=0x7ff8b003af90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1355617847 2023-12-07T10:11:02.134 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.483+0000 7ff8c5d9b700 1 --2- 172.21.15.119:0/3939945030 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7ff8b0038ad0 0x7ff8b003af90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-07T10:11:02.134 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.884+0000 7ff8c5d9b700 1 -- 172.21.15.119:0/3939945030 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7ff8b0038ad0 msgr2=0x7ff8b003af90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1355617847 2023-12-07T10:11:02.134 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:54.884+0000 7ff8c5d9b700 1 --2- 172.21.15.119:0/3939945030 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7ff8b0038ad0 0x7ff8b003af90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-07T10:11:02.134 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:55.685+0000 7ff8c5d9b700 1 -- 172.21.15.119:0/3939945030 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7ff8b0038ad0 msgr2=0x7ff8b003af90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1355617847 2023-12-07T10:11:02.134 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:55.685+0000 7ff8c5d9b700 1 --2- 172.21.15.119:0/3939945030 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7ff8b0038ad0 0x7ff8b003af90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-07T10:11:02.134 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:57.287+0000 7ff8c5d9b700 1 -- 172.21.15.119:0/3939945030 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7ff8b0038ad0 msgr2=0x7ff8b003af90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1355617847 2023-12-07T10:11:02.134 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:10:57.287+0000 7ff8c5d9b700 1 --2- 172.21.15.119:0/3939945030 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7ff8b0038ad0 0x7ff8b003af90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-07T10:11:02.134 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:00.489+0000 7ff8c5d9b700 1 -- 172.21.15.119:0/3939945030 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7ff8b0038ad0 msgr2=0x7ff8b003af90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1355617847 2023-12-07T10:11:02.134 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:00.489+0000 7ff8c5d9b700 1 --2- 172.21.15.119:0/3939945030 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7ff8b0038ad0 0x7ff8b003af90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-07T10:11:02.134 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:00.588+0000 7ff8b77fe700 1 -- 172.21.15.119:0/3939945030 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mgrmap(e 10) v1 ==== 45519+0+0 (secure 0 0 0) 0x7ff8b8019b50 con 0x7ff8c8106c60 2023-12-07T10:11:02.134 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:00.588+0000 7ff8b77fe700 1 -- 172.21.15.119:0/3939945030 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7ff8b0038ad0 msgr2=0x7ff8b003af90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-07T10:11:02.134 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:00.588+0000 7ff8b77fe700 1 --2- 172.21.15.119:0/3939945030 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7ff8b0038ad0 0x7ff8b003af90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:02.134 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:01.590+0000 7ff8b77fe700 1 -- 172.21.15.119:0/3939945030 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 11) v1 ==== 45646+0+0 (secure 0 0 0) 0x7ff8b804e6e0 con 0x7ff8c8106c60 2023-12-07T10:11:02.134 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:01.591+0000 7ff8b77fe700 1 --2- 172.21.15.119:0/3939945030 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff8b003c540 0x7ff8b003e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:02.135 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:01.591+0000 7ff8b77fe700 1 -- 172.21.15.119:0/3939945030 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8b003b6a0 con 0x7ff8b003c540 2023-12-07T10:11:02.135 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:01.591+0000 7ff8c5d9b700 1 --2- 172.21.15.119:0/3939945030 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff8b003c540 0x7ff8b003e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:02.135 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:01.592+0000 7ff8c5d9b700 1 --2- 172.21.15.119:0/3939945030 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff8b003c540 0x7ff8b003e930 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc003a10 tx=0x7ff8bc009270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:02.135 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:01.593+0000 7ff8b77fe700 1 -- 172.21.15.119:0/3939945030 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7ff8b003b6a0 con 0x7ff8b003c540 2023-12-07T10:11:02.135 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:01.600+0000 7ff8ccf25700 1 -- 172.21.15.119:0/3939945030 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7ff8c8062c40 con 0x7ff8b003c540 2023-12-07T10:11:02.135 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:01.600+0000 7ff8b77fe700 1 -- 172.21.15.119:0/3939945030 <== mgr.14150 v2:172.21.15.119:6800/1344846367 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7ff8c8062c40 con 0x7ff8b003c540 2023-12-07T10:11:02.135 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:01.601+0000 7ff8ccf25700 1 -- 172.21.15.119:0/3939945030 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff8b003c540 msgr2=0x7ff8b003e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:02.135 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:01.601+0000 7ff8ccf25700 1 --2- 172.21.15.119:0/3939945030 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff8b003c540 0x7ff8b003e930 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc003a10 tx=0x7ff8bc009270 comp rx=0 tx=0).stop 2023-12-07T10:11:02.135 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:01.601+0000 7ff8ccf25700 1 -- 172.21.15.119:0/3939945030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8c8106c60 msgr2=0x7ff8c807cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:02.135 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:01.601+0000 7ff8ccf25700 1 --2- 172.21.15.119:0/3939945030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8c8106c60 0x7ff8c807cb10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8003e60 tx=0x7ff8b8011a30 comp rx=0 tx=0).stop 2023-12-07T10:11:02.135 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:01.601+0000 7ff8ccf25700 1 -- 172.21.15.119:0/3939945030 shutdown_connections 2023-12-07T10:11:02.135 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:01.601+0000 7ff8ccf25700 1 --2- 172.21.15.119:0/3939945030 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff8b003c540 0x7ff8b003e930 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:02.135 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:01.601+0000 7ff8ccf25700 1 --2- 172.21.15.119:0/3939945030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8c8106c60 0x7ff8c807cb10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:02.136 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:01.601+0000 7ff8ccf25700 1 --2- 172.21.15.119:0/3939945030 >> [v2:172.21.15.119:6800/1355617847,v1:172.21.15.119:6801/1355617847] conn(0x7ff8b0038ad0 0x7ff8b003af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:02.136 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:01.601+0000 7ff8ccf25700 1 -- 172.21.15.119:0/3939945030 >> 172.21.15.119:0/3939945030 conn(0x7ff8c8102220 msgr2=0x7ff8c8102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:02.136 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:01.601+0000 7ff8ccf25700 1 -- 172.21.15.119:0/3939945030 shutdown_connections 2023-12-07T10:11:02.136 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:01.601+0000 7ff8ccf25700 1 -- 172.21.15.119:0/3939945030 wait complete. 2023-12-07T10:11:02.136 INFO:teuthology.orchestra.run.smithi119.stdout:mgr epoch 9 is available 2023-12-07T10:11:02.136 INFO:teuthology.orchestra.run.smithi119.stdout:Generating a dashboard self-signed certificate... 2023-12-07T10:11:02.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:02 smithi119 ceph-mon[97908]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-12-07T10:11:02.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:02 smithi119 ceph-mon[97908]: mgrmap e11: x(active, since 1.00947s) 2023-12-07T10:11:02.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:02 smithi119 ceph-mon[97908]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-12-07T10:11:03.712 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-12-07T10:11:03.712 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.870+0000 7fc61c37d700 1 Processor -- start 2023-12-07T10:11:03.712 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.871+0000 7fc61c37d700 1 -- start start 2023-12-07T10:11:03.712 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.871+0000 7fc61c37d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc614108e70 0x7fc614109290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:03.712 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.871+0000 7fc61c37d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6141097d0 con 0x7fc614108e70 2023-12-07T10:11:03.712 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.871+0000 7fc61a119700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc614108e70 0x7fc614109290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:03.712 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.872+0000 7fc61a119700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc614108e70 0x7fc614109290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37406/0 (socket says 172.21.15.119:37406) 2023-12-07T10:11:03.712 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.872+0000 7fc61a119700 1 -- 172.21.15.119:0/202828435 learned_addr learned my addr 172.21.15.119:0/202828435 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:11:03.712 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.872+0000 7fc61a119700 1 -- 172.21.15.119:0/202828435 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc614109910 con 0x7fc614108e70 2023-12-07T10:11:03.713 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.872+0000 7fc61a119700 1 --2- 172.21.15.119:0/202828435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc614108e70 0x7fc614109290 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc6100098a0 tx=0x7fc610009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c85bba1ade73b4 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:03.713 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.872+0000 7fc619117700 1 -- 172.21.15.119:0/202828435 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc610004160 con 0x7fc614108e70 2023-12-07T10:11:03.713 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.872+0000 7fc619117700 1 -- 172.21.15.119:0/202828435 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6100042c0 con 0x7fc614108e70 2023-12-07T10:11:03.713 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.872+0000 7fc619117700 1 -- 172.21.15.119:0/202828435 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6100119b0 con 0x7fc614108e70 2023-12-07T10:11:03.713 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.873+0000 7fc61c37d700 1 -- 172.21.15.119:0/202828435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc614108e70 msgr2=0x7fc614109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:03.713 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.873+0000 7fc61c37d700 1 --2- 172.21.15.119:0/202828435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc614108e70 0x7fc614109290 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc6100098a0 tx=0x7fc610009bb0 comp rx=0 tx=0).stop 2023-12-07T10:11:03.713 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.873+0000 7fc61c37d700 1 -- 172.21.15.119:0/202828435 shutdown_connections 2023-12-07T10:11:03.713 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.873+0000 7fc61c37d700 1 --2- 172.21.15.119:0/202828435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc614108e70 0x7fc614109290 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:03.713 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.873+0000 7fc61c37d700 1 -- 172.21.15.119:0/202828435 >> 172.21.15.119:0/202828435 conn(0x7fc61407c240 msgr2=0x7fc61407c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:03.713 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.873+0000 7fc61c37d700 1 -- 172.21.15.119:0/202828435 shutdown_connections 2023-12-07T10:11:03.713 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.873+0000 7fc61c37d700 1 -- 172.21.15.119:0/202828435 wait complete. 2023-12-07T10:11:03.713 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.874+0000 7fc61c37d700 1 Processor -- start 2023-12-07T10:11:03.713 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.874+0000 7fc61c37d700 1 -- start start 2023-12-07T10:11:03.713 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.874+0000 7fc61c37d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc614108e70 0x7fc61419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:03.713 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.874+0000 7fc61c37d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61419d100 con 0x7fc614108e70 2023-12-07T10:11:03.714 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.875+0000 7fc61a119700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc614108e70 0x7fc61419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:03.714 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.875+0000 7fc61a119700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc614108e70 0x7fc61419cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37422/0 (socket says 172.21.15.119:37422) 2023-12-07T10:11:03.714 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.875+0000 7fc61a119700 1 -- 172.21.15.119:0/2626123467 learned_addr learned my addr 172.21.15.119:0/2626123467 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:11:03.714 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.875+0000 7fc61a119700 1 -- 172.21.15.119:0/2626123467 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc610009550 con 0x7fc614108e70 2023-12-07T10:11:03.714 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.875+0000 7fc61a119700 1 --2- 172.21.15.119:0/2626123467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc614108e70 0x7fc61419cbc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc610006af0 tx=0x7fc610011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:03.714 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.876+0000 7fc60b7fe700 1 -- 172.21.15.119:0/2626123467 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc610003a40 con 0x7fc614108e70 2023-12-07T10:11:03.714 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.876+0000 7fc60b7fe700 1 -- 172.21.15.119:0/2626123467 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc610003ba0 con 0x7fc614108e70 2023-12-07T10:11:03.714 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.876+0000 7fc61c37d700 1 -- 172.21.15.119:0/2626123467 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc61419d300 con 0x7fc614108e70 2023-12-07T10:11:03.714 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.876+0000 7fc60b7fe700 1 -- 172.21.15.119:0/2626123467 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6100196c0 con 0x7fc614108e70 2023-12-07T10:11:03.714 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.876+0000 7fc61c37d700 1 -- 172.21.15.119:0/2626123467 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc61419d7a0 con 0x7fc614108e70 2023-12-07T10:11:03.714 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.877+0000 7fc61c37d700 1 -- 172.21.15.119:0/2626123467 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc61410ac90 con 0x7fc614108e70 2023-12-07T10:11:03.714 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.877+0000 7fc60b7fe700 1 -- 172.21.15.119:0/2626123467 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 12) v1 ==== 45752+0+0 (secure 0 0 0) 0x7fc61001aa80 con 0x7fc614108e70 2023-12-07T10:11:03.714 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.878+0000 7fc60b7fe700 1 --2- 172.21.15.119:0/2626123467 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc600038aa0 0x7fc60003af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:03.714 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.878+0000 7fc60b7fe700 1 -- 172.21.15.119:0/2626123467 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc61004cfa0 con 0x7fc614108e70 2023-12-07T10:11:03.715 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.878+0000 7fc619918700 1 --2- 172.21.15.119:0/2626123467 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc600038aa0 0x7fc60003af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:03.715 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.878+0000 7fc619918700 1 --2- 172.21.15.119:0/2626123467 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc600038aa0 0x7fc60003af60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc604006fd0 tx=0x7fc604006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:03.715 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:02.886+0000 7fc60b7fe700 1 -- 172.21.15.119:0/2626123467 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc61001a430 con 0x7fc614108e70 2023-12-07T10:11:03.715 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:03.114+0000 7fc61c37d700 1 -- 172.21.15.119:0/2626123467 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fc614106c50 con 0x7fc600038aa0 2023-12-07T10:11:03.715 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:03.189+0000 7fc60b7fe700 1 -- 172.21.15.119:0/2626123467 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fc614106c50 con 0x7fc600038aa0 2023-12-07T10:11:03.715 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:03.192+0000 7fc61c37d700 1 -- 172.21.15.119:0/2626123467 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc600038aa0 msgr2=0x7fc60003af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:03.715 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:03.192+0000 7fc61c37d700 1 --2- 172.21.15.119:0/2626123467 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc600038aa0 0x7fc60003af60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc604006fd0 tx=0x7fc604006e40 comp rx=0 tx=0).stop 2023-12-07T10:11:03.715 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:03.192+0000 7fc61c37d700 1 -- 172.21.15.119:0/2626123467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc614108e70 msgr2=0x7fc61419cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:03.715 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:03.192+0000 7fc61c37d700 1 --2- 172.21.15.119:0/2626123467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc614108e70 0x7fc61419cbc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc610006af0 tx=0x7fc610011a10 comp rx=0 tx=0).stop 2023-12-07T10:11:03.715 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:03.192+0000 7fc61c37d700 1 -- 172.21.15.119:0/2626123467 shutdown_connections 2023-12-07T10:11:03.715 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:03.192+0000 7fc61c37d700 1 --2- 172.21.15.119:0/2626123467 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc600038aa0 0x7fc60003af60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:03.715 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:03.192+0000 7fc61c37d700 1 --2- 172.21.15.119:0/2626123467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc614108e70 0x7fc61419cbc0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:03.715 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:03.192+0000 7fc61c37d700 1 -- 172.21.15.119:0/2626123467 >> 172.21.15.119:0/2626123467 conn(0x7fc61407c240 msgr2=0x7fc614106540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:03.715 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:03.192+0000 7fc61c37d700 1 -- 172.21.15.119:0/2626123467 shutdown_connections 2023-12-07T10:11:03.716 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:03.193+0000 7fc61c37d700 1 -- 172.21.15.119:0/2626123467 wait complete. 2023-12-07T10:11:03.716 INFO:teuthology.orchestra.run.smithi119.stdout:Creating initial admin user... 2023-12-07T10:11:03.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:03 smithi119 ceph-mon[97908]: mgrmap e12: x(active, since 2s) 2023-12-07T10:11:03.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:03 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:03.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:03 smithi119 ceph-mon[97908]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:11:03.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:03 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:03.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:03 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:05.203 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$tDr.X7D.eypYzUMIMRGO9uoFBTNtnnX3Kcvx.hJ4F/DXyIo7CLr.2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1701943864, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-12-07T10:11:05.203 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.289+0000 7fac28a8a700 1 Processor -- start 2023-12-07T10:11:05.203 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.289+0000 7fac28a8a700 1 -- start start 2023-12-07T10:11:05.203 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.290+0000 7fac28a8a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac2407ce70 0x7fac2407b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:05.203 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.290+0000 7fac28a8a700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2407b8a0 con 0x7fac2407ce70 2023-12-07T10:11:05.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.290+0000 7fac2259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac2407ce70 0x7fac2407b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:05.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.290+0000 7fac2259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac2407ce70 0x7fac2407b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37426/0 (socket says 172.21.15.119:37426) 2023-12-07T10:11:05.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.290+0000 7fac2259c700 1 -- 172.21.15.119:0/2580956769 learned_addr learned my addr 172.21.15.119:0/2580956769 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:11:05.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.290+0000 7fac2259c700 1 -- 172.21.15.119:0/2580956769 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac2407b9e0 con 0x7fac2407ce70 2023-12-07T10:11:05.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.291+0000 7fac2259c700 1 --2- 172.21.15.119:0/2580956769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac2407ce70 0x7fac2407b2d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fac140098a0 tx=0x7fac14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d739d803d045ea6a server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:05.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.291+0000 7fac2159a700 1 -- 172.21.15.119:0/2580956769 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac14004160 con 0x7fac2407ce70 2023-12-07T10:11:05.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.291+0000 7fac2159a700 1 -- 172.21.15.119:0/2580956769 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac140042c0 con 0x7fac2407ce70 2023-12-07T10:11:05.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.291+0000 7fac2159a700 1 -- 172.21.15.119:0/2580956769 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac140119b0 con 0x7fac2407ce70 2023-12-07T10:11:05.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.291+0000 7fac28a8a700 1 -- 172.21.15.119:0/2580956769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac2407ce70 msgr2=0x7fac2407b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:05.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.291+0000 7fac28a8a700 1 --2- 172.21.15.119:0/2580956769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac2407ce70 0x7fac2407b2d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fac140098a0 tx=0x7fac14009bb0 comp rx=0 tx=0).stop 2023-12-07T10:11:05.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.292+0000 7fac28a8a700 1 -- 172.21.15.119:0/2580956769 shutdown_connections 2023-12-07T10:11:05.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.292+0000 7fac28a8a700 1 --2- 172.21.15.119:0/2580956769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac2407ce70 0x7fac2407b2d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:05.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.292+0000 7fac28a8a700 1 -- 172.21.15.119:0/2580956769 >> 172.21.15.119:0/2580956769 conn(0x7fac24102220 msgr2=0x7fac24104640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:05.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.292+0000 7fac28a8a700 1 -- 172.21.15.119:0/2580956769 shutdown_connections 2023-12-07T10:11:05.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.292+0000 7fac28a8a700 1 -- 172.21.15.119:0/2580956769 wait complete. 2023-12-07T10:11:05.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.293+0000 7fac28a8a700 1 Processor -- start 2023-12-07T10:11:05.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.293+0000 7fac28a8a700 1 -- start start 2023-12-07T10:11:05.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.293+0000 7fac28a8a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac2407ce70 0x7fac2419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:05.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.293+0000 7fac28a8a700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2419d0f0 con 0x7fac2407ce70 2023-12-07T10:11:05.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.293+0000 7fac2259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac2407ce70 0x7fac2419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:05.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.294+0000 7fac2259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac2407ce70 0x7fac2419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37430/0 (socket says 172.21.15.119:37430) 2023-12-07T10:11:05.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.294+0000 7fac2259c700 1 -- 172.21.15.119:0/4127488902 learned_addr learned my addr 172.21.15.119:0/4127488902 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:11:05.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.294+0000 7fac2259c700 1 -- 172.21.15.119:0/4127488902 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac14009550 con 0x7fac2407ce70 2023-12-07T10:11:05.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.294+0000 7fac2259c700 1 --2- 172.21.15.119:0/4127488902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac2407ce70 0x7fac2419cbb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fac14011670 tx=0x7fac14011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:05.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.294+0000 7fac137fe700 1 -- 172.21.15.119:0/4127488902 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac14003b20 con 0x7fac2407ce70 2023-12-07T10:11:05.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.294+0000 7fac137fe700 1 -- 172.21.15.119:0/4127488902 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac14003c80 con 0x7fac2407ce70 2023-12-07T10:11:05.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.294+0000 7fac137fe700 1 -- 172.21.15.119:0/4127488902 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac14019770 con 0x7fac2407ce70 2023-12-07T10:11:05.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.294+0000 7fac28a8a700 1 -- 172.21.15.119:0/4127488902 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac2419d2f0 con 0x7fac2407ce70 2023-12-07T10:11:05.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.295+0000 7fac28a8a700 1 -- 172.21.15.119:0/4127488902 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac2419d790 con 0x7fac2407ce70 2023-12-07T10:11:05.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.296+0000 7fac137fe700 1 -- 172.21.15.119:0/4127488902 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 12) v1 ==== 45752+0+0 (secure 0 0 0) 0x7fac1401ab60 con 0x7fac2407ce70 2023-12-07T10:11:05.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.296+0000 7fac137fe700 1 --2- 172.21.15.119:0/4127488902 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fac0c038aa0 0x7fac0c03af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:05.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.296+0000 7fac137fe700 1 -- 172.21.15.119:0/4127488902 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fac1404cfe0 con 0x7fac2407ce70 2023-12-07T10:11:05.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.296+0000 7fac28a8a700 1 -- 172.21.15.119:0/4127488902 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac2410d0a0 con 0x7fac2407ce70 2023-12-07T10:11:05.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.296+0000 7fac21d9b700 1 --2- 172.21.15.119:0/4127488902 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fac0c038aa0 0x7fac0c03af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:05.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.297+0000 7fac21d9b700 1 --2- 172.21.15.119:0/4127488902 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fac0c038aa0 0x7fac0c03af60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fac18006fd0 tx=0x7fac18006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:05.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.301+0000 7fac137fe700 1 -- 172.21.15.119:0/4127488902 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fac1401a430 con 0x7fac2407ce70 2023-12-07T10:11:05.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.535+0000 7fac28a8a700 1 -- 172.21.15.119:0/4127488902 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fac2410d290 con 0x7fac0c038aa0 2023-12-07T10:11:05.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.764+0000 7fac137fe700 1 -- 172.21.15.119:0/4127488902 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fac2410d290 con 0x7fac0c038aa0 2023-12-07T10:11:05.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.767+0000 7fac28a8a700 1 -- 172.21.15.119:0/4127488902 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fac0c038aa0 msgr2=0x7fac0c03af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:05.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.767+0000 7fac28a8a700 1 --2- 172.21.15.119:0/4127488902 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fac0c038aa0 0x7fac0c03af60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fac18006fd0 tx=0x7fac18006e40 comp rx=0 tx=0).stop 2023-12-07T10:11:05.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.767+0000 7fac28a8a700 1 -- 172.21.15.119:0/4127488902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac2407ce70 msgr2=0x7fac2419cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:05.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.767+0000 7fac28a8a700 1 --2- 172.21.15.119:0/4127488902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac2407ce70 0x7fac2419cbb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fac14011670 tx=0x7fac14011a30 comp rx=0 tx=0).stop 2023-12-07T10:11:05.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.767+0000 7fac28a8a700 1 -- 172.21.15.119:0/4127488902 shutdown_connections 2023-12-07T10:11:05.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.767+0000 7fac28a8a700 1 --2- 172.21.15.119:0/4127488902 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fac0c038aa0 0x7fac0c03af60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:05.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.767+0000 7fac28a8a700 1 --2- 172.21.15.119:0/4127488902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac2407ce70 0x7fac2419cbb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:05.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.767+0000 7fac28a8a700 1 -- 172.21.15.119:0/4127488902 >> 172.21.15.119:0/4127488902 conn(0x7fac24102220 msgr2=0x7fac24102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:05.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.767+0000 7fac28a8a700 1 -- 172.21.15.119:0/4127488902 shutdown_connections 2023-12-07T10:11:05.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:04.767+0000 7fac28a8a700 1 -- 172.21.15.119:0/4127488902 wait complete. 2023-12-07T10:11:05.207 INFO:teuthology.orchestra.run.smithi119.stdout:Fetching dashboard port number... 2023-12-07T10:11:05.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:05 smithi119 ceph-mon[97908]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:11:05.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:05 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:05.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:05 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:06.485 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 8443 2023-12-07T10:11:06.485 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.773+0000 7f8e61467700 1 Processor -- start 2023-12-07T10:11:06.485 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.773+0000 7f8e61467700 1 -- start start 2023-12-07T10:11:06.485 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.773+0000 7f8e61467700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e5c106c60 0x7f8e5c107080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:06.485 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.773+0000 7f8e61467700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c107650 con 0x7f8e5c106c60 2023-12-07T10:11:06.485 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.774+0000 7f8e5affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e5c106c60 0x7f8e5c107080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:06.486 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.774+0000 7f8e5affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e5c106c60 0x7f8e5c107080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37432/0 (socket says 172.21.15.119:37432) 2023-12-07T10:11:06.486 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.774+0000 7f8e5affd700 1 -- 172.21.15.119:0/1457306625 learned_addr learned my addr 172.21.15.119:0/1457306625 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:11:06.486 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.774+0000 7f8e5affd700 1 -- 172.21.15.119:0/1457306625 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5c107e60 con 0x7f8e5c106c60 2023-12-07T10:11:06.486 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.774+0000 7f8e5affd700 1 --2- 172.21.15.119:0/1457306625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e5c106c60 0x7f8e5c107080 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c0098a0 tx=0x7f8e4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9abecd8a5fa40d server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:06.486 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.775+0000 7f8e59ffb700 1 -- 172.21.15.119:0/1457306625 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e4c004160 con 0x7f8e5c106c60 2023-12-07T10:11:06.486 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.775+0000 7f8e59ffb700 1 -- 172.21.15.119:0/1457306625 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e4c0042c0 con 0x7f8e5c106c60 2023-12-07T10:11:06.486 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.775+0000 7f8e59ffb700 1 -- 172.21.15.119:0/1457306625 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e4c0119b0 con 0x7f8e5c106c60 2023-12-07T10:11:06.486 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.775+0000 7f8e61467700 1 -- 172.21.15.119:0/1457306625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e5c106c60 msgr2=0x7f8e5c107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:06.486 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.776+0000 7f8e61467700 1 --2- 172.21.15.119:0/1457306625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e5c106c60 0x7f8e5c107080 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c0098a0 tx=0x7f8e4c009bb0 comp rx=0 tx=0).stop 2023-12-07T10:11:06.486 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.776+0000 7f8e61467700 1 -- 172.21.15.119:0/1457306625 shutdown_connections 2023-12-07T10:11:06.486 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.776+0000 7f8e61467700 1 --2- 172.21.15.119:0/1457306625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e5c106c60 0x7f8e5c107080 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:06.486 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.776+0000 7f8e61467700 1 -- 172.21.15.119:0/1457306625 >> 172.21.15.119:0/1457306625 conn(0x7f8e5c102220 msgr2=0x7f8e5c104640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:06.486 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.776+0000 7f8e61467700 1 -- 172.21.15.119:0/1457306625 shutdown_connections 2023-12-07T10:11:06.487 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.776+0000 7f8e61467700 1 -- 172.21.15.119:0/1457306625 wait complete. 2023-12-07T10:11:06.487 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.777+0000 7f8e61467700 1 Processor -- start 2023-12-07T10:11:06.487 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.777+0000 7f8e61467700 1 -- start start 2023-12-07T10:11:06.487 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.777+0000 7f8e61467700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e5c106c60 0x7f8e5c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:06.487 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.777+0000 7f8e61467700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c19d1c0 con 0x7f8e5c106c60 2023-12-07T10:11:06.487 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.778+0000 7f8e5affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e5c106c60 0x7f8e5c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:06.487 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.778+0000 7f8e5affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e5c106c60 0x7f8e5c19cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37434/0 (socket says 172.21.15.119:37434) 2023-12-07T10:11:06.487 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.778+0000 7f8e5affd700 1 -- 172.21.15.119:0/3532030729 learned_addr learned my addr 172.21.15.119:0/3532030729 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:11:06.487 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.778+0000 7f8e5affd700 1 -- 172.21.15.119:0/3532030729 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c009550 con 0x7f8e5c106c60 2023-12-07T10:11:06.487 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.778+0000 7f8e5affd700 1 --2- 172.21.15.119:0/3532030729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e5c106c60 0x7f8e5c19cc80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c003e60 tx=0x7f8e4c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:06.487 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.779+0000 7f8e43fff700 1 -- 172.21.15.119:0/3532030729 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e4c003a60 con 0x7f8e5c106c60 2023-12-07T10:11:06.487 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.779+0000 7f8e43fff700 1 -- 172.21.15.119:0/3532030729 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e4c003bc0 con 0x7f8e5c106c60 2023-12-07T10:11:06.487 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.779+0000 7f8e61467700 1 -- 172.21.15.119:0/3532030729 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e5c19d3c0 con 0x7f8e5c106c60 2023-12-07T10:11:06.487 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.779+0000 7f8e43fff700 1 -- 172.21.15.119:0/3532030729 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e4c0196c0 con 0x7f8e5c106c60 2023-12-07T10:11:06.488 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.779+0000 7f8e61467700 1 -- 172.21.15.119:0/3532030729 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5c19d860 con 0x7f8e5c106c60 2023-12-07T10:11:06.488 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.780+0000 7f8e61467700 1 -- 172.21.15.119:0/3532030729 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e5c10b090 con 0x7f8e5c106c60 2023-12-07T10:11:06.488 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.780+0000 7f8e43fff700 1 -- 172.21.15.119:0/3532030729 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 12) v1 ==== 45752+0+0 (secure 0 0 0) 0x7f8e4c01ab60 con 0x7f8e5c106c60 2023-12-07T10:11:06.488 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.781+0000 7f8e43fff700 1 --2- 172.21.15.119:0/3532030729 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8e44038b20 0x7f8e4403afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:06.488 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.781+0000 7f8e43fff700 1 -- 172.21.15.119:0/3532030729 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8e4c04cf00 con 0x7f8e5c106c60 2023-12-07T10:11:06.488 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.782+0000 7f8e5a7fc700 1 --2- 172.21.15.119:0/3532030729 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8e44038b20 0x7f8e4403afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:06.488 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.784+0000 7f8e5a7fc700 1 --2- 172.21.15.119:0/3532030729 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8e44038b20 0x7f8e4403afe0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8e50006fd0 tx=0x7f8e50006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:06.488 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:05.788+0000 7f8e43fff700 1 -- 172.21.15.119:0/3532030729 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8e4c019930 con 0x7f8e5c106c60 2023-12-07T10:11:06.488 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:06.003+0000 7f8e61467700 1 -- 172.21.15.119:0/3532030729 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f8e5c1a0150 con 0x7f8e5c106c60 2023-12-07T10:11:06.488 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:06.004+0000 7f8e43fff700 1 -- 172.21.15.119:0/3532030729 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f8e4c018350 con 0x7f8e5c106c60 2023-12-07T10:11:06.488 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:06.006+0000 7f8e61467700 1 -- 172.21.15.119:0/3532030729 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8e44038b20 msgr2=0x7f8e4403afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:06.488 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:06.006+0000 7f8e61467700 1 --2- 172.21.15.119:0/3532030729 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8e44038b20 0x7f8e4403afe0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8e50006fd0 tx=0x7f8e50006e40 comp rx=0 tx=0).stop 2023-12-07T10:11:06.488 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:06.006+0000 7f8e61467700 1 -- 172.21.15.119:0/3532030729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e5c106c60 msgr2=0x7f8e5c19cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:06.488 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:06.006+0000 7f8e61467700 1 --2- 172.21.15.119:0/3532030729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e5c106c60 0x7f8e5c19cc80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c003e60 tx=0x7f8e4c011a30 comp rx=0 tx=0).stop 2023-12-07T10:11:06.489 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:06.006+0000 7f8e61467700 1 -- 172.21.15.119:0/3532030729 shutdown_connections 2023-12-07T10:11:06.489 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:06.006+0000 7f8e61467700 1 --2- 172.21.15.119:0/3532030729 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8e44038b20 0x7f8e4403afe0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:06.489 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:06.006+0000 7f8e61467700 1 --2- 172.21.15.119:0/3532030729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e5c106c60 0x7f8e5c19cc80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:06.489 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:06.006+0000 7f8e61467700 1 -- 172.21.15.119:0/3532030729 >> 172.21.15.119:0/3532030729 conn(0x7f8e5c102220 msgr2=0x7f8e5c102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:06.489 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:06.006+0000 7f8e61467700 1 -- 172.21.15.119:0/3532030729 shutdown_connections 2023-12-07T10:11:06.489 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:06.006+0000 7f8e61467700 1 -- 172.21.15.119:0/3532030729 wait complete. 2023-12-07T10:11:06.505 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-07T10:11:06.505 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout disabled 2023-12-07T10:11:06.525 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-07T10:11:06.525 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout inactive 2023-12-07T10:11:06.525 INFO:teuthology.orchestra.run.smithi119.stdout:firewalld.service is not enabled 2023-12-07T10:11:06.525 INFO:teuthology.orchestra.run.smithi119.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-12-07T10:11:06.528 INFO:teuthology.orchestra.run.smithi119.stdout:Ceph Dashboard is now available at: 2023-12-07T10:11:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:11:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: URL: https://smithi119.front.sepia.ceph.com:8443/ 2023-12-07T10:11:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: User: admin 2023-12-07T10:11:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: Password: khsx8dr5s0 2023-12-07T10:11:06.528 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:11:06.528 INFO:teuthology.orchestra.run.smithi119.stdout:Saving cluster configuration to /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config directory 2023-12-07T10:11:06.529 INFO:teuthology.orchestra.run.smithi119.stdout:Enabling autotune for osd_memory_target 2023-12-07T10:11:06.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:06 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3532030729' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-12-07T10:11:07.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:07 smithi119 ceph-mon[97908]: mgrmap e13: x(active, since 6s) 2023-12-07T10:11:08.019 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.308+0000 7f965b122700 1 Processor -- start 2023-12-07T10:11:08.020 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.308+0000 7f965b122700 1 -- start start 2023-12-07T10:11:08.020 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.308+0000 7f965b122700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f965407ce90 0x7f965407b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:08.020 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.308+0000 7f965b122700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965407b8c0 con 0x7f965407ce90 2023-12-07T10:11:08.020 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.308+0000 7f9658ebe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f965407ce90 0x7f965407b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:08.020 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.308+0000 7f9658ebe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f965407ce90 0x7f965407b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37440/0 (socket says 172.21.15.119:37440) 2023-12-07T10:11:08.021 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.308+0000 7f9658ebe700 1 -- 172.21.15.119:0/2959655534 learned_addr learned my addr 172.21.15.119:0/2959655534 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:11:08.021 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.309+0000 7f9658ebe700 1 -- 172.21.15.119:0/2959655534 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965407ba00 con 0x7f965407ce90 2023-12-07T10:11:08.021 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.309+0000 7f9658ebe700 1 --2- 172.21.15.119:0/2959655534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f965407ce90 0x7f965407b2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f96480098a0 tx=0x7f9648009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ecfa4de297f972 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:08.021 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.309+0000 7f96537fe700 1 -- 172.21.15.119:0/2959655534 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9648004160 con 0x7f965407ce90 2023-12-07T10:11:08.021 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.309+0000 7f96537fe700 1 -- 172.21.15.119:0/2959655534 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96480042c0 con 0x7f965407ce90 2023-12-07T10:11:08.021 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.309+0000 7f96537fe700 1 -- 172.21.15.119:0/2959655534 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96480119b0 con 0x7f965407ce90 2023-12-07T10:11:08.022 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.310+0000 7f965b122700 1 -- 172.21.15.119:0/2959655534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f965407ce90 msgr2=0x7f965407b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:08.022 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.310+0000 7f965b122700 1 --2- 172.21.15.119:0/2959655534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f965407ce90 0x7f965407b2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f96480098a0 tx=0x7f9648009bb0 comp rx=0 tx=0).stop 2023-12-07T10:11:08.022 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.310+0000 7f965b122700 1 -- 172.21.15.119:0/2959655534 shutdown_connections 2023-12-07T10:11:08.022 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.310+0000 7f965b122700 1 --2- 172.21.15.119:0/2959655534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f965407ce90 0x7f965407b2f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:08.022 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.310+0000 7f965b122700 1 -- 172.21.15.119:0/2959655534 >> 172.21.15.119:0/2959655534 conn(0x7f9654102220 msgr2=0x7f9654104640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:08.022 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.310+0000 7f965b122700 1 -- 172.21.15.119:0/2959655534 shutdown_connections 2023-12-07T10:11:08.022 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.310+0000 7f965b122700 1 -- 172.21.15.119:0/2959655534 wait complete. 2023-12-07T10:11:08.023 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.311+0000 7f965b122700 1 Processor -- start 2023-12-07T10:11:08.023 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.311+0000 7f965b122700 1 -- start start 2023-12-07T10:11:08.023 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.311+0000 7f965b122700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f965407ce90 0x7f965419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:08.023 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.311+0000 7f965b122700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965419d140 con 0x7f965407ce90 2023-12-07T10:11:08.023 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.312+0000 7f9658ebe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f965407ce90 0x7f965419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:08.023 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.312+0000 7f9658ebe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f965407ce90 0x7f965419cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37448/0 (socket says 172.21.15.119:37448) 2023-12-07T10:11:08.023 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.312+0000 7f9658ebe700 1 -- 172.21.15.119:0/2335532676 learned_addr learned my addr 172.21.15.119:0/2335532676 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:11:08.024 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.312+0000 7f9658ebe700 1 -- 172.21.15.119:0/2335532676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9648009550 con 0x7f965407ce90 2023-12-07T10:11:08.024 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.312+0000 7f9658ebe700 1 --2- 172.21.15.119:0/2335532676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f965407ce90 0x7f965419cc00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9648003e60 tx=0x7f9648011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:08.024 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.312+0000 7f9651ffb700 1 -- 172.21.15.119:0/2335532676 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9648003a60 con 0x7f965407ce90 2023-12-07T10:11:08.024 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.312+0000 7f9651ffb700 1 -- 172.21.15.119:0/2335532676 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9648003bc0 con 0x7f965407ce90 2023-12-07T10:11:08.024 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.312+0000 7f9651ffb700 1 -- 172.21.15.119:0/2335532676 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96480196c0 con 0x7f965407ce90 2023-12-07T10:11:08.024 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.312+0000 7f965b122700 1 -- 172.21.15.119:0/2335532676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f965419d340 con 0x7f965407ce90 2023-12-07T10:11:08.024 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.313+0000 7f965b122700 1 -- 172.21.15.119:0/2335532676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f965419d7e0 con 0x7f965407ce90 2023-12-07T10:11:08.025 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.313+0000 7f965b122700 1 -- 172.21.15.119:0/2335532676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f965410d080 con 0x7f965407ce90 2023-12-07T10:11:08.025 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.313+0000 7f9651ffb700 1 -- 172.21.15.119:0/2335532676 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45796+0+0 (secure 0 0 0) 0x7f964801ab60 con 0x7f965407ce90 2023-12-07T10:11:08.025 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.314+0000 7f9651ffb700 1 --2- 172.21.15.119:0/2335532676 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f9644038b70 0x7f964403b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:08.025 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.314+0000 7f9651ffb700 1 -- 172.21.15.119:0/2335532676 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f964804cfd0 con 0x7f965407ce90 2023-12-07T10:11:08.026 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.314+0000 7f9653fff700 1 --2- 172.21.15.119:0/2335532676 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f9644038b70 0x7f964403b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:08.026 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.315+0000 7f9653fff700 1 --2- 172.21.15.119:0/2335532676 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f9644038b70 0x7f964403b030 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9640006fd0 tx=0x7f9640006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:08.026 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.320+0000 7f9651ffb700 1 -- 172.21.15.119:0/2335532676 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9648018b00 con 0x7f965407ce90 2023-12-07T10:11:08.026 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.519+0000 7f965b122700 1 -- 172.21.15.119:0/2335532676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f965410d290 con 0x7f965407ce90 2023-12-07T10:11:08.026 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.520+0000 7f9651ffb700 1 -- 172.21.15.119:0/2335532676 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f96480281c0 con 0x7f965407ce90 2023-12-07T10:11:08.026 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.522+0000 7f965b122700 1 -- 172.21.15.119:0/2335532676 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f9644038b70 msgr2=0x7f964403b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:08.027 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.522+0000 7f965b122700 1 --2- 172.21.15.119:0/2335532676 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f9644038b70 0x7f964403b030 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9640006fd0 tx=0x7f9640006e40 comp rx=0 tx=0).stop 2023-12-07T10:11:08.027 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.522+0000 7f965b122700 1 -- 172.21.15.119:0/2335532676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f965407ce90 msgr2=0x7f965419cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:08.027 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.522+0000 7f965b122700 1 --2- 172.21.15.119:0/2335532676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f965407ce90 0x7f965419cc00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9648003e60 tx=0x7f9648011a30 comp rx=0 tx=0).stop 2023-12-07T10:11:08.027 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.522+0000 7f965b122700 1 -- 172.21.15.119:0/2335532676 shutdown_connections 2023-12-07T10:11:08.027 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.522+0000 7f965b122700 1 --2- 172.21.15.119:0/2335532676 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f9644038b70 0x7f964403b030 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:08.027 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.522+0000 7f965b122700 1 --2- 172.21.15.119:0/2335532676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f965407ce90 0x7f965419cc00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:08.027 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.522+0000 7f965b122700 1 -- 172.21.15.119:0/2335532676 >> 172.21.15.119:0/2335532676 conn(0x7f9654102220 msgr2=0x7f9654102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:08.028 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.523+0000 7f965b122700 1 -- 172.21.15.119:0/2335532676 shutdown_connections 2023-12-07T10:11:08.028 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:07.523+0000 7f965b122700 1 -- 172.21.15.119:0/2335532676 wait complete. 2023-12-07T10:11:09.297 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.591+0000 7f4101240700 1 Processor -- start 2023-12-07T10:11:09.297 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.591+0000 7f4101240700 1 -- start start 2023-12-07T10:11:09.297 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.591+0000 7f4101240700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40fc07ceb0 0x7f40fc07b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:09.298 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.591+0000 7f4101240700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc07b8e0 con 0x7f40fc07ceb0 2023-12-07T10:11:09.298 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.592+0000 7f40fad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40fc07ceb0 0x7f40fc07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:09.298 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.592+0000 7f40fad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40fc07ceb0 0x7f40fc07b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37456/0 (socket says 172.21.15.119:37456) 2023-12-07T10:11:09.298 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.592+0000 7f40fad9d700 1 -- 172.21.15.119:0/298626947 learned_addr learned my addr 172.21.15.119:0/298626947 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:11:09.298 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.592+0000 7f40fad9d700 1 -- 172.21.15.119:0/298626947 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40fc07ba20 con 0x7f40fc07ceb0 2023-12-07T10:11:09.298 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.592+0000 7f40fad9d700 1 --2- 172.21.15.119:0/298626947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40fc07ceb0 0x7f40fc07b310 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0098a0 tx=0x7f40ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9f09f5c6159b73 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:09.298 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.592+0000 7f40f9d9b700 1 -- 172.21.15.119:0/298626947 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40ec004160 con 0x7f40fc07ceb0 2023-12-07T10:11:09.298 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.592+0000 7f40f9d9b700 1 -- 172.21.15.119:0/298626947 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40ec0042c0 con 0x7f40fc07ceb0 2023-12-07T10:11:09.298 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.592+0000 7f40f9d9b700 1 -- 172.21.15.119:0/298626947 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40ec0119b0 con 0x7f40fc07ceb0 2023-12-07T10:11:09.298 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.593+0000 7f4101240700 1 -- 172.21.15.119:0/298626947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40fc07ceb0 msgr2=0x7f40fc07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:09.298 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.593+0000 7f4101240700 1 --2- 172.21.15.119:0/298626947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40fc07ceb0 0x7f40fc07b310 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0098a0 tx=0x7f40ec009bb0 comp rx=0 tx=0).stop 2023-12-07T10:11:09.298 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.593+0000 7f4101240700 1 -- 172.21.15.119:0/298626947 shutdown_connections 2023-12-07T10:11:09.298 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.593+0000 7f4101240700 1 --2- 172.21.15.119:0/298626947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40fc07ceb0 0x7f40fc07b310 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:09.299 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.593+0000 7f4101240700 1 -- 172.21.15.119:0/298626947 >> 172.21.15.119:0/298626947 conn(0x7f40fc102220 msgr2=0x7f40fc104660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:09.299 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.593+0000 7f4101240700 1 -- 172.21.15.119:0/298626947 shutdown_connections 2023-12-07T10:11:09.299 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.593+0000 7f4101240700 1 -- 172.21.15.119:0/298626947 wait complete. 2023-12-07T10:11:09.299 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.594+0000 7f4101240700 1 Processor -- start 2023-12-07T10:11:09.299 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.594+0000 7f4101240700 1 -- start start 2023-12-07T10:11:09.299 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.595+0000 7f4101240700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40fc07ceb0 0x7f40fc19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:09.299 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.595+0000 7f4101240700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc19d150 con 0x7f40fc07ceb0 2023-12-07T10:11:09.299 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.595+0000 7f40fad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40fc07ceb0 0x7f40fc19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:09.299 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.595+0000 7f40fad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40fc07ceb0 0x7f40fc19cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37460/0 (socket says 172.21.15.119:37460) 2023-12-07T10:11:09.299 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.595+0000 7f40fad9d700 1 -- 172.21.15.119:0/2035816413 learned_addr learned my addr 172.21.15.119:0/2035816413 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:11:09.299 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.595+0000 7f40fad9d700 1 -- 172.21.15.119:0/2035816413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec009550 con 0x7f40fc07ceb0 2023-12-07T10:11:09.299 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.595+0000 7f40fad9d700 1 --2- 172.21.15.119:0/2035816413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40fc07ceb0 0x7f40fc19cc10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f40ec003e60 tx=0x7f40ec011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:09.299 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.596+0000 7f40e3fff700 1 -- 172.21.15.119:0/2035816413 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40ec003a60 con 0x7f40fc07ceb0 2023-12-07T10:11:09.299 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.596+0000 7f40e3fff700 1 -- 172.21.15.119:0/2035816413 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40ec003bc0 con 0x7f40fc07ceb0 2023-12-07T10:11:09.300 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.596+0000 7f40e3fff700 1 -- 172.21.15.119:0/2035816413 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40ec0196c0 con 0x7f40fc07ceb0 2023-12-07T10:11:09.300 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.596+0000 7f4101240700 1 -- 172.21.15.119:0/2035816413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40fc19d350 con 0x7f40fc07ceb0 2023-12-07T10:11:09.300 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.596+0000 7f4101240700 1 -- 172.21.15.119:0/2035816413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40fc19d7f0 con 0x7f40fc07ceb0 2023-12-07T10:11:09.300 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.597+0000 7f40e3fff700 1 -- 172.21.15.119:0/2035816413 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45796+0+0 (secure 0 0 0) 0x7f40ec01ab60 con 0x7f40fc07ceb0 2023-12-07T10:11:09.300 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.597+0000 7f4101240700 1 -- 172.21.15.119:0/2035816413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40fc10d080 con 0x7f40fc07ceb0 2023-12-07T10:11:09.300 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.597+0000 7f40e3fff700 1 --2- 172.21.15.119:0/2035816413 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f40e4038b70 0x7f40e403b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:09.300 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.597+0000 7f40e3fff700 1 -- 172.21.15.119:0/2035816413 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f40ec04cfd0 con 0x7f40fc07ceb0 2023-12-07T10:11:09.300 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.598+0000 7f40fa59c700 1 --2- 172.21.15.119:0/2035816413 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f40e4038b70 0x7f40e403b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:09.300 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.598+0000 7f40fa59c700 1 --2- 172.21.15.119:0/2035816413 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f40e4038b70 0x7f40e403b030 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f40f0006fd0 tx=0x7f40f0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:09.300 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.602+0000 7f40e3fff700 1 -- 172.21.15.119:0/2035816413 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f40ec018b00 con 0x7f40fc07ceb0 2023-12-07T10:11:09.300 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.905+0000 7f4101240700 1 -- 172.21.15.119:0/2035816413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f40fc10d2f0 con 0x7f40fc07ceb0 2023-12-07T10:11:09.300 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.908+0000 7f40e3fff700 1 -- 172.21.15.119:0/2035816413 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7f40ec0281c0 con 0x7f40fc07ceb0 2023-12-07T10:11:09.300 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.911+0000 7f4101240700 1 -- 172.21.15.119:0/2035816413 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f40e4038b70 msgr2=0x7f40e403b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:09.300 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.911+0000 7f4101240700 1 --2- 172.21.15.119:0/2035816413 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f40e4038b70 0x7f40e403b030 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f40f0006fd0 tx=0x7f40f0006e40 comp rx=0 tx=0).stop 2023-12-07T10:11:09.300 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.911+0000 7f4101240700 1 -- 172.21.15.119:0/2035816413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40fc07ceb0 msgr2=0x7f40fc19cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:09.301 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.911+0000 7f4101240700 1 --2- 172.21.15.119:0/2035816413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40fc07ceb0 0x7f40fc19cc10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f40ec003e60 tx=0x7f40ec011a30 comp rx=0 tx=0).stop 2023-12-07T10:11:09.301 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.911+0000 7f4101240700 1 -- 172.21.15.119:0/2035816413 shutdown_connections 2023-12-07T10:11:09.301 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.911+0000 7f4101240700 1 --2- 172.21.15.119:0/2035816413 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f40e4038b70 0x7f40e403b030 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:09.301 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.911+0000 7f4101240700 1 --2- 172.21.15.119:0/2035816413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40fc07ceb0 0x7f40fc19cc10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:09.301 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.911+0000 7f4101240700 1 -- 172.21.15.119:0/2035816413 >> 172.21.15.119:0/2035816413 conn(0x7f40fc102220 msgr2=0x7f40fc102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:09.301 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.911+0000 7f4101240700 1 -- 172.21.15.119:0/2035816413 shutdown_connections 2023-12-07T10:11:09.301 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-12-07T10:11:08.911+0000 7f4101240700 1 -- 172.21.15.119:0/2035816413 wait complete. 2023-12-07T10:11:09.301 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-12-07T10:11:09.301 INFO:teuthology.orchestra.run.smithi119.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-12-07T10:11:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:11:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-12-07T10:11:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:11:09.301 INFO:teuthology.orchestra.run.smithi119.stdout:Or, if you are only running a single cluster on this host: 2023-12-07T10:11:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:11:09.301 INFO:teuthology.orchestra.run.smithi119.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-12-07T10:11:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:11:09.302 INFO:teuthology.orchestra.run.smithi119.stdout:Please consider enabling telemetry to help improve Ceph: 2023-12-07T10:11:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:11:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: ceph telemetry on 2023-12-07T10:11:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:11:09.302 INFO:teuthology.orchestra.run.smithi119.stdout:For more information see: 2023-12-07T10:11:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:11:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-12-07T10:11:09.302 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:11:09.302 INFO:teuthology.orchestra.run.smithi119.stdout:Bootstrap complete. 2023-12-07T10:11:09.362 INFO:tasks.cephadm:Fetching config... 2023-12-07T10:11:09.362 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-12-07T10:11:09.362 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-12-07T10:11:09.405 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-12-07T10:11:09.405 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-12-07T10:11:09.405 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-12-07T10:11:09.491 INFO:tasks.cephadm:Fetching mon keyring... 2023-12-07T10:11:09.491 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-12-07T10:11:09.491 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/keyring of=/dev/stdout 2023-12-07T10:11:09.600 INFO:tasks.cephadm:Fetching pub ssh key... 2023-12-07T10:11:09.601 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-12-07T10:11:09.601 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-12-07T10:11:09.686 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-12-07T10:11:09.687 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+j/G7Kagnmwi5u9jUXZynEt7Ze+IxYbWrDeLNakvmCiKMqyj4BuMYFNI9SBpMg6XAYeOhcGKGQuONJyqQEXcemcPoZYyOcpFURxoOCxAcnHsUCLTFGZ3ZpceBVERR3nqgf1UA54+9LYttoDERldFZVD1kYCe+AQbkEBMNhMH6gZ/n0BSWx05V4A7HGxV++vCndUKLIM2ZfJddvpc82cy6ASrGAy7jSE8HtdQQ3Yq/h6yXFyZSIIwT0XSueDMThFtRSa9D3EQ1fbZsYRN4+LAKsJgcveNGksb0qmRLItpK2hpQHsv1uXtDOgvvI5pZhWic7Lv6O+toncRhOyEc1oEbQBRYYJI6ZXWCh3zH7UvjKycSNu7nVM+LV68Td49/Fk04pso7V9HNt9ipjpPcO1t859H0lD6HcUKVOwOmSIa6E3tSqHR+B22CjYgjWTmS2i8Y870DEnFuSXPY1QcdFSYkTSrkf2RicfXQ8WH36ID1IPRG92EPWr7R7Ld5F1SAQoM= ceph-a4c642c6-94e8-11ee-95a2-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-07T10:11:09.838 INFO:teuthology.orchestra.run.smithi119.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+j/G7Kagnmwi5u9jUXZynEt7Ze+IxYbWrDeLNakvmCiKMqyj4BuMYFNI9SBpMg6XAYeOhcGKGQuONJyqQEXcemcPoZYyOcpFURxoOCxAcnHsUCLTFGZ3ZpceBVERR3nqgf1UA54+9LYttoDERldFZVD1kYCe+AQbkEBMNhMH6gZ/n0BSWx05V4A7HGxV++vCndUKLIM2ZfJddvpc82cy6ASrGAy7jSE8HtdQQ3Yq/h6yXFyZSIIwT0XSueDMThFtRSa9D3EQ1fbZsYRN4+LAKsJgcveNGksb0qmRLItpK2hpQHsv1uXtDOgvvI5pZhWic7Lv6O+toncRhOyEc1oEbQBRYYJI6ZXWCh3zH7UvjKycSNu7nVM+LV68Td49/Fk04pso7V9HNt9ipjpPcO1t859H0lD6HcUKVOwOmSIa6E3tSqHR+B22CjYgjWTmS2i8Y870DEnFuSXPY1QcdFSYkTSrkf2RicfXQ8WH36ID1IPRG92EPWr7R7Ld5F1SAQoM= ceph-a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:11:09.870 DEBUG:teuthology.orchestra.run.smithi153:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+j/G7Kagnmwi5u9jUXZynEt7Ze+IxYbWrDeLNakvmCiKMqyj4BuMYFNI9SBpMg6XAYeOhcGKGQuONJyqQEXcemcPoZYyOcpFURxoOCxAcnHsUCLTFGZ3ZpceBVERR3nqgf1UA54+9LYttoDERldFZVD1kYCe+AQbkEBMNhMH6gZ/n0BSWx05V4A7HGxV++vCndUKLIM2ZfJddvpc82cy6ASrGAy7jSE8HtdQQ3Yq/h6yXFyZSIIwT0XSueDMThFtRSa9D3EQ1fbZsYRN4+LAKsJgcveNGksb0qmRLItpK2hpQHsv1uXtDOgvvI5pZhWic7Lv6O+toncRhOyEc1oEbQBRYYJI6ZXWCh3zH7UvjKycSNu7nVM+LV68Td49/Fk04pso7V9HNt9ipjpPcO1t859H0lD6HcUKVOwOmSIa6E3tSqHR+B22CjYgjWTmS2i8Y870DEnFuSXPY1QcdFSYkTSrkf2RicfXQ8WH36ID1IPRG92EPWr7R7Ld5F1SAQoM= ceph-a4c642c6-94e8-11ee-95a2-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-07T10:11:09.917 INFO:teuthology.orchestra.run.smithi153.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+j/G7Kagnmwi5u9jUXZynEt7Ze+IxYbWrDeLNakvmCiKMqyj4BuMYFNI9SBpMg6XAYeOhcGKGQuONJyqQEXcemcPoZYyOcpFURxoOCxAcnHsUCLTFGZ3ZpceBVERR3nqgf1UA54+9LYttoDERldFZVD1kYCe+AQbkEBMNhMH6gZ/n0BSWx05V4A7HGxV++vCndUKLIM2ZfJddvpc82cy6ASrGAy7jSE8HtdQQ3Yq/h6yXFyZSIIwT0XSueDMThFtRSa9D3EQ1fbZsYRN4+LAKsJgcveNGksb0qmRLItpK2hpQHsv1uXtDOgvvI5pZhWic7Lv6O+toncRhOyEc1oEbQBRYYJI6ZXWCh3zH7UvjKycSNu7nVM+LV68Td49/Fk04pso7V9HNt9ipjpPcO1t859H0lD6HcUKVOwOmSIa6E3tSqHR+B22CjYgjWTmS2i8Y870DEnFuSXPY1QcdFSYkTSrkf2RicfXQ8WH36ID1IPRG92EPWr7R7Ld5F1SAQoM= ceph-a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:11:09.929 DEBUG:teuthology.orchestra.run.smithi159:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+j/G7Kagnmwi5u9jUXZynEt7Ze+IxYbWrDeLNakvmCiKMqyj4BuMYFNI9SBpMg6XAYeOhcGKGQuONJyqQEXcemcPoZYyOcpFURxoOCxAcnHsUCLTFGZ3ZpceBVERR3nqgf1UA54+9LYttoDERldFZVD1kYCe+AQbkEBMNhMH6gZ/n0BSWx05V4A7HGxV++vCndUKLIM2ZfJddvpc82cy6ASrGAy7jSE8HtdQQ3Yq/h6yXFyZSIIwT0XSueDMThFtRSa9D3EQ1fbZsYRN4+LAKsJgcveNGksb0qmRLItpK2hpQHsv1uXtDOgvvI5pZhWic7Lv6O+toncRhOyEc1oEbQBRYYJI6ZXWCh3zH7UvjKycSNu7nVM+LV68Td49/Fk04pso7V9HNt9ipjpPcO1t859H0lD6HcUKVOwOmSIa6E3tSqHR+B22CjYgjWTmS2i8Y870DEnFuSXPY1QcdFSYkTSrkf2RicfXQ8WH36ID1IPRG92EPWr7R7Ld5F1SAQoM= ceph-a4c642c6-94e8-11ee-95a2-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-07T10:11:09.975 INFO:teuthology.orchestra.run.smithi159.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+j/G7Kagnmwi5u9jUXZynEt7Ze+IxYbWrDeLNakvmCiKMqyj4BuMYFNI9SBpMg6XAYeOhcGKGQuONJyqQEXcemcPoZYyOcpFURxoOCxAcnHsUCLTFGZ3ZpceBVERR3nqgf1UA54+9LYttoDERldFZVD1kYCe+AQbkEBMNhMH6gZ/n0BSWx05V4A7HGxV++vCndUKLIM2ZfJddvpc82cy6ASrGAy7jSE8HtdQQ3Yq/h6yXFyZSIIwT0XSueDMThFtRSa9D3EQ1fbZsYRN4+LAKsJgcveNGksb0qmRLItpK2hpQHsv1uXtDOgvvI5pZhWic7Lv6O+toncRhOyEc1oEbQBRYYJI6ZXWCh3zH7UvjKycSNu7nVM+LV68Td49/Fk04pso7V9HNt9ipjpPcO1t859H0lD6HcUKVOwOmSIa6E3tSqHR+B22CjYgjWTmS2i8Y870DEnFuSXPY1QcdFSYkTSrkf2RicfXQ8WH36ID1IPRG92EPWr7R7Ld5F1SAQoM= ceph-a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:11:09.988 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-12-07T10:11:10.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:09 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2035816413' entity='client.admin' 2023-12-07T10:11:10.770 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:11:12.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.625+0000 7ff43a71c700 1 -- 172.21.15.119:0/1240301832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff434100f70 msgr2=0x7ff434101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:12.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.625+0000 7ff43a71c700 1 --2- 172.21.15.119:0/1240301832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff434100f70 0x7ff434101390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff424009910 tx=0x7ff424009c20 comp rx=0 tx=0).stop 2023-12-07T10:11:12.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.625+0000 7ff43a71c700 1 -- 172.21.15.119:0/1240301832 shutdown_connections 2023-12-07T10:11:12.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.625+0000 7ff43a71c700 1 --2- 172.21.15.119:0/1240301832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff434100f70 0x7ff434101390 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:12.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.625+0000 7ff43a71c700 1 -- 172.21.15.119:0/1240301832 >> 172.21.15.119:0/1240301832 conn(0x7ff4340fc530 msgr2=0x7ff4340fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:12.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.625+0000 7ff43a71c700 1 -- 172.21.15.119:0/1240301832 shutdown_connections 2023-12-07T10:11:12.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.625+0000 7ff43a71c700 1 -- 172.21.15.119:0/1240301832 wait complete. 2023-12-07T10:11:12.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.626+0000 7ff43a71c700 1 Processor -- start 2023-12-07T10:11:12.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.626+0000 7ff43a71c700 1 -- start start 2023-12-07T10:11:12.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.626+0000 7ff43a71c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff434100f70 0x7ff434198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:12.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.626+0000 7ff43a71c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff434198b60 con 0x7ff434100f70 2023-12-07T10:11:12.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.626+0000 7ff433fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff434100f70 0x7ff434198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:12.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.626+0000 7ff433fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff434100f70 0x7ff434198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37470/0 (socket says 172.21.15.119:37470) 2023-12-07T10:11:12.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.627+0000 7ff433fff700 1 -- 172.21.15.119:0/1881709635 learned_addr learned my addr 172.21.15.119:0/1881709635 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:11:12.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.627+0000 7ff433fff700 1 -- 172.21.15.119:0/1881709635 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4240095f0 con 0x7ff434100f70 2023-12-07T10:11:12.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.627+0000 7ff433fff700 1 --2- 172.21.15.119:0/1881709635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff434100f70 0x7ff434198620 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff424005950 tx=0x7ff4240057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:12.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.627+0000 7ff4317fa700 1 -- 172.21.15.119:0/1881709635 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff42401d070 con 0x7ff434100f70 2023-12-07T10:11:12.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.627+0000 7ff4317fa700 1 -- 172.21.15.119:0/1881709635 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff42400bc10 con 0x7ff434100f70 2023-12-07T10:11:12.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.627+0000 7ff4317fa700 1 -- 172.21.15.119:0/1881709635 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff424003a10 con 0x7ff434100f70 2023-12-07T10:11:12.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.627+0000 7ff43a71c700 1 -- 172.21.15.119:0/1881709635 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff434198d60 con 0x7ff434100f70 2023-12-07T10:11:12.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.627+0000 7ff43a71c700 1 -- 172.21.15.119:0/1881709635 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff434199200 con 0x7ff434100f70 2023-12-07T10:11:12.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.628+0000 7ff4317fa700 1 -- 172.21.15.119:0/1881709635 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45796+0+0 (secure 0 0 0) 0x7ff424004390 con 0x7ff434100f70 2023-12-07T10:11:12.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.628+0000 7ff43a71c700 1 -- 172.21.15.119:0/1881709635 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff434192090 con 0x7ff434100f70 2023-12-07T10:11:12.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.629+0000 7ff4317fa700 1 --2- 172.21.15.119:0/1881709635 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff41c038b40 0x7ff41c03b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:12.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.629+0000 7ff4317fa700 1 -- 172.21.15.119:0/1881709635 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff42404c9d0 con 0x7ff434100f70 2023-12-07T10:11:12.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.629+0000 7ff4337fe700 1 --2- 172.21.15.119:0/1881709635 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff41c038b40 0x7ff41c03b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:12.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.630+0000 7ff4337fe700 1 --2- 172.21.15.119:0/1881709635 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff41c038b40 0x7ff41c03b000 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff428006fd0 tx=0x7ff428006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:12.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.634+0000 7ff4317fa700 1 -- 172.21.15.119:0/1881709635 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff42402abf0 con 0x7ff434100f70 2023-12-07T10:11:12.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.833+0000 7ff43a71c700 1 -- 172.21.15.119:0/1881709635 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7ff434062c40 con 0x7ff434100f70 2023-12-07T10:11:12.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.836+0000 7ff4317fa700 1 -- 172.21.15.119:0/1881709635 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7ff434062c40 con 0x7ff434100f70 2023-12-07T10:11:12.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.838+0000 7ff43a71c700 1 -- 172.21.15.119:0/1881709635 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff41c038b40 msgr2=0x7ff41c03b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:12.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.838+0000 7ff43a71c700 1 --2- 172.21.15.119:0/1881709635 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff41c038b40 0x7ff41c03b000 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff428006fd0 tx=0x7ff428006e40 comp rx=0 tx=0).stop 2023-12-07T10:11:12.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.838+0000 7ff43a71c700 1 -- 172.21.15.119:0/1881709635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff434100f70 msgr2=0x7ff434198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:12.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.838+0000 7ff43a71c700 1 --2- 172.21.15.119:0/1881709635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff434100f70 0x7ff434198620 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff424005950 tx=0x7ff4240057c0 comp rx=0 tx=0).stop 2023-12-07T10:11:12.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.838+0000 7ff43a71c700 1 -- 172.21.15.119:0/1881709635 shutdown_connections 2023-12-07T10:11:12.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.838+0000 7ff43a71c700 1 --2- 172.21.15.119:0/1881709635 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff41c038b40 0x7ff41c03b000 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:12.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.838+0000 7ff43a71c700 1 --2- 172.21.15.119:0/1881709635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff434100f70 0x7ff434198620 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:12.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.838+0000 7ff43a71c700 1 -- 172.21.15.119:0/1881709635 >> 172.21.15.119:0/1881709635 conn(0x7ff4340fc530 msgr2=0x7ff4340fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:12.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.839+0000 7ff43a71c700 1 -- 172.21.15.119:0/1881709635 shutdown_connections 2023-12-07T10:11:12.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:12.839+0000 7ff43a71c700 1 -- 172.21.15.119:0/1881709635 wait complete. 2023-12-07T10:11:13.241 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-12-07T10:11:13.241 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-12-07T10:11:13.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:13.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:13.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi119", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:11:13.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:13.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:11:13.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:13.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:13 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1881709635' entity='client.admin' 2023-12-07T10:11:13.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:11:13.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:14.008 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:11:15.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.528+0000 7f17a79d3700 1 -- 172.21.15.119:0/2891738745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17a0100f70 msgr2=0x7f17a0101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:15.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.528+0000 7f17a79d3700 1 --2- 172.21.15.119:0/2891738745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17a0100f70 0x7f17a0101390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f179c009910 tx=0x7f179c009c20 comp rx=0 tx=0).stop 2023-12-07T10:11:15.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.528+0000 7f17a79d3700 1 -- 172.21.15.119:0/2891738745 shutdown_connections 2023-12-07T10:11:15.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.528+0000 7f17a79d3700 1 --2- 172.21.15.119:0/2891738745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17a0100f70 0x7f17a0101390 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:15.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.528+0000 7f17a79d3700 1 -- 172.21.15.119:0/2891738745 >> 172.21.15.119:0/2891738745 conn(0x7f17a00fc530 msgr2=0x7f17a00fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:15.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.528+0000 7f17a79d3700 1 -- 172.21.15.119:0/2891738745 shutdown_connections 2023-12-07T10:11:15.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.528+0000 7f17a79d3700 1 -- 172.21.15.119:0/2891738745 wait complete. 2023-12-07T10:11:15.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.530+0000 7f17a79d3700 1 Processor -- start 2023-12-07T10:11:15.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.530+0000 7f17a79d3700 1 -- start start 2023-12-07T10:11:15.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.530+0000 7f17a79d3700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17a0100f70 0x7f17a0198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:15.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.531+0000 7f17a79d3700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a0198b60 con 0x7f17a0100f70 2023-12-07T10:11:15.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.531+0000 7f17a576f700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17a0100f70 0x7f17a0198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:15.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.531+0000 7f17a576f700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17a0100f70 0x7f17a0198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52170/0 (socket says 172.21.15.119:52170) 2023-12-07T10:11:15.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.531+0000 7f17a576f700 1 -- 172.21.15.119:0/1162028151 learned_addr learned my addr 172.21.15.119:0/1162028151 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:11:15.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.531+0000 7f17a576f700 1 -- 172.21.15.119:0/1162028151 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f179c0095f0 con 0x7f17a0100f70 2023-12-07T10:11:15.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.531+0000 7f17a576f700 1 --2- 172.21.15.119:0/1162028151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17a0100f70 0x7f17a0198620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f179c005950 tx=0x7f179c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:15.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.532+0000 7f17967fc700 1 -- 172.21.15.119:0/1162028151 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f179c01d070 con 0x7f17a0100f70 2023-12-07T10:11:15.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.532+0000 7f17a79d3700 1 -- 172.21.15.119:0/1162028151 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a0198d60 con 0x7f17a0100f70 2023-12-07T10:11:15.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.532+0000 7f17967fc700 1 -- 172.21.15.119:0/1162028151 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f179c00bc10 con 0x7f17a0100f70 2023-12-07T10:11:15.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.532+0000 7f17967fc700 1 -- 172.21.15.119:0/1162028151 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f179c003a10 con 0x7f17a0100f70 2023-12-07T10:11:15.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.532+0000 7f17a79d3700 1 -- 172.21.15.119:0/1162028151 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a0199200 con 0x7f17a0100f70 2023-12-07T10:11:15.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.533+0000 7f17967fc700 1 -- 172.21.15.119:0/1162028151 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45796+0+0 (secure 0 0 0) 0x7f179c003cd0 con 0x7f17a0100f70 2023-12-07T10:11:15.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.533+0000 7f17a79d3700 1 -- 172.21.15.119:0/1162028151 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17a0192090 con 0x7f17a0100f70 2023-12-07T10:11:15.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.533+0000 7f17967fc700 1 --2- 172.21.15.119:0/1162028151 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f178c038af0 0x7f178c03afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:15.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.533+0000 7f17967fc700 1 -- 172.21.15.119:0/1162028151 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f179c04c9f0 con 0x7f17a0100f70 2023-12-07T10:11:15.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.533+0000 7f17a4f6e700 1 --2- 172.21.15.119:0/1162028151 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f178c038af0 0x7f178c03afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:15.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.534+0000 7f17a4f6e700 1 --2- 172.21.15.119:0/1162028151 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f178c038af0 0x7f178c03afb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1790006fd0 tx=0x7f1790006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:15.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.538+0000 7f17967fc700 1 -- 172.21.15.119:0/1162028151 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f179c00ba70 con 0x7f17a0100f70 2023-12-07T10:11:15.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.743+0000 7f17a79d3700 1 -- 172.21.15.119:0/1162028151 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f17a0061a50 con 0x7f178c038af0 2023-12-07T10:11:15.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.750+0000 7f17967fc700 1 -- 172.21.15.119:0/1162028151 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f17a0061a50 con 0x7f178c038af0 2023-12-07T10:11:15.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.752+0000 7f17a79d3700 1 -- 172.21.15.119:0/1162028151 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f178c038af0 msgr2=0x7f178c03afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:15.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.752+0000 7f17a79d3700 1 --2- 172.21.15.119:0/1162028151 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f178c038af0 0x7f178c03afb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1790006fd0 tx=0x7f1790006e40 comp rx=0 tx=0).stop 2023-12-07T10:11:15.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.752+0000 7f17a79d3700 1 -- 172.21.15.119:0/1162028151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17a0100f70 msgr2=0x7f17a0198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:15.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.752+0000 7f17a79d3700 1 --2- 172.21.15.119:0/1162028151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17a0100f70 0x7f17a0198620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f179c005950 tx=0x7f179c0057c0 comp rx=0 tx=0).stop 2023-12-07T10:11:15.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.752+0000 7f17a79d3700 1 -- 172.21.15.119:0/1162028151 shutdown_connections 2023-12-07T10:11:15.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.752+0000 7f17a79d3700 1 --2- 172.21.15.119:0/1162028151 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f178c038af0 0x7f178c03afb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:15.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.752+0000 7f17a79d3700 1 --2- 172.21.15.119:0/1162028151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17a0100f70 0x7f17a0198620 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:15.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.752+0000 7f17a79d3700 1 -- 172.21.15.119:0/1162028151 >> 172.21.15.119:0/1162028151 conn(0x7f17a00fc530 msgr2=0x7f17a00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:15.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.752+0000 7f17a79d3700 1 -- 172.21.15.119:0/1162028151 shutdown_connections 2023-12-07T10:11:15.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:15.752+0000 7f17a79d3700 1 -- 172.21.15.119:0/1162028151 wait complete. 2023-12-07T10:11:16.296 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi153 2023-12-07T10:11:16.296 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-12-07T10:11:16.296 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/etc/ceph/ceph.conf 2023-12-07T10:11:16.317 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-12-07T10:11:16.318 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-07T10:11:16.377 INFO:tasks.cephadm:Adding host smithi153 to orchestrator... 2023-12-07T10:11:16.378 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph orch host add smithi153 2023-12-07T10:11:17.193 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:11:17.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:16 smithi119 ceph-mon[97908]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:11:17.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:16 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:17.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:16 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:11:17.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:16 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:11:17.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:16 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:11:17.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:16 smithi119 ceph-mon[97908]: Updating smithi119:/etc/ceph/ceph.conf 2023-12-07T10:11:18.079 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:17 smithi119 ceph-mon[97908]: Updating smithi119:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:11:18.079 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:17 smithi119 ceph-mon[97908]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2023-12-07T10:11:18.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.767+0000 7f15a5179700 1 -- 172.21.15.119:0/1642125584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a0101170 msgr2=0x7f15a0101550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:18.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.767+0000 7f15a5179700 1 --2- 172.21.15.119:0/1642125584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a0101170 0x7f15a0101550 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1590009910 tx=0x7f1590009c20 comp rx=0 tx=0).stop 2023-12-07T10:11:18.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.767+0000 7f15a5179700 1 -- 172.21.15.119:0/1642125584 shutdown_connections 2023-12-07T10:11:18.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.767+0000 7f15a5179700 1 --2- 172.21.15.119:0/1642125584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a0101170 0x7f15a0101550 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:18.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.767+0000 7f15a5179700 1 -- 172.21.15.119:0/1642125584 >> 172.21.15.119:0/1642125584 conn(0x7f15a00fca40 msgr2=0x7f15a00fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:18.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.767+0000 7f15a5179700 1 -- 172.21.15.119:0/1642125584 shutdown_connections 2023-12-07T10:11:18.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.767+0000 7f15a5179700 1 -- 172.21.15.119:0/1642125584 wait complete. 2023-12-07T10:11:18.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.768+0000 7f15a5179700 1 Processor -- start 2023-12-07T10:11:18.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.768+0000 7f15a5179700 1 -- start start 2023-12-07T10:11:18.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.769+0000 7f15a5179700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a0101170 0x7f15a019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:18.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.769+0000 7f15a5179700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a019ec50 con 0x7f15a0101170 2023-12-07T10:11:18.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.769+0000 7f159ed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a0101170 0x7f15a019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:18.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.769+0000 7f159ed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a0101170 0x7f15a019e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52182/0 (socket says 172.21.15.119:52182) 2023-12-07T10:11:18.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.769+0000 7f159ed9d700 1 -- 172.21.15.119:0/2312023625 learned_addr learned my addr 172.21.15.119:0/2312023625 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:11:18.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.769+0000 7f159ed9d700 1 -- 172.21.15.119:0/2312023625 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15900095f0 con 0x7f15a0101170 2023-12-07T10:11:18.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.769+0000 7f159ed9d700 1 --2- 172.21.15.119:0/2312023625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a0101170 0x7f15a019e710 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1590006010 tx=0x7f1590004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:18.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.770+0000 7f1587fff700 1 -- 172.21.15.119:0/2312023625 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f159001d070 con 0x7f15a0101170 2023-12-07T10:11:18.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.770+0000 7f1587fff700 1 -- 172.21.15.119:0/2312023625 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f159000bc10 con 0x7f15a0101170 2023-12-07T10:11:18.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.770+0000 7f15a5179700 1 -- 172.21.15.119:0/2312023625 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a019edf0 con 0x7f15a0101170 2023-12-07T10:11:18.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.770+0000 7f1587fff700 1 -- 172.21.15.119:0/2312023625 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15900038a0 con 0x7f15a0101170 2023-12-07T10:11:18.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.770+0000 7f15a5179700 1 -- 172.21.15.119:0/2312023625 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a019f210 con 0x7f15a0101170 2023-12-07T10:11:18.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.771+0000 7f1587fff700 1 -- 172.21.15.119:0/2312023625 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45796+0+0 (secure 0 0 0) 0x7f159000b6f0 con 0x7f15a0101170 2023-12-07T10:11:18.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.771+0000 7f15a5179700 1 -- 172.21.15.119:0/2312023625 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a0102650 con 0x7f15a0101170 2023-12-07T10:11:18.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.771+0000 7f1587fff700 1 --2- 172.21.15.119:0/2312023625 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1588038b90 0x7f158803b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:18.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.771+0000 7f1587fff700 1 -- 172.21.15.119:0/2312023625 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f159004c890 con 0x7f15a0101170 2023-12-07T10:11:18.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.772+0000 7f159e59c700 1 --2- 172.21.15.119:0/2312023625 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1588038b90 0x7f158803b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:18.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.772+0000 7f159e59c700 1 --2- 172.21.15.119:0/2312023625 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1588038b90 0x7f158803b050 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1594006fd0 tx=0x7f1594006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:18.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.776+0000 7f1587fff700 1 -- 172.21.15.119:0/2312023625 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1590029540 con 0x7f15a0101170 2023-12-07T10:11:18.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:18.980+0000 7f15a5179700 1 -- 172.21.15.119:0/2312023625 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi153", "target": ["mon-mgr", ""]}) v1 -- 0x7f15a010b370 con 0x7f1588038b90 2023-12-07T10:11:19.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:19 smithi119 ceph-mon[97908]: Updating smithi119:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-07T10:11:19.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:19 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:19.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:19 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:19.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:19 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:20.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:20 smithi119 ceph-mon[97908]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi153", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:11:21.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:21 smithi119 ceph-mon[97908]: Deploying cephadm binary to smithi153 2023-12-07T10:11:22.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:22 smithi119 ceph-mon[97908]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:22.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:22.901+0000 7f1587fff700 1 -- 172.21.15.119:0/2312023625 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f15a010b370 con 0x7f1588038b90 2023-12-07T10:11:22.905 INFO:teuthology.orchestra.run.smithi119.stdout:Added host 'smithi153' with addr '172.21.15.153' 2023-12-07T10:11:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:22.904+0000 7f15a5179700 1 -- 172.21.15.119:0/2312023625 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1588038b90 msgr2=0x7f158803b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:22.904+0000 7f15a5179700 1 --2- 172.21.15.119:0/2312023625 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1588038b90 0x7f158803b050 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1594006fd0 tx=0x7f1594006e40 comp rx=0 tx=0).stop 2023-12-07T10:11:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:22.904+0000 7f15a5179700 1 -- 172.21.15.119:0/2312023625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a0101170 msgr2=0x7f15a019e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:22.904+0000 7f15a5179700 1 --2- 172.21.15.119:0/2312023625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a0101170 0x7f15a019e710 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1590006010 tx=0x7f1590004ff0 comp rx=0 tx=0).stop 2023-12-07T10:11:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:22.904+0000 7f15a5179700 1 -- 172.21.15.119:0/2312023625 shutdown_connections 2023-12-07T10:11:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:22.904+0000 7f15a5179700 1 --2- 172.21.15.119:0/2312023625 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1588038b90 0x7f158803b050 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:22.904+0000 7f15a5179700 1 --2- 172.21.15.119:0/2312023625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a0101170 0x7f15a019e710 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:22.904+0000 7f15a5179700 1 -- 172.21.15.119:0/2312023625 >> 172.21.15.119:0/2312023625 conn(0x7f15a00fca40 msgr2=0x7f15a0073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:22.904+0000 7f15a5179700 1 -- 172.21.15.119:0/2312023625 shutdown_connections 2023-12-07T10:11:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:22.904+0000 7f15a5179700 1 -- 172.21.15.119:0/2312023625 wait complete. 2023-12-07T10:11:23.400 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph orch host ls --format=json 2023-12-07T10:11:24.176 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:11:24.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:23 smithi119 ceph-mon[97908]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:24.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:23 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:24.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:23 smithi119 ceph-mon[97908]: Added host smithi153 2023-12-07T10:11:24.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:23 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:11:24.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:23 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:25.456 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:25 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:25.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.700+0000 7f6a5889d700 1 -- 172.21.15.119:0/557276532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a54101170 msgr2=0x7f6a54101550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:25.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.700+0000 7f6a5889d700 1 --2- 172.21.15.119:0/557276532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a54101170 0x7f6a54101550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6a44009910 tx=0x7f6a44009c20 comp rx=0 tx=0).stop 2023-12-07T10:11:25.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.700+0000 7f6a5889d700 1 -- 172.21.15.119:0/557276532 shutdown_connections 2023-12-07T10:11:25.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.700+0000 7f6a5889d700 1 --2- 172.21.15.119:0/557276532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a54101170 0x7f6a54101550 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:25.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.700+0000 7f6a5889d700 1 -- 172.21.15.119:0/557276532 >> 172.21.15.119:0/557276532 conn(0x7f6a540fca40 msgr2=0x7f6a540fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:25.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.700+0000 7f6a5889d700 1 -- 172.21.15.119:0/557276532 shutdown_connections 2023-12-07T10:11:25.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.700+0000 7f6a5889d700 1 -- 172.21.15.119:0/557276532 wait complete. 2023-12-07T10:11:25.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.701+0000 7f6a5889d700 1 Processor -- start 2023-12-07T10:11:25.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.701+0000 7f6a5889d700 1 -- start start 2023-12-07T10:11:25.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.702+0000 7f6a5889d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a54101170 0x7f6a5419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:25.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.702+0000 7f6a5889d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5419ec50 con 0x7f6a54101170 2023-12-07T10:11:25.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.702+0000 7f6a5259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a54101170 0x7f6a5419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:25.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.702+0000 7f6a5259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a54101170 0x7f6a5419e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35622/0 (socket says 172.21.15.119:35622) 2023-12-07T10:11:25.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.702+0000 7f6a5259c700 1 -- 172.21.15.119:0/2629279107 learned_addr learned my addr 172.21.15.119:0/2629279107 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:11:25.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.702+0000 7f6a5259c700 1 -- 172.21.15.119:0/2629279107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a440095f0 con 0x7f6a54101170 2023-12-07T10:11:25.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.702+0000 7f6a5259c700 1 --2- 172.21.15.119:0/2629279107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a54101170 0x7f6a5419e710 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6a44006010 tx=0x7f6a44004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:25.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.703+0000 7f6a437fe700 1 -- 172.21.15.119:0/2629279107 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6a4401d070 con 0x7f6a54101170 2023-12-07T10:11:25.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.703+0000 7f6a437fe700 1 -- 172.21.15.119:0/2629279107 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a4400bc10 con 0x7f6a54101170 2023-12-07T10:11:25.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.703+0000 7f6a5889d700 1 -- 172.21.15.119:0/2629279107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a5419edf0 con 0x7f6a54101170 2023-12-07T10:11:25.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.703+0000 7f6a437fe700 1 -- 172.21.15.119:0/2629279107 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6a440038a0 con 0x7f6a54101170 2023-12-07T10:11:25.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.703+0000 7f6a5889d700 1 -- 172.21.15.119:0/2629279107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5419f210 con 0x7f6a54101170 2023-12-07T10:11:25.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.704+0000 7f6a5889d700 1 -- 172.21.15.119:0/2629279107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a54102650 con 0x7f6a54101170 2023-12-07T10:11:25.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.704+0000 7f6a437fe700 1 -- 172.21.15.119:0/2629279107 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45796+0+0 (secure 0 0 0) 0x7f6a4400b6f0 con 0x7f6a54101170 2023-12-07T10:11:25.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.705+0000 7f6a437fe700 1 --2- 172.21.15.119:0/2629279107 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6a3c038b40 0x7f6a3c03b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:25.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.705+0000 7f6a437fe700 1 -- 172.21.15.119:0/2629279107 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6a4404c850 con 0x7f6a54101170 2023-12-07T10:11:25.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.705+0000 7f6a51d9b700 1 --2- 172.21.15.119:0/2629279107 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6a3c038b40 0x7f6a3c03b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:25.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.706+0000 7f6a51d9b700 1 --2- 172.21.15.119:0/2629279107 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6a3c038b40 0x7f6a3c03b000 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6a48006fd0 tx=0x7f6a48006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:25.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.710+0000 7f6a437fe700 1 -- 172.21.15.119:0/2629279107 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f6a4400bd80 con 0x7f6a54101170 2023-12-07T10:11:25.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.912+0000 7f6a5889d700 1 -- 172.21.15.119:0/2629279107 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6a5410b260 con 0x7f6a3c038b40 2023-12-07T10:11:25.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.914+0000 7f6a437fe700 1 -- 172.21.15.119:0/2629279107 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f6a5410b260 con 0x7f6a3c038b40 2023-12-07T10:11:25.917 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:11:25.917 INFO:teuthology.orchestra.run.smithi119.stdout:[{"addr": "172.21.15.119", "hostname": "smithi119", "labels": [], "status": ""}, {"addr": "172.21.15.153", "hostname": "smithi153", "labels": [], "status": ""}] 2023-12-07T10:11:25.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.916+0000 7f6a5889d700 1 -- 172.21.15.119:0/2629279107 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6a3c038b40 msgr2=0x7f6a3c03b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:25.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.916+0000 7f6a5889d700 1 --2- 172.21.15.119:0/2629279107 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6a3c038b40 0x7f6a3c03b000 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6a48006fd0 tx=0x7f6a48006e40 comp rx=0 tx=0).stop 2023-12-07T10:11:25.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.916+0000 7f6a5889d700 1 -- 172.21.15.119:0/2629279107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a54101170 msgr2=0x7f6a5419e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:25.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.916+0000 7f6a5889d700 1 --2- 172.21.15.119:0/2629279107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a54101170 0x7f6a5419e710 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6a44006010 tx=0x7f6a44004ff0 comp rx=0 tx=0).stop 2023-12-07T10:11:25.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.916+0000 7f6a5889d700 1 -- 172.21.15.119:0/2629279107 shutdown_connections 2023-12-07T10:11:25.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.916+0000 7f6a5889d700 1 --2- 172.21.15.119:0/2629279107 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6a3c038b40 0x7f6a3c03b000 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:25.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.916+0000 7f6a5889d700 1 --2- 172.21.15.119:0/2629279107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a54101170 0x7f6a5419e710 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:25.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.916+0000 7f6a5889d700 1 -- 172.21.15.119:0/2629279107 >> 172.21.15.119:0/2629279107 conn(0x7f6a540fca40 msgr2=0x7f6a54073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:25.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.916+0000 7f6a5889d700 1 -- 172.21.15.119:0/2629279107 shutdown_connections 2023-12-07T10:11:25.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:25.916+0000 7f6a5889d700 1 -- 172.21.15.119:0/2629279107 wait complete. 2023-12-07T10:11:26.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:26 smithi119 ceph-mon[97908]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:26.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:26 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:26.458 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi159 2023-12-07T10:11:26.459 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-12-07T10:11:26.459 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/etc/ceph/ceph.conf 2023-12-07T10:11:26.479 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-12-07T10:11:26.479 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-07T10:11:26.538 INFO:tasks.cephadm:Adding host smithi159 to orchestrator... 2023-12-07T10:11:26.539 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph orch host add smithi159 2023-12-07T10:11:27.305 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:11:27.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:27 smithi119 ceph-mon[97908]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:11:28.135 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:28 smithi119 ceph-mon[97908]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:28.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.852+0000 7f6209af1700 1 -- 172.21.15.119:0/556293082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6204101170 msgr2=0x7f6204101550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:28.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.852+0000 7f6209af1700 1 --2- 172.21.15.119:0/556293082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6204101170 0x7f6204101550 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f61f4009910 tx=0x7f61f4009c20 comp rx=0 tx=0).stop 2023-12-07T10:11:28.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.852+0000 7f6209af1700 1 -- 172.21.15.119:0/556293082 shutdown_connections 2023-12-07T10:11:28.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.852+0000 7f6209af1700 1 --2- 172.21.15.119:0/556293082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6204101170 0x7f6204101550 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:28.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.852+0000 7f6209af1700 1 -- 172.21.15.119:0/556293082 >> 172.21.15.119:0/556293082 conn(0x7f62040fca40 msgr2=0x7f62040fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:28.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.852+0000 7f6209af1700 1 -- 172.21.15.119:0/556293082 shutdown_connections 2023-12-07T10:11:28.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.852+0000 7f6209af1700 1 -- 172.21.15.119:0/556293082 wait complete. 2023-12-07T10:11:28.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.853+0000 7f6209af1700 1 Processor -- start 2023-12-07T10:11:28.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.853+0000 7f6209af1700 1 -- start start 2023-12-07T10:11:28.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.854+0000 7f6209af1700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6204101170 0x7f6204190940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:28.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.854+0000 7f6209af1700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6204194870 con 0x7f6204101170 2023-12-07T10:11:28.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.854+0000 7f62037fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6204101170 0x7f6204190940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:28.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.854+0000 7f62037fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6204101170 0x7f6204190940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35630/0 (socket says 172.21.15.119:35630) 2023-12-07T10:11:28.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.854+0000 7f62037fe700 1 -- 172.21.15.119:0/2774630259 learned_addr learned my addr 172.21.15.119:0/2774630259 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:11:28.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.854+0000 7f62037fe700 1 -- 172.21.15.119:0/2774630259 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f40095f0 con 0x7f6204101170 2023-12-07T10:11:28.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.855+0000 7f62037fe700 1 --2- 172.21.15.119:0/2774630259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6204101170 0x7f6204190940 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f61f4006010 tx=0x7f61f4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:28.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.855+0000 7f6200ff9700 1 -- 172.21.15.119:0/2774630259 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61f401d070 con 0x7f6204101170 2023-12-07T10:11:28.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.855+0000 7f6200ff9700 1 -- 172.21.15.119:0/2774630259 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61f400bc10 con 0x7f6204101170 2023-12-07T10:11:28.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.855+0000 7f6209af1700 1 -- 172.21.15.119:0/2774630259 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6204194a70 con 0x7f6204101170 2023-12-07T10:11:28.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.855+0000 7f6200ff9700 1 -- 172.21.15.119:0/2774630259 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61f40038a0 con 0x7f6204101170 2023-12-07T10:11:28.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.855+0000 7f6209af1700 1 -- 172.21.15.119:0/2774630259 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6204191050 con 0x7f6204101170 2023-12-07T10:11:28.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.856+0000 7f6200ff9700 1 -- 172.21.15.119:0/2774630259 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45796+0+0 (secure 0 0 0) 0x7f61f400b6f0 con 0x7f6204101170 2023-12-07T10:11:28.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.856+0000 7f6209af1700 1 -- 172.21.15.119:0/2774630259 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6204102650 con 0x7f6204101170 2023-12-07T10:11:28.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.856+0000 7f6200ff9700 1 --2- 172.21.15.119:0/2774630259 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f61ec038b40 0x7f61ec03b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:28.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.856+0000 7f6200ff9700 1 -- 172.21.15.119:0/2774630259 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f61f404c830 con 0x7f6204101170 2023-12-07T10:11:28.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.857+0000 7f6202ffd700 1 --2- 172.21.15.119:0/2774630259 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f61ec038b40 0x7f61ec03b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:28.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.857+0000 7f6202ffd700 1 --2- 172.21.15.119:0/2774630259 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f61ec038b40 0x7f61ec03b000 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f61f8006fd0 tx=0x7f61f8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:28.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:28.861+0000 7f6200ff9700 1 -- 172.21.15.119:0/2774630259 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f61f4003a00 con 0x7f6204101170 2023-12-07T10:11:29.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:29.069+0000 7f6209af1700 1 -- 172.21.15.119:0/2774630259 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi159", "target": ["mon-mgr", ""]}) v1 -- 0x7f6204191f90 con 0x7f61ec038b40 2023-12-07T10:11:29.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:29 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:29.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:29 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:29.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:29 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:29.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:29 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:29.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:29 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi153", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:11:29.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:29 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:11:29.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:29 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:11:29.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:29 smithi119 ceph-mon[97908]: Updating smithi153:/etc/ceph/ceph.conf 2023-12-07T10:11:29.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:29 smithi119 ceph-mon[97908]: Updating smithi153:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:11:29.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:29 smithi119 ceph-mon[97908]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:29.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:29 smithi119 ceph-mon[97908]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2023-12-07T10:11:29.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:29 smithi119 ceph-mon[97908]: Updating smithi153:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-07T10:11:29.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:29 smithi119 ceph-mon[97908]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi159", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:11:29.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:29 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:29.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:29 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:29.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:29 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:30.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:30 smithi119 ceph-mon[97908]: Deploying cephadm binary to smithi159 2023-12-07T10:11:31.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:31 smithi119 ceph-mon[97908]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:33.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:33.170+0000 7f6200ff9700 1 -- 172.21.15.119:0/2774630259 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f6204191f90 con 0x7f61ec038b40 2023-12-07T10:11:33.173 INFO:teuthology.orchestra.run.smithi119.stdout:Added host 'smithi159' with addr '172.21.15.159' 2023-12-07T10:11:33.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:33.173+0000 7f6209af1700 1 -- 172.21.15.119:0/2774630259 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f61ec038b40 msgr2=0x7f61ec03b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:33.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:33.173+0000 7f6209af1700 1 --2- 172.21.15.119:0/2774630259 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f61ec038b40 0x7f61ec03b000 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f61f8006fd0 tx=0x7f61f8006e40 comp rx=0 tx=0).stop 2023-12-07T10:11:33.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:33.173+0000 7f6209af1700 1 -- 172.21.15.119:0/2774630259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6204101170 msgr2=0x7f6204190940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:33.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:33.173+0000 7f6209af1700 1 --2- 172.21.15.119:0/2774630259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6204101170 0x7f6204190940 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f61f4006010 tx=0x7f61f4004ff0 comp rx=0 tx=0).stop 2023-12-07T10:11:33.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:33.173+0000 7f6209af1700 1 -- 172.21.15.119:0/2774630259 shutdown_connections 2023-12-07T10:11:33.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:33.173+0000 7f6209af1700 1 --2- 172.21.15.119:0/2774630259 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f61ec038b40 0x7f61ec03b000 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:33.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:33.173+0000 7f6209af1700 1 --2- 172.21.15.119:0/2774630259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6204101170 0x7f6204190940 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:33.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:33.173+0000 7f6209af1700 1 -- 172.21.15.119:0/2774630259 >> 172.21.15.119:0/2774630259 conn(0x7f62040fca40 msgr2=0x7f6204073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:33.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:33.173+0000 7f6209af1700 1 -- 172.21.15.119:0/2774630259 shutdown_connections 2023-12-07T10:11:33.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:33.173+0000 7f6209af1700 1 -- 172.21.15.119:0/2774630259 wait complete. 2023-12-07T10:11:33.604 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph orch host ls --format=json 2023-12-07T10:11:33.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:33 smithi119 ceph-mon[97908]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:33.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:33 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:33.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:33 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:11:33.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:33 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:34.372 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:11:34.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:34 smithi119 ceph-mon[97908]: Added host smithi159 2023-12-07T10:11:34.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:34 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:35.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.873+0000 7f0c1a65d700 1 -- 172.21.15.119:0/2025153126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c14100fa0 msgr2=0x7f0c14101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:35.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.873+0000 7f0c1a65d700 1 --2- 172.21.15.119:0/2025153126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c14100fa0 0x7f0c14101380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009910 tx=0x7f0c10009c20 comp rx=0 tx=0).stop 2023-12-07T10:11:35.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.873+0000 7f0c1a65d700 1 -- 172.21.15.119:0/2025153126 shutdown_connections 2023-12-07T10:11:35.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.873+0000 7f0c1a65d700 1 --2- 172.21.15.119:0/2025153126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c14100fa0 0x7f0c14101380 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:35.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.873+0000 7f0c1a65d700 1 -- 172.21.15.119:0/2025153126 >> 172.21.15.119:0/2025153126 conn(0x7f0c140fc850 msgr2=0x7f0c140fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:35.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.873+0000 7f0c1a65d700 1 -- 172.21.15.119:0/2025153126 shutdown_connections 2023-12-07T10:11:35.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.873+0000 7f0c1a65d700 1 -- 172.21.15.119:0/2025153126 wait complete. 2023-12-07T10:11:35.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.874+0000 7f0c1a65d700 1 Processor -- start 2023-12-07T10:11:35.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.874+0000 7f0c1a65d700 1 -- start start 2023-12-07T10:11:35.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.875+0000 7f0c1a65d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c14100fa0 0x7f0c14073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:35.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.875+0000 7f0c1a65d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c140757d0 con 0x7f0c14100fa0 2023-12-07T10:11:35.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.875+0000 7f0c1965b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c14100fa0 0x7f0c14073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:35.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.875+0000 7f0c1965b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c14100fa0 0x7f0c14073910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58864/0 (socket says 172.21.15.119:58864) 2023-12-07T10:11:35.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.875+0000 7f0c1965b700 1 -- 172.21.15.119:0/1571693437 learned_addr learned my addr 172.21.15.119:0/1571693437 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:11:35.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.875+0000 7f0c1965b700 1 -- 172.21.15.119:0/1571693437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c100095f0 con 0x7f0c14100fa0 2023-12-07T10:11:35.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.876+0000 7f0c1965b700 1 --2- 172.21.15.119:0/1571693437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c14100fa0 0x7f0c14073910 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0c10006010 tx=0x7f0c10004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:35.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.876+0000 7f0c0a7fc700 1 -- 172.21.15.119:0/1571693437 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c1001d070 con 0x7f0c14100fa0 2023-12-07T10:11:35.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.876+0000 7f0c0a7fc700 1 -- 172.21.15.119:0/1571693437 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c1000bc10 con 0x7f0c14100fa0 2023-12-07T10:11:35.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.876+0000 7f0c1a65d700 1 -- 172.21.15.119:0/1571693437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c14073e50 con 0x7f0c14100fa0 2023-12-07T10:11:35.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.876+0000 7f0c0a7fc700 1 -- 172.21.15.119:0/1571693437 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c100038a0 con 0x7f0c14100fa0 2023-12-07T10:11:35.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.876+0000 7f0c1a65d700 1 -- 172.21.15.119:0/1571693437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c14074270 con 0x7f0c14100fa0 2023-12-07T10:11:35.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.877+0000 7f0c0a7fc700 1 -- 172.21.15.119:0/1571693437 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45796+0+0 (secure 0 0 0) 0x7f0c1000b6f0 con 0x7f0c14100fa0 2023-12-07T10:11:35.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.877+0000 7f0c1a65d700 1 -- 172.21.15.119:0/1571693437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c14109000 con 0x7f0c14100fa0 2023-12-07T10:11:35.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.877+0000 7f0c0a7fc700 1 --2- 172.21.15.119:0/1571693437 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0c00038b40 0x7f0c0003b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:35.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.877+0000 7f0c0a7fc700 1 -- 172.21.15.119:0/1571693437 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0c1004c7f0 con 0x7f0c14100fa0 2023-12-07T10:11:35.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.878+0000 7f0c18e5a700 1 --2- 172.21.15.119:0/1571693437 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0c00038b40 0x7f0c0003b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:35.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.878+0000 7f0c18e5a700 1 --2- 172.21.15.119:0/1571693437 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0c00038b40 0x7f0c0003b000 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0c04006fd0 tx=0x7f0c04006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:35.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:35.883+0000 7f0c0a7fc700 1 -- 172.21.15.119:0/1571693437 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0c10003d30 con 0x7f0c14100fa0 2023-12-07T10:11:35.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:35 smithi119 ceph-mon[97908]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:35.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:35 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:36.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:36.088+0000 7f0c1a65d700 1 -- 172.21.15.119:0/1571693437 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c14074f60 con 0x7f0c00038b40 2023-12-07T10:11:36.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:36.089+0000 7f0c0a7fc700 1 -- 172.21.15.119:0/1571693437 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+241 (secure 0 0 0) 0x7f0c14074f60 con 0x7f0c00038b40 2023-12-07T10:11:36.092 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:11:36.092 INFO:teuthology.orchestra.run.smithi119.stdout:[{"addr": "172.21.15.119", "hostname": "smithi119", "labels": [], "status": ""}, {"addr": "172.21.15.153", "hostname": "smithi153", "labels": [], "status": ""}, {"addr": "172.21.15.159", "hostname": "smithi159", "labels": [], "status": ""}] 2023-12-07T10:11:36.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:36.091+0000 7f0c1a65d700 1 -- 172.21.15.119:0/1571693437 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0c00038b40 msgr2=0x7f0c0003b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:36.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:36.091+0000 7f0c1a65d700 1 --2- 172.21.15.119:0/1571693437 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0c00038b40 0x7f0c0003b000 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0c04006fd0 tx=0x7f0c04006e40 comp rx=0 tx=0).stop 2023-12-07T10:11:36.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:36.091+0000 7f0c1a65d700 1 -- 172.21.15.119:0/1571693437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c14100fa0 msgr2=0x7f0c14073910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:36.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:36.091+0000 7f0c1a65d700 1 --2- 172.21.15.119:0/1571693437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c14100fa0 0x7f0c14073910 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0c10006010 tx=0x7f0c10004ff0 comp rx=0 tx=0).stop 2023-12-07T10:11:36.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:36.091+0000 7f0c1a65d700 1 -- 172.21.15.119:0/1571693437 shutdown_connections 2023-12-07T10:11:36.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:36.091+0000 7f0c1a65d700 1 --2- 172.21.15.119:0/1571693437 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0c00038b40 0x7f0c0003b000 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:36.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:36.091+0000 7f0c1a65d700 1 --2- 172.21.15.119:0/1571693437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c14100fa0 0x7f0c14073910 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:36.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:36.091+0000 7f0c1a65d700 1 -- 172.21.15.119:0/1571693437 >> 172.21.15.119:0/1571693437 conn(0x7f0c140fc850 msgr2=0x7f0c14102970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:36.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:36.092+0000 7f0c1a65d700 1 -- 172.21.15.119:0/1571693437 shutdown_connections 2023-12-07T10:11:36.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:36.092+0000 7f0c1a65d700 1 -- 172.21.15.119:0/1571693437 wait complete. 2023-12-07T10:11:36.558 INFO:tasks.cephadm:Setting crush tunables to default 2023-12-07T10:11:36.558 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph osd crush tunables default 2023-12-07T10:11:36.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:36 smithi119 ceph-mon[97908]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:11:37.330 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:11:37.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:37 smithi119 ceph-mon[97908]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:38.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.941+0000 7f0ba689f700 1 -- 172.21.15.119:0/2844895766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ba0073a40 msgr2=0x7f0ba0073e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:38.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.941+0000 7f0ba689f700 1 --2- 172.21.15.119:0/2844895766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ba0073a40 0x7f0ba0073e20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009910 tx=0x7f0b90009c20 comp rx=0 tx=0).stop 2023-12-07T10:11:38.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.941+0000 7f0ba689f700 1 -- 172.21.15.119:0/2844895766 shutdown_connections 2023-12-07T10:11:38.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.941+0000 7f0ba689f700 1 --2- 172.21.15.119:0/2844895766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ba0073a40 0x7f0ba0073e20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:38.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.941+0000 7f0ba689f700 1 -- 172.21.15.119:0/2844895766 >> 172.21.15.119:0/2844895766 conn(0x7f0ba00fca40 msgr2=0x7f0ba00fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:38.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.941+0000 7f0ba689f700 1 -- 172.21.15.119:0/2844895766 shutdown_connections 2023-12-07T10:11:38.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.941+0000 7f0ba689f700 1 -- 172.21.15.119:0/2844895766 wait complete. 2023-12-07T10:11:38.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.943+0000 7f0ba689f700 1 Processor -- start 2023-12-07T10:11:38.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.943+0000 7f0ba689f700 1 -- start start 2023-12-07T10:11:38.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.943+0000 7f0ba689f700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ba0073a40 0x7f0ba0198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:38.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.943+0000 7f0ba689f700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba0198fa0 con 0x7f0ba0073a40 2023-12-07T10:11:38.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.943+0000 7f0b9ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ba0073a40 0x7f0ba0198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:38.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.943+0000 7f0b9ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ba0073a40 0x7f0ba0198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58884/0 (socket says 172.21.15.119:58884) 2023-12-07T10:11:38.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.943+0000 7f0b9ffff700 1 -- 172.21.15.119:0/3463676143 learned_addr learned my addr 172.21.15.119:0/3463676143 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:11:38.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.944+0000 7f0b9ffff700 1 -- 172.21.15.119:0/3463676143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b900095f0 con 0x7f0ba0073a40 2023-12-07T10:11:38.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.944+0000 7f0b9ffff700 1 --2- 172.21.15.119:0/3463676143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ba0073a40 0x7f0ba0198a60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0b90006010 tx=0x7f0b90004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:38.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.944+0000 7f0b9d7fa700 1 -- 172.21.15.119:0/3463676143 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b9001d070 con 0x7f0ba0073a40 2023-12-07T10:11:38.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.944+0000 7f0b9d7fa700 1 -- 172.21.15.119:0/3463676143 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b9000bc10 con 0x7f0ba0073a40 2023-12-07T10:11:38.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.944+0000 7f0b9d7fa700 1 -- 172.21.15.119:0/3463676143 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b900038f0 con 0x7f0ba0073a40 2023-12-07T10:11:38.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.944+0000 7f0ba689f700 1 -- 172.21.15.119:0/3463676143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba01991a0 con 0x7f0ba0073a40 2023-12-07T10:11:38.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.945+0000 7f0ba689f700 1 -- 172.21.15.119:0/3463676143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba01995c0 con 0x7f0ba0073a40 2023-12-07T10:11:38.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.945+0000 7f0b9d7fa700 1 -- 172.21.15.119:0/3463676143 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45796+0+0 (secure 0 0 0) 0x7f0b90003a50 con 0x7f0ba0073a40 2023-12-07T10:11:38.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.946+0000 7f0b86ffd700 1 -- 172.21.15.119:0/3463676143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b800052f0 con 0x7f0ba0073a40 2023-12-07T10:11:38.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.946+0000 7f0b9d7fa700 1 --2- 172.21.15.119:0/3463676143 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0b88038b40 0x7f0b8803b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:38.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.946+0000 7f0b9d7fa700 1 -- 172.21.15.119:0/3463676143 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0b9004c9f0 con 0x7f0ba0073a40 2023-12-07T10:11:38.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.946+0000 7f0b9f7fe700 1 --2- 172.21.15.119:0/3463676143 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0b88038b40 0x7f0b8803b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:38.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.947+0000 7f0b9f7fe700 1 --2- 172.21.15.119:0/3463676143 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0b88038b40 0x7f0b8803b000 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0b94006fd0 tx=0x7f0b94006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:38.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:38.951+0000 7f0b9d7fa700 1 -- 172.21.15.119:0/3463676143 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0b9002aba0 con 0x7f0ba0073a40 2023-12-07T10:11:39.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:39.159+0000 7f0b86ffd700 1 -- 172.21.15.119:0/3463676143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f0b80005160 con 0x7f0ba0073a40 2023-12-07T10:11:39.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:39.497+0000 7f0b9d7fa700 1 -- 172.21.15.119:0/3463676143 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f0b90026020 con 0x7f0ba0073a40 2023-12-07T10:11:39.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:39.500+0000 7f0b86ffd700 1 -- 172.21.15.119:0/3463676143 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0b88038b40 msgr2=0x7f0b8803b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:39.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:39.500+0000 7f0b86ffd700 1 --2- 172.21.15.119:0/3463676143 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0b88038b40 0x7f0b8803b000 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0b94006fd0 tx=0x7f0b94006e40 comp rx=0 tx=0).stop 2023-12-07T10:11:39.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:39.500+0000 7f0b86ffd700 1 -- 172.21.15.119:0/3463676143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ba0073a40 msgr2=0x7f0ba0198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:39.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:39.500+0000 7f0b86ffd700 1 --2- 172.21.15.119:0/3463676143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ba0073a40 0x7f0ba0198a60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0b90006010 tx=0x7f0b90004ff0 comp rx=0 tx=0).stop 2023-12-07T10:11:39.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:39.500+0000 7f0b86ffd700 1 -- 172.21.15.119:0/3463676143 shutdown_connections 2023-12-07T10:11:39.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:39.500+0000 7f0b86ffd700 1 --2- 172.21.15.119:0/3463676143 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0b88038b40 0x7f0b8803b000 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:39.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:39.500+0000 7f0b86ffd700 1 --2- 172.21.15.119:0/3463676143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ba0073a40 0x7f0ba0198a60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:39.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:39.500+0000 7f0b86ffd700 1 -- 172.21.15.119:0/3463676143 >> 172.21.15.119:0/3463676143 conn(0x7f0ba00fca40 msgr2=0x7f0ba01071c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:39.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:39.500+0000 7f0b86ffd700 1 -- 172.21.15.119:0/3463676143 shutdown_connections 2023-12-07T10:11:39.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:11:39.500+0000 7f0b86ffd700 1 -- 172.21.15.119:0/3463676143 wait complete. 2023-12-07T10:11:39.504 INFO:teuthology.orchestra.run.smithi119.stderr:adjusted tunables profile to default 2023-12-07T10:11:39.971 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:39 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:39.971 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:39 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:39.971 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:39 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:39.971 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:39 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:39.971 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:39 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:11:39.972 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:39 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:11:39.972 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:39 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:11:39.972 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:39 smithi119 ceph-mon[97908]: Updating smithi159:/etc/ceph/ceph.conf 2023-12-07T10:11:39.972 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:39 smithi119 ceph-mon[97908]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:39.972 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:39 smithi119 ceph-mon[97908]: Updating smithi159:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:11:39.972 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:39 smithi119 ceph-mon[97908]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2023-12-07T10:11:39.972 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:39 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3463676143' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-12-07T10:11:39.972 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:39 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:39.972 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:39 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:39.972 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:39 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:40.014 INFO:tasks.cephadm:Adding mon.a on smithi119 2023-12-07T10:11:40.014 INFO:tasks.cephadm:Adding mon.b on smithi153 2023-12-07T10:11:40.014 INFO:tasks.cephadm:Adding mon.c on smithi159 2023-12-07T10:11:40.014 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph orch apply mon '3;smithi119:172.21.15.119=a;smithi153:172.21.15.153=b;smithi159:172.21.15.159=c' 2023-12-07T10:11:40.638 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:11:40.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:40 smithi119 ceph-mon[97908]: Updating smithi159:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-07T10:11:40.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:40 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3463676143' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-12-07T10:11:40.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:40 smithi119 ceph-mon[97908]: osdmap e4: 0 total, 0 up, 0 in 2023-12-07T10:11:41.710 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.706+0000 7f2e02906700 1 -- 172.21.15.159:0/170546364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dfc100760 msgr2=0x7f2dfc100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:41.711 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.706+0000 7f2e02906700 1 --2- 172.21.15.159:0/170546364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dfc100760 0x7f2dfc100b40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2de4009910 tx=0x7f2de4009c20 comp rx=0 tx=0).stop 2023-12-07T10:11:41.711 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.706+0000 7f2e02906700 1 -- 172.21.15.159:0/170546364 shutdown_connections 2023-12-07T10:11:41.711 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.706+0000 7f2e02906700 1 --2- 172.21.15.159:0/170546364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dfc100760 0x7f2dfc100b40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:41.711 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.706+0000 7f2e02906700 1 -- 172.21.15.159:0/170546364 >> 172.21.15.159:0/170546364 conn(0x7f2dfc0fb070 msgr2=0x7f2dfc0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:41.711 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.706+0000 7f2e02906700 1 -- 172.21.15.159:0/170546364 shutdown_connections 2023-12-07T10:11:41.711 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.706+0000 7f2e02906700 1 -- 172.21.15.159:0/170546364 wait complete. 2023-12-07T10:11:41.711 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.707+0000 7f2e02906700 1 Processor -- start 2023-12-07T10:11:41.711 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.707+0000 7f2e02906700 1 -- start start 2023-12-07T10:11:41.711 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.707+0000 7f2e02906700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dfc100760 0x7f2dfc1953f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:41.711 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.707+0000 7f2e02906700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc100230 con 0x7f2dfc100760 2023-12-07T10:11:41.711 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.707+0000 7f2dfbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dfc100760 0x7f2dfc1953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:41.711 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.708+0000 7f2dfbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dfc100760 0x7f2dfc1953f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.159:39342/0 (socket says 172.21.15.159:39342) 2023-12-07T10:11:41.712 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.708+0000 7f2dfbfff700 1 -- 172.21.15.159:0/1871097903 learned_addr learned my addr 172.21.15.159:0/1871097903 (peer_addr_for_me v2:172.21.15.159:0/0) 2023-12-07T10:11:41.713 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.708+0000 7f2dfbfff700 1 -- 172.21.15.159:0/1871097903 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de40095f0 con 0x7f2dfc100760 2023-12-07T10:11:41.713 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.708+0000 7f2dfbfff700 1 --2- 172.21.15.159:0/1871097903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dfc100760 0x7f2dfc1953f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2de4006010 tx=0x7f2de4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:41.713 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.708+0000 7f2df97fa700 1 -- 172.21.15.159:0/1871097903 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2de401d070 con 0x7f2dfc100760 2023-12-07T10:11:41.713 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.708+0000 7f2e02906700 1 -- 172.21.15.159:0/1871097903 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dfc195930 con 0x7f2dfc100760 2023-12-07T10:11:41.713 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.708+0000 7f2df97fa700 1 -- 172.21.15.159:0/1871097903 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2de400bc10 con 0x7f2dfc100760 2023-12-07T10:11:41.713 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.708+0000 7f2e02906700 1 -- 172.21.15.159:0/1871097903 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dfc195dd0 con 0x7f2dfc100760 2023-12-07T10:11:41.713 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.708+0000 7f2df97fa700 1 -- 172.21.15.159:0/1871097903 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2de40038a0 con 0x7f2dfc100760 2023-12-07T10:11:41.714 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.709+0000 7f2e02906700 1 -- 172.21.15.159:0/1871097903 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dfc100370 con 0x7f2dfc100760 2023-12-07T10:11:41.714 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.709+0000 7f2df97fa700 1 -- 172.21.15.159:0/1871097903 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45796+0+0 (secure 0 0 0) 0x7f2de400b6f0 con 0x7f2dfc100760 2023-12-07T10:11:41.714 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.709+0000 7f2df97fa700 1 --2- 172.21.15.159:0/1871097903 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2de8038af0 0x7f2de803afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:41.714 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.709+0000 7f2df97fa700 1 -- 172.21.15.159:0/1871097903 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2de404c800 con 0x7f2dfc100760 2023-12-07T10:11:41.714 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.710+0000 7f2dfb7fe700 1 --2- 172.21.15.159:0/1871097903 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2de8038af0 0x7f2de803afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:41.714 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.710+0000 7f2dfb7fe700 1 --2- 172.21.15.159:0/1871097903 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2de8038af0 0x7f2de803afb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2dec006fd0 tx=0x7f2dec006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:41.718 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.715+0000 7f2df97fa700 1 -- 172.21.15.159:0/1871097903 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2de4003ad0 con 0x7f2dfc100760 2023-12-07T10:11:41.915 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.911+0000 7f2e02906700 1 -- 172.21.15.159:0/1871097903 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi119:172.21.15.119=a;smithi153:172.21.15.153=b;smithi159:172.21.15.159=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f2dfc196080 con 0x7f2de8038af0 2023-12-07T10:11:41.921 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.919+0000 7f2df97fa700 1 -- 172.21.15.159:0/1871097903 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f2dfc196080 con 0x7f2de8038af0 2023-12-07T10:11:41.922 INFO:teuthology.orchestra.run.smithi159.stdout:Scheduled mon update... 2023-12-07T10:11:41.923 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.920+0000 7f2e02906700 1 -- 172.21.15.159:0/1871097903 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2de8038af0 msgr2=0x7f2de803afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:41.924 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.920+0000 7f2e02906700 1 --2- 172.21.15.159:0/1871097903 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2de8038af0 0x7f2de803afb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2dec006fd0 tx=0x7f2dec006e40 comp rx=0 tx=0).stop 2023-12-07T10:11:41.924 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.920+0000 7f2e02906700 1 -- 172.21.15.159:0/1871097903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dfc100760 msgr2=0x7f2dfc1953f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:41.924 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.920+0000 7f2e02906700 1 --2- 172.21.15.159:0/1871097903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dfc100760 0x7f2dfc1953f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2de4006010 tx=0x7f2de4004ff0 comp rx=0 tx=0).stop 2023-12-07T10:11:41.924 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.920+0000 7f2e02906700 1 -- 172.21.15.159:0/1871097903 shutdown_connections 2023-12-07T10:11:41.924 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.920+0000 7f2e02906700 1 --2- 172.21.15.159:0/1871097903 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2de8038af0 0x7f2de803afb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:41.924 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.920+0000 7f2e02906700 1 --2- 172.21.15.159:0/1871097903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dfc100760 0x7f2dfc1953f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:41.924 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.920+0000 7f2e02906700 1 -- 172.21.15.159:0/1871097903 >> 172.21.15.159:0/1871097903 conn(0x7f2dfc0fb070 msgr2=0x7f2dfc101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:41.924 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.920+0000 7f2e02906700 1 -- 172.21.15.159:0/1871097903 shutdown_connections 2023-12-07T10:11:41.924 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:41.921+0000 7f2e02906700 1 -- 172.21.15.159:0/1871097903 wait complete. 2023-12-07T10:11:41.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:41 smithi119 ceph-mon[97908]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:42.319 DEBUG:teuthology.orchestra.run.smithi153:mon.b> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mon.b.service 2023-12-07T10:11:42.322 DEBUG:teuthology.orchestra.run.smithi159:mon.c> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mon.c.service 2023-12-07T10:11:42.324 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-07T10:11:42.324 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-07T10:11:42.354 INFO:journalctl@ceph.mon.c.smithi159.stdout:-- Logs begin at Thu 2023-12-07 09:49:21 UTC. -- 2023-12-07T10:11:42.357 INFO:journalctl@ceph.mon.b.smithi153.stdout:-- Logs begin at Thu 2023-12-07 09:49:13 UTC. -- 2023-12-07T10:11:43.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:42 smithi119 ceph-mon[97908]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi119:172.21.15.119=a;smithi153:172.21.15.153=b;smithi159:172.21.15.159=c", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:11:43.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:42 smithi119 ceph-mon[97908]: Saving service mon spec with placement smithi119:172.21.15.119=a;smithi153:172.21.15.153=b;smithi159:172.21.15.159=c;count:3 2023-12-07T10:11:43.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:42 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:43.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:42 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:11:43.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:42 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:11:43.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:42 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:11:43.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:42 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:43.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:42 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-07T10:11:43.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:42 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:11:43.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:42 smithi119 ceph-mon[97908]: Deploying daemon mon.c on smithi159 2023-12-07T10:11:43.307 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:11:44.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:43 smithi119 ceph-mon[97908]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:44.842 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.838+0000 7fa561dc2700 1 -- 172.21.15.159:0/541421794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa55c100080 msgr2=0x7fa55c100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:44.842 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.838+0000 7fa561dc2700 1 --2- 172.21.15.159:0/541421794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa55c100080 0x7fa55c100460 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa54c009910 tx=0x7fa54c009c20 comp rx=0 tx=0).stop 2023-12-07T10:11:44.843 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.839+0000 7fa561dc2700 1 -- 172.21.15.159:0/541421794 shutdown_connections 2023-12-07T10:11:44.843 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.839+0000 7fa561dc2700 1 --2- 172.21.15.159:0/541421794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa55c100080 0x7fa55c100460 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:44.843 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.839+0000 7fa561dc2700 1 -- 172.21.15.159:0/541421794 >> 172.21.15.159:0/541421794 conn(0x7fa55c076a80 msgr2=0x7fa55c0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:44.849 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.845+0000 7fa561dc2700 1 -- 172.21.15.159:0/541421794 shutdown_connections 2023-12-07T10:11:44.849 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.845+0000 7fa561dc2700 1 -- 172.21.15.159:0/541421794 wait complete. 2023-12-07T10:11:44.849 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.846+0000 7fa561dc2700 1 Processor -- start 2023-12-07T10:11:44.851 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.847+0000 7fa561dc2700 1 -- start start 2023-12-07T10:11:44.851 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.848+0000 7fa561dc2700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa55c100080 0x7fa55c072c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:44.851 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.848+0000 7fa561dc2700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c0731b0 con 0x7fa55c100080 2023-12-07T10:11:44.851 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.848+0000 7fa55b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa55c100080 0x7fa55c072c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:44.851 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.848+0000 7fa55b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa55c100080 0x7fa55c072c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.159:39348/0 (socket says 172.21.15.159:39348) 2023-12-07T10:11:44.851 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.848+0000 7fa55b7fe700 1 -- 172.21.15.159:0/592520202 learned_addr learned my addr 172.21.15.159:0/592520202 (peer_addr_for_me v2:172.21.15.159:0/0) 2023-12-07T10:11:44.852 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.848+0000 7fa55b7fe700 1 -- 172.21.15.159:0/592520202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54c0095f0 con 0x7fa55c100080 2023-12-07T10:11:44.852 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.849+0000 7fa55b7fe700 1 --2- 172.21.15.159:0/592520202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa55c100080 0x7fa55c072c70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa54c004f40 tx=0x7fa54c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:44.853 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.850+0000 7fa558ff9700 1 -- 172.21.15.159:0/592520202 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa54c01d070 con 0x7fa55c100080 2023-12-07T10:11:44.853 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.850+0000 7fa558ff9700 1 -- 172.21.15.159:0/592520202 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa54c00bc10 con 0x7fa55c100080 2023-12-07T10:11:44.853 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.850+0000 7fa561dc2700 1 -- 172.21.15.159:0/592520202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa55c0733b0 con 0x7fa55c100080 2023-12-07T10:11:44.853 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.850+0000 7fa558ff9700 1 -- 172.21.15.159:0/592520202 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa54c0037d0 con 0x7fa55c100080 2023-12-07T10:11:44.853 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.850+0000 7fa561dc2700 1 -- 172.21.15.159:0/592520202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa55c070db0 con 0x7fa55c100080 2023-12-07T10:11:44.854 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.851+0000 7fa558ff9700 1 -- 172.21.15.159:0/592520202 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45796+0+0 (secure 0 0 0) 0x7fa54c003990 con 0x7fa55c100080 2023-12-07T10:11:44.854 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.851+0000 7fa561dc2700 1 -- 172.21.15.159:0/592520202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa55c101560 con 0x7fa55c100080 2023-12-07T10:11:44.854 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.851+0000 7fa558ff9700 1 --2- 172.21.15.159:0/592520202 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa544038b90 0x7fa54403b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:44.855 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.851+0000 7fa558ff9700 1 -- 172.21.15.159:0/592520202 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa54c04c830 con 0x7fa55c100080 2023-12-07T10:11:44.855 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.852+0000 7fa55affd700 1 --2- 172.21.15.159:0/592520202 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa544038b90 0x7fa54403b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:44.855 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.852+0000 7fa55affd700 1 --2- 172.21.15.159:0/592520202 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa544038b90 0x7fa54403b050 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa550006fd0 tx=0x7fa550006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:44.859 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:44.856+0000 7fa558ff9700 1 -- 172.21.15.159:0/592520202 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa54c02aa00 con 0x7fa55c100080 2023-12-07T10:11:45.186 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:45.181+0000 7fa561dc2700 1 -- 172.21.15.159:0/592520202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa55c062c40 con 0x7fa55c100080 2023-12-07T10:11:45.186 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:45.182+0000 7fa558ff9700 1 -- 172.21.15.159:0/592520202 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7fa54c026070 con 0x7fa55c100080 2023-12-07T10:11:45.186 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-12-07T10:11:45.186 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":1,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","modified":"2023-12-07T10:09:56.560924Z","created":"2023-12-07T10:09:56.560924Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-07T10:11:45.187 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:45.184+0000 7fa561dc2700 1 -- 172.21.15.159:0/592520202 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa544038b90 msgr2=0x7fa54403b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:45.188 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:45.184+0000 7fa561dc2700 1 --2- 172.21.15.159:0/592520202 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa544038b90 0x7fa54403b050 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa550006fd0 tx=0x7fa550006e40 comp rx=0 tx=0).stop 2023-12-07T10:11:45.188 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:45.184+0000 7fa561dc2700 1 -- 172.21.15.159:0/592520202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa55c100080 msgr2=0x7fa55c072c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:45.188 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:45.184+0000 7fa561dc2700 1 --2- 172.21.15.159:0/592520202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa55c100080 0x7fa55c072c70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa54c004f40 tx=0x7fa54c005020 comp rx=0 tx=0).stop 2023-12-07T10:11:45.188 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:45.184+0000 7fa561dc2700 1 -- 172.21.15.159:0/592520202 shutdown_connections 2023-12-07T10:11:45.188 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:45.184+0000 7fa561dc2700 1 --2- 172.21.15.159:0/592520202 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa544038b90 0x7fa54403b050 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:45.188 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:45.184+0000 7fa561dc2700 1 --2- 172.21.15.159:0/592520202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa55c100080 0x7fa55c072c70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:45.188 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:45.184+0000 7fa561dc2700 1 -- 172.21.15.159:0/592520202 >> 172.21.15.159:0/592520202 conn(0x7fa55c076a80 msgr2=0x7fa55c105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:45.188 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:45.184+0000 7fa561dc2700 1 -- 172.21.15.159:0/592520202 shutdown_connections 2023-12-07T10:11:45.188 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:45.184+0000 7fa561dc2700 1 -- 172.21.15.159:0/592520202 wait complete. 2023-12-07T10:11:45.188 INFO:teuthology.orchestra.run.smithi159.stderr:dumped monmap epoch 1 2023-12-07T10:11:45.209 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:44 smithi159 systemd[1]: Starting Ceph mon.c for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:11:45.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 podman[134663]: 2023-12-07T10:11:46.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 bash[134663]: 472f244eb51fb460c1f9815e370d0b95fc1c94ac6bc1546825fd14d507836a07 2023-12-07T10:11:46.016 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: set uid:gid to 167:167 (ceph:ceph) 2023-12-07T10:11:46.016 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: ceph version 19.0.0-18-g3bd0bfa4 (3bd0bfa458b4399e7266c56d8215a2cf3a6091a0) reef (dev), process ceph-mon, pid 7 2023-12-07T10:11:46.016 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: pidfile_write: ignore empty --pid-file 2023-12-07T10:11:46.016 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: load: jerasure load: lrc 2023-12-07T10:11:46.016 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: RocksDB version: 7.9.2 2023-12-07T10:11:46.016 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Git sha 0 2023-12-07T10:11:46.016 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Compile date 2023-12-07 06:56:28 2023-12-07T10:11:46.016 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: DB SUMMARY 2023-12-07T10:11:46.016 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: DB Session ID: 5DYL69LJP07DQNKUYIG5 2023-12-07T10:11:46.017 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: CURRENT file: CURRENT 2023-12-07T10:11:46.017 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: IDENTITY file: IDENTITY 2023-12-07T10:11:46.017 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-12-07T10:11:46.017 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-12-07T10:11:46.017 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-12-07T10:11:46.017 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.error_if_exists: 0 2023-12-07T10:11:46.017 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.create_if_missing: 0 2023-12-07T10:11:46.017 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.paranoid_checks: 1 2023-12-07T10:11:46.017 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.flush_verify_memtable_count: 1 2023-12-07T10:11:46.017 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-12-07T10:11:46.017 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-12-07T10:11:46.017 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.env: 0x559382290a60 2023-12-07T10:11:46.017 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.fs: PosixFileSystem 2023-12-07T10:11:46.018 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.info_log: 0x559384391740 2023-12-07T10:11:46.018 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.max_file_opening_threads: 16 2023-12-07T10:11:46.018 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.statistics: (nil) 2023-12-07T10:11:46.018 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.use_fsync: 0 2023-12-07T10:11:46.018 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.max_log_file_size: 0 2023-12-07T10:11:46.018 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-07T10:11:46.018 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.log_file_time_to_roll: 0 2023-12-07T10:11:46.018 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.keep_log_file_num: 1000 2023-12-07T10:11:46.018 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.recycle_log_file_num: 0 2023-12-07T10:11:46.018 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.allow_fallocate: 1 2023-12-07T10:11:46.018 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.allow_mmap_reads: 0 2023-12-07T10:11:46.019 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.allow_mmap_writes: 0 2023-12-07T10:11:46.019 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.use_direct_reads: 0 2023-12-07T10:11:46.019 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-07T10:11:46.019 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.create_missing_column_families: 0 2023-12-07T10:11:46.019 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.db_log_dir: 2023-12-07T10:11:46.019 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.wal_dir: 2023-12-07T10:11:46.019 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.table_cache_numshardbits: 6 2023-12-07T10:11:46.019 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.WAL_ttl_seconds: 0 2023-12-07T10:11:46.019 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.WAL_size_limit_MB: 0 2023-12-07T10:11:46.019 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-07T10:11:46.019 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-07T10:11:46.019 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.is_fd_close_on_exec: 1 2023-12-07T10:11:46.019 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.advise_random_on_open: 1 2023-12-07T10:11:46.019 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.db_write_buffer_size: 0 2023-12-07T10:11:46.020 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.write_buffer_manager: 0x559384347360 2023-12-07T10:11:46.020 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-07T10:11:46.020 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-07T10:11:46.020 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.use_adaptive_mutex: 0 2023-12-07T10:11:46.020 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.rate_limiter: (nil) 2023-12-07T10:11:46.020 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-07T10:11:46.020 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.wal_recovery_mode: 2 2023-12-07T10:11:46.020 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.enable_thread_tracking: 0 2023-12-07T10:11:46.020 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.enable_pipelined_write: 0 2023-12-07T10:11:46.020 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.unordered_write: 0 2023-12-07T10:11:46.020 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-07T10:11:46.020 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-07T10:11:46.020 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-07T10:11:46.020 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-07T10:11:46.021 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.row_cache: None 2023-12-07T10:11:46.021 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.wal_filter: None 2023-12-07T10:11:46.021 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-07T10:11:46.021 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.allow_ingest_behind: 0 2023-12-07T10:11:46.021 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.two_write_queues: 0 2023-12-07T10:11:46.021 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.manual_wal_flush: 0 2023-12-07T10:11:46.021 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.wal_compression: 0 2023-12-07T10:11:46.021 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.atomic_flush: 0 2023-12-07T10:11:46.021 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-07T10:11:46.021 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.persist_stats_to_disk: 0 2023-12-07T10:11:46.021 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.write_dbid_to_manifest: 0 2023-12-07T10:11:46.021 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.log_readahead_size: 0 2023-12-07T10:11:46.021 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-12-07T10:11:46.021 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.best_efforts_recovery: 0 2023-12-07T10:11:46.022 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-12-07T10:11:46.022 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-12-07T10:11:46.022 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.allow_data_in_errors: 0 2023-12-07T10:11:46.022 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.db_host_id: __hostname__ 2023-12-07T10:11:46.022 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.enforce_single_del_contracts: true 2023-12-07T10:11:46.022 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.max_background_jobs: 2 2023-12-07T10:11:46.022 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.max_background_compactions: -1 2023-12-07T10:11:46.022 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.max_subcompactions: 1 2023-12-07T10:11:46.022 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-07T10:11:46.022 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-07T10:11:46.022 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.delayed_write_rate : 16777216 2023-12-07T10:11:46.023 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.max_total_wal_size: 0 2023-12-07T10:11:46.023 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-07T10:11:46.023 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.stats_dump_period_sec: 600 2023-12-07T10:11:46.023 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.stats_persist_period_sec: 600 2023-12-07T10:11:46.023 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-07T10:11:46.023 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.max_open_files: -1 2023-12-07T10:11:46.023 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.bytes_per_sync: 0 2023-12-07T10:11:46.023 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.wal_bytes_per_sync: 0 2023-12-07T10:11:46.023 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.strict_bytes_per_sync: 0 2023-12-07T10:11:46.023 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.compaction_readahead_size: 0 2023-12-07T10:11:46.023 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.max_background_flushes: -1 2023-12-07T10:11:46.023 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Compression algorithms supported: 2023-12-07T10:11:46.023 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-07T10:11:46.023 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: kZSTD supported: 0 2023-12-07T10:11:46.023 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: kXpressCompression supported: 0 2023-12-07T10:11:46.024 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: kLZ4HCCompression supported: 1 2023-12-07T10:11:46.024 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: kZlibCompression supported: 1 2023-12-07T10:11:46.024 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: kSnappyCompression supported: 1 2023-12-07T10:11:46.024 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: kLZ4Compression supported: 1 2023-12-07T10:11:46.024 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: kBZip2Compression supported: 0 2023-12-07T10:11:46.024 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Fast CRC32 supported: Supported on x86 2023-12-07T10:11:46.024 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: DMutex implementation: pthread_mutex_t 2023-12-07T10:11:46.024 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-12-07T10:11:46.024 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-12-07T10:11:46.024 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-07T10:11:46.024 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.merge_operator: 2023-12-07T10:11:46.024 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.compaction_filter: None 2023-12-07T10:11:46.024 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.compaction_filter_factory: None 2023-12-07T10:11:46.024 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.sst_partitioner_factory: None 2023-12-07T10:11:46.025 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.memtable_factory: SkipListFactory 2023-12-07T10:11:46.025 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.table_factory: BlockBasedTable 2023-12-07T10:11:46.025 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559384391860) 2023-12-07T10:11:46.025 INFO:journalctl@ceph.mon.c.smithi159.stdout: cache_index_and_filter_blocks: 1 2023-12-07T10:11:46.025 INFO:journalctl@ceph.mon.c.smithi159.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-12-07T10:11:46.025 INFO:journalctl@ceph.mon.c.smithi159.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-07T10:11:46.025 INFO:journalctl@ceph.mon.c.smithi159.stdout: pin_top_level_index_and_filter: 1 2023-12-07T10:11:46.025 INFO:journalctl@ceph.mon.c.smithi159.stdout: index_type: 0 2023-12-07T10:11:46.025 INFO:journalctl@ceph.mon.c.smithi159.stdout: data_block_index_type: 0 2023-12-07T10:11:46.025 INFO:journalctl@ceph.mon.c.smithi159.stdout: index_shortening: 1 2023-12-07T10:11:46.025 INFO:journalctl@ceph.mon.c.smithi159.stdout: data_block_hash_table_util_ratio: 0.750000 2023-12-07T10:11:46.025 INFO:journalctl@ceph.mon.c.smithi159.stdout: checksum: 4 2023-12-07T10:11:46.025 INFO:journalctl@ceph.mon.c.smithi159.stdout: no_block_cache: 0 2023-12-07T10:11:46.025 INFO:journalctl@ceph.mon.c.smithi159.stdout: block_cache: 0x5593843e7090 2023-12-07T10:11:46.025 INFO:journalctl@ceph.mon.c.smithi159.stdout: block_cache_name: BinnedLRUCache 2023-12-07T10:11:46.026 INFO:journalctl@ceph.mon.c.smithi159.stdout: block_cache_options: 2023-12-07T10:11:46.026 INFO:journalctl@ceph.mon.c.smithi159.stdout: capacity : 536870912 2023-12-07T10:11:46.026 INFO:journalctl@ceph.mon.c.smithi159.stdout: num_shard_bits : 4 2023-12-07T10:11:46.026 INFO:journalctl@ceph.mon.c.smithi159.stdout: strict_capacity_limit : 0 2023-12-07T10:11:46.026 INFO:journalctl@ceph.mon.c.smithi159.stdout: high_pri_pool_ratio: 0.000 2023-12-07T10:11:46.026 INFO:journalctl@ceph.mon.c.smithi159.stdout: block_cache_compressed: (nil) 2023-12-07T10:11:46.026 INFO:journalctl@ceph.mon.c.smithi159.stdout: persistent_cache: (nil) 2023-12-07T10:11:46.026 INFO:journalctl@ceph.mon.c.smithi159.stdout: block_size: 4096 2023-12-07T10:11:46.026 INFO:journalctl@ceph.mon.c.smithi159.stdout: block_size_deviation: 10 2023-12-07T10:11:46.026 INFO:journalctl@ceph.mon.c.smithi159.stdout: block_restart_interval: 16 2023-12-07T10:11:46.026 INFO:journalctl@ceph.mon.c.smithi159.stdout: index_block_restart_interval: 1 2023-12-07T10:11:46.026 INFO:journalctl@ceph.mon.c.smithi159.stdout: metadata_block_size: 4096 2023-12-07T10:11:46.026 INFO:journalctl@ceph.mon.c.smithi159.stdout: partition_filters: 0 2023-12-07T10:11:46.027 INFO:journalctl@ceph.mon.c.smithi159.stdout: use_delta_encoding: 1 2023-12-07T10:11:46.027 INFO:journalctl@ceph.mon.c.smithi159.stdout: filter_policy: bloomfilter 2023-12-07T10:11:46.027 INFO:journalctl@ceph.mon.c.smithi159.stdout: whole_key_filtering: 1 2023-12-07T10:11:46.027 INFO:journalctl@ceph.mon.c.smithi159.stdout: verify_compression: 0 2023-12-07T10:11:46.027 INFO:journalctl@ceph.mon.c.smithi159.stdout: read_amp_bytes_per_bit: 0 2023-12-07T10:11:46.027 INFO:journalctl@ceph.mon.c.smithi159.stdout: format_version: 5 2023-12-07T10:11:46.027 INFO:journalctl@ceph.mon.c.smithi159.stdout: enable_index_compression: 1 2023-12-07T10:11:46.027 INFO:journalctl@ceph.mon.c.smithi159.stdout: block_align: 0 2023-12-07T10:11:46.027 INFO:journalctl@ceph.mon.c.smithi159.stdout: max_auto_readahead_size: 262144 2023-12-07T10:11:46.027 INFO:journalctl@ceph.mon.c.smithi159.stdout: prepopulate_block_cache: 0 2023-12-07T10:11:46.027 INFO:journalctl@ceph.mon.c.smithi159.stdout: initial_auto_readahead_size: 8192 2023-12-07T10:11:46.027 INFO:journalctl@ceph.mon.c.smithi159.stdout: num_file_reads_for_auto_readahead: 2 2023-12-07T10:11:46.027 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.write_buffer_size: 33554432 2023-12-07T10:11:46.028 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.max_write_buffer_number: 2 2023-12-07T10:11:46.028 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.compression: NoCompression 2023-12-07T10:11:46.028 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.bottommost_compression: Disabled 2023-12-07T10:11:46.028 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.prefix_extractor: nullptr 2023-12-07T10:11:46.028 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-07T10:11:46.028 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.num_levels: 7 2023-12-07T10:11:46.028 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-07T10:11:46.028 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-07T10:11:46.028 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-07T10:11:46.028 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-07T10:11:46.028 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-07T10:11:46.028 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-07T10:11:46.028 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-07T10:11:46.029 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-07T10:11:46.029 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-12-07T10:11:46.029 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-07T10:11:46.029 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-12-07T10:11:46.029 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-12-07T10:11:46.029 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.compression_opts.window_bits: -14 2023-12-07T10:11:46.029 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.compression_opts.level: 32767 2023-12-07T10:11:46.029 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.compression_opts.strategy: 0 2023-12-07T10:11:46.029 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-07T10:11:46.029 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-07T10:11:46.029 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-12-07T10:11:46.029 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-12-07T10:11:46.029 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.compression_opts.enabled: false 2023-12-07T10:11:46.029 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-12-07T10:11:46.029 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-07T10:11:46.030 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-07T10:11:46.030 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-07T10:11:46.030 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.target_file_size_base: 67108864 2023-12-07T10:11:46.030 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.target_file_size_multiplier: 1 2023-12-07T10:11:46.030 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-07T10:11:46.030 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-07T10:11:46.030 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-07T10:11:46.030 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-07T10:11:46.030 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-07T10:11:46.030 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-07T10:11:46.030 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-07T10:11:46.031 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-07T10:11:46.031 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-07T10:11:46.031 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-07T10:11:46.031 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-07T10:11:46.031 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-07T10:11:46.031 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-12-07T10:11:46.031 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.arena_block_size: 1048576 2023-12-07T10:11:46.031 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-07T10:11:46.031 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-07T10:11:46.031 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.disable_auto_compactions: 0 2023-12-07T10:11:46.031 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-07T10:11:46.032 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-07T10:11:46.033 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-07T10:11:46.033 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-07T10:11:46.033 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-07T10:11:46.033 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-07T10:11:46.033 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-07T10:11:46.033 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-07T10:11:46.033 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-07T10:11:46.033 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-07T10:11:46.033 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.table_properties_collectors: 2023-12-07T10:11:46.033 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.inplace_update_support: 0 2023-12-07T10:11:46.033 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.inplace_update_num_locks: 10000 2023-12-07T10:11:46.033 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-07T10:11:46.033 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-07T10:11:46.033 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.memtable_huge_page_size: 0 2023-12-07T10:11:46.033 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.bloom_locality: 0 2023-12-07T10:11:46.034 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.max_successive_merges: 0 2023-12-07T10:11:46.034 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.optimize_filters_for_hits: 0 2023-12-07T10:11:46.034 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.paranoid_file_checks: 0 2023-12-07T10:11:46.034 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.force_consistency_checks: 1 2023-12-07T10:11:46.034 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.report_bg_io_stats: 0 2023-12-07T10:11:46.034 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.ttl: 2592000 2023-12-07T10:11:46.034 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.periodic_compaction_seconds: 0 2023-12-07T10:11:46.034 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-12-07T10:11:46.034 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-12-07T10:11:46.034 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.enable_blob_files: false 2023-12-07T10:11:46.034 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.min_blob_size: 0 2023-12-07T10:11:46.034 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.blob_file_size: 268435456 2023-12-07T10:11:46.034 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.blob_compression_type: NoCompression 2023-12-07T10:11:46.034 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.enable_blob_garbage_collection: false 2023-12-07T10:11:46.035 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-12-07T10:11:46.035 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-12-07T10:11:46.035 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-12-07T10:11:46.035 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.blob_file_starting_level: 0 2023-12-07T10:11:46.035 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-12-07T10:11:46.035 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-12-07T10:11:46.035 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-12-07T10:11:46.035 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c24afd36-c84c-49f0-896b-c98398d67e9d 2023-12-07T10:11:46.035 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701943905690872, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-12-07T10:11:46.035 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-12-07T10:11:46.035 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701943905691299, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1701943905, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c24afd36-c84c-49f0-896b-c98398d67e9d", "db_session_id": "5DYL69LJP07DQNKUYIG5", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-12-07T10:11:46.035 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701943905691369, "job": 1, "event": "recovery_finished"} 2023-12-07T10:11:46.035 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-12-07T10:11:46.035 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-12-07T10:11:46.036 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559384482000 2023-12-07T10:11:46.036 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: DB pointer 0x55938446e000 2023-12-07T10:11:46.036 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-12-07T10:11:46.036 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-12-07T10:11:46.036 INFO:journalctl@ceph.mon.c.smithi159.stdout: ** DB Stats ** 2023-12-07T10:11:46.036 INFO:journalctl@ceph.mon.c.smithi159.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-07T10:11:46.036 INFO:journalctl@ceph.mon.c.smithi159.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-07T10:11:46.036 INFO:journalctl@ceph.mon.c.smithi159.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-07T10:11:46.036 INFO:journalctl@ceph.mon.c.smithi159.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-07T10:11:46.036 INFO:journalctl@ceph.mon.c.smithi159.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-07T10:11:46.036 INFO:journalctl@ceph.mon.c.smithi159.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-07T10:11:46.036 INFO:journalctl@ceph.mon.c.smithi159.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-07T10:11:46.037 INFO:journalctl@ceph.mon.c.smithi159.stdout: 2023-12-07T10:11:46.037 INFO:journalctl@ceph.mon.c.smithi159.stdout: ** Compaction Stats [default] ** 2023-12-07T10:11:46.037 INFO:journalctl@ceph.mon.c.smithi159.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-07T10:11:46.037 INFO:journalctl@ceph.mon.c.smithi159.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-12-07T10:11:46.037 INFO:journalctl@ceph.mon.c.smithi159.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-07T10:11:46.037 INFO:journalctl@ceph.mon.c.smithi159.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-07T10:11:46.038 INFO:journalctl@ceph.mon.c.smithi159.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-07T10:11:46.038 INFO:journalctl@ceph.mon.c.smithi159.stdout: 2023-12-07T10:11:46.038 INFO:journalctl@ceph.mon.c.smithi159.stdout: ** Compaction Stats [default] ** 2023-12-07T10:11:46.038 INFO:journalctl@ceph.mon.c.smithi159.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-07T10:11:46.038 INFO:journalctl@ceph.mon.c.smithi159.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-07T10:11:46.038 INFO:journalctl@ceph.mon.c.smithi159.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-07T10:11:46.038 INFO:journalctl@ceph.mon.c.smithi159.stdout: 2023-12-07T10:11:46.038 INFO:journalctl@ceph.mon.c.smithi159.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-12-07T10:11:46.039 INFO:journalctl@ceph.mon.c.smithi159.stdout: 2023-12-07T10:11:46.039 INFO:journalctl@ceph.mon.c.smithi159.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-07T10:11:46.039 INFO:journalctl@ceph.mon.c.smithi159.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-12-07T10:11:46.039 INFO:journalctl@ceph.mon.c.smithi159.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-07T10:11:46.039 INFO:journalctl@ceph.mon.c.smithi159.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-12-07T10:11:46.039 INFO:journalctl@ceph.mon.c.smithi159.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-12-07T10:11:46.039 INFO:journalctl@ceph.mon.c.smithi159.stdout: AddFile(Keys): cumulative 0, interval 0 2023-12-07T10:11:46.040 INFO:journalctl@ceph.mon.c.smithi159.stdout: Cumulative compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-07T10:11:46.040 INFO:journalctl@ceph.mon.c.smithi159.stdout: Interval compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-07T10:11:46.040 INFO:journalctl@ceph.mon.c.smithi159.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-07T10:11:46.040 INFO:journalctl@ceph.mon.c.smithi159.stdout: Block cache BinnedLRUCache@0x5593843e7090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2023-12-07T10:11:46.040 INFO:journalctl@ceph.mon.c.smithi159.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-12-07T10:11:46.040 INFO:journalctl@ceph.mon.c.smithi159.stdout: 2023-12-07T10:11:46.040 INFO:journalctl@ceph.mon.c.smithi159.stdout: ** File Read Latency Histogram By Level [default] ** 2023-12-07T10:11:46.040 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-12-07T10:11:46.040 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: using public_addr v2:172.21.15.159:0/0 -> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] 2023-12-07T10:11:46.040 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: starting mon.c rank -1 at public addrs [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] at bind addrs [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:11:46.040 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: mon.c@-1(???) e0 preinit fsid a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:11:46.040 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: mon.c@-1(synchronizing).mds e1 new map 2023-12-07T10:11:46.040 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: mon.c@-1(synchronizing).mds e1 print_map 2023-12-07T10:11:46.040 INFO:journalctl@ceph.mon.c.smithi159.stdout: e1 2023-12-07T10:11:46.041 INFO:journalctl@ceph.mon.c.smithi159.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-12-07T10:11:46.041 INFO:journalctl@ceph.mon.c.smithi159.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-12-07T10:11:46.041 INFO:journalctl@ceph.mon.c.smithi159.stdout: legacy client fscid: -1 2023-12-07T10:11:46.041 INFO:journalctl@ceph.mon.c.smithi159.stdout: 2023-12-07T10:11:46.041 INFO:journalctl@ceph.mon.c.smithi159.stdout: No filesystems configured 2023-12-07T10:11:46.041 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-12-07T10:11:46.041 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-12-07T10:11:46.041 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-12-07T10:11:46.041 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-12-07T10:11:46.041 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-12-07T10:11:46.041 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-12-07T10:11:46.041 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-12-07T10:11:46.042 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-07T10:11:46.042 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-07T10:11:46.042 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-07T10:11:46.042 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:46.042 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:46.042 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:46.042 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:46.042 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:46.042 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi153", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:11:46.042 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:11:46.042 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:11:46.042 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: Updating smithi153:/etc/ceph/ceph.conf 2023-12-07T10:11:46.042 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: Updating smithi153:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:11:46.042 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:46.043 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2023-12-07T10:11:46.043 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: Updating smithi153:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-07T10:11:46.043 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi159", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:11:46.043 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:46.043 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:46.043 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:46.043 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: Deploying cephadm binary to smithi159 2023-12-07T10:11:46.043 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:46.043 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:46.043 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:46.043 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:11:46.043 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:46.043 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: Added host smithi159 2023-12-07T10:11:46.043 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:46.044 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:46.044 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:46.044 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:11:46.044 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:46.044 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:46.044 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:46.044 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:46.044 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:46.044 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:11:46.044 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:11:46.044 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:11:46.044 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: Updating smithi159:/etc/ceph/ceph.conf 2023-12-07T10:11:46.044 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:46.044 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: Updating smithi159:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:11:46.044 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2023-12-07T10:11:46.044 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3463676143' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-12-07T10:11:46.045 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:46.045 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:46.045 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:46.045 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: Updating smithi159:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-07T10:11:46.045 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3463676143' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-12-07T10:11:46.045 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: osdmap e4: 0 total, 0 up, 0 in 2023-12-07T10:11:46.045 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:46.045 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi119:172.21.15.119=a;smithi153:172.21.15.153=b;smithi159:172.21.15.159=c", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:11:46.045 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: Saving service mon spec with placement smithi119:172.21.15.119=a;smithi153:172.21.15.153=b;smithi159:172.21.15.159=c;count:3 2023-12-07T10:11:46.045 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:46.045 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:11:46.045 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:11:46.045 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:11:46.045 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:46.045 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-07T10:11:46.046 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:11:46.046 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: Deploying daemon mon.c on smithi159 2023-12-07T10:11:46.046 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:46.046 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 ceph-mon[134726]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-12-07T10:11:46.046 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:45 smithi159 systemd[1]: Started Ceph mon.c for a4c642c6-94e8-11ee-95a2-87774f69a715. 2023-12-07T10:11:46.651 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-07T10:11:46.651 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-07T10:11:47.257 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.c/config 2023-12-07T10:11:51.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:50 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T10:11:51.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:50 smithi119 ceph-mon[97908]: mon.a calling monitor election 2023-12-07T10:11:51.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:50 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:11:51.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:50 smithi119 ceph-mon[97908]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:51.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:50 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:11:51.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:50 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:11:51.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:50 smithi119 ceph-mon[97908]: mon.c calling monitor election 2023-12-07T10:11:51.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:50 smithi119 ceph-mon[97908]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:51.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:50 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:11:51.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:50 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:11:51.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:50 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:11:51.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:50 smithi119 ceph-mon[97908]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-07T10:11:51.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:50 smithi119 ceph-mon[97908]: monmap e2: 2 mons at {a=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],c=[v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-07T10:11:51.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:50 smithi119 ceph-mon[97908]: fsmap 2023-12-07T10:11:51.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:50 smithi119 ceph-mon[97908]: osdmap e4: 0 total, 0 up, 0 in 2023-12-07T10:11:51.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:50 smithi119 ceph-mon[97908]: mgrmap e13: x(active, since 50s) 2023-12-07T10:11:51.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:50 smithi119 ceph-mon[97908]: overall HEALTH_OK 2023-12-07T10:11:51.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:50 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:51.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:50 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:51.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:50 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:51.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:50 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-07T10:11:51.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:50 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:11:51.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:50 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T10:11:51.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:50 smithi159 ceph-mon[134726]: mon.a calling monitor election 2023-12-07T10:11:51.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:50 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:11:51.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:50 smithi159 ceph-mon[134726]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:51.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:50 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:11:51.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:50 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:11:51.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:50 smithi159 ceph-mon[134726]: mon.c calling monitor election 2023-12-07T10:11:51.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:50 smithi159 ceph-mon[134726]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:51.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:50 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:11:51.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:50 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:11:51.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:50 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:11:51.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:50 smithi159 ceph-mon[134726]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-07T10:11:51.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:50 smithi159 ceph-mon[134726]: monmap e2: 2 mons at {a=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],c=[v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-07T10:11:51.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:50 smithi159 ceph-mon[134726]: fsmap 2023-12-07T10:11:51.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:50 smithi159 ceph-mon[134726]: osdmap e4: 0 total, 0 up, 0 in 2023-12-07T10:11:51.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:50 smithi159 ceph-mon[134726]: mgrmap e13: x(active, since 50s) 2023-12-07T10:11:51.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:50 smithi159 ceph-mon[134726]: overall HEALTH_OK 2023-12-07T10:11:51.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:50 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:51.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:50 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:51.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:50 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:51.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:50 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-07T10:11:51.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:50 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:11:51.458 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.453+0000 7f0b535dc700 1 -- 172.21.15.159:0/3478594690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b380035c0 msgr2=0x7f0b38005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:51.458 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.453+0000 7f0b535dc700 1 --2- 172.21.15.159:0/3478594690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b380035c0 0x7f0b38005a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c100f60 tx=0x7f0b4800feb0 comp rx=0 tx=0).stop 2023-12-07T10:11:51.458 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.453+0000 7f0b535dc700 1 -- 172.21.15.159:0/3478594690 shutdown_connections 2023-12-07T10:11:51.458 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.453+0000 7f0b535dc700 1 --2- 172.21.15.159:0/3478594690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b380035c0 0x7f0b38005a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:51.458 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.453+0000 7f0b535dc700 1 -- 172.21.15.159:0/3478594690 >> 172.21.15.159:0/3478594690 conn(0x7f0b4c0fcb40 msgr2=0x7f0b4c0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:51.458 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.453+0000 7f0b535dc700 1 -- 172.21.15.159:0/3478594690 shutdown_connections 2023-12-07T10:11:51.458 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.453+0000 7f0b535dc700 1 -- 172.21.15.159:0/3478594690 wait complete. 2023-12-07T10:11:51.458 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.454+0000 7f0b535dc700 1 Processor -- start 2023-12-07T10:11:51.459 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.454+0000 7f0b535dc700 1 -- start start 2023-12-07T10:11:51.459 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.454+0000 7f0b535dc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b380035c0 0x7f0b4c1969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:51.459 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.454+0000 7f0b535dc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b4c196f30 0x7f0b4c19b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:51.459 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.454+0000 7f0b535dc700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c197470 con 0x7f0b4c196f30 2023-12-07T10:11:51.459 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.454+0000 7f0b535dc700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c1975e0 con 0x7f0b380035c0 2023-12-07T10:11:51.459 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.454+0000 7f0b51378700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b380035c0 0x7f0b4c1969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:51.459 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.455+0000 7f0b51378700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b380035c0 0x7f0b4c1969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42420/0 (socket says 172.21.15.159:42420) 2023-12-07T10:11:51.459 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.455+0000 7f0b51378700 1 -- 172.21.15.159:0/2650305084 learned_addr learned my addr 172.21.15.159:0/2650305084 (peer_addr_for_me v2:172.21.15.159:0/0) 2023-12-07T10:11:51.459 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.455+0000 7f0b51378700 1 -- 172.21.15.159:0/2650305084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b380035c0 msgr2=0x7f0b4c1969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-12-07T10:11:51.459 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.455+0000 7f0b51378700 1 -- 172.21.15.159:0/2650305084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b380035c0 msgr2=0x7f0b4c1969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-12-07T10:11:51.459 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.455+0000 7f0b51378700 1 --2- 172.21.15.159:0/2650305084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b380035c0 0x7f0b4c1969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-07T10:11:51.459 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.455+0000 7f0b51378700 1 --2- 172.21.15.159:0/2650305084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b380035c0 0x7f0b4c1969f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-07T10:11:51.459 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.455+0000 7f0b50b77700 1 --2- 172.21.15.159:0/2650305084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b4c196f30 0x7f0b4c19b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:51.461 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.455+0000 7f0b50b77700 1 -- 172.21.15.159:0/2650305084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b380035c0 msgr2=0x7f0b4c1969f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-07T10:11:51.461 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.455+0000 7f0b50b77700 1 --2- 172.21.15.159:0/2650305084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b380035c0 0x7f0b4c1969f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:51.461 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.455+0000 7f0b50b77700 1 -- 172.21.15.159:0/2650305084 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4800f900 con 0x7f0b4c196f30 2023-12-07T10:11:51.461 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.455+0000 7f0b50b77700 1 --2- 172.21.15.159:0/2650305084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b4c196f30 0x7f0b4c19b220 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00b4d0 tx=0x7f0b3c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:51.461 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.456+0000 7f0b427fc700 1 -- 172.21.15.159:0/2650305084 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f0b3c011630 con 0x7f0b4c196f30 2023-12-07T10:11:51.461 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.456+0000 7f0b427fc700 1 -- 172.21.15.159:0/2650305084 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b3c011c50 con 0x7f0b4c196f30 2023-12-07T10:11:51.461 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.456+0000 7f0b535dc700 1 -- 172.21.15.159:0/2650305084 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b4c19b760 con 0x7f0b4c196f30 2023-12-07T10:11:51.461 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.456+0000 7f0b427fc700 1 -- 172.21.15.159:0/2650305084 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f0b3c010d90 con 0x7f0b4c196f30 2023-12-07T10:11:51.461 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.456+0000 7f0b535dc700 1 -- 172.21.15.159:0/2650305084 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b4c19bcb0 con 0x7f0b4c196f30 2023-12-07T10:11:51.461 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.457+0000 7f0b427fc700 1 -- 172.21.15.159:0/2650305084 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45796+0+0 (secure 0 0 0) 0x7f0b3c01a400 con 0x7f0b4c196f30 2023-12-07T10:11:51.462 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.457+0000 7f0b427fc700 1 --2- 172.21.15.159:0/2650305084 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0b30038930 0x7f0b3003adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:51.462 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.457+0000 7f0b535dc700 1 -- 172.21.15.159:0/2650305084 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b4c102770 con 0x7f0b4c196f30 2023-12-07T10:11:51.462 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.457+0000 7f0b427fc700 1 -- 172.21.15.159:0/2650305084 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0b3c04b3f0 con 0x7f0b4c196f30 2023-12-07T10:11:51.462 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.458+0000 7f0b51378700 1 --2- 172.21.15.159:0/2650305084 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0b30038930 0x7f0b3003adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:51.462 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.458+0000 7f0b51378700 1 --2- 172.21.15.159:0/2650305084 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0b30038930 0x7f0b3003adf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0b480059d0 tx=0x7f0b48005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:51.466 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.463+0000 7f0b427fc700 1 -- 172.21.15.159:0/2650305084 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0b3c0178b0 con 0x7f0b4c196f30 2023-12-07T10:11:51.736 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.733+0000 7f0b535dc700 1 -- 172.21.15.159:0/2650305084 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0b4c19c080 con 0x7f0b4c196f30 2023-12-07T10:11:51.736 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.733+0000 7f0b427fc700 1 -- 172.21.15.159:0/2650305084 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1010 (secure 0 0 0) 0x7f0b3c03e020 con 0x7f0b4c196f30 2023-12-07T10:11:51.737 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-12-07T10:11:51.737 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":2,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","modified":"2023-12-07T10:11:45.725100Z","created":"2023-12-07T10:09:56.560924Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:3300","nonce":0},{"type":"v1","addr":"172.21.15.159:6789","nonce":0}]},"addr":"172.21.15.159:6789/0","public_addr":"172.21.15.159:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-12-07T10:11:51.739 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.735+0000 7f0b535dc700 1 -- 172.21.15.159:0/2650305084 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0b30038930 msgr2=0x7f0b3003adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:51.739 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.735+0000 7f0b535dc700 1 --2- 172.21.15.159:0/2650305084 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0b30038930 0x7f0b3003adf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0b480059d0 tx=0x7f0b48005960 comp rx=0 tx=0).stop 2023-12-07T10:11:51.739 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.735+0000 7f0b535dc700 1 -- 172.21.15.159:0/2650305084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b4c196f30 msgr2=0x7f0b4c19b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:51.739 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.735+0000 7f0b535dc700 1 --2- 172.21.15.159:0/2650305084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b4c196f30 0x7f0b4c19b220 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00b4d0 tx=0x7f0b3c00b890 comp rx=0 tx=0).stop 2023-12-07T10:11:51.739 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.736+0000 7f0b535dc700 1 -- 172.21.15.159:0/2650305084 shutdown_connections 2023-12-07T10:11:51.739 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.736+0000 7f0b535dc700 1 --2- 172.21.15.159:0/2650305084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b380035c0 0x7f0b4c1969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:51.739 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.736+0000 7f0b535dc700 1 --2- 172.21.15.159:0/2650305084 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0b30038930 0x7f0b3003adf0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:51.739 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.736+0000 7f0b535dc700 1 --2- 172.21.15.159:0/2650305084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b4c196f30 0x7f0b4c19b220 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:51.739 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.736+0000 7f0b535dc700 1 -- 172.21.15.159:0/2650305084 >> 172.21.15.159:0/2650305084 conn(0x7f0b4c0fcb40 msgr2=0x7f0b4c0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:51.739 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.736+0000 7f0b535dc700 1 -- 172.21.15.159:0/2650305084 shutdown_connections 2023-12-07T10:11:51.739 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:51.736+0000 7f0b535dc700 1 -- 172.21.15.159:0/2650305084 wait complete. 2023-12-07T10:11:51.740 INFO:teuthology.orchestra.run.smithi159.stderr:dumped monmap epoch 2 2023-12-07T10:11:52.099 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:51 smithi159 ceph-mon[134726]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:52.099 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:51 smithi159 ceph-mon[134726]: Deploying daemon mon.b on smithi153 2023-12-07T10:11:52.099 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:51 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:11:52.099 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:51 smithi159 ceph-mon[134726]: from='client.? 172.21.15.159:0/2650305084' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-07T10:11:52.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:51 smithi119 ceph-mon[97908]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:52.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:51 smithi119 ceph-mon[97908]: Deploying daemon mon.b on smithi153 2023-12-07T10:11:52.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:51 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:11:52.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:51 smithi119 ceph-mon[97908]: from='client.? 172.21.15.159:0/2650305084' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-07T10:11:52.980 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:11:52 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:11:52.725+0000 7f8aae255700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-12-07T10:11:53.127 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-07T10:11:53.128 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-07T10:11:53.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 systemd[1]: Starting Ceph mon.b for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:11:53.733 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.c/config 2023-12-07T10:11:54.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 podman[133586]: 2023-12-07T10:11:54.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: set uid:gid to 167:167 (ceph:ceph) 2023-12-07T10:11:54.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: ceph version 19.0.0-18-g3bd0bfa4 (3bd0bfa458b4399e7266c56d8215a2cf3a6091a0) reef (dev), process ceph-mon, pid 7 2023-12-07T10:11:54.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: pidfile_write: ignore empty --pid-file 2023-12-07T10:11:54.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 bash[133586]: c83140c64b80347607b4028e4cf7af801b6baa38578078d19c1c63ba1c1415d4 2023-12-07T10:11:54.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: load: jerasure load: lrc 2023-12-07T10:11:54.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: RocksDB version: 7.9.2 2023-12-07T10:11:54.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Git sha 0 2023-12-07T10:11:54.033 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Compile date 2023-12-07 06:56:28 2023-12-07T10:11:54.033 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: DB SUMMARY 2023-12-07T10:11:54.033 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: DB Session ID: G8R34DJPRY6CFKCZDAEO 2023-12-07T10:11:54.033 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: CURRENT file: CURRENT 2023-12-07T10:11:54.033 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: IDENTITY file: IDENTITY 2023-12-07T10:11:54.033 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-12-07T10:11:54.033 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-12-07T10:11:54.033 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-12-07T10:11:54.033 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.error_if_exists: 0 2023-12-07T10:11:54.033 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.create_if_missing: 0 2023-12-07T10:11:54.033 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.paranoid_checks: 1 2023-12-07T10:11:54.033 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.flush_verify_memtable_count: 1 2023-12-07T10:11:54.033 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-12-07T10:11:54.034 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-12-07T10:11:54.034 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.env: 0x55a049e11a60 2023-12-07T10:11:54.034 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.fs: PosixFileSystem 2023-12-07T10:11:54.034 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.info_log: 0x55a04b723740 2023-12-07T10:11:54.034 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.max_file_opening_threads: 16 2023-12-07T10:11:54.034 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.statistics: (nil) 2023-12-07T10:11:54.034 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.use_fsync: 0 2023-12-07T10:11:54.034 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.max_log_file_size: 0 2023-12-07T10:11:54.034 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-07T10:11:54.034 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.log_file_time_to_roll: 0 2023-12-07T10:11:54.034 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.keep_log_file_num: 1000 2023-12-07T10:11:54.034 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.recycle_log_file_num: 0 2023-12-07T10:11:54.034 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.allow_fallocate: 1 2023-12-07T10:11:54.034 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.allow_mmap_reads: 0 2023-12-07T10:11:54.035 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.allow_mmap_writes: 0 2023-12-07T10:11:54.035 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.use_direct_reads: 0 2023-12-07T10:11:54.035 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-07T10:11:54.035 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.create_missing_column_families: 0 2023-12-07T10:11:54.035 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.db_log_dir: 2023-12-07T10:11:54.035 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.wal_dir: 2023-12-07T10:11:54.035 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.table_cache_numshardbits: 6 2023-12-07T10:11:54.035 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.WAL_ttl_seconds: 0 2023-12-07T10:11:54.035 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.WAL_size_limit_MB: 0 2023-12-07T10:11:54.035 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-07T10:11:54.035 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-07T10:11:54.035 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.is_fd_close_on_exec: 1 2023-12-07T10:11:54.035 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.advise_random_on_open: 1 2023-12-07T10:11:54.035 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.db_write_buffer_size: 0 2023-12-07T10:11:54.035 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.write_buffer_manager: 0x55a04b6d9360 2023-12-07T10:11:54.035 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-07T10:11:54.036 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-07T10:11:54.036 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.use_adaptive_mutex: 0 2023-12-07T10:11:54.036 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.rate_limiter: (nil) 2023-12-07T10:11:54.036 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-07T10:11:54.036 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.wal_recovery_mode: 2 2023-12-07T10:11:54.036 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.enable_thread_tracking: 0 2023-12-07T10:11:54.036 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.enable_pipelined_write: 0 2023-12-07T10:11:54.036 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.unordered_write: 0 2023-12-07T10:11:54.036 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-07T10:11:54.036 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-07T10:11:54.036 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-07T10:11:54.036 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-07T10:11:54.036 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.row_cache: None 2023-12-07T10:11:54.036 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.wal_filter: None 2023-12-07T10:11:54.037 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-07T10:11:54.037 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.allow_ingest_behind: 0 2023-12-07T10:11:54.037 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.two_write_queues: 0 2023-12-07T10:11:54.037 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.manual_wal_flush: 0 2023-12-07T10:11:54.037 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.wal_compression: 0 2023-12-07T10:11:54.037 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.atomic_flush: 0 2023-12-07T10:11:54.037 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-07T10:11:54.037 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.persist_stats_to_disk: 0 2023-12-07T10:11:54.037 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.write_dbid_to_manifest: 0 2023-12-07T10:11:54.037 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.log_readahead_size: 0 2023-12-07T10:11:54.037 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-12-07T10:11:54.037 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.best_efforts_recovery: 0 2023-12-07T10:11:54.037 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-12-07T10:11:54.037 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-12-07T10:11:54.038 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.allow_data_in_errors: 0 2023-12-07T10:11:54.038 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.db_host_id: __hostname__ 2023-12-07T10:11:54.038 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.enforce_single_del_contracts: true 2023-12-07T10:11:54.038 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.max_background_jobs: 2 2023-12-07T10:11:54.038 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.max_background_compactions: -1 2023-12-07T10:11:54.038 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.max_subcompactions: 1 2023-12-07T10:11:54.038 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-07T10:11:54.038 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-07T10:11:54.038 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.delayed_write_rate : 16777216 2023-12-07T10:11:54.038 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.max_total_wal_size: 0 2023-12-07T10:11:54.038 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-07T10:11:54.038 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.stats_dump_period_sec: 600 2023-12-07T10:11:54.038 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.stats_persist_period_sec: 600 2023-12-07T10:11:54.039 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-07T10:11:54.039 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.max_open_files: -1 2023-12-07T10:11:54.039 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.bytes_per_sync: 0 2023-12-07T10:11:54.039 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.wal_bytes_per_sync: 0 2023-12-07T10:11:54.039 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.strict_bytes_per_sync: 0 2023-12-07T10:11:54.039 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.compaction_readahead_size: 0 2023-12-07T10:11:54.039 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.max_background_flushes: -1 2023-12-07T10:11:54.039 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Compression algorithms supported: 2023-12-07T10:11:54.039 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-07T10:11:54.039 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: kZSTD supported: 0 2023-12-07T10:11:54.039 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: kXpressCompression supported: 0 2023-12-07T10:11:54.039 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: kLZ4HCCompression supported: 1 2023-12-07T10:11:54.039 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: kZlibCompression supported: 1 2023-12-07T10:11:54.040 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: kSnappyCompression supported: 1 2023-12-07T10:11:54.040 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: kLZ4Compression supported: 1 2023-12-07T10:11:54.040 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: kBZip2Compression supported: 0 2023-12-07T10:11:54.040 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Fast CRC32 supported: Supported on x86 2023-12-07T10:11:54.040 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: DMutex implementation: pthread_mutex_t 2023-12-07T10:11:54.040 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-12-07T10:11:54.040 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-12-07T10:11:54.040 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-07T10:11:54.040 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.merge_operator: 2023-12-07T10:11:54.040 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.compaction_filter: None 2023-12-07T10:11:54.040 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.compaction_filter_factory: None 2023-12-07T10:11:54.040 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.sst_partitioner_factory: None 2023-12-07T10:11:54.040 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.memtable_factory: SkipListFactory 2023-12-07T10:11:54.040 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.table_factory: BlockBasedTable 2023-12-07T10:11:54.041 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a04b723860) 2023-12-07T10:11:54.041 INFO:journalctl@ceph.mon.b.smithi153.stdout: cache_index_and_filter_blocks: 1 2023-12-07T10:11:54.041 INFO:journalctl@ceph.mon.b.smithi153.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-12-07T10:11:54.041 INFO:journalctl@ceph.mon.b.smithi153.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-07T10:11:54.041 INFO:journalctl@ceph.mon.b.smithi153.stdout: pin_top_level_index_and_filter: 1 2023-12-07T10:11:54.041 INFO:journalctl@ceph.mon.b.smithi153.stdout: index_type: 0 2023-12-07T10:11:54.041 INFO:journalctl@ceph.mon.b.smithi153.stdout: data_block_index_type: 0 2023-12-07T10:11:54.041 INFO:journalctl@ceph.mon.b.smithi153.stdout: index_shortening: 1 2023-12-07T10:11:54.041 INFO:journalctl@ceph.mon.b.smithi153.stdout: data_block_hash_table_util_ratio: 0.750000 2023-12-07T10:11:54.041 INFO:journalctl@ceph.mon.b.smithi153.stdout: checksum: 4 2023-12-07T10:11:54.041 INFO:journalctl@ceph.mon.b.smithi153.stdout: no_block_cache: 0 2023-12-07T10:11:54.041 INFO:journalctl@ceph.mon.b.smithi153.stdout: block_cache: 0x55a04b779090 2023-12-07T10:11:54.041 INFO:journalctl@ceph.mon.b.smithi153.stdout: block_cache_name: BinnedLRUCache 2023-12-07T10:11:54.041 INFO:journalctl@ceph.mon.b.smithi153.stdout: block_cache_options: 2023-12-07T10:11:54.042 INFO:journalctl@ceph.mon.b.smithi153.stdout: capacity : 536870912 2023-12-07T10:11:54.042 INFO:journalctl@ceph.mon.b.smithi153.stdout: num_shard_bits : 4 2023-12-07T10:11:54.042 INFO:journalctl@ceph.mon.b.smithi153.stdout: strict_capacity_limit : 0 2023-12-07T10:11:54.042 INFO:journalctl@ceph.mon.b.smithi153.stdout: high_pri_pool_ratio: 0.000 2023-12-07T10:11:54.042 INFO:journalctl@ceph.mon.b.smithi153.stdout: block_cache_compressed: (nil) 2023-12-07T10:11:54.042 INFO:journalctl@ceph.mon.b.smithi153.stdout: persistent_cache: (nil) 2023-12-07T10:11:54.042 INFO:journalctl@ceph.mon.b.smithi153.stdout: block_size: 4096 2023-12-07T10:11:54.042 INFO:journalctl@ceph.mon.b.smithi153.stdout: block_size_deviation: 10 2023-12-07T10:11:54.042 INFO:journalctl@ceph.mon.b.smithi153.stdout: block_restart_interval: 16 2023-12-07T10:11:54.042 INFO:journalctl@ceph.mon.b.smithi153.stdout: index_block_restart_interval: 1 2023-12-07T10:11:54.042 INFO:journalctl@ceph.mon.b.smithi153.stdout: metadata_block_size: 4096 2023-12-07T10:11:54.042 INFO:journalctl@ceph.mon.b.smithi153.stdout: partition_filters: 0 2023-12-07T10:11:54.043 INFO:journalctl@ceph.mon.b.smithi153.stdout: use_delta_encoding: 1 2023-12-07T10:11:54.043 INFO:journalctl@ceph.mon.b.smithi153.stdout: filter_policy: bloomfilter 2023-12-07T10:11:54.043 INFO:journalctl@ceph.mon.b.smithi153.stdout: whole_key_filtering: 1 2023-12-07T10:11:54.043 INFO:journalctl@ceph.mon.b.smithi153.stdout: verify_compression: 0 2023-12-07T10:11:54.043 INFO:journalctl@ceph.mon.b.smithi153.stdout: read_amp_bytes_per_bit: 0 2023-12-07T10:11:54.043 INFO:journalctl@ceph.mon.b.smithi153.stdout: format_version: 5 2023-12-07T10:11:54.043 INFO:journalctl@ceph.mon.b.smithi153.stdout: enable_index_compression: 1 2023-12-07T10:11:54.043 INFO:journalctl@ceph.mon.b.smithi153.stdout: block_align: 0 2023-12-07T10:11:54.043 INFO:journalctl@ceph.mon.b.smithi153.stdout: max_auto_readahead_size: 262144 2023-12-07T10:11:54.043 INFO:journalctl@ceph.mon.b.smithi153.stdout: prepopulate_block_cache: 0 2023-12-07T10:11:54.043 INFO:journalctl@ceph.mon.b.smithi153.stdout: initial_auto_readahead_size: 8192 2023-12-07T10:11:54.043 INFO:journalctl@ceph.mon.b.smithi153.stdout: num_file_reads_for_auto_readahead: 2 2023-12-07T10:11:54.043 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.write_buffer_size: 33554432 2023-12-07T10:11:54.043 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.max_write_buffer_number: 2 2023-12-07T10:11:54.044 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.compression: NoCompression 2023-12-07T10:11:54.044 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.bottommost_compression: Disabled 2023-12-07T10:11:54.044 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.prefix_extractor: nullptr 2023-12-07T10:11:54.044 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-07T10:11:54.044 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.num_levels: 7 2023-12-07T10:11:54.044 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-07T10:11:54.044 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-07T10:11:54.044 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-07T10:11:54.044 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-07T10:11:54.044 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-07T10:11:54.044 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-07T10:11:54.044 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-07T10:11:54.044 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-07T10:11:54.044 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-12-07T10:11:54.045 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-07T10:11:54.045 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-12-07T10:11:54.045 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-12-07T10:11:54.045 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.compression_opts.window_bits: -14 2023-12-07T10:11:54.045 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.compression_opts.level: 32767 2023-12-07T10:11:54.045 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.compression_opts.strategy: 0 2023-12-07T10:11:54.045 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-07T10:11:54.045 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-07T10:11:54.045 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-12-07T10:11:54.045 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-12-07T10:11:54.045 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.compression_opts.enabled: false 2023-12-07T10:11:54.045 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-12-07T10:11:54.045 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-07T10:11:54.045 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-07T10:11:54.046 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-07T10:11:54.046 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.target_file_size_base: 67108864 2023-12-07T10:11:54.046 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.target_file_size_multiplier: 1 2023-12-07T10:11:54.046 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-07T10:11:54.046 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-07T10:11:54.046 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-07T10:11:54.046 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-07T10:11:54.046 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-07T10:11:54.046 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-07T10:11:54.046 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-07T10:11:54.046 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-07T10:11:54.046 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-07T10:11:54.047 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-07T10:11:54.047 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-07T10:11:54.047 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-07T10:11:54.047 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-12-07T10:11:54.047 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.arena_block_size: 1048576 2023-12-07T10:11:54.047 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-07T10:11:54.047 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-07T10:11:54.047 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.disable_auto_compactions: 0 2023-12-07T10:11:54.048 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-07T10:11:54.048 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-07T10:11:54.049 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-07T10:11:54.049 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-07T10:11:54.049 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-07T10:11:54.049 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-07T10:11:54.049 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-07T10:11:54.049 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-07T10:11:54.049 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-07T10:11:54.049 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-07T10:11:54.049 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.table_properties_collectors: 2023-12-07T10:11:54.049 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.inplace_update_support: 0 2023-12-07T10:11:54.049 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.inplace_update_num_locks: 10000 2023-12-07T10:11:54.049 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-07T10:11:54.049 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-07T10:11:54.050 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.memtable_huge_page_size: 0 2023-12-07T10:11:54.050 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.bloom_locality: 0 2023-12-07T10:11:54.050 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.max_successive_merges: 0 2023-12-07T10:11:54.050 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.optimize_filters_for_hits: 0 2023-12-07T10:11:54.050 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.paranoid_file_checks: 0 2023-12-07T10:11:54.050 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.force_consistency_checks: 1 2023-12-07T10:11:54.050 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.report_bg_io_stats: 0 2023-12-07T10:11:54.050 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.ttl: 2592000 2023-12-07T10:11:54.050 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.periodic_compaction_seconds: 0 2023-12-07T10:11:54.050 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-12-07T10:11:54.050 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-12-07T10:11:54.050 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.enable_blob_files: false 2023-12-07T10:11:54.050 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.min_blob_size: 0 2023-12-07T10:11:54.050 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.blob_file_size: 268435456 2023-12-07T10:11:54.051 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.blob_compression_type: NoCompression 2023-12-07T10:11:54.051 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.enable_blob_garbage_collection: false 2023-12-07T10:11:54.051 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-12-07T10:11:54.051 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-12-07T10:11:54.051 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-12-07T10:11:54.051 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.blob_file_starting_level: 0 2023-12-07T10:11:54.051 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-12-07T10:11:54.051 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-12-07T10:11:54.051 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-12-07T10:11:54.051 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7cfa8bab-30b6-4802-950e-f9abdcce617a 2023-12-07T10:11:54.051 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701943913832609, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-12-07T10:11:54.051 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-12-07T10:11:54.051 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701943913833173, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1701943913, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7cfa8bab-30b6-4802-950e-f9abdcce617a", "db_session_id": "G8R34DJPRY6CFKCZDAEO", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-12-07T10:11:54.052 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701943913833265, "job": 1, "event": "recovery_finished"} 2023-12-07T10:11:54.052 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-12-07T10:11:54.052 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-12-07T10:11:54.052 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a04b814000 2023-12-07T10:11:54.052 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: DB pointer 0x55a04b800000 2023-12-07T10:11:54.052 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-12-07T10:11:54.052 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-12-07T10:11:54.052 INFO:journalctl@ceph.mon.b.smithi153.stdout: ** DB Stats ** 2023-12-07T10:11:54.052 INFO:journalctl@ceph.mon.b.smithi153.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-07T10:11:54.052 INFO:journalctl@ceph.mon.b.smithi153.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-07T10:11:54.052 INFO:journalctl@ceph.mon.b.smithi153.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-07T10:11:54.052 INFO:journalctl@ceph.mon.b.smithi153.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-07T10:11:54.052 INFO:journalctl@ceph.mon.b.smithi153.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-07T10:11:54.052 INFO:journalctl@ceph.mon.b.smithi153.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-07T10:11:54.053 INFO:journalctl@ceph.mon.b.smithi153.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-07T10:11:54.053 INFO:journalctl@ceph.mon.b.smithi153.stdout: 2023-12-07T10:11:54.053 INFO:journalctl@ceph.mon.b.smithi153.stdout: ** Compaction Stats [default] ** 2023-12-07T10:11:54.053 INFO:journalctl@ceph.mon.b.smithi153.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-07T10:11:54.053 INFO:journalctl@ceph.mon.b.smithi153.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-12-07T10:11:54.053 INFO:journalctl@ceph.mon.b.smithi153.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-07T10:11:54.053 INFO:journalctl@ceph.mon.b.smithi153.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-07T10:11:54.053 INFO:journalctl@ceph.mon.b.smithi153.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-07T10:11:54.053 INFO:journalctl@ceph.mon.b.smithi153.stdout: 2023-12-07T10:11:54.053 INFO:journalctl@ceph.mon.b.smithi153.stdout: ** Compaction Stats [default] ** 2023-12-07T10:11:54.053 INFO:journalctl@ceph.mon.b.smithi153.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-07T10:11:54.053 INFO:journalctl@ceph.mon.b.smithi153.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-07T10:11:54.054 INFO:journalctl@ceph.mon.b.smithi153.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-07T10:11:54.054 INFO:journalctl@ceph.mon.b.smithi153.stdout: 2023-12-07T10:11:54.054 INFO:journalctl@ceph.mon.b.smithi153.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-12-07T10:11:54.054 INFO:journalctl@ceph.mon.b.smithi153.stdout: 2023-12-07T10:11:54.054 INFO:journalctl@ceph.mon.b.smithi153.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-07T10:11:54.054 INFO:journalctl@ceph.mon.b.smithi153.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-12-07T10:11:54.054 INFO:journalctl@ceph.mon.b.smithi153.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-07T10:11:54.054 INFO:journalctl@ceph.mon.b.smithi153.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-12-07T10:11:54.054 INFO:journalctl@ceph.mon.b.smithi153.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-12-07T10:11:54.054 INFO:journalctl@ceph.mon.b.smithi153.stdout: AddFile(Keys): cumulative 0, interval 0 2023-12-07T10:11:54.054 INFO:journalctl@ceph.mon.b.smithi153.stdout: Cumulative compaction: 0.00 GB write, 0.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-07T10:11:54.054 INFO:journalctl@ceph.mon.b.smithi153.stdout: Interval compaction: 0.00 GB write, 0.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-07T10:11:54.054 INFO:journalctl@ceph.mon.b.smithi153.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-07T10:11:54.054 INFO:journalctl@ceph.mon.b.smithi153.stdout: Block cache BinnedLRUCache@0x55a04b779090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-12-07T10:11:54.055 INFO:journalctl@ceph.mon.b.smithi153.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-12-07T10:11:54.055 INFO:journalctl@ceph.mon.b.smithi153.stdout: 2023-12-07T10:11:54.055 INFO:journalctl@ceph.mon.b.smithi153.stdout: ** File Read Latency Histogram By Level [default] ** 2023-12-07T10:11:54.055 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-12-07T10:11:54.055 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: using public_addr v2:172.21.15.153:0/0 -> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] 2023-12-07T10:11:54.055 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: starting mon.b rank -1 at public addrs [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] at bind addrs [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:11:54.055 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: mon.b@-1(???) e0 preinit fsid a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:11:54.055 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 systemd[1]: Started Ceph mon.b for a4c642c6-94e8-11ee-95a2-87774f69a715. 2023-12-07T10:11:54.055 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: mon.b@-1(synchronizing).mds e1 new map 2023-12-07T10:11:54.055 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: mon.b@-1(synchronizing).mds e1 print_map 2023-12-07T10:11:54.055 INFO:journalctl@ceph.mon.b.smithi153.stdout: e1 2023-12-07T10:11:54.055 INFO:journalctl@ceph.mon.b.smithi153.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-12-07T10:11:54.055 INFO:journalctl@ceph.mon.b.smithi153.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-12-07T10:11:54.055 INFO:journalctl@ceph.mon.b.smithi153.stdout: legacy client fscid: -1 2023-12-07T10:11:54.056 INFO:journalctl@ceph.mon.b.smithi153.stdout: 2023-12-07T10:11:54.056 INFO:journalctl@ceph.mon.b.smithi153.stdout: No filesystems configured 2023-12-07T10:11:54.056 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-12-07T10:11:54.056 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-12-07T10:11:54.056 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-12-07T10:11:54.056 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-12-07T10:11:54.056 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-12-07T10:11:54.056 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-12-07T10:11:54.056 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-12-07T10:11:54.056 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-07T10:11:54.056 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-07T10:11:54.056 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-07T10:11:54.056 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:54.056 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:54.057 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:54.057 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:54.057 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:54.057 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi153", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:11:54.057 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:11:54.057 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:11:54.057 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: Updating smithi153:/etc/ceph/ceph.conf 2023-12-07T10:11:54.057 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: Updating smithi153:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:11:54.057 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:54.057 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2023-12-07T10:11:54.057 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: Updating smithi153:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-07T10:11:54.057 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi159", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:11:54.057 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:54.057 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:54.058 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:54.058 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: Deploying cephadm binary to smithi159 2023-12-07T10:11:54.058 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:54.058 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:54.058 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:54.058 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:11:54.058 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:54.058 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: Added host smithi159 2023-12-07T10:11:54.058 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:54.058 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:54.058 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:54.058 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:11:54.058 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:54.058 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:54.059 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:54.059 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:54.059 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:54.059 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:11:54.059 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:11:54.059 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:11:54.059 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: Updating smithi159:/etc/ceph/ceph.conf 2023-12-07T10:11:54.059 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:54.059 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: Updating smithi159:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:11:54.059 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2023-12-07T10:11:54.059 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3463676143' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-12-07T10:11:54.059 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:54.060 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:54.060 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:54.060 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: Updating smithi159:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-07T10:11:54.060 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3463676143' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-12-07T10:11:54.060 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: osdmap e4: 0 total, 0 up, 0 in 2023-12-07T10:11:54.060 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:54.060 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi119:172.21.15.119=a;smithi153:172.21.15.153=b;smithi159:172.21.15.159=c", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:11:54.060 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: Saving service mon spec with placement smithi119:172.21.15.119=a;smithi153:172.21.15.153=b;smithi159:172.21.15.159=c;count:3 2023-12-07T10:11:54.060 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:54.060 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:11:54.060 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:11:54.060 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:11:54.060 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:54.060 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-07T10:11:54.060 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:11:54.061 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: Deploying daemon mon.c on smithi159 2023-12-07T10:11:54.061 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:54.061 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T10:11:54.061 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: mon.a calling monitor election 2023-12-07T10:11:54.061 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:11:54.061 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:54.061 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:11:54.061 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:11:54.061 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: mon.c calling monitor election 2023-12-07T10:11:54.061 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:54.061 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:11:54.061 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:11:54.061 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:11:54.061 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-07T10:11:54.062 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: monmap e2: 2 mons at {a=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],c=[v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-07T10:11:54.062 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: fsmap 2023-12-07T10:11:54.062 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: osdmap e4: 0 total, 0 up, 0 in 2023-12-07T10:11:54.062 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: mgrmap e13: x(active, since 50s) 2023-12-07T10:11:54.062 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: overall HEALTH_OK 2023-12-07T10:11:54.062 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:54.062 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:54.062 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:54.062 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-07T10:11:54.062 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:11:54.062 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:54.062 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: Deploying daemon mon.b on smithi153 2023-12-07T10:11:54.062 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:11:54.062 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: from='client.? 172.21.15.159:0/2650305084' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-07T10:11:54.063 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:54.063 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:11:53 smithi153 ceph-mon[133625]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-12-07T10:11:54.125 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:53 smithi159 ceph-mon[134726]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:54.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:53 smithi119 ceph-mon[97908]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:59.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:58 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T10:11:59.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:58 smithi119 ceph-mon[97908]: mon.c calling monitor election 2023-12-07T10:11:59.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:58 smithi119 ceph-mon[97908]: mon.a calling monitor election 2023-12-07T10:11:59.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:58 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:11:59.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:58 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:11:59.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:58 smithi119 ceph-mon[97908]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:59.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:58 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:11:59.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:58 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:11:59.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:58 smithi119 ceph-mon[97908]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:59.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:58 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:11:59.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:58 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:11:59.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:58 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:11:59.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:58 smithi119 ceph-mon[97908]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-07T10:11:59.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:58 smithi119 ceph-mon[97908]: monmap e3: 3 mons at {a=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],b=[v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0],c=[v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-07T10:11:59.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:58 smithi119 ceph-mon[97908]: fsmap 2023-12-07T10:11:59.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:58 smithi119 ceph-mon[97908]: osdmap e4: 0 total, 0 up, 0 in 2023-12-07T10:11:59.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:58 smithi119 ceph-mon[97908]: mgrmap e13: x(active, since 58s) 2023-12-07T10:11:59.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:58 smithi119 ceph-mon[97908]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-12-07T10:11:59.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:58 smithi119 ceph-mon[97908]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-12-07T10:11:59.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:58 smithi119 ceph-mon[97908]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-12-07T10:11:59.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:58 smithi119 ceph-mon[97908]: mon.b (rank 2) addr [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] is down (out of quorum) 2023-12-07T10:11:59.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:58 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:59.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:58 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:59.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:58 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T10:11:59.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:58 smithi159 ceph-mon[134726]: mon.c calling monitor election 2023-12-07T10:11:59.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:58 smithi159 ceph-mon[134726]: mon.a calling monitor election 2023-12-07T10:11:59.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:58 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:11:59.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:58 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:11:59.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:58 smithi159 ceph-mon[134726]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:59.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:58 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:11:59.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:58 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:11:59.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:58 smithi159 ceph-mon[134726]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:11:59.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:58 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:11:59.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:58 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:11:59.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:58 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:11:59.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:58 smithi159 ceph-mon[134726]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-07T10:11:59.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:58 smithi159 ceph-mon[134726]: monmap e3: 3 mons at {a=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],b=[v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0],c=[v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-07T10:11:59.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:58 smithi159 ceph-mon[134726]: fsmap 2023-12-07T10:11:59.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:58 smithi159 ceph-mon[134726]: osdmap e4: 0 total, 0 up, 0 in 2023-12-07T10:11:59.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:58 smithi159 ceph-mon[134726]: mgrmap e13: x(active, since 58s) 2023-12-07T10:11:59.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:58 smithi159 ceph-mon[134726]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-12-07T10:11:59.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:58 smithi159 ceph-mon[134726]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-12-07T10:11:59.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:58 smithi159 ceph-mon[134726]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-12-07T10:11:59.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:58 smithi159 ceph-mon[134726]: mon.b (rank 2) addr [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] is down (out of quorum) 2023-12-07T10:11:59.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:58 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:59.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:58 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:11:59.317 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.313+0000 7f6082ffd700 1 -- 172.21.15.159:0/1233526703 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607401d440 con 0x7f606c0035c0 2023-12-07T10:11:59.317 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.313+0000 7f608a433700 1 -- 172.21.15.159:0/1233526703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f606c0035c0 msgr2=0x7f606c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:59.317 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.313+0000 7f608a433700 1 --2- 172.21.15.159:0/1233526703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f606c0035c0 0x7f606c005a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f60840ffe80 tx=0x7f607400fe60 comp rx=0 tx=0).stop 2023-12-07T10:11:59.317 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.313+0000 7f608a433700 1 -- 172.21.15.159:0/1233526703 shutdown_connections 2023-12-07T10:11:59.317 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.313+0000 7f608a433700 1 --2- 172.21.15.159:0/1233526703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f606c0035c0 0x7f606c005a50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:59.317 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.313+0000 7f608a433700 1 -- 172.21.15.159:0/1233526703 >> 172.21.15.159:0/1233526703 conn(0x7f6084076bf0 msgr2=0x7f60840fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:59.317 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.313+0000 7f608a433700 1 -- 172.21.15.159:0/1233526703 shutdown_connections 2023-12-07T10:11:59.317 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.313+0000 7f608a433700 1 -- 172.21.15.159:0/1233526703 wait complete. 2023-12-07T10:11:59.320 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.314+0000 7f608a433700 1 Processor -- start 2023-12-07T10:11:59.320 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.314+0000 7f608a433700 1 -- start start 2023-12-07T10:11:59.320 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.315+0000 7f608a433700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f606c0035c0 0x7f60841978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:59.320 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.315+0000 7f608a433700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6084197df0 0x7f6084198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:59.320 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.315+0000 7f608a433700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60841a0920 0x7f60841a2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:59.320 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.315+0000 7f608a433700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60841a33b0 con 0x7f6084197df0 2023-12-07T10:11:59.320 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.315+0000 7f608a433700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60841a3520 con 0x7f60841a0920 2023-12-07T10:11:59.320 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.315+0000 7f608a433700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60841a36e0 con 0x7f606c0035c0 2023-12-07T10:11:59.320 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.315+0000 7f60837fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6084197df0 0x7f6084198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:59.320 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.315+0000 7f60837fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6084197df0 0x7f6084198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.159:39406/0 (socket says 172.21.15.159:39406) 2023-12-07T10:11:59.320 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.315+0000 7f60837fe700 1 -- 172.21.15.159:0/2840820500 learned_addr learned my addr 172.21.15.159:0/2840820500 (peer_addr_for_me v2:172.21.15.159:0/0) 2023-12-07T10:11:59.321 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.315+0000 7f6083fff700 1 --2- 172.21.15.159:0/2840820500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f606c0035c0 0x7f60841978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:59.321 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.315+0000 7f60889d0700 1 --2- 172.21.15.159:0/2840820500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60841a0920 0x7f60841a2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:59.321 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.315+0000 7f6083fff700 1 -- 172.21.15.159:0/2840820500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f606c0035c0 msgr2=0x7f60841978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-12-07T10:11:59.321 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.315+0000 7f6083fff700 1 -- 172.21.15.159:0/2840820500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f606c0035c0 msgr2=0x7f60841978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-12-07T10:11:59.321 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.315+0000 7f6083fff700 1 --2- 172.21.15.159:0/2840820500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f606c0035c0 0x7f60841978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-07T10:11:59.321 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.315+0000 7f60889d0700 1 -- 172.21.15.159:0/2840820500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60841a0920 msgr2=0x7f60841a2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-12-07T10:11:59.321 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.315+0000 7f60889d0700 1 -- 172.21.15.159:0/2840820500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60841a0920 msgr2=0x7f60841a2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-12-07T10:11:59.321 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.315+0000 7f6083fff700 1 --2- 172.21.15.159:0/2840820500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f606c0035c0 0x7f60841978b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-07T10:11:59.321 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.315+0000 7f60889d0700 1 --2- 172.21.15.159:0/2840820500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60841a0920 0x7f60841a2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-07T10:11:59.321 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.315+0000 7f60889d0700 1 --2- 172.21.15.159:0/2840820500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60841a0920 0x7f60841a2d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-07T10:11:59.321 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.316+0000 7f60837fe700 1 -- 172.21.15.159:0/2840820500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f606c0035c0 msgr2=0x7f60841978b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-07T10:11:59.321 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.316+0000 7f60837fe700 1 --2- 172.21.15.159:0/2840820500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f606c0035c0 0x7f60841978b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:59.321 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.316+0000 7f60837fe700 1 -- 172.21.15.159:0/2840820500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60841a0920 msgr2=0x7f60841a2d10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-07T10:11:59.321 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.316+0000 7f60837fe700 1 --2- 172.21.15.159:0/2840820500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60841a0920 0x7f60841a2d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:59.322 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.316+0000 7f60837fe700 1 -- 172.21.15.159:0/2840820500 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607400f8b0 con 0x7f6084197df0 2023-12-07T10:11:59.322 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.316+0000 7f60837fe700 1 --2- 172.21.15.159:0/2840820500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6084197df0 0x7f6084198270 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f607800b780 tx=0x7f607800bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:59.322 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.316+0000 7f60817fa700 1 -- 172.21.15.159:0/2840820500 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6078004230 con 0x7f6084197df0 2023-12-07T10:11:59.322 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.316+0000 7f608a433700 1 -- 172.21.15.159:0/2840820500 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60841a39c0 con 0x7f6084197df0 2023-12-07T10:11:59.322 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.316+0000 7f608a433700 1 -- 172.21.15.159:0/2840820500 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60841a3f10 con 0x7f6084197df0 2023-12-07T10:11:59.322 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.316+0000 7f60817fa700 1 -- 172.21.15.159:0/2840820500 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60780093d0 con 0x7f6084197df0 2023-12-07T10:11:59.322 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.316+0000 7f60817fa700 1 -- 172.21.15.159:0/2840820500 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6078011aa0 con 0x7f6084197df0 2023-12-07T10:11:59.323 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.317+0000 7f60817fa700 1 -- 172.21.15.159:0/2840820500 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45796+0+0 (secure 0 0 0) 0x7f6078011c00 con 0x7f6084197df0 2023-12-07T10:11:59.323 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.317+0000 7f608a433700 1 -- 172.21.15.159:0/2840820500 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6084101690 con 0x7f6084197df0 2023-12-07T10:11:59.323 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.318+0000 7f60817fa700 1 --2- 172.21.15.159:0/2840820500 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6070038f70 0x7f607003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:11:59.323 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.318+0000 7f60817fa700 1 -- 172.21.15.159:0/2840820500 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f607804afa0 con 0x7f6084197df0 2023-12-07T10:11:59.323 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.318+0000 7f6083fff700 1 --2- 172.21.15.159:0/2840820500 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6070038f70 0x7f607003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:11:59.323 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.318+0000 7f6083fff700 1 --2- 172.21.15.159:0/2840820500 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6070038f70 0x7f607003b430 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6074020040 tx=0x7f6074001e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:11:59.325 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.322+0000 7f60817fa700 1 -- 172.21.15.159:0/2840820500 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f607801ae40 con 0x7f6084197df0 2023-12-07T10:11:59.617 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.614+0000 7f608a433700 1 -- 172.21.15.159:0/2840820500 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6084107ca0 con 0x7f6084197df0 2023-12-07T10:11:59.618 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.615+0000 7f60817fa700 1 -- 172.21.15.159:0/2840820500 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1278 (secure 0 0 0) 0x7f6078018350 con 0x7f6084197df0 2023-12-07T10:11:59.618 INFO:teuthology.orchestra.run.smithi159.stdout: 2023-12-07T10:11:59.618 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":3,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","modified":"2023-12-07T10:11:53.861158Z","created":"2023-12-07T10:09:56.560924Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:3300","nonce":0},{"type":"v1","addr":"172.21.15.159:6789","nonce":0}]},"addr":"172.21.15.159:6789/0","public_addr":"172.21.15.159:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-12-07T10:11:59.620 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.617+0000 7f608a433700 1 -- 172.21.15.159:0/2840820500 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6070038f70 msgr2=0x7f607003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:59.620 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.617+0000 7f608a433700 1 --2- 172.21.15.159:0/2840820500 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6070038f70 0x7f607003b430 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6074020040 tx=0x7f6074001e50 comp rx=0 tx=0).stop 2023-12-07T10:11:59.621 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.617+0000 7f608a433700 1 -- 172.21.15.159:0/2840820500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6084197df0 msgr2=0x7f6084198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:11:59.621 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.617+0000 7f608a433700 1 --2- 172.21.15.159:0/2840820500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6084197df0 0x7f6084198270 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f607800b780 tx=0x7f607800bb40 comp rx=0 tx=0).stop 2023-12-07T10:11:59.621 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.617+0000 7f608a433700 1 -- 172.21.15.159:0/2840820500 shutdown_connections 2023-12-07T10:11:59.621 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.617+0000 7f608a433700 1 --2- 172.21.15.159:0/2840820500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60841a0920 0x7f60841a2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:59.621 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.617+0000 7f608a433700 1 --2- 172.21.15.159:0/2840820500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f606c0035c0 0x7f60841978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:59.621 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.617+0000 7f608a433700 1 --2- 172.21.15.159:0/2840820500 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6070038f70 0x7f607003b430 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:59.621 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.617+0000 7f608a433700 1 --2- 172.21.15.159:0/2840820500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6084197df0 0x7f6084198270 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:11:59.621 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.617+0000 7f608a433700 1 -- 172.21.15.159:0/2840820500 >> 172.21.15.159:0/2840820500 conn(0x7f6084076bf0 msgr2=0x7f6084103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:11:59.621 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.617+0000 7f608a433700 1 -- 172.21.15.159:0/2840820500 shutdown_connections 2023-12-07T10:11:59.621 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:11:59.617+0000 7f608a433700 1 -- 172.21.15.159:0/2840820500 wait complete. 2023-12-07T10:11:59.621 INFO:teuthology.orchestra.run.smithi159.stderr:dumped monmap epoch 3 2023-12-07T10:12:00.003 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-12-07T10:12:00.003 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph config generate-minimal-conf 2023-12-07T10:12:00.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:59 smithi119 ceph-mon[97908]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:00.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:59 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:00.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:59 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:00.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:59 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:12:00.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:59 smithi119 ceph-mon[97908]: from='client.? 172.21.15.159:0/2840820500' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-07T10:12:00.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:11:59 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:12:00.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:59 smithi159 ceph-mon[134726]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:00.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:59 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:00.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:59 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:00.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:59 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:12:00.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:59 smithi159 ceph-mon[134726]: from='client.? 172.21.15.159:0/2840820500' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-07T10:12:00.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:11:59 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:12:00.788 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:12:01.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T10:12:01.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: mon.c calling monitor election 2023-12-07T10:12:01.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: mon.a calling monitor election 2023-12-07T10:12:01.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:12:01.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:12:01.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:01.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:12:01.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:12:01.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:01.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:12:01.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:12:01.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:12:01.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-07T10:12:01.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: monmap e3: 3 mons at {a=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],b=[v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0],c=[v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-07T10:12:01.283 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: fsmap 2023-12-07T10:12:01.283 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: osdmap e4: 0 total, 0 up, 0 in 2023-12-07T10:12:01.283 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: mgrmap e13: x(active, since 58s) 2023-12-07T10:12:01.283 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-12-07T10:12:01.283 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-12-07T10:12:01.283 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-12-07T10:12:01.283 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: mon.b (rank 2) addr [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] is down (out of quorum) 2023-12-07T10:12:01.283 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:01.283 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:01.283 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:01.283 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:01.283 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:01.283 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:12:01.283 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: from='client.? 172.21.15.159:0/2840820500' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-07T10:12:01.283 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:12:01.861 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:01 smithi119 ceph-mon[97908]: mon.b calling monitor election 2023-12-07T10:12:01.861 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:01 smithi119 ceph-mon[97908]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:01.861 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:01 smithi119 ceph-mon[97908]: Updating smithi119:/etc/ceph/ceph.conf 2023-12-07T10:12:01.861 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:01 smithi119 ceph-mon[97908]: Updating smithi153:/etc/ceph/ceph.conf 2023-12-07T10:12:01.861 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:01 smithi119 ceph-mon[97908]: Updating smithi159:/etc/ceph/ceph.conf 2023-12-07T10:12:01.861 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:01 smithi119 ceph-mon[97908]: mon.b calling monitor election 2023-12-07T10:12:01.861 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:01 smithi119 ceph-mon[97908]: mon.c calling monitor election 2023-12-07T10:12:01.861 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:01 smithi119 ceph-mon[97908]: mon.a calling monitor election 2023-12-07T10:12:01.862 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:01 smithi119 ceph-mon[97908]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-12-07T10:12:01.862 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:01 smithi119 ceph-mon[97908]: monmap e3: 3 mons at {a=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],b=[v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0],c=[v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-07T10:12:01.862 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:01 smithi119 ceph-mon[97908]: fsmap 2023-12-07T10:12:01.862 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:01 smithi119 ceph-mon[97908]: osdmap e4: 0 total, 0 up, 0 in 2023-12-07T10:12:01.862 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:01 smithi119 ceph-mon[97908]: mgrmap e13: x(active, since 60s) 2023-12-07T10:12:01.862 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:01 smithi119 ceph-mon[97908]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-12-07T10:12:01.862 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:01 smithi119 ceph-mon[97908]: Cluster is now healthy 2023-12-07T10:12:01.862 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:01 smithi119 ceph-mon[97908]: overall HEALTH_OK 2023-12-07T10:12:01.862 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:01 smithi119 ceph-mon[97908]: Updating smithi153:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:12:01.862 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:01 smithi119 ceph-mon[97908]: Updating smithi159:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:12:01.862 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:01.862 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:01.862 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:01.862 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:02.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:01 smithi159 ceph-mon[134726]: mon.b calling monitor election 2023-12-07T10:12:02.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:01 smithi159 ceph-mon[134726]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:02.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:01 smithi159 ceph-mon[134726]: Updating smithi119:/etc/ceph/ceph.conf 2023-12-07T10:12:02.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:01 smithi159 ceph-mon[134726]: Updating smithi153:/etc/ceph/ceph.conf 2023-12-07T10:12:02.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:01 smithi159 ceph-mon[134726]: Updating smithi159:/etc/ceph/ceph.conf 2023-12-07T10:12:02.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:01 smithi159 ceph-mon[134726]: mon.b calling monitor election 2023-12-07T10:12:02.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:01 smithi159 ceph-mon[134726]: mon.c calling monitor election 2023-12-07T10:12:02.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:01 smithi159 ceph-mon[134726]: mon.a calling monitor election 2023-12-07T10:12:02.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:01 smithi159 ceph-mon[134726]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-12-07T10:12:02.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:01 smithi159 ceph-mon[134726]: monmap e3: 3 mons at {a=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],b=[v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0],c=[v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-07T10:12:02.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:01 smithi159 ceph-mon[134726]: fsmap 2023-12-07T10:12:02.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:01 smithi159 ceph-mon[134726]: osdmap e4: 0 total, 0 up, 0 in 2023-12-07T10:12:02.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:01 smithi159 ceph-mon[134726]: mgrmap e13: x(active, since 60s) 2023-12-07T10:12:02.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:01 smithi159 ceph-mon[134726]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-12-07T10:12:02.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:01 smithi159 ceph-mon[134726]: Cluster is now healthy 2023-12-07T10:12:02.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:01 smithi159 ceph-mon[134726]: overall HEALTH_OK 2023-12-07T10:12:02.016 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:01 smithi159 ceph-mon[134726]: Updating smithi153:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:12:02.016 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:01 smithi159 ceph-mon[134726]: Updating smithi159:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:12:02.016 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:02.016 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:02.016 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:02.016 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:02.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:01 smithi153 ceph-mon[133625]: mon.b calling monitor election 2023-12-07T10:12:02.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:01 smithi153 ceph-mon[133625]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:02.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:01 smithi153 ceph-mon[133625]: Updating smithi119:/etc/ceph/ceph.conf 2023-12-07T10:12:02.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:01 smithi153 ceph-mon[133625]: Updating smithi153:/etc/ceph/ceph.conf 2023-12-07T10:12:02.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:01 smithi153 ceph-mon[133625]: Updating smithi159:/etc/ceph/ceph.conf 2023-12-07T10:12:02.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:01 smithi153 ceph-mon[133625]: mon.b calling monitor election 2023-12-07T10:12:02.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:01 smithi153 ceph-mon[133625]: mon.c calling monitor election 2023-12-07T10:12:02.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:01 smithi153 ceph-mon[133625]: mon.a calling monitor election 2023-12-07T10:12:02.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:01 smithi153 ceph-mon[133625]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-12-07T10:12:02.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:01 smithi153 ceph-mon[133625]: monmap e3: 3 mons at {a=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],b=[v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0],c=[v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-07T10:12:02.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:01 smithi153 ceph-mon[133625]: fsmap 2023-12-07T10:12:02.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:01 smithi153 ceph-mon[133625]: osdmap e4: 0 total, 0 up, 0 in 2023-12-07T10:12:02.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:01 smithi153 ceph-mon[133625]: mgrmap e13: x(active, since 60s) 2023-12-07T10:12:02.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:01 smithi153 ceph-mon[133625]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-12-07T10:12:02.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:01 smithi153 ceph-mon[133625]: Cluster is now healthy 2023-12-07T10:12:02.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:01 smithi153 ceph-mon[133625]: overall HEALTH_OK 2023-12-07T10:12:02.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:01 smithi153 ceph-mon[133625]: Updating smithi153:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:12:02.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:01 smithi153 ceph-mon[133625]: Updating smithi159:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:12:02.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:02.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:02.033 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:02.033 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:02.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.521+0000 7f3fe806c700 1 -- 172.21.15.119:0/3445317431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fe0101170 msgr2=0x7f3fe0101550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:12:02.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.521+0000 7f3fe806c700 1 --2- 172.21.15.119:0/3445317431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fe0101170 0x7f3fe0101550 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc009910 tx=0x7f3fdc009c20 comp rx=0 tx=0).stop 2023-12-07T10:12:02.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.522+0000 7f3fe806c700 1 -- 172.21.15.119:0/3445317431 shutdown_connections 2023-12-07T10:12:02.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.522+0000 7f3fe806c700 1 --2- 172.21.15.119:0/3445317431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fe0101170 0x7f3fe0101550 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:02.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.522+0000 7f3fe806c700 1 -- 172.21.15.119:0/3445317431 >> 172.21.15.119:0/3445317431 conn(0x7f3fe00fca40 msgr2=0x7f3fe00fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:12:02.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.522+0000 7f3fe806c700 1 -- 172.21.15.119:0/3445317431 shutdown_connections 2023-12-07T10:12:02.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.522+0000 7f3fe806c700 1 -- 172.21.15.119:0/3445317431 wait complete. 2023-12-07T10:12:02.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.523+0000 7f3fe806c700 1 Processor -- start 2023-12-07T10:12:02.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.523+0000 7f3fe806c700 1 -- start start 2023-12-07T10:12:02.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.523+0000 7f3fe806c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fe0101170 0x7f3fe019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:12:02.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.523+0000 7f3fe806c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fe019ee80 0x7f3fe0198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:12:02.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.523+0000 7f3fe806c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fe0198fc0 0x7f3fe0199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:12:02.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.523+0000 7f3fe806c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe019f480 con 0x7f3fe019ee80 2023-12-07T10:12:02.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.523+0000 7f3fe806c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe01a5760 con 0x7f3fe0198fc0 2023-12-07T10:12:02.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.523+0000 7f3fe806c700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe01a58a0 con 0x7f3fe0101170 2023-12-07T10:12:02.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.523+0000 7f3fe5607700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fe019ee80 0x7f3fe0198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:12:02.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.523+0000 7f3fe5607700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fe019ee80 0x7f3fe0198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49646/0 (socket says 172.21.15.119:49646) 2023-12-07T10:12:02.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.523+0000 7f3fe5607700 1 -- 172.21.15.119:0/265343595 learned_addr learned my addr 172.21.15.119:0/265343595 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:12:02.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.524+0000 7f3fe5e08700 1 --2- 172.21.15.119:0/265343595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fe0101170 0x7f3fe019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:12:02.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.524+0000 7f3fe6609700 1 --2- 172.21.15.119:0/265343595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fe0198fc0 0x7f3fe0199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:12:02.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.524+0000 7f3fe5607700 1 -- 172.21.15.119:0/265343595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fe0101170 msgr2=0x7f3fe019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:12:02.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.524+0000 7f3fe5607700 1 --2- 172.21.15.119:0/265343595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fe0101170 0x7f3fe019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:02.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.524+0000 7f3fe5607700 1 -- 172.21.15.119:0/265343595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fe0198fc0 msgr2=0x7f3fe0199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:12:02.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.524+0000 7f3fe5607700 1 --2- 172.21.15.119:0/265343595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fe0198fc0 0x7f3fe0199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:02.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.524+0000 7f3fe5607700 1 -- 172.21.15.119:0/265343595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fdc0095f0 con 0x7f3fe019ee80 2023-12-07T10:12:02.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.524+0000 7f3fe5607700 1 --2- 172.21.15.119:0/265343595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fe019ee80 0x7f3fe0198a80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000b7b0 tx=0x7f3fd000bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:12:02.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.524+0000 7f3fd6ffd700 1 -- 172.21.15.119:0/265343595 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0004230 con 0x7f3fe019ee80 2023-12-07T10:12:02.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.524+0000 7f3fd6ffd700 1 -- 172.21.15.119:0/265343595 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fd0004390 con 0x7f3fe019ee80 2023-12-07T10:12:02.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.524+0000 7f3fe806c700 1 -- 172.21.15.119:0/265343595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe01a5b80 con 0x7f3fe019ee80 2023-12-07T10:12:02.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.524+0000 7f3fd6ffd700 1 -- 172.21.15.119:0/265343595 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0011ad0 con 0x7f3fe019ee80 2023-12-07T10:12:02.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.524+0000 7f3fe806c700 1 -- 172.21.15.119:0/265343595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe01a60d0 con 0x7f3fe019ee80 2023-12-07T10:12:02.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.526+0000 7f3fd6ffd700 1 -- 172.21.15.119:0/265343595 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45796+0+0 (secure 0 0 0) 0x7f3fd00036a0 con 0x7f3fe019ee80 2023-12-07T10:12:02.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.526+0000 7f3fe806c700 1 -- 172.21.15.119:0/265343595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe0102650 con 0x7f3fe019ee80 2023-12-07T10:12:02.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.526+0000 7f3fd6ffd700 1 --2- 172.21.15.119:0/265343595 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3fcc038fc0 0x7f3fcc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:12:02.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.526+0000 7f3fd6ffd700 1 -- 172.21.15.119:0/265343595 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3fd004b170 con 0x7f3fe019ee80 2023-12-07T10:12:02.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.526+0000 7f3fe5e08700 1 --2- 172.21.15.119:0/265343595 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3fcc038fc0 0x7f3fcc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:12:02.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.527+0000 7f3fe5e08700 1 --2- 172.21.15.119:0/265343595 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3fcc038fc0 0x7f3fcc03b480 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc004690 tx=0x7f3fdc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:12:02.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.531+0000 7f3fd6ffd700 1 -- 172.21.15.119:0/265343595 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3fd0004500 con 0x7f3fe019ee80 2023-12-07T10:12:02.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.739+0000 7f3fe806c700 1 -- 172.21.15.119:0/265343595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f3fe0062c40 con 0x7f3fe019ee80 2023-12-07T10:12:02.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.739+0000 7f3fd6ffd700 1 -- 172.21.15.119:0/265343595 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+277 (secure 0 0 0) 0x7f3fd0021710 con 0x7f3fe019ee80 2023-12-07T10:12:02.742 INFO:teuthology.orchestra.run.smithi119.stdout:# minimal ceph.conf for a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:12:02.742 INFO:teuthology.orchestra.run.smithi119.stdout:[global] 2023-12-07T10:12:02.743 INFO:teuthology.orchestra.run.smithi119.stdout: fsid = a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:12:02.743 INFO:teuthology.orchestra.run.smithi119.stdout: mon_host = [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] 2023-12-07T10:12:02.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.741+0000 7f3fe806c700 1 -- 172.21.15.119:0/265343595 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3fcc038fc0 msgr2=0x7f3fcc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:12:02.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.741+0000 7f3fe806c700 1 --2- 172.21.15.119:0/265343595 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3fcc038fc0 0x7f3fcc03b480 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc004690 tx=0x7f3fdc005fb0 comp rx=0 tx=0).stop 2023-12-07T10:12:02.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.741+0000 7f3fe806c700 1 -- 172.21.15.119:0/265343595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fe019ee80 msgr2=0x7f3fe0198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:12:02.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.741+0000 7f3fe806c700 1 --2- 172.21.15.119:0/265343595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fe019ee80 0x7f3fe0198a80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000b7b0 tx=0x7f3fd000bac0 comp rx=0 tx=0).stop 2023-12-07T10:12:02.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.742+0000 7f3fe806c700 1 -- 172.21.15.119:0/265343595 shutdown_connections 2023-12-07T10:12:02.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.742+0000 7f3fe806c700 1 --2- 172.21.15.119:0/265343595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fe0198fc0 0x7f3fe0199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:02.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.742+0000 7f3fe806c700 1 --2- 172.21.15.119:0/265343595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fe0101170 0x7f3fe019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:02.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.742+0000 7f3fe806c700 1 --2- 172.21.15.119:0/265343595 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3fcc038fc0 0x7f3fcc03b480 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:02.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.742+0000 7f3fe806c700 1 --2- 172.21.15.119:0/265343595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fe019ee80 0x7f3fe0198a80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:02.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.742+0000 7f3fe806c700 1 -- 172.21.15.119:0/265343595 >> 172.21.15.119:0/265343595 conn(0x7f3fe00fca40 msgr2=0x7f3fe0073980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:12:02.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.742+0000 7f3fe806c700 1 -- 172.21.15.119:0/265343595 shutdown_connections 2023-12-07T10:12:02.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:02.742+0000 7f3fe806c700 1 -- 172.21.15.119:0/265343595 wait complete. 2023-12-07T10:12:03.040 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:12:02 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:12:02.861+0000 7f8aae255700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-12-07T10:12:03.041 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:02 smithi119 ceph-mon[97908]: Updating smithi119:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:12:03.041 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:02 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:03.041 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:02 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:03.041 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:02 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:03.041 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:02 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:12:03.041 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:02 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:03.041 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:02 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:03.041 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:02 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:03.041 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:02 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:03.041 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:02 smithi119 ceph-mon[97908]: Reconfiguring mon.a (unknown last config time)... 2023-12-07T10:12:03.041 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:02 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-07T10:12:03.042 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:02 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-07T10:12:03.042 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:02 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:03.042 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:02 smithi119 ceph-mon[97908]: Reconfiguring daemon mon.a on smithi119 2023-12-07T10:12:03.042 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:02 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/265343595' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:03.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:02 smithi159 ceph-mon[134726]: Updating smithi119:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:12:03.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:02 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:03.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:02 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:03.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:02 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:03.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:02 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:12:03.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:02 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:03.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:02 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:03.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:02 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:03.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:02 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:03.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:02 smithi159 ceph-mon[134726]: Reconfiguring mon.a (unknown last config time)... 2023-12-07T10:12:03.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:02 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-07T10:12:03.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:02 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-07T10:12:03.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:02 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:03.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:02 smithi159 ceph-mon[134726]: Reconfiguring daemon mon.a on smithi119 2023-12-07T10:12:03.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:02 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/265343595' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:03.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:02 smithi153 ceph-mon[133625]: Updating smithi119:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:12:03.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:02 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:03.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:02 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:03.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:02 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:03.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:02 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:12:03.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:02 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:03.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:02 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:03.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:02 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:03.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:02 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:03.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:02 smithi153 ceph-mon[133625]: Reconfiguring mon.a (unknown last config time)... 2023-12-07T10:12:03.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:02 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-07T10:12:03.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:02 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-07T10:12:03.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:02 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:03.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:02 smithi153 ceph-mon[133625]: Reconfiguring daemon mon.a on smithi119 2023-12-07T10:12:03.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:02 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/265343595' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:03.683 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-12-07T10:12:03.683 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-12-07T10:12:03.683 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.conf 2023-12-07T10:12:03.758 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-12-07T10:12:03.758 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-07T10:12:03.866 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-12-07T10:12:03.867 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/ceph/ceph.conf 2023-12-07T10:12:03.898 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-12-07T10:12:03.899 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-07T10:12:03.967 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-12-07T10:12:03.968 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.conf 2023-12-07T10:12:04.004 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-12-07T10:12:04.004 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-07T10:12:04.074 INFO:tasks.cephadm:Adding mgr.x on smithi119 2023-12-07T10:12:04.075 INFO:tasks.cephadm:Adding mgr.y on smithi153 2023-12-07T10:12:04.075 INFO:tasks.cephadm:Adding mgr.z on smithi159 2023-12-07T10:12:04.075 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph orch apply mgr '3;smithi119=x;smithi153=y;smithi159=z' 2023-12-07T10:12:04.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:04 smithi159 ceph-mon[134726]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:04.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:04 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:04.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:04 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:04.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:04 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-07T10:12:04.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:04 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-07T10:12:04.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:04 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:04.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:04 smithi153 ceph-mon[133625]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:04.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:04 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:04.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:04 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:04.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:04 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-07T10:12:04.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:04 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-07T10:12:04.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:04 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:04.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:04 smithi119 ceph-mon[97908]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:04.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:04 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:04.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:04 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:04.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:04 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-07T10:12:04.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:04 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-07T10:12:04.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:04 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:04.705 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.c/config 2023-12-07T10:12:05.226 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:05 smithi159 ceph-mon[134726]: Reconfiguring mon.b (monmap changed)... 2023-12-07T10:12:05.226 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:05 smithi159 ceph-mon[134726]: Reconfiguring daemon mon.b on smithi153 2023-12-07T10:12:05.226 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:05 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:05.226 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:05 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:05.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:05 smithi119 ceph-mon[97908]: Reconfiguring mon.b (monmap changed)... 2023-12-07T10:12:05.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:05 smithi119 ceph-mon[97908]: Reconfiguring daemon mon.b on smithi153 2023-12-07T10:12:05.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:05 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:05.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:05 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:05.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:05 smithi153 ceph-mon[133625]: Reconfiguring mon.b (monmap changed)... 2023-12-07T10:12:05.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:05 smithi153 ceph-mon[133625]: Reconfiguring daemon mon.b on smithi153 2023-12-07T10:12:05.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:05 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:05.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:05 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:05.985 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.980+0000 7fbdfb1f3700 1 -- 172.21.15.159:0/2656872144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdf40fffa0 msgr2=0x7fbdf41003c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:12:05.986 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.980+0000 7fbdfb1f3700 1 --2- 172.21.15.159:0/2656872144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdf40fffa0 0x7fbdf41003c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbdec00b0a0 tx=0x7fbdec00b3b0 comp rx=0 tx=0).stop 2023-12-07T10:12:05.986 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.981+0000 7fbdfb1f3700 1 -- 172.21.15.159:0/2656872144 shutdown_connections 2023-12-07T10:12:05.986 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.981+0000 7fbdfb1f3700 1 --2- 172.21.15.159:0/2656872144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdf40fffa0 0x7fbdf41003c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:05.986 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.981+0000 7fbdfb1f3700 1 -- 172.21.15.159:0/2656872144 >> 172.21.15.159:0/2656872144 conn(0x7fbdf40fb560 msgr2=0x7fbdf40fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:12:05.986 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.981+0000 7fbdfb1f3700 1 -- 172.21.15.159:0/2656872144 shutdown_connections 2023-12-07T10:12:05.986 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.981+0000 7fbdfb1f3700 1 -- 172.21.15.159:0/2656872144 wait complete. 2023-12-07T10:12:05.986 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.981+0000 7fbdfb1f3700 1 Processor -- start 2023-12-07T10:12:05.986 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.981+0000 7fbdfb1f3700 1 -- start start 2023-12-07T10:12:05.986 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.981+0000 7fbdfb1f3700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdf40fffa0 0x7fbdf4069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:12:05.986 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.981+0000 7fbdfb1f3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdf419b820 0x7fbdf419dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:12:05.986 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.981+0000 7fbdfb1f3700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdf419e150 0x7fbdf41a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:12:05.987 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.981+0000 7fbdfb1f3700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf41a0b60 con 0x7fbdf40fffa0 2023-12-07T10:12:05.987 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.981+0000 7fbdfb1f3700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf41a0cd0 con 0x7fbdf419b820 2023-12-07T10:12:05.987 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.981+0000 7fbdfb1f3700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf41a0e90 con 0x7fbdf419e150 2023-12-07T10:12:05.987 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.981+0000 7fbdf9790700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdf419e150 0x7fbdf41a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:12:05.987 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.982+0000 7fbdf8f8f700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdf40fffa0 0x7fbdf4069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:12:05.987 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.982+0000 7fbdf9790700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdf419e150 0x7fbdf41a0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42460/0 (socket says 172.21.15.159:42460) 2023-12-07T10:12:05.987 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.982+0000 7fbdf9790700 1 -- 172.21.15.159:0/2285027585 learned_addr learned my addr 172.21.15.159:0/2285027585 (peer_addr_for_me v2:172.21.15.159:0/0) 2023-12-07T10:12:05.987 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.982+0000 7fbdf3fff700 1 --2- 172.21.15.159:0/2285027585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdf419b820 0x7fbdf419dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:12:05.987 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.982+0000 7fbdf9790700 1 -- 172.21.15.159:0/2285027585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdf419b820 msgr2=0x7fbdf419dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:12:05.987 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.982+0000 7fbdf9790700 1 --2- 172.21.15.159:0/2285027585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdf419b820 0x7fbdf419dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:05.987 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.982+0000 7fbdf9790700 1 -- 172.21.15.159:0/2285027585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdf40fffa0 msgr2=0x7fbdf4069760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:12:05.987 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.982+0000 7fbdf9790700 1 --2- 172.21.15.159:0/2285027585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdf40fffa0 0x7fbdf4069760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:05.988 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.982+0000 7fbdf9790700 1 -- 172.21.15.159:0/2285027585 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdec009d00 con 0x7fbdf419e150 2023-12-07T10:12:05.988 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.982+0000 7fbdf9790700 1 --2- 172.21.15.159:0/2285027585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdf419e150 0x7fbdf41a0540 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbde400b550 tx=0x7fbde400b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:12:05.988 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.982+0000 7fbdf1ffb700 1 -- 172.21.15.159:0/2285027585 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde4011830 con 0x7fbdf419e150 2023-12-07T10:12:05.988 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.982+0000 7fbdf1ffb700 1 -- 172.21.15.159:0/2285027585 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbde4011e50 con 0x7fbdf419e150 2023-12-07T10:12:05.988 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.982+0000 7fbdfb1f3700 1 -- 172.21.15.159:0/2285027585 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf41a1170 con 0x7fbdf419e150 2023-12-07T10:12:05.988 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.982+0000 7fbdf1ffb700 1 -- 172.21.15.159:0/2285027585 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde4010e40 con 0x7fbdf419e150 2023-12-07T10:12:05.988 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.982+0000 7fbdfb1f3700 1 -- 172.21.15.159:0/2285027585 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf41a16a0 con 0x7fbdf419e150 2023-12-07T10:12:05.989 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.983+0000 7fbdf1ffb700 1 -- 172.21.15.159:0/2285027585 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 13) v1 ==== 45796+0+0 (secure 0 0 0) 0x7fbde401a400 con 0x7fbdf419e150 2023-12-07T10:12:05.989 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.983+0000 7fbdf1ffb700 1 --2- 172.21.15.159:0/2285027585 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fbde0038f70 0x7fbde003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:12:05.989 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.983+0000 7fbdf1ffb700 1 -- 172.21.15.159:0/2285027585 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbde404b170 con 0x7fbdf419e150 2023-12-07T10:12:05.989 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.983+0000 7fbdfb1f3700 1 -- 172.21.15.159:0/2285027585 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf4062c40 con 0x7fbdf419e150 2023-12-07T10:12:05.989 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.984+0000 7fbdf8f8f700 1 --2- 172.21.15.159:0/2285027585 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fbde0038f70 0x7fbde003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:12:05.989 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.984+0000 7fbdf8f8f700 1 --2- 172.21.15.159:0/2285027585 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fbde0038f70 0x7fbde003b430 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbdec000f80 tx=0x7fbdec005040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:12:05.991 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:05.989+0000 7fbdf1ffb700 1 -- 172.21.15.159:0/2285027585 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fbde40178b0 con 0x7fbdf419e150 2023-12-07T10:12:06.204 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:06.200+0000 7fbdfb1f3700 1 -- 172.21.15.159:0/2285027585 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi119=x;smithi153=y;smithi159=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fbdf40fd570 con 0x7fbde0038f70 2023-12-07T10:12:06.210 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:06.207+0000 7fbdf1ffb700 1 -- 172.21.15.159:0/2285027585 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fbdf40fd570 con 0x7fbde0038f70 2023-12-07T10:12:06.211 INFO:teuthology.orchestra.run.smithi159.stdout:Scheduled mgr update... 2023-12-07T10:12:06.213 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:06.210+0000 7fbdfb1f3700 1 -- 172.21.15.159:0/2285027585 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fbde0038f70 msgr2=0x7fbde003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:12:06.213 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:06.210+0000 7fbdfb1f3700 1 --2- 172.21.15.159:0/2285027585 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fbde0038f70 0x7fbde003b430 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbdec000f80 tx=0x7fbdec005040 comp rx=0 tx=0).stop 2023-12-07T10:12:06.213 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:06.210+0000 7fbdfb1f3700 1 -- 172.21.15.159:0/2285027585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdf419e150 msgr2=0x7fbdf41a0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:12:06.213 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:06.210+0000 7fbdfb1f3700 1 --2- 172.21.15.159:0/2285027585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdf419e150 0x7fbdf41a0540 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbde400b550 tx=0x7fbde400b860 comp rx=0 tx=0).stop 2023-12-07T10:12:06.213 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:06.210+0000 7fbdfb1f3700 1 -- 172.21.15.159:0/2285027585 shutdown_connections 2023-12-07T10:12:06.213 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:06.210+0000 7fbdfb1f3700 1 --2- 172.21.15.159:0/2285027585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdf419e150 0x7fbdf41a0540 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:06.213 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:06.210+0000 7fbdfb1f3700 1 --2- 172.21.15.159:0/2285027585 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fbde0038f70 0x7fbde003b430 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:06.213 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:06.210+0000 7fbdfb1f3700 1 --2- 172.21.15.159:0/2285027585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdf40fffa0 0x7fbdf4069760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:06.213 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:06.210+0000 7fbdfb1f3700 1 --2- 172.21.15.159:0/2285027585 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdf419b820 0x7fbdf419dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:06.214 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:06.210+0000 7fbdfb1f3700 1 -- 172.21.15.159:0/2285027585 >> 172.21.15.159:0/2285027585 conn(0x7fbdf40fb560 msgr2=0x7fbdf40fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:12:06.214 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:06.210+0000 7fbdfb1f3700 1 -- 172.21.15.159:0/2285027585 shutdown_connections 2023-12-07T10:12:06.214 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:12:06.210+0000 7fbdfb1f3700 1 -- 172.21.15.159:0/2285027585 wait complete. 2023-12-07T10:12:06.384 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:06 smithi159 ceph-mon[134726]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:06.384 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:06 smithi159 ceph-mon[134726]: Reconfiguring mon.c (monmap changed)... 2023-12-07T10:12:06.384 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:06 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-07T10:12:06.384 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:06 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-07T10:12:06.384 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:06 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:06.385 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:06 smithi159 ceph-mon[134726]: Reconfiguring daemon mon.c on smithi159 2023-12-07T10:12:06.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:06 smithi119 ceph-mon[97908]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:06.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:06 smithi119 ceph-mon[97908]: Reconfiguring mon.c (monmap changed)... 2023-12-07T10:12:06.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:06 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-07T10:12:06.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:06 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-07T10:12:06.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:06 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:06.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:06 smithi119 ceph-mon[97908]: Reconfiguring daemon mon.c on smithi159 2023-12-07T10:12:06.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:06 smithi153 ceph-mon[133625]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:06.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:06 smithi153 ceph-mon[133625]: Reconfiguring mon.c (monmap changed)... 2023-12-07T10:12:06.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:06 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-07T10:12:06.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:06 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-07T10:12:06.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:06 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:06.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:06 smithi153 ceph-mon[133625]: Reconfiguring daemon mon.c on smithi159 2023-12-07T10:12:06.667 DEBUG:teuthology.orchestra.run.smithi153:mgr.y> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mgr.y.service 2023-12-07T10:12:06.669 DEBUG:teuthology.orchestra.run.smithi159:mgr.z> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mgr.z.service 2023-12-07T10:12:06.672 INFO:tasks.cephadm:Deploying OSDs... 2023-12-07T10:12:06.673 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-12-07T10:12:06.673 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/scratch_devs of=/dev/stdout 2023-12-07T10:12:06.702 INFO:journalctl@ceph.mgr.z.smithi159.stdout:-- Logs begin at Thu 2023-12-07 09:49:21 UTC. -- 2023-12-07T10:12:06.705 INFO:journalctl@ceph.mgr.y.smithi153.stdout:-- Logs begin at Thu 2023-12-07 09:49:13 UTC. -- 2023-12-07T10:12:06.722 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-07T10:12:06.722 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_1 2023-12-07T10:12:06.809 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-07T10:12:06.809 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-07T10:12:06.809 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-12-07T10:12:06.809 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-07T10:12:06.809 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2023-12-07T10:12:06.809 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2023-12-07 10:11:10.737740632 +0000 2023-12-07T10:12:06.810 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2023-12-07 10:05:23.129454079 +0000 2023-12-07T10:12:06.810 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2023-12-07 10:05:23.129454079 +0000 2023-12-07T10:12:06.810 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2023-12-07 10:05:23.100454773 +0000 2023-12-07T10:12:06.810 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-07T10:12:06.917 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2023-12-07T10:12:06.917 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2023-12-07T10:12:06.918 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000317853 s, 1.6 MB/s 2023-12-07T10:12:06.920 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-07T10:12:06.966 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_2 2023-12-07T10:12:07.050 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-07T10:12:07.050 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-07T10:12:07.050 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-12-07T10:12:07.050 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-07T10:12:07.050 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2023-12-07T10:12:07.050 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2023-12-07 10:11:10.813738803 +0000 2023-12-07T10:12:07.051 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2023-12-07 10:05:23.950434429 +0000 2023-12-07T10:12:07.051 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2023-12-07 10:05:23.950434429 +0000 2023-12-07T10:12:07.051 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2023-12-07 10:05:23.923435075 +0000 2023-12-07T10:12:07.051 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-07T10:12:07.160 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2023-12-07T10:12:07.160 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2023-12-07T10:12:07.160 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000284601 s, 1.8 MB/s 2023-12-07T10:12:07.162 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-07T10:12:07.207 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_3 2023-12-07T10:12:07.292 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-07T10:12:07.292 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-07T10:12:07.292 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-12-07T10:12:07.292 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-07T10:12:07.292 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2023-12-07T10:12:07.292 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2023-12-07 10:11:10.888736999 +0000 2023-12-07T10:12:07.292 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2023-12-07 10:05:24.754415186 +0000 2023-12-07T10:12:07.293 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2023-12-07 10:05:24.754415186 +0000 2023-12-07T10:12:07.293 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2023-12-07 10:05:24.727415833 +0000 2023-12-07T10:12:07.293 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-07T10:12:07.400 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2023-12-07T10:12:07.400 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2023-12-07T10:12:07.401 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000305216 s, 1.7 MB/s 2023-12-07T10:12:07.403 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-07T10:12:07.447 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_4 2023-12-07T10:12:07.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:07 smithi119 ceph-mon[97908]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi119=x;smithi153=y;smithi159=z", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:12:07.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:07 smithi119 ceph-mon[97908]: Saving service mgr spec with placement smithi119=x;smithi153=y;smithi159=z;count:3 2023-12-07T10:12:07.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:07 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:07.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:07 smithi119 ceph-mon[97908]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:07.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:07 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:07.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:07 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:07.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:07 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:12:07.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:07 smithi159 ceph-mon[134726]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi119=x;smithi153=y;smithi159=z", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:12:07.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:07 smithi159 ceph-mon[134726]: Saving service mgr spec with placement smithi119=x;smithi153=y;smithi159=z;count:3 2023-12-07T10:12:07.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:07 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:07.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:07 smithi159 ceph-mon[134726]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:07.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:07 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:07.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:07 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:07.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:07 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:12:07.529 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-07T10:12:07.529 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-07T10:12:07.529 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-12-07T10:12:07.529 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-07T10:12:07.529 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2023-12-07T10:12:07.529 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2023-12-07 10:11:10.960735266 +0000 2023-12-07T10:12:07.529 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2023-12-07 10:05:25.588395226 +0000 2023-12-07T10:12:07.529 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2023-12-07 10:05:25.588395226 +0000 2023-12-07T10:12:07.529 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2023-12-07 10:05:25.558395944 +0000 2023-12-07T10:12:07.530 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-07T10:12:07.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:07 smithi153 ceph-mon[133625]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi119=x;smithi153=y;smithi159=z", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:12:07.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:07 smithi153 ceph-mon[133625]: Saving service mgr spec with placement smithi119=x;smithi153=y;smithi159=z;count:3 2023-12-07T10:12:07.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:07 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:07.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:07 smithi153 ceph-mon[133625]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:07.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:07 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:07.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:07 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:07.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:07 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:12:07.601 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2023-12-07T10:12:07.601 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2023-12-07T10:12:07.601 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000283011 s, 1.8 MB/s 2023-12-07T10:12:07.603 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-07T10:12:07.649 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-12-07T10:12:07.649 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/scratch_devs of=/dev/stdout 2023-12-07T10:12:07.670 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-07T10:12:07.670 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_1 2023-12-07T10:12:07.731 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-07T10:12:07.731 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-07T10:12:07.731 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 6h/6d Inode: 386483 Links: 1 2023-12-07T10:12:07.731 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-07T10:12:07.731 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2023-12-07T10:12:07.731 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2023-12-07 10:11:53.320353695 +0000 2023-12-07T10:12:07.732 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2023-12-07 10:03:14.183088015 +0000 2023-12-07T10:12:07.732 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2023-12-07 10:03:14.183088015 +0000 2023-12-07T10:12:07.732 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: - 2023-12-07T10:12:07.732 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-07T10:12:07.801 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2023-12-07T10:12:07.801 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2023-12-07T10:12:07.801 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000227298 s, 2.3 MB/s 2023-12-07T10:12:07.802 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-07T10:12:07.865 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_2 2023-12-07T10:12:07.924 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-07T10:12:07.924 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-07T10:12:07.924 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 6h/6d Inode: 390548 Links: 1 2023-12-07T10:12:07.924 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-07T10:12:07.924 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2023-12-07T10:12:07.924 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2023-12-07 10:11:53.320353695 +0000 2023-12-07T10:12:07.924 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2023-12-07 10:03:14.712078744 +0000 2023-12-07T10:12:07.924 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2023-12-07 10:03:14.712078744 +0000 2023-12-07T10:12:07.924 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: - 2023-12-07T10:12:07.924 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-07T10:12:07.991 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2023-12-07T10:12:07.991 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2023-12-07T10:12:07.992 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000233765 s, 2.2 MB/s 2023-12-07T10:12:07.993 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-07T10:12:08.054 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_3 2023-12-07T10:12:08.112 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-07T10:12:08.112 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-07T10:12:08.112 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 6h/6d Inode: 383873 Links: 1 2023-12-07T10:12:08.112 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-07T10:12:08.112 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2023-12-07T10:12:08.112 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2023-12-07 10:11:53.320353695 +0000 2023-12-07T10:12:08.112 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2023-12-07 10:03:15.241069472 +0000 2023-12-07T10:12:08.112 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2023-12-07 10:03:15.241069472 +0000 2023-12-07T10:12:08.113 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: - 2023-12-07T10:12:08.113 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-07T10:12:08.179 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2023-12-07T10:12:08.179 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2023-12-07T10:12:08.179 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000226848 s, 2.3 MB/s 2023-12-07T10:12:08.181 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-07T10:12:08.239 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_4 2023-12-07T10:12:08.297 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-07T10:12:08.297 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-07T10:12:08.297 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 6h/6d Inode: 385700 Links: 1 2023-12-07T10:12:08.297 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-07T10:12:08.297 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2023-12-07T10:12:08.297 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2023-12-07 10:11:53.320353695 +0000 2023-12-07T10:12:08.297 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2023-12-07 10:03:15.772060165 +0000 2023-12-07T10:12:08.297 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2023-12-07 10:03:15.772060165 +0000 2023-12-07T10:12:08.297 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: - 2023-12-07T10:12:08.297 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-07T10:12:08.363 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2023-12-07T10:12:08.363 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2023-12-07T10:12:08.363 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000177555 s, 2.9 MB/s 2023-12-07T10:12:08.365 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-07T10:12:08.422 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2023-12-07T10:12:08.423 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/scratch_devs of=/dev/stdout 2023-12-07T10:12:08.440 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-07T10:12:08.440 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_1 2023-12-07T10:12:08.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:08 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:08.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:08 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:12:08.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:08 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:08.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:08 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-07T10:12:08.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:08 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-07T10:12:08.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:08 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-07T10:12:08.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:08 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:08.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:08 smithi119 ceph-mon[97908]: Deploying daemon mgr.z on smithi159 2023-12-07T10:12:08.498 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-07T10:12:08.498 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-07T10:12:08.498 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 6h/6d Inode: 389622 Links: 1 2023-12-07T10:12:08.498 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-07T10:12:08.498 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2023-12-07T10:12:08.498 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2023-12-07 10:11:44.887664004 +0000 2023-12-07T10:12:08.498 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2023-12-07 10:03:05.005122714 +0000 2023-12-07T10:12:08.498 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2023-12-07 10:03:05.005122714 +0000 2023-12-07T10:12:08.498 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: - 2023-12-07T10:12:08.499 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-07T10:12:08.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:08 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:08.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:08 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:12:08.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:08 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:08.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:08 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-07T10:12:08.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:08 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-07T10:12:08.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:08 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-07T10:12:08.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:08 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:08.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:08 smithi159 ceph-mon[134726]: Deploying daemon mgr.z on smithi159 2023-12-07T10:12:08.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:08 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:08.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:08 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:12:08.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:08 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:08.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:08 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-07T10:12:08.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:08 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-07T10:12:08.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:08 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-07T10:12:08.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:08 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:08.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:08 smithi153 ceph-mon[133625]: Deploying daemon mgr.z on smithi159 2023-12-07T10:12:08.541 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2023-12-07T10:12:08.541 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2023-12-07T10:12:08.541 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.00026606 s, 1.9 MB/s 2023-12-07T10:12:08.542 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-07T10:12:08.601 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_2 2023-12-07T10:12:08.657 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-07T10:12:08.658 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-07T10:12:08.658 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 6h/6d Inode: 387728 Links: 1 2023-12-07T10:12:08.658 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-07T10:12:08.658 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2023-12-07T10:12:08.658 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2023-12-07 10:11:44.887664004 +0000 2023-12-07T10:12:08.658 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2023-12-07 10:03:05.526113004 +0000 2023-12-07T10:12:08.658 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2023-12-07 10:03:05.526113004 +0000 2023-12-07T10:12:08.658 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: - 2023-12-07T10:12:08.659 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-07T10:12:08.723 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2023-12-07T10:12:08.724 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2023-12-07T10:12:08.724 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000225197 s, 2.3 MB/s 2023-12-07T10:12:08.725 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-07T10:12:08.783 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_3 2023-12-07T10:12:08.850 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-07T10:12:08.850 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-07T10:12:08.850 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 6h/6d Inode: 392443 Links: 1 2023-12-07T10:12:08.850 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-07T10:12:08.850 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2023-12-07T10:12:08.851 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2023-12-07 10:11:44.887664004 +0000 2023-12-07T10:12:08.851 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2023-12-07 10:03:06.031103592 +0000 2023-12-07T10:12:08.851 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2023-12-07 10:03:06.031103592 +0000 2023-12-07T10:12:08.851 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: - 2023-12-07T10:12:08.852 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-07T10:12:08.918 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2023-12-07T10:12:08.918 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2023-12-07T10:12:08.919 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000243911 s, 2.1 MB/s 2023-12-07T10:12:08.919 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-07T10:12:08.987 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_4 2023-12-07T10:12:09.044 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-07T10:12:09.044 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-07T10:12:09.044 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 6h/6d Inode: 388592 Links: 1 2023-12-07T10:12:09.044 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-07T10:12:09.044 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2023-12-07T10:12:09.044 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2023-12-07 10:12:08.998209622 +0000 2023-12-07T10:12:09.044 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2023-12-07 10:03:06.528094328 +0000 2023-12-07T10:12:09.044 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2023-12-07 10:03:06.528094328 +0000 2023-12-07T10:12:09.045 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: - 2023-12-07T10:12:09.045 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-07T10:12:09.112 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:09 smithi159 systemd[1]: Starting Ceph mgr.z for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:12:09.115 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2023-12-07T10:12:09.115 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2023-12-07T10:12:09.115 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.00020505 s, 2.5 MB/s 2023-12-07T10:12:09.117 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-07T10:12:09.177 INFO:tasks.cephadm:Deploying osd.0 on smithi119 with /dev/vg_nvme/lv_4... 2023-12-07T10:12:09.177 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-12-07T10:12:09.362 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:09 smithi159 ceph-mon[134726]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:09.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:09 smithi119 ceph-mon[97908]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:09.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:09 smithi153 ceph-mon[133625]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:09.639 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:09 smithi159 podman[136818]: 2023-12-07T10:12:09.639 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:09 smithi159 bash[136818]: d5c6b8816696f976629a5853fffeb0865a51adf963337ed9d8db2905298c1e08 2023-12-07T10:12:09.640 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:09 smithi159 ceph-mgr[136859]: -- 172.21.15.159:0/187292753 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5652e3e601a0 con 0x5652e3e36c00 2023-12-07T10:12:09.894 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:09 smithi159 systemd[1]: Started Ceph mgr.z for a4c642c6-94e8-11ee-95a2-87774f69a715. 2023-12-07T10:12:09.894 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:09 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:12:09.721+0000 7fb9973e8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-07T10:12:09.894 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:09 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:12:09.802+0000 7fb9973e8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-07T10:12:09.894 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:09 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:12:09.890+0000 7fb9973e8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-07T10:12:09.947 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:12:10.153 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:09 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:12:09.983+0000 7fb9973e8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-07T10:12:10.514 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:10 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:12:10.150+0000 7fb9973e8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-07T10:12:10.788 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:10 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:10.788 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:10 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:10.788 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:10 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:10.788 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:10 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-07T10:12:10.788 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:10 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-07T10:12:10.788 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:10 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-07T10:12:10.788 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:10 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:10.788 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:10 smithi119 ceph-mon[97908]: Deploying daemon mgr.y on smithi153 2023-12-07T10:12:10.826 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:10 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:12:10.512+0000 7fb9973e8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-07T10:12:10.826 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:10 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:12:10.641+0000 7fb9973e8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-07T10:12:10.826 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:10 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:12:10.729+0000 7fb9973e8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-07T10:12:10.826 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:10 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:12:10.823+0000 7fb9973e8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-07T10:12:10.827 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:10 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:10.827 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:10 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:10.827 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:10 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:10.827 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:10 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-07T10:12:10.827 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:10 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-07T10:12:10.827 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:10 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-07T10:12:10.827 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:10 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:10.827 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:10 smithi159 ceph-mon[134726]: Deploying daemon mgr.y on smithi153 2023-12-07T10:12:10.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:10 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:10.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:10 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:10.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:10 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:10.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:10 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-07T10:12:10.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:10 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-07T10:12:10.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:10 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-07T10:12:10.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:10 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:10.938 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:10 smithi153 ceph-mon[133625]: Deploying daemon mgr.y on smithi153 2023-12-07T10:12:11.256 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:11 smithi153 systemd[1]: Starting Ceph mgr.y for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:12:11.514 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:11 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:12:11.241+0000 7fb9973e8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-07T10:12:11.843 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:11 smithi153 ceph-mon[133625]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:11.843 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:11 smithi153 podman[135063]: 2023-12-07T10:12:11.843 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:11 smithi153 ceph-mgr[135104]: -- 172.21.15.153:0/1146826055 <== mon.2 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55dfb423c1a0 con 0x55dfb4212c00 2023-12-07T10:12:11.843 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:11 smithi153 bash[135063]: 78ec1f418011ed11d81c8d13030f995a22a547ab294e8168204d68cb2f7de99a 2023-12-07T10:12:11.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:11 smithi119 ceph-mon[97908]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:11.996 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:11 smithi159 ceph-mon[134726]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:11.997 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:11 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:12:11.910+0000 7fb9973e8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-07T10:12:12.069 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-12-07T10:12:12.069 INFO:teuthology.orchestra.run.smithi119.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-12-07T10:12:12.069 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10+0 records in 2023-12-07T10:12:12.069 INFO:teuthology.orchestra.run.smithi119.stderr:10+0 records out 2023-12-07T10:12:12.069 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0513503 s, 204 MB/s 2023-12-07T10:12:12.069 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping successful for: 2023-12-07T10:12:12.175 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:11 smithi153 systemd[1]: Started Ceph mgr.y for a4c642c6-94e8-11ee-95a2-87774f69a715. 2023-12-07T10:12:12.176 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:11 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:12:11.902+0000 7f3ac5bde200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-07T10:12:12.176 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:11 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:12:11.983+0000 7f3ac5bde200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-07T10:12:12.176 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:12 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:12:12.072+0000 7f3ac5bde200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-07T10:12:12.176 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:12 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:12:12.173+0000 7f3ac5bde200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-07T10:12:12.264 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:11 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:12:11.993+0000 7fb9973e8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-07T10:12:12.531 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:12 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:12:12.358+0000 7f3ac5bde200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-07T10:12:12.637 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi119:vg_nvme/lv_4 2023-12-07T10:12:13.031 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:12 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:12:12.743+0000 7f3ac5bde200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-07T10:12:13.031 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:12 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:12:12.879+0000 7f3ac5bde200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-07T10:12:13.031 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:12 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:12:12.969+0000 7f3ac5bde200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-07T10:12:13.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:12 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:13.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:12 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:13.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:12 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:13.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:12 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:13.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:12 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:12:13.080 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:12 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:13.080 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:12 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:13.081 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:12 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:13.081 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:12 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:13.081 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:12 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:12:13.081 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:12 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:12:12.795+0000 7fb9973e8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-07T10:12:13.081 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:12 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:12:12.896+0000 7fb9973e8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-07T10:12:13.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:12 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:13.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:12 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:13.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:12 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:13.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:12 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:13.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:12 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:12:13.411 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:12:13.514 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:13 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:12:13.077+0000 7fb9973e8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-07T10:12:13.525 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:13 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:12:13.070+0000 7f3ac5bde200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-07T10:12:13.966 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:13 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:12:13.520+0000 7fb9973e8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-07T10:12:13.967 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:13 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:12:13.609+0000 7fb9973e8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-07T10:12:13.967 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:13 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:12:13.759+0000 7fb9973e8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-07T10:12:13.967 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:13 smithi119 ceph-mon[97908]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:13.967 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:13.967 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:14.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:14.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:14.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:14.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:12:14.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:14.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-07T10:12:14.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-07T10:12:14.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:14.264 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:14 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:12:14.026+0000 7fb9973e8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-07T10:12:14.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:13 smithi159 ceph-mon[134726]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:14.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:13 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:14.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:13 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:14.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:13 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:14.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:13 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:14.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:13 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:14.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:13 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:12:14.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:13 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:14.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:13 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-07T10:12:14.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:13 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-07T10:12:14.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:13 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:14.281 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:13 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:12:13.522+0000 7f3ac5bde200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-07T10:12:14.281 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:14 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:12:14.211+0000 7f3ac5bde200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-07T10:12:14.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:13 smithi153 ceph-mon[133625]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:14.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:13 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:14.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:13 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:14.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:13 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:14.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:13 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:14.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:13 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:14.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:13 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:12:14.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:13 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:14.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:13 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-07T10:12:14.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:13 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-07T10:12:14.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:13 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:14.781 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:14 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:12:14.292+0000 7f3ac5bde200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-07T10:12:14.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.980+0000 7f2c51ebe700 1 -- 172.21.15.119:0/1893203607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c4c103200 msgr2=0x7f2c4c109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:12:14.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.980+0000 7f2c51ebe700 1 --2- 172.21.15.119:0/1893203607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c4c103200 0x7f2c4c109ab0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2c44009fc0 tx=0x7f2c4400b330 comp rx=0 tx=0).stop 2023-12-07T10:12:14.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.980+0000 7f2c51ebe700 1 -- 172.21.15.119:0/1893203607 shutdown_connections 2023-12-07T10:12:14.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.980+0000 7f2c51ebe700 1 --2- 172.21.15.119:0/1893203607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c4c103200 0x7f2c4c109ab0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:14.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.980+0000 7f2c51ebe700 1 --2- 172.21.15.119:0/1893203607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c4c101660 0x7f2c4c101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:14.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.980+0000 7f2c51ebe700 1 --2- 172.21.15.119:0/1893203607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c4c102860 0x7f2c4c102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:14.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.980+0000 7f2c51ebe700 1 -- 172.21.15.119:0/1893203607 >> 172.21.15.119:0/1893203607 conn(0x7f2c4c0fcc20 msgr2=0x7f2c4c0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:12:14.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.980+0000 7f2c51ebe700 1 -- 172.21.15.119:0/1893203607 shutdown_connections 2023-12-07T10:12:14.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.980+0000 7f2c51ebe700 1 -- 172.21.15.119:0/1893203607 wait complete. 2023-12-07T10:12:14.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.981+0000 7f2c51ebe700 1 Processor -- start 2023-12-07T10:12:14.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.981+0000 7f2c51ebe700 1 -- start start 2023-12-07T10:12:14.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.981+0000 7f2c51ebe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c4c101660 0x7f2c4c199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:12:14.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.981+0000 7f2c51ebe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c4c102860 0x7f2c4c199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:12:14.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.981+0000 7f2c51ebe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c4c103200 0x7f2c4c1a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:12:14.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.981+0000 7f2c51ebe700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c1a13b0 con 0x7f2c4c103200 2023-12-07T10:12:14.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.981+0000 7f2c51ebe700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c1a14f0 con 0x7f2c4c101660 2023-12-07T10:12:14.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.981+0000 7f2c51ebe700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c1a1680 con 0x7f2c4c102860 2023-12-07T10:12:14.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.982+0000 7f2c4affd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c4c102860 0x7f2c4c199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:12:14.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.982+0000 7f2c4affd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c4c102860 0x7f2c4c199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41010/0 (socket says 172.21.15.119:41010) 2023-12-07T10:12:14.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.982+0000 7f2c4affd700 1 -- 172.21.15.119:0/1867364921 learned_addr learned my addr 172.21.15.119:0/1867364921 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:12:14.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.982+0000 7f2c4affd700 1 -- 172.21.15.119:0/1867364921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c4c101660 msgr2=0x7f2c4c199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:12:14.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.982+0000 7f2c4affd700 1 --2- 172.21.15.119:0/1867364921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c4c101660 0x7f2c4c199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:14.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.982+0000 7f2c4affd700 1 -- 172.21.15.119:0/1867364921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c4c103200 msgr2=0x7f2c4c1a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:12:14.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.982+0000 7f2c4bfff700 1 --2- 172.21.15.119:0/1867364921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c4c103200 0x7f2c4c1a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:12:14.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.982+0000 7f2c4affd700 1 --2- 172.21.15.119:0/1867364921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c4c103200 0x7f2c4c1a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:14.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.982+0000 7f2c4affd700 1 -- 172.21.15.119:0/1867364921 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3c0095f0 con 0x7f2c4c102860 2023-12-07T10:12:14.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.982+0000 7f2c4bfff700 1 --2- 172.21.15.119:0/1867364921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c4c103200 0x7f2c4c1a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:12:14.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.982+0000 7f2c4affd700 1 --2- 172.21.15.119:0/1867364921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c4c102860 0x7f2c4c199db0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000e9d0 tx=0x7f2c4000ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:12:14.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.982+0000 7f2c48ff9700 1 -- 172.21.15.119:0/1867364921 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c40009790 con 0x7f2c4c102860 2023-12-07T10:12:14.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.983+0000 7f2c48ff9700 1 -- 172.21.15.119:0/1867364921 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c4000cbd0 con 0x7f2c4c102860 2023-12-07T10:12:14.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.983+0000 7f2c51ebe700 1 -- 172.21.15.119:0/1867364921 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c44009c40 con 0x7f2c4c102860 2023-12-07T10:12:14.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.983+0000 7f2c48ff9700 1 -- 172.21.15.119:0/1867364921 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c40018680 con 0x7f2c4c102860 2023-12-07T10:12:14.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.983+0000 7f2c51ebe700 1 -- 172.21.15.119:0/1867364921 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c4c1a1cd0 con 0x7f2c4c102860 2023-12-07T10:12:14.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.984+0000 7f2c48ff9700 1 -- 172.21.15.119:0/1867364921 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 13) v1 ==== 45796+0+0 (secure 0 0 0) 0x7f2c400187e0 con 0x7f2c4c102860 2023-12-07T10:12:14.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.984+0000 7f2c51ebe700 1 -- 172.21.15.119:0/1867364921 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c4c04f2d0 con 0x7f2c4c102860 2023-12-07T10:12:14.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.984+0000 7f2c48ff9700 1 --2- 172.21.15.119:0/1867364921 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2c34038d20 0x7f2c3403b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:12:14.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.984+0000 7f2c48ff9700 1 -- 172.21.15.119:0/1867364921 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2c40014070 con 0x7f2c4c102860 2023-12-07T10:12:14.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.984+0000 7f2c4b7fe700 1 --2- 172.21.15.119:0/1867364921 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2c34038d20 0x7f2c3403b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:12:14.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.985+0000 7f2c4b7fe700 1 --2- 172.21.15.119:0/1867364921 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2c34038d20 0x7f2c3403b1e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0095c0 tx=0x7f2c3c009510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:12:14.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:14.989+0000 7f2c48ff9700 1 -- 172.21.15.119:0/1867364921 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2c4001f730 con 0x7f2c4c102860 2023-12-07T10:12:15.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:14 smithi159 ceph-mon[134726]: Reconfiguring mgr.x (unknown last config time)... 2023-12-07T10:12:15.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:14 smithi159 ceph-mon[134726]: Reconfiguring daemon mgr.x on smithi119 2023-12-07T10:12:15.015 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:14 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:12:14.720+0000 7fb9973e8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-07T10:12:15.015 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:14 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:12:14.850+0000 7fb9973e8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-07T10:12:15.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:15.217+0000 7f2c51ebe700 1 -- 172.21.15.119:0/1867364921 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c4c074f60 con 0x7f2c34038d20 2023-12-07T10:12:15.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:14 smithi119 ceph-mon[97908]: Reconfiguring mgr.x (unknown last config time)... 2023-12-07T10:12:15.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:14 smithi119 ceph-mon[97908]: Reconfiguring daemon mgr.x on smithi119 2023-12-07T10:12:15.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:14 smithi153 ceph-mon[133625]: Reconfiguring mgr.x (unknown last config time)... 2023-12-07T10:12:15.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:14 smithi153 ceph-mon[133625]: Reconfiguring daemon mgr.x on smithi119 2023-12-07T10:12:15.281 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:15 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:12:15.065+0000 7f3ac5bde200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-07T10:12:15.281 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:15 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:12:15.154+0000 7f3ac5bde200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-07T10:12:15.374 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:15 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:12:15.044+0000 7fb9973e8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-07T10:12:15.375 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:15 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:12:15.155+0000 7fb9973e8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-07T10:12:15.757 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:15 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:12:15.325+0000 7f3ac5bde200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-07T10:12:16.031 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:15 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:12:15.753+0000 7f3ac5bde200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-07T10:12:16.031 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:15 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:12:15.841+0000 7f3ac5bde200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-07T10:12:16.031 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:15 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:12:15.992+0000 7f3ac5bde200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-07T10:12:16.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:15 smithi153 ceph-mon[133625]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:16.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:15 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:12:16.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:15 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:12:16.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:15 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:16.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:15 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:16.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:15 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:16.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:15 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:16.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:15 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:12:16.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:15 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:16.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:15 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:12:16.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:15 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:16.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:15 smithi119 ceph-mon[97908]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:16.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:15 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:12:16.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:15 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:12:16.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:15 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:16.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:15 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:16.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:15 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:16.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:15 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:16.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:15 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:12:16.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:15 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:16.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:15 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:12:16.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:15 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:16.240 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:15 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:12:15.371+0000 7fb9973e8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-07T10:12:16.240 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:15 smithi159 ceph-mon[134726]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:16.241 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:15 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:12:16.241 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:15 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:12:16.241 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:15 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:16.241 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:15 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:16.241 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:15 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:16.241 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:15 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:16.241 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:15 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:12:16.241 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:15 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:16.241 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:15 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:12:16.241 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:15 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:16.514 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:12:16 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:12:16.237+0000 7fb9973e8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-07T10:12:16.531 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:16 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:12:16.256+0000 7f3ac5bde200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-07T10:12:16.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:16.983+0000 7f2c48ff9700 1 -- 172.21.15.119:0/1867364921 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mgrmap(e 14) v1 ==== 91112+0+0 (secure 0 0 0) 0x7f2c4001fd30 con 0x7f2c4c102860 2023-12-07T10:12:17.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:16 smithi159 ceph-mon[134726]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:12:17.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:16 smithi159 ceph-mon[134726]: Standby manager daemon z started 2023-12-07T10:12:17.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:16 smithi159 ceph-mon[134726]: from='mgr.? 172.21.15.159:0/2157851835' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-07T10:12:17.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:16 smithi159 ceph-mon[134726]: from='mgr.? 172.21.15.159:0/2157851835' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-07T10:12:17.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:16 smithi159 ceph-mon[134726]: from='mgr.? 172.21.15.159:0/2157851835' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-07T10:12:17.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:16 smithi159 ceph-mon[134726]: from='mgr.? 172.21.15.159:0/2157851835' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-07T10:12:17.281 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:16 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:12:16.957+0000 7f3ac5bde200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-07T10:12:17.281 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:17 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:12:17.091+0000 7f3ac5bde200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-07T10:12:17.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:16 smithi153 ceph-mon[133625]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:12:17.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:16 smithi153 ceph-mon[133625]: Standby manager daemon z started 2023-12-07T10:12:17.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:16 smithi153 ceph-mon[133625]: from='mgr.? 172.21.15.159:0/2157851835' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-07T10:12:17.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:16 smithi153 ceph-mon[133625]: from='mgr.? 172.21.15.159:0/2157851835' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-07T10:12:17.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:16 smithi153 ceph-mon[133625]: from='mgr.? 172.21.15.159:0/2157851835' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-07T10:12:17.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:16 smithi153 ceph-mon[133625]: from='mgr.? 172.21.15.159:0/2157851835' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-07T10:12:17.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:16 smithi119 ceph-mon[97908]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:12:17.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:16 smithi119 ceph-mon[97908]: Standby manager daemon z started 2023-12-07T10:12:17.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:16 smithi119 ceph-mon[97908]: from='mgr.? 172.21.15.159:0/2157851835' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-07T10:12:17.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:16 smithi119 ceph-mon[97908]: from='mgr.? 172.21.15.159:0/2157851835' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-07T10:12:17.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:16 smithi119 ceph-mon[97908]: from='mgr.? 172.21.15.159:0/2157851835' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-07T10:12:17.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:16 smithi119 ceph-mon[97908]: from='mgr.? 172.21.15.159:0/2157851835' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-07T10:12:17.615 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:17 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:12:17.282+0000 7f3ac5bde200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-07T10:12:17.615 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:17 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:12:17.394+0000 7f3ac5bde200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-07T10:12:17.984 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:17 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:12:17.612+0000 7f3ac5bde200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-07T10:12:18.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:17 smithi159 ceph-mon[134726]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:18.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:17 smithi159 ceph-mon[134726]: mgrmap e14: x(active, since 76s), standbys: z 2023-12-07T10:12:18.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:17 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-07T10:12:18.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:17 smithi153 ceph-mon[133625]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:18.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:17 smithi153 ceph-mon[133625]: mgrmap e14: x(active, since 76s), standbys: z 2023-12-07T10:12:18.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:17 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-07T10:12:18.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:17 smithi119 ceph-mon[97908]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:18.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:17 smithi119 ceph-mon[97908]: mgrmap e14: x(active, since 76s), standbys: z 2023-12-07T10:12:18.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:17 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-07T10:12:18.781 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:12:18 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:12:18.484+0000 7f3ac5bde200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-07T10:12:19.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:18 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/540303182' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56b5818e-e625-40f1-8838-3ddcf2d34ea7"}]: dispatch 2023-12-07T10:12:19.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:18 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/540303182' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56b5818e-e625-40f1-8838-3ddcf2d34ea7"}]': finished 2023-12-07T10:12:19.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:18 smithi159 ceph-mon[134726]: osdmap e5: 1 total, 0 up, 1 in 2023-12-07T10:12:19.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:18 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T10:12:19.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:18 smithi159 ceph-mon[134726]: Standby manager daemon y started 2023-12-07T10:12:19.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:18 smithi159 ceph-mon[134726]: from='mgr.? 172.21.15.153:0/73720057' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-07T10:12:19.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:18 smithi159 ceph-mon[134726]: from='mgr.? 172.21.15.153:0/73720057' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-07T10:12:19.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:18 smithi159 ceph-mon[134726]: from='mgr.? 172.21.15.153:0/73720057' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-07T10:12:19.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:18 smithi159 ceph-mon[134726]: from='mgr.? 172.21.15.153:0/73720057' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-07T10:12:19.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:18 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/540303182' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56b5818e-e625-40f1-8838-3ddcf2d34ea7"}]: dispatch 2023-12-07T10:12:19.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:18 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/540303182' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56b5818e-e625-40f1-8838-3ddcf2d34ea7"}]': finished 2023-12-07T10:12:19.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:18 smithi153 ceph-mon[133625]: osdmap e5: 1 total, 0 up, 1 in 2023-12-07T10:12:19.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:18 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T10:12:19.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:18 smithi153 ceph-mon[133625]: Standby manager daemon y started 2023-12-07T10:12:19.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:18 smithi153 ceph-mon[133625]: from='mgr.? 172.21.15.153:0/73720057' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-07T10:12:19.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:18 smithi153 ceph-mon[133625]: from='mgr.? 172.21.15.153:0/73720057' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-07T10:12:19.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:18 smithi153 ceph-mon[133625]: from='mgr.? 172.21.15.153:0/73720057' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-07T10:12:19.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:18 smithi153 ceph-mon[133625]: from='mgr.? 172.21.15.153:0/73720057' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-07T10:12:19.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:19.468+0000 7f2c48ff9700 1 -- 172.21.15.119:0/1867364921 <== mon.1 v2:172.21.15.159:3300/0 8 ==== mgrmap(e 15) v1 ==== 136428+0+0 (secure 0 0 0) 0x7f2c400173d0 con 0x7f2c4c102860 2023-12-07T10:12:19.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:18 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/540303182' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56b5818e-e625-40f1-8838-3ddcf2d34ea7"}]: dispatch 2023-12-07T10:12:19.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:18 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/540303182' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56b5818e-e625-40f1-8838-3ddcf2d34ea7"}]': finished 2023-12-07T10:12:19.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:18 smithi119 ceph-mon[97908]: osdmap e5: 1 total, 0 up, 1 in 2023-12-07T10:12:19.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:18 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T10:12:19.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:18 smithi119 ceph-mon[97908]: Standby manager daemon y started 2023-12-07T10:12:19.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:18 smithi119 ceph-mon[97908]: from='mgr.? 172.21.15.153:0/73720057' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-07T10:12:19.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:18 smithi119 ceph-mon[97908]: from='mgr.? 172.21.15.153:0/73720057' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-07T10:12:19.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:18 smithi119 ceph-mon[97908]: from='mgr.? 172.21.15.153:0/73720057' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-07T10:12:19.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:18 smithi119 ceph-mon[97908]: from='mgr.? 172.21.15.153:0/73720057' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-07T10:12:20.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:19 smithi159 ceph-mon[134726]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:20.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:19 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/216968256' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:12:20.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:19 smithi159 ceph-mon[134726]: mgrmap e15: x(active, since 78s), standbys: z, y 2023-12-07T10:12:20.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:19 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-07T10:12:20.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:19 smithi153 ceph-mon[133625]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:20.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:19 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/216968256' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:12:20.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:19 smithi153 ceph-mon[133625]: mgrmap e15: x(active, since 78s), standbys: z, y 2023-12-07T10:12:20.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:19 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-07T10:12:20.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:19 smithi119 ceph-mon[97908]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:20.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:19 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/216968256' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:12:20.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:19 smithi119 ceph-mon[97908]: mgrmap e15: x(active, since 78s), standbys: z, y 2023-12-07T10:12:20.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:19 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-07T10:12:21.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:21 smithi119 ceph-mon[97908]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:21.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:21 smithi159 ceph-mon[134726]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:21.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:21 smithi153 ceph-mon[133625]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:23.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:23 smithi119 ceph-mon[97908]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:24.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:23 smithi159 ceph-mon[134726]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:24.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:23 smithi153 ceph-mon[133625]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:25.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:25 smithi119 ceph-mon[97908]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:26.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:25 smithi159 ceph-mon[134726]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:26.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:25 smithi153 ceph-mon[133625]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:26.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:26 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-07T10:12:26.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:26 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:26.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:26 smithi119 ceph-mon[97908]: Deploying daemon osd.0 on smithi119 2023-12-07T10:12:27.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:26 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-07T10:12:27.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:26 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:27.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:26 smithi159 ceph-mon[134726]: Deploying daemon osd.0 on smithi119 2023-12-07T10:12:27.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:26 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-07T10:12:27.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:26 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:27.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:26 smithi153 ceph-mon[133625]: Deploying daemon osd.0 on smithi119 2023-12-07T10:12:27.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:27 smithi119 ceph-mon[97908]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:28.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:27 smithi159 ceph-mon[134726]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:28.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:27 smithi153 ceph-mon[133625]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:29.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:29 smithi119 ceph-mon[97908]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:30.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:29 smithi159 ceph-mon[134726]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:30.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:29 smithi153 ceph-mon[133625]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:31.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:31 smithi119 ceph-mon[97908]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:31.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:31 smithi159 ceph-mon[134726]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:31.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:31 smithi153 ceph-mon[133625]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:32.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:32 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:12:32.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:32 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:32.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:32 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:32.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:32 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:12:32.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:32 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:32.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:32 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:32.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:32 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:12:32.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:32 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:32.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:32 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:33.761 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:33 smithi119 ceph-mon[97908]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:33.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:33 smithi159 ceph-mon[134726]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:33.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:33 smithi153 ceph-mon[133625]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:35.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:35.975+0000 7f2c48ff9700 1 -- 172.21.15.119:0/1867364921 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2c4c074f60 con 0x7f2c34038d20 2023-12-07T10:12:35.979 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 0 on host 'smithi119' 2023-12-07T10:12:35.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:35.978+0000 7f2c51ebe700 1 -- 172.21.15.119:0/1867364921 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2c34038d20 msgr2=0x7f2c3403b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:12:35.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:35.978+0000 7f2c51ebe700 1 --2- 172.21.15.119:0/1867364921 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2c34038d20 0x7f2c3403b1e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0095c0 tx=0x7f2c3c009510 comp rx=0 tx=0).stop 2023-12-07T10:12:35.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:35.978+0000 7f2c51ebe700 1 -- 172.21.15.119:0/1867364921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c4c102860 msgr2=0x7f2c4c199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:12:35.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:35.978+0000 7f2c51ebe700 1 --2- 172.21.15.119:0/1867364921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c4c102860 0x7f2c4c199db0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000e9d0 tx=0x7f2c4000ed90 comp rx=0 tx=0).stop 2023-12-07T10:12:35.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:35.978+0000 7f2c51ebe700 1 -- 172.21.15.119:0/1867364921 shutdown_connections 2023-12-07T10:12:35.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:35.978+0000 7f2c51ebe700 1 --2- 172.21.15.119:0/1867364921 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2c34038d20 0x7f2c3403b1e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:35.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:35.978+0000 7f2c51ebe700 1 --2- 172.21.15.119:0/1867364921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c4c103200 0x7f2c4c1a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:35.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:35.978+0000 7f2c51ebe700 1 --2- 172.21.15.119:0/1867364921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2c4c101660 0x7f2c4c199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:35.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:35.978+0000 7f2c51ebe700 1 --2- 172.21.15.119:0/1867364921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c4c102860 0x7f2c4c199db0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:35.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:35.978+0000 7f2c51ebe700 1 -- 172.21.15.119:0/1867364921 >> 172.21.15.119:0/1867364921 conn(0x7f2c4c0fcc20 msgr2=0x7f2c4c073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:12:35.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:35.978+0000 7f2c51ebe700 1 -- 172.21.15.119:0/1867364921 shutdown_connections 2023-12-07T10:12:35.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:12:35.978+0000 7f2c51ebe700 1 -- 172.21.15.119:0/1867364921 wait complete. 2023-12-07T10:12:36.122 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:35 smithi119 ceph-mon[97908]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:36.122 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:35 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:36.122 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:35 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:36.122 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:35 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:36.122 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:35 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:12:36.122 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:35 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:36.122 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:35 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:36.122 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:35 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:12:36.122 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:35 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:36.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:35 smithi159 ceph-mon[134726]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:36.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:35 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:36.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:35 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:36.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:35 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:36.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:35 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:12:36.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:35 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:36.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:35 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:36.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:35 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:12:36.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:35 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:36.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:35 smithi153 ceph-mon[133625]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:36.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:35 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:36.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:35 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:36.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:35 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:36.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:35 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:12:36.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:35 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:36.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:35 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:36.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:35 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:12:36.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:35 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:36.467 DEBUG:teuthology.orchestra.run.smithi119:osd.0> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.0.service 2023-12-07T10:12:36.471 INFO:tasks.cephadm:Deploying osd.1 on smithi153 with /dev/vg_nvme/lv_4... 2023-12-07T10:12:36.472 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-12-07T10:12:36.541 INFO:journalctl@ceph.osd.0.smithi119.stdout:-- Logs begin at Thu 2023-12-07 09:53:13 UTC. -- 2023-12-07T10:12:37.073 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T10:12:37.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:36 smithi119 ceph-mon[97908]: from='osd.0 [v2:172.21.15.119:6802/2141950133,v1:172.21.15.119:6803/2141950133]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-07T10:12:37.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:36 smithi119 ceph-mon[97908]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-07T10:12:37.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:36 smithi159 ceph-mon[134726]: from='osd.0 [v2:172.21.15.119:6802/2141950133,v1:172.21.15.119:6803/2141950133]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-07T10:12:37.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:36 smithi159 ceph-mon[134726]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-07T10:12:37.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:36 smithi153 ceph-mon[133625]: from='osd.0 [v2:172.21.15.119:6802/2141950133,v1:172.21.15.119:6803/2141950133]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-07T10:12:37.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:36 smithi153 ceph-mon[133625]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-07T10:12:38.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:37 smithi159 ceph-mon[134726]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:38.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:37 smithi159 ceph-mon[134726]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-07T10:12:38.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:37 smithi159 ceph-mon[134726]: osdmap e6: 1 total, 0 up, 1 in 2023-12-07T10:12:38.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:37 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T10:12:38.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:37 smithi159 ceph-mon[134726]: from='osd.0 [v2:172.21.15.119:6802/2141950133,v1:172.21.15.119:6803/2141950133]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2023-12-07T10:12:38.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:37 smithi159 ceph-mon[134726]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2023-12-07T10:12:38.271 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:37 smithi153 ceph-mon[133625]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:38.271 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:37 smithi153 ceph-mon[133625]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-07T10:12:38.271 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:37 smithi153 ceph-mon[133625]: osdmap e6: 1 total, 0 up, 1 in 2023-12-07T10:12:38.271 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:37 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T10:12:38.271 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:37 smithi153 ceph-mon[133625]: from='osd.0 [v2:172.21.15.119:6802/2141950133,v1:172.21.15.119:6803/2141950133]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2023-12-07T10:12:38.271 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:37 smithi153 ceph-mon[133625]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2023-12-07T10:12:38.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:37 smithi119 ceph-mon[97908]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:38.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:37 smithi119 ceph-mon[97908]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-07T10:12:38.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:37 smithi119 ceph-mon[97908]: osdmap e6: 1 total, 0 up, 1 in 2023-12-07T10:12:38.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:37 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T10:12:38.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:37 smithi119 ceph-mon[97908]: from='osd.0 [v2:172.21.15.119:6802/2141950133,v1:172.21.15.119:6803/2141950133]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2023-12-07T10:12:38.482 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:37 smithi119 ceph-mon[97908]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2023-12-07T10:12:38.482 INFO:journalctl@ceph.osd.0.smithi119.stdout:Dec 07 10:12:38 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-0[113022]: 2023-12-07T10:12:38.001+0000 7fbbdd303700 -1 osd.0 0 waiting for initial osdmap 2023-12-07T10:12:38.482 INFO:journalctl@ceph.osd.0.smithi119.stdout:Dec 07 10:12:38 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-0[113022]: 2023-12-07T10:12:38.005+0000 7fbbd58f6700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-07T10:12:38.788 INFO:teuthology.orchestra.run.smithi153.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-12-07T10:12:38.788 INFO:teuthology.orchestra.run.smithi153.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-12-07T10:12:38.788 INFO:teuthology.orchestra.run.smithi153.stderr: stderr: 10+0 records in 2023-12-07T10:12:38.788 INFO:teuthology.orchestra.run.smithi153.stderr:10+0 records out 2023-12-07T10:12:38.788 INFO:teuthology.orchestra.run.smithi153.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0150351 s, 697 MB/s 2023-12-07T10:12:38.789 INFO:teuthology.orchestra.run.smithi153.stderr:--> Zapping successful for: 2023-12-07T10:12:39.228 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:38 smithi153 ceph-mon[133625]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2023-12-07T10:12:39.229 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:38 smithi153 ceph-mon[133625]: osdmap e7: 1 total, 0 up, 1 in 2023-12-07T10:12:39.229 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:38 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T10:12:39.229 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:38 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T10:12:39.230 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi153:vg_nvme/lv_4 2023-12-07T10:12:39.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:38 smithi159 ceph-mon[134726]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2023-12-07T10:12:39.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:38 smithi159 ceph-mon[134726]: osdmap e7: 1 total, 0 up, 1 in 2023-12-07T10:12:39.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:38 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T10:12:39.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:38 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T10:12:39.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:38 smithi119 ceph-mon[97908]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2023-12-07T10:12:39.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:38 smithi119 ceph-mon[97908]: osdmap e7: 1 total, 0 up, 1 in 2023-12-07T10:12:39.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:38 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T10:12:39.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:38 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T10:12:39.819 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T10:12:40.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:39 smithi119 ceph-mon[97908]: purged_snaps scrub starts 2023-12-07T10:12:40.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:39 smithi119 ceph-mon[97908]: purged_snaps scrub ok 2023-12-07T10:12:40.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:39 smithi119 ceph-mon[97908]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:40.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:39 smithi119 ceph-mon[97908]: osd.0 [v2:172.21.15.119:6802/2141950133,v1:172.21.15.119:6803/2141950133] boot 2023-12-07T10:12:40.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:39 smithi119 ceph-mon[97908]: osdmap e8: 1 total, 1 up, 1 in 2023-12-07T10:12:40.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:39 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T10:12:40.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:39 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:40.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:39 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:40.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:39 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:12:40.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:39 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:40.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:39 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:40.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:39 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:12:40.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:39 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:40.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:39 smithi159 ceph-mon[134726]: purged_snaps scrub starts 2023-12-07T10:12:40.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:39 smithi159 ceph-mon[134726]: purged_snaps scrub ok 2023-12-07T10:12:40.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:39 smithi159 ceph-mon[134726]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:40.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:39 smithi159 ceph-mon[134726]: osd.0 [v2:172.21.15.119:6802/2141950133,v1:172.21.15.119:6803/2141950133] boot 2023-12-07T10:12:40.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:39 smithi159 ceph-mon[134726]: osdmap e8: 1 total, 1 up, 1 in 2023-12-07T10:12:40.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:39 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T10:12:40.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:39 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:40.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:39 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:40.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:39 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:12:40.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:39 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:40.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:39 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:40.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:39 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:12:40.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:39 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:40.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:39 smithi153 ceph-mon[133625]: purged_snaps scrub starts 2023-12-07T10:12:40.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:39 smithi153 ceph-mon[133625]: purged_snaps scrub ok 2023-12-07T10:12:40.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:39 smithi153 ceph-mon[133625]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-07T10:12:40.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:39 smithi153 ceph-mon[133625]: osd.0 [v2:172.21.15.119:6802/2141950133,v1:172.21.15.119:6803/2141950133] boot 2023-12-07T10:12:40.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:39 smithi153 ceph-mon[133625]: osdmap e8: 1 total, 1 up, 1 in 2023-12-07T10:12:40.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:39 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T10:12:40.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:39 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:40.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:39 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:40.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:39 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:12:40.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:39 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:40.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:39 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:40.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:39 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:12:40.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:39 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:41.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.036+0000 7ffbbc59f700 1 -- 172.21.15.153:0/2022041804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbb4100550 msgr2=0x7ffbb4100970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:12:41.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.036+0000 7ffbbc59f700 1 --2- 172.21.15.153:0/2022041804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbb4100550 0x7ffbb4100970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ffba40099c0 tx=0x7ffba4009cd0 comp rx=0 tx=0).stop 2023-12-07T10:12:41.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.037+0000 7ffbbc59f700 1 -- 172.21.15.153:0/2022041804 shutdown_connections 2023-12-07T10:12:41.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.037+0000 7ffbbc59f700 1 --2- 172.21.15.153:0/2022041804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbb41020f0 0x7ffbb41089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:41.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.037+0000 7ffbbc59f700 1 --2- 172.21.15.153:0/2022041804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbb4100550 0x7ffbb4100970 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:41.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.037+0000 7ffbbc59f700 1 --2- 172.21.15.153:0/2022041804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbb4101750 0x7ffbb4101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:41.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.037+0000 7ffbbc59f700 1 -- 172.21.15.153:0/2022041804 >> 172.21.15.153:0/2022041804 conn(0x7ffbb40fbb10 msgr2=0x7ffbb40fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:12:41.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.037+0000 7ffbbc59f700 1 -- 172.21.15.153:0/2022041804 shutdown_connections 2023-12-07T10:12:41.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.037+0000 7ffbbc59f700 1 -- 172.21.15.153:0/2022041804 wait complete. 2023-12-07T10:12:41.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.037+0000 7ffbbc59f700 1 Processor -- start 2023-12-07T10:12:41.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.037+0000 7ffbbc59f700 1 -- start start 2023-12-07T10:12:41.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.037+0000 7ffbbc59f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbb4100550 0x7ffbb406a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:12:41.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.037+0000 7ffbbc59f700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbb4101750 0x7ffbb419c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:12:41.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.037+0000 7ffbbc59f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbb41020f0 0x7ffbb419e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:12:41.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.037+0000 7ffbbc59f700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb419f140 con 0x7ffbb4101750 2023-12-07T10:12:41.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.037+0000 7ffbbc59f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb419f2b0 con 0x7ffbb4100550 2023-12-07T10:12:41.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.037+0000 7ffbbc59f700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb419f470 con 0x7ffbb41020f0 2023-12-07T10:12:41.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.037+0000 7ffbba33b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbb4100550 0x7ffbb406a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:12:41.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.037+0000 7ffbba33b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbb4100550 0x7ffbb406a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55638/0 (socket says 172.21.15.153:55638) 2023-12-07T10:12:41.044 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.037+0000 7ffbba33b700 1 -- 172.21.15.153:0/3155010808 learned_addr learned my addr 172.21.15.153:0/3155010808 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-12-07T10:12:41.044 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.038+0000 7ffbbab3c700 1 --2- 172.21.15.153:0/3155010808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbb41020f0 0x7ffbb419e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:12:41.044 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.038+0000 7ffbba33b700 1 -- 172.21.15.153:0/3155010808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbb41020f0 msgr2=0x7ffbb419e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:12:41.044 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.038+0000 7ffbba33b700 1 --2- 172.21.15.153:0/3155010808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbb41020f0 0x7ffbb419e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:41.044 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.038+0000 7ffbb9b3a700 1 --2- 172.21.15.153:0/3155010808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbb4101750 0x7ffbb419c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:12:41.044 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.038+0000 7ffbba33b700 1 -- 172.21.15.153:0/3155010808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbb4101750 msgr2=0x7ffbb419c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:12:41.044 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.038+0000 7ffbba33b700 1 --2- 172.21.15.153:0/3155010808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbb4101750 0x7ffbb419c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:41.044 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.038+0000 7ffbba33b700 1 -- 172.21.15.153:0/3155010808 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba40095f0 con 0x7ffbb4100550 2023-12-07T10:12:41.044 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.038+0000 7ffbbab3c700 1 --2- 172.21.15.153:0/3155010808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbb41020f0 0x7ffbb419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:12:41.044 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.038+0000 7ffbba33b700 1 --2- 172.21.15.153:0/3155010808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbb4100550 0x7ffbb406a350 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffba4009f50 tx=0x7ffba4003970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:12:41.044 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.038+0000 7ffbab7fe700 1 -- 172.21.15.153:0/3155010808 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba401d070 con 0x7ffbb4100550 2023-12-07T10:12:41.044 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.038+0000 7ffbab7fe700 1 -- 172.21.15.153:0/3155010808 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffba4022470 con 0x7ffbb4100550 2023-12-07T10:12:41.044 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.038+0000 7ffbbc59f700 1 -- 172.21.15.153:0/3155010808 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb419f6f0 con 0x7ffbb4100550 2023-12-07T10:12:41.045 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.038+0000 7ffbab7fe700 1 -- 172.21.15.153:0/3155010808 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba4022ea0 con 0x7ffbb4100550 2023-12-07T10:12:41.045 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.038+0000 7ffbbc59f700 1 -- 172.21.15.153:0/3155010808 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb419fbe0 con 0x7ffbb4100550 2023-12-07T10:12:41.045 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.039+0000 7ffbbc59f700 1 -- 172.21.15.153:0/3155010808 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb4107a80 con 0x7ffbb4100550 2023-12-07T10:12:41.046 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.040+0000 7ffbab7fe700 1 -- 172.21.15.153:0/3155010808 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 15) v1 ==== 136428+0+0 (secure 0 0 0) 0x7ffba4022800 con 0x7ffbb4100550 2023-12-07T10:12:41.046 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.041+0000 7ffbab7fe700 1 --2- 172.21.15.153:0/3155010808 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ffba00a1b00 0x7ffba00a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:12:41.046 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.041+0000 7ffbab7fe700 1 -- 172.21.15.153:0/3155010808 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7ffba40cd330 con 0x7ffbb4100550 2023-12-07T10:12:41.046 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.041+0000 7ffbb9b3a700 1 --2- 172.21.15.153:0/3155010808 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ffba00a1b00 0x7ffba00a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:12:41.046 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.042+0000 7ffbb9b3a700 1 --2- 172.21.15.153:0/3155010808 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ffba00a1b00 0x7ffba00a3fc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0006fd0 tx=0x7ffbb0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:12:41.048 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.044+0000 7ffbab7fe700 1 -- 172.21.15.153:0/3155010808 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ffba4098250 con 0x7ffbb4100550 2023-12-07T10:12:41.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:41.240+0000 7ffbbc59f700 1 -- 172.21.15.153:0/3155010808 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7ffbb402d3f0 con 0x7ffba00a1b00 2023-12-07T10:12:41.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:40 smithi159 ceph-mon[134726]: Detected new or changed devices on smithi119 2023-12-07T10:12:41.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:40 smithi159 ceph-mon[134726]: Adjusting osd_memory_target on smithi119 to 17271M 2023-12-07T10:12:41.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:40 smithi159 ceph-mon[134726]: osdmap e9: 1 total, 1 up, 1 in 2023-12-07T10:12:41.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:40 smithi153 ceph-mon[133625]: Detected new or changed devices on smithi119 2023-12-07T10:12:41.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:40 smithi153 ceph-mon[133625]: Adjusting osd_memory_target on smithi119 to 17271M 2023-12-07T10:12:41.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:40 smithi153 ceph-mon[133625]: osdmap e9: 1 total, 1 up, 1 in 2023-12-07T10:12:41.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:40 smithi119 ceph-mon[97908]: Detected new or changed devices on smithi119 2023-12-07T10:12:41.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:40 smithi119 ceph-mon[97908]: Adjusting osd_memory_target on smithi119 to 17271M 2023-12-07T10:12:41.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:40 smithi119 ceph-mon[97908]: osdmap e9: 1 total, 1 up, 1 in 2023-12-07T10:12:42.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:42 smithi159 ceph-mon[134726]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:42.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:42 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:12:42.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:42 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:12:42.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:42 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:42.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:42 smithi153 ceph-mon[133625]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:42.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:42 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:12:42.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:42 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:12:42.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:42 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:42.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:42 smithi119 ceph-mon[97908]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:42.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:42 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:12:42.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:42 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:12:42.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:42 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:43.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:43 smithi159 ceph-mon[134726]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:12:43.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:43 smithi153 ceph-mon[133625]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:12:43.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:43 smithi119 ceph-mon[97908]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:12:44.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:44 smithi159 ceph-mon[134726]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:44.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:44 smithi159 ceph-mon[134726]: from='client.? 172.21.15.153:0/4293859533' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36489a6a-5278-4802-9525-9871521722e6"}]: dispatch 2023-12-07T10:12:44.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:44 smithi159 ceph-mon[134726]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36489a6a-5278-4802-9525-9871521722e6"}]: dispatch 2023-12-07T10:12:44.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:44 smithi159 ceph-mon[134726]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36489a6a-5278-4802-9525-9871521722e6"}]': finished 2023-12-07T10:12:44.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:44 smithi159 ceph-mon[134726]: osdmap e10: 2 total, 1 up, 2 in 2023-12-07T10:12:44.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:44 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T10:12:44.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:44 smithi153 ceph-mon[133625]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:44.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:44 smithi153 ceph-mon[133625]: from='client.? 172.21.15.153:0/4293859533' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36489a6a-5278-4802-9525-9871521722e6"}]: dispatch 2023-12-07T10:12:44.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:44 smithi153 ceph-mon[133625]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36489a6a-5278-4802-9525-9871521722e6"}]: dispatch 2023-12-07T10:12:44.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:44 smithi153 ceph-mon[133625]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36489a6a-5278-4802-9525-9871521722e6"}]': finished 2023-12-07T10:12:44.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:44 smithi153 ceph-mon[133625]: osdmap e10: 2 total, 1 up, 2 in 2023-12-07T10:12:44.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:44 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T10:12:44.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:44 smithi119 ceph-mon[97908]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:44.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:44 smithi119 ceph-mon[97908]: from='client.? 172.21.15.153:0/4293859533' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36489a6a-5278-4802-9525-9871521722e6"}]: dispatch 2023-12-07T10:12:44.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:44 smithi119 ceph-mon[97908]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36489a6a-5278-4802-9525-9871521722e6"}]: dispatch 2023-12-07T10:12:44.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:44 smithi119 ceph-mon[97908]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36489a6a-5278-4802-9525-9871521722e6"}]': finished 2023-12-07T10:12:44.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:44 smithi119 ceph-mon[97908]: osdmap e10: 2 total, 1 up, 2 in 2023-12-07T10:12:44.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:44 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T10:12:45.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:45 smithi153 ceph-mon[133625]: from='client.? 172.21.15.153:0/3314963251' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:12:45.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:45 smithi119 ceph-mon[97908]: from='client.? 172.21.15.153:0/3314963251' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:12:45.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:45 smithi159 ceph-mon[134726]: from='client.? 172.21.15.153:0/3314963251' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:12:46.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:46 smithi153 ceph-mon[133625]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:46.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:46 smithi119 ceph-mon[97908]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:46.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:46 smithi159 ceph-mon[134726]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:48.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:48 smithi153 ceph-mon[133625]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:48.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:48 smithi119 ceph-mon[97908]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:48.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:48 smithi159 ceph-mon[134726]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:50.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:50 smithi153 ceph-mon[133625]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:50.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:50 smithi119 ceph-mon[97908]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:50.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:50 smithi159 ceph-mon[134726]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:51.254 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:51 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-07T10:12:51.254 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:51 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:51.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:51 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-07T10:12:51.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:51 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:51.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:51 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-07T10:12:51.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:51 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:52.139 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:52 smithi153 ceph-mon[133625]: Deploying daemon osd.1 on smithi153 2023-12-07T10:12:52.139 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:52 smithi153 ceph-mon[133625]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:52.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:52 smithi119 ceph-mon[97908]: Deploying daemon osd.1 on smithi153 2023-12-07T10:12:52.482 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:52 smithi119 ceph-mon[97908]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:52.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:52 smithi159 ceph-mon[134726]: Deploying daemon osd.1 on smithi153 2023-12-07T10:12:52.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:52 smithi159 ceph-mon[134726]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:54.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:54 smithi153 ceph-mon[133625]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:54.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:54 smithi119 ceph-mon[97908]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:54.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:54 smithi159 ceph-mon[134726]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:56.038 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:55 smithi153 ceph-mon[133625]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:56.038 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:55 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:56.038 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:55 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:12:56.038 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:55 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:56.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:55 smithi119 ceph-mon[97908]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:56.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:55 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:56.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:55 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:12:56.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:55 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:56.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:55 smithi159 ceph-mon[134726]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:56.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:55 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:56.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:55 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:12:56.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:55 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:57.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:57.955+0000 7ffbab7fe700 1 -- 172.21.15.153:0/3155010808 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ffbb402d3f0 con 0x7ffba00a1b00 2023-12-07T10:12:57.958 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 1 on host 'smithi153' 2023-12-07T10:12:57.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:57.957+0000 7ffbbc59f700 1 -- 172.21.15.153:0/3155010808 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ffba00a1b00 msgr2=0x7ffba00a3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:12:57.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:57.957+0000 7ffbbc59f700 1 --2- 172.21.15.153:0/3155010808 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ffba00a1b00 0x7ffba00a3fc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0006fd0 tx=0x7ffbb0008040 comp rx=0 tx=0).stop 2023-12-07T10:12:57.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:57.957+0000 7ffbbc59f700 1 -- 172.21.15.153:0/3155010808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbb4100550 msgr2=0x7ffbb406a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:12:57.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:57.957+0000 7ffbbc59f700 1 --2- 172.21.15.153:0/3155010808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbb4100550 0x7ffbb406a350 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffba4009f50 tx=0x7ffba4003970 comp rx=0 tx=0).stop 2023-12-07T10:12:57.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:57.957+0000 7ffbbc59f700 1 -- 172.21.15.153:0/3155010808 shutdown_connections 2023-12-07T10:12:57.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:57.957+0000 7ffbbc59f700 1 --2- 172.21.15.153:0/3155010808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbb41020f0 0x7ffbb419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:57.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:57.957+0000 7ffbbc59f700 1 --2- 172.21.15.153:0/3155010808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffbb4100550 0x7ffbb406a350 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:57.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:57.957+0000 7ffbbc59f700 1 --2- 172.21.15.153:0/3155010808 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ffba00a1b00 0x7ffba00a3fc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:57.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:57.957+0000 7ffbbc59f700 1 --2- 172.21.15.153:0/3155010808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbb4101750 0x7ffbb419c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:12:57.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:57.957+0000 7ffbbc59f700 1 -- 172.21.15.153:0/3155010808 >> 172.21.15.153:0/3155010808 conn(0x7ffbb40fbb10 msgr2=0x7ffbb4104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:12:57.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:57.958+0000 7ffbbc59f700 1 -- 172.21.15.153:0/3155010808 shutdown_connections 2023-12-07T10:12:57.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:12:57.958+0000 7ffbbc59f700 1 -- 172.21.15.153:0/3155010808 wait complete. 2023-12-07T10:12:58.094 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:57 smithi153 ceph-mon[133625]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:58.094 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:57 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:58.094 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:57 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:58.095 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:57 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:58.095 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:57 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:12:58.095 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:57 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:58.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:57 smithi119 ceph-mon[97908]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:58.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:57 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:58.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:57 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:58.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:57 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:58.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:57 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:12:58.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:57 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:58.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:57 smithi159 ceph-mon[134726]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:12:58.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:57 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:58.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:57 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:58.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:57 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:12:58.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:57 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:12:58.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:57 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:58.418 DEBUG:teuthology.orchestra.run.smithi153:osd.1> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.1.service 2023-12-07T10:12:58.420 INFO:tasks.cephadm:Deploying osd.2 on smithi159 with /dev/vg_nvme/lv_4... 2023-12-07T10:12:58.420 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-12-07T10:12:58.451 INFO:journalctl@ceph.osd.1.smithi153.stdout:-- Logs begin at Thu 2023-12-07 09:49:13 UTC. -- 2023-12-07T10:12:58.781 INFO:journalctl@ceph.osd.1.smithi153.stdout:Dec 07 10:12:58 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-1[138700]: 2023-12-07T10:12:58.538+0000 7f9fca609640 -1 osd.1 0 log_to_monitors true 2023-12-07T10:12:59.027 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.c/config 2023-12-07T10:12:59.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:58 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:59.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:58 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:59.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:58 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:12:59.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:58 smithi119 ceph-mon[97908]: from='osd.1 [v2:172.21.15.153:6800/3390875370,v1:172.21.15.153:6801/3390875370]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-07T10:12:59.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:58 smithi119 ceph-mon[97908]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-07T10:12:59.241 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:58 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:59.242 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:58 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:59.242 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:58 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:12:59.242 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:58 smithi159 ceph-mon[134726]: from='osd.1 [v2:172.21.15.153:6800/3390875370,v1:172.21.15.153:6801/3390875370]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-07T10:12:59.242 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:58 smithi159 ceph-mon[134726]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-07T10:12:59.252 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:58 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:59.252 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:58 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:12:59.252 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:58 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:12:59.252 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:58 smithi153 ceph-mon[133625]: from='osd.1 [v2:172.21.15.153:6800/3390875370,v1:172.21.15.153:6801/3390875370]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-07T10:12:59.252 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:58 smithi153 ceph-mon[133625]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-07T10:13:00.197 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:59 smithi159 ceph-mon[134726]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:13:00.197 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:59 smithi159 ceph-mon[134726]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-07T10:13:00.197 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:59 smithi159 ceph-mon[134726]: from='osd.1 [v2:172.21.15.153:6800/3390875370,v1:172.21.15.153:6801/3390875370]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2023-12-07T10:13:00.197 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:59 smithi159 ceph-mon[134726]: osdmap e11: 2 total, 1 up, 2 in 2023-12-07T10:13:00.197 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:59 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T10:13:00.197 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:12:59 smithi159 ceph-mon[134726]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2023-12-07T10:13:00.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:59 smithi119 ceph-mon[97908]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:13:00.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:59 smithi119 ceph-mon[97908]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-07T10:13:00.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:59 smithi119 ceph-mon[97908]: from='osd.1 [v2:172.21.15.153:6800/3390875370,v1:172.21.15.153:6801/3390875370]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2023-12-07T10:13:00.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:59 smithi119 ceph-mon[97908]: osdmap e11: 2 total, 1 up, 2 in 2023-12-07T10:13:00.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:59 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T10:13:00.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:12:59 smithi119 ceph-mon[97908]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2023-12-07T10:13:00.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:59 smithi153 ceph-mon[133625]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:13:00.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:59 smithi153 ceph-mon[133625]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-07T10:13:00.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:59 smithi153 ceph-mon[133625]: from='osd.1 [v2:172.21.15.153:6800/3390875370,v1:172.21.15.153:6801/3390875370]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2023-12-07T10:13:00.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:59 smithi153 ceph-mon[133625]: osdmap e11: 2 total, 1 up, 2 in 2023-12-07T10:13:00.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:59 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T10:13:00.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:12:59 smithi153 ceph-mon[133625]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2023-12-07T10:13:00.283 INFO:journalctl@ceph.osd.1.smithi153.stdout:Dec 07 10:12:59 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-1[138700]: 2023-12-07T10:12:59.922+0000 7f9fbfc80700 -1 osd.1 0 waiting for initial osdmap 2023-12-07T10:13:00.283 INFO:journalctl@ceph.osd.1.smithi153.stdout:Dec 07 10:12:59 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-1[138700]: 2023-12-07T10:12:59.925+0000 7f9fb9a76700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-07T10:13:00.673 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-12-07T10:13:00.673 INFO:teuthology.orchestra.run.smithi159.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-12-07T10:13:00.673 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10+0 records in 2023-12-07T10:13:00.673 INFO:teuthology.orchestra.run.smithi159.stderr:10+0 records out 2023-12-07T10:13:00.673 INFO:teuthology.orchestra.run.smithi159.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0128732 s, 815 MB/s 2023-12-07T10:13:00.674 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping successful for: 2023-12-07T10:13:01.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:00 smithi159 ceph-mon[134726]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2023-12-07T10:13:01.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:00 smithi159 ceph-mon[134726]: osdmap e12: 2 total, 1 up, 2 in 2023-12-07T10:13:01.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:00 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T10:13:01.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:00 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T10:13:01.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:00 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T10:13:01.089 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi159:vg_nvme/lv_4 2023-12-07T10:13:01.161 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:00 smithi153 ceph-mon[133625]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2023-12-07T10:13:01.161 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:00 smithi153 ceph-mon[133625]: osdmap e12: 2 total, 1 up, 2 in 2023-12-07T10:13:01.161 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T10:13:01.161 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T10:13:01.161 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T10:13:01.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:00 smithi119 ceph-mon[97908]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2023-12-07T10:13:01.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:00 smithi119 ceph-mon[97908]: osdmap e12: 2 total, 1 up, 2 in 2023-12-07T10:13:01.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:00 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T10:13:01.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:00 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T10:13:01.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:00 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T10:13:01.681 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.c/config 2023-12-07T10:13:02.218 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:01 smithi159 ceph-mon[134726]: purged_snaps scrub starts 2023-12-07T10:13:02.218 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:01 smithi159 ceph-mon[134726]: purged_snaps scrub ok 2023-12-07T10:13:02.218 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:01 smithi159 ceph-mon[134726]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:13:02.218 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:01 smithi159 ceph-mon[134726]: osd.1 [v2:172.21.15.153:6800/3390875370,v1:172.21.15.153:6801/3390875370] boot 2023-12-07T10:13:02.218 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:01 smithi159 ceph-mon[134726]: osdmap e13: 2 total, 2 up, 2 in 2023-12-07T10:13:02.218 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T10:13:02.218 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:02.218 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:02.218 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:13:02.218 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:02.218 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:02.218 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:13:02.218 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:02.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:01 smithi119 ceph-mon[97908]: purged_snaps scrub starts 2023-12-07T10:13:02.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:01 smithi119 ceph-mon[97908]: purged_snaps scrub ok 2023-12-07T10:13:02.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:01 smithi119 ceph-mon[97908]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:13:02.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:01 smithi119 ceph-mon[97908]: osd.1 [v2:172.21.15.153:6800/3390875370,v1:172.21.15.153:6801/3390875370] boot 2023-12-07T10:13:02.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:01 smithi119 ceph-mon[97908]: osdmap e13: 2 total, 2 up, 2 in 2023-12-07T10:13:02.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T10:13:02.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:02.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:02.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:13:02.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:02.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:02.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:13:02.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:02.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:01 smithi153 ceph-mon[133625]: purged_snaps scrub starts 2023-12-07T10:13:02.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:01 smithi153 ceph-mon[133625]: purged_snaps scrub ok 2023-12-07T10:13:02.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:01 smithi153 ceph-mon[133625]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-07T10:13:02.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:01 smithi153 ceph-mon[133625]: osd.1 [v2:172.21.15.153:6800/3390875370,v1:172.21.15.153:6801/3390875370] boot 2023-12-07T10:13:02.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:01 smithi153 ceph-mon[133625]: osdmap e13: 2 total, 2 up, 2 in 2023-12-07T10:13:02.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T10:13:02.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:02.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:02.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:13:02.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:02.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:02.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:13:02.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:02.870 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.864+0000 7fa0f3952700 1 -- 172.21.15.159:0/4076385208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0ec1007a0 msgr2=0x7fa0ec100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:13:02.871 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.864+0000 7fa0f3952700 1 --2- 172.21.15.159:0/4076385208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0ec1007a0 0x7fa0ec100c20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009990 tx=0x7fa0e8009ca0 comp rx=0 tx=0).stop 2023-12-07T10:13:02.871 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.864+0000 7fa0f3952700 1 -- 172.21.15.159:0/4076385208 shutdown_connections 2023-12-07T10:13:02.871 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.864+0000 7fa0f3952700 1 --2- 172.21.15.159:0/4076385208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0ec101160 0x7fa0ec108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:13:02.871 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.864+0000 7fa0f3952700 1 --2- 172.21.15.159:0/4076385208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0ec101bd0 0x7fa0ec101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:13:02.871 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.864+0000 7fa0f3952700 1 --2- 172.21.15.159:0/4076385208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0ec1007a0 0x7fa0ec100c20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:13:02.871 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.864+0000 7fa0f3952700 1 -- 172.21.15.159:0/4076385208 >> 172.21.15.159:0/4076385208 conn(0x7fa0ec0fb270 msgr2=0x7fa0ec0fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:13:02.871 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.864+0000 7fa0f3952700 1 -- 172.21.15.159:0/4076385208 shutdown_connections 2023-12-07T10:13:02.872 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.864+0000 7fa0f3952700 1 -- 172.21.15.159:0/4076385208 wait complete. 2023-12-07T10:13:02.872 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.865+0000 7fa0f3952700 1 Processor -- start 2023-12-07T10:13:02.872 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.865+0000 7fa0f3952700 1 -- start start 2023-12-07T10:13:02.872 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.865+0000 7fa0f3952700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0ec1007a0 0x7fa0ec1044f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:13:02.872 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.865+0000 7fa0f3952700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0ec101160 0x7fa0ec102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:13:02.872 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.865+0000 7fa0f3952700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0ec101bd0 0x7fa0ec1030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:13:02.872 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.865+0000 7fa0f3952700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec103870 con 0x7fa0ec101bd0 2023-12-07T10:13:02.873 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.865+0000 7fa0f3952700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec1039e0 con 0x7fa0ec1007a0 2023-12-07T10:13:02.873 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.865+0000 7fa0f3952700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec1a15e0 con 0x7fa0ec101160 2023-12-07T10:13:02.873 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.865+0000 7fa0f0eed700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0ec101160 0x7fa0ec102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:13:02.873 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.865+0000 7fa0f0eed700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0ec101160 0x7fa0ec102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42490/0 (socket says 172.21.15.159:42490) 2023-12-07T10:13:02.873 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.865+0000 7fa0f0eed700 1 -- 172.21.15.159:0/1638362222 learned_addr learned my addr 172.21.15.159:0/1638362222 (peer_addr_for_me v2:172.21.15.159:0/0) 2023-12-07T10:13:02.873 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.865+0000 7fa0f16ee700 1 --2- 172.21.15.159:0/1638362222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0ec1007a0 0x7fa0ec1044f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:13:02.873 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.865+0000 7fa0f0eed700 1 -- 172.21.15.159:0/1638362222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0ec1007a0 msgr2=0x7fa0ec1044f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:13:02.873 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.865+0000 7fa0f0eed700 1 --2- 172.21.15.159:0/1638362222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0ec1007a0 0x7fa0ec1044f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:13:02.873 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.865+0000 7fa0f0eed700 1 -- 172.21.15.159:0/1638362222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0ec101bd0 msgr2=0x7fa0ec1030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:13:02.873 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.865+0000 7fa0f0eed700 1 --2- 172.21.15.159:0/1638362222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0ec101bd0 0x7fa0ec1030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:13:02.873 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.865+0000 7fa0f0eed700 1 -- 172.21.15.159:0/1638362222 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e80095f0 con 0x7fa0ec101160 2023-12-07T10:13:02.873 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.865+0000 7fa0f0eed700 1 --2- 172.21.15.159:0/1638362222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0ec101160 0x7fa0ec102b90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800b5c0 tx=0x7fa0e800f720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:13:02.874 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.866+0000 7fa0e27fc700 1 -- 172.21.15.159:0/1638362222 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e801d070 con 0x7fa0ec101160 2023-12-07T10:13:02.874 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.866+0000 7fa0e27fc700 1 -- 172.21.15.159:0/1638362222 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0e800fc10 con 0x7fa0ec101160 2023-12-07T10:13:02.874 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.866+0000 7fa0f3952700 1 -- 172.21.15.159:0/1638362222 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0ec1a1810 con 0x7fa0ec101160 2023-12-07T10:13:02.874 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.866+0000 7fa0e27fc700 1 -- 172.21.15.159:0/1638362222 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8017650 con 0x7fa0ec101160 2023-12-07T10:13:02.874 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.866+0000 7fa0f3952700 1 -- 172.21.15.159:0/1638362222 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0ec1a1c20 con 0x7fa0ec101160 2023-12-07T10:13:02.874 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.867+0000 7fa0f3952700 1 -- 172.21.15.159:0/1638362222 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0ec04f2d0 con 0x7fa0ec101160 2023-12-07T10:13:02.874 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.867+0000 7fa0e27fc700 1 -- 172.21.15.159:0/1638362222 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 15) v1 ==== 136428+0+0 (secure 0 0 0) 0x7fa0e8022470 con 0x7fa0ec101160 2023-12-07T10:13:02.875 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.868+0000 7fa0e27fc700 1 --2- 172.21.15.159:0/1638362222 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa0d80a1a80 0x7fa0d80a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:13:02.875 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.868+0000 7fa0e27fc700 1 -- 172.21.15.159:0/1638362222 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7fa0e80cd4b0 con 0x7fa0ec101160 2023-12-07T10:13:02.875 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.869+0000 7fa0f16ee700 1 --2- 172.21.15.159:0/1638362222 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa0d80a1a80 0x7fa0d80a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:13:02.875 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.869+0000 7fa0f16ee700 1 --2- 172.21.15.159:0/1638362222 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa0d80a1a80 0x7fa0d80a3f40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc005ea0 tx=0x7fa0dc005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:13:02.875 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:02.873+0000 7fa0e27fc700 1 -- 172.21.15.159:0/1638362222 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa0e80982d0 con 0x7fa0ec101160 2023-12-07T10:13:03.076 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:03.073+0000 7fa0f3952700 1 -- 172.21.15.159:0/1638362222 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0ec0fd450 con 0x7fa0d80a1a80 2023-12-07T10:13:03.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:02 smithi119 ceph-mon[97908]: Detected new or changed devices on smithi153 2023-12-07T10:13:03.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:02 smithi119 ceph-mon[97908]: Adjusting osd_memory_target on smithi153 to 17168M 2023-12-07T10:13:03.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:02 smithi119 ceph-mon[97908]: osdmap e14: 2 total, 2 up, 2 in 2023-12-07T10:13:03.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:02 smithi159 ceph-mon[134726]: Detected new or changed devices on smithi153 2023-12-07T10:13:03.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:02 smithi159 ceph-mon[134726]: Adjusting osd_memory_target on smithi153 to 17168M 2023-12-07T10:13:03.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:02 smithi159 ceph-mon[134726]: osdmap e14: 2 total, 2 up, 2 in 2023-12-07T10:13:03.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:02 smithi153 ceph-mon[133625]: Detected new or changed devices on smithi153 2023-12-07T10:13:03.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:02 smithi153 ceph-mon[133625]: Adjusting osd_memory_target on smithi153 to 17168M 2023-12-07T10:13:03.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:02 smithi153 ceph-mon[133625]: osdmap e14: 2 total, 2 up, 2 in 2023-12-07T10:13:04.208 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:03 smithi159 ceph-mon[134726]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:04.208 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:03 smithi159 ceph-mon[134726]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:13:04.209 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:03 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:13:04.209 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:03 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:13:04.209 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:03 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:04.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:03 smithi119 ceph-mon[97908]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:04.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:03 smithi119 ceph-mon[97908]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:13:04.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:03 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:13:04.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:03 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:13:04.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:03 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:04.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:03 smithi153 ceph-mon[133625]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:04.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:03 smithi153 ceph-mon[133625]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:13:04.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:03 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:13:04.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:03 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:13:04.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:03 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:06.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:05 smithi119 ceph-mon[97908]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:06.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:05 smithi119 ceph-mon[97908]: from='client.? 172.21.15.159:0/3961856589' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a937ad26-5e9b-4c8a-b65a-626a8feda6fa"}]: dispatch 2023-12-07T10:13:06.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:05 smithi119 ceph-mon[97908]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a937ad26-5e9b-4c8a-b65a-626a8feda6fa"}]: dispatch 2023-12-07T10:13:06.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:05 smithi119 ceph-mon[97908]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a937ad26-5e9b-4c8a-b65a-626a8feda6fa"}]': finished 2023-12-07T10:13:06.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:05 smithi119 ceph-mon[97908]: osdmap e15: 3 total, 2 up, 3 in 2023-12-07T10:13:06.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:05 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T10:13:06.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:05 smithi159 ceph-mon[134726]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:06.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:05 smithi159 ceph-mon[134726]: from='client.? 172.21.15.159:0/3961856589' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a937ad26-5e9b-4c8a-b65a-626a8feda6fa"}]: dispatch 2023-12-07T10:13:06.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:05 smithi159 ceph-mon[134726]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a937ad26-5e9b-4c8a-b65a-626a8feda6fa"}]: dispatch 2023-12-07T10:13:06.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:05 smithi159 ceph-mon[134726]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a937ad26-5e9b-4c8a-b65a-626a8feda6fa"}]': finished 2023-12-07T10:13:06.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:05 smithi159 ceph-mon[134726]: osdmap e15: 3 total, 2 up, 3 in 2023-12-07T10:13:06.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:05 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T10:13:06.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:05 smithi153 ceph-mon[133625]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:06.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:05 smithi153 ceph-mon[133625]: from='client.? 172.21.15.159:0/3961856589' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a937ad26-5e9b-4c8a-b65a-626a8feda6fa"}]: dispatch 2023-12-07T10:13:06.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:05 smithi153 ceph-mon[133625]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a937ad26-5e9b-4c8a-b65a-626a8feda6fa"}]: dispatch 2023-12-07T10:13:06.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:05 smithi153 ceph-mon[133625]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a937ad26-5e9b-4c8a-b65a-626a8feda6fa"}]': finished 2023-12-07T10:13:06.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:05 smithi153 ceph-mon[133625]: osdmap e15: 3 total, 2 up, 3 in 2023-12-07T10:13:06.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:05 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T10:13:07.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:06 smithi119 ceph-mon[97908]: from='client.? 172.21.15.159:0/456790163' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:13:07.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:06 smithi159 ceph-mon[134726]: from='client.? 172.21.15.159:0/456790163' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:13:07.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:06 smithi153 ceph-mon[133625]: from='client.? 172.21.15.159:0/456790163' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:13:08.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:07 smithi119 ceph-mon[97908]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:08.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:07 smithi159 ceph-mon[134726]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:08.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:07 smithi153 ceph-mon[133625]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:10.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:09 smithi119 ceph-mon[97908]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:10.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:09 smithi159 ceph-mon[134726]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:10.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:09 smithi153 ceph-mon[133625]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:11.730 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:11 smithi159 ceph-mon[134726]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:11.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:11 smithi119 ceph-mon[97908]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:11.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:11 smithi153 ceph-mon[133625]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:12.713 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:12 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-07T10:13:12.713 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:12 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:12.713 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:12 smithi159 ceph-mon[134726]: Deploying daemon osd.2 on smithi159 2023-12-07T10:13:12.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:12 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-07T10:13:12.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:12 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:12.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:12 smithi119 ceph-mon[97908]: Deploying daemon osd.2 on smithi159 2023-12-07T10:13:12.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:12 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-07T10:13:12.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:12 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:12.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:12 smithi153 ceph-mon[133625]: Deploying daemon osd.2 on smithi159 2023-12-07T10:13:13.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:13 smithi119 ceph-mon[97908]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:13.765 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:13 smithi159 ceph-mon[134726]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:13.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:13 smithi153 ceph-mon[133625]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:15.765 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:15 smithi159 ceph-mon[134726]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:15.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:15 smithi119 ceph-mon[97908]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:16.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:15 smithi153 ceph-mon[133625]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:17.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:17 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:13:17.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:17 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:17.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:17 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:17.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:17 smithi159 ceph-mon[134726]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:17.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:17 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:13:17.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:17 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:17.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:17 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:17.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:17 smithi119 ceph-mon[97908]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:17.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:17 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:13:17.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:17 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:17.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:17 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:17.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:17 smithi153 ceph-mon[133625]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:19.042 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:19.038+0000 7fa0e27fc700 1 -- 172.21.15.159:0/1638362222 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa0ec0fd450 con 0x7fa0d80a1a80 2023-12-07T10:13:19.042 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 2 on host 'smithi159' 2023-12-07T10:13:19.044 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:19.040+0000 7fa0f3952700 1 -- 172.21.15.159:0/1638362222 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa0d80a1a80 msgr2=0x7fa0d80a3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:13:19.044 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:19.040+0000 7fa0f3952700 1 --2- 172.21.15.159:0/1638362222 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa0d80a1a80 0x7fa0d80a3f40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc005ea0 tx=0x7fa0dc005e30 comp rx=0 tx=0).stop 2023-12-07T10:13:19.044 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:19.040+0000 7fa0f3952700 1 -- 172.21.15.159:0/1638362222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0ec101160 msgr2=0x7fa0ec102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:13:19.044 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:19.040+0000 7fa0f3952700 1 --2- 172.21.15.159:0/1638362222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0ec101160 0x7fa0ec102b90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800b5c0 tx=0x7fa0e800f720 comp rx=0 tx=0).stop 2023-12-07T10:13:19.044 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:19.040+0000 7fa0f3952700 1 -- 172.21.15.159:0/1638362222 shutdown_connections 2023-12-07T10:13:19.044 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:19.040+0000 7fa0f3952700 1 --2- 172.21.15.159:0/1638362222 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa0d80a1a80 0x7fa0d80a3f40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:13:19.044 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:19.040+0000 7fa0f3952700 1 --2- 172.21.15.159:0/1638362222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0ec101bd0 0x7fa0ec1030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:13:19.044 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:19.040+0000 7fa0f3952700 1 --2- 172.21.15.159:0/1638362222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0ec1007a0 0x7fa0ec1044f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:13:19.044 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:19.040+0000 7fa0f3952700 1 --2- 172.21.15.159:0/1638362222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0ec101160 0x7fa0ec102b90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:13:19.044 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:19.040+0000 7fa0f3952700 1 -- 172.21.15.159:0/1638362222 >> 172.21.15.159:0/1638362222 conn(0x7fa0ec0fb270 msgr2=0x7fa0ec0fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:13:19.044 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:19.040+0000 7fa0f3952700 1 -- 172.21.15.159:0/1638362222 shutdown_connections 2023-12-07T10:13:19.044 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:13:19.041+0000 7fa0f3952700 1 -- 172.21.15.159:0/1638362222 wait complete. 2023-12-07T10:13:19.476 DEBUG:teuthology.orchestra.run.smithi159:osd.2> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.2.service 2023-12-07T10:13:19.478 INFO:tasks.cephadm:Deploying osd.3 on smithi119 with /dev/vg_nvme/lv_3... 2023-12-07T10:13:19.478 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-12-07T10:13:19.510 INFO:journalctl@ceph.osd.2.smithi159.stdout:-- Logs begin at Thu 2023-12-07 09:49:21 UTC. -- 2023-12-07T10:13:20.239 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:20 smithi159 ceph-mon[134726]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:20.239 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:20 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:20.240 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:20 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:20.240 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:20 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:20.240 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:20 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:20.255 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:13:20.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:20 smithi153 ceph-mon[133625]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:20.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:20 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:20.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:20 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:20.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:20 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:20.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:20 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:20.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:20 smithi119 ceph-mon[97908]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:20.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:20 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:20.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:20 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:20.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:20 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:20.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:20 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:20.514 INFO:journalctl@ceph.osd.2.smithi159.stdout:Dec 07 10:13:20 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-2[140541]: 2023-12-07T10:13:20.391+0000 7f3112602640 -1 osd.2 0 log_to_monitors true 2023-12-07T10:13:21.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:21 smithi119 ceph-mon[97908]: from='osd.2 [v2:172.21.15.159:6800/708101637,v1:172.21.15.159:6801/708101637]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-07T10:13:21.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:21 smithi119 ceph-mon[97908]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-07T10:13:21.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:21 smithi153 ceph-mon[133625]: from='osd.2 [v2:172.21.15.159:6800/708101637,v1:172.21.15.159:6801/708101637]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-07T10:13:21.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:21 smithi153 ceph-mon[133625]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-07T10:13:21.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:21 smithi159 ceph-mon[134726]: from='osd.2 [v2:172.21.15.159:6800/708101637,v1:172.21.15.159:6801/708101637]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-07T10:13:21.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:21 smithi159 ceph-mon[134726]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-07T10:13:22.307 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-12-07T10:13:22.307 INFO:teuthology.orchestra.run.smithi119.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-12-07T10:13:22.307 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10+0 records in 2023-12-07T10:13:22.307 INFO:teuthology.orchestra.run.smithi119.stderr:10+0 records out 2023-12-07T10:13:22.307 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0305089 s, 344 MB/s 2023-12-07T10:13:22.307 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping successful for: 2023-12-07T10:13:22.345 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:22 smithi119 ceph-mon[97908]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:22.345 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:22 smithi119 ceph-mon[97908]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-07T10:13:22.345 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:22 smithi119 ceph-mon[97908]: osdmap e16: 3 total, 2 up, 3 in 2023-12-07T10:13:22.345 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:22 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T10:13:22.345 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:22 smithi119 ceph-mon[97908]: from='osd.2 [v2:172.21.15.159:6800/708101637,v1:172.21.15.159:6801/708101637]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-12-07T10:13:22.345 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:22 smithi119 ceph-mon[97908]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-12-07T10:13:22.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:22 smithi159 ceph-mon[134726]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:22.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:22 smithi159 ceph-mon[134726]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-07T10:13:22.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:22 smithi159 ceph-mon[134726]: osdmap e16: 3 total, 2 up, 3 in 2023-12-07T10:13:22.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:22 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T10:13:22.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:22 smithi159 ceph-mon[134726]: from='osd.2 [v2:172.21.15.159:6800/708101637,v1:172.21.15.159:6801/708101637]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-12-07T10:13:22.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:22 smithi159 ceph-mon[134726]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-12-07T10:13:22.515 INFO:journalctl@ceph.osd.2.smithi159.stdout:Dec 07 10:13:22 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-2[140541]: 2023-12-07T10:13:22.036+0000 7f3107c79700 -1 osd.2 0 waiting for initial osdmap 2023-12-07T10:13:22.515 INFO:journalctl@ceph.osd.2.smithi159.stdout:Dec 07 10:13:22 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-2[140541]: 2023-12-07T10:13:22.040+0000 7f3102270700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-07T10:13:22.534 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:22 smithi153 ceph-mon[133625]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:22.534 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:22 smithi153 ceph-mon[133625]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-07T10:13:22.534 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:22 smithi153 ceph-mon[133625]: osdmap e16: 3 total, 2 up, 3 in 2023-12-07T10:13:22.534 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:22 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T10:13:22.534 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:22 smithi153 ceph-mon[133625]: from='osd.2 [v2:172.21.15.159:6800/708101637,v1:172.21.15.159:6801/708101637]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-12-07T10:13:22.534 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:22 smithi153 ceph-mon[133625]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-12-07T10:13:22.738 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi119:vg_nvme/lv_3 2023-12-07T10:13:23.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:23 smithi119 ceph-mon[97908]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-12-07T10:13:23.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:23 smithi119 ceph-mon[97908]: osdmap e17: 3 total, 2 up, 3 in 2023-12-07T10:13:23.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:23 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T10:13:23.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:23 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T10:13:23.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:23 smithi119 ceph-mon[97908]: Detected new or changed devices on smithi159 2023-12-07T10:13:23.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:23 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:23.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:23 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:23.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:23 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:13:23.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:23 smithi119 ceph-mon[97908]: Adjusting osd_memory_target on smithi159 to 17168M 2023-12-07T10:13:23.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:23 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:23.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:23 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:23.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:23 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:13:23.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:23 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:23.482 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:23 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:13:23.482 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:23 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:23.482 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:23 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:13:23.482 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:23 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:23.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:23 smithi159 ceph-mon[134726]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-12-07T10:13:23.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:23 smithi159 ceph-mon[134726]: osdmap e17: 3 total, 2 up, 3 in 2023-12-07T10:13:23.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:23 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T10:13:23.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:23 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T10:13:23.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:23 smithi159 ceph-mon[134726]: Detected new or changed devices on smithi159 2023-12-07T10:13:23.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:23 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:23.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:23 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:23.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:23 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:13:23.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:23 smithi159 ceph-mon[134726]: Adjusting osd_memory_target on smithi159 to 17168M 2023-12-07T10:13:23.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:23 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:23.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:23 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:23.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:23 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:13:23.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:23 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:23.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:23 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:13:23.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:23 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:23.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:23 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:13:23.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:23 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:23.516 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:13:23.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:23 smithi153 ceph-mon[133625]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-12-07T10:13:23.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:23 smithi153 ceph-mon[133625]: osdmap e17: 3 total, 2 up, 3 in 2023-12-07T10:13:23.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:23 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T10:13:23.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:23 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T10:13:23.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:23 smithi153 ceph-mon[133625]: Detected new or changed devices on smithi159 2023-12-07T10:13:23.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:23 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:23.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:23 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:23.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:23 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:13:23.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:23 smithi153 ceph-mon[133625]: Adjusting osd_memory_target on smithi159 to 17168M 2023-12-07T10:13:23.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:23 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:23.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:23 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:23.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:23 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:13:23.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:23 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:23.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:23 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:13:23.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:23 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:23.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:23 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:13:23.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:23 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:24.449 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:24 smithi119 ceph-mon[97908]: purged_snaps scrub starts 2023-12-07T10:13:24.450 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:24 smithi119 ceph-mon[97908]: purged_snaps scrub ok 2023-12-07T10:13:24.450 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:24 smithi119 ceph-mon[97908]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:24.450 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:24 smithi119 ceph-mon[97908]: osd.2 [v2:172.21.15.159:6800/708101637,v1:172.21.15.159:6801/708101637] boot 2023-12-07T10:13:24.450 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:24 smithi119 ceph-mon[97908]: osdmap e18: 3 total, 3 up, 3 in 2023-12-07T10:13:24.450 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:24 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T10:13:24.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:24 smithi159 ceph-mon[134726]: purged_snaps scrub starts 2023-12-07T10:13:24.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:24 smithi159 ceph-mon[134726]: purged_snaps scrub ok 2023-12-07T10:13:24.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:24 smithi159 ceph-mon[134726]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:24.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:24 smithi159 ceph-mon[134726]: osd.2 [v2:172.21.15.159:6800/708101637,v1:172.21.15.159:6801/708101637] boot 2023-12-07T10:13:24.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:24 smithi159 ceph-mon[134726]: osdmap e18: 3 total, 3 up, 3 in 2023-12-07T10:13:24.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:24 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T10:13:24.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:24 smithi153 ceph-mon[133625]: purged_snaps scrub starts 2023-12-07T10:13:24.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:24 smithi153 ceph-mon[133625]: purged_snaps scrub ok 2023-12-07T10:13:24.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:24 smithi153 ceph-mon[133625]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-07T10:13:24.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:24 smithi153 ceph-mon[133625]: osd.2 [v2:172.21.15.159:6800/708101637,v1:172.21.15.159:6801/708101637] boot 2023-12-07T10:13:24.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:24 smithi153 ceph-mon[133625]: osdmap e18: 3 total, 3 up, 3 in 2023-12-07T10:13:24.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:24 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T10:13:25.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.100+0000 7f15a9bd2700 1 -- 172.21.15.119:0/3774194012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a4101660 msgr2=0x7f15a4101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:13:25.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.100+0000 7f15a9bd2700 1 --2- 172.21.15.119:0/3774194012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a4101660 0x7f15a4101a80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f15940099c0 tx=0x7f1594009cd0 comp rx=0 tx=0).stop 2023-12-07T10:13:25.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.100+0000 7f15a9bd2700 1 -- 172.21.15.119:0/3774194012 shutdown_connections 2023-12-07T10:13:25.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.100+0000 7f15a9bd2700 1 --2- 172.21.15.119:0/3774194012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15a4103200 0x7f15a4109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:13:25.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.100+0000 7f15a9bd2700 1 --2- 172.21.15.119:0/3774194012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a4101660 0x7f15a4101a80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:13:25.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.100+0000 7f15a9bd2700 1 --2- 172.21.15.119:0/3774194012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a4102860 0x7f15a4102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:13:25.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.100+0000 7f15a9bd2700 1 -- 172.21.15.119:0/3774194012 >> 172.21.15.119:0/3774194012 conn(0x7f15a40fcc20 msgr2=0x7f15a40ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:13:25.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.100+0000 7f15a9bd2700 1 -- 172.21.15.119:0/3774194012 shutdown_connections 2023-12-07T10:13:25.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.100+0000 7f15a9bd2700 1 -- 172.21.15.119:0/3774194012 wait complete. 2023-12-07T10:13:25.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.101+0000 7f15a9bd2700 1 Processor -- start 2023-12-07T10:13:25.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.101+0000 7f15a9bd2700 1 -- start start 2023-12-07T10:13:25.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.101+0000 7f15a9bd2700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a4101660 0x7f15a41997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:13:25.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.101+0000 7f15a9bd2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a4102860 0x7f15a4199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:13:25.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.101+0000 7f15a9bd2700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15a4103200 0x7f15a41a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:13:25.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.101+0000 7f15a9bd2700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a41a1320 con 0x7f15a4101660 2023-12-07T10:13:25.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.101+0000 7f15a9bd2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a41a1460 con 0x7f15a4102860 2023-12-07T10:13:25.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.101+0000 7f15a9bd2700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a41a15f0 con 0x7f15a4103200 2023-12-07T10:13:25.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.102+0000 7f15a37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a4101660 0x7f15a41997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:13:25.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.102+0000 7f15a2ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a4102860 0x7f15a4199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:13:25.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.102+0000 7f15a37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a4101660 0x7f15a41997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37226/0 (socket says 172.21.15.119:37226) 2023-12-07T10:13:25.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.102+0000 7f15a37fe700 1 -- 172.21.15.119:0/2080167662 learned_addr learned my addr 172.21.15.119:0/2080167662 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:13:25.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.102+0000 7f15a2ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a4102860 0x7f15a4199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:47190/0 (socket says 172.21.15.119:47190) 2023-12-07T10:13:25.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.102+0000 7f15a3fff700 1 --2- 172.21.15.119:0/2080167662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15a4103200 0x7f15a41a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:13:25.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.102+0000 7f15a2ffd700 1 -- 172.21.15.119:0/2080167662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15a4103200 msgr2=0x7f15a41a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:13:25.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.102+0000 7f15a2ffd700 1 --2- 172.21.15.119:0/2080167662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15a4103200 0x7f15a41a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:13:25.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.102+0000 7f15a2ffd700 1 -- 172.21.15.119:0/2080167662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a4101660 msgr2=0x7f15a41997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:13:25.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.102+0000 7f15a2ffd700 1 --2- 172.21.15.119:0/2080167662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a4101660 0x7f15a41997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:13:25.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.102+0000 7f15a2ffd700 1 -- 172.21.15.119:0/2080167662 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15940095f0 con 0x7f15a4102860 2023-12-07T10:13:25.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.102+0000 7f15a2ffd700 1 --2- 172.21.15.119:0/2080167662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a4102860 0x7f15a4199d20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f159800d950 tx=0x7f159800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:13:25.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.103+0000 7f15a0ff9700 1 -- 172.21.15.119:0/2080167662 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159800f890 con 0x7f15a4102860 2023-12-07T10:13:25.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.103+0000 7f15a9bd2700 1 -- 172.21.15.119:0/2080167662 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a41a18d0 con 0x7f15a4102860 2023-12-07T10:13:25.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.103+0000 7f15a0ff9700 1 -- 172.21.15.119:0/2080167662 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1598010460 con 0x7f15a4102860 2023-12-07T10:13:25.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.103+0000 7f15a0ff9700 1 -- 172.21.15.119:0/2080167662 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159800e5f0 con 0x7f15a4102860 2023-12-07T10:13:25.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.103+0000 7f15a9bd2700 1 -- 172.21.15.119:0/2080167662 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a41a1dd0 con 0x7f15a4102860 2023-12-07T10:13:25.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.104+0000 7f15a9bd2700 1 -- 172.21.15.119:0/2080167662 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a404f2d0 con 0x7f15a4102860 2023-12-07T10:13:25.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.106+0000 7f15a0ff9700 1 -- 172.21.15.119:0/2080167662 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 15) v1 ==== 136428+0+0 (secure 0 0 0) 0x7f15980105d0 con 0x7f15a4102860 2023-12-07T10:13:25.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.107+0000 7f15a0ff9700 1 --2- 172.21.15.119:0/2080167662 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f158c0a1a80 0x7f158c0a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:13:25.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.107+0000 7f15a0ff9700 1 -- 172.21.15.119:0/2080167662 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f15980cc160 con 0x7f15a4102860 2023-12-07T10:13:25.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.107+0000 7f15a37fe700 1 --2- 172.21.15.119:0/2080167662 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f158c0a1a80 0x7f158c0a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:13:25.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.108+0000 7f15a37fe700 1 --2- 172.21.15.119:0/2080167662 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f158c0a1a80 0x7f158c0a3f40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1594000c00 tx=0x7f1594005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:13:25.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.110+0000 7f15a0ff9700 1 -- 172.21.15.119:0/2080167662 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1598096b10 con 0x7f15a4102860 2023-12-07T10:13:25.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:25.321+0000 7f15a9bd2700 1 -- 172.21.15.119:0/2080167662 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f15a41097f0 con 0x7f158c0a1a80 2023-12-07T10:13:25.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:25 smithi119 ceph-mon[97908]: osdmap e19: 3 total, 3 up, 3 in 2023-12-07T10:13:25.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:25 smithi119 ceph-mon[97908]: pgmap v81: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-07T10:13:25.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:25 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-07T10:13:25.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:25 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:13:25.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:25 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:13:25.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:25 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:25.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:25 smithi159 ceph-mon[134726]: osdmap e19: 3 total, 3 up, 3 in 2023-12-07T10:13:25.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:25 smithi159 ceph-mon[134726]: pgmap v81: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-07T10:13:25.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:25 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-07T10:13:25.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:25 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:13:25.765 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:25 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:13:25.765 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:25 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:25.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:25 smithi153 ceph-mon[133625]: osdmap e19: 3 total, 3 up, 3 in 2023-12-07T10:13:25.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:25 smithi153 ceph-mon[133625]: pgmap v81: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-07T10:13:25.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:25 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-07T10:13:25.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:25 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:13:25.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:25 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:13:25.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:25 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:26.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:26 smithi119 ceph-mon[97908]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:13:26.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:26 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-07T10:13:26.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:26 smithi119 ceph-mon[97908]: osdmap e20: 3 total, 3 up, 3 in 2023-12-07T10:13:26.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:26 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-07T10:13:26.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:26 smithi159 ceph-mon[134726]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:13:26.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:26 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-07T10:13:26.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:26 smithi159 ceph-mon[134726]: osdmap e20: 3 total, 3 up, 3 in 2023-12-07T10:13:26.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:26 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-07T10:13:26.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:26 smithi153 ceph-mon[133625]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:13:26.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:26 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-07T10:13:26.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:26 smithi153 ceph-mon[133625]: osdmap e20: 3 total, 3 up, 3 in 2023-12-07T10:13:26.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:26 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-07T10:13:27.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:27 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-07T10:13:27.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:27 smithi119 ceph-mon[97908]: osdmap e21: 3 total, 3 up, 3 in 2023-12-07T10:13:27.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:27 smithi119 ceph-mon[97908]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-07T10:13:27.731 INFO:journalctl@ceph.osd.0.smithi119.stdout:Dec 07 10:13:27 smithi119 sudo[119256]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-07T10:13:27.731 INFO:journalctl@ceph.osd.0.smithi119.stdout:Dec 07 10:13:27 smithi119 sudo[119256]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-07T10:13:27.731 INFO:journalctl@ceph.osd.0.smithi119.stdout:Dec 07 10:13:27 smithi119 sudo[119256]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-07T10:13:27.731 INFO:journalctl@ceph.osd.0.smithi119.stdout:Dec 07 10:13:27 smithi119 sudo[119256]: pam_unix(sudo:session): session closed for user root 2023-12-07T10:13:27.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:27 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-07T10:13:27.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:27 smithi159 ceph-mon[134726]: osdmap e21: 3 total, 3 up, 3 in 2023-12-07T10:13:27.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:27 smithi159 ceph-mon[134726]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-07T10:13:27.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:27 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-07T10:13:27.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:27 smithi153 ceph-mon[133625]: osdmap e21: 3 total, 3 up, 3 in 2023-12-07T10:13:27.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:27 smithi153 ceph-mon[133625]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-07T10:13:28.230 INFO:journalctl@ceph.osd.0.smithi119.stdout:Dec 07 10:13:27 smithi119 sudo[119282]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-07T10:13:28.230 INFO:journalctl@ceph.osd.0.smithi119.stdout:Dec 07 10:13:27 smithi119 sudo[119282]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-07T10:13:28.230 INFO:journalctl@ceph.osd.0.smithi119.stdout:Dec 07 10:13:27 smithi119 sudo[119282]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-07T10:13:28.230 INFO:journalctl@ceph.osd.0.smithi119.stdout:Dec 07 10:13:27 smithi119 sudo[119282]: pam_unix(sudo:session): session closed for user root 2023-12-07T10:13:28.474 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:28 smithi153 ceph-mon[133625]: osdmap e22: 3 total, 3 up, 3 in 2023-12-07T10:13:28.474 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:28 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:28.474 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:28 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/507836737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26ef8300-aa07-48f6-b31d-8b0f2217ffcc"}]: dispatch 2023-12-07T10:13:28.475 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:28 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/507836737' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26ef8300-aa07-48f6-b31d-8b0f2217ffcc"}]': finished 2023-12-07T10:13:28.475 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:28 smithi153 ceph-mon[133625]: osdmap e23: 4 total, 3 up, 4 in 2023-12-07T10:13:28.475 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:28 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T10:13:28.475 INFO:journalctl@ceph.osd.1.smithi153.stdout:Dec 07 10:13:28 smithi153 sudo[144054]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-07T10:13:28.475 INFO:journalctl@ceph.osd.1.smithi153.stdout:Dec 07 10:13:28 smithi153 sudo[144054]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-07T10:13:28.475 INFO:journalctl@ceph.osd.1.smithi153.stdout:Dec 07 10:13:28 smithi153 sudo[144054]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-07T10:13:28.475 INFO:journalctl@ceph.osd.1.smithi153.stdout:Dec 07 10:13:28 smithi153 sudo[144054]: pam_unix(sudo:session): session closed for user root 2023-12-07T10:13:28.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:28 smithi119 ceph-mon[97908]: osdmap e22: 3 total, 3 up, 3 in 2023-12-07T10:13:28.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:28 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:28.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:28 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/507836737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26ef8300-aa07-48f6-b31d-8b0f2217ffcc"}]: dispatch 2023-12-07T10:13:28.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:28 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/507836737' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26ef8300-aa07-48f6-b31d-8b0f2217ffcc"}]': finished 2023-12-07T10:13:28.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:28 smithi119 ceph-mon[97908]: osdmap e23: 4 total, 3 up, 4 in 2023-12-07T10:13:28.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:28 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T10:13:28.740 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:28 smithi159 ceph-mon[134726]: osdmap e22: 3 total, 3 up, 3 in 2023-12-07T10:13:28.740 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:28 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:28.740 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:28 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/507836737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26ef8300-aa07-48f6-b31d-8b0f2217ffcc"}]: dispatch 2023-12-07T10:13:28.740 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:28 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/507836737' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26ef8300-aa07-48f6-b31d-8b0f2217ffcc"}]': finished 2023-12-07T10:13:28.740 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:28 smithi159 ceph-mon[134726]: osdmap e23: 4 total, 3 up, 4 in 2023-12-07T10:13:28.741 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:28 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T10:13:28.781 INFO:journalctl@ceph.osd.1.smithi153.stdout:Dec 07 10:13:28 smithi153 sudo[144057]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-07T10:13:28.781 INFO:journalctl@ceph.osd.1.smithi153.stdout:Dec 07 10:13:28 smithi153 sudo[144057]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-07T10:13:28.781 INFO:journalctl@ceph.osd.1.smithi153.stdout:Dec 07 10:13:28 smithi153 sudo[144057]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-07T10:13:28.781 INFO:journalctl@ceph.osd.1.smithi153.stdout:Dec 07 10:13:28 smithi153 sudo[144057]: pam_unix(sudo:session): session closed for user root 2023-12-07T10:13:28.996 INFO:journalctl@ceph.osd.2.smithi159.stdout:Dec 07 10:13:28 smithi159 sudo[145982]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-07T10:13:28.996 INFO:journalctl@ceph.osd.2.smithi159.stdout:Dec 07 10:13:28 smithi159 sudo[145982]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-07T10:13:28.996 INFO:journalctl@ceph.osd.2.smithi159.stdout:Dec 07 10:13:28 smithi159 sudo[145982]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-07T10:13:28.996 INFO:journalctl@ceph.osd.2.smithi159.stdout:Dec 07 10:13:28 smithi159 sudo[145982]: pam_unix(sudo:session): session closed for user root 2023-12-07T10:13:29.264 INFO:journalctl@ceph.osd.2.smithi159.stdout:Dec 07 10:13:28 smithi159 sudo[145985]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-07T10:13:29.264 INFO:journalctl@ceph.osd.2.smithi159.stdout:Dec 07 10:13:28 smithi159 sudo[145985]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-07T10:13:29.264 INFO:journalctl@ceph.osd.2.smithi159.stdout:Dec 07 10:13:28 smithi159 sudo[145985]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-07T10:13:29.264 INFO:journalctl@ceph.osd.2.smithi159.stdout:Dec 07 10:13:28 smithi159 sudo[145985]: pam_unix(sudo:session): session closed for user root 2023-12-07T10:13:29.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:29 smithi119 sudo[119397]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-07T10:13:29.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:29 smithi119 sudo[119397]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-07T10:13:29.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:29 smithi119 sudo[119397]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-07T10:13:29.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:29 smithi119 sudo[119397]: pam_unix(sudo:session): session closed for user root 2023-12-07T10:13:30.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:30.095+0000 7f15a0ff9700 1 -- 172.21.15.119:0/2080167662 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f15980108d0 con 0x7f15a4102860 2023-12-07T10:13:30.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:30 smithi119 sudo[119588]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-07T10:13:30.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:30 smithi119 sudo[119588]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-07T10:13:30.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:30 smithi119 sudo[119588]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-07T10:13:30.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:30 smithi119 sudo[119588]: pam_unix(sudo:session): session closed for user root 2023-12-07T10:13:30.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:30 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:30.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:30 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:30.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:30 smithi119 ceph-mon[97908]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-07T10:13:30.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:30 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:30.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:30 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:30.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:30 smithi153 ceph-mon[133625]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-07T10:13:30.579 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:30 smithi159 sudo[145996]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-07T10:13:30.579 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:30 smithi159 sudo[145996]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-07T10:13:30.579 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:30 smithi159 sudo[145996]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-07T10:13:30.579 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:30 smithi159 sudo[145996]: pam_unix(sudo:session): session closed for user root 2023-12-07T10:13:30.579 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:30 smithi159 sudo[145999]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-07T10:13:31.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:30 smithi159 sudo[145999]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-07T10:13:31.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:30 smithi159 sudo[145999]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-07T10:13:31.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:30 smithi159 sudo[145999]: pam_unix(sudo:session): session closed for user root 2023-12-07T10:13:31.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:30 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:31.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:30 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:31.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:30 smithi159 ceph-mon[134726]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-07T10:13:31.073 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:30 smithi153 sudo[144060]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-07T10:13:31.073 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:30 smithi153 sudo[144060]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-07T10:13:31.073 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:30 smithi153 sudo[144060]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-07T10:13:31.073 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:30 smithi153 sudo[144060]: pam_unix(sudo:session): session closed for user root 2023-12-07T10:13:31.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:31 smithi119 ceph-mon[97908]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-07T10:13:31.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:31 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1602800579' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:13:31.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:31 smithi119 ceph-mon[97908]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-07T10:13:31.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:31 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T10:13:31.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:31 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:13:31.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:31 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:13:31.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:31 smithi119 ceph-mon[97908]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-07T10:13:31.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:31 smithi119 ceph-mon[97908]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-12-07T10:13:31.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:31 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T10:13:31.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:31 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:13:31.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:31 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:13:31.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:31 smithi119 ceph-mon[97908]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-07T10:13:31.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:31 smithi119 ceph-mon[97908]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-07T10:13:31.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:31 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T10:13:31.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:31 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:13:31.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:31 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:13:31.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:31 smithi119 ceph-mon[97908]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-07T10:13:31.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:31 smithi159 ceph-mon[134726]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-07T10:13:31.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:31 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1602800579' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:13:31.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:31 smithi159 ceph-mon[134726]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-07T10:13:31.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:31 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T10:13:31.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:31 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:13:31.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:31 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:13:31.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:31 smithi159 ceph-mon[134726]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-07T10:13:31.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:31 smithi159 ceph-mon[134726]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-12-07T10:13:31.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:31 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T10:13:31.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:31 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:13:31.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:31 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:13:31.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:31 smithi159 ceph-mon[134726]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-07T10:13:31.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:31 smithi159 ceph-mon[134726]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-07T10:13:31.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:31 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T10:13:31.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:31 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:13:31.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:31 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:13:31.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:31 smithi159 ceph-mon[134726]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-07T10:13:31.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:31 smithi153 sudo[144063]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-07T10:13:31.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:31 smithi153 sudo[144063]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-07T10:13:31.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:31 smithi153 sudo[144063]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-07T10:13:31.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:31 smithi153 sudo[144063]: pam_unix(sudo:session): session closed for user root 2023-12-07T10:13:31.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:31 smithi153 ceph-mon[133625]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-07T10:13:31.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:31 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1602800579' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:13:31.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:31 smithi153 ceph-mon[133625]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-07T10:13:31.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:31 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T10:13:31.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:31 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:13:31.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:31 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:13:31.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:31 smithi153 ceph-mon[133625]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-07T10:13:31.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:31 smithi153 ceph-mon[133625]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-12-07T10:13:31.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:31 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T10:13:31.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:31 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:13:31.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:31 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:13:31.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:31 smithi153 ceph-mon[133625]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-07T10:13:31.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:31 smithi153 ceph-mon[133625]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-07T10:13:31.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:31 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T10:13:31.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:31 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:13:31.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:31 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:13:31.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:31 smithi153 ceph-mon[133625]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-07T10:13:32.388 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:32 smithi119 ceph-mon[97908]: pgmap v88: 1 pgs: 1 active+clean; 449 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-12-07T10:13:32.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:32 smithi159 ceph-mon[134726]: pgmap v88: 1 pgs: 1 active+clean; 449 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-12-07T10:13:32.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:32 smithi153 ceph-mon[133625]: pgmap v88: 1 pgs: 1 active+clean; 449 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-12-07T10:13:34.203 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:34 smithi119 ceph-mon[97908]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 7.7 KiB/s rd, 351 KiB/s wr, 24 op/s 2023-12-07T10:13:34.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:34 smithi159 ceph-mon[134726]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 7.7 KiB/s rd, 351 KiB/s wr, 24 op/s 2023-12-07T10:13:34.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:34 smithi153 ceph-mon[133625]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 7.7 KiB/s rd, 351 KiB/s wr, 24 op/s 2023-12-07T10:13:36.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:36 smithi119 ceph-mon[97908]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 5.5 KiB/s rd, 254 KiB/s wr, 17 op/s 2023-12-07T10:13:36.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:36 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-07T10:13:36.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:36 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:36.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:36 smithi159 ceph-mon[134726]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 5.5 KiB/s rd, 254 KiB/s wr, 17 op/s 2023-12-07T10:13:36.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:36 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-07T10:13:36.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:36 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:36.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:36 smithi153 ceph-mon[133625]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 5.5 KiB/s rd, 254 KiB/s wr, 17 op/s 2023-12-07T10:13:36.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:36 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-07T10:13:36.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:36 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:37.186 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:37 smithi119 ceph-mon[97908]: Deploying daemon osd.3 on smithi119 2023-12-07T10:13:37.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:37 smithi159 ceph-mon[134726]: Deploying daemon osd.3 on smithi119 2023-12-07T10:13:37.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:37 smithi153 ceph-mon[133625]: Deploying daemon osd.3 on smithi119 2023-12-07T10:13:38.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:38 smithi119 ceph-mon[97908]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 4.3 KiB/s rd, 198 KiB/s wr, 13 op/s 2023-12-07T10:13:38.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:38 smithi159 ceph-mon[134726]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 4.3 KiB/s rd, 198 KiB/s wr, 13 op/s 2023-12-07T10:13:38.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:38 smithi153 ceph-mon[133625]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 4.3 KiB/s rd, 198 KiB/s wr, 13 op/s 2023-12-07T10:13:40.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:40 smithi119 ceph-mon[97908]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.9 KiB/s rd, 179 KiB/s wr, 12 op/s 2023-12-07T10:13:40.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:40 smithi159 ceph-mon[134726]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.9 KiB/s rd, 179 KiB/s wr, 12 op/s 2023-12-07T10:13:40.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:40 smithi153 ceph-mon[133625]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.9 KiB/s rd, 179 KiB/s wr, 12 op/s 2023-12-07T10:13:41.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:41 smithi119 ceph-mon[97908]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-12-07T10:13:41.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:41 smithi159 ceph-mon[134726]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-12-07T10:13:41.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:41 smithi153 ceph-mon[133625]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-12-07T10:13:43.431 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:43 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:43.432 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:43 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:43.432 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:43 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:13:43.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:43 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:43.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:43 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:43.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:43 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:13:43.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:43 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:43.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:43 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:43.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:43 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:13:44.203 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:44 smithi119 ceph-mon[97908]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-12-07T10:13:44.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:44 smithi159 ceph-mon[134726]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-12-07T10:13:44.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:44 smithi153 ceph-mon[133625]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-12-07T10:13:45.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:45.932+0000 7f15a0ff9700 1 -- 172.21.15.119:0/2080167662 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f15a41097f0 con 0x7f158c0a1a80 2023-12-07T10:13:45.935 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 3 on host 'smithi119' 2023-12-07T10:13:45.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:45.934+0000 7f15a9bd2700 1 -- 172.21.15.119:0/2080167662 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f158c0a1a80 msgr2=0x7f158c0a3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:13:45.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:45.934+0000 7f15a9bd2700 1 --2- 172.21.15.119:0/2080167662 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f158c0a1a80 0x7f158c0a3f40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1594000c00 tx=0x7f1594005a90 comp rx=0 tx=0).stop 2023-12-07T10:13:45.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:45.934+0000 7f15a9bd2700 1 -- 172.21.15.119:0/2080167662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a4102860 msgr2=0x7f15a4199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:13:45.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:45.934+0000 7f15a9bd2700 1 --2- 172.21.15.119:0/2080167662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a4102860 0x7f15a4199d20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f159800d950 tx=0x7f159800dc60 comp rx=0 tx=0).stop 2023-12-07T10:13:45.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:45.934+0000 7f15a9bd2700 1 -- 172.21.15.119:0/2080167662 shutdown_connections 2023-12-07T10:13:45.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:45.934+0000 7f15a9bd2700 1 --2- 172.21.15.119:0/2080167662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15a4103200 0x7f15a41a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:13:45.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:45.935+0000 7f15a9bd2700 1 --2- 172.21.15.119:0/2080167662 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f158c0a1a80 0x7f158c0a3f40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:13:45.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:45.935+0000 7f15a9bd2700 1 --2- 172.21.15.119:0/2080167662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a4101660 0x7f15a41997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:13:45.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:45.935+0000 7f15a9bd2700 1 --2- 172.21.15.119:0/2080167662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a4102860 0x7f15a4199d20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:13:45.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:45.935+0000 7f15a9bd2700 1 -- 172.21.15.119:0/2080167662 >> 172.21.15.119:0/2080167662 conn(0x7f15a40fcc20 msgr2=0x7f15a41080d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:13:45.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:45.935+0000 7f15a9bd2700 1 -- 172.21.15.119:0/2080167662 shutdown_connections 2023-12-07T10:13:45.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:13:45.935+0000 7f15a9bd2700 1 -- 172.21.15.119:0/2080167662 wait complete. 2023-12-07T10:13:46.101 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:45 smithi119 ceph-mon[97908]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-12-07T10:13:46.101 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:45 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:46.101 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:45 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:46.101 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:45 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:46.102 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:45 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:13:46.102 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:45 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:46.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:45 smithi159 ceph-mon[134726]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-12-07T10:13:46.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:46.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:46.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:46.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:13:46.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:46.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:45 smithi153 ceph-mon[133625]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-12-07T10:13:46.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:45 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:46.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:45 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:46.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:45 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:46.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:45 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:13:46.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:45 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:46.353 DEBUG:teuthology.orchestra.run.smithi119:osd.3> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.3.service 2023-12-07T10:13:46.357 INFO:tasks.cephadm:Deploying osd.4 on smithi153 with /dev/vg_nvme/lv_3... 2023-12-07T10:13:46.358 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-12-07T10:13:46.426 INFO:journalctl@ceph.osd.3.smithi119.stdout:-- Logs begin at Thu 2023-12-07 09:53:13 UTC. -- 2023-12-07T10:13:46.962 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T10:13:47.174 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:46 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:47.174 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:46 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:47.174 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:46 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:13:47.175 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:46 smithi119 ceph-mon[97908]: from='osd.3 [v2:172.21.15.119:6810/3228047257,v1:172.21.15.119:6811/3228047257]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-07T10:13:47.176 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:46 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:47.176 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:46 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:47.176 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:46 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:13:47.176 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:46 smithi153 ceph-mon[133625]: from='osd.3 [v2:172.21.15.119:6810/3228047257,v1:172.21.15.119:6811/3228047257]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-07T10:13:47.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:46 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:47.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:46 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:47.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:46 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:13:47.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:46 smithi159 ceph-mon[134726]: from='osd.3 [v2:172.21.15.119:6810/3228047257,v1:172.21.15.119:6811/3228047257]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-07T10:13:48.231 INFO:journalctl@ceph.osd.3.smithi119.stdout:Dec 07 10:13:47 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-3[121584]: 2023-12-07T10:13:47.921+0000 7f06da138700 -1 osd.3 0 waiting for initial osdmap 2023-12-07T10:13:48.231 INFO:journalctl@ceph.osd.3.smithi119.stdout:Dec 07 10:13:47 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-3[121584]: 2023-12-07T10:13:47.930+0000 7f06d272b700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-07T10:13:48.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:47 smithi119 ceph-mon[97908]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-12-07T10:13:48.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:47 smithi119 ceph-mon[97908]: from='osd.3 [v2:172.21.15.119:6810/3228047257,v1:172.21.15.119:6811/3228047257]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-07T10:13:48.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:47 smithi119 ceph-mon[97908]: osdmap e24: 4 total, 3 up, 4 in 2023-12-07T10:13:48.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:47 smithi119 ceph-mon[97908]: from='osd.3 [v2:172.21.15.119:6810/3228047257,v1:172.21.15.119:6811/3228047257]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2023-12-07T10:13:48.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:47 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T10:13:48.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:47 smithi159 ceph-mon[134726]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-12-07T10:13:48.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:47 smithi159 ceph-mon[134726]: from='osd.3 [v2:172.21.15.119:6810/3228047257,v1:172.21.15.119:6811/3228047257]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-07T10:13:48.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:47 smithi159 ceph-mon[134726]: osdmap e24: 4 total, 3 up, 4 in 2023-12-07T10:13:48.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:47 smithi159 ceph-mon[134726]: from='osd.3 [v2:172.21.15.119:6810/3228047257,v1:172.21.15.119:6811/3228047257]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2023-12-07T10:13:48.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:47 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T10:13:48.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:47 smithi153 ceph-mon[133625]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-12-07T10:13:48.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:47 smithi153 ceph-mon[133625]: from='osd.3 [v2:172.21.15.119:6810/3228047257,v1:172.21.15.119:6811/3228047257]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-07T10:13:48.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:47 smithi153 ceph-mon[133625]: osdmap e24: 4 total, 3 up, 4 in 2023-12-07T10:13:48.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:47 smithi153 ceph-mon[133625]: from='osd.3 [v2:172.21.15.119:6810/3228047257,v1:172.21.15.119:6811/3228047257]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2023-12-07T10:13:48.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:47 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T10:13:48.606 INFO:teuthology.orchestra.run.smithi153.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-12-07T10:13:48.607 INFO:teuthology.orchestra.run.smithi153.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-12-07T10:13:48.607 INFO:teuthology.orchestra.run.smithi153.stderr: stderr: 10+0 records in 2023-12-07T10:13:48.607 INFO:teuthology.orchestra.run.smithi153.stderr:10+0 records out 2023-12-07T10:13:48.607 INFO:teuthology.orchestra.run.smithi153.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0119497 s, 877 MB/s 2023-12-07T10:13:48.607 INFO:teuthology.orchestra.run.smithi153.stderr: stderr: 2023-12-07T10:13:48.607 INFO:teuthology.orchestra.run.smithi153.stderr:--> Zapping successful for: 2023-12-07T10:13:49.019 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi153:vg_nvme/lv_3 2023-12-07T10:13:49.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:48 smithi119 ceph-mon[97908]: from='osd.3 [v2:172.21.15.119:6810/3228047257,v1:172.21.15.119:6811/3228047257]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2023-12-07T10:13:49.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:48 smithi119 ceph-mon[97908]: osdmap e25: 4 total, 3 up, 4 in 2023-12-07T10:13:49.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:48 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T10:13:49.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:48 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T10:13:49.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:48 smithi159 ceph-mon[134726]: from='osd.3 [v2:172.21.15.119:6810/3228047257,v1:172.21.15.119:6811/3228047257]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2023-12-07T10:13:49.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:48 smithi159 ceph-mon[134726]: osdmap e25: 4 total, 3 up, 4 in 2023-12-07T10:13:49.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:48 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T10:13:49.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:48 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T10:13:49.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:48 smithi153 ceph-mon[133625]: from='osd.3 [v2:172.21.15.119:6810/3228047257,v1:172.21.15.119:6811/3228047257]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2023-12-07T10:13:49.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:48 smithi153 ceph-mon[133625]: osdmap e25: 4 total, 3 up, 4 in 2023-12-07T10:13:49.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:48 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T10:13:49.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:48 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T10:13:49.613 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T10:13:50.160 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:49 smithi153 ceph-mon[133625]: purged_snaps scrub starts 2023-12-07T10:13:50.161 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:49 smithi153 ceph-mon[133625]: purged_snaps scrub ok 2023-12-07T10:13:50.161 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:49 smithi153 ceph-mon[133625]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-12-07T10:13:50.161 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:49 smithi153 ceph-mon[133625]: osd.3 [v2:172.21.15.119:6810/3228047257,v1:172.21.15.119:6811/3228047257] boot 2023-12-07T10:13:50.161 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:49 smithi153 ceph-mon[133625]: osdmap e26: 4 total, 4 up, 4 in 2023-12-07T10:13:50.161 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:49 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T10:13:50.161 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:49 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:50.161 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:49 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:50.161 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:49 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:13:50.161 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:49 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:13:50.161 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:49 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:50.161 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:49 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:50.161 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:49 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:13:50.162 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:49 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:50.162 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:49 smithi153 ceph-mon[133625]: osdmap e27: 4 total, 4 up, 4 in 2023-12-07T10:13:50.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:49 smithi119 ceph-mon[97908]: purged_snaps scrub starts 2023-12-07T10:13:50.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:49 smithi119 ceph-mon[97908]: purged_snaps scrub ok 2023-12-07T10:13:50.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:49 smithi119 ceph-mon[97908]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-12-07T10:13:50.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:49 smithi119 ceph-mon[97908]: osd.3 [v2:172.21.15.119:6810/3228047257,v1:172.21.15.119:6811/3228047257] boot 2023-12-07T10:13:50.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:49 smithi119 ceph-mon[97908]: osdmap e26: 4 total, 4 up, 4 in 2023-12-07T10:13:50.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:49 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T10:13:50.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:49 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:50.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:49 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:50.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:49 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:13:50.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:49 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:13:50.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:49 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:50.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:49 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:50.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:49 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:13:50.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:49 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:50.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:49 smithi119 ceph-mon[97908]: osdmap e27: 4 total, 4 up, 4 in 2023-12-07T10:13:50.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:49 smithi159 ceph-mon[134726]: purged_snaps scrub starts 2023-12-07T10:13:50.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:49 smithi159 ceph-mon[134726]: purged_snaps scrub ok 2023-12-07T10:13:50.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:49 smithi159 ceph-mon[134726]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-12-07T10:13:50.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:49 smithi159 ceph-mon[134726]: osd.3 [v2:172.21.15.119:6810/3228047257,v1:172.21.15.119:6811/3228047257] boot 2023-12-07T10:13:50.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:49 smithi159 ceph-mon[134726]: osdmap e26: 4 total, 4 up, 4 in 2023-12-07T10:13:50.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:49 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T10:13:50.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:49 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:50.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:49 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:50.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:49 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:13:50.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:49 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:13:50.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:49 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:50.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:49 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:50.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:49 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:13:50.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:49 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:13:50.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:49 smithi159 ceph-mon[134726]: osdmap e27: 4 total, 4 up, 4 in 2023-12-07T10:13:50.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.867+0000 7f916feb2700 1 -- 172.21.15.153:0/1480358312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91681020f0 msgr2=0x7f91681089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:13:50.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.867+0000 7f916feb2700 1 --2- 172.21.15.153:0/1480358312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91681020f0 0x7f91681089a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f916000b0d0 tx=0x7f916000b3e0 comp rx=0 tx=0).stop 2023-12-07T10:13:50.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.867+0000 7f916feb2700 1 -- 172.21.15.153:0/1480358312 shutdown_connections 2023-12-07T10:13:50.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.867+0000 7f916feb2700 1 --2- 172.21.15.153:0/1480358312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91681020f0 0x7f91681089a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:13:50.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.867+0000 7f916feb2700 1 --2- 172.21.15.153:0/1480358312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9168100550 0x7f9168100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:13:50.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.867+0000 7f916feb2700 1 --2- 172.21.15.153:0/1480358312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9168101750 0x7f9168101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:13:50.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.867+0000 7f916feb2700 1 -- 172.21.15.153:0/1480358312 >> 172.21.15.153:0/1480358312 conn(0x7f91680fbb10 msgr2=0x7f91680fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:13:50.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.867+0000 7f916feb2700 1 -- 172.21.15.153:0/1480358312 shutdown_connections 2023-12-07T10:13:50.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.867+0000 7f916feb2700 1 -- 172.21.15.153:0/1480358312 wait complete. 2023-12-07T10:13:50.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.868+0000 7f916feb2700 1 Processor -- start 2023-12-07T10:13:50.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.868+0000 7f916feb2700 1 -- start start 2023-12-07T10:13:50.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.868+0000 7f916feb2700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9168100550 0x7f9168196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:13:50.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.868+0000 7f916feb2700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9168101750 0x7f9168196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:13:50.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.868+0000 7f916feb2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91681020f0 0x7f916819daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:13:50.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.868+0000 7f916feb2700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916819e0c0 con 0x7f9168100550 2023-12-07T10:13:50.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.868+0000 7f916feb2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916819e200 con 0x7f91681020f0 2023-12-07T10:13:50.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.868+0000 7f916feb2700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916819e390 con 0x7f9168101750 2023-12-07T10:13:50.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.868+0000 7f916e44f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91681020f0 0x7f916819daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:13:50.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.868+0000 7f916e44f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91681020f0 0x7f916819daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55720/0 (socket says 172.21.15.153:55720) 2023-12-07T10:13:50.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.868+0000 7f916e44f700 1 -- 172.21.15.153:0/4227913602 learned_addr learned my addr 172.21.15.153:0/4227913602 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-12-07T10:13:50.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.868+0000 7f916e44f700 1 -- 172.21.15.153:0/4227913602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9168101750 msgr2=0x7f9168196ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:13:50.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.868+0000 7f916d44d700 1 --2- 172.21.15.153:0/4227913602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9168101750 0x7f9168196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:13:50.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.868+0000 7f916e44f700 1 --2- 172.21.15.153:0/4227913602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9168101750 0x7f9168196ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:13:50.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.868+0000 7f916e44f700 1 -- 172.21.15.153:0/4227913602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9168100550 msgr2=0x7f9168196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:13:50.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.868+0000 7f916e44f700 1 --2- 172.21.15.153:0/4227913602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9168100550 0x7f9168196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:13:50.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.868+0000 7f916e44f700 1 -- 172.21.15.153:0/4227913602 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9160009d00 con 0x7f91681020f0 2023-12-07T10:13:50.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.868+0000 7f916e44f700 1 --2- 172.21.15.153:0/4227913602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91681020f0 0x7f916819daf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9160007bd0 tx=0x7f9160009570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:13:50.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.869+0000 7f915effd700 1 -- 172.21.15.153:0/4227913602 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916000e040 con 0x7f91681020f0 2023-12-07T10:13:50.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.869+0000 7f915effd700 1 -- 172.21.15.153:0/4227913602 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9160004e60 con 0x7f91681020f0 2023-12-07T10:13:50.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.869+0000 7f916feb2700 1 -- 172.21.15.153:0/4227913602 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916819e610 con 0x7f91681020f0 2023-12-07T10:13:50.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.869+0000 7f915effd700 1 -- 172.21.15.153:0/4227913602 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9160005670 con 0x7f91681020f0 2023-12-07T10:13:50.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.869+0000 7f916feb2700 1 -- 172.21.15.153:0/4227913602 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916819ea80 con 0x7f91681020f0 2023-12-07T10:13:50.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.870+0000 7f916feb2700 1 -- 172.21.15.153:0/4227913602 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91681079e0 con 0x7f91681020f0 2023-12-07T10:13:50.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.871+0000 7f915effd700 1 -- 172.21.15.153:0/4227913602 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f91600057d0 con 0x7f91681020f0 2023-12-07T10:13:50.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.871+0000 7f915effd700 1 --2- 172.21.15.153:0/4227913602 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f91540a1b90 0x7f91540a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:13:50.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.871+0000 7f915effd700 1 -- 172.21.15.153:0/4227913602 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f91600cd530 con 0x7f91681020f0 2023-12-07T10:13:50.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.872+0000 7f916dc4e700 1 --2- 172.21.15.153:0/4227913602 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f91540a1b90 0x7f91540a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:13:50.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.872+0000 7f916dc4e700 1 --2- 172.21.15.153:0/4227913602 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f91540a1b90 0x7f91540a4050 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f91681015b0 tx=0x7f9164005ca0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:13:50.877 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:50.875+0000 7f915effd700 1 -- 172.21.15.153:0/4227913602 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9160097be0 con 0x7f91681020f0 2023-12-07T10:13:51.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:13:51.077+0000 7f916feb2700 1 -- 172.21.15.153:0/4227913602 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f916802d3f0 con 0x7f91540a1b90 2023-12-07T10:13:51.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:50 smithi119 ceph-mon[97908]: Detected new or changed devices on smithi119 2023-12-07T10:13:51.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:50 smithi119 ceph-mon[97908]: Adjusting osd_memory_target on smithi119 to 8635M 2023-12-07T10:13:51.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:50 smithi159 ceph-mon[134726]: Detected new or changed devices on smithi119 2023-12-07T10:13:51.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:50 smithi159 ceph-mon[134726]: Adjusting osd_memory_target on smithi119 to 8635M 2023-12-07T10:13:51.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:50 smithi153 ceph-mon[133625]: Detected new or changed devices on smithi119 2023-12-07T10:13:51.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:50 smithi153 ceph-mon[133625]: Adjusting osd_memory_target on smithi119 to 8635M 2023-12-07T10:13:52.227 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:51 smithi153 ceph-mon[133625]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-12-07T10:13:52.227 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:51 smithi153 ceph-mon[133625]: osdmap e28: 4 total, 4 up, 4 in 2023-12-07T10:13:52.227 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:51 smithi153 ceph-mon[133625]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:13:52.227 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:51 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:13:52.227 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:51 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:13:52.227 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:51 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:52.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:51 smithi119 ceph-mon[97908]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-12-07T10:13:52.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:51 smithi119 ceph-mon[97908]: osdmap e28: 4 total, 4 up, 4 in 2023-12-07T10:13:52.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:51 smithi119 ceph-mon[97908]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:13:52.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:51 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:13:52.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:51 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:13:52.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:51 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:52.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:51 smithi159 ceph-mon[134726]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-12-07T10:13:52.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:51 smithi159 ceph-mon[134726]: osdmap e28: 4 total, 4 up, 4 in 2023-12-07T10:13:52.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:51 smithi159 ceph-mon[134726]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:13:52.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:51 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:13:52.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:51 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:13:52.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:51 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:13:53.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:52 smithi153 ceph-mon[133625]: osdmap e29: 4 total, 4 up, 4 in 2023-12-07T10:13:53.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:52 smithi119 ceph-mon[97908]: osdmap e29: 4 total, 4 up, 4 in 2023-12-07T10:13:53.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:52 smithi159 ceph-mon[134726]: osdmap e29: 4 total, 4 up, 4 in 2023-12-07T10:13:54.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:53 smithi119 ceph-mon[97908]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-07T10:13:54.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:53 smithi119 ceph-mon[97908]: osdmap e30: 4 total, 4 up, 4 in 2023-12-07T10:13:54.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:53 smithi119 ceph-mon[97908]: from='client.? 172.21.15.153:0/2300026947' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "000ee4b9-9896-4901-8cc3-c85fdfc8cbaf"}]: dispatch 2023-12-07T10:13:54.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:53 smithi119 ceph-mon[97908]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "000ee4b9-9896-4901-8cc3-c85fdfc8cbaf"}]: dispatch 2023-12-07T10:13:54.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:53 smithi119 ceph-mon[97908]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "000ee4b9-9896-4901-8cc3-c85fdfc8cbaf"}]': finished 2023-12-07T10:13:54.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:53 smithi119 ceph-mon[97908]: osdmap e31: 5 total, 4 up, 5 in 2023-12-07T10:13:54.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:53 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T10:13:54.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:53 smithi159 ceph-mon[134726]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-07T10:13:54.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:53 smithi159 ceph-mon[134726]: osdmap e30: 4 total, 4 up, 4 in 2023-12-07T10:13:54.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:53 smithi159 ceph-mon[134726]: from='client.? 172.21.15.153:0/2300026947' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "000ee4b9-9896-4901-8cc3-c85fdfc8cbaf"}]: dispatch 2023-12-07T10:13:54.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:53 smithi159 ceph-mon[134726]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "000ee4b9-9896-4901-8cc3-c85fdfc8cbaf"}]: dispatch 2023-12-07T10:13:54.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:53 smithi159 ceph-mon[134726]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "000ee4b9-9896-4901-8cc3-c85fdfc8cbaf"}]': finished 2023-12-07T10:13:54.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:53 smithi159 ceph-mon[134726]: osdmap e31: 5 total, 4 up, 5 in 2023-12-07T10:13:54.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:53 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T10:13:54.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:53 smithi153 ceph-mon[133625]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-07T10:13:54.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:53 smithi153 ceph-mon[133625]: osdmap e30: 4 total, 4 up, 4 in 2023-12-07T10:13:54.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:53 smithi153 ceph-mon[133625]: from='client.? 172.21.15.153:0/2300026947' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "000ee4b9-9896-4901-8cc3-c85fdfc8cbaf"}]: dispatch 2023-12-07T10:13:54.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:53 smithi153 ceph-mon[133625]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "000ee4b9-9896-4901-8cc3-c85fdfc8cbaf"}]: dispatch 2023-12-07T10:13:54.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:53 smithi153 ceph-mon[133625]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "000ee4b9-9896-4901-8cc3-c85fdfc8cbaf"}]': finished 2023-12-07T10:13:54.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:53 smithi153 ceph-mon[133625]: osdmap e31: 5 total, 4 up, 5 in 2023-12-07T10:13:54.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:53 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T10:13:55.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:54 smithi119 ceph-mon[97908]: from='client.? 172.21.15.153:0/252238183' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:13:55.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:54 smithi159 ceph-mon[134726]: from='client.? 172.21.15.153:0/252238183' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:13:55.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:54 smithi153 ceph-mon[133625]: from='client.? 172.21.15.153:0/252238183' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:13:56.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:55 smithi119 ceph-mon[97908]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-07T10:13:56.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:55 smithi159 ceph-mon[134726]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-07T10:13:56.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:55 smithi153 ceph-mon[133625]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-07T10:13:58.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:57 smithi119 ceph-mon[97908]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2023-12-07T10:13:58.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:57 smithi159 ceph-mon[134726]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2023-12-07T10:13:58.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:57 smithi153 ceph-mon[133625]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2023-12-07T10:14:00.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:13:59 smithi153 ceph-mon[133625]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-12-07T10:14:00.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:13:59 smithi119 ceph-mon[97908]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-12-07T10:14:00.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:13:59 smithi159 ceph-mon[134726]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-12-07T10:14:01.159 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-07T10:14:01.159 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:01.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:00 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-07T10:14:01.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:00 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:01.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:00 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-07T10:14:01.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:00 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:02.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:01 smithi153 ceph-mon[133625]: Deploying daemon osd.4 on smithi153 2023-12-07T10:14:02.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:01 smithi153 ceph-mon[133625]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-07T10:14:02.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:01 smithi119 ceph-mon[97908]: Deploying daemon osd.4 on smithi153 2023-12-07T10:14:02.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:01 smithi119 ceph-mon[97908]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-07T10:14:02.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:01 smithi159 ceph-mon[134726]: Deploying daemon osd.4 on smithi153 2023-12-07T10:14:02.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:01 smithi159 ceph-mon[134726]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-07T10:14:04.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:03 smithi119 ceph-mon[97908]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-07T10:14:04.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:03 smithi159 ceph-mon[134726]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-07T10:14:04.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:03 smithi153 ceph-mon[133625]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-07T10:14:06.025 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:05 smithi153 ceph-mon[133625]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-07T10:14:06.025 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:05 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:06.025 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:05 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:06.025 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:05 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:14:06.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:05 smithi119 ceph-mon[97908]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-07T10:14:06.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:05 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:06.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:05 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:06.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:05 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:14:06.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:05 smithi159 ceph-mon[134726]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-07T10:14:06.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:05 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:06.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:05 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:06.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:05 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:14:07.693 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 4 on host 'smithi153' 2023-12-07T10:14:07.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:14:07.689+0000 7f915effd700 1 -- 172.21.15.153:0/4227913602 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f916802d3f0 con 0x7f91540a1b90 2023-12-07T10:14:07.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:14:07.691+0000 7f916feb2700 1 -- 172.21.15.153:0/4227913602 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f91540a1b90 msgr2=0x7f91540a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:14:07.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:14:07.691+0000 7f916feb2700 1 --2- 172.21.15.153:0/4227913602 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f91540a1b90 0x7f91540a4050 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f91681015b0 tx=0x7f9164005ca0 comp rx=0 tx=0).stop 2023-12-07T10:14:07.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:14:07.691+0000 7f916feb2700 1 -- 172.21.15.153:0/4227913602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91681020f0 msgr2=0x7f916819daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:14:07.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:14:07.691+0000 7f916feb2700 1 --2- 172.21.15.153:0/4227913602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91681020f0 0x7f916819daf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9160007bd0 tx=0x7f9160009570 comp rx=0 tx=0).stop 2023-12-07T10:14:07.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:14:07.691+0000 7f916feb2700 1 -- 172.21.15.153:0/4227913602 shutdown_connections 2023-12-07T10:14:07.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:14:07.691+0000 7f916feb2700 1 --2- 172.21.15.153:0/4227913602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91681020f0 0x7f916819daf0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:14:07.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:14:07.691+0000 7f916feb2700 1 --2- 172.21.15.153:0/4227913602 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f91540a1b90 0x7f91540a4050 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:14:07.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:14:07.691+0000 7f916feb2700 1 --2- 172.21.15.153:0/4227913602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9168100550 0x7f9168196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:14:07.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:14:07.691+0000 7f916feb2700 1 --2- 172.21.15.153:0/4227913602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9168101750 0x7f9168196ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:14:07.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:14:07.691+0000 7f916feb2700 1 -- 172.21.15.153:0/4227913602 >> 172.21.15.153:0/4227913602 conn(0x7f91680fbb10 msgr2=0x7f9168104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:14:07.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:14:07.691+0000 7f916feb2700 1 -- 172.21.15.153:0/4227913602 shutdown_connections 2023-12-07T10:14:07.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:14:07.691+0000 7f916feb2700 1 -- 172.21.15.153:0/4227913602 wait complete. 2023-12-07T10:14:08.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:07 smithi153 ceph-mon[133625]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-07T10:14:08.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:07 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:08.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:07 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:08.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:07 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:08.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:07 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:14:08.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:07 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:08.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:07 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:08.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:07 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:08.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:07 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:14:08.110 DEBUG:teuthology.orchestra.run.smithi153:osd.4> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.4.service 2023-12-07T10:14:08.113 INFO:tasks.cephadm:Deploying osd.5 on smithi159 with /dev/vg_nvme/lv_3... 2023-12-07T10:14:08.113 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-12-07T10:14:08.154 INFO:journalctl@ceph.osd.4.smithi153.stdout:-- Logs begin at Thu 2023-12-07 09:49:13 UTC. -- 2023-12-07T10:14:08.157 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:07 smithi159 ceph-mon[134726]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-07T10:14:08.157 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:07 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:08.157 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:07 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:08.157 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:07 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:08.157 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:07 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:14:08.157 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:07 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:08.157 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:07 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:08.157 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:07 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:08.157 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:07 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:14:08.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:07 smithi119 ceph-mon[97908]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-07T10:14:08.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:07 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:08.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:07 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:08.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:07 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:08.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:07 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:14:08.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:07 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:08.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:07 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:08.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:07 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:08.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:07 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:14:08.729 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.c/config 2023-12-07T10:14:08.983 INFO:journalctl@ceph.osd.4.smithi153.stdout:Dec 07 10:14:08 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-4[147230]: 2023-12-07T10:14:08.759+0000 7f16c4b69640 -1 osd.4 0 log_to_monitors true 2023-12-07T10:14:08.984 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:08 smithi153 ceph-mon[133625]: from='osd.4 [v2:172.21.15.153:6808/4021998299,v1:172.21.15.153:6809/4021998299]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-07T10:14:08.984 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:08 smithi153 ceph-mon[133625]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-07T10:14:09.124 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:08 smithi159 ceph-mon[134726]: from='osd.4 [v2:172.21.15.153:6808/4021998299,v1:172.21.15.153:6809/4021998299]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-07T10:14:09.125 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:08 smithi159 ceph-mon[134726]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-07T10:14:09.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:08 smithi119 ceph-mon[97908]: from='osd.4 [v2:172.21.15.153:6808/4021998299,v1:172.21.15.153:6809/4021998299]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-07T10:14:09.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:08 smithi119 ceph-mon[97908]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-07T10:14:10.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:09 smithi159 ceph-mon[134726]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-07T10:14:10.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:09 smithi159 ceph-mon[134726]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-07T10:14:10.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:09 smithi159 ceph-mon[134726]: from='osd.4 [v2:172.21.15.153:6808/4021998299,v1:172.21.15.153:6809/4021998299]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2023-12-07T10:14:10.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:09 smithi159 ceph-mon[134726]: osdmap e32: 5 total, 4 up, 5 in 2023-12-07T10:14:10.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:09 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T10:14:10.016 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:09 smithi159 ceph-mon[134726]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2023-12-07T10:14:10.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:09 smithi119 ceph-mon[97908]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-07T10:14:10.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:09 smithi119 ceph-mon[97908]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-07T10:14:10.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:09 smithi119 ceph-mon[97908]: from='osd.4 [v2:172.21.15.153:6808/4021998299,v1:172.21.15.153:6809/4021998299]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2023-12-07T10:14:10.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:09 smithi119 ceph-mon[97908]: osdmap e32: 5 total, 4 up, 5 in 2023-12-07T10:14:10.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:09 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T10:14:10.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:09 smithi119 ceph-mon[97908]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2023-12-07T10:14:10.250 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-12-07T10:14:10.250 INFO:teuthology.orchestra.run.smithi159.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-12-07T10:14:10.250 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10+0 records in 2023-12-07T10:14:10.250 INFO:teuthology.orchestra.run.smithi159.stderr:10+0 records out 2023-12-07T10:14:10.250 INFO:teuthology.orchestra.run.smithi159.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110376 s, 950 MB/s 2023-12-07T10:14:10.251 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 2023-12-07T10:14:10.251 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping successful for: 2023-12-07T10:14:10.281 INFO:journalctl@ceph.osd.4.smithi153.stdout:Dec 07 10:14:09 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-4[147230]: 2023-12-07T10:14:09.804+0000 7f16ba9e1700 -1 osd.4 0 waiting for initial osdmap 2023-12-07T10:14:10.281 INFO:journalctl@ceph.osd.4.smithi153.stdout:Dec 07 10:14:09 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-4[147230]: 2023-12-07T10:14:09.810+0000 7f16b3fd6700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-07T10:14:10.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:09 smithi153 ceph-mon[133625]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-07T10:14:10.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:09 smithi153 ceph-mon[133625]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-07T10:14:10.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:09 smithi153 ceph-mon[133625]: from='osd.4 [v2:172.21.15.153:6808/4021998299,v1:172.21.15.153:6809/4021998299]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2023-12-07T10:14:10.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:09 smithi153 ceph-mon[133625]: osdmap e32: 5 total, 4 up, 5 in 2023-12-07T10:14:10.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:09 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T10:14:10.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:09 smithi153 ceph-mon[133625]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2023-12-07T10:14:10.687 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi159:vg_nvme/lv_3 2023-12-07T10:14:11.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:10 smithi159 ceph-mon[134726]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2023-12-07T10:14:11.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:10 smithi159 ceph-mon[134726]: osdmap e33: 5 total, 4 up, 5 in 2023-12-07T10:14:11.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:10 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T10:14:11.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:10 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T10:14:11.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:10 smithi119 ceph-mon[97908]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2023-12-07T10:14:11.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:10 smithi119 ceph-mon[97908]: osdmap e33: 5 total, 4 up, 5 in 2023-12-07T10:14:11.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:10 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T10:14:11.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:10 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T10:14:11.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:10 smithi153 ceph-mon[133625]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2023-12-07T10:14:11.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:10 smithi153 ceph-mon[133625]: osdmap e33: 5 total, 4 up, 5 in 2023-12-07T10:14:11.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:10 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T10:14:11.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:10 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T10:14:11.321 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.c/config 2023-12-07T10:14:12.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:11 smithi119 ceph-mon[97908]: purged_snaps scrub starts 2023-12-07T10:14:12.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:11 smithi119 ceph-mon[97908]: purged_snaps scrub ok 2023-12-07T10:14:12.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:11 smithi119 ceph-mon[97908]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-07T10:14:12.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:11 smithi119 ceph-mon[97908]: osd.4 [v2:172.21.15.153:6808/4021998299,v1:172.21.15.153:6809/4021998299] boot 2023-12-07T10:14:12.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:11 smithi119 ceph-mon[97908]: osdmap e34: 5 total, 5 up, 5 in 2023-12-07T10:14:12.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:11 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T10:14:12.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:11 smithi119 ceph-mon[97908]: Detected new or changed devices on smithi153 2023-12-07T10:14:12.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:11 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:12.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:11 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:12.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:11 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:14:12.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:11 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:14:12.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:11 smithi119 ceph-mon[97908]: Adjusting osd_memory_target on smithi153 to 8584M 2023-12-07T10:14:12.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:11 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:12.232 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:11 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:12.233 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:11 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:14:12.233 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:11 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:12.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:11 smithi159 ceph-mon[134726]: purged_snaps scrub starts 2023-12-07T10:14:12.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:11 smithi159 ceph-mon[134726]: purged_snaps scrub ok 2023-12-07T10:14:12.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:11 smithi159 ceph-mon[134726]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-07T10:14:12.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:11 smithi159 ceph-mon[134726]: osd.4 [v2:172.21.15.153:6808/4021998299,v1:172.21.15.153:6809/4021998299] boot 2023-12-07T10:14:12.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:11 smithi159 ceph-mon[134726]: osdmap e34: 5 total, 5 up, 5 in 2023-12-07T10:14:12.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:11 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T10:14:12.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:11 smithi159 ceph-mon[134726]: Detected new or changed devices on smithi153 2023-12-07T10:14:12.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:11 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:12.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:11 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:12.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:11 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:14:12.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:11 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:14:12.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:11 smithi159 ceph-mon[134726]: Adjusting osd_memory_target on smithi153 to 8584M 2023-12-07T10:14:12.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:11 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:12.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:11 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:12.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:11 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:14:12.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:11 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:12.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:11 smithi153 ceph-mon[133625]: purged_snaps scrub starts 2023-12-07T10:14:12.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:11 smithi153 ceph-mon[133625]: purged_snaps scrub ok 2023-12-07T10:14:12.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:11 smithi153 ceph-mon[133625]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-07T10:14:12.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:11 smithi153 ceph-mon[133625]: osd.4 [v2:172.21.15.153:6808/4021998299,v1:172.21.15.153:6809/4021998299] boot 2023-12-07T10:14:12.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:11 smithi153 ceph-mon[133625]: osdmap e34: 5 total, 5 up, 5 in 2023-12-07T10:14:12.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:11 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T10:14:12.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:11 smithi153 ceph-mon[133625]: Detected new or changed devices on smithi153 2023-12-07T10:14:12.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:11 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:12.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:11 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:12.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:11 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:14:12.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:11 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:14:12.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:11 smithi153 ceph-mon[133625]: Adjusting osd_memory_target on smithi153 to 8584M 2023-12-07T10:14:12.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:11 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:12.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:11 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:12.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:11 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:14:12.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:11 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:12.514 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.511+0000 7f23d0414700 1 -- 172.21.15.159:0/2604767905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23c81011c0 msgr2=0x7f23c8108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:14:12.515 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.511+0000 7f23d0414700 1 --2- 172.21.15.159:0/2604767905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23c81011c0 0x7f23c8108a60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f23c000b0a0 tx=0x7f23c000b3b0 comp rx=0 tx=0).stop 2023-12-07T10:14:12.515 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.511+0000 7f23d0414700 1 -- 172.21.15.159:0/2604767905 shutdown_connections 2023-12-07T10:14:12.515 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.511+0000 7f23d0414700 1 --2- 172.21.15.159:0/2604767905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23c81011c0 0x7f23c8108a60 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:14:12.515 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.511+0000 7f23d0414700 1 --2- 172.21.15.159:0/2604767905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23c8101ca0 0x7f23c81020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:14:12.515 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.511+0000 7f23d0414700 1 --2- 172.21.15.159:0/2604767905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23c8100800 0x7f23c8100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:14:12.515 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.511+0000 7f23d0414700 1 -- 172.21.15.159:0/2604767905 >> 172.21.15.159:0/2604767905 conn(0x7f23c80fb2b0 msgr2=0x7f23c80fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:14:12.515 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.511+0000 7f23d0414700 1 -- 172.21.15.159:0/2604767905 shutdown_connections 2023-12-07T10:14:12.515 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.511+0000 7f23d0414700 1 -- 172.21.15.159:0/2604767905 wait complete. 2023-12-07T10:14:12.515 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.512+0000 7f23d0414700 1 Processor -- start 2023-12-07T10:14:12.515 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.512+0000 7f23d0414700 1 -- start start 2023-12-07T10:14:12.517 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.512+0000 7f23d0414700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23c8100800 0x7f23c81966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:14:12.517 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.512+0000 7f23d0414700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23c81011c0 0x7f23c8196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:14:12.517 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.512+0000 7f23d0414700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23c8101ca0 0x7f23c819dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:14:12.517 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.512+0000 7f23d0414700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c819e1f0 con 0x7f23c8101ca0 2023-12-07T10:14:12.517 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.512+0000 7f23d0414700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c819e330 con 0x7f23c81011c0 2023-12-07T10:14:12.517 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.512+0000 7f23d0414700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c819e4c0 con 0x7f23c8100800 2023-12-07T10:14:12.517 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.512+0000 7f23ce1b0700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23c8100800 0x7f23c81966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:14:12.517 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.512+0000 7f23ce1b0700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23c8100800 0x7f23c81966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42570/0 (socket says 172.21.15.159:42570) 2023-12-07T10:14:12.517 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.512+0000 7f23ce1b0700 1 -- 172.21.15.159:0/1646533421 learned_addr learned my addr 172.21.15.159:0/1646533421 (peer_addr_for_me v2:172.21.15.159:0/0) 2023-12-07T10:14:12.517 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.512+0000 7f23cd9af700 1 --2- 172.21.15.159:0/1646533421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23c81011c0 0x7f23c8196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:14:12.518 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.512+0000 7f23ce1b0700 1 -- 172.21.15.159:0/1646533421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23c81011c0 msgr2=0x7f23c8196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:14:12.518 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.512+0000 7f23ce1b0700 1 --2- 172.21.15.159:0/1646533421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23c81011c0 0x7f23c8196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:14:12.518 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.512+0000 7f23ce1b0700 1 -- 172.21.15.159:0/1646533421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23c8101ca0 msgr2=0x7f23c819dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:14:12.518 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.512+0000 7f23ce1b0700 1 --2- 172.21.15.159:0/1646533421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23c8101ca0 0x7f23c819dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:14:12.518 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.512+0000 7f23ce1b0700 1 -- 172.21.15.159:0/1646533421 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c0009d00 con 0x7f23c8100800 2023-12-07T10:14:12.518 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.513+0000 7f23ce1b0700 1 --2- 172.21.15.159:0/1646533421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23c8100800 0x7f23c81966b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f23c400c9a0 tx=0x7f23c400ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:14:12.518 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.513+0000 7f23bf7fe700 1 -- 172.21.15.159:0/1646533421 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c4004220 con 0x7f23c8100800 2023-12-07T10:14:12.518 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.513+0000 7f23d0414700 1 -- 172.21.15.159:0/1646533421 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c819e7a0 con 0x7f23c8100800 2023-12-07T10:14:12.518 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.513+0000 7f23bf7fe700 1 -- 172.21.15.159:0/1646533421 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23c4007d20 con 0x7f23c8100800 2023-12-07T10:14:12.518 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.513+0000 7f23bf7fe700 1 -- 172.21.15.159:0/1646533421 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c4003c00 con 0x7f23c8100800 2023-12-07T10:14:12.518 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.513+0000 7f23d0414700 1 -- 172.21.15.159:0/1646533421 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c819ecf0 con 0x7f23c8100800 2023-12-07T10:14:12.518 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.514+0000 7f23d0414700 1 -- 172.21.15.159:0/1646533421 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c8107a50 con 0x7f23c8100800 2023-12-07T10:14:12.521 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.515+0000 7f23bf7fe700 1 -- 172.21.15.159:0/1646533421 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f23c4007860 con 0x7f23c8100800 2023-12-07T10:14:12.521 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.515+0000 7f23bf7fe700 1 --2- 172.21.15.159:0/1646533421 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f23b40a1b10 0x7f23b40a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:14:12.521 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.515+0000 7f23bf7fe700 1 -- 172.21.15.159:0/1646533421 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f23c40cc110 con 0x7f23c8100800 2023-12-07T10:14:12.521 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.516+0000 7f23cd9af700 1 --2- 172.21.15.159:0/1646533421 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f23b40a1b10 0x7f23b40a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:14:12.521 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.516+0000 7f23cd9af700 1 --2- 172.21.15.159:0/1646533421 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f23b40a1b10 0x7f23b40a3fd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009a10 tx=0x7f23b8009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:14:12.523 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.520+0000 7f23bf7fe700 1 -- 172.21.15.159:0/1646533421 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f23c40965a0 con 0x7f23c8100800 2023-12-07T10:14:12.718 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:12.715+0000 7f23d0414700 1 -- 172.21.15.159:0/1646533421 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f23c8061a50 con 0x7f23b40a1b10 2023-12-07T10:14:13.723 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:13 smithi159 ceph-mon[134726]: osdmap e35: 5 total, 5 up, 5 in 2023-12-07T10:14:13.723 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:13 smithi159 ceph-mon[134726]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:13.723 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:13 smithi159 ceph-mon[134726]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:14:13.723 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:13 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:14:13.723 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:13 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:14:13.723 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:13 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:13.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:13 smithi119 ceph-mon[97908]: osdmap e35: 5 total, 5 up, 5 in 2023-12-07T10:14:13.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:13 smithi119 ceph-mon[97908]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:13.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:13 smithi119 ceph-mon[97908]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:14:13.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:14:13.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:14:13.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:13.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:13 smithi153 ceph-mon[133625]: osdmap e35: 5 total, 5 up, 5 in 2023-12-07T10:14:13.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:13 smithi153 ceph-mon[133625]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:13.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:13 smithi153 ceph-mon[133625]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:14:13.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:13 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:14:13.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:13 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:14:13.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:13 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:15.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:15 smithi119 ceph-mon[97908]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:15.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:15 smithi119 ceph-mon[97908]: from='client.? 172.21.15.159:0/2954668113' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b2fc167-262d-474f-89f3-47bd29424421"}]: dispatch 2023-12-07T10:14:15.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:15 smithi119 ceph-mon[97908]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b2fc167-262d-474f-89f3-47bd29424421"}]: dispatch 2023-12-07T10:14:15.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:15 smithi119 ceph-mon[97908]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8b2fc167-262d-474f-89f3-47bd29424421"}]': finished 2023-12-07T10:14:15.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:15 smithi119 ceph-mon[97908]: osdmap e36: 6 total, 5 up, 6 in 2023-12-07T10:14:15.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:15 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T10:14:16.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:15 smithi159 ceph-mon[134726]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:16.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:15 smithi159 ceph-mon[134726]: from='client.? 172.21.15.159:0/2954668113' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b2fc167-262d-474f-89f3-47bd29424421"}]: dispatch 2023-12-07T10:14:16.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:15 smithi159 ceph-mon[134726]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b2fc167-262d-474f-89f3-47bd29424421"}]: dispatch 2023-12-07T10:14:16.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:15 smithi159 ceph-mon[134726]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8b2fc167-262d-474f-89f3-47bd29424421"}]': finished 2023-12-07T10:14:16.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:15 smithi159 ceph-mon[134726]: osdmap e36: 6 total, 5 up, 6 in 2023-12-07T10:14:16.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:15 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T10:14:16.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:15 smithi153 ceph-mon[133625]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:16.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:15 smithi153 ceph-mon[133625]: from='client.? 172.21.15.159:0/2954668113' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b2fc167-262d-474f-89f3-47bd29424421"}]: dispatch 2023-12-07T10:14:16.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:15 smithi153 ceph-mon[133625]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b2fc167-262d-474f-89f3-47bd29424421"}]: dispatch 2023-12-07T10:14:16.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:15 smithi153 ceph-mon[133625]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8b2fc167-262d-474f-89f3-47bd29424421"}]': finished 2023-12-07T10:14:16.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:15 smithi153 ceph-mon[133625]: osdmap e36: 6 total, 5 up, 6 in 2023-12-07T10:14:16.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:15 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T10:14:16.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:16 smithi119 ceph-mon[97908]: from='client.? 172.21.15.159:0/836562861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:14:17.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:16 smithi159 ceph-mon[134726]: from='client.? 172.21.15.159:0/836562861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:14:17.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:16 smithi153 ceph-mon[133625]: from='client.? 172.21.15.159:0/836562861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:14:17.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:17 smithi119 ceph-mon[97908]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:18.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:17 smithi159 ceph-mon[134726]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:18.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:17 smithi153 ceph-mon[133625]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:19.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:19 smithi119 ceph-mon[97908]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:20.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:19 smithi159 ceph-mon[134726]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:20.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:19 smithi153 ceph-mon[133625]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:21.627 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:21 smithi159 ceph-mon[134726]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:21.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:21 smithi119 ceph-mon[97908]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:21.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:21 smithi153 ceph-mon[133625]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:22.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:22 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-07T10:14:22.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:22 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:22.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:22 smithi159 ceph-mon[134726]: Deploying daemon osd.5 on smithi159 2023-12-07T10:14:22.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:22 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-07T10:14:22.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:22 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:22.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:22 smithi119 ceph-mon[97908]: Deploying daemon osd.5 on smithi159 2023-12-07T10:14:22.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:22 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-07T10:14:22.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:22 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:22.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:22 smithi153 ceph-mon[133625]: Deploying daemon osd.5 on smithi159 2023-12-07T10:14:23.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:23 smithi119 ceph-mon[97908]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:23.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:23 smithi159 ceph-mon[134726]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:23.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:23 smithi153 ceph-mon[133625]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:25.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:25 smithi119 ceph-mon[97908]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:26.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:25 smithi159 ceph-mon[134726]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:26.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:25 smithi153 ceph-mon[133625]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:27.080 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:26 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:27.080 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:26 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:27.080 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:26 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:14:27.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:26 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:27.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:26 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:27.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:26 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:14:27.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:26 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:27.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:26 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:27.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:26 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:14:28.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:27 smithi119 ceph-mon[97908]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:28.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:27 smithi159 ceph-mon[134726]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:28.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:27 smithi153 ceph-mon[133625]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:28.734 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:28.731+0000 7f23bf7fe700 1 -- 172.21.15.159:0/1646533421 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f23c8061a50 con 0x7f23b40a1b10 2023-12-07T10:14:28.735 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 5 on host 'smithi159' 2023-12-07T10:14:28.737 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:28.733+0000 7f23d0414700 1 -- 172.21.15.159:0/1646533421 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f23b40a1b10 msgr2=0x7f23b40a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:14:28.737 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:28.733+0000 7f23d0414700 1 --2- 172.21.15.159:0/1646533421 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f23b40a1b10 0x7f23b40a3fd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009a10 tx=0x7f23b8009190 comp rx=0 tx=0).stop 2023-12-07T10:14:28.737 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:28.733+0000 7f23d0414700 1 -- 172.21.15.159:0/1646533421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23c8100800 msgr2=0x7f23c81966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:14:28.737 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:28.733+0000 7f23d0414700 1 --2- 172.21.15.159:0/1646533421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23c8100800 0x7f23c81966b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f23c400c9a0 tx=0x7f23c400ccb0 comp rx=0 tx=0).stop 2023-12-07T10:14:28.737 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:28.734+0000 7f23d0414700 1 -- 172.21.15.159:0/1646533421 shutdown_connections 2023-12-07T10:14:28.737 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:28.734+0000 7f23d0414700 1 --2- 172.21.15.159:0/1646533421 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f23b40a1b10 0x7f23b40a3fd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:14:28.737 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:28.734+0000 7f23d0414700 1 --2- 172.21.15.159:0/1646533421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23c8101ca0 0x7f23c819dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:14:28.737 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:28.734+0000 7f23d0414700 1 --2- 172.21.15.159:0/1646533421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23c8100800 0x7f23c81966b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:14:28.738 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:28.734+0000 7f23d0414700 1 --2- 172.21.15.159:0/1646533421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23c81011c0 0x7f23c8196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:14:28.738 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:28.734+0000 7f23d0414700 1 -- 172.21.15.159:0/1646533421 >> 172.21.15.159:0/1646533421 conn(0x7f23c80fb2b0 msgr2=0x7f23c8104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:14:28.738 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:28.734+0000 7f23d0414700 1 -- 172.21.15.159:0/1646533421 shutdown_connections 2023-12-07T10:14:28.738 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:14:28.734+0000 7f23d0414700 1 -- 172.21.15.159:0/1646533421 wait complete. 2023-12-07T10:14:29.217 DEBUG:teuthology.orchestra.run.smithi159:osd.5> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.5.service 2023-12-07T10:14:29.220 INFO:tasks.cephadm:Deploying osd.6 on smithi119 with /dev/vg_nvme/lv_2... 2023-12-07T10:14:29.220 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-12-07T10:14:29.249 INFO:journalctl@ceph.osd.5.smithi159.stdout:-- Logs begin at Thu 2023-12-07 09:49:21 UTC. -- 2023-12-07T10:14:29.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:29 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:29.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:29 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:29.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:29 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:29.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:29 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:14:29.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:29 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:29.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:29 smithi119 ceph-mon[97908]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:29.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:29 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:29.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:29 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:29.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:29 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:14:29.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:29 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:29.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:29 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:29.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:29 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:29.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:29 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:14:29.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:29 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:29.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:29 smithi159 ceph-mon[134726]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:29.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:29 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:29.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:29 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:29.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:29 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:14:29.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:29 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:29.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:29 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:29.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:29 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:29.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:29 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:14:29.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:29 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:29.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:29 smithi153 ceph-mon[133625]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:29.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:29 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:29.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:29 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:29.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:29 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:14:30.018 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:14:30.181 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:14:29 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:14:29.903+0000 7f9315521640 -1 osd.5 0 log_to_monitors true 2023-12-07T10:14:30.444 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:30 smithi159 ceph-mon[134726]: from='osd.5 [v2:172.21.15.159:6808/1832540768,v1:172.21.15.159:6809/1832540768]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-07T10:14:30.444 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:30 smithi159 ceph-mon[134726]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-07T10:14:30.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:30 smithi119 ceph-mon[97908]: from='osd.5 [v2:172.21.15.159:6808/1832540768,v1:172.21.15.159:6809/1832540768]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-07T10:14:30.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:30 smithi119 ceph-mon[97908]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-07T10:14:30.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:30 smithi153 ceph-mon[133625]: from='osd.5 [v2:172.21.15.159:6808/1832540768,v1:172.21.15.159:6809/1832540768]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-07T10:14:30.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:30 smithi153 ceph-mon[133625]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-07T10:14:31.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:31 smithi119 ceph-mon[97908]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-07T10:14:31.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:31 smithi119 ceph-mon[97908]: osdmap e37: 6 total, 5 up, 6 in 2023-12-07T10:14:31.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:31 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T10:14:31.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:31 smithi119 ceph-mon[97908]: from='osd.5 [v2:172.21.15.159:6808/1832540768,v1:172.21.15.159:6809/1832540768]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-12-07T10:14:31.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:31 smithi119 ceph-mon[97908]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-12-07T10:14:31.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:31 smithi119 ceph-mon[97908]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:31.495 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:14:31 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:14:31.206+0000 7f930ab98700 -1 osd.5 0 waiting for initial osdmap 2023-12-07T10:14:31.495 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:14:31 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:14:31.213+0000 7f930518f700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-07T10:14:31.495 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:31 smithi159 ceph-mon[134726]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-07T10:14:31.495 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:31 smithi159 ceph-mon[134726]: osdmap e37: 6 total, 5 up, 6 in 2023-12-07T10:14:31.496 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:31 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T10:14:31.496 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:31 smithi159 ceph-mon[134726]: from='osd.5 [v2:172.21.15.159:6808/1832540768,v1:172.21.15.159:6809/1832540768]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-12-07T10:14:31.496 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:31 smithi159 ceph-mon[134726]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-12-07T10:14:31.496 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:31 smithi159 ceph-mon[134726]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:31.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:31 smithi153 ceph-mon[133625]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-07T10:14:31.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:31 smithi153 ceph-mon[133625]: osdmap e37: 6 total, 5 up, 6 in 2023-12-07T10:14:31.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:31 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T10:14:31.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:31 smithi153 ceph-mon[133625]: from='osd.5 [v2:172.21.15.159:6808/1832540768,v1:172.21.15.159:6809/1832540768]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-12-07T10:14:31.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:31 smithi153 ceph-mon[133625]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-12-07T10:14:31.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:31 smithi153 ceph-mon[133625]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:32.018 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-12-07T10:14:32.018 INFO:teuthology.orchestra.run.smithi119.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-12-07T10:14:32.018 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10+0 records in 2023-12-07T10:14:32.018 INFO:teuthology.orchestra.run.smithi119.stderr:10+0 records out 2023-12-07T10:14:32.018 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0564043 s, 186 MB/s 2023-12-07T10:14:32.018 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping successful for: 2023-12-07T10:14:32.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:32 smithi119 ceph-mon[97908]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-12-07T10:14:32.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:32 smithi119 ceph-mon[97908]: osdmap e38: 6 total, 5 up, 6 in 2023-12-07T10:14:32.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:32 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T10:14:32.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:32 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T10:14:32.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:32 smithi119 ceph-mon[97908]: Detected new or changed devices on smithi159 2023-12-07T10:14:32.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:32 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:32.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:32 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:32.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:32 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:14:32.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:32 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:14:32.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:32 smithi119 ceph-mon[97908]: Adjusting osd_memory_target on smithi159 to 8584M 2023-12-07T10:14:32.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:32 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:32.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:32 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:32.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:32 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:14:32.408 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:32 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:32.501 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi119:vg_nvme/lv_2 2023-12-07T10:14:32.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:32 smithi159 ceph-mon[134726]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-12-07T10:14:32.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:32 smithi159 ceph-mon[134726]: osdmap e38: 6 total, 5 up, 6 in 2023-12-07T10:14:32.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:32 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T10:14:32.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:32 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T10:14:32.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:32 smithi159 ceph-mon[134726]: Detected new or changed devices on smithi159 2023-12-07T10:14:32.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:32 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:32.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:32 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:32.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:32 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:14:32.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:32 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:14:32.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:32 smithi159 ceph-mon[134726]: Adjusting osd_memory_target on smithi159 to 8584M 2023-12-07T10:14:32.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:32 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:32.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:32 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:32.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:32 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:14:32.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:32 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:32.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:32 smithi153 ceph-mon[133625]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-12-07T10:14:32.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:32 smithi153 ceph-mon[133625]: osdmap e38: 6 total, 5 up, 6 in 2023-12-07T10:14:32.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:32 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T10:14:32.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:32 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T10:14:32.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:32 smithi153 ceph-mon[133625]: Detected new or changed devices on smithi159 2023-12-07T10:14:32.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:32 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:32.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:32 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:32.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:32 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:14:32.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:32 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:14:32.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:32 smithi153 ceph-mon[133625]: Adjusting osd_memory_target on smithi159 to 8584M 2023-12-07T10:14:32.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:32 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:32.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:32 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:32.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:32 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:14:32.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:32 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:33.280 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:14:33.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:33 smithi119 ceph-mon[97908]: purged_snaps scrub starts 2023-12-07T10:14:33.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:33 smithi119 ceph-mon[97908]: purged_snaps scrub ok 2023-12-07T10:14:33.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:33 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T10:14:33.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:33 smithi119 ceph-mon[97908]: osd.5 [v2:172.21.15.159:6808/1832540768,v1:172.21.15.159:6809/1832540768] boot 2023-12-07T10:14:33.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:33 smithi119 ceph-mon[97908]: osdmap e39: 6 total, 6 up, 6 in 2023-12-07T10:14:33.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:33 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T10:14:33.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:33 smithi119 ceph-mon[97908]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:33.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:33 smithi159 ceph-mon[134726]: purged_snaps scrub starts 2023-12-07T10:14:33.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:33 smithi159 ceph-mon[134726]: purged_snaps scrub ok 2023-12-07T10:14:33.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:33 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T10:14:33.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:33 smithi159 ceph-mon[134726]: osd.5 [v2:172.21.15.159:6808/1832540768,v1:172.21.15.159:6809/1832540768] boot 2023-12-07T10:14:33.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:33 smithi159 ceph-mon[134726]: osdmap e39: 6 total, 6 up, 6 in 2023-12-07T10:14:33.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:33 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T10:14:33.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:33 smithi159 ceph-mon[134726]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:33.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:33 smithi153 ceph-mon[133625]: purged_snaps scrub starts 2023-12-07T10:14:33.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:33 smithi153 ceph-mon[133625]: purged_snaps scrub ok 2023-12-07T10:14:33.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:33 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T10:14:33.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:33 smithi153 ceph-mon[133625]: osd.5 [v2:172.21.15.159:6808/1832540768,v1:172.21.15.159:6809/1832540768] boot 2023-12-07T10:14:33.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:33 smithi153 ceph-mon[133625]: osdmap e39: 6 total, 6 up, 6 in 2023-12-07T10:14:33.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:33 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T10:14:33.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:33 smithi153 ceph-mon[133625]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-07T10:14:34.452 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:34 smithi119 ceph-mon[97908]: osdmap e40: 6 total, 6 up, 6 in 2023-12-07T10:14:34.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:34 smithi159 ceph-mon[134726]: osdmap e40: 6 total, 6 up, 6 in 2023-12-07T10:14:34.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:34 smithi153 ceph-mon[133625]: osdmap e40: 6 total, 6 up, 6 in 2023-12-07T10:14:34.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.702+0000 7f7d0dbe0700 1 -- 172.21.15.119:0/1185417676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d08103200 msgr2=0x7f7d08109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:14:34.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.702+0000 7f7d0dbe0700 1 --2- 172.21.15.119:0/1185417676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d08103200 0x7f7d08109ab0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000b0d0 tx=0x7f7d0000b3e0 comp rx=0 tx=0).stop 2023-12-07T10:14:34.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.702+0000 7f7d0dbe0700 1 -- 172.21.15.119:0/1185417676 shutdown_connections 2023-12-07T10:14:34.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.702+0000 7f7d0dbe0700 1 --2- 172.21.15.119:0/1185417676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d08103200 0x7f7d08109ab0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:14:34.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.702+0000 7f7d0dbe0700 1 --2- 172.21.15.119:0/1185417676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d08101660 0x7f7d08101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:14:34.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.702+0000 7f7d0dbe0700 1 --2- 172.21.15.119:0/1185417676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d08102860 0x7f7d08102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:14:34.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.702+0000 7f7d0dbe0700 1 -- 172.21.15.119:0/1185417676 >> 172.21.15.119:0/1185417676 conn(0x7f7d080fcc20 msgr2=0x7f7d080ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:14:34.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.703+0000 7f7d0dbe0700 1 -- 172.21.15.119:0/1185417676 shutdown_connections 2023-12-07T10:14:34.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.703+0000 7f7d0dbe0700 1 -- 172.21.15.119:0/1185417676 wait complete. 2023-12-07T10:14:34.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.704+0000 7f7d0dbe0700 1 Processor -- start 2023-12-07T10:14:34.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.704+0000 7f7d0dbe0700 1 -- start start 2023-12-07T10:14:34.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.704+0000 7f7d0dbe0700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d08101660 0x7f7d08197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:14:34.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.704+0000 7f7d0dbe0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d08102860 0x7f7d08197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:14:34.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.704+0000 7f7d0dbe0700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d08103200 0x7f7d0819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:14:34.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.704+0000 7f7d0dbe0700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0819f1c0 con 0x7f7d08103200 2023-12-07T10:14:34.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.704+0000 7f7d0dbe0700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0819f300 con 0x7f7d08102860 2023-12-07T10:14:34.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.704+0000 7f7d0dbe0700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0819f490 con 0x7f7d08101660 2023-12-07T10:14:34.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.704+0000 7f7d06ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d08102860 0x7f7d08197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:14:34.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.704+0000 7f7d06ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d08102860 0x7f7d08197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:39796/0 (socket says 172.21.15.119:39796) 2023-12-07T10:14:34.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.704+0000 7f7d06ffd700 1 -- 172.21.15.119:0/3217911615 learned_addr learned my addr 172.21.15.119:0/3217911615 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:14:34.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.704+0000 7f7d077fe700 1 --2- 172.21.15.119:0/3217911615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d08101660 0x7f7d08197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:14:34.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.704+0000 7f7d07fff700 1 --2- 172.21.15.119:0/3217911615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d08103200 0x7f7d0819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:14:34.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.705+0000 7f7d06ffd700 1 -- 172.21.15.119:0/3217911615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d08101660 msgr2=0x7f7d08197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:14:34.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.705+0000 7f7d06ffd700 1 --2- 172.21.15.119:0/3217911615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d08101660 0x7f7d08197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:14:34.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.705+0000 7f7d06ffd700 1 -- 172.21.15.119:0/3217911615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d08103200 msgr2=0x7f7d0819ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:14:34.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.705+0000 7f7d06ffd700 1 --2- 172.21.15.119:0/3217911615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d08103200 0x7f7d0819ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:14:34.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.705+0000 7f7d06ffd700 1 -- 172.21.15.119:0/3217911615 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d00009d00 con 0x7f7d08102860 2023-12-07T10:14:34.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.705+0000 7f7d06ffd700 1 --2- 172.21.15.119:0/3217911615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d08102860 0x7f7d08197bc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00e9c0 tx=0x7f7cfc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:14:34.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.705+0000 7f7d04ff9700 1 -- 172.21.15.119:0/3217911615 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc0097b0 con 0x7f7d08102860 2023-12-07T10:14:34.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.705+0000 7f7d0dbe0700 1 -- 172.21.15.119:0/3217911615 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d0819f770 con 0x7f7d08102860 2023-12-07T10:14:34.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.705+0000 7f7d04ff9700 1 -- 172.21.15.119:0/3217911615 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7cfc00ee60 con 0x7f7d08102860 2023-12-07T10:14:34.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.705+0000 7f7d04ff9700 1 -- 172.21.15.119:0/3217911615 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc0054e0 con 0x7f7d08102860 2023-12-07T10:14:34.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.705+0000 7f7d0dbe0700 1 -- 172.21.15.119:0/3217911615 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d0819fc70 con 0x7f7d08102860 2023-12-07T10:14:34.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.706+0000 7f7d0dbe0700 1 -- 172.21.15.119:0/3217911615 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d08108a20 con 0x7f7d08102860 2023-12-07T10:14:34.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.708+0000 7f7d04ff9700 1 -- 172.21.15.119:0/3217911615 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f7cfc004750 con 0x7f7d08102860 2023-12-07T10:14:34.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.710+0000 7f7d04ff9700 1 --2- 172.21.15.119:0/3217911615 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f7cf00a1b90 0x7f7cf00a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:14:34.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.710+0000 7f7d04ff9700 1 -- 172.21.15.119:0/3217911615 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f7cfc014070 con 0x7f7d08102860 2023-12-07T10:14:34.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.710+0000 7f7d077fe700 1 --2- 172.21.15.119:0/3217911615 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f7cf00a1b90 0x7f7cf00a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:14:34.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.710+0000 7f7d077fe700 1 --2- 172.21.15.119:0/3217911615 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f7cf00a1b90 0x7f7cf00a4050 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7d081026c0 tx=0x7f7cf8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:14:34.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.712+0000 7f7d04ff9700 1 -- 172.21.15.119:0/3217911615 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7cfc096cf0 con 0x7f7d08102860 2023-12-07T10:14:34.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:34.922+0000 7f7d0dbe0700 1 -- 172.21.15.119:0/3217911615 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d0802d3c0 con 0x7f7cf00a1b90 2023-12-07T10:14:35.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:35 smithi159 ceph-mon[134726]: osdmap e41: 6 total, 6 up, 6 in 2023-12-07T10:14:35.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:35 smithi159 ceph-mon[134726]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T10:14:35.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:35 smithi159 ceph-mon[134726]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:14:35.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:35 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:14:35.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:35 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:14:35.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:35 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:35.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:35 smithi153 ceph-mon[133625]: osdmap e41: 6 total, 6 up, 6 in 2023-12-07T10:14:35.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:35 smithi153 ceph-mon[133625]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T10:14:35.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:35 smithi153 ceph-mon[133625]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:14:35.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:35 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:14:35.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:35 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:14:35.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:35 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:35.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:35 smithi119 ceph-mon[97908]: osdmap e41: 6 total, 6 up, 6 in 2023-12-07T10:14:35.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:35 smithi119 ceph-mon[97908]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T10:14:35.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:35 smithi119 ceph-mon[97908]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:14:35.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:35 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:14:35.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:35 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:14:35.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:35 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:36.350 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:36 smithi119 ceph-mon[97908]: osdmap e42: 6 total, 6 up, 6 in 2023-12-07T10:14:36.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:36 smithi159 ceph-mon[134726]: osdmap e42: 6 total, 6 up, 6 in 2023-12-07T10:14:36.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:36 smithi153 ceph-mon[133625]: osdmap e42: 6 total, 6 up, 6 in 2023-12-07T10:14:37.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:37 smithi159 ceph-mon[134726]: osdmap e43: 6 total, 6 up, 6 in 2023-12-07T10:14:37.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:37 smithi159 ceph-mon[134726]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T10:14:37.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:37 smithi153 ceph-mon[133625]: osdmap e43: 6 total, 6 up, 6 in 2023-12-07T10:14:37.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:37 smithi153 ceph-mon[133625]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T10:14:37.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:37 smithi119 ceph-mon[97908]: osdmap e43: 6 total, 6 up, 6 in 2023-12-07T10:14:37.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:37 smithi119 ceph-mon[97908]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T10:14:38.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:38 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3353650646' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9aa6749-46cf-4b2a-8114-cbd034219d30"}]: dispatch 2023-12-07T10:14:38.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:38 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3353650646' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9aa6749-46cf-4b2a-8114-cbd034219d30"}]': finished 2023-12-07T10:14:38.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:38 smithi159 ceph-mon[134726]: osdmap e44: 7 total, 6 up, 7 in 2023-12-07T10:14:38.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:38 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T10:14:38.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:38 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3353650646' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9aa6749-46cf-4b2a-8114-cbd034219d30"}]: dispatch 2023-12-07T10:14:38.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:38 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3353650646' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9aa6749-46cf-4b2a-8114-cbd034219d30"}]': finished 2023-12-07T10:14:38.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:38 smithi153 ceph-mon[133625]: osdmap e44: 7 total, 6 up, 7 in 2023-12-07T10:14:38.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:38 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T10:14:38.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:38 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3353650646' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9aa6749-46cf-4b2a-8114-cbd034219d30"}]: dispatch 2023-12-07T10:14:38.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:38 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3353650646' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9aa6749-46cf-4b2a-8114-cbd034219d30"}]': finished 2023-12-07T10:14:38.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:38 smithi119 ceph-mon[97908]: osdmap e44: 7 total, 6 up, 7 in 2023-12-07T10:14:38.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:38 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T10:14:39.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:39 smithi159 ceph-mon[134726]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2023-12-07T10:14:39.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:39 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/527663010' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:14:39.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:39 smithi153 ceph-mon[133625]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2023-12-07T10:14:39.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:39 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/527663010' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:14:39.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:39 smithi119 ceph-mon[97908]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2023-12-07T10:14:39.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:39 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/527663010' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:14:41.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:41 smithi119 ceph-mon[97908]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-07T10:14:41.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:41 smithi159 ceph-mon[134726]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-07T10:14:41.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:41 smithi153 ceph-mon[133625]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-07T10:14:43.941 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:43 smithi119 ceph-mon[97908]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-12-07T10:14:44.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:43 smithi159 ceph-mon[134726]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-12-07T10:14:44.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:43 smithi153 ceph-mon[133625]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-12-07T10:14:45.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:45 smithi119 ceph-mon[97908]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-12-07T10:14:45.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:45 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-07T10:14:45.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:45 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:46.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:45 smithi159 ceph-mon[134726]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-12-07T10:14:46.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-07T10:14:46.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:46.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:45 smithi153 ceph-mon[133625]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-12-07T10:14:46.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:45 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-07T10:14:46.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:45 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:46.720 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:46 smithi119 ceph-mon[97908]: Deploying daemon osd.6 on smithi119 2023-12-07T10:14:47.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:46 smithi159 ceph-mon[134726]: Deploying daemon osd.6 on smithi119 2023-12-07T10:14:47.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:46 smithi153 ceph-mon[133625]: Deploying daemon osd.6 on smithi119 2023-12-07T10:14:47.966 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:47 smithi119 ceph-mon[97908]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-07T10:14:48.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:47 smithi159 ceph-mon[134726]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-07T10:14:48.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:47 smithi153 ceph-mon[133625]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-07T10:14:49.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:49 smithi119 ceph-mon[97908]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T10:14:50.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:49 smithi159 ceph-mon[134726]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T10:14:50.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:49 smithi153 ceph-mon[133625]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T10:14:51.516 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:51 smithi119 ceph-mon[97908]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T10:14:51.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:51 smithi159 ceph-mon[134726]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T10:14:51.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:51 smithi153 ceph-mon[133625]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T10:14:52.842 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:52 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:52.842 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:52 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:52.842 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:52 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:14:53.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:52 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:53.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:52 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:53.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:52 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:14:53.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:52 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:53.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:52 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:53.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:52 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:14:53.880 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:53 smithi119 ceph-mon[97908]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T10:14:54.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:53 smithi159 ceph-mon[134726]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T10:14:54.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:53 smithi153 ceph-mon[133625]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T10:14:55.140 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 6 on host 'smithi119' 2023-12-07T10:14:55.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:55.137+0000 7f7d04ff9700 1 -- 172.21.15.119:0/3217911615 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7d0802d3c0 con 0x7f7cf00a1b90 2023-12-07T10:14:55.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:55.140+0000 7f7d0dbe0700 1 -- 172.21.15.119:0/3217911615 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f7cf00a1b90 msgr2=0x7f7cf00a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:14:55.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:55.140+0000 7f7d0dbe0700 1 --2- 172.21.15.119:0/3217911615 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f7cf00a1b90 0x7f7cf00a4050 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7d081026c0 tx=0x7f7cf8009260 comp rx=0 tx=0).stop 2023-12-07T10:14:55.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:55.140+0000 7f7d0dbe0700 1 -- 172.21.15.119:0/3217911615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d08102860 msgr2=0x7f7d08197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:14:55.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:55.140+0000 7f7d0dbe0700 1 --2- 172.21.15.119:0/3217911615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d08102860 0x7f7d08197bc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00e9c0 tx=0x7f7cfc00ecd0 comp rx=0 tx=0).stop 2023-12-07T10:14:55.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:55.140+0000 7f7d0dbe0700 1 -- 172.21.15.119:0/3217911615 shutdown_connections 2023-12-07T10:14:55.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:55.140+0000 7f7d0dbe0700 1 --2- 172.21.15.119:0/3217911615 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f7cf00a1b90 0x7f7cf00a4050 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:14:55.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:55.140+0000 7f7d0dbe0700 1 --2- 172.21.15.119:0/3217911615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d08103200 0x7f7d0819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:14:55.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:55.140+0000 7f7d0dbe0700 1 --2- 172.21.15.119:0/3217911615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d08101660 0x7f7d08197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:14:55.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:55.140+0000 7f7d0dbe0700 1 --2- 172.21.15.119:0/3217911615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d08102860 0x7f7d08197bc0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:14:55.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:55.140+0000 7f7d0dbe0700 1 -- 172.21.15.119:0/3217911615 >> 172.21.15.119:0/3217911615 conn(0x7f7d080fcc20 msgr2=0x7f7d08105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:14:55.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:55.140+0000 7f7d0dbe0700 1 -- 172.21.15.119:0/3217911615 shutdown_connections 2023-12-07T10:14:55.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:14:55.140+0000 7f7d0dbe0700 1 -- 172.21.15.119:0/3217911615 wait complete. 2023-12-07T10:14:55.615 DEBUG:teuthology.orchestra.run.smithi119:osd.6> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.6.service 2023-12-07T10:14:55.621 INFO:tasks.cephadm:Deploying osd.7 on smithi153 with /dev/vg_nvme/lv_2... 2023-12-07T10:14:55.621 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-12-07T10:14:55.706 INFO:journalctl@ceph.osd.6.smithi119.stdout:-- Logs begin at Thu 2023-12-07 09:53:13 UTC. -- 2023-12-07T10:14:55.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:55 smithi119 ceph-mon[97908]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T10:14:55.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:55 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:55.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:55 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:55.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:55 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:55.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:55 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:14:55.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:55 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:55.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:55 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:14:55.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:55 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:55.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:55 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:55.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:55 smithi119 ceph-mon[97908]: from='osd.6 [v2:172.21.15.119:6818/1162436840,v1:172.21.15.119:6819/1162436840]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-07T10:14:55.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:55 smithi119 ceph-mon[97908]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-07T10:14:56.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:55 smithi159 ceph-mon[134726]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T10:14:56.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:55 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:56.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:55 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:56.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:55 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:56.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:55 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:14:56.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:55 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:56.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:55 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:14:56.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:55 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:56.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:55 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:56.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:55 smithi159 ceph-mon[134726]: from='osd.6 [v2:172.21.15.119:6818/1162436840,v1:172.21.15.119:6819/1162436840]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-07T10:14:56.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:55 smithi159 ceph-mon[134726]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-07T10:14:56.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:55 smithi153 ceph-mon[133625]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T10:14:56.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:55 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:56.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:55 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:56.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:55 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:56.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:55 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:14:56.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:55 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:56.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:55 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:14:56.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:55 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:56.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:55 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:56.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:55 smithi153 ceph-mon[133625]: from='osd.6 [v2:172.21.15.119:6818/1162436840,v1:172.21.15.119:6819/1162436840]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-07T10:14:56.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:55 smithi153 ceph-mon[133625]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-07T10:14:56.227 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T10:14:57.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:57 smithi119 ceph-mon[97908]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-07T10:14:57.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:57 smithi119 ceph-mon[97908]: osdmap e45: 7 total, 6 up, 7 in 2023-12-07T10:14:57.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:57 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T10:14:57.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:57 smithi119 ceph-mon[97908]: from='osd.6 [v2:172.21.15.119:6818/1162436840,v1:172.21.15.119:6819/1162436840]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2023-12-07T10:14:57.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:57 smithi119 ceph-mon[97908]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2023-12-07T10:14:57.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:57 smithi119 ceph-mon[97908]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T10:14:57.731 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:14:57 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:14:57.457+0000 7f2cec66e700 -1 osd.6 0 waiting for initial osdmap 2023-12-07T10:14:57.731 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:14:57 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:14:57.472+0000 7f2ce6464700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-07T10:14:57.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:57 smithi159 ceph-mon[134726]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-07T10:14:57.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:57 smithi159 ceph-mon[134726]: osdmap e45: 7 total, 6 up, 7 in 2023-12-07T10:14:57.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:57 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T10:14:57.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:57 smithi159 ceph-mon[134726]: from='osd.6 [v2:172.21.15.119:6818/1162436840,v1:172.21.15.119:6819/1162436840]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2023-12-07T10:14:57.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:57 smithi159 ceph-mon[134726]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2023-12-07T10:14:57.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:57 smithi159 ceph-mon[134726]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T10:14:57.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:57 smithi153 ceph-mon[133625]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-07T10:14:57.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:57 smithi153 ceph-mon[133625]: osdmap e45: 7 total, 6 up, 7 in 2023-12-07T10:14:57.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:57 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T10:14:57.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:57 smithi153 ceph-mon[133625]: from='osd.6 [v2:172.21.15.119:6818/1162436840,v1:172.21.15.119:6819/1162436840]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2023-12-07T10:14:57.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:57 smithi153 ceph-mon[133625]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2023-12-07T10:14:57.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:57 smithi153 ceph-mon[133625]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-07T10:14:57.859 INFO:teuthology.orchestra.run.smithi153.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-12-07T10:14:57.859 INFO:teuthology.orchestra.run.smithi153.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-12-07T10:14:57.859 INFO:teuthology.orchestra.run.smithi153.stderr: stderr: 10+0 records in 2023-12-07T10:14:57.859 INFO:teuthology.orchestra.run.smithi153.stderr:10+0 records out 2023-12-07T10:14:57.859 INFO:teuthology.orchestra.run.smithi153.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0138255 s, 758 MB/s 2023-12-07T10:14:57.859 INFO:teuthology.orchestra.run.smithi153.stderr:--> Zapping successful for: 2023-12-07T10:14:58.286 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi153:vg_nvme/lv_2 2023-12-07T10:14:58.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:58 smithi153 ceph-mon[133625]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2023-12-07T10:14:58.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:58 smithi153 ceph-mon[133625]: osdmap e46: 7 total, 6 up, 7 in 2023-12-07T10:14:58.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:58 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T10:14:58.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:58 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T10:14:58.714 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:58 smithi119 ceph-mon[97908]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2023-12-07T10:14:58.714 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:58 smithi119 ceph-mon[97908]: osdmap e46: 7 total, 6 up, 7 in 2023-12-07T10:14:58.714 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:58 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T10:14:58.714 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:58 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T10:14:58.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:58 smithi159 ceph-mon[134726]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2023-12-07T10:14:58.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:58 smithi159 ceph-mon[134726]: osdmap e46: 7 total, 6 up, 7 in 2023-12-07T10:14:58.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:58 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T10:14:58.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:58 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T10:14:58.881 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T10:14:59.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:59 smithi119 ceph-mon[97908]: purged_snaps scrub starts 2023-12-07T10:14:59.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:59 smithi119 ceph-mon[97908]: purged_snaps scrub ok 2023-12-07T10:14:59.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:59 smithi119 ceph-mon[97908]: osd.6 [v2:172.21.15.119:6818/1162436840,v1:172.21.15.119:6819/1162436840] boot 2023-12-07T10:14:59.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:59 smithi119 ceph-mon[97908]: osdmap e47: 7 total, 7 up, 7 in 2023-12-07T10:14:59.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:59 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T10:14:59.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:59 smithi119 ceph-mon[97908]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:14:59.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:59 smithi119 ceph-mon[97908]: Detected new or changed devices on smithi119 2023-12-07T10:14:59.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:59 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:59.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:59 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:59.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:59 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:14:59.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:59 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:14:59.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:59 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:14:59.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:59 smithi119 ceph-mon[97908]: Adjusting osd_memory_target on smithi119 to 5757M 2023-12-07T10:14:59.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:59 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:59.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:59 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:59.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:59 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:14:59.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:14:59 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:59.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:59 smithi159 ceph-mon[134726]: purged_snaps scrub starts 2023-12-07T10:14:59.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:59 smithi159 ceph-mon[134726]: purged_snaps scrub ok 2023-12-07T10:14:59.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:59 smithi159 ceph-mon[134726]: osd.6 [v2:172.21.15.119:6818/1162436840,v1:172.21.15.119:6819/1162436840] boot 2023-12-07T10:14:59.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:59 smithi159 ceph-mon[134726]: osdmap e47: 7 total, 7 up, 7 in 2023-12-07T10:14:59.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:59 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T10:14:59.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:59 smithi159 ceph-mon[134726]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:14:59.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:59 smithi159 ceph-mon[134726]: Detected new or changed devices on smithi119 2023-12-07T10:14:59.765 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:59 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:59.765 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:59 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:59.765 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:59 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:14:59.765 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:59 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:14:59.765 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:59 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:14:59.765 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:59 smithi159 ceph-mon[134726]: Adjusting osd_memory_target on smithi119 to 5757M 2023-12-07T10:14:59.765 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:59 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:59.765 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:59 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:59.765 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:59 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:14:59.765 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:14:59 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:59.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:59 smithi153 ceph-mon[133625]: purged_snaps scrub starts 2023-12-07T10:14:59.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:59 smithi153 ceph-mon[133625]: purged_snaps scrub ok 2023-12-07T10:14:59.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:59 smithi153 ceph-mon[133625]: osd.6 [v2:172.21.15.119:6818/1162436840,v1:172.21.15.119:6819/1162436840] boot 2023-12-07T10:14:59.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:59 smithi153 ceph-mon[133625]: osdmap e47: 7 total, 7 up, 7 in 2023-12-07T10:14:59.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:59 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T10:14:59.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:59 smithi153 ceph-mon[133625]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:14:59.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:59 smithi153 ceph-mon[133625]: Detected new or changed devices on smithi119 2023-12-07T10:14:59.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:59 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:59.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:59 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:59.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:59 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:14:59.782 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:59 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:14:59.782 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:59 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:14:59.782 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:59 smithi153 ceph-mon[133625]: Adjusting osd_memory_target on smithi119 to 5757M 2023-12-07T10:14:59.782 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:59 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:14:59.782 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:59 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:14:59.782 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:59 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:14:59.782 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:14:59 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:00.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.076+0000 7f35dcecc700 1 -- 172.21.15.153:0/3945510935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35d81007a0 msgr2=0x7f35d8100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:15:00.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.076+0000 7f35dcecc700 1 --2- 172.21.15.153:0/3945510935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35d81007a0 0x7f35d8100c20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009990 tx=0x7f35c8009ca0 comp rx=0 tx=0).stop 2023-12-07T10:15:00.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.077+0000 7f35dcecc700 1 -- 172.21.15.153:0/3945510935 shutdown_connections 2023-12-07T10:15:00.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.077+0000 7f35dcecc700 1 --2- 172.21.15.153:0/3945510935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35d8101160 0x7f35d8108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:15:00.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.077+0000 7f35dcecc700 1 --2- 172.21.15.153:0/3945510935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35d8101bd0 0x7f35d8101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:15:00.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.077+0000 7f35dcecc700 1 --2- 172.21.15.153:0/3945510935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35d81007a0 0x7f35d8100c20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:15:00.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.077+0000 7f35dcecc700 1 -- 172.21.15.153:0/3945510935 >> 172.21.15.153:0/3945510935 conn(0x7f35d80fb270 msgr2=0x7f35d80fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:15:00.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.077+0000 7f35dcecc700 1 -- 172.21.15.153:0/3945510935 shutdown_connections 2023-12-07T10:15:00.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.077+0000 7f35dcecc700 1 -- 172.21.15.153:0/3945510935 wait complete. 2023-12-07T10:15:00.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.077+0000 7f35dcecc700 1 Processor -- start 2023-12-07T10:15:00.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.077+0000 7f35dcecc700 1 -- start start 2023-12-07T10:15:00.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.078+0000 7f35dcecc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35d81007a0 0x7f35d8196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:15:00.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.078+0000 7f35dcecc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35d8101160 0x7f35d8196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:15:00.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.078+0000 7f35dcecc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35d8101bd0 0x7f35d819d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:15:00.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.078+0000 7f35dcecc700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d819df80 con 0x7f35d81007a0 2023-12-07T10:15:00.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.078+0000 7f35dcecc700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d80ff690 con 0x7f35d8101160 2023-12-07T10:15:00.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.078+0000 7f35dcecc700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d80ff850 con 0x7f35d8101bd0 2023-12-07T10:15:00.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.078+0000 7f35d5d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35d8101160 0x7f35d8196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:15:00.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.078+0000 7f35d5d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35d8101160 0x7f35d8196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55828/0 (socket says 172.21.15.153:55828) 2023-12-07T10:15:00.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.078+0000 7f35d5d9b700 1 -- 172.21.15.153:0/2571478392 learned_addr learned my addr 172.21.15.153:0/2571478392 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-12-07T10:15:00.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.078+0000 7f35d659c700 1 --2- 172.21.15.153:0/2571478392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35d81007a0 0x7f35d8196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:15:00.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.078+0000 7f35d5d9b700 1 -- 172.21.15.153:0/2571478392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35d8101bd0 msgr2=0x7f35d819d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:15:00.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.078+0000 7f35d5d9b700 1 --2- 172.21.15.153:0/2571478392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35d8101bd0 0x7f35d819d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:15:00.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.078+0000 7f35d5d9b700 1 -- 172.21.15.153:0/2571478392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35d81007a0 msgr2=0x7f35d8196440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:15:00.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.078+0000 7f35d5d9b700 1 --2- 172.21.15.153:0/2571478392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35d81007a0 0x7f35d8196440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:15:00.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.078+0000 7f35d5d9b700 1 -- 172.21.15.153:0/2571478392 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c80095f0 con 0x7f35d8101160 2023-12-07T10:15:00.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.078+0000 7f35d5d9b700 1 --2- 172.21.15.153:0/2571478392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35d8101160 0x7f35d8196980 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f35c8000c00 tx=0x7f35c8005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:15:00.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.078+0000 7f35cf7fe700 1 -- 172.21.15.153:0/2571478392 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c801d070 con 0x7f35d8101160 2023-12-07T10:15:00.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.078+0000 7f35cf7fe700 1 -- 172.21.15.153:0/2571478392 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35c8004ca0 con 0x7f35d8101160 2023-12-07T10:15:00.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.078+0000 7f35cf7fe700 1 -- 172.21.15.153:0/2571478392 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c80055f0 con 0x7f35d8101160 2023-12-07T10:15:00.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.078+0000 7f35dcecc700 1 -- 172.21.15.153:0/2571478392 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d80ffad0 con 0x7f35d8101160 2023-12-07T10:15:00.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.078+0000 7f35dcecc700 1 -- 172.21.15.153:0/2571478392 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d80fff10 con 0x7f35d8101160 2023-12-07T10:15:00.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.079+0000 7f35dcecc700 1 -- 172.21.15.153:0/2571478392 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d8107a10 con 0x7f35d8101160 2023-12-07T10:15:00.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.080+0000 7f35cf7fe700 1 -- 172.21.15.153:0/2571478392 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f35c800fc40 con 0x7f35d8101160 2023-12-07T10:15:00.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.081+0000 7f35cf7fe700 1 --2- 172.21.15.153:0/2571478392 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f35c40a1b90 0x7f35c40a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:15:00.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.081+0000 7f35cf7fe700 1 -- 172.21.15.153:0/2571478392 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f35c80cdd30 con 0x7f35d8101160 2023-12-07T10:15:00.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.081+0000 7f35d659c700 1 --2- 172.21.15.153:0/2571478392 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f35c40a1b90 0x7f35c40a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:15:00.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.082+0000 7f35d659c700 1 --2- 172.21.15.153:0/2571478392 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f35c40a1b90 0x7f35c40a4050 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009bf0 tx=0x7f35c0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:15:00.086 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.084+0000 7f35cf7fe700 1 -- 172.21.15.153:0/2571478392 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f35c8097ef0 con 0x7f35d8101160 2023-12-07T10:15:00.282 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:00.280+0000 7f35dcecc700 1 -- 172.21.15.153:0/2571478392 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f35d8100190 con 0x7f35c40a1b90 2023-12-07T10:15:00.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:00 smithi119 ceph-mon[97908]: osdmap e48: 7 total, 7 up, 7 in 2023-12-07T10:15:00.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:00 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:15:00.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:00 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:15:00.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:00 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:00.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:00 smithi159 ceph-mon[134726]: osdmap e48: 7 total, 7 up, 7 in 2023-12-07T10:15:00.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:00 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:15:00.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:00 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:15:00.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:00 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:00.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:00 smithi153 ceph-mon[133625]: osdmap e48: 7 total, 7 up, 7 in 2023-12-07T10:15:00.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:15:00.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:15:00.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:01.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:01 smithi119 ceph-mon[97908]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:15:01.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:01 smithi119 ceph-mon[97908]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:01.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:01 smithi159 ceph-mon[134726]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:15:01.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:01 smithi159 ceph-mon[134726]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:01.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:01 smithi153 ceph-mon[133625]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:15:01.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:01 smithi153 ceph-mon[133625]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:03.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:03 smithi119 ceph-mon[97908]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:03.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:03 smithi119 ceph-mon[97908]: from='client.? 172.21.15.153:0/1946948058' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39216361-e1c4-4fef-86cb-a4707656b547"}]: dispatch 2023-12-07T10:15:03.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:03 smithi119 ceph-mon[97908]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39216361-e1c4-4fef-86cb-a4707656b547"}]: dispatch 2023-12-07T10:15:03.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:03 smithi119 ceph-mon[97908]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "39216361-e1c4-4fef-86cb-a4707656b547"}]': finished 2023-12-07T10:15:03.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:03 smithi119 ceph-mon[97908]: osdmap e49: 8 total, 7 up, 8 in 2023-12-07T10:15:03.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:03 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T10:15:03.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:03 smithi119 ceph-mon[97908]: from='client.? 172.21.15.153:0/2797212457' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:15:04.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:03 smithi159 ceph-mon[134726]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:04.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:03 smithi159 ceph-mon[134726]: from='client.? 172.21.15.153:0/1946948058' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39216361-e1c4-4fef-86cb-a4707656b547"}]: dispatch 2023-12-07T10:15:04.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:03 smithi159 ceph-mon[134726]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39216361-e1c4-4fef-86cb-a4707656b547"}]: dispatch 2023-12-07T10:15:04.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:03 smithi159 ceph-mon[134726]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "39216361-e1c4-4fef-86cb-a4707656b547"}]': finished 2023-12-07T10:15:04.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:03 smithi159 ceph-mon[134726]: osdmap e49: 8 total, 7 up, 8 in 2023-12-07T10:15:04.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:03 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T10:15:04.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:03 smithi159 ceph-mon[134726]: from='client.? 172.21.15.153:0/2797212457' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:15:04.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:03 smithi153 ceph-mon[133625]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:04.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:03 smithi153 ceph-mon[133625]: from='client.? 172.21.15.153:0/1946948058' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39216361-e1c4-4fef-86cb-a4707656b547"}]: dispatch 2023-12-07T10:15:04.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:03 smithi153 ceph-mon[133625]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39216361-e1c4-4fef-86cb-a4707656b547"}]: dispatch 2023-12-07T10:15:04.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:03 smithi153 ceph-mon[133625]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "39216361-e1c4-4fef-86cb-a4707656b547"}]': finished 2023-12-07T10:15:04.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:03 smithi153 ceph-mon[133625]: osdmap e49: 8 total, 7 up, 8 in 2023-12-07T10:15:04.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:03 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T10:15:04.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:03 smithi153 ceph-mon[133625]: from='client.? 172.21.15.153:0/2797212457' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:15:05.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:05 smithi119 ceph-mon[97908]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:06.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:05 smithi159 ceph-mon[134726]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:06.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:05 smithi153 ceph-mon[133625]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:07.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:07 smithi119 ceph-mon[97908]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:08.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:07 smithi159 ceph-mon[134726]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:08.024 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:07 smithi153 ceph-mon[133625]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:09.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:09 smithi119 ceph-mon[97908]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:09.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:09 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-07T10:15:09.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:09 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:10.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:09 smithi159 ceph-mon[134726]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:10.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:09 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-07T10:15:10.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:09 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:10.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:09 smithi153 ceph-mon[133625]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:10.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:09 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-07T10:15:10.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:09 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:10.963 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:10 smithi153 ceph-mon[133625]: Deploying daemon osd.7 on smithi153 2023-12-07T10:15:10.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:10 smithi119 ceph-mon[97908]: Deploying daemon osd.7 on smithi153 2023-12-07T10:15:11.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:10 smithi159 ceph-mon[134726]: Deploying daemon osd.7 on smithi153 2023-12-07T10:15:11.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:11 smithi119 ceph-mon[97908]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:11.984 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:11 smithi153 ceph-mon[133625]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:12.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:11 smithi159 ceph-mon[134726]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:13.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:13 smithi119 ceph-mon[97908]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:14.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:13 smithi159 ceph-mon[134726]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:14.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:13 smithi153 ceph-mon[133625]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:15.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:14 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:15.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:14 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:15.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:14 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:15:15.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:14 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:15.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:14 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:15.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:14 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:15:15.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:14 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:15.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:14 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:15.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:14 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:15:16.017 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:16 smithi153 ceph-mon[133625]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:16.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:16 smithi159 ceph-mon[134726]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:16.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:16 smithi119 ceph-mon[97908]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:16.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:16.943+0000 7f35cf7fe700 1 -- 172.21.15.153:0/2571478392 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f35d8100190 con 0x7f35c40a1b90 2023-12-07T10:15:16.947 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 7 on host 'smithi153' 2023-12-07T10:15:16.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:16.945+0000 7f35dcecc700 1 -- 172.21.15.153:0/2571478392 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f35c40a1b90 msgr2=0x7f35c40a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:15:16.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:16.945+0000 7f35dcecc700 1 --2- 172.21.15.153:0/2571478392 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f35c40a1b90 0x7f35c40a4050 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009bf0 tx=0x7f35c0009260 comp rx=0 tx=0).stop 2023-12-07T10:15:16.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:16.945+0000 7f35dcecc700 1 -- 172.21.15.153:0/2571478392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35d8101160 msgr2=0x7f35d8196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:15:16.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:16.945+0000 7f35dcecc700 1 --2- 172.21.15.153:0/2571478392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35d8101160 0x7f35d8196980 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f35c8000c00 tx=0x7f35c8005ef0 comp rx=0 tx=0).stop 2023-12-07T10:15:16.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:16.946+0000 7f35dcecc700 1 -- 172.21.15.153:0/2571478392 shutdown_connections 2023-12-07T10:15:16.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:16.946+0000 7f35dcecc700 1 --2- 172.21.15.153:0/2571478392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35d8101bd0 0x7f35d819d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:15:16.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:16.946+0000 7f35dcecc700 1 --2- 172.21.15.153:0/2571478392 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f35c40a1b90 0x7f35c40a4050 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:15:16.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:16.946+0000 7f35dcecc700 1 --2- 172.21.15.153:0/2571478392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35d81007a0 0x7f35d8196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:15:16.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:16.946+0000 7f35dcecc700 1 --2- 172.21.15.153:0/2571478392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35d8101160 0x7f35d8196980 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:15:16.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:16.946+0000 7f35dcecc700 1 -- 172.21.15.153:0/2571478392 >> 172.21.15.153:0/2571478392 conn(0x7f35d80fb270 msgr2=0x7f35d8104b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:15:16.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:16.946+0000 7f35dcecc700 1 -- 172.21.15.153:0/2571478392 shutdown_connections 2023-12-07T10:15:16.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:15:16.946+0000 7f35dcecc700 1 -- 172.21.15.153:0/2571478392 wait complete. 2023-12-07T10:15:17.376 DEBUG:teuthology.orchestra.run.smithi153:osd.7> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.7.service 2023-12-07T10:15:17.378 INFO:tasks.cephadm:Deploying osd.8 on smithi159 with /dev/vg_nvme/lv_2... 2023-12-07T10:15:17.378 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-12-07T10:15:17.408 INFO:journalctl@ceph.osd.7.smithi153.stdout:-- Logs begin at Thu 2023-12-07 09:49:13 UTC. -- 2023-12-07T10:15:17.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:17 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:17.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:17 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:17.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:17 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:17.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:17 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:15:17.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:17 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:17.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:17 smithi119 ceph-mon[97908]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:17.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:17 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:17.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:17 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:17.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:17 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:15:17.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:17 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:17.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:17 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:17.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:17 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:17.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:17 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:15:17.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:17 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:17.765 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:17 smithi159 ceph-mon[134726]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:17.765 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:17 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:17.765 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:17 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:17.765 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:17 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:15:17.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:17 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:17.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:17 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:17.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:17 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:17.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:17 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:15:17.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:17 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:17.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:17 smithi153 ceph-mon[133625]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:17.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:17 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:17.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:17 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:17.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:17 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:15:17.984 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.c/config 2023-12-07T10:15:18.154 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:15:18 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:15:18.014+0000 7f6790f1c640 -1 osd.7 0 log_to_monitors true 2023-12-07T10:15:18.696 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:18 smithi153 ceph-mon[133625]: from='osd.7 [v2:172.21.15.153:6816/3254653457,v1:172.21.15.153:6817/3254653457]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-07T10:15:18.696 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:18 smithi153 ceph-mon[133625]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-07T10:15:18.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:18 smithi119 ceph-mon[97908]: from='osd.7 [v2:172.21.15.153:6816/3254653457,v1:172.21.15.153:6817/3254653457]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-07T10:15:18.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:18 smithi119 ceph-mon[97908]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-07T10:15:18.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:18 smithi159 ceph-mon[134726]: from='osd.7 [v2:172.21.15.153:6816/3254653457,v1:172.21.15.153:6817/3254653457]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-07T10:15:18.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:18 smithi159 ceph-mon[134726]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-07T10:15:19.499 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-12-07T10:15:19.500 INFO:teuthology.orchestra.run.smithi159.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-12-07T10:15:19.500 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10+0 records in 2023-12-07T10:15:19.500 INFO:teuthology.orchestra.run.smithi159.stderr:10+0 records out 2023-12-07T10:15:19.500 INFO:teuthology.orchestra.run.smithi159.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124652 s, 841 MB/s 2023-12-07T10:15:19.500 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping successful for: 2023-12-07T10:15:19.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:19 smithi159 ceph-mon[134726]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-07T10:15:19.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:19 smithi159 ceph-mon[134726]: from='osd.7 [v2:172.21.15.153:6816/3254653457,v1:172.21.15.153:6817/3254653457]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2023-12-07T10:15:19.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:19 smithi159 ceph-mon[134726]: osdmap e50: 8 total, 7 up, 8 in 2023-12-07T10:15:19.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:19 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T10:15:19.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:19 smithi159 ceph-mon[134726]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2023-12-07T10:15:19.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:19 smithi159 ceph-mon[134726]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:19.781 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:15:19 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:15:19.486+0000 7f6786593700 -1 osd.7 0 waiting for initial osdmap 2023-12-07T10:15:19.781 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:15:19 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:15:19.501+0000 7f6780389700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-07T10:15:19.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:19 smithi153 ceph-mon[133625]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-07T10:15:19.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:19 smithi153 ceph-mon[133625]: from='osd.7 [v2:172.21.15.153:6816/3254653457,v1:172.21.15.153:6817/3254653457]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2023-12-07T10:15:19.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:19 smithi153 ceph-mon[133625]: osdmap e50: 8 total, 7 up, 8 in 2023-12-07T10:15:19.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:19 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T10:15:19.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:19 smithi153 ceph-mon[133625]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2023-12-07T10:15:19.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:19 smithi153 ceph-mon[133625]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:19.947 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi159:vg_nvme/lv_2 2023-12-07T10:15:19.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:19 smithi119 ceph-mon[97908]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-07T10:15:19.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:19 smithi119 ceph-mon[97908]: from='osd.7 [v2:172.21.15.153:6816/3254653457,v1:172.21.15.153:6817/3254653457]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2023-12-07T10:15:19.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:19 smithi119 ceph-mon[97908]: osdmap e50: 8 total, 7 up, 8 in 2023-12-07T10:15:19.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:19 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T10:15:19.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:19 smithi119 ceph-mon[97908]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2023-12-07T10:15:19.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:19 smithi119 ceph-mon[97908]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-12-07T10:15:20.543 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.c/config 2023-12-07T10:15:20.752 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:20 smithi159 ceph-mon[134726]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2023-12-07T10:15:20.752 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:20 smithi159 ceph-mon[134726]: osdmap e51: 8 total, 7 up, 8 in 2023-12-07T10:15:20.752 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:20 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T10:15:20.752 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:20 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T10:15:20.752 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:20 smithi159 ceph-mon[134726]: Detected new or changed devices on smithi153 2023-12-07T10:15:20.752 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:20 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:20.753 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:20 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:20.753 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:20 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:15:20.753 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:20 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:15:20.753 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:20 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:15:20.753 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:20 smithi159 ceph-mon[134726]: Adjusting osd_memory_target on smithi153 to 5722M 2023-12-07T10:15:20.753 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:20 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:20.753 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:20 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:20.753 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:20 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:15:20.753 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:20 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:20.753 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:20 smithi159 ceph-mon[134726]: osd.7 [v2:172.21.15.153:6816/3254653457,v1:172.21.15.153:6817/3254653457] boot 2023-12-07T10:15:20.753 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:20 smithi159 ceph-mon[134726]: osdmap e52: 8 total, 8 up, 8 in 2023-12-07T10:15:20.753 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:20 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T10:15:20.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:20 smithi153 ceph-mon[133625]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2023-12-07T10:15:20.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:20 smithi153 ceph-mon[133625]: osdmap e51: 8 total, 7 up, 8 in 2023-12-07T10:15:20.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:20 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T10:15:20.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:20 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T10:15:20.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:20 smithi153 ceph-mon[133625]: Detected new or changed devices on smithi153 2023-12-07T10:15:20.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:20 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:20.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:20 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:20.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:20 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:15:20.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:20 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:15:20.782 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:20 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:15:20.782 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:20 smithi153 ceph-mon[133625]: Adjusting osd_memory_target on smithi153 to 5722M 2023-12-07T10:15:20.782 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:20 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:20.782 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:20 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:20.782 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:20 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:15:20.782 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:20 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:20.782 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:20 smithi153 ceph-mon[133625]: osd.7 [v2:172.21.15.153:6816/3254653457,v1:172.21.15.153:6817/3254653457] boot 2023-12-07T10:15:20.782 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:20 smithi153 ceph-mon[133625]: osdmap e52: 8 total, 8 up, 8 in 2023-12-07T10:15:20.782 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:20 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T10:15:20.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:20 smithi119 ceph-mon[97908]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2023-12-07T10:15:20.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:20 smithi119 ceph-mon[97908]: osdmap e51: 8 total, 7 up, 8 in 2023-12-07T10:15:20.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:20 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T10:15:20.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:20 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T10:15:20.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:20 smithi119 ceph-mon[97908]: Detected new or changed devices on smithi153 2023-12-07T10:15:20.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:20 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:20.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:20 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:20.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:20 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:15:20.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:20 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:15:20.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:20 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:15:20.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:20 smithi119 ceph-mon[97908]: Adjusting osd_memory_target on smithi153 to 5722M 2023-12-07T10:15:20.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:20 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:20.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:20 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:20.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:20 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:15:20.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:20 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:20.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:20 smithi119 ceph-mon[97908]: osd.7 [v2:172.21.15.153:6816/3254653457,v1:172.21.15.153:6817/3254653457] boot 2023-12-07T10:15:20.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:20 smithi119 ceph-mon[97908]: osdmap e52: 8 total, 8 up, 8 in 2023-12-07T10:15:20.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:20 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T10:15:21.712 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.707+0000 7fc8802e2700 1 -- 172.21.15.159:0/2327271737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc878101750 msgr2=0x7fc878101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:15:21.712 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.707+0000 7fc8802e2700 1 --2- 172.21.15.159:0/2327271737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc878101750 0x7fc878101bb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc868009990 tx=0x7fc868009ca0 comp rx=0 tx=0).stop 2023-12-07T10:15:21.712 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.707+0000 7fc8802e2700 1 -- 172.21.15.159:0/2327271737 shutdown_connections 2023-12-07T10:15:21.712 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.707+0000 7fc8802e2700 1 --2- 172.21.15.159:0/2327271737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8781020f0 0x7fc8781089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:15:21.713 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.707+0000 7fc8802e2700 1 --2- 172.21.15.159:0/2327271737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc878100550 0x7fc878100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:15:21.713 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.707+0000 7fc8802e2700 1 --2- 172.21.15.159:0/2327271737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc878101750 0x7fc878101bb0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:15:21.713 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.707+0000 7fc8802e2700 1 -- 172.21.15.159:0/2327271737 >> 172.21.15.159:0/2327271737 conn(0x7fc8780fbb10 msgr2=0x7fc8780fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:15:21.713 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.707+0000 7fc8802e2700 1 -- 172.21.15.159:0/2327271737 shutdown_connections 2023-12-07T10:15:21.713 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.707+0000 7fc8802e2700 1 -- 172.21.15.159:0/2327271737 wait complete. 2023-12-07T10:15:21.713 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.708+0000 7fc8802e2700 1 Processor -- start 2023-12-07T10:15:21.713 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.708+0000 7fc8802e2700 1 -- start start 2023-12-07T10:15:21.713 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.708+0000 7fc8802e2700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc878100550 0x7fc878197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:15:21.713 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.708+0000 7fc8802e2700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc878101750 0x7fc878197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:15:21.713 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.708+0000 7fc8802e2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8781020f0 0x7fc87819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:15:21.713 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.708+0000 7fc8802e2700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87819f150 con 0x7fc878101750 2023-12-07T10:15:21.713 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.708+0000 7fc8802e2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87819f290 con 0x7fc8781020f0 2023-12-07T10:15:21.713 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.708+0000 7fc8802e2700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87819f420 con 0x7fc878100550 2023-12-07T10:15:21.714 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.708+0000 7fc87e07e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc878100550 0x7fc878197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:15:21.714 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.708+0000 7fc87e07e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc878100550 0x7fc878197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42686/0 (socket says 172.21.15.159:42686) 2023-12-07T10:15:21.714 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.708+0000 7fc87e07e700 1 -- 172.21.15.159:0/2919764054 learned_addr learned my addr 172.21.15.159:0/2919764054 (peer_addr_for_me v2:172.21.15.159:0/0) 2023-12-07T10:15:21.714 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.708+0000 7fc87e87f700 1 --2- 172.21.15.159:0/2919764054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8781020f0 0x7fc87819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:15:21.714 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.708+0000 7fc87e07e700 1 -- 172.21.15.159:0/2919764054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8781020f0 msgr2=0x7fc87819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:15:21.714 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.708+0000 7fc87e07e700 1 --2- 172.21.15.159:0/2919764054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8781020f0 0x7fc87819eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:15:21.714 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.708+0000 7fc87e07e700 1 -- 172.21.15.159:0/2919764054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc878101750 msgr2=0x7fc878197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:15:21.714 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.708+0000 7fc87e07e700 1 --2- 172.21.15.159:0/2919764054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc878101750 0x7fc878197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:15:21.714 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.708+0000 7fc87e07e700 1 -- 172.21.15.159:0/2919764054 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8680095f0 con 0x7fc878100550 2023-12-07T10:15:21.714 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.708+0000 7fc87e07e700 1 --2- 172.21.15.159:0/2919764054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc878100550 0x7fc878197660 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc87400b530 tx=0x7fc87400b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:15:21.714 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.708+0000 7fc86f7fe700 1 -- 172.21.15.159:0/2919764054 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc874011830 con 0x7fc878100550 2023-12-07T10:15:21.714 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.709+0000 7fc8802e2700 1 -- 172.21.15.159:0/2919764054 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc87819f700 con 0x7fc878100550 2023-12-07T10:15:21.714 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.709+0000 7fc86f7fe700 1 -- 172.21.15.159:0/2919764054 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc874011e50 con 0x7fc878100550 2023-12-07T10:15:21.715 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.709+0000 7fc86f7fe700 1 -- 172.21.15.159:0/2919764054 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87400f2e0 con 0x7fc878100550 2023-12-07T10:15:21.715 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.709+0000 7fc8802e2700 1 -- 172.21.15.159:0/2919764054 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc87819fc00 con 0x7fc878100550 2023-12-07T10:15:21.715 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.710+0000 7fc8802e2700 1 -- 172.21.15.159:0/2919764054 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc87804f2d0 con 0x7fc878100550 2023-12-07T10:15:21.715 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.711+0000 7fc86f7fe700 1 -- 172.21.15.159:0/2919764054 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fc874010580 con 0x7fc878100550 2023-12-07T10:15:21.716 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.711+0000 7fc86f7fe700 1 --2- 172.21.15.159:0/2919764054 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc8640a1b10 0x7fc8640a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:15:21.716 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.712+0000 7fc86f7fe700 1 -- 172.21.15.159:0/2919764054 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5426+0+0 (secure 0 0 0) 0x7fc8740cc540 con 0x7fc878100550 2023-12-07T10:15:21.716 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.712+0000 7fc87d87d700 1 --2- 172.21.15.159:0/2919764054 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc8640a1b10 0x7fc8640a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:15:21.716 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.713+0000 7fc87d87d700 1 --2- 172.21.15.159:0/2919764054 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc8640a1b10 0x7fc8640a3fd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc8680098e0 tx=0x7fc868009510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:15:21.719 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.716+0000 7fc86f7fe700 1 -- 172.21.15.159:0/2919764054 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc8740964a0 con 0x7fc878100550 2023-12-07T10:15:21.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:21 smithi159 ceph-mon[134726]: purged_snaps scrub starts 2023-12-07T10:15:21.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:21 smithi159 ceph-mon[134726]: purged_snaps scrub ok 2023-12-07T10:15:21.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:21 smithi159 ceph-mon[134726]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:21.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:21 smithi159 ceph-mon[134726]: osdmap e53: 8 total, 8 up, 8 in 2023-12-07T10:15:21.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:21 smithi153 ceph-mon[133625]: purged_snaps scrub starts 2023-12-07T10:15:21.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:21 smithi153 ceph-mon[133625]: purged_snaps scrub ok 2023-12-07T10:15:21.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:21 smithi153 ceph-mon[133625]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:21.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:21 smithi153 ceph-mon[133625]: osdmap e53: 8 total, 8 up, 8 in 2023-12-07T10:15:21.919 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:21.916+0000 7fc8802e2700 1 -- 172.21.15.159:0/2919764054 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8780696c0 con 0x7fc8640a1b10 2023-12-07T10:15:21.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:21 smithi119 ceph-mon[97908]: purged_snaps scrub starts 2023-12-07T10:15:21.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:21 smithi119 ceph-mon[97908]: purged_snaps scrub ok 2023-12-07T10:15:21.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:21 smithi119 ceph-mon[97908]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:21.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:21 smithi119 ceph-mon[97908]: osdmap e53: 8 total, 8 up, 8 in 2023-12-07T10:15:22.751 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:22 smithi159 ceph-mon[134726]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:15:22.751 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:22 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:15:22.751 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:22 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:15:22.751 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:22 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:22.751 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:22 smithi159 ceph-mon[134726]: osdmap e54: 8 total, 8 up, 8 in 2023-12-07T10:15:22.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:22 smithi153 ceph-mon[133625]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:15:22.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:22 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:15:22.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:22 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:15:22.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:22 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:22.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:22 smithi153 ceph-mon[133625]: osdmap e54: 8 total, 8 up, 8 in 2023-12-07T10:15:22.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:22 smithi119 ceph-mon[97908]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:15:22.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:22 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:15:22.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:22 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:15:22.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:22 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:22.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:22 smithi119 ceph-mon[97908]: osdmap e54: 8 total, 8 up, 8 in 2023-12-07T10:15:23.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:23 smithi159 ceph-mon[134726]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:23.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:23 smithi153 ceph-mon[133625]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:23.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:23 smithi119 ceph-mon[97908]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:24.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:24 smithi159 ceph-mon[134726]: osdmap e55: 8 total, 8 up, 8 in 2023-12-07T10:15:24.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:24 smithi159 ceph-mon[134726]: from='client.? 172.21.15.159:0/3592337055' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cceb6ebc-7c02-48ea-9d16-b7a016dd449b"}]: dispatch 2023-12-07T10:15:24.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:24 smithi159 ceph-mon[134726]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cceb6ebc-7c02-48ea-9d16-b7a016dd449b"}]: dispatch 2023-12-07T10:15:24.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:24 smithi159 ceph-mon[134726]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cceb6ebc-7c02-48ea-9d16-b7a016dd449b"}]': finished 2023-12-07T10:15:24.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:24 smithi159 ceph-mon[134726]: osdmap e56: 9 total, 8 up, 9 in 2023-12-07T10:15:24.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:24 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-07T10:15:24.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:24 smithi153 ceph-mon[133625]: osdmap e55: 8 total, 8 up, 8 in 2023-12-07T10:15:24.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:24 smithi153 ceph-mon[133625]: from='client.? 172.21.15.159:0/3592337055' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cceb6ebc-7c02-48ea-9d16-b7a016dd449b"}]: dispatch 2023-12-07T10:15:24.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:24 smithi153 ceph-mon[133625]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cceb6ebc-7c02-48ea-9d16-b7a016dd449b"}]: dispatch 2023-12-07T10:15:24.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:24 smithi153 ceph-mon[133625]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cceb6ebc-7c02-48ea-9d16-b7a016dd449b"}]': finished 2023-12-07T10:15:24.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:24 smithi153 ceph-mon[133625]: osdmap e56: 9 total, 8 up, 9 in 2023-12-07T10:15:24.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:24 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-07T10:15:24.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:24 smithi119 ceph-mon[97908]: osdmap e55: 8 total, 8 up, 8 in 2023-12-07T10:15:24.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:24 smithi119 ceph-mon[97908]: from='client.? 172.21.15.159:0/3592337055' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cceb6ebc-7c02-48ea-9d16-b7a016dd449b"}]: dispatch 2023-12-07T10:15:24.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:24 smithi119 ceph-mon[97908]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cceb6ebc-7c02-48ea-9d16-b7a016dd449b"}]: dispatch 2023-12-07T10:15:24.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:24 smithi119 ceph-mon[97908]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cceb6ebc-7c02-48ea-9d16-b7a016dd449b"}]': finished 2023-12-07T10:15:24.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:24 smithi119 ceph-mon[97908]: osdmap e56: 9 total, 8 up, 9 in 2023-12-07T10:15:24.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:24 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-07T10:15:25.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:25 smithi159 ceph-mon[134726]: pgmap v178: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:25.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:25 smithi159 ceph-mon[134726]: from='client.? 172.21.15.159:0/2988099474' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:15:25.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:25 smithi153 ceph-mon[133625]: pgmap v178: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:25.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:25 smithi153 ceph-mon[133625]: from='client.? 172.21.15.159:0/2988099474' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:15:25.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:25 smithi119 ceph-mon[97908]: pgmap v178: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:25.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:25 smithi119 ceph-mon[97908]: from='client.? 172.21.15.159:0/2988099474' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:15:27.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:27 smithi119 ceph-mon[97908]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:28.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:27 smithi159 ceph-mon[134726]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:28.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:27 smithi153 ceph-mon[133625]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:29.799 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:29 smithi159 ceph-mon[134726]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:29.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:29 smithi119 ceph-mon[97908]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:30.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:29 smithi153 ceph-mon[133625]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:31.717 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:31 smithi159 ceph-mon[134726]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:31.717 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:31 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-12-07T10:15:31.717 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:31 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:31.717 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:31 smithi159 ceph-mon[134726]: Deploying daemon osd.8 on smithi159 2023-12-07T10:15:31.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:31 smithi119 ceph-mon[97908]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:31.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:31 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-12-07T10:15:31.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:31 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:31.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:31 smithi119 ceph-mon[97908]: Deploying daemon osd.8 on smithi159 2023-12-07T10:15:31.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:31 smithi153 ceph-mon[133625]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:31.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:31 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-12-07T10:15:31.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:31 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:31.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:31 smithi153 ceph-mon[133625]: Deploying daemon osd.8 on smithi159 2023-12-07T10:15:33.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:33 smithi119 ceph-mon[97908]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:33.998 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:33 smithi159 ceph-mon[134726]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:34.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:33 smithi153 ceph-mon[133625]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:35.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:35 smithi159 ceph-mon[134726]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:35.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:35 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:35.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:35 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:35.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:35 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:15:35.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:35 smithi119 ceph-mon[97908]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:35.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:35 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:35.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:35 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:35.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:35 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:15:36.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:35 smithi153 ceph-mon[133625]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:36.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:35 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:36.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:35 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:36.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:35 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:15:37.979 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:37 smithi159 ceph-mon[134726]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:37.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:37 smithi119 ceph-mon[97908]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:38.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:37 smithi153 ceph-mon[133625]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:38.336 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:38.333+0000 7fc86f7fe700 1 -- 172.21.15.159:0/2919764054 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc8780696c0 con 0x7fc8640a1b10 2023-12-07T10:15:38.336 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 8 on host 'smithi159' 2023-12-07T10:15:38.338 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:38.335+0000 7fc8802e2700 1 -- 172.21.15.159:0/2919764054 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc8640a1b10 msgr2=0x7fc8640a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:15:38.338 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:38.335+0000 7fc8802e2700 1 --2- 172.21.15.159:0/2919764054 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc8640a1b10 0x7fc8640a3fd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc8680098e0 tx=0x7fc868009510 comp rx=0 tx=0).stop 2023-12-07T10:15:38.338 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:38.335+0000 7fc8802e2700 1 -- 172.21.15.159:0/2919764054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc878100550 msgr2=0x7fc878197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:15:38.338 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:38.335+0000 7fc8802e2700 1 --2- 172.21.15.159:0/2919764054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc878100550 0x7fc878197660 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc87400b530 tx=0x7fc87400b8f0 comp rx=0 tx=0).stop 2023-12-07T10:15:38.338 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:38.335+0000 7fc8802e2700 1 -- 172.21.15.159:0/2919764054 shutdown_connections 2023-12-07T10:15:38.338 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:38.335+0000 7fc8802e2700 1 --2- 172.21.15.159:0/2919764054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8781020f0 0x7fc87819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:15:38.338 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:38.335+0000 7fc8802e2700 1 --2- 172.21.15.159:0/2919764054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc878100550 0x7fc878197660 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:15:38.339 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:38.335+0000 7fc8802e2700 1 --2- 172.21.15.159:0/2919764054 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc8640a1b10 0x7fc8640a3fd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:15:38.339 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:38.335+0000 7fc8802e2700 1 --2- 172.21.15.159:0/2919764054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc878101750 0x7fc878197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:15:38.339 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:38.335+0000 7fc8802e2700 1 -- 172.21.15.159:0/2919764054 >> 172.21.15.159:0/2919764054 conn(0x7fc8780fbb10 msgr2=0x7fc878068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:15:38.339 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:38.335+0000 7fc8802e2700 1 -- 172.21.15.159:0/2919764054 shutdown_connections 2023-12-07T10:15:38.339 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:15:38.335+0000 7fc8802e2700 1 -- 172.21.15.159:0/2919764054 wait complete. 2023-12-07T10:15:38.748 DEBUG:teuthology.orchestra.run.smithi159:osd.8> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.8.service 2023-12-07T10:15:38.750 INFO:tasks.cephadm:Deploying osd.9 on smithi119 with /dev/vg_nvme/lv_1... 2023-12-07T10:15:38.750 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-12-07T10:15:38.780 INFO:journalctl@ceph.osd.8.smithi159.stdout:-- Logs begin at Thu 2023-12-07 09:49:21 UTC. -- 2023-12-07T10:15:39.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:38 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:39.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:38 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:39.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:38 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:39.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:38 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:15:39.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:38 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:39.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:38 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:39.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:38 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:39.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:38 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:15:39.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:38 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:39.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:38 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:39.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:38 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:39.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:38 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:15:39.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:38 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:39.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:38 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:39.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:38 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:39.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:38 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:15:39.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:38 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:39.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:38 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:39.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:38 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:39.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:38 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:15:39.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:38 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:39.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:38 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:39.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:38 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:39.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:38 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:15:39.514 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:15:39 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:15:39.423+0000 7fdd52116640 -1 osd.8 0 log_to_monitors true 2023-12-07T10:15:39.527 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:15:40.090 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:39 smithi119 ceph-mon[97908]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:40.090 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:39 smithi119 ceph-mon[97908]: from='osd.8 [v2:172.21.15.159:6816/2600946380,v1:172.21.15.159:6817/2600946380]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-07T10:15:40.090 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:39 smithi119 ceph-mon[97908]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-07T10:15:40.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:39 smithi159 ceph-mon[134726]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:40.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:39 smithi159 ceph-mon[134726]: from='osd.8 [v2:172.21.15.159:6816/2600946380,v1:172.21.15.159:6817/2600946380]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-07T10:15:40.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:39 smithi159 ceph-mon[134726]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-07T10:15:40.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:39 smithi153 ceph-mon[133625]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:40.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:39 smithi153 ceph-mon[133625]: from='osd.8 [v2:172.21.15.159:6816/2600946380,v1:172.21.15.159:6817/2600946380]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-07T10:15:40.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:39 smithi153 ceph-mon[133625]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-07T10:15:41.618 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-12-07T10:15:41.618 INFO:teuthology.orchestra.run.smithi119.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-12-07T10:15:41.618 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10+0 records in 2023-12-07T10:15:41.618 INFO:teuthology.orchestra.run.smithi119.stderr:10+0 records out 2023-12-07T10:15:41.618 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0309111 s, 339 MB/s 2023-12-07T10:15:41.618 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping successful for: 2023-12-07T10:15:41.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:41 smithi119 ceph-mon[97908]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-12-07T10:15:41.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:41 smithi119 ceph-mon[97908]: osdmap e57: 9 total, 8 up, 9 in 2023-12-07T10:15:41.654 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:41 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-07T10:15:41.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:41 smithi119 ceph-mon[97908]: from='osd.8 [v2:172.21.15.159:6816/2600946380,v1:172.21.15.159:6817/2600946380]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-12-07T10:15:41.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:41 smithi119 ceph-mon[97908]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-12-07T10:15:41.655 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:41 smithi119 ceph-mon[97908]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:41.764 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:15:41 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:15:41.333+0000 7fdd4778d700 -1 osd.8 0 waiting for initial osdmap 2023-12-07T10:15:41.764 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:15:41 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:15:41.344+0000 7fdd3f57f700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-07T10:15:41.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:41 smithi159 ceph-mon[134726]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-12-07T10:15:41.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:41 smithi159 ceph-mon[134726]: osdmap e57: 9 total, 8 up, 9 in 2023-12-07T10:15:41.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:41 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-07T10:15:41.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:41 smithi159 ceph-mon[134726]: from='osd.8 [v2:172.21.15.159:6816/2600946380,v1:172.21.15.159:6817/2600946380]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-12-07T10:15:41.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:41 smithi159 ceph-mon[134726]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-12-07T10:15:41.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:41 smithi159 ceph-mon[134726]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:41.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:41 smithi153 ceph-mon[133625]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-12-07T10:15:41.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:41 smithi153 ceph-mon[133625]: osdmap e57: 9 total, 8 up, 9 in 2023-12-07T10:15:41.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:41 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-07T10:15:41.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:41 smithi153 ceph-mon[133625]: from='osd.8 [v2:172.21.15.159:6816/2600946380,v1:172.21.15.159:6817/2600946380]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-12-07T10:15:41.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:41 smithi153 ceph-mon[133625]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-12-07T10:15:41.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:41 smithi153 ceph-mon[133625]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-07T10:15:42.119 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi119:vg_nvme/lv_1 2023-12-07T10:15:42.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:42 smithi119 ceph-mon[97908]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-12-07T10:15:42.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:42 smithi119 ceph-mon[97908]: osdmap e58: 9 total, 8 up, 9 in 2023-12-07T10:15:42.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:42 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-07T10:15:42.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:42 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-07T10:15:42.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:42 smithi119 ceph-mon[97908]: Detected new or changed devices on smithi159 2023-12-07T10:15:42.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:42 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:42.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:42 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:42.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:42 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:15:42.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:42 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:15:42.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:42 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:15:42.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:42 smithi119 ceph-mon[97908]: Adjusting osd_memory_target on smithi159 to 5722M 2023-12-07T10:15:42.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:42 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:42.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:42 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:42.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:42 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:15:42.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:42 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:42.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:42 smithi159 ceph-mon[134726]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-12-07T10:15:42.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:42 smithi159 ceph-mon[134726]: osdmap e58: 9 total, 8 up, 9 in 2023-12-07T10:15:42.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:42 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-07T10:15:42.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:42 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-07T10:15:42.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:42 smithi159 ceph-mon[134726]: Detected new or changed devices on smithi159 2023-12-07T10:15:42.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:42 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:42.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:42 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:42.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:42 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:15:42.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:42 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:15:42.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:42 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:15:42.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:42 smithi159 ceph-mon[134726]: Adjusting osd_memory_target on smithi159 to 5722M 2023-12-07T10:15:42.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:42 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:42.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:42 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:42.765 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:42 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:15:42.765 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:42 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:42.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:42 smithi153 ceph-mon[133625]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-12-07T10:15:42.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:42 smithi153 ceph-mon[133625]: osdmap e58: 9 total, 8 up, 9 in 2023-12-07T10:15:42.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:42 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-07T10:15:42.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:42 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-07T10:15:42.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:42 smithi153 ceph-mon[133625]: Detected new or changed devices on smithi159 2023-12-07T10:15:42.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:42 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:42.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:42 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:42.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:42 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:15:42.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:42 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:15:42.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:42 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:15:42.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:42 smithi153 ceph-mon[133625]: Adjusting osd_memory_target on smithi159 to 5722M 2023-12-07T10:15:42.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:42 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:42.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:42 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:42.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:42 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:15:42.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:42 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:15:42.897 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:15:43.448 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:43 smithi119 ceph-mon[97908]: purged_snaps scrub starts 2023-12-07T10:15:43.448 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:43 smithi119 ceph-mon[97908]: purged_snaps scrub ok 2023-12-07T10:15:43.448 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:43 smithi119 ceph-mon[97908]: osd.8 [v2:172.21.15.159:6816/2600946380,v1:172.21.15.159:6817/2600946380] boot 2023-12-07T10:15:43.448 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:43 smithi119 ceph-mon[97908]: osdmap e59: 9 total, 9 up, 9 in 2023-12-07T10:15:43.448 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:43 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-07T10:15:43.448 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:43 smithi119 ceph-mon[97908]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:15:43.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:43 smithi159 ceph-mon[134726]: purged_snaps scrub starts 2023-12-07T10:15:43.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:43 smithi159 ceph-mon[134726]: purged_snaps scrub ok 2023-12-07T10:15:43.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:43 smithi159 ceph-mon[134726]: osd.8 [v2:172.21.15.159:6816/2600946380,v1:172.21.15.159:6817/2600946380] boot 2023-12-07T10:15:43.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:43 smithi159 ceph-mon[134726]: osdmap e59: 9 total, 9 up, 9 in 2023-12-07T10:15:43.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:43 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-07T10:15:43.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:43 smithi159 ceph-mon[134726]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:15:43.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:43 smithi153 ceph-mon[133625]: purged_snaps scrub starts 2023-12-07T10:15:43.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:43 smithi153 ceph-mon[133625]: purged_snaps scrub ok 2023-12-07T10:15:43.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:43 smithi153 ceph-mon[133625]: osd.8 [v2:172.21.15.159:6816/2600946380,v1:172.21.15.159:6817/2600946380] boot 2023-12-07T10:15:43.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:43 smithi153 ceph-mon[133625]: osdmap e59: 9 total, 9 up, 9 in 2023-12-07T10:15:43.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:43 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-07T10:15:43.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:43 smithi153 ceph-mon[133625]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:15:44.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.411+0000 7f7676031700 1 -- 172.21.15.119:0/1489063144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7670103200 msgr2=0x7f7670109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:15:44.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.411+0000 7f7676031700 1 --2- 172.21.15.119:0/1489063144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7670103200 0x7f7670109ab0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7668009fc0 tx=0x7f766800b330 comp rx=0 tx=0).stop 2023-12-07T10:15:44.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.411+0000 7f7676031700 1 -- 172.21.15.119:0/1489063144 shutdown_connections 2023-12-07T10:15:44.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.411+0000 7f7676031700 1 --2- 172.21.15.119:0/1489063144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7670103200 0x7f7670109ab0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:15:44.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.411+0000 7f7676031700 1 --2- 172.21.15.119:0/1489063144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7670101660 0x7f7670101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:15:44.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.411+0000 7f7676031700 1 --2- 172.21.15.119:0/1489063144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7670102860 0x7f7670102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:15:44.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.411+0000 7f7676031700 1 -- 172.21.15.119:0/1489063144 >> 172.21.15.119:0/1489063144 conn(0x7f76700fcc20 msgr2=0x7f76700ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:15:44.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.411+0000 7f7676031700 1 -- 172.21.15.119:0/1489063144 shutdown_connections 2023-12-07T10:15:44.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.411+0000 7f7676031700 1 -- 172.21.15.119:0/1489063144 wait complete. 2023-12-07T10:15:44.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.412+0000 7f7676031700 1 Processor -- start 2023-12-07T10:15:44.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.412+0000 7f7676031700 1 -- start start 2023-12-07T10:15:44.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.412+0000 7f7676031700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7670101660 0x7f7670199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:15:44.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.412+0000 7f7676031700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7670102860 0x7f7670199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:15:44.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.412+0000 7f7676031700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7670103200 0x7f76701a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:15:44.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.412+0000 7f7676031700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76701a1390 con 0x7f7670102860 2023-12-07T10:15:44.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.412+0000 7f7676031700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76701a14d0 con 0x7f7670103200 2023-12-07T10:15:44.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.412+0000 7f7676031700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76701a1660 con 0x7f7670101660 2023-12-07T10:15:44.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.413+0000 7f766f7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7670101660 0x7f7670199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:15:44.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.413+0000 7f766effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7670102860 0x7f7670199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:15:44.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.413+0000 7f766f7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7670101660 0x7f7670199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:48436/0 (socket says 172.21.15.119:48436) 2023-12-07T10:15:44.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.413+0000 7f766f7fe700 1 -- 172.21.15.119:0/3009981602 learned_addr learned my addr 172.21.15.119:0/3009981602 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:15:44.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.413+0000 7f766ffff700 1 --2- 172.21.15.119:0/3009981602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7670103200 0x7f76701a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:15:44.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.413+0000 7f766f7fe700 1 -- 172.21.15.119:0/3009981602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7670103200 msgr2=0x7f76701a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:15:44.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.413+0000 7f766f7fe700 1 --2- 172.21.15.119:0/3009981602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7670103200 0x7f76701a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:15:44.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.413+0000 7f766f7fe700 1 -- 172.21.15.119:0/3009981602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7670102860 msgr2=0x7f7670199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:15:44.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.413+0000 7f766f7fe700 1 --2- 172.21.15.119:0/3009981602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7670102860 0x7f7670199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:15:44.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.413+0000 7f766f7fe700 1 -- 172.21.15.119:0/3009981602 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76600095f0 con 0x7f7670101660 2023-12-07T10:15:44.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.413+0000 7f766f7fe700 1 --2- 172.21.15.119:0/3009981602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7670101660 0x7f7670199850 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f766000ec20 tx=0x7f766000ef30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:15:44.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.414+0000 7f766cff9700 1 -- 172.21.15.119:0/3009981602 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7660009a10 con 0x7f7670101660 2023-12-07T10:15:44.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.414+0000 7f7676031700 1 -- 172.21.15.119:0/3009981602 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7668009c40 con 0x7f7670101660 2023-12-07T10:15:44.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.414+0000 7f766cff9700 1 -- 172.21.15.119:0/3009981602 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f766000cc90 con 0x7f7670101660 2023-12-07T10:15:44.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.414+0000 7f766cff9700 1 -- 172.21.15.119:0/3009981602 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76600187e0 con 0x7f7670101660 2023-12-07T10:15:44.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.414+0000 7f7676031700 1 -- 172.21.15.119:0/3009981602 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76701a1cb0 con 0x7f7670101660 2023-12-07T10:15:44.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.415+0000 7f7676031700 1 -- 172.21.15.119:0/3009981602 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f767004f2d0 con 0x7f7670101660 2023-12-07T10:15:44.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.417+0000 7f766cff9700 1 -- 172.21.15.119:0/3009981602 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f76600109f0 con 0x7f7670101660 2023-12-07T10:15:44.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.418+0000 7f766cff9700 1 --2- 172.21.15.119:0/3009981602 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f76580a1b10 0x7f76580a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:15:44.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.418+0000 7f766cff9700 1 -- 172.21.15.119:0/3009981602 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f7660014070 con 0x7f7670101660 2023-12-07T10:15:44.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.419+0000 7f766effd700 1 --2- 172.21.15.119:0/3009981602 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f76580a1b10 0x7f76580a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:15:44.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.419+0000 7f766effd700 1 --2- 172.21.15.119:0/3009981602 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f76580a1b10 0x7f76580a3fd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7664009c60 tx=0x7f7664009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:15:44.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.421+0000 7f766cff9700 1 -- 172.21.15.119:0/3009981602 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f76600976f0 con 0x7f7670101660 2023-12-07T10:15:44.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:44 smithi119 ceph-mon[97908]: osdmap e60: 9 total, 9 up, 9 in 2023-12-07T10:15:44.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:15:44.630+0000 7f7676031700 1 -- 172.21.15.119:0/3009981602 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7670074f60 con 0x7f76580a1b10 2023-12-07T10:15:44.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:44 smithi159 ceph-mon[134726]: osdmap e60: 9 total, 9 up, 9 in 2023-12-07T10:15:44.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:44 smithi153 ceph-mon[133625]: osdmap e60: 9 total, 9 up, 9 in 2023-12-07T10:15:45.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:45 smithi119 ceph-mon[97908]: osdmap e61: 9 total, 9 up, 9 in 2023-12-07T10:15:45.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:45 smithi119 ceph-mon[97908]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:15:45.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:45 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:15:45.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:45 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:15:45.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:45 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:45.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:45 smithi119 ceph-mon[97908]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:15:45.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:45 smithi159 ceph-mon[134726]: osdmap e61: 9 total, 9 up, 9 in 2023-12-07T10:15:45.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:45 smithi159 ceph-mon[134726]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:15:45.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:15:45.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:15:45.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:45.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:45 smithi159 ceph-mon[134726]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:15:45.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:45 smithi153 ceph-mon[133625]: osdmap e61: 9 total, 9 up, 9 in 2023-12-07T10:15:45.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:45 smithi153 ceph-mon[133625]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:15:45.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:45 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:15:45.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:45 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:15:45.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:45 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:45.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:45 smithi153 ceph-mon[133625]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:15:46.617 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:46 smithi119 ceph-mon[97908]: osdmap e62: 9 total, 9 up, 9 in 2023-12-07T10:15:46.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:46 smithi159 ceph-mon[134726]: osdmap e62: 9 total, 9 up, 9 in 2023-12-07T10:15:46.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:46 smithi153 ceph-mon[133625]: osdmap e62: 9 total, 9 up, 9 in 2023-12-07T10:15:47.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:47 smithi119 ceph-mon[97908]: osdmap e63: 9 total, 9 up, 9 in 2023-12-07T10:15:47.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:47 smithi119 ceph-mon[97908]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-07T10:15:47.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:47 smithi159 ceph-mon[134726]: osdmap e63: 9 total, 9 up, 9 in 2023-12-07T10:15:47.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:47 smithi159 ceph-mon[134726]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-07T10:15:47.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:47 smithi153 ceph-mon[133625]: osdmap e63: 9 total, 9 up, 9 in 2023-12-07T10:15:47.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:47 smithi153 ceph-mon[133625]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-07T10:15:48.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:48 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/940451906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f337a2e4-e065-4a9f-91d9-e3615de2d6f8"}]: dispatch 2023-12-07T10:15:48.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:48 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/940451906' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f337a2e4-e065-4a9f-91d9-e3615de2d6f8"}]': finished 2023-12-07T10:15:48.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:48 smithi119 ceph-mon[97908]: osdmap e64: 10 total, 9 up, 10 in 2023-12-07T10:15:48.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:48 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-07T10:15:48.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:48 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/940451906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f337a2e4-e065-4a9f-91d9-e3615de2d6f8"}]: dispatch 2023-12-07T10:15:48.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:48 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/940451906' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f337a2e4-e065-4a9f-91d9-e3615de2d6f8"}]': finished 2023-12-07T10:15:48.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:48 smithi159 ceph-mon[134726]: osdmap e64: 10 total, 9 up, 10 in 2023-12-07T10:15:48.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:48 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-07T10:15:48.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:48 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/940451906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f337a2e4-e065-4a9f-91d9-e3615de2d6f8"}]: dispatch 2023-12-07T10:15:48.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:48 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/940451906' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f337a2e4-e065-4a9f-91d9-e3615de2d6f8"}]': finished 2023-12-07T10:15:48.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:48 smithi153 ceph-mon[133625]: osdmap e64: 10 total, 9 up, 10 in 2023-12-07T10:15:48.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:48 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-07T10:15:49.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:49 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2877195397' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:15:49.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:49 smithi119 ceph-mon[97908]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 134 KiB/s, 0 objects/s recovering 2023-12-07T10:15:49.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:49 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2877195397' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:15:49.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:49 smithi159 ceph-mon[134726]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 134 KiB/s, 0 objects/s recovering 2023-12-07T10:15:49.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:49 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2877195397' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:15:49.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:49 smithi153 ceph-mon[133625]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 134 KiB/s, 0 objects/s recovering 2023-12-07T10:15:51.718 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:51 smithi119 ceph-mon[97908]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-07T10:15:51.765 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:51 smithi159 ceph-mon[134726]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-07T10:15:51.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:51 smithi153 ceph-mon[133625]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-07T10:15:53.889 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:53 smithi119 ceph-mon[97908]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-12-07T10:15:54.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:53 smithi159 ceph-mon[134726]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-12-07T10:15:54.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:53 smithi153 ceph-mon[133625]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-12-07T10:15:55.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:55 smithi119 ceph-mon[97908]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:15:55.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:55 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-12-07T10:15:55.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:55 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:56.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:55 smithi159 ceph-mon[134726]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:15:56.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:55 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-12-07T10:15:56.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:55 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:56.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:55 smithi153 ceph-mon[133625]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:15:56.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:55 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-12-07T10:15:56.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:55 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:15:56.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:56 smithi119 ceph-mon[97908]: Deploying daemon osd.9 on smithi119 2023-12-07T10:15:57.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:56 smithi159 ceph-mon[134726]: Deploying daemon osd.9 on smithi119 2023-12-07T10:15:57.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:56 smithi153 ceph-mon[133625]: Deploying daemon osd.9 on smithi119 2023-12-07T10:15:57.949 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:57 smithi119 ceph-mon[97908]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:15:58.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:57 smithi159 ceph-mon[134726]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:15:58.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:57 smithi153 ceph-mon[133625]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:15:59.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:15:59 smithi119 ceph-mon[97908]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:16:00.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:15:59 smithi159 ceph-mon[134726]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:16:00.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:15:59 smithi153 ceph-mon[133625]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:16:01.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:01 smithi119 ceph-mon[97908]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:16:01.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:01.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:01.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:01 smithi159 ceph-mon[134726]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:16:01.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:01.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:01.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:01 smithi153 ceph-mon[133625]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:16:01.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:01.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:02.710 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:02 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:16:02.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:02 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:16:02.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:02 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:16:03.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:03 smithi119 ceph-mon[97908]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:16:03.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:03 smithi159 ceph-mon[134726]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:16:03.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:03 smithi153 ceph-mon[133625]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:16:04.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:04.995+0000 7f766cff9700 1 -- 172.21.15.119:0/3009981602 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7670074f60 con 0x7f76580a1b10 2023-12-07T10:16:04.998 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 9 on host 'smithi119' 2023-12-07T10:16:05.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:04.997+0000 7f7676031700 1 -- 172.21.15.119:0/3009981602 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f76580a1b10 msgr2=0x7f76580a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:05.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:04.997+0000 7f7676031700 1 --2- 172.21.15.119:0/3009981602 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f76580a1b10 0x7f76580a3fd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7664009c60 tx=0x7f7664009260 comp rx=0 tx=0).stop 2023-12-07T10:16:05.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:04.997+0000 7f7676031700 1 -- 172.21.15.119:0/3009981602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7670101660 msgr2=0x7f7670199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:05.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:04.997+0000 7f7676031700 1 --2- 172.21.15.119:0/3009981602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7670101660 0x7f7670199850 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f766000ec20 tx=0x7f766000ef30 comp rx=0 tx=0).stop 2023-12-07T10:16:05.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:04.997+0000 7f7676031700 1 -- 172.21.15.119:0/3009981602 shutdown_connections 2023-12-07T10:16:05.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:04.997+0000 7f7676031700 1 --2- 172.21.15.119:0/3009981602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7670103200 0x7f76701a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:05.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:04.997+0000 7f7676031700 1 --2- 172.21.15.119:0/3009981602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7670101660 0x7f7670199850 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:05.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:04.997+0000 7f7676031700 1 --2- 172.21.15.119:0/3009981602 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f76580a1b10 0x7f76580a3fd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:05.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:04.997+0000 7f7676031700 1 --2- 172.21.15.119:0/3009981602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7670102860 0x7f7670199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:05.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:04.998+0000 7f7676031700 1 -- 172.21.15.119:0/3009981602 >> 172.21.15.119:0/3009981602 conn(0x7f76700fcc20 msgr2=0x7f7670073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:16:05.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:04.998+0000 7f7676031700 1 -- 172.21.15.119:0/3009981602 shutdown_connections 2023-12-07T10:16:05.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:04.998+0000 7f7676031700 1 -- 172.21.15.119:0/3009981602 wait complete. 2023-12-07T10:16:05.442 DEBUG:teuthology.orchestra.run.smithi119:osd.9> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.9.service 2023-12-07T10:16:05.447 INFO:tasks.cephadm:Deploying osd.10 on smithi153 with /dev/vg_nvme/lv_1... 2023-12-07T10:16:05.447 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-12-07T10:16:05.518 INFO:journalctl@ceph.osd.9.smithi119.stdout:-- Logs begin at Thu 2023-12-07 09:53:13 UTC. -- 2023-12-07T10:16:06.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:05 smithi159 ceph-mon[134726]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:16:06.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:05 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:06.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:05 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:06.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:05 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:06.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:05 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:16:06.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:05 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:06.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:05 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:06.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:05 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:06.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:05 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:16:06.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:05 smithi159 ceph-mon[134726]: from='osd.9 [v2:172.21.15.119:6826/1940737285,v1:172.21.15.119:6827/1940737285]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-12-07T10:16:06.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:05 smithi153 ceph-mon[133625]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:16:06.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:05 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:06.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:05 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:06.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:05 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:06.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:05 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:16:06.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:05 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:06.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:05 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:06.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:05 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:06.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:05 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:16:06.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:05 smithi153 ceph-mon[133625]: from='osd.9 [v2:172.21.15.119:6826/1940737285,v1:172.21.15.119:6827/1940737285]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-12-07T10:16:06.050 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T10:16:06.106 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:05 smithi119 ceph-mon[97908]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:16:06.106 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:05 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:06.106 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:05 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:06.106 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:05 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:06.106 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:05 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:16:06.106 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:05 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:06.106 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:05 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:06.107 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:05 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:06.107 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:05 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:16:06.107 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:05 smithi119 ceph-mon[97908]: from='osd.9 [v2:172.21.15.119:6826/1940737285,v1:172.21.15.119:6827/1940737285]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-12-07T10:16:07.169 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:06 smithi119 ceph-mon[97908]: from='osd.9 [v2:172.21.15.119:6826/1940737285,v1:172.21.15.119:6827/1940737285]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-12-07T10:16:07.169 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:06 smithi119 ceph-mon[97908]: osdmap e65: 10 total, 9 up, 10 in 2023-12-07T10:16:07.169 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:06 smithi119 ceph-mon[97908]: from='osd.9 [v2:172.21.15.119:6826/1940737285,v1:172.21.15.119:6827/1940737285]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2023-12-07T10:16:07.169 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:06 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-07T10:16:07.170 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:16:07 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:16:07.004+0000 7f6e22f44700 -1 osd.9 0 waiting for initial osdmap 2023-12-07T10:16:07.170 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:16:07 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:16:07.018+0000 7f6e1dd3c700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-07T10:16:07.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:06 smithi159 ceph-mon[134726]: from='osd.9 [v2:172.21.15.119:6826/1940737285,v1:172.21.15.119:6827/1940737285]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-12-07T10:16:07.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:06 smithi159 ceph-mon[134726]: osdmap e65: 10 total, 9 up, 10 in 2023-12-07T10:16:07.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:06 smithi159 ceph-mon[134726]: from='osd.9 [v2:172.21.15.119:6826/1940737285,v1:172.21.15.119:6827/1940737285]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2023-12-07T10:16:07.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:06 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-07T10:16:07.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:06 smithi153 ceph-mon[133625]: from='osd.9 [v2:172.21.15.119:6826/1940737285,v1:172.21.15.119:6827/1940737285]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-12-07T10:16:07.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:06 smithi153 ceph-mon[133625]: osdmap e65: 10 total, 9 up, 10 in 2023-12-07T10:16:07.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:06 smithi153 ceph-mon[133625]: from='osd.9 [v2:172.21.15.119:6826/1940737285,v1:172.21.15.119:6827/1940737285]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2023-12-07T10:16:07.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:06 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-07T10:16:07.597 INFO:teuthology.orchestra.run.smithi153.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-12-07T10:16:07.598 INFO:teuthology.orchestra.run.smithi153.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-12-07T10:16:07.598 INFO:teuthology.orchestra.run.smithi153.stderr: stderr: 10+0 records in 2023-12-07T10:16:07.598 INFO:teuthology.orchestra.run.smithi153.stderr:10+0 records out 2023-12-07T10:16:07.598 INFO:teuthology.orchestra.run.smithi153.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128803 s, 814 MB/s 2023-12-07T10:16:07.598 INFO:teuthology.orchestra.run.smithi153.stderr:--> Zapping successful for: 2023-12-07T10:16:08.005 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi153:vg_nvme/lv_1 2023-12-07T10:16:08.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:08 smithi159 ceph-mon[134726]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:16:08.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:08 smithi159 ceph-mon[134726]: from='osd.9 [v2:172.21.15.119:6826/1940737285,v1:172.21.15.119:6827/1940737285]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2023-12-07T10:16:08.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:08 smithi159 ceph-mon[134726]: osdmap e66: 10 total, 9 up, 10 in 2023-12-07T10:16:08.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:08 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-07T10:16:08.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:08 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-07T10:16:08.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:08 smithi153 ceph-mon[133625]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:16:08.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:08 smithi153 ceph-mon[133625]: from='osd.9 [v2:172.21.15.119:6826/1940737285,v1:172.21.15.119:6827/1940737285]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2023-12-07T10:16:08.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:08 smithi153 ceph-mon[133625]: osdmap e66: 10 total, 9 up, 10 in 2023-12-07T10:16:08.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:08 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-07T10:16:08.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:08 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-07T10:16:08.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:08 smithi119 ceph-mon[97908]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:16:08.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:08 smithi119 ceph-mon[97908]: from='osd.9 [v2:172.21.15.119:6826/1940737285,v1:172.21.15.119:6827/1940737285]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2023-12-07T10:16:08.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:08 smithi119 ceph-mon[97908]: osdmap e66: 10 total, 9 up, 10 in 2023-12-07T10:16:08.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:08 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-07T10:16:08.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:08 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-07T10:16:08.597 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T10:16:09.120 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:09 smithi153 ceph-mon[133625]: purged_snaps scrub starts 2023-12-07T10:16:09.120 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:09 smithi153 ceph-mon[133625]: purged_snaps scrub ok 2023-12-07T10:16:09.120 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:09 smithi153 ceph-mon[133625]: osd.9 [v2:172.21.15.119:6826/1940737285,v1:172.21.15.119:6827/1940737285] boot 2023-12-07T10:16:09.120 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:09 smithi153 ceph-mon[133625]: osdmap e67: 10 total, 10 up, 10 in 2023-12-07T10:16:09.121 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:09 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-07T10:16:09.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:09 smithi119 ceph-mon[97908]: purged_snaps scrub starts 2023-12-07T10:16:09.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:09 smithi119 ceph-mon[97908]: purged_snaps scrub ok 2023-12-07T10:16:09.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:09 smithi119 ceph-mon[97908]: osd.9 [v2:172.21.15.119:6826/1940737285,v1:172.21.15.119:6827/1940737285] boot 2023-12-07T10:16:09.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:09 smithi119 ceph-mon[97908]: osdmap e67: 10 total, 10 up, 10 in 2023-12-07T10:16:09.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:09 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-07T10:16:09.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:09 smithi159 ceph-mon[134726]: purged_snaps scrub starts 2023-12-07T10:16:09.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:09 smithi159 ceph-mon[134726]: purged_snaps scrub ok 2023-12-07T10:16:09.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:09 smithi159 ceph-mon[134726]: osd.9 [v2:172.21.15.119:6826/1940737285,v1:172.21.15.119:6827/1940737285] boot 2023-12-07T10:16:09.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:09 smithi159 ceph-mon[134726]: osdmap e67: 10 total, 10 up, 10 in 2023-12-07T10:16:09.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:09 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-07T10:16:09.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.842+0000 7fd51ac73700 1 -- 172.21.15.153:0/1792452732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd514101750 msgr2=0x7fd514101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:09.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.842+0000 7fd51ac73700 1 --2- 172.21.15.153:0/1792452732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd514101750 0x7fd514101bb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd5000099c0 tx=0x7fd500009cd0 comp rx=0 tx=0).stop 2023-12-07T10:16:09.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.842+0000 7fd51ac73700 1 -- 172.21.15.153:0/1792452732 shutdown_connections 2023-12-07T10:16:09.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.842+0000 7fd51ac73700 1 --2- 172.21.15.153:0/1792452732 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5141020f0 0x7fd5141089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:09.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.842+0000 7fd51ac73700 1 --2- 172.21.15.153:0/1792452732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd514100550 0x7fd514100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:09.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.842+0000 7fd51ac73700 1 --2- 172.21.15.153:0/1792452732 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd514101750 0x7fd514101bb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:09.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.842+0000 7fd51ac73700 1 -- 172.21.15.153:0/1792452732 >> 172.21.15.153:0/1792452732 conn(0x7fd5140fbb10 msgr2=0x7fd5140fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:16:09.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.843+0000 7fd51ac73700 1 -- 172.21.15.153:0/1792452732 shutdown_connections 2023-12-07T10:16:09.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.843+0000 7fd51ac73700 1 -- 172.21.15.153:0/1792452732 wait complete. 2023-12-07T10:16:09.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.843+0000 7fd51ac73700 1 Processor -- start 2023-12-07T10:16:09.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.843+0000 7fd51ac73700 1 -- start start 2023-12-07T10:16:09.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.843+0000 7fd51ac73700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd514100550 0x7fd514196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:16:09.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.843+0000 7fd51ac73700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd514101750 0x7fd5141969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:16:09.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.843+0000 7fd51ac73700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5141020f0 0x7fd51419da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:16:09.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.843+0000 7fd51ac73700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51419dfd0 con 0x7fd514101750 2023-12-07T10:16:09.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.843+0000 7fd51ac73700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51419e110 con 0x7fd514100550 2023-12-07T10:16:09.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.843+0000 7fd51ac73700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51419e2a0 con 0x7fd5141020f0 2023-12-07T10:16:09.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.843+0000 7fd518a0f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd514100550 0x7fd514196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:16:09.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.843+0000 7fd518a0f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd514100550 0x7fd514196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55960/0 (socket says 172.21.15.153:55960) 2023-12-07T10:16:09.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.843+0000 7fd518a0f700 1 -- 172.21.15.153:0/3754118631 learned_addr learned my addr 172.21.15.153:0/3754118631 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-12-07T10:16:09.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.844+0000 7fd513fff700 1 --2- 172.21.15.153:0/3754118631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd514101750 0x7fd5141969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:16:09.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.844+0000 7fd518a0f700 1 -- 172.21.15.153:0/3754118631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5141020f0 msgr2=0x7fd51419da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:09.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.844+0000 7fd518a0f700 1 --2- 172.21.15.153:0/3754118631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5141020f0 0x7fd51419da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:09.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.844+0000 7fd518a0f700 1 -- 172.21.15.153:0/3754118631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd514101750 msgr2=0x7fd5141969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:09.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.844+0000 7fd518a0f700 1 --2- 172.21.15.153:0/3754118631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd514101750 0x7fd5141969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:09.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.844+0000 7fd518a0f700 1 -- 172.21.15.153:0/3754118631 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5000095f0 con 0x7fd514100550 2023-12-07T10:16:09.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.844+0000 7fd518a0f700 1 --2- 172.21.15.153:0/3754118631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd514100550 0x7fd514196490 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd50800d980 tx=0x7fd50800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:16:09.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.844+0000 7fd511ffb700 1 -- 172.21.15.153:0/3754118631 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd508004020 con 0x7fd514100550 2023-12-07T10:16:09.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.844+0000 7fd51ac73700 1 -- 172.21.15.153:0/3754118631 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5140690d0 con 0x7fd514100550 2023-12-07T10:16:09.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.844+0000 7fd511ffb700 1 -- 172.21.15.153:0/3754118631 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd508009730 con 0x7fd514100550 2023-12-07T10:16:09.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.844+0000 7fd511ffb700 1 -- 172.21.15.153:0/3754118631 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd508003b20 con 0x7fd514100550 2023-12-07T10:16:09.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.844+0000 7fd51ac73700 1 -- 172.21.15.153:0/3754118631 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd514069620 con 0x7fd514100550 2023-12-07T10:16:09.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.845+0000 7fd51ac73700 1 -- 172.21.15.153:0/3754118631 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd51404f2d0 con 0x7fd514100550 2023-12-07T10:16:09.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.846+0000 7fd511ffb700 1 -- 172.21.15.153:0/3754118631 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fd5080098a0 con 0x7fd514100550 2023-12-07T10:16:09.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.847+0000 7fd511ffb700 1 --2- 172.21.15.153:0/3754118631 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd5040a1b10 0x7fd5040a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:16:09.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.847+0000 7fd511ffb700 1 -- 172.21.15.153:0/3754118631 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fd5080ccaf0 con 0x7fd514100550 2023-12-07T10:16:09.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.847+0000 7fd513fff700 1 --2- 172.21.15.153:0/3754118631 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd5040a1b10 0x7fd5040a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:16:09.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.847+0000 7fd513fff700 1 --2- 172.21.15.153:0/3754118631 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd5040a1b10 0x7fd5040a3fd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd500009990 tx=0x7fd500005bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:16:09.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:09.851+0000 7fd511ffb700 1 -- 172.21.15.153:0/3754118631 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd508096670 con 0x7fd514100550 2023-12-07T10:16:10.052 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:10.049+0000 7fd51ac73700 1 -- 172.21.15.153:0/3754118631 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd514107550 con 0x7fd5040a1b10 2023-12-07T10:16:10.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:10 smithi159 ceph-mon[134726]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:16:10.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:10 smithi159 ceph-mon[134726]: osdmap e68: 10 total, 10 up, 10 in 2023-12-07T10:16:10.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:10 smithi159 ceph-mon[134726]: Detected new or changed devices on smithi119 2023-12-07T10:16:10.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:10 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:10.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:10 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:10.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:10 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:10.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:10 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:10.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:10 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:10.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:10 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:10.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:10 smithi159 ceph-mon[134726]: Adjusting osd_memory_target on smithi119 to 4317M 2023-12-07T10:16:10.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:10 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:10.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:10 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:10.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:10 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:16:10.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:10 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:10.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:10 smithi153 ceph-mon[133625]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:16:10.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:10 smithi153 ceph-mon[133625]: osdmap e68: 10 total, 10 up, 10 in 2023-12-07T10:16:10.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:10 smithi153 ceph-mon[133625]: Detected new or changed devices on smithi119 2023-12-07T10:16:10.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:10 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:10.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:10 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:10.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:10 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:10.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:10 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:10.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:10 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:10.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:10 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:10.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:10 smithi153 ceph-mon[133625]: Adjusting osd_memory_target on smithi119 to 4317M 2023-12-07T10:16:10.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:10 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:10.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:10 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:10.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:10 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:16:10.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:10 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:10.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:10 smithi119 ceph-mon[97908]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-07T10:16:10.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:10 smithi119 ceph-mon[97908]: osdmap e68: 10 total, 10 up, 10 in 2023-12-07T10:16:10.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:10 smithi119 ceph-mon[97908]: Detected new or changed devices on smithi119 2023-12-07T10:16:10.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:10 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:10.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:10 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:10.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:10 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:10.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:10 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:10.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:10 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:10.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:10 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:10.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:10 smithi119 ceph-mon[97908]: Adjusting osd_memory_target on smithi119 to 4317M 2023-12-07T10:16:10.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:10 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:10.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:10 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:10.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:10 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:16:10.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:10 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:11.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:11 smithi153 ceph-mon[133625]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:16:11.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:11 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:16:11.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:11 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:16:11.187 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:11 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:11.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:11 smithi119 ceph-mon[97908]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:16:11.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:11 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:16:11.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:11 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:16:11.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:11 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:11.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:11 smithi159 ceph-mon[134726]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:16:11.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:11 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:16:11.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:11 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:16:11.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:11 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:12.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:12 smithi153 ceph-mon[133625]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:12.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:12 smithi119 ceph-mon[97908]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:12.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:12 smithi159 ceph-mon[134726]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:13.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:13 smithi153 ceph-mon[133625]: from='client.? 172.21.15.153:0/3842580372' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a3bc347-85f4-4878-9797-d9da96137e56"}]: dispatch 2023-12-07T10:16:13.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:13 smithi153 ceph-mon[133625]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a3bc347-85f4-4878-9797-d9da96137e56"}]: dispatch 2023-12-07T10:16:13.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:13 smithi153 ceph-mon[133625]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a3bc347-85f4-4878-9797-d9da96137e56"}]': finished 2023-12-07T10:16:13.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:13 smithi153 ceph-mon[133625]: osdmap e69: 11 total, 10 up, 11 in 2023-12-07T10:16:13.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:13 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-07T10:16:13.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:13 smithi119 ceph-mon[97908]: from='client.? 172.21.15.153:0/3842580372' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a3bc347-85f4-4878-9797-d9da96137e56"}]: dispatch 2023-12-07T10:16:13.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:13 smithi119 ceph-mon[97908]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a3bc347-85f4-4878-9797-d9da96137e56"}]: dispatch 2023-12-07T10:16:13.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:13 smithi119 ceph-mon[97908]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a3bc347-85f4-4878-9797-d9da96137e56"}]': finished 2023-12-07T10:16:13.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:13 smithi119 ceph-mon[97908]: osdmap e69: 11 total, 10 up, 11 in 2023-12-07T10:16:13.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-07T10:16:13.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:13 smithi159 ceph-mon[134726]: from='client.? 172.21.15.153:0/3842580372' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a3bc347-85f4-4878-9797-d9da96137e56"}]: dispatch 2023-12-07T10:16:13.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:13 smithi159 ceph-mon[134726]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a3bc347-85f4-4878-9797-d9da96137e56"}]: dispatch 2023-12-07T10:16:13.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:13 smithi159 ceph-mon[134726]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a3bc347-85f4-4878-9797-d9da96137e56"}]': finished 2023-12-07T10:16:13.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:13 smithi159 ceph-mon[134726]: osdmap e69: 11 total, 10 up, 11 in 2023-12-07T10:16:13.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:13 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-07T10:16:14.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:14 smithi153 ceph-mon[133625]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:14.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:14 smithi153 ceph-mon[133625]: from='client.? 172.21.15.153:0/630596702' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:16:14.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:14 smithi119 ceph-mon[97908]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:14.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:14 smithi119 ceph-mon[97908]: from='client.? 172.21.15.153:0/630596702' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:16:14.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:14 smithi159 ceph-mon[134726]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:14.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:14 smithi159 ceph-mon[134726]: from='client.? 172.21.15.153:0/630596702' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:16:16.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:16 smithi153 ceph-mon[133625]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:16.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:16 smithi119 ceph-mon[97908]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:16.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:16 smithi159 ceph-mon[134726]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:18.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:18 smithi153 ceph-mon[133625]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:18.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:18 smithi119 ceph-mon[97908]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:18.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:18 smithi159 ceph-mon[134726]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:19.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:19 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-12-07T10:16:19.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:19 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:19.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:19 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-12-07T10:16:19.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:19 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:19.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:19 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-12-07T10:16:19.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:19 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:20.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:20 smithi153 ceph-mon[133625]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:20.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:20 smithi153 ceph-mon[133625]: Deploying daemon osd.10 on smithi153 2023-12-07T10:16:20.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:20 smithi119 ceph-mon[97908]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:20.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:20 smithi119 ceph-mon[97908]: Deploying daemon osd.10 on smithi153 2023-12-07T10:16:20.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:20 smithi159 ceph-mon[134726]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:20.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:20 smithi159 ceph-mon[134726]: Deploying daemon osd.10 on smithi153 2023-12-07T10:16:21.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:21 smithi119 ceph-mon[97908]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:21.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:21 smithi159 ceph-mon[134726]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:21.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:21 smithi153 ceph-mon[133625]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:23.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:23 smithi119 ceph-mon[97908]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:23.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:23 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:23.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:23 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:23.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:23 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:16:24.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:23 smithi159 ceph-mon[134726]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:24.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:23 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:24.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:23 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:24.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:23 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:16:24.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:23 smithi153 ceph-mon[133625]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:24.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:23 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:24.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:23 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:24.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:23 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:16:25.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:25 smithi119 ceph-mon[97908]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:25.984 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:25 smithi153 ceph-mon[133625]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:26.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:25 smithi159 ceph-mon[134726]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:26.214 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:26.211+0000 7fd511ffb700 1 -- 172.21.15.153:0/3754118631 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fd514107550 con 0x7fd5040a1b10 2023-12-07T10:16:26.214 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 10 on host 'smithi153' 2023-12-07T10:16:26.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:26.213+0000 7fd51ac73700 1 -- 172.21.15.153:0/3754118631 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd5040a1b10 msgr2=0x7fd5040a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:26.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:26.213+0000 7fd51ac73700 1 --2- 172.21.15.153:0/3754118631 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd5040a1b10 0x7fd5040a3fd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd500009990 tx=0x7fd500005bc0 comp rx=0 tx=0).stop 2023-12-07T10:16:26.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:26.214+0000 7fd51ac73700 1 -- 172.21.15.153:0/3754118631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd514100550 msgr2=0x7fd514196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:26.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:26.214+0000 7fd51ac73700 1 --2- 172.21.15.153:0/3754118631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd514100550 0x7fd514196490 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd50800d980 tx=0x7fd50800dc90 comp rx=0 tx=0).stop 2023-12-07T10:16:26.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:26.214+0000 7fd51ac73700 1 -- 172.21.15.153:0/3754118631 shutdown_connections 2023-12-07T10:16:26.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:26.214+0000 7fd51ac73700 1 --2- 172.21.15.153:0/3754118631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5141020f0 0x7fd51419da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:26.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:26.214+0000 7fd51ac73700 1 --2- 172.21.15.153:0/3754118631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd514100550 0x7fd514196490 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:26.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:26.214+0000 7fd51ac73700 1 --2- 172.21.15.153:0/3754118631 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd5040a1b10 0x7fd5040a3fd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:26.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:26.214+0000 7fd51ac73700 1 --2- 172.21.15.153:0/3754118631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd514101750 0x7fd5141969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:26.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:26.214+0000 7fd51ac73700 1 -- 172.21.15.153:0/3754118631 >> 172.21.15.153:0/3754118631 conn(0x7fd5140fbb10 msgr2=0x7fd514104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:16:26.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:26.214+0000 7fd51ac73700 1 -- 172.21.15.153:0/3754118631 shutdown_connections 2023-12-07T10:16:26.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-12-07T10:16:26.214+0000 7fd51ac73700 1 -- 172.21.15.153:0/3754118631 wait complete. 2023-12-07T10:16:26.595 DEBUG:teuthology.orchestra.run.smithi153:osd.10> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.10.service 2023-12-07T10:16:26.597 INFO:tasks.cephadm:Deploying osd.11 on smithi159 with /dev/vg_nvme/lv_1... 2023-12-07T10:16:26.597 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-12-07T10:16:26.630 INFO:journalctl@ceph.osd.10.smithi153.stdout:-- Logs begin at Thu 2023-12-07 09:49:13 UTC. -- 2023-12-07T10:16:27.203 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.c/config 2023-12-07T10:16:27.416 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:27 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:27.416 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:27 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:27.416 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:27 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:27.416 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:27 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:16:27.416 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:27 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:27.416 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:27 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:27.416 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:27 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:27.416 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:27 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:16:27.453 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:16:27 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:16:27.386+0000 7fb9028e0640 -1 osd.10 0 log_to_monitors true 2023-12-07T10:16:27.453 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:27 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:27.454 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:27 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:27.454 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:27 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:27.454 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:27 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:16:27.454 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:27 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:27.454 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:27 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:27.454 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:27 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:27.454 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:27 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:16:27.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:27 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:27.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:27 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:27.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:27 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:27.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:27 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:16:27.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:27 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:27.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:27 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:27.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:27 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:27.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:27 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:16:28.232 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:28 smithi159 ceph-mon[134726]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:28.232 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:28 smithi159 ceph-mon[134726]: from='osd.10 [v2:172.21.15.153:6824/663834864,v1:172.21.15.153:6825/663834864]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-07T10:16:28.232 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:28 smithi159 ceph-mon[134726]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-07T10:16:28.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:28 smithi119 ceph-mon[97908]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:28.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:28 smithi119 ceph-mon[97908]: from='osd.10 [v2:172.21.15.153:6824/663834864,v1:172.21.15.153:6825/663834864]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-07T10:16:28.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:28 smithi119 ceph-mon[97908]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-07T10:16:28.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:28 smithi153 ceph-mon[133625]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:28.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:28 smithi153 ceph-mon[133625]: from='osd.10 [v2:172.21.15.153:6824/663834864,v1:172.21.15.153:6825/663834864]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-07T10:16:28.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:28 smithi153 ceph-mon[133625]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-07T10:16:28.746 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-12-07T10:16:28.746 INFO:teuthology.orchestra.run.smithi159.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-12-07T10:16:28.746 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10+0 records in 2023-12-07T10:16:28.746 INFO:teuthology.orchestra.run.smithi159.stderr:10+0 records out 2023-12-07T10:16:28.746 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011115 s, 943 MB/s 2023-12-07T10:16:28.746 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping successful for: 2023-12-07T10:16:29.163 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi159:vg_nvme/lv_1 2023-12-07T10:16:29.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:29 smithi119 ceph-mon[97908]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-12-07T10:16:29.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:29 smithi119 ceph-mon[97908]: osdmap e70: 11 total, 10 up, 11 in 2023-12-07T10:16:29.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:29 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-07T10:16:29.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:29 smithi119 ceph-mon[97908]: from='osd.10 [v2:172.21.15.153:6824/663834864,v1:172.21.15.153:6825/663834864]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2023-12-07T10:16:29.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:29 smithi119 ceph-mon[97908]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2023-12-07T10:16:29.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:29 smithi119 ceph-mon[97908]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:29.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:29 smithi159 ceph-mon[134726]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-12-07T10:16:29.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:29 smithi159 ceph-mon[134726]: osdmap e70: 11 total, 10 up, 11 in 2023-12-07T10:16:29.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:29 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-07T10:16:29.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:29 smithi159 ceph-mon[134726]: from='osd.10 [v2:172.21.15.153:6824/663834864,v1:172.21.15.153:6825/663834864]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2023-12-07T10:16:29.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:29 smithi159 ceph-mon[134726]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2023-12-07T10:16:29.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:29 smithi159 ceph-mon[134726]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:29.530 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:16:29 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:16:29.217+0000 7fb8f7f57700 -1 osd.10 0 waiting for initial osdmap 2023-12-07T10:16:29.530 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:16:29 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:16:29.231+0000 7fb8f154c700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-07T10:16:29.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:29 smithi153 ceph-mon[133625]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-12-07T10:16:29.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:29 smithi153 ceph-mon[133625]: osdmap e70: 11 total, 10 up, 11 in 2023-12-07T10:16:29.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:29 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-07T10:16:29.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:29 smithi153 ceph-mon[133625]: from='osd.10 [v2:172.21.15.153:6824/663834864,v1:172.21.15.153:6825/663834864]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2023-12-07T10:16:29.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:29 smithi153 ceph-mon[133625]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2023-12-07T10:16:29.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:29 smithi153 ceph-mon[133625]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-07T10:16:29.765 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.c/config 2023-12-07T10:16:30.221 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:30 smithi159 ceph-mon[134726]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2023-12-07T10:16:30.221 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:30 smithi159 ceph-mon[134726]: osdmap e71: 11 total, 10 up, 11 in 2023-12-07T10:16:30.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:30 smithi119 ceph-mon[97908]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2023-12-07T10:16:30.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:30 smithi119 ceph-mon[97908]: osdmap e71: 11 total, 10 up, 11 in 2023-12-07T10:16:30.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:30 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-07T10:16:30.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:30 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-07T10:16:30.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:30 smithi119 ceph-mon[97908]: Detected new or changed devices on smithi153 2023-12-07T10:16:30.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:30 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:30.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:30 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:30.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:30 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:30.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:30 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:30.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:30 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:30.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:30 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:30.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:30 smithi119 ceph-mon[97908]: Adjusting osd_memory_target on smithi153 to 4292M 2023-12-07T10:16:30.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:30 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:30.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:30 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:30.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:30 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:16:30.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:30 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:30.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:30 smithi119 ceph-mon[97908]: osd.10 [v2:172.21.15.153:6824/663834864,v1:172.21.15.153:6825/663834864] boot 2023-12-07T10:16:30.481 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:30 smithi119 ceph-mon[97908]: osdmap e72: 11 total, 11 up, 11 in 2023-12-07T10:16:30.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:30 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-07T10:16:30.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:30 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-07T10:16:30.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:30 smithi159 ceph-mon[134726]: Detected new or changed devices on smithi153 2023-12-07T10:16:30.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:30 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:30.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:30 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:30.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:30 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:30.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:30 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:30.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:30 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:30.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:30 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:30.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:30 smithi159 ceph-mon[134726]: Adjusting osd_memory_target on smithi153 to 4292M 2023-12-07T10:16:30.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:30 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:30.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:30 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:30.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:30 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:16:30.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:30 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:30.493 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:30 smithi159 ceph-mon[134726]: osd.10 [v2:172.21.15.153:6824/663834864,v1:172.21.15.153:6825/663834864] boot 2023-12-07T10:16:30.494 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:30 smithi159 ceph-mon[134726]: osdmap e72: 11 total, 11 up, 11 in 2023-12-07T10:16:30.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:30 smithi153 ceph-mon[133625]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2023-12-07T10:16:30.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:30 smithi153 ceph-mon[133625]: osdmap e71: 11 total, 10 up, 11 in 2023-12-07T10:16:30.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:30 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-07T10:16:30.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:30 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-07T10:16:30.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:30 smithi153 ceph-mon[133625]: Detected new or changed devices on smithi153 2023-12-07T10:16:30.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:30 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:30.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:30 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:30.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:30 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:30.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:30 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:30.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:30 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:30.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:30 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:30.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:30 smithi153 ceph-mon[133625]: Adjusting osd_memory_target on smithi153 to 4292M 2023-12-07T10:16:30.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:30 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:30.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:30 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:30.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:30 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:16:30.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:30 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:30.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:30 smithi153 ceph-mon[133625]: osd.10 [v2:172.21.15.153:6824/663834864,v1:172.21.15.153:6825/663834864] boot 2023-12-07T10:16:30.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:30 smithi153 ceph-mon[133625]: osdmap e72: 11 total, 11 up, 11 in 2023-12-07T10:16:30.967 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.964+0000 7f8b78bfd700 1 -- 172.21.15.159:0/3596160142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b74104ab0 msgr2=0x7f8b74106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:30.968 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.964+0000 7f8b78bfd700 1 --2- 172.21.15.159:0/3596160142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b74104ab0 0x7f8b74106ea0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8b64009990 tx=0x7f8b64009ca0 comp rx=0 tx=0).stop 2023-12-07T10:16:30.968 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.965+0000 7f8b78bfd700 1 -- 172.21.15.159:0/3596160142 shutdown_connections 2023-12-07T10:16:30.968 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.965+0000 7f8b78bfd700 1 --2- 172.21.15.159:0/3596160142 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b741073e0 0x7f8b741097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:30.968 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.965+0000 7f8b78bfd700 1 --2- 172.21.15.159:0/3596160142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b74102180 0x7f8b74104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:30.968 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.965+0000 7f8b78bfd700 1 --2- 172.21.15.159:0/3596160142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b74104ab0 0x7f8b74106ea0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:30.968 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.965+0000 7f8b78bfd700 1 -- 172.21.15.159:0/3596160142 >> 172.21.15.159:0/3596160142 conn(0x7f8b740fbb10 msgr2=0x7f8b740fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:16:30.968 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.965+0000 7f8b78bfd700 1 -- 172.21.15.159:0/3596160142 shutdown_connections 2023-12-07T10:16:30.968 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.965+0000 7f8b78bfd700 1 -- 172.21.15.159:0/3596160142 wait complete. 2023-12-07T10:16:30.968 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.965+0000 7f8b78bfd700 1 Processor -- start 2023-12-07T10:16:30.970 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.965+0000 7f8b78bfd700 1 -- start start 2023-12-07T10:16:30.970 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.965+0000 7f8b78bfd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b74102180 0x7f8b74101a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:16:30.970 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.966+0000 7f8b78bfd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b74104ab0 0x7f8b74100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:16:30.970 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.966+0000 7f8b78bfd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b741073e0 0x7f8b74100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:16:30.970 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.966+0000 7f8b78bfd700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b74100e10 con 0x7f8b741073e0 2023-12-07T10:16:30.970 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.966+0000 7f8b78bfd700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b74100f80 con 0x7f8b74104ab0 2023-12-07T10:16:30.970 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.966+0000 7f8b78bfd700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b741a1640 con 0x7f8b74102180 2023-12-07T10:16:30.971 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.966+0000 7f8b7259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b74102180 0x7f8b74101a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:16:30.971 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.966+0000 7f8b7259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b74102180 0x7f8b74101a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42830/0 (socket says 172.21.15.159:42830) 2023-12-07T10:16:30.971 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.966+0000 7f8b7259c700 1 -- 172.21.15.159:0/916474317 learned_addr learned my addr 172.21.15.159:0/916474317 (peer_addr_for_me v2:172.21.15.159:0/0) 2023-12-07T10:16:30.971 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.966+0000 7f8b71d9b700 1 --2- 172.21.15.159:0/916474317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b74104ab0 0x7f8b74100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:16:30.971 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.966+0000 7f8b7259c700 1 -- 172.21.15.159:0/916474317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b74104ab0 msgr2=0x7f8b74100130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:30.971 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.966+0000 7f8b7259c700 1 --2- 172.21.15.159:0/916474317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b74104ab0 0x7f8b74100130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:30.971 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.966+0000 7f8b7259c700 1 -- 172.21.15.159:0/916474317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b741073e0 msgr2=0x7f8b74100670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:30.971 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.966+0000 7f8b7259c700 1 --2- 172.21.15.159:0/916474317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b741073e0 0x7f8b74100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:30.971 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.966+0000 7f8b7259c700 1 -- 172.21.15.159:0/916474317 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b640095f0 con 0x7f8b74102180 2023-12-07T10:16:30.971 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.966+0000 7f8b7259c700 1 --2- 172.21.15.159:0/916474317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b74102180 0x7f8b74101a90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800b530 tx=0x7f8b6800b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:16:30.971 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.966+0000 7f8b637fe700 1 -- 172.21.15.159:0/916474317 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b68011830 con 0x7f8b74102180 2023-12-07T10:16:30.971 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.966+0000 7f8b78bfd700 1 -- 172.21.15.159:0/916474317 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b741a18d0 con 0x7f8b74102180 2023-12-07T10:16:30.971 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.966+0000 7f8b637fe700 1 -- 172.21.15.159:0/916474317 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b68011e50 con 0x7f8b74102180 2023-12-07T10:16:30.971 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.966+0000 7f8b637fe700 1 -- 172.21.15.159:0/916474317 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6800f2e0 con 0x7f8b74102180 2023-12-07T10:16:30.972 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.966+0000 7f8b78bfd700 1 -- 172.21.15.159:0/916474317 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b741a1dd0 con 0x7f8b74102180 2023-12-07T10:16:30.972 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.967+0000 7f8b78bfd700 1 -- 172.21.15.159:0/916474317 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b740fd690 con 0x7f8b74102180 2023-12-07T10:16:30.972 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.968+0000 7f8b637fe700 1 -- 172.21.15.159:0/916474317 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f8b68010380 con 0x7f8b74102180 2023-12-07T10:16:30.973 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.969+0000 7f8b637fe700 1 --2- 172.21.15.159:0/916474317 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8b5c0a1b10 0x7f8b5c0a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:16:30.973 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.969+0000 7f8b637fe700 1 -- 172.21.15.159:0/916474317 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f8b680cd040 con 0x7f8b74102180 2023-12-07T10:16:30.973 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.969+0000 7f8b71d9b700 1 --2- 172.21.15.159:0/916474317 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8b5c0a1b10 0x7f8b5c0a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:16:30.973 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.970+0000 7f8b71d9b700 1 --2- 172.21.15.159:0/916474317 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8b5c0a1b10 0x7f8b5c0a3fd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400b5c0 tx=0x7f8b64003790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:16:30.975 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:30.973+0000 7f8b637fe700 1 -- 172.21.15.159:0/916474317 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8b68096a90 con 0x7f8b74102180 2023-12-07T10:16:31.171 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:31.168+0000 7f8b78bfd700 1 -- 172.21.15.159:0/916474317 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b7402d3c0 con 0x7f8b5c0a1b10 2023-12-07T10:16:31.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:31 smithi119 ceph-mon[97908]: purged_snaps scrub starts 2023-12-07T10:16:31.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:31 smithi119 ceph-mon[97908]: purged_snaps scrub ok 2023-12-07T10:16:31.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:31 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-07T10:16:31.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:31 smithi119 ceph-mon[97908]: osdmap e73: 11 total, 11 up, 11 in 2023-12-07T10:16:31.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:31 smithi119 ceph-mon[97908]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-07T10:16:31.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:31 smithi119 ceph-mon[97908]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:16:31.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:31 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:16:31.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:31 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:16:31.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:31 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:31.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:31 smithi159 ceph-mon[134726]: purged_snaps scrub starts 2023-12-07T10:16:31.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:31 smithi159 ceph-mon[134726]: purged_snaps scrub ok 2023-12-07T10:16:31.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:31 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-07T10:16:31.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:31 smithi159 ceph-mon[134726]: osdmap e73: 11 total, 11 up, 11 in 2023-12-07T10:16:31.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:31 smithi159 ceph-mon[134726]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-07T10:16:31.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:31 smithi159 ceph-mon[134726]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:16:31.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:31 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:16:31.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:31 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:16:31.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:31 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:31.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:31 smithi153 ceph-mon[133625]: purged_snaps scrub starts 2023-12-07T10:16:31.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:31 smithi153 ceph-mon[133625]: purged_snaps scrub ok 2023-12-07T10:16:31.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:31 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-07T10:16:31.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:31 smithi153 ceph-mon[133625]: osdmap e73: 11 total, 11 up, 11 in 2023-12-07T10:16:31.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:31 smithi153 ceph-mon[133625]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-07T10:16:31.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:31 smithi153 ceph-mon[133625]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:16:31.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:31 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-07T10:16:31.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:31 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-07T10:16:31.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:31 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:32.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:32 smithi119 ceph-mon[97908]: osdmap e74: 11 total, 11 up, 11 in 2023-12-07T10:16:32.765 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:32 smithi159 ceph-mon[134726]: osdmap e74: 11 total, 11 up, 11 in 2023-12-07T10:16:32.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:32 smithi153 ceph-mon[133625]: osdmap e74: 11 total, 11 up, 11 in 2023-12-07T10:16:33.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:33 smithi119 ceph-mon[97908]: osdmap e75: 11 total, 11 up, 11 in 2023-12-07T10:16:33.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:33 smithi119 ceph-mon[97908]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-12-07T10:16:33.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:33 smithi159 ceph-mon[134726]: osdmap e75: 11 total, 11 up, 11 in 2023-12-07T10:16:33.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:33 smithi159 ceph-mon[134726]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-12-07T10:16:33.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:33 smithi153 ceph-mon[133625]: osdmap e75: 11 total, 11 up, 11 in 2023-12-07T10:16:33.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:33 smithi153 ceph-mon[133625]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-12-07T10:16:34.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:34 smithi119 ceph-mon[97908]: osdmap e76: 11 total, 11 up, 11 in 2023-12-07T10:16:34.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:34 smithi119 ceph-mon[97908]: from='client.? 172.21.15.159:0/2557424000' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0fc5909-279d-4172-977c-2dc2da0a7e34"}]: dispatch 2023-12-07T10:16:34.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:34 smithi119 ceph-mon[97908]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0fc5909-279d-4172-977c-2dc2da0a7e34"}]: dispatch 2023-12-07T10:16:34.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:34 smithi119 ceph-mon[97908]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0fc5909-279d-4172-977c-2dc2da0a7e34"}]': finished 2023-12-07T10:16:34.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:34 smithi119 ceph-mon[97908]: osdmap e77: 12 total, 11 up, 12 in 2023-12-07T10:16:34.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:34 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-07T10:16:34.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:34 smithi119 ceph-mon[97908]: from='client.? 172.21.15.159:0/3882614484' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:16:34.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:34 smithi159 ceph-mon[134726]: osdmap e76: 11 total, 11 up, 11 in 2023-12-07T10:16:34.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:34 smithi159 ceph-mon[134726]: from='client.? 172.21.15.159:0/2557424000' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0fc5909-279d-4172-977c-2dc2da0a7e34"}]: dispatch 2023-12-07T10:16:34.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:34 smithi159 ceph-mon[134726]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0fc5909-279d-4172-977c-2dc2da0a7e34"}]: dispatch 2023-12-07T10:16:34.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:34 smithi159 ceph-mon[134726]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0fc5909-279d-4172-977c-2dc2da0a7e34"}]': finished 2023-12-07T10:16:34.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:34 smithi159 ceph-mon[134726]: osdmap e77: 12 total, 11 up, 12 in 2023-12-07T10:16:34.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:34 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-07T10:16:34.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:34 smithi159 ceph-mon[134726]: from='client.? 172.21.15.159:0/3882614484' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:16:34.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:34 smithi153 ceph-mon[133625]: osdmap e76: 11 total, 11 up, 11 in 2023-12-07T10:16:34.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:34 smithi153 ceph-mon[133625]: from='client.? 172.21.15.159:0/2557424000' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0fc5909-279d-4172-977c-2dc2da0a7e34"}]: dispatch 2023-12-07T10:16:34.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:34 smithi153 ceph-mon[133625]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0fc5909-279d-4172-977c-2dc2da0a7e34"}]: dispatch 2023-12-07T10:16:34.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:34 smithi153 ceph-mon[133625]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0fc5909-279d-4172-977c-2dc2da0a7e34"}]': finished 2023-12-07T10:16:34.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:34 smithi153 ceph-mon[133625]: osdmap e77: 12 total, 11 up, 12 in 2023-12-07T10:16:34.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:34 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-07T10:16:34.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:34 smithi153 ceph-mon[133625]: from='client.? 172.21.15.159:0/3882614484' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-07T10:16:35.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:35 smithi159 ceph-mon[134726]: pgmap v234: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-12-07T10:16:35.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:35 smithi153 ceph-mon[133625]: pgmap v234: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-12-07T10:16:35.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:35 smithi119 ceph-mon[97908]: pgmap v234: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-12-07T10:16:37.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:37 smithi119 ceph-mon[97908]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2023-12-07T10:16:38.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:37 smithi159 ceph-mon[134726]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2023-12-07T10:16:38.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:37 smithi153 ceph-mon[133625]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2023-12-07T10:16:39.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:39 smithi119 ceph-mon[97908]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-12-07T10:16:40.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:39 smithi159 ceph-mon[134726]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-12-07T10:16:40.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:39 smithi153 ceph-mon[133625]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-12-07T10:16:40.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:40 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-12-07T10:16:40.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:40 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:40.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:40 smithi119 ceph-mon[97908]: Deploying daemon osd.11 on smithi159 2023-12-07T10:16:41.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:40 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-12-07T10:16:41.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:40 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:41.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:40 smithi159 ceph-mon[134726]: Deploying daemon osd.11 on smithi159 2023-12-07T10:16:41.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:40 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-12-07T10:16:41.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:40 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:41.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:40 smithi153 ceph-mon[133625]: Deploying daemon osd.11 on smithi159 2023-12-07T10:16:41.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:41 smithi119 ceph-mon[97908]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-07T10:16:41.992 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:41 smithi159 ceph-mon[134726]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-07T10:16:42.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:41 smithi153 ceph-mon[133625]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-07T10:16:43.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:43 smithi119 ceph-mon[97908]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-12-07T10:16:44.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:43 smithi159 ceph-mon[134726]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-12-07T10:16:44.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:43 smithi153 ceph-mon[133625]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-12-07T10:16:45.890 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:45.891 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:45.891 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:16:45.891 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:45 smithi159 ceph-mon[134726]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-12-07T10:16:45.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:45 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:45.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:45 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:45.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:45 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:16:45.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:45 smithi119 ceph-mon[97908]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-12-07T10:16:46.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:45 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:46.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:45 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:46.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:45 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:16:46.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:45 smithi153 ceph-mon[133625]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-12-07T10:16:47.435 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:47.432+0000 7f8b637fe700 1 -- 172.21.15.159:0/916474317 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f8b7402d3c0 con 0x7f8b5c0a1b10 2023-12-07T10:16:47.435 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 11 on host 'smithi159' 2023-12-07T10:16:47.437 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:47.435+0000 7f8b78bfd700 1 -- 172.21.15.159:0/916474317 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8b5c0a1b10 msgr2=0x7f8b5c0a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:47.437 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:47.435+0000 7f8b78bfd700 1 --2- 172.21.15.159:0/916474317 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8b5c0a1b10 0x7f8b5c0a3fd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400b5c0 tx=0x7f8b64003790 comp rx=0 tx=0).stop 2023-12-07T10:16:47.437 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:47.435+0000 7f8b78bfd700 1 -- 172.21.15.159:0/916474317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b74102180 msgr2=0x7f8b74101a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:47.437 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:47.435+0000 7f8b78bfd700 1 --2- 172.21.15.159:0/916474317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b74102180 0x7f8b74101a90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800b530 tx=0x7f8b6800b840 comp rx=0 tx=0).stop 2023-12-07T10:16:47.437 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:47.435+0000 7f8b78bfd700 1 -- 172.21.15.159:0/916474317 shutdown_connections 2023-12-07T10:16:47.438 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:47.435+0000 7f8b78bfd700 1 --2- 172.21.15.159:0/916474317 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8b5c0a1b10 0x7f8b5c0a3fd0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:47.438 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:47.435+0000 7f8b78bfd700 1 --2- 172.21.15.159:0/916474317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b741073e0 0x7f8b74100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:47.438 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:47.435+0000 7f8b78bfd700 1 --2- 172.21.15.159:0/916474317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b74102180 0x7f8b74101a90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:47.438 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:47.435+0000 7f8b78bfd700 1 --2- 172.21.15.159:0/916474317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b74104ab0 0x7f8b74100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:47.438 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:47.435+0000 7f8b78bfd700 1 -- 172.21.15.159:0/916474317 >> 172.21.15.159:0/916474317 conn(0x7f8b740fbb10 msgr2=0x7f8b74102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:16:47.438 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:47.435+0000 7f8b78bfd700 1 -- 172.21.15.159:0/916474317 shutdown_connections 2023-12-07T10:16:47.438 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:16:47.435+0000 7f8b78bfd700 1 -- 172.21.15.159:0/916474317 wait complete. 2023-12-07T10:16:47.862 DEBUG:teuthology.orchestra.run.smithi159:osd.11> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.11.service 2023-12-07T10:16:47.866 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-12-07T10:16:47.867 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph osd stat -f json 2023-12-07T10:16:47.901 INFO:journalctl@ceph.osd.11.smithi159.stdout:-- Logs begin at Thu 2023-12-07 09:49:21 UTC. -- 2023-12-07T10:16:47.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:47 smithi119 ceph-mon[97908]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-07T10:16:47.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:47 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:47.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:47 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:48.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:47 smithi159 ceph-mon[134726]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-07T10:16:48.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:47 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:48.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:47 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:48.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:47 smithi153 ceph-mon[133625]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-07T10:16:48.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:47 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:48.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:47 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:48.643 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:16:48.907 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:16:48 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:16:48.597+0000 7f3469343640 -1 osd.11 0 log_to_monitors true 2023-12-07T10:16:49.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:48 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:49.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:48 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:49.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:48 smithi119 ceph-mon[97908]: from='osd.11 [v2:172.21.15.159:6824/1262216348,v1:172.21.15.159:6825/1262216348]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-07T10:16:49.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:48 smithi119 ceph-mon[97908]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-07T10:16:49.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:48 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:49.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:48 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:49.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:48 smithi159 ceph-mon[134726]: from='osd.11 [v2:172.21.15.159:6824/1262216348,v1:172.21.15.159:6825/1262216348]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-07T10:16:49.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:48 smithi159 ceph-mon[134726]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-07T10:16:49.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:48 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:49.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:48 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:49.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:48 smithi153 ceph-mon[133625]: from='osd.11 [v2:172.21.15.159:6824/1262216348,v1:172.21.15.159:6825/1262216348]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-07T10:16:49.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:48 smithi153 ceph-mon[133625]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-07T10:16:50.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.163+0000 7f299c0f1700 1 -- 172.21.15.119:0/3520467113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2994103200 msgr2=0x7f2994109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:50.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.163+0000 7f299c0f1700 1 --2- 172.21.15.119:0/3520467113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2994103200 0x7f2994109ab0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f298c00b0d0 tx=0x7f298c00b3e0 comp rx=0 tx=0).stop 2023-12-07T10:16:50.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.163+0000 7f299c0f1700 1 -- 172.21.15.119:0/3520467113 shutdown_connections 2023-12-07T10:16:50.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.163+0000 7f299c0f1700 1 --2- 172.21.15.119:0/3520467113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2994103200 0x7f2994109ab0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:50.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.163+0000 7f299c0f1700 1 --2- 172.21.15.119:0/3520467113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2994101660 0x7f2994101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:50.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.163+0000 7f299c0f1700 1 --2- 172.21.15.119:0/3520467113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2994102860 0x7f2994102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:50.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.163+0000 7f299c0f1700 1 -- 172.21.15.119:0/3520467113 >> 172.21.15.119:0/3520467113 conn(0x7f29940fcc20 msgr2=0x7f29940ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:16:50.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.163+0000 7f299c0f1700 1 -- 172.21.15.119:0/3520467113 shutdown_connections 2023-12-07T10:16:50.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.163+0000 7f299c0f1700 1 -- 172.21.15.119:0/3520467113 wait complete. 2023-12-07T10:16:50.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.164+0000 7f299c0f1700 1 Processor -- start 2023-12-07T10:16:50.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.164+0000 7f299c0f1700 1 -- start start 2023-12-07T10:16:50.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.164+0000 7f299c0f1700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2994101660 0x7f2994075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:16:50.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.164+0000 7f299c0f1700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2994102860 0x7f2994073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:16:50.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.164+0000 7f299c0f1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2994103200 0x7f2994073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:16:50.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.164+0000 7f299c0f1700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2994074620 con 0x7f2994101660 2023-12-07T10:16:50.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.164+0000 7f299c0f1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2994074790 con 0x7f2994103200 2023-12-07T10:16:50.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.164+0000 7f299c0f1700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29941a1770 con 0x7f2994102860 2023-12-07T10:16:50.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.165+0000 7f2999e8d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2994101660 0x7f2994075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:16:50.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.165+0000 7f299968c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2994102860 0x7f2994073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:16:50.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.165+0000 7f299968c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2994102860 0x7f2994073a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:52414/0 (socket says 172.21.15.119:52414) 2023-12-07T10:16:50.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.165+0000 7f299968c700 1 -- 172.21.15.119:0/323284065 learned_addr learned my addr 172.21.15.119:0/323284065 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:16:50.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.165+0000 7f2999e8d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2994101660 0x7f2994075310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58336/0 (socket says 172.21.15.119:58336) 2023-12-07T10:16:50.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.165+0000 7f299a68e700 1 --2- 172.21.15.119:0/323284065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2994103200 0x7f2994073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:16:50.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.165+0000 7f2999e8d700 1 -- 172.21.15.119:0/323284065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2994102860 msgr2=0x7f2994073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:50.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.165+0000 7f2999e8d700 1 --2- 172.21.15.119:0/323284065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2994102860 0x7f2994073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:50.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.165+0000 7f2999e8d700 1 -- 172.21.15.119:0/323284065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2994103200 msgr2=0x7f2994073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:50.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.165+0000 7f2999e8d700 1 --2- 172.21.15.119:0/323284065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2994103200 0x7f2994073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:50.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.165+0000 7f2999e8d700 1 -- 172.21.15.119:0/323284065 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298c009d00 con 0x7f2994101660 2023-12-07T10:16:50.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.165+0000 7f299a68e700 1 --2- 172.21.15.119:0/323284065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2994103200 0x7f2994073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:16:50.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.165+0000 7f2999e8d700 1 --2- 172.21.15.119:0/323284065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2994101660 0x7f2994075310 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f299000c940 tx=0x7f299000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:16:50.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.166+0000 7f298affd700 1 -- 172.21.15.119:0/323284065 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2990007aa0 con 0x7f2994101660 2023-12-07T10:16:50.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.166+0000 7f299c0f1700 1 -- 172.21.15.119:0/323284065 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29941a19d0 con 0x7f2994101660 2023-12-07T10:16:50.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.166+0000 7f298affd700 1 -- 172.21.15.119:0/323284065 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f299000ce90 con 0x7f2994101660 2023-12-07T10:16:50.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.166+0000 7f298affd700 1 -- 172.21.15.119:0/323284065 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299000f490 con 0x7f2994101660 2023-12-07T10:16:50.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.166+0000 7f299c0f1700 1 -- 172.21.15.119:0/323284065 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29941a1ef0 con 0x7f2994101660 2023-12-07T10:16:50.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.167+0000 7f299c0f1700 1 -- 172.21.15.119:0/323284065 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2994108a20 con 0x7f2994101660 2023-12-07T10:16:50.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.169+0000 7f298affd700 1 -- 172.21.15.119:0/323284065 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f2990018480 con 0x7f2994101660 2023-12-07T10:16:50.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.170+0000 7f298affd700 1 --2- 172.21.15.119:0/323284065 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f29800a1b10 0x7f29800a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:16:50.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.170+0000 7f298affd700 1 -- 172.21.15.119:0/323284065 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f2990013070 con 0x7f2994101660 2023-12-07T10:16:50.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.170+0000 7f299968c700 1 --2- 172.21.15.119:0/323284065 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f29800a1b10 0x7f29800a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:16:50.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.171+0000 7f299968c700 1 --2- 172.21.15.119:0/323284065 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f29800a1b10 0x7f29800a3fd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2984009ad0 tx=0x7f2984009290 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:16:50.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.174+0000 7f298affd700 1 -- 172.21.15.119:0/323284065 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2990096bf0 con 0x7f2994101660 2023-12-07T10:16:50.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:49 smithi119 ceph-mon[97908]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-12-07T10:16:50.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:49 smithi119 ceph-mon[97908]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-12-07T10:16:50.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:49 smithi119 ceph-mon[97908]: from='osd.11 [v2:172.21.15.159:6824/1262216348,v1:172.21.15.159:6825/1262216348]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-12-07T10:16:50.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:49 smithi119 ceph-mon[97908]: osdmap e78: 12 total, 11 up, 12 in 2023-12-07T10:16:50.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:49 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-07T10:16:50.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:49 smithi119 ceph-mon[97908]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-12-07T10:16:50.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:49 smithi159 ceph-mon[134726]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-12-07T10:16:50.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:49 smithi159 ceph-mon[134726]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-12-07T10:16:50.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:49 smithi159 ceph-mon[134726]: from='osd.11 [v2:172.21.15.159:6824/1262216348,v1:172.21.15.159:6825/1262216348]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-12-07T10:16:50.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:49 smithi159 ceph-mon[134726]: osdmap e78: 12 total, 11 up, 12 in 2023-12-07T10:16:50.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:49 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-07T10:16:50.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:49 smithi159 ceph-mon[134726]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-12-07T10:16:50.265 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:16:49 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:16:49.927+0000 7f345e9ba700 -1 osd.11 0 waiting for initial osdmap 2023-12-07T10:16:50.265 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:16:49 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:16:49.940+0000 7f34577ae700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-07T10:16:50.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:49 smithi153 ceph-mon[133625]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-12-07T10:16:50.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:49 smithi153 ceph-mon[133625]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-12-07T10:16:50.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:49 smithi153 ceph-mon[133625]: from='osd.11 [v2:172.21.15.159:6824/1262216348,v1:172.21.15.159:6825/1262216348]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-12-07T10:16:50.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:49 smithi153 ceph-mon[133625]: osdmap e78: 12 total, 11 up, 12 in 2023-12-07T10:16:50.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:49 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-07T10:16:50.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:49 smithi153 ceph-mon[133625]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2023-12-07T10:16:50.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.375+0000 7f299c0f1700 1 -- 172.21.15.119:0/323284065 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f299402d350 con 0x7f2994101660 2023-12-07T10:16:50.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.375+0000 7f298affd700 1 -- 172.21.15.119:0/323284065 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7f299009a210 con 0x7f2994101660 2023-12-07T10:16:50.378 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:16:50.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.378+0000 7f299c0f1700 1 -- 172.21.15.119:0/323284065 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f29800a1b10 msgr2=0x7f29800a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:50.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.378+0000 7f299c0f1700 1 --2- 172.21.15.119:0/323284065 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f29800a1b10 0x7f29800a3fd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2984009ad0 tx=0x7f2984009290 comp rx=0 tx=0).stop 2023-12-07T10:16:50.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.378+0000 7f299c0f1700 1 -- 172.21.15.119:0/323284065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2994101660 msgr2=0x7f2994075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:50.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.378+0000 7f299c0f1700 1 --2- 172.21.15.119:0/323284065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2994101660 0x7f2994075310 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f299000c940 tx=0x7f299000cc50 comp rx=0 tx=0).stop 2023-12-07T10:16:50.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.378+0000 7f299c0f1700 1 -- 172.21.15.119:0/323284065 shutdown_connections 2023-12-07T10:16:50.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.378+0000 7f299c0f1700 1 --2- 172.21.15.119:0/323284065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2994103200 0x7f2994073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:50.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.378+0000 7f299c0f1700 1 --2- 172.21.15.119:0/323284065 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f29800a1b10 0x7f29800a3fd0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:50.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.378+0000 7f299c0f1700 1 --2- 172.21.15.119:0/323284065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2994101660 0x7f2994075310 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.378+0000 7f299c0f1700 1 --2- 172.21.15.119:0/323284065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2994102860 0x7f2994073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.378+0000 7f299c0f1700 1 -- 172.21.15.119:0/323284065 >> 172.21.15.119:0/323284065 conn(0x7f29940fcc20 msgr2=0x7f2994105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:16:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.378+0000 7f299c0f1700 1 -- 172.21.15.119:0/323284065 shutdown_connections 2023-12-07T10:16:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:50.378+0000 7f299c0f1700 1 -- 172.21.15.119:0/323284065 wait complete. 2023-12-07T10:16:50.830 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":79,"num_osds":12,"num_up_osds":11,"osd_up_since":1701944190,"num_in_osds":12,"osd_in_since":1701944193,"num_remapped_pgs":0} 2023-12-07T10:16:51.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:50 smithi119 ceph-mon[97908]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-12-07T10:16:51.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:50 smithi119 ceph-mon[97908]: osdmap e79: 12 total, 11 up, 12 in 2023-12-07T10:16:51.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:50 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-07T10:16:51.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:50 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-07T10:16:51.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:50 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/323284065' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-07T10:16:51.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:50 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:51.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:50 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:51.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:50 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:51.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:50 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:51.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:50 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:51.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:50 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:51.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:50 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:51.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:50 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:51.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:50 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:16:51.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:50 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:51.231 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:50 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-07T10:16:51.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:50 smithi159 ceph-mon[134726]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-12-07T10:16:51.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:50 smithi159 ceph-mon[134726]: osdmap e79: 12 total, 11 up, 12 in 2023-12-07T10:16:51.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:50 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-07T10:16:51.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:50 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-07T10:16:51.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:50 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/323284065' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-07T10:16:51.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:50 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:51.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:50 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:51.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:50 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:51.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:50 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:51.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:50 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:51.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:50 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:51.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:50 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:51.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:50 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:51.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:50 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:16:51.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:50 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:51.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:50 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-07T10:16:51.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:50 smithi153 ceph-mon[133625]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2023-12-07T10:16:51.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:50 smithi153 ceph-mon[133625]: osdmap e79: 12 total, 11 up, 12 in 2023-12-07T10:16:51.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:50 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-07T10:16:51.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:50 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-07T10:16:51.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:50 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/323284065' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-07T10:16:51.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:50 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:51.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:50 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:51.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:50 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:51.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:50 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:51.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:50 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:51.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:50 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:16:51.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:50 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:51.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:50 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:51.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:50 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:16:51.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:50 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:51.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:50 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-07T10:16:51.831 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph osd stat -f json 2023-12-07T10:16:52.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:51 smithi119 ceph-mon[97908]: purged_snaps scrub starts 2023-12-07T10:16:52.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:51 smithi119 ceph-mon[97908]: purged_snaps scrub ok 2023-12-07T10:16:52.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:51 smithi119 ceph-mon[97908]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-12-07T10:16:52.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:51 smithi119 ceph-mon[97908]: Detected new or changed devices on smithi159 2023-12-07T10:16:52.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:51 smithi119 ceph-mon[97908]: Adjusting osd_memory_target on smithi159 to 4292M 2023-12-07T10:16:52.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:51 smithi119 ceph-mon[97908]: osd.11 [v2:172.21.15.159:6824/1262216348,v1:172.21.15.159:6825/1262216348] boot 2023-12-07T10:16:52.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:51 smithi119 ceph-mon[97908]: osdmap e80: 12 total, 12 up, 12 in 2023-12-07T10:16:52.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:51 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-07T10:16:52.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:51 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:16:52.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:51 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:52.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:51 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:16:52.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:51 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:52.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:51 smithi159 ceph-mon[134726]: purged_snaps scrub starts 2023-12-07T10:16:52.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:51 smithi159 ceph-mon[134726]: purged_snaps scrub ok 2023-12-07T10:16:52.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:51 smithi159 ceph-mon[134726]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-12-07T10:16:52.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:51 smithi159 ceph-mon[134726]: Detected new or changed devices on smithi159 2023-12-07T10:16:52.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:51 smithi159 ceph-mon[134726]: Adjusting osd_memory_target on smithi159 to 4292M 2023-12-07T10:16:52.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:51 smithi159 ceph-mon[134726]: osd.11 [v2:172.21.15.159:6824/1262216348,v1:172.21.15.159:6825/1262216348] boot 2023-12-07T10:16:52.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:51 smithi159 ceph-mon[134726]: osdmap e80: 12 total, 12 up, 12 in 2023-12-07T10:16:52.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:51 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-07T10:16:52.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:51 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:16:52.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:51 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:52.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:51 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:16:52.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:51 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:52.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:51 smithi153 ceph-mon[133625]: purged_snaps scrub starts 2023-12-07T10:16:52.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:51 smithi153 ceph-mon[133625]: purged_snaps scrub ok 2023-12-07T10:16:52.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:51 smithi153 ceph-mon[133625]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-12-07T10:16:52.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:51 smithi153 ceph-mon[133625]: Detected new or changed devices on smithi159 2023-12-07T10:16:52.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:51 smithi153 ceph-mon[133625]: Adjusting osd_memory_target on smithi159 to 4292M 2023-12-07T10:16:52.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:51 smithi153 ceph-mon[133625]: osd.11 [v2:172.21.15.159:6824/1262216348,v1:172.21.15.159:6825/1262216348] boot 2023-12-07T10:16:52.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:51 smithi153 ceph-mon[133625]: osdmap e80: 12 total, 12 up, 12 in 2023-12-07T10:16:52.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:51 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-07T10:16:52.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:51 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:16:52.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:51 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:16:52.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:51 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:16:52.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:51 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:16:52.610 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:16:53.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:53 smithi119 ceph-mon[97908]: osdmap e81: 12 total, 12 up, 12 in 2023-12-07T10:16:53.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:53 smithi119 ceph-mon[97908]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:16:53.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:53 smithi159 ceph-mon[134726]: osdmap e81: 12 total, 12 up, 12 in 2023-12-07T10:16:53.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:53 smithi159 ceph-mon[134726]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:16:53.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:53 smithi153 ceph-mon[133625]: osdmap e81: 12 total, 12 up, 12 in 2023-12-07T10:16:53.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:53 smithi153 ceph-mon[133625]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:16:54.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.128+0000 7f32ef396700 1 -- 172.21.15.119:0/4255334532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32e8101660 msgr2=0x7f32e8101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:54.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.128+0000 7f32ef396700 1 --2- 172.21.15.119:0/4255334532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32e8101660 0x7f32e8101a80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f32e40099c0 tx=0x7f32e4009cd0 comp rx=0 tx=0).stop 2023-12-07T10:16:54.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.128+0000 7f32ef396700 1 -- 172.21.15.119:0/4255334532 shutdown_connections 2023-12-07T10:16:54.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.128+0000 7f32ef396700 1 --2- 172.21.15.119:0/4255334532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32e8103200 0x7f32e8109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:54.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.128+0000 7f32ef396700 1 --2- 172.21.15.119:0/4255334532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32e8101660 0x7f32e8101a80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:54.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.128+0000 7f32ef396700 1 --2- 172.21.15.119:0/4255334532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32e8102860 0x7f32e8102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:54.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.128+0000 7f32ef396700 1 -- 172.21.15.119:0/4255334532 >> 172.21.15.119:0/4255334532 conn(0x7f32e80fcc20 msgr2=0x7f32e80ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:16:54.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.128+0000 7f32ef396700 1 -- 172.21.15.119:0/4255334532 shutdown_connections 2023-12-07T10:16:54.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.128+0000 7f32ef396700 1 -- 172.21.15.119:0/4255334532 wait complete. 2023-12-07T10:16:54.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.130+0000 7f32ef396700 1 Processor -- start 2023-12-07T10:16:54.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.130+0000 7f32ef396700 1 -- start start 2023-12-07T10:16:54.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.130+0000 7f32ef396700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32e8101660 0x7f32e8197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:16:54.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.130+0000 7f32ef396700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32e8102860 0x7f32e8197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:16:54.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.130+0000 7f32ef396700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32e8103200 0x7f32e819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:16:54.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.130+0000 7f32ef396700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e819f140 con 0x7f32e8103200 2023-12-07T10:16:54.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.130+0000 7f32ef396700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e819f280 con 0x7f32e8102860 2023-12-07T10:16:54.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.130+0000 7f32ef396700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e819f410 con 0x7f32e8101660 2023-12-07T10:16:54.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.131+0000 7f32ed933700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32e8103200 0x7f32e819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:16:54.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.131+0000 7f32ed132700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32e8101660 0x7f32e8197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:16:54.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.131+0000 7f32ed933700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32e8103200 0x7f32e819eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55206/0 (socket says 172.21.15.119:55206) 2023-12-07T10:16:54.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.131+0000 7f32ed933700 1 -- 172.21.15.119:0/3839383775 learned_addr learned my addr 172.21.15.119:0/3839383775 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:16:54.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.131+0000 7f32ec931700 1 --2- 172.21.15.119:0/3839383775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32e8102860 0x7f32e8197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:16:54.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.131+0000 7f32ed933700 1 -- 172.21.15.119:0/3839383775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32e8101660 msgr2=0x7f32e8197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:54.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.131+0000 7f32ed933700 1 --2- 172.21.15.119:0/3839383775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32e8101660 0x7f32e8197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:54.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.131+0000 7f32ed933700 1 -- 172.21.15.119:0/3839383775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32e8102860 msgr2=0x7f32e8197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:54.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.131+0000 7f32ed933700 1 --2- 172.21.15.119:0/3839383775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32e8102860 0x7f32e8197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:54.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.131+0000 7f32ed933700 1 -- 172.21.15.119:0/3839383775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d8009520 con 0x7f32e8103200 2023-12-07T10:16:54.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.131+0000 7f32ed933700 1 --2- 172.21.15.119:0/3839383775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32e8103200 0x7f32e819eb70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f32e000ef60 tx=0x7f32e000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:16:54.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.131+0000 7f32de7fc700 1 -- 172.21.15.119:0/3839383775 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e0018070 con 0x7f32e8103200 2023-12-07T10:16:54.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.131+0000 7f32de7fc700 1 -- 172.21.15.119:0/3839383775 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32e000f040 con 0x7f32e8103200 2023-12-07T10:16:54.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.131+0000 7f32ef396700 1 -- 172.21.15.119:0/3839383775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e40095f0 con 0x7f32e8103200 2023-12-07T10:16:54.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.131+0000 7f32de7fc700 1 -- 172.21.15.119:0/3839383775 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e000ccc0 con 0x7f32e8103200 2023-12-07T10:16:54.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.132+0000 7f32ef396700 1 -- 172.21.15.119:0/3839383775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e819fa50 con 0x7f32e8103200 2023-12-07T10:16:54.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.133+0000 7f32ef396700 1 -- 172.21.15.119:0/3839383775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32e8108b40 con 0x7f32e8103200 2023-12-07T10:16:54.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.135+0000 7f32de7fc700 1 -- 172.21.15.119:0/3839383775 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f32e0003a00 con 0x7f32e8103200 2023-12-07T10:16:54.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.136+0000 7f32de7fc700 1 --2- 172.21.15.119:0/3839383775 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f32d40a1b90 0x7f32d40a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:16:54.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.136+0000 7f32de7fc700 1 -- 172.21.15.119:0/3839383775 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f32e00cd540 con 0x7f32e8103200 2023-12-07T10:16:54.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.136+0000 7f32ed132700 1 --2- 172.21.15.119:0/3839383775 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f32d40a1b90 0x7f32d40a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:16:54.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.137+0000 7f32ed132700 1 --2- 172.21.15.119:0/3839383775 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f32d40a1b90 0x7f32d40a4050 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f32e4009990 tx=0x7f32e4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:16:54.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.139+0000 7f32de7fc700 1 -- 172.21.15.119:0/3839383775 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f32e0096df0 con 0x7f32e8103200 2023-12-07T10:16:54.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.338+0000 7f32ef396700 1 -- 172.21.15.119:0/3839383775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f32e804f2d0 con 0x7f32e8103200 2023-12-07T10:16:54.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.339+0000 7f32de7fc700 1 -- 172.21.15.119:0/3839383775 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) v1 ==== 74+0+133 (secure 0 0 0) 0x7f32e009a410 con 0x7f32e8103200 2023-12-07T10:16:54.341 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:16:54.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.341+0000 7f32ef396700 1 -- 172.21.15.119:0/3839383775 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f32d40a1b90 msgr2=0x7f32d40a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:54.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.341+0000 7f32ef396700 1 --2- 172.21.15.119:0/3839383775 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f32d40a1b90 0x7f32d40a4050 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f32e4009990 tx=0x7f32e4000bc0 comp rx=0 tx=0).stop 2023-12-07T10:16:54.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.341+0000 7f32ef396700 1 -- 172.21.15.119:0/3839383775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32e8103200 msgr2=0x7f32e819eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:54.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.341+0000 7f32ef396700 1 --2- 172.21.15.119:0/3839383775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32e8103200 0x7f32e819eb70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f32e000ef60 tx=0x7f32e000ef90 comp rx=0 tx=0).stop 2023-12-07T10:16:54.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.341+0000 7f32ef396700 1 -- 172.21.15.119:0/3839383775 shutdown_connections 2023-12-07T10:16:54.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.341+0000 7f32ef396700 1 --2- 172.21.15.119:0/3839383775 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f32d40a1b90 0x7f32d40a4050 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:54.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.341+0000 7f32ef396700 1 --2- 172.21.15.119:0/3839383775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32e8103200 0x7f32e819eb70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:54.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.341+0000 7f32ef396700 1 --2- 172.21.15.119:0/3839383775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32e8101660 0x7f32e8197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:54.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.341+0000 7f32ef396700 1 --2- 172.21.15.119:0/3839383775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32e8102860 0x7f32e8197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:54.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.341+0000 7f32ef396700 1 -- 172.21.15.119:0/3839383775 >> 172.21.15.119:0/3839383775 conn(0x7f32e80fcc20 msgr2=0x7f32e8105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:16:54.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.341+0000 7f32ef396700 1 -- 172.21.15.119:0/3839383775 shutdown_connections 2023-12-07T10:16:54.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:54.341+0000 7f32ef396700 1 -- 172.21.15.119:0/3839383775 wait complete. 2023-12-07T10:16:54.712 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:54 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3839383775' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-07T10:16:54.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:54 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3839383775' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-07T10:16:54.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:54 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3839383775' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-07T10:16:54.795 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1701944210,"num_in_osds":12,"osd_in_since":1701944193,"num_remapped_pgs":0} 2023-12-07T10:16:54.796 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph osd dump --format=json 2023-12-07T10:16:55.566 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:16:55.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:55 smithi159 ceph-mon[134726]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:16:55.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:55 smithi153 ceph-mon[133625]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:16:55.841 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:55 smithi119 ceph-mon[97908]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:16:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.986+0000 7f084b4d1700 1 -- 172.21.15.119:0/3258851001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0844107730 msgr2=0x7f0844107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.986+0000 7f084b4d1700 1 --2- 172.21.15.119:0/3258851001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0844107730 0x7f0844107b10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0840009990 tx=0x7f0840009ca0 comp rx=0 tx=0).stop 2023-12-07T10:16:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.986+0000 7f084b4d1700 1 -- 172.21.15.119:0/3258851001 shutdown_connections 2023-12-07T10:16:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.986+0000 7f084b4d1700 1 --2- 172.21.15.119:0/3258851001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0844102080 0x7f084410e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.986+0000 7f084b4d1700 1 --2- 172.21.15.119:0/3258851001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0844107730 0x7f0844107b10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.986+0000 7f084b4d1700 1 --2- 172.21.15.119:0/3258851001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08441016c0 0x7f0844101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.986+0000 7f084b4d1700 1 -- 172.21.15.119:0/3258851001 >> 172.21.15.119:0/3258851001 conn(0x7f0844078800 msgr2=0x7f0844078c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:16:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.986+0000 7f084b4d1700 1 -- 172.21.15.119:0/3258851001 shutdown_connections 2023-12-07T10:16:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.986+0000 7f084b4d1700 1 -- 172.21.15.119:0/3258851001 wait complete. 2023-12-07T10:16:56.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.987+0000 7f084b4d1700 1 Processor -- start 2023-12-07T10:16:56.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.987+0000 7f084b4d1700 1 -- start start 2023-12-07T10:16:56.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.988+0000 7f084b4d1700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08441016c0 0x7f084419f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:16:56.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.988+0000 7f084b4d1700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0844102080 0x7f084419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:16:56.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.988+0000 7f084b4d1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0844107730 0x7f0844199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:16:56.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.988+0000 7f084b4d1700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084419a0e0 con 0x7f0844102080 2023-12-07T10:16:56.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.988+0000 7f084b4d1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084419a220 con 0x7f0844107730 2023-12-07T10:16:56.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.988+0000 7f084b4d1700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084419a3e0 con 0x7f08441016c0 2023-12-07T10:16:56.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.988+0000 7f084926d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08441016c0 0x7f084419f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:16:56.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.988+0000 7f0848a6c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0844102080 0x7f084419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:16:56.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.988+0000 7f084926d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08441016c0 0x7f084419f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:48912/0 (socket says 172.21.15.119:48912) 2023-12-07T10:16:56.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.988+0000 7f084926d700 1 -- 172.21.15.119:0/707371637 learned_addr learned my addr 172.21.15.119:0/707371637 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:16:56.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.988+0000 7f0849a6e700 1 --2- 172.21.15.119:0/707371637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0844107730 0x7f0844199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:16:56.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.988+0000 7f084926d700 1 -- 172.21.15.119:0/707371637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0844107730 msgr2=0x7f0844199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:56.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.988+0000 7f084926d700 1 --2- 172.21.15.119:0/707371637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0844107730 0x7f0844199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:56.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.989+0000 7f084926d700 1 -- 172.21.15.119:0/707371637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0844102080 msgr2=0x7f084419fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:56.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.989+0000 7f084926d700 1 --2- 172.21.15.119:0/707371637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0844102080 0x7f084419fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:56.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.989+0000 7f084926d700 1 -- 172.21.15.119:0/707371637 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08400095f0 con 0x7f08441016c0 2023-12-07T10:16:56.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.989+0000 7f0849a6e700 1 --2- 172.21.15.119:0/707371637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0844107730 0x7f0844199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:16:56.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.989+0000 7f0848a6c700 1 --2- 172.21.15.119:0/707371637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0844102080 0x7f084419fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:16:56.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.989+0000 7f084926d700 1 --2- 172.21.15.119:0/707371637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08441016c0 0x7f084419f870 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0840005770 tx=0x7f08400057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:16:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.989+0000 7f083a7fc700 1 -- 172.21.15.119:0/707371637 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084001d070 con 0x7f08441016c0 2023-12-07T10:16:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.989+0000 7f083a7fc700 1 -- 172.21.15.119:0/707371637 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0840021410 con 0x7f08441016c0 2023-12-07T10:16:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.989+0000 7f084b4d1700 1 -- 172.21.15.119:0/707371637 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f084419a660 con 0x7f08441016c0 2023-12-07T10:16:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.989+0000 7f083a7fc700 1 -- 172.21.15.119:0/707371637 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0840017650 con 0x7f08441016c0 2023-12-07T10:16:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.989+0000 7f084b4d1700 1 -- 172.21.15.119:0/707371637 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08441a6910 con 0x7f08441016c0 2023-12-07T10:16:56.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.990+0000 7f084b4d1700 1 -- 172.21.15.119:0/707371637 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0844104980 con 0x7f08441016c0 2023-12-07T10:16:56.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.992+0000 7f083a7fc700 1 -- 172.21.15.119:0/707371637 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f08400217f0 con 0x7f08441016c0 2023-12-07T10:16:56.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.993+0000 7f083a7fc700 1 --2- 172.21.15.119:0/707371637 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f08300a1b10 0x7f08300a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:16:56.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.994+0000 7f083a7fc700 1 -- 172.21.15.119:0/707371637 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f08400ce480 con 0x7f08441016c0 2023-12-07T10:16:56.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.994+0000 7f0848a6c700 1 --2- 172.21.15.119:0/707371637 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f08300a1b10 0x7f08300a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:16:56.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.994+0000 7f0848a6c700 1 --2- 172.21.15.119:0/707371637 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f08300a1b10 0x7f08300a3fd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f08440791c0 tx=0x7f0834009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:16:57.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:56.998+0000 7f083a7fc700 1 -- 172.21.15.119:0/707371637 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0840097d30 con 0x7f08441016c0 2023-12-07T10:16:57.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:57.198+0000 7f084b4d1700 1 -- 172.21.15.119:0/707371637 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f084419af60 con 0x7f08441016c0 2023-12-07T10:16:57.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:57.199+0000 7f083a7fc700 1 -- 172.21.15.119:0/707371637 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18492 (secure 0 0 0) 0x7f08400b9e60 con 0x7f08441016c0 2023-12-07T10:16:57.202 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:16:57.202 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":81,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","created":"2023-12-07T10:09:59.945863+0000","modified":"2023-12-07T10:16:52.466300+0000","last_up_change":"2023-12-07T10:16:50.929427+0000","last_in_change":"2023-12-07T10:16:33.558158+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T10:13:25.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"56b5818e-e625-40f1-8838-3ddcf2d34ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6803","nonce":2141950133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6805","nonce":2141950133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6809","nonce":2141950133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6807","nonce":2141950133}]},"public_addr":"172.21.15.119:6803/2141950133","cluster_addr":"172.21.15.119:6805/2141950133","heartbeat_back_addr":"172.21.15.119:6809/2141950133","heartbeat_front_addr":"172.21.15.119:6807/2141950133","state":["exists","up"]},{"osd":1,"uuid":"36489a6a-5278-4802-9525-9871521722e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6801","nonce":3390875370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6803","nonce":3390875370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6807","nonce":3390875370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6805","nonce":3390875370}]},"public_addr":"172.21.15.153:6801/3390875370","cluster_addr":"172.21.15.153:6803/3390875370","heartbeat_back_addr":"172.21.15.153:6807/3390875370","heartbeat_front_addr":"172.21.15.153:6805/3390875370","state":["exists","up"]},{"osd":2,"uuid":"a937ad26-5e9b-4c8a-b65a-626a8feda6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6801","nonce":708101637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6803","nonce":708101637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6807","nonce":708101637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6805","nonce":708101637}]},"public_addr":"172.21.15.159:6801/708101637","cluster_addr":"172.21.15.159:6803/708101637","heartbeat_back_addr":"172.21.15.159:6807/708101637","heartbeat_front_addr":"172.21.15.159:6805/708101637","state":["exists","up"]},{"osd":3,"uuid":"26ef8300-aa07-48f6-b31d-8b0f2217ffcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6811","nonce":3228047257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6813","nonce":3228047257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6817","nonce":3228047257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6815","nonce":3228047257}]},"public_addr":"172.21.15.119:6811/3228047257","cluster_addr":"172.21.15.119:6813/3228047257","heartbeat_back_addr":"172.21.15.119:6817/3228047257","heartbeat_front_addr":"172.21.15.119:6815/3228047257","state":["exists","up"]},{"osd":4,"uuid":"000ee4b9-9896-4901-8cc3-c85fdfc8cbaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6809","nonce":4021998299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6811","nonce":4021998299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6815","nonce":4021998299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6813","nonce":4021998299}]},"public_addr":"172.21.15.153:6809/4021998299","cluster_addr":"172.21.15.153:6811/4021998299","heartbeat_back_addr":"172.21.15.153:6815/4021998299","heartbeat_front_addr":"172.21.15.153:6813/4021998299","state":["exists","up"]},{"osd":5,"uuid":"8b2fc167-262d-474f-89f3-47bd29424421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6809","nonce":1832540768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6811","nonce":1832540768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6815","nonce":1832540768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6813","nonce":1832540768}]},"public_addr":"172.21.15.159:6809/1832540768","cluster_addr":"172.21.15.159:6811/1832540768","heartbeat_back_addr":"172.21.15.159:6815/1832540768","heartbeat_front_addr":"172.21.15.159:6813/1832540768","state":["exists","up"]},{"osd":6,"uuid":"f9aa6749-46cf-4b2a-8114-cbd034219d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162436840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162436840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6825","nonce":1162436840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162436840}]},"public_addr":"172.21.15.119:6819/1162436840","cluster_addr":"172.21.15.119:6821/1162436840","heartbeat_back_addr":"172.21.15.119:6825/1162436840","heartbeat_front_addr":"172.21.15.119:6823/1162436840","state":["exists","up"]},{"osd":7,"uuid":"39216361-e1c4-4fef-86cb-a4707656b547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6817","nonce":3254653457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6819","nonce":3254653457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6823","nonce":3254653457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6821","nonce":3254653457}]},"public_addr":"172.21.15.153:6817/3254653457","cluster_addr":"172.21.15.153:6819/3254653457","heartbeat_back_addr":"172.21.15.153:6823/3254653457","heartbeat_front_addr":"172.21.15.153:6821/3254653457","state":["exists","up"]},{"osd":8,"uuid":"cceb6ebc-7c02-48ea-9d16-b7a016dd449b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6817","nonce":2600946380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6819","nonce":2600946380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6823","nonce":2600946380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6821","nonce":2600946380}]},"public_addr":"172.21.15.159:6817/2600946380","cluster_addr":"172.21.15.159:6819/2600946380","heartbeat_back_addr":"172.21.15.159:6823/2600946380","heartbeat_front_addr":"172.21.15.159:6821/2600946380","state":["exists","up"]},{"osd":9,"uuid":"f337a2e4-e065-4a9f-91d9-e3615de2d6f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6827","nonce":1940737285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6829","nonce":1940737285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6833","nonce":1940737285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6831","nonce":1940737285}]},"public_addr":"172.21.15.119:6827/1940737285","cluster_addr":"172.21.15.119:6829/1940737285","heartbeat_back_addr":"172.21.15.119:6833/1940737285","heartbeat_front_addr":"172.21.15.119:6831/1940737285","state":["exists","up"]},{"osd":10,"uuid":"3a3bc347-85f4-4878-9797-d9da96137e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6825","nonce":663834864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6827","nonce":663834864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6831","nonce":663834864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6829","nonce":663834864}]},"public_addr":"172.21.15.153:6825/663834864","cluster_addr":"172.21.15.153:6827/663834864","heartbeat_back_addr":"172.21.15.153:6831/663834864","heartbeat_front_addr":"172.21.15.153:6829/663834864","state":["exists","up"]},{"osd":11,"uuid":"a0fc5909-279d-4172-977c-2dc2da0a7e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6825","nonce":1262216348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6827","nonce":1262216348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6831","nonce":1262216348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6829","nonce":1262216348}]},"public_addr":"172.21.15.159:6825/1262216348","cluster_addr":"172.21.15.159:6827/1262216348","heartbeat_back_addr":"172.21.15.159:6831/1262216348","heartbeat_front_addr":"172.21.15.159:6829/1262216348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:37.488109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:59.543060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:21.381585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:47.112729+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:09.732150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:30.893729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:56.678710+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:19.004256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:40.388818+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:06.391734+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:28.394053+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:49.634348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1628639392":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6801/1355617847":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6800/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/362307836":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/128964558":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/2278813554":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/1411933614":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/1685698277":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6800/1355617847":"2023-12-08T10:11:00.585565+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T10:16:57.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:57.201+0000 7f084b4d1700 1 -- 172.21.15.119:0/707371637 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f08300a1b10 msgr2=0x7f08300a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:57.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:57.201+0000 7f084b4d1700 1 --2- 172.21.15.119:0/707371637 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f08300a1b10 0x7f08300a3fd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f08440791c0 tx=0x7f0834009260 comp rx=0 tx=0).stop 2023-12-07T10:16:57.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:57.202+0000 7f084b4d1700 1 -- 172.21.15.119:0/707371637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08441016c0 msgr2=0x7f084419f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:57.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:57.202+0000 7f084b4d1700 1 --2- 172.21.15.119:0/707371637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08441016c0 0x7f084419f870 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0840005770 tx=0x7f08400057a0 comp rx=0 tx=0).stop 2023-12-07T10:16:57.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:57.202+0000 7f084b4d1700 1 -- 172.21.15.119:0/707371637 shutdown_connections 2023-12-07T10:16:57.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:57.202+0000 7f084b4d1700 1 --2- 172.21.15.119:0/707371637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0844107730 0x7f0844199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:57.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:57.202+0000 7f084b4d1700 1 --2- 172.21.15.119:0/707371637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08441016c0 0x7f084419f870 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:57.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:57.202+0000 7f084b4d1700 1 --2- 172.21.15.119:0/707371637 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f08300a1b10 0x7f08300a3fd0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:57.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:57.202+0000 7f084b4d1700 1 --2- 172.21.15.119:0/707371637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0844102080 0x7f084419fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:57.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:57.202+0000 7f084b4d1700 1 -- 172.21.15.119:0/707371637 >> 172.21.15.119:0/707371637 conn(0x7f0844078800 msgr2=0x7f084410cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:16:57.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:57.202+0000 7f084b4d1700 1 -- 172.21.15.119:0/707371637 shutdown_connections 2023-12-07T10:16:57.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:57.202+0000 7f084b4d1700 1 -- 172.21.15.119:0/707371637 wait complete. 2023-12-07T10:16:57.656 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-07T10:13:25.194366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-07T10:16:57.656 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph osd pool get .mgr pg_num 2023-12-07T10:16:57.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:57 smithi119 ceph-mon[97908]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:16:57.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:57 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/707371637' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:16:58.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:57 smithi159 ceph-mon[134726]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:16:58.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:57 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/707371637' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:16:58.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:57 smithi153 ceph-mon[133625]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:16:58.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:57 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/707371637' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:16:58.430 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:16:59.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.816+0000 7fbd29edd700 1 -- 172.21.15.119:0/3100784217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd241016c0 msgr2=0x7fbd24101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:59.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.816+0000 7fbd29edd700 1 --2- 172.21.15.119:0/3100784217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd241016c0 0x7fbd24101b40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009990 tx=0x7fbd18009ca0 comp rx=0 tx=0).stop 2023-12-07T10:16:59.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.817+0000 7fbd29edd700 1 -- 172.21.15.119:0/3100784217 shutdown_connections 2023-12-07T10:16:59.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.817+0000 7fbd29edd700 1 --2- 172.21.15.119:0/3100784217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd24102080 0x7fbd2410e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:59.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.817+0000 7fbd29edd700 1 --2- 172.21.15.119:0/3100784217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd24107730 0x7fbd24107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:59.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.817+0000 7fbd29edd700 1 --2- 172.21.15.119:0/3100784217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd241016c0 0x7fbd24101b40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:59.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.817+0000 7fbd29edd700 1 -- 172.21.15.119:0/3100784217 >> 172.21.15.119:0/3100784217 conn(0x7fbd24078800 msgr2=0x7fbd24078c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:16:59.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.817+0000 7fbd29edd700 1 -- 172.21.15.119:0/3100784217 shutdown_connections 2023-12-07T10:16:59.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.817+0000 7fbd29edd700 1 -- 172.21.15.119:0/3100784217 wait complete. 2023-12-07T10:16:59.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.818+0000 7fbd29edd700 1 Processor -- start 2023-12-07T10:16:59.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.818+0000 7fbd29edd700 1 -- start start 2023-12-07T10:16:59.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.818+0000 7fbd29edd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd241016c0 0x7fbd24197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:16:59.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.818+0000 7fbd29edd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd24102080 0x7fbd24197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:16:59.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.818+0000 7fbd29edd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd24107730 0x7fbd2419c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:16:59.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.818+0000 7fbd29edd700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2419c920 con 0x7fbd241016c0 2023-12-07T10:16:59.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.818+0000 7fbd29edd700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2419ca90 con 0x7fbd24102080 2023-12-07T10:16:59.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.818+0000 7fbd29edd700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2419cc50 con 0x7fbd24107730 2023-12-07T10:16:59.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.818+0000 7fbd237fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd241016c0 0x7fbd24197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:16:59.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.818+0000 7fbd237fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd241016c0 0x7fbd24197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55262/0 (socket says 172.21.15.119:55262) 2023-12-07T10:16:59.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.818+0000 7fbd237fe700 1 -- 172.21.15.119:0/68505341 learned_addr learned my addr 172.21.15.119:0/68505341 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:16:59.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.819+0000 7fbd22ffd700 1 --2- 172.21.15.119:0/68505341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd24102080 0x7fbd24197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:16:59.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.819+0000 7fbd23fff700 1 --2- 172.21.15.119:0/68505341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd24107730 0x7fbd2419c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:16:59.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.819+0000 7fbd237fe700 1 -- 172.21.15.119:0/68505341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd24107730 msgr2=0x7fbd2419c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:59.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.819+0000 7fbd237fe700 1 --2- 172.21.15.119:0/68505341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd24107730 0x7fbd2419c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:59.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.819+0000 7fbd237fe700 1 -- 172.21.15.119:0/68505341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd24102080 msgr2=0x7fbd24197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:16:59.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.819+0000 7fbd237fe700 1 --2- 172.21.15.119:0/68505341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd24102080 0x7fbd24197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:16:59.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.819+0000 7fbd237fe700 1 -- 172.21.15.119:0/68505341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd180095f0 con 0x7fbd241016c0 2023-12-07T10:16:59.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.819+0000 7fbd237fe700 1 --2- 172.21.15.119:0/68505341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd241016c0 0x7fbd24197a30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400d950 tx=0x7fbd1400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:16:59.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.819+0000 7fbd20ff9700 1 -- 172.21.15.119:0/68505341 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1400f890 con 0x7fbd241016c0 2023-12-07T10:16:59.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.819+0000 7fbd20ff9700 1 -- 172.21.15.119:0/68505341 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd14010460 con 0x7fbd241016c0 2023-12-07T10:16:59.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.819+0000 7fbd29edd700 1 -- 172.21.15.119:0/68505341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd2419cf30 con 0x7fbd241016c0 2023-12-07T10:16:59.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.819+0000 7fbd20ff9700 1 -- 172.21.15.119:0/68505341 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1400e610 con 0x7fbd241016c0 2023-12-07T10:16:59.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.820+0000 7fbd29edd700 1 -- 172.21.15.119:0/68505341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd241a47d0 con 0x7fbd241016c0 2023-12-07T10:16:59.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.821+0000 7fbd29edd700 1 -- 172.21.15.119:0/68505341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd24109bc0 con 0x7fbd241016c0 2023-12-07T10:16:59.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.822+0000 7fbd20ff9700 1 -- 172.21.15.119:0/68505341 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fbd140108d0 con 0x7fbd241016c0 2023-12-07T10:16:59.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.824+0000 7fbd20ff9700 1 --2- 172.21.15.119:0/68505341 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fbd0c0a1b10 0x7fbd0c0a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:16:59.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.824+0000 7fbd20ff9700 1 -- 172.21.15.119:0/68505341 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbd140ccf30 con 0x7fbd241016c0 2023-12-07T10:16:59.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.824+0000 7fbd22ffd700 1 --2- 172.21.15.119:0/68505341 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fbd0c0a1b10 0x7fbd0c0a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:16:59.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.825+0000 7fbd22ffd700 1 --2- 172.21.15.119:0/68505341 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fbd0c0a1b10 0x7fbd0c0a3fd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbd24198f90 tx=0x7fbd18005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:16:59.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:16:59.828+0000 7fbd20ff9700 1 -- 172.21.15.119:0/68505341 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fbd14096750 con 0x7fbd241016c0 2023-12-07T10:16:59.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:16:59 smithi119 ceph-mon[97908]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:00.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:16:59 smithi159 ceph-mon[134726]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:00.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:00.026+0000 7fbd29edd700 1 -- 172.21.15.119:0/68505341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbd24198d40 con 0x7fbd241016c0 2023-12-07T10:17:00.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:00.026+0000 7fbd20ff9700 1 -- 172.21.15.119:0/68505341 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbd14099d70 con 0x7fbd241016c0 2023-12-07T10:17:00.029 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-12-07T10:17:00.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:16:59 smithi153 ceph-mon[133625]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:00.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:00.028+0000 7fbd29edd700 1 -- 172.21.15.119:0/68505341 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fbd0c0a1b10 msgr2=0x7fbd0c0a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:00.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:00.028+0000 7fbd29edd700 1 --2- 172.21.15.119:0/68505341 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fbd0c0a1b10 0x7fbd0c0a3fd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbd24198f90 tx=0x7fbd18005fd0 comp rx=0 tx=0).stop 2023-12-07T10:17:00.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:00.028+0000 7fbd29edd700 1 -- 172.21.15.119:0/68505341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd241016c0 msgr2=0x7fbd24197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:00.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:00.028+0000 7fbd29edd700 1 --2- 172.21.15.119:0/68505341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd241016c0 0x7fbd24197a30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400d950 tx=0x7fbd1400dc60 comp rx=0 tx=0).stop 2023-12-07T10:17:00.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:00.029+0000 7fbd29edd700 1 -- 172.21.15.119:0/68505341 shutdown_connections 2023-12-07T10:17:00.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:00.029+0000 7fbd29edd700 1 --2- 172.21.15.119:0/68505341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd24107730 0x7fbd2419c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:00.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:00.029+0000 7fbd29edd700 1 --2- 172.21.15.119:0/68505341 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fbd0c0a1b10 0x7fbd0c0a3fd0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:00.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:00.029+0000 7fbd29edd700 1 --2- 172.21.15.119:0/68505341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd241016c0 0x7fbd24197a30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:00.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:00.029+0000 7fbd29edd700 1 --2- 172.21.15.119:0/68505341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd24102080 0x7fbd24197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:00.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:00.029+0000 7fbd29edd700 1 -- 172.21.15.119:0/68505341 >> 172.21.15.119:0/68505341 conn(0x7fbd24078800 msgr2=0x7fbd24100390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:00.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:00.029+0000 7fbd29edd700 1 -- 172.21.15.119:0/68505341 shutdown_connections 2023-12-07T10:17:00.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:00.029+0000 7fbd29edd700 1 -- 172.21.15.119:0/68505341 wait complete. 2023-12-07T10:17:00.414 INFO:tasks.cephadm:Adding mds.a on smithi119 2023-12-07T10:17:00.414 INFO:tasks.cephadm:Adding mds.d on smithi119 2023-12-07T10:17:00.414 INFO:tasks.cephadm:Adding mds.g on smithi119 2023-12-07T10:17:00.414 INFO:tasks.cephadm:Adding mds.j on smithi119 2023-12-07T10:17:00.414 INFO:tasks.cephadm:Adding mds.b on smithi153 2023-12-07T10:17:00.414 INFO:tasks.cephadm:Adding mds.e on smithi153 2023-12-07T10:17:00.414 INFO:tasks.cephadm:Adding mds.h on smithi153 2023-12-07T10:17:00.415 INFO:tasks.cephadm:Adding mds.k on smithi153 2023-12-07T10:17:00.415 INFO:tasks.cephadm:Adding mds.c on smithi159 2023-12-07T10:17:00.415 INFO:tasks.cephadm:Adding mds.f on smithi159 2023-12-07T10:17:00.415 INFO:tasks.cephadm:Adding mds.i on smithi159 2023-12-07T10:17:00.415 INFO:tasks.cephadm:Adding mds.l on smithi159 2023-12-07T10:17:00.415 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph orch apply mds all '12;smithi119=a;smithi119=d;smithi119=g;smithi119=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l' 2023-12-07T10:17:00.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:00 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/68505341' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:17:01.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:00 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/68505341' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:17:01.025 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.c/config 2023-12-07T10:17:01.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:00 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/68505341' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:17:01.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:01 smithi119 ceph-mon[97908]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:02.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:01 smithi159 ceph-mon[134726]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:02.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:01 smithi153 ceph-mon[133625]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:02.212 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.210+0000 7fe0484b9700 1 -- 172.21.15.159:0/1601584183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe040100550 msgr2=0x7fe040100970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:02.213 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.210+0000 7fe0484b9700 1 --2- 172.21.15.159:0/1601584183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe040100550 0x7fe040100970 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe030009990 tx=0x7fe030009ca0 comp rx=0 tx=0).stop 2023-12-07T10:17:02.213 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.210+0000 7fe0484b9700 1 -- 172.21.15.159:0/1601584183 shutdown_connections 2023-12-07T10:17:02.213 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.210+0000 7fe0484b9700 1 --2- 172.21.15.159:0/1601584183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0401020f0 0x7fe0401089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:02.213 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.210+0000 7fe0484b9700 1 --2- 172.21.15.159:0/1601584183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe040100550 0x7fe040100970 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:02.213 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.210+0000 7fe0484b9700 1 --2- 172.21.15.159:0/1601584183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe040101750 0x7fe040101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:02.213 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.210+0000 7fe0484b9700 1 -- 172.21.15.159:0/1601584183 >> 172.21.15.159:0/1601584183 conn(0x7fe0400fbb10 msgr2=0x7fe0400fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:02.213 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.210+0000 7fe0484b9700 1 -- 172.21.15.159:0/1601584183 shutdown_connections 2023-12-07T10:17:02.213 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.210+0000 7fe0484b9700 1 -- 172.21.15.159:0/1601584183 wait complete. 2023-12-07T10:17:02.215 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.211+0000 7fe0484b9700 1 Processor -- start 2023-12-07T10:17:02.215 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.211+0000 7fe0484b9700 1 -- start start 2023-12-07T10:17:02.215 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.211+0000 7fe0484b9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe040100550 0x7fe040069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:02.215 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.211+0000 7fe0484b9700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe040101750 0x7fe04019c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:02.215 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.211+0000 7fe0484b9700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0401020f0 0x7fe04019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:02.216 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.211+0000 7fe0484b9700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04019f0d0 con 0x7fe0401020f0 2023-12-07T10:17:02.216 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.211+0000 7fe0484b9700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04019f240 con 0x7fe040100550 2023-12-07T10:17:02.216 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.211+0000 7fe0484b9700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04019f400 con 0x7fe040101750 2023-12-07T10:17:02.216 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.211+0000 7fe045a54700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe040101750 0x7fe04019c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:02.216 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.211+0000 7fe045a54700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe040101750 0x7fe04019c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42960/0 (socket says 172.21.15.159:42960) 2023-12-07T10:17:02.216 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.211+0000 7fe045a54700 1 -- 172.21.15.159:0/2329057962 learned_addr learned my addr 172.21.15.159:0/2329057962 (peer_addr_for_me v2:172.21.15.159:0/0) 2023-12-07T10:17:02.216 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.211+0000 7fe046255700 1 --2- 172.21.15.159:0/2329057962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe040100550 0x7fe040069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:02.216 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.211+0000 7fe045a54700 1 -- 172.21.15.159:0/2329057962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe040100550 msgr2=0x7fe040069ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:02.216 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.211+0000 7fe045a54700 1 --2- 172.21.15.159:0/2329057962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe040100550 0x7fe040069ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:02.216 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.211+0000 7fe045a54700 1 -- 172.21.15.159:0/2329057962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0401020f0 msgr2=0x7fe04019e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:02.216 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.211+0000 7fe045a54700 1 --2- 172.21.15.159:0/2329057962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0401020f0 0x7fe04019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:02.216 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.211+0000 7fe045a54700 1 -- 172.21.15.159:0/2329057962 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0300095f0 con 0x7fe040101750 2023-12-07T10:17:02.216 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.211+0000 7fe045a54700 1 --2- 172.21.15.159:0/2329057962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe040101750 0x7fe04019c360 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe03c00d950 tx=0x7fe03c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:02.217 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.212+0000 7fe0377fe700 1 -- 172.21.15.159:0/2329057962 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c009940 con 0x7fe040101750 2023-12-07T10:17:02.217 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.212+0000 7fe0377fe700 1 -- 172.21.15.159:0/2329057962 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe03c010460 con 0x7fe040101750 2023-12-07T10:17:02.217 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.212+0000 7fe0377fe700 1 -- 172.21.15.159:0/2329057962 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c00f620 con 0x7fe040101750 2023-12-07T10:17:02.217 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.212+0000 7fe0484b9700 1 -- 172.21.15.159:0/2329057962 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe04019f6e0 con 0x7fe040101750 2023-12-07T10:17:02.217 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.212+0000 7fe0484b9700 1 -- 172.21.15.159:0/2329057962 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe04019fb90 con 0x7fe040101750 2023-12-07T10:17:02.217 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.213+0000 7fe0484b9700 1 -- 172.21.15.159:0/2329057962 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe040107a30 con 0x7fe040101750 2023-12-07T10:17:02.218 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.214+0000 7fe0377fe700 1 -- 172.21.15.159:0/2329057962 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fe03c0108d0 con 0x7fe040101750 2023-12-07T10:17:02.218 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.214+0000 7fe0377fe700 1 --2- 172.21.15.159:0/2329057962 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe02c0a1b10 0x7fe02c0a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:02.218 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.215+0000 7fe0377fe700 1 -- 172.21.15.159:0/2329057962 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe03c0ccf70 con 0x7fe040101750 2023-12-07T10:17:02.218 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.215+0000 7fe046255700 1 --2- 172.21.15.159:0/2329057962 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe02c0a1b10 0x7fe02c0a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:02.218 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.215+0000 7fe046255700 1 --2- 172.21.15.159:0/2329057962 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe02c0a1b10 0x7fe02c0a3fd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe030003790 tx=0x7fe030005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:02.221 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.219+0000 7fe0377fe700 1 -- 172.21.15.159:0/2329057962 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe03c1002f0 con 0x7fe040101750 2023-12-07T10:17:02.418 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.416+0000 7fe0484b9700 1 -- 172.21.15.159:0/2329057962 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi119=a;smithi119=d;smithi119=g;smithi119=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fe04002d5b0 con 0x7fe02c0a1b10 2023-12-07T10:17:02.427 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.425+0000 7fe0377fe700 1 -- 172.21.15.159:0/2329057962 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fe04002d5b0 con 0x7fe02c0a1b10 2023-12-07T10:17:02.427 INFO:teuthology.orchestra.run.smithi159.stdout:Scheduled mds.all update... 2023-12-07T10:17:02.429 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.427+0000 7fe0484b9700 1 -- 172.21.15.159:0/2329057962 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe02c0a1b10 msgr2=0x7fe02c0a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:02.429 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.427+0000 7fe0484b9700 1 --2- 172.21.15.159:0/2329057962 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe02c0a1b10 0x7fe02c0a3fd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe030003790 tx=0x7fe030005fd0 comp rx=0 tx=0).stop 2023-12-07T10:17:02.429 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.427+0000 7fe0484b9700 1 -- 172.21.15.159:0/2329057962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe040101750 msgr2=0x7fe04019c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:02.429 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.427+0000 7fe0484b9700 1 --2- 172.21.15.159:0/2329057962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe040101750 0x7fe04019c360 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe03c00d950 tx=0x7fe03c00dc60 comp rx=0 tx=0).stop 2023-12-07T10:17:02.429 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.427+0000 7fe0484b9700 1 -- 172.21.15.159:0/2329057962 shutdown_connections 2023-12-07T10:17:02.429 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.427+0000 7fe0484b9700 1 --2- 172.21.15.159:0/2329057962 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe02c0a1b10 0x7fe02c0a3fd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:02.429 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.427+0000 7fe0484b9700 1 --2- 172.21.15.159:0/2329057962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0401020f0 0x7fe04019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:02.430 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.427+0000 7fe0484b9700 1 --2- 172.21.15.159:0/2329057962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe040100550 0x7fe040069ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:02.430 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.427+0000 7fe0484b9700 1 --2- 172.21.15.159:0/2329057962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe040101750 0x7fe04019c360 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:02.430 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.427+0000 7fe0484b9700 1 -- 172.21.15.159:0/2329057962 >> 172.21.15.159:0/2329057962 conn(0x7fe0400fbb10 msgr2=0x7fe040104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:02.430 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.427+0000 7fe0484b9700 1 -- 172.21.15.159:0/2329057962 shutdown_connections 2023-12-07T10:17:02.430 INFO:teuthology.orchestra.run.smithi159.stderr:2023-12-07T10:17:02.427+0000 7fe0484b9700 1 -- 172.21.15.159:0/2329057962 wait complete. 2023-12-07T10:17:02.773 DEBUG:teuthology.orchestra.run.smithi119:mds.a> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.a.service 2023-12-07T10:17:02.779 DEBUG:teuthology.orchestra.run.smithi119:mds.d> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.d.service 2023-12-07T10:17:02.825 DEBUG:teuthology.orchestra.run.smithi119:mds.g> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.g.service 2023-12-07T10:17:02.857 INFO:journalctl@ceph.mds.a.smithi119.stdout:-- Logs begin at Thu 2023-12-07 09:53:13 UTC. -- 2023-12-07T10:17:02.861 DEBUG:teuthology.orchestra.run.smithi119:mds.j> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.j.service 2023-12-07T10:17:02.897 INFO:journalctl@ceph.mds.d.smithi119.stdout:-- Logs begin at Thu 2023-12-07 09:53:13 UTC. -- 2023-12-07T10:17:02.900 DEBUG:teuthology.orchestra.run.smithi153:mds.b> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.b.service 2023-12-07T10:17:02.903 DEBUG:teuthology.orchestra.run.smithi153:mds.e> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.e.service 2023-12-07T10:17:02.931 INFO:journalctl@ceph.mds.g.smithi119.stdout:-- Logs begin at Thu 2023-12-07 09:53:13 UTC. -- 2023-12-07T10:17:02.935 INFO:journalctl@ceph.mds.b.smithi153.stdout:-- Logs begin at Thu 2023-12-07 09:49:13 UTC. -- 2023-12-07T10:17:02.936 DEBUG:teuthology.orchestra.run.smithi153:mds.h> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.h.service 2023-12-07T10:17:02.962 INFO:journalctl@ceph.mds.e.smithi153.stdout:-- Logs begin at Thu 2023-12-07 09:49:13 UTC. -- 2023-12-07T10:17:02.963 DEBUG:teuthology.orchestra.run.smithi153:mds.k> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.k.service 2023-12-07T10:17:02.969 INFO:journalctl@ceph.mds.j.smithi119.stdout:-- Logs begin at Thu 2023-12-07 09:53:13 UTC. -- 2023-12-07T10:17:02.991 INFO:journalctl@ceph.mds.h.smithi153.stdout:-- Logs begin at Thu 2023-12-07 09:49:13 UTC. -- 2023-12-07T10:17:02.991 DEBUG:teuthology.orchestra.run.smithi159:mds.c> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.c.service 2023-12-07T10:17:02.994 DEBUG:teuthology.orchestra.run.smithi159:mds.f> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.f.service 2023-12-07T10:17:03.017 INFO:journalctl@ceph.mds.k.smithi153.stdout:-- Logs begin at Thu 2023-12-07 09:49:13 UTC. -- 2023-12-07T10:17:03.023 INFO:journalctl@ceph.mds.c.smithi159.stdout:-- Logs begin at Thu 2023-12-07 09:49:21 UTC. -- 2023-12-07T10:17:03.024 DEBUG:teuthology.orchestra.run.smithi159:mds.i> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.i.service 2023-12-07T10:17:03.050 INFO:journalctl@ceph.mds.f.smithi159.stdout:-- Logs begin at Thu 2023-12-07 09:49:21 UTC. -- 2023-12-07T10:17:03.051 DEBUG:teuthology.orchestra.run.smithi159:mds.l> sudo journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.l.service 2023-12-07T10:17:03.076 INFO:journalctl@ceph.mds.i.smithi159.stdout:-- Logs begin at Thu 2023-12-07 09:49:21 UTC. -- 2023-12-07T10:17:03.077 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-12-07T10:17:03.078 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-07T10:17:03.103 INFO:journalctl@ceph.mds.l.smithi159.stdout:-- Logs begin at Thu 2023-12-07 09:49:21 UTC. -- 2023-12-07T10:17:03.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.311+0000 7f2d53fff700 1 -- 172.21.15.119:0/1904131958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d4c1030f0 msgr2=0x7f2d4c1099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:03.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.311+0000 7f2d53fff700 1 --2- 172.21.15.119:0/1904131958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d4c1030f0 0x7f2d4c1099a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400b0d0 tx=0x7f2d4400b3e0 comp rx=0 tx=0).stop 2023-12-07T10:17:03.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.312+0000 7f2d53fff700 1 -- 172.21.15.119:0/1904131958 shutdown_connections 2023-12-07T10:17:03.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.312+0000 7f2d53fff700 1 --2- 172.21.15.119:0/1904131958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d4c1030f0 0x7f2d4c1099a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:03.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.312+0000 7f2d53fff700 1 --2- 172.21.15.119:0/1904131958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d4c101560 0x7f2d4c101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:03.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.312+0000 7f2d53fff700 1 --2- 172.21.15.119:0/1904131958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d4c102750 0x7f2d4c102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:03.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.312+0000 7f2d53fff700 1 -- 172.21.15.119:0/1904131958 >> 172.21.15.119:0/1904131958 conn(0x7f2d4c0fcab0 msgr2=0x7f2d4c0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:03.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.312+0000 7f2d53fff700 1 -- 172.21.15.119:0/1904131958 shutdown_connections 2023-12-07T10:17:03.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.312+0000 7f2d53fff700 1 -- 172.21.15.119:0/1904131958 wait complete. 2023-12-07T10:17:03.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.313+0000 7f2d53fff700 1 Processor -- start 2023-12-07T10:17:03.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.313+0000 7f2d53fff700 1 -- start start 2023-12-07T10:17:03.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.314+0000 7f2d53fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d4c101560 0x7f2d4c1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:03.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.314+0000 7f2d53fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d4c102750 0x7f2d4c199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:03.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.314+0000 7f2d53fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d4c1030f0 0x7f2d4c1a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:03.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.314+0000 7f2d53fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c1a1310 con 0x7f2d4c101560 2023-12-07T10:17:03.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.314+0000 7f2d53fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c1a1450 con 0x7f2d4c102750 2023-12-07T10:17:03.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.314+0000 7f2d53fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c1a15e0 con 0x7f2d4c1030f0 2023-12-07T10:17:03.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.314+0000 7f2d51d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d4c101560 0x7f2d4c1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:03.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.314+0000 7f2d51d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d4c101560 0x7f2d4c1997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37068/0 (socket says 172.21.15.119:37068) 2023-12-07T10:17:03.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.314+0000 7f2d5259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d4c1030f0 0x7f2d4c1a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:03.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.314+0000 7f2d51d9b700 1 -- 172.21.15.119:0/247473518 learned_addr learned my addr 172.21.15.119:0/247473518 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:03.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.314+0000 7f2d5159a700 1 --2- 172.21.15.119:0/247473518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d4c102750 0x7f2d4c199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:03.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.314+0000 7f2d5259c700 1 -- 172.21.15.119:0/247473518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d4c102750 msgr2=0x7f2d4c199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:03.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.314+0000 7f2d5259c700 1 --2- 172.21.15.119:0/247473518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d4c102750 0x7f2d4c199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:03.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.314+0000 7f2d5259c700 1 -- 172.21.15.119:0/247473518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d4c101560 msgr2=0x7f2d4c1997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:03.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.314+0000 7f2d5259c700 1 --2- 172.21.15.119:0/247473518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d4c101560 0x7f2d4c1997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:03.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.314+0000 7f2d5259c700 1 -- 172.21.15.119:0/247473518 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d44009d00 con 0x7f2d4c1030f0 2023-12-07T10:17:03.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.315+0000 7f2d5259c700 1 --2- 172.21.15.119:0/247473518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d4c1030f0 0x7f2d4c1a0d40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2d44007b60 tx=0x7f2d44008e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:03.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.315+0000 7f2d42ffd700 1 -- 172.21.15.119:0/247473518 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4400e040 con 0x7f2d4c1030f0 2023-12-07T10:17:03.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.315+0000 7f2d42ffd700 1 -- 172.21.15.119:0/247473518 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d44005070 con 0x7f2d4c1030f0 2023-12-07T10:17:03.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.315+0000 7f2d53fff700 1 -- 172.21.15.119:0/247473518 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d4c1a1860 con 0x7f2d4c1030f0 2023-12-07T10:17:03.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.315+0000 7f2d42ffd700 1 -- 172.21.15.119:0/247473518 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4401d6d0 con 0x7f2d4c1030f0 2023-12-07T10:17:03.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.315+0000 7f2d53fff700 1 -- 172.21.15.119:0/247473518 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d4c1a1c80 con 0x7f2d4c1030f0 2023-12-07T10:17:03.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.316+0000 7f2d53fff700 1 -- 172.21.15.119:0/247473518 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d4c068350 con 0x7f2d4c1030f0 2023-12-07T10:17:03.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.318+0000 7f2d42ffd700 1 -- 172.21.15.119:0/247473518 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f2d440051e0 con 0x7f2d4c1030f0 2023-12-07T10:17:03.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.319+0000 7f2d42ffd700 1 --2- 172.21.15.119:0/247473518 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2d380a1b10 0x7f2d380a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:03.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.319+0000 7f2d42ffd700 1 -- 172.21.15.119:0/247473518 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2d440cdfc0 con 0x7f2d4c1030f0 2023-12-07T10:17:03.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.319+0000 7f2d51d9b700 1 --2- 172.21.15.119:0/247473518 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2d380a1b10 0x7f2d380a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:03.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.320+0000 7f2d51d9b700 1 --2- 172.21.15.119:0/247473518 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2d380a1b10 0x7f2d380a3fd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2d48005950 tx=0x7f2d48009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:03.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.324+0000 7f2d42ffd700 1 -- 172.21.15.119:0/247473518 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2d44097900 con 0x7f2d4c1030f0 2023-12-07T10:17:03.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.532+0000 7f2d53fff700 1 -- 172.21.15.119:0/247473518 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d4c10cdd0 con 0x7f2d4c1030f0 2023-12-07T10:17:03.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.533+0000 7f2d42ffd700 1 -- 172.21.15.119:0/247473518 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18492 (secure 0 0 0) 0x7f2d4409af20 con 0x7f2d4c1030f0 2023-12-07T10:17:03.535 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:17:03.535 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":81,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","created":"2023-12-07T10:09:59.945863+0000","modified":"2023-12-07T10:16:52.466300+0000","last_up_change":"2023-12-07T10:16:50.929427+0000","last_in_change":"2023-12-07T10:16:33.558158+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T10:13:25.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"56b5818e-e625-40f1-8838-3ddcf2d34ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6803","nonce":2141950133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6805","nonce":2141950133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6809","nonce":2141950133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6807","nonce":2141950133}]},"public_addr":"172.21.15.119:6803/2141950133","cluster_addr":"172.21.15.119:6805/2141950133","heartbeat_back_addr":"172.21.15.119:6809/2141950133","heartbeat_front_addr":"172.21.15.119:6807/2141950133","state":["exists","up"]},{"osd":1,"uuid":"36489a6a-5278-4802-9525-9871521722e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6801","nonce":3390875370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6803","nonce":3390875370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6807","nonce":3390875370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6805","nonce":3390875370}]},"public_addr":"172.21.15.153:6801/3390875370","cluster_addr":"172.21.15.153:6803/3390875370","heartbeat_back_addr":"172.21.15.153:6807/3390875370","heartbeat_front_addr":"172.21.15.153:6805/3390875370","state":["exists","up"]},{"osd":2,"uuid":"a937ad26-5e9b-4c8a-b65a-626a8feda6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6801","nonce":708101637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6803","nonce":708101637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6807","nonce":708101637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6805","nonce":708101637}]},"public_addr":"172.21.15.159:6801/708101637","cluster_addr":"172.21.15.159:6803/708101637","heartbeat_back_addr":"172.21.15.159:6807/708101637","heartbeat_front_addr":"172.21.15.159:6805/708101637","state":["exists","up"]},{"osd":3,"uuid":"26ef8300-aa07-48f6-b31d-8b0f2217ffcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6811","nonce":3228047257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6813","nonce":3228047257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6817","nonce":3228047257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6815","nonce":3228047257}]},"public_addr":"172.21.15.119:6811/3228047257","cluster_addr":"172.21.15.119:6813/3228047257","heartbeat_back_addr":"172.21.15.119:6817/3228047257","heartbeat_front_addr":"172.21.15.119:6815/3228047257","state":["exists","up"]},{"osd":4,"uuid":"000ee4b9-9896-4901-8cc3-c85fdfc8cbaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6809","nonce":4021998299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6811","nonce":4021998299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6815","nonce":4021998299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6813","nonce":4021998299}]},"public_addr":"172.21.15.153:6809/4021998299","cluster_addr":"172.21.15.153:6811/4021998299","heartbeat_back_addr":"172.21.15.153:6815/4021998299","heartbeat_front_addr":"172.21.15.153:6813/4021998299","state":["exists","up"]},{"osd":5,"uuid":"8b2fc167-262d-474f-89f3-47bd29424421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6809","nonce":1832540768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6811","nonce":1832540768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6815","nonce":1832540768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6813","nonce":1832540768}]},"public_addr":"172.21.15.159:6809/1832540768","cluster_addr":"172.21.15.159:6811/1832540768","heartbeat_back_addr":"172.21.15.159:6815/1832540768","heartbeat_front_addr":"172.21.15.159:6813/1832540768","state":["exists","up"]},{"osd":6,"uuid":"f9aa6749-46cf-4b2a-8114-cbd034219d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162436840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162436840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6825","nonce":1162436840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162436840}]},"public_addr":"172.21.15.119:6819/1162436840","cluster_addr":"172.21.15.119:6821/1162436840","heartbeat_back_addr":"172.21.15.119:6825/1162436840","heartbeat_front_addr":"172.21.15.119:6823/1162436840","state":["exists","up"]},{"osd":7,"uuid":"39216361-e1c4-4fef-86cb-a4707656b547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6817","nonce":3254653457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6819","nonce":3254653457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6823","nonce":3254653457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6821","nonce":3254653457}]},"public_addr":"172.21.15.153:6817/3254653457","cluster_addr":"172.21.15.153:6819/3254653457","heartbeat_back_addr":"172.21.15.153:6823/3254653457","heartbeat_front_addr":"172.21.15.153:6821/3254653457","state":["exists","up"]},{"osd":8,"uuid":"cceb6ebc-7c02-48ea-9d16-b7a016dd449b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6817","nonce":2600946380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6819","nonce":2600946380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6823","nonce":2600946380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6821","nonce":2600946380}]},"public_addr":"172.21.15.159:6817/2600946380","cluster_addr":"172.21.15.159:6819/2600946380","heartbeat_back_addr":"172.21.15.159:6823/2600946380","heartbeat_front_addr":"172.21.15.159:6821/2600946380","state":["exists","up"]},{"osd":9,"uuid":"f337a2e4-e065-4a9f-91d9-e3615de2d6f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6827","nonce":1940737285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6829","nonce":1940737285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6833","nonce":1940737285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6831","nonce":1940737285}]},"public_addr":"172.21.15.119:6827/1940737285","cluster_addr":"172.21.15.119:6829/1940737285","heartbeat_back_addr":"172.21.15.119:6833/1940737285","heartbeat_front_addr":"172.21.15.119:6831/1940737285","state":["exists","up"]},{"osd":10,"uuid":"3a3bc347-85f4-4878-9797-d9da96137e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6825","nonce":663834864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6827","nonce":663834864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6831","nonce":663834864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6829","nonce":663834864}]},"public_addr":"172.21.15.153:6825/663834864","cluster_addr":"172.21.15.153:6827/663834864","heartbeat_back_addr":"172.21.15.153:6831/663834864","heartbeat_front_addr":"172.21.15.153:6829/663834864","state":["exists","up"]},{"osd":11,"uuid":"a0fc5909-279d-4172-977c-2dc2da0a7e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6825","nonce":1262216348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6827","nonce":1262216348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6831","nonce":1262216348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6829","nonce":1262216348}]},"public_addr":"172.21.15.159:6825/1262216348","cluster_addr":"172.21.15.159:6827/1262216348","heartbeat_back_addr":"172.21.15.159:6831/1262216348","heartbeat_front_addr":"172.21.15.159:6829/1262216348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:37.488109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:59.543060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:21.381585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:47.112729+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:09.732150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:30.893729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:56.678710+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:19.004256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:40.388818+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:06.391734+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:28.394053+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:49.634348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1628639392":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6801/1355617847":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6800/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/362307836":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/128964558":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/2278813554":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/1411933614":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/1685698277":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6800/1355617847":"2023-12-08T10:11:00.585565+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T10:17:03.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.535+0000 7f2d53fff700 1 -- 172.21.15.119:0/247473518 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2d380a1b10 msgr2=0x7f2d380a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:03.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.535+0000 7f2d53fff700 1 --2- 172.21.15.119:0/247473518 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2d380a1b10 0x7f2d380a3fd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2d48005950 tx=0x7f2d48009260 comp rx=0 tx=0).stop 2023-12-07T10:17:03.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.535+0000 7f2d53fff700 1 -- 172.21.15.119:0/247473518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d4c1030f0 msgr2=0x7f2d4c1a0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:03.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.535+0000 7f2d53fff700 1 --2- 172.21.15.119:0/247473518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d4c1030f0 0x7f2d4c1a0d40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2d44007b60 tx=0x7f2d44008e40 comp rx=0 tx=0).stop 2023-12-07T10:17:03.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.535+0000 7f2d53fff700 1 -- 172.21.15.119:0/247473518 shutdown_connections 2023-12-07T10:17:03.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.535+0000 7f2d53fff700 1 --2- 172.21.15.119:0/247473518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d4c1030f0 0x7f2d4c1a0d40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:03.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.535+0000 7f2d53fff700 1 --2- 172.21.15.119:0/247473518 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2d380a1b10 0x7f2d380a3fd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:03.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.535+0000 7f2d53fff700 1 --2- 172.21.15.119:0/247473518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d4c101560 0x7f2d4c1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:03.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.535+0000 7f2d53fff700 1 --2- 172.21.15.119:0/247473518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d4c102750 0x7f2d4c199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:03.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.535+0000 7f2d53fff700 1 -- 172.21.15.119:0/247473518 >> 172.21.15.119:0/247473518 conn(0x7f2d4c0fcab0 msgr2=0x7f2d4c073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:03.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.535+0000 7f2d53fff700 1 -- 172.21.15.119:0/247473518 shutdown_connections 2023-12-07T10:17:03.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.535+0000 7f2d53fff700 1 -- 172.21.15.119:0/247473518 wait complete. 2023-12-07T10:17:03.557 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-07T10:13:25.194366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-07T10:17:03.558 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-07T10:17:03.702 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:03 smithi159 ceph-mon[134726]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi119=a;smithi119=d;smithi119=g;smithi119=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:17:03.702 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:03 smithi159 ceph-mon[134726]: Saving service mds.all spec with placement smithi119=a;smithi119=d;smithi119=g;smithi119=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l;count:12 2023-12-07T10:17:03.702 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:03 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:03.702 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:03 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:17:03.703 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:03 smithi159 ceph-mon[134726]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:03.703 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:03 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:03.703 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:03 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:17:03.703 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:03 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:03.703 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:03 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:03.703 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:03 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:03.703 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:03 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:03.703 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:03 smithi159 ceph-mon[134726]: Deploying daemon mds.l on smithi159 2023-12-07T10:17:03.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:03 smithi119 ceph-mon[97908]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi119=a;smithi119=d;smithi119=g;smithi119=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:17:03.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:03 smithi119 ceph-mon[97908]: Saving service mds.all spec with placement smithi119=a;smithi119=d;smithi119=g;smithi119=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l;count:12 2023-12-07T10:17:03.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:03 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:03.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:03 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:17:03.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:03 smithi119 ceph-mon[97908]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:03.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:03 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:03.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:03 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:17:03.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:03 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:03.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:03 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:03.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:03 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:03.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:03 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:03.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:03 smithi119 ceph-mon[97908]: Deploying daemon mds.l on smithi159 2023-12-07T10:17:03.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.756+0000 7f1c123ff700 1 -- 172.21.15.119:0/2939845486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c0c1015c0 msgr2=0x7f1c0c1019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:03.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.756+0000 7f1c123ff700 1 --2- 172.21.15.119:0/2939845486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c0c1015c0 0x7f1c0c1019e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc009990 tx=0x7f1bfc009ca0 comp rx=0 tx=0).stop 2023-12-07T10:17:03.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.756+0000 7f1c123ff700 1 -- 172.21.15.119:0/2939845486 shutdown_connections 2023-12-07T10:17:03.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.756+0000 7f1c123ff700 1 --2- 172.21.15.119:0/2939845486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0c103150 0x7f1c0c109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:03.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.756+0000 7f1c123ff700 1 --2- 172.21.15.119:0/2939845486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c0c1015c0 0x7f1c0c1019e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:03.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.756+0000 7f1c123ff700 1 --2- 172.21.15.119:0/2939845486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c0c1027b0 0x7f1c0c102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:03.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.756+0000 7f1c123ff700 1 -- 172.21.15.119:0/2939845486 >> 172.21.15.119:0/2939845486 conn(0x7f1c0c0fcb10 msgr2=0x7f1c0c0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:03.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.756+0000 7f1c123ff700 1 -- 172.21.15.119:0/2939845486 shutdown_connections 2023-12-07T10:17:03.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.756+0000 7f1c123ff700 1 -- 172.21.15.119:0/2939845486 wait complete. 2023-12-07T10:17:03.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.757+0000 7f1c123ff700 1 Processor -- start 2023-12-07T10:17:03.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.757+0000 7f1c123ff700 1 -- start start 2023-12-07T10:17:03.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.758+0000 7f1c123ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0c1015c0 0x7f1c0c1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:03.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.758+0000 7f1c123ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c0c1027b0 0x7f1c0c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:03.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.758+0000 7f1c123ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c0c103150 0x7f1c0c19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:03.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.758+0000 7f1c123ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c19f0c0 con 0x7f1c0c1015c0 2023-12-07T10:17:03.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.758+0000 7f1c123ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c19f200 con 0x7f1c0c103150 2023-12-07T10:17:03.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.758+0000 7f1c123ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c19f390 con 0x7f1c0c1027b0 2023-12-07T10:17:03.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.758+0000 7f1c0bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0c1015c0 0x7f1c0c1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:03.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.758+0000 7f1c0bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0c1015c0 0x7f1c0c1975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37090/0 (socket says 172.21.15.119:37090) 2023-12-07T10:17:03.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.758+0000 7f1c0bfff700 1 -- 172.21.15.119:0/2354910869 learned_addr learned my addr 172.21.15.119:0/2354910869 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:03.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.758+0000 7f1c0b7fe700 1 --2- 172.21.15.119:0/2354910869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c0c1027b0 0x7f1c0c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:03.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.758+0000 7f1c1099c700 1 --2- 172.21.15.119:0/2354910869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c0c103150 0x7f1c0c19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:03.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.758+0000 7f1c0bfff700 1 -- 172.21.15.119:0/2354910869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c0c1027b0 msgr2=0x7f1c0c197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:03.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.758+0000 7f1c0bfff700 1 --2- 172.21.15.119:0/2354910869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c0c1027b0 0x7f1c0c197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:03.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.758+0000 7f1c0bfff700 1 -- 172.21.15.119:0/2354910869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c0c103150 msgr2=0x7f1c0c19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:03.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.758+0000 7f1c0bfff700 1 --2- 172.21.15.119:0/2354910869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c0c103150 0x7f1c0c19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:03.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.758+0000 7f1c0bfff700 1 -- 172.21.15.119:0/2354910869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c04009c40 con 0x7f1c0c1015c0 2023-12-07T10:17:03.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.758+0000 7f1c0bfff700 1 --2- 172.21.15.119:0/2354910869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0c1015c0 0x7f1c0c1975e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc000c00 tx=0x7f1bfc009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:03.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.759+0000 7f1c097fa700 1 -- 172.21.15.119:0/2354910869 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc01d070 con 0x7f1c0c1015c0 2023-12-07T10:17:03.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.759+0000 7f1c123ff700 1 -- 172.21.15.119:0/2354910869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bfc0095f0 con 0x7f1c0c1015c0 2023-12-07T10:17:03.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.759+0000 7f1c097fa700 1 -- 172.21.15.119:0/2354910869 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bfc021410 con 0x7f1c0c1015c0 2023-12-07T10:17:03.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.759+0000 7f1c097fa700 1 -- 172.21.15.119:0/2354910869 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc017650 con 0x7f1c0c1015c0 2023-12-07T10:17:03.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.759+0000 7f1c123ff700 1 -- 172.21.15.119:0/2354910869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c0c19f8b0 con 0x7f1c0c1015c0 2023-12-07T10:17:03.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.760+0000 7f1c123ff700 1 -- 172.21.15.119:0/2354910869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c0c108ae0 con 0x7f1c0c1015c0 2023-12-07T10:17:03.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.762+0000 7f1c097fa700 1 -- 172.21.15.119:0/2354910869 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f1bfc021580 con 0x7f1c0c1015c0 2023-12-07T10:17:03.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.763+0000 7f1c097fa700 1 --2- 172.21.15.119:0/2354910869 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1bf40a1b10 0x7f1bf40a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:03.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.763+0000 7f1c097fa700 1 -- 172.21.15.119:0/2354910869 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1bfc0ceab0 con 0x7f1c0c1015c0 2023-12-07T10:17:03.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.763+0000 7f1c0b7fe700 1 --2- 172.21.15.119:0/2354910869 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1bf40a1b10 0x7f1bf40a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:03.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.764+0000 7f1c0b7fe700 1 --2- 172.21.15.119:0/2354910869 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1bf40a1b10 0x7f1bf40a3fd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009c60 tx=0x7f1c00009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:03.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.765+0000 7f1c097fa700 1 -- 172.21.15.119:0/2354910869 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1bfc098360 con 0x7f1c0c1015c0 2023-12-07T10:17:03.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:03 smithi153 ceph-mon[133625]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi119=a;smithi119=d;smithi119=g;smithi119=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:17:03.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:03 smithi153 ceph-mon[133625]: Saving service mds.all spec with placement smithi119=a;smithi119=d;smithi119=g;smithi119=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l;count:12 2023-12-07T10:17:03.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:03 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:03.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:03 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:17:03.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:03 smithi153 ceph-mon[133625]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:03.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:03 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:03.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:03 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:17:03.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:03 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:03.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:03 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:03.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:03 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:03.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:03 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:03.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:03 smithi153 ceph-mon[133625]: Deploying daemon mds.l on smithi159 2023-12-07T10:17:03.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.967+0000 7f1c123ff700 1 -- 172.21.15.119:0/2354910869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1c0c1084c0 con 0x7f1c0c1015c0 2023-12-07T10:17:03.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.967+0000 7f1c097fa700 1 -- 172.21.15.119:0/2354910869 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1bfc09b980 con 0x7f1c0c1015c0 2023-12-07T10:17:03.969 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-12-07T10:17:03.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.970+0000 7f1c123ff700 1 -- 172.21.15.119:0/2354910869 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1bf40a1b10 msgr2=0x7f1bf40a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:03.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.970+0000 7f1c123ff700 1 --2- 172.21.15.119:0/2354910869 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1bf40a1b10 0x7f1bf40a3fd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009c60 tx=0x7f1c00009260 comp rx=0 tx=0).stop 2023-12-07T10:17:03.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.970+0000 7f1c123ff700 1 -- 172.21.15.119:0/2354910869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0c1015c0 msgr2=0x7f1c0c1975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:03.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.970+0000 7f1c123ff700 1 --2- 172.21.15.119:0/2354910869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0c1015c0 0x7f1c0c1975e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc000c00 tx=0x7f1bfc009580 comp rx=0 tx=0).stop 2023-12-07T10:17:03.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.970+0000 7f1c123ff700 1 -- 172.21.15.119:0/2354910869 shutdown_connections 2023-12-07T10:17:03.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.970+0000 7f1c123ff700 1 --2- 172.21.15.119:0/2354910869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c0c103150 0x7f1c0c19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:03.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.970+0000 7f1c123ff700 1 --2- 172.21.15.119:0/2354910869 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1bf40a1b10 0x7f1bf40a3fd0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:03.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.970+0000 7f1c123ff700 1 --2- 172.21.15.119:0/2354910869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0c1015c0 0x7f1c0c1975e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:03.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.970+0000 7f1c123ff700 1 --2- 172.21.15.119:0/2354910869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c0c1027b0 0x7f1c0c197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:03.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.970+0000 7f1c123ff700 1 -- 172.21.15.119:0/2354910869 >> 172.21.15.119:0/2354910869 conn(0x7f1c0c0fcb10 msgr2=0x7f1c0c105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:03.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.970+0000 7f1c123ff700 1 -- 172.21.15.119:0/2354910869 shutdown_connections 2023-12-07T10:17:03.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:03.970+0000 7f1c123ff700 1 -- 172.21.15.119:0/2354910869 wait complete. 2023-12-07T10:17:03.991 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:17:04.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.185+0000 7fc30dfff700 1 -- 172.21.15.119:0/3615140180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc308102750 msgr2=0x7fc308102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:04.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.185+0000 7fc30dfff700 1 --2- 172.21.15.119:0/3615140180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc308102750 0x7fc308102bb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc009990 tx=0x7fc2fc009ca0 comp rx=0 tx=0).stop 2023-12-07T10:17:04.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.185+0000 7fc30dfff700 1 -- 172.21.15.119:0/3615140180 shutdown_connections 2023-12-07T10:17:04.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.185+0000 7fc30dfff700 1 --2- 172.21.15.119:0/3615140180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3081030f0 0x7fc3081099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:04.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.185+0000 7fc30dfff700 1 --2- 172.21.15.119:0/3615140180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc308101560 0x7fc308101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:04.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.185+0000 7fc30dfff700 1 --2- 172.21.15.119:0/3615140180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc308102750 0x7fc308102bb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:04.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.185+0000 7fc30dfff700 1 -- 172.21.15.119:0/3615140180 >> 172.21.15.119:0/3615140180 conn(0x7fc3080fcab0 msgr2=0x7fc3080fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:04.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.185+0000 7fc30dfff700 1 -- 172.21.15.119:0/3615140180 shutdown_connections 2023-12-07T10:17:04.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.185+0000 7fc30dfff700 1 -- 172.21.15.119:0/3615140180 wait complete. 2023-12-07T10:17:04.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.186+0000 7fc30dfff700 1 Processor -- start 2023-12-07T10:17:04.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.186+0000 7fc30dfff700 1 -- start start 2023-12-07T10:17:04.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.187+0000 7fc30dfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc308101560 0x7fc308199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:04.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.187+0000 7fc30dfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc308102750 0x7fc308199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:04.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.187+0000 7fc30dfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3081030f0 0x7fc3081a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:04.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.187+0000 7fc30dfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3081a1250 con 0x7fc3081030f0 2023-12-07T10:17:04.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.187+0000 7fc30dfff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3081a1390 con 0x7fc308101560 2023-12-07T10:17:04.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.187+0000 7fc30dfff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3081a1520 con 0x7fc308102750 2023-12-07T10:17:04.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.187+0000 7fc306ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc308102750 0x7fc308199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:04.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.187+0000 7fc306ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc308102750 0x7fc308199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:59794/0 (socket says 172.21.15.119:59794) 2023-12-07T10:17:04.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.187+0000 7fc307fff700 1 --2- 172.21.15.119:0/381075008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3081030f0 0x7fc3081a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:04.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.187+0000 7fc306ffd700 1 -- 172.21.15.119:0/381075008 learned_addr learned my addr 172.21.15.119:0/381075008 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:04.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.187+0000 7fc3077fe700 1 --2- 172.21.15.119:0/381075008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc308101560 0x7fc308199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:04.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.187+0000 7fc3077fe700 1 -- 172.21.15.119:0/381075008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc308102750 msgr2=0x7fc308199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:04.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.187+0000 7fc3077fe700 1 --2- 172.21.15.119:0/381075008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc308102750 0x7fc308199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:04.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.187+0000 7fc3077fe700 1 -- 172.21.15.119:0/381075008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3081030f0 msgr2=0x7fc3081a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:04.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.187+0000 7fc3077fe700 1 --2- 172.21.15.119:0/381075008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3081030f0 0x7fc3081a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:04.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.188+0000 7fc3077fe700 1 -- 172.21.15.119:0/381075008 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2fc0095f0 con 0x7fc308101560 2023-12-07T10:17:04.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.188+0000 7fc307fff700 1 --2- 172.21.15.119:0/381075008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3081030f0 0x7fc3081a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:17:04.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.188+0000 7fc3077fe700 1 --2- 172.21.15.119:0/381075008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc308101560 0x7fc308199720 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800e9c0 tx=0x7fc2f800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:04.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.188+0000 7fc304ff9700 1 -- 172.21.15.119:0/381075008 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8004220 con 0x7fc308101560 2023-12-07T10:17:04.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.188+0000 7fc30dfff700 1 -- 172.21.15.119:0/381075008 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3081a1800 con 0x7fc308101560 2023-12-07T10:17:04.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.188+0000 7fc304ff9700 1 -- 172.21.15.119:0/381075008 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2f8004380 con 0x7fc308101560 2023-12-07T10:17:04.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.188+0000 7fc304ff9700 1 -- 172.21.15.119:0/381075008 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8003be0 con 0x7fc308101560 2023-12-07T10:17:04.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.188+0000 7fc30dfff700 1 -- 172.21.15.119:0/381075008 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3081a1d60 con 0x7fc308101560 2023-12-07T10:17:04.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.189+0000 7fc30dfff700 1 -- 172.21.15.119:0/381075008 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc308068350 con 0x7fc308101560 2023-12-07T10:17:04.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.191+0000 7fc304ff9700 1 -- 172.21.15.119:0/381075008 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fc2f8003d40 con 0x7fc308101560 2023-12-07T10:17:04.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.192+0000 7fc304ff9700 1 --2- 172.21.15.119:0/381075008 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc2f00a1b10 0x7fc2f00a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:04.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.193+0000 7fc304ff9700 1 -- 172.21.15.119:0/381075008 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc2f8014070 con 0x7fc308101560 2023-12-07T10:17:04.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.193+0000 7fc306ffd700 1 --2- 172.21.15.119:0/381075008 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc2f00a1b10 0x7fc2f00a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:04.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.193+0000 7fc306ffd700 1 --2- 172.21.15.119:0/381075008 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc2f00a1b10 0x7fc2f00a3fd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00b2e0 tx=0x7fc2fc00b750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:04.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.195+0000 7fc304ff9700 1 -- 172.21.15.119:0/381075008 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc2f8096de0 con 0x7fc308101560 2023-12-07T10:17:04.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.447+0000 7fc30dfff700 1 -- 172.21.15.119:0/381075008 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc30810cdd0 con 0x7fc308101560 2023-12-07T10:17:04.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.448+0000 7fc304ff9700 1 -- 172.21.15.119:0/381075008 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7fc2f809a400 con 0x7fc308101560 2023-12-07T10:17:04.450 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:17:04.450 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-12-07T10:17:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.450+0000 7fc30dfff700 1 -- 172.21.15.119:0/381075008 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc2f00a1b10 msgr2=0x7fc2f00a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.450+0000 7fc30dfff700 1 --2- 172.21.15.119:0/381075008 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc2f00a1b10 0x7fc2f00a3fd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00b2e0 tx=0x7fc2fc00b750 comp rx=0 tx=0).stop 2023-12-07T10:17:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.450+0000 7fc30dfff700 1 -- 172.21.15.119:0/381075008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc308101560 msgr2=0x7fc308199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.450+0000 7fc30dfff700 1 --2- 172.21.15.119:0/381075008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc308101560 0x7fc308199720 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800e9c0 tx=0x7fc2f800ecd0 comp rx=0 tx=0).stop 2023-12-07T10:17:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.451+0000 7fc30dfff700 1 -- 172.21.15.119:0/381075008 shutdown_connections 2023-12-07T10:17:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.451+0000 7fc30dfff700 1 --2- 172.21.15.119:0/381075008 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc2f00a1b10 0x7fc2f00a3fd0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.451+0000 7fc30dfff700 1 --2- 172.21.15.119:0/381075008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3081030f0 0x7fc3081a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.451+0000 7fc30dfff700 1 --2- 172.21.15.119:0/381075008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc308101560 0x7fc308199720 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.451+0000 7fc30dfff700 1 --2- 172.21.15.119:0/381075008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc308102750 0x7fc308199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.451+0000 7fc30dfff700 1 -- 172.21.15.119:0/381075008 >> 172.21.15.119:0/381075008 conn(0x7fc3080fcab0 msgr2=0x7fc308107f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.451+0000 7fc30dfff700 1 -- 172.21.15.119:0/381075008 shutdown_connections 2023-12-07T10:17:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:04.451+0000 7fc30dfff700 1 -- 172.21.15.119:0/381075008 wait complete. 2023-12-07T10:17:04.454 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 1 2023-12-07T10:17:04.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:04 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/247473518' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:17:04.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:04 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2354910869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:17:04.764 INFO:journalctl@ceph.mds.l.smithi159.stdout:Dec 07 10:17:04 smithi159 systemd[1]: Starting Ceph mds.l for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:17:04.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:04 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/247473518' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:17:04.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:04 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2354910869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:17:04.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:04 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/247473518' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:17:04.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:04 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2354910869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:17:05.143 INFO:journalctl@ceph.mds.l.smithi159.stdout:Dec 07 10:17:04 smithi159 podman[171556]: 2023-12-07T10:17:05.144 INFO:journalctl@ceph.mds.l.smithi159.stdout:Dec 07 10:17:05 smithi159 ceph-mds[171596]: -- 172.21.15.159:0/2465825694 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c728f3c1a0 con 0x55c728f12c00 2023-12-07T10:17:05.144 INFO:journalctl@ceph.mds.l.smithi159.stdout:Dec 07 10:17:05 smithi159 bash[171556]: 19f143714d131c043fad1067ece8d385206648d3a065cc4438289d819f3fbcf6 2023-12-07T10:17:05.144 INFO:journalctl@ceph.mds.l.smithi159.stdout:Dec 07 10:17:05 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-l[171572]: starting mds.l at 2023-12-07T10:17:05.439 INFO:journalctl@ceph.mds.l.smithi159.stdout:Dec 07 10:17:05 smithi159 systemd[1]: Started Ceph mds.l for a4c642c6-94e8-11ee-95a2-87774f69a715. 2023-12-07T10:17:05.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:05 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/381075008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:05.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:05 smithi119 ceph-mon[97908]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:05.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:05 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.159:6832/935461464,v1:172.21.15.159:6833/935461464] up:boot 2023-12-07T10:17:05.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:05 smithi119 ceph-mon[97908]: fsmap 1 up:standby 2023-12-07T10:17:05.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:05 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-07T10:17:05.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:05 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:05.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:05 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:05.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:05 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:05.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:05 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:05.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:05 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:05.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:05 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:05.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:05 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/381075008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:05.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:05 smithi159 ceph-mon[134726]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:05.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:05 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.159:6832/935461464,v1:172.21.15.159:6833/935461464] up:boot 2023-12-07T10:17:05.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:05 smithi159 ceph-mon[134726]: fsmap 1 up:standby 2023-12-07T10:17:05.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:05 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-07T10:17:05.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:05 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:05.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:05 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:05.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:05 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:05.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:05 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:05.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:05 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:05.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:05 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:05.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:05 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/381075008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:05.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:05 smithi153 ceph-mon[133625]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:05.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:05 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.159:6832/935461464,v1:172.21.15.159:6833/935461464] up:boot 2023-12-07T10:17:05.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:05 smithi153 ceph-mon[133625]: fsmap 1 up:standby 2023-12-07T10:17:05.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:05 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-07T10:17:05.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:05 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:05.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:05 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:05.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:05 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:05.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:05 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:05.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:05 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:05.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:05 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:06.472 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:17:06.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.677+0000 7fbcbb3ff700 1 -- 172.21.15.119:0/3189421406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcb4101560 msgr2=0x7fbcb4101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:06.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.677+0000 7fbcbb3ff700 1 --2- 172.21.15.119:0/3189421406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcb4101560 0x7fbcb4101980 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00099c0 tx=0x7fbcb0009cd0 comp rx=0 tx=0).stop 2023-12-07T10:17:06.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.677+0000 7fbcbb3ff700 1 -- 172.21.15.119:0/3189421406 shutdown_connections 2023-12-07T10:17:06.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.677+0000 7fbcbb3ff700 1 --2- 172.21.15.119:0/3189421406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcb41030f0 0x7fbcb41099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:06.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.677+0000 7fbcbb3ff700 1 --2- 172.21.15.119:0/3189421406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcb4101560 0x7fbcb4101980 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:06.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.677+0000 7fbcbb3ff700 1 --2- 172.21.15.119:0/3189421406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbcb4102750 0x7fbcb4102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:06.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.677+0000 7fbcbb3ff700 1 -- 172.21.15.119:0/3189421406 >> 172.21.15.119:0/3189421406 conn(0x7fbcb40fcab0 msgr2=0x7fbcb40fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:06.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.677+0000 7fbcbb3ff700 1 -- 172.21.15.119:0/3189421406 shutdown_connections 2023-12-07T10:17:06.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.677+0000 7fbcbb3ff700 1 -- 172.21.15.119:0/3189421406 wait complete. 2023-12-07T10:17:06.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.678+0000 7fbcbb3ff700 1 Processor -- start 2023-12-07T10:17:06.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.678+0000 7fbcbb3ff700 1 -- start start 2023-12-07T10:17:06.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.678+0000 7fbcbb3ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcb4101560 0x7fbcb4199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:06.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.678+0000 7fbcbb3ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbcb4102750 0x7fbcb4199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:06.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.678+0000 7fbcbb3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcb41030f0 0x7fbcb41a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:06.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.678+0000 7fbcbb3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb41a12c0 con 0x7fbcb41030f0 2023-12-07T10:17:06.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.678+0000 7fbcbb3ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb41a1400 con 0x7fbcb4102750 2023-12-07T10:17:06.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.678+0000 7fbcbb3ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb41a1590 con 0x7fbcb4101560 2023-12-07T10:17:06.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.679+0000 7fbcb999c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcb41030f0 0x7fbcb41a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:06.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.679+0000 7fbcb999c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcb41030f0 0x7fbcb41a0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37118/0 (socket says 172.21.15.119:37118) 2023-12-07T10:17:06.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.679+0000 7fbcb999c700 1 -- 172.21.15.119:0/3832629826 learned_addr learned my addr 172.21.15.119:0/3832629826 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:06.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.679+0000 7fbcb899a700 1 --2- 172.21.15.119:0/3832629826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbcb4102750 0x7fbcb4199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:06.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.679+0000 7fbcb919b700 1 --2- 172.21.15.119:0/3832629826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcb4101560 0x7fbcb4199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:06.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.679+0000 7fbcb999c700 1 -- 172.21.15.119:0/3832629826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcb4101560 msgr2=0x7fbcb4199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:06.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.679+0000 7fbcb999c700 1 --2- 172.21.15.119:0/3832629826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcb4101560 0x7fbcb4199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:06.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.679+0000 7fbcb999c700 1 -- 172.21.15.119:0/3832629826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbcb4102750 msgr2=0x7fbcb4199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:06.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.679+0000 7fbcb999c700 1 --2- 172.21.15.119:0/3832629826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbcb4102750 0x7fbcb4199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:06.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.679+0000 7fbcb999c700 1 -- 172.21.15.119:0/3832629826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb00095f0 con 0x7fbcb41030f0 2023-12-07T10:17:06.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.679+0000 7fbcb999c700 1 --2- 172.21.15.119:0/3832629826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcb41030f0 0x7fbcb41a0cf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbcac007ee0 tx=0x7fbcac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:06.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.680+0000 7fbcaa7fc700 1 -- 172.21.15.119:0/3832629826 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac017070 con 0x7fbcb41030f0 2023-12-07T10:17:06.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.680+0000 7fbcaa7fc700 1 -- 172.21.15.119:0/3832629826 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcac00f040 con 0x7fbcb41030f0 2023-12-07T10:17:06.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.680+0000 7fbcbb3ff700 1 -- 172.21.15.119:0/3832629826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb41a1870 con 0x7fbcb41030f0 2023-12-07T10:17:06.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.680+0000 7fbcaa7fc700 1 -- 172.21.15.119:0/3832629826 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac005740 con 0x7fbcb41030f0 2023-12-07T10:17:06.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.680+0000 7fbcbb3ff700 1 -- 172.21.15.119:0/3832629826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb41a1d80 con 0x7fbcb41030f0 2023-12-07T10:17:06.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.681+0000 7fbcbb3ff700 1 -- 172.21.15.119:0/3832629826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb4068350 con 0x7fbcb41030f0 2023-12-07T10:17:06.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.683+0000 7fbcaa7fc700 1 -- 172.21.15.119:0/3832629826 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fbcac00dc60 con 0x7fbcb41030f0 2023-12-07T10:17:06.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.684+0000 7fbcaa7fc700 1 --2- 172.21.15.119:0/3832629826 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fbca00a1b90 0x7fbca00a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:06.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.684+0000 7fbcaa7fc700 1 -- 172.21.15.119:0/3832629826 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbcac0cc940 con 0x7fbcb41030f0 2023-12-07T10:17:06.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.684+0000 7fbcb919b700 1 --2- 172.21.15.119:0/3832629826 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fbca00a1b90 0x7fbca00a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:06.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.685+0000 7fbcb919b700 1 --2- 172.21.15.119:0/3832629826 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fbca00a1b90 0x7fbca00a4050 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0006010 tx=0x7fbcb000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:06.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.688+0000 7fbcaa7fc700 1 -- 172.21.15.119:0/3832629826 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fbcac096160 con 0x7fbcb41030f0 2023-12-07T10:17:06.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:06 smithi119 ceph-mon[97908]: Deploying daemon mds.k on smithi153 2023-12-07T10:17:06.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:06 smithi159 ceph-mon[134726]: Deploying daemon mds.k on smithi153 2023-12-07T10:17:06.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:06 smithi153 ceph-mon[133625]: Deploying daemon mds.k on smithi153 2023-12-07T10:17:06.781 INFO:journalctl@ceph.mds.k.smithi153.stdout:Dec 07 10:17:06 smithi153 systemd[1]: Starting Ceph mds.k for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:17:06.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.934+0000 7fbcbb3ff700 1 -- 172.21.15.119:0/3832629826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbcb410ce50 con 0x7fbcb41030f0 2023-12-07T10:17:06.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.934+0000 7fbcaa7fc700 1 -- 172.21.15.119:0/3832629826 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1325 (secure 0 0 0) 0x7fbcac099780 con 0x7fbcb41030f0 2023-12-07T10:17:06.936 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:17:06.936 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-12-07T10:17:06.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.936+0000 7fbcbb3ff700 1 -- 172.21.15.119:0/3832629826 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fbca00a1b90 msgr2=0x7fbca00a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:06.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.936+0000 7fbcbb3ff700 1 --2- 172.21.15.119:0/3832629826 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fbca00a1b90 0x7fbca00a4050 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0006010 tx=0x7fbcb000b540 comp rx=0 tx=0).stop 2023-12-07T10:17:06.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.936+0000 7fbcbb3ff700 1 -- 172.21.15.119:0/3832629826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcb41030f0 msgr2=0x7fbcb41a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:06.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.936+0000 7fbcbb3ff700 1 --2- 172.21.15.119:0/3832629826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcb41030f0 0x7fbcb41a0cf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbcac007ee0 tx=0x7fbcac00d3b0 comp rx=0 tx=0).stop 2023-12-07T10:17:06.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.936+0000 7fbcbb3ff700 1 -- 172.21.15.119:0/3832629826 shutdown_connections 2023-12-07T10:17:06.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.936+0000 7fbcbb3ff700 1 --2- 172.21.15.119:0/3832629826 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fbca00a1b90 0x7fbca00a4050 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:06.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.936+0000 7fbcbb3ff700 1 --2- 172.21.15.119:0/3832629826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcb41030f0 0x7fbcb41a0cf0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:06.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.936+0000 7fbcbb3ff700 1 --2- 172.21.15.119:0/3832629826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcb4101560 0x7fbcb4199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:06.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.936+0000 7fbcbb3ff700 1 --2- 172.21.15.119:0/3832629826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbcb4102750 0x7fbcb4199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:06.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.936+0000 7fbcbb3ff700 1 -- 172.21.15.119:0/3832629826 >> 172.21.15.119:0/3832629826 conn(0x7fbcb40fcab0 msgr2=0x7fbcb4107f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:06.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.936+0000 7fbcbb3ff700 1 -- 172.21.15.119:0/3832629826 shutdown_connections 2023-12-07T10:17:06.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:06.937+0000 7fbcbb3ff700 1 -- 172.21.15.119:0/3832629826 wait complete. 2023-12-07T10:17:06.940 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 2 2023-12-07T10:17:07.234 INFO:journalctl@ceph.mds.k.smithi153.stdout:Dec 07 10:17:06 smithi153 podman[169405]: 2023-12-07T10:17:07.234 INFO:journalctl@ceph.mds.k.smithi153.stdout:Dec 07 10:17:07 smithi153 bash[169405]: b447bfee8ee450d2fc675fce5a0aa8fd936ba584dfcb52309136437aa48f0a56 2023-12-07T10:17:07.234 INFO:journalctl@ceph.mds.k.smithi153.stdout:Dec 07 10:17:07 smithi153 ceph-mds[169445]: -- 172.21.15.153:0/1629388880 <== mon.2 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55981499e1a0 con 0x559814974800 2023-12-07T10:17:07.234 INFO:journalctl@ceph.mds.k.smithi153.stdout:Dec 07 10:17:07 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-k[169421]: starting mds.k at 2023-12-07T10:17:07.530 INFO:journalctl@ceph.mds.k.smithi153.stdout:Dec 07 10:17:07 smithi153 systemd[1]: Started Ceph mds.k for a4c642c6-94e8-11ee-95a2-87774f69a715. 2023-12-07T10:17:07.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:07 smithi153 ceph-mon[133625]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:07.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:07 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3832629826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:07.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:07 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:07.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:07 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:07.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:07 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:07.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:07 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:07.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:07 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:07.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:07 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:07.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:07 smithi119 ceph-mon[97908]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:07.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:07 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3832629826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:07.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:07 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:07.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:07 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:07.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:07 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:07.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:07 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:07.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:07 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:07.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:07 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:07.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:07 smithi159 ceph-mon[134726]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:07.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:07 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3832629826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:07.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:07 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:07.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:07 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:07.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:07 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:07.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:07 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:07.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:07 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:07.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:07 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:08.544 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:08 smithi119 ceph-mon[97908]: Deploying daemon mds.j on smithi119 2023-12-07T10:17:08.544 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:08 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.153:6832/209059922,v1:172.21.15.153:6833/209059922] up:boot 2023-12-07T10:17:08.544 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:08 smithi119 ceph-mon[97908]: fsmap 2 up:standby 2023-12-07T10:17:08.544 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:08 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-07T10:17:08.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:08 smithi159 ceph-mon[134726]: Deploying daemon mds.j on smithi119 2023-12-07T10:17:08.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:08 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.153:6832/209059922,v1:172.21.15.153:6833/209059922] up:boot 2023-12-07T10:17:08.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:08 smithi159 ceph-mon[134726]: fsmap 2 up:standby 2023-12-07T10:17:08.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:08 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-07T10:17:08.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:08 smithi153 ceph-mon[133625]: Deploying daemon mds.j on smithi119 2023-12-07T10:17:08.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:08 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.153:6832/209059922,v1:172.21.15.153:6833/209059922] up:boot 2023-12-07T10:17:08.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:08 smithi153 ceph-mon[133625]: fsmap 2 up:standby 2023-12-07T10:17:08.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:08 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-07T10:17:08.960 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:17:09.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.262+0000 7f30537ff700 1 -- 172.21.15.119:0/2498788073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f304c102750 msgr2=0x7f304c102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:09.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.262+0000 7f30537ff700 1 --2- 172.21.15.119:0/2498788073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f304c102750 0x7f304c102bb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f303c0099c0 tx=0x7f303c009cd0 comp rx=0 tx=0).stop 2023-12-07T10:17:09.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.262+0000 7f30537ff700 1 -- 172.21.15.119:0/2498788073 shutdown_connections 2023-12-07T10:17:09.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.262+0000 7f30537ff700 1 --2- 172.21.15.119:0/2498788073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f304c1030f0 0x7f304c1099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:09.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.262+0000 7f30537ff700 1 --2- 172.21.15.119:0/2498788073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f304c101560 0x7f304c101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:09.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.262+0000 7f30537ff700 1 --2- 172.21.15.119:0/2498788073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f304c102750 0x7f304c102bb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:09.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.262+0000 7f30537ff700 1 -- 172.21.15.119:0/2498788073 >> 172.21.15.119:0/2498788073 conn(0x7f304c0fcab0 msgr2=0x7f304c0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:09.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.262+0000 7f30537ff700 1 -- 172.21.15.119:0/2498788073 shutdown_connections 2023-12-07T10:17:09.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.262+0000 7f30537ff700 1 -- 172.21.15.119:0/2498788073 wait complete. 2023-12-07T10:17:09.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.264+0000 7f30537ff700 1 Processor -- start 2023-12-07T10:17:09.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.264+0000 7f30537ff700 1 -- start start 2023-12-07T10:17:09.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.264+0000 7f30537ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f304c101560 0x7f304c075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:09.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.264+0000 7f30537ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f304c102750 0x7f304c073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:09.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.264+0000 7f30537ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f304c1030f0 0x7f304c073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:09.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.264+0000 7f30537ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c0746e0 con 0x7f304c1030f0 2023-12-07T10:17:09.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.264+0000 7f30537ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c074850 con 0x7f304c102750 2023-12-07T10:17:09.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.264+0000 7f30537ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c1a1550 con 0x7f304c101560 2023-12-07T10:17:09.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.264+0000 7f3050d9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f304c102750 0x7f304c073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:09.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.264+0000 7f3050d9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f304c102750 0x7f304c073a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:40000/0 (socket says 172.21.15.119:40000) 2023-12-07T10:17:09.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.264+0000 7f3050d9a700 1 -- 172.21.15.119:0/1518343861 learned_addr learned my addr 172.21.15.119:0/1518343861 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:09.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.264+0000 7f3051d9c700 1 --2- 172.21.15.119:0/1518343861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f304c1030f0 0x7f304c073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:09.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.264+0000 7f305159b700 1 --2- 172.21.15.119:0/1518343861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f304c101560 0x7f304c075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:09.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.265+0000 7f3050d9a700 1 -- 172.21.15.119:0/1518343861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f304c101560 msgr2=0x7f304c075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:09.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.265+0000 7f3050d9a700 1 --2- 172.21.15.119:0/1518343861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f304c101560 0x7f304c075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:09.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.265+0000 7f3050d9a700 1 -- 172.21.15.119:0/1518343861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f304c1030f0 msgr2=0x7f304c073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:09.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.265+0000 7f3050d9a700 1 --2- 172.21.15.119:0/1518343861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f304c1030f0 0x7f304c073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:09.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.265+0000 7f3050d9a700 1 -- 172.21.15.119:0/1518343861 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3048009520 con 0x7f304c102750 2023-12-07T10:17:09.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.265+0000 7f3050d9a700 1 --2- 172.21.15.119:0/1518343861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f304c102750 0x7f304c073a00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f303c003730 tx=0x7f303c00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:09.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.265+0000 7f30427fc700 1 -- 172.21.15.119:0/1518343861 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c01d070 con 0x7f304c102750 2023-12-07T10:17:09.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.265+0000 7f30537ff700 1 -- 172.21.15.119:0/1518343861 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f303c0095f0 con 0x7f304c102750 2023-12-07T10:17:09.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.265+0000 7f30427fc700 1 -- 172.21.15.119:0/1518343861 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f303c00fe70 con 0x7f304c102750 2023-12-07T10:17:09.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.265+0000 7f30427fc700 1 -- 172.21.15.119:0/1518343861 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c017650 con 0x7f304c102750 2023-12-07T10:17:09.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.265+0000 7f30537ff700 1 -- 172.21.15.119:0/1518343861 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f304c1a1a10 con 0x7f304c102750 2023-12-07T10:17:09.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.266+0000 7f30537ff700 1 -- 172.21.15.119:0/1518343861 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f304c1089d0 con 0x7f304c102750 2023-12-07T10:17:09.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.268+0000 7f30427fc700 1 -- 172.21.15.119:0/1518343861 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f303c022470 con 0x7f304c102750 2023-12-07T10:17:09.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.270+0000 7f30427fc700 1 --2- 172.21.15.119:0/1518343861 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f30380a1b10 0x7f30380a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:09.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.270+0000 7f30427fc700 1 -- 172.21.15.119:0/1518343861 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f303c0ce8a0 con 0x7f304c102750 2023-12-07T10:17:09.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.270+0000 7f305159b700 1 --2- 172.21.15.119:0/1518343861 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f30380a1b10 0x7f30380a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:09.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.271+0000 7f305159b700 1 --2- 172.21.15.119:0/1518343861 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f30380a1b10 0x7f30380a3fd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f304c1025b0 tx=0x7f3048009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:09.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.272+0000 7f30427fc700 1 -- 172.21.15.119:0/1518343861 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f303c098150 con 0x7f304c102750 2023-12-07T10:17:09.447 INFO:journalctl@ceph.mds.j.smithi119.stdout:Dec 07 10:17:09 smithi119 systemd[1]: Starting Ceph mds.j for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:17:09.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.529+0000 7f30537ff700 1 -- 172.21.15.119:0/1518343861 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f304c068320 con 0x7f304c102750 2023-12-07T10:17:09.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.530+0000 7f30427fc700 1 -- 172.21.15.119:0/1518343861 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2116 (secure 0 0 0) 0x7f303c09b770 con 0x7f304c102750 2023-12-07T10:17:09.533 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:17:09.533 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-12-07T10:17:09.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.532+0000 7f30537ff700 1 -- 172.21.15.119:0/1518343861 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f30380a1b10 msgr2=0x7f30380a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:09.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.532+0000 7f30537ff700 1 --2- 172.21.15.119:0/1518343861 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f30380a1b10 0x7f30380a3fd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f304c1025b0 tx=0x7f3048009260 comp rx=0 tx=0).stop 2023-12-07T10:17:09.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.532+0000 7f30537ff700 1 -- 172.21.15.119:0/1518343861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f304c102750 msgr2=0x7f304c073a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:09.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.532+0000 7f30537ff700 1 --2- 172.21.15.119:0/1518343861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f304c102750 0x7f304c073a00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f303c003730 tx=0x7f303c00fa00 comp rx=0 tx=0).stop 2023-12-07T10:17:09.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.532+0000 7f30537ff700 1 -- 172.21.15.119:0/1518343861 shutdown_connections 2023-12-07T10:17:09.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.532+0000 7f30537ff700 1 --2- 172.21.15.119:0/1518343861 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f30380a1b10 0x7f30380a3fd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:09.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.532+0000 7f30537ff700 1 --2- 172.21.15.119:0/1518343861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f304c1030f0 0x7f304c073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:09.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.532+0000 7f30537ff700 1 --2- 172.21.15.119:0/1518343861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f304c101560 0x7f304c075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:09.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.532+0000 7f30537ff700 1 --2- 172.21.15.119:0/1518343861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f304c102750 0x7f304c073a00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:09.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.532+0000 7f30537ff700 1 -- 172.21.15.119:0/1518343861 >> 172.21.15.119:0/1518343861 conn(0x7f304c0fcab0 msgr2=0x7f304c105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:09.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.533+0000 7f30537ff700 1 -- 172.21.15.119:0/1518343861 shutdown_connections 2023-12-07T10:17:09.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:09.533+0000 7f30537ff700 1 -- 172.21.15.119:0/1518343861 wait complete. 2023-12-07T10:17:09.537 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 3 2023-12-07T10:17:09.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:09 smithi159 ceph-mon[134726]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:09.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:09 smithi153 ceph-mon[133625]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:09.906 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:09 smithi119 ceph-mon[97908]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:10.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Dec 07 10:17:09 smithi119 podman[145334]: 2023-12-07T10:17:10.158 INFO:journalctl@ceph.mds.j.smithi119.stdout:Dec 07 10:17:10 smithi119 bash[145334]: 77c6708926be54e00fce0e9c491bc59511b5d2a8ece0449b3969988069d17b72 2023-12-07T10:17:10.479 INFO:journalctl@ceph.mds.j.smithi119.stdout:Dec 07 10:17:10 smithi119 ceph-mds[145374]: -- 172.21.15.119:0/3050255135 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e3af9be1a0 con 0x55e3af996c00 2023-12-07T10:17:10.479 INFO:journalctl@ceph.mds.j.smithi119.stdout:Dec 07 10:17:10 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-j[145350]: starting mds.j at 2023-12-07T10:17:10.479 INFO:journalctl@ceph.mds.j.smithi119.stdout:Dec 07 10:17:10 smithi119 systemd[1]: Started Ceph mds.j for a4c642c6-94e8-11ee-95a2-87774f69a715. 2023-12-07T10:17:10.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:10 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1518343861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:10.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:10 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:10.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:10 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:10.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:10 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:10.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:10 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:10.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:10 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:10.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:10 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:10.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:10 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1518343861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:10.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:10 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:10.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:10 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:10.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:10 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:10.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:10 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:10.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:10 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:10.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:10 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:10.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:10 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1518343861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:10.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:10 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:10.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:10 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:10.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:10 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:10.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:10 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:10.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:10 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:10.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:10 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:11.557 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:17:11.608 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:11 smithi119 ceph-mon[97908]: Deploying daemon mds.d on smithi119 2023-12-07T10:17:11.608 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:11 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.119:6834/1556173472,v1:172.21.15.119:6835/1556173472] up:boot 2023-12-07T10:17:11.608 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:11 smithi119 ceph-mon[97908]: fsmap 3 up:standby 2023-12-07T10:17:11.608 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:11 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-07T10:17:11.608 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:11 smithi119 ceph-mon[97908]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:11.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:11 smithi159 ceph-mon[134726]: Deploying daemon mds.d on smithi119 2023-12-07T10:17:11.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:11 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.119:6834/1556173472,v1:172.21.15.119:6835/1556173472] up:boot 2023-12-07T10:17:11.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:11 smithi159 ceph-mon[134726]: fsmap 3 up:standby 2023-12-07T10:17:11.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:11 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-07T10:17:11.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:11 smithi159 ceph-mon[134726]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:11.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.763+0000 7faa41bff700 1 -- 172.21.15.119:0/2465498625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3c101560 msgr2=0x7faa3c101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:11.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.763+0000 7faa41bff700 1 --2- 172.21.15.119:0/2465498625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3c101560 0x7faa3c101980 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faa2c009920 tx=0x7faa2c009c30 comp rx=0 tx=0).stop 2023-12-07T10:17:11.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.763+0000 7faa41bff700 1 -- 172.21.15.119:0/2465498625 shutdown_connections 2023-12-07T10:17:11.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.763+0000 7faa41bff700 1 --2- 172.21.15.119:0/2465498625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa3c1030f0 0x7faa3c1099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:11.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.763+0000 7faa41bff700 1 --2- 172.21.15.119:0/2465498625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3c101560 0x7faa3c101980 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:11.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.763+0000 7faa41bff700 1 --2- 172.21.15.119:0/2465498625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa3c102750 0x7faa3c102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:11.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.763+0000 7faa41bff700 1 -- 172.21.15.119:0/2465498625 >> 172.21.15.119:0/2465498625 conn(0x7faa3c0fcab0 msgr2=0x7faa3c0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:11.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.763+0000 7faa41bff700 1 -- 172.21.15.119:0/2465498625 shutdown_connections 2023-12-07T10:17:11.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.763+0000 7faa41bff700 1 -- 172.21.15.119:0/2465498625 wait complete. 2023-12-07T10:17:11.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.764+0000 7faa41bff700 1 Processor -- start 2023-12-07T10:17:11.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.764+0000 7faa41bff700 1 -- start start 2023-12-07T10:17:11.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.765+0000 7faa41bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa3c101560 0x7faa3c075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:11.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.765+0000 7faa41bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3c102750 0x7faa3c073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:11.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.765+0000 7faa41bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa3c1030f0 0x7faa3c073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:11.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.765+0000 7faa41bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c19b480 con 0x7faa3c102750 2023-12-07T10:17:11.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.765+0000 7faa41bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c074480 con 0x7faa3c1030f0 2023-12-07T10:17:11.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.765+0000 7faa41bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c074640 con 0x7faa3c101560 2023-12-07T10:17:11.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.765+0000 7faa33fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3c102750 0x7faa3c073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:11.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.765+0000 7faa33fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3c102750 0x7faa3c073a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37162/0 (socket says 172.21.15.119:37162) 2023-12-07T10:17:11.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.765+0000 7faa33fff700 1 -- 172.21.15.119:0/2917320433 learned_addr learned my addr 172.21.15.119:0/2917320433 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:11.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.765+0000 7faa3b7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa3c101560 0x7faa3c075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:11.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.765+0000 7faa3bfff700 1 --2- 172.21.15.119:0/2917320433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa3c1030f0 0x7faa3c073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:11.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.765+0000 7faa3b7fe700 1 -- 172.21.15.119:0/2917320433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa3c1030f0 msgr2=0x7faa3c073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:11.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.765+0000 7faa3b7fe700 1 --2- 172.21.15.119:0/2917320433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa3c1030f0 0x7faa3c073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:11.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.766+0000 7faa3b7fe700 1 -- 172.21.15.119:0/2917320433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3c102750 msgr2=0x7faa3c073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:11.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.766+0000 7faa3b7fe700 1 --2- 172.21.15.119:0/2917320433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3c102750 0x7faa3c073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:11.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.766+0000 7faa3b7fe700 1 -- 172.21.15.119:0/2917320433 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2c009520 con 0x7faa3c101560 2023-12-07T10:17:11.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.766+0000 7faa33fff700 1 --2- 172.21.15.119:0/2917320433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3c102750 0x7faa3c073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:17:11.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.766+0000 7faa3b7fe700 1 --2- 172.21.15.119:0/2917320433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa3c101560 0x7faa3c075370 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faa2c009230 tx=0x7faa2c00bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:11.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.766+0000 7faa397fa700 1 -- 172.21.15.119:0/2917320433 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c01c070 con 0x7faa3c101560 2023-12-07T10:17:11.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.766+0000 7faa41bff700 1 -- 172.21.15.119:0/2917320433 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa3c074860 con 0x7faa3c101560 2023-12-07T10:17:11.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.766+0000 7faa397fa700 1 -- 172.21.15.119:0/2917320433 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa2c00bb20 con 0x7faa3c101560 2023-12-07T10:17:11.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.766+0000 7faa397fa700 1 -- 172.21.15.119:0/2917320433 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c0049c0 con 0x7faa3c101560 2023-12-07T10:17:11.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.766+0000 7faa41bff700 1 -- 172.21.15.119:0/2917320433 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa3c1a6070 con 0x7faa3c101560 2023-12-07T10:17:11.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.767+0000 7faa41bff700 1 -- 172.21.15.119:0/2917320433 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa3c108a80 con 0x7faa3c101560 2023-12-07T10:17:11.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.769+0000 7faa397fa700 1 -- 172.21.15.119:0/2917320433 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7faa2c020070 con 0x7faa3c101560 2023-12-07T10:17:11.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.770+0000 7faa397fa700 1 --2- 172.21.15.119:0/2917320433 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7faa280a1b10 0x7faa280a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:11.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.770+0000 7faa397fa700 1 -- 172.21.15.119:0/2917320433 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7faa2c0cdd00 con 0x7faa3c101560 2023-12-07T10:17:11.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.770+0000 7faa33fff700 1 --2- 172.21.15.119:0/2917320433 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7faa280a1b10 0x7faa280a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:11.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.771+0000 7faa33fff700 1 --2- 172.21.15.119:0/2917320433 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7faa280a1b10 0x7faa280a3fd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faa24005950 tx=0x7faa24009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:11.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:11.773+0000 7faa397fa700 1 -- 172.21.15.119:0/2917320433 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7faa2c0975b0 con 0x7faa3c101560 2023-12-07T10:17:11.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:11 smithi153 ceph-mon[133625]: Deploying daemon mds.d on smithi119 2023-12-07T10:17:11.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:11 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.119:6834/1556173472,v1:172.21.15.119:6835/1556173472] up:boot 2023-12-07T10:17:11.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:11 smithi153 ceph-mon[133625]: fsmap 3 up:standby 2023-12-07T10:17:11.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:11 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-07T10:17:11.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:11 smithi153 ceph-mon[133625]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:12.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:12.025+0000 7faa41bff700 1 -- 172.21.15.119:0/2917320433 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa3c068350 con 0x7faa3c101560 2023-12-07T10:17:12.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:12.026+0000 7faa397fa700 1 -- 172.21.15.119:0/2917320433 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7faa2c09abd0 con 0x7faa3c101560 2023-12-07T10:17:12.028 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:17:12.028 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-12-07T10:17:12.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:12.029+0000 7faa41bff700 1 -- 172.21.15.119:0/2917320433 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7faa280a1b10 msgr2=0x7faa280a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:12.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:12.029+0000 7faa41bff700 1 --2- 172.21.15.119:0/2917320433 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7faa280a1b10 0x7faa280a3fd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faa24005950 tx=0x7faa24009310 comp rx=0 tx=0).stop 2023-12-07T10:17:12.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:12.029+0000 7faa41bff700 1 -- 172.21.15.119:0/2917320433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa3c101560 msgr2=0x7faa3c075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:12.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:12.029+0000 7faa41bff700 1 --2- 172.21.15.119:0/2917320433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa3c101560 0x7faa3c075370 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faa2c009230 tx=0x7faa2c00bef0 comp rx=0 tx=0).stop 2023-12-07T10:17:12.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:12.029+0000 7faa41bff700 1 -- 172.21.15.119:0/2917320433 shutdown_connections 2023-12-07T10:17:12.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:12.029+0000 7faa41bff700 1 --2- 172.21.15.119:0/2917320433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa3c1030f0 0x7faa3c073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:12.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:12.029+0000 7faa41bff700 1 --2- 172.21.15.119:0/2917320433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa3c101560 0x7faa3c075370 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:12.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:12.029+0000 7faa41bff700 1 --2- 172.21.15.119:0/2917320433 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7faa280a1b10 0x7faa280a3fd0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:12.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:12.029+0000 7faa41bff700 1 --2- 172.21.15.119:0/2917320433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3c102750 0x7faa3c073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:12.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:12.029+0000 7faa41bff700 1 -- 172.21.15.119:0/2917320433 >> 172.21.15.119:0/2917320433 conn(0x7faa3c0fcab0 msgr2=0x7faa3c105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:12.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:12.029+0000 7faa41bff700 1 -- 172.21.15.119:0/2917320433 shutdown_connections 2023-12-07T10:17:12.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:12.029+0000 7faa41bff700 1 -- 172.21.15.119:0/2917320433 wait complete. 2023-12-07T10:17:12.032 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 4 2023-12-07T10:17:12.599 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:12 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2917320433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:12.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:12 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2917320433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:12.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:12 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2917320433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:12.979 INFO:journalctl@ceph.mds.d.smithi119.stdout:Dec 07 10:17:12 smithi119 systemd[1]: Starting Ceph mds.d for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:17:13.370 INFO:journalctl@ceph.mds.d.smithi119.stdout:Dec 07 10:17:13 smithi119 podman[145718]: 2023-12-07T10:17:13.370 INFO:journalctl@ceph.mds.d.smithi119.stdout:Dec 07 10:17:13 smithi119 bash[145718]: cfc4c1daf62756db57f1c38004034a03dc23958a33c52e221a7ec485b91593d4 2023-12-07T10:17:13.370 INFO:journalctl@ceph.mds.d.smithi119.stdout:Dec 07 10:17:13 smithi119 ceph-mds[145758]: -- 172.21.15.119:0/2414879587 <== mon.0 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a09a0be1a0 con 0x55a09a097000 2023-12-07T10:17:13.370 INFO:journalctl@ceph.mds.d.smithi119.stdout:Dec 07 10:17:13 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-d[145734]: starting mds.d at 2023-12-07T10:17:13.729 INFO:journalctl@ceph.mds.d.smithi119.stdout:Dec 07 10:17:13 smithi119 systemd[1]: Started Ceph mds.d for a4c642c6-94e8-11ee-95a2-87774f69a715. 2023-12-07T10:17:13.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:13 smithi119 ceph-mon[97908]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:13.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:13.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:13.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:13.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:13.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:13.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:13 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:13.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:13 smithi159 ceph-mon[134726]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:13.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:13 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:13.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:13 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:13.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:13 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:13.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:13 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:13.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:13 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:13.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:13 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:13.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:13 smithi153 ceph-mon[133625]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:13.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:13 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:13.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:13 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:13.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:13 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:13.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:13 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:13.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:13 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:13.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:13 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:14.053 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:17:14.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.257+0000 7fd56a5ff700 1 -- 172.21.15.119:0/2060027820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd564101560 msgr2=0x7fd564101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:14.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.257+0000 7fd56a5ff700 1 --2- 172.21.15.119:0/2060027820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd564101560 0x7fd564101980 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd5540099c0 tx=0x7fd554009cd0 comp rx=0 tx=0).stop 2023-12-07T10:17:14.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.257+0000 7fd56a5ff700 1 -- 172.21.15.119:0/2060027820 shutdown_connections 2023-12-07T10:17:14.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.257+0000 7fd56a5ff700 1 --2- 172.21.15.119:0/2060027820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5641030f0 0x7fd5641099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:14.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.257+0000 7fd56a5ff700 1 --2- 172.21.15.119:0/2060027820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd564101560 0x7fd564101980 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:14.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.257+0000 7fd56a5ff700 1 --2- 172.21.15.119:0/2060027820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd564102750 0x7fd564102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:14.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.257+0000 7fd56a5ff700 1 -- 172.21.15.119:0/2060027820 >> 172.21.15.119:0/2060027820 conn(0x7fd5640fcab0 msgr2=0x7fd5640fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:14.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.257+0000 7fd56a5ff700 1 -- 172.21.15.119:0/2060027820 shutdown_connections 2023-12-07T10:17:14.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.257+0000 7fd56a5ff700 1 -- 172.21.15.119:0/2060027820 wait complete. 2023-12-07T10:17:14.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.258+0000 7fd56a5ff700 1 Processor -- start 2023-12-07T10:17:14.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.258+0000 7fd56a5ff700 1 -- start start 2023-12-07T10:17:14.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.258+0000 7fd56a5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd564101560 0x7fd564199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:14.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.258+0000 7fd56a5ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd564102750 0x7fd564199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:14.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.258+0000 7fd56a5ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5641030f0 0x7fd5641a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:14.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.258+0000 7fd56a5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5641a1260 con 0x7fd564101560 2023-12-07T10:17:14.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.258+0000 7fd56a5ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5641a13a0 con 0x7fd564102750 2023-12-07T10:17:14.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.258+0000 7fd56a5ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5641a1530 con 0x7fd5641030f0 2023-12-07T10:17:14.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.259+0000 7fd563fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd564101560 0x7fd564199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:14.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.259+0000 7fd563fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd564101560 0x7fd564199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45306/0 (socket says 172.21.15.119:45306) 2023-12-07T10:17:14.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.259+0000 7fd563fff700 1 -- 172.21.15.119:0/118780956 learned_addr learned my addr 172.21.15.119:0/118780956 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:14.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.259+0000 7fd55bfff700 1 --2- 172.21.15.119:0/118780956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd564102750 0x7fd564199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:14.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.259+0000 7fd568b9c700 1 --2- 172.21.15.119:0/118780956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5641030f0 0x7fd5641a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:14.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.259+0000 7fd563fff700 1 -- 172.21.15.119:0/118780956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5641030f0 msgr2=0x7fd5641a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:14.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.259+0000 7fd563fff700 1 --2- 172.21.15.119:0/118780956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5641030f0 0x7fd5641a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:14.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.259+0000 7fd563fff700 1 -- 172.21.15.119:0/118780956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd564102750 msgr2=0x7fd564199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:14.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.259+0000 7fd563fff700 1 --2- 172.21.15.119:0/118780956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd564102750 0x7fd564199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:14.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.259+0000 7fd563fff700 1 -- 172.21.15.119:0/118780956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5540095f0 con 0x7fd564101560 2023-12-07T10:17:14.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.259+0000 7fd568b9c700 1 --2- 172.21.15.119:0/118780956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5641030f0 0x7fd5641a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:17:14.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.259+0000 7fd55bfff700 1 --2- 172.21.15.119:0/118780956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd564102750 0x7fd564199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:17:14.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.259+0000 7fd563fff700 1 --2- 172.21.15.119:0/118780956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd564101560 0x7fd564199780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd554006010 tx=0x7fd55400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:14.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.259+0000 7fd561ffb700 1 -- 172.21.15.119:0/118780956 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55401d070 con 0x7fd564101560 2023-12-07T10:17:14.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.259+0000 7fd561ffb700 1 -- 172.21.15.119:0/118780956 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5540054e0 con 0x7fd564101560 2023-12-07T10:17:14.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.259+0000 7fd56a5ff700 1 -- 172.21.15.119:0/118780956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5641a17b0 con 0x7fd564101560 2023-12-07T10:17:14.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.259+0000 7fd561ffb700 1 -- 172.21.15.119:0/118780956 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd554004c10 con 0x7fd564101560 2023-12-07T10:17:14.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.259+0000 7fd56a5ff700 1 -- 172.21.15.119:0/118780956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5641a1bd0 con 0x7fd564101560 2023-12-07T10:17:14.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.261+0000 7fd56a5ff700 1 -- 172.21.15.119:0/118780956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd564068350 con 0x7fd564101560 2023-12-07T10:17:14.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.262+0000 7fd561ffb700 1 -- 172.21.15.119:0/118780956 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fd55400fbb0 con 0x7fd564101560 2023-12-07T10:17:14.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.263+0000 7fd561ffb700 1 --2- 172.21.15.119:0/118780956 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd5500a1b90 0x7fd5500a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:14.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.264+0000 7fd561ffb700 1 -- 172.21.15.119:0/118780956 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd5540cdc30 con 0x7fd564101560 2023-12-07T10:17:14.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.264+0000 7fd55bfff700 1 --2- 172.21.15.119:0/118780956 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd5500a1b90 0x7fd5500a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:14.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.264+0000 7fd55bfff700 1 --2- 172.21.15.119:0/118780956 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd5500a1b90 0x7fd5500a4050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009cb0 tx=0x7fd54c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:14.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.268+0000 7fd561ffb700 1 -- 172.21.15.119:0/118780956 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd5540974e0 con 0x7fd564101560 2023-12-07T10:17:14.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.516+0000 7fd56a5ff700 1 -- 172.21.15.119:0/118780956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5641a1f60 con 0x7fd564101560 2023-12-07T10:17:14.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.517+0000 7fd561ffb700 1 -- 172.21.15.119:0/118780956 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3701 (secure 0 0 0) 0x7fd55409ab00 con 0x7fd564101560 2023-12-07T10:17:14.520 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:17:14.520 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-12-07T10:17:14.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.519+0000 7fd56a5ff700 1 -- 172.21.15.119:0/118780956 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd5500a1b90 msgr2=0x7fd5500a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:14.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.519+0000 7fd56a5ff700 1 --2- 172.21.15.119:0/118780956 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd5500a1b90 0x7fd5500a4050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009cb0 tx=0x7fd54c009260 comp rx=0 tx=0).stop 2023-12-07T10:17:14.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.519+0000 7fd56a5ff700 1 -- 172.21.15.119:0/118780956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd564101560 msgr2=0x7fd564199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:14.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.519+0000 7fd56a5ff700 1 --2- 172.21.15.119:0/118780956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd564101560 0x7fd564199780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd554006010 tx=0x7fd55400f740 comp rx=0 tx=0).stop 2023-12-07T10:17:14.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.519+0000 7fd56a5ff700 1 -- 172.21.15.119:0/118780956 shutdown_connections 2023-12-07T10:17:14.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.519+0000 7fd56a5ff700 1 --2- 172.21.15.119:0/118780956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5641030f0 0x7fd5641a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:14.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.519+0000 7fd56a5ff700 1 --2- 172.21.15.119:0/118780956 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd5500a1b90 0x7fd5500a4050 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:14.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.519+0000 7fd56a5ff700 1 --2- 172.21.15.119:0/118780956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd564101560 0x7fd564199780 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:14.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.519+0000 7fd56a5ff700 1 --2- 172.21.15.119:0/118780956 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd564102750 0x7fd564199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:14.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.519+0000 7fd56a5ff700 1 -- 172.21.15.119:0/118780956 >> 172.21.15.119:0/118780956 conn(0x7fd5640fcab0 msgr2=0x7fd564073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:14.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.519+0000 7fd56a5ff700 1 -- 172.21.15.119:0/118780956 shutdown_connections 2023-12-07T10:17:14.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:14.519+0000 7fd56a5ff700 1 -- 172.21.15.119:0/118780956 wait complete. 2023-12-07T10:17:14.523 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 5 2023-12-07T10:17:14.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:14 smithi159 ceph-mon[134726]: Deploying daemon mds.e on smithi153 2023-12-07T10:17:14.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:14 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.119:6836/184007634,v1:172.21.15.119:6837/184007634] up:boot 2023-12-07T10:17:14.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:14 smithi159 ceph-mon[134726]: fsmap 4 up:standby 2023-12-07T10:17:14.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:14 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-07T10:17:14.769 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:14 smithi153 ceph-mon[133625]: Deploying daemon mds.e on smithi153 2023-12-07T10:17:14.769 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:14 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.119:6836/184007634,v1:172.21.15.119:6837/184007634] up:boot 2023-12-07T10:17:14.769 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:14 smithi153 ceph-mon[133625]: fsmap 4 up:standby 2023-12-07T10:17:14.769 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:14 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-07T10:17:14.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:14 smithi119 ceph-mon[97908]: Deploying daemon mds.e on smithi153 2023-12-07T10:17:14.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:14 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.119:6836/184007634,v1:172.21.15.119:6837/184007634] up:boot 2023-12-07T10:17:14.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:14 smithi119 ceph-mon[97908]: fsmap 4 up:standby 2023-12-07T10:17:14.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:14 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-07T10:17:15.376 INFO:journalctl@ceph.mds.e.smithi153.stdout:Dec 07 10:17:15 smithi153 systemd[1]: Starting Ceph mds.e for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:17:15.641 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:15 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/118780956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:15.641 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:15 smithi153 ceph-mon[133625]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:15.641 INFO:journalctl@ceph.mds.e.smithi153.stdout:Dec 07 10:17:15 smithi153 podman[169743]: 2023-12-07T10:17:15.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:15 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/118780956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:15.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:15 smithi119 ceph-mon[97908]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:16.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:15 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/118780956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:16.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:15 smithi159 ceph-mon[134726]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:16.030 INFO:journalctl@ceph.mds.e.smithi153.stdout:Dec 07 10:17:15 smithi153 ceph-mds[169783]: -- 172.21.15.153:0/2525640655 <== mon.2 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559aa4ea01a0 con 0x559aa4e76c00 2023-12-07T10:17:16.030 INFO:journalctl@ceph.mds.e.smithi153.stdout:Dec 07 10:17:15 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-e[169759]: starting mds.e at 2023-12-07T10:17:16.030 INFO:journalctl@ceph.mds.e.smithi153.stdout:Dec 07 10:17:15 smithi153 bash[169743]: 19ed7a677328b6737c8253e058f42f4d3dcd38dfe27e74bcaa9e9f6508ae2b42 2023-12-07T10:17:16.030 INFO:journalctl@ceph.mds.e.smithi153.stdout:Dec 07 10:17:15 smithi153 systemd[1]: Started Ceph mds.e for a4c642c6-94e8-11ee-95a2-87774f69a715. 2023-12-07T10:17:16.541 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:17:16.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.746+0000 7f2757bff700 1 -- 172.21.15.119:0/654459731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27501030f0 msgr2=0x7f27501099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:16.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.746+0000 7f2757bff700 1 --2- 172.21.15.119:0/654459731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27501030f0 0x7f27501099a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f274800b0d0 tx=0x7f274800b3e0 comp rx=0 tx=0).stop 2023-12-07T10:17:16.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.746+0000 7f2757bff700 1 -- 172.21.15.119:0/654459731 shutdown_connections 2023-12-07T10:17:16.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.746+0000 7f2757bff700 1 --2- 172.21.15.119:0/654459731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27501030f0 0x7f27501099a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:16.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.746+0000 7f2757bff700 1 --2- 172.21.15.119:0/654459731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2750101560 0x7f2750101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:16.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.746+0000 7f2757bff700 1 --2- 172.21.15.119:0/654459731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2750102750 0x7f2750102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:16.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.746+0000 7f2757bff700 1 -- 172.21.15.119:0/654459731 >> 172.21.15.119:0/654459731 conn(0x7f27500fcab0 msgr2=0x7f27500fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:16.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.746+0000 7f2757bff700 1 -- 172.21.15.119:0/654459731 shutdown_connections 2023-12-07T10:17:16.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.746+0000 7f2757bff700 1 -- 172.21.15.119:0/654459731 wait complete. 2023-12-07T10:17:16.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.747+0000 7f2757bff700 1 Processor -- start 2023-12-07T10:17:16.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.747+0000 7f2757bff700 1 -- start start 2023-12-07T10:17:16.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.747+0000 7f2757bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2750101560 0x7f27501997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:16.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.747+0000 7f2757bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2750102750 0x7f2750199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:16.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.747+0000 7f2757bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27501030f0 0x7f27501a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:16.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.747+0000 7f2757bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27501a1310 con 0x7f2750102750 2023-12-07T10:17:16.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.747+0000 7f2757bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27501a1450 con 0x7f27501030f0 2023-12-07T10:17:16.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.747+0000 7f2757bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27501a15e0 con 0x7f2750101560 2023-12-07T10:17:16.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.747+0000 7f275599b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2750101560 0x7f27501997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:16.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.747+0000 7f275599b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2750101560 0x7f27501997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:58724/0 (socket says 172.21.15.119:58724) 2023-12-07T10:17:16.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.747+0000 7f275599b700 1 -- 172.21.15.119:0/2470302962 learned_addr learned my addr 172.21.15.119:0/2470302962 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:16.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.747+0000 7f275519a700 1 --2- 172.21.15.119:0/2470302962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2750102750 0x7f2750199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:16.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.748+0000 7f275619c700 1 --2- 172.21.15.119:0/2470302962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27501030f0 0x7f27501a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:16.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.748+0000 7f275599b700 1 -- 172.21.15.119:0/2470302962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27501030f0 msgr2=0x7f27501a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:16.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.748+0000 7f275599b700 1 --2- 172.21.15.119:0/2470302962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27501030f0 0x7f27501a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:16.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.748+0000 7f275599b700 1 -- 172.21.15.119:0/2470302962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2750102750 msgr2=0x7f2750199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:16.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.748+0000 7f275599b700 1 --2- 172.21.15.119:0/2470302962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2750102750 0x7f2750199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:16.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.748+0000 7f275599b700 1 -- 172.21.15.119:0/2470302962 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2748009d00 con 0x7f2750101560 2023-12-07T10:17:16.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.748+0000 7f275599b700 1 --2- 172.21.15.119:0/2470302962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2750101560 0x7f27501997e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f274c00e970 tx=0x7f274c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:16.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.748+0000 7f2746ffd700 1 -- 172.21.15.119:0/2470302962 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274c009790 con 0x7f2750101560 2023-12-07T10:17:16.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.748+0000 7f2757bff700 1 -- 172.21.15.119:0/2470302962 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27501a18c0 con 0x7f2750101560 2023-12-07T10:17:16.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.749+0000 7f2746ffd700 1 -- 172.21.15.119:0/2470302962 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f274c00c7e0 con 0x7f2750101560 2023-12-07T10:17:16.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.749+0000 7f2746ffd700 1 -- 172.21.15.119:0/2470302962 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274c003c70 con 0x7f2750101560 2023-12-07T10:17:16.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.749+0000 7f2757bff700 1 -- 172.21.15.119:0/2470302962 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27501a1e20 con 0x7f2750101560 2023-12-07T10:17:16.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.749+0000 7f2757bff700 1 -- 172.21.15.119:0/2470302962 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2750068350 con 0x7f2750101560 2023-12-07T10:17:16.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.751+0000 7f2746ffd700 1 -- 172.21.15.119:0/2470302962 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f274c00cc50 con 0x7f2750101560 2023-12-07T10:17:16.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.753+0000 7f2746ffd700 1 --2- 172.21.15.119:0/2470302962 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f273c0a1b10 0x7f273c0a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:16.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.753+0000 7f2746ffd700 1 -- 172.21.15.119:0/2470302962 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f274c014070 con 0x7f2750101560 2023-12-07T10:17:16.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.753+0000 7f275519a700 1 --2- 172.21.15.119:0/2470302962 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f273c0a1b10 0x7f273c0a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:16.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.753+0000 7f275519a700 1 --2- 172.21.15.119:0/2470302962 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f273c0a1b10 0x7f273c0a3fd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2740007be0 tx=0x7f27400073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:16.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:16.757+0000 7f2746ffd700 1 -- 172.21.15.119:0/2470302962 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f274c096b50 con 0x7f2750101560 2023-12-07T10:17:16.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:16 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:16.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:16 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:16.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:16 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:16.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:16 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:16.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:16 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:16.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:16 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:16.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:16 smithi119 ceph-mon[97908]: Deploying daemon mds.f on smithi159 2023-12-07T10:17:16.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:16 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] up:boot 2023-12-07T10:17:16.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:16 smithi119 ceph-mon[97908]: fsmap 5 up:standby 2023-12-07T10:17:16.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:16 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-07T10:17:17.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:17.003+0000 7f2757bff700 1 -- 172.21.15.119:0/2470302962 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f275010cdd0 con 0x7f2750101560 2023-12-07T10:17:17.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:17.004+0000 7f2746ffd700 1 -- 172.21.15.119:0/2470302962 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4495 (secure 0 0 0) 0x7f274c09a170 con 0x7f2750101560 2023-12-07T10:17:17.007 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:17:17.007 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-12-07T10:17:17.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:17.006+0000 7f2757bff700 1 -- 172.21.15.119:0/2470302962 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f273c0a1b10 msgr2=0x7f273c0a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:17.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:17.006+0000 7f2757bff700 1 --2- 172.21.15.119:0/2470302962 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f273c0a1b10 0x7f273c0a3fd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2740007be0 tx=0x7f27400073d0 comp rx=0 tx=0).stop 2023-12-07T10:17:17.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:17.006+0000 7f2757bff700 1 -- 172.21.15.119:0/2470302962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2750101560 msgr2=0x7f27501997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:17.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:17.006+0000 7f2757bff700 1 --2- 172.21.15.119:0/2470302962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2750101560 0x7f27501997e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f274c00e970 tx=0x7f274c00ec80 comp rx=0 tx=0).stop 2023-12-07T10:17:17.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:17.006+0000 7f2757bff700 1 -- 172.21.15.119:0/2470302962 shutdown_connections 2023-12-07T10:17:17.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:17.007+0000 7f2757bff700 1 --2- 172.21.15.119:0/2470302962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27501030f0 0x7f27501a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:17.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:17.007+0000 7f2757bff700 1 --2- 172.21.15.119:0/2470302962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2750101560 0x7f27501997e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:17.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:17.007+0000 7f2757bff700 1 --2- 172.21.15.119:0/2470302962 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f273c0a1b10 0x7f273c0a3fd0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:17.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:17.007+0000 7f2757bff700 1 --2- 172.21.15.119:0/2470302962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2750102750 0x7f2750199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:17.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:17.007+0000 7f2757bff700 1 -- 172.21.15.119:0/2470302962 >> 172.21.15.119:0/2470302962 conn(0x7f27500fcab0 msgr2=0x7f2750073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:17.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:17.007+0000 7f2757bff700 1 -- 172.21.15.119:0/2470302962 shutdown_connections 2023-12-07T10:17:17.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:17.007+0000 7f2757bff700 1 -- 172.21.15.119:0/2470302962 wait complete. 2023-12-07T10:17:17.011 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 6 2023-12-07T10:17:17.149 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:16 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:17.149 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:16 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:17.149 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:16 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:17.149 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:16 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:17.149 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:16 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:17.149 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:16 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:17.149 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:16 smithi159 ceph-mon[134726]: Deploying daemon mds.f on smithi159 2023-12-07T10:17:17.149 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:16 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] up:boot 2023-12-07T10:17:17.149 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:16 smithi159 ceph-mon[134726]: fsmap 5 up:standby 2023-12-07T10:17:17.150 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:16 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-07T10:17:17.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:16 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:17.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:16 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:17.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:16 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:17.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:16 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:17.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:16 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:17.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:16 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:17.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:16 smithi153 ceph-mon[133625]: Deploying daemon mds.f on smithi159 2023-12-07T10:17:17.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:16 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] up:boot 2023-12-07T10:17:17.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:16 smithi153 ceph-mon[133625]: fsmap 5 up:standby 2023-12-07T10:17:17.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:16 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-07T10:17:17.514 INFO:journalctl@ceph.mds.f.smithi159.stdout:Dec 07 10:17:17 smithi159 systemd[1]: Starting Ceph mds.f for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:17:17.961 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:17 smithi159 ceph-mon[134726]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:17.961 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:17 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2470302962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:17.962 INFO:journalctl@ceph.mds.f.smithi159.stdout:Dec 07 10:17:17 smithi159 podman[171894]: 2023-12-07T10:17:17.962 INFO:journalctl@ceph.mds.f.smithi159.stdout:Dec 07 10:17:17 smithi159 ceph-mds[171934]: -- 172.21.15.159:0/3783474918 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55be4d2841a0 con 0x55be4d25ac00 2023-12-07T10:17:18.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:17 smithi119 ceph-mon[97908]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:18.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:17 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2470302962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:18.263 INFO:journalctl@ceph.mds.f.smithi159.stdout:Dec 07 10:17:17 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-f[171910]: starting mds.f at 2023-12-07T10:17:18.263 INFO:journalctl@ceph.mds.f.smithi159.stdout:Dec 07 10:17:17 smithi159 bash[171894]: 6be0ddf23a18c2871bffccdad6d7cf323d0d18b81c6b42f593a6b73e28c39311 2023-12-07T10:17:18.263 INFO:journalctl@ceph.mds.f.smithi159.stdout:Dec 07 10:17:18 smithi159 systemd[1]: Started Ceph mds.f for a4c642c6-94e8-11ee-95a2-87774f69a715. 2023-12-07T10:17:18.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:17 smithi153 ceph-mon[133625]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:18.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:17 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2470302962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:19.029 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:17:19.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.234+0000 7f3a231ff700 1 -- 172.21.15.119:0/1952514001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a1c101560 msgr2=0x7f3a1c101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:19.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.234+0000 7f3a231ff700 1 --2- 172.21.15.119:0/1952514001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a1c101560 0x7f3a1c101980 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3a100099c0 tx=0x7f3a10009cd0 comp rx=0 tx=0).stop 2023-12-07T10:17:19.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.234+0000 7f3a231ff700 1 -- 172.21.15.119:0/1952514001 shutdown_connections 2023-12-07T10:17:19.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.234+0000 7f3a231ff700 1 --2- 172.21.15.119:0/1952514001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a1c1030f0 0x7f3a1c1099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:19.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.234+0000 7f3a231ff700 1 --2- 172.21.15.119:0/1952514001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a1c101560 0x7f3a1c101980 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:19.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.234+0000 7f3a231ff700 1 --2- 172.21.15.119:0/1952514001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a1c102750 0x7f3a1c102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:19.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.234+0000 7f3a231ff700 1 -- 172.21.15.119:0/1952514001 >> 172.21.15.119:0/1952514001 conn(0x7f3a1c0fcab0 msgr2=0x7f3a1c0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:19.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.234+0000 7f3a231ff700 1 -- 172.21.15.119:0/1952514001 shutdown_connections 2023-12-07T10:17:19.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.234+0000 7f3a231ff700 1 -- 172.21.15.119:0/1952514001 wait complete. 2023-12-07T10:17:19.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.235+0000 7f3a231ff700 1 Processor -- start 2023-12-07T10:17:19.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.235+0000 7f3a231ff700 1 -- start start 2023-12-07T10:17:19.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.235+0000 7f3a231ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a1c101560 0x7f3a1c199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:19.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.235+0000 7f3a231ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a1c102750 0x7f3a1c199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:19.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.235+0000 7f3a231ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a1c1030f0 0x7f3a1c1a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:19.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.236+0000 7f3a231ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c1a1260 con 0x7f3a1c101560 2023-12-07T10:17:19.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.236+0000 7f3a231ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c1a13a0 con 0x7f3a1c1030f0 2023-12-07T10:17:19.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.236+0000 7f3a231ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c1a1530 con 0x7f3a1c102750 2023-12-07T10:17:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.236+0000 7f3a20f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a1c101560 0x7f3a1c199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.236+0000 7f3a1bfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a1c102750 0x7f3a1c199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.236+0000 7f3a20f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a1c101560 0x7f3a1c199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45344/0 (socket says 172.21.15.119:45344) 2023-12-07T10:17:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.236+0000 7f3a20f9b700 1 -- 172.21.15.119:0/606205640 learned_addr learned my addr 172.21.15.119:0/606205640 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.236+0000 7f3a2179c700 1 --2- 172.21.15.119:0/606205640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a1c1030f0 0x7f3a1c1a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.236+0000 7f3a20f9b700 1 -- 172.21.15.119:0/606205640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a1c102750 msgr2=0x7f3a1c199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.236+0000 7f3a20f9b700 1 --2- 172.21.15.119:0/606205640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a1c102750 0x7f3a1c199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.236+0000 7f3a20f9b700 1 -- 172.21.15.119:0/606205640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a1c1030f0 msgr2=0x7f3a1c1a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.236+0000 7f3a20f9b700 1 --2- 172.21.15.119:0/606205640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a1c1030f0 0x7f3a1c1a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.236+0000 7f3a20f9b700 1 -- 172.21.15.119:0/606205640 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a100095f0 con 0x7f3a1c101560 2023-12-07T10:17:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.236+0000 7f3a1bfff700 1 --2- 172.21.15.119:0/606205640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a1c102750 0x7f3a1c199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:17:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.236+0000 7f3a2179c700 1 --2- 172.21.15.119:0/606205640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a1c1030f0 0x7f3a1c1a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:17:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.236+0000 7f3a20f9b700 1 --2- 172.21.15.119:0/606205640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a1c101560 0x7f3a1c199730 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3a10003730 tx=0x7f3a1000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.237+0000 7f3a19ffb700 1 -- 172.21.15.119:0/606205640 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1001d070 con 0x7f3a1c101560 2023-12-07T10:17:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.237+0000 7f3a19ffb700 1 -- 172.21.15.119:0/606205640 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a1000fe70 con 0x7f3a1c101560 2023-12-07T10:17:19.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.237+0000 7f3a19ffb700 1 -- 172.21.15.119:0/606205640 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a100173f0 con 0x7f3a1c101560 2023-12-07T10:17:19.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.237+0000 7f3a231ff700 1 -- 172.21.15.119:0/606205640 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1c1a17b0 con 0x7f3a1c101560 2023-12-07T10:17:19.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.237+0000 7f3a231ff700 1 -- 172.21.15.119:0/606205640 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1c1a1b80 con 0x7f3a1c101560 2023-12-07T10:17:19.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.238+0000 7f3a231ff700 1 -- 172.21.15.119:0/606205640 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a1c068350 con 0x7f3a1c101560 2023-12-07T10:17:19.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.240+0000 7f3a19ffb700 1 -- 172.21.15.119:0/606205640 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f3a10022470 con 0x7f3a1c101560 2023-12-07T10:17:19.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.241+0000 7f3a19ffb700 1 --2- 172.21.15.119:0/606205640 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3a0c0a1b10 0x7f3a0c0a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:19.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.241+0000 7f3a19ffb700 1 -- 172.21.15.119:0/606205640 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3a100ce3b0 con 0x7f3a1c101560 2023-12-07T10:17:19.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.241+0000 7f3a1bfff700 1 --2- 172.21.15.119:0/606205640 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3a0c0a1b10 0x7f3a0c0a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:19.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.242+0000 7f3a1bfff700 1 --2- 172.21.15.119:0/606205640 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3a0c0a1b10 0x7f3a0c0a3fd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3a08011490 tx=0x7f3a08009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:19.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.245+0000 7f3a19ffb700 1 -- 172.21.15.119:0/606205640 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3a10097c60 con 0x7f3a1c101560 2023-12-07T10:17:19.372 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:19 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:19.372 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:19 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:19.372 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:19 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:19.372 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:19 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:19.373 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:19 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:19.373 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:19 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:19.373 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:19 smithi159 ceph-mon[134726]: Deploying daemon mds.c on smithi159 2023-12-07T10:17:19.373 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:19 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] up:boot 2023-12-07T10:17:19.373 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:19 smithi159 ceph-mon[134726]: fsmap 6 up:standby 2023-12-07T10:17:19.373 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:19 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-07T10:17:19.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:19 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:19.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:19 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:19.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:19 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:19.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:19 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:19.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:19 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:19.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:19 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:19.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:19 smithi119 ceph-mon[97908]: Deploying daemon mds.c on smithi159 2023-12-07T10:17:19.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:19 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] up:boot 2023-12-07T10:17:19.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:19 smithi119 ceph-mon[97908]: fsmap 6 up:standby 2023-12-07T10:17:19.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:19 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-07T10:17:19.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.492+0000 7f3a231ff700 1 -- 172.21.15.119:0/606205640 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a1c10d000 con 0x7f3a1c101560 2023-12-07T10:17:19.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.492+0000 7f3a19ffb700 1 -- 172.21.15.119:0/606205640 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5289 (secure 0 0 0) 0x7f3a1009b280 con 0x7f3a1c101560 2023-12-07T10:17:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:17:19.496 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2023-12-07T10:17:19.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.495+0000 7f3a231ff700 1 -- 172.21.15.119:0/606205640 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3a0c0a1b10 msgr2=0x7f3a0c0a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:19.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.495+0000 7f3a231ff700 1 --2- 172.21.15.119:0/606205640 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3a0c0a1b10 0x7f3a0c0a3fd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3a08011490 tx=0x7f3a08009260 comp rx=0 tx=0).stop 2023-12-07T10:17:19.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.495+0000 7f3a231ff700 1 -- 172.21.15.119:0/606205640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a1c101560 msgr2=0x7f3a1c199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:19.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.495+0000 7f3a231ff700 1 --2- 172.21.15.119:0/606205640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a1c101560 0x7f3a1c199730 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3a10003730 tx=0x7f3a1000fa00 comp rx=0 tx=0).stop 2023-12-07T10:17:19.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.495+0000 7f3a231ff700 1 -- 172.21.15.119:0/606205640 shutdown_connections 2023-12-07T10:17:19.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.495+0000 7f3a231ff700 1 --2- 172.21.15.119:0/606205640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a1c1030f0 0x7f3a1c1a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:19.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.495+0000 7f3a231ff700 1 --2- 172.21.15.119:0/606205640 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3a0c0a1b10 0x7f3a0c0a3fd0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:19.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.495+0000 7f3a231ff700 1 --2- 172.21.15.119:0/606205640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a1c101560 0x7f3a1c199730 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:19.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.495+0000 7f3a231ff700 1 --2- 172.21.15.119:0/606205640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a1c102750 0x7f3a1c199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:19.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.495+0000 7f3a231ff700 1 -- 172.21.15.119:0/606205640 >> 172.21.15.119:0/606205640 conn(0x7f3a1c0fcab0 msgr2=0x7f3a1c107f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:19.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.495+0000 7f3a231ff700 1 -- 172.21.15.119:0/606205640 shutdown_connections 2023-12-07T10:17:19.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:19.495+0000 7f3a231ff700 1 -- 172.21.15.119:0/606205640 wait complete. 2023-12-07T10:17:19.499 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 7 2023-12-07T10:17:19.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:19 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:19.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:19 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:19.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:19 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:19.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:19 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:19.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:19 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:19.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:19 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:19.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:19 smithi153 ceph-mon[133625]: Deploying daemon mds.c on smithi159 2023-12-07T10:17:19.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:19 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] up:boot 2023-12-07T10:17:19.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:19 smithi153 ceph-mon[133625]: fsmap 6 up:standby 2023-12-07T10:17:19.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:19 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-07T10:17:19.994 INFO:journalctl@ceph.mds.c.smithi159.stdout:Dec 07 10:17:19 smithi159 systemd[1]: Starting Ceph mds.c for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:17:20.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:20 smithi159 ceph-mon[134726]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:20.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:20 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/606205640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:20.264 INFO:journalctl@ceph.mds.c.smithi159.stdout:Dec 07 10:17:19 smithi159 podman[172234]: 2023-12-07T10:17:20.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:20 smithi119 ceph-mon[97908]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:20.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:20 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/606205640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:20.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:20 smithi153 ceph-mon[133625]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:20.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:20 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/606205640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:20.763 INFO:journalctl@ceph.mds.c.smithi159.stdout:Dec 07 10:17:20 smithi159 ceph-mds[172274]: -- 172.21.15.159:0/1298843804 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559abda321a0 con 0x559abda08c00 2023-12-07T10:17:20.763 INFO:journalctl@ceph.mds.c.smithi159.stdout:Dec 07 10:17:20 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-c[172250]: starting mds.c at 2023-12-07T10:17:20.764 INFO:journalctl@ceph.mds.c.smithi159.stdout:Dec 07 10:17:20 smithi159 bash[172234]: 64599de510fc50d2e1e6abb8564b9ff76e3d66470f890bf14ef13a84c58393d4 2023-12-07T10:17:20.764 INFO:journalctl@ceph.mds.c.smithi159.stdout:Dec 07 10:17:20 smithi159 systemd[1]: Started Ceph mds.c for a4c642c6-94e8-11ee-95a2-87774f69a715. 2023-12-07T10:17:21.518 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:17:21.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.723+0000 7f25411ff700 1 -- 172.21.15.119:0/2370019079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f253c101560 msgr2=0x7f253c101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:21.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.723+0000 7f25411ff700 1 --2- 172.21.15.119:0/2370019079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f253c101560 0x7f253c101980 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f252c0099c0 tx=0x7f252c009cd0 comp rx=0 tx=0).stop 2023-12-07T10:17:21.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.724+0000 7f25411ff700 1 -- 172.21.15.119:0/2370019079 shutdown_connections 2023-12-07T10:17:21.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.724+0000 7f25411ff700 1 --2- 172.21.15.119:0/2370019079 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f253c1030f0 0x7f253c1099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:21.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.724+0000 7f25411ff700 1 --2- 172.21.15.119:0/2370019079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f253c101560 0x7f253c101980 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:21.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.724+0000 7f25411ff700 1 --2- 172.21.15.119:0/2370019079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f253c102750 0x7f253c102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:21.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.724+0000 7f25411ff700 1 -- 172.21.15.119:0/2370019079 >> 172.21.15.119:0/2370019079 conn(0x7f253c0fcab0 msgr2=0x7f253c0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:21.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.724+0000 7f25411ff700 1 -- 172.21.15.119:0/2370019079 shutdown_connections 2023-12-07T10:17:21.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.724+0000 7f25411ff700 1 -- 172.21.15.119:0/2370019079 wait complete. 2023-12-07T10:17:21.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.725+0000 7f25411ff700 1 Processor -- start 2023-12-07T10:17:21.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.725+0000 7f25411ff700 1 -- start start 2023-12-07T10:17:21.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.726+0000 7f25411ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f253c101560 0x7f253c199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:21.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.726+0000 7f25411ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f253c102750 0x7f253c199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:21.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.726+0000 7f25411ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f253c1030f0 0x7f253c1a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:21.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.726+0000 7f25411ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c1a1260 con 0x7f253c101560 2023-12-07T10:17:21.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.726+0000 7f25411ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c1a13a0 con 0x7f253c102750 2023-12-07T10:17:21.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.726+0000 7f25411ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c1a1530 con 0x7f253c1030f0 2023-12-07T10:17:21.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.726+0000 7f253ad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f253c101560 0x7f253c199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:21.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.726+0000 7f253ad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f253c101560 0x7f253c199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45360/0 (socket says 172.21.15.119:45360) 2023-12-07T10:17:21.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.726+0000 7f253ad9d700 1 -- 172.21.15.119:0/3187966036 learned_addr learned my addr 172.21.15.119:0/3187966036 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:21.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.726+0000 7f253b59e700 1 --2- 172.21.15.119:0/3187966036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f253c1030f0 0x7f253c1a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:21.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.726+0000 7f2533fff700 1 --2- 172.21.15.119:0/3187966036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f253c102750 0x7f253c199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:21.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.726+0000 7f253ad9d700 1 -- 172.21.15.119:0/3187966036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f253c1030f0 msgr2=0x7f253c1a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:21.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.726+0000 7f253ad9d700 1 --2- 172.21.15.119:0/3187966036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f253c1030f0 0x7f253c1a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:21.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.726+0000 7f253ad9d700 1 -- 172.21.15.119:0/3187966036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f253c102750 msgr2=0x7f253c199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:21.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.726+0000 7f253ad9d700 1 --2- 172.21.15.119:0/3187966036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f253c102750 0x7f253c199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:21.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.726+0000 7f253ad9d700 1 -- 172.21.15.119:0/3187966036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252c0095f0 con 0x7f253c101560 2023-12-07T10:17:21.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.727+0000 7f253ad9d700 1 --2- 172.21.15.119:0/3187966036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f253c101560 0x7f253c199730 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f252c006010 tx=0x7f252c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:21.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.727+0000 7f2538d99700 1 -- 172.21.15.119:0/3187966036 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c01d070 con 0x7f253c101560 2023-12-07T10:17:21.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.727+0000 7f2538d99700 1 -- 172.21.15.119:0/3187966036 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f252c005020 con 0x7f253c101560 2023-12-07T10:17:21.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.727+0000 7f25411ff700 1 -- 172.21.15.119:0/3187966036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253c1a17b0 con 0x7f253c101560 2023-12-07T10:17:21.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.727+0000 7f2538d99700 1 -- 172.21.15.119:0/3187966036 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c004b10 con 0x7f253c101560 2023-12-07T10:17:21.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:21 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:21.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:21 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:21.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:21 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:21.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:21 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:21.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:21 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:21.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:21 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:21.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:21 smithi119 ceph-mon[97908]: Deploying daemon mds.h on smithi153 2023-12-07T10:17:21.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:21 smithi119 ceph-mon[97908]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:21.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:21 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] up:boot 2023-12-07T10:17:21.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:21 smithi119 ceph-mon[97908]: fsmap 7 up:standby 2023-12-07T10:17:21.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:21 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-07T10:17:21.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.727+0000 7f25411ff700 1 -- 172.21.15.119:0/3187966036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253c1a1bd0 con 0x7f253c101560 2023-12-07T10:17:21.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.728+0000 7f25411ff700 1 -- 172.21.15.119:0/3187966036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f253c068350 con 0x7f253c101560 2023-12-07T10:17:21.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.730+0000 7f2538d99700 1 -- 172.21.15.119:0/3187966036 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f252c005190 con 0x7f253c101560 2023-12-07T10:17:21.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.731+0000 7f2538d99700 1 --2- 172.21.15.119:0/3187966036 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f25280a1b90 0x7f25280a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:21.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.731+0000 7f2538d99700 1 -- 172.21.15.119:0/3187966036 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f252c0cdb00 con 0x7f253c101560 2023-12-07T10:17:21.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.732+0000 7f2533fff700 1 --2- 172.21.15.119:0/3187966036 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f25280a1b90 0x7f25280a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:21.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.732+0000 7f2533fff700 1 --2- 172.21.15.119:0/3187966036 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f25280a1b90 0x7f25280a4050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2524009c60 tx=0x7f2524009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:21.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.736+0000 7f2538d99700 1 -- 172.21.15.119:0/3187966036 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f252c0cfe60 con 0x7f253c101560 2023-12-07T10:17:21.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:21 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:21.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:21 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:21.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:21 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:21.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:21 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:21.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:21 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:21.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:21 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:21.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:21 smithi159 ceph-mon[134726]: Deploying daemon mds.h on smithi153 2023-12-07T10:17:21.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:21 smithi159 ceph-mon[134726]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:21.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:21 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] up:boot 2023-12-07T10:17:21.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:21 smithi159 ceph-mon[134726]: fsmap 7 up:standby 2023-12-07T10:17:21.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:21 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-07T10:17:21.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:21 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:21.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:21 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:21.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:21 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:21.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:21 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:21.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:21 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:21.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:21 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:21.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:21 smithi153 ceph-mon[133625]: Deploying daemon mds.h on smithi153 2023-12-07T10:17:21.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:21 smithi153 ceph-mon[133625]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:21.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:21 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] up:boot 2023-12-07T10:17:21.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:21 smithi153 ceph-mon[133625]: fsmap 7 up:standby 2023-12-07T10:17:21.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:21 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-07T10:17:21.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.983+0000 7f25411ff700 1 -- 172.21.15.119:0/3187966036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f253c10cde0 con 0x7f253c101560 2023-12-07T10:17:21.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.984+0000 7f2538d99700 1 -- 172.21.15.119:0/3187966036 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6083 (secure 0 0 0) 0x7f252c0973b0 con 0x7f253c101560 2023-12-07T10:17:21.986 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:17:21.986 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-12-07T10:17:21.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.986+0000 7f25411ff700 1 -- 172.21.15.119:0/3187966036 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f25280a1b90 msgr2=0x7f25280a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:21.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.986+0000 7f25411ff700 1 --2- 172.21.15.119:0/3187966036 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f25280a1b90 0x7f25280a4050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2524009c60 tx=0x7f2524009260 comp rx=0 tx=0).stop 2023-12-07T10:17:21.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.986+0000 7f25411ff700 1 -- 172.21.15.119:0/3187966036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f253c101560 msgr2=0x7f253c199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:21.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.986+0000 7f25411ff700 1 --2- 172.21.15.119:0/3187966036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f253c101560 0x7f253c199730 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f252c006010 tx=0x7f252c00f690 comp rx=0 tx=0).stop 2023-12-07T10:17:21.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.987+0000 7f25411ff700 1 -- 172.21.15.119:0/3187966036 shutdown_connections 2023-12-07T10:17:21.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.987+0000 7f25411ff700 1 --2- 172.21.15.119:0/3187966036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f253c1030f0 0x7f253c1a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:21.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.987+0000 7f25411ff700 1 --2- 172.21.15.119:0/3187966036 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f25280a1b90 0x7f25280a4050 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:21.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.987+0000 7f25411ff700 1 --2- 172.21.15.119:0/3187966036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f253c101560 0x7f253c199730 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:21.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.987+0000 7f25411ff700 1 --2- 172.21.15.119:0/3187966036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f253c102750 0x7f253c199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:21.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.987+0000 7f25411ff700 1 -- 172.21.15.119:0/3187966036 >> 172.21.15.119:0/3187966036 conn(0x7f253c0fcab0 msgr2=0x7f253c107f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:21.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.987+0000 7f25411ff700 1 -- 172.21.15.119:0/3187966036 shutdown_connections 2023-12-07T10:17:21.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:21.987+0000 7f25411ff700 1 -- 172.21.15.119:0/3187966036 wait complete. 2023-12-07T10:17:21.990 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 8 2023-12-07T10:17:22.280 INFO:journalctl@ceph.mds.h.smithi153.stdout:Dec 07 10:17:22 smithi153 systemd[1]: Starting Ceph mds.h for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:17:22.630 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:22 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3187966036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:22.631 INFO:journalctl@ceph.mds.h.smithi153.stdout:Dec 07 10:17:22 smithi153 podman[170087]: 2023-12-07T10:17:22.631 INFO:journalctl@ceph.mds.h.smithi153.stdout:Dec 07 10:17:22 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-h[170103]: starting mds.h at 2023-12-07T10:17:22.631 INFO:journalctl@ceph.mds.h.smithi153.stdout:Dec 07 10:17:22 smithi153 bash[170087]: b7582912e5e5e6c1d75050f0e0e1b3de491a45611f3f0ca12662eab70382f2a1 2023-12-07T10:17:22.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:22 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3187966036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:22.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:22 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3187966036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:23.030 INFO:journalctl@ceph.mds.h.smithi153.stdout:Dec 07 10:17:22 smithi153 systemd[1]: Started Ceph mds.h for a4c642c6-94e8-11ee-95a2-87774f69a715. 2023-12-07T10:17:23.963 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:23 smithi159 ceph-mon[134726]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:23.963 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:23 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:23.963 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:23 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:23.963 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:23 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:23.963 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:23 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:23.963 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:23 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:23.964 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:23 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:23.964 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:23 smithi159 ceph-mon[134726]: Deploying daemon mds.i on smithi159 2023-12-07T10:17:23.964 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:23 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.153:6836/2925503246,v1:172.21.15.153:6837/2925503246] up:boot 2023-12-07T10:17:23.964 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:23 smithi159 ceph-mon[134726]: fsmap 8 up:standby 2023-12-07T10:17:23.964 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:23 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-07T10:17:23.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:23 smithi119 ceph-mon[97908]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:23.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:23 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:23.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:23 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:23.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:23 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:23.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:23 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:23.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:23 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:23.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:23 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:23.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:23 smithi119 ceph-mon[97908]: Deploying daemon mds.i on smithi159 2023-12-07T10:17:23.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:23 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.153:6836/2925503246,v1:172.21.15.153:6837/2925503246] up:boot 2023-12-07T10:17:23.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:23 smithi119 ceph-mon[97908]: fsmap 8 up:standby 2023-12-07T10:17:23.981 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:23 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-07T10:17:24.008 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:17:24.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:23 smithi153 ceph-mon[133625]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:24.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:23 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:24.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:23 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:24.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:23 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:24.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:23 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:24.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:23 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:24.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:23 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:24.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:23 smithi153 ceph-mon[133625]: Deploying daemon mds.i on smithi159 2023-12-07T10:17:24.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:23 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.153:6836/2925503246,v1:172.21.15.153:6837/2925503246] up:boot 2023-12-07T10:17:24.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:23 smithi153 ceph-mon[133625]: fsmap 8 up:standby 2023-12-07T10:17:24.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:23 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-07T10:17:24.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.214+0000 7f700fdff700 1 -- 172.21.15.119:0/4076853892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7008101560 msgr2=0x7f7008101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:24.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.214+0000 7f700fdff700 1 --2- 172.21.15.119:0/4076853892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7008101560 0x7f7008101980 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7004009990 tx=0x7f7004009ca0 comp rx=0 tx=0).stop 2023-12-07T10:17:24.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.214+0000 7f700fdff700 1 -- 172.21.15.119:0/4076853892 shutdown_connections 2023-12-07T10:17:24.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.214+0000 7f700fdff700 1 --2- 172.21.15.119:0/4076853892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70081030f0 0x7f70081099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:24.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.214+0000 7f700fdff700 1 --2- 172.21.15.119:0/4076853892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7008101560 0x7f7008101980 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:24.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.214+0000 7f700fdff700 1 --2- 172.21.15.119:0/4076853892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7008102750 0x7f7008102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:24.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.214+0000 7f700fdff700 1 -- 172.21.15.119:0/4076853892 >> 172.21.15.119:0/4076853892 conn(0x7f70080fcab0 msgr2=0x7f70080fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:24.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.214+0000 7f700fdff700 1 -- 172.21.15.119:0/4076853892 shutdown_connections 2023-12-07T10:17:24.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.214+0000 7f700fdff700 1 -- 172.21.15.119:0/4076853892 wait complete. 2023-12-07T10:17:24.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.215+0000 7f700fdff700 1 Processor -- start 2023-12-07T10:17:24.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.215+0000 7f700fdff700 1 -- start start 2023-12-07T10:17:24.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.216+0000 7f700fdff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7008101560 0x7f70081996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:24.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.216+0000 7f700fdff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7008102750 0x7f7008199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:24.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.216+0000 7f700fdff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70081030f0 0x7f70081a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:24.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.216+0000 7f700fdff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70081a1220 con 0x7f70081030f0 2023-12-07T10:17:24.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.216+0000 7f700fdff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70081a1360 con 0x7f7008101560 2023-12-07T10:17:24.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.216+0000 7f700fdff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70081a14f0 con 0x7f7008102750 2023-12-07T10:17:24.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.216+0000 7f700d39a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7008102750 0x7f7008199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:24.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.216+0000 7f700d39a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7008102750 0x7f7008199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:42040/0 (socket says 172.21.15.119:42040) 2023-12-07T10:17:24.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.216+0000 7f700d39a700 1 -- 172.21.15.119:0/3257840156 learned_addr learned my addr 172.21.15.119:0/3257840156 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:24.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.216+0000 7f700e39c700 1 --2- 172.21.15.119:0/3257840156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70081030f0 0x7f70081a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:24.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.216+0000 7f700db9b700 1 --2- 172.21.15.119:0/3257840156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7008101560 0x7f70081996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:24.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.216+0000 7f700d39a700 1 -- 172.21.15.119:0/3257840156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7008101560 msgr2=0x7f70081996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:24.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.216+0000 7f700d39a700 1 --2- 172.21.15.119:0/3257840156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7008101560 0x7f70081996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:24.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.216+0000 7f700d39a700 1 -- 172.21.15.119:0/3257840156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70081030f0 msgr2=0x7f70081a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:24.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.216+0000 7f700d39a700 1 --2- 172.21.15.119:0/3257840156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70081030f0 0x7f70081a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:24.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.216+0000 7f700d39a700 1 -- 172.21.15.119:0/3257840156 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff8009520 con 0x7f7008102750 2023-12-07T10:17:24.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.216+0000 7f700d39a700 1 --2- 172.21.15.119:0/3257840156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7008102750 0x7f7008199c30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800eb30 tx=0x7f6ff800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:24.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.217+0000 7f6ffeffd700 1 -- 172.21.15.119:0/3257840156 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8009880 con 0x7f7008102750 2023-12-07T10:17:24.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.217+0000 7f700fdff700 1 -- 172.21.15.119:0/3257840156 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70040095f0 con 0x7f7008102750 2023-12-07T10:17:24.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.217+0000 7f6ffeffd700 1 -- 172.21.15.119:0/3257840156 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ff80044e0 con 0x7f7008102750 2023-12-07T10:17:24.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.217+0000 7f6ffeffd700 1 -- 172.21.15.119:0/3257840156 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8010430 con 0x7f7008102750 2023-12-07T10:17:24.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.217+0000 7f700fdff700 1 -- 172.21.15.119:0/3257840156 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70081a1b50 con 0x7f7008102750 2023-12-07T10:17:24.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.218+0000 7f700fdff700 1 -- 172.21.15.119:0/3257840156 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7008068350 con 0x7f7008102750 2023-12-07T10:17:24.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.220+0000 7f6ffeffd700 1 -- 172.21.15.119:0/3257840156 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f6ff8003ee0 con 0x7f7008102750 2023-12-07T10:17:24.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.221+0000 7f6ffeffd700 1 --2- 172.21.15.119:0/3257840156 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6ff40a1b10 0x7f6ff40a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:24.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.221+0000 7f6ffeffd700 1 -- 172.21.15.119:0/3257840156 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6ff8014070 con 0x7f7008102750 2023-12-07T10:17:24.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.221+0000 7f700db9b700 1 --2- 172.21.15.119:0/3257840156 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6ff40a1b10 0x7f6ff40a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:24.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.222+0000 7f700db9b700 1 --2- 172.21.15.119:0/3257840156 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6ff40a1b10 0x7f6ff40a3fd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f700400b7b0 tx=0x7f7004005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:24.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.232+0000 7f6ffeffd700 1 -- 172.21.15.119:0/3257840156 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f6ff8097040 con 0x7f7008102750 2023-12-07T10:17:24.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.479+0000 7f700fdff700 1 -- 172.21.15.119:0/3257840156 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f700810cd80 con 0x7f7008102750 2023-12-07T10:17:24.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.480+0000 7f6ffeffd700 1 -- 172.21.15.119:0/3257840156 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6877 (secure 0 0 0) 0x7f6ff80ba090 con 0x7f7008102750 2023-12-07T10:17:24.483 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:17:24.483 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2925503246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2925503246},{"type":"v1","addr":"172.21.15.153:6837","nonce":2925503246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-12-07T10:17:24.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.483+0000 7f700fdff700 1 -- 172.21.15.119:0/3257840156 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6ff40a1b10 msgr2=0x7f6ff40a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:24.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.483+0000 7f700fdff700 1 --2- 172.21.15.119:0/3257840156 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6ff40a1b10 0x7f6ff40a3fd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f700400b7b0 tx=0x7f7004005fb0 comp rx=0 tx=0).stop 2023-12-07T10:17:24.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.483+0000 7f700fdff700 1 -- 172.21.15.119:0/3257840156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7008102750 msgr2=0x7f7008199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:24.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.483+0000 7f700fdff700 1 --2- 172.21.15.119:0/3257840156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7008102750 0x7f7008199c30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800eb30 tx=0x7f6ff800eef0 comp rx=0 tx=0).stop 2023-12-07T10:17:24.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.483+0000 7f700fdff700 1 -- 172.21.15.119:0/3257840156 shutdown_connections 2023-12-07T10:17:24.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.483+0000 7f700fdff700 1 --2- 172.21.15.119:0/3257840156 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6ff40a1b10 0x7f6ff40a3fd0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:24.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.483+0000 7f700fdff700 1 --2- 172.21.15.119:0/3257840156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70081030f0 0x7f70081a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:24.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.483+0000 7f700fdff700 1 --2- 172.21.15.119:0/3257840156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7008101560 0x7f70081996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:24.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.483+0000 7f700fdff700 1 --2- 172.21.15.119:0/3257840156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7008102750 0x7f7008199c30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:24.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.483+0000 7f700fdff700 1 -- 172.21.15.119:0/3257840156 >> 172.21.15.119:0/3257840156 conn(0x7f70080fcab0 msgr2=0x7f7008107f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:24.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.483+0000 7f700fdff700 1 -- 172.21.15.119:0/3257840156 shutdown_connections 2023-12-07T10:17:24.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:24.483+0000 7f700fdff700 1 -- 172.21.15.119:0/3257840156 wait complete. 2023-12-07T10:17:24.488 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 9 2023-12-07T10:17:24.568 INFO:journalctl@ceph.mds.i.smithi159.stdout:Dec 07 10:17:24 smithi159 systemd[1]: Starting Ceph mds.i for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:17:24.850 INFO:journalctl@ceph.mds.i.smithi159.stdout:Dec 07 10:17:24 smithi159 podman[172569]: 2023-12-07T10:17:24.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:24 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3257840156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:24.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:24 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3257840156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:25.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:24 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3257840156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:25.263 INFO:journalctl@ceph.mds.i.smithi159.stdout:Dec 07 10:17:24 smithi159 ceph-mds[172609]: -- 172.21.15.159:0/1825419403 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562df2bfe1a0 con 0x562df2bd5000 2023-12-07T10:17:25.263 INFO:journalctl@ceph.mds.i.smithi159.stdout:Dec 07 10:17:24 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-i[172585]: starting mds.i at 2023-12-07T10:17:25.263 INFO:journalctl@ceph.mds.i.smithi159.stdout:Dec 07 10:17:24 smithi159 bash[172569]: bb11777dcfded4b4a0973f2748ec719daddda78c775bd19242b5abb52c2a174d 2023-12-07T10:17:25.263 INFO:journalctl@ceph.mds.i.smithi159.stdout:Dec 07 10:17:24 smithi159 systemd[1]: Started Ceph mds.i for a4c642c6-94e8-11ee-95a2-87774f69a715. 2023-12-07T10:17:26.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:26 smithi159 ceph-mon[134726]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:26.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:26 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:26.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:26 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:26.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:26 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:26.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:26 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:26.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:26 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:26.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:26 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:26.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:26 smithi159 ceph-mon[134726]: Deploying daemon mds.g on smithi119 2023-12-07T10:17:26.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:26 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.159:6838/3984962900,v1:172.21.15.159:6839/3984962900] up:boot 2023-12-07T10:17:26.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:26 smithi159 ceph-mon[134726]: fsmap 9 up:standby 2023-12-07T10:17:26.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:26 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-07T10:17:26.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:26 smithi153 ceph-mon[133625]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:26.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:26 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:26.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:26 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:26.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:26 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:26.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:26 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:26.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:26 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:26.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:26 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:26.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:26 smithi153 ceph-mon[133625]: Deploying daemon mds.g on smithi119 2023-12-07T10:17:26.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:26 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.159:6838/3984962900,v1:172.21.15.159:6839/3984962900] up:boot 2023-12-07T10:17:26.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:26 smithi153 ceph-mon[133625]: fsmap 9 up:standby 2023-12-07T10:17:26.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:26 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-07T10:17:26.353 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:26 smithi119 ceph-mon[97908]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:26.353 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:26 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:26.353 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:26 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:26.353 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:26 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:26.354 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:26 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:26.354 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:26 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:26.354 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:26 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:26.354 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:26 smithi119 ceph-mon[97908]: Deploying daemon mds.g on smithi119 2023-12-07T10:17:26.354 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:26 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.159:6838/3984962900,v1:172.21.15.159:6839/3984962900] up:boot 2023-12-07T10:17:26.354 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:26 smithi119 ceph-mon[97908]: fsmap 9 up:standby 2023-12-07T10:17:26.354 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:26 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-07T10:17:26.506 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:17:26.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.713+0000 7f79f55ff700 1 -- 172.21.15.119:0/3955873788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79f0075650 msgr2=0x7f79f0073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:26.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.713+0000 7f79f55ff700 1 --2- 172.21.15.119:0/3955873788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79f0075650 0x7f79f0073aa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f79e00098f0 tx=0x7f79e0009c00 comp rx=0 tx=0).stop 2023-12-07T10:17:26.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.713+0000 7f79f55ff700 1 -- 172.21.15.119:0/3955873788 shutdown_connections 2023-12-07T10:17:26.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.713+0000 7f79f55ff700 1 --2- 172.21.15.119:0/3955873788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79f010db30 0x7f79f010ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:26.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.713+0000 7f79f55ff700 1 --2- 172.21.15.119:0/3955873788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79f0075650 0x7f79f0073aa0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:26.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.713+0000 7f79f55ff700 1 --2- 172.21.15.119:0/3955873788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79f0074070 0x7f79f00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:26.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.713+0000 7f79f55ff700 1 -- 172.21.15.119:0/3955873788 >> 172.21.15.119:0/3955873788 conn(0x7f79f00fcab0 msgr2=0x7f79f00fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:26.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.713+0000 7f79f55ff700 1 -- 172.21.15.119:0/3955873788 shutdown_connections 2023-12-07T10:17:26.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.713+0000 7f79f55ff700 1 -- 172.21.15.119:0/3955873788 wait complete. 2023-12-07T10:17:26.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.714+0000 7f79f55ff700 1 Processor -- start 2023-12-07T10:17:26.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.714+0000 7f79f55ff700 1 -- start start 2023-12-07T10:17:26.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.715+0000 7f79f55ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79f0074070 0x7f79f019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.715+0000 7f79f55ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79f0075650 0x7f79f019e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.715+0000 7f79f55ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79f010db30 0x7f79f01a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.715+0000 7f79f55ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f01a56b0 con 0x7f79f010db30 2023-12-07T10:17:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.715+0000 7f79f55ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f01a57f0 con 0x7f79f0074070 2023-12-07T10:17:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.715+0000 7f79f55ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f01a5980 con 0x7f79f0075650 2023-12-07T10:17:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.715+0000 7f79eeffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79f0074070 0x7f79f019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.715+0000 7f79eeffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79f0074070 0x7f79f019db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:53794/0 (socket says 172.21.15.119:53794) 2023-12-07T10:17:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.715+0000 7f79eeffd700 1 -- 172.21.15.119:0/1885083924 learned_addr learned my addr 172.21.15.119:0/1885083924 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.715+0000 7f79ef7fe700 1 --2- 172.21.15.119:0/1885083924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79f010db30 0x7f79f01a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.715+0000 7f79ee7fc700 1 --2- 172.21.15.119:0/1885083924 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79f0075650 0x7f79f019e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.715+0000 7f79eeffd700 1 -- 172.21.15.119:0/1885083924 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79f0075650 msgr2=0x7f79f019e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:26.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.715+0000 7f79eeffd700 1 --2- 172.21.15.119:0/1885083924 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79f0075650 0x7f79f019e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:26.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.715+0000 7f79eeffd700 1 -- 172.21.15.119:0/1885083924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79f010db30 msgr2=0x7f79f01a50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:26.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.715+0000 7f79eeffd700 1 --2- 172.21.15.119:0/1885083924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79f010db30 0x7f79f01a50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:26.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.715+0000 7f79eeffd700 1 -- 172.21.15.119:0/1885083924 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e0009520 con 0x7f79f0074070 2023-12-07T10:17:26.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.716+0000 7f79eeffd700 1 --2- 172.21.15.119:0/1885083924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79f0074070 0x7f79f019db80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f79e0009810 tx=0x7f79e000b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:26.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.716+0000 7f79d7fff700 1 -- 172.21.15.119:0/1885083924 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e001c070 con 0x7f79f0074070 2023-12-07T10:17:26.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.716+0000 7f79f55ff700 1 -- 172.21.15.119:0/1885083924 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f01a5c00 con 0x7f79f0074070 2023-12-07T10:17:26.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.716+0000 7f79d7fff700 1 -- 172.21.15.119:0/1885083924 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79e000f7d0 con 0x7f79f0074070 2023-12-07T10:17:26.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.716+0000 7f79f55ff700 1 -- 172.21.15.119:0/1885083924 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f01a6080 con 0x7f79f0074070 2023-12-07T10:17:26.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.716+0000 7f79d7fff700 1 -- 172.21.15.119:0/1885083924 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0003970 con 0x7f79f0074070 2023-12-07T10:17:26.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.717+0000 7f79f55ff700 1 -- 172.21.15.119:0/1885083924 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f0103d90 con 0x7f79f0074070 2023-12-07T10:17:26.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.719+0000 7f79d7fff700 1 -- 172.21.15.119:0/1885083924 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f79e0020030 con 0x7f79f0074070 2023-12-07T10:17:26.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.720+0000 7f79d7fff700 1 --2- 172.21.15.119:0/1885083924 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f79d80a1b90 0x7f79d80a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:26.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.721+0000 7f79d7fff700 1 -- 172.21.15.119:0/1885083924 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f79e00ce060 con 0x7f79f0074070 2023-12-07T10:17:26.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.721+0000 7f79ee7fc700 1 --2- 172.21.15.119:0/1885083924 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f79d80a1b90 0x7f79d80a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:26.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.721+0000 7f79ee7fc700 1 --2- 172.21.15.119:0/1885083924 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f79d80a1b90 0x7f79d80a4050 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f79f0075370 tx=0x7f79e4009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:26.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.723+0000 7f79d7fff700 1 -- 172.21.15.119:0/1885083924 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f79e0097910 con 0x7f79f0074070 2023-12-07T10:17:26.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.994+0000 7f79f55ff700 1 -- 172.21.15.119:0/1885083924 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79f0068350 con 0x7f79f0074070 2023-12-07T10:17:26.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.996+0000 7f79d7fff700 1 -- 172.21.15.119:0/1885083924 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7673 (secure 0 0 0) 0x7f79e009af30 con 0x7f79f0074070 2023-12-07T10:17:26.998 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:17:26.998 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2925503246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2925503246},{"type":"v1","addr":"172.21.15.153:6837","nonce":2925503246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/3984962900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3984962900},{"type":"v1","addr":"172.21.15.159:6839","nonce":3984962900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-12-07T10:17:27.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.998+0000 7f79f55ff700 1 -- 172.21.15.119:0/1885083924 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f79d80a1b90 msgr2=0x7f79d80a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:27.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.998+0000 7f79f55ff700 1 --2- 172.21.15.119:0/1885083924 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f79d80a1b90 0x7f79d80a4050 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f79f0075370 tx=0x7f79e4009310 comp rx=0 tx=0).stop 2023-12-07T10:17:27.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.998+0000 7f79f55ff700 1 -- 172.21.15.119:0/1885083924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79f0074070 msgr2=0x7f79f019db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:27.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.998+0000 7f79f55ff700 1 --2- 172.21.15.119:0/1885083924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79f0074070 0x7f79f019db80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f79e0009810 tx=0x7f79e000b790 comp rx=0 tx=0).stop 2023-12-07T10:17:27.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.998+0000 7f79f55ff700 1 -- 172.21.15.119:0/1885083924 shutdown_connections 2023-12-07T10:17:27.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.998+0000 7f79f55ff700 1 --2- 172.21.15.119:0/1885083924 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f79d80a1b90 0x7f79d80a4050 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:27.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.998+0000 7f79f55ff700 1 --2- 172.21.15.119:0/1885083924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79f010db30 0x7f79f01a50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:27.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.998+0000 7f79f55ff700 1 --2- 172.21.15.119:0/1885083924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79f0074070 0x7f79f019db80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:27.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.998+0000 7f79f55ff700 1 --2- 172.21.15.119:0/1885083924 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79f0075650 0x7f79f019e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:27.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.998+0000 7f79f55ff700 1 -- 172.21.15.119:0/1885083924 >> 172.21.15.119:0/1885083924 conn(0x7f79f00fcab0 msgr2=0x7f79f0100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:27.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.998+0000 7f79f55ff700 1 -- 172.21.15.119:0/1885083924 shutdown_connections 2023-12-07T10:17:27.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:26.998+0000 7f79f55ff700 1 -- 172.21.15.119:0/1885083924 wait complete. 2023-12-07T10:17:27.001 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 10 2023-12-07T10:17:27.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:27 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1885083924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:27.334 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:27 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1885083924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:27.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:27 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1885083924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:27.729 INFO:journalctl@ceph.mds.g.smithi119.stdout:Dec 07 10:17:27 smithi119 systemd[1]: Starting Ceph mds.g for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:17:28.108 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:28 smithi119 ceph-mon[97908]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:28.108 INFO:journalctl@ceph.mds.g.smithi119.stdout:Dec 07 10:17:27 smithi119 podman[146335]: 2023-12-07T10:17:28.108 INFO:journalctl@ceph.mds.g.smithi119.stdout:Dec 07 10:17:28 smithi119 bash[146335]: e6b70bca1e3e8c5f9ad2a0bd33fbb64279314d177a5ef71f07ae8c3d67481db1 2023-12-07T10:17:28.108 INFO:journalctl@ceph.mds.g.smithi119.stdout:Dec 07 10:17:28 smithi119 ceph-mds[146375]: -- 172.21.15.119:0/1015439385 <== mon.0 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fcae41a1a0 con 0x55fcae3f2800 2023-12-07T10:17:28.109 INFO:journalctl@ceph.mds.g.smithi119.stdout:Dec 07 10:17:28 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-g[146351]: starting mds.g at 2023-12-07T10:17:28.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:28 smithi153 ceph-mon[133625]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:28.479 INFO:journalctl@ceph.mds.g.smithi119.stdout:Dec 07 10:17:28 smithi119 systemd[1]: Started Ceph mds.g for a4c642c6-94e8-11ee-95a2-87774f69a715. 2023-12-07T10:17:28.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:28 smithi159 ceph-mon[134726]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:29.022 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:17:29.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.225+0000 7effa0fff700 1 -- 172.21.15.119:0/490033701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff9c1030f0 msgr2=0x7eff9c1099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:29.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.225+0000 7effa0fff700 1 --2- 172.21.15.119:0/490033701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff9c1030f0 0x7eff9c1099a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7eff9400b0d0 tx=0x7eff9400b3e0 comp rx=0 tx=0).stop 2023-12-07T10:17:29.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.225+0000 7effa0fff700 1 -- 172.21.15.119:0/490033701 shutdown_connections 2023-12-07T10:17:29.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.225+0000 7effa0fff700 1 --2- 172.21.15.119:0/490033701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff9c1030f0 0x7eff9c1099a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:29.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.225+0000 7effa0fff700 1 --2- 172.21.15.119:0/490033701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff9c101560 0x7eff9c101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:29.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.225+0000 7effa0fff700 1 --2- 172.21.15.119:0/490033701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff9c102750 0x7eff9c102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:29.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.225+0000 7effa0fff700 1 -- 172.21.15.119:0/490033701 >> 172.21.15.119:0/490033701 conn(0x7eff9c0fcab0 msgr2=0x7eff9c0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:29.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.225+0000 7effa0fff700 1 -- 172.21.15.119:0/490033701 shutdown_connections 2023-12-07T10:17:29.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.225+0000 7effa0fff700 1 -- 172.21.15.119:0/490033701 wait complete. 2023-12-07T10:17:29.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.227+0000 7effa0fff700 1 Processor -- start 2023-12-07T10:17:29.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.227+0000 7effa0fff700 1 -- start start 2023-12-07T10:17:29.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.227+0000 7effa0fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff9c101560 0x7eff9c075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:29.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.227+0000 7effa0fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff9c102750 0x7eff9c073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:29.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.227+0000 7effa0fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff9c1030f0 0x7eff9c073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:29.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.227+0000 7effa0fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c0746e0 con 0x7eff9c1030f0 2023-12-07T10:17:29.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.227+0000 7effa0fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c074850 con 0x7eff9c101560 2023-12-07T10:17:29.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.227+0000 7effa0fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c1a1640 con 0x7eff9c102750 2023-12-07T10:17:29.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.228+0000 7eff9a59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff9c101560 0x7eff9c075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:29.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.228+0000 7eff9ad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff9c1030f0 0x7eff9c073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:29.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.228+0000 7eff9a59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff9c101560 0x7eff9c075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:53834/0 (socket says 172.21.15.119:53834) 2023-12-07T10:17:29.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.228+0000 7eff9a59c700 1 -- 172.21.15.119:0/4240155686 learned_addr learned my addr 172.21.15.119:0/4240155686 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:29.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.228+0000 7eff9ad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff9c1030f0 0x7eff9c073f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49794/0 (socket says 172.21.15.119:49794) 2023-12-07T10:17:29.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.228+0000 7eff99d9b700 1 --2- 172.21.15.119:0/4240155686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff9c102750 0x7eff9c073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:29.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.228+0000 7eff9a59c700 1 -- 172.21.15.119:0/4240155686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff9c102750 msgr2=0x7eff9c073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:29.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.228+0000 7eff9a59c700 1 --2- 172.21.15.119:0/4240155686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff9c102750 0x7eff9c073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:29.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.228+0000 7eff9a59c700 1 -- 172.21.15.119:0/4240155686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff9c1030f0 msgr2=0x7eff9c073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:29.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.228+0000 7eff9a59c700 1 --2- 172.21.15.119:0/4240155686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff9c1030f0 0x7eff9c073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:29.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.228+0000 7eff9a59c700 1 -- 172.21.15.119:0/4240155686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff94009d00 con 0x7eff9c101560 2023-12-07T10:17:29.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.228+0000 7eff99d9b700 1 --2- 172.21.15.119:0/4240155686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff9c102750 0x7eff9c073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:17:29.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.228+0000 7eff9ad9d700 1 --2- 172.21.15.119:0/4240155686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff9c1030f0 0x7eff9c073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:17:29.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.229+0000 7eff9a59c700 1 --2- 172.21.15.119:0/4240155686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff9c101560 0x7eff9c075370 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00e970 tx=0x7eff8c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:29.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.231+0000 7eff8b7fe700 1 -- 172.21.15.119:0/4240155686 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c009790 con 0x7eff9c101560 2023-12-07T10:17:29.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.231+0000 7eff8b7fe700 1 -- 172.21.15.119:0/4240155686 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff8c00eec0 con 0x7eff9c101560 2023-12-07T10:17:29.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.231+0000 7effa0fff700 1 -- 172.21.15.119:0/4240155686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff9c1a18d0 con 0x7eff9c101560 2023-12-07T10:17:29.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.231+0000 7eff8b7fe700 1 -- 172.21.15.119:0/4240155686 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c003c90 con 0x7eff9c101560 2023-12-07T10:17:29.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.231+0000 7effa0fff700 1 -- 172.21.15.119:0/4240155686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff9c1a1de0 con 0x7eff9c101560 2023-12-07T10:17:29.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.232+0000 7effa0fff700 1 -- 172.21.15.119:0/4240155686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff9c108960 con 0x7eff9c101560 2023-12-07T10:17:29.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.234+0000 7eff8b7fe700 1 -- 172.21.15.119:0/4240155686 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7eff8c00cc70 con 0x7eff9c101560 2023-12-07T10:17:29.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.235+0000 7eff8b7fe700 1 --2- 172.21.15.119:0/4240155686 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7eff840a1c60 0x7eff840a4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:29.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.236+0000 7eff8b7fe700 1 -- 172.21.15.119:0/4240155686 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7eff8c014070 con 0x7eff9c101560 2023-12-07T10:17:29.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.236+0000 7eff99d9b700 1 --2- 172.21.15.119:0/4240155686 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7eff840a1c60 0x7eff840a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:29.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.236+0000 7eff99d9b700 1 --2- 172.21.15.119:0/4240155686 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7eff840a1c60 0x7eff840a4120 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7eff90009cb0 tx=0x7eff90009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:29.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.239+0000 7eff8b7fe700 1 -- 172.21.15.119:0/4240155686 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7eff8c096ab0 con 0x7eff9c101560 2023-12-07T10:17:29.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:29 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:29.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:29 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:29.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:29 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:29.396 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:29 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:29.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:29 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:29.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:29 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:29.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:29 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] up:boot 2023-12-07T10:17:29.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:29 smithi119 ceph-mon[97908]: fsmap 10 up:standby 2023-12-07T10:17:29.397 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:29 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-07T10:17:29.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.502+0000 7effa0fff700 1 -- 172.21.15.119:0/4240155686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff9c068350 con 0x7eff9c101560 2023-12-07T10:17:29.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.503+0000 7eff8b7fe700 1 -- 172.21.15.119:0/4240155686 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8468 (secure 0 0 0) 0x7eff8c09a0d0 con 0x7eff9c101560 2023-12-07T10:17:29.506 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:17:29.506 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2925503246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2925503246},{"type":"v1","addr":"172.21.15.153:6837","nonce":2925503246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/3984962900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3984962900},{"type":"v1","addr":"172.21.15.159:6839","nonce":3984962900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24542,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6839/1920762602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1920762602},{"type":"v1","addr":"172.21.15.119:6839","nonce":1920762602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[]} 2023-12-07T10:17:29.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.506+0000 7effa0fff700 1 -- 172.21.15.119:0/4240155686 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7eff840a1c60 msgr2=0x7eff840a4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:29.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.506+0000 7effa0fff700 1 --2- 172.21.15.119:0/4240155686 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7eff840a1c60 0x7eff840a4120 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7eff90009cb0 tx=0x7eff90009260 comp rx=0 tx=0).stop 2023-12-07T10:17:29.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.506+0000 7effa0fff700 1 -- 172.21.15.119:0/4240155686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff9c101560 msgr2=0x7eff9c075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:29.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.506+0000 7effa0fff700 1 --2- 172.21.15.119:0/4240155686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff9c101560 0x7eff9c075370 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00e970 tx=0x7eff8c00ec80 comp rx=0 tx=0).stop 2023-12-07T10:17:29.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.506+0000 7effa0fff700 1 -- 172.21.15.119:0/4240155686 shutdown_connections 2023-12-07T10:17:29.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.506+0000 7effa0fff700 1 --2- 172.21.15.119:0/4240155686 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7eff840a1c60 0x7eff840a4120 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:29.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.506+0000 7effa0fff700 1 --2- 172.21.15.119:0/4240155686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff9c1030f0 0x7eff9c073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:29.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.506+0000 7effa0fff700 1 --2- 172.21.15.119:0/4240155686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff9c101560 0x7eff9c075370 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:29.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.506+0000 7effa0fff700 1 --2- 172.21.15.119:0/4240155686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff9c102750 0x7eff9c073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:29.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.506+0000 7effa0fff700 1 -- 172.21.15.119:0/4240155686 >> 172.21.15.119:0/4240155686 conn(0x7eff9c0fcab0 msgr2=0x7eff9c105aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:29.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.506+0000 7effa0fff700 1 -- 172.21.15.119:0/4240155686 shutdown_connections 2023-12-07T10:17:29.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:29.506+0000 7effa0fff700 1 -- 172.21.15.119:0/4240155686 wait complete. 2023-12-07T10:17:29.510 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 11 2023-12-07T10:17:29.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:29 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:29.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:29 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:29.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:29 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:29.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:29 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:29.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:29 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:29.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:29 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:29.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:29 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] up:boot 2023-12-07T10:17:29.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:29 smithi159 ceph-mon[134726]: fsmap 10 up:standby 2023-12-07T10:17:29.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:29 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-07T10:17:29.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:29 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:29.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:29 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:29.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:29 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:29.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:29 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:29.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:29 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:29.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:29 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:29.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:29 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] up:boot 2023-12-07T10:17:29.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:29 smithi153 ceph-mon[133625]: fsmap 10 up:standby 2023-12-07T10:17:29.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:29 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-07T10:17:30.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:30 smithi119 ceph-mon[97908]: Deploying daemon mds.a on smithi119 2023-12-07T10:17:30.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:30 smithi119 ceph-mon[97908]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:30.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:30 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/4240155686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:30.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:30 smithi159 ceph-mon[134726]: Deploying daemon mds.a on smithi119 2023-12-07T10:17:30.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:30 smithi159 ceph-mon[134726]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:30.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:30 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/4240155686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:30.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:30 smithi153 ceph-mon[133625]: Deploying daemon mds.a on smithi119 2023-12-07T10:17:30.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:30 smithi153 ceph-mon[133625]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:30.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:30 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/4240155686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:30.980 INFO:journalctl@ceph.mds.a.smithi119.stdout:Dec 07 10:17:30 smithi119 systemd[1]: Starting Ceph mds.a for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:17:31.285 INFO:journalctl@ceph.mds.a.smithi119.stdout:Dec 07 10:17:31 smithi119 podman[146720]: 2023-12-07T10:17:31.529 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:17:31.592 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:31 smithi119 ceph-mon[97908]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:31.592 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:31 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:31.592 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:31 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:31.592 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:31 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:31.592 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:31 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:31.592 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:31 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:31.592 INFO:journalctl@ceph.mds.a.smithi119.stdout:Dec 07 10:17:31 smithi119 bash[146720]: 192ab581164456335f2e9d71b605f593bc14ef04ae728a2a2709a6827d408425 2023-12-07T10:17:31.593 INFO:journalctl@ceph.mds.a.smithi119.stdout:Dec 07 10:17:31 smithi119 ceph-mds[146760]: -- 172.21.15.119:0/1156830975 <== mon.0 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5625e29f81a0 con 0x5625e29d1000 2023-12-07T10:17:31.593 INFO:journalctl@ceph.mds.a.smithi119.stdout:Dec 07 10:17:31 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-a[146736]: starting mds.a at 2023-12-07T10:17:31.593 INFO:journalctl@ceph.mds.a.smithi119.stdout:Dec 07 10:17:31 smithi119 systemd[1]: Started Ceph mds.a for a4c642c6-94e8-11ee-95a2-87774f69a715. 2023-12-07T10:17:31.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.728+0000 7f5c3a3ff700 1 -- 172.21.15.119:0/30599632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c341030f0 msgr2=0x7f5c341099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:31.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.728+0000 7f5c3a3ff700 1 --2- 172.21.15.119:0/30599632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c341030f0 0x7f5c341099a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00b0d0 tx=0x7f5c2c00b3e0 comp rx=0 tx=0).stop 2023-12-07T10:17:31.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.729+0000 7f5c3a3ff700 1 -- 172.21.15.119:0/30599632 shutdown_connections 2023-12-07T10:17:31.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.729+0000 7f5c3a3ff700 1 --2- 172.21.15.119:0/30599632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c341030f0 0x7f5c341099a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:31.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.729+0000 7f5c3a3ff700 1 --2- 172.21.15.119:0/30599632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c34101560 0x7f5c34101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:31.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.729+0000 7f5c3a3ff700 1 --2- 172.21.15.119:0/30599632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c34102750 0x7f5c34102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:31.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.729+0000 7f5c3a3ff700 1 -- 172.21.15.119:0/30599632 >> 172.21.15.119:0/30599632 conn(0x7f5c340fcab0 msgr2=0x7f5c340fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:31.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.729+0000 7f5c3a3ff700 1 -- 172.21.15.119:0/30599632 shutdown_connections 2023-12-07T10:17:31.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.729+0000 7f5c3a3ff700 1 -- 172.21.15.119:0/30599632 wait complete. 2023-12-07T10:17:31.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.730+0000 7f5c3a3ff700 1 Processor -- start 2023-12-07T10:17:31.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.730+0000 7f5c3a3ff700 1 -- start start 2023-12-07T10:17:31.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.731+0000 7f5c3a3ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c34101560 0x7f5c34199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:31.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.731+0000 7f5c3a3ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c34102750 0x7f5c34199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:31.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.731+0000 7f5c3a3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c341030f0 0x7f5c341a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:31.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.731+0000 7f5c3a3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c341a1330 con 0x7f5c341030f0 2023-12-07T10:17:31.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.731+0000 7f5c3a3ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c341a1470 con 0x7f5c34101560 2023-12-07T10:17:31.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.731+0000 7f5c3a3ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c341a1600 con 0x7f5c34102750 2023-12-07T10:17:31.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.731+0000 7f5c3899c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c341030f0 0x7f5c341a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:31.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.731+0000 7f5c33fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c34101560 0x7f5c34199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:31.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.731+0000 7f5c3899c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c341030f0 0x7f5c341a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49816/0 (socket says 172.21.15.119:49816) 2023-12-07T10:17:31.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.731+0000 7f5c33fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c34101560 0x7f5c34199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:53868/0 (socket says 172.21.15.119:53868) 2023-12-07T10:17:31.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.731+0000 7f5c3899c700 1 -- 172.21.15.119:0/2186960493 learned_addr learned my addr 172.21.15.119:0/2186960493 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:31.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.731+0000 7f5c337fe700 1 --2- 172.21.15.119:0/2186960493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c34102750 0x7f5c34199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:31.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.731+0000 7f5c3899c700 1 -- 172.21.15.119:0/2186960493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c34102750 msgr2=0x7f5c34199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:31.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.731+0000 7f5c3899c700 1 --2- 172.21.15.119:0/2186960493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c34102750 0x7f5c34199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:31.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.731+0000 7f5c3899c700 1 -- 172.21.15.119:0/2186960493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c34101560 msgr2=0x7f5c34199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:31.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.731+0000 7f5c3899c700 1 --2- 172.21.15.119:0/2186960493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c34101560 0x7f5c34199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:31.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.731+0000 7f5c3899c700 1 -- 172.21.15.119:0/2186960493 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2c009d00 con 0x7f5c341030f0 2023-12-07T10:17:31.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.732+0000 7f5c3899c700 1 --2- 172.21.15.119:0/2186960493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c341030f0 0x7f5c341a0d60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c007b60 tx=0x7f5c2c008e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:31.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.732+0000 7f5c317fa700 1 -- 172.21.15.119:0/2186960493 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c00e040 con 0x7f5c341030f0 2023-12-07T10:17:31.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.732+0000 7f5c317fa700 1 -- 172.21.15.119:0/2186960493 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c2c01d930 con 0x7f5c341030f0 2023-12-07T10:17:31.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.732+0000 7f5c3a3ff700 1 -- 172.21.15.119:0/2186960493 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c341a1880 con 0x7f5c341030f0 2023-12-07T10:17:31.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.732+0000 7f5c317fa700 1 -- 172.21.15.119:0/2186960493 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c01b9f0 con 0x7f5c341030f0 2023-12-07T10:17:31.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.732+0000 7f5c3a3ff700 1 -- 172.21.15.119:0/2186960493 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c341a1d50 con 0x7f5c341030f0 2023-12-07T10:17:31.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.733+0000 7f5c3a3ff700 1 -- 172.21.15.119:0/2186960493 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c34068350 con 0x7f5c341030f0 2023-12-07T10:17:31.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.735+0000 7f5c317fa700 1 -- 172.21.15.119:0/2186960493 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f5c2c01daa0 con 0x7f5c341030f0 2023-12-07T10:17:31.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.736+0000 7f5c317fa700 1 --2- 172.21.15.119:0/2186960493 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5c1c0a1b90 0x7f5c1c0a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:31.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.736+0000 7f5c317fa700 1 -- 172.21.15.119:0/2186960493 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5c2c0ce4e0 con 0x7f5c341030f0 2023-12-07T10:17:31.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.736+0000 7f5c33fff700 1 --2- 172.21.15.119:0/2186960493 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5c1c0a1b90 0x7f5c1c0a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:31.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.737+0000 7f5c33fff700 1 --2- 172.21.15.119:0/2186960493 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5c1c0a1b90 0x7f5c1c0a4050 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5c341025b0 tx=0x7f5c24009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:31.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.740+0000 7f5c317fa700 1 -- 172.21.15.119:0/2186960493 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5c2c097d90 con 0x7f5c341030f0 2023-12-07T10:17:31.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:31 smithi159 ceph-mon[134726]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:31.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:31 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:31.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:31 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:31.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:31 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:31.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:31 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:31.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:31 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:31.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:31 smithi153 ceph-mon[133625]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:31.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:31 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:31.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:31 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:31.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:31 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:31.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:31 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-07T10:17:31.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:31 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-07T10:17:31.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.985+0000 7f5c3a3ff700 1 -- 172.21.15.119:0/2186960493 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c3410cdd0 con 0x7f5c341030f0 2023-12-07T10:17:31.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.985+0000 7f5c317fa700 1 -- 172.21.15.119:0/2186960493 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9263 (secure 0 0 0) 0x7f5c2c09b3b0 con 0x7f5c341030f0 2023-12-07T10:17:31.987 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:17:31.988 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14535,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6841/1791998028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1791998028},{"type":"v1","addr":"172.21.15.119:6841","nonce":1791998028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2925503246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2925503246},{"type":"v1","addr":"172.21.15.153:6837","nonce":2925503246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/3984962900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3984962900},{"type":"v1","addr":"172.21.15.159:6839","nonce":3984962900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24542,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6839/1920762602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1920762602},{"type":"v1","addr":"172.21.15.119:6839","nonce":1920762602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[]} 2023-12-07T10:17:31.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.988+0000 7f5c3a3ff700 1 -- 172.21.15.119:0/2186960493 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5c1c0a1b90 msgr2=0x7f5c1c0a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:31.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.988+0000 7f5c3a3ff700 1 --2- 172.21.15.119:0/2186960493 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5c1c0a1b90 0x7f5c1c0a4050 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5c341025b0 tx=0x7f5c24009260 comp rx=0 tx=0).stop 2023-12-07T10:17:31.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.988+0000 7f5c3a3ff700 1 -- 172.21.15.119:0/2186960493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c341030f0 msgr2=0x7f5c341a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:31.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.988+0000 7f5c3a3ff700 1 --2- 172.21.15.119:0/2186960493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c341030f0 0x7f5c341a0d60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c007b60 tx=0x7f5c2c008e40 comp rx=0 tx=0).stop 2023-12-07T10:17:31.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.988+0000 7f5c3a3ff700 1 -- 172.21.15.119:0/2186960493 shutdown_connections 2023-12-07T10:17:31.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.988+0000 7f5c3a3ff700 1 --2- 172.21.15.119:0/2186960493 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5c1c0a1b90 0x7f5c1c0a4050 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:31.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.988+0000 7f5c3a3ff700 1 --2- 172.21.15.119:0/2186960493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c341030f0 0x7f5c341a0d60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:31.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.988+0000 7f5c3a3ff700 1 --2- 172.21.15.119:0/2186960493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c34101560 0x7f5c34199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:31.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.988+0000 7f5c3a3ff700 1 --2- 172.21.15.119:0/2186960493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c34102750 0x7f5c34199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:31.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.988+0000 7f5c3a3ff700 1 -- 172.21.15.119:0/2186960493 >> 172.21.15.119:0/2186960493 conn(0x7f5c340fcab0 msgr2=0x7f5c34073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:31.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.988+0000 7f5c3a3ff700 1 -- 172.21.15.119:0/2186960493 shutdown_connections 2023-12-07T10:17:31.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:31.988+0000 7f5c3a3ff700 1 -- 172.21.15.119:0/2186960493 wait complete. 2023-12-07T10:17:31.991 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 12 2023-12-07T10:17:32.760 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:32 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:32.760 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:32 smithi153 ceph-mon[133625]: Deploying daemon mds.b on smithi153 2023-12-07T10:17:32.760 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:32 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.119:6840/1791998028,v1:172.21.15.119:6841/1791998028] up:boot 2023-12-07T10:17:32.760 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:32 smithi153 ceph-mon[133625]: fsmap 11 up:standby 2023-12-07T10:17:32.760 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:32 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-07T10:17:32.760 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:32 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2186960493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:32.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:32 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:32.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:32 smithi159 ceph-mon[134726]: Deploying daemon mds.b on smithi153 2023-12-07T10:17:32.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:32 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.119:6840/1791998028,v1:172.21.15.119:6841/1791998028] up:boot 2023-12-07T10:17:32.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:32 smithi159 ceph-mon[134726]: fsmap 11 up:standby 2023-12-07T10:17:32.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:32 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-07T10:17:32.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:32 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2186960493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:32.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:32 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:32.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:32 smithi119 ceph-mon[97908]: Deploying daemon mds.b on smithi153 2023-12-07T10:17:32.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:32 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.119:6840/1791998028,v1:172.21.15.119:6841/1791998028] up:boot 2023-12-07T10:17:32.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:32 smithi119 ceph-mon[97908]: fsmap 11 up:standby 2023-12-07T10:17:32.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:32 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-07T10:17:32.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:32 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2186960493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:33.030 INFO:journalctl@ceph.mds.b.smithi153.stdout:Dec 07 10:17:33 smithi153 systemd[1]: Starting Ceph mds.b for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:17:33.593 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:33 smithi153 ceph-mon[133625]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:33.593 INFO:journalctl@ceph.mds.b.smithi153.stdout:Dec 07 10:17:33 smithi153 podman[170426]: 2023-12-07T10:17:33.593 INFO:journalctl@ceph.mds.b.smithi153.stdout:Dec 07 10:17:33 smithi153 ceph-mds[170466]: -- 172.21.15.153:0/109436054 <== mon.2 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f4a4c021a0 con 0x55f4a4bd8800 2023-12-07T10:17:33.593 INFO:journalctl@ceph.mds.b.smithi153.stdout:Dec 07 10:17:33 smithi153 bash[170426]: 828dc654c7574ec64305b12c392b17bd66376735d5a05158d6bd9f96a4da9589 2023-12-07T10:17:33.593 INFO:journalctl@ceph.mds.b.smithi153.stdout:Dec 07 10:17:33 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-b[170442]: starting mds.b at 2023-12-07T10:17:33.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:33 smithi159 ceph-mon[134726]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:33.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:33 smithi119 ceph-mon[97908]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:34.011 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:17:34.030 INFO:journalctl@ceph.mds.b.smithi153.stdout:Dec 07 10:17:33 smithi153 systemd[1]: Started Ceph mds.b for a4c642c6-94e8-11ee-95a2-87774f69a715. 2023-12-07T10:17:34.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.212+0000 7f921cdff700 1 -- 172.21.15.119:0/1294275631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9218074070 msgr2=0x7f92180744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:34.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.212+0000 7f921cdff700 1 --2- 172.21.15.119:0/1294275631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9218074070 0x7f92180744f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f920c009990 tx=0x7f920c009ca0 comp rx=0 tx=0).stop 2023-12-07T10:17:34.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.212+0000 7f921cdff700 1 -- 172.21.15.119:0/1294275631 shutdown_connections 2023-12-07T10:17:34.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.212+0000 7f921cdff700 1 --2- 172.21.15.119:0/1294275631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9218109420 0x7f921810b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:34.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.212+0000 7f921cdff700 1 --2- 172.21.15.119:0/1294275631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9218075650 0x7f9218073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:34.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.212+0000 7f921cdff700 1 --2- 172.21.15.119:0/1294275631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9218074070 0x7f92180744f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:34.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.212+0000 7f921cdff700 1 -- 172.21.15.119:0/1294275631 >> 172.21.15.119:0/1294275631 conn(0x7f92180fcab0 msgr2=0x7f92180fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:34.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.212+0000 7f921cdff700 1 -- 172.21.15.119:0/1294275631 shutdown_connections 2023-12-07T10:17:34.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.212+0000 7f921cdff700 1 -- 172.21.15.119:0/1294275631 wait complete. 2023-12-07T10:17:34.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.213+0000 7f921cdff700 1 Processor -- start 2023-12-07T10:17:34.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.213+0000 7f921cdff700 1 -- start start 2023-12-07T10:17:34.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.214+0000 7f921cdff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9218074070 0x7f9218199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:34.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.214+0000 7f921cdff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9218075650 0x7f9218199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:34.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.214+0000 7f921cdff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9218109420 0x7f92181a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:34.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.214+0000 7f921cdff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92181a1260 con 0x7f9218074070 2023-12-07T10:17:34.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.214+0000 7f921cdff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92181a13a0 con 0x7f9218109420 2023-12-07T10:17:34.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.214+0000 7f921cdff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92181a1530 con 0x7f9218075650 2023-12-07T10:17:34.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.214+0000 7f9215d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9218075650 0x7f9218199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:34.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.214+0000 7f921659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9218074070 0x7f9218199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:34.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.214+0000 7f9215d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9218075650 0x7f9218199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:46048/0 (socket says 172.21.15.119:46048) 2023-12-07T10:17:34.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.214+0000 7f9215d9b700 1 -- 172.21.15.119:0/1416747131 learned_addr learned my addr 172.21.15.119:0/1416747131 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:34.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.214+0000 7f9216d9d700 1 --2- 172.21.15.119:0/1416747131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9218109420 0x7f92181a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:34.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.214+0000 7f9215d9b700 1 -- 172.21.15.119:0/1416747131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9218109420 msgr2=0x7f92181a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:34.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.214+0000 7f9215d9b700 1 --2- 172.21.15.119:0/1416747131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9218109420 0x7f92181a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:34.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.214+0000 7f9215d9b700 1 -- 172.21.15.119:0/1416747131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9218074070 msgr2=0x7f9218199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:34.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.214+0000 7f9215d9b700 1 --2- 172.21.15.119:0/1416747131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9218074070 0x7f9218199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:34.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.214+0000 7f9215d9b700 1 -- 172.21.15.119:0/1416747131 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920c0095f0 con 0x7f9218075650 2023-12-07T10:17:34.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.215+0000 7f9215d9b700 1 --2- 172.21.15.119:0/1416747131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9218075650 0x7f9218199cc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f920c000c00 tx=0x7f920c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:34.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.215+0000 7f92077fe700 1 -- 172.21.15.119:0/1416747131 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c01d070 con 0x7f9218075650 2023-12-07T10:17:34.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.215+0000 7f921cdff700 1 -- 172.21.15.119:0/1416747131 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92181a17b0 con 0x7f9218075650 2023-12-07T10:17:34.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.215+0000 7f92077fe700 1 -- 172.21.15.119:0/1416747131 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f920c021410 con 0x7f9218075650 2023-12-07T10:17:34.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.215+0000 7f92077fe700 1 -- 172.21.15.119:0/1416747131 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c017650 con 0x7f9218075650 2023-12-07T10:17:34.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.215+0000 7f921cdff700 1 -- 172.21.15.119:0/1416747131 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92181a1bd0 con 0x7f9218075650 2023-12-07T10:17:34.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.216+0000 7f921cdff700 1 -- 172.21.15.119:0/1416747131 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92181081e0 con 0x7f9218075650 2023-12-07T10:17:34.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.218+0000 7f92077fe700 1 -- 172.21.15.119:0/1416747131 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f920c021580 con 0x7f9218075650 2023-12-07T10:17:34.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.219+0000 7f92077fe700 1 --2- 172.21.15.119:0/1416747131 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f92000a1b10 0x7f92000a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:34.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.219+0000 7f92077fe700 1 -- 172.21.15.119:0/1416747131 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f920c0ce610 con 0x7f9218075650 2023-12-07T10:17:34.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.219+0000 7f921659c700 1 --2- 172.21.15.119:0/1416747131 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f92000a1b10 0x7f92000a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:34.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.220+0000 7f921659c700 1 --2- 172.21.15.119:0/1416747131 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f92000a1b10 0x7f92000a3fd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9208009c60 tx=0x7f9208009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:34.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.221+0000 7f92077fe700 1 -- 172.21.15.119:0/1416747131 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f920c097ec0 con 0x7f9218075650 2023-12-07T10:17:34.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.479+0000 7f921cdff700 1 -- 172.21.15.119:0/1416747131 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9218068350 con 0x7f9218075650 2023-12-07T10:17:34.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.481+0000 7f92077fe700 1 -- 172.21.15.119:0/1416747131 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9263 (secure 0 0 0) 0x7f920c09b4e0 con 0x7f9218075650 2023-12-07T10:17:34.483 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:17:34.483 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14535,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6841/1791998028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1791998028},{"type":"v1","addr":"172.21.15.119:6841","nonce":1791998028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2925503246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2925503246},{"type":"v1","addr":"172.21.15.153:6837","nonce":2925503246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/3984962900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3984962900},{"type":"v1","addr":"172.21.15.159:6839","nonce":3984962900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24542,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6839/1920762602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1920762602},{"type":"v1","addr":"172.21.15.119:6839","nonce":1920762602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[]} 2023-12-07T10:17:34.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.483+0000 7f921cdff700 1 -- 172.21.15.119:0/1416747131 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f92000a1b10 msgr2=0x7f92000a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:34.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.483+0000 7f921cdff700 1 --2- 172.21.15.119:0/1416747131 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f92000a1b10 0x7f92000a3fd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9208009c60 tx=0x7f9208009260 comp rx=0 tx=0).stop 2023-12-07T10:17:34.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.483+0000 7f921cdff700 1 -- 172.21.15.119:0/1416747131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9218075650 msgr2=0x7f9218199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:34.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.483+0000 7f921cdff700 1 --2- 172.21.15.119:0/1416747131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9218075650 0x7f9218199cc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f920c000c00 tx=0x7f920c009580 comp rx=0 tx=0).stop 2023-12-07T10:17:34.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.483+0000 7f921cdff700 1 -- 172.21.15.119:0/1416747131 shutdown_connections 2023-12-07T10:17:34.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.483+0000 7f921cdff700 1 --2- 172.21.15.119:0/1416747131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9218109420 0x7f92181a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:34.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.483+0000 7f921cdff700 1 --2- 172.21.15.119:0/1416747131 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f92000a1b10 0x7f92000a3fd0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:34.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.483+0000 7f921cdff700 1 --2- 172.21.15.119:0/1416747131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9218074070 0x7f9218199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:34.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.483+0000 7f921cdff700 1 --2- 172.21.15.119:0/1416747131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9218075650 0x7f9218199cc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:34.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.483+0000 7f921cdff700 1 -- 172.21.15.119:0/1416747131 >> 172.21.15.119:0/1416747131 conn(0x7f92180fcab0 msgr2=0x7f9218105320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:34.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.483+0000 7f921cdff700 1 -- 172.21.15.119:0/1416747131 shutdown_connections 2023-12-07T10:17:34.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:34.483+0000 7f921cdff700 1 -- 172.21.15.119:0/1416747131 wait complete. 2023-12-07T10:17:34.487 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 12 2023-12-07T10:17:34.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:34 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:34.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:34 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:34.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:34 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:34.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:34 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:34.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:34 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:34.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:34 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:17:34.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:34 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1416747131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:34.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:34 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] up:boot 2023-12-07T10:17:34.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:34 smithi119 ceph-mon[97908]: fsmap 12 up:standby 2023-12-07T10:17:34.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:34 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-07T10:17:35.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:34 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:35.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:34 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:35.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:34 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:35.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:34 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:35.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:34 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:35.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:34 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:17:35.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:34 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1416747131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:35.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:34 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] up:boot 2023-12-07T10:17:35.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:34 smithi159 ceph-mon[134726]: fsmap 12 up:standby 2023-12-07T10:17:35.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:34 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-07T10:17:35.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:34 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:35.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:34 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:35.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:34 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:35.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:34 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:35.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:34 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:35.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:34 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:17:35.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:34 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1416747131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:35.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:34 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] up:boot 2023-12-07T10:17:35.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:34 smithi153 ceph-mon[133625]: fsmap 12 up:standby 2023-12-07T10:17:35.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:34 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-07T10:17:35.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:35 smithi119 ceph-mon[97908]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:36.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:35 smithi159 ceph-mon[134726]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:36.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:35 smithi153 ceph-mon[133625]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:36.509 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:17:36.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.719+0000 7f9a835ff700 1 -- 172.21.15.119:0/915594952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a7c074070 msgr2=0x7f9a7c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:36.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.719+0000 7f9a835ff700 1 --2- 172.21.15.119:0/915594952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a7c074070 0x7f9a7c0744f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c009990 tx=0x7f9a6c009ca0 comp rx=0 tx=0).stop 2023-12-07T10:17:36.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.719+0000 7f9a835ff700 1 -- 172.21.15.119:0/915594952 shutdown_connections 2023-12-07T10:17:36.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.719+0000 7f9a835ff700 1 --2- 172.21.15.119:0/915594952 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a7c10db30 0x7f9a7c10ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:36.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.719+0000 7f9a835ff700 1 --2- 172.21.15.119:0/915594952 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a7c075650 0x7f9a7c073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:36.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.719+0000 7f9a835ff700 1 --2- 172.21.15.119:0/915594952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a7c074070 0x7f9a7c0744f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:36.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.719+0000 7f9a835ff700 1 -- 172.21.15.119:0/915594952 >> 172.21.15.119:0/915594952 conn(0x7f9a7c0fcab0 msgr2=0x7f9a7c0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:36.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.719+0000 7f9a835ff700 1 -- 172.21.15.119:0/915594952 shutdown_connections 2023-12-07T10:17:36.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.719+0000 7f9a835ff700 1 -- 172.21.15.119:0/915594952 wait complete. 2023-12-07T10:17:36.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.720+0000 7f9a835ff700 1 Processor -- start 2023-12-07T10:17:36.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.720+0000 7f9a835ff700 1 -- start start 2023-12-07T10:17:36.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.720+0000 7f9a835ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a7c074070 0x7f9a7c19dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:36.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.720+0000 7f9a835ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a7c075650 0x7f9a7c19e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:36.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.720+0000 7f9a835ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a7c10db30 0x7f9a7c1a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:36.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.720+0000 7f9a835ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c1a56e0 con 0x7f9a7c074070 2023-12-07T10:17:36.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.720+0000 7f9a835ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c1a5820 con 0x7f9a7c10db30 2023-12-07T10:17:36.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.720+0000 7f9a835ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c1a59b0 con 0x7f9a7c075650 2023-12-07T10:17:36.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.721+0000 7f9a8139b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a7c074070 0x7f9a7c19dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:36.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.721+0000 7f9a8139b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a7c074070 0x7f9a7c19dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60936/0 (socket says 172.21.15.119:60936) 2023-12-07T10:17:36.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.721+0000 7f9a8139b700 1 -- 172.21.15.119:0/418534560 learned_addr learned my addr 172.21.15.119:0/418534560 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:36.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.721+0000 7f9a80b9a700 1 --2- 172.21.15.119:0/418534560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a7c075650 0x7f9a7c19e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:36.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.721+0000 7f9a81b9c700 1 --2- 172.21.15.119:0/418534560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a7c10db30 0x7f9a7c1a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:36.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.721+0000 7f9a8139b700 1 -- 172.21.15.119:0/418534560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a7c075650 msgr2=0x7f9a7c19e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:36.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.721+0000 7f9a8139b700 1 --2- 172.21.15.119:0/418534560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a7c075650 0x7f9a7c19e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:36.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.721+0000 7f9a8139b700 1 -- 172.21.15.119:0/418534560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a7c10db30 msgr2=0x7f9a7c1a5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:36.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.721+0000 7f9a8139b700 1 --2- 172.21.15.119:0/418534560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a7c10db30 0x7f9a7c1a5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:36.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.721+0000 7f9a8139b700 1 -- 172.21.15.119:0/418534560 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6c0095f0 con 0x7f9a7c074070 2023-12-07T10:17:36.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.721+0000 7f9a81b9c700 1 --2- 172.21.15.119:0/418534560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a7c10db30 0x7f9a7c1a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:17:36.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.721+0000 7f9a8139b700 1 --2- 172.21.15.119:0/418534560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a7c074070 0x7f9a7c19dc00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800e9a0 tx=0x7f9a7800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:36.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.722+0000 7f9a727fc700 1 -- 172.21.15.119:0/418534560 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a78009790 con 0x7f9a7c074070 2023-12-07T10:17:36.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.722+0000 7f9a835ff700 1 -- 172.21.15.119:0/418534560 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a7c1a5c90 con 0x7f9a7c074070 2023-12-07T10:17:36.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.722+0000 7f9a727fc700 1 -- 172.21.15.119:0/418534560 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a7800cbd0 con 0x7f9a7c074070 2023-12-07T10:17:36.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.722+0000 7f9a727fc700 1 -- 172.21.15.119:0/418534560 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a78018680 con 0x7f9a7c074070 2023-12-07T10:17:36.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.722+0000 7f9a835ff700 1 -- 172.21.15.119:0/418534560 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7c1a61a0 con 0x7f9a7c074070 2023-12-07T10:17:36.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.723+0000 7f9a835ff700 1 -- 172.21.15.119:0/418534560 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a7c103d90 con 0x7f9a7c074070 2023-12-07T10:17:36.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.725+0000 7f9a727fc700 1 -- 172.21.15.119:0/418534560 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f9a7800cd40 con 0x7f9a7c074070 2023-12-07T10:17:36.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.726+0000 7f9a727fc700 1 --2- 172.21.15.119:0/418534560 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f9a680a1b10 0x7f9a680a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:36.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.726+0000 7f9a727fc700 1 -- 172.21.15.119:0/418534560 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9a78014070 con 0x7f9a7c074070 2023-12-07T10:17:36.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.727+0000 7f9a80b9a700 1 --2- 172.21.15.119:0/418534560 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f9a680a1b10 0x7f9a680a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:36.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.727+0000 7f9a80b9a700 1 --2- 172.21.15.119:0/418534560 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f9a680a1b10 0x7f9a680a3fd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00b2e0 tx=0x7f9a6c00b750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:36.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.731+0000 7f9a727fc700 1 -- 172.21.15.119:0/418534560 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9a780522f0 con 0x7f9a7c074070 2023-12-07T10:17:36.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:36 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:36.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.992+0000 7f9a835ff700 1 -- 172.21.15.119:0/418534560 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a7c068350 con 0x7f9a7c074070 2023-12-07T10:17:36.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.992+0000 7f9a727fc700 1 -- 172.21.15.119:0/418534560 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10055 (secure 0 0 0) 0x7f9a78096c60 con 0x7f9a7c074070 2023-12-07T10:17:36.994 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:17:36.995 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14535,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6841/1791998028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1791998028},{"type":"v1","addr":"172.21.15.119:6841","nonce":1791998028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2925503246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2925503246},{"type":"v1","addr":"172.21.15.153:6837","nonce":2925503246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/3984962900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3984962900},{"type":"v1","addr":"172.21.15.159:6839","nonce":3984962900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24542,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6839/1920762602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1920762602},{"type":"v1","addr":"172.21.15.119:6839","nonce":1920762602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6839/566825184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":566825184},{"type":"v1","addr":"172.21.15.153:6839","nonce":566825184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-12-07T10:17:36.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.995+0000 7f9a835ff700 1 -- 172.21.15.119:0/418534560 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f9a680a1b10 msgr2=0x7f9a680a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:36.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.995+0000 7f9a835ff700 1 --2- 172.21.15.119:0/418534560 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f9a680a1b10 0x7f9a680a3fd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00b2e0 tx=0x7f9a6c00b750 comp rx=0 tx=0).stop 2023-12-07T10:17:36.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.995+0000 7f9a835ff700 1 -- 172.21.15.119:0/418534560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a7c074070 msgr2=0x7f9a7c19dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:36.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.995+0000 7f9a835ff700 1 --2- 172.21.15.119:0/418534560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a7c074070 0x7f9a7c19dc00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800e9a0 tx=0x7f9a7800ecb0 comp rx=0 tx=0).stop 2023-12-07T10:17:36.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.995+0000 7f9a835ff700 1 -- 172.21.15.119:0/418534560 shutdown_connections 2023-12-07T10:17:36.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.995+0000 7f9a835ff700 1 --2- 172.21.15.119:0/418534560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a7c10db30 0x7f9a7c1a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:36.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.995+0000 7f9a835ff700 1 --2- 172.21.15.119:0/418534560 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f9a680a1b10 0x7f9a680a3fd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:36.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.995+0000 7f9a835ff700 1 --2- 172.21.15.119:0/418534560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a7c074070 0x7f9a7c19dc00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:36.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.995+0000 7f9a835ff700 1 --2- 172.21.15.119:0/418534560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a7c075650 0x7f9a7c19e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:36.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.995+0000 7f9a835ff700 1 -- 172.21.15.119:0/418534560 >> 172.21.15.119:0/418534560 conn(0x7f9a7c0fcab0 msgr2=0x7f9a7c100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:36.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.995+0000 7f9a835ff700 1 -- 172.21.15.119:0/418534560 shutdown_connections 2023-12-07T10:17:36.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:36.995+0000 7f9a835ff700 1 -- 172.21.15.119:0/418534560 wait complete. 2023-12-07T10:17:36.998 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 13 2023-12-07T10:17:37.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:36 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:37.016 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-07T10:17:37.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:36 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:37.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.220+0000 7f70d43ff700 1 -- 172.21.15.119:0/2833922606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70cc104960 msgr2=0x7f70cc104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:37.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.220+0000 7f70d43ff700 1 --2- 172.21.15.119:0/2833922606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70cc104960 0x7f70cc104dc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0099c0 tx=0x7f70bc009cd0 comp rx=0 tx=0).stop 2023-12-07T10:17:37.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.221+0000 7f70d43ff700 1 -- 172.21.15.119:0/2833922606 shutdown_connections 2023-12-07T10:17:37.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.221+0000 7f70d43ff700 1 --2- 172.21.15.119:0/2833922606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70cc105300 0x7f70cc10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:37.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.221+0000 7f70d43ff700 1 --2- 172.21.15.119:0/2833922606 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70cc103770 0x7f70cc103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:37.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.221+0000 7f70d43ff700 1 --2- 172.21.15.119:0/2833922606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70cc104960 0x7f70cc104dc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:37.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.221+0000 7f70d43ff700 1 -- 172.21.15.119:0/2833922606 >> 172.21.15.119:0/2833922606 conn(0x7f70cc0fece0 msgr2=0x7f70cc101120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:37.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.221+0000 7f70d43ff700 1 -- 172.21.15.119:0/2833922606 shutdown_connections 2023-12-07T10:17:37.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.221+0000 7f70d43ff700 1 -- 172.21.15.119:0/2833922606 wait complete. 2023-12-07T10:17:37.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.222+0000 7f70d43ff700 1 Processor -- start 2023-12-07T10:17:37.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.222+0000 7f70d43ff700 1 -- start start 2023-12-07T10:17:37.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.223+0000 7f70d43ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70cc103770 0x7f70cc1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:37.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.223+0000 7f70d43ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70cc104960 0x7f70cc199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:37.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.223+0000 7f70d43ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70cc105300 0x7f70cc1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:37.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.223+0000 7f70d43ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc1a11d0 con 0x7f70cc104960 2023-12-07T10:17:37.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.223+0000 7f70d43ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc1a1310 con 0x7f70cc105300 2023-12-07T10:17:37.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.223+0000 7f70d43ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc1a14a0 con 0x7f70cc103770 2023-12-07T10:17:37.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.223+0000 7f70d199a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70cc104960 0x7f70cc199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:37.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.223+0000 7f70d219b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70cc103770 0x7f70cc1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:37.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.223+0000 7f70d219b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70cc103770 0x7f70cc1996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:46080/0 (socket says 172.21.15.119:46080) 2023-12-07T10:17:37.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.223+0000 7f70d219b700 1 -- 172.21.15.119:0/1659963677 learned_addr learned my addr 172.21.15.119:0/1659963677 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:37.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.223+0000 7f70d299c700 1 --2- 172.21.15.119:0/1659963677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70cc105300 0x7f70cc1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:37.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.223+0000 7f70d219b700 1 -- 172.21.15.119:0/1659963677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70cc105300 msgr2=0x7f70cc1a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:37.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.223+0000 7f70d219b700 1 --2- 172.21.15.119:0/1659963677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70cc105300 0x7f70cc1a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:37.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.223+0000 7f70d219b700 1 -- 172.21.15.119:0/1659963677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70cc104960 msgr2=0x7f70cc199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:37.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.223+0000 7f70d219b700 1 --2- 172.21.15.119:0/1659963677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70cc104960 0x7f70cc199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:37.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.223+0000 7f70d219b700 1 -- 172.21.15.119:0/1659963677 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70bc0095f0 con 0x7f70cc103770 2023-12-07T10:17:37.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.224+0000 7f70d219b700 1 --2- 172.21.15.119:0/1659963677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70cc103770 0x7f70cc1996a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f70c800e970 tx=0x7f70c800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:37.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.224+0000 7f70c37fe700 1 -- 172.21.15.119:0/1659963677 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c8009790 con 0x7f70cc103770 2023-12-07T10:17:37.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.224+0000 7f70d43ff700 1 -- 172.21.15.119:0/1659963677 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70cc1a1780 con 0x7f70cc103770 2023-12-07T10:17:37.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.224+0000 7f70c37fe700 1 -- 172.21.15.119:0/1659963677 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70c800c7e0 con 0x7f70cc103770 2023-12-07T10:17:37.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.224+0000 7f70c37fe700 1 -- 172.21.15.119:0/1659963677 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c8003c70 con 0x7f70cc103770 2023-12-07T10:17:37.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.224+0000 7f70d43ff700 1 -- 172.21.15.119:0/1659963677 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70cc1a1ce0 con 0x7f70cc103770 2023-12-07T10:17:37.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.225+0000 7f70d43ff700 1 -- 172.21.15.119:0/1659963677 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70cc10abf0 con 0x7f70cc103770 2023-12-07T10:17:37.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.227+0000 7f70c37fe700 1 -- 172.21.15.119:0/1659963677 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f70c800cc50 con 0x7f70cc103770 2023-12-07T10:17:37.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.228+0000 7f70c37fe700 1 --2- 172.21.15.119:0/1659963677 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f70b80a1b10 0x7f70b80a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:37.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.228+0000 7f70c37fe700 1 -- 172.21.15.119:0/1659963677 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f70c8014070 con 0x7f70cc103770 2023-12-07T10:17:37.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.228+0000 7f70d199a700 1 --2- 172.21.15.119:0/1659963677 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f70b80a1b10 0x7f70b80a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:37.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.229+0000 7f70d199a700 1 --2- 172.21.15.119:0/1659963677 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f70b80a1b10 0x7f70b80a3fd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f70bc003730 tx=0x7f70bc00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:37.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.232+0000 7f70c37fe700 1 -- 172.21.15.119:0/1659963677 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f70c8096bb0 con 0x7f70cc103770 2023-12-07T10:17:37.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.459+0000 7f70d43ff700 1 -- 172.21.15.119:0/1659963677 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70cc0687a0 con 0x7f70cc103770 2023-12-07T10:17:37.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.461+0000 7f70c37fe700 1 -- 172.21.15.119:0/1659963677 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18492 (secure 0 0 0) 0x7f70c8003dd0 con 0x7f70cc103770 2023-12-07T10:17:37.463 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:17:37.463 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":81,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","created":"2023-12-07T10:09:59.945863+0000","modified":"2023-12-07T10:16:52.466300+0000","last_up_change":"2023-12-07T10:16:50.929427+0000","last_in_change":"2023-12-07T10:16:33.558158+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T10:13:25.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"56b5818e-e625-40f1-8838-3ddcf2d34ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6803","nonce":2141950133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6805","nonce":2141950133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6809","nonce":2141950133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6807","nonce":2141950133}]},"public_addr":"172.21.15.119:6803/2141950133","cluster_addr":"172.21.15.119:6805/2141950133","heartbeat_back_addr":"172.21.15.119:6809/2141950133","heartbeat_front_addr":"172.21.15.119:6807/2141950133","state":["exists","up"]},{"osd":1,"uuid":"36489a6a-5278-4802-9525-9871521722e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6801","nonce":3390875370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6803","nonce":3390875370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6807","nonce":3390875370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6805","nonce":3390875370}]},"public_addr":"172.21.15.153:6801/3390875370","cluster_addr":"172.21.15.153:6803/3390875370","heartbeat_back_addr":"172.21.15.153:6807/3390875370","heartbeat_front_addr":"172.21.15.153:6805/3390875370","state":["exists","up"]},{"osd":2,"uuid":"a937ad26-5e9b-4c8a-b65a-626a8feda6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6801","nonce":708101637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6803","nonce":708101637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6807","nonce":708101637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6805","nonce":708101637}]},"public_addr":"172.21.15.159:6801/708101637","cluster_addr":"172.21.15.159:6803/708101637","heartbeat_back_addr":"172.21.15.159:6807/708101637","heartbeat_front_addr":"172.21.15.159:6805/708101637","state":["exists","up"]},{"osd":3,"uuid":"26ef8300-aa07-48f6-b31d-8b0f2217ffcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6811","nonce":3228047257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6813","nonce":3228047257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6817","nonce":3228047257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6815","nonce":3228047257}]},"public_addr":"172.21.15.119:6811/3228047257","cluster_addr":"172.21.15.119:6813/3228047257","heartbeat_back_addr":"172.21.15.119:6817/3228047257","heartbeat_front_addr":"172.21.15.119:6815/3228047257","state":["exists","up"]},{"osd":4,"uuid":"000ee4b9-9896-4901-8cc3-c85fdfc8cbaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6809","nonce":4021998299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6811","nonce":4021998299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6815","nonce":4021998299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6813","nonce":4021998299}]},"public_addr":"172.21.15.153:6809/4021998299","cluster_addr":"172.21.15.153:6811/4021998299","heartbeat_back_addr":"172.21.15.153:6815/4021998299","heartbeat_front_addr":"172.21.15.153:6813/4021998299","state":["exists","up"]},{"osd":5,"uuid":"8b2fc167-262d-474f-89f3-47bd29424421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6809","nonce":1832540768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6811","nonce":1832540768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6815","nonce":1832540768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6813","nonce":1832540768}]},"public_addr":"172.21.15.159:6809/1832540768","cluster_addr":"172.21.15.159:6811/1832540768","heartbeat_back_addr":"172.21.15.159:6815/1832540768","heartbeat_front_addr":"172.21.15.159:6813/1832540768","state":["exists","up"]},{"osd":6,"uuid":"f9aa6749-46cf-4b2a-8114-cbd034219d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162436840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162436840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6825","nonce":1162436840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162436840}]},"public_addr":"172.21.15.119:6819/1162436840","cluster_addr":"172.21.15.119:6821/1162436840","heartbeat_back_addr":"172.21.15.119:6825/1162436840","heartbeat_front_addr":"172.21.15.119:6823/1162436840","state":["exists","up"]},{"osd":7,"uuid":"39216361-e1c4-4fef-86cb-a4707656b547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6817","nonce":3254653457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6819","nonce":3254653457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6823","nonce":3254653457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6821","nonce":3254653457}]},"public_addr":"172.21.15.153:6817/3254653457","cluster_addr":"172.21.15.153:6819/3254653457","heartbeat_back_addr":"172.21.15.153:6823/3254653457","heartbeat_front_addr":"172.21.15.153:6821/3254653457","state":["exists","up"]},{"osd":8,"uuid":"cceb6ebc-7c02-48ea-9d16-b7a016dd449b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6817","nonce":2600946380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6819","nonce":2600946380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6823","nonce":2600946380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6821","nonce":2600946380}]},"public_addr":"172.21.15.159:6817/2600946380","cluster_addr":"172.21.15.159:6819/2600946380","heartbeat_back_addr":"172.21.15.159:6823/2600946380","heartbeat_front_addr":"172.21.15.159:6821/2600946380","state":["exists","up"]},{"osd":9,"uuid":"f337a2e4-e065-4a9f-91d9-e3615de2d6f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6827","nonce":1940737285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6829","nonce":1940737285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6833","nonce":1940737285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6831","nonce":1940737285}]},"public_addr":"172.21.15.119:6827/1940737285","cluster_addr":"172.21.15.119:6829/1940737285","heartbeat_back_addr":"172.21.15.119:6833/1940737285","heartbeat_front_addr":"172.21.15.119:6831/1940737285","state":["exists","up"]},{"osd":10,"uuid":"3a3bc347-85f4-4878-9797-d9da96137e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6825","nonce":663834864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6827","nonce":663834864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6831","nonce":663834864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6829","nonce":663834864}]},"public_addr":"172.21.15.153:6825/663834864","cluster_addr":"172.21.15.153:6827/663834864","heartbeat_back_addr":"172.21.15.153:6831/663834864","heartbeat_front_addr":"172.21.15.153:6829/663834864","state":["exists","up"]},{"osd":11,"uuid":"a0fc5909-279d-4172-977c-2dc2da0a7e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6825","nonce":1262216348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6827","nonce":1262216348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6831","nonce":1262216348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6829","nonce":1262216348}]},"public_addr":"172.21.15.159:6825/1262216348","cluster_addr":"172.21.15.159:6827/1262216348","heartbeat_back_addr":"172.21.15.159:6831/1262216348","heartbeat_front_addr":"172.21.15.159:6829/1262216348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:37.488109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:59.543060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:21.381585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:47.112729+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:09.732150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:30.893729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:56.678710+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:19.004256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:40.388818+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:06.391734+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:28.394053+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:49.634348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1628639392":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6801/1355617847":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6800/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/362307836":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/128964558":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/2278813554":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/1411933614":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/1685698277":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6800/1355617847":"2023-12-08T10:11:00.585565+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T10:17:37.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.463+0000 7f70d43ff700 1 -- 172.21.15.119:0/1659963677 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f70b80a1b10 msgr2=0x7f70b80a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:37.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.463+0000 7f70d43ff700 1 --2- 172.21.15.119:0/1659963677 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f70b80a1b10 0x7f70b80a3fd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f70bc003730 tx=0x7f70bc00b540 comp rx=0 tx=0).stop 2023-12-07T10:17:37.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.463+0000 7f70d43ff700 1 -- 172.21.15.119:0/1659963677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70cc103770 msgr2=0x7f70cc1996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:37.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.463+0000 7f70d43ff700 1 --2- 172.21.15.119:0/1659963677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70cc103770 0x7f70cc1996a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f70c800e970 tx=0x7f70c800ec80 comp rx=0 tx=0).stop 2023-12-07T10:17:37.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.463+0000 7f70d43ff700 1 -- 172.21.15.119:0/1659963677 shutdown_connections 2023-12-07T10:17:37.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.463+0000 7f70d43ff700 1 --2- 172.21.15.119:0/1659963677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70cc105300 0x7f70cc1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:37.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.463+0000 7f70d43ff700 1 --2- 172.21.15.119:0/1659963677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70cc103770 0x7f70cc1996a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:37.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.463+0000 7f70d43ff700 1 --2- 172.21.15.119:0/1659963677 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f70b80a1b10 0x7f70b80a3fd0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:37.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.463+0000 7f70d43ff700 1 --2- 172.21.15.119:0/1659963677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70cc104960 0x7f70cc199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:37.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.463+0000 7f70d43ff700 1 -- 172.21.15.119:0/1659963677 >> 172.21.15.119:0/1659963677 conn(0x7f70cc0fece0 msgr2=0x7f70cc107d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:37.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.463+0000 7f70d43ff700 1 -- 172.21.15.119:0/1659963677 shutdown_connections 2023-12-07T10:17:37.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.463+0000 7f70d43ff700 1 -- 172.21.15.119:0/1659963677 wait complete. 2023-12-07T10:17:37.485 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-07T10:13:25.194366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-07T10:17:37.485 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-07T10:17:37.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.693+0000 7fa2f99ff700 1 -- 172.21.15.119:0/2790467222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2f41027d0 msgr2=0x7fa2f4102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:37.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.693+0000 7fa2f99ff700 1 --2- 172.21.15.119:0/2790467222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2f41027d0 0x7fa2f4102c30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc0099c0 tx=0x7fa2dc009cd0 comp rx=0 tx=0).stop 2023-12-07T10:17:37.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.694+0000 7fa2f99ff700 1 -- 172.21.15.119:0/2790467222 shutdown_connections 2023-12-07T10:17:37.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.694+0000 7fa2f99ff700 1 --2- 172.21.15.119:0/2790467222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2f4103170 0x7fa2f4109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:37.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.694+0000 7fa2f99ff700 1 --2- 172.21.15.119:0/2790467222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2f41015e0 0x7fa2f4101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:37.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.694+0000 7fa2f99ff700 1 --2- 172.21.15.119:0/2790467222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2f41027d0 0x7fa2f4102c30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:37.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.694+0000 7fa2f99ff700 1 -- 172.21.15.119:0/2790467222 >> 172.21.15.119:0/2790467222 conn(0x7fa2f40fcb50 msgr2=0x7fa2f40fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:37.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.694+0000 7fa2f99ff700 1 -- 172.21.15.119:0/2790467222 shutdown_connections 2023-12-07T10:17:37.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.694+0000 7fa2f99ff700 1 -- 172.21.15.119:0/2790467222 wait complete. 2023-12-07T10:17:37.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.695+0000 7fa2f99ff700 1 Processor -- start 2023-12-07T10:17:37.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.695+0000 7fa2f99ff700 1 -- start start 2023-12-07T10:17:37.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.695+0000 7fa2f99ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2f41015e0 0x7fa2f4075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:37.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.695+0000 7fa2f99ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2f41027d0 0x7fa2f4073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:37.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.695+0000 7fa2f99ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2f4103170 0x7fa2f4073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:37.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.695+0000 7fa2f99ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f40746e0 con 0x7fa2f41027d0 2023-12-07T10:17:37.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.695+0000 7fa2f99ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f4074850 con 0x7fa2f41015e0 2023-12-07T10:17:37.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.695+0000 7fa2f99ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f41a15b0 con 0x7fa2f4103170 2023-12-07T10:17:37.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.695+0000 7fa2f2ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2f41015e0 0x7fa2f4075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:37.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.695+0000 7fa2ebfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2f41027d0 0x7fa2f4073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:37.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.695+0000 7fa2f2ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2f41015e0 0x7fa2f4075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:55202/0 (socket says 172.21.15.119:55202) 2023-12-07T10:17:37.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.695+0000 7fa2f2ffd700 1 -- 172.21.15.119:0/2334686994 learned_addr learned my addr 172.21.15.119:0/2334686994 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:37.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.696+0000 7fa2f37fe700 1 --2- 172.21.15.119:0/2334686994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2f4103170 0x7fa2f4073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:37.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.696+0000 7fa2f2ffd700 1 -- 172.21.15.119:0/2334686994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2f4103170 msgr2=0x7fa2f4073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:37.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.696+0000 7fa2f2ffd700 1 --2- 172.21.15.119:0/2334686994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2f4103170 0x7fa2f4073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:37.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.696+0000 7fa2f2ffd700 1 -- 172.21.15.119:0/2334686994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2f41027d0 msgr2=0x7fa2f4073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:37.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.696+0000 7fa2f2ffd700 1 --2- 172.21.15.119:0/2334686994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2f41027d0 0x7fa2f4073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:37.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.696+0000 7fa2f2ffd700 1 -- 172.21.15.119:0/2334686994 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2dc0095f0 con 0x7fa2f41015e0 2023-12-07T10:17:37.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.696+0000 7fa2f2ffd700 1 --2- 172.21.15.119:0/2334686994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2f41015e0 0x7fa2f4075370 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400e970 tx=0x7fa2e400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:37.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.696+0000 7fa2f89fd700 1 -- 172.21.15.119:0/2334686994 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4009790 con 0x7fa2f41015e0 2023-12-07T10:17:37.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.696+0000 7fa2f89fd700 1 -- 172.21.15.119:0/2334686994 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2e400c780 con 0x7fa2f41015e0 2023-12-07T10:17:37.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.696+0000 7fa2f99ff700 1 -- 172.21.15.119:0/2334686994 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f41a1840 con 0x7fa2f41015e0 2023-12-07T10:17:37.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.696+0000 7fa2f89fd700 1 -- 172.21.15.119:0/2334686994 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4003be0 con 0x7fa2f41015e0 2023-12-07T10:17:37.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.697+0000 7fa2f99ff700 1 -- 172.21.15.119:0/2334686994 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f41a1d50 con 0x7fa2f41015e0 2023-12-07T10:17:37.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.697+0000 7fa2f99ff700 1 -- 172.21.15.119:0/2334686994 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2f4108a80 con 0x7fa2f41015e0 2023-12-07T10:17:37.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.701+0000 7fa2f89fd700 1 -- 172.21.15.119:0/2334686994 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fa2e4003d60 con 0x7fa2f41015e0 2023-12-07T10:17:37.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.702+0000 7fa2f89fd700 1 --2- 172.21.15.119:0/2334686994 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa2e00a1b10 0x7fa2e00a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:37.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.703+0000 7fa2f89fd700 1 -- 172.21.15.119:0/2334686994 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa2e4014070 con 0x7fa2f41015e0 2023-12-07T10:17:37.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.703+0000 7fa2ebfff700 1 --2- 172.21.15.119:0/2334686994 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa2e00a1b10 0x7fa2e00a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:37.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.703+0000 7fa2ebfff700 1 --2- 172.21.15.119:0/2334686994 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa2e00a1b10 0x7fa2e00a3fd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc003730 tx=0x7fa2dc00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:37.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.703+0000 7fa2f89fd700 1 -- 172.21.15.119:0/2334686994 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa2e4096ca0 con 0x7fa2f41015e0 2023-12-07T10:17:37.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.914+0000 7fa2f99ff700 1 -- 172.21.15.119:0/2334686994 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa2f410ce50 con 0x7fa2f41015e0 2023-12-07T10:17:37.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.914+0000 7fa2f89fd700 1 -- 172.21.15.119:0/2334686994 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa2e409a2c0 con 0x7fa2f41015e0 2023-12-07T10:17:37.917 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-12-07T10:17:37.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.917+0000 7fa2f99ff700 1 -- 172.21.15.119:0/2334686994 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa2e00a1b10 msgr2=0x7fa2e00a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:37.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.917+0000 7fa2f99ff700 1 --2- 172.21.15.119:0/2334686994 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa2e00a1b10 0x7fa2e00a3fd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc003730 tx=0x7fa2dc00b540 comp rx=0 tx=0).stop 2023-12-07T10:17:37.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.917+0000 7fa2f99ff700 1 -- 172.21.15.119:0/2334686994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2f41015e0 msgr2=0x7fa2f4075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:37.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.917+0000 7fa2f99ff700 1 --2- 172.21.15.119:0/2334686994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2f41015e0 0x7fa2f4075370 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400e970 tx=0x7fa2e400ec80 comp rx=0 tx=0).stop 2023-12-07T10:17:37.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.917+0000 7fa2f99ff700 1 -- 172.21.15.119:0/2334686994 shutdown_connections 2023-12-07T10:17:37.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.917+0000 7fa2f99ff700 1 --2- 172.21.15.119:0/2334686994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2f4103170 0x7fa2f4073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:37.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.917+0000 7fa2f99ff700 1 --2- 172.21.15.119:0/2334686994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2f41015e0 0x7fa2f4075370 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:37.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.917+0000 7fa2f99ff700 1 --2- 172.21.15.119:0/2334686994 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa2e00a1b10 0x7fa2e00a3fd0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:37.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.917+0000 7fa2f99ff700 1 --2- 172.21.15.119:0/2334686994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2f41027d0 0x7fa2f4073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:37.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.917+0000 7fa2f99ff700 1 -- 172.21.15.119:0/2334686994 >> 172.21.15.119:0/2334686994 conn(0x7fa2f40fcb50 msgr2=0x7fa2f4105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:37.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.917+0000 7fa2f99ff700 1 -- 172.21.15.119:0/2334686994 shutdown_connections 2023-12-07T10:17:37.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:37.918+0000 7fa2f99ff700 1 -- 172.21.15.119:0/2334686994 wait complete. 2023-12-07T10:17:37.937 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:37 smithi119 ceph-mon[97908]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:37.937 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:37 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:37.938 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:37 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:37.938 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:37 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:37.938 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:37 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:37.938 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:37 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/418534560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:37.938 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:37 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1659963677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:17:37.938 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:37 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:37.938 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:37 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:37.938 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:37 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:37.938 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:37 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:17:37.939 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:37 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:37.940 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-07T10:17:38.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:37 smithi159 ceph-mon[134726]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:38.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:37 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:38.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:37 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:38.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:37 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:38.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:37 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:38.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:37 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/418534560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:38.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:37 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1659963677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:17:38.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:37 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:38.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:37 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:38.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:37 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:38.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:37 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:17:38.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:37 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:38.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:37 smithi153 ceph-mon[133625]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:38.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:37 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:38.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:37 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:38.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:37 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:38.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:37 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:38.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:37 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/418534560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:38.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:37 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1659963677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:17:38.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:37 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:38.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:37 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:38.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:37 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:38.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:37 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:17:38.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:37 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:38.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.139+0000 7f142b5ff700 1 -- 172.21.15.119:0/226078965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1424075650 msgr2=0x7f1424073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:38.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.139+0000 7f142b5ff700 1 --2- 172.21.15.119:0/226078965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1424075650 0x7f1424073aa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1420009990 tx=0x7f1420009ca0 comp rx=0 tx=0).stop 2023-12-07T10:17:38.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.139+0000 7f142b5ff700 1 -- 172.21.15.119:0/226078965 shutdown_connections 2023-12-07T10:17:38.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.139+0000 7f142b5ff700 1 --2- 172.21.15.119:0/226078965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f142410db30 0x7f142410ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:38.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.139+0000 7f142b5ff700 1 --2- 172.21.15.119:0/226078965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1424075650 0x7f1424073aa0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:38.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.139+0000 7f142b5ff700 1 --2- 172.21.15.119:0/226078965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1424074070 0x7f14240744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:38.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.139+0000 7f142b5ff700 1 -- 172.21.15.119:0/226078965 >> 172.21.15.119:0/226078965 conn(0x7f14240fcab0 msgr2=0x7f14240fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:38.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.139+0000 7f142b5ff700 1 -- 172.21.15.119:0/226078965 shutdown_connections 2023-12-07T10:17:38.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.139+0000 7f142b5ff700 1 -- 172.21.15.119:0/226078965 wait complete. 2023-12-07T10:17:38.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.140+0000 7f142b5ff700 1 Processor -- start 2023-12-07T10:17:38.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.140+0000 7f142b5ff700 1 -- start start 2023-12-07T10:17:38.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.140+0000 7f142b5ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1424074070 0x7f142419db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:38.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.140+0000 7f142b5ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1424075650 0x7f142419e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:38.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.141+0000 7f142b5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f142410db30 0x7f14241a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:38.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.141+0000 7f142b5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14241a5650 con 0x7f142410db30 2023-12-07T10:17:38.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.141+0000 7f142b5ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14241a5790 con 0x7f1424075650 2023-12-07T10:17:38.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.141+0000 7f142b5ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14241a5920 con 0x7f1424074070 2023-12-07T10:17:38.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.141+0000 7f1429b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f142410db30 0x7f14241a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:38.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.141+0000 7f1429b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f142410db30 0x7f14241a5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32774/0 (socket says 172.21.15.119:32774) 2023-12-07T10:17:38.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.141+0000 7f1429b9c700 1 -- 172.21.15.119:0/1362130449 learned_addr learned my addr 172.21.15.119:0/1362130449 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:38.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.141+0000 7f142939b700 1 --2- 172.21.15.119:0/1362130449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1424074070 0x7f142419db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:38.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.141+0000 7f1428b9a700 1 --2- 172.21.15.119:0/1362130449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1424075650 0x7f142419e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:38.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.141+0000 7f1429b9c700 1 -- 172.21.15.119:0/1362130449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1424074070 msgr2=0x7f142419db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:38.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.141+0000 7f1429b9c700 1 --2- 172.21.15.119:0/1362130449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1424074070 0x7f142419db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:38.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.141+0000 7f1429b9c700 1 -- 172.21.15.119:0/1362130449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1424075650 msgr2=0x7f142419e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:38.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.141+0000 7f1429b9c700 1 --2- 172.21.15.119:0/1362130449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1424075650 0x7f142419e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:38.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.141+0000 7f1429b9c700 1 -- 172.21.15.119:0/1362130449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14200095f0 con 0x7f142410db30 2023-12-07T10:17:38.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.141+0000 7f142939b700 1 --2- 172.21.15.119:0/1362130449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1424074070 0x7f142419db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:17:38.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.141+0000 7f1429b9c700 1 --2- 172.21.15.119:0/1362130449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f142410db30 0x7f14241a5080 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f141c007ee0 tx=0x7f141c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:38.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.142+0000 7f141a7fc700 1 -- 172.21.15.119:0/1362130449 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c017070 con 0x7f142410db30 2023-12-07T10:17:38.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.142+0000 7f141a7fc700 1 -- 172.21.15.119:0/1362130449 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f141c00f040 con 0x7f142410db30 2023-12-07T10:17:38.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.142+0000 7f141a7fc700 1 -- 172.21.15.119:0/1362130449 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c005580 con 0x7f142410db30 2023-12-07T10:17:38.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.142+0000 7f142b5ff700 1 -- 172.21.15.119:0/1362130449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14241a5c00 con 0x7f142410db30 2023-12-07T10:17:38.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.142+0000 7f142b5ff700 1 -- 172.21.15.119:0/1362130449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14241a60c0 con 0x7f142410db30 2023-12-07T10:17:38.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.143+0000 7f142b5ff700 1 -- 172.21.15.119:0/1362130449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1424103d90 con 0x7f142410db30 2023-12-07T10:17:38.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.145+0000 7f141a7fc700 1 -- 172.21.15.119:0/1362130449 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f141c00db20 con 0x7f142410db30 2023-12-07T10:17:38.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.147+0000 7f141a7fc700 1 --2- 172.21.15.119:0/1362130449 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f14100a1b90 0x7f14100a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:38.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.147+0000 7f141a7fc700 1 -- 172.21.15.119:0/1362130449 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f141c0cc860 con 0x7f142410db30 2023-12-07T10:17:38.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.147+0000 7f142939b700 1 --2- 172.21.15.119:0/1362130449 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f14100a1b90 0x7f14100a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:38.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.148+0000 7f142939b700 1 --2- 172.21.15.119:0/1362130449 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f14100a1b90 0x7f14100a4050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f142000b2e0 tx=0x7f1420005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:38.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.151+0000 7f141a7fc700 1 -- 172.21.15.119:0/1362130449 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f141c096080 con 0x7f142410db30 2023-12-07T10:17:38.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.367+0000 7f142b5ff700 1 -- 172.21.15.119:0/1362130449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1424068350 con 0x7f142410db30 2023-12-07T10:17:38.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.367+0000 7f141a7fc700 1 -- 172.21.15.119:0/1362130449 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7f141c0996a0 con 0x7f142410db30 2023-12-07T10:17:38.369 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:17:38.369 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-12-07T10:17:38.370 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:17:38.370 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2023-12-07T10:17:38.370 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2023-12-07T10:17:38.370 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:17:38.370 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-12-07T10:17:38.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.370+0000 7f142b5ff700 1 -- 172.21.15.119:0/1362130449 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f14100a1b90 msgr2=0x7f14100a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:38.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.370+0000 7f142b5ff700 1 --2- 172.21.15.119:0/1362130449 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f14100a1b90 0x7f14100a4050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f142000b2e0 tx=0x7f1420005fd0 comp rx=0 tx=0).stop 2023-12-07T10:17:38.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.370+0000 7f142b5ff700 1 -- 172.21.15.119:0/1362130449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f142410db30 msgr2=0x7f14241a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:38.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.370+0000 7f142b5ff700 1 --2- 172.21.15.119:0/1362130449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f142410db30 0x7f14241a5080 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f141c007ee0 tx=0x7f141c00d3b0 comp rx=0 tx=0).stop 2023-12-07T10:17:38.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.370+0000 7f142b5ff700 1 -- 172.21.15.119:0/1362130449 shutdown_connections 2023-12-07T10:17:38.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.370+0000 7f142b5ff700 1 --2- 172.21.15.119:0/1362130449 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f14100a1b90 0x7f14100a4050 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:38.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.370+0000 7f142b5ff700 1 --2- 172.21.15.119:0/1362130449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f142410db30 0x7f14241a5080 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:38.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.370+0000 7f142b5ff700 1 --2- 172.21.15.119:0/1362130449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1424074070 0x7f142419db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:38.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.370+0000 7f142b5ff700 1 --2- 172.21.15.119:0/1362130449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1424075650 0x7f142419e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:38.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.370+0000 7f142b5ff700 1 -- 172.21.15.119:0/1362130449 >> 172.21.15.119:0/1362130449 conn(0x7f14240fcab0 msgr2=0x7f1424100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:38.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.370+0000 7f142b5ff700 1 -- 172.21.15.119:0/1362130449 shutdown_connections 2023-12-07T10:17:38.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.370+0000 7f142b5ff700 1 -- 172.21.15.119:0/1362130449 wait complete. 2023-12-07T10:17:38.393 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-07T10:17:38.393 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-07T10:17:38.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.601+0000 7f513539d700 1 -- 172.21.15.119:0/3263830231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5130102650 msgr2=0x7f5130102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:38.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.601+0000 7f513539d700 1 --2- 172.21.15.119:0/3263830231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5130102650 0x7f5130102ab0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f51240099c0 tx=0x7f5124009cd0 comp rx=0 tx=0).stop 2023-12-07T10:17:38.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.601+0000 7f513539d700 1 -- 172.21.15.119:0/3263830231 shutdown_connections 2023-12-07T10:17:38.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.601+0000 7f513539d700 1 --2- 172.21.15.119:0/3263830231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5130102ff0 0x7f51301098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:38.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.601+0000 7f513539d700 1 --2- 172.21.15.119:0/3263830231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5130101460 0x7f5130101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:38.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.601+0000 7f513539d700 1 --2- 172.21.15.119:0/3263830231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5130102650 0x7f5130102ab0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:38.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.601+0000 7f513539d700 1 -- 172.21.15.119:0/3263830231 >> 172.21.15.119:0/3263830231 conn(0x7f51300fc9f0 msgr2=0x7f51300fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:38.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.601+0000 7f513539d700 1 -- 172.21.15.119:0/3263830231 shutdown_connections 2023-12-07T10:17:38.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.601+0000 7f513539d700 1 -- 172.21.15.119:0/3263830231 wait complete. 2023-12-07T10:17:38.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.602+0000 7f513539d700 1 Processor -- start 2023-12-07T10:17:38.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.603+0000 7f513539d700 1 -- start start 2023-12-07T10:17:38.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.603+0000 7f513539d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5130101460 0x7f51301973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:38.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.603+0000 7f513539d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5130102650 0x7f5130197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:38.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.603+0000 7f513539d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5130102ff0 0x7f513019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:38.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.603+0000 7f513539d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513019ef00 con 0x7f5130101460 2023-12-07T10:17:38.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.603+0000 7f513539d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513019f040 con 0x7f5130102650 2023-12-07T10:17:38.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.603+0000 7f513539d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513019f1d0 con 0x7f5130102ff0 2023-12-07T10:17:38.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.603+0000 7f512ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5130101460 0x7f51301973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:38.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.603+0000 7f512f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5130102650 0x7f5130197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:38.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.603+0000 7f512ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5130101460 0x7f51301973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32798/0 (socket says 172.21.15.119:32798) 2023-12-07T10:17:38.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.603+0000 7f512f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5130102650 0x7f5130197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:55222/0 (socket says 172.21.15.119:55222) 2023-12-07T10:17:38.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.603+0000 7f512ffff700 1 -- 172.21.15.119:0/4188806220 learned_addr learned my addr 172.21.15.119:0/4188806220 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:38.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.603+0000 7f5134b9c700 1 --2- 172.21.15.119:0/4188806220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5130102ff0 0x7f513019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:38.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.604+0000 7f512ffff700 1 -- 172.21.15.119:0/4188806220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5130102ff0 msgr2=0x7f513019e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:38.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.604+0000 7f512ffff700 1 --2- 172.21.15.119:0/4188806220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5130102ff0 0x7f513019e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:38.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.604+0000 7f512ffff700 1 -- 172.21.15.119:0/4188806220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5130102650 msgr2=0x7f5130197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:38.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.604+0000 7f512ffff700 1 --2- 172.21.15.119:0/4188806220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5130102650 0x7f5130197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:38.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.604+0000 7f512ffff700 1 -- 172.21.15.119:0/4188806220 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5120009520 con 0x7f5130101460 2023-12-07T10:17:38.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.604+0000 7f512f7fe700 1 --2- 172.21.15.119:0/4188806220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5130102650 0x7f5130197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:17:38.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.604+0000 7f512ffff700 1 --2- 172.21.15.119:0/4188806220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5130101460 0x7f51301973d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f512000ea50 tx=0x7f512000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:38.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.604+0000 7f512d7fa700 1 -- 172.21.15.119:0/4188806220 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512000cbd0 con 0x7f5130101460 2023-12-07T10:17:38.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.604+0000 7f513539d700 1 -- 172.21.15.119:0/4188806220 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51240095f0 con 0x7f5130101460 2023-12-07T10:17:38.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.604+0000 7f512d7fa700 1 -- 172.21.15.119:0/4188806220 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f512000cd30 con 0x7f5130101460 2023-12-07T10:17:38.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.604+0000 7f512d7fa700 1 -- 172.21.15.119:0/4188806220 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5120018680 con 0x7f5130101460 2023-12-07T10:17:38.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.604+0000 7f513539d700 1 -- 172.21.15.119:0/4188806220 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5130073c70 con 0x7f5130101460 2023-12-07T10:17:38.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.605+0000 7f513539d700 1 -- 172.21.15.119:0/4188806220 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5130108810 con 0x7f5130101460 2023-12-07T10:17:38.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.609+0000 7f512d7fa700 1 -- 172.21.15.119:0/4188806220 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f5120010430 con 0x7f5130101460 2023-12-07T10:17:38.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.610+0000 7f512d7fa700 1 --2- 172.21.15.119:0/4188806220 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f51180a1b10 0x7f51180a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:38.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.610+0000 7f512d7fa700 1 -- 172.21.15.119:0/4188806220 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5120014070 con 0x7f5130101460 2023-12-07T10:17:38.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.610+0000 7f512f7fe700 1 --2- 172.21.15.119:0/4188806220 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f51180a1b10 0x7f51180a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:38.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.611+0000 7f512f7fe700 1 --2- 172.21.15.119:0/4188806220 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f51180a1b10 0x7f51180a3fd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5124003730 tx=0x7f512401e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:38.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.612+0000 7f512d7fa700 1 -- 172.21.15.119:0/4188806220 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f51200974d0 con 0x7f5130101460 2023-12-07T10:17:38.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:38.830+0000 7f513539d700 1 -- 172.21.15.119:0/4188806220 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5130068650 con 0x7f5130101460 2023-12-07T10:17:38.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:38 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:17:38.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:38 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2334686994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:17:38.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:38 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1362130449' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-12-07T10:17:39.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:38 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:17:39.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:38 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2334686994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:17:39.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:38 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1362130449' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-12-07T10:17:39.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:38 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:17:39.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:38 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2334686994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:17:39.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:38 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1362130449' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-12-07T10:17:39.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.725+0000 7f512d7fa700 1 -- 172.21.15.119:0/4188806220 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7f512009aaf0 con 0x7f5130101460 2023-12-07T10:17:39.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.730+0000 7f513539d700 1 -- 172.21.15.119:0/4188806220 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f51180a1b10 msgr2=0x7f51180a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:39.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.731+0000 7f513539d700 1 --2- 172.21.15.119:0/4188806220 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f51180a1b10 0x7f51180a3fd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5124003730 tx=0x7f512401e040 comp rx=0 tx=0).stop 2023-12-07T10:17:39.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.731+0000 7f513539d700 1 -- 172.21.15.119:0/4188806220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5130101460 msgr2=0x7f51301973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:39.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.731+0000 7f513539d700 1 --2- 172.21.15.119:0/4188806220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5130101460 0x7f51301973d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f512000ea50 tx=0x7f512000ed60 comp rx=0 tx=0).stop 2023-12-07T10:17:39.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.732+0000 7f513539d700 1 -- 172.21.15.119:0/4188806220 shutdown_connections 2023-12-07T10:17:39.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.732+0000 7f513539d700 1 --2- 172.21.15.119:0/4188806220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5130102ff0 0x7f513019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:39.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.732+0000 7f513539d700 1 --2- 172.21.15.119:0/4188806220 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f51180a1b10 0x7f51180a3fd0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:39.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.732+0000 7f513539d700 1 --2- 172.21.15.119:0/4188806220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5130101460 0x7f51301973d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:39.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.732+0000 7f513539d700 1 --2- 172.21.15.119:0/4188806220 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5130102650 0x7f5130197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:39.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.732+0000 7f513539d700 1 -- 172.21.15.119:0/4188806220 >> 172.21.15.119:0/4188806220 conn(0x7f51300fc9f0 msgr2=0x7f5130105950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:39.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.732+0000 7f513539d700 1 -- 172.21.15.119:0/4188806220 shutdown_connections 2023-12-07T10:17:39.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.733+0000 7f513539d700 1 -- 172.21.15.119:0/4188806220 wait complete. 2023-12-07T10:17:39.737 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2023-12-07T10:17:39.760 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-12-07T10:17:39.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:39 smithi119 ceph-mon[97908]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:39.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:39 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/4188806220' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-07T10:17:39.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.984+0000 7f773b5ff700 1 -- 172.21.15.119:0/2627394315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7734103150 msgr2=0x7f7734109a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:39.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.984+0000 7f773b5ff700 1 --2- 172.21.15.119:0/2627394315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7734103150 0x7f7734109a00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f772c009920 tx=0x7f772c009c30 comp rx=0 tx=0).stop 2023-12-07T10:17:39.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.984+0000 7f773b5ff700 1 -- 172.21.15.119:0/2627394315 shutdown_connections 2023-12-07T10:17:39.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.984+0000 7f773b5ff700 1 --2- 172.21.15.119:0/2627394315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7734103150 0x7f7734109a00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:39.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.984+0000 7f773b5ff700 1 --2- 172.21.15.119:0/2627394315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77341015c0 0x7f77341019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:39.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.984+0000 7f773b5ff700 1 --2- 172.21.15.119:0/2627394315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77341027b0 0x7f7734102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:39.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.984+0000 7f773b5ff700 1 -- 172.21.15.119:0/2627394315 >> 172.21.15.119:0/2627394315 conn(0x7f77340fcb10 msgr2=0x7f77340fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:39.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.984+0000 7f773b5ff700 1 -- 172.21.15.119:0/2627394315 shutdown_connections 2023-12-07T10:17:39.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.984+0000 7f773b5ff700 1 -- 172.21.15.119:0/2627394315 wait complete. 2023-12-07T10:17:39.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.985+0000 7f773b5ff700 1 Processor -- start 2023-12-07T10:17:39.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.985+0000 7f773b5ff700 1 -- start start 2023-12-07T10:17:39.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.985+0000 7f773b5ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77341015c0 0x7f7734199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:39.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.985+0000 7f773b5ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77341027b0 0x7f7734199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:39.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.985+0000 7f773b5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7734103150 0x7f77341a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:39.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.985+0000 7f773b5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77341a1280 con 0x7f7734103150 2023-12-07T10:17:39.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.985+0000 7f773b5ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77341a13c0 con 0x7f77341027b0 2023-12-07T10:17:39.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.985+0000 7f773b5ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77341a1550 con 0x7f77341015c0 2023-12-07T10:17:39.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.985+0000 7f7738b9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77341027b0 0x7f7734199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:39.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.985+0000 7f7738b9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77341027b0 0x7f7734199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:55228/0 (socket says 172.21.15.119:55228) 2023-12-07T10:17:39.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.985+0000 7f7738b9a700 1 -- 172.21.15.119:0/875664337 learned_addr learned my addr 172.21.15.119:0/875664337 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:39.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.985+0000 7f773939b700 1 --2- 172.21.15.119:0/875664337 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77341015c0 0x7f7734199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:39.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.985+0000 7f7739b9c700 1 --2- 172.21.15.119:0/875664337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7734103150 0x7f77341a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:39.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.986+0000 7f7738b9a700 1 -- 172.21.15.119:0/875664337 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77341015c0 msgr2=0x7f7734199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:39.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.986+0000 7f7738b9a700 1 --2- 172.21.15.119:0/875664337 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77341015c0 0x7f7734199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:39.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.986+0000 7f7738b9a700 1 -- 172.21.15.119:0/875664337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7734103150 msgr2=0x7f77341a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:39.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.986+0000 7f7738b9a700 1 --2- 172.21.15.119:0/875664337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7734103150 0x7f77341a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:39.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.986+0000 7f7738b9a700 1 -- 172.21.15.119:0/875664337 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77240095f0 con 0x7f77341027b0 2023-12-07T10:17:39.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.986+0000 7f773939b700 1 --2- 172.21.15.119:0/875664337 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77341015c0 0x7f7734199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:17:39.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.986+0000 7f7739b9c700 1 --2- 172.21.15.119:0/875664337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7734103150 0x7f77341a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:17:39.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.986+0000 7f7738b9a700 1 --2- 172.21.15.119:0/875664337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77341027b0 0x7f7734199c90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f772400ec10 tx=0x7f772400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:39.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.986+0000 7f772a7fc700 1 -- 172.21.15.119:0/875664337 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772400cc90 con 0x7f77341027b0 2023-12-07T10:17:39.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.986+0000 7f773b5ff700 1 -- 172.21.15.119:0/875664337 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f772c009520 con 0x7f77341027b0 2023-12-07T10:17:39.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.986+0000 7f772a7fc700 1 -- 172.21.15.119:0/875664337 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77240044e0 con 0x7f77341027b0 2023-12-07T10:17:39.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.986+0000 7f772a7fc700 1 -- 172.21.15.119:0/875664337 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7724010430 con 0x7f77341027b0 2023-12-07T10:17:39.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.986+0000 7f773b5ff700 1 -- 172.21.15.119:0/875664337 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77341a1bb0 con 0x7f77341027b0 2023-12-07T10:17:39.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.987+0000 7f773b5ff700 1 -- 172.21.15.119:0/875664337 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77340683c0 con 0x7f77341027b0 2023-12-07T10:17:39.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.989+0000 7f772a7fc700 1 -- 172.21.15.119:0/875664337 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f7724003fb0 con 0x7f77341027b0 2023-12-07T10:17:39.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.990+0000 7f772a7fc700 1 --2- 172.21.15.119:0/875664337 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f77200a1b10 0x7f77200a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:39.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.991+0000 7f772a7fc700 1 -- 172.21.15.119:0/875664337 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f7724014070 con 0x7f77341027b0 2023-12-07T10:17:39.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.991+0000 7f773939b700 1 --2- 172.21.15.119:0/875664337 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f77200a1b10 0x7f77200a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:39.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.991+0000 7f773939b700 1 --2- 172.21.15.119:0/875664337 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f77200a1b10 0x7f77200a3fd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7734102610 tx=0x7f773000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:39.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:39.993+0000 7f772a7fc700 1 -- 172.21.15.119:0/875664337 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7724096f60 con 0x7f77341027b0 2023-12-07T10:17:40.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:39 smithi159 ceph-mon[134726]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:40.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:39 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/4188806220' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-07T10:17:40.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:39 smithi153 ceph-mon[133625]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:40.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:39 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/4188806220' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-07T10:17:40.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:40.207+0000 7f773b5ff700 1 -- 172.21.15.119:0/875664337 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f773410ce30 con 0x7f77341027b0 2023-12-07T10:17:40.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:40.738+0000 7f772a7fc700 1 -- 172.21.15.119:0/875664337 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) v1 ==== 171+0+0 (secure 0 0 0) 0x7f772409a580 con 0x7f77341027b0 2023-12-07T10:17:40.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:40.749+0000 7f773b5ff700 1 -- 172.21.15.119:0/875664337 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f77200a1b10 msgr2=0x7f77200a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:40.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:40.749+0000 7f773b5ff700 1 --2- 172.21.15.119:0/875664337 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f77200a1b10 0x7f77200a3fd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7734102610 tx=0x7f773000b040 comp rx=0 tx=0).stop 2023-12-07T10:17:40.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:40.749+0000 7f773b5ff700 1 -- 172.21.15.119:0/875664337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77341027b0 msgr2=0x7f7734199c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:40.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:40.749+0000 7f773b5ff700 1 --2- 172.21.15.119:0/875664337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77341027b0 0x7f7734199c90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f772400ec10 tx=0x7f772400ef20 comp rx=0 tx=0).stop 2023-12-07T10:17:40.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:40.750+0000 7f773b5ff700 1 -- 172.21.15.119:0/875664337 shutdown_connections 2023-12-07T10:17:40.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:40.750+0000 7f773b5ff700 1 --2- 172.21.15.119:0/875664337 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f77200a1b10 0x7f77200a3fd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:40.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:40.750+0000 7f773b5ff700 1 --2- 172.21.15.119:0/875664337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7734103150 0x7f77341a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:40.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:40.750+0000 7f773b5ff700 1 --2- 172.21.15.119:0/875664337 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77341015c0 0x7f7734199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:40.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:40.750+0000 7f773b5ff700 1 --2- 172.21.15.119:0/875664337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77341027b0 0x7f7734199c90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:40.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:40.750+0000 7f773b5ff700 1 -- 172.21.15.119:0/875664337 >> 172.21.15.119:0/875664337 conn(0x7f77340fcb10 msgr2=0x7f7734073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:40.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:40.751+0000 7f773b5ff700 1 -- 172.21.15.119:0/875664337 shutdown_connections 2023-12-07T10:17:40.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:40.751+0000 7f773b5ff700 1 -- 172.21.15.119:0/875664337 wait complete. 2023-12-07T10:17:40.759 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2023-12-07T10:17:40.785 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-07T10:17:41.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.001+0000 7f4fe5fff700 1 -- 172.21.15.119:0/318451973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fe0074070 msgr2=0x7f4fe00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:41.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.001+0000 7f4fe5fff700 1 --2- 172.21.15.119:0/318451973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fe0074070 0x7f4fe00744f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009990 tx=0x7f4fd4009ca0 comp rx=0 tx=0).stop 2023-12-07T10:17:41.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.001+0000 7f4fe5fff700 1 -- 172.21.15.119:0/318451973 shutdown_connections 2023-12-07T10:17:41.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.001+0000 7f4fe5fff700 1 --2- 172.21.15.119:0/318451973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fe010db90 0x7f4fe010ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:41.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.001+0000 7f4fe5fff700 1 --2- 172.21.15.119:0/318451973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe0075650 0x7f4fe0073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:41.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.001+0000 7f4fe5fff700 1 --2- 172.21.15.119:0/318451973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fe0074070 0x7f4fe00744f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:41.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.001+0000 7f4fe5fff700 1 -- 172.21.15.119:0/318451973 >> 172.21.15.119:0/318451973 conn(0x7f4fe00fcb10 msgr2=0x7f4fe00fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:41.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.002+0000 7f4fe5fff700 1 -- 172.21.15.119:0/318451973 shutdown_connections 2023-12-07T10:17:41.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.002+0000 7f4fe5fff700 1 -- 172.21.15.119:0/318451973 wait complete. 2023-12-07T10:17:41.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.002+0000 7f4fe5fff700 1 Processor -- start 2023-12-07T10:17:41.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.003+0000 7f4fe5fff700 1 -- start start 2023-12-07T10:17:41.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.003+0000 7f4fe5fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fe0074070 0x7f4fe019dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:41.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.003+0000 7f4fe5fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fe0075650 0x7f4fe019e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:41.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.003+0000 7f4fe5fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe010db90 0x7f4fe01a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:41.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.003+0000 7f4fe5fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe01a56a0 con 0x7f4fe010db90 2023-12-07T10:17:41.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.003+0000 7f4fe5fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe01a57e0 con 0x7f4fe0075650 2023-12-07T10:17:41.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.003+0000 7f4fe5fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe01a5970 con 0x7f4fe0074070 2023-12-07T10:17:41.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.003+0000 7f4fdffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe010db90 0x7f4fe01a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:41.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.003+0000 7f4fdf7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fe0074070 0x7f4fe019dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:41.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.003+0000 7f4fdeffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fe0075650 0x7f4fe019e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:41.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.003+0000 7f4fdffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe010db90 0x7f4fe01a50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32828/0 (socket says 172.21.15.119:32828) 2023-12-07T10:17:41.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.003+0000 7f4fdffff700 1 -- 172.21.15.119:0/1236934120 learned_addr learned my addr 172.21.15.119:0/1236934120 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:41.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.004+0000 7f4fdffff700 1 -- 172.21.15.119:0/1236934120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fe0074070 msgr2=0x7f4fe019dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:41.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.004+0000 7f4fdffff700 1 --2- 172.21.15.119:0/1236934120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fe0074070 0x7f4fe019dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:41.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.004+0000 7f4fdffff700 1 -- 172.21.15.119:0/1236934120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fe0075650 msgr2=0x7f4fe019e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:41.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.004+0000 7f4fdffff700 1 --2- 172.21.15.119:0/1236934120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fe0075650 0x7f4fe019e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:41.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.004+0000 7f4fdffff700 1 -- 172.21.15.119:0/1236934120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd40095f0 con 0x7f4fe010db90 2023-12-07T10:17:41.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.004+0000 7f4fdffff700 1 --2- 172.21.15.119:0/1236934120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe010db90 0x7f4fe01a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8007ee0 tx=0x7f4fd800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:41.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.004+0000 7f4fdcff9700 1 -- 172.21.15.119:0/1236934120 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8017070 con 0x7f4fe010db90 2023-12-07T10:17:41.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.004+0000 7f4fe5fff700 1 -- 172.21.15.119:0/1236934120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe01a5c50 con 0x7f4fe010db90 2023-12-07T10:17:41.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.004+0000 7f4fdcff9700 1 -- 172.21.15.119:0/1236934120 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fd800f040 con 0x7f4fe010db90 2023-12-07T10:17:41.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.004+0000 7f4fe5fff700 1 -- 172.21.15.119:0/1236934120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe01a6160 con 0x7f4fe010db90 2023-12-07T10:17:41.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.004+0000 7f4fdcff9700 1 -- 172.21.15.119:0/1236934120 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8005530 con 0x7f4fe010db90 2023-12-07T10:17:41.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.005+0000 7f4fe5fff700 1 -- 172.21.15.119:0/1236934120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fcc005310 con 0x7f4fe010db90 2023-12-07T10:17:41.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.008+0000 7f4fdcff9700 1 -- 172.21.15.119:0/1236934120 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f4fd800db20 con 0x7f4fe010db90 2023-12-07T10:17:41.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.009+0000 7f4fdcff9700 1 --2- 172.21.15.119:0/1236934120 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4fc80a1b10 0x7f4fc80a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:41.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.009+0000 7f4fdcff9700 1 -- 172.21.15.119:0/1236934120 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f4fd80ccab0 con 0x7f4fe010db90 2023-12-07T10:17:41.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.009+0000 7f4fdf7fe700 1 --2- 172.21.15.119:0/1236934120 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4fc80a1b10 0x7f4fc80a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:41.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.010+0000 7f4fdf7fe700 1 --2- 172.21.15.119:0/1236934120 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4fc80a1b10 0x7f4fc80a3fd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009c60 tx=0x7f4fd0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:41.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:40 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/4188806220' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-12-07T10:17:41.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:40 smithi159 ceph-mon[134726]: osdmap e82: 12 total, 12 up, 12 in 2023-12-07T10:17:41.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:40 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/875664337' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-12-07T10:17:41.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:40 smithi159 ceph-mon[134726]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-12-07T10:17:41.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:40 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:41.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.016+0000 7f4fdcff9700 1 -- 172.21.15.119:0/1236934120 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4fd8096070 con 0x7f4fe010db90 2023-12-07T10:17:41.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:40 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/4188806220' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-12-07T10:17:41.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:40 smithi153 ceph-mon[133625]: osdmap e82: 12 total, 12 up, 12 in 2023-12-07T10:17:41.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:40 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/875664337' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-12-07T10:17:41.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:40 smithi153 ceph-mon[133625]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-12-07T10:17:41.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:40 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:41.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:40 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/4188806220' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-12-07T10:17:41.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:40 smithi119 ceph-mon[97908]: osdmap e82: 12 total, 12 up, 12 in 2023-12-07T10:17:41.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:40 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/875664337' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-12-07T10:17:41.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:40 smithi119 ceph-mon[97908]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-12-07T10:17:41.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:40 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:41.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.276+0000 7f4fe5fff700 1 -- 172.21.15.119:0/1236934120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4fcc005890 con 0x7f4fe010db90 2023-12-07T10:17:41.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.749+0000 7f4fdcff9700 1 -- 172.21.15.119:0/1236934120 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f4fd8009390 con 0x7f4fe010db90 2023-12-07T10:17:41.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.752+0000 7f4fe5fff700 1 -- 172.21.15.119:0/1236934120 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4fc80a1b10 msgr2=0x7f4fc80a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:41.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.752+0000 7f4fe5fff700 1 --2- 172.21.15.119:0/1236934120 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4fc80a1b10 0x7f4fc80a3fd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009c60 tx=0x7f4fd0009260 comp rx=0 tx=0).stop 2023-12-07T10:17:41.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.752+0000 7f4fe5fff700 1 -- 172.21.15.119:0/1236934120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe010db90 msgr2=0x7f4fe01a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:41.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.752+0000 7f4fe5fff700 1 --2- 172.21.15.119:0/1236934120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe010db90 0x7f4fe01a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8007ee0 tx=0x7f4fd800d3b0 comp rx=0 tx=0).stop 2023-12-07T10:17:41.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.752+0000 7f4fe5fff700 1 -- 172.21.15.119:0/1236934120 shutdown_connections 2023-12-07T10:17:41.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.752+0000 7f4fe5fff700 1 --2- 172.21.15.119:0/1236934120 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4fc80a1b10 0x7f4fc80a3fd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:41.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.752+0000 7f4fe5fff700 1 --2- 172.21.15.119:0/1236934120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe010db90 0x7f4fe01a50d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:41.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.752+0000 7f4fe5fff700 1 --2- 172.21.15.119:0/1236934120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fe0074070 0x7f4fe019dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:41.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.752+0000 7f4fe5fff700 1 --2- 172.21.15.119:0/1236934120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fe0075650 0x7f4fe019e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:41.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.752+0000 7f4fe5fff700 1 -- 172.21.15.119:0/1236934120 >> 172.21.15.119:0/1236934120 conn(0x7f4fe00fcb10 msgr2=0x7f4fe0100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:41.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.752+0000 7f4fe5fff700 1 -- 172.21.15.119:0/1236934120 shutdown_connections 2023-12-07T10:17:41.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.752+0000 7f4fe5fff700 1 -- 172.21.15.119:0/1236934120 wait complete. 2023-12-07T10:17:41.755 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 2 and data pool 3 2023-12-07T10:17:41.776 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-12-07T10:17:41.776 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-12-07T10:17:41.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.977+0000 7f7214dff700 1 -- 172.21.15.119:0/2704739593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72101015c0 msgr2=0x7f72101019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:41.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.977+0000 7f7214dff700 1 --2- 172.21.15.119:0/2704739593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72101015c0 0x7f72101019e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f72000099c0 tx=0x7f7200009cd0 comp rx=0 tx=0).stop 2023-12-07T10:17:41.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.978+0000 7f7214dff700 1 -- 172.21.15.119:0/2704739593 shutdown_connections 2023-12-07T10:17:41.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.978+0000 7f7214dff700 1 --2- 172.21.15.119:0/2704739593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7210103150 0x7f7210109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:41.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.978+0000 7f7214dff700 1 --2- 172.21.15.119:0/2704739593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72101015c0 0x7f72101019e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:41.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.978+0000 7f7214dff700 1 --2- 172.21.15.119:0/2704739593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72101027b0 0x7f7210102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:41.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.978+0000 7f7214dff700 1 -- 172.21.15.119:0/2704739593 >> 172.21.15.119:0/2704739593 conn(0x7f72100fcb10 msgr2=0x7f72100fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:41.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.978+0000 7f7214dff700 1 -- 172.21.15.119:0/2704739593 shutdown_connections 2023-12-07T10:17:41.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.978+0000 7f7214dff700 1 -- 172.21.15.119:0/2704739593 wait complete. 2023-12-07T10:17:41.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.979+0000 7f7214dff700 1 Processor -- start 2023-12-07T10:17:41.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.979+0000 7f7214dff700 1 -- start start 2023-12-07T10:17:41.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.980+0000 7f7214dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72101015c0 0x7f7210075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:41.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.980+0000 7f7214dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72101027b0 0x7f7210073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:41.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.980+0000 7f7214dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7210103150 0x7f7210073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:41.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.980+0000 7f7214dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72100746e0 con 0x7f7210103150 2023-12-07T10:17:41.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.980+0000 7f7214dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7210074850 con 0x7f72101027b0 2023-12-07T10:17:41.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.980+0000 7f7214dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72101a15e0 con 0x7f72101015c0 2023-12-07T10:17:41.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.980+0000 7f7207fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72101027b0 0x7f7210073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:41.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.980+0000 7f720e59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72101015c0 0x7f7210075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:41.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.980+0000 7f7207fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72101027b0 0x7f7210073a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:55272/0 (socket says 172.21.15.119:55272) 2023-12-07T10:17:41.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.980+0000 7f7207fff700 1 -- 172.21.15.119:0/142538845 learned_addr learned my addr 172.21.15.119:0/142538845 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:41.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.980+0000 7f720ed9d700 1 --2- 172.21.15.119:0/142538845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7210103150 0x7f7210073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:41.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.980+0000 7f7207fff700 1 -- 172.21.15.119:0/142538845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72101015c0 msgr2=0x7f7210075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:41.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.980+0000 7f7207fff700 1 --2- 172.21.15.119:0/142538845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72101015c0 0x7f7210075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:41.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.980+0000 7f7207fff700 1 -- 172.21.15.119:0/142538845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7210103150 msgr2=0x7f7210073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:41.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.980+0000 7f7207fff700 1 --2- 172.21.15.119:0/142538845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7210103150 0x7f7210073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:41.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.980+0000 7f7207fff700 1 -- 172.21.15.119:0/142538845 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72000095f0 con 0x7f72101027b0 2023-12-07T10:17:41.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.981+0000 7f7207fff700 1 --2- 172.21.15.119:0/142538845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72101027b0 0x7f7210073a00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f71f800e9a0 tx=0x7f71f800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:41.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.981+0000 7f72077fe700 1 -- 172.21.15.119:0/142538845 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f8009790 con 0x7f72101027b0 2023-12-07T10:17:41.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.981+0000 7f7214dff700 1 -- 172.21.15.119:0/142538845 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72101a1870 con 0x7f72101027b0 2023-12-07T10:17:41.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.981+0000 7f72077fe700 1 -- 172.21.15.119:0/142538845 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71f800ce70 con 0x7f72101027b0 2023-12-07T10:17:41.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.981+0000 7f72077fe700 1 -- 172.21.15.119:0/142538845 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f8018420 con 0x7f72101027b0 2023-12-07T10:17:41.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.981+0000 7f7214dff700 1 -- 172.21.15.119:0/142538845 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72101a1d80 con 0x7f72101027b0 2023-12-07T10:17:41.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.982+0000 7f7214dff700 1 -- 172.21.15.119:0/142538845 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7210108a10 con 0x7f72101027b0 2023-12-07T10:17:41.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.984+0000 7f72077fe700 1 -- 172.21.15.119:0/142538845 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f71f80106f0 con 0x7f72101027b0 2023-12-07T10:17:41.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.985+0000 7f72077fe700 1 --2- 172.21.15.119:0/142538845 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f71fc0a1b10 0x7f71fc0a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:41.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.985+0000 7f72077fe700 1 -- 172.21.15.119:0/142538845 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f71f8014070 con 0x7f72101027b0 2023-12-07T10:17:41.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.985+0000 7f720e59c700 1 --2- 172.21.15.119:0/142538845 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f71fc0a1b10 0x7f71fc0a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:41.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.986+0000 7f720e59c700 1 --2- 172.21.15.119:0/142538845 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f71fc0a1b10 0x7f71fc0a3fd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7200009990 tx=0x7f7200000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:41.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:41.987+0000 7f72077fe700 1 -- 172.21.15.119:0/142538845 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f71f8096b60 con 0x7f72101027b0 2023-12-07T10:17:42.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:41 smithi159 ceph-mon[134726]: pgmap v272: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:42.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:41 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:42.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:41 smithi159 ceph-mon[134726]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-12-07T10:17:42.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:41 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:42.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:41 smithi159 ceph-mon[134726]: osdmap e83: 12 total, 12 up, 12 in 2023-12-07T10:17:42.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:41 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:42.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:41 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1236934120' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-07T10:17:42.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:41 smithi159 ceph-mon[134726]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-12-07T10:17:42.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:41 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:42.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:41 smithi159 ceph-mon[134726]: osdmap e84: 12 total, 12 up, 12 in 2023-12-07T10:17:42.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:41 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:42.015 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:41 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:42.016 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:41 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:17:42.016 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:41 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:42.016 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:41 smithi159 ceph-mon[134726]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-07T10:17:42.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:41 smithi153 ceph-mon[133625]: pgmap v272: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:42.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:41 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:42.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:41 smithi153 ceph-mon[133625]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-12-07T10:17:42.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:41 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:42.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:41 smithi153 ceph-mon[133625]: osdmap e83: 12 total, 12 up, 12 in 2023-12-07T10:17:42.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:41 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:42.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:41 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1236934120' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-07T10:17:42.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:41 smithi153 ceph-mon[133625]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-12-07T10:17:42.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:41 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:42.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:41 smithi153 ceph-mon[133625]: osdmap e84: 12 total, 12 up, 12 in 2023-12-07T10:17:42.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:41 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:42.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:41 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:42.032 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:41 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:17:42.033 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:41 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:42.033 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:41 smithi153 ceph-mon[133625]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-07T10:17:42.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.199+0000 7f7214dff700 1 -- 172.21.15.119:0/142538845 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f72100683c0 con 0x7f72101027b0 2023-12-07T10:17:42.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:41 smithi119 ceph-mon[97908]: pgmap v272: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:42.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:41 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:42.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:41 smithi119 ceph-mon[97908]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-12-07T10:17:42.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:41 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:42.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:41 smithi119 ceph-mon[97908]: osdmap e83: 12 total, 12 up, 12 in 2023-12-07T10:17:42.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:41 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:42.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:41 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1236934120' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-07T10:17:42.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:41 smithi119 ceph-mon[97908]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-12-07T10:17:42.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:41 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:42.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:41 smithi119 ceph-mon[97908]: osdmap e84: 12 total, 12 up, 12 in 2023-12-07T10:17:42.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:41 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:42.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:41 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:17:42.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:41 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:17:42.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:41 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:17:42.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:41 smithi119 ceph-mon[97908]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-07T10:17:42.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.594+0000 7f72077fe700 1 -- 172.21.15.119:0/142538845 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) v1 ==== 157+0+0 (secure 0 0 0) 0x7f71f809a180 con 0x7f72101027b0 2023-12-07T10:17:42.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.597+0000 7f7214dff700 1 -- 172.21.15.119:0/142538845 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f71fc0a1b10 msgr2=0x7f71fc0a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:42.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.597+0000 7f7214dff700 1 --2- 172.21.15.119:0/142538845 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f71fc0a1b10 0x7f71fc0a3fd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7200009990 tx=0x7f7200000bc0 comp rx=0 tx=0).stop 2023-12-07T10:17:42.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.597+0000 7f7214dff700 1 -- 172.21.15.119:0/142538845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72101027b0 msgr2=0x7f7210073a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:42.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.597+0000 7f7214dff700 1 --2- 172.21.15.119:0/142538845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72101027b0 0x7f7210073a00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f71f800e9a0 tx=0x7f71f800ecb0 comp rx=0 tx=0).stop 2023-12-07T10:17:42.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.597+0000 7f7214dff700 1 -- 172.21.15.119:0/142538845 shutdown_connections 2023-12-07T10:17:42.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.597+0000 7f7214dff700 1 --2- 172.21.15.119:0/142538845 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f71fc0a1b10 0x7f71fc0a3fd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:42.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.597+0000 7f7214dff700 1 --2- 172.21.15.119:0/142538845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7210103150 0x7f7210073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:42.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.597+0000 7f7214dff700 1 --2- 172.21.15.119:0/142538845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72101015c0 0x7f7210075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:42.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.597+0000 7f7214dff700 1 --2- 172.21.15.119:0/142538845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72101027b0 0x7f7210073a00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:42.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.597+0000 7f7214dff700 1 -- 172.21.15.119:0/142538845 >> 172.21.15.119:0/142538845 conn(0x7f72100fcb10 msgr2=0x7f7210105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:42.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.597+0000 7f7214dff700 1 -- 172.21.15.119:0/142538845 shutdown_connections 2023-12-07T10:17:42.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.597+0000 7f7214dff700 1 -- 172.21.15.119:0/142538845 wait complete. 2023-12-07T10:17:42.619 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-12-07T10:17:42.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.827+0000 7f90b7fff700 1 -- 172.21.15.119:0/3898414023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b01014f0 msgr2=0x7f90b0101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:42.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.827+0000 7f90b7fff700 1 --2- 172.21.15.119:0/3898414023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b01014f0 0x7f90b0101910 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0099c0 tx=0x7f90ac009cd0 comp rx=0 tx=0).stop 2023-12-07T10:17:42.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.827+0000 7f90b7fff700 1 -- 172.21.15.119:0/3898414023 shutdown_connections 2023-12-07T10:17:42.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.827+0000 7f90b7fff700 1 --2- 172.21.15.119:0/3898414023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90b0103080 0x7f90b0109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:42.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.827+0000 7f90b7fff700 1 --2- 172.21.15.119:0/3898414023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b01014f0 0x7f90b0101910 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:42.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.827+0000 7f90b7fff700 1 --2- 172.21.15.119:0/3898414023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90b01026e0 0x7f90b0102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:42.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.827+0000 7f90b7fff700 1 -- 172.21.15.119:0/3898414023 >> 172.21.15.119:0/3898414023 conn(0x7f90b00fca80 msgr2=0x7f90b00feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:42.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.827+0000 7f90b7fff700 1 -- 172.21.15.119:0/3898414023 shutdown_connections 2023-12-07T10:17:42.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.827+0000 7f90b7fff700 1 -- 172.21.15.119:0/3898414023 wait complete. 2023-12-07T10:17:42.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.828+0000 7f90b7fff700 1 Processor -- start 2023-12-07T10:17:42.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.828+0000 7f90b7fff700 1 -- start start 2023-12-07T10:17:42.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.828+0000 7f90b7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b01014f0 0x7f90b0197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:42.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.828+0000 7f90b7fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90b01026e0 0x7f90b0197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:42.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.828+0000 7f90b7fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90b0103080 0x7f90b019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:42.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.828+0000 7f90b7fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b019ef60 con 0x7f90b01014f0 2023-12-07T10:17:42.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.828+0000 7f90b7fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b019f0a0 con 0x7f90b0103080 2023-12-07T10:17:42.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.828+0000 7f90b7fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b019f230 con 0x7f90b01026e0 2023-12-07T10:17:42.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.828+0000 7f90b5d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b01014f0 0x7f90b0197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:42.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.828+0000 7f90b5d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b01014f0 0x7f90b0197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42072/0 (socket says 172.21.15.119:42072) 2023-12-07T10:17:42.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.828+0000 7f90b5d9b700 1 -- 172.21.15.119:0/3980807835 learned_addr learned my addr 172.21.15.119:0/3980807835 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:42.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.829+0000 7f90b659c700 1 --2- 172.21.15.119:0/3980807835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90b0103080 0x7f90b019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:42.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.829+0000 7f90b559a700 1 --2- 172.21.15.119:0/3980807835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90b01026e0 0x7f90b0197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:42.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.829+0000 7f90b5d9b700 1 -- 172.21.15.119:0/3980807835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90b01026e0 msgr2=0x7f90b0197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:42.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.829+0000 7f90b5d9b700 1 --2- 172.21.15.119:0/3980807835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90b01026e0 0x7f90b0197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:42.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.829+0000 7f90b5d9b700 1 -- 172.21.15.119:0/3980807835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90b0103080 msgr2=0x7f90b019e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:42.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.829+0000 7f90b5d9b700 1 --2- 172.21.15.119:0/3980807835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90b0103080 0x7f90b019e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:42.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.829+0000 7f90b5d9b700 1 -- 172.21.15.119:0/3980807835 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ac0095f0 con 0x7f90b01014f0 2023-12-07T10:17:42.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.829+0000 7f90b5d9b700 1 --2- 172.21.15.119:0/3980807835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b01014f0 0x7f90b0197430 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f90ac003730 tx=0x7f90ac00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:42.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.829+0000 7f90a6ffd700 1 -- 172.21.15.119:0/3980807835 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ac01d070 con 0x7f90b01014f0 2023-12-07T10:17:42.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.829+0000 7f90a6ffd700 1 -- 172.21.15.119:0/3980807835 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90ac00fe70 con 0x7f90b01014f0 2023-12-07T10:17:42.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.829+0000 7f90b7fff700 1 -- 172.21.15.119:0/3980807835 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b019f4b0 con 0x7f90b01014f0 2023-12-07T10:17:42.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.829+0000 7f90a6ffd700 1 -- 172.21.15.119:0/3980807835 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ac004b30 con 0x7f90b01014f0 2023-12-07T10:17:42.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.830+0000 7f90b7fff700 1 -- 172.21.15.119:0/3980807835 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b019f930 con 0x7f90b01014f0 2023-12-07T10:17:42.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.831+0000 7f90b7fff700 1 -- 172.21.15.119:0/3980807835 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b01088f0 con 0x7f90b01014f0 2023-12-07T10:17:42.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.832+0000 7f90a6ffd700 1 -- 172.21.15.119:0/3980807835 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f90ac004cf0 con 0x7f90b01014f0 2023-12-07T10:17:42.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.834+0000 7f90a6ffd700 1 --2- 172.21.15.119:0/3980807835 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f909c0a1b80 0x7f909c0a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:42.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.834+0000 7f90a6ffd700 1 -- 172.21.15.119:0/3980807835 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f90ac0ce3e0 con 0x7f90b01014f0 2023-12-07T10:17:42.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.834+0000 7f90b559a700 1 --2- 172.21.15.119:0/3980807835 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f909c0a1b80 0x7f909c0a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:42.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.834+0000 7f90b559a700 1 --2- 172.21.15.119:0/3980807835 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f909c0a1b80 0x7f909c0a4040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009bf0 tx=0x7f90a0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:42.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:42.838+0000 7f90a6ffd700 1 -- 172.21.15.119:0/3980807835 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f90ac097850 con 0x7f90b01014f0 2023-12-07T10:17:42.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:42 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1236934120' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-12-07T10:17:42.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:42 smithi119 ceph-mon[97908]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-12-07T10:17:42.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:42 smithi119 ceph-mon[97908]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-12-07T10:17:42.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:42 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/142538845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-12-07T10:17:42.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:42 smithi119 ceph-mon[97908]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-12-07T10:17:42.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:42 smithi119 ceph-mon[97908]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-12-07T10:17:42.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:42 smithi119 ceph-mon[97908]: osdmap e85: 12 total, 12 up, 12 in 2023-12-07T10:17:43.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:42 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1236934120' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-12-07T10:17:43.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:42 smithi159 ceph-mon[134726]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-12-07T10:17:43.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:42 smithi159 ceph-mon[134726]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-12-07T10:17:43.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:42 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/142538845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-12-07T10:17:43.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:42 smithi159 ceph-mon[134726]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-12-07T10:17:43.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:42 smithi159 ceph-mon[134726]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-12-07T10:17:43.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:42 smithi159 ceph-mon[134726]: osdmap e85: 12 total, 12 up, 12 in 2023-12-07T10:17:43.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:42 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1236934120' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-12-07T10:17:43.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:42 smithi153 ceph-mon[133625]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-12-07T10:17:43.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:42 smithi153 ceph-mon[133625]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-12-07T10:17:43.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:42 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/142538845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-12-07T10:17:43.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:42 smithi153 ceph-mon[133625]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-12-07T10:17:43.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:42 smithi153 ceph-mon[133625]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-12-07T10:17:43.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:42 smithi153 ceph-mon[133625]: osdmap e85: 12 total, 12 up, 12 in 2023-12-07T10:17:43.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:43.038+0000 7f90b7fff700 1 -- 172.21.15.119:0/3980807835 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f90b0068270 con 0x7f90b01014f0 2023-12-07T10:17:44.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:43 smithi159 ceph-mon[134726]: pgmap v276: 97 pgs: 6 creating+peering, 61 unknown, 30 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 5 op/s 2023-12-07T10:17:44.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:43 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] up:active 2023-12-07T10:17:44.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:43 smithi159 ceph-mon[134726]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-07T10:17:44.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:43 smithi159 ceph-mon[134726]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-07T10:17:44.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:43 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3980807835' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-12-07T10:17:44.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:43 smithi159 ceph-mon[134726]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-12-07T10:17:44.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:43 smithi159 ceph-mon[134726]: Cluster is now healthy 2023-12-07T10:17:44.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:43 smithi153 ceph-mon[133625]: pgmap v276: 97 pgs: 6 creating+peering, 61 unknown, 30 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 5 op/s 2023-12-07T10:17:44.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:43 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] up:active 2023-12-07T10:17:44.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:43 smithi153 ceph-mon[133625]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-07T10:17:44.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:43 smithi153 ceph-mon[133625]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-07T10:17:44.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:43 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3980807835' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-12-07T10:17:44.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:43 smithi153 ceph-mon[133625]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-12-07T10:17:44.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:43 smithi153 ceph-mon[133625]: Cluster is now healthy 2023-12-07T10:17:44.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:43 smithi119 ceph-mon[97908]: pgmap v276: 97 pgs: 6 creating+peering, 61 unknown, 30 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 5 op/s 2023-12-07T10:17:44.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:43 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] up:active 2023-12-07T10:17:44.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:43 smithi119 ceph-mon[97908]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-07T10:17:44.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:43 smithi119 ceph-mon[97908]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-07T10:17:44.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:43 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3980807835' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-12-07T10:17:44.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:43 smithi119 ceph-mon[97908]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-12-07T10:17:44.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:43 smithi119 ceph-mon[97908]: Cluster is now healthy 2023-12-07T10:17:44.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:44.771+0000 7f90a6ffd700 1 -- 172.21.15.119:0/3980807835 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) v1 ==== 228+0+0 (secure 0 0 0) 0x7f90ac09ae70 con 0x7f90b01014f0 2023-12-07T10:17:44.775 INFO:journalctl@ceph.mds.b.smithi153.stdout:Dec 07 10:17:44 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-b[170442]: 2023-12-07T10:17:44.499+0000 7f05571b3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-07T10:17:44.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:44.774+0000 7f90b7fff700 1 -- 172.21.15.119:0/3980807835 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f909c0a1b80 msgr2=0x7f909c0a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:44.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:44.774+0000 7f90b7fff700 1 --2- 172.21.15.119:0/3980807835 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f909c0a1b80 0x7f909c0a4040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009bf0 tx=0x7f90a0009260 comp rx=0 tx=0).stop 2023-12-07T10:17:44.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:44.774+0000 7f90b7fff700 1 -- 172.21.15.119:0/3980807835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b01014f0 msgr2=0x7f90b0197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:44.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:44.774+0000 7f90b7fff700 1 --2- 172.21.15.119:0/3980807835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b01014f0 0x7f90b0197430 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f90ac003730 tx=0x7f90ac00fa00 comp rx=0 tx=0).stop 2023-12-07T10:17:44.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:44.775+0000 7f90b7fff700 1 -- 172.21.15.119:0/3980807835 shutdown_connections 2023-12-07T10:17:44.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:44.775+0000 7f90b7fff700 1 --2- 172.21.15.119:0/3980807835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90b0103080 0x7f90b019e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:44.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:44.775+0000 7f90b7fff700 1 --2- 172.21.15.119:0/3980807835 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f909c0a1b80 0x7f909c0a4040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:44.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:44.775+0000 7f90b7fff700 1 --2- 172.21.15.119:0/3980807835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b01014f0 0x7f90b0197430 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:44.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:44.775+0000 7f90b7fff700 1 --2- 172.21.15.119:0/3980807835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90b01026e0 0x7f90b0197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:44.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:44.775+0000 7f90b7fff700 1 -- 172.21.15.119:0/3980807835 >> 172.21.15.119:0/3980807835 conn(0x7f90b00fca80 msgr2=0x7f90b0105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:44.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:44.775+0000 7f90b7fff700 1 -- 172.21.15.119:0/3980807835 shutdown_connections 2023-12-07T10:17:44.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:44.775+0000 7f90b7fff700 1 -- 172.21.15.119:0/3980807835 wait complete. 2023-12-07T10:17:44.778 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data_ec' created 2023-12-07T10:17:44.803 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-12-07T10:17:45.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.004+0000 7f7cc07ff700 1 -- 172.21.15.119:0/3604721475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7cb81027b0 msgr2=0x7f7cb8102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:45.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.004+0000 7f7cc07ff700 1 --2- 172.21.15.119:0/3604721475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7cb81027b0 0x7f7cb8102c10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009990 tx=0x7f7ca8009ca0 comp rx=0 tx=0).stop 2023-12-07T10:17:45.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.004+0000 7f7cc07ff700 1 -- 172.21.15.119:0/3604721475 shutdown_connections 2023-12-07T10:17:45.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.004+0000 7f7cc07ff700 1 --2- 172.21.15.119:0/3604721475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb8103150 0x7f7cb8109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:45.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.004+0000 7f7cc07ff700 1 --2- 172.21.15.119:0/3604721475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb81015c0 0x7f7cb81019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:45.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.004+0000 7f7cc07ff700 1 --2- 172.21.15.119:0/3604721475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7cb81027b0 0x7f7cb8102c10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:45.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.004+0000 7f7cc07ff700 1 -- 172.21.15.119:0/3604721475 >> 172.21.15.119:0/3604721475 conn(0x7f7cb80fcb10 msgr2=0x7f7cb80fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:45.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.004+0000 7f7cc07ff700 1 -- 172.21.15.119:0/3604721475 shutdown_connections 2023-12-07T10:17:45.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.004+0000 7f7cc07ff700 1 -- 172.21.15.119:0/3604721475 wait complete. 2023-12-07T10:17:45.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.005+0000 7f7cc07ff700 1 Processor -- start 2023-12-07T10:17:45.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.005+0000 7f7cc07ff700 1 -- start start 2023-12-07T10:17:45.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.005+0000 7f7cc07ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7cb81015c0 0x7f7cb8199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:45.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.005+0000 7f7cc07ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb81027b0 0x7f7cb8199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:45.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.006+0000 7f7cc07ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb8103150 0x7f7cb81a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:45.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.006+0000 7f7cc07ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb81a1190 con 0x7f7cb81027b0 2023-12-07T10:17:45.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.006+0000 7f7cc07ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb81a12d0 con 0x7f7cb8103150 2023-12-07T10:17:45.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.006+0000 7f7cc07ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb81a1460 con 0x7f7cb81015c0 2023-12-07T10:17:45.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.006+0000 7f7cbdd9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb81027b0 0x7f7cb8199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:45.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.006+0000 7f7cbdd9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb81027b0 0x7f7cb8199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42092/0 (socket says 172.21.15.119:42092) 2023-12-07T10:17:45.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.006+0000 7f7cbe59b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7cb81015c0 0x7f7cb8199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:45.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.006+0000 7f7cbdd9a700 1 -- 172.21.15.119:0/1113060369 learned_addr learned my addr 172.21.15.119:0/1113060369 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:45.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.006+0000 7f7cbed9c700 1 --2- 172.21.15.119:0/1113060369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb8103150 0x7f7cb81a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:45.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.006+0000 7f7cbe59b700 1 -- 172.21.15.119:0/1113060369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb8103150 msgr2=0x7f7cb81a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:45.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.006+0000 7f7cbe59b700 1 --2- 172.21.15.119:0/1113060369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb8103150 0x7f7cb81a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:45.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.006+0000 7f7cbe59b700 1 -- 172.21.15.119:0/1113060369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb81027b0 msgr2=0x7f7cb8199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:45.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.006+0000 7f7cbe59b700 1 --2- 172.21.15.119:0/1113060369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb81027b0 0x7f7cb8199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:45.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.006+0000 7f7cbe59b700 1 -- 172.21.15.119:0/1113060369 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca80095f0 con 0x7f7cb81015c0 2023-12-07T10:17:45.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.006+0000 7f7cbdd9a700 1 --2- 172.21.15.119:0/1113060369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb81027b0 0x7f7cb8199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:17:45.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.007+0000 7f7cbe59b700 1 --2- 172.21.15.119:0/1113060369 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7cb81015c0 0x7f7cb8199660 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7cb400c940 tx=0x7f7cb400cc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:45.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.007+0000 7f7caf7fe700 1 -- 172.21.15.119:0/1113060369 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4004230 con 0x7f7cb81015c0 2023-12-07T10:17:45.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.007+0000 7f7cc07ff700 1 -- 172.21.15.119:0/1113060369 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb81a1740 con 0x7f7cb81015c0 2023-12-07T10:17:45.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.007+0000 7f7caf7fe700 1 -- 172.21.15.119:0/1113060369 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7cb4004390 con 0x7f7cb81015c0 2023-12-07T10:17:45.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.007+0000 7f7caf7fe700 1 -- 172.21.15.119:0/1113060369 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4003b20 con 0x7f7cb81015c0 2023-12-07T10:17:45.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.007+0000 7f7cc07ff700 1 -- 172.21.15.119:0/1113060369 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb81a1ca0 con 0x7f7cb81015c0 2023-12-07T10:17:45.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.008+0000 7f7cc07ff700 1 -- 172.21.15.119:0/1113060369 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb80683c0 con 0x7f7cb81015c0 2023-12-07T10:17:45.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.010+0000 7f7caf7fe700 1 -- 172.21.15.119:0/1113060369 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f7cb4003ca0 con 0x7f7cb81015c0 2023-12-07T10:17:45.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.011+0000 7f7caf7fe700 1 --2- 172.21.15.119:0/1113060369 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f7ca40a1b10 0x7f7ca40a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:45.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.011+0000 7f7caf7fe700 1 -- 172.21.15.119:0/1113060369 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f7cb4013070 con 0x7f7cb81015c0 2023-12-07T10:17:45.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.012+0000 7f7cbdd9a700 1 --2- 172.21.15.119:0/1113060369 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f7ca40a1b10 0x7f7ca40a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:45.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.012+0000 7f7cbdd9a700 1 --2- 172.21.15.119:0/1113060369 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f7ca40a1b10 0x7f7ca40a3fd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8006010 tx=0x7f7ca800b770 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:45.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.014+0000 7f7caf7fe700 1 -- 172.21.15.119:0/1113060369 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7cb4096b10 con 0x7f7cb81015c0 2023-12-07T10:17:45.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:44 smithi153 ceph-mon[133625]: osdmap e86: 12 total, 12 up, 12 in 2023-12-07T10:17:45.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.218+0000 7f7cc07ff700 1 -- 172.21.15.119:0/1113060369 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f7cb810cde0 con 0x7f7cb81015c0 2023-12-07T10:17:45.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:44 smithi119 ceph-mon[97908]: osdmap e86: 12 total, 12 up, 12 in 2023-12-07T10:17:45.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:44 smithi159 ceph-mon[134726]: osdmap e86: 12 total, 12 up, 12 in 2023-12-07T10:17:45.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.777+0000 7f7caf7fe700 1 -- 172.21.15.119:0/1113060369 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7cb4007e60 con 0x7f7cb81015c0 2023-12-07T10:17:45.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.781+0000 7f7cc07ff700 1 -- 172.21.15.119:0/1113060369 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f7ca40a1b10 msgr2=0x7f7ca40a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:45.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.782+0000 7f7cc07ff700 1 --2- 172.21.15.119:0/1113060369 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f7ca40a1b10 0x7f7ca40a3fd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8006010 tx=0x7f7ca800b770 comp rx=0 tx=0).stop 2023-12-07T10:17:45.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.782+0000 7f7cc07ff700 1 -- 172.21.15.119:0/1113060369 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7cb81015c0 msgr2=0x7f7cb8199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:45.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.782+0000 7f7cc07ff700 1 --2- 172.21.15.119:0/1113060369 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7cb81015c0 0x7f7cb8199660 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7cb400c940 tx=0x7f7cb400cc50 comp rx=0 tx=0).stop 2023-12-07T10:17:45.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.782+0000 7f7cc07ff700 1 -- 172.21.15.119:0/1113060369 shutdown_connections 2023-12-07T10:17:45.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.782+0000 7f7cc07ff700 1 --2- 172.21.15.119:0/1113060369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb8103150 0x7f7cb81a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:45.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.782+0000 7f7cc07ff700 1 --2- 172.21.15.119:0/1113060369 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7cb81015c0 0x7f7cb8199660 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:45.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.782+0000 7f7cc07ff700 1 --2- 172.21.15.119:0/1113060369 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f7ca40a1b10 0x7f7ca40a3fd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:45.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.782+0000 7f7cc07ff700 1 --2- 172.21.15.119:0/1113060369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cb81027b0 0x7f7cb8199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:45.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.782+0000 7f7cc07ff700 1 -- 172.21.15.119:0/1113060369 >> 172.21.15.119:0/1113060369 conn(0x7f7cb80fcb10 msgr2=0x7f7cb8073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:45.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.782+0000 7f7cc07ff700 1 -- 172.21.15.119:0/1113060369 shutdown_connections 2023-12-07T10:17:45.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:45.782+0000 7f7cc07ff700 1 -- 172.21.15.119:0/1113060369 wait complete. 2023-12-07T10:17:45.792 INFO:teuthology.orchestra.run.smithi119.stderr:set pool 4 allow_ec_overwrites to true 2023-12-07T10:17:45.817 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-12-07T10:17:46.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.016+0000 7f9b67fff700 1 -- 172.21.15.119:0/749645174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b601015c0 msgr2=0x7f9b601019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:46.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.016+0000 7f9b67fff700 1 --2- 172.21.15.119:0/749645174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b601015c0 0x7f9b601019e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c0099c0 tx=0x7f9b5c009cd0 comp rx=0 tx=0).stop 2023-12-07T10:17:46.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.016+0000 7f9b67fff700 1 -- 172.21.15.119:0/749645174 shutdown_connections 2023-12-07T10:17:46.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.016+0000 7f9b67fff700 1 --2- 172.21.15.119:0/749645174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b60103150 0x7f9b60109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:46.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.016+0000 7f9b67fff700 1 --2- 172.21.15.119:0/749645174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b601015c0 0x7f9b601019e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:46.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.016+0000 7f9b67fff700 1 --2- 172.21.15.119:0/749645174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b601027b0 0x7f9b60102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:46.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.016+0000 7f9b67fff700 1 -- 172.21.15.119:0/749645174 >> 172.21.15.119:0/749645174 conn(0x7f9b600fcb10 msgr2=0x7f9b600fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:46.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.016+0000 7f9b67fff700 1 -- 172.21.15.119:0/749645174 shutdown_connections 2023-12-07T10:17:46.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.016+0000 7f9b67fff700 1 -- 172.21.15.119:0/749645174 wait complete. 2023-12-07T10:17:46.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.017+0000 7f9b67fff700 1 Processor -- start 2023-12-07T10:17:46.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.017+0000 7f9b67fff700 1 -- start start 2023-12-07T10:17:46.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.017+0000 7f9b67fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b601015c0 0x7f9b60197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:46.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.017+0000 7f9b67fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b601027b0 0x7f9b60197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:46.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.017+0000 7f9b67fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b60103150 0x7f9b6019ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:46.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.017+0000 7f9b67fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6019f060 con 0x7f9b601027b0 2023-12-07T10:17:46.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.017+0000 7f9b67fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6019f1a0 con 0x7f9b60103150 2023-12-07T10:17:46.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.017+0000 7f9b67fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6019f330 con 0x7f9b601015c0 2023-12-07T10:17:46.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.018+0000 7f9b6559a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b601027b0 0x7f9b60197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:46.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.018+0000 7f9b65d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b601015c0 0x7f9b60197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:46.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.018+0000 7f9b6559a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b601027b0 0x7f9b60197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42110/0 (socket says 172.21.15.119:42110) 2023-12-07T10:17:46.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.018+0000 7f9b6559a700 1 -- 172.21.15.119:0/879077932 learned_addr learned my addr 172.21.15.119:0/879077932 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:46.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.018+0000 7f9b6659c700 1 --2- 172.21.15.119:0/879077932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b60103150 0x7f9b6019ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:46.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.018+0000 7f9b6559a700 1 -- 172.21.15.119:0/879077932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b601015c0 msgr2=0x7f9b60197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:46.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.018+0000 7f9b6559a700 1 --2- 172.21.15.119:0/879077932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b601015c0 0x7f9b60197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:46.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.018+0000 7f9b6559a700 1 -- 172.21.15.119:0/879077932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b60103150 msgr2=0x7f9b6019ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:46.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.018+0000 7f9b6559a700 1 --2- 172.21.15.119:0/879077932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b60103150 0x7f9b6019ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:46.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.018+0000 7f9b6559a700 1 -- 172.21.15.119:0/879077932 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b5c0095f0 con 0x7f9b601027b0 2023-12-07T10:17:46.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.018+0000 7f9b6659c700 1 --2- 172.21.15.119:0/879077932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b60103150 0x7f9b6019ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:17:46.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.018+0000 7f9b6559a700 1 --2- 172.21.15.119:0/879077932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b601027b0 0x7f9b60197a70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000d980 tx=0x7f9b5000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:46.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.018+0000 7f9b56ffd700 1 -- 172.21.15.119:0/879077932 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50004220 con 0x7f9b601027b0 2023-12-07T10:17:46.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.019+0000 7f9b56ffd700 1 -- 172.21.15.119:0/879077932 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b50007640 con 0x7f9b601027b0 2023-12-07T10:17:46.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.019+0000 7f9b67fff700 1 -- 172.21.15.119:0/879077932 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b6019f610 con 0x7f9b601027b0 2023-12-07T10:17:46.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.019+0000 7f9b56ffd700 1 -- 172.21.15.119:0/879077932 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50005610 con 0x7f9b601027b0 2023-12-07T10:17:46.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.019+0000 7f9b67fff700 1 -- 172.21.15.119:0/879077932 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b6019fb20 con 0x7f9b601027b0 2023-12-07T10:17:46.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.020+0000 7f9b67fff700 1 -- 172.21.15.119:0/879077932 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b60108a90 con 0x7f9b601027b0 2023-12-07T10:17:46.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.022+0000 7f9b56ffd700 1 -- 172.21.15.119:0/879077932 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f9b50004750 con 0x7f9b601027b0 2023-12-07T10:17:46.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.023+0000 7f9b56ffd700 1 --2- 172.21.15.119:0/879077932 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f9b4c0a1b10 0x7f9b4c0a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:46.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.023+0000 7f9b56ffd700 1 -- 172.21.15.119:0/879077932 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f9b500cdbc0 con 0x7f9b601027b0 2023-12-07T10:17:46.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.023+0000 7f9b65d9b700 1 --2- 172.21.15.119:0/879077932 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f9b4c0a1b10 0x7f9b4c0a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:46.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.024+0000 7f9b65d9b700 1 --2- 172.21.15.119:0/879077932 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f9b4c0a1b10 0x7f9b4c0a3fd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c006010 tx=0x7f9b5c00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:46.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.026+0000 7f9b56ffd700 1 -- 172.21.15.119:0/879077932 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9b50096db0 con 0x7f9b601027b0 2023-12-07T10:17:46.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:45 smithi119 ceph-mon[97908]: pgmap v278: 97 pgs: 6 creating+peering, 20 unknown, 71 active+clean; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2023-12-07T10:17:46.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:45 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3980807835' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-12-07T10:17:46.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:45 smithi119 ceph-mon[97908]: osdmap e87: 12 total, 12 up, 12 in 2023-12-07T10:17:46.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:45 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1113060369' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-12-07T10:17:46.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:45 smithi119 ceph-mon[97908]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-12-07T10:17:46.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:45 smithi159 ceph-mon[134726]: pgmap v278: 97 pgs: 6 creating+peering, 20 unknown, 71 active+clean; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2023-12-07T10:17:46.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:45 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3980807835' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-12-07T10:17:46.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:45 smithi159 ceph-mon[134726]: osdmap e87: 12 total, 12 up, 12 in 2023-12-07T10:17:46.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:45 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1113060369' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-12-07T10:17:46.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:45 smithi159 ceph-mon[134726]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-12-07T10:17:46.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.275+0000 7f9b67fff700 1 -- 172.21.15.119:0/879077932 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f9b600683c0 con 0x7f9b601027b0 2023-12-07T10:17:46.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:45 smithi153 ceph-mon[133625]: pgmap v278: 97 pgs: 6 creating+peering, 20 unknown, 71 active+clean; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2023-12-07T10:17:46.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:45 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3980807835' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-12-07T10:17:46.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:45 smithi153 ceph-mon[133625]: osdmap e87: 12 total, 12 up, 12 in 2023-12-07T10:17:46.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:45 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1113060369' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-12-07T10:17:46.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:45 smithi153 ceph-mon[133625]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-12-07T10:17:46.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.789+0000 7f9b56ffd700 1 -- 172.21.15.119:0/879077932 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f9b50083e60 con 0x7f9b601027b0 2023-12-07T10:17:46.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.792+0000 7f9b67fff700 1 -- 172.21.15.119:0/879077932 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f9b4c0a1b10 msgr2=0x7f9b4c0a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:46.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.792+0000 7f9b67fff700 1 --2- 172.21.15.119:0/879077932 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f9b4c0a1b10 0x7f9b4c0a3fd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c006010 tx=0x7f9b5c00b540 comp rx=0 tx=0).stop 2023-12-07T10:17:46.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.792+0000 7f9b67fff700 1 -- 172.21.15.119:0/879077932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b601027b0 msgr2=0x7f9b60197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:46.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.792+0000 7f9b67fff700 1 --2- 172.21.15.119:0/879077932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b601027b0 0x7f9b60197a70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000d980 tx=0x7f9b5000dc90 comp rx=0 tx=0).stop 2023-12-07T10:17:46.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.792+0000 7f9b67fff700 1 -- 172.21.15.119:0/879077932 shutdown_connections 2023-12-07T10:17:46.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.792+0000 7f9b67fff700 1 --2- 172.21.15.119:0/879077932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b60103150 0x7f9b6019ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:46.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.792+0000 7f9b67fff700 1 --2- 172.21.15.119:0/879077932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b601015c0 0x7f9b60197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:46.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.792+0000 7f9b67fff700 1 --2- 172.21.15.119:0/879077932 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f9b4c0a1b10 0x7f9b4c0a3fd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:46.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.792+0000 7f9b67fff700 1 --2- 172.21.15.119:0/879077932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b601027b0 0x7f9b60197a70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:46.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.792+0000 7f9b67fff700 1 -- 172.21.15.119:0/879077932 >> 172.21.15.119:0/879077932 conn(0x7f9b600fcb10 msgr2=0x7f9b60105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:46.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.792+0000 7f9b67fff700 1 -- 172.21.15.119:0/879077932 shutdown_connections 2023-12-07T10:17:46.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:46.792+0000 7f9b67fff700 1 -- 172.21.15.119:0/879077932 wait complete. 2023-12-07T10:17:46.795 INFO:teuthology.orchestra.run.smithi119.stderr:added data pool 4 to fsmap 2023-12-07T10:17:46.813 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:17:47.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.008+0000 7f4e4edff700 1 -- 172.21.15.119:0/1269709549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e48102750 msgr2=0x7f4e48102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:47.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.008+0000 7f4e4edff700 1 --2- 172.21.15.119:0/1269709549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e48102750 0x7f4e48102bb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4e340099c0 tx=0x7f4e34009cd0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.008+0000 7f4e4edff700 1 -- 172.21.15.119:0/1269709549 shutdown_connections 2023-12-07T10:17:47.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.008+0000 7f4e4edff700 1 --2- 172.21.15.119:0/1269709549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e481030f0 0x7f4e481099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.008+0000 7f4e4edff700 1 --2- 172.21.15.119:0/1269709549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e48101560 0x7f4e48101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.008+0000 7f4e4edff700 1 --2- 172.21.15.119:0/1269709549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e48102750 0x7f4e48102bb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.008+0000 7f4e4edff700 1 -- 172.21.15.119:0/1269709549 >> 172.21.15.119:0/1269709549 conn(0x7f4e480fcab0 msgr2=0x7f4e480fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:47.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.008+0000 7f4e4edff700 1 -- 172.21.15.119:0/1269709549 shutdown_connections 2023-12-07T10:17:47.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.008+0000 7f4e4edff700 1 -- 172.21.15.119:0/1269709549 wait complete. 2023-12-07T10:17:47.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.009+0000 7f4e4edff700 1 Processor -- start 2023-12-07T10:17:47.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.009+0000 7f4e4edff700 1 -- start start 2023-12-07T10:17:47.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.010+0000 7f4e4edff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e48101560 0x7f4e48197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:47.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.010+0000 7f4e4edff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e48102750 0x7f4e48197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:47.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.010+0000 7f4e4edff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e481030f0 0x7f4e4819ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:47.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.010+0000 7f4e4edff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4819f030 con 0x7f4e48102750 2023-12-07T10:17:47.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.010+0000 7f4e4edff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4819f170 con 0x7f4e48101560 2023-12-07T10:17:47.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.010+0000 7f4e4edff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4819f300 con 0x7f4e481030f0 2023-12-07T10:17:47.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.010+0000 7f4e47fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e48102750 0x7f4e48197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:47.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.010+0000 7f4e47fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e48102750 0x7f4e48197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42122/0 (socket says 172.21.15.119:42122) 2023-12-07T10:17:47.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.010+0000 7f4e47fff700 1 -- 172.21.15.119:0/1190987535 learned_addr learned my addr 172.21.15.119:0/1190987535 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:47.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.010+0000 7f4e4d39c700 1 --2- 172.21.15.119:0/1190987535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e481030f0 0x7f4e4819ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:47.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.010+0000 7f4e4cb9b700 1 --2- 172.21.15.119:0/1190987535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e48101560 0x7f4e48197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:47.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.010+0000 7f4e47fff700 1 -- 172.21.15.119:0/1190987535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e481030f0 msgr2=0x7f4e4819ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:47.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.010+0000 7f4e47fff700 1 --2- 172.21.15.119:0/1190987535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e481030f0 0x7f4e4819ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.010+0000 7f4e47fff700 1 -- 172.21.15.119:0/1190987535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e48101560 msgr2=0x7f4e48197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:47.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.010+0000 7f4e47fff700 1 --2- 172.21.15.119:0/1190987535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e48101560 0x7f4e48197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.010+0000 7f4e47fff700 1 -- 172.21.15.119:0/1190987535 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e340095f0 con 0x7f4e48102750 2023-12-07T10:17:47.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.010+0000 7f4e47fff700 1 --2- 172.21.15.119:0/1190987535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e48102750 0x7f4e48197a40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4e34003730 tx=0x7f4e3400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:47.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.011+0000 7f4e45ffb700 1 -- 172.21.15.119:0/1190987535 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3401d070 con 0x7f4e48102750 2023-12-07T10:17:47.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.011+0000 7f4e45ffb700 1 -- 172.21.15.119:0/1190987535 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e3400bda0 con 0x7f4e48102750 2023-12-07T10:17:47.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.011+0000 7f4e4edff700 1 -- 172.21.15.119:0/1190987535 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e4819f580 con 0x7f4e48102750 2023-12-07T10:17:47.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.011+0000 7f4e45ffb700 1 -- 172.21.15.119:0/1190987535 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e34004bd0 con 0x7f4e48102750 2023-12-07T10:17:47.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.011+0000 7f4e4edff700 1 -- 172.21.15.119:0/1190987535 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e4819fa00 con 0x7f4e48102750 2023-12-07T10:17:47.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.012+0000 7f4e4edff700 1 -- 172.21.15.119:0/1190987535 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e48108a30 con 0x7f4e48102750 2023-12-07T10:17:47.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.014+0000 7f4e45ffb700 1 -- 172.21.15.119:0/1190987535 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f4e34004d30 con 0x7f4e48102750 2023-12-07T10:17:47.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.016+0000 7f4e45ffb700 1 --2- 172.21.15.119:0/1190987535 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4e380a1b90 0x7f4e380a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:47.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.016+0000 7f4e45ffb700 1 -- 172.21.15.119:0/1190987535 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4e340ce550 con 0x7f4e48102750 2023-12-07T10:17:47.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.016+0000 7f4e4cb9b700 1 --2- 172.21.15.119:0/1190987535 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4e380a1b90 0x7f4e380a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:47.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.017+0000 7f4e4cb9b700 1 --2- 172.21.15.119:0/1190987535 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4e380a1b90 0x7f4e380a4050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4e481025b0 tx=0x7f4e3c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:47.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.020+0000 7f4e45ffb700 1 -- 172.21.15.119:0/1190987535 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4e340977b0 con 0x7f4e48102750 2023-12-07T10:17:47.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:46 smithi119 ceph-mon[97908]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-12-07T10:17:47.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:46 smithi119 ceph-mon[97908]: osdmap e88: 12 total, 12 up, 12 in 2023-12-07T10:17:47.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:46 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/879077932' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-12-07T10:17:47.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:46 smithi159 ceph-mon[134726]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-12-07T10:17:47.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:46 smithi159 ceph-mon[134726]: osdmap e88: 12 total, 12 up, 12 in 2023-12-07T10:17:47.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:46 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/879077932' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-12-07T10:17:47.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.266+0000 7f4e4edff700 1 -- 172.21.15.119:0/1190987535 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e48068350 con 0x7f4e48102750 2023-12-07T10:17:47.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.267+0000 7f4e45ffb700 1 -- 172.21.15.119:0/1190987535 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11273 (secure 0 0 0) 0x7f4e3409add0 con 0x7f4e48102750 2023-12-07T10:17:47.268 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:17:47.269 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14535,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6841/1791998028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1791998028},{"type":"v1","addr":"172.21.15.119:6841","nonce":1791998028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2925503246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2925503246},{"type":"v1","addr":"172.21.15.153:6837","nonce":2925503246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/3984962900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3984962900},{"type":"v1","addr":"172.21.15.159:6839","nonce":3984962900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24542,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6839/1920762602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1920762602},{"type":"v1","addr":"172.21.15.119:6839","nonce":1920762602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-07T10:17:41.278311+0000","modified":"2023-12-07T10:17:46.277320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24566},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/566825184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":566825184},{"type":"v1","addr":"172.21.15.153:6839","nonce":566825184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-12-07T10:17:47.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.269+0000 7f4e4edff700 1 -- 172.21.15.119:0/1190987535 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4e380a1b90 msgr2=0x7f4e380a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:47.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.269+0000 7f4e4edff700 1 --2- 172.21.15.119:0/1190987535 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4e380a1b90 0x7f4e380a4050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4e481025b0 tx=0x7f4e3c009260 comp rx=0 tx=0).stop 2023-12-07T10:17:47.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.269+0000 7f4e4edff700 1 -- 172.21.15.119:0/1190987535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e48102750 msgr2=0x7f4e48197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:47.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.269+0000 7f4e4edff700 1 --2- 172.21.15.119:0/1190987535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e48102750 0x7f4e48197a40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4e34003730 tx=0x7f4e3400fa00 comp rx=0 tx=0).stop 2023-12-07T10:17:47.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.269+0000 7f4e4edff700 1 -- 172.21.15.119:0/1190987535 shutdown_connections 2023-12-07T10:17:47.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.269+0000 7f4e4edff700 1 --2- 172.21.15.119:0/1190987535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e481030f0 0x7f4e4819ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.269+0000 7f4e4edff700 1 --2- 172.21.15.119:0/1190987535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e48101560 0x7f4e48197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.269+0000 7f4e4edff700 1 --2- 172.21.15.119:0/1190987535 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4e380a1b90 0x7f4e380a4050 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.269+0000 7f4e4edff700 1 --2- 172.21.15.119:0/1190987535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e48102750 0x7f4e48197a40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.269+0000 7f4e4edff700 1 -- 172.21.15.119:0/1190987535 >> 172.21.15.119:0/1190987535 conn(0x7f4e480fcab0 msgr2=0x7f4e48105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:47.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.269+0000 7f4e4edff700 1 -- 172.21.15.119:0/1190987535 shutdown_connections 2023-12-07T10:17:47.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.269+0000 7f4e4edff700 1 -- 172.21.15.119:0/1190987535 wait complete. 2023-12-07T10:17:47.274 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 17 2023-12-07T10:17:47.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:46 smithi153 ceph-mon[133625]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-12-07T10:17:47.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:46 smithi153 ceph-mon[133625]: osdmap e88: 12 total, 12 up, 12 in 2023-12-07T10:17:47.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:46 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/879077932' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-12-07T10:17:47.290 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-07T10:17:47.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.485+0000 7fbdb55ff700 1 -- 172.21.15.119:0/498697236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdb0102750 msgr2=0x7fbdb0102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:47.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.485+0000 7fbdb55ff700 1 --2- 172.21.15.119:0/498697236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdb0102750 0x7fbdb0102bb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbda40099c0 tx=0x7fbda4009cd0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.485+0000 7fbdb55ff700 1 -- 172.21.15.119:0/498697236 shutdown_connections 2023-12-07T10:17:47.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.485+0000 7fbdb55ff700 1 --2- 172.21.15.119:0/498697236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdb01030f0 0x7fbdb01099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.485+0000 7fbdb55ff700 1 --2- 172.21.15.119:0/498697236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdb0101560 0x7fbdb0101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.485+0000 7fbdb55ff700 1 --2- 172.21.15.119:0/498697236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdb0102750 0x7fbdb0102bb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.485+0000 7fbdb55ff700 1 -- 172.21.15.119:0/498697236 >> 172.21.15.119:0/498697236 conn(0x7fbdb00fcab0 msgr2=0x7fbdb00fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:47.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.485+0000 7fbdb55ff700 1 -- 172.21.15.119:0/498697236 shutdown_connections 2023-12-07T10:17:47.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.485+0000 7fbdb55ff700 1 -- 172.21.15.119:0/498697236 wait complete. 2023-12-07T10:17:47.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.486+0000 7fbdb55ff700 1 Processor -- start 2023-12-07T10:17:47.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.486+0000 7fbdb55ff700 1 -- start start 2023-12-07T10:17:47.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.487+0000 7fbdb55ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdb0101560 0x7fbdb0199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:47.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.487+0000 7fbdb55ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdb0102750 0x7fbdb0199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:47.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.487+0000 7fbdb55ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdb01030f0 0x7fbdb01a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:47.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.487+0000 7fbdb55ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb01a1250 con 0x7fbdb0102750 2023-12-07T10:17:47.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.487+0000 7fbdb55ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb01a1390 con 0x7fbdb0101560 2023-12-07T10:17:47.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.487+0000 7fbdb55ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb01a1520 con 0x7fbdb01030f0 2023-12-07T10:17:47.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.487+0000 7fbdae7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdb0102750 0x7fbdb0199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:47.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.487+0000 7fbdaeffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdb0101560 0x7fbdb0199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:47.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.487+0000 7fbdae7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdb0102750 0x7fbdb0199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42138/0 (socket says 172.21.15.119:42138) 2023-12-07T10:17:47.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.487+0000 7fbdaeffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdb0101560 0x7fbdb0199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:40168/0 (socket says 172.21.15.119:40168) 2023-12-07T10:17:47.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.487+0000 7fbdae7fc700 1 -- 172.21.15.119:0/1997046409 learned_addr learned my addr 172.21.15.119:0/1997046409 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:47.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.487+0000 7fbdaf7fe700 1 --2- 172.21.15.119:0/1997046409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdb01030f0 0x7fbdb01a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:47.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.487+0000 7fbdaeffd700 1 -- 172.21.15.119:0/1997046409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdb01030f0 msgr2=0x7fbdb01a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:47.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.487+0000 7fbdaeffd700 1 --2- 172.21.15.119:0/1997046409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdb01030f0 0x7fbdb01a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.487+0000 7fbdaeffd700 1 -- 172.21.15.119:0/1997046409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdb0102750 msgr2=0x7fbdb0199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:47.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.487+0000 7fbdaeffd700 1 --2- 172.21.15.119:0/1997046409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdb0102750 0x7fbdb0199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.487+0000 7fbdaeffd700 1 -- 172.21.15.119:0/1997046409 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda40095f0 con 0x7fbdb0101560 2023-12-07T10:17:47.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.487+0000 7fbdae7fc700 1 --2- 172.21.15.119:0/1997046409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdb0102750 0x7fbdb0199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:17:47.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.488+0000 7fbdaeffd700 1 --2- 172.21.15.119:0/1997046409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdb0101560 0x7fbdb0199720 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbda000e9c0 tx=0x7fbda000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:47.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.488+0000 7fbd97fff700 1 -- 172.21.15.119:0/1997046409 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda0004220 con 0x7fbdb0101560 2023-12-07T10:17:47.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.488+0000 7fbdb55ff700 1 -- 172.21.15.119:0/1997046409 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb01a1800 con 0x7fbdb0101560 2023-12-07T10:17:47.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.488+0000 7fbd97fff700 1 -- 172.21.15.119:0/1997046409 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbda0004380 con 0x7fbdb0101560 2023-12-07T10:17:47.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.488+0000 7fbd97fff700 1 -- 172.21.15.119:0/1997046409 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda0003be0 con 0x7fbdb0101560 2023-12-07T10:17:47.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.488+0000 7fbdb55ff700 1 -- 172.21.15.119:0/1997046409 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb01a1d10 con 0x7fbdb0101560 2023-12-07T10:17:47.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.489+0000 7fbdb55ff700 1 -- 172.21.15.119:0/1997046409 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb0068350 con 0x7fbdb0101560 2023-12-07T10:17:47.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.491+0000 7fbd97fff700 1 -- 172.21.15.119:0/1997046409 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fbda0003d40 con 0x7fbdb0101560 2023-12-07T10:17:47.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.492+0000 7fbd97fff700 1 --2- 172.21.15.119:0/1997046409 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fbd980a1b10 0x7fbd980a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:47.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.493+0000 7fbd97fff700 1 -- 172.21.15.119:0/1997046409 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fbda0014070 con 0x7fbdb0101560 2023-12-07T10:17:47.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.493+0000 7fbdae7fc700 1 --2- 172.21.15.119:0/1997046409 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fbd980a1b10 0x7fbd980a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:47.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.493+0000 7fbdae7fc700 1 --2- 172.21.15.119:0/1997046409 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fbd980a1b10 0x7fbd980a3fd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbda4004800 tx=0x7fbda401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:47.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.495+0000 7fbd97fff700 1 -- 172.21.15.119:0/1997046409 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fbda0096de0 con 0x7fbdb0101560 2023-12-07T10:17:47.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.696+0000 7fbdb55ff700 1 -- 172.21.15.119:0/1997046409 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbdb010cfa0 con 0x7fbdb0101560 2023-12-07T10:17:47.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.698+0000 7fbd97fff700 1 -- 172.21.15.119:0/1997046409 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24030 (secure 0 0 0) 0x7fbdb010cfa0 con 0x7fbdb0101560 2023-12-07T10:17:47.702 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:17:47.702 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":89,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","created":"2023-12-07T10:09:59.945863+0000","modified":"2023-12-07T10:17:46.277123+0000","last_up_change":"2023-12-07T10:16:50.929427+0000","last_in_change":"2023-12-07T10:16:33.558158+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T10:13:25.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-07T10:17:39.347813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-07T10:17:40.714501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-07T10:17:44.257760+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"56b5818e-e625-40f1-8838-3ddcf2d34ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6803","nonce":2141950133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6805","nonce":2141950133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6809","nonce":2141950133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6807","nonce":2141950133}]},"public_addr":"172.21.15.119:6803/2141950133","cluster_addr":"172.21.15.119:6805/2141950133","heartbeat_back_addr":"172.21.15.119:6809/2141950133","heartbeat_front_addr":"172.21.15.119:6807/2141950133","state":["exists","up"]},{"osd":1,"uuid":"36489a6a-5278-4802-9525-9871521722e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6801","nonce":3390875370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6803","nonce":3390875370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6807","nonce":3390875370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6805","nonce":3390875370}]},"public_addr":"172.21.15.153:6801/3390875370","cluster_addr":"172.21.15.153:6803/3390875370","heartbeat_back_addr":"172.21.15.153:6807/3390875370","heartbeat_front_addr":"172.21.15.153:6805/3390875370","state":["exists","up"]},{"osd":2,"uuid":"a937ad26-5e9b-4c8a-b65a-626a8feda6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6801","nonce":708101637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6803","nonce":708101637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6807","nonce":708101637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6805","nonce":708101637}]},"public_addr":"172.21.15.159:6801/708101637","cluster_addr":"172.21.15.159:6803/708101637","heartbeat_back_addr":"172.21.15.159:6807/708101637","heartbeat_front_addr":"172.21.15.159:6805/708101637","state":["exists","up"]},{"osd":3,"uuid":"26ef8300-aa07-48f6-b31d-8b0f2217ffcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6811","nonce":3228047257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6813","nonce":3228047257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6817","nonce":3228047257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6815","nonce":3228047257}]},"public_addr":"172.21.15.119:6811/3228047257","cluster_addr":"172.21.15.119:6813/3228047257","heartbeat_back_addr":"172.21.15.119:6817/3228047257","heartbeat_front_addr":"172.21.15.119:6815/3228047257","state":["exists","up"]},{"osd":4,"uuid":"000ee4b9-9896-4901-8cc3-c85fdfc8cbaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6809","nonce":4021998299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6811","nonce":4021998299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6815","nonce":4021998299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6813","nonce":4021998299}]},"public_addr":"172.21.15.153:6809/4021998299","cluster_addr":"172.21.15.153:6811/4021998299","heartbeat_back_addr":"172.21.15.153:6815/4021998299","heartbeat_front_addr":"172.21.15.153:6813/4021998299","state":["exists","up"]},{"osd":5,"uuid":"8b2fc167-262d-474f-89f3-47bd29424421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6809","nonce":1832540768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6811","nonce":1832540768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6815","nonce":1832540768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6813","nonce":1832540768}]},"public_addr":"172.21.15.159:6809/1832540768","cluster_addr":"172.21.15.159:6811/1832540768","heartbeat_back_addr":"172.21.15.159:6815/1832540768","heartbeat_front_addr":"172.21.15.159:6813/1832540768","state":["exists","up"]},{"osd":6,"uuid":"f9aa6749-46cf-4b2a-8114-cbd034219d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162436840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162436840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6825","nonce":1162436840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162436840}]},"public_addr":"172.21.15.119:6819/1162436840","cluster_addr":"172.21.15.119:6821/1162436840","heartbeat_back_addr":"172.21.15.119:6825/1162436840","heartbeat_front_addr":"172.21.15.119:6823/1162436840","state":["exists","up"]},{"osd":7,"uuid":"39216361-e1c4-4fef-86cb-a4707656b547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6817","nonce":3254653457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6819","nonce":3254653457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6823","nonce":3254653457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6821","nonce":3254653457}]},"public_addr":"172.21.15.153:6817/3254653457","cluster_addr":"172.21.15.153:6819/3254653457","heartbeat_back_addr":"172.21.15.153:6823/3254653457","heartbeat_front_addr":"172.21.15.153:6821/3254653457","state":["exists","up"]},{"osd":8,"uuid":"cceb6ebc-7c02-48ea-9d16-b7a016dd449b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6817","nonce":2600946380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6819","nonce":2600946380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6823","nonce":2600946380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6821","nonce":2600946380}]},"public_addr":"172.21.15.159:6817/2600946380","cluster_addr":"172.21.15.159:6819/2600946380","heartbeat_back_addr":"172.21.15.159:6823/2600946380","heartbeat_front_addr":"172.21.15.159:6821/2600946380","state":["exists","up"]},{"osd":9,"uuid":"f337a2e4-e065-4a9f-91d9-e3615de2d6f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6827","nonce":1940737285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6829","nonce":1940737285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6833","nonce":1940737285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6831","nonce":1940737285}]},"public_addr":"172.21.15.119:6827/1940737285","cluster_addr":"172.21.15.119:6829/1940737285","heartbeat_back_addr":"172.21.15.119:6833/1940737285","heartbeat_front_addr":"172.21.15.119:6831/1940737285","state":["exists","up"]},{"osd":10,"uuid":"3a3bc347-85f4-4878-9797-d9da96137e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6825","nonce":663834864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6827","nonce":663834864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6831","nonce":663834864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6829","nonce":663834864}]},"public_addr":"172.21.15.153:6825/663834864","cluster_addr":"172.21.15.153:6827/663834864","heartbeat_back_addr":"172.21.15.153:6831/663834864","heartbeat_front_addr":"172.21.15.153:6829/663834864","state":["exists","up"]},{"osd":11,"uuid":"a0fc5909-279d-4172-977c-2dc2da0a7e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6825","nonce":1262216348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6827","nonce":1262216348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6831","nonce":1262216348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6829","nonce":1262216348}]},"public_addr":"172.21.15.159:6825/1262216348","cluster_addr":"172.21.15.159:6827/1262216348","heartbeat_back_addr":"172.21.15.159:6831/1262216348","heartbeat_front_addr":"172.21.15.159:6829/1262216348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:37.488109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:59.543060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:21.381585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:47.112729+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:09.732150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:30.893729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:56.678710+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:19.004256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:40.388818+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:06.391734+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:28.394053+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:49.634348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1628639392":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6801/1355617847":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6800/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/362307836":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/128964558":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/2278813554":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/1411933614":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/1685698277":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6800/1355617847":"2023-12-08T10:11:00.585565+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T10:17:47.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.701+0000 7fbdb55ff700 1 -- 172.21.15.119:0/1997046409 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fbd980a1b10 msgr2=0x7fbd980a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:47.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.701+0000 7fbdb55ff700 1 --2- 172.21.15.119:0/1997046409 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fbd980a1b10 0x7fbd980a3fd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbda4004800 tx=0x7fbda401a040 comp rx=0 tx=0).stop 2023-12-07T10:17:47.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.701+0000 7fbdb55ff700 1 -- 172.21.15.119:0/1997046409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdb0101560 msgr2=0x7fbdb0199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:47.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.701+0000 7fbdb55ff700 1 --2- 172.21.15.119:0/1997046409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdb0101560 0x7fbdb0199720 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbda000e9c0 tx=0x7fbda000ecd0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.701+0000 7fbdb55ff700 1 -- 172.21.15.119:0/1997046409 shutdown_connections 2023-12-07T10:17:47.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.701+0000 7fbdb55ff700 1 --2- 172.21.15.119:0/1997046409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdb01030f0 0x7fbdb01a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.701+0000 7fbdb55ff700 1 --2- 172.21.15.119:0/1997046409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdb0101560 0x7fbdb0199720 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.701+0000 7fbdb55ff700 1 --2- 172.21.15.119:0/1997046409 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fbd980a1b10 0x7fbd980a3fd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.701+0000 7fbdb55ff700 1 --2- 172.21.15.119:0/1997046409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdb0102750 0x7fbdb0199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.701+0000 7fbdb55ff700 1 -- 172.21.15.119:0/1997046409 >> 172.21.15.119:0/1997046409 conn(0x7fbdb00fcab0 msgr2=0x7fbdb0073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:47.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.701+0000 7fbdb55ff700 1 -- 172.21.15.119:0/1997046409 shutdown_connections 2023-12-07T10:17:47.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.701+0000 7fbdb55ff700 1 -- 172.21.15.119:0/1997046409 wait complete. 2023-12-07T10:17:47.723 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-07T10:17:47.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.920+0000 7f8b25bff700 1 -- 172.21.15.119:0/1753034722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b20101560 msgr2=0x7f8b20101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:47.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.920+0000 7f8b25bff700 1 --2- 172.21.15.119:0/1753034722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b20101560 0x7f8b20101980 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8b100099c0 tx=0x7f8b10009cd0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.920+0000 7f8b25bff700 1 -- 172.21.15.119:0/1753034722 shutdown_connections 2023-12-07T10:17:47.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.920+0000 7f8b25bff700 1 --2- 172.21.15.119:0/1753034722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b201030f0 0x7f8b201099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.920+0000 7f8b25bff700 1 --2- 172.21.15.119:0/1753034722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b20101560 0x7f8b20101980 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.920+0000 7f8b25bff700 1 --2- 172.21.15.119:0/1753034722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b20102750 0x7f8b20102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.920+0000 7f8b25bff700 1 -- 172.21.15.119:0/1753034722 >> 172.21.15.119:0/1753034722 conn(0x7f8b200fcab0 msgr2=0x7f8b200fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:47.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.920+0000 7f8b25bff700 1 -- 172.21.15.119:0/1753034722 shutdown_connections 2023-12-07T10:17:47.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.920+0000 7f8b25bff700 1 -- 172.21.15.119:0/1753034722 wait complete. 2023-12-07T10:17:47.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.921+0000 7f8b25bff700 1 Processor -- start 2023-12-07T10:17:47.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.921+0000 7f8b25bff700 1 -- start start 2023-12-07T10:17:47.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.921+0000 7f8b25bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b20101560 0x7f8b20075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:47.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.921+0000 7f8b25bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b20102750 0x7f8b20073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:47.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.921+0000 7f8b25bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b201030f0 0x7f8b20073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:47.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.922+0000 7f8b25bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b200746e0 con 0x7f8b20102750 2023-12-07T10:17:47.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.922+0000 7f8b25bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b20074850 con 0x7f8b201030f0 2023-12-07T10:17:47.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.922+0000 7f8b25bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b201a1570 con 0x7f8b20101560 2023-12-07T10:17:47.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.922+0000 7f8b1effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b20102750 0x7f8b20073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:47.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.922+0000 7f8b1effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b20102750 0x7f8b20073a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42148/0 (socket says 172.21.15.119:42148) 2023-12-07T10:17:47.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.922+0000 7f8b1effd700 1 -- 172.21.15.119:0/3717179615 learned_addr learned my addr 172.21.15.119:0/3717179615 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:47.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.922+0000 7f8b1f7fe700 1 --2- 172.21.15.119:0/3717179615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b20101560 0x7f8b20075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:47.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.922+0000 7f8b1ffff700 1 --2- 172.21.15.119:0/3717179615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b201030f0 0x7f8b20073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:47.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.922+0000 7f8b1effd700 1 -- 172.21.15.119:0/3717179615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b20101560 msgr2=0x7f8b20075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:47.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.922+0000 7f8b1effd700 1 --2- 172.21.15.119:0/3717179615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b20101560 0x7f8b20075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.922+0000 7f8b1effd700 1 -- 172.21.15.119:0/3717179615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b201030f0 msgr2=0x7f8b20073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:47.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.922+0000 7f8b1effd700 1 --2- 172.21.15.119:0/3717179615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b201030f0 0x7f8b20073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:47.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.922+0000 7f8b1effd700 1 -- 172.21.15.119:0/3717179615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b100095f0 con 0x7f8b20102750 2023-12-07T10:17:47.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.922+0000 7f8b1ffff700 1 --2- 172.21.15.119:0/3717179615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b201030f0 0x7f8b20073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:17:47.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.922+0000 7f8b1effd700 1 --2- 172.21.15.119:0/3717179615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b20102750 0x7f8b20073a00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400e9a0 tx=0x7f8b1400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:47.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.923+0000 7f8b1cff9700 1 -- 172.21.15.119:0/3717179615 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14009790 con 0x7f8b20102750 2023-12-07T10:17:47.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.923+0000 7f8b1cff9700 1 -- 172.21.15.119:0/3717179615 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b1400cb70 con 0x7f8b20102750 2023-12-07T10:17:47.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.923+0000 7f8b1cff9700 1 -- 172.21.15.119:0/3717179615 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14010ea0 con 0x7f8b20102750 2023-12-07T10:17:47.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.923+0000 7f8b25bff700 1 -- 172.21.15.119:0/3717179615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b201a1800 con 0x7f8b20102750 2023-12-07T10:17:47.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.923+0000 7f8b25bff700 1 -- 172.21.15.119:0/3717179615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b201a1cc0 con 0x7f8b20102750 2023-12-07T10:17:47.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.924+0000 7f8b25bff700 1 -- 172.21.15.119:0/3717179615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b20108a30 con 0x7f8b20102750 2023-12-07T10:17:47.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.926+0000 7f8b1cff9700 1 -- 172.21.15.119:0/3717179615 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f8b1400cce0 con 0x7f8b20102750 2023-12-07T10:17:47.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.927+0000 7f8b1cff9700 1 --2- 172.21.15.119:0/3717179615 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8b080a1b10 0x7f8b080a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:47.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.927+0000 7f8b1cff9700 1 -- 172.21.15.119:0/3717179615 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8b14014070 con 0x7f8b20102750 2023-12-07T10:17:47.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.927+0000 7f8b1f7fe700 1 --2- 172.21.15.119:0/3717179615 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8b080a1b10 0x7f8b080a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:47.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.928+0000 7f8b1f7fe700 1 --2- 172.21.15.119:0/3717179615 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8b080a1b10 0x7f8b080a3fd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8b10009990 tx=0x7f8b10000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:47.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:47.930+0000 7f8b1cff9700 1 -- 172.21.15.119:0/3717179615 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8b140522f0 con 0x7f8b20102750 2023-12-07T10:17:47.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:47 smithi119 ceph-mon[97908]: pgmap v281: 129 pgs: 3 creating+peering, 29 unknown, 97 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2023-12-07T10:17:47.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:47 smithi119 ceph-mon[97908]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-07T10:17:47.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:47 smithi119 ceph-mon[97908]: osdmap e89: 12 total, 12 up, 12 in 2023-12-07T10:17:47.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:47 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/879077932' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-12-07T10:17:47.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:47 smithi119 ceph-mon[97908]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-07T10:17:47.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:47 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1190987535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:47.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:47 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1997046409' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:17:48.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.136+0000 7f8b25bff700 1 -- 172.21.15.119:0/3717179615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b20068350 con 0x7f8b20102750 2023-12-07T10:17:48.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.138+0000 7f8b1cff9700 1 -- 172.21.15.119:0/3717179615 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24030 (secure 0 0 0) 0x7f8b14096bf0 con 0x7f8b20102750 2023-12-07T10:17:48.140 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:17:48.140 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":89,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","created":"2023-12-07T10:09:59.945863+0000","modified":"2023-12-07T10:17:46.277123+0000","last_up_change":"2023-12-07T10:16:50.929427+0000","last_in_change":"2023-12-07T10:16:33.558158+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T10:13:25.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-07T10:17:39.347813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-07T10:17:40.714501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-07T10:17:44.257760+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"56b5818e-e625-40f1-8838-3ddcf2d34ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6803","nonce":2141950133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6805","nonce":2141950133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6809","nonce":2141950133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6807","nonce":2141950133}]},"public_addr":"172.21.15.119:6803/2141950133","cluster_addr":"172.21.15.119:6805/2141950133","heartbeat_back_addr":"172.21.15.119:6809/2141950133","heartbeat_front_addr":"172.21.15.119:6807/2141950133","state":["exists","up"]},{"osd":1,"uuid":"36489a6a-5278-4802-9525-9871521722e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6801","nonce":3390875370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6803","nonce":3390875370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6807","nonce":3390875370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6805","nonce":3390875370}]},"public_addr":"172.21.15.153:6801/3390875370","cluster_addr":"172.21.15.153:6803/3390875370","heartbeat_back_addr":"172.21.15.153:6807/3390875370","heartbeat_front_addr":"172.21.15.153:6805/3390875370","state":["exists","up"]},{"osd":2,"uuid":"a937ad26-5e9b-4c8a-b65a-626a8feda6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6801","nonce":708101637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6803","nonce":708101637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6807","nonce":708101637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6805","nonce":708101637}]},"public_addr":"172.21.15.159:6801/708101637","cluster_addr":"172.21.15.159:6803/708101637","heartbeat_back_addr":"172.21.15.159:6807/708101637","heartbeat_front_addr":"172.21.15.159:6805/708101637","state":["exists","up"]},{"osd":3,"uuid":"26ef8300-aa07-48f6-b31d-8b0f2217ffcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6811","nonce":3228047257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6813","nonce":3228047257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6817","nonce":3228047257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6815","nonce":3228047257}]},"public_addr":"172.21.15.119:6811/3228047257","cluster_addr":"172.21.15.119:6813/3228047257","heartbeat_back_addr":"172.21.15.119:6817/3228047257","heartbeat_front_addr":"172.21.15.119:6815/3228047257","state":["exists","up"]},{"osd":4,"uuid":"000ee4b9-9896-4901-8cc3-c85fdfc8cbaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6809","nonce":4021998299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6811","nonce":4021998299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6815","nonce":4021998299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6813","nonce":4021998299}]},"public_addr":"172.21.15.153:6809/4021998299","cluster_addr":"172.21.15.153:6811/4021998299","heartbeat_back_addr":"172.21.15.153:6815/4021998299","heartbeat_front_addr":"172.21.15.153:6813/4021998299","state":["exists","up"]},{"osd":5,"uuid":"8b2fc167-262d-474f-89f3-47bd29424421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6809","nonce":1832540768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6811","nonce":1832540768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6815","nonce":1832540768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6813","nonce":1832540768}]},"public_addr":"172.21.15.159:6809/1832540768","cluster_addr":"172.21.15.159:6811/1832540768","heartbeat_back_addr":"172.21.15.159:6815/1832540768","heartbeat_front_addr":"172.21.15.159:6813/1832540768","state":["exists","up"]},{"osd":6,"uuid":"f9aa6749-46cf-4b2a-8114-cbd034219d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162436840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162436840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6825","nonce":1162436840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162436840}]},"public_addr":"172.21.15.119:6819/1162436840","cluster_addr":"172.21.15.119:6821/1162436840","heartbeat_back_addr":"172.21.15.119:6825/1162436840","heartbeat_front_addr":"172.21.15.119:6823/1162436840","state":["exists","up"]},{"osd":7,"uuid":"39216361-e1c4-4fef-86cb-a4707656b547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6817","nonce":3254653457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6819","nonce":3254653457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6823","nonce":3254653457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6821","nonce":3254653457}]},"public_addr":"172.21.15.153:6817/3254653457","cluster_addr":"172.21.15.153:6819/3254653457","heartbeat_back_addr":"172.21.15.153:6823/3254653457","heartbeat_front_addr":"172.21.15.153:6821/3254653457","state":["exists","up"]},{"osd":8,"uuid":"cceb6ebc-7c02-48ea-9d16-b7a016dd449b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6817","nonce":2600946380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6819","nonce":2600946380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6823","nonce":2600946380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6821","nonce":2600946380}]},"public_addr":"172.21.15.159:6817/2600946380","cluster_addr":"172.21.15.159:6819/2600946380","heartbeat_back_addr":"172.21.15.159:6823/2600946380","heartbeat_front_addr":"172.21.15.159:6821/2600946380","state":["exists","up"]},{"osd":9,"uuid":"f337a2e4-e065-4a9f-91d9-e3615de2d6f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6827","nonce":1940737285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6829","nonce":1940737285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6833","nonce":1940737285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6831","nonce":1940737285}]},"public_addr":"172.21.15.119:6827/1940737285","cluster_addr":"172.21.15.119:6829/1940737285","heartbeat_back_addr":"172.21.15.119:6833/1940737285","heartbeat_front_addr":"172.21.15.119:6831/1940737285","state":["exists","up"]},{"osd":10,"uuid":"3a3bc347-85f4-4878-9797-d9da96137e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6825","nonce":663834864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6827","nonce":663834864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6831","nonce":663834864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6829","nonce":663834864}]},"public_addr":"172.21.15.153:6825/663834864","cluster_addr":"172.21.15.153:6827/663834864","heartbeat_back_addr":"172.21.15.153:6831/663834864","heartbeat_front_addr":"172.21.15.153:6829/663834864","state":["exists","up"]},{"osd":11,"uuid":"a0fc5909-279d-4172-977c-2dc2da0a7e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6825","nonce":1262216348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6827","nonce":1262216348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6831","nonce":1262216348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6829","nonce":1262216348}]},"public_addr":"172.21.15.159:6825/1262216348","cluster_addr":"172.21.15.159:6827/1262216348","heartbeat_back_addr":"172.21.15.159:6831/1262216348","heartbeat_front_addr":"172.21.15.159:6829/1262216348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:37.488109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:59.543060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:21.381585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:47.112729+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:09.732150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:30.893729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:56.678710+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:19.004256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:40.388818+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:06.391734+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:28.394053+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:49.634348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1628639392":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6801/1355617847":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6800/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/362307836":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/128964558":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/2278813554":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/1411933614":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/1685698277":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6800/1355617847":"2023-12-08T10:11:00.585565+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T10:17:48.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.140+0000 7f8b25bff700 1 -- 172.21.15.119:0/3717179615 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8b080a1b10 msgr2=0x7f8b080a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:48.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.140+0000 7f8b25bff700 1 --2- 172.21.15.119:0/3717179615 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8b080a1b10 0x7f8b080a3fd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8b10009990 tx=0x7f8b10000bc0 comp rx=0 tx=0).stop 2023-12-07T10:17:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.140+0000 7f8b25bff700 1 -- 172.21.15.119:0/3717179615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b20102750 msgr2=0x7f8b20073a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.140+0000 7f8b25bff700 1 --2- 172.21.15.119:0/3717179615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b20102750 0x7f8b20073a00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400e9a0 tx=0x7f8b1400ecb0 comp rx=0 tx=0).stop 2023-12-07T10:17:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.140+0000 7f8b25bff700 1 -- 172.21.15.119:0/3717179615 shutdown_connections 2023-12-07T10:17:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.140+0000 7f8b25bff700 1 --2- 172.21.15.119:0/3717179615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b201030f0 0x7f8b20073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.140+0000 7f8b25bff700 1 --2- 172.21.15.119:0/3717179615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b20101560 0x7f8b20075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.140+0000 7f8b25bff700 1 --2- 172.21.15.119:0/3717179615 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8b080a1b10 0x7f8b080a3fd0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.140+0000 7f8b25bff700 1 --2- 172.21.15.119:0/3717179615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b20102750 0x7f8b20073a00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.140+0000 7f8b25bff700 1 -- 172.21.15.119:0/3717179615 >> 172.21.15.119:0/3717179615 conn(0x7f8b200fcab0 msgr2=0x7f8b20105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.141+0000 7f8b25bff700 1 -- 172.21.15.119:0/3717179615 shutdown_connections 2023-12-07T10:17:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.141+0000 7f8b25bff700 1 -- 172.21.15.119:0/3717179615 wait complete. 2023-12-07T10:17:48.176 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:17:48.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:47 smithi159 ceph-mon[134726]: pgmap v281: 129 pgs: 3 creating+peering, 29 unknown, 97 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2023-12-07T10:17:48.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:47 smithi159 ceph-mon[134726]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-07T10:17:48.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:47 smithi159 ceph-mon[134726]: osdmap e89: 12 total, 12 up, 12 in 2023-12-07T10:17:48.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:47 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/879077932' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-12-07T10:17:48.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:47 smithi159 ceph-mon[134726]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-07T10:17:48.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:47 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1190987535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:48.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:47 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1997046409' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:17:48.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:47 smithi153 ceph-mon[133625]: pgmap v281: 129 pgs: 3 creating+peering, 29 unknown, 97 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2023-12-07T10:17:48.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:47 smithi153 ceph-mon[133625]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-07T10:17:48.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:47 smithi153 ceph-mon[133625]: osdmap e89: 12 total, 12 up, 12 in 2023-12-07T10:17:48.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:47 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/879077932' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-12-07T10:17:48.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:47 smithi153 ceph-mon[133625]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-07T10:17:48.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:47 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1190987535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:48.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:47 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1997046409' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:17:48.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.372+0000 7f2bd49ff700 1 -- 172.21.15.119:0/473511375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bd0101560 msgr2=0x7f2bd0101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:48.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.372+0000 7f2bd49ff700 1 --2- 172.21.15.119:0/473511375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bd0101560 0x7f2bd0101980 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00099c0 tx=0x7f2bc0009cd0 comp rx=0 tx=0).stop 2023-12-07T10:17:48.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.372+0000 7f2bd49ff700 1 -- 172.21.15.119:0/473511375 shutdown_connections 2023-12-07T10:17:48.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.372+0000 7f2bd49ff700 1 --2- 172.21.15.119:0/473511375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bd01030f0 0x7f2bd01099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:48.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.372+0000 7f2bd49ff700 1 --2- 172.21.15.119:0/473511375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bd0101560 0x7f2bd0101980 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:48.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.372+0000 7f2bd49ff700 1 --2- 172.21.15.119:0/473511375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bd0102750 0x7f2bd0102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:48.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.372+0000 7f2bd49ff700 1 -- 172.21.15.119:0/473511375 >> 172.21.15.119:0/473511375 conn(0x7f2bd00fcab0 msgr2=0x7f2bd00fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:48.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.372+0000 7f2bd49ff700 1 -- 172.21.15.119:0/473511375 shutdown_connections 2023-12-07T10:17:48.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.372+0000 7f2bd49ff700 1 -- 172.21.15.119:0/473511375 wait complete. 2023-12-07T10:17:48.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.373+0000 7f2bd49ff700 1 Processor -- start 2023-12-07T10:17:48.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.373+0000 7f2bd49ff700 1 -- start start 2023-12-07T10:17:48.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.374+0000 7f2bd49ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bd0101560 0x7f2bd0197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:48.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.374+0000 7f2bd49ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bd0102750 0x7f2bd0197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:48.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.374+0000 7f2bd49ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bd01030f0 0x7f2bd019ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:48.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.374+0000 7f2bd49ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd019f060 con 0x7f2bd01030f0 2023-12-07T10:17:48.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.374+0000 7f2bd49ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd019f1a0 con 0x7f2bd0102750 2023-12-07T10:17:48.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.374+0000 7f2bd49ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd019f330 con 0x7f2bd0101560 2023-12-07T10:17:48.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.374+0000 7f2bced9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bd01030f0 0x7f2bd019ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:48.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.374+0000 7f2bced9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bd01030f0 0x7f2bd019ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42166/0 (socket says 172.21.15.119:42166) 2023-12-07T10:17:48.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.374+0000 7f2bced9d700 1 -- 172.21.15.119:0/578618655 learned_addr learned my addr 172.21.15.119:0/578618655 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:48.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.374+0000 7f2bce59c700 1 --2- 172.21.15.119:0/578618655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bd0101560 0x7f2bd0197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:48.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.374+0000 7f2bc7fff700 1 --2- 172.21.15.119:0/578618655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bd0102750 0x7f2bd0197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:48.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.375+0000 7f2bced9d700 1 -- 172.21.15.119:0/578618655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bd0101560 msgr2=0x7f2bd0197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:48.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.375+0000 7f2bced9d700 1 --2- 172.21.15.119:0/578618655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bd0101560 0x7f2bd0197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:48.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.375+0000 7f2bced9d700 1 -- 172.21.15.119:0/578618655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bd0102750 msgr2=0x7f2bd0197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:48.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.375+0000 7f2bced9d700 1 --2- 172.21.15.119:0/578618655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bd0102750 0x7f2bd0197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:48.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.375+0000 7f2bced9d700 1 -- 172.21.15.119:0/578618655 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc00095f0 con 0x7f2bd01030f0 2023-12-07T10:17:48.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.375+0000 7f2bced9d700 1 --2- 172.21.15.119:0/578618655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bd01030f0 0x7f2bd019ea90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8007ee0 tx=0x7f2bc800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:48.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.375+0000 7f2bc77fe700 1 -- 172.21.15.119:0/578618655 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc800dd20 con 0x7f2bd01030f0 2023-12-07T10:17:48.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.375+0000 7f2bc77fe700 1 -- 172.21.15.119:0/578618655 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bc800f040 con 0x7f2bd01030f0 2023-12-07T10:17:48.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.375+0000 7f2bc77fe700 1 -- 172.21.15.119:0/578618655 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc80048a0 con 0x7f2bd01030f0 2023-12-07T10:17:48.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.375+0000 7f2bd49ff700 1 -- 172.21.15.119:0/578618655 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd019f610 con 0x7f2bd01030f0 2023-12-07T10:17:48.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.375+0000 7f2bd49ff700 1 -- 172.21.15.119:0/578618655 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd019fad0 con 0x7f2bd01030f0 2023-12-07T10:17:48.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.376+0000 7f2bd49ff700 1 -- 172.21.15.119:0/578618655 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd0108a50 con 0x7f2bd01030f0 2023-12-07T10:17:48.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.378+0000 7f2bc77fe700 1 -- 172.21.15.119:0/578618655 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f2bc8003a00 con 0x7f2bd01030f0 2023-12-07T10:17:48.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.379+0000 7f2bc77fe700 1 --2- 172.21.15.119:0/578618655 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2bbc0a1b10 0x7f2bbc0a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:48.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.380+0000 7f2bc77fe700 1 -- 172.21.15.119:0/578618655 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2bc80cd860 con 0x7f2bd01030f0 2023-12-07T10:17:48.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.380+0000 7f2bce59c700 1 --2- 172.21.15.119:0/578618655 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2bbc0a1b10 0x7f2bbc0a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:48.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.380+0000 7f2bce59c700 1 --2- 172.21.15.119:0/578618655 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2bbc0a1b10 0x7f2bbc0a3fd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0003730 tx=0x7f2bc0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:48.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.382+0000 7f2bc77fe700 1 -- 172.21.15.119:0/578618655 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2bc8096a30 con 0x7f2bd01030f0 2023-12-07T10:17:48.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.624+0000 7f2bd49ff700 1 -- 172.21.15.119:0/578618655 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2bd0068350 con 0x7f2bd01030f0 2023-12-07T10:17:48.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.625+0000 7f2bc77fe700 1 -- 172.21.15.119:0/578618655 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11273 (secure 0 0 0) 0x7f2bc809a050 con 0x7f2bd01030f0 2023-12-07T10:17:48.627 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:17:48.627 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14535,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6841/1791998028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1791998028},{"type":"v1","addr":"172.21.15.119:6841","nonce":1791998028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2925503246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2925503246},{"type":"v1","addr":"172.21.15.153:6837","nonce":2925503246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/3984962900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3984962900},{"type":"v1","addr":"172.21.15.159:6839","nonce":3984962900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24542,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6839/1920762602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1920762602},{"type":"v1","addr":"172.21.15.119:6839","nonce":1920762602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-07T10:17:41.278311+0000","modified":"2023-12-07T10:17:46.277320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24566},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/566825184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":566825184},{"type":"v1","addr":"172.21.15.153:6839","nonce":566825184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-12-07T10:17:48.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.627+0000 7f2bd49ff700 1 -- 172.21.15.119:0/578618655 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2bbc0a1b10 msgr2=0x7f2bbc0a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:48.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.627+0000 7f2bd49ff700 1 --2- 172.21.15.119:0/578618655 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2bbc0a1b10 0x7f2bbc0a3fd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0003730 tx=0x7f2bc0000bc0 comp rx=0 tx=0).stop 2023-12-07T10:17:48.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.628+0000 7f2bd49ff700 1 -- 172.21.15.119:0/578618655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bd01030f0 msgr2=0x7f2bd019ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:48.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.628+0000 7f2bd49ff700 1 --2- 172.21.15.119:0/578618655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bd01030f0 0x7f2bd019ea90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8007ee0 tx=0x7f2bc800d3b0 comp rx=0 tx=0).stop 2023-12-07T10:17:48.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.628+0000 7f2bd49ff700 1 -- 172.21.15.119:0/578618655 shutdown_connections 2023-12-07T10:17:48.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.628+0000 7f2bd49ff700 1 --2- 172.21.15.119:0/578618655 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2bbc0a1b10 0x7f2bbc0a3fd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:48.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.628+0000 7f2bd49ff700 1 --2- 172.21.15.119:0/578618655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bd01030f0 0x7f2bd019ea90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:48.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.628+0000 7f2bd49ff700 1 --2- 172.21.15.119:0/578618655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bd0101560 0x7f2bd0197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:48.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.628+0000 7f2bd49ff700 1 --2- 172.21.15.119:0/578618655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bd0102750 0x7f2bd0197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:48.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.628+0000 7f2bd49ff700 1 -- 172.21.15.119:0/578618655 >> 172.21.15.119:0/578618655 conn(0x7f2bd00fcab0 msgr2=0x7f2bd0105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:48.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.628+0000 7f2bd49ff700 1 -- 172.21.15.119:0/578618655 shutdown_connections 2023-12-07T10:17:48.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.628+0000 7f2bd49ff700 1 -- 172.21.15.119:0/578618655 wait complete. 2023-12-07T10:17:48.631 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 17 2023-12-07T10:17:48.649 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-07T10:17:48.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.852+0000 7f935dfff700 1 -- 172.21.15.119:0/2773575454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9358101560 msgr2=0x7f9358101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:48.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.852+0000 7f935dfff700 1 --2- 172.21.15.119:0/2773575454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9358101560 0x7f9358101980 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f93480099c0 tx=0x7f9348009cd0 comp rx=0 tx=0).stop 2023-12-07T10:17:48.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.852+0000 7f935dfff700 1 -- 172.21.15.119:0/2773575454 shutdown_connections 2023-12-07T10:17:48.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.852+0000 7f935dfff700 1 --2- 172.21.15.119:0/2773575454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93581030f0 0x7f93581099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:48.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.852+0000 7f935dfff700 1 --2- 172.21.15.119:0/2773575454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9358101560 0x7f9358101980 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:48.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.852+0000 7f935dfff700 1 --2- 172.21.15.119:0/2773575454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9358102750 0x7f9358102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:48.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.852+0000 7f935dfff700 1 -- 172.21.15.119:0/2773575454 >> 172.21.15.119:0/2773575454 conn(0x7f93580fcab0 msgr2=0x7f93580fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:48.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.852+0000 7f935dfff700 1 -- 172.21.15.119:0/2773575454 shutdown_connections 2023-12-07T10:17:48.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.852+0000 7f935dfff700 1 -- 172.21.15.119:0/2773575454 wait complete. 2023-12-07T10:17:48.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.853+0000 7f935dfff700 1 Processor -- start 2023-12-07T10:17:48.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.853+0000 7f935dfff700 1 -- start start 2023-12-07T10:17:48.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.853+0000 7f935dfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9358101560 0x7f9358197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:48.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.853+0000 7f935dfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9358102750 0x7f9358197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:48.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.853+0000 7f935dfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93581030f0 0x7f935819ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:48.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.853+0000 7f935dfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935819f040 con 0x7f9358101560 2023-12-07T10:17:48.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.853+0000 7f935dfff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935819f180 con 0x7f9358102750 2023-12-07T10:17:48.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.853+0000 7f935dfff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935819f310 con 0x7f93581030f0 2023-12-07T10:17:48.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.854+0000 7f93577fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9358101560 0x7f9358197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:48.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.854+0000 7f934ffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9358102750 0x7f9358197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:48.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.854+0000 7f93577fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9358101560 0x7f9358197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42180/0 (socket says 172.21.15.119:42180) 2023-12-07T10:17:48.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.854+0000 7f93577fe700 1 -- 172.21.15.119:0/3188243765 learned_addr learned my addr 172.21.15.119:0/3188243765 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:48.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.854+0000 7f9357fff700 1 --2- 172.21.15.119:0/3188243765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93581030f0 0x7f935819ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:48.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.854+0000 7f934ffff700 1 -- 172.21.15.119:0/3188243765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93581030f0 msgr2=0x7f935819ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:48.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.854+0000 7f934ffff700 1 --2- 172.21.15.119:0/3188243765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93581030f0 0x7f935819ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:48.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.854+0000 7f934ffff700 1 -- 172.21.15.119:0/3188243765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9358101560 msgr2=0x7f9358197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:48.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.854+0000 7f934ffff700 1 --2- 172.21.15.119:0/3188243765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9358101560 0x7f9358197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:48.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.854+0000 7f934ffff700 1 -- 172.21.15.119:0/3188243765 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93480095f0 con 0x7f9358102750 2023-12-07T10:17:48.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.854+0000 7f93577fe700 1 --2- 172.21.15.119:0/3188243765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9358101560 0x7f9358197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:17:48.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.854+0000 7f934ffff700 1 --2- 172.21.15.119:0/3188243765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9358102750 0x7f9358197a50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f934000e9f0 tx=0x7f934000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:48.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.854+0000 7f93557fa700 1 -- 172.21.15.119:0/3188243765 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934000c9c0 con 0x7f9358102750 2023-12-07T10:17:48.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.854+0000 7f93557fa700 1 -- 172.21.15.119:0/3188243765 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9340010430 con 0x7f9358102750 2023-12-07T10:17:48.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.854+0000 7f935dfff700 1 -- 172.21.15.119:0/3188243765 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935819f5f0 con 0x7f9358102750 2023-12-07T10:17:48.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.855+0000 7f93557fa700 1 -- 172.21.15.119:0/3188243765 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340018680 con 0x7f9358102750 2023-12-07T10:17:48.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.855+0000 7f935dfff700 1 -- 172.21.15.119:0/3188243765 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935819fb00 con 0x7f9358102750 2023-12-07T10:17:48.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.855+0000 7f935dfff700 1 -- 172.21.15.119:0/3188243765 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9358108a30 con 0x7f9358102750 2023-12-07T10:17:48.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.857+0000 7f93557fa700 1 -- 172.21.15.119:0/3188243765 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f9340010860 con 0x7f9358102750 2023-12-07T10:17:48.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.859+0000 7f93557fa700 1 --2- 172.21.15.119:0/3188243765 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f93440a1b10 0x7f93440a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:48.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.859+0000 7f93557fa700 1 -- 172.21.15.119:0/3188243765 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9340014070 con 0x7f9358102750 2023-12-07T10:17:48.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.859+0000 7f93577fe700 1 --2- 172.21.15.119:0/3188243765 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f93440a1b10 0x7f93440a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:48.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.859+0000 7f93577fe700 1 --2- 172.21.15.119:0/3188243765 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f93440a1b10 0x7f93440a3fd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9348000c00 tx=0x7f9348009530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:48.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:48.861+0000 7f93557fa700 1 -- 172.21.15.119:0/3188243765 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9340096ab0 con 0x7f9358102750 2023-12-07T10:17:48.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:48 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3717179615' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:17:48.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:48 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/578618655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:49.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.065+0000 7f935dfff700 1 -- 172.21.15.119:0/3188243765 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9358068350 con 0x7f9358102750 2023-12-07T10:17:49.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.067+0000 7f93557fa700 1 -- 172.21.15.119:0/3188243765 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24030 (secure 0 0 0) 0x7f934009a0d0 con 0x7f9358102750 2023-12-07T10:17:49.069 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:17:49.069 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":89,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","created":"2023-12-07T10:09:59.945863+0000","modified":"2023-12-07T10:17:46.277123+0000","last_up_change":"2023-12-07T10:16:50.929427+0000","last_in_change":"2023-12-07T10:16:33.558158+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T10:13:25.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-07T10:17:39.347813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-07T10:17:40.714501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-07T10:17:44.257760+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"56b5818e-e625-40f1-8838-3ddcf2d34ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6803","nonce":2141950133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6805","nonce":2141950133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6809","nonce":2141950133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6807","nonce":2141950133}]},"public_addr":"172.21.15.119:6803/2141950133","cluster_addr":"172.21.15.119:6805/2141950133","heartbeat_back_addr":"172.21.15.119:6809/2141950133","heartbeat_front_addr":"172.21.15.119:6807/2141950133","state":["exists","up"]},{"osd":1,"uuid":"36489a6a-5278-4802-9525-9871521722e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6801","nonce":3390875370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6803","nonce":3390875370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6807","nonce":3390875370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6805","nonce":3390875370}]},"public_addr":"172.21.15.153:6801/3390875370","cluster_addr":"172.21.15.153:6803/3390875370","heartbeat_back_addr":"172.21.15.153:6807/3390875370","heartbeat_front_addr":"172.21.15.153:6805/3390875370","state":["exists","up"]},{"osd":2,"uuid":"a937ad26-5e9b-4c8a-b65a-626a8feda6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6801","nonce":708101637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6803","nonce":708101637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6807","nonce":708101637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6805","nonce":708101637}]},"public_addr":"172.21.15.159:6801/708101637","cluster_addr":"172.21.15.159:6803/708101637","heartbeat_back_addr":"172.21.15.159:6807/708101637","heartbeat_front_addr":"172.21.15.159:6805/708101637","state":["exists","up"]},{"osd":3,"uuid":"26ef8300-aa07-48f6-b31d-8b0f2217ffcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6811","nonce":3228047257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6813","nonce":3228047257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6817","nonce":3228047257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6815","nonce":3228047257}]},"public_addr":"172.21.15.119:6811/3228047257","cluster_addr":"172.21.15.119:6813/3228047257","heartbeat_back_addr":"172.21.15.119:6817/3228047257","heartbeat_front_addr":"172.21.15.119:6815/3228047257","state":["exists","up"]},{"osd":4,"uuid":"000ee4b9-9896-4901-8cc3-c85fdfc8cbaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6809","nonce":4021998299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6811","nonce":4021998299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6815","nonce":4021998299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6813","nonce":4021998299}]},"public_addr":"172.21.15.153:6809/4021998299","cluster_addr":"172.21.15.153:6811/4021998299","heartbeat_back_addr":"172.21.15.153:6815/4021998299","heartbeat_front_addr":"172.21.15.153:6813/4021998299","state":["exists","up"]},{"osd":5,"uuid":"8b2fc167-262d-474f-89f3-47bd29424421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6809","nonce":1832540768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6811","nonce":1832540768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6815","nonce":1832540768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6813","nonce":1832540768}]},"public_addr":"172.21.15.159:6809/1832540768","cluster_addr":"172.21.15.159:6811/1832540768","heartbeat_back_addr":"172.21.15.159:6815/1832540768","heartbeat_front_addr":"172.21.15.159:6813/1832540768","state":["exists","up"]},{"osd":6,"uuid":"f9aa6749-46cf-4b2a-8114-cbd034219d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162436840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162436840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6825","nonce":1162436840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162436840}]},"public_addr":"172.21.15.119:6819/1162436840","cluster_addr":"172.21.15.119:6821/1162436840","heartbeat_back_addr":"172.21.15.119:6825/1162436840","heartbeat_front_addr":"172.21.15.119:6823/1162436840","state":["exists","up"]},{"osd":7,"uuid":"39216361-e1c4-4fef-86cb-a4707656b547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6817","nonce":3254653457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6819","nonce":3254653457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6823","nonce":3254653457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6821","nonce":3254653457}]},"public_addr":"172.21.15.153:6817/3254653457","cluster_addr":"172.21.15.153:6819/3254653457","heartbeat_back_addr":"172.21.15.153:6823/3254653457","heartbeat_front_addr":"172.21.15.153:6821/3254653457","state":["exists","up"]},{"osd":8,"uuid":"cceb6ebc-7c02-48ea-9d16-b7a016dd449b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6817","nonce":2600946380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6819","nonce":2600946380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6823","nonce":2600946380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6821","nonce":2600946380}]},"public_addr":"172.21.15.159:6817/2600946380","cluster_addr":"172.21.15.159:6819/2600946380","heartbeat_back_addr":"172.21.15.159:6823/2600946380","heartbeat_front_addr":"172.21.15.159:6821/2600946380","state":["exists","up"]},{"osd":9,"uuid":"f337a2e4-e065-4a9f-91d9-e3615de2d6f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6827","nonce":1940737285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6829","nonce":1940737285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6833","nonce":1940737285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6831","nonce":1940737285}]},"public_addr":"172.21.15.119:6827/1940737285","cluster_addr":"172.21.15.119:6829/1940737285","heartbeat_back_addr":"172.21.15.119:6833/1940737285","heartbeat_front_addr":"172.21.15.119:6831/1940737285","state":["exists","up"]},{"osd":10,"uuid":"3a3bc347-85f4-4878-9797-d9da96137e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6825","nonce":663834864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6827","nonce":663834864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6831","nonce":663834864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6829","nonce":663834864}]},"public_addr":"172.21.15.153:6825/663834864","cluster_addr":"172.21.15.153:6827/663834864","heartbeat_back_addr":"172.21.15.153:6831/663834864","heartbeat_front_addr":"172.21.15.153:6829/663834864","state":["exists","up"]},{"osd":11,"uuid":"a0fc5909-279d-4172-977c-2dc2da0a7e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6825","nonce":1262216348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6827","nonce":1262216348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6831","nonce":1262216348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6829","nonce":1262216348}]},"public_addr":"172.21.15.159:6825/1262216348","cluster_addr":"172.21.15.159:6827/1262216348","heartbeat_back_addr":"172.21.15.159:6831/1262216348","heartbeat_front_addr":"172.21.15.159:6829/1262216348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:37.488109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:59.543060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:21.381585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:47.112729+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:09.732150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:30.893729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:56.678710+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:19.004256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:40.388818+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:06.391734+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:28.394053+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:49.634348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1628639392":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6801/1355617847":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6800/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/362307836":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/128964558":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/2278813554":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/1411933614":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/1685698277":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6800/1355617847":"2023-12-08T10:11:00.585565+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T10:17:49.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.069+0000 7f935dfff700 1 -- 172.21.15.119:0/3188243765 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f93440a1b10 msgr2=0x7f93440a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:49.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.069+0000 7f935dfff700 1 --2- 172.21.15.119:0/3188243765 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f93440a1b10 0x7f93440a3fd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9348000c00 tx=0x7f9348009530 comp rx=0 tx=0).stop 2023-12-07T10:17:49.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.069+0000 7f935dfff700 1 -- 172.21.15.119:0/3188243765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9358102750 msgr2=0x7f9358197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:49.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.069+0000 7f935dfff700 1 --2- 172.21.15.119:0/3188243765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9358102750 0x7f9358197a50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f934000e9f0 tx=0x7f934000ed00 comp rx=0 tx=0).stop 2023-12-07T10:17:49.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.069+0000 7f935dfff700 1 -- 172.21.15.119:0/3188243765 shutdown_connections 2023-12-07T10:17:49.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.069+0000 7f935dfff700 1 --2- 172.21.15.119:0/3188243765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93581030f0 0x7f935819ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.069+0000 7f935dfff700 1 --2- 172.21.15.119:0/3188243765 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f93440a1b10 0x7f93440a3fd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.069+0000 7f935dfff700 1 --2- 172.21.15.119:0/3188243765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9358101560 0x7f9358197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.069+0000 7f935dfff700 1 --2- 172.21.15.119:0/3188243765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9358102750 0x7f9358197a50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.069+0000 7f935dfff700 1 -- 172.21.15.119:0/3188243765 >> 172.21.15.119:0/3188243765 conn(0x7f93580fcab0 msgr2=0x7f9358105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:49.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.070+0000 7f935dfff700 1 -- 172.21.15.119:0/3188243765 shutdown_connections 2023-12-07T10:17:49.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.070+0000 7f935dfff700 1 -- 172.21.15.119:0/3188243765 wait complete. 2023-12-07T10:17:49.091 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-07T10:17:41.278311+0000', 'modified': '2023-12-07T10:17:46.277320+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24566}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24566': {'gid': 24566, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.153:6839/566825184', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6838', 'nonce': 566825184}, {'type': 'v1', 'addr': '172.21.15.153:6839', 'nonce': 566825184}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-12-07T10:17:49.091 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-07T10:17:49.092 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-07T10:17:49.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:48 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3717179615' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:17:49.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:48 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/578618655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:49.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:48 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3717179615' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:17:49.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:48 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/578618655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:17:49.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.286+0000 7f316fdff700 1 -- 172.21.15.119:0/243681846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3168101540 msgr2=0x7f3168101960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:49.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.286+0000 7f316fdff700 1 --2- 172.21.15.119:0/243681846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3168101540 0x7f3168101960 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f31640099c0 tx=0x7f3164009cd0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.286+0000 7f316fdff700 1 -- 172.21.15.119:0/243681846 shutdown_connections 2023-12-07T10:17:49.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.286+0000 7f316fdff700 1 --2- 172.21.15.119:0/243681846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31681030d0 0x7f3168109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.286+0000 7f316fdff700 1 --2- 172.21.15.119:0/243681846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3168101540 0x7f3168101960 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.286+0000 7f316fdff700 1 --2- 172.21.15.119:0/243681846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3168102730 0x7f3168102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.286+0000 7f316fdff700 1 -- 172.21.15.119:0/243681846 >> 172.21.15.119:0/243681846 conn(0x7f31680fcab0 msgr2=0x7f31680feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:49.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.286+0000 7f316fdff700 1 -- 172.21.15.119:0/243681846 shutdown_connections 2023-12-07T10:17:49.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.286+0000 7f316fdff700 1 -- 172.21.15.119:0/243681846 wait complete. 2023-12-07T10:17:49.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.287+0000 7f316fdff700 1 Processor -- start 2023-12-07T10:17:49.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.287+0000 7f316fdff700 1 -- start start 2023-12-07T10:17:49.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.287+0000 7f316fdff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3168101540 0x7f3168199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:49.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.287+0000 7f316fdff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3168102730 0x7f3168199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:49.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.287+0000 7f316fdff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31681030d0 0x7f31681a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:49.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.287+0000 7f316fdff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31681a1240 con 0x7f3168102730 2023-12-07T10:17:49.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.287+0000 7f316fdff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31681a1380 con 0x7f3168101540 2023-12-07T10:17:49.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.287+0000 7f316fdff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31681a1510 con 0x7f31681030d0 2023-12-07T10:17:49.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.288+0000 7f316d39a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3168102730 0x7f3168199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:49.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.288+0000 7f316d39a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3168102730 0x7f3168199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42196/0 (socket says 172.21.15.119:42196) 2023-12-07T10:17:49.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.288+0000 7f316d39a700 1 -- 172.21.15.119:0/389630754 learned_addr learned my addr 172.21.15.119:0/389630754 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:49.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.288+0000 7f316db9b700 1 --2- 172.21.15.119:0/389630754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3168101540 0x7f3168199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:49.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.288+0000 7f316e39c700 1 --2- 172.21.15.119:0/389630754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31681030d0 0x7f31681a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:49.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.288+0000 7f316d39a700 1 -- 172.21.15.119:0/389630754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31681030d0 msgr2=0x7f31681a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:49.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.288+0000 7f316d39a700 1 --2- 172.21.15.119:0/389630754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31681030d0 0x7f31681a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.288+0000 7f316d39a700 1 -- 172.21.15.119:0/389630754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3168101540 msgr2=0x7f3168199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:49.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.288+0000 7f316d39a700 1 --2- 172.21.15.119:0/389630754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3168101540 0x7f3168199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.288+0000 7f316d39a700 1 -- 172.21.15.119:0/389630754 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31640095f0 con 0x7f3168102730 2023-12-07T10:17:49.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.288+0000 7f316d39a700 1 --2- 172.21.15.119:0/389630754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3168102730 0x7f3168199c50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f315800e9a0 tx=0x7f315800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:49.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.288+0000 7f315effd700 1 -- 172.21.15.119:0/389630754 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3158009790 con 0x7f3168102730 2023-12-07T10:17:49.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.288+0000 7f316fdff700 1 -- 172.21.15.119:0/389630754 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31681a17f0 con 0x7f3168102730 2023-12-07T10:17:49.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.288+0000 7f315effd700 1 -- 172.21.15.119:0/389630754 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f315800cbd0 con 0x7f3168102730 2023-12-07T10:17:49.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.289+0000 7f315effd700 1 -- 172.21.15.119:0/389630754 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3158018680 con 0x7f3168102730 2023-12-07T10:17:49.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.289+0000 7f316fdff700 1 -- 172.21.15.119:0/389630754 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31681a1d00 con 0x7f3168102730 2023-12-07T10:17:49.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.291+0000 7f315effd700 1 -- 172.21.15.119:0/389630754 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f315800cd40 con 0x7f3168102730 2023-12-07T10:17:49.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.293+0000 7f31837f0b80 1 Processor -- start 2023-12-07T10:17:49.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.293+0000 7f31837f0b80 1 -- start start 2023-12-07T10:17:49.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.293+0000 7f315effd700 1 --2- 172.21.15.119:0/389630754 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f31540a1b10 0x7f31540a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:49.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.293+0000 7f31837f0b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1a130 0x558ddaa1a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:49.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.293+0000 7f31837f0b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ddaa1aa90 0x558ddaa1b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:49.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.293+0000 7f31837f0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ddaa1b980 0x558ddaa1be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:49.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.293+0000 7f31837f0b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dda98e770 con 0x558ddaa1a130 2023-12-07T10:17:49.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.293+0000 7f31837f0b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dda975750 con 0x558ddaa1aa90 2023-12-07T10:17:49.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.293+0000 7f31837f0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dda973070 con 0x558ddaa1b980 2023-12-07T10:17:49.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.293+0000 7f315effd700 1 -- 172.21.15.119:0/389630754 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3158014070 con 0x7f3168102730 2023-12-07T10:17:49.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.293+0000 7f316db9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1a130 0x558ddaa1a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:49.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.293+0000 7f316d39a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ddaa1aa90 0x558ddaa1b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:49.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.293+0000 7f316e39c700 1 --2- 172.21.15.119:0/389630754 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f31540a1b10 0x7f31540a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:49.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.293+0000 7f316db9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1a130 0x558ddaa1a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42204/0 (socket says 172.21.15.119:42204) 2023-12-07T10:17:49.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.293+0000 7f316d39a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ddaa1aa90 0x558ddaa1b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:40232/0 (socket says 172.21.15.119:40232) 2023-12-07T10:17:49.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.293+0000 7f316db9b700 1 -- 172.21.15.119:0/2031864427 learned_addr learned my addr 172.21.15.119:0/2031864427 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:49.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.294+0000 7f316e39c700 1 --2- 172.21.15.119:0/2031864427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ddaa1b980 0x558ddaa1be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:49.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.294+0000 7f316d39a700 1 -- 172.21.15.119:0/2031864427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ddaa1b980 msgr2=0x558ddaa1be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:17:49.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.294+0000 7f316d39a700 1 --2- 172.21.15.119:0/2031864427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ddaa1b980 0x558ddaa1be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.294+0000 7f316d39a700 1 -- 172.21.15.119:0/2031864427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1a130 msgr2=0x558ddaa1a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:17:49.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.294+0000 7f316d39a700 1 --2- 172.21.15.119:0/2031864427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1a130 0x558ddaa1a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.294+0000 7f316d39a700 1 -- 172.21.15.119:0/2031864427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dda7eb5e0 con 0x558ddaa1aa90 2023-12-07T10:17:49.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.294+0000 7f316db9b700 1 --2- 172.21.15.119:0/2031864427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1a130 0x558ddaa1a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:17:49.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.294+0000 7f316e39c700 1 --2- 172.21.15.119:0/389630754 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f31540a1b10 0x7f31540a3fd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x558ddaa1b160 tx=0x7f31600090b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:49.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.294+0000 7f316d39a700 1 --2- 172.21.15.119:0/2031864427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ddaa1aa90 0x558ddaa1b3a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f315809a6a0 tx=0x7f315809b0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fceada8749ddbac server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:49.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.294+0000 7f3153fff700 1 -- 172.21.15.119:0/2031864427 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31580b8ea0 con 0x558ddaa1aa90 2023-12-07T10:17:49.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.294+0000 7f3153fff700 1 -- 172.21.15.119:0/2031864427 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f315809eaf0 con 0x558ddaa1aa90 2023-12-07T10:17:49.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.294+0000 7f3153fff700 1 -- 172.21.15.119:0/2031864427 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31580ba0e0 con 0x558ddaa1aa90 2023-12-07T10:17:49.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.295+0000 7f31837f0b80 1 -- 172.21.15.119:0/2031864427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ddaa1aa90 msgr2=0x558ddaa1b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:49.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.295+0000 7f31837f0b80 1 --2- 172.21.15.119:0/2031864427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ddaa1aa90 0x558ddaa1b3a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f315809a6a0 tx=0x7f315809b0a0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.295+0000 7f31837f0b80 1 -- 172.21.15.119:0/2031864427 shutdown_connections 2023-12-07T10:17:49.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.295+0000 7f31837f0b80 1 --2- 172.21.15.119:0/2031864427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ddaa1b980 0x558ddaa1be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.295+0000 7f31837f0b80 1 --2- 172.21.15.119:0/2031864427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1a130 0x558ddaa1a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.295+0000 7f31837f0b80 1 --2- 172.21.15.119:0/2031864427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ddaa1aa90 0x558ddaa1b3a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.295+0000 7f31837f0b80 1 -- 172.21.15.119:0/2031864427 >> 172.21.15.119:0/2031864427 conn(0x558dda949310 msgr2=0x558ddaa16b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:49.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.295+0000 7f31837f0b80 1 -- 172.21.15.119:0/2031864427 shutdown_connections 2023-12-07T10:17:49.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.295+0000 7f31837f0b80 1 -- 172.21.15.119:0/2031864427 wait complete. 2023-12-07T10:17:49.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.296+0000 7f31837f0b80 1 -- start start 2023-12-07T10:17:49.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.296+0000 7f31837f0b80 1 Processor -- start 2023-12-07T10:17:49.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.297+0000 7f31837f0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ddaa1a130 0x558ddaa23fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:49.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.297+0000 7f31837f0b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1aa90 0x558ddaa24510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:49.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.297+0000 7f31837f0b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ddaa1b980 0x558ddaa2b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:49.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.297+0000 7f31837f0b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dda96e960 con 0x558ddaa1aa90 2023-12-07T10:17:49.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.297+0000 7f31837f0b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dda928510 con 0x558ddaa1b980 2023-12-07T10:17:49.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.297+0000 7f31837f0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dda9281b0 con 0x558ddaa1a130 2023-12-07T10:17:49.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.297+0000 7f31837f0b80 10 client.0 _ll_register_callbacks cb 0x558dda90f8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:17:49.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.297+0000 7f316db9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ddaa1a130 0x558ddaa23fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:49.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.297+0000 7f316e39c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ddaa1b980 0x558ddaa2b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:49.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.297+0000 7f316db9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ddaa1a130 0x558ddaa23fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:58300/0 (socket says 172.21.15.119:58300) 2023-12-07T10:17:49.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.297+0000 7f316db9b700 1 -- 172.21.15.119:0/3620403469 learned_addr learned my addr 172.21.15.119:0/3620403469 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:49.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.297+0000 7f316d39a700 1 --2- 172.21.15.119:0/3620403469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1aa90 0x558ddaa24510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:49.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.297+0000 7f316db9b700 1 -- 172.21.15.119:0/3620403469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ddaa1b980 msgr2=0x558ddaa2b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:17:49.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.297+0000 7f316db9b700 1 --2- 172.21.15.119:0/3620403469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ddaa1b980 0x558ddaa2b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.297+0000 7f316db9b700 1 -- 172.21.15.119:0/3620403469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1aa90 msgr2=0x558ddaa24510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:17:49.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.297+0000 7f316db9b700 1 --2- 172.21.15.119:0/3620403469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1aa90 0x558ddaa24510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.297+0000 7f316db9b700 1 -- 172.21.15.119:0/3620403469 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316810a2d0 con 0x558ddaa1a130 2023-12-07T10:17:49.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.297+0000 7f316d39a700 1 --2- 172.21.15.119:0/3620403469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1aa90 0x558ddaa24510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:17:49.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.298+0000 7f316db9b700 1 --2- 172.21.15.119:0/3620403469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ddaa1a130 0x558ddaa23fd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f31640099c0 tx=0x7f3164009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc4dd55902afa931 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:49.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.298+0000 7f31517fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:17:49.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.298+0000 7f31517fa700 1 -- 172.21.15.119:0/3620403469 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3164027070 con 0x558ddaa1a130 2023-12-07T10:17:49.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.298+0000 7f31837f0b80 1 -- 172.21.15.119:0/3620403469 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ddaa0dab0 con 0x558ddaa1a130 2023-12-07T10:17:49.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.298+0000 7f31517fa700 1 -- 172.21.15.119:0/3620403469 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f31640052e0 con 0x558ddaa1a130 2023-12-07T10:17:49.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.298+0000 7f31517fa700 1 -- 172.21.15.119:0/3620403469 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f316401d040 con 0x558ddaa1a130 2023-12-07T10:17:49.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.298+0000 7f31837f0b80 10 client.24685 fetch_fsmap learned FSMap version 17 2023-12-07T10:17:49.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.298+0000 7f31837f0b80 1 -- 172.21.15.119:0/3620403469 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x7f31681a1d00 con 0x558ddaa1a130 2023-12-07T10:17:49.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.298+0000 7f31517fa700 1 -- 172.21.15.119:0/3620403469 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558ddaa0dab0 con 0x558ddaa1a130 2023-12-07T10:17:49.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.299+0000 7f31517fa700 1 -- 172.21.15.119:0/3620403469 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f316402a020 con 0x558ddaa1a130 2023-12-07T10:17:49.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.299+0000 7f31837f0b80 10 client.24685 fetch_fsmap finished waiting for FSMap version 17 2023-12-07T10:17:49.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.299+0000 7f31837f0b80 10 client.24685 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-07T10:17:49.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.299+0000 7f31837f0b80 20 client.24685 populate_metadata read hostname 'smithi119' 2023-12-07T10:17:49.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.299+0000 7f31837f0b80 1 --2- 172.21.15.119:0/3620403469 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558ddaa2fc10 0x558ddaa2e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:49.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.299+0000 7f31837f0b80 4 client.24685 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:17:49.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.299+0000 7f31837f0b80 1 -- 172.21.15.119:0/3620403469 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558dda9e9750 con 0x558ddaa2fc10 2023-12-07T10:17:49.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.299+0000 7f316d39a700 1 --2- 172.21.15.119:0/3620403469 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558ddaa2fc10 0x558ddaa2e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:49.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.300+0000 7f31517fa700 10 client.24685 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:17:49.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.300+0000 7f316d39a700 1 --2- 172.21.15.119:0/3620403469 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558ddaa2fc10 0x558ddaa2e6f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de61eb2ce3a9769b server_cookie=6efc6d2340d22f34 in_seq=0 out_seq=0 2023-12-07T10:17:49.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.300+0000 7f31517fa700 1 -- 172.21.15.119:0/3620403469 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558dda9e9750 con 0x558ddaa2fc10 2023-12-07T10:17:49.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.300+0000 7f31517fa700 10 client.24685 handle_command_reply: tid=0 2023-12-07T10:17:49.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.301+0000 7f31837f0b80 1 client.24685 shutdown 2023-12-07T10:17:49.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.301+0000 7f31837f0b80 1 -- 172.21.15.119:0/3620403469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ddaa1a130 msgr2=0x558ddaa23fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:49.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.301+0000 7f31837f0b80 1 --2- 172.21.15.119:0/3620403469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ddaa1a130 0x558ddaa23fd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f31640099c0 tx=0x7f3164009510 comp rx=0 tx=0).stop 2023-12-07T10:17:49.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.301+0000 7f31837f0b80 1 -- 172.21.15.119:0/3620403469 shutdown_connections 2023-12-07T10:17:49.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.301+0000 7f31837f0b80 1 --2- 172.21.15.119:0/3620403469 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558ddaa2fc10 0x558ddaa2e6f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.301+0000 7f31837f0b80 1 --2- 172.21.15.119:0/3620403469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ddaa1b980 0x558ddaa2b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.301+0000 7f31837f0b80 1 --2- 172.21.15.119:0/3620403469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ddaa1a130 0x558ddaa23fd0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.301+0000 7f31837f0b80 1 --2- 172.21.15.119:0/3620403469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1aa90 0x558ddaa24510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.301+0000 7f31517fa700 0 client.24685 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:17:49.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.301+0000 7f31837f0b80 1 -- 172.21.15.119:0/3620403469 >> 172.21.15.119:0/3620403469 conn(0x558dda949310 msgr2=0x558ddaa15990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:49.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.301+0000 7f31837f0b80 1 -- 172.21.15.119:0/3620403469 shutdown_connections 2023-12-07T10:17:49.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.301+0000 7f31837f0b80 1 -- 172.21.15.119:0/3620403469 wait complete. 2023-12-07T10:17:49.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.301+0000 7f31837f0b80 20 client.24685 trim_cache size 0 max 16384 2023-12-07T10:17:49.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.319+0000 7f31837f0b80 1 Processor -- start 2023-12-07T10:17:49.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.319+0000 7f31837f0b80 1 -- start start 2023-12-07T10:17:49.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.320+0000 7f31837f0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ddaa1a130 0x558ddaa1e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:49.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.320+0000 7f31837f0b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ddaa1aa90 0x558ddaa1ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:49.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.320+0000 7f31837f0b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1b980 0x558ddaa2b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:49.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.320+0000 7f31837f0b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dda7ed8d0 con 0x558ddaa1b980 2023-12-07T10:17:49.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.320+0000 7f31837f0b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dda7f9de0 con 0x558ddaa1aa90 2023-12-07T10:17:49.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.320+0000 7f31837f0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dda90a360 con 0x558ddaa1a130 2023-12-07T10:17:49.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.320+0000 7f316e39c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1b980 0x558ddaa2b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:49.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.320+0000 7f316e39c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1b980 0x558ddaa2b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42220/0 (socket says 172.21.15.119:42220) 2023-12-07T10:17:49.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.320+0000 7f316e39c700 1 -- 172.21.15.119:0/817168504 learned_addr learned my addr 172.21.15.119:0/817168504 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:49.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.320+0000 7f316db9b700 1 --2- 172.21.15.119:0/817168504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ddaa1a130 0x558ddaa1e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:49.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.320+0000 7f316d39a700 1 --2- 172.21.15.119:0/817168504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ddaa1aa90 0x558ddaa1ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:49.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.320+0000 7f316e39c700 1 -- 172.21.15.119:0/817168504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ddaa1a130 msgr2=0x558ddaa1e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:17:49.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.320+0000 7f316e39c700 1 --2- 172.21.15.119:0/817168504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ddaa1a130 0x558ddaa1e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.320+0000 7f316e39c700 1 -- 172.21.15.119:0/817168504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ddaa1aa90 msgr2=0x558ddaa1ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:17:49.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.320+0000 7f316e39c700 1 --2- 172.21.15.119:0/817168504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ddaa1aa90 0x558ddaa1ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.320+0000 7f316e39c700 1 -- 172.21.15.119:0/817168504 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dda797d90 con 0x558ddaa1b980 2023-12-07T10:17:49.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.321+0000 7f316e39c700 1 --2- 172.21.15.119:0/817168504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1b980 0x558ddaa2b510 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3160016930 tx=0x7f3160007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc42cb6a9995fb1 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:49.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.321+0000 7f315cbff700 1 -- 172.21.15.119:0/817168504 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316002a070 con 0x558ddaa1b980 2023-12-07T10:17:49.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.321+0000 7f315cbff700 1 -- 172.21.15.119:0/817168504 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3160020040 con 0x558ddaa1b980 2023-12-07T10:17:49.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.321+0000 7f315cbff700 1 -- 172.21.15.119:0/817168504 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316001f050 con 0x558ddaa1b980 2023-12-07T10:17:49.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.321+0000 7f31837f0b80 1 -- 172.21.15.119:0/817168504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1b980 msgr2=0x558ddaa2b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:49.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.322+0000 7f31837f0b80 1 --2- 172.21.15.119:0/817168504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1b980 0x558ddaa2b510 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3160016930 tx=0x7f3160007f90 comp rx=0 tx=0).stop 2023-12-07T10:17:49.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.322+0000 7f31837f0b80 1 -- 172.21.15.119:0/817168504 shutdown_connections 2023-12-07T10:17:49.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.322+0000 7f31837f0b80 1 --2- 172.21.15.119:0/817168504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1b980 0x558ddaa2b510 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.322+0000 7f31837f0b80 1 --2- 172.21.15.119:0/817168504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ddaa1a130 0x558ddaa1e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.322+0000 7f31837f0b80 1 --2- 172.21.15.119:0/817168504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ddaa1aa90 0x558ddaa1ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.322+0000 7f31837f0b80 1 -- 172.21.15.119:0/817168504 >> 172.21.15.119:0/817168504 conn(0x558dda949310 msgr2=0x558ddaa231b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:49.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.322+0000 7f31837f0b80 1 -- 172.21.15.119:0/817168504 shutdown_connections 2023-12-07T10:17:49.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.322+0000 7f31837f0b80 1 -- 172.21.15.119:0/817168504 wait complete. 2023-12-07T10:17:49.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.322+0000 7f31837f0b80 1 -- start start 2023-12-07T10:17:49.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.322+0000 7f31837f0b80 1 Processor -- start 2023-12-07T10:17:49.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.323+0000 7f31837f0b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1a130 0x558ddaa2b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:49.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.323+0000 7f31837f0b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ddaa1aa90 0x558ddaa27d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:49.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.323+0000 7f31837f0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ddaa1b980 0x558ddaa28280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:49.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.323+0000 7f31837f0b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ddaa287c0 con 0x558ddaa1a130 2023-12-07T10:17:49.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.323+0000 7f31837f0b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ddaa28930 con 0x558ddaa1aa90 2023-12-07T10:17:49.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.323+0000 7f31837f0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ddaa85cd0 con 0x558ddaa1b980 2023-12-07T10:17:49.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.323+0000 7f31837f0b80 10 client.0 _ll_register_callbacks cb 0x558dda90f8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:17:49.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.323+0000 7f316db9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1a130 0x558ddaa2b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:49.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.323+0000 7f316db9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1a130 0x558ddaa2b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42228/0 (socket says 172.21.15.119:42228) 2023-12-07T10:17:49.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.323+0000 7f316db9b700 1 -- 172.21.15.119:0/470130063 learned_addr learned my addr 172.21.15.119:0/470130063 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:49.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.323+0000 7f316e39c700 1 --2- 172.21.15.119:0/470130063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ddaa1b980 0x558ddaa28280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:49.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.323+0000 7f316d39a700 1 --2- 172.21.15.119:0/470130063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ddaa1aa90 0x558ddaa27d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:49.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.323+0000 7f316db9b700 1 -- 172.21.15.119:0/470130063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ddaa1b980 msgr2=0x558ddaa28280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:17:49.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.323+0000 7f316db9b700 1 --2- 172.21.15.119:0/470130063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ddaa1b980 0x558ddaa28280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.323+0000 7f316db9b700 1 -- 172.21.15.119:0/470130063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ddaa1aa90 msgr2=0x558ddaa27d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:17:49.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.323+0000 7f316db9b700 1 --2- 172.21.15.119:0/470130063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ddaa1aa90 0x558ddaa27d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.323+0000 7f316db9b700 1 -- 172.21.15.119:0/470130063 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3160016670 con 0x558ddaa1a130 2023-12-07T10:17:49.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.323+0000 7f316db9b700 1 --2- 172.21.15.119:0/470130063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1a130 0x558ddaa2b220 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3164006010 tx=0x7f31640054f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646bf20e4412d14c server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:49.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.323+0000 7f31517fa700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:17:49.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.324+0000 7f31517fa700 1 -- 172.21.15.119:0/470130063 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316401d070 con 0x558ddaa1a130 2023-12-07T10:17:49.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.324+0000 7f31837f0b80 1 -- 172.21.15.119:0/470130063 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f315809eaf0 con 0x558ddaa1a130 2023-12-07T10:17:49.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.324+0000 7f31517fa700 1 -- 172.21.15.119:0/470130063 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3164005660 con 0x558ddaa1a130 2023-12-07T10:17:49.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.324+0000 7f31517fa700 1 -- 172.21.15.119:0/470130063 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3164027040 con 0x558ddaa1a130 2023-12-07T10:17:49.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.324+0000 7f31837f0b80 10 client.14658 fetch_fsmap learned FSMap version 17 2023-12-07T10:17:49.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.324+0000 7f31837f0b80 1 -- 172.21.15.119:0/470130063 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x558dda780c10 con 0x558ddaa1a130 2023-12-07T10:17:49.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.324+0000 7f31517fa700 1 -- 172.21.15.119:0/470130063 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f315809eaf0 con 0x558ddaa1a130 2023-12-07T10:17:49.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.324+0000 7f31517fa700 1 -- 172.21.15.119:0/470130063 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f316400b260 con 0x558ddaa1a130 2023-12-07T10:17:49.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.324+0000 7f31837f0b80 10 client.14658 fetch_fsmap finished waiting for FSMap version 17 2023-12-07T10:17:49.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.324+0000 7f31837f0b80 10 client.14658 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-07T10:17:49.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.324+0000 7f31837f0b80 20 client.14658 populate_metadata read hostname 'smithi119' 2023-12-07T10:17:49.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.324+0000 7f31837f0b80 1 --2- 172.21.15.119:0/470130063 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558ddaa8aa70 0x558ddaa86ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:49.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.324+0000 7f31837f0b80 4 client.14658 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-07T10:17:49.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.324+0000 7f31837f0b80 1 -- 172.21.15.119:0/470130063 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558dda934f80 con 0x558ddaa8aa70 2023-12-07T10:17:49.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.325+0000 7f316d39a700 1 --2- 172.21.15.119:0/470130063 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558ddaa8aa70 0x558ddaa86ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:49.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.325+0000 7f316d39a700 1 --2- 172.21.15.119:0/470130063 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558ddaa8aa70 0x558ddaa86ef0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9155061904b4eed8 server_cookie=56f8cb66f401e095 in_seq=0 out_seq=0 2023-12-07T10:17:49.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.325+0000 7f31517fa700 10 client.14658 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:17:49.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.325+0000 7f31517fa700 1 -- 172.21.15.119:0/470130063 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x558dda934f80 con 0x558ddaa8aa70 2023-12-07T10:17:49.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.325+0000 7f31517fa700 10 client.14658 handle_command_reply: tid=0 2023-12-07T10:17:49.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.325+0000 7f31837f0b80 1 client.14658 shutdown 2023-12-07T10:17:49.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.325+0000 7f31837f0b80 1 -- 172.21.15.119:0/470130063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1a130 msgr2=0x558ddaa2b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:49.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.325+0000 7f31837f0b80 1 --2- 172.21.15.119:0/470130063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1a130 0x558ddaa2b220 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3164006010 tx=0x7f31640054f0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.325+0000 7f31837f0b80 1 -- 172.21.15.119:0/470130063 shutdown_connections 2023-12-07T10:17:49.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.325+0000 7f31837f0b80 1 --2- 172.21.15.119:0/470130063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ddaa1b980 0x558ddaa28280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.325+0000 7f31837f0b80 1 --2- 172.21.15.119:0/470130063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558ddaa1a130 0x558ddaa2b220 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.325+0000 7f31837f0b80 1 --2- 172.21.15.119:0/470130063 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558ddaa8aa70 0x558ddaa86ef0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.325+0000 7f31837f0b80 1 --2- 172.21.15.119:0/470130063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558ddaa1aa90 0x558ddaa27d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.326+0000 7f31517fa700 0 client.14658 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:17:49.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.326+0000 7f31837f0b80 1 -- 172.21.15.119:0/470130063 >> 172.21.15.119:0/470130063 conn(0x558dda949310 msgr2=0x558ddaa231b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:49.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.326+0000 7f31837f0b80 1 -- 172.21.15.119:0/470130063 shutdown_connections 2023-12-07T10:17:49.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.326+0000 7f31837f0b80 1 -- 172.21.15.119:0/470130063 wait complete. 2023-12-07T10:17:49.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.326+0000 7f31837f0b80 20 client.14658 trim_cache size 0 max 16384 2023-12-07T10:17:49.335 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:17:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "a4c642c6-94e8-11ee-95a2-87774f69a715", 2023-12-07T10:17:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-12-07T10:17:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 24566, 2023-12-07T10:17:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-12-07T10:17:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-12-07T10:17:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-12-07T10:17:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 7.5746018770000001, 2023-12-07T10:17:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 17, 2023-12-07T10:17:49.335 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 89, 2023-12-07T10:17:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 89, 2023-12-07T10:17:49.336 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 15.787117975999999 2023-12-07T10:17:49.336 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:17:49.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.326+0000 7f316fdff700 1 -- 172.21.15.119:0/389630754 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f31540a1b10 msgr2=0x7f31540a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:49.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.326+0000 7f316fdff700 1 --2- 172.21.15.119:0/389630754 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f31540a1b10 0x7f31540a3fd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x558ddaa1b160 tx=0x7f31600090b0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.326+0000 7f316fdff700 1 -- 172.21.15.119:0/389630754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3168102730 msgr2=0x7f3168199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:49.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.326+0000 7f316fdff700 1 --2- 172.21.15.119:0/389630754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3168102730 0x7f3168199c50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f315800e9a0 tx=0x7f315800ed60 comp rx=0 tx=0).stop 2023-12-07T10:17:49.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.326+0000 7f316fdff700 1 -- 172.21.15.119:0/389630754 shutdown_connections 2023-12-07T10:17:49.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.326+0000 7f316fdff700 1 --2- 172.21.15.119:0/389630754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31681030d0 0x7f31681a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.326+0000 7f316fdff700 1 --2- 172.21.15.119:0/389630754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3168101540 0x7f3168199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.326+0000 7f316fdff700 1 --2- 172.21.15.119:0/389630754 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f31540a1b10 0x7f31540a3fd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.326+0000 7f316fdff700 1 --2- 172.21.15.119:0/389630754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3168102730 0x7f3168199c50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:49.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.326+0000 7f316fdff700 1 -- 172.21.15.119:0/389630754 >> 172.21.15.119:0/389630754 conn(0x7f31680fcab0 msgr2=0x7f3168073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:49.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.327+0000 7f316fdff700 1 -- 172.21.15.119:0/389630754 shutdown_connections 2023-12-07T10:17:49.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:49.327+0000 7f316fdff700 1 -- 172.21.15.119:0/389630754 wait complete. 2023-12-07T10:17:49.346 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-07T10:17:49.346 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-12-07T10:17:49.347 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph fs ls 2023-12-07T10:17:50.115 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:17:50.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:49 smithi119 ceph-mon[97908]: pgmap v283: 129 pgs: 3 creating+peering, 10 unknown, 116 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 2 op/s 2023-12-07T10:17:50.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:49 smithi119 ceph-mon[97908]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-12-07T10:17:50.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:49 smithi119 ceph-mon[97908]: Cluster is now healthy 2023-12-07T10:17:50.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:49 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3188243765' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:17:50.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:49 smithi159 ceph-mon[134726]: pgmap v283: 129 pgs: 3 creating+peering, 10 unknown, 116 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 2 op/s 2023-12-07T10:17:50.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:49 smithi159 ceph-mon[134726]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-12-07T10:17:50.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:49 smithi159 ceph-mon[134726]: Cluster is now healthy 2023-12-07T10:17:50.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:49 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3188243765' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:17:50.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:49 smithi153 ceph-mon[133625]: pgmap v283: 129 pgs: 3 creating+peering, 10 unknown, 116 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 2 op/s 2023-12-07T10:17:50.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:49 smithi153 ceph-mon[133625]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-12-07T10:17:50.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:49 smithi153 ceph-mon[133625]: Cluster is now healthy 2023-12-07T10:17:50.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:49 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3188243765' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:17:51.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.592+0000 7fc033afb700 1 -- 172.21.15.119:0/3540584043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc02c107730 msgr2=0x7fc02c107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:51.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.592+0000 7fc033afb700 1 --2- 172.21.15.119:0/3540584043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc02c107730 0x7fc02c107b10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc028009990 tx=0x7fc028009ca0 comp rx=0 tx=0).stop 2023-12-07T10:17:51.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.592+0000 7fc033afb700 1 -- 172.21.15.119:0/3540584043 shutdown_connections 2023-12-07T10:17:51.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.592+0000 7fc033afb700 1 --2- 172.21.15.119:0/3540584043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc02c102080 0x7fc02c10e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:51.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.592+0000 7fc033afb700 1 --2- 172.21.15.119:0/3540584043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc02c107730 0x7fc02c107b10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:51.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.592+0000 7fc033afb700 1 --2- 172.21.15.119:0/3540584043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc02c1016c0 0x7fc02c101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:51.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.592+0000 7fc033afb700 1 -- 172.21.15.119:0/3540584043 >> 172.21.15.119:0/3540584043 conn(0x7fc02c078800 msgr2=0x7fc02c078c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:51.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.592+0000 7fc033afb700 1 -- 172.21.15.119:0/3540584043 shutdown_connections 2023-12-07T10:17:51.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.592+0000 7fc033afb700 1 -- 172.21.15.119:0/3540584043 wait complete. 2023-12-07T10:17:51.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.593+0000 7fc033afb700 1 Processor -- start 2023-12-07T10:17:51.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.593+0000 7fc033afb700 1 -- start start 2023-12-07T10:17:51.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.594+0000 7fc033afb700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc02c1016c0 0x7fc02c19f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:51.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.594+0000 7fc033afb700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc02c102080 0x7fc02c19fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:51.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.594+0000 7fc033afb700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc02c107730 0x7fc02c199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:51.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.594+0000 7fc033afb700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c19a080 con 0x7fc02c102080 2023-12-07T10:17:51.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.594+0000 7fc033afb700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c19a1f0 con 0x7fc02c1016c0 2023-12-07T10:17:51.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.594+0000 7fc033afb700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c19a3b0 con 0x7fc02c107730 2023-12-07T10:17:51.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.594+0000 7fc031096700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc02c102080 0x7fc02c19fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:51.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.594+0000 7fc031096700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc02c102080 0x7fc02c19fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42246/0 (socket says 172.21.15.119:42246) 2023-12-07T10:17:51.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.594+0000 7fc031096700 1 -- 172.21.15.119:0/2797725402 learned_addr learned my addr 172.21.15.119:0/2797725402 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:51.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.594+0000 7fc031897700 1 --2- 172.21.15.119:0/2797725402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc02c1016c0 0x7fc02c19f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:51.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.594+0000 7fc032098700 1 --2- 172.21.15.119:0/2797725402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc02c107730 0x7fc02c199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:51.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.594+0000 7fc031096700 1 -- 172.21.15.119:0/2797725402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc02c107730 msgr2=0x7fc02c199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:51.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.594+0000 7fc031096700 1 --2- 172.21.15.119:0/2797725402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc02c107730 0x7fc02c199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:51.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.594+0000 7fc031096700 1 -- 172.21.15.119:0/2797725402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc02c1016c0 msgr2=0x7fc02c19f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:51.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.594+0000 7fc031096700 1 --2- 172.21.15.119:0/2797725402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc02c1016c0 0x7fc02c19f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:51.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.594+0000 7fc031096700 1 -- 172.21.15.119:0/2797725402 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0280095f0 con 0x7fc02c102080 2023-12-07T10:17:51.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.595+0000 7fc031096700 1 --2- 172.21.15.119:0/2797725402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc02c102080 0x7fc02c19fe20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc01c00e9a0 tx=0x7fc01c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:51.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.595+0000 7fc022ffd700 1 -- 172.21.15.119:0/2797725402 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c009790 con 0x7fc02c102080 2023-12-07T10:17:51.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.595+0000 7fc033afb700 1 -- 172.21.15.119:0/2797725402 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc02c19a690 con 0x7fc02c102080 2023-12-07T10:17:51.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.595+0000 7fc022ffd700 1 -- 172.21.15.119:0/2797725402 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc01c00cbd0 con 0x7fc02c102080 2023-12-07T10:17:51.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.595+0000 7fc022ffd700 1 -- 172.21.15.119:0/2797725402 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c018680 con 0x7fc02c102080 2023-12-07T10:17:51.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.595+0000 7fc033afb700 1 -- 172.21.15.119:0/2797725402 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc02c1a68d0 con 0x7fc02c102080 2023-12-07T10:17:51.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.596+0000 7fc033afb700 1 -- 172.21.15.119:0/2797725402 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc02c104980 con 0x7fc02c102080 2023-12-07T10:17:51.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.598+0000 7fc022ffd700 1 -- 172.21.15.119:0/2797725402 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fc01c00cd40 con 0x7fc02c102080 2023-12-07T10:17:51.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.599+0000 7fc022ffd700 1 --2- 172.21.15.119:0/2797725402 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc0180a1b10 0x7fc0180a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:51.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.599+0000 7fc022ffd700 1 -- 172.21.15.119:0/2797725402 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc01c014070 con 0x7fc02c102080 2023-12-07T10:17:51.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.600+0000 7fc031897700 1 --2- 172.21.15.119:0/2797725402 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc0180a1b10 0x7fc0180a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:51.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.600+0000 7fc031897700 1 --2- 172.21.15.119:0/2797725402 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc0180a1b10 0x7fc0180a3fd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc02800b2e0 tx=0x7fc028005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:51.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.604+0000 7fc022ffd700 1 -- 172.21.15.119:0/2797725402 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc01c0522f0 con 0x7fc02c102080 2023-12-07T10:17:51.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.847+0000 7fc033afb700 1 -- 172.21.15.119:0/2797725402 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc02c19aee0 con 0x7fc02c102080 2023-12-07T10:17:51.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.848+0000 7fc022ffd700 1 -- 172.21.15.119:0/2797725402 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7fc01c096c60 con 0x7fc02c102080 2023-12-07T10:17:51.850 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-12-07T10:17:51.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.850+0000 7fc033afb700 1 -- 172.21.15.119:0/2797725402 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc0180a1b10 msgr2=0x7fc0180a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:51.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.850+0000 7fc033afb700 1 --2- 172.21.15.119:0/2797725402 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc0180a1b10 0x7fc0180a3fd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc02800b2e0 tx=0x7fc028005960 comp rx=0 tx=0).stop 2023-12-07T10:17:51.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.850+0000 7fc033afb700 1 -- 172.21.15.119:0/2797725402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc02c102080 msgr2=0x7fc02c19fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:51.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.850+0000 7fc033afb700 1 --2- 172.21.15.119:0/2797725402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc02c102080 0x7fc02c19fe20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc01c00e9a0 tx=0x7fc01c00ecb0 comp rx=0 tx=0).stop 2023-12-07T10:17:51.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.850+0000 7fc033afb700 1 -- 172.21.15.119:0/2797725402 shutdown_connections 2023-12-07T10:17:51.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.850+0000 7fc033afb700 1 --2- 172.21.15.119:0/2797725402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc02c107730 0x7fc02c199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:51.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.850+0000 7fc033afb700 1 --2- 172.21.15.119:0/2797725402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc02c1016c0 0x7fc02c19f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:51.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.850+0000 7fc033afb700 1 --2- 172.21.15.119:0/2797725402 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc0180a1b10 0x7fc0180a3fd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:51.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.850+0000 7fc033afb700 1 --2- 172.21.15.119:0/2797725402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc02c102080 0x7fc02c19fe20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:51.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.850+0000 7fc033afb700 1 -- 172.21.15.119:0/2797725402 >> 172.21.15.119:0/2797725402 conn(0x7fc02c078800 msgr2=0x7fc02c10cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:51.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.850+0000 7fc033afb700 1 -- 172.21.15.119:0/2797725402 shutdown_connections 2023-12-07T10:17:51.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:51.851+0000 7fc033afb700 1 -- 172.21.15.119:0/2797725402 wait complete. 2023-12-07T10:17:52.220 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:51 smithi119 ceph-mon[97908]: pgmap v284: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2023-12-07T10:17:52.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:51 smithi159 ceph-mon[134726]: pgmap v284: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2023-12-07T10:17:52.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:51 smithi153 ceph-mon[133625]: pgmap v284: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2023-12-07T10:17:52.299 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-07T10:17:52.299 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-07T10:17:52.299 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-12-07T10:17:52.299 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-12-07T10:17:52.299 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-12-07T10:17:52.299 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-07T10:17:52.299 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-07T10:17:52.299 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-07T10:17:52.300 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-12-07T10:17:52.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:17:52.300 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-12-07T10:17:52.345 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-07T10:17:52.345 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-07T10:17:52.345 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-12-07T10:17:52.346 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-12-07T10:17:52.346 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-12-07T10:17:52.346 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-12-07T10:17:52.346 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-07T10:17:52.346 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-12-07T10:17:52.346 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-12-07T10:17:52.346 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-07T10:17:52.346 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-12-07T10:17:52.346 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-12-07T10:17:52.346 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-07T10:17:52.346 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-12-07T10:17:52.347 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-07T10:17:52.347 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-12-07T10:17:52.347 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-07T10:17:52.347 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 41739sec preferred_lft 41739sec 2023-12-07T10:17:52.347 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::9e03:31a5:7c84:6136/64 scope link noprefixroute 2023-12-07T10:17:52.347 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-12-07T10:17:52.348 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-07T10:17:52.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:17:52.348 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-12-07T10:17:52.348 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-12-07T10:17:52.348 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add name ceph-brx type bridge 2023-12-07T10:17:52.348 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip addr flush dev ceph-brx 2023-12-07T10:17:52.348 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set ceph-brx up 2023-12-07T10:17:52.348 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-07T10:17:52.349 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-12-07T10:17:52.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:17:52.803 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-07T10:17:52.876 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-12-07T10:17:52.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:17:52.879 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-12-07T10:17:52.946 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-12-07T10:17:52.946 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-07T10:17:52.947 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-07T10:17:52.947 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-07T10:17:52.947 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-07T10:17:52.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:17:52.948 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-12-07T10:17:52.948 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-12-07T10:17:52.948 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-07T10:17:52.948 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-07T10:17:52.948 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-07T10:17:52.948 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-12-07T10:17:53.049 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:52 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2797725402' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-07T10:17:53.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:52 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2797725402' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-07T10:17:53.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:52 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2797725402' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-07T10:17:53.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:17:53.673 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-12-07T10:17:53.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:17:53.717 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list-id 2023-12-07T10:17:53.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:17:53.801 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-12-07T10:17:53.801 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-12-07T10:17:53.802 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-07T10:17:53.802 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-12-07T10:17:53.802 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-12-07T10:17:53.942 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:53 smithi119 ceph-mon[97908]: pgmap v285: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 388 B/s wr, 0 op/s 2023-12-07T10:17:53.985 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-12-07T10:17:53.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:17:53.985 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-12-07T10:17:53.985 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-12-07T10:17:53.985 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-12-07T10:17:53.986 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-07T10:17:53.986 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-12-07T10:17:53.986 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-12-07T10:17:53.986 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-12-07T10:17:53.986 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-12-07T10:17:54.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:53 smithi159 ceph-mon[134726]: pgmap v285: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 388 B/s wr, 0 op/s 2023-12-07T10:17:54.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:17:54.264 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-12-07T10:17:54.264 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-12-07T10:17:54.264 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2023-12-07T10:17:54.265 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set dev brx.0 master ceph-brx 2023-12-07T10:17:54.265 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-12-07T10:17:54.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:53 smithi153 ceph-mon[133625]: pgmap v285: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 388 B/s wr, 0 op/s 2023-12-07T10:17:54.435 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-12-07T10:17:54.435 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-07T10:17:54.435 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-12-07T10:17:54.478 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-12-07T10:17:54.479 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-07T10:17:54.479 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-12-07T10:17:54.561 DEBUG:teuthology.orchestra.run.smithi119:> sudo modprobe fuse 2023-12-07T10:17:54.671 DEBUG:teuthology.orchestra.run.smithi119:> cat /proc/self/mounts | awk '{print $2}' 2023-12-07T10:17:54.716 INFO:teuthology.orchestra.run.smithi119.stdout:/sys 2023-12-07T10:17:54.716 INFO:teuthology.orchestra.run.smithi119.stdout:/proc 2023-12-07T10:17:54.716 INFO:teuthology.orchestra.run.smithi119.stdout:/dev 2023-12-07T10:17:54.716 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/security 2023-12-07T10:17:54.716 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/shm 2023-12-07T10:17:54.716 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/pts 2023-12-07T10:17:54.716 INFO:teuthology.orchestra.run.smithi119.stdout:/run 2023-12-07T10:17:54.716 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup 2023-12-07T10:17:54.716 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup/systemd 2023-12-07T10:17:54.716 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/pstore 2023-12-07T10:17:54.716 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-12-07T10:17:54.717 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup/cpuset 2023-12-07T10:17:54.717 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup/devices 2023-12-07T10:17:54.717 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup/pids 2023-12-07T10:17:54.717 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup/blkio 2023-12-07T10:17:54.717 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup/freezer 2023-12-07T10:17:54.717 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup/perf_event 2023-12-07T10:17:54.717 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-12-07T10:17:54.717 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup/memory 2023-12-07T10:17:54.717 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup/hugetlb 2023-12-07T10:17:54.717 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/tracing 2023-12-07T10:17:54.717 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/config 2023-12-07T10:17:54.717 INFO:teuthology.orchestra.run.smithi119.stdout:/ 2023-12-07T10:17:54.717 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/selinux 2023-12-07T10:17:54.717 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/mqueue 2023-12-07T10:17:54.717 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/debug 2023-12-07T10:17:54.718 INFO:teuthology.orchestra.run.smithi119.stdout:/proc/sys/fs/binfmt_misc 2023-12-07T10:17:54.718 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/hugepages 2023-12-07T10:17:54.718 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/fuse/connections 2023-12-07T10:17:54.718 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/nfs/rpc_pipefs 2023-12-07T10:17:54.718 INFO:teuthology.orchestra.run.smithi119.stdout:/run/user/1000 2023-12-07T10:17:54.718 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph 2023-12-07T10:17:54.718 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/containers/storage/overlay 2023-12-07T10:17:54.718 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/containers/storage/overlay/94bb8aa2c375f56012694782590080dec9ba5b314d15ac84afa4b081b2ebf79b/merged 2023-12-07T10:17:54.718 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/containers/storage/overlay/bbeb8f0c70f78e52f69dcffc3a76b5e5b7b2fa89e5aecc3364074fd24e7f7370/merged 2023-12-07T10:17:54.718 INFO:teuthology.orchestra.run.smithi119.stdout:/run/user/0 2023-12-07T10:17:54.718 INFO:teuthology.orchestra.run.smithi119.stdout:/proc/sys/fs/binfmt_misc 2023-12-07T10:17:54.718 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/containers/storage/overlay/fb938085bf938dca2ac12f760375de694a5aad90d0c97f7f3ed1a53ee9df19d1/merged 2023-12-07T10:17:54.719 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/containers/storage/overlay/e3b6c058f929cffaacc05317ef3ade43152453a2853efdf257f132518c8b30f2/merged 2023-12-07T10:17:54.719 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/containers/storage/overlay/db1482fa00767390e0caa938e2d2329eebcf39230121e249f9e9682e887d19b7/merged 2023-12-07T10:17:54.719 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/containers/storage/overlay/2230e27a3e49e9b75348d545fa1f2a58c1f1193297d29f7b48c8ca08cd0e59e6/merged 2023-12-07T10:17:54.719 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/containers/storage/overlay/760f8af43848714c12d17dd65ca1a5e8f15925bd6ba1cb1c3182f3c918d80f15/merged 2023-12-07T10:17:54.719 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/containers/storage/overlay/8fae3477a54b4cd3db1cd04e10b4f8d31b8d36c6c7fe73c5cb64304ed9298f8c/merged 2023-12-07T10:17:54.719 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/containers/storage/overlay/9841295f6850ed447d649104b3a3e5338a38b821da290e4a12f4c84938c11b9c/merged 2023-12-07T10:17:54.719 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/containers/storage/overlay/e1a9a41ea1213b79e9159322ec5a6ae50c4f0e54d055cabbe1c3e8064731eb91/merged 2023-12-07T10:17:54.719 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns 2023-12-07T10:17:54.719 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-07T10:17:54.719 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-07T10:17:54.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:17:54.720 DEBUG:teuthology.orchestra.run.smithi119:> ls /sys/fs/fuse/connections 2023-12-07T10:17:54.804 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-07T10:17:54.804 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-12-07T10:17:54.850 DEBUG:teuthology.orchestra.run.smithi119:> sudo modprobe fuse 2023-12-07T10:17:54.920 DEBUG:teuthology.orchestra.run.smithi119:> cat /proc/self/mounts | awk '{print $2}' 2023-12-07T10:17:54.965 INFO:teuthology.orchestra.run.smithi119.stdout:/sys 2023-12-07T10:17:54.965 INFO:teuthology.orchestra.run.smithi119.stdout:/proc 2023-12-07T10:17:54.965 INFO:teuthology.orchestra.run.smithi119.stdout:/dev 2023-12-07T10:17:54.965 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/security 2023-12-07T10:17:54.965 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/shm 2023-12-07T10:17:54.965 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/pts 2023-12-07T10:17:54.966 INFO:teuthology.orchestra.run.smithi119.stdout:/run 2023-12-07T10:17:54.966 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup 2023-12-07T10:17:54.966 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup/systemd 2023-12-07T10:17:54.966 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/pstore 2023-12-07T10:17:54.966 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-12-07T10:17:54.966 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup/cpuset 2023-12-07T10:17:54.966 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup/devices 2023-12-07T10:17:54.966 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup/pids 2023-12-07T10:17:54.966 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup/blkio 2023-12-07T10:17:54.966 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup/freezer 2023-12-07T10:17:54.966 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup/perf_event 2023-12-07T10:17:54.966 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-12-07T10:17:54.966 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup/memory 2023-12-07T10:17:54.967 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup/hugetlb 2023-12-07T10:17:54.967 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/tracing 2023-12-07T10:17:54.967 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/config 2023-12-07T10:17:54.967 INFO:teuthology.orchestra.run.smithi119.stdout:/ 2023-12-07T10:17:54.967 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/selinux 2023-12-07T10:17:54.967 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/mqueue 2023-12-07T10:17:54.967 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/debug 2023-12-07T10:17:54.967 INFO:teuthology.orchestra.run.smithi119.stdout:/proc/sys/fs/binfmt_misc 2023-12-07T10:17:54.967 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/hugepages 2023-12-07T10:17:54.967 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/fuse/connections 2023-12-07T10:17:54.967 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/nfs/rpc_pipefs 2023-12-07T10:17:54.967 INFO:teuthology.orchestra.run.smithi119.stdout:/run/user/1000 2023-12-07T10:17:54.967 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph 2023-12-07T10:17:54.967 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/containers/storage/overlay 2023-12-07T10:17:54.967 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/containers/storage/overlay/94bb8aa2c375f56012694782590080dec9ba5b314d15ac84afa4b081b2ebf79b/merged 2023-12-07T10:17:54.968 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/containers/storage/overlay/bbeb8f0c70f78e52f69dcffc3a76b5e5b7b2fa89e5aecc3364074fd24e7f7370/merged 2023-12-07T10:17:54.968 INFO:teuthology.orchestra.run.smithi119.stdout:/run/user/0 2023-12-07T10:17:54.968 INFO:teuthology.orchestra.run.smithi119.stdout:/proc/sys/fs/binfmt_misc 2023-12-07T10:17:54.968 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/containers/storage/overlay/fb938085bf938dca2ac12f760375de694a5aad90d0c97f7f3ed1a53ee9df19d1/merged 2023-12-07T10:17:54.968 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/containers/storage/overlay/e3b6c058f929cffaacc05317ef3ade43152453a2853efdf257f132518c8b30f2/merged 2023-12-07T10:17:54.968 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/containers/storage/overlay/db1482fa00767390e0caa938e2d2329eebcf39230121e249f9e9682e887d19b7/merged 2023-12-07T10:17:54.968 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/containers/storage/overlay/2230e27a3e49e9b75348d545fa1f2a58c1f1193297d29f7b48c8ca08cd0e59e6/merged 2023-12-07T10:17:54.968 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/containers/storage/overlay/760f8af43848714c12d17dd65ca1a5e8f15925bd6ba1cb1c3182f3c918d80f15/merged 2023-12-07T10:17:54.968 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/containers/storage/overlay/8fae3477a54b4cd3db1cd04e10b4f8d31b8d36c6c7fe73c5cb64304ed9298f8c/merged 2023-12-07T10:17:54.968 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/containers/storage/overlay/9841295f6850ed447d649104b3a3e5338a38b821da290e4a12f4c84938c11b9c/merged 2023-12-07T10:17:54.968 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/containers/storage/overlay/e1a9a41ea1213b79e9159322ec5a6ae50c4f0e54d055cabbe1c3e8064731eb91/merged 2023-12-07T10:17:54.968 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns 2023-12-07T10:17:54.968 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-07T10:17:54.968 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-07T10:17:54.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:17:54.970 DEBUG:teuthology.orchestra.run.smithi119:> ls /sys/fs/fuse/connections 2023-12-07T10:17:55.041 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi119.stderr:2023-12-07T10:17:55.038+0000 7f4763ef85c0 -1 init, newargv = 0x5609175942d0 newargc=15 2023-12-07T10:17:55.041 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi119.stderr:ceph-fuse[149244]: starting ceph client 2023-12-07T10:17:55.052 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi119.stderr:ceph-fuse[149244]: starting fuse 2023-12-07T10:17:55.057 INFO:teuthology.orchestra.run.smithi119.stdout:121 2023-12-07T10:17:55.057 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2023-12-07T10:17:55.057 DEBUG:teuthology.orchestra.run.smithi119:> sudo stdin-killer -- python3 -c ' 2023-12-07T10:17:55.057 DEBUG:teuthology.orchestra.run.smithi119:> import glob 2023-12-07T10:17:55.058 DEBUG:teuthology.orchestra.run.smithi119:> import re 2023-12-07T10:17:55.058 DEBUG:teuthology.orchestra.run.smithi119:> import os 2023-12-07T10:17:55.058 DEBUG:teuthology.orchestra.run.smithi119:> import subprocess 2023-12-07T10:17:55.058 DEBUG:teuthology.orchestra.run.smithi119:> 2023-12-07T10:17:55.058 DEBUG:teuthology.orchestra.run.smithi119:> def _find_admin_socket(client_name): 2023-12-07T10:17:55.058 DEBUG:teuthology.orchestra.run.smithi119:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-12-07T10:17:55.058 DEBUG:teuthology.orchestra.run.smithi119:> files = glob.glob(asok_path) 2023-12-07T10:17:55.058 DEBUG:teuthology.orchestra.run.smithi119:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-12-07T10:17:55.058 DEBUG:teuthology.orchestra.run.smithi119:> 2023-12-07T10:17:55.058 DEBUG:teuthology.orchestra.run.smithi119:> # Given a non-glob path, it better be there 2023-12-07T10:17:55.058 DEBUG:teuthology.orchestra.run.smithi119:> if "*" not in asok_path: 2023-12-07T10:17:55.058 DEBUG:teuthology.orchestra.run.smithi119:> assert(len(files) == 1) 2023-12-07T10:17:55.059 DEBUG:teuthology.orchestra.run.smithi119:> return files[0] 2023-12-07T10:17:55.059 DEBUG:teuthology.orchestra.run.smithi119:> 2023-12-07T10:17:55.059 DEBUG:teuthology.orchestra.run.smithi119:> for f in files: 2023-12-07T10:17:55.059 DEBUG:teuthology.orchestra.run.smithi119:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-07T10:17:55.059 DEBUG:teuthology.orchestra.run.smithi119:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-07T10:17:55.059 DEBUG:teuthology.orchestra.run.smithi119:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-07T10:17:55.059 DEBUG:teuthology.orchestra.run.smithi119:> contents = proc_f.read() 2023-12-07T10:17:55.059 DEBUG:teuthology.orchestra.run.smithi119:> if mountpoint in contents: 2023-12-07T10:17:55.059 DEBUG:teuthology.orchestra.run.smithi119:> return f 2023-12-07T10:17:55.059 DEBUG:teuthology.orchestra.run.smithi119:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-07T10:17:55.059 DEBUG:teuthology.orchestra.run.smithi119:> 2023-12-07T10:17:55.060 DEBUG:teuthology.orchestra.run.smithi119:> print(_find_admin_socket("client.admin")) 2023-12-07T10:17:55.060 DEBUG:teuthology.orchestra.run.smithi119:> ' 2023-12-07T10:17:55.240 INFO:teuthology.orchestra.run.smithi119.stdout:/var/run/ceph/ceph-client.admin.149244.asok 2023-12-07T10:17:55.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-07T10:17:55.253 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.149244.asok 2023-12-07T10:17:55.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:17:55.253 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.149244.asok status 2023-12-07T10:17:55.477 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:17:55.477 INFO:teuthology.orchestra.run.smithi119.stdout: "metadata": { 2023-12-07T10:17:55.478 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_sha1": "3bd0bfa458b4399e7266c56d8215a2cf3a6091a0", 2023-12-07T10:17:55.478 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_version": "ceph version 19.0.0-18-g3bd0bfa4 (3bd0bfa458b4399e7266c56d8215a2cf3a6091a0) reef (dev)", 2023-12-07T10:17:55.478 INFO:teuthology.orchestra.run.smithi119.stdout: "entity_id": "admin", 2023-12-07T10:17:55.478 INFO:teuthology.orchestra.run.smithi119.stdout: "hostname": "smithi119", 2023-12-07T10:17:55.478 INFO:teuthology.orchestra.run.smithi119.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-12-07T10:17:55.478 INFO:teuthology.orchestra.run.smithi119.stdout: "pid": "149244", 2023-12-07T10:17:55.478 INFO:teuthology.orchestra.run.smithi119.stdout: "root": "/" 2023-12-07T10:17:55.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:17:55.478 INFO:teuthology.orchestra.run.smithi119.stdout: "dentry_count": 0, 2023-12-07T10:17:55.478 INFO:teuthology.orchestra.run.smithi119.stdout: "dentry_pinned_count": 0, 2023-12-07T10:17:55.478 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 14670, 2023-12-07T10:17:55.478 INFO:teuthology.orchestra.run.smithi119.stdout: "inst": { 2023-12-07T10:17:55.478 INFO:teuthology.orchestra.run.smithi119.stdout: "name": { 2023-12-07T10:17:55.478 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "client", 2023-12-07T10:17:55.479 INFO:teuthology.orchestra.run.smithi119.stdout: "num": 14670 2023-12-07T10:17:55.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:17:55.479 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": { 2023-12-07T10:17:55.479 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2023-12-07T10:17:55.479 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "192.168.0.1:0", 2023-12-07T10:17:55.479 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 2028483176 2023-12-07T10:17:55.479 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:17:55.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:17:55.479 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": { 2023-12-07T10:17:55.479 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2023-12-07T10:17:55.479 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "192.168.0.1:0", 2023-12-07T10:17:55.479 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 2028483176 2023-12-07T10:17:55.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:17:55.480 INFO:teuthology.orchestra.run.smithi119.stdout: "inst_str": "client.14670 192.168.0.1:0/2028483176", 2023-12-07T10:17:55.480 INFO:teuthology.orchestra.run.smithi119.stdout: "addr_str": "192.168.0.1:0/2028483176", 2023-12-07T10:17:55.480 INFO:teuthology.orchestra.run.smithi119.stdout: "inode_count": 1, 2023-12-07T10:17:55.480 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_epoch": 17, 2023-12-07T10:17:55.480 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_epoch": 89, 2023-12-07T10:17:55.480 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_epoch_barrier": 0, 2023-12-07T10:17:55.480 INFO:teuthology.orchestra.run.smithi119.stdout: "blocklisted": false, 2023-12-07T10:17:55.480 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs" 2023-12-07T10:17:55.480 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:17:55.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:17:55.493 DEBUG:teuthology.orchestra.run.smithi119:> stat --file-system '--printf=%T 2023-12-07T10:17:55.493 DEBUG:teuthology.orchestra.run.smithi119:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-12-07T10:17:55.537 INFO:teuthology.orchestra.run.smithi119.stdout:fuseblk 2023-12-07T10:17:55.537 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-12-07T10:17:55.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:17:55.537 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-12-07T10:17:55.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:17:55.647 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-12-07T10:17:55.772 INFO:teuthology.orchestra.run.smithi119.stdout:# file: . 2023-12-07T10:17:55.772 INFO:teuthology.orchestra.run.smithi119.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-12-07T10:17:55.772 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:17:55.773 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-12-07T10:17:55.773 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-07T10:17:55.817 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-07T10:17:55.917 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:55 smithi119 ceph-mon[97908]: pgmap v286: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:56.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.051+0000 7f32501ff700 1 -- 172.21.15.119:0/3205843240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3248102750 msgr2=0x7f3248102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:56.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.051+0000 7f32501ff700 1 --2- 172.21.15.119:0/3205843240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3248102750 0x7f3248102bb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f32380099c0 tx=0x7f3238009cd0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.051+0000 7f32501ff700 1 -- 172.21.15.119:0/3205843240 shutdown_connections 2023-12-07T10:17:56.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.051+0000 7f32501ff700 1 --2- 172.21.15.119:0/3205843240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32481030f0 0x7f32481099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.051+0000 7f32501ff700 1 --2- 172.21.15.119:0/3205843240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3248101560 0x7f3248101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.051+0000 7f32501ff700 1 --2- 172.21.15.119:0/3205843240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3248102750 0x7f3248102bb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.051+0000 7f32501ff700 1 -- 172.21.15.119:0/3205843240 >> 172.21.15.119:0/3205843240 conn(0x7f32480fcab0 msgr2=0x7f32480fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:56.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.051+0000 7f32501ff700 1 -- 172.21.15.119:0/3205843240 shutdown_connections 2023-12-07T10:17:56.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.051+0000 7f32501ff700 1 -- 172.21.15.119:0/3205843240 wait complete. 2023-12-07T10:17:56.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.052+0000 7f32501ff700 1 Processor -- start 2023-12-07T10:17:56.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.052+0000 7f32501ff700 1 -- start start 2023-12-07T10:17:56.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.052+0000 7f32501ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3248101560 0x7f32481974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:56.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.052+0000 7f32501ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3248102750 0x7f3248197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:56.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.052+0000 7f32501ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32481030f0 0x7f324819ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:56.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.052+0000 7f32501ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324819f010 con 0x7f32481030f0 2023-12-07T10:17:56.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.052+0000 7f32501ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324819f150 con 0x7f3248102750 2023-12-07T10:17:56.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.052+0000 7f32501ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324819f2e0 con 0x7f3248101560 2023-12-07T10:17:56.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.053+0000 7f324df9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3248101560 0x7f32481974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:56.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.053+0000 7f324e79c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32481030f0 0x7f324819ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:56.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.053+0000 7f324df9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3248101560 0x7f32481974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:43188/0 (socket says 172.21.15.119:43188) 2023-12-07T10:17:56.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.053+0000 7f324e79c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32481030f0 0x7f324819ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39966/0 (socket says 172.21.15.119:39966) 2023-12-07T10:17:56.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.053+0000 7f324df9b700 1 -- 172.21.15.119:0/4198997736 learned_addr learned my addr 172.21.15.119:0/4198997736 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:56.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.053+0000 7f324d79a700 1 --2- 172.21.15.119:0/4198997736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3248102750 0x7f3248197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:56.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.053+0000 7f324e79c700 1 -- 172.21.15.119:0/4198997736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3248101560 msgr2=0x7f32481974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:56.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.053+0000 7f324e79c700 1 --2- 172.21.15.119:0/4198997736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3248101560 0x7f32481974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.053+0000 7f324e79c700 1 -- 172.21.15.119:0/4198997736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3248102750 msgr2=0x7f3248197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:56.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.053+0000 7f324e79c700 1 --2- 172.21.15.119:0/4198997736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3248102750 0x7f3248197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.053+0000 7f324e79c700 1 -- 172.21.15.119:0/4198997736 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32380095f0 con 0x7f32481030f0 2023-12-07T10:17:56.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.053+0000 7f324d79a700 1 --2- 172.21.15.119:0/4198997736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3248102750 0x7f3248197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:17:56.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.053+0000 7f324e79c700 1 --2- 172.21.15.119:0/4198997736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32481030f0 0x7f324819ea40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3240007f10 tx=0x7f324000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:56.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.054+0000 7f323effd700 1 -- 172.21.15.119:0/4198997736 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324000ddd0 con 0x7f32481030f0 2023-12-07T10:17:56.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.054+0000 7f323effd700 1 -- 172.21.15.119:0/4198997736 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f324000f040 con 0x7f32481030f0 2023-12-07T10:17:56.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.054+0000 7f32501ff700 1 -- 172.21.15.119:0/4198997736 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f324819f5c0 con 0x7f32481030f0 2023-12-07T10:17:56.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.054+0000 7f323effd700 1 -- 172.21.15.119:0/4198997736 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3240005a90 con 0x7f32481030f0 2023-12-07T10:17:56.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.054+0000 7f32501ff700 1 -- 172.21.15.119:0/4198997736 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f324819fa80 con 0x7f32481030f0 2023-12-07T10:17:56.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.055+0000 7f32501ff700 1 -- 172.21.15.119:0/4198997736 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3248108910 con 0x7f32481030f0 2023-12-07T10:17:56.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.058+0000 7f323effd700 1 -- 172.21.15.119:0/4198997736 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f3240004ad0 con 0x7f32481030f0 2023-12-07T10:17:56.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.059+0000 7f323effd700 1 --2- 172.21.15.119:0/4198997736 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f32340a1b10 0x7f32340a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:56.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.059+0000 7f323effd700 1 -- 172.21.15.119:0/4198997736 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f32400cd8e0 con 0x7f32481030f0 2023-12-07T10:17:56.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.060+0000 7f324df9b700 1 --2- 172.21.15.119:0/4198997736 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f32340a1b10 0x7f32340a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:56.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.060+0000 7f324df9b700 1 --2- 172.21.15.119:0/4198997736 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f32340a1b10 0x7f32340a3fd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3244005950 tx=0x7f3244009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:56.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.062+0000 7f323effd700 1 -- 172.21.15.119:0/4198997736 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3240096b10 con 0x7f32481030f0 2023-12-07T10:17:56.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:55 smithi159 ceph-mon[134726]: pgmap v286: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:56.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.266+0000 7f32501ff700 1 -- 172.21.15.119:0/4198997736 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f324810cdd0 con 0x7f32481030f0 2023-12-07T10:17:56.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.268+0000 7f323effd700 1 -- 172.21.15.119:0/4198997736 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24030 (secure 0 0 0) 0x7f324009a130 con 0x7f32481030f0 2023-12-07T10:17:56.271 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:17:56.271 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":89,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","created":"2023-12-07T10:09:59.945863+0000","modified":"2023-12-07T10:17:46.277123+0000","last_up_change":"2023-12-07T10:16:50.929427+0000","last_in_change":"2023-12-07T10:16:33.558158+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T10:13:25.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-07T10:17:39.347813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-07T10:17:40.714501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-07T10:17:44.257760+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"56b5818e-e625-40f1-8838-3ddcf2d34ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6803","nonce":2141950133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6805","nonce":2141950133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6809","nonce":2141950133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6807","nonce":2141950133}]},"public_addr":"172.21.15.119:6803/2141950133","cluster_addr":"172.21.15.119:6805/2141950133","heartbeat_back_addr":"172.21.15.119:6809/2141950133","heartbeat_front_addr":"172.21.15.119:6807/2141950133","state":["exists","up"]},{"osd":1,"uuid":"36489a6a-5278-4802-9525-9871521722e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6801","nonce":3390875370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6803","nonce":3390875370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6807","nonce":3390875370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6805","nonce":3390875370}]},"public_addr":"172.21.15.153:6801/3390875370","cluster_addr":"172.21.15.153:6803/3390875370","heartbeat_back_addr":"172.21.15.153:6807/3390875370","heartbeat_front_addr":"172.21.15.153:6805/3390875370","state":["exists","up"]},{"osd":2,"uuid":"a937ad26-5e9b-4c8a-b65a-626a8feda6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6801","nonce":708101637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6803","nonce":708101637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6807","nonce":708101637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6805","nonce":708101637}]},"public_addr":"172.21.15.159:6801/708101637","cluster_addr":"172.21.15.159:6803/708101637","heartbeat_back_addr":"172.21.15.159:6807/708101637","heartbeat_front_addr":"172.21.15.159:6805/708101637","state":["exists","up"]},{"osd":3,"uuid":"26ef8300-aa07-48f6-b31d-8b0f2217ffcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6811","nonce":3228047257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6813","nonce":3228047257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6817","nonce":3228047257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6815","nonce":3228047257}]},"public_addr":"172.21.15.119:6811/3228047257","cluster_addr":"172.21.15.119:6813/3228047257","heartbeat_back_addr":"172.21.15.119:6817/3228047257","heartbeat_front_addr":"172.21.15.119:6815/3228047257","state":["exists","up"]},{"osd":4,"uuid":"000ee4b9-9896-4901-8cc3-c85fdfc8cbaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6809","nonce":4021998299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6811","nonce":4021998299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6815","nonce":4021998299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6813","nonce":4021998299}]},"public_addr":"172.21.15.153:6809/4021998299","cluster_addr":"172.21.15.153:6811/4021998299","heartbeat_back_addr":"172.21.15.153:6815/4021998299","heartbeat_front_addr":"172.21.15.153:6813/4021998299","state":["exists","up"]},{"osd":5,"uuid":"8b2fc167-262d-474f-89f3-47bd29424421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6809","nonce":1832540768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6811","nonce":1832540768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6815","nonce":1832540768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6813","nonce":1832540768}]},"public_addr":"172.21.15.159:6809/1832540768","cluster_addr":"172.21.15.159:6811/1832540768","heartbeat_back_addr":"172.21.15.159:6815/1832540768","heartbeat_front_addr":"172.21.15.159:6813/1832540768","state":["exists","up"]},{"osd":6,"uuid":"f9aa6749-46cf-4b2a-8114-cbd034219d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162436840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162436840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6825","nonce":1162436840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162436840}]},"public_addr":"172.21.15.119:6819/1162436840","cluster_addr":"172.21.15.119:6821/1162436840","heartbeat_back_addr":"172.21.15.119:6825/1162436840","heartbeat_front_addr":"172.21.15.119:6823/1162436840","state":["exists","up"]},{"osd":7,"uuid":"39216361-e1c4-4fef-86cb-a4707656b547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6817","nonce":3254653457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6819","nonce":3254653457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6823","nonce":3254653457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6821","nonce":3254653457}]},"public_addr":"172.21.15.153:6817/3254653457","cluster_addr":"172.21.15.153:6819/3254653457","heartbeat_back_addr":"172.21.15.153:6823/3254653457","heartbeat_front_addr":"172.21.15.153:6821/3254653457","state":["exists","up"]},{"osd":8,"uuid":"cceb6ebc-7c02-48ea-9d16-b7a016dd449b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6817","nonce":2600946380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6819","nonce":2600946380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6823","nonce":2600946380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6821","nonce":2600946380}]},"public_addr":"172.21.15.159:6817/2600946380","cluster_addr":"172.21.15.159:6819/2600946380","heartbeat_back_addr":"172.21.15.159:6823/2600946380","heartbeat_front_addr":"172.21.15.159:6821/2600946380","state":["exists","up"]},{"osd":9,"uuid":"f337a2e4-e065-4a9f-91d9-e3615de2d6f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6827","nonce":1940737285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6829","nonce":1940737285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6833","nonce":1940737285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6831","nonce":1940737285}]},"public_addr":"172.21.15.119:6827/1940737285","cluster_addr":"172.21.15.119:6829/1940737285","heartbeat_back_addr":"172.21.15.119:6833/1940737285","heartbeat_front_addr":"172.21.15.119:6831/1940737285","state":["exists","up"]},{"osd":10,"uuid":"3a3bc347-85f4-4878-9797-d9da96137e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6825","nonce":663834864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6827","nonce":663834864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6831","nonce":663834864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6829","nonce":663834864}]},"public_addr":"172.21.15.153:6825/663834864","cluster_addr":"172.21.15.153:6827/663834864","heartbeat_back_addr":"172.21.15.153:6831/663834864","heartbeat_front_addr":"172.21.15.153:6829/663834864","state":["exists","up"]},{"osd":11,"uuid":"a0fc5909-279d-4172-977c-2dc2da0a7e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6825","nonce":1262216348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6827","nonce":1262216348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6831","nonce":1262216348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6829","nonce":1262216348}]},"public_addr":"172.21.15.159:6825/1262216348","cluster_addr":"172.21.15.159:6827/1262216348","heartbeat_back_addr":"172.21.15.159:6831/1262216348","heartbeat_front_addr":"172.21.15.159:6829/1262216348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:37.488109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:59.543060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:21.381585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:47.112729+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:09.732150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:30.893729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:56.678710+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:19.004256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:40.388818+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:06.391734+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:28.394053+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:49.634348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1628639392":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6801/1355617847":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6800/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/362307836":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/128964558":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/2278813554":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/1411933614":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/1685698277":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6800/1355617847":"2023-12-08T10:11:00.585565+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T10:17:56.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.270+0000 7f32501ff700 1 -- 172.21.15.119:0/4198997736 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f32340a1b10 msgr2=0x7f32340a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:56.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.270+0000 7f32501ff700 1 --2- 172.21.15.119:0/4198997736 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f32340a1b10 0x7f32340a3fd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3244005950 tx=0x7f3244009260 comp rx=0 tx=0).stop 2023-12-07T10:17:56.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.270+0000 7f32501ff700 1 -- 172.21.15.119:0/4198997736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32481030f0 msgr2=0x7f324819ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:56.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.270+0000 7f32501ff700 1 --2- 172.21.15.119:0/4198997736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32481030f0 0x7f324819ea40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3240007f10 tx=0x7f324000d3b0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.270+0000 7f32501ff700 1 -- 172.21.15.119:0/4198997736 shutdown_connections 2023-12-07T10:17:56.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.270+0000 7f32501ff700 1 --2- 172.21.15.119:0/4198997736 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f32340a1b10 0x7f32340a3fd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.270+0000 7f32501ff700 1 --2- 172.21.15.119:0/4198997736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32481030f0 0x7f324819ea40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.270+0000 7f32501ff700 1 --2- 172.21.15.119:0/4198997736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3248101560 0x7f32481974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.270+0000 7f32501ff700 1 --2- 172.21.15.119:0/4198997736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3248102750 0x7f3248197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.270+0000 7f32501ff700 1 -- 172.21.15.119:0/4198997736 >> 172.21.15.119:0/4198997736 conn(0x7f32480fcab0 msgr2=0x7f3248105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:56.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.271+0000 7f32501ff700 1 -- 172.21.15.119:0/4198997736 shutdown_connections 2023-12-07T10:17:56.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.271+0000 7f32501ff700 1 -- 172.21.15.119:0/4198997736 wait complete. 2023-12-07T10:17:56.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:55 smithi153 ceph-mon[133625]: pgmap v286: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:17:56.292 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-07T10:13:25.194366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-07T10:17:39.347813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-07T10:17:40.714501+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-07T10:17:44.257760+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-07T10:17:56.292 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-07T10:17:56.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.488+0000 7fdd34dff700 1 -- 172.21.15.119:0/1578018750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd301015c0 msgr2=0x7fdd301019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:56.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.488+0000 7fdd34dff700 1 --2- 172.21.15.119:0/1578018750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd301015c0 0x7fdd301019e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdd200099c0 tx=0x7fdd20009cd0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.488+0000 7fdd34dff700 1 -- 172.21.15.119:0/1578018750 shutdown_connections 2023-12-07T10:17:56.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.488+0000 7fdd34dff700 1 --2- 172.21.15.119:0/1578018750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd30103150 0x7fdd30109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.488+0000 7fdd34dff700 1 --2- 172.21.15.119:0/1578018750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd301015c0 0x7fdd301019e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.488+0000 7fdd34dff700 1 --2- 172.21.15.119:0/1578018750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd301027b0 0x7fdd30102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.488+0000 7fdd34dff700 1 -- 172.21.15.119:0/1578018750 >> 172.21.15.119:0/1578018750 conn(0x7fdd300fcb10 msgr2=0x7fdd300fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:56.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.488+0000 7fdd34dff700 1 -- 172.21.15.119:0/1578018750 shutdown_connections 2023-12-07T10:17:56.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.488+0000 7fdd34dff700 1 -- 172.21.15.119:0/1578018750 wait complete. 2023-12-07T10:17:56.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.489+0000 7fdd34dff700 1 Processor -- start 2023-12-07T10:17:56.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.489+0000 7fdd34dff700 1 -- start start 2023-12-07T10:17:56.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.490+0000 7fdd34dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd301015c0 0x7fdd301997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:56.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.490+0000 7fdd34dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd301027b0 0x7fdd30199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:56.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.490+0000 7fdd34dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd30103150 0x7fdd301a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:56.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.490+0000 7fdd34dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd301a12d0 con 0x7fdd30103150 2023-12-07T10:17:56.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.490+0000 7fdd34dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd301a1410 con 0x7fdd301027b0 2023-12-07T10:17:56.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.490+0000 7fdd34dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd301a15a0 con 0x7fdd301015c0 2023-12-07T10:17:56.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.490+0000 7fdd2e59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd301015c0 0x7fdd301997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:56.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.490+0000 7fdd2e59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd301015c0 0x7fdd301997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:43212/0 (socket says 172.21.15.119:43212) 2023-12-07T10:17:56.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.490+0000 7fdd2e59c700 1 -- 172.21.15.119:0/1480633631 learned_addr learned my addr 172.21.15.119:0/1480633631 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:56.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.490+0000 7fdd2ed9d700 1 --2- 172.21.15.119:0/1480633631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd30103150 0x7fdd301a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:56.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.490+0000 7fdd2dd9b700 1 --2- 172.21.15.119:0/1480633631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd301027b0 0x7fdd30199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:56.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.490+0000 7fdd2e59c700 1 -- 172.21.15.119:0/1480633631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd301027b0 msgr2=0x7fdd30199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:56.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.490+0000 7fdd2e59c700 1 --2- 172.21.15.119:0/1480633631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd301027b0 0x7fdd30199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.490+0000 7fdd2e59c700 1 -- 172.21.15.119:0/1480633631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd30103150 msgr2=0x7fdd301a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:56.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.490+0000 7fdd2e59c700 1 --2- 172.21.15.119:0/1480633631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd30103150 0x7fdd301a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.490+0000 7fdd2e59c700 1 -- 172.21.15.119:0/1480633631 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd200095f0 con 0x7fdd301015c0 2023-12-07T10:17:56.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.491+0000 7fdd2e59c700 1 --2- 172.21.15.119:0/1480633631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd301015c0 0x7fdd301997a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdd20003730 tx=0x7fdd2000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:56.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.491+0000 7fdd1f7fe700 1 -- 172.21.15.119:0/1480633631 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2001d070 con 0x7fdd301015c0 2023-12-07T10:17:56.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.491+0000 7fdd34dff700 1 -- 172.21.15.119:0/1480633631 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd301a1820 con 0x7fdd301015c0 2023-12-07T10:17:56.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.491+0000 7fdd1f7fe700 1 -- 172.21.15.119:0/1480633631 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd2000bda0 con 0x7fdd301015c0 2023-12-07T10:17:56.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.491+0000 7fdd1f7fe700 1 -- 172.21.15.119:0/1480633631 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd200173f0 con 0x7fdd301015c0 2023-12-07T10:17:56.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.491+0000 7fdd34dff700 1 -- 172.21.15.119:0/1480633631 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd301a1ca0 con 0x7fdd301015c0 2023-12-07T10:17:56.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.492+0000 7fdd34dff700 1 -- 172.21.15.119:0/1480633631 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd300683c0 con 0x7fdd301015c0 2023-12-07T10:17:56.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.494+0000 7fdd1f7fe700 1 -- 172.21.15.119:0/1480633631 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fdd20022470 con 0x7fdd301015c0 2023-12-07T10:17:56.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.495+0000 7fdd1f7fe700 1 --2- 172.21.15.119:0/1480633631 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fdd180a1b90 0x7fdd180a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:56.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.496+0000 7fdd1f7fe700 1 -- 172.21.15.119:0/1480633631 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fdd200cece0 con 0x7fdd301015c0 2023-12-07T10:17:56.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.496+0000 7fdd2dd9b700 1 --2- 172.21.15.119:0/1480633631 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fdd180a1b90 0x7fdd180a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:56.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.496+0000 7fdd2dd9b700 1 --2- 172.21.15.119:0/1480633631 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fdd180a1b90 0x7fdd180a4050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdd24009c60 tx=0x7fdd24009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:56.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.498+0000 7fdd1f7fe700 1 -- 172.21.15.119:0/1480633631 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fdd20097f40 con 0x7fdd301015c0 2023-12-07T10:17:56.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.698+0000 7fdd34dff700 1 -- 172.21.15.119:0/1480633631 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdd3010cfd0 con 0x7fdd301015c0 2023-12-07T10:17:56.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.698+0000 7fdd1f7fe700 1 -- 172.21.15.119:0/1480633631 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdd200270a0 con 0x7fdd301015c0 2023-12-07T10:17:56.701 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-12-07T10:17:56.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.701+0000 7fdd34dff700 1 -- 172.21.15.119:0/1480633631 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fdd180a1b90 msgr2=0x7fdd180a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:56.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.701+0000 7fdd34dff700 1 --2- 172.21.15.119:0/1480633631 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fdd180a1b90 0x7fdd180a4050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdd24009c60 tx=0x7fdd24009260 comp rx=0 tx=0).stop 2023-12-07T10:17:56.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.701+0000 7fdd34dff700 1 -- 172.21.15.119:0/1480633631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd301015c0 msgr2=0x7fdd301997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:56.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.701+0000 7fdd34dff700 1 --2- 172.21.15.119:0/1480633631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd301015c0 0x7fdd301997a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdd20003730 tx=0x7fdd2000fa00 comp rx=0 tx=0).stop 2023-12-07T10:17:56.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.701+0000 7fdd34dff700 1 -- 172.21.15.119:0/1480633631 shutdown_connections 2023-12-07T10:17:56.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.701+0000 7fdd34dff700 1 --2- 172.21.15.119:0/1480633631 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fdd180a1b90 0x7fdd180a4050 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.701+0000 7fdd34dff700 1 --2- 172.21.15.119:0/1480633631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd30103150 0x7fdd301a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.701+0000 7fdd34dff700 1 --2- 172.21.15.119:0/1480633631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd301015c0 0x7fdd301997a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.701+0000 7fdd34dff700 1 --2- 172.21.15.119:0/1480633631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd301027b0 0x7fdd30199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.701+0000 7fdd34dff700 1 -- 172.21.15.119:0/1480633631 >> 172.21.15.119:0/1480633631 conn(0x7fdd300fcb10 msgr2=0x7fdd30073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:56.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.701+0000 7fdd34dff700 1 -- 172.21.15.119:0/1480633631 shutdown_connections 2023-12-07T10:17:56.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.701+0000 7fdd34dff700 1 -- 172.21.15.119:0/1480633631 wait complete. 2023-12-07T10:17:56.723 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-07T10:17:56.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.923+0000 7f41e9bff700 1 -- 172.21.15.119:0/3989670849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41e41027b0 msgr2=0x7f41e4102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:56.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.923+0000 7f41e9bff700 1 --2- 172.21.15.119:0/3989670849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41e41027b0 0x7f41e4102c10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f41d8009990 tx=0x7f41d8009ca0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.923+0000 7f41e9bff700 1 -- 172.21.15.119:0/3989670849 shutdown_connections 2023-12-07T10:17:56.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.923+0000 7f41e9bff700 1 --2- 172.21.15.119:0/3989670849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41e4103150 0x7f41e4109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.923+0000 7f41e9bff700 1 --2- 172.21.15.119:0/3989670849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41e41015c0 0x7f41e41019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.923+0000 7f41e9bff700 1 --2- 172.21.15.119:0/3989670849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41e41027b0 0x7f41e4102c10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.923+0000 7f41e9bff700 1 -- 172.21.15.119:0/3989670849 >> 172.21.15.119:0/3989670849 conn(0x7f41e40fcb10 msgr2=0x7f41e40fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:56.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.923+0000 7f41e9bff700 1 -- 172.21.15.119:0/3989670849 shutdown_connections 2023-12-07T10:17:56.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.923+0000 7f41e9bff700 1 -- 172.21.15.119:0/3989670849 wait complete. 2023-12-07T10:17:56.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.924+0000 7f41e9bff700 1 Processor -- start 2023-12-07T10:17:56.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.924+0000 7f41e9bff700 1 -- start start 2023-12-07T10:17:56.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.924+0000 7f41e9bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41e41015c0 0x7f41e4199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:56.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.924+0000 7f41e9bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41e41027b0 0x7f41e4199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:56.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.924+0000 7f41e9bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41e4103150 0x7f41e41a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:56.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.924+0000 7f41e9bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e41a1250 con 0x7f41e4103150 2023-12-07T10:17:56.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.924+0000 7f41e9bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e41a1390 con 0x7f41e41015c0 2023-12-07T10:17:56.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.924+0000 7f41e9bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e41a1520 con 0x7f41e41027b0 2023-12-07T10:17:56.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.925+0000 7f41e37fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41e41015c0 0x7f41e4199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:56.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.925+0000 7f41e3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41e4103150 0x7f41e41a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:56.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.925+0000 7f41e37fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41e41015c0 0x7f41e4199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:40136/0 (socket says 172.21.15.119:40136) 2023-12-07T10:17:56.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.925+0000 7f41e37fe700 1 -- 172.21.15.119:0/2193739385 learned_addr learned my addr 172.21.15.119:0/2193739385 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:56.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.925+0000 7f41e2ffd700 1 --2- 172.21.15.119:0/2193739385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41e41027b0 0x7f41e4199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:56.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.925+0000 7f41e3fff700 1 -- 172.21.15.119:0/2193739385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41e41027b0 msgr2=0x7f41e4199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:56.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.925+0000 7f41e3fff700 1 --2- 172.21.15.119:0/2193739385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41e41027b0 0x7f41e4199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.925+0000 7f41e3fff700 1 -- 172.21.15.119:0/2193739385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41e41015c0 msgr2=0x7f41e4199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:56.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.925+0000 7f41e3fff700 1 --2- 172.21.15.119:0/2193739385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41e41015c0 0x7f41e4199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:56.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.925+0000 7f41e3fff700 1 -- 172.21.15.119:0/2193739385 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d80095f0 con 0x7f41e4103150 2023-12-07T10:17:56.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.925+0000 7f41e37fe700 1 --2- 172.21.15.119:0/2193739385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41e41015c0 0x7f41e4199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:17:56.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.925+0000 7f41e3fff700 1 --2- 172.21.15.119:0/2193739385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41e4103150 0x7f41e41a0c80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00efc0 tx=0x7f41dc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:56.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.926+0000 7f41e0ff9700 1 -- 172.21.15.119:0/2193739385 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc018070 con 0x7f41e4103150 2023-12-07T10:17:56.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.926+0000 7f41e9bff700 1 -- 172.21.15.119:0/2193739385 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e41a17a0 con 0x7f41e4103150 2023-12-07T10:17:56.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.926+0000 7f41e0ff9700 1 -- 172.21.15.119:0/2193739385 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41dc00f040 con 0x7f41e4103150 2023-12-07T10:17:56.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.926+0000 7f41e0ff9700 1 -- 172.21.15.119:0/2193739385 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc00cd40 con 0x7f41e4103150 2023-12-07T10:17:56.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.926+0000 7f41e9bff700 1 -- 172.21.15.119:0/2193739385 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e41a1c60 con 0x7f41e4103150 2023-12-07T10:17:56.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.927+0000 7f41e9bff700 1 -- 172.21.15.119:0/2193739385 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e40683c0 con 0x7f41e4103150 2023-12-07T10:17:56.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.929+0000 7f41e0ff9700 1 -- 172.21.15.119:0/2193739385 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f41dc003a00 con 0x7f41e4103150 2023-12-07T10:17:56.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.930+0000 7f41e0ff9700 1 --2- 172.21.15.119:0/2193739385 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f41cc0a1b10 0x7f41cc0a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:56.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.930+0000 7f41e0ff9700 1 -- 172.21.15.119:0/2193739385 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f41dc0cd370 con 0x7f41e4103150 2023-12-07T10:17:56.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.930+0000 7f41e37fe700 1 --2- 172.21.15.119:0/2193739385 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f41cc0a1b10 0x7f41cc0a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:56.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.931+0000 7f41e37fe700 1 --2- 172.21.15.119:0/2193739385 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f41cc0a1b10 0x7f41cc0a3fd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f41d4005950 tx=0x7f41d4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:56.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:56.934+0000 7f41e0ff9700 1 -- 172.21.15.119:0/2193739385 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f41dc096630 con 0x7f41e4103150 2023-12-07T10:17:56.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:56 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/4198997736' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:17:56.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:56 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1480633631' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:17:57.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.137+0000 7f41e9bff700 1 -- 172.21.15.119:0/2193739385 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f41e41a1f50 con 0x7f41e4103150 2023-12-07T10:17:57.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.137+0000 7f41e0ff9700 1 -- 172.21.15.119:0/2193739385 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f41dc099c50 con 0x7f41e4103150 2023-12-07T10:17:57.140 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-12-07T10:17:57.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.140+0000 7f41e9bff700 1 -- 172.21.15.119:0/2193739385 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f41cc0a1b10 msgr2=0x7f41cc0a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:57.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.140+0000 7f41e9bff700 1 --2- 172.21.15.119:0/2193739385 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f41cc0a1b10 0x7f41cc0a3fd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f41d4005950 tx=0x7f41d4009260 comp rx=0 tx=0).stop 2023-12-07T10:17:57.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.140+0000 7f41e9bff700 1 -- 172.21.15.119:0/2193739385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41e4103150 msgr2=0x7f41e41a0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:57.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.140+0000 7f41e9bff700 1 --2- 172.21.15.119:0/2193739385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41e4103150 0x7f41e41a0c80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00efc0 tx=0x7f41dc00c3a0 comp rx=0 tx=0).stop 2023-12-07T10:17:57.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.140+0000 7f41e9bff700 1 -- 172.21.15.119:0/2193739385 shutdown_connections 2023-12-07T10:17:57.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.140+0000 7f41e9bff700 1 --2- 172.21.15.119:0/2193739385 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f41cc0a1b10 0x7f41cc0a3fd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:57.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.140+0000 7f41e9bff700 1 --2- 172.21.15.119:0/2193739385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41e4103150 0x7f41e41a0c80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:57.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.140+0000 7f41e9bff700 1 --2- 172.21.15.119:0/2193739385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41e41015c0 0x7f41e4199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:57.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.140+0000 7f41e9bff700 1 --2- 172.21.15.119:0/2193739385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41e41027b0 0x7f41e4199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:57.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.140+0000 7f41e9bff700 1 -- 172.21.15.119:0/2193739385 >> 172.21.15.119:0/2193739385 conn(0x7f41e40fcb10 msgr2=0x7f41e4107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:57.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.140+0000 7f41e9bff700 1 -- 172.21.15.119:0/2193739385 shutdown_connections 2023-12-07T10:17:57.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.140+0000 7f41e9bff700 1 -- 172.21.15.119:0/2193739385 wait complete. 2023-12-07T10:17:57.161 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-07T10:17:57.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:56 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/4198997736' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:17:57.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:56 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1480633631' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:17:57.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:56 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/4198997736' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:17:57.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:56 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1480633631' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:17:57.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.356+0000 7f04ca1ff700 1 -- 172.21.15.119:0/3759524810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04c41015c0 msgr2=0x7f04c41019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:57.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.356+0000 7f04ca1ff700 1 --2- 172.21.15.119:0/3759524810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04c41015c0 0x7f04c41019e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f04b40099c0 tx=0x7f04b4009cd0 comp rx=0 tx=0).stop 2023-12-07T10:17:57.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.356+0000 7f04ca1ff700 1 -- 172.21.15.119:0/3759524810 shutdown_connections 2023-12-07T10:17:57.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.356+0000 7f04ca1ff700 1 --2- 172.21.15.119:0/3759524810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04c4103150 0x7f04c4109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:57.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.356+0000 7f04ca1ff700 1 --2- 172.21.15.119:0/3759524810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04c41015c0 0x7f04c41019e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:57.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.356+0000 7f04ca1ff700 1 --2- 172.21.15.119:0/3759524810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04c41027b0 0x7f04c4102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:57.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.356+0000 7f04ca1ff700 1 -- 172.21.15.119:0/3759524810 >> 172.21.15.119:0/3759524810 conn(0x7f04c40fcb10 msgr2=0x7f04c40fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:57.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.356+0000 7f04ca1ff700 1 -- 172.21.15.119:0/3759524810 shutdown_connections 2023-12-07T10:17:57.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.356+0000 7f04ca1ff700 1 -- 172.21.15.119:0/3759524810 wait complete. 2023-12-07T10:17:57.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.358+0000 7f04ca1ff700 1 Processor -- start 2023-12-07T10:17:57.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.358+0000 7f04ca1ff700 1 -- start start 2023-12-07T10:17:57.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.358+0000 7f04ca1ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04c41015c0 0x7f04c41997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:57.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.358+0000 7f04ca1ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04c41027b0 0x7f04c4199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:57.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.358+0000 7f04ca1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04c4103150 0x7f04c41a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:57.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.358+0000 7f04ca1ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c41a12d0 con 0x7f04c41027b0 2023-12-07T10:17:57.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.358+0000 7f04ca1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c41a1410 con 0x7f04c4103150 2023-12-07T10:17:57.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.358+0000 7f04ca1ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c41a15a0 con 0x7f04c41015c0 2023-12-07T10:17:57.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.359+0000 7f04c2ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04c41027b0 0x7f04c4199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:57.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.359+0000 7f04c2ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04c41027b0 0x7f04c4199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40012/0 (socket says 172.21.15.119:40012) 2023-12-07T10:17:57.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.359+0000 7f04c2ffd700 1 -- 172.21.15.119:0/2873010544 learned_addr learned my addr 172.21.15.119:0/2873010544 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:57.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.359+0000 7f04c37fe700 1 --2- 172.21.15.119:0/2873010544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04c41015c0 0x7f04c41997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:57.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.359+0000 7f04c3fff700 1 --2- 172.21.15.119:0/2873010544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04c4103150 0x7f04c41a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:57.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.359+0000 7f04c2ffd700 1 -- 172.21.15.119:0/2873010544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04c41015c0 msgr2=0x7f04c41997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:57.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.359+0000 7f04c2ffd700 1 --2- 172.21.15.119:0/2873010544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04c41015c0 0x7f04c41997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:57.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.359+0000 7f04c2ffd700 1 -- 172.21.15.119:0/2873010544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04c4103150 msgr2=0x7f04c41a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:57.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.359+0000 7f04c2ffd700 1 --2- 172.21.15.119:0/2873010544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04c4103150 0x7f04c41a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:57.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.359+0000 7f04c2ffd700 1 -- 172.21.15.119:0/2873010544 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b40095f0 con 0x7f04c41027b0 2023-12-07T10:17:57.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.359+0000 7f04c2ffd700 1 --2- 172.21.15.119:0/2873010544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04c41027b0 0x7f04c4199d30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f04b800d980 tx=0x7f04b800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:57.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.359+0000 7f04c0ff9700 1 -- 172.21.15.119:0/2873010544 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8004220 con 0x7f04c41027b0 2023-12-07T10:17:57.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.360+0000 7f04ca1ff700 1 -- 172.21.15.119:0/2873010544 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c41a1880 con 0x7f04c41027b0 2023-12-07T10:17:57.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.360+0000 7f04c0ff9700 1 -- 172.21.15.119:0/2873010544 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04b8007640 con 0x7f04c41027b0 2023-12-07T10:17:57.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.360+0000 7f04ca1ff700 1 -- 172.21.15.119:0/2873010544 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c41a1de0 con 0x7f04c41027b0 2023-12-07T10:17:57.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.360+0000 7f04c0ff9700 1 -- 172.21.15.119:0/2873010544 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8005630 con 0x7f04c41027b0 2023-12-07T10:17:57.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.361+0000 7f04ca1ff700 1 -- 172.21.15.119:0/2873010544 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c40683c0 con 0x7f04c41027b0 2023-12-07T10:17:57.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.363+0000 7f04c0ff9700 1 -- 172.21.15.119:0/2873010544 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f04b8004750 con 0x7f04c41027b0 2023-12-07T10:17:57.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.364+0000 7f04c0ff9700 1 --2- 172.21.15.119:0/2873010544 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f04ac0a1b10 0x7f04ac0a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:57.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.364+0000 7f04c0ff9700 1 -- 172.21.15.119:0/2873010544 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f04b80cd5b0 con 0x7f04c41027b0 2023-12-07T10:17:57.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.364+0000 7f04c37fe700 1 --2- 172.21.15.119:0/2873010544 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f04ac0a1b10 0x7f04ac0a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:57.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.365+0000 7f04c37fe700 1 --2- 172.21.15.119:0/2873010544 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f04ac0a1b10 0x7f04ac0a3fd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f04b4003730 tx=0x7f04b4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:57.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.368+0000 7f04c0ff9700 1 -- 172.21.15.119:0/2873010544 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f04b8096780 con 0x7f04c41027b0 2023-12-07T10:17:57.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.568+0000 7f04ca1ff700 1 -- 172.21.15.119:0/2873010544 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f04c41a2070 con 0x7f04c41027b0 2023-12-07T10:17:57.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.569+0000 7f04c0ff9700 1 -- 172.21.15.119:0/2873010544 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f04b80160a0 con 0x7f04c41027b0 2023-12-07T10:17:57.572 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-12-07T10:17:57.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.571+0000 7f04ca1ff700 1 -- 172.21.15.119:0/2873010544 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f04ac0a1b10 msgr2=0x7f04ac0a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:57.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.571+0000 7f04ca1ff700 1 --2- 172.21.15.119:0/2873010544 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f04ac0a1b10 0x7f04ac0a3fd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f04b4003730 tx=0x7f04b4000bc0 comp rx=0 tx=0).stop 2023-12-07T10:17:57.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.571+0000 7f04ca1ff700 1 -- 172.21.15.119:0/2873010544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04c41027b0 msgr2=0x7f04c4199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:57.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.571+0000 7f04ca1ff700 1 --2- 172.21.15.119:0/2873010544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04c41027b0 0x7f04c4199d30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f04b800d980 tx=0x7f04b800dc90 comp rx=0 tx=0).stop 2023-12-07T10:17:57.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.571+0000 7f04ca1ff700 1 -- 172.21.15.119:0/2873010544 shutdown_connections 2023-12-07T10:17:57.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.571+0000 7f04ca1ff700 1 --2- 172.21.15.119:0/2873010544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04c4103150 0x7f04c41a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:57.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.571+0000 7f04ca1ff700 1 --2- 172.21.15.119:0/2873010544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04c41015c0 0x7f04c41997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:57.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.571+0000 7f04ca1ff700 1 --2- 172.21.15.119:0/2873010544 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f04ac0a1b10 0x7f04ac0a3fd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:57.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.571+0000 7f04ca1ff700 1 --2- 172.21.15.119:0/2873010544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04c41027b0 0x7f04c4199d30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:57.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.571+0000 7f04ca1ff700 1 -- 172.21.15.119:0/2873010544 >> 172.21.15.119:0/2873010544 conn(0x7f04c40fcb10 msgr2=0x7f04c4107fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:57.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.572+0000 7f04ca1ff700 1 -- 172.21.15.119:0/2873010544 shutdown_connections 2023-12-07T10:17:57.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.572+0000 7f04ca1ff700 1 -- 172.21.15.119:0/2873010544 wait complete. 2023-12-07T10:17:57.592 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-07T10:17:57.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.793+0000 7faf411ff700 1 -- 172.21.15.119:0/3222696681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf3c1015c0 msgr2=0x7faf3c1019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:57.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.793+0000 7faf411ff700 1 --2- 172.21.15.119:0/3222696681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf3c1015c0 0x7faf3c1019e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7faf2c0099c0 tx=0x7faf2c009cd0 comp rx=0 tx=0).stop 2023-12-07T10:17:57.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.793+0000 7faf411ff700 1 -- 172.21.15.119:0/3222696681 shutdown_connections 2023-12-07T10:17:57.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.793+0000 7faf411ff700 1 --2- 172.21.15.119:0/3222696681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf3c103150 0x7faf3c109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:57.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.793+0000 7faf411ff700 1 --2- 172.21.15.119:0/3222696681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf3c1015c0 0x7faf3c1019e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:57.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.793+0000 7faf411ff700 1 --2- 172.21.15.119:0/3222696681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf3c1027b0 0x7faf3c102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:57.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.793+0000 7faf411ff700 1 -- 172.21.15.119:0/3222696681 >> 172.21.15.119:0/3222696681 conn(0x7faf3c0fcb10 msgr2=0x7faf3c0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:57.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.793+0000 7faf411ff700 1 -- 172.21.15.119:0/3222696681 shutdown_connections 2023-12-07T10:17:57.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.793+0000 7faf411ff700 1 -- 172.21.15.119:0/3222696681 wait complete. 2023-12-07T10:17:57.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.794+0000 7faf411ff700 1 Processor -- start 2023-12-07T10:17:57.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.794+0000 7faf411ff700 1 -- start start 2023-12-07T10:17:57.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.794+0000 7faf411ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf3c1015c0 0x7faf3c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:57.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.794+0000 7faf411ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf3c1027b0 0x7faf3c199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:57.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.794+0000 7faf411ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf3c103150 0x7faf3c1a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:57.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.794+0000 7faf411ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c1a1290 con 0x7faf3c1027b0 2023-12-07T10:17:57.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.794+0000 7faf411ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c1a13d0 con 0x7faf3c1015c0 2023-12-07T10:17:57.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.794+0000 7faf411ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c1a1560 con 0x7faf3c103150 2023-12-07T10:17:57.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.795+0000 7faf33fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf3c1027b0 0x7faf3c199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:57.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.795+0000 7faf33fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf3c1027b0 0x7faf3c199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40028/0 (socket says 172.21.15.119:40028) 2023-12-07T10:17:57.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.795+0000 7faf33fff700 1 -- 172.21.15.119:0/4212177360 learned_addr learned my addr 172.21.15.119:0/4212177360 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:57.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.795+0000 7faf3ad9d700 1 --2- 172.21.15.119:0/4212177360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf3c1015c0 0x7faf3c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:57.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.795+0000 7faf3b59e700 1 --2- 172.21.15.119:0/4212177360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf3c103150 0x7faf3c1a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:57.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.795+0000 7faf33fff700 1 -- 172.21.15.119:0/4212177360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf3c103150 msgr2=0x7faf3c1a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:57.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.795+0000 7faf33fff700 1 --2- 172.21.15.119:0/4212177360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf3c103150 0x7faf3c1a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:57.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.795+0000 7faf33fff700 1 -- 172.21.15.119:0/4212177360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf3c1015c0 msgr2=0x7faf3c1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:57.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.795+0000 7faf33fff700 1 --2- 172.21.15.119:0/4212177360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf3c1015c0 0x7faf3c1997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:57.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.795+0000 7faf33fff700 1 -- 172.21.15.119:0/4212177360 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf2c0095f0 con 0x7faf3c1027b0 2023-12-07T10:17:57.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.795+0000 7faf3b59e700 1 --2- 172.21.15.119:0/4212177360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf3c103150 0x7faf3c1a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:17:57.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.795+0000 7faf3ad9d700 1 --2- 172.21.15.119:0/4212177360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf3c1015c0 0x7faf3c1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:17:57.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.795+0000 7faf33fff700 1 --2- 172.21.15.119:0/4212177360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf3c1027b0 0x7faf3c199cf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faf2400e9a0 tx=0x7faf2400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:57.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.796+0000 7faf38d99700 1 -- 172.21.15.119:0/4212177360 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf24009790 con 0x7faf3c1027b0 2023-12-07T10:17:57.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.796+0000 7faf38d99700 1 -- 172.21.15.119:0/4212177360 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf2400cbd0 con 0x7faf3c1027b0 2023-12-07T10:17:57.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.796+0000 7faf38d99700 1 -- 172.21.15.119:0/4212177360 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf24018680 con 0x7faf3c1027b0 2023-12-07T10:17:57.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.796+0000 7faf411ff700 1 -- 172.21.15.119:0/4212177360 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf3c1a1840 con 0x7faf3c1027b0 2023-12-07T10:17:57.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.796+0000 7faf411ff700 1 -- 172.21.15.119:0/4212177360 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3c1a1d00 con 0x7faf3c1027b0 2023-12-07T10:17:57.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.797+0000 7faf411ff700 1 -- 172.21.15.119:0/4212177360 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3c0683c0 con 0x7faf3c1027b0 2023-12-07T10:17:57.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.799+0000 7faf38d99700 1 -- 172.21.15.119:0/4212177360 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7faf2400cd40 con 0x7faf3c1027b0 2023-12-07T10:17:57.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.801+0000 7faf38d99700 1 --2- 172.21.15.119:0/4212177360 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7faf280a1b90 0x7faf280a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:57.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.801+0000 7faf38d99700 1 -- 172.21.15.119:0/4212177360 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7faf24014070 con 0x7faf3c1027b0 2023-12-07T10:17:57.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.801+0000 7faf3ad9d700 1 --2- 172.21.15.119:0/4212177360 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7faf280a1b90 0x7faf280a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:57.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.802+0000 7faf3ad9d700 1 --2- 172.21.15.119:0/4212177360 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7faf280a1b90 0x7faf280a4050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009990 tx=0x7faf2c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:57.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:57.805+0000 7faf38d99700 1 -- 172.21.15.119:0/4212177360 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7faf240522f0 con 0x7faf3c1027b0 2023-12-07T10:17:57.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:57 smithi119 ceph-mon[97908]: pgmap v287: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 0 op/s 2023-12-07T10:17:57.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:57 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2193739385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:17:57.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:57 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2873010544' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:17:58.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.010+0000 7faf411ff700 1 -- 172.21.15.119:0/4212177360 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7faf3c1a2020 con 0x7faf3c1027b0 2023-12-07T10:17:58.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.010+0000 7faf38d99700 1 -- 172.21.15.119:0/4212177360 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7faf24096bc0 con 0x7faf3c1027b0 2023-12-07T10:17:58.013 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-12-07T10:17:58.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.013+0000 7faf411ff700 1 -- 172.21.15.119:0/4212177360 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7faf280a1b90 msgr2=0x7faf280a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:58.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.013+0000 7faf411ff700 1 --2- 172.21.15.119:0/4212177360 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7faf280a1b90 0x7faf280a4050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009990 tx=0x7faf2c01a040 comp rx=0 tx=0).stop 2023-12-07T10:17:58.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.013+0000 7faf411ff700 1 -- 172.21.15.119:0/4212177360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf3c1027b0 msgr2=0x7faf3c199cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:58.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.013+0000 7faf411ff700 1 --2- 172.21.15.119:0/4212177360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf3c1027b0 0x7faf3c199cf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faf2400e9a0 tx=0x7faf2400ecb0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.013+0000 7faf411ff700 1 -- 172.21.15.119:0/4212177360 shutdown_connections 2023-12-07T10:17:58.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.013+0000 7faf411ff700 1 --2- 172.21.15.119:0/4212177360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf3c103150 0x7faf3c1a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.013+0000 7faf411ff700 1 --2- 172.21.15.119:0/4212177360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf3c1015c0 0x7faf3c1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.013+0000 7faf411ff700 1 --2- 172.21.15.119:0/4212177360 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7faf280a1b90 0x7faf280a4050 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.013+0000 7faf411ff700 1 --2- 172.21.15.119:0/4212177360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf3c1027b0 0x7faf3c199cf0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.013+0000 7faf411ff700 1 -- 172.21.15.119:0/4212177360 >> 172.21.15.119:0/4212177360 conn(0x7faf3c0fcb10 msgr2=0x7faf3c107fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:58.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.013+0000 7faf411ff700 1 -- 172.21.15.119:0/4212177360 shutdown_connections 2023-12-07T10:17:58.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.013+0000 7faf411ff700 1 -- 172.21.15.119:0/4212177360 wait complete. 2023-12-07T10:17:58.034 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-07T10:17:58.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.230+0000 7fd2599ff700 1 -- 172.21.15.119:0/2524858863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd254103770 msgr2=0x7fd254103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:58.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.230+0000 7fd2599ff700 1 --2- 172.21.15.119:0/2524858863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd254103770 0x7fd254103b90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd2440099c0 tx=0x7fd244009cd0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.230+0000 7fd2599ff700 1 -- 172.21.15.119:0/2524858863 shutdown_connections 2023-12-07T10:17:58.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.230+0000 7fd2599ff700 1 --2- 172.21.15.119:0/2524858863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd254105300 0x7fd25410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.230+0000 7fd2599ff700 1 --2- 172.21.15.119:0/2524858863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd254103770 0x7fd254103b90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.230+0000 7fd2599ff700 1 --2- 172.21.15.119:0/2524858863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd254104960 0x7fd254104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.230+0000 7fd2599ff700 1 -- 172.21.15.119:0/2524858863 >> 172.21.15.119:0/2524858863 conn(0x7fd2540fece0 msgr2=0x7fd254101120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:58.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.230+0000 7fd2599ff700 1 -- 172.21.15.119:0/2524858863 shutdown_connections 2023-12-07T10:17:58.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.230+0000 7fd2599ff700 1 -- 172.21.15.119:0/2524858863 wait complete. 2023-12-07T10:17:58.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.231+0000 7fd2599ff700 1 Processor -- start 2023-12-07T10:17:58.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.231+0000 7fd2599ff700 1 -- start start 2023-12-07T10:17:58.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.231+0000 7fd2599ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd254103770 0x7fd254199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:58.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.231+0000 7fd2599ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd254104960 0x7fd254199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:58.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.232+0000 7fd2599ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd254105300 0x7fd2541a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:58.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.232+0000 7fd2599ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2541a1230 con 0x7fd254104960 2023-12-07T10:17:58.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.232+0000 7fd2599ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2541a1370 con 0x7fd254103770 2023-12-07T10:17:58.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.232+0000 7fd2599ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2541a1500 con 0x7fd254105300 2023-12-07T10:17:58.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.232+0000 7fd24a7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd254104960 0x7fd254199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:58.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.232+0000 7fd24a7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd254104960 0x7fd254199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40048/0 (socket says 172.21.15.119:40048) 2023-12-07T10:17:58.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.232+0000 7fd24a7fc700 1 -- 172.21.15.119:0/730188305 learned_addr learned my addr 172.21.15.119:0/730188305 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:58.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.232+0000 7fd252ffd700 1 --2- 172.21.15.119:0/730188305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd254103770 0x7fd254199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:58.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.232+0000 7fd2537fe700 1 --2- 172.21.15.119:0/730188305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd254105300 0x7fd2541a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:58.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.232+0000 7fd24a7fc700 1 -- 172.21.15.119:0/730188305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd254105300 msgr2=0x7fd2541a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:58.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.232+0000 7fd24a7fc700 1 --2- 172.21.15.119:0/730188305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd254105300 0x7fd2541a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.232+0000 7fd24a7fc700 1 -- 172.21.15.119:0/730188305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd254103770 msgr2=0x7fd254199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:58.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.232+0000 7fd24a7fc700 1 --2- 172.21.15.119:0/730188305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd254103770 0x7fd254199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.232+0000 7fd24a7fc700 1 -- 172.21.15.119:0/730188305 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2440095f0 con 0x7fd254104960 2023-12-07T10:17:58.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.232+0000 7fd2537fe700 1 --2- 172.21.15.119:0/730188305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd254105300 0x7fd2541a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:17:58.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.232+0000 7fd24a7fc700 1 --2- 172.21.15.119:0/730188305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd254104960 0x7fd254199c40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00e9a0 tx=0x7fd23c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:58.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.233+0000 7fd2589fd700 1 -- 172.21.15.119:0/730188305 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c009790 con 0x7fd254104960 2023-12-07T10:17:58.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.233+0000 7fd2599ff700 1 -- 172.21.15.119:0/730188305 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2541a17e0 con 0x7fd254104960 2023-12-07T10:17:58.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.233+0000 7fd2589fd700 1 -- 172.21.15.119:0/730188305 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd23c00cbd0 con 0x7fd254104960 2023-12-07T10:17:58.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.233+0000 7fd2589fd700 1 -- 172.21.15.119:0/730188305 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c018680 con 0x7fd254104960 2023-12-07T10:17:58.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.233+0000 7fd2599ff700 1 -- 172.21.15.119:0/730188305 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2541a1cf0 con 0x7fd254104960 2023-12-07T10:17:58.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.234+0000 7fd2599ff700 1 -- 172.21.15.119:0/730188305 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd25410ab70 con 0x7fd254104960 2023-12-07T10:17:58.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.236+0000 7fd2589fd700 1 -- 172.21.15.119:0/730188305 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fd23c00cd40 con 0x7fd254104960 2023-12-07T10:17:58.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.237+0000 7fd2589fd700 1 --2- 172.21.15.119:0/730188305 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd2340a1b10 0x7fd2340a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:58.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.237+0000 7fd2589fd700 1 -- 172.21.15.119:0/730188305 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd23c014070 con 0x7fd254104960 2023-12-07T10:17:58.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.237+0000 7fd252ffd700 1 --2- 172.21.15.119:0/730188305 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd2340a1b10 0x7fd2340a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:58.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.238+0000 7fd252ffd700 1 --2- 172.21.15.119:0/730188305 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd2340a1b10 0x7fd2340a3fd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd244003730 tx=0x7fd244000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:58.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.239+0000 7fd2589fd700 1 -- 172.21.15.119:0/730188305 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd23c096f10 con 0x7fd254104960 2023-12-07T10:17:58.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:57 smithi159 ceph-mon[134726]: pgmap v287: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 0 op/s 2023-12-07T10:17:58.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:57 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2193739385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:17:58.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:57 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2873010544' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:17:58.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:57 smithi153 ceph-mon[133625]: pgmap v287: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 0 op/s 2023-12-07T10:17:58.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:57 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2193739385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:17:58.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:57 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2873010544' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:17:58.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.454+0000 7fd2599ff700 1 -- 172.21.15.119:0/730188305 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd254068320 con 0x7fd254104960 2023-12-07T10:17:58.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.455+0000 7fd2589fd700 1 -- 172.21.15.119:0/730188305 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) v1 ==== 81+0+600 (secure 0 0 0) 0x7fd23c09a530 con 0x7fd254104960 2023-12-07T10:17:58.458 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/1628639392 2023-12-08T10:11:00.585565+0000 2023-12-07T10:17:58.458 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6801/1355617847 2023-12-08T10:11:00.585565+0000 2023-12-07T10:17:58.458 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6800/3515858087 2023-12-08T10:10:32.177930+0000 2023-12-07T10:17:58.458 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/362307836 2023-12-08T10:10:32.177930+0000 2023-12-07T10:17:58.458 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/128964558 2023-12-08T10:11:00.585565+0000 2023-12-07T10:17:58.458 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/2278813554 2023-12-08T10:10:32.177930+0000 2023-12-07T10:17:58.458 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6801/3515858087 2023-12-08T10:10:32.177930+0000 2023-12-07T10:17:58.458 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/1411933614 2023-12-08T10:11:00.585565+0000 2023-12-07T10:17:58.459 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/1685698277 2023-12-08T10:10:32.177930+0000 2023-12-07T10:17:58.459 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6800/1355617847 2023-12-08T10:11:00.585565+0000 2023-12-07T10:17:58.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.457+0000 7fd2599ff700 1 -- 172.21.15.119:0/730188305 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd2340a1b10 msgr2=0x7fd2340a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:58.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.457+0000 7fd2599ff700 1 --2- 172.21.15.119:0/730188305 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd2340a1b10 0x7fd2340a3fd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd244003730 tx=0x7fd244000bc0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.457+0000 7fd2599ff700 1 -- 172.21.15.119:0/730188305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd254104960 msgr2=0x7fd254199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:58.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.457+0000 7fd2599ff700 1 --2- 172.21.15.119:0/730188305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd254104960 0x7fd254199c40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00e9a0 tx=0x7fd23c00ecb0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.457+0000 7fd2599ff700 1 -- 172.21.15.119:0/730188305 shutdown_connections 2023-12-07T10:17:58.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.457+0000 7fd2599ff700 1 --2- 172.21.15.119:0/730188305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd254105300 0x7fd2541a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.457+0000 7fd2599ff700 1 --2- 172.21.15.119:0/730188305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd254103770 0x7fd254199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.457+0000 7fd2599ff700 1 --2- 172.21.15.119:0/730188305 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd2340a1b10 0x7fd2340a3fd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.457+0000 7fd2599ff700 1 --2- 172.21.15.119:0/730188305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd254104960 0x7fd254199c40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.457+0000 7fd2599ff700 1 -- 172.21.15.119:0/730188305 >> 172.21.15.119:0/730188305 conn(0x7fd2540fece0 msgr2=0x7fd254107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:58.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.458+0000 7fd2599ff700 1 -- 172.21.15.119:0/730188305 shutdown_connections 2023-12-07T10:17:58.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.458+0000 7fd2599ff700 1 -- 172.21.15.119:0/730188305 wait complete. 2023-12-07T10:17:58.461 INFO:teuthology.orchestra.run.smithi119.stderr:listed 10 entries 2023-12-07T10:17:58.478 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-12-07T10:17:58.478 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-07T10:17:58.522 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-07T10:17:58.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.755+0000 7f1d04fff700 1 -- 172.21.15.119:0/2817637247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d00102750 msgr2=0x7f1d00102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:58.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.755+0000 7f1d04fff700 1 --2- 172.21.15.119:0/2817637247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d00102750 0x7f1d00102bb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40099c0 tx=0x7f1cf4009cd0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.755+0000 7f1d04fff700 1 -- 172.21.15.119:0/2817637247 shutdown_connections 2023-12-07T10:17:58.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.755+0000 7f1d04fff700 1 --2- 172.21.15.119:0/2817637247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d001030f0 0x7f1d001099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.755+0000 7f1d04fff700 1 --2- 172.21.15.119:0/2817637247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d00101560 0x7f1d00101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.755+0000 7f1d04fff700 1 --2- 172.21.15.119:0/2817637247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d00102750 0x7f1d00102bb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.755+0000 7f1d04fff700 1 -- 172.21.15.119:0/2817637247 >> 172.21.15.119:0/2817637247 conn(0x7f1d000fcab0 msgr2=0x7f1d000fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:58.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.755+0000 7f1d04fff700 1 -- 172.21.15.119:0/2817637247 shutdown_connections 2023-12-07T10:17:58.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.755+0000 7f1d04fff700 1 -- 172.21.15.119:0/2817637247 wait complete. 2023-12-07T10:17:58.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.756+0000 7f1d04fff700 1 Processor -- start 2023-12-07T10:17:58.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.756+0000 7f1d04fff700 1 -- start start 2023-12-07T10:17:58.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.757+0000 7f1d04fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d00101560 0x7f1d00075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:58.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.757+0000 7f1d04fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d00102750 0x7f1d00073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:58.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.757+0000 7f1d04fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d001030f0 0x7f1d00073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:58.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.757+0000 7f1d04fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d000746e0 con 0x7f1d00101560 2023-12-07T10:17:58.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.757+0000 7f1d04fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d00074850 con 0x7f1d001030f0 2023-12-07T10:17:58.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.757+0000 7f1d04fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d001a1520 con 0x7f1d00102750 2023-12-07T10:17:58.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.757+0000 7f1cfe59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d00101560 0x7f1d00075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:58.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.757+0000 7f1cfe59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d00101560 0x7f1d00075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40064/0 (socket says 172.21.15.119:40064) 2023-12-07T10:17:58.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.757+0000 7f1cfdd9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d00102750 0x7f1d00073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:58.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.757+0000 7f1cfe59c700 1 -- 172.21.15.119:0/1107066863 learned_addr learned my addr 172.21.15.119:0/1107066863 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:58.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.757+0000 7f1cfed9d700 1 --2- 172.21.15.119:0/1107066863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d001030f0 0x7f1d00073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:58.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.757+0000 7f1cfe59c700 1 -- 172.21.15.119:0/1107066863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d00102750 msgr2=0x7f1d00073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:58.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.757+0000 7f1cfe59c700 1 --2- 172.21.15.119:0/1107066863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d00102750 0x7f1d00073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.757+0000 7f1cfe59c700 1 -- 172.21.15.119:0/1107066863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d001030f0 msgr2=0x7f1d00073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:58.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.757+0000 7f1cfe59c700 1 --2- 172.21.15.119:0/1107066863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d001030f0 0x7f1d00073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.757+0000 7f1cfe59c700 1 -- 172.21.15.119:0/1107066863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf0009520 con 0x7f1d00101560 2023-12-07T10:17:58.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.757+0000 7f1cfed9d700 1 --2- 172.21.15.119:0/1107066863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d001030f0 0x7f1d00073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:17:58.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.758+0000 7f1cfe59c700 1 --2- 172.21.15.119:0/1107066863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d00101560 0x7f1d00075370 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000ea50 tx=0x7f1cf000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:58.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.758+0000 7f1cef7fe700 1 -- 172.21.15.119:0/1107066863 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf000cb70 con 0x7f1d00101560 2023-12-07T10:17:58.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.758+0000 7f1cef7fe700 1 -- 172.21.15.119:0/1107066863 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cf000ccd0 con 0x7f1d00101560 2023-12-07T10:17:58.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.758+0000 7f1d04fff700 1 -- 172.21.15.119:0/1107066863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf40095f0 con 0x7f1d00101560 2023-12-07T10:17:58.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.758+0000 7f1cef7fe700 1 -- 172.21.15.119:0/1107066863 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0018680 con 0x7f1d00101560 2023-12-07T10:17:58.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.758+0000 7f1d04fff700 1 -- 172.21.15.119:0/1107066863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d001a1b80 con 0x7f1d00101560 2023-12-07T10:17:58.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.759+0000 7f1d04fff700 1 -- 172.21.15.119:0/1107066863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d001089e0 con 0x7f1d00101560 2023-12-07T10:17:58.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.761+0000 7f1cef7fe700 1 -- 172.21.15.119:0/1107066863 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f1cf0010430 con 0x7f1d00101560 2023-12-07T10:17:58.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.762+0000 7f1cef7fe700 1 --2- 172.21.15.119:0/1107066863 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1ce80a1b90 0x7f1ce80a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:58.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.762+0000 7f1cef7fe700 1 -- 172.21.15.119:0/1107066863 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1cf0014070 con 0x7f1d00101560 2023-12-07T10:17:58.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.762+0000 7f1cfdd9b700 1 --2- 172.21.15.119:0/1107066863 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1ce80a1b90 0x7f1ce80a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:58.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.763+0000 7f1cfdd9b700 1 --2- 172.21.15.119:0/1107066863 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1ce80a1b90 0x7f1ce80a4050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4009990 tx=0x7f1cf4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:58.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.766+0000 7f1cef7fe700 1 -- 172.21.15.119:0/1107066863 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1cf0096ef0 con 0x7f1d00101560 2023-12-07T10:17:58.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.971+0000 7f1d04fff700 1 -- 172.21.15.119:0/1107066863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d00068320 con 0x7f1d00101560 2023-12-07T10:17:58.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.973+0000 7f1cef7fe700 1 -- 172.21.15.119:0/1107066863 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24030 (secure 0 0 0) 0x7f1cf009a510 con 0x7f1d00101560 2023-12-07T10:17:58.976 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:17:58.976 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":89,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","created":"2023-12-07T10:09:59.945863+0000","modified":"2023-12-07T10:17:46.277123+0000","last_up_change":"2023-12-07T10:16:50.929427+0000","last_in_change":"2023-12-07T10:16:33.558158+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T10:13:25.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-07T10:17:39.347813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-07T10:17:40.714501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-07T10:17:44.257760+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"56b5818e-e625-40f1-8838-3ddcf2d34ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6803","nonce":2141950133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6805","nonce":2141950133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6809","nonce":2141950133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6807","nonce":2141950133}]},"public_addr":"172.21.15.119:6803/2141950133","cluster_addr":"172.21.15.119:6805/2141950133","heartbeat_back_addr":"172.21.15.119:6809/2141950133","heartbeat_front_addr":"172.21.15.119:6807/2141950133","state":["exists","up"]},{"osd":1,"uuid":"36489a6a-5278-4802-9525-9871521722e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6801","nonce":3390875370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6803","nonce":3390875370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6807","nonce":3390875370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6805","nonce":3390875370}]},"public_addr":"172.21.15.153:6801/3390875370","cluster_addr":"172.21.15.153:6803/3390875370","heartbeat_back_addr":"172.21.15.153:6807/3390875370","heartbeat_front_addr":"172.21.15.153:6805/3390875370","state":["exists","up"]},{"osd":2,"uuid":"a937ad26-5e9b-4c8a-b65a-626a8feda6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6801","nonce":708101637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6803","nonce":708101637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6807","nonce":708101637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6805","nonce":708101637}]},"public_addr":"172.21.15.159:6801/708101637","cluster_addr":"172.21.15.159:6803/708101637","heartbeat_back_addr":"172.21.15.159:6807/708101637","heartbeat_front_addr":"172.21.15.159:6805/708101637","state":["exists","up"]},{"osd":3,"uuid":"26ef8300-aa07-48f6-b31d-8b0f2217ffcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6811","nonce":3228047257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6813","nonce":3228047257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6817","nonce":3228047257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6815","nonce":3228047257}]},"public_addr":"172.21.15.119:6811/3228047257","cluster_addr":"172.21.15.119:6813/3228047257","heartbeat_back_addr":"172.21.15.119:6817/3228047257","heartbeat_front_addr":"172.21.15.119:6815/3228047257","state":["exists","up"]},{"osd":4,"uuid":"000ee4b9-9896-4901-8cc3-c85fdfc8cbaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6809","nonce":4021998299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6811","nonce":4021998299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6815","nonce":4021998299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6813","nonce":4021998299}]},"public_addr":"172.21.15.153:6809/4021998299","cluster_addr":"172.21.15.153:6811/4021998299","heartbeat_back_addr":"172.21.15.153:6815/4021998299","heartbeat_front_addr":"172.21.15.153:6813/4021998299","state":["exists","up"]},{"osd":5,"uuid":"8b2fc167-262d-474f-89f3-47bd29424421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6809","nonce":1832540768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6811","nonce":1832540768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6815","nonce":1832540768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6813","nonce":1832540768}]},"public_addr":"172.21.15.159:6809/1832540768","cluster_addr":"172.21.15.159:6811/1832540768","heartbeat_back_addr":"172.21.15.159:6815/1832540768","heartbeat_front_addr":"172.21.15.159:6813/1832540768","state":["exists","up"]},{"osd":6,"uuid":"f9aa6749-46cf-4b2a-8114-cbd034219d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162436840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162436840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6825","nonce":1162436840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162436840}]},"public_addr":"172.21.15.119:6819/1162436840","cluster_addr":"172.21.15.119:6821/1162436840","heartbeat_back_addr":"172.21.15.119:6825/1162436840","heartbeat_front_addr":"172.21.15.119:6823/1162436840","state":["exists","up"]},{"osd":7,"uuid":"39216361-e1c4-4fef-86cb-a4707656b547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6817","nonce":3254653457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6819","nonce":3254653457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6823","nonce":3254653457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6821","nonce":3254653457}]},"public_addr":"172.21.15.153:6817/3254653457","cluster_addr":"172.21.15.153:6819/3254653457","heartbeat_back_addr":"172.21.15.153:6823/3254653457","heartbeat_front_addr":"172.21.15.153:6821/3254653457","state":["exists","up"]},{"osd":8,"uuid":"cceb6ebc-7c02-48ea-9d16-b7a016dd449b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6817","nonce":2600946380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6819","nonce":2600946380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6823","nonce":2600946380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6821","nonce":2600946380}]},"public_addr":"172.21.15.159:6817/2600946380","cluster_addr":"172.21.15.159:6819/2600946380","heartbeat_back_addr":"172.21.15.159:6823/2600946380","heartbeat_front_addr":"172.21.15.159:6821/2600946380","state":["exists","up"]},{"osd":9,"uuid":"f337a2e4-e065-4a9f-91d9-e3615de2d6f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6827","nonce":1940737285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6829","nonce":1940737285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6833","nonce":1940737285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6831","nonce":1940737285}]},"public_addr":"172.21.15.119:6827/1940737285","cluster_addr":"172.21.15.119:6829/1940737285","heartbeat_back_addr":"172.21.15.119:6833/1940737285","heartbeat_front_addr":"172.21.15.119:6831/1940737285","state":["exists","up"]},{"osd":10,"uuid":"3a3bc347-85f4-4878-9797-d9da96137e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6825","nonce":663834864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6827","nonce":663834864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6831","nonce":663834864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6829","nonce":663834864}]},"public_addr":"172.21.15.153:6825/663834864","cluster_addr":"172.21.15.153:6827/663834864","heartbeat_back_addr":"172.21.15.153:6831/663834864","heartbeat_front_addr":"172.21.15.153:6829/663834864","state":["exists","up"]},{"osd":11,"uuid":"a0fc5909-279d-4172-977c-2dc2da0a7e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6825","nonce":1262216348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6827","nonce":1262216348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6831","nonce":1262216348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6829","nonce":1262216348}]},"public_addr":"172.21.15.159:6825/1262216348","cluster_addr":"172.21.15.159:6827/1262216348","heartbeat_back_addr":"172.21.15.159:6831/1262216348","heartbeat_front_addr":"172.21.15.159:6829/1262216348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:37.488109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:59.543060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:21.381585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:47.112729+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:09.732150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:30.893729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:56.678710+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:19.004256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:40.388818+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:06.391734+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:28.394053+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:49.634348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1628639392":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6801/1355617847":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6800/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/362307836":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/128964558":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/2278813554":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/1411933614":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/1685698277":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6800/1355617847":"2023-12-08T10:11:00.585565+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T10:17:58.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.975+0000 7f1d04fff700 1 -- 172.21.15.119:0/1107066863 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1ce80a1b90 msgr2=0x7f1ce80a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:58.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.975+0000 7f1d04fff700 1 --2- 172.21.15.119:0/1107066863 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1ce80a1b90 0x7f1ce80a4050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4009990 tx=0x7f1cf4000bc0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.975+0000 7f1d04fff700 1 -- 172.21.15.119:0/1107066863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d00101560 msgr2=0x7f1d00075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:58.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.975+0000 7f1d04fff700 1 --2- 172.21.15.119:0/1107066863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d00101560 0x7f1d00075370 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000ea50 tx=0x7f1cf000ed60 comp rx=0 tx=0).stop 2023-12-07T10:17:58.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.975+0000 7f1d04fff700 1 -- 172.21.15.119:0/1107066863 shutdown_connections 2023-12-07T10:17:58.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.975+0000 7f1d04fff700 1 --2- 172.21.15.119:0/1107066863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d001030f0 0x7f1d00073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.975+0000 7f1d04fff700 1 --2- 172.21.15.119:0/1107066863 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1ce80a1b90 0x7f1ce80a4050 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.975+0000 7f1d04fff700 1 --2- 172.21.15.119:0/1107066863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d00101560 0x7f1d00075370 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.975+0000 7f1d04fff700 1 --2- 172.21.15.119:0/1107066863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d00102750 0x7f1d00073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:58.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.975+0000 7f1d04fff700 1 -- 172.21.15.119:0/1107066863 >> 172.21.15.119:0/1107066863 conn(0x7f1d000fcab0 msgr2=0x7f1d00105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:58.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.975+0000 7f1d04fff700 1 -- 172.21.15.119:0/1107066863 shutdown_connections 2023-12-07T10:17:58.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:58.975+0000 7f1d04fff700 1 -- 172.21.15.119:0/1107066863 wait complete. 2023-12-07T10:17:58.997 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-07T10:13:25.194366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-07T10:17:39.347813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-07T10:17:40.714501+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-07T10:17:44.257760+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-07T10:17:58.998 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-07T10:17:59.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.194+0000 7f7170dff700 1 -- 172.21.15.119:0/714658946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f716c1015c0 msgr2=0x7f716c1019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:59.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.194+0000 7f7170dff700 1 --2- 172.21.15.119:0/714658946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f716c1015c0 0x7f716c1019e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f715c009990 tx=0x7f715c009ca0 comp rx=0 tx=0).stop 2023-12-07T10:17:59.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.194+0000 7f7170dff700 1 -- 172.21.15.119:0/714658946 shutdown_connections 2023-12-07T10:17:59.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.194+0000 7f7170dff700 1 --2- 172.21.15.119:0/714658946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f716c103150 0x7f716c109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:59.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.194+0000 7f7170dff700 1 --2- 172.21.15.119:0/714658946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f716c1015c0 0x7f716c1019e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:59.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.194+0000 7f7170dff700 1 --2- 172.21.15.119:0/714658946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f716c1027b0 0x7f716c102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:59.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.194+0000 7f7170dff700 1 -- 172.21.15.119:0/714658946 >> 172.21.15.119:0/714658946 conn(0x7f716c0fcb10 msgr2=0x7f716c0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:59.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.195+0000 7f7170dff700 1 -- 172.21.15.119:0/714658946 shutdown_connections 2023-12-07T10:17:59.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.195+0000 7f7170dff700 1 -- 172.21.15.119:0/714658946 wait complete. 2023-12-07T10:17:59.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.195+0000 7f7170dff700 1 Processor -- start 2023-12-07T10:17:59.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.196+0000 7f7170dff700 1 -- start start 2023-12-07T10:17:59.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.196+0000 7f7170dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f716c1015c0 0x7f716c075320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:59.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.196+0000 7f7170dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f716c1027b0 0x7f716c073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.196+0000 7f7170dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f716c103150 0x7f716c073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.196+0000 7f7170dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c074640 con 0x7f716c1015c0 2023-12-07T10:17:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.196+0000 7f7170dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c0747b0 con 0x7f716c1027b0 2023-12-07T10:17:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.196+0000 7f7170dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c1a15d0 con 0x7f716c103150 2023-12-07T10:17:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.196+0000 7f716a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f716c1015c0 0x7f716c075320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.196+0000 7f7169d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f716c1027b0 0x7f716c073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.196+0000 7f7169d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f716c1027b0 0x7f716c073a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:40248/0 (socket says 172.21.15.119:40248) 2023-12-07T10:17:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.196+0000 7f716a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f716c1015c0 0x7f716c075320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40090/0 (socket says 172.21.15.119:40090) 2023-12-07T10:17:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.196+0000 7f7169d9b700 1 -- 172.21.15.119:0/2950355290 learned_addr learned my addr 172.21.15.119:0/2950355290 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.197+0000 7f716ad9d700 1 --2- 172.21.15.119:0/2950355290 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f716c103150 0x7f716c073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.197+0000 7f7169d9b700 1 -- 172.21.15.119:0/2950355290 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f716c103150 msgr2=0x7f716c073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.197+0000 7f7169d9b700 1 --2- 172.21.15.119:0/2950355290 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f716c103150 0x7f716c073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.197+0000 7f7169d9b700 1 -- 172.21.15.119:0/2950355290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f716c1015c0 msgr2=0x7f716c075320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.197+0000 7f7169d9b700 1 --2- 172.21.15.119:0/2950355290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f716c1015c0 0x7f716c075320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.197+0000 7f7169d9b700 1 -- 172.21.15.119:0/2950355290 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715c0095f0 con 0x7f716c1027b0 2023-12-07T10:17:59.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.197+0000 7f7169d9b700 1 --2- 172.21.15.119:0/2950355290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f716c1027b0 0x7f716c073a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f716000e9a0 tx=0x7f716000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:59.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.197+0000 7f715b7fe700 1 -- 172.21.15.119:0/2950355290 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160009790 con 0x7f716c1027b0 2023-12-07T10:17:59.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.197+0000 7f715b7fe700 1 -- 172.21.15.119:0/2950355290 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f716000cbd0 con 0x7f716c1027b0 2023-12-07T10:17:59.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.197+0000 7f7170dff700 1 -- 172.21.15.119:0/2950355290 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716c1a1830 con 0x7f716c1027b0 2023-12-07T10:17:59.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.197+0000 7f715b7fe700 1 -- 172.21.15.119:0/2950355290 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160018680 con 0x7f716c1027b0 2023-12-07T10:17:59.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.197+0000 7f7170dff700 1 -- 172.21.15.119:0/2950355290 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f716c1a1d40 con 0x7f716c1027b0 2023-12-07T10:17:59.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.198+0000 7f7170dff700 1 -- 172.21.15.119:0/2950355290 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f716c108a90 con 0x7f716c1027b0 2023-12-07T10:17:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.200+0000 7f715b7fe700 1 -- 172.21.15.119:0/2950355290 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f716000cd40 con 0x7f716c1027b0 2023-12-07T10:17:59.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.202+0000 7f715b7fe700 1 --2- 172.21.15.119:0/2950355290 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f71540a1b10 0x7f71540a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:59.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.202+0000 7f715b7fe700 1 -- 172.21.15.119:0/2950355290 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7160014070 con 0x7f716c1027b0 2023-12-07T10:17:59.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.202+0000 7f716a59c700 1 --2- 172.21.15.119:0/2950355290 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f71540a1b10 0x7f71540a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:59.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.202+0000 7f716a59c700 1 --2- 172.21.15.119:0/2950355290 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f71540a1b10 0x7f71540a3fd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f715c00b680 tx=0x7f715c00b750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:59.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.204+0000 7f715b7fe700 1 -- 172.21.15.119:0/2950355290 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7160096f10 con 0x7f716c1027b0 2023-12-07T10:17:59.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:58 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/4212177360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:17:59.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:58 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/730188305' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-07T10:17:59.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:58 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/4212177360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:17:59.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:58 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/730188305' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-07T10:17:59.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:58 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/4212177360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:17:59.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:58 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/730188305' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-07T10:17:59.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.403+0000 7f7170dff700 1 -- 172.21.15.119:0/2950355290 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f716c10cdf0 con 0x7f716c1027b0 2023-12-07T10:17:59.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.404+0000 7f715b7fe700 1 -- 172.21.15.119:0/2950355290 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f716009a530 con 0x7f716c1027b0 2023-12-07T10:17:59.407 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-12-07T10:17:59.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.406+0000 7f7170dff700 1 -- 172.21.15.119:0/2950355290 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f71540a1b10 msgr2=0x7f71540a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:59.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.406+0000 7f7170dff700 1 --2- 172.21.15.119:0/2950355290 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f71540a1b10 0x7f71540a3fd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f715c00b680 tx=0x7f715c00b750 comp rx=0 tx=0).stop 2023-12-07T10:17:59.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.406+0000 7f7170dff700 1 -- 172.21.15.119:0/2950355290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f716c1027b0 msgr2=0x7f716c073a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:59.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.406+0000 7f7170dff700 1 --2- 172.21.15.119:0/2950355290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f716c1027b0 0x7f716c073a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f716000e9a0 tx=0x7f716000ecb0 comp rx=0 tx=0).stop 2023-12-07T10:17:59.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.406+0000 7f7170dff700 1 -- 172.21.15.119:0/2950355290 shutdown_connections 2023-12-07T10:17:59.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.406+0000 7f7170dff700 1 --2- 172.21.15.119:0/2950355290 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f716c103150 0x7f716c073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:59.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.406+0000 7f7170dff700 1 --2- 172.21.15.119:0/2950355290 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f71540a1b10 0x7f71540a3fd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:59.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.406+0000 7f7170dff700 1 --2- 172.21.15.119:0/2950355290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f716c1015c0 0x7f716c075320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:59.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.406+0000 7f7170dff700 1 --2- 172.21.15.119:0/2950355290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f716c1027b0 0x7f716c073a00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:59.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.407+0000 7f7170dff700 1 -- 172.21.15.119:0/2950355290 >> 172.21.15.119:0/2950355290 conn(0x7f716c0fcb10 msgr2=0x7f716c105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:59.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.407+0000 7f7170dff700 1 -- 172.21.15.119:0/2950355290 shutdown_connections 2023-12-07T10:17:59.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.407+0000 7f7170dff700 1 -- 172.21.15.119:0/2950355290 wait complete. 2023-12-07T10:17:59.428 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-07T10:17:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.625+0000 7f8e2dbff700 1 -- 172.21.15.119:0/1188616141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e281015c0 msgr2=0x7f8e281019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.625+0000 7f8e2dbff700 1 --2- 172.21.15.119:0/1188616141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e281015c0 0x7f8e281019e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8e180099c0 tx=0x7f8e18009cd0 comp rx=0 tx=0).stop 2023-12-07T10:17:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.625+0000 7f8e2dbff700 1 -- 172.21.15.119:0/1188616141 shutdown_connections 2023-12-07T10:17:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.625+0000 7f8e2dbff700 1 --2- 172.21.15.119:0/1188616141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e28103150 0x7f8e28109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.625+0000 7f8e2dbff700 1 --2- 172.21.15.119:0/1188616141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e281015c0 0x7f8e281019e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.625+0000 7f8e2dbff700 1 --2- 172.21.15.119:0/1188616141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e281027b0 0x7f8e28102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.625+0000 7f8e2dbff700 1 -- 172.21.15.119:0/1188616141 >> 172.21.15.119:0/1188616141 conn(0x7f8e280fcb10 msgr2=0x7f8e280fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.625+0000 7f8e2dbff700 1 -- 172.21.15.119:0/1188616141 shutdown_connections 2023-12-07T10:17:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.625+0000 7f8e2dbff700 1 -- 172.21.15.119:0/1188616141 wait complete. 2023-12-07T10:17:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.626+0000 7f8e2dbff700 1 Processor -- start 2023-12-07T10:17:59.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.626+0000 7f8e2dbff700 1 -- start start 2023-12-07T10:17:59.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.627+0000 7f8e2dbff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e281015c0 0x7f8e28199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:59.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.627+0000 7f8e2dbff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e281027b0 0x7f8e28199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:59.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.627+0000 7f8e2dbff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e28103150 0x7f8e281a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:59.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.627+0000 7f8e2dbff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e281a12f0 con 0x7f8e281027b0 2023-12-07T10:17:59.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.627+0000 7f8e2dbff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e281a1430 con 0x7f8e281015c0 2023-12-07T10:17:59.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.627+0000 7f8e2dbff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e281a15c0 con 0x7f8e28103150 2023-12-07T10:17:59.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.627+0000 7f8e26ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e281027b0 0x7f8e28199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:59.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.627+0000 7f8e26ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e281027b0 0x7f8e28199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40118/0 (socket says 172.21.15.119:40118) 2023-12-07T10:17:59.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.627+0000 7f8e26ffd700 1 -- 172.21.15.119:0/398963150 learned_addr learned my addr 172.21.15.119:0/398963150 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:17:59.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.627+0000 7f8e277fe700 1 --2- 172.21.15.119:0/398963150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e281015c0 0x7f8e28199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:59.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.627+0000 7f8e27fff700 1 --2- 172.21.15.119:0/398963150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e28103150 0x7f8e281a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:59.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.627+0000 7f8e26ffd700 1 -- 172.21.15.119:0/398963150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e28103150 msgr2=0x7f8e281a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:59.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.627+0000 7f8e26ffd700 1 --2- 172.21.15.119:0/398963150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e28103150 0x7f8e281a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:59.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.627+0000 7f8e26ffd700 1 -- 172.21.15.119:0/398963150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e281015c0 msgr2=0x7f8e28199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:59.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.627+0000 7f8e26ffd700 1 --2- 172.21.15.119:0/398963150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e281015c0 0x7f8e28199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:59.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.627+0000 7f8e26ffd700 1 -- 172.21.15.119:0/398963150 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e180095f0 con 0x7f8e281027b0 2023-12-07T10:17:59.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.627+0000 7f8e26ffd700 1 --2- 172.21.15.119:0/398963150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e281027b0 0x7f8e28199d50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00e9a0 tx=0x7f8e1c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:59.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.628+0000 7f8e24ff9700 1 -- 172.21.15.119:0/398963150 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c009790 con 0x7f8e281027b0 2023-12-07T10:17:59.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.628+0000 7f8e24ff9700 1 -- 172.21.15.119:0/398963150 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e1c00cbd0 con 0x7f8e281027b0 2023-12-07T10:17:59.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.628+0000 7f8e2dbff700 1 -- 172.21.15.119:0/398963150 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e281a18a0 con 0x7f8e281027b0 2023-12-07T10:17:59.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.628+0000 7f8e24ff9700 1 -- 172.21.15.119:0/398963150 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c018680 con 0x7f8e281027b0 2023-12-07T10:17:59.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.628+0000 7f8e2dbff700 1 -- 172.21.15.119:0/398963150 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e281a1e00 con 0x7f8e281027b0 2023-12-07T10:17:59.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.629+0000 7f8e2dbff700 1 -- 172.21.15.119:0/398963150 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e280683c0 con 0x7f8e281027b0 2023-12-07T10:17:59.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.631+0000 7f8e24ff9700 1 -- 172.21.15.119:0/398963150 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f8e1c00cd40 con 0x7f8e281027b0 2023-12-07T10:17:59.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.632+0000 7f8e24ff9700 1 --2- 172.21.15.119:0/398963150 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8e100a1b90 0x7f8e100a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:17:59.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.632+0000 7f8e24ff9700 1 -- 172.21.15.119:0/398963150 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8e1c014070 con 0x7f8e281027b0 2023-12-07T10:17:59.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.632+0000 7f8e277fe700 1 --2- 172.21.15.119:0/398963150 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8e100a1b90 0x7f8e100a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:17:59.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.633+0000 7f8e277fe700 1 --2- 172.21.15.119:0/398963150 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8e100a1b90 0x7f8e100a4050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8e18009990 tx=0x7f8e1801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:17:59.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.636+0000 7f8e24ff9700 1 -- 172.21.15.119:0/398963150 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8e1c0522f0 con 0x7f8e281027b0 2023-12-07T10:17:59.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.842+0000 7f8e2dbff700 1 -- 172.21.15.119:0/398963150 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8e281a2090 con 0x7f8e281027b0 2023-12-07T10:17:59.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.842+0000 7f8e24ff9700 1 -- 172.21.15.119:0/398963150 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8e1c096bc0 con 0x7f8e281027b0 2023-12-07T10:17:59.846 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-12-07T10:17:59.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.845+0000 7f8e2dbff700 1 -- 172.21.15.119:0/398963150 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8e100a1b90 msgr2=0x7f8e100a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:59.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.845+0000 7f8e2dbff700 1 --2- 172.21.15.119:0/398963150 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8e100a1b90 0x7f8e100a4050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8e18009990 tx=0x7f8e1801a040 comp rx=0 tx=0).stop 2023-12-07T10:17:59.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.845+0000 7f8e2dbff700 1 -- 172.21.15.119:0/398963150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e281027b0 msgr2=0x7f8e28199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:17:59.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.845+0000 7f8e2dbff700 1 --2- 172.21.15.119:0/398963150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e281027b0 0x7f8e28199d50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00e9a0 tx=0x7f8e1c00ecb0 comp rx=0 tx=0).stop 2023-12-07T10:17:59.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.845+0000 7f8e2dbff700 1 -- 172.21.15.119:0/398963150 shutdown_connections 2023-12-07T10:17:59.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.845+0000 7f8e2dbff700 1 --2- 172.21.15.119:0/398963150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e28103150 0x7f8e281a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:59.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.845+0000 7f8e2dbff700 1 --2- 172.21.15.119:0/398963150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e281015c0 0x7f8e28199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:59.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.845+0000 7f8e2dbff700 1 --2- 172.21.15.119:0/398963150 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8e100a1b90 0x7f8e100a4050 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:59.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.845+0000 7f8e2dbff700 1 --2- 172.21.15.119:0/398963150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e281027b0 0x7f8e28199d50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:17:59.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.845+0000 7f8e2dbff700 1 -- 172.21.15.119:0/398963150 >> 172.21.15.119:0/398963150 conn(0x7f8e280fcb10 msgr2=0x7f8e28073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:17:59.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.845+0000 7f8e2dbff700 1 -- 172.21.15.119:0/398963150 shutdown_connections 2023-12-07T10:17:59.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:17:59.845+0000 7f8e2dbff700 1 -- 172.21.15.119:0/398963150 wait complete. 2023-12-07T10:17:59.866 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-07T10:18:00.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.063+0000 7f09a23ff700 1 -- 172.21.15.119:0/3376453116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f099c1015c0 msgr2=0x7f099c1019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:00.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.063+0000 7f09a23ff700 1 --2- 172.21.15.119:0/3376453116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f099c1015c0 0x7f099c1019e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f098c0099c0 tx=0x7f098c009cd0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.063+0000 7f09a23ff700 1 -- 172.21.15.119:0/3376453116 shutdown_connections 2023-12-07T10:18:00.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.063+0000 7f09a23ff700 1 --2- 172.21.15.119:0/3376453116 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f099c103150 0x7f099c109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.063+0000 7f09a23ff700 1 --2- 172.21.15.119:0/3376453116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f099c1015c0 0x7f099c1019e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.063+0000 7f09a23ff700 1 --2- 172.21.15.119:0/3376453116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f099c1027b0 0x7f099c102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.063+0000 7f09a23ff700 1 -- 172.21.15.119:0/3376453116 >> 172.21.15.119:0/3376453116 conn(0x7f099c0fcb10 msgr2=0x7f099c0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:00.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.063+0000 7f09a23ff700 1 -- 172.21.15.119:0/3376453116 shutdown_connections 2023-12-07T10:18:00.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.063+0000 7f09a23ff700 1 -- 172.21.15.119:0/3376453116 wait complete. 2023-12-07T10:18:00.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.064+0000 7f09a23ff700 1 Processor -- start 2023-12-07T10:18:00.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.064+0000 7f09a23ff700 1 -- start start 2023-12-07T10:18:00.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.064+0000 7f09a23ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f099c1015c0 0x7f099c199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:00.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.064+0000 7f09a23ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f099c1027b0 0x7f099c199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:00.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.064+0000 7f09a23ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f099c103150 0x7f099c1a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:00.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.064+0000 7f09a23ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c1a12b0 con 0x7f099c1015c0 2023-12-07T10:18:00.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.064+0000 7f09a23ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c1a13f0 con 0x7f099c103150 2023-12-07T10:18:00.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.064+0000 7f09a23ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c1a1580 con 0x7f099c1027b0 2023-12-07T10:18:00.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.064+0000 7f099bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f099c1015c0 0x7f099c199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:00.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.064+0000 7f099bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f099c1015c0 0x7f099c199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40140/0 (socket says 172.21.15.119:40140) 2023-12-07T10:18:00.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.064+0000 7f099bfff700 1 -- 172.21.15.119:0/3225345954 learned_addr learned my addr 172.21.15.119:0/3225345954 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:00.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.064+0000 7f0993fff700 1 --2- 172.21.15.119:0/3225345954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f099c1027b0 0x7f099c199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:00.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.064+0000 7f09a099c700 1 --2- 172.21.15.119:0/3225345954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f099c103150 0x7f099c1a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:00.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.065+0000 7f099bfff700 1 -- 172.21.15.119:0/3225345954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f099c1027b0 msgr2=0x7f099c199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:00.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.065+0000 7f099bfff700 1 --2- 172.21.15.119:0/3225345954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f099c1027b0 0x7f099c199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.065+0000 7f099bfff700 1 -- 172.21.15.119:0/3225345954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f099c103150 msgr2=0x7f099c1a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:00.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.065+0000 7f099bfff700 1 --2- 172.21.15.119:0/3225345954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f099c103150 0x7f099c1a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.065+0000 7f099bfff700 1 -- 172.21.15.119:0/3225345954 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098c0095f0 con 0x7f099c1015c0 2023-12-07T10:18:00.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.065+0000 7f099bfff700 1 --2- 172.21.15.119:0/3225345954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f099c1015c0 0x7f099c199780 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f098c006010 tx=0x7f098c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:00.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.065+0000 7f0999ffb700 1 -- 172.21.15.119:0/3225345954 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c01d070 con 0x7f099c1015c0 2023-12-07T10:18:00.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.065+0000 7f0999ffb700 1 -- 172.21.15.119:0/3225345954 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f098c003f90 con 0x7f099c1015c0 2023-12-07T10:18:00.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.065+0000 7f0999ffb700 1 -- 172.21.15.119:0/3225345954 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c022cb0 con 0x7f099c1015c0 2023-12-07T10:18:00.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.065+0000 7f09a23ff700 1 -- 172.21.15.119:0/3225345954 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f099c1a1800 con 0x7f099c1015c0 2023-12-07T10:18:00.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.065+0000 7f09a23ff700 1 -- 172.21.15.119:0/3225345954 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f099c1a1bd0 con 0x7f099c1015c0 2023-12-07T10:18:00.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.066+0000 7f09a23ff700 1 -- 172.21.15.119:0/3225345954 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f099c0683c0 con 0x7f099c1015c0 2023-12-07T10:18:00.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.068+0000 7f0999ffb700 1 -- 172.21.15.119:0/3225345954 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f098c022470 con 0x7f099c1015c0 2023-12-07T10:18:00.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.070+0000 7f0999ffb700 1 --2- 172.21.15.119:0/3225345954 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f09880a1b90 0x7f09880a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:00.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.070+0000 7f0999ffb700 1 -- 172.21.15.119:0/3225345954 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f098c0cf270 con 0x7f099c1015c0 2023-12-07T10:18:00.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.070+0000 7f0993fff700 1 --2- 172.21.15.119:0/3225345954 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f09880a1b90 0x7f09880a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:00.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.070+0000 7f0993fff700 1 --2- 172.21.15.119:0/3225345954 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f09880a1b90 0x7f09880a4050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0984007c60 tx=0x7f09840073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:00.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.072+0000 7f0999ffb700 1 -- 172.21.15.119:0/3225345954 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f098c0984d0 con 0x7f099c1015c0 2023-12-07T10:18:00.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:59 smithi119 ceph-mon[97908]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 86 B/s wr, 0 op/s 2023-12-07T10:18:00.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:59 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1107066863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:18:00.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:17:59 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2950355290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:18:00.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:59 smithi159 ceph-mon[134726]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 86 B/s wr, 0 op/s 2023-12-07T10:18:00.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:59 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1107066863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:18:00.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:17:59 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2950355290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:18:00.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.274+0000 7f09a23ff700 1 -- 172.21.15.119:0/3225345954 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f099c10d090 con 0x7f099c1015c0 2023-12-07T10:18:00.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.274+0000 7f0999ffb700 1 -- 172.21.15.119:0/3225345954 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f098c0270a0 con 0x7f099c1015c0 2023-12-07T10:18:00.277 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-12-07T10:18:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.277+0000 7f09a23ff700 1 -- 172.21.15.119:0/3225345954 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f09880a1b90 msgr2=0x7f09880a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.277+0000 7f09a23ff700 1 --2- 172.21.15.119:0/3225345954 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f09880a1b90 0x7f09880a4050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0984007c60 tx=0x7f09840073d0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.277+0000 7f09a23ff700 1 -- 172.21.15.119:0/3225345954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f099c1015c0 msgr2=0x7f099c199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.277+0000 7f09a23ff700 1 --2- 172.21.15.119:0/3225345954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f099c1015c0 0x7f099c199780 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f098c006010 tx=0x7f098c005740 comp rx=0 tx=0).stop 2023-12-07T10:18:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.277+0000 7f09a23ff700 1 -- 172.21.15.119:0/3225345954 shutdown_connections 2023-12-07T10:18:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.277+0000 7f09a23ff700 1 --2- 172.21.15.119:0/3225345954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f099c103150 0x7f099c1a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.277+0000 7f09a23ff700 1 --2- 172.21.15.119:0/3225345954 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f09880a1b90 0x7f09880a4050 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.277+0000 7f09a23ff700 1 --2- 172.21.15.119:0/3225345954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f099c1015c0 0x7f099c199780 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.277+0000 7f09a23ff700 1 --2- 172.21.15.119:0/3225345954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f099c1027b0 0x7f099c199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.277+0000 7f09a23ff700 1 -- 172.21.15.119:0/3225345954 >> 172.21.15.119:0/3225345954 conn(0x7f099c0fcb10 msgr2=0x7f099c073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:00.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.277+0000 7f09a23ff700 1 -- 172.21.15.119:0/3225345954 shutdown_connections 2023-12-07T10:18:00.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.277+0000 7f09a23ff700 1 -- 172.21.15.119:0/3225345954 wait complete. 2023-12-07T10:18:00.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:59 smithi153 ceph-mon[133625]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 86 B/s wr, 0 op/s 2023-12-07T10:18:00.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:59 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1107066863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:18:00.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:17:59 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2950355290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:18:00.298 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-07T10:18:00.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.494+0000 7f1bfe7ff700 1 -- 172.21.15.119:0/169191820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bf81027b0 msgr2=0x7f1bf8102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:00.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.494+0000 7f1bfe7ff700 1 --2- 172.21.15.119:0/169191820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bf81027b0 0x7f1bf8102c10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1bec0099c0 tx=0x7f1bec009cd0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.494+0000 7f1bfe7ff700 1 -- 172.21.15.119:0/169191820 shutdown_connections 2023-12-07T10:18:00.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.494+0000 7f1bfe7ff700 1 --2- 172.21.15.119:0/169191820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1bf8103150 0x7f1bf8109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.494+0000 7f1bfe7ff700 1 --2- 172.21.15.119:0/169191820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bf81015c0 0x7f1bf81019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.494+0000 7f1bfe7ff700 1 --2- 172.21.15.119:0/169191820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bf81027b0 0x7f1bf8102c10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.494+0000 7f1bfe7ff700 1 -- 172.21.15.119:0/169191820 >> 172.21.15.119:0/169191820 conn(0x7f1bf80fcb10 msgr2=0x7f1bf80fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:00.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.494+0000 7f1bfe7ff700 1 -- 172.21.15.119:0/169191820 shutdown_connections 2023-12-07T10:18:00.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.494+0000 7f1bfe7ff700 1 -- 172.21.15.119:0/169191820 wait complete. 2023-12-07T10:18:00.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.495+0000 7f1bfe7ff700 1 Processor -- start 2023-12-07T10:18:00.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.495+0000 7f1bfe7ff700 1 -- start start 2023-12-07T10:18:00.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.495+0000 7f1bfe7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1bf81015c0 0x7f1bf81996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:00.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.495+0000 7f1bfe7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bf81027b0 0x7f1bf8199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:00.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.495+0000 7f1bfe7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bf8103150 0x7f1bf81a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:00.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.495+0000 7f1bfe7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf81a11f0 con 0x7f1bf8103150 2023-12-07T10:18:00.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.495+0000 7f1bfe7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf81a1330 con 0x7f1bf81027b0 2023-12-07T10:18:00.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.495+0000 7f1bfe7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf81a14c0 con 0x7f1bf81015c0 2023-12-07T10:18:00.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.496+0000 7f1bfcd9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bf8103150 0x7f1bf81a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:00.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.496+0000 7f1bf7fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1bf81015c0 0x7f1bf81996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:00.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.496+0000 7f1bf77fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bf81027b0 0x7f1bf8199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:00.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.496+0000 7f1bfcd9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bf8103150 0x7f1bf81a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40156/0 (socket says 172.21.15.119:40156) 2023-12-07T10:18:00.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.496+0000 7f1bfcd9c700 1 -- 172.21.15.119:0/804504957 learned_addr learned my addr 172.21.15.119:0/804504957 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:00.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.496+0000 7f1bfcd9c700 1 -- 172.21.15.119:0/804504957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1bf81015c0 msgr2=0x7f1bf81996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:00.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.496+0000 7f1bfcd9c700 1 --2- 172.21.15.119:0/804504957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1bf81015c0 0x7f1bf81996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.496+0000 7f1bfcd9c700 1 -- 172.21.15.119:0/804504957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bf81027b0 msgr2=0x7f1bf8199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:00.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.496+0000 7f1bfcd9c700 1 --2- 172.21.15.119:0/804504957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bf81027b0 0x7f1bf8199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.496+0000 7f1bfcd9c700 1 -- 172.21.15.119:0/804504957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bec0095f0 con 0x7f1bf8103150 2023-12-07T10:18:00.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.496+0000 7f1bfcd9c700 1 --2- 172.21.15.119:0/804504957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bf8103150 0x7f1bf81a0c20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000ef30 tx=0x7f1bf000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:00.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.497+0000 7f1bf57fa700 1 -- 172.21.15.119:0/804504957 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0018070 con 0x7f1bf8103150 2023-12-07T10:18:00.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.497+0000 7f1bf57fa700 1 -- 172.21.15.119:0/804504957 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bf000f040 con 0x7f1bf8103150 2023-12-07T10:18:00.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.497+0000 7f1bfe7ff700 1 -- 172.21.15.119:0/804504957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bf81a1740 con 0x7f1bf8103150 2023-12-07T10:18:00.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.497+0000 7f1bf57fa700 1 -- 172.21.15.119:0/804504957 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf000cd00 con 0x7f1bf8103150 2023-12-07T10:18:00.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.497+0000 7f1bfe7ff700 1 -- 172.21.15.119:0/804504957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bf81a1c00 con 0x7f1bf8103150 2023-12-07T10:18:00.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.498+0000 7f1bfe7ff700 1 -- 172.21.15.119:0/804504957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bf80683c0 con 0x7f1bf8103150 2023-12-07T10:18:00.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.500+0000 7f1bf57fa700 1 -- 172.21.15.119:0/804504957 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f1bf0003a00 con 0x7f1bf8103150 2023-12-07T10:18:00.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.501+0000 7f1bf57fa700 1 --2- 172.21.15.119:0/804504957 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1be00a1b10 0x7f1be00a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:00.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.501+0000 7f1bf57fa700 1 -- 172.21.15.119:0/804504957 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1bf00cd360 con 0x7f1bf8103150 2023-12-07T10:18:00.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.501+0000 7f1bf7fff700 1 --2- 172.21.15.119:0/804504957 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1be00a1b10 0x7f1be00a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:00.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.502+0000 7f1bf7fff700 1 --2- 172.21.15.119:0/804504957 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1be00a1b10 0x7f1be00a3fd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8102610 tx=0x7f1be8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:00.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.505+0000 7f1bf57fa700 1 -- 172.21.15.119:0/804504957 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1bf00965c0 con 0x7f1bf8103150 2023-12-07T10:18:00.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.708+0000 7f1bfe7ff700 1 -- 172.21.15.119:0/804504957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1bf81a1ef0 con 0x7f1bf8103150 2023-12-07T10:18:00.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.709+0000 7f1bf57fa700 1 -- 172.21.15.119:0/804504957 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1bf0099be0 con 0x7f1bf8103150 2023-12-07T10:18:00.712 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-12-07T10:18:00.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.711+0000 7f1bfe7ff700 1 -- 172.21.15.119:0/804504957 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1be00a1b10 msgr2=0x7f1be00a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:00.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.711+0000 7f1bfe7ff700 1 --2- 172.21.15.119:0/804504957 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1be00a1b10 0x7f1be00a3fd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8102610 tx=0x7f1be8009260 comp rx=0 tx=0).stop 2023-12-07T10:18:00.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.711+0000 7f1bfe7ff700 1 -- 172.21.15.119:0/804504957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bf8103150 msgr2=0x7f1bf81a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:00.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.711+0000 7f1bfe7ff700 1 --2- 172.21.15.119:0/804504957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bf8103150 0x7f1bf81a0c20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000ef30 tx=0x7f1bf000c3a0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.711+0000 7f1bfe7ff700 1 -- 172.21.15.119:0/804504957 shutdown_connections 2023-12-07T10:18:00.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.711+0000 7f1bfe7ff700 1 --2- 172.21.15.119:0/804504957 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1be00a1b10 0x7f1be00a3fd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.711+0000 7f1bfe7ff700 1 --2- 172.21.15.119:0/804504957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bf8103150 0x7f1bf81a0c20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.711+0000 7f1bfe7ff700 1 --2- 172.21.15.119:0/804504957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1bf81015c0 0x7f1bf81996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.711+0000 7f1bfe7ff700 1 --2- 172.21.15.119:0/804504957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bf81027b0 0x7f1bf8199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.711+0000 7f1bfe7ff700 1 -- 172.21.15.119:0/804504957 >> 172.21.15.119:0/804504957 conn(0x7f1bf80fcb10 msgr2=0x7f1bf8073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:00.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.711+0000 7f1bfe7ff700 1 -- 172.21.15.119:0/804504957 shutdown_connections 2023-12-07T10:18:00.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.711+0000 7f1bfe7ff700 1 -- 172.21.15.119:0/804504957 wait complete. 2023-12-07T10:18:00.734 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-07T10:18:00.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.937+0000 7f63925ff700 1 -- 172.21.15.119:0/1705107297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f638c101540 msgr2=0x7f638c101960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:00.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.937+0000 7f63925ff700 1 --2- 172.21.15.119:0/1705107297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f638c101540 0x7f638c101960 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f637c0099c0 tx=0x7f637c009cd0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.937+0000 7f63925ff700 1 -- 172.21.15.119:0/1705107297 shutdown_connections 2023-12-07T10:18:00.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.937+0000 7f63925ff700 1 --2- 172.21.15.119:0/1705107297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f638c1030d0 0x7f638c109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.937+0000 7f63925ff700 1 --2- 172.21.15.119:0/1705107297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f638c101540 0x7f638c101960 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.937+0000 7f63925ff700 1 --2- 172.21.15.119:0/1705107297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f638c102730 0x7f638c102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.937+0000 7f63925ff700 1 -- 172.21.15.119:0/1705107297 >> 172.21.15.119:0/1705107297 conn(0x7f638c0fcab0 msgr2=0x7f638c0feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:00.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.937+0000 7f63925ff700 1 -- 172.21.15.119:0/1705107297 shutdown_connections 2023-12-07T10:18:00.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.937+0000 7f63925ff700 1 -- 172.21.15.119:0/1705107297 wait complete. 2023-12-07T10:18:00.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.938+0000 7f63925ff700 1 Processor -- start 2023-12-07T10:18:00.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.938+0000 7f63925ff700 1 -- start start 2023-12-07T10:18:00.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.939+0000 7f63925ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f638c101540 0x7f638c075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:00.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.939+0000 7f63925ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f638c102730 0x7f638c073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:00.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.939+0000 7f63925ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f638c1030d0 0x7f638c073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:00.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.939+0000 7f63925ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c0746e0 con 0x7f638c102730 2023-12-07T10:18:00.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.939+0000 7f63925ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c074850 con 0x7f638c1030d0 2023-12-07T10:18:00.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.939+0000 7f63925ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c1a1510 con 0x7f638c101540 2023-12-07T10:18:00.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.939+0000 7f638b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f638c102730 0x7f638c073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:00.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.939+0000 7f638bfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f638c101540 0x7f638c075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:00.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.939+0000 7f638b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f638c102730 0x7f638c073a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40186/0 (socket says 172.21.15.119:40186) 2023-12-07T10:18:00.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.939+0000 7f638b7fe700 1 -- 172.21.15.119:0/510134659 learned_addr learned my addr 172.21.15.119:0/510134659 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:00.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.939+0000 7f6390b9c700 1 --2- 172.21.15.119:0/510134659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f638c1030d0 0x7f638c073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:00.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.939+0000 7f638b7fe700 1 -- 172.21.15.119:0/510134659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f638c101540 msgr2=0x7f638c075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:00.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.939+0000 7f638b7fe700 1 --2- 172.21.15.119:0/510134659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f638c101540 0x7f638c075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.939+0000 7f638b7fe700 1 -- 172.21.15.119:0/510134659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f638c1030d0 msgr2=0x7f638c073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:00.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.939+0000 7f638b7fe700 1 --2- 172.21.15.119:0/510134659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f638c1030d0 0x7f638c073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:00.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.939+0000 7f638b7fe700 1 -- 172.21.15.119:0/510134659 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637c0095f0 con 0x7f638c102730 2023-12-07T10:18:00.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.940+0000 7f638b7fe700 1 --2- 172.21.15.119:0/510134659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f638c102730 0x7f638c073a00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f638000e9a0 tx=0x7f638000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:00.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.940+0000 7f63897fa700 1 -- 172.21.15.119:0/510134659 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6380009790 con 0x7f638c102730 2023-12-07T10:18:00.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.940+0000 7f63897fa700 1 -- 172.21.15.119:0/510134659 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f638000ce70 con 0x7f638c102730 2023-12-07T10:18:00.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.940+0000 7f63897fa700 1 -- 172.21.15.119:0/510134659 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6380018420 con 0x7f638c102730 2023-12-07T10:18:00.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.940+0000 7f63925ff700 1 -- 172.21.15.119:0/510134659 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f638c1a17a0 con 0x7f638c102730 2023-12-07T10:18:00.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.940+0000 7f63925ff700 1 -- 172.21.15.119:0/510134659 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f638c1a1cb0 con 0x7f638c102730 2023-12-07T10:18:00.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.941+0000 7f63925ff700 1 -- 172.21.15.119:0/510134659 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f638c108a10 con 0x7f638c102730 2023-12-07T10:18:00.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.943+0000 7f63897fa700 1 -- 172.21.15.119:0/510134659 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f63800106f0 con 0x7f638c102730 2023-12-07T10:18:00.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.945+0000 7f63897fa700 1 --2- 172.21.15.119:0/510134659 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f63740a1b90 0x7f63740a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:00.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.945+0000 7f63897fa700 1 -- 172.21.15.119:0/510134659 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f6380014070 con 0x7f638c102730 2023-12-07T10:18:00.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.945+0000 7f638bfff700 1 --2- 172.21.15.119:0/510134659 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f63740a1b90 0x7f63740a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:00.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.946+0000 7f638bfff700 1 --2- 172.21.15.119:0/510134659 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f63740a1b90 0x7f63740a4050 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f637c003730 tx=0x7f637c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:00.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:00.949+0000 7f63897fa700 1 -- 172.21.15.119:0/510134659 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f63800522f0 con 0x7f638c102730 2023-12-07T10:18:01.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:01.167+0000 7f63925ff700 1 -- 172.21.15.119:0/510134659 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f638c02d350 con 0x7f638c102730 2023-12-07T10:18:01.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:01.168+0000 7f63897fa700 1 -- 172.21.15.119:0/510134659 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) v1 ==== 81+0+600 (secure 0 0 0) 0x7f6380096ab0 con 0x7f638c102730 2023-12-07T10:18:01.171 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/1628639392 2023-12-08T10:11:00.585565+0000 2023-12-07T10:18:01.171 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6801/1355617847 2023-12-08T10:11:00.585565+0000 2023-12-07T10:18:01.171 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6800/3515858087 2023-12-08T10:10:32.177930+0000 2023-12-07T10:18:01.171 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/362307836 2023-12-08T10:10:32.177930+0000 2023-12-07T10:18:01.171 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/128964558 2023-12-08T10:11:00.585565+0000 2023-12-07T10:18:01.171 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/2278813554 2023-12-08T10:10:32.177930+0000 2023-12-07T10:18:01.171 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6801/3515858087 2023-12-08T10:10:32.177930+0000 2023-12-07T10:18:01.171 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/1411933614 2023-12-08T10:11:00.585565+0000 2023-12-07T10:18:01.171 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/1685698277 2023-12-08T10:10:32.177930+0000 2023-12-07T10:18:01.171 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6800/1355617847 2023-12-08T10:11:00.585565+0000 2023-12-07T10:18:01.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:01.170+0000 7f63925ff700 1 -- 172.21.15.119:0/510134659 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f63740a1b90 msgr2=0x7f63740a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:01.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:01.170+0000 7f63925ff700 1 --2- 172.21.15.119:0/510134659 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f63740a1b90 0x7f63740a4050 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f637c003730 tx=0x7f637c000bc0 comp rx=0 tx=0).stop 2023-12-07T10:18:01.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:01.170+0000 7f63925ff700 1 -- 172.21.15.119:0/510134659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f638c102730 msgr2=0x7f638c073a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:01.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:01.170+0000 7f63925ff700 1 --2- 172.21.15.119:0/510134659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f638c102730 0x7f638c073a00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f638000e9a0 tx=0x7f638000ecb0 comp rx=0 tx=0).stop 2023-12-07T10:18:01.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:01.170+0000 7f63925ff700 1 -- 172.21.15.119:0/510134659 shutdown_connections 2023-12-07T10:18:01.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:01.170+0000 7f63925ff700 1 --2- 172.21.15.119:0/510134659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f638c1030d0 0x7f638c073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:01.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:01.170+0000 7f63925ff700 1 --2- 172.21.15.119:0/510134659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f638c101540 0x7f638c075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:01.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:01.170+0000 7f63925ff700 1 --2- 172.21.15.119:0/510134659 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f63740a1b90 0x7f63740a4050 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:01.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:01.170+0000 7f63925ff700 1 --2- 172.21.15.119:0/510134659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f638c102730 0x7f638c073a00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:01.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:01.170+0000 7f63925ff700 1 -- 172.21.15.119:0/510134659 >> 172.21.15.119:0/510134659 conn(0x7f638c0fcab0 msgr2=0x7f638c105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:01.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:01.170+0000 7f63925ff700 1 -- 172.21.15.119:0/510134659 shutdown_connections 2023-12-07T10:18:01.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:01.170+0000 7f63925ff700 1 -- 172.21.15.119:0/510134659 wait complete. 2023-12-07T10:18:01.174 INFO:teuthology.orchestra.run.smithi119.stderr:listed 10 entries 2023-12-07T10:18:01.189 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:00 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/398963150' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:18:01.189 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:00 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3225345954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:18:01.190 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:00 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-12-07T10:18:01.190 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:00 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-12-07T10:18:01.190 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:00 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2023-12-07T10:18:01.190 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:00 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]: dispatch 2023-12-07T10:18:01.190 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:00 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2023-12-07T10:18:01.190 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:00 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6, 5, 7]}]: dispatch 2023-12-07T10:18:01.190 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:00 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2023-12-07T10:18:01.190 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:00 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-12-07T10:18:01.190 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:00 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2023-12-07T10:18:01.190 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:00 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/804504957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:18:01.192 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi119.front.sepia.ceph.com... 2023-12-07T10:18:01.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:18:01.192 DEBUG:teuthology.orchestra.run.smithi119:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-12-07T10:18:01.257 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi119.stderr:ceph-fuse[149244]: fuse finished with error 0 and tester_r 0 2023-12-07T10:18:01.260 INFO:teuthology.orchestra.run:waiting for 300 2023-12-07T10:18:01.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:00 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/398963150' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:18:01.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:00 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3225345954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:18:01.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:00 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-12-07T10:18:01.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:00 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-12-07T10:18:01.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:00 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2023-12-07T10:18:01.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:00 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]: dispatch 2023-12-07T10:18:01.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:00 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2023-12-07T10:18:01.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:00 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6, 5, 7]}]: dispatch 2023-12-07T10:18:01.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:00 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2023-12-07T10:18:01.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:00 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-12-07T10:18:01.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:00 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2023-12-07T10:18:01.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:00 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/804504957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:18:01.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:00 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/398963150' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:18:01.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:00 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3225345954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:18:01.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-12-07T10:18:01.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-12-07T10:18:01.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2023-12-07T10:18:01.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]: dispatch 2023-12-07T10:18:01.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2023-12-07T10:18:01.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6, 5, 7]}]: dispatch 2023-12-07T10:18:01.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2023-12-07T10:18:01.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-12-07T10:18:01.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:00 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2023-12-07T10:18:01.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:00 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/804504957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:18:02.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:01 smithi119 ceph-mon[97908]: pgmap v289: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-12-07T10:18:02.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-12-07T10:18:02.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-12-07T10:18:02.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2023-12-07T10:18:02.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]': finished 2023-12-07T10:18:02.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2023-12-07T10:18:02.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6, 5, 7]}]': finished 2023-12-07T10:18:02.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2023-12-07T10:18:02.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-12-07T10:18:02.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2023-12-07T10:18:02.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:01 smithi119 ceph-mon[97908]: osdmap e90: 12 total, 12 up, 12 in 2023-12-07T10:18:02.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:01 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/510134659' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-07T10:18:02.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:01 smithi159 ceph-mon[134726]: pgmap v289: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-12-07T10:18:02.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-12-07T10:18:02.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-12-07T10:18:02.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2023-12-07T10:18:02.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]': finished 2023-12-07T10:18:02.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2023-12-07T10:18:02.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6, 5, 7]}]': finished 2023-12-07T10:18:02.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2023-12-07T10:18:02.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-12-07T10:18:02.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2023-12-07T10:18:02.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:01 smithi159 ceph-mon[134726]: osdmap e90: 12 total, 12 up, 12 in 2023-12-07T10:18:02.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:01 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/510134659' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-07T10:18:02.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:01 smithi153 ceph-mon[133625]: pgmap v289: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-12-07T10:18:02.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-12-07T10:18:02.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-12-07T10:18:02.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2023-12-07T10:18:02.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]': finished 2023-12-07T10:18:02.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2023-12-07T10:18:02.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6, 5, 7]}]': finished 2023-12-07T10:18:02.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2023-12-07T10:18:02.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-12-07T10:18:02.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2023-12-07T10:18:02.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:01 smithi153 ceph-mon[133625]: osdmap e90: 12 total, 12 up, 12 in 2023-12-07T10:18:02.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:01 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/510134659' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-07T10:18:03.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:02 smithi119 ceph-mon[97908]: osdmap e91: 12 total, 12 up, 12 in 2023-12-07T10:18:03.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:02 smithi159 ceph-mon[134726]: osdmap e91: 12 total, 12 up, 12 in 2023-12-07T10:18:03.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:02 smithi153 ceph-mon[133625]: osdmap e91: 12 total, 12 up, 12 in 2023-12-07T10:18:04.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:03 smithi119 ceph-mon[97908]: pgmap v292: 129 pgs: 1 activating, 128 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-12-07T10:18:04.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:03 smithi159 ceph-mon[134726]: pgmap v292: 129 pgs: 1 activating, 128 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-12-07T10:18:04.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:03 smithi153 ceph-mon[133625]: pgmap v292: 129 pgs: 1 activating, 128 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-12-07T10:18:06.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:05 smithi119 ceph-mon[97908]: pgmap v293: 129 pgs: 1 activating, 128 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-12-07T10:18:06.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:05 smithi159 ceph-mon[134726]: pgmap v293: 129 pgs: 1 activating, 128 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-12-07T10:18:06.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:05 smithi153 ceph-mon[133625]: pgmap v293: 129 pgs: 1 activating, 128 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-12-07T10:18:07.362 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-12-07T10:18:07.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:18:07.362 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-12-07T10:18:07.424 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-07T10:18:07.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.623+0000 7fdc661ff700 1 -- 172.21.15.119:0/1577408065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc60107270 msgr2=0x7fdc60109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:07.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.623+0000 7fdc661ff700 1 --2- 172.21.15.119:0/1577408065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc60107270 0x7fdc60109680 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800b0a0 tx=0x7fdc5800b3b0 comp rx=0 tx=0).stop 2023-12-07T10:18:07.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.623+0000 7fdc661ff700 1 -- 172.21.15.119:0/1577408065 shutdown_connections 2023-12-07T10:18:07.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.623+0000 7fdc661ff700 1 --2- 172.21.15.119:0/1577408065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc60107270 0x7fdc60109680 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:07.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.623+0000 7fdc661ff700 1 --2- 172.21.15.119:0/1577408065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc60102010 0x7fdc60104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:07.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.623+0000 7fdc661ff700 1 --2- 172.21.15.119:0/1577408065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc60104940 0x7fdc60106d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:07.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.623+0000 7fdc661ff700 1 -- 172.21.15.119:0/1577408065 >> 172.21.15.119:0/1577408065 conn(0x7fdc600fb9c0 msgr2=0x7fdc600fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:07.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.623+0000 7fdc661ff700 1 -- 172.21.15.119:0/1577408065 shutdown_connections 2023-12-07T10:18:07.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.623+0000 7fdc661ff700 1 -- 172.21.15.119:0/1577408065 wait complete. 2023-12-07T10:18:07.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.624+0000 7fdc661ff700 1 Processor -- start 2023-12-07T10:18:07.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.624+0000 7fdc661ff700 1 -- start start 2023-12-07T10:18:07.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.624+0000 7fdc661ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc60102010 0x7fdc60101930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:07.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.624+0000 7fdc661ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc60104940 0x7fdc600fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:07.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.625+0000 7fdc661ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc60107270 0x7fdc60100500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:07.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.625+0000 7fdc661ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc60100ca0 con 0x7fdc60104940 2023-12-07T10:18:07.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.625+0000 7fdc661ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc60100e10 con 0x7fdc60102010 2023-12-07T10:18:07.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.625+0000 7fdc661ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc601a15e0 con 0x7fdc60107270 2023-12-07T10:18:07.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.625+0000 7fdc57fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc60104940 0x7fdc600fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:07.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.625+0000 7fdc57fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc60104940 0x7fdc600fffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50760/0 (socket says 172.21.15.119:50760) 2023-12-07T10:18:07.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.625+0000 7fdc57fff700 1 -- 172.21.15.119:0/1880573717 learned_addr learned my addr 172.21.15.119:0/1880573717 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:07.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.625+0000 7fdc5f7fe700 1 --2- 172.21.15.119:0/1880573717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc60102010 0x7fdc60101930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:07.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.625+0000 7fdc5ffff700 1 --2- 172.21.15.119:0/1880573717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc60107270 0x7fdc60100500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:07.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.625+0000 7fdc57fff700 1 -- 172.21.15.119:0/1880573717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc60107270 msgr2=0x7fdc60100500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:07.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.625+0000 7fdc57fff700 1 --2- 172.21.15.119:0/1880573717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc60107270 0x7fdc60100500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:07.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.625+0000 7fdc57fff700 1 -- 172.21.15.119:0/1880573717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc60102010 msgr2=0x7fdc60101930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:07.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.625+0000 7fdc57fff700 1 --2- 172.21.15.119:0/1880573717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc60102010 0x7fdc60101930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:07.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.625+0000 7fdc57fff700 1 -- 172.21.15.119:0/1880573717 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc58009d00 con 0x7fdc60104940 2023-12-07T10:18:07.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.625+0000 7fdc5f7fe700 1 --2- 172.21.15.119:0/1880573717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc60102010 0x7fdc60101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:18:07.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.625+0000 7fdc57fff700 1 --2- 172.21.15.119:0/1880573717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc60104940 0x7fdc600fffc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800e9a0 tx=0x7fdc4800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:07.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.626+0000 7fdc5d7fa700 1 -- 172.21.15.119:0/1880573717 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48009790 con 0x7fdc60104940 2023-12-07T10:18:07.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.626+0000 7fdc5d7fa700 1 -- 172.21.15.119:0/1880573717 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc4800cb70 con 0x7fdc60104940 2023-12-07T10:18:07.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.626+0000 7fdc661ff700 1 -- 172.21.15.119:0/1880573717 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc601a1870 con 0x7fdc60104940 2023-12-07T10:18:07.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.626+0000 7fdc5d7fa700 1 -- 172.21.15.119:0/1880573717 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48018680 con 0x7fdc60104940 2023-12-07T10:18:07.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.626+0000 7fdc661ff700 1 -- 172.21.15.119:0/1880573717 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc601a1dd0 con 0x7fdc60104940 2023-12-07T10:18:07.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.627+0000 7fdc661ff700 1 -- 172.21.15.119:0/1880573717 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc600fd540 con 0x7fdc60104940 2023-12-07T10:18:07.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.628+0000 7fdc5d7fa700 1 -- 172.21.15.119:0/1880573717 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fdc4800cce0 con 0x7fdc60104940 2023-12-07T10:18:07.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.630+0000 7fdc5d7fa700 1 --2- 172.21.15.119:0/1880573717 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fdc4c0a1b90 0x7fdc4c0a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:07.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.630+0000 7fdc5d7fa700 1 -- 172.21.15.119:0/1880573717 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fdc48014070 con 0x7fdc60104940 2023-12-07T10:18:07.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.630+0000 7fdc5f7fe700 1 --2- 172.21.15.119:0/1880573717 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fdc4c0a1b90 0x7fdc4c0a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:07.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.631+0000 7fdc5f7fe700 1 --2- 172.21.15.119:0/1880573717 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fdc4c0a1b90 0x7fdc4c0a4050 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009cb0 tx=0x7fdc50009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:07.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.634+0000 7fdc5d7fa700 1 -- 172.21.15.119:0/1880573717 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fdc480522f0 con 0x7fdc60104940 2023-12-07T10:18:07.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.835+0000 7fdc661ff700 1 -- 172.21.15.119:0/1880573717 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc600fcf20 con 0x7fdc60104940 2023-12-07T10:18:07.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.837+0000 7fdc5d7fa700 1 -- 172.21.15.119:0/1880573717 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24528 (secure 0 0 0) 0x7fdc48096c20 con 0x7fdc60104940 2023-12-07T10:18:07.839 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:18:07.839 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":91,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","created":"2023-12-07T10:09:59.945863+0000","modified":"2023-12-07T10:18:01.843682+0000","last_up_change":"2023-12-07T10:16:50.929427+0000","last_in_change":"2023-12-07T10:16:33.558158+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T10:13:25.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-07T10:17:39.347813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-07T10:17:40.714501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-07T10:17:44.257760+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"56b5818e-e625-40f1-8838-3ddcf2d34ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6803","nonce":2141950133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6805","nonce":2141950133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6809","nonce":2141950133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6807","nonce":2141950133}]},"public_addr":"172.21.15.119:6803/2141950133","cluster_addr":"172.21.15.119:6805/2141950133","heartbeat_back_addr":"172.21.15.119:6809/2141950133","heartbeat_front_addr":"172.21.15.119:6807/2141950133","state":["exists","up"]},{"osd":1,"uuid":"36489a6a-5278-4802-9525-9871521722e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6801","nonce":3390875370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6803","nonce":3390875370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6807","nonce":3390875370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6805","nonce":3390875370}]},"public_addr":"172.21.15.153:6801/3390875370","cluster_addr":"172.21.15.153:6803/3390875370","heartbeat_back_addr":"172.21.15.153:6807/3390875370","heartbeat_front_addr":"172.21.15.153:6805/3390875370","state":["exists","up"]},{"osd":2,"uuid":"a937ad26-5e9b-4c8a-b65a-626a8feda6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6801","nonce":708101637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6803","nonce":708101637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6807","nonce":708101637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6805","nonce":708101637}]},"public_addr":"172.21.15.159:6801/708101637","cluster_addr":"172.21.15.159:6803/708101637","heartbeat_back_addr":"172.21.15.159:6807/708101637","heartbeat_front_addr":"172.21.15.159:6805/708101637","state":["exists","up"]},{"osd":3,"uuid":"26ef8300-aa07-48f6-b31d-8b0f2217ffcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6811","nonce":3228047257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6813","nonce":3228047257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6817","nonce":3228047257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6815","nonce":3228047257}]},"public_addr":"172.21.15.119:6811/3228047257","cluster_addr":"172.21.15.119:6813/3228047257","heartbeat_back_addr":"172.21.15.119:6817/3228047257","heartbeat_front_addr":"172.21.15.119:6815/3228047257","state":["exists","up"]},{"osd":4,"uuid":"000ee4b9-9896-4901-8cc3-c85fdfc8cbaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6809","nonce":4021998299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6811","nonce":4021998299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6815","nonce":4021998299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6813","nonce":4021998299}]},"public_addr":"172.21.15.153:6809/4021998299","cluster_addr":"172.21.15.153:6811/4021998299","heartbeat_back_addr":"172.21.15.153:6815/4021998299","heartbeat_front_addr":"172.21.15.153:6813/4021998299","state":["exists","up"]},{"osd":5,"uuid":"8b2fc167-262d-474f-89f3-47bd29424421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6809","nonce":1832540768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6811","nonce":1832540768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6815","nonce":1832540768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6813","nonce":1832540768}]},"public_addr":"172.21.15.159:6809/1832540768","cluster_addr":"172.21.15.159:6811/1832540768","heartbeat_back_addr":"172.21.15.159:6815/1832540768","heartbeat_front_addr":"172.21.15.159:6813/1832540768","state":["exists","up"]},{"osd":6,"uuid":"f9aa6749-46cf-4b2a-8114-cbd034219d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162436840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162436840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6825","nonce":1162436840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162436840}]},"public_addr":"172.21.15.119:6819/1162436840","cluster_addr":"172.21.15.119:6821/1162436840","heartbeat_back_addr":"172.21.15.119:6825/1162436840","heartbeat_front_addr":"172.21.15.119:6823/1162436840","state":["exists","up"]},{"osd":7,"uuid":"39216361-e1c4-4fef-86cb-a4707656b547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6817","nonce":3254653457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6819","nonce":3254653457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6823","nonce":3254653457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6821","nonce":3254653457}]},"public_addr":"172.21.15.153:6817/3254653457","cluster_addr":"172.21.15.153:6819/3254653457","heartbeat_back_addr":"172.21.15.153:6823/3254653457","heartbeat_front_addr":"172.21.15.153:6821/3254653457","state":["exists","up"]},{"osd":8,"uuid":"cceb6ebc-7c02-48ea-9d16-b7a016dd449b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6817","nonce":2600946380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6819","nonce":2600946380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6823","nonce":2600946380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6821","nonce":2600946380}]},"public_addr":"172.21.15.159:6817/2600946380","cluster_addr":"172.21.15.159:6819/2600946380","heartbeat_back_addr":"172.21.15.159:6823/2600946380","heartbeat_front_addr":"172.21.15.159:6821/2600946380","state":["exists","up"]},{"osd":9,"uuid":"f337a2e4-e065-4a9f-91d9-e3615de2d6f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6827","nonce":1940737285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6829","nonce":1940737285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6833","nonce":1940737285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6831","nonce":1940737285}]},"public_addr":"172.21.15.119:6827/1940737285","cluster_addr":"172.21.15.119:6829/1940737285","heartbeat_back_addr":"172.21.15.119:6833/1940737285","heartbeat_front_addr":"172.21.15.119:6831/1940737285","state":["exists","up"]},{"osd":10,"uuid":"3a3bc347-85f4-4878-9797-d9da96137e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6825","nonce":663834864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6827","nonce":663834864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6831","nonce":663834864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6829","nonce":663834864}]},"public_addr":"172.21.15.153:6825/663834864","cluster_addr":"172.21.15.153:6827/663834864","heartbeat_back_addr":"172.21.15.153:6831/663834864","heartbeat_front_addr":"172.21.15.153:6829/663834864","state":["exists","up"]},{"osd":11,"uuid":"a0fc5909-279d-4172-977c-2dc2da0a7e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6825","nonce":1262216348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6827","nonce":1262216348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6831","nonce":1262216348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6829","nonce":1262216348}]},"public_addr":"172.21.15.159:6825/1262216348","cluster_addr":"172.21.15.159:6827/1262216348","heartbeat_back_addr":"172.21.15.159:6831/1262216348","heartbeat_front_addr":"172.21.15.159:6829/1262216348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:37.488109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:59.543060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:21.381585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:47.112729+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:09.732150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:30.893729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:56.678710+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:19.004256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:40.388818+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:06.391734+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:28.394053+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:49.634348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1628639392":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6801/1355617847":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6800/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/362307836":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/128964558":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/2278813554":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/1411933614":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/1685698277":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6800/1355617847":"2023-12-08T10:11:00.585565+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T10:18:07.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.839+0000 7fdc661ff700 1 -- 172.21.15.119:0/1880573717 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fdc4c0a1b90 msgr2=0x7fdc4c0a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:07.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.839+0000 7fdc661ff700 1 --2- 172.21.15.119:0/1880573717 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fdc4c0a1b90 0x7fdc4c0a4050 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009cb0 tx=0x7fdc50009260 comp rx=0 tx=0).stop 2023-12-07T10:18:07.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.839+0000 7fdc661ff700 1 -- 172.21.15.119:0/1880573717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc60104940 msgr2=0x7fdc600fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:07.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.839+0000 7fdc661ff700 1 --2- 172.21.15.119:0/1880573717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc60104940 0x7fdc600fffc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800e9a0 tx=0x7fdc4800ed60 comp rx=0 tx=0).stop 2023-12-07T10:18:07.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.839+0000 7fdc661ff700 1 -- 172.21.15.119:0/1880573717 shutdown_connections 2023-12-07T10:18:07.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.839+0000 7fdc661ff700 1 --2- 172.21.15.119:0/1880573717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc60107270 0x7fdc60100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:07.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.839+0000 7fdc661ff700 1 --2- 172.21.15.119:0/1880573717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc60102010 0x7fdc60101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:07.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.839+0000 7fdc661ff700 1 --2- 172.21.15.119:0/1880573717 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fdc4c0a1b90 0x7fdc4c0a4050 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:07.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.839+0000 7fdc661ff700 1 --2- 172.21.15.119:0/1880573717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc60104940 0x7fdc600fffc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:07.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.839+0000 7fdc661ff700 1 -- 172.21.15.119:0/1880573717 >> 172.21.15.119:0/1880573717 conn(0x7fdc600fb9c0 msgr2=0x7fdc60102950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:07.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.839+0000 7fdc661ff700 1 -- 172.21.15.119:0/1880573717 shutdown_connections 2023-12-07T10:18:07.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:07.839+0000 7fdc661ff700 1 -- 172.21.15.119:0/1880573717 wait complete. 2023-12-07T10:18:07.861 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-07T10:18:08.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.056+0000 7f431b9ff700 1 -- 172.21.15.119:0/2114631299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4314107220 msgr2=0x7f4314109690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:08.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.056+0000 7f431b9ff700 1 --2- 172.21.15.119:0/2114631299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4314107220 0x7f4314109690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f430c009990 tx=0x7f430c009ca0 comp rx=0 tx=0).stop 2023-12-07T10:18:08.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.056+0000 7f431b9ff700 1 -- 172.21.15.119:0/2114631299 shutdown_connections 2023-12-07T10:18:08.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.056+0000 7f431b9ff700 1 --2- 172.21.15.119:0/2114631299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4314107220 0x7f4314109690 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:08.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.056+0000 7f431b9ff700 1 --2- 172.21.15.119:0/2114631299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4314101fc0 0x7f43141043b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:08.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.056+0000 7f431b9ff700 1 --2- 172.21.15.119:0/2114631299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43141048f0 0x7f4314106ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:08.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.056+0000 7f431b9ff700 1 -- 172.21.15.119:0/2114631299 >> 172.21.15.119:0/2114631299 conn(0x7f43140fb9d0 msgr2=0x7f43140fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:08.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.056+0000 7f431b9ff700 1 -- 172.21.15.119:0/2114631299 shutdown_connections 2023-12-07T10:18:08.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.056+0000 7f431b9ff700 1 -- 172.21.15.119:0/2114631299 wait complete. 2023-12-07T10:18:08.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.057+0000 7f431b9ff700 1 Processor -- start 2023-12-07T10:18:08.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.058+0000 7f431b9ff700 1 -- start start 2023-12-07T10:18:08.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.058+0000 7f431b9ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4314101fc0 0x7f4314197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:08.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.058+0000 7f431b9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43141048f0 0x7f4314197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:08.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.058+0000 7f431b9ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4314107220 0x7f431419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:08.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.058+0000 7f431b9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431419ef60 con 0x7f43141048f0 2023-12-07T10:18:08.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.058+0000 7f431b9ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431419f0a0 con 0x7f4314101fc0 2023-12-07T10:18:08.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.058+0000 7f431b9ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431419f230 con 0x7f4314107220 2023-12-07T10:18:08.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.058+0000 7f4318f9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43141048f0 0x7f4314197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:08.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.059+0000 7f4318f9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43141048f0 0x7f4314197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50780/0 (socket says 172.21.15.119:50780) 2023-12-07T10:18:08.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.059+0000 7f4318f9a700 1 -- 172.21.15.119:0/3731855591 learned_addr learned my addr 172.21.15.119:0/3731855591 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:08.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.059+0000 7f431979b700 1 --2- 172.21.15.119:0/3731855591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4314101fc0 0x7f4314197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:08.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.059+0000 7f4319f9c700 1 --2- 172.21.15.119:0/3731855591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4314107220 0x7f431419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:08.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.059+0000 7f4318f9a700 1 -- 172.21.15.119:0/3731855591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4314107220 msgr2=0x7f431419e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:08.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.059+0000 7f4318f9a700 1 --2- 172.21.15.119:0/3731855591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4314107220 0x7f431419e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:08.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.059+0000 7f4318f9a700 1 -- 172.21.15.119:0/3731855591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4314101fc0 msgr2=0x7f4314197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:08.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.059+0000 7f4318f9a700 1 --2- 172.21.15.119:0/3731855591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4314101fc0 0x7f4314197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:08.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.059+0000 7f4318f9a700 1 -- 172.21.15.119:0/3731855591 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430c0095f0 con 0x7f43141048f0 2023-12-07T10:18:08.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.059+0000 7f4318f9a700 1 --2- 172.21.15.119:0/3731855591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43141048f0 0x7f4314197970 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f430400e9a0 tx=0x7f430400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:08.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.059+0000 7f430a7fc700 1 -- 172.21.15.119:0/3731855591 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4304009790 con 0x7f43141048f0 2023-12-07T10:18:08.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.059+0000 7f430a7fc700 1 -- 172.21.15.119:0/3731855591 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4304010430 con 0x7f43141048f0 2023-12-07T10:18:08.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.059+0000 7f430a7fc700 1 -- 172.21.15.119:0/3731855591 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4304018680 con 0x7f43141048f0 2023-12-07T10:18:08.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.059+0000 7f431b9ff700 1 -- 172.21.15.119:0/3731855591 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f431419f510 con 0x7f43141048f0 2023-12-07T10:18:08.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.060+0000 7f431b9ff700 1 -- 172.21.15.119:0/3731855591 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431419f9d0 con 0x7f43141048f0 2023-12-07T10:18:08.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.060+0000 7f431b9ff700 1 -- 172.21.15.119:0/3731855591 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43140fd550 con 0x7f43141048f0 2023-12-07T10:18:08.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.062+0000 7f430a7fc700 1 -- 172.21.15.119:0/3731855591 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f43040105a0 con 0x7f43141048f0 2023-12-07T10:18:08.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.064+0000 7f430a7fc700 1 --2- 172.21.15.119:0/3731855591 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f43000a1b10 0x7f43000a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:08.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.064+0000 7f430a7fc700 1 -- 172.21.15.119:0/3731855591 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f4304014070 con 0x7f43141048f0 2023-12-07T10:18:08.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.064+0000 7f431979b700 1 --2- 172.21.15.119:0/3731855591 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f43000a1b10 0x7f43000a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:08.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.065+0000 7f431979b700 1 --2- 172.21.15.119:0/3731855591 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f43000a1b10 0x7f43000a3fd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f431000f540 tx=0x7f431000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:08.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.067+0000 7f430a7fc700 1 -- 172.21.15.119:0/3731855591 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f43040522f0 con 0x7f43141048f0 2023-12-07T10:18:08.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:07 smithi119 ceph-mon[97908]: pgmap v294: 129 pgs: 1 activating, 128 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-12-07T10:18:08.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:07 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1880573717' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:18:08.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:07 smithi159 ceph-mon[134726]: pgmap v294: 129 pgs: 1 activating, 128 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-12-07T10:18:08.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:07 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1880573717' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:18:08.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.270+0000 7f431b9ff700 1 -- 172.21.15.119:0/3731855591 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f431410cdf0 con 0x7f43141048f0 2023-12-07T10:18:08.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.272+0000 7f430a7fc700 1 -- 172.21.15.119:0/3731855591 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24528 (secure 0 0 0) 0x7f4304096b70 con 0x7f43141048f0 2023-12-07T10:18:08.274 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:18:08.274 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":91,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","created":"2023-12-07T10:09:59.945863+0000","modified":"2023-12-07T10:18:01.843682+0000","last_up_change":"2023-12-07T10:16:50.929427+0000","last_in_change":"2023-12-07T10:16:33.558158+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T10:13:25.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-07T10:17:39.347813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-07T10:17:40.714501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-07T10:17:44.257760+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"56b5818e-e625-40f1-8838-3ddcf2d34ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6803","nonce":2141950133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6805","nonce":2141950133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6809","nonce":2141950133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6807","nonce":2141950133}]},"public_addr":"172.21.15.119:6803/2141950133","cluster_addr":"172.21.15.119:6805/2141950133","heartbeat_back_addr":"172.21.15.119:6809/2141950133","heartbeat_front_addr":"172.21.15.119:6807/2141950133","state":["exists","up"]},{"osd":1,"uuid":"36489a6a-5278-4802-9525-9871521722e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6801","nonce":3390875370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6803","nonce":3390875370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6807","nonce":3390875370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6805","nonce":3390875370}]},"public_addr":"172.21.15.153:6801/3390875370","cluster_addr":"172.21.15.153:6803/3390875370","heartbeat_back_addr":"172.21.15.153:6807/3390875370","heartbeat_front_addr":"172.21.15.153:6805/3390875370","state":["exists","up"]},{"osd":2,"uuid":"a937ad26-5e9b-4c8a-b65a-626a8feda6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6801","nonce":708101637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6803","nonce":708101637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6807","nonce":708101637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6805","nonce":708101637}]},"public_addr":"172.21.15.159:6801/708101637","cluster_addr":"172.21.15.159:6803/708101637","heartbeat_back_addr":"172.21.15.159:6807/708101637","heartbeat_front_addr":"172.21.15.159:6805/708101637","state":["exists","up"]},{"osd":3,"uuid":"26ef8300-aa07-48f6-b31d-8b0f2217ffcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6811","nonce":3228047257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6813","nonce":3228047257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6817","nonce":3228047257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6815","nonce":3228047257}]},"public_addr":"172.21.15.119:6811/3228047257","cluster_addr":"172.21.15.119:6813/3228047257","heartbeat_back_addr":"172.21.15.119:6817/3228047257","heartbeat_front_addr":"172.21.15.119:6815/3228047257","state":["exists","up"]},{"osd":4,"uuid":"000ee4b9-9896-4901-8cc3-c85fdfc8cbaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6809","nonce":4021998299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6811","nonce":4021998299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6815","nonce":4021998299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6813","nonce":4021998299}]},"public_addr":"172.21.15.153:6809/4021998299","cluster_addr":"172.21.15.153:6811/4021998299","heartbeat_back_addr":"172.21.15.153:6815/4021998299","heartbeat_front_addr":"172.21.15.153:6813/4021998299","state":["exists","up"]},{"osd":5,"uuid":"8b2fc167-262d-474f-89f3-47bd29424421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6809","nonce":1832540768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6811","nonce":1832540768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6815","nonce":1832540768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6813","nonce":1832540768}]},"public_addr":"172.21.15.159:6809/1832540768","cluster_addr":"172.21.15.159:6811/1832540768","heartbeat_back_addr":"172.21.15.159:6815/1832540768","heartbeat_front_addr":"172.21.15.159:6813/1832540768","state":["exists","up"]},{"osd":6,"uuid":"f9aa6749-46cf-4b2a-8114-cbd034219d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162436840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162436840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6825","nonce":1162436840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162436840}]},"public_addr":"172.21.15.119:6819/1162436840","cluster_addr":"172.21.15.119:6821/1162436840","heartbeat_back_addr":"172.21.15.119:6825/1162436840","heartbeat_front_addr":"172.21.15.119:6823/1162436840","state":["exists","up"]},{"osd":7,"uuid":"39216361-e1c4-4fef-86cb-a4707656b547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6817","nonce":3254653457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6819","nonce":3254653457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6823","nonce":3254653457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6821","nonce":3254653457}]},"public_addr":"172.21.15.153:6817/3254653457","cluster_addr":"172.21.15.153:6819/3254653457","heartbeat_back_addr":"172.21.15.153:6823/3254653457","heartbeat_front_addr":"172.21.15.153:6821/3254653457","state":["exists","up"]},{"osd":8,"uuid":"cceb6ebc-7c02-48ea-9d16-b7a016dd449b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6817","nonce":2600946380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6819","nonce":2600946380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6823","nonce":2600946380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6821","nonce":2600946380}]},"public_addr":"172.21.15.159:6817/2600946380","cluster_addr":"172.21.15.159:6819/2600946380","heartbeat_back_addr":"172.21.15.159:6823/2600946380","heartbeat_front_addr":"172.21.15.159:6821/2600946380","state":["exists","up"]},{"osd":9,"uuid":"f337a2e4-e065-4a9f-91d9-e3615de2d6f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6827","nonce":1940737285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6829","nonce":1940737285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6833","nonce":1940737285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6831","nonce":1940737285}]},"public_addr":"172.21.15.119:6827/1940737285","cluster_addr":"172.21.15.119:6829/1940737285","heartbeat_back_addr":"172.21.15.119:6833/1940737285","heartbeat_front_addr":"172.21.15.119:6831/1940737285","state":["exists","up"]},{"osd":10,"uuid":"3a3bc347-85f4-4878-9797-d9da96137e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6825","nonce":663834864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6827","nonce":663834864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6831","nonce":663834864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6829","nonce":663834864}]},"public_addr":"172.21.15.153:6825/663834864","cluster_addr":"172.21.15.153:6827/663834864","heartbeat_back_addr":"172.21.15.153:6831/663834864","heartbeat_front_addr":"172.21.15.153:6829/663834864","state":["exists","up"]},{"osd":11,"uuid":"a0fc5909-279d-4172-977c-2dc2da0a7e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6825","nonce":1262216348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6827","nonce":1262216348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6831","nonce":1262216348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6829","nonce":1262216348}]},"public_addr":"172.21.15.159:6825/1262216348","cluster_addr":"172.21.15.159:6827/1262216348","heartbeat_back_addr":"172.21.15.159:6831/1262216348","heartbeat_front_addr":"172.21.15.159:6829/1262216348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:37.488109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:59.543060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:21.381585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:47.112729+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:09.732150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:30.893729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:56.678710+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:19.004256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:40.388818+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:06.391734+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:28.394053+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:49.634348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1628639392":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6801/1355617847":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6800/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/362307836":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/128964558":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/2278813554":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/1411933614":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/1685698277":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6800/1355617847":"2023-12-08T10:11:00.585565+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T10:18:08.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.274+0000 7f431b9ff700 1 -- 172.21.15.119:0/3731855591 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f43000a1b10 msgr2=0x7f43000a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:08.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.274+0000 7f431b9ff700 1 --2- 172.21.15.119:0/3731855591 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f43000a1b10 0x7f43000a3fd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f431000f540 tx=0x7f431000b040 comp rx=0 tx=0).stop 2023-12-07T10:18:08.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.274+0000 7f431b9ff700 1 -- 172.21.15.119:0/3731855591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43141048f0 msgr2=0x7f4314197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:08.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.274+0000 7f431b9ff700 1 --2- 172.21.15.119:0/3731855591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43141048f0 0x7f4314197970 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f430400e9a0 tx=0x7f430400ecb0 comp rx=0 tx=0).stop 2023-12-07T10:18:08.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.274+0000 7f431b9ff700 1 -- 172.21.15.119:0/3731855591 shutdown_connections 2023-12-07T10:18:08.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.274+0000 7f431b9ff700 1 --2- 172.21.15.119:0/3731855591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4314107220 0x7f431419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:08.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.274+0000 7f431b9ff700 1 --2- 172.21.15.119:0/3731855591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4314101fc0 0x7f4314197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:08.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.274+0000 7f431b9ff700 1 --2- 172.21.15.119:0/3731855591 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f43000a1b10 0x7f43000a3fd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:08.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.274+0000 7f431b9ff700 1 --2- 172.21.15.119:0/3731855591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43141048f0 0x7f4314197970 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:08.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.274+0000 7f431b9ff700 1 -- 172.21.15.119:0/3731855591 >> 172.21.15.119:0/3731855591 conn(0x7f43140fb9d0 msgr2=0x7f4314102900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:08.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.274+0000 7f431b9ff700 1 -- 172.21.15.119:0/3731855591 shutdown_connections 2023-12-07T10:18:08.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.274+0000 7f431b9ff700 1 -- 172.21.15.119:0/3731855591 wait complete. 2023-12-07T10:18:08.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:07 smithi153 ceph-mon[133625]: pgmap v294: 129 pgs: 1 activating, 128 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-12-07T10:18:08.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:07 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1880573717' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:18:08.295 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-07T10:18:08.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.492+0000 7fc17d7ff700 1 -- 172.21.15.119:0/2099849780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1781027b0 msgr2=0x7fc178102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:08.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.492+0000 7fc17d7ff700 1 --2- 172.21.15.119:0/2099849780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1781027b0 0x7fc178102c10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0099c0 tx=0x7fc16c009cd0 comp rx=0 tx=0).stop 2023-12-07T10:18:08.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.493+0000 7fc17d7ff700 1 -- 172.21.15.119:0/2099849780 shutdown_connections 2023-12-07T10:18:08.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.493+0000 7fc17d7ff700 1 --2- 172.21.15.119:0/2099849780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc178103150 0x7fc178109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:08.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.493+0000 7fc17d7ff700 1 --2- 172.21.15.119:0/2099849780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1781015c0 0x7fc1781019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:08.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.493+0000 7fc17d7ff700 1 --2- 172.21.15.119:0/2099849780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1781027b0 0x7fc178102c10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:08.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.493+0000 7fc17d7ff700 1 -- 172.21.15.119:0/2099849780 >> 172.21.15.119:0/2099849780 conn(0x7fc1780fcb10 msgr2=0x7fc1780fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:08.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.493+0000 7fc17d7ff700 1 -- 172.21.15.119:0/2099849780 shutdown_connections 2023-12-07T10:18:08.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.493+0000 7fc17d7ff700 1 -- 172.21.15.119:0/2099849780 wait complete. 2023-12-07T10:18:08.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.494+0000 7fc17d7ff700 1 Processor -- start 2023-12-07T10:18:08.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.494+0000 7fc17d7ff700 1 -- start start 2023-12-07T10:18:08.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.494+0000 7fc17d7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1781015c0 0x7fc178199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:08.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.494+0000 7fc17d7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1781027b0 0x7fc178199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:08.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.494+0000 7fc17d7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc178103150 0x7fc1781a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:08.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.494+0000 7fc17d7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1781a1280 con 0x7fc1781015c0 2023-12-07T10:18:08.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.494+0000 7fc17d7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1781a13c0 con 0x7fc178103150 2023-12-07T10:18:08.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.494+0000 7fc17d7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1781a1550 con 0x7fc1781027b0 2023-12-07T10:18:08.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.494+0000 7fc176ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1781015c0 0x7fc178199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:08.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.494+0000 7fc176ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1781015c0 0x7fc178199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50812/0 (socket says 172.21.15.119:50812) 2023-12-07T10:18:08.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.494+0000 7fc176ffd700 1 -- 172.21.15.119:0/2815895827 learned_addr learned my addr 172.21.15.119:0/2815895827 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:08.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.495+0000 7fc1767fc700 1 --2- 172.21.15.119:0/2815895827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1781027b0 0x7fc178199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:08.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.495+0000 7fc1777fe700 1 --2- 172.21.15.119:0/2815895827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc178103150 0x7fc1781a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:08.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.495+0000 7fc176ffd700 1 -- 172.21.15.119:0/2815895827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1781027b0 msgr2=0x7fc178199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:08.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.495+0000 7fc176ffd700 1 --2- 172.21.15.119:0/2815895827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1781027b0 0x7fc178199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:08.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.495+0000 7fc176ffd700 1 -- 172.21.15.119:0/2815895827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc178103150 msgr2=0x7fc1781a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:08.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.495+0000 7fc176ffd700 1 --2- 172.21.15.119:0/2815895827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc178103150 0x7fc1781a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:08.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.495+0000 7fc176ffd700 1 -- 172.21.15.119:0/2815895827 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc170009c40 con 0x7fc1781015c0 2023-12-07T10:18:08.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.495+0000 7fc176ffd700 1 --2- 172.21.15.119:0/2815895827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1781015c0 0x7fc178199750 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc16800e970 tx=0x7fc16800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:08.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.495+0000 7fc15ffff700 1 -- 172.21.15.119:0/2815895827 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc168009790 con 0x7fc1781015c0 2023-12-07T10:18:08.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.495+0000 7fc15ffff700 1 -- 172.21.15.119:0/2815895827 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc16800c7c0 con 0x7fc1781015c0 2023-12-07T10:18:08.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.495+0000 7fc17d7ff700 1 -- 172.21.15.119:0/2815895827 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc16c0095f0 con 0x7fc1781015c0 2023-12-07T10:18:08.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.495+0000 7fc15ffff700 1 -- 172.21.15.119:0/2815895827 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc168003c20 con 0x7fc1781015c0 2023-12-07T10:18:08.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.495+0000 7fc17d7ff700 1 -- 172.21.15.119:0/2815895827 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1781a1c00 con 0x7fc1781015c0 2023-12-07T10:18:08.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.496+0000 7fc17d7ff700 1 -- 172.21.15.119:0/2815895827 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1780683c0 con 0x7fc1781015c0 2023-12-07T10:18:08.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.498+0000 7fc15ffff700 1 -- 172.21.15.119:0/2815895827 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fc16800c930 con 0x7fc1781015c0 2023-12-07T10:18:08.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.500+0000 7fc15ffff700 1 --2- 172.21.15.119:0/2815895827 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc1600a1b90 0x7fc1600a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:08.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.500+0000 7fc15ffff700 1 -- 172.21.15.119:0/2815895827 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fc168014070 con 0x7fc1781015c0 2023-12-07T10:18:08.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.500+0000 7fc1767fc700 1 --2- 172.21.15.119:0/2815895827 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc1600a1b90 0x7fc1600a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:08.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.500+0000 7fc1767fc700 1 --2- 172.21.15.119:0/2815895827 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc1600a1b90 0x7fc1600a4050 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc16c006010 tx=0x7fc16c00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:08.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.503+0000 7fc15ffff700 1 -- 172.21.15.119:0/2815895827 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc168096af0 con 0x7fc1781015c0 2023-12-07T10:18:08.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.756+0000 7fc17d7ff700 1 -- 172.21.15.119:0/2815895827 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc17810ce30 con 0x7fc1781015c0 2023-12-07T10:18:08.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.872+0000 7fc15ffff700 1 -- 172.21.15.119:0/2815895827 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc16809a110 con 0x7fc1781015c0 2023-12-07T10:18:08.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.874+0000 7fc17d7ff700 1 -- 172.21.15.119:0/2815895827 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc1600a1b90 msgr2=0x7fc1600a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:08.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.874+0000 7fc17d7ff700 1 --2- 172.21.15.119:0/2815895827 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc1600a1b90 0x7fc1600a4050 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc16c006010 tx=0x7fc16c00b540 comp rx=0 tx=0).stop 2023-12-07T10:18:08.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.875+0000 7fc17d7ff700 1 -- 172.21.15.119:0/2815895827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1781015c0 msgr2=0x7fc178199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:08.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.875+0000 7fc17d7ff700 1 --2- 172.21.15.119:0/2815895827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1781015c0 0x7fc178199750 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc16800e970 tx=0x7fc16800ec80 comp rx=0 tx=0).stop 2023-12-07T10:18:08.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.875+0000 7fc17d7ff700 1 -- 172.21.15.119:0/2815895827 shutdown_connections 2023-12-07T10:18:08.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.875+0000 7fc17d7ff700 1 --2- 172.21.15.119:0/2815895827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc178103150 0x7fc1781a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:08.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.875+0000 7fc17d7ff700 1 --2- 172.21.15.119:0/2815895827 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc1600a1b90 0x7fc1600a4050 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:08.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.875+0000 7fc17d7ff700 1 --2- 172.21.15.119:0/2815895827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1781015c0 0x7fc178199750 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:08.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.875+0000 7fc17d7ff700 1 --2- 172.21.15.119:0/2815895827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1781027b0 0x7fc178199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:08.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.875+0000 7fc17d7ff700 1 -- 172.21.15.119:0/2815895827 >> 172.21.15.119:0/2815895827 conn(0x7fc1780fcb10 msgr2=0x7fc178073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:08.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.875+0000 7fc17d7ff700 1 -- 172.21.15.119:0/2815895827 shutdown_connections 2023-12-07T10:18:08.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:08.875+0000 7fc17d7ff700 1 -- 172.21.15.119:0/2815895827 wait complete. 2023-12-07T10:18:08.895 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2023-12-07T10:18:08.896 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-12-07T10:18:09.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.094+0000 7f1cc2bff700 1 -- 172.21.15.119:0/1392632517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cbc1094d0 msgr2=0x7f1cbc10b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:09.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.094+0000 7f1cc2bff700 1 --2- 172.21.15.119:0/1392632517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cbc1094d0 0x7f1cbc10b960 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400b0a0 tx=0x7f1cb400b3b0 comp rx=0 tx=0).stop 2023-12-07T10:18:09.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.094+0000 7f1cc2bff700 1 -- 172.21.15.119:0/1392632517 shutdown_connections 2023-12-07T10:18:09.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.094+0000 7f1cc2bff700 1 --2- 172.21.15.119:0/1392632517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cbc1094d0 0x7f1cbc10b960 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:09.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.094+0000 7f1cc2bff700 1 --2- 172.21.15.119:0/1392632517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cbc0755d0 0x7f1cbc073a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:09.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.094+0000 7f1cc2bff700 1 --2- 172.21.15.119:0/1392632517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cbc073ff0 0x7f1cbc074470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:09.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.094+0000 7f1cc2bff700 1 -- 172.21.15.119:0/1392632517 >> 172.21.15.119:0/1392632517 conn(0x7f1cbc0fcb50 msgr2=0x7f1cbc0fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:09.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.094+0000 7f1cc2bff700 1 -- 172.21.15.119:0/1392632517 shutdown_connections 2023-12-07T10:18:09.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.094+0000 7f1cc2bff700 1 -- 172.21.15.119:0/1392632517 wait complete. 2023-12-07T10:18:09.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.095+0000 7f1cc2bff700 1 Processor -- start 2023-12-07T10:18:09.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.095+0000 7f1cc2bff700 1 -- start start 2023-12-07T10:18:09.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.095+0000 7f1cc2bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cbc073ff0 0x7f1cbc199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:09.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.095+0000 7f1cc2bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cbc0755d0 0x7f1cbc199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:09.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.095+0000 7f1cc2bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cbc1094d0 0x7f1cbc1a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:09.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.095+0000 7f1cc2bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc1a1340 con 0x7f1cbc073ff0 2023-12-07T10:18:09.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.095+0000 7f1cc2bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc1a1480 con 0x7f1cbc1094d0 2023-12-07T10:18:09.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.095+0000 7f1cc2bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc1a1610 con 0x7f1cbc0755d0 2023-12-07T10:18:09.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.095+0000 7f1cc099b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cbc073ff0 0x7f1cbc199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:09.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.095+0000 7f1cc099b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cbc073ff0 0x7f1cbc199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50836/0 (socket says 172.21.15.119:50836) 2023-12-07T10:18:09.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.095+0000 7f1cc099b700 1 -- 172.21.15.119:0/897278444 learned_addr learned my addr 172.21.15.119:0/897278444 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:09.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.096+0000 7f1cc119c700 1 --2- 172.21.15.119:0/897278444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cbc1094d0 0x7f1cbc1a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:09.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.096+0000 7f1cbbfff700 1 --2- 172.21.15.119:0/897278444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cbc0755d0 0x7f1cbc199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:09.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.096+0000 7f1cc099b700 1 -- 172.21.15.119:0/897278444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cbc0755d0 msgr2=0x7f1cbc199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:09.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.096+0000 7f1cc099b700 1 --2- 172.21.15.119:0/897278444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cbc0755d0 0x7f1cbc199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:09.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.096+0000 7f1cc099b700 1 -- 172.21.15.119:0/897278444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cbc1094d0 msgr2=0x7f1cbc1a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:09.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.096+0000 7f1cc099b700 1 --2- 172.21.15.119:0/897278444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cbc1094d0 0x7f1cbc1a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:09.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.096+0000 7f1cc099b700 1 -- 172.21.15.119:0/897278444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb4009d00 con 0x7f1cbc073ff0 2023-12-07T10:18:09.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.096+0000 7f1cc099b700 1 --2- 172.21.15.119:0/897278444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cbc073ff0 0x7f1cbc199810 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000e9a0 tx=0x7f1cb000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:09.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.096+0000 7f1cb9ffb700 1 -- 172.21.15.119:0/897278444 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0009790 con 0x7f1cbc073ff0 2023-12-07T10:18:09.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.096+0000 7f1cc2bff700 1 -- 172.21.15.119:0/897278444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cbc1a18f0 con 0x7f1cbc073ff0 2023-12-07T10:18:09.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.096+0000 7f1cb9ffb700 1 -- 172.21.15.119:0/897278444 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cb000cbd0 con 0x7f1cbc073ff0 2023-12-07T10:18:09.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.096+0000 7f1cb9ffb700 1 -- 172.21.15.119:0/897278444 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0018680 con 0x7f1cbc073ff0 2023-12-07T10:18:09.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.096+0000 7f1cc2bff700 1 -- 172.21.15.119:0/897278444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cbc1a1e00 con 0x7f1cbc073ff0 2023-12-07T10:18:09.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.097+0000 7f1cc2bff700 1 -- 172.21.15.119:0/897278444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cbc1082e0 con 0x7f1cbc073ff0 2023-12-07T10:18:09.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.099+0000 7f1cb9ffb700 1 -- 172.21.15.119:0/897278444 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f1cb000cd40 con 0x7f1cbc073ff0 2023-12-07T10:18:09.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.101+0000 7f1cb9ffb700 1 --2- 172.21.15.119:0/897278444 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1cac0a1b10 0x7f1cac0a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:09.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.101+0000 7f1cb9ffb700 1 -- 172.21.15.119:0/897278444 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f1cb0014070 con 0x7f1cbc073ff0 2023-12-07T10:18:09.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.101+0000 7f1cbbfff700 1 --2- 172.21.15.119:0/897278444 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1cac0a1b10 0x7f1cac0a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:09.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.101+0000 7f1cbbfff700 1 --2- 172.21.15.119:0/897278444 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1cac0a1b10 0x7f1cac0a3fd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1ca800bbd0 tx=0x7f1ca800b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:09.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.102+0000 7f1cb9ffb700 1 -- 172.21.15.119:0/897278444 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1cb0096e80 con 0x7f1cbc073ff0 2023-12-07T10:18:09.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:08 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3731855591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:18:09.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:08 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2815895827' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-12-07T10:18:09.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:08 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3731855591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:18:09.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:08 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2815895827' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-12-07T10:18:09.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:08 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3731855591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:18:09.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:08 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2815895827' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-12-07T10:18:09.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.347+0000 7f1cc2bff700 1 -- 172.21.15.119:0/897278444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f1cbc10f020 con 0x7f1cbc073ff0 2023-12-07T10:18:09.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.880+0000 7f1cb9ffb700 1 -- 172.21.15.119:0/897278444 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f1cb009a4a0 con 0x7f1cbc073ff0 2023-12-07T10:18:09.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.885+0000 7f1cc2bff700 1 -- 172.21.15.119:0/897278444 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1cac0a1b10 msgr2=0x7f1cac0a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:09.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.885+0000 7f1cc2bff700 1 --2- 172.21.15.119:0/897278444 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1cac0a1b10 0x7f1cac0a3fd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1ca800bbd0 tx=0x7f1ca800b5b0 comp rx=0 tx=0).stop 2023-12-07T10:18:09.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.885+0000 7f1cc2bff700 1 -- 172.21.15.119:0/897278444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cbc073ff0 msgr2=0x7f1cbc199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:09.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.885+0000 7f1cc2bff700 1 --2- 172.21.15.119:0/897278444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cbc073ff0 0x7f1cbc199810 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000e9a0 tx=0x7f1cb000ecb0 comp rx=0 tx=0).stop 2023-12-07T10:18:09.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.885+0000 7f1cc2bff700 1 -- 172.21.15.119:0/897278444 shutdown_connections 2023-12-07T10:18:09.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.885+0000 7f1cc2bff700 1 --2- 172.21.15.119:0/897278444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1cbc1094d0 0x7f1cbc1a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:09.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.885+0000 7f1cc2bff700 1 --2- 172.21.15.119:0/897278444 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1cac0a1b10 0x7f1cac0a3fd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:09.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.885+0000 7f1cc2bff700 1 --2- 172.21.15.119:0/897278444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cbc073ff0 0x7f1cbc199810 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:09.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.885+0000 7f1cc2bff700 1 --2- 172.21.15.119:0/897278444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cbc0755d0 0x7f1cbc199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:09.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.885+0000 7f1cc2bff700 1 -- 172.21.15.119:0/897278444 >> 172.21.15.119:0/897278444 conn(0x7f1cbc0fcb50 msgr2=0x7f1cbc105420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:09.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.885+0000 7f1cc2bff700 1 -- 172.21.15.119:0/897278444 shutdown_connections 2023-12-07T10:18:09.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:09.885+0000 7f1cc2bff700 1 -- 172.21.15.119:0/897278444 wait complete. 2023-12-07T10:18:09.917 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-12-07T10:18:10.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.117+0000 7fb0f51ff700 1 -- 172.21.15.119:0/1895973625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0f01027b0 msgr2=0x7fb0f0102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:10.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.117+0000 7fb0f51ff700 1 --2- 172.21.15.119:0/1895973625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0f01027b0 0x7fb0f0102c10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009920 tx=0x7fb0d8009c30 comp rx=0 tx=0).stop 2023-12-07T10:18:10.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.117+0000 7fb0f51ff700 1 -- 172.21.15.119:0/1895973625 shutdown_connections 2023-12-07T10:18:10.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.117+0000 7fb0f51ff700 1 --2- 172.21.15.119:0/1895973625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0f0103150 0x7fb0f0109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:10.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.117+0000 7fb0f51ff700 1 --2- 172.21.15.119:0/1895973625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0f01015c0 0x7fb0f01019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:10.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.117+0000 7fb0f51ff700 1 --2- 172.21.15.119:0/1895973625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0f01027b0 0x7fb0f0102c10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:10.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.117+0000 7fb0f51ff700 1 -- 172.21.15.119:0/1895973625 >> 172.21.15.119:0/1895973625 conn(0x7fb0f00fcb10 msgr2=0x7fb0f00fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:10.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.117+0000 7fb0f51ff700 1 -- 172.21.15.119:0/1895973625 shutdown_connections 2023-12-07T10:18:10.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.117+0000 7fb0f51ff700 1 -- 172.21.15.119:0/1895973625 wait complete. 2023-12-07T10:18:10.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.118+0000 7fb0f51ff700 1 Processor -- start 2023-12-07T10:18:10.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.118+0000 7fb0f51ff700 1 -- start start 2023-12-07T10:18:10.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.118+0000 7fb0f51ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0f01015c0 0x7fb0f0075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:10.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.118+0000 7fb0f51ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0f01027b0 0x7fb0f0073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:10.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.118+0000 7fb0f51ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0f0103150 0x7fb0f0073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:10.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.118+0000 7fb0f51ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00746e0 con 0x7fb0f01015c0 2023-12-07T10:18:10.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.118+0000 7fb0f51ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f0074850 con 0x7fb0f0103150 2023-12-07T10:18:10.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.118+0000 7fb0f51ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f01a1600 con 0x7fb0f01027b0 2023-12-07T10:18:10.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.118+0000 7fb0eed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0f01015c0 0x7fb0f0075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:10.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.118+0000 7fb0eed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0f01015c0 0x7fb0f0075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50846/0 (socket says 172.21.15.119:50846) 2023-12-07T10:18:10.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.118+0000 7fb0ef59e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0f0103150 0x7fb0f0073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:10.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.118+0000 7fb0eed9d700 1 -- 172.21.15.119:0/872044875 learned_addr learned my addr 172.21.15.119:0/872044875 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:10.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.119+0000 7fb0e7fff700 1 --2- 172.21.15.119:0/872044875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0f01027b0 0x7fb0f0073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:10.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.119+0000 7fb0eed9d700 1 -- 172.21.15.119:0/872044875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0f01027b0 msgr2=0x7fb0f0073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:10.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.119+0000 7fb0eed9d700 1 --2- 172.21.15.119:0/872044875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0f01027b0 0x7fb0f0073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:10.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.119+0000 7fb0eed9d700 1 -- 172.21.15.119:0/872044875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0f0103150 msgr2=0x7fb0f0073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:10.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.119+0000 7fb0eed9d700 1 --2- 172.21.15.119:0/872044875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0f0103150 0x7fb0f0073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:10.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.119+0000 7fb0eed9d700 1 -- 172.21.15.119:0/872044875 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d8009520 con 0x7fb0f01015c0 2023-12-07T10:18:10.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.119+0000 7fb0ef59e700 1 --2- 172.21.15.119:0/872044875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0f0103150 0x7fb0f0073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:18:10.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.119+0000 7fb0eed9d700 1 --2- 172.21.15.119:0/872044875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0f01015c0 0x7fb0f0075370 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000e970 tx=0x7fb0e000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:10.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.119+0000 7fb0ecd99700 1 -- 172.21.15.119:0/872044875 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0009790 con 0x7fb0f01015c0 2023-12-07T10:18:10.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.119+0000 7fb0f51ff700 1 -- 172.21.15.119:0/872044875 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f01a1890 con 0x7fb0f01015c0 2023-12-07T10:18:10.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.119+0000 7fb0ecd99700 1 -- 172.21.15.119:0/872044875 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0e000c780 con 0x7fb0f01015c0 2023-12-07T10:18:10.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.119+0000 7fb0ecd99700 1 -- 172.21.15.119:0/872044875 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0003be0 con 0x7fb0f01015c0 2023-12-07T10:18:10.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.119+0000 7fb0f51ff700 1 -- 172.21.15.119:0/872044875 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f01a1df0 con 0x7fb0f01015c0 2023-12-07T10:18:10.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.121+0000 7fb0f51ff700 1 -- 172.21.15.119:0/872044875 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f0108ae0 con 0x7fb0f01015c0 2023-12-07T10:18:10.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.122+0000 7fb0ecd99700 1 -- 172.21.15.119:0/872044875 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fb0e000cbf0 con 0x7fb0f01015c0 2023-12-07T10:18:10.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.124+0000 7fb0ecd99700 1 --2- 172.21.15.119:0/872044875 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fb0dc0a1b10 0x7fb0dc0a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:10.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.124+0000 7fb0ecd99700 1 -- 172.21.15.119:0/872044875 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fb0e0014070 con 0x7fb0f01015c0 2023-12-07T10:18:10.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.124+0000 7fb0e7fff700 1 --2- 172.21.15.119:0/872044875 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fb0dc0a1b10 0x7fb0dc0a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:10.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.125+0000 7fb0e7fff700 1 --2- 172.21.15.119:0/872044875 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fb0dc0a1b10 0x7fb0dc0a3fd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb0d800fa40 tx=0x7fb0d8019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:10.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.127+0000 7fb0ecd99700 1 -- 172.21.15.119:0/872044875 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fb0e0096b60 con 0x7fb0f01015c0 2023-12-07T10:18:10.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:09 smithi119 ceph-mon[97908]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-12-07T10:18:10.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:09 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2815895827' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-12-07T10:18:10.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:09 smithi119 ceph-mon[97908]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-07T10:18:10.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:09 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/897278444' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-12-07T10:18:10.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:09 smithi159 ceph-mon[134726]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-12-07T10:18:10.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:09 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2815895827' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-12-07T10:18:10.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:09 smithi159 ceph-mon[134726]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-07T10:18:10.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:09 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/897278444' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-12-07T10:18:10.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:09 smithi153 ceph-mon[133625]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2023-12-07T10:18:10.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:09 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2815895827' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-12-07T10:18:10.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:09 smithi153 ceph-mon[133625]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-07T10:18:10.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:09 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/897278444' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-12-07T10:18:10.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.377+0000 7fb0f51ff700 1 -- 172.21.15.119:0/872044875 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fb0f01084c0 con 0x7fb0f01015c0 2023-12-07T10:18:10.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.724+0000 7fb0ecd99700 1 -- 172.21.15.119:0/872044875 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7fb0e009a180 con 0x7fb0f01015c0 2023-12-07T10:18:10.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.727+0000 7fb0f51ff700 1 -- 172.21.15.119:0/872044875 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fb0dc0a1b10 msgr2=0x7fb0dc0a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:10.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.727+0000 7fb0f51ff700 1 --2- 172.21.15.119:0/872044875 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fb0dc0a1b10 0x7fb0dc0a3fd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb0d800fa40 tx=0x7fb0d8019040 comp rx=0 tx=0).stop 2023-12-07T10:18:10.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.727+0000 7fb0f51ff700 1 -- 172.21.15.119:0/872044875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0f01015c0 msgr2=0x7fb0f0075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:10.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.727+0000 7fb0f51ff700 1 --2- 172.21.15.119:0/872044875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0f01015c0 0x7fb0f0075370 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000e970 tx=0x7fb0e000ec80 comp rx=0 tx=0).stop 2023-12-07T10:18:10.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.727+0000 7fb0f51ff700 1 -- 172.21.15.119:0/872044875 shutdown_connections 2023-12-07T10:18:10.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.727+0000 7fb0f51ff700 1 --2- 172.21.15.119:0/872044875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0f0103150 0x7fb0f0073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:10.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.727+0000 7fb0f51ff700 1 --2- 172.21.15.119:0/872044875 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fb0dc0a1b10 0x7fb0dc0a3fd0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:10.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.727+0000 7fb0f51ff700 1 --2- 172.21.15.119:0/872044875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0f01015c0 0x7fb0f0075370 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:10.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.727+0000 7fb0f51ff700 1 --2- 172.21.15.119:0/872044875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0f01027b0 0x7fb0f0073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:10.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.727+0000 7fb0f51ff700 1 -- 172.21.15.119:0/872044875 >> 172.21.15.119:0/872044875 conn(0x7fb0f00fcb10 msgr2=0x7fb0f0105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:10.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.727+0000 7fb0f51ff700 1 -- 172.21.15.119:0/872044875 shutdown_connections 2023-12-07T10:18:10.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.727+0000 7fb0f51ff700 1 -- 172.21.15.119:0/872044875 wait complete. 2023-12-07T10:18:10.753 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-12-07T10:18:10.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.955+0000 7f3d5a7ff700 1 -- 172.21.15.119:0/268575991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d541027b0 msgr2=0x7f3d54102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:10.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.955+0000 7f3d5a7ff700 1 --2- 172.21.15.119:0/268575991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d541027b0 0x7f3d54102c10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3d480099c0 tx=0x7f3d48009cd0 comp rx=0 tx=0).stop 2023-12-07T10:18:10.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.955+0000 7f3d5a7ff700 1 -- 172.21.15.119:0/268575991 shutdown_connections 2023-12-07T10:18:10.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.955+0000 7f3d5a7ff700 1 --2- 172.21.15.119:0/268575991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d54103150 0x7f3d54109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:10.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.955+0000 7f3d5a7ff700 1 --2- 172.21.15.119:0/268575991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d541015c0 0x7f3d541019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:10.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.955+0000 7f3d5a7ff700 1 --2- 172.21.15.119:0/268575991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d541027b0 0x7f3d54102c10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:10.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.955+0000 7f3d5a7ff700 1 -- 172.21.15.119:0/268575991 >> 172.21.15.119:0/268575991 conn(0x7f3d540fcb10 msgr2=0x7f3d540fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:10.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.955+0000 7f3d5a7ff700 1 -- 172.21.15.119:0/268575991 shutdown_connections 2023-12-07T10:18:10.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.955+0000 7f3d5a7ff700 1 -- 172.21.15.119:0/268575991 wait complete. 2023-12-07T10:18:10.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.956+0000 7f3d5a7ff700 1 Processor -- start 2023-12-07T10:18:10.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.956+0000 7f3d5a7ff700 1 -- start start 2023-12-07T10:18:10.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.956+0000 7f3d5a7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d541015c0 0x7f3d54075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:10.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.956+0000 7f3d5a7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d541027b0 0x7f3d54073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:10.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.956+0000 7f3d5a7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d54103150 0x7f3d54073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:10.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.956+0000 7f3d5a7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d540746e0 con 0x7f3d54103150 2023-12-07T10:18:10.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.956+0000 7f3d5a7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d54074850 con 0x7f3d541015c0 2023-12-07T10:18:10.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.956+0000 7f3d5a7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d541a1570 con 0x7f3d541027b0 2023-12-07T10:18:10.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.957+0000 7f3d58d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d54103150 0x7f3d54073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:10.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.957+0000 7f3d58d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d54103150 0x7f3d54073f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50870/0 (socket says 172.21.15.119:50870) 2023-12-07T10:18:10.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.957+0000 7f3d58d9c700 1 -- 172.21.15.119:0/3655435289 learned_addr learned my addr 172.21.15.119:0/3655435289 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:10.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.957+0000 7f3d53fff700 1 --2- 172.21.15.119:0/3655435289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d541015c0 0x7f3d54075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:10.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.957+0000 7f3d537fe700 1 --2- 172.21.15.119:0/3655435289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d541027b0 0x7f3d54073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:10.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.957+0000 7f3d58d9c700 1 -- 172.21.15.119:0/3655435289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d541027b0 msgr2=0x7f3d54073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:10.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.957+0000 7f3d58d9c700 1 --2- 172.21.15.119:0/3655435289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d541027b0 0x7f3d54073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:10.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.957+0000 7f3d58d9c700 1 -- 172.21.15.119:0/3655435289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d541015c0 msgr2=0x7f3d54075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:10.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.957+0000 7f3d58d9c700 1 --2- 172.21.15.119:0/3655435289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d541015c0 0x7f3d54075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:10.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.957+0000 7f3d58d9c700 1 -- 172.21.15.119:0/3655435289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d480095f0 con 0x7f3d54103150 2023-12-07T10:18:10.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.957+0000 7f3d58d9c700 1 --2- 172.21.15.119:0/3655435289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d54103150 0x7f3d54073f40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c007ee0 tx=0x7f3d4c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:10.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.958+0000 7f3d517fa700 1 -- 172.21.15.119:0/3655435289 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c017070 con 0x7f3d54103150 2023-12-07T10:18:10.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.958+0000 7f3d517fa700 1 -- 172.21.15.119:0/3655435289 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d4c00f040 con 0x7f3d54103150 2023-12-07T10:18:10.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.958+0000 7f3d5a7ff700 1 -- 172.21.15.119:0/3655435289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d541a1800 con 0x7f3d54103150 2023-12-07T10:18:10.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.958+0000 7f3d517fa700 1 -- 172.21.15.119:0/3655435289 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c0054b0 con 0x7f3d54103150 2023-12-07T10:18:10.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.958+0000 7f3d5a7ff700 1 -- 172.21.15.119:0/3655435289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d541a1d10 con 0x7f3d54103150 2023-12-07T10:18:10.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.959+0000 7f3d5a7ff700 1 -- 172.21.15.119:0/3655435289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d54108a90 con 0x7f3d54103150 2023-12-07T10:18:10.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.961+0000 7f3d517fa700 1 -- 172.21.15.119:0/3655435289 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f3d4c004ad0 con 0x7f3d54103150 2023-12-07T10:18:10.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.962+0000 7f3d517fa700 1 --2- 172.21.15.119:0/3655435289 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3d3c0a1b90 0x7f3d3c0a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:10.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.962+0000 7f3d517fa700 1 -- 172.21.15.119:0/3655435289 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3d4c0ccff0 con 0x7f3d54103150 2023-12-07T10:18:10.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.962+0000 7f3d53fff700 1 --2- 172.21.15.119:0/3655435289 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3d3c0a1b90 0x7f3d3c0a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:10.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.963+0000 7f3d53fff700 1 --2- 172.21.15.119:0/3655435289 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3d3c0a1b90 0x7f3d3c0a4050 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3d54102610 tx=0x7f3d44009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:10.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:10.966+0000 7f3d517fa700 1 -- 172.21.15.119:0/3655435289 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3d4c096110 con 0x7f3d54103150 2023-12-07T10:18:11.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.215+0000 7f3d5a7ff700 1 -- 172.21.15.119:0/3655435289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f3d540683c0 con 0x7f3d54103150 2023-12-07T10:18:11.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:10 smithi119 ceph-mon[97908]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-12-07T10:18:11.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:10 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/897278444' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-12-07T10:18:11.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:10 smithi119 ceph-mon[97908]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-12-07T10:18:11.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:10 smithi119 ceph-mon[97908]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-07T10:18:11.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:10 smithi119 ceph-mon[97908]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2023-12-07T10:18:11.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:10 smithi119 ceph-mon[97908]: daemon mds.g is now active in filesystem cephfs as rank 1 2023-12-07T10:18:11.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:10 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/872044875' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-12-07T10:18:11.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:10 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/872044875' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-12-07T10:18:11.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:10 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] up:active 2023-12-07T10:18:11.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:10 smithi119 ceph-mon[97908]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-12-07T10:18:11.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:10 smithi119 ceph-mon[97908]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 2 up:standby-replay 8 up:standby 2023-12-07T10:18:11.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:10 smithi119 ceph-mon[97908]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=c=up:creating} 2 up:standby-replay 7 up:standby 2023-12-07T10:18:11.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:10 smithi119 ceph-mon[97908]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-12-07T10:18:11.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:10 smithi159 ceph-mon[134726]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-12-07T10:18:11.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:10 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/897278444' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-12-07T10:18:11.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:10 smithi159 ceph-mon[134726]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-12-07T10:18:11.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:10 smithi159 ceph-mon[134726]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-07T10:18:11.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:10 smithi159 ceph-mon[134726]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2023-12-07T10:18:11.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:10 smithi159 ceph-mon[134726]: daemon mds.g is now active in filesystem cephfs as rank 1 2023-12-07T10:18:11.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:10 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/872044875' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-12-07T10:18:11.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:10 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/872044875' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-12-07T10:18:11.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:10 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] up:active 2023-12-07T10:18:11.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:10 smithi159 ceph-mon[134726]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-12-07T10:18:11.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:10 smithi159 ceph-mon[134726]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 2 up:standby-replay 8 up:standby 2023-12-07T10:18:11.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:10 smithi159 ceph-mon[134726]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=c=up:creating} 2 up:standby-replay 7 up:standby 2023-12-07T10:18:11.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:10 smithi159 ceph-mon[134726]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-12-07T10:18:11.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:10 smithi153 ceph-mon[133625]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-12-07T10:18:11.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:10 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/897278444' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-12-07T10:18:11.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:10 smithi153 ceph-mon[133625]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-12-07T10:18:11.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:10 smithi153 ceph-mon[133625]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-07T10:18:11.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:10 smithi153 ceph-mon[133625]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2023-12-07T10:18:11.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:10 smithi153 ceph-mon[133625]: daemon mds.g is now active in filesystem cephfs as rank 1 2023-12-07T10:18:11.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:10 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/872044875' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-12-07T10:18:11.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:10 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/872044875' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-12-07T10:18:11.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:10 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] up:active 2023-12-07T10:18:11.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:10 smithi153 ceph-mon[133625]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-12-07T10:18:11.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:10 smithi153 ceph-mon[133625]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 2 up:standby-replay 8 up:standby 2023-12-07T10:18:11.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:10 smithi153 ceph-mon[133625]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=c=up:creating} 2 up:standby-replay 7 up:standby 2023-12-07T10:18:11.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:10 smithi153 ceph-mon[133625]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-12-07T10:18:11.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.731+0000 7f3d517fa700 1 -- 172.21.15.119:0/3655435289 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7f3d4c099730 con 0x7f3d54103150 2023-12-07T10:18:11.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.734+0000 7f3d5a7ff700 1 -- 172.21.15.119:0/3655435289 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3d3c0a1b90 msgr2=0x7f3d3c0a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:11.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.734+0000 7f3d5a7ff700 1 --2- 172.21.15.119:0/3655435289 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3d3c0a1b90 0x7f3d3c0a4050 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3d54102610 tx=0x7f3d44009260 comp rx=0 tx=0).stop 2023-12-07T10:18:11.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.734+0000 7f3d5a7ff700 1 -- 172.21.15.119:0/3655435289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d54103150 msgr2=0x7f3d54073f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:11.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.734+0000 7f3d5a7ff700 1 --2- 172.21.15.119:0/3655435289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d54103150 0x7f3d54073f40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c007ee0 tx=0x7f3d4c00d3b0 comp rx=0 tx=0).stop 2023-12-07T10:18:11.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.734+0000 7f3d5a7ff700 1 -- 172.21.15.119:0/3655435289 shutdown_connections 2023-12-07T10:18:11.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.734+0000 7f3d5a7ff700 1 --2- 172.21.15.119:0/3655435289 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3d3c0a1b90 0x7f3d3c0a4050 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:11.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.734+0000 7f3d5a7ff700 1 --2- 172.21.15.119:0/3655435289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d54103150 0x7f3d54073f40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:11.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.734+0000 7f3d5a7ff700 1 --2- 172.21.15.119:0/3655435289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d541015c0 0x7f3d54075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:11.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.734+0000 7f3d5a7ff700 1 --2- 172.21.15.119:0/3655435289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d541027b0 0x7f3d54073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:11.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.734+0000 7f3d5a7ff700 1 -- 172.21.15.119:0/3655435289 >> 172.21.15.119:0/3655435289 conn(0x7f3d540fcb10 msgr2=0x7f3d54105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:11.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.735+0000 7f3d5a7ff700 1 -- 172.21.15.119:0/3655435289 shutdown_connections 2023-12-07T10:18:11.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.735+0000 7f3d5a7ff700 1 -- 172.21.15.119:0/3655435289 wait complete. 2023-12-07T10:18:11.761 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2023-12-07T10:18:11.761 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-12-07T10:18:11.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.965+0000 7fb800bff700 1 -- 172.21.15.119:0/65436987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7fc102090 msgr2=0x7fb7fc104480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:11.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.965+0000 7fb800bff700 1 --2- 172.21.15.119:0/65436987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7fc102090 0x7fb7fc104480 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec009990 tx=0x7fb7ec009ca0 comp rx=0 tx=0).stop 2023-12-07T10:18:11.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.965+0000 7fb800bff700 1 -- 172.21.15.119:0/65436987 shutdown_connections 2023-12-07T10:18:11.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.965+0000 7fb800bff700 1 --2- 172.21.15.119:0/65436987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7fc1072f0 0x7fb7fc109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:11.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.965+0000 7fb800bff700 1 --2- 172.21.15.119:0/65436987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7fc102090 0x7fb7fc104480 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:11.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.965+0000 7fb800bff700 1 --2- 172.21.15.119:0/65436987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7fc1049c0 0x7fb7fc106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:11.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.965+0000 7fb800bff700 1 -- 172.21.15.119:0/65436987 >> 172.21.15.119:0/65436987 conn(0x7fb7fc0fba20 msgr2=0x7fb7fc0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:11.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.965+0000 7fb800bff700 1 -- 172.21.15.119:0/65436987 shutdown_connections 2023-12-07T10:18:11.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.965+0000 7fb800bff700 1 -- 172.21.15.119:0/65436987 wait complete. 2023-12-07T10:18:11.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.966+0000 7fb800bff700 1 Processor -- start 2023-12-07T10:18:11.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.966+0000 7fb800bff700 1 -- start start 2023-12-07T10:18:11.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.966+0000 7fb800bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7fc102090 0x7fb7fc1019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:11.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.966+0000 7fb800bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7fc1049c0 0x7fb7fc100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:11.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.966+0000 7fb800bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7fc1072f0 0x7fb7fc100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:11.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.966+0000 7fb800bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc100d20 con 0x7fb7fc1049c0 2023-12-07T10:18:11.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.966+0000 7fb800bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc100e90 con 0x7fb7fc102090 2023-12-07T10:18:11.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.966+0000 7fb800bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc1a1580 con 0x7fb7fc1072f0 2023-12-07T10:18:11.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.967+0000 7fb7fa59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7fc102090 0x7fb7fc1019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:11.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.967+0000 7fb7f3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7fc1049c0 0x7fb7fc100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:11.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.967+0000 7fb7fa59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7fc102090 0x7fb7fc1019b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:34296/0 (socket says 172.21.15.119:34296) 2023-12-07T10:18:11.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.967+0000 7fb7fa59c700 1 -- 172.21.15.119:0/240465061 learned_addr learned my addr 172.21.15.119:0/240465061 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:11.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.967+0000 7fb7fad9d700 1 --2- 172.21.15.119:0/240465061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7fc1072f0 0x7fb7fc100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:11.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.967+0000 7fb7fa59c700 1 -- 172.21.15.119:0/240465061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7fc1072f0 msgr2=0x7fb7fc100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:11.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.967+0000 7fb7fa59c700 1 --2- 172.21.15.119:0/240465061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7fc1072f0 0x7fb7fc100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:11.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.967+0000 7fb7fa59c700 1 -- 172.21.15.119:0/240465061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7fc1049c0 msgr2=0x7fb7fc100040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:11.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.967+0000 7fb7fa59c700 1 --2- 172.21.15.119:0/240465061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7fc1049c0 0x7fb7fc100040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:11.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.967+0000 7fb7fa59c700 1 -- 172.21.15.119:0/240465061 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ec0095f0 con 0x7fb7fc102090 2023-12-07T10:18:11.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.967+0000 7fb7f3fff700 1 --2- 172.21.15.119:0/240465061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7fc1049c0 0x7fb7fc100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:18:11.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.967+0000 7fb7fa59c700 1 --2- 172.21.15.119:0/240465061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7fc102090 0x7fb7fc1019b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00b680 tx=0x7fb7ec009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:11.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.968+0000 7fb7f37fe700 1 -- 172.21.15.119:0/240465061 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec01d070 con 0x7fb7fc102090 2023-12-07T10:18:11.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.968+0000 7fb7f37fe700 1 -- 172.21.15.119:0/240465061 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7ec022470 con 0x7fb7fc102090 2023-12-07T10:18:11.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.968+0000 7fb7f37fe700 1 -- 172.21.15.119:0/240465061 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec017650 con 0x7fb7fc102090 2023-12-07T10:18:11.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.968+0000 7fb800bff700 1 -- 172.21.15.119:0/240465061 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7fc1a17b0 con 0x7fb7fc102090 2023-12-07T10:18:11.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.968+0000 7fb800bff700 1 -- 172.21.15.119:0/240465061 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7fc1a1b20 con 0x7fb7fc102090 2023-12-07T10:18:11.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.969+0000 7fb800bff700 1 -- 172.21.15.119:0/240465061 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7fc0fd5a0 con 0x7fb7fc102090 2023-12-07T10:18:11.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.971+0000 7fb7f37fe700 1 -- 172.21.15.119:0/240465061 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fb7ec0225e0 con 0x7fb7fc102090 2023-12-07T10:18:11.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.972+0000 7fb7f37fe700 1 --2- 172.21.15.119:0/240465061 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fb7e80a1b10 0x7fb7e80a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:11.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.972+0000 7fb7f37fe700 1 -- 172.21.15.119:0/240465061 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fb7ec0ced80 con 0x7fb7fc102090 2023-12-07T10:18:11.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.972+0000 7fb7f3fff700 1 --2- 172.21.15.119:0/240465061 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fb7e80a1b10 0x7fb7e80a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:11.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.973+0000 7fb7f3fff700 1 --2- 172.21.15.119:0/240465061 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fb7e80a1b10 0x7fb7e80a3fd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4011490 tx=0x7fb7e4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:11.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:11.974+0000 7fb7f37fe700 1 -- 172.21.15.119:0/240465061 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fb7ec097ed0 con 0x7fb7fc102090 2023-12-07T10:18:12.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.206+0000 7fb800bff700 1 -- 172.21.15.119:0/240465061 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7fc061a50 con 0x7fb7e80a1b10 2023-12-07T10:18:12.217 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:11 smithi119 ceph-mon[97908]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 312 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2023-12-07T10:18:12.217 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:11 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3655435289' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-12-07T10:18:12.217 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:11 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3655435289' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-12-07T10:18:12.217 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:11 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] up:active 2023-12-07T10:18:12.217 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:11 smithi119 ceph-mon[97908]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-12-07T10:18:12.217 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:11 smithi119 ceph-mon[97908]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2023-12-07T10:18:12.217 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:11 smithi119 ceph-mon[97908]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=c=up:active,3=f=up:creating} 2 up:standby-replay 6 up:standby 2023-12-07T10:18:12.217 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:11 smithi119 ceph-mon[97908]: daemon mds.f is now active in filesystem cephfs as rank 3 2023-12-07T10:18:12.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.255+0000 7fb7f37fe700 1 -- 172.21.15.119:0/240465061 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb7fc061a50 con 0x7fb7e80a1b10 2023-12-07T10:18:12.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.258+0000 7fb800bff700 1 -- 172.21.15.119:0/240465061 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fb7e80a1b10 msgr2=0x7fb7e80a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:12.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.258+0000 7fb800bff700 1 --2- 172.21.15.119:0/240465061 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fb7e80a1b10 0x7fb7e80a3fd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4011490 tx=0x7fb7e4009260 comp rx=0 tx=0).stop 2023-12-07T10:18:12.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.258+0000 7fb800bff700 1 -- 172.21.15.119:0/240465061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7fc102090 msgr2=0x7fb7fc1019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:12.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.258+0000 7fb800bff700 1 --2- 172.21.15.119:0/240465061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7fc102090 0x7fb7fc1019b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00b680 tx=0x7fb7ec009580 comp rx=0 tx=0).stop 2023-12-07T10:18:12.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.258+0000 7fb800bff700 1 -- 172.21.15.119:0/240465061 shutdown_connections 2023-12-07T10:18:12.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.258+0000 7fb800bff700 1 --2- 172.21.15.119:0/240465061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7fc1072f0 0x7fb7fc100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:12.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.258+0000 7fb800bff700 1 --2- 172.21.15.119:0/240465061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7fc102090 0x7fb7fc1019b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:12.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.258+0000 7fb800bff700 1 --2- 172.21.15.119:0/240465061 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fb7e80a1b10 0x7fb7e80a3fd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:12.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.258+0000 7fb800bff700 1 --2- 172.21.15.119:0/240465061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7fc1049c0 0x7fb7fc100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:12.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.258+0000 7fb800bff700 1 -- 172.21.15.119:0/240465061 >> 172.21.15.119:0/240465061 conn(0x7fb7fc0fba20 msgr2=0x7fb7fc102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:12.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.258+0000 7fb800bff700 1 -- 172.21.15.119:0/240465061 shutdown_connections 2023-12-07T10:18:12.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.258+0000 7fb800bff700 1 -- 172.21.15.119:0/240465061 wait complete. 2023-12-07T10:18:12.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:11 smithi159 ceph-mon[134726]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 312 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2023-12-07T10:18:12.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:11 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3655435289' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-12-07T10:18:12.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:11 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3655435289' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-12-07T10:18:12.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:11 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] up:active 2023-12-07T10:18:12.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:11 smithi159 ceph-mon[134726]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-12-07T10:18:12.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:11 smithi159 ceph-mon[134726]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2023-12-07T10:18:12.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:11 smithi159 ceph-mon[134726]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=c=up:active,3=f=up:creating} 2 up:standby-replay 6 up:standby 2023-12-07T10:18:12.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:11 smithi159 ceph-mon[134726]: daemon mds.f is now active in filesystem cephfs as rank 3 2023-12-07T10:18:12.278 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-12-07T10:18:12.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:11 smithi153 ceph-mon[133625]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 312 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2023-12-07T10:18:12.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:11 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3655435289' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-12-07T10:18:12.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:11 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3655435289' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-12-07T10:18:12.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:11 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] up:active 2023-12-07T10:18:12.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:11 smithi153 ceph-mon[133625]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-12-07T10:18:12.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:11 smithi153 ceph-mon[133625]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2023-12-07T10:18:12.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:11 smithi153 ceph-mon[133625]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=c=up:active,3=f=up:creating} 2 up:standby-replay 6 up:standby 2023-12-07T10:18:12.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:11 smithi153 ceph-mon[133625]: daemon mds.f is now active in filesystem cephfs as rank 3 2023-12-07T10:18:12.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.476+0000 7fe4409ff700 1 -- 172.21.15.119:0/4144793382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe43c075650 msgr2=0x7fe43c073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:12.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.476+0000 7fe4409ff700 1 --2- 172.21.15.119:0/4144793382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe43c075650 0x7fe43c073aa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe42c009990 tx=0x7fe42c009ca0 comp rx=0 tx=0).stop 2023-12-07T10:18:12.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.477+0000 7fe4409ff700 1 -- 172.21.15.119:0/4144793382 shutdown_connections 2023-12-07T10:18:12.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.477+0000 7fe4409ff700 1 --2- 172.21.15.119:0/4144793382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe43c109480 0x7fe43c10b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:12.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.477+0000 7fe4409ff700 1 --2- 172.21.15.119:0/4144793382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe43c075650 0x7fe43c073aa0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:12.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.477+0000 7fe4409ff700 1 --2- 172.21.15.119:0/4144793382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe43c074070 0x7fe43c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:12.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.477+0000 7fe4409ff700 1 -- 172.21.15.119:0/4144793382 >> 172.21.15.119:0/4144793382 conn(0x7fe43c0fcb10 msgr2=0x7fe43c0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:12.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.477+0000 7fe4409ff700 1 -- 172.21.15.119:0/4144793382 shutdown_connections 2023-12-07T10:18:12.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.477+0000 7fe4409ff700 1 -- 172.21.15.119:0/4144793382 wait complete. 2023-12-07T10:18:12.480 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:18:12 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:18:12.213+0000 7f8a57732700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-07T10:18:12.480 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:18:12 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:18:12.213+0000 7f8a57732700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-07T10:18:12.480 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:18:12 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:18:12.213+0000 7f8a57732700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-07T10:18:12.480 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:18:12 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:18:12.213+0000 7f8a57732700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-07T10:18:12.480 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:18:12 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:18:12.213+0000 7f8a57732700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-07T10:18:12.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.478+0000 7fe4409ff700 1 Processor -- start 2023-12-07T10:18:12.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.478+0000 7fe4409ff700 1 -- start start 2023-12-07T10:18:12.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.478+0000 7fe4409ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe43c074070 0x7fe43c199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:12.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.478+0000 7fe4409ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe43c075650 0x7fe43c199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:12.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.478+0000 7fe4409ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe43c109480 0x7fe43c1a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:12.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.478+0000 7fe4409ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c1a1270 con 0x7fe43c075650 2023-12-07T10:18:12.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.478+0000 7fe4409ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c1a13b0 con 0x7fe43c109480 2023-12-07T10:18:12.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.478+0000 7fe4409ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c1a1540 con 0x7fe43c074070 2023-12-07T10:18:12.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.479+0000 7fe439d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe43c075650 0x7fe43c199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:12.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.479+0000 7fe43a59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe43c074070 0x7fe43c199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:12.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.479+0000 7fe439d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe43c075650 0x7fe43c199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50920/0 (socket says 172.21.15.119:50920) 2023-12-07T10:18:12.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.479+0000 7fe439d9b700 1 -- 172.21.15.119:0/1082056023 learned_addr learned my addr 172.21.15.119:0/1082056023 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:12.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.479+0000 7fe43ad9d700 1 --2- 172.21.15.119:0/1082056023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe43c109480 0x7fe43c1a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:12.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.479+0000 7fe43a59c700 1 -- 172.21.15.119:0/1082056023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe43c109480 msgr2=0x7fe43c1a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:12.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.479+0000 7fe43a59c700 1 --2- 172.21.15.119:0/1082056023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe43c109480 0x7fe43c1a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:12.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.479+0000 7fe43a59c700 1 -- 172.21.15.119:0/1082056023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe43c075650 msgr2=0x7fe43c199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:12.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.479+0000 7fe43a59c700 1 --2- 172.21.15.119:0/1082056023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe43c075650 0x7fe43c199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:12.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.479+0000 7fe43a59c700 1 -- 172.21.15.119:0/1082056023 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42c0095f0 con 0x7fe43c074070 2023-12-07T10:18:12.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.479+0000 7fe439d9b700 1 --2- 172.21.15.119:0/1082056023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe43c075650 0x7fe43c199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:18:12.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.479+0000 7fe43a59c700 1 --2- 172.21.15.119:0/1082056023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe43c074070 0x7fe43c199740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0098e0 tx=0x7fe42c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:12.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.479+0000 7fe42b7fe700 1 -- 172.21.15.119:0/1082056023 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c01d070 con 0x7fe43c074070 2023-12-07T10:18:12.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.479+0000 7fe42b7fe700 1 -- 172.21.15.119:0/1082056023 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe42c021410 con 0x7fe43c074070 2023-12-07T10:18:12.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.479+0000 7fe4409ff700 1 -- 172.21.15.119:0/1082056023 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe43c1a17c0 con 0x7fe43c074070 2023-12-07T10:18:12.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.480+0000 7fe42b7fe700 1 -- 172.21.15.119:0/1082056023 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c017650 con 0x7fe43c074070 2023-12-07T10:18:12.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.480+0000 7fe4409ff700 1 -- 172.21.15.119:0/1082056023 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe43c1a1be0 con 0x7fe43c074070 2023-12-07T10:18:12.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.481+0000 7fe4409ff700 1 -- 172.21.15.119:0/1082056023 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe43c108240 con 0x7fe43c074070 2023-12-07T10:18:12.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.483+0000 7fe42b7fe700 1 -- 172.21.15.119:0/1082056023 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fe42c021580 con 0x7fe43c074070 2023-12-07T10:18:12.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.484+0000 7fe42b7fe700 1 --2- 172.21.15.119:0/1082056023 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe4240a1b10 0x7fe4240a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:12.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.484+0000 7fe42b7fe700 1 -- 172.21.15.119:0/1082056023 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fe42c0cf290 con 0x7fe43c074070 2023-12-07T10:18:12.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.484+0000 7fe439d9b700 1 --2- 172.21.15.119:0/1082056023 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe4240a1b10 0x7fe4240a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:12.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.485+0000 7fe439d9b700 1 --2- 172.21.15.119:0/1082056023 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe4240a1b10 0x7fe4240a3fd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe430005950 tx=0x7fe430009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:12.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.486+0000 7fe42b7fe700 1 -- 172.21.15.119:0/1082056023 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe42c0983e0 con 0x7fe43c074070 2023-12-07T10:18:12.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.724+0000 7fe4409ff700 1 -- 172.21.15.119:0/1082056023 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fe43c061a50 con 0x7fe4240a1b10 2023-12-07T10:18:12.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.729+0000 7fe42b7fe700 1 -- 172.21.15.119:0/1082056023 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fe43c061a50 con 0x7fe4240a1b10 2023-12-07T10:18:12.731 INFO:teuthology.orchestra.run.smithi119.stdout:/volumes/_nogroup/sv_0/748e819e-db1d-4a24-9aa5-ed46fc6ad582 2023-12-07T10:18:12.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.731+0000 7fe4409ff700 1 -- 172.21.15.119:0/1082056023 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe4240a1b10 msgr2=0x7fe4240a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:12.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.731+0000 7fe4409ff700 1 --2- 172.21.15.119:0/1082056023 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe4240a1b10 0x7fe4240a3fd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe430005950 tx=0x7fe430009260 comp rx=0 tx=0).stop 2023-12-07T10:18:12.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.731+0000 7fe4409ff700 1 -- 172.21.15.119:0/1082056023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe43c074070 msgr2=0x7fe43c199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:12.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.731+0000 7fe4409ff700 1 --2- 172.21.15.119:0/1082056023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe43c074070 0x7fe43c199740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0098e0 tx=0x7fe42c009580 comp rx=0 tx=0).stop 2023-12-07T10:18:12.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.731+0000 7fe4409ff700 1 -- 172.21.15.119:0/1082056023 shutdown_connections 2023-12-07T10:18:12.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.731+0000 7fe4409ff700 1 --2- 172.21.15.119:0/1082056023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe43c109480 0x7fe43c1a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:12.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.731+0000 7fe4409ff700 1 --2- 172.21.15.119:0/1082056023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe43c074070 0x7fe43c199740 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:12.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.731+0000 7fe4409ff700 1 --2- 172.21.15.119:0/1082056023 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe4240a1b10 0x7fe4240a3fd0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:12.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.731+0000 7fe4409ff700 1 --2- 172.21.15.119:0/1082056023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe43c075650 0x7fe43c199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:12.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.731+0000 7fe4409ff700 1 -- 172.21.15.119:0/1082056023 >> 172.21.15.119:0/1082056023 conn(0x7fe43c0fcb10 msgr2=0x7fe43c105380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:12.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.731+0000 7fe4409ff700 1 -- 172.21.15.119:0/1082056023 shutdown_connections 2023-12-07T10:18:12.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.731+0000 7fe4409ff700 1 -- 172.21.15.119:0/1082056023 wait complete. 2023-12-07T10:18:12.756 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-12-07T10:18:12.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.958+0000 7f02459ff700 1 -- 172.21.15.119:0/332943573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0240075650 msgr2=0x7f0240073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:12.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.958+0000 7f02459ff700 1 --2- 172.21.15.119:0/332943573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0240075650 0x7f0240073aa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f02300098f0 tx=0x7f0230009c00 comp rx=0 tx=0).stop 2023-12-07T10:18:12.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.958+0000 7f02459ff700 1 -- 172.21.15.119:0/332943573 shutdown_connections 2023-12-07T10:18:12.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.958+0000 7f02459ff700 1 --2- 172.21.15.119:0/332943573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f024010db90 0x7f024010ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:12.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.958+0000 7f02459ff700 1 --2- 172.21.15.119:0/332943573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0240075650 0x7f0240073aa0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:12.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.958+0000 7f02459ff700 1 --2- 172.21.15.119:0/332943573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0240074070 0x7f02400744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:12.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.958+0000 7f02459ff700 1 -- 172.21.15.119:0/332943573 >> 172.21.15.119:0/332943573 conn(0x7f02400fcb10 msgr2=0x7f02400fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:12.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.959+0000 7f02459ff700 1 -- 172.21.15.119:0/332943573 shutdown_connections 2023-12-07T10:18:12.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.959+0000 7f02459ff700 1 -- 172.21.15.119:0/332943573 wait complete. 2023-12-07T10:18:12.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.960+0000 7f02459ff700 1 Processor -- start 2023-12-07T10:18:12.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.960+0000 7f02459ff700 1 -- start start 2023-12-07T10:18:12.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.960+0000 7f02459ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0240074070 0x7f024019db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:12.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.960+0000 7f02459ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0240075650 0x7f024019e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:12.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.960+0000 7f02459ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f024010db90 0x7f02401a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:12.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.960+0000 7f02459ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02401a56c0 con 0x7f024010db90 2023-12-07T10:18:12.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.960+0000 7f02459ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02401a5800 con 0x7f0240075650 2023-12-07T10:18:12.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.960+0000 7f02459ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02401a5990 con 0x7f0240074070 2023-12-07T10:18:12.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.960+0000 7f023f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f024010db90 0x7f02401a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:12.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.960+0000 7f023effd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0240074070 0x7f024019db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:12.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.960+0000 7f023f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f024010db90 0x7f02401a50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53998/0 (socket says 172.21.15.119:53998) 2023-12-07T10:18:12.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.960+0000 7f023f7fe700 1 -- 172.21.15.119:0/2037924762 learned_addr learned my addr 172.21.15.119:0/2037924762 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:12.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.960+0000 7f023e7fc700 1 --2- 172.21.15.119:0/2037924762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0240075650 0x7f024019e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:12.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.961+0000 7f023f7fe700 1 -- 172.21.15.119:0/2037924762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0240074070 msgr2=0x7f024019db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:12.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.961+0000 7f023f7fe700 1 --2- 172.21.15.119:0/2037924762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0240074070 0x7f024019db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:12.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.961+0000 7f023f7fe700 1 -- 172.21.15.119:0/2037924762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0240075650 msgr2=0x7f024019e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:12.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.961+0000 7f023f7fe700 1 --2- 172.21.15.119:0/2037924762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0240075650 0x7f024019e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:12.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.961+0000 7f023f7fe700 1 -- 172.21.15.119:0/2037924762 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0230009520 con 0x7f024010db90 2023-12-07T10:18:12.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.961+0000 7f023e7fc700 1 --2- 172.21.15.119:0/2037924762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0240075650 0x7f024019e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:18:12.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.961+0000 7f023effd700 1 --2- 172.21.15.119:0/2037924762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0240074070 0x7f024019db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:18:12.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.961+0000 7f023f7fe700 1 --2- 172.21.15.119:0/2037924762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f024010db90 0x7f02401a50f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f023800c110 tx=0x7f023800c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:12.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.961+0000 7f02449fd700 1 -- 172.21.15.119:0/2037924762 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0238017070 con 0x7f024010db90 2023-12-07T10:18:12.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.961+0000 7f02459ff700 1 -- 172.21.15.119:0/2037924762 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02401a5c10 con 0x7f024010db90 2023-12-07T10:18:12.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.961+0000 7f02449fd700 1 -- 172.21.15.119:0/2037924762 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f023800e050 con 0x7f024010db90 2023-12-07T10:18:12.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.961+0000 7f02449fd700 1 -- 172.21.15.119:0/2037924762 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02380078f0 con 0x7f024010db90 2023-12-07T10:18:12.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.961+0000 7f02459ff700 1 -- 172.21.15.119:0/2037924762 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02401a60d0 con 0x7f024010db90 2023-12-07T10:18:12.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.962+0000 7f02459ff700 1 -- 172.21.15.119:0/2037924762 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0240103df0 con 0x7f024010db90 2023-12-07T10:18:12.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.965+0000 7f02449fd700 1 -- 172.21.15.119:0/2037924762 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f0238007a50 con 0x7f024010db90 2023-12-07T10:18:12.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.966+0000 7f02449fd700 1 --2- 172.21.15.119:0/2037924762 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f02280a1b10 0x7f02280a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:12.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.966+0000 7f02449fd700 1 -- 172.21.15.119:0/2037924762 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f02380cdd60 con 0x7f024010db90 2023-12-07T10:18:12.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.966+0000 7f023effd700 1 --2- 172.21.15.119:0/2037924762 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f02280a1b10 0x7f02280a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:12.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.967+0000 7f023effd700 1 --2- 172.21.15.119:0/2037924762 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f02280a1b10 0x7f02280a3fd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f023000fa40 tx=0x7f0230019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:12.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:12.968+0000 7f02449fd700 1 -- 172.21.15.119:0/2037924762 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0238096e20 con 0x7f024010db90 2023-12-07T10:18:13.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.201+0000 7f02459ff700 1 -- 172.21.15.119:0/2037924762 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f024002d3c0 con 0x7f02280a1b10 2023-12-07T10:18:13.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.227+0000 7f02449fd700 1 -- 172.21.15.119:0/2037924762 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f024002d3c0 con 0x7f02280a1b10 2023-12-07T10:18:13.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.229+0000 7f02459ff700 1 -- 172.21.15.119:0/2037924762 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f02280a1b10 msgr2=0x7f02280a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:13.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.229+0000 7f02459ff700 1 --2- 172.21.15.119:0/2037924762 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f02280a1b10 0x7f02280a3fd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f023000fa40 tx=0x7f0230019040 comp rx=0 tx=0).stop 2023-12-07T10:18:13.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.229+0000 7f02459ff700 1 -- 172.21.15.119:0/2037924762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f024010db90 msgr2=0x7f02401a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:13.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.229+0000 7f02459ff700 1 --2- 172.21.15.119:0/2037924762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f024010db90 0x7f02401a50f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f023800c110 tx=0x7f023800c420 comp rx=0 tx=0).stop 2023-12-07T10:18:13.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.229+0000 7f02459ff700 1 -- 172.21.15.119:0/2037924762 shutdown_connections 2023-12-07T10:18:13.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.229+0000 7f02459ff700 1 --2- 172.21.15.119:0/2037924762 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f02280a1b10 0x7f02280a3fd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:13.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.229+0000 7f02459ff700 1 --2- 172.21.15.119:0/2037924762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f024010db90 0x7f02401a50f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:13.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.229+0000 7f02459ff700 1 --2- 172.21.15.119:0/2037924762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0240074070 0x7f024019db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:13.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.229+0000 7f02459ff700 1 --2- 172.21.15.119:0/2037924762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0240075650 0x7f024019e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:13.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.229+0000 7f02459ff700 1 -- 172.21.15.119:0/2037924762 >> 172.21.15.119:0/2037924762 conn(0x7f02400fcb10 msgr2=0x7f0240100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:13.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.230+0000 7f02459ff700 1 -- 172.21.15.119:0/2037924762 shutdown_connections 2023-12-07T10:18:13.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.230+0000 7f02459ff700 1 -- 172.21.15.119:0/2037924762 wait complete. 2023-12-07T10:18:13.250 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-12-07T10:18:13.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.449+0000 7f51ef3ff700 1 -- 172.21.15.119:0/2237841816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51e8103170 msgr2=0x7f51e8109a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:13.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.449+0000 7f51ef3ff700 1 --2- 172.21.15.119:0/2237841816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51e8103170 0x7f51e8109a20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f51e000b0d0 tx=0x7f51e000b3e0 comp rx=0 tx=0).stop 2023-12-07T10:18:13.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.449+0000 7f51ef3ff700 1 -- 172.21.15.119:0/2237841816 shutdown_connections 2023-12-07T10:18:13.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.449+0000 7f51ef3ff700 1 --2- 172.21.15.119:0/2237841816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51e8103170 0x7f51e8109a20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:13.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.449+0000 7f51ef3ff700 1 --2- 172.21.15.119:0/2237841816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51e81015e0 0x7f51e8101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:13.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.449+0000 7f51ef3ff700 1 --2- 172.21.15.119:0/2237841816 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51e81027d0 0x7f51e8102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:13.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.449+0000 7f51ef3ff700 1 -- 172.21.15.119:0/2237841816 >> 172.21.15.119:0/2237841816 conn(0x7f51e80fcb50 msgr2=0x7f51e80fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:13.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.450+0000 7f51ef3ff700 1 -- 172.21.15.119:0/2237841816 shutdown_connections 2023-12-07T10:18:13.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.450+0000 7f51ef3ff700 1 -- 172.21.15.119:0/2237841816 wait complete. 2023-12-07T10:18:13.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.450+0000 7f51ef3ff700 1 Processor -- start 2023-12-07T10:18:13.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.451+0000 7f51ef3ff700 1 -- start start 2023-12-07T10:18:13.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.451+0000 7f51ef3ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51e81015e0 0x7f51e8075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:13.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.451+0000 7f51ef3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51e81027d0 0x7f51e8073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:13.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.451+0000 7f51ef3ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51e8103170 0x7f51e8073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:13.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.451+0000 7f51ef3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e80746e0 con 0x7f51e81027d0 2023-12-07T10:18:13.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.451+0000 7f51ef3ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e8074850 con 0x7f51e8103170 2023-12-07T10:18:13.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.451+0000 7f51ef3ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e81a16b0 con 0x7f51e81015e0 2023-12-07T10:18:13.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.451+0000 7f51ed19b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51e81015e0 0x7f51e8075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:13.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.451+0000 7f51ec99a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51e81027d0 0x7f51e8073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:13.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.451+0000 7f51ed19b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51e81015e0 0x7f51e8075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:60304/0 (socket says 172.21.15.119:60304) 2023-12-07T10:18:13.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.451+0000 7f51ed19b700 1 -- 172.21.15.119:0/1860937026 learned_addr learned my addr 172.21.15.119:0/1860937026 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:13.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.451+0000 7f51ed99c700 1 --2- 172.21.15.119:0/1860937026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51e8103170 0x7f51e8073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:13.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.452+0000 7f51ed19b700 1 -- 172.21.15.119:0/1860937026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51e8103170 msgr2=0x7f51e8073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:13.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.452+0000 7f51ed19b700 1 --2- 172.21.15.119:0/1860937026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51e8103170 0x7f51e8073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:13.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.452+0000 7f51ed19b700 1 -- 172.21.15.119:0/1860937026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51e81027d0 msgr2=0x7f51e8073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:13.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.452+0000 7f51ed19b700 1 --2- 172.21.15.119:0/1860937026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51e81027d0 0x7f51e8073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:13.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.452+0000 7f51ed19b700 1 -- 172.21.15.119:0/1860937026 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e0009d00 con 0x7f51e81015e0 2023-12-07T10:18:13.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.452+0000 7f51ed99c700 1 --2- 172.21.15.119:0/1860937026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51e8103170 0x7f51e8073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:18:13.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.452+0000 7f51ec99a700 1 --2- 172.21.15.119:0/1860937026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51e81027d0 0x7f51e8073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:18:13.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.452+0000 7f51ed19b700 1 --2- 172.21.15.119:0/1860937026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51e81015e0 0x7f51e8075370 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f51e400e970 tx=0x7f51e400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:13.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.452+0000 7f51de7fc700 1 -- 172.21.15.119:0/1860937026 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e4009790 con 0x7f51e81015e0 2023-12-07T10:18:13.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.452+0000 7f51ef3ff700 1 -- 172.21.15.119:0/1860937026 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e81a18e0 con 0x7f51e81015e0 2023-12-07T10:18:13.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.452+0000 7f51de7fc700 1 -- 172.21.15.119:0/1860937026 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51e400eec0 con 0x7f51e81015e0 2023-12-07T10:18:13.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.452+0000 7f51de7fc700 1 -- 172.21.15.119:0/1860937026 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e4003cf0 con 0x7f51e81015e0 2023-12-07T10:18:13.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.452+0000 7f51ef3ff700 1 -- 172.21.15.119:0/1860937026 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e81a1e40 con 0x7f51e81015e0 2023-12-07T10:18:13.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.453+0000 7f51ef3ff700 1 -- 172.21.15.119:0/1860937026 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e8108990 con 0x7f51e81015e0 2023-12-07T10:18:13.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.455+0000 7f51de7fc700 1 -- 172.21.15.119:0/1860937026 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f51e400ccd0 con 0x7f51e81015e0 2023-12-07T10:18:13.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.457+0000 7f51de7fc700 1 --2- 172.21.15.119:0/1860937026 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f51d40a1b10 0x7f51d40a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:13.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.457+0000 7f51de7fc700 1 -- 172.21.15.119:0/1860937026 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f51e4014070 con 0x7f51e81015e0 2023-12-07T10:18:13.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.457+0000 7f51ec99a700 1 --2- 172.21.15.119:0/1860937026 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f51d40a1b10 0x7f51d40a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:13.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.457+0000 7f51ec99a700 1 --2- 172.21.15.119:0/1860937026 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f51d40a1b10 0x7f51d40a3fd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f51d8011490 tx=0x7f51d8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:13.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.459+0000 7f51de7fc700 1 -- 172.21.15.119:0/1860937026 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f51e4096d70 con 0x7f51e81015e0 2023-12-07T10:18:13.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.708+0000 7f51ef3ff700 1 -- 172.21.15.119:0/1860937026 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f51e8061a50 con 0x7f51d40a1b10 2023-12-07T10:18:13.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.713+0000 7f51de7fc700 1 -- 172.21.15.119:0/1860937026 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f51e8061a50 con 0x7f51d40a1b10 2023-12-07T10:18:13.715 INFO:teuthology.orchestra.run.smithi119.stdout:/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69 2023-12-07T10:18:13.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.715+0000 7f51ef3ff700 1 -- 172.21.15.119:0/1860937026 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f51d40a1b10 msgr2=0x7f51d40a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:13.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.715+0000 7f51ef3ff700 1 --2- 172.21.15.119:0/1860937026 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f51d40a1b10 0x7f51d40a3fd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f51d8011490 tx=0x7f51d8009260 comp rx=0 tx=0).stop 2023-12-07T10:18:13.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.716+0000 7f51ef3ff700 1 -- 172.21.15.119:0/1860937026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51e81015e0 msgr2=0x7f51e8075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:13.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.716+0000 7f51ef3ff700 1 --2- 172.21.15.119:0/1860937026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51e81015e0 0x7f51e8075370 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f51e400e970 tx=0x7f51e400ec80 comp rx=0 tx=0).stop 2023-12-07T10:18:13.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.716+0000 7f51ef3ff700 1 -- 172.21.15.119:0/1860937026 shutdown_connections 2023-12-07T10:18:13.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.716+0000 7f51ef3ff700 1 --2- 172.21.15.119:0/1860937026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51e8103170 0x7f51e8073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:13.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.716+0000 7f51ef3ff700 1 --2- 172.21.15.119:0/1860937026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51e81015e0 0x7f51e8075370 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:13.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.716+0000 7f51ef3ff700 1 --2- 172.21.15.119:0/1860937026 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f51d40a1b10 0x7f51d40a3fd0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:13.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.716+0000 7f51ef3ff700 1 --2- 172.21.15.119:0/1860937026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51e81027d0 0x7f51e8073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:13.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.716+0000 7f51ef3ff700 1 -- 172.21.15.119:0/1860937026 >> 172.21.15.119:0/1860937026 conn(0x7f51e80fcb50 msgr2=0x7f51e8105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:13.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.716+0000 7f51ef3ff700 1 -- 172.21.15.119:0/1860937026 shutdown_connections 2023-12-07T10:18:13.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.716+0000 7f51ef3ff700 1 -- 172.21.15.119:0/1860937026 wait complete. 2023-12-07T10:18:13.736 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:18:13.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.940+0000 7f3e9e9ff700 1 -- 172.21.15.119:0/2736208750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e981030f0 msgr2=0x7f3e981099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:13.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.940+0000 7f3e9e9ff700 1 --2- 172.21.15.119:0/2736208750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e981030f0 0x7f3e981099a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000b0d0 tx=0x7f3e9000b3e0 comp rx=0 tx=0).stop 2023-12-07T10:18:13.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.941+0000 7f3e9e9ff700 1 -- 172.21.15.119:0/2736208750 shutdown_connections 2023-12-07T10:18:13.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.941+0000 7f3e9e9ff700 1 --2- 172.21.15.119:0/2736208750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e981030f0 0x7f3e981099a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:13.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.941+0000 7f3e9e9ff700 1 --2- 172.21.15.119:0/2736208750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e98101560 0x7f3e98101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:13.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.941+0000 7f3e9e9ff700 1 --2- 172.21.15.119:0/2736208750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e98102750 0x7f3e98102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:13.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.941+0000 7f3e9e9ff700 1 -- 172.21.15.119:0/2736208750 >> 172.21.15.119:0/2736208750 conn(0x7f3e980fcab0 msgr2=0x7f3e980fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:13.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.941+0000 7f3e9e9ff700 1 -- 172.21.15.119:0/2736208750 shutdown_connections 2023-12-07T10:18:13.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.941+0000 7f3e9e9ff700 1 -- 172.21.15.119:0/2736208750 wait complete. 2023-12-07T10:18:13.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.942+0000 7f3e9e9ff700 1 Processor -- start 2023-12-07T10:18:13.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.942+0000 7f3e9e9ff700 1 -- start start 2023-12-07T10:18:13.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.942+0000 7f3e9e9ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e98101560 0x7f3e98075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:13.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.942+0000 7f3e9e9ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e98102750 0x7f3e98073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:13.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.942+0000 7f3e9e9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e981030f0 0x7f3e98073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:13.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.942+0000 7f3e9e9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e980746e0 con 0x7f3e981030f0 2023-12-07T10:18:13.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.942+0000 7f3e9e9ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e98074850 con 0x7f3e98101560 2023-12-07T10:18:13.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.942+0000 7f3e9e9ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e981a1660 con 0x7f3e98102750 2023-12-07T10:18:13.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.942+0000 7f3e97fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e98101560 0x7f3e98075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:13.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.942+0000 7f3e9cf9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e981030f0 0x7f3e98073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:13.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.942+0000 7f3e97fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e98101560 0x7f3e98075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:34772/0 (socket says 172.21.15.119:34772) 2023-12-07T10:18:13.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.942+0000 7f3e97fff700 1 -- 172.21.15.119:0/3870923656 learned_addr learned my addr 172.21.15.119:0/3870923656 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:13.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.942+0000 7f3e977fe700 1 --2- 172.21.15.119:0/3870923656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e98102750 0x7f3e98073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:13.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.943+0000 7f3e97fff700 1 -- 172.21.15.119:0/3870923656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e98102750 msgr2=0x7f3e98073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:13.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.943+0000 7f3e97fff700 1 --2- 172.21.15.119:0/3870923656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e98102750 0x7f3e98073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:13.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.943+0000 7f3e97fff700 1 -- 172.21.15.119:0/3870923656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e981030f0 msgr2=0x7f3e98073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:13.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.943+0000 7f3e97fff700 1 --2- 172.21.15.119:0/3870923656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e981030f0 0x7f3e98073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:13.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.943+0000 7f3e97fff700 1 -- 172.21.15.119:0/3870923656 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e90009d00 con 0x7f3e98101560 2023-12-07T10:18:13.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.943+0000 7f3e977fe700 1 --2- 172.21.15.119:0/3870923656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e98102750 0x7f3e98073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:18:13.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.943+0000 7f3e9cf9c700 1 --2- 172.21.15.119:0/3870923656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e981030f0 0x7f3e98073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:18:13.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.943+0000 7f3e97fff700 1 --2- 172.21.15.119:0/3870923656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e98101560 0x7f3e98075370 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3e8800e970 tx=0x7f3e8800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:13.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.943+0000 7f3e957fa700 1 -- 172.21.15.119:0/3870923656 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e88009790 con 0x7f3e98101560 2023-12-07T10:18:13.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.943+0000 7f3e9e9ff700 1 -- 172.21.15.119:0/3870923656 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e981a18f0 con 0x7f3e98101560 2023-12-07T10:18:13.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.943+0000 7f3e957fa700 1 -- 172.21.15.119:0/3870923656 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e8800c780 con 0x7f3e98101560 2023-12-07T10:18:13.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.943+0000 7f3e957fa700 1 -- 172.21.15.119:0/3870923656 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e88003be0 con 0x7f3e98101560 2023-12-07T10:18:13.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.943+0000 7f3e9e9ff700 1 -- 172.21.15.119:0/3870923656 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e981a1e50 con 0x7f3e98101560 2023-12-07T10:18:13.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.944+0000 7f3e9e9ff700 1 -- 172.21.15.119:0/3870923656 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e98108910 con 0x7f3e98101560 2023-12-07T10:18:13.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.946+0000 7f3e957fa700 1 -- 172.21.15.119:0/3870923656 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f3e8800cbf0 con 0x7f3e98101560 2023-12-07T10:18:13.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.948+0000 7f3e957fa700 1 --2- 172.21.15.119:0/3870923656 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3e800a1ba0 0x7f3e800a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:13.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.948+0000 7f3e957fa700 1 -- 172.21.15.119:0/3870923656 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3e88014070 con 0x7f3e98101560 2023-12-07T10:18:13.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.948+0000 7f3e977fe700 1 --2- 172.21.15.119:0/3870923656 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3e800a1ba0 0x7f3e800a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:13.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.949+0000 7f3e977fe700 1 --2- 172.21.15.119:0/3870923656 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3e800a1ba0 0x7f3e800a4060 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c011490 tx=0x7f3e8c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:13.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:13.951+0000 7f3e957fa700 1 -- 172.21.15.119:0/3870923656 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3e88096be0 con 0x7f3e98101560 2023-12-07T10:18:14.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:13 smithi159 ceph-mon[134726]: from='client.24812 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:18:14.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:13 smithi159 ceph-mon[134726]: pgmap v297: 129 pgs: 129 active+clean; 582 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 567 B/s rd, 1.9 KiB/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2023-12-07T10:18:14.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:13 smithi159 ceph-mon[134726]: from='client.24829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:18:14.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:13 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] up:active 2023-12-07T10:18:14.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:13 smithi159 ceph-mon[134726]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-12-07T10:18:14.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:13 smithi159 ceph-mon[134726]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-12-07T10:18:14.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:13 smithi159 ceph-mon[134726]: Cluster is now healthy 2023-12-07T10:18:14.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:13 smithi159 ceph-mon[134726]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=c=up:active,3=f=up:active} 2 up:standby-replay 6 up:standby 2023-12-07T10:18:14.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:13 smithi159 ceph-mon[134726]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2023-12-07T10:18:14.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:13 smithi159 ceph-mon[134726]: daemon mds.e is now active in filesystem cephfs as rank 4 2023-12-07T10:18:14.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:13 smithi153 ceph-mon[133625]: from='client.24812 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:18:14.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:13 smithi153 ceph-mon[133625]: pgmap v297: 129 pgs: 129 active+clean; 582 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 567 B/s rd, 1.9 KiB/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2023-12-07T10:18:14.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:13 smithi153 ceph-mon[133625]: from='client.24829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:18:14.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:13 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] up:active 2023-12-07T10:18:14.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:13 smithi153 ceph-mon[133625]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-12-07T10:18:14.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:13 smithi153 ceph-mon[133625]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-12-07T10:18:14.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:13 smithi153 ceph-mon[133625]: Cluster is now healthy 2023-12-07T10:18:14.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:13 smithi153 ceph-mon[133625]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=c=up:active,3=f=up:active} 2 up:standby-replay 6 up:standby 2023-12-07T10:18:14.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:13 smithi153 ceph-mon[133625]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2023-12-07T10:18:14.031 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:13 smithi153 ceph-mon[133625]: daemon mds.e is now active in filesystem cephfs as rank 4 2023-12-07T10:18:14.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.195+0000 7f3e9e9ff700 1 -- 172.21.15.119:0/3870923656 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e98068350 con 0x7f3e98101560 2023-12-07T10:18:14.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.196+0000 7f3e957fa700 1 -- 172.21.15.119:0/3870923656 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11384 (secure 0 0 0) 0x7f3e8809a200 con 0x7f3e98101560 2023-12-07T10:18:14.198 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:18:14.198 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-07T10:17:41.278311+0000","modified":"2023-12-07T10:18:13.752388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24517,"mds_3":24505,"mds_4":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14535":{"gid":14535,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6841/1791998028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1791998028},{"type":"v1","addr":"172.21.15.119:6841","nonce":1791998028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24505":{"gid":24505,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/2925503246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2925503246},{"type":"v1","addr":"172.21.15.153:6837","nonce":2925503246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6839/3984962900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3984962900},{"type":"v1","addr":"172.21.15.159:6839","nonce":3984962900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.119:6839/1920762602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1920762602},{"type":"v1","addr":"172.21.15.119:6839","nonce":1920762602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/566825184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":566825184},{"type":"v1","addr":"172.21.15.153:6839","nonce":566825184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-07T10:18:14.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.199+0000 7f3e9e9ff700 1 -- 172.21.15.119:0/3870923656 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3e800a1ba0 msgr2=0x7f3e800a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:14.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.199+0000 7f3e9e9ff700 1 --2- 172.21.15.119:0/3870923656 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3e800a1ba0 0x7f3e800a4060 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c011490 tx=0x7f3e8c009260 comp rx=0 tx=0).stop 2023-12-07T10:18:14.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.199+0000 7f3e9e9ff700 1 -- 172.21.15.119:0/3870923656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e98101560 msgr2=0x7f3e98075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:14.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.199+0000 7f3e9e9ff700 1 --2- 172.21.15.119:0/3870923656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e98101560 0x7f3e98075370 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3e8800e970 tx=0x7f3e8800ec80 comp rx=0 tx=0).stop 2023-12-07T10:18:14.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.199+0000 7f3e9e9ff700 1 -- 172.21.15.119:0/3870923656 shutdown_connections 2023-12-07T10:18:14.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.199+0000 7f3e9e9ff700 1 --2- 172.21.15.119:0/3870923656 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3e800a1ba0 0x7f3e800a4060 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:14.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.199+0000 7f3e9e9ff700 1 --2- 172.21.15.119:0/3870923656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e981030f0 0x7f3e98073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:14.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.199+0000 7f3e9e9ff700 1 --2- 172.21.15.119:0/3870923656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e98101560 0x7f3e98075370 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:14.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.199+0000 7f3e9e9ff700 1 --2- 172.21.15.119:0/3870923656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e98102750 0x7f3e98073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:14.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.199+0000 7f3e9e9ff700 1 -- 172.21.15.119:0/3870923656 >> 172.21.15.119:0/3870923656 conn(0x7f3e980fcab0 msgr2=0x7f3e98105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:14.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.199+0000 7f3e9e9ff700 1 -- 172.21.15.119:0/3870923656 shutdown_connections 2023-12-07T10:18:14.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.199+0000 7f3e9e9ff700 1 -- 172.21.15.119:0/3870923656 wait complete. 2023-12-07T10:18:14.202 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 28 2023-12-07T10:18:14.217 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:13 smithi119 ceph-mon[97908]: from='client.24812 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:18:14.217 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:13 smithi119 ceph-mon[97908]: pgmap v297: 129 pgs: 129 active+clean; 582 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 567 B/s rd, 1.9 KiB/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2023-12-07T10:18:14.217 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:13 smithi119 ceph-mon[97908]: from='client.24829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:18:14.217 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:13 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] up:active 2023-12-07T10:18:14.217 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:13 smithi119 ceph-mon[97908]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-12-07T10:18:14.218 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:13 smithi119 ceph-mon[97908]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-12-07T10:18:14.218 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:13 smithi119 ceph-mon[97908]: Cluster is now healthy 2023-12-07T10:18:14.218 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:13 smithi119 ceph-mon[97908]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=c=up:active,3=f=up:active} 2 up:standby-replay 6 up:standby 2023-12-07T10:18:14.218 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:13 smithi119 ceph-mon[97908]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2023-12-07T10:18:14.218 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:13 smithi119 ceph-mon[97908]: daemon mds.e is now active in filesystem cephfs as rank 4 2023-12-07T10:18:14.219 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-07T10:18:14.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.415+0000 7f807e1ff700 1 -- 172.21.15.119:0/1921661874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8078101560 msgr2=0x7f8078101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:14.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.415+0000 7f807e1ff700 1 --2- 172.21.15.119:0/1921661874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8078101560 0x7f8078101980 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f806800b0d0 tx=0x7f806800b3e0 comp rx=0 tx=0).stop 2023-12-07T10:18:14.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.415+0000 7f807e1ff700 1 -- 172.21.15.119:0/1921661874 shutdown_connections 2023-12-07T10:18:14.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.415+0000 7f807e1ff700 1 --2- 172.21.15.119:0/1921661874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80781030f0 0x7f80781099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:14.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.415+0000 7f807e1ff700 1 --2- 172.21.15.119:0/1921661874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8078101560 0x7f8078101980 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:14.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.415+0000 7f807e1ff700 1 --2- 172.21.15.119:0/1921661874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8078102750 0x7f8078102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:14.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.415+0000 7f807e1ff700 1 -- 172.21.15.119:0/1921661874 >> 172.21.15.119:0/1921661874 conn(0x7f80780fcab0 msgr2=0x7f80780fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:14.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.415+0000 7f807e1ff700 1 -- 172.21.15.119:0/1921661874 shutdown_connections 2023-12-07T10:18:14.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.415+0000 7f807e1ff700 1 -- 172.21.15.119:0/1921661874 wait complete. 2023-12-07T10:18:14.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.416+0000 7f807e1ff700 1 Processor -- start 2023-12-07T10:18:14.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.416+0000 7f807e1ff700 1 -- start start 2023-12-07T10:18:14.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.416+0000 7f807e1ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8078101560 0x7f8078075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:14.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.416+0000 7f807e1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8078102750 0x7f8078073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:14.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.416+0000 7f807e1ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80781030f0 0x7f8078073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:14.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.416+0000 7f807e1ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80780746e0 con 0x7f80781030f0 2023-12-07T10:18:14.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.416+0000 7f807e1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8078074850 con 0x7f8078102750 2023-12-07T10:18:14.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.416+0000 7f807e1ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80781a1700 con 0x7f8078101560 2023-12-07T10:18:14.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.417+0000 7f8077fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80781030f0 0x7f8078073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:14.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.417+0000 7f8077fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80781030f0 0x7f8078073f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54046/0 (socket says 172.21.15.119:54046) 2023-12-07T10:18:14.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.417+0000 7f806ffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8078102750 0x7f8078073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:14.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.417+0000 7f8077fff700 1 -- 172.21.15.119:0/2023940432 learned_addr learned my addr 172.21.15.119:0/2023940432 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:14.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.417+0000 7f80777fe700 1 --2- 172.21.15.119:0/2023940432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8078101560 0x7f8078075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:14.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.417+0000 7f8077fff700 1 -- 172.21.15.119:0/2023940432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8078101560 msgr2=0x7f8078075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:14.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.417+0000 7f8077fff700 1 --2- 172.21.15.119:0/2023940432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8078101560 0x7f8078075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:14.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.417+0000 7f8077fff700 1 -- 172.21.15.119:0/2023940432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8078102750 msgr2=0x7f8078073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:14.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.417+0000 7f8077fff700 1 --2- 172.21.15.119:0/2023940432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8078102750 0x7f8078073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:14.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.417+0000 7f8077fff700 1 -- 172.21.15.119:0/2023940432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8068009d00 con 0x7f80781030f0 2023-12-07T10:18:14.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.417+0000 7f8077fff700 1 --2- 172.21.15.119:0/2023940432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80781030f0 0x7f8078073f40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f807000e940 tx=0x7f807000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:14.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.417+0000 7f80757fa700 1 -- 172.21.15.119:0/2023940432 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80700097b0 con 0x7f80781030f0 2023-12-07T10:18:14.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.417+0000 7f80757fa700 1 -- 172.21.15.119:0/2023940432 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8070005020 con 0x7f80781030f0 2023-12-07T10:18:14.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.417+0000 7f807e1ff700 1 -- 172.21.15.119:0/2023940432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80781a1990 con 0x7f80781030f0 2023-12-07T10:18:14.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.418+0000 7f80757fa700 1 -- 172.21.15.119:0/2023940432 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807000cbe0 con 0x7f80781030f0 2023-12-07T10:18:14.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.418+0000 7f807e1ff700 1 -- 172.21.15.119:0/2023940432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80781a1e50 con 0x7f80781030f0 2023-12-07T10:18:14.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.419+0000 7f807e1ff700 1 -- 172.21.15.119:0/2023940432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8078108a50 con 0x7f80781030f0 2023-12-07T10:18:14.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.420+0000 7f80757fa700 1 -- 172.21.15.119:0/2023940432 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f8070005400 con 0x7f80781030f0 2023-12-07T10:18:14.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.422+0000 7f80757fa700 1 --2- 172.21.15.119:0/2023940432 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f80640a1ba0 0x7f80640a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:14.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.422+0000 7f80757fa700 1 -- 172.21.15.119:0/2023940432 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f8070014070 con 0x7f80781030f0 2023-12-07T10:18:14.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.422+0000 7f80777fe700 1 --2- 172.21.15.119:0/2023940432 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f80640a1ba0 0x7f80640a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:14.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.423+0000 7f80777fe700 1 --2- 172.21.15.119:0/2023940432 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f80640a1ba0 0x7f80640a4060 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8068009fd0 tx=0x7f806800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:14.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.426+0000 7f80757fa700 1 -- 172.21.15.119:0/2023940432 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8070096bc0 con 0x7f80781030f0 2023-12-07T10:18:14.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.629+0000 7f807e1ff700 1 -- 172.21.15.119:0/2023940432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8078068350 con 0x7f80781030f0 2023-12-07T10:18:14.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.631+0000 7f80757fa700 1 -- 172.21.15.119:0/2023940432 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24528 (secure 0 0 0) 0x7f8070005190 con 0x7f80781030f0 2023-12-07T10:18:14.633 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:18:14.633 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":91,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","created":"2023-12-07T10:09:59.945863+0000","modified":"2023-12-07T10:18:01.843682+0000","last_up_change":"2023-12-07T10:16:50.929427+0000","last_in_change":"2023-12-07T10:16:33.558158+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T10:13:25.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-07T10:17:39.347813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-07T10:17:40.714501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-07T10:17:44.257760+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"56b5818e-e625-40f1-8838-3ddcf2d34ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6803","nonce":2141950133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6805","nonce":2141950133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6809","nonce":2141950133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6807","nonce":2141950133}]},"public_addr":"172.21.15.119:6803/2141950133","cluster_addr":"172.21.15.119:6805/2141950133","heartbeat_back_addr":"172.21.15.119:6809/2141950133","heartbeat_front_addr":"172.21.15.119:6807/2141950133","state":["exists","up"]},{"osd":1,"uuid":"36489a6a-5278-4802-9525-9871521722e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6801","nonce":3390875370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6803","nonce":3390875370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6807","nonce":3390875370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6805","nonce":3390875370}]},"public_addr":"172.21.15.153:6801/3390875370","cluster_addr":"172.21.15.153:6803/3390875370","heartbeat_back_addr":"172.21.15.153:6807/3390875370","heartbeat_front_addr":"172.21.15.153:6805/3390875370","state":["exists","up"]},{"osd":2,"uuid":"a937ad26-5e9b-4c8a-b65a-626a8feda6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6801","nonce":708101637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6803","nonce":708101637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6807","nonce":708101637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6805","nonce":708101637}]},"public_addr":"172.21.15.159:6801/708101637","cluster_addr":"172.21.15.159:6803/708101637","heartbeat_back_addr":"172.21.15.159:6807/708101637","heartbeat_front_addr":"172.21.15.159:6805/708101637","state":["exists","up"]},{"osd":3,"uuid":"26ef8300-aa07-48f6-b31d-8b0f2217ffcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6811","nonce":3228047257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6813","nonce":3228047257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6817","nonce":3228047257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6815","nonce":3228047257}]},"public_addr":"172.21.15.119:6811/3228047257","cluster_addr":"172.21.15.119:6813/3228047257","heartbeat_back_addr":"172.21.15.119:6817/3228047257","heartbeat_front_addr":"172.21.15.119:6815/3228047257","state":["exists","up"]},{"osd":4,"uuid":"000ee4b9-9896-4901-8cc3-c85fdfc8cbaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6809","nonce":4021998299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6811","nonce":4021998299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6815","nonce":4021998299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6813","nonce":4021998299}]},"public_addr":"172.21.15.153:6809/4021998299","cluster_addr":"172.21.15.153:6811/4021998299","heartbeat_back_addr":"172.21.15.153:6815/4021998299","heartbeat_front_addr":"172.21.15.153:6813/4021998299","state":["exists","up"]},{"osd":5,"uuid":"8b2fc167-262d-474f-89f3-47bd29424421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6809","nonce":1832540768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6811","nonce":1832540768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6815","nonce":1832540768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6813","nonce":1832540768}]},"public_addr":"172.21.15.159:6809/1832540768","cluster_addr":"172.21.15.159:6811/1832540768","heartbeat_back_addr":"172.21.15.159:6815/1832540768","heartbeat_front_addr":"172.21.15.159:6813/1832540768","state":["exists","up"]},{"osd":6,"uuid":"f9aa6749-46cf-4b2a-8114-cbd034219d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162436840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162436840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6825","nonce":1162436840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162436840}]},"public_addr":"172.21.15.119:6819/1162436840","cluster_addr":"172.21.15.119:6821/1162436840","heartbeat_back_addr":"172.21.15.119:6825/1162436840","heartbeat_front_addr":"172.21.15.119:6823/1162436840","state":["exists","up"]},{"osd":7,"uuid":"39216361-e1c4-4fef-86cb-a4707656b547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6817","nonce":3254653457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6819","nonce":3254653457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6823","nonce":3254653457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6821","nonce":3254653457}]},"public_addr":"172.21.15.153:6817/3254653457","cluster_addr":"172.21.15.153:6819/3254653457","heartbeat_back_addr":"172.21.15.153:6823/3254653457","heartbeat_front_addr":"172.21.15.153:6821/3254653457","state":["exists","up"]},{"osd":8,"uuid":"cceb6ebc-7c02-48ea-9d16-b7a016dd449b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6817","nonce":2600946380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6819","nonce":2600946380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6823","nonce":2600946380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6821","nonce":2600946380}]},"public_addr":"172.21.15.159:6817/2600946380","cluster_addr":"172.21.15.159:6819/2600946380","heartbeat_back_addr":"172.21.15.159:6823/2600946380","heartbeat_front_addr":"172.21.15.159:6821/2600946380","state":["exists","up"]},{"osd":9,"uuid":"f337a2e4-e065-4a9f-91d9-e3615de2d6f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6827","nonce":1940737285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6829","nonce":1940737285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6833","nonce":1940737285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6831","nonce":1940737285}]},"public_addr":"172.21.15.119:6827/1940737285","cluster_addr":"172.21.15.119:6829/1940737285","heartbeat_back_addr":"172.21.15.119:6833/1940737285","heartbeat_front_addr":"172.21.15.119:6831/1940737285","state":["exists","up"]},{"osd":10,"uuid":"3a3bc347-85f4-4878-9797-d9da96137e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6825","nonce":663834864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6827","nonce":663834864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6831","nonce":663834864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6829","nonce":663834864}]},"public_addr":"172.21.15.153:6825/663834864","cluster_addr":"172.21.15.153:6827/663834864","heartbeat_back_addr":"172.21.15.153:6831/663834864","heartbeat_front_addr":"172.21.15.153:6829/663834864","state":["exists","up"]},{"osd":11,"uuid":"a0fc5909-279d-4172-977c-2dc2da0a7e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6825","nonce":1262216348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6827","nonce":1262216348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6831","nonce":1262216348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6829","nonce":1262216348}]},"public_addr":"172.21.15.159:6825/1262216348","cluster_addr":"172.21.15.159:6827/1262216348","heartbeat_back_addr":"172.21.15.159:6831/1262216348","heartbeat_front_addr":"172.21.15.159:6829/1262216348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:37.488109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:59.543060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:21.381585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:47.112729+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:09.732150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:30.893729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:56.678710+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:19.004256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:40.388818+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:06.391734+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:28.394053+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:49.634348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1628639392":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6801/1355617847":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6800/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/362307836":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/128964558":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/2278813554":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/1411933614":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/1685698277":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6800/1355617847":"2023-12-08T10:11:00.585565+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T10:18:14.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.633+0000 7f807e1ff700 1 -- 172.21.15.119:0/2023940432 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f80640a1ba0 msgr2=0x7f80640a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:14.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.633+0000 7f807e1ff700 1 --2- 172.21.15.119:0/2023940432 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f80640a1ba0 0x7f80640a4060 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8068009fd0 tx=0x7f806800b040 comp rx=0 tx=0).stop 2023-12-07T10:18:14.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.633+0000 7f807e1ff700 1 -- 172.21.15.119:0/2023940432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80781030f0 msgr2=0x7f8078073f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:14.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.633+0000 7f807e1ff700 1 --2- 172.21.15.119:0/2023940432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80781030f0 0x7f8078073f40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f807000e940 tx=0x7f807000ec50 comp rx=0 tx=0).stop 2023-12-07T10:18:14.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.633+0000 7f807e1ff700 1 -- 172.21.15.119:0/2023940432 shutdown_connections 2023-12-07T10:18:14.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.633+0000 7f807e1ff700 1 --2- 172.21.15.119:0/2023940432 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f80640a1ba0 0x7f80640a4060 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:14.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.633+0000 7f807e1ff700 1 --2- 172.21.15.119:0/2023940432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80781030f0 0x7f8078073f40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:14.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.633+0000 7f807e1ff700 1 --2- 172.21.15.119:0/2023940432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8078101560 0x7f8078075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:14.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.633+0000 7f807e1ff700 1 --2- 172.21.15.119:0/2023940432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8078102750 0x7f8078073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:14.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.633+0000 7f807e1ff700 1 -- 172.21.15.119:0/2023940432 >> 172.21.15.119:0/2023940432 conn(0x7f80780fcab0 msgr2=0x7f8078105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:14.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.633+0000 7f807e1ff700 1 -- 172.21.15.119:0/2023940432 shutdown_connections 2023-12-07T10:18:14.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.633+0000 7f807e1ff700 1 -- 172.21.15.119:0/2023940432 wait complete. 2023-12-07T10:18:14.656 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-07T10:18:14.657 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-07T10:18:14.780 INFO:journalctl@ceph.mds.b.smithi153.stdout:Dec 07 10:18:14 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-b[170442]: 2023-12-07T10:18:14.501+0000 7f05571b3700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2023-12-07T10:18:14.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:14 smithi153 ceph-mon[133625]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:18:14.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:14 smithi153 ceph-mon[133625]: from='client.24841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:18:14.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:14 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] up:active 2023-12-07T10:18:14.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:14 smithi153 ceph-mon[133625]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2023-12-07T10:18:14.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:14 smithi153 ceph-mon[133625]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-12-07T10:18:14.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:14 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:18:14.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:14 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3870923656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:18:14.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:14 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2023940432' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:18:14.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.854+0000 7fd6e3fff700 1 -- 172.21.15.119:0/1950740721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6dc100780 msgr2=0x7fd6dc100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:14.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.854+0000 7fd6e3fff700 1 --2- 172.21.15.119:0/1950740721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6dc100780 0x7fd6dc100c00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc009990 tx=0x7fd6cc009ca0 comp rx=0 tx=0).stop 2023-12-07T10:18:14.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.854+0000 7fd6e3fff700 1 -- 172.21.15.119:0/1950740721 shutdown_connections 2023-12-07T10:18:14.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.854+0000 7fd6e3fff700 1 --2- 172.21.15.119:0/1950740721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6dc101140 0x7fd6dc1015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:14.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.854+0000 7fd6e3fff700 1 --2- 172.21.15.119:0/1950740721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6dc0ffe20 0x7fd6dc100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:14.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.854+0000 7fd6e3fff700 1 --2- 172.21.15.119:0/1950740721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6dc100780 0x7fd6dc100c00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:14.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.854+0000 7fd6e3fff700 1 -- 172.21.15.119:0/1950740721 >> 172.21.15.119:0/1950740721 conn(0x7fd6dc0fb9c0 msgr2=0x7fd6dc0fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:14.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.854+0000 7fd6e3fff700 1 -- 172.21.15.119:0/1950740721 shutdown_connections 2023-12-07T10:18:14.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.854+0000 7fd6e3fff700 1 -- 172.21.15.119:0/1950740721 wait complete. 2023-12-07T10:18:14.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.855+0000 7fd6e3fff700 1 Processor -- start 2023-12-07T10:18:14.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.855+0000 7fd6e3fff700 1 -- start start 2023-12-07T10:18:14.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.855+0000 7fd6e3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6dc0ffe20 0x7fd6dc199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:14.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.855+0000 7fd6e3fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6dc100780 0x7fd6dc199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:14.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.855+0000 7fd6e3fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6dc101140 0x7fd6dc1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:14.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.855+0000 7fd6e3fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc1a11c0 con 0x7fd6dc0ffe20 2023-12-07T10:18:14.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.855+0000 7fd6e3fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc1a1300 con 0x7fd6dc100780 2023-12-07T10:18:14.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.855+0000 7fd6e3fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc1a1490 con 0x7fd6dc101140 2023-12-07T10:18:14.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.856+0000 7fd6e1d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6dc0ffe20 0x7fd6dc199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:14.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.856+0000 7fd6e1d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6dc0ffe20 0x7fd6dc199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54052/0 (socket says 172.21.15.119:54052) 2023-12-07T10:18:14.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.856+0000 7fd6e1d9b700 1 -- 172.21.15.119:0/3170855152 learned_addr learned my addr 172.21.15.119:0/3170855152 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:14.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.856+0000 7fd6e159a700 1 --2- 172.21.15.119:0/3170855152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6dc100780 0x7fd6dc199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:14.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.856+0000 7fd6e259c700 1 --2- 172.21.15.119:0/3170855152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6dc101140 0x7fd6dc1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:14.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.856+0000 7fd6e1d9b700 1 -- 172.21.15.119:0/3170855152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6dc101140 msgr2=0x7fd6dc1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:14.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.856+0000 7fd6e1d9b700 1 --2- 172.21.15.119:0/3170855152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6dc101140 0x7fd6dc1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:14.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.856+0000 7fd6e1d9b700 1 -- 172.21.15.119:0/3170855152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6dc100780 msgr2=0x7fd6dc199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:14.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.856+0000 7fd6e1d9b700 1 --2- 172.21.15.119:0/3170855152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6dc100780 0x7fd6dc199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:14.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.856+0000 7fd6e1d9b700 1 -- 172.21.15.119:0/3170855152 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc0095f0 con 0x7fd6dc0ffe20 2023-12-07T10:18:14.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.856+0000 7fd6e1d9b700 1 --2- 172.21.15.119:0/3170855152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6dc0ffe20 0x7fd6dc199690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd6d800d950 tx=0x7fd6d800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:14.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.857+0000 7fd6d2ffd700 1 -- 172.21.15.119:0/3170855152 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d800f890 con 0x7fd6dc0ffe20 2023-12-07T10:18:14.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.857+0000 7fd6d2ffd700 1 -- 172.21.15.119:0/3170855152 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6d8010460 con 0x7fd6dc0ffe20 2023-12-07T10:18:14.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.857+0000 7fd6e3fff700 1 -- 172.21.15.119:0/3170855152 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6dc1a1770 con 0x7fd6dc0ffe20 2023-12-07T10:18:14.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.857+0000 7fd6d2ffd700 1 -- 172.21.15.119:0/3170855152 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d800e610 con 0x7fd6dc0ffe20 2023-12-07T10:18:14.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.857+0000 7fd6e3fff700 1 -- 172.21.15.119:0/3170855152 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6dc1a1cd0 con 0x7fd6dc0ffe20 2023-12-07T10:18:14.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.858+0000 7fd6e3fff700 1 -- 172.21.15.119:0/3170855152 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6dc10ae50 con 0x7fd6dc0ffe20 2023-12-07T10:18:14.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.859+0000 7fd6d2ffd700 1 -- 172.21.15.119:0/3170855152 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7fd6d80108d0 con 0x7fd6dc0ffe20 2023-12-07T10:18:14.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.861+0000 7fd6d2ffd700 1 --2- 172.21.15.119:0/3170855152 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd6c80a1c20 0x7fd6c80a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:14.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.861+0000 7fd6d2ffd700 1 -- 172.21.15.119:0/3170855152 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd6d80cd7f0 con 0x7fd6dc0ffe20 2023-12-07T10:18:14.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.861+0000 7fd6e159a700 1 --2- 172.21.15.119:0/3170855152 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd6c80a1c20 0x7fd6c80a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:14.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.862+0000 7fd6e159a700 1 --2- 172.21.15.119:0/3170855152 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd6c80a1c20 0x7fd6c80a40e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00b5c0 tx=0x7fd6cc003790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:14.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:14.865+0000 7fd6d2ffd700 1 -- 172.21.15.119:0/3170855152 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd6d8096750 con 0x7fd6dc0ffe20 2023-12-07T10:18:14.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:14 smithi119 ceph-mon[97908]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:18:14.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:14 smithi119 ceph-mon[97908]: from='client.24841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:18:14.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:14 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] up:active 2023-12-07T10:18:14.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:14 smithi119 ceph-mon[97908]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2023-12-07T10:18:14.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:14 smithi119 ceph-mon[97908]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-12-07T10:18:14.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:14 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:18:14.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:14 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3870923656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:18:14.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:14 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2023940432' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:18:15.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:14 smithi159 ceph-mon[134726]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:18:15.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:14 smithi159 ceph-mon[134726]: from='client.24841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:18:15.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:14 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] up:active 2023-12-07T10:18:15.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:14 smithi159 ceph-mon[134726]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2023-12-07T10:18:15.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:14 smithi159 ceph-mon[134726]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-12-07T10:18:15.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:14 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:18:15.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:14 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3870923656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:18:15.014 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:14 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2023940432' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:18:15.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.065+0000 7fd6e3fff700 1 -- 172.21.15.119:0/3170855152 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd6dc068320 con 0x7fd6c80a1c20 2023-12-07T10:18:15.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.080+0000 7fd6d2ffd700 1 -- 172.21.15.119:0/3170855152 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422574 (secure 0 0 0) 0x7fd6dc068320 con 0x7fd6c80a1c20 2023-12-07T10:18:15.083 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:18:15.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.086+0000 7fd6e3fff700 1 -- 172.21.15.119:0/3170855152 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd6c80a1c20 msgr2=0x7fd6c80a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:15.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.086+0000 7fd6e3fff700 1 --2- 172.21.15.119:0/3170855152 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd6c80a1c20 0x7fd6c80a40e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00b5c0 tx=0x7fd6cc003790 comp rx=0 tx=0).stop 2023-12-07T10:18:15.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.086+0000 7fd6e3fff700 1 -- 172.21.15.119:0/3170855152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6dc0ffe20 msgr2=0x7fd6dc199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:15.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.086+0000 7fd6e3fff700 1 --2- 172.21.15.119:0/3170855152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6dc0ffe20 0x7fd6dc199690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd6d800d950 tx=0x7fd6d800dc60 comp rx=0 tx=0).stop 2023-12-07T10:18:15.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.086+0000 7fd6e3fff700 1 -- 172.21.15.119:0/3170855152 shutdown_connections 2023-12-07T10:18:15.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.086+0000 7fd6e3fff700 1 --2- 172.21.15.119:0/3170855152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6dc101140 0x7fd6dc1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:15.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.086+0000 7fd6e3fff700 1 --2- 172.21.15.119:0/3170855152 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd6c80a1c20 0x7fd6c80a40e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:15.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.086+0000 7fd6e3fff700 1 --2- 172.21.15.119:0/3170855152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6dc0ffe20 0x7fd6dc199690 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:15.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.086+0000 7fd6e3fff700 1 --2- 172.21.15.119:0/3170855152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6dc100780 0x7fd6dc199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:15.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.086+0000 7fd6e3fff700 1 -- 172.21.15.119:0/3170855152 >> 172.21.15.119:0/3170855152 conn(0x7fd6dc0fb9c0 msgr2=0x7fd6dc0fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:15.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.086+0000 7fd6e3fff700 1 -- 172.21.15.119:0/3170855152 shutdown_connections 2023-12-07T10:18:15.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.086+0000 7fd6e3fff700 1 -- 172.21.15.119:0/3170855152 wait complete. 2023-12-07T10:18:15.092 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-12-07T10:18:15.107 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2023-12-07T10:18:14.676298+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":599634,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":157,"num_read_kb":136,"num_write":353,"num_write_kb":3002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":53,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":289,"ondisk_log_size":289,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338056,"kb_used_data":13848,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603768,"statfs":{"total":1151940427776,"available":1151594258432,"internally_reserved":0,"allocated":14180352,"data_stored":7753374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":331855161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4769,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":125,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003497"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849988+0000","last_change":"2023-12-07T10:17:41.614206+0000","last_active":"2023-12-07T10:18:00.849988+0000","last_peered":"2023-12-07T10:18:00.849988+0000","last_clean":"2023-12-07T10:18:00.849988+0000","last_became_active":"2023-12-07T10:17:41.613553+0000","last_became_peered":"2023-12-07T10:17:41.613553+0000","last_unstale":"2023-12-07T10:18:00.849988+0000","last_undegraded":"2023-12-07T10:18:00.849988+0000","last_fullsized":"2023-12-07T10:18:00.849988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:10:22.198414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852964+0000","last_change":"2023-12-07T10:17:41.616917+0000","last_active":"2023-12-07T10:18:00.852964+0000","last_peered":"2023-12-07T10:18:00.852964+0000","last_clean":"2023-12-07T10:18:00.852964+0000","last_became_active":"2023-12-07T10:17:41.615345+0000","last_became_peered":"2023-12-07T10:17:41.615345+0000","last_unstale":"2023-12-07T10:18:00.852964+0000","last_undegraded":"2023-12-07T10:18:00.852964+0000","last_fullsized":"2023-12-07T10:18:00.852964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:22:25.566738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.853214+0000","last_change":"2023-12-07T10:17:41.619767+0000","last_active":"2023-12-07T10:18:00.853214+0000","last_peered":"2023-12-07T10:18:00.853214+0000","last_clean":"2023-12-07T10:18:00.853214+0000","last_became_active":"2023-12-07T10:17:41.619362+0000","last_became_peered":"2023-12-07T10:17:41.619362+0000","last_unstale":"2023-12-07T10:18:00.853214+0000","last_undegraded":"2023-12-07T10:18:00.853214+0000","last_fullsized":"2023-12-07T10:18:00.853214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:12:55.458520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.862478+0000","last_change":"2023-12-07T10:18:01.861930+0000","last_active":"2023-12-07T10:18:01.862478+0000","last_peered":"2023-12-07T10:18:01.862478+0000","last_clean":"2023-12-07T10:18:01.862478+0000","last_became_active":"2023-12-07T10:18:01.861570+0000","last_became_peered":"2023-12-07T10:18:01.861570+0000","last_unstale":"2023-12-07T10:18:01.862478+0000","last_undegraded":"2023-12-07T10:18:01.862478+0000","last_fullsized":"2023-12-07T10:18:01.862478+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:29:45.234261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852580+0000","last_change":"2023-12-07T10:17:41.613890+0000","last_active":"2023-12-07T10:18:00.852580+0000","last_peered":"2023-12-07T10:18:00.852580+0000","last_clean":"2023-12-07T10:18:00.852580+0000","last_became_active":"2023-12-07T10:17:41.613073+0000","last_became_peered":"2023-12-07T10:17:41.613073+0000","last_unstale":"2023-12-07T10:18:00.852580+0000","last_undegraded":"2023-12-07T10:18:00.852580+0000","last_fullsized":"2023-12-07T10:18:00.852580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:17:14.339002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.854664+0000","last_change":"2023-12-07T10:17:41.609585+0000","last_active":"2023-12-07T10:18:00.854664+0000","last_peered":"2023-12-07T10:18:00.854664+0000","last_clean":"2023-12-07T10:18:00.854664+0000","last_became_active":"2023-12-07T10:17:41.609256+0000","last_became_peered":"2023-12-07T10:17:41.609256+0000","last_unstale":"2023-12-07T10:18:00.854664+0000","last_undegraded":"2023-12-07T10:18:00.854664+0000","last_fullsized":"2023-12-07T10:18:00.854664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:20:15.103208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856396+0000","last_change":"2023-12-07T10:17:41.606138+0000","last_active":"2023-12-07T10:18:00.856396+0000","last_peered":"2023-12-07T10:18:00.856396+0000","last_clean":"2023-12-07T10:18:00.856396+0000","last_became_active":"2023-12-07T10:17:41.605818+0000","last_became_peered":"2023-12-07T10:17:41.605818+0000","last_unstale":"2023-12-07T10:18:00.856396+0000","last_undegraded":"2023-12-07T10:18:00.856396+0000","last_fullsized":"2023-12-07T10:18:00.856396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:38:39.244077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.257383+0000","last_change":"2023-12-07T10:18:02.255959+0000","last_active":"2023-12-07T10:18:02.257383+0000","last_peered":"2023-12-07T10:18:02.257383+0000","last_clean":"2023-12-07T10:18:02.257383+0000","last_became_active":"2023-12-07T10:18:02.255560+0000","last_became_peered":"2023-12-07T10:18:02.255560+0000","last_unstale":"2023-12-07T10:18:02.257383+0000","last_undegraded":"2023-12-07T10:18:02.257383+0000","last_fullsized":"2023-12-07T10:18:02.257383+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:05:49.065538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.847884+0000","last_change":"2023-12-07T10:17:41.610225+0000","last_active":"2023-12-07T10:18:00.847884+0000","last_peered":"2023-12-07T10:18:00.847884+0000","last_clean":"2023-12-07T10:18:00.847884+0000","last_became_active":"2023-12-07T10:17:41.609875+0000","last_became_peered":"2023-12-07T10:17:41.609875+0000","last_unstale":"2023-12-07T10:18:00.847884+0000","last_undegraded":"2023-12-07T10:18:00.847884+0000","last_fullsized":"2023-12-07T10:18:00.847884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:00:43.136176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855342+0000","last_change":"2023-12-07T10:17:41.617335+0000","last_active":"2023-12-07T10:18:00.855342+0000","last_peered":"2023-12-07T10:18:00.855342+0000","last_clean":"2023-12-07T10:18:00.855342+0000","last_became_active":"2023-12-07T10:17:41.617005+0000","last_became_peered":"2023-12-07T10:17:41.617005+0000","last_unstale":"2023-12-07T10:18:00.855342+0000","last_undegraded":"2023-12-07T10:18:00.855342+0000","last_fullsized":"2023-12-07T10:18:00.855342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:43:36.855465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.857002+0000","last_change":"2023-12-07T10:17:41.608104+0000","last_active":"2023-12-07T10:18:00.857002+0000","last_peered":"2023-12-07T10:18:00.857002+0000","last_clean":"2023-12-07T10:18:00.857002+0000","last_became_active":"2023-12-07T10:17:41.607769+0000","last_became_peered":"2023-12-07T10:17:41.607769+0000","last_unstale":"2023-12-07T10:18:00.857002+0000","last_undegraded":"2023-12-07T10:18:00.857002+0000","last_fullsized":"2023-12-07T10:18:00.857002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:47:04.853044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852617+0000","last_change":"2023-12-07T10:17:41.611585+0000","last_active":"2023-12-07T10:18:00.852617+0000","last_peered":"2023-12-07T10:18:00.852617+0000","last_clean":"2023-12-07T10:18:00.852617+0000","last_became_active":"2023-12-07T10:17:41.611202+0000","last_became_peered":"2023-12-07T10:17:41.611202+0000","last_unstale":"2023-12-07T10:18:00.852617+0000","last_undegraded":"2023-12-07T10:18:00.852617+0000","last_fullsized":"2023-12-07T10:18:00.852617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:17:01.649424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855084+0000","last_change":"2023-12-07T10:17:41.611116+0000","last_active":"2023-12-07T10:18:00.855084+0000","last_peered":"2023-12-07T10:18:00.855084+0000","last_clean":"2023-12-07T10:18:00.855084+0000","last_became_active":"2023-12-07T10:17:41.610791+0000","last_became_peered":"2023-12-07T10:17:41.610791+0000","last_unstale":"2023-12-07T10:18:00.855084+0000","last_undegraded":"2023-12-07T10:18:00.855084+0000","last_fullsized":"2023-12-07T10:18:00.855084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:48:08.822235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852233+0000","last_change":"2023-12-07T10:17:41.617447+0000","last_active":"2023-12-07T10:18:00.852233+0000","last_peered":"2023-12-07T10:18:00.852233+0000","last_clean":"2023-12-07T10:18:00.852233+0000","last_became_active":"2023-12-07T10:17:41.612638+0000","last_became_peered":"2023-12-07T10:17:41.612638+0000","last_unstale":"2023-12-07T10:18:00.852233+0000","last_undegraded":"2023-12-07T10:18:00.852233+0000","last_fullsized":"2023-12-07T10:18:00.852233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:11:30.384367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852009+0000","last_change":"2023-12-07T10:17:41.611607+0000","last_active":"2023-12-07T10:18:00.852009+0000","last_peered":"2023-12-07T10:18:00.852009+0000","last_clean":"2023-12-07T10:18:00.852009+0000","last_became_active":"2023-12-07T10:17:41.611331+0000","last_became_peered":"2023-12-07T10:17:41.611331+0000","last_unstale":"2023-12-07T10:18:00.852009+0000","last_undegraded":"2023-12-07T10:18:00.852009+0000","last_fullsized":"2023-12-07T10:18:00.852009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:17:53.049922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.847334+0000","last_change":"2023-12-07T10:17:41.610528+0000","last_active":"2023-12-07T10:18:00.847334+0000","last_peered":"2023-12-07T10:18:00.847334+0000","last_clean":"2023-12-07T10:18:00.847334+0000","last_became_active":"2023-12-07T10:17:41.610216+0000","last_became_peered":"2023-12-07T10:17:41.610216+0000","last_unstale":"2023-12-07T10:18:00.847334+0000","last_undegraded":"2023-12-07T10:18:00.847334+0000","last_fullsized":"2023-12-07T10:18:00.847334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:11:36.749537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849875+0000","last_change":"2023-12-07T10:17:41.613967+0000","last_active":"2023-12-07T10:18:00.849875+0000","last_peered":"2023-12-07T10:18:00.849875+0000","last_clean":"2023-12-07T10:18:00.849875+0000","last_became_active":"2023-12-07T10:17:41.613116+0000","last_became_peered":"2023-12-07T10:17:41.613116+0000","last_unstale":"2023-12-07T10:18:00.849875+0000","last_undegraded":"2023-12-07T10:18:00.849875+0000","last_fullsized":"2023-12-07T10:18:00.849875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:46:32.444025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.846417+0000","last_change":"2023-12-07T10:17:41.605478+0000","last_active":"2023-12-07T10:18:00.846417+0000","last_peered":"2023-12-07T10:18:00.846417+0000","last_clean":"2023-12-07T10:18:00.846417+0000","last_became_active":"2023-12-07T10:17:41.605110+0000","last_became_peered":"2023-12-07T10:17:41.605110+0000","last_unstale":"2023-12-07T10:18:00.846417+0000","last_undegraded":"2023-12-07T10:18:00.846417+0000","last_fullsized":"2023-12-07T10:18:00.846417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:53:56.410196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.256377+0000","last_change":"2023-12-07T10:18:02.256377+0000","last_active":"2023-12-07T10:18:02.256377+0000","last_peered":"2023-12-07T10:18:02.256377+0000","last_clean":"2023-12-07T10:18:02.256377+0000","last_became_active":"2023-12-07T10:18:02.255987+0000","last_became_peered":"2023-12-07T10:18:02.255987+0000","last_unstale":"2023-12-07T10:18:02.256377+0000","last_undegraded":"2023-12-07T10:18:02.256377+0000","last_fullsized":"2023-12-07T10:18:02.256377+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:23:34.909187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849924+0000","last_change":"2023-12-07T10:17:45.799925+0000","last_active":"2023-12-07T10:18:00.849924+0000","last_peered":"2023-12-07T10:18:00.849924+0000","last_clean":"2023-12-07T10:18:00.849924+0000","last_became_active":"2023-12-07T10:17:45.799505+0000","last_became_peered":"2023-12-07T10:17:45.799505+0000","last_unstale":"2023-12-07T10:18:00.849924+0000","last_undegraded":"2023-12-07T10:18:00.849924+0000","last_fullsized":"2023-12-07T10:18:00.849924+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:52:51.470333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'13","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.757302+0000","last_change":"2023-12-07T10:17:40.755066+0000","last_active":"2023-12-07T10:18:12.757302+0000","last_peered":"2023-12-07T10:18:12.757302+0000","last_clean":"2023-12-07T10:18:12.757302+0000","last_became_active":"2023-12-07T10:17:40.754709+0000","last_became_peered":"2023-12-07T10:17:40.754709+0000","last_unstale":"2023-12-07T10:18:12.757302+0000","last_undegraded":"2023-12-07T10:18:12.757302+0000","last_fullsized":"2023-12-07T10:18:12.757302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:18:31.148592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849407+0000","last_change":"2023-12-07T10:17:41.619107+0000","last_active":"2023-12-07T10:18:00.849407+0000","last_peered":"2023-12-07T10:18:00.849407+0000","last_clean":"2023-12-07T10:18:00.849407+0000","last_became_active":"2023-12-07T10:17:41.617057+0000","last_became_peered":"2023-12-07T10:17:41.617057+0000","last_unstale":"2023-12-07T10:18:00.849407+0000","last_undegraded":"2023-12-07T10:18:00.849407+0000","last_fullsized":"2023-12-07T10:18:00.849407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:53:16.402933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852207+0000","last_change":"2023-12-07T10:17:45.797582+0000","last_active":"2023-12-07T10:18:00.852207+0000","last_peered":"2023-12-07T10:18:00.852207+0000","last_clean":"2023-12-07T10:18:00.852207+0000","last_became_active":"2023-12-07T10:17:45.796358+0000","last_became_peered":"2023-12-07T10:17:45.796358+0000","last_unstale":"2023-12-07T10:18:00.852207+0000","last_undegraded":"2023-12-07T10:18:00.852207+0000","last_fullsized":"2023-12-07T10:18:00.852207+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:53:19.551459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.857161+0000","last_change":"2023-12-07T10:17:40.762155+0000","last_active":"2023-12-07T10:18:01.857161+0000","last_peered":"2023-12-07T10:18:01.857161+0000","last_clean":"2023-12-07T10:18:01.857161+0000","last_became_active":"2023-12-07T10:17:40.761776+0000","last_became_peered":"2023-12-07T10:17:40.761776+0000","last_unstale":"2023-12-07T10:18:01.857161+0000","last_undegraded":"2023-12-07T10:18:01.857161+0000","last_fullsized":"2023-12-07T10:18:01.857161+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:15:06.096331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.853107+0000","last_change":"2023-12-07T10:17:41.614999+0000","last_active":"2023-12-07T10:18:00.853107+0000","last_peered":"2023-12-07T10:18:00.853107+0000","last_clean":"2023-12-07T10:18:00.853107+0000","last_became_active":"2023-12-07T10:17:41.614673+0000","last_became_peered":"2023-12-07T10:17:41.614673+0000","last_unstale":"2023-12-07T10:18:00.853107+0000","last_undegraded":"2023-12-07T10:18:00.853107+0000","last_fullsized":"2023-12-07T10:18:00.853107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:34:45.378843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852388+0000","last_change":"2023-12-07T10:17:45.799399+0000","last_active":"2023-12-07T10:18:00.852388+0000","last_peered":"2023-12-07T10:18:00.852388+0000","last_clean":"2023-12-07T10:18:00.852388+0000","last_became_active":"2023-12-07T10:17:45.799072+0000","last_became_peered":"2023-12-07T10:17:45.799072+0000","last_unstale":"2023-12-07T10:18:00.852388+0000","last_undegraded":"2023-12-07T10:18:00.852388+0000","last_fullsized":"2023-12-07T10:18:00.852388+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:34:54.663334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:11.748573+0000","last_change":"2023-12-07T10:18:01.993128+0000","last_active":"2023-12-07T10:18:11.748573+0000","last_peered":"2023-12-07T10:18:11.748573+0000","last_clean":"2023-12-07T10:18:11.748573+0000","last_became_active":"2023-12-07T10:18:01.990244+0000","last_became_peered":"2023-12-07T10:18:01.990244+0000","last_unstale":"2023-12-07T10:18:11.748573+0000","last_undegraded":"2023-12-07T10:18:11.748573+0000","last_fullsized":"2023-12-07T10:18:11.748573+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:42:53.778363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.847633+0000","last_change":"2023-12-07T10:17:41.612903+0000","last_active":"2023-12-07T10:18:00.847633+0000","last_peered":"2023-12-07T10:18:00.847633+0000","last_clean":"2023-12-07T10:18:00.847633+0000","last_became_active":"2023-12-07T10:17:41.611968+0000","last_became_peered":"2023-12-07T10:17:41.611968+0000","last_unstale":"2023-12-07T10:18:00.847633+0000","last_undegraded":"2023-12-07T10:18:00.847633+0000","last_fullsized":"2023-12-07T10:18:00.847633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:34:08.442828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852245+0000","last_change":"2023-12-07T10:17:45.803858+0000","last_active":"2023-12-07T10:18:00.852245+0000","last_peered":"2023-12-07T10:18:00.852245+0000","last_clean":"2023-12-07T10:18:00.852245+0000","last_became_active":"2023-12-07T10:17:45.803481+0000","last_became_peered":"2023-12-07T10:17:45.803481+0000","last_unstale":"2023-12-07T10:18:00.852245+0000","last_undegraded":"2023-12-07T10:18:00.852245+0000","last_fullsized":"2023-12-07T10:18:00.852245+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:08:03.728554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852884+0000","last_change":"2023-12-07T10:17:40.751866+0000","last_active":"2023-12-07T10:18:00.852884+0000","last_peered":"2023-12-07T10:18:00.852884+0000","last_clean":"2023-12-07T10:18:00.852884+0000","last_became_active":"2023-12-07T10:17:40.751492+0000","last_became_peered":"2023-12-07T10:17:40.751492+0000","last_unstale":"2023-12-07T10:18:00.852884+0000","last_undegraded":"2023-12-07T10:18:00.852884+0000","last_fullsized":"2023-12-07T10:18:00.852884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:42:53.839651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855907+0000","last_change":"2023-12-07T10:17:41.611326+0000","last_active":"2023-12-07T10:18:00.855907+0000","last_peered":"2023-12-07T10:18:00.855907+0000","last_clean":"2023-12-07T10:18:00.855907+0000","last_became_active":"2023-12-07T10:17:41.610863+0000","last_became_peered":"2023-12-07T10:17:41.610863+0000","last_unstale":"2023-12-07T10:18:00.855907+0000","last_undegraded":"2023-12-07T10:18:00.855907+0000","last_fullsized":"2023-12-07T10:18:00.855907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:07:55.801356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.850230+0000","last_change":"2023-12-07T10:17:45.791409+0000","last_active":"2023-12-07T10:18:00.850230+0000","last_peered":"2023-12-07T10:18:00.850230+0000","last_clean":"2023-12-07T10:18:00.850230+0000","last_became_active":"2023-12-07T10:17:45.791022+0000","last_became_peered":"2023-12-07T10:17:45.791022+0000","last_unstale":"2023-12-07T10:18:00.850230+0000","last_undegraded":"2023-12-07T10:18:00.850230+0000","last_fullsized":"2023-12-07T10:18:00.850230+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:57:14.617409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.738486+0000","last_change":"2023-12-07T10:17:40.745318+0000","last_active":"2023-12-07T10:18:12.738486+0000","last_peered":"2023-12-07T10:18:12.738486+0000","last_clean":"2023-12-07T10:18:12.738486+0000","last_became_active":"2023-12-07T10:17:40.745008+0000","last_became_peered":"2023-12-07T10:17:40.745008+0000","last_unstale":"2023-12-07T10:18:12.738486+0000","last_undegraded":"2023-12-07T10:18:12.738486+0000","last_fullsized":"2023-12-07T10:18:12.738486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:42:28.503421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1010,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.853726+0000","last_change":"2023-12-07T10:17:41.605121+0000","last_active":"2023-12-07T10:18:00.853726+0000","last_peered":"2023-12-07T10:18:00.853726+0000","last_clean":"2023-12-07T10:18:00.853726+0000","last_became_active":"2023-12-07T10:17:41.604770+0000","last_became_peered":"2023-12-07T10:17:41.604770+0000","last_unstale":"2023-12-07T10:18:00.853726+0000","last_undegraded":"2023-12-07T10:18:00.853726+0000","last_fullsized":"2023-12-07T10:18:00.853726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:51:02.151155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.850337+0000","last_change":"2023-12-07T10:17:45.809039+0000","last_active":"2023-12-07T10:18:00.850337+0000","last_peered":"2023-12-07T10:18:00.850337+0000","last_clean":"2023-12-07T10:18:00.850337+0000","last_became_active":"2023-12-07T10:17:45.808622+0000","last_became_peered":"2023-12-07T10:17:45.808622+0000","last_unstale":"2023-12-07T10:18:00.850337+0000","last_undegraded":"2023-12-07T10:18:00.850337+0000","last_fullsized":"2023-12-07T10:18:00.850337+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:16:02.835931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.854526+0000","last_change":"2023-12-07T10:17:40.757876+0000","last_active":"2023-12-07T10:18:00.854526+0000","last_peered":"2023-12-07T10:18:00.854526+0000","last_clean":"2023-12-07T10:18:00.854526+0000","last_became_active":"2023-12-07T10:17:40.757613+0000","last_became_peered":"2023-12-07T10:17:40.757613+0000","last_unstale":"2023-12-07T10:18:00.854526+0000","last_undegraded":"2023-12-07T10:18:00.854526+0000","last_fullsized":"2023-12-07T10:18:00.854526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:30:40.423092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856026+0000","last_change":"2023-12-07T10:17:41.611575+0000","last_active":"2023-12-07T10:18:00.856026+0000","last_peered":"2023-12-07T10:18:00.856026+0000","last_clean":"2023-12-07T10:18:00.856026+0000","last_became_active":"2023-12-07T10:17:41.611042+0000","last_became_peered":"2023-12-07T10:17:41.611042+0000","last_unstale":"2023-12-07T10:18:00.856026+0000","last_undegraded":"2023-12-07T10:18:00.856026+0000","last_fullsized":"2023-12-07T10:18:00.856026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:54:05.589792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849821+0000","last_change":"2023-12-07T10:17:45.796640+0000","last_active":"2023-12-07T10:18:00.849821+0000","last_peered":"2023-12-07T10:18:00.849821+0000","last_clean":"2023-12-07T10:18:00.849821+0000","last_became_active":"2023-12-07T10:17:45.796183+0000","last_became_peered":"2023-12-07T10:17:45.796183+0000","last_unstale":"2023-12-07T10:18:00.849821+0000","last_undegraded":"2023-12-07T10:18:00.849821+0000","last_fullsized":"2023-12-07T10:18:00.849821+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:41:56.890662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:10.744033+0000","last_change":"2023-12-07T10:17:40.760283+0000","last_active":"2023-12-07T10:18:10.744033+0000","last_peered":"2023-12-07T10:18:10.744033+0000","last_clean":"2023-12-07T10:18:10.744033+0000","last_became_active":"2023-12-07T10:17:40.759398+0000","last_became_peered":"2023-12-07T10:17:40.759398+0000","last_unstale":"2023-12-07T10:18:10.744033+0000","last_undegraded":"2023-12-07T10:18:10.744033+0000","last_fullsized":"2023-12-07T10:18:10.744033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:34:37.365490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849107+0000","last_change":"2023-12-07T10:17:41.604150+0000","last_active":"2023-12-07T10:18:00.849107+0000","last_peered":"2023-12-07T10:18:00.849107+0000","last_clean":"2023-12-07T10:18:00.849107+0000","last_became_active":"2023-12-07T10:17:41.603688+0000","last_became_peered":"2023-12-07T10:17:41.603688+0000","last_unstale":"2023-12-07T10:18:00.849107+0000","last_undegraded":"2023-12-07T10:18:00.849107+0000","last_fullsized":"2023-12-07T10:18:00.849107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:52:43.735609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852965+0000","last_change":"2023-12-07T10:17:45.806569+0000","last_active":"2023-12-07T10:18:00.852965+0000","last_peered":"2023-12-07T10:18:00.852965+0000","last_clean":"2023-12-07T10:18:00.852965+0000","last_became_active":"2023-12-07T10:17:45.806099+0000","last_became_peered":"2023-12-07T10:17:45.806099+0000","last_unstale":"2023-12-07T10:18:00.852965+0000","last_undegraded":"2023-12-07T10:18:00.852965+0000","last_fullsized":"2023-12-07T10:18:00.852965+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:02:19.977670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:11.747819+0000","last_change":"2023-12-07T10:17:40.744388+0000","last_active":"2023-12-07T10:18:11.747819+0000","last_peered":"2023-12-07T10:18:11.747819+0000","last_clean":"2023-12-07T10:18:11.747819+0000","last_became_active":"2023-12-07T10:17:40.743887+0000","last_became_peered":"2023-12-07T10:17:40.743887+0000","last_unstale":"2023-12-07T10:18:11.747819+0000","last_undegraded":"2023-12-07T10:18:11.747819+0000","last_fullsized":"2023-12-07T10:18:11.747819+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:18:39.102099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.845569+0000","last_change":"2023-12-07T10:17:41.595616+0000","last_active":"2023-12-07T10:18:00.845569+0000","last_peered":"2023-12-07T10:18:00.845569+0000","last_clean":"2023-12-07T10:18:00.845569+0000","last_became_active":"2023-12-07T10:17:41.595256+0000","last_became_peered":"2023-12-07T10:17:41.595256+0000","last_unstale":"2023-12-07T10:18:00.845569+0000","last_undegraded":"2023-12-07T10:18:00.845569+0000","last_fullsized":"2023-12-07T10:18:00.845569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:08:00.156043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849668+0000","last_change":"2023-12-07T10:17:45.794523+0000","last_active":"2023-12-07T10:18:00.849668+0000","last_peered":"2023-12-07T10:18:00.849668+0000","last_clean":"2023-12-07T10:18:00.849668+0000","last_became_active":"2023-12-07T10:17:45.794139+0000","last_became_peered":"2023-12-07T10:17:45.794139+0000","last_unstale":"2023-12-07T10:18:00.849668+0000","last_undegraded":"2023-12-07T10:18:00.849668+0000","last_fullsized":"2023-12-07T10:18:00.849668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:43:24.767627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'3","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:09.900730+0000","last_change":"2023-12-07T10:17:40.759620+0000","last_active":"2023-12-07T10:18:09.900730+0000","last_peered":"2023-12-07T10:18:09.900730+0000","last_clean":"2023-12-07T10:18:09.900730+0000","last_became_active":"2023-12-07T10:17:40.759372+0000","last_became_peered":"2023-12-07T10:17:40.759372+0000","last_unstale":"2023-12-07T10:18:09.900730+0000","last_undegraded":"2023-12-07T10:18:09.900730+0000","last_fullsized":"2023-12-07T10:18:09.900730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:46:14.459409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.847937+0000","last_change":"2023-12-07T10:17:41.601152+0000","last_active":"2023-12-07T10:18:00.847937+0000","last_peered":"2023-12-07T10:18:00.847937+0000","last_clean":"2023-12-07T10:18:00.847937+0000","last_became_active":"2023-12-07T10:17:41.600147+0000","last_became_peered":"2023-12-07T10:17:41.600147+0000","last_unstale":"2023-12-07T10:18:00.847937+0000","last_undegraded":"2023-12-07T10:18:00.847937+0000","last_fullsized":"2023-12-07T10:18:00.847937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:40:46.671489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.845356+0000","last_change":"2023-12-07T10:17:45.796229+0000","last_active":"2023-12-07T10:18:00.845356+0000","last_peered":"2023-12-07T10:18:00.845356+0000","last_clean":"2023-12-07T10:18:00.845356+0000","last_became_active":"2023-12-07T10:17:45.795582+0000","last_became_peered":"2023-12-07T10:17:45.795582+0000","last_unstale":"2023-12-07T10:18:00.845356+0000","last_undegraded":"2023-12-07T10:18:00.845356+0000","last_fullsized":"2023-12-07T10:18:00.845356+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:29:32.853975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:09.901229+0000","last_change":"2023-12-07T10:17:40.748353+0000","last_active":"2023-12-07T10:18:09.901229+0000","last_peered":"2023-12-07T10:18:09.901229+0000","last_clean":"2023-12-07T10:18:09.901229+0000","last_became_active":"2023-12-07T10:17:40.748050+0000","last_became_peered":"2023-12-07T10:17:40.748050+0000","last_unstale":"2023-12-07T10:18:09.901229+0000","last_undegraded":"2023-12-07T10:18:09.901229+0000","last_fullsized":"2023-12-07T10:18:09.901229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:18:15.298534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.256728+0000","last_change":"2023-12-07T10:18:02.256728+0000","last_active":"2023-12-07T10:18:02.256728+0000","last_peered":"2023-12-07T10:18:02.256728+0000","last_clean":"2023-12-07T10:18:02.256728+0000","last_became_active":"2023-12-07T10:18:02.256271+0000","last_became_peered":"2023-12-07T10:18:02.256271+0000","last_unstale":"2023-12-07T10:18:02.256728+0000","last_undegraded":"2023-12-07T10:18:02.256728+0000","last_fullsized":"2023-12-07T10:18:02.256728+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:27:00.198934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852553+0000","last_change":"2023-12-07T10:17:45.796903+0000","last_active":"2023-12-07T10:18:00.852553+0000","last_peered":"2023-12-07T10:18:00.852553+0000","last_clean":"2023-12-07T10:18:00.852553+0000","last_became_active":"2023-12-07T10:17:45.796454+0000","last_became_peered":"2023-12-07T10:17:45.796454+0000","last_unstale":"2023-12-07T10:18:00.852553+0000","last_undegraded":"2023-12-07T10:18:00.852553+0000","last_fullsized":"2023-12-07T10:18:00.852553+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:46:36.955396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:10.734628+0000","last_change":"2023-12-07T10:17:40.756045+0000","last_active":"2023-12-07T10:18:10.734628+0000","last_peered":"2023-12-07T10:18:10.734628+0000","last_clean":"2023-12-07T10:18:10.734628+0000","last_became_active":"2023-12-07T10:17:40.755763+0000","last_became_peered":"2023-12-07T10:17:40.755763+0000","last_unstale":"2023-12-07T10:18:10.734628+0000","last_undegraded":"2023-12-07T10:18:10.734628+0000","last_fullsized":"2023-12-07T10:18:10.734628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:01:08.009076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.847716+0000","last_change":"2023-12-07T10:17:41.611178+0000","last_active":"2023-12-07T10:18:00.847716+0000","last_peered":"2023-12-07T10:18:00.847716+0000","last_clean":"2023-12-07T10:18:00.847716+0000","last_became_active":"2023-12-07T10:17:41.610750+0000","last_became_peered":"2023-12-07T10:17:41.610750+0000","last_unstale":"2023-12-07T10:18:00.847716+0000","last_undegraded":"2023-12-07T10:18:00.847716+0000","last_fullsized":"2023-12-07T10:18:00.847716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:56:30.763523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.845439+0000","last_change":"2023-12-07T10:17:45.800183+0000","last_active":"2023-12-07T10:18:00.845439+0000","last_peered":"2023-12-07T10:18:00.845439+0000","last_clean":"2023-12-07T10:18:00.845439+0000","last_became_active":"2023-12-07T10:17:45.798598+0000","last_became_peered":"2023-12-07T10:17:45.798598+0000","last_unstale":"2023-12-07T10:18:00.845439+0000","last_undegraded":"2023-12-07T10:18:00.845439+0000","last_fullsized":"2023-12-07T10:18:00.845439+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:06:22.162951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'12","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:09.901114+0000","last_change":"2023-12-07T10:17:40.752481+0000","last_active":"2023-12-07T10:18:09.901114+0000","last_peered":"2023-12-07T10:18:09.901114+0000","last_clean":"2023-12-07T10:18:09.901114+0000","last_became_active":"2023-12-07T10:17:40.752158+0000","last_became_peered":"2023-12-07T10:17:40.752158+0000","last_unstale":"2023-12-07T10:18:09.901114+0000","last_undegraded":"2023-12-07T10:18:09.901114+0000","last_fullsized":"2023-12-07T10:18:09.901114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:43:27.963014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3840,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.853035+0000","last_change":"2023-12-07T10:17:41.614107+0000","last_active":"2023-12-07T10:18:00.853035+0000","last_peered":"2023-12-07T10:18:00.853035+0000","last_clean":"2023-12-07T10:18:00.853035+0000","last_became_active":"2023-12-07T10:17:41.613743+0000","last_became_peered":"2023-12-07T10:17:41.613743+0000","last_unstale":"2023-12-07T10:18:00.853035+0000","last_undegraded":"2023-12-07T10:18:00.853035+0000","last_fullsized":"2023-12-07T10:18:00.853035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:49:03.200387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856289+0000","last_change":"2023-12-07T10:17:45.804282+0000","last_active":"2023-12-07T10:18:00.856289+0000","last_peered":"2023-12-07T10:18:00.856289+0000","last_clean":"2023-12-07T10:18:00.856289+0000","last_became_active":"2023-12-07T10:17:45.803794+0000","last_became_peered":"2023-12-07T10:17:45.803794+0000","last_unstale":"2023-12-07T10:18:00.856289+0000","last_undegraded":"2023-12-07T10:18:00.856289+0000","last_fullsized":"2023-12-07T10:18:00.856289+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:31:26.887565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.756900+0000","last_change":"2023-12-07T10:17:40.747204+0000","last_active":"2023-12-07T10:18:12.756900+0000","last_peered":"2023-12-07T10:18:12.756900+0000","last_clean":"2023-12-07T10:18:12.756900+0000","last_became_active":"2023-12-07T10:17:40.746916+0000","last_became_peered":"2023-12-07T10:17:40.746916+0000","last_unstale":"2023-12-07T10:18:12.756900+0000","last_undegraded":"2023-12-07T10:18:12.756900+0000","last_fullsized":"2023-12-07T10:18:12.756900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:53:49.228320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855125+0000","last_change":"2023-12-07T10:17:41.606895+0000","last_active":"2023-12-07T10:18:00.855125+0000","last_peered":"2023-12-07T10:18:00.855125+0000","last_clean":"2023-12-07T10:18:00.855125+0000","last_became_active":"2023-12-07T10:17:41.606596+0000","last_became_peered":"2023-12-07T10:17:41.606596+0000","last_unstale":"2023-12-07T10:18:00.855125+0000","last_undegraded":"2023-12-07T10:18:00.855125+0000","last_fullsized":"2023-12-07T10:18:00.855125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:34:25.166906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849382+0000","last_change":"2023-12-07T10:17:45.807101+0000","last_active":"2023-12-07T10:18:00.849382+0000","last_peered":"2023-12-07T10:18:00.849382+0000","last_clean":"2023-12-07T10:18:00.849382+0000","last_became_active":"2023-12-07T10:17:45.806770+0000","last_became_peered":"2023-12-07T10:17:45.806770+0000","last_unstale":"2023-12-07T10:18:00.849382+0000","last_undegraded":"2023-12-07T10:18:00.849382+0000","last_fullsized":"2023-12-07T10:18:00.849382+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:39:32.522968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:11.750248+0000","last_change":"2023-12-07T10:17:40.754856+0000","last_active":"2023-12-07T10:18:11.750248+0000","last_peered":"2023-12-07T10:18:11.750248+0000","last_clean":"2023-12-07T10:18:11.750248+0000","last_became_active":"2023-12-07T10:17:40.754488+0000","last_became_peered":"2023-12-07T10:17:40.754488+0000","last_unstale":"2023-12-07T10:18:11.750248+0000","last_undegraded":"2023-12-07T10:18:11.750248+0000","last_fullsized":"2023-12-07T10:18:11.750248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:44:50.960216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856759+0000","last_change":"2023-12-07T10:17:41.606963+0000","last_active":"2023-12-07T10:18:00.856759+0000","last_peered":"2023-12-07T10:18:00.856759+0000","last_clean":"2023-12-07T10:18:00.856759+0000","last_became_active":"2023-12-07T10:17:41.606408+0000","last_became_peered":"2023-12-07T10:17:41.606408+0000","last_unstale":"2023-12-07T10:18:00.856759+0000","last_undegraded":"2023-12-07T10:18:00.856759+0000","last_fullsized":"2023-12-07T10:18:00.856759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:25:47.906649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.846470+0000","last_change":"2023-12-07T10:17:45.803867+0000","last_active":"2023-12-07T10:18:00.846470+0000","last_peered":"2023-12-07T10:18:00.846470+0000","last_clean":"2023-12-07T10:18:00.846470+0000","last_became_active":"2023-12-07T10:17:45.802664+0000","last_became_peered":"2023-12-07T10:17:45.802664+0000","last_unstale":"2023-12-07T10:18:00.846470+0000","last_undegraded":"2023-12-07T10:18:00.846470+0000","last_fullsized":"2023-12-07T10:18:00.846470+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:32:33.975307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:11.747193+0000","last_change":"2023-12-07T10:17:40.747494+0000","last_active":"2023-12-07T10:18:11.747193+0000","last_peered":"2023-12-07T10:18:11.747193+0000","last_clean":"2023-12-07T10:18:11.747193+0000","last_became_active":"2023-12-07T10:17:40.747157+0000","last_became_peered":"2023-12-07T10:17:40.747157+0000","last_unstale":"2023-12-07T10:18:11.747193+0000","last_undegraded":"2023-12-07T10:18:11.747193+0000","last_fullsized":"2023-12-07T10:18:11.747193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:45:24.395418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856481+0000","last_change":"2023-12-07T10:17:41.609495+0000","last_active":"2023-12-07T10:18:00.856481+0000","last_peered":"2023-12-07T10:18:00.856481+0000","last_clean":"2023-12-07T10:18:00.856481+0000","last_became_active":"2023-12-07T10:17:41.609181+0000","last_became_peered":"2023-12-07T10:17:41.609181+0000","last_unstale":"2023-12-07T10:18:00.856481+0000","last_undegraded":"2023-12-07T10:18:00.856481+0000","last_fullsized":"2023-12-07T10:18:00.856481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:20:33.473824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855858+0000","last_change":"2023-12-07T10:17:45.808862+0000","last_active":"2023-12-07T10:18:00.855858+0000","last_peered":"2023-12-07T10:18:00.855858+0000","last_clean":"2023-12-07T10:18:00.855858+0000","last_became_active":"2023-12-07T10:17:45.808461+0000","last_became_peered":"2023-12-07T10:17:45.808461+0000","last_unstale":"2023-12-07T10:18:00.855858+0000","last_undegraded":"2023-12-07T10:18:00.855858+0000","last_fullsized":"2023-12-07T10:18:00.855858+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:09:12.598126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'16","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:10.743667+0000","last_change":"2023-12-07T10:18:02.263709+0000","last_active":"2023-12-07T10:18:10.743667+0000","last_peered":"2023-12-07T10:18:10.743667+0000","last_clean":"2023-12-07T10:18:10.743667+0000","last_became_active":"2023-12-07T10:18:02.255788+0000","last_became_peered":"2023-12-07T10:18:02.255788+0000","last_unstale":"2023-12-07T10:18:10.743667+0000","last_undegraded":"2023-12-07T10:18:10.743667+0000","last_fullsized":"2023-12-07T10:18:10.743667+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:12:04.976226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.854358+0000","last_change":"2023-12-07T10:17:41.610509+0000","last_active":"2023-12-07T10:18:00.854358+0000","last_peered":"2023-12-07T10:18:00.854358+0000","last_clean":"2023-12-07T10:18:00.854358+0000","last_became_active":"2023-12-07T10:17:41.610121+0000","last_became_peered":"2023-12-07T10:17:41.610121+0000","last_unstale":"2023-12-07T10:18:00.854358+0000","last_undegraded":"2023-12-07T10:18:00.854358+0000","last_fullsized":"2023-12-07T10:18:00.854358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:53:17.365880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.854272+0000","last_change":"2023-12-07T10:17:45.795818+0000","last_active":"2023-12-07T10:18:00.854272+0000","last_peered":"2023-12-07T10:18:00.854272+0000","last_clean":"2023-12-07T10:18:00.854272+0000","last_became_active":"2023-12-07T10:17:45.795477+0000","last_became_peered":"2023-12-07T10:17:45.795477+0000","last_unstale":"2023-12-07T10:18:00.854272+0000","last_undegraded":"2023-12-07T10:18:00.854272+0000","last_fullsized":"2023-12-07T10:18:00.854272+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:17:21.494654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.859473+0000","last_change":"2023-12-07T10:17:40.751791+0000","last_active":"2023-12-07T10:18:01.859473+0000","last_peered":"2023-12-07T10:18:01.859473+0000","last_clean":"2023-12-07T10:18:01.859473+0000","last_became_active":"2023-12-07T10:17:40.751502+0000","last_became_peered":"2023-12-07T10:17:40.751502+0000","last_unstale":"2023-12-07T10:18:01.859473+0000","last_undegraded":"2023-12-07T10:18:01.859473+0000","last_fullsized":"2023-12-07T10:18:01.859473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:15:52.319265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.256247+0000","last_change":"2023-12-07T10:18:02.256247+0000","last_active":"2023-12-07T10:18:02.256247+0000","last_peered":"2023-12-07T10:18:02.256247+0000","last_clean":"2023-12-07T10:18:02.256247+0000","last_became_active":"2023-12-07T10:18:02.255732+0000","last_became_peered":"2023-12-07T10:18:02.255732+0000","last_unstale":"2023-12-07T10:18:02.256247+0000","last_undegraded":"2023-12-07T10:18:02.256247+0000","last_fullsized":"2023-12-07T10:18:02.256247+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:00:06.516804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.853635+0000","last_change":"2023-12-07T10:17:45.791966+0000","last_active":"2023-12-07T10:18:00.853635+0000","last_peered":"2023-12-07T10:18:00.853635+0000","last_clean":"2023-12-07T10:18:00.853635+0000","last_became_active":"2023-12-07T10:17:45.791613+0000","last_became_peered":"2023-12-07T10:17:45.791613+0000","last_unstale":"2023-12-07T10:18:00.853635+0000","last_undegraded":"2023-12-07T10:18:00.853635+0000","last_fullsized":"2023-12-07T10:18:00.853635+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:13:02.291289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.853559+0000","last_change":"2023-12-07T10:17:40.762896+0000","last_active":"2023-12-07T10:18:01.853559+0000","last_peered":"2023-12-07T10:18:01.853559+0000","last_clean":"2023-12-07T10:18:01.853559+0000","last_became_active":"2023-12-07T10:17:40.762427+0000","last_became_peered":"2023-12-07T10:17:40.762427+0000","last_unstale":"2023-12-07T10:18:01.853559+0000","last_undegraded":"2023-12-07T10:18:01.853559+0000","last_fullsized":"2023-12-07T10:18:01.853559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:56:35.589548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.854524+0000","last_change":"2023-12-07T10:17:41.610354+0000","last_active":"2023-12-07T10:18:00.854524+0000","last_peered":"2023-12-07T10:18:00.854524+0000","last_clean":"2023-12-07T10:18:00.854524+0000","last_became_active":"2023-12-07T10:17:41.609888+0000","last_became_peered":"2023-12-07T10:17:41.609888+0000","last_unstale":"2023-12-07T10:18:00.854524+0000","last_undegraded":"2023-12-07T10:18:00.854524+0000","last_fullsized":"2023-12-07T10:18:00.854524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:48:19.976949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855124+0000","last_change":"2023-12-07T10:17:45.807324+0000","last_active":"2023-12-07T10:18:00.855124+0000","last_peered":"2023-12-07T10:18:00.855124+0000","last_clean":"2023-12-07T10:18:00.855124+0000","last_became_active":"2023-12-07T10:17:45.806036+0000","last_became_peered":"2023-12-07T10:17:45.806036+0000","last_unstale":"2023-12-07T10:18:00.855124+0000","last_undegraded":"2023-12-07T10:18:00.855124+0000","last_fullsized":"2023-12-07T10:18:00.855124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:54:36.847992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856573+0000","last_change":"2023-12-07T10:17:40.754247+0000","last_active":"2023-12-07T10:18:00.856573+0000","last_peered":"2023-12-07T10:18:00.856573+0000","last_clean":"2023-12-07T10:18:00.856573+0000","last_became_active":"2023-12-07T10:17:40.753964+0000","last_became_peered":"2023-12-07T10:17:40.753964+0000","last_unstale":"2023-12-07T10:18:00.856573+0000","last_undegraded":"2023-12-07T10:18:00.856573+0000","last_fullsized":"2023-12-07T10:18:00.856573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:22:04.832399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.854494+0000","last_change":"2023-12-07T10:17:41.610516+0000","last_active":"2023-12-07T10:18:00.854494+0000","last_peered":"2023-12-07T10:18:00.854494+0000","last_clean":"2023-12-07T10:18:00.854494+0000","last_became_active":"2023-12-07T10:17:41.610184+0000","last_became_peered":"2023-12-07T10:17:41.610184+0000","last_unstale":"2023-12-07T10:18:00.854494+0000","last_undegraded":"2023-12-07T10:18:00.854494+0000","last_fullsized":"2023-12-07T10:18:00.854494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:53:43.100723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855260+0000","last_change":"2023-12-07T10:17:45.804624+0000","last_active":"2023-12-07T10:18:00.855260+0000","last_peered":"2023-12-07T10:18:00.855260+0000","last_clean":"2023-12-07T10:18:00.855260+0000","last_became_active":"2023-12-07T10:17:45.803667+0000","last_became_peered":"2023-12-07T10:17:45.803667+0000","last_unstale":"2023-12-07T10:18:00.855260+0000","last_undegraded":"2023-12-07T10:18:00.855260+0000","last_fullsized":"2023-12-07T10:18:00.855260+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:55:09.642795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855573+0000","last_change":"2023-12-07T10:17:41.618433+0000","last_active":"2023-12-07T10:18:00.855573+0000","last_peered":"2023-12-07T10:18:00.855573+0000","last_clean":"2023-12-07T10:18:00.855573+0000","last_became_active":"2023-12-07T10:17:41.617942+0000","last_became_peered":"2023-12-07T10:17:41.617942+0000","last_unstale":"2023-12-07T10:18:00.855573+0000","last_undegraded":"2023-12-07T10:18:00.855573+0000","last_fullsized":"2023-12-07T10:18:00.855573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:26:48.166157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:10.745062+0000","last_change":"2023-12-07T10:17:40.761907+0000","last_active":"2023-12-07T10:18:10.745062+0000","last_peered":"2023-12-07T10:18:10.745062+0000","last_clean":"2023-12-07T10:18:10.745062+0000","last_became_active":"2023-12-07T10:17:40.761564+0000","last_became_peered":"2023-12-07T10:17:40.761564+0000","last_unstale":"2023-12-07T10:18:10.745062+0000","last_undegraded":"2023-12-07T10:18:10.745062+0000","last_fullsized":"2023-12-07T10:18:10.745062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:57:22.304352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856852+0000","last_change":"2023-12-07T10:17:45.792684+0000","last_active":"2023-12-07T10:18:00.856852+0000","last_peered":"2023-12-07T10:18:00.856852+0000","last_clean":"2023-12-07T10:18:00.856852+0000","last_became_active":"2023-12-07T10:17:45.792172+0000","last_became_peered":"2023-12-07T10:17:45.792172+0000","last_unstale":"2023-12-07T10:18:00.856852+0000","last_undegraded":"2023-12-07T10:18:00.856852+0000","last_fullsized":"2023-12-07T10:18:00.856852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:24:31.084727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849807+0000","last_change":"2023-12-07T10:17:41.612582+0000","last_active":"2023-12-07T10:18:00.849807+0000","last_peered":"2023-12-07T10:18:00.849807+0000","last_clean":"2023-12-07T10:18:00.849807+0000","last_became_active":"2023-12-07T10:17:41.612244+0000","last_became_peered":"2023-12-07T10:17:41.612244+0000","last_unstale":"2023-12-07T10:18:00.849807+0000","last_undegraded":"2023-12-07T10:18:00.849807+0000","last_fullsized":"2023-12-07T10:18:00.849807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:41:53.506037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:11.749673+0000","last_change":"2023-12-07T10:17:40.748565+0000","last_active":"2023-12-07T10:18:11.749673+0000","last_peered":"2023-12-07T10:18:11.749673+0000","last_clean":"2023-12-07T10:18:11.749673+0000","last_became_active":"2023-12-07T10:17:40.748250+0000","last_became_peered":"2023-12-07T10:17:40.748250+0000","last_unstale":"2023-12-07T10:18:11.749673+0000","last_undegraded":"2023-12-07T10:18:11.749673+0000","last_fullsized":"2023-12-07T10:18:11.749673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:26:28.202440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":326,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856336+0000","last_change":"2023-12-07T10:16:33.482173+0000","last_active":"2023-12-07T10:18:00.856336+0000","last_peered":"2023-12-07T10:18:00.856336+0000","last_clean":"2023-12-07T10:18:00.856336+0000","last_became_active":"2023-12-07T10:16:33.481699+0000","last_became_peered":"2023-12-07T10:16:33.481699+0000","last_unstale":"2023-12-07T10:18:00.856336+0000","last_undegraded":"2023-12-07T10:18:00.856336+0000","last_fullsized":"2023-12-07T10:18:00.856336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:13:25.388258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:13:25.388258+0000","last_clean_scrub_stamp":"2023-12-07T10:13:25.388258+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:57:09.583677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856130+0000","last_change":"2023-12-07T10:17:45.806289+0000","last_active":"2023-12-07T10:18:00.856130+0000","last_peered":"2023-12-07T10:18:00.856130+0000","last_clean":"2023-12-07T10:18:00.856130+0000","last_became_active":"2023-12-07T10:17:45.805954+0000","last_became_peered":"2023-12-07T10:17:45.805954+0000","last_unstale":"2023-12-07T10:18:00.856130+0000","last_undegraded":"2023-12-07T10:18:00.856130+0000","last_fullsized":"2023-12-07T10:18:00.856130+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:21:09.955380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855381+0000","last_change":"2023-12-07T10:17:41.612322+0000","last_active":"2023-12-07T10:18:00.855381+0000","last_peered":"2023-12-07T10:18:00.855381+0000","last_clean":"2023-12-07T10:18:00.855381+0000","last_became_active":"2023-12-07T10:17:41.612039+0000","last_became_peered":"2023-12-07T10:17:41.612039+0000","last_unstale":"2023-12-07T10:18:00.855381+0000","last_undegraded":"2023-12-07T10:18:00.855381+0000","last_fullsized":"2023-12-07T10:18:00.855381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:40:49.883742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.853755+0000","last_change":"2023-12-07T10:17:40.749694+0000","last_active":"2023-12-07T10:18:01.853755+0000","last_peered":"2023-12-07T10:18:01.853755+0000","last_clean":"2023-12-07T10:18:01.853755+0000","last_became_active":"2023-12-07T10:17:40.749393+0000","last_became_peered":"2023-12-07T10:17:40.749393+0000","last_unstale":"2023-12-07T10:18:01.853755+0000","last_undegraded":"2023-12-07T10:18:01.853755+0000","last_fullsized":"2023-12-07T10:18:01.853755+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:15:36.949062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849339+0000","last_change":"2023-12-07T10:17:45.805769+0000","last_active":"2023-12-07T10:18:00.849339+0000","last_peered":"2023-12-07T10:18:00.849339+0000","last_clean":"2023-12-07T10:18:00.849339+0000","last_became_active":"2023-12-07T10:17:45.805346+0000","last_became_peered":"2023-12-07T10:17:45.805346+0000","last_unstale":"2023-12-07T10:18:00.849339+0000","last_undegraded":"2023-12-07T10:18:00.849339+0000","last_fullsized":"2023-12-07T10:18:00.849339+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:47:47.781933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'3","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:11.749601+0000","last_change":"2023-12-07T10:18:01.862558+0000","last_active":"2023-12-07T10:18:11.749601+0000","last_peered":"2023-12-07T10:18:11.749601+0000","last_clean":"2023-12-07T10:18:11.749601+0000","last_became_active":"2023-12-07T10:18:01.862063+0000","last_became_peered":"2023-12-07T10:18:01.862063+0000","last_unstale":"2023-12-07T10:18:11.749601+0000","last_undegraded":"2023-12-07T10:18:11.749601+0000","last_fullsized":"2023-12-07T10:18:11.749601+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:25:32.652160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855264+0000","last_change":"2023-12-07T10:17:41.617199+0000","last_active":"2023-12-07T10:18:00.855264+0000","last_peered":"2023-12-07T10:18:00.855264+0000","last_clean":"2023-12-07T10:18:00.855264+0000","last_became_active":"2023-12-07T10:17:41.616901+0000","last_became_peered":"2023-12-07T10:17:41.616901+0000","last_unstale":"2023-12-07T10:18:00.855264+0000","last_undegraded":"2023-12-07T10:18:00.855264+0000","last_fullsized":"2023-12-07T10:18:00.855264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:23:59.433861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849025+0000","last_change":"2023-12-07T10:17:45.790914+0000","last_active":"2023-12-07T10:18:00.849025+0000","last_peered":"2023-12-07T10:18:00.849025+0000","last_clean":"2023-12-07T10:18:00.849025+0000","last_became_active":"2023-12-07T10:17:45.790428+0000","last_became_peered":"2023-12-07T10:17:45.790428+0000","last_unstale":"2023-12-07T10:18:00.849025+0000","last_undegraded":"2023-12-07T10:18:00.849025+0000","last_fullsized":"2023-12-07T10:18:00.849025+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:03:57.516300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:11.750949+0000","last_change":"2023-12-07T10:18:01.863765+0000","last_active":"2023-12-07T10:18:11.750949+0000","last_peered":"2023-12-07T10:18:11.750949+0000","last_clean":"2023-12-07T10:18:11.750949+0000","last_became_active":"2023-12-07T10:18:01.858926+0000","last_became_peered":"2023-12-07T10:18:01.858926+0000","last_unstale":"2023-12-07T10:18:11.750949+0000","last_undegraded":"2023-12-07T10:18:11.750949+0000","last_fullsized":"2023-12-07T10:18:11.750949+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:44:19.663592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849322+0000","last_change":"2023-12-07T10:17:41.608513+0000","last_active":"2023-12-07T10:18:00.849322+0000","last_peered":"2023-12-07T10:18:00.849322+0000","last_clean":"2023-12-07T10:18:00.849322+0000","last_became_active":"2023-12-07T10:17:41.605049+0000","last_became_peered":"2023-12-07T10:17:41.605049+0000","last_unstale":"2023-12-07T10:18:00.849322+0000","last_undegraded":"2023-12-07T10:18:00.849322+0000","last_fullsized":"2023-12-07T10:18:00.849322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:33:00.730827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.850303+0000","last_change":"2023-12-07T10:17:45.792583+0000","last_active":"2023-12-07T10:18:00.850303+0000","last_peered":"2023-12-07T10:18:00.850303+0000","last_clean":"2023-12-07T10:18:00.850303+0000","last_became_active":"2023-12-07T10:17:45.792162+0000","last_became_peered":"2023-12-07T10:17:45.792162+0000","last_unstale":"2023-12-07T10:18:00.850303+0000","last_undegraded":"2023-12-07T10:18:00.850303+0000","last_fullsized":"2023-12-07T10:18:00.850303+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:47:30.901477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:09.902945+0000","last_change":"2023-12-07T10:17:40.750826+0000","last_active":"2023-12-07T10:18:09.902945+0000","last_peered":"2023-12-07T10:18:09.902945+0000","last_clean":"2023-12-07T10:18:09.902945+0000","last_became_active":"2023-12-07T10:17:40.750519+0000","last_became_peered":"2023-12-07T10:17:40.750519+0000","last_unstale":"2023-12-07T10:18:09.902945+0000","last_undegraded":"2023-12-07T10:18:09.902945+0000","last_fullsized":"2023-12-07T10:18:09.902945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:21:01.008897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.853040+0000","last_change":"2023-12-07T10:17:41.609488+0000","last_active":"2023-12-07T10:18:00.853040+0000","last_peered":"2023-12-07T10:18:00.853040+0000","last_clean":"2023-12-07T10:18:00.853040+0000","last_became_active":"2023-12-07T10:17:41.609061+0000","last_became_peered":"2023-12-07T10:17:41.609061+0000","last_unstale":"2023-12-07T10:18:00.853040+0000","last_undegraded":"2023-12-07T10:18:00.853040+0000","last_fullsized":"2023-12-07T10:18:00.853040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:39:32.729502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852501+0000","last_change":"2023-12-07T10:17:45.797573+0000","last_active":"2023-12-07T10:18:00.852501+0000","last_peered":"2023-12-07T10:18:00.852501+0000","last_clean":"2023-12-07T10:18:00.852501+0000","last_became_active":"2023-12-07T10:17:45.797119+0000","last_became_peered":"2023-12-07T10:17:45.797119+0000","last_unstale":"2023-12-07T10:18:00.852501+0000","last_undegraded":"2023-12-07T10:18:00.852501+0000","last_fullsized":"2023-12-07T10:18:00.852501+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:17:27.100470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:09.901504+0000","last_change":"2023-12-07T10:17:40.753596+0000","last_active":"2023-12-07T10:18:09.901504+0000","last_peered":"2023-12-07T10:18:09.901504+0000","last_clean":"2023-12-07T10:18:09.901504+0000","last_became_active":"2023-12-07T10:17:40.753251+0000","last_became_peered":"2023-12-07T10:17:40.753251+0000","last_unstale":"2023-12-07T10:18:09.901504+0000","last_undegraded":"2023-12-07T10:18:09.901504+0000","last_fullsized":"2023-12-07T10:18:09.901504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:08:30.868794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852298+0000","last_change":"2023-12-07T10:17:41.599689+0000","last_active":"2023-12-07T10:18:00.852298+0000","last_peered":"2023-12-07T10:18:00.852298+0000","last_clean":"2023-12-07T10:18:00.852298+0000","last_became_active":"2023-12-07T10:17:41.599358+0000","last_became_peered":"2023-12-07T10:17:41.599358+0000","last_unstale":"2023-12-07T10:18:00.852298+0000","last_undegraded":"2023-12-07T10:18:00.852298+0000","last_fullsized":"2023-12-07T10:18:00.852298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:44:19.742560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.846613+0000","last_change":"2023-12-07T10:17:45.800231+0000","last_active":"2023-12-07T10:18:00.846613+0000","last_peered":"2023-12-07T10:18:00.846613+0000","last_clean":"2023-12-07T10:18:00.846613+0000","last_became_active":"2023-12-07T10:17:45.799585+0000","last_became_peered":"2023-12-07T10:17:45.799585+0000","last_unstale":"2023-12-07T10:18:00.846613+0000","last_undegraded":"2023-12-07T10:18:00.846613+0000","last_fullsized":"2023-12-07T10:18:00.846613+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:08:36.887582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.756917+0000","last_change":"2023-12-07T10:17:40.758663+0000","last_active":"2023-12-07T10:18:12.756917+0000","last_peered":"2023-12-07T10:18:12.756917+0000","last_clean":"2023-12-07T10:18:12.756917+0000","last_became_active":"2023-12-07T10:17:40.753611+0000","last_became_peered":"2023-12-07T10:17:40.753611+0000","last_unstale":"2023-12-07T10:18:12.756917+0000","last_undegraded":"2023-12-07T10:18:12.756917+0000","last_fullsized":"2023-12-07T10:18:12.756917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:00:59.256468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849063+0000","last_change":"2023-12-07T10:17:41.616546+0000","last_active":"2023-12-07T10:18:00.849063+0000","last_peered":"2023-12-07T10:18:00.849063+0000","last_clean":"2023-12-07T10:18:00.849063+0000","last_became_active":"2023-12-07T10:17:41.615780+0000","last_became_peered":"2023-12-07T10:17:41.615780+0000","last_unstale":"2023-12-07T10:18:00.849063+0000","last_undegraded":"2023-12-07T10:18:00.849063+0000","last_fullsized":"2023-12-07T10:18:00.849063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:22:32.455784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852693+0000","last_change":"2023-12-07T10:17:45.797550+0000","last_active":"2023-12-07T10:18:00.852693+0000","last_peered":"2023-12-07T10:18:00.852693+0000","last_clean":"2023-12-07T10:18:00.852693+0000","last_became_active":"2023-12-07T10:17:45.797140+0000","last_became_peered":"2023-12-07T10:17:45.797140+0000","last_unstale":"2023-12-07T10:18:00.852693+0000","last_undegraded":"2023-12-07T10:18:00.852693+0000","last_fullsized":"2023-12-07T10:18:00.852693+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:53:44.984904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'3","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.965660+0000","last_change":"2023-12-07T10:17:40.748410+0000","last_active":"2023-12-07T10:18:12.965660+0000","last_peered":"2023-12-07T10:18:12.965660+0000","last_clean":"2023-12-07T10:18:12.965660+0000","last_became_active":"2023-12-07T10:17:40.748069+0000","last_became_peered":"2023-12-07T10:17:40.748069+0000","last_unstale":"2023-12-07T10:18:12.965660+0000","last_undegraded":"2023-12-07T10:18:12.965660+0000","last_fullsized":"2023-12-07T10:18:12.965660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:51:48.278974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849910+0000","last_change":"2023-12-07T10:17:41.609248+0000","last_active":"2023-12-07T10:18:00.849910+0000","last_peered":"2023-12-07T10:18:00.849910+0000","last_clean":"2023-12-07T10:18:00.849910+0000","last_became_active":"2023-12-07T10:17:41.608943+0000","last_became_peered":"2023-12-07T10:17:41.608943+0000","last_unstale":"2023-12-07T10:18:00.849910+0000","last_undegraded":"2023-12-07T10:18:00.849910+0000","last_fullsized":"2023-12-07T10:18:00.849910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:29:32.322273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852512+0000","last_change":"2023-12-07T10:17:45.794855+0000","last_active":"2023-12-07T10:18:00.852512+0000","last_peered":"2023-12-07T10:18:00.852512+0000","last_clean":"2023-12-07T10:18:00.852512+0000","last_became_active":"2023-12-07T10:17:45.794410+0000","last_became_peered":"2023-12-07T10:17:45.794410+0000","last_unstale":"2023-12-07T10:18:00.852512+0000","last_undegraded":"2023-12-07T10:18:00.852512+0000","last_fullsized":"2023-12-07T10:18:00.852512+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:57:25.118036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:09.900503+0000","last_change":"2023-12-07T10:17:40.750471+0000","last_active":"2023-12-07T10:18:09.900503+0000","last_peered":"2023-12-07T10:18:09.900503+0000","last_clean":"2023-12-07T10:18:09.900503+0000","last_became_active":"2023-12-07T10:17:40.750176+0000","last_became_peered":"2023-12-07T10:17:40.750176+0000","last_unstale":"2023-12-07T10:18:09.900503+0000","last_undegraded":"2023-12-07T10:18:09.900503+0000","last_fullsized":"2023-12-07T10:18:09.900503+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:04:22.305343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856679+0000","last_change":"2023-12-07T10:17:41.614087+0000","last_active":"2023-12-07T10:18:00.856679+0000","last_peered":"2023-12-07T10:18:00.856679+0000","last_clean":"2023-12-07T10:18:00.856679+0000","last_became_active":"2023-12-07T10:17:41.607689+0000","last_became_peered":"2023-12-07T10:17:41.607689+0000","last_unstale":"2023-12-07T10:18:00.856679+0000","last_undegraded":"2023-12-07T10:18:00.856679+0000","last_fullsized":"2023-12-07T10:18:00.856679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:50:10.316085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.847396+0000","last_change":"2023-12-07T10:17:45.791234+0000","last_active":"2023-12-07T10:18:00.847396+0000","last_peered":"2023-12-07T10:18:00.847396+0000","last_clean":"2023-12-07T10:18:00.847396+0000","last_became_active":"2023-12-07T10:17:45.790785+0000","last_became_peered":"2023-12-07T10:17:45.790785+0000","last_unstale":"2023-12-07T10:18:00.847396+0000","last_undegraded":"2023-12-07T10:18:00.847396+0000","last_fullsized":"2023-12-07T10:18:00.847396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:46:24.980528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'5","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:13.772767+0000","last_change":"2023-12-07T10:17:40.753645+0000","last_active":"2023-12-07T10:18:13.772767+0000","last_peered":"2023-12-07T10:18:13.772767+0000","last_clean":"2023-12-07T10:18:13.772767+0000","last_became_active":"2023-12-07T10:17:40.753326+0000","last_became_peered":"2023-12-07T10:17:40.753326+0000","last_unstale":"2023-12-07T10:18:13.772767+0000","last_undegraded":"2023-12-07T10:18:13.772767+0000","last_fullsized":"2023-12-07T10:18:13.772767+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:43:03.617591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.857048+0000","last_change":"2023-12-07T10:17:41.607968+0000","last_active":"2023-12-07T10:18:00.857048+0000","last_peered":"2023-12-07T10:18:00.857048+0000","last_clean":"2023-12-07T10:18:00.857048+0000","last_became_active":"2023-12-07T10:17:41.607621+0000","last_became_peered":"2023-12-07T10:17:41.607621+0000","last_unstale":"2023-12-07T10:18:00.857048+0000","last_undegraded":"2023-12-07T10:18:00.857048+0000","last_fullsized":"2023-12-07T10:18:00.857048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:55:10.224738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852172+0000","last_change":"2023-12-07T10:17:45.793678+0000","last_active":"2023-12-07T10:18:00.852172+0000","last_peered":"2023-12-07T10:18:00.852172+0000","last_clean":"2023-12-07T10:18:00.852172+0000","last_became_active":"2023-12-07T10:17:45.793283+0000","last_became_peered":"2023-12-07T10:17:45.793283+0000","last_unstale":"2023-12-07T10:18:00.852172+0000","last_undegraded":"2023-12-07T10:18:00.852172+0000","last_fullsized":"2023-12-07T10:18:00.852172+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:33:56.501107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:11.749969+0000","last_change":"2023-12-07T10:17:40.762069+0000","last_active":"2023-12-07T10:18:11.749969+0000","last_peered":"2023-12-07T10:18:11.749969+0000","last_clean":"2023-12-07T10:18:11.749969+0000","last_became_active":"2023-12-07T10:17:40.761752+0000","last_became_peered":"2023-12-07T10:17:40.761752+0000","last_unstale":"2023-12-07T10:18:11.749969+0000","last_undegraded":"2023-12-07T10:18:11.749969+0000","last_fullsized":"2023-12-07T10:18:11.749969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:57:55.736144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856757+0000","last_change":"2023-12-07T10:17:41.607347+0000","last_active":"2023-12-07T10:18:00.856757+0000","last_peered":"2023-12-07T10:18:00.856757+0000","last_clean":"2023-12-07T10:18:00.856757+0000","last_became_active":"2023-12-07T10:17:41.606618+0000","last_became_peered":"2023-12-07T10:17:41.606618+0000","last_unstale":"2023-12-07T10:18:00.856757+0000","last_undegraded":"2023-12-07T10:18:00.856757+0000","last_fullsized":"2023-12-07T10:18:00.856757+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:53:34.748130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855194+0000","last_change":"2023-12-07T10:17:45.791459+0000","last_active":"2023-12-07T10:18:00.855194+0000","last_peered":"2023-12-07T10:18:00.855194+0000","last_clean":"2023-12-07T10:18:00.855194+0000","last_became_active":"2023-12-07T10:17:45.791070+0000","last_became_peered":"2023-12-07T10:17:45.791070+0000","last_unstale":"2023-12-07T10:18:00.855194+0000","last_undegraded":"2023-12-07T10:18:00.855194+0000","last_fullsized":"2023-12-07T10:18:00.855194+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:46:35.985990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.756703+0000","last_change":"2023-12-07T10:17:40.762726+0000","last_active":"2023-12-07T10:18:12.756703+0000","last_peered":"2023-12-07T10:18:12.756703+0000","last_clean":"2023-12-07T10:18:12.756703+0000","last_became_active":"2023-12-07T10:17:40.762378+0000","last_became_peered":"2023-12-07T10:17:40.762378+0000","last_unstale":"2023-12-07T10:18:12.756703+0000","last_undegraded":"2023-12-07T10:18:12.756703+0000","last_fullsized":"2023-12-07T10:18:12.756703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:42:06.193834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856972+0000","last_change":"2023-12-07T10:17:41.602356+0000","last_active":"2023-12-07T10:18:00.856972+0000","last_peered":"2023-12-07T10:18:00.856972+0000","last_clean":"2023-12-07T10:18:00.856972+0000","last_became_active":"2023-12-07T10:17:41.602025+0000","last_became_peered":"2023-12-07T10:17:41.602025+0000","last_unstale":"2023-12-07T10:18:00.856972+0000","last_undegraded":"2023-12-07T10:18:00.856972+0000","last_fullsized":"2023-12-07T10:18:00.856972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:33:04.994510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849064+0000","last_change":"2023-12-07T10:17:41.616766+0000","last_active":"2023-12-07T10:18:00.849064+0000","last_peered":"2023-12-07T10:18:00.849064+0000","last_clean":"2023-12-07T10:18:00.849064+0000","last_became_active":"2023-12-07T10:17:41.616221+0000","last_became_peered":"2023-12-07T10:17:41.616221+0000","last_unstale":"2023-12-07T10:18:00.849064+0000","last_undegraded":"2023-12-07T10:18:00.849064+0000","last_fullsized":"2023-12-07T10:18:00.849064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:16:00.393772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.850406+0000","last_change":"2023-12-07T10:17:41.613646+0000","last_active":"2023-12-07T10:18:00.850406+0000","last_peered":"2023-12-07T10:18:00.850406+0000","last_clean":"2023-12-07T10:18:00.850406+0000","last_became_active":"2023-12-07T10:17:41.613257+0000","last_became_peered":"2023-12-07T10:17:41.613257+0000","last_unstale":"2023-12-07T10:18:00.850406+0000","last_undegraded":"2023-12-07T10:18:00.850406+0000","last_fullsized":"2023-12-07T10:18:00.850406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:10:33.416955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849853+0000","last_change":"2023-12-07T10:17:41.620017+0000","last_active":"2023-12-07T10:18:00.849853+0000","last_peered":"2023-12-07T10:18:00.849853+0000","last_clean":"2023-12-07T10:18:00.849853+0000","last_became_active":"2023-12-07T10:17:41.619647+0000","last_became_peered":"2023-12-07T10:17:41.619647+0000","last_unstale":"2023-12-07T10:18:00.849853+0000","last_undegraded":"2023-12-07T10:18:00.849853+0000","last_fullsized":"2023-12-07T10:18:00.849853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:36:42.891160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852495+0000","last_change":"2023-12-07T10:17:41.610803+0000","last_active":"2023-12-07T10:18:00.852495+0000","last_peered":"2023-12-07T10:18:00.852495+0000","last_clean":"2023-12-07T10:18:00.852495+0000","last_became_active":"2023-12-07T10:17:41.610388+0000","last_became_peered":"2023-12-07T10:17:41.610388+0000","last_unstale":"2023-12-07T10:18:00.852495+0000","last_undegraded":"2023-12-07T10:18:00.852495+0000","last_fullsized":"2023-12-07T10:18:00.852495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:33:25.787281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855345+0000","last_change":"2023-12-07T10:17:41.615916+0000","last_active":"2023-12-07T10:18:00.855345+0000","last_peered":"2023-12-07T10:18:00.855345+0000","last_clean":"2023-12-07T10:18:00.855345+0000","last_became_active":"2023-12-07T10:17:41.615591+0000","last_became_peered":"2023-12-07T10:17:41.615591+0000","last_unstale":"2023-12-07T10:18:00.855345+0000","last_undegraded":"2023-12-07T10:18:00.855345+0000","last_fullsized":"2023-12-07T10:18:00.855345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:40:45.541571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.850876+0000","last_change":"2023-12-07T10:17:41.614735+0000","last_active":"2023-12-07T10:18:00.850876+0000","last_peered":"2023-12-07T10:18:00.850876+0000","last_clean":"2023-12-07T10:18:00.850876+0000","last_became_active":"2023-12-07T10:17:41.614388+0000","last_became_peered":"2023-12-07T10:17:41.614388+0000","last_unstale":"2023-12-07T10:18:00.850876+0000","last_undegraded":"2023-12-07T10:18:00.850876+0000","last_fullsized":"2023-12-07T10:18:00.850876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:10:07.196841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.850893+0000","last_change":"2023-12-07T10:17:41.620059+0000","last_active":"2023-12-07T10:18:00.850893+0000","last_peered":"2023-12-07T10:18:00.850893+0000","last_clean":"2023-12-07T10:18:00.850893+0000","last_became_active":"2023-12-07T10:17:41.619770+0000","last_became_peered":"2023-12-07T10:17:41.619770+0000","last_unstale":"2023-12-07T10:18:00.850893+0000","last_undegraded":"2023-12-07T10:18:00.850893+0000","last_fullsized":"2023-12-07T10:18:00.850893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:13:43.421776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849472+0000","last_change":"2023-12-07T10:17:41.609720+0000","last_active":"2023-12-07T10:18:00.849472+0000","last_peered":"2023-12-07T10:18:00.849472+0000","last_clean":"2023-12-07T10:18:00.849472+0000","last_became_active":"2023-12-07T10:17:41.609417+0000","last_became_peered":"2023-12-07T10:17:41.609417+0000","last_unstale":"2023-12-07T10:18:00.849472+0000","last_undegraded":"2023-12-07T10:18:00.849472+0000","last_fullsized":"2023-12-07T10:18:00.849472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:48:47.768754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855496+0000","last_change":"2023-12-07T10:17:41.620952+0000","last_active":"2023-12-07T10:18:00.855496+0000","last_peered":"2023-12-07T10:18:00.855496+0000","last_clean":"2023-12-07T10:18:00.855496+0000","last_became_active":"2023-12-07T10:17:41.620560+0000","last_became_peered":"2023-12-07T10:17:41.620560+0000","last_unstale":"2023-12-07T10:18:00.855496+0000","last_undegraded":"2023-12-07T10:18:00.855496+0000","last_fullsized":"2023-12-07T10:18:00.855496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:51:37.699511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849382+0000","last_change":"2023-12-07T10:17:41.611908+0000","last_active":"2023-12-07T10:18:00.849382+0000","last_peered":"2023-12-07T10:18:00.849382+0000","last_clean":"2023-12-07T10:18:00.849382+0000","last_became_active":"2023-12-07T10:17:41.610433+0000","last_became_peered":"2023-12-07T10:17:41.610433+0000","last_unstale":"2023-12-07T10:18:00.849382+0000","last_undegraded":"2023-12-07T10:18:00.849382+0000","last_fullsized":"2023-12-07T10:18:00.849382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:42:53.906221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.847644+0000","last_change":"2023-12-07T10:17:41.604116+0000","last_active":"2023-12-07T10:18:00.847644+0000","last_peered":"2023-12-07T10:18:00.847644+0000","last_clean":"2023-12-07T10:18:00.847644+0000","last_became_active":"2023-12-07T10:17:41.603880+0000","last_became_peered":"2023-12-07T10:17:41.603880+0000","last_unstale":"2023-12-07T10:18:00.847644+0000","last_undegraded":"2023-12-07T10:18:00.847644+0000","last_fullsized":"2023-12-07T10:18:00.847644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:56:27.343169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855007+0000","last_change":"2023-12-07T10:17:41.610497+0000","last_active":"2023-12-07T10:18:00.855007+0000","last_peered":"2023-12-07T10:18:00.855007+0000","last_clean":"2023-12-07T10:18:00.855007+0000","last_became_active":"2023-12-07T10:17:41.609956+0000","last_became_peered":"2023-12-07T10:17:41.609956+0000","last_unstale":"2023-12-07T10:18:00.855007+0000","last_undegraded":"2023-12-07T10:18:00.855007+0000","last_fullsized":"2023-12-07T10:18:00.855007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:04:03.967377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849491+0000","last_change":"2023-12-07T10:17:41.612284+0000","last_active":"2023-12-07T10:18:00.849491+0000","last_peered":"2023-12-07T10:18:00.849491+0000","last_clean":"2023-12-07T10:18:00.849491+0000","last_became_active":"2023-12-07T10:17:41.611917+0000","last_became_peered":"2023-12-07T10:17:41.611917+0000","last_unstale":"2023-12-07T10:18:00.849491+0000","last_undegraded":"2023-12-07T10:18:00.849491+0000","last_fullsized":"2023-12-07T10:18:00.849491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:02:27.355883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9266,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":172,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":53,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":64726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383697,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":493950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.65900000000000003}]},{"osd":1,"last update":"Thu Dec 7 10:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.68200000000000005}]},{"osd":2,"last update":"Thu Dec 7 10:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Dec 7 10:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.81999999999999995}]},{"osd":5,"last update":"Thu Dec 7 10:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.71599999999999997}]},{"osd":6,"last update":"Thu Dec 7 10:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.83799999999999997}]},{"osd":7,"last update":"Thu Dec 7 10:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.86599999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":9,"last update":"Thu Dec 7 10:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.98499999999999999}]},{"osd":10,"last update":"Thu Dec 7 10:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.93500000000000005}]}]},{"osd":10,"up_from":72,"seq":309237645333,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1622016,"data_stored":1086807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":1,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.627},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.59299999999999997}]},{"osd":2,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.76900000000000002}]},{"osd":3,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.79000000000000004}]},{"osd":4,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.751}]},{"osd":5,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.93899999999999995}]},{"osd":6,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.95199999999999996}]},{"osd":7,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.63900000000000001}]},{"osd":8,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.87},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.90200000000000002}]},{"osd":9,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.999}]},{"osd":11,"last update":"Thu Dec 7 10:17:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":1.026}]}]},{"osd":9,"up_from":67,"seq":287762808859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":502050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.60399999999999998,"15min":0.62},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.92600000000000005,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58099999999999996,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74299999999999999,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.39100000000000001}]},{"osd":1,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.69399999999999995,"15min":0.70699999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97999999999999998,"5min":1.899,"15min":1.899},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.77700000000000002,"15min":0.79300000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.94399999999999995,"5min":1.409,"15min":1.409},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.80000000000000004,"15min":0.81899999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.91800000000000004,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.873,"15min":0.89800000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.92900000000000005,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":0.59799999999999998}]},{"osd":3,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.72799999999999998,"15min":0.746},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.878,"5min":1.47,"15min":1.47},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.81699999999999995,"15min":0.85499999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84799999999999998,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.42999999999999999}]},{"osd":4,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.80000000000000004,"15min":0.82999999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.85099999999999998,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.88,"15min":0.90200000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.109,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.92800000000000005,"15min":0.95599999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.089,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.84699999999999998,"15min":0.876},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.76000000000000001,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":0.66000000000000003}]},{"osd":6,"last update":"Thu Dec 7 10:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":0.749}]},{"osd":7,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.93400000000000005,"15min":0.95799999999999996},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.0820000000000001,"5min":1.5,"15min":1.5},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":1.016,"15min":1.052},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.1519999999999999,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":0.67500000000000004}]},{"osd":8,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.95599999999999996,"15min":0.97399999999999998},"min":{"1min":0.63800000000000001,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0600000000000001,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":1.0409999999999999,"15min":1.071},"min":{"1min":0.626,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.194,"5min":2.6539999999999999,"15min":2.6539999999999999},"last":0.64200000000000002}]},{"osd":10,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.6240000000000001,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.752},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.3889999999999998,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.74199999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]}]},{"osd":8,"up_from":59,"seq":253403070496,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":493452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77900000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76200000000000001,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65500000000000003}]},{"osd":1,"last update":"Thu Dec 7 10:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.56999999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77400000000000002,"5min":1,"15min":1},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57499999999999996,"15min":0.57399999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79600000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.85299999999999998}]},{"osd":2,"last update":"Thu Dec 7 10:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.44600000000000001,"15min":0.439},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.85299999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.435},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.436,"15min":0.42899999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.56899999999999995}]},{"osd":3,"last update":"Thu Dec 7 10:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58899999999999997,"15min":0.59299999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.58199999999999996},"min":{"1min":0.45500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.67100000000000004}]},{"osd":4,"last update":"Thu Dec 7 10:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57799999999999996,"15min":0.57999999999999996},"min":{"1min":0.373,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79900000000000004,"5min":0.876,"15min":0.876},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.627,"15min":0.628},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88100000000000001,"5min":1.079,"15min":1.079},"last":0.59999999999999998}]},{"osd":5,"last update":"Thu Dec 7 10:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51300000000000001,"15min":0.50800000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.85299999999999998,"5min":1.024,"15min":1.024},"last":0.505}]},{"osd":6,"last update":"Thu Dec 7 10:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64900000000000002,"15min":0.64800000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93500000000000005,"5min":1.046,"15min":1.046},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.66400000000000003,"15min":0.65600000000000003},"min":{"1min":0.46200000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.70499999999999996}]},{"osd":7,"last update":"Thu Dec 7 10:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.65200000000000002,"15min":0.64300000000000002},"min":{"1min":0.42499999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.90900000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.78600000000000003}]},{"osd":9,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.71999999999999997,"15min":0.71499999999999997},"min":{"1min":0.53900000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.74399999999999999,"15min":0.73399999999999999},"min":{"1min":0.59899999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.83299999999999996}]},{"osd":10,"last update":"Thu Dec 7 10:17:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.91800000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299428,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1609728,"data_stored":1083977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.58199999999999996,"15min":0.55200000000000005},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.5369999999999999,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56599999999999995,"15min":0.56000000000000005},"min":{"1min":0.39700000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.36899999999999999}]},{"osd":1,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.436,"15min":0.433},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.45700000000000002,"15min":0.45500000000000002},"min":{"1min":0.28299999999999997,"5min":0.253,"15min":0.253},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.36499999999999999}]},{"osd":2,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.57299999999999995,"15min":0.57999999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.56299999999999994}]},{"osd":3,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.57899999999999996,"15min":0.56899999999999995},"min":{"1min":0.437,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.622,"15min":0.61499999999999999},"min":{"1min":0.47499999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53500000000000003,"15min":0.53700000000000003},"min":{"1min":0.27900000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.76600000000000001,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40799999999999997}]},{"osd":5,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64100000000000001,"15min":0.64500000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.81799999999999995,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81399999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.52600000000000002}]},{"osd":6,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.69799999999999995,"15min":0.69099999999999995},"min":{"1min":0.44400000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.92600000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.69599999999999995,"15min":0.69499999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.54500000000000004}]},{"osd":8,"last update":"Thu Dec 7 10:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.433,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72699999999999998,"15min":0.72899999999999998},"min":{"1min":0.51500000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.66800000000000004}]},{"osd":9,"last update":"Thu Dec 7 10:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.628},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.60399999999999998}]},{"osd":10,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.68300000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]}]},{"osd":6,"up_from":47,"seq":201863462953,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":493458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47399999999999998},"min":{"1min":0.34499999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.63400000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.48299999999999998,"15min":0.47199999999999998},"min":{"1min":0.38300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61499999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.65200000000000002,"15min":0.65800000000000003},"min":{"1min":0.442,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80900000000000005,"5min":1.462,"15min":1.462},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68000000000000005,"15min":0.66400000000000003},"min":{"1min":0.54600000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.52700000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.72999999999999998,"15min":0.752},"min":{"1min":0.51000000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.95899999999999996,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.57399999999999995}]},{"osd":3,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.63600000000000001,"15min":0.65900000000000003},"min":{"1min":0.37,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78400000000000003,"5min":1.605,"15min":1.605},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.83899999999999997,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.54200000000000004}]},{"osd":4,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73999999999999999,"15min":0.75},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.032,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.747,"15min":0.74299999999999999},"min":{"1min":0.54700000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.77100000000000002}]},{"osd":5,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79200000000000004,"15min":0.80100000000000005},"min":{"1min":0.51100000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0840000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77800000000000002,"15min":0.78500000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95399999999999996,"5min":1.75,"15min":1.75},"last":0.82599999999999996}]},{"osd":7,"last update":"Thu Dec 7 10:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.78500000000000003,"15min":0.78700000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.347,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.79600000000000004,"15min":0.78600000000000003},"min":{"1min":0.59499999999999997,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.89400000000000002}]},{"osd":8,"last update":"Thu Dec 7 10:17:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.84399999999999997,"15min":0.86399999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.92800000000000005,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.87,"15min":0.88900000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0269999999999999,"5min":1.581,"15min":1.581},"last":0.81100000000000005}]},{"osd":9,"last update":"Thu Dec 7 10:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.86499999999999999,"15min":0.871},"min":{"1min":0.66800000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1160000000000001,"5min":1.179,"15min":1.179},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.84499999999999997,"15min":0.84899999999999998},"min":{"1min":0.65700000000000003,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.0600000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.91600000000000004}]},{"osd":10,"last update":"Thu Dec 7 10:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.94099999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":497158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60899999999999999,"15min":0.55900000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.96699999999999997,"5min":6.8120000000000003,"15min":6.8120000000000003},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.55800000000000005,"15min":0.54100000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.63500000000000001}]},{"osd":2,"last update":"Thu Dec 7 10:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.55900000000000005,"15min":0.53100000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.52800000000000002,"15min":0.5},"min":{"1min":0.35799999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78000000000000003,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.68600000000000005}]},{"osd":3,"last update":"Thu Dec 7 10:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58999999999999997,"15min":0.55800000000000005},"min":{"1min":0.40899999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.878,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.752},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.54800000000000004,"15min":0.52500000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83599999999999997,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.56000000000000005}]},{"osd":4,"last update":"Thu Dec 7 10:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.40600000000000003,"15min":0.38600000000000001},"min":{"1min":0.28699999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.44,"15min":0.40799999999999997},"min":{"1min":0.318,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.53500000000000003}]},{"osd":5,"last update":"Thu Dec 7 10:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.61399999999999999,"15min":0.57399999999999995},"min":{"1min":0.51400000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.008,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.57799999999999996,"15min":0.55200000000000005},"min":{"1min":0.47999999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.73599999999999999}]},{"osd":6,"last update":"Thu Dec 7 10:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.69299999999999995,"15min":0.66700000000000004},"min":{"1min":0.49199999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93700000000000006,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.65400000000000003,"15min":0.64200000000000002},"min":{"1min":0.52900000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.85699999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.59499999999999997}]},{"osd":7,"last update":"Thu Dec 7 10:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.40300000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.64400000000000002,"15min":0.624},"min":{"1min":0.44600000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.71799999999999997}]},{"osd":8,"last update":"Thu Dec 7 10:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.74099999999999999,"15min":0.72499999999999998},"min":{"1min":0.60299999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0620000000000001,"5min":1.175,"15min":1.175},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.65200000000000002,"15min":0.63800000000000001},"min":{"1min":0.55500000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.69799999999999995}]},{"osd":9,"last update":"Thu Dec 7 10:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.94699999999999995}]},{"osd":10,"last update":"Thu Dec 7 10:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.77000000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":492826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Dec 7 10:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.56200000000000006},"min":{"1min":0.40200000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.55900000000000005,"15min":0.55300000000000005},"min":{"1min":0.432,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.76400000000000001}]},{"osd":2,"last update":"Thu Dec 7 10:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59999999999999998,"15min":0.58399999999999996},"min":{"1min":0.41699999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82799999999999996,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.86499999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.82399999999999995}]},{"osd":3,"last update":"Thu Dec 7 10:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.50900000000000001,"15min":0.48499999999999999},"min":{"1min":0.36699999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":1.032},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.51000000000000001,"15min":0.49299999999999999},"min":{"1min":0.34300000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.91100000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":1.008}]},{"osd":4,"last update":"Thu Dec 7 10:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.57099999999999995,"15min":0.55700000000000005},"min":{"1min":0.439,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.61199999999999999,"15min":0.60099999999999998},"min":{"1min":0.41999999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.69999999999999996}]},{"osd":5,"last update":"Thu Dec 7 10:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.63900000000000001,"15min":0.624},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91000000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.626,"15min":0.624},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85399999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.628}]},{"osd":6,"last update":"Thu Dec 7 10:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.627,"15min":0.625},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79000000000000004,"5min":1.02,"15min":1.02},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64100000000000001,"15min":0.63400000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.58199999999999996}]},{"osd":7,"last update":"Thu Dec 7 10:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.55600000000000005,"5min":0.499,"15min":0.499},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.496},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.57299999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.60799999999999998}]},{"osd":8,"last update":"Thu Dec 7 10:17:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.78300000000000003,"15min":0.79400000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.79700000000000004,"15min":0.80800000000000005},"min":{"1min":0.54500000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.89400000000000002,"5min":1.171,"15min":1.171},"last":0.875}]},{"osd":9,"last update":"Thu Dec 7 10:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.85799999999999998}]},{"osd":10,"last update":"Thu Dec 7 10:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.84299999999999997}]},{"osd":11,"last update":"Thu Dec 7 10:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":1.085},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":1.0589999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":533543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53200000000000003,"15min":0.504},"min":{"1min":0.439,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.42399999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.52000000000000002}]},{"osd":1,"last update":"Thu Dec 7 10:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.55300000000000005,"15min":0.51900000000000002},"min":{"1min":0.40699999999999997,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79900000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.53800000000000003,"15min":0.51500000000000001},"min":{"1min":0.40000000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76700000000000002,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.58799999999999997}]},{"osd":3,"last update":"Thu Dec 7 10:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.58799999999999997,"15min":0.55300000000000005},"min":{"1min":0.45000000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.58299999999999996,"15min":0.55700000000000005},"min":{"1min":0.40999999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.622}]},{"osd":4,"last update":"Thu Dec 7 10:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.55500000000000005,"15min":0.53500000000000003},"min":{"1min":0.42399999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55700000000000005,"15min":0.52900000000000003},"min":{"1min":0.38,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85699999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.79000000000000004}]},{"osd":5,"last update":"Thu Dec 7 10:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.45400000000000001,"15min":0.435},"min":{"1min":0.313,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76500000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.47799999999999998,"15min":0.46400000000000002},"min":{"1min":0.311,"5min":0.307,"15min":0.307},"max":{"1min":0.82799999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.54500000000000004}]},{"osd":6,"last update":"Thu Dec 7 10:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.68100000000000005,"15min":0.67000000000000004},"min":{"1min":0.501,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.91800000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.67200000000000004,"15min":0.65400000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.63600000000000001}]},{"osd":7,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.68300000000000005,"15min":0.68999999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89900000000000002,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.67900000000000005,"15min":0.68700000000000006},"min":{"1min":0.41599999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.68500000000000005}]},{"osd":8,"last update":"Thu Dec 7 10:17:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.87,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.66000000000000003,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93799999999999994,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.71199999999999997}]},{"osd":9,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.78800000000000003,"15min":0.78400000000000003},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.754},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0389999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.72799999999999998}]},{"osd":10,"last update":"Thu Dec 7 10:17:32 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.84099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]}]},{"osd":3,"up_from":26,"seq":111669149751,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":492835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.435,"15min":0.39800000000000002},"min":{"1min":0.309,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.44900000000000001,"15min":0.40699999999999997},"min":{"1min":0.33700000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.77600000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.42099999999999999}]},{"osd":1,"last update":"Thu Dec 7 10:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.65000000000000002,"15min":0.57399999999999995},"min":{"1min":0.54700000000000004,"5min":0.438,"15min":0.438},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61299999999999999,"15min":0.55300000000000005},"min":{"1min":0.496,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84499999999999997,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.624}]},{"osd":2,"last update":"Thu Dec 7 10:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59899999999999998,"15min":0.58999999999999997},"min":{"1min":0.42899999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.754,"5min":1.087,"15min":1.087},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.65800000000000003,"15min":0.60999999999999999},"min":{"1min":0.48499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.016,"5min":1.177,"15min":1.177},"last":0.58699999999999997}]},{"osd":4,"last update":"Thu Dec 7 10:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.65200000000000002,"15min":0.60899999999999999},"min":{"1min":0.56999999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.61799999999999999,"15min":0.58599999999999997},"min":{"1min":0.498,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Dec 7 10:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.61899999999999999,"15min":0.60899999999999999},"min":{"1min":0.45900000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.70299999999999996,"15min":0.69299999999999995},"min":{"1min":0.48799999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0940000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.64600000000000002}]},{"osd":6,"last update":"Thu Dec 7 10:18:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.46999999999999997,"5min":0.377,"15min":0.377},"max":{"1min":1.0369999999999999,"5min":1.282,"15min":1.282},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60999999999999999,"15min":0.61799999999999999},"min":{"1min":0.39700000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71199999999999997,"5min":1.292,"15min":1.292},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Dec 7 10:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.77400000000000002,"15min":0.76500000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.1799999999999999,"5min":1.323,"15min":1.323},"last":0.88},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.55000000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.83099999999999996}]},{"osd":8,"last update":"Thu Dec 7 10:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.872}]},{"osd":9,"last update":"Thu Dec 7 10:18:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.88,"15min":0.88600000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.224,"5min":1.242,"15min":1.242},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.80100000000000005,"15min":0.81000000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.86299999999999999}]},{"osd":10,"last update":"Thu Dec 7 10:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.94499999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]}]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":493492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.52100000000000002,"15min":0.49099999999999999},"min":{"1min":0.41499999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76500000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Dec 7 10:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.39300000000000002,"15min":0.38400000000000001},"min":{"1min":0.27600000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.39700000000000002,"15min":0.38300000000000001},"min":{"1min":0.26200000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.47699999999999998}]},{"osd":2,"last update":"Thu Dec 7 10:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.57399999999999995,"15min":0.55500000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56299999999999994,"15min":0.53600000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.59599999999999997}]},{"osd":3,"last update":"Thu Dec 7 10:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.55900000000000005,"15min":0.53500000000000003},"min":{"1min":0.40899999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90700000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58199999999999996,"15min":0.56100000000000005},"min":{"1min":0.39900000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88900000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.78500000000000003}]},{"osd":5,"last update":"Thu Dec 7 10:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.627},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.59699999999999998,"15min":0.57699999999999996},"min":{"1min":0.40200000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Dec 7 10:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.41399999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64800000000000002,"15min":0.63600000000000001},"min":{"1min":0.501,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91900000000000004,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.85199999999999998}]},{"osd":7,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.52300000000000002,"15min":0.50900000000000001},"min":{"1min":0.35499999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.56599999999999995,"15min":0.54900000000000004},"min":{"1min":0.39000000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.50900000000000001}]},{"osd":8,"last update":"Thu Dec 7 10:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.68700000000000006,"15min":0.67100000000000004},"min":{"1min":0.437,"5min":0.432,"15min":0.432},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.69399999999999995,"15min":0.67600000000000005},"min":{"1min":0.52600000000000002,"5min":0.502,"15min":0.502},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.64600000000000002}]},{"osd":9,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.76700000000000002,"15min":0.76900000000000002},"min":{"1min":0.57399999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.98799999999999999,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.77100000000000002,"15min":0.75800000000000001},"min":{"1min":0.58699999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.88600000000000001}]},{"osd":10,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.745},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.87}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]}]},{"osd":5,"up_from":39,"seq":167503724590,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1646592,"data_stored":1089826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.373,"5min":0.318,"15min":0.318},"max":{"1min":0.748,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53600000000000003,"15min":0.51800000000000002},"min":{"1min":0.34399999999999997,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.73099999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.626}]},{"osd":1,"last update":"Thu Dec 7 10:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.52100000000000002,"15min":0.503},"min":{"1min":0.35499999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76600000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53400000000000003,"15min":0.52200000000000002},"min":{"1min":0.375,"5min":0.312,"15min":0.312},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.61099999999999999}]},{"osd":2,"last update":"Thu Dec 7 10:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.38500000000000001,"15min":0.373},"min":{"1min":0.27400000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.40200000000000002,"15min":0.38200000000000001},"min":{"1min":0.28999999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.70499999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.54700000000000004}]},{"osd":3,"last update":"Thu Dec 7 10:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55600000000000005,"15min":0.53100000000000003},"min":{"1min":0.38200000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82599999999999996,"5min":0.88,"15min":0.88},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56499999999999995,"15min":0.54700000000000004},"min":{"1min":0.44600000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57499999999999996}]},{"osd":4,"last update":"Thu Dec 7 10:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57599999999999996,"15min":0.55400000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83399999999999996,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.67000000000000004}]},{"osd":6,"last update":"Thu Dec 7 10:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.61799999999999999,"15min":0.58599999999999997},"min":{"1min":0.42999999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.69499999999999995,"15min":0.63200000000000001},"min":{"1min":0.45000000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":5.9720000000000004,"5min":5.9720000000000004,"15min":5.9720000000000004},"last":0.64100000000000001}]},{"osd":7,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.747},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.78500000000000003}]},{"osd":8,"last update":"Thu Dec 7 10:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.624,"15min":0.61899999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.80900000000000005}]},{"osd":9,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.78700000000000003,"15min":0.755},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":4.532,"5min":4.532,"15min":4.532},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76400000000000001,"15min":0.76300000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.89300000000000002}]},{"osd":10,"last update":"Thu Dec 7 10:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":2.927,"5min":2.927,"15min":2.927},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.86099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":45420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-07T10:18:15.118 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-07T10:18:15.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.344+0000 7f107d7ff700 1 -- 172.21.15.119:0/1892768669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1078101560 msgr2=0x7f1078101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:15.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.344+0000 7f107d7ff700 1 --2- 172.21.15.119:0/1892768669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1078101560 0x7f1078101980 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f10680099c0 tx=0x7f1068009cd0 comp rx=0 tx=0).stop 2023-12-07T10:18:15.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.344+0000 7f107d7ff700 1 -- 172.21.15.119:0/1892768669 shutdown_connections 2023-12-07T10:18:15.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.344+0000 7f107d7ff700 1 --2- 172.21.15.119:0/1892768669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10781030f0 0x7f10781099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:15.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.344+0000 7f107d7ff700 1 --2- 172.21.15.119:0/1892768669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1078101560 0x7f1078101980 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:15.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.344+0000 7f107d7ff700 1 --2- 172.21.15.119:0/1892768669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1078102750 0x7f1078102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:15.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.344+0000 7f107d7ff700 1 -- 172.21.15.119:0/1892768669 >> 172.21.15.119:0/1892768669 conn(0x7f10780fcab0 msgr2=0x7f10780fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:15.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.344+0000 7f107d7ff700 1 -- 172.21.15.119:0/1892768669 shutdown_connections 2023-12-07T10:18:15.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.344+0000 7f107d7ff700 1 -- 172.21.15.119:0/1892768669 wait complete. 2023-12-07T10:18:15.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.345+0000 7f107d7ff700 1 Processor -- start 2023-12-07T10:18:15.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.345+0000 7f107d7ff700 1 -- start start 2023-12-07T10:18:15.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.346+0000 7f107d7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1078101560 0x7f1078199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:15.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.346+0000 7f107d7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1078102750 0x7f1078199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:15.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.346+0000 7f107d7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10781030f0 0x7f10781a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:15.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.346+0000 7f107d7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10781a1280 con 0x7f10781030f0 2023-12-07T10:18:15.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.346+0000 7f107d7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10781a13c0 con 0x7f1078101560 2023-12-07T10:18:15.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.346+0000 7f107d7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10781a1550 con 0x7f1078102750 2023-12-07T10:18:15.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.346+0000 7f1076ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1078101560 0x7f1078199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:15.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.346+0000 7f10777fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10781030f0 0x7f10781a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:15.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.346+0000 7f1076ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1078101560 0x7f1078199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:34822/0 (socket says 172.21.15.119:34822) 2023-12-07T10:18:15.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.346+0000 7f1076ffd700 1 -- 172.21.15.119:0/1532429802 learned_addr learned my addr 172.21.15.119:0/1532429802 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:15.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.346+0000 7f10767fc700 1 --2- 172.21.15.119:0/1532429802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1078102750 0x7f1078199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:15.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.346+0000 7f10777fe700 1 -- 172.21.15.119:0/1532429802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1078102750 msgr2=0x7f1078199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:15.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.346+0000 7f10777fe700 1 --2- 172.21.15.119:0/1532429802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1078102750 0x7f1078199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:15.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.346+0000 7f10777fe700 1 -- 172.21.15.119:0/1532429802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1078101560 msgr2=0x7f1078199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:15.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.346+0000 7f10777fe700 1 --2- 172.21.15.119:0/1532429802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1078101560 0x7f1078199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:15.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.346+0000 7f10777fe700 1 -- 172.21.15.119:0/1532429802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1070009c40 con 0x7f10781030f0 2023-12-07T10:18:15.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.347+0000 7f10777fe700 1 --2- 172.21.15.119:0/1532429802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10781030f0 0x7f10781a0cb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f107000efb0 tx=0x7f107000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:15.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.347+0000 7f105ffff700 1 -- 172.21.15.119:0/1532429802 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1070003a00 con 0x7f10781030f0 2023-12-07T10:18:15.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.347+0000 7f105ffff700 1 -- 172.21.15.119:0/1532429802 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f107000f040 con 0x7f10781030f0 2023-12-07T10:18:15.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.347+0000 7f107d7ff700 1 -- 172.21.15.119:0/1532429802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10680095f0 con 0x7f10781030f0 2023-12-07T10:18:15.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.347+0000 7f105ffff700 1 -- 172.21.15.119:0/1532429802 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1070004540 con 0x7f10781030f0 2023-12-07T10:18:15.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.347+0000 7f107d7ff700 1 -- 172.21.15.119:0/1532429802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10781a1b60 con 0x7f10781030f0 2023-12-07T10:18:15.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.348+0000 7f107d7ff700 1 -- 172.21.15.119:0/1532429802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1078068350 con 0x7f10781030f0 2023-12-07T10:18:15.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.350+0000 7f105ffff700 1 -- 172.21.15.119:0/1532429802 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f1070003bf0 con 0x7f10781030f0 2023-12-07T10:18:15.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.351+0000 7f105ffff700 1 --2- 172.21.15.119:0/1532429802 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f10600a1ba0 0x7f10600a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:15.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.351+0000 7f105ffff700 1 -- 172.21.15.119:0/1532429802 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f10700cd6d0 con 0x7f10781030f0 2023-12-07T10:18:15.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.351+0000 7f1076ffd700 1 --2- 172.21.15.119:0/1532429802 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f10600a1ba0 0x7f10600a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:15.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.352+0000 7f1076ffd700 1 --2- 172.21.15.119:0/1532429802 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f10600a1ba0 0x7f10600a4060 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1068000c00 tx=0x7f1068005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:15.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.355+0000 7f105ffff700 1 -- 172.21.15.119:0/1532429802 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f10700966c0 con 0x7f10781030f0 2023-12-07T10:18:15.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.553+0000 7f107d7ff700 1 -- 172.21.15.119:0/1532429802 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1078109690 con 0x7f10600a1ba0 2023-12-07T10:18:15.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.565+0000 7f105ffff700 1 -- 172.21.15.119:0/1532429802 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422574 (secure 0 0 0) 0x7f1078109690 con 0x7f10600a1ba0 2023-12-07T10:18:15.575 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:18:15.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.570+0000 7f107d7ff700 1 -- 172.21.15.119:0/1532429802 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f10600a1ba0 msgr2=0x7f10600a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:15.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.570+0000 7f107d7ff700 1 --2- 172.21.15.119:0/1532429802 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f10600a1ba0 0x7f10600a4060 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1068000c00 tx=0x7f1068005a90 comp rx=0 tx=0).stop 2023-12-07T10:18:15.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.571+0000 7f107d7ff700 1 -- 172.21.15.119:0/1532429802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10781030f0 msgr2=0x7f10781a0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:15.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.571+0000 7f107d7ff700 1 --2- 172.21.15.119:0/1532429802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10781030f0 0x7f10781a0cb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f107000efb0 tx=0x7f107000c3a0 comp rx=0 tx=0).stop 2023-12-07T10:18:15.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.571+0000 7f107d7ff700 1 -- 172.21.15.119:0/1532429802 shutdown_connections 2023-12-07T10:18:15.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.571+0000 7f107d7ff700 1 --2- 172.21.15.119:0/1532429802 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f10600a1ba0 0x7f10600a4060 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:15.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.571+0000 7f107d7ff700 1 --2- 172.21.15.119:0/1532429802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10781030f0 0x7f10781a0cb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:15.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.571+0000 7f107d7ff700 1 --2- 172.21.15.119:0/1532429802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1078101560 0x7f1078199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:15.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.571+0000 7f107d7ff700 1 --2- 172.21.15.119:0/1532429802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1078102750 0x7f1078199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:15.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.571+0000 7f107d7ff700 1 -- 172.21.15.119:0/1532429802 >> 172.21.15.119:0/1532429802 conn(0x7f10780fcab0 msgr2=0x7f1078107f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:15.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.571+0000 7f107d7ff700 1 -- 172.21.15.119:0/1532429802 shutdown_connections 2023-12-07T10:18:15.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:15.571+0000 7f107d7ff700 1 -- 172.21.15.119:0/1532429802 wait complete. 2023-12-07T10:18:15.581 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-12-07T10:18:15.592 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2023-12-07T10:18:14.676298+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":599634,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":157,"num_read_kb":136,"num_write":353,"num_write_kb":3002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":53,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":289,"ondisk_log_size":289,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338056,"kb_used_data":13848,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603768,"statfs":{"total":1151940427776,"available":1151594258432,"internally_reserved":0,"allocated":14180352,"data_stored":7753374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":331855161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4769,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":125,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003497"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849988+0000","last_change":"2023-12-07T10:17:41.614206+0000","last_active":"2023-12-07T10:18:00.849988+0000","last_peered":"2023-12-07T10:18:00.849988+0000","last_clean":"2023-12-07T10:18:00.849988+0000","last_became_active":"2023-12-07T10:17:41.613553+0000","last_became_peered":"2023-12-07T10:17:41.613553+0000","last_unstale":"2023-12-07T10:18:00.849988+0000","last_undegraded":"2023-12-07T10:18:00.849988+0000","last_fullsized":"2023-12-07T10:18:00.849988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:10:22.198414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852964+0000","last_change":"2023-12-07T10:17:41.616917+0000","last_active":"2023-12-07T10:18:00.852964+0000","last_peered":"2023-12-07T10:18:00.852964+0000","last_clean":"2023-12-07T10:18:00.852964+0000","last_became_active":"2023-12-07T10:17:41.615345+0000","last_became_peered":"2023-12-07T10:17:41.615345+0000","last_unstale":"2023-12-07T10:18:00.852964+0000","last_undegraded":"2023-12-07T10:18:00.852964+0000","last_fullsized":"2023-12-07T10:18:00.852964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:22:25.566738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.853214+0000","last_change":"2023-12-07T10:17:41.619767+0000","last_active":"2023-12-07T10:18:00.853214+0000","last_peered":"2023-12-07T10:18:00.853214+0000","last_clean":"2023-12-07T10:18:00.853214+0000","last_became_active":"2023-12-07T10:17:41.619362+0000","last_became_peered":"2023-12-07T10:17:41.619362+0000","last_unstale":"2023-12-07T10:18:00.853214+0000","last_undegraded":"2023-12-07T10:18:00.853214+0000","last_fullsized":"2023-12-07T10:18:00.853214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:12:55.458520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.862478+0000","last_change":"2023-12-07T10:18:01.861930+0000","last_active":"2023-12-07T10:18:01.862478+0000","last_peered":"2023-12-07T10:18:01.862478+0000","last_clean":"2023-12-07T10:18:01.862478+0000","last_became_active":"2023-12-07T10:18:01.861570+0000","last_became_peered":"2023-12-07T10:18:01.861570+0000","last_unstale":"2023-12-07T10:18:01.862478+0000","last_undegraded":"2023-12-07T10:18:01.862478+0000","last_fullsized":"2023-12-07T10:18:01.862478+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:29:45.234261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852580+0000","last_change":"2023-12-07T10:17:41.613890+0000","last_active":"2023-12-07T10:18:00.852580+0000","last_peered":"2023-12-07T10:18:00.852580+0000","last_clean":"2023-12-07T10:18:00.852580+0000","last_became_active":"2023-12-07T10:17:41.613073+0000","last_became_peered":"2023-12-07T10:17:41.613073+0000","last_unstale":"2023-12-07T10:18:00.852580+0000","last_undegraded":"2023-12-07T10:18:00.852580+0000","last_fullsized":"2023-12-07T10:18:00.852580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:17:14.339002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.854664+0000","last_change":"2023-12-07T10:17:41.609585+0000","last_active":"2023-12-07T10:18:00.854664+0000","last_peered":"2023-12-07T10:18:00.854664+0000","last_clean":"2023-12-07T10:18:00.854664+0000","last_became_active":"2023-12-07T10:17:41.609256+0000","last_became_peered":"2023-12-07T10:17:41.609256+0000","last_unstale":"2023-12-07T10:18:00.854664+0000","last_undegraded":"2023-12-07T10:18:00.854664+0000","last_fullsized":"2023-12-07T10:18:00.854664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:20:15.103208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856396+0000","last_change":"2023-12-07T10:17:41.606138+0000","last_active":"2023-12-07T10:18:00.856396+0000","last_peered":"2023-12-07T10:18:00.856396+0000","last_clean":"2023-12-07T10:18:00.856396+0000","last_became_active":"2023-12-07T10:17:41.605818+0000","last_became_peered":"2023-12-07T10:17:41.605818+0000","last_unstale":"2023-12-07T10:18:00.856396+0000","last_undegraded":"2023-12-07T10:18:00.856396+0000","last_fullsized":"2023-12-07T10:18:00.856396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:38:39.244077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.257383+0000","last_change":"2023-12-07T10:18:02.255959+0000","last_active":"2023-12-07T10:18:02.257383+0000","last_peered":"2023-12-07T10:18:02.257383+0000","last_clean":"2023-12-07T10:18:02.257383+0000","last_became_active":"2023-12-07T10:18:02.255560+0000","last_became_peered":"2023-12-07T10:18:02.255560+0000","last_unstale":"2023-12-07T10:18:02.257383+0000","last_undegraded":"2023-12-07T10:18:02.257383+0000","last_fullsized":"2023-12-07T10:18:02.257383+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:05:49.065538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.847884+0000","last_change":"2023-12-07T10:17:41.610225+0000","last_active":"2023-12-07T10:18:00.847884+0000","last_peered":"2023-12-07T10:18:00.847884+0000","last_clean":"2023-12-07T10:18:00.847884+0000","last_became_active":"2023-12-07T10:17:41.609875+0000","last_became_peered":"2023-12-07T10:17:41.609875+0000","last_unstale":"2023-12-07T10:18:00.847884+0000","last_undegraded":"2023-12-07T10:18:00.847884+0000","last_fullsized":"2023-12-07T10:18:00.847884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:00:43.136176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855342+0000","last_change":"2023-12-07T10:17:41.617335+0000","last_active":"2023-12-07T10:18:00.855342+0000","last_peered":"2023-12-07T10:18:00.855342+0000","last_clean":"2023-12-07T10:18:00.855342+0000","last_became_active":"2023-12-07T10:17:41.617005+0000","last_became_peered":"2023-12-07T10:17:41.617005+0000","last_unstale":"2023-12-07T10:18:00.855342+0000","last_undegraded":"2023-12-07T10:18:00.855342+0000","last_fullsized":"2023-12-07T10:18:00.855342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:43:36.855465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.857002+0000","last_change":"2023-12-07T10:17:41.608104+0000","last_active":"2023-12-07T10:18:00.857002+0000","last_peered":"2023-12-07T10:18:00.857002+0000","last_clean":"2023-12-07T10:18:00.857002+0000","last_became_active":"2023-12-07T10:17:41.607769+0000","last_became_peered":"2023-12-07T10:17:41.607769+0000","last_unstale":"2023-12-07T10:18:00.857002+0000","last_undegraded":"2023-12-07T10:18:00.857002+0000","last_fullsized":"2023-12-07T10:18:00.857002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:47:04.853044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852617+0000","last_change":"2023-12-07T10:17:41.611585+0000","last_active":"2023-12-07T10:18:00.852617+0000","last_peered":"2023-12-07T10:18:00.852617+0000","last_clean":"2023-12-07T10:18:00.852617+0000","last_became_active":"2023-12-07T10:17:41.611202+0000","last_became_peered":"2023-12-07T10:17:41.611202+0000","last_unstale":"2023-12-07T10:18:00.852617+0000","last_undegraded":"2023-12-07T10:18:00.852617+0000","last_fullsized":"2023-12-07T10:18:00.852617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:17:01.649424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855084+0000","last_change":"2023-12-07T10:17:41.611116+0000","last_active":"2023-12-07T10:18:00.855084+0000","last_peered":"2023-12-07T10:18:00.855084+0000","last_clean":"2023-12-07T10:18:00.855084+0000","last_became_active":"2023-12-07T10:17:41.610791+0000","last_became_peered":"2023-12-07T10:17:41.610791+0000","last_unstale":"2023-12-07T10:18:00.855084+0000","last_undegraded":"2023-12-07T10:18:00.855084+0000","last_fullsized":"2023-12-07T10:18:00.855084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:48:08.822235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852233+0000","last_change":"2023-12-07T10:17:41.617447+0000","last_active":"2023-12-07T10:18:00.852233+0000","last_peered":"2023-12-07T10:18:00.852233+0000","last_clean":"2023-12-07T10:18:00.852233+0000","last_became_active":"2023-12-07T10:17:41.612638+0000","last_became_peered":"2023-12-07T10:17:41.612638+0000","last_unstale":"2023-12-07T10:18:00.852233+0000","last_undegraded":"2023-12-07T10:18:00.852233+0000","last_fullsized":"2023-12-07T10:18:00.852233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:11:30.384367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852009+0000","last_change":"2023-12-07T10:17:41.611607+0000","last_active":"2023-12-07T10:18:00.852009+0000","last_peered":"2023-12-07T10:18:00.852009+0000","last_clean":"2023-12-07T10:18:00.852009+0000","last_became_active":"2023-12-07T10:17:41.611331+0000","last_became_peered":"2023-12-07T10:17:41.611331+0000","last_unstale":"2023-12-07T10:18:00.852009+0000","last_undegraded":"2023-12-07T10:18:00.852009+0000","last_fullsized":"2023-12-07T10:18:00.852009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:17:53.049922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.847334+0000","last_change":"2023-12-07T10:17:41.610528+0000","last_active":"2023-12-07T10:18:00.847334+0000","last_peered":"2023-12-07T10:18:00.847334+0000","last_clean":"2023-12-07T10:18:00.847334+0000","last_became_active":"2023-12-07T10:17:41.610216+0000","last_became_peered":"2023-12-07T10:17:41.610216+0000","last_unstale":"2023-12-07T10:18:00.847334+0000","last_undegraded":"2023-12-07T10:18:00.847334+0000","last_fullsized":"2023-12-07T10:18:00.847334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:11:36.749537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849875+0000","last_change":"2023-12-07T10:17:41.613967+0000","last_active":"2023-12-07T10:18:00.849875+0000","last_peered":"2023-12-07T10:18:00.849875+0000","last_clean":"2023-12-07T10:18:00.849875+0000","last_became_active":"2023-12-07T10:17:41.613116+0000","last_became_peered":"2023-12-07T10:17:41.613116+0000","last_unstale":"2023-12-07T10:18:00.849875+0000","last_undegraded":"2023-12-07T10:18:00.849875+0000","last_fullsized":"2023-12-07T10:18:00.849875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:46:32.444025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.846417+0000","last_change":"2023-12-07T10:17:41.605478+0000","last_active":"2023-12-07T10:18:00.846417+0000","last_peered":"2023-12-07T10:18:00.846417+0000","last_clean":"2023-12-07T10:18:00.846417+0000","last_became_active":"2023-12-07T10:17:41.605110+0000","last_became_peered":"2023-12-07T10:17:41.605110+0000","last_unstale":"2023-12-07T10:18:00.846417+0000","last_undegraded":"2023-12-07T10:18:00.846417+0000","last_fullsized":"2023-12-07T10:18:00.846417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:53:56.410196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.256377+0000","last_change":"2023-12-07T10:18:02.256377+0000","last_active":"2023-12-07T10:18:02.256377+0000","last_peered":"2023-12-07T10:18:02.256377+0000","last_clean":"2023-12-07T10:18:02.256377+0000","last_became_active":"2023-12-07T10:18:02.255987+0000","last_became_peered":"2023-12-07T10:18:02.255987+0000","last_unstale":"2023-12-07T10:18:02.256377+0000","last_undegraded":"2023-12-07T10:18:02.256377+0000","last_fullsized":"2023-12-07T10:18:02.256377+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:23:34.909187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849924+0000","last_change":"2023-12-07T10:17:45.799925+0000","last_active":"2023-12-07T10:18:00.849924+0000","last_peered":"2023-12-07T10:18:00.849924+0000","last_clean":"2023-12-07T10:18:00.849924+0000","last_became_active":"2023-12-07T10:17:45.799505+0000","last_became_peered":"2023-12-07T10:17:45.799505+0000","last_unstale":"2023-12-07T10:18:00.849924+0000","last_undegraded":"2023-12-07T10:18:00.849924+0000","last_fullsized":"2023-12-07T10:18:00.849924+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:52:51.470333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'13","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.757302+0000","last_change":"2023-12-07T10:17:40.755066+0000","last_active":"2023-12-07T10:18:12.757302+0000","last_peered":"2023-12-07T10:18:12.757302+0000","last_clean":"2023-12-07T10:18:12.757302+0000","last_became_active":"2023-12-07T10:17:40.754709+0000","last_became_peered":"2023-12-07T10:17:40.754709+0000","last_unstale":"2023-12-07T10:18:12.757302+0000","last_undegraded":"2023-12-07T10:18:12.757302+0000","last_fullsized":"2023-12-07T10:18:12.757302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:18:31.148592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849407+0000","last_change":"2023-12-07T10:17:41.619107+0000","last_active":"2023-12-07T10:18:00.849407+0000","last_peered":"2023-12-07T10:18:00.849407+0000","last_clean":"2023-12-07T10:18:00.849407+0000","last_became_active":"2023-12-07T10:17:41.617057+0000","last_became_peered":"2023-12-07T10:17:41.617057+0000","last_unstale":"2023-12-07T10:18:00.849407+0000","last_undegraded":"2023-12-07T10:18:00.849407+0000","last_fullsized":"2023-12-07T10:18:00.849407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:53:16.402933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852207+0000","last_change":"2023-12-07T10:17:45.797582+0000","last_active":"2023-12-07T10:18:00.852207+0000","last_peered":"2023-12-07T10:18:00.852207+0000","last_clean":"2023-12-07T10:18:00.852207+0000","last_became_active":"2023-12-07T10:17:45.796358+0000","last_became_peered":"2023-12-07T10:17:45.796358+0000","last_unstale":"2023-12-07T10:18:00.852207+0000","last_undegraded":"2023-12-07T10:18:00.852207+0000","last_fullsized":"2023-12-07T10:18:00.852207+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:53:19.551459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.857161+0000","last_change":"2023-12-07T10:17:40.762155+0000","last_active":"2023-12-07T10:18:01.857161+0000","last_peered":"2023-12-07T10:18:01.857161+0000","last_clean":"2023-12-07T10:18:01.857161+0000","last_became_active":"2023-12-07T10:17:40.761776+0000","last_became_peered":"2023-12-07T10:17:40.761776+0000","last_unstale":"2023-12-07T10:18:01.857161+0000","last_undegraded":"2023-12-07T10:18:01.857161+0000","last_fullsized":"2023-12-07T10:18:01.857161+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:15:06.096331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.853107+0000","last_change":"2023-12-07T10:17:41.614999+0000","last_active":"2023-12-07T10:18:00.853107+0000","last_peered":"2023-12-07T10:18:00.853107+0000","last_clean":"2023-12-07T10:18:00.853107+0000","last_became_active":"2023-12-07T10:17:41.614673+0000","last_became_peered":"2023-12-07T10:17:41.614673+0000","last_unstale":"2023-12-07T10:18:00.853107+0000","last_undegraded":"2023-12-07T10:18:00.853107+0000","last_fullsized":"2023-12-07T10:18:00.853107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:34:45.378843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852388+0000","last_change":"2023-12-07T10:17:45.799399+0000","last_active":"2023-12-07T10:18:00.852388+0000","last_peered":"2023-12-07T10:18:00.852388+0000","last_clean":"2023-12-07T10:18:00.852388+0000","last_became_active":"2023-12-07T10:17:45.799072+0000","last_became_peered":"2023-12-07T10:17:45.799072+0000","last_unstale":"2023-12-07T10:18:00.852388+0000","last_undegraded":"2023-12-07T10:18:00.852388+0000","last_fullsized":"2023-12-07T10:18:00.852388+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:34:54.663334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:11.748573+0000","last_change":"2023-12-07T10:18:01.993128+0000","last_active":"2023-12-07T10:18:11.748573+0000","last_peered":"2023-12-07T10:18:11.748573+0000","last_clean":"2023-12-07T10:18:11.748573+0000","last_became_active":"2023-12-07T10:18:01.990244+0000","last_became_peered":"2023-12-07T10:18:01.990244+0000","last_unstale":"2023-12-07T10:18:11.748573+0000","last_undegraded":"2023-12-07T10:18:11.748573+0000","last_fullsized":"2023-12-07T10:18:11.748573+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:42:53.778363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.847633+0000","last_change":"2023-12-07T10:17:41.612903+0000","last_active":"2023-12-07T10:18:00.847633+0000","last_peered":"2023-12-07T10:18:00.847633+0000","last_clean":"2023-12-07T10:18:00.847633+0000","last_became_active":"2023-12-07T10:17:41.611968+0000","last_became_peered":"2023-12-07T10:17:41.611968+0000","last_unstale":"2023-12-07T10:18:00.847633+0000","last_undegraded":"2023-12-07T10:18:00.847633+0000","last_fullsized":"2023-12-07T10:18:00.847633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:34:08.442828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852245+0000","last_change":"2023-12-07T10:17:45.803858+0000","last_active":"2023-12-07T10:18:00.852245+0000","last_peered":"2023-12-07T10:18:00.852245+0000","last_clean":"2023-12-07T10:18:00.852245+0000","last_became_active":"2023-12-07T10:17:45.803481+0000","last_became_peered":"2023-12-07T10:17:45.803481+0000","last_unstale":"2023-12-07T10:18:00.852245+0000","last_undegraded":"2023-12-07T10:18:00.852245+0000","last_fullsized":"2023-12-07T10:18:00.852245+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:08:03.728554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852884+0000","last_change":"2023-12-07T10:17:40.751866+0000","last_active":"2023-12-07T10:18:00.852884+0000","last_peered":"2023-12-07T10:18:00.852884+0000","last_clean":"2023-12-07T10:18:00.852884+0000","last_became_active":"2023-12-07T10:17:40.751492+0000","last_became_peered":"2023-12-07T10:17:40.751492+0000","last_unstale":"2023-12-07T10:18:00.852884+0000","last_undegraded":"2023-12-07T10:18:00.852884+0000","last_fullsized":"2023-12-07T10:18:00.852884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:42:53.839651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855907+0000","last_change":"2023-12-07T10:17:41.611326+0000","last_active":"2023-12-07T10:18:00.855907+0000","last_peered":"2023-12-07T10:18:00.855907+0000","last_clean":"2023-12-07T10:18:00.855907+0000","last_became_active":"2023-12-07T10:17:41.610863+0000","last_became_peered":"2023-12-07T10:17:41.610863+0000","last_unstale":"2023-12-07T10:18:00.855907+0000","last_undegraded":"2023-12-07T10:18:00.855907+0000","last_fullsized":"2023-12-07T10:18:00.855907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:07:55.801356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.850230+0000","last_change":"2023-12-07T10:17:45.791409+0000","last_active":"2023-12-07T10:18:00.850230+0000","last_peered":"2023-12-07T10:18:00.850230+0000","last_clean":"2023-12-07T10:18:00.850230+0000","last_became_active":"2023-12-07T10:17:45.791022+0000","last_became_peered":"2023-12-07T10:17:45.791022+0000","last_unstale":"2023-12-07T10:18:00.850230+0000","last_undegraded":"2023-12-07T10:18:00.850230+0000","last_fullsized":"2023-12-07T10:18:00.850230+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:57:14.617409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.738486+0000","last_change":"2023-12-07T10:17:40.745318+0000","last_active":"2023-12-07T10:18:12.738486+0000","last_peered":"2023-12-07T10:18:12.738486+0000","last_clean":"2023-12-07T10:18:12.738486+0000","last_became_active":"2023-12-07T10:17:40.745008+0000","last_became_peered":"2023-12-07T10:17:40.745008+0000","last_unstale":"2023-12-07T10:18:12.738486+0000","last_undegraded":"2023-12-07T10:18:12.738486+0000","last_fullsized":"2023-12-07T10:18:12.738486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:42:28.503421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1010,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.853726+0000","last_change":"2023-12-07T10:17:41.605121+0000","last_active":"2023-12-07T10:18:00.853726+0000","last_peered":"2023-12-07T10:18:00.853726+0000","last_clean":"2023-12-07T10:18:00.853726+0000","last_became_active":"2023-12-07T10:17:41.604770+0000","last_became_peered":"2023-12-07T10:17:41.604770+0000","last_unstale":"2023-12-07T10:18:00.853726+0000","last_undegraded":"2023-12-07T10:18:00.853726+0000","last_fullsized":"2023-12-07T10:18:00.853726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:51:02.151155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.850337+0000","last_change":"2023-12-07T10:17:45.809039+0000","last_active":"2023-12-07T10:18:00.850337+0000","last_peered":"2023-12-07T10:18:00.850337+0000","last_clean":"2023-12-07T10:18:00.850337+0000","last_became_active":"2023-12-07T10:17:45.808622+0000","last_became_peered":"2023-12-07T10:17:45.808622+0000","last_unstale":"2023-12-07T10:18:00.850337+0000","last_undegraded":"2023-12-07T10:18:00.850337+0000","last_fullsized":"2023-12-07T10:18:00.850337+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:16:02.835931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.854526+0000","last_change":"2023-12-07T10:17:40.757876+0000","last_active":"2023-12-07T10:18:00.854526+0000","last_peered":"2023-12-07T10:18:00.854526+0000","last_clean":"2023-12-07T10:18:00.854526+0000","last_became_active":"2023-12-07T10:17:40.757613+0000","last_became_peered":"2023-12-07T10:17:40.757613+0000","last_unstale":"2023-12-07T10:18:00.854526+0000","last_undegraded":"2023-12-07T10:18:00.854526+0000","last_fullsized":"2023-12-07T10:18:00.854526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:30:40.423092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856026+0000","last_change":"2023-12-07T10:17:41.611575+0000","last_active":"2023-12-07T10:18:00.856026+0000","last_peered":"2023-12-07T10:18:00.856026+0000","last_clean":"2023-12-07T10:18:00.856026+0000","last_became_active":"2023-12-07T10:17:41.611042+0000","last_became_peered":"2023-12-07T10:17:41.611042+0000","last_unstale":"2023-12-07T10:18:00.856026+0000","last_undegraded":"2023-12-07T10:18:00.856026+0000","last_fullsized":"2023-12-07T10:18:00.856026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:54:05.589792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849821+0000","last_change":"2023-12-07T10:17:45.796640+0000","last_active":"2023-12-07T10:18:00.849821+0000","last_peered":"2023-12-07T10:18:00.849821+0000","last_clean":"2023-12-07T10:18:00.849821+0000","last_became_active":"2023-12-07T10:17:45.796183+0000","last_became_peered":"2023-12-07T10:17:45.796183+0000","last_unstale":"2023-12-07T10:18:00.849821+0000","last_undegraded":"2023-12-07T10:18:00.849821+0000","last_fullsized":"2023-12-07T10:18:00.849821+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:41:56.890662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:10.744033+0000","last_change":"2023-12-07T10:17:40.760283+0000","last_active":"2023-12-07T10:18:10.744033+0000","last_peered":"2023-12-07T10:18:10.744033+0000","last_clean":"2023-12-07T10:18:10.744033+0000","last_became_active":"2023-12-07T10:17:40.759398+0000","last_became_peered":"2023-12-07T10:17:40.759398+0000","last_unstale":"2023-12-07T10:18:10.744033+0000","last_undegraded":"2023-12-07T10:18:10.744033+0000","last_fullsized":"2023-12-07T10:18:10.744033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:34:37.365490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849107+0000","last_change":"2023-12-07T10:17:41.604150+0000","last_active":"2023-12-07T10:18:00.849107+0000","last_peered":"2023-12-07T10:18:00.849107+0000","last_clean":"2023-12-07T10:18:00.849107+0000","last_became_active":"2023-12-07T10:17:41.603688+0000","last_became_peered":"2023-12-07T10:17:41.603688+0000","last_unstale":"2023-12-07T10:18:00.849107+0000","last_undegraded":"2023-12-07T10:18:00.849107+0000","last_fullsized":"2023-12-07T10:18:00.849107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:52:43.735609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852965+0000","last_change":"2023-12-07T10:17:45.806569+0000","last_active":"2023-12-07T10:18:00.852965+0000","last_peered":"2023-12-07T10:18:00.852965+0000","last_clean":"2023-12-07T10:18:00.852965+0000","last_became_active":"2023-12-07T10:17:45.806099+0000","last_became_peered":"2023-12-07T10:17:45.806099+0000","last_unstale":"2023-12-07T10:18:00.852965+0000","last_undegraded":"2023-12-07T10:18:00.852965+0000","last_fullsized":"2023-12-07T10:18:00.852965+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:02:19.977670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:11.747819+0000","last_change":"2023-12-07T10:17:40.744388+0000","last_active":"2023-12-07T10:18:11.747819+0000","last_peered":"2023-12-07T10:18:11.747819+0000","last_clean":"2023-12-07T10:18:11.747819+0000","last_became_active":"2023-12-07T10:17:40.743887+0000","last_became_peered":"2023-12-07T10:17:40.743887+0000","last_unstale":"2023-12-07T10:18:11.747819+0000","last_undegraded":"2023-12-07T10:18:11.747819+0000","last_fullsized":"2023-12-07T10:18:11.747819+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:18:39.102099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.845569+0000","last_change":"2023-12-07T10:17:41.595616+0000","last_active":"2023-12-07T10:18:00.845569+0000","last_peered":"2023-12-07T10:18:00.845569+0000","last_clean":"2023-12-07T10:18:00.845569+0000","last_became_active":"2023-12-07T10:17:41.595256+0000","last_became_peered":"2023-12-07T10:17:41.595256+0000","last_unstale":"2023-12-07T10:18:00.845569+0000","last_undegraded":"2023-12-07T10:18:00.845569+0000","last_fullsized":"2023-12-07T10:18:00.845569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:08:00.156043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849668+0000","last_change":"2023-12-07T10:17:45.794523+0000","last_active":"2023-12-07T10:18:00.849668+0000","last_peered":"2023-12-07T10:18:00.849668+0000","last_clean":"2023-12-07T10:18:00.849668+0000","last_became_active":"2023-12-07T10:17:45.794139+0000","last_became_peered":"2023-12-07T10:17:45.794139+0000","last_unstale":"2023-12-07T10:18:00.849668+0000","last_undegraded":"2023-12-07T10:18:00.849668+0000","last_fullsized":"2023-12-07T10:18:00.849668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:43:24.767627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'3","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:09.900730+0000","last_change":"2023-12-07T10:17:40.759620+0000","last_active":"2023-12-07T10:18:09.900730+0000","last_peered":"2023-12-07T10:18:09.900730+0000","last_clean":"2023-12-07T10:18:09.900730+0000","last_became_active":"2023-12-07T10:17:40.759372+0000","last_became_peered":"2023-12-07T10:17:40.759372+0000","last_unstale":"2023-12-07T10:18:09.900730+0000","last_undegraded":"2023-12-07T10:18:09.900730+0000","last_fullsized":"2023-12-07T10:18:09.900730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:46:14.459409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.847937+0000","last_change":"2023-12-07T10:17:41.601152+0000","last_active":"2023-12-07T10:18:00.847937+0000","last_peered":"2023-12-07T10:18:00.847937+0000","last_clean":"2023-12-07T10:18:00.847937+0000","last_became_active":"2023-12-07T10:17:41.600147+0000","last_became_peered":"2023-12-07T10:17:41.600147+0000","last_unstale":"2023-12-07T10:18:00.847937+0000","last_undegraded":"2023-12-07T10:18:00.847937+0000","last_fullsized":"2023-12-07T10:18:00.847937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:40:46.671489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.845356+0000","last_change":"2023-12-07T10:17:45.796229+0000","last_active":"2023-12-07T10:18:00.845356+0000","last_peered":"2023-12-07T10:18:00.845356+0000","last_clean":"2023-12-07T10:18:00.845356+0000","last_became_active":"2023-12-07T10:17:45.795582+0000","last_became_peered":"2023-12-07T10:17:45.795582+0000","last_unstale":"2023-12-07T10:18:00.845356+0000","last_undegraded":"2023-12-07T10:18:00.845356+0000","last_fullsized":"2023-12-07T10:18:00.845356+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:29:32.853975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:09.901229+0000","last_change":"2023-12-07T10:17:40.748353+0000","last_active":"2023-12-07T10:18:09.901229+0000","last_peered":"2023-12-07T10:18:09.901229+0000","last_clean":"2023-12-07T10:18:09.901229+0000","last_became_active":"2023-12-07T10:17:40.748050+0000","last_became_peered":"2023-12-07T10:17:40.748050+0000","last_unstale":"2023-12-07T10:18:09.901229+0000","last_undegraded":"2023-12-07T10:18:09.901229+0000","last_fullsized":"2023-12-07T10:18:09.901229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:18:15.298534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.256728+0000","last_change":"2023-12-07T10:18:02.256728+0000","last_active":"2023-12-07T10:18:02.256728+0000","last_peered":"2023-12-07T10:18:02.256728+0000","last_clean":"2023-12-07T10:18:02.256728+0000","last_became_active":"2023-12-07T10:18:02.256271+0000","last_became_peered":"2023-12-07T10:18:02.256271+0000","last_unstale":"2023-12-07T10:18:02.256728+0000","last_undegraded":"2023-12-07T10:18:02.256728+0000","last_fullsized":"2023-12-07T10:18:02.256728+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:27:00.198934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852553+0000","last_change":"2023-12-07T10:17:45.796903+0000","last_active":"2023-12-07T10:18:00.852553+0000","last_peered":"2023-12-07T10:18:00.852553+0000","last_clean":"2023-12-07T10:18:00.852553+0000","last_became_active":"2023-12-07T10:17:45.796454+0000","last_became_peered":"2023-12-07T10:17:45.796454+0000","last_unstale":"2023-12-07T10:18:00.852553+0000","last_undegraded":"2023-12-07T10:18:00.852553+0000","last_fullsized":"2023-12-07T10:18:00.852553+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:46:36.955396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:10.734628+0000","last_change":"2023-12-07T10:17:40.756045+0000","last_active":"2023-12-07T10:18:10.734628+0000","last_peered":"2023-12-07T10:18:10.734628+0000","last_clean":"2023-12-07T10:18:10.734628+0000","last_became_active":"2023-12-07T10:17:40.755763+0000","last_became_peered":"2023-12-07T10:17:40.755763+0000","last_unstale":"2023-12-07T10:18:10.734628+0000","last_undegraded":"2023-12-07T10:18:10.734628+0000","last_fullsized":"2023-12-07T10:18:10.734628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:01:08.009076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.847716+0000","last_change":"2023-12-07T10:17:41.611178+0000","last_active":"2023-12-07T10:18:00.847716+0000","last_peered":"2023-12-07T10:18:00.847716+0000","last_clean":"2023-12-07T10:18:00.847716+0000","last_became_active":"2023-12-07T10:17:41.610750+0000","last_became_peered":"2023-12-07T10:17:41.610750+0000","last_unstale":"2023-12-07T10:18:00.847716+0000","last_undegraded":"2023-12-07T10:18:00.847716+0000","last_fullsized":"2023-12-07T10:18:00.847716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:56:30.763523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.845439+0000","last_change":"2023-12-07T10:17:45.800183+0000","last_active":"2023-12-07T10:18:00.845439+0000","last_peered":"2023-12-07T10:18:00.845439+0000","last_clean":"2023-12-07T10:18:00.845439+0000","last_became_active":"2023-12-07T10:17:45.798598+0000","last_became_peered":"2023-12-07T10:17:45.798598+0000","last_unstale":"2023-12-07T10:18:00.845439+0000","last_undegraded":"2023-12-07T10:18:00.845439+0000","last_fullsized":"2023-12-07T10:18:00.845439+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:06:22.162951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'12","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:09.901114+0000","last_change":"2023-12-07T10:17:40.752481+0000","last_active":"2023-12-07T10:18:09.901114+0000","last_peered":"2023-12-07T10:18:09.901114+0000","last_clean":"2023-12-07T10:18:09.901114+0000","last_became_active":"2023-12-07T10:17:40.752158+0000","last_became_peered":"2023-12-07T10:17:40.752158+0000","last_unstale":"2023-12-07T10:18:09.901114+0000","last_undegraded":"2023-12-07T10:18:09.901114+0000","last_fullsized":"2023-12-07T10:18:09.901114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:43:27.963014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3840,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.853035+0000","last_change":"2023-12-07T10:17:41.614107+0000","last_active":"2023-12-07T10:18:00.853035+0000","last_peered":"2023-12-07T10:18:00.853035+0000","last_clean":"2023-12-07T10:18:00.853035+0000","last_became_active":"2023-12-07T10:17:41.613743+0000","last_became_peered":"2023-12-07T10:17:41.613743+0000","last_unstale":"2023-12-07T10:18:00.853035+0000","last_undegraded":"2023-12-07T10:18:00.853035+0000","last_fullsized":"2023-12-07T10:18:00.853035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:49:03.200387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856289+0000","last_change":"2023-12-07T10:17:45.804282+0000","last_active":"2023-12-07T10:18:00.856289+0000","last_peered":"2023-12-07T10:18:00.856289+0000","last_clean":"2023-12-07T10:18:00.856289+0000","last_became_active":"2023-12-07T10:17:45.803794+0000","last_became_peered":"2023-12-07T10:17:45.803794+0000","last_unstale":"2023-12-07T10:18:00.856289+0000","last_undegraded":"2023-12-07T10:18:00.856289+0000","last_fullsized":"2023-12-07T10:18:00.856289+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:31:26.887565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.756900+0000","last_change":"2023-12-07T10:17:40.747204+0000","last_active":"2023-12-07T10:18:12.756900+0000","last_peered":"2023-12-07T10:18:12.756900+0000","last_clean":"2023-12-07T10:18:12.756900+0000","last_became_active":"2023-12-07T10:17:40.746916+0000","last_became_peered":"2023-12-07T10:17:40.746916+0000","last_unstale":"2023-12-07T10:18:12.756900+0000","last_undegraded":"2023-12-07T10:18:12.756900+0000","last_fullsized":"2023-12-07T10:18:12.756900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:53:49.228320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855125+0000","last_change":"2023-12-07T10:17:41.606895+0000","last_active":"2023-12-07T10:18:00.855125+0000","last_peered":"2023-12-07T10:18:00.855125+0000","last_clean":"2023-12-07T10:18:00.855125+0000","last_became_active":"2023-12-07T10:17:41.606596+0000","last_became_peered":"2023-12-07T10:17:41.606596+0000","last_unstale":"2023-12-07T10:18:00.855125+0000","last_undegraded":"2023-12-07T10:18:00.855125+0000","last_fullsized":"2023-12-07T10:18:00.855125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:34:25.166906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849382+0000","last_change":"2023-12-07T10:17:45.807101+0000","last_active":"2023-12-07T10:18:00.849382+0000","last_peered":"2023-12-07T10:18:00.849382+0000","last_clean":"2023-12-07T10:18:00.849382+0000","last_became_active":"2023-12-07T10:17:45.806770+0000","last_became_peered":"2023-12-07T10:17:45.806770+0000","last_unstale":"2023-12-07T10:18:00.849382+0000","last_undegraded":"2023-12-07T10:18:00.849382+0000","last_fullsized":"2023-12-07T10:18:00.849382+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:39:32.522968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:11.750248+0000","last_change":"2023-12-07T10:17:40.754856+0000","last_active":"2023-12-07T10:18:11.750248+0000","last_peered":"2023-12-07T10:18:11.750248+0000","last_clean":"2023-12-07T10:18:11.750248+0000","last_became_active":"2023-12-07T10:17:40.754488+0000","last_became_peered":"2023-12-07T10:17:40.754488+0000","last_unstale":"2023-12-07T10:18:11.750248+0000","last_undegraded":"2023-12-07T10:18:11.750248+0000","last_fullsized":"2023-12-07T10:18:11.750248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:44:50.960216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856759+0000","last_change":"2023-12-07T10:17:41.606963+0000","last_active":"2023-12-07T10:18:00.856759+0000","last_peered":"2023-12-07T10:18:00.856759+0000","last_clean":"2023-12-07T10:18:00.856759+0000","last_became_active":"2023-12-07T10:17:41.606408+0000","last_became_peered":"2023-12-07T10:17:41.606408+0000","last_unstale":"2023-12-07T10:18:00.856759+0000","last_undegraded":"2023-12-07T10:18:00.856759+0000","last_fullsized":"2023-12-07T10:18:00.856759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:25:47.906649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.846470+0000","last_change":"2023-12-07T10:17:45.803867+0000","last_active":"2023-12-07T10:18:00.846470+0000","last_peered":"2023-12-07T10:18:00.846470+0000","last_clean":"2023-12-07T10:18:00.846470+0000","last_became_active":"2023-12-07T10:17:45.802664+0000","last_became_peered":"2023-12-07T10:17:45.802664+0000","last_unstale":"2023-12-07T10:18:00.846470+0000","last_undegraded":"2023-12-07T10:18:00.846470+0000","last_fullsized":"2023-12-07T10:18:00.846470+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:32:33.975307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:11.747193+0000","last_change":"2023-12-07T10:17:40.747494+0000","last_active":"2023-12-07T10:18:11.747193+0000","last_peered":"2023-12-07T10:18:11.747193+0000","last_clean":"2023-12-07T10:18:11.747193+0000","last_became_active":"2023-12-07T10:17:40.747157+0000","last_became_peered":"2023-12-07T10:17:40.747157+0000","last_unstale":"2023-12-07T10:18:11.747193+0000","last_undegraded":"2023-12-07T10:18:11.747193+0000","last_fullsized":"2023-12-07T10:18:11.747193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:45:24.395418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856481+0000","last_change":"2023-12-07T10:17:41.609495+0000","last_active":"2023-12-07T10:18:00.856481+0000","last_peered":"2023-12-07T10:18:00.856481+0000","last_clean":"2023-12-07T10:18:00.856481+0000","last_became_active":"2023-12-07T10:17:41.609181+0000","last_became_peered":"2023-12-07T10:17:41.609181+0000","last_unstale":"2023-12-07T10:18:00.856481+0000","last_undegraded":"2023-12-07T10:18:00.856481+0000","last_fullsized":"2023-12-07T10:18:00.856481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:20:33.473824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855858+0000","last_change":"2023-12-07T10:17:45.808862+0000","last_active":"2023-12-07T10:18:00.855858+0000","last_peered":"2023-12-07T10:18:00.855858+0000","last_clean":"2023-12-07T10:18:00.855858+0000","last_became_active":"2023-12-07T10:17:45.808461+0000","last_became_peered":"2023-12-07T10:17:45.808461+0000","last_unstale":"2023-12-07T10:18:00.855858+0000","last_undegraded":"2023-12-07T10:18:00.855858+0000","last_fullsized":"2023-12-07T10:18:00.855858+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:09:12.598126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'16","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:10.743667+0000","last_change":"2023-12-07T10:18:02.263709+0000","last_active":"2023-12-07T10:18:10.743667+0000","last_peered":"2023-12-07T10:18:10.743667+0000","last_clean":"2023-12-07T10:18:10.743667+0000","last_became_active":"2023-12-07T10:18:02.255788+0000","last_became_peered":"2023-12-07T10:18:02.255788+0000","last_unstale":"2023-12-07T10:18:10.743667+0000","last_undegraded":"2023-12-07T10:18:10.743667+0000","last_fullsized":"2023-12-07T10:18:10.743667+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:12:04.976226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.854358+0000","last_change":"2023-12-07T10:17:41.610509+0000","last_active":"2023-12-07T10:18:00.854358+0000","last_peered":"2023-12-07T10:18:00.854358+0000","last_clean":"2023-12-07T10:18:00.854358+0000","last_became_active":"2023-12-07T10:17:41.610121+0000","last_became_peered":"2023-12-07T10:17:41.610121+0000","last_unstale":"2023-12-07T10:18:00.854358+0000","last_undegraded":"2023-12-07T10:18:00.854358+0000","last_fullsized":"2023-12-07T10:18:00.854358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:53:17.365880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.854272+0000","last_change":"2023-12-07T10:17:45.795818+0000","last_active":"2023-12-07T10:18:00.854272+0000","last_peered":"2023-12-07T10:18:00.854272+0000","last_clean":"2023-12-07T10:18:00.854272+0000","last_became_active":"2023-12-07T10:17:45.795477+0000","last_became_peered":"2023-12-07T10:17:45.795477+0000","last_unstale":"2023-12-07T10:18:00.854272+0000","last_undegraded":"2023-12-07T10:18:00.854272+0000","last_fullsized":"2023-12-07T10:18:00.854272+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:17:21.494654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.859473+0000","last_change":"2023-12-07T10:17:40.751791+0000","last_active":"2023-12-07T10:18:01.859473+0000","last_peered":"2023-12-07T10:18:01.859473+0000","last_clean":"2023-12-07T10:18:01.859473+0000","last_became_active":"2023-12-07T10:17:40.751502+0000","last_became_peered":"2023-12-07T10:17:40.751502+0000","last_unstale":"2023-12-07T10:18:01.859473+0000","last_undegraded":"2023-12-07T10:18:01.859473+0000","last_fullsized":"2023-12-07T10:18:01.859473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:15:52.319265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.256247+0000","last_change":"2023-12-07T10:18:02.256247+0000","last_active":"2023-12-07T10:18:02.256247+0000","last_peered":"2023-12-07T10:18:02.256247+0000","last_clean":"2023-12-07T10:18:02.256247+0000","last_became_active":"2023-12-07T10:18:02.255732+0000","last_became_peered":"2023-12-07T10:18:02.255732+0000","last_unstale":"2023-12-07T10:18:02.256247+0000","last_undegraded":"2023-12-07T10:18:02.256247+0000","last_fullsized":"2023-12-07T10:18:02.256247+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:00:06.516804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.853635+0000","last_change":"2023-12-07T10:17:45.791966+0000","last_active":"2023-12-07T10:18:00.853635+0000","last_peered":"2023-12-07T10:18:00.853635+0000","last_clean":"2023-12-07T10:18:00.853635+0000","last_became_active":"2023-12-07T10:17:45.791613+0000","last_became_peered":"2023-12-07T10:17:45.791613+0000","last_unstale":"2023-12-07T10:18:00.853635+0000","last_undegraded":"2023-12-07T10:18:00.853635+0000","last_fullsized":"2023-12-07T10:18:00.853635+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:13:02.291289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.853559+0000","last_change":"2023-12-07T10:17:40.762896+0000","last_active":"2023-12-07T10:18:01.853559+0000","last_peered":"2023-12-07T10:18:01.853559+0000","last_clean":"2023-12-07T10:18:01.853559+0000","last_became_active":"2023-12-07T10:17:40.762427+0000","last_became_peered":"2023-12-07T10:17:40.762427+0000","last_unstale":"2023-12-07T10:18:01.853559+0000","last_undegraded":"2023-12-07T10:18:01.853559+0000","last_fullsized":"2023-12-07T10:18:01.853559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:56:35.589548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.854524+0000","last_change":"2023-12-07T10:17:41.610354+0000","last_active":"2023-12-07T10:18:00.854524+0000","last_peered":"2023-12-07T10:18:00.854524+0000","last_clean":"2023-12-07T10:18:00.854524+0000","last_became_active":"2023-12-07T10:17:41.609888+0000","last_became_peered":"2023-12-07T10:17:41.609888+0000","last_unstale":"2023-12-07T10:18:00.854524+0000","last_undegraded":"2023-12-07T10:18:00.854524+0000","last_fullsized":"2023-12-07T10:18:00.854524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:48:19.976949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855124+0000","last_change":"2023-12-07T10:17:45.807324+0000","last_active":"2023-12-07T10:18:00.855124+0000","last_peered":"2023-12-07T10:18:00.855124+0000","last_clean":"2023-12-07T10:18:00.855124+0000","last_became_active":"2023-12-07T10:17:45.806036+0000","last_became_peered":"2023-12-07T10:17:45.806036+0000","last_unstale":"2023-12-07T10:18:00.855124+0000","last_undegraded":"2023-12-07T10:18:00.855124+0000","last_fullsized":"2023-12-07T10:18:00.855124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:54:36.847992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856573+0000","last_change":"2023-12-07T10:17:40.754247+0000","last_active":"2023-12-07T10:18:00.856573+0000","last_peered":"2023-12-07T10:18:00.856573+0000","last_clean":"2023-12-07T10:18:00.856573+0000","last_became_active":"2023-12-07T10:17:40.753964+0000","last_became_peered":"2023-12-07T10:17:40.753964+0000","last_unstale":"2023-12-07T10:18:00.856573+0000","last_undegraded":"2023-12-07T10:18:00.856573+0000","last_fullsized":"2023-12-07T10:18:00.856573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:22:04.832399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.854494+0000","last_change":"2023-12-07T10:17:41.610516+0000","last_active":"2023-12-07T10:18:00.854494+0000","last_peered":"2023-12-07T10:18:00.854494+0000","last_clean":"2023-12-07T10:18:00.854494+0000","last_became_active":"2023-12-07T10:17:41.610184+0000","last_became_peered":"2023-12-07T10:17:41.610184+0000","last_unstale":"2023-12-07T10:18:00.854494+0000","last_undegraded":"2023-12-07T10:18:00.854494+0000","last_fullsized":"2023-12-07T10:18:00.854494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:53:43.100723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855260+0000","last_change":"2023-12-07T10:17:45.804624+0000","last_active":"2023-12-07T10:18:00.855260+0000","last_peered":"2023-12-07T10:18:00.855260+0000","last_clean":"2023-12-07T10:18:00.855260+0000","last_became_active":"2023-12-07T10:17:45.803667+0000","last_became_peered":"2023-12-07T10:17:45.803667+0000","last_unstale":"2023-12-07T10:18:00.855260+0000","last_undegraded":"2023-12-07T10:18:00.855260+0000","last_fullsized":"2023-12-07T10:18:00.855260+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:55:09.642795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855573+0000","last_change":"2023-12-07T10:17:41.618433+0000","last_active":"2023-12-07T10:18:00.855573+0000","last_peered":"2023-12-07T10:18:00.855573+0000","last_clean":"2023-12-07T10:18:00.855573+0000","last_became_active":"2023-12-07T10:17:41.617942+0000","last_became_peered":"2023-12-07T10:17:41.617942+0000","last_unstale":"2023-12-07T10:18:00.855573+0000","last_undegraded":"2023-12-07T10:18:00.855573+0000","last_fullsized":"2023-12-07T10:18:00.855573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:26:48.166157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:10.745062+0000","last_change":"2023-12-07T10:17:40.761907+0000","last_active":"2023-12-07T10:18:10.745062+0000","last_peered":"2023-12-07T10:18:10.745062+0000","last_clean":"2023-12-07T10:18:10.745062+0000","last_became_active":"2023-12-07T10:17:40.761564+0000","last_became_peered":"2023-12-07T10:17:40.761564+0000","last_unstale":"2023-12-07T10:18:10.745062+0000","last_undegraded":"2023-12-07T10:18:10.745062+0000","last_fullsized":"2023-12-07T10:18:10.745062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:57:22.304352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856852+0000","last_change":"2023-12-07T10:17:45.792684+0000","last_active":"2023-12-07T10:18:00.856852+0000","last_peered":"2023-12-07T10:18:00.856852+0000","last_clean":"2023-12-07T10:18:00.856852+0000","last_became_active":"2023-12-07T10:17:45.792172+0000","last_became_peered":"2023-12-07T10:17:45.792172+0000","last_unstale":"2023-12-07T10:18:00.856852+0000","last_undegraded":"2023-12-07T10:18:00.856852+0000","last_fullsized":"2023-12-07T10:18:00.856852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:24:31.084727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849807+0000","last_change":"2023-12-07T10:17:41.612582+0000","last_active":"2023-12-07T10:18:00.849807+0000","last_peered":"2023-12-07T10:18:00.849807+0000","last_clean":"2023-12-07T10:18:00.849807+0000","last_became_active":"2023-12-07T10:17:41.612244+0000","last_became_peered":"2023-12-07T10:17:41.612244+0000","last_unstale":"2023-12-07T10:18:00.849807+0000","last_undegraded":"2023-12-07T10:18:00.849807+0000","last_fullsized":"2023-12-07T10:18:00.849807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:41:53.506037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:11.749673+0000","last_change":"2023-12-07T10:17:40.748565+0000","last_active":"2023-12-07T10:18:11.749673+0000","last_peered":"2023-12-07T10:18:11.749673+0000","last_clean":"2023-12-07T10:18:11.749673+0000","last_became_active":"2023-12-07T10:17:40.748250+0000","last_became_peered":"2023-12-07T10:17:40.748250+0000","last_unstale":"2023-12-07T10:18:11.749673+0000","last_undegraded":"2023-12-07T10:18:11.749673+0000","last_fullsized":"2023-12-07T10:18:11.749673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:26:28.202440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":326,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856336+0000","last_change":"2023-12-07T10:16:33.482173+0000","last_active":"2023-12-07T10:18:00.856336+0000","last_peered":"2023-12-07T10:18:00.856336+0000","last_clean":"2023-12-07T10:18:00.856336+0000","last_became_active":"2023-12-07T10:16:33.481699+0000","last_became_peered":"2023-12-07T10:16:33.481699+0000","last_unstale":"2023-12-07T10:18:00.856336+0000","last_undegraded":"2023-12-07T10:18:00.856336+0000","last_fullsized":"2023-12-07T10:18:00.856336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:13:25.388258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:13:25.388258+0000","last_clean_scrub_stamp":"2023-12-07T10:13:25.388258+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:57:09.583677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856130+0000","last_change":"2023-12-07T10:17:45.806289+0000","last_active":"2023-12-07T10:18:00.856130+0000","last_peered":"2023-12-07T10:18:00.856130+0000","last_clean":"2023-12-07T10:18:00.856130+0000","last_became_active":"2023-12-07T10:17:45.805954+0000","last_became_peered":"2023-12-07T10:17:45.805954+0000","last_unstale":"2023-12-07T10:18:00.856130+0000","last_undegraded":"2023-12-07T10:18:00.856130+0000","last_fullsized":"2023-12-07T10:18:00.856130+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:21:09.955380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855381+0000","last_change":"2023-12-07T10:17:41.612322+0000","last_active":"2023-12-07T10:18:00.855381+0000","last_peered":"2023-12-07T10:18:00.855381+0000","last_clean":"2023-12-07T10:18:00.855381+0000","last_became_active":"2023-12-07T10:17:41.612039+0000","last_became_peered":"2023-12-07T10:17:41.612039+0000","last_unstale":"2023-12-07T10:18:00.855381+0000","last_undegraded":"2023-12-07T10:18:00.855381+0000","last_fullsized":"2023-12-07T10:18:00.855381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:40:49.883742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.853755+0000","last_change":"2023-12-07T10:17:40.749694+0000","last_active":"2023-12-07T10:18:01.853755+0000","last_peered":"2023-12-07T10:18:01.853755+0000","last_clean":"2023-12-07T10:18:01.853755+0000","last_became_active":"2023-12-07T10:17:40.749393+0000","last_became_peered":"2023-12-07T10:17:40.749393+0000","last_unstale":"2023-12-07T10:18:01.853755+0000","last_undegraded":"2023-12-07T10:18:01.853755+0000","last_fullsized":"2023-12-07T10:18:01.853755+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:15:36.949062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849339+0000","last_change":"2023-12-07T10:17:45.805769+0000","last_active":"2023-12-07T10:18:00.849339+0000","last_peered":"2023-12-07T10:18:00.849339+0000","last_clean":"2023-12-07T10:18:00.849339+0000","last_became_active":"2023-12-07T10:17:45.805346+0000","last_became_peered":"2023-12-07T10:17:45.805346+0000","last_unstale":"2023-12-07T10:18:00.849339+0000","last_undegraded":"2023-12-07T10:18:00.849339+0000","last_fullsized":"2023-12-07T10:18:00.849339+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:47:47.781933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'3","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:11.749601+0000","last_change":"2023-12-07T10:18:01.862558+0000","last_active":"2023-12-07T10:18:11.749601+0000","last_peered":"2023-12-07T10:18:11.749601+0000","last_clean":"2023-12-07T10:18:11.749601+0000","last_became_active":"2023-12-07T10:18:01.862063+0000","last_became_peered":"2023-12-07T10:18:01.862063+0000","last_unstale":"2023-12-07T10:18:11.749601+0000","last_undegraded":"2023-12-07T10:18:11.749601+0000","last_fullsized":"2023-12-07T10:18:11.749601+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:25:32.652160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855264+0000","last_change":"2023-12-07T10:17:41.617199+0000","last_active":"2023-12-07T10:18:00.855264+0000","last_peered":"2023-12-07T10:18:00.855264+0000","last_clean":"2023-12-07T10:18:00.855264+0000","last_became_active":"2023-12-07T10:17:41.616901+0000","last_became_peered":"2023-12-07T10:17:41.616901+0000","last_unstale":"2023-12-07T10:18:00.855264+0000","last_undegraded":"2023-12-07T10:18:00.855264+0000","last_fullsized":"2023-12-07T10:18:00.855264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:23:59.433861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849025+0000","last_change":"2023-12-07T10:17:45.790914+0000","last_active":"2023-12-07T10:18:00.849025+0000","last_peered":"2023-12-07T10:18:00.849025+0000","last_clean":"2023-12-07T10:18:00.849025+0000","last_became_active":"2023-12-07T10:17:45.790428+0000","last_became_peered":"2023-12-07T10:17:45.790428+0000","last_unstale":"2023-12-07T10:18:00.849025+0000","last_undegraded":"2023-12-07T10:18:00.849025+0000","last_fullsized":"2023-12-07T10:18:00.849025+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:03:57.516300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'8","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:11.750949+0000","last_change":"2023-12-07T10:18:01.863765+0000","last_active":"2023-12-07T10:18:11.750949+0000","last_peered":"2023-12-07T10:18:11.750949+0000","last_clean":"2023-12-07T10:18:11.750949+0000","last_became_active":"2023-12-07T10:18:01.858926+0000","last_became_peered":"2023-12-07T10:18:01.858926+0000","last_unstale":"2023-12-07T10:18:11.750949+0000","last_undegraded":"2023-12-07T10:18:11.750949+0000","last_fullsized":"2023-12-07T10:18:11.750949+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:44:19.663592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849322+0000","last_change":"2023-12-07T10:17:41.608513+0000","last_active":"2023-12-07T10:18:00.849322+0000","last_peered":"2023-12-07T10:18:00.849322+0000","last_clean":"2023-12-07T10:18:00.849322+0000","last_became_active":"2023-12-07T10:17:41.605049+0000","last_became_peered":"2023-12-07T10:17:41.605049+0000","last_unstale":"2023-12-07T10:18:00.849322+0000","last_undegraded":"2023-12-07T10:18:00.849322+0000","last_fullsized":"2023-12-07T10:18:00.849322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:33:00.730827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.850303+0000","last_change":"2023-12-07T10:17:45.792583+0000","last_active":"2023-12-07T10:18:00.850303+0000","last_peered":"2023-12-07T10:18:00.850303+0000","last_clean":"2023-12-07T10:18:00.850303+0000","last_became_active":"2023-12-07T10:17:45.792162+0000","last_became_peered":"2023-12-07T10:17:45.792162+0000","last_unstale":"2023-12-07T10:18:00.850303+0000","last_undegraded":"2023-12-07T10:18:00.850303+0000","last_fullsized":"2023-12-07T10:18:00.850303+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:47:30.901477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:09.902945+0000","last_change":"2023-12-07T10:17:40.750826+0000","last_active":"2023-12-07T10:18:09.902945+0000","last_peered":"2023-12-07T10:18:09.902945+0000","last_clean":"2023-12-07T10:18:09.902945+0000","last_became_active":"2023-12-07T10:17:40.750519+0000","last_became_peered":"2023-12-07T10:17:40.750519+0000","last_unstale":"2023-12-07T10:18:09.902945+0000","last_undegraded":"2023-12-07T10:18:09.902945+0000","last_fullsized":"2023-12-07T10:18:09.902945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:21:01.008897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.853040+0000","last_change":"2023-12-07T10:17:41.609488+0000","last_active":"2023-12-07T10:18:00.853040+0000","last_peered":"2023-12-07T10:18:00.853040+0000","last_clean":"2023-12-07T10:18:00.853040+0000","last_became_active":"2023-12-07T10:17:41.609061+0000","last_became_peered":"2023-12-07T10:17:41.609061+0000","last_unstale":"2023-12-07T10:18:00.853040+0000","last_undegraded":"2023-12-07T10:18:00.853040+0000","last_fullsized":"2023-12-07T10:18:00.853040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:39:32.729502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852501+0000","last_change":"2023-12-07T10:17:45.797573+0000","last_active":"2023-12-07T10:18:00.852501+0000","last_peered":"2023-12-07T10:18:00.852501+0000","last_clean":"2023-12-07T10:18:00.852501+0000","last_became_active":"2023-12-07T10:17:45.797119+0000","last_became_peered":"2023-12-07T10:17:45.797119+0000","last_unstale":"2023-12-07T10:18:00.852501+0000","last_undegraded":"2023-12-07T10:18:00.852501+0000","last_fullsized":"2023-12-07T10:18:00.852501+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:17:27.100470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:09.901504+0000","last_change":"2023-12-07T10:17:40.753596+0000","last_active":"2023-12-07T10:18:09.901504+0000","last_peered":"2023-12-07T10:18:09.901504+0000","last_clean":"2023-12-07T10:18:09.901504+0000","last_became_active":"2023-12-07T10:17:40.753251+0000","last_became_peered":"2023-12-07T10:17:40.753251+0000","last_unstale":"2023-12-07T10:18:09.901504+0000","last_undegraded":"2023-12-07T10:18:09.901504+0000","last_fullsized":"2023-12-07T10:18:09.901504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:08:30.868794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852298+0000","last_change":"2023-12-07T10:17:41.599689+0000","last_active":"2023-12-07T10:18:00.852298+0000","last_peered":"2023-12-07T10:18:00.852298+0000","last_clean":"2023-12-07T10:18:00.852298+0000","last_became_active":"2023-12-07T10:17:41.599358+0000","last_became_peered":"2023-12-07T10:17:41.599358+0000","last_unstale":"2023-12-07T10:18:00.852298+0000","last_undegraded":"2023-12-07T10:18:00.852298+0000","last_fullsized":"2023-12-07T10:18:00.852298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:44:19.742560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.846613+0000","last_change":"2023-12-07T10:17:45.800231+0000","last_active":"2023-12-07T10:18:00.846613+0000","last_peered":"2023-12-07T10:18:00.846613+0000","last_clean":"2023-12-07T10:18:00.846613+0000","last_became_active":"2023-12-07T10:17:45.799585+0000","last_became_peered":"2023-12-07T10:17:45.799585+0000","last_unstale":"2023-12-07T10:18:00.846613+0000","last_undegraded":"2023-12-07T10:18:00.846613+0000","last_fullsized":"2023-12-07T10:18:00.846613+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:08:36.887582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.756917+0000","last_change":"2023-12-07T10:17:40.758663+0000","last_active":"2023-12-07T10:18:12.756917+0000","last_peered":"2023-12-07T10:18:12.756917+0000","last_clean":"2023-12-07T10:18:12.756917+0000","last_became_active":"2023-12-07T10:17:40.753611+0000","last_became_peered":"2023-12-07T10:17:40.753611+0000","last_unstale":"2023-12-07T10:18:12.756917+0000","last_undegraded":"2023-12-07T10:18:12.756917+0000","last_fullsized":"2023-12-07T10:18:12.756917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:00:59.256468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849063+0000","last_change":"2023-12-07T10:17:41.616546+0000","last_active":"2023-12-07T10:18:00.849063+0000","last_peered":"2023-12-07T10:18:00.849063+0000","last_clean":"2023-12-07T10:18:00.849063+0000","last_became_active":"2023-12-07T10:17:41.615780+0000","last_became_peered":"2023-12-07T10:17:41.615780+0000","last_unstale":"2023-12-07T10:18:00.849063+0000","last_undegraded":"2023-12-07T10:18:00.849063+0000","last_fullsized":"2023-12-07T10:18:00.849063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:22:32.455784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852693+0000","last_change":"2023-12-07T10:17:45.797550+0000","last_active":"2023-12-07T10:18:00.852693+0000","last_peered":"2023-12-07T10:18:00.852693+0000","last_clean":"2023-12-07T10:18:00.852693+0000","last_became_active":"2023-12-07T10:17:45.797140+0000","last_became_peered":"2023-12-07T10:17:45.797140+0000","last_unstale":"2023-12-07T10:18:00.852693+0000","last_undegraded":"2023-12-07T10:18:00.852693+0000","last_fullsized":"2023-12-07T10:18:00.852693+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:53:44.984904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'3","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.965660+0000","last_change":"2023-12-07T10:17:40.748410+0000","last_active":"2023-12-07T10:18:12.965660+0000","last_peered":"2023-12-07T10:18:12.965660+0000","last_clean":"2023-12-07T10:18:12.965660+0000","last_became_active":"2023-12-07T10:17:40.748069+0000","last_became_peered":"2023-12-07T10:17:40.748069+0000","last_unstale":"2023-12-07T10:18:12.965660+0000","last_undegraded":"2023-12-07T10:18:12.965660+0000","last_fullsized":"2023-12-07T10:18:12.965660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:51:48.278974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849910+0000","last_change":"2023-12-07T10:17:41.609248+0000","last_active":"2023-12-07T10:18:00.849910+0000","last_peered":"2023-12-07T10:18:00.849910+0000","last_clean":"2023-12-07T10:18:00.849910+0000","last_became_active":"2023-12-07T10:17:41.608943+0000","last_became_peered":"2023-12-07T10:17:41.608943+0000","last_unstale":"2023-12-07T10:18:00.849910+0000","last_undegraded":"2023-12-07T10:18:00.849910+0000","last_fullsized":"2023-12-07T10:18:00.849910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:29:32.322273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852512+0000","last_change":"2023-12-07T10:17:45.794855+0000","last_active":"2023-12-07T10:18:00.852512+0000","last_peered":"2023-12-07T10:18:00.852512+0000","last_clean":"2023-12-07T10:18:00.852512+0000","last_became_active":"2023-12-07T10:17:45.794410+0000","last_became_peered":"2023-12-07T10:17:45.794410+0000","last_unstale":"2023-12-07T10:18:00.852512+0000","last_undegraded":"2023-12-07T10:18:00.852512+0000","last_fullsized":"2023-12-07T10:18:00.852512+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:57:25.118036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:09.900503+0000","last_change":"2023-12-07T10:17:40.750471+0000","last_active":"2023-12-07T10:18:09.900503+0000","last_peered":"2023-12-07T10:18:09.900503+0000","last_clean":"2023-12-07T10:18:09.900503+0000","last_became_active":"2023-12-07T10:17:40.750176+0000","last_became_peered":"2023-12-07T10:17:40.750176+0000","last_unstale":"2023-12-07T10:18:09.900503+0000","last_undegraded":"2023-12-07T10:18:09.900503+0000","last_fullsized":"2023-12-07T10:18:09.900503+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:04:22.305343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856679+0000","last_change":"2023-12-07T10:17:41.614087+0000","last_active":"2023-12-07T10:18:00.856679+0000","last_peered":"2023-12-07T10:18:00.856679+0000","last_clean":"2023-12-07T10:18:00.856679+0000","last_became_active":"2023-12-07T10:17:41.607689+0000","last_became_peered":"2023-12-07T10:17:41.607689+0000","last_unstale":"2023-12-07T10:18:00.856679+0000","last_undegraded":"2023-12-07T10:18:00.856679+0000","last_fullsized":"2023-12-07T10:18:00.856679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:50:10.316085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.847396+0000","last_change":"2023-12-07T10:17:45.791234+0000","last_active":"2023-12-07T10:18:00.847396+0000","last_peered":"2023-12-07T10:18:00.847396+0000","last_clean":"2023-12-07T10:18:00.847396+0000","last_became_active":"2023-12-07T10:17:45.790785+0000","last_became_peered":"2023-12-07T10:17:45.790785+0000","last_unstale":"2023-12-07T10:18:00.847396+0000","last_undegraded":"2023-12-07T10:18:00.847396+0000","last_fullsized":"2023-12-07T10:18:00.847396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:46:24.980528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'5","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:13.772767+0000","last_change":"2023-12-07T10:17:40.753645+0000","last_active":"2023-12-07T10:18:13.772767+0000","last_peered":"2023-12-07T10:18:13.772767+0000","last_clean":"2023-12-07T10:18:13.772767+0000","last_became_active":"2023-12-07T10:17:40.753326+0000","last_became_peered":"2023-12-07T10:17:40.753326+0000","last_unstale":"2023-12-07T10:18:13.772767+0000","last_undegraded":"2023-12-07T10:18:13.772767+0000","last_fullsized":"2023-12-07T10:18:13.772767+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:43:03.617591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.857048+0000","last_change":"2023-12-07T10:17:41.607968+0000","last_active":"2023-12-07T10:18:00.857048+0000","last_peered":"2023-12-07T10:18:00.857048+0000","last_clean":"2023-12-07T10:18:00.857048+0000","last_became_active":"2023-12-07T10:17:41.607621+0000","last_became_peered":"2023-12-07T10:17:41.607621+0000","last_unstale":"2023-12-07T10:18:00.857048+0000","last_undegraded":"2023-12-07T10:18:00.857048+0000","last_fullsized":"2023-12-07T10:18:00.857048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:55:10.224738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852172+0000","last_change":"2023-12-07T10:17:45.793678+0000","last_active":"2023-12-07T10:18:00.852172+0000","last_peered":"2023-12-07T10:18:00.852172+0000","last_clean":"2023-12-07T10:18:00.852172+0000","last_became_active":"2023-12-07T10:17:45.793283+0000","last_became_peered":"2023-12-07T10:17:45.793283+0000","last_unstale":"2023-12-07T10:18:00.852172+0000","last_undegraded":"2023-12-07T10:18:00.852172+0000","last_fullsized":"2023-12-07T10:18:00.852172+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:33:56.501107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:11.749969+0000","last_change":"2023-12-07T10:17:40.762069+0000","last_active":"2023-12-07T10:18:11.749969+0000","last_peered":"2023-12-07T10:18:11.749969+0000","last_clean":"2023-12-07T10:18:11.749969+0000","last_became_active":"2023-12-07T10:17:40.761752+0000","last_became_peered":"2023-12-07T10:17:40.761752+0000","last_unstale":"2023-12-07T10:18:11.749969+0000","last_undegraded":"2023-12-07T10:18:11.749969+0000","last_fullsized":"2023-12-07T10:18:11.749969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:57:55.736144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856757+0000","last_change":"2023-12-07T10:17:41.607347+0000","last_active":"2023-12-07T10:18:00.856757+0000","last_peered":"2023-12-07T10:18:00.856757+0000","last_clean":"2023-12-07T10:18:00.856757+0000","last_became_active":"2023-12-07T10:17:41.606618+0000","last_became_peered":"2023-12-07T10:17:41.606618+0000","last_unstale":"2023-12-07T10:18:00.856757+0000","last_undegraded":"2023-12-07T10:18:00.856757+0000","last_fullsized":"2023-12-07T10:18:00.856757+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:53:34.748130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855194+0000","last_change":"2023-12-07T10:17:45.791459+0000","last_active":"2023-12-07T10:18:00.855194+0000","last_peered":"2023-12-07T10:18:00.855194+0000","last_clean":"2023-12-07T10:18:00.855194+0000","last_became_active":"2023-12-07T10:17:45.791070+0000","last_became_peered":"2023-12-07T10:17:45.791070+0000","last_unstale":"2023-12-07T10:18:00.855194+0000","last_undegraded":"2023-12-07T10:18:00.855194+0000","last_fullsized":"2023-12-07T10:18:00.855194+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:46:35.985990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.756703+0000","last_change":"2023-12-07T10:17:40.762726+0000","last_active":"2023-12-07T10:18:12.756703+0000","last_peered":"2023-12-07T10:18:12.756703+0000","last_clean":"2023-12-07T10:18:12.756703+0000","last_became_active":"2023-12-07T10:17:40.762378+0000","last_became_peered":"2023-12-07T10:17:40.762378+0000","last_unstale":"2023-12-07T10:18:12.756703+0000","last_undegraded":"2023-12-07T10:18:12.756703+0000","last_fullsized":"2023-12-07T10:18:12.756703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:42:06.193834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856972+0000","last_change":"2023-12-07T10:17:41.602356+0000","last_active":"2023-12-07T10:18:00.856972+0000","last_peered":"2023-12-07T10:18:00.856972+0000","last_clean":"2023-12-07T10:18:00.856972+0000","last_became_active":"2023-12-07T10:17:41.602025+0000","last_became_peered":"2023-12-07T10:17:41.602025+0000","last_unstale":"2023-12-07T10:18:00.856972+0000","last_undegraded":"2023-12-07T10:18:00.856972+0000","last_fullsized":"2023-12-07T10:18:00.856972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:33:04.994510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849064+0000","last_change":"2023-12-07T10:17:41.616766+0000","last_active":"2023-12-07T10:18:00.849064+0000","last_peered":"2023-12-07T10:18:00.849064+0000","last_clean":"2023-12-07T10:18:00.849064+0000","last_became_active":"2023-12-07T10:17:41.616221+0000","last_became_peered":"2023-12-07T10:17:41.616221+0000","last_unstale":"2023-12-07T10:18:00.849064+0000","last_undegraded":"2023-12-07T10:18:00.849064+0000","last_fullsized":"2023-12-07T10:18:00.849064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:16:00.393772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.850406+0000","last_change":"2023-12-07T10:17:41.613646+0000","last_active":"2023-12-07T10:18:00.850406+0000","last_peered":"2023-12-07T10:18:00.850406+0000","last_clean":"2023-12-07T10:18:00.850406+0000","last_became_active":"2023-12-07T10:17:41.613257+0000","last_became_peered":"2023-12-07T10:17:41.613257+0000","last_unstale":"2023-12-07T10:18:00.850406+0000","last_undegraded":"2023-12-07T10:18:00.850406+0000","last_fullsized":"2023-12-07T10:18:00.850406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:10:33.416955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849853+0000","last_change":"2023-12-07T10:17:41.620017+0000","last_active":"2023-12-07T10:18:00.849853+0000","last_peered":"2023-12-07T10:18:00.849853+0000","last_clean":"2023-12-07T10:18:00.849853+0000","last_became_active":"2023-12-07T10:17:41.619647+0000","last_became_peered":"2023-12-07T10:17:41.619647+0000","last_unstale":"2023-12-07T10:18:00.849853+0000","last_undegraded":"2023-12-07T10:18:00.849853+0000","last_fullsized":"2023-12-07T10:18:00.849853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:36:42.891160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852495+0000","last_change":"2023-12-07T10:17:41.610803+0000","last_active":"2023-12-07T10:18:00.852495+0000","last_peered":"2023-12-07T10:18:00.852495+0000","last_clean":"2023-12-07T10:18:00.852495+0000","last_became_active":"2023-12-07T10:17:41.610388+0000","last_became_peered":"2023-12-07T10:17:41.610388+0000","last_unstale":"2023-12-07T10:18:00.852495+0000","last_undegraded":"2023-12-07T10:18:00.852495+0000","last_fullsized":"2023-12-07T10:18:00.852495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:33:25.787281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855345+0000","last_change":"2023-12-07T10:17:41.615916+0000","last_active":"2023-12-07T10:18:00.855345+0000","last_peered":"2023-12-07T10:18:00.855345+0000","last_clean":"2023-12-07T10:18:00.855345+0000","last_became_active":"2023-12-07T10:17:41.615591+0000","last_became_peered":"2023-12-07T10:17:41.615591+0000","last_unstale":"2023-12-07T10:18:00.855345+0000","last_undegraded":"2023-12-07T10:18:00.855345+0000","last_fullsized":"2023-12-07T10:18:00.855345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:40:45.541571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.850876+0000","last_change":"2023-12-07T10:17:41.614735+0000","last_active":"2023-12-07T10:18:00.850876+0000","last_peered":"2023-12-07T10:18:00.850876+0000","last_clean":"2023-12-07T10:18:00.850876+0000","last_became_active":"2023-12-07T10:17:41.614388+0000","last_became_peered":"2023-12-07T10:17:41.614388+0000","last_unstale":"2023-12-07T10:18:00.850876+0000","last_undegraded":"2023-12-07T10:18:00.850876+0000","last_fullsized":"2023-12-07T10:18:00.850876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:10:07.196841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.850893+0000","last_change":"2023-12-07T10:17:41.620059+0000","last_active":"2023-12-07T10:18:00.850893+0000","last_peered":"2023-12-07T10:18:00.850893+0000","last_clean":"2023-12-07T10:18:00.850893+0000","last_became_active":"2023-12-07T10:17:41.619770+0000","last_became_peered":"2023-12-07T10:17:41.619770+0000","last_unstale":"2023-12-07T10:18:00.850893+0000","last_undegraded":"2023-12-07T10:18:00.850893+0000","last_fullsized":"2023-12-07T10:18:00.850893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:13:43.421776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849472+0000","last_change":"2023-12-07T10:17:41.609720+0000","last_active":"2023-12-07T10:18:00.849472+0000","last_peered":"2023-12-07T10:18:00.849472+0000","last_clean":"2023-12-07T10:18:00.849472+0000","last_became_active":"2023-12-07T10:17:41.609417+0000","last_became_peered":"2023-12-07T10:17:41.609417+0000","last_unstale":"2023-12-07T10:18:00.849472+0000","last_undegraded":"2023-12-07T10:18:00.849472+0000","last_fullsized":"2023-12-07T10:18:00.849472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:48:47.768754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855496+0000","last_change":"2023-12-07T10:17:41.620952+0000","last_active":"2023-12-07T10:18:00.855496+0000","last_peered":"2023-12-07T10:18:00.855496+0000","last_clean":"2023-12-07T10:18:00.855496+0000","last_became_active":"2023-12-07T10:17:41.620560+0000","last_became_peered":"2023-12-07T10:17:41.620560+0000","last_unstale":"2023-12-07T10:18:00.855496+0000","last_undegraded":"2023-12-07T10:18:00.855496+0000","last_fullsized":"2023-12-07T10:18:00.855496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:51:37.699511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849382+0000","last_change":"2023-12-07T10:17:41.611908+0000","last_active":"2023-12-07T10:18:00.849382+0000","last_peered":"2023-12-07T10:18:00.849382+0000","last_clean":"2023-12-07T10:18:00.849382+0000","last_became_active":"2023-12-07T10:17:41.610433+0000","last_became_peered":"2023-12-07T10:17:41.610433+0000","last_unstale":"2023-12-07T10:18:00.849382+0000","last_undegraded":"2023-12-07T10:18:00.849382+0000","last_fullsized":"2023-12-07T10:18:00.849382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:42:53.906221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.847644+0000","last_change":"2023-12-07T10:17:41.604116+0000","last_active":"2023-12-07T10:18:00.847644+0000","last_peered":"2023-12-07T10:18:00.847644+0000","last_clean":"2023-12-07T10:18:00.847644+0000","last_became_active":"2023-12-07T10:17:41.603880+0000","last_became_peered":"2023-12-07T10:17:41.603880+0000","last_unstale":"2023-12-07T10:18:00.847644+0000","last_undegraded":"2023-12-07T10:18:00.847644+0000","last_fullsized":"2023-12-07T10:18:00.847644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:56:27.343169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855007+0000","last_change":"2023-12-07T10:17:41.610497+0000","last_active":"2023-12-07T10:18:00.855007+0000","last_peered":"2023-12-07T10:18:00.855007+0000","last_clean":"2023-12-07T10:18:00.855007+0000","last_became_active":"2023-12-07T10:17:41.609956+0000","last_became_peered":"2023-12-07T10:17:41.609956+0000","last_unstale":"2023-12-07T10:18:00.855007+0000","last_undegraded":"2023-12-07T10:18:00.855007+0000","last_fullsized":"2023-12-07T10:18:00.855007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:04:03.967377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849491+0000","last_change":"2023-12-07T10:17:41.612284+0000","last_active":"2023-12-07T10:18:00.849491+0000","last_peered":"2023-12-07T10:18:00.849491+0000","last_clean":"2023-12-07T10:18:00.849491+0000","last_became_active":"2023-12-07T10:17:41.611917+0000","last_became_peered":"2023-12-07T10:17:41.611917+0000","last_unstale":"2023-12-07T10:18:00.849491+0000","last_undegraded":"2023-12-07T10:18:00.849491+0000","last_fullsized":"2023-12-07T10:18:00.849491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:02:27.355883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9266,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":172,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":53,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":64726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383697,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":493950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.65900000000000003}]},{"osd":1,"last update":"Thu Dec 7 10:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.68200000000000005}]},{"osd":2,"last update":"Thu Dec 7 10:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Dec 7 10:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.81999999999999995}]},{"osd":5,"last update":"Thu Dec 7 10:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.71599999999999997}]},{"osd":6,"last update":"Thu Dec 7 10:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.83799999999999997}]},{"osd":7,"last update":"Thu Dec 7 10:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.86599999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":9,"last update":"Thu Dec 7 10:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.98499999999999999}]},{"osd":10,"last update":"Thu Dec 7 10:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.93500000000000005}]}]},{"osd":10,"up_from":72,"seq":309237645333,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1622016,"data_stored":1086807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":1,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.627},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.59299999999999997}]},{"osd":2,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.76900000000000002}]},{"osd":3,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.79000000000000004}]},{"osd":4,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.751}]},{"osd":5,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.93899999999999995}]},{"osd":6,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.95199999999999996}]},{"osd":7,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.63900000000000001}]},{"osd":8,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.87},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.90200000000000002}]},{"osd":9,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.999}]},{"osd":11,"last update":"Thu Dec 7 10:17:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":1.026}]}]},{"osd":9,"up_from":67,"seq":287762808859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":502050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.60399999999999998,"15min":0.62},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.92600000000000005,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58099999999999996,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74299999999999999,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.39100000000000001}]},{"osd":1,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.69399999999999995,"15min":0.70699999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97999999999999998,"5min":1.899,"15min":1.899},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.77700000000000002,"15min":0.79300000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.94399999999999995,"5min":1.409,"15min":1.409},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.80000000000000004,"15min":0.81899999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.91800000000000004,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.873,"15min":0.89800000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.92900000000000005,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":0.59799999999999998}]},{"osd":3,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.72799999999999998,"15min":0.746},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.878,"5min":1.47,"15min":1.47},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.81699999999999995,"15min":0.85499999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84799999999999998,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.42999999999999999}]},{"osd":4,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.80000000000000004,"15min":0.82999999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.85099999999999998,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.88,"15min":0.90200000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.109,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.92800000000000005,"15min":0.95599999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.089,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.84699999999999998,"15min":0.876},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.76000000000000001,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":0.66000000000000003}]},{"osd":6,"last update":"Thu Dec 7 10:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":0.749}]},{"osd":7,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.93400000000000005,"15min":0.95799999999999996},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.0820000000000001,"5min":1.5,"15min":1.5},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":1.016,"15min":1.052},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.1519999999999999,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":0.67500000000000004}]},{"osd":8,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.95599999999999996,"15min":0.97399999999999998},"min":{"1min":0.63800000000000001,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0600000000000001,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":1.0409999999999999,"15min":1.071},"min":{"1min":0.626,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.194,"5min":2.6539999999999999,"15min":2.6539999999999999},"last":0.64200000000000002}]},{"osd":10,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.6240000000000001,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.752},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.3889999999999998,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.74199999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]}]},{"osd":8,"up_from":59,"seq":253403070496,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":493452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77900000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76200000000000001,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65500000000000003}]},{"osd":1,"last update":"Thu Dec 7 10:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.56999999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77400000000000002,"5min":1,"15min":1},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57499999999999996,"15min":0.57399999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79600000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.85299999999999998}]},{"osd":2,"last update":"Thu Dec 7 10:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.44600000000000001,"15min":0.439},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.85299999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.435},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.436,"15min":0.42899999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.56899999999999995}]},{"osd":3,"last update":"Thu Dec 7 10:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58899999999999997,"15min":0.59299999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.58199999999999996},"min":{"1min":0.45500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.67100000000000004}]},{"osd":4,"last update":"Thu Dec 7 10:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57799999999999996,"15min":0.57999999999999996},"min":{"1min":0.373,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79900000000000004,"5min":0.876,"15min":0.876},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.627,"15min":0.628},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88100000000000001,"5min":1.079,"15min":1.079},"last":0.59999999999999998}]},{"osd":5,"last update":"Thu Dec 7 10:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51300000000000001,"15min":0.50800000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.85299999999999998,"5min":1.024,"15min":1.024},"last":0.505}]},{"osd":6,"last update":"Thu Dec 7 10:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64900000000000002,"15min":0.64800000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93500000000000005,"5min":1.046,"15min":1.046},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.66400000000000003,"15min":0.65600000000000003},"min":{"1min":0.46200000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.70499999999999996}]},{"osd":7,"last update":"Thu Dec 7 10:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.65200000000000002,"15min":0.64300000000000002},"min":{"1min":0.42499999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.90900000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.78600000000000003}]},{"osd":9,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.71999999999999997,"15min":0.71499999999999997},"min":{"1min":0.53900000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.74399999999999999,"15min":0.73399999999999999},"min":{"1min":0.59899999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.83299999999999996}]},{"osd":10,"last update":"Thu Dec 7 10:17:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.91800000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299428,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1609728,"data_stored":1083977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.58199999999999996,"15min":0.55200000000000005},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.5369999999999999,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56599999999999995,"15min":0.56000000000000005},"min":{"1min":0.39700000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.36899999999999999}]},{"osd":1,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.436,"15min":0.433},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.45700000000000002,"15min":0.45500000000000002},"min":{"1min":0.28299999999999997,"5min":0.253,"15min":0.253},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.36499999999999999}]},{"osd":2,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.57299999999999995,"15min":0.57999999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.56299999999999994}]},{"osd":3,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.57899999999999996,"15min":0.56899999999999995},"min":{"1min":0.437,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.622,"15min":0.61499999999999999},"min":{"1min":0.47499999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53500000000000003,"15min":0.53700000000000003},"min":{"1min":0.27900000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.76600000000000001,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40799999999999997}]},{"osd":5,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64100000000000001,"15min":0.64500000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.81799999999999995,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81399999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.52600000000000002}]},{"osd":6,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.69799999999999995,"15min":0.69099999999999995},"min":{"1min":0.44400000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.92600000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.69599999999999995,"15min":0.69499999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.54500000000000004}]},{"osd":8,"last update":"Thu Dec 7 10:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.433,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72699999999999998,"15min":0.72899999999999998},"min":{"1min":0.51500000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.66800000000000004}]},{"osd":9,"last update":"Thu Dec 7 10:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.628},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.60399999999999998}]},{"osd":10,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.68300000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]}]},{"osd":6,"up_from":47,"seq":201863462953,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":493458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47399999999999998},"min":{"1min":0.34499999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.63400000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.48299999999999998,"15min":0.47199999999999998},"min":{"1min":0.38300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61499999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.65200000000000002,"15min":0.65800000000000003},"min":{"1min":0.442,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80900000000000005,"5min":1.462,"15min":1.462},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68000000000000005,"15min":0.66400000000000003},"min":{"1min":0.54600000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.52700000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.72999999999999998,"15min":0.752},"min":{"1min":0.51000000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.95899999999999996,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.57399999999999995}]},{"osd":3,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.63600000000000001,"15min":0.65900000000000003},"min":{"1min":0.37,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78400000000000003,"5min":1.605,"15min":1.605},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.83899999999999997,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.54200000000000004}]},{"osd":4,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73999999999999999,"15min":0.75},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.032,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.747,"15min":0.74299999999999999},"min":{"1min":0.54700000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.77100000000000002}]},{"osd":5,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79200000000000004,"15min":0.80100000000000005},"min":{"1min":0.51100000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0840000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77800000000000002,"15min":0.78500000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95399999999999996,"5min":1.75,"15min":1.75},"last":0.82599999999999996}]},{"osd":7,"last update":"Thu Dec 7 10:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.78500000000000003,"15min":0.78700000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.347,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.79600000000000004,"15min":0.78600000000000003},"min":{"1min":0.59499999999999997,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.89400000000000002}]},{"osd":8,"last update":"Thu Dec 7 10:17:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.84399999999999997,"15min":0.86399999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.92800000000000005,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.87,"15min":0.88900000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0269999999999999,"5min":1.581,"15min":1.581},"last":0.81100000000000005}]},{"osd":9,"last update":"Thu Dec 7 10:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.86499999999999999,"15min":0.871},"min":{"1min":0.66800000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1160000000000001,"5min":1.179,"15min":1.179},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.84499999999999997,"15min":0.84899999999999998},"min":{"1min":0.65700000000000003,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.0600000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.91600000000000004}]},{"osd":10,"last update":"Thu Dec 7 10:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.94099999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":497158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60899999999999999,"15min":0.55900000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.96699999999999997,"5min":6.8120000000000003,"15min":6.8120000000000003},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.55800000000000005,"15min":0.54100000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.63500000000000001}]},{"osd":2,"last update":"Thu Dec 7 10:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.55900000000000005,"15min":0.53100000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.52800000000000002,"15min":0.5},"min":{"1min":0.35799999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78000000000000003,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.68600000000000005}]},{"osd":3,"last update":"Thu Dec 7 10:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58999999999999997,"15min":0.55800000000000005},"min":{"1min":0.40899999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.878,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.752},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.54800000000000004,"15min":0.52500000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83599999999999997,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.56000000000000005}]},{"osd":4,"last update":"Thu Dec 7 10:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.40600000000000003,"15min":0.38600000000000001},"min":{"1min":0.28699999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.44,"15min":0.40799999999999997},"min":{"1min":0.318,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.53500000000000003}]},{"osd":5,"last update":"Thu Dec 7 10:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.61399999999999999,"15min":0.57399999999999995},"min":{"1min":0.51400000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.008,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.57799999999999996,"15min":0.55200000000000005},"min":{"1min":0.47999999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.73599999999999999}]},{"osd":6,"last update":"Thu Dec 7 10:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.69299999999999995,"15min":0.66700000000000004},"min":{"1min":0.49199999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93700000000000006,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.65400000000000003,"15min":0.64200000000000002},"min":{"1min":0.52900000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.85699999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.59499999999999997}]},{"osd":7,"last update":"Thu Dec 7 10:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.40300000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.64400000000000002,"15min":0.624},"min":{"1min":0.44600000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.71799999999999997}]},{"osd":8,"last update":"Thu Dec 7 10:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.74099999999999999,"15min":0.72499999999999998},"min":{"1min":0.60299999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0620000000000001,"5min":1.175,"15min":1.175},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.65200000000000002,"15min":0.63800000000000001},"min":{"1min":0.55500000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.69799999999999995}]},{"osd":9,"last update":"Thu Dec 7 10:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.94699999999999995}]},{"osd":10,"last update":"Thu Dec 7 10:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.77000000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":492826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Dec 7 10:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56799999999999995,"15min":0.56200000000000006},"min":{"1min":0.40200000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.55900000000000005,"15min":0.55300000000000005},"min":{"1min":0.432,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.76400000000000001}]},{"osd":2,"last update":"Thu Dec 7 10:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59999999999999998,"15min":0.58399999999999996},"min":{"1min":0.41699999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82799999999999996,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.86499999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.82399999999999995}]},{"osd":3,"last update":"Thu Dec 7 10:16:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.50900000000000001,"15min":0.48499999999999999},"min":{"1min":0.36699999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":1.032},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.51000000000000001,"15min":0.49299999999999999},"min":{"1min":0.34300000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.91100000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":1.008}]},{"osd":4,"last update":"Thu Dec 7 10:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.57099999999999995,"15min":0.55700000000000005},"min":{"1min":0.439,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.61199999999999999,"15min":0.60099999999999998},"min":{"1min":0.41999999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.69999999999999996}]},{"osd":5,"last update":"Thu Dec 7 10:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.63900000000000001,"15min":0.624},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91000000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.626,"15min":0.624},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85399999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.628}]},{"osd":6,"last update":"Thu Dec 7 10:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.627,"15min":0.625},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79000000000000004,"5min":1.02,"15min":1.02},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64100000000000001,"15min":0.63400000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.58199999999999996}]},{"osd":7,"last update":"Thu Dec 7 10:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.55600000000000005,"5min":0.499,"15min":0.499},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.496},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.57299999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.60799999999999998}]},{"osd":8,"last update":"Thu Dec 7 10:17:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.78300000000000003,"15min":0.79400000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.79700000000000004,"15min":0.80800000000000005},"min":{"1min":0.54500000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.89400000000000002,"5min":1.171,"15min":1.171},"last":0.875}]},{"osd":9,"last update":"Thu Dec 7 10:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.85799999999999998}]},{"osd":10,"last update":"Thu Dec 7 10:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.84299999999999997}]},{"osd":11,"last update":"Thu Dec 7 10:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":1.085},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":1.0589999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":533543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53200000000000003,"15min":0.504},"min":{"1min":0.439,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.42399999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.52000000000000002}]},{"osd":1,"last update":"Thu Dec 7 10:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.55300000000000005,"15min":0.51900000000000002},"min":{"1min":0.40699999999999997,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79900000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.53800000000000003,"15min":0.51500000000000001},"min":{"1min":0.40000000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76700000000000002,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.58799999999999997}]},{"osd":3,"last update":"Thu Dec 7 10:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.58799999999999997,"15min":0.55300000000000005},"min":{"1min":0.45000000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.58299999999999996,"15min":0.55700000000000005},"min":{"1min":0.40999999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.622}]},{"osd":4,"last update":"Thu Dec 7 10:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.55500000000000005,"15min":0.53500000000000003},"min":{"1min":0.42399999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55700000000000005,"15min":0.52900000000000003},"min":{"1min":0.38,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85699999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.79000000000000004}]},{"osd":5,"last update":"Thu Dec 7 10:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.45400000000000001,"15min":0.435},"min":{"1min":0.313,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76500000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.47799999999999998,"15min":0.46400000000000002},"min":{"1min":0.311,"5min":0.307,"15min":0.307},"max":{"1min":0.82799999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.54500000000000004}]},{"osd":6,"last update":"Thu Dec 7 10:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.68100000000000005,"15min":0.67000000000000004},"min":{"1min":0.501,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.91800000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.67200000000000004,"15min":0.65400000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.63600000000000001}]},{"osd":7,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.68300000000000005,"15min":0.68999999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89900000000000002,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.67900000000000005,"15min":0.68700000000000006},"min":{"1min":0.41599999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.68500000000000005}]},{"osd":8,"last update":"Thu Dec 7 10:17:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.87,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.66000000000000003,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93799999999999994,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.71199999999999997}]},{"osd":9,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.78800000000000003,"15min":0.78400000000000003},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.754},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0389999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.72799999999999998}]},{"osd":10,"last update":"Thu Dec 7 10:17:32 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.84099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]}]},{"osd":3,"up_from":26,"seq":111669149751,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":492835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.435,"15min":0.39800000000000002},"min":{"1min":0.309,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.44900000000000001,"15min":0.40699999999999997},"min":{"1min":0.33700000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.77600000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.42099999999999999}]},{"osd":1,"last update":"Thu Dec 7 10:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.65000000000000002,"15min":0.57399999999999995},"min":{"1min":0.54700000000000004,"5min":0.438,"15min":0.438},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61299999999999999,"15min":0.55300000000000005},"min":{"1min":0.496,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84499999999999997,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.624}]},{"osd":2,"last update":"Thu Dec 7 10:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59899999999999998,"15min":0.58999999999999997},"min":{"1min":0.42899999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.754,"5min":1.087,"15min":1.087},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.65800000000000003,"15min":0.60999999999999999},"min":{"1min":0.48499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.016,"5min":1.177,"15min":1.177},"last":0.58699999999999997}]},{"osd":4,"last update":"Thu Dec 7 10:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.65200000000000002,"15min":0.60899999999999999},"min":{"1min":0.56999999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.61799999999999999,"15min":0.58599999999999997},"min":{"1min":0.498,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Dec 7 10:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.61899999999999999,"15min":0.60899999999999999},"min":{"1min":0.45900000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.70299999999999996,"15min":0.69299999999999995},"min":{"1min":0.48799999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0940000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.64600000000000002}]},{"osd":6,"last update":"Thu Dec 7 10:18:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.46999999999999997,"5min":0.377,"15min":0.377},"max":{"1min":1.0369999999999999,"5min":1.282,"15min":1.282},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60999999999999999,"15min":0.61799999999999999},"min":{"1min":0.39700000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71199999999999997,"5min":1.292,"15min":1.292},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Dec 7 10:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.77400000000000002,"15min":0.76500000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.1799999999999999,"5min":1.323,"15min":1.323},"last":0.88},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.55000000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.83099999999999996}]},{"osd":8,"last update":"Thu Dec 7 10:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.872}]},{"osd":9,"last update":"Thu Dec 7 10:18:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.88,"15min":0.88600000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.224,"5min":1.242,"15min":1.242},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.80100000000000005,"15min":0.81000000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.86299999999999999}]},{"osd":10,"last update":"Thu Dec 7 10:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.94499999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]}]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":493492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.52100000000000002,"15min":0.49099999999999999},"min":{"1min":0.41499999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76500000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Dec 7 10:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.39300000000000002,"15min":0.38400000000000001},"min":{"1min":0.27600000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.39700000000000002,"15min":0.38300000000000001},"min":{"1min":0.26200000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.47699999999999998}]},{"osd":2,"last update":"Thu Dec 7 10:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.57399999999999995,"15min":0.55500000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56299999999999994,"15min":0.53600000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.59599999999999997}]},{"osd":3,"last update":"Thu Dec 7 10:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.55900000000000005,"15min":0.53500000000000003},"min":{"1min":0.40899999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90700000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58199999999999996,"15min":0.56100000000000005},"min":{"1min":0.39900000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88900000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.78500000000000003}]},{"osd":5,"last update":"Thu Dec 7 10:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.627},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.59699999999999998,"15min":0.57699999999999996},"min":{"1min":0.40200000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Dec 7 10:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.41399999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64800000000000002,"15min":0.63600000000000001},"min":{"1min":0.501,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91900000000000004,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.85199999999999998}]},{"osd":7,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.52300000000000002,"15min":0.50900000000000001},"min":{"1min":0.35499999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.56599999999999995,"15min":0.54900000000000004},"min":{"1min":0.39000000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.50900000000000001}]},{"osd":8,"last update":"Thu Dec 7 10:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.68700000000000006,"15min":0.67100000000000004},"min":{"1min":0.437,"5min":0.432,"15min":0.432},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.69399999999999995,"15min":0.67600000000000005},"min":{"1min":0.52600000000000002,"5min":0.502,"15min":0.502},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.64600000000000002}]},{"osd":9,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.76700000000000002,"15min":0.76900000000000002},"min":{"1min":0.57399999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.98799999999999999,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.77100000000000002,"15min":0.75800000000000001},"min":{"1min":0.58699999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.88600000000000001}]},{"osd":10,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.745},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.87}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]}]},{"osd":5,"up_from":39,"seq":167503724590,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1646592,"data_stored":1089826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.373,"5min":0.318,"15min":0.318},"max":{"1min":0.748,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53600000000000003,"15min":0.51800000000000002},"min":{"1min":0.34399999999999997,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.73099999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.626}]},{"osd":1,"last update":"Thu Dec 7 10:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.52100000000000002,"15min":0.503},"min":{"1min":0.35499999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76600000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53400000000000003,"15min":0.52200000000000002},"min":{"1min":0.375,"5min":0.312,"15min":0.312},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.61099999999999999}]},{"osd":2,"last update":"Thu Dec 7 10:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.38500000000000001,"15min":0.373},"min":{"1min":0.27400000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.40200000000000002,"15min":0.38200000000000001},"min":{"1min":0.28999999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.70499999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.54700000000000004}]},{"osd":3,"last update":"Thu Dec 7 10:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55600000000000005,"15min":0.53100000000000003},"min":{"1min":0.38200000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82599999999999996,"5min":0.88,"15min":0.88},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56499999999999995,"15min":0.54700000000000004},"min":{"1min":0.44600000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57499999999999996}]},{"osd":4,"last update":"Thu Dec 7 10:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57599999999999996,"15min":0.55400000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83399999999999996,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.67000000000000004}]},{"osd":6,"last update":"Thu Dec 7 10:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.61799999999999999,"15min":0.58599999999999997},"min":{"1min":0.42999999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.69499999999999995,"15min":0.63200000000000001},"min":{"1min":0.45000000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":5.9720000000000004,"5min":5.9720000000000004,"15min":5.9720000000000004},"last":0.64100000000000001}]},{"osd":7,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.747},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.78500000000000003}]},{"osd":8,"last update":"Thu Dec 7 10:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.624,"15min":0.61899999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.80900000000000005}]},{"osd":9,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.78700000000000003,"15min":0.755},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":4.532,"5min":4.532,"15min":4.532},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76400000000000001,"15min":0.76300000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.89300000000000002}]},{"osd":10,"last update":"Thu Dec 7 10:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":2.927,"5min":2.927,"15min":2.927},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.86099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":45420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-07T10:18:15.600 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-07T10:18:15.601 INFO:tasks.cephadm:Setting up client nodes... 2023-12-07T10:18:15.601 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-12-07T10:18:15.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:15 smithi119 ceph-mon[97908]: pgmap v298: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 3.9 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2023-12-07T10:18:15.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:15 smithi119 ceph-mon[97908]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:18:16.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:15 smithi159 ceph-mon[134726]: pgmap v298: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 3.9 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2023-12-07T10:18:16.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:15 smithi159 ceph-mon[134726]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:18:16.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:15 smithi153 ceph-mon[133625]: pgmap v298: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 3.9 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2023-12-07T10:18:16.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:15 smithi153 ceph-mon[133625]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:18:16.410 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:16.956 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:16 smithi119 ceph-mon[97908]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:18:17.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:16 smithi153 ceph-mon[133625]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:18:17.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:16 smithi159 ceph-mon[134726]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:18:17.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.926+0000 7f942742c700 1 -- 172.21.15.119:0/3532665828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9420101620 msgr2=0x7f9420101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:17.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.926+0000 7f942742c700 1 --2- 172.21.15.119:0/3532665828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9420101620 0x7f9420101a40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f941c0099c0 tx=0x7f941c009cd0 comp rx=0 tx=0).stop 2023-12-07T10:18:17.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.926+0000 7f942742c700 1 -- 172.21.15.119:0/3532665828 shutdown_connections 2023-12-07T10:18:17.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.926+0000 7f942742c700 1 --2- 172.21.15.119:0/3532665828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94201031c0 0x7f9420109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:17.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.926+0000 7f942742c700 1 --2- 172.21.15.119:0/3532665828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9420101620 0x7f9420101a40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:17.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.926+0000 7f942742c700 1 --2- 172.21.15.119:0/3532665828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9420102820 0x7f9420102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:17.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.926+0000 7f942742c700 1 -- 172.21.15.119:0/3532665828 >> 172.21.15.119:0/3532665828 conn(0x7f94200fcbc0 msgr2=0x7f94200ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:17.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.926+0000 7f942742c700 1 -- 172.21.15.119:0/3532665828 shutdown_connections 2023-12-07T10:18:17.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.927+0000 7f942742c700 1 -- 172.21.15.119:0/3532665828 wait complete. 2023-12-07T10:18:17.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.927+0000 7f942742c700 1 Processor -- start 2023-12-07T10:18:17.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.928+0000 7f942742c700 1 -- start start 2023-12-07T10:18:17.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.928+0000 7f942742c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9420101620 0x7f9420075360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:17.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.928+0000 7f942742c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9420102820 0x7f9420073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:17.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.928+0000 7f942742c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94201031c0 0x7f9420073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:17.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.928+0000 7f942742c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94200746e0 con 0x7f94201031c0 2023-12-07T10:18:17.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.928+0000 7f942742c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9420074850 con 0x7f9420101620 2023-12-07T10:18:17.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.928+0000 7f942742c700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94201a1560 con 0x7f9420102820 2023-12-07T10:18:17.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.928+0000 7f94251c8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9420101620 0x7f9420075360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:17.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.928+0000 7f94259c9700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94201031c0 0x7f9420073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:17.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.928+0000 7f94251c8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9420101620 0x7f9420075360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:34842/0 (socket says 172.21.15.119:34842) 2023-12-07T10:18:17.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.928+0000 7f94251c8700 1 -- 172.21.15.119:0/331920862 learned_addr learned my addr 172.21.15.119:0/331920862 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:17.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.928+0000 7f94249c7700 1 --2- 172.21.15.119:0/331920862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9420102820 0x7f9420073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:17.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.929+0000 7f94259c9700 1 -- 172.21.15.119:0/331920862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9420102820 msgr2=0x7f9420073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:17.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.929+0000 7f94259c9700 1 --2- 172.21.15.119:0/331920862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9420102820 0x7f9420073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:17.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.929+0000 7f94259c9700 1 -- 172.21.15.119:0/331920862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9420101620 msgr2=0x7f9420075360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:17.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.929+0000 7f94259c9700 1 --2- 172.21.15.119:0/331920862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9420101620 0x7f9420075360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:17.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.929+0000 7f94259c9700 1 -- 172.21.15.119:0/331920862 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c0095f0 con 0x7f94201031c0 2023-12-07T10:18:17.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.929+0000 7f94259c9700 1 --2- 172.21.15.119:0/331920862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94201031c0 0x7f9420073f40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9418007f10 tx=0x7f941800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:17.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.929+0000 7f94167fc700 1 -- 172.21.15.119:0/331920862 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9418004db0 con 0x7f94201031c0 2023-12-07T10:18:17.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.929+0000 7f94167fc700 1 -- 172.21.15.119:0/331920862 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f941800f040 con 0x7f94201031c0 2023-12-07T10:18:17.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.929+0000 7f94167fc700 1 -- 172.21.15.119:0/331920862 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9418005990 con 0x7f94201031c0 2023-12-07T10:18:17.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.929+0000 7f942742c700 1 -- 172.21.15.119:0/331920862 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94201a17f0 con 0x7f94201031c0 2023-12-07T10:18:17.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.929+0000 7f942742c700 1 -- 172.21.15.119:0/331920862 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94201a1ca0 con 0x7f94201031c0 2023-12-07T10:18:17.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.930+0000 7f942742c700 1 -- 172.21.15.119:0/331920862 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9420108a50 con 0x7f94201031c0 2023-12-07T10:18:17.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.932+0000 7f94167fc700 1 -- 172.21.15.119:0/331920862 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f941800db20 con 0x7f94201031c0 2023-12-07T10:18:17.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.934+0000 7f94167fc700 1 --2- 172.21.15.119:0/331920862 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f940c0a1ba0 0x7f940c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:17.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.934+0000 7f94167fc700 1 -- 172.21.15.119:0/331920862 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f94180cd310 con 0x7f94201031c0 2023-12-07T10:18:17.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.934+0000 7f94251c8700 1 --2- 172.21.15.119:0/331920862 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f940c0a1ba0 0x7f940c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:17.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.934+0000 7f94251c8700 1 --2- 172.21.15.119:0/331920862 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f940c0a1ba0 0x7f940c0a4060 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f941c003820 tx=0x7f941c005bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:17.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:17.938+0000 7f94167fc700 1 -- 172.21.15.119:0/331920862 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f941804f2f0 con 0x7f94201031c0 2023-12-07T10:18:17.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:17 smithi119 ceph-mon[97908]: pgmap v299: 129 pgs: 129 active+clean; 625 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2023-12-07T10:18:18.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:17 smithi153 ceph-mon[133625]: pgmap v299: 129 pgs: 129 active+clean; 625 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2023-12-07T10:18:18.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:18.217+0000 7f942742c700 1 -- 172.21.15.119:0/331920862 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f942004f2d0 con 0x7f94201031c0 2023-12-07T10:18:18.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:18.220+0000 7f94167fc700 1 -- 172.21.15.119:0/331920862 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f9418096270 con 0x7f94201031c0 2023-12-07T10:18:18.222 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2023-12-07T10:18:18.222 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDqm3FlZLoQDRAAThIHuoqF6WrLIK81dC+wRQ== 2023-12-07T10:18:18.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:18.222+0000 7f942742c700 1 -- 172.21.15.119:0/331920862 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f940c0a1ba0 msgr2=0x7f940c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:18.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:18.222+0000 7f942742c700 1 --2- 172.21.15.119:0/331920862 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f940c0a1ba0 0x7f940c0a4060 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f941c003820 tx=0x7f941c005bc0 comp rx=0 tx=0).stop 2023-12-07T10:18:18.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:18.222+0000 7f942742c700 1 -- 172.21.15.119:0/331920862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94201031c0 msgr2=0x7f9420073f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:18.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:18.222+0000 7f942742c700 1 --2- 172.21.15.119:0/331920862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94201031c0 0x7f9420073f40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9418007f10 tx=0x7f941800d3b0 comp rx=0 tx=0).stop 2023-12-07T10:18:18.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:18.223+0000 7f942742c700 1 -- 172.21.15.119:0/331920862 shutdown_connections 2023-12-07T10:18:18.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:18.223+0000 7f942742c700 1 --2- 172.21.15.119:0/331920862 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f940c0a1ba0 0x7f940c0a4060 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:18.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:18.223+0000 7f942742c700 1 --2- 172.21.15.119:0/331920862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94201031c0 0x7f9420073f40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:18.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:18.223+0000 7f942742c700 1 --2- 172.21.15.119:0/331920862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9420101620 0x7f9420075360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:18.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:18.223+0000 7f942742c700 1 --2- 172.21.15.119:0/331920862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9420102820 0x7f9420073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:18.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:18.223+0000 7f942742c700 1 -- 172.21.15.119:0/331920862 >> 172.21.15.119:0/331920862 conn(0x7f94200fcbc0 msgr2=0x7f9420105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:18.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:18.223+0000 7f942742c700 1 -- 172.21.15.119:0/331920862 shutdown_connections 2023-12-07T10:18:18.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:18.223+0000 7f942742c700 1 -- 172.21.15.119:0/331920862 wait complete. 2023-12-07T10:18:18.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:17 smithi159 ceph-mon[134726]: pgmap v299: 129 pgs: 129 active+clean; 625 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2023-12-07T10:18:18.713 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-12-07T10:18:18.714 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-12-07T10:18:18.714 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-12-07T10:18:18.815 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-12-07T10:18:18.815 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-12-07T10:18:18.816 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph mgr dump --format=json 2023-12-07T10:18:18.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:18 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/331920862' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-07T10:18:18.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:18 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/331920862' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-07T10:18:19.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:18 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/331920862' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-07T10:18:19.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:18 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/331920862' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-07T10:18:19.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:18 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/331920862' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-07T10:18:19.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:18 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/331920862' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-07T10:18:19.625 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:20.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:19 smithi153 ceph-mon[133625]: pgmap v300: 129 pgs: 129 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.9 KiB/s wr, 20 op/s 2023-12-07T10:18:20.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:19 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:18:20.071 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:19 smithi119 ceph-mon[97908]: pgmap v300: 129 pgs: 129 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.9 KiB/s wr, 20 op/s 2023-12-07T10:18:20.071 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:19 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:18:20.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:19 smithi159 ceph-mon[134726]: pgmap v300: 129 pgs: 129 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.9 KiB/s wr, 20 op/s 2023-12-07T10:18:20.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:19 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:18:21.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.104+0000 7f02a9b86700 1 -- 172.21.15.119:0/202242208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02a4102080 msgr2=0x7f02a410e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:21.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.104+0000 7f02a9b86700 1 --2- 172.21.15.119:0/202242208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02a4102080 0x7f02a410e470 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f029c00b0a0 tx=0x7f029c00b3b0 comp rx=0 tx=0).stop 2023-12-07T10:18:21.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.104+0000 7f02a9b86700 1 -- 172.21.15.119:0/202242208 shutdown_connections 2023-12-07T10:18:21.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.104+0000 7f02a9b86700 1 --2- 172.21.15.119:0/202242208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02a4102080 0x7f02a410e470 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:21.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.104+0000 7f02a9b86700 1 --2- 172.21.15.119:0/202242208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02a4107730 0x7f02a4107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:21.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.104+0000 7f02a9b86700 1 --2- 172.21.15.119:0/202242208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02a41016c0 0x7f02a4101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:21.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.104+0000 7f02a9b86700 1 -- 172.21.15.119:0/202242208 >> 172.21.15.119:0/202242208 conn(0x7f02a4078800 msgr2=0x7f02a4078c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:21.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.104+0000 7f02a9b86700 1 -- 172.21.15.119:0/202242208 shutdown_connections 2023-12-07T10:18:21.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.104+0000 7f02a9b86700 1 -- 172.21.15.119:0/202242208 wait complete. 2023-12-07T10:18:21.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.105+0000 7f02a9b86700 1 Processor -- start 2023-12-07T10:18:21.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.106+0000 7f02a9b86700 1 -- start start 2023-12-07T10:18:21.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.106+0000 7f02a9b86700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02a41016c0 0x7f02a419f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:21.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.106+0000 7f02a9b86700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02a4102080 0x7f02a419fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:21.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.106+0000 7f02a9b86700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02a4107730 0x7f02a4199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:21.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.106+0000 7f02a9b86700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a419a140 con 0x7f02a41016c0 2023-12-07T10:18:21.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.106+0000 7f02a9b86700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a419a2b0 con 0x7f02a4102080 2023-12-07T10:18:21.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.106+0000 7f02a9b86700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a419a470 con 0x7f02a4107730 2023-12-07T10:18:21.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.106+0000 7f02a37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02a41016c0 0x7f02a419f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:21.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.106+0000 7f02a37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02a41016c0 0x7f02a419f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54102/0 (socket says 172.21.15.119:54102) 2023-12-07T10:18:21.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.106+0000 7f02a37fe700 1 -- 172.21.15.119:0/4255603085 learned_addr learned my addr 172.21.15.119:0/4255603085 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:21.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.106+0000 7f02a2ffd700 1 --2- 172.21.15.119:0/4255603085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02a4102080 0x7f02a419fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:21.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.106+0000 7f02a3fff700 1 --2- 172.21.15.119:0/4255603085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02a4107730 0x7f02a4199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:21.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.107+0000 7f02a37fe700 1 -- 172.21.15.119:0/4255603085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02a4107730 msgr2=0x7f02a4199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:21.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.107+0000 7f02a37fe700 1 --2- 172.21.15.119:0/4255603085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02a4107730 0x7f02a4199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:21.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.107+0000 7f02a37fe700 1 -- 172.21.15.119:0/4255603085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02a4102080 msgr2=0x7f02a419fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:21.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.107+0000 7f02a37fe700 1 --2- 172.21.15.119:0/4255603085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02a4102080 0x7f02a419fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:21.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.107+0000 7f02a37fe700 1 -- 172.21.15.119:0/4255603085 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029c009d00 con 0x7f02a41016c0 2023-12-07T10:18:21.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.107+0000 7f02a37fe700 1 --2- 172.21.15.119:0/4255603085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02a41016c0 0x7f02a419f9a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f029400d950 tx=0x7f029400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:21.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.107+0000 7f02a0ff9700 1 -- 172.21.15.119:0/4255603085 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029400f890 con 0x7f02a41016c0 2023-12-07T10:18:21.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.107+0000 7f02a0ff9700 1 -- 172.21.15.119:0/4255603085 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0294010460 con 0x7f02a41016c0 2023-12-07T10:18:21.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.107+0000 7f02a9b86700 1 -- 172.21.15.119:0/4255603085 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a419a750 con 0x7f02a41016c0 2023-12-07T10:18:21.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.107+0000 7f02a0ff9700 1 -- 172.21.15.119:0/4255603085 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029400e610 con 0x7f02a41016c0 2023-12-07T10:18:21.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.107+0000 7f02a9b86700 1 -- 172.21.15.119:0/4255603085 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a41a6990 con 0x7f02a41016c0 2023-12-07T10:18:21.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.108+0000 7f02a9b86700 1 -- 172.21.15.119:0/4255603085 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02a4102d50 con 0x7f02a41016c0 2023-12-07T10:18:21.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.110+0000 7f02a0ff9700 1 -- 172.21.15.119:0/4255603085 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f02940108d0 con 0x7f02a41016c0 2023-12-07T10:18:21.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.112+0000 7f02a0ff9700 1 --2- 172.21.15.119:0/4255603085 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f028c0a1c20 0x7f028c0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:21.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.112+0000 7f02a0ff9700 1 -- 172.21.15.119:0/4255603085 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f02940cd740 con 0x7f02a41016c0 2023-12-07T10:18:21.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.112+0000 7f02a2ffd700 1 --2- 172.21.15.119:0/4255603085 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f028c0a1c20 0x7f028c0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:21.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.112+0000 7f02a2ffd700 1 --2- 172.21.15.119:0/4255603085 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f028c0a1c20 0x7f028c0a40e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f02a419aca0 tx=0x7f02980073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:21.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.116+0000 7f02a0ff9700 1 -- 172.21.15.119:0/4255603085 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f02940966a0 con 0x7f02a41016c0 2023-12-07T10:18:21.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.370+0000 7f02a9b86700 1 -- 172.21.15.119:0/4255603085 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f02a404f2d0 con 0x7f02a41016c0 2023-12-07T10:18:21.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.376+0000 7f02a0ff9700 1 -- 172.21.15.119:0/4255603085 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+261226 (secure 0 0 0) 0x7f0294099cc0 con 0x7f02a41016c0 2023-12-07T10:18:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:18:21.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.380+0000 7f02a9b86700 1 -- 172.21.15.119:0/4255603085 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f028c0a1c20 msgr2=0x7f028c0a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:21.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.380+0000 7f02a9b86700 1 --2- 172.21.15.119:0/4255603085 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f028c0a1c20 0x7f028c0a40e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f02a419aca0 tx=0x7f02980073d0 comp rx=0 tx=0).stop 2023-12-07T10:18:21.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.380+0000 7f02a9b86700 1 -- 172.21.15.119:0/4255603085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02a41016c0 msgr2=0x7f02a419f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:21.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.380+0000 7f02a9b86700 1 --2- 172.21.15.119:0/4255603085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02a41016c0 0x7f02a419f9a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f029400d950 tx=0x7f029400dc60 comp rx=0 tx=0).stop 2023-12-07T10:18:21.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.380+0000 7f02a9b86700 1 -- 172.21.15.119:0/4255603085 shutdown_connections 2023-12-07T10:18:21.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.380+0000 7f02a9b86700 1 --2- 172.21.15.119:0/4255603085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02a4107730 0x7f02a4199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:21.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.380+0000 7f02a9b86700 1 --2- 172.21.15.119:0/4255603085 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f028c0a1c20 0x7f028c0a40e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:21.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.380+0000 7f02a9b86700 1 --2- 172.21.15.119:0/4255603085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02a41016c0 0x7f02a419f9a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:21.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.380+0000 7f02a9b86700 1 --2- 172.21.15.119:0/4255603085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02a4102080 0x7f02a419fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:21.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.380+0000 7f02a9b86700 1 -- 172.21.15.119:0/4255603085 >> 172.21.15.119:0/4255603085 conn(0x7f02a4078800 msgr2=0x7f02a410cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:21.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.381+0000 7f02a9b86700 1 -- 172.21.15.119:0/4255603085 shutdown_connections 2023-12-07T10:18:21.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:21.381+0000 7f02a9b86700 1 -- 172.21.15.119:0/4255603085 wait complete. 2023-12-07T10:18:21.927 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":17,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":1344846367},{"type":"v1","addr":"172.21.15.119:6801","nonce":1344846367}]},"active_addr":"172.21.15.119:6801/1344846367","active_change":"2023-12-07T10:11:00.585683+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.119:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":2580180729}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":2490678270}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":149289687}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":2050367825}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":3359396837}]}]} 2023-12-07T10:18:21.932 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-12-07T10:18:21.932 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-12-07T10:18:21.933 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph osd dump --format=json 2023-12-07T10:18:22.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:21 smithi153 ceph-mon[133625]: pgmap v301: 129 pgs: 129 active+clean; 645 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 24 op/s 2023-12-07T10:18:22.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:21 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/4255603085' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-07T10:18:22.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:21 smithi119 ceph-mon[97908]: pgmap v301: 129 pgs: 129 active+clean; 645 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 24 op/s 2023-12-07T10:18:22.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:21 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/4255603085' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-07T10:18:22.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:21 smithi159 ceph-mon[134726]: pgmap v301: 129 pgs: 129 active+clean; 645 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 24 op/s 2023-12-07T10:18:22.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:21 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/4255603085' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-07T10:18:22.703 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:23.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:23 smithi119 ceph-mon[97908]: pgmap v302: 129 pgs: 129 active+clean; 645 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 24 op/s 2023-12-07T10:18:24.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:23 smithi153 ceph-mon[133625]: pgmap v302: 129 pgs: 129 active+clean; 645 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 24 op/s 2023-12-07T10:18:24.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.238+0000 7fd931409700 1 -- 172.21.15.119:0/1272612908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd92c102080 msgr2=0x7fd92c10e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:24.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.238+0000 7fd931409700 1 --2- 172.21.15.119:0/1272612908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd92c102080 0x7fd92c10e470 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd924009f90 tx=0x7fd92400b330 comp rx=0 tx=0).stop 2023-12-07T10:18:24.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.238+0000 7fd931409700 1 -- 172.21.15.119:0/1272612908 shutdown_connections 2023-12-07T10:18:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.238+0000 7fd931409700 1 --2- 172.21.15.119:0/1272612908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd92c102080 0x7fd92c10e470 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.238+0000 7fd931409700 1 --2- 172.21.15.119:0/1272612908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd92c107730 0x7fd92c107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.238+0000 7fd931409700 1 --2- 172.21.15.119:0/1272612908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd92c1016c0 0x7fd92c101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.238+0000 7fd931409700 1 -- 172.21.15.119:0/1272612908 >> 172.21.15.119:0/1272612908 conn(0x7fd92c078800 msgr2=0x7fd92c078c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.238+0000 7fd931409700 1 -- 172.21.15.119:0/1272612908 shutdown_connections 2023-12-07T10:18:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.238+0000 7fd931409700 1 -- 172.21.15.119:0/1272612908 wait complete. 2023-12-07T10:18:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.240+0000 7fd931409700 1 Processor -- start 2023-12-07T10:18:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.240+0000 7fd931409700 1 -- start start 2023-12-07T10:18:24.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.240+0000 7fd931409700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd92c1016c0 0x7fd92c19d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:24.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.240+0000 7fd931409700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd92c102080 0x7fd92c19dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.240+0000 7fd931409700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd92c107730 0x7fd92c197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.241+0000 7fd931409700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c197f70 con 0x7fd92c102080 2023-12-07T10:18:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.241+0000 7fd931409700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c1980e0 con 0x7fd92c1016c0 2023-12-07T10:18:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.241+0000 7fd931409700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c1982a0 con 0x7fd92c107730 2023-12-07T10:18:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.241+0000 7fd92a7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd92c102080 0x7fd92c19dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.241+0000 7fd92a7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd92c102080 0x7fd92c19dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54936/0 (socket says 172.21.15.119:54936) 2023-12-07T10:18:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.241+0000 7fd92a7fc700 1 -- 172.21.15.119:0/201055241 learned_addr learned my addr 172.21.15.119:0/201055241 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.241+0000 7fd92affd700 1 --2- 172.21.15.119:0/201055241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd92c1016c0 0x7fd92c19d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.241+0000 7fd92b7fe700 1 --2- 172.21.15.119:0/201055241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd92c107730 0x7fd92c197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.241+0000 7fd92a7fc700 1 -- 172.21.15.119:0/201055241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd92c107730 msgr2=0x7fd92c197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.241+0000 7fd92a7fc700 1 --2- 172.21.15.119:0/201055241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd92c107730 0x7fd92c197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.241+0000 7fd92a7fc700 1 -- 172.21.15.119:0/201055241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd92c1016c0 msgr2=0x7fd92c19d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.241+0000 7fd92a7fc700 1 --2- 172.21.15.119:0/201055241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd92c1016c0 0x7fd92c19d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.241+0000 7fd92a7fc700 1 -- 172.21.15.119:0/201055241 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9200095f0 con 0x7fd92c102080 2023-12-07T10:18:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.241+0000 7fd92b7fe700 1 --2- 172.21.15.119:0/201055241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd92c107730 0x7fd92c197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:18:24.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.242+0000 7fd92a7fc700 1 --2- 172.21.15.119:0/201055241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd92c102080 0x7fd92c19dd00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd92000ec10 tx=0x7fd92000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:24.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.242+0000 7fd913fff700 1 -- 172.21.15.119:0/201055241 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92000cc90 con 0x7fd92c102080 2023-12-07T10:18:24.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.242+0000 7fd913fff700 1 -- 172.21.15.119:0/201055241 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9200044e0 con 0x7fd92c102080 2023-12-07T10:18:24.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.242+0000 7fd931409700 1 -- 172.21.15.119:0/201055241 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd924009c40 con 0x7fd92c102080 2023-12-07T10:18:24.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.242+0000 7fd913fff700 1 -- 172.21.15.119:0/201055241 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd920010430 con 0x7fd92c102080 2023-12-07T10:18:24.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.242+0000 7fd931409700 1 -- 172.21.15.119:0/201055241 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd92c1a45e0 con 0x7fd92c102080 2023-12-07T10:18:24.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.243+0000 7fd931409700 1 -- 172.21.15.119:0/201055241 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd92c109bc0 con 0x7fd92c102080 2023-12-07T10:18:24.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.245+0000 7fd913fff700 1 -- 172.21.15.119:0/201055241 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7fd920003fb0 con 0x7fd92c102080 2023-12-07T10:18:24.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.246+0000 7fd913fff700 1 --2- 172.21.15.119:0/201055241 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd9140a1ba0 0x7fd9140a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:24.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.246+0000 7fd913fff700 1 -- 172.21.15.119:0/201055241 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd920014070 con 0x7fd92c102080 2023-12-07T10:18:24.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.246+0000 7fd92affd700 1 --2- 172.21.15.119:0/201055241 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd9140a1ba0 0x7fd9140a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:24.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.247+0000 7fd92affd700 1 --2- 172.21.15.119:0/201055241 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd9140a1ba0 0x7fd9140a4060 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd91c009c60 tx=0x7fd91c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:24.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.250+0000 7fd913fff700 1 -- 172.21.15.119:0/201055241 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd920096f60 con 0x7fd92c102080 2023-12-07T10:18:24.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:23 smithi159 ceph-mon[134726]: pgmap v302: 129 pgs: 129 active+clean; 645 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 24 op/s 2023-12-07T10:18:24.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.450+0000 7fd931409700 1 -- 172.21.15.119:0/201055241 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd92c198dc0 con 0x7fd92c102080 2023-12-07T10:18:24.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.452+0000 7fd913fff700 1 -- 172.21.15.119:0/201055241 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24528 (secure 0 0 0) 0x7fd92009a580 con 0x7fd92c102080 2023-12-07T10:18:24.454 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:18:24.454 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":91,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","created":"2023-12-07T10:09:59.945863+0000","modified":"2023-12-07T10:18:01.843682+0000","last_up_change":"2023-12-07T10:16:50.929427+0000","last_in_change":"2023-12-07T10:16:33.558158+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T10:13:25.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-07T10:17:39.347813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-07T10:17:40.714501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-07T10:17:44.257760+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"56b5818e-e625-40f1-8838-3ddcf2d34ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6803","nonce":2141950133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6805","nonce":2141950133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6809","nonce":2141950133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6807","nonce":2141950133}]},"public_addr":"172.21.15.119:6803/2141950133","cluster_addr":"172.21.15.119:6805/2141950133","heartbeat_back_addr":"172.21.15.119:6809/2141950133","heartbeat_front_addr":"172.21.15.119:6807/2141950133","state":["exists","up"]},{"osd":1,"uuid":"36489a6a-5278-4802-9525-9871521722e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6801","nonce":3390875370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6803","nonce":3390875370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6807","nonce":3390875370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6805","nonce":3390875370}]},"public_addr":"172.21.15.153:6801/3390875370","cluster_addr":"172.21.15.153:6803/3390875370","heartbeat_back_addr":"172.21.15.153:6807/3390875370","heartbeat_front_addr":"172.21.15.153:6805/3390875370","state":["exists","up"]},{"osd":2,"uuid":"a937ad26-5e9b-4c8a-b65a-626a8feda6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6801","nonce":708101637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6803","nonce":708101637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6807","nonce":708101637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6805","nonce":708101637}]},"public_addr":"172.21.15.159:6801/708101637","cluster_addr":"172.21.15.159:6803/708101637","heartbeat_back_addr":"172.21.15.159:6807/708101637","heartbeat_front_addr":"172.21.15.159:6805/708101637","state":["exists","up"]},{"osd":3,"uuid":"26ef8300-aa07-48f6-b31d-8b0f2217ffcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6811","nonce":3228047257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6813","nonce":3228047257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6817","nonce":3228047257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6815","nonce":3228047257}]},"public_addr":"172.21.15.119:6811/3228047257","cluster_addr":"172.21.15.119:6813/3228047257","heartbeat_back_addr":"172.21.15.119:6817/3228047257","heartbeat_front_addr":"172.21.15.119:6815/3228047257","state":["exists","up"]},{"osd":4,"uuid":"000ee4b9-9896-4901-8cc3-c85fdfc8cbaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6809","nonce":4021998299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6811","nonce":4021998299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6815","nonce":4021998299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6813","nonce":4021998299}]},"public_addr":"172.21.15.153:6809/4021998299","cluster_addr":"172.21.15.153:6811/4021998299","heartbeat_back_addr":"172.21.15.153:6815/4021998299","heartbeat_front_addr":"172.21.15.153:6813/4021998299","state":["exists","up"]},{"osd":5,"uuid":"8b2fc167-262d-474f-89f3-47bd29424421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6809","nonce":1832540768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6811","nonce":1832540768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6815","nonce":1832540768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6813","nonce":1832540768}]},"public_addr":"172.21.15.159:6809/1832540768","cluster_addr":"172.21.15.159:6811/1832540768","heartbeat_back_addr":"172.21.15.159:6815/1832540768","heartbeat_front_addr":"172.21.15.159:6813/1832540768","state":["exists","up"]},{"osd":6,"uuid":"f9aa6749-46cf-4b2a-8114-cbd034219d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162436840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162436840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6825","nonce":1162436840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162436840}]},"public_addr":"172.21.15.119:6819/1162436840","cluster_addr":"172.21.15.119:6821/1162436840","heartbeat_back_addr":"172.21.15.119:6825/1162436840","heartbeat_front_addr":"172.21.15.119:6823/1162436840","state":["exists","up"]},{"osd":7,"uuid":"39216361-e1c4-4fef-86cb-a4707656b547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6817","nonce":3254653457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6819","nonce":3254653457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6823","nonce":3254653457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6821","nonce":3254653457}]},"public_addr":"172.21.15.153:6817/3254653457","cluster_addr":"172.21.15.153:6819/3254653457","heartbeat_back_addr":"172.21.15.153:6823/3254653457","heartbeat_front_addr":"172.21.15.153:6821/3254653457","state":["exists","up"]},{"osd":8,"uuid":"cceb6ebc-7c02-48ea-9d16-b7a016dd449b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6817","nonce":2600946380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6819","nonce":2600946380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6823","nonce":2600946380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6821","nonce":2600946380}]},"public_addr":"172.21.15.159:6817/2600946380","cluster_addr":"172.21.15.159:6819/2600946380","heartbeat_back_addr":"172.21.15.159:6823/2600946380","heartbeat_front_addr":"172.21.15.159:6821/2600946380","state":["exists","up"]},{"osd":9,"uuid":"f337a2e4-e065-4a9f-91d9-e3615de2d6f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6827","nonce":1940737285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6829","nonce":1940737285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6833","nonce":1940737285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6831","nonce":1940737285}]},"public_addr":"172.21.15.119:6827/1940737285","cluster_addr":"172.21.15.119:6829/1940737285","heartbeat_back_addr":"172.21.15.119:6833/1940737285","heartbeat_front_addr":"172.21.15.119:6831/1940737285","state":["exists","up"]},{"osd":10,"uuid":"3a3bc347-85f4-4878-9797-d9da96137e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6825","nonce":663834864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6827","nonce":663834864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6831","nonce":663834864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6829","nonce":663834864}]},"public_addr":"172.21.15.153:6825/663834864","cluster_addr":"172.21.15.153:6827/663834864","heartbeat_back_addr":"172.21.15.153:6831/663834864","heartbeat_front_addr":"172.21.15.153:6829/663834864","state":["exists","up"]},{"osd":11,"uuid":"a0fc5909-279d-4172-977c-2dc2da0a7e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6825","nonce":1262216348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6827","nonce":1262216348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6831","nonce":1262216348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6829","nonce":1262216348}]},"public_addr":"172.21.15.159:6825/1262216348","cluster_addr":"172.21.15.159:6827/1262216348","heartbeat_back_addr":"172.21.15.159:6831/1262216348","heartbeat_front_addr":"172.21.15.159:6829/1262216348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:37.488109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:59.543060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:21.381585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:47.112729+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:09.732150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:30.893729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:56.678710+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:19.004256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:40.388818+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:06.391734+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:28.394053+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:49.634348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1628639392":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6801/1355617847":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6800/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/362307836":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/128964558":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/2278813554":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/1411933614":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/1685698277":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6800/1355617847":"2023-12-08T10:11:00.585565+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T10:18:24.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.454+0000 7fd931409700 1 -- 172.21.15.119:0/201055241 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd9140a1ba0 msgr2=0x7fd9140a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:24.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.454+0000 7fd931409700 1 --2- 172.21.15.119:0/201055241 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd9140a1ba0 0x7fd9140a4060 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd91c009c60 tx=0x7fd91c009260 comp rx=0 tx=0).stop 2023-12-07T10:18:24.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.454+0000 7fd931409700 1 -- 172.21.15.119:0/201055241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd92c102080 msgr2=0x7fd92c19dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:24.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.454+0000 7fd931409700 1 --2- 172.21.15.119:0/201055241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd92c102080 0x7fd92c19dd00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd92000ec10 tx=0x7fd92000ef20 comp rx=0 tx=0).stop 2023-12-07T10:18:24.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.454+0000 7fd931409700 1 -- 172.21.15.119:0/201055241 shutdown_connections 2023-12-07T10:18:24.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.454+0000 7fd931409700 1 --2- 172.21.15.119:0/201055241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd92c107730 0x7fd92c197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:24.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.454+0000 7fd931409700 1 --2- 172.21.15.119:0/201055241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd92c1016c0 0x7fd92c19d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:24.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.454+0000 7fd931409700 1 --2- 172.21.15.119:0/201055241 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd9140a1ba0 0x7fd9140a4060 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:24.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.454+0000 7fd931409700 1 --2- 172.21.15.119:0/201055241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd92c102080 0x7fd92c19dd00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:24.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.454+0000 7fd931409700 1 -- 172.21.15.119:0/201055241 >> 172.21.15.119:0/201055241 conn(0x7fd92c078800 msgr2=0x7fd92c1002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:24.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.454+0000 7fd931409700 1 -- 172.21.15.119:0/201055241 shutdown_connections 2023-12-07T10:18:24.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:24.454+0000 7fd931409700 1 -- 172.21.15.119:0/201055241 wait complete. 2023-12-07T10:18:24.907 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-12-07T10:18:24.908 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph osd dump --format=json 2023-12-07T10:18:25.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:24 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/201055241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:18:25.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:24 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/201055241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:18:25.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:24 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/201055241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:18:25.682 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:26.111 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:25 smithi119 ceph-mon[97908]: pgmap v303: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 KiB/s wr, 22 op/s 2023-12-07T10:18:26.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:25 smithi159 ceph-mon[134726]: pgmap v303: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 KiB/s wr, 22 op/s 2023-12-07T10:18:26.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:25 smithi153 ceph-mon[133625]: pgmap v303: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 KiB/s wr, 22 op/s 2023-12-07T10:18:27.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.227+0000 7fb6a15c8700 1 -- 172.21.15.119:0/3020381863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb69c073840 msgr2=0x7fb69c073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:27.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.227+0000 7fb6a15c8700 1 --2- 172.21.15.119:0/3020381863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb69c073840 0x7fb69c073c20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb68c009990 tx=0x7fb68c009ca0 comp rx=0 tx=0).stop 2023-12-07T10:18:27.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.227+0000 7fb6a15c8700 1 -- 172.21.15.119:0/3020381863 shutdown_connections 2023-12-07T10:18:27.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.227+0000 7fb6a15c8700 1 --2- 172.21.15.119:0/3020381863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb69c10e070 0x7fb69c110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:27.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.227+0000 7fb6a15c8700 1 --2- 172.21.15.119:0/3020381863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb69c073840 0x7fb69c073c20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:27.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.227+0000 7fb6a15c8700 1 --2- 172.21.15.119:0/3020381863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb69c074160 0x7fb69c10db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:27.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.227+0000 7fb6a15c8700 1 -- 172.21.15.119:0/3020381863 >> 172.21.15.119:0/3020381863 conn(0x7fb69c078800 msgr2=0x7fb69c078c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:27.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.227+0000 7fb6a15c8700 1 -- 172.21.15.119:0/3020381863 shutdown_connections 2023-12-07T10:18:27.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.227+0000 7fb6a15c8700 1 -- 172.21.15.119:0/3020381863 wait complete. 2023-12-07T10:18:27.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.228+0000 7fb6a15c8700 1 Processor -- start 2023-12-07T10:18:27.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.229+0000 7fb6a15c8700 1 -- start start 2023-12-07T10:18:27.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.229+0000 7fb6a15c8700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb69c073840 0x7fb69c19f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:27.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.229+0000 7fb6a15c8700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb69c074160 0x7fb69c19fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:27.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.229+0000 7fb6a15c8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb69c10e070 0x7fb69c199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:27.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.229+0000 7fb6a15c8700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c19a180 con 0x7fb69c073840 2023-12-07T10:18:27.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.229+0000 7fb6a15c8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c19a2c0 con 0x7fb69c10e070 2023-12-07T10:18:27.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.229+0000 7fb6a15c8700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c19a480 con 0x7fb69c074160 2023-12-07T10:18:27.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.229+0000 7fb69affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb69c073840 0x7fb69c19f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:27.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.229+0000 7fb69affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb69c073840 0x7fb69c19f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54954/0 (socket says 172.21.15.119:54954) 2023-12-07T10:18:27.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.229+0000 7fb69affd700 1 -- 172.21.15.119:0/2551417782 learned_addr learned my addr 172.21.15.119:0/2551417782 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:27.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.229+0000 7fb69b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb69c10e070 0x7fb69c199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:27.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.229+0000 7fb69a7fc700 1 --2- 172.21.15.119:0/2551417782 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb69c074160 0x7fb69c19fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:27.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.230+0000 7fb69affd700 1 -- 172.21.15.119:0/2551417782 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb69c074160 msgr2=0x7fb69c19fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:27.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.230+0000 7fb69affd700 1 --2- 172.21.15.119:0/2551417782 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb69c074160 0x7fb69c19fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:27.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.230+0000 7fb69affd700 1 -- 172.21.15.119:0/2551417782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb69c10e070 msgr2=0x7fb69c199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:27.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.230+0000 7fb69affd700 1 --2- 172.21.15.119:0/2551417782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb69c10e070 0x7fb69c199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:27.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.230+0000 7fb69affd700 1 -- 172.21.15.119:0/2551417782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb68c0095f0 con 0x7fb69c073840 2023-12-07T10:18:27.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.230+0000 7fb69b7fe700 1 --2- 172.21.15.119:0/2551417782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb69c10e070 0x7fb69c199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:18:27.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.230+0000 7fb69affd700 1 --2- 172.21.15.119:0/2551417782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb69c073840 0x7fb69c19f910 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb68c000c00 tx=0x7fb68c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:27.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.230+0000 7fb683fff700 1 -- 172.21.15.119:0/2551417782 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c01d070 con 0x7fb69c073840 2023-12-07T10:18:27.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.230+0000 7fb683fff700 1 -- 172.21.15.119:0/2551417782 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb68c022470 con 0x7fb69c073840 2023-12-07T10:18:27.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.230+0000 7fb6a15c8700 1 -- 172.21.15.119:0/2551417782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb69c19a700 con 0x7fb69c073840 2023-12-07T10:18:27.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.230+0000 7fb683fff700 1 -- 172.21.15.119:0/2551417782 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c017650 con 0x7fb69c073840 2023-12-07T10:18:27.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.230+0000 7fb6a15c8700 1 -- 172.21.15.119:0/2551417782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb69c1a69b0 con 0x7fb69c073840 2023-12-07T10:18:27.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.232+0000 7fb6a15c8700 1 -- 172.21.15.119:0/2551417782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb69c109280 con 0x7fb69c073840 2023-12-07T10:18:27.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.233+0000 7fb683fff700 1 -- 172.21.15.119:0/2551417782 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7fb68c0225e0 con 0x7fb69c073840 2023-12-07T10:18:27.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.235+0000 7fb683fff700 1 --2- 172.21.15.119:0/2551417782 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fb6840a1c20 0x7fb6840a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:27.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.235+0000 7fb683fff700 1 -- 172.21.15.119:0/2551417782 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fb68c0ced50 con 0x7fb69c073840 2023-12-07T10:18:27.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.235+0000 7fb69a7fc700 1 --2- 172.21.15.119:0/2551417782 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fb6840a1c20 0x7fb6840a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:27.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.236+0000 7fb69a7fc700 1 --2- 172.21.15.119:0/2551417782 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fb6840a1c20 0x7fb6840a40e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb69c10fd30 tx=0x7fb69000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:27.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.239+0000 7fb683fff700 1 -- 172.21.15.119:0/2551417782 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fb68c097d40 con 0x7fb69c073840 2023-12-07T10:18:27.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.437+0000 7fb6a15c8700 1 -- 172.21.15.119:0/2551417782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb69c074ed0 con 0x7fb69c073840 2023-12-07T10:18:27.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.439+0000 7fb683fff700 1 -- 172.21.15.119:0/2551417782 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24528 (secure 0 0 0) 0x7fb69c074ed0 con 0x7fb69c073840 2023-12-07T10:18:27.441 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:18:27.441 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":91,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","created":"2023-12-07T10:09:59.945863+0000","modified":"2023-12-07T10:18:01.843682+0000","last_up_change":"2023-12-07T10:16:50.929427+0000","last_in_change":"2023-12-07T10:16:33.558158+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T10:13:25.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-07T10:17:39.347813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-07T10:17:40.714501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-07T10:17:44.257760+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"56b5818e-e625-40f1-8838-3ddcf2d34ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6803","nonce":2141950133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6805","nonce":2141950133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6809","nonce":2141950133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6807","nonce":2141950133}]},"public_addr":"172.21.15.119:6803/2141950133","cluster_addr":"172.21.15.119:6805/2141950133","heartbeat_back_addr":"172.21.15.119:6809/2141950133","heartbeat_front_addr":"172.21.15.119:6807/2141950133","state":["exists","up"]},{"osd":1,"uuid":"36489a6a-5278-4802-9525-9871521722e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6801","nonce":3390875370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6803","nonce":3390875370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6807","nonce":3390875370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6805","nonce":3390875370}]},"public_addr":"172.21.15.153:6801/3390875370","cluster_addr":"172.21.15.153:6803/3390875370","heartbeat_back_addr":"172.21.15.153:6807/3390875370","heartbeat_front_addr":"172.21.15.153:6805/3390875370","state":["exists","up"]},{"osd":2,"uuid":"a937ad26-5e9b-4c8a-b65a-626a8feda6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6801","nonce":708101637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6803","nonce":708101637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6807","nonce":708101637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6805","nonce":708101637}]},"public_addr":"172.21.15.159:6801/708101637","cluster_addr":"172.21.15.159:6803/708101637","heartbeat_back_addr":"172.21.15.159:6807/708101637","heartbeat_front_addr":"172.21.15.159:6805/708101637","state":["exists","up"]},{"osd":3,"uuid":"26ef8300-aa07-48f6-b31d-8b0f2217ffcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6811","nonce":3228047257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6813","nonce":3228047257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6817","nonce":3228047257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6815","nonce":3228047257}]},"public_addr":"172.21.15.119:6811/3228047257","cluster_addr":"172.21.15.119:6813/3228047257","heartbeat_back_addr":"172.21.15.119:6817/3228047257","heartbeat_front_addr":"172.21.15.119:6815/3228047257","state":["exists","up"]},{"osd":4,"uuid":"000ee4b9-9896-4901-8cc3-c85fdfc8cbaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6809","nonce":4021998299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6811","nonce":4021998299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6815","nonce":4021998299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6813","nonce":4021998299}]},"public_addr":"172.21.15.153:6809/4021998299","cluster_addr":"172.21.15.153:6811/4021998299","heartbeat_back_addr":"172.21.15.153:6815/4021998299","heartbeat_front_addr":"172.21.15.153:6813/4021998299","state":["exists","up"]},{"osd":5,"uuid":"8b2fc167-262d-474f-89f3-47bd29424421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6809","nonce":1832540768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6811","nonce":1832540768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6815","nonce":1832540768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6813","nonce":1832540768}]},"public_addr":"172.21.15.159:6809/1832540768","cluster_addr":"172.21.15.159:6811/1832540768","heartbeat_back_addr":"172.21.15.159:6815/1832540768","heartbeat_front_addr":"172.21.15.159:6813/1832540768","state":["exists","up"]},{"osd":6,"uuid":"f9aa6749-46cf-4b2a-8114-cbd034219d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162436840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162436840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6825","nonce":1162436840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162436840}]},"public_addr":"172.21.15.119:6819/1162436840","cluster_addr":"172.21.15.119:6821/1162436840","heartbeat_back_addr":"172.21.15.119:6825/1162436840","heartbeat_front_addr":"172.21.15.119:6823/1162436840","state":["exists","up"]},{"osd":7,"uuid":"39216361-e1c4-4fef-86cb-a4707656b547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6817","nonce":3254653457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6819","nonce":3254653457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6823","nonce":3254653457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6821","nonce":3254653457}]},"public_addr":"172.21.15.153:6817/3254653457","cluster_addr":"172.21.15.153:6819/3254653457","heartbeat_back_addr":"172.21.15.153:6823/3254653457","heartbeat_front_addr":"172.21.15.153:6821/3254653457","state":["exists","up"]},{"osd":8,"uuid":"cceb6ebc-7c02-48ea-9d16-b7a016dd449b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6817","nonce":2600946380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6819","nonce":2600946380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6823","nonce":2600946380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6821","nonce":2600946380}]},"public_addr":"172.21.15.159:6817/2600946380","cluster_addr":"172.21.15.159:6819/2600946380","heartbeat_back_addr":"172.21.15.159:6823/2600946380","heartbeat_front_addr":"172.21.15.159:6821/2600946380","state":["exists","up"]},{"osd":9,"uuid":"f337a2e4-e065-4a9f-91d9-e3615de2d6f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6827","nonce":1940737285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6829","nonce":1940737285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6833","nonce":1940737285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6831","nonce":1940737285}]},"public_addr":"172.21.15.119:6827/1940737285","cluster_addr":"172.21.15.119:6829/1940737285","heartbeat_back_addr":"172.21.15.119:6833/1940737285","heartbeat_front_addr":"172.21.15.119:6831/1940737285","state":["exists","up"]},{"osd":10,"uuid":"3a3bc347-85f4-4878-9797-d9da96137e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6825","nonce":663834864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6827","nonce":663834864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6831","nonce":663834864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6829","nonce":663834864}]},"public_addr":"172.21.15.153:6825/663834864","cluster_addr":"172.21.15.153:6827/663834864","heartbeat_back_addr":"172.21.15.153:6831/663834864","heartbeat_front_addr":"172.21.15.153:6829/663834864","state":["exists","up"]},{"osd":11,"uuid":"a0fc5909-279d-4172-977c-2dc2da0a7e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6825","nonce":1262216348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6827","nonce":1262216348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6831","nonce":1262216348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6829","nonce":1262216348}]},"public_addr":"172.21.15.159:6825/1262216348","cluster_addr":"172.21.15.159:6827/1262216348","heartbeat_back_addr":"172.21.15.159:6831/1262216348","heartbeat_front_addr":"172.21.15.159:6829/1262216348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:37.488109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:59.543060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:21.381585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:47.112729+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:09.732150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:30.893729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:56.678710+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:19.004256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:40.388818+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:06.391734+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:28.394053+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:49.634348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1628639392":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6801/1355617847":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6800/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/362307836":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/128964558":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/2278813554":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/1411933614":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/1685698277":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6800/1355617847":"2023-12-08T10:11:00.585565+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T10:18:27.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.441+0000 7fb6a15c8700 1 -- 172.21.15.119:0/2551417782 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fb6840a1c20 msgr2=0x7fb6840a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:27.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.441+0000 7fb6a15c8700 1 --2- 172.21.15.119:0/2551417782 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fb6840a1c20 0x7fb6840a40e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb69c10fd30 tx=0x7fb69000b410 comp rx=0 tx=0).stop 2023-12-07T10:18:27.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.441+0000 7fb6a15c8700 1 -- 172.21.15.119:0/2551417782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb69c073840 msgr2=0x7fb69c19f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:27.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.441+0000 7fb6a15c8700 1 --2- 172.21.15.119:0/2551417782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb69c073840 0x7fb69c19f910 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb68c000c00 tx=0x7fb68c005740 comp rx=0 tx=0).stop 2023-12-07T10:18:27.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.441+0000 7fb6a15c8700 1 -- 172.21.15.119:0/2551417782 shutdown_connections 2023-12-07T10:18:27.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.441+0000 7fb6a15c8700 1 --2- 172.21.15.119:0/2551417782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb69c10e070 0x7fb69c199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:27.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.441+0000 7fb6a15c8700 1 --2- 172.21.15.119:0/2551417782 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fb6840a1c20 0x7fb6840a40e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:27.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.441+0000 7fb6a15c8700 1 --2- 172.21.15.119:0/2551417782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb69c073840 0x7fb69c19f910 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:27.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.441+0000 7fb6a15c8700 1 --2- 172.21.15.119:0/2551417782 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb69c074160 0x7fb69c19fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:27.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.441+0000 7fb6a15c8700 1 -- 172.21.15.119:0/2551417782 >> 172.21.15.119:0/2551417782 conn(0x7fb69c078800 msgr2=0x7fb69c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:27.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.441+0000 7fb6a15c8700 1 -- 172.21.15.119:0/2551417782 shutdown_connections 2023-12-07T10:18:27.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:27.441+0000 7fb6a15c8700 1 -- 172.21.15.119:0/2551417782 wait complete. 2023-12-07T10:18:27.986 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2023-12-07T10:18:27.987 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2023-12-07T10:18:27.987 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2023-12-07T10:18:27.987 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2023-12-07T10:18:27.988 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2023-12-07T10:18:27.988 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2023-12-07T10:18:27.989 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2023-12-07T10:18:27.989 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2023-12-07T10:18:27.989 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2023-12-07T10:18:27.990 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2023-12-07T10:18:27.990 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2023-12-07T10:18:27.990 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2023-12-07T10:18:28.114 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:27 smithi119 ceph-mon[97908]: pgmap v304: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.3 KiB/s wr, 17 op/s 2023-12-07T10:18:28.114 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:27 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2551417782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:18:28.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:27 smithi159 ceph-mon[134726]: pgmap v304: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.3 KiB/s wr, 17 op/s 2023-12-07T10:18:28.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:27 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2551417782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:18:28.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:27 smithi153 ceph-mon[133625]: pgmap v304: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.3 KiB/s wr, 17 op/s 2023-12-07T10:18:28.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:27 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2551417782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:18:29.652 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:29.827 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:29.886 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:30.107 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:30.107 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:30.121 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:29 smithi119 ceph-mon[97908]: pgmap v305: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.6 KiB/s wr, 14 op/s 2023-12-07T10:18:30.150 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:30.168 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:30.172 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:30.183 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:30.205 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:30.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:29 smithi159 ceph-mon[134726]: pgmap v305: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.6 KiB/s wr, 14 op/s 2023-12-07T10:18:30.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:29 smithi153 ceph-mon[133625]: pgmap v305: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.6 KiB/s wr, 14 op/s 2023-12-07T10:18:31.268 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:31.269 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:32.190 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:31 smithi119 ceph-mon[97908]: pgmap v306: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.3 KiB/s wr, 13 op/s 2023-12-07T10:18:32.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:31 smithi159 ceph-mon[134726]: pgmap v306: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.3 KiB/s wr, 13 op/s 2023-12-07T10:18:32.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:31 smithi153 ceph-mon[133625]: pgmap v306: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.3 KiB/s wr, 13 op/s 2023-12-07T10:18:33.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:33 smithi119 ceph-mon[97908]: pgmap v307: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2023-12-07T10:18:34.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:33 smithi159 ceph-mon[134726]: pgmap v307: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2023-12-07T10:18:34.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:33 smithi153 ceph-mon[133625]: pgmap v307: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2023-12-07T10:18:35.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.360+0000 7f15ad231700 1 -- 172.21.15.119:0/62498001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a8102080 msgr2=0x7f15a810e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:35.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.360+0000 7f15ad231700 1 --2- 172.21.15.119:0/62498001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a8102080 0x7f15a810e470 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f15a000b0a0 tx=0x7f15a000b3b0 comp rx=0 tx=0).stop 2023-12-07T10:18:35.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.360+0000 7f15ad231700 1 -- 172.21.15.119:0/62498001 shutdown_connections 2023-12-07T10:18:35.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.360+0000 7f15ad231700 1 --2- 172.21.15.119:0/62498001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a8102080 0x7f15a810e470 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:35.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.360+0000 7f15ad231700 1 --2- 172.21.15.119:0/62498001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15a8107730 0x7f15a8107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:35.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.360+0000 7f15ad231700 1 --2- 172.21.15.119:0/62498001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a81016c0 0x7f15a8101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:35.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.360+0000 7f15ad231700 1 -- 172.21.15.119:0/62498001 >> 172.21.15.119:0/62498001 conn(0x7f15a8078800 msgr2=0x7f15a8078c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:35.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.361+0000 7f15ad231700 1 -- 172.21.15.119:0/62498001 shutdown_connections 2023-12-07T10:18:35.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.361+0000 7f15ad231700 1 -- 172.21.15.119:0/62498001 wait complete. 2023-12-07T10:18:35.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.362+0000 7f15ad231700 1 Processor -- start 2023-12-07T10:18:35.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.362+0000 7f15ad231700 1 -- start start 2023-12-07T10:18:35.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.363+0000 7f15ad231700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15a81016c0 0x7f15a819d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:35.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.363+0000 7f15ad231700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a8102080 0x7f15a819dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:35.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.363+0000 7f15ad231700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a8107730 0x7f15a81979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:35.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.363+0000 7f15ad231700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a8198050 con 0x7f15a8107730 2023-12-07T10:18:35.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.363+0000 7f15ad231700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a81981c0 con 0x7f15a8102080 2023-12-07T10:18:35.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.363+0000 7f15ad231700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a8198380 con 0x7f15a81016c0 2023-12-07T10:18:35.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.363+0000 7f15a759e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a8107730 0x7f15a81979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:35.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.363+0000 7f15a659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a8102080 0x7f15a819dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:35.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.363+0000 7f15a6d9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15a81016c0 0x7f15a819d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:35.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.363+0000 7f15a659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a8102080 0x7f15a819dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:35184/0 (socket says 172.21.15.119:35184) 2023-12-07T10:18:35.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.363+0000 7f15a759e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a8107730 0x7f15a81979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44050/0 (socket says 172.21.15.119:44050) 2023-12-07T10:18:35.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.363+0000 7f15a659c700 1 -- 172.21.15.119:0/1296384337 learned_addr learned my addr 172.21.15.119:0/1296384337 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:35.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.364+0000 7f15a759e700 1 -- 172.21.15.119:0/1296384337 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15a81016c0 msgr2=0x7f15a819d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:35.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.364+0000 7f15a759e700 1 --2- 172.21.15.119:0/1296384337 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15a81016c0 0x7f15a819d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:35.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.364+0000 7f15a759e700 1 -- 172.21.15.119:0/1296384337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a8102080 msgr2=0x7f15a819dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:35.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.364+0000 7f15a759e700 1 --2- 172.21.15.119:0/1296384337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a8102080 0x7f15a819dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:35.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.364+0000 7f15a759e700 1 -- 172.21.15.119:0/1296384337 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a0009d00 con 0x7f15a8107730 2023-12-07T10:18:35.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.364+0000 7f15a759e700 1 --2- 172.21.15.119:0/1296384337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a8107730 0x7f15a81979e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f15a000b070 tx=0x7f15a0007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:35.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.364+0000 7f158ffff700 1 -- 172.21.15.119:0/1296384337 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a000e040 con 0x7f15a8107730 2023-12-07T10:18:35.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.364+0000 7f158ffff700 1 -- 172.21.15.119:0/1296384337 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15a001d470 con 0x7f15a8107730 2023-12-07T10:18:35.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.364+0000 7f15ad231700 1 -- 172.21.15.119:0/1296384337 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a8198600 con 0x7f15a8107730 2023-12-07T10:18:35.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.365+0000 7f158ffff700 1 -- 172.21.15.119:0/1296384337 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a0005280 con 0x7f15a8107730 2023-12-07T10:18:35.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.365+0000 7f15ad231700 1 -- 172.21.15.119:0/1296384337 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a81a4840 con 0x7f15a8107730 2023-12-07T10:18:35.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.366+0000 7f15ad231700 1 -- 172.21.15.119:0/1296384337 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f15a8111820 con 0x7f15a8107730 2023-12-07T10:18:35.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.369+0000 7f158ffff700 1 -- 172.21.15.119:0/1296384337 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f15a001d5e0 con 0x7f15a8107730 2023-12-07T10:18:35.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.370+0000 7f158ffff700 1 --2- 172.21.15.119:0/1296384337 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f15900a1ba0 0x7f15900a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:35.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.371+0000 7f158ffff700 1 -- 172.21.15.119:0/1296384337 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f15a00cf090 con 0x7f15a8107730 2023-12-07T10:18:35.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.371+0000 7f15a6d9d700 1 --2- 172.21.15.119:0/1296384337 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f15900a1ba0 0x7f15900a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:35.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.371+0000 7f158ffff700 1 --2- 172.21.15.119:0/1296384337 >> [v2:172.21.15.119:6802/2141950133,v1:172.21.15.119:6803/2141950133] conn(0x7f15900acac0 0x7f15900aef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:35.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.371+0000 7f158ffff700 1 -- 172.21.15.119:0/1296384337 --> [v2:172.21.15.119:6802/2141950133,v1:172.21.15.119:6803/2141950133] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f15900af630 con 0x7f15900acac0 2023-12-07T10:18:35.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.371+0000 7f158ffff700 1 -- 172.21.15.119:0/1296384337 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15a00cf400 con 0x7f15a8107730 2023-12-07T10:18:35.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.371+0000 7f15a659c700 1 --2- 172.21.15.119:0/1296384337 >> [v2:172.21.15.119:6802/2141950133,v1:172.21.15.119:6803/2141950133] conn(0x7f15900acac0 0x7f15900aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:35.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.371+0000 7f15a6d9d700 1 --2- 172.21.15.119:0/1296384337 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f15900a1ba0 0x7f15900a4060 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1598009bd0 tx=0x7f1598009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:35.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.372+0000 7f15a659c700 1 --2- 172.21.15.119:0/1296384337 >> [v2:172.21.15.119:6802/2141950133,v1:172.21.15.119:6803/2141950133] conn(0x7f15900acac0 0x7f15900aef60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:35.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.373+0000 7f158ffff700 1 -- 172.21.15.119:0/1296384337 <== osd.0 v2:172.21.15.119:6802/2141950133 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f15900af630 con 0x7f15900acac0 2023-12-07T10:18:35.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.390+0000 7f15ad231700 1 -- 172.21.15.119:0/1296384337 --> [v2:172.21.15.119:6802/2141950133,v1:172.21.15.119:6803/2141950133] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f15a8061d10 con 0x7f15900acac0 2023-12-07T10:18:35.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.390+0000 7f158ffff700 1 -- 172.21.15.119:0/1296384337 <== osd.0 v2:172.21.15.119:6802/2141950133 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f15a8061d10 con 0x7f15900acac0 2023-12-07T10:18:35.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.391+0000 7f15ad231700 1 -- 172.21.15.119:0/1296384337 >> [v2:172.21.15.119:6802/2141950133,v1:172.21.15.119:6803/2141950133] conn(0x7f15900acac0 msgr2=0x7f15900aef60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:35.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.391+0000 7f15ad231700 1 --2- 172.21.15.119:0/1296384337 >> [v2:172.21.15.119:6802/2141950133,v1:172.21.15.119:6803/2141950133] conn(0x7f15900acac0 0x7f15900aef60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:35.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.391+0000 7f15ad231700 1 -- 172.21.15.119:0/1296384337 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f15900a1ba0 msgr2=0x7f15900a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:35.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.391+0000 7f15ad231700 1 --2- 172.21.15.119:0/1296384337 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f15900a1ba0 0x7f15900a4060 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1598009bd0 tx=0x7f1598009260 comp rx=0 tx=0).stop 2023-12-07T10:18:35.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.391+0000 7f15ad231700 1 -- 172.21.15.119:0/1296384337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a8107730 msgr2=0x7f15a81979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:35.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.391+0000 7f15ad231700 1 --2- 172.21.15.119:0/1296384337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a8107730 0x7f15a81979e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f15a000b070 tx=0x7f15a0007860 comp rx=0 tx=0).stop 2023-12-07T10:18:35.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.391+0000 7f15a759e700 1 -- 172.21.15.119:0/1296384337 reap_dead start 2023-12-07T10:18:35.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.391+0000 7f15ad231700 1 -- 172.21.15.119:0/1296384337 shutdown_connections 2023-12-07T10:18:35.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.391+0000 7f15ad231700 1 -- 172.21.15.119:0/1296384337 >> 172.21.15.119:0/1296384337 conn(0x7f15a8078800 msgr2=0x7f15a8100300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:35.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.391+0000 7f15ad231700 1 -- 172.21.15.119:0/1296384337 shutdown_connections 2023-12-07T10:18:35.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:34.391+0000 7f15ad231700 1 -- 172.21.15.119:0/1296384337 wait complete. 2023-12-07T10:18:35.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:35 smithi119 ceph-mon[97908]: pgmap v308: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-12-07T10:18:36.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:35 smithi159 ceph-mon[134726]: pgmap v308: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-12-07T10:18:36.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:35 smithi153 ceph-mon[133625]: pgmap v308: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-12-07T10:18:36.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.295+0000 7f729fdd0700 1 -- 172.21.15.119:0/3913464498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72980694f0 msgr2=0x7f7298069970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:36.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.295+0000 7f729fdd0700 1 --2- 172.21.15.119:0/3913464498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72980694f0 0x7f7298069970 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7288009990 tx=0x7f7288009ca0 comp rx=0 tx=0).stop 2023-12-07T10:18:36.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.295+0000 7f729fdd0700 1 -- 172.21.15.119:0/3913464498 shutdown_connections 2023-12-07T10:18:36.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.295+0000 7f729fdd0700 1 --2- 172.21.15.119:0/3913464498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7298104070 0x7f7298104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:36.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.295+0000 7f729fdd0700 1 --2- 172.21.15.119:0/3913464498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7298106510 0x7f7298068fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:36.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.295+0000 7f729fdd0700 1 --2- 172.21.15.119:0/3913464498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72980694f0 0x7f7298069970 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:36.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.295+0000 7f729fdd0700 1 -- 172.21.15.119:0/3913464498 >> 172.21.15.119:0/3913464498 conn(0x7f72980765b0 msgr2=0x7f72980769c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:36.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.295+0000 7f729fdd0700 1 -- 172.21.15.119:0/3913464498 shutdown_connections 2023-12-07T10:18:36.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.295+0000 7f729fdd0700 1 -- 172.21.15.119:0/3913464498 wait complete. 2023-12-07T10:18:36.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.296+0000 7f729fdd0700 1 Processor -- start 2023-12-07T10:18:36.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.297+0000 7f729fdd0700 1 -- start start 2023-12-07T10:18:36.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.297+0000 7f729fdd0700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72980694f0 0x7f729819dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:36.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.297+0000 7f729fdd0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7298104070 0x7f729819e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:36.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.297+0000 7f729fdd0700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7298106510 0x7f72981a28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:36.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.297+0000 7f729fdd0700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72981a2ed0 con 0x7f72980694f0 2023-12-07T10:18:36.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.297+0000 7f729fdd0700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72981a3040 con 0x7f7298104070 2023-12-07T10:18:36.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.297+0000 7f729fdd0700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72981a3200 con 0x7f7298106510 2023-12-07T10:18:36.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.297+0000 7f729d36b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7298104070 0x7f729819e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:36.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.298+0000 7f729d36b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7298104070 0x7f729819e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:35202/0 (socket says 172.21.15.119:35202) 2023-12-07T10:18:36.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.298+0000 7f729d36b700 1 -- 172.21.15.119:0/3384507532 learned_addr learned my addr 172.21.15.119:0/3384507532 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:36.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.298+0000 7f729db6c700 1 --2- 172.21.15.119:0/3384507532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72980694f0 0x7f729819dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:36.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.298+0000 7f729e36d700 1 --2- 172.21.15.119:0/3384507532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7298106510 0x7f72981a28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:36.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.298+0000 7f729d36b700 1 -- 172.21.15.119:0/3384507532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7298106510 msgr2=0x7f72981a28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:36.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.298+0000 7f729d36b700 1 --2- 172.21.15.119:0/3384507532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7298106510 0x7f72981a28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:36.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.298+0000 7f729d36b700 1 -- 172.21.15.119:0/3384507532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72980694f0 msgr2=0x7f729819dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:36.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.298+0000 7f729d36b700 1 --2- 172.21.15.119:0/3384507532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72980694f0 0x7f729819dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:36.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.298+0000 7f729d36b700 1 -- 172.21.15.119:0/3384507532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7294009520 con 0x7f7298104070 2023-12-07T10:18:36.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.298+0000 7f729e36d700 1 --2- 172.21.15.119:0/3384507532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7298106510 0x7f72981a28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:18:36.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.298+0000 7f729db6c700 1 --2- 172.21.15.119:0/3384507532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72980694f0 0x7f729819dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:18:36.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.298+0000 7f729d36b700 1 --2- 172.21.15.119:0/3384507532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7298104070 0x7f729819e510 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7288005770 tx=0x7f7288005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:36.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.299+0000 7f728effd700 1 -- 172.21.15.119:0/3384507532 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728801d070 con 0x7f7298104070 2023-12-07T10:18:36.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.299+0000 7f729fdd0700 1 -- 172.21.15.119:0/3384507532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72880095f0 con 0x7f7298104070 2023-12-07T10:18:36.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.299+0000 7f728effd700 1 -- 172.21.15.119:0/3384507532 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7288021410 con 0x7f7298104070 2023-12-07T10:18:36.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.299+0000 7f728effd700 1 -- 172.21.15.119:0/3384507532 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7288017650 con 0x7f7298104070 2023-12-07T10:18:36.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.299+0000 7f729fdd0700 1 -- 172.21.15.119:0/3384507532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72981aab70 con 0x7f7298104070 2023-12-07T10:18:36.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.300+0000 7f729fdd0700 1 -- 172.21.15.119:0/3384507532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7298117d60 con 0x7f7298104070 2023-12-07T10:18:36.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.302+0000 7f728effd700 1 -- 172.21.15.119:0/3384507532 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f72880217f0 con 0x7f7298104070 2023-12-07T10:18:36.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.303+0000 7f728effd700 1 --2- 172.21.15.119:0/3384507532 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f72840a1ba0 0x7f72840a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:36.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.303+0000 7f728effd700 1 -- 172.21.15.119:0/3384507532 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f72880cf750 con 0x7f7298104070 2023-12-07T10:18:36.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.304+0000 7f728effd700 1 --2- 172.21.15.119:0/3384507532 >> [v2:172.21.15.159:6800/708101637,v1:172.21.15.159:6801/708101637] conn(0x7f72840acb40 0x7f72840aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:36.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.304+0000 7f728effd700 1 -- 172.21.15.119:0/3384507532 --> [v2:172.21.15.159:6800/708101637,v1:172.21.15.159:6801/708101637] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f72840af6b0 con 0x7f72840acb40 2023-12-07T10:18:36.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.304+0000 7f728effd700 1 -- 172.21.15.119:0/3384507532 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72880cfb60 con 0x7f7298104070 2023-12-07T10:18:36.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.304+0000 7f729db6c700 1 --2- 172.21.15.119:0/3384507532 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f72840a1ba0 0x7f72840a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:36.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.305+0000 7f729db6c700 1 --2- 172.21.15.119:0/3384507532 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f72840a1ba0 0x7f72840a4060 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7294009e20 tx=0x7f7294009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:36.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.305+0000 7f729e36d700 1 --2- 172.21.15.119:0/3384507532 >> [v2:172.21.15.159:6800/708101637,v1:172.21.15.159:6801/708101637] conn(0x7f72840acb40 0x7f72840aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:36.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.305+0000 7f729e36d700 1 --2- 172.21.15.119:0/3384507532 >> [v2:172.21.15.159:6800/708101637,v1:172.21.15.159:6801/708101637] conn(0x7f72840acb40 0x7f72840aefe0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:36.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.306+0000 7f728effd700 1 -- 172.21.15.119:0/3384507532 <== osd.2 v2:172.21.15.159:6800/708101637 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f72840af6b0 con 0x7f72840acb40 2023-12-07T10:18:36.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.326+0000 7f729fdd0700 1 -- 172.21.15.119:0/3384507532 --> [v2:172.21.15.159:6800/708101637,v1:172.21.15.159:6801/708101637] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f72981029a0 con 0x7f72840acb40 2023-12-07T10:18:36.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.326+0000 7f728effd700 1 -- 172.21.15.119:0/3384507532 <== osd.2 v2:172.21.15.159:6800/708101637 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f72981029a0 con 0x7f72840acb40 2023-12-07T10:18:36.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.327+0000 7f729fdd0700 1 -- 172.21.15.119:0/3384507532 >> [v2:172.21.15.159:6800/708101637,v1:172.21.15.159:6801/708101637] conn(0x7f72840acb40 msgr2=0x7f72840aefe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:36.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.327+0000 7f729fdd0700 1 --2- 172.21.15.119:0/3384507532 >> [v2:172.21.15.159:6800/708101637,v1:172.21.15.159:6801/708101637] conn(0x7f72840acb40 0x7f72840aefe0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:36.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.327+0000 7f729fdd0700 1 -- 172.21.15.119:0/3384507532 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f72840a1ba0 msgr2=0x7f72840a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:36.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.327+0000 7f729fdd0700 1 --2- 172.21.15.119:0/3384507532 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f72840a1ba0 0x7f72840a4060 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7294009e20 tx=0x7f7294009260 comp rx=0 tx=0).stop 2023-12-07T10:18:36.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.327+0000 7f729fdd0700 1 -- 172.21.15.119:0/3384507532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7298104070 msgr2=0x7f729819e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:36.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.327+0000 7f729fdd0700 1 --2- 172.21.15.119:0/3384507532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7298104070 0x7f729819e510 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7288005770 tx=0x7f7288005e80 comp rx=0 tx=0).stop 2023-12-07T10:18:36.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.327+0000 7f729e36d700 1 -- 172.21.15.119:0/3384507532 reap_dead start 2023-12-07T10:18:36.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.327+0000 7f729fdd0700 1 -- 172.21.15.119:0/3384507532 shutdown_connections 2023-12-07T10:18:36.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.327+0000 7f729fdd0700 1 -- 172.21.15.119:0/3384507532 >> 172.21.15.119:0/3384507532 conn(0x7f72980765b0 msgr2=0x7f7298101040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:36.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.328+0000 7f729fdd0700 1 -- 172.21.15.119:0/3384507532 shutdown_connections 2023-12-07T10:18:36.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.328+0000 7f729fdd0700 1 -- 172.21.15.119:0/3384507532 wait complete. 2023-12-07T10:18:36.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.901+0000 7f88eb046700 1 -- 172.21.15.119:0/1885491429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88e4107730 msgr2=0x7f88e4107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:36.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.901+0000 7f88eb046700 1 --2- 172.21.15.119:0/1885491429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88e4107730 0x7f88e4107b10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009990 tx=0x7f88d8009ca0 comp rx=0 tx=0).stop 2023-12-07T10:18:36.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.901+0000 7f88eb046700 1 -- 172.21.15.119:0/1885491429 shutdown_connections 2023-12-07T10:18:36.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.901+0000 7f88eb046700 1 --2- 172.21.15.119:0/1885491429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88e4102080 0x7f88e410e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:36.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.901+0000 7f88eb046700 1 --2- 172.21.15.119:0/1885491429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88e4107730 0x7f88e4107b10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:36.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.901+0000 7f88eb046700 1 --2- 172.21.15.119:0/1885491429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88e41016c0 0x7f88e4101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:36.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.901+0000 7f88eb046700 1 -- 172.21.15.119:0/1885491429 >> 172.21.15.119:0/1885491429 conn(0x7f88e4078800 msgr2=0x7f88e4078c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:36.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.901+0000 7f88eb046700 1 -- 172.21.15.119:0/1885491429 shutdown_connections 2023-12-07T10:18:36.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.901+0000 7f88eb046700 1 -- 172.21.15.119:0/1885491429 wait complete. 2023-12-07T10:18:36.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.902+0000 7f88eb046700 1 Processor -- start 2023-12-07T10:18:36.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.902+0000 7f88eb046700 1 -- start start 2023-12-07T10:18:36.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.903+0000 7f88eb046700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88e41016c0 0x7f88e419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:36.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.903+0000 7f88eb046700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88e4102080 0x7f88e419dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:36.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.903+0000 7f88eb046700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88e4107730 0x7f88e4197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:36.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.903+0000 7f88eb046700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e4197ec0 con 0x7f88e4102080 2023-12-07T10:18:36.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.903+0000 7f88eb046700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e4198030 con 0x7f88e41016c0 2023-12-07T10:18:36.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.903+0000 7f88eb046700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e41981f0 con 0x7f88e4107730 2023-12-07T10:18:36.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.903+0000 7f88e3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88e4102080 0x7f88e419dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:36.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.903+0000 7f88e8de2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88e41016c0 0x7f88e419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:36.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.903+0000 7f88e8de2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88e41016c0 0x7f88e419d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:35218/0 (socket says 172.21.15.119:35218) 2023-12-07T10:18:36.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.903+0000 7f88e8de2700 1 -- 172.21.15.119:0/3146022381 learned_addr learned my addr 172.21.15.119:0/3146022381 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:36.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.903+0000 7f88e95e3700 1 --2- 172.21.15.119:0/3146022381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88e4107730 0x7f88e4197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:36.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.903+0000 7f88e3fff700 1 -- 172.21.15.119:0/3146022381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88e4107730 msgr2=0x7f88e4197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:36.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.903+0000 7f88e3fff700 1 --2- 172.21.15.119:0/3146022381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88e4107730 0x7f88e4197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:36.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.903+0000 7f88e3fff700 1 -- 172.21.15.119:0/3146022381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88e41016c0 msgr2=0x7f88e419d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:36.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.903+0000 7f88e3fff700 1 --2- 172.21.15.119:0/3146022381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88e41016c0 0x7f88e419d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:36.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.903+0000 7f88e3fff700 1 -- 172.21.15.119:0/3146022381 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d80095f0 con 0x7f88e4102080 2023-12-07T10:18:36.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.904+0000 7f88e3fff700 1 --2- 172.21.15.119:0/3146022381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88e4102080 0x7f88e419dc50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f88d000e9f0 tx=0x7f88d000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:36.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.904+0000 7f88e1ffb700 1 -- 172.21.15.119:0/3146022381 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d000c9c0 con 0x7f88e4102080 2023-12-07T10:18:36.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.904+0000 7f88eb046700 1 -- 172.21.15.119:0/3146022381 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e41984d0 con 0x7f88e4102080 2023-12-07T10:18:36.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.904+0000 7f88e1ffb700 1 -- 172.21.15.119:0/3146022381 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88d0010430 con 0x7f88e4102080 2023-12-07T10:18:36.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.904+0000 7f88e1ffb700 1 -- 172.21.15.119:0/3146022381 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d0018680 con 0x7f88e4102080 2023-12-07T10:18:36.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.904+0000 7f88eb046700 1 -- 172.21.15.119:0/3146022381 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e41a4700 con 0x7f88e4102080 2023-12-07T10:18:36.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.905+0000 7f88eb046700 1 -- 172.21.15.119:0/3146022381 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f88e41117d0 con 0x7f88e4102080 2023-12-07T10:18:36.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.907+0000 7f88e1ffb700 1 -- 172.21.15.119:0/3146022381 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f88d0010860 con 0x7f88e4102080 2023-12-07T10:18:36.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.908+0000 7f88e1ffb700 1 --2- 172.21.15.119:0/3146022381 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f88d40a1ba0 0x7f88d40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:36.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.908+0000 7f88e1ffb700 1 -- 172.21.15.119:0/3146022381 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f88d0014070 con 0x7f88e4102080 2023-12-07T10:18:36.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.908+0000 7f88e8de2700 1 --2- 172.21.15.119:0/3146022381 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f88d40a1ba0 0x7f88d40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:36.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.909+0000 7f88e1ffb700 1 --2- 172.21.15.119:0/3146022381 >> [v2:172.21.15.159:6816/2600946380,v1:172.21.15.159:6817/2600946380] conn(0x7f88d40acb40 0x7f88d40aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.909+0000 7f88e1ffb700 1 -- 172.21.15.119:0/3146022381 --> [v2:172.21.15.159:6816/2600946380,v1:172.21.15.159:6817/2600946380] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f88d40af6b0 con 0x7f88d40acb40 2023-12-07T10:18:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.909+0000 7f88e1ffb700 1 -- 172.21.15.119:0/3146022381 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88d00ce210 con 0x7f88e4102080 2023-12-07T10:18:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.909+0000 7f88e95e3700 1 --2- 172.21.15.119:0/3146022381 >> [v2:172.21.15.159:6816/2600946380,v1:172.21.15.159:6817/2600946380] conn(0x7f88d40acb40 0x7f88d40aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.909+0000 7f88e8de2700 1 --2- 172.21.15.119:0/3146022381 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f88d40a1ba0 0x7f88d40a4060 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f88d800b6f0 tx=0x7f88d8005bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.909+0000 7f88e95e3700 1 --2- 172.21.15.119:0/3146022381 >> [v2:172.21.15.159:6816/2600946380,v1:172.21.15.159:6817/2600946380] conn(0x7f88d40acb40 0x7f88d40aefe0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:36.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.910+0000 7f88e1ffb700 1 -- 172.21.15.119:0/3146022381 <== osd.8 v2:172.21.15.159:6816/2600946380 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f88d40af6b0 con 0x7f88d40acb40 2023-12-07T10:18:36.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.928+0000 7f88eb046700 1 -- 172.21.15.119:0/3146022381 --> [v2:172.21.15.159:6816/2600946380,v1:172.21.15.159:6817/2600946380] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f88e4061d10 con 0x7f88d40acb40 2023-12-07T10:18:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.929+0000 7f88e1ffb700 1 -- 172.21.15.119:0/3146022381 <== osd.8 v2:172.21.15.159:6816/2600946380 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f88e4061d10 con 0x7f88d40acb40 2023-12-07T10:18:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.929+0000 7f88eb046700 1 -- 172.21.15.119:0/3146022381 >> [v2:172.21.15.159:6816/2600946380,v1:172.21.15.159:6817/2600946380] conn(0x7f88d40acb40 msgr2=0x7f88d40aefe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.929+0000 7f88eb046700 1 --2- 172.21.15.119:0/3146022381 >> [v2:172.21.15.159:6816/2600946380,v1:172.21.15.159:6817/2600946380] conn(0x7f88d40acb40 0x7f88d40aefe0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.930+0000 7f88eb046700 1 -- 172.21.15.119:0/3146022381 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f88d40a1ba0 msgr2=0x7f88d40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.930+0000 7f88eb046700 1 --2- 172.21.15.119:0/3146022381 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f88d40a1ba0 0x7f88d40a4060 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f88d800b6f0 tx=0x7f88d8005bc0 comp rx=0 tx=0).stop 2023-12-07T10:18:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.930+0000 7f88eb046700 1 -- 172.21.15.119:0/3146022381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88e4102080 msgr2=0x7f88e419dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.930+0000 7f88eb046700 1 --2- 172.21.15.119:0/3146022381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88e4102080 0x7f88e419dc50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f88d000e9f0 tx=0x7f88d000ed00 comp rx=0 tx=0).stop 2023-12-07T10:18:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.930+0000 7f88e95e3700 1 -- 172.21.15.119:0/3146022381 reap_dead start 2023-12-07T10:18:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.930+0000 7f88eb046700 1 -- 172.21.15.119:0/3146022381 shutdown_connections 2023-12-07T10:18:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.930+0000 7f88eb046700 1 -- 172.21.15.119:0/3146022381 >> 172.21.15.119:0/3146022381 conn(0x7f88e4078800 msgr2=0x7f88e4100400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.930+0000 7f88eb046700 1 -- 172.21.15.119:0/3146022381 shutdown_connections 2023-12-07T10:18:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:36.930+0000 7f88eb046700 1 -- 172.21.15.119:0/3146022381 wait complete. 2023-12-07T10:18:37.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:37 smithi119 ceph-mon[97908]: pgmap v309: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 10 op/s 2023-12-07T10:18:38.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.986+0000 7f3ac9e66700 1 -- 172.21.15.119:0/237389198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ac4100f30 msgr2=0x7f3ac410d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:38.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.986+0000 7f3ac9e66700 1 --2- 172.21.15.119:0/237389198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ac4100f30 0x7f3ac410d370 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00b0d0 tx=0x7f3abc00b3e0 comp rx=0 tx=0).stop 2023-12-07T10:18:38.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.986+0000 7f3ac9e66700 1 -- 172.21.15.119:0/237389198 shutdown_connections 2023-12-07T10:18:38.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.986+0000 7f3ac9e66700 1 --2- 172.21.15.119:0/237389198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ac4100f30 0x7f3ac410d370 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:38.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.986+0000 7f3ac9e66700 1 --2- 172.21.15.119:0/237389198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ac41065a0 0x7f3ac4106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:38.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.986+0000 7f3ac9e66700 1 --2- 172.21.15.119:0/237389198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ac4100570 0x7f3ac41009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:38.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.986+0000 7f3ac9e66700 1 -- 172.21.15.119:0/237389198 >> 172.21.15.119:0/237389198 conn(0x7f3ac40765b0 msgr2=0x7f3ac40769c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:38.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.986+0000 7f3ac9e66700 1 -- 172.21.15.119:0/237389198 shutdown_connections 2023-12-07T10:18:38.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.986+0000 7f3ac9e66700 1 -- 172.21.15.119:0/237389198 wait complete. 2023-12-07T10:18:38.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.988+0000 7f3ac9e66700 1 Processor -- start 2023-12-07T10:18:38.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.988+0000 7f3ac9e66700 1 -- start start 2023-12-07T10:18:38.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.989+0000 7f3ac9e66700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ac4100570 0x7f3ac4198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:38.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.989+0000 7f3ac9e66700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ac4100f30 0x7f3ac4198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:38.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.989+0000 7f3ac9e66700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ac41065a0 0x7f3ac41994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:38.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.989+0000 7f3ac9e66700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac41a3d10 con 0x7f3ac41065a0 2023-12-07T10:18:38.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.989+0000 7f3ac9e66700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac41a3e80 con 0x7f3ac4100f30 2023-12-07T10:18:38.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.989+0000 7f3ac9e66700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac41a4040 con 0x7f3ac4100570 2023-12-07T10:18:38.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.989+0000 7f3ac3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ac41065a0 0x7f3ac41994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:38.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.989+0000 7f3ac3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ac41065a0 0x7f3ac41994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44114/0 (socket says 172.21.15.119:44114) 2023-12-07T10:18:38.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.989+0000 7f3ac3fff700 1 -- 172.21.15.119:0/3618091652 learned_addr learned my addr 172.21.15.119:0/3618091652 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:38.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.989+0000 7f3ac37fe700 1 --2- 172.21.15.119:0/3618091652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ac4100570 0x7f3ac4198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:38.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.989+0000 7f3ac2ffd700 1 --2- 172.21.15.119:0/3618091652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ac4100f30 0x7f3ac4198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:38.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.989+0000 7f3ac37fe700 1 -- 172.21.15.119:0/3618091652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ac4100f30 msgr2=0x7f3ac4198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:38.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.990+0000 7f3ac37fe700 1 --2- 172.21.15.119:0/3618091652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ac4100f30 0x7f3ac4198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:38.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.990+0000 7f3ac37fe700 1 -- 172.21.15.119:0/3618091652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ac41065a0 msgr2=0x7f3ac41994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:38.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.990+0000 7f3ac37fe700 1 --2- 172.21.15.119:0/3618091652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ac41065a0 0x7f3ac41994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:38.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.990+0000 7f3ac37fe700 1 -- 172.21.15.119:0/3618091652 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3abc009d00 con 0x7f3ac4100570 2023-12-07T10:18:38.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.990+0000 7f3ac3fff700 1 --2- 172.21.15.119:0/3618091652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ac41065a0 0x7f3ac41994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:18:38.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.990+0000 7f3ac2ffd700 1 --2- 172.21.15.119:0/3618091652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ac4100f30 0x7f3ac4198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:18:38.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.990+0000 7f3ac37fe700 1 --2- 172.21.15.119:0/3618091652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ac4100570 0x7f3ac4198a00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3ab400e9f0 tx=0x7f3ab400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:38.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.990+0000 7f3ac0ff9700 1 -- 172.21.15.119:0/3618091652 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab400c9c0 con 0x7f3ac4100570 2023-12-07T10:18:38.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.990+0000 7f3ac0ff9700 1 -- 172.21.15.119:0/3618091652 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ab4010430 con 0x7f3ac4100570 2023-12-07T10:18:38.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.990+0000 7f3ac9e66700 1 -- 172.21.15.119:0/3618091652 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac41a4320 con 0x7f3ac4100570 2023-12-07T10:18:38.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.990+0000 7f3ac0ff9700 1 -- 172.21.15.119:0/3618091652 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab4018680 con 0x7f3ac4100570 2023-12-07T10:18:38.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.990+0000 7f3ac9e66700 1 -- 172.21.15.119:0/3618091652 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac41a4870 con 0x7f3ac4100570 2023-12-07T10:18:38.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.991+0000 7f3ac9e66700 1 -- 172.21.15.119:0/3618091652 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3ac4110720 con 0x7f3ac4100570 2023-12-07T10:18:38.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.993+0000 7f3ac0ff9700 1 -- 172.21.15.119:0/3618091652 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f3ab400cb20 con 0x7f3ac4100570 2023-12-07T10:18:38.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.995+0000 7f3ac0ff9700 1 --2- 172.21.15.119:0/3618091652 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3aac0a1c20 0x7f3aac0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:38.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.995+0000 7f3ac0ff9700 1 -- 172.21.15.119:0/3618091652 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3ab4014070 con 0x7f3ac4100570 2023-12-07T10:18:38.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.995+0000 7f3ac2ffd700 1 --2- 172.21.15.119:0/3618091652 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3aac0a1c20 0x7f3aac0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:38.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.995+0000 7f3ac0ff9700 1 --2- 172.21.15.119:0/3618091652 >> [v2:172.21.15.119:6810/3228047257,v1:172.21.15.119:6811/3228047257] conn(0x7f3aac0acbc0 0x7f3aac0af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:38.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.996+0000 7f3ac0ff9700 1 -- 172.21.15.119:0/3618091652 --> [v2:172.21.15.119:6810/3228047257,v1:172.21.15.119:6811/3228047257] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3aac0af730 con 0x7f3aac0acbc0 2023-12-07T10:18:38.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.996+0000 7f3ac0ff9700 1 -- 172.21.15.119:0/3618091652 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ab40d22a0 con 0x7f3ac4100570 2023-12-07T10:18:38.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.996+0000 7f3ac3fff700 1 --2- 172.21.15.119:0/3618091652 >> [v2:172.21.15.119:6810/3228047257,v1:172.21.15.119:6811/3228047257] conn(0x7f3aac0acbc0 0x7f3aac0af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:38.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.997+0000 7f3ac2ffd700 1 --2- 172.21.15.119:0/3618091652 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3aac0a1c20 0x7f3aac0a40e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4069770 tx=0x7f3ab8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:38.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.997+0000 7f3ac3fff700 1 --2- 172.21.15.119:0/3618091652 >> [v2:172.21.15.119:6810/3228047257,v1:172.21.15.119:6811/3228047257] conn(0x7f3aac0acbc0 0x7f3aac0af060 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:38.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.998+0000 7f3ac0ff9700 1 -- 172.21.15.119:0/3618091652 <== osd.3 v2:172.21.15.119:6810/3228047257 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f3aac0af730 con 0x7f3aac0acbc0 2023-12-07T10:18:38.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.019+0000 7f3ac9e66700 1 -- 172.21.15.119:0/3618091652 --> [v2:172.21.15.119:6810/3228047257,v1:172.21.15.119:6811/3228047257] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3ac4101c10 con 0x7f3aac0acbc0 2023-12-07T10:18:38.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.020+0000 7f3ac0ff9700 1 -- 172.21.15.119:0/3618091652 <== osd.3 v2:172.21.15.119:6810/3228047257 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3ac4101c10 con 0x7f3aac0acbc0 2023-12-07T10:18:38.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.020+0000 7f3ac9e66700 1 -- 172.21.15.119:0/3618091652 >> [v2:172.21.15.119:6810/3228047257,v1:172.21.15.119:6811/3228047257] conn(0x7f3aac0acbc0 msgr2=0x7f3aac0af060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:38.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.020+0000 7f3ac9e66700 1 --2- 172.21.15.119:0/3618091652 >> [v2:172.21.15.119:6810/3228047257,v1:172.21.15.119:6811/3228047257] conn(0x7f3aac0acbc0 0x7f3aac0af060 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:38.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.020+0000 7f3ac9e66700 1 -- 172.21.15.119:0/3618091652 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3aac0a1c20 msgr2=0x7f3aac0a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:38.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.020+0000 7f3ac9e66700 1 --2- 172.21.15.119:0/3618091652 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3aac0a1c20 0x7f3aac0a40e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4069770 tx=0x7f3ab8009260 comp rx=0 tx=0).stop 2023-12-07T10:18:38.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.020+0000 7f3ac9e66700 1 -- 172.21.15.119:0/3618091652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ac4100570 msgr2=0x7f3ac4198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:38.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.020+0000 7f3ac9e66700 1 --2- 172.21.15.119:0/3618091652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ac4100570 0x7f3ac4198a00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3ab400e9f0 tx=0x7f3ab400ed00 comp rx=0 tx=0).stop 2023-12-07T10:18:38.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.020+0000 7f3ac3fff700 1 -- 172.21.15.119:0/3618091652 reap_dead start 2023-12-07T10:18:38.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.020+0000 7f3ac9e66700 1 -- 172.21.15.119:0/3618091652 shutdown_connections 2023-12-07T10:18:38.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.020+0000 7f3ac9e66700 1 -- 172.21.15.119:0/3618091652 >> 172.21.15.119:0/3618091652 conn(0x7f3ac40765b0 msgr2=0x7f3ac40ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:38.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.020+0000 7f3ac9e66700 1 -- 172.21.15.119:0/3618091652 shutdown_connections 2023-12-07T10:18:38.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.020+0000 7f3ac9e66700 1 -- 172.21.15.119:0/3618091652 wait complete. 2023-12-07T10:18:38.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.634+0000 7fd0b1f8f700 1 -- 172.21.15.119:0/3713680807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0ac1016c0 msgr2=0x7fd0ac101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:38.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.634+0000 7fd0b1f8f700 1 --2- 172.21.15.119:0/3713680807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0ac1016c0 0x7fd0ac101b40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009990 tx=0x7fd0a0009ca0 comp rx=0 tx=0).stop 2023-12-07T10:18:38.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.634+0000 7fd0b1f8f700 1 -- 172.21.15.119:0/3713680807 shutdown_connections 2023-12-07T10:18:38.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.634+0000 7fd0b1f8f700 1 --2- 172.21.15.119:0/3713680807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0ac102080 0x7fd0ac10e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:38.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.634+0000 7fd0b1f8f700 1 --2- 172.21.15.119:0/3713680807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0ac107730 0x7fd0ac107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:38.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.634+0000 7fd0b1f8f700 1 --2- 172.21.15.119:0/3713680807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0ac1016c0 0x7fd0ac101b40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:38.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.634+0000 7fd0b1f8f700 1 -- 172.21.15.119:0/3713680807 >> 172.21.15.119:0/3713680807 conn(0x7fd0ac078800 msgr2=0x7fd0ac078c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:38.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.634+0000 7fd0b1f8f700 1 -- 172.21.15.119:0/3713680807 shutdown_connections 2023-12-07T10:18:38.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.634+0000 7fd0b1f8f700 1 -- 172.21.15.119:0/3713680807 wait complete. 2023-12-07T10:18:38.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.635+0000 7fd0b1f8f700 1 Processor -- start 2023-12-07T10:18:38.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.635+0000 7fd0b1f8f700 1 -- start start 2023-12-07T10:18:38.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.635+0000 7fd0b1f8f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0ac1016c0 0x7fd0ac19f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:38.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.635+0000 7fd0b1f8f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0ac102080 0x7fd0ac19fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:38.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.635+0000 7fd0b1f8f700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0ac107730 0x7fd0ac199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:38.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.635+0000 7fd0b1f8f700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac19a0a0 con 0x7fd0ac107730 2023-12-07T10:18:38.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.635+0000 7fd0b1f8f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac19a210 con 0x7fd0ac102080 2023-12-07T10:18:38.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.635+0000 7fd0b1f8f700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac19a3d0 con 0x7fd0ac1016c0 2023-12-07T10:18:38.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.636+0000 7fd0abfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0ac107730 0x7fd0ac199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:38.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.636+0000 7fd0abfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0ac107730 0x7fd0ac199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44104/0 (socket says 172.21.15.119:44104) 2023-12-07T10:18:38.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.636+0000 7fd0abfff700 1 -- 172.21.15.119:0/2987337593 learned_addr learned my addr 172.21.15.119:0/2987337593 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:38.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.636+0000 7fd0ab7fe700 1 --2- 172.21.15.119:0/2987337593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0ac1016c0 0x7fd0ac19f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:38.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.636+0000 7fd0aaffd700 1 --2- 172.21.15.119:0/2987337593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0ac102080 0x7fd0ac19fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:38.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.636+0000 7fd0abfff700 1 -- 172.21.15.119:0/2987337593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0ac1016c0 msgr2=0x7fd0ac19f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:38.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.636+0000 7fd0abfff700 1 --2- 172.21.15.119:0/2987337593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0ac1016c0 0x7fd0ac19f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:38.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.636+0000 7fd0abfff700 1 -- 172.21.15.119:0/2987337593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0ac102080 msgr2=0x7fd0ac19fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:38.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.636+0000 7fd0abfff700 1 --2- 172.21.15.119:0/2987337593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0ac102080 0x7fd0ac19fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:38.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.636+0000 7fd0abfff700 1 -- 172.21.15.119:0/2987337593 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd09c009520 con 0x7fd0ac107730 2023-12-07T10:18:38.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.636+0000 7fd0ab7fe700 1 --2- 172.21.15.119:0/2987337593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0ac1016c0 0x7fd0ac19f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:18:38.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.636+0000 7fd0aaffd700 1 --2- 172.21.15.119:0/2987337593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0ac102080 0x7fd0ac19fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:18:38.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.636+0000 7fd0abfff700 1 --2- 172.21.15.119:0/2987337593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0ac107730 0x7fd0ac199a40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd0a400efc0 tx=0x7fd0a400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:38.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.637+0000 7fd0a8ff9700 1 -- 172.21.15.119:0/2987337593 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a4018070 con 0x7fd0ac107730 2023-12-07T10:18:38.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.637+0000 7fd0a8ff9700 1 -- 172.21.15.119:0/2987337593 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0a400f040 con 0x7fd0ac107730 2023-12-07T10:18:38.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.637+0000 7fd0b1f8f700 1 -- 172.21.15.119:0/2987337593 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a00095f0 con 0x7fd0ac107730 2023-12-07T10:18:38.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.637+0000 7fd0a8ff9700 1 -- 172.21.15.119:0/2987337593 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a400cd80 con 0x7fd0ac107730 2023-12-07T10:18:38.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.637+0000 7fd0b1f8f700 1 -- 172.21.15.119:0/2987337593 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0ac1a6720 con 0x7fd0ac107730 2023-12-07T10:18:38.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.638+0000 7fd0b1f8f700 1 -- 172.21.15.119:0/2987337593 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd0ac111820 con 0x7fd0ac107730 2023-12-07T10:18:38.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.640+0000 7fd0a8ff9700 1 -- 172.21.15.119:0/2987337593 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7fd0a4003a00 con 0x7fd0ac107730 2023-12-07T10:18:38.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.641+0000 7fd0a8ff9700 1 --2- 172.21.15.119:0/2987337593 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd0940a1c20 0x7fd0940a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:38.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.641+0000 7fd0a8ff9700 1 -- 172.21.15.119:0/2987337593 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd0a40cdf00 con 0x7fd0ac107730 2023-12-07T10:18:38.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.641+0000 7fd0ab7fe700 1 --2- 172.21.15.119:0/2987337593 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd0940a1c20 0x7fd0940a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:38.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.641+0000 7fd0a8ff9700 1 --2- 172.21.15.119:0/2987337593 >> [v2:172.21.15.119:6826/1940737285,v1:172.21.15.119:6827/1940737285] conn(0x7fd0940acbc0 0x7fd0940af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:38.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.641+0000 7fd0a8ff9700 1 -- 172.21.15.119:0/2987337593 --> [v2:172.21.15.119:6826/1940737285,v1:172.21.15.119:6827/1940737285] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd0940af730 con 0x7fd0940acbc0 2023-12-07T10:18:38.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.641+0000 7fd0a8ff9700 1 -- 172.21.15.119:0/2987337593 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0a40ce270 con 0x7fd0ac107730 2023-12-07T10:18:38.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.642+0000 7fd0aaffd700 1 --2- 172.21.15.119:0/2987337593 >> [v2:172.21.15.119:6826/1940737285,v1:172.21.15.119:6827/1940737285] conn(0x7fd0940acbc0 0x7fd0940af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:38.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.642+0000 7fd0ab7fe700 1 --2- 172.21.15.119:0/2987337593 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd0940a1c20 0x7fd0940a40e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd09c009e40 tx=0x7fd09c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:38.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.642+0000 7fd0aaffd700 1 --2- 172.21.15.119:0/2987337593 >> [v2:172.21.15.119:6826/1940737285,v1:172.21.15.119:6827/1940737285] conn(0x7fd0940acbc0 0x7fd0940af060 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:38.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.643+0000 7fd0a8ff9700 1 -- 172.21.15.119:0/2987337593 <== osd.9 v2:172.21.15.119:6826/1940737285 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fd0940af730 con 0x7fd0940acbc0 2023-12-07T10:18:38.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.660+0000 7fd0b1f8f700 1 -- 172.21.15.119:0/2987337593 --> [v2:172.21.15.119:6826/1940737285,v1:172.21.15.119:6827/1940737285] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd0ac100ac0 con 0x7fd0940acbc0 2023-12-07T10:18:38.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.661+0000 7fd0a8ff9700 1 -- 172.21.15.119:0/2987337593 <== osd.9 v2:172.21.15.119:6826/1940737285 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd0ac100ac0 con 0x7fd0940acbc0 2023-12-07T10:18:38.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.661+0000 7fd0b1f8f700 1 -- 172.21.15.119:0/2987337593 >> [v2:172.21.15.119:6826/1940737285,v1:172.21.15.119:6827/1940737285] conn(0x7fd0940acbc0 msgr2=0x7fd0940af060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:38.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.661+0000 7fd0b1f8f700 1 --2- 172.21.15.119:0/2987337593 >> [v2:172.21.15.119:6826/1940737285,v1:172.21.15.119:6827/1940737285] conn(0x7fd0940acbc0 0x7fd0940af060 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:38.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.661+0000 7fd0b1f8f700 1 -- 172.21.15.119:0/2987337593 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd0940a1c20 msgr2=0x7fd0940a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:38.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.661+0000 7fd0b1f8f700 1 --2- 172.21.15.119:0/2987337593 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd0940a1c20 0x7fd0940a40e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd09c009e40 tx=0x7fd09c009260 comp rx=0 tx=0).stop 2023-12-07T10:18:38.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.661+0000 7fd0b1f8f700 1 -- 172.21.15.119:0/2987337593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0ac107730 msgr2=0x7fd0ac199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:38.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.661+0000 7fd0b1f8f700 1 --2- 172.21.15.119:0/2987337593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0ac107730 0x7fd0ac199a40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd0a400efc0 tx=0x7fd0a400c3a0 comp rx=0 tx=0).stop 2023-12-07T10:18:38.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.661+0000 7fd0abfff700 1 -- 172.21.15.119:0/2987337593 reap_dead start 2023-12-07T10:18:38.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.662+0000 7fd0b1f8f700 1 -- 172.21.15.119:0/2987337593 shutdown_connections 2023-12-07T10:18:38.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.662+0000 7fd0b1f8f700 1 -- 172.21.15.119:0/2987337593 >> 172.21.15.119:0/2987337593 conn(0x7fd0ac078800 msgr2=0x7fd0ac1000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:38.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.662+0000 7fd0b1f8f700 1 -- 172.21.15.119:0/2987337593 shutdown_connections 2023-12-07T10:18:38.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:37.662+0000 7fd0b1f8f700 1 -- 172.21.15.119:0/2987337593 wait complete. 2023-12-07T10:18:38.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:37 smithi159 ceph-mon[134726]: pgmap v309: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 10 op/s 2023-12-07T10:18:38.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:37 smithi153 ceph-mon[133625]: pgmap v309: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 10 op/s 2023-12-07T10:18:38.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.038+0000 7f5b82c33700 1 -- 172.21.15.119:0/2304989847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b7c107730 msgr2=0x7f5b7c107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:38.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.038+0000 7f5b82c33700 1 --2- 172.21.15.119:0/2304989847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b7c107730 0x7f5b7c107b10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009990 tx=0x7f5b70009ca0 comp rx=0 tx=0).stop 2023-12-07T10:18:38.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.038+0000 7f5b82c33700 1 -- 172.21.15.119:0/2304989847 shutdown_connections 2023-12-07T10:18:38.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.038+0000 7f5b82c33700 1 --2- 172.21.15.119:0/2304989847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b7c102080 0x7f5b7c10e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:38.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.038+0000 7f5b82c33700 1 --2- 172.21.15.119:0/2304989847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b7c107730 0x7f5b7c107b10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:38.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.038+0000 7f5b82c33700 1 --2- 172.21.15.119:0/2304989847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b7c1016c0 0x7f5b7c101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:38.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.038+0000 7f5b82c33700 1 -- 172.21.15.119:0/2304989847 >> 172.21.15.119:0/2304989847 conn(0x7f5b7c078800 msgr2=0x7f5b7c078c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:38.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.038+0000 7f5b82c33700 1 -- 172.21.15.119:0/2304989847 shutdown_connections 2023-12-07T10:18:38.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.038+0000 7f5b82c33700 1 -- 172.21.15.119:0/2304989847 wait complete. 2023-12-07T10:18:38.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.039+0000 7f5b82c33700 1 Processor -- start 2023-12-07T10:18:38.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.040+0000 7f5b82c33700 1 -- start start 2023-12-07T10:18:38.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.040+0000 7f5b82c33700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b7c1016c0 0x7f5b7c19f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:38.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.040+0000 7f5b82c33700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b7c102080 0x7f5b7c19fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:38.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.040+0000 7f5b82c33700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b7c107730 0x7f5b7c199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:38.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.040+0000 7f5b82c33700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c19a0a0 con 0x7f5b7c107730 2023-12-07T10:18:38.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.040+0000 7f5b82c33700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c19a210 con 0x7f5b7c1016c0 2023-12-07T10:18:38.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.040+0000 7f5b82c33700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c19a3d0 con 0x7f5b7c102080 2023-12-07T10:18:38.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.041+0000 7f5b809cf700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b7c1016c0 0x7f5b7c19f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:38.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.041+0000 7f5b811d0700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b7c107730 0x7f5b7c199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:38.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.041+0000 7f5b809cf700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b7c1016c0 0x7f5b7c19f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:35264/0 (socket says 172.21.15.119:35264) 2023-12-07T10:18:38.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.041+0000 7f5b809cf700 1 -- 172.21.15.119:0/3146242674 learned_addr learned my addr 172.21.15.119:0/3146242674 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:38.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.041+0000 7f5b7bfff700 1 --2- 172.21.15.119:0/3146242674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b7c102080 0x7f5b7c19fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:38.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.041+0000 7f5b809cf700 1 -- 172.21.15.119:0/3146242674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b7c102080 msgr2=0x7f5b7c19fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:38.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.041+0000 7f5b809cf700 1 --2- 172.21.15.119:0/3146242674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b7c102080 0x7f5b7c19fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:38.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.041+0000 7f5b809cf700 1 -- 172.21.15.119:0/3146242674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b7c107730 msgr2=0x7f5b7c199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:38.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.041+0000 7f5b809cf700 1 --2- 172.21.15.119:0/3146242674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b7c107730 0x7f5b7c199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:38.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.041+0000 7f5b809cf700 1 -- 172.21.15.119:0/3146242674 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b700095f0 con 0x7f5b7c1016c0 2023-12-07T10:18:38.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.041+0000 7f5b809cf700 1 --2- 172.21.15.119:0/3146242674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b7c1016c0 0x7f5b7c19f900 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5b70005770 tx=0x7f5b700057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:38.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.042+0000 7f5b79ffb700 1 -- 172.21.15.119:0/3146242674 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7001d070 con 0x7f5b7c1016c0 2023-12-07T10:18:38.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.042+0000 7f5b79ffb700 1 -- 172.21.15.119:0/3146242674 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b70021410 con 0x7f5b7c1016c0 2023-12-07T10:18:38.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.042+0000 7f5b82c33700 1 -- 172.21.15.119:0/3146242674 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b7c19a650 con 0x7f5b7c1016c0 2023-12-07T10:18:38.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.042+0000 7f5b79ffb700 1 -- 172.21.15.119:0/3146242674 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b70017650 con 0x7f5b7c1016c0 2023-12-07T10:18:38.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.042+0000 7f5b82c33700 1 -- 172.21.15.119:0/3146242674 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b7c1a68f0 con 0x7f5b7c1016c0 2023-12-07T10:18:38.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.043+0000 7f5b82c33700 1 -- 172.21.15.119:0/3146242674 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5b7c1117d0 con 0x7f5b7c1016c0 2023-12-07T10:18:38.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.045+0000 7f5b79ffb700 1 -- 172.21.15.119:0/3146242674 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f5b700217f0 con 0x7f5b7c1016c0 2023-12-07T10:18:38.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.046+0000 7f5b79ffb700 1 --2- 172.21.15.119:0/3146242674 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5b6c0a1c20 0x7f5b6c0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:38.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.046+0000 7f5b79ffb700 1 -- 172.21.15.119:0/3146242674 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f5b700cecd0 con 0x7f5b7c1016c0 2023-12-07T10:18:38.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.046+0000 7f5b7bfff700 1 --2- 172.21.15.119:0/3146242674 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5b6c0a1c20 0x7f5b6c0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:38.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.046+0000 7f5b79ffb700 1 --2- 172.21.15.119:0/3146242674 >> [v2:172.21.15.153:6800/3390875370,v1:172.21.15.153:6801/3390875370] conn(0x7f5b6c0aca90 0x7f5b6c0aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:38.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.046+0000 7f5b79ffb700 1 -- 172.21.15.119:0/3146242674 --> [v2:172.21.15.153:6800/3390875370,v1:172.21.15.153:6801/3390875370] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5b6c0af600 con 0x7f5b6c0aca90 2023-12-07T10:18:38.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.046+0000 7f5b79ffb700 1 -- 172.21.15.119:0/3146242674 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b700d42a0 con 0x7f5b7c1016c0 2023-12-07T10:18:38.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.047+0000 7f5b811d0700 1 --2- 172.21.15.119:0/3146242674 >> [v2:172.21.15.153:6800/3390875370,v1:172.21.15.153:6801/3390875370] conn(0x7f5b6c0aca90 0x7f5b6c0aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:38.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.047+0000 7f5b7bfff700 1 --2- 172.21.15.119:0/3146242674 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5b6c0a1c20 0x7f5b6c0a40e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c19ac00 tx=0x7f5b68009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:38.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.048+0000 7f5b811d0700 1 --2- 172.21.15.119:0/3146242674 >> [v2:172.21.15.153:6800/3390875370,v1:172.21.15.153:6801/3390875370] conn(0x7f5b6c0aca90 0x7f5b6c0aef30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:38.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.049+0000 7f5b79ffb700 1 -- 172.21.15.119:0/3146242674 <== osd.1 v2:172.21.15.153:6800/3390875370 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f5b6c0af600 con 0x7f5b6c0aca90 2023-12-07T10:18:38.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.066+0000 7f5b82c33700 1 -- 172.21.15.119:0/3146242674 --> [v2:172.21.15.153:6800/3390875370,v1:172.21.15.153:6801/3390875370] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5b7c061d10 con 0x7f5b6c0aca90 2023-12-07T10:18:38.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.067+0000 7f5b79ffb700 1 -- 172.21.15.119:0/3146242674 <== osd.1 v2:172.21.15.153:6800/3390875370 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5b7c061d10 con 0x7f5b6c0aca90 2023-12-07T10:18:38.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.067+0000 7f5b82c33700 1 -- 172.21.15.119:0/3146242674 >> [v2:172.21.15.153:6800/3390875370,v1:172.21.15.153:6801/3390875370] conn(0x7f5b6c0aca90 msgr2=0x7f5b6c0aef30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:38.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.067+0000 7f5b82c33700 1 --2- 172.21.15.119:0/3146242674 >> [v2:172.21.15.153:6800/3390875370,v1:172.21.15.153:6801/3390875370] conn(0x7f5b6c0aca90 0x7f5b6c0aef30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:38.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.067+0000 7f5b82c33700 1 -- 172.21.15.119:0/3146242674 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5b6c0a1c20 msgr2=0x7f5b6c0a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:38.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.067+0000 7f5b82c33700 1 --2- 172.21.15.119:0/3146242674 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5b6c0a1c20 0x7f5b6c0a40e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c19ac00 tx=0x7f5b68009260 comp rx=0 tx=0).stop 2023-12-07T10:18:38.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.067+0000 7f5b82c33700 1 -- 172.21.15.119:0/3146242674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b7c1016c0 msgr2=0x7f5b7c19f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:38.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.067+0000 7f5b82c33700 1 --2- 172.21.15.119:0/3146242674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b7c1016c0 0x7f5b7c19f900 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5b70005770 tx=0x7f5b700057a0 comp rx=0 tx=0).stop 2023-12-07T10:18:38.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.067+0000 7f5b811d0700 1 -- 172.21.15.119:0/3146242674 reap_dead start 2023-12-07T10:18:38.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.067+0000 7f5b82c33700 1 -- 172.21.15.119:0/3146242674 shutdown_connections 2023-12-07T10:18:38.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.067+0000 7f5b82c33700 1 -- 172.21.15.119:0/3146242674 >> 172.21.15.119:0/3146242674 conn(0x7f5b7c078800 msgr2=0x7f5b7c10cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:38.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.067+0000 7f5b82c33700 1 -- 172.21.15.119:0/3146242674 shutdown_connections 2023-12-07T10:18:38.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.067+0000 7f5b82c33700 1 -- 172.21.15.119:0/3146242674 wait complete. 2023-12-07T10:18:39.021 INFO:teuthology.orchestra.run.smithi119.stdout:34359738441 2023-12-07T10:18:39.022 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.0 2023-12-07T10:18:39.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.927+0000 7fa535f1c700 1 -- 172.21.15.119:0/2118090170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5301065a0 msgr2=0x7fa530106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:39.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.927+0000 7fa535f1c700 1 --2- 172.21.15.119:0/2118090170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5301065a0 0x7fa530106980 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa5200099c0 tx=0x7fa520009cd0 comp rx=0 tx=0).stop 2023-12-07T10:18:39.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.927+0000 7fa535f1c700 1 -- 172.21.15.119:0/2118090170 shutdown_connections 2023-12-07T10:18:39.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.927+0000 7fa535f1c700 1 --2- 172.21.15.119:0/2118090170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa530100f30 0x7fa53010d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:39.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.927+0000 7fa535f1c700 1 --2- 172.21.15.119:0/2118090170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5301065a0 0x7fa530106980 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:39.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.927+0000 7fa535f1c700 1 --2- 172.21.15.119:0/2118090170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa530100570 0x7fa5301009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:39.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.927+0000 7fa535f1c700 1 -- 172.21.15.119:0/2118090170 >> 172.21.15.119:0/2118090170 conn(0x7fa5300765b0 msgr2=0x7fa5300769c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:39.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.927+0000 7fa535f1c700 1 -- 172.21.15.119:0/2118090170 shutdown_connections 2023-12-07T10:18:39.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.928+0000 7fa535f1c700 1 -- 172.21.15.119:0/2118090170 wait complete. 2023-12-07T10:18:39.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.929+0000 7fa535f1c700 1 Processor -- start 2023-12-07T10:18:39.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.929+0000 7fa535f1c700 1 -- start start 2023-12-07T10:18:39.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.930+0000 7fa535f1c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa530100570 0x7fa530197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:39.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.930+0000 7fa535f1c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa530100f30 0x7fa530197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:39.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.930+0000 7fa535f1c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5301065a0 0x7fa53019c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:39.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.930+0000 7fa535f1c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53019c8a0 con 0x7fa530100f30 2023-12-07T10:18:39.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.930+0000 7fa535f1c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53019ca10 con 0x7fa5301065a0 2023-12-07T10:18:39.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.930+0000 7fa535f1c700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53019cbd0 con 0x7fa530100570 2023-12-07T10:18:39.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.930+0000 7fa52f7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa530100570 0x7fa530197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:39.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.930+0000 7fa52effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa530100f30 0x7fa530197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:39.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.930+0000 7fa52f7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa530100570 0x7fa530197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:36498/0 (socket says 172.21.15.119:36498) 2023-12-07T10:18:39.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.930+0000 7fa52f7fe700 1 -- 172.21.15.119:0/111122512 learned_addr learned my addr 172.21.15.119:0/111122512 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:39.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.930+0000 7fa52ffff700 1 --2- 172.21.15.119:0/111122512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5301065a0 0x7fa53019c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:39.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.931+0000 7fa52f7fe700 1 -- 172.21.15.119:0/111122512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5301065a0 msgr2=0x7fa53019c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:39.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.931+0000 7fa52f7fe700 1 --2- 172.21.15.119:0/111122512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5301065a0 0x7fa53019c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:39.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.931+0000 7fa52f7fe700 1 -- 172.21.15.119:0/111122512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa530100f30 msgr2=0x7fa530197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:39.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.931+0000 7fa52f7fe700 1 --2- 172.21.15.119:0/111122512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa530100f30 0x7fa530197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:39.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.931+0000 7fa52f7fe700 1 -- 172.21.15.119:0/111122512 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5200095f0 con 0x7fa530100570 2023-12-07T10:18:39.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.931+0000 7fa52effd700 1 --2- 172.21.15.119:0/111122512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa530100f30 0x7fa530197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:18:39.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.931+0000 7fa52f7fe700 1 --2- 172.21.15.119:0/111122512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa530100570 0x7fa530197a60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa52000b580 tx=0x7fa52000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:39.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.932+0000 7fa52cff9700 1 -- 172.21.15.119:0/111122512 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52001c070 con 0x7fa530100570 2023-12-07T10:18:39.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.932+0000 7fa535f1c700 1 -- 172.21.15.119:0/111122512 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa53019ce50 con 0x7fa530100570 2023-12-07T10:18:39.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.932+0000 7fa52cff9700 1 -- 172.21.15.119:0/111122512 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa52000fd80 con 0x7fa530100570 2023-12-07T10:18:39.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.932+0000 7fa52cff9700 1 -- 172.21.15.119:0/111122512 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa520022c40 con 0x7fa530100570 2023-12-07T10:18:39.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.932+0000 7fa535f1c700 1 -- 172.21.15.119:0/111122512 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5301a4730 con 0x7fa530100570 2023-12-07T10:18:39.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.933+0000 7fa535f1c700 1 -- 172.21.15.119:0/111122512 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa530110720 con 0x7fa530100570 2023-12-07T10:18:39.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.935+0000 7fa52cff9700 1 -- 172.21.15.119:0/111122512 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7fa520020070 con 0x7fa530100570 2023-12-07T10:18:39.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.937+0000 7fa52cff9700 1 --2- 172.21.15.119:0/111122512 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa5180a1c20 0x7fa5180a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:39.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.937+0000 7fa52cff9700 1 -- 172.21.15.119:0/111122512 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fa5200ce780 con 0x7fa530100570 2023-12-07T10:18:39.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.937+0000 7fa52effd700 1 --2- 172.21.15.119:0/111122512 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa5180a1c20 0x7fa5180a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:39.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.937+0000 7fa52cff9700 1 --2- 172.21.15.119:0/111122512 >> [v2:172.21.15.153:6824/663834864,v1:172.21.15.153:6825/663834864] conn(0x7fa5180aca10 0x7fa5180aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:39.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.937+0000 7fa52cff9700 1 -- 172.21.15.119:0/111122512 --> [v2:172.21.15.153:6824/663834864,v1:172.21.15.153:6825/663834864] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa5180af580 con 0x7fa5180aca10 2023-12-07T10:18:39.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.937+0000 7fa52cff9700 1 -- 172.21.15.119:0/111122512 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5200d0d60 con 0x7fa530100570 2023-12-07T10:18:39.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.937+0000 7fa52ffff700 1 --2- 172.21.15.119:0/111122512 >> [v2:172.21.15.153:6824/663834864,v1:172.21.15.153:6825/663834864] conn(0x7fa5180aca10 0x7fa5180aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:39.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.937+0000 7fa52effd700 1 --2- 172.21.15.119:0/111122512 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa5180a1c20 0x7fa5180a40e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa530076ab0 tx=0x7fa524009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:39.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.938+0000 7fa52ffff700 1 --2- 172.21.15.119:0/111122512 >> [v2:172.21.15.153:6824/663834864,v1:172.21.15.153:6825/663834864] conn(0x7fa5180aca10 0x7fa5180aeeb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:39.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.939+0000 7fa52cff9700 1 -- 172.21.15.119:0/111122512 <== osd.10 v2:172.21.15.153:6824/663834864 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fa5180af580 con 0x7fa5180aca10 2023-12-07T10:18:39.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.956+0000 7fa535f1c700 1 -- 172.21.15.119:0/111122512 --> [v2:172.21.15.153:6824/663834864,v1:172.21.15.153:6825/663834864] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa5301044e0 con 0x7fa5180aca10 2023-12-07T10:18:39.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.957+0000 7fa52cff9700 1 -- 172.21.15.119:0/111122512 <== osd.10 v2:172.21.15.153:6824/663834864 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa5301044e0 con 0x7fa5180aca10 2023-12-07T10:18:39.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.957+0000 7fa535f1c700 1 -- 172.21.15.119:0/111122512 >> [v2:172.21.15.153:6824/663834864,v1:172.21.15.153:6825/663834864] conn(0x7fa5180aca10 msgr2=0x7fa5180aeeb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:39.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.957+0000 7fa535f1c700 1 --2- 172.21.15.119:0/111122512 >> [v2:172.21.15.153:6824/663834864,v1:172.21.15.153:6825/663834864] conn(0x7fa5180aca10 0x7fa5180aeeb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:39.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.957+0000 7fa535f1c700 1 -- 172.21.15.119:0/111122512 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa5180a1c20 msgr2=0x7fa5180a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:39.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.957+0000 7fa535f1c700 1 --2- 172.21.15.119:0/111122512 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa5180a1c20 0x7fa5180a40e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa530076ab0 tx=0x7fa524009260 comp rx=0 tx=0).stop 2023-12-07T10:18:39.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.957+0000 7fa535f1c700 1 -- 172.21.15.119:0/111122512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa530100570 msgr2=0x7fa530197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:39.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.957+0000 7fa535f1c700 1 --2- 172.21.15.119:0/111122512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa530100570 0x7fa530197a60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa52000b580 tx=0x7fa52000f690 comp rx=0 tx=0).stop 2023-12-07T10:18:39.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.957+0000 7fa52ffff700 1 -- 172.21.15.119:0/111122512 reap_dead start 2023-12-07T10:18:39.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.957+0000 7fa535f1c700 1 -- 172.21.15.119:0/111122512 shutdown_connections 2023-12-07T10:18:39.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.957+0000 7fa535f1c700 1 -- 172.21.15.119:0/111122512 >> 172.21.15.119:0/111122512 conn(0x7fa5300765b0 msgr2=0x7fa5300ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:39.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.957+0000 7fa535f1c700 1 -- 172.21.15.119:0/111122512 shutdown_connections 2023-12-07T10:18:39.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:38.957+0000 7fa535f1c700 1 -- 172.21.15.119:0/111122512 wait complete. 2023-12-07T10:18:39.790 INFO:teuthology.orchestra.run.smithi119.stdout:111669149756 2023-12-07T10:18:39.790 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.3 2023-12-07T10:18:39.791 INFO:teuthology.orchestra.run.smithi119.stdout:55834574918 2023-12-07T10:18:39.791 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.1 2023-12-07T10:18:39.896 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:39 smithi119 ceph-mon[97908]: pgmap v310: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2023-12-07T10:18:40.153 INFO:teuthology.orchestra.run.smithi119.stdout:253403070501 2023-12-07T10:18:40.153 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.8 2023-12-07T10:18:40.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:39 smithi159 ceph-mon[134726]: pgmap v310: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2023-12-07T10:18:40.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:39 smithi153 ceph-mon[133625]: pgmap v310: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2023-12-07T10:18:40.405 INFO:teuthology.orchestra.run.smithi119.stdout:77309411394 2023-12-07T10:18:40.406 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.2 2023-12-07T10:18:40.565 INFO:teuthology.orchestra.run.smithi119.stdout:309237645339 2023-12-07T10:18:40.565 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.10 2023-12-07T10:18:40.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.835+0000 7f32bdbd7700 1 -- 172.21.15.119:0/2604714382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b810fdb0 msgr2=0x7f32b8108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:40.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.835+0000 7f32bdbd7700 1 --2- 172.21.15.119:0/2604714382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b810fdb0 0x7f32b8108610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009990 tx=0x7f32b0009ca0 comp rx=0 tx=0).stop 2023-12-07T10:18:40.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.836+0000 7f32bdbd7700 1 -- 172.21.15.119:0/2604714382 shutdown_connections 2023-12-07T10:18:40.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.836+0000 7f32bdbd7700 1 --2- 172.21.15.119:0/2604714382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32b811adf0 0x7f32b811d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:40.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.836+0000 7f32bdbd7700 1 --2- 172.21.15.119:0/2604714382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b810fdb0 0x7f32b8108610 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:40.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.836+0000 7f32bdbd7700 1 --2- 172.21.15.119:0/2604714382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32b8108b50 0x7f32b8108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:40.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.836+0000 7f32bdbd7700 1 -- 172.21.15.119:0/2604714382 >> 172.21.15.119:0/2604714382 conn(0x7f32b806c6e0 msgr2=0x7f32b806caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:40.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.836+0000 7f32bdbd7700 1 -- 172.21.15.119:0/2604714382 shutdown_connections 2023-12-07T10:18:40.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.836+0000 7f32bdbd7700 1 -- 172.21.15.119:0/2604714382 wait complete. 2023-12-07T10:18:40.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.837+0000 7f32bdbd7700 1 Processor -- start 2023-12-07T10:18:40.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.837+0000 7f32bdbd7700 1 -- start start 2023-12-07T10:18:40.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.838+0000 7f32bdbd7700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32b8108b50 0x7f32b81ac640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:40.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.838+0000 7f32bdbd7700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32b810fdb0 0x7f32b81acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:40.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.838+0000 7f32bdbd7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b811adf0 0x7f32b81a67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:40.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.838+0000 7f32bdbd7700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b81a6e50 con 0x7f32b810fdb0 2023-12-07T10:18:40.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.838+0000 7f32bdbd7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b81a6fc0 con 0x7f32b811adf0 2023-12-07T10:18:40.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.838+0000 7f32bdbd7700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b81a7180 con 0x7f32b8108b50 2023-12-07T10:18:40.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.838+0000 7f32b6ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32b810fdb0 0x7f32b81acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:40.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.839+0000 7f32b6ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32b810fdb0 0x7f32b81acb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44172/0 (socket says 172.21.15.119:44172) 2023-12-07T10:18:40.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.839+0000 7f32b6ffd700 1 -- 172.21.15.119:0/651793339 learned_addr learned my addr 172.21.15.119:0/651793339 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:40.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.839+0000 7f32b6ffd700 1 -- 172.21.15.119:0/651793339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32b8108b50 msgr2=0x7f32b81ac640 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-07T10:18:40.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.839+0000 7f32b6ffd700 1 --2- 172.21.15.119:0/651793339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32b8108b50 0x7f32b81ac640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:40.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.839+0000 7f32b6ffd700 1 -- 172.21.15.119:0/651793339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b811adf0 msgr2=0x7f32b81a67a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-07T10:18:40.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.839+0000 7f32b6ffd700 1 --2- 172.21.15.119:0/651793339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b811adf0 0x7f32b81a67a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:40.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.839+0000 7f32b6ffd700 1 -- 172.21.15.119:0/651793339 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b00095f0 con 0x7f32b810fdb0 2023-12-07T10:18:40.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.840+0000 7f32b6ffd700 1 --2- 172.21.15.119:0/651793339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32b810fdb0 0x7f32b81acb80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f32a000ed60 tx=0x7f32a000c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:40.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.840+0000 7f32b4ff9700 1 -- 172.21.15.119:0/651793339 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a0009790 con 0x7f32b810fdb0 2023-12-07T10:18:40.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.840+0000 7f32bdbd7700 1 -- 172.21.15.119:0/651793339 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b81a7460 con 0x7f32b810fdb0 2023-12-07T10:18:40.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.840+0000 7f32bdbd7700 1 -- 172.21.15.119:0/651793339 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b81b3670 con 0x7f32b810fdb0 2023-12-07T10:18:40.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.841+0000 7f32b4ff9700 1 -- 172.21.15.119:0/651793339 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32a000cbc0 con 0x7f32b810fdb0 2023-12-07T10:18:40.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.841+0000 7f32b4ff9700 1 -- 172.21.15.119:0/651793339 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a0009790 con 0x7f32b810fdb0 2023-12-07T10:18:40.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.847+0000 7f32bdbd7700 1 -- 172.21.15.119:0/651793339 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f32b8120740 con 0x7f32b810fdb0 2023-12-07T10:18:40.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.848+0000 7f32b4ff9700 1 -- 172.21.15.119:0/651793339 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f32a0010430 con 0x7f32b810fdb0 2023-12-07T10:18:40.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.851+0000 7f32b4ff9700 1 --2- 172.21.15.119:0/651793339 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f32a40a1cd0 0x7f32a40a4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:40.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.852+0000 7f32b4ff9700 1 -- 172.21.15.119:0/651793339 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f32a0014070 con 0x7f32b810fdb0 2023-12-07T10:18:40.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.852+0000 7f32b4ff9700 1 --2- 172.21.15.119:0/651793339 >> [v2:172.21.15.119:6818/1162436840,v1:172.21.15.119:6819/1162436840] conn(0x7f32a40acc70 0x7f32a40af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:40.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.853+0000 7f32b4ff9700 1 -- 172.21.15.119:0/651793339 --> [v2:172.21.15.119:6818/1162436840,v1:172.21.15.119:6819/1162436840] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f32a40af7e0 con 0x7f32a40acc70 2023-12-07T10:18:40.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.853+0000 7f32b4ff9700 1 -- 172.21.15.119:0/651793339 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32a00d22c0 con 0x7f32b810fdb0 2023-12-07T10:18:40.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.853+0000 7f32b77fe700 1 --2- 172.21.15.119:0/651793339 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f32a40a1cd0 0x7f32a40a4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:40.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.854+0000 7f32b77fe700 1 --2- 172.21.15.119:0/651793339 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f32a40a1cd0 0x7f32a40a4190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f32b0006010 tx=0x7f32b000b610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:40.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.856+0000 7f32b7fff700 1 --2- 172.21.15.119:0/651793339 >> [v2:172.21.15.119:6818/1162436840,v1:172.21.15.119:6819/1162436840] conn(0x7f32a40acc70 0x7f32a40af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:40.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.857+0000 7f32b7fff700 1 --2- 172.21.15.119:0/651793339 >> [v2:172.21.15.119:6818/1162436840,v1:172.21.15.119:6819/1162436840] conn(0x7f32a40acc70 0x7f32a40af110 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:40.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.858+0000 7f32b4ff9700 1 -- 172.21.15.119:0/651793339 <== osd.6 v2:172.21.15.119:6818/1162436840 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f32a40af7e0 con 0x7f32a40acc70 2023-12-07T10:18:40.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.887+0000 7f32bdbd7700 1 -- 172.21.15.119:0/651793339 --> [v2:172.21.15.119:6818/1162436840,v1:172.21.15.119:6819/1162436840] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f32b8119730 con 0x7f32a40acc70 2023-12-07T10:18:40.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.888+0000 7f32b4ff9700 1 -- 172.21.15.119:0/651793339 <== osd.6 v2:172.21.15.119:6818/1162436840 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f32b8119730 con 0x7f32a40acc70 2023-12-07T10:18:40.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.888+0000 7f32bdbd7700 1 -- 172.21.15.119:0/651793339 >> [v2:172.21.15.119:6818/1162436840,v1:172.21.15.119:6819/1162436840] conn(0x7f32a40acc70 msgr2=0x7f32a40af110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:40.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.888+0000 7f32bdbd7700 1 --2- 172.21.15.119:0/651793339 >> [v2:172.21.15.119:6818/1162436840,v1:172.21.15.119:6819/1162436840] conn(0x7f32a40acc70 0x7f32a40af110 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:40.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.888+0000 7f32bdbd7700 1 -- 172.21.15.119:0/651793339 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f32a40a1cd0 msgr2=0x7f32a40a4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:40.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.888+0000 7f32bdbd7700 1 --2- 172.21.15.119:0/651793339 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f32a40a1cd0 0x7f32a40a4190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f32b0006010 tx=0x7f32b000b610 comp rx=0 tx=0).stop 2023-12-07T10:18:40.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.888+0000 7f32bdbd7700 1 -- 172.21.15.119:0/651793339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32b810fdb0 msgr2=0x7f32b81acb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:40.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.888+0000 7f32bdbd7700 1 --2- 172.21.15.119:0/651793339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32b810fdb0 0x7f32b81acb80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f32a000ed60 tx=0x7f32a000c5b0 comp rx=0 tx=0).stop 2023-12-07T10:18:40.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.889+0000 7f32b7fff700 1 -- 172.21.15.119:0/651793339 reap_dead start 2023-12-07T10:18:40.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.889+0000 7f32bdbd7700 1 -- 172.21.15.119:0/651793339 shutdown_connections 2023-12-07T10:18:40.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.889+0000 7f32bdbd7700 1 -- 172.21.15.119:0/651793339 >> 172.21.15.119:0/651793339 conn(0x7f32b806c6e0 msgr2=0x7f32b8118de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:40.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.889+0000 7f32bdbd7700 1 -- 172.21.15.119:0/651793339 shutdown_connections 2023-12-07T10:18:40.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.889+0000 7f32bdbd7700 1 -- 172.21.15.119:0/651793339 wait complete. 2023-12-07T10:18:40.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.922+0000 7f66dc5d4700 1 -- 172.21.15.119:0/348324988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66d40698d0 msgr2=0x7f66d410dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:40.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.922+0000 7f66dc5d4700 1 --2- 172.21.15.119:0/348324988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66d40698d0 0x7f66d410dbe0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f66c40098f0 tx=0x7f66c4009c00 comp rx=0 tx=0).stop 2023-12-07T10:18:40.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.922+0000 7f66dc5d4700 1 -- 172.21.15.119:0/348324988 shutdown_connections 2023-12-07T10:18:40.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.922+0000 7f66dc5d4700 1 --2- 172.21.15.119:0/348324988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66d410e120 0x7f66d4110510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:40.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.922+0000 7f66dc5d4700 1 --2- 172.21.15.119:0/348324988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66d4068fb0 0x7f66d4069390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:40.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.922+0000 7f66dc5d4700 1 --2- 172.21.15.119:0/348324988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66d40698d0 0x7f66d410dbe0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:40.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.922+0000 7f66dc5d4700 1 -- 172.21.15.119:0/348324988 >> 172.21.15.119:0/348324988 conn(0x7f66d40765b0 msgr2=0x7f66d40769c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:40.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.922+0000 7f66dc5d4700 1 -- 172.21.15.119:0/348324988 shutdown_connections 2023-12-07T10:18:40.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.922+0000 7f66dc5d4700 1 -- 172.21.15.119:0/348324988 wait complete. 2023-12-07T10:18:40.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.924+0000 7f66dc5d4700 1 Processor -- start 2023-12-07T10:18:40.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.924+0000 7f66dc5d4700 1 -- start start 2023-12-07T10:18:40.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.925+0000 7f66dc5d4700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66d4068fb0 0x7f66d419f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:40.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.925+0000 7f66dc5d4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66d40698d0 0x7f66d419ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:40.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.925+0000 7f66dc5d4700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66d410e120 0x7f66d4199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:40.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.925+0000 7f66dc5d4700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d419a180 con 0x7f66d4068fb0 2023-12-07T10:18:40.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.925+0000 7f66dc5d4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d419a2f0 con 0x7f66d40698d0 2023-12-07T10:18:40.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.925+0000 7f66dc5d4700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d419a4b0 con 0x7f66d410e120 2023-12-07T10:18:40.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.925+0000 7f66d9b6f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66d40698d0 0x7f66d419ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:40.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.925+0000 7f66d9b6f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66d40698d0 0x7f66d419ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:35334/0 (socket says 172.21.15.119:35334) 2023-12-07T10:18:40.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.925+0000 7f66d9b6f700 1 -- 172.21.15.119:0/2690129475 learned_addr learned my addr 172.21.15.119:0/2690129475 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:40.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.925+0000 7f66da370700 1 --2- 172.21.15.119:0/2690129475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66d4068fb0 0x7f66d419f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:40.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.925+0000 7f66dab71700 1 --2- 172.21.15.119:0/2690129475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66d410e120 0x7f66d4199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:40.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.926+0000 7f66d9b6f700 1 -- 172.21.15.119:0/2690129475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66d410e120 msgr2=0x7f66d4199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:40.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.926+0000 7f66d9b6f700 1 --2- 172.21.15.119:0/2690129475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66d410e120 0x7f66d4199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:40.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.926+0000 7f66d9b6f700 1 -- 172.21.15.119:0/2690129475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66d4068fb0 msgr2=0x7f66d419f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:40.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.926+0000 7f66d9b6f700 1 --2- 172.21.15.119:0/2690129475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66d4068fb0 0x7f66d419f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:40.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.926+0000 7f66d9b6f700 1 -- 172.21.15.119:0/2690129475 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c4009520 con 0x7f66d40698d0 2023-12-07T10:18:40.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.926+0000 7f66dab71700 1 --2- 172.21.15.119:0/2690129475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66d410e120 0x7f66d4199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:18:40.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.926+0000 7f66d9b6f700 1 --2- 172.21.15.119:0/2690129475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66d40698d0 0x7f66d419ff20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f66c4005950 tx=0x7f66c4010ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:40.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.926+0000 7f66cb7fe700 1 -- 172.21.15.119:0/2690129475 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c401c070 con 0x7f66d40698d0 2023-12-07T10:18:40.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.927+0000 7f66cb7fe700 1 -- 172.21.15.119:0/2690129475 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66c400bba0 con 0x7f66d40698d0 2023-12-07T10:18:40.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.927+0000 7f66dc5d4700 1 -- 172.21.15.119:0/2690129475 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d419a730 con 0x7f66d40698d0 2023-12-07T10:18:40.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.927+0000 7f66cb7fe700 1 -- 172.21.15.119:0/2690129475 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c40039c0 con 0x7f66d40698d0 2023-12-07T10:18:40.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.927+0000 7f66dc5d4700 1 -- 172.21.15.119:0/2690129475 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d41a69d0 con 0x7f66d40698d0 2023-12-07T10:18:40.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.928+0000 7f66dc5d4700 1 -- 172.21.15.119:0/2690129475 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f66d4113a60 con 0x7f66d40698d0 2023-12-07T10:18:40.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.930+0000 7f66cb7fe700 1 -- 172.21.15.119:0/2690129475 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f66c4020030 con 0x7f66d40698d0 2023-12-07T10:18:40.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.931+0000 7f66cb7fe700 1 --2- 172.21.15.119:0/2690129475 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f66c00a1ba0 0x7f66c00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:40.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.932+0000 7f66cb7fe700 1 -- 172.21.15.119:0/2690129475 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f66c40ce9b0 con 0x7f66d40698d0 2023-12-07T10:18:40.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.932+0000 7f66da370700 1 --2- 172.21.15.119:0/2690129475 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f66c00a1ba0 0x7f66c00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:40.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.932+0000 7f66cb7fe700 1 --2- 172.21.15.119:0/2690129475 >> [v2:172.21.15.159:6824/1262216348,v1:172.21.15.159:6825/1262216348] conn(0x7f66c00acb40 0x7f66c00aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:40.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.932+0000 7f66cb7fe700 1 -- 172.21.15.119:0/2690129475 --> [v2:172.21.15.159:6824/1262216348,v1:172.21.15.159:6825/1262216348] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f66c00af6b0 con 0x7f66c00acb40 2023-12-07T10:18:40.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.932+0000 7f66cb7fe700 1 -- 172.21.15.119:0/2690129475 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66c40d32a0 con 0x7f66d40698d0 2023-12-07T10:18:40.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.933+0000 7f66dab71700 1 --2- 172.21.15.119:0/2690129475 >> [v2:172.21.15.159:6824/1262216348,v1:172.21.15.159:6825/1262216348] conn(0x7f66c00acb40 0x7f66c00aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:40.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.933+0000 7f66da370700 1 --2- 172.21.15.119:0/2690129475 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f66c00a1ba0 0x7f66c00a4060 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f66d0005fd0 tx=0x7f66d0009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:40.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.933+0000 7f66dab71700 1 --2- 172.21.15.119:0/2690129475 >> [v2:172.21.15.159:6824/1262216348,v1:172.21.15.159:6825/1262216348] conn(0x7f66c00acb40 0x7f66c00aefe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:40.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.934+0000 7f66cb7fe700 1 -- 172.21.15.119:0/2690129475 <== osd.11 v2:172.21.15.159:6824/1262216348 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f66c00af6b0 con 0x7f66c00acb40 2023-12-07T10:18:40.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.957+0000 7f66dc5d4700 1 -- 172.21.15.119:0/2690129475 --> [v2:172.21.15.159:6824/1262216348,v1:172.21.15.159:6825/1262216348] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f66d4061d10 con 0x7f66c00acb40 2023-12-07T10:18:40.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.958+0000 7f66cb7fe700 1 -- 172.21.15.119:0/2690129475 <== osd.11 v2:172.21.15.159:6824/1262216348 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f66d4061d10 con 0x7f66c00acb40 2023-12-07T10:18:40.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.958+0000 7f66dc5d4700 1 -- 172.21.15.119:0/2690129475 >> [v2:172.21.15.159:6824/1262216348,v1:172.21.15.159:6825/1262216348] conn(0x7f66c00acb40 msgr2=0x7f66c00aefe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:40.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.958+0000 7f66dc5d4700 1 --2- 172.21.15.119:0/2690129475 >> [v2:172.21.15.159:6824/1262216348,v1:172.21.15.159:6825/1262216348] conn(0x7f66c00acb40 0x7f66c00aefe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:40.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.958+0000 7f66dc5d4700 1 -- 172.21.15.119:0/2690129475 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f66c00a1ba0 msgr2=0x7f66c00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:40.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.959+0000 7f66dc5d4700 1 --2- 172.21.15.119:0/2690129475 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f66c00a1ba0 0x7f66c00a4060 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f66d0005fd0 tx=0x7f66d0009310 comp rx=0 tx=0).stop 2023-12-07T10:18:40.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.959+0000 7f66dc5d4700 1 -- 172.21.15.119:0/2690129475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66d40698d0 msgr2=0x7f66d419ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:40.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.959+0000 7f66dc5d4700 1 --2- 172.21.15.119:0/2690129475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66d40698d0 0x7f66d419ff20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f66c4005950 tx=0x7f66c4010ff0 comp rx=0 tx=0).stop 2023-12-07T10:18:40.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.959+0000 7f66dab71700 1 -- 172.21.15.119:0/2690129475 reap_dead start 2023-12-07T10:18:40.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.959+0000 7f66dc5d4700 1 -- 172.21.15.119:0/2690129475 shutdown_connections 2023-12-07T10:18:40.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.959+0000 7f66dc5d4700 1 -- 172.21.15.119:0/2690129475 >> 172.21.15.119:0/2690129475 conn(0x7f66d40765b0 msgr2=0x7f66d410f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:40.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.959+0000 7f66dc5d4700 1 -- 172.21.15.119:0/2690129475 shutdown_connections 2023-12-07T10:18:40.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.959+0000 7f66dc5d4700 1 -- 172.21.15.119:0/2690129475 wait complete. 2023-12-07T10:18:41.385 INFO:teuthology.orchestra.run.smithi119.stdout:287762808864 2023-12-07T10:18:41.386 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.9 2023-12-07T10:18:41.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.792+0000 7fef8efbe700 1 -- 172.21.15.119:0/4272545226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef88068fb0 msgr2=0x7fef88069390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:41.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.792+0000 7fef8efbe700 1 --2- 172.21.15.119:0/4272545226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef88068fb0 0x7fef88069390 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fef740098f0 tx=0x7fef74009c00 comp rx=0 tx=0).stop 2023-12-07T10:18:41.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.792+0000 7fef8efbe700 1 -- 172.21.15.119:0/4272545226 shutdown_connections 2023-12-07T10:18:41.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.792+0000 7fef8efbe700 1 --2- 172.21.15.119:0/4272545226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef8810e120 0x7fef88110510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:41.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.792+0000 7fef8efbe700 1 --2- 172.21.15.119:0/4272545226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef88068fb0 0x7fef88069390 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:41.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.792+0000 7fef8efbe700 1 --2- 172.21.15.119:0/4272545226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef880698d0 0x7fef8810dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:41.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.792+0000 7fef8efbe700 1 -- 172.21.15.119:0/4272545226 >> 172.21.15.119:0/4272545226 conn(0x7fef880765b0 msgr2=0x7fef880769c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:41.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.792+0000 7fef8efbe700 1 -- 172.21.15.119:0/4272545226 shutdown_connections 2023-12-07T10:18:41.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.793+0000 7fef8efbe700 1 -- 172.21.15.119:0/4272545226 wait complete. 2023-12-07T10:18:41.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.794+0000 7fef8efbe700 1 Processor -- start 2023-12-07T10:18:41.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.794+0000 7fef8efbe700 1 -- start start 2023-12-07T10:18:41.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.795+0000 7fef8efbe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef88068fb0 0x7fef8819f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:41.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.795+0000 7fef8efbe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef880698d0 0x7fef8819ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:41.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.795+0000 7fef8efbe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef8810e120 0x7fef88199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:41.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.795+0000 7fef8efbe700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8819a190 con 0x7fef8810e120 2023-12-07T10:18:41.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.795+0000 7fef8efbe700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8819a300 con 0x7fef880698d0 2023-12-07T10:18:41.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.795+0000 7fef8efbe700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8819a4c0 con 0x7fef88068fb0 2023-12-07T10:18:41.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.796+0000 7fef87fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef880698d0 0x7fef8819ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:41.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.796+0000 7fef87fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef880698d0 0x7fef8819ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:35304/0 (socket says 172.21.15.119:35304) 2023-12-07T10:18:41.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.796+0000 7fef87fff700 1 -- 172.21.15.119:0/489683025 learned_addr learned my addr 172.21.15.119:0/489683025 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:41.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.796+0000 7fef8cd5a700 1 --2- 172.21.15.119:0/489683025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef88068fb0 0x7fef8819f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:41.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.796+0000 7fef87fff700 1 -- 172.21.15.119:0/489683025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef88068fb0 msgr2=0x7fef8819f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:41.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.796+0000 7fef87fff700 1 --2- 172.21.15.119:0/489683025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef88068fb0 0x7fef8819f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:41.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.796+0000 7fef87fff700 1 -- 172.21.15.119:0/489683025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef8810e120 msgr2=0x7fef88199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:41.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.796+0000 7fef8d55b700 1 --2- 172.21.15.119:0/489683025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef8810e120 0x7fef88199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:41.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.796+0000 7fef87fff700 1 --2- 172.21.15.119:0/489683025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef8810e120 0x7fef88199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:41.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.797+0000 7fef87fff700 1 -- 172.21.15.119:0/489683025 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef80009d00 con 0x7fef880698d0 2023-12-07T10:18:41.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.797+0000 7fef8d55b700 1 --2- 172.21.15.119:0/489683025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef8810e120 0x7fef88199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:18:41.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.797+0000 7fef87fff700 1 --2- 172.21.15.119:0/489683025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef880698d0 0x7fef8819ff30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00e9a0 tx=0x7fef7c00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:41.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.797+0000 7fef85ffb700 1 -- 172.21.15.119:0/489683025 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c00cbd0 con 0x7fef880698d0 2023-12-07T10:18:41.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.797+0000 7fef8efbe700 1 -- 172.21.15.119:0/489683025 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef74009520 con 0x7fef880698d0 2023-12-07T10:18:41.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.797+0000 7fef8efbe700 1 -- 172.21.15.119:0/489683025 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8819a990 con 0x7fef880698d0 2023-12-07T10:18:41.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.798+0000 7fef8efbe700 1 -- 172.21.15.119:0/489683025 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fef88113a50 con 0x7fef880698d0 2023-12-07T10:18:41.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.799+0000 7fef85ffb700 1 -- 172.21.15.119:0/489683025 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef7c00cd30 con 0x7fef880698d0 2023-12-07T10:18:41.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.799+0000 7fef85ffb700 1 -- 172.21.15.119:0/489683025 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c018710 con 0x7fef880698d0 2023-12-07T10:18:41.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.802+0000 7fef85ffb700 1 -- 172.21.15.119:0/489683025 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7fef7c010980 con 0x7fef880698d0 2023-12-07T10:18:41.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.803+0000 7fef85ffb700 1 --2- 172.21.15.119:0/489683025 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fef780a1ca0 0x7fef780a4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:41.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.803+0000 7fef85ffb700 1 -- 172.21.15.119:0/489683025 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fef7c014070 con 0x7fef880698d0 2023-12-07T10:18:41.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.804+0000 7fef85ffb700 1 --2- 172.21.15.119:0/489683025 >> [v2:172.21.15.153:6808/4021998299,v1:172.21.15.153:6809/4021998299] conn(0x7fef780acc40 0x7fef780af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:41.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.804+0000 7fef85ffb700 1 -- 172.21.15.119:0/489683025 --> [v2:172.21.15.153:6808/4021998299,v1:172.21.15.153:6809/4021998299] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef780af7b0 con 0x7fef780acc40 2023-12-07T10:18:41.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.804+0000 7fef85ffb700 1 -- 172.21.15.119:0/489683025 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef7c0ceaf0 con 0x7fef880698d0 2023-12-07T10:18:41.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.804+0000 7fef8cd5a700 1 --2- 172.21.15.119:0/489683025 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fef780a1ca0 0x7fef780a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:41.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.805+0000 7fef8cd5a700 1 --2- 172.21.15.119:0/489683025 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fef780a1ca0 0x7fef780a4160 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fef7400c010 tx=0x7fef74009f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:41.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.805+0000 7fef8d55b700 1 --2- 172.21.15.119:0/489683025 >> [v2:172.21.15.153:6808/4021998299,v1:172.21.15.153:6809/4021998299] conn(0x7fef780acc40 0x7fef780af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:41.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.806+0000 7fef8d55b700 1 --2- 172.21.15.119:0/489683025 >> [v2:172.21.15.153:6808/4021998299,v1:172.21.15.153:6809/4021998299] conn(0x7fef780acc40 0x7fef780af0e0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.808+0000 7fef85ffb700 1 -- 172.21.15.119:0/489683025 <== osd.4 v2:172.21.15.153:6808/4021998299 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fef780af7b0 con 0x7fef780acc40 2023-12-07T10:18:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.831+0000 7fef8efbe700 1 -- 172.21.15.119:0/489683025 --> [v2:172.21.15.153:6808/4021998299,v1:172.21.15.153:6809/4021998299] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fef88061d10 con 0x7fef780acc40 2023-12-07T10:18:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.844+0000 7fef85ffb700 1 -- 172.21.15.119:0/489683025 <== osd.4 v2:172.21.15.153:6808/4021998299 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fef88061d10 con 0x7fef780acc40 2023-12-07T10:18:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.845+0000 7fef8efbe700 1 -- 172.21.15.119:0/489683025 >> [v2:172.21.15.153:6808/4021998299,v1:172.21.15.153:6809/4021998299] conn(0x7fef780acc40 msgr2=0x7fef780af0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.845+0000 7fef8efbe700 1 --2- 172.21.15.119:0/489683025 >> [v2:172.21.15.153:6808/4021998299,v1:172.21.15.153:6809/4021998299] conn(0x7fef780acc40 0x7fef780af0e0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.845+0000 7fef8efbe700 1 -- 172.21.15.119:0/489683025 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fef780a1ca0 msgr2=0x7fef780a4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.845+0000 7fef8efbe700 1 --2- 172.21.15.119:0/489683025 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fef780a1ca0 0x7fef780a4160 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fef7400c010 tx=0x7fef74009f50 comp rx=0 tx=0).stop 2023-12-07T10:18:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.845+0000 7fef8efbe700 1 -- 172.21.15.119:0/489683025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef880698d0 msgr2=0x7fef8819ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.845+0000 7fef8efbe700 1 --2- 172.21.15.119:0/489683025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef880698d0 0x7fef8819ff30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00e9a0 tx=0x7fef7c00ed60 comp rx=0 tx=0).stop 2023-12-07T10:18:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.845+0000 7fef8d55b700 1 -- 172.21.15.119:0/489683025 reap_dead start 2023-12-07T10:18:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.846+0000 7fef8efbe700 1 -- 172.21.15.119:0/489683025 shutdown_connections 2023-12-07T10:18:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.846+0000 7fef8efbe700 1 -- 172.21.15.119:0/489683025 >> 172.21.15.119:0/489683025 conn(0x7fef880765b0 msgr2=0x7fef8810f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.847+0000 7fef8efbe700 1 -- 172.21.15.119:0/489683025 shutdown_connections 2023-12-07T10:18:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:40.847+0000 7fef8efbe700 1 -- 172.21.15.119:0/489683025 wait complete. 2023-12-07T10:18:41.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.535+0000 7fa085c63700 1 -- 172.21.15.119:0/1031333981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa080107630 msgr2=0x7fa080107a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:41.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.535+0000 7fa085c63700 1 --2- 172.21.15.119:0/1031333981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa080107630 0x7fa080107a10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa070009990 tx=0x7fa070009ca0 comp rx=0 tx=0).stop 2023-12-07T10:18:41.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.535+0000 7fa085c63700 1 -- 172.21.15.119:0/1031333981 shutdown_connections 2023-12-07T10:18:41.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.535+0000 7fa085c63700 1 --2- 172.21.15.119:0/1031333981 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa080069970 0x7fa08010e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:41.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.535+0000 7fa085c63700 1 --2- 172.21.15.119:0/1031333981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa080107630 0x7fa080107a10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:41.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.535+0000 7fa085c63700 1 --2- 172.21.15.119:0/1031333981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa080068fb0 0x7fa080069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:41.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.535+0000 7fa085c63700 1 -- 172.21.15.119:0/1031333981 >> 172.21.15.119:0/1031333981 conn(0x7fa0800765b0 msgr2=0x7fa0800769c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:41.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.535+0000 7fa085c63700 1 -- 172.21.15.119:0/1031333981 shutdown_connections 2023-12-07T10:18:41.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.535+0000 7fa085c63700 1 -- 172.21.15.119:0/1031333981 wait complete. 2023-12-07T10:18:41.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.536+0000 7fa085c63700 1 Processor -- start 2023-12-07T10:18:41.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.536+0000 7fa085c63700 1 -- start start 2023-12-07T10:18:41.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.537+0000 7fa085c63700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa080068fb0 0x7fa08019f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:41.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.537+0000 7fa085c63700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa080069970 0x7fa08019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:41.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.537+0000 7fa085c63700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa080107630 0x7fa080199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:41.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.537+0000 7fa085c63700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08019a070 con 0x7fa080068fb0 2023-12-07T10:18:41.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.537+0000 7fa085c63700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08019a1e0 con 0x7fa080107630 2023-12-07T10:18:41.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.537+0000 7fa085c63700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08019a3a0 con 0x7fa080069970 2023-12-07T10:18:41.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.537+0000 7fa07f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa080068fb0 0x7fa08019f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:41.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.537+0000 7fa07f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa080068fb0 0x7fa08019f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44212/0 (socket says 172.21.15.119:44212) 2023-12-07T10:18:41.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.537+0000 7fa07f7fe700 1 -- 172.21.15.119:0/1010518506 learned_addr learned my addr 172.21.15.119:0/1010518506 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:41.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.537+0000 7fa07ffff700 1 --2- 172.21.15.119:0/1010518506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa080107630 0x7fa080199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:41.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.537+0000 7fa07effd700 1 --2- 172.21.15.119:0/1010518506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa080069970 0x7fa08019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:41.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.538+0000 7fa07f7fe700 1 -- 172.21.15.119:0/1010518506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa080069970 msgr2=0x7fa08019fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:41.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.538+0000 7fa07f7fe700 1 --2- 172.21.15.119:0/1010518506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa080069970 0x7fa08019fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:41.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.538+0000 7fa07f7fe700 1 -- 172.21.15.119:0/1010518506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa080107630 msgr2=0x7fa080199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:41.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.538+0000 7fa07f7fe700 1 --2- 172.21.15.119:0/1010518506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa080107630 0x7fa080199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:41.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.538+0000 7fa07f7fe700 1 -- 172.21.15.119:0/1010518506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0700095f0 con 0x7fa080068fb0 2023-12-07T10:18:41.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.538+0000 7fa07effd700 1 --2- 172.21.15.119:0/1010518506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa080069970 0x7fa08019fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:18:41.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.538+0000 7fa07f7fe700 1 --2- 172.21.15.119:0/1010518506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa080068fb0 0x7fa08019f8d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa070005770 tx=0x7fa0700057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:41.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.539+0000 7fa07cff9700 1 -- 172.21.15.119:0/1010518506 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07001d070 con 0x7fa080068fb0 2023-12-07T10:18:41.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.539+0000 7fa07cff9700 1 -- 172.21.15.119:0/1010518506 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa070021410 con 0x7fa080068fb0 2023-12-07T10:18:41.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.539+0000 7fa085c63700 1 -- 172.21.15.119:0/1010518506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa08019a620 con 0x7fa080068fb0 2023-12-07T10:18:41.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.539+0000 7fa07cff9700 1 -- 172.21.15.119:0/1010518506 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa070017650 con 0x7fa080068fb0 2023-12-07T10:18:41.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.539+0000 7fa085c63700 1 -- 172.21.15.119:0/1010518506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0801a68c0 con 0x7fa080068fb0 2023-12-07T10:18:41.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.540+0000 7fa085c63700 1 -- 172.21.15.119:0/1010518506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa0801117b0 con 0x7fa080068fb0 2023-12-07T10:18:41.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.543+0000 7fa07cff9700 1 -- 172.21.15.119:0/1010518506 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7fa0700217f0 con 0x7fa080068fb0 2023-12-07T10:18:41.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.544+0000 7fa07cff9700 1 --2- 172.21.15.119:0/1010518506 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa0680a1ba0 0x7fa0680a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:41.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.544+0000 7fa07cff9700 1 -- 172.21.15.119:0/1010518506 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fa0700cf650 con 0x7fa080068fb0 2023-12-07T10:18:41.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.544+0000 7fa07effd700 1 --2- 172.21.15.119:0/1010518506 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa0680a1ba0 0x7fa0680a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:41.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.544+0000 7fa07cff9700 1 --2- 172.21.15.119:0/1010518506 >> [v2:172.21.15.159:6808/1832540768,v1:172.21.15.159:6809/1832540768] conn(0x7fa0680acb40 0x7fa0680aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:41.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.544+0000 7fa07cff9700 1 -- 172.21.15.119:0/1010518506 --> [v2:172.21.15.159:6808/1832540768,v1:172.21.15.159:6809/1832540768] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0680af6b0 con 0x7fa0680acb40 2023-12-07T10:18:41.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.544+0000 7fa07cff9700 1 -- 172.21.15.119:0/1010518506 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0700cf9c0 con 0x7fa080068fb0 2023-12-07T10:18:41.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.545+0000 7fa07ffff700 1 --2- 172.21.15.119:0/1010518506 >> [v2:172.21.15.159:6808/1832540768,v1:172.21.15.159:6809/1832540768] conn(0x7fa0680acb40 0x7fa0680aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:41.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.545+0000 7fa07effd700 1 --2- 172.21.15.119:0/1010518506 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa0680a1ba0 0x7fa0680a4060 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa08019abd0 tx=0x7fa074009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:41.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.545+0000 7fa07ffff700 1 --2- 172.21.15.119:0/1010518506 >> [v2:172.21.15.159:6808/1832540768,v1:172.21.15.159:6809/1832540768] conn(0x7fa0680acb40 0x7fa0680aefe0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:41.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.546+0000 7fa07cff9700 1 -- 172.21.15.119:0/1010518506 <== osd.5 v2:172.21.15.159:6808/1832540768 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fa0680af6b0 con 0x7fa0680acb40 2023-12-07T10:18:41.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.564+0000 7fa085c63700 1 -- 172.21.15.119:0/1010518506 --> [v2:172.21.15.159:6808/1832540768,v1:172.21.15.159:6809/1832540768] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa0800ff990 con 0x7fa0680acb40 2023-12-07T10:18:41.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.564+0000 7fa07cff9700 1 -- 172.21.15.119:0/1010518506 <== osd.5 v2:172.21.15.159:6808/1832540768 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa0800ff990 con 0x7fa0680acb40 2023-12-07T10:18:41.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.565+0000 7fa085c63700 1 -- 172.21.15.119:0/1010518506 >> [v2:172.21.15.159:6808/1832540768,v1:172.21.15.159:6809/1832540768] conn(0x7fa0680acb40 msgr2=0x7fa0680aefe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:41.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.565+0000 7fa085c63700 1 --2- 172.21.15.119:0/1010518506 >> [v2:172.21.15.159:6808/1832540768,v1:172.21.15.159:6809/1832540768] conn(0x7fa0680acb40 0x7fa0680aefe0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:41.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.565+0000 7fa085c63700 1 -- 172.21.15.119:0/1010518506 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa0680a1ba0 msgr2=0x7fa0680a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:41.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.565+0000 7fa085c63700 1 --2- 172.21.15.119:0/1010518506 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa0680a1ba0 0x7fa0680a4060 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa08019abd0 tx=0x7fa074009260 comp rx=0 tx=0).stop 2023-12-07T10:18:41.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.565+0000 7fa085c63700 1 -- 172.21.15.119:0/1010518506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa080068fb0 msgr2=0x7fa08019f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:41.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.565+0000 7fa085c63700 1 --2- 172.21.15.119:0/1010518506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa080068fb0 0x7fa08019f8d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa070005770 tx=0x7fa0700057a0 comp rx=0 tx=0).stop 2023-12-07T10:18:41.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.565+0000 7fa07ffff700 1 -- 172.21.15.119:0/1010518506 reap_dead start 2023-12-07T10:18:41.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.565+0000 7fa085c63700 1 -- 172.21.15.119:0/1010518506 shutdown_connections 2023-12-07T10:18:41.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.565+0000 7fa085c63700 1 -- 172.21.15.119:0/1010518506 >> 172.21.15.119:0/1010518506 conn(0x7fa0800765b0 msgr2=0x7fa0800fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:41.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.565+0000 7fa085c63700 1 -- 172.21.15.119:0/1010518506 shutdown_connections 2023-12-07T10:18:41.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.565+0000 7fa085c63700 1 -- 172.21.15.119:0/1010518506 wait complete. 2023-12-07T10:18:41.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.715+0000 7f999475f700 1 -- 172.21.15.119:0/1947929116 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f998c107730 msgr2=0x7f998c107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:41.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.715+0000 7f999475f700 1 --2- 172.21.15.119:0/1947929116 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f998c107730 0x7f998c107b10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f99880098f0 tx=0x7f9988009c00 comp rx=0 tx=0).stop 2023-12-07T10:18:41.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.715+0000 7f999475f700 1 -- 172.21.15.119:0/1947929116 shutdown_connections 2023-12-07T10:18:41.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.715+0000 7f999475f700 1 --2- 172.21.15.119:0/1947929116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f998c102080 0x7f998c10e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:41.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.715+0000 7f999475f700 1 --2- 172.21.15.119:0/1947929116 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f998c107730 0x7f998c107b10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:41.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.715+0000 7f999475f700 1 --2- 172.21.15.119:0/1947929116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f998c1016c0 0x7f998c101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:41.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.715+0000 7f999475f700 1 -- 172.21.15.119:0/1947929116 >> 172.21.15.119:0/1947929116 conn(0x7f998c078800 msgr2=0x7f998c078c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:41.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.715+0000 7f999475f700 1 -- 172.21.15.119:0/1947929116 shutdown_connections 2023-12-07T10:18:41.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.715+0000 7f999475f700 1 -- 172.21.15.119:0/1947929116 wait complete. 2023-12-07T10:18:41.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.716+0000 7f999475f700 1 Processor -- start 2023-12-07T10:18:41.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.716+0000 7f999475f700 1 -- start start 2023-12-07T10:18:41.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.717+0000 7f999475f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f998c1016c0 0x7f998c19f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:41.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.717+0000 7f999475f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f998c102080 0x7f998c19fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:41.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.717+0000 7f999475f700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f998c107730 0x7f998c199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:41.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.717+0000 7f999475f700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998c19a090 con 0x7f998c107730 2023-12-07T10:18:41.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.717+0000 7f999475f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998c19a200 con 0x7f998c1016c0 2023-12-07T10:18:41.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.717+0000 7f999475f700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998c19a3c0 con 0x7f998c102080 2023-12-07T10:18:41.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.717+0000 7f9991cfa700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f998c102080 0x7f998c19fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:41.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.717+0000 7f9991cfa700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f998c102080 0x7f998c19fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:36576/0 (socket says 172.21.15.119:36576) 2023-12-07T10:18:41.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.717+0000 7f9991cfa700 1 -- 172.21.15.119:0/2893321031 learned_addr learned my addr 172.21.15.119:0/2893321031 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:41.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.717+0000 7f99924fb700 1 --2- 172.21.15.119:0/2893321031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f998c1016c0 0x7f998c19f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:41.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.717+0000 7f9992cfc700 1 --2- 172.21.15.119:0/2893321031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f998c107730 0x7f998c199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:41.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.718+0000 7f9991cfa700 1 -- 172.21.15.119:0/2893321031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f998c1016c0 msgr2=0x7f998c19f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:41.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.718+0000 7f9991cfa700 1 --2- 172.21.15.119:0/2893321031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f998c1016c0 0x7f998c19f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:41.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.718+0000 7f9991cfa700 1 -- 172.21.15.119:0/2893321031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f998c107730 msgr2=0x7f998c199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:41.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.718+0000 7f9991cfa700 1 --2- 172.21.15.119:0/2893321031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f998c107730 0x7f998c199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:41.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.718+0000 7f9991cfa700 1 -- 172.21.15.119:0/2893321031 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9988009520 con 0x7f998c102080 2023-12-07T10:18:41.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.718+0000 7f9992cfc700 1 --2- 172.21.15.119:0/2893321031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f998c107730 0x7f998c199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:18:41.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.718+0000 7f9991cfa700 1 --2- 172.21.15.119:0/2893321031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f998c102080 0x7f998c19fe30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f997c00e9c0 tx=0x7f997c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:41.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.718+0000 7f99837fe700 1 -- 172.21.15.119:0/2893321031 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c00cbd0 con 0x7f998c102080 2023-12-07T10:18:41.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.718+0000 7f999475f700 1 -- 172.21.15.119:0/2893321031 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f998c19a6a0 con 0x7f998c102080 2023-12-07T10:18:41.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.718+0000 7f99837fe700 1 -- 172.21.15.119:0/2893321031 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f997c00eeb0 con 0x7f998c102080 2023-12-07T10:18:41.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.718+0000 7f99837fe700 1 -- 172.21.15.119:0/2893321031 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c005580 con 0x7f998c102080 2023-12-07T10:18:41.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.718+0000 7f999475f700 1 -- 172.21.15.119:0/2893321031 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f998c1a68e0 con 0x7f998c102080 2023-12-07T10:18:41.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.719+0000 7f999475f700 1 -- 172.21.15.119:0/2893321031 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f998c1117d0 con 0x7f998c102080 2023-12-07T10:18:41.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.722+0000 7f99837fe700 1 -- 172.21.15.119:0/2893321031 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f997c004750 con 0x7f998c102080 2023-12-07T10:18:41.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.724+0000 7f99837fe700 1 --2- 172.21.15.119:0/2893321031 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f99780a1c20 0x7f99780a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:41.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.724+0000 7f99837fe700 1 -- 172.21.15.119:0/2893321031 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f997c014070 con 0x7f998c102080 2023-12-07T10:18:41.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.724+0000 7f99924fb700 1 --2- 172.21.15.119:0/2893321031 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f99780a1c20 0x7f99780a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:41.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.724+0000 7f99837fe700 1 --2- 172.21.15.119:0/2893321031 >> [v2:172.21.15.153:6816/3254653457,v1:172.21.15.153:6817/3254653457] conn(0x7f99780acbc0 0x7f99780af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:41.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.724+0000 7f99837fe700 1 -- 172.21.15.119:0/2893321031 --> [v2:172.21.15.153:6816/3254653457,v1:172.21.15.153:6817/3254653457] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f99780af730 con 0x7f99780acbc0 2023-12-07T10:18:41.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.724+0000 7f99837fe700 1 -- 172.21.15.119:0/2893321031 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f997c0ce5b0 con 0x7f998c102080 2023-12-07T10:18:41.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.725+0000 7f9992cfc700 1 --2- 172.21.15.119:0/2893321031 >> [v2:172.21.15.153:6816/3254653457,v1:172.21.15.153:6817/3254653457] conn(0x7f99780acbc0 0x7f99780af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:41.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.725+0000 7f99924fb700 1 --2- 172.21.15.119:0/2893321031 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f99780a1c20 0x7f99780a40e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9988009fd0 tx=0x7f9988019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:41.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.725+0000 7f9992cfc700 1 --2- 172.21.15.119:0/2893321031 >> [v2:172.21.15.153:6816/3254653457,v1:172.21.15.153:6817/3254653457] conn(0x7f99780acbc0 0x7f99780af060 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:41.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.726+0000 7f99837fe700 1 -- 172.21.15.119:0/2893321031 <== osd.7 v2:172.21.15.153:6816/3254653457 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f99780af730 con 0x7f99780acbc0 2023-12-07T10:18:41.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.753+0000 7f999475f700 1 -- 172.21.15.119:0/2893321031 --> [v2:172.21.15.153:6816/3254653457,v1:172.21.15.153:6817/3254653457] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f998c061c90 con 0x7f99780acbc0 2023-12-07T10:18:41.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.753+0000 7f99837fe700 1 -- 172.21.15.119:0/2893321031 <== osd.7 v2:172.21.15.153:6816/3254653457 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f998c061c90 con 0x7f99780acbc0 2023-12-07T10:18:41.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.754+0000 7f999475f700 1 -- 172.21.15.119:0/2893321031 >> [v2:172.21.15.153:6816/3254653457,v1:172.21.15.153:6817/3254653457] conn(0x7f99780acbc0 msgr2=0x7f99780af060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:41.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.754+0000 7f999475f700 1 --2- 172.21.15.119:0/2893321031 >> [v2:172.21.15.153:6816/3254653457,v1:172.21.15.153:6817/3254653457] conn(0x7f99780acbc0 0x7f99780af060 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:41.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.754+0000 7f999475f700 1 -- 172.21.15.119:0/2893321031 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f99780a1c20 msgr2=0x7f99780a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:41.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.754+0000 7f999475f700 1 --2- 172.21.15.119:0/2893321031 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f99780a1c20 0x7f99780a40e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9988009fd0 tx=0x7f9988019040 comp rx=0 tx=0).stop 2023-12-07T10:18:41.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.754+0000 7f999475f700 1 -- 172.21.15.119:0/2893321031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f998c102080 msgr2=0x7f998c19fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:41.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.754+0000 7f999475f700 1 --2- 172.21.15.119:0/2893321031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f998c102080 0x7f998c19fe30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f997c00e9c0 tx=0x7f997c00ecd0 comp rx=0 tx=0).stop 2023-12-07T10:18:41.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.754+0000 7f9992cfc700 1 -- 172.21.15.119:0/2893321031 reap_dead start 2023-12-07T10:18:41.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.754+0000 7f999475f700 1 -- 172.21.15.119:0/2893321031 shutdown_connections 2023-12-07T10:18:41.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.754+0000 7f999475f700 1 -- 172.21.15.119:0/2893321031 >> 172.21.15.119:0/2893321031 conn(0x7f998c078800 msgr2=0x7f998c10cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:41.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.754+0000 7f999475f700 1 -- 172.21.15.119:0/2893321031 shutdown_connections 2023-12-07T10:18:41.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:41.754+0000 7f999475f700 1 -- 172.21.15.119:0/2893321031 wait complete. 2023-12-07T10:18:41.881 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:41 smithi119 ceph-mon[97908]: pgmap v311: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2023-12-07T10:18:41.994 INFO:teuthology.orchestra.run.smithi119.stdout:343597383703 2023-12-07T10:18:41.997 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.11 2023-12-07T10:18:42.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:41 smithi159 ceph-mon[134726]: pgmap v311: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2023-12-07T10:18:42.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:41 smithi153 ceph-mon[133625]: pgmap v311: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2023-12-07T10:18:42.554 INFO:teuthology.orchestra.run.smithi119.stdout:167503724596 2023-12-07T10:18:42.555 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.5 2023-12-07T10:18:42.613 INFO:teuthology.orchestra.run.smithi119.stdout:201863462959 2023-12-07T10:18:42.613 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.6 2023-12-07T10:18:43.008 INFO:teuthology.orchestra.run.smithi119.stdout:146028888120 2023-12-07T10:18:43.009 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.4 2023-12-07T10:18:43.078 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:42 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:18:43.156 INFO:teuthology.orchestra.run.smithi119.stdout:223338299435 2023-12-07T10:18:43.157 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.7 2023-12-07T10:18:43.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:42 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:18:43.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:42 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:18:44.053 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:44.097 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:44.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:43 smithi119 ceph-mon[97908]: pgmap v312: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2023-12-07T10:18:44.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:43 smithi159 ceph-mon[134726]: pgmap v312: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2023-12-07T10:18:44.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:43 smithi153 ceph-mon[133625]: pgmap v312: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2023-12-07T10:18:44.470 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:44.473 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:44.476 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:44.524 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:44.542 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:44.620 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:44.633 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:44.637 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:45.762 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:45 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:18:45.762 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:45 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:18:45.762 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:45 smithi119 ceph-mon[97908]: pgmap v313: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-07T10:18:45.763 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:45 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:18:45.763 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:45 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:18:46.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:18:46.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:18:46.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:45 smithi159 ceph-mon[134726]: pgmap v313: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-07T10:18:46.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:18:46.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:45 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:18:46.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:45 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:18:46.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:45 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:18:46.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:45 smithi153 ceph-mon[133625]: pgmap v313: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-07T10:18:46.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:45 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:18:46.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:45 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:18:46.242 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:46.670 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:47.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:47 smithi119 ceph-mon[97908]: pgmap v314: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-12-07T10:18:48.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:47 smithi159 ceph-mon[134726]: pgmap v314: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-12-07T10:18:48.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:47 smithi153 ceph-mon[133625]: pgmap v314: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-12-07T10:18:49.921 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:49 smithi119 ceph-mon[97908]: pgmap v315: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-07T10:18:49.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.929+0000 7f92a4c36700 1 -- 172.21.15.119:0/1973058492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92a010c260 msgr2=0x7f92a010c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:49.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.929+0000 7f92a4c36700 1 --2- 172.21.15.119:0/1973058492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92a010c260 0x7f92a010c640 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9290009990 tx=0x7f9290009ca0 comp rx=0 tx=0).stop 2023-12-07T10:18:49.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.929+0000 7f92a4c36700 1 -- 172.21.15.119:0/1973058492 shutdown_connections 2023-12-07T10:18:49.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.929+0000 7f92a4c36700 1 --2- 172.21.15.119:0/1973058492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a01112f0 0x7f92a0118ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:49.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.929+0000 7f92a4c36700 1 --2- 172.21.15.119:0/1973058492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92a010c260 0x7f92a010c640 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:49.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.929+0000 7f92a4c36700 1 --2- 172.21.15.119:0/1973058492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92a010cc10 0x7f92a0110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:49.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.929+0000 7f92a4c36700 1 -- 172.21.15.119:0/1973058492 >> 172.21.15.119:0/1973058492 conn(0x7f92a006c710 msgr2=0x7f92a010a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:49.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.930+0000 7f92a4c36700 1 -- 172.21.15.119:0/1973058492 shutdown_connections 2023-12-07T10:18:49.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.930+0000 7f92a4c36700 1 -- 172.21.15.119:0/1973058492 wait complete. 2023-12-07T10:18:49.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.931+0000 7f92a4c36700 1 Processor -- start 2023-12-07T10:18:49.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.931+0000 7f92a4c36700 1 -- start start 2023-12-07T10:18:49.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.932+0000 7f92a4c36700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92a010c260 0x7f92a01a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:49.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.932+0000 7f92a4c36700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92a010cc10 0x7f92a01a29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:49.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.932+0000 7f92a4c36700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a01112f0 0x7f92a01a6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:49.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.932+0000 7f92a4c36700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a01a7360 con 0x7f92a010c260 2023-12-07T10:18:49.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.932+0000 7f92a4c36700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a01a74d0 con 0x7f92a01112f0 2023-12-07T10:18:49.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.932+0000 7f92a4c36700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a01a7690 con 0x7f92a010cc10 2023-12-07T10:18:49.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.932+0000 7f929e59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92a010c260 0x7f92a01a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:49.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.932+0000 7f929e59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92a010c260 0x7f92a01a2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56368/0 (socket says 172.21.15.119:56368) 2023-12-07T10:18:49.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.932+0000 7f929e59c700 1 -- 172.21.15.119:0/2198324164 learned_addr learned my addr 172.21.15.119:0/2198324164 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:49.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.932+0000 7f929dd9b700 1 --2- 172.21.15.119:0/2198324164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92a010cc10 0x7f92a01a29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:49.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.933+0000 7f929ed9d700 1 --2- 172.21.15.119:0/2198324164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a01112f0 0x7f92a01a6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:49.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.933+0000 7f929e59c700 1 -- 172.21.15.119:0/2198324164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92a010cc10 msgr2=0x7f92a01a29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:49.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.933+0000 7f929e59c700 1 --2- 172.21.15.119:0/2198324164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92a010cc10 0x7f92a01a29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:49.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.933+0000 7f929e59c700 1 -- 172.21.15.119:0/2198324164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a01112f0 msgr2=0x7f92a01a6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:49.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.933+0000 7f929e59c700 1 --2- 172.21.15.119:0/2198324164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a01112f0 0x7f92a01a6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:49.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.933+0000 7f929e59c700 1 -- 172.21.15.119:0/2198324164 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92900095f0 con 0x7f92a010c260 2023-12-07T10:18:49.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.933+0000 7f929ed9d700 1 --2- 172.21.15.119:0/2198324164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a01112f0 0x7f92a01a6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:18:49.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.933+0000 7f929e59c700 1 --2- 172.21.15.119:0/2198324164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92a010c260 0x7f92a01a2460 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9290000c00 tx=0x7f9290005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:49.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.934+0000 7f928f7fe700 1 -- 172.21.15.119:0/2198324164 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929001d070 con 0x7f92a010c260 2023-12-07T10:18:49.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.934+0000 7f928f7fe700 1 -- 172.21.15.119:0/2198324164 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9290022470 con 0x7f92a010c260 2023-12-07T10:18:49.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.934+0000 7f928f7fe700 1 -- 172.21.15.119:0/2198324164 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9290017650 con 0x7f92a010c260 2023-12-07T10:18:49.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.934+0000 7f92a4c36700 1 -- 172.21.15.119:0/2198324164 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a01a7910 con 0x7f92a010c260 2023-12-07T10:18:49.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.934+0000 7f92a4c36700 1 -- 172.21.15.119:0/2198324164 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a01af000 con 0x7f92a010c260 2023-12-07T10:18:49.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.935+0000 7f92a4c36700 1 -- 172.21.15.119:0/2198324164 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a01145b0 con 0x7f92a010c260 2023-12-07T10:18:49.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.939+0000 7f928f7fe700 1 -- 172.21.15.119:0/2198324164 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f92900225e0 con 0x7f92a010c260 2023-12-07T10:18:49.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.942+0000 7f928f7fe700 1 --2- 172.21.15.119:0/2198324164 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f92880a1ca0 0x7f92880a4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:49.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.942+0000 7f928f7fe700 1 -- 172.21.15.119:0/2198324164 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f92900cf3d0 con 0x7f92a010c260 2023-12-07T10:18:49.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.942+0000 7f929dd9b700 1 --2- 172.21.15.119:0/2198324164 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f92880a1ca0 0x7f92880a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:49.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.943+0000 7f928f7fe700 1 -- 172.21.15.119:0/2198324164 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f92900cf7b0 con 0x7f92a010c260 2023-12-07T10:18:49.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:49.943+0000 7f929dd9b700 1 --2- 172.21.15.119:0/2198324164 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f92880a1ca0 0x7f92880a4160 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f92a01a39c0 tx=0x7f9294009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:50.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:49 smithi159 ceph-mon[134726]: pgmap v315: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-07T10:18:50.029 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:49 smithi153 ceph-mon[133625]: pgmap v315: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-07T10:18:50.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.168+0000 7f92a4c36700 1 -- 172.21.15.119:0/2198324164 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f92a010d6f0 con 0x7f92a010c260 2023-12-07T10:18:50.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.168+0000 7f928f7fe700 1 -- 172.21.15.119:0/2198324164 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9290027020 con 0x7f92a010c260 2023-12-07T10:18:50.170 INFO:teuthology.orchestra.run.smithi119.stdout:253403070504 2023-12-07T10:18:50.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.171+0000 7f92a4c36700 1 -- 172.21.15.119:0/2198324164 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f92880a1ca0 msgr2=0x7f92880a4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:50.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.171+0000 7f92a4c36700 1 --2- 172.21.15.119:0/2198324164 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f92880a1ca0 0x7f92880a4160 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f92a01a39c0 tx=0x7f9294009260 comp rx=0 tx=0).stop 2023-12-07T10:18:50.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.171+0000 7f92a4c36700 1 -- 172.21.15.119:0/2198324164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92a010c260 msgr2=0x7f92a01a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:50.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.171+0000 7f92a4c36700 1 --2- 172.21.15.119:0/2198324164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92a010c260 0x7f92a01a2460 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9290000c00 tx=0x7f9290005740 comp rx=0 tx=0).stop 2023-12-07T10:18:50.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.171+0000 7f92a4c36700 1 -- 172.21.15.119:0/2198324164 shutdown_connections 2023-12-07T10:18:50.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.171+0000 7f92a4c36700 1 --2- 172.21.15.119:0/2198324164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a01112f0 0x7f92a01a6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:50.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.171+0000 7f92a4c36700 1 --2- 172.21.15.119:0/2198324164 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f92880a1ca0 0x7f92880a4160 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:50.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.171+0000 7f92a4c36700 1 --2- 172.21.15.119:0/2198324164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92a010c260 0x7f92a01a2460 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:50.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.171+0000 7f92a4c36700 1 --2- 172.21.15.119:0/2198324164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92a010cc10 0x7f92a01a29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:50.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.171+0000 7f92a4c36700 1 -- 172.21.15.119:0/2198324164 >> 172.21.15.119:0/2198324164 conn(0x7f92a006c710 msgr2=0x7f92a010a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:50.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.171+0000 7f92a4c36700 1 -- 172.21.15.119:0/2198324164 shutdown_connections 2023-12-07T10:18:50.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.171+0000 7f92a4c36700 1 -- 172.21.15.119:0/2198324164 wait complete. 2023-12-07T10:18:50.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.522+0000 7f276192f700 1 -- 172.21.15.119:0/76566953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f275c1016c0 msgr2=0x7f275c101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:50.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.522+0000 7f276192f700 1 --2- 172.21.15.119:0/76566953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f275c1016c0 0x7f275c101b40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2750009990 tx=0x7f2750009ca0 comp rx=0 tx=0).stop 2023-12-07T10:18:50.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.522+0000 7f276192f700 1 -- 172.21.15.119:0/76566953 shutdown_connections 2023-12-07T10:18:50.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.522+0000 7f276192f700 1 --2- 172.21.15.119:0/76566953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f275c102080 0x7f275c10e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:50.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.522+0000 7f276192f700 1 --2- 172.21.15.119:0/76566953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f275c107730 0x7f275c107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:50.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.522+0000 7f276192f700 1 --2- 172.21.15.119:0/76566953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f275c1016c0 0x7f275c101b40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:50.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.522+0000 7f276192f700 1 -- 172.21.15.119:0/76566953 >> 172.21.15.119:0/76566953 conn(0x7f275c078800 msgr2=0x7f275c078c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:50.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.522+0000 7f276192f700 1 -- 172.21.15.119:0/76566953 shutdown_connections 2023-12-07T10:18:50.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.522+0000 7f276192f700 1 -- 172.21.15.119:0/76566953 wait complete. 2023-12-07T10:18:50.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.523+0000 7f276192f700 1 Processor -- start 2023-12-07T10:18:50.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.523+0000 7f276192f700 1 -- start start 2023-12-07T10:18:50.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.524+0000 7f276192f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f275c1016c0 0x7f275c197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:50.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.524+0000 7f276192f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f275c102080 0x7f275c197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:50.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.524+0000 7f276192f700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f275c107730 0x7f275c19c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:50.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.524+0000 7f276192f700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c19c920 con 0x7f275c107730 2023-12-07T10:18:50.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.524+0000 7f276192f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c19ca90 con 0x7f275c102080 2023-12-07T10:18:50.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.524+0000 7f276192f700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c19cc50 con 0x7f275c1016c0 2023-12-07T10:18:50.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.524+0000 7f275affd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f275c1016c0 0x7f275c197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:50.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.524+0000 7f275b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f275c107730 0x7f275c19c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:50.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.524+0000 7f275b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f275c107730 0x7f275c19c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56390/0 (socket says 172.21.15.119:56390) 2023-12-07T10:18:50.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.524+0000 7f275affd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f275c1016c0 0x7f275c197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:37350/0 (socket says 172.21.15.119:37350) 2023-12-07T10:18:50.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.524+0000 7f275b7fe700 1 -- 172.21.15.119:0/1411809513 learned_addr learned my addr 172.21.15.119:0/1411809513 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:50.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.524+0000 7f275a7fc700 1 --2- 172.21.15.119:0/1411809513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f275c102080 0x7f275c197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:50.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.524+0000 7f275b7fe700 1 -- 172.21.15.119:0/1411809513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f275c1016c0 msgr2=0x7f275c197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:50.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.524+0000 7f275b7fe700 1 --2- 172.21.15.119:0/1411809513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f275c1016c0 0x7f275c197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:50.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.524+0000 7f275b7fe700 1 -- 172.21.15.119:0/1411809513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f275c102080 msgr2=0x7f275c197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:50.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.524+0000 7f275b7fe700 1 --2- 172.21.15.119:0/1411809513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f275c102080 0x7f275c197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:50.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.524+0000 7f275b7fe700 1 -- 172.21.15.119:0/1411809513 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27500095f0 con 0x7f275c107730 2023-12-07T10:18:50.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.524+0000 7f275affd700 1 --2- 172.21.15.119:0/1411809513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f275c1016c0 0x7f275c197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:18:50.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.525+0000 7f275b7fe700 1 --2- 172.21.15.119:0/1411809513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f275c107730 0x7f275c19c350 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f275400efc0 tx=0x7f275400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:50.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.525+0000 7f276092d700 1 -- 172.21.15.119:0/1411809513 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2754018070 con 0x7f275c107730 2023-12-07T10:18:50.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.525+0000 7f276092d700 1 -- 172.21.15.119:0/1411809513 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f275400f040 con 0x7f275c107730 2023-12-07T10:18:50.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.525+0000 7f276192f700 1 -- 172.21.15.119:0/1411809513 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f275c19ced0 con 0x7f275c107730 2023-12-07T10:18:50.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.525+0000 7f276092d700 1 -- 172.21.15.119:0/1411809513 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275400cc10 con 0x7f275c107730 2023-12-07T10:18:50.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.525+0000 7f276192f700 1 -- 172.21.15.119:0/1411809513 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f275c19d1b0 con 0x7f275c107730 2023-12-07T10:18:50.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.526+0000 7f276192f700 1 -- 172.21.15.119:0/1411809513 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f275c109bc0 con 0x7f275c107730 2023-12-07T10:18:50.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.528+0000 7f276092d700 1 -- 172.21.15.119:0/1411809513 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f2754003a00 con 0x7f275c107730 2023-12-07T10:18:50.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.529+0000 7f276092d700 1 --2- 172.21.15.119:0/1411809513 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f27440a1ba0 0x7f27440a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:50.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.529+0000 7f276092d700 1 -- 172.21.15.119:0/1411809513 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f27540cd4b0 con 0x7f275c107730 2023-12-07T10:18:50.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.530+0000 7f275affd700 1 --2- 172.21.15.119:0/1411809513 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f27440a1ba0 0x7f27440a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:50.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.530+0000 7f275affd700 1 --2- 172.21.15.119:0/1411809513 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f27440a1ba0 0x7f27440a4060 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f274c009fd0 tx=0x7f274c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:50.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.534+0000 7f276092d700 1 -- 172.21.15.119:0/1411809513 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f27540964a0 con 0x7f275c107730 2023-12-07T10:18:50.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.747+0000 7f276192f700 1 -- 172.21.15.119:0/1411809513 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f275c102d50 con 0x7f275c107730 2023-12-07T10:18:50.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.748+0000 7f276092d700 1 -- 172.21.15.119:0/1411809513 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2754099ac0 con 0x7f275c107730 2023-12-07T10:18:50.749 INFO:teuthology.orchestra.run.smithi119.stdout:111669149758 2023-12-07T10:18:50.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.750+0000 7f276192f700 1 -- 172.21.15.119:0/1411809513 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f27440a1ba0 msgr2=0x7f27440a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:50.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.750+0000 7f276192f700 1 --2- 172.21.15.119:0/1411809513 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f27440a1ba0 0x7f27440a4060 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f274c009fd0 tx=0x7f274c009260 comp rx=0 tx=0).stop 2023-12-07T10:18:50.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.750+0000 7f276192f700 1 -- 172.21.15.119:0/1411809513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f275c107730 msgr2=0x7f275c19c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:50.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.750+0000 7f276192f700 1 --2- 172.21.15.119:0/1411809513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f275c107730 0x7f275c19c350 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f275400efc0 tx=0x7f275400c3a0 comp rx=0 tx=0).stop 2023-12-07T10:18:50.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.750+0000 7f276192f700 1 -- 172.21.15.119:0/1411809513 shutdown_connections 2023-12-07T10:18:50.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.750+0000 7f276192f700 1 --2- 172.21.15.119:0/1411809513 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f27440a1ba0 0x7f27440a4060 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:50.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.750+0000 7f276192f700 1 --2- 172.21.15.119:0/1411809513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f275c107730 0x7f275c19c350 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:50.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.750+0000 7f276192f700 1 --2- 172.21.15.119:0/1411809513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f275c1016c0 0x7f275c197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:50.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.750+0000 7f276192f700 1 --2- 172.21.15.119:0/1411809513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f275c102080 0x7f275c197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:50.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.750+0000 7f276192f700 1 -- 172.21.15.119:0/1411809513 >> 172.21.15.119:0/1411809513 conn(0x7f275c078800 msgr2=0x7f275c100390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:50.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.750+0000 7f276192f700 1 -- 172.21.15.119:0/1411809513 shutdown_connections 2023-12-07T10:18:50.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:50.750+0000 7f276192f700 1 -- 172.21.15.119:0/1411809513 wait complete. 2023-12-07T10:18:51.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:50 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2198324164' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-12-07T10:18:51.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:50 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1411809513' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-07T10:18:51.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:50 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2198324164' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-12-07T10:18:51.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:50 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1411809513' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-07T10:18:51.086 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:50 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2198324164' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-12-07T10:18:51.086 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:50 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1411809513' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-07T10:18:51.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.576+0000 7f25083c9700 1 -- 172.21.15.119:0/2603905596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25001037c0 msgr2=0x7f2500103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:51.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.576+0000 7f25083c9700 1 --2- 172.21.15.119:0/2603905596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25001037c0 0x7f2500103c40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009990 tx=0x7f24f0009ca0 comp rx=0 tx=0).stop 2023-12-07T10:18:51.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.576+0000 7f25083c9700 1 -- 172.21.15.119:0/2603905596 shutdown_connections 2023-12-07T10:18:51.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.576+0000 7f25083c9700 1 --2- 172.21.15.119:0/2603905596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2500104180 0x7f2500110690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:51.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.576+0000 7f25083c9700 1 --2- 172.21.15.119:0/2603905596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25001097c0 0x7f2500109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:51.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.576+0000 7f25083c9700 1 --2- 172.21.15.119:0/2603905596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25001037c0 0x7f2500103c40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:51.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.576+0000 7f25083c9700 1 -- 172.21.15.119:0/2603905596 >> 172.21.15.119:0/2603905596 conn(0x7f25000ff2d0 msgr2=0x7f25001016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:51.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.576+0000 7f25083c9700 1 -- 172.21.15.119:0/2603905596 shutdown_connections 2023-12-07T10:18:51.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.576+0000 7f25083c9700 1 -- 172.21.15.119:0/2603905596 wait complete. 2023-12-07T10:18:51.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.577+0000 7f25083c9700 1 Processor -- start 2023-12-07T10:18:51.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.577+0000 7f25083c9700 1 -- start start 2023-12-07T10:18:51.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.578+0000 7f25083c9700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25001037c0 0x7f250019f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:51.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.578+0000 7f25083c9700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2500104180 0x7f250019fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:51.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.578+0000 7f25083c9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25001097c0 0x7f2500199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:51.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.578+0000 7f25083c9700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250019a0d0 con 0x7f2500104180 2023-12-07T10:18:51.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.578+0000 7f25083c9700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250019a240 con 0x7f25001097c0 2023-12-07T10:18:51.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.578+0000 7f25083c9700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250019a400 con 0x7f25001037c0 2023-12-07T10:18:51.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.578+0000 7f2505964700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2500104180 0x7f250019fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.578+0000 7f2505964700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2500104180 0x7f250019fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56414/0 (socket says 172.21.15.119:56414) 2023-12-07T10:18:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.578+0000 7f2505964700 1 -- 172.21.15.119:0/3787658987 learned_addr learned my addr 172.21.15.119:0/3787658987 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.578+0000 7f2506165700 1 --2- 172.21.15.119:0/3787658987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25001037c0 0x7f250019f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.578+0000 7f2506966700 1 --2- 172.21.15.119:0/3787658987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25001097c0 0x7f2500199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.579+0000 7f2505964700 1 -- 172.21.15.119:0/3787658987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25001037c0 msgr2=0x7f250019f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.579+0000 7f2505964700 1 --2- 172.21.15.119:0/3787658987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25001037c0 0x7f250019f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.579+0000 7f2505964700 1 -- 172.21.15.119:0/3787658987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25001097c0 msgr2=0x7f2500199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.579+0000 7f2505964700 1 --2- 172.21.15.119:0/3787658987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25001097c0 0x7f2500199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.579+0000 7f2505964700 1 -- 172.21.15.119:0/3787658987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24fc009520 con 0x7f2500104180 2023-12-07T10:18:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.579+0000 7f2505964700 1 --2- 172.21.15.119:0/3787658987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2500104180 0x7f250019fe20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f24f0000c00 tx=0x7f24f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.579+0000 7f24f77fe700 1 -- 172.21.15.119:0/3787658987 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f001d070 con 0x7f2500104180 2023-12-07T10:18:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.579+0000 7f24f77fe700 1 -- 172.21.15.119:0/3787658987 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24f0022470 con 0x7f2500104180 2023-12-07T10:18:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.579+0000 7f25083c9700 1 -- 172.21.15.119:0/3787658987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f00095f0 con 0x7f2500104180 2023-12-07T10:18:51.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.579+0000 7f24f77fe700 1 -- 172.21.15.119:0/3787658987 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f0017650 con 0x7f2500104180 2023-12-07T10:18:51.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.579+0000 7f25083c9700 1 -- 172.21.15.119:0/3787658987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25001a6700 con 0x7f2500104180 2023-12-07T10:18:51.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.580+0000 7f25083c9700 1 -- 172.21.15.119:0/3787658987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f250010bde0 con 0x7f2500104180 2023-12-07T10:18:51.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.582+0000 7f24f77fe700 1 -- 172.21.15.119:0/3787658987 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f24f00225e0 con 0x7f2500104180 2023-12-07T10:18:51.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.584+0000 7f24f77fe700 1 --2- 172.21.15.119:0/3787658987 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f24ec0a1ba0 0x7f24ec0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:51.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.584+0000 7f24f77fe700 1 -- 172.21.15.119:0/3787658987 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f24f00cec20 con 0x7f2500104180 2023-12-07T10:18:51.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.584+0000 7f2506165700 1 --2- 172.21.15.119:0/3787658987 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f24ec0a1ba0 0x7f24ec0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:51.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.585+0000 7f2506165700 1 --2- 172.21.15.119:0/3787658987 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f24ec0a1ba0 0x7f24ec0a4060 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009d70 tx=0x7f24fc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:51.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.588+0000 7f24f77fe700 1 -- 172.21.15.119:0/3787658987 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f24f0097c10 con 0x7f2500104180 2023-12-07T10:18:51.795 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070501 got 253403070504 for osd.8 2023-12-07T10:18:51.795 DEBUG:teuthology.parallel:result is None 2023-12-07T10:18:51.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.805+0000 7f25083c9700 1 -- 172.21.15.119:0/3787658987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f2500104f00 con 0x7f2500104180 2023-12-07T10:18:51.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.805+0000 7f24f77fe700 1 -- 172.21.15.119:0/3787658987 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f24f009b230 con 0x7f2500104180 2023-12-07T10:18:51.808 INFO:teuthology.orchestra.run.smithi119.stdout:343597383705 2023-12-07T10:18:51.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.808+0000 7f25083c9700 1 -- 172.21.15.119:0/3787658987 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f24ec0a1ba0 msgr2=0x7f24ec0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:51.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.808+0000 7f25083c9700 1 --2- 172.21.15.119:0/3787658987 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f24ec0a1ba0 0x7f24ec0a4060 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009d70 tx=0x7f24fc009260 comp rx=0 tx=0).stop 2023-12-07T10:18:51.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.808+0000 7f25083c9700 1 -- 172.21.15.119:0/3787658987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2500104180 msgr2=0x7f250019fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:51.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.808+0000 7f25083c9700 1 --2- 172.21.15.119:0/3787658987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2500104180 0x7f250019fe20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f24f0000c00 tx=0x7f24f0004730 comp rx=0 tx=0).stop 2023-12-07T10:18:51.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.808+0000 7f25083c9700 1 -- 172.21.15.119:0/3787658987 shutdown_connections 2023-12-07T10:18:51.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.808+0000 7f25083c9700 1 --2- 172.21.15.119:0/3787658987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25001097c0 0x7f2500199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:51.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.808+0000 7f25083c9700 1 --2- 172.21.15.119:0/3787658987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25001037c0 0x7f250019f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:51.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.808+0000 7f25083c9700 1 --2- 172.21.15.119:0/3787658987 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f24ec0a1ba0 0x7f24ec0a4060 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:51.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.808+0000 7f25083c9700 1 --2- 172.21.15.119:0/3787658987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2500104180 0x7f250019fe20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:51.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.808+0000 7f25083c9700 1 -- 172.21.15.119:0/3787658987 >> 172.21.15.119:0/3787658987 conn(0x7f25000ff2d0 msgr2=0x7f25000ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:51.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.808+0000 7f25083c9700 1 -- 172.21.15.119:0/3787658987 shutdown_connections 2023-12-07T10:18:51.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:51.808+0000 7f25083c9700 1 -- 172.21.15.119:0/3787658987 wait complete. 2023-12-07T10:18:51.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:51 smithi119 ceph-mon[97908]: pgmap v316: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2023-12-07T10:18:52.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:51 smithi159 ceph-mon[134726]: pgmap v316: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2023-12-07T10:18:52.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:51 smithi153 ceph-mon[133625]: pgmap v316: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2023-12-07T10:18:52.990 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:52 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3787658987' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-12-07T10:18:53.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:52 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3787658987' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-12-07T10:18:53.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:52 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3787658987' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-12-07T10:18:53.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.476+0000 7fc3fe8f9700 1 -- 172.21.15.119:0/1744357074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3f8068fb0 msgr2=0x7fc3f8069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:53.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.476+0000 7fc3fe8f9700 1 --2- 172.21.15.119:0/1744357074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3f8068fb0 0x7fc3f8069430 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000b0d0 tx=0x7fc3e000b3e0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.476+0000 7fc3fe8f9700 1 -- 172.21.15.119:0/1744357074 shutdown_connections 2023-12-07T10:18:53.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.476+0000 7fc3fe8f9700 1 --2- 172.21.15.119:0/1744357074 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3f8069970 0x7fc3f810e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.476+0000 7fc3fe8f9700 1 --2- 172.21.15.119:0/1744357074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3f8107630 0x7fc3f8107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.476+0000 7fc3fe8f9700 1 --2- 172.21.15.119:0/1744357074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3f8068fb0 0x7fc3f8069430 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.476+0000 7fc3fe8f9700 1 -- 172.21.15.119:0/1744357074 >> 172.21.15.119:0/1744357074 conn(0x7fc3f80765b0 msgr2=0x7fc3f80769c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:53.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.476+0000 7fc3fe8f9700 1 -- 172.21.15.119:0/1744357074 shutdown_connections 2023-12-07T10:18:53.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.476+0000 7fc3fe8f9700 1 -- 172.21.15.119:0/1744357074 wait complete. 2023-12-07T10:18:53.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.478+0000 7fc3fe8f9700 1 Processor -- start 2023-12-07T10:18:53.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.478+0000 7fc3fe8f9700 1 -- start start 2023-12-07T10:18:53.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.478+0000 7fc3fe8f9700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3f8068fb0 0x7fc3f8199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:53.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.478+0000 7fc3fe8f9700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3f8069970 0x7fc3f819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:53.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.478+0000 7fc3fe8f9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3f8107630 0x7fc3f819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:53.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.478+0000 7fc3fe8f9700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f819ec60 con 0x7fc3f8069970 2023-12-07T10:18:53.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.478+0000 7fc3fe8f9700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f819edd0 con 0x7fc3f8107630 2023-12-07T10:18:53.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.478+0000 7fc3fe8f9700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f819ef90 con 0x7fc3f8068fb0 2023-12-07T10:18:53.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.479+0000 7fc3ef7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3f8069970 0x7fc3f819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:53.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.479+0000 7fc3ef7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3f8069970 0x7fc3f819a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58676/0 (socket says 172.21.15.119:58676) 2023-12-07T10:18:53.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.479+0000 7fc3ef7fe700 1 -- 172.21.15.119:0/884215178 learned_addr learned my addr 172.21.15.119:0/884215178 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:53.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.479+0000 7fc3fce96700 1 --2- 172.21.15.119:0/884215178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3f8107630 0x7fc3f819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:53.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.479+0000 7fc3ef7fe700 1 -- 172.21.15.119:0/884215178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3f8068fb0 msgr2=0x7fc3f8199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:53.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.479+0000 7fc3ef7fe700 1 --2- 172.21.15.119:0/884215178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3f8068fb0 0x7fc3f8199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.479+0000 7fc3ef7fe700 1 -- 172.21.15.119:0/884215178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3f8107630 msgr2=0x7fc3f819e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:53.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.479+0000 7fc3ef7fe700 1 --2- 172.21.15.119:0/884215178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3f8107630 0x7fc3f819e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.479+0000 7fc3ef7fe700 1 -- 172.21.15.119:0/884215178 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e8009520 con 0x7fc3f8069970 2023-12-07T10:18:53.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.480+0000 7fc3ef7fe700 1 --2- 172.21.15.119:0/884215178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3f8069970 0x7fc3f819a2a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000bd80 tx=0x7fc3e000bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:53.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.480+0000 7fc3f5ffb700 1 -- 172.21.15.119:0/884215178 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e000e040 con 0x7fc3f8069970 2023-12-07T10:18:53.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.480+0000 7fc3fe8f9700 1 -- 172.21.15.119:0/884215178 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e0009d00 con 0x7fc3f8069970 2023-12-07T10:18:53.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.480+0000 7fc3f5ffb700 1 -- 172.21.15.119:0/884215178 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3e001b8e0 con 0x7fc3f8069970 2023-12-07T10:18:53.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.480+0000 7fc3f5ffb700 1 -- 172.21.15.119:0/884215178 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0012890 con 0x7fc3f8069970 2023-12-07T10:18:53.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.481+0000 7fc3fe8f9700 1 -- 172.21.15.119:0/884215178 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f81083d0 con 0x7fc3f8069970 2023-12-07T10:18:53.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.482+0000 7fc3fe8f9700 1 -- 172.21.15.119:0/884215178 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f8102d00 con 0x7fc3f8069970 2023-12-07T10:18:53.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.483+0000 7fc3f5ffb700 1 -- 172.21.15.119:0/884215178 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7fc3e001ba50 con 0x7fc3f8069970 2023-12-07T10:18:53.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.485+0000 7fc3f5ffb700 1 --2- 172.21.15.119:0/884215178 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc3d80a1c20 0x7fc3d80a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:53.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.485+0000 7fc3f5ffb700 1 -- 172.21.15.119:0/884215178 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fc3e00ce520 con 0x7fc3f8069970 2023-12-07T10:18:53.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.485+0000 7fc3f7fff700 1 --2- 172.21.15.119:0/884215178 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc3d80a1c20 0x7fc3d80a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:53.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.486+0000 7fc3f7fff700 1 --2- 172.21.15.119:0/884215178 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc3d80a1c20 0x7fc3d80a40e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8005d90 tx=0x7fc3e8005ce0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:53.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.489+0000 7fc3f5ffb700 1 -- 172.21.15.119:0/884215178 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc3e0097570 con 0x7fc3f8069970 2023-12-07T10:18:53.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.434+0000 7f03bb5b5700 1 -- 172.21.15.119:0/2503763707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b41016c0 msgr2=0x7f03b4101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:53.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.434+0000 7f03bb5b5700 1 --2- 172.21.15.119:0/2503763707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b41016c0 0x7f03b4101b40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009990 tx=0x7f03a4009ca0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.434+0000 7f03bb5b5700 1 -- 172.21.15.119:0/2503763707 shutdown_connections 2023-12-07T10:18:53.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.434+0000 7f03bb5b5700 1 --2- 172.21.15.119:0/2503763707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03b4102080 0x7f03b410e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.434+0000 7f03bb5b5700 1 --2- 172.21.15.119:0/2503763707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03b4107730 0x7f03b4107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.434+0000 7f03bb5b5700 1 --2- 172.21.15.119:0/2503763707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b41016c0 0x7f03b4101b40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.434+0000 7f03bb5b5700 1 -- 172.21.15.119:0/2503763707 >> 172.21.15.119:0/2503763707 conn(0x7f03b4078800 msgr2=0x7f03b4078c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:53.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.434+0000 7f03bb5b5700 1 -- 172.21.15.119:0/2503763707 shutdown_connections 2023-12-07T10:18:53.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.434+0000 7f03bb5b5700 1 -- 172.21.15.119:0/2503763707 wait complete. 2023-12-07T10:18:53.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.435+0000 7f03bb5b5700 1 Processor -- start 2023-12-07T10:18:53.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.435+0000 7f03bb5b5700 1 -- start start 2023-12-07T10:18:53.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.436+0000 7f03bb5b5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03b41016c0 0x7f03b4073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:53.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.436+0000 7f03bb5b5700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03b4102080 0x7f03b4073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:53.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.436+0000 7f03bb5b5700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b4107730 0x7f03b4074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:53.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.436+0000 7f03bb5b5700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b40758d0 con 0x7f03b4107730 2023-12-07T10:18:53.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.436+0000 7f03bb5b5700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b41aa3a0 con 0x7f03b41016c0 2023-12-07T10:18:53.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.436+0000 7f03bb5b5700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b41aa560 con 0x7f03b4102080 2023-12-07T10:18:53.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.436+0000 7f03b9b52700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b4107730 0x7f03b4074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:53.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.436+0000 7f03b9b52700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b4107730 0x7f03b4074390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58660/0 (socket says 172.21.15.119:58660) 2023-12-07T10:18:53.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.436+0000 7f03b9b52700 1 -- 172.21.15.119:0/560814942 learned_addr learned my addr 172.21.15.119:0/560814942 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:53.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.436+0000 7f03b8b50700 1 --2- 172.21.15.119:0/560814942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03b4102080 0x7f03b4073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:53.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.436+0000 7f03b9351700 1 --2- 172.21.15.119:0/560814942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03b41016c0 0x7f03b4073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:53.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.436+0000 7f03b9b52700 1 -- 172.21.15.119:0/560814942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03b4102080 msgr2=0x7f03b4073e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:53.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.436+0000 7f03b9b52700 1 --2- 172.21.15.119:0/560814942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03b4102080 0x7f03b4073e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.436+0000 7f03b9b52700 1 -- 172.21.15.119:0/560814942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03b41016c0 msgr2=0x7f03b4073910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:53.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.436+0000 7f03b9b52700 1 --2- 172.21.15.119:0/560814942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03b41016c0 0x7f03b4073910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.436+0000 7f03b9b52700 1 -- 172.21.15.119:0/560814942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a40095f0 con 0x7f03b4107730 2023-12-07T10:18:53.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.437+0000 7f03b9b52700 1 --2- 172.21.15.119:0/560814942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b4107730 0x7f03b4074390 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00efc0 tx=0x7f03ac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:53.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.437+0000 7f03aa7fc700 1 -- 172.21.15.119:0/560814942 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac018070 con 0x7f03b4107730 2023-12-07T10:18:53.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.437+0000 7f03bb5b5700 1 -- 172.21.15.119:0/560814942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b41aa7e0 con 0x7f03b4107730 2023-12-07T10:18:53.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.437+0000 7f03aa7fc700 1 -- 172.21.15.119:0/560814942 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03ac00f040 con 0x7f03b4107730 2023-12-07T10:18:53.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.437+0000 7f03aa7fc700 1 -- 172.21.15.119:0/560814942 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac00cc60 con 0x7f03b4107730 2023-12-07T10:18:53.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.437+0000 7f03bb5b5700 1 -- 172.21.15.119:0/560814942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b41aace0 con 0x7f03b4107730 2023-12-07T10:18:53.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.438+0000 7f03bb5b5700 1 -- 172.21.15.119:0/560814942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b4109bc0 con 0x7f03b4107730 2023-12-07T10:18:53.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.440+0000 7f03aa7fc700 1 -- 172.21.15.119:0/560814942 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f03ac003a00 con 0x7f03b4107730 2023-12-07T10:18:53.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.441+0000 7f03aa7fc700 1 --2- 172.21.15.119:0/560814942 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f03a00a1ba0 0x7f03a00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:53.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.441+0000 7f03aa7fc700 1 -- 172.21.15.119:0/560814942 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f03ac0cd390 con 0x7f03b4107730 2023-12-07T10:18:53.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.442+0000 7f03b9351700 1 --2- 172.21.15.119:0/560814942 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f03a00a1ba0 0x7f03a00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:53.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.442+0000 7f03b9351700 1 --2- 172.21.15.119:0/560814942 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f03a00a1ba0 0x7f03a00a4060 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f03b0009be0 tx=0x7f03b0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:53.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.446+0000 7f03aa7fc700 1 -- 172.21.15.119:0/560814942 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f03ac096380 con 0x7f03b4107730 2023-12-07T10:18:53.698 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383703 got 343597383705 for osd.11 2023-12-07T10:18:53.698 DEBUG:teuthology.parallel:result is None 2023-12-07T10:18:53.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.725+0000 7f03bb5b5700 1 -- 172.21.15.119:0/560814942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f03b4074df0 con 0x7f03b4107730 2023-12-07T10:18:53.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.725+0000 7f03aa7fc700 1 -- 172.21.15.119:0/560814942 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f03ac0999a0 con 0x7f03b4107730 2023-12-07T10:18:53.727 INFO:teuthology.orchestra.run.smithi119.stdout:201863462961 2023-12-07T10:18:53.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.728+0000 7f03bb5b5700 1 -- 172.21.15.119:0/560814942 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f03a00a1ba0 msgr2=0x7f03a00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:53.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.728+0000 7f03bb5b5700 1 --2- 172.21.15.119:0/560814942 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f03a00a1ba0 0x7f03a00a4060 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f03b0009be0 tx=0x7f03b0009260 comp rx=0 tx=0).stop 2023-12-07T10:18:53.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.728+0000 7f03bb5b5700 1 -- 172.21.15.119:0/560814942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b4107730 msgr2=0x7f03b4074390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:53.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.728+0000 7f03bb5b5700 1 --2- 172.21.15.119:0/560814942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b4107730 0x7f03b4074390 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00efc0 tx=0x7f03ac00c3a0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.728+0000 7f03bb5b5700 1 -- 172.21.15.119:0/560814942 shutdown_connections 2023-12-07T10:18:53.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.728+0000 7f03bb5b5700 1 --2- 172.21.15.119:0/560814942 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f03a00a1ba0 0x7f03a00a4060 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.728+0000 7f03bb5b5700 1 --2- 172.21.15.119:0/560814942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b4107730 0x7f03b4074390 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.728+0000 7f03bb5b5700 1 --2- 172.21.15.119:0/560814942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03b41016c0 0x7f03b4073910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.728+0000 7f03bb5b5700 1 --2- 172.21.15.119:0/560814942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03b4102080 0x7f03b4073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.728+0000 7f03bb5b5700 1 -- 172.21.15.119:0/560814942 >> 172.21.15.119:0/560814942 conn(0x7f03b4078800 msgr2=0x7f03b41002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:53.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.728+0000 7f03bb5b5700 1 -- 172.21.15.119:0/560814942 shutdown_connections 2023-12-07T10:18:53.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.729+0000 7f03bb5b5700 1 -- 172.21.15.119:0/560814942 wait complete. 2023-12-07T10:18:53.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.798+0000 7f02402ee700 1 -- 172.21.15.119:0/2109482505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0238073d80 msgr2=0x7f0238074200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:53.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.798+0000 7f02402ee700 1 --2- 172.21.15.119:0/2109482505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0238073d80 0x7f0238074200 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f02280098f0 tx=0x7f0228009c00 comp rx=0 tx=0).stop 2023-12-07T10:18:53.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.798+0000 7f02402ee700 1 -- 172.21.15.119:0/2109482505 shutdown_connections 2023-12-07T10:18:53.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.798+0000 7f02402ee700 1 --2- 172.21.15.119:0/2109482505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0238112390 0x7f02381147c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.798+0000 7f02402ee700 1 --2- 172.21.15.119:0/2109482505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0238075590 0x7f0238073840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.798+0000 7f02402ee700 1 --2- 172.21.15.119:0/2109482505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0238073d80 0x7f0238074200 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.798+0000 7f02402ee700 1 -- 172.21.15.119:0/2109482505 >> 172.21.15.119:0/2109482505 conn(0x7f0238078800 msgr2=0x7f0238078c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:53.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.798+0000 7f02402ee700 1 -- 172.21.15.119:0/2109482505 shutdown_connections 2023-12-07T10:18:53.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.798+0000 7f02402ee700 1 -- 172.21.15.119:0/2109482505 wait complete. 2023-12-07T10:18:53.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.799+0000 7f02402ee700 1 Processor -- start 2023-12-07T10:18:53.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.800+0000 7f02402ee700 1 -- start start 2023-12-07T10:18:53.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.800+0000 7f02402ee700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0238073d80 0x7f023819e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:53.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.800+0000 7f02402ee700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0238075590 0x7f023819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:53.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.800+0000 7f02402ee700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0238112390 0x7f02381a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:53.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.800+0000 7f02402ee700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02381a3080 con 0x7f0238073d80 2023-12-07T10:18:53.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.800+0000 7f02402ee700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02381a31f0 con 0x7f0238112390 2023-12-07T10:18:53.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.800+0000 7f02402ee700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02381a33b0 con 0x7f0238075590 2023-12-07T10:18:53.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.801+0000 7f023d889700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0238075590 0x7f023819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:53.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.801+0000 7f023d889700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0238075590 0x7f023819e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:42154/0 (socket says 172.21.15.119:42154) 2023-12-07T10:18:53.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.801+0000 7f023d889700 1 -- 172.21.15.119:0/3411266669 learned_addr learned my addr 172.21.15.119:0/3411266669 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:53.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.801+0000 7f023e08a700 1 --2- 172.21.15.119:0/3411266669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0238073d80 0x7f023819e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:53.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.801+0000 7f023e88b700 1 --2- 172.21.15.119:0/3411266669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0238112390 0x7f02381a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:53.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.801+0000 7f023d889700 1 -- 172.21.15.119:0/3411266669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0238112390 msgr2=0x7f02381a2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:53.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.801+0000 7f023d889700 1 --2- 172.21.15.119:0/3411266669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0238112390 0x7f02381a2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.801+0000 7f023d889700 1 -- 172.21.15.119:0/3411266669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0238073d80 msgr2=0x7f023819e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:53.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.801+0000 7f023d889700 1 --2- 172.21.15.119:0/3411266669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0238073d80 0x7f023819e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.801+0000 7f023d889700 1 -- 172.21.15.119:0/3411266669 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0230009d00 con 0x7f0238075590 2023-12-07T10:18:53.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.801+0000 7f023d889700 1 --2- 172.21.15.119:0/3411266669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0238075590 0x7f023819e6c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0228005950 tx=0x7f022800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:53.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.802+0000 7f022f7fe700 1 -- 172.21.15.119:0/3411266669 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022801c070 con 0x7f0238075590 2023-12-07T10:18:53.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.802+0000 7f022f7fe700 1 -- 172.21.15.119:0/3411266669 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f022800baa0 con 0x7f0238075590 2023-12-07T10:18:53.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.802+0000 7f02402ee700 1 -- 172.21.15.119:0/3411266669 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0228009520 con 0x7f0238075590 2023-12-07T10:18:53.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.802+0000 7f022f7fe700 1 -- 172.21.15.119:0/3411266669 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02280038e0 con 0x7f0238075590 2023-12-07T10:18:53.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.802+0000 7f02402ee700 1 -- 172.21.15.119:0/3411266669 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02381aad20 con 0x7f0238075590 2023-12-07T10:18:53.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.803+0000 7f02402ee700 1 -- 172.21.15.119:0/3411266669 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0238074ed0 con 0x7f0238075590 2023-12-07T10:18:53.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.805+0000 7f022f7fe700 1 -- 172.21.15.119:0/3411266669 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f0228020030 con 0x7f0238075590 2023-12-07T10:18:53.812 INFO:teuthology.orchestra.run.smithi119.stdout:309237645342 2023-12-07T10:18:53.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.783+0000 7fc3fe8f9700 1 -- 172.21.15.119:0/884215178 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fc3f819b070 con 0x7fc3f8069970 2023-12-07T10:18:53.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.784+0000 7fc3f5ffb700 1 -- 172.21.15.119:0/884215178 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fc3e009ab90 con 0x7fc3f8069970 2023-12-07T10:18:53.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.787+0000 7fc3fe8f9700 1 -- 172.21.15.119:0/884215178 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc3d80a1c20 msgr2=0x7fc3d80a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:53.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.787+0000 7fc3fe8f9700 1 --2- 172.21.15.119:0/884215178 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc3d80a1c20 0x7fc3d80a40e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8005d90 tx=0x7fc3e8005ce0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.787+0000 7fc3fe8f9700 1 -- 172.21.15.119:0/884215178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3f8069970 msgr2=0x7fc3f819a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:53.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.787+0000 7fc3fe8f9700 1 --2- 172.21.15.119:0/884215178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3f8069970 0x7fc3f819a2a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000bd80 tx=0x7fc3e000bdb0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.788+0000 7fc3fe8f9700 1 -- 172.21.15.119:0/884215178 shutdown_connections 2023-12-07T10:18:53.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.788+0000 7fc3fe8f9700 1 --2- 172.21.15.119:0/884215178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3f8107630 0x7fc3f819e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.788+0000 7fc3fe8f9700 1 --2- 172.21.15.119:0/884215178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3f8068fb0 0x7fc3f8199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.788+0000 7fc3fe8f9700 1 --2- 172.21.15.119:0/884215178 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc3d80a1c20 0x7fc3d80a40e0 secure :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8005d90 tx=0x7fc3e8005ce0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.788+0000 7fc3fe8f9700 1 --2- 172.21.15.119:0/884215178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3f8069970 0x7fc3f819a2a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:53.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.788+0000 7fc3fe8f9700 1 -- 172.21.15.119:0/884215178 >> 172.21.15.119:0/884215178 conn(0x7fc3f80765b0 msgr2=0x7fc3f810c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:53.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.788+0000 7fc3fe8f9700 1 -- 172.21.15.119:0/884215178 shutdown_connections 2023-12-07T10:18:53.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.788+0000 7fc3fe8f9700 1 -- 172.21.15.119:0/884215178 wait complete. 2023-12-07T10:18:53.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.807+0000 7f022f7fe700 1 --2- 172.21.15.119:0/3411266669 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f02240a1c20 0x7f02240a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:53.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.807+0000 7f022f7fe700 1 -- 172.21.15.119:0/3411266669 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f02280ce740 con 0x7f0238075590 2023-12-07T10:18:53.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.808+0000 7f023e08a700 1 --2- 172.21.15.119:0/3411266669 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f02240a1c20 0x7f02240a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:53.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.809+0000 7f023e08a700 1 --2- 172.21.15.119:0/3411266669 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f02240a1c20 0x7f02240a40e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0234009c60 tx=0x7f0234009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:53.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.811+0000 7f022f7fe700 1 -- 172.21.15.119:0/3411266669 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0228097730 con 0x7f0238075590 2023-12-07T10:18:54.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.941+0000 7fa440ac5700 1 -- 172.21.15.119:0/2361860192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa43c068fb0 msgr2=0x7fa43c069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:54.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.941+0000 7fa440ac5700 1 --2- 172.21.15.119:0/2361860192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa43c068fb0 0x7fa43c069430 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa43000fb60 tx=0x7fa43000fe70 comp rx=0 tx=0).stop 2023-12-07T10:18:54.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.941+0000 7fa440ac5700 1 -- 172.21.15.119:0/2361860192 shutdown_connections 2023-12-07T10:18:54.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.941+0000 7fa440ac5700 1 --2- 172.21.15.119:0/2361860192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa43c069970 0x7fa43c10e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.941+0000 7fa440ac5700 1 --2- 172.21.15.119:0/2361860192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa43c107660 0x7fa43c107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.941+0000 7fa440ac5700 1 --2- 172.21.15.119:0/2361860192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa43c068fb0 0x7fa43c069430 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.941+0000 7fa440ac5700 1 -- 172.21.15.119:0/2361860192 >> 172.21.15.119:0/2361860192 conn(0x7fa43c0765b0 msgr2=0x7fa43c0769c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:54.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.941+0000 7fa440ac5700 1 -- 172.21.15.119:0/2361860192 shutdown_connections 2023-12-07T10:18:54.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.942+0000 7fa440ac5700 1 -- 172.21.15.119:0/2361860192 wait complete. 2023-12-07T10:18:54.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.943+0000 7fa440ac5700 1 Processor -- start 2023-12-07T10:18:54.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.943+0000 7fa440ac5700 1 -- start start 2023-12-07T10:18:54.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.944+0000 7fa440ac5700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa43c068fb0 0x7fa43c1ac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:54.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.944+0000 7fa440ac5700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa43c069970 0x7fa43c1acb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:54.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.944+0000 7fa440ac5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa43c107660 0x7fa43c1a6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:54.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.944+0000 7fa440ac5700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c1a6d80 con 0x7fa43c068fb0 2023-12-07T10:18:54.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.944+0000 7fa440ac5700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c1a6ef0 con 0x7fa43c107660 2023-12-07T10:18:54.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.944+0000 7fa440ac5700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c1a70b0 con 0x7fa43c069970 2023-12-07T10:18:54.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.945+0000 7fa43a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa43c068fb0 0x7fa43c1ac5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:54.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.945+0000 7fa43a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa43c068fb0 0x7fa43c1ac5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58724/0 (socket says 172.21.15.119:58724) 2023-12-07T10:18:54.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.945+0000 7fa43a59c700 1 -- 172.21.15.119:0/3864890360 learned_addr learned my addr 172.21.15.119:0/3864890360 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:54.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.945+0000 7fa43ad9d700 1 --2- 172.21.15.119:0/3864890360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa43c107660 0x7fa43c1a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:54.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.945+0000 7fa43a59c700 1 -- 172.21.15.119:0/3864890360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa43c069970 msgr2=0x7fa43c1acb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:54.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.945+0000 7fa43a59c700 1 --2- 172.21.15.119:0/3864890360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa43c069970 0x7fa43c1acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.945+0000 7fa43a59c700 1 -- 172.21.15.119:0/3864890360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa43c107660 msgr2=0x7fa43c1a6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:54.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.945+0000 7fa43a59c700 1 --2- 172.21.15.119:0/3864890360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa43c107660 0x7fa43c1a6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.945+0000 7fa43a59c700 1 -- 172.21.15.119:0/3864890360 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa43000f790 con 0x7fa43c068fb0 2023-12-07T10:18:54.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.945+0000 7fa43ad9d700 1 --2- 172.21.15.119:0/3864890360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa43c107660 0x7fa43c1a6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:18:54.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.946+0000 7fa43a59c700 1 --2- 172.21.15.119:0/3864890360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa43c068fb0 0x7fa43c1ac5e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa42c00e9c0 tx=0x7fa42c00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:54.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.946+0000 7fa42b7fe700 1 -- 172.21.15.119:0/3864890360 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42c00cbd0 con 0x7fa43c068fb0 2023-12-07T10:18:54.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.946+0000 7fa42b7fe700 1 -- 172.21.15.119:0/3864890360 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa42c00cd30 con 0x7fa43c068fb0 2023-12-07T10:18:54.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.946+0000 7fa42b7fe700 1 -- 172.21.15.119:0/3864890360 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42c005580 con 0x7fa43c068fb0 2023-12-07T10:18:54.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.947+0000 7fa440ac5700 1 -- 172.21.15.119:0/3864890360 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa43c1a7390 con 0x7fa43c068fb0 2023-12-07T10:18:54.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.947+0000 7fa440ac5700 1 -- 172.21.15.119:0/3864890360 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa43c1b35d0 con 0x7fa43c068fb0 2023-12-07T10:18:54.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.949+0000 7fa440ac5700 1 -- 172.21.15.119:0/3864890360 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa43c109ba0 con 0x7fa43c068fb0 2023-12-07T10:18:54.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.951+0000 7fa42b7fe700 1 -- 172.21.15.119:0/3864890360 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7fa42c004750 con 0x7fa43c068fb0 2023-12-07T10:18:54.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.952+0000 7fa42b7fe700 1 --2- 172.21.15.119:0/3864890360 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa4240a1ca0 0x7fa4240a4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:54.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.953+0000 7fa42b7fe700 1 -- 172.21.15.119:0/3864890360 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fa42c014070 con 0x7fa43c068fb0 2023-12-07T10:18:54.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.953+0000 7fa439d9b700 1 --2- 172.21.15.119:0/3864890360 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa4240a1ca0 0x7fa4240a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:54.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.954+0000 7fa439d9b700 1 --2- 172.21.15.119:0/3864890360 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa4240a1ca0 0x7fa4240a4160 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa43000fb30 tx=0x7fa4300096e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:54.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.960+0000 7fa42b7fe700 1 -- 172.21.15.119:0/3864890360 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa42c096ba0 con 0x7fa43c068fb0 2023-12-07T10:18:54.022 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:53 smithi119 ceph-mon[97908]: pgmap v317: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-07T10:18:54.023 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:53 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/560814942' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-07T10:18:54.029 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:53 smithi153 ceph-mon[133625]: pgmap v317: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-07T10:18:54.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:53 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/560814942' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-07T10:18:54.046 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149756 got 111669149758 for osd.3 2023-12-07T10:18:54.046 DEBUG:teuthology.parallel:result is None 2023-12-07T10:18:54.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.082+0000 7f02402ee700 1 -- 172.21.15.119:0/3411266669 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f023819f490 con 0x7f0238075590 2023-12-07T10:18:54.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.082+0000 7f022f7fe700 1 -- 172.21.15.119:0/3411266669 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f022809ad50 con 0x7f0238075590 2023-12-07T10:18:54.084 INFO:teuthology.orchestra.run.smithi119.stdout:146028888123 2023-12-07T10:18:54.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.086+0000 7f02402ee700 1 -- 172.21.15.119:0/3411266669 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f02240a1c20 msgr2=0x7f02240a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:54.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.086+0000 7f02402ee700 1 --2- 172.21.15.119:0/3411266669 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f02240a1c20 0x7f02240a40e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0234009c60 tx=0x7f0234009260 comp rx=0 tx=0).stop 2023-12-07T10:18:54.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.086+0000 7f02402ee700 1 -- 172.21.15.119:0/3411266669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0238075590 msgr2=0x7f023819e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:54.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.086+0000 7f02402ee700 1 --2- 172.21.15.119:0/3411266669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0238075590 0x7f023819e6c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0228005950 tx=0x7f022800fa00 comp rx=0 tx=0).stop 2023-12-07T10:18:54.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.086+0000 7f02402ee700 1 -- 172.21.15.119:0/3411266669 shutdown_connections 2023-12-07T10:18:54.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.086+0000 7f02402ee700 1 --2- 172.21.15.119:0/3411266669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0238112390 0x7f02381a2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.086+0000 7f02402ee700 1 --2- 172.21.15.119:0/3411266669 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f02240a1c20 0x7f02240a40e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.086+0000 7f02402ee700 1 --2- 172.21.15.119:0/3411266669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0238073d80 0x7f023819e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.086+0000 7f02402ee700 1 --2- 172.21.15.119:0/3411266669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0238075590 0x7f023819e6c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.086+0000 7f02402ee700 1 -- 172.21.15.119:0/3411266669 >> 172.21.15.119:0/3411266669 conn(0x7f0238078800 msgr2=0x7f0238111060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:54.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.086+0000 7f02402ee700 1 -- 172.21.15.119:0/3411266669 shutdown_connections 2023-12-07T10:18:54.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.087+0000 7f02402ee700 1 -- 172.21.15.119:0/3411266669 wait complete. 2023-12-07T10:18:54.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.226+0000 7f73b484f700 1 -- 172.21.15.119:0/562193077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73a809d560 msgr2=0x7f73a80a0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:54.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.226+0000 7f73b484f700 1 --2- 172.21.15.119:0/562193077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73a809d560 0x7f73a80a0fa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f73a401a9a0 tx=0x7f73a401acb0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.227+0000 7f73b484f700 1 -- 172.21.15.119:0/562193077 shutdown_connections 2023-12-07T10:18:54.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.227+0000 7f73b484f700 1 --2- 172.21.15.119:0/562193077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73a80a1680 0x7f73a80a9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.227+0000 7f73b484f700 1 --2- 172.21.15.119:0/562193077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73a809cbb0 0x7f73a809cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.227+0000 7f73b484f700 1 --2- 172.21.15.119:0/562193077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73a809d560 0x7f73a80a0fa0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.227+0000 7f73b484f700 1 -- 172.21.15.119:0/562193077 >> 172.21.15.119:0/562193077 conn(0x7f73a80092e0 msgr2=0x7f73a80096f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:54.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.227+0000 7f73b484f700 1 -- 172.21.15.119:0/562193077 shutdown_connections 2023-12-07T10:18:54.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.227+0000 7f73b484f700 1 -- 172.21.15.119:0/562193077 wait complete. 2023-12-07T10:18:54.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.229+0000 7f73b484f700 1 Processor -- start 2023-12-07T10:18:54.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.229+0000 7f73b484f700 1 -- start start 2023-12-07T10:18:54.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.230+0000 7f73b484f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73a809cbb0 0x7f73a8014ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:54.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.230+0000 7f73b484f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73a809d560 0x7f73a8015440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:54.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.230+0000 7f73b484f700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73a80a1680 0x7f73a800e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:54.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.230+0000 7f73b484f700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a800e800 con 0x7f73a80a1680 2023-12-07T10:18:54.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.230+0000 7f73b484f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a800e970 con 0x7f73a809cbb0 2023-12-07T10:18:54.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.230+0000 7f73b484f700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a800eb30 con 0x7f73a809d560 2023-12-07T10:18:54.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.230+0000 7f73af59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73a80a1680 0x7f73a800e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:54.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.230+0000 7f73af59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73a80a1680 0x7f73a800e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58746/0 (socket says 172.21.15.119:58746) 2023-12-07T10:18:54.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.230+0000 7f73af59e700 1 -- 172.21.15.119:0/3229659963 learned_addr learned my addr 172.21.15.119:0/3229659963 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:54.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.231+0000 7f73af59e700 1 -- 172.21.15.119:0/3229659963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73a809d560 msgr2=0x7f73a8015440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-07T10:18:54.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.231+0000 7f73af59e700 1 --2- 172.21.15.119:0/3229659963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73a809d560 0x7f73a8015440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.231+0000 7f73af59e700 1 -- 172.21.15.119:0/3229659963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73a809cbb0 msgr2=0x7f73a8014ec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-07T10:18:54.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.231+0000 7f73af59e700 1 --2- 172.21.15.119:0/3229659963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73a809cbb0 0x7f73a8014ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.231+0000 7f73af59e700 1 -- 172.21.15.119:0/3229659963 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a401a600 con 0x7f73a80a1680 2023-12-07T10:18:54.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.231+0000 7f73af59e700 1 --2- 172.21.15.119:0/3229659963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73a80a1680 0x7f73a800e030 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f73b004f5a0 tx=0x7f73b006bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:54.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.232+0000 7f7397fff700 1 -- 172.21.15.119:0/3229659963 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b007a070 con 0x7f73a80a1680 2023-12-07T10:18:54.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.232+0000 7f7397fff700 1 -- 172.21.15.119:0/3229659963 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73b0070cb0 con 0x7f73a80a1680 2023-12-07T10:18:54.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.232+0000 7f7397fff700 1 -- 172.21.15.119:0/3229659963 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b0076c60 con 0x7f73a80a1680 2023-12-07T10:18:54.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.232+0000 7f73b484f700 1 -- 172.21.15.119:0/3229659963 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a800ee10 con 0x7f73a80a1680 2023-12-07T10:18:54.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.232+0000 7f73b484f700 1 -- 172.21.15.119:0/3229659963 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a8012cf0 con 0x7f73a80a1680 2023-12-07T10:18:54.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.878+0000 7f1b7d98b700 1 -- 172.21.15.119:0/2119435297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b78075590 msgr2=0x7f1b78073840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:54.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.878+0000 7f1b7d98b700 1 --2- 172.21.15.119:0/2119435297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b78075590 0x7f1b78073840 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009990 tx=0x7f1b68009ca0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.878+0000 7f1b7d98b700 1 -- 172.21.15.119:0/2119435297 shutdown_connections 2023-12-07T10:18:54.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.878+0000 7f1b7d98b700 1 --2- 172.21.15.119:0/2119435297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b78112390 0x7f1b781147c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.878+0000 7f1b7d98b700 1 --2- 172.21.15.119:0/2119435297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b78075590 0x7f1b78073840 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.878+0000 7f1b7d98b700 1 --2- 172.21.15.119:0/2119435297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b78073d80 0x7f1b78074200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.878+0000 7f1b7d98b700 1 -- 172.21.15.119:0/2119435297 >> 172.21.15.119:0/2119435297 conn(0x7f1b78078800 msgr2=0x7f1b78078c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:54.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.879+0000 7f1b7d98b700 1 -- 172.21.15.119:0/2119435297 shutdown_connections 2023-12-07T10:18:54.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.879+0000 7f1b7d98b700 1 -- 172.21.15.119:0/2119435297 wait complete. 2023-12-07T10:18:54.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.880+0000 7f1b7d98b700 1 Processor -- start 2023-12-07T10:18:54.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.880+0000 7f1b7d98b700 1 -- start start 2023-12-07T10:18:54.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.880+0000 7f1b7d98b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b78073d80 0x7f1b7819e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:54.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.880+0000 7f1b7d98b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b78075590 0x7f1b7819e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:54.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.880+0000 7f1b7d98b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b78112390 0x7f1b781a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:54.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.880+0000 7f1b7d98b700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b781a3020 con 0x7f1b78073d80 2023-12-07T10:18:54.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.880+0000 7f1b7d98b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b781a3190 con 0x7f1b78075590 2023-12-07T10:18:54.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.880+0000 7f1b7d98b700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b781a3350 con 0x7f1b78112390 2023-12-07T10:18:54.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.881+0000 7f1b76ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b78073d80 0x7f1b7819e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:54.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.881+0000 7f1b76ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b78073d80 0x7f1b7819e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58710/0 (socket says 172.21.15.119:58710) 2023-12-07T10:18:54.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.881+0000 7f1b76ffd700 1 -- 172.21.15.119:0/2564943684 learned_addr learned my addr 172.21.15.119:0/2564943684 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:54.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.881+0000 7f1b767fc700 1 --2- 172.21.15.119:0/2564943684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b78075590 0x7f1b7819e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:54.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.881+0000 7f1b777fe700 1 --2- 172.21.15.119:0/2564943684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b78112390 0x7f1b781a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:54.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.881+0000 7f1b777fe700 1 -- 172.21.15.119:0/2564943684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b78075590 msgr2=0x7f1b7819e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:54.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.881+0000 7f1b777fe700 1 --2- 172.21.15.119:0/2564943684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b78075590 0x7f1b7819e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.881+0000 7f1b777fe700 1 -- 172.21.15.119:0/2564943684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b78073d80 msgr2=0x7f1b7819e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:54.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.881+0000 7f1b777fe700 1 --2- 172.21.15.119:0/2564943684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b78073d80 0x7f1b7819e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.881+0000 7f1b777fe700 1 -- 172.21.15.119:0/2564943684 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b680095f0 con 0x7f1b78112390 2023-12-07T10:18:54.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.881+0000 7f1b76ffd700 1 --2- 172.21.15.119:0/2564943684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b78073d80 0x7f1b7819e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:18:54.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.882+0000 7f1b777fe700 1 --2- 172.21.15.119:0/2564943684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b78112390 0x7f1b781a2a10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1b7000ef90 tx=0x7f1b7000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:54.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.882+0000 7f1b7c989700 1 -- 172.21.15.119:0/2564943684 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b70018070 con 0x7f1b78112390 2023-12-07T10:18:54.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.882+0000 7f1b7d98b700 1 -- 172.21.15.119:0/2564943684 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b781a35d0 con 0x7f1b78112390 2023-12-07T10:18:54.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.882+0000 7f1b7c989700 1 -- 172.21.15.119:0/2564943684 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b7000f040 con 0x7f1b78112390 2023-12-07T10:18:54.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.882+0000 7f1b7c989700 1 -- 172.21.15.119:0/2564943684 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7000cb60 con 0x7f1b78112390 2023-12-07T10:18:54.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.882+0000 7f1b7d98b700 1 -- 172.21.15.119:0/2564943684 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b781aae90 con 0x7f1b78112390 2023-12-07T10:18:54.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.883+0000 7f1b7d98b700 1 -- 172.21.15.119:0/2564943684 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b78074ed0 con 0x7f1b78112390 2023-12-07T10:18:54.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.887+0000 7f1b7c989700 1 -- 172.21.15.119:0/2564943684 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f1b7000cd40 con 0x7f1b78112390 2023-12-07T10:18:54.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.888+0000 7f1b7c989700 1 --2- 172.21.15.119:0/2564943684 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1b600a1ba0 0x7f1b600a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:54.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.888+0000 7f1b7c989700 1 -- 172.21.15.119:0/2564943684 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f1b700cd4e0 con 0x7f1b78112390 2023-12-07T10:18:54.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.889+0000 7f1b76ffd700 1 --2- 172.21.15.119:0/2564943684 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1b600a1ba0 0x7f1b600a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:54.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.890+0000 7f1b76ffd700 1 --2- 172.21.15.119:0/2564943684 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1b600a1ba0 0x7f1b600a4060 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1b68006010 tx=0x7f1b6800b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:54.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:53.893+0000 7f1b7c989700 1 -- 172.21.15.119:0/2564943684 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1b70096530 con 0x7f1b78112390 2023-12-07T10:18:54.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.237+0000 7f73b484f700 1 -- 172.21.15.119:0/3229659963 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a809db70 con 0x7f73a80a1680 2023-12-07T10:18:54.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.243+0000 7f7397fff700 1 -- 172.21.15.119:0/3229659963 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f73b007f410 con 0x7f73a80a1680 2023-12-07T10:18:54.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.246+0000 7f7397fff700 1 --2- 172.21.15.119:0/3229659963 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f73980a1d70 0x7f73980a4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:54.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.246+0000 7f7397fff700 1 -- 172.21.15.119:0/3229659963 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f73b012fb00 con 0x7f73a80a1680 2023-12-07T10:18:54.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.246+0000 7f73aed9d700 1 --2- 172.21.15.119:0/3229659963 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f73980a1d70 0x7f73980a4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:54.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.247+0000 7f7397fff700 1 -- 172.21.15.119:0/3229659963 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f73b00f8c00 con 0x7f73a80a1680 2023-12-07T10:18:54.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.247+0000 7f73aed9d700 1 --2- 172.21.15.119:0/3229659963 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f73980a1d70 0x7f73980a4230 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f73a000a9b0 tx=0x7f73a0005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:54.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.248+0000 7f1b7d98b700 1 -- 172.21.15.119:0/2564943684 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1b7819f430 con 0x7f1b78112390 2023-12-07T10:18:54.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.250+0000 7f1b7c989700 1 -- 172.21.15.119:0/2564943684 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1b70099b50 con 0x7f1b78112390 2023-12-07T10:18:54.253 INFO:teuthology.orchestra.run.smithi119.stdout:34359738445 2023-12-07T10:18:54.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.254+0000 7f1b7d98b700 1 -- 172.21.15.119:0/2564943684 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1b600a1ba0 msgr2=0x7f1b600a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:54.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.254+0000 7f1b7d98b700 1 --2- 172.21.15.119:0/2564943684 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1b600a1ba0 0x7f1b600a4060 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1b68006010 tx=0x7f1b6800b260 comp rx=0 tx=0).stop 2023-12-07T10:18:54.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.254+0000 7f1b7d98b700 1 -- 172.21.15.119:0/2564943684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b78112390 msgr2=0x7f1b781a2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:54.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.254+0000 7f1b7d98b700 1 --2- 172.21.15.119:0/2564943684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b78112390 0x7f1b781a2a10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1b7000ef90 tx=0x7f1b7000c3a0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.254+0000 7f1b7d98b700 1 -- 172.21.15.119:0/2564943684 shutdown_connections 2023-12-07T10:18:54.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.254+0000 7f1b7d98b700 1 --2- 172.21.15.119:0/2564943684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b78112390 0x7f1b781a2a10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.254+0000 7f1b7d98b700 1 --2- 172.21.15.119:0/2564943684 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1b600a1ba0 0x7f1b600a4060 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.254+0000 7f1b7d98b700 1 --2- 172.21.15.119:0/2564943684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b78073d80 0x7f1b7819e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.254+0000 7f1b7d98b700 1 --2- 172.21.15.119:0/2564943684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b78075590 0x7f1b7819e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.254+0000 7f1b7d98b700 1 -- 172.21.15.119:0/2564943684 >> 172.21.15.119:0/2564943684 conn(0x7f1b78078800 msgr2=0x7f1b78111000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:54.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.255+0000 7f1b7d98b700 1 -- 172.21.15.119:0/2564943684 shutdown_connections 2023-12-07T10:18:54.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.255+0000 7f1b7d98b700 1 -- 172.21.15.119:0/2564943684 wait complete. 2023-12-07T10:18:54.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:53 smithi159 ceph-mon[134726]: pgmap v317: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-07T10:18:54.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:53 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/560814942' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-07T10:18:54.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.269+0000 7fa440ac5700 1 -- 172.21.15.119:0/3864890360 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fa43c102d30 con 0x7fa43c068fb0 2023-12-07T10:18:54.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.270+0000 7fa42b7fe700 1 -- 172.21.15.119:0/3864890360 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa42c09a1c0 con 0x7fa43c068fb0 2023-12-07T10:18:54.272 INFO:teuthology.orchestra.run.smithi119.stdout:287762808867 2023-12-07T10:18:54.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.273+0000 7fa440ac5700 1 -- 172.21.15.119:0/3864890360 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa4240a1ca0 msgr2=0x7fa4240a4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:54.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.273+0000 7fa440ac5700 1 --2- 172.21.15.119:0/3864890360 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa4240a1ca0 0x7fa4240a4160 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa43000fb30 tx=0x7fa4300096e0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.273+0000 7fa440ac5700 1 -- 172.21.15.119:0/3864890360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa43c068fb0 msgr2=0x7fa43c1ac5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:54.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.273+0000 7fa440ac5700 1 --2- 172.21.15.119:0/3864890360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa43c068fb0 0x7fa43c1ac5e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa42c00e9c0 tx=0x7fa42c00ed80 comp rx=0 tx=0).stop 2023-12-07T10:18:54.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.273+0000 7fa440ac5700 1 -- 172.21.15.119:0/3864890360 shutdown_connections 2023-12-07T10:18:54.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.273+0000 7fa440ac5700 1 --2- 172.21.15.119:0/3864890360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa43c107660 0x7fa43c1a6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.273+0000 7fa440ac5700 1 --2- 172.21.15.119:0/3864890360 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa4240a1ca0 0x7fa4240a4160 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.273+0000 7fa440ac5700 1 --2- 172.21.15.119:0/3864890360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa43c068fb0 0x7fa43c1ac5e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.273+0000 7fa440ac5700 1 --2- 172.21.15.119:0/3864890360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa43c069970 0x7fa43c1acb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.273+0000 7fa440ac5700 1 -- 172.21.15.119:0/3864890360 >> 172.21.15.119:0/3864890360 conn(0x7fa43c0765b0 msgr2=0x7fa43c0ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:54.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.274+0000 7fa440ac5700 1 -- 172.21.15.119:0/3864890360 shutdown_connections 2023-12-07T10:18:54.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.274+0000 7fa440ac5700 1 -- 172.21.15.119:0/3864890360 wait complete. 2023-12-07T10:18:54.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.473+0000 7fe023a23700 1 -- 172.21.15.119:0/2660636884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe01c10e070 msgr2=0x7fe01c110460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:54.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.473+0000 7fe023a23700 1 --2- 172.21.15.119:0/2660636884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe01c10e070 0x7fe01c110460 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe01400b0a0 tx=0x7fe01400b3b0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.473+0000 7fe023a23700 1 -- 172.21.15.119:0/2660636884 shutdown_connections 2023-12-07T10:18:54.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.473+0000 7fe023a23700 1 --2- 172.21.15.119:0/2660636884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe01c10e070 0x7fe01c110460 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.473+0000 7fe023a23700 1 --2- 172.21.15.119:0/2660636884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe01c073840 0x7fe01c073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.473+0000 7fe023a23700 1 --2- 172.21.15.119:0/2660636884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe01c074160 0x7fe01c10db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.473+0000 7fe023a23700 1 -- 172.21.15.119:0/2660636884 >> 172.21.15.119:0/2660636884 conn(0x7fe01c078800 msgr2=0x7fe01c078c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:54.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.473+0000 7fe023a23700 1 -- 172.21.15.119:0/2660636884 shutdown_connections 2023-12-07T10:18:54.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.473+0000 7fe023a23700 1 -- 172.21.15.119:0/2660636884 wait complete. 2023-12-07T10:18:54.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.474+0000 7fe023a23700 1 Processor -- start 2023-12-07T10:18:54.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.474+0000 7fe023a23700 1 -- start start 2023-12-07T10:18:54.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.475+0000 7fe023a23700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe01c073840 0x7fe01c199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:54.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.475+0000 7fe023a23700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe01c074160 0x7fe01c19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:54.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.475+0000 7fe023a23700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe01c10e070 0x7fe01c19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:54.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.475+0000 7fe023a23700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c19ed10 con 0x7fe01c073840 2023-12-07T10:18:54.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.475+0000 7fe023a23700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c19ee80 con 0x7fe01c10e070 2023-12-07T10:18:54.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.475+0000 7fe023a23700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c19f040 con 0x7fe01c074160 2023-12-07T10:18:54.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.475+0000 7fe0217bf700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe01c073840 0x7fe01c199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:54.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.475+0000 7fe0217bf700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe01c073840 0x7fe01c199e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58754/0 (socket says 172.21.15.119:58754) 2023-12-07T10:18:54.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.475+0000 7fe0217bf700 1 -- 172.21.15.119:0/1384545877 learned_addr learned my addr 172.21.15.119:0/1384545877 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:54.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.475+0000 7fe021fc0700 1 --2- 172.21.15.119:0/1384545877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe01c10e070 0x7fe01c19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:54.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.475+0000 7fe020fbe700 1 --2- 172.21.15.119:0/1384545877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe01c074160 0x7fe01c19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:54.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.475+0000 7fe0217bf700 1 -- 172.21.15.119:0/1384545877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe01c074160 msgr2=0x7fe01c19a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:54.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.475+0000 7fe0217bf700 1 --2- 172.21.15.119:0/1384545877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe01c074160 0x7fe01c19a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.475+0000 7fe0217bf700 1 -- 172.21.15.119:0/1384545877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe01c10e070 msgr2=0x7fe01c19e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:54.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.475+0000 7fe0217bf700 1 --2- 172.21.15.119:0/1384545877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe01c10e070 0x7fe01c19e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.475+0000 7fe0217bf700 1 -- 172.21.15.119:0/1384545877 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe014009d00 con 0x7fe01c073840 2023-12-07T10:18:54.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.475+0000 7fe021fc0700 1 --2- 172.21.15.119:0/1384545877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe01c10e070 0x7fe01c19e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:18:54.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.476+0000 7fe0217bf700 1 --2- 172.21.15.119:0/1384545877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe01c073840 0x7fe01c199e60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe01800e9a0 tx=0x7fe01800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:54.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.476+0000 7fe0127fc700 1 -- 172.21.15.119:0/1384545877 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe018009790 con 0x7fe01c073840 2023-12-07T10:18:54.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.476+0000 7fe023a23700 1 -- 172.21.15.119:0/1384545877 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01c19f320 con 0x7fe01c073840 2023-12-07T10:18:54.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.476+0000 7fe0127fc700 1 -- 172.21.15.119:0/1384545877 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe018010430 con 0x7fe01c073840 2023-12-07T10:18:54.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.476+0000 7fe0127fc700 1 -- 172.21.15.119:0/1384545877 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe018018680 con 0x7fe01c073840 2023-12-07T10:18:54.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.476+0000 7fe023a23700 1 -- 172.21.15.119:0/1384545877 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01c1a6b80 con 0x7fe01c073840 2023-12-07T10:18:54.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.477+0000 7fe023a23700 1 -- 172.21.15.119:0/1384545877 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe01c109280 con 0x7fe01c073840 2023-12-07T10:18:54.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.479+0000 7fe0127fc700 1 -- 172.21.15.119:0/1384545877 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7fe0180105a0 con 0x7fe01c073840 2023-12-07T10:18:54.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.481+0000 7fe0127fc700 1 --2- 172.21.15.119:0/1384545877 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe0080a1ba0 0x7fe0080a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:54.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.481+0000 7fe0127fc700 1 -- 172.21.15.119:0/1384545877 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fe018014070 con 0x7fe01c073840 2023-12-07T10:18:54.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.481+0000 7fe020fbe700 1 --2- 172.21.15.119:0/1384545877 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe0080a1ba0 0x7fe0080a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:54.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.481+0000 7fe020fbe700 1 --2- 172.21.15.119:0/1384545877 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe0080a1ba0 0x7fe0080a4060 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe01c19b370 tx=0x7fe00c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:54.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.489+0000 7fe0127fc700 1 -- 172.21.15.119:0/1384545877 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe0180522f0 con 0x7fe01c073840 2023-12-07T10:18:54.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.519+0000 7f73b484f700 1 -- 172.21.15.119:0/3229659963 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f73a800f480 con 0x7f73a80a1680 2023-12-07T10:18:54.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.520+0000 7f7397fff700 1 -- 172.21.15.119:0/3229659963 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f73b00fc220 con 0x7f73a80a1680 2023-12-07T10:18:54.522 INFO:teuthology.orchestra.run.smithi119.stdout:77309411397 2023-12-07T10:18:54.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.523+0000 7f73b484f700 1 -- 172.21.15.119:0/3229659963 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f73980a1d70 msgr2=0x7f73980a4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:54.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.523+0000 7f73b484f700 1 --2- 172.21.15.119:0/3229659963 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f73980a1d70 0x7f73980a4230 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f73a000a9b0 tx=0x7f73a0005c90 comp rx=0 tx=0).stop 2023-12-07T10:18:54.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.523+0000 7f73b484f700 1 -- 172.21.15.119:0/3229659963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73a80a1680 msgr2=0x7f73a800e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:54.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.523+0000 7f73b484f700 1 --2- 172.21.15.119:0/3229659963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73a80a1680 0x7f73a800e030 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f73b004f5a0 tx=0x7f73b006bcd0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.523+0000 7f73b484f700 1 -- 172.21.15.119:0/3229659963 shutdown_connections 2023-12-07T10:18:54.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.523+0000 7f73b484f700 1 --2- 172.21.15.119:0/3229659963 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f73980a1d70 0x7f73980a4230 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.523+0000 7f73b484f700 1 --2- 172.21.15.119:0/3229659963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73a80a1680 0x7f73a800e030 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.523+0000 7f73b484f700 1 --2- 172.21.15.119:0/3229659963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73a809cbb0 0x7f73a8014ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.523+0000 7f73b484f700 1 --2- 172.21.15.119:0/3229659963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73a809d560 0x7f73a8015440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.523+0000 7f73b484f700 1 -- 172.21.15.119:0/3229659963 >> 172.21.15.119:0/3229659963 conn(0x7f73a80092e0 msgr2=0x7f73a809aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:54.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.523+0000 7f73b484f700 1 -- 172.21.15.119:0/3229659963 shutdown_connections 2023-12-07T10:18:54.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.524+0000 7f73b484f700 1 -- 172.21.15.119:0/3229659963 wait complete. 2023-12-07T10:18:54.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.716+0000 7fe023a23700 1 -- 172.21.15.119:0/1384545877 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe01c04f2d0 con 0x7fe01c073840 2023-12-07T10:18:54.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.716+0000 7fe0127fc700 1 -- 172.21.15.119:0/1384545877 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe018096ba0 con 0x7fe01c073840 2023-12-07T10:18:54.718 INFO:teuthology.orchestra.run.smithi119.stdout:55834574921 2023-12-07T10:18:54.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.718+0000 7fe023a23700 1 -- 172.21.15.119:0/1384545877 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe0080a1ba0 msgr2=0x7fe0080a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:54.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.718+0000 7fe023a23700 1 --2- 172.21.15.119:0/1384545877 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe0080a1ba0 0x7fe0080a4060 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe01c19b370 tx=0x7fe00c009260 comp rx=0 tx=0).stop 2023-12-07T10:18:54.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.718+0000 7fe023a23700 1 -- 172.21.15.119:0/1384545877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe01c073840 msgr2=0x7fe01c199e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:54.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.718+0000 7fe023a23700 1 --2- 172.21.15.119:0/1384545877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe01c073840 0x7fe01c199e60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe01800e9a0 tx=0x7fe01800ecb0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.718+0000 7fe023a23700 1 -- 172.21.15.119:0/1384545877 shutdown_connections 2023-12-07T10:18:54.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.718+0000 7fe023a23700 1 --2- 172.21.15.119:0/1384545877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe01c10e070 0x7fe01c19e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.718+0000 7fe023a23700 1 --2- 172.21.15.119:0/1384545877 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe0080a1ba0 0x7fe0080a4060 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.718+0000 7fe023a23700 1 --2- 172.21.15.119:0/1384545877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe01c073840 0x7fe01c199e60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.718+0000 7fe023a23700 1 --2- 172.21.15.119:0/1384545877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe01c074160 0x7fe01c19a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:54.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.718+0000 7fe023a23700 1 -- 172.21.15.119:0/1384545877 >> 172.21.15.119:0/1384545877 conn(0x7fe01c078800 msgr2=0x7fe01c0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:54.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.719+0000 7fe023a23700 1 -- 172.21.15.119:0/1384545877 shutdown_connections 2023-12-07T10:18:54.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:54.719+0000 7fe023a23700 1 -- 172.21.15.119:0/1384545877 wait complete. 2023-12-07T10:18:54.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:54 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/884215178' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-12-07T10:18:54.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:54 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3411266669' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-07T10:18:54.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:54 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2564943684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-07T10:18:54.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:54 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3864890360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-12-07T10:18:54.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:54 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3229659963' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-07T10:18:54.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:54 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1384545877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-07T10:18:55.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:54 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/884215178' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-12-07T10:18:55.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:54 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3411266669' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-07T10:18:55.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:54 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2564943684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-07T10:18:55.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:54 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3864890360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-12-07T10:18:55.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:54 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3229659963' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-07T10:18:55.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:54 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1384545877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-07T10:18:55.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:54 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/884215178' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-12-07T10:18:55.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:54 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3411266669' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-07T10:18:55.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:54 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2564943684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-07T10:18:55.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:54 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3864890360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-12-07T10:18:55.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:54 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3229659963' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-07T10:18:55.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:54 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1384545877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-07T10:18:55.491 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574918 got 55834574921 for osd.1 2023-12-07T10:18:55.523 DEBUG:teuthology.parallel:result is None 2023-12-07T10:18:55.555 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888120 got 146028888123 for osd.4 2023-12-07T10:18:55.556 DEBUG:teuthology.parallel:result is None 2023-12-07T10:18:55.923 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:55 smithi119 ceph-mon[97908]: pgmap v318: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2023-12-07T10:18:56.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:55 smithi159 ceph-mon[134726]: pgmap v318: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2023-12-07T10:18:56.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:55 smithi153 ceph-mon[133625]: pgmap v318: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2023-12-07T10:18:56.482 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462959 got 201863462961 for osd.6 2023-12-07T10:18:56.483 DEBUG:teuthology.parallel:result is None 2023-12-07T10:18:56.709 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645339 got 309237645342 for osd.10 2023-12-07T10:18:56.709 DEBUG:teuthology.parallel:result is None 2023-12-07T10:18:56.863 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808864 got 287762808867 for osd.9 2023-12-07T10:18:56.863 DEBUG:teuthology.parallel:result is None 2023-12-07T10:18:56.881 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411394 got 77309411397 for osd.2 2023-12-07T10:18:56.881 DEBUG:teuthology.parallel:result is None 2023-12-07T10:18:57.086 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738441 got 34359738445 for osd.0 2023-12-07T10:18:57.086 DEBUG:teuthology.parallel:result is None 2023-12-07T10:18:57.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.158+0000 7fe0cb93d700 1 -- 172.21.15.119:0/1529162435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c41016c0 msgr2=0x7fe0c4101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:57.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.158+0000 7fe0cb93d700 1 --2- 172.21.15.119:0/1529162435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c41016c0 0x7fe0c4101b40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009990 tx=0x7fe0b4009ca0 comp rx=0 tx=0).stop 2023-12-07T10:18:57.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.158+0000 7fe0cb93d700 1 -- 172.21.15.119:0/1529162435 shutdown_connections 2023-12-07T10:18:57.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.158+0000 7fe0cb93d700 1 --2- 172.21.15.119:0/1529162435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0c4102080 0x7fe0c410e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:57.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.158+0000 7fe0cb93d700 1 --2- 172.21.15.119:0/1529162435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0c4107730 0x7fe0c4107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:57.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.158+0000 7fe0cb93d700 1 --2- 172.21.15.119:0/1529162435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c41016c0 0x7fe0c4101b40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:57.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.158+0000 7fe0cb93d700 1 -- 172.21.15.119:0/1529162435 >> 172.21.15.119:0/1529162435 conn(0x7fe0c4078800 msgr2=0x7fe0c4078c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:57.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.158+0000 7fe0cb93d700 1 -- 172.21.15.119:0/1529162435 shutdown_connections 2023-12-07T10:18:57.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.158+0000 7fe0cb93d700 1 -- 172.21.15.119:0/1529162435 wait complete. 2023-12-07T10:18:57.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.159+0000 7fe0cb93d700 1 Processor -- start 2023-12-07T10:18:57.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.159+0000 7fe0cb93d700 1 -- start start 2023-12-07T10:18:57.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.160+0000 7fe0cb93d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c41016c0 0x7fe0c419f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:57.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.160+0000 7fe0cb93d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0c4102080 0x7fe0c419fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:57.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.160+0000 7fe0cb93d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0c4107730 0x7fe0c4199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:57.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.160+0000 7fe0cb93d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c419a0a0 con 0x7fe0c41016c0 2023-12-07T10:18:57.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.160+0000 7fe0cb93d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c419a1e0 con 0x7fe0c4107730 2023-12-07T10:18:57.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.160+0000 7fe0cb93d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c419a3a0 con 0x7fe0c4102080 2023-12-07T10:18:57.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.160+0000 7fe0c96d9700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c41016c0 0x7fe0c419f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:57.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.160+0000 7fe0c96d9700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c41016c0 0x7fe0c419f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58780/0 (socket says 172.21.15.119:58780) 2023-12-07T10:18:57.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.160+0000 7fe0c96d9700 1 -- 172.21.15.119:0/1735686578 learned_addr learned my addr 172.21.15.119:0/1735686578 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:57.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.160+0000 7fe0c8ed8700 1 --2- 172.21.15.119:0/1735686578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0c4102080 0x7fe0c419fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:57.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.160+0000 7fe0c9eda700 1 --2- 172.21.15.119:0/1735686578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0c4107730 0x7fe0c4199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:57.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.161+0000 7fe0c96d9700 1 -- 172.21.15.119:0/1735686578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0c4102080 msgr2=0x7fe0c419fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:57.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.161+0000 7fe0c96d9700 1 --2- 172.21.15.119:0/1735686578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0c4102080 0x7fe0c419fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:57.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.161+0000 7fe0c96d9700 1 -- 172.21.15.119:0/1735686578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0c4107730 msgr2=0x7fe0c4199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:57.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.161+0000 7fe0c96d9700 1 --2- 172.21.15.119:0/1735686578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0c4107730 0x7fe0c4199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:57.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.161+0000 7fe0c96d9700 1 -- 172.21.15.119:0/1735686578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b40095f0 con 0x7fe0c41016c0 2023-12-07T10:18:57.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.161+0000 7fe0c9eda700 1 --2- 172.21.15.119:0/1735686578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0c4107730 0x7fe0c4199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:18:57.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.161+0000 7fe0c96d9700 1 --2- 172.21.15.119:0/1735686578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c41016c0 0x7fe0c419f830 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe0c000e9a0 tx=0x7fe0c000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:57.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.161+0000 7fe0ba7fc700 1 -- 172.21.15.119:0/1735686578 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0009790 con 0x7fe0c41016c0 2023-12-07T10:18:57.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.161+0000 7fe0ba7fc700 1 -- 172.21.15.119:0/1735686578 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0c000cbd0 con 0x7fe0c41016c0 2023-12-07T10:18:57.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.161+0000 7fe0ba7fc700 1 -- 172.21.15.119:0/1735686578 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0018680 con 0x7fe0c41016c0 2023-12-07T10:18:57.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.161+0000 7fe0cb93d700 1 -- 172.21.15.119:0/1735686578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c419a680 con 0x7fe0c41016c0 2023-12-07T10:18:57.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.161+0000 7fe0cb93d700 1 -- 172.21.15.119:0/1735686578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c41a68d0 con 0x7fe0c41016c0 2023-12-07T10:18:57.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.162+0000 7fe0cb93d700 1 -- 172.21.15.119:0/1735686578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c4104980 con 0x7fe0c41016c0 2023-12-07T10:18:57.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.164+0000 7fe0ba7fc700 1 -- 172.21.15.119:0/1735686578 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7fe0c000cd40 con 0x7fe0c41016c0 2023-12-07T10:18:57.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.166+0000 7fe0ba7fc700 1 --2- 172.21.15.119:0/1735686578 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe0b00a1c20 0x7fe0b00a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:57.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.166+0000 7fe0ba7fc700 1 -- 172.21.15.119:0/1735686578 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fe0c0014070 con 0x7fe0c41016c0 2023-12-07T10:18:57.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.166+0000 7fe0c8ed8700 1 --2- 172.21.15.119:0/1735686578 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe0b00a1c20 0x7fe0b00a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:57.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.167+0000 7fe0c8ed8700 1 --2- 172.21.15.119:0/1735686578 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe0b00a1c20 0x7fe0b00a40e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4079180 tx=0x7fe0b4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:57.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.170+0000 7fe0ba7fc700 1 -- 172.21.15.119:0/1735686578 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe0c00522f0 con 0x7fe0c41016c0 2023-12-07T10:18:57.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.225+0000 7fe44ec08700 1 -- 172.21.15.119:0/769871880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4481016c0 msgr2=0x7fe448101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:57.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.225+0000 7fe44ec08700 1 --2- 172.21.15.119:0/769871880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4481016c0 0x7fe448101b40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe434009990 tx=0x7fe434009ca0 comp rx=0 tx=0).stop 2023-12-07T10:18:57.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.225+0000 7fe44ec08700 1 -- 172.21.15.119:0/769871880 shutdown_connections 2023-12-07T10:18:57.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.225+0000 7fe44ec08700 1 --2- 172.21.15.119:0/769871880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe448102080 0x7fe44810e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:57.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.225+0000 7fe44ec08700 1 --2- 172.21.15.119:0/769871880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe448107730 0x7fe448107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:57.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.225+0000 7fe44ec08700 1 --2- 172.21.15.119:0/769871880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4481016c0 0x7fe448101b40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:57.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.225+0000 7fe44ec08700 1 -- 172.21.15.119:0/769871880 >> 172.21.15.119:0/769871880 conn(0x7fe448078800 msgr2=0x7fe448078c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:57.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.225+0000 7fe44ec08700 1 -- 172.21.15.119:0/769871880 shutdown_connections 2023-12-07T10:18:57.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.225+0000 7fe44ec08700 1 -- 172.21.15.119:0/769871880 wait complete. 2023-12-07T10:18:57.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.226+0000 7fe44ec08700 1 Processor -- start 2023-12-07T10:18:57.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.226+0000 7fe44ec08700 1 -- start start 2023-12-07T10:18:57.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.227+0000 7fe44ec08700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4481016c0 0x7fe44819f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:57.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.227+0000 7fe44ec08700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe448102080 0x7fe44819fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:57.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.227+0000 7fe44ec08700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe448107730 0x7fe448199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:57.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.227+0000 7fe44ec08700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44819a0a0 con 0x7fe448102080 2023-12-07T10:18:57.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.227+0000 7fe44ec08700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44819a210 con 0x7fe448107730 2023-12-07T10:18:57.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.227+0000 7fe44ec08700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44819a3d0 con 0x7fe4481016c0 2023-12-07T10:18:57.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.227+0000 7fe447fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe448102080 0x7fe44819fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:57.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.227+0000 7fe447fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe448102080 0x7fe44819fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58798/0 (socket says 172.21.15.119:58798) 2023-12-07T10:18:57.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.227+0000 7fe447fff700 1 -- 172.21.15.119:0/2391065784 learned_addr learned my addr 172.21.15.119:0/2391065784 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:18:57.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.227+0000 7fe44d1a5700 1 --2- 172.21.15.119:0/2391065784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe448107730 0x7fe448199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:57.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.227+0000 7fe44c9a4700 1 --2- 172.21.15.119:0/2391065784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4481016c0 0x7fe44819f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:57.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.227+0000 7fe447fff700 1 -- 172.21.15.119:0/2391065784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4481016c0 msgr2=0x7fe44819f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:57.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.227+0000 7fe447fff700 1 --2- 172.21.15.119:0/2391065784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4481016c0 0x7fe44819f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:57.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.227+0000 7fe447fff700 1 -- 172.21.15.119:0/2391065784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe448107730 msgr2=0x7fe448199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:57.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.227+0000 7fe447fff700 1 --2- 172.21.15.119:0/2391065784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe448107730 0x7fe448199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:57.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.227+0000 7fe447fff700 1 -- 172.21.15.119:0/2391065784 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4340095f0 con 0x7fe448102080 2023-12-07T10:18:57.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.227+0000 7fe44c9a4700 1 --2- 172.21.15.119:0/2391065784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4481016c0 0x7fe44819f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:18:57.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.228+0000 7fe447fff700 1 --2- 172.21.15.119:0/2391065784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe448102080 0x7fe44819fe40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe434005910 tx=0x7fe434004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:57.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.228+0000 7fe445ffb700 1 -- 172.21.15.119:0/2391065784 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43401d070 con 0x7fe448102080 2023-12-07T10:18:57.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.228+0000 7fe445ffb700 1 -- 172.21.15.119:0/2391065784 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe434022470 con 0x7fe448102080 2023-12-07T10:18:57.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.228+0000 7fe445ffb700 1 -- 172.21.15.119:0/2391065784 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe434017650 con 0x7fe448102080 2023-12-07T10:18:57.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.228+0000 7fe44ec08700 1 -- 172.21.15.119:0/2391065784 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe44819a650 con 0x7fe448102080 2023-12-07T10:18:57.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.228+0000 7fe44ec08700 1 -- 172.21.15.119:0/2391065784 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4481a68f0 con 0x7fe448102080 2023-12-07T10:18:57.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.229+0000 7fe4337fe700 1 -- 172.21.15.119:0/2391065784 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4280052f0 con 0x7fe448102080 2023-12-07T10:18:57.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.231+0000 7fe445ffb700 1 -- 172.21.15.119:0/2391065784 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7fe4340225e0 con 0x7fe448102080 2023-12-07T10:18:57.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.232+0000 7fe445ffb700 1 --2- 172.21.15.119:0/2391065784 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe4380a1c20 0x7fe4380a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:18:57.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.233+0000 7fe445ffb700 1 -- 172.21.15.119:0/2391065784 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fe4340ced00 con 0x7fe448102080 2023-12-07T10:18:57.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.233+0000 7fe44c9a4700 1 --2- 172.21.15.119:0/2391065784 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe4380a1c20 0x7fe4380a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:18:57.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.233+0000 7fe44c9a4700 1 --2- 172.21.15.119:0/2391065784 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe4380a1c20 0x7fe4380a40e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009cb0 tx=0x7fe43c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:18:57.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.236+0000 7fe445ffb700 1 -- 172.21.15.119:0/2391065784 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe434097cf0 con 0x7fe448102080 2023-12-07T10:18:57.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.391+0000 7fe0cb93d700 1 -- 172.21.15.119:0/1735686578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fe0c4102d50 con 0x7fe0c41016c0 2023-12-07T10:18:57.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.392+0000 7fe0ba7fc700 1 -- 172.21.15.119:0/1735686578 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe0c0096bf0 con 0x7fe0c41016c0 2023-12-07T10:18:57.393 INFO:teuthology.orchestra.run.smithi119.stdout:223338299438 2023-12-07T10:18:57.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.394+0000 7fe0cb93d700 1 -- 172.21.15.119:0/1735686578 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe0b00a1c20 msgr2=0x7fe0b00a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:57.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.394+0000 7fe0cb93d700 1 --2- 172.21.15.119:0/1735686578 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe0b00a1c20 0x7fe0b00a40e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4079180 tx=0x7fe0b4005fd0 comp rx=0 tx=0).stop 2023-12-07T10:18:57.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.394+0000 7fe0cb93d700 1 -- 172.21.15.119:0/1735686578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c41016c0 msgr2=0x7fe0c419f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:57.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.394+0000 7fe0cb93d700 1 --2- 172.21.15.119:0/1735686578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c41016c0 0x7fe0c419f830 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe0c000e9a0 tx=0x7fe0c000ecb0 comp rx=0 tx=0).stop 2023-12-07T10:18:57.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.394+0000 7fe0cb93d700 1 -- 172.21.15.119:0/1735686578 shutdown_connections 2023-12-07T10:18:57.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.394+0000 7fe0cb93d700 1 --2- 172.21.15.119:0/1735686578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0c4107730 0x7fe0c4199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:57.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.394+0000 7fe0cb93d700 1 --2- 172.21.15.119:0/1735686578 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe0b00a1c20 0x7fe0b00a40e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:57.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.394+0000 7fe0cb93d700 1 --2- 172.21.15.119:0/1735686578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c41016c0 0x7fe0c419f830 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:57.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.394+0000 7fe0cb93d700 1 --2- 172.21.15.119:0/1735686578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0c4102080 0x7fe0c419fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:57.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.394+0000 7fe0cb93d700 1 -- 172.21.15.119:0/1735686578 >> 172.21.15.119:0/1735686578 conn(0x7fe0c4078800 msgr2=0x7fe0c410cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:57.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.394+0000 7fe0cb93d700 1 -- 172.21.15.119:0/1735686578 shutdown_connections 2023-12-07T10:18:57.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.394+0000 7fe0cb93d700 1 -- 172.21.15.119:0/1735686578 wait complete. 2023-12-07T10:18:57.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.477+0000 7fe4337fe700 1 -- 172.21.15.119:0/2391065784 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe428005160 con 0x7fe448102080 2023-12-07T10:18:57.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.477+0000 7fe445ffb700 1 -- 172.21.15.119:0/2391065784 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe43409b310 con 0x7fe448102080 2023-12-07T10:18:57.480 INFO:teuthology.orchestra.run.smithi119.stdout:167503724599 2023-12-07T10:18:57.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.480+0000 7fe4337fe700 1 -- 172.21.15.119:0/2391065784 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe4380a1c20 msgr2=0x7fe4380a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:57.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.480+0000 7fe4337fe700 1 --2- 172.21.15.119:0/2391065784 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe4380a1c20 0x7fe4380a40e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009cb0 tx=0x7fe43c009260 comp rx=0 tx=0).stop 2023-12-07T10:18:57.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.480+0000 7fe4337fe700 1 -- 172.21.15.119:0/2391065784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe448102080 msgr2=0x7fe44819fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:18:57.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.480+0000 7fe4337fe700 1 --2- 172.21.15.119:0/2391065784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe448102080 0x7fe44819fe40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe434005910 tx=0x7fe434004730 comp rx=0 tx=0).stop 2023-12-07T10:18:57.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.480+0000 7fe4337fe700 1 -- 172.21.15.119:0/2391065784 shutdown_connections 2023-12-07T10:18:57.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.480+0000 7fe4337fe700 1 --2- 172.21.15.119:0/2391065784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe448107730 0x7fe448199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:57.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.480+0000 7fe4337fe700 1 --2- 172.21.15.119:0/2391065784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4481016c0 0x7fe44819f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:57.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.480+0000 7fe4337fe700 1 --2- 172.21.15.119:0/2391065784 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe4380a1c20 0x7fe4380a40e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:57.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.480+0000 7fe4337fe700 1 --2- 172.21.15.119:0/2391065784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe448102080 0x7fe44819fe40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:18:57.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.480+0000 7fe4337fe700 1 -- 172.21.15.119:0/2391065784 >> 172.21.15.119:0/2391065784 conn(0x7fe448078800 msgr2=0x7fe44810cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:18:57.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.480+0000 7fe4337fe700 1 -- 172.21.15.119:0/2391065784 shutdown_connections 2023-12-07T10:18:57.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:18:57.480+0000 7fe4337fe700 1 -- 172.21.15.119:0/2391065784 wait complete. 2023-12-07T10:18:58.019 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299435 got 223338299438 for osd.7 2023-12-07T10:18:58.019 DEBUG:teuthology.parallel:result is None 2023-12-07T10:18:58.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:57 smithi119 ceph-mon[97908]: pgmap v319: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-12-07T10:18:58.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:57 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1735686578' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-07T10:18:58.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:57 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2391065784' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-07T10:18:58.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:57 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:18:58.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:57 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:18:58.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:57 smithi159 ceph-mon[134726]: pgmap v319: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-12-07T10:18:58.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:57 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1735686578' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-07T10:18:58.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:57 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2391065784' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-07T10:18:58.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:57 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:18:58.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:57 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:18:58.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:57 smithi153 ceph-mon[133625]: pgmap v319: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-12-07T10:18:58.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:57 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1735686578' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-07T10:18:58.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:57 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2391065784' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-07T10:18:58.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:57 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:18:58.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:57 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:18:58.310 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724596 got 167503724599 for osd.5 2023-12-07T10:18:58.311 DEBUG:teuthology.parallel:result is None 2023-12-07T10:18:58.311 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-12-07T10:18:58.311 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph pg dump --format=json 2023-12-07T10:18:59.104 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:18:59.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:58 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:18:59.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:58 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:18:59.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:58 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:18:59.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:58 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:18:59.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:58 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:18:59.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:58 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:18:59.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:58 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:18:59.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:58 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:18:59.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:58 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:18:59.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:18:59 smithi119 ceph-mon[97908]: pgmap v320: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-07T10:19:00.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:18:59 smithi159 ceph-mon[134726]: pgmap v320: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-07T10:19:00.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:18:59 smithi153 ceph-mon[133625]: pgmap v320: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-07T10:19:00.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.648+0000 7f4f9399d700 1 -- 172.21.15.119:0/1349954523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f8c107730 msgr2=0x7f4f8c107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:00.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.648+0000 7f4f9399d700 1 --2- 172.21.15.119:0/1349954523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f8c107730 0x7f4f8c107b10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009990 tx=0x7f4f88009ca0 comp rx=0 tx=0).stop 2023-12-07T10:19:00.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.649+0000 7f4f9399d700 1 -- 172.21.15.119:0/1349954523 shutdown_connections 2023-12-07T10:19:00.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.649+0000 7f4f9399d700 1 --2- 172.21.15.119:0/1349954523 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f8c102080 0x7f4f8c10e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:00.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.649+0000 7f4f9399d700 1 --2- 172.21.15.119:0/1349954523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f8c107730 0x7f4f8c107b10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:00.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.649+0000 7f4f9399d700 1 --2- 172.21.15.119:0/1349954523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f8c1016c0 0x7f4f8c101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:00.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.649+0000 7f4f9399d700 1 -- 172.21.15.119:0/1349954523 >> 172.21.15.119:0/1349954523 conn(0x7f4f8c078800 msgr2=0x7f4f8c078c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:00.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.649+0000 7f4f9399d700 1 -- 172.21.15.119:0/1349954523 shutdown_connections 2023-12-07T10:19:00.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.649+0000 7f4f9399d700 1 -- 172.21.15.119:0/1349954523 wait complete. 2023-12-07T10:19:00.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.650+0000 7f4f9399d700 1 Processor -- start 2023-12-07T10:19:00.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.650+0000 7f4f9399d700 1 -- start start 2023-12-07T10:19:00.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.650+0000 7f4f9399d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f8c1016c0 0x7f4f8c19f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:00.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.650+0000 7f4f9399d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f8c102080 0x7f4f8c19fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:00.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.650+0000 7f4f9399d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f8c107730 0x7f4f8c199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:00.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.650+0000 7f4f9399d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c19a080 con 0x7f4f8c102080 2023-12-07T10:19:00.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.650+0000 7f4f9399d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c19a1f0 con 0x7f4f8c1016c0 2023-12-07T10:19:00.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.650+0000 7f4f9399d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c19a3b0 con 0x7f4f8c107730 2023-12-07T10:19:00.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.651+0000 7f4f91739700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f8c1016c0 0x7f4f8c19f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:00.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.651+0000 7f4f90f38700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f8c102080 0x7f4f8c19fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:00.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.651+0000 7f4f91739700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f8c1016c0 0x7f4f8c19f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:58218/0 (socket says 172.21.15.119:58218) 2023-12-07T10:19:00.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.651+0000 7f4f91739700 1 -- 172.21.15.119:0/1694035507 learned_addr learned my addr 172.21.15.119:0/1694035507 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:00.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.651+0000 7f4f91f3a700 1 --2- 172.21.15.119:0/1694035507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f8c107730 0x7f4f8c199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:00.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.651+0000 7f4f91739700 1 -- 172.21.15.119:0/1694035507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f8c107730 msgr2=0x7f4f8c199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:00.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.651+0000 7f4f91739700 1 --2- 172.21.15.119:0/1694035507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f8c107730 0x7f4f8c199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:00.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.651+0000 7f4f91739700 1 -- 172.21.15.119:0/1694035507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f8c102080 msgr2=0x7f4f8c19fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:00.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.651+0000 7f4f91739700 1 --2- 172.21.15.119:0/1694035507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f8c102080 0x7f4f8c19fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:00.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.651+0000 7f4f91739700 1 -- 172.21.15.119:0/1694035507 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f880095f0 con 0x7f4f8c1016c0 2023-12-07T10:19:00.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.651+0000 7f4f91f3a700 1 --2- 172.21.15.119:0/1694035507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f8c107730 0x7f4f8c199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:19:00.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.651+0000 7f4f90f38700 1 --2- 172.21.15.119:0/1694035507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f8c102080 0x7f4f8c19fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:00.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.651+0000 7f4f91739700 1 --2- 172.21.15.119:0/1694035507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f8c1016c0 0x7f4f8c19f8e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4f88000c00 tx=0x7f4f88005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:00.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.652+0000 7f4f827fc700 1 -- 172.21.15.119:0/1694035507 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8801d070 con 0x7f4f8c1016c0 2023-12-07T10:19:00.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.652+0000 7f4f827fc700 1 -- 172.21.15.119:0/1694035507 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f88022470 con 0x7f4f8c1016c0 2023-12-07T10:19:00.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.652+0000 7f4f9399d700 1 -- 172.21.15.119:0/1694035507 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f8c19a630 con 0x7f4f8c1016c0 2023-12-07T10:19:00.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.652+0000 7f4f827fc700 1 -- 172.21.15.119:0/1694035507 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f88017650 con 0x7f4f8c1016c0 2023-12-07T10:19:00.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.652+0000 7f4f9399d700 1 -- 172.21.15.119:0/1694035507 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f8c1a68d0 con 0x7f4f8c1016c0 2023-12-07T10:19:00.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.653+0000 7f4f9399d700 1 -- 172.21.15.119:0/1694035507 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f8c104980 con 0x7f4f8c1016c0 2023-12-07T10:19:00.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.655+0000 7f4f827fc700 1 -- 172.21.15.119:0/1694035507 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f4f880225e0 con 0x7f4f8c1016c0 2023-12-07T10:19:00.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.656+0000 7f4f827fc700 1 --2- 172.21.15.119:0/1694035507 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4f780a1c20 0x7f4f780a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:00.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.656+0000 7f4f827fc700 1 -- 172.21.15.119:0/1694035507 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f4f880cf340 con 0x7f4f8c1016c0 2023-12-07T10:19:00.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.656+0000 7f4f90f38700 1 --2- 172.21.15.119:0/1694035507 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4f780a1c20 0x7f4f780a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:00.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.657+0000 7f4f90f38700 1 --2- 172.21.15.119:0/1694035507 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4f780a1c20 0x7f4f780a40e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c19abe0 tx=0x7f4f7c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:00.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.659+0000 7f4f827fc700 1 -- 172.21.15.119:0/1694035507 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4f88098390 con 0x7f4f8c1016c0 2023-12-07T10:19:00.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.874+0000 7f4f9399d700 1 -- 172.21.15.119:0/1694035507 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4f8c19aee0 con 0x7f4f780a1c20 2023-12-07T10:19:00.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.887+0000 7f4f827fc700 1 -- 172.21.15.119:0/1694035507 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426731 (secure 0 0 0) 0x7f4f8c19aee0 con 0x7f4f780a1c20 2023-12-07T10:19:00.894 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:19:00.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.892+0000 7f4f9399d700 1 -- 172.21.15.119:0/1694035507 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4f780a1c20 msgr2=0x7f4f780a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:00.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.892+0000 7f4f9399d700 1 --2- 172.21.15.119:0/1694035507 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4f780a1c20 0x7f4f780a40e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c19abe0 tx=0x7f4f7c009260 comp rx=0 tx=0).stop 2023-12-07T10:19:00.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.892+0000 7f4f9399d700 1 -- 172.21.15.119:0/1694035507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f8c1016c0 msgr2=0x7f4f8c19f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:00.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.892+0000 7f4f9399d700 1 --2- 172.21.15.119:0/1694035507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f8c1016c0 0x7f4f8c19f8e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4f88000c00 tx=0x7f4f88005740 comp rx=0 tx=0).stop 2023-12-07T10:19:00.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.892+0000 7f4f9399d700 1 -- 172.21.15.119:0/1694035507 shutdown_connections 2023-12-07T10:19:00.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.892+0000 7f4f9399d700 1 --2- 172.21.15.119:0/1694035507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f8c107730 0x7f4f8c199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:00.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.892+0000 7f4f9399d700 1 --2- 172.21.15.119:0/1694035507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f8c1016c0 0x7f4f8c19f8e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:00.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.892+0000 7f4f9399d700 1 --2- 172.21.15.119:0/1694035507 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4f780a1c20 0x7f4f780a40e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:00.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.892+0000 7f4f9399d700 1 --2- 172.21.15.119:0/1694035507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f8c102080 0x7f4f8c19fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:00.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.892+0000 7f4f9399d700 1 -- 172.21.15.119:0/1694035507 >> 172.21.15.119:0/1694035507 conn(0x7f4f8c078800 msgr2=0x7f4f8c10cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:00.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.892+0000 7f4f9399d700 1 -- 172.21.15.119:0/1694035507 shutdown_connections 2023-12-07T10:19:00.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:00.892+0000 7f4f9399d700 1 -- 172.21.15.119:0/1694035507 wait complete. 2023-12-07T10:19:00.899 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-12-07T10:19:01.401 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2023-12-07T10:19:00.689531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":672259,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":649,"num_read_kb":465,"num_write":426,"num_write_kb":3092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338348,"kb_used_data":14156,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603476,"statfs":{"total":1151940427776,"available":1151593959424,"internally_reserved":0,"allocated":14495744,"data_stored":7960646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":331855161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003141"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849988+0000","last_change":"2023-12-07T10:17:41.614206+0000","last_active":"2023-12-07T10:18:00.849988+0000","last_peered":"2023-12-07T10:18:00.849988+0000","last_clean":"2023-12-07T10:18:00.849988+0000","last_became_active":"2023-12-07T10:17:41.613553+0000","last_became_peered":"2023-12-07T10:17:41.613553+0000","last_unstale":"2023-12-07T10:18:00.849988+0000","last_undegraded":"2023-12-07T10:18:00.849988+0000","last_fullsized":"2023-12-07T10:18:00.849988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:10:22.198414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852964+0000","last_change":"2023-12-07T10:17:41.616917+0000","last_active":"2023-12-07T10:18:00.852964+0000","last_peered":"2023-12-07T10:18:00.852964+0000","last_clean":"2023-12-07T10:18:00.852964+0000","last_became_active":"2023-12-07T10:17:41.615345+0000","last_became_peered":"2023-12-07T10:17:41.615345+0000","last_unstale":"2023-12-07T10:18:00.852964+0000","last_undegraded":"2023-12-07T10:18:00.852964+0000","last_fullsized":"2023-12-07T10:18:00.852964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:22:25.566738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.853214+0000","last_change":"2023-12-07T10:17:41.619767+0000","last_active":"2023-12-07T10:18:00.853214+0000","last_peered":"2023-12-07T10:18:00.853214+0000","last_clean":"2023-12-07T10:18:00.853214+0000","last_became_active":"2023-12-07T10:17:41.619362+0000","last_became_peered":"2023-12-07T10:17:41.619362+0000","last_unstale":"2023-12-07T10:18:00.853214+0000","last_undegraded":"2023-12-07T10:18:00.853214+0000","last_fullsized":"2023-12-07T10:18:00.853214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:12:55.458520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.862478+0000","last_change":"2023-12-07T10:18:01.861930+0000","last_active":"2023-12-07T10:18:01.862478+0000","last_peered":"2023-12-07T10:18:01.862478+0000","last_clean":"2023-12-07T10:18:01.862478+0000","last_became_active":"2023-12-07T10:18:01.861570+0000","last_became_peered":"2023-12-07T10:18:01.861570+0000","last_unstale":"2023-12-07T10:18:01.862478+0000","last_undegraded":"2023-12-07T10:18:01.862478+0000","last_fullsized":"2023-12-07T10:18:01.862478+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:29:45.234261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852580+0000","last_change":"2023-12-07T10:17:41.613890+0000","last_active":"2023-12-07T10:18:00.852580+0000","last_peered":"2023-12-07T10:18:00.852580+0000","last_clean":"2023-12-07T10:18:00.852580+0000","last_became_active":"2023-12-07T10:17:41.613073+0000","last_became_peered":"2023-12-07T10:17:41.613073+0000","last_unstale":"2023-12-07T10:18:00.852580+0000","last_undegraded":"2023-12-07T10:18:00.852580+0000","last_fullsized":"2023-12-07T10:18:00.852580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:17:14.339002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.854664+0000","last_change":"2023-12-07T10:17:41.609585+0000","last_active":"2023-12-07T10:18:00.854664+0000","last_peered":"2023-12-07T10:18:00.854664+0000","last_clean":"2023-12-07T10:18:00.854664+0000","last_became_active":"2023-12-07T10:17:41.609256+0000","last_became_peered":"2023-12-07T10:17:41.609256+0000","last_unstale":"2023-12-07T10:18:00.854664+0000","last_undegraded":"2023-12-07T10:18:00.854664+0000","last_fullsized":"2023-12-07T10:18:00.854664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:20:15.103208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856396+0000","last_change":"2023-12-07T10:17:41.606138+0000","last_active":"2023-12-07T10:18:00.856396+0000","last_peered":"2023-12-07T10:18:00.856396+0000","last_clean":"2023-12-07T10:18:00.856396+0000","last_became_active":"2023-12-07T10:17:41.605818+0000","last_became_peered":"2023-12-07T10:17:41.605818+0000","last_unstale":"2023-12-07T10:18:00.856396+0000","last_undegraded":"2023-12-07T10:18:00.856396+0000","last_fullsized":"2023-12-07T10:18:00.856396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:38:39.244077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.257383+0000","last_change":"2023-12-07T10:18:02.255959+0000","last_active":"2023-12-07T10:18:02.257383+0000","last_peered":"2023-12-07T10:18:02.257383+0000","last_clean":"2023-12-07T10:18:02.257383+0000","last_became_active":"2023-12-07T10:18:02.255560+0000","last_became_peered":"2023-12-07T10:18:02.255560+0000","last_unstale":"2023-12-07T10:18:02.257383+0000","last_undegraded":"2023-12-07T10:18:02.257383+0000","last_fullsized":"2023-12-07T10:18:02.257383+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:05:49.065538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.847884+0000","last_change":"2023-12-07T10:17:41.610225+0000","last_active":"2023-12-07T10:18:00.847884+0000","last_peered":"2023-12-07T10:18:00.847884+0000","last_clean":"2023-12-07T10:18:00.847884+0000","last_became_active":"2023-12-07T10:17:41.609875+0000","last_became_peered":"2023-12-07T10:17:41.609875+0000","last_unstale":"2023-12-07T10:18:00.847884+0000","last_undegraded":"2023-12-07T10:18:00.847884+0000","last_fullsized":"2023-12-07T10:18:00.847884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:00:43.136176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855342+0000","last_change":"2023-12-07T10:17:41.617335+0000","last_active":"2023-12-07T10:18:00.855342+0000","last_peered":"2023-12-07T10:18:00.855342+0000","last_clean":"2023-12-07T10:18:00.855342+0000","last_became_active":"2023-12-07T10:17:41.617005+0000","last_became_peered":"2023-12-07T10:17:41.617005+0000","last_unstale":"2023-12-07T10:18:00.855342+0000","last_undegraded":"2023-12-07T10:18:00.855342+0000","last_fullsized":"2023-12-07T10:18:00.855342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:43:36.855465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.857002+0000","last_change":"2023-12-07T10:17:41.608104+0000","last_active":"2023-12-07T10:18:00.857002+0000","last_peered":"2023-12-07T10:18:00.857002+0000","last_clean":"2023-12-07T10:18:00.857002+0000","last_became_active":"2023-12-07T10:17:41.607769+0000","last_became_peered":"2023-12-07T10:17:41.607769+0000","last_unstale":"2023-12-07T10:18:00.857002+0000","last_undegraded":"2023-12-07T10:18:00.857002+0000","last_fullsized":"2023-12-07T10:18:00.857002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:47:04.853044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852617+0000","last_change":"2023-12-07T10:17:41.611585+0000","last_active":"2023-12-07T10:18:00.852617+0000","last_peered":"2023-12-07T10:18:00.852617+0000","last_clean":"2023-12-07T10:18:00.852617+0000","last_became_active":"2023-12-07T10:17:41.611202+0000","last_became_peered":"2023-12-07T10:17:41.611202+0000","last_unstale":"2023-12-07T10:18:00.852617+0000","last_undegraded":"2023-12-07T10:18:00.852617+0000","last_fullsized":"2023-12-07T10:18:00.852617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:17:01.649424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855084+0000","last_change":"2023-12-07T10:17:41.611116+0000","last_active":"2023-12-07T10:18:00.855084+0000","last_peered":"2023-12-07T10:18:00.855084+0000","last_clean":"2023-12-07T10:18:00.855084+0000","last_became_active":"2023-12-07T10:17:41.610791+0000","last_became_peered":"2023-12-07T10:17:41.610791+0000","last_unstale":"2023-12-07T10:18:00.855084+0000","last_undegraded":"2023-12-07T10:18:00.855084+0000","last_fullsized":"2023-12-07T10:18:00.855084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:48:08.822235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852233+0000","last_change":"2023-12-07T10:17:41.617447+0000","last_active":"2023-12-07T10:18:00.852233+0000","last_peered":"2023-12-07T10:18:00.852233+0000","last_clean":"2023-12-07T10:18:00.852233+0000","last_became_active":"2023-12-07T10:17:41.612638+0000","last_became_peered":"2023-12-07T10:17:41.612638+0000","last_unstale":"2023-12-07T10:18:00.852233+0000","last_undegraded":"2023-12-07T10:18:00.852233+0000","last_fullsized":"2023-12-07T10:18:00.852233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:11:30.384367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852009+0000","last_change":"2023-12-07T10:17:41.611607+0000","last_active":"2023-12-07T10:18:00.852009+0000","last_peered":"2023-12-07T10:18:00.852009+0000","last_clean":"2023-12-07T10:18:00.852009+0000","last_became_active":"2023-12-07T10:17:41.611331+0000","last_became_peered":"2023-12-07T10:17:41.611331+0000","last_unstale":"2023-12-07T10:18:00.852009+0000","last_undegraded":"2023-12-07T10:18:00.852009+0000","last_fullsized":"2023-12-07T10:18:00.852009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:17:53.049922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.847334+0000","last_change":"2023-12-07T10:17:41.610528+0000","last_active":"2023-12-07T10:18:00.847334+0000","last_peered":"2023-12-07T10:18:00.847334+0000","last_clean":"2023-12-07T10:18:00.847334+0000","last_became_active":"2023-12-07T10:17:41.610216+0000","last_became_peered":"2023-12-07T10:17:41.610216+0000","last_unstale":"2023-12-07T10:18:00.847334+0000","last_undegraded":"2023-12-07T10:18:00.847334+0000","last_fullsized":"2023-12-07T10:18:00.847334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:11:36.749537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849875+0000","last_change":"2023-12-07T10:17:41.613967+0000","last_active":"2023-12-07T10:18:00.849875+0000","last_peered":"2023-12-07T10:18:00.849875+0000","last_clean":"2023-12-07T10:18:00.849875+0000","last_became_active":"2023-12-07T10:17:41.613116+0000","last_became_peered":"2023-12-07T10:17:41.613116+0000","last_unstale":"2023-12-07T10:18:00.849875+0000","last_undegraded":"2023-12-07T10:18:00.849875+0000","last_fullsized":"2023-12-07T10:18:00.849875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:46:32.444025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.846417+0000","last_change":"2023-12-07T10:17:41.605478+0000","last_active":"2023-12-07T10:18:00.846417+0000","last_peered":"2023-12-07T10:18:00.846417+0000","last_clean":"2023-12-07T10:18:00.846417+0000","last_became_active":"2023-12-07T10:17:41.605110+0000","last_became_peered":"2023-12-07T10:17:41.605110+0000","last_unstale":"2023-12-07T10:18:00.846417+0000","last_undegraded":"2023-12-07T10:18:00.846417+0000","last_fullsized":"2023-12-07T10:18:00.846417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:53:56.410196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.256377+0000","last_change":"2023-12-07T10:18:02.256377+0000","last_active":"2023-12-07T10:18:02.256377+0000","last_peered":"2023-12-07T10:18:02.256377+0000","last_clean":"2023-12-07T10:18:02.256377+0000","last_became_active":"2023-12-07T10:18:02.255987+0000","last_became_peered":"2023-12-07T10:18:02.255987+0000","last_unstale":"2023-12-07T10:18:02.256377+0000","last_undegraded":"2023-12-07T10:18:02.256377+0000","last_fullsized":"2023-12-07T10:18:02.256377+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:23:34.909187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849924+0000","last_change":"2023-12-07T10:17:45.799925+0000","last_active":"2023-12-07T10:18:00.849924+0000","last_peered":"2023-12-07T10:18:00.849924+0000","last_clean":"2023-12-07T10:18:00.849924+0000","last_became_active":"2023-12-07T10:17:45.799505+0000","last_became_peered":"2023-12-07T10:17:45.799505+0000","last_unstale":"2023-12-07T10:18:00.849924+0000","last_undegraded":"2023-12-07T10:18:00.849924+0000","last_fullsized":"2023-12-07T10:18:00.849924+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:52:51.470333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'13","reported_seq":79,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:57.831212+0000","last_change":"2023-12-07T10:17:40.755066+0000","last_active":"2023-12-07T10:18:57.831212+0000","last_peered":"2023-12-07T10:18:57.831212+0000","last_clean":"2023-12-07T10:18:57.831212+0000","last_became_active":"2023-12-07T10:17:40.754709+0000","last_became_peered":"2023-12-07T10:17:40.754709+0000","last_unstale":"2023-12-07T10:18:57.831212+0000","last_undegraded":"2023-12-07T10:18:57.831212+0000","last_fullsized":"2023-12-07T10:18:57.831212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:18:31.148592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":47,"num_read_kb":47,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849407+0000","last_change":"2023-12-07T10:17:41.619107+0000","last_active":"2023-12-07T10:18:00.849407+0000","last_peered":"2023-12-07T10:18:00.849407+0000","last_clean":"2023-12-07T10:18:00.849407+0000","last_became_active":"2023-12-07T10:17:41.617057+0000","last_became_peered":"2023-12-07T10:17:41.617057+0000","last_unstale":"2023-12-07T10:18:00.849407+0000","last_undegraded":"2023-12-07T10:18:00.849407+0000","last_fullsized":"2023-12-07T10:18:00.849407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:53:16.402933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852207+0000","last_change":"2023-12-07T10:17:45.797582+0000","last_active":"2023-12-07T10:18:00.852207+0000","last_peered":"2023-12-07T10:18:00.852207+0000","last_clean":"2023-12-07T10:18:00.852207+0000","last_became_active":"2023-12-07T10:17:45.796358+0000","last_became_peered":"2023-12-07T10:17:45.796358+0000","last_unstale":"2023-12-07T10:18:00.852207+0000","last_undegraded":"2023-12-07T10:18:00.852207+0000","last_fullsized":"2023-12-07T10:18:00.852207+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:53:19.551459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.857161+0000","last_change":"2023-12-07T10:17:40.762155+0000","last_active":"2023-12-07T10:18:01.857161+0000","last_peered":"2023-12-07T10:18:01.857161+0000","last_clean":"2023-12-07T10:18:01.857161+0000","last_became_active":"2023-12-07T10:17:40.761776+0000","last_became_peered":"2023-12-07T10:17:40.761776+0000","last_unstale":"2023-12-07T10:18:01.857161+0000","last_undegraded":"2023-12-07T10:18:01.857161+0000","last_fullsized":"2023-12-07T10:18:01.857161+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:15:06.096331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.853107+0000","last_change":"2023-12-07T10:17:41.614999+0000","last_active":"2023-12-07T10:18:00.853107+0000","last_peered":"2023-12-07T10:18:00.853107+0000","last_clean":"2023-12-07T10:18:00.853107+0000","last_became_active":"2023-12-07T10:17:41.614673+0000","last_became_peered":"2023-12-07T10:17:41.614673+0000","last_unstale":"2023-12-07T10:18:00.853107+0000","last_undegraded":"2023-12-07T10:18:00.853107+0000","last_fullsized":"2023-12-07T10:18:00.853107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:34:45.378843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852388+0000","last_change":"2023-12-07T10:17:45.799399+0000","last_active":"2023-12-07T10:18:00.852388+0000","last_peered":"2023-12-07T10:18:00.852388+0000","last_clean":"2023-12-07T10:18:00.852388+0000","last_became_active":"2023-12-07T10:17:45.799072+0000","last_became_peered":"2023-12-07T10:17:45.799072+0000","last_unstale":"2023-12-07T10:18:00.852388+0000","last_undegraded":"2023-12-07T10:18:00.852388+0000","last_fullsized":"2023-12-07T10:18:00.852388+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:34:54.663334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:13.768961+0000","last_change":"2023-12-07T10:18:01.993128+0000","last_active":"2023-12-07T10:18:13.768961+0000","last_peered":"2023-12-07T10:18:13.768961+0000","last_clean":"2023-12-07T10:18:13.768961+0000","last_became_active":"2023-12-07T10:18:01.990244+0000","last_became_peered":"2023-12-07T10:18:01.990244+0000","last_unstale":"2023-12-07T10:18:13.768961+0000","last_undegraded":"2023-12-07T10:18:13.768961+0000","last_fullsized":"2023-12-07T10:18:13.768961+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:42:53.778363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.847633+0000","last_change":"2023-12-07T10:17:41.612903+0000","last_active":"2023-12-07T10:18:00.847633+0000","last_peered":"2023-12-07T10:18:00.847633+0000","last_clean":"2023-12-07T10:18:00.847633+0000","last_became_active":"2023-12-07T10:17:41.611968+0000","last_became_peered":"2023-12-07T10:17:41.611968+0000","last_unstale":"2023-12-07T10:18:00.847633+0000","last_undegraded":"2023-12-07T10:18:00.847633+0000","last_fullsized":"2023-12-07T10:18:00.847633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:34:08.442828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852245+0000","last_change":"2023-12-07T10:17:45.803858+0000","last_active":"2023-12-07T10:18:00.852245+0000","last_peered":"2023-12-07T10:18:00.852245+0000","last_clean":"2023-12-07T10:18:00.852245+0000","last_became_active":"2023-12-07T10:17:45.803481+0000","last_became_peered":"2023-12-07T10:17:45.803481+0000","last_unstale":"2023-12-07T10:18:00.852245+0000","last_undegraded":"2023-12-07T10:18:00.852245+0000","last_fullsized":"2023-12-07T10:18:00.852245+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:08:03.728554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.758206+0000","last_change":"2023-12-07T10:17:40.751866+0000","last_active":"2023-12-07T10:18:12.758206+0000","last_peered":"2023-12-07T10:18:12.758206+0000","last_clean":"2023-12-07T10:18:12.758206+0000","last_became_active":"2023-12-07T10:17:40.751492+0000","last_became_peered":"2023-12-07T10:17:40.751492+0000","last_unstale":"2023-12-07T10:18:12.758206+0000","last_undegraded":"2023-12-07T10:18:12.758206+0000","last_fullsized":"2023-12-07T10:18:12.758206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:42:53.839651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855907+0000","last_change":"2023-12-07T10:17:41.611326+0000","last_active":"2023-12-07T10:18:00.855907+0000","last_peered":"2023-12-07T10:18:00.855907+0000","last_clean":"2023-12-07T10:18:00.855907+0000","last_became_active":"2023-12-07T10:17:41.610863+0000","last_became_peered":"2023-12-07T10:17:41.610863+0000","last_unstale":"2023-12-07T10:18:00.855907+0000","last_undegraded":"2023-12-07T10:18:00.855907+0000","last_fullsized":"2023-12-07T10:18:00.855907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:07:55.801356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.850230+0000","last_change":"2023-12-07T10:17:45.791409+0000","last_active":"2023-12-07T10:18:00.850230+0000","last_peered":"2023-12-07T10:18:00.850230+0000","last_clean":"2023-12-07T10:18:00.850230+0000","last_became_active":"2023-12-07T10:17:45.791022+0000","last_became_peered":"2023-12-07T10:17:45.791022+0000","last_unstale":"2023-12-07T10:18:00.850230+0000","last_undegraded":"2023-12-07T10:18:00.850230+0000","last_fullsized":"2023-12-07T10:18:00.850230+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:57:14.617409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'9","reported_seq":81,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:56.805755+0000","last_change":"2023-12-07T10:17:40.745318+0000","last_active":"2023-12-07T10:18:56.805755+0000","last_peered":"2023-12-07T10:18:56.805755+0000","last_clean":"2023-12-07T10:18:56.805755+0000","last_became_active":"2023-12-07T10:17:40.745008+0000","last_became_peered":"2023-12-07T10:17:40.745008+0000","last_unstale":"2023-12-07T10:18:56.805755+0000","last_undegraded":"2023-12-07T10:18:56.805755+0000","last_fullsized":"2023-12-07T10:18:56.805755+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:42:28.503421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11004,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":14,"num_write":6,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.853726+0000","last_change":"2023-12-07T10:17:41.605121+0000","last_active":"2023-12-07T10:18:00.853726+0000","last_peered":"2023-12-07T10:18:00.853726+0000","last_clean":"2023-12-07T10:18:00.853726+0000","last_became_active":"2023-12-07T10:17:41.604770+0000","last_became_peered":"2023-12-07T10:17:41.604770+0000","last_unstale":"2023-12-07T10:18:00.853726+0000","last_undegraded":"2023-12-07T10:18:00.853726+0000","last_fullsized":"2023-12-07T10:18:00.853726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:51:02.151155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.850337+0000","last_change":"2023-12-07T10:17:45.809039+0000","last_active":"2023-12-07T10:18:00.850337+0000","last_peered":"2023-12-07T10:18:00.850337+0000","last_clean":"2023-12-07T10:18:00.850337+0000","last_became_active":"2023-12-07T10:17:45.808622+0000","last_became_peered":"2023-12-07T10:17:45.808622+0000","last_unstale":"2023-12-07T10:18:00.850337+0000","last_undegraded":"2023-12-07T10:18:00.850337+0000","last_fullsized":"2023-12-07T10:18:00.850337+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:16:02.835931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:11.747711+0000","last_change":"2023-12-07T10:17:40.757876+0000","last_active":"2023-12-07T10:18:11.747711+0000","last_peered":"2023-12-07T10:18:11.747711+0000","last_clean":"2023-12-07T10:18:11.747711+0000","last_became_active":"2023-12-07T10:17:40.757613+0000","last_became_peered":"2023-12-07T10:17:40.757613+0000","last_unstale":"2023-12-07T10:18:11.747711+0000","last_undegraded":"2023-12-07T10:18:11.747711+0000","last_fullsized":"2023-12-07T10:18:11.747711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:30:40.423092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856026+0000","last_change":"2023-12-07T10:17:41.611575+0000","last_active":"2023-12-07T10:18:00.856026+0000","last_peered":"2023-12-07T10:18:00.856026+0000","last_clean":"2023-12-07T10:18:00.856026+0000","last_became_active":"2023-12-07T10:17:41.611042+0000","last_became_peered":"2023-12-07T10:17:41.611042+0000","last_unstale":"2023-12-07T10:18:00.856026+0000","last_undegraded":"2023-12-07T10:18:00.856026+0000","last_fullsized":"2023-12-07T10:18:00.856026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:54:05.589792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849821+0000","last_change":"2023-12-07T10:17:45.796640+0000","last_active":"2023-12-07T10:18:00.849821+0000","last_peered":"2023-12-07T10:18:00.849821+0000","last_clean":"2023-12-07T10:18:00.849821+0000","last_became_active":"2023-12-07T10:17:45.796183+0000","last_became_peered":"2023-12-07T10:17:45.796183+0000","last_unstale":"2023-12-07T10:18:00.849821+0000","last_undegraded":"2023-12-07T10:18:00.849821+0000","last_fullsized":"2023-12-07T10:18:00.849821+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:41:56.890662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:10.744033+0000","last_change":"2023-12-07T10:17:40.760283+0000","last_active":"2023-12-07T10:18:10.744033+0000","last_peered":"2023-12-07T10:18:10.744033+0000","last_clean":"2023-12-07T10:18:10.744033+0000","last_became_active":"2023-12-07T10:17:40.759398+0000","last_became_peered":"2023-12-07T10:17:40.759398+0000","last_unstale":"2023-12-07T10:18:10.744033+0000","last_undegraded":"2023-12-07T10:18:10.744033+0000","last_fullsized":"2023-12-07T10:18:10.744033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:34:37.365490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849107+0000","last_change":"2023-12-07T10:17:41.604150+0000","last_active":"2023-12-07T10:18:00.849107+0000","last_peered":"2023-12-07T10:18:00.849107+0000","last_clean":"2023-12-07T10:18:00.849107+0000","last_became_active":"2023-12-07T10:17:41.603688+0000","last_became_peered":"2023-12-07T10:17:41.603688+0000","last_unstale":"2023-12-07T10:18:00.849107+0000","last_undegraded":"2023-12-07T10:18:00.849107+0000","last_fullsized":"2023-12-07T10:18:00.849107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:52:43.735609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852965+0000","last_change":"2023-12-07T10:17:45.806569+0000","last_active":"2023-12-07T10:18:00.852965+0000","last_peered":"2023-12-07T10:18:00.852965+0000","last_clean":"2023-12-07T10:18:00.852965+0000","last_became_active":"2023-12-07T10:17:45.806099+0000","last_became_peered":"2023-12-07T10:17:45.806099+0000","last_unstale":"2023-12-07T10:18:00.852965+0000","last_undegraded":"2023-12-07T10:18:00.852965+0000","last_fullsized":"2023-12-07T10:18:00.852965+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:02:19.977670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.757927+0000","last_change":"2023-12-07T10:17:40.744388+0000","last_active":"2023-12-07T10:18:12.757927+0000","last_peered":"2023-12-07T10:18:12.757927+0000","last_clean":"2023-12-07T10:18:12.757927+0000","last_became_active":"2023-12-07T10:17:40.743887+0000","last_became_peered":"2023-12-07T10:17:40.743887+0000","last_unstale":"2023-12-07T10:18:12.757927+0000","last_undegraded":"2023-12-07T10:18:12.757927+0000","last_fullsized":"2023-12-07T10:18:12.757927+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:18:39.102099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.845569+0000","last_change":"2023-12-07T10:17:41.595616+0000","last_active":"2023-12-07T10:18:00.845569+0000","last_peered":"2023-12-07T10:18:00.845569+0000","last_clean":"2023-12-07T10:18:00.845569+0000","last_became_active":"2023-12-07T10:17:41.595256+0000","last_became_peered":"2023-12-07T10:17:41.595256+0000","last_unstale":"2023-12-07T10:18:00.845569+0000","last_undegraded":"2023-12-07T10:18:00.845569+0000","last_fullsized":"2023-12-07T10:18:00.845569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:08:00.156043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849668+0000","last_change":"2023-12-07T10:17:45.794523+0000","last_active":"2023-12-07T10:18:00.849668+0000","last_peered":"2023-12-07T10:18:00.849668+0000","last_clean":"2023-12-07T10:18:00.849668+0000","last_became_active":"2023-12-07T10:17:45.794139+0000","last_became_peered":"2023-12-07T10:17:45.794139+0000","last_unstale":"2023-12-07T10:18:00.849668+0000","last_undegraded":"2023-12-07T10:18:00.849668+0000","last_fullsized":"2023-12-07T10:18:00.849668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:43:24.767627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":120,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:59.809174+0000","last_change":"2023-12-07T10:17:40.759620+0000","last_active":"2023-12-07T10:18:59.809174+0000","last_peered":"2023-12-07T10:18:59.809174+0000","last_clean":"2023-12-07T10:18:59.809174+0000","last_became_active":"2023-12-07T10:17:40.759372+0000","last_became_peered":"2023-12-07T10:17:40.759372+0000","last_unstale":"2023-12-07T10:18:59.809174+0000","last_undegraded":"2023-12-07T10:18:59.809174+0000","last_fullsized":"2023-12-07T10:18:59.809174+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:46:14.459409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.847937+0000","last_change":"2023-12-07T10:17:41.601152+0000","last_active":"2023-12-07T10:18:00.847937+0000","last_peered":"2023-12-07T10:18:00.847937+0000","last_clean":"2023-12-07T10:18:00.847937+0000","last_became_active":"2023-12-07T10:17:41.600147+0000","last_became_peered":"2023-12-07T10:17:41.600147+0000","last_unstale":"2023-12-07T10:18:00.847937+0000","last_undegraded":"2023-12-07T10:18:00.847937+0000","last_fullsized":"2023-12-07T10:18:00.847937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:40:46.671489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.845356+0000","last_change":"2023-12-07T10:17:45.796229+0000","last_active":"2023-12-07T10:18:00.845356+0000","last_peered":"2023-12-07T10:18:00.845356+0000","last_clean":"2023-12-07T10:18:00.845356+0000","last_became_active":"2023-12-07T10:17:45.795582+0000","last_became_peered":"2023-12-07T10:17:45.795582+0000","last_unstale":"2023-12-07T10:18:00.845356+0000","last_undegraded":"2023-12-07T10:18:00.845356+0000","last_fullsized":"2023-12-07T10:18:00.845356+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:29:32.853975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.758506+0000","last_change":"2023-12-07T10:17:40.748353+0000","last_active":"2023-12-07T10:18:12.758506+0000","last_peered":"2023-12-07T10:18:12.758506+0000","last_clean":"2023-12-07T10:18:12.758506+0000","last_became_active":"2023-12-07T10:17:40.748050+0000","last_became_peered":"2023-12-07T10:17:40.748050+0000","last_unstale":"2023-12-07T10:18:12.758506+0000","last_undegraded":"2023-12-07T10:18:12.758506+0000","last_fullsized":"2023-12-07T10:18:12.758506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:18:15.298534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.256728+0000","last_change":"2023-12-07T10:18:02.256728+0000","last_active":"2023-12-07T10:18:02.256728+0000","last_peered":"2023-12-07T10:18:02.256728+0000","last_clean":"2023-12-07T10:18:02.256728+0000","last_became_active":"2023-12-07T10:18:02.256271+0000","last_became_peered":"2023-12-07T10:18:02.256271+0000","last_unstale":"2023-12-07T10:18:02.256728+0000","last_undegraded":"2023-12-07T10:18:02.256728+0000","last_fullsized":"2023-12-07T10:18:02.256728+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:27:00.198934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852553+0000","last_change":"2023-12-07T10:17:45.796903+0000","last_active":"2023-12-07T10:18:00.852553+0000","last_peered":"2023-12-07T10:18:00.852553+0000","last_clean":"2023-12-07T10:18:00.852553+0000","last_became_active":"2023-12-07T10:17:45.796454+0000","last_became_peered":"2023-12-07T10:17:45.796454+0000","last_unstale":"2023-12-07T10:18:00.852553+0000","last_undegraded":"2023-12-07T10:18:00.852553+0000","last_fullsized":"2023-12-07T10:18:00.852553+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:46:36.955396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:10.734628+0000","last_change":"2023-12-07T10:17:40.756045+0000","last_active":"2023-12-07T10:18:10.734628+0000","last_peered":"2023-12-07T10:18:10.734628+0000","last_clean":"2023-12-07T10:18:10.734628+0000","last_became_active":"2023-12-07T10:17:40.755763+0000","last_became_peered":"2023-12-07T10:17:40.755763+0000","last_unstale":"2023-12-07T10:18:10.734628+0000","last_undegraded":"2023-12-07T10:18:10.734628+0000","last_fullsized":"2023-12-07T10:18:10.734628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:01:08.009076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.847716+0000","last_change":"2023-12-07T10:17:41.611178+0000","last_active":"2023-12-07T10:18:00.847716+0000","last_peered":"2023-12-07T10:18:00.847716+0000","last_clean":"2023-12-07T10:18:00.847716+0000","last_became_active":"2023-12-07T10:17:41.610750+0000","last_became_peered":"2023-12-07T10:17:41.610750+0000","last_unstale":"2023-12-07T10:18:00.847716+0000","last_undegraded":"2023-12-07T10:18:00.847716+0000","last_fullsized":"2023-12-07T10:18:00.847716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:56:30.763523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.845439+0000","last_change":"2023-12-07T10:17:45.800183+0000","last_active":"2023-12-07T10:18:00.845439+0000","last_peered":"2023-12-07T10:18:00.845439+0000","last_clean":"2023-12-07T10:18:00.845439+0000","last_became_active":"2023-12-07T10:17:45.798598+0000","last_became_peered":"2023-12-07T10:17:45.798598+0000","last_unstale":"2023-12-07T10:18:00.845439+0000","last_undegraded":"2023-12-07T10:18:00.845439+0000","last_fullsized":"2023-12-07T10:18:00.845439+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:06:22.162951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":160,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:59.812240+0000","last_change":"2023-12-07T10:17:40.752481+0000","last_active":"2023-12-07T10:18:59.812240+0000","last_peered":"2023-12-07T10:18:59.812240+0000","last_clean":"2023-12-07T10:18:59.812240+0000","last_became_active":"2023-12-07T10:17:40.752158+0000","last_became_peered":"2023-12-07T10:17:40.752158+0000","last_unstale":"2023-12-07T10:18:59.812240+0000","last_undegraded":"2023-12-07T10:18:59.812240+0000","last_fullsized":"2023-12-07T10:18:59.812240+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:43:27.963014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64453,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":107,"num_read_kb":116,"num_write":36,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.853035+0000","last_change":"2023-12-07T10:17:41.614107+0000","last_active":"2023-12-07T10:18:00.853035+0000","last_peered":"2023-12-07T10:18:00.853035+0000","last_clean":"2023-12-07T10:18:00.853035+0000","last_became_active":"2023-12-07T10:17:41.613743+0000","last_became_peered":"2023-12-07T10:17:41.613743+0000","last_unstale":"2023-12-07T10:18:00.853035+0000","last_undegraded":"2023-12-07T10:18:00.853035+0000","last_fullsized":"2023-12-07T10:18:00.853035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:49:03.200387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856289+0000","last_change":"2023-12-07T10:17:45.804282+0000","last_active":"2023-12-07T10:18:00.856289+0000","last_peered":"2023-12-07T10:18:00.856289+0000","last_clean":"2023-12-07T10:18:00.856289+0000","last_became_active":"2023-12-07T10:17:45.803794+0000","last_became_peered":"2023-12-07T10:17:45.803794+0000","last_unstale":"2023-12-07T10:18:00.856289+0000","last_undegraded":"2023-12-07T10:18:00.856289+0000","last_fullsized":"2023-12-07T10:18:00.856289+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:31:26.887565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.756900+0000","last_change":"2023-12-07T10:17:40.747204+0000","last_active":"2023-12-07T10:18:12.756900+0000","last_peered":"2023-12-07T10:18:12.756900+0000","last_clean":"2023-12-07T10:18:12.756900+0000","last_became_active":"2023-12-07T10:17:40.746916+0000","last_became_peered":"2023-12-07T10:17:40.746916+0000","last_unstale":"2023-12-07T10:18:12.756900+0000","last_undegraded":"2023-12-07T10:18:12.756900+0000","last_fullsized":"2023-12-07T10:18:12.756900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:53:49.228320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855125+0000","last_change":"2023-12-07T10:17:41.606895+0000","last_active":"2023-12-07T10:18:00.855125+0000","last_peered":"2023-12-07T10:18:00.855125+0000","last_clean":"2023-12-07T10:18:00.855125+0000","last_became_active":"2023-12-07T10:17:41.606596+0000","last_became_peered":"2023-12-07T10:17:41.606596+0000","last_unstale":"2023-12-07T10:18:00.855125+0000","last_undegraded":"2023-12-07T10:18:00.855125+0000","last_fullsized":"2023-12-07T10:18:00.855125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:34:25.166906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849382+0000","last_change":"2023-12-07T10:17:45.807101+0000","last_active":"2023-12-07T10:18:00.849382+0000","last_peered":"2023-12-07T10:18:00.849382+0000","last_clean":"2023-12-07T10:18:00.849382+0000","last_became_active":"2023-12-07T10:17:45.806770+0000","last_became_peered":"2023-12-07T10:17:45.806770+0000","last_unstale":"2023-12-07T10:18:00.849382+0000","last_undegraded":"2023-12-07T10:18:00.849382+0000","last_fullsized":"2023-12-07T10:18:00.849382+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:39:32.522968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.756459+0000","last_change":"2023-12-07T10:17:40.754856+0000","last_active":"2023-12-07T10:18:12.756459+0000","last_peered":"2023-12-07T10:18:12.756459+0000","last_clean":"2023-12-07T10:18:12.756459+0000","last_became_active":"2023-12-07T10:17:40.754488+0000","last_became_peered":"2023-12-07T10:17:40.754488+0000","last_unstale":"2023-12-07T10:18:12.756459+0000","last_undegraded":"2023-12-07T10:18:12.756459+0000","last_fullsized":"2023-12-07T10:18:12.756459+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:44:50.960216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856759+0000","last_change":"2023-12-07T10:17:41.606963+0000","last_active":"2023-12-07T10:18:00.856759+0000","last_peered":"2023-12-07T10:18:00.856759+0000","last_clean":"2023-12-07T10:18:00.856759+0000","last_became_active":"2023-12-07T10:17:41.606408+0000","last_became_peered":"2023-12-07T10:17:41.606408+0000","last_unstale":"2023-12-07T10:18:00.856759+0000","last_undegraded":"2023-12-07T10:18:00.856759+0000","last_fullsized":"2023-12-07T10:18:00.856759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:25:47.906649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.846470+0000","last_change":"2023-12-07T10:17:45.803867+0000","last_active":"2023-12-07T10:18:00.846470+0000","last_peered":"2023-12-07T10:18:00.846470+0000","last_clean":"2023-12-07T10:18:00.846470+0000","last_became_active":"2023-12-07T10:17:45.802664+0000","last_became_peered":"2023-12-07T10:17:45.802664+0000","last_unstale":"2023-12-07T10:18:00.846470+0000","last_undegraded":"2023-12-07T10:18:00.846470+0000","last_fullsized":"2023-12-07T10:18:00.846470+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:32:33.975307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":71,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:55.829483+0000","last_change":"2023-12-07T10:17:40.747494+0000","last_active":"2023-12-07T10:18:55.829483+0000","last_peered":"2023-12-07T10:18:55.829483+0000","last_clean":"2023-12-07T10:18:55.829483+0000","last_became_active":"2023-12-07T10:17:40.747157+0000","last_became_peered":"2023-12-07T10:17:40.747157+0000","last_unstale":"2023-12-07T10:18:55.829483+0000","last_undegraded":"2023-12-07T10:18:55.829483+0000","last_fullsized":"2023-12-07T10:18:55.829483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:45:24.395418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":976,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":47,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856481+0000","last_change":"2023-12-07T10:17:41.609495+0000","last_active":"2023-12-07T10:18:00.856481+0000","last_peered":"2023-12-07T10:18:00.856481+0000","last_clean":"2023-12-07T10:18:00.856481+0000","last_became_active":"2023-12-07T10:17:41.609181+0000","last_became_peered":"2023-12-07T10:17:41.609181+0000","last_unstale":"2023-12-07T10:18:00.856481+0000","last_undegraded":"2023-12-07T10:18:00.856481+0000","last_fullsized":"2023-12-07T10:18:00.856481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:20:33.473824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855858+0000","last_change":"2023-12-07T10:17:45.808862+0000","last_active":"2023-12-07T10:18:00.855858+0000","last_peered":"2023-12-07T10:18:00.855858+0000","last_clean":"2023-12-07T10:18:00.855858+0000","last_became_active":"2023-12-07T10:17:45.808461+0000","last_became_peered":"2023-12-07T10:17:45.808461+0000","last_unstale":"2023-12-07T10:18:00.855858+0000","last_undegraded":"2023-12-07T10:18:00.855858+0000","last_fullsized":"2023-12-07T10:18:00.855858+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:09:12.598126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.756367+0000","last_change":"2023-12-07T10:18:02.263709+0000","last_active":"2023-12-07T10:18:12.756367+0000","last_peered":"2023-12-07T10:18:12.756367+0000","last_clean":"2023-12-07T10:18:12.756367+0000","last_became_active":"2023-12-07T10:18:02.255788+0000","last_became_peered":"2023-12-07T10:18:02.255788+0000","last_unstale":"2023-12-07T10:18:12.756367+0000","last_undegraded":"2023-12-07T10:18:12.756367+0000","last_fullsized":"2023-12-07T10:18:12.756367+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:12:04.976226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.854358+0000","last_change":"2023-12-07T10:17:41.610509+0000","last_active":"2023-12-07T10:18:00.854358+0000","last_peered":"2023-12-07T10:18:00.854358+0000","last_clean":"2023-12-07T10:18:00.854358+0000","last_became_active":"2023-12-07T10:17:41.610121+0000","last_became_peered":"2023-12-07T10:17:41.610121+0000","last_unstale":"2023-12-07T10:18:00.854358+0000","last_undegraded":"2023-12-07T10:18:00.854358+0000","last_fullsized":"2023-12-07T10:18:00.854358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:53:17.365880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.854272+0000","last_change":"2023-12-07T10:17:45.795818+0000","last_active":"2023-12-07T10:18:00.854272+0000","last_peered":"2023-12-07T10:18:00.854272+0000","last_clean":"2023-12-07T10:18:00.854272+0000","last_became_active":"2023-12-07T10:17:45.795477+0000","last_became_peered":"2023-12-07T10:17:45.795477+0000","last_unstale":"2023-12-07T10:18:00.854272+0000","last_undegraded":"2023-12-07T10:18:00.854272+0000","last_fullsized":"2023-12-07T10:18:00.854272+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:17:21.494654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:10.731357+0000","last_change":"2023-12-07T10:17:40.751791+0000","last_active":"2023-12-07T10:18:10.731357+0000","last_peered":"2023-12-07T10:18:10.731357+0000","last_clean":"2023-12-07T10:18:10.731357+0000","last_became_active":"2023-12-07T10:17:40.751502+0000","last_became_peered":"2023-12-07T10:17:40.751502+0000","last_unstale":"2023-12-07T10:18:10.731357+0000","last_undegraded":"2023-12-07T10:18:10.731357+0000","last_fullsized":"2023-12-07T10:18:10.731357+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:15:52.319265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.256247+0000","last_change":"2023-12-07T10:18:02.256247+0000","last_active":"2023-12-07T10:18:02.256247+0000","last_peered":"2023-12-07T10:18:02.256247+0000","last_clean":"2023-12-07T10:18:02.256247+0000","last_became_active":"2023-12-07T10:18:02.255732+0000","last_became_peered":"2023-12-07T10:18:02.255732+0000","last_unstale":"2023-12-07T10:18:02.256247+0000","last_undegraded":"2023-12-07T10:18:02.256247+0000","last_fullsized":"2023-12-07T10:18:02.256247+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:00:06.516804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.853635+0000","last_change":"2023-12-07T10:17:45.791966+0000","last_active":"2023-12-07T10:18:00.853635+0000","last_peered":"2023-12-07T10:18:00.853635+0000","last_clean":"2023-12-07T10:18:00.853635+0000","last_became_active":"2023-12-07T10:17:45.791613+0000","last_became_peered":"2023-12-07T10:17:45.791613+0000","last_unstale":"2023-12-07T10:18:00.853635+0000","last_undegraded":"2023-12-07T10:18:00.853635+0000","last_fullsized":"2023-12-07T10:18:00.853635+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:13:02.291289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.853559+0000","last_change":"2023-12-07T10:17:40.762896+0000","last_active":"2023-12-07T10:18:01.853559+0000","last_peered":"2023-12-07T10:18:01.853559+0000","last_clean":"2023-12-07T10:18:01.853559+0000","last_became_active":"2023-12-07T10:17:40.762427+0000","last_became_peered":"2023-12-07T10:17:40.762427+0000","last_unstale":"2023-12-07T10:18:01.853559+0000","last_undegraded":"2023-12-07T10:18:01.853559+0000","last_fullsized":"2023-12-07T10:18:01.853559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:56:35.589548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.854524+0000","last_change":"2023-12-07T10:17:41.610354+0000","last_active":"2023-12-07T10:18:00.854524+0000","last_peered":"2023-12-07T10:18:00.854524+0000","last_clean":"2023-12-07T10:18:00.854524+0000","last_became_active":"2023-12-07T10:17:41.609888+0000","last_became_peered":"2023-12-07T10:17:41.609888+0000","last_unstale":"2023-12-07T10:18:00.854524+0000","last_undegraded":"2023-12-07T10:18:00.854524+0000","last_fullsized":"2023-12-07T10:18:00.854524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:48:19.976949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855124+0000","last_change":"2023-12-07T10:17:45.807324+0000","last_active":"2023-12-07T10:18:00.855124+0000","last_peered":"2023-12-07T10:18:00.855124+0000","last_clean":"2023-12-07T10:18:00.855124+0000","last_became_active":"2023-12-07T10:17:45.806036+0000","last_became_peered":"2023-12-07T10:17:45.806036+0000","last_unstale":"2023-12-07T10:18:00.855124+0000","last_undegraded":"2023-12-07T10:18:00.855124+0000","last_fullsized":"2023-12-07T10:18:00.855124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:54:36.847992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'5","reported_seq":72,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:58.834748+0000","last_change":"2023-12-07T10:17:40.754247+0000","last_active":"2023-12-07T10:18:58.834748+0000","last_peered":"2023-12-07T10:18:58.834748+0000","last_clean":"2023-12-07T10:18:58.834748+0000","last_became_active":"2023-12-07T10:17:40.753964+0000","last_became_peered":"2023-12-07T10:17:40.753964+0000","last_unstale":"2023-12-07T10:18:58.834748+0000","last_undegraded":"2023-12-07T10:18:58.834748+0000","last_fullsized":"2023-12-07T10:18:58.834748+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:22:04.832399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.854494+0000","last_change":"2023-12-07T10:17:41.610516+0000","last_active":"2023-12-07T10:18:00.854494+0000","last_peered":"2023-12-07T10:18:00.854494+0000","last_clean":"2023-12-07T10:18:00.854494+0000","last_became_active":"2023-12-07T10:17:41.610184+0000","last_became_peered":"2023-12-07T10:17:41.610184+0000","last_unstale":"2023-12-07T10:18:00.854494+0000","last_undegraded":"2023-12-07T10:18:00.854494+0000","last_fullsized":"2023-12-07T10:18:00.854494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:53:43.100723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855260+0000","last_change":"2023-12-07T10:17:45.804624+0000","last_active":"2023-12-07T10:18:00.855260+0000","last_peered":"2023-12-07T10:18:00.855260+0000","last_clean":"2023-12-07T10:18:00.855260+0000","last_became_active":"2023-12-07T10:17:45.803667+0000","last_became_peered":"2023-12-07T10:17:45.803667+0000","last_unstale":"2023-12-07T10:18:00.855260+0000","last_undegraded":"2023-12-07T10:18:00.855260+0000","last_fullsized":"2023-12-07T10:18:00.855260+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:55:09.642795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855573+0000","last_change":"2023-12-07T10:17:41.618433+0000","last_active":"2023-12-07T10:18:00.855573+0000","last_peered":"2023-12-07T10:18:00.855573+0000","last_clean":"2023-12-07T10:18:00.855573+0000","last_became_active":"2023-12-07T10:17:41.617942+0000","last_became_peered":"2023-12-07T10:17:41.617942+0000","last_unstale":"2023-12-07T10:18:00.855573+0000","last_undegraded":"2023-12-07T10:18:00.855573+0000","last_fullsized":"2023-12-07T10:18:00.855573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:26:48.166157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":70,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:56.830170+0000","last_change":"2023-12-07T10:17:40.761907+0000","last_active":"2023-12-07T10:18:56.830170+0000","last_peered":"2023-12-07T10:18:56.830170+0000","last_clean":"2023-12-07T10:18:56.830170+0000","last_became_active":"2023-12-07T10:17:40.761564+0000","last_became_peered":"2023-12-07T10:17:40.761564+0000","last_unstale":"2023-12-07T10:18:56.830170+0000","last_undegraded":"2023-12-07T10:18:56.830170+0000","last_fullsized":"2023-12-07T10:18:56.830170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:57:22.304352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":46,"num_read_kb":46,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.854203+0000","last_change":"2023-12-07T10:17:45.792684+0000","last_active":"2023-12-07T10:18:01.854203+0000","last_peered":"2023-12-07T10:18:01.854203+0000","last_clean":"2023-12-07T10:18:01.854203+0000","last_became_active":"2023-12-07T10:17:45.792172+0000","last_became_peered":"2023-12-07T10:17:45.792172+0000","last_unstale":"2023-12-07T10:18:01.854203+0000","last_undegraded":"2023-12-07T10:18:01.854203+0000","last_fullsized":"2023-12-07T10:18:01.854203+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:24:31.084727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849807+0000","last_change":"2023-12-07T10:17:41.612582+0000","last_active":"2023-12-07T10:18:00.849807+0000","last_peered":"2023-12-07T10:18:00.849807+0000","last_clean":"2023-12-07T10:18:00.849807+0000","last_became_active":"2023-12-07T10:17:41.612244+0000","last_became_peered":"2023-12-07T10:17:41.612244+0000","last_unstale":"2023-12-07T10:18:00.849807+0000","last_undegraded":"2023-12-07T10:18:00.849807+0000","last_fullsized":"2023-12-07T10:18:00.849807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:41:53.506037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:13.770053+0000","last_change":"2023-12-07T10:17:40.748565+0000","last_active":"2023-12-07T10:18:13.770053+0000","last_peered":"2023-12-07T10:18:13.770053+0000","last_clean":"2023-12-07T10:18:13.770053+0000","last_became_active":"2023-12-07T10:17:40.748250+0000","last_became_peered":"2023-12-07T10:17:40.748250+0000","last_unstale":"2023-12-07T10:18:13.770053+0000","last_undegraded":"2023-12-07T10:18:13.770053+0000","last_fullsized":"2023-12-07T10:18:13.770053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:26:28.202440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":326,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856336+0000","last_change":"2023-12-07T10:16:33.482173+0000","last_active":"2023-12-07T10:18:00.856336+0000","last_peered":"2023-12-07T10:18:00.856336+0000","last_clean":"2023-12-07T10:18:00.856336+0000","last_became_active":"2023-12-07T10:16:33.481699+0000","last_became_peered":"2023-12-07T10:16:33.481699+0000","last_unstale":"2023-12-07T10:18:00.856336+0000","last_undegraded":"2023-12-07T10:18:00.856336+0000","last_fullsized":"2023-12-07T10:18:00.856336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:13:25.388258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:13:25.388258+0000","last_clean_scrub_stamp":"2023-12-07T10:13:25.388258+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:57:09.583677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856130+0000","last_change":"2023-12-07T10:17:45.806289+0000","last_active":"2023-12-07T10:18:00.856130+0000","last_peered":"2023-12-07T10:18:00.856130+0000","last_clean":"2023-12-07T10:18:00.856130+0000","last_became_active":"2023-12-07T10:17:45.805954+0000","last_became_peered":"2023-12-07T10:17:45.805954+0000","last_unstale":"2023-12-07T10:18:00.856130+0000","last_undegraded":"2023-12-07T10:18:00.856130+0000","last_fullsized":"2023-12-07T10:18:00.856130+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:21:09.955380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855381+0000","last_change":"2023-12-07T10:17:41.612322+0000","last_active":"2023-12-07T10:18:00.855381+0000","last_peered":"2023-12-07T10:18:00.855381+0000","last_clean":"2023-12-07T10:18:00.855381+0000","last_became_active":"2023-12-07T10:17:41.612039+0000","last_became_peered":"2023-12-07T10:17:41.612039+0000","last_unstale":"2023-12-07T10:18:00.855381+0000","last_undegraded":"2023-12-07T10:18:00.855381+0000","last_fullsized":"2023-12-07T10:18:00.855381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:40:49.883742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:10.742637+0000","last_change":"2023-12-07T10:17:40.749694+0000","last_active":"2023-12-07T10:18:10.742637+0000","last_peered":"2023-12-07T10:18:10.742637+0000","last_clean":"2023-12-07T10:18:10.742637+0000","last_became_active":"2023-12-07T10:17:40.749393+0000","last_became_peered":"2023-12-07T10:17:40.749393+0000","last_unstale":"2023-12-07T10:18:10.742637+0000","last_undegraded":"2023-12-07T10:18:10.742637+0000","last_fullsized":"2023-12-07T10:18:10.742637+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:15:36.949062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849339+0000","last_change":"2023-12-07T10:17:45.805769+0000","last_active":"2023-12-07T10:18:00.849339+0000","last_peered":"2023-12-07T10:18:00.849339+0000","last_clean":"2023-12-07T10:18:00.849339+0000","last_became_active":"2023-12-07T10:17:45.805346+0000","last_became_peered":"2023-12-07T10:17:45.805346+0000","last_unstale":"2023-12-07T10:18:00.849339+0000","last_undegraded":"2023-12-07T10:18:00.849339+0000","last_fullsized":"2023-12-07T10:18:00.849339+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:47:47.781933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:11.749790+0000","last_change":"2023-12-07T10:18:01.862558+0000","last_active":"2023-12-07T10:18:11.749790+0000","last_peered":"2023-12-07T10:18:11.749790+0000","last_clean":"2023-12-07T10:18:11.749790+0000","last_became_active":"2023-12-07T10:18:01.862063+0000","last_became_peered":"2023-12-07T10:18:01.862063+0000","last_unstale":"2023-12-07T10:18:11.749790+0000","last_undegraded":"2023-12-07T10:18:11.749790+0000","last_fullsized":"2023-12-07T10:18:11.749790+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:25:32.652160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855264+0000","last_change":"2023-12-07T10:17:41.617199+0000","last_active":"2023-12-07T10:18:00.855264+0000","last_peered":"2023-12-07T10:18:00.855264+0000","last_clean":"2023-12-07T10:18:00.855264+0000","last_became_active":"2023-12-07T10:17:41.616901+0000","last_became_peered":"2023-12-07T10:17:41.616901+0000","last_unstale":"2023-12-07T10:18:00.855264+0000","last_undegraded":"2023-12-07T10:18:00.855264+0000","last_fullsized":"2023-12-07T10:18:00.855264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:23:59.433861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849025+0000","last_change":"2023-12-07T10:17:45.790914+0000","last_active":"2023-12-07T10:18:00.849025+0000","last_peered":"2023-12-07T10:18:00.849025+0000","last_clean":"2023-12-07T10:18:00.849025+0000","last_became_active":"2023-12-07T10:17:45.790428+0000","last_became_peered":"2023-12-07T10:17:45.790428+0000","last_unstale":"2023-12-07T10:18:00.849025+0000","last_undegraded":"2023-12-07T10:18:00.849025+0000","last_fullsized":"2023-12-07T10:18:00.849025+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:03:57.516300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'10","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:18.543684+0000","last_change":"2023-12-07T10:18:01.863765+0000","last_active":"2023-12-07T10:18:18.543684+0000","last_peered":"2023-12-07T10:18:18.543684+0000","last_clean":"2023-12-07T10:18:18.543684+0000","last_became_active":"2023-12-07T10:18:01.858926+0000","last_became_peered":"2023-12-07T10:18:01.858926+0000","last_unstale":"2023-12-07T10:18:18.543684+0000","last_undegraded":"2023-12-07T10:18:18.543684+0000","last_fullsized":"2023-12-07T10:18:18.543684+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:44:19.663592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849322+0000","last_change":"2023-12-07T10:17:41.608513+0000","last_active":"2023-12-07T10:18:00.849322+0000","last_peered":"2023-12-07T10:18:00.849322+0000","last_clean":"2023-12-07T10:18:00.849322+0000","last_became_active":"2023-12-07T10:17:41.605049+0000","last_became_peered":"2023-12-07T10:17:41.605049+0000","last_unstale":"2023-12-07T10:18:00.849322+0000","last_undegraded":"2023-12-07T10:18:00.849322+0000","last_fullsized":"2023-12-07T10:18:00.849322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:33:00.730827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.850303+0000","last_change":"2023-12-07T10:17:45.792583+0000","last_active":"2023-12-07T10:18:00.850303+0000","last_peered":"2023-12-07T10:18:00.850303+0000","last_clean":"2023-12-07T10:18:00.850303+0000","last_became_active":"2023-12-07T10:17:45.792162+0000","last_became_peered":"2023-12-07T10:17:45.792162+0000","last_unstale":"2023-12-07T10:18:00.850303+0000","last_undegraded":"2023-12-07T10:18:00.850303+0000","last_fullsized":"2023-12-07T10:18:00.850303+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:47:30.901477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.757478+0000","last_change":"2023-12-07T10:17:40.750826+0000","last_active":"2023-12-07T10:18:12.757478+0000","last_peered":"2023-12-07T10:18:12.757478+0000","last_clean":"2023-12-07T10:18:12.757478+0000","last_became_active":"2023-12-07T10:17:40.750519+0000","last_became_peered":"2023-12-07T10:17:40.750519+0000","last_unstale":"2023-12-07T10:18:12.757478+0000","last_undegraded":"2023-12-07T10:18:12.757478+0000","last_fullsized":"2023-12-07T10:18:12.757478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:21:01.008897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.853040+0000","last_change":"2023-12-07T10:17:41.609488+0000","last_active":"2023-12-07T10:18:00.853040+0000","last_peered":"2023-12-07T10:18:00.853040+0000","last_clean":"2023-12-07T10:18:00.853040+0000","last_became_active":"2023-12-07T10:17:41.609061+0000","last_became_peered":"2023-12-07T10:17:41.609061+0000","last_unstale":"2023-12-07T10:18:00.853040+0000","last_undegraded":"2023-12-07T10:18:00.853040+0000","last_fullsized":"2023-12-07T10:18:00.853040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:39:32.729502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852501+0000","last_change":"2023-12-07T10:17:45.797573+0000","last_active":"2023-12-07T10:18:00.852501+0000","last_peered":"2023-12-07T10:18:00.852501+0000","last_clean":"2023-12-07T10:18:00.852501+0000","last_became_active":"2023-12-07T10:17:45.797119+0000","last_became_peered":"2023-12-07T10:17:45.797119+0000","last_unstale":"2023-12-07T10:18:00.852501+0000","last_undegraded":"2023-12-07T10:18:00.852501+0000","last_fullsized":"2023-12-07T10:18:00.852501+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:17:27.100470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:09.901504+0000","last_change":"2023-12-07T10:17:40.753596+0000","last_active":"2023-12-07T10:18:09.901504+0000","last_peered":"2023-12-07T10:18:09.901504+0000","last_clean":"2023-12-07T10:18:09.901504+0000","last_became_active":"2023-12-07T10:17:40.753251+0000","last_became_peered":"2023-12-07T10:17:40.753251+0000","last_unstale":"2023-12-07T10:18:09.901504+0000","last_undegraded":"2023-12-07T10:18:09.901504+0000","last_fullsized":"2023-12-07T10:18:09.901504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:08:30.868794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852298+0000","last_change":"2023-12-07T10:17:41.599689+0000","last_active":"2023-12-07T10:18:00.852298+0000","last_peered":"2023-12-07T10:18:00.852298+0000","last_clean":"2023-12-07T10:18:00.852298+0000","last_became_active":"2023-12-07T10:17:41.599358+0000","last_became_peered":"2023-12-07T10:17:41.599358+0000","last_unstale":"2023-12-07T10:18:00.852298+0000","last_undegraded":"2023-12-07T10:18:00.852298+0000","last_fullsized":"2023-12-07T10:18:00.852298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:44:19.742560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.846613+0000","last_change":"2023-12-07T10:17:45.800231+0000","last_active":"2023-12-07T10:18:00.846613+0000","last_peered":"2023-12-07T10:18:00.846613+0000","last_clean":"2023-12-07T10:18:00.846613+0000","last_became_active":"2023-12-07T10:17:45.799585+0000","last_became_peered":"2023-12-07T10:17:45.799585+0000","last_unstale":"2023-12-07T10:18:00.846613+0000","last_undegraded":"2023-12-07T10:18:00.846613+0000","last_fullsized":"2023-12-07T10:18:00.846613+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:08:36.887582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.756917+0000","last_change":"2023-12-07T10:17:40.758663+0000","last_active":"2023-12-07T10:18:12.756917+0000","last_peered":"2023-12-07T10:18:12.756917+0000","last_clean":"2023-12-07T10:18:12.756917+0000","last_became_active":"2023-12-07T10:17:40.753611+0000","last_became_peered":"2023-12-07T10:17:40.753611+0000","last_unstale":"2023-12-07T10:18:12.756917+0000","last_undegraded":"2023-12-07T10:18:12.756917+0000","last_fullsized":"2023-12-07T10:18:12.756917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:00:59.256468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849063+0000","last_change":"2023-12-07T10:17:41.616546+0000","last_active":"2023-12-07T10:18:00.849063+0000","last_peered":"2023-12-07T10:18:00.849063+0000","last_clean":"2023-12-07T10:18:00.849063+0000","last_became_active":"2023-12-07T10:17:41.615780+0000","last_became_peered":"2023-12-07T10:17:41.615780+0000","last_unstale":"2023-12-07T10:18:00.849063+0000","last_undegraded":"2023-12-07T10:18:00.849063+0000","last_fullsized":"2023-12-07T10:18:00.849063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:22:32.455784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.254713+0000","last_change":"2023-12-07T10:17:45.797550+0000","last_active":"2023-12-07T10:18:02.254713+0000","last_peered":"2023-12-07T10:18:02.254713+0000","last_clean":"2023-12-07T10:18:02.254713+0000","last_became_active":"2023-12-07T10:17:45.797140+0000","last_became_peered":"2023-12-07T10:17:45.797140+0000","last_unstale":"2023-12-07T10:18:02.254713+0000","last_undegraded":"2023-12-07T10:18:02.254713+0000","last_fullsized":"2023-12-07T10:18:02.254713+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:53:44.984904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:13.048342+0000","last_change":"2023-12-07T10:17:40.748410+0000","last_active":"2023-12-07T10:18:13.048342+0000","last_peered":"2023-12-07T10:18:13.048342+0000","last_clean":"2023-12-07T10:18:13.048342+0000","last_became_active":"2023-12-07T10:17:40.748069+0000","last_became_peered":"2023-12-07T10:17:40.748069+0000","last_unstale":"2023-12-07T10:18:13.048342+0000","last_undegraded":"2023-12-07T10:18:13.048342+0000","last_fullsized":"2023-12-07T10:18:13.048342+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:51:48.278974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849910+0000","last_change":"2023-12-07T10:17:41.609248+0000","last_active":"2023-12-07T10:18:00.849910+0000","last_peered":"2023-12-07T10:18:00.849910+0000","last_clean":"2023-12-07T10:18:00.849910+0000","last_became_active":"2023-12-07T10:17:41.608943+0000","last_became_peered":"2023-12-07T10:17:41.608943+0000","last_unstale":"2023-12-07T10:18:00.849910+0000","last_undegraded":"2023-12-07T10:18:00.849910+0000","last_fullsized":"2023-12-07T10:18:00.849910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:29:32.322273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852512+0000","last_change":"2023-12-07T10:17:45.794855+0000","last_active":"2023-12-07T10:18:00.852512+0000","last_peered":"2023-12-07T10:18:00.852512+0000","last_clean":"2023-12-07T10:18:00.852512+0000","last_became_active":"2023-12-07T10:17:45.794410+0000","last_became_peered":"2023-12-07T10:17:45.794410+0000","last_unstale":"2023-12-07T10:18:00.852512+0000","last_undegraded":"2023-12-07T10:18:00.852512+0000","last_fullsized":"2023-12-07T10:18:00.852512+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:57:25.118036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:09.900503+0000","last_change":"2023-12-07T10:17:40.750471+0000","last_active":"2023-12-07T10:18:09.900503+0000","last_peered":"2023-12-07T10:18:09.900503+0000","last_clean":"2023-12-07T10:18:09.900503+0000","last_became_active":"2023-12-07T10:17:40.750176+0000","last_became_peered":"2023-12-07T10:17:40.750176+0000","last_unstale":"2023-12-07T10:18:09.900503+0000","last_undegraded":"2023-12-07T10:18:09.900503+0000","last_fullsized":"2023-12-07T10:18:09.900503+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:04:22.305343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856679+0000","last_change":"2023-12-07T10:17:41.614087+0000","last_active":"2023-12-07T10:18:00.856679+0000","last_peered":"2023-12-07T10:18:00.856679+0000","last_clean":"2023-12-07T10:18:00.856679+0000","last_became_active":"2023-12-07T10:17:41.607689+0000","last_became_peered":"2023-12-07T10:17:41.607689+0000","last_unstale":"2023-12-07T10:18:00.856679+0000","last_undegraded":"2023-12-07T10:18:00.856679+0000","last_fullsized":"2023-12-07T10:18:00.856679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:50:10.316085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.847396+0000","last_change":"2023-12-07T10:17:45.791234+0000","last_active":"2023-12-07T10:18:00.847396+0000","last_peered":"2023-12-07T10:18:00.847396+0000","last_clean":"2023-12-07T10:18:00.847396+0000","last_became_active":"2023-12-07T10:17:45.790785+0000","last_became_peered":"2023-12-07T10:17:45.790785+0000","last_unstale":"2023-12-07T10:18:00.847396+0000","last_undegraded":"2023-12-07T10:18:00.847396+0000","last_fullsized":"2023-12-07T10:18:00.847396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:46:24.980528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'5","reported_seq":72,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:57.831918+0000","last_change":"2023-12-07T10:17:40.753645+0000","last_active":"2023-12-07T10:18:57.831918+0000","last_peered":"2023-12-07T10:18:57.831918+0000","last_clean":"2023-12-07T10:18:57.831918+0000","last_became_active":"2023-12-07T10:17:40.753326+0000","last_became_peered":"2023-12-07T10:17:40.753326+0000","last_unstale":"2023-12-07T10:18:57.831918+0000","last_undegraded":"2023-12-07T10:18:57.831918+0000","last_fullsized":"2023-12-07T10:18:57.831918+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:43:03.617591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":48,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.857048+0000","last_change":"2023-12-07T10:17:41.607968+0000","last_active":"2023-12-07T10:18:00.857048+0000","last_peered":"2023-12-07T10:18:00.857048+0000","last_clean":"2023-12-07T10:18:00.857048+0000","last_became_active":"2023-12-07T10:17:41.607621+0000","last_became_peered":"2023-12-07T10:17:41.607621+0000","last_unstale":"2023-12-07T10:18:00.857048+0000","last_undegraded":"2023-12-07T10:18:00.857048+0000","last_fullsized":"2023-12-07T10:18:00.857048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:55:10.224738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852172+0000","last_change":"2023-12-07T10:17:45.793678+0000","last_active":"2023-12-07T10:18:00.852172+0000","last_peered":"2023-12-07T10:18:00.852172+0000","last_clean":"2023-12-07T10:18:00.852172+0000","last_became_active":"2023-12-07T10:17:45.793283+0000","last_became_peered":"2023-12-07T10:17:45.793283+0000","last_unstale":"2023-12-07T10:18:00.852172+0000","last_undegraded":"2023-12-07T10:18:00.852172+0000","last_fullsized":"2023-12-07T10:18:00.852172+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:33:56.501107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:11.749969+0000","last_change":"2023-12-07T10:17:40.762069+0000","last_active":"2023-12-07T10:18:11.749969+0000","last_peered":"2023-12-07T10:18:11.749969+0000","last_clean":"2023-12-07T10:18:11.749969+0000","last_became_active":"2023-12-07T10:17:40.761752+0000","last_became_peered":"2023-12-07T10:17:40.761752+0000","last_unstale":"2023-12-07T10:18:11.749969+0000","last_undegraded":"2023-12-07T10:18:11.749969+0000","last_fullsized":"2023-12-07T10:18:11.749969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:57:55.736144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856757+0000","last_change":"2023-12-07T10:17:41.607347+0000","last_active":"2023-12-07T10:18:00.856757+0000","last_peered":"2023-12-07T10:18:00.856757+0000","last_clean":"2023-12-07T10:18:00.856757+0000","last_became_active":"2023-12-07T10:17:41.606618+0000","last_became_peered":"2023-12-07T10:17:41.606618+0000","last_unstale":"2023-12-07T10:18:00.856757+0000","last_undegraded":"2023-12-07T10:18:00.856757+0000","last_fullsized":"2023-12-07T10:18:00.856757+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:53:34.748130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855194+0000","last_change":"2023-12-07T10:17:45.791459+0000","last_active":"2023-12-07T10:18:00.855194+0000","last_peered":"2023-12-07T10:18:00.855194+0000","last_clean":"2023-12-07T10:18:00.855194+0000","last_became_active":"2023-12-07T10:17:45.791070+0000","last_became_peered":"2023-12-07T10:17:45.791070+0000","last_unstale":"2023-12-07T10:18:00.855194+0000","last_undegraded":"2023-12-07T10:18:00.855194+0000","last_fullsized":"2023-12-07T10:18:00.855194+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:46:35.985990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.756703+0000","last_change":"2023-12-07T10:17:40.762726+0000","last_active":"2023-12-07T10:18:12.756703+0000","last_peered":"2023-12-07T10:18:12.756703+0000","last_clean":"2023-12-07T10:18:12.756703+0000","last_became_active":"2023-12-07T10:17:40.762378+0000","last_became_peered":"2023-12-07T10:17:40.762378+0000","last_unstale":"2023-12-07T10:18:12.756703+0000","last_undegraded":"2023-12-07T10:18:12.756703+0000","last_fullsized":"2023-12-07T10:18:12.756703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:42:06.193834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856972+0000","last_change":"2023-12-07T10:17:41.602356+0000","last_active":"2023-12-07T10:18:00.856972+0000","last_peered":"2023-12-07T10:18:00.856972+0000","last_clean":"2023-12-07T10:18:00.856972+0000","last_became_active":"2023-12-07T10:17:41.602025+0000","last_became_peered":"2023-12-07T10:17:41.602025+0000","last_unstale":"2023-12-07T10:18:00.856972+0000","last_undegraded":"2023-12-07T10:18:00.856972+0000","last_fullsized":"2023-12-07T10:18:00.856972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:33:04.994510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849064+0000","last_change":"2023-12-07T10:17:41.616766+0000","last_active":"2023-12-07T10:18:00.849064+0000","last_peered":"2023-12-07T10:18:00.849064+0000","last_clean":"2023-12-07T10:18:00.849064+0000","last_became_active":"2023-12-07T10:17:41.616221+0000","last_became_peered":"2023-12-07T10:17:41.616221+0000","last_unstale":"2023-12-07T10:18:00.849064+0000","last_undegraded":"2023-12-07T10:18:00.849064+0000","last_fullsized":"2023-12-07T10:18:00.849064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:16:00.393772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.850406+0000","last_change":"2023-12-07T10:17:41.613646+0000","last_active":"2023-12-07T10:18:00.850406+0000","last_peered":"2023-12-07T10:18:00.850406+0000","last_clean":"2023-12-07T10:18:00.850406+0000","last_became_active":"2023-12-07T10:17:41.613257+0000","last_became_peered":"2023-12-07T10:17:41.613257+0000","last_unstale":"2023-12-07T10:18:00.850406+0000","last_undegraded":"2023-12-07T10:18:00.850406+0000","last_fullsized":"2023-12-07T10:18:00.850406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:10:33.416955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849853+0000","last_change":"2023-12-07T10:17:41.620017+0000","last_active":"2023-12-07T10:18:00.849853+0000","last_peered":"2023-12-07T10:18:00.849853+0000","last_clean":"2023-12-07T10:18:00.849853+0000","last_became_active":"2023-12-07T10:17:41.619647+0000","last_became_peered":"2023-12-07T10:17:41.619647+0000","last_unstale":"2023-12-07T10:18:00.849853+0000","last_undegraded":"2023-12-07T10:18:00.849853+0000","last_fullsized":"2023-12-07T10:18:00.849853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:36:42.891160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852495+0000","last_change":"2023-12-07T10:17:41.610803+0000","last_active":"2023-12-07T10:18:00.852495+0000","last_peered":"2023-12-07T10:18:00.852495+0000","last_clean":"2023-12-07T10:18:00.852495+0000","last_became_active":"2023-12-07T10:17:41.610388+0000","last_became_peered":"2023-12-07T10:17:41.610388+0000","last_unstale":"2023-12-07T10:18:00.852495+0000","last_undegraded":"2023-12-07T10:18:00.852495+0000","last_fullsized":"2023-12-07T10:18:00.852495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:33:25.787281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855345+0000","last_change":"2023-12-07T10:17:41.615916+0000","last_active":"2023-12-07T10:18:00.855345+0000","last_peered":"2023-12-07T10:18:00.855345+0000","last_clean":"2023-12-07T10:18:00.855345+0000","last_became_active":"2023-12-07T10:17:41.615591+0000","last_became_peered":"2023-12-07T10:17:41.615591+0000","last_unstale":"2023-12-07T10:18:00.855345+0000","last_undegraded":"2023-12-07T10:18:00.855345+0000","last_fullsized":"2023-12-07T10:18:00.855345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:40:45.541571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.850876+0000","last_change":"2023-12-07T10:17:41.614735+0000","last_active":"2023-12-07T10:18:00.850876+0000","last_peered":"2023-12-07T10:18:00.850876+0000","last_clean":"2023-12-07T10:18:00.850876+0000","last_became_active":"2023-12-07T10:17:41.614388+0000","last_became_peered":"2023-12-07T10:17:41.614388+0000","last_unstale":"2023-12-07T10:18:00.850876+0000","last_undegraded":"2023-12-07T10:18:00.850876+0000","last_fullsized":"2023-12-07T10:18:00.850876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:10:07.196841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.850893+0000","last_change":"2023-12-07T10:17:41.620059+0000","last_active":"2023-12-07T10:18:00.850893+0000","last_peered":"2023-12-07T10:18:00.850893+0000","last_clean":"2023-12-07T10:18:00.850893+0000","last_became_active":"2023-12-07T10:17:41.619770+0000","last_became_peered":"2023-12-07T10:17:41.619770+0000","last_unstale":"2023-12-07T10:18:00.850893+0000","last_undegraded":"2023-12-07T10:18:00.850893+0000","last_fullsized":"2023-12-07T10:18:00.850893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:13:43.421776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849472+0000","last_change":"2023-12-07T10:17:41.609720+0000","last_active":"2023-12-07T10:18:00.849472+0000","last_peered":"2023-12-07T10:18:00.849472+0000","last_clean":"2023-12-07T10:18:00.849472+0000","last_became_active":"2023-12-07T10:17:41.609417+0000","last_became_peered":"2023-12-07T10:17:41.609417+0000","last_unstale":"2023-12-07T10:18:00.849472+0000","last_undegraded":"2023-12-07T10:18:00.849472+0000","last_fullsized":"2023-12-07T10:18:00.849472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:48:47.768754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855496+0000","last_change":"2023-12-07T10:17:41.620952+0000","last_active":"2023-12-07T10:18:00.855496+0000","last_peered":"2023-12-07T10:18:00.855496+0000","last_clean":"2023-12-07T10:18:00.855496+0000","last_became_active":"2023-12-07T10:17:41.620560+0000","last_became_peered":"2023-12-07T10:17:41.620560+0000","last_unstale":"2023-12-07T10:18:00.855496+0000","last_undegraded":"2023-12-07T10:18:00.855496+0000","last_fullsized":"2023-12-07T10:18:00.855496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:51:37.699511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849382+0000","last_change":"2023-12-07T10:17:41.611908+0000","last_active":"2023-12-07T10:18:00.849382+0000","last_peered":"2023-12-07T10:18:00.849382+0000","last_clean":"2023-12-07T10:18:00.849382+0000","last_became_active":"2023-12-07T10:17:41.610433+0000","last_became_peered":"2023-12-07T10:17:41.610433+0000","last_unstale":"2023-12-07T10:18:00.849382+0000","last_undegraded":"2023-12-07T10:18:00.849382+0000","last_fullsized":"2023-12-07T10:18:00.849382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:42:53.906221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.847644+0000","last_change":"2023-12-07T10:17:41.604116+0000","last_active":"2023-12-07T10:18:00.847644+0000","last_peered":"2023-12-07T10:18:00.847644+0000","last_clean":"2023-12-07T10:18:00.847644+0000","last_became_active":"2023-12-07T10:17:41.603880+0000","last_became_peered":"2023-12-07T10:17:41.603880+0000","last_unstale":"2023-12-07T10:18:00.847644+0000","last_undegraded":"2023-12-07T10:18:00.847644+0000","last_fullsized":"2023-12-07T10:18:00.847644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:56:27.343169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855007+0000","last_change":"2023-12-07T10:17:41.610497+0000","last_active":"2023-12-07T10:18:00.855007+0000","last_peered":"2023-12-07T10:18:00.855007+0000","last_clean":"2023-12-07T10:18:00.855007+0000","last_became_active":"2023-12-07T10:17:41.609956+0000","last_became_peered":"2023-12-07T10:17:41.609956+0000","last_unstale":"2023-12-07T10:18:00.855007+0000","last_undegraded":"2023-12-07T10:18:00.855007+0000","last_fullsized":"2023-12-07T10:18:00.855007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:04:03.967377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849491+0000","last_change":"2023-12-07T10:17:41.612284+0000","last_active":"2023-12-07T10:18:00.849491+0000","last_peered":"2023-12-07T10:18:00.849491+0000","last_clean":"2023-12-07T10:18:00.849491+0000","last_became_active":"2023-12-07T10:17:41.611917+0000","last_became_peered":"2023-12-07T10:17:41.611917+0000","last_unstale":"2023-12-07T10:18:00.849491+0000","last_undegraded":"2023-12-07T10:18:00.849491+0000","last_fullsized":"2023-12-07T10:18:00.849491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:02:27.355883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":81645,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":513,"num_read_kb":346,"num_write":243,"num_write_kb":152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":244935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383707,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":498136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.434,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.629},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.72299999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.52600000000000002}]},{"osd":1,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.62,"15min":0.60499999999999998},"min":{"1min":0.505,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.40899999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.76800000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.40899999999999997}]},{"osd":2,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.45900000000000002,"15min":0.45000000000000001},"min":{"1min":0.36899999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.46999999999999997,"15min":0.45700000000000002},"min":{"1min":0.376,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.42999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.67900000000000005,"15min":0.67200000000000004},"min":{"1min":0.48899999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90300000000000002,"5min":1.046,"15min":1.046},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.84699999999999998,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.57799999999999996}]},{"osd":5,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.56799999999999995,"15min":0.55700000000000005},"min":{"1min":0.40200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80700000000000005,"5min":1.034,"15min":1.034},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.59199999999999997,"15min":0.57999999999999996},"min":{"1min":0.51300000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92900000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.51300000000000001}]},{"osd":6,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.46200000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90700000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.72399999999999998,"15min":0.71599999999999997},"min":{"1min":0.60399999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91300000000000003,"5min":1.363,"15min":1.363},"last":0.68200000000000005}]},{"osd":7,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.76100000000000001,"15min":0.751},"min":{"1min":0.63700000000000001,"5min":0.501,"15min":0.501},"max":{"1min":0.97899999999999998,"5min":1.25,"15min":1.25},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.70599999999999996,"15min":0.69599999999999995},"min":{"1min":0.51500000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95399999999999996,"5min":1.085,"15min":1.085},"last":0.54300000000000004}]},{"osd":8,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.72299999999999998}]},{"osd":9,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.78000000000000003,"15min":0.76600000000000001},"min":{"1min":0.69799999999999995,"5min":0.496,"15min":0.496},"max":{"1min":0.96799999999999997,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.753},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.78700000000000003,"15min":0.77500000000000002},"min":{"1min":0.71899999999999997,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.014,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.81599999999999995}]},{"osd":10,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.751,"15min":0.74099999999999999},"min":{"1min":0.59999999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.996,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.79300000000000004,"15min":0.78300000000000003},"min":{"1min":0.70999999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.999,"5min":1.411,"15min":1.411},"last":0.80400000000000005}]}]},{"osd":10,"up_from":72,"seq":309237645344,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1716224,"data_stored":1149350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.3610000000000002,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.66000000000000003}]},{"osd":1,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.48599999999999999,"15min":0.49099999999999999},"min":{"1min":0.34100000000000003,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.627,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.49299999999999999,"15min":0.495},"min":{"1min":0.36299999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.53400000000000003}]},{"osd":2,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82699999999999996,"5min":1.008,"15min":1.008},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.42699999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81299999999999994,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.64400000000000002}]},{"osd":3,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.45500000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.78300000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.67900000000000005,"15min":0.68799999999999994},"min":{"1min":0.48199999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.79000000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.63}]},{"osd":4,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57599999999999996,"15min":0.58299999999999996},"min":{"1min":0.33300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71799999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58599999999999997,"15min":0.59099999999999997},"min":{"1min":0.40699999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.751,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51600000000000001}]},{"osd":5,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.95499999999999996,"5min":1.22,"15min":1.22},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72699999999999998,"15min":0.72799999999999998},"min":{"1min":0.49299999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93899999999999995,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.70799999999999996}]},{"osd":6,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.747,"15min":0.751},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.89200000000000002,"5min":1.054,"15min":1.054},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78900000000000003,"15min":0.78700000000000003},"min":{"1min":0.48499999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.74199999999999999}]},{"osd":7,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.63900000000000001}]},{"osd":8,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.79600000000000004,"15min":0.80500000000000005},"min":{"1min":0.53100000000000003,"5min":0.503,"15min":0.503},"max":{"1min":0.90700000000000003,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.83099999999999996,"15min":0.84199999999999997},"min":{"1min":0.629,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.91700000000000004,"5min":1.165,"15min":1.165},"last":0.84199999999999997}]},{"osd":9,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.83399999999999996,"15min":0.83999999999999997},"min":{"1min":0.622,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.98399999999999999,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.85199999999999998,"15min":0.86599999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.92400000000000004}]},{"osd":11,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.81699999999999995,"15min":0.82099999999999995},"min":{"1min":0.56899999999999995,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.95599999999999996,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81999999999999995,"15min":0.82199999999999995},"min":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.026,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.879}]}]},{"osd":9,"up_from":67,"seq":287762808869,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":502050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.60399999999999998,"15min":0.62},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.92600000000000005,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.626},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58099999999999996,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74299999999999999,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.80200000000000005}]},{"osd":1,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.69399999999999995,"15min":0.70699999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97999999999999998,"5min":1.899,"15min":1.899},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.77700000000000002,"15min":0.79300000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.94399999999999995,"5min":1.409,"15min":1.409},"last":0.66700000000000004}]},{"osd":2,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.80000000000000004,"15min":0.81899999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.91800000000000004,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.873,"15min":0.89800000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.92900000000000005,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":0.73699999999999999}]},{"osd":3,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.72799999999999998,"15min":0.746},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.878,"5min":1.47,"15min":1.47},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.81699999999999995,"15min":0.85499999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84799999999999998,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.69899999999999995}]},{"osd":4,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.80000000000000004,"15min":0.82999999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.85099999999999998,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.88,"15min":0.90200000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.109,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.88}]},{"osd":5,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.92800000000000005,"15min":0.95599999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.089,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.84699999999999998,"15min":0.876},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.76000000000000001,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":0.58099999999999996}]},{"osd":6,"last update":"Thu Dec 7 10:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":0.749}]},{"osd":7,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.93400000000000005,"15min":0.95799999999999996},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.0820000000000001,"5min":1.5,"15min":1.5},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":1.016,"15min":1.052},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.1519999999999999,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":0.96899999999999997}]},{"osd":8,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.95599999999999996,"15min":0.97399999999999998},"min":{"1min":0.63800000000000001,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0600000000000001,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.85699999999999998,"5min":1.0409999999999999,"15min":1.071},"min":{"1min":0.626,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.194,"5min":2.6539999999999999,"15min":2.6539999999999999},"last":0.76700000000000002}]},{"osd":10,"last update":"Thu Dec 7 10:18:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":1.02,"15min":1.052},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.2889999999999999,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.93899999999999995,"15min":0.96899999999999997},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.0189999999999999,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.82899999999999996}]},{"osd":11,"last update":"Thu Dec 7 10:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.78400000000000003}]}]},{"osd":8,"up_from":59,"seq":253403070506,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":498220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55900000000000005,"15min":0.56299999999999994},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.76500000000000001}]},{"osd":1,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.59199999999999997,"15min":0.57699999999999996},"min":{"1min":0.45100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86299999999999999,"5min":1,"15min":1},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.60299999999999998,"15min":0.58299999999999996},"min":{"1min":0.52800000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.54300000000000004}]},{"osd":2,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.46200000000000002,"15min":0.44400000000000001},"min":{"1min":0.317,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.76700000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.46300000000000002,"15min":0.438},"min":{"1min":0.29699999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.44900000000000001}]},{"osd":3,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59799999999999998,"15min":0.58599999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.83199999999999996,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.74099999999999999}]},{"osd":4,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.498,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.86899999999999999,"5min":0.876,"15min":0.876},"last":0.496},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.64400000000000002,"15min":0.63400000000000001},"min":{"1min":0.54400000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89100000000000001,"5min":1.079,"15min":1.079},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.54100000000000004,"15min":0.51700000000000002},"min":{"1min":0.34799999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56799999999999995,"15min":0.55300000000000005},"min":{"1min":0.38200000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.84199999999999997,"5min":1.024,"15min":1.024},"last":0.58299999999999996}]},{"osd":6,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.67400000000000004,"15min":0.65600000000000003},"min":{"1min":0.499,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.96099999999999997,"5min":1.046,"15min":1.046},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.68999999999999995,"15min":0.66400000000000003},"min":{"1min":0.48699999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.78100000000000003}]},{"osd":7,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.68700000000000006,"15min":0.65500000000000003},"min":{"1min":0.57999999999999996,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.69199999999999995,"15min":0.67500000000000004},"min":{"1min":0.58699999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.95599999999999996,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.86799999999999999}]},{"osd":9,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.71999999999999997,"15min":0.71499999999999997},"min":{"1min":0.53900000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.74399999999999999,"15min":0.73399999999999999},"min":{"1min":0.59899999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.91500000000000004}]},{"osd":10,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.74399999999999999,"15min":0.72699999999999998},"min":{"1min":0.626,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.74299999999999999,"15min":0.72799999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.78900000000000003}]},{"osd":11,"last update":"Thu Dec 7 10:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.88800000000000001}]}]},{"osd":7,"up_from":52,"seq":223338299439,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1646592,"data_stored":1104877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.58199999999999996,"15min":0.55200000000000005},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.5369999999999999,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56599999999999995,"15min":0.56000000000000005},"min":{"1min":0.39700000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.36899999999999999}]},{"osd":1,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.434,"15min":0.432},"min":{"1min":0.29699999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60499999999999998,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45600000000000002},"min":{"1min":0.254,"5min":0.253,"15min":0.253},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.434,"5min":0.314,"15min":0.314},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.504},{"interface":"front","average":{"1min":0.626,"5min":0.57899999999999996,"15min":0.57099999999999995},"min":{"1min":0.39600000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.66400000000000003}]},{"osd":3,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73499999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.45500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.79900000000000004,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51900000000000002,"15min":0.51600000000000001},"min":{"1min":0.315,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.748,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.52400000000000002,"15min":0.53300000000000003},"min":{"1min":0.33600000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.77400000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.46200000000000002}]},{"osd":5,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63900000000000001,"15min":0.64400000000000002},"min":{"1min":0.45600000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.79200000000000004,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.443,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.84999999999999998,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.81299999999999994}]},{"osd":6,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85699999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.68600000000000005,"15min":0.69199999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94999999999999996,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.58899999999999997}]},{"osd":8,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70099999999999996,"15min":0.69599999999999995},"min":{"1min":0.46200000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90600000000000003,"5min":1.012,"15min":1.012},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72399999999999998,"15min":0.72799999999999998},"min":{"1min":0.52900000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83399999999999996,"5min":1.026,"15min":1.026},"last":0.64800000000000002}]},{"osd":9,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.73599999999999999,"15min":0.74199999999999999},"min":{"1min":0.54600000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88500000000000001,"5min":1.075,"15min":1.075},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74199999999999999,"15min":0.75},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.92300000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.78600000000000003}]},{"osd":10,"last update":"Thu Dec 7 10:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.76300000000000001,"15min":0.76600000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.96099999999999997,"5min":1.03,"15min":1.03},"last":0.63},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73999999999999999,"15min":0.74199999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.89200000000000002,"5min":1.002,"15min":1.002},"last":0.754}]},{"osd":11,"last update":"Thu Dec 7 10:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.70199999999999996}]}]},{"osd":6,"up_from":47,"seq":201863462963,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":493458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47399999999999998},"min":{"1min":0.34499999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.63400000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.374},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.48299999999999998,"15min":0.47199999999999998},"min":{"1min":0.38300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61499999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.65200000000000002,"15min":0.65800000000000003},"min":{"1min":0.442,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80900000000000005,"5min":1.462,"15min":1.462},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68000000000000005,"15min":0.66400000000000003},"min":{"1min":0.54600000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.73199999999999998}]},{"osd":2,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.52700000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.72999999999999998,"15min":0.752},"min":{"1min":0.51000000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.95899999999999996,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.77200000000000002}]},{"osd":3,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.63600000000000001,"15min":0.65900000000000003},"min":{"1min":0.37,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78400000000000003,"5min":1.605,"15min":1.605},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.83899999999999997,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.69599999999999995}]},{"osd":4,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73999999999999999,"15min":0.75},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.032,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.747,"15min":0.74299999999999999},"min":{"1min":0.54700000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.56799999999999995}]},{"osd":5,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79200000000000004,"15min":0.80100000000000005},"min":{"1min":0.51100000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0840000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77800000000000002,"15min":0.78500000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95399999999999996,"5min":1.75,"15min":1.75},"last":0.85899999999999999}]},{"osd":7,"last update":"Thu Dec 7 10:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.78000000000000003,"15min":0.78500000000000003},"min":{"1min":0.47399999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0489999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.80400000000000005,"15min":0.78800000000000003},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.97999999999999998,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.84199999999999997}]},{"osd":8,"last update":"Thu Dec 7 10:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.85999999999999999,"15min":0.86899999999999999},"min":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.569,"5min":2.569,"15min":2.569},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.88600000000000001,"15min":0.89400000000000002},"min":{"1min":0.53400000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":3.1850000000000001,"5min":3.1850000000000001,"15min":3.1850000000000001},"last":0.80900000000000005}]},{"osd":9,"last update":"Thu Dec 7 10:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.86499999999999999,"15min":0.871},"min":{"1min":0.66800000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1160000000000001,"5min":1.179,"15min":1.179},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.84499999999999997,"15min":0.84899999999999998},"min":{"1min":0.65700000000000003,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.0600000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.88300000000000001}]},{"osd":10,"last update":"Thu Dec 7 10:18:34 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.85999999999999999,"15min":0.84999999999999998},"min":{"1min":0.58299999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.3980000000000001,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.88600000000000001,"15min":0.88300000000000001},"min":{"1min":0.73899999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.89400000000000002}]},{"osd":11,"last update":"Thu Dec 7 10:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.91000000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":558941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.627,"15min":0.56499999999999995},"min":{"1min":0.39200000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.92500000000000004,"5min":6.8120000000000003,"15min":6.8120000000000003},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.57299999999999995,"15min":0.54600000000000004},"min":{"1min":0.433,"5min":0.376,"15min":0.376},"max":{"1min":0.82999999999999996,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.67900000000000005}]},{"osd":2,"last update":"Thu Dec 7 10:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.59499999999999997,"15min":0.54300000000000004},"min":{"1min":0.46899999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.87,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.55800000000000005,"15min":0.51000000000000001},"min":{"1min":0.42999999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.58999999999999997}]},{"osd":3,"last update":"Thu Dec 7 10:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60499999999999998,"15min":0.56299999999999994},"min":{"1min":0.39400000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.82399999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.37,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.67200000000000004,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Dec 7 10:18:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.437,"15min":0.39700000000000002},"min":{"1min":0.35599999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.47899999999999998,"15min":0.42099999999999999},"min":{"1min":0.36599999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.77100000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.40400000000000003}]},{"osd":5,"last update":"Thu Dec 7 10:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.65700000000000003,"15min":0.58799999999999997},"min":{"1min":0.45000000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.98099999999999998,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.60599999999999998,"15min":0.56200000000000006},"min":{"1min":0.51600000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82099999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.627}]},{"osd":6,"last update":"Thu Dec 7 10:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.69299999999999995,"15min":0.66700000000000004},"min":{"1min":0.49199999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93700000000000006,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.65400000000000003,"15min":0.64200000000000002},"min":{"1min":0.52900000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.85699999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.66100000000000003}]},{"osd":7,"last update":"Thu Dec 7 10:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.54900000000000004,"15min":0.52600000000000002},"min":{"1min":0.38100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.438},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.67300000000000004,"15min":0.63400000000000001},"min":{"1min":0.46300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93799999999999994,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.64000000000000001}]},{"osd":8,"last update":"Thu Dec 7 10:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.77100000000000002,"15min":0.73499999999999999},"min":{"1min":0.70999999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.026,"5min":1.175,"15min":1.175},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67100000000000004,"15min":0.64500000000000002},"min":{"1min":0.55500000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86699999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.60899999999999999}]},{"osd":9,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.76300000000000001,"15min":0.754},"min":{"1min":0.58299999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.85399999999999998,"15min":0.84899999999999998},"min":{"1min":0.65700000000000003,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2050000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.76700000000000002}]},{"osd":10,"last update":"Thu Dec 7 10:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.73999999999999999,"15min":0.73699999999999999},"min":{"1min":0.63200000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.75700000000000001,"15min":0.751},"min":{"1min":0.61099999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.747}]},{"osd":11,"last update":"Thu Dec 7 10:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.871},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.78600000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":494914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.59899999999999998,"15min":0.57199999999999995},"min":{"1min":0.443,"5min":0.375,"15min":0.375},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.41899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.68600000000000005}]},{"osd":2,"last update":"Thu Dec 7 10:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.63400000000000001,"15min":0.59599999999999997},"min":{"1min":0.44500000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.3420000000000001,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.64600000000000002,"15min":0.60499999999999998},"min":{"1min":0.42299999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":3.4140000000000001,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Dec 7 10:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":3.484,"5min":3.484,"15min":3.484},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":3.1379999999999999,"5min":3.1379999999999999,"15min":3.1379999999999999},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Dec 7 10:18:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.59999999999999998,"15min":0.56599999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91700000000000004,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.754},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.623,"15min":0.60499999999999998},"min":{"1min":0.46000000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82799999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Dec 7 10:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.63900000000000001,"15min":0.624},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91000000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.626,"15min":0.624},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85399999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.628}]},{"osd":6,"last update":"Thu Dec 7 10:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.627,"15min":0.625},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79000000000000004,"5min":1.02,"15min":1.02},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64100000000000001,"15min":0.63400000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.44700000000000001}]},{"osd":7,"last update":"Thu Dec 7 10:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":3.0430000000000001,"5min":3.0430000000000001,"15min":3.0430000000000001},"last":0.71899999999999997}]},{"osd":8,"last update":"Thu Dec 7 10:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.81499999999999995,"15min":0.80500000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":3.75,"5min":3.75,"15min":3.75},"last":0.622},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.80600000000000005,"15min":0.81100000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.84299999999999997}]},{"osd":9,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.84899999999999998,"15min":0.86199999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0609999999999999,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.82999999999999996,"15min":0.83999999999999997},"min":{"1min":0.501,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.67400000000000004}]},{"osd":10,"last update":"Thu Dec 7 10:18:35 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.79900000000000004,"15min":0.77900000000000003},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":3.0979999999999999,"5min":3.0979999999999999,"15min":3.0979999999999999},"last":1.1259999999999999},{"interface":"front","average":{"1min":0.88,"5min":0.85199999999999998,"15min":0.84699999999999998},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.383,"5min":2.383,"15min":2.383},"last":1.2669999999999999}]},{"osd":11,"last update":"Thu Dec 7 10:18:55 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.84599999999999997,"15min":0.81799999999999995},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.7879999999999998,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.82399999999999995,"15min":0.80200000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.1749999999999998,"5min":3.1749999999999998,"15min":3.1749999999999998},"last":1.079}]}]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":569431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.51000000000000001},"min":{"1min":0.41599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55100000000000005,"15min":0.52000000000000002},"min":{"1min":0.35899999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.64600000000000002}]},{"osd":1,"last update":"Thu Dec 7 10:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.58099999999999996,"15min":0.52800000000000002},"min":{"1min":0.42199999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.56200000000000006,"15min":0.52300000000000002},"min":{"1min":0.39800000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79700000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.70899999999999996}]},{"osd":3,"last update":"Thu Dec 7 10:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.58799999999999997,"15min":0.55300000000000005},"min":{"1min":0.45000000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.58299999999999996,"15min":0.55700000000000005},"min":{"1min":0.40999999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Dec 7 10:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.79300000000000004}]},{"osd":5,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.47899999999999998,"15min":0.443},"min":{"1min":0.39400000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.77200000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.502},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.501,"15min":0.47199999999999998},"min":{"1min":0.379,"5min":0.307,"15min":0.307},"max":{"1min":0.81299999999999994,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.47899999999999998}]},{"osd":6,"last update":"Thu Dec 7 10:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.68100000000000005,"15min":0.67000000000000004},"min":{"1min":0.501,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.91800000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.436},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.67200000000000004,"15min":0.65400000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.74199999999999999}]},{"osd":7,"last update":"Thu Dec 7 10:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.63100000000000001}]},{"osd":8,"last update":"Thu Dec 7 10:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.625,"15min":0.60399999999999998},"min":{"1min":0.53400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.68000000000000005,"15min":0.66400000000000003},"min":{"1min":0.60799999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0229999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.67800000000000005}]},{"osd":9,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.78800000000000003,"15min":0.78400000000000003},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0389999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.80700000000000005}]},{"osd":10,"last update":"Thu Dec 7 10:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.76600000000000001,"15min":0.752},"min":{"1min":0.57999999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.77800000000000002,"15min":0.76600000000000001},"min":{"1min":0.57099999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.84299999999999997}]},{"osd":11,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.82099999999999995}]}]},{"osd":3,"up_from":26,"seq":111669149761,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":492835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.435,"15min":0.39800000000000002},"min":{"1min":0.309,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.624},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.44900000000000001,"15min":0.40699999999999997},"min":{"1min":0.33700000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.77600000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.51500000000000001}]},{"osd":1,"last update":"Thu Dec 7 10:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.65000000000000002,"15min":0.57399999999999995},"min":{"1min":0.54700000000000004,"5min":0.438,"15min":0.438},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61299999999999999,"15min":0.55300000000000005},"min":{"1min":0.496,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84499999999999997,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.89000000000000001}]},{"osd":2,"last update":"Thu Dec 7 10:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59899999999999998,"15min":0.58999999999999997},"min":{"1min":0.42899999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.754,"5min":1.087,"15min":1.087},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.65800000000000003,"15min":0.60999999999999999},"min":{"1min":0.48499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.016,"5min":1.177,"15min":1.177},"last":0.91500000000000004}]},{"osd":4,"last update":"Thu Dec 7 10:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.68600000000000005,"15min":0.62},"min":{"1min":0.503,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.018,"5min":1.208,"15min":1.208},"last":1.032},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63200000000000001,"15min":0.58999999999999997},"min":{"1min":0.46999999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83899999999999997,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.65700000000000003}]},{"osd":5,"last update":"Thu Dec 7 10:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.623,"15min":0.61099999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.82099999999999995,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.71399999999999997,"15min":0.69599999999999995},"min":{"1min":0.60499999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96199999999999997,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.0760000000000001}]},{"osd":6,"last update":"Thu Dec 7 10:18:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.46999999999999997,"5min":0.377,"15min":0.377},"max":{"1min":1.0369999999999999,"5min":1.282,"15min":1.282},"last":1.179},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60999999999999999,"15min":0.61799999999999999},"min":{"1min":0.39700000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71199999999999997,"5min":1.292,"15min":1.292},"last":0.86599999999999999}]},{"osd":7,"last update":"Thu Dec 7 10:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78600000000000003,"15min":0.76900000000000002},"min":{"1min":0.59599999999999997,"5min":0.498,"15min":0.498},"max":{"1min":1.0780000000000001,"5min":1.323,"15min":1.323},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.56799999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96999999999999997,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.93799999999999994}]},{"osd":8,"last update":"Thu Dec 7 10:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.872}]},{"osd":9,"last update":"Thu Dec 7 10:18:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.88,"15min":0.88600000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.224,"5min":1.242,"15min":1.242},"last":1.1479999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.80100000000000005,"15min":0.81000000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":1.0009999999999999}]},{"osd":10,"last update":"Thu Dec 7 10:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.78200000000000003,"15min":0.79100000000000004},"min":{"1min":0.59099999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.90500000000000003,"5min":1.335,"15min":1.335},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.878,"5min":0.92100000000000004,"15min":0.92800000000000005},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.1180000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":1.2190000000000001}]},{"osd":11,"last update":"Thu Dec 7 10:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":1.2509999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":1.056}]}]},{"osd":4,"up_from":34,"seq":146028888124,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":498032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.52100000000000002,"15min":0.49099999999999999},"min":{"1min":0.41499999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76500000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Dec 7 10:18:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.41099999999999998,"15min":0.39000000000000001},"min":{"1min":0.254,"5min":0.215,"15min":0.215},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.41499999999999998,"15min":0.38900000000000001},"min":{"1min":0.308,"5min":0.22,"15min":0.22},"max":{"1min":0.55200000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.39200000000000002}]},{"osd":2,"last update":"Thu Dec 7 10:18:17 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58899999999999997,"15min":0.56000000000000005},"min":{"1min":0.42699999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.875,"5min":1.323,"15min":1.323},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58499999999999996,"15min":0.54300000000000004},"min":{"1min":0.45700000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84299999999999997,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.66800000000000004}]},{"osd":3,"last update":"Thu Dec 7 10:18:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58099999999999996,"15min":0.54300000000000004},"min":{"1min":0.437,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.874,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.60899999999999999,"15min":0.56999999999999995},"min":{"1min":0.45700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91200000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.61499999999999999}]},{"osd":5,"last update":"Thu Dec 7 10:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.63100000000000001,"15min":0.59299999999999997},"min":{"1min":0.55000000000000004,"5min":0.374,"15min":0.374},"max":{"1min":1.0009999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.61699999999999999,"15min":0.58399999999999996},"min":{"1min":0.47599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82699999999999996,"5min":1.258,"15min":1.258},"last":0.502}]},{"osd":6,"last update":"Thu Dec 7 10:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.41399999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.434},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64800000000000002,"15min":0.63600000000000001},"min":{"1min":0.501,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91900000000000004,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.54700000000000004}]},{"osd":7,"last update":"Thu Dec 7 10:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.55000000000000004,"15min":0.51800000000000002},"min":{"1min":0.38100000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76800000000000002,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.35599999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.84399999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.56699999999999995}]},{"osd":8,"last update":"Thu Dec 7 10:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.71299999999999997,"15min":0.67900000000000005},"min":{"1min":0.57099999999999995,"5min":0.432,"15min":0.432},"max":{"1min":0.90000000000000002,"5min":1.381,"15min":1.381},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71199999999999997,"15min":0.68200000000000005},"min":{"1min":0.59099999999999997,"5min":0.502,"15min":0.502},"max":{"1min":0.872,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.749}]},{"osd":9,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.76700000000000002,"15min":0.76900000000000002},"min":{"1min":0.57399999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.98799999999999999,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.77100000000000002,"15min":0.75800000000000001},"min":{"1min":0.58699999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.76100000000000001}]},{"osd":10,"last update":"Thu Dec 7 10:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.60999999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.54600000000000004,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.73399999999999999}]},{"osd":11,"last update":"Thu Dec 7 10:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.873},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.78100000000000003}]}]},{"osd":5,"up_from":39,"seq":167503724600,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1658880,"data_stored":1100402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.54700000000000004,"15min":0.51700000000000002},"min":{"1min":0.377,"5min":0.318,"15min":0.318},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54700000000000004,"15min":0.52200000000000002},"min":{"1min":0.37,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79000000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54600000000000004}]},{"osd":1,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.56499999999999995,"15min":0.51700000000000002},"min":{"1min":0.505,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.59799999999999998,"15min":0.54400000000000004},"min":{"1min":0.52200000000000002,"5min":0.312,"15min":0.312},"max":{"1min":3.0070000000000001,"5min":3.0070000000000001,"15min":3.0070000000000001},"last":0.64200000000000002}]},{"osd":2,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.42399999999999999,"15min":0.38600000000000001},"min":{"1min":0.40200000000000002,"5min":0.246,"15min":0.246},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.45300000000000001,"15min":0.39900000000000002},"min":{"1min":0.35699999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.851,"5min":1.851,"15min":1.851},"last":0.41899999999999998}]},{"osd":3,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.57999999999999996,"15min":0.53900000000000003},"min":{"1min":0.43099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.59099999999999997,"15min":0.55600000000000005},"min":{"1min":0.40500000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.56899999999999995}]},{"osd":4,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.63100000000000001,"15min":0.57299999999999995},"min":{"1min":0.58199999999999996,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.621,"15min":0.57899999999999996},"min":{"1min":0.47699999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.61299999999999999}]},{"osd":6,"last update":"Thu Dec 7 10:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.61799999999999999,"15min":0.58599999999999997},"min":{"1min":0.42999999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.69499999999999995,"15min":0.63200000000000001},"min":{"1min":0.45000000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":5.9720000000000004,"5min":5.9720000000000004,"15min":5.9720000000000004},"last":0.78400000000000003}]},{"osd":7,"last update":"Thu Dec 7 10:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.66100000000000003,"15min":0.626},"min":{"1min":0.38400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.66600000000000004,"15min":0.63800000000000001},"min":{"1min":0.443,"5min":0.437,"15min":0.437},"max":{"1min":0.96999999999999997,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.628}]},{"osd":8,"last update":"Thu Dec 7 10:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.65000000000000002,"15min":0.628},"min":{"1min":0.45600000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.052,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.751},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.61899999999999999,"15min":0.59899999999999998},"min":{"1min":0.5,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92000000000000004,"5min":1.135,"15min":1.135},"last":0.68300000000000005}]},{"osd":9,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.78700000000000003,"15min":0.755},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":4.532,"5min":4.532,"15min":4.532},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76400000000000001,"15min":0.76300000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.83299999999999996}]},{"osd":10,"last update":"Thu Dec 7 10:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.84799999999999998,"15min":0.84099999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.7649999999999999,"5min":2.927,"15min":2.927},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.1519999999999999,"5min":0.85899999999999999,"15min":0.81000000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":5.7709999999999999,"5min":5.7709999999999999,"15min":5.7709999999999999},"last":0.76500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":76695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":13202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":13912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-07T10:19:01.409 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph pg dump --format=json 2023-12-07T10:19:02.190 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:19:02.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:01 smithi119 ceph-mon[97908]: pgmap v321: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-07T10:19:02.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]: dispatch 2023-12-07T10:19:02.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2023-12-07T10:19:02.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2023-12-07T10:19:02.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:01 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]: dispatch 2023-12-07T10:19:02.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:01 smithi119 ceph-mon[97908]: from='client.25028 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:19:02.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:01 smithi159 ceph-mon[134726]: pgmap v321: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-07T10:19:02.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]: dispatch 2023-12-07T10:19:02.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2023-12-07T10:19:02.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2023-12-07T10:19:02.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:01 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]: dispatch 2023-12-07T10:19:02.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:01 smithi159 ceph-mon[134726]: from='client.25028 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:19:02.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:01 smithi153 ceph-mon[133625]: pgmap v321: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-07T10:19:02.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]: dispatch 2023-12-07T10:19:02.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2023-12-07T10:19:02.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2023-12-07T10:19:02.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:01 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]: dispatch 2023-12-07T10:19:02.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:01 smithi153 ceph-mon[133625]: from='client.25028 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:19:02.971 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:02 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]': finished 2023-12-07T10:19:02.972 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:02 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2023-12-07T10:19:02.972 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:02 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2023-12-07T10:19:02.972 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:02 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]': finished 2023-12-07T10:19:02.972 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:02 smithi119 ceph-mon[97908]: osdmap e92: 12 total, 12 up, 12 in 2023-12-07T10:19:03.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:02 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]': finished 2023-12-07T10:19:03.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:02 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2023-12-07T10:19:03.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:02 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2023-12-07T10:19:03.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:02 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]': finished 2023-12-07T10:19:03.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:02 smithi159 ceph-mon[134726]: osdmap e92: 12 total, 12 up, 12 in 2023-12-07T10:19:03.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:02 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]': finished 2023-12-07T10:19:03.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:02 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2023-12-07T10:19:03.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:02 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2023-12-07T10:19:03.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:02 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]': finished 2023-12-07T10:19:03.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:02 smithi153 ceph-mon[133625]: osdmap e92: 12 total, 12 up, 12 in 2023-12-07T10:19:03.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.624+0000 7fafdee71700 1 -- 172.21.15.119:0/3263607845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafd81016c0 msgr2=0x7fafd8101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:03.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.624+0000 7fafdee71700 1 --2- 172.21.15.119:0/3263607845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafd81016c0 0x7fafd8101b40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fafc4009990 tx=0x7fafc4009ca0 comp rx=0 tx=0).stop 2023-12-07T10:19:03.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.624+0000 7fafdee71700 1 -- 172.21.15.119:0/3263607845 shutdown_connections 2023-12-07T10:19:03.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.624+0000 7fafdee71700 1 --2- 172.21.15.119:0/3263607845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafd8102080 0x7fafd810e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:03.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.624+0000 7fafdee71700 1 --2- 172.21.15.119:0/3263607845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafd8107730 0x7fafd8107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:03.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.624+0000 7fafdee71700 1 --2- 172.21.15.119:0/3263607845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafd81016c0 0x7fafd8101b40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:03.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.624+0000 7fafdee71700 1 -- 172.21.15.119:0/3263607845 >> 172.21.15.119:0/3263607845 conn(0x7fafd8078800 msgr2=0x7fafd8078c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:03.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.624+0000 7fafdee71700 1 -- 172.21.15.119:0/3263607845 shutdown_connections 2023-12-07T10:19:03.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.624+0000 7fafdee71700 1 -- 172.21.15.119:0/3263607845 wait complete. 2023-12-07T10:19:03.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.625+0000 7fafdee71700 1 Processor -- start 2023-12-07T10:19:03.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.626+0000 7fafdee71700 1 -- start start 2023-12-07T10:19:03.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.626+0000 7fafdee71700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafd81016c0 0x7fafd8073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:03.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.626+0000 7fafdee71700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafd8102080 0x7fafd8073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:03.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.626+0000 7fafdee71700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafd8107730 0x7fafd8074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:03.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.626+0000 7fafdee71700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd81a5dd0 con 0x7fafd8107730 2023-12-07T10:19:03.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.626+0000 7fafdee71700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd81a5f40 con 0x7fafd8102080 2023-12-07T10:19:03.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.626+0000 7fafdee71700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd81a6100 con 0x7fafd81016c0 2023-12-07T10:19:03.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.626+0000 7fafdd40e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafd8107730 0x7fafd8074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:03.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.626+0000 7fafdd40e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafd8107730 0x7fafd8074390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49010/0 (socket says 172.21.15.119:49010) 2023-12-07T10:19:03.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.626+0000 7fafdd40e700 1 -- 172.21.15.119:0/2040594392 learned_addr learned my addr 172.21.15.119:0/2040594392 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:03.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.626+0000 7fafdcc0d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafd81016c0 0x7fafd8073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:03.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.626+0000 7fafd7fff700 1 --2- 172.21.15.119:0/2040594392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafd8102080 0x7fafd8073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:03.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.627+0000 7fafdd40e700 1 -- 172.21.15.119:0/2040594392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafd81016c0 msgr2=0x7fafd8073910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:03.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.627+0000 7fafdd40e700 1 --2- 172.21.15.119:0/2040594392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafd81016c0 0x7fafd8073910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:03.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.627+0000 7fafdd40e700 1 -- 172.21.15.119:0/2040594392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafd8102080 msgr2=0x7fafd8073e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:03.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.627+0000 7fafdd40e700 1 --2- 172.21.15.119:0/2040594392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafd8102080 0x7fafd8073e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:03.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.627+0000 7fafdd40e700 1 -- 172.21.15.119:0/2040594392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc40095f0 con 0x7fafd8107730 2023-12-07T10:19:03.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.627+0000 7fafdd40e700 1 --2- 172.21.15.119:0/2040594392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafd8107730 0x7fafd8074390 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fafd000efc0 tx=0x7fafd000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:03.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.627+0000 7fafd5ffb700 1 -- 172.21.15.119:0/2040594392 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd0018070 con 0x7fafd8107730 2023-12-07T10:19:03.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.627+0000 7fafd5ffb700 1 -- 172.21.15.119:0/2040594392 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafd000f040 con 0x7fafd8107730 2023-12-07T10:19:03.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.627+0000 7fafdee71700 1 -- 172.21.15.119:0/2040594392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafd81a6380 con 0x7fafd8107730 2023-12-07T10:19:03.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.627+0000 7fafd5ffb700 1 -- 172.21.15.119:0/2040594392 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd000cc10 con 0x7fafd8107730 2023-12-07T10:19:03.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.627+0000 7fafdee71700 1 -- 172.21.15.119:0/2040594392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd81a6830 con 0x7fafd8107730 2023-12-07T10:19:03.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.628+0000 7fafdee71700 1 -- 172.21.15.119:0/2040594392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd8109bc0 con 0x7fafd8107730 2023-12-07T10:19:03.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.630+0000 7fafd5ffb700 1 -- 172.21.15.119:0/2040594392 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7fafd0003a00 con 0x7fafd8107730 2023-12-07T10:19:03.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.631+0000 7fafd5ffb700 1 --2- 172.21.15.119:0/2040594392 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fafc80a1ba0 0x7fafc80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:03.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.632+0000 7fafd5ffb700 1 -- 172.21.15.119:0/2040594392 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fafd00cd520 con 0x7fafd8107730 2023-12-07T10:19:03.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.632+0000 7fafdcc0d700 1 --2- 172.21.15.119:0/2040594392 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fafc80a1ba0 0x7fafc80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:03.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.632+0000 7fafdcc0d700 1 --2- 172.21.15.119:0/2040594392 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fafc80a1ba0 0x7fafc80a4060 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009be0 tx=0x7fafcc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:03.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.636+0000 7fafd5ffb700 1 -- 172.21.15.119:0/2040594392 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fafd00964a0 con 0x7fafd8107730 2023-12-07T10:19:03.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.834+0000 7fafdee71700 1 -- 172.21.15.119:0/2040594392 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fafd8074e40 con 0x7fafc80a1ba0 2023-12-07T10:19:03.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.847+0000 7fafd5ffb700 1 -- 172.21.15.119:0/2040594392 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426511 (secure 0 0 0) 0x7fafd8074e40 con 0x7fafc80a1ba0 2023-12-07T10:19:03.855 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:19:03.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.853+0000 7fafdee71700 1 -- 172.21.15.119:0/2040594392 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fafc80a1ba0 msgr2=0x7fafc80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:03.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.853+0000 7fafdee71700 1 --2- 172.21.15.119:0/2040594392 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fafc80a1ba0 0x7fafc80a4060 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009be0 tx=0x7fafcc009260 comp rx=0 tx=0).stop 2023-12-07T10:19:03.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.853+0000 7fafdee71700 1 -- 172.21.15.119:0/2040594392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafd8107730 msgr2=0x7fafd8074390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:03.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.853+0000 7fafdee71700 1 --2- 172.21.15.119:0/2040594392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafd8107730 0x7fafd8074390 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fafd000efc0 tx=0x7fafd000c3a0 comp rx=0 tx=0).stop 2023-12-07T10:19:03.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.853+0000 7fafdee71700 1 -- 172.21.15.119:0/2040594392 shutdown_connections 2023-12-07T10:19:03.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.853+0000 7fafdee71700 1 --2- 172.21.15.119:0/2040594392 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fafc80a1ba0 0x7fafc80a4060 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:03.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.853+0000 7fafdee71700 1 --2- 172.21.15.119:0/2040594392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafd8107730 0x7fafd8074390 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:03.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.853+0000 7fafdee71700 1 --2- 172.21.15.119:0/2040594392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafd81016c0 0x7fafd8073910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:03.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.853+0000 7fafdee71700 1 --2- 172.21.15.119:0/2040594392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafd8102080 0x7fafd8073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:03.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.853+0000 7fafdee71700 1 -- 172.21.15.119:0/2040594392 >> 172.21.15.119:0/2040594392 conn(0x7fafd8078800 msgr2=0x7fafd81002e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:03.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.853+0000 7fafdee71700 1 -- 172.21.15.119:0/2040594392 shutdown_connections 2023-12-07T10:19:03.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:03.853+0000 7fafdee71700 1 -- 172.21.15.119:0/2040594392 wait complete. 2023-12-07T10:19:03.859 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-12-07T10:19:04.070 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:03 smithi119 ceph-mon[97908]: pgmap v323: 129 pgs: 4 peering, 125 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-12-07T10:19:04.070 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:03 smithi119 ceph-mon[97908]: Health check failed: Reduced data availability: 2 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2023-12-07T10:19:04.070 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:03 smithi119 ceph-mon[97908]: osdmap e93: 12 total, 12 up, 12 in 2023-12-07T10:19:04.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:03 smithi159 ceph-mon[134726]: pgmap v323: 129 pgs: 4 peering, 125 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-12-07T10:19:04.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:03 smithi159 ceph-mon[134726]: Health check failed: Reduced data availability: 2 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2023-12-07T10:19:04.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:03 smithi159 ceph-mon[134726]: osdmap e93: 12 total, 12 up, 12 in 2023-12-07T10:19:04.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:03 smithi153 ceph-mon[133625]: pgmap v323: 129 pgs: 4 peering, 125 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-12-07T10:19:04.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:03 smithi153 ceph-mon[133625]: Health check failed: Reduced data availability: 2 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2023-12-07T10:19:04.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:03 smithi153 ceph-mon[133625]: osdmap e93: 12 total, 12 up, 12 in 2023-12-07T10:19:04.390 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2023-12-07T10:19:02.806850+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":672259,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":653,"num_read_kb":465,"num_write":426,"num_write_kb":3092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338368,"kb_used_data":14172,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603456,"statfs":{"total":1151940427776,"available":1151593938944,"internally_reserved":0,"allocated":14512128,"data_stored":7970077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":331855161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.118784"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.856154+0000","last_change":"2023-12-07T10:17:41.614206+0000","last_active":"2023-12-07T10:18:01.856154+0000","last_peered":"2023-12-07T10:18:01.856154+0000","last_clean":"2023-12-07T10:18:01.856154+0000","last_became_active":"2023-12-07T10:17:41.613553+0000","last_became_peered":"2023-12-07T10:17:41.613553+0000","last_unstale":"2023-12-07T10:18:01.856154+0000","last_undegraded":"2023-12-07T10:18:01.856154+0000","last_fullsized":"2023-12-07T10:18:01.856154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:10:22.198414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852964+0000","last_change":"2023-12-07T10:17:41.616917+0000","last_active":"2023-12-07T10:18:00.852964+0000","last_peered":"2023-12-07T10:18:00.852964+0000","last_clean":"2023-12-07T10:18:00.852964+0000","last_became_active":"2023-12-07T10:17:41.615345+0000","last_became_peered":"2023-12-07T10:17:41.615345+0000","last_unstale":"2023-12-07T10:18:00.852964+0000","last_undegraded":"2023-12-07T10:18:00.852964+0000","last_fullsized":"2023-12-07T10:18:00.852964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:22:25.566738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.853214+0000","last_change":"2023-12-07T10:17:41.619767+0000","last_active":"2023-12-07T10:18:00.853214+0000","last_peered":"2023-12-07T10:18:00.853214+0000","last_clean":"2023-12-07T10:18:00.853214+0000","last_became_active":"2023-12-07T10:17:41.619362+0000","last_became_peered":"2023-12-07T10:17:41.619362+0000","last_unstale":"2023-12-07T10:18:00.853214+0000","last_undegraded":"2023-12-07T10:18:00.853214+0000","last_fullsized":"2023-12-07T10:18:00.853214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:12:55.458520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.862478+0000","last_change":"2023-12-07T10:18:01.861930+0000","last_active":"2023-12-07T10:18:01.862478+0000","last_peered":"2023-12-07T10:18:01.862478+0000","last_clean":"2023-12-07T10:18:01.862478+0000","last_became_active":"2023-12-07T10:18:01.861570+0000","last_became_peered":"2023-12-07T10:18:01.861570+0000","last_unstale":"2023-12-07T10:18:01.862478+0000","last_undegraded":"2023-12-07T10:18:01.862478+0000","last_fullsized":"2023-12-07T10:18:01.862478+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:29:45.234261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.858385+0000","last_change":"2023-12-07T10:17:41.613890+0000","last_active":"2023-12-07T10:18:01.858385+0000","last_peered":"2023-12-07T10:18:01.858385+0000","last_clean":"2023-12-07T10:18:01.858385+0000","last_became_active":"2023-12-07T10:17:41.613073+0000","last_became_peered":"2023-12-07T10:17:41.613073+0000","last_unstale":"2023-12-07T10:18:01.858385+0000","last_undegraded":"2023-12-07T10:18:01.858385+0000","last_fullsized":"2023-12-07T10:18:01.858385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:17:14.339002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.854664+0000","last_change":"2023-12-07T10:17:41.609585+0000","last_active":"2023-12-07T10:18:00.854664+0000","last_peered":"2023-12-07T10:18:00.854664+0000","last_clean":"2023-12-07T10:18:00.854664+0000","last_became_active":"2023-12-07T10:17:41.609256+0000","last_became_peered":"2023-12-07T10:17:41.609256+0000","last_unstale":"2023-12-07T10:18:00.854664+0000","last_undegraded":"2023-12-07T10:18:00.854664+0000","last_fullsized":"2023-12-07T10:18:00.854664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:20:15.103208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856396+0000","last_change":"2023-12-07T10:17:41.606138+0000","last_active":"2023-12-07T10:18:00.856396+0000","last_peered":"2023-12-07T10:18:00.856396+0000","last_clean":"2023-12-07T10:18:00.856396+0000","last_became_active":"2023-12-07T10:17:41.605818+0000","last_became_peered":"2023-12-07T10:17:41.605818+0000","last_unstale":"2023-12-07T10:18:00.856396+0000","last_undegraded":"2023-12-07T10:18:00.856396+0000","last_fullsized":"2023-12-07T10:18:00.856396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:38:39.244077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.257383+0000","last_change":"2023-12-07T10:18:02.255959+0000","last_active":"2023-12-07T10:18:02.257383+0000","last_peered":"2023-12-07T10:18:02.257383+0000","last_clean":"2023-12-07T10:18:02.257383+0000","last_became_active":"2023-12-07T10:18:02.255560+0000","last_became_peered":"2023-12-07T10:18:02.255560+0000","last_unstale":"2023-12-07T10:18:02.257383+0000","last_undegraded":"2023-12-07T10:18:02.257383+0000","last_fullsized":"2023-12-07T10:18:02.257383+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:05:49.065538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.989614+0000","last_change":"2023-12-07T10:17:41.610225+0000","last_active":"2023-12-07T10:18:01.989614+0000","last_peered":"2023-12-07T10:18:01.989614+0000","last_clean":"2023-12-07T10:18:01.989614+0000","last_became_active":"2023-12-07T10:17:41.609875+0000","last_became_peered":"2023-12-07T10:17:41.609875+0000","last_unstale":"2023-12-07T10:18:01.989614+0000","last_undegraded":"2023-12-07T10:18:01.989614+0000","last_fullsized":"2023-12-07T10:18:01.989614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:00:43.136176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855342+0000","last_change":"2023-12-07T10:17:41.617335+0000","last_active":"2023-12-07T10:18:00.855342+0000","last_peered":"2023-12-07T10:18:00.855342+0000","last_clean":"2023-12-07T10:18:00.855342+0000","last_became_active":"2023-12-07T10:17:41.617005+0000","last_became_peered":"2023-12-07T10:17:41.617005+0000","last_unstale":"2023-12-07T10:18:00.855342+0000","last_undegraded":"2023-12-07T10:18:00.855342+0000","last_fullsized":"2023-12-07T10:18:00.855342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:43:36.855465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.857002+0000","last_change":"2023-12-07T10:17:41.608104+0000","last_active":"2023-12-07T10:18:00.857002+0000","last_peered":"2023-12-07T10:18:00.857002+0000","last_clean":"2023-12-07T10:18:00.857002+0000","last_became_active":"2023-12-07T10:17:41.607769+0000","last_became_peered":"2023-12-07T10:17:41.607769+0000","last_unstale":"2023-12-07T10:18:00.857002+0000","last_undegraded":"2023-12-07T10:18:00.857002+0000","last_fullsized":"2023-12-07T10:18:00.857002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:47:04.853044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.254656+0000","last_change":"2023-12-07T10:17:41.611585+0000","last_active":"2023-12-07T10:18:02.254656+0000","last_peered":"2023-12-07T10:18:02.254656+0000","last_clean":"2023-12-07T10:18:02.254656+0000","last_became_active":"2023-12-07T10:17:41.611202+0000","last_became_peered":"2023-12-07T10:17:41.611202+0000","last_unstale":"2023-12-07T10:18:02.254656+0000","last_undegraded":"2023-12-07T10:18:02.254656+0000","last_fullsized":"2023-12-07T10:18:02.254656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:17:01.649424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855084+0000","last_change":"2023-12-07T10:17:41.611116+0000","last_active":"2023-12-07T10:18:00.855084+0000","last_peered":"2023-12-07T10:18:00.855084+0000","last_clean":"2023-12-07T10:18:00.855084+0000","last_became_active":"2023-12-07T10:17:41.610791+0000","last_became_peered":"2023-12-07T10:17:41.610791+0000","last_unstale":"2023-12-07T10:18:00.855084+0000","last_undegraded":"2023-12-07T10:18:00.855084+0000","last_fullsized":"2023-12-07T10:18:00.855084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:48:08.822235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852233+0000","last_change":"2023-12-07T10:17:41.617447+0000","last_active":"2023-12-07T10:18:00.852233+0000","last_peered":"2023-12-07T10:18:00.852233+0000","last_clean":"2023-12-07T10:18:00.852233+0000","last_became_active":"2023-12-07T10:17:41.612638+0000","last_became_peered":"2023-12-07T10:17:41.612638+0000","last_unstale":"2023-12-07T10:18:00.852233+0000","last_undegraded":"2023-12-07T10:18:00.852233+0000","last_fullsized":"2023-12-07T10:18:00.852233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:11:30.384367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852009+0000","last_change":"2023-12-07T10:17:41.611607+0000","last_active":"2023-12-07T10:18:00.852009+0000","last_peered":"2023-12-07T10:18:00.852009+0000","last_clean":"2023-12-07T10:18:00.852009+0000","last_became_active":"2023-12-07T10:17:41.611331+0000","last_became_peered":"2023-12-07T10:17:41.611331+0000","last_unstale":"2023-12-07T10:18:00.852009+0000","last_undegraded":"2023-12-07T10:18:00.852009+0000","last_fullsized":"2023-12-07T10:18:00.852009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:17:53.049922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.988971+0000","last_change":"2023-12-07T10:17:41.610528+0000","last_active":"2023-12-07T10:18:01.988971+0000","last_peered":"2023-12-07T10:18:01.988971+0000","last_clean":"2023-12-07T10:18:01.988971+0000","last_became_active":"2023-12-07T10:17:41.610216+0000","last_became_peered":"2023-12-07T10:17:41.610216+0000","last_unstale":"2023-12-07T10:18:01.988971+0000","last_undegraded":"2023-12-07T10:18:01.988971+0000","last_fullsized":"2023-12-07T10:18:01.988971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:11:36.749537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.856068+0000","last_change":"2023-12-07T10:17:41.613967+0000","last_active":"2023-12-07T10:18:01.856068+0000","last_peered":"2023-12-07T10:18:01.856068+0000","last_clean":"2023-12-07T10:18:01.856068+0000","last_became_active":"2023-12-07T10:17:41.613116+0000","last_became_peered":"2023-12-07T10:17:41.613116+0000","last_unstale":"2023-12-07T10:18:01.856068+0000","last_undegraded":"2023-12-07T10:18:01.856068+0000","last_fullsized":"2023-12-07T10:18:01.856068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:46:32.444025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.846417+0000","last_change":"2023-12-07T10:17:41.605478+0000","last_active":"2023-12-07T10:18:00.846417+0000","last_peered":"2023-12-07T10:18:00.846417+0000","last_clean":"2023-12-07T10:18:00.846417+0000","last_became_active":"2023-12-07T10:17:41.605110+0000","last_became_peered":"2023-12-07T10:17:41.605110+0000","last_unstale":"2023-12-07T10:18:00.846417+0000","last_undegraded":"2023-12-07T10:18:00.846417+0000","last_fullsized":"2023-12-07T10:18:00.846417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:53:56.410196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.256377+0000","last_change":"2023-12-07T10:18:02.256377+0000","last_active":"2023-12-07T10:18:02.256377+0000","last_peered":"2023-12-07T10:18:02.256377+0000","last_clean":"2023-12-07T10:18:02.256377+0000","last_became_active":"2023-12-07T10:18:02.255987+0000","last_became_peered":"2023-12-07T10:18:02.255987+0000","last_unstale":"2023-12-07T10:18:02.256377+0000","last_undegraded":"2023-12-07T10:18:02.256377+0000","last_fullsized":"2023-12-07T10:18:02.256377+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:23:34.909187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849924+0000","last_change":"2023-12-07T10:17:45.799925+0000","last_active":"2023-12-07T10:18:00.849924+0000","last_peered":"2023-12-07T10:18:00.849924+0000","last_clean":"2023-12-07T10:18:00.849924+0000","last_became_active":"2023-12-07T10:17:45.799505+0000","last_became_peered":"2023-12-07T10:17:45.799505+0000","last_unstale":"2023-12-07T10:18:00.849924+0000","last_undegraded":"2023-12-07T10:18:00.849924+0000","last_fullsized":"2023-12-07T10:18:00.849924+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:52:51.470333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'13","reported_seq":79,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:57.831212+0000","last_change":"2023-12-07T10:17:40.755066+0000","last_active":"2023-12-07T10:18:57.831212+0000","last_peered":"2023-12-07T10:18:57.831212+0000","last_clean":"2023-12-07T10:18:57.831212+0000","last_became_active":"2023-12-07T10:17:40.754709+0000","last_became_peered":"2023-12-07T10:17:40.754709+0000","last_unstale":"2023-12-07T10:18:57.831212+0000","last_undegraded":"2023-12-07T10:18:57.831212+0000","last_fullsized":"2023-12-07T10:18:57.831212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:18:31.148592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":47,"num_read_kb":47,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849407+0000","last_change":"2023-12-07T10:17:41.619107+0000","last_active":"2023-12-07T10:18:00.849407+0000","last_peered":"2023-12-07T10:18:00.849407+0000","last_clean":"2023-12-07T10:18:00.849407+0000","last_became_active":"2023-12-07T10:17:41.617057+0000","last_became_peered":"2023-12-07T10:17:41.617057+0000","last_unstale":"2023-12-07T10:18:00.849407+0000","last_undegraded":"2023-12-07T10:18:00.849407+0000","last_fullsized":"2023-12-07T10:18:00.849407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:53:16.402933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.255056+0000","last_change":"2023-12-07T10:17:45.797582+0000","last_active":"2023-12-07T10:18:02.255056+0000","last_peered":"2023-12-07T10:18:02.255056+0000","last_clean":"2023-12-07T10:18:02.255056+0000","last_became_active":"2023-12-07T10:17:45.796358+0000","last_became_peered":"2023-12-07T10:17:45.796358+0000","last_unstale":"2023-12-07T10:18:02.255056+0000","last_undegraded":"2023-12-07T10:18:02.255056+0000","last_fullsized":"2023-12-07T10:18:02.255056+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:53:19.551459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.857161+0000","last_change":"2023-12-07T10:17:40.762155+0000","last_active":"2023-12-07T10:18:01.857161+0000","last_peered":"2023-12-07T10:18:01.857161+0000","last_clean":"2023-12-07T10:18:01.857161+0000","last_became_active":"2023-12-07T10:17:40.761776+0000","last_became_peered":"2023-12-07T10:17:40.761776+0000","last_unstale":"2023-12-07T10:18:01.857161+0000","last_undegraded":"2023-12-07T10:18:01.857161+0000","last_fullsized":"2023-12-07T10:18:01.857161+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:15:06.096331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.853107+0000","last_change":"2023-12-07T10:17:41.614999+0000","last_active":"2023-12-07T10:18:00.853107+0000","last_peered":"2023-12-07T10:18:00.853107+0000","last_clean":"2023-12-07T10:18:00.853107+0000","last_became_active":"2023-12-07T10:17:41.614673+0000","last_became_peered":"2023-12-07T10:17:41.614673+0000","last_unstale":"2023-12-07T10:18:00.853107+0000","last_undegraded":"2023-12-07T10:18:00.853107+0000","last_fullsized":"2023-12-07T10:18:00.853107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:34:45.378843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"peering","last_fresh":"2023-12-07T10:18:02.254326+0000","last_change":"2023-12-07T10:19:01.805042+0000","last_active":"2023-12-07T10:19:01.805042+0000","last_peered":"2023-12-07T10:18:02.254326+0000","last_clean":"2023-12-07T10:18:02.254326+0000","last_became_active":"2023-12-07T10:17:45.799072+0000","last_became_peered":"2023-12-07T10:17:45.799072+0000","last_unstale":"2023-12-07T10:18:02.254326+0000","last_undegraded":"2023-12-07T10:18:02.254326+0000","last_fullsized":"2023-12-07T10:18:02.254326+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5,9,10],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:15.647183+0000","last_change":"2023-12-07T10:18:01.993128+0000","last_active":"2023-12-07T10:18:15.647183+0000","last_peered":"2023-12-07T10:18:15.647183+0000","last_clean":"2023-12-07T10:18:15.647183+0000","last_became_active":"2023-12-07T10:18:01.990244+0000","last_became_peered":"2023-12-07T10:18:01.990244+0000","last_unstale":"2023-12-07T10:18:15.647183+0000","last_undegraded":"2023-12-07T10:18:15.647183+0000","last_fullsized":"2023-12-07T10:18:15.647183+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:42:53.778363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.989335+0000","last_change":"2023-12-07T10:17:41.612903+0000","last_active":"2023-12-07T10:18:01.989335+0000","last_peered":"2023-12-07T10:18:01.989335+0000","last_clean":"2023-12-07T10:18:01.989335+0000","last_became_active":"2023-12-07T10:17:41.611968+0000","last_became_peered":"2023-12-07T10:17:41.611968+0000","last_unstale":"2023-12-07T10:18:01.989335+0000","last_undegraded":"2023-12-07T10:18:01.989335+0000","last_fullsized":"2023-12-07T10:18:01.989335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:34:08.442828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.254202+0000","last_change":"2023-12-07T10:17:45.803858+0000","last_active":"2023-12-07T10:18:02.254202+0000","last_peered":"2023-12-07T10:18:02.254202+0000","last_clean":"2023-12-07T10:18:02.254202+0000","last_became_active":"2023-12-07T10:17:45.803481+0000","last_became_peered":"2023-12-07T10:17:45.803481+0000","last_unstale":"2023-12-07T10:18:02.254202+0000","last_undegraded":"2023-12-07T10:18:02.254202+0000","last_fullsized":"2023-12-07T10:18:02.254202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:08:03.728554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:13.774156+0000","last_change":"2023-12-07T10:17:40.751866+0000","last_active":"2023-12-07T10:18:13.774156+0000","last_peered":"2023-12-07T10:18:13.774156+0000","last_clean":"2023-12-07T10:18:13.774156+0000","last_became_active":"2023-12-07T10:17:40.751492+0000","last_became_peered":"2023-12-07T10:17:40.751492+0000","last_unstale":"2023-12-07T10:18:13.774156+0000","last_undegraded":"2023-12-07T10:18:13.774156+0000","last_fullsized":"2023-12-07T10:18:13.774156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:42:53.839651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855907+0000","last_change":"2023-12-07T10:17:41.611326+0000","last_active":"2023-12-07T10:18:00.855907+0000","last_peered":"2023-12-07T10:18:00.855907+0000","last_clean":"2023-12-07T10:18:00.855907+0000","last_became_active":"2023-12-07T10:17:41.610863+0000","last_became_peered":"2023-12-07T10:17:41.610863+0000","last_unstale":"2023-12-07T10:18:00.855907+0000","last_undegraded":"2023-12-07T10:18:00.855907+0000","last_fullsized":"2023-12-07T10:18:00.855907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:07:55.801356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.850230+0000","last_change":"2023-12-07T10:17:45.791409+0000","last_active":"2023-12-07T10:18:00.850230+0000","last_peered":"2023-12-07T10:18:00.850230+0000","last_clean":"2023-12-07T10:18:00.850230+0000","last_became_active":"2023-12-07T10:17:45.791022+0000","last_became_peered":"2023-12-07T10:17:45.791022+0000","last_unstale":"2023-12-07T10:18:00.850230+0000","last_undegraded":"2023-12-07T10:18:00.850230+0000","last_fullsized":"2023-12-07T10:18:00.850230+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:57:14.617409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'9","reported_seq":81,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:56.805755+0000","last_change":"2023-12-07T10:17:40.745318+0000","last_active":"2023-12-07T10:18:56.805755+0000","last_peered":"2023-12-07T10:18:56.805755+0000","last_clean":"2023-12-07T10:18:56.805755+0000","last_became_active":"2023-12-07T10:17:40.745008+0000","last_became_peered":"2023-12-07T10:17:40.745008+0000","last_unstale":"2023-12-07T10:18:56.805755+0000","last_undegraded":"2023-12-07T10:18:56.805755+0000","last_fullsized":"2023-12-07T10:18:56.805755+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:42:28.503421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11004,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":14,"num_write":6,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.858287+0000","last_change":"2023-12-07T10:17:41.605121+0000","last_active":"2023-12-07T10:18:01.858287+0000","last_peered":"2023-12-07T10:18:01.858287+0000","last_clean":"2023-12-07T10:18:01.858287+0000","last_became_active":"2023-12-07T10:17:41.604770+0000","last_became_peered":"2023-12-07T10:17:41.604770+0000","last_unstale":"2023-12-07T10:18:01.858287+0000","last_undegraded":"2023-12-07T10:18:01.858287+0000","last_fullsized":"2023-12-07T10:18:01.858287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:51:02.151155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.850337+0000","last_change":"2023-12-07T10:17:45.809039+0000","last_active":"2023-12-07T10:18:00.850337+0000","last_peered":"2023-12-07T10:18:00.850337+0000","last_clean":"2023-12-07T10:18:00.850337+0000","last_became_active":"2023-12-07T10:17:45.808622+0000","last_became_peered":"2023-12-07T10:17:45.808622+0000","last_unstale":"2023-12-07T10:18:00.850337+0000","last_undegraded":"2023-12-07T10:18:00.850337+0000","last_fullsized":"2023-12-07T10:18:00.850337+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:16:02.835931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:11.747711+0000","last_change":"2023-12-07T10:17:40.757876+0000","last_active":"2023-12-07T10:18:11.747711+0000","last_peered":"2023-12-07T10:18:11.747711+0000","last_clean":"2023-12-07T10:18:11.747711+0000","last_became_active":"2023-12-07T10:17:40.757613+0000","last_became_peered":"2023-12-07T10:17:40.757613+0000","last_unstale":"2023-12-07T10:18:11.747711+0000","last_undegraded":"2023-12-07T10:18:11.747711+0000","last_fullsized":"2023-12-07T10:18:11.747711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:30:40.423092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856026+0000","last_change":"2023-12-07T10:17:41.611575+0000","last_active":"2023-12-07T10:18:00.856026+0000","last_peered":"2023-12-07T10:18:00.856026+0000","last_clean":"2023-12-07T10:18:00.856026+0000","last_became_active":"2023-12-07T10:17:41.611042+0000","last_became_peered":"2023-12-07T10:17:41.611042+0000","last_unstale":"2023-12-07T10:18:00.856026+0000","last_undegraded":"2023-12-07T10:18:00.856026+0000","last_fullsized":"2023-12-07T10:18:00.856026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:54:05.589792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.856029+0000","last_change":"2023-12-07T10:17:45.796640+0000","last_active":"2023-12-07T10:18:01.856029+0000","last_peered":"2023-12-07T10:18:01.856029+0000","last_clean":"2023-12-07T10:18:01.856029+0000","last_became_active":"2023-12-07T10:17:45.796183+0000","last_became_peered":"2023-12-07T10:17:45.796183+0000","last_unstale":"2023-12-07T10:18:01.856029+0000","last_undegraded":"2023-12-07T10:18:01.856029+0000","last_fullsized":"2023-12-07T10:18:01.856029+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:41:56.890662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:10.744033+0000","last_change":"2023-12-07T10:17:40.760283+0000","last_active":"2023-12-07T10:18:10.744033+0000","last_peered":"2023-12-07T10:18:10.744033+0000","last_clean":"2023-12-07T10:18:10.744033+0000","last_became_active":"2023-12-07T10:17:40.759398+0000","last_became_peered":"2023-12-07T10:17:40.759398+0000","last_unstale":"2023-12-07T10:18:10.744033+0000","last_undegraded":"2023-12-07T10:18:10.744033+0000","last_fullsized":"2023-12-07T10:18:10.744033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:34:37.365490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849107+0000","last_change":"2023-12-07T10:17:41.604150+0000","last_active":"2023-12-07T10:18:00.849107+0000","last_peered":"2023-12-07T10:18:00.849107+0000","last_clean":"2023-12-07T10:18:00.849107+0000","last_became_active":"2023-12-07T10:17:41.603688+0000","last_became_peered":"2023-12-07T10:17:41.603688+0000","last_unstale":"2023-12-07T10:18:00.849107+0000","last_undegraded":"2023-12-07T10:18:00.849107+0000","last_fullsized":"2023-12-07T10:18:00.849107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:52:43.735609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852965+0000","last_change":"2023-12-07T10:17:45.806569+0000","last_active":"2023-12-07T10:18:00.852965+0000","last_peered":"2023-12-07T10:18:00.852965+0000","last_clean":"2023-12-07T10:18:00.852965+0000","last_became_active":"2023-12-07T10:17:45.806099+0000","last_became_peered":"2023-12-07T10:17:45.806099+0000","last_unstale":"2023-12-07T10:18:00.852965+0000","last_undegraded":"2023-12-07T10:18:00.852965+0000","last_fullsized":"2023-12-07T10:18:00.852965+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:02:19.977670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.757927+0000","last_change":"2023-12-07T10:17:40.744388+0000","last_active":"2023-12-07T10:18:12.757927+0000","last_peered":"2023-12-07T10:18:12.757927+0000","last_clean":"2023-12-07T10:18:12.757927+0000","last_became_active":"2023-12-07T10:17:40.743887+0000","last_became_peered":"2023-12-07T10:17:40.743887+0000","last_unstale":"2023-12-07T10:18:12.757927+0000","last_undegraded":"2023-12-07T10:18:12.757927+0000","last_fullsized":"2023-12-07T10:18:12.757927+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:18:39.102099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.845569+0000","last_change":"2023-12-07T10:17:41.595616+0000","last_active":"2023-12-07T10:18:00.845569+0000","last_peered":"2023-12-07T10:18:00.845569+0000","last_clean":"2023-12-07T10:18:00.845569+0000","last_became_active":"2023-12-07T10:17:41.595256+0000","last_became_peered":"2023-12-07T10:17:41.595256+0000","last_unstale":"2023-12-07T10:18:00.845569+0000","last_undegraded":"2023-12-07T10:18:00.845569+0000","last_fullsized":"2023-12-07T10:18:00.845569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:08:00.156043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849668+0000","last_change":"2023-12-07T10:17:45.794523+0000","last_active":"2023-12-07T10:18:00.849668+0000","last_peered":"2023-12-07T10:18:00.849668+0000","last_clean":"2023-12-07T10:18:00.849668+0000","last_became_active":"2023-12-07T10:17:45.794139+0000","last_became_peered":"2023-12-07T10:17:45.794139+0000","last_unstale":"2023-12-07T10:18:00.849668+0000","last_undegraded":"2023-12-07T10:18:00.849668+0000","last_fullsized":"2023-12-07T10:18:00.849668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:43:24.767627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":120,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:59.809174+0000","last_change":"2023-12-07T10:17:40.759620+0000","last_active":"2023-12-07T10:18:59.809174+0000","last_peered":"2023-12-07T10:18:59.809174+0000","last_clean":"2023-12-07T10:18:59.809174+0000","last_became_active":"2023-12-07T10:17:40.759372+0000","last_became_peered":"2023-12-07T10:17:40.759372+0000","last_unstale":"2023-12-07T10:18:59.809174+0000","last_undegraded":"2023-12-07T10:18:59.809174+0000","last_fullsized":"2023-12-07T10:18:59.809174+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:46:14.459409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.989575+0000","last_change":"2023-12-07T10:17:41.601152+0000","last_active":"2023-12-07T10:18:01.989575+0000","last_peered":"2023-12-07T10:18:01.989575+0000","last_clean":"2023-12-07T10:18:01.989575+0000","last_became_active":"2023-12-07T10:17:41.600147+0000","last_became_peered":"2023-12-07T10:17:41.600147+0000","last_unstale":"2023-12-07T10:18:01.989575+0000","last_undegraded":"2023-12-07T10:18:01.989575+0000","last_fullsized":"2023-12-07T10:18:01.989575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:40:46.671489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.845356+0000","last_change":"2023-12-07T10:17:45.796229+0000","last_active":"2023-12-07T10:18:00.845356+0000","last_peered":"2023-12-07T10:18:00.845356+0000","last_clean":"2023-12-07T10:18:00.845356+0000","last_became_active":"2023-12-07T10:17:45.795582+0000","last_became_peered":"2023-12-07T10:17:45.795582+0000","last_unstale":"2023-12-07T10:18:00.845356+0000","last_undegraded":"2023-12-07T10:18:00.845356+0000","last_fullsized":"2023-12-07T10:18:00.845356+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:29:32.853975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.758506+0000","last_change":"2023-12-07T10:17:40.748353+0000","last_active":"2023-12-07T10:18:12.758506+0000","last_peered":"2023-12-07T10:18:12.758506+0000","last_clean":"2023-12-07T10:18:12.758506+0000","last_became_active":"2023-12-07T10:17:40.748050+0000","last_became_peered":"2023-12-07T10:17:40.748050+0000","last_unstale":"2023-12-07T10:18:12.758506+0000","last_undegraded":"2023-12-07T10:18:12.758506+0000","last_fullsized":"2023-12-07T10:18:12.758506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:18:15.298534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.256728+0000","last_change":"2023-12-07T10:18:02.256728+0000","last_active":"2023-12-07T10:18:02.256728+0000","last_peered":"2023-12-07T10:18:02.256728+0000","last_clean":"2023-12-07T10:18:02.256728+0000","last_became_active":"2023-12-07T10:18:02.256271+0000","last_became_peered":"2023-12-07T10:18:02.256271+0000","last_unstale":"2023-12-07T10:18:02.256728+0000","last_undegraded":"2023-12-07T10:18:02.256728+0000","last_fullsized":"2023-12-07T10:18:02.256728+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:27:00.198934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.254711+0000","last_change":"2023-12-07T10:17:45.796903+0000","last_active":"2023-12-07T10:18:02.254711+0000","last_peered":"2023-12-07T10:18:02.254711+0000","last_clean":"2023-12-07T10:18:02.254711+0000","last_became_active":"2023-12-07T10:17:45.796454+0000","last_became_peered":"2023-12-07T10:17:45.796454+0000","last_unstale":"2023-12-07T10:18:02.254711+0000","last_undegraded":"2023-12-07T10:18:02.254711+0000","last_fullsized":"2023-12-07T10:18:02.254711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:46:36.955396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:10.734628+0000","last_change":"2023-12-07T10:17:40.756045+0000","last_active":"2023-12-07T10:18:10.734628+0000","last_peered":"2023-12-07T10:18:10.734628+0000","last_clean":"2023-12-07T10:18:10.734628+0000","last_became_active":"2023-12-07T10:17:40.755763+0000","last_became_peered":"2023-12-07T10:17:40.755763+0000","last_unstale":"2023-12-07T10:18:10.734628+0000","last_undegraded":"2023-12-07T10:18:10.734628+0000","last_fullsized":"2023-12-07T10:18:10.734628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:01:08.009076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.989591+0000","last_change":"2023-12-07T10:17:41.611178+0000","last_active":"2023-12-07T10:18:01.989591+0000","last_peered":"2023-12-07T10:18:01.989591+0000","last_clean":"2023-12-07T10:18:01.989591+0000","last_became_active":"2023-12-07T10:17:41.610750+0000","last_became_peered":"2023-12-07T10:17:41.610750+0000","last_unstale":"2023-12-07T10:18:01.989591+0000","last_undegraded":"2023-12-07T10:18:01.989591+0000","last_fullsized":"2023-12-07T10:18:01.989591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:56:30.763523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.845439+0000","last_change":"2023-12-07T10:17:45.800183+0000","last_active":"2023-12-07T10:18:00.845439+0000","last_peered":"2023-12-07T10:18:00.845439+0000","last_clean":"2023-12-07T10:18:00.845439+0000","last_became_active":"2023-12-07T10:17:45.798598+0000","last_became_peered":"2023-12-07T10:17:45.798598+0000","last_unstale":"2023-12-07T10:18:00.845439+0000","last_undegraded":"2023-12-07T10:18:00.845439+0000","last_fullsized":"2023-12-07T10:18:00.845439+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:06:22.162951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":160,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:59.812240+0000","last_change":"2023-12-07T10:17:40.752481+0000","last_active":"2023-12-07T10:18:59.812240+0000","last_peered":"2023-12-07T10:18:59.812240+0000","last_clean":"2023-12-07T10:18:59.812240+0000","last_became_active":"2023-12-07T10:17:40.752158+0000","last_became_peered":"2023-12-07T10:17:40.752158+0000","last_unstale":"2023-12-07T10:18:59.812240+0000","last_undegraded":"2023-12-07T10:18:59.812240+0000","last_fullsized":"2023-12-07T10:18:59.812240+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:43:27.963014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64453,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":107,"num_read_kb":116,"num_write":36,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.859129+0000","last_change":"2023-12-07T10:17:41.614107+0000","last_active":"2023-12-07T10:18:01.859129+0000","last_peered":"2023-12-07T10:18:01.859129+0000","last_clean":"2023-12-07T10:18:01.859129+0000","last_became_active":"2023-12-07T10:17:41.613743+0000","last_became_peered":"2023-12-07T10:17:41.613743+0000","last_unstale":"2023-12-07T10:18:01.859129+0000","last_undegraded":"2023-12-07T10:18:01.859129+0000","last_fullsized":"2023-12-07T10:18:01.859129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:49:03.200387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856289+0000","last_change":"2023-12-07T10:17:45.804282+0000","last_active":"2023-12-07T10:18:00.856289+0000","last_peered":"2023-12-07T10:18:00.856289+0000","last_clean":"2023-12-07T10:18:00.856289+0000","last_became_active":"2023-12-07T10:17:45.803794+0000","last_became_peered":"2023-12-07T10:17:45.803794+0000","last_unstale":"2023-12-07T10:18:00.856289+0000","last_undegraded":"2023-12-07T10:18:00.856289+0000","last_fullsized":"2023-12-07T10:18:00.856289+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:31:26.887565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.756900+0000","last_change":"2023-12-07T10:17:40.747204+0000","last_active":"2023-12-07T10:18:12.756900+0000","last_peered":"2023-12-07T10:18:12.756900+0000","last_clean":"2023-12-07T10:18:12.756900+0000","last_became_active":"2023-12-07T10:17:40.746916+0000","last_became_peered":"2023-12-07T10:17:40.746916+0000","last_unstale":"2023-12-07T10:18:12.756900+0000","last_undegraded":"2023-12-07T10:18:12.756900+0000","last_fullsized":"2023-12-07T10:18:12.756900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:53:49.228320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855125+0000","last_change":"2023-12-07T10:17:41.606895+0000","last_active":"2023-12-07T10:18:00.855125+0000","last_peered":"2023-12-07T10:18:00.855125+0000","last_clean":"2023-12-07T10:18:00.855125+0000","last_became_active":"2023-12-07T10:17:41.606596+0000","last_became_peered":"2023-12-07T10:17:41.606596+0000","last_unstale":"2023-12-07T10:18:00.855125+0000","last_undegraded":"2023-12-07T10:18:00.855125+0000","last_fullsized":"2023-12-07T10:18:00.855125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:34:25.166906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.856131+0000","last_change":"2023-12-07T10:17:45.807101+0000","last_active":"2023-12-07T10:18:01.856131+0000","last_peered":"2023-12-07T10:18:01.856131+0000","last_clean":"2023-12-07T10:18:01.856131+0000","last_became_active":"2023-12-07T10:17:45.806770+0000","last_became_peered":"2023-12-07T10:17:45.806770+0000","last_unstale":"2023-12-07T10:18:01.856131+0000","last_undegraded":"2023-12-07T10:18:01.856131+0000","last_fullsized":"2023-12-07T10:18:01.856131+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:39:32.522968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.756459+0000","last_change":"2023-12-07T10:17:40.754856+0000","last_active":"2023-12-07T10:18:12.756459+0000","last_peered":"2023-12-07T10:18:12.756459+0000","last_clean":"2023-12-07T10:18:12.756459+0000","last_became_active":"2023-12-07T10:17:40.754488+0000","last_became_peered":"2023-12-07T10:17:40.754488+0000","last_unstale":"2023-12-07T10:18:12.756459+0000","last_undegraded":"2023-12-07T10:18:12.756459+0000","last_fullsized":"2023-12-07T10:18:12.756459+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:44:50.960216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856759+0000","last_change":"2023-12-07T10:17:41.606963+0000","last_active":"2023-12-07T10:18:00.856759+0000","last_peered":"2023-12-07T10:18:00.856759+0000","last_clean":"2023-12-07T10:18:00.856759+0000","last_became_active":"2023-12-07T10:17:41.606408+0000","last_became_peered":"2023-12-07T10:17:41.606408+0000","last_unstale":"2023-12-07T10:18:00.856759+0000","last_undegraded":"2023-12-07T10:18:00.856759+0000","last_fullsized":"2023-12-07T10:18:00.856759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:25:47.906649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":2,"reported_epoch":92,"state":"peering","last_fresh":"2023-12-07T10:17:45.779804+0000","last_change":"2023-12-07T10:19:02.353784+0000","last_active":"2023-12-07T10:17:44.766631+0000","last_peered":"2023-12-07T10:17:44.766631+0000","last_clean":"2023-12-07T10:17:44.766631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-07T10:17:45.779804+0000","last_undegraded":"2023-12-07T10:17:45.779804+0000","last_fullsized":"2023-12-07T10:17:45.779804+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":76,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:19:00.835987+0000","last_change":"2023-12-07T10:17:40.747494+0000","last_active":"2023-12-07T10:19:00.835987+0000","last_peered":"2023-12-07T10:19:00.835987+0000","last_clean":"2023-12-07T10:19:00.835987+0000","last_became_active":"2023-12-07T10:17:40.747157+0000","last_became_peered":"2023-12-07T10:17:40.747157+0000","last_unstale":"2023-12-07T10:19:00.835987+0000","last_undegraded":"2023-12-07T10:19:00.835987+0000","last_fullsized":"2023-12-07T10:19:00.835987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:45:24.395418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":976,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856481+0000","last_change":"2023-12-07T10:17:41.609495+0000","last_active":"2023-12-07T10:18:00.856481+0000","last_peered":"2023-12-07T10:18:00.856481+0000","last_clean":"2023-12-07T10:18:00.856481+0000","last_became_active":"2023-12-07T10:17:41.609181+0000","last_became_peered":"2023-12-07T10:17:41.609181+0000","last_unstale":"2023-12-07T10:18:00.856481+0000","last_undegraded":"2023-12-07T10:18:00.856481+0000","last_fullsized":"2023-12-07T10:18:00.856481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:20:33.473824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855858+0000","last_change":"2023-12-07T10:17:45.808862+0000","last_active":"2023-12-07T10:18:00.855858+0000","last_peered":"2023-12-07T10:18:00.855858+0000","last_clean":"2023-12-07T10:18:00.855858+0000","last_became_active":"2023-12-07T10:17:45.808461+0000","last_became_peered":"2023-12-07T10:17:45.808461+0000","last_unstale":"2023-12-07T10:18:00.855858+0000","last_undegraded":"2023-12-07T10:18:00.855858+0000","last_fullsized":"2023-12-07T10:18:00.855858+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:09:12.598126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:13.769981+0000","last_change":"2023-12-07T10:18:02.263709+0000","last_active":"2023-12-07T10:18:13.769981+0000","last_peered":"2023-12-07T10:18:13.769981+0000","last_clean":"2023-12-07T10:18:13.769981+0000","last_became_active":"2023-12-07T10:18:02.255788+0000","last_became_peered":"2023-12-07T10:18:02.255788+0000","last_unstale":"2023-12-07T10:18:13.769981+0000","last_undegraded":"2023-12-07T10:18:13.769981+0000","last_fullsized":"2023-12-07T10:18:13.769981+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:12:04.976226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.854358+0000","last_change":"2023-12-07T10:17:41.610509+0000","last_active":"2023-12-07T10:18:00.854358+0000","last_peered":"2023-12-07T10:18:00.854358+0000","last_clean":"2023-12-07T10:18:00.854358+0000","last_became_active":"2023-12-07T10:17:41.610121+0000","last_became_peered":"2023-12-07T10:17:41.610121+0000","last_unstale":"2023-12-07T10:18:00.854358+0000","last_undegraded":"2023-12-07T10:18:00.854358+0000","last_fullsized":"2023-12-07T10:18:00.854358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:53:17.365880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.854272+0000","last_change":"2023-12-07T10:17:45.795818+0000","last_active":"2023-12-07T10:18:00.854272+0000","last_peered":"2023-12-07T10:18:00.854272+0000","last_clean":"2023-12-07T10:18:00.854272+0000","last_became_active":"2023-12-07T10:17:45.795477+0000","last_became_peered":"2023-12-07T10:17:45.795477+0000","last_unstale":"2023-12-07T10:18:00.854272+0000","last_undegraded":"2023-12-07T10:18:00.854272+0000","last_fullsized":"2023-12-07T10:18:00.854272+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:17:21.494654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:15.281962+0000","last_change":"2023-12-07T10:17:40.751791+0000","last_active":"2023-12-07T10:18:15.281962+0000","last_peered":"2023-12-07T10:18:15.281962+0000","last_clean":"2023-12-07T10:18:15.281962+0000","last_became_active":"2023-12-07T10:17:40.751502+0000","last_became_peered":"2023-12-07T10:17:40.751502+0000","last_unstale":"2023-12-07T10:18:15.281962+0000","last_undegraded":"2023-12-07T10:18:15.281962+0000","last_fullsized":"2023-12-07T10:18:15.281962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:15:52.319265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.257186+0000","last_change":"2023-12-07T10:18:02.256247+0000","last_active":"2023-12-07T10:18:02.257186+0000","last_peered":"2023-12-07T10:18:02.257186+0000","last_clean":"2023-12-07T10:18:02.257186+0000","last_became_active":"2023-12-07T10:18:02.255732+0000","last_became_peered":"2023-12-07T10:18:02.255732+0000","last_unstale":"2023-12-07T10:18:02.257186+0000","last_undegraded":"2023-12-07T10:18:02.257186+0000","last_fullsized":"2023-12-07T10:18:02.257186+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:00:06.516804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.859099+0000","last_change":"2023-12-07T10:17:45.791966+0000","last_active":"2023-12-07T10:18:01.859099+0000","last_peered":"2023-12-07T10:18:01.859099+0000","last_clean":"2023-12-07T10:18:01.859099+0000","last_became_active":"2023-12-07T10:17:45.791613+0000","last_became_peered":"2023-12-07T10:17:45.791613+0000","last_unstale":"2023-12-07T10:18:01.859099+0000","last_undegraded":"2023-12-07T10:18:01.859099+0000","last_fullsized":"2023-12-07T10:18:01.859099+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:13:02.291289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.853559+0000","last_change":"2023-12-07T10:17:40.762896+0000","last_active":"2023-12-07T10:18:01.853559+0000","last_peered":"2023-12-07T10:18:01.853559+0000","last_clean":"2023-12-07T10:18:01.853559+0000","last_became_active":"2023-12-07T10:17:40.762427+0000","last_became_peered":"2023-12-07T10:17:40.762427+0000","last_unstale":"2023-12-07T10:18:01.853559+0000","last_undegraded":"2023-12-07T10:18:01.853559+0000","last_fullsized":"2023-12-07T10:18:01.853559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:56:35.589548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.854524+0000","last_change":"2023-12-07T10:17:41.610354+0000","last_active":"2023-12-07T10:18:00.854524+0000","last_peered":"2023-12-07T10:18:00.854524+0000","last_clean":"2023-12-07T10:18:00.854524+0000","last_became_active":"2023-12-07T10:17:41.609888+0000","last_became_peered":"2023-12-07T10:17:41.609888+0000","last_unstale":"2023-12-07T10:18:00.854524+0000","last_undegraded":"2023-12-07T10:18:00.854524+0000","last_fullsized":"2023-12-07T10:18:00.854524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:48:19.976949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855124+0000","last_change":"2023-12-07T10:17:45.807324+0000","last_active":"2023-12-07T10:18:00.855124+0000","last_peered":"2023-12-07T10:18:00.855124+0000","last_clean":"2023-12-07T10:18:00.855124+0000","last_became_active":"2023-12-07T10:17:45.806036+0000","last_became_peered":"2023-12-07T10:17:45.806036+0000","last_unstale":"2023-12-07T10:18:00.855124+0000","last_undegraded":"2023-12-07T10:18:00.855124+0000","last_fullsized":"2023-12-07T10:18:00.855124+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:54:36.847992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'5","reported_seq":72,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:58.834748+0000","last_change":"2023-12-07T10:17:40.754247+0000","last_active":"2023-12-07T10:18:58.834748+0000","last_peered":"2023-12-07T10:18:58.834748+0000","last_clean":"2023-12-07T10:18:58.834748+0000","last_became_active":"2023-12-07T10:17:40.753964+0000","last_became_peered":"2023-12-07T10:17:40.753964+0000","last_unstale":"2023-12-07T10:18:58.834748+0000","last_undegraded":"2023-12-07T10:18:58.834748+0000","last_fullsized":"2023-12-07T10:18:58.834748+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:22:04.832399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.854494+0000","last_change":"2023-12-07T10:17:41.610516+0000","last_active":"2023-12-07T10:18:00.854494+0000","last_peered":"2023-12-07T10:18:00.854494+0000","last_clean":"2023-12-07T10:18:00.854494+0000","last_became_active":"2023-12-07T10:17:41.610184+0000","last_became_peered":"2023-12-07T10:17:41.610184+0000","last_unstale":"2023-12-07T10:18:00.854494+0000","last_undegraded":"2023-12-07T10:18:00.854494+0000","last_fullsized":"2023-12-07T10:18:00.854494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:53:43.100723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855260+0000","last_change":"2023-12-07T10:17:45.804624+0000","last_active":"2023-12-07T10:18:00.855260+0000","last_peered":"2023-12-07T10:18:00.855260+0000","last_clean":"2023-12-07T10:18:00.855260+0000","last_became_active":"2023-12-07T10:17:45.803667+0000","last_became_peered":"2023-12-07T10:17:45.803667+0000","last_unstale":"2023-12-07T10:18:00.855260+0000","last_undegraded":"2023-12-07T10:18:00.855260+0000","last_fullsized":"2023-12-07T10:18:00.855260+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:55:09.642795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855573+0000","last_change":"2023-12-07T10:17:41.618433+0000","last_active":"2023-12-07T10:18:00.855573+0000","last_peered":"2023-12-07T10:18:00.855573+0000","last_clean":"2023-12-07T10:18:00.855573+0000","last_became_active":"2023-12-07T10:17:41.617942+0000","last_became_peered":"2023-12-07T10:17:41.617942+0000","last_unstale":"2023-12-07T10:18:00.855573+0000","last_undegraded":"2023-12-07T10:18:00.855573+0000","last_fullsized":"2023-12-07T10:18:00.855573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:26:48.166157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":70,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:56.830170+0000","last_change":"2023-12-07T10:17:40.761907+0000","last_active":"2023-12-07T10:18:56.830170+0000","last_peered":"2023-12-07T10:18:56.830170+0000","last_clean":"2023-12-07T10:18:56.830170+0000","last_became_active":"2023-12-07T10:17:40.761564+0000","last_became_peered":"2023-12-07T10:17:40.761564+0000","last_unstale":"2023-12-07T10:18:56.830170+0000","last_undegraded":"2023-12-07T10:18:56.830170+0000","last_fullsized":"2023-12-07T10:18:56.830170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:57:22.304352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":46,"num_read_kb":46,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.854203+0000","last_change":"2023-12-07T10:17:45.792684+0000","last_active":"2023-12-07T10:18:01.854203+0000","last_peered":"2023-12-07T10:18:01.854203+0000","last_clean":"2023-12-07T10:18:01.854203+0000","last_became_active":"2023-12-07T10:17:45.792172+0000","last_became_peered":"2023-12-07T10:17:45.792172+0000","last_unstale":"2023-12-07T10:18:01.854203+0000","last_undegraded":"2023-12-07T10:18:01.854203+0000","last_fullsized":"2023-12-07T10:18:01.854203+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:24:31.084727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849807+0000","last_change":"2023-12-07T10:17:41.612582+0000","last_active":"2023-12-07T10:18:00.849807+0000","last_peered":"2023-12-07T10:18:00.849807+0000","last_clean":"2023-12-07T10:18:00.849807+0000","last_became_active":"2023-12-07T10:17:41.612244+0000","last_became_peered":"2023-12-07T10:17:41.612244+0000","last_unstale":"2023-12-07T10:18:00.849807+0000","last_undegraded":"2023-12-07T10:18:00.849807+0000","last_fullsized":"2023-12-07T10:18:00.849807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:41:53.506037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:13.770423+0000","last_change":"2023-12-07T10:17:40.748565+0000","last_active":"2023-12-07T10:18:13.770423+0000","last_peered":"2023-12-07T10:18:13.770423+0000","last_clean":"2023-12-07T10:18:13.770423+0000","last_became_active":"2023-12-07T10:17:40.748250+0000","last_became_peered":"2023-12-07T10:17:40.748250+0000","last_unstale":"2023-12-07T10:18:13.770423+0000","last_undegraded":"2023-12-07T10:18:13.770423+0000","last_fullsized":"2023-12-07T10:18:13.770423+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:26:28.202440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":326,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856336+0000","last_change":"2023-12-07T10:16:33.482173+0000","last_active":"2023-12-07T10:18:00.856336+0000","last_peered":"2023-12-07T10:18:00.856336+0000","last_clean":"2023-12-07T10:18:00.856336+0000","last_became_active":"2023-12-07T10:16:33.481699+0000","last_became_peered":"2023-12-07T10:16:33.481699+0000","last_unstale":"2023-12-07T10:18:00.856336+0000","last_undegraded":"2023-12-07T10:18:00.856336+0000","last_fullsized":"2023-12-07T10:18:00.856336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:13:25.388258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:13:25.388258+0000","last_clean_scrub_stamp":"2023-12-07T10:13:25.388258+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:57:09.583677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856130+0000","last_change":"2023-12-07T10:17:45.806289+0000","last_active":"2023-12-07T10:18:00.856130+0000","last_peered":"2023-12-07T10:18:00.856130+0000","last_clean":"2023-12-07T10:18:00.856130+0000","last_became_active":"2023-12-07T10:17:45.805954+0000","last_became_peered":"2023-12-07T10:17:45.805954+0000","last_unstale":"2023-12-07T10:18:00.856130+0000","last_undegraded":"2023-12-07T10:18:00.856130+0000","last_fullsized":"2023-12-07T10:18:00.856130+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:21:09.955380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855381+0000","last_change":"2023-12-07T10:17:41.612322+0000","last_active":"2023-12-07T10:18:00.855381+0000","last_peered":"2023-12-07T10:18:00.855381+0000","last_clean":"2023-12-07T10:18:00.855381+0000","last_became_active":"2023-12-07T10:17:41.612039+0000","last_became_peered":"2023-12-07T10:17:41.612039+0000","last_unstale":"2023-12-07T10:18:00.855381+0000","last_undegraded":"2023-12-07T10:18:00.855381+0000","last_fullsized":"2023-12-07T10:18:00.855381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:40:49.883742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:10.742637+0000","last_change":"2023-12-07T10:17:40.749694+0000","last_active":"2023-12-07T10:18:10.742637+0000","last_peered":"2023-12-07T10:18:10.742637+0000","last_clean":"2023-12-07T10:18:10.742637+0000","last_became_active":"2023-12-07T10:17:40.749393+0000","last_became_peered":"2023-12-07T10:17:40.749393+0000","last_unstale":"2023-12-07T10:18:10.742637+0000","last_undegraded":"2023-12-07T10:18:10.742637+0000","last_fullsized":"2023-12-07T10:18:10.742637+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:15:36.949062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.855939+0000","last_change":"2023-12-07T10:17:45.805769+0000","last_active":"2023-12-07T10:18:01.855939+0000","last_peered":"2023-12-07T10:18:01.855939+0000","last_clean":"2023-12-07T10:18:01.855939+0000","last_became_active":"2023-12-07T10:17:45.805346+0000","last_became_peered":"2023-12-07T10:17:45.805346+0000","last_unstale":"2023-12-07T10:18:01.855939+0000","last_undegraded":"2023-12-07T10:18:01.855939+0000","last_fullsized":"2023-12-07T10:18:01.855939+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:47:47.781933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.756983+0000","last_change":"2023-12-07T10:18:01.862558+0000","last_active":"2023-12-07T10:18:12.756983+0000","last_peered":"2023-12-07T10:18:12.756983+0000","last_clean":"2023-12-07T10:18:12.756983+0000","last_became_active":"2023-12-07T10:18:01.862063+0000","last_became_peered":"2023-12-07T10:18:01.862063+0000","last_unstale":"2023-12-07T10:18:12.756983+0000","last_undegraded":"2023-12-07T10:18:12.756983+0000","last_fullsized":"2023-12-07T10:18:12.756983+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:25:32.652160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855264+0000","last_change":"2023-12-07T10:17:41.617199+0000","last_active":"2023-12-07T10:18:00.855264+0000","last_peered":"2023-12-07T10:18:00.855264+0000","last_clean":"2023-12-07T10:18:00.855264+0000","last_became_active":"2023-12-07T10:17:41.616901+0000","last_became_peered":"2023-12-07T10:17:41.616901+0000","last_unstale":"2023-12-07T10:18:00.855264+0000","last_undegraded":"2023-12-07T10:18:00.855264+0000","last_fullsized":"2023-12-07T10:18:00.855264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:23:59.433861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849025+0000","last_change":"2023-12-07T10:17:45.790914+0000","last_active":"2023-12-07T10:18:00.849025+0000","last_peered":"2023-12-07T10:18:00.849025+0000","last_clean":"2023-12-07T10:18:00.849025+0000","last_became_active":"2023-12-07T10:17:45.790428+0000","last_became_peered":"2023-12-07T10:17:45.790428+0000","last_unstale":"2023-12-07T10:18:00.849025+0000","last_undegraded":"2023-12-07T10:18:00.849025+0000","last_fullsized":"2023-12-07T10:18:00.849025+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:03:57.516300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'10","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:18.543684+0000","last_change":"2023-12-07T10:18:01.863765+0000","last_active":"2023-12-07T10:18:18.543684+0000","last_peered":"2023-12-07T10:18:18.543684+0000","last_clean":"2023-12-07T10:18:18.543684+0000","last_became_active":"2023-12-07T10:18:01.858926+0000","last_became_peered":"2023-12-07T10:18:01.858926+0000","last_unstale":"2023-12-07T10:18:18.543684+0000","last_undegraded":"2023-12-07T10:18:18.543684+0000","last_fullsized":"2023-12-07T10:18:18.543684+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:44:19.663592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849322+0000","last_change":"2023-12-07T10:17:41.608513+0000","last_active":"2023-12-07T10:18:00.849322+0000","last_peered":"2023-12-07T10:18:00.849322+0000","last_clean":"2023-12-07T10:18:00.849322+0000","last_became_active":"2023-12-07T10:17:41.605049+0000","last_became_peered":"2023-12-07T10:17:41.605049+0000","last_unstale":"2023-12-07T10:18:00.849322+0000","last_undegraded":"2023-12-07T10:18:00.849322+0000","last_fullsized":"2023-12-07T10:18:00.849322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:33:00.730827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.850303+0000","last_change":"2023-12-07T10:17:45.792583+0000","last_active":"2023-12-07T10:18:00.850303+0000","last_peered":"2023-12-07T10:18:00.850303+0000","last_clean":"2023-12-07T10:18:00.850303+0000","last_became_active":"2023-12-07T10:17:45.792162+0000","last_became_peered":"2023-12-07T10:17:45.792162+0000","last_unstale":"2023-12-07T10:18:00.850303+0000","last_undegraded":"2023-12-07T10:18:00.850303+0000","last_fullsized":"2023-12-07T10:18:00.850303+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:47:30.901477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:13.769485+0000","last_change":"2023-12-07T10:17:40.750826+0000","last_active":"2023-12-07T10:18:13.769485+0000","last_peered":"2023-12-07T10:18:13.769485+0000","last_clean":"2023-12-07T10:18:13.769485+0000","last_became_active":"2023-12-07T10:17:40.750519+0000","last_became_peered":"2023-12-07T10:17:40.750519+0000","last_unstale":"2023-12-07T10:18:13.769485+0000","last_undegraded":"2023-12-07T10:18:13.769485+0000","last_fullsized":"2023-12-07T10:18:13.769485+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:21:01.008897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.853040+0000","last_change":"2023-12-07T10:17:41.609488+0000","last_active":"2023-12-07T10:18:00.853040+0000","last_peered":"2023-12-07T10:18:00.853040+0000","last_clean":"2023-12-07T10:18:00.853040+0000","last_became_active":"2023-12-07T10:17:41.609061+0000","last_became_peered":"2023-12-07T10:17:41.609061+0000","last_unstale":"2023-12-07T10:18:00.853040+0000","last_undegraded":"2023-12-07T10:18:00.853040+0000","last_fullsized":"2023-12-07T10:18:00.853040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:39:32.729502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.254528+0000","last_change":"2023-12-07T10:17:45.797573+0000","last_active":"2023-12-07T10:18:02.254528+0000","last_peered":"2023-12-07T10:18:02.254528+0000","last_clean":"2023-12-07T10:18:02.254528+0000","last_became_active":"2023-12-07T10:17:45.797119+0000","last_became_peered":"2023-12-07T10:17:45.797119+0000","last_unstale":"2023-12-07T10:18:02.254528+0000","last_undegraded":"2023-12-07T10:18:02.254528+0000","last_fullsized":"2023-12-07T10:18:02.254528+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:17:27.100470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:09.901985+0000","last_change":"2023-12-07T10:17:40.753596+0000","last_active":"2023-12-07T10:18:09.901985+0000","last_peered":"2023-12-07T10:18:09.901985+0000","last_clean":"2023-12-07T10:18:09.901985+0000","last_became_active":"2023-12-07T10:17:40.753251+0000","last_became_peered":"2023-12-07T10:17:40.753251+0000","last_unstale":"2023-12-07T10:18:09.901985+0000","last_undegraded":"2023-12-07T10:18:09.901985+0000","last_fullsized":"2023-12-07T10:18:09.901985+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:08:30.868794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.857325+0000","last_change":"2023-12-07T10:17:41.599689+0000","last_active":"2023-12-07T10:18:01.857325+0000","last_peered":"2023-12-07T10:18:01.857325+0000","last_clean":"2023-12-07T10:18:01.857325+0000","last_became_active":"2023-12-07T10:17:41.599358+0000","last_became_peered":"2023-12-07T10:17:41.599358+0000","last_unstale":"2023-12-07T10:18:01.857325+0000","last_undegraded":"2023-12-07T10:18:01.857325+0000","last_fullsized":"2023-12-07T10:18:01.857325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:44:19.742560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":2,"reported_epoch":92,"state":"peering","last_fresh":"2023-12-07T10:17:45.780383+0000","last_change":"2023-12-07T10:19:02.353800+0000","last_active":"2023-12-07T10:17:44.766631+0000","last_peered":"2023-12-07T10:17:44.766631+0000","last_clean":"2023-12-07T10:17:44.766631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-07T10:17:45.780383+0000","last_undegraded":"2023-12-07T10:17:45.780383+0000","last_fullsized":"2023-12-07T10:17:45.780383+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.756917+0000","last_change":"2023-12-07T10:17:40.758663+0000","last_active":"2023-12-07T10:18:12.756917+0000","last_peered":"2023-12-07T10:18:12.756917+0000","last_clean":"2023-12-07T10:18:12.756917+0000","last_became_active":"2023-12-07T10:17:40.753611+0000","last_became_peered":"2023-12-07T10:17:40.753611+0000","last_unstale":"2023-12-07T10:18:12.756917+0000","last_undegraded":"2023-12-07T10:18:12.756917+0000","last_fullsized":"2023-12-07T10:18:12.756917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:00:59.256468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.855712+0000","last_change":"2023-12-07T10:17:41.616546+0000","last_active":"2023-12-07T10:18:01.855712+0000","last_peered":"2023-12-07T10:18:01.855712+0000","last_clean":"2023-12-07T10:18:01.855712+0000","last_became_active":"2023-12-07T10:17:41.615780+0000","last_became_peered":"2023-12-07T10:17:41.615780+0000","last_unstale":"2023-12-07T10:18:01.855712+0000","last_undegraded":"2023-12-07T10:18:01.855712+0000","last_fullsized":"2023-12-07T10:18:01.855712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:22:32.455784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:13.223835+0000","last_change":"2023-12-07T10:17:45.797550+0000","last_active":"2023-12-07T10:18:13.223835+0000","last_peered":"2023-12-07T10:18:13.223835+0000","last_clean":"2023-12-07T10:18:13.223835+0000","last_became_active":"2023-12-07T10:17:45.797140+0000","last_became_peered":"2023-12-07T10:17:45.797140+0000","last_unstale":"2023-12-07T10:18:13.223835+0000","last_undegraded":"2023-12-07T10:18:13.223835+0000","last_fullsized":"2023-12-07T10:18:13.223835+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:53:44.984904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:13.048342+0000","last_change":"2023-12-07T10:17:40.748410+0000","last_active":"2023-12-07T10:18:13.048342+0000","last_peered":"2023-12-07T10:18:13.048342+0000","last_clean":"2023-12-07T10:18:13.048342+0000","last_became_active":"2023-12-07T10:17:40.748069+0000","last_became_peered":"2023-12-07T10:17:40.748069+0000","last_unstale":"2023-12-07T10:18:13.048342+0000","last_undegraded":"2023-12-07T10:18:13.048342+0000","last_fullsized":"2023-12-07T10:18:13.048342+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:51:48.278974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849910+0000","last_change":"2023-12-07T10:17:41.609248+0000","last_active":"2023-12-07T10:18:00.849910+0000","last_peered":"2023-12-07T10:18:00.849910+0000","last_clean":"2023-12-07T10:18:00.849910+0000","last_became_active":"2023-12-07T10:17:41.608943+0000","last_became_peered":"2023-12-07T10:17:41.608943+0000","last_unstale":"2023-12-07T10:18:00.849910+0000","last_undegraded":"2023-12-07T10:18:00.849910+0000","last_fullsized":"2023-12-07T10:18:00.849910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:29:32.322273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.254578+0000","last_change":"2023-12-07T10:17:45.794855+0000","last_active":"2023-12-07T10:18:02.254578+0000","last_peered":"2023-12-07T10:18:02.254578+0000","last_clean":"2023-12-07T10:18:02.254578+0000","last_became_active":"2023-12-07T10:17:45.794410+0000","last_became_peered":"2023-12-07T10:17:45.794410+0000","last_unstale":"2023-12-07T10:18:02.254578+0000","last_undegraded":"2023-12-07T10:18:02.254578+0000","last_fullsized":"2023-12-07T10:18:02.254578+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:57:25.118036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:10.731035+0000","last_change":"2023-12-07T10:17:40.750471+0000","last_active":"2023-12-07T10:18:10.731035+0000","last_peered":"2023-12-07T10:18:10.731035+0000","last_clean":"2023-12-07T10:18:10.731035+0000","last_became_active":"2023-12-07T10:17:40.750176+0000","last_became_peered":"2023-12-07T10:17:40.750176+0000","last_unstale":"2023-12-07T10:18:10.731035+0000","last_undegraded":"2023-12-07T10:18:10.731035+0000","last_fullsized":"2023-12-07T10:18:10.731035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:04:22.305343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856679+0000","last_change":"2023-12-07T10:17:41.614087+0000","last_active":"2023-12-07T10:18:00.856679+0000","last_peered":"2023-12-07T10:18:00.856679+0000","last_clean":"2023-12-07T10:18:00.856679+0000","last_became_active":"2023-12-07T10:17:41.607689+0000","last_became_peered":"2023-12-07T10:17:41.607689+0000","last_unstale":"2023-12-07T10:18:00.856679+0000","last_undegraded":"2023-12-07T10:18:00.856679+0000","last_fullsized":"2023-12-07T10:18:00.856679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:50:10.316085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"peering","last_fresh":"2023-12-07T10:18:01.989081+0000","last_change":"2023-12-07T10:19:01.811088+0000","last_active":"2023-12-07T10:19:01.811088+0000","last_peered":"2023-12-07T10:18:01.989081+0000","last_clean":"2023-12-07T10:18:01.989081+0000","last_became_active":"2023-12-07T10:17:45.790785+0000","last_became_peered":"2023-12-07T10:17:45.790785+0000","last_unstale":"2023-12-07T10:18:01.989081+0000","last_undegraded":"2023-12-07T10:18:01.989081+0000","last_fullsized":"2023-12-07T10:18:01.989081+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,6,7,8],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'5","reported_seq":72,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:57.831918+0000","last_change":"2023-12-07T10:17:40.753645+0000","last_active":"2023-12-07T10:18:57.831918+0000","last_peered":"2023-12-07T10:18:57.831918+0000","last_clean":"2023-12-07T10:18:57.831918+0000","last_became_active":"2023-12-07T10:17:40.753326+0000","last_became_peered":"2023-12-07T10:17:40.753326+0000","last_unstale":"2023-12-07T10:18:57.831918+0000","last_undegraded":"2023-12-07T10:18:57.831918+0000","last_fullsized":"2023-12-07T10:18:57.831918+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:43:03.617591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":48,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.857048+0000","last_change":"2023-12-07T10:17:41.607968+0000","last_active":"2023-12-07T10:18:00.857048+0000","last_peered":"2023-12-07T10:18:00.857048+0000","last_clean":"2023-12-07T10:18:00.857048+0000","last_became_active":"2023-12-07T10:17:41.607621+0000","last_became_peered":"2023-12-07T10:17:41.607621+0000","last_unstale":"2023-12-07T10:18:00.857048+0000","last_undegraded":"2023-12-07T10:18:00.857048+0000","last_fullsized":"2023-12-07T10:18:00.857048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:55:10.224738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852172+0000","last_change":"2023-12-07T10:17:45.793678+0000","last_active":"2023-12-07T10:18:00.852172+0000","last_peered":"2023-12-07T10:18:00.852172+0000","last_clean":"2023-12-07T10:18:00.852172+0000","last_became_active":"2023-12-07T10:17:45.793283+0000","last_became_peered":"2023-12-07T10:17:45.793283+0000","last_unstale":"2023-12-07T10:18:00.852172+0000","last_undegraded":"2023-12-07T10:18:00.852172+0000","last_fullsized":"2023-12-07T10:18:00.852172+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:33:56.501107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:11.749969+0000","last_change":"2023-12-07T10:17:40.762069+0000","last_active":"2023-12-07T10:18:11.749969+0000","last_peered":"2023-12-07T10:18:11.749969+0000","last_clean":"2023-12-07T10:18:11.749969+0000","last_became_active":"2023-12-07T10:17:40.761752+0000","last_became_peered":"2023-12-07T10:17:40.761752+0000","last_unstale":"2023-12-07T10:18:11.749969+0000","last_undegraded":"2023-12-07T10:18:11.749969+0000","last_fullsized":"2023-12-07T10:18:11.749969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:57:55.736144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856757+0000","last_change":"2023-12-07T10:17:41.607347+0000","last_active":"2023-12-07T10:18:00.856757+0000","last_peered":"2023-12-07T10:18:00.856757+0000","last_clean":"2023-12-07T10:18:00.856757+0000","last_became_active":"2023-12-07T10:17:41.606618+0000","last_became_peered":"2023-12-07T10:17:41.606618+0000","last_unstale":"2023-12-07T10:18:00.856757+0000","last_undegraded":"2023-12-07T10:18:00.856757+0000","last_fullsized":"2023-12-07T10:18:00.856757+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:53:34.748130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855194+0000","last_change":"2023-12-07T10:17:45.791459+0000","last_active":"2023-12-07T10:18:00.855194+0000","last_peered":"2023-12-07T10:18:00.855194+0000","last_clean":"2023-12-07T10:18:00.855194+0000","last_became_active":"2023-12-07T10:17:45.791070+0000","last_became_peered":"2023-12-07T10:17:45.791070+0000","last_unstale":"2023-12-07T10:18:00.855194+0000","last_undegraded":"2023-12-07T10:18:00.855194+0000","last_fullsized":"2023-12-07T10:18:00.855194+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:46:35.985990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.756703+0000","last_change":"2023-12-07T10:17:40.762726+0000","last_active":"2023-12-07T10:18:12.756703+0000","last_peered":"2023-12-07T10:18:12.756703+0000","last_clean":"2023-12-07T10:18:12.756703+0000","last_became_active":"2023-12-07T10:17:40.762378+0000","last_became_peered":"2023-12-07T10:17:40.762378+0000","last_unstale":"2023-12-07T10:18:12.756703+0000","last_undegraded":"2023-12-07T10:18:12.756703+0000","last_fullsized":"2023-12-07T10:18:12.756703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:42:06.193834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.856972+0000","last_change":"2023-12-07T10:17:41.602356+0000","last_active":"2023-12-07T10:18:00.856972+0000","last_peered":"2023-12-07T10:18:00.856972+0000","last_clean":"2023-12-07T10:18:00.856972+0000","last_became_active":"2023-12-07T10:17:41.602025+0000","last_became_peered":"2023-12-07T10:17:41.602025+0000","last_unstale":"2023-12-07T10:18:00.856972+0000","last_undegraded":"2023-12-07T10:18:00.856972+0000","last_fullsized":"2023-12-07T10:18:00.856972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:33:04.994510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.855751+0000","last_change":"2023-12-07T10:17:41.616766+0000","last_active":"2023-12-07T10:18:01.855751+0000","last_peered":"2023-12-07T10:18:01.855751+0000","last_clean":"2023-12-07T10:18:01.855751+0000","last_became_active":"2023-12-07T10:17:41.616221+0000","last_became_peered":"2023-12-07T10:17:41.616221+0000","last_unstale":"2023-12-07T10:18:01.855751+0000","last_undegraded":"2023-12-07T10:18:01.855751+0000","last_fullsized":"2023-12-07T10:18:01.855751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:16:00.393772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.850406+0000","last_change":"2023-12-07T10:17:41.613646+0000","last_active":"2023-12-07T10:18:00.850406+0000","last_peered":"2023-12-07T10:18:00.850406+0000","last_clean":"2023-12-07T10:18:00.850406+0000","last_became_active":"2023-12-07T10:17:41.613257+0000","last_became_peered":"2023-12-07T10:17:41.613257+0000","last_unstale":"2023-12-07T10:18:00.850406+0000","last_undegraded":"2023-12-07T10:18:00.850406+0000","last_fullsized":"2023-12-07T10:18:00.850406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:10:33.416955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849853+0000","last_change":"2023-12-07T10:17:41.620017+0000","last_active":"2023-12-07T10:18:00.849853+0000","last_peered":"2023-12-07T10:18:00.849853+0000","last_clean":"2023-12-07T10:18:00.849853+0000","last_became_active":"2023-12-07T10:17:41.619647+0000","last_became_peered":"2023-12-07T10:17:41.619647+0000","last_unstale":"2023-12-07T10:18:00.849853+0000","last_undegraded":"2023-12-07T10:18:00.849853+0000","last_fullsized":"2023-12-07T10:18:00.849853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:36:42.891160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.852495+0000","last_change":"2023-12-07T10:17:41.610803+0000","last_active":"2023-12-07T10:18:00.852495+0000","last_peered":"2023-12-07T10:18:00.852495+0000","last_clean":"2023-12-07T10:18:00.852495+0000","last_became_active":"2023-12-07T10:17:41.610388+0000","last_became_peered":"2023-12-07T10:17:41.610388+0000","last_unstale":"2023-12-07T10:18:00.852495+0000","last_undegraded":"2023-12-07T10:18:00.852495+0000","last_fullsized":"2023-12-07T10:18:00.852495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:33:25.787281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855345+0000","last_change":"2023-12-07T10:17:41.615916+0000","last_active":"2023-12-07T10:18:00.855345+0000","last_peered":"2023-12-07T10:18:00.855345+0000","last_clean":"2023-12-07T10:18:00.855345+0000","last_became_active":"2023-12-07T10:17:41.615591+0000","last_became_peered":"2023-12-07T10:17:41.615591+0000","last_unstale":"2023-12-07T10:18:00.855345+0000","last_undegraded":"2023-12-07T10:18:00.855345+0000","last_fullsized":"2023-12-07T10:18:00.855345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:40:45.541571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.856162+0000","last_change":"2023-12-07T10:17:41.614735+0000","last_active":"2023-12-07T10:18:01.856162+0000","last_peered":"2023-12-07T10:18:01.856162+0000","last_clean":"2023-12-07T10:18:01.856162+0000","last_became_active":"2023-12-07T10:17:41.614388+0000","last_became_peered":"2023-12-07T10:17:41.614388+0000","last_unstale":"2023-12-07T10:18:01.856162+0000","last_undegraded":"2023-12-07T10:18:01.856162+0000","last_fullsized":"2023-12-07T10:18:01.856162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:10:07.196841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.850893+0000","last_change":"2023-12-07T10:17:41.620059+0000","last_active":"2023-12-07T10:18:00.850893+0000","last_peered":"2023-12-07T10:18:00.850893+0000","last_clean":"2023-12-07T10:18:00.850893+0000","last_became_active":"2023-12-07T10:17:41.619770+0000","last_became_peered":"2023-12-07T10:17:41.619770+0000","last_unstale":"2023-12-07T10:18:00.850893+0000","last_undegraded":"2023-12-07T10:18:00.850893+0000","last_fullsized":"2023-12-07T10:18:00.850893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:13:43.421776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849472+0000","last_change":"2023-12-07T10:17:41.609720+0000","last_active":"2023-12-07T10:18:00.849472+0000","last_peered":"2023-12-07T10:18:00.849472+0000","last_clean":"2023-12-07T10:18:00.849472+0000","last_became_active":"2023-12-07T10:17:41.609417+0000","last_became_peered":"2023-12-07T10:17:41.609417+0000","last_unstale":"2023-12-07T10:18:00.849472+0000","last_undegraded":"2023-12-07T10:18:00.849472+0000","last_fullsized":"2023-12-07T10:18:00.849472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:48:47.768754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855496+0000","last_change":"2023-12-07T10:17:41.620952+0000","last_active":"2023-12-07T10:18:00.855496+0000","last_peered":"2023-12-07T10:18:00.855496+0000","last_clean":"2023-12-07T10:18:00.855496+0000","last_became_active":"2023-12-07T10:17:41.620560+0000","last_became_peered":"2023-12-07T10:17:41.620560+0000","last_unstale":"2023-12-07T10:18:00.855496+0000","last_undegraded":"2023-12-07T10:18:00.855496+0000","last_fullsized":"2023-12-07T10:18:00.855496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:51:37.699511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.849382+0000","last_change":"2023-12-07T10:17:41.611908+0000","last_active":"2023-12-07T10:18:00.849382+0000","last_peered":"2023-12-07T10:18:00.849382+0000","last_clean":"2023-12-07T10:18:00.849382+0000","last_became_active":"2023-12-07T10:17:41.610433+0000","last_became_peered":"2023-12-07T10:17:41.610433+0000","last_unstale":"2023-12-07T10:18:00.849382+0000","last_undegraded":"2023-12-07T10:18:00.849382+0000","last_fullsized":"2023-12-07T10:18:00.849382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:42:53.906221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.989268+0000","last_change":"2023-12-07T10:17:41.604116+0000","last_active":"2023-12-07T10:18:01.989268+0000","last_peered":"2023-12-07T10:18:01.989268+0000","last_clean":"2023-12-07T10:18:01.989268+0000","last_became_active":"2023-12-07T10:17:41.603880+0000","last_became_peered":"2023-12-07T10:17:41.603880+0000","last_unstale":"2023-12-07T10:18:01.989268+0000","last_undegraded":"2023-12-07T10:18:01.989268+0000","last_fullsized":"2023-12-07T10:18:01.989268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:56:27.343169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-12-07T10:18:00.855007+0000","last_change":"2023-12-07T10:17:41.610497+0000","last_active":"2023-12-07T10:18:00.855007+0000","last_peered":"2023-12-07T10:18:00.855007+0000","last_clean":"2023-12-07T10:18:00.855007+0000","last_became_active":"2023-12-07T10:17:41.609956+0000","last_became_peered":"2023-12-07T10:17:41.609956+0000","last_unstale":"2023-12-07T10:18:00.855007+0000","last_undegraded":"2023-12-07T10:18:00.855007+0000","last_fullsized":"2023-12-07T10:18:00.855007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:04:03.967377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.856279+0000","last_change":"2023-12-07T10:17:41.612284+0000","last_active":"2023-12-07T10:18:01.856279+0000","last_peered":"2023-12-07T10:18:01.856279+0000","last_clean":"2023-12-07T10:18:01.856279+0000","last_became_active":"2023-12-07T10:17:41.611917+0000","last_became_peered":"2023-12-07T10:17:41.611917+0000","last_unstale":"2023-12-07T10:18:01.856279+0000","last_undegraded":"2023-12-07T10:18:01.856279+0000","last_fullsized":"2023-12-07T10:18:01.856279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:02:27.355883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":81645,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":517,"num_read_kb":346,"num_write":243,"num_write_kb":152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":244935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383708,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":498136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.434,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.72299999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.64300000000000002}]},{"osd":1,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.62,"15min":0.60499999999999998},"min":{"1min":0.505,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.40899999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.76800000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.45900000000000002,"15min":0.45000000000000001},"min":{"1min":0.36899999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.46999999999999997,"15min":0.45700000000000002},"min":{"1min":0.376,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.38400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.67900000000000005,"15min":0.67200000000000004},"min":{"1min":0.48899999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90300000000000002,"5min":1.046,"15min":1.046},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.84699999999999998,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.56799999999999995,"15min":0.55700000000000005},"min":{"1min":0.40200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80700000000000005,"5min":1.034,"15min":1.034},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.59199999999999997,"15min":0.57999999999999996},"min":{"1min":0.51300000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92900000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.442}]},{"osd":6,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.46200000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90700000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.72399999999999998,"15min":0.71599999999999997},"min":{"1min":0.60399999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91300000000000003,"5min":1.363,"15min":1.363},"last":0.79800000000000004}]},{"osd":7,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.76100000000000001,"15min":0.751},"min":{"1min":0.63700000000000001,"5min":0.501,"15min":0.501},"max":{"1min":0.97899999999999998,"5min":1.25,"15min":1.25},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.70599999999999996,"15min":0.69599999999999995},"min":{"1min":0.51500000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95399999999999996,"5min":1.085,"15min":1.085},"last":0.60399999999999998}]},{"osd":8,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.70299999999999996}]},{"osd":9,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.78000000000000003,"15min":0.76600000000000001},"min":{"1min":0.69799999999999995,"5min":0.496,"15min":0.496},"max":{"1min":0.96799999999999997,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.78700000000000003,"15min":0.77500000000000002},"min":{"1min":0.71899999999999997,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.014,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.78000000000000003}]},{"osd":10,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.751,"15min":0.74099999999999999},"min":{"1min":0.59999999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.996,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.79300000000000004,"15min":0.78300000000000003},"min":{"1min":0.70999999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.999,"5min":1.411,"15min":1.411},"last":0.754}]}]},{"osd":10,"up_from":72,"seq":309237645344,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1716224,"data_stored":1149350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.3610000000000002,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.66000000000000003}]},{"osd":1,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.48599999999999999,"15min":0.49099999999999999},"min":{"1min":0.34100000000000003,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.627,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.49299999999999999,"15min":0.495},"min":{"1min":0.36299999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.53400000000000003}]},{"osd":2,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82699999999999996,"5min":1.008,"15min":1.008},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.42699999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81299999999999994,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.64400000000000002}]},{"osd":3,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.45500000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.78300000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.67900000000000005,"15min":0.68799999999999994},"min":{"1min":0.48199999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.79000000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.63}]},{"osd":4,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57599999999999996,"15min":0.58299999999999996},"min":{"1min":0.33300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71799999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58599999999999997,"15min":0.59099999999999997},"min":{"1min":0.40699999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.751,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51600000000000001}]},{"osd":5,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.95499999999999996,"5min":1.22,"15min":1.22},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72699999999999998,"15min":0.72799999999999998},"min":{"1min":0.49299999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93899999999999995,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.70799999999999996}]},{"osd":6,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.747,"15min":0.751},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.89200000000000002,"5min":1.054,"15min":1.054},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78900000000000003,"15min":0.78700000000000003},"min":{"1min":0.48499999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.74199999999999999}]},{"osd":7,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.63900000000000001}]},{"osd":8,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.79600000000000004,"15min":0.80500000000000005},"min":{"1min":0.53100000000000003,"5min":0.503,"15min":0.503},"max":{"1min":0.90700000000000003,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.83099999999999996,"15min":0.84199999999999997},"min":{"1min":0.629,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.91700000000000004,"5min":1.165,"15min":1.165},"last":0.84199999999999997}]},{"osd":9,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.83399999999999996,"15min":0.83999999999999997},"min":{"1min":0.622,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.98399999999999999,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.85199999999999998,"15min":0.86599999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.92400000000000004}]},{"osd":11,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.81699999999999995,"15min":0.82099999999999995},"min":{"1min":0.56899999999999995,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.95599999999999996,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81999999999999995,"15min":0.82199999999999995},"min":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.026,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.879}]}]},{"osd":9,"up_from":67,"seq":287762808869,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":502050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.60399999999999998,"15min":0.62},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.92600000000000005,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.626},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58099999999999996,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74299999999999999,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.80200000000000005}]},{"osd":1,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.69399999999999995,"15min":0.70699999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97999999999999998,"5min":1.899,"15min":1.899},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.77700000000000002,"15min":0.79300000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.94399999999999995,"5min":1.409,"15min":1.409},"last":0.66700000000000004}]},{"osd":2,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.80000000000000004,"15min":0.81899999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.91800000000000004,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.873,"15min":0.89800000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.92900000000000005,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":0.73699999999999999}]},{"osd":3,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.72799999999999998,"15min":0.746},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.878,"5min":1.47,"15min":1.47},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.81699999999999995,"15min":0.85499999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84799999999999998,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.69899999999999995}]},{"osd":4,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.80000000000000004,"15min":0.82999999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.85099999999999998,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.88,"15min":0.90200000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.109,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.88}]},{"osd":5,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.92800000000000005,"15min":0.95599999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.089,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.84699999999999998,"15min":0.876},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.76000000000000001,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":0.58099999999999996}]},{"osd":6,"last update":"Thu Dec 7 10:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":0.749}]},{"osd":7,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.93400000000000005,"15min":0.95799999999999996},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.0820000000000001,"5min":1.5,"15min":1.5},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":1.016,"15min":1.052},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.1519999999999999,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":0.96899999999999997}]},{"osd":8,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.95599999999999996,"15min":0.97399999999999998},"min":{"1min":0.63800000000000001,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0600000000000001,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.85699999999999998,"5min":1.0409999999999999,"15min":1.071},"min":{"1min":0.626,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.194,"5min":2.6539999999999999,"15min":2.6539999999999999},"last":0.76700000000000002}]},{"osd":10,"last update":"Thu Dec 7 10:18:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":1.02,"15min":1.052},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.2889999999999999,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.93899999999999995,"15min":0.96899999999999997},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.0189999999999999,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.82899999999999996}]},{"osd":11,"last update":"Thu Dec 7 10:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.78400000000000003}]}]},{"osd":8,"up_from":59,"seq":253403070507,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":507651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55900000000000005,"15min":0.56299999999999994},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.51200000000000001}]},{"osd":1,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.59199999999999997,"15min":0.57699999999999996},"min":{"1min":0.45100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86299999999999999,"5min":1,"15min":1},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.60299999999999998,"15min":0.58299999999999996},"min":{"1min":0.52800000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.70099999999999996}]},{"osd":2,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.46200000000000002,"15min":0.44400000000000001},"min":{"1min":0.317,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.76700000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.46300000000000002,"15min":0.438},"min":{"1min":0.29699999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.43099999999999999}]},{"osd":3,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59799999999999998,"15min":0.58599999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.83199999999999996,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.72199999999999998}]},{"osd":4,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.498,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.86899999999999999,"5min":0.876,"15min":0.876},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.64400000000000002,"15min":0.63400000000000001},"min":{"1min":0.54400000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89100000000000001,"5min":1.079,"15min":1.079},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.54100000000000004,"15min":0.51700000000000002},"min":{"1min":0.34799999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56799999999999995,"15min":0.55300000000000005},"min":{"1min":0.38200000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.84199999999999997,"5min":1.024,"15min":1.024},"last":0.747}]},{"osd":6,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.67400000000000004,"15min":0.65600000000000003},"min":{"1min":0.499,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.96099999999999997,"5min":1.046,"15min":1.046},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.68999999999999995,"15min":0.66400000000000003},"min":{"1min":0.48699999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.80000000000000004}]},{"osd":7,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.68700000000000006,"15min":0.65500000000000003},"min":{"1min":0.57999999999999996,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.69199999999999995,"15min":0.67500000000000004},"min":{"1min":0.58699999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.95599999999999996,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.81299999999999994}]},{"osd":9,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.71999999999999997,"15min":0.71499999999999997},"min":{"1min":0.53900000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.74399999999999999,"15min":0.73399999999999999},"min":{"1min":0.59899999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.93100000000000005}]},{"osd":10,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.74399999999999999,"15min":0.72699999999999998},"min":{"1min":0.626,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.74299999999999999,"15min":0.72799999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.82999999999999996}]},{"osd":11,"last update":"Thu Dec 7 10:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.88200000000000001}]}]},{"osd":7,"up_from":52,"seq":223338299439,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1646592,"data_stored":1104877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.58199999999999996,"15min":0.55200000000000005},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.5369999999999999,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56599999999999995,"15min":0.56000000000000005},"min":{"1min":0.39700000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.36899999999999999}]},{"osd":1,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.434,"15min":0.432},"min":{"1min":0.29699999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60499999999999998,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45600000000000002},"min":{"1min":0.254,"5min":0.253,"15min":0.253},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.434,"5min":0.314,"15min":0.314},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.504},{"interface":"front","average":{"1min":0.626,"5min":0.57899999999999996,"15min":0.57099999999999995},"min":{"1min":0.39600000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.66400000000000003}]},{"osd":3,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73499999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.45500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.79900000000000004,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51900000000000002,"15min":0.51600000000000001},"min":{"1min":0.315,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.748,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.52400000000000002,"15min":0.53300000000000003},"min":{"1min":0.33600000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.77400000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.46200000000000002}]},{"osd":5,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63900000000000001,"15min":0.64400000000000002},"min":{"1min":0.45600000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.79200000000000004,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.443,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.84999999999999998,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.81299999999999994}]},{"osd":6,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85699999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.68600000000000005,"15min":0.69199999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94999999999999996,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.58899999999999997}]},{"osd":8,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70099999999999996,"15min":0.69599999999999995},"min":{"1min":0.46200000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90600000000000003,"5min":1.012,"15min":1.012},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72399999999999998,"15min":0.72799999999999998},"min":{"1min":0.52900000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83399999999999996,"5min":1.026,"15min":1.026},"last":0.64800000000000002}]},{"osd":9,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.73599999999999999,"15min":0.74199999999999999},"min":{"1min":0.54600000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88500000000000001,"5min":1.075,"15min":1.075},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74199999999999999,"15min":0.75},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.92300000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.78600000000000003}]},{"osd":10,"last update":"Thu Dec 7 10:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.76300000000000001,"15min":0.76600000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.96099999999999997,"5min":1.03,"15min":1.03},"last":0.63},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73999999999999999,"15min":0.74199999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.89200000000000002,"5min":1.002,"15min":1.002},"last":0.754}]},{"osd":11,"last update":"Thu Dec 7 10:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.70199999999999996}]}]},{"osd":6,"up_from":47,"seq":201863462963,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":493458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47399999999999998},"min":{"1min":0.34499999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.63400000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.374},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.48299999999999998,"15min":0.47199999999999998},"min":{"1min":0.38300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61499999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.65200000000000002,"15min":0.65800000000000003},"min":{"1min":0.442,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80900000000000005,"5min":1.462,"15min":1.462},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68000000000000005,"15min":0.66400000000000003},"min":{"1min":0.54600000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.73199999999999998}]},{"osd":2,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.52700000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.72999999999999998,"15min":0.752},"min":{"1min":0.51000000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.95899999999999996,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.77200000000000002}]},{"osd":3,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.63600000000000001,"15min":0.65900000000000003},"min":{"1min":0.37,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78400000000000003,"5min":1.605,"15min":1.605},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.83899999999999997,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.69599999999999995}]},{"osd":4,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73999999999999999,"15min":0.75},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.032,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.747,"15min":0.74299999999999999},"min":{"1min":0.54700000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.56799999999999995}]},{"osd":5,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79200000000000004,"15min":0.80100000000000005},"min":{"1min":0.51100000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0840000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77800000000000002,"15min":0.78500000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95399999999999996,"5min":1.75,"15min":1.75},"last":0.85899999999999999}]},{"osd":7,"last update":"Thu Dec 7 10:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.78000000000000003,"15min":0.78500000000000003},"min":{"1min":0.47399999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0489999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.80400000000000005,"15min":0.78800000000000003},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.97999999999999998,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.84199999999999997}]},{"osd":8,"last update":"Thu Dec 7 10:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.85999999999999999,"15min":0.86899999999999999},"min":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.569,"5min":2.569,"15min":2.569},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.88600000000000001,"15min":0.89400000000000002},"min":{"1min":0.53400000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":3.1850000000000001,"5min":3.1850000000000001,"15min":3.1850000000000001},"last":0.80900000000000005}]},{"osd":9,"last update":"Thu Dec 7 10:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.86499999999999999,"15min":0.871},"min":{"1min":0.66800000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1160000000000001,"5min":1.179,"15min":1.179},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.84499999999999997,"15min":0.84899999999999998},"min":{"1min":0.65700000000000003,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.0600000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.88300000000000001}]},{"osd":10,"last update":"Thu Dec 7 10:18:34 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.85999999999999999,"15min":0.84999999999999998},"min":{"1min":0.58299999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.3980000000000001,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.88600000000000001,"15min":0.88300000000000001},"min":{"1min":0.73899999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.89400000000000002}]},{"osd":11,"last update":"Thu Dec 7 10:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.91000000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":558941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.627,"15min":0.56499999999999995},"min":{"1min":0.39200000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.92500000000000004,"5min":6.8120000000000003,"15min":6.8120000000000003},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.57299999999999995,"15min":0.54600000000000004},"min":{"1min":0.433,"5min":0.376,"15min":0.376},"max":{"1min":0.82999999999999996,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.67900000000000005}]},{"osd":2,"last update":"Thu Dec 7 10:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.59499999999999997,"15min":0.54300000000000004},"min":{"1min":0.46899999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.87,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.55800000000000005,"15min":0.51000000000000001},"min":{"1min":0.42999999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.58999999999999997}]},{"osd":3,"last update":"Thu Dec 7 10:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60499999999999998,"15min":0.56299999999999994},"min":{"1min":0.39400000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.82399999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.37,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.67200000000000004,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Dec 7 10:18:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.437,"15min":0.39700000000000002},"min":{"1min":0.35599999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.47899999999999998,"15min":0.42099999999999999},"min":{"1min":0.36599999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.77100000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.40400000000000003}]},{"osd":5,"last update":"Thu Dec 7 10:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.65700000000000003,"15min":0.58799999999999997},"min":{"1min":0.45000000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.98099999999999998,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.60599999999999998,"15min":0.56200000000000006},"min":{"1min":0.51600000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82099999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.627}]},{"osd":6,"last update":"Thu Dec 7 10:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.69299999999999995,"15min":0.66700000000000004},"min":{"1min":0.49199999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93700000000000006,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.65400000000000003,"15min":0.64200000000000002},"min":{"1min":0.52900000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.85699999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.66100000000000003}]},{"osd":7,"last update":"Thu Dec 7 10:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.54900000000000004,"15min":0.52600000000000002},"min":{"1min":0.38100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.438},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.67300000000000004,"15min":0.63400000000000001},"min":{"1min":0.46300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93799999999999994,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.64000000000000001}]},{"osd":8,"last update":"Thu Dec 7 10:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.77100000000000002,"15min":0.73499999999999999},"min":{"1min":0.70999999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.026,"5min":1.175,"15min":1.175},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67100000000000004,"15min":0.64500000000000002},"min":{"1min":0.55500000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86699999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.60899999999999999}]},{"osd":9,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.76300000000000001,"15min":0.754},"min":{"1min":0.58299999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.85399999999999998,"15min":0.84899999999999998},"min":{"1min":0.65700000000000003,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2050000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.76700000000000002}]},{"osd":10,"last update":"Thu Dec 7 10:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.73999999999999999,"15min":0.73699999999999999},"min":{"1min":0.63200000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.75700000000000001,"15min":0.751},"min":{"1min":0.61099999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.747}]},{"osd":11,"last update":"Thu Dec 7 10:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.871},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.78600000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":494914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.59899999999999998,"15min":0.57199999999999995},"min":{"1min":0.443,"5min":0.375,"15min":0.375},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.41899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.68600000000000005}]},{"osd":2,"last update":"Thu Dec 7 10:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.63400000000000001,"15min":0.59599999999999997},"min":{"1min":0.44500000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.3420000000000001,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.64600000000000002,"15min":0.60499999999999998},"min":{"1min":0.42299999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":3.4140000000000001,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Dec 7 10:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":3.484,"5min":3.484,"15min":3.484},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":3.1379999999999999,"5min":3.1379999999999999,"15min":3.1379999999999999},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Dec 7 10:18:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.59999999999999998,"15min":0.56599999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91700000000000004,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.754},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.623,"15min":0.60499999999999998},"min":{"1min":0.46000000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82799999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Dec 7 10:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.63900000000000001,"15min":0.624},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91000000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.626,"15min":0.624},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85399999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.628}]},{"osd":6,"last update":"Thu Dec 7 10:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.627,"15min":0.625},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79000000000000004,"5min":1.02,"15min":1.02},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64100000000000001,"15min":0.63400000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.44700000000000001}]},{"osd":7,"last update":"Thu Dec 7 10:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":3.0430000000000001,"5min":3.0430000000000001,"15min":3.0430000000000001},"last":0.71899999999999997}]},{"osd":8,"last update":"Thu Dec 7 10:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.81499999999999995,"15min":0.80500000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":3.75,"5min":3.75,"15min":3.75},"last":0.622},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.80600000000000005,"15min":0.81100000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.84299999999999997}]},{"osd":9,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.84899999999999998,"15min":0.86199999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0609999999999999,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.82999999999999996,"15min":0.83999999999999997},"min":{"1min":0.501,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.67400000000000004}]},{"osd":10,"last update":"Thu Dec 7 10:18:35 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.79900000000000004,"15min":0.77900000000000003},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":3.0979999999999999,"5min":3.0979999999999999,"15min":3.0979999999999999},"last":1.1259999999999999},{"interface":"front","average":{"1min":0.88,"5min":0.85199999999999998,"15min":0.84699999999999998},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.383,"5min":2.383,"15min":2.383},"last":1.2669999999999999}]},{"osd":11,"last update":"Thu Dec 7 10:18:55 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.84599999999999997,"15min":0.81799999999999995},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.7879999999999998,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.82399999999999995,"15min":0.80200000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.1749999999999998,"5min":3.1749999999999998,"15min":3.1749999999999998},"last":1.079}]}]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":569431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.51000000000000001},"min":{"1min":0.41599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55100000000000005,"15min":0.52000000000000002},"min":{"1min":0.35899999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.64600000000000002}]},{"osd":1,"last update":"Thu Dec 7 10:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.58099999999999996,"15min":0.52800000000000002},"min":{"1min":0.42199999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.56200000000000006,"15min":0.52300000000000002},"min":{"1min":0.39800000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79700000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.70899999999999996}]},{"osd":3,"last update":"Thu Dec 7 10:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.58799999999999997,"15min":0.55300000000000005},"min":{"1min":0.45000000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.58299999999999996,"15min":0.55700000000000005},"min":{"1min":0.40999999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Dec 7 10:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.79300000000000004}]},{"osd":5,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.47899999999999998,"15min":0.443},"min":{"1min":0.39400000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.77200000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.502},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.501,"15min":0.47199999999999998},"min":{"1min":0.379,"5min":0.307,"15min":0.307},"max":{"1min":0.81299999999999994,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.47899999999999998}]},{"osd":6,"last update":"Thu Dec 7 10:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.68100000000000005,"15min":0.67000000000000004},"min":{"1min":0.501,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.91800000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.436},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.67200000000000004,"15min":0.65400000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.74199999999999999}]},{"osd":7,"last update":"Thu Dec 7 10:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.63100000000000001}]},{"osd":8,"last update":"Thu Dec 7 10:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.625,"15min":0.60399999999999998},"min":{"1min":0.53400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.68000000000000005,"15min":0.66400000000000003},"min":{"1min":0.60799999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0229999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.67800000000000005}]},{"osd":9,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.78800000000000003,"15min":0.78400000000000003},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0389999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.80700000000000005}]},{"osd":10,"last update":"Thu Dec 7 10:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.76600000000000001,"15min":0.752},"min":{"1min":0.57999999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.77800000000000002,"15min":0.76600000000000001},"min":{"1min":0.57099999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.84299999999999997}]},{"osd":11,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.82099999999999995}]}]},{"osd":3,"up_from":26,"seq":111669149761,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":492835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.435,"15min":0.39800000000000002},"min":{"1min":0.309,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.624},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.44900000000000001,"15min":0.40699999999999997},"min":{"1min":0.33700000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.77600000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.51500000000000001}]},{"osd":1,"last update":"Thu Dec 7 10:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.65000000000000002,"15min":0.57399999999999995},"min":{"1min":0.54700000000000004,"5min":0.438,"15min":0.438},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61299999999999999,"15min":0.55300000000000005},"min":{"1min":0.496,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84499999999999997,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.89000000000000001}]},{"osd":2,"last update":"Thu Dec 7 10:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59899999999999998,"15min":0.58999999999999997},"min":{"1min":0.42899999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.754,"5min":1.087,"15min":1.087},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.65800000000000003,"15min":0.60999999999999999},"min":{"1min":0.48499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.016,"5min":1.177,"15min":1.177},"last":0.91500000000000004}]},{"osd":4,"last update":"Thu Dec 7 10:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.68600000000000005,"15min":0.62},"min":{"1min":0.503,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.018,"5min":1.208,"15min":1.208},"last":1.032},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63200000000000001,"15min":0.58999999999999997},"min":{"1min":0.46999999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83899999999999997,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.65700000000000003}]},{"osd":5,"last update":"Thu Dec 7 10:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.623,"15min":0.61099999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.82099999999999995,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.71399999999999997,"15min":0.69599999999999995},"min":{"1min":0.60499999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96199999999999997,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.0760000000000001}]},{"osd":6,"last update":"Thu Dec 7 10:18:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.46999999999999997,"5min":0.377,"15min":0.377},"max":{"1min":1.0369999999999999,"5min":1.282,"15min":1.282},"last":1.179},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60999999999999999,"15min":0.61799999999999999},"min":{"1min":0.39700000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71199999999999997,"5min":1.292,"15min":1.292},"last":0.86599999999999999}]},{"osd":7,"last update":"Thu Dec 7 10:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78600000000000003,"15min":0.76900000000000002},"min":{"1min":0.59599999999999997,"5min":0.498,"15min":0.498},"max":{"1min":1.0780000000000001,"5min":1.323,"15min":1.323},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.56799999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96999999999999997,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.93799999999999994}]},{"osd":8,"last update":"Thu Dec 7 10:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.872}]},{"osd":9,"last update":"Thu Dec 7 10:18:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.88,"15min":0.88600000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.224,"5min":1.242,"15min":1.242},"last":1.1479999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.80100000000000005,"15min":0.81000000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":1.0009999999999999}]},{"osd":10,"last update":"Thu Dec 7 10:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.78200000000000003,"15min":0.79100000000000004},"min":{"1min":0.59099999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.90500000000000003,"5min":1.335,"15min":1.335},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.878,"5min":0.92100000000000004,"15min":0.92800000000000005},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.1180000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":1.2190000000000001}]},{"osd":11,"last update":"Thu Dec 7 10:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":1.2509999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":1.056}]}]},{"osd":4,"up_from":34,"seq":146028888125,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":498032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.52100000000000002,"15min":0.49099999999999999},"min":{"1min":0.41499999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76500000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Dec 7 10:18:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.41099999999999998,"15min":0.39000000000000001},"min":{"1min":0.254,"5min":0.215,"15min":0.215},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.41499999999999998,"15min":0.38900000000000001},"min":{"1min":0.308,"5min":0.22,"15min":0.22},"max":{"1min":0.55200000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.53400000000000003}]},{"osd":2,"last update":"Thu Dec 7 10:18:17 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58899999999999997,"15min":0.56000000000000005},"min":{"1min":0.42699999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.875,"5min":1.323,"15min":1.323},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58499999999999996,"15min":0.54300000000000004},"min":{"1min":0.45700000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84299999999999997,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.59999999999999998}]},{"osd":3,"last update":"Thu Dec 7 10:18:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58099999999999996,"15min":0.54300000000000004},"min":{"1min":0.437,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.874,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.60899999999999999,"15min":0.56999999999999995},"min":{"1min":0.45700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91200000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.79400000000000004}]},{"osd":5,"last update":"Thu Dec 7 10:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.63100000000000001,"15min":0.59299999999999997},"min":{"1min":0.55000000000000004,"5min":0.374,"15min":0.374},"max":{"1min":1.0009999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.61699999999999999,"15min":0.58399999999999996},"min":{"1min":0.47599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82699999999999996,"5min":1.258,"15min":1.258},"last":0.63100000000000001}]},{"osd":6,"last update":"Thu Dec 7 10:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.41399999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64800000000000002,"15min":0.63600000000000001},"min":{"1min":0.501,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91900000000000004,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.69299999999999995}]},{"osd":7,"last update":"Thu Dec 7 10:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.55000000000000004,"15min":0.51800000000000002},"min":{"1min":0.38100000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76800000000000002,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.35599999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.84399999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.72699999999999998}]},{"osd":8,"last update":"Thu Dec 7 10:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.71299999999999997,"15min":0.67900000000000005},"min":{"1min":0.57099999999999995,"5min":0.432,"15min":0.432},"max":{"1min":0.90000000000000002,"5min":1.381,"15min":1.381},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71199999999999997,"15min":0.68200000000000005},"min":{"1min":0.59099999999999997,"5min":0.502,"15min":0.502},"max":{"1min":0.872,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.75}]},{"osd":9,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.76700000000000002,"15min":0.76900000000000002},"min":{"1min":0.57399999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.98799999999999999,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.77100000000000002,"15min":0.75800000000000001},"min":{"1min":0.58699999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.874}]},{"osd":10,"last update":"Thu Dec 7 10:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.60999999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.54600000000000004,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.80500000000000005}]},{"osd":11,"last update":"Thu Dec 7 10:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.83599999999999997}]}]},{"osd":5,"up_from":39,"seq":167503724600,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1658880,"data_stored":1100402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.54700000000000004,"15min":0.51700000000000002},"min":{"1min":0.377,"5min":0.318,"15min":0.318},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54700000000000004,"15min":0.52200000000000002},"min":{"1min":0.37,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79000000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54600000000000004}]},{"osd":1,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.56499999999999995,"15min":0.51700000000000002},"min":{"1min":0.505,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.59799999999999998,"15min":0.54400000000000004},"min":{"1min":0.52200000000000002,"5min":0.312,"15min":0.312},"max":{"1min":3.0070000000000001,"5min":3.0070000000000001,"15min":3.0070000000000001},"last":0.64200000000000002}]},{"osd":2,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.42399999999999999,"15min":0.38600000000000001},"min":{"1min":0.40200000000000002,"5min":0.246,"15min":0.246},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.45300000000000001,"15min":0.39900000000000002},"min":{"1min":0.35699999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.851,"5min":1.851,"15min":1.851},"last":0.41899999999999998}]},{"osd":3,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.57999999999999996,"15min":0.53900000000000003},"min":{"1min":0.43099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.59099999999999997,"15min":0.55600000000000005},"min":{"1min":0.40500000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.56899999999999995}]},{"osd":4,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.63100000000000001,"15min":0.57299999999999995},"min":{"1min":0.58199999999999996,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.621,"15min":0.57899999999999996},"min":{"1min":0.47699999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.61299999999999999}]},{"osd":6,"last update":"Thu Dec 7 10:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.61799999999999999,"15min":0.58599999999999997},"min":{"1min":0.42999999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.69499999999999995,"15min":0.63200000000000001},"min":{"1min":0.45000000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":5.9720000000000004,"5min":5.9720000000000004,"15min":5.9720000000000004},"last":0.78400000000000003}]},{"osd":7,"last update":"Thu Dec 7 10:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.66100000000000003,"15min":0.626},"min":{"1min":0.38400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.66600000000000004,"15min":0.63800000000000001},"min":{"1min":0.443,"5min":0.437,"15min":0.437},"max":{"1min":0.96999999999999997,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.628}]},{"osd":8,"last update":"Thu Dec 7 10:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.65000000000000002,"15min":0.628},"min":{"1min":0.45600000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.052,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.751},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.61899999999999999,"15min":0.59899999999999998},"min":{"1min":0.5,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92000000000000004,"5min":1.135,"15min":1.135},"last":0.68300000000000005}]},{"osd":9,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.78700000000000003,"15min":0.755},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":4.532,"5min":4.532,"15min":4.532},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76400000000000001,"15min":0.76300000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.83299999999999996}]},{"osd":10,"last update":"Thu Dec 7 10:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.84799999999999998,"15min":0.84099999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.7649999999999999,"5min":2.927,"15min":2.927},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.1519999999999999,"5min":0.85899999999999999,"15min":0.81000000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":5.7709999999999999,"5min":5.7709999999999999,"15min":5.7709999999999999},"last":0.76500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":76695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":13202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":13912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-07T10:19:04.398 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph pg dump --format=json 2023-12-07T10:19:05.171 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:19:05.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:04 smithi119 ceph-mon[97908]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:19:05.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:04 smithi159 ceph-mon[134726]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:19:05.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:04 smithi153 ceph-mon[133625]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:19:06.037 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:05 smithi119 ceph-mon[97908]: pgmap v325: 129 pgs: 4 peering, 125 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2023-12-07T10:19:06.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:05 smithi159 ceph-mon[134726]: pgmap v325: 129 pgs: 4 peering, 125 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2023-12-07T10:19:06.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:05 smithi153 ceph-mon[133625]: pgmap v325: 129 pgs: 4 peering, 125 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2023-12-07T10:19:06.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.697+0000 7ff8e5927700 1 -- 172.21.15.119:0/3237692233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8e0107730 msgr2=0x7ff8e0107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:06.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.697+0000 7ff8e5927700 1 --2- 172.21.15.119:0/3237692233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8e0107730 0x7ff8e0107b10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009990 tx=0x7ff8d0009ca0 comp rx=0 tx=0).stop 2023-12-07T10:19:06.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.697+0000 7ff8e5927700 1 -- 172.21.15.119:0/3237692233 shutdown_connections 2023-12-07T10:19:06.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.697+0000 7ff8e5927700 1 --2- 172.21.15.119:0/3237692233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8e0102080 0x7ff8e010e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:06.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.697+0000 7ff8e5927700 1 --2- 172.21.15.119:0/3237692233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8e0107730 0x7ff8e0107b10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:06.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.697+0000 7ff8e5927700 1 --2- 172.21.15.119:0/3237692233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8e01016c0 0x7ff8e0101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:06.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.697+0000 7ff8e5927700 1 -- 172.21.15.119:0/3237692233 >> 172.21.15.119:0/3237692233 conn(0x7ff8e0078800 msgr2=0x7ff8e0078c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:06.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.697+0000 7ff8e5927700 1 -- 172.21.15.119:0/3237692233 shutdown_connections 2023-12-07T10:19:06.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.697+0000 7ff8e5927700 1 -- 172.21.15.119:0/3237692233 wait complete. 2023-12-07T10:19:06.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.698+0000 7ff8e5927700 1 Processor -- start 2023-12-07T10:19:06.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.698+0000 7ff8e5927700 1 -- start start 2023-12-07T10:19:06.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.699+0000 7ff8e5927700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8e01016c0 0x7ff8e019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:06.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.699+0000 7ff8e5927700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8e0102080 0x7ff8e019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:06.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.699+0000 7ff8e5927700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8e0107730 0x7ff8e0199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:06.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.699+0000 7ff8e5927700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e019a060 con 0x7ff8e01016c0 2023-12-07T10:19:06.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.699+0000 7ff8e5927700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e019a1a0 con 0x7ff8e0102080 2023-12-07T10:19:06.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.699+0000 7ff8e5927700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e019a360 con 0x7ff8e0107730 2023-12-07T10:19:06.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.699+0000 7ff8deffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8e01016c0 0x7ff8e019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:06.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.699+0000 7ff8deffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8e01016c0 0x7ff8e019f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49034/0 (socket says 172.21.15.119:49034) 2023-12-07T10:19:06.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.699+0000 7ff8deffd700 1 -- 172.21.15.119:0/1706468757 learned_addr learned my addr 172.21.15.119:0/1706468757 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:06.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.699+0000 7ff8de7fc700 1 --2- 172.21.15.119:0/1706468757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8e0102080 0x7ff8e019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:06.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.699+0000 7ff8df7fe700 1 --2- 172.21.15.119:0/1706468757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8e0107730 0x7ff8e0199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:06.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.700+0000 7ff8deffd700 1 -- 172.21.15.119:0/1706468757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8e0107730 msgr2=0x7ff8e0199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:06.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.700+0000 7ff8deffd700 1 --2- 172.21.15.119:0/1706468757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8e0107730 0x7ff8e0199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:06.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.700+0000 7ff8deffd700 1 -- 172.21.15.119:0/1706468757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8e0102080 msgr2=0x7ff8e019fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:06.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.700+0000 7ff8deffd700 1 --2- 172.21.15.119:0/1706468757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8e0102080 0x7ff8e019fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:06.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.700+0000 7ff8deffd700 1 -- 172.21.15.119:0/1706468757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d00095f0 con 0x7ff8e01016c0 2023-12-07T10:19:06.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.700+0000 7ff8deffd700 1 --2- 172.21.15.119:0/1706468757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8e01016c0 0x7ff8e019f7f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff8d00095c0 tx=0x7ff8d000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:06.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.700+0000 7ff8e4925700 1 -- 172.21.15.119:0/1706468757 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d001d070 con 0x7ff8e01016c0 2023-12-07T10:19:06.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.700+0000 7ff8e4925700 1 -- 172.21.15.119:0/1706468757 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8d0005050 con 0x7ff8e01016c0 2023-12-07T10:19:06.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.700+0000 7ff8e5927700 1 -- 172.21.15.119:0/1706468757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e019a5e0 con 0x7ff8e01016c0 2023-12-07T10:19:06.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.700+0000 7ff8e4925700 1 -- 172.21.15.119:0/1706468757 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0004a70 con 0x7ff8e01016c0 2023-12-07T10:19:06.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.700+0000 7ff8e5927700 1 -- 172.21.15.119:0/1706468757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e01a6890 con 0x7ff8e01016c0 2023-12-07T10:19:06.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.701+0000 7ff8e5927700 1 -- 172.21.15.119:0/1706468757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e0104980 con 0x7ff8e01016c0 2023-12-07T10:19:06.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.703+0000 7ff8e4925700 1 -- 172.21.15.119:0/1706468757 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7ff8d00051c0 con 0x7ff8e01016c0 2023-12-07T10:19:06.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.705+0000 7ff8e4925700 1 --2- 172.21.15.119:0/1706468757 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff8c80a1ba0 0x7ff8c80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:06.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.705+0000 7ff8e4925700 1 -- 172.21.15.119:0/1706468757 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7ff8d00ce8b0 con 0x7ff8e01016c0 2023-12-07T10:19:06.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.705+0000 7ff8de7fc700 1 --2- 172.21.15.119:0/1706468757 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff8c80a1ba0 0x7ff8c80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:06.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.706+0000 7ff8de7fc700 1 --2- 172.21.15.119:0/1706468757 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff8c80a1ba0 0x7ff8c80a4060 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0079100 tx=0x7ff8d400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:06.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.709+0000 7ff8e4925700 1 -- 172.21.15.119:0/1706468757 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff8d0097830 con 0x7ff8e01016c0 2023-12-07T10:19:06.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.908+0000 7ff8e5927700 1 -- 172.21.15.119:0/1706468757 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff8e019aee0 con 0x7ff8c80a1ba0 2023-12-07T10:19:06.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.920+0000 7ff8e4925700 1 -- 172.21.15.119:0/1706468757 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426690 (secure 0 0 0) 0x7ff8e019aee0 con 0x7ff8c80a1ba0 2023-12-07T10:19:06.928 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:19:06.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.927+0000 7ff8e5927700 1 -- 172.21.15.119:0/1706468757 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff8c80a1ba0 msgr2=0x7ff8c80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:06.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.927+0000 7ff8e5927700 1 --2- 172.21.15.119:0/1706468757 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff8c80a1ba0 0x7ff8c80a4060 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0079100 tx=0x7ff8d400b410 comp rx=0 tx=0).stop 2023-12-07T10:19:06.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.927+0000 7ff8e5927700 1 -- 172.21.15.119:0/1706468757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8e01016c0 msgr2=0x7ff8e019f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:06.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.927+0000 7ff8e5927700 1 --2- 172.21.15.119:0/1706468757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8e01016c0 0x7ff8e019f7f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff8d00095c0 tx=0x7ff8d000f690 comp rx=0 tx=0).stop 2023-12-07T10:19:06.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.927+0000 7ff8e5927700 1 -- 172.21.15.119:0/1706468757 shutdown_connections 2023-12-07T10:19:06.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.927+0000 7ff8e5927700 1 --2- 172.21.15.119:0/1706468757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8e0107730 0x7ff8e0199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:06.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.927+0000 7ff8e5927700 1 --2- 172.21.15.119:0/1706468757 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff8c80a1ba0 0x7ff8c80a4060 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:06.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.927+0000 7ff8e5927700 1 --2- 172.21.15.119:0/1706468757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8e01016c0 0x7ff8e019f7f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:06.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.927+0000 7ff8e5927700 1 --2- 172.21.15.119:0/1706468757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8e0102080 0x7ff8e019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:06.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.927+0000 7ff8e5927700 1 -- 172.21.15.119:0/1706468757 >> 172.21.15.119:0/1706468757 conn(0x7ff8e0078800 msgr2=0x7ff8e010cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:06.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.927+0000 7ff8e5927700 1 -- 172.21.15.119:0/1706468757 shutdown_connections 2023-12-07T10:19:06.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:06.927+0000 7ff8e5927700 1 -- 172.21.15.119:0/1706468757 wait complete. 2023-12-07T10:19:06.932 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-12-07T10:19:07.373 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2023-12-07T10:19:06.691204+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":672259,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":698,"num_read_kb":490,"num_write":426,"num_write_kb":3092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":420,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338648,"kb_used_data":14412,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603176,"statfs":{"total":1151940427776,"available":1151593652224,"internally_reserved":0,"allocated":14757888,"data_stored":8110898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":331855161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002180"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.856154+0000","last_change":"2023-12-07T10:17:41.614206+0000","last_active":"2023-12-07T10:18:01.856154+0000","last_peered":"2023-12-07T10:18:01.856154+0000","last_clean":"2023-12-07T10:18:01.856154+0000","last_became_active":"2023-12-07T10:17:41.613553+0000","last_became_peered":"2023-12-07T10:17:41.613553+0000","last_unstale":"2023-12-07T10:18:01.856154+0000","last_undegraded":"2023-12-07T10:18:01.856154+0000","last_fullsized":"2023-12-07T10:18:01.856154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:10:22.198414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803801+0000","last_change":"2023-12-07T10:17:41.616917+0000","last_active":"2023-12-07T10:19:01.803801+0000","last_peered":"2023-12-07T10:19:01.803801+0000","last_clean":"2023-12-07T10:19:01.803801+0000","last_became_active":"2023-12-07T10:17:41.615345+0000","last_became_peered":"2023-12-07T10:17:41.615345+0000","last_unstale":"2023-12-07T10:19:01.803801+0000","last_undegraded":"2023-12-07T10:19:01.803801+0000","last_fullsized":"2023-12-07T10:19:01.803801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:22:25.566738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803844+0000","last_change":"2023-12-07T10:17:41.619767+0000","last_active":"2023-12-07T10:19:01.803844+0000","last_peered":"2023-12-07T10:19:01.803844+0000","last_clean":"2023-12-07T10:19:01.803844+0000","last_became_active":"2023-12-07T10:17:41.619362+0000","last_became_peered":"2023-12-07T10:17:41.619362+0000","last_unstale":"2023-12-07T10:19:01.803844+0000","last_undegraded":"2023-12-07T10:19:01.803844+0000","last_fullsized":"2023-12-07T10:19:01.803844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:12:55.458520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807439+0000","last_change":"2023-12-07T10:18:01.861930+0000","last_active":"2023-12-07T10:19:01.807439+0000","last_peered":"2023-12-07T10:19:01.807439+0000","last_clean":"2023-12-07T10:19:01.807439+0000","last_became_active":"2023-12-07T10:18:01.861570+0000","last_became_peered":"2023-12-07T10:18:01.861570+0000","last_unstale":"2023-12-07T10:19:01.807439+0000","last_undegraded":"2023-12-07T10:19:01.807439+0000","last_fullsized":"2023-12-07T10:19:01.807439+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:29:45.234261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.858385+0000","last_change":"2023-12-07T10:17:41.613890+0000","last_active":"2023-12-07T10:18:01.858385+0000","last_peered":"2023-12-07T10:18:01.858385+0000","last_clean":"2023-12-07T10:18:01.858385+0000","last_became_active":"2023-12-07T10:17:41.613073+0000","last_became_peered":"2023-12-07T10:17:41.613073+0000","last_unstale":"2023-12-07T10:18:01.858385+0000","last_undegraded":"2023-12-07T10:18:01.858385+0000","last_fullsized":"2023-12-07T10:18:01.858385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:17:14.339002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802869+0000","last_change":"2023-12-07T10:17:41.609585+0000","last_active":"2023-12-07T10:19:01.802869+0000","last_peered":"2023-12-07T10:19:01.802869+0000","last_clean":"2023-12-07T10:19:01.802869+0000","last_became_active":"2023-12-07T10:17:41.609256+0000","last_became_peered":"2023-12-07T10:17:41.609256+0000","last_unstale":"2023-12-07T10:19:01.802869+0000","last_undegraded":"2023-12-07T10:19:01.802869+0000","last_fullsized":"2023-12-07T10:19:01.802869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:20:15.103208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811012+0000","last_change":"2023-12-07T10:17:41.606138+0000","last_active":"2023-12-07T10:19:01.811012+0000","last_peered":"2023-12-07T10:19:01.811012+0000","last_clean":"2023-12-07T10:19:01.811012+0000","last_became_active":"2023-12-07T10:17:41.605818+0000","last_became_peered":"2023-12-07T10:17:41.605818+0000","last_unstale":"2023-12-07T10:19:01.811012+0000","last_undegraded":"2023-12-07T10:19:01.811012+0000","last_fullsized":"2023-12-07T10:19:01.811012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:38:39.244077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.806921+0000","last_change":"2023-12-07T10:18:02.255959+0000","last_active":"2023-12-07T10:19:01.806921+0000","last_peered":"2023-12-07T10:19:01.806921+0000","last_clean":"2023-12-07T10:19:01.806921+0000","last_became_active":"2023-12-07T10:18:02.255560+0000","last_became_peered":"2023-12-07T10:18:02.255560+0000","last_unstale":"2023-12-07T10:19:01.806921+0000","last_undegraded":"2023-12-07T10:19:01.806921+0000","last_fullsized":"2023-12-07T10:19:01.806921+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:05:49.065538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.989614+0000","last_change":"2023-12-07T10:17:41.610225+0000","last_active":"2023-12-07T10:18:01.989614+0000","last_peered":"2023-12-07T10:18:01.989614+0000","last_clean":"2023-12-07T10:18:01.989614+0000","last_became_active":"2023-12-07T10:17:41.609875+0000","last_became_peered":"2023-12-07T10:17:41.609875+0000","last_unstale":"2023-12-07T10:18:01.989614+0000","last_undegraded":"2023-12-07T10:18:01.989614+0000","last_fullsized":"2023-12-07T10:18:01.989614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:00:43.136176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803348+0000","last_change":"2023-12-07T10:17:41.617335+0000","last_active":"2023-12-07T10:19:01.803348+0000","last_peered":"2023-12-07T10:19:01.803348+0000","last_clean":"2023-12-07T10:19:01.803348+0000","last_became_active":"2023-12-07T10:17:41.617005+0000","last_became_peered":"2023-12-07T10:17:41.617005+0000","last_unstale":"2023-12-07T10:19:01.803348+0000","last_undegraded":"2023-12-07T10:19:01.803348+0000","last_fullsized":"2023-12-07T10:19:01.803348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:43:36.855465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811670+0000","last_change":"2023-12-07T10:17:41.608104+0000","last_active":"2023-12-07T10:19:01.811670+0000","last_peered":"2023-12-07T10:19:01.811670+0000","last_clean":"2023-12-07T10:19:01.811670+0000","last_became_active":"2023-12-07T10:17:41.607769+0000","last_became_peered":"2023-12-07T10:17:41.607769+0000","last_unstale":"2023-12-07T10:19:01.811670+0000","last_undegraded":"2023-12-07T10:19:01.811670+0000","last_fullsized":"2023-12-07T10:19:01.811670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:47:04.853044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.254656+0000","last_change":"2023-12-07T10:17:41.611585+0000","last_active":"2023-12-07T10:18:02.254656+0000","last_peered":"2023-12-07T10:18:02.254656+0000","last_clean":"2023-12-07T10:18:02.254656+0000","last_became_active":"2023-12-07T10:17:41.611202+0000","last_became_peered":"2023-12-07T10:17:41.611202+0000","last_unstale":"2023-12-07T10:18:02.254656+0000","last_undegraded":"2023-12-07T10:18:02.254656+0000","last_fullsized":"2023-12-07T10:18:02.254656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:17:01.649424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813519+0000","last_change":"2023-12-07T10:17:41.611116+0000","last_active":"2023-12-07T10:19:01.813519+0000","last_peered":"2023-12-07T10:19:01.813519+0000","last_clean":"2023-12-07T10:19:01.813519+0000","last_became_active":"2023-12-07T10:17:41.610791+0000","last_became_peered":"2023-12-07T10:17:41.610791+0000","last_unstale":"2023-12-07T10:19:01.813519+0000","last_undegraded":"2023-12-07T10:19:01.813519+0000","last_fullsized":"2023-12-07T10:19:01.813519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:48:08.822235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.804022+0000","last_change":"2023-12-07T10:17:41.617447+0000","last_active":"2023-12-07T10:19:01.804022+0000","last_peered":"2023-12-07T10:19:01.804022+0000","last_clean":"2023-12-07T10:19:01.804022+0000","last_became_active":"2023-12-07T10:17:41.612638+0000","last_became_peered":"2023-12-07T10:17:41.612638+0000","last_unstale":"2023-12-07T10:19:01.804022+0000","last_undegraded":"2023-12-07T10:19:01.804022+0000","last_fullsized":"2023-12-07T10:19:01.804022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:11:30.384367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803409+0000","last_change":"2023-12-07T10:17:41.611607+0000","last_active":"2023-12-07T10:19:01.803409+0000","last_peered":"2023-12-07T10:19:01.803409+0000","last_clean":"2023-12-07T10:19:01.803409+0000","last_became_active":"2023-12-07T10:17:41.611331+0000","last_became_peered":"2023-12-07T10:17:41.611331+0000","last_unstale":"2023-12-07T10:19:01.803409+0000","last_undegraded":"2023-12-07T10:19:01.803409+0000","last_fullsized":"2023-12-07T10:19:01.803409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:17:53.049922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.988971+0000","last_change":"2023-12-07T10:17:41.610528+0000","last_active":"2023-12-07T10:18:01.988971+0000","last_peered":"2023-12-07T10:18:01.988971+0000","last_clean":"2023-12-07T10:18:01.988971+0000","last_became_active":"2023-12-07T10:17:41.610216+0000","last_became_peered":"2023-12-07T10:17:41.610216+0000","last_unstale":"2023-12-07T10:18:01.988971+0000","last_undegraded":"2023-12-07T10:18:01.988971+0000","last_fullsized":"2023-12-07T10:18:01.988971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:11:36.749537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.856068+0000","last_change":"2023-12-07T10:17:41.613967+0000","last_active":"2023-12-07T10:18:01.856068+0000","last_peered":"2023-12-07T10:18:01.856068+0000","last_clean":"2023-12-07T10:18:01.856068+0000","last_became_active":"2023-12-07T10:17:41.613116+0000","last_became_peered":"2023-12-07T10:17:41.613116+0000","last_unstale":"2023-12-07T10:18:01.856068+0000","last_undegraded":"2023-12-07T10:18:01.856068+0000","last_fullsized":"2023-12-07T10:18:01.856068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:46:32.444025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807559+0000","last_change":"2023-12-07T10:17:41.605478+0000","last_active":"2023-12-07T10:19:01.807559+0000","last_peered":"2023-12-07T10:19:01.807559+0000","last_clean":"2023-12-07T10:19:01.807559+0000","last_became_active":"2023-12-07T10:17:41.605110+0000","last_became_peered":"2023-12-07T10:17:41.605110+0000","last_unstale":"2023-12-07T10:19:01.807559+0000","last_undegraded":"2023-12-07T10:19:01.807559+0000","last_fullsized":"2023-12-07T10:19:01.807559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:53:56.410196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810309+0000","last_change":"2023-12-07T10:18:02.256377+0000","last_active":"2023-12-07T10:19:01.810309+0000","last_peered":"2023-12-07T10:19:01.810309+0000","last_clean":"2023-12-07T10:19:01.810309+0000","last_became_active":"2023-12-07T10:18:02.255987+0000","last_became_peered":"2023-12-07T10:18:02.255987+0000","last_unstale":"2023-12-07T10:19:01.810309+0000","last_undegraded":"2023-12-07T10:19:01.810309+0000","last_fullsized":"2023-12-07T10:19:01.810309+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:23:34.909187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.809857+0000","last_change":"2023-12-07T10:17:45.799925+0000","last_active":"2023-12-07T10:19:01.809857+0000","last_peered":"2023-12-07T10:19:01.809857+0000","last_clean":"2023-12-07T10:19:01.809857+0000","last_became_active":"2023-12-07T10:17:45.799505+0000","last_became_peered":"2023-12-07T10:17:45.799505+0000","last_unstale":"2023-12-07T10:19:01.809857+0000","last_undegraded":"2023-12-07T10:19:01.809857+0000","last_fullsized":"2023-12-07T10:19:01.809857+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:52:51.470333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'13","reported_seq":86,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.838801+0000","last_change":"2023-12-07T10:17:40.755066+0000","last_active":"2023-12-07T10:19:02.838801+0000","last_peered":"2023-12-07T10:19:02.838801+0000","last_clean":"2023-12-07T10:19:02.838801+0000","last_became_active":"2023-12-07T10:17:40.754709+0000","last_became_peered":"2023-12-07T10:17:40.754709+0000","last_unstale":"2023-12-07T10:19:02.838801+0000","last_undegraded":"2023-12-07T10:19:02.838801+0000","last_fullsized":"2023-12-07T10:19:02.838801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:18:31.148592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803641+0000","last_change":"2023-12-07T10:17:41.619107+0000","last_active":"2023-12-07T10:19:01.803641+0000","last_peered":"2023-12-07T10:19:01.803641+0000","last_clean":"2023-12-07T10:19:01.803641+0000","last_became_active":"2023-12-07T10:17:41.617057+0000","last_became_peered":"2023-12-07T10:17:41.617057+0000","last_unstale":"2023-12-07T10:19:01.803641+0000","last_undegraded":"2023-12-07T10:19:01.803641+0000","last_fullsized":"2023-12-07T10:19:01.803641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:53:16.402933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.255056+0000","last_change":"2023-12-07T10:17:45.797582+0000","last_active":"2023-12-07T10:18:02.255056+0000","last_peered":"2023-12-07T10:18:02.255056+0000","last_clean":"2023-12-07T10:18:02.255056+0000","last_became_active":"2023-12-07T10:17:45.796358+0000","last_became_peered":"2023-12-07T10:17:45.796358+0000","last_unstale":"2023-12-07T10:18:02.255056+0000","last_undegraded":"2023-12-07T10:18:02.255056+0000","last_fullsized":"2023-12-07T10:18:02.255056+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:53:19.551459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803712+0000","last_change":"2023-12-07T10:17:40.762155+0000","last_active":"2023-12-07T10:19:01.803712+0000","last_peered":"2023-12-07T10:19:01.803712+0000","last_clean":"2023-12-07T10:19:01.803712+0000","last_became_active":"2023-12-07T10:17:40.761776+0000","last_became_peered":"2023-12-07T10:17:40.761776+0000","last_unstale":"2023-12-07T10:19:01.803712+0000","last_undegraded":"2023-12-07T10:19:01.803712+0000","last_fullsized":"2023-12-07T10:19:01.803712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:15:06.096331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.805463+0000","last_change":"2023-12-07T10:17:41.614999+0000","last_active":"2023-12-07T10:19:01.805463+0000","last_peered":"2023-12-07T10:19:01.805463+0000","last_clean":"2023-12-07T10:19:01.805463+0000","last_became_active":"2023-12-07T10:17:41.614673+0000","last_became_peered":"2023-12-07T10:17:41.614673+0000","last_unstale":"2023-12-07T10:19:01.805463+0000","last_undegraded":"2023-12-07T10:19:01.805463+0000","last_fullsized":"2023-12-07T10:19:01.805463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:34:45.378843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"peering","last_fresh":"2023-12-07T10:18:02.254326+0000","last_change":"2023-12-07T10:19:01.805042+0000","last_active":"2023-12-07T10:19:01.805042+0000","last_peered":"2023-12-07T10:18:02.254326+0000","last_clean":"2023-12-07T10:18:02.254326+0000","last_became_active":"2023-12-07T10:17:45.799072+0000","last_became_peered":"2023-12-07T10:17:45.799072+0000","last_unstale":"2023-12-07T10:18:02.254326+0000","last_undegraded":"2023-12-07T10:18:02.254326+0000","last_fullsized":"2023-12-07T10:18:02.254326+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5,9,10],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:15.647183+0000","last_change":"2023-12-07T10:18:01.993128+0000","last_active":"2023-12-07T10:18:15.647183+0000","last_peered":"2023-12-07T10:18:15.647183+0000","last_clean":"2023-12-07T10:18:15.647183+0000","last_became_active":"2023-12-07T10:18:01.990244+0000","last_became_peered":"2023-12-07T10:18:01.990244+0000","last_unstale":"2023-12-07T10:18:15.647183+0000","last_undegraded":"2023-12-07T10:18:15.647183+0000","last_fullsized":"2023-12-07T10:18:15.647183+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:42:53.778363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.989335+0000","last_change":"2023-12-07T10:17:41.612903+0000","last_active":"2023-12-07T10:18:01.989335+0000","last_peered":"2023-12-07T10:18:01.989335+0000","last_clean":"2023-12-07T10:18:01.989335+0000","last_became_active":"2023-12-07T10:17:41.611968+0000","last_became_peered":"2023-12-07T10:17:41.611968+0000","last_unstale":"2023-12-07T10:18:01.989335+0000","last_undegraded":"2023-12-07T10:18:01.989335+0000","last_fullsized":"2023-12-07T10:18:01.989335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:34:08.442828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.254202+0000","last_change":"2023-12-07T10:17:45.803858+0000","last_active":"2023-12-07T10:18:02.254202+0000","last_peered":"2023-12-07T10:18:02.254202+0000","last_clean":"2023-12-07T10:18:02.254202+0000","last_became_active":"2023-12-07T10:17:45.803481+0000","last_became_peered":"2023-12-07T10:17:45.803481+0000","last_unstale":"2023-12-07T10:18:02.254202+0000","last_undegraded":"2023-12-07T10:18:02.254202+0000","last_fullsized":"2023-12-07T10:18:02.254202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:08:03.728554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:13.774156+0000","last_change":"2023-12-07T10:17:40.751866+0000","last_active":"2023-12-07T10:18:13.774156+0000","last_peered":"2023-12-07T10:18:13.774156+0000","last_clean":"2023-12-07T10:18:13.774156+0000","last_became_active":"2023-12-07T10:17:40.751492+0000","last_became_peered":"2023-12-07T10:17:40.751492+0000","last_unstale":"2023-12-07T10:18:13.774156+0000","last_undegraded":"2023-12-07T10:18:13.774156+0000","last_fullsized":"2023-12-07T10:18:13.774156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:42:53.839651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808255+0000","last_change":"2023-12-07T10:17:41.611326+0000","last_active":"2023-12-07T10:19:01.808255+0000","last_peered":"2023-12-07T10:19:01.808255+0000","last_clean":"2023-12-07T10:19:01.808255+0000","last_became_active":"2023-12-07T10:17:41.610863+0000","last_became_peered":"2023-12-07T10:17:41.610863+0000","last_unstale":"2023-12-07T10:19:01.808255+0000","last_undegraded":"2023-12-07T10:19:01.808255+0000","last_fullsized":"2023-12-07T10:19:01.808255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:07:55.801356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810238+0000","last_change":"2023-12-07T10:17:45.791409+0000","last_active":"2023-12-07T10:19:01.810238+0000","last_peered":"2023-12-07T10:19:01.810238+0000","last_clean":"2023-12-07T10:19:01.810238+0000","last_became_active":"2023-12-07T10:17:45.791022+0000","last_became_peered":"2023-12-07T10:17:45.791022+0000","last_unstale":"2023-12-07T10:19:01.810238+0000","last_undegraded":"2023-12-07T10:19:01.810238+0000","last_fullsized":"2023-12-07T10:19:01.810238+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:57:14.617409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'9","reported_seq":88,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.818392+0000","last_change":"2023-12-07T10:17:40.745318+0000","last_active":"2023-12-07T10:19:02.818392+0000","last_peered":"2023-12-07T10:19:02.818392+0000","last_clean":"2023-12-07T10:19:02.818392+0000","last_became_active":"2023-12-07T10:17:40.745008+0000","last_became_peered":"2023-12-07T10:17:40.745008+0000","last_unstale":"2023-12-07T10:19:02.818392+0000","last_undegraded":"2023-12-07T10:19:02.818392+0000","last_fullsized":"2023-12-07T10:19:02.818392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:42:28.503421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11004,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":14,"num_write":6,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.858287+0000","last_change":"2023-12-07T10:17:41.605121+0000","last_active":"2023-12-07T10:18:01.858287+0000","last_peered":"2023-12-07T10:18:01.858287+0000","last_clean":"2023-12-07T10:18:01.858287+0000","last_became_active":"2023-12-07T10:17:41.604770+0000","last_became_peered":"2023-12-07T10:17:41.604770+0000","last_unstale":"2023-12-07T10:18:01.858287+0000","last_undegraded":"2023-12-07T10:18:01.858287+0000","last_fullsized":"2023-12-07T10:18:01.858287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:51:02.151155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810286+0000","last_change":"2023-12-07T10:17:45.809039+0000","last_active":"2023-12-07T10:19:01.810286+0000","last_peered":"2023-12-07T10:19:01.810286+0000","last_clean":"2023-12-07T10:19:01.810286+0000","last_became_active":"2023-12-07T10:17:45.808622+0000","last_became_peered":"2023-12-07T10:17:45.808622+0000","last_unstale":"2023-12-07T10:19:01.810286+0000","last_undegraded":"2023-12-07T10:19:01.810286+0000","last_fullsized":"2023-12-07T10:19:01.810286+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:16:02.835931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802888+0000","last_change":"2023-12-07T10:17:40.757876+0000","last_active":"2023-12-07T10:19:01.802888+0000","last_peered":"2023-12-07T10:19:01.802888+0000","last_clean":"2023-12-07T10:19:01.802888+0000","last_became_active":"2023-12-07T10:17:40.757613+0000","last_became_peered":"2023-12-07T10:17:40.757613+0000","last_unstale":"2023-12-07T10:19:01.802888+0000","last_undegraded":"2023-12-07T10:19:01.802888+0000","last_fullsized":"2023-12-07T10:19:01.802888+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:30:40.423092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808115+0000","last_change":"2023-12-07T10:17:41.611575+0000","last_active":"2023-12-07T10:19:01.808115+0000","last_peered":"2023-12-07T10:19:01.808115+0000","last_clean":"2023-12-07T10:19:01.808115+0000","last_became_active":"2023-12-07T10:17:41.611042+0000","last_became_peered":"2023-12-07T10:17:41.611042+0000","last_unstale":"2023-12-07T10:19:01.808115+0000","last_undegraded":"2023-12-07T10:19:01.808115+0000","last_fullsized":"2023-12-07T10:19:01.808115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:54:05.589792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.856029+0000","last_change":"2023-12-07T10:17:45.796640+0000","last_active":"2023-12-07T10:18:01.856029+0000","last_peered":"2023-12-07T10:18:01.856029+0000","last_clean":"2023-12-07T10:18:01.856029+0000","last_became_active":"2023-12-07T10:17:45.796183+0000","last_became_peered":"2023-12-07T10:17:45.796183+0000","last_unstale":"2023-12-07T10:18:01.856029+0000","last_undegraded":"2023-12-07T10:18:01.856029+0000","last_fullsized":"2023-12-07T10:18:01.856029+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:41:56.890662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813362+0000","last_change":"2023-12-07T10:17:40.760283+0000","last_active":"2023-12-07T10:19:01.813362+0000","last_peered":"2023-12-07T10:19:01.813362+0000","last_clean":"2023-12-07T10:19:01.813362+0000","last_became_active":"2023-12-07T10:17:40.759398+0000","last_became_peered":"2023-12-07T10:17:40.759398+0000","last_unstale":"2023-12-07T10:19:01.813362+0000","last_undegraded":"2023-12-07T10:19:01.813362+0000","last_fullsized":"2023-12-07T10:19:01.813362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:34:37.365490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802564+0000","last_change":"2023-12-07T10:17:41.604150+0000","last_active":"2023-12-07T10:19:01.802564+0000","last_peered":"2023-12-07T10:19:01.802564+0000","last_clean":"2023-12-07T10:19:01.802564+0000","last_became_active":"2023-12-07T10:17:41.603688+0000","last_became_peered":"2023-12-07T10:17:41.603688+0000","last_unstale":"2023-12-07T10:19:01.802564+0000","last_undegraded":"2023-12-07T10:19:01.802564+0000","last_fullsized":"2023-12-07T10:19:01.802564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:52:43.735609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803710+0000","last_change":"2023-12-07T10:17:45.806569+0000","last_active":"2023-12-07T10:19:01.803710+0000","last_peered":"2023-12-07T10:19:01.803710+0000","last_clean":"2023-12-07T10:19:01.803710+0000","last_became_active":"2023-12-07T10:17:45.806099+0000","last_became_peered":"2023-12-07T10:17:45.806099+0000","last_unstale":"2023-12-07T10:19:01.803710+0000","last_undegraded":"2023-12-07T10:19:01.803710+0000","last_fullsized":"2023-12-07T10:19:01.803710+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:02:19.977670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807289+0000","last_change":"2023-12-07T10:17:40.744388+0000","last_active":"2023-12-07T10:19:01.807289+0000","last_peered":"2023-12-07T10:19:01.807289+0000","last_clean":"2023-12-07T10:19:01.807289+0000","last_became_active":"2023-12-07T10:17:40.743887+0000","last_became_peered":"2023-12-07T10:17:40.743887+0000","last_unstale":"2023-12-07T10:19:01.807289+0000","last_undegraded":"2023-12-07T10:19:01.807289+0000","last_fullsized":"2023-12-07T10:19:01.807289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:18:39.102099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807114+0000","last_change":"2023-12-07T10:17:41.595616+0000","last_active":"2023-12-07T10:19:01.807114+0000","last_peered":"2023-12-07T10:19:01.807114+0000","last_clean":"2023-12-07T10:19:01.807114+0000","last_became_active":"2023-12-07T10:17:41.595256+0000","last_became_peered":"2023-12-07T10:17:41.595256+0000","last_unstale":"2023-12-07T10:19:01.807114+0000","last_undegraded":"2023-12-07T10:19:01.807114+0000","last_fullsized":"2023-12-07T10:19:01.807114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:08:00.156043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802753+0000","last_change":"2023-12-07T10:17:45.794523+0000","last_active":"2023-12-07T10:19:01.802753+0000","last_peered":"2023-12-07T10:19:01.802753+0000","last_clean":"2023-12-07T10:19:01.802753+0000","last_became_active":"2023-12-07T10:17:45.794139+0000","last_became_peered":"2023-12-07T10:17:45.794139+0000","last_unstale":"2023-12-07T10:19:01.802753+0000","last_undegraded":"2023-12-07T10:19:01.802753+0000","last_fullsized":"2023-12-07T10:19:01.802753+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:43:24.767627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":132,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:04.821173+0000","last_change":"2023-12-07T10:17:40.759620+0000","last_active":"2023-12-07T10:19:04.821173+0000","last_peered":"2023-12-07T10:19:04.821173+0000","last_clean":"2023-12-07T10:19:04.821173+0000","last_became_active":"2023-12-07T10:17:40.759372+0000","last_became_peered":"2023-12-07T10:17:40.759372+0000","last_unstale":"2023-12-07T10:19:04.821173+0000","last_undegraded":"2023-12-07T10:19:04.821173+0000","last_fullsized":"2023-12-07T10:19:04.821173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:46:14.459409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.989575+0000","last_change":"2023-12-07T10:17:41.601152+0000","last_active":"2023-12-07T10:18:01.989575+0000","last_peered":"2023-12-07T10:18:01.989575+0000","last_clean":"2023-12-07T10:18:01.989575+0000","last_became_active":"2023-12-07T10:17:41.600147+0000","last_became_peered":"2023-12-07T10:17:41.600147+0000","last_unstale":"2023-12-07T10:18:01.989575+0000","last_undegraded":"2023-12-07T10:18:01.989575+0000","last_fullsized":"2023-12-07T10:18:01.989575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:40:46.671489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807019+0000","last_change":"2023-12-07T10:17:45.796229+0000","last_active":"2023-12-07T10:19:01.807019+0000","last_peered":"2023-12-07T10:19:01.807019+0000","last_clean":"2023-12-07T10:19:01.807019+0000","last_became_active":"2023-12-07T10:17:45.795582+0000","last_became_peered":"2023-12-07T10:17:45.795582+0000","last_unstale":"2023-12-07T10:19:01.807019+0000","last_undegraded":"2023-12-07T10:19:01.807019+0000","last_fullsized":"2023-12-07T10:19:01.807019+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:29:32.853975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811078+0000","last_change":"2023-12-07T10:17:40.748353+0000","last_active":"2023-12-07T10:19:01.811078+0000","last_peered":"2023-12-07T10:19:01.811078+0000","last_clean":"2023-12-07T10:19:01.811078+0000","last_became_active":"2023-12-07T10:17:40.748050+0000","last_became_peered":"2023-12-07T10:17:40.748050+0000","last_unstale":"2023-12-07T10:19:01.811078+0000","last_undegraded":"2023-12-07T10:19:01.811078+0000","last_fullsized":"2023-12-07T10:19:01.811078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:18:15.298534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.805543+0000","last_change":"2023-12-07T10:18:02.256728+0000","last_active":"2023-12-07T10:19:01.805543+0000","last_peered":"2023-12-07T10:19:01.805543+0000","last_clean":"2023-12-07T10:19:01.805543+0000","last_became_active":"2023-12-07T10:18:02.256271+0000","last_became_peered":"2023-12-07T10:18:02.256271+0000","last_unstale":"2023-12-07T10:19:01.805543+0000","last_undegraded":"2023-12-07T10:19:01.805543+0000","last_fullsized":"2023-12-07T10:19:01.805543+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:27:00.198934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.254711+0000","last_change":"2023-12-07T10:17:45.796903+0000","last_active":"2023-12-07T10:18:02.254711+0000","last_peered":"2023-12-07T10:18:02.254711+0000","last_clean":"2023-12-07T10:18:02.254711+0000","last_became_active":"2023-12-07T10:17:45.796454+0000","last_became_peered":"2023-12-07T10:17:45.796454+0000","last_unstale":"2023-12-07T10:18:02.254711+0000","last_undegraded":"2023-12-07T10:18:02.254711+0000","last_fullsized":"2023-12-07T10:18:02.254711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:46:36.955396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808245+0000","last_change":"2023-12-07T10:17:40.756045+0000","last_active":"2023-12-07T10:19:01.808245+0000","last_peered":"2023-12-07T10:19:01.808245+0000","last_clean":"2023-12-07T10:19:01.808245+0000","last_became_active":"2023-12-07T10:17:40.755763+0000","last_became_peered":"2023-12-07T10:17:40.755763+0000","last_unstale":"2023-12-07T10:19:01.808245+0000","last_undegraded":"2023-12-07T10:19:01.808245+0000","last_fullsized":"2023-12-07T10:19:01.808245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:01:08.009076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.989591+0000","last_change":"2023-12-07T10:17:41.611178+0000","last_active":"2023-12-07T10:18:01.989591+0000","last_peered":"2023-12-07T10:18:01.989591+0000","last_clean":"2023-12-07T10:18:01.989591+0000","last_became_active":"2023-12-07T10:17:41.610750+0000","last_became_peered":"2023-12-07T10:17:41.610750+0000","last_unstale":"2023-12-07T10:18:01.989591+0000","last_undegraded":"2023-12-07T10:18:01.989591+0000","last_fullsized":"2023-12-07T10:18:01.989591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:56:30.763523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807197+0000","last_change":"2023-12-07T10:17:45.800183+0000","last_active":"2023-12-07T10:19:01.807197+0000","last_peered":"2023-12-07T10:19:01.807197+0000","last_clean":"2023-12-07T10:19:01.807197+0000","last_became_active":"2023-12-07T10:17:45.798598+0000","last_became_peered":"2023-12-07T10:17:45.798598+0000","last_unstale":"2023-12-07T10:19:01.807197+0000","last_undegraded":"2023-12-07T10:19:01.807197+0000","last_fullsized":"2023-12-07T10:19:01.807197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:06:22.162951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":172,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:04.821208+0000","last_change":"2023-12-07T10:17:40.752481+0000","last_active":"2023-12-07T10:19:04.821208+0000","last_peered":"2023-12-07T10:19:04.821208+0000","last_clean":"2023-12-07T10:19:04.821208+0000","last_became_active":"2023-12-07T10:17:40.752158+0000","last_became_peered":"2023-12-07T10:17:40.752158+0000","last_unstale":"2023-12-07T10:19:04.821208+0000","last_undegraded":"2023-12-07T10:19:04.821208+0000","last_fullsized":"2023-12-07T10:19:04.821208+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:43:27.963014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64453,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":117,"num_read_kb":121,"num_write":36,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.859129+0000","last_change":"2023-12-07T10:17:41.614107+0000","last_active":"2023-12-07T10:18:01.859129+0000","last_peered":"2023-12-07T10:18:01.859129+0000","last_clean":"2023-12-07T10:18:01.859129+0000","last_became_active":"2023-12-07T10:17:41.613743+0000","last_became_peered":"2023-12-07T10:17:41.613743+0000","last_unstale":"2023-12-07T10:18:01.859129+0000","last_undegraded":"2023-12-07T10:18:01.859129+0000","last_fullsized":"2023-12-07T10:18:01.859129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:49:03.200387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808863+0000","last_change":"2023-12-07T10:17:45.804282+0000","last_active":"2023-12-07T10:19:01.808863+0000","last_peered":"2023-12-07T10:19:01.808863+0000","last_clean":"2023-12-07T10:19:01.808863+0000","last_became_active":"2023-12-07T10:17:45.803794+0000","last_became_peered":"2023-12-07T10:17:45.803794+0000","last_unstale":"2023-12-07T10:19:01.808863+0000","last_undegraded":"2023-12-07T10:19:01.808863+0000","last_fullsized":"2023-12-07T10:19:01.808863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:31:26.887565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810171+0000","last_change":"2023-12-07T10:17:40.747204+0000","last_active":"2023-12-07T10:19:01.810171+0000","last_peered":"2023-12-07T10:19:01.810171+0000","last_clean":"2023-12-07T10:19:01.810171+0000","last_became_active":"2023-12-07T10:17:40.746916+0000","last_became_peered":"2023-12-07T10:17:40.746916+0000","last_unstale":"2023-12-07T10:19:01.810171+0000","last_undegraded":"2023-12-07T10:19:01.810171+0000","last_fullsized":"2023-12-07T10:19:01.810171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:53:49.228320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.814021+0000","last_change":"2023-12-07T10:17:41.606895+0000","last_active":"2023-12-07T10:19:01.814021+0000","last_peered":"2023-12-07T10:19:01.814021+0000","last_clean":"2023-12-07T10:19:01.814021+0000","last_became_active":"2023-12-07T10:17:41.606596+0000","last_became_peered":"2023-12-07T10:17:41.606596+0000","last_unstale":"2023-12-07T10:19:01.814021+0000","last_undegraded":"2023-12-07T10:19:01.814021+0000","last_fullsized":"2023-12-07T10:19:01.814021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:34:25.166906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.856131+0000","last_change":"2023-12-07T10:17:45.807101+0000","last_active":"2023-12-07T10:18:01.856131+0000","last_peered":"2023-12-07T10:18:01.856131+0000","last_clean":"2023-12-07T10:18:01.856131+0000","last_became_active":"2023-12-07T10:17:45.806770+0000","last_became_peered":"2023-12-07T10:17:45.806770+0000","last_unstale":"2023-12-07T10:18:01.856131+0000","last_undegraded":"2023-12-07T10:18:01.856131+0000","last_fullsized":"2023-12-07T10:18:01.856131+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:39:32.522968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808371+0000","last_change":"2023-12-07T10:17:40.754856+0000","last_active":"2023-12-07T10:19:01.808371+0000","last_peered":"2023-12-07T10:19:01.808371+0000","last_clean":"2023-12-07T10:19:01.808371+0000","last_became_active":"2023-12-07T10:17:40.754488+0000","last_became_peered":"2023-12-07T10:17:40.754488+0000","last_unstale":"2023-12-07T10:19:01.808371+0000","last_undegraded":"2023-12-07T10:19:01.808371+0000","last_fullsized":"2023-12-07T10:19:01.808371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:44:50.960216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811734+0000","last_change":"2023-12-07T10:17:41.606963+0000","last_active":"2023-12-07T10:19:01.811734+0000","last_peered":"2023-12-07T10:19:01.811734+0000","last_clean":"2023-12-07T10:19:01.811734+0000","last_became_active":"2023-12-07T10:17:41.606408+0000","last_became_peered":"2023-12-07T10:17:41.606408+0000","last_unstale":"2023-12-07T10:19:01.811734+0000","last_undegraded":"2023-12-07T10:19:01.811734+0000","last_fullsized":"2023-12-07T10:19:01.811734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:25:47.906649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":2,"reported_epoch":92,"state":"peering","last_fresh":"2023-12-07T10:17:45.779804+0000","last_change":"2023-12-07T10:19:02.353784+0000","last_active":"2023-12-07T10:17:44.766631+0000","last_peered":"2023-12-07T10:17:44.766631+0000","last_clean":"2023-12-07T10:17:44.766631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-07T10:17:45.779804+0000","last_undegraded":"2023-12-07T10:17:45.779804+0000","last_fullsized":"2023-12-07T10:17:45.779804+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":76,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:19:00.835987+0000","last_change":"2023-12-07T10:17:40.747494+0000","last_active":"2023-12-07T10:19:00.835987+0000","last_peered":"2023-12-07T10:19:00.835987+0000","last_clean":"2023-12-07T10:19:00.835987+0000","last_became_active":"2023-12-07T10:17:40.747157+0000","last_became_peered":"2023-12-07T10:17:40.747157+0000","last_unstale":"2023-12-07T10:19:00.835987+0000","last_undegraded":"2023-12-07T10:19:00.835987+0000","last_fullsized":"2023-12-07T10:19:00.835987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:45:24.395418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":976,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813024+0000","last_change":"2023-12-07T10:17:41.609495+0000","last_active":"2023-12-07T10:19:01.813024+0000","last_peered":"2023-12-07T10:19:01.813024+0000","last_clean":"2023-12-07T10:19:01.813024+0000","last_became_active":"2023-12-07T10:17:41.609181+0000","last_became_peered":"2023-12-07T10:17:41.609181+0000","last_unstale":"2023-12-07T10:19:01.813024+0000","last_undegraded":"2023-12-07T10:19:01.813024+0000","last_fullsized":"2023-12-07T10:19:01.813024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:20:33.473824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813855+0000","last_change":"2023-12-07T10:17:45.808862+0000","last_active":"2023-12-07T10:19:01.813855+0000","last_peered":"2023-12-07T10:19:01.813855+0000","last_clean":"2023-12-07T10:19:01.813855+0000","last_became_active":"2023-12-07T10:17:45.808461+0000","last_became_peered":"2023-12-07T10:17:45.808461+0000","last_unstale":"2023-12-07T10:19:01.813855+0000","last_undegraded":"2023-12-07T10:19:01.813855+0000","last_fullsized":"2023-12-07T10:19:01.813855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:09:12.598126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:13.769981+0000","last_change":"2023-12-07T10:18:02.263709+0000","last_active":"2023-12-07T10:18:13.769981+0000","last_peered":"2023-12-07T10:18:13.769981+0000","last_clean":"2023-12-07T10:18:13.769981+0000","last_became_active":"2023-12-07T10:18:02.255788+0000","last_became_peered":"2023-12-07T10:18:02.255788+0000","last_unstale":"2023-12-07T10:18:13.769981+0000","last_undegraded":"2023-12-07T10:18:13.769981+0000","last_fullsized":"2023-12-07T10:18:13.769981+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:12:04.976226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808356+0000","last_change":"2023-12-07T10:17:41.610509+0000","last_active":"2023-12-07T10:19:01.808356+0000","last_peered":"2023-12-07T10:19:01.808356+0000","last_clean":"2023-12-07T10:19:01.808356+0000","last_became_active":"2023-12-07T10:17:41.610121+0000","last_became_peered":"2023-12-07T10:17:41.610121+0000","last_unstale":"2023-12-07T10:19:01.808356+0000","last_undegraded":"2023-12-07T10:19:01.808356+0000","last_fullsized":"2023-12-07T10:19:01.808356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:53:17.365880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808256+0000","last_change":"2023-12-07T10:17:45.795818+0000","last_active":"2023-12-07T10:19:01.808256+0000","last_peered":"2023-12-07T10:19:01.808256+0000","last_clean":"2023-12-07T10:19:01.808256+0000","last_became_active":"2023-12-07T10:17:45.795477+0000","last_became_peered":"2023-12-07T10:17:45.795477+0000","last_unstale":"2023-12-07T10:19:01.808256+0000","last_undegraded":"2023-12-07T10:19:01.808256+0000","last_fullsized":"2023-12-07T10:19:01.808256+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:17:21.494654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:15.281962+0000","last_change":"2023-12-07T10:17:40.751791+0000","last_active":"2023-12-07T10:18:15.281962+0000","last_peered":"2023-12-07T10:18:15.281962+0000","last_clean":"2023-12-07T10:18:15.281962+0000","last_became_active":"2023-12-07T10:17:40.751502+0000","last_became_peered":"2023-12-07T10:17:40.751502+0000","last_unstale":"2023-12-07T10:18:15.281962+0000","last_undegraded":"2023-12-07T10:18:15.281962+0000","last_fullsized":"2023-12-07T10:18:15.281962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:15:52.319265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.257186+0000","last_change":"2023-12-07T10:18:02.256247+0000","last_active":"2023-12-07T10:18:02.257186+0000","last_peered":"2023-12-07T10:18:02.257186+0000","last_clean":"2023-12-07T10:18:02.257186+0000","last_became_active":"2023-12-07T10:18:02.255732+0000","last_became_peered":"2023-12-07T10:18:02.255732+0000","last_unstale":"2023-12-07T10:18:02.257186+0000","last_undegraded":"2023-12-07T10:18:02.257186+0000","last_fullsized":"2023-12-07T10:18:02.257186+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:00:06.516804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.859099+0000","last_change":"2023-12-07T10:17:45.791966+0000","last_active":"2023-12-07T10:18:01.859099+0000","last_peered":"2023-12-07T10:18:01.859099+0000","last_clean":"2023-12-07T10:18:01.859099+0000","last_became_active":"2023-12-07T10:17:45.791613+0000","last_became_peered":"2023-12-07T10:17:45.791613+0000","last_unstale":"2023-12-07T10:18:01.859099+0000","last_undegraded":"2023-12-07T10:18:01.859099+0000","last_fullsized":"2023-12-07T10:18:01.859099+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:13:02.291289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807344+0000","last_change":"2023-12-07T10:17:40.762896+0000","last_active":"2023-12-07T10:19:01.807344+0000","last_peered":"2023-12-07T10:19:01.807344+0000","last_clean":"2023-12-07T10:19:01.807344+0000","last_became_active":"2023-12-07T10:17:40.762427+0000","last_became_peered":"2023-12-07T10:17:40.762427+0000","last_unstale":"2023-12-07T10:19:01.807344+0000","last_undegraded":"2023-12-07T10:19:01.807344+0000","last_fullsized":"2023-12-07T10:19:01.807344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:56:35.589548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803085+0000","last_change":"2023-12-07T10:17:41.610354+0000","last_active":"2023-12-07T10:19:01.803085+0000","last_peered":"2023-12-07T10:19:01.803085+0000","last_clean":"2023-12-07T10:19:01.803085+0000","last_became_active":"2023-12-07T10:17:41.609888+0000","last_became_peered":"2023-12-07T10:17:41.609888+0000","last_unstale":"2023-12-07T10:19:01.803085+0000","last_undegraded":"2023-12-07T10:19:01.803085+0000","last_fullsized":"2023-12-07T10:19:01.803085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:48:19.976949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813526+0000","last_change":"2023-12-07T10:17:45.807324+0000","last_active":"2023-12-07T10:19:01.813526+0000","last_peered":"2023-12-07T10:19:01.813526+0000","last_clean":"2023-12-07T10:19:01.813526+0000","last_became_active":"2023-12-07T10:17:45.806036+0000","last_became_peered":"2023-12-07T10:17:45.806036+0000","last_unstale":"2023-12-07T10:19:01.813526+0000","last_undegraded":"2023-12-07T10:19:01.813526+0000","last_fullsized":"2023-12-07T10:19:01.813526+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:54:36.847992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'5","reported_seq":79,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.840754+0000","last_change":"2023-12-07T10:17:40.754247+0000","last_active":"2023-12-07T10:19:03.840754+0000","last_peered":"2023-12-07T10:19:03.840754+0000","last_clean":"2023-12-07T10:19:03.840754+0000","last_became_active":"2023-12-07T10:17:40.753964+0000","last_became_peered":"2023-12-07T10:17:40.753964+0000","last_unstale":"2023-12-07T10:19:03.840754+0000","last_undegraded":"2023-12-07T10:19:03.840754+0000","last_fullsized":"2023-12-07T10:19:03.840754+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:22:04.832399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802944+0000","last_change":"2023-12-07T10:17:41.610516+0000","last_active":"2023-12-07T10:19:01.802944+0000","last_peered":"2023-12-07T10:19:01.802944+0000","last_clean":"2023-12-07T10:19:01.802944+0000","last_became_active":"2023-12-07T10:17:41.610184+0000","last_became_peered":"2023-12-07T10:17:41.610184+0000","last_unstale":"2023-12-07T10:19:01.802944+0000","last_undegraded":"2023-12-07T10:19:01.802944+0000","last_fullsized":"2023-12-07T10:19:01.802944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:53:43.100723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813880+0000","last_change":"2023-12-07T10:17:45.804624+0000","last_active":"2023-12-07T10:19:01.813880+0000","last_peered":"2023-12-07T10:19:01.813880+0000","last_clean":"2023-12-07T10:19:01.813880+0000","last_became_active":"2023-12-07T10:17:45.803667+0000","last_became_peered":"2023-12-07T10:17:45.803667+0000","last_unstale":"2023-12-07T10:19:01.813880+0000","last_undegraded":"2023-12-07T10:19:01.813880+0000","last_fullsized":"2023-12-07T10:19:01.813880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:55:09.642795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807386+0000","last_change":"2023-12-07T10:17:41.618433+0000","last_active":"2023-12-07T10:19:01.807386+0000","last_peered":"2023-12-07T10:19:01.807386+0000","last_clean":"2023-12-07T10:19:01.807386+0000","last_became_active":"2023-12-07T10:17:41.617942+0000","last_became_peered":"2023-12-07T10:17:41.617942+0000","last_unstale":"2023-12-07T10:19:01.807386+0000","last_undegraded":"2023-12-07T10:19:01.807386+0000","last_fullsized":"2023-12-07T10:19:01.807386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:26:48.166157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":77,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.839457+0000","last_change":"2023-12-07T10:17:40.761907+0000","last_active":"2023-12-07T10:19:02.839457+0000","last_peered":"2023-12-07T10:19:02.839457+0000","last_clean":"2023-12-07T10:19:02.839457+0000","last_became_active":"2023-12-07T10:17:40.761564+0000","last_became_peered":"2023-12-07T10:17:40.761564+0000","last_unstale":"2023-12-07T10:19:02.839457+0000","last_undegraded":"2023-12-07T10:19:02.839457+0000","last_fullsized":"2023-12-07T10:19:02.839457+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:57:22.304352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811777+0000","last_change":"2023-12-07T10:17:45.792684+0000","last_active":"2023-12-07T10:19:01.811777+0000","last_peered":"2023-12-07T10:19:01.811777+0000","last_clean":"2023-12-07T10:19:01.811777+0000","last_became_active":"2023-12-07T10:17:45.792172+0000","last_became_peered":"2023-12-07T10:17:45.792172+0000","last_unstale":"2023-12-07T10:19:01.811777+0000","last_undegraded":"2023-12-07T10:19:01.811777+0000","last_fullsized":"2023-12-07T10:19:01.811777+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:24:31.084727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802651+0000","last_change":"2023-12-07T10:17:41.612582+0000","last_active":"2023-12-07T10:19:01.802651+0000","last_peered":"2023-12-07T10:19:01.802651+0000","last_clean":"2023-12-07T10:19:01.802651+0000","last_became_active":"2023-12-07T10:17:41.612244+0000","last_became_peered":"2023-12-07T10:17:41.612244+0000","last_unstale":"2023-12-07T10:19:01.802651+0000","last_undegraded":"2023-12-07T10:19:01.802651+0000","last_fullsized":"2023-12-07T10:19:01.802651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:41:53.506037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:13.770423+0000","last_change":"2023-12-07T10:17:40.748565+0000","last_active":"2023-12-07T10:18:13.770423+0000","last_peered":"2023-12-07T10:18:13.770423+0000","last_clean":"2023-12-07T10:18:13.770423+0000","last_became_active":"2023-12-07T10:17:40.748250+0000","last_became_peered":"2023-12-07T10:17:40.748250+0000","last_unstale":"2023-12-07T10:18:13.770423+0000","last_undegraded":"2023-12-07T10:18:13.770423+0000","last_fullsized":"2023-12-07T10:18:13.770423+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:26:28.202440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":328,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.812930+0000","last_change":"2023-12-07T10:16:33.482173+0000","last_active":"2023-12-07T10:19:01.812930+0000","last_peered":"2023-12-07T10:19:01.812930+0000","last_clean":"2023-12-07T10:19:01.812930+0000","last_became_active":"2023-12-07T10:16:33.481699+0000","last_became_peered":"2023-12-07T10:16:33.481699+0000","last_unstale":"2023-12-07T10:19:01.812930+0000","last_undegraded":"2023-12-07T10:19:01.812930+0000","last_fullsized":"2023-12-07T10:19:01.812930+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:13:25.388258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:13:25.388258+0000","last_clean_scrub_stamp":"2023-12-07T10:13:25.388258+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:57:09.583677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807871+0000","last_change":"2023-12-07T10:17:45.806289+0000","last_active":"2023-12-07T10:19:01.807871+0000","last_peered":"2023-12-07T10:19:01.807871+0000","last_clean":"2023-12-07T10:19:01.807871+0000","last_became_active":"2023-12-07T10:17:45.805954+0000","last_became_peered":"2023-12-07T10:17:45.805954+0000","last_unstale":"2023-12-07T10:19:01.807871+0000","last_undegraded":"2023-12-07T10:19:01.807871+0000","last_fullsized":"2023-12-07T10:19:01.807871+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:21:09.955380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813281+0000","last_change":"2023-12-07T10:17:41.612322+0000","last_active":"2023-12-07T10:19:01.813281+0000","last_peered":"2023-12-07T10:19:01.813281+0000","last_clean":"2023-12-07T10:19:01.813281+0000","last_became_active":"2023-12-07T10:17:41.612039+0000","last_became_peered":"2023-12-07T10:17:41.612039+0000","last_unstale":"2023-12-07T10:19:01.813281+0000","last_undegraded":"2023-12-07T10:19:01.813281+0000","last_fullsized":"2023-12-07T10:19:01.813281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:40:49.883742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.812980+0000","last_change":"2023-12-07T10:17:40.749694+0000","last_active":"2023-12-07T10:19:01.812980+0000","last_peered":"2023-12-07T10:19:01.812980+0000","last_clean":"2023-12-07T10:19:01.812980+0000","last_became_active":"2023-12-07T10:17:40.749393+0000","last_became_peered":"2023-12-07T10:17:40.749393+0000","last_unstale":"2023-12-07T10:19:01.812980+0000","last_undegraded":"2023-12-07T10:19:01.812980+0000","last_fullsized":"2023-12-07T10:19:01.812980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:15:36.949062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.855939+0000","last_change":"2023-12-07T10:17:45.805769+0000","last_active":"2023-12-07T10:18:01.855939+0000","last_peered":"2023-12-07T10:18:01.855939+0000","last_clean":"2023-12-07T10:18:01.855939+0000","last_became_active":"2023-12-07T10:17:45.805346+0000","last_became_peered":"2023-12-07T10:17:45.805346+0000","last_unstale":"2023-12-07T10:18:01.855939+0000","last_undegraded":"2023-12-07T10:18:01.855939+0000","last_fullsized":"2023-12-07T10:18:01.855939+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:47:47.781933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:12.756983+0000","last_change":"2023-12-07T10:18:01.862558+0000","last_active":"2023-12-07T10:18:12.756983+0000","last_peered":"2023-12-07T10:18:12.756983+0000","last_clean":"2023-12-07T10:18:12.756983+0000","last_became_active":"2023-12-07T10:18:01.862063+0000","last_became_peered":"2023-12-07T10:18:01.862063+0000","last_unstale":"2023-12-07T10:18:12.756983+0000","last_undegraded":"2023-12-07T10:18:12.756983+0000","last_fullsized":"2023-12-07T10:18:12.756983+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:25:32.652160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813607+0000","last_change":"2023-12-07T10:17:41.617199+0000","last_active":"2023-12-07T10:19:01.813607+0000","last_peered":"2023-12-07T10:19:01.813607+0000","last_clean":"2023-12-07T10:19:01.813607+0000","last_became_active":"2023-12-07T10:17:41.616901+0000","last_became_peered":"2023-12-07T10:17:41.616901+0000","last_unstale":"2023-12-07T10:19:01.813607+0000","last_undegraded":"2023-12-07T10:19:01.813607+0000","last_fullsized":"2023-12-07T10:19:01.813607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:23:59.433861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802553+0000","last_change":"2023-12-07T10:17:45.790914+0000","last_active":"2023-12-07T10:19:01.802553+0000","last_peered":"2023-12-07T10:19:01.802553+0000","last_clean":"2023-12-07T10:19:01.802553+0000","last_became_active":"2023-12-07T10:17:45.790428+0000","last_became_peered":"2023-12-07T10:17:45.790428+0000","last_unstale":"2023-12-07T10:19:01.802553+0000","last_undegraded":"2023-12-07T10:19:01.802553+0000","last_fullsized":"2023-12-07T10:19:01.802553+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:03:57.516300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807601+0000","last_change":"2023-12-07T10:18:01.863765+0000","last_active":"2023-12-07T10:19:01.807601+0000","last_peered":"2023-12-07T10:19:01.807601+0000","last_clean":"2023-12-07T10:19:01.807601+0000","last_became_active":"2023-12-07T10:18:01.858926+0000","last_became_peered":"2023-12-07T10:18:01.858926+0000","last_unstale":"2023-12-07T10:19:01.807601+0000","last_undegraded":"2023-12-07T10:19:01.807601+0000","last_fullsized":"2023-12-07T10:19:01.807601+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:44:19.663592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.804412+0000","last_change":"2023-12-07T10:17:41.608513+0000","last_active":"2023-12-07T10:19:01.804412+0000","last_peered":"2023-12-07T10:19:01.804412+0000","last_clean":"2023-12-07T10:19:01.804412+0000","last_became_active":"2023-12-07T10:17:41.605049+0000","last_became_peered":"2023-12-07T10:17:41.605049+0000","last_unstale":"2023-12-07T10:19:01.804412+0000","last_undegraded":"2023-12-07T10:19:01.804412+0000","last_fullsized":"2023-12-07T10:19:01.804412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:33:00.730827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.809743+0000","last_change":"2023-12-07T10:17:45.792583+0000","last_active":"2023-12-07T10:19:01.809743+0000","last_peered":"2023-12-07T10:19:01.809743+0000","last_clean":"2023-12-07T10:19:01.809743+0000","last_became_active":"2023-12-07T10:17:45.792162+0000","last_became_peered":"2023-12-07T10:17:45.792162+0000","last_unstale":"2023-12-07T10:19:01.809743+0000","last_undegraded":"2023-12-07T10:19:01.809743+0000","last_fullsized":"2023-12-07T10:19:01.809743+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:47:30.901477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:13.769485+0000","last_change":"2023-12-07T10:17:40.750826+0000","last_active":"2023-12-07T10:18:13.769485+0000","last_peered":"2023-12-07T10:18:13.769485+0000","last_clean":"2023-12-07T10:18:13.769485+0000","last_became_active":"2023-12-07T10:17:40.750519+0000","last_became_peered":"2023-12-07T10:17:40.750519+0000","last_unstale":"2023-12-07T10:18:13.769485+0000","last_undegraded":"2023-12-07T10:18:13.769485+0000","last_fullsized":"2023-12-07T10:18:13.769485+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:21:01.008897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.805305+0000","last_change":"2023-12-07T10:17:41.609488+0000","last_active":"2023-12-07T10:19:01.805305+0000","last_peered":"2023-12-07T10:19:01.805305+0000","last_clean":"2023-12-07T10:19:01.805305+0000","last_became_active":"2023-12-07T10:17:41.609061+0000","last_became_peered":"2023-12-07T10:17:41.609061+0000","last_unstale":"2023-12-07T10:19:01.805305+0000","last_undegraded":"2023-12-07T10:19:01.805305+0000","last_fullsized":"2023-12-07T10:19:01.805305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:39:32.729502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.254528+0000","last_change":"2023-12-07T10:17:45.797573+0000","last_active":"2023-12-07T10:18:02.254528+0000","last_peered":"2023-12-07T10:18:02.254528+0000","last_clean":"2023-12-07T10:18:02.254528+0000","last_became_active":"2023-12-07T10:17:45.797119+0000","last_became_peered":"2023-12-07T10:17:45.797119+0000","last_unstale":"2023-12-07T10:18:02.254528+0000","last_undegraded":"2023-12-07T10:18:02.254528+0000","last_fullsized":"2023-12-07T10:18:02.254528+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:17:27.100470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:09.901985+0000","last_change":"2023-12-07T10:17:40.753596+0000","last_active":"2023-12-07T10:18:09.901985+0000","last_peered":"2023-12-07T10:18:09.901985+0000","last_clean":"2023-12-07T10:18:09.901985+0000","last_became_active":"2023-12-07T10:17:40.753251+0000","last_became_peered":"2023-12-07T10:17:40.753251+0000","last_unstale":"2023-12-07T10:18:09.901985+0000","last_undegraded":"2023-12-07T10:18:09.901985+0000","last_fullsized":"2023-12-07T10:18:09.901985+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:08:30.868794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.857325+0000","last_change":"2023-12-07T10:17:41.599689+0000","last_active":"2023-12-07T10:18:01.857325+0000","last_peered":"2023-12-07T10:18:01.857325+0000","last_clean":"2023-12-07T10:18:01.857325+0000","last_became_active":"2023-12-07T10:17:41.599358+0000","last_became_peered":"2023-12-07T10:17:41.599358+0000","last_unstale":"2023-12-07T10:18:01.857325+0000","last_undegraded":"2023-12-07T10:18:01.857325+0000","last_fullsized":"2023-12-07T10:18:01.857325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:44:19.742560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":2,"reported_epoch":92,"state":"peering","last_fresh":"2023-12-07T10:17:45.780383+0000","last_change":"2023-12-07T10:19:02.353800+0000","last_active":"2023-12-07T10:17:44.766631+0000","last_peered":"2023-12-07T10:17:44.766631+0000","last_clean":"2023-12-07T10:17:44.766631+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-07T10:17:45.780383+0000","last_undegraded":"2023-12-07T10:17:45.780383+0000","last_fullsized":"2023-12-07T10:17:45.780383+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802597+0000","last_change":"2023-12-07T10:17:40.758663+0000","last_active":"2023-12-07T10:19:01.802597+0000","last_peered":"2023-12-07T10:19:01.802597+0000","last_clean":"2023-12-07T10:19:01.802597+0000","last_became_active":"2023-12-07T10:17:40.753611+0000","last_became_peered":"2023-12-07T10:17:40.753611+0000","last_unstale":"2023-12-07T10:19:01.802597+0000","last_undegraded":"2023-12-07T10:19:01.802597+0000","last_fullsized":"2023-12-07T10:19:01.802597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:00:59.256468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.855712+0000","last_change":"2023-12-07T10:17:41.616546+0000","last_active":"2023-12-07T10:18:01.855712+0000","last_peered":"2023-12-07T10:18:01.855712+0000","last_clean":"2023-12-07T10:18:01.855712+0000","last_became_active":"2023-12-07T10:17:41.615780+0000","last_became_peered":"2023-12-07T10:17:41.615780+0000","last_unstale":"2023-12-07T10:18:01.855712+0000","last_undegraded":"2023-12-07T10:18:01.855712+0000","last_fullsized":"2023-12-07T10:18:01.855712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:22:32.455784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:13.223835+0000","last_change":"2023-12-07T10:17:45.797550+0000","last_active":"2023-12-07T10:18:13.223835+0000","last_peered":"2023-12-07T10:18:13.223835+0000","last_clean":"2023-12-07T10:18:13.223835+0000","last_became_active":"2023-12-07T10:17:45.797140+0000","last_became_peered":"2023-12-07T10:17:45.797140+0000","last_unstale":"2023-12-07T10:18:13.223835+0000","last_undegraded":"2023-12-07T10:18:13.223835+0000","last_fullsized":"2023-12-07T10:18:13.223835+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:53:44.984904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.809964+0000","last_change":"2023-12-07T10:17:40.748410+0000","last_active":"2023-12-07T10:19:01.809964+0000","last_peered":"2023-12-07T10:19:01.809964+0000","last_clean":"2023-12-07T10:19:01.809964+0000","last_became_active":"2023-12-07T10:17:40.748069+0000","last_became_peered":"2023-12-07T10:17:40.748069+0000","last_unstale":"2023-12-07T10:19:01.809964+0000","last_undegraded":"2023-12-07T10:19:01.809964+0000","last_fullsized":"2023-12-07T10:19:01.809964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:51:48.278974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803033+0000","last_change":"2023-12-07T10:17:41.609248+0000","last_active":"2023-12-07T10:19:01.803033+0000","last_peered":"2023-12-07T10:19:01.803033+0000","last_clean":"2023-12-07T10:19:01.803033+0000","last_became_active":"2023-12-07T10:17:41.608943+0000","last_became_peered":"2023-12-07T10:17:41.608943+0000","last_unstale":"2023-12-07T10:19:01.803033+0000","last_undegraded":"2023-12-07T10:19:01.803033+0000","last_fullsized":"2023-12-07T10:19:01.803033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:29:32.322273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:02.254578+0000","last_change":"2023-12-07T10:17:45.794855+0000","last_active":"2023-12-07T10:18:02.254578+0000","last_peered":"2023-12-07T10:18:02.254578+0000","last_clean":"2023-12-07T10:18:02.254578+0000","last_became_active":"2023-12-07T10:17:45.794410+0000","last_became_peered":"2023-12-07T10:17:45.794410+0000","last_unstale":"2023-12-07T10:18:02.254578+0000","last_undegraded":"2023-12-07T10:18:02.254578+0000","last_fullsized":"2023-12-07T10:18:02.254578+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:57:25.118036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:10.731035+0000","last_change":"2023-12-07T10:17:40.750471+0000","last_active":"2023-12-07T10:18:10.731035+0000","last_peered":"2023-12-07T10:18:10.731035+0000","last_clean":"2023-12-07T10:18:10.731035+0000","last_became_active":"2023-12-07T10:17:40.750176+0000","last_became_peered":"2023-12-07T10:17:40.750176+0000","last_unstale":"2023-12-07T10:18:10.731035+0000","last_undegraded":"2023-12-07T10:18:10.731035+0000","last_fullsized":"2023-12-07T10:18:10.731035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:04:22.305343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.812625+0000","last_change":"2023-12-07T10:17:41.614087+0000","last_active":"2023-12-07T10:19:01.812625+0000","last_peered":"2023-12-07T10:19:01.812625+0000","last_clean":"2023-12-07T10:19:01.812625+0000","last_became_active":"2023-12-07T10:17:41.607689+0000","last_became_peered":"2023-12-07T10:17:41.607689+0000","last_unstale":"2023-12-07T10:19:01.812625+0000","last_undegraded":"2023-12-07T10:19:01.812625+0000","last_fullsized":"2023-12-07T10:19:01.812625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:50:10.316085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"peering","last_fresh":"2023-12-07T10:18:01.989081+0000","last_change":"2023-12-07T10:19:01.811088+0000","last_active":"2023-12-07T10:19:01.811088+0000","last_peered":"2023-12-07T10:18:01.989081+0000","last_clean":"2023-12-07T10:18:01.989081+0000","last_became_active":"2023-12-07T10:17:45.790785+0000","last_became_peered":"2023-12-07T10:17:45.790785+0000","last_unstale":"2023-12-07T10:18:01.989081+0000","last_undegraded":"2023-12-07T10:18:01.989081+0000","last_fullsized":"2023-12-07T10:18:01.989081+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,6,7,8],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'5","reported_seq":79,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.840769+0000","last_change":"2023-12-07T10:17:40.753645+0000","last_active":"2023-12-07T10:19:02.840769+0000","last_peered":"2023-12-07T10:19:02.840769+0000","last_clean":"2023-12-07T10:19:02.840769+0000","last_became_active":"2023-12-07T10:17:40.753326+0000","last_became_peered":"2023-12-07T10:17:40.753326+0000","last_unstale":"2023-12-07T10:19:02.840769+0000","last_undegraded":"2023-12-07T10:19:02.840769+0000","last_fullsized":"2023-12-07T10:19:02.840769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:43:03.617591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811725+0000","last_change":"2023-12-07T10:17:41.607968+0000","last_active":"2023-12-07T10:19:01.811725+0000","last_peered":"2023-12-07T10:19:01.811725+0000","last_clean":"2023-12-07T10:19:01.811725+0000","last_became_active":"2023-12-07T10:17:41.607621+0000","last_became_peered":"2023-12-07T10:17:41.607621+0000","last_unstale":"2023-12-07T10:19:01.811725+0000","last_undegraded":"2023-12-07T10:19:01.811725+0000","last_fullsized":"2023-12-07T10:19:01.811725+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:55:10.224738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803538+0000","last_change":"2023-12-07T10:17:45.793678+0000","last_active":"2023-12-07T10:19:01.803538+0000","last_peered":"2023-12-07T10:19:01.803538+0000","last_clean":"2023-12-07T10:19:01.803538+0000","last_became_active":"2023-12-07T10:17:45.793283+0000","last_became_peered":"2023-12-07T10:17:45.793283+0000","last_unstale":"2023-12-07T10:19:01.803538+0000","last_undegraded":"2023-12-07T10:19:01.803538+0000","last_fullsized":"2023-12-07T10:19:01.803538+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:33:56.501107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808567+0000","last_change":"2023-12-07T10:17:40.762069+0000","last_active":"2023-12-07T10:19:01.808567+0000","last_peered":"2023-12-07T10:19:01.808567+0000","last_clean":"2023-12-07T10:19:01.808567+0000","last_became_active":"2023-12-07T10:17:40.761752+0000","last_became_peered":"2023-12-07T10:17:40.761752+0000","last_unstale":"2023-12-07T10:19:01.808567+0000","last_undegraded":"2023-12-07T10:19:01.808567+0000","last_fullsized":"2023-12-07T10:19:01.808567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:57:55.736144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811362+0000","last_change":"2023-12-07T10:17:41.607347+0000","last_active":"2023-12-07T10:19:01.811362+0000","last_peered":"2023-12-07T10:19:01.811362+0000","last_clean":"2023-12-07T10:19:01.811362+0000","last_became_active":"2023-12-07T10:17:41.606618+0000","last_became_peered":"2023-12-07T10:17:41.606618+0000","last_unstale":"2023-12-07T10:19:01.811362+0000","last_undegraded":"2023-12-07T10:19:01.811362+0000","last_fullsized":"2023-12-07T10:19:01.811362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:53:34.748130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807110+0000","last_change":"2023-12-07T10:17:45.791459+0000","last_active":"2023-12-07T10:19:01.807110+0000","last_peered":"2023-12-07T10:19:01.807110+0000","last_clean":"2023-12-07T10:19:01.807110+0000","last_became_active":"2023-12-07T10:17:45.791070+0000","last_became_peered":"2023-12-07T10:17:45.791070+0000","last_unstale":"2023-12-07T10:19:01.807110+0000","last_undegraded":"2023-12-07T10:19:01.807110+0000","last_fullsized":"2023-12-07T10:19:01.807110+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:46:35.985990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813745+0000","last_change":"2023-12-07T10:17:40.762726+0000","last_active":"2023-12-07T10:19:01.813745+0000","last_peered":"2023-12-07T10:19:01.813745+0000","last_clean":"2023-12-07T10:19:01.813745+0000","last_became_active":"2023-12-07T10:17:40.762378+0000","last_became_peered":"2023-12-07T10:17:40.762378+0000","last_unstale":"2023-12-07T10:19:01.813745+0000","last_undegraded":"2023-12-07T10:19:01.813745+0000","last_fullsized":"2023-12-07T10:19:01.813745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:42:06.193834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811313+0000","last_change":"2023-12-07T10:17:41.602356+0000","last_active":"2023-12-07T10:19:01.811313+0000","last_peered":"2023-12-07T10:19:01.811313+0000","last_clean":"2023-12-07T10:19:01.811313+0000","last_became_active":"2023-12-07T10:17:41.602025+0000","last_became_peered":"2023-12-07T10:17:41.602025+0000","last_unstale":"2023-12-07T10:19:01.811313+0000","last_undegraded":"2023-12-07T10:19:01.811313+0000","last_fullsized":"2023-12-07T10:19:01.811313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:33:04.994510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.855751+0000","last_change":"2023-12-07T10:17:41.616766+0000","last_active":"2023-12-07T10:18:01.855751+0000","last_peered":"2023-12-07T10:18:01.855751+0000","last_clean":"2023-12-07T10:18:01.855751+0000","last_became_active":"2023-12-07T10:17:41.616221+0000","last_became_peered":"2023-12-07T10:17:41.616221+0000","last_unstale":"2023-12-07T10:18:01.855751+0000","last_undegraded":"2023-12-07T10:18:01.855751+0000","last_fullsized":"2023-12-07T10:18:01.855751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:16:00.393772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810004+0000","last_change":"2023-12-07T10:17:41.613646+0000","last_active":"2023-12-07T10:19:01.810004+0000","last_peered":"2023-12-07T10:19:01.810004+0000","last_clean":"2023-12-07T10:19:01.810004+0000","last_became_active":"2023-12-07T10:17:41.613257+0000","last_became_peered":"2023-12-07T10:17:41.613257+0000","last_unstale":"2023-12-07T10:19:01.810004+0000","last_undegraded":"2023-12-07T10:19:01.810004+0000","last_fullsized":"2023-12-07T10:19:01.810004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:10:33.416955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810008+0000","last_change":"2023-12-07T10:17:41.620017+0000","last_active":"2023-12-07T10:19:01.810008+0000","last_peered":"2023-12-07T10:19:01.810008+0000","last_clean":"2023-12-07T10:19:01.810008+0000","last_became_active":"2023-12-07T10:17:41.619647+0000","last_became_peered":"2023-12-07T10:17:41.619647+0000","last_unstale":"2023-12-07T10:19:01.810008+0000","last_undegraded":"2023-12-07T10:19:01.810008+0000","last_fullsized":"2023-12-07T10:19:01.810008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:36:42.891160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803726+0000","last_change":"2023-12-07T10:17:41.610803+0000","last_active":"2023-12-07T10:19:01.803726+0000","last_peered":"2023-12-07T10:19:01.803726+0000","last_clean":"2023-12-07T10:19:01.803726+0000","last_became_active":"2023-12-07T10:17:41.610388+0000","last_became_peered":"2023-12-07T10:17:41.610388+0000","last_unstale":"2023-12-07T10:19:01.803726+0000","last_undegraded":"2023-12-07T10:19:01.803726+0000","last_fullsized":"2023-12-07T10:19:01.803726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:33:25.787281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813690+0000","last_change":"2023-12-07T10:17:41.615916+0000","last_active":"2023-12-07T10:19:01.813690+0000","last_peered":"2023-12-07T10:19:01.813690+0000","last_clean":"2023-12-07T10:19:01.813690+0000","last_became_active":"2023-12-07T10:17:41.615591+0000","last_became_peered":"2023-12-07T10:17:41.615591+0000","last_unstale":"2023-12-07T10:19:01.813690+0000","last_undegraded":"2023-12-07T10:19:01.813690+0000","last_fullsized":"2023-12-07T10:19:01.813690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:40:45.541571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.856162+0000","last_change":"2023-12-07T10:17:41.614735+0000","last_active":"2023-12-07T10:18:01.856162+0000","last_peered":"2023-12-07T10:18:01.856162+0000","last_clean":"2023-12-07T10:18:01.856162+0000","last_became_active":"2023-12-07T10:17:41.614388+0000","last_became_peered":"2023-12-07T10:17:41.614388+0000","last_unstale":"2023-12-07T10:18:01.856162+0000","last_undegraded":"2023-12-07T10:18:01.856162+0000","last_fullsized":"2023-12-07T10:18:01.856162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:10:07.196841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810348+0000","last_change":"2023-12-07T10:17:41.620059+0000","last_active":"2023-12-07T10:19:01.810348+0000","last_peered":"2023-12-07T10:19:01.810348+0000","last_clean":"2023-12-07T10:19:01.810348+0000","last_became_active":"2023-12-07T10:17:41.619770+0000","last_became_peered":"2023-12-07T10:17:41.619770+0000","last_unstale":"2023-12-07T10:19:01.810348+0000","last_undegraded":"2023-12-07T10:19:01.810348+0000","last_fullsized":"2023-12-07T10:19:01.810348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:13:43.421776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.805006+0000","last_change":"2023-12-07T10:17:41.609720+0000","last_active":"2023-12-07T10:19:01.805006+0000","last_peered":"2023-12-07T10:19:01.805006+0000","last_clean":"2023-12-07T10:19:01.805006+0000","last_became_active":"2023-12-07T10:17:41.609417+0000","last_became_peered":"2023-12-07T10:17:41.609417+0000","last_unstale":"2023-12-07T10:19:01.805006+0000","last_undegraded":"2023-12-07T10:19:01.805006+0000","last_fullsized":"2023-12-07T10:19:01.805006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:48:47.768754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807362+0000","last_change":"2023-12-07T10:17:41.620952+0000","last_active":"2023-12-07T10:19:01.807362+0000","last_peered":"2023-12-07T10:19:01.807362+0000","last_clean":"2023-12-07T10:19:01.807362+0000","last_became_active":"2023-12-07T10:17:41.620560+0000","last_became_peered":"2023-12-07T10:17:41.620560+0000","last_unstale":"2023-12-07T10:19:01.807362+0000","last_undegraded":"2023-12-07T10:19:01.807362+0000","last_fullsized":"2023-12-07T10:19:01.807362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:51:37.699511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802559+0000","last_change":"2023-12-07T10:17:41.611908+0000","last_active":"2023-12-07T10:19:01.802559+0000","last_peered":"2023-12-07T10:19:01.802559+0000","last_clean":"2023-12-07T10:19:01.802559+0000","last_became_active":"2023-12-07T10:17:41.610433+0000","last_became_peered":"2023-12-07T10:17:41.610433+0000","last_unstale":"2023-12-07T10:19:01.802559+0000","last_undegraded":"2023-12-07T10:19:01.802559+0000","last_fullsized":"2023-12-07T10:19:01.802559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:42:53.906221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.989268+0000","last_change":"2023-12-07T10:17:41.604116+0000","last_active":"2023-12-07T10:18:01.989268+0000","last_peered":"2023-12-07T10:18:01.989268+0000","last_clean":"2023-12-07T10:18:01.989268+0000","last_became_active":"2023-12-07T10:17:41.603880+0000","last_became_peered":"2023-12-07T10:17:41.603880+0000","last_unstale":"2023-12-07T10:18:01.989268+0000","last_undegraded":"2023-12-07T10:18:01.989268+0000","last_fullsized":"2023-12-07T10:18:01.989268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:56:27.343169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813422+0000","last_change":"2023-12-07T10:17:41.610497+0000","last_active":"2023-12-07T10:19:01.813422+0000","last_peered":"2023-12-07T10:19:01.813422+0000","last_clean":"2023-12-07T10:19:01.813422+0000","last_became_active":"2023-12-07T10:17:41.609956+0000","last_became_peered":"2023-12-07T10:17:41.609956+0000","last_unstale":"2023-12-07T10:19:01.813422+0000","last_undegraded":"2023-12-07T10:19:01.813422+0000","last_fullsized":"2023-12-07T10:19:01.813422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:04:03.967377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-07T10:18:01.856279+0000","last_change":"2023-12-07T10:17:41.612284+0000","last_active":"2023-12-07T10:18:01.856279+0000","last_peered":"2023-12-07T10:18:01.856279+0000","last_clean":"2023-12-07T10:18:01.856279+0000","last_became_active":"2023-12-07T10:17:41.611917+0000","last_became_peered":"2023-12-07T10:17:41.611917+0000","last_unstale":"2023-12-07T10:18:01.856279+0000","last_undegraded":"2023-12-07T10:18:01.856279+0000","last_fullsized":"2023-12-07T10:18:01.856279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:02:27.355883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":81645,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":562,"num_read_kb":371,"num_write":243,"num_write_kb":152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":244935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383708,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":498136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.434,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.72299999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.64300000000000002}]},{"osd":1,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.62,"15min":0.60499999999999998},"min":{"1min":0.505,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.40899999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.76800000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.45900000000000002,"15min":0.45000000000000001},"min":{"1min":0.36899999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.46999999999999997,"15min":0.45700000000000002},"min":{"1min":0.376,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.38400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.67900000000000005,"15min":0.67200000000000004},"min":{"1min":0.48899999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90300000000000002,"5min":1.046,"15min":1.046},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.84699999999999998,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.56799999999999995,"15min":0.55700000000000005},"min":{"1min":0.40200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80700000000000005,"5min":1.034,"15min":1.034},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.59199999999999997,"15min":0.57999999999999996},"min":{"1min":0.51300000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92900000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.442}]},{"osd":6,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.46200000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90700000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.72399999999999998,"15min":0.71599999999999997},"min":{"1min":0.60399999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91300000000000003,"5min":1.363,"15min":1.363},"last":0.79800000000000004}]},{"osd":7,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.76100000000000001,"15min":0.751},"min":{"1min":0.63700000000000001,"5min":0.501,"15min":0.501},"max":{"1min":0.97899999999999998,"5min":1.25,"15min":1.25},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.70599999999999996,"15min":0.69599999999999995},"min":{"1min":0.51500000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95399999999999996,"5min":1.085,"15min":1.085},"last":0.60399999999999998}]},{"osd":8,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.70299999999999996}]},{"osd":9,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.78000000000000003,"15min":0.76600000000000001},"min":{"1min":0.69799999999999995,"5min":0.496,"15min":0.496},"max":{"1min":0.96799999999999997,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.78700000000000003,"15min":0.77500000000000002},"min":{"1min":0.71899999999999997,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.014,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.78000000000000003}]},{"osd":10,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.751,"15min":0.74099999999999999},"min":{"1min":0.59999999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.996,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.79300000000000004,"15min":0.78300000000000003},"min":{"1min":0.70999999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.999,"5min":1.411,"15min":1.411},"last":0.754}]}]},{"osd":10,"up_from":72,"seq":309237645345,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28732,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716420,"statfs":{"total":95995035648,"available":95965614080,"internally_reserved":0,"allocated":1748992,"data_stored":1168120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.3610000000000002,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.56499999999999995}]},{"osd":1,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.48599999999999999,"15min":0.49099999999999999},"min":{"1min":0.34100000000000003,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.627,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.49299999999999999,"15min":0.495},"min":{"1min":0.36299999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.43099999999999999}]},{"osd":2,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82699999999999996,"5min":1.008,"15min":1.008},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.42699999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81299999999999994,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.46400000000000002}]},{"osd":3,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.45500000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.78300000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.67900000000000005,"15min":0.68799999999999994},"min":{"1min":0.48199999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.79000000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.60299999999999998}]},{"osd":4,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57599999999999996,"15min":0.58299999999999996},"min":{"1min":0.33300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71799999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58599999999999997,"15min":0.59099999999999997},"min":{"1min":0.40699999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.751,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.63400000000000001}]},{"osd":5,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.95499999999999996,"5min":1.22,"15min":1.22},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72699999999999998,"15min":0.72799999999999998},"min":{"1min":0.49299999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93899999999999995,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.64600000000000002}]},{"osd":6,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.747,"15min":0.751},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.89200000000000002,"5min":1.054,"15min":1.054},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78900000000000003,"15min":0.78700000000000003},"min":{"1min":0.48499999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.69099999999999995}]},{"osd":7,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.63900000000000001}]},{"osd":8,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.79600000000000004,"15min":0.80500000000000005},"min":{"1min":0.53100000000000003,"5min":0.503,"15min":0.503},"max":{"1min":0.90700000000000003,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.83099999999999996,"15min":0.84199999999999997},"min":{"1min":0.629,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.91700000000000004,"5min":1.165,"15min":1.165},"last":0.67700000000000005}]},{"osd":9,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.83399999999999996,"15min":0.83999999999999997},"min":{"1min":0.622,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.98399999999999999,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.85199999999999998,"15min":0.86599999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.70599999999999996}]},{"osd":11,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.81699999999999995,"15min":0.82099999999999995},"min":{"1min":0.56899999999999995,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.95599999999999996,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81999999999999995,"15min":0.82199999999999995},"min":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.026,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.72799999999999998}]}]},{"osd":9,"up_from":67,"seq":287762808870,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":520820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.60399999999999998,"15min":0.62},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.92600000000000005,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58099999999999996,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74299999999999999,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.51200000000000001}]},{"osd":1,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.69399999999999995,"15min":0.70699999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97999999999999998,"5min":1.899,"15min":1.899},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.77700000000000002,"15min":0.79300000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.94399999999999995,"5min":1.409,"15min":1.409},"last":0.55800000000000005}]},{"osd":2,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.80000000000000004,"15min":0.81899999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.91800000000000004,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.873,"15min":0.89800000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.92900000000000005,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":0.75800000000000001}]},{"osd":3,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.72799999999999998,"15min":0.746},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.878,"5min":1.47,"15min":1.47},"last":0.434},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.81699999999999995,"15min":0.85499999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84799999999999998,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.45200000000000001}]},{"osd":4,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.80000000000000004,"15min":0.82999999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.85099999999999998,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.88,"15min":0.90200000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.109,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.60799999999999998}]},{"osd":5,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.92800000000000005,"15min":0.95599999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.089,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.84699999999999998,"15min":0.876},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.76000000000000001,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":0.72799999999999998}]},{"osd":6,"last update":"Thu Dec 7 10:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":0.749}]},{"osd":7,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.93400000000000005,"15min":0.95799999999999996},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.0820000000000001,"5min":1.5,"15min":1.5},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.80400000000000005,"5min":1.016,"15min":1.052},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.1519999999999999,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":0.73999999999999999}]},{"osd":8,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.95599999999999996,"15min":0.97399999999999998},"min":{"1min":0.63800000000000001,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0600000000000001,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":1.0409999999999999,"15min":1.071},"min":{"1min":0.626,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.194,"5min":2.6539999999999999,"15min":2.6539999999999999},"last":0.63300000000000001}]},{"osd":10,"last update":"Thu Dec 7 10:18:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":1.02,"15min":1.052},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.2889999999999999,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.93899999999999995,"15min":0.96899999999999997},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.0189999999999999,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.57299999999999995}]},{"osd":11,"last update":"Thu Dec 7 10:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67200000000000004}]}]},{"osd":8,"up_from":59,"seq":253403070507,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":507651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55900000000000005,"15min":0.56299999999999994},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.51200000000000001}]},{"osd":1,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.59199999999999997,"15min":0.57699999999999996},"min":{"1min":0.45100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86299999999999999,"5min":1,"15min":1},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.60299999999999998,"15min":0.58299999999999996},"min":{"1min":0.52800000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.70099999999999996}]},{"osd":2,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.46200000000000002,"15min":0.44400000000000001},"min":{"1min":0.317,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.76700000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.46300000000000002,"15min":0.438},"min":{"1min":0.29699999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.43099999999999999}]},{"osd":3,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59799999999999998,"15min":0.58599999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.83199999999999996,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.72199999999999998}]},{"osd":4,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.498,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.86899999999999999,"5min":0.876,"15min":0.876},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.64400000000000002,"15min":0.63400000000000001},"min":{"1min":0.54400000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89100000000000001,"5min":1.079,"15min":1.079},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.54100000000000004,"15min":0.51700000000000002},"min":{"1min":0.34799999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56799999999999995,"15min":0.55300000000000005},"min":{"1min":0.38200000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.84199999999999997,"5min":1.024,"15min":1.024},"last":0.747}]},{"osd":6,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.67400000000000004,"15min":0.65600000000000003},"min":{"1min":0.499,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.96099999999999997,"5min":1.046,"15min":1.046},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.68999999999999995,"15min":0.66400000000000003},"min":{"1min":0.48699999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.80000000000000004}]},{"osd":7,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.68700000000000006,"15min":0.65500000000000003},"min":{"1min":0.57999999999999996,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.69199999999999995,"15min":0.67500000000000004},"min":{"1min":0.58699999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.95599999999999996,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.81299999999999994}]},{"osd":9,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.71999999999999997,"15min":0.71499999999999997},"min":{"1min":0.53900000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.74399999999999999,"15min":0.73399999999999999},"min":{"1min":0.59899999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.93100000000000005}]},{"osd":10,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.74399999999999999,"15min":0.72699999999999998},"min":{"1min":0.626,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.74299999999999999,"15min":0.72799999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.82999999999999996}]},{"osd":11,"last update":"Thu Dec 7 10:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.88200000000000001}]}]},{"osd":7,"up_from":52,"seq":223338299440,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1679360,"data_stored":1123647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.58199999999999996,"15min":0.55200000000000005},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.5369999999999999,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56599999999999995,"15min":0.56000000000000005},"min":{"1min":0.39700000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.36899999999999999}]},{"osd":1,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.434,"15min":0.432},"min":{"1min":0.29699999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60499999999999998,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45600000000000002},"min":{"1min":0.254,"5min":0.253,"15min":0.253},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.36199999999999999}]},{"osd":2,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.434,"5min":0.314,"15min":0.314},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.626,"5min":0.57899999999999996,"15min":0.57099999999999995},"min":{"1min":0.39600000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.53800000000000003}]},{"osd":3,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73499999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.496},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.45500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.79900000000000004,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.55500000000000005}]},{"osd":4,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51900000000000002,"15min":0.51600000000000001},"min":{"1min":0.315,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.748,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.438},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.52400000000000002,"15min":0.53300000000000003},"min":{"1min":0.33600000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.77400000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40500000000000003}]},{"osd":5,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63900000000000001,"15min":0.64400000000000002},"min":{"1min":0.45600000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.79200000000000004,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.443,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.84999999999999998,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.63700000000000001}]},{"osd":6,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85699999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.68600000000000005,"15min":0.69199999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94999999999999996,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.71799999999999997}]},{"osd":8,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70099999999999996,"15min":0.69599999999999995},"min":{"1min":0.46200000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90600000000000003,"5min":1.012,"15min":1.012},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72399999999999998,"15min":0.72799999999999998},"min":{"1min":0.52900000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83399999999999996,"5min":1.026,"15min":1.026},"last":0.66300000000000003}]},{"osd":9,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.73599999999999999,"15min":0.74199999999999999},"min":{"1min":0.54600000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88500000000000001,"5min":1.075,"15min":1.075},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74199999999999999,"15min":0.75},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.92300000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.75600000000000001}]},{"osd":10,"last update":"Thu Dec 7 10:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.76300000000000001,"15min":0.76600000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.96099999999999997,"5min":1.03,"15min":1.03},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73999999999999999,"15min":0.74199999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.89200000000000002,"5min":1.002,"15min":1.002},"last":0.67800000000000005}]},{"osd":11,"last update":"Thu Dec 7 10:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.70399999999999996}]}]},{"osd":6,"up_from":47,"seq":201863462964,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":512228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47399999999999998},"min":{"1min":0.34499999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.63400000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.374},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.48299999999999998,"15min":0.47199999999999998},"min":{"1min":0.38300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61499999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.65200000000000002,"15min":0.65800000000000003},"min":{"1min":0.442,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80900000000000005,"5min":1.462,"15min":1.462},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68000000000000005,"15min":0.66400000000000003},"min":{"1min":0.54600000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.73199999999999998}]},{"osd":2,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.52700000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.72999999999999998,"15min":0.752},"min":{"1min":0.51000000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.95899999999999996,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.77200000000000002}]},{"osd":3,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.63600000000000001,"15min":0.65900000000000003},"min":{"1min":0.37,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78400000000000003,"5min":1.605,"15min":1.605},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.83899999999999997,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.69599999999999995}]},{"osd":4,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73999999999999999,"15min":0.75},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.032,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.747,"15min":0.74299999999999999},"min":{"1min":0.54700000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.56799999999999995}]},{"osd":5,"last update":"Thu Dec 7 10:18:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79200000000000004,"15min":0.80100000000000005},"min":{"1min":0.51100000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0840000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77800000000000002,"15min":0.78500000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95399999999999996,"5min":1.75,"15min":1.75},"last":0.85899999999999999}]},{"osd":7,"last update":"Thu Dec 7 10:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.78000000000000003,"15min":0.78500000000000003},"min":{"1min":0.47399999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0489999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.80400000000000005,"15min":0.78800000000000003},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.97999999999999998,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.84199999999999997}]},{"osd":8,"last update":"Thu Dec 7 10:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.85999999999999999,"15min":0.86899999999999999},"min":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.569,"5min":2.569,"15min":2.569},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.88600000000000001,"15min":0.89400000000000002},"min":{"1min":0.53400000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":3.1850000000000001,"5min":3.1850000000000001,"15min":3.1850000000000001},"last":0.80900000000000005}]},{"osd":9,"last update":"Thu Dec 7 10:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.86499999999999999,"15min":0.871},"min":{"1min":0.66800000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1160000000000001,"5min":1.179,"15min":1.179},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.84499999999999997,"15min":0.84899999999999998},"min":{"1min":0.65700000000000003,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.0600000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.88300000000000001}]},{"osd":10,"last update":"Thu Dec 7 10:18:34 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.85999999999999999,"15min":0.84999999999999998},"min":{"1min":0.58299999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.3980000000000001,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.88600000000000001,"15min":0.88300000000000001},"min":{"1min":0.73899999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.89400000000000002}]},{"osd":11,"last update":"Thu Dec 7 10:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.91000000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":558941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.627,"15min":0.56499999999999995},"min":{"1min":0.39200000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.92500000000000004,"5min":6.8120000000000003,"15min":6.8120000000000003},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.57299999999999995,"15min":0.54600000000000004},"min":{"1min":0.433,"5min":0.376,"15min":0.376},"max":{"1min":0.82999999999999996,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.67900000000000005}]},{"osd":2,"last update":"Thu Dec 7 10:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.59499999999999997,"15min":0.54300000000000004},"min":{"1min":0.46899999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.87,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.55800000000000005,"15min":0.51000000000000001},"min":{"1min":0.42999999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.58999999999999997}]},{"osd":3,"last update":"Thu Dec 7 10:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60499999999999998,"15min":0.56299999999999994},"min":{"1min":0.39400000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.82399999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.37,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.67200000000000004,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Dec 7 10:18:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.437,"15min":0.39700000000000002},"min":{"1min":0.35599999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.47899999999999998,"15min":0.42099999999999999},"min":{"1min":0.36599999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.77100000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.40400000000000003}]},{"osd":5,"last update":"Thu Dec 7 10:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.65700000000000003,"15min":0.58799999999999997},"min":{"1min":0.45000000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.98099999999999998,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.60599999999999998,"15min":0.56200000000000006},"min":{"1min":0.51600000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82099999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.627}]},{"osd":6,"last update":"Thu Dec 7 10:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.69299999999999995,"15min":0.66700000000000004},"min":{"1min":0.49199999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93700000000000006,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.65400000000000003,"15min":0.64200000000000002},"min":{"1min":0.52900000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.85699999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.66100000000000003}]},{"osd":7,"last update":"Thu Dec 7 10:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.54900000000000004,"15min":0.52600000000000002},"min":{"1min":0.38100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.438},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.67300000000000004,"15min":0.63400000000000001},"min":{"1min":0.46300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93799999999999994,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.64000000000000001}]},{"osd":8,"last update":"Thu Dec 7 10:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.77100000000000002,"15min":0.73499999999999999},"min":{"1min":0.70999999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.026,"5min":1.175,"15min":1.175},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67100000000000004,"15min":0.64500000000000002},"min":{"1min":0.55500000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86699999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.60899999999999999}]},{"osd":9,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.76300000000000001,"15min":0.754},"min":{"1min":0.58299999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.85399999999999998,"15min":0.84899999999999998},"min":{"1min":0.65700000000000003,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2050000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.76700000000000002}]},{"osd":10,"last update":"Thu Dec 7 10:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.73999999999999999,"15min":0.73699999999999999},"min":{"1min":0.63200000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.75700000000000001,"15min":0.751},"min":{"1min":0.61099999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.747}]},{"osd":11,"last update":"Thu Dec 7 10:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.871},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.78600000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1081344,"data_stored":513684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.59899999999999998,"15min":0.57199999999999995},"min":{"1min":0.443,"5min":0.375,"15min":0.375},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.41899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.57499999999999996}]},{"osd":2,"last update":"Thu Dec 7 10:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.63400000000000001,"15min":0.59599999999999997},"min":{"1min":0.44500000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.3420000000000001,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.64600000000000002,"15min":0.60499999999999998},"min":{"1min":0.42299999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":3.4140000000000001,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.70899999999999996}]},{"osd":3,"last update":"Thu Dec 7 10:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":3.484,"5min":3.484,"15min":3.484},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":3.1379999999999999,"5min":3.1379999999999999,"15min":3.1379999999999999},"last":0.46899999999999997}]},{"osd":4,"last update":"Thu Dec 7 10:18:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.59999999999999998,"15min":0.56599999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91700000000000004,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.623,"15min":0.60499999999999998},"min":{"1min":0.46000000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82799999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Dec 7 10:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.63900000000000001,"15min":0.624},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91000000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.626,"15min":0.624},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85399999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.628}]},{"osd":6,"last update":"Thu Dec 7 10:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.627,"15min":0.625},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79000000000000004,"5min":1.02,"15min":1.02},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64100000000000001,"15min":0.63400000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.51700000000000002}]},{"osd":7,"last update":"Thu Dec 7 10:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.622},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":3.0430000000000001,"5min":3.0430000000000001,"15min":3.0430000000000001},"last":0.66500000000000004}]},{"osd":8,"last update":"Thu Dec 7 10:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.81499999999999995,"15min":0.80500000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":3.75,"5min":3.75,"15min":3.75},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.80600000000000005,"15min":0.81100000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.75900000000000001}]},{"osd":9,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.84899999999999998,"15min":0.86199999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0609999999999999,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.82999999999999996,"15min":0.83999999999999997},"min":{"1min":0.501,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.746}]},{"osd":10,"last update":"Thu Dec 7 10:18:35 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.79900000000000004,"15min":0.77900000000000003},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":3.0979999999999999,"5min":3.0979999999999999,"15min":3.0979999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.88,"5min":0.85199999999999998,"15min":0.84699999999999998},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.383,"5min":2.383,"15min":2.383},"last":0.80400000000000005}]},{"osd":11,"last update":"Thu Dec 7 10:18:55 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.84599999999999997,"15min":0.81799999999999995},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.7879999999999998,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.82399999999999995,"15min":0.80200000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.1749999999999998,"5min":3.1749999999999998,"15min":3.1749999999999998},"last":0.72199999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":588201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.51000000000000001},"min":{"1min":0.41599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55100000000000005,"15min":0.52000000000000002},"min":{"1min":0.35899999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.69399999999999995}]},{"osd":1,"last update":"Thu Dec 7 10:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.58099999999999996,"15min":0.52800000000000002},"min":{"1min":0.42199999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.56200000000000006,"15min":0.52300000000000002},"min":{"1min":0.39800000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79700000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61799999999999999}]},{"osd":3,"last update":"Thu Dec 7 10:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.60899999999999999,"15min":0.56000000000000005},"min":{"1min":0.41399999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.432},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60399999999999998,"15min":0.56399999999999995},"min":{"1min":0.42699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.95299999999999996,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.45500000000000002}]},{"osd":4,"last update":"Thu Dec 7 10:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.88500000000000001}]},{"osd":5,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.47899999999999998,"15min":0.443},"min":{"1min":0.39400000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.77200000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.501,"15min":0.47199999999999998},"min":{"1min":0.379,"5min":0.307,"15min":0.307},"max":{"1min":0.81299999999999994,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.53400000000000003}]},{"osd":6,"last update":"Thu Dec 7 10:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.68100000000000005,"15min":0.67000000000000004},"min":{"1min":0.501,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.91800000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.67200000000000004,"15min":0.65400000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.79600000000000004}]},{"osd":7,"last update":"Thu Dec 7 10:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.67000000000000004}]},{"osd":8,"last update":"Thu Dec 7 10:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.625,"15min":0.60399999999999998},"min":{"1min":0.53400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.751},{"interface":"front","average":{"1min":0.755,"5min":0.68000000000000005,"15min":0.66400000000000003},"min":{"1min":0.60799999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0229999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.65200000000000002}]},{"osd":9,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.78800000000000003,"15min":0.78400000000000003},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0389999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.83899999999999997}]},{"osd":10,"last update":"Thu Dec 7 10:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.76600000000000001,"15min":0.752},"min":{"1min":0.57999999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.871},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.77800000000000002,"15min":0.76600000000000001},"min":{"1min":0.57099999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.82199999999999995}]},{"osd":11,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.73499999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149762,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":511605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44900000000000001,"15min":0.40300000000000002},"min":{"1min":0.33500000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.624,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.46700000000000003,"15min":0.41299999999999998},"min":{"1min":0.35599999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.72599999999999998,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.64500000000000002}]},{"osd":1,"last update":"Thu Dec 7 10:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.68899999999999995,"15min":0.58699999999999997},"min":{"1min":0.55300000000000005,"5min":0.438,"15min":0.438},"max":{"1min":0.98199999999999998,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.56100000000000005},"min":{"1min":0.41899999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89000000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.79300000000000004}]},{"osd":2,"last update":"Thu Dec 7 10:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.59399999999999997},"min":{"1min":0.44600000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86799999999999999,"5min":1.087,"15min":1.087},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68200000000000005,"15min":0.61799999999999999},"min":{"1min":0.53800000000000003,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91500000000000004,"5min":1.177,"15min":1.177},"last":0.84799999999999998}]},{"osd":4,"last update":"Thu Dec 7 10:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.68600000000000005,"15min":0.62},"min":{"1min":0.503,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.018,"5min":1.208,"15min":1.208},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63200000000000001,"15min":0.58999999999999997},"min":{"1min":0.46999999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83899999999999997,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.747}]},{"osd":5,"last update":"Thu Dec 7 10:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.623,"15min":0.61099999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.82099999999999995,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.71399999999999997,"15min":0.69599999999999995},"min":{"1min":0.60499999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96199999999999997,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.78100000000000003}]},{"osd":6,"last update":"Thu Dec 7 10:18:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.46999999999999997,"5min":0.377,"15min":0.377},"max":{"1min":1.0369999999999999,"5min":1.282,"15min":1.282},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60999999999999999,"15min":0.61799999999999999},"min":{"1min":0.39700000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71199999999999997,"5min":1.292,"15min":1.292},"last":0.56100000000000005}]},{"osd":7,"last update":"Thu Dec 7 10:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78600000000000003,"15min":0.76900000000000002},"min":{"1min":0.59599999999999997,"5min":0.498,"15min":0.498},"max":{"1min":1.0780000000000001,"5min":1.323,"15min":1.323},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.56799999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96999999999999997,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.83599999999999997}]},{"osd":8,"last update":"Thu Dec 7 10:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.872}]},{"osd":9,"last update":"Thu Dec 7 10:18:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.88,"15min":0.88600000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.224,"5min":1.242,"15min":1.242},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.80100000000000005,"15min":0.81000000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.86099999999999999}]},{"osd":10,"last update":"Thu Dec 7 10:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.78200000000000003,"15min":0.79100000000000004},"min":{"1min":0.59099999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.90500000000000003,"5min":1.335,"15min":1.335},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.878,"5min":0.92100000000000004,"15min":0.92800000000000005},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.1180000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.80600000000000005}]},{"osd":11,"last update":"Thu Dec 7 10:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.89100000000000001}]}]},{"osd":4,"up_from":34,"seq":146028888125,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":498032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.52100000000000002,"15min":0.49099999999999999},"min":{"1min":0.41499999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76500000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Dec 7 10:18:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.41099999999999998,"15min":0.39000000000000001},"min":{"1min":0.254,"5min":0.215,"15min":0.215},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.41499999999999998,"15min":0.38900000000000001},"min":{"1min":0.308,"5min":0.22,"15min":0.22},"max":{"1min":0.55200000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.53400000000000003}]},{"osd":2,"last update":"Thu Dec 7 10:18:17 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58899999999999997,"15min":0.56000000000000005},"min":{"1min":0.42699999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.875,"5min":1.323,"15min":1.323},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58499999999999996,"15min":0.54300000000000004},"min":{"1min":0.45700000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84299999999999997,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.59999999999999998}]},{"osd":3,"last update":"Thu Dec 7 10:18:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58099999999999996,"15min":0.54300000000000004},"min":{"1min":0.437,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.874,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.60899999999999999,"15min":0.56999999999999995},"min":{"1min":0.45700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91200000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.79400000000000004}]},{"osd":5,"last update":"Thu Dec 7 10:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.63100000000000001,"15min":0.59299999999999997},"min":{"1min":0.55000000000000004,"5min":0.374,"15min":0.374},"max":{"1min":1.0009999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.61699999999999999,"15min":0.58399999999999996},"min":{"1min":0.47599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82699999999999996,"5min":1.258,"15min":1.258},"last":0.63100000000000001}]},{"osd":6,"last update":"Thu Dec 7 10:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.41399999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64800000000000002,"15min":0.63600000000000001},"min":{"1min":0.501,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91900000000000004,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.69299999999999995}]},{"osd":7,"last update":"Thu Dec 7 10:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.55000000000000004,"15min":0.51800000000000002},"min":{"1min":0.38100000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76800000000000002,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.35599999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.84399999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.72699999999999998}]},{"osd":8,"last update":"Thu Dec 7 10:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.71299999999999997,"15min":0.67900000000000005},"min":{"1min":0.57099999999999995,"5min":0.432,"15min":0.432},"max":{"1min":0.90000000000000002,"5min":1.381,"15min":1.381},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71199999999999997,"15min":0.68200000000000005},"min":{"1min":0.59099999999999997,"5min":0.502,"15min":0.502},"max":{"1min":0.872,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.75}]},{"osd":9,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.76700000000000002,"15min":0.76900000000000002},"min":{"1min":0.57399999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.98799999999999999,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.77100000000000002,"15min":0.75800000000000001},"min":{"1min":0.58699999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.874}]},{"osd":10,"last update":"Thu Dec 7 10:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.60999999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.54600000000000004,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.80500000000000005}]},{"osd":11,"last update":"Thu Dec 7 10:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.83599999999999997}]}]},{"osd":5,"up_from":39,"seq":167503724601,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1675264,"data_stored":1109833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.54700000000000004,"15min":0.51700000000000002},"min":{"1min":0.377,"5min":0.318,"15min":0.318},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54700000000000004,"15min":0.52200000000000002},"min":{"1min":0.37,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79000000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.41899999999999998}]},{"osd":1,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.56499999999999995,"15min":0.51700000000000002},"min":{"1min":0.505,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.59799999999999998,"15min":0.54400000000000004},"min":{"1min":0.52200000000000002,"5min":0.312,"15min":0.312},"max":{"1min":3.0070000000000001,"5min":3.0070000000000001,"15min":3.0070000000000001},"last":0.48599999999999999}]},{"osd":2,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.42399999999999999,"15min":0.38600000000000001},"min":{"1min":0.40200000000000002,"5min":0.246,"15min":0.246},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.443},{"interface":"front","average":{"1min":0.628,"5min":0.45300000000000001,"15min":0.39900000000000002},"min":{"1min":0.35699999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.851,"5min":1.851,"15min":1.851},"last":0.39400000000000002}]},{"osd":3,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.57999999999999996,"15min":0.53900000000000003},"min":{"1min":0.43099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.59099999999999997,"15min":0.55600000000000005},"min":{"1min":0.40500000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.57199999999999995}]},{"osd":4,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.63100000000000001,"15min":0.57299999999999995},"min":{"1min":0.58199999999999996,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.621,"15min":0.57899999999999996},"min":{"1min":0.47699999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.47099999999999997}]},{"osd":6,"last update":"Thu Dec 7 10:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.61799999999999999,"15min":0.58599999999999997},"min":{"1min":0.42999999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.69499999999999995,"15min":0.63200000000000001},"min":{"1min":0.45000000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":5.9720000000000004,"5min":5.9720000000000004,"15min":5.9720000000000004},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Dec 7 10:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.66100000000000003,"15min":0.626},"min":{"1min":0.38400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.66600000000000004,"15min":0.63800000000000001},"min":{"1min":0.443,"5min":0.437,"15min":0.437},"max":{"1min":0.96999999999999997,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.69399999999999995}]},{"osd":8,"last update":"Thu Dec 7 10:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.65000000000000002,"15min":0.628},"min":{"1min":0.45600000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.052,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.61899999999999999,"15min":0.59899999999999998},"min":{"1min":0.5,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92000000000000004,"5min":1.135,"15min":1.135},"last":0.58999999999999997}]},{"osd":9,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.78700000000000003,"15min":0.755},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":4.532,"5min":4.532,"15min":4.532},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76400000000000001,"15min":0.76300000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.80800000000000005}]},{"osd":10,"last update":"Thu Dec 7 10:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.84799999999999998,"15min":0.84099999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.7649999999999999,"5min":2.927,"15min":2.927},"last":0.748},{"interface":"front","average":{"1min":1.1519999999999999,"5min":0.85899999999999999,"15min":0.81000000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":5.7709999999999999,"5min":5.7709999999999999,"15min":5.7709999999999999},"last":0.76400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":76695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":13202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":13912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-07T10:19:07.380 INFO:tasks.cephadm.ceph_manager.ceph:PG 4.1b is not active+clean 2023-12-07T10:19:07.380 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '4.1b', 'version': "0'0", 'reported_seq': 16, 'reported_epoch': 91, 'state': 'peering', 'last_fresh': '2023-12-07T10:18:02.254326+0000', 'last_change': '2023-12-07T10:19:01.805042+0000', 'last_active': '2023-12-07T10:19:01.805042+0000', 'last_peered': '2023-12-07T10:18:02.254326+0000', 'last_clean': '2023-12-07T10:18:02.254326+0000', 'last_became_active': '2023-12-07T10:17:45.799072+0000', 'last_became_peered': '2023-12-07T10:17:45.799072+0000', 'last_unstale': '2023-12-07T10:18:02.254326+0000', 'last_undegraded': '2023-12-07T10:18:02.254326+0000', 'last_fullsized': '2023-12-07T10:18:02.254326+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 88, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-07T10:17:44.766631+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-07T10:17:44.766631+0000', 'last_clean_scrub_stamp': '2023-12-07T10:17:44.766631+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3, 10, 5], 'acting': [4, 3, 10, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3, 5, 9, 10], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-07T10:19:07.380 INFO:tasks.cephadm.ceph_manager.ceph:PG 4.17 is not active+clean 2023-12-07T10:19:07.380 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '4.17', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 92, 'state': 'peering', 'last_fresh': '2023-12-07T10:17:45.779804+0000', 'last_change': '2023-12-07T10:19:02.353784+0000', 'last_active': '2023-12-07T10:17:44.766631+0000', 'last_peered': '2023-12-07T10:17:44.766631+0000', 'last_clean': '2023-12-07T10:17:44.766631+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-07T10:17:45.779804+0000', 'last_undegraded': '2023-12-07T10:17:45.779804+0000', 'last_fullsized': '2023-12-07T10:17:45.779804+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 88, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-07T10:17:44.766631+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-07T10:17:44.766631+0000', 'last_clean_scrub_stamp': '2023-12-07T10:17:44.766631+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [8, 7, 0, 3], 'acting': [8, 7, 0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 8, 'acting_primary': 8, 'purged_snaps': []} 2023-12-07T10:19:07.381 INFO:tasks.cephadm.ceph_manager.ceph:PG 4.f is not active+clean 2023-12-07T10:19:07.381 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '4.f', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 92, 'state': 'peering', 'last_fresh': '2023-12-07T10:17:45.780383+0000', 'last_change': '2023-12-07T10:19:02.353800+0000', 'last_active': '2023-12-07T10:17:44.766631+0000', 'last_peered': '2023-12-07T10:17:44.766631+0000', 'last_clean': '2023-12-07T10:17:44.766631+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-07T10:17:45.780383+0000', 'last_undegraded': '2023-12-07T10:17:45.780383+0000', 'last_fullsized': '2023-12-07T10:17:45.780383+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 88, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-07T10:17:44.766631+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-07T10:17:44.766631+0000', 'last_clean_scrub_stamp': '2023-12-07T10:17:44.766631+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [8, 3, 2, 5], 'acting': [8, 3, 2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 8, 'acting_primary': 8, 'purged_snaps': []} 2023-12-07T10:19:07.381 INFO:tasks.cephadm.ceph_manager.ceph:PG 4.a is not active+clean 2023-12-07T10:19:07.381 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '4.a', 'version': "0'0", 'reported_seq': 16, 'reported_epoch': 91, 'state': 'peering', 'last_fresh': '2023-12-07T10:18:01.989081+0000', 'last_change': '2023-12-07T10:19:01.811088+0000', 'last_active': '2023-12-07T10:19:01.811088+0000', 'last_peered': '2023-12-07T10:18:01.989081+0000', 'last_clean': '2023-12-07T10:18:01.989081+0000', 'last_became_active': '2023-12-07T10:17:45.790785+0000', 'last_became_peered': '2023-12-07T10:17:45.790785+0000', 'last_unstale': '2023-12-07T10:18:01.989081+0000', 'last_undegraded': '2023-12-07T10:18:01.989081+0000', 'last_fullsized': '2023-12-07T10:18:01.989081+0000', 'mapping_epoch': 92, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 88, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-07T10:17:44.766631+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-07T10:17:44.766631+0000', 'last_clean_scrub_stamp': '2023-12-07T10:17:44.766631+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 8, 6, 7], 'acting': [1, 8, 6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5, 6, 7, 8], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-07T10:19:07.381 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph status --format=json 2023-12-07T10:19:08.156 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:19:08.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:07 smithi119 ceph-mon[97908]: pgmap v326: 129 pgs: 4 peering, 125 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2023-12-07T10:19:08.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:07 smithi119 ceph-mon[97908]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:19:08.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:07 smithi159 ceph-mon[134726]: pgmap v326: 129 pgs: 4 peering, 125 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2023-12-07T10:19:08.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:07 smithi159 ceph-mon[134726]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:19:08.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:07 smithi153 ceph-mon[133625]: pgmap v326: 129 pgs: 4 peering, 125 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2023-12-07T10:19:08.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:07 smithi153 ceph-mon[133625]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:19:09.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:08 smithi119 ceph-mon[97908]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 4 pgs peering) 2023-12-07T10:19:09.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:08 smithi119 ceph-mon[97908]: Cluster is now healthy 2023-12-07T10:19:09.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:08 smithi159 ceph-mon[134726]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 4 pgs peering) 2023-12-07T10:19:09.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:08 smithi159 ceph-mon[134726]: Cluster is now healthy 2023-12-07T10:19:09.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:08 smithi153 ceph-mon[133625]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 4 pgs peering) 2023-12-07T10:19:09.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:08 smithi153 ceph-mon[133625]: Cluster is now healthy 2023-12-07T10:19:09.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.871+0000 7f70b618a700 1 -- 172.21.15.119:0/296454440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b0104180 msgr2=0x7f70b0110690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:09.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.871+0000 7f70b618a700 1 --2- 172.21.15.119:0/296454440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b0104180 0x7f70b0110690 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f70a800b0a0 tx=0x7f70a800b3b0 comp rx=0 tx=0).stop 2023-12-07T10:19:09.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.872+0000 7f70b618a700 1 -- 172.21.15.119:0/296454440 shutdown_connections 2023-12-07T10:19:09.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.872+0000 7f70b618a700 1 --2- 172.21.15.119:0/296454440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b0104180 0x7f70b0110690 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:09.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.872+0000 7f70b618a700 1 --2- 172.21.15.119:0/296454440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70b01097c0 0x7f70b0109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:09.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.872+0000 7f70b618a700 1 --2- 172.21.15.119:0/296454440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70b01037c0 0x7f70b0103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:09.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.872+0000 7f70b618a700 1 -- 172.21.15.119:0/296454440 >> 172.21.15.119:0/296454440 conn(0x7f70b00ff2d0 msgr2=0x7f70b01016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:09.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.872+0000 7f70b618a700 1 -- 172.21.15.119:0/296454440 shutdown_connections 2023-12-07T10:19:09.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.872+0000 7f70b618a700 1 -- 172.21.15.119:0/296454440 wait complete. 2023-12-07T10:19:09.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.873+0000 7f70b618a700 1 Processor -- start 2023-12-07T10:19:09.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.873+0000 7f70b618a700 1 -- start start 2023-12-07T10:19:09.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.873+0000 7f70b618a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b01037c0 0x7f70b019fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:09.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.873+0000 7f70b618a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70b0104180 0x7f70b01a0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:09.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.873+0000 7f70b618a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70b01097c0 0x7f70b0199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.873+0000 7f70b618a700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b019a320 con 0x7f70b01037c0 2023-12-07T10:19:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.873+0000 7f70b618a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b019a490 con 0x7f70b01097c0 2023-12-07T10:19:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.873+0000 7f70b618a700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b019a650 con 0x7f70b0104180 2023-12-07T10:19:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.874+0000 7f70af7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b01037c0 0x7f70b019fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.874+0000 7f70af7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b01037c0 0x7f70b019fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49060/0 (socket says 172.21.15.119:49060) 2023-12-07T10:19:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.874+0000 7f70af7fe700 1 -- 172.21.15.119:0/206791784 learned_addr learned my addr 172.21.15.119:0/206791784 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.874+0000 7f70aeffd700 1 --2- 172.21.15.119:0/206791784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70b0104180 0x7f70b01a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.874+0000 7f70affff700 1 --2- 172.21.15.119:0/206791784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70b01097c0 0x7f70b0199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.874+0000 7f70af7fe700 1 -- 172.21.15.119:0/206791784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70b0104180 msgr2=0x7f70b01a0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.874+0000 7f70af7fe700 1 --2- 172.21.15.119:0/206791784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70b0104180 0x7f70b01a0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:09.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.874+0000 7f70af7fe700 1 -- 172.21.15.119:0/206791784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70b01097c0 msgr2=0x7f70b0199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:09.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.874+0000 7f70af7fe700 1 --2- 172.21.15.119:0/206791784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70b01097c0 0x7f70b0199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:09.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.874+0000 7f70af7fe700 1 -- 172.21.15.119:0/206791784 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a8009d00 con 0x7f70b01037c0 2023-12-07T10:19:09.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.874+0000 7f70af7fe700 1 --2- 172.21.15.119:0/206791784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b01037c0 0x7f70b019fac0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f70a000e9a0 tx=0x7f70a000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:09.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.875+0000 7f70acff9700 1 -- 172.21.15.119:0/206791784 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0009790 con 0x7f70b01037c0 2023-12-07T10:19:09.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.875+0000 7f70acff9700 1 -- 172.21.15.119:0/206791784 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70a0010430 con 0x7f70b01037c0 2023-12-07T10:19:09.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.875+0000 7f70acff9700 1 -- 172.21.15.119:0/206791784 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0018680 con 0x7f70b01037c0 2023-12-07T10:19:09.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.875+0000 7f70b618a700 1 -- 172.21.15.119:0/206791784 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b019a930 con 0x7f70b01037c0 2023-12-07T10:19:09.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.875+0000 7f70b618a700 1 -- 172.21.15.119:0/206791784 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b01a6a40 con 0x7f70b01037c0 2023-12-07T10:19:09.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.877+0000 7f70b618a700 1 -- 172.21.15.119:0/206791784 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b010bde0 con 0x7f70b01037c0 2023-12-07T10:19:09.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.878+0000 7f70acff9700 1 -- 172.21.15.119:0/206791784 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f70a00105a0 con 0x7f70b01037c0 2023-12-07T10:19:09.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.880+0000 7f70acff9700 1 --2- 172.21.15.119:0/206791784 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f70980a1ba0 0x7f70980a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:09.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.880+0000 7f70acff9700 1 -- 172.21.15.119:0/206791784 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f70a0014070 con 0x7f70b01037c0 2023-12-07T10:19:09.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.880+0000 7f70aeffd700 1 --2- 172.21.15.119:0/206791784 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f70980a1ba0 0x7f70980a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:09.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.880+0000 7f70aeffd700 1 --2- 172.21.15.119:0/206791784 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f70980a1ba0 0x7f70980a4060 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f70b019b2e0 tx=0x7f70a4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:09.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:09.883+0000 7f70acff9700 1 -- 172.21.15.119:0/206791784 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f70a00522f0 con 0x7f70b01037c0 2023-12-07T10:19:09.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:09 smithi119 ceph-mon[97908]: pgmap v327: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-07T10:19:10.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:10.164+0000 7f70b618a700 1 -- 172.21.15.119:0/206791784 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f70b019b080 con 0x7f70b01037c0 2023-12-07T10:19:10.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:10.164+0000 7f70acff9700 1 -- 172.21.15.119:0/206791784 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+2975 (secure 0 0 0) 0x7f70a0096b40 con 0x7f70b01037c0 2023-12-07T10:19:10.166 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:19:10.166 INFO:teuthology.orchestra.run.smithi119.stdout:{"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":429,"monmap":{"epoch":3,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":93,"num_osds":12,"num_up_osds":12,"osd_up_since":1701944210,"num_in_osds":12,"osd_in_since":1701944193,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":129}],"num_pgs":129,"num_pools":4,"num_objects":102,"data_bytes":672259,"bytes_used":346935296,"bytes_avail":1151593492480,"bytes_total":1151940427776,"read_bytes_sec":3838,"read_op_per_sec":7,"write_op_per_sec":0},"fsmap":{"epoch":29,"id":1,"up":5,"in":5,"max":5,"by_rank":[{"filesystem_id":1,"rank":4,"name":"a","status":"up:standby-replay","gid":14535},{"filesystem_id":1,"rank":3,"name":"l","status":"up:standby-replay","gid":24445},{"filesystem_id":1,"rank":2,"name":"k","status":"up:standby-replay","gid":24452},{"filesystem_id":1,"rank":4,"name":"e","status":"up:active","gid":24488},{"filesystem_id":1,"rank":3,"name":"f","status":"up:active","gid":24505},{"filesystem_id":1,"rank":2,"name":"c","status":"up:active","gid":24517},{"filesystem_id":1,"rank":1,"name":"h","status":"up:standby-replay","gid":24518},{"filesystem_id":1,"rank":0,"name":"i","status":"up:standby-replay","gid":24538},{"filesystem_id":1,"rank":1,"name":"g","status":"up:active","gid":24542},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24566}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.119:8443/"}},"servicemap":{"epoch":23,"modified":"2023-12-07T10:18:44.685334+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"g":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"k":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"l":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-12-07T10:19:10.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:10.167+0000 7f70b618a700 1 -- 172.21.15.119:0/206791784 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f70980a1ba0 msgr2=0x7f70980a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:10.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:10.167+0000 7f70b618a700 1 --2- 172.21.15.119:0/206791784 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f70980a1ba0 0x7f70980a4060 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f70b019b2e0 tx=0x7f70a4009260 comp rx=0 tx=0).stop 2023-12-07T10:19:10.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:10.167+0000 7f70b618a700 1 -- 172.21.15.119:0/206791784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b01037c0 msgr2=0x7f70b019fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:10.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:10.167+0000 7f70b618a700 1 --2- 172.21.15.119:0/206791784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b01037c0 0x7f70b019fac0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f70a000e9a0 tx=0x7f70a000ecb0 comp rx=0 tx=0).stop 2023-12-07T10:19:10.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:10.167+0000 7f70b618a700 1 -- 172.21.15.119:0/206791784 shutdown_connections 2023-12-07T10:19:10.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:10.167+0000 7f70b618a700 1 --2- 172.21.15.119:0/206791784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70b01097c0 0x7f70b0199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:10.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:10.167+0000 7f70b618a700 1 --2- 172.21.15.119:0/206791784 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f70980a1ba0 0x7f70980a4060 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:10.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:10.167+0000 7f70b618a700 1 --2- 172.21.15.119:0/206791784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b01037c0 0x7f70b019fac0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:10.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:10.167+0000 7f70b618a700 1 --2- 172.21.15.119:0/206791784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70b0104180 0x7f70b01a0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:10.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:10.167+0000 7f70b618a700 1 -- 172.21.15.119:0/206791784 >> 172.21.15.119:0/206791784 conn(0x7f70b00ff2d0 msgr2=0x7f70b00ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:10.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:10.167+0000 7f70b618a700 1 -- 172.21.15.119:0/206791784 shutdown_connections 2023-12-07T10:19:10.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:10.167+0000 7f70b618a700 1 -- 172.21.15.119:0/206791784 wait complete. 2023-12-07T10:19:10.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:09 smithi159 ceph-mon[134726]: pgmap v327: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-07T10:19:10.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:09 smithi153 ceph-mon[133625]: pgmap v327: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-07T10:19:10.656 INFO:tasks.cephadm.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-12-07T10:19:10.657 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph pg dump --format=json 2023-12-07T10:19:10.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:10 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/206791784' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-12-07T10:19:11.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:10 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/206791784' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-12-07T10:19:11.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:10 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/206791784' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-12-07T10:19:11.434 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:19:11.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:11 smithi119 ceph-mon[97908]: pgmap v328: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2023-12-07T10:19:12.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:11 smithi159 ceph-mon[134726]: pgmap v328: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2023-12-07T10:19:12.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:11 smithi153 ceph-mon[133625]: pgmap v328: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2023-12-07T10:19:12.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.947+0000 7f3325581700 1 -- 172.21.15.119:0/2296711304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3320074160 msgr2=0x7f332010db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:12.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.947+0000 7f3325581700 1 --2- 172.21.15.119:0/2296711304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3320074160 0x7f332010db30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f33140098f0 tx=0x7f3314009c00 comp rx=0 tx=0).stop 2023-12-07T10:19:12.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.947+0000 7f3325581700 1 -- 172.21.15.119:0/2296711304 shutdown_connections 2023-12-07T10:19:12.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.947+0000 7f3325581700 1 --2- 172.21.15.119:0/2296711304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f332010e070 0x7f3320110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:12.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.947+0000 7f3325581700 1 --2- 172.21.15.119:0/2296711304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3320073840 0x7f3320073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:12.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.947+0000 7f3325581700 1 --2- 172.21.15.119:0/2296711304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3320074160 0x7f332010db30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:12.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.947+0000 7f3325581700 1 -- 172.21.15.119:0/2296711304 >> 172.21.15.119:0/2296711304 conn(0x7f3320078800 msgr2=0x7f3320078c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:12.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.948+0000 7f3325581700 1 -- 172.21.15.119:0/2296711304 shutdown_connections 2023-12-07T10:19:12.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.948+0000 7f3325581700 1 -- 172.21.15.119:0/2296711304 wait complete. 2023-12-07T10:19:12.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.949+0000 7f3325581700 1 Processor -- start 2023-12-07T10:19:12.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.949+0000 7f3325581700 1 -- start start 2023-12-07T10:19:12.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.949+0000 7f3325581700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3320073840 0x7f3320199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:12.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.949+0000 7f3325581700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3320074160 0x7f332019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:12.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.949+0000 7f3325581700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f332010e070 0x7f332019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:12.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.949+0000 7f3325581700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332019ec70 con 0x7f332010e070 2023-12-07T10:19:12.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.949+0000 7f3325581700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332019ede0 con 0x7f3320074160 2023-12-07T10:19:12.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.949+0000 7f3325581700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332019efa0 con 0x7f3320073840 2023-12-07T10:19:12.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.949+0000 7f331e7fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3320074160 0x7f332019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:12.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.949+0000 7f331e7fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3320074160 0x7f332019a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:32852/0 (socket says 172.21.15.119:32852) 2023-12-07T10:19:12.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.949+0000 7f331e7fc700 1 -- 172.21.15.119:0/2870507599 learned_addr learned my addr 172.21.15.119:0/2870507599 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:12.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.949+0000 7f331f7fe700 1 --2- 172.21.15.119:0/2870507599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f332010e070 0x7f332019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:12.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.950+0000 7f331effd700 1 --2- 172.21.15.119:0/2870507599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3320073840 0x7f3320199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:12.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.950+0000 7f331e7fc700 1 -- 172.21.15.119:0/2870507599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3320073840 msgr2=0x7f3320199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:12.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.950+0000 7f331e7fc700 1 --2- 172.21.15.119:0/2870507599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3320073840 0x7f3320199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:12.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.950+0000 7f331e7fc700 1 -- 172.21.15.119:0/2870507599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f332010e070 msgr2=0x7f332019e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:12.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.950+0000 7f331e7fc700 1 --2- 172.21.15.119:0/2870507599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f332010e070 0x7f332019e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:12.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.950+0000 7f331e7fc700 1 -- 172.21.15.119:0/2870507599 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3318009d00 con 0x7f3320074160 2023-12-07T10:19:12.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.950+0000 7f331e7fc700 1 --2- 172.21.15.119:0/2870507599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3320074160 0x7f332019a300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3314005950 tx=0x7f331400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:12.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.950+0000 7f3307fff700 1 -- 172.21.15.119:0/2870507599 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331401c070 con 0x7f3320074160 2023-12-07T10:19:12.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.950+0000 7f3307fff700 1 -- 172.21.15.119:0/2870507599 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f331400baa0 con 0x7f3320074160 2023-12-07T10:19:12.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.950+0000 7f3325581700 1 -- 172.21.15.119:0/2870507599 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3314009520 con 0x7f3320074160 2023-12-07T10:19:12.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.950+0000 7f3307fff700 1 -- 172.21.15.119:0/2870507599 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33140038e0 con 0x7f3320074160 2023-12-07T10:19:12.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.951+0000 7f3325581700 1 -- 172.21.15.119:0/2870507599 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33201a6910 con 0x7f3320074160 2023-12-07T10:19:12.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.951+0000 7f3325581700 1 -- 172.21.15.119:0/2870507599 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3320109280 con 0x7f3320074160 2023-12-07T10:19:12.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.953+0000 7f3307fff700 1 -- 172.21.15.119:0/2870507599 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f3314020030 con 0x7f3320074160 2023-12-07T10:19:12.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.955+0000 7f3307fff700 1 --2- 172.21.15.119:0/2870507599 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f33080a1c20 0x7f33080a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:12.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.955+0000 7f3307fff700 1 -- 172.21.15.119:0/2870507599 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f33140ce970 con 0x7f3320074160 2023-12-07T10:19:12.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.955+0000 7f331effd700 1 --2- 172.21.15.119:0/2870507599 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f33080a1c20 0x7f33080a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:12.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.956+0000 7f331effd700 1 --2- 172.21.15.119:0/2870507599 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f33080a1c20 0x7f33080a40e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3310009c60 tx=0x7f3310009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:12.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:12.957+0000 7f3307fff700 1 -- 172.21.15.119:0/2870507599 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f33140978f0 con 0x7f3320074160 2023-12-07T10:19:13.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:13.151+0000 7f3325581700 1 -- 172.21.15.119:0/2870507599 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f332004f2d0 con 0x7f33080a1c20 2023-12-07T10:19:13.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:13.164+0000 7f3307fff700 1 -- 172.21.15.119:0/2870507599 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426852 (secure 0 0 0) 0x7f332004f2d0 con 0x7f33080a1c20 2023-12-07T10:19:13.174 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:19:13.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:13.170+0000 7f3325581700 1 -- 172.21.15.119:0/2870507599 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f33080a1c20 msgr2=0x7f33080a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:13.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:13.170+0000 7f3325581700 1 --2- 172.21.15.119:0/2870507599 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f33080a1c20 0x7f33080a40e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3310009c60 tx=0x7f3310009260 comp rx=0 tx=0).stop 2023-12-07T10:19:13.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:13.170+0000 7f3325581700 1 -- 172.21.15.119:0/2870507599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3320074160 msgr2=0x7f332019a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:13.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:13.170+0000 7f3325581700 1 --2- 172.21.15.119:0/2870507599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3320074160 0x7f332019a300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3314005950 tx=0x7f331400fa00 comp rx=0 tx=0).stop 2023-12-07T10:19:13.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:13.170+0000 7f3325581700 1 -- 172.21.15.119:0/2870507599 shutdown_connections 2023-12-07T10:19:13.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:13.170+0000 7f3325581700 1 --2- 172.21.15.119:0/2870507599 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f33080a1c20 0x7f33080a40e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:13.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:13.170+0000 7f3325581700 1 --2- 172.21.15.119:0/2870507599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f332010e070 0x7f332019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:13.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:13.170+0000 7f3325581700 1 --2- 172.21.15.119:0/2870507599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3320073840 0x7f3320199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:13.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:13.170+0000 7f3325581700 1 --2- 172.21.15.119:0/2870507599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3320074160 0x7f332019a300 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:13.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:13.170+0000 7f3325581700 1 -- 172.21.15.119:0/2870507599 >> 172.21.15.119:0/2870507599 conn(0x7f3320078800 msgr2=0x7f33200fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:13.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:13.170+0000 7f3325581700 1 -- 172.21.15.119:0/2870507599 shutdown_connections 2023-12-07T10:19:13.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:13.170+0000 7f3325581700 1 -- 172.21.15.119:0/2870507599 wait complete. 2023-12-07T10:19:13.178 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-12-07T10:19:13.735 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2023-12-07T10:19:12.692915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":672259,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":753,"num_read_kb":515,"num_write":426,"num_write_kb":3092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338804,"kb_used_data":14540,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603020,"statfs":{"total":1151940427776,"available":1151593492480,"internally_reserved":0,"allocated":14888960,"data_stored":8185886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":331855161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":100,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002912"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.899237+0000","last_change":"2023-12-07T10:17:41.614206+0000","last_active":"2023-12-07T10:19:01.899237+0000","last_peered":"2023-12-07T10:19:01.899237+0000","last_clean":"2023-12-07T10:19:01.899237+0000","last_became_active":"2023-12-07T10:17:41.613553+0000","last_became_peered":"2023-12-07T10:17:41.613553+0000","last_unstale":"2023-12-07T10:19:01.899237+0000","last_undegraded":"2023-12-07T10:19:01.899237+0000","last_fullsized":"2023-12-07T10:19:01.899237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:10:22.198414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803801+0000","last_change":"2023-12-07T10:17:41.616917+0000","last_active":"2023-12-07T10:19:01.803801+0000","last_peered":"2023-12-07T10:19:01.803801+0000","last_clean":"2023-12-07T10:19:01.803801+0000","last_became_active":"2023-12-07T10:17:41.615345+0000","last_became_peered":"2023-12-07T10:17:41.615345+0000","last_unstale":"2023-12-07T10:19:01.803801+0000","last_undegraded":"2023-12-07T10:19:01.803801+0000","last_fullsized":"2023-12-07T10:19:01.803801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:22:25.566738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803844+0000","last_change":"2023-12-07T10:17:41.619767+0000","last_active":"2023-12-07T10:19:01.803844+0000","last_peered":"2023-12-07T10:19:01.803844+0000","last_clean":"2023-12-07T10:19:01.803844+0000","last_became_active":"2023-12-07T10:17:41.619362+0000","last_became_peered":"2023-12-07T10:17:41.619362+0000","last_unstale":"2023-12-07T10:19:01.803844+0000","last_undegraded":"2023-12-07T10:19:01.803844+0000","last_fullsized":"2023-12-07T10:19:01.803844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:12:55.458520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807439+0000","last_change":"2023-12-07T10:18:01.861930+0000","last_active":"2023-12-07T10:19:01.807439+0000","last_peered":"2023-12-07T10:19:01.807439+0000","last_clean":"2023-12-07T10:19:01.807439+0000","last_became_active":"2023-12-07T10:18:01.861570+0000","last_became_peered":"2023-12-07T10:18:01.861570+0000","last_unstale":"2023-12-07T10:19:01.807439+0000","last_undegraded":"2023-12-07T10:19:01.807439+0000","last_fullsized":"2023-12-07T10:19:01.807439+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:29:45.234261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808566+0000","last_change":"2023-12-07T10:17:41.613890+0000","last_active":"2023-12-07T10:19:01.808566+0000","last_peered":"2023-12-07T10:19:01.808566+0000","last_clean":"2023-12-07T10:19:01.808566+0000","last_became_active":"2023-12-07T10:17:41.613073+0000","last_became_peered":"2023-12-07T10:17:41.613073+0000","last_unstale":"2023-12-07T10:19:01.808566+0000","last_undegraded":"2023-12-07T10:19:01.808566+0000","last_fullsized":"2023-12-07T10:19:01.808566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:17:14.339002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802869+0000","last_change":"2023-12-07T10:17:41.609585+0000","last_active":"2023-12-07T10:19:01.802869+0000","last_peered":"2023-12-07T10:19:01.802869+0000","last_clean":"2023-12-07T10:19:01.802869+0000","last_became_active":"2023-12-07T10:17:41.609256+0000","last_became_peered":"2023-12-07T10:17:41.609256+0000","last_unstale":"2023-12-07T10:19:01.802869+0000","last_undegraded":"2023-12-07T10:19:01.802869+0000","last_fullsized":"2023-12-07T10:19:01.802869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:20:15.103208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811012+0000","last_change":"2023-12-07T10:17:41.606138+0000","last_active":"2023-12-07T10:19:01.811012+0000","last_peered":"2023-12-07T10:19:01.811012+0000","last_clean":"2023-12-07T10:19:01.811012+0000","last_became_active":"2023-12-07T10:17:41.605818+0000","last_became_peered":"2023-12-07T10:17:41.605818+0000","last_unstale":"2023-12-07T10:19:01.811012+0000","last_undegraded":"2023-12-07T10:19:01.811012+0000","last_fullsized":"2023-12-07T10:19:01.811012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:38:39.244077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.806921+0000","last_change":"2023-12-07T10:18:02.255959+0000","last_active":"2023-12-07T10:19:01.806921+0000","last_peered":"2023-12-07T10:19:01.806921+0000","last_clean":"2023-12-07T10:19:01.806921+0000","last_became_active":"2023-12-07T10:18:02.255560+0000","last_became_peered":"2023-12-07T10:18:02.255560+0000","last_unstale":"2023-12-07T10:19:01.806921+0000","last_undegraded":"2023-12-07T10:19:01.806921+0000","last_fullsized":"2023-12-07T10:19:01.806921+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:05:49.065538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811362+0000","last_change":"2023-12-07T10:17:41.610225+0000","last_active":"2023-12-07T10:19:01.811362+0000","last_peered":"2023-12-07T10:19:01.811362+0000","last_clean":"2023-12-07T10:19:01.811362+0000","last_became_active":"2023-12-07T10:17:41.609875+0000","last_became_peered":"2023-12-07T10:17:41.609875+0000","last_unstale":"2023-12-07T10:19:01.811362+0000","last_undegraded":"2023-12-07T10:19:01.811362+0000","last_fullsized":"2023-12-07T10:19:01.811362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:00:43.136176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803348+0000","last_change":"2023-12-07T10:17:41.617335+0000","last_active":"2023-12-07T10:19:01.803348+0000","last_peered":"2023-12-07T10:19:01.803348+0000","last_clean":"2023-12-07T10:19:01.803348+0000","last_became_active":"2023-12-07T10:17:41.617005+0000","last_became_peered":"2023-12-07T10:17:41.617005+0000","last_unstale":"2023-12-07T10:19:01.803348+0000","last_undegraded":"2023-12-07T10:19:01.803348+0000","last_fullsized":"2023-12-07T10:19:01.803348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:43:36.855465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811670+0000","last_change":"2023-12-07T10:17:41.608104+0000","last_active":"2023-12-07T10:19:01.811670+0000","last_peered":"2023-12-07T10:19:01.811670+0000","last_clean":"2023-12-07T10:19:01.811670+0000","last_became_active":"2023-12-07T10:17:41.607769+0000","last_became_peered":"2023-12-07T10:17:41.607769+0000","last_unstale":"2023-12-07T10:19:01.811670+0000","last_undegraded":"2023-12-07T10:19:01.811670+0000","last_fullsized":"2023-12-07T10:19:01.811670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:47:04.853044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.805183+0000","last_change":"2023-12-07T10:17:41.611585+0000","last_active":"2023-12-07T10:19:01.805183+0000","last_peered":"2023-12-07T10:19:01.805183+0000","last_clean":"2023-12-07T10:19:01.805183+0000","last_became_active":"2023-12-07T10:17:41.611202+0000","last_became_peered":"2023-12-07T10:17:41.611202+0000","last_unstale":"2023-12-07T10:19:01.805183+0000","last_undegraded":"2023-12-07T10:19:01.805183+0000","last_fullsized":"2023-12-07T10:19:01.805183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:17:01.649424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813519+0000","last_change":"2023-12-07T10:17:41.611116+0000","last_active":"2023-12-07T10:19:01.813519+0000","last_peered":"2023-12-07T10:19:01.813519+0000","last_clean":"2023-12-07T10:19:01.813519+0000","last_became_active":"2023-12-07T10:17:41.610791+0000","last_became_peered":"2023-12-07T10:17:41.610791+0000","last_unstale":"2023-12-07T10:19:01.813519+0000","last_undegraded":"2023-12-07T10:19:01.813519+0000","last_fullsized":"2023-12-07T10:19:01.813519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:48:08.822235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.804022+0000","last_change":"2023-12-07T10:17:41.617447+0000","last_active":"2023-12-07T10:19:01.804022+0000","last_peered":"2023-12-07T10:19:01.804022+0000","last_clean":"2023-12-07T10:19:01.804022+0000","last_became_active":"2023-12-07T10:17:41.612638+0000","last_became_peered":"2023-12-07T10:17:41.612638+0000","last_unstale":"2023-12-07T10:19:01.804022+0000","last_undegraded":"2023-12-07T10:19:01.804022+0000","last_fullsized":"2023-12-07T10:19:01.804022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:11:30.384367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803409+0000","last_change":"2023-12-07T10:17:41.611607+0000","last_active":"2023-12-07T10:19:01.803409+0000","last_peered":"2023-12-07T10:19:01.803409+0000","last_clean":"2023-12-07T10:19:01.803409+0000","last_became_active":"2023-12-07T10:17:41.611331+0000","last_became_peered":"2023-12-07T10:17:41.611331+0000","last_unstale":"2023-12-07T10:19:01.803409+0000","last_undegraded":"2023-12-07T10:19:01.803409+0000","last_fullsized":"2023-12-07T10:19:01.803409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:17:53.049922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810651+0000","last_change":"2023-12-07T10:17:41.610528+0000","last_active":"2023-12-07T10:19:01.810651+0000","last_peered":"2023-12-07T10:19:01.810651+0000","last_clean":"2023-12-07T10:19:01.810651+0000","last_became_active":"2023-12-07T10:17:41.610216+0000","last_became_peered":"2023-12-07T10:17:41.610216+0000","last_unstale":"2023-12-07T10:19:01.810651+0000","last_undegraded":"2023-12-07T10:19:01.810651+0000","last_fullsized":"2023-12-07T10:19:01.810651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:11:36.749537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.899192+0000","last_change":"2023-12-07T10:17:41.613967+0000","last_active":"2023-12-07T10:19:01.899192+0000","last_peered":"2023-12-07T10:19:01.899192+0000","last_clean":"2023-12-07T10:19:01.899192+0000","last_became_active":"2023-12-07T10:17:41.613116+0000","last_became_peered":"2023-12-07T10:17:41.613116+0000","last_unstale":"2023-12-07T10:19:01.899192+0000","last_undegraded":"2023-12-07T10:19:01.899192+0000","last_fullsized":"2023-12-07T10:19:01.899192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:46:32.444025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807559+0000","last_change":"2023-12-07T10:17:41.605478+0000","last_active":"2023-12-07T10:19:01.807559+0000","last_peered":"2023-12-07T10:19:01.807559+0000","last_clean":"2023-12-07T10:19:01.807559+0000","last_became_active":"2023-12-07T10:17:41.605110+0000","last_became_peered":"2023-12-07T10:17:41.605110+0000","last_unstale":"2023-12-07T10:19:01.807559+0000","last_undegraded":"2023-12-07T10:19:01.807559+0000","last_fullsized":"2023-12-07T10:19:01.807559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:53:56.410196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810309+0000","last_change":"2023-12-07T10:18:02.256377+0000","last_active":"2023-12-07T10:19:01.810309+0000","last_peered":"2023-12-07T10:19:01.810309+0000","last_clean":"2023-12-07T10:19:01.810309+0000","last_became_active":"2023-12-07T10:18:02.255987+0000","last_became_peered":"2023-12-07T10:18:02.255987+0000","last_unstale":"2023-12-07T10:19:01.810309+0000","last_undegraded":"2023-12-07T10:19:01.810309+0000","last_fullsized":"2023-12-07T10:19:01.810309+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:23:34.909187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.809857+0000","last_change":"2023-12-07T10:17:45.799925+0000","last_active":"2023-12-07T10:19:01.809857+0000","last_peered":"2023-12-07T10:19:01.809857+0000","last_clean":"2023-12-07T10:19:01.809857+0000","last_became_active":"2023-12-07T10:17:45.799505+0000","last_became_peered":"2023-12-07T10:17:45.799505+0000","last_unstale":"2023-12-07T10:19:01.809857+0000","last_undegraded":"2023-12-07T10:19:01.809857+0000","last_fullsized":"2023-12-07T10:19:01.809857+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:52:51.470333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'13","reported_seq":91,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:07.845146+0000","last_change":"2023-12-07T10:17:40.755066+0000","last_active":"2023-12-07T10:19:07.845146+0000","last_peered":"2023-12-07T10:19:07.845146+0000","last_clean":"2023-12-07T10:19:07.845146+0000","last_became_active":"2023-12-07T10:17:40.754709+0000","last_became_peered":"2023-12-07T10:17:40.754709+0000","last_unstale":"2023-12-07T10:19:07.845146+0000","last_undegraded":"2023-12-07T10:19:07.845146+0000","last_fullsized":"2023-12-07T10:19:07.845146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:18:31.148592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":57,"num_read_kb":57,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803641+0000","last_change":"2023-12-07T10:17:41.619107+0000","last_active":"2023-12-07T10:19:01.803641+0000","last_peered":"2023-12-07T10:19:01.803641+0000","last_clean":"2023-12-07T10:19:01.803641+0000","last_became_active":"2023-12-07T10:17:41.617057+0000","last_became_peered":"2023-12-07T10:17:41.617057+0000","last_unstale":"2023-12-07T10:19:01.803641+0000","last_undegraded":"2023-12-07T10:19:01.803641+0000","last_fullsized":"2023-12-07T10:19:01.803641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:53:16.402933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.804762+0000","last_change":"2023-12-07T10:17:45.797582+0000","last_active":"2023-12-07T10:19:01.804762+0000","last_peered":"2023-12-07T10:19:01.804762+0000","last_clean":"2023-12-07T10:19:01.804762+0000","last_became_active":"2023-12-07T10:17:45.796358+0000","last_became_peered":"2023-12-07T10:17:45.796358+0000","last_unstale":"2023-12-07T10:19:01.804762+0000","last_undegraded":"2023-12-07T10:19:01.804762+0000","last_fullsized":"2023-12-07T10:19:01.804762+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:53:19.551459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803712+0000","last_change":"2023-12-07T10:17:40.762155+0000","last_active":"2023-12-07T10:19:01.803712+0000","last_peered":"2023-12-07T10:19:01.803712+0000","last_clean":"2023-12-07T10:19:01.803712+0000","last_became_active":"2023-12-07T10:17:40.761776+0000","last_became_peered":"2023-12-07T10:17:40.761776+0000","last_unstale":"2023-12-07T10:19:01.803712+0000","last_undegraded":"2023-12-07T10:19:01.803712+0000","last_fullsized":"2023-12-07T10:19:01.803712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:15:06.096331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.805463+0000","last_change":"2023-12-07T10:17:41.614999+0000","last_active":"2023-12-07T10:19:01.805463+0000","last_peered":"2023-12-07T10:19:01.805463+0000","last_clean":"2023-12-07T10:19:01.805463+0000","last_became_active":"2023-12-07T10:17:41.614673+0000","last_became_peered":"2023-12-07T10:17:41.614673+0000","last_unstale":"2023-12-07T10:19:01.805463+0000","last_undegraded":"2023-12-07T10:19:01.805463+0000","last_fullsized":"2023-12-07T10:19:01.805463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:34:45.378843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.215249+0000","last_change":"2023-12-07T10:19:03.215249+0000","last_active":"2023-12-07T10:19:03.215249+0000","last_peered":"2023-12-07T10:19:03.215249+0000","last_clean":"2023-12-07T10:19:03.215249+0000","last_became_active":"2023-12-07T10:19:03.214647+0000","last_became_peered":"2023-12-07T10:19:03.214647+0000","last_unstale":"2023-12-07T10:19:03.215249+0000","last_undegraded":"2023-12-07T10:19:03.215249+0000","last_fullsized":"2023-12-07T10:19:03.215249+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:41:01.893661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808809+0000","last_change":"2023-12-07T10:18:01.993128+0000","last_active":"2023-12-07T10:19:01.808809+0000","last_peered":"2023-12-07T10:19:01.808809+0000","last_clean":"2023-12-07T10:19:01.808809+0000","last_became_active":"2023-12-07T10:18:01.990244+0000","last_became_peered":"2023-12-07T10:18:01.990244+0000","last_unstale":"2023-12-07T10:19:01.808809+0000","last_undegraded":"2023-12-07T10:19:01.808809+0000","last_fullsized":"2023-12-07T10:19:01.808809+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:42:53.778363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810926+0000","last_change":"2023-12-07T10:17:41.612903+0000","last_active":"2023-12-07T10:19:01.810926+0000","last_peered":"2023-12-07T10:19:01.810926+0000","last_clean":"2023-12-07T10:19:01.810926+0000","last_became_active":"2023-12-07T10:17:41.611968+0000","last_became_peered":"2023-12-07T10:17:41.611968+0000","last_unstale":"2023-12-07T10:19:01.810926+0000","last_undegraded":"2023-12-07T10:19:01.810926+0000","last_fullsized":"2023-12-07T10:19:01.810926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:34:08.442828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.804826+0000","last_change":"2023-12-07T10:17:45.803858+0000","last_active":"2023-12-07T10:19:01.804826+0000","last_peered":"2023-12-07T10:19:01.804826+0000","last_clean":"2023-12-07T10:19:01.804826+0000","last_became_active":"2023-12-07T10:17:45.803481+0000","last_became_peered":"2023-12-07T10:17:45.803481+0000","last_unstale":"2023-12-07T10:19:01.804826+0000","last_undegraded":"2023-12-07T10:19:01.804826+0000","last_fullsized":"2023-12-07T10:19:01.804826+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:08:03.728554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808719+0000","last_change":"2023-12-07T10:17:40.751866+0000","last_active":"2023-12-07T10:19:01.808719+0000","last_peered":"2023-12-07T10:19:01.808719+0000","last_clean":"2023-12-07T10:19:01.808719+0000","last_became_active":"2023-12-07T10:17:40.751492+0000","last_became_peered":"2023-12-07T10:17:40.751492+0000","last_unstale":"2023-12-07T10:19:01.808719+0000","last_undegraded":"2023-12-07T10:19:01.808719+0000","last_fullsized":"2023-12-07T10:19:01.808719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:42:53.839651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808255+0000","last_change":"2023-12-07T10:17:41.611326+0000","last_active":"2023-12-07T10:19:01.808255+0000","last_peered":"2023-12-07T10:19:01.808255+0000","last_clean":"2023-12-07T10:19:01.808255+0000","last_became_active":"2023-12-07T10:17:41.610863+0000","last_became_peered":"2023-12-07T10:17:41.610863+0000","last_unstale":"2023-12-07T10:19:01.808255+0000","last_undegraded":"2023-12-07T10:19:01.808255+0000","last_fullsized":"2023-12-07T10:19:01.808255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:07:55.801356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810238+0000","last_change":"2023-12-07T10:17:45.791409+0000","last_active":"2023-12-07T10:19:01.810238+0000","last_peered":"2023-12-07T10:19:01.810238+0000","last_clean":"2023-12-07T10:19:01.810238+0000","last_became_active":"2023-12-07T10:17:45.791022+0000","last_became_peered":"2023-12-07T10:17:45.791022+0000","last_unstale":"2023-12-07T10:19:01.810238+0000","last_undegraded":"2023-12-07T10:19:01.810238+0000","last_fullsized":"2023-12-07T10:19:01.810238+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:57:14.617409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'9","reported_seq":93,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:06.824597+0000","last_change":"2023-12-07T10:17:40.745318+0000","last_active":"2023-12-07T10:19:06.824597+0000","last_peered":"2023-12-07T10:19:06.824597+0000","last_clean":"2023-12-07T10:19:06.824597+0000","last_became_active":"2023-12-07T10:17:40.745008+0000","last_became_peered":"2023-12-07T10:17:40.745008+0000","last_unstale":"2023-12-07T10:19:06.824597+0000","last_undegraded":"2023-12-07T10:19:06.824597+0000","last_fullsized":"2023-12-07T10:19:06.824597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:42:28.503421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11004,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":14,"num_write":6,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808088+0000","last_change":"2023-12-07T10:17:41.605121+0000","last_active":"2023-12-07T10:19:01.808088+0000","last_peered":"2023-12-07T10:19:01.808088+0000","last_clean":"2023-12-07T10:19:01.808088+0000","last_became_active":"2023-12-07T10:17:41.604770+0000","last_became_peered":"2023-12-07T10:17:41.604770+0000","last_unstale":"2023-12-07T10:19:01.808088+0000","last_undegraded":"2023-12-07T10:19:01.808088+0000","last_fullsized":"2023-12-07T10:19:01.808088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:51:02.151155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810286+0000","last_change":"2023-12-07T10:17:45.809039+0000","last_active":"2023-12-07T10:19:01.810286+0000","last_peered":"2023-12-07T10:19:01.810286+0000","last_clean":"2023-12-07T10:19:01.810286+0000","last_became_active":"2023-12-07T10:17:45.808622+0000","last_became_peered":"2023-12-07T10:17:45.808622+0000","last_unstale":"2023-12-07T10:19:01.810286+0000","last_undegraded":"2023-12-07T10:19:01.810286+0000","last_fullsized":"2023-12-07T10:19:01.810286+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:16:02.835931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802888+0000","last_change":"2023-12-07T10:17:40.757876+0000","last_active":"2023-12-07T10:19:01.802888+0000","last_peered":"2023-12-07T10:19:01.802888+0000","last_clean":"2023-12-07T10:19:01.802888+0000","last_became_active":"2023-12-07T10:17:40.757613+0000","last_became_peered":"2023-12-07T10:17:40.757613+0000","last_unstale":"2023-12-07T10:19:01.802888+0000","last_undegraded":"2023-12-07T10:19:01.802888+0000","last_fullsized":"2023-12-07T10:19:01.802888+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:30:40.423092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808115+0000","last_change":"2023-12-07T10:17:41.611575+0000","last_active":"2023-12-07T10:19:01.808115+0000","last_peered":"2023-12-07T10:19:01.808115+0000","last_clean":"2023-12-07T10:19:01.808115+0000","last_became_active":"2023-12-07T10:17:41.611042+0000","last_became_peered":"2023-12-07T10:17:41.611042+0000","last_unstale":"2023-12-07T10:19:01.808115+0000","last_undegraded":"2023-12-07T10:19:01.808115+0000","last_fullsized":"2023-12-07T10:19:01.808115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:54:05.589792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.899155+0000","last_change":"2023-12-07T10:17:45.796640+0000","last_active":"2023-12-07T10:19:01.899155+0000","last_peered":"2023-12-07T10:19:01.899155+0000","last_clean":"2023-12-07T10:19:01.899155+0000","last_became_active":"2023-12-07T10:17:45.796183+0000","last_became_peered":"2023-12-07T10:17:45.796183+0000","last_unstale":"2023-12-07T10:19:01.899155+0000","last_undegraded":"2023-12-07T10:19:01.899155+0000","last_fullsized":"2023-12-07T10:19:01.899155+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:41:56.890662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813362+0000","last_change":"2023-12-07T10:17:40.760283+0000","last_active":"2023-12-07T10:19:01.813362+0000","last_peered":"2023-12-07T10:19:01.813362+0000","last_clean":"2023-12-07T10:19:01.813362+0000","last_became_active":"2023-12-07T10:17:40.759398+0000","last_became_peered":"2023-12-07T10:17:40.759398+0000","last_unstale":"2023-12-07T10:19:01.813362+0000","last_undegraded":"2023-12-07T10:19:01.813362+0000","last_fullsized":"2023-12-07T10:19:01.813362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:34:37.365490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802564+0000","last_change":"2023-12-07T10:17:41.604150+0000","last_active":"2023-12-07T10:19:01.802564+0000","last_peered":"2023-12-07T10:19:01.802564+0000","last_clean":"2023-12-07T10:19:01.802564+0000","last_became_active":"2023-12-07T10:17:41.603688+0000","last_became_peered":"2023-12-07T10:17:41.603688+0000","last_unstale":"2023-12-07T10:19:01.802564+0000","last_undegraded":"2023-12-07T10:19:01.802564+0000","last_fullsized":"2023-12-07T10:19:01.802564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:52:43.735609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803710+0000","last_change":"2023-12-07T10:17:45.806569+0000","last_active":"2023-12-07T10:19:01.803710+0000","last_peered":"2023-12-07T10:19:01.803710+0000","last_clean":"2023-12-07T10:19:01.803710+0000","last_became_active":"2023-12-07T10:17:45.806099+0000","last_became_peered":"2023-12-07T10:17:45.806099+0000","last_unstale":"2023-12-07T10:19:01.803710+0000","last_undegraded":"2023-12-07T10:19:01.803710+0000","last_fullsized":"2023-12-07T10:19:01.803710+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:02:19.977670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807289+0000","last_change":"2023-12-07T10:17:40.744388+0000","last_active":"2023-12-07T10:19:01.807289+0000","last_peered":"2023-12-07T10:19:01.807289+0000","last_clean":"2023-12-07T10:19:01.807289+0000","last_became_active":"2023-12-07T10:17:40.743887+0000","last_became_peered":"2023-12-07T10:17:40.743887+0000","last_unstale":"2023-12-07T10:19:01.807289+0000","last_undegraded":"2023-12-07T10:19:01.807289+0000","last_fullsized":"2023-12-07T10:19:01.807289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:18:39.102099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807114+0000","last_change":"2023-12-07T10:17:41.595616+0000","last_active":"2023-12-07T10:19:01.807114+0000","last_peered":"2023-12-07T10:19:01.807114+0000","last_clean":"2023-12-07T10:19:01.807114+0000","last_became_active":"2023-12-07T10:17:41.595256+0000","last_became_peered":"2023-12-07T10:17:41.595256+0000","last_unstale":"2023-12-07T10:19:01.807114+0000","last_undegraded":"2023-12-07T10:19:01.807114+0000","last_fullsized":"2023-12-07T10:19:01.807114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:08:00.156043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802753+0000","last_change":"2023-12-07T10:17:45.794523+0000","last_active":"2023-12-07T10:19:01.802753+0000","last_peered":"2023-12-07T10:19:01.802753+0000","last_clean":"2023-12-07T10:19:01.802753+0000","last_became_active":"2023-12-07T10:17:45.794139+0000","last_became_peered":"2023-12-07T10:17:45.794139+0000","last_unstale":"2023-12-07T10:19:01.802753+0000","last_undegraded":"2023-12-07T10:19:01.802753+0000","last_fullsized":"2023-12-07T10:19:01.802753+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:43:24.767627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":142,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:09.828395+0000","last_change":"2023-12-07T10:17:40.759620+0000","last_active":"2023-12-07T10:19:09.828395+0000","last_peered":"2023-12-07T10:19:09.828395+0000","last_clean":"2023-12-07T10:19:09.828395+0000","last_became_active":"2023-12-07T10:17:40.759372+0000","last_became_peered":"2023-12-07T10:17:40.759372+0000","last_unstale":"2023-12-07T10:19:09.828395+0000","last_undegraded":"2023-12-07T10:19:09.828395+0000","last_fullsized":"2023-12-07T10:19:09.828395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:46:14.459409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811706+0000","last_change":"2023-12-07T10:17:41.601152+0000","last_active":"2023-12-07T10:19:01.811706+0000","last_peered":"2023-12-07T10:19:01.811706+0000","last_clean":"2023-12-07T10:19:01.811706+0000","last_became_active":"2023-12-07T10:17:41.600147+0000","last_became_peered":"2023-12-07T10:17:41.600147+0000","last_unstale":"2023-12-07T10:19:01.811706+0000","last_undegraded":"2023-12-07T10:19:01.811706+0000","last_fullsized":"2023-12-07T10:19:01.811706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:40:46.671489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807019+0000","last_change":"2023-12-07T10:17:45.796229+0000","last_active":"2023-12-07T10:19:01.807019+0000","last_peered":"2023-12-07T10:19:01.807019+0000","last_clean":"2023-12-07T10:19:01.807019+0000","last_became_active":"2023-12-07T10:17:45.795582+0000","last_became_peered":"2023-12-07T10:17:45.795582+0000","last_unstale":"2023-12-07T10:19:01.807019+0000","last_undegraded":"2023-12-07T10:19:01.807019+0000","last_fullsized":"2023-12-07T10:19:01.807019+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:29:32.853975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811078+0000","last_change":"2023-12-07T10:17:40.748353+0000","last_active":"2023-12-07T10:19:01.811078+0000","last_peered":"2023-12-07T10:19:01.811078+0000","last_clean":"2023-12-07T10:19:01.811078+0000","last_became_active":"2023-12-07T10:17:40.748050+0000","last_became_peered":"2023-12-07T10:17:40.748050+0000","last_unstale":"2023-12-07T10:19:01.811078+0000","last_undegraded":"2023-12-07T10:19:01.811078+0000","last_fullsized":"2023-12-07T10:19:01.811078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:18:15.298534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.805543+0000","last_change":"2023-12-07T10:18:02.256728+0000","last_active":"2023-12-07T10:19:01.805543+0000","last_peered":"2023-12-07T10:19:01.805543+0000","last_clean":"2023-12-07T10:19:01.805543+0000","last_became_active":"2023-12-07T10:18:02.256271+0000","last_became_peered":"2023-12-07T10:18:02.256271+0000","last_unstale":"2023-12-07T10:19:01.805543+0000","last_undegraded":"2023-12-07T10:19:01.805543+0000","last_fullsized":"2023-12-07T10:19:01.805543+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:27:00.198934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.805652+0000","last_change":"2023-12-07T10:17:45.796903+0000","last_active":"2023-12-07T10:19:01.805652+0000","last_peered":"2023-12-07T10:19:01.805652+0000","last_clean":"2023-12-07T10:19:01.805652+0000","last_became_active":"2023-12-07T10:17:45.796454+0000","last_became_peered":"2023-12-07T10:17:45.796454+0000","last_unstale":"2023-12-07T10:19:01.805652+0000","last_undegraded":"2023-12-07T10:19:01.805652+0000","last_fullsized":"2023-12-07T10:19:01.805652+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:46:36.955396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808245+0000","last_change":"2023-12-07T10:17:40.756045+0000","last_active":"2023-12-07T10:19:01.808245+0000","last_peered":"2023-12-07T10:19:01.808245+0000","last_clean":"2023-12-07T10:19:01.808245+0000","last_became_active":"2023-12-07T10:17:40.755763+0000","last_became_peered":"2023-12-07T10:17:40.755763+0000","last_unstale":"2023-12-07T10:19:01.808245+0000","last_undegraded":"2023-12-07T10:19:01.808245+0000","last_fullsized":"2023-12-07T10:19:01.808245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:01:08.009076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811097+0000","last_change":"2023-12-07T10:17:41.611178+0000","last_active":"2023-12-07T10:19:01.811097+0000","last_peered":"2023-12-07T10:19:01.811097+0000","last_clean":"2023-12-07T10:19:01.811097+0000","last_became_active":"2023-12-07T10:17:41.610750+0000","last_became_peered":"2023-12-07T10:17:41.610750+0000","last_unstale":"2023-12-07T10:19:01.811097+0000","last_undegraded":"2023-12-07T10:19:01.811097+0000","last_fullsized":"2023-12-07T10:19:01.811097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:56:30.763523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807197+0000","last_change":"2023-12-07T10:17:45.800183+0000","last_active":"2023-12-07T10:19:01.807197+0000","last_peered":"2023-12-07T10:19:01.807197+0000","last_clean":"2023-12-07T10:19:01.807197+0000","last_became_active":"2023-12-07T10:17:45.798598+0000","last_became_peered":"2023-12-07T10:17:45.798598+0000","last_unstale":"2023-12-07T10:19:01.807197+0000","last_undegraded":"2023-12-07T10:19:01.807197+0000","last_fullsized":"2023-12-07T10:19:01.807197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:06:22.162951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":182,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:09.828363+0000","last_change":"2023-12-07T10:17:40.752481+0000","last_active":"2023-12-07T10:19:09.828363+0000","last_peered":"2023-12-07T10:19:09.828363+0000","last_clean":"2023-12-07T10:19:09.828363+0000","last_became_active":"2023-12-07T10:17:40.752158+0000","last_became_peered":"2023-12-07T10:17:40.752158+0000","last_unstale":"2023-12-07T10:19:09.828363+0000","last_undegraded":"2023-12-07T10:19:09.828363+0000","last_fullsized":"2023-12-07T10:19:09.828363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:43:27.963014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64453,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":127,"num_read_kb":126,"num_write":36,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808644+0000","last_change":"2023-12-07T10:17:41.614107+0000","last_active":"2023-12-07T10:19:01.808644+0000","last_peered":"2023-12-07T10:19:01.808644+0000","last_clean":"2023-12-07T10:19:01.808644+0000","last_became_active":"2023-12-07T10:17:41.613743+0000","last_became_peered":"2023-12-07T10:17:41.613743+0000","last_unstale":"2023-12-07T10:19:01.808644+0000","last_undegraded":"2023-12-07T10:19:01.808644+0000","last_fullsized":"2023-12-07T10:19:01.808644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:49:03.200387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808863+0000","last_change":"2023-12-07T10:17:45.804282+0000","last_active":"2023-12-07T10:19:01.808863+0000","last_peered":"2023-12-07T10:19:01.808863+0000","last_clean":"2023-12-07T10:19:01.808863+0000","last_became_active":"2023-12-07T10:17:45.803794+0000","last_became_peered":"2023-12-07T10:17:45.803794+0000","last_unstale":"2023-12-07T10:19:01.808863+0000","last_undegraded":"2023-12-07T10:19:01.808863+0000","last_fullsized":"2023-12-07T10:19:01.808863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:31:26.887565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810171+0000","last_change":"2023-12-07T10:17:40.747204+0000","last_active":"2023-12-07T10:19:01.810171+0000","last_peered":"2023-12-07T10:19:01.810171+0000","last_clean":"2023-12-07T10:19:01.810171+0000","last_became_active":"2023-12-07T10:17:40.746916+0000","last_became_peered":"2023-12-07T10:17:40.746916+0000","last_unstale":"2023-12-07T10:19:01.810171+0000","last_undegraded":"2023-12-07T10:19:01.810171+0000","last_fullsized":"2023-12-07T10:19:01.810171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:53:49.228320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.814021+0000","last_change":"2023-12-07T10:17:41.606895+0000","last_active":"2023-12-07T10:19:01.814021+0000","last_peered":"2023-12-07T10:19:01.814021+0000","last_clean":"2023-12-07T10:19:01.814021+0000","last_became_active":"2023-12-07T10:17:41.606596+0000","last_became_peered":"2023-12-07T10:17:41.606596+0000","last_unstale":"2023-12-07T10:19:01.814021+0000","last_undegraded":"2023-12-07T10:19:01.814021+0000","last_fullsized":"2023-12-07T10:19:01.814021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:34:25.166906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.898878+0000","last_change":"2023-12-07T10:17:45.807101+0000","last_active":"2023-12-07T10:19:01.898878+0000","last_peered":"2023-12-07T10:19:01.898878+0000","last_clean":"2023-12-07T10:19:01.898878+0000","last_became_active":"2023-12-07T10:17:45.806770+0000","last_became_peered":"2023-12-07T10:17:45.806770+0000","last_unstale":"2023-12-07T10:19:01.898878+0000","last_undegraded":"2023-12-07T10:19:01.898878+0000","last_fullsized":"2023-12-07T10:19:01.898878+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:39:32.522968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808371+0000","last_change":"2023-12-07T10:17:40.754856+0000","last_active":"2023-12-07T10:19:01.808371+0000","last_peered":"2023-12-07T10:19:01.808371+0000","last_clean":"2023-12-07T10:19:01.808371+0000","last_became_active":"2023-12-07T10:17:40.754488+0000","last_became_peered":"2023-12-07T10:17:40.754488+0000","last_unstale":"2023-12-07T10:19:01.808371+0000","last_undegraded":"2023-12-07T10:19:01.808371+0000","last_fullsized":"2023-12-07T10:19:01.808371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:44:50.960216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811734+0000","last_change":"2023-12-07T10:17:41.606963+0000","last_active":"2023-12-07T10:19:01.811734+0000","last_peered":"2023-12-07T10:19:01.811734+0000","last_clean":"2023-12-07T10:19:01.811734+0000","last_became_active":"2023-12-07T10:17:41.606408+0000","last_became_peered":"2023-12-07T10:17:41.606408+0000","last_unstale":"2023-12-07T10:19:01.811734+0000","last_undegraded":"2023-12-07T10:19:01.811734+0000","last_fullsized":"2023-12-07T10:19:01.811734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:25:47.906649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.384918+0000","last_change":"2023-12-07T10:19:03.384918+0000","last_active":"2023-12-07T10:19:03.384918+0000","last_peered":"2023-12-07T10:19:03.384918+0000","last_clean":"2023-12-07T10:19:03.384918+0000","last_became_active":"2023-12-07T10:19:03.384613+0000","last_became_peered":"2023-12-07T10:19:03.384613+0000","last_unstale":"2023-12-07T10:19:03.384918+0000","last_undegraded":"2023-12-07T10:19:03.384918+0000","last_fullsized":"2023-12-07T10:19:03.384918+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:07:29.385953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":87,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:09.848760+0000","last_change":"2023-12-07T10:17:40.747494+0000","last_active":"2023-12-07T10:19:09.848760+0000","last_peered":"2023-12-07T10:19:09.848760+0000","last_clean":"2023-12-07T10:19:09.848760+0000","last_became_active":"2023-12-07T10:17:40.747157+0000","last_became_peered":"2023-12-07T10:17:40.747157+0000","last_unstale":"2023-12-07T10:19:09.848760+0000","last_undegraded":"2023-12-07T10:19:09.848760+0000","last_fullsized":"2023-12-07T10:19:09.848760+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:45:24.395418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":976,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":61,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813024+0000","last_change":"2023-12-07T10:17:41.609495+0000","last_active":"2023-12-07T10:19:01.813024+0000","last_peered":"2023-12-07T10:19:01.813024+0000","last_clean":"2023-12-07T10:19:01.813024+0000","last_became_active":"2023-12-07T10:17:41.609181+0000","last_became_peered":"2023-12-07T10:17:41.609181+0000","last_unstale":"2023-12-07T10:19:01.813024+0000","last_undegraded":"2023-12-07T10:19:01.813024+0000","last_fullsized":"2023-12-07T10:19:01.813024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:20:33.473824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813855+0000","last_change":"2023-12-07T10:17:45.808862+0000","last_active":"2023-12-07T10:19:01.813855+0000","last_peered":"2023-12-07T10:19:01.813855+0000","last_clean":"2023-12-07T10:19:01.813855+0000","last_became_active":"2023-12-07T10:17:45.808461+0000","last_became_peered":"2023-12-07T10:17:45.808461+0000","last_unstale":"2023-12-07T10:19:01.813855+0000","last_undegraded":"2023-12-07T10:19:01.813855+0000","last_fullsized":"2023-12-07T10:19:01.813855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:09:12.598126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807550+0000","last_change":"2023-12-07T10:18:02.263709+0000","last_active":"2023-12-07T10:19:01.807550+0000","last_peered":"2023-12-07T10:19:01.807550+0000","last_clean":"2023-12-07T10:19:01.807550+0000","last_became_active":"2023-12-07T10:18:02.255788+0000","last_became_peered":"2023-12-07T10:18:02.255788+0000","last_unstale":"2023-12-07T10:19:01.807550+0000","last_undegraded":"2023-12-07T10:19:01.807550+0000","last_fullsized":"2023-12-07T10:19:01.807550+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:12:04.976226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808356+0000","last_change":"2023-12-07T10:17:41.610509+0000","last_active":"2023-12-07T10:19:01.808356+0000","last_peered":"2023-12-07T10:19:01.808356+0000","last_clean":"2023-12-07T10:19:01.808356+0000","last_became_active":"2023-12-07T10:17:41.610121+0000","last_became_peered":"2023-12-07T10:17:41.610121+0000","last_unstale":"2023-12-07T10:19:01.808356+0000","last_undegraded":"2023-12-07T10:19:01.808356+0000","last_fullsized":"2023-12-07T10:19:01.808356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:53:17.365880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808256+0000","last_change":"2023-12-07T10:17:45.795818+0000","last_active":"2023-12-07T10:19:01.808256+0000","last_peered":"2023-12-07T10:19:01.808256+0000","last_clean":"2023-12-07T10:19:01.808256+0000","last_became_active":"2023-12-07T10:17:45.795477+0000","last_became_peered":"2023-12-07T10:17:45.795477+0000","last_unstale":"2023-12-07T10:19:01.808256+0000","last_undegraded":"2023-12-07T10:19:01.808256+0000","last_fullsized":"2023-12-07T10:19:01.808256+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:17:21.494654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.809420+0000","last_change":"2023-12-07T10:17:40.751791+0000","last_active":"2023-12-07T10:19:01.809420+0000","last_peered":"2023-12-07T10:19:01.809420+0000","last_clean":"2023-12-07T10:19:01.809420+0000","last_became_active":"2023-12-07T10:17:40.751502+0000","last_became_peered":"2023-12-07T10:17:40.751502+0000","last_unstale":"2023-12-07T10:19:01.809420+0000","last_undegraded":"2023-12-07T10:19:01.809420+0000","last_fullsized":"2023-12-07T10:19:01.809420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:15:52.319265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.899134+0000","last_change":"2023-12-07T10:18:02.256247+0000","last_active":"2023-12-07T10:19:01.899134+0000","last_peered":"2023-12-07T10:19:01.899134+0000","last_clean":"2023-12-07T10:19:01.899134+0000","last_became_active":"2023-12-07T10:18:02.255732+0000","last_became_peered":"2023-12-07T10:18:02.255732+0000","last_unstale":"2023-12-07T10:19:01.899134+0000","last_undegraded":"2023-12-07T10:19:01.899134+0000","last_fullsized":"2023-12-07T10:19:01.899134+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:00:06.516804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808004+0000","last_change":"2023-12-07T10:17:45.791966+0000","last_active":"2023-12-07T10:19:01.808004+0000","last_peered":"2023-12-07T10:19:01.808004+0000","last_clean":"2023-12-07T10:19:01.808004+0000","last_became_active":"2023-12-07T10:17:45.791613+0000","last_became_peered":"2023-12-07T10:17:45.791613+0000","last_unstale":"2023-12-07T10:19:01.808004+0000","last_undegraded":"2023-12-07T10:19:01.808004+0000","last_fullsized":"2023-12-07T10:19:01.808004+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:13:02.291289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807344+0000","last_change":"2023-12-07T10:17:40.762896+0000","last_active":"2023-12-07T10:19:01.807344+0000","last_peered":"2023-12-07T10:19:01.807344+0000","last_clean":"2023-12-07T10:19:01.807344+0000","last_became_active":"2023-12-07T10:17:40.762427+0000","last_became_peered":"2023-12-07T10:17:40.762427+0000","last_unstale":"2023-12-07T10:19:01.807344+0000","last_undegraded":"2023-12-07T10:19:01.807344+0000","last_fullsized":"2023-12-07T10:19:01.807344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:56:35.589548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803085+0000","last_change":"2023-12-07T10:17:41.610354+0000","last_active":"2023-12-07T10:19:01.803085+0000","last_peered":"2023-12-07T10:19:01.803085+0000","last_clean":"2023-12-07T10:19:01.803085+0000","last_became_active":"2023-12-07T10:17:41.609888+0000","last_became_peered":"2023-12-07T10:17:41.609888+0000","last_unstale":"2023-12-07T10:19:01.803085+0000","last_undegraded":"2023-12-07T10:19:01.803085+0000","last_fullsized":"2023-12-07T10:19:01.803085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:48:19.976949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813526+0000","last_change":"2023-12-07T10:17:45.807324+0000","last_active":"2023-12-07T10:19:01.813526+0000","last_peered":"2023-12-07T10:19:01.813526+0000","last_clean":"2023-12-07T10:19:01.813526+0000","last_became_active":"2023-12-07T10:17:45.806036+0000","last_became_peered":"2023-12-07T10:17:45.806036+0000","last_unstale":"2023-12-07T10:19:01.813526+0000","last_undegraded":"2023-12-07T10:19:01.813526+0000","last_fullsized":"2023-12-07T10:19:01.813526+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:54:36.847992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'5","reported_seq":84,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:08.847177+0000","last_change":"2023-12-07T10:17:40.754247+0000","last_active":"2023-12-07T10:19:08.847177+0000","last_peered":"2023-12-07T10:19:08.847177+0000","last_clean":"2023-12-07T10:19:08.847177+0000","last_became_active":"2023-12-07T10:17:40.753964+0000","last_became_peered":"2023-12-07T10:17:40.753964+0000","last_unstale":"2023-12-07T10:19:08.847177+0000","last_undegraded":"2023-12-07T10:19:08.847177+0000","last_fullsized":"2023-12-07T10:19:08.847177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:22:04.832399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802944+0000","last_change":"2023-12-07T10:17:41.610516+0000","last_active":"2023-12-07T10:19:01.802944+0000","last_peered":"2023-12-07T10:19:01.802944+0000","last_clean":"2023-12-07T10:19:01.802944+0000","last_became_active":"2023-12-07T10:17:41.610184+0000","last_became_peered":"2023-12-07T10:17:41.610184+0000","last_unstale":"2023-12-07T10:19:01.802944+0000","last_undegraded":"2023-12-07T10:19:01.802944+0000","last_fullsized":"2023-12-07T10:19:01.802944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:53:43.100723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813880+0000","last_change":"2023-12-07T10:17:45.804624+0000","last_active":"2023-12-07T10:19:01.813880+0000","last_peered":"2023-12-07T10:19:01.813880+0000","last_clean":"2023-12-07T10:19:01.813880+0000","last_became_active":"2023-12-07T10:17:45.803667+0000","last_became_peered":"2023-12-07T10:17:45.803667+0000","last_unstale":"2023-12-07T10:19:01.813880+0000","last_undegraded":"2023-12-07T10:19:01.813880+0000","last_fullsized":"2023-12-07T10:19:01.813880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:55:09.642795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807386+0000","last_change":"2023-12-07T10:17:41.618433+0000","last_active":"2023-12-07T10:19:01.807386+0000","last_peered":"2023-12-07T10:19:01.807386+0000","last_clean":"2023-12-07T10:19:01.807386+0000","last_became_active":"2023-12-07T10:17:41.617942+0000","last_became_peered":"2023-12-07T10:17:41.617942+0000","last_unstale":"2023-12-07T10:19:01.807386+0000","last_undegraded":"2023-12-07T10:19:01.807386+0000","last_fullsized":"2023-12-07T10:19:01.807386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:26:48.166157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":82,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:06.844338+0000","last_change":"2023-12-07T10:17:40.761907+0000","last_active":"2023-12-07T10:19:06.844338+0000","last_peered":"2023-12-07T10:19:06.844338+0000","last_clean":"2023-12-07T10:19:06.844338+0000","last_became_active":"2023-12-07T10:17:40.761564+0000","last_became_peered":"2023-12-07T10:17:40.761564+0000","last_unstale":"2023-12-07T10:19:06.844338+0000","last_undegraded":"2023-12-07T10:19:06.844338+0000","last_fullsized":"2023-12-07T10:19:06.844338+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:57:22.304352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811777+0000","last_change":"2023-12-07T10:17:45.792684+0000","last_active":"2023-12-07T10:19:01.811777+0000","last_peered":"2023-12-07T10:19:01.811777+0000","last_clean":"2023-12-07T10:19:01.811777+0000","last_became_active":"2023-12-07T10:17:45.792172+0000","last_became_peered":"2023-12-07T10:17:45.792172+0000","last_unstale":"2023-12-07T10:19:01.811777+0000","last_undegraded":"2023-12-07T10:19:01.811777+0000","last_fullsized":"2023-12-07T10:19:01.811777+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:24:31.084727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802651+0000","last_change":"2023-12-07T10:17:41.612582+0000","last_active":"2023-12-07T10:19:01.802651+0000","last_peered":"2023-12-07T10:19:01.802651+0000","last_clean":"2023-12-07T10:19:01.802651+0000","last_became_active":"2023-12-07T10:17:41.612244+0000","last_became_peered":"2023-12-07T10:17:41.612244+0000","last_unstale":"2023-12-07T10:19:01.802651+0000","last_undegraded":"2023-12-07T10:19:01.802651+0000","last_fullsized":"2023-12-07T10:19:01.802651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:41:53.506037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.805642+0000","last_change":"2023-12-07T10:17:40.748565+0000","last_active":"2023-12-07T10:19:01.805642+0000","last_peered":"2023-12-07T10:19:01.805642+0000","last_clean":"2023-12-07T10:19:01.805642+0000","last_became_active":"2023-12-07T10:17:40.748250+0000","last_became_peered":"2023-12-07T10:17:40.748250+0000","last_unstale":"2023-12-07T10:19:01.805642+0000","last_undegraded":"2023-12-07T10:19:01.805642+0000","last_fullsized":"2023-12-07T10:19:01.805642+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:26:28.202440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":328,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.812930+0000","last_change":"2023-12-07T10:16:33.482173+0000","last_active":"2023-12-07T10:19:01.812930+0000","last_peered":"2023-12-07T10:19:01.812930+0000","last_clean":"2023-12-07T10:19:01.812930+0000","last_became_active":"2023-12-07T10:16:33.481699+0000","last_became_peered":"2023-12-07T10:16:33.481699+0000","last_unstale":"2023-12-07T10:19:01.812930+0000","last_undegraded":"2023-12-07T10:19:01.812930+0000","last_fullsized":"2023-12-07T10:19:01.812930+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:13:25.388258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:13:25.388258+0000","last_clean_scrub_stamp":"2023-12-07T10:13:25.388258+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:57:09.583677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807871+0000","last_change":"2023-12-07T10:17:45.806289+0000","last_active":"2023-12-07T10:19:01.807871+0000","last_peered":"2023-12-07T10:19:01.807871+0000","last_clean":"2023-12-07T10:19:01.807871+0000","last_became_active":"2023-12-07T10:17:45.805954+0000","last_became_peered":"2023-12-07T10:17:45.805954+0000","last_unstale":"2023-12-07T10:19:01.807871+0000","last_undegraded":"2023-12-07T10:19:01.807871+0000","last_fullsized":"2023-12-07T10:19:01.807871+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:21:09.955380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813281+0000","last_change":"2023-12-07T10:17:41.612322+0000","last_active":"2023-12-07T10:19:01.813281+0000","last_peered":"2023-12-07T10:19:01.813281+0000","last_clean":"2023-12-07T10:19:01.813281+0000","last_became_active":"2023-12-07T10:17:41.612039+0000","last_became_peered":"2023-12-07T10:17:41.612039+0000","last_unstale":"2023-12-07T10:19:01.813281+0000","last_undegraded":"2023-12-07T10:19:01.813281+0000","last_fullsized":"2023-12-07T10:19:01.813281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:40:49.883742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.812980+0000","last_change":"2023-12-07T10:17:40.749694+0000","last_active":"2023-12-07T10:19:01.812980+0000","last_peered":"2023-12-07T10:19:01.812980+0000","last_clean":"2023-12-07T10:19:01.812980+0000","last_became_active":"2023-12-07T10:17:40.749393+0000","last_became_peered":"2023-12-07T10:17:40.749393+0000","last_unstale":"2023-12-07T10:19:01.812980+0000","last_undegraded":"2023-12-07T10:19:01.812980+0000","last_fullsized":"2023-12-07T10:19:01.812980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:15:36.949062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.899040+0000","last_change":"2023-12-07T10:17:45.805769+0000","last_active":"2023-12-07T10:19:01.899040+0000","last_peered":"2023-12-07T10:19:01.899040+0000","last_clean":"2023-12-07T10:19:01.899040+0000","last_became_active":"2023-12-07T10:17:45.805346+0000","last_became_peered":"2023-12-07T10:17:45.805346+0000","last_unstale":"2023-12-07T10:19:01.899040+0000","last_undegraded":"2023-12-07T10:19:01.899040+0000","last_fullsized":"2023-12-07T10:19:01.899040+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:47:47.781933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.899044+0000","last_change":"2023-12-07T10:18:01.862558+0000","last_active":"2023-12-07T10:19:01.899044+0000","last_peered":"2023-12-07T10:19:01.899044+0000","last_clean":"2023-12-07T10:19:01.899044+0000","last_became_active":"2023-12-07T10:18:01.862063+0000","last_became_peered":"2023-12-07T10:18:01.862063+0000","last_unstale":"2023-12-07T10:19:01.899044+0000","last_undegraded":"2023-12-07T10:19:01.899044+0000","last_fullsized":"2023-12-07T10:19:01.899044+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:25:32.652160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813607+0000","last_change":"2023-12-07T10:17:41.617199+0000","last_active":"2023-12-07T10:19:01.813607+0000","last_peered":"2023-12-07T10:19:01.813607+0000","last_clean":"2023-12-07T10:19:01.813607+0000","last_became_active":"2023-12-07T10:17:41.616901+0000","last_became_peered":"2023-12-07T10:17:41.616901+0000","last_unstale":"2023-12-07T10:19:01.813607+0000","last_undegraded":"2023-12-07T10:19:01.813607+0000","last_fullsized":"2023-12-07T10:19:01.813607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:23:59.433861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802553+0000","last_change":"2023-12-07T10:17:45.790914+0000","last_active":"2023-12-07T10:19:01.802553+0000","last_peered":"2023-12-07T10:19:01.802553+0000","last_clean":"2023-12-07T10:19:01.802553+0000","last_became_active":"2023-12-07T10:17:45.790428+0000","last_became_peered":"2023-12-07T10:17:45.790428+0000","last_unstale":"2023-12-07T10:19:01.802553+0000","last_undegraded":"2023-12-07T10:19:01.802553+0000","last_fullsized":"2023-12-07T10:19:01.802553+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:03:57.516300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807601+0000","last_change":"2023-12-07T10:18:01.863765+0000","last_active":"2023-12-07T10:19:01.807601+0000","last_peered":"2023-12-07T10:19:01.807601+0000","last_clean":"2023-12-07T10:19:01.807601+0000","last_became_active":"2023-12-07T10:18:01.858926+0000","last_became_peered":"2023-12-07T10:18:01.858926+0000","last_unstale":"2023-12-07T10:19:01.807601+0000","last_undegraded":"2023-12-07T10:19:01.807601+0000","last_fullsized":"2023-12-07T10:19:01.807601+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:44:19.663592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.804412+0000","last_change":"2023-12-07T10:17:41.608513+0000","last_active":"2023-12-07T10:19:01.804412+0000","last_peered":"2023-12-07T10:19:01.804412+0000","last_clean":"2023-12-07T10:19:01.804412+0000","last_became_active":"2023-12-07T10:17:41.605049+0000","last_became_peered":"2023-12-07T10:17:41.605049+0000","last_unstale":"2023-12-07T10:19:01.804412+0000","last_undegraded":"2023-12-07T10:19:01.804412+0000","last_fullsized":"2023-12-07T10:19:01.804412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:33:00.730827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.809743+0000","last_change":"2023-12-07T10:17:45.792583+0000","last_active":"2023-12-07T10:19:01.809743+0000","last_peered":"2023-12-07T10:19:01.809743+0000","last_clean":"2023-12-07T10:19:01.809743+0000","last_became_active":"2023-12-07T10:17:45.792162+0000","last_became_peered":"2023-12-07T10:17:45.792162+0000","last_unstale":"2023-12-07T10:19:01.809743+0000","last_undegraded":"2023-12-07T10:19:01.809743+0000","last_fullsized":"2023-12-07T10:19:01.809743+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:47:30.901477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.805380+0000","last_change":"2023-12-07T10:17:40.750826+0000","last_active":"2023-12-07T10:19:01.805380+0000","last_peered":"2023-12-07T10:19:01.805380+0000","last_clean":"2023-12-07T10:19:01.805380+0000","last_became_active":"2023-12-07T10:17:40.750519+0000","last_became_peered":"2023-12-07T10:17:40.750519+0000","last_unstale":"2023-12-07T10:19:01.805380+0000","last_undegraded":"2023-12-07T10:19:01.805380+0000","last_fullsized":"2023-12-07T10:19:01.805380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:21:01.008897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.805305+0000","last_change":"2023-12-07T10:17:41.609488+0000","last_active":"2023-12-07T10:19:01.805305+0000","last_peered":"2023-12-07T10:19:01.805305+0000","last_clean":"2023-12-07T10:19:01.805305+0000","last_became_active":"2023-12-07T10:17:41.609061+0000","last_became_peered":"2023-12-07T10:17:41.609061+0000","last_unstale":"2023-12-07T10:19:01.805305+0000","last_undegraded":"2023-12-07T10:19:01.805305+0000","last_fullsized":"2023-12-07T10:19:01.805305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:39:32.729502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.805100+0000","last_change":"2023-12-07T10:17:45.797573+0000","last_active":"2023-12-07T10:19:01.805100+0000","last_peered":"2023-12-07T10:19:01.805100+0000","last_clean":"2023-12-07T10:19:01.805100+0000","last_became_active":"2023-12-07T10:17:45.797119+0000","last_became_peered":"2023-12-07T10:17:45.797119+0000","last_unstale":"2023-12-07T10:19:01.805100+0000","last_undegraded":"2023-12-07T10:19:01.805100+0000","last_fullsized":"2023-12-07T10:19:01.805100+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:17:27.100470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810670+0000","last_change":"2023-12-07T10:17:40.753596+0000","last_active":"2023-12-07T10:19:01.810670+0000","last_peered":"2023-12-07T10:19:01.810670+0000","last_clean":"2023-12-07T10:19:01.810670+0000","last_became_active":"2023-12-07T10:17:40.753251+0000","last_became_peered":"2023-12-07T10:17:40.753251+0000","last_unstale":"2023-12-07T10:19:01.810670+0000","last_undegraded":"2023-12-07T10:19:01.810670+0000","last_fullsized":"2023-12-07T10:19:01.810670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:08:30.868794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807743+0000","last_change":"2023-12-07T10:17:41.599689+0000","last_active":"2023-12-07T10:19:01.807743+0000","last_peered":"2023-12-07T10:19:01.807743+0000","last_clean":"2023-12-07T10:19:01.807743+0000","last_became_active":"2023-12-07T10:17:41.599358+0000","last_became_peered":"2023-12-07T10:17:41.599358+0000","last_unstale":"2023-12-07T10:19:01.807743+0000","last_undegraded":"2023-12-07T10:19:01.807743+0000","last_fullsized":"2023-12-07T10:19:01.807743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:44:19.742560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.215169+0000","last_change":"2023-12-07T10:19:03.215169+0000","last_active":"2023-12-07T10:19:03.215169+0000","last_peered":"2023-12-07T10:19:03.215169+0000","last_clean":"2023-12-07T10:19:03.215169+0000","last_became_active":"2023-12-07T10:19:03.214754+0000","last_became_peered":"2023-12-07T10:19:03.214754+0000","last_unstale":"2023-12-07T10:19:03.215169+0000","last_undegraded":"2023-12-07T10:19:03.215169+0000","last_fullsized":"2023-12-07T10:19:03.215169+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:51:41.461131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802597+0000","last_change":"2023-12-07T10:17:40.758663+0000","last_active":"2023-12-07T10:19:01.802597+0000","last_peered":"2023-12-07T10:19:01.802597+0000","last_clean":"2023-12-07T10:19:01.802597+0000","last_became_active":"2023-12-07T10:17:40.753611+0000","last_became_peered":"2023-12-07T10:17:40.753611+0000","last_unstale":"2023-12-07T10:19:01.802597+0000","last_undegraded":"2023-12-07T10:19:01.802597+0000","last_fullsized":"2023-12-07T10:19:01.802597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:00:59.256468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.898763+0000","last_change":"2023-12-07T10:17:41.616546+0000","last_active":"2023-12-07T10:19:01.898763+0000","last_peered":"2023-12-07T10:19:01.898763+0000","last_clean":"2023-12-07T10:19:01.898763+0000","last_became_active":"2023-12-07T10:17:41.615780+0000","last_became_peered":"2023-12-07T10:17:41.615780+0000","last_unstale":"2023-12-07T10:19:01.898763+0000","last_undegraded":"2023-12-07T10:19:01.898763+0000","last_fullsized":"2023-12-07T10:19:01.898763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:22:32.455784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.805077+0000","last_change":"2023-12-07T10:17:45.797550+0000","last_active":"2023-12-07T10:19:01.805077+0000","last_peered":"2023-12-07T10:19:01.805077+0000","last_clean":"2023-12-07T10:19:01.805077+0000","last_became_active":"2023-12-07T10:17:45.797140+0000","last_became_peered":"2023-12-07T10:17:45.797140+0000","last_unstale":"2023-12-07T10:19:01.805077+0000","last_undegraded":"2023-12-07T10:19:01.805077+0000","last_fullsized":"2023-12-07T10:19:01.805077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:53:44.984904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.809964+0000","last_change":"2023-12-07T10:17:40.748410+0000","last_active":"2023-12-07T10:19:01.809964+0000","last_peered":"2023-12-07T10:19:01.809964+0000","last_clean":"2023-12-07T10:19:01.809964+0000","last_became_active":"2023-12-07T10:17:40.748069+0000","last_became_peered":"2023-12-07T10:17:40.748069+0000","last_unstale":"2023-12-07T10:19:01.809964+0000","last_undegraded":"2023-12-07T10:19:01.809964+0000","last_fullsized":"2023-12-07T10:19:01.809964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:51:48.278974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803033+0000","last_change":"2023-12-07T10:17:41.609248+0000","last_active":"2023-12-07T10:19:01.803033+0000","last_peered":"2023-12-07T10:19:01.803033+0000","last_clean":"2023-12-07T10:19:01.803033+0000","last_became_active":"2023-12-07T10:17:41.608943+0000","last_became_peered":"2023-12-07T10:17:41.608943+0000","last_unstale":"2023-12-07T10:19:01.803033+0000","last_undegraded":"2023-12-07T10:19:01.803033+0000","last_fullsized":"2023-12-07T10:19:01.803033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:29:32.322273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.805250+0000","last_change":"2023-12-07T10:17:45.794855+0000","last_active":"2023-12-07T10:19:01.805250+0000","last_peered":"2023-12-07T10:19:01.805250+0000","last_clean":"2023-12-07T10:19:01.805250+0000","last_became_active":"2023-12-07T10:17:45.794410+0000","last_became_peered":"2023-12-07T10:17:45.794410+0000","last_unstale":"2023-12-07T10:19:01.805250+0000","last_undegraded":"2023-12-07T10:19:01.805250+0000","last_fullsized":"2023-12-07T10:19:01.805250+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:57:25.118036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808678+0000","last_change":"2023-12-07T10:17:40.750471+0000","last_active":"2023-12-07T10:19:01.808678+0000","last_peered":"2023-12-07T10:19:01.808678+0000","last_clean":"2023-12-07T10:19:01.808678+0000","last_became_active":"2023-12-07T10:17:40.750176+0000","last_became_peered":"2023-12-07T10:17:40.750176+0000","last_unstale":"2023-12-07T10:19:01.808678+0000","last_undegraded":"2023-12-07T10:19:01.808678+0000","last_fullsized":"2023-12-07T10:19:01.808678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:04:22.305343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.812625+0000","last_change":"2023-12-07T10:17:41.614087+0000","last_active":"2023-12-07T10:19:01.812625+0000","last_peered":"2023-12-07T10:19:01.812625+0000","last_clean":"2023-12-07T10:19:01.812625+0000","last_became_active":"2023-12-07T10:17:41.607689+0000","last_became_peered":"2023-12-07T10:17:41.607689+0000","last_unstale":"2023-12-07T10:19:01.812625+0000","last_undegraded":"2023-12-07T10:19:01.812625+0000","last_fullsized":"2023-12-07T10:19:01.812625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:50:10.316085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.841001+0000","last_change":"2023-12-07T10:19:02.841001+0000","last_active":"2023-12-07T10:19:02.841001+0000","last_peered":"2023-12-07T10:19:02.841001+0000","last_clean":"2023-12-07T10:19:02.841001+0000","last_became_active":"2023-12-07T10:19:02.840595+0000","last_became_peered":"2023-12-07T10:19:02.840595+0000","last_unstale":"2023-12-07T10:19:02.841001+0000","last_undegraded":"2023-12-07T10:19:02.841001+0000","last_fullsized":"2023-12-07T10:19:02.841001+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:18:35.590140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'5","reported_seq":84,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:07.847749+0000","last_change":"2023-12-07T10:17:40.753645+0000","last_active":"2023-12-07T10:19:07.847749+0000","last_peered":"2023-12-07T10:19:07.847749+0000","last_clean":"2023-12-07T10:19:07.847749+0000","last_became_active":"2023-12-07T10:17:40.753326+0000","last_became_peered":"2023-12-07T10:17:40.753326+0000","last_unstale":"2023-12-07T10:19:07.847749+0000","last_undegraded":"2023-12-07T10:19:07.847749+0000","last_fullsized":"2023-12-07T10:19:07.847749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:43:03.617591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811725+0000","last_change":"2023-12-07T10:17:41.607968+0000","last_active":"2023-12-07T10:19:01.811725+0000","last_peered":"2023-12-07T10:19:01.811725+0000","last_clean":"2023-12-07T10:19:01.811725+0000","last_became_active":"2023-12-07T10:17:41.607621+0000","last_became_peered":"2023-12-07T10:17:41.607621+0000","last_unstale":"2023-12-07T10:19:01.811725+0000","last_undegraded":"2023-12-07T10:19:01.811725+0000","last_fullsized":"2023-12-07T10:19:01.811725+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:55:10.224738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803538+0000","last_change":"2023-12-07T10:17:45.793678+0000","last_active":"2023-12-07T10:19:01.803538+0000","last_peered":"2023-12-07T10:19:01.803538+0000","last_clean":"2023-12-07T10:19:01.803538+0000","last_became_active":"2023-12-07T10:17:45.793283+0000","last_became_peered":"2023-12-07T10:17:45.793283+0000","last_unstale":"2023-12-07T10:19:01.803538+0000","last_undegraded":"2023-12-07T10:19:01.803538+0000","last_fullsized":"2023-12-07T10:19:01.803538+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:33:56.501107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808567+0000","last_change":"2023-12-07T10:17:40.762069+0000","last_active":"2023-12-07T10:19:01.808567+0000","last_peered":"2023-12-07T10:19:01.808567+0000","last_clean":"2023-12-07T10:19:01.808567+0000","last_became_active":"2023-12-07T10:17:40.761752+0000","last_became_peered":"2023-12-07T10:17:40.761752+0000","last_unstale":"2023-12-07T10:19:01.808567+0000","last_undegraded":"2023-12-07T10:19:01.808567+0000","last_fullsized":"2023-12-07T10:19:01.808567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:57:55.736144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811362+0000","last_change":"2023-12-07T10:17:41.607347+0000","last_active":"2023-12-07T10:19:01.811362+0000","last_peered":"2023-12-07T10:19:01.811362+0000","last_clean":"2023-12-07T10:19:01.811362+0000","last_became_active":"2023-12-07T10:17:41.606618+0000","last_became_peered":"2023-12-07T10:17:41.606618+0000","last_unstale":"2023-12-07T10:19:01.811362+0000","last_undegraded":"2023-12-07T10:19:01.811362+0000","last_fullsized":"2023-12-07T10:19:01.811362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:53:34.748130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807110+0000","last_change":"2023-12-07T10:17:45.791459+0000","last_active":"2023-12-07T10:19:01.807110+0000","last_peered":"2023-12-07T10:19:01.807110+0000","last_clean":"2023-12-07T10:19:01.807110+0000","last_became_active":"2023-12-07T10:17:45.791070+0000","last_became_peered":"2023-12-07T10:17:45.791070+0000","last_unstale":"2023-12-07T10:19:01.807110+0000","last_undegraded":"2023-12-07T10:19:01.807110+0000","last_fullsized":"2023-12-07T10:19:01.807110+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:46:35.985990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813745+0000","last_change":"2023-12-07T10:17:40.762726+0000","last_active":"2023-12-07T10:19:01.813745+0000","last_peered":"2023-12-07T10:19:01.813745+0000","last_clean":"2023-12-07T10:19:01.813745+0000","last_became_active":"2023-12-07T10:17:40.762378+0000","last_became_peered":"2023-12-07T10:17:40.762378+0000","last_unstale":"2023-12-07T10:19:01.813745+0000","last_undegraded":"2023-12-07T10:19:01.813745+0000","last_fullsized":"2023-12-07T10:19:01.813745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:42:06.193834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811313+0000","last_change":"2023-12-07T10:17:41.602356+0000","last_active":"2023-12-07T10:19:01.811313+0000","last_peered":"2023-12-07T10:19:01.811313+0000","last_clean":"2023-12-07T10:19:01.811313+0000","last_became_active":"2023-12-07T10:17:41.602025+0000","last_became_peered":"2023-12-07T10:17:41.602025+0000","last_unstale":"2023-12-07T10:19:01.811313+0000","last_undegraded":"2023-12-07T10:19:01.811313+0000","last_fullsized":"2023-12-07T10:19:01.811313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:33:04.994510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.898923+0000","last_change":"2023-12-07T10:17:41.616766+0000","last_active":"2023-12-07T10:19:01.898923+0000","last_peered":"2023-12-07T10:19:01.898923+0000","last_clean":"2023-12-07T10:19:01.898923+0000","last_became_active":"2023-12-07T10:17:41.616221+0000","last_became_peered":"2023-12-07T10:17:41.616221+0000","last_unstale":"2023-12-07T10:19:01.898923+0000","last_undegraded":"2023-12-07T10:19:01.898923+0000","last_fullsized":"2023-12-07T10:19:01.898923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:16:00.393772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810004+0000","last_change":"2023-12-07T10:17:41.613646+0000","last_active":"2023-12-07T10:19:01.810004+0000","last_peered":"2023-12-07T10:19:01.810004+0000","last_clean":"2023-12-07T10:19:01.810004+0000","last_became_active":"2023-12-07T10:17:41.613257+0000","last_became_peered":"2023-12-07T10:17:41.613257+0000","last_unstale":"2023-12-07T10:19:01.810004+0000","last_undegraded":"2023-12-07T10:19:01.810004+0000","last_fullsized":"2023-12-07T10:19:01.810004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:10:33.416955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810008+0000","last_change":"2023-12-07T10:17:41.620017+0000","last_active":"2023-12-07T10:19:01.810008+0000","last_peered":"2023-12-07T10:19:01.810008+0000","last_clean":"2023-12-07T10:19:01.810008+0000","last_became_active":"2023-12-07T10:17:41.619647+0000","last_became_peered":"2023-12-07T10:17:41.619647+0000","last_unstale":"2023-12-07T10:19:01.810008+0000","last_undegraded":"2023-12-07T10:19:01.810008+0000","last_fullsized":"2023-12-07T10:19:01.810008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:36:42.891160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803726+0000","last_change":"2023-12-07T10:17:41.610803+0000","last_active":"2023-12-07T10:19:01.803726+0000","last_peered":"2023-12-07T10:19:01.803726+0000","last_clean":"2023-12-07T10:19:01.803726+0000","last_became_active":"2023-12-07T10:17:41.610388+0000","last_became_peered":"2023-12-07T10:17:41.610388+0000","last_unstale":"2023-12-07T10:19:01.803726+0000","last_undegraded":"2023-12-07T10:19:01.803726+0000","last_fullsized":"2023-12-07T10:19:01.803726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:33:25.787281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813690+0000","last_change":"2023-12-07T10:17:41.615916+0000","last_active":"2023-12-07T10:19:01.813690+0000","last_peered":"2023-12-07T10:19:01.813690+0000","last_clean":"2023-12-07T10:19:01.813690+0000","last_became_active":"2023-12-07T10:17:41.615591+0000","last_became_peered":"2023-12-07T10:17:41.615591+0000","last_unstale":"2023-12-07T10:19:01.813690+0000","last_undegraded":"2023-12-07T10:19:01.813690+0000","last_fullsized":"2023-12-07T10:19:01.813690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:40:45.541571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.899185+0000","last_change":"2023-12-07T10:17:41.614735+0000","last_active":"2023-12-07T10:19:01.899185+0000","last_peered":"2023-12-07T10:19:01.899185+0000","last_clean":"2023-12-07T10:19:01.899185+0000","last_became_active":"2023-12-07T10:17:41.614388+0000","last_became_peered":"2023-12-07T10:17:41.614388+0000","last_unstale":"2023-12-07T10:19:01.899185+0000","last_undegraded":"2023-12-07T10:19:01.899185+0000","last_fullsized":"2023-12-07T10:19:01.899185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:10:07.196841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810348+0000","last_change":"2023-12-07T10:17:41.620059+0000","last_active":"2023-12-07T10:19:01.810348+0000","last_peered":"2023-12-07T10:19:01.810348+0000","last_clean":"2023-12-07T10:19:01.810348+0000","last_became_active":"2023-12-07T10:17:41.619770+0000","last_became_peered":"2023-12-07T10:17:41.619770+0000","last_unstale":"2023-12-07T10:19:01.810348+0000","last_undegraded":"2023-12-07T10:19:01.810348+0000","last_fullsized":"2023-12-07T10:19:01.810348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:13:43.421776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.805006+0000","last_change":"2023-12-07T10:17:41.609720+0000","last_active":"2023-12-07T10:19:01.805006+0000","last_peered":"2023-12-07T10:19:01.805006+0000","last_clean":"2023-12-07T10:19:01.805006+0000","last_became_active":"2023-12-07T10:17:41.609417+0000","last_became_peered":"2023-12-07T10:17:41.609417+0000","last_unstale":"2023-12-07T10:19:01.805006+0000","last_undegraded":"2023-12-07T10:19:01.805006+0000","last_fullsized":"2023-12-07T10:19:01.805006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:48:47.768754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807362+0000","last_change":"2023-12-07T10:17:41.620952+0000","last_active":"2023-12-07T10:19:01.807362+0000","last_peered":"2023-12-07T10:19:01.807362+0000","last_clean":"2023-12-07T10:19:01.807362+0000","last_became_active":"2023-12-07T10:17:41.620560+0000","last_became_peered":"2023-12-07T10:17:41.620560+0000","last_unstale":"2023-12-07T10:19:01.807362+0000","last_undegraded":"2023-12-07T10:19:01.807362+0000","last_fullsized":"2023-12-07T10:19:01.807362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:51:37.699511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802559+0000","last_change":"2023-12-07T10:17:41.611908+0000","last_active":"2023-12-07T10:19:01.802559+0000","last_peered":"2023-12-07T10:19:01.802559+0000","last_clean":"2023-12-07T10:19:01.802559+0000","last_became_active":"2023-12-07T10:17:41.610433+0000","last_became_peered":"2023-12-07T10:17:41.610433+0000","last_unstale":"2023-12-07T10:19:01.802559+0000","last_undegraded":"2023-12-07T10:19:01.802559+0000","last_fullsized":"2023-12-07T10:19:01.802559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:42:53.906221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811612+0000","last_change":"2023-12-07T10:17:41.604116+0000","last_active":"2023-12-07T10:19:01.811612+0000","last_peered":"2023-12-07T10:19:01.811612+0000","last_clean":"2023-12-07T10:19:01.811612+0000","last_became_active":"2023-12-07T10:17:41.603880+0000","last_became_peered":"2023-12-07T10:17:41.603880+0000","last_unstale":"2023-12-07T10:19:01.811612+0000","last_undegraded":"2023-12-07T10:19:01.811612+0000","last_fullsized":"2023-12-07T10:19:01.811612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:56:27.343169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813422+0000","last_change":"2023-12-07T10:17:41.610497+0000","last_active":"2023-12-07T10:19:01.813422+0000","last_peered":"2023-12-07T10:19:01.813422+0000","last_clean":"2023-12-07T10:19:01.813422+0000","last_became_active":"2023-12-07T10:17:41.609956+0000","last_became_peered":"2023-12-07T10:17:41.609956+0000","last_unstale":"2023-12-07T10:19:01.813422+0000","last_undegraded":"2023-12-07T10:19:01.813422+0000","last_fullsized":"2023-12-07T10:19:01.813422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:04:03.967377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.899100+0000","last_change":"2023-12-07T10:17:41.612284+0000","last_active":"2023-12-07T10:19:01.899100+0000","last_peered":"2023-12-07T10:19:01.899100+0000","last_clean":"2023-12-07T10:19:01.899100+0000","last_became_active":"2023-12-07T10:17:41.611917+0000","last_became_peered":"2023-12-07T10:17:41.611917+0000","last_unstale":"2023-12-07T10:19:01.899100+0000","last_undegraded":"2023-12-07T10:19:01.899100+0000","last_fullsized":"2023-12-07T10:19:01.899100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:02:27.355883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":81645,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":617,"num_read_kb":396,"num_write":243,"num_write_kb":152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":244935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383710,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":516906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.434,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.503},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.72299999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.62,"15min":0.60499999999999998},"min":{"1min":0.505,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.40899999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.76800000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.68899999999999995}]},{"osd":2,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.45900000000000002,"15min":0.45000000000000001},"min":{"1min":0.36899999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.46999999999999997,"15min":0.45700000000000002},"min":{"1min":0.376,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.45300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.67900000000000005,"15min":0.67200000000000004},"min":{"1min":0.48899999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90300000000000002,"5min":1.046,"15min":1.046},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.84699999999999998,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.59599999999999997}]},{"osd":5,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.56799999999999995,"15min":0.55700000000000005},"min":{"1min":0.40200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80700000000000005,"5min":1.034,"15min":1.034},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.59199999999999997,"15min":0.57999999999999996},"min":{"1min":0.51300000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92900000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.60599999999999998}]},{"osd":6,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.46200000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90700000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.72399999999999998,"15min":0.71599999999999997},"min":{"1min":0.60399999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91300000000000003,"5min":1.363,"15min":1.363},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.76100000000000001,"15min":0.751},"min":{"1min":0.63700000000000001,"5min":0.501,"15min":0.501},"max":{"1min":0.97899999999999998,"5min":1.25,"15min":1.25},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.70599999999999996,"15min":0.69599999999999995},"min":{"1min":0.51500000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95399999999999996,"5min":1.085,"15min":1.085},"last":0.54800000000000004}]},{"osd":8,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.58499999999999996}]},{"osd":9,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.78000000000000003,"15min":0.76600000000000001},"min":{"1min":0.69799999999999995,"5min":0.496,"15min":0.496},"max":{"1min":0.96799999999999997,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.78700000000000003,"15min":0.77500000000000002},"min":{"1min":0.71899999999999997,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.014,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.79800000000000004}]},{"osd":10,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.751,"15min":0.74099999999999999},"min":{"1min":0.59999999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.996,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.79300000000000004,"15min":0.78300000000000003},"min":{"1min":0.70999999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.999,"5min":1.411,"15min":1.411},"last":0.70299999999999996}]}]},{"osd":10,"up_from":72,"seq":309237645346,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28732,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716420,"statfs":{"total":95995035648,"available":95965614080,"internally_reserved":0,"allocated":1748992,"data_stored":1168120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.3610000000000002,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.54000000000000004}]},{"osd":1,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.48599999999999999,"15min":0.49099999999999999},"min":{"1min":0.34100000000000003,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.627,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.49299999999999999,"15min":0.495},"min":{"1min":0.36299999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.40100000000000002}]},{"osd":2,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82699999999999996,"5min":1.008,"15min":1.008},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.42699999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81299999999999994,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.78900000000000003}]},{"osd":3,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.45500000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.78300000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.67900000000000005,"15min":0.68799999999999994},"min":{"1min":0.48199999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.79000000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.59199999999999997}]},{"osd":4,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57599999999999996,"15min":0.58299999999999996},"min":{"1min":0.33300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71799999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58599999999999997,"15min":0.59099999999999997},"min":{"1min":0.40699999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.751,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.42899999999999999}]},{"osd":5,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.95499999999999996,"5min":1.22,"15min":1.22},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72699999999999998,"15min":0.72799999999999998},"min":{"1min":0.49299999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93899999999999995,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.73299999999999998}]},{"osd":6,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.747,"15min":0.751},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.89200000000000002,"5min":1.054,"15min":1.054},"last":0.497},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78900000000000003,"15min":0.78700000000000003},"min":{"1min":0.48499999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.64700000000000002}]},{"osd":7,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.63900000000000001}]},{"osd":8,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.79600000000000004,"15min":0.80500000000000005},"min":{"1min":0.53100000000000003,"5min":0.503,"15min":0.503},"max":{"1min":0.90700000000000003,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.83099999999999996,"15min":0.84199999999999997},"min":{"1min":0.629,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.91700000000000004,"5min":1.165,"15min":1.165},"last":0.71799999999999997}]},{"osd":9,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.83399999999999996,"15min":0.83999999999999997},"min":{"1min":0.622,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.98399999999999999,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.753},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.85199999999999998,"15min":0.86599999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.66700000000000004}]},{"osd":11,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.81699999999999995,"15min":0.82099999999999995},"min":{"1min":0.56899999999999995,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.95599999999999996,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81999999999999995,"15min":0.82199999999999995},"min":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.026,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.77100000000000002}]}]},{"osd":9,"up_from":67,"seq":287762808871,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":520820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.60399999999999998,"15min":0.62},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.92600000000000005,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58099999999999996,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74299999999999999,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.48799999999999999}]},{"osd":1,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.69399999999999995,"15min":0.70699999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97999999999999998,"5min":1.899,"15min":1.899},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.77700000000000002,"15min":0.79300000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.94399999999999995,"5min":1.409,"15min":1.409},"last":0.69299999999999995}]},{"osd":2,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.80000000000000004,"15min":0.81899999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.91800000000000004,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.873,"15min":0.89800000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.92900000000000005,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":0.77000000000000002}]},{"osd":3,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.72799999999999998,"15min":0.746},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.878,"5min":1.47,"15min":1.47},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.81699999999999995,"15min":0.85499999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.84799999999999998,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.63}]},{"osd":4,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.80000000000000004,"15min":0.82999999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.85099999999999998,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.752,"5min":0.88,"15min":0.90200000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.109,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.57499999999999996}]},{"osd":5,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.92800000000000005,"15min":0.95599999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.089,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.748},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.84699999999999998,"15min":0.876},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.76000000000000001,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":0.85499999999999998}]},{"osd":6,"last update":"Thu Dec 7 10:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":0.749}]},{"osd":7,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.93400000000000005,"15min":0.95799999999999996},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.0820000000000001,"5min":1.5,"15min":1.5},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":1.016,"15min":1.052},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.1519999999999999,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":0.67500000000000004}]},{"osd":8,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.95599999999999996,"15min":0.97399999999999998},"min":{"1min":0.63800000000000001,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0600000000000001,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":1.0409999999999999,"15min":1.071},"min":{"1min":0.626,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.194,"5min":2.6539999999999999,"15min":2.6539999999999999},"last":0.86399999999999999}]},{"osd":10,"last update":"Thu Dec 7 10:18:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":1.02,"15min":1.052},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.2889999999999999,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.93899999999999995,"15min":0.96899999999999997},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.0189999999999999,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.72699999999999998}]},{"osd":11,"last update":"Thu Dec 7 10:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.80200000000000005}]}]},{"osd":8,"up_from":59,"seq":253403070509,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":516990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.499},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55900000000000005,"15min":0.56299999999999994},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.55700000000000005}]},{"osd":1,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.59199999999999997,"15min":0.57699999999999996},"min":{"1min":0.45100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86299999999999999,"5min":1,"15min":1},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.60299999999999998,"15min":0.58299999999999996},"min":{"1min":0.52800000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.70199999999999996}]},{"osd":2,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.46200000000000002,"15min":0.44400000000000001},"min":{"1min":0.317,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.76700000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.46300000000000002,"15min":0.438},"min":{"1min":0.29699999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.64800000000000002}]},{"osd":3,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59799999999999998,"15min":0.58599999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.83199999999999996,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.498,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.86899999999999999,"5min":0.876,"15min":0.876},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.64400000000000002,"15min":0.63400000000000001},"min":{"1min":0.54400000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89100000000000001,"5min":1.079,"15min":1.079},"last":0.81200000000000006}]},{"osd":5,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.54100000000000004,"15min":0.51700000000000002},"min":{"1min":0.34799999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56799999999999995,"15min":0.55300000000000005},"min":{"1min":0.38200000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.84199999999999997,"5min":1.024,"15min":1.024},"last":0.82399999999999995}]},{"osd":6,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.67400000000000004,"15min":0.65600000000000003},"min":{"1min":0.499,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.96099999999999997,"5min":1.046,"15min":1.046},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.68999999999999995,"15min":0.66400000000000003},"min":{"1min":0.48699999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.75600000000000001}]},{"osd":7,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.68700000000000006,"15min":0.65500000000000003},"min":{"1min":0.57999999999999996,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.69199999999999995,"15min":0.67500000000000004},"min":{"1min":0.58699999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.95599999999999996,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.89500000000000002}]},{"osd":9,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.71999999999999997,"15min":0.71499999999999997},"min":{"1min":0.53900000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.74399999999999999,"15min":0.73399999999999999},"min":{"1min":0.59899999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.88300000000000001}]},{"osd":10,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.74399999999999999,"15min":0.72699999999999998},"min":{"1min":0.626,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.74299999999999999,"15min":0.72799999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.90300000000000002}]},{"osd":11,"last update":"Thu Dec 7 10:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.85899999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299441,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1679360,"data_stored":1123647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.58199999999999996,"15min":0.55200000000000005},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.5369999999999999,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56599999999999995,"15min":0.56000000000000005},"min":{"1min":0.39700000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.36899999999999999}]},{"osd":1,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.434,"15min":0.432},"min":{"1min":0.29699999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60499999999999998,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.378},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45600000000000002},"min":{"1min":0.254,"5min":0.253,"15min":0.253},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.52600000000000002}]},{"osd":2,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.434,"5min":0.314,"15min":0.314},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.57899999999999996,"15min":0.57099999999999995},"min":{"1min":0.39600000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73499999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.45500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.79900000000000004,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64000000000000001}]},{"osd":4,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51900000000000002,"15min":0.51600000000000001},"min":{"1min":0.315,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.748,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.52400000000000002,"15min":0.53300000000000003},"min":{"1min":0.33600000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.77400000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.44400000000000001}]},{"osd":5,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63900000000000001,"15min":0.64400000000000002},"min":{"1min":0.45600000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.79200000000000004,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.443,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.84999999999999998,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.61599999999999999}]},{"osd":6,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85699999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.68600000000000005,"15min":0.69199999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94999999999999996,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.71799999999999997}]},{"osd":8,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70099999999999996,"15min":0.69599999999999995},"min":{"1min":0.46200000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90600000000000003,"5min":1.012,"15min":1.012},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72399999999999998,"15min":0.72799999999999998},"min":{"1min":0.52900000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83399999999999996,"5min":1.026,"15min":1.026},"last":0.80500000000000005}]},{"osd":9,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.73599999999999999,"15min":0.74199999999999999},"min":{"1min":0.54600000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88500000000000001,"5min":1.075,"15min":1.075},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74199999999999999,"15min":0.75},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.92300000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.73499999999999999}]},{"osd":10,"last update":"Thu Dec 7 10:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.76300000000000001,"15min":0.76600000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.96099999999999997,"5min":1.03,"15min":1.03},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73999999999999999,"15min":0.74199999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.89200000000000002,"5min":1.002,"15min":1.002},"last":0.68200000000000005}]},{"osd":11,"last update":"Thu Dec 7 10:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.79200000000000004}]}]},{"osd":6,"up_from":47,"seq":201863462965,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":512228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.5,"15min":0.48199999999999998},"min":{"1min":0.374,"5min":0.316,"15min":0.316},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.53000000000000003,"15min":0.48799999999999999},"min":{"1min":0.35399999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.45100000000000001}]},{"osd":1,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.69299999999999995,"15min":0.67100000000000004},"min":{"1min":0.56399999999999995,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":3.077,"5min":3.077,"15min":3.077},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.70599999999999996,"15min":0.67300000000000004},"min":{"1min":0.55300000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.113,"5min":2.113,"15min":2.113},"last":0.69199999999999995}]},{"osd":2,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.71999999999999997,"15min":0.68200000000000005},"min":{"1min":0.51400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.42299999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":3.153,"5min":3.153,"15min":3.153},"last":0.49099999999999999}]},{"osd":3,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":3.1200000000000001,"5min":3.1200000000000001,"15min":3.1200000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.67600000000000005,"15min":0.65100000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.51200000000000001}]},{"osd":4,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.77900000000000003,"15min":0.76300000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":3.2629999999999999,"5min":3.2629999999999999,"15min":3.2629999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.76000000000000001,"15min":0.748},"min":{"1min":0.56799999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.58099999999999996}]},{"osd":5,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.79600000000000004,"15min":0.80200000000000005},"min":{"1min":0.628,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.81200000000000006,"15min":0.79600000000000004},"min":{"1min":0.55200000000000005,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.5369999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.85799999999999998}]},{"osd":7,"last update":"Thu Dec 7 10:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.78000000000000003,"15min":0.78500000000000003},"min":{"1min":0.47399999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0489999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.80400000000000005,"15min":0.78800000000000003},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.97999999999999998,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.74399999999999999}]},{"osd":8,"last update":"Thu Dec 7 10:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.85999999999999999,"15min":0.86899999999999999},"min":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.569,"5min":2.569,"15min":2.569},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.88600000000000001,"15min":0.89400000000000002},"min":{"1min":0.53400000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":3.1850000000000001,"5min":3.1850000000000001,"15min":3.1850000000000001},"last":0.79200000000000004}]},{"osd":9,"last update":"Thu Dec 7 10:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.86499999999999999,"15min":0.871},"min":{"1min":0.66800000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1160000000000001,"5min":1.179,"15min":1.179},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.84499999999999997,"15min":0.84899999999999998},"min":{"1min":0.65700000000000003,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.0600000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.89700000000000002}]},{"osd":10,"last update":"Thu Dec 7 10:18:34 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.85999999999999999,"15min":0.84999999999999998},"min":{"1min":0.58299999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.3980000000000001,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.88600000000000001,"15min":0.88300000000000001},"min":{"1min":0.73899999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.84199999999999997}]},{"osd":11,"last update":"Thu Dec 7 10:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.92700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1126400,"data_stored":577711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.627,"15min":0.56499999999999995},"min":{"1min":0.39200000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.92500000000000004,"5min":6.8120000000000003,"15min":6.8120000000000003},"last":0.497},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.57299999999999995,"15min":0.54600000000000004},"min":{"1min":0.433,"5min":0.376,"15min":0.376},"max":{"1min":0.82999999999999996,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.67500000000000004}]},{"osd":2,"last update":"Thu Dec 7 10:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.59499999999999997,"15min":0.54300000000000004},"min":{"1min":0.46899999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.87,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.55800000000000005,"15min":0.51000000000000001},"min":{"1min":0.42999999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.77800000000000002}]},{"osd":3,"last update":"Thu Dec 7 10:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60499999999999998,"15min":0.56299999999999994},"min":{"1min":0.39400000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.82399999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.37,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.67200000000000004,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Dec 7 10:18:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.437,"15min":0.39700000000000002},"min":{"1min":0.35599999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.47899999999999998,"15min":0.42099999999999999},"min":{"1min":0.36599999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.77100000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.73199999999999998}]},{"osd":5,"last update":"Thu Dec 7 10:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.65700000000000003,"15min":0.58799999999999997},"min":{"1min":0.45000000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.98099999999999998,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.60599999999999998,"15min":0.56200000000000006},"min":{"1min":0.51600000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82099999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.82299999999999995}]},{"osd":6,"last update":"Thu Dec 7 10:19:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.71799999999999997,"15min":0.67600000000000005},"min":{"1min":0.54000000000000004,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.101,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65700000000000003,"15min":0.64300000000000002},"min":{"1min":0.49199999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.83099999999999996,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.70799999999999996}]},{"osd":7,"last update":"Thu Dec 7 10:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.54900000000000004,"15min":0.52600000000000002},"min":{"1min":0.38100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.67300000000000004,"15min":0.63400000000000001},"min":{"1min":0.46300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93799999999999994,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.95099999999999996}]},{"osd":8,"last update":"Thu Dec 7 10:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.77100000000000002,"15min":0.73499999999999999},"min":{"1min":0.70999999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.026,"5min":1.175,"15min":1.175},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67100000000000004,"15min":0.64500000000000002},"min":{"1min":0.55500000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86699999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.76200000000000001}]},{"osd":9,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.76300000000000001,"15min":0.754},"min":{"1min":0.58299999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.85399999999999998,"15min":0.84899999999999998},"min":{"1min":0.65700000000000003,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2050000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":1.028}]},{"osd":10,"last update":"Thu Dec 7 10:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.73999999999999999,"15min":0.73699999999999999},"min":{"1min":0.63200000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.75700000000000001,"15min":0.751},"min":{"1min":0.61099999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.93100000000000005}]},{"osd":11,"last update":"Thu Dec 7 10:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":1.002},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.90100000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1081344,"data_stored":513684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.59899999999999998,"15min":0.57199999999999995},"min":{"1min":0.443,"5min":0.375,"15min":0.375},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.41899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Dec 7 10:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.63400000000000001,"15min":0.59599999999999997},"min":{"1min":0.44500000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.3420000000000001,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.752},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.64600000000000002,"15min":0.60499999999999998},"min":{"1min":0.42299999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":3.4140000000000001,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.63700000000000001}]},{"osd":3,"last update":"Thu Dec 7 10:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":3.484,"5min":3.484,"15min":3.484},"last":0.441},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":3.1379999999999999,"5min":3.1379999999999999,"15min":3.1379999999999999},"last":0.51200000000000001}]},{"osd":4,"last update":"Thu Dec 7 10:18:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.59999999999999998,"15min":0.56599999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91700000000000004,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.623,"15min":0.60499999999999998},"min":{"1min":0.46000000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82799999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.66600000000000004}]},{"osd":5,"last update":"Thu Dec 7 10:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.63900000000000001,"15min":0.624},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91000000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.626,"15min":0.624},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85399999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.628}]},{"osd":6,"last update":"Thu Dec 7 10:19:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63600000000000001,"15min":0.628},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":2.5,"5min":2.5,"15min":2.5},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.67200000000000004,"15min":0.64400000000000002},"min":{"1min":0.38100000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.452,"5min":3.452,"15min":3.452},"last":0.48799999999999999}]},{"osd":7,"last update":"Thu Dec 7 10:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":3.0430000000000001,"5min":3.0430000000000001,"15min":3.0430000000000001},"last":0.73299999999999998}]},{"osd":8,"last update":"Thu Dec 7 10:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.81499999999999995,"15min":0.80500000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":3.75,"5min":3.75,"15min":3.75},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.80600000000000005,"15min":0.81100000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.0249999999999999}]},{"osd":9,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.84899999999999998,"15min":0.86199999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0609999999999999,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.82999999999999996,"15min":0.83999999999999997},"min":{"1min":0.501,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.77100000000000002}]},{"osd":10,"last update":"Thu Dec 7 10:18:35 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.79900000000000004,"15min":0.77900000000000003},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":3.0979999999999999,"5min":3.0979999999999999,"15min":3.0979999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.88,"5min":0.85199999999999998,"15min":0.84699999999999998},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.383,"5min":2.383,"15min":2.383},"last":1.0580000000000001}]},{"osd":11,"last update":"Thu Dec 7 10:18:55 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.84599999999999997,"15min":0.81799999999999995},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.7879999999999998,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.82399999999999995,"15min":0.80200000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.1749999999999998,"5min":3.1749999999999998,"15min":3.1749999999999998},"last":0.86099999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":588201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.51000000000000001},"min":{"1min":0.41599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55100000000000005,"15min":0.52000000000000002},"min":{"1min":0.35899999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.51400000000000001}]},{"osd":1,"last update":"Thu Dec 7 10:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.58099999999999996,"15min":0.52800000000000002},"min":{"1min":0.42199999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.56200000000000006,"15min":0.52300000000000002},"min":{"1min":0.39800000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79700000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.65500000000000003}]},{"osd":3,"last update":"Thu Dec 7 10:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.60899999999999999,"15min":0.56000000000000005},"min":{"1min":0.41399999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60399999999999998,"15min":0.56399999999999995},"min":{"1min":0.42699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.95299999999999996,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.59999999999999998}]},{"osd":4,"last update":"Thu Dec 7 10:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.89000000000000001}]},{"osd":5,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.47899999999999998,"15min":0.443},"min":{"1min":0.39400000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.77200000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.501,"15min":0.47199999999999998},"min":{"1min":0.379,"5min":0.307,"15min":0.307},"max":{"1min":0.81299999999999994,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.68000000000000005}]},{"osd":6,"last update":"Thu Dec 7 10:19:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68000000000000005,"15min":0.67000000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.69199999999999995,"15min":0.66000000000000003},"min":{"1min":0.57699999999999996,"5min":0.5,"15min":0.5},"max":{"1min":0.96699999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.79700000000000004}]},{"osd":7,"last update":"Thu Dec 7 10:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.77800000000000002}]},{"osd":8,"last update":"Thu Dec 7 10:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.625,"15min":0.60399999999999998},"min":{"1min":0.53400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.68000000000000005,"15min":0.66400000000000003},"min":{"1min":0.60799999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0229999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.751}]},{"osd":9,"last update":"Thu Dec 7 10:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.78800000000000003,"15min":0.78400000000000003},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0389999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":1.0209999999999999}]},{"osd":10,"last update":"Thu Dec 7 10:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.76600000000000001,"15min":0.752},"min":{"1min":0.57999999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.77800000000000002,"15min":0.76600000000000001},"min":{"1min":0.57099999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.94199999999999995}]},{"osd":11,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.97799999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149763,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":511605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44900000000000001,"15min":0.40300000000000002},"min":{"1min":0.33500000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.624,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.46700000000000003,"15min":0.41299999999999998},"min":{"1min":0.35599999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.72599999999999998,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.375}]},{"osd":1,"last update":"Thu Dec 7 10:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.68899999999999995,"15min":0.58699999999999997},"min":{"1min":0.55300000000000005,"5min":0.438,"15min":0.438},"max":{"1min":0.98199999999999998,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.56100000000000005},"min":{"1min":0.41899999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89000000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.66800000000000004}]},{"osd":2,"last update":"Thu Dec 7 10:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.59399999999999997},"min":{"1min":0.44600000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86799999999999999,"5min":1.087,"15min":1.087},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68200000000000005,"15min":0.61799999999999999},"min":{"1min":0.53800000000000003,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91500000000000004,"5min":1.177,"15min":1.177},"last":0.79600000000000004}]},{"osd":4,"last update":"Thu Dec 7 10:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.68600000000000005,"15min":0.62},"min":{"1min":0.503,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.018,"5min":1.208,"15min":1.208},"last":0.62},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63200000000000001,"15min":0.58999999999999997},"min":{"1min":0.46999999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83899999999999997,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.70099999999999996}]},{"osd":5,"last update":"Thu Dec 7 10:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.623,"15min":0.61099999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.82099999999999995,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.71399999999999997,"15min":0.69599999999999995},"min":{"1min":0.60499999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96199999999999997,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.75700000000000001}]},{"osd":6,"last update":"Thu Dec 7 10:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65300000000000002,"15min":0.64200000000000002},"min":{"1min":0.40999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":1.179,"5min":1.282,"15min":1.282},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.60299999999999998,"15min":0.61599999999999999},"min":{"1min":0.40699999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.86599999999999999,"5min":1.292,"15min":1.292},"last":0.48399999999999999}]},{"osd":7,"last update":"Thu Dec 7 10:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78600000000000003,"15min":0.76900000000000002},"min":{"1min":0.59599999999999997,"5min":0.498,"15min":0.498},"max":{"1min":1.0780000000000001,"5min":1.323,"15min":1.323},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.56799999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96999999999999997,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.73999999999999999}]},{"osd":8,"last update":"Thu Dec 7 10:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.872}]},{"osd":9,"last update":"Thu Dec 7 10:18:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.88,"15min":0.88600000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.224,"5min":1.242,"15min":1.242},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.80100000000000005,"15min":0.81000000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.77500000000000002}]},{"osd":10,"last update":"Thu Dec 7 10:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.78200000000000003,"15min":0.79100000000000004},"min":{"1min":0.59099999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.90500000000000003,"5min":1.335,"15min":1.335},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.878,"5min":0.92100000000000004,"15min":0.92800000000000005},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.1180000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.86799999999999999}]},{"osd":11,"last update":"Thu Dec 7 10:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.81299999999999994}]}]},{"osd":4,"up_from":34,"seq":146028888127,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":516802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.52100000000000002,"15min":0.49099999999999999},"min":{"1min":0.41499999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76500000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Dec 7 10:18:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.41099999999999998,"15min":0.39000000000000001},"min":{"1min":0.254,"5min":0.215,"15min":0.215},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.41499999999999998,"15min":0.38900000000000001},"min":{"1min":0.308,"5min":0.22,"15min":0.22},"max":{"1min":0.55200000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.61399999999999999}]},{"osd":2,"last update":"Thu Dec 7 10:18:17 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58899999999999997,"15min":0.56000000000000005},"min":{"1min":0.42699999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.875,"5min":1.323,"15min":1.323},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58499999999999996,"15min":0.54300000000000004},"min":{"1min":0.45700000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84299999999999997,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.68600000000000005}]},{"osd":3,"last update":"Thu Dec 7 10:18:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58099999999999996,"15min":0.54300000000000004},"min":{"1min":0.437,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.874,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.873},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.60899999999999999,"15min":0.56999999999999995},"min":{"1min":0.45700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91200000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.82599999999999996}]},{"osd":5,"last update":"Thu Dec 7 10:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.63100000000000001,"15min":0.59299999999999997},"min":{"1min":0.55000000000000004,"5min":0.374,"15min":0.374},"max":{"1min":1.0009999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.61699999999999999,"15min":0.58399999999999996},"min":{"1min":0.47599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82699999999999996,"5min":1.258,"15min":1.258},"last":0.72999999999999998}]},{"osd":6,"last update":"Thu Dec 7 10:19:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65300000000000002,"15min":0.65100000000000002},"min":{"1min":0.434,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90000000000000002,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65200000000000002,"15min":0.63800000000000001},"min":{"1min":0.49399999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85199999999999998,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.84299999999999997}]},{"osd":7,"last update":"Thu Dec 7 10:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.55000000000000004,"15min":0.51800000000000002},"min":{"1min":0.38100000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76800000000000002,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.35599999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.84399999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.96099999999999997}]},{"osd":8,"last update":"Thu Dec 7 10:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.71299999999999997,"15min":0.67900000000000005},"min":{"1min":0.57099999999999995,"5min":0.432,"15min":0.432},"max":{"1min":0.90000000000000002,"5min":1.381,"15min":1.381},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71199999999999997,"15min":0.68200000000000005},"min":{"1min":0.59099999999999997,"5min":0.502,"15min":0.502},"max":{"1min":0.872,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.91900000000000004}]},{"osd":9,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.76700000000000002,"15min":0.76900000000000002},"min":{"1min":0.57399999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.98799999999999999,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.77100000000000002,"15min":0.75800000000000001},"min":{"1min":0.58699999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.0289999999999999}]},{"osd":10,"last update":"Thu Dec 7 10:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.60999999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.54600000000000004,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.97499999999999998}]},{"osd":11,"last update":"Thu Dec 7 10:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":1.0129999999999999}]}]},{"osd":5,"up_from":39,"seq":167503724602,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28676,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716476,"statfs":{"total":95995035648,"available":95965671424,"internally_reserved":0,"allocated":1691648,"data_stored":1119172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.54700000000000004,"15min":0.51700000000000002},"min":{"1min":0.377,"5min":0.318,"15min":0.318},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54700000000000004,"15min":0.52200000000000002},"min":{"1min":0.37,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79000000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.52100000000000002}]},{"osd":1,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.56499999999999995,"15min":0.51700000000000002},"min":{"1min":0.505,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.59799999999999998,"15min":0.54400000000000004},"min":{"1min":0.52200000000000002,"5min":0.312,"15min":0.312},"max":{"1min":3.0070000000000001,"5min":3.0070000000000001,"15min":3.0070000000000001},"last":0.68200000000000005}]},{"osd":2,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.42399999999999999,"15min":0.38600000000000001},"min":{"1min":0.40200000000000002,"5min":0.246,"15min":0.246},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.45300000000000001,"15min":0.39900000000000002},"min":{"1min":0.35699999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.851,"5min":1.851,"15min":1.851},"last":0.36799999999999999}]},{"osd":3,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.57999999999999996,"15min":0.53900000000000003},"min":{"1min":0.43099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.59099999999999997,"15min":0.55600000000000005},"min":{"1min":0.40500000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.48299999999999998}]},{"osd":4,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.63100000000000001,"15min":0.57299999999999995},"min":{"1min":0.58199999999999996,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.621,"15min":0.57899999999999996},"min":{"1min":0.47699999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.749}]},{"osd":6,"last update":"Thu Dec 7 10:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63800000000000001,"15min":0.59299999999999997},"min":{"1min":0.46200000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86599999999999999,"5min":1.272,"15min":1.272},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.64000000000000001},"min":{"1min":0.56200000000000006,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96099999999999997,"5min":5.9720000000000004,"15min":5.9720000000000004},"last":0.63200000000000001}]},{"osd":7,"last update":"Thu Dec 7 10:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.66100000000000003,"15min":0.626},"min":{"1min":0.38400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.66600000000000004,"15min":0.63800000000000001},"min":{"1min":0.443,"5min":0.437,"15min":0.437},"max":{"1min":0.96999999999999997,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.62}]},{"osd":8,"last update":"Thu Dec 7 10:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.65000000000000002,"15min":0.628},"min":{"1min":0.45600000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.052,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.61899999999999999,"15min":0.59899999999999998},"min":{"1min":0.5,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92000000000000004,"5min":1.135,"15min":1.135},"last":0.60199999999999998}]},{"osd":9,"last update":"Thu Dec 7 10:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.78700000000000003,"15min":0.755},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":4.532,"5min":4.532,"15min":4.532},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76400000000000001,"15min":0.76300000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.69799999999999995}]},{"osd":10,"last update":"Thu Dec 7 10:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.84799999999999998,"15min":0.84099999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.7649999999999999,"5min":2.927,"15min":2.927},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.1519999999999999,"5min":0.85899999999999999,"15min":0.81000000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":5.7709999999999999,"5min":5.7709999999999999,"15min":5.7709999999999999},"last":0.72499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":76695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":13202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":13912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-07T10:19:13.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:13 smithi119 ceph-mon[97908]: pgmap v329: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-12-07T10:19:13.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:13 smithi119 ceph-mon[97908]: from='client.25052 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:19:14.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:13 smithi159 ceph-mon[134726]: pgmap v329: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-12-07T10:19:14.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:13 smithi159 ceph-mon[134726]: from='client.25052 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:19:14.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:13 smithi153 ceph-mon[133625]: pgmap v329: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-12-07T10:19:14.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:13 smithi153 ceph-mon[133625]: from='client.25052 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:19:16.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:15 smithi119 ceph-mon[97908]: pgmap v330: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-07T10:19:16.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:15 smithi159 ceph-mon[134726]: pgmap v330: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-07T10:19:16.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:15 smithi153 ceph-mon[133625]: pgmap v330: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-07T10:19:16.743 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph pg dump --format=json 2023-12-07T10:19:17.539 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:19:18.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:17 smithi119 ceph-mon[97908]: pgmap v331: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-12-07T10:19:18.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:17 smithi159 ceph-mon[134726]: pgmap v331: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-12-07T10:19:18.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:17 smithi153 ceph-mon[133625]: pgmap v331: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-12-07T10:19:19.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.296+0000 7f75ac7af700 1 -- 172.21.15.119:0/3344545652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75a4107730 msgr2=0x7f75a4107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:19.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.296+0000 7f75ac7af700 1 --2- 172.21.15.119:0/3344545652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75a4107730 0x7f75a4107b10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009990 tx=0x7f75a0009ca0 comp rx=0 tx=0).stop 2023-12-07T10:19:19.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.296+0000 7f75ac7af700 1 -- 172.21.15.119:0/3344545652 shutdown_connections 2023-12-07T10:19:19.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.296+0000 7f75ac7af700 1 --2- 172.21.15.119:0/3344545652 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75a4102080 0x7f75a410e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:19.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.296+0000 7f75ac7af700 1 --2- 172.21.15.119:0/3344545652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75a4107730 0x7f75a4107b10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:19.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.296+0000 7f75ac7af700 1 --2- 172.21.15.119:0/3344545652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75a41016c0 0x7f75a4101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:19.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.296+0000 7f75ac7af700 1 -- 172.21.15.119:0/3344545652 >> 172.21.15.119:0/3344545652 conn(0x7f75a4078800 msgr2=0x7f75a4078c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:19.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.296+0000 7f75ac7af700 1 -- 172.21.15.119:0/3344545652 shutdown_connections 2023-12-07T10:19:19.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.296+0000 7f75ac7af700 1 -- 172.21.15.119:0/3344545652 wait complete. 2023-12-07T10:19:19.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.297+0000 7f75ac7af700 1 Processor -- start 2023-12-07T10:19:19.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.298+0000 7f75ac7af700 1 -- start start 2023-12-07T10:19:19.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.298+0000 7f75ac7af700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75a41016c0 0x7f75a419f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:19.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.298+0000 7f75ac7af700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75a4102080 0x7f75a419fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:19.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.298+0000 7f75ac7af700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75a4107730 0x7f75a4199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:19.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.298+0000 7f75ac7af700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a419a060 con 0x7f75a4102080 2023-12-07T10:19:19.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.298+0000 7f75ac7af700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a419a1d0 con 0x7f75a4107730 2023-12-07T10:19:19.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.298+0000 7f75ac7af700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a419a390 con 0x7f75a41016c0 2023-12-07T10:19:19.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.298+0000 7f75a9d4a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75a4102080 0x7f75a419fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:19.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.298+0000 7f75a9d4a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75a4102080 0x7f75a419fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60908/0 (socket says 172.21.15.119:60908) 2023-12-07T10:19:19.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.298+0000 7f75a9d4a700 1 -- 172.21.15.119:0/3621130365 learned_addr learned my addr 172.21.15.119:0/3621130365 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:19.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.298+0000 7f75aa54b700 1 --2- 172.21.15.119:0/3621130365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75a41016c0 0x7f75a419f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:19.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.298+0000 7f75aad4c700 1 --2- 172.21.15.119:0/3621130365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75a4107730 0x7f75a4199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:19.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.299+0000 7f75a9d4a700 1 -- 172.21.15.119:0/3621130365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75a41016c0 msgr2=0x7f75a419f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:19.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.299+0000 7f75a9d4a700 1 --2- 172.21.15.119:0/3621130365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75a41016c0 0x7f75a419f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:19.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.299+0000 7f75a9d4a700 1 -- 172.21.15.119:0/3621130365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75a4107730 msgr2=0x7f75a4199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:19.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.299+0000 7f75a9d4a700 1 --2- 172.21.15.119:0/3621130365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75a4107730 0x7f75a4199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:19.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.299+0000 7f75a9d4a700 1 -- 172.21.15.119:0/3621130365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a00095f0 con 0x7f75a4102080 2023-12-07T10:19:19.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.299+0000 7f75a9d4a700 1 --2- 172.21.15.119:0/3621130365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75a4102080 0x7f75a419fe00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f759400e9a0 tx=0x7f759400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:19.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.299+0000 7f759b7fe700 1 -- 172.21.15.119:0/3621130365 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7594009790 con 0x7f75a4102080 2023-12-07T10:19:19.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.299+0000 7f759b7fe700 1 -- 172.21.15.119:0/3621130365 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7594010430 con 0x7f75a4102080 2023-12-07T10:19:19.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.299+0000 7f75ac7af700 1 -- 172.21.15.119:0/3621130365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a419a670 con 0x7f75a4102080 2023-12-07T10:19:19.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.299+0000 7f759b7fe700 1 -- 172.21.15.119:0/3621130365 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7594018680 con 0x7f75a4102080 2023-12-07T10:19:19.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.299+0000 7f75ac7af700 1 -- 172.21.15.119:0/3621130365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a41a68b0 con 0x7f75a4102080 2023-12-07T10:19:19.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.300+0000 7f75ac7af700 1 -- 172.21.15.119:0/3621130365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75a4104980 con 0x7f75a4102080 2023-12-07T10:19:19.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.302+0000 7f759b7fe700 1 -- 172.21.15.119:0/3621130365 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f75940105a0 con 0x7f75a4102080 2023-12-07T10:19:19.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.303+0000 7f759b7fe700 1 --2- 172.21.15.119:0/3621130365 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f75900a1c20 0x7f75900a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:19.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.304+0000 7f759b7fe700 1 -- 172.21.15.119:0/3621130365 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f7594014070 con 0x7f75a4102080 2023-12-07T10:19:19.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.304+0000 7f75aa54b700 1 --2- 172.21.15.119:0/3621130365 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f75900a1c20 0x7f75900a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:19.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.304+0000 7f75aa54b700 1 --2- 172.21.15.119:0/3621130365 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f75900a1c20 0x7f75900a40e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f75a0006010 tx=0x7f75a000b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:19.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.308+0000 7f759b7fe700 1 -- 172.21.15.119:0/3621130365 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7594096b40 con 0x7f75a4102080 2023-12-07T10:19:19.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.501+0000 7f75ac7af700 1 -- 172.21.15.119:0/3621130365 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f75a419af30 con 0x7f75900a1c20 2023-12-07T10:19:19.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.515+0000 7f759b7fe700 1 -- 172.21.15.119:0/3621130365 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426719 (secure 0 0 0) 0x7f75a419af30 con 0x7f75900a1c20 2023-12-07T10:19:19.523 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:19:19.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.520+0000 7f75ac7af700 1 -- 172.21.15.119:0/3621130365 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f75900a1c20 msgr2=0x7f75900a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:19.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.520+0000 7f75ac7af700 1 --2- 172.21.15.119:0/3621130365 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f75900a1c20 0x7f75900a40e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f75a0006010 tx=0x7f75a000b260 comp rx=0 tx=0).stop 2023-12-07T10:19:19.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.520+0000 7f75ac7af700 1 -- 172.21.15.119:0/3621130365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75a4102080 msgr2=0x7f75a419fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:19.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.520+0000 7f75ac7af700 1 --2- 172.21.15.119:0/3621130365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75a4102080 0x7f75a419fe00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f759400e9a0 tx=0x7f759400ecb0 comp rx=0 tx=0).stop 2023-12-07T10:19:19.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.520+0000 7f75ac7af700 1 -- 172.21.15.119:0/3621130365 shutdown_connections 2023-12-07T10:19:19.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.520+0000 7f75ac7af700 1 --2- 172.21.15.119:0/3621130365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75a4107730 0x7f75a4199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:19.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.520+0000 7f75ac7af700 1 --2- 172.21.15.119:0/3621130365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75a41016c0 0x7f75a419f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:19.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.520+0000 7f75ac7af700 1 --2- 172.21.15.119:0/3621130365 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f75900a1c20 0x7f75900a40e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:19.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.520+0000 7f75ac7af700 1 --2- 172.21.15.119:0/3621130365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75a4102080 0x7f75a419fe00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:19.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.520+0000 7f75ac7af700 1 -- 172.21.15.119:0/3621130365 >> 172.21.15.119:0/3621130365 conn(0x7f75a4078800 msgr2=0x7f75a410cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:19.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.520+0000 7f75ac7af700 1 -- 172.21.15.119:0/3621130365 shutdown_connections 2023-12-07T10:19:19.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:19.520+0000 7f75ac7af700 1 -- 172.21.15.119:0/3621130365 wait complete. 2023-12-07T10:19:19.528 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-12-07T10:19:20.033 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2023-12-07T10:19:18.694835+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":672259,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":813,"num_read_kb":545,"num_write":426,"num_write_kb":3092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338804,"kb_used_data":14540,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603020,"statfs":{"total":1151940427776,"available":1151593492480,"internally_reserved":0,"allocated":14888960,"data_stored":8185886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":331855161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003631"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.899237+0000","last_change":"2023-12-07T10:17:41.614206+0000","last_active":"2023-12-07T10:19:01.899237+0000","last_peered":"2023-12-07T10:19:01.899237+0000","last_clean":"2023-12-07T10:19:01.899237+0000","last_became_active":"2023-12-07T10:17:41.613553+0000","last_became_peered":"2023-12-07T10:17:41.613553+0000","last_unstale":"2023-12-07T10:19:01.899237+0000","last_undegraded":"2023-12-07T10:19:01.899237+0000","last_fullsized":"2023-12-07T10:19:01.899237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:10:22.198414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803801+0000","last_change":"2023-12-07T10:17:41.616917+0000","last_active":"2023-12-07T10:19:01.803801+0000","last_peered":"2023-12-07T10:19:01.803801+0000","last_clean":"2023-12-07T10:19:01.803801+0000","last_became_active":"2023-12-07T10:17:41.615345+0000","last_became_peered":"2023-12-07T10:17:41.615345+0000","last_unstale":"2023-12-07T10:19:01.803801+0000","last_undegraded":"2023-12-07T10:19:01.803801+0000","last_fullsized":"2023-12-07T10:19:01.803801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:22:25.566738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803844+0000","last_change":"2023-12-07T10:17:41.619767+0000","last_active":"2023-12-07T10:19:01.803844+0000","last_peered":"2023-12-07T10:19:01.803844+0000","last_clean":"2023-12-07T10:19:01.803844+0000","last_became_active":"2023-12-07T10:17:41.619362+0000","last_became_peered":"2023-12-07T10:17:41.619362+0000","last_unstale":"2023-12-07T10:19:01.803844+0000","last_undegraded":"2023-12-07T10:19:01.803844+0000","last_fullsized":"2023-12-07T10:19:01.803844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:12:55.458520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807439+0000","last_change":"2023-12-07T10:18:01.861930+0000","last_active":"2023-12-07T10:19:01.807439+0000","last_peered":"2023-12-07T10:19:01.807439+0000","last_clean":"2023-12-07T10:19:01.807439+0000","last_became_active":"2023-12-07T10:18:01.861570+0000","last_became_peered":"2023-12-07T10:18:01.861570+0000","last_unstale":"2023-12-07T10:19:01.807439+0000","last_undegraded":"2023-12-07T10:19:01.807439+0000","last_fullsized":"2023-12-07T10:19:01.807439+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:29:45.234261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808566+0000","last_change":"2023-12-07T10:17:41.613890+0000","last_active":"2023-12-07T10:19:01.808566+0000","last_peered":"2023-12-07T10:19:01.808566+0000","last_clean":"2023-12-07T10:19:01.808566+0000","last_became_active":"2023-12-07T10:17:41.613073+0000","last_became_peered":"2023-12-07T10:17:41.613073+0000","last_unstale":"2023-12-07T10:19:01.808566+0000","last_undegraded":"2023-12-07T10:19:01.808566+0000","last_fullsized":"2023-12-07T10:19:01.808566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:17:14.339002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802869+0000","last_change":"2023-12-07T10:17:41.609585+0000","last_active":"2023-12-07T10:19:01.802869+0000","last_peered":"2023-12-07T10:19:01.802869+0000","last_clean":"2023-12-07T10:19:01.802869+0000","last_became_active":"2023-12-07T10:17:41.609256+0000","last_became_peered":"2023-12-07T10:17:41.609256+0000","last_unstale":"2023-12-07T10:19:01.802869+0000","last_undegraded":"2023-12-07T10:19:01.802869+0000","last_fullsized":"2023-12-07T10:19:01.802869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:20:15.103208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811012+0000","last_change":"2023-12-07T10:17:41.606138+0000","last_active":"2023-12-07T10:19:01.811012+0000","last_peered":"2023-12-07T10:19:01.811012+0000","last_clean":"2023-12-07T10:19:01.811012+0000","last_became_active":"2023-12-07T10:17:41.605818+0000","last_became_peered":"2023-12-07T10:17:41.605818+0000","last_unstale":"2023-12-07T10:19:01.811012+0000","last_undegraded":"2023-12-07T10:19:01.811012+0000","last_fullsized":"2023-12-07T10:19:01.811012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:38:39.244077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.806921+0000","last_change":"2023-12-07T10:18:02.255959+0000","last_active":"2023-12-07T10:19:01.806921+0000","last_peered":"2023-12-07T10:19:01.806921+0000","last_clean":"2023-12-07T10:19:01.806921+0000","last_became_active":"2023-12-07T10:18:02.255560+0000","last_became_peered":"2023-12-07T10:18:02.255560+0000","last_unstale":"2023-12-07T10:19:01.806921+0000","last_undegraded":"2023-12-07T10:19:01.806921+0000","last_fullsized":"2023-12-07T10:19:01.806921+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:05:49.065538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811362+0000","last_change":"2023-12-07T10:17:41.610225+0000","last_active":"2023-12-07T10:19:01.811362+0000","last_peered":"2023-12-07T10:19:01.811362+0000","last_clean":"2023-12-07T10:19:01.811362+0000","last_became_active":"2023-12-07T10:17:41.609875+0000","last_became_peered":"2023-12-07T10:17:41.609875+0000","last_unstale":"2023-12-07T10:19:01.811362+0000","last_undegraded":"2023-12-07T10:19:01.811362+0000","last_fullsized":"2023-12-07T10:19:01.811362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:00:43.136176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803348+0000","last_change":"2023-12-07T10:17:41.617335+0000","last_active":"2023-12-07T10:19:01.803348+0000","last_peered":"2023-12-07T10:19:01.803348+0000","last_clean":"2023-12-07T10:19:01.803348+0000","last_became_active":"2023-12-07T10:17:41.617005+0000","last_became_peered":"2023-12-07T10:17:41.617005+0000","last_unstale":"2023-12-07T10:19:01.803348+0000","last_undegraded":"2023-12-07T10:19:01.803348+0000","last_fullsized":"2023-12-07T10:19:01.803348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:43:36.855465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811670+0000","last_change":"2023-12-07T10:17:41.608104+0000","last_active":"2023-12-07T10:19:01.811670+0000","last_peered":"2023-12-07T10:19:01.811670+0000","last_clean":"2023-12-07T10:19:01.811670+0000","last_became_active":"2023-12-07T10:17:41.607769+0000","last_became_peered":"2023-12-07T10:17:41.607769+0000","last_unstale":"2023-12-07T10:19:01.811670+0000","last_undegraded":"2023-12-07T10:19:01.811670+0000","last_fullsized":"2023-12-07T10:19:01.811670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:47:04.853044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.805183+0000","last_change":"2023-12-07T10:17:41.611585+0000","last_active":"2023-12-07T10:19:01.805183+0000","last_peered":"2023-12-07T10:19:01.805183+0000","last_clean":"2023-12-07T10:19:01.805183+0000","last_became_active":"2023-12-07T10:17:41.611202+0000","last_became_peered":"2023-12-07T10:17:41.611202+0000","last_unstale":"2023-12-07T10:19:01.805183+0000","last_undegraded":"2023-12-07T10:19:01.805183+0000","last_fullsized":"2023-12-07T10:19:01.805183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:17:01.649424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813519+0000","last_change":"2023-12-07T10:17:41.611116+0000","last_active":"2023-12-07T10:19:01.813519+0000","last_peered":"2023-12-07T10:19:01.813519+0000","last_clean":"2023-12-07T10:19:01.813519+0000","last_became_active":"2023-12-07T10:17:41.610791+0000","last_became_peered":"2023-12-07T10:17:41.610791+0000","last_unstale":"2023-12-07T10:19:01.813519+0000","last_undegraded":"2023-12-07T10:19:01.813519+0000","last_fullsized":"2023-12-07T10:19:01.813519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:48:08.822235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.804022+0000","last_change":"2023-12-07T10:17:41.617447+0000","last_active":"2023-12-07T10:19:01.804022+0000","last_peered":"2023-12-07T10:19:01.804022+0000","last_clean":"2023-12-07T10:19:01.804022+0000","last_became_active":"2023-12-07T10:17:41.612638+0000","last_became_peered":"2023-12-07T10:17:41.612638+0000","last_unstale":"2023-12-07T10:19:01.804022+0000","last_undegraded":"2023-12-07T10:19:01.804022+0000","last_fullsized":"2023-12-07T10:19:01.804022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:11:30.384367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803409+0000","last_change":"2023-12-07T10:17:41.611607+0000","last_active":"2023-12-07T10:19:01.803409+0000","last_peered":"2023-12-07T10:19:01.803409+0000","last_clean":"2023-12-07T10:19:01.803409+0000","last_became_active":"2023-12-07T10:17:41.611331+0000","last_became_peered":"2023-12-07T10:17:41.611331+0000","last_unstale":"2023-12-07T10:19:01.803409+0000","last_undegraded":"2023-12-07T10:19:01.803409+0000","last_fullsized":"2023-12-07T10:19:01.803409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:17:53.049922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810651+0000","last_change":"2023-12-07T10:17:41.610528+0000","last_active":"2023-12-07T10:19:01.810651+0000","last_peered":"2023-12-07T10:19:01.810651+0000","last_clean":"2023-12-07T10:19:01.810651+0000","last_became_active":"2023-12-07T10:17:41.610216+0000","last_became_peered":"2023-12-07T10:17:41.610216+0000","last_unstale":"2023-12-07T10:19:01.810651+0000","last_undegraded":"2023-12-07T10:19:01.810651+0000","last_fullsized":"2023-12-07T10:19:01.810651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:11:36.749537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.899192+0000","last_change":"2023-12-07T10:17:41.613967+0000","last_active":"2023-12-07T10:19:01.899192+0000","last_peered":"2023-12-07T10:19:01.899192+0000","last_clean":"2023-12-07T10:19:01.899192+0000","last_became_active":"2023-12-07T10:17:41.613116+0000","last_became_peered":"2023-12-07T10:17:41.613116+0000","last_unstale":"2023-12-07T10:19:01.899192+0000","last_undegraded":"2023-12-07T10:19:01.899192+0000","last_fullsized":"2023-12-07T10:19:01.899192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:46:32.444025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807559+0000","last_change":"2023-12-07T10:17:41.605478+0000","last_active":"2023-12-07T10:19:01.807559+0000","last_peered":"2023-12-07T10:19:01.807559+0000","last_clean":"2023-12-07T10:19:01.807559+0000","last_became_active":"2023-12-07T10:17:41.605110+0000","last_became_peered":"2023-12-07T10:17:41.605110+0000","last_unstale":"2023-12-07T10:19:01.807559+0000","last_undegraded":"2023-12-07T10:19:01.807559+0000","last_fullsized":"2023-12-07T10:19:01.807559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:53:56.410196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810309+0000","last_change":"2023-12-07T10:18:02.256377+0000","last_active":"2023-12-07T10:19:01.810309+0000","last_peered":"2023-12-07T10:19:01.810309+0000","last_clean":"2023-12-07T10:19:01.810309+0000","last_became_active":"2023-12-07T10:18:02.255987+0000","last_became_peered":"2023-12-07T10:18:02.255987+0000","last_unstale":"2023-12-07T10:19:01.810309+0000","last_undegraded":"2023-12-07T10:19:01.810309+0000","last_fullsized":"2023-12-07T10:19:01.810309+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:23:34.909187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.809857+0000","last_change":"2023-12-07T10:17:45.799925+0000","last_active":"2023-12-07T10:19:01.809857+0000","last_peered":"2023-12-07T10:19:01.809857+0000","last_clean":"2023-12-07T10:19:01.809857+0000","last_became_active":"2023-12-07T10:17:45.799505+0000","last_became_peered":"2023-12-07T10:17:45.799505+0000","last_unstale":"2023-12-07T10:19:01.809857+0000","last_undegraded":"2023-12-07T10:19:01.809857+0000","last_fullsized":"2023-12-07T10:19:01.809857+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:52:51.470333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'13","reported_seq":96,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:12.852006+0000","last_change":"2023-12-07T10:17:40.755066+0000","last_active":"2023-12-07T10:19:12.852006+0000","last_peered":"2023-12-07T10:19:12.852006+0000","last_clean":"2023-12-07T10:19:12.852006+0000","last_became_active":"2023-12-07T10:17:40.754709+0000","last_became_peered":"2023-12-07T10:17:40.754709+0000","last_unstale":"2023-12-07T10:19:12.852006+0000","last_undegraded":"2023-12-07T10:19:12.852006+0000","last_fullsized":"2023-12-07T10:19:12.852006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:18:31.148592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":62,"num_read_kb":62,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803641+0000","last_change":"2023-12-07T10:17:41.619107+0000","last_active":"2023-12-07T10:19:01.803641+0000","last_peered":"2023-12-07T10:19:01.803641+0000","last_clean":"2023-12-07T10:19:01.803641+0000","last_became_active":"2023-12-07T10:17:41.617057+0000","last_became_peered":"2023-12-07T10:17:41.617057+0000","last_unstale":"2023-12-07T10:19:01.803641+0000","last_undegraded":"2023-12-07T10:19:01.803641+0000","last_fullsized":"2023-12-07T10:19:01.803641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:53:16.402933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.804762+0000","last_change":"2023-12-07T10:17:45.797582+0000","last_active":"2023-12-07T10:19:01.804762+0000","last_peered":"2023-12-07T10:19:01.804762+0000","last_clean":"2023-12-07T10:19:01.804762+0000","last_became_active":"2023-12-07T10:17:45.796358+0000","last_became_peered":"2023-12-07T10:17:45.796358+0000","last_unstale":"2023-12-07T10:19:01.804762+0000","last_undegraded":"2023-12-07T10:19:01.804762+0000","last_fullsized":"2023-12-07T10:19:01.804762+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:53:19.551459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803712+0000","last_change":"2023-12-07T10:17:40.762155+0000","last_active":"2023-12-07T10:19:01.803712+0000","last_peered":"2023-12-07T10:19:01.803712+0000","last_clean":"2023-12-07T10:19:01.803712+0000","last_became_active":"2023-12-07T10:17:40.761776+0000","last_became_peered":"2023-12-07T10:17:40.761776+0000","last_unstale":"2023-12-07T10:19:01.803712+0000","last_undegraded":"2023-12-07T10:19:01.803712+0000","last_fullsized":"2023-12-07T10:19:01.803712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:15:06.096331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.805463+0000","last_change":"2023-12-07T10:17:41.614999+0000","last_active":"2023-12-07T10:19:01.805463+0000","last_peered":"2023-12-07T10:19:01.805463+0000","last_clean":"2023-12-07T10:19:01.805463+0000","last_became_active":"2023-12-07T10:17:41.614673+0000","last_became_peered":"2023-12-07T10:17:41.614673+0000","last_unstale":"2023-12-07T10:19:01.805463+0000","last_undegraded":"2023-12-07T10:19:01.805463+0000","last_fullsized":"2023-12-07T10:19:01.805463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:34:45.378843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.215249+0000","last_change":"2023-12-07T10:19:03.215249+0000","last_active":"2023-12-07T10:19:03.215249+0000","last_peered":"2023-12-07T10:19:03.215249+0000","last_clean":"2023-12-07T10:19:03.215249+0000","last_became_active":"2023-12-07T10:19:03.214647+0000","last_became_peered":"2023-12-07T10:19:03.214647+0000","last_unstale":"2023-12-07T10:19:03.215249+0000","last_undegraded":"2023-12-07T10:19:03.215249+0000","last_fullsized":"2023-12-07T10:19:03.215249+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:41:01.893661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808809+0000","last_change":"2023-12-07T10:18:01.993128+0000","last_active":"2023-12-07T10:19:01.808809+0000","last_peered":"2023-12-07T10:19:01.808809+0000","last_clean":"2023-12-07T10:19:01.808809+0000","last_became_active":"2023-12-07T10:18:01.990244+0000","last_became_peered":"2023-12-07T10:18:01.990244+0000","last_unstale":"2023-12-07T10:19:01.808809+0000","last_undegraded":"2023-12-07T10:19:01.808809+0000","last_fullsized":"2023-12-07T10:19:01.808809+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:42:53.778363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810926+0000","last_change":"2023-12-07T10:17:41.612903+0000","last_active":"2023-12-07T10:19:01.810926+0000","last_peered":"2023-12-07T10:19:01.810926+0000","last_clean":"2023-12-07T10:19:01.810926+0000","last_became_active":"2023-12-07T10:17:41.611968+0000","last_became_peered":"2023-12-07T10:17:41.611968+0000","last_unstale":"2023-12-07T10:19:01.810926+0000","last_undegraded":"2023-12-07T10:19:01.810926+0000","last_fullsized":"2023-12-07T10:19:01.810926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:34:08.442828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.804826+0000","last_change":"2023-12-07T10:17:45.803858+0000","last_active":"2023-12-07T10:19:01.804826+0000","last_peered":"2023-12-07T10:19:01.804826+0000","last_clean":"2023-12-07T10:19:01.804826+0000","last_became_active":"2023-12-07T10:17:45.803481+0000","last_became_peered":"2023-12-07T10:17:45.803481+0000","last_unstale":"2023-12-07T10:19:01.804826+0000","last_undegraded":"2023-12-07T10:19:01.804826+0000","last_fullsized":"2023-12-07T10:19:01.804826+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:08:03.728554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808719+0000","last_change":"2023-12-07T10:17:40.751866+0000","last_active":"2023-12-07T10:19:01.808719+0000","last_peered":"2023-12-07T10:19:01.808719+0000","last_clean":"2023-12-07T10:19:01.808719+0000","last_became_active":"2023-12-07T10:17:40.751492+0000","last_became_peered":"2023-12-07T10:17:40.751492+0000","last_unstale":"2023-12-07T10:19:01.808719+0000","last_undegraded":"2023-12-07T10:19:01.808719+0000","last_fullsized":"2023-12-07T10:19:01.808719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:42:53.839651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808255+0000","last_change":"2023-12-07T10:17:41.611326+0000","last_active":"2023-12-07T10:19:01.808255+0000","last_peered":"2023-12-07T10:19:01.808255+0000","last_clean":"2023-12-07T10:19:01.808255+0000","last_became_active":"2023-12-07T10:17:41.610863+0000","last_became_peered":"2023-12-07T10:17:41.610863+0000","last_unstale":"2023-12-07T10:19:01.808255+0000","last_undegraded":"2023-12-07T10:19:01.808255+0000","last_fullsized":"2023-12-07T10:19:01.808255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:07:55.801356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810238+0000","last_change":"2023-12-07T10:17:45.791409+0000","last_active":"2023-12-07T10:19:01.810238+0000","last_peered":"2023-12-07T10:19:01.810238+0000","last_clean":"2023-12-07T10:19:01.810238+0000","last_became_active":"2023-12-07T10:17:45.791022+0000","last_became_peered":"2023-12-07T10:17:45.791022+0000","last_unstale":"2023-12-07T10:19:01.810238+0000","last_undegraded":"2023-12-07T10:19:01.810238+0000","last_fullsized":"2023-12-07T10:19:01.810238+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:57:14.617409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'9","reported_seq":103,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:16.838158+0000","last_change":"2023-12-07T10:17:40.745318+0000","last_active":"2023-12-07T10:19:16.838158+0000","last_peered":"2023-12-07T10:19:16.838158+0000","last_clean":"2023-12-07T10:19:16.838158+0000","last_became_active":"2023-12-07T10:17:40.745008+0000","last_became_peered":"2023-12-07T10:17:40.745008+0000","last_unstale":"2023-12-07T10:19:16.838158+0000","last_undegraded":"2023-12-07T10:19:16.838158+0000","last_fullsized":"2023-12-07T10:19:16.838158+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:42:28.503421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11004,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":14,"num_write":6,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808088+0000","last_change":"2023-12-07T10:17:41.605121+0000","last_active":"2023-12-07T10:19:01.808088+0000","last_peered":"2023-12-07T10:19:01.808088+0000","last_clean":"2023-12-07T10:19:01.808088+0000","last_became_active":"2023-12-07T10:17:41.604770+0000","last_became_peered":"2023-12-07T10:17:41.604770+0000","last_unstale":"2023-12-07T10:19:01.808088+0000","last_undegraded":"2023-12-07T10:19:01.808088+0000","last_fullsized":"2023-12-07T10:19:01.808088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:51:02.151155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810286+0000","last_change":"2023-12-07T10:17:45.809039+0000","last_active":"2023-12-07T10:19:01.810286+0000","last_peered":"2023-12-07T10:19:01.810286+0000","last_clean":"2023-12-07T10:19:01.810286+0000","last_became_active":"2023-12-07T10:17:45.808622+0000","last_became_peered":"2023-12-07T10:17:45.808622+0000","last_unstale":"2023-12-07T10:19:01.810286+0000","last_undegraded":"2023-12-07T10:19:01.810286+0000","last_fullsized":"2023-12-07T10:19:01.810286+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:16:02.835931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802888+0000","last_change":"2023-12-07T10:17:40.757876+0000","last_active":"2023-12-07T10:19:01.802888+0000","last_peered":"2023-12-07T10:19:01.802888+0000","last_clean":"2023-12-07T10:19:01.802888+0000","last_became_active":"2023-12-07T10:17:40.757613+0000","last_became_peered":"2023-12-07T10:17:40.757613+0000","last_unstale":"2023-12-07T10:19:01.802888+0000","last_undegraded":"2023-12-07T10:19:01.802888+0000","last_fullsized":"2023-12-07T10:19:01.802888+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:30:40.423092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808115+0000","last_change":"2023-12-07T10:17:41.611575+0000","last_active":"2023-12-07T10:19:01.808115+0000","last_peered":"2023-12-07T10:19:01.808115+0000","last_clean":"2023-12-07T10:19:01.808115+0000","last_became_active":"2023-12-07T10:17:41.611042+0000","last_became_peered":"2023-12-07T10:17:41.611042+0000","last_unstale":"2023-12-07T10:19:01.808115+0000","last_undegraded":"2023-12-07T10:19:01.808115+0000","last_fullsized":"2023-12-07T10:19:01.808115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:54:05.589792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.899155+0000","last_change":"2023-12-07T10:17:45.796640+0000","last_active":"2023-12-07T10:19:01.899155+0000","last_peered":"2023-12-07T10:19:01.899155+0000","last_clean":"2023-12-07T10:19:01.899155+0000","last_became_active":"2023-12-07T10:17:45.796183+0000","last_became_peered":"2023-12-07T10:17:45.796183+0000","last_unstale":"2023-12-07T10:19:01.899155+0000","last_undegraded":"2023-12-07T10:19:01.899155+0000","last_fullsized":"2023-12-07T10:19:01.899155+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:41:56.890662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813362+0000","last_change":"2023-12-07T10:17:40.760283+0000","last_active":"2023-12-07T10:19:01.813362+0000","last_peered":"2023-12-07T10:19:01.813362+0000","last_clean":"2023-12-07T10:19:01.813362+0000","last_became_active":"2023-12-07T10:17:40.759398+0000","last_became_peered":"2023-12-07T10:17:40.759398+0000","last_unstale":"2023-12-07T10:19:01.813362+0000","last_undegraded":"2023-12-07T10:19:01.813362+0000","last_fullsized":"2023-12-07T10:19:01.813362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:34:37.365490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802564+0000","last_change":"2023-12-07T10:17:41.604150+0000","last_active":"2023-12-07T10:19:01.802564+0000","last_peered":"2023-12-07T10:19:01.802564+0000","last_clean":"2023-12-07T10:19:01.802564+0000","last_became_active":"2023-12-07T10:17:41.603688+0000","last_became_peered":"2023-12-07T10:17:41.603688+0000","last_unstale":"2023-12-07T10:19:01.802564+0000","last_undegraded":"2023-12-07T10:19:01.802564+0000","last_fullsized":"2023-12-07T10:19:01.802564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:52:43.735609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803710+0000","last_change":"2023-12-07T10:17:45.806569+0000","last_active":"2023-12-07T10:19:01.803710+0000","last_peered":"2023-12-07T10:19:01.803710+0000","last_clean":"2023-12-07T10:19:01.803710+0000","last_became_active":"2023-12-07T10:17:45.806099+0000","last_became_peered":"2023-12-07T10:17:45.806099+0000","last_unstale":"2023-12-07T10:19:01.803710+0000","last_undegraded":"2023-12-07T10:19:01.803710+0000","last_fullsized":"2023-12-07T10:19:01.803710+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:02:19.977670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807289+0000","last_change":"2023-12-07T10:17:40.744388+0000","last_active":"2023-12-07T10:19:01.807289+0000","last_peered":"2023-12-07T10:19:01.807289+0000","last_clean":"2023-12-07T10:19:01.807289+0000","last_became_active":"2023-12-07T10:17:40.743887+0000","last_became_peered":"2023-12-07T10:17:40.743887+0000","last_unstale":"2023-12-07T10:19:01.807289+0000","last_undegraded":"2023-12-07T10:19:01.807289+0000","last_fullsized":"2023-12-07T10:19:01.807289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:18:39.102099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807114+0000","last_change":"2023-12-07T10:17:41.595616+0000","last_active":"2023-12-07T10:19:01.807114+0000","last_peered":"2023-12-07T10:19:01.807114+0000","last_clean":"2023-12-07T10:19:01.807114+0000","last_became_active":"2023-12-07T10:17:41.595256+0000","last_became_peered":"2023-12-07T10:17:41.595256+0000","last_unstale":"2023-12-07T10:19:01.807114+0000","last_undegraded":"2023-12-07T10:19:01.807114+0000","last_fullsized":"2023-12-07T10:19:01.807114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:08:00.156043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802753+0000","last_change":"2023-12-07T10:17:45.794523+0000","last_active":"2023-12-07T10:19:01.802753+0000","last_peered":"2023-12-07T10:19:01.802753+0000","last_clean":"2023-12-07T10:19:01.802753+0000","last_became_active":"2023-12-07T10:17:45.794139+0000","last_became_peered":"2023-12-07T10:17:45.794139+0000","last_unstale":"2023-12-07T10:19:01.802753+0000","last_undegraded":"2023-12-07T10:19:01.802753+0000","last_fullsized":"2023-12-07T10:19:01.802753+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:43:24.767627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:14.834968+0000","last_change":"2023-12-07T10:17:40.759620+0000","last_active":"2023-12-07T10:19:14.834968+0000","last_peered":"2023-12-07T10:19:14.834968+0000","last_clean":"2023-12-07T10:19:14.834968+0000","last_became_active":"2023-12-07T10:17:40.759372+0000","last_became_peered":"2023-12-07T10:17:40.759372+0000","last_unstale":"2023-12-07T10:19:14.834968+0000","last_undegraded":"2023-12-07T10:19:14.834968+0000","last_fullsized":"2023-12-07T10:19:14.834968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:46:14.459409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":127,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811706+0000","last_change":"2023-12-07T10:17:41.601152+0000","last_active":"2023-12-07T10:19:01.811706+0000","last_peered":"2023-12-07T10:19:01.811706+0000","last_clean":"2023-12-07T10:19:01.811706+0000","last_became_active":"2023-12-07T10:17:41.600147+0000","last_became_peered":"2023-12-07T10:17:41.600147+0000","last_unstale":"2023-12-07T10:19:01.811706+0000","last_undegraded":"2023-12-07T10:19:01.811706+0000","last_fullsized":"2023-12-07T10:19:01.811706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:40:46.671489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807019+0000","last_change":"2023-12-07T10:17:45.796229+0000","last_active":"2023-12-07T10:19:01.807019+0000","last_peered":"2023-12-07T10:19:01.807019+0000","last_clean":"2023-12-07T10:19:01.807019+0000","last_became_active":"2023-12-07T10:17:45.795582+0000","last_became_peered":"2023-12-07T10:17:45.795582+0000","last_unstale":"2023-12-07T10:19:01.807019+0000","last_undegraded":"2023-12-07T10:19:01.807019+0000","last_fullsized":"2023-12-07T10:19:01.807019+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:29:32.853975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811078+0000","last_change":"2023-12-07T10:17:40.748353+0000","last_active":"2023-12-07T10:19:01.811078+0000","last_peered":"2023-12-07T10:19:01.811078+0000","last_clean":"2023-12-07T10:19:01.811078+0000","last_became_active":"2023-12-07T10:17:40.748050+0000","last_became_peered":"2023-12-07T10:17:40.748050+0000","last_unstale":"2023-12-07T10:19:01.811078+0000","last_undegraded":"2023-12-07T10:19:01.811078+0000","last_fullsized":"2023-12-07T10:19:01.811078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:18:15.298534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.805543+0000","last_change":"2023-12-07T10:18:02.256728+0000","last_active":"2023-12-07T10:19:01.805543+0000","last_peered":"2023-12-07T10:19:01.805543+0000","last_clean":"2023-12-07T10:19:01.805543+0000","last_became_active":"2023-12-07T10:18:02.256271+0000","last_became_peered":"2023-12-07T10:18:02.256271+0000","last_unstale":"2023-12-07T10:19:01.805543+0000","last_undegraded":"2023-12-07T10:19:01.805543+0000","last_fullsized":"2023-12-07T10:19:01.805543+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:27:00.198934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.805652+0000","last_change":"2023-12-07T10:17:45.796903+0000","last_active":"2023-12-07T10:19:01.805652+0000","last_peered":"2023-12-07T10:19:01.805652+0000","last_clean":"2023-12-07T10:19:01.805652+0000","last_became_active":"2023-12-07T10:17:45.796454+0000","last_became_peered":"2023-12-07T10:17:45.796454+0000","last_unstale":"2023-12-07T10:19:01.805652+0000","last_undegraded":"2023-12-07T10:19:01.805652+0000","last_fullsized":"2023-12-07T10:19:01.805652+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:46:36.955396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808245+0000","last_change":"2023-12-07T10:17:40.756045+0000","last_active":"2023-12-07T10:19:01.808245+0000","last_peered":"2023-12-07T10:19:01.808245+0000","last_clean":"2023-12-07T10:19:01.808245+0000","last_became_active":"2023-12-07T10:17:40.755763+0000","last_became_peered":"2023-12-07T10:17:40.755763+0000","last_unstale":"2023-12-07T10:19:01.808245+0000","last_undegraded":"2023-12-07T10:19:01.808245+0000","last_fullsized":"2023-12-07T10:19:01.808245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:01:08.009076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811097+0000","last_change":"2023-12-07T10:17:41.611178+0000","last_active":"2023-12-07T10:19:01.811097+0000","last_peered":"2023-12-07T10:19:01.811097+0000","last_clean":"2023-12-07T10:19:01.811097+0000","last_became_active":"2023-12-07T10:17:41.610750+0000","last_became_peered":"2023-12-07T10:17:41.610750+0000","last_unstale":"2023-12-07T10:19:01.811097+0000","last_undegraded":"2023-12-07T10:19:01.811097+0000","last_fullsized":"2023-12-07T10:19:01.811097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:56:30.763523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807197+0000","last_change":"2023-12-07T10:17:45.800183+0000","last_active":"2023-12-07T10:19:01.807197+0000","last_peered":"2023-12-07T10:19:01.807197+0000","last_clean":"2023-12-07T10:19:01.807197+0000","last_became_active":"2023-12-07T10:17:45.798598+0000","last_became_peered":"2023-12-07T10:17:45.798598+0000","last_unstale":"2023-12-07T10:19:01.807197+0000","last_undegraded":"2023-12-07T10:19:01.807197+0000","last_fullsized":"2023-12-07T10:19:01.807197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:06:22.162951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":192,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:14.835101+0000","last_change":"2023-12-07T10:17:40.752481+0000","last_active":"2023-12-07T10:19:14.835101+0000","last_peered":"2023-12-07T10:19:14.835101+0000","last_clean":"2023-12-07T10:19:14.835101+0000","last_became_active":"2023-12-07T10:17:40.752158+0000","last_became_peered":"2023-12-07T10:17:40.752158+0000","last_unstale":"2023-12-07T10:19:14.835101+0000","last_undegraded":"2023-12-07T10:19:14.835101+0000","last_fullsized":"2023-12-07T10:19:14.835101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:43:27.963014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64453,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":137,"num_read_kb":131,"num_write":36,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808644+0000","last_change":"2023-12-07T10:17:41.614107+0000","last_active":"2023-12-07T10:19:01.808644+0000","last_peered":"2023-12-07T10:19:01.808644+0000","last_clean":"2023-12-07T10:19:01.808644+0000","last_became_active":"2023-12-07T10:17:41.613743+0000","last_became_peered":"2023-12-07T10:17:41.613743+0000","last_unstale":"2023-12-07T10:19:01.808644+0000","last_undegraded":"2023-12-07T10:19:01.808644+0000","last_fullsized":"2023-12-07T10:19:01.808644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:49:03.200387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808863+0000","last_change":"2023-12-07T10:17:45.804282+0000","last_active":"2023-12-07T10:19:01.808863+0000","last_peered":"2023-12-07T10:19:01.808863+0000","last_clean":"2023-12-07T10:19:01.808863+0000","last_became_active":"2023-12-07T10:17:45.803794+0000","last_became_peered":"2023-12-07T10:17:45.803794+0000","last_unstale":"2023-12-07T10:19:01.808863+0000","last_undegraded":"2023-12-07T10:19:01.808863+0000","last_fullsized":"2023-12-07T10:19:01.808863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:31:26.887565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810171+0000","last_change":"2023-12-07T10:17:40.747204+0000","last_active":"2023-12-07T10:19:01.810171+0000","last_peered":"2023-12-07T10:19:01.810171+0000","last_clean":"2023-12-07T10:19:01.810171+0000","last_became_active":"2023-12-07T10:17:40.746916+0000","last_became_peered":"2023-12-07T10:17:40.746916+0000","last_unstale":"2023-12-07T10:19:01.810171+0000","last_undegraded":"2023-12-07T10:19:01.810171+0000","last_fullsized":"2023-12-07T10:19:01.810171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:53:49.228320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.814021+0000","last_change":"2023-12-07T10:17:41.606895+0000","last_active":"2023-12-07T10:19:01.814021+0000","last_peered":"2023-12-07T10:19:01.814021+0000","last_clean":"2023-12-07T10:19:01.814021+0000","last_became_active":"2023-12-07T10:17:41.606596+0000","last_became_peered":"2023-12-07T10:17:41.606596+0000","last_unstale":"2023-12-07T10:19:01.814021+0000","last_undegraded":"2023-12-07T10:19:01.814021+0000","last_fullsized":"2023-12-07T10:19:01.814021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:34:25.166906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.898878+0000","last_change":"2023-12-07T10:17:45.807101+0000","last_active":"2023-12-07T10:19:01.898878+0000","last_peered":"2023-12-07T10:19:01.898878+0000","last_clean":"2023-12-07T10:19:01.898878+0000","last_became_active":"2023-12-07T10:17:45.806770+0000","last_became_peered":"2023-12-07T10:17:45.806770+0000","last_unstale":"2023-12-07T10:19:01.898878+0000","last_undegraded":"2023-12-07T10:19:01.898878+0000","last_fullsized":"2023-12-07T10:19:01.898878+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:39:32.522968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808371+0000","last_change":"2023-12-07T10:17:40.754856+0000","last_active":"2023-12-07T10:19:01.808371+0000","last_peered":"2023-12-07T10:19:01.808371+0000","last_clean":"2023-12-07T10:19:01.808371+0000","last_became_active":"2023-12-07T10:17:40.754488+0000","last_became_peered":"2023-12-07T10:17:40.754488+0000","last_unstale":"2023-12-07T10:19:01.808371+0000","last_undegraded":"2023-12-07T10:19:01.808371+0000","last_fullsized":"2023-12-07T10:19:01.808371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:44:50.960216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811734+0000","last_change":"2023-12-07T10:17:41.606963+0000","last_active":"2023-12-07T10:19:01.811734+0000","last_peered":"2023-12-07T10:19:01.811734+0000","last_clean":"2023-12-07T10:19:01.811734+0000","last_became_active":"2023-12-07T10:17:41.606408+0000","last_became_peered":"2023-12-07T10:17:41.606408+0000","last_unstale":"2023-12-07T10:19:01.811734+0000","last_undegraded":"2023-12-07T10:19:01.811734+0000","last_fullsized":"2023-12-07T10:19:01.811734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:25:47.906649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.384918+0000","last_change":"2023-12-07T10:19:03.384918+0000","last_active":"2023-12-07T10:19:03.384918+0000","last_peered":"2023-12-07T10:19:03.384918+0000","last_clean":"2023-12-07T10:19:03.384918+0000","last_became_active":"2023-12-07T10:19:03.384613+0000","last_became_peered":"2023-12-07T10:19:03.384613+0000","last_unstale":"2023-12-07T10:19:03.384918+0000","last_undegraded":"2023-12-07T10:19:03.384918+0000","last_fullsized":"2023-12-07T10:19:03.384918+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:07:29.385953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":92,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:14.855656+0000","last_change":"2023-12-07T10:17:40.747494+0000","last_active":"2023-12-07T10:19:14.855656+0000","last_peered":"2023-12-07T10:19:14.855656+0000","last_clean":"2023-12-07T10:19:14.855656+0000","last_became_active":"2023-12-07T10:17:40.747157+0000","last_became_peered":"2023-12-07T10:17:40.747157+0000","last_unstale":"2023-12-07T10:19:14.855656+0000","last_undegraded":"2023-12-07T10:19:14.855656+0000","last_fullsized":"2023-12-07T10:19:14.855656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:45:24.395418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":976,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813024+0000","last_change":"2023-12-07T10:17:41.609495+0000","last_active":"2023-12-07T10:19:01.813024+0000","last_peered":"2023-12-07T10:19:01.813024+0000","last_clean":"2023-12-07T10:19:01.813024+0000","last_became_active":"2023-12-07T10:17:41.609181+0000","last_became_peered":"2023-12-07T10:17:41.609181+0000","last_unstale":"2023-12-07T10:19:01.813024+0000","last_undegraded":"2023-12-07T10:19:01.813024+0000","last_fullsized":"2023-12-07T10:19:01.813024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:20:33.473824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813855+0000","last_change":"2023-12-07T10:17:45.808862+0000","last_active":"2023-12-07T10:19:01.813855+0000","last_peered":"2023-12-07T10:19:01.813855+0000","last_clean":"2023-12-07T10:19:01.813855+0000","last_became_active":"2023-12-07T10:17:45.808461+0000","last_became_peered":"2023-12-07T10:17:45.808461+0000","last_unstale":"2023-12-07T10:19:01.813855+0000","last_undegraded":"2023-12-07T10:19:01.813855+0000","last_fullsized":"2023-12-07T10:19:01.813855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:09:12.598126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807550+0000","last_change":"2023-12-07T10:18:02.263709+0000","last_active":"2023-12-07T10:19:01.807550+0000","last_peered":"2023-12-07T10:19:01.807550+0000","last_clean":"2023-12-07T10:19:01.807550+0000","last_became_active":"2023-12-07T10:18:02.255788+0000","last_became_peered":"2023-12-07T10:18:02.255788+0000","last_unstale":"2023-12-07T10:19:01.807550+0000","last_undegraded":"2023-12-07T10:19:01.807550+0000","last_fullsized":"2023-12-07T10:19:01.807550+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:12:04.976226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808356+0000","last_change":"2023-12-07T10:17:41.610509+0000","last_active":"2023-12-07T10:19:01.808356+0000","last_peered":"2023-12-07T10:19:01.808356+0000","last_clean":"2023-12-07T10:19:01.808356+0000","last_became_active":"2023-12-07T10:17:41.610121+0000","last_became_peered":"2023-12-07T10:17:41.610121+0000","last_unstale":"2023-12-07T10:19:01.808356+0000","last_undegraded":"2023-12-07T10:19:01.808356+0000","last_fullsized":"2023-12-07T10:19:01.808356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:53:17.365880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808256+0000","last_change":"2023-12-07T10:17:45.795818+0000","last_active":"2023-12-07T10:19:01.808256+0000","last_peered":"2023-12-07T10:19:01.808256+0000","last_clean":"2023-12-07T10:19:01.808256+0000","last_became_active":"2023-12-07T10:17:45.795477+0000","last_became_peered":"2023-12-07T10:17:45.795477+0000","last_unstale":"2023-12-07T10:19:01.808256+0000","last_undegraded":"2023-12-07T10:19:01.808256+0000","last_fullsized":"2023-12-07T10:19:01.808256+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:17:21.494654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.809420+0000","last_change":"2023-12-07T10:17:40.751791+0000","last_active":"2023-12-07T10:19:01.809420+0000","last_peered":"2023-12-07T10:19:01.809420+0000","last_clean":"2023-12-07T10:19:01.809420+0000","last_became_active":"2023-12-07T10:17:40.751502+0000","last_became_peered":"2023-12-07T10:17:40.751502+0000","last_unstale":"2023-12-07T10:19:01.809420+0000","last_undegraded":"2023-12-07T10:19:01.809420+0000","last_fullsized":"2023-12-07T10:19:01.809420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:15:52.319265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.899134+0000","last_change":"2023-12-07T10:18:02.256247+0000","last_active":"2023-12-07T10:19:01.899134+0000","last_peered":"2023-12-07T10:19:01.899134+0000","last_clean":"2023-12-07T10:19:01.899134+0000","last_became_active":"2023-12-07T10:18:02.255732+0000","last_became_peered":"2023-12-07T10:18:02.255732+0000","last_unstale":"2023-12-07T10:19:01.899134+0000","last_undegraded":"2023-12-07T10:19:01.899134+0000","last_fullsized":"2023-12-07T10:19:01.899134+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:00:06.516804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808004+0000","last_change":"2023-12-07T10:17:45.791966+0000","last_active":"2023-12-07T10:19:01.808004+0000","last_peered":"2023-12-07T10:19:01.808004+0000","last_clean":"2023-12-07T10:19:01.808004+0000","last_became_active":"2023-12-07T10:17:45.791613+0000","last_became_peered":"2023-12-07T10:17:45.791613+0000","last_unstale":"2023-12-07T10:19:01.808004+0000","last_undegraded":"2023-12-07T10:19:01.808004+0000","last_fullsized":"2023-12-07T10:19:01.808004+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:13:02.291289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807344+0000","last_change":"2023-12-07T10:17:40.762896+0000","last_active":"2023-12-07T10:19:01.807344+0000","last_peered":"2023-12-07T10:19:01.807344+0000","last_clean":"2023-12-07T10:19:01.807344+0000","last_became_active":"2023-12-07T10:17:40.762427+0000","last_became_peered":"2023-12-07T10:17:40.762427+0000","last_unstale":"2023-12-07T10:19:01.807344+0000","last_undegraded":"2023-12-07T10:19:01.807344+0000","last_fullsized":"2023-12-07T10:19:01.807344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:56:35.589548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803085+0000","last_change":"2023-12-07T10:17:41.610354+0000","last_active":"2023-12-07T10:19:01.803085+0000","last_peered":"2023-12-07T10:19:01.803085+0000","last_clean":"2023-12-07T10:19:01.803085+0000","last_became_active":"2023-12-07T10:17:41.609888+0000","last_became_peered":"2023-12-07T10:17:41.609888+0000","last_unstale":"2023-12-07T10:19:01.803085+0000","last_undegraded":"2023-12-07T10:19:01.803085+0000","last_fullsized":"2023-12-07T10:19:01.803085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:48:19.976949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813526+0000","last_change":"2023-12-07T10:17:45.807324+0000","last_active":"2023-12-07T10:19:01.813526+0000","last_peered":"2023-12-07T10:19:01.813526+0000","last_clean":"2023-12-07T10:19:01.813526+0000","last_became_active":"2023-12-07T10:17:45.806036+0000","last_became_peered":"2023-12-07T10:17:45.806036+0000","last_unstale":"2023-12-07T10:19:01.813526+0000","last_undegraded":"2023-12-07T10:19:01.813526+0000","last_fullsized":"2023-12-07T10:19:01.813526+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:54:36.847992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'5","reported_seq":89,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:13.853837+0000","last_change":"2023-12-07T10:17:40.754247+0000","last_active":"2023-12-07T10:19:13.853837+0000","last_peered":"2023-12-07T10:19:13.853837+0000","last_clean":"2023-12-07T10:19:13.853837+0000","last_became_active":"2023-12-07T10:17:40.753964+0000","last_became_peered":"2023-12-07T10:17:40.753964+0000","last_unstale":"2023-12-07T10:19:13.853837+0000","last_undegraded":"2023-12-07T10:19:13.853837+0000","last_fullsized":"2023-12-07T10:19:13.853837+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:22:04.832399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802944+0000","last_change":"2023-12-07T10:17:41.610516+0000","last_active":"2023-12-07T10:19:01.802944+0000","last_peered":"2023-12-07T10:19:01.802944+0000","last_clean":"2023-12-07T10:19:01.802944+0000","last_became_active":"2023-12-07T10:17:41.610184+0000","last_became_peered":"2023-12-07T10:17:41.610184+0000","last_unstale":"2023-12-07T10:19:01.802944+0000","last_undegraded":"2023-12-07T10:19:01.802944+0000","last_fullsized":"2023-12-07T10:19:01.802944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:53:43.100723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813880+0000","last_change":"2023-12-07T10:17:45.804624+0000","last_active":"2023-12-07T10:19:01.813880+0000","last_peered":"2023-12-07T10:19:01.813880+0000","last_clean":"2023-12-07T10:19:01.813880+0000","last_became_active":"2023-12-07T10:17:45.803667+0000","last_became_peered":"2023-12-07T10:17:45.803667+0000","last_unstale":"2023-12-07T10:19:01.813880+0000","last_undegraded":"2023-12-07T10:19:01.813880+0000","last_fullsized":"2023-12-07T10:19:01.813880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:55:09.642795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807386+0000","last_change":"2023-12-07T10:17:41.618433+0000","last_active":"2023-12-07T10:19:01.807386+0000","last_peered":"2023-12-07T10:19:01.807386+0000","last_clean":"2023-12-07T10:19:01.807386+0000","last_became_active":"2023-12-07T10:17:41.617942+0000","last_became_peered":"2023-12-07T10:17:41.617942+0000","last_unstale":"2023-12-07T10:19:01.807386+0000","last_undegraded":"2023-12-07T10:19:01.807386+0000","last_fullsized":"2023-12-07T10:19:01.807386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:26:48.166157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":92,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:16.858070+0000","last_change":"2023-12-07T10:17:40.761907+0000","last_active":"2023-12-07T10:19:16.858070+0000","last_peered":"2023-12-07T10:19:16.858070+0000","last_clean":"2023-12-07T10:19:16.858070+0000","last_became_active":"2023-12-07T10:17:40.761564+0000","last_became_peered":"2023-12-07T10:17:40.761564+0000","last_unstale":"2023-12-07T10:19:16.858070+0000","last_undegraded":"2023-12-07T10:19:16.858070+0000","last_fullsized":"2023-12-07T10:19:16.858070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:57:22.304352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811777+0000","last_change":"2023-12-07T10:17:45.792684+0000","last_active":"2023-12-07T10:19:01.811777+0000","last_peered":"2023-12-07T10:19:01.811777+0000","last_clean":"2023-12-07T10:19:01.811777+0000","last_became_active":"2023-12-07T10:17:45.792172+0000","last_became_peered":"2023-12-07T10:17:45.792172+0000","last_unstale":"2023-12-07T10:19:01.811777+0000","last_undegraded":"2023-12-07T10:19:01.811777+0000","last_fullsized":"2023-12-07T10:19:01.811777+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:24:31.084727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802651+0000","last_change":"2023-12-07T10:17:41.612582+0000","last_active":"2023-12-07T10:19:01.802651+0000","last_peered":"2023-12-07T10:19:01.802651+0000","last_clean":"2023-12-07T10:19:01.802651+0000","last_became_active":"2023-12-07T10:17:41.612244+0000","last_became_peered":"2023-12-07T10:17:41.612244+0000","last_unstale":"2023-12-07T10:19:01.802651+0000","last_undegraded":"2023-12-07T10:19:01.802651+0000","last_fullsized":"2023-12-07T10:19:01.802651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:41:53.506037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.805642+0000","last_change":"2023-12-07T10:17:40.748565+0000","last_active":"2023-12-07T10:19:01.805642+0000","last_peered":"2023-12-07T10:19:01.805642+0000","last_clean":"2023-12-07T10:19:01.805642+0000","last_became_active":"2023-12-07T10:17:40.748250+0000","last_became_peered":"2023-12-07T10:17:40.748250+0000","last_unstale":"2023-12-07T10:19:01.805642+0000","last_undegraded":"2023-12-07T10:19:01.805642+0000","last_fullsized":"2023-12-07T10:19:01.805642+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:26:28.202440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":328,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.812930+0000","last_change":"2023-12-07T10:16:33.482173+0000","last_active":"2023-12-07T10:19:01.812930+0000","last_peered":"2023-12-07T10:19:01.812930+0000","last_clean":"2023-12-07T10:19:01.812930+0000","last_became_active":"2023-12-07T10:16:33.481699+0000","last_became_peered":"2023-12-07T10:16:33.481699+0000","last_unstale":"2023-12-07T10:19:01.812930+0000","last_undegraded":"2023-12-07T10:19:01.812930+0000","last_fullsized":"2023-12-07T10:19:01.812930+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:13:25.388258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:13:25.388258+0000","last_clean_scrub_stamp":"2023-12-07T10:13:25.388258+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:57:09.583677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807871+0000","last_change":"2023-12-07T10:17:45.806289+0000","last_active":"2023-12-07T10:19:01.807871+0000","last_peered":"2023-12-07T10:19:01.807871+0000","last_clean":"2023-12-07T10:19:01.807871+0000","last_became_active":"2023-12-07T10:17:45.805954+0000","last_became_peered":"2023-12-07T10:17:45.805954+0000","last_unstale":"2023-12-07T10:19:01.807871+0000","last_undegraded":"2023-12-07T10:19:01.807871+0000","last_fullsized":"2023-12-07T10:19:01.807871+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:21:09.955380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813281+0000","last_change":"2023-12-07T10:17:41.612322+0000","last_active":"2023-12-07T10:19:01.813281+0000","last_peered":"2023-12-07T10:19:01.813281+0000","last_clean":"2023-12-07T10:19:01.813281+0000","last_became_active":"2023-12-07T10:17:41.612039+0000","last_became_peered":"2023-12-07T10:17:41.612039+0000","last_unstale":"2023-12-07T10:19:01.813281+0000","last_undegraded":"2023-12-07T10:19:01.813281+0000","last_fullsized":"2023-12-07T10:19:01.813281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:40:49.883742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.812980+0000","last_change":"2023-12-07T10:17:40.749694+0000","last_active":"2023-12-07T10:19:01.812980+0000","last_peered":"2023-12-07T10:19:01.812980+0000","last_clean":"2023-12-07T10:19:01.812980+0000","last_became_active":"2023-12-07T10:17:40.749393+0000","last_became_peered":"2023-12-07T10:17:40.749393+0000","last_unstale":"2023-12-07T10:19:01.812980+0000","last_undegraded":"2023-12-07T10:19:01.812980+0000","last_fullsized":"2023-12-07T10:19:01.812980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:15:36.949062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.899040+0000","last_change":"2023-12-07T10:17:45.805769+0000","last_active":"2023-12-07T10:19:01.899040+0000","last_peered":"2023-12-07T10:19:01.899040+0000","last_clean":"2023-12-07T10:19:01.899040+0000","last_became_active":"2023-12-07T10:17:45.805346+0000","last_became_peered":"2023-12-07T10:17:45.805346+0000","last_unstale":"2023-12-07T10:19:01.899040+0000","last_undegraded":"2023-12-07T10:19:01.899040+0000","last_fullsized":"2023-12-07T10:19:01.899040+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:47:47.781933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.899044+0000","last_change":"2023-12-07T10:18:01.862558+0000","last_active":"2023-12-07T10:19:01.899044+0000","last_peered":"2023-12-07T10:19:01.899044+0000","last_clean":"2023-12-07T10:19:01.899044+0000","last_became_active":"2023-12-07T10:18:01.862063+0000","last_became_peered":"2023-12-07T10:18:01.862063+0000","last_unstale":"2023-12-07T10:19:01.899044+0000","last_undegraded":"2023-12-07T10:19:01.899044+0000","last_fullsized":"2023-12-07T10:19:01.899044+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:25:32.652160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813607+0000","last_change":"2023-12-07T10:17:41.617199+0000","last_active":"2023-12-07T10:19:01.813607+0000","last_peered":"2023-12-07T10:19:01.813607+0000","last_clean":"2023-12-07T10:19:01.813607+0000","last_became_active":"2023-12-07T10:17:41.616901+0000","last_became_peered":"2023-12-07T10:17:41.616901+0000","last_unstale":"2023-12-07T10:19:01.813607+0000","last_undegraded":"2023-12-07T10:19:01.813607+0000","last_fullsized":"2023-12-07T10:19:01.813607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:23:59.433861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802553+0000","last_change":"2023-12-07T10:17:45.790914+0000","last_active":"2023-12-07T10:19:01.802553+0000","last_peered":"2023-12-07T10:19:01.802553+0000","last_clean":"2023-12-07T10:19:01.802553+0000","last_became_active":"2023-12-07T10:17:45.790428+0000","last_became_peered":"2023-12-07T10:17:45.790428+0000","last_unstale":"2023-12-07T10:19:01.802553+0000","last_undegraded":"2023-12-07T10:19:01.802553+0000","last_fullsized":"2023-12-07T10:19:01.802553+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:03:57.516300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807601+0000","last_change":"2023-12-07T10:18:01.863765+0000","last_active":"2023-12-07T10:19:01.807601+0000","last_peered":"2023-12-07T10:19:01.807601+0000","last_clean":"2023-12-07T10:19:01.807601+0000","last_became_active":"2023-12-07T10:18:01.858926+0000","last_became_peered":"2023-12-07T10:18:01.858926+0000","last_unstale":"2023-12-07T10:19:01.807601+0000","last_undegraded":"2023-12-07T10:19:01.807601+0000","last_fullsized":"2023-12-07T10:19:01.807601+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:44:19.663592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.804412+0000","last_change":"2023-12-07T10:17:41.608513+0000","last_active":"2023-12-07T10:19:01.804412+0000","last_peered":"2023-12-07T10:19:01.804412+0000","last_clean":"2023-12-07T10:19:01.804412+0000","last_became_active":"2023-12-07T10:17:41.605049+0000","last_became_peered":"2023-12-07T10:17:41.605049+0000","last_unstale":"2023-12-07T10:19:01.804412+0000","last_undegraded":"2023-12-07T10:19:01.804412+0000","last_fullsized":"2023-12-07T10:19:01.804412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:33:00.730827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.809743+0000","last_change":"2023-12-07T10:17:45.792583+0000","last_active":"2023-12-07T10:19:01.809743+0000","last_peered":"2023-12-07T10:19:01.809743+0000","last_clean":"2023-12-07T10:19:01.809743+0000","last_became_active":"2023-12-07T10:17:45.792162+0000","last_became_peered":"2023-12-07T10:17:45.792162+0000","last_unstale":"2023-12-07T10:19:01.809743+0000","last_undegraded":"2023-12-07T10:19:01.809743+0000","last_fullsized":"2023-12-07T10:19:01.809743+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:47:30.901477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.805380+0000","last_change":"2023-12-07T10:17:40.750826+0000","last_active":"2023-12-07T10:19:01.805380+0000","last_peered":"2023-12-07T10:19:01.805380+0000","last_clean":"2023-12-07T10:19:01.805380+0000","last_became_active":"2023-12-07T10:17:40.750519+0000","last_became_peered":"2023-12-07T10:17:40.750519+0000","last_unstale":"2023-12-07T10:19:01.805380+0000","last_undegraded":"2023-12-07T10:19:01.805380+0000","last_fullsized":"2023-12-07T10:19:01.805380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:21:01.008897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.805305+0000","last_change":"2023-12-07T10:17:41.609488+0000","last_active":"2023-12-07T10:19:01.805305+0000","last_peered":"2023-12-07T10:19:01.805305+0000","last_clean":"2023-12-07T10:19:01.805305+0000","last_became_active":"2023-12-07T10:17:41.609061+0000","last_became_peered":"2023-12-07T10:17:41.609061+0000","last_unstale":"2023-12-07T10:19:01.805305+0000","last_undegraded":"2023-12-07T10:19:01.805305+0000","last_fullsized":"2023-12-07T10:19:01.805305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:39:32.729502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.805100+0000","last_change":"2023-12-07T10:17:45.797573+0000","last_active":"2023-12-07T10:19:01.805100+0000","last_peered":"2023-12-07T10:19:01.805100+0000","last_clean":"2023-12-07T10:19:01.805100+0000","last_became_active":"2023-12-07T10:17:45.797119+0000","last_became_peered":"2023-12-07T10:17:45.797119+0000","last_unstale":"2023-12-07T10:19:01.805100+0000","last_undegraded":"2023-12-07T10:19:01.805100+0000","last_fullsized":"2023-12-07T10:19:01.805100+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:17:27.100470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810670+0000","last_change":"2023-12-07T10:17:40.753596+0000","last_active":"2023-12-07T10:19:01.810670+0000","last_peered":"2023-12-07T10:19:01.810670+0000","last_clean":"2023-12-07T10:19:01.810670+0000","last_became_active":"2023-12-07T10:17:40.753251+0000","last_became_peered":"2023-12-07T10:17:40.753251+0000","last_unstale":"2023-12-07T10:19:01.810670+0000","last_undegraded":"2023-12-07T10:19:01.810670+0000","last_fullsized":"2023-12-07T10:19:01.810670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:08:30.868794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807743+0000","last_change":"2023-12-07T10:17:41.599689+0000","last_active":"2023-12-07T10:19:01.807743+0000","last_peered":"2023-12-07T10:19:01.807743+0000","last_clean":"2023-12-07T10:19:01.807743+0000","last_became_active":"2023-12-07T10:17:41.599358+0000","last_became_peered":"2023-12-07T10:17:41.599358+0000","last_unstale":"2023-12-07T10:19:01.807743+0000","last_undegraded":"2023-12-07T10:19:01.807743+0000","last_fullsized":"2023-12-07T10:19:01.807743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:44:19.742560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.215169+0000","last_change":"2023-12-07T10:19:03.215169+0000","last_active":"2023-12-07T10:19:03.215169+0000","last_peered":"2023-12-07T10:19:03.215169+0000","last_clean":"2023-12-07T10:19:03.215169+0000","last_became_active":"2023-12-07T10:19:03.214754+0000","last_became_peered":"2023-12-07T10:19:03.214754+0000","last_unstale":"2023-12-07T10:19:03.215169+0000","last_undegraded":"2023-12-07T10:19:03.215169+0000","last_fullsized":"2023-12-07T10:19:03.215169+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:51:41.461131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802597+0000","last_change":"2023-12-07T10:17:40.758663+0000","last_active":"2023-12-07T10:19:01.802597+0000","last_peered":"2023-12-07T10:19:01.802597+0000","last_clean":"2023-12-07T10:19:01.802597+0000","last_became_active":"2023-12-07T10:17:40.753611+0000","last_became_peered":"2023-12-07T10:17:40.753611+0000","last_unstale":"2023-12-07T10:19:01.802597+0000","last_undegraded":"2023-12-07T10:19:01.802597+0000","last_fullsized":"2023-12-07T10:19:01.802597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:00:59.256468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.898763+0000","last_change":"2023-12-07T10:17:41.616546+0000","last_active":"2023-12-07T10:19:01.898763+0000","last_peered":"2023-12-07T10:19:01.898763+0000","last_clean":"2023-12-07T10:19:01.898763+0000","last_became_active":"2023-12-07T10:17:41.615780+0000","last_became_peered":"2023-12-07T10:17:41.615780+0000","last_unstale":"2023-12-07T10:19:01.898763+0000","last_undegraded":"2023-12-07T10:19:01.898763+0000","last_fullsized":"2023-12-07T10:19:01.898763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:22:32.455784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.805077+0000","last_change":"2023-12-07T10:17:45.797550+0000","last_active":"2023-12-07T10:19:01.805077+0000","last_peered":"2023-12-07T10:19:01.805077+0000","last_clean":"2023-12-07T10:19:01.805077+0000","last_became_active":"2023-12-07T10:17:45.797140+0000","last_became_peered":"2023-12-07T10:17:45.797140+0000","last_unstale":"2023-12-07T10:19:01.805077+0000","last_undegraded":"2023-12-07T10:19:01.805077+0000","last_fullsized":"2023-12-07T10:19:01.805077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:53:44.984904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.809964+0000","last_change":"2023-12-07T10:17:40.748410+0000","last_active":"2023-12-07T10:19:01.809964+0000","last_peered":"2023-12-07T10:19:01.809964+0000","last_clean":"2023-12-07T10:19:01.809964+0000","last_became_active":"2023-12-07T10:17:40.748069+0000","last_became_peered":"2023-12-07T10:17:40.748069+0000","last_unstale":"2023-12-07T10:19:01.809964+0000","last_undegraded":"2023-12-07T10:19:01.809964+0000","last_fullsized":"2023-12-07T10:19:01.809964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:51:48.278974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803033+0000","last_change":"2023-12-07T10:17:41.609248+0000","last_active":"2023-12-07T10:19:01.803033+0000","last_peered":"2023-12-07T10:19:01.803033+0000","last_clean":"2023-12-07T10:19:01.803033+0000","last_became_active":"2023-12-07T10:17:41.608943+0000","last_became_peered":"2023-12-07T10:17:41.608943+0000","last_unstale":"2023-12-07T10:19:01.803033+0000","last_undegraded":"2023-12-07T10:19:01.803033+0000","last_fullsized":"2023-12-07T10:19:01.803033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:29:32.322273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.805250+0000","last_change":"2023-12-07T10:17:45.794855+0000","last_active":"2023-12-07T10:19:01.805250+0000","last_peered":"2023-12-07T10:19:01.805250+0000","last_clean":"2023-12-07T10:19:01.805250+0000","last_became_active":"2023-12-07T10:17:45.794410+0000","last_became_peered":"2023-12-07T10:17:45.794410+0000","last_unstale":"2023-12-07T10:19:01.805250+0000","last_undegraded":"2023-12-07T10:19:01.805250+0000","last_fullsized":"2023-12-07T10:19:01.805250+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:57:25.118036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808678+0000","last_change":"2023-12-07T10:17:40.750471+0000","last_active":"2023-12-07T10:19:01.808678+0000","last_peered":"2023-12-07T10:19:01.808678+0000","last_clean":"2023-12-07T10:19:01.808678+0000","last_became_active":"2023-12-07T10:17:40.750176+0000","last_became_peered":"2023-12-07T10:17:40.750176+0000","last_unstale":"2023-12-07T10:19:01.808678+0000","last_undegraded":"2023-12-07T10:19:01.808678+0000","last_fullsized":"2023-12-07T10:19:01.808678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:04:22.305343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.812625+0000","last_change":"2023-12-07T10:17:41.614087+0000","last_active":"2023-12-07T10:19:01.812625+0000","last_peered":"2023-12-07T10:19:01.812625+0000","last_clean":"2023-12-07T10:19:01.812625+0000","last_became_active":"2023-12-07T10:17:41.607689+0000","last_became_peered":"2023-12-07T10:17:41.607689+0000","last_unstale":"2023-12-07T10:19:01.812625+0000","last_undegraded":"2023-12-07T10:19:01.812625+0000","last_fullsized":"2023-12-07T10:19:01.812625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:50:10.316085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.841001+0000","last_change":"2023-12-07T10:19:02.841001+0000","last_active":"2023-12-07T10:19:02.841001+0000","last_peered":"2023-12-07T10:19:02.841001+0000","last_clean":"2023-12-07T10:19:02.841001+0000","last_became_active":"2023-12-07T10:19:02.840595+0000","last_became_peered":"2023-12-07T10:19:02.840595+0000","last_unstale":"2023-12-07T10:19:02.841001+0000","last_undegraded":"2023-12-07T10:19:02.841001+0000","last_fullsized":"2023-12-07T10:19:02.841001+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:18:35.590140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'5","reported_seq":89,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:12.854772+0000","last_change":"2023-12-07T10:17:40.753645+0000","last_active":"2023-12-07T10:19:12.854772+0000","last_peered":"2023-12-07T10:19:12.854772+0000","last_clean":"2023-12-07T10:19:12.854772+0000","last_became_active":"2023-12-07T10:17:40.753326+0000","last_became_peered":"2023-12-07T10:17:40.753326+0000","last_unstale":"2023-12-07T10:19:12.854772+0000","last_undegraded":"2023-12-07T10:19:12.854772+0000","last_fullsized":"2023-12-07T10:19:12.854772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:43:03.617591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":63,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811725+0000","last_change":"2023-12-07T10:17:41.607968+0000","last_active":"2023-12-07T10:19:01.811725+0000","last_peered":"2023-12-07T10:19:01.811725+0000","last_clean":"2023-12-07T10:19:01.811725+0000","last_became_active":"2023-12-07T10:17:41.607621+0000","last_became_peered":"2023-12-07T10:17:41.607621+0000","last_unstale":"2023-12-07T10:19:01.811725+0000","last_undegraded":"2023-12-07T10:19:01.811725+0000","last_fullsized":"2023-12-07T10:19:01.811725+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:55:10.224738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803538+0000","last_change":"2023-12-07T10:17:45.793678+0000","last_active":"2023-12-07T10:19:01.803538+0000","last_peered":"2023-12-07T10:19:01.803538+0000","last_clean":"2023-12-07T10:19:01.803538+0000","last_became_active":"2023-12-07T10:17:45.793283+0000","last_became_peered":"2023-12-07T10:17:45.793283+0000","last_unstale":"2023-12-07T10:19:01.803538+0000","last_undegraded":"2023-12-07T10:19:01.803538+0000","last_fullsized":"2023-12-07T10:19:01.803538+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:33:56.501107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808567+0000","last_change":"2023-12-07T10:17:40.762069+0000","last_active":"2023-12-07T10:19:01.808567+0000","last_peered":"2023-12-07T10:19:01.808567+0000","last_clean":"2023-12-07T10:19:01.808567+0000","last_became_active":"2023-12-07T10:17:40.761752+0000","last_became_peered":"2023-12-07T10:17:40.761752+0000","last_unstale":"2023-12-07T10:19:01.808567+0000","last_undegraded":"2023-12-07T10:19:01.808567+0000","last_fullsized":"2023-12-07T10:19:01.808567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:57:55.736144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811362+0000","last_change":"2023-12-07T10:17:41.607347+0000","last_active":"2023-12-07T10:19:01.811362+0000","last_peered":"2023-12-07T10:19:01.811362+0000","last_clean":"2023-12-07T10:19:01.811362+0000","last_became_active":"2023-12-07T10:17:41.606618+0000","last_became_peered":"2023-12-07T10:17:41.606618+0000","last_unstale":"2023-12-07T10:19:01.811362+0000","last_undegraded":"2023-12-07T10:19:01.811362+0000","last_fullsized":"2023-12-07T10:19:01.811362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:53:34.748130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807110+0000","last_change":"2023-12-07T10:17:45.791459+0000","last_active":"2023-12-07T10:19:01.807110+0000","last_peered":"2023-12-07T10:19:01.807110+0000","last_clean":"2023-12-07T10:19:01.807110+0000","last_became_active":"2023-12-07T10:17:45.791070+0000","last_became_peered":"2023-12-07T10:17:45.791070+0000","last_unstale":"2023-12-07T10:19:01.807110+0000","last_undegraded":"2023-12-07T10:19:01.807110+0000","last_fullsized":"2023-12-07T10:19:01.807110+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:46:35.985990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813745+0000","last_change":"2023-12-07T10:17:40.762726+0000","last_active":"2023-12-07T10:19:01.813745+0000","last_peered":"2023-12-07T10:19:01.813745+0000","last_clean":"2023-12-07T10:19:01.813745+0000","last_became_active":"2023-12-07T10:17:40.762378+0000","last_became_peered":"2023-12-07T10:17:40.762378+0000","last_unstale":"2023-12-07T10:19:01.813745+0000","last_undegraded":"2023-12-07T10:19:01.813745+0000","last_fullsized":"2023-12-07T10:19:01.813745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:42:06.193834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811313+0000","last_change":"2023-12-07T10:17:41.602356+0000","last_active":"2023-12-07T10:19:01.811313+0000","last_peered":"2023-12-07T10:19:01.811313+0000","last_clean":"2023-12-07T10:19:01.811313+0000","last_became_active":"2023-12-07T10:17:41.602025+0000","last_became_peered":"2023-12-07T10:17:41.602025+0000","last_unstale":"2023-12-07T10:19:01.811313+0000","last_undegraded":"2023-12-07T10:19:01.811313+0000","last_fullsized":"2023-12-07T10:19:01.811313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:33:04.994510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.898923+0000","last_change":"2023-12-07T10:17:41.616766+0000","last_active":"2023-12-07T10:19:01.898923+0000","last_peered":"2023-12-07T10:19:01.898923+0000","last_clean":"2023-12-07T10:19:01.898923+0000","last_became_active":"2023-12-07T10:17:41.616221+0000","last_became_peered":"2023-12-07T10:17:41.616221+0000","last_unstale":"2023-12-07T10:19:01.898923+0000","last_undegraded":"2023-12-07T10:19:01.898923+0000","last_fullsized":"2023-12-07T10:19:01.898923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:16:00.393772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810004+0000","last_change":"2023-12-07T10:17:41.613646+0000","last_active":"2023-12-07T10:19:01.810004+0000","last_peered":"2023-12-07T10:19:01.810004+0000","last_clean":"2023-12-07T10:19:01.810004+0000","last_became_active":"2023-12-07T10:17:41.613257+0000","last_became_peered":"2023-12-07T10:17:41.613257+0000","last_unstale":"2023-12-07T10:19:01.810004+0000","last_undegraded":"2023-12-07T10:19:01.810004+0000","last_fullsized":"2023-12-07T10:19:01.810004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:10:33.416955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810008+0000","last_change":"2023-12-07T10:17:41.620017+0000","last_active":"2023-12-07T10:19:01.810008+0000","last_peered":"2023-12-07T10:19:01.810008+0000","last_clean":"2023-12-07T10:19:01.810008+0000","last_became_active":"2023-12-07T10:17:41.619647+0000","last_became_peered":"2023-12-07T10:17:41.619647+0000","last_unstale":"2023-12-07T10:19:01.810008+0000","last_undegraded":"2023-12-07T10:19:01.810008+0000","last_fullsized":"2023-12-07T10:19:01.810008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:36:42.891160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.803726+0000","last_change":"2023-12-07T10:17:41.610803+0000","last_active":"2023-12-07T10:19:01.803726+0000","last_peered":"2023-12-07T10:19:01.803726+0000","last_clean":"2023-12-07T10:19:01.803726+0000","last_became_active":"2023-12-07T10:17:41.610388+0000","last_became_peered":"2023-12-07T10:17:41.610388+0000","last_unstale":"2023-12-07T10:19:01.803726+0000","last_undegraded":"2023-12-07T10:19:01.803726+0000","last_fullsized":"2023-12-07T10:19:01.803726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:33:25.787281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813690+0000","last_change":"2023-12-07T10:17:41.615916+0000","last_active":"2023-12-07T10:19:01.813690+0000","last_peered":"2023-12-07T10:19:01.813690+0000","last_clean":"2023-12-07T10:19:01.813690+0000","last_became_active":"2023-12-07T10:17:41.615591+0000","last_became_peered":"2023-12-07T10:17:41.615591+0000","last_unstale":"2023-12-07T10:19:01.813690+0000","last_undegraded":"2023-12-07T10:19:01.813690+0000","last_fullsized":"2023-12-07T10:19:01.813690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:40:45.541571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.899185+0000","last_change":"2023-12-07T10:17:41.614735+0000","last_active":"2023-12-07T10:19:01.899185+0000","last_peered":"2023-12-07T10:19:01.899185+0000","last_clean":"2023-12-07T10:19:01.899185+0000","last_became_active":"2023-12-07T10:17:41.614388+0000","last_became_peered":"2023-12-07T10:17:41.614388+0000","last_unstale":"2023-12-07T10:19:01.899185+0000","last_undegraded":"2023-12-07T10:19:01.899185+0000","last_fullsized":"2023-12-07T10:19:01.899185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:10:07.196841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810348+0000","last_change":"2023-12-07T10:17:41.620059+0000","last_active":"2023-12-07T10:19:01.810348+0000","last_peered":"2023-12-07T10:19:01.810348+0000","last_clean":"2023-12-07T10:19:01.810348+0000","last_became_active":"2023-12-07T10:17:41.619770+0000","last_became_peered":"2023-12-07T10:17:41.619770+0000","last_unstale":"2023-12-07T10:19:01.810348+0000","last_undegraded":"2023-12-07T10:19:01.810348+0000","last_fullsized":"2023-12-07T10:19:01.810348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:13:43.421776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.805006+0000","last_change":"2023-12-07T10:17:41.609720+0000","last_active":"2023-12-07T10:19:01.805006+0000","last_peered":"2023-12-07T10:19:01.805006+0000","last_clean":"2023-12-07T10:19:01.805006+0000","last_became_active":"2023-12-07T10:17:41.609417+0000","last_became_peered":"2023-12-07T10:17:41.609417+0000","last_unstale":"2023-12-07T10:19:01.805006+0000","last_undegraded":"2023-12-07T10:19:01.805006+0000","last_fullsized":"2023-12-07T10:19:01.805006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:48:47.768754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807362+0000","last_change":"2023-12-07T10:17:41.620952+0000","last_active":"2023-12-07T10:19:01.807362+0000","last_peered":"2023-12-07T10:19:01.807362+0000","last_clean":"2023-12-07T10:19:01.807362+0000","last_became_active":"2023-12-07T10:17:41.620560+0000","last_became_peered":"2023-12-07T10:17:41.620560+0000","last_unstale":"2023-12-07T10:19:01.807362+0000","last_undegraded":"2023-12-07T10:19:01.807362+0000","last_fullsized":"2023-12-07T10:19:01.807362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:51:37.699511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802559+0000","last_change":"2023-12-07T10:17:41.611908+0000","last_active":"2023-12-07T10:19:01.802559+0000","last_peered":"2023-12-07T10:19:01.802559+0000","last_clean":"2023-12-07T10:19:01.802559+0000","last_became_active":"2023-12-07T10:17:41.610433+0000","last_became_peered":"2023-12-07T10:17:41.610433+0000","last_unstale":"2023-12-07T10:19:01.802559+0000","last_undegraded":"2023-12-07T10:19:01.802559+0000","last_fullsized":"2023-12-07T10:19:01.802559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:42:53.906221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.811612+0000","last_change":"2023-12-07T10:17:41.604116+0000","last_active":"2023-12-07T10:19:01.811612+0000","last_peered":"2023-12-07T10:19:01.811612+0000","last_clean":"2023-12-07T10:19:01.811612+0000","last_became_active":"2023-12-07T10:17:41.603880+0000","last_became_peered":"2023-12-07T10:17:41.603880+0000","last_unstale":"2023-12-07T10:19:01.811612+0000","last_undegraded":"2023-12-07T10:19:01.811612+0000","last_fullsized":"2023-12-07T10:19:01.811612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:56:27.343169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813422+0000","last_change":"2023-12-07T10:17:41.610497+0000","last_active":"2023-12-07T10:19:01.813422+0000","last_peered":"2023-12-07T10:19:01.813422+0000","last_clean":"2023-12-07T10:19:01.813422+0000","last_became_active":"2023-12-07T10:17:41.609956+0000","last_became_peered":"2023-12-07T10:17:41.609956+0000","last_unstale":"2023-12-07T10:19:01.813422+0000","last_undegraded":"2023-12-07T10:19:01.813422+0000","last_fullsized":"2023-12-07T10:19:01.813422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:04:03.967377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.899100+0000","last_change":"2023-12-07T10:17:41.612284+0000","last_active":"2023-12-07T10:19:01.899100+0000","last_peered":"2023-12-07T10:19:01.899100+0000","last_clean":"2023-12-07T10:19:01.899100+0000","last_became_active":"2023-12-07T10:17:41.611917+0000","last_became_peered":"2023-12-07T10:17:41.611917+0000","last_unstale":"2023-12-07T10:19:01.899100+0000","last_undegraded":"2023-12-07T10:19:01.899100+0000","last_fullsized":"2023-12-07T10:19:01.899100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:02:27.355883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":81645,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":677,"num_read_kb":426,"num_write":243,"num_write_kb":152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":244935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383711,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":516906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.434,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.72299999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.76400000000000001}]},{"osd":1,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.62,"15min":0.60499999999999998},"min":{"1min":0.505,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.40899999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.76800000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.45900000000000002,"15min":0.45000000000000001},"min":{"1min":0.36899999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.626},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.46999999999999997,"15min":0.45700000000000002},"min":{"1min":0.376,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.496}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.67900000000000005,"15min":0.67200000000000004},"min":{"1min":0.48899999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90300000000000002,"5min":1.046,"15min":1.046},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.84699999999999998,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.66400000000000003}]},{"osd":5,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.56799999999999995,"15min":0.55700000000000005},"min":{"1min":0.40200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80700000000000005,"5min":1.034,"15min":1.034},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.59199999999999997,"15min":0.57999999999999996},"min":{"1min":0.51300000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92900000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.441}]},{"osd":6,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.46200000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90700000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.72399999999999998,"15min":0.71599999999999997},"min":{"1min":0.60399999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91300000000000003,"5min":1.363,"15min":1.363},"last":0.84699999999999998}]},{"osd":7,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.76100000000000001,"15min":0.751},"min":{"1min":0.63700000000000001,"5min":0.501,"15min":0.501},"max":{"1min":0.97899999999999998,"5min":1.25,"15min":1.25},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.70599999999999996,"15min":0.69599999999999995},"min":{"1min":0.51500000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95399999999999996,"5min":1.085,"15min":1.085},"last":0.747}]},{"osd":8,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.69799999999999995}]},{"osd":9,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.78000000000000003,"15min":0.76600000000000001},"min":{"1min":0.69799999999999995,"5min":0.496,"15min":0.496},"max":{"1min":0.96799999999999997,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.874},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.78700000000000003,"15min":0.77500000000000002},"min":{"1min":0.71899999999999997,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.014,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.78100000000000003}]},{"osd":10,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.751,"15min":0.74099999999999999},"min":{"1min":0.59999999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.996,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.79300000000000004,"15min":0.78300000000000003},"min":{"1min":0.70999999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.999,"5min":1.411,"15min":1.411},"last":0.81200000000000006}]}]},{"osd":10,"up_from":72,"seq":309237645347,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28732,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716420,"statfs":{"total":95995035648,"available":95965614080,"internally_reserved":0,"allocated":1748992,"data_stored":1168120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.3610000000000002,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.48599999999999999,"15min":0.49099999999999999},"min":{"1min":0.34100000000000003,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.627,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.49299999999999999,"15min":0.495},"min":{"1min":0.36299999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.51300000000000001}]},{"osd":2,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82699999999999996,"5min":1.008,"15min":1.008},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.42699999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81299999999999994,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.77200000000000002}]},{"osd":3,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.45500000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.78300000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.62},{"interface":"front","average":{"1min":0.626,"5min":0.67900000000000005,"15min":0.68799999999999994},"min":{"1min":0.48199999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.79000000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.746}]},{"osd":4,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57599999999999996,"15min":0.58299999999999996},"min":{"1min":0.33300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71799999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58599999999999997,"15min":0.59099999999999997},"min":{"1min":0.40699999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.751,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.95499999999999996,"5min":1.22,"15min":1.22},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72699999999999998,"15min":0.72799999999999998},"min":{"1min":0.49299999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93899999999999995,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.81999999999999995}]},{"osd":6,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.747,"15min":0.751},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.89200000000000002,"5min":1.054,"15min":1.054},"last":0.871},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78900000000000003,"15min":0.78700000000000003},"min":{"1min":0.48499999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.83699999999999997}]},{"osd":7,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.63900000000000001}]},{"osd":8,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.79600000000000004,"15min":0.80500000000000005},"min":{"1min":0.53100000000000003,"5min":0.503,"15min":0.503},"max":{"1min":0.90700000000000003,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.83099999999999996,"15min":0.84199999999999997},"min":{"1min":0.629,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.91700000000000004,"5min":1.165,"15min":1.165},"last":0.90300000000000002}]},{"osd":9,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.83399999999999996,"15min":0.83999999999999997},"min":{"1min":0.622,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.98399999999999999,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.85199999999999998,"15min":0.86599999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.95099999999999996}]},{"osd":11,"last update":"Thu Dec 7 10:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.81699999999999995,"15min":0.82099999999999995},"min":{"1min":0.56899999999999995,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.95599999999999996,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81999999999999995,"15min":0.82199999999999995},"min":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.026,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.98299999999999998}]}]},{"osd":9,"up_from":67,"seq":287762808872,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":520820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.56599999999999995,"15min":0.60799999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.626,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57199999999999995,"15min":0.58499999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.42099999999999999}]},{"osd":1,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.66900000000000004,"15min":0.69899999999999995},"min":{"1min":0.45300000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.70899999999999996,"5min":1.899,"15min":1.899},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.745,"15min":0.78300000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.85799999999999998,"5min":1.409,"15min":1.409},"last":0.46400000000000002}]},{"osd":2,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":0.89700000000000002,"15min":0.85099999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":11.782999999999999,"5min":11.782999999999999,"15min":11.782999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.82999999999999996,"15min":0.88400000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.94499999999999995,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":0.79800000000000004}]},{"osd":3,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.68400000000000005,"15min":0.73099999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91400000000000003,"5min":1.47,"15min":1.47},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.753,"15min":0.83299999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.032,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.47399999999999998}]},{"osd":4,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":0.879,"15min":0.85599999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":11.808999999999999,"5min":11.808999999999999,"15min":11.808999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.82999999999999996,"15min":0.88500000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.88,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.51100000000000001}]},{"osd":5,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.87,"15min":0.93600000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.91800000000000004,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":1.3109999999999999,"5min":0.93100000000000005,"15min":0.90400000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":11.863,"5min":11.863,"15min":11.863},"last":0.81100000000000005}]},{"osd":6,"last update":"Thu Dec 7 10:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":0.749}]},{"osd":7,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.879,"15min":0.93999999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86099999999999999,"5min":1.5,"15min":1.5},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.94899999999999995,"15min":1.0289999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.96899999999999997,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":0.51900000000000002}]},{"osd":8,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.032,"15min":1},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":11.831,"5min":11.831,"15min":11.831},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.96999999999999997,"15min":1.048},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.006,"5min":2.6539999999999999,"15min":2.6539999999999999},"last":0.82499999999999996}]},{"osd":10,"last update":"Thu Dec 7 10:18:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":1.02,"15min":1.052},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.2889999999999999,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.93899999999999995,"15min":0.96899999999999997},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.0189999999999999,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.56999999999999995}]},{"osd":11,"last update":"Thu Dec 7 10:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.78600000000000003}]}]},{"osd":8,"up_from":59,"seq":253403070510,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":516990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55900000000000005,"15min":0.56299999999999994},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.58499999999999996}]},{"osd":1,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.59199999999999997,"15min":0.57699999999999996},"min":{"1min":0.45100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86299999999999999,"5min":1,"15min":1},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.60299999999999998,"15min":0.58299999999999996},"min":{"1min":0.52800000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.72899999999999998}]},{"osd":2,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.46200000000000002,"15min":0.44400000000000001},"min":{"1min":0.317,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.76700000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.46300000000000002,"15min":0.438},"min":{"1min":0.29699999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.44400000000000001}]},{"osd":3,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59799999999999998,"15min":0.58599999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.83199999999999996,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.81599999999999995}]},{"osd":4,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.498,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.86899999999999999,"5min":0.876,"15min":0.876},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.64400000000000002,"15min":0.63400000000000001},"min":{"1min":0.54400000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89100000000000001,"5min":1.079,"15min":1.079},"last":0.69499999999999995}]},{"osd":5,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.54100000000000004,"15min":0.51700000000000002},"min":{"1min":0.34799999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.56799999999999995,"15min":0.55300000000000005},"min":{"1min":0.38200000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.84199999999999997,"5min":1.024,"15min":1.024},"last":0.49399999999999999}]},{"osd":6,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.67400000000000004,"15min":0.65600000000000003},"min":{"1min":0.499,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.96099999999999997,"5min":1.046,"15min":1.046},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.68999999999999995,"15min":0.66400000000000003},"min":{"1min":0.48699999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.80000000000000004}]},{"osd":7,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.68700000000000006,"15min":0.65500000000000003},"min":{"1min":0.57999999999999996,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.69199999999999995,"15min":0.67500000000000004},"min":{"1min":0.58699999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.95599999999999996,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.71499999999999997}]},{"osd":9,"last update":"Thu Dec 7 10:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.73899999999999999,"15min":0.72099999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":1.5269999999999999},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.76900000000000002,"15min":0.74299999999999999},"min":{"1min":0.62,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":1.4930000000000001}]},{"osd":10,"last update":"Thu Dec 7 10:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.74399999999999999,"15min":0.72699999999999998},"min":{"1min":0.626,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.74299999999999999,"15min":0.72799999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.78300000000000003}]},{"osd":11,"last update":"Thu Dec 7 10:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.751}]}]},{"osd":7,"up_from":52,"seq":223338299442,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1679360,"data_stored":1123647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.58199999999999996,"15min":0.55200000000000005},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.5369999999999999,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56599999999999995,"15min":0.56000000000000005},"min":{"1min":0.39700000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.36899999999999999}]},{"osd":1,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.434,"15min":0.432},"min":{"1min":0.29699999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60499999999999998,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45600000000000002},"min":{"1min":0.254,"5min":0.253,"15min":0.253},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53300000000000003}]},{"osd":2,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.434,"5min":0.314,"15min":0.314},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.57899999999999996,"15min":0.57099999999999995},"min":{"1min":0.39600000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.65800000000000003}]},{"osd":3,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73499999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.45500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.79900000000000004,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.46899999999999997}]},{"osd":4,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51900000000000002,"15min":0.51600000000000001},"min":{"1min":0.315,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.748,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.52400000000000002,"15min":0.53300000000000003},"min":{"1min":0.33600000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.77400000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.44800000000000001}]},{"osd":5,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63900000000000001,"15min":0.64400000000000002},"min":{"1min":0.45600000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.79200000000000004,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.443,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.84999999999999998,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.72599999999999998}]},{"osd":6,"last update":"Thu Dec 7 10:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85699999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.68600000000000005,"15min":0.69199999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94999999999999996,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.61099999999999999}]},{"osd":8,"last update":"Thu Dec 7 10:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70099999999999996,"15min":0.69599999999999995},"min":{"1min":0.46200000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90600000000000003,"5min":1.012,"15min":1.012},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72399999999999998,"15min":0.72799999999999998},"min":{"1min":0.52900000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83399999999999996,"5min":1.026,"15min":1.026},"last":0.748}]},{"osd":9,"last update":"Thu Dec 7 10:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73599999999999999,"15min":0.74199999999999999},"min":{"1min":0.55700000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94399999999999995,"5min":1.075,"15min":1.075},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.75800000000000001,"15min":0.755},"min":{"1min":0.56399999999999995,"5min":0.505,"15min":0.505},"max":{"1min":2.1989999999999998,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":0.77100000000000002}]},{"osd":10,"last update":"Thu Dec 7 10:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.76300000000000001,"15min":0.76600000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.96099999999999997,"5min":1.03,"15min":1.03},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73999999999999999,"15min":0.74199999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.89200000000000002,"5min":1.002,"15min":1.002},"last":0.81000000000000005}]},{"osd":11,"last update":"Thu Dec 7 10:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.85099999999999998}]}]},{"osd":6,"up_from":47,"seq":201863462967,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":512228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.5,"15min":0.48199999999999998},"min":{"1min":0.374,"5min":0.316,"15min":0.316},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.53000000000000003,"15min":0.48799999999999999},"min":{"1min":0.35399999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.34599999999999997}]},{"osd":1,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.69299999999999995,"15min":0.67100000000000004},"min":{"1min":0.56399999999999995,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":3.077,"5min":3.077,"15min":3.077},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.70599999999999996,"15min":0.67300000000000004},"min":{"1min":0.55300000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.113,"5min":2.113,"15min":2.113},"last":0.79800000000000004}]},{"osd":2,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.71999999999999997,"15min":0.68200000000000005},"min":{"1min":0.51400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.42299999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":3.153,"5min":3.153,"15min":3.153},"last":0.496}]},{"osd":3,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":3.1200000000000001,"5min":3.1200000000000001,"15min":3.1200000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.67600000000000005,"15min":0.65100000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.51900000000000002}]},{"osd":4,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.77900000000000003,"15min":0.76300000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":3.2629999999999999,"5min":3.2629999999999999,"15min":3.2629999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.76000000000000001,"15min":0.748},"min":{"1min":0.56799999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.747}]},{"osd":5,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.79600000000000004,"15min":0.80200000000000005},"min":{"1min":0.628,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.81200000000000006,"15min":0.79600000000000004},"min":{"1min":0.55200000000000005,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.5369999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.60999999999999999}]},{"osd":7,"last update":"Thu Dec 7 10:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.78000000000000003,"15min":0.78500000000000003},"min":{"1min":0.47399999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0489999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.80400000000000005,"15min":0.78800000000000003},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.97999999999999998,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.77600000000000002}]},{"osd":8,"last update":"Thu Dec 7 10:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.85999999999999999,"15min":0.86899999999999999},"min":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.569,"5min":2.569,"15min":2.569},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.88600000000000001,"15min":0.89400000000000002},"min":{"1min":0.53400000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":3.1850000000000001,"5min":3.1850000000000001,"15min":3.1850000000000001},"last":0.56499999999999995}]},{"osd":9,"last update":"Thu Dec 7 10:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.872,"15min":0.874},"min":{"1min":0.66000000000000003,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.88100000000000001,"15min":0.86099999999999999},"min":{"1min":0.63300000000000001,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.6600000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":0.76100000000000001}]},{"osd":10,"last update":"Thu Dec 7 10:18:34 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.85999999999999999,"15min":0.84999999999999998},"min":{"1min":0.58299999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.3980000000000001,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.88600000000000001,"15min":0.88300000000000001},"min":{"1min":0.73899999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.81899999999999995}]},{"osd":11,"last update":"Thu Dec 7 10:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.72899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1126400,"data_stored":577711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64300000000000002,"15min":0.57099999999999995},"min":{"1min":0.38200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.93100000000000005,"5min":6.8120000000000003,"15min":6.8120000000000003},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58699999999999997,"15min":0.55100000000000005},"min":{"1min":0.44600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.81999999999999995,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.68899999999999995}]},{"osd":2,"last update":"Thu Dec 7 10:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.59499999999999997,"15min":0.54300000000000004},"min":{"1min":0.46899999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.87,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.55800000000000005,"15min":0.51000000000000001},"min":{"1min":0.42999999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.76300000000000001}]},{"osd":3,"last update":"Thu Dec 7 10:18:56 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60499999999999998,"15min":0.56299999999999994},"min":{"1min":0.39400000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.82399999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.745},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.37,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.67200000000000004,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.61699999999999999}]},{"osd":4,"last update":"Thu Dec 7 10:18:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.437,"15min":0.39700000000000002},"min":{"1min":0.35599999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.47899999999999998,"15min":0.42099999999999999},"min":{"1min":0.36599999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.77100000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.58499999999999996}]},{"osd":5,"last update":"Thu Dec 7 10:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.65700000000000003,"15min":0.58799999999999997},"min":{"1min":0.45000000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.98099999999999998,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.60599999999999998,"15min":0.56200000000000006},"min":{"1min":0.51600000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82099999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.80700000000000005}]},{"osd":6,"last update":"Thu Dec 7 10:19:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.71799999999999997,"15min":0.67600000000000005},"min":{"1min":0.54000000000000004,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.101,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65700000000000003,"15min":0.64300000000000002},"min":{"1min":0.49199999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.83099999999999996,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.78000000000000003}]},{"osd":7,"last update":"Thu Dec 7 10:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.54900000000000004,"15min":0.52600000000000002},"min":{"1min":0.38100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.67300000000000004,"15min":0.63400000000000001},"min":{"1min":0.46300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93799999999999994,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.872}]},{"osd":8,"last update":"Thu Dec 7 10:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.77100000000000002,"15min":0.73499999999999999},"min":{"1min":0.70999999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.026,"5min":1.175,"15min":1.175},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67100000000000004,"15min":0.64500000000000002},"min":{"1min":0.55500000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86699999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.84399999999999997}]},{"osd":9,"last update":"Thu Dec 7 10:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.76900000000000002,"15min":0.75600000000000001},"min":{"1min":0.55900000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.96699999999999997,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.86299999999999999,"15min":0.85199999999999998},"min":{"1min":0.63300000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0660000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":1.0089999999999999}]},{"osd":10,"last update":"Thu Dec 7 10:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.73999999999999999,"15min":0.73699999999999999},"min":{"1min":0.63200000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.75700000000000001,"15min":0.751},"min":{"1min":0.61099999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.85899999999999999}]},{"osd":11,"last update":"Thu Dec 7 10:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.96299999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1081344,"data_stored":513684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Dec 7 10:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.628,"15min":0.58199999999999996},"min":{"1min":0.40899999999999997,"5min":0.375,"15min":0.375},"max":{"1min":2.3929999999999998,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.61099999999999999,"15min":0.57099999999999995},"min":{"1min":0.39500000000000002,"5min":0.34399999999999997,"15min":0.34300000000000003},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.55300000000000005}]},{"osd":2,"last update":"Thu Dec 7 10:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.63400000000000001,"15min":0.59599999999999997},"min":{"1min":0.44500000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.3420000000000001,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.64600000000000002,"15min":0.60499999999999998},"min":{"1min":0.42299999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":3.4140000000000001,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.73899999999999999}]},{"osd":3,"last update":"Thu Dec 7 10:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":3.484,"5min":3.484,"15min":3.484},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":3.1379999999999999,"5min":3.1379999999999999,"15min":3.1379999999999999},"last":0.60299999999999998}]},{"osd":4,"last update":"Thu Dec 7 10:18:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.59999999999999998,"15min":0.56599999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91700000000000004,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.623,"15min":0.60499999999999998},"min":{"1min":0.46000000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82799999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.72899999999999998}]},{"osd":5,"last update":"Thu Dec 7 10:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.63900000000000001,"15min":0.624},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91000000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.626,"15min":0.624},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85399999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.628}]},{"osd":6,"last update":"Thu Dec 7 10:19:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63600000000000001,"15min":0.628},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":2.5,"5min":2.5,"15min":2.5},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.67200000000000004,"15min":0.64400000000000002},"min":{"1min":0.38100000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.452,"5min":3.452,"15min":3.452},"last":0.66200000000000003}]},{"osd":7,"last update":"Thu Dec 7 10:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":3.0430000000000001,"5min":3.0430000000000001,"15min":3.0430000000000001},"last":0.67800000000000005}]},{"osd":8,"last update":"Thu Dec 7 10:18:46 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.81499999999999995,"15min":0.80500000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":3.75,"5min":3.75,"15min":3.75},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.80600000000000005,"15min":0.81100000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.69399999999999995}]},{"osd":9,"last update":"Thu Dec 7 10:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.84299999999999997,"15min":0.85999999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.4180000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.49199999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":3.7050000000000001,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.85599999999999998}]},{"osd":10,"last update":"Thu Dec 7 10:18:35 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.79900000000000004,"15min":0.77900000000000003},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":3.0979999999999999,"5min":3.0979999999999999,"15min":3.0979999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.88,"5min":0.85199999999999998,"15min":0.84699999999999998},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.383,"5min":2.383,"15min":2.383},"last":0.88}]},{"osd":11,"last update":"Thu Dec 7 10:18:55 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.84599999999999997,"15min":0.81799999999999995},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.7879999999999998,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.82399999999999995,"15min":0.80200000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.1749999999999998,"5min":3.1749999999999998,"15min":3.1749999999999998},"last":0.82099999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":588201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.51000000000000001},"min":{"1min":0.41599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55100000000000005,"15min":0.52000000000000002},"min":{"1min":0.35899999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.79300000000000004}]},{"osd":1,"last update":"Thu Dec 7 10:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.58099999999999996,"15min":0.52800000000000002},"min":{"1min":0.42199999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.871},{"interface":"front","average":{"1min":0.622,"5min":0.56200000000000006,"15min":0.52300000000000002},"min":{"1min":0.39800000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79700000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.81499999999999995}]},{"osd":3,"last update":"Thu Dec 7 10:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.60899999999999999,"15min":0.56000000000000005},"min":{"1min":0.41399999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60399999999999998,"15min":0.56399999999999995},"min":{"1min":0.42699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.95299999999999996,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.65600000000000003}]},{"osd":4,"last update":"Thu Dec 7 10:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.98299999999999998}]},{"osd":5,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.47899999999999998,"15min":0.443},"min":{"1min":0.39400000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.77200000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.501,"15min":0.47199999999999998},"min":{"1min":0.379,"5min":0.307,"15min":0.307},"max":{"1min":0.81299999999999994,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Dec 7 10:19:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68000000000000005,"15min":0.67000000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.69199999999999995,"15min":0.66000000000000003},"min":{"1min":0.57699999999999996,"5min":0.5,"15min":0.5},"max":{"1min":0.96699999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.89900000000000002}]},{"osd":7,"last update":"Thu Dec 7 10:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.88800000000000001}]},{"osd":8,"last update":"Thu Dec 7 10:18:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.625,"15min":0.60399999999999998},"min":{"1min":0.53400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.68000000000000005,"15min":0.66400000000000003},"min":{"1min":0.60799999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0229999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.73799999999999999}]},{"osd":9,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79000000000000004,"15min":0.78400000000000003},"min":{"1min":0.64000000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.038,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.873,"5min":0.83899999999999997,"15min":0.83399999999999996},"min":{"1min":0.65500000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0680000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.92500000000000004}]},{"osd":10,"last update":"Thu Dec 7 10:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.76600000000000001,"15min":0.752},"min":{"1min":0.57999999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.77800000000000002,"15min":0.76600000000000001},"min":{"1min":0.57099999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.94599999999999995}]},{"osd":11,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":1.022}]}]},{"osd":3,"up_from":26,"seq":111669149764,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":511605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44900000000000001,"15min":0.40300000000000002},"min":{"1min":0.33500000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.624,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.46700000000000003,"15min":0.41299999999999998},"min":{"1min":0.35599999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.72599999999999998,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.38}]},{"osd":1,"last update":"Thu Dec 7 10:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.68899999999999995,"15min":0.58699999999999997},"min":{"1min":0.55300000000000005,"5min":0.438,"15min":0.438},"max":{"1min":0.98199999999999998,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.56100000000000005},"min":{"1min":0.41899999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89000000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.63700000000000001}]},{"osd":2,"last update":"Thu Dec 7 10:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.59399999999999997},"min":{"1min":0.44600000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86799999999999999,"5min":1.087,"15min":1.087},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68200000000000005,"15min":0.61799999999999999},"min":{"1min":0.53800000000000003,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91500000000000004,"5min":1.177,"15min":1.177},"last":0.64700000000000002}]},{"osd":4,"last update":"Thu Dec 7 10:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.68600000000000005,"15min":0.62},"min":{"1min":0.503,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.018,"5min":1.208,"15min":1.208},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63200000000000001,"15min":0.58999999999999997},"min":{"1min":0.46999999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83899999999999997,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.48399999999999999}]},{"osd":5,"last update":"Thu Dec 7 10:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.623,"15min":0.61099999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.82099999999999995,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.71399999999999997,"15min":0.69599999999999995},"min":{"1min":0.60499999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96199999999999997,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.754}]},{"osd":6,"last update":"Thu Dec 7 10:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65300000000000002,"15min":0.64200000000000002},"min":{"1min":0.40999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":1.179,"5min":1.282,"15min":1.282},"last":0.501},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.60299999999999998,"15min":0.61599999999999999},"min":{"1min":0.40699999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.86599999999999999,"5min":1.292,"15min":1.292},"last":0.53700000000000003}]},{"osd":7,"last update":"Thu Dec 7 10:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78600000000000003,"15min":0.76900000000000002},"min":{"1min":0.59599999999999997,"5min":0.498,"15min":0.498},"max":{"1min":1.0780000000000001,"5min":1.323,"15min":1.323},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.56799999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96999999999999997,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.52100000000000002}]},{"osd":8,"last update":"Thu Dec 7 10:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.872}]},{"osd":9,"last update":"Thu Dec 7 10:18:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.88,"15min":0.88600000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.224,"5min":1.242,"15min":1.242},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.80100000000000005,"15min":0.81000000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.73699999999999999}]},{"osd":10,"last update":"Thu Dec 7 10:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.78200000000000003,"15min":0.79100000000000004},"min":{"1min":0.59099999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.90500000000000003,"5min":1.335,"15min":1.335},"last":0.625},{"interface":"front","average":{"1min":0.878,"5min":0.92100000000000004,"15min":0.92800000000000005},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.1180000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.72199999999999998}]},{"osd":11,"last update":"Thu Dec 7 10:18:41 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.69599999999999995}]}]},{"osd":4,"up_from":34,"seq":146028888128,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":516802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.52100000000000002,"15min":0.49099999999999999},"min":{"1min":0.41499999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76500000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Dec 7 10:18:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.41099999999999998,"15min":0.39000000000000001},"min":{"1min":0.254,"5min":0.215,"15min":0.215},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.41499999999999998,"15min":0.38900000000000001},"min":{"1min":0.308,"5min":0.22,"15min":0.22},"max":{"1min":0.55200000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.38700000000000001}]},{"osd":2,"last update":"Thu Dec 7 10:18:17 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58899999999999997,"15min":0.56000000000000005},"min":{"1min":0.42699999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.875,"5min":1.323,"15min":1.323},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58499999999999996,"15min":0.54300000000000004},"min":{"1min":0.45700000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84299999999999997,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Dec 7 10:18:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58099999999999996,"15min":0.54300000000000004},"min":{"1min":0.437,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.874,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.60899999999999999,"15min":0.56999999999999995},"min":{"1min":0.45700000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91200000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.57999999999999996}]},{"osd":5,"last update":"Thu Dec 7 10:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.63100000000000001,"15min":0.59299999999999997},"min":{"1min":0.55000000000000004,"5min":0.374,"15min":0.374},"max":{"1min":1.0009999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.623},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.61699999999999999,"15min":0.58399999999999996},"min":{"1min":0.47599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82699999999999996,"5min":1.258,"15min":1.258},"last":0.64300000000000002}]},{"osd":6,"last update":"Thu Dec 7 10:19:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65300000000000002,"15min":0.65100000000000002},"min":{"1min":0.434,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90000000000000002,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65200000000000002,"15min":0.63800000000000001},"min":{"1min":0.49399999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85199999999999998,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.48699999999999999}]},{"osd":7,"last update":"Thu Dec 7 10:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.55000000000000004,"15min":0.51800000000000002},"min":{"1min":0.38100000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76800000000000002,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.35599999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.84399999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.46300000000000002}]},{"osd":8,"last update":"Thu Dec 7 10:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.71299999999999997,"15min":0.67900000000000005},"min":{"1min":0.57099999999999995,"5min":0.432,"15min":0.432},"max":{"1min":0.90000000000000002,"5min":1.381,"15min":1.381},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71199999999999997,"15min":0.68200000000000005},"min":{"1min":0.59099999999999997,"5min":0.502,"15min":0.502},"max":{"1min":0.872,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.70099999999999996}]},{"osd":9,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.76100000000000001,"15min":0.76700000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.93500000000000005,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78000000000000003,"15min":0.76100000000000001},"min":{"1min":0.59399999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0289999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.77400000000000002}]},{"osd":10,"last update":"Thu Dec 7 10:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.60999999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.54600000000000004,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.65500000000000003}]},{"osd":11,"last update":"Thu Dec 7 10:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.75600000000000001}]}]},{"osd":5,"up_from":39,"seq":167503724604,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28676,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716476,"statfs":{"total":95995035648,"available":95965671424,"internally_reserved":0,"allocated":1691648,"data_stored":1119172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.54700000000000004,"15min":0.51700000000000002},"min":{"1min":0.377,"5min":0.318,"15min":0.318},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54700000000000004,"15min":0.52200000000000002},"min":{"1min":0.37,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79000000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.67600000000000005}]},{"osd":1,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.56499999999999995,"15min":0.51700000000000002},"min":{"1min":0.505,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.59799999999999998,"15min":0.54400000000000004},"min":{"1min":0.52200000000000002,"5min":0.312,"15min":0.312},"max":{"1min":3.0070000000000001,"5min":3.0070000000000001,"15min":3.0070000000000001},"last":0.58799999999999997}]},{"osd":2,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.42399999999999999,"15min":0.38600000000000001},"min":{"1min":0.40200000000000002,"5min":0.246,"15min":0.246},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.45300000000000001,"15min":0.39900000000000002},"min":{"1min":0.35699999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.851,"5min":1.851,"15min":1.851},"last":0.48599999999999999}]},{"osd":3,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.57999999999999996,"15min":0.53900000000000003},"min":{"1min":0.43099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.59099999999999997,"15min":0.55600000000000005},"min":{"1min":0.40500000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.80300000000000005}]},{"osd":4,"last update":"Thu Dec 7 10:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.63100000000000001,"15min":0.57299999999999995},"min":{"1min":0.58199999999999996,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.621,"15min":0.57899999999999996},"min":{"1min":0.47699999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Dec 7 10:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63800000000000001,"15min":0.59299999999999997},"min":{"1min":0.46200000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86599999999999999,"5min":1.272,"15min":1.272},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.64000000000000001},"min":{"1min":0.56200000000000006,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96099999999999997,"5min":5.9720000000000004,"15min":5.9720000000000004},"last":0.87}]},{"osd":7,"last update":"Thu Dec 7 10:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.66100000000000003,"15min":0.626},"min":{"1min":0.38400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.66600000000000004,"15min":0.63800000000000001},"min":{"1min":0.443,"5min":0.437,"15min":0.437},"max":{"1min":0.96999999999999997,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.70199999999999996}]},{"osd":8,"last update":"Thu Dec 7 10:18:51 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.65000000000000002,"15min":0.628},"min":{"1min":0.45600000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.052,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.61899999999999999,"15min":0.59899999999999998},"min":{"1min":0.5,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92000000000000004,"5min":1.135,"15min":1.135},"last":0.72099999999999997}]},{"osd":9,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.75700000000000001},"min":{"1min":0.56200000000000006,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.98499999999999999,"5min":4.532,"15min":4.532},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77300000000000002,"15min":0.76600000000000001},"min":{"1min":0.55000000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.01,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.85499999999999998}]},{"osd":10,"last update":"Thu Dec 7 10:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.84799999999999998,"15min":0.84099999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.7649999999999999,"5min":2.927,"15min":2.927},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.1519999999999999,"5min":0.85899999999999999,"15min":0.81000000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":5.7709999999999999,"5min":5.7709999999999999,"15min":5.7709999999999999},"last":0.82899999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":76695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":13202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":13912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-07T10:19:20.040 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-12-07T10:19:20.040 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-12-07T10:19:20.040 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-12-07T10:19:20.040 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-07T10:19:20.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:19 smithi119 ceph-mon[97908]: pgmap v332: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-07T10:19:20.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:19 smithi159 ceph-mon[134726]: pgmap v332: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-07T10:19:20.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:19 smithi153 ceph-mon[133625]: pgmap v332: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-07T10:19:20.818 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:19:21.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:20 smithi119 ceph-mon[97908]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:19:21.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:20 smithi159 ceph-mon[134726]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:19:21.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:20 smithi153 ceph-mon[133625]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:19:21.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:21 smithi119 ceph-mon[97908]: pgmap v333: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-07T10:19:22.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:21 smithi159 ceph-mon[134726]: pgmap v333: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-07T10:19:22.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:21 smithi153 ceph-mon[133625]: pgmap v333: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-07T10:19:22.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.359+0000 7f53c92c0700 1 -- 172.21.15.119:0/698788534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53c41016c0 msgr2=0x7f53c4101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:22.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.359+0000 7f53c92c0700 1 --2- 172.21.15.119:0/698788534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53c41016c0 0x7f53c4101b40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f53b8009990 tx=0x7f53b8009ca0 comp rx=0 tx=0).stop 2023-12-07T10:19:22.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.359+0000 7f53c92c0700 1 -- 172.21.15.119:0/698788534 shutdown_connections 2023-12-07T10:19:22.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.359+0000 7f53c92c0700 1 --2- 172.21.15.119:0/698788534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53c4102080 0x7f53c410e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:22.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.359+0000 7f53c92c0700 1 --2- 172.21.15.119:0/698788534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53c4107730 0x7f53c4107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:22.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.359+0000 7f53c92c0700 1 --2- 172.21.15.119:0/698788534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53c41016c0 0x7f53c4101b40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:22.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.359+0000 7f53c92c0700 1 -- 172.21.15.119:0/698788534 >> 172.21.15.119:0/698788534 conn(0x7f53c4078800 msgr2=0x7f53c4078c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:22.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.359+0000 7f53c92c0700 1 -- 172.21.15.119:0/698788534 shutdown_connections 2023-12-07T10:19:22.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.359+0000 7f53c92c0700 1 -- 172.21.15.119:0/698788534 wait complete. 2023-12-07T10:19:22.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.360+0000 7f53c92c0700 1 Processor -- start 2023-12-07T10:19:22.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.361+0000 7f53c92c0700 1 -- start start 2023-12-07T10:19:22.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.361+0000 7f53c92c0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53c41016c0 0x7f53c4073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:22.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.361+0000 7f53c92c0700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53c4102080 0x7f53c4073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:22.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.361+0000 7f53c92c0700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53c4107730 0x7f53c4074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:22.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.361+0000 7f53c92c0700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c41a5df0 con 0x7f53c4102080 2023-12-07T10:19:22.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.361+0000 7f53c92c0700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c41a5f60 con 0x7f53c41016c0 2023-12-07T10:19:22.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.361+0000 7f53c92c0700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c41a6120 con 0x7f53c4107730 2023-12-07T10:19:22.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.361+0000 7f53c27fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53c4102080 0x7f53c4073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:22.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.361+0000 7f53c27fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53c4102080 0x7f53c4073e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60938/0 (socket says 172.21.15.119:60938) 2023-12-07T10:19:22.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.361+0000 7f53c27fc700 1 -- 172.21.15.119:0/3915317344 learned_addr learned my addr 172.21.15.119:0/3915317344 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:22.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.361+0000 7f53c37fe700 1 --2- 172.21.15.119:0/3915317344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53c4107730 0x7f53c4074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:22.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.361+0000 7f53c2ffd700 1 --2- 172.21.15.119:0/3915317344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53c41016c0 0x7f53c4073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:22.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.362+0000 7f53c27fc700 1 -- 172.21.15.119:0/3915317344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53c4107730 msgr2=0x7f53c4074390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:22.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.362+0000 7f53c27fc700 1 --2- 172.21.15.119:0/3915317344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53c4107730 0x7f53c4074390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:22.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.362+0000 7f53c27fc700 1 -- 172.21.15.119:0/3915317344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53c41016c0 msgr2=0x7f53c4073910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:22.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.362+0000 7f53c27fc700 1 --2- 172.21.15.119:0/3915317344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53c41016c0 0x7f53c4073910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:22.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.362+0000 7f53c27fc700 1 -- 172.21.15.119:0/3915317344 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b80095f0 con 0x7f53c4102080 2023-12-07T10:19:22.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.362+0000 7f53c27fc700 1 --2- 172.21.15.119:0/3915317344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53c4102080 0x7f53c4073e50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f53b8005910 tx=0x7f53b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:22.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.362+0000 7f53abfff700 1 -- 172.21.15.119:0/3915317344 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b801d070 con 0x7f53c4102080 2023-12-07T10:19:22.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.362+0000 7f53abfff700 1 -- 172.21.15.119:0/3915317344 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53b8022470 con 0x7f53c4102080 2023-12-07T10:19:22.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.362+0000 7f53c92c0700 1 -- 172.21.15.119:0/3915317344 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c41a63a0 con 0x7f53c4102080 2023-12-07T10:19:22.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.362+0000 7f53abfff700 1 -- 172.21.15.119:0/3915317344 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b8017650 con 0x7f53c4102080 2023-12-07T10:19:22.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.362+0000 7f53c92c0700 1 -- 172.21.15.119:0/3915317344 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c41a67b0 con 0x7f53c4102080 2023-12-07T10:19:22.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.363+0000 7f53c92c0700 1 -- 172.21.15.119:0/3915317344 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c4109bc0 con 0x7f53c4102080 2023-12-07T10:19:22.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.365+0000 7f53abfff700 1 -- 172.21.15.119:0/3915317344 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f53b80225e0 con 0x7f53c4102080 2023-12-07T10:19:22.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.367+0000 7f53abfff700 1 --2- 172.21.15.119:0/3915317344 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f53ac0a1ba0 0x7f53ac0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:22.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.367+0000 7f53abfff700 1 -- 172.21.15.119:0/3915317344 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f53b80cf140 con 0x7f53c4102080 2023-12-07T10:19:22.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.367+0000 7f53c2ffd700 1 --2- 172.21.15.119:0/3915317344 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f53ac0a1ba0 0x7f53ac0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:22.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.367+0000 7f53c2ffd700 1 --2- 172.21.15.119:0/3915317344 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f53ac0a1ba0 0x7f53ac0a4060 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f53b4009be0 tx=0x7f53b4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:22.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.369+0000 7f53abfff700 1 -- 172.21.15.119:0/3915317344 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f53b80980c0 con 0x7f53c4102080 2023-12-07T10:19:22.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.610+0000 7f53c92c0700 1 -- 172.21.15.119:0/3915317344 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f53c4074df0 con 0x7f53c4102080 2023-12-07T10:19:22.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.611+0000 7f53abfff700 1 -- 172.21.15.119:0/3915317344 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f53b809b6e0 con 0x7f53c4102080 2023-12-07T10:19:22.614 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:19:22.614 INFO:teuthology.orchestra.run.smithi119.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-12-07T10:19:22.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.613+0000 7f53c92c0700 1 -- 172.21.15.119:0/3915317344 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f53ac0a1ba0 msgr2=0x7f53ac0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:22.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.613+0000 7f53c92c0700 1 --2- 172.21.15.119:0/3915317344 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f53ac0a1ba0 0x7f53ac0a4060 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f53b4009be0 tx=0x7f53b4009260 comp rx=0 tx=0).stop 2023-12-07T10:19:22.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.613+0000 7f53c92c0700 1 -- 172.21.15.119:0/3915317344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53c4102080 msgr2=0x7f53c4073e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:22.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.613+0000 7f53c92c0700 1 --2- 172.21.15.119:0/3915317344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53c4102080 0x7f53c4073e50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f53b8005910 tx=0x7f53b8004730 comp rx=0 tx=0).stop 2023-12-07T10:19:22.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.613+0000 7f53c92c0700 1 -- 172.21.15.119:0/3915317344 shutdown_connections 2023-12-07T10:19:22.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.613+0000 7f53c92c0700 1 --2- 172.21.15.119:0/3915317344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53c4107730 0x7f53c4074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:22.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.613+0000 7f53c92c0700 1 --2- 172.21.15.119:0/3915317344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53c41016c0 0x7f53c4073910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:22.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.613+0000 7f53c92c0700 1 --2- 172.21.15.119:0/3915317344 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f53ac0a1ba0 0x7f53ac0a4060 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:22.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.613+0000 7f53c92c0700 1 --2- 172.21.15.119:0/3915317344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53c4102080 0x7f53c4073e50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:22.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.613+0000 7f53c92c0700 1 -- 172.21.15.119:0/3915317344 >> 172.21.15.119:0/3915317344 conn(0x7f53c4078800 msgr2=0x7f53c4100390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:22.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.613+0000 7f53c92c0700 1 -- 172.21.15.119:0/3915317344 shutdown_connections 2023-12-07T10:19:22.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:22.613+0000 7f53c92c0700 1 -- 172.21.15.119:0/3915317344 wait complete. 2023-12-07T10:19:23.120 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-12-07T10:19:23.121 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-07T10:19:23.183 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:22 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3915317344' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T10:19:23.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:22 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3915317344' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T10:19:23.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:22 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3915317344' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-07T10:19:23.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.321+0000 7f0ab59ff700 1 -- 172.21.15.119:0/271102714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab0101560 msgr2=0x7f0ab0101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:23.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.321+0000 7f0ab59ff700 1 --2- 172.21.15.119:0/271102714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab0101560 0x7f0ab0101980 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0aa00099c0 tx=0x7f0aa0009cd0 comp rx=0 tx=0).stop 2023-12-07T10:19:23.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.321+0000 7f0ab59ff700 1 -- 172.21.15.119:0/271102714 shutdown_connections 2023-12-07T10:19:23.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.321+0000 7f0ab59ff700 1 --2- 172.21.15.119:0/271102714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ab01030f0 0x7f0ab01099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:23.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.321+0000 7f0ab59ff700 1 --2- 172.21.15.119:0/271102714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab0101560 0x7f0ab0101980 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:23.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.321+0000 7f0ab59ff700 1 --2- 172.21.15.119:0/271102714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ab0102750 0x7f0ab0102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:23.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.321+0000 7f0ab59ff700 1 -- 172.21.15.119:0/271102714 >> 172.21.15.119:0/271102714 conn(0x7f0ab00fcab0 msgr2=0x7f0ab00fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:23.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.321+0000 7f0ab59ff700 1 -- 172.21.15.119:0/271102714 shutdown_connections 2023-12-07T10:19:23.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.321+0000 7f0ab59ff700 1 -- 172.21.15.119:0/271102714 wait complete. 2023-12-07T10:19:23.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.322+0000 7f0ab59ff700 1 Processor -- start 2023-12-07T10:19:23.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.322+0000 7f0ab59ff700 1 -- start start 2023-12-07T10:19:23.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.322+0000 7f0ab59ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ab0101560 0x7f0ab0197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:23.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.322+0000 7f0ab59ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ab0102750 0x7f0ab0197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:23.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.322+0000 7f0ab59ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab01030f0 0x7f0ab019ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:23.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.322+0000 7f0ab59ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab019f040 con 0x7f0ab01030f0 2023-12-07T10:19:23.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.322+0000 7f0ab59ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab019f180 con 0x7f0ab0101560 2023-12-07T10:19:23.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.322+0000 7f0ab59ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab019f310 con 0x7f0ab0102750 2023-12-07T10:19:23.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.323+0000 7f0aaeffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ab0101560 0x7f0ab0197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:23.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.323+0000 7f0aaf7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab01030f0 0x7f0ab019ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:23.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.323+0000 7f0aaeffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ab0101560 0x7f0ab0197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:40720/0 (socket says 172.21.15.119:40720) 2023-12-07T10:19:23.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.323+0000 7f0aaeffd700 1 -- 172.21.15.119:0/4255117389 learned_addr learned my addr 172.21.15.119:0/4255117389 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:23.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.323+0000 7f0aae7fc700 1 --2- 172.21.15.119:0/4255117389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ab0102750 0x7f0ab0197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:23.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.323+0000 7f0aaeffd700 1 -- 172.21.15.119:0/4255117389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ab0102750 msgr2=0x7f0ab0197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:23.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.323+0000 7f0aaeffd700 1 --2- 172.21.15.119:0/4255117389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ab0102750 0x7f0ab0197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:23.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.323+0000 7f0aaeffd700 1 -- 172.21.15.119:0/4255117389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab01030f0 msgr2=0x7f0ab019ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:23.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.323+0000 7f0aaeffd700 1 --2- 172.21.15.119:0/4255117389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab01030f0 0x7f0ab019ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:23.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.323+0000 7f0aaeffd700 1 -- 172.21.15.119:0/4255117389 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa4009520 con 0x7f0ab0101560 2023-12-07T10:19:23.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.323+0000 7f0aaf7fe700 1 --2- 172.21.15.119:0/4255117389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab01030f0 0x7f0ab019ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:23.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.323+0000 7f0aaeffd700 1 --2- 172.21.15.119:0/4255117389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ab0101560 0x7f0ab0197510 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0003730 tx=0x7f0aa000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:23.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.323+0000 7f0ab49fd700 1 -- 172.21.15.119:0/4255117389 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa001d070 con 0x7f0ab0101560 2023-12-07T10:19:23.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.324+0000 7f0ab59ff700 1 -- 172.21.15.119:0/4255117389 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa00095f0 con 0x7f0ab0101560 2023-12-07T10:19:23.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.324+0000 7f0ab49fd700 1 -- 172.21.15.119:0/4255117389 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0aa0005050 con 0x7f0ab0101560 2023-12-07T10:19:23.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.324+0000 7f0ab49fd700 1 -- 172.21.15.119:0/4255117389 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0004a50 con 0x7f0ab0101560 2023-12-07T10:19:23.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.324+0000 7f0ab59ff700 1 -- 172.21.15.119:0/4255117389 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab019f830 con 0x7f0ab0101560 2023-12-07T10:19:23.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.325+0000 7f0ab59ff700 1 -- 172.21.15.119:0/4255117389 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab0108a30 con 0x7f0ab0101560 2023-12-07T10:19:23.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.327+0000 7f0ab49fd700 1 -- 172.21.15.119:0/4255117389 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f0aa00051c0 con 0x7f0ab0101560 2023-12-07T10:19:23.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.328+0000 7f0ab49fd700 1 --2- 172.21.15.119:0/4255117389 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0a980a1ba0 0x7f0a980a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:23.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.328+0000 7f0ab49fd700 1 -- 172.21.15.119:0/4255117389 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f0aa00cec10 con 0x7f0ab0101560 2023-12-07T10:19:23.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.328+0000 7f0aae7fc700 1 --2- 172.21.15.119:0/4255117389 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0a980a1ba0 0x7f0a980a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:23.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.329+0000 7f0aae7fc700 1 --2- 172.21.15.119:0/4255117389 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0a980a1ba0 0x7f0a980a4060 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009df0 tx=0x7f0aa4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:23.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.330+0000 7f0ab49fd700 1 -- 172.21.15.119:0/4255117389 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0aa0097b90 con 0x7f0ab0101560 2023-12-07T10:19:23.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.533+0000 7f0ab59ff700 1 -- 172.21.15.119:0/4255117389 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ab010cdf0 con 0x7f0ab0101560 2023-12-07T10:19:23.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.536+0000 7f0ab49fd700 1 -- 172.21.15.119:0/4255117389 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24715 (secure 0 0 0) 0x7f0aa009b1b0 con 0x7f0ab0101560 2023-12-07T10:19:23.538 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:19:23.538 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":93,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","created":"2023-12-07T10:09:59.945863+0000","modified":"2023-12-07T10:19:02.800626+0000","last_up_change":"2023-12-07T10:16:50.929427+0000","last_in_change":"2023-12-07T10:16:33.558158+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T10:13:25.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-07T10:17:39.347813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-07T10:17:40.714501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-07T10:17:44.257760+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"56b5818e-e625-40f1-8838-3ddcf2d34ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6803","nonce":2141950133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6805","nonce":2141950133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6809","nonce":2141950133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6807","nonce":2141950133}]},"public_addr":"172.21.15.119:6803/2141950133","cluster_addr":"172.21.15.119:6805/2141950133","heartbeat_back_addr":"172.21.15.119:6809/2141950133","heartbeat_front_addr":"172.21.15.119:6807/2141950133","state":["exists","up"]},{"osd":1,"uuid":"36489a6a-5278-4802-9525-9871521722e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6801","nonce":3390875370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6803","nonce":3390875370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6807","nonce":3390875370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6805","nonce":3390875370}]},"public_addr":"172.21.15.153:6801/3390875370","cluster_addr":"172.21.15.153:6803/3390875370","heartbeat_back_addr":"172.21.15.153:6807/3390875370","heartbeat_front_addr":"172.21.15.153:6805/3390875370","state":["exists","up"]},{"osd":2,"uuid":"a937ad26-5e9b-4c8a-b65a-626a8feda6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6801","nonce":708101637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6803","nonce":708101637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6807","nonce":708101637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6805","nonce":708101637}]},"public_addr":"172.21.15.159:6801/708101637","cluster_addr":"172.21.15.159:6803/708101637","heartbeat_back_addr":"172.21.15.159:6807/708101637","heartbeat_front_addr":"172.21.15.159:6805/708101637","state":["exists","up"]},{"osd":3,"uuid":"26ef8300-aa07-48f6-b31d-8b0f2217ffcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6811","nonce":3228047257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6813","nonce":3228047257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6817","nonce":3228047257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6815","nonce":3228047257}]},"public_addr":"172.21.15.119:6811/3228047257","cluster_addr":"172.21.15.119:6813/3228047257","heartbeat_back_addr":"172.21.15.119:6817/3228047257","heartbeat_front_addr":"172.21.15.119:6815/3228047257","state":["exists","up"]},{"osd":4,"uuid":"000ee4b9-9896-4901-8cc3-c85fdfc8cbaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6809","nonce":4021998299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6811","nonce":4021998299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6815","nonce":4021998299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6813","nonce":4021998299}]},"public_addr":"172.21.15.153:6809/4021998299","cluster_addr":"172.21.15.153:6811/4021998299","heartbeat_back_addr":"172.21.15.153:6815/4021998299","heartbeat_front_addr":"172.21.15.153:6813/4021998299","state":["exists","up"]},{"osd":5,"uuid":"8b2fc167-262d-474f-89f3-47bd29424421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6809","nonce":1832540768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6811","nonce":1832540768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6815","nonce":1832540768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6813","nonce":1832540768}]},"public_addr":"172.21.15.159:6809/1832540768","cluster_addr":"172.21.15.159:6811/1832540768","heartbeat_back_addr":"172.21.15.159:6815/1832540768","heartbeat_front_addr":"172.21.15.159:6813/1832540768","state":["exists","up"]},{"osd":6,"uuid":"f9aa6749-46cf-4b2a-8114-cbd034219d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162436840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162436840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6825","nonce":1162436840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162436840}]},"public_addr":"172.21.15.119:6819/1162436840","cluster_addr":"172.21.15.119:6821/1162436840","heartbeat_back_addr":"172.21.15.119:6825/1162436840","heartbeat_front_addr":"172.21.15.119:6823/1162436840","state":["exists","up"]},{"osd":7,"uuid":"39216361-e1c4-4fef-86cb-a4707656b547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6817","nonce":3254653457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6819","nonce":3254653457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6823","nonce":3254653457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6821","nonce":3254653457}]},"public_addr":"172.21.15.153:6817/3254653457","cluster_addr":"172.21.15.153:6819/3254653457","heartbeat_back_addr":"172.21.15.153:6823/3254653457","heartbeat_front_addr":"172.21.15.153:6821/3254653457","state":["exists","up"]},{"osd":8,"uuid":"cceb6ebc-7c02-48ea-9d16-b7a016dd449b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6817","nonce":2600946380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6819","nonce":2600946380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6823","nonce":2600946380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6821","nonce":2600946380}]},"public_addr":"172.21.15.159:6817/2600946380","cluster_addr":"172.21.15.159:6819/2600946380","heartbeat_back_addr":"172.21.15.159:6823/2600946380","heartbeat_front_addr":"172.21.15.159:6821/2600946380","state":["exists","up"]},{"osd":9,"uuid":"f337a2e4-e065-4a9f-91d9-e3615de2d6f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6827","nonce":1940737285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6829","nonce":1940737285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6833","nonce":1940737285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6831","nonce":1940737285}]},"public_addr":"172.21.15.119:6827/1940737285","cluster_addr":"172.21.15.119:6829/1940737285","heartbeat_back_addr":"172.21.15.119:6833/1940737285","heartbeat_front_addr":"172.21.15.119:6831/1940737285","state":["exists","up"]},{"osd":10,"uuid":"3a3bc347-85f4-4878-9797-d9da96137e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6825","nonce":663834864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6827","nonce":663834864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6831","nonce":663834864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6829","nonce":663834864}]},"public_addr":"172.21.15.153:6825/663834864","cluster_addr":"172.21.15.153:6827/663834864","heartbeat_back_addr":"172.21.15.153:6831/663834864","heartbeat_front_addr":"172.21.15.153:6829/663834864","state":["exists","up"]},{"osd":11,"uuid":"a0fc5909-279d-4172-977c-2dc2da0a7e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6825","nonce":1262216348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6827","nonce":1262216348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6831","nonce":1262216348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6829","nonce":1262216348}]},"public_addr":"172.21.15.159:6825/1262216348","cluster_addr":"172.21.15.159:6827/1262216348","heartbeat_back_addr":"172.21.15.159:6831/1262216348","heartbeat_front_addr":"172.21.15.159:6829/1262216348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:37.488109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:59.543060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:21.381585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:47.112729+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:09.732150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:30.893729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:56.678710+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:19.004256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:40.388818+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:06.391734+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:28.394053+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:49.634348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1628639392":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6801/1355617847":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6800/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/362307836":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/128964558":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/2278813554":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/1411933614":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/1685698277":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6800/1355617847":"2023-12-08T10:11:00.585565+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T10:19:23.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.538+0000 7f0ab59ff700 1 -- 172.21.15.119:0/4255117389 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0a980a1ba0 msgr2=0x7f0a980a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:23.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.538+0000 7f0ab59ff700 1 --2- 172.21.15.119:0/4255117389 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0a980a1ba0 0x7f0a980a4060 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009df0 tx=0x7f0aa4009260 comp rx=0 tx=0).stop 2023-12-07T10:19:23.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.538+0000 7f0ab59ff700 1 -- 172.21.15.119:0/4255117389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ab0101560 msgr2=0x7f0ab0197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:23.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.538+0000 7f0ab59ff700 1 --2- 172.21.15.119:0/4255117389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ab0101560 0x7f0ab0197510 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0003730 tx=0x7f0aa000f690 comp rx=0 tx=0).stop 2023-12-07T10:19:23.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.538+0000 7f0ab59ff700 1 -- 172.21.15.119:0/4255117389 shutdown_connections 2023-12-07T10:19:23.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.538+0000 7f0ab59ff700 1 --2- 172.21.15.119:0/4255117389 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0a980a1ba0 0x7f0a980a4060 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:23.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.538+0000 7f0ab59ff700 1 --2- 172.21.15.119:0/4255117389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab01030f0 0x7f0ab019ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:23.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.538+0000 7f0ab59ff700 1 --2- 172.21.15.119:0/4255117389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ab0101560 0x7f0ab0197510 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:23.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.538+0000 7f0ab59ff700 1 --2- 172.21.15.119:0/4255117389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ab0102750 0x7f0ab0197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:23.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.538+0000 7f0ab59ff700 1 -- 172.21.15.119:0/4255117389 >> 172.21.15.119:0/4255117389 conn(0x7f0ab00fcab0 msgr2=0x7f0ab0105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:23.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.538+0000 7f0ab59ff700 1 -- 172.21.15.119:0/4255117389 shutdown_connections 2023-12-07T10:19:23.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.538+0000 7f0ab59ff700 1 -- 172.21.15.119:0/4255117389 wait complete. 2023-12-07T10:19:23.560 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-07T10:13:25.194366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-07T10:17:39.347813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-07T10:17:40.714501+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-07T10:17:44.257760+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-07T10:19:23.561 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-07T10:19:23.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.758+0000 7f3603fff700 1 -- 172.21.15.119:0/810232637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35fc1020f0 msgr2=0x7f35fc1044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:23.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.758+0000 7f3603fff700 1 --2- 172.21.15.119:0/810232637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35fc1020f0 0x7f35fc1044e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009990 tx=0x7f35f8009ca0 comp rx=0 tx=0).stop 2023-12-07T10:19:23.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.758+0000 7f3603fff700 1 -- 172.21.15.119:0/810232637 shutdown_connections 2023-12-07T10:19:23.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.758+0000 7f3603fff700 1 --2- 172.21.15.119:0/810232637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35fc107350 0x7f35fc109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:23.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.758+0000 7f3603fff700 1 --2- 172.21.15.119:0/810232637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35fc1020f0 0x7f35fc1044e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:23.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.758+0000 7f3603fff700 1 --2- 172.21.15.119:0/810232637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35fc104a20 0x7f35fc106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:23.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.758+0000 7f3603fff700 1 -- 172.21.15.119:0/810232637 >> 172.21.15.119:0/810232637 conn(0x7f35fc0fba20 msgr2=0x7f35fc0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:23.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.758+0000 7f3603fff700 1 -- 172.21.15.119:0/810232637 shutdown_connections 2023-12-07T10:19:23.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.758+0000 7f3603fff700 1 -- 172.21.15.119:0/810232637 wait complete. 2023-12-07T10:19:23.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.759+0000 7f3603fff700 1 Processor -- start 2023-12-07T10:19:23.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.759+0000 7f3603fff700 1 -- start start 2023-12-07T10:19:23.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.760+0000 7f3603fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35fc1020f0 0x7f35fc199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:23.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.760+0000 7f3603fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35fc104a20 0x7f35fc199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:23.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.760+0000 7f3603fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35fc107350 0x7f35fc1a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:23.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.760+0000 7f3603fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc1a1260 con 0x7f35fc104a20 2023-12-07T10:19:23.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.760+0000 7f3603fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc1a13a0 con 0x7f35fc1020f0 2023-12-07T10:19:23.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.760+0000 7f3603fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc1a1530 con 0x7f35fc107350 2023-12-07T10:19:23.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.760+0000 7f360159a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35fc104a20 0x7f35fc199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:23.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.760+0000 7f360159a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35fc104a20 0x7f35fc199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51740/0 (socket says 172.21.15.119:51740) 2023-12-07T10:19:23.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.760+0000 7f360159a700 1 -- 172.21.15.119:0/3052987003 learned_addr learned my addr 172.21.15.119:0/3052987003 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:23.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.760+0000 7f3601d9b700 1 --2- 172.21.15.119:0/3052987003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35fc1020f0 0x7f35fc199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:23.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.760+0000 7f360259c700 1 --2- 172.21.15.119:0/3052987003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35fc107350 0x7f35fc1a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:23.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.760+0000 7f360159a700 1 -- 172.21.15.119:0/3052987003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35fc107350 msgr2=0x7f35fc1a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:23.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.760+0000 7f360159a700 1 --2- 172.21.15.119:0/3052987003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35fc107350 0x7f35fc1a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:23.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.760+0000 7f360159a700 1 -- 172.21.15.119:0/3052987003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35fc1020f0 msgr2=0x7f35fc199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:23.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.760+0000 7f360159a700 1 --2- 172.21.15.119:0/3052987003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35fc1020f0 0x7f35fc199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:23.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.760+0000 7f360159a700 1 -- 172.21.15.119:0/3052987003 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f80095f0 con 0x7f35fc104a20 2023-12-07T10:19:23.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.760+0000 7f360259c700 1 --2- 172.21.15.119:0/3052987003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35fc107350 0x7f35fc1a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:19:23.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.760+0000 7f360159a700 1 --2- 172.21.15.119:0/3052987003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35fc104a20 0x7f35fc199c70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00d980 tx=0x7f35ec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:23.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.761+0000 7f35f2ffd700 1 -- 172.21.15.119:0/3052987003 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec004220 con 0x7f35fc104a20 2023-12-07T10:19:23.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.761+0000 7f35f2ffd700 1 -- 172.21.15.119:0/3052987003 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35ec007640 con 0x7f35fc104a20 2023-12-07T10:19:23.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.761+0000 7f35f2ffd700 1 -- 172.21.15.119:0/3052987003 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec005610 con 0x7f35fc104a20 2023-12-07T10:19:23.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.761+0000 7f3603fff700 1 -- 172.21.15.119:0/3052987003 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35fc1a1810 con 0x7f35fc104a20 2023-12-07T10:19:23.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.761+0000 7f3603fff700 1 -- 172.21.15.119:0/3052987003 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35fc1a1d20 con 0x7f35fc104a20 2023-12-07T10:19:23.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.762+0000 7f3603fff700 1 -- 172.21.15.119:0/3052987003 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35fc108ed0 con 0x7f35fc104a20 2023-12-07T10:19:23.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.764+0000 7f35f2ffd700 1 -- 172.21.15.119:0/3052987003 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f35ec004750 con 0x7f35fc104a20 2023-12-07T10:19:23.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.765+0000 7f35f2ffd700 1 --2- 172.21.15.119:0/3052987003 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f35e80a1c20 0x7f35e80a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:23.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.765+0000 7f35f2ffd700 1 -- 172.21.15.119:0/3052987003 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f35ec0cde10 con 0x7f35fc104a20 2023-12-07T10:19:23.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.765+0000 7f3601d9b700 1 --2- 172.21.15.119:0/3052987003 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f35e80a1c20 0x7f35e80a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:23.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.766+0000 7f3601d9b700 1 --2- 172.21.15.119:0/3052987003 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f35e80a1c20 0x7f35e80a40e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f35f800b5c0 tx=0x7f35f8003790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:23.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.767+0000 7f35f2ffd700 1 -- 172.21.15.119:0/3052987003 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f35ec096d00 con 0x7f35fc104a20 2023-12-07T10:19:23.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.977+0000 7f3603fff700 1 -- 172.21.15.119:0/3052987003 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f35fc10ce20 con 0x7f35fc104a20 2023-12-07T10:19:23.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.977+0000 7f35f2ffd700 1 -- 172.21.15.119:0/3052987003 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f35ec0160a0 con 0x7f35fc104a20 2023-12-07T10:19:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-12-07T10:19:23.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.980+0000 7f3603fff700 1 -- 172.21.15.119:0/3052987003 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f35e80a1c20 msgr2=0x7f35e80a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:23.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.980+0000 7f3603fff700 1 --2- 172.21.15.119:0/3052987003 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f35e80a1c20 0x7f35e80a40e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f35f800b5c0 tx=0x7f35f8003790 comp rx=0 tx=0).stop 2023-12-07T10:19:23.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.980+0000 7f3603fff700 1 -- 172.21.15.119:0/3052987003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35fc104a20 msgr2=0x7f35fc199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:23.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.980+0000 7f3603fff700 1 --2- 172.21.15.119:0/3052987003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35fc104a20 0x7f35fc199c70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00d980 tx=0x7f35ec00dc90 comp rx=0 tx=0).stop 2023-12-07T10:19:23.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.980+0000 7f3603fff700 1 -- 172.21.15.119:0/3052987003 shutdown_connections 2023-12-07T10:19:23.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.980+0000 7f3603fff700 1 --2- 172.21.15.119:0/3052987003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35fc107350 0x7f35fc1a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:23.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.980+0000 7f3603fff700 1 --2- 172.21.15.119:0/3052987003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35fc1020f0 0x7f35fc199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:23.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.980+0000 7f3603fff700 1 --2- 172.21.15.119:0/3052987003 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f35e80a1c20 0x7f35e80a40e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:23.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.980+0000 7f3603fff700 1 --2- 172.21.15.119:0/3052987003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35fc104a20 0x7f35fc199c70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:23.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.980+0000 7f3603fff700 1 -- 172.21.15.119:0/3052987003 >> 172.21.15.119:0/3052987003 conn(0x7f35fc0fba20 msgr2=0x7f35fc101860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:23.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.980+0000 7f3603fff700 1 -- 172.21.15.119:0/3052987003 shutdown_connections 2023-12-07T10:19:23.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:23.980+0000 7f3603fff700 1 -- 172.21.15.119:0/3052987003 wait complete. 2023-12-07T10:19:24.003 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-07T10:19:24.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.201+0000 7f928edff700 1 -- 172.21.15.119:0/702814992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92881027b0 msgr2=0x7f9288102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:24.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.201+0000 7f928edff700 1 --2- 172.21.15.119:0/702814992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92881027b0 0x7f9288102c10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f92740099c0 tx=0x7f9274009cd0 comp rx=0 tx=0).stop 2023-12-07T10:19:24.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.201+0000 7f928edff700 1 -- 172.21.15.119:0/702814992 shutdown_connections 2023-12-07T10:19:24.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.201+0000 7f928edff700 1 --2- 172.21.15.119:0/702814992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9288103150 0x7f9288109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:24.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.201+0000 7f928edff700 1 --2- 172.21.15.119:0/702814992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92881015c0 0x7f92881019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:24.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.201+0000 7f928edff700 1 --2- 172.21.15.119:0/702814992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92881027b0 0x7f9288102c10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:24.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.201+0000 7f928edff700 1 -- 172.21.15.119:0/702814992 >> 172.21.15.119:0/702814992 conn(0x7f92880fcb10 msgr2=0x7f92880fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:24.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.201+0000 7f928edff700 1 -- 172.21.15.119:0/702814992 shutdown_connections 2023-12-07T10:19:24.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.201+0000 7f928edff700 1 -- 172.21.15.119:0/702814992 wait complete. 2023-12-07T10:19:24.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.202+0000 7f928edff700 1 Processor -- start 2023-12-07T10:19:24.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.202+0000 7f928edff700 1 -- start start 2023-12-07T10:19:24.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.203+0000 7f928edff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92881015c0 0x7f9288197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:24.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.203+0000 7f928edff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92881027b0 0x7f9288197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:24.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.203+0000 7f928edff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9288103150 0x7f928819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:24.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.203+0000 7f928edff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928819f070 con 0x7f92881015c0 2023-12-07T10:19:24.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.203+0000 7f928edff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928819f1b0 con 0x7f92881027b0 2023-12-07T10:19:24.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.203+0000 7f928edff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928819f340 con 0x7f9288103150 2023-12-07T10:19:24.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.203+0000 7f928cb9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92881015c0 0x7f9288197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:24.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.203+0000 7f928cb9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92881015c0 0x7f9288197540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51760/0 (socket says 172.21.15.119:51760) 2023-12-07T10:19:24.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.203+0000 7f928cb9b700 1 -- 172.21.15.119:0/1922395558 learned_addr learned my addr 172.21.15.119:0/1922395558 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:24.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.203+0000 7f9287fff700 1 --2- 172.21.15.119:0/1922395558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92881027b0 0x7f9288197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:24.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.203+0000 7f928d39c700 1 --2- 172.21.15.119:0/1922395558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9288103150 0x7f928819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:24.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.203+0000 7f928cb9b700 1 -- 172.21.15.119:0/1922395558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9288103150 msgr2=0x7f928819eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:24.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.203+0000 7f928cb9b700 1 --2- 172.21.15.119:0/1922395558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9288103150 0x7f928819eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:24.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.203+0000 7f928cb9b700 1 -- 172.21.15.119:0/1922395558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92881027b0 msgr2=0x7f9288197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:24.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.203+0000 7f928cb9b700 1 --2- 172.21.15.119:0/1922395558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92881027b0 0x7f9288197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:24.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.203+0000 7f928cb9b700 1 -- 172.21.15.119:0/1922395558 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92740095f0 con 0x7f92881015c0 2023-12-07T10:19:24.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.204+0000 7f928cb9b700 1 --2- 172.21.15.119:0/1922395558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92881015c0 0x7f9288197540 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f927c00e970 tx=0x7f927c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:24.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.204+0000 7f9285ffb700 1 -- 172.21.15.119:0/1922395558 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c009790 con 0x7f92881015c0 2023-12-07T10:19:24.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.204+0000 7f928edff700 1 -- 172.21.15.119:0/1922395558 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928819f620 con 0x7f92881015c0 2023-12-07T10:19:24.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.204+0000 7f9285ffb700 1 -- 172.21.15.119:0/1922395558 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f927c00c7e0 con 0x7f92881015c0 2023-12-07T10:19:24.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.204+0000 7f9285ffb700 1 -- 172.21.15.119:0/1922395558 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c003c70 con 0x7f92881015c0 2023-12-07T10:19:24.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.204+0000 7f928edff700 1 -- 172.21.15.119:0/1922395558 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f928819fb30 con 0x7f92881015c0 2023-12-07T10:19:24.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.205+0000 7f928edff700 1 -- 172.21.15.119:0/1922395558 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9288108a90 con 0x7f92881015c0 2023-12-07T10:19:24.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.207+0000 7f9285ffb700 1 -- 172.21.15.119:0/1922395558 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f927c00cc50 con 0x7f92881015c0 2023-12-07T10:19:24.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.208+0000 7f9285ffb700 1 --2- 172.21.15.119:0/1922395558 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f92780a1ba0 0x7f92780a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:24.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.209+0000 7f9285ffb700 1 -- 172.21.15.119:0/1922395558 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f927c014070 con 0x7f92881015c0 2023-12-07T10:19:24.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.209+0000 7f9287fff700 1 --2- 172.21.15.119:0/1922395558 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f92780a1ba0 0x7f92780a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:24.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.209+0000 7f9287fff700 1 --2- 172.21.15.119:0/1922395558 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f92780a1ba0 0x7f92780a4060 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9274006010 tx=0x7f927400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:24.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.212+0000 7f9285ffb700 1 -- 172.21.15.119:0/1922395558 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f927c096be0 con 0x7f92881015c0 2023-12-07T10:19:24.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:23 smithi119 ceph-mon[97908]: pgmap v334: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-07T10:19:24.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:23 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/4255117389' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:19:24.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:23 smithi159 ceph-mon[134726]: pgmap v334: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-07T10:19:24.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:23 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/4255117389' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:19:24.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:23 smithi153 ceph-mon[133625]: pgmap v334: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-07T10:19:24.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:23 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/4255117389' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:19:24.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.413+0000 7f928edff700 1 -- 172.21.15.119:0/1922395558 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f928810ce30 con 0x7f92881015c0 2023-12-07T10:19:24.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.413+0000 7f9285ffb700 1 -- 172.21.15.119:0/1922395558 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f927c003dd0 con 0x7f92881015c0 2023-12-07T10:19:24.416 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-12-07T10:19:24.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.415+0000 7f928edff700 1 -- 172.21.15.119:0/1922395558 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f92780a1ba0 msgr2=0x7f92780a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:24.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.415+0000 7f928edff700 1 --2- 172.21.15.119:0/1922395558 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f92780a1ba0 0x7f92780a4060 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9274006010 tx=0x7f927400b540 comp rx=0 tx=0).stop 2023-12-07T10:19:24.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.415+0000 7f928edff700 1 -- 172.21.15.119:0/1922395558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92881015c0 msgr2=0x7f9288197540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:24.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.415+0000 7f928edff700 1 --2- 172.21.15.119:0/1922395558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92881015c0 0x7f9288197540 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f927c00e970 tx=0x7f927c00ec80 comp rx=0 tx=0).stop 2023-12-07T10:19:24.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.415+0000 7f928edff700 1 -- 172.21.15.119:0/1922395558 shutdown_connections 2023-12-07T10:19:24.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.415+0000 7f928edff700 1 --2- 172.21.15.119:0/1922395558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9288103150 0x7f928819eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:24.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.415+0000 7f928edff700 1 --2- 172.21.15.119:0/1922395558 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f92780a1ba0 0x7f92780a4060 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:24.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.415+0000 7f928edff700 1 --2- 172.21.15.119:0/1922395558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92881015c0 0x7f9288197540 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:24.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.415+0000 7f928edff700 1 --2- 172.21.15.119:0/1922395558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92881027b0 0x7f9288197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:24.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.415+0000 7f928edff700 1 -- 172.21.15.119:0/1922395558 >> 172.21.15.119:0/1922395558 conn(0x7f92880fcb10 msgr2=0x7f9288105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:24.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.416+0000 7f928edff700 1 -- 172.21.15.119:0/1922395558 shutdown_connections 2023-12-07T10:19:24.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.416+0000 7f928edff700 1 -- 172.21.15.119:0/1922395558 wait complete. 2023-12-07T10:19:24.438 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-07T10:19:24.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.637+0000 7f3b6c1ff700 1 -- 172.21.15.119:0/536273042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b641015c0 msgr2=0x7f3b641019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:24.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.637+0000 7f3b6c1ff700 1 --2- 172.21.15.119:0/536273042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b641015c0 0x7f3b641019e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3b600099c0 tx=0x7f3b60009cd0 comp rx=0 tx=0).stop 2023-12-07T10:19:24.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.637+0000 7f3b6c1ff700 1 -- 172.21.15.119:0/536273042 shutdown_connections 2023-12-07T10:19:24.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.637+0000 7f3b6c1ff700 1 --2- 172.21.15.119:0/536273042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b64103150 0x7f3b64109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:24.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.637+0000 7f3b6c1ff700 1 --2- 172.21.15.119:0/536273042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b641015c0 0x7f3b641019e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:24.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.637+0000 7f3b6c1ff700 1 --2- 172.21.15.119:0/536273042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b641027b0 0x7f3b64102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:24.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.637+0000 7f3b6c1ff700 1 -- 172.21.15.119:0/536273042 >> 172.21.15.119:0/536273042 conn(0x7f3b640fcb10 msgr2=0x7f3b640fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:24.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.637+0000 7f3b6c1ff700 1 -- 172.21.15.119:0/536273042 shutdown_connections 2023-12-07T10:19:24.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.637+0000 7f3b6c1ff700 1 -- 172.21.15.119:0/536273042 wait complete. 2023-12-07T10:19:24.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.638+0000 7f3b6c1ff700 1 Processor -- start 2023-12-07T10:19:24.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.638+0000 7f3b6c1ff700 1 -- start start 2023-12-07T10:19:24.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.638+0000 7f3b6c1ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b641015c0 0x7f3b64197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:24.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.638+0000 7f3b6c1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b641027b0 0x7f3b64197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:24.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.638+0000 7f3b6c1ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b64103150 0x7f3b6419eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:24.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.638+0000 7f3b6c1ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6419f080 con 0x7f3b64103150 2023-12-07T10:19:24.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.638+0000 7f3b6c1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6419f1c0 con 0x7f3b641027b0 2023-12-07T10:19:24.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.638+0000 7f3b6c1ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6419f350 con 0x7f3b641015c0 2023-12-07T10:19:24.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.639+0000 7f3b6a79c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b64103150 0x7f3b6419eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:24.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.639+0000 7f3b6a79c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b64103150 0x7f3b6419eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51780/0 (socket says 172.21.15.119:51780) 2023-12-07T10:19:24.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.639+0000 7f3b6a79c700 1 -- 172.21.15.119:0/2424224215 learned_addr learned my addr 172.21.15.119:0/2424224215 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:24.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.639+0000 7f3b6979a700 1 --2- 172.21.15.119:0/2424224215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b641027b0 0x7f3b64197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:24.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.639+0000 7f3b69f9b700 1 --2- 172.21.15.119:0/2424224215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b641015c0 0x7f3b64197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:24.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.639+0000 7f3b6a79c700 1 -- 172.21.15.119:0/2424224215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b641015c0 msgr2=0x7f3b64197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:24.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.639+0000 7f3b6a79c700 1 --2- 172.21.15.119:0/2424224215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b641015c0 0x7f3b64197550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:24.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.639+0000 7f3b6a79c700 1 -- 172.21.15.119:0/2424224215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b641027b0 msgr2=0x7f3b64197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:24.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.639+0000 7f3b6a79c700 1 --2- 172.21.15.119:0/2424224215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b641027b0 0x7f3b64197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:24.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.639+0000 7f3b6a79c700 1 -- 172.21.15.119:0/2424224215 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b600095f0 con 0x7f3b64103150 2023-12-07T10:19:24.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.639+0000 7f3b6a79c700 1 --2- 172.21.15.119:0/2424224215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b64103150 0x7f3b6419eab0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c00ef30 tx=0x7f3b5c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:24.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.640+0000 7f3b5affd700 1 -- 172.21.15.119:0/2424224215 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c018070 con 0x7f3b64103150 2023-12-07T10:19:24.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.640+0000 7f3b5affd700 1 -- 172.21.15.119:0/2424224215 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b5c00f040 con 0x7f3b64103150 2023-12-07T10:19:24.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.640+0000 7f3b6c1ff700 1 -- 172.21.15.119:0/2424224215 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b6419f5d0 con 0x7f3b64103150 2023-12-07T10:19:24.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.640+0000 7f3b5affd700 1 -- 172.21.15.119:0/2424224215 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c00cd00 con 0x7f3b64103150 2023-12-07T10:19:24.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.640+0000 7f3b6c1ff700 1 -- 172.21.15.119:0/2424224215 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b6419fa90 con 0x7f3b64103150 2023-12-07T10:19:24.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.641+0000 7f3b6c1ff700 1 -- 172.21.15.119:0/2424224215 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b64108a90 con 0x7f3b64103150 2023-12-07T10:19:24.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.643+0000 7f3b5affd700 1 -- 172.21.15.119:0/2424224215 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f3b5c003a00 con 0x7f3b64103150 2023-12-07T10:19:24.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.644+0000 7f3b5affd700 1 --2- 172.21.15.119:0/2424224215 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3b500a1ba0 0x7f3b500a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:24.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.644+0000 7f3b5affd700 1 -- 172.21.15.119:0/2424224215 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f3b5c0cdfa0 con 0x7f3b64103150 2023-12-07T10:19:24.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.644+0000 7f3b69f9b700 1 --2- 172.21.15.119:0/2424224215 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3b500a1ba0 0x7f3b500a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:24.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.645+0000 7f3b69f9b700 1 --2- 172.21.15.119:0/2424224215 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3b500a1ba0 0x7f3b500a4060 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3b60003730 tx=0x7f3b60000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:24.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.646+0000 7f3b5affd700 1 -- 172.21.15.119:0/2424224215 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3b5c096f80 con 0x7f3b64103150 2023-12-07T10:19:24.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.851+0000 7f3b6c1ff700 1 -- 172.21.15.119:0/2424224215 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3b6410cf50 con 0x7f3b64103150 2023-12-07T10:19:24.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.851+0000 7f3b5affd700 1 -- 172.21.15.119:0/2424224215 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3b5c09a5a0 con 0x7f3b64103150 2023-12-07T10:19:24.853 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-12-07T10:19:24.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.854+0000 7f3b6c1ff700 1 -- 172.21.15.119:0/2424224215 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3b500a1ba0 msgr2=0x7f3b500a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:24.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.854+0000 7f3b6c1ff700 1 --2- 172.21.15.119:0/2424224215 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3b500a1ba0 0x7f3b500a4060 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3b60003730 tx=0x7f3b60000bc0 comp rx=0 tx=0).stop 2023-12-07T10:19:24.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.854+0000 7f3b6c1ff700 1 -- 172.21.15.119:0/2424224215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b64103150 msgr2=0x7f3b6419eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:24.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.854+0000 7f3b6c1ff700 1 --2- 172.21.15.119:0/2424224215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b64103150 0x7f3b6419eab0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c00ef30 tx=0x7f3b5c00c3a0 comp rx=0 tx=0).stop 2023-12-07T10:19:24.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.854+0000 7f3b6c1ff700 1 -- 172.21.15.119:0/2424224215 shutdown_connections 2023-12-07T10:19:24.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.854+0000 7f3b6c1ff700 1 --2- 172.21.15.119:0/2424224215 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3b500a1ba0 0x7f3b500a4060 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:24.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.854+0000 7f3b6c1ff700 1 --2- 172.21.15.119:0/2424224215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b64103150 0x7f3b6419eab0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:24.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.854+0000 7f3b6c1ff700 1 --2- 172.21.15.119:0/2424224215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b641015c0 0x7f3b64197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:24.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.854+0000 7f3b6c1ff700 1 --2- 172.21.15.119:0/2424224215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b641027b0 0x7f3b64197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:24.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.854+0000 7f3b6c1ff700 1 -- 172.21.15.119:0/2424224215 >> 172.21.15.119:0/2424224215 conn(0x7f3b640fcb10 msgr2=0x7f3b64105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:24.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.854+0000 7f3b6c1ff700 1 -- 172.21.15.119:0/2424224215 shutdown_connections 2023-12-07T10:19:24.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:24.854+0000 7f3b6c1ff700 1 -- 172.21.15.119:0/2424224215 wait complete. 2023-12-07T10:19:24.875 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-07T10:19:25.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.075+0000 7f4ed05ff700 1 -- 172.21.15.119:0/1854738415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ec8103150 msgr2=0x7f4ec8109a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:25.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.075+0000 7f4ed05ff700 1 --2- 172.21.15.119:0/1854738415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ec8103150 0x7f4ec8109a00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000b0d0 tx=0x7f4ec000b3e0 comp rx=0 tx=0).stop 2023-12-07T10:19:25.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.075+0000 7f4ed05ff700 1 -- 172.21.15.119:0/1854738415 shutdown_connections 2023-12-07T10:19:25.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.075+0000 7f4ed05ff700 1 --2- 172.21.15.119:0/1854738415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ec8103150 0x7f4ec8109a00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:25.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.075+0000 7f4ed05ff700 1 --2- 172.21.15.119:0/1854738415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ec81015c0 0x7f4ec81019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:25.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.075+0000 7f4ed05ff700 1 --2- 172.21.15.119:0/1854738415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ec81027b0 0x7f4ec8102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:25.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.075+0000 7f4ed05ff700 1 -- 172.21.15.119:0/1854738415 >> 172.21.15.119:0/1854738415 conn(0x7f4ec80fcb10 msgr2=0x7f4ec80fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:25.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.075+0000 7f4ed05ff700 1 -- 172.21.15.119:0/1854738415 shutdown_connections 2023-12-07T10:19:25.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.075+0000 7f4ed05ff700 1 -- 172.21.15.119:0/1854738415 wait complete. 2023-12-07T10:19:25.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.076+0000 7f4ed05ff700 1 Processor -- start 2023-12-07T10:19:25.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.077+0000 7f4ed05ff700 1 -- start start 2023-12-07T10:19:25.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.077+0000 7f4ed05ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ec81015c0 0x7f4ec8199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:25.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.077+0000 7f4ed05ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ec81027b0 0x7f4ec8199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:25.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.077+0000 7f4ed05ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ec8103150 0x7f4ec81a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:25.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.077+0000 7f4ed05ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec81a13c0 con 0x7f4ec8103150 2023-12-07T10:19:25.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.077+0000 7f4ed05ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec81a1500 con 0x7f4ec81015c0 2023-12-07T10:19:25.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.077+0000 7f4ed05ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec81a1690 con 0x7f4ec81027b0 2023-12-07T10:19:25.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.077+0000 7f4ece39b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ec81015c0 0x7f4ec8199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:25.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.077+0000 7f4ece39b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ec81015c0 0x7f4ec8199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:40802/0 (socket says 172.21.15.119:40802) 2023-12-07T10:19:25.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.077+0000 7f4ece39b700 1 -- 172.21.15.119:0/734271270 learned_addr learned my addr 172.21.15.119:0/734271270 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:25.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.077+0000 7f4eceb9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ec8103150 0x7f4ec81a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:25.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.077+0000 7f4ecdb9a700 1 --2- 172.21.15.119:0/734271270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ec81027b0 0x7f4ec8199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:25.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.078+0000 7f4eceb9c700 1 -- 172.21.15.119:0/734271270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ec81027b0 msgr2=0x7f4ec8199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:25.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.078+0000 7f4eceb9c700 1 --2- 172.21.15.119:0/734271270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ec81027b0 0x7f4ec8199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:25.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.078+0000 7f4eceb9c700 1 -- 172.21.15.119:0/734271270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ec81015c0 msgr2=0x7f4ec8199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:25.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.078+0000 7f4eceb9c700 1 --2- 172.21.15.119:0/734271270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ec81015c0 0x7f4ec8199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:25.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.078+0000 7f4eceb9c700 1 -- 172.21.15.119:0/734271270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec0009d00 con 0x7f4ec8103150 2023-12-07T10:19:25.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.078+0000 7f4ece39b700 1 --2- 172.21.15.119:0/734271270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ec81015c0 0x7f4ec8199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:25.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.078+0000 7f4eceb9c700 1 --2- 172.21.15.119:0/734271270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ec8103150 0x7f4ec81a0df0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0007b60 tx=0x7f4ec0008e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:25.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.078+0000 7f4ebf7fe700 1 -- 172.21.15.119:0/734271270 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec000e040 con 0x7f4ec8103150 2023-12-07T10:19:25.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.078+0000 7f4ebf7fe700 1 -- 172.21.15.119:0/734271270 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ec0005070 con 0x7f4ec8103150 2023-12-07T10:19:25.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.078+0000 7f4ed05ff700 1 -- 172.21.15.119:0/734271270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec81a1910 con 0x7f4ec8103150 2023-12-07T10:19:25.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.078+0000 7f4ebf7fe700 1 -- 172.21.15.119:0/734271270 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec001d6d0 con 0x7f4ec8103150 2023-12-07T10:19:25.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.078+0000 7f4ed05ff700 1 -- 172.21.15.119:0/734271270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec81a1d30 con 0x7f4ec8103150 2023-12-07T10:19:25.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.079+0000 7f4ed05ff700 1 -- 172.21.15.119:0/734271270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec80683c0 con 0x7f4ec8103150 2023-12-07T10:19:25.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.081+0000 7f4ebf7fe700 1 -- 172.21.15.119:0/734271270 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f4ec00051e0 con 0x7f4ec8103150 2023-12-07T10:19:25.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.083+0000 7f4ebf7fe700 1 --2- 172.21.15.119:0/734271270 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4eb40a1ba0 0x7f4eb40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:25.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.083+0000 7f4ebf7fe700 1 -- 172.21.15.119:0/734271270 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f4ec00cea10 con 0x7f4ec8103150 2023-12-07T10:19:25.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.083+0000 7f4ece39b700 1 --2- 172.21.15.119:0/734271270 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4eb40a1ba0 0x7f4eb40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:25.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.083+0000 7f4ece39b700 1 --2- 172.21.15.119:0/734271270 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4eb40a1ba0 0x7f4eb40a4060 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4007fd0 tx=0x7f4ec4007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:25.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.086+0000 7f4ebf7fe700 1 -- 172.21.15.119:0/734271270 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4ec0097a20 con 0x7f4ec8103150 2023-12-07T10:19:25.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:24 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3052987003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:19:25.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:24 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1922395558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:19:25.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:24 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2424224215' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:19:25.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:24 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3052987003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:19:25.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:24 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1922395558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:19:25.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:24 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2424224215' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:19:25.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:24 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3052987003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:19:25.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:24 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1922395558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:19:25.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:24 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2424224215' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:19:25.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.288+0000 7f4ed05ff700 1 -- 172.21.15.119:0/734271270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4ec81a2090 con 0x7f4ec8103150 2023-12-07T10:19:25.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.288+0000 7f4ebf7fe700 1 -- 172.21.15.119:0/734271270 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4ec009b040 con 0x7f4ec8103150 2023-12-07T10:19:25.290 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-12-07T10:19:25.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.291+0000 7f4ed05ff700 1 -- 172.21.15.119:0/734271270 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4eb40a1ba0 msgr2=0x7f4eb40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:25.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.291+0000 7f4ed05ff700 1 --2- 172.21.15.119:0/734271270 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4eb40a1ba0 0x7f4eb40a4060 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4007fd0 tx=0x7f4ec4007480 comp rx=0 tx=0).stop 2023-12-07T10:19:25.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.291+0000 7f4ed05ff700 1 -- 172.21.15.119:0/734271270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ec8103150 msgr2=0x7f4ec81a0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:25.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.291+0000 7f4ed05ff700 1 --2- 172.21.15.119:0/734271270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ec8103150 0x7f4ec81a0df0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0007b60 tx=0x7f4ec0008e90 comp rx=0 tx=0).stop 2023-12-07T10:19:25.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.291+0000 7f4ed05ff700 1 -- 172.21.15.119:0/734271270 shutdown_connections 2023-12-07T10:19:25.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.291+0000 7f4ed05ff700 1 --2- 172.21.15.119:0/734271270 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4eb40a1ba0 0x7f4eb40a4060 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:25.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.291+0000 7f4ed05ff700 1 --2- 172.21.15.119:0/734271270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ec8103150 0x7f4ec81a0df0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:25.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.291+0000 7f4ed05ff700 1 --2- 172.21.15.119:0/734271270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ec81015c0 0x7f4ec8199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:25.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.291+0000 7f4ed05ff700 1 --2- 172.21.15.119:0/734271270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ec81027b0 0x7f4ec8199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:25.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.291+0000 7f4ed05ff700 1 -- 172.21.15.119:0/734271270 >> 172.21.15.119:0/734271270 conn(0x7f4ec80fcb10 msgr2=0x7f4ec8073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:25.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.291+0000 7f4ed05ff700 1 -- 172.21.15.119:0/734271270 shutdown_connections 2023-12-07T10:19:25.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.291+0000 7f4ed05ff700 1 -- 172.21.15.119:0/734271270 wait complete. 2023-12-07T10:19:25.311 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:19:25.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.507+0000 7fcb5bbff700 1 -- 172.21.15.119:0/1680848668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb541030f0 msgr2=0x7fcb541099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:25.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.507+0000 7fcb5bbff700 1 --2- 172.21.15.119:0/1680848668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb541030f0 0x7fcb541099a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00b0d0 tx=0x7fcb4c00b3e0 comp rx=0 tx=0).stop 2023-12-07T10:19:25.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.507+0000 7fcb5bbff700 1 -- 172.21.15.119:0/1680848668 shutdown_connections 2023-12-07T10:19:25.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.507+0000 7fcb5bbff700 1 --2- 172.21.15.119:0/1680848668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb541030f0 0x7fcb541099a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:25.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.507+0000 7fcb5bbff700 1 --2- 172.21.15.119:0/1680848668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb54101560 0x7fcb54101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:25.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.507+0000 7fcb5bbff700 1 --2- 172.21.15.119:0/1680848668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb54102750 0x7fcb54102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:25.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.507+0000 7fcb5bbff700 1 -- 172.21.15.119:0/1680848668 >> 172.21.15.119:0/1680848668 conn(0x7fcb540fcab0 msgr2=0x7fcb540fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:25.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.507+0000 7fcb5bbff700 1 -- 172.21.15.119:0/1680848668 shutdown_connections 2023-12-07T10:19:25.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.507+0000 7fcb5bbff700 1 -- 172.21.15.119:0/1680848668 wait complete. 2023-12-07T10:19:25.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.508+0000 7fcb5bbff700 1 Processor -- start 2023-12-07T10:19:25.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.508+0000 7fcb5bbff700 1 -- start start 2023-12-07T10:19:25.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.508+0000 7fcb5bbff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb54101560 0x7fcb541997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:25.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.508+0000 7fcb5bbff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb54102750 0x7fcb54199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:25.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.508+0000 7fcb5bbff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb541030f0 0x7fcb541a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:25.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.508+0000 7fcb5bbff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb541a1310 con 0x7fcb541030f0 2023-12-07T10:19:25.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.508+0000 7fcb5bbff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb541a1450 con 0x7fcb54102750 2023-12-07T10:19:25.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.508+0000 7fcb5bbff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb541a15e0 con 0x7fcb54101560 2023-12-07T10:19:25.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.509+0000 7fcb5a19c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb541030f0 0x7fcb541a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:25.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.509+0000 7fcb5919a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb54102750 0x7fcb54199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:25.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.509+0000 7fcb5a19c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb541030f0 0x7fcb541a0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51824/0 (socket says 172.21.15.119:51824) 2023-12-07T10:19:25.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.509+0000 7fcb5a19c700 1 -- 172.21.15.119:0/2213222486 learned_addr learned my addr 172.21.15.119:0/2213222486 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:25.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.509+0000 7fcb5999b700 1 --2- 172.21.15.119:0/2213222486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb54101560 0x7fcb541997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:25.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.509+0000 7fcb5919a700 1 -- 172.21.15.119:0/2213222486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb54101560 msgr2=0x7fcb541997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:25.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.509+0000 7fcb5919a700 1 --2- 172.21.15.119:0/2213222486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb54101560 0x7fcb541997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:25.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.509+0000 7fcb5919a700 1 -- 172.21.15.119:0/2213222486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb541030f0 msgr2=0x7fcb541a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:25.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.509+0000 7fcb5919a700 1 --2- 172.21.15.119:0/2213222486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb541030f0 0x7fcb541a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:25.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.509+0000 7fcb5919a700 1 -- 172.21.15.119:0/2213222486 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb50009520 con 0x7fcb54102750 2023-12-07T10:19:25.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.509+0000 7fcb5a19c700 1 --2- 172.21.15.119:0/2213222486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb541030f0 0x7fcb541a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:25.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.509+0000 7fcb5919a700 1 --2- 172.21.15.119:0/2213222486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb54102750 0x7fcb54199d20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400e9f0 tx=0x7fcb4400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:25.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.510+0000 7fcb4affd700 1 -- 172.21.15.119:0/2213222486 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4400c9c0 con 0x7fcb54102750 2023-12-07T10:19:25.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.510+0000 7fcb5bbff700 1 -- 172.21.15.119:0/2213222486 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb4c009d00 con 0x7fcb54102750 2023-12-07T10:19:25.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.510+0000 7fcb4affd700 1 -- 172.21.15.119:0/2213222486 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb44010430 con 0x7fcb54102750 2023-12-07T10:19:25.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.510+0000 7fcb4affd700 1 -- 172.21.15.119:0/2213222486 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb44018680 con 0x7fcb54102750 2023-12-07T10:19:25.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.510+0000 7fcb5bbff700 1 -- 172.21.15.119:0/2213222486 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb541a1c90 con 0x7fcb54102750 2023-12-07T10:19:25.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.511+0000 7fcb5bbff700 1 -- 172.21.15.119:0/2213222486 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb54068350 con 0x7fcb54102750 2023-12-07T10:19:25.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.513+0000 7fcb4affd700 1 -- 172.21.15.119:0/2213222486 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7fcb44010860 con 0x7fcb54102750 2023-12-07T10:19:25.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.514+0000 7fcb4affd700 1 --2- 172.21.15.119:0/2213222486 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fcb400a1ba0 0x7fcb400a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:25.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.514+0000 7fcb4affd700 1 -- 172.21.15.119:0/2213222486 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fcb44014070 con 0x7fcb54102750 2023-12-07T10:19:25.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.514+0000 7fcb5999b700 1 --2- 172.21.15.119:0/2213222486 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fcb400a1ba0 0x7fcb400a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:25.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.515+0000 7fcb5999b700 1 --2- 172.21.15.119:0/2213222486 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fcb400a1ba0 0x7fcb400a4060 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcb541025b0 tx=0x7fcb50009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:25.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.517+0000 7fcb4affd700 1 -- 172.21.15.119:0/2213222486 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fcb44096cd0 con 0x7fcb54102750 2023-12-07T10:19:25.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.768+0000 7fcb5bbff700 1 -- 172.21.15.119:0/2213222486 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb5410cdd0 con 0x7fcb54102750 2023-12-07T10:19:25.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.770+0000 7fcb4affd700 1 -- 172.21.15.119:0/2213222486 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11385 (secure 0 0 0) 0x7fcb4409a2f0 con 0x7fcb54102750 2023-12-07T10:19:25.771 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:19:25.772 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-07T10:17:41.278311+0000","modified":"2023-12-07T10:18:18.543356+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24517,"mds_3":24505,"mds_4":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14535":{"gid":14535,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6841/1791998028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1791998028},{"type":"v1","addr":"172.21.15.119:6841","nonce":1791998028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24505":{"gid":24505,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/2925503246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2925503246},{"type":"v1","addr":"172.21.15.153:6837","nonce":2925503246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6839/3984962900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3984962900},{"type":"v1","addr":"172.21.15.159:6839","nonce":3984962900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.119:6839/1920762602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1920762602},{"type":"v1","addr":"172.21.15.119:6839","nonce":1920762602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/566825184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":566825184},{"type":"v1","addr":"172.21.15.153:6839","nonce":566825184}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-07T10:19:25.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.772+0000 7fcb5bbff700 1 -- 172.21.15.119:0/2213222486 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fcb400a1ba0 msgr2=0x7fcb400a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:25.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.772+0000 7fcb5bbff700 1 --2- 172.21.15.119:0/2213222486 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fcb400a1ba0 0x7fcb400a4060 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcb541025b0 tx=0x7fcb50009260 comp rx=0 tx=0).stop 2023-12-07T10:19:25.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.772+0000 7fcb5bbff700 1 -- 172.21.15.119:0/2213222486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb54102750 msgr2=0x7fcb54199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:25.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.772+0000 7fcb5bbff700 1 --2- 172.21.15.119:0/2213222486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb54102750 0x7fcb54199d20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400e9f0 tx=0x7fcb4400ed00 comp rx=0 tx=0).stop 2023-12-07T10:19:25.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.773+0000 7fcb5bbff700 1 -- 172.21.15.119:0/2213222486 shutdown_connections 2023-12-07T10:19:25.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.773+0000 7fcb5bbff700 1 --2- 172.21.15.119:0/2213222486 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fcb400a1ba0 0x7fcb400a4060 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:25.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.773+0000 7fcb5bbff700 1 --2- 172.21.15.119:0/2213222486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb541030f0 0x7fcb541a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:25.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.773+0000 7fcb5bbff700 1 --2- 172.21.15.119:0/2213222486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb54101560 0x7fcb541997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:25.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.773+0000 7fcb5bbff700 1 --2- 172.21.15.119:0/2213222486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb54102750 0x7fcb54199d20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:25.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.773+0000 7fcb5bbff700 1 -- 172.21.15.119:0/2213222486 >> 172.21.15.119:0/2213222486 conn(0x7fcb540fcab0 msgr2=0x7fcb54073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:25.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.773+0000 7fcb5bbff700 1 -- 172.21.15.119:0/2213222486 shutdown_connections 2023-12-07T10:19:25.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.773+0000 7fcb5bbff700 1 -- 172.21.15.119:0/2213222486 wait complete. 2023-12-07T10:19:25.776 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 29 2023-12-07T10:19:25.793 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-07T10:19:25.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.990+0000 7f9400dff700 1 -- 172.21.15.119:0/1171892132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93fc1030f0 msgr2=0x7f93fc1099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:25.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.990+0000 7f9400dff700 1 --2- 172.21.15.119:0/1171892132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93fc1030f0 0x7f93fc1099a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f93f400b0d0 tx=0x7f93f400b3e0 comp rx=0 tx=0).stop 2023-12-07T10:19:25.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.990+0000 7f9400dff700 1 -- 172.21.15.119:0/1171892132 shutdown_connections 2023-12-07T10:19:25.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.990+0000 7f9400dff700 1 --2- 172.21.15.119:0/1171892132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93fc1030f0 0x7f93fc1099a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:25.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.990+0000 7f9400dff700 1 --2- 172.21.15.119:0/1171892132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93fc101560 0x7f93fc101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:25.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.990+0000 7f9400dff700 1 --2- 172.21.15.119:0/1171892132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93fc102750 0x7f93fc102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:25.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.990+0000 7f9400dff700 1 -- 172.21.15.119:0/1171892132 >> 172.21.15.119:0/1171892132 conn(0x7f93fc0fcab0 msgr2=0x7f93fc0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:25.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.990+0000 7f9400dff700 1 -- 172.21.15.119:0/1171892132 shutdown_connections 2023-12-07T10:19:25.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.990+0000 7f9400dff700 1 -- 172.21.15.119:0/1171892132 wait complete. 2023-12-07T10:19:25.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.991+0000 7f9400dff700 1 Processor -- start 2023-12-07T10:19:25.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.991+0000 7f9400dff700 1 -- start start 2023-12-07T10:19:25.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.991+0000 7f9400dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93fc101560 0x7f93fc199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:25.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.991+0000 7f9400dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93fc102750 0x7f93fc199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:25.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.991+0000 7f9400dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93fc1030f0 0x7f93fc1a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:25.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.991+0000 7f9400dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc1a1330 con 0x7f93fc102750 2023-12-07T10:19:25.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.991+0000 7f9400dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc1a1470 con 0x7f93fc101560 2023-12-07T10:19:25.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.991+0000 7f9400dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc1a1600 con 0x7f93fc1030f0 2023-12-07T10:19:25.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.991+0000 7f93f3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93fc102750 0x7f93fc199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:25.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.991+0000 7f93fa59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93fc101560 0x7f93fc199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:25.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.992+0000 7f93f3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93fc102750 0x7f93fc199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51856/0 (socket says 172.21.15.119:51856) 2023-12-07T10:19:25.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.992+0000 7f93f3fff700 1 -- 172.21.15.119:0/3004780384 learned_addr learned my addr 172.21.15.119:0/3004780384 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:25.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.992+0000 7f93fad9d700 1 --2- 172.21.15.119:0/3004780384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93fc1030f0 0x7f93fc1a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:25.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.992+0000 7f93f3fff700 1 -- 172.21.15.119:0/3004780384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93fc1030f0 msgr2=0x7f93fc1a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:25.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.992+0000 7f93f3fff700 1 --2- 172.21.15.119:0/3004780384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93fc1030f0 0x7f93fc1a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:25.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.992+0000 7f93f3fff700 1 -- 172.21.15.119:0/3004780384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93fc101560 msgr2=0x7f93fc199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:25.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.992+0000 7f93f3fff700 1 --2- 172.21.15.119:0/3004780384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93fc101560 0x7f93fc199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:25.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.992+0000 7f93f3fff700 1 -- 172.21.15.119:0/3004780384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f4009d00 con 0x7f93fc102750 2023-12-07T10:19:25.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.992+0000 7f93fad9d700 1 --2- 172.21.15.119:0/3004780384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93fc1030f0 0x7f93fc1a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:19:25.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.992+0000 7f93f3fff700 1 --2- 172.21.15.119:0/3004780384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93fc102750 0x7f93fc199d40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f93e400e9a0 tx=0x7f93e400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:25.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.992+0000 7f93f37fe700 1 -- 172.21.15.119:0/3004780384 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e4009790 con 0x7f93fc102750 2023-12-07T10:19:25.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.992+0000 7f93f37fe700 1 -- 172.21.15.119:0/3004780384 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93e400cbd0 con 0x7f93fc102750 2023-12-07T10:19:25.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.992+0000 7f9400dff700 1 -- 172.21.15.119:0/3004780384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93fc1a18e0 con 0x7f93fc102750 2023-12-07T10:19:25.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.992+0000 7f93f37fe700 1 -- 172.21.15.119:0/3004780384 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e4018680 con 0x7f93fc102750 2023-12-07T10:19:25.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.993+0000 7f9400dff700 1 -- 172.21.15.119:0/3004780384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93fc1a1df0 con 0x7f93fc102750 2023-12-07T10:19:25.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.993+0000 7f9400dff700 1 -- 172.21.15.119:0/3004780384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93fc068350 con 0x7f93fc102750 2023-12-07T10:19:25.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.995+0000 7f93f37fe700 1 -- 172.21.15.119:0/3004780384 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f93e400cd40 con 0x7f93fc102750 2023-12-07T10:19:25.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.997+0000 7f93f37fe700 1 --2- 172.21.15.119:0/3004780384 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f93e80a1ba0 0x7f93e80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:25.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.998+0000 7f93f37fe700 1 -- 172.21.15.119:0/3004780384 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f93e4014070 con 0x7f93fc102750 2023-12-07T10:19:25.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.998+0000 7f93fa59c700 1 --2- 172.21.15.119:0/3004780384 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f93e80a1ba0 0x7f93e80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:25.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:25.998+0000 7f93fa59c700 1 --2- 172.21.15.119:0/3004780384 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f93e80a1ba0 0x7f93e80a4060 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f93fc1025b0 tx=0x7f93ec007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:26.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.002+0000 7f93f37fe700 1 -- 172.21.15.119:0/3004780384 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f93e40522f0 con 0x7f93fc102750 2023-12-07T10:19:26.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.203+0000 7f9400dff700 1 -- 172.21.15.119:0/3004780384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93fc10cff0 con 0x7f93fc102750 2023-12-07T10:19:26.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.204+0000 7f93f37fe700 1 -- 172.21.15.119:0/3004780384 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24715 (secure 0 0 0) 0x7f93e4096c00 con 0x7f93fc102750 2023-12-07T10:19:26.206 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:19:26.207 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":93,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","created":"2023-12-07T10:09:59.945863+0000","modified":"2023-12-07T10:19:02.800626+0000","last_up_change":"2023-12-07T10:16:50.929427+0000","last_in_change":"2023-12-07T10:16:33.558158+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T10:13:25.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-07T10:17:39.347813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-07T10:17:40.714501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-07T10:17:44.257760+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"56b5818e-e625-40f1-8838-3ddcf2d34ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6803","nonce":2141950133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6805","nonce":2141950133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6809","nonce":2141950133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6807","nonce":2141950133}]},"public_addr":"172.21.15.119:6803/2141950133","cluster_addr":"172.21.15.119:6805/2141950133","heartbeat_back_addr":"172.21.15.119:6809/2141950133","heartbeat_front_addr":"172.21.15.119:6807/2141950133","state":["exists","up"]},{"osd":1,"uuid":"36489a6a-5278-4802-9525-9871521722e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6801","nonce":3390875370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6803","nonce":3390875370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6807","nonce":3390875370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6805","nonce":3390875370}]},"public_addr":"172.21.15.153:6801/3390875370","cluster_addr":"172.21.15.153:6803/3390875370","heartbeat_back_addr":"172.21.15.153:6807/3390875370","heartbeat_front_addr":"172.21.15.153:6805/3390875370","state":["exists","up"]},{"osd":2,"uuid":"a937ad26-5e9b-4c8a-b65a-626a8feda6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6801","nonce":708101637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6803","nonce":708101637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6807","nonce":708101637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6805","nonce":708101637}]},"public_addr":"172.21.15.159:6801/708101637","cluster_addr":"172.21.15.159:6803/708101637","heartbeat_back_addr":"172.21.15.159:6807/708101637","heartbeat_front_addr":"172.21.15.159:6805/708101637","state":["exists","up"]},{"osd":3,"uuid":"26ef8300-aa07-48f6-b31d-8b0f2217ffcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6811","nonce":3228047257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6813","nonce":3228047257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6817","nonce":3228047257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6815","nonce":3228047257}]},"public_addr":"172.21.15.119:6811/3228047257","cluster_addr":"172.21.15.119:6813/3228047257","heartbeat_back_addr":"172.21.15.119:6817/3228047257","heartbeat_front_addr":"172.21.15.119:6815/3228047257","state":["exists","up"]},{"osd":4,"uuid":"000ee4b9-9896-4901-8cc3-c85fdfc8cbaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6809","nonce":4021998299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6811","nonce":4021998299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6815","nonce":4021998299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6813","nonce":4021998299}]},"public_addr":"172.21.15.153:6809/4021998299","cluster_addr":"172.21.15.153:6811/4021998299","heartbeat_back_addr":"172.21.15.153:6815/4021998299","heartbeat_front_addr":"172.21.15.153:6813/4021998299","state":["exists","up"]},{"osd":5,"uuid":"8b2fc167-262d-474f-89f3-47bd29424421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6809","nonce":1832540768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6811","nonce":1832540768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6815","nonce":1832540768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6813","nonce":1832540768}]},"public_addr":"172.21.15.159:6809/1832540768","cluster_addr":"172.21.15.159:6811/1832540768","heartbeat_back_addr":"172.21.15.159:6815/1832540768","heartbeat_front_addr":"172.21.15.159:6813/1832540768","state":["exists","up"]},{"osd":6,"uuid":"f9aa6749-46cf-4b2a-8114-cbd034219d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162436840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162436840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6825","nonce":1162436840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162436840}]},"public_addr":"172.21.15.119:6819/1162436840","cluster_addr":"172.21.15.119:6821/1162436840","heartbeat_back_addr":"172.21.15.119:6825/1162436840","heartbeat_front_addr":"172.21.15.119:6823/1162436840","state":["exists","up"]},{"osd":7,"uuid":"39216361-e1c4-4fef-86cb-a4707656b547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6817","nonce":3254653457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6819","nonce":3254653457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6823","nonce":3254653457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6821","nonce":3254653457}]},"public_addr":"172.21.15.153:6817/3254653457","cluster_addr":"172.21.15.153:6819/3254653457","heartbeat_back_addr":"172.21.15.153:6823/3254653457","heartbeat_front_addr":"172.21.15.153:6821/3254653457","state":["exists","up"]},{"osd":8,"uuid":"cceb6ebc-7c02-48ea-9d16-b7a016dd449b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6817","nonce":2600946380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6819","nonce":2600946380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6823","nonce":2600946380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6821","nonce":2600946380}]},"public_addr":"172.21.15.159:6817/2600946380","cluster_addr":"172.21.15.159:6819/2600946380","heartbeat_back_addr":"172.21.15.159:6823/2600946380","heartbeat_front_addr":"172.21.15.159:6821/2600946380","state":["exists","up"]},{"osd":9,"uuid":"f337a2e4-e065-4a9f-91d9-e3615de2d6f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6827","nonce":1940737285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6829","nonce":1940737285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6833","nonce":1940737285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6831","nonce":1940737285}]},"public_addr":"172.21.15.119:6827/1940737285","cluster_addr":"172.21.15.119:6829/1940737285","heartbeat_back_addr":"172.21.15.119:6833/1940737285","heartbeat_front_addr":"172.21.15.119:6831/1940737285","state":["exists","up"]},{"osd":10,"uuid":"3a3bc347-85f4-4878-9797-d9da96137e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6825","nonce":663834864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6827","nonce":663834864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6831","nonce":663834864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6829","nonce":663834864}]},"public_addr":"172.21.15.153:6825/663834864","cluster_addr":"172.21.15.153:6827/663834864","heartbeat_back_addr":"172.21.15.153:6831/663834864","heartbeat_front_addr":"172.21.15.153:6829/663834864","state":["exists","up"]},{"osd":11,"uuid":"a0fc5909-279d-4172-977c-2dc2da0a7e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6825","nonce":1262216348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6827","nonce":1262216348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6831","nonce":1262216348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6829","nonce":1262216348}]},"public_addr":"172.21.15.159:6825/1262216348","cluster_addr":"172.21.15.159:6827/1262216348","heartbeat_back_addr":"172.21.15.159:6831/1262216348","heartbeat_front_addr":"172.21.15.159:6829/1262216348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:37.488109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:59.543060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:21.381585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:47.112729+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:09.732150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:30.893729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:56.678710+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:19.004256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:40.388818+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:06.391734+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:28.394053+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:49.634348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1628639392":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6801/1355617847":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6800/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/362307836":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/128964558":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/2278813554":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/1411933614":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/1685698277":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6800/1355617847":"2023-12-08T10:11:00.585565+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T10:19:26.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.207+0000 7f9400dff700 1 -- 172.21.15.119:0/3004780384 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f93e80a1ba0 msgr2=0x7f93e80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:26.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.207+0000 7f9400dff700 1 --2- 172.21.15.119:0/3004780384 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f93e80a1ba0 0x7f93e80a4060 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f93fc1025b0 tx=0x7f93ec007400 comp rx=0 tx=0).stop 2023-12-07T10:19:26.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.207+0000 7f9400dff700 1 -- 172.21.15.119:0/3004780384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93fc102750 msgr2=0x7f93fc199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:26.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.207+0000 7f9400dff700 1 --2- 172.21.15.119:0/3004780384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93fc102750 0x7f93fc199d40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f93e400e9a0 tx=0x7f93e400ecb0 comp rx=0 tx=0).stop 2023-12-07T10:19:26.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.207+0000 7f9400dff700 1 -- 172.21.15.119:0/3004780384 shutdown_connections 2023-12-07T10:19:26.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.207+0000 7f9400dff700 1 --2- 172.21.15.119:0/3004780384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93fc1030f0 0x7f93fc1a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:26.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.207+0000 7f9400dff700 1 --2- 172.21.15.119:0/3004780384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93fc101560 0x7f93fc199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:26.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.207+0000 7f9400dff700 1 --2- 172.21.15.119:0/3004780384 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f93e80a1ba0 0x7f93e80a4060 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:26.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.207+0000 7f9400dff700 1 --2- 172.21.15.119:0/3004780384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93fc102750 0x7f93fc199d40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:26.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.207+0000 7f9400dff700 1 -- 172.21.15.119:0/3004780384 >> 172.21.15.119:0/3004780384 conn(0x7f93fc0fcab0 msgr2=0x7f93fc073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:26.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.207+0000 7f9400dff700 1 -- 172.21.15.119:0/3004780384 shutdown_connections 2023-12-07T10:19:26.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.207+0000 7f9400dff700 1 -- 172.21.15.119:0/3004780384 wait complete. 2023-12-07T10:19:26.225 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:25 smithi119 ceph-mon[97908]: pgmap v335: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-12-07T10:19:26.225 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:25 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/734271270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:19:26.225 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:25 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2213222486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:19:26.228 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-07T10:13:25.194366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-07T10:17:39.347813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-07T10:17:40.714501+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-07T10:17:44.257760+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-07T10:19:26.228 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-07T10:19:26.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:25 smithi159 ceph-mon[134726]: pgmap v335: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-12-07T10:19:26.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:25 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/734271270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:19:26.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:25 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2213222486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:19:26.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:25 smithi153 ceph-mon[133625]: pgmap v335: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-12-07T10:19:26.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:25 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/734271270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:19:26.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:25 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2213222486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:19:26.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.426+0000 7f5b48fff700 1 -- 172.21.15.119:0/3409996858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b441027b0 msgr2=0x7f5b44102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:26.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.426+0000 7f5b48fff700 1 --2- 172.21.15.119:0/3409996858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b441027b0 0x7f5b44102c10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5b380099c0 tx=0x7f5b38009cd0 comp rx=0 tx=0).stop 2023-12-07T10:19:26.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.426+0000 7f5b48fff700 1 -- 172.21.15.119:0/3409996858 shutdown_connections 2023-12-07T10:19:26.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.426+0000 7f5b48fff700 1 --2- 172.21.15.119:0/3409996858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b44103150 0x7f5b44109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:26.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.426+0000 7f5b48fff700 1 --2- 172.21.15.119:0/3409996858 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b441015c0 0x7f5b441019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:26.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.426+0000 7f5b48fff700 1 --2- 172.21.15.119:0/3409996858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b441027b0 0x7f5b44102c10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:26.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.426+0000 7f5b48fff700 1 -- 172.21.15.119:0/3409996858 >> 172.21.15.119:0/3409996858 conn(0x7f5b440fcb10 msgr2=0x7f5b440fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:26.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.426+0000 7f5b48fff700 1 -- 172.21.15.119:0/3409996858 shutdown_connections 2023-12-07T10:19:26.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.426+0000 7f5b48fff700 1 -- 172.21.15.119:0/3409996858 wait complete. 2023-12-07T10:19:26.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.427+0000 7f5b48fff700 1 Processor -- start 2023-12-07T10:19:26.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.427+0000 7f5b48fff700 1 -- start start 2023-12-07T10:19:26.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.427+0000 7f5b48fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b441015c0 0x7f5b441997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:26.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.427+0000 7f5b48fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b441027b0 0x7f5b44199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:26.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.427+0000 7f5b48fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b44103150 0x7f5b441a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:26.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.428+0000 7f5b48fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b441a1280 con 0x7f5b441027b0 2023-12-07T10:19:26.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.428+0000 7f5b48fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b441a13c0 con 0x7f5b44103150 2023-12-07T10:19:26.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.428+0000 7f5b48fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b441a1550 con 0x7f5b441015c0 2023-12-07T10:19:26.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.428+0000 7f5b41d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b441027b0 0x7f5b44199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:26.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.428+0000 7f5b41d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b441027b0 0x7f5b44199ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51882/0 (socket says 172.21.15.119:51882) 2023-12-07T10:19:26.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.428+0000 7f5b41d9b700 1 -- 172.21.15.119:0/1134512168 learned_addr learned my addr 172.21.15.119:0/1134512168 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:26.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.428+0000 7f5b4259c700 1 --2- 172.21.15.119:0/1134512168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b441015c0 0x7f5b441997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:26.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.428+0000 7f5b42d9d700 1 --2- 172.21.15.119:0/1134512168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b44103150 0x7f5b441a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:26.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.428+0000 7f5b41d9b700 1 -- 172.21.15.119:0/1134512168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b441015c0 msgr2=0x7f5b441997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:26.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.428+0000 7f5b41d9b700 1 --2- 172.21.15.119:0/1134512168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b441015c0 0x7f5b441997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:26.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.428+0000 7f5b41d9b700 1 -- 172.21.15.119:0/1134512168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b44103150 msgr2=0x7f5b441a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:26.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.428+0000 7f5b41d9b700 1 --2- 172.21.15.119:0/1134512168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b44103150 0x7f5b441a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:26.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.428+0000 7f5b41d9b700 1 -- 172.21.15.119:0/1134512168 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b380095f0 con 0x7f5b441027b0 2023-12-07T10:19:26.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.428+0000 7f5b41d9b700 1 --2- 172.21.15.119:0/1134512168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b441027b0 0x7f5b44199ce0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5b38003730 tx=0x7f5b3800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:26.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.429+0000 7f5b337fe700 1 -- 172.21.15.119:0/1134512168 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3801d070 con 0x7f5b441027b0 2023-12-07T10:19:26.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.429+0000 7f5b48fff700 1 -- 172.21.15.119:0/1134512168 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b441a17d0 con 0x7f5b441027b0 2023-12-07T10:19:26.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.429+0000 7f5b337fe700 1 -- 172.21.15.119:0/1134512168 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b38005050 con 0x7f5b441027b0 2023-12-07T10:19:26.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.429+0000 7f5b337fe700 1 -- 172.21.15.119:0/1134512168 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b38004b10 con 0x7f5b441027b0 2023-12-07T10:19:26.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.429+0000 7f5b48fff700 1 -- 172.21.15.119:0/1134512168 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b441a1c50 con 0x7f5b441027b0 2023-12-07T10:19:26.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.430+0000 7f5b48fff700 1 -- 172.21.15.119:0/1134512168 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b440683c0 con 0x7f5b441027b0 2023-12-07T10:19:26.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.433+0000 7f5b337fe700 1 -- 172.21.15.119:0/1134512168 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f5b380051c0 con 0x7f5b441027b0 2023-12-07T10:19:26.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.434+0000 7f5b337fe700 1 --2- 172.21.15.119:0/1134512168 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5b2c0a1ba0 0x7f5b2c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:26.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.434+0000 7f5b337fe700 1 -- 172.21.15.119:0/1134512168 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f5b380cebe0 con 0x7f5b441027b0 2023-12-07T10:19:26.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.435+0000 7f5b4259c700 1 --2- 172.21.15.119:0/1134512168 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5b2c0a1ba0 0x7f5b2c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:26.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.435+0000 7f5b4259c700 1 --2- 172.21.15.119:0/1134512168 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5b2c0a1ba0 0x7f5b2c0a4060 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5b44102610 tx=0x7f5b34009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:26.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.436+0000 7f5b337fe700 1 -- 172.21.15.119:0/1134512168 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5b38097b60 con 0x7f5b441027b0 2023-12-07T10:19:26.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.637+0000 7f5b48fff700 1 -- 172.21.15.119:0/1134512168 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5b441a1f30 con 0x7f5b441027b0 2023-12-07T10:19:26.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.637+0000 7f5b337fe700 1 -- 172.21.15.119:0/1134512168 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5b3809b180 con 0x7f5b441027b0 2023-12-07T10:19:26.639 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-12-07T10:19:26.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.639+0000 7f5b48fff700 1 -- 172.21.15.119:0/1134512168 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5b2c0a1ba0 msgr2=0x7f5b2c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:26.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.639+0000 7f5b48fff700 1 --2- 172.21.15.119:0/1134512168 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5b2c0a1ba0 0x7f5b2c0a4060 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5b44102610 tx=0x7f5b34009260 comp rx=0 tx=0).stop 2023-12-07T10:19:26.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.639+0000 7f5b48fff700 1 -- 172.21.15.119:0/1134512168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b441027b0 msgr2=0x7f5b44199ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:26.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.639+0000 7f5b48fff700 1 --2- 172.21.15.119:0/1134512168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b441027b0 0x7f5b44199ce0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5b38003730 tx=0x7f5b3800f690 comp rx=0 tx=0).stop 2023-12-07T10:19:26.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.640+0000 7f5b48fff700 1 -- 172.21.15.119:0/1134512168 shutdown_connections 2023-12-07T10:19:26.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.640+0000 7f5b48fff700 1 --2- 172.21.15.119:0/1134512168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b44103150 0x7f5b441a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:26.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.640+0000 7f5b48fff700 1 --2- 172.21.15.119:0/1134512168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b441015c0 0x7f5b441997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:26.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.640+0000 7f5b48fff700 1 --2- 172.21.15.119:0/1134512168 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5b2c0a1ba0 0x7f5b2c0a4060 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:26.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.640+0000 7f5b48fff700 1 --2- 172.21.15.119:0/1134512168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b441027b0 0x7f5b44199ce0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:26.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.640+0000 7f5b48fff700 1 -- 172.21.15.119:0/1134512168 >> 172.21.15.119:0/1134512168 conn(0x7f5b440fcb10 msgr2=0x7f5b44073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:26.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.640+0000 7f5b48fff700 1 -- 172.21.15.119:0/1134512168 shutdown_connections 2023-12-07T10:19:26.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.640+0000 7f5b48fff700 1 -- 172.21.15.119:0/1134512168 wait complete. 2023-12-07T10:19:26.660 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-07T10:19:26.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.860+0000 7f6a11bff700 1 -- 172.21.15.119:0/2265305928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a0c1027b0 msgr2=0x7f6a0c102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:26.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.860+0000 7f6a11bff700 1 --2- 172.21.15.119:0/2265305928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a0c1027b0 0x7f6a0c102c10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f69f40099c0 tx=0x7f69f4009cd0 comp rx=0 tx=0).stop 2023-12-07T10:19:26.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.860+0000 7f6a11bff700 1 -- 172.21.15.119:0/2265305928 shutdown_connections 2023-12-07T10:19:26.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.860+0000 7f6a11bff700 1 --2- 172.21.15.119:0/2265305928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a0c103150 0x7f6a0c109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:26.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.860+0000 7f6a11bff700 1 --2- 172.21.15.119:0/2265305928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a0c1015c0 0x7f6a0c1019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:26.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.860+0000 7f6a11bff700 1 --2- 172.21.15.119:0/2265305928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a0c1027b0 0x7f6a0c102c10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:26.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.860+0000 7f6a11bff700 1 -- 172.21.15.119:0/2265305928 >> 172.21.15.119:0/2265305928 conn(0x7f6a0c0fcb10 msgr2=0x7f6a0c0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:26.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.860+0000 7f6a11bff700 1 -- 172.21.15.119:0/2265305928 shutdown_connections 2023-12-07T10:19:26.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.860+0000 7f6a11bff700 1 -- 172.21.15.119:0/2265305928 wait complete. 2023-12-07T10:19:26.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.862+0000 7f6a11bff700 1 Processor -- start 2023-12-07T10:19:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.862+0000 7f6a11bff700 1 -- start start 2023-12-07T10:19:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.862+0000 7f6a11bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a0c1015c0 0x7f6a0c197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.862+0000 7f6a11bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a0c1027b0 0x7f6a0c197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.862+0000 7f6a11bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a0c103150 0x7f6a0c19ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.862+0000 7f6a11bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c19f050 con 0x7f6a0c103150 2023-12-07T10:19:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.862+0000 7f6a11bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c19f190 con 0x7f6a0c1015c0 2023-12-07T10:19:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.862+0000 7f6a11bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c19f320 con 0x7f6a0c1027b0 2023-12-07T10:19:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.863+0000 7f6a0bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a0c103150 0x7f6a0c19ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.863+0000 7f6a0b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a0c1015c0 0x7f6a0c197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.863+0000 7f6a0bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a0c103150 0x7f6a0c19ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51894/0 (socket says 172.21.15.119:51894) 2023-12-07T10:19:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.863+0000 7f6a0b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a0c1015c0 0x7f6a0c197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:40880/0 (socket says 172.21.15.119:40880) 2023-12-07T10:19:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.863+0000 7f6a0bfff700 1 -- 172.21.15.119:0/87414444 learned_addr learned my addr 172.21.15.119:0/87414444 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.863+0000 7f6a03fff700 1 --2- 172.21.15.119:0/87414444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a0c1027b0 0x7f6a0c197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:26.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.863+0000 7f6a0bfff700 1 -- 172.21.15.119:0/87414444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a0c1027b0 msgr2=0x7f6a0c197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:26.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.863+0000 7f6a0bfff700 1 --2- 172.21.15.119:0/87414444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a0c1027b0 0x7f6a0c197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:26.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.863+0000 7f6a0bfff700 1 -- 172.21.15.119:0/87414444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a0c1015c0 msgr2=0x7f6a0c197520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:26.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.863+0000 7f6a0bfff700 1 --2- 172.21.15.119:0/87414444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a0c1015c0 0x7f6a0c197520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:26.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.863+0000 7f6a0bfff700 1 -- 172.21.15.119:0/87414444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69fc009520 con 0x7f6a0c103150 2023-12-07T10:19:26.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.863+0000 7f6a0bfff700 1 --2- 172.21.15.119:0/87414444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a0c103150 0x7f6a0c19ea80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400ef30 tx=0x7f6a0400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:26.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.864+0000 7f6a097fa700 1 -- 172.21.15.119:0/87414444 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a04018070 con 0x7f6a0c103150 2023-12-07T10:19:26.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.864+0000 7f6a097fa700 1 -- 172.21.15.119:0/87414444 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a0400f040 con 0x7f6a0c103150 2023-12-07T10:19:26.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.864+0000 7f6a11bff700 1 -- 172.21.15.119:0/87414444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f40095f0 con 0x7f6a0c103150 2023-12-07T10:19:26.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.864+0000 7f6a097fa700 1 -- 172.21.15.119:0/87414444 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0400cd00 con 0x7f6a0c103150 2023-12-07T10:19:26.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.864+0000 7f6a11bff700 1 -- 172.21.15.119:0/87414444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a0c073b70 con 0x7f6a0c103150 2023-12-07T10:19:26.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.865+0000 7f6a11bff700 1 -- 172.21.15.119:0/87414444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a0c108a40 con 0x7f6a0c103150 2023-12-07T10:19:26.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.867+0000 7f6a097fa700 1 -- 172.21.15.119:0/87414444 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f6a04003a00 con 0x7f6a0c103150 2023-12-07T10:19:26.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.868+0000 7f6a097fa700 1 --2- 172.21.15.119:0/87414444 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f69f80a1ba0 0x7f69f80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:26.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.868+0000 7f6a097fa700 1 -- 172.21.15.119:0/87414444 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f6a040cdfa0 con 0x7f6a0c103150 2023-12-07T10:19:26.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.868+0000 7f6a0b7fe700 1 --2- 172.21.15.119:0/87414444 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f69f80a1ba0 0x7f69f80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:26.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.869+0000 7f6a0b7fe700 1 --2- 172.21.15.119:0/87414444 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f69f80a1ba0 0x7f69f80a4060 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c102610 tx=0x7f69fc012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:26.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:26.871+0000 7f6a097fa700 1 -- 172.21.15.119:0/87414444 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f6a04096f80 con 0x7f6a0c103150 2023-12-07T10:19:27.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.077+0000 7f6a11bff700 1 -- 172.21.15.119:0/87414444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6a0c10cde0 con 0x7f6a0c103150 2023-12-07T10:19:27.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.078+0000 7f6a097fa700 1 -- 172.21.15.119:0/87414444 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6a0409a5a0 con 0x7f6a0c103150 2023-12-07T10:19:27.080 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-12-07T10:19:27.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.080+0000 7f6a11bff700 1 -- 172.21.15.119:0/87414444 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f69f80a1ba0 msgr2=0x7f69f80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:27.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.080+0000 7f6a11bff700 1 --2- 172.21.15.119:0/87414444 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f69f80a1ba0 0x7f69f80a4060 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c102610 tx=0x7f69fc012040 comp rx=0 tx=0).stop 2023-12-07T10:19:27.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.080+0000 7f6a11bff700 1 -- 172.21.15.119:0/87414444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a0c103150 msgr2=0x7f6a0c19ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:27.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.080+0000 7f6a11bff700 1 --2- 172.21.15.119:0/87414444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a0c103150 0x7f6a0c19ea80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400ef30 tx=0x7f6a0400c3a0 comp rx=0 tx=0).stop 2023-12-07T10:19:27.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.080+0000 7f6a11bff700 1 -- 172.21.15.119:0/87414444 shutdown_connections 2023-12-07T10:19:27.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.080+0000 7f6a11bff700 1 --2- 172.21.15.119:0/87414444 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f69f80a1ba0 0x7f69f80a4060 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:27.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.080+0000 7f6a11bff700 1 --2- 172.21.15.119:0/87414444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a0c103150 0x7f6a0c19ea80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:27.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.080+0000 7f6a11bff700 1 --2- 172.21.15.119:0/87414444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a0c1015c0 0x7f6a0c197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:27.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.080+0000 7f6a11bff700 1 --2- 172.21.15.119:0/87414444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a0c1027b0 0x7f6a0c197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:27.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.080+0000 7f6a11bff700 1 -- 172.21.15.119:0/87414444 >> 172.21.15.119:0/87414444 conn(0x7f6a0c0fcb10 msgr2=0x7f6a0c105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:27.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.080+0000 7f6a11bff700 1 -- 172.21.15.119:0/87414444 shutdown_connections 2023-12-07T10:19:27.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.081+0000 7f6a11bff700 1 -- 172.21.15.119:0/87414444 wait complete. 2023-12-07T10:19:27.101 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-07T10:19:27.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:26 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3004780384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:19:27.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:26 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1134512168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:19:27.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:26 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3004780384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:19:27.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:26 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1134512168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:19:27.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:26 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3004780384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:19:27.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:26 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1134512168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:19:27.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.301+0000 7fcac97ff700 1 -- 172.21.15.119:0/2485304248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcac4100780 msgr2=0x7fcac4100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:27.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.301+0000 7fcac97ff700 1 --2- 172.21.15.119:0/2485304248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcac4100780 0x7fcac4100c00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009990 tx=0x7fcab8009ca0 comp rx=0 tx=0).stop 2023-12-07T10:19:27.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.301+0000 7fcac97ff700 1 -- 172.21.15.119:0/2485304248 shutdown_connections 2023-12-07T10:19:27.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.301+0000 7fcac97ff700 1 --2- 172.21.15.119:0/2485304248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcac4101140 0x7fcac41015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:27.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.301+0000 7fcac97ff700 1 --2- 172.21.15.119:0/2485304248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcac40ffe20 0x7fcac4100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:27.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.301+0000 7fcac97ff700 1 --2- 172.21.15.119:0/2485304248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcac4100780 0x7fcac4100c00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:27.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.301+0000 7fcac97ff700 1 -- 172.21.15.119:0/2485304248 >> 172.21.15.119:0/2485304248 conn(0x7fcac40fba20 msgr2=0x7fcac40fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:27.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.301+0000 7fcac97ff700 1 -- 172.21.15.119:0/2485304248 shutdown_connections 2023-12-07T10:19:27.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.301+0000 7fcac97ff700 1 -- 172.21.15.119:0/2485304248 wait complete. 2023-12-07T10:19:27.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.302+0000 7fcac97ff700 1 Processor -- start 2023-12-07T10:19:27.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.302+0000 7fcac97ff700 1 -- start start 2023-12-07T10:19:27.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.303+0000 7fcac97ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcac40ffe20 0x7fcac4199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:27.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.303+0000 7fcac97ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcac4100780 0x7fcac4199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:27.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.303+0000 7fcac97ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcac4101140 0x7fcac41a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:27.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.303+0000 7fcac97ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac41a1250 con 0x7fcac4100780 2023-12-07T10:19:27.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.303+0000 7fcac97ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac41a1390 con 0x7fcac4101140 2023-12-07T10:19:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.303+0000 7fcac97ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac41a1520 con 0x7fcac40ffe20 2023-12-07T10:19:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.303+0000 7fcac37fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcac4101140 0x7fcac41a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.303+0000 7fcac37fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcac4101140 0x7fcac41a0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:40898/0 (socket says 172.21.15.119:40898) 2023-12-07T10:19:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.303+0000 7fcac37fe700 1 -- 172.21.15.119:0/1184367567 learned_addr learned my addr 172.21.15.119:0/1184367567 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.303+0000 7fcac27fc700 1 --2- 172.21.15.119:0/1184367567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcac4100780 0x7fcac4199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.303+0000 7fcac2ffd700 1 --2- 172.21.15.119:0/1184367567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcac40ffe20 0x7fcac4199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.303+0000 7fcac37fe700 1 -- 172.21.15.119:0/1184367567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcac40ffe20 msgr2=0x7fcac4199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.303+0000 7fcac37fe700 1 --2- 172.21.15.119:0/1184367567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcac40ffe20 0x7fcac4199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.303+0000 7fcac37fe700 1 -- 172.21.15.119:0/1184367567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcac4100780 msgr2=0x7fcac4199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.303+0000 7fcac37fe700 1 --2- 172.21.15.119:0/1184367567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcac4100780 0x7fcac4199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.303+0000 7fcac37fe700 1 -- 172.21.15.119:0/1184367567 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab80095f0 con 0x7fcac4101140 2023-12-07T10:19:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.303+0000 7fcac37fe700 1 --2- 172.21.15.119:0/1184367567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcac4101140 0x7fcac41a0c80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcabc00ef30 tx=0x7fcabc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:27.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.304+0000 7fcaabfff700 1 -- 172.21.15.119:0/1184367567 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc018070 con 0x7fcac4101140 2023-12-07T10:19:27.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.304+0000 7fcaabfff700 1 -- 172.21.15.119:0/1184367567 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcabc00f040 con 0x7fcac4101140 2023-12-07T10:19:27.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.304+0000 7fcac97ff700 1 -- 172.21.15.119:0/1184367567 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac41a17a0 con 0x7fcac4101140 2023-12-07T10:19:27.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.304+0000 7fcaabfff700 1 -- 172.21.15.119:0/1184367567 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc00cd00 con 0x7fcac4101140 2023-12-07T10:19:27.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.304+0000 7fcac97ff700 1 -- 172.21.15.119:0/1184367567 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac41a1c60 con 0x7fcac4101140 2023-12-07T10:19:27.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.305+0000 7fcac97ff700 1 -- 172.21.15.119:0/1184367567 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac410ae40 con 0x7fcac4101140 2023-12-07T10:19:27.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.307+0000 7fcaabfff700 1 -- 172.21.15.119:0/1184367567 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7fcabc003a00 con 0x7fcac4101140 2023-12-07T10:19:27.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.308+0000 7fcaabfff700 1 --2- 172.21.15.119:0/1184367567 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fcaac0a1ba0 0x7fcaac0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:27.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.308+0000 7fcaabfff700 1 -- 172.21.15.119:0/1184367567 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fcabc0cdfe0 con 0x7fcac4101140 2023-12-07T10:19:27.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.309+0000 7fcac2ffd700 1 --2- 172.21.15.119:0/1184367567 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fcaac0a1ba0 0x7fcaac0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:27.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.309+0000 7fcac2ffd700 1 --2- 172.21.15.119:0/1184367567 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fcaac0a1ba0 0x7fcaac0a4060 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009c60 tx=0x7fcab4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:27.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.310+0000 7fcaabfff700 1 -- 172.21.15.119:0/1184367567 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fcabc096f80 con 0x7fcac4101140 2023-12-07T10:19:27.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.512+0000 7fcac97ff700 1 -- 172.21.15.119:0/1184367567 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcac410f0f0 con 0x7fcac4101140 2023-12-07T10:19:27.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.513+0000 7fcaabfff700 1 -- 172.21.15.119:0/1184367567 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcabc09a5a0 con 0x7fcac4101140 2023-12-07T10:19:27.514 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-12-07T10:19:27.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.515+0000 7fcac97ff700 1 -- 172.21.15.119:0/1184367567 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fcaac0a1ba0 msgr2=0x7fcaac0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:27.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.515+0000 7fcac97ff700 1 --2- 172.21.15.119:0/1184367567 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fcaac0a1ba0 0x7fcaac0a4060 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009c60 tx=0x7fcab4009260 comp rx=0 tx=0).stop 2023-12-07T10:19:27.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.515+0000 7fcac97ff700 1 -- 172.21.15.119:0/1184367567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcac4101140 msgr2=0x7fcac41a0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:27.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.515+0000 7fcac97ff700 1 --2- 172.21.15.119:0/1184367567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcac4101140 0x7fcac41a0c80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcabc00ef30 tx=0x7fcabc00c3a0 comp rx=0 tx=0).stop 2023-12-07T10:19:27.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.515+0000 7fcac97ff700 1 -- 172.21.15.119:0/1184367567 shutdown_connections 2023-12-07T10:19:27.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.515+0000 7fcac97ff700 1 --2- 172.21.15.119:0/1184367567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcac4101140 0x7fcac41a0c80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:27.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.515+0000 7fcac97ff700 1 --2- 172.21.15.119:0/1184367567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcac40ffe20 0x7fcac4199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:27.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.515+0000 7fcac97ff700 1 --2- 172.21.15.119:0/1184367567 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fcaac0a1ba0 0x7fcaac0a4060 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:27.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.515+0000 7fcac97ff700 1 --2- 172.21.15.119:0/1184367567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcac4100780 0x7fcac4199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:27.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.515+0000 7fcac97ff700 1 -- 172.21.15.119:0/1184367567 >> 172.21.15.119:0/1184367567 conn(0x7fcac40fba20 msgr2=0x7fcac40fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:27.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.516+0000 7fcac97ff700 1 -- 172.21.15.119:0/1184367567 shutdown_connections 2023-12-07T10:19:27.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.516+0000 7fcac97ff700 1 -- 172.21.15.119:0/1184367567 wait complete. 2023-12-07T10:19:27.536 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-07T10:19:27.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.732+0000 7f34431ff700 1 -- 172.21.15.119:0/1778585438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f343c103150 msgr2=0x7f343c109a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:27.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.732+0000 7f34431ff700 1 --2- 172.21.15.119:0/1778585438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f343c103150 0x7f343c109a00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f343400b0d0 tx=0x7f343400b3e0 comp rx=0 tx=0).stop 2023-12-07T10:19:27.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.733+0000 7f34431ff700 1 -- 172.21.15.119:0/1778585438 shutdown_connections 2023-12-07T10:19:27.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.733+0000 7f34431ff700 1 --2- 172.21.15.119:0/1778585438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f343c103150 0x7f343c109a00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:27.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.733+0000 7f34431ff700 1 --2- 172.21.15.119:0/1778585438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f343c1015c0 0x7f343c1019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:27.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.733+0000 7f34431ff700 1 --2- 172.21.15.119:0/1778585438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f343c1027b0 0x7f343c102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:27.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.733+0000 7f34431ff700 1 -- 172.21.15.119:0/1778585438 >> 172.21.15.119:0/1778585438 conn(0x7f343c0fcb10 msgr2=0x7f343c0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:27.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.733+0000 7f34431ff700 1 -- 172.21.15.119:0/1778585438 shutdown_connections 2023-12-07T10:19:27.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.733+0000 7f34431ff700 1 -- 172.21.15.119:0/1778585438 wait complete. 2023-12-07T10:19:27.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.734+0000 7f34431ff700 1 Processor -- start 2023-12-07T10:19:27.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.734+0000 7f34431ff700 1 -- start start 2023-12-07T10:19:27.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.734+0000 7f34431ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f343c1015c0 0x7f343c199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:27.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.734+0000 7f34431ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f343c1027b0 0x7f343c199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:27.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.734+0000 7f34431ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f343c103150 0x7f343c1a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:27.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.734+0000 7f34431ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c1a13c0 con 0x7f343c1027b0 2023-12-07T10:19:27.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.734+0000 7f34431ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c1a1500 con 0x7f343c1015c0 2023-12-07T10:19:27.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.734+0000 7f34431ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c1a1690 con 0x7f343c103150 2023-12-07T10:19:27.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.734+0000 7f343bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f343c1027b0 0x7f343c199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:27.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.734+0000 7f3440f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f343c1015c0 0x7f343c199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:27.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.734+0000 7f3440f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f343c1015c0 0x7f343c199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:40920/0 (socket says 172.21.15.119:40920) 2023-12-07T10:19:27.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.734+0000 7f3440f9b700 1 -- 172.21.15.119:0/3001650339 learned_addr learned my addr 172.21.15.119:0/3001650339 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:27.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.734+0000 7f344179c700 1 --2- 172.21.15.119:0/3001650339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f343c103150 0x7f343c1a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:27.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.735+0000 7f3440f9b700 1 -- 172.21.15.119:0/3001650339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f343c103150 msgr2=0x7f343c1a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:27.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.735+0000 7f3440f9b700 1 --2- 172.21.15.119:0/3001650339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f343c103150 0x7f343c1a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:27.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.735+0000 7f3440f9b700 1 -- 172.21.15.119:0/3001650339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f343c1027b0 msgr2=0x7f343c199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:27.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.735+0000 7f3440f9b700 1 --2- 172.21.15.119:0/3001650339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f343c1027b0 0x7f343c199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:27.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.735+0000 7f3440f9b700 1 -- 172.21.15.119:0/3001650339 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3428009520 con 0x7f343c1015c0 2023-12-07T10:19:27.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.735+0000 7f343bfff700 1 --2- 172.21.15.119:0/3001650339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f343c1027b0 0x7f343c199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:27.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.735+0000 7f344179c700 1 --2- 172.21.15.119:0/3001650339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f343c103150 0x7f343c1a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:27.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.735+0000 7f3440f9b700 1 --2- 172.21.15.119:0/3001650339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f343c1015c0 0x7f343c199890 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f343000e970 tx=0x7f343000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:27.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.735+0000 7f3439ffb700 1 -- 172.21.15.119:0/3001650339 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430009790 con 0x7f343c1015c0 2023-12-07T10:19:27.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.735+0000 7f3439ffb700 1 -- 172.21.15.119:0/3001650339 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f343000c7e0 con 0x7f343c1015c0 2023-12-07T10:19:27.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.735+0000 7f34431ff700 1 -- 172.21.15.119:0/3001650339 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3434009d00 con 0x7f343c1015c0 2023-12-07T10:19:27.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.735+0000 7f3439ffb700 1 -- 172.21.15.119:0/3001650339 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430003c70 con 0x7f343c1015c0 2023-12-07T10:19:27.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.735+0000 7f34431ff700 1 -- 172.21.15.119:0/3001650339 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f343c1a1cf0 con 0x7f343c1015c0 2023-12-07T10:19:27.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.736+0000 7f34431ff700 1 -- 172.21.15.119:0/3001650339 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f343c0683c0 con 0x7f343c1015c0 2023-12-07T10:19:27.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.738+0000 7f3439ffb700 1 -- 172.21.15.119:0/3001650339 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f343000cc50 con 0x7f343c1015c0 2023-12-07T10:19:27.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.740+0000 7f3439ffb700 1 --2- 172.21.15.119:0/3001650339 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f342c0a1ba0 0x7f342c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:27.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.740+0000 7f3439ffb700 1 -- 172.21.15.119:0/3001650339 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f3430014070 con 0x7f343c1015c0 2023-12-07T10:19:27.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.740+0000 7f343bfff700 1 --2- 172.21.15.119:0/3001650339 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f342c0a1ba0 0x7f342c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:27.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.741+0000 7f343bfff700 1 --2- 172.21.15.119:0/3001650339 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f342c0a1ba0 0x7f342c0a4060 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3428009fd0 tx=0x7f3428009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:27.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.742+0000 7f3439ffb700 1 -- 172.21.15.119:0/3001650339 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3430096cf0 con 0x7f343c1015c0 2023-12-07T10:19:27.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.948+0000 7f34431ff700 1 -- 172.21.15.119:0/3001650339 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f343c10d050 con 0x7f343c1015c0 2023-12-07T10:19:27.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.949+0000 7f3439ffb700 1 -- 172.21.15.119:0/3001650339 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3430003dd0 con 0x7f343c1015c0 2023-12-07T10:19:27.951 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-12-07T10:19:27.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.952+0000 7f34431ff700 1 -- 172.21.15.119:0/3001650339 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f342c0a1ba0 msgr2=0x7f342c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:27.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.952+0000 7f34431ff700 1 --2- 172.21.15.119:0/3001650339 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f342c0a1ba0 0x7f342c0a4060 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3428009fd0 tx=0x7f3428009260 comp rx=0 tx=0).stop 2023-12-07T10:19:27.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.952+0000 7f34431ff700 1 -- 172.21.15.119:0/3001650339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f343c1015c0 msgr2=0x7f343c199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:27.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.952+0000 7f34431ff700 1 --2- 172.21.15.119:0/3001650339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f343c1015c0 0x7f343c199890 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f343000e970 tx=0x7f343000ec80 comp rx=0 tx=0).stop 2023-12-07T10:19:27.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.952+0000 7f34431ff700 1 -- 172.21.15.119:0/3001650339 shutdown_connections 2023-12-07T10:19:27.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.952+0000 7f34431ff700 1 --2- 172.21.15.119:0/3001650339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f343c103150 0x7f343c1a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:27.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.952+0000 7f34431ff700 1 --2- 172.21.15.119:0/3001650339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f343c1015c0 0x7f343c199890 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:27.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.952+0000 7f34431ff700 1 --2- 172.21.15.119:0/3001650339 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f342c0a1ba0 0x7f342c0a4060 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:27.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.952+0000 7f34431ff700 1 --2- 172.21.15.119:0/3001650339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f343c1027b0 0x7f343c199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:27.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.952+0000 7f34431ff700 1 -- 172.21.15.119:0/3001650339 >> 172.21.15.119:0/3001650339 conn(0x7f343c0fcb10 msgr2=0x7f343c073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:27.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.952+0000 7f34431ff700 1 -- 172.21.15.119:0/3001650339 shutdown_connections 2023-12-07T10:19:27.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:27.952+0000 7f34431ff700 1 -- 172.21.15.119:0/3001650339 wait complete. 2023-12-07T10:19:27.973 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:19:28.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.167+0000 7f9b749ff700 1 -- 172.21.15.119:0/3862651311 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b70101560 msgr2=0x7f9b70101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:28.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.167+0000 7f9b749ff700 1 --2- 172.21.15.119:0/3862651311 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b70101560 0x7f9b70101980 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9b60009990 tx=0x7f9b60009ca0 comp rx=0 tx=0).stop 2023-12-07T10:19:28.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.167+0000 7f9b749ff700 1 -- 172.21.15.119:0/3862651311 shutdown_connections 2023-12-07T10:19:28.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.167+0000 7f9b749ff700 1 --2- 172.21.15.119:0/3862651311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b701030f0 0x7f9b701099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:28.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.167+0000 7f9b749ff700 1 --2- 172.21.15.119:0/3862651311 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b70101560 0x7f9b70101980 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:28.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.167+0000 7f9b749ff700 1 --2- 172.21.15.119:0/3862651311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b70102750 0x7f9b70102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:28.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.167+0000 7f9b749ff700 1 -- 172.21.15.119:0/3862651311 >> 172.21.15.119:0/3862651311 conn(0x7f9b700fcab0 msgr2=0x7f9b700fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:28.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.167+0000 7f9b749ff700 1 -- 172.21.15.119:0/3862651311 shutdown_connections 2023-12-07T10:19:28.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.168+0000 7f9b749ff700 1 -- 172.21.15.119:0/3862651311 wait complete. 2023-12-07T10:19:28.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.168+0000 7f9b749ff700 1 Processor -- start 2023-12-07T10:19:28.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.169+0000 7f9b749ff700 1 -- start start 2023-12-07T10:19:28.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.169+0000 7f9b749ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b70101560 0x7f9b70075320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:28.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.169+0000 7f9b749ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b70102750 0x7f9b70073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:28.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.169+0000 7f9b749ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b701030f0 0x7f9b70073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:28.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.169+0000 7f9b749ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b70074640 con 0x7f9b70101560 2023-12-07T10:19:28.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.169+0000 7f9b749ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b700747b0 con 0x7f9b70102750 2023-12-07T10:19:28.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.169+0000 7f9b749ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b701a15a0 con 0x7f9b701030f0 2023-12-07T10:19:28.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.169+0000 7f9b6e59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b70101560 0x7f9b70075320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:28.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.169+0000 7f9b6e59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b70101560 0x7f9b70075320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51950/0 (socket says 172.21.15.119:51950) 2023-12-07T10:19:28.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.169+0000 7f9b6e59c700 1 -- 172.21.15.119:0/136502542 learned_addr learned my addr 172.21.15.119:0/136502542 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:28.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.169+0000 7f9b6ed9d700 1 --2- 172.21.15.119:0/136502542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b701030f0 0x7f9b70073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:28.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.169+0000 7f9b67fff700 1 --2- 172.21.15.119:0/136502542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b70102750 0x7f9b70073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:28.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.169+0000 7f9b6e59c700 1 -- 172.21.15.119:0/136502542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b701030f0 msgr2=0x7f9b70073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:28.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.169+0000 7f9b6e59c700 1 --2- 172.21.15.119:0/136502542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b701030f0 0x7f9b70073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:28.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.169+0000 7f9b6e59c700 1 -- 172.21.15.119:0/136502542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b70102750 msgr2=0x7f9b70073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:28.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.169+0000 7f9b6e59c700 1 --2- 172.21.15.119:0/136502542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b70102750 0x7f9b70073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:28.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.169+0000 7f9b6e59c700 1 -- 172.21.15.119:0/136502542 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b600095f0 con 0x7f9b70101560 2023-12-07T10:19:28.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.170+0000 7f9b6ed9d700 1 --2- 172.21.15.119:0/136502542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b701030f0 0x7f9b70073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:28.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.170+0000 7f9b6e59c700 1 --2- 172.21.15.119:0/136502542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b70101560 0x7f9b70075320 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9b6000b7b0 tx=0x7f9b60005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:28.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.170+0000 7f9b677fe700 1 -- 172.21.15.119:0/136502542 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6001d070 con 0x7f9b70101560 2023-12-07T10:19:28.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.170+0000 7f9b749ff700 1 -- 172.21.15.119:0/136502542 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b701a17a0 con 0x7f9b70101560 2023-12-07T10:19:28.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.170+0000 7f9b677fe700 1 -- 172.21.15.119:0/136502542 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b60022470 con 0x7f9b70101560 2023-12-07T10:19:28.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.170+0000 7f9b677fe700 1 -- 172.21.15.119:0/136502542 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b60017650 con 0x7f9b70101560 2023-12-07T10:19:28.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.170+0000 7f9b749ff700 1 -- 172.21.15.119:0/136502542 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b701a1c20 con 0x7f9b70101560 2023-12-07T10:19:28.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.171+0000 7f9b749ff700 1 -- 172.21.15.119:0/136502542 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b70108aa0 con 0x7f9b70101560 2023-12-07T10:19:28.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.174+0000 7f9b677fe700 1 -- 172.21.15.119:0/136502542 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f9b60022890 con 0x7f9b70101560 2023-12-07T10:19:28.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.176+0000 7f9b677fe700 1 --2- 172.21.15.119:0/136502542 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f9b5c0a1ba0 0x7f9b5c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:28.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.176+0000 7f9b677fe700 1 -- 172.21.15.119:0/136502542 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f9b600cf630 con 0x7f9b70101560 2023-12-07T10:19:28.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.176+0000 7f9b67fff700 1 --2- 172.21.15.119:0/136502542 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f9b5c0a1ba0 0x7f9b5c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:28.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.177+0000 7f9b67fff700 1 --2- 172.21.15.119:0/136502542 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f9b5c0a1ba0 0x7f9b5c0a4060 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9b58009c60 tx=0x7f9b58009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:28.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.178+0000 7f9b677fe700 1 -- 172.21.15.119:0/136502542 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9b600985b0 con 0x7f9b70101560 2023-12-07T10:19:28.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:27 smithi119 ceph-mon[97908]: pgmap v336: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-12-07T10:19:28.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:27 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/87414444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:19:28.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:27 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1184367567' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:19:28.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:27 smithi159 ceph-mon[134726]: pgmap v336: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-12-07T10:19:28.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:27 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/87414444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:19:28.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:27 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1184367567' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:19:28.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:27 smithi153 ceph-mon[133625]: pgmap v336: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-12-07T10:19:28.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:27 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/87414444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:19:28.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:27 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1184367567' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:19:28.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.420+0000 7f9b749ff700 1 -- 172.21.15.119:0/136502542 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b70068350 con 0x7f9b70101560 2023-12-07T10:19:28.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.420+0000 7f9b677fe700 1 -- 172.21.15.119:0/136502542 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11385 (secure 0 0 0) 0x7f9b6009bbd0 con 0x7f9b70101560 2023-12-07T10:19:28.422 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:19:28.422 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-07T10:17:41.278311+0000","modified":"2023-12-07T10:18:18.543356+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24517,"mds_3":24505,"mds_4":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14535":{"gid":14535,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6841/1791998028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1791998028},{"type":"v1","addr":"172.21.15.119:6841","nonce":1791998028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24505":{"gid":24505,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/2925503246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2925503246},{"type":"v1","addr":"172.21.15.153:6837","nonce":2925503246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6839/3984962900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3984962900},{"type":"v1","addr":"172.21.15.159:6839","nonce":3984962900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.119:6839/1920762602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1920762602},{"type":"v1","addr":"172.21.15.119:6839","nonce":1920762602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/566825184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":566825184},{"type":"v1","addr":"172.21.15.153:6839","nonce":566825184}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-07T10:19:28.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.423+0000 7f9b749ff700 1 -- 172.21.15.119:0/136502542 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f9b5c0a1ba0 msgr2=0x7f9b5c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:28.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.423+0000 7f9b749ff700 1 --2- 172.21.15.119:0/136502542 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f9b5c0a1ba0 0x7f9b5c0a4060 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9b58009c60 tx=0x7f9b58009260 comp rx=0 tx=0).stop 2023-12-07T10:19:28.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.423+0000 7f9b749ff700 1 -- 172.21.15.119:0/136502542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b70101560 msgr2=0x7f9b70075320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:28.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.423+0000 7f9b749ff700 1 --2- 172.21.15.119:0/136502542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b70101560 0x7f9b70075320 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9b6000b7b0 tx=0x7f9b60005720 comp rx=0 tx=0).stop 2023-12-07T10:19:28.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.423+0000 7f9b749ff700 1 -- 172.21.15.119:0/136502542 shutdown_connections 2023-12-07T10:19:28.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.423+0000 7f9b749ff700 1 --2- 172.21.15.119:0/136502542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b701030f0 0x7f9b70073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:28.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.423+0000 7f9b749ff700 1 --2- 172.21.15.119:0/136502542 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f9b5c0a1ba0 0x7f9b5c0a4060 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:28.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.423+0000 7f9b749ff700 1 --2- 172.21.15.119:0/136502542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b70101560 0x7f9b70075320 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:28.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.423+0000 7f9b749ff700 1 --2- 172.21.15.119:0/136502542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b70102750 0x7f9b70073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:28.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.423+0000 7f9b749ff700 1 -- 172.21.15.119:0/136502542 >> 172.21.15.119:0/136502542 conn(0x7f9b700fcab0 msgr2=0x7f9b70105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:28.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.423+0000 7f9b749ff700 1 -- 172.21.15.119:0/136502542 shutdown_connections 2023-12-07T10:19:28.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.423+0000 7f9b749ff700 1 -- 172.21.15.119:0/136502542 wait complete. 2023-12-07T10:19:28.426 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 29 2023-12-07T10:19:28.444 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-07T10:19:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.639+0000 7efbf11ff700 1 -- 172.21.15.119:0/1154763609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efbec075650 msgr2=0x7efbec073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.639+0000 7efbf11ff700 1 --2- 172.21.15.119:0/1154763609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efbec075650 0x7efbec073aa0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efbdc009990 tx=0x7efbdc009ca0 comp rx=0 tx=0).stop 2023-12-07T10:19:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.639+0000 7efbf11ff700 1 -- 172.21.15.119:0/1154763609 shutdown_connections 2023-12-07T10:19:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.639+0000 7efbf11ff700 1 --2- 172.21.15.119:0/1154763609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbec109420 0x7efbec10b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.639+0000 7efbf11ff700 1 --2- 172.21.15.119:0/1154763609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efbec075650 0x7efbec073aa0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.639+0000 7efbf11ff700 1 --2- 172.21.15.119:0/1154763609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbec074070 0x7efbec0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.639+0000 7efbf11ff700 1 -- 172.21.15.119:0/1154763609 >> 172.21.15.119:0/1154763609 conn(0x7efbec0fcab0 msgr2=0x7efbec0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.639+0000 7efbf11ff700 1 -- 172.21.15.119:0/1154763609 shutdown_connections 2023-12-07T10:19:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.640+0000 7efbf11ff700 1 -- 172.21.15.119:0/1154763609 wait complete. 2023-12-07T10:19:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.640+0000 7efbf11ff700 1 Processor -- start 2023-12-07T10:19:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.641+0000 7efbf11ff700 1 -- start start 2023-12-07T10:19:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.641+0000 7efbf11ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efbec074070 0x7efbec199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.641+0000 7efbf11ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbec075650 0x7efbec199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.641+0000 7efbf11ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbec109420 0x7efbec1a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.641+0000 7efbf11ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbec1a11e0 con 0x7efbec075650 2023-12-07T10:19:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.641+0000 7efbf11ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbec1a1320 con 0x7efbec109420 2023-12-07T10:19:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.641+0000 7efbf11ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbec1a14b0 con 0x7efbec074070 2023-12-07T10:19:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.641+0000 7efbea59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbec075650 0x7efbec199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.641+0000 7efbeb59e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbec109420 0x7efbec1a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.641+0000 7efbead9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efbec074070 0x7efbec199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.641+0000 7efbead9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efbec074070 0x7efbec199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:32826/0 (socket says 172.21.15.119:32826) 2023-12-07T10:19:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.641+0000 7efbead9d700 1 -- 172.21.15.119:0/700796348 learned_addr learned my addr 172.21.15.119:0/700796348 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.641+0000 7efbeb59e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbec109420 0x7efbec1a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:40962/0 (socket says 172.21.15.119:40962) 2023-12-07T10:19:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.642+0000 7efbead9d700 1 -- 172.21.15.119:0/700796348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbec109420 msgr2=0x7efbec1a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.642+0000 7efbead9d700 1 --2- 172.21.15.119:0/700796348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbec109420 0x7efbec1a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.642+0000 7efbead9d700 1 -- 172.21.15.119:0/700796348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbec075650 msgr2=0x7efbec199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.642+0000 7efbead9d700 1 --2- 172.21.15.119:0/700796348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbec075650 0x7efbec199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.642+0000 7efbead9d700 1 -- 172.21.15.119:0/700796348 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbdc0095f0 con 0x7efbec074070 2023-12-07T10:19:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.642+0000 7efbeb59e700 1 --2- 172.21.15.119:0/700796348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbec109420 0x7efbec1a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.642+0000 7efbea59c700 1 --2- 172.21.15.119:0/700796348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbec075650 0x7efbec199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.642+0000 7efbead9d700 1 --2- 172.21.15.119:0/700796348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efbec074070 0x7efbec199700 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efbdc0098e0 tx=0x7efbdc005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.642+0000 7efbd3fff700 1 -- 172.21.15.119:0/700796348 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbdc01d070 con 0x7efbec074070 2023-12-07T10:19:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.642+0000 7efbf11ff700 1 -- 172.21.15.119:0/700796348 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbec1a1730 con 0x7efbec074070 2023-12-07T10:19:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.642+0000 7efbd3fff700 1 -- 172.21.15.119:0/700796348 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efbdc022470 con 0x7efbec074070 2023-12-07T10:19:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.642+0000 7efbd3fff700 1 -- 172.21.15.119:0/700796348 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbdc017650 con 0x7efbec074070 2023-12-07T10:19:28.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.642+0000 7efbf11ff700 1 -- 172.21.15.119:0/700796348 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbec1a1b50 con 0x7efbec074070 2023-12-07T10:19:28.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.643+0000 7efbf11ff700 1 -- 172.21.15.119:0/700796348 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbec1081e0 con 0x7efbec074070 2023-12-07T10:19:28.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.645+0000 7efbd3fff700 1 -- 172.21.15.119:0/700796348 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7efbdc0225e0 con 0x7efbec074070 2023-12-07T10:19:28.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.646+0000 7efbd3fff700 1 --2- 172.21.15.119:0/700796348 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7efbd40a1ba0 0x7efbd40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:28.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.647+0000 7efbd3fff700 1 -- 172.21.15.119:0/700796348 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7efbdc0cf5a0 con 0x7efbec074070 2023-12-07T10:19:28.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.647+0000 7efbea59c700 1 --2- 172.21.15.119:0/700796348 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7efbd40a1ba0 0x7efbd40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:28.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.647+0000 7efbea59c700 1 --2- 172.21.15.119:0/700796348 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7efbd40a1ba0 0x7efbd40a4060 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efbec075370 tx=0x7efbe0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.649+0000 7efbd3fff700 1 -- 172.21.15.119:0/700796348 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7efbdc0985b0 con 0x7efbec074070 2023-12-07T10:19:28.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.856+0000 7efbf11ff700 1 -- 172.21.15.119:0/700796348 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efbec068350 con 0x7efbec074070 2023-12-07T10:19:28.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.858+0000 7efbd3fff700 1 -- 172.21.15.119:0/700796348 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24715 (secure 0 0 0) 0x7efbdc09bbd0 con 0x7efbec074070 2023-12-07T10:19:28.860 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:19:28.860 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":93,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","created":"2023-12-07T10:09:59.945863+0000","modified":"2023-12-07T10:19:02.800626+0000","last_up_change":"2023-12-07T10:16:50.929427+0000","last_in_change":"2023-12-07T10:16:33.558158+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T10:13:25.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-07T10:17:39.347813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-07T10:17:40.714501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-07T10:17:44.257760+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"56b5818e-e625-40f1-8838-3ddcf2d34ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6803","nonce":2141950133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6805","nonce":2141950133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6809","nonce":2141950133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6807","nonce":2141950133}]},"public_addr":"172.21.15.119:6803/2141950133","cluster_addr":"172.21.15.119:6805/2141950133","heartbeat_back_addr":"172.21.15.119:6809/2141950133","heartbeat_front_addr":"172.21.15.119:6807/2141950133","state":["exists","up"]},{"osd":1,"uuid":"36489a6a-5278-4802-9525-9871521722e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6801","nonce":3390875370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6803","nonce":3390875370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6807","nonce":3390875370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6805","nonce":3390875370}]},"public_addr":"172.21.15.153:6801/3390875370","cluster_addr":"172.21.15.153:6803/3390875370","heartbeat_back_addr":"172.21.15.153:6807/3390875370","heartbeat_front_addr":"172.21.15.153:6805/3390875370","state":["exists","up"]},{"osd":2,"uuid":"a937ad26-5e9b-4c8a-b65a-626a8feda6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6801","nonce":708101637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6803","nonce":708101637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6807","nonce":708101637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6805","nonce":708101637}]},"public_addr":"172.21.15.159:6801/708101637","cluster_addr":"172.21.15.159:6803/708101637","heartbeat_back_addr":"172.21.15.159:6807/708101637","heartbeat_front_addr":"172.21.15.159:6805/708101637","state":["exists","up"]},{"osd":3,"uuid":"26ef8300-aa07-48f6-b31d-8b0f2217ffcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6811","nonce":3228047257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6813","nonce":3228047257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6817","nonce":3228047257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6815","nonce":3228047257}]},"public_addr":"172.21.15.119:6811/3228047257","cluster_addr":"172.21.15.119:6813/3228047257","heartbeat_back_addr":"172.21.15.119:6817/3228047257","heartbeat_front_addr":"172.21.15.119:6815/3228047257","state":["exists","up"]},{"osd":4,"uuid":"000ee4b9-9896-4901-8cc3-c85fdfc8cbaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6809","nonce":4021998299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6811","nonce":4021998299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6815","nonce":4021998299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6813","nonce":4021998299}]},"public_addr":"172.21.15.153:6809/4021998299","cluster_addr":"172.21.15.153:6811/4021998299","heartbeat_back_addr":"172.21.15.153:6815/4021998299","heartbeat_front_addr":"172.21.15.153:6813/4021998299","state":["exists","up"]},{"osd":5,"uuid":"8b2fc167-262d-474f-89f3-47bd29424421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6809","nonce":1832540768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6811","nonce":1832540768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6815","nonce":1832540768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6813","nonce":1832540768}]},"public_addr":"172.21.15.159:6809/1832540768","cluster_addr":"172.21.15.159:6811/1832540768","heartbeat_back_addr":"172.21.15.159:6815/1832540768","heartbeat_front_addr":"172.21.15.159:6813/1832540768","state":["exists","up"]},{"osd":6,"uuid":"f9aa6749-46cf-4b2a-8114-cbd034219d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162436840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162436840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6825","nonce":1162436840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162436840}]},"public_addr":"172.21.15.119:6819/1162436840","cluster_addr":"172.21.15.119:6821/1162436840","heartbeat_back_addr":"172.21.15.119:6825/1162436840","heartbeat_front_addr":"172.21.15.119:6823/1162436840","state":["exists","up"]},{"osd":7,"uuid":"39216361-e1c4-4fef-86cb-a4707656b547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6817","nonce":3254653457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6819","nonce":3254653457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6823","nonce":3254653457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6821","nonce":3254653457}]},"public_addr":"172.21.15.153:6817/3254653457","cluster_addr":"172.21.15.153:6819/3254653457","heartbeat_back_addr":"172.21.15.153:6823/3254653457","heartbeat_front_addr":"172.21.15.153:6821/3254653457","state":["exists","up"]},{"osd":8,"uuid":"cceb6ebc-7c02-48ea-9d16-b7a016dd449b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6817","nonce":2600946380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6819","nonce":2600946380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6823","nonce":2600946380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6821","nonce":2600946380}]},"public_addr":"172.21.15.159:6817/2600946380","cluster_addr":"172.21.15.159:6819/2600946380","heartbeat_back_addr":"172.21.15.159:6823/2600946380","heartbeat_front_addr":"172.21.15.159:6821/2600946380","state":["exists","up"]},{"osd":9,"uuid":"f337a2e4-e065-4a9f-91d9-e3615de2d6f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6827","nonce":1940737285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6829","nonce":1940737285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6833","nonce":1940737285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6831","nonce":1940737285}]},"public_addr":"172.21.15.119:6827/1940737285","cluster_addr":"172.21.15.119:6829/1940737285","heartbeat_back_addr":"172.21.15.119:6833/1940737285","heartbeat_front_addr":"172.21.15.119:6831/1940737285","state":["exists","up"]},{"osd":10,"uuid":"3a3bc347-85f4-4878-9797-d9da96137e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6825","nonce":663834864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6827","nonce":663834864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6831","nonce":663834864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6829","nonce":663834864}]},"public_addr":"172.21.15.153:6825/663834864","cluster_addr":"172.21.15.153:6827/663834864","heartbeat_back_addr":"172.21.15.153:6831/663834864","heartbeat_front_addr":"172.21.15.153:6829/663834864","state":["exists","up"]},{"osd":11,"uuid":"a0fc5909-279d-4172-977c-2dc2da0a7e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6825","nonce":1262216348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6827","nonce":1262216348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6831","nonce":1262216348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6829","nonce":1262216348}]},"public_addr":"172.21.15.159:6825/1262216348","cluster_addr":"172.21.15.159:6827/1262216348","heartbeat_back_addr":"172.21.15.159:6831/1262216348","heartbeat_front_addr":"172.21.15.159:6829/1262216348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:37.488109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:59.543060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:21.381585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:47.112729+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:09.732150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:30.893729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:56.678710+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:19.004256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:40.388818+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:06.391734+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:28.394053+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:49.634348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1628639392":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6801/1355617847":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6800/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/362307836":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/128964558":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/2278813554":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/1411933614":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/1685698277":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6800/1355617847":"2023-12-08T10:11:00.585565+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T10:19:28.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.861+0000 7efbf11ff700 1 -- 172.21.15.119:0/700796348 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7efbd40a1ba0 msgr2=0x7efbd40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:28.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.861+0000 7efbf11ff700 1 --2- 172.21.15.119:0/700796348 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7efbd40a1ba0 0x7efbd40a4060 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efbec075370 tx=0x7efbe0009260 comp rx=0 tx=0).stop 2023-12-07T10:19:28.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.861+0000 7efbf11ff700 1 -- 172.21.15.119:0/700796348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efbec074070 msgr2=0x7efbec199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:28.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.861+0000 7efbf11ff700 1 --2- 172.21.15.119:0/700796348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efbec074070 0x7efbec199700 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efbdc0098e0 tx=0x7efbdc005720 comp rx=0 tx=0).stop 2023-12-07T10:19:28.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.861+0000 7efbf11ff700 1 -- 172.21.15.119:0/700796348 shutdown_connections 2023-12-07T10:19:28.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.861+0000 7efbf11ff700 1 --2- 172.21.15.119:0/700796348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbec109420 0x7efbec1a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:28.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.861+0000 7efbf11ff700 1 --2- 172.21.15.119:0/700796348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efbec074070 0x7efbec199700 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:28.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.861+0000 7efbf11ff700 1 --2- 172.21.15.119:0/700796348 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7efbd40a1ba0 0x7efbd40a4060 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:28.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.861+0000 7efbf11ff700 1 --2- 172.21.15.119:0/700796348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbec075650 0x7efbec199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:28.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.861+0000 7efbf11ff700 1 -- 172.21.15.119:0/700796348 >> 172.21.15.119:0/700796348 conn(0x7efbec0fcab0 msgr2=0x7efbec105320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:28.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.861+0000 7efbf11ff700 1 -- 172.21.15.119:0/700796348 shutdown_connections 2023-12-07T10:19:28.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:28.861+0000 7efbf11ff700 1 -- 172.21.15.119:0/700796348 wait complete. 2023-12-07T10:19:28.883 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:19:29.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.080+0000 7f7b047ff700 1 -- 172.21.15.119:0/221227091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7afc1030f0 msgr2=0x7f7afc1099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:29.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.080+0000 7f7b047ff700 1 --2- 172.21.15.119:0/221227091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7afc1030f0 0x7f7afc1099a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7af400b0d0 tx=0x7f7af400b3e0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.081+0000 7f7b047ff700 1 -- 172.21.15.119:0/221227091 shutdown_connections 2023-12-07T10:19:29.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.081+0000 7f7b047ff700 1 --2- 172.21.15.119:0/221227091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7afc1030f0 0x7f7afc1099a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.081+0000 7f7b047ff700 1 --2- 172.21.15.119:0/221227091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7afc101560 0x7f7afc101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.081+0000 7f7b047ff700 1 --2- 172.21.15.119:0/221227091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7afc102750 0x7f7afc102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.081+0000 7f7b047ff700 1 -- 172.21.15.119:0/221227091 >> 172.21.15.119:0/221227091 conn(0x7f7afc0fcab0 msgr2=0x7f7afc0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:29.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.081+0000 7f7b047ff700 1 -- 172.21.15.119:0/221227091 shutdown_connections 2023-12-07T10:19:29.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.081+0000 7f7b047ff700 1 -- 172.21.15.119:0/221227091 wait complete. 2023-12-07T10:19:29.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.082+0000 7f7b047ff700 1 Processor -- start 2023-12-07T10:19:29.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.082+0000 7f7b047ff700 1 -- start start 2023-12-07T10:19:29.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.082+0000 7f7b047ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7afc101560 0x7f7afc1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.082+0000 7f7b047ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7afc102750 0x7f7afc199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.082+0000 7f7b047ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7afc1030f0 0x7f7afc1a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.082+0000 7f7b047ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc1a1310 con 0x7f7afc1030f0 2023-12-07T10:19:29.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.082+0000 7f7b047ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc1a1450 con 0x7f7afc101560 2023-12-07T10:19:29.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.082+0000 7f7b047ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc1a15e0 con 0x7f7afc102750 2023-12-07T10:19:29.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.082+0000 7f7b02d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7afc1030f0 0x7f7afc1a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.082+0000 7f7b0259b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7afc101560 0x7f7afc1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.082+0000 7f7b02d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7afc1030f0 0x7f7afc1a0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51988/0 (socket says 172.21.15.119:51988) 2023-12-07T10:19:29.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.082+0000 7f7b02d9c700 1 -- 172.21.15.119:0/1851376352 learned_addr learned my addr 172.21.15.119:0/1851376352 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:29.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.082+0000 7f7b01d9a700 1 --2- 172.21.15.119:0/1851376352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7afc102750 0x7f7afc199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.083+0000 7f7b02d9c700 1 -- 172.21.15.119:0/1851376352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7afc102750 msgr2=0x7f7afc199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:29.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.083+0000 7f7b02d9c700 1 --2- 172.21.15.119:0/1851376352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7afc102750 0x7f7afc199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.083+0000 7f7b02d9c700 1 -- 172.21.15.119:0/1851376352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7afc101560 msgr2=0x7f7afc1997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:29.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.083+0000 7f7b02d9c700 1 --2- 172.21.15.119:0/1851376352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7afc101560 0x7f7afc1997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.083+0000 7f7b02d9c700 1 -- 172.21.15.119:0/1851376352 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af4009d00 con 0x7f7afc1030f0 2023-12-07T10:19:29.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.083+0000 7f7b02d9c700 1 --2- 172.21.15.119:0/1851376352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7afc1030f0 0x7f7afc1a0d40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7af4007b60 tx=0x7f7af4008e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:29.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.083+0000 7f7af37fe700 1 -- 172.21.15.119:0/1851376352 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af400e040 con 0x7f7afc1030f0 2023-12-07T10:19:29.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.083+0000 7f7af37fe700 1 -- 172.21.15.119:0/1851376352 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7af401d930 con 0x7f7afc1030f0 2023-12-07T10:19:29.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.083+0000 7f7af37fe700 1 -- 172.21.15.119:0/1851376352 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af401b9f0 con 0x7f7afc1030f0 2023-12-07T10:19:29.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.083+0000 7f7b047ff700 1 -- 172.21.15.119:0/1851376352 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7afc1a1860 con 0x7f7afc1030f0 2023-12-07T10:19:29.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.083+0000 7f7b047ff700 1 -- 172.21.15.119:0/1851376352 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7afc1a1c90 con 0x7f7afc1030f0 2023-12-07T10:19:29.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.084+0000 7f7b047ff700 1 -- 172.21.15.119:0/1851376352 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7afc068350 con 0x7f7afc1030f0 2023-12-07T10:19:29.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.086+0000 7f7af37fe700 1 -- 172.21.15.119:0/1851376352 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f7af401daa0 con 0x7f7afc1030f0 2023-12-07T10:19:29.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.088+0000 7f7af37fe700 1 --2- 172.21.15.119:0/1851376352 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f7ae80a1c20 0x7f7ae80a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.088+0000 7f7af37fe700 1 -- 172.21.15.119:0/1851376352 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f7af40cf810 con 0x7f7afc1030f0 2023-12-07T10:19:29.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.088+0000 7f7b0259b700 1 --2- 172.21.15.119:0/1851376352 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f7ae80a1c20 0x7f7ae80a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.088+0000 7f7b0259b700 1 --2- 172.21.15.119:0/1851376352 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f7ae80a1c20 0x7f7ae80a40e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7afc1025b0 tx=0x7f7af8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:29.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.090+0000 7f7af37fe700 1 -- 172.21.15.119:0/1851376352 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7af40987f0 con 0x7f7afc1030f0 2023-12-07T10:19:29.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:28 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3001650339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:19:29.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:28 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/136502542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:19:29.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:28 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/700796348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:19:29.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:28 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3001650339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:19:29.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:28 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/136502542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:19:29.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:28 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/700796348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:19:29.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:28 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3001650339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:19:29.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:28 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/136502542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:19:29.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:28 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/700796348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:19:29.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.333+0000 7f7b047ff700 1 -- 172.21.15.119:0/1851376352 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7afc10cdd0 con 0x7f7afc1030f0 2023-12-07T10:19:29.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.334+0000 7f7af37fe700 1 -- 172.21.15.119:0/1851376352 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11385 (secure 0 0 0) 0x7f7af409be10 con 0x7f7afc1030f0 2023-12-07T10:19:29.335 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:19:29.335 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-07T10:17:41.278311+0000","modified":"2023-12-07T10:18:18.543356+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24517,"mds_3":24505,"mds_4":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14535":{"gid":14535,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6841/1791998028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1791998028},{"type":"v1","addr":"172.21.15.119:6841","nonce":1791998028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24505":{"gid":24505,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/2925503246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2925503246},{"type":"v1","addr":"172.21.15.153:6837","nonce":2925503246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6839/3984962900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3984962900},{"type":"v1","addr":"172.21.15.159:6839","nonce":3984962900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.119:6839/1920762602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1920762602},{"type":"v1","addr":"172.21.15.119:6839","nonce":1920762602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/566825184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":566825184},{"type":"v1","addr":"172.21.15.153:6839","nonce":566825184}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-07T10:19:29.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.336+0000 7f7b047ff700 1 -- 172.21.15.119:0/1851376352 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f7ae80a1c20 msgr2=0x7f7ae80a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:29.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.336+0000 7f7b047ff700 1 --2- 172.21.15.119:0/1851376352 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f7ae80a1c20 0x7f7ae80a40e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7afc1025b0 tx=0x7f7af8009260 comp rx=0 tx=0).stop 2023-12-07T10:19:29.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.336+0000 7f7b047ff700 1 -- 172.21.15.119:0/1851376352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7afc1030f0 msgr2=0x7f7afc1a0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:29.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.336+0000 7f7b047ff700 1 --2- 172.21.15.119:0/1851376352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7afc1030f0 0x7f7afc1a0d40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7af4007b60 tx=0x7f7af4008e40 comp rx=0 tx=0).stop 2023-12-07T10:19:29.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.336+0000 7f7b047ff700 1 -- 172.21.15.119:0/1851376352 shutdown_connections 2023-12-07T10:19:29.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.336+0000 7f7b047ff700 1 --2- 172.21.15.119:0/1851376352 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f7ae80a1c20 0x7f7ae80a40e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.336+0000 7f7b047ff700 1 --2- 172.21.15.119:0/1851376352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7afc1030f0 0x7f7afc1a0d40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.336+0000 7f7b047ff700 1 --2- 172.21.15.119:0/1851376352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7afc101560 0x7f7afc1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.336+0000 7f7b047ff700 1 --2- 172.21.15.119:0/1851376352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7afc102750 0x7f7afc199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.336+0000 7f7b047ff700 1 -- 172.21.15.119:0/1851376352 >> 172.21.15.119:0/1851376352 conn(0x7f7afc0fcab0 msgr2=0x7f7afc073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:29.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.336+0000 7f7b047ff700 1 -- 172.21.15.119:0/1851376352 shutdown_connections 2023-12-07T10:19:29.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.336+0000 7f7b047ff700 1 -- 172.21.15.119:0/1851376352 wait complete. 2023-12-07T10:19:29.339 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 29 2023-12-07T10:19:29.357 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-12-07T10:17:41.278311+0000', 'modified': '2023-12-07T10:18:18.543356+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24566, 'mds_1': 24542, 'mds_2': 24517, 'mds_3': 24505, 'mds_4': 24488}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14535': {'gid': 14535, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.119:6841/1791998028', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6840', 'nonce': 1791998028}, {'type': 'v1', 'addr': '172.21.15.119:6841', 'nonce': 1791998028}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24445': {'gid': 24445, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6833/935461464', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6832', 'nonce': 935461464}, {'type': 'v1', 'addr': '172.21.15.159:6833', 'nonce': 935461464}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24452': {'gid': 24452, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.153:6833/209059922', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6832', 'nonce': 209059922}, {'type': 'v1', 'addr': '172.21.15.153:6833', 'nonce': 209059922}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24488': {'gid': 24488, 'name': 'e', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.153:6835/1295649464', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6834', 'nonce': 1295649464}, {'type': 'v1', 'addr': '172.21.15.153:6835', 'nonce': 1295649464}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24505': {'gid': 24505, 'name': 'f', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.159:6835/2406380792', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 2406380792}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 2406380792}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24517': {'gid': 24517, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.159:6837/3106097520', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6836', 'nonce': 3106097520}, {'type': 'v1', 'addr': '172.21.15.159:6837', 'nonce': 3106097520}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24518': {'gid': 24518, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.153:6837/2925503246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6836', 'nonce': 2925503246}, {'type': 'v1', 'addr': '172.21.15.153:6837', 'nonce': 2925503246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24538': {'gid': 24538, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6839/3984962900', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6838', 'nonce': 3984962900}, {'type': 'v1', 'addr': '172.21.15.159:6839', 'nonce': 3984962900}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24542': {'gid': 24542, 'name': 'g', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.119:6839/1920762602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6838', 'nonce': 1920762602}, {'type': 'v1', 'addr': '172.21.15.119:6839', 'nonce': 1920762602}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24566': {'gid': 24566, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.153:6839/566825184', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6838', 'nonce': 566825184}, {'type': 'v1', 'addr': '172.21.15.153:6839', 'nonce': 566825184}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-07T10:19:29.358 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-12-07T10:19:29.358 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-12-07T10:19:29.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.554+0000 7f0f4afff700 1 -- 172.21.15.119:0/3713329936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f44102730 msgr2=0x7f0f44102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:29.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.554+0000 7f0f4afff700 1 --2- 172.21.15.119:0/3713329936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f44102730 0x7f0f44102b90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0f300099c0 tx=0x7f0f30009cd0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.554+0000 7f0f4afff700 1 -- 172.21.15.119:0/3713329936 shutdown_connections 2023-12-07T10:19:29.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.554+0000 7f0f4afff700 1 --2- 172.21.15.119:0/3713329936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f441030d0 0x7f0f44109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.554+0000 7f0f4afff700 1 --2- 172.21.15.119:0/3713329936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f44101540 0x7f0f44101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.554+0000 7f0f4afff700 1 --2- 172.21.15.119:0/3713329936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f44102730 0x7f0f44102b90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.554+0000 7f0f4afff700 1 -- 172.21.15.119:0/3713329936 >> 172.21.15.119:0/3713329936 conn(0x7f0f440fcab0 msgr2=0x7f0f440feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:29.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.554+0000 7f0f4afff700 1 -- 172.21.15.119:0/3713329936 shutdown_connections 2023-12-07T10:19:29.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.554+0000 7f0f4afff700 1 -- 172.21.15.119:0/3713329936 wait complete. 2023-12-07T10:19:29.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.555+0000 7f0f4afff700 1 Processor -- start 2023-12-07T10:19:29.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.555+0000 7f0f4afff700 1 -- start start 2023-12-07T10:19:29.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.556+0000 7f0f4afff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f44101540 0x7f0f44075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.556+0000 7f0f4afff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f44102730 0x7f0f44073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.556+0000 7f0f4afff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f441030d0 0x7f0f44073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.556+0000 7f0f4afff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f440746e0 con 0x7f0f44101540 2023-12-07T10:19:29.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.556+0000 7f0f4afff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f44074850 con 0x7f0f44102730 2023-12-07T10:19:29.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.556+0000 7f0f4afff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f441a1510 con 0x7f0f441030d0 2023-12-07T10:19:29.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.556+0000 7f0f48d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f44101540 0x7f0f44075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.556+0000 7f0f48d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f44101540 0x7f0f44075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52018/0 (socket says 172.21.15.119:52018) 2023-12-07T10:19:29.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.556+0000 7f0f48d9b700 1 -- 172.21.15.119:0/24079923 learned_addr learned my addr 172.21.15.119:0/24079923 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:29.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.556+0000 7f0f4959c700 1 --2- 172.21.15.119:0/24079923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f441030d0 0x7f0f44073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.556+0000 7f0f43fff700 1 --2- 172.21.15.119:0/24079923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f44102730 0x7f0f44073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.556+0000 7f0f48d9b700 1 -- 172.21.15.119:0/24079923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f441030d0 msgr2=0x7f0f44073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:29.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.556+0000 7f0f48d9b700 1 --2- 172.21.15.119:0/24079923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f441030d0 0x7f0f44073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.556+0000 7f0f48d9b700 1 -- 172.21.15.119:0/24079923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f44102730 msgr2=0x7f0f44073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:29.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.556+0000 7f0f48d9b700 1 --2- 172.21.15.119:0/24079923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f44102730 0x7f0f44073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.556+0000 7f0f48d9b700 1 -- 172.21.15.119:0/24079923 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3c009c40 con 0x7f0f44101540 2023-12-07T10:19:29.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.556+0000 7f0f48d9b700 1 --2- 172.21.15.119:0/24079923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f44101540 0x7f0f44075370 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800e970 tx=0x7f0f3800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:29.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.557+0000 7f0f41ffb700 1 -- 172.21.15.119:0/24079923 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f38009790 con 0x7f0f44101540 2023-12-07T10:19:29.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.557+0000 7f0f41ffb700 1 -- 172.21.15.119:0/24079923 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f3800eeb0 con 0x7f0f44101540 2023-12-07T10:19:29.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.557+0000 7f0f41ffb700 1 -- 172.21.15.119:0/24079923 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f38003cb0 con 0x7f0f44101540 2023-12-07T10:19:29.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.557+0000 7f0f4afff700 1 -- 172.21.15.119:0/24079923 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f300095f0 con 0x7f0f44101540 2023-12-07T10:19:29.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.557+0000 7f0f4afff700 1 -- 172.21.15.119:0/24079923 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f441a1b20 con 0x7f0f44101540 2023-12-07T10:19:29.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.560+0000 7f0f41ffb700 1 -- 172.21.15.119:0/24079923 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f0f38003e10 con 0x7f0f44101540 2023-12-07T10:19:29.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.561+0000 7f0f5ea18b80 1 Processor -- start 2023-12-07T10:19:29.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.561+0000 7f0f5ea18b80 1 -- start start 2023-12-07T10:19:29.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.561+0000 7f0f5ea18b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faa5ef5130 0x55faa5ef5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.561+0000 7f0f5ea18b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faa5ef5a90 0x55faa5ef63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.561+0000 7f0f5ea18b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa5ef6980 0x55faa5ef6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.561+0000 7f0f5ea18b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa5e69770 con 0x55faa5ef5a90 2023-12-07T10:19:29.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.561+0000 7f0f5ea18b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa5e50750 con 0x55faa5ef6980 2023-12-07T10:19:29.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.561+0000 7f0f5ea18b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa5e4e070 con 0x55faa5ef5130 2023-12-07T10:19:29.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.561+0000 7f0f41ffb700 1 --2- 172.21.15.119:0/24079923 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0f340a1c20 0x7f0f340a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.562+0000 7f0f48d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faa5ef5a90 0x55faa5ef63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.562+0000 7f0f41ffb700 1 -- 172.21.15.119:0/24079923 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f0f38014070 con 0x7f0f44101540 2023-12-07T10:19:29.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.562+0000 7f0f43fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa5ef6980 0x55faa5ef6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.562+0000 7f0f48d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faa5ef5a90 0x55faa5ef63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52026/0 (socket says 172.21.15.119:52026) 2023-12-07T10:19:29.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.562+0000 7f0f48d9b700 1 -- 172.21.15.119:0/2164822883 learned_addr learned my addr 172.21.15.119:0/2164822883 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:29.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.562+0000 7f0f4959c700 1 --2- 172.21.15.119:0/24079923 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0f340a1c20 0x7f0f340a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.562+0000 7f0f4959c700 1 --2- 172.21.15.119:0/2164822883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faa5ef5130 0x55faa5ef5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.562+0000 7f0f43fff700 1 -- 172.21.15.119:0/2164822883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faa5ef5130 msgr2=0x55faa5ef5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:29.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.562+0000 7f0f43fff700 1 --2- 172.21.15.119:0/2164822883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faa5ef5130 0x55faa5ef5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.562+0000 7f0f43fff700 1 -- 172.21.15.119:0/2164822883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faa5ef5a90 msgr2=0x55faa5ef63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:29.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.562+0000 7f0f43fff700 1 --2- 172.21.15.119:0/2164822883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faa5ef5a90 0x55faa5ef63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.562+0000 7f0f43fff700 1 -- 172.21.15.119:0/2164822883 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55faa5cc65e0 con 0x55faa5ef6980 2023-12-07T10:19:29.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.562+0000 7f0f48d9b700 1 --2- 172.21.15.119:0/2164822883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faa5ef5a90 0x55faa5ef63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:29.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.562+0000 7f0f43fff700 1 --2- 172.21.15.119:0/2164822883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa5ef6980 0x55faa5ef6e00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0f300098e0 tx=0x7f0f30005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfd833be36fc949b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:29.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.562+0000 7f0f4959c700 1 --2- 172.21.15.119:0/24079923 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0f340a1c20 0x7f0f340a40e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009fd0 tx=0x7f0f3c009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:29.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.563+0000 7f0f2effd700 1 -- 172.21.15.119:0/2164822883 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f30023ad0 con 0x55faa5ef6980 2023-12-07T10:19:29.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.563+0000 7f0f2effd700 1 -- 172.21.15.119:0/2164822883 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f30005530 con 0x55faa5ef6980 2023-12-07T10:19:29.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.563+0000 7f0f2effd700 1 -- 172.21.15.119:0/2164822883 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3000bc50 con 0x55faa5ef6980 2023-12-07T10:19:29.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.563+0000 7f0f5ea18b80 1 -- 172.21.15.119:0/2164822883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa5ef6980 msgr2=0x55faa5ef6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:29.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.563+0000 7f0f5ea18b80 1 --2- 172.21.15.119:0/2164822883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa5ef6980 0x55faa5ef6e00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0f300098e0 tx=0x7f0f30005e80 comp rx=0 tx=0).stop 2023-12-07T10:19:29.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.563+0000 7f0f5ea18b80 1 -- 172.21.15.119:0/2164822883 shutdown_connections 2023-12-07T10:19:29.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.563+0000 7f0f5ea18b80 1 --2- 172.21.15.119:0/2164822883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa5ef6980 0x55faa5ef6e00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.563+0000 7f0f5ea18b80 1 --2- 172.21.15.119:0/2164822883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faa5ef5130 0x55faa5ef5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.563+0000 7f0f5ea18b80 1 --2- 172.21.15.119:0/2164822883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faa5ef5a90 0x55faa5ef63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.563+0000 7f0f5ea18b80 1 -- 172.21.15.119:0/2164822883 >> 172.21.15.119:0/2164822883 conn(0x55faa5e24310 msgr2=0x55faa5ef1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:29.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.564+0000 7f0f5ea18b80 1 -- 172.21.15.119:0/2164822883 shutdown_connections 2023-12-07T10:19:29.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.564+0000 7f0f5ea18b80 1 -- 172.21.15.119:0/2164822883 wait complete. 2023-12-07T10:19:29.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.564+0000 7f0f5ea18b80 1 -- start start 2023-12-07T10:19:29.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.564+0000 7f0f5ea18b80 1 Processor -- start 2023-12-07T10:19:29.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.565+0000 7f0f5ea18b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa5ef5130 0x55faa5efef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.565+0000 7f0f5ea18b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faa5ef5a90 0x55faa5eff470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.565+0000 7f0f5ea18b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faa5ef6980 0x55faa5f064b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.565+0000 7f0f5ea18b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa5e49960 con 0x55faa5ef6980 2023-12-07T10:19:29.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.565+0000 7f0f5ea18b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa5e03510 con 0x55faa5ef5130 2023-12-07T10:19:29.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.565+0000 7f0f5ea18b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa5e031b0 con 0x55faa5ef5a90 2023-12-07T10:19:29.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.565+0000 7f0f5ea18b80 10 client.0 _ll_register_callbacks cb 0x55faa5dea8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:19:29.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.565+0000 7f0f43fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faa5ef5a90 0x55faa5eff470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.565+0000 7f0f4959c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faa5ef6980 0x55faa5f064b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.565+0000 7f0f43fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faa5ef5a90 0x55faa5eff470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:32902/0 (socket says 172.21.15.119:32902) 2023-12-07T10:19:29.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.565+0000 7f0f43fff700 1 -- 172.21.15.119:0/1711594198 learned_addr learned my addr 172.21.15.119:0/1711594198 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:29.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.565+0000 7f0f48d9b700 1 --2- 172.21.15.119:0/1711594198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa5ef5130 0x55faa5efef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.565+0000 7f0f43fff700 1 -- 172.21.15.119:0/1711594198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa5ef5130 msgr2=0x55faa5efef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:29.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.565+0000 7f0f43fff700 1 --2- 172.21.15.119:0/1711594198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa5ef5130 0x55faa5efef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.565+0000 7f0f43fff700 1 -- 172.21.15.119:0/1711594198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faa5ef6980 msgr2=0x55faa5f064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:29.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.565+0000 7f0f43fff700 1 --2- 172.21.15.119:0/1711594198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faa5ef6980 0x55faa5f064b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.565+0000 7f0f43fff700 1 -- 172.21.15.119:0/1711594198 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f441a1b20 con 0x55faa5ef5a90 2023-12-07T10:19:29.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.566+0000 7f0f43fff700 1 --2- 172.21.15.119:0/1711594198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faa5ef5a90 0x55faa5eff470 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0f30000c00 tx=0x7f0f30023b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eedd6240d54970f1 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:29.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.566+0000 7f0f23fff700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:19:29.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.566+0000 7f0f23fff700 1 -- 172.21.15.119:0/1711594198 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f300056a0 con 0x55faa5ef5a90 2023-12-07T10:19:29.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.566+0000 7f0f5ea18b80 1 -- 172.21.15.119:0/1711594198 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55faa5ee8ab0 con 0x55faa5ef5a90 2023-12-07T10:19:29.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.566+0000 7f0f23fff700 1 -- 172.21.15.119:0/1711594198 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f0f30023e20 con 0x55faa5ef5a90 2023-12-07T10:19:29.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.566+0000 7f0f23fff700 1 -- 172.21.15.119:0/1711594198 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f300052e0 con 0x55faa5ef5a90 2023-12-07T10:19:29.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.566+0000 7f0f5ea18b80 10 client.25141 fetch_fsmap learned FSMap version 29 2023-12-07T10:19:29.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.566+0000 7f0f5ea18b80 1 -- 172.21.15.119:0/1711594198 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f0f4410a2d0 con 0x55faa5ef5a90 2023-12-07T10:19:29.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.567+0000 7f0f23fff700 1 -- 172.21.15.119:0/1711594198 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55faa5ee8ab0 con 0x55faa5ef5a90 2023-12-07T10:19:29.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.567+0000 7f0f23fff700 1 -- 172.21.15.119:0/1711594198 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f0f3000f610 con 0x55faa5ef5a90 2023-12-07T10:19:29.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.567+0000 7f0f5ea18b80 10 client.25141 fetch_fsmap finished waiting for FSMap version 29 2023-12-07T10:19:29.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.567+0000 7f0f5ea18b80 10 client.25141 resolve_mds: resolved name 'e' to daemon mds.e 2023-12-07T10:19:29.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.567+0000 7f0f5ea18b80 20 client.25141 populate_metadata read hostname 'smithi119' 2023-12-07T10:19:29.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.567+0000 7f0f5ea18b80 1 --2- 172.21.15.119:0/1711594198 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x55faa5f0a610 0x55faa5f09480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.567+0000 7f0f5ea18b80 4 client.25141 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:19:29.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.567+0000 7f0f5ea18b80 1 -- 172.21.15.119:0/1711594198 --> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55faa5ec4750 con 0x55faa5f0a610 2023-12-07T10:19:29.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.567+0000 7f0f48d9b700 1 --2- 172.21.15.119:0/1711594198 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x55faa5f0a610 0x55faa5f09480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.568+0000 7f0f48d9b700 1 --2- 172.21.15.119:0/1711594198 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x55faa5f0a610 0x55faa5f09480 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4888d5a87ce44d35 server_cookie=29a13ba98a267bb in_seq=0 out_seq=0 2023-12-07T10:19:29.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.568+0000 7f0f23fff700 10 client.25141 ms_handle_connect on v2:172.21.15.153:6834/1295649464 2023-12-07T10:19:29.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.568+0000 7f0f23fff700 1 -- 172.21.15.119:0/1711594198 <== mds.4 v2:172.21.15.153:6834/1295649464 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55faa5ec4750 con 0x55faa5f0a610 2023-12-07T10:19:29.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.568+0000 7f0f23fff700 10 client.25141 handle_command_reply: tid=0 2023-12-07T10:19:29.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.569+0000 7f0f5ea18b80 1 client.25141 shutdown 2023-12-07T10:19:29.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.569+0000 7f0f5ea18b80 1 -- 172.21.15.119:0/1711594198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faa5ef5a90 msgr2=0x55faa5eff470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:29.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.569+0000 7f0f5ea18b80 1 --2- 172.21.15.119:0/1711594198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faa5ef5a90 0x55faa5eff470 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0f30000c00 tx=0x7f0f30023b80 comp rx=0 tx=0).stop 2023-12-07T10:19:29.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.569+0000 7f0f5ea18b80 1 -- 172.21.15.119:0/1711594198 shutdown_connections 2023-12-07T10:19:29.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.569+0000 7f0f5ea18b80 1 --2- 172.21.15.119:0/1711594198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faa5ef6980 0x55faa5f064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.569+0000 7f0f5ea18b80 1 --2- 172.21.15.119:0/1711594198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa5ef5130 0x55faa5efef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.569+0000 7f0f5ea18b80 1 --2- 172.21.15.119:0/1711594198 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x55faa5f0a610 0x55faa5f09480 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.569+0000 7f0f5ea18b80 1 --2- 172.21.15.119:0/1711594198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faa5ef5a90 0x55faa5eff470 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.569+0000 7f0f23fff700 0 client.25141 ms_handle_reset on v2:172.21.15.153:6834/1295649464 2023-12-07T10:19:29.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.569+0000 7f0f5ea18b80 1 -- 172.21.15.119:0/1711594198 >> 172.21.15.119:0/1711594198 conn(0x55faa5e24310 msgr2=0x55faa5ef0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:29.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.569+0000 7f0f5ea18b80 1 -- 172.21.15.119:0/1711594198 shutdown_connections 2023-12-07T10:19:29.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.569+0000 7f0f5ea18b80 1 -- 172.21.15.119:0/1711594198 wait complete. 2023-12-07T10:19:29.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.569+0000 7f0f5ea18b80 20 client.25141 trim_cache size 0 max 16384 2023-12-07T10:19:29.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.587+0000 7f0f5ea18b80 1 Processor -- start 2023-12-07T10:19:29.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.588+0000 7f0f5ea18b80 1 -- start start 2023-12-07T10:19:29.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.588+0000 7f0f5ea18b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faa5ef5130 0x55faa5efdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.588+0000 7f0f5ea18b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa5ef5a90 0x55faa5f063d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.588+0000 7f0f5ea18b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faa5ef6980 0x55faa5f12fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.588+0000 7f0f5ea18b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa5cc88d0 con 0x55faa5ef5130 2023-12-07T10:19:29.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.588+0000 7f0f5ea18b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa5e031b0 con 0x55faa5ef5a90 2023-12-07T10:19:29.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.588+0000 7f0f5ea18b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa5ef11d0 con 0x55faa5ef6980 2023-12-07T10:19:29.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.588+0000 7f0f48d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faa5ef5130 0x55faa5efdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.588+0000 7f0f48d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faa5ef5130 0x55faa5efdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52032/0 (socket says 172.21.15.119:52032) 2023-12-07T10:19:29.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.588+0000 7f0f48d9b700 1 -- 172.21.15.119:0/2534295984 learned_addr learned my addr 172.21.15.119:0/2534295984 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:29.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.588+0000 7f0f43fff700 1 --2- 172.21.15.119:0/2534295984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa5ef5a90 0x55faa5f063d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.588+0000 7f0f4959c700 1 --2- 172.21.15.119:0/2534295984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faa5ef6980 0x55faa5f12fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.588+0000 7f0f48d9b700 1 -- 172.21.15.119:0/2534295984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faa5ef6980 msgr2=0x55faa5f12fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:29.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.588+0000 7f0f48d9b700 1 --2- 172.21.15.119:0/2534295984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faa5ef6980 0x55faa5f12fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.589+0000 7f0f48d9b700 1 -- 172.21.15.119:0/2534295984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa5ef5a90 msgr2=0x55faa5f063d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:29.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.589+0000 7f0f48d9b700 1 --2- 172.21.15.119:0/2534295984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa5ef5a90 0x55faa5f063d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.589+0000 7f0f48d9b700 1 -- 172.21.15.119:0/2534295984 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55faa5c72d90 con 0x55faa5ef5130 2023-12-07T10:19:29.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.589+0000 7f0f43fff700 1 --2- 172.21.15.119:0/2534295984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa5ef5a90 0x55faa5f063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:29.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.589+0000 7f0f48d9b700 1 --2- 172.21.15.119:0/2534295984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faa5ef5130 0x55faa5efdc10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0f380972a0 tx=0x7f0f38007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210937b0c42a0fb4 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:29.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.589+0000 7f0f2f7fe700 1 -- 172.21.15.119:0/2534295984 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3809a190 con 0x55faa5ef5130 2023-12-07T10:19:29.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.589+0000 7f0f2f7fe700 1 -- 172.21.15.119:0/2534295984 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f38007e40 con 0x55faa5ef5130 2023-12-07T10:19:29.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.589+0000 7f0f2f7fe700 1 -- 172.21.15.119:0/2534295984 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f380850b0 con 0x55faa5ef5130 2023-12-07T10:19:29.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.590+0000 7f0f5ea18b80 1 -- 172.21.15.119:0/2534295984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faa5ef5130 msgr2=0x55faa5efdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:29.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.590+0000 7f0f5ea18b80 1 --2- 172.21.15.119:0/2534295984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faa5ef5130 0x55faa5efdc10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0f380972a0 tx=0x7f0f38007dd0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.590+0000 7f0f5ea18b80 1 -- 172.21.15.119:0/2534295984 shutdown_connections 2023-12-07T10:19:29.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.590+0000 7f0f5ea18b80 1 --2- 172.21.15.119:0/2534295984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faa5ef6980 0x55faa5f12fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.590+0000 7f0f5ea18b80 1 --2- 172.21.15.119:0/2534295984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faa5ef5130 0x55faa5efdc10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.590+0000 7f0f5ea18b80 1 --2- 172.21.15.119:0/2534295984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa5ef5a90 0x55faa5f063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.590+0000 7f0f5ea18b80 1 -- 172.21.15.119:0/2534295984 >> 172.21.15.119:0/2534295984 conn(0x55faa5e24310 msgr2=0x55faa5efe440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:29.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.590+0000 7f0f5ea18b80 1 -- 172.21.15.119:0/2534295984 shutdown_connections 2023-12-07T10:19:29.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.590+0000 7f0f5ea18b80 1 -- 172.21.15.119:0/2534295984 wait complete. 2023-12-07T10:19:29.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.590+0000 7f0f5ea18b80 1 -- start start 2023-12-07T10:19:29.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.590+0000 7f0f5ea18b80 1 Processor -- start 2023-12-07T10:19:29.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.591+0000 7f0f5ea18b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faa5ef5130 0x55faa5f05fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.591+0000 7f0f5ea18b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faa5ef5a90 0x55faa5f02a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.591+0000 7f0f5ea18b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa5ef6980 0x55faa5f02f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.591+0000 7f0f5ea18b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa5f067c0 con 0x55faa5ef5130 2023-12-07T10:19:29.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.591+0000 7f0f5ea18b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa5f034f0 con 0x55faa5ef6980 2023-12-07T10:19:29.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.591+0000 7f0f5ea18b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa5f036b0 con 0x55faa5ef5a90 2023-12-07T10:19:29.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.591+0000 7f0f5ea18b80 10 client.0 _ll_register_callbacks cb 0x55faa5dea8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:19:29.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.591+0000 7f0f48d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faa5ef5130 0x55faa5f05fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.591+0000 7f0f48d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faa5ef5130 0x55faa5f05fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52034/0 (socket says 172.21.15.119:52034) 2023-12-07T10:19:29.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.591+0000 7f0f48d9b700 1 -- 172.21.15.119:0/2708154710 learned_addr learned my addr 172.21.15.119:0/2708154710 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:29.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.591+0000 7f0f4959c700 1 --2- 172.21.15.119:0/2708154710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa5ef6980 0x55faa5f02f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.591+0000 7f0f43fff700 1 --2- 172.21.15.119:0/2708154710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faa5ef5a90 0x55faa5f02a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.592+0000 7f0f48d9b700 1 -- 172.21.15.119:0/2708154710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faa5ef5a90 msgr2=0x55faa5f02a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:29.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.592+0000 7f0f48d9b700 1 --2- 172.21.15.119:0/2708154710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faa5ef5a90 0x55faa5f02a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.592+0000 7f0f48d9b700 1 -- 172.21.15.119:0/2708154710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa5ef6980 msgr2=0x55faa5f02f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:29.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.592+0000 7f0f48d9b700 1 --2- 172.21.15.119:0/2708154710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa5ef6980 0x55faa5f02f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.592+0000 7f0f48d9b700 1 -- 172.21.15.119:0/2708154710 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f4410a2d0 con 0x55faa5ef5130 2023-12-07T10:19:29.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.592+0000 7f0f43fff700 1 --2- 172.21.15.119:0/2708154710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faa5ef5a90 0x55faa5f02a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:19:29.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.592+0000 7f0f48d9b700 1 --2- 172.21.15.119:0/2708154710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faa5ef5130 0x55faa5f05fa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0f38004800 tx=0x7f0f3809a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bb6d0f3cffef11 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:29.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.592+0000 7f0f23fff700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:19:29.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.592+0000 7f0f23fff700 1 -- 172.21.15.119:0/2708154710 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f380850b0 con 0x55faa5ef5130 2023-12-07T10:19:29.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.592+0000 7f0f23fff700 1 -- 172.21.15.119:0/2708154710 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f0f3809af30 con 0x55faa5ef5130 2023-12-07T10:19:29.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.592+0000 7f0f5ea18b80 1 -- 172.21.15.119:0/2708154710 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f30005530 con 0x55faa5ef5130 2023-12-07T10:19:29.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.592+0000 7f0f23fff700 1 -- 172.21.15.119:0/2708154710 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f380b3060 con 0x55faa5ef5130 2023-12-07T10:19:29.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.592+0000 7f0f5ea18b80 10 client.15138 fetch_fsmap learned FSMap version 29 2023-12-07T10:19:29.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.592+0000 7f0f5ea18b80 1 -- 172.21.15.119:0/2708154710 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f0f300095f0 con 0x55faa5ef5130 2023-12-07T10:19:29.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.593+0000 7f0f23fff700 1 -- 172.21.15.119:0/2708154710 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f30005530 con 0x55faa5ef5130 2023-12-07T10:19:29.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.593+0000 7f0f23fff700 1 -- 172.21.15.119:0/2708154710 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f0f3809aa90 con 0x55faa5ef5130 2023-12-07T10:19:29.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.593+0000 7f0f5ea18b80 10 client.15138 fetch_fsmap finished waiting for FSMap version 29 2023-12-07T10:19:29.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.593+0000 7f0f5ea18b80 10 client.15138 resolve_mds: resolved name 'e' to daemon mds.e 2023-12-07T10:19:29.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.593+0000 7f0f5ea18b80 20 client.15138 populate_metadata read hostname 'smithi119' 2023-12-07T10:19:29.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.593+0000 7f0f5ea18b80 1 --2- 172.21.15.119:0/2708154710 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x55faa5f65790 0x55faa5f61c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.593+0000 7f0f5ea18b80 4 client.15138 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-07T10:19:29.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.593+0000 7f0f5ea18b80 1 -- 172.21.15.119:0/2708154710 --> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55faa5e0ff80 con 0x55faa5f65790 2023-12-07T10:19:29.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.593+0000 7f0f43fff700 1 --2- 172.21.15.119:0/2708154710 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x55faa5f65790 0x55faa5f61c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.594+0000 7f0f43fff700 1 --2- 172.21.15.119:0/2708154710 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x55faa5f65790 0x55faa5f61c50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=280d8e3f03b60798 server_cookie=3680a81997365f13 in_seq=0 out_seq=0 2023-12-07T10:19:29.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.594+0000 7f0f23fff700 10 client.15138 ms_handle_connect on v2:172.21.15.153:6834/1295649464 2023-12-07T10:19:29.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.594+0000 7f0f23fff700 1 -- 172.21.15.119:0/2708154710 <== mds.4 v2:172.21.15.153:6834/1295649464 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55faa5e0ff80 con 0x55faa5f65790 2023-12-07T10:19:29.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.594+0000 7f0f23fff700 10 client.15138 handle_command_reply: tid=0 2023-12-07T10:19:29.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.594+0000 7f0f5ea18b80 1 client.15138 shutdown 2023-12-07T10:19:29.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.594+0000 7f0f5ea18b80 1 -- 172.21.15.119:0/2708154710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faa5ef5130 msgr2=0x55faa5f05fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:29.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.594+0000 7f0f5ea18b80 1 --2- 172.21.15.119:0/2708154710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faa5ef5130 0x55faa5f05fa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0f38004800 tx=0x7f0f3809a210 comp rx=0 tx=0).stop 2023-12-07T10:19:29.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.594+0000 7f0f5ea18b80 1 -- 172.21.15.119:0/2708154710 shutdown_connections 2023-12-07T10:19:29.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.594+0000 7f0f5ea18b80 1 --2- 172.21.15.119:0/2708154710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55faa5ef6980 0x55faa5f02f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.594+0000 7f0f5ea18b80 1 --2- 172.21.15.119:0/2708154710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faa5ef5130 0x55faa5f05fa0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.594+0000 7f0f5ea18b80 1 --2- 172.21.15.119:0/2708154710 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x55faa5f65790 0x55faa5f61c50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.594+0000 7f0f5ea18b80 1 --2- 172.21.15.119:0/2708154710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faa5ef5a90 0x55faa5f02a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.594+0000 7f0f23fff700 0 client.15138 ms_handle_reset on v2:172.21.15.153:6834/1295649464 2023-12-07T10:19:29.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.594+0000 7f0f5ea18b80 1 -- 172.21.15.119:0/2708154710 >> 172.21.15.119:0/2708154710 conn(0x55faa5e24310 msgr2=0x55faa5ef15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:29.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.594+0000 7f0f5ea18b80 1 -- 172.21.15.119:0/2708154710 shutdown_connections 2023-12-07T10:19:29.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.594+0000 7f0f5ea18b80 1 -- 172.21.15.119:0/2708154710 wait complete. 2023-12-07T10:19:29.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.594+0000 7f0f5ea18b80 20 client.15138 trim_cache size 0 max 16384 2023-12-07T10:19:29.605 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:19:29.605 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "a4c642c6-94e8-11ee-95a2-87774f69a715", 2023-12-07T10:19:29.605 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 4, 2023-12-07T10:19:29.605 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 24488, 2023-12-07T10:19:29.605 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-12-07T10:19:29.605 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-12-07T10:19:29.605 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-12-07T10:19:29.606 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 76.848618590000001, 2023-12-07T10:19:29.606 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 29, 2023-12-07T10:19:29.606 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 93, 2023-12-07T10:19:29.606 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 91, 2023-12-07T10:19:29.606 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 133.95251132499999 2023-12-07T10:19:29.606 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:19:29.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.595+0000 7f0f4afff700 1 -- 172.21.15.119:0/24079923 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0f340a1c20 msgr2=0x7f0f340a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:29.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.595+0000 7f0f4afff700 1 --2- 172.21.15.119:0/24079923 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0f340a1c20 0x7f0f340a40e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009fd0 tx=0x7f0f3c009920 comp rx=0 tx=0).stop 2023-12-07T10:19:29.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.595+0000 7f0f4afff700 1 -- 172.21.15.119:0/24079923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f44101540 msgr2=0x7f0f44075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:29.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.595+0000 7f0f4afff700 1 --2- 172.21.15.119:0/24079923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f44101540 0x7f0f44075370 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800e970 tx=0x7f0f3800ed30 comp rx=0 tx=0).stop 2023-12-07T10:19:29.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.595+0000 7f0f4afff700 1 -- 172.21.15.119:0/24079923 shutdown_connections 2023-12-07T10:19:29.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.595+0000 7f0f4afff700 1 --2- 172.21.15.119:0/24079923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f441030d0 0x7f0f44073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.595+0000 7f0f4afff700 1 --2- 172.21.15.119:0/24079923 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0f340a1c20 0x7f0f340a40e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.595+0000 7f0f4afff700 1 --2- 172.21.15.119:0/24079923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f44101540 0x7f0f44075370 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.595+0000 7f0f4afff700 1 --2- 172.21.15.119:0/24079923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f44102730 0x7f0f44073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.595+0000 7f0f4afff700 1 -- 172.21.15.119:0/24079923 >> 172.21.15.119:0/24079923 conn(0x7f0f440fcab0 msgr2=0x7f0f44105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:29.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.595+0000 7f0f4afff700 1 -- 172.21.15.119:0/24079923 shutdown_connections 2023-12-07T10:19:29.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.595+0000 7f0f4afff700 1 -- 172.21.15.119:0/24079923 wait complete. 2023-12-07T10:19:29.615 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-12-07T10:19:29.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.814+0000 7f98169ff700 1 -- 172.21.15.119:0/2251493322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9810104950 msgr2=0x7f9810106d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:29.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.814+0000 7f98169ff700 1 --2- 172.21.15.119:0/2251493322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9810104950 0x7f9810106d40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f98040098f0 tx=0x7f9804009c00 comp rx=0 tx=0).stop 2023-12-07T10:19:29.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.814+0000 7f98169ff700 1 -- 172.21.15.119:0/2251493322 shutdown_connections 2023-12-07T10:19:29.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.814+0000 7f98169ff700 1 --2- 172.21.15.119:0/2251493322 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9810107280 0x7f9810109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.814+0000 7f98169ff700 1 --2- 172.21.15.119:0/2251493322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9810102020 0x7f9810104410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.814+0000 7f98169ff700 1 --2- 172.21.15.119:0/2251493322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9810104950 0x7f9810106d40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.814+0000 7f98169ff700 1 -- 172.21.15.119:0/2251493322 >> 172.21.15.119:0/2251493322 conn(0x7f98100fb9b0 msgr2=0x7f98100fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:29.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.814+0000 7f98169ff700 1 -- 172.21.15.119:0/2251493322 shutdown_connections 2023-12-07T10:19:29.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.814+0000 7f98169ff700 1 -- 172.21.15.119:0/2251493322 wait complete. 2023-12-07T10:19:29.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.815+0000 7f98169ff700 1 Processor -- start 2023-12-07T10:19:29.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.815+0000 7f98169ff700 1 -- start start 2023-12-07T10:19:29.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.816+0000 7f98169ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9810102020 0x7f9810199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.816+0000 7f98169ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9810104950 0x7f9810199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.816+0000 7f98169ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9810107280 0x7f98101a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.816+0000 7f98169ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98101a1280 con 0x7f9810104950 2023-12-07T10:19:29.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.816+0000 7f98169ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98101a13c0 con 0x7f9810107280 2023-12-07T10:19:29.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.816+0000 7f98169ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98101a1550 con 0x7f9810102020 2023-12-07T10:19:29.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.816+0000 7f980f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9810104950 0x7f9810199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.816+0000 7f980f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9810104950 0x7f9810199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52046/0 (socket says 172.21.15.119:52046) 2023-12-07T10:19:29.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.816+0000 7f980f7fe700 1 -- 172.21.15.119:0/1552544191 learned_addr learned my addr 172.21.15.119:0/1552544191 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:29.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.816+0000 7f980ffff700 1 --2- 172.21.15.119:0/1552544191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9810102020 0x7f9810199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.816+0000 7f9814f9c700 1 --2- 172.21.15.119:0/1552544191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9810107280 0x7f98101a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.816+0000 7f980f7fe700 1 -- 172.21.15.119:0/1552544191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9810102020 msgr2=0x7f9810199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:29.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.816+0000 7f980f7fe700 1 --2- 172.21.15.119:0/1552544191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9810102020 0x7f9810199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.816+0000 7f980f7fe700 1 -- 172.21.15.119:0/1552544191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9810107280 msgr2=0x7f98101a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:29.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.816+0000 7f980f7fe700 1 --2- 172.21.15.119:0/1552544191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9810107280 0x7f98101a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.816+0000 7f980f7fe700 1 -- 172.21.15.119:0/1552544191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98000095f0 con 0x7f9810104950 2023-12-07T10:19:29.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.816+0000 7f9814f9c700 1 --2- 172.21.15.119:0/1552544191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9810107280 0x7f98101a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:19:29.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.817+0000 7f980f7fe700 1 --2- 172.21.15.119:0/1552544191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9810104950 0x7f9810199c90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9804009810 tx=0x7f980400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:29.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.817+0000 7f980d7fa700 1 -- 172.21.15.119:0/1552544191 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980401c070 con 0x7f9810104950 2023-12-07T10:19:29.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.817+0000 7f980d7fa700 1 -- 172.21.15.119:0/1552544191 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f980400f7d0 con 0x7f9810104950 2023-12-07T10:19:29.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.817+0000 7f98169ff700 1 -- 172.21.15.119:0/1552544191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9804009520 con 0x7f9810104950 2023-12-07T10:19:29.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.817+0000 7f980d7fa700 1 -- 172.21.15.119:0/1552544191 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9804003a60 con 0x7f9810104950 2023-12-07T10:19:29.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.817+0000 7f98169ff700 1 -- 172.21.15.119:0/1552544191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98101a1a60 con 0x7f9810104950 2023-12-07T10:19:29.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.820+0000 7f980d7fa700 1 -- 172.21.15.119:0/1552544191 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f9804020030 con 0x7f9810104950 2023-12-07T10:19:29.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.821+0000 7f980d7fa700 1 --2- 172.21.15.119:0/1552544191 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f97f80a1ba0 0x7f97f80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.822+0000 7f982a399b80 1 Processor -- start 2023-12-07T10:19:29.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.822+0000 7f982a399b80 1 -- start start 2023-12-07T10:19:29.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.822+0000 7f980d7fa700 1 -- 172.21.15.119:0/1552544191 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f98040ceaf0 con 0x7f9810104950 2023-12-07T10:19:29.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.822+0000 7f980ffff700 1 --2- 172.21.15.119:0/1552544191 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f97f80a1ba0 0x7f97f80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.822+0000 7f982a399b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca2f130 0x556f7ca2f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.822+0000 7f982a399b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f7ca2fa90 0x556f7ca303a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.822+0000 7f982a399b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f7ca30980 0x556f7ca30e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.822+0000 7f982a399b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f7c9a3770 con 0x556f7ca2f130 2023-12-07T10:19:29.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.822+0000 7f982a399b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f7c98a750 con 0x556f7ca30980 2023-12-07T10:19:29.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.822+0000 7f982a399b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f7c988070 con 0x556f7ca2fa90 2023-12-07T10:19:29.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.822+0000 7f980ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca2f130 0x556f7ca2f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.822+0000 7f980f7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f7ca2fa90 0x556f7ca303a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.822+0000 7f980ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca2f130 0x556f7ca2f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52048/0 (socket says 172.21.15.119:52048) 2023-12-07T10:19:29.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.822+0000 7f980ffff700 1 -- 172.21.15.119:0/1685192172 learned_addr learned my addr 172.21.15.119:0/1685192172 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:29.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.822+0000 7f9814f9c700 1 --2- 172.21.15.119:0/1685192172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f7ca30980 0x556f7ca30e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.823+0000 7f980f7fe700 1 -- 172.21.15.119:0/1685192172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f7ca30980 msgr2=0x556f7ca30e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:29.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.823+0000 7f980f7fe700 1 --2- 172.21.15.119:0/1685192172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f7ca30980 0x556f7ca30e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.823+0000 7f980f7fe700 1 -- 172.21.15.119:0/1685192172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca2f130 msgr2=0x556f7ca2f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:29.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.823+0000 7f980f7fe700 1 --2- 172.21.15.119:0/1685192172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca2f130 0x556f7ca2f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.823+0000 7f980f7fe700 1 -- 172.21.15.119:0/1685192172 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f7c8005e0 con 0x556f7ca2fa90 2023-12-07T10:19:29.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.823+0000 7f980ffff700 1 --2- 172.21.15.119:0/1552544191 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f97f80a1ba0 0x7f97f80a4060 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9800013760 tx=0x7f9800013b20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:29.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.823+0000 7f980f7fe700 1 --2- 172.21.15.119:0/1685192172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f7ca2fa90 0x556f7ca303a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f98040cc0d0 tx=0x7f980409b5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20a72f67fd9690db server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:29.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.823+0000 7f97f67fc700 1 -- 172.21.15.119:0/1685192172 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98040bb070 con 0x556f7ca2fa90 2023-12-07T10:19:29.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.823+0000 7f97f67fc700 1 -- 172.21.15.119:0/1685192172 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f980400bbc0 con 0x556f7ca2fa90 2023-12-07T10:19:29.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.823+0000 7f97f67fc700 1 -- 172.21.15.119:0/1685192172 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9804097e40 con 0x556f7ca2fa90 2023-12-07T10:19:29.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.824+0000 7f982a399b80 1 -- 172.21.15.119:0/1685192172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f7ca2fa90 msgr2=0x556f7ca303a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:29.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.824+0000 7f982a399b80 1 --2- 172.21.15.119:0/1685192172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f7ca2fa90 0x556f7ca303a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f98040cc0d0 tx=0x7f980409b5f0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.824+0000 7f982a399b80 1 -- 172.21.15.119:0/1685192172 shutdown_connections 2023-12-07T10:19:29.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.824+0000 7f982a399b80 1 --2- 172.21.15.119:0/1685192172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f7ca30980 0x556f7ca30e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.824+0000 7f982a399b80 1 --2- 172.21.15.119:0/1685192172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca2f130 0x556f7ca2f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.824+0000 7f982a399b80 1 --2- 172.21.15.119:0/1685192172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f7ca2fa90 0x556f7ca303a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.824+0000 7f982a399b80 1 -- 172.21.15.119:0/1685192172 >> 172.21.15.119:0/1685192172 conn(0x556f7c95e310 msgr2=0x556f7ca2bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:29.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.824+0000 7f982a399b80 1 -- 172.21.15.119:0/1685192172 shutdown_connections 2023-12-07T10:19:29.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.824+0000 7f982a399b80 1 -- 172.21.15.119:0/1685192172 wait complete. 2023-12-07T10:19:29.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.825+0000 7f982a399b80 1 -- start start 2023-12-07T10:19:29.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.825+0000 7f982a399b80 1 Processor -- start 2023-12-07T10:19:29.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.826+0000 7f982a399b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca2f130 0x556f7ca38c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.826+0000 7f982a399b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f7ca2fa90 0x556f7ca391b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.826+0000 7f982a399b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f7ca30980 0x556f7ca401a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.826+0000 7f982a399b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f7c983960 con 0x556f7ca2f130 2023-12-07T10:19:29.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.826+0000 7f982a399b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f7c93d510 con 0x556f7ca30980 2023-12-07T10:19:29.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.826+0000 7f982a399b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f7c93d1b0 con 0x556f7ca2fa90 2023-12-07T10:19:29.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.826+0000 7f982a399b80 10 client.0 _ll_register_callbacks cb 0x556f7c9248f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:19:29.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.826+0000 7f980ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca2f130 0x556f7ca38c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.826+0000 7f980f7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f7ca2fa90 0x556f7ca391b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.826+0000 7f980ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca2f130 0x556f7ca38c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52064/0 (socket says 172.21.15.119:52064) 2023-12-07T10:19:29.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.826+0000 7f980ffff700 1 -- 172.21.15.119:0/3855528999 learned_addr learned my addr 172.21.15.119:0/3855528999 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:29.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.826+0000 7f980f7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f7ca2fa90 0x556f7ca391b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:32956/0 (socket says 172.21.15.119:32956) 2023-12-07T10:19:29.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.826+0000 7f9814f9c700 1 --2- 172.21.15.119:0/3855528999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f7ca30980 0x556f7ca401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.826+0000 7f980f7fe700 1 -- 172.21.15.119:0/3855528999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f7ca30980 msgr2=0x556f7ca401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:29.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.826+0000 7f980f7fe700 1 --2- 172.21.15.119:0/3855528999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f7ca30980 0x556f7ca401a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.826+0000 7f980f7fe700 1 -- 172.21.15.119:0/3855528999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca2f130 msgr2=0x556f7ca38c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:29.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.826+0000 7f980f7fe700 1 --2- 172.21.15.119:0/3855528999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca2f130 0x556f7ca38c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.826+0000 7f980f7fe700 1 -- 172.21.15.119:0/3855528999 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98101a1a60 con 0x556f7ca2fa90 2023-12-07T10:19:29.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.826+0000 7f9814f9c700 1 --2- 172.21.15.119:0/3855528999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f7ca30980 0x556f7ca401a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:19:29.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.826+0000 7f980ffff700 1 --2- 172.21.15.119:0/3855528999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca2f130 0x556f7ca38c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:29.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.827+0000 7f980f7fe700 1 --2- 172.21.15.119:0/3855528999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f7ca2fa90 0x556f7ca391b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f98040bb1d0 tx=0x7f9804005e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8098c608d981e768 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:29.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.827+0000 7f97ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:19:29.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.827+0000 7f97ef7fe700 1 -- 172.21.15.119:0/3855528999 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980409be30 con 0x556f7ca2fa90 2023-12-07T10:19:29.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.827+0000 7f97ef7fe700 1 -- 172.21.15.119:0/3855528999 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f980409b730 con 0x556f7ca2fa90 2023-12-07T10:19:29.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.827+0000 7f982a399b80 1 -- 172.21.15.119:0/3855528999 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f7ca22ab0 con 0x556f7ca2fa90 2023-12-07T10:19:29.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.827+0000 7f97ef7fe700 1 -- 172.21.15.119:0/3855528999 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f980409b990 con 0x556f7ca2fa90 2023-12-07T10:19:29.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.827+0000 7f982a399b80 10 client.25159 fetch_fsmap learned FSMap version 29 2023-12-07T10:19:29.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.827+0000 7f982a399b80 1 -- 172.21.15.119:0/3855528999 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556f7c7acd90 con 0x556f7ca2fa90 2023-12-07T10:19:29.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.827+0000 7f97ef7fe700 1 -- 172.21.15.119:0/3855528999 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556f7ca22ab0 con 0x556f7ca2fa90 2023-12-07T10:19:29.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.827+0000 7f97ef7fe700 1 -- 172.21.15.119:0/3855528999 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f98040b8030 con 0x556f7ca2fa90 2023-12-07T10:19:29.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.828+0000 7f982a399b80 10 client.25159 fetch_fsmap finished waiting for FSMap version 29 2023-12-07T10:19:29.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.828+0000 7f982a399b80 10 client.25159 resolve_mds: resolved name 'f' to daemon mds.f 2023-12-07T10:19:29.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.828+0000 7f982a399b80 20 client.25159 populate_metadata read hostname 'smithi119' 2023-12-07T10:19:29.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.828+0000 7f982a399b80 1 --2- 172.21.15.119:0/3855528999 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x556f7ca44250 0x556f7ca43170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.828+0000 7f982a399b80 4 client.25159 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:19:29.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.828+0000 7f982a399b80 1 -- 172.21.15.119:0/3855528999 --> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f7c9fe750 con 0x556f7ca44250 2023-12-07T10:19:29.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.828+0000 7f980ffff700 1 --2- 172.21.15.119:0/3855528999 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x556f7ca44250 0x556f7ca43170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.828+0000 7f97ef7fe700 10 client.25159 ms_handle_connect on v2:172.21.15.159:6834/2406380792 2023-12-07T10:19:29.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.828+0000 7f980ffff700 1 --2- 172.21.15.119:0/3855528999 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x556f7ca44250 0x556f7ca43170 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c3027f01a5b9d374 server_cookie=61ba5b0f1c78abd3 in_seq=0 out_seq=0 2023-12-07T10:19:29.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.829+0000 7f97ef7fe700 1 -- 172.21.15.119:0/3855528999 <== mds.3 v2:172.21.15.159:6834/2406380792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556f7c9fe750 con 0x556f7ca44250 2023-12-07T10:19:29.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.829+0000 7f97ef7fe700 10 client.25159 handle_command_reply: tid=0 2023-12-07T10:19:29.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.829+0000 7f982a399b80 1 client.25159 shutdown 2023-12-07T10:19:29.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.830+0000 7f982a399b80 1 -- 172.21.15.119:0/3855528999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f7ca2fa90 msgr2=0x556f7ca391b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:29.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.830+0000 7f982a399b80 1 --2- 172.21.15.119:0/3855528999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f7ca2fa90 0x556f7ca391b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f98040bb1d0 tx=0x7f9804005e30 comp rx=0 tx=0).stop 2023-12-07T10:19:29.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.830+0000 7f982a399b80 1 -- 172.21.15.119:0/3855528999 shutdown_connections 2023-12-07T10:19:29.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.830+0000 7f982a399b80 1 --2- 172.21.15.119:0/3855528999 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x556f7ca44250 0x556f7ca43170 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.830+0000 7f982a399b80 1 --2- 172.21.15.119:0/3855528999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f7ca30980 0x556f7ca401a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.830+0000 7f97ef7fe700 0 client.25159 ms_handle_reset on v2:172.21.15.159:6834/2406380792 2023-12-07T10:19:29.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.830+0000 7f982a399b80 1 --2- 172.21.15.119:0/3855528999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca2f130 0x556f7ca38c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.830+0000 7f982a399b80 1 --2- 172.21.15.119:0/3855528999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f7ca2fa90 0x556f7ca391b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.830+0000 7f982a399b80 1 -- 172.21.15.119:0/3855528999 >> 172.21.15.119:0/3855528999 conn(0x556f7c95e310 msgr2=0x556f7ca2a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:29.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.830+0000 7f982a399b80 1 -- 172.21.15.119:0/3855528999 shutdown_connections 2023-12-07T10:19:29.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.830+0000 7f982a399b80 1 -- 172.21.15.119:0/3855528999 wait complete. 2023-12-07T10:19:29.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.830+0000 7f982a399b80 20 client.25159 trim_cache size 0 max 16384 2023-12-07T10:19:29.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.848+0000 7f982a399b80 1 Processor -- start 2023-12-07T10:19:29.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.849+0000 7f982a399b80 1 -- start start 2023-12-07T10:19:29.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.849+0000 7f982a399b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f7ca2f130 0x556f7ca4cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.849+0000 7f982a399b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f7ca2fa90 0x556f7ca40160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.849+0000 7f982a399b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca30980 0x556f7ca338f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.849+0000 7f982a399b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f7c8028d0 con 0x556f7ca30980 2023-12-07T10:19:29.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.849+0000 7f982a399b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f7c988070 con 0x556f7ca2f130 2023-12-07T10:19:29.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.849+0000 7f982a399b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f7ca2b1d0 con 0x556f7ca2fa90 2023-12-07T10:19:29.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.849+0000 7f9814f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca30980 0x556f7ca338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.849+0000 7f9814f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca30980 0x556f7ca338f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52072/0 (socket says 172.21.15.119:52072) 2023-12-07T10:19:29.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.849+0000 7f9814f9c700 1 -- 172.21.15.119:0/4212148377 learned_addr learned my addr 172.21.15.119:0/4212148377 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:29.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.849+0000 7f980f7fe700 1 --2- 172.21.15.119:0/4212148377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f7ca2fa90 0x556f7ca40160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.849+0000 7f980ffff700 1 --2- 172.21.15.119:0/4212148377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f7ca2f130 0x556f7ca4cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.850+0000 7f9814f9c700 1 -- 172.21.15.119:0/4212148377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f7ca2fa90 msgr2=0x556f7ca40160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:29.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.850+0000 7f9814f9c700 1 --2- 172.21.15.119:0/4212148377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f7ca2fa90 0x556f7ca40160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.850+0000 7f9814f9c700 1 -- 172.21.15.119:0/4212148377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f7ca2f130 msgr2=0x556f7ca4cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:29.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.850+0000 7f9814f9c700 1 --2- 172.21.15.119:0/4212148377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f7ca2f130 0x556f7ca4cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.850+0000 7f9814f9c700 1 -- 172.21.15.119:0/4212148377 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f7c795c10 con 0x556f7ca30980 2023-12-07T10:19:29.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.850+0000 7f980f7fe700 1 --2- 172.21.15.119:0/4212148377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f7ca2fa90 0x556f7ca40160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:29.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.850+0000 7f9814f9c700 1 --2- 172.21.15.119:0/4212148377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca30980 0x556f7ca338f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9808007660 tx=0x7f980800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58855371d09501 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:29.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.850+0000 7f97f6ffd700 1 -- 172.21.15.119:0/4212148377 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9808007d50 con 0x556f7ca30980 2023-12-07T10:19:29.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.850+0000 7f97f6ffd700 1 -- 172.21.15.119:0/4212148377 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f980800f030 con 0x556f7ca30980 2023-12-07T10:19:29.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.850+0000 7f97f6ffd700 1 -- 172.21.15.119:0/4212148377 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9808016e70 con 0x556f7ca30980 2023-12-07T10:19:29.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.851+0000 7f982a399b80 1 -- 172.21.15.119:0/4212148377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca30980 msgr2=0x556f7ca338f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:29.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.851+0000 7f982a399b80 1 --2- 172.21.15.119:0/4212148377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca30980 0x556f7ca338f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9808007660 tx=0x7f980800b040 comp rx=0 tx=0).stop 2023-12-07T10:19:29.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.851+0000 7f982a399b80 1 -- 172.21.15.119:0/4212148377 shutdown_connections 2023-12-07T10:19:29.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.851+0000 7f982a399b80 1 --2- 172.21.15.119:0/4212148377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca30980 0x556f7ca338f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.851+0000 7f982a399b80 1 --2- 172.21.15.119:0/4212148377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f7ca2f130 0x556f7ca4cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.851+0000 7f982a399b80 1 --2- 172.21.15.119:0/4212148377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f7ca2fa90 0x556f7ca40160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.851+0000 7f982a399b80 1 -- 172.21.15.119:0/4212148377 >> 172.21.15.119:0/4212148377 conn(0x556f7c95e310 msgr2=0x556f7ca2b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:29.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.851+0000 7f982a399b80 1 -- 172.21.15.119:0/4212148377 shutdown_connections 2023-12-07T10:19:29.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.851+0000 7f982a399b80 1 -- 172.21.15.119:0/4212148377 wait complete. 2023-12-07T10:19:29.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.851+0000 7f982a399b80 1 -- start start 2023-12-07T10:19:29.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.851+0000 7f982a399b80 1 Processor -- start 2023-12-07T10:19:29.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.852+0000 7f982a399b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f7ca2f130 0x556f7ca3fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.852+0000 7f982a399b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca2fa90 0x556f7ca40190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.852+0000 7f982a399b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f7ca30980 0x556f7ca3c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.852+0000 7f982a399b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f7ca3cfb0 con 0x556f7ca2fa90 2023-12-07T10:19:29.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.852+0000 7f982a399b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f7ca3d120 con 0x556f7ca2f130 2023-12-07T10:19:29.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.852+0000 7f982a399b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f7ca3d2e0 con 0x556f7ca30980 2023-12-07T10:19:29.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.852+0000 7f982a399b80 10 client.0 _ll_register_callbacks cb 0x556f7c9248f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:19:29.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.852+0000 7f980f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca2fa90 0x556f7ca40190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.852+0000 7f980ffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f7ca2f130 0x556f7ca3fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.852+0000 7f980f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca2fa90 0x556f7ca40190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52076/0 (socket says 172.21.15.119:52076) 2023-12-07T10:19:29.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.852+0000 7f9814f9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f7ca30980 0x556f7ca3c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.852+0000 7f980f7fe700 1 -- 172.21.15.119:0/530716301 learned_addr learned my addr 172.21.15.119:0/530716301 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:29.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.852+0000 7f980ffff700 1 -- 172.21.15.119:0/530716301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f7ca30980 msgr2=0x556f7ca3c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:29.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.853+0000 7f980ffff700 1 --2- 172.21.15.119:0/530716301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f7ca30980 0x556f7ca3c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.853+0000 7f980ffff700 1 -- 172.21.15.119:0/530716301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca2fa90 msgr2=0x556f7ca40190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:29.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.853+0000 7f980ffff700 1 --2- 172.21.15.119:0/530716301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca2fa90 0x556f7ca40190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.853+0000 7f980ffff700 1 -- 172.21.15.119:0/530716301 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98101a1a60 con 0x556f7ca2f130 2023-12-07T10:19:29.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.853+0000 7f980f7fe700 1 --2- 172.21.15.119:0/530716301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca2fa90 0x556f7ca40190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:29.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.853+0000 7f980ffff700 1 --2- 172.21.15.119:0/530716301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f7ca2f130 0x556f7ca3fc50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x556f7c983a30 tx=0x7f980001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c59d1c4484e4947e server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:29.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.853+0000 7f97ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:19:29.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.853+0000 7f97ef7fe700 1 -- 172.21.15.119:0/530716301 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9800018040 con 0x556f7ca2f130 2023-12-07T10:19:29.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.853+0000 7f982a399b80 1 -- 172.21.15.119:0/530716301 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f980400bbc0 con 0x556f7ca2f130 2023-12-07T10:19:29.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.853+0000 7f97ef7fe700 1 -- 172.21.15.119:0/530716301 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f98000120d0 con 0x556f7ca2f130 2023-12-07T10:19:29.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.853+0000 7f97ef7fe700 1 -- 172.21.15.119:0/530716301 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9800012290 con 0x556f7ca2f130 2023-12-07T10:19:29.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.853+0000 7f982a399b80 10 client.25181 fetch_fsmap learned FSMap version 29 2023-12-07T10:19:29.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.853+0000 7f982a399b80 1 -- 172.21.15.119:0/530716301 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f98080092c0 con 0x556f7ca2f130 2023-12-07T10:19:29.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.854+0000 7f97ef7fe700 1 -- 172.21.15.119:0/530716301 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f980400bbc0 con 0x556f7ca2f130 2023-12-07T10:19:29.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.854+0000 7f97ef7fe700 1 -- 172.21.15.119:0/530716301 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f9800029050 con 0x556f7ca2f130 2023-12-07T10:19:29.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.854+0000 7f982a399b80 10 client.25181 fetch_fsmap finished waiting for FSMap version 29 2023-12-07T10:19:29.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.854+0000 7f982a399b80 10 client.25181 resolve_mds: resolved name 'f' to daemon mds.f 2023-12-07T10:19:29.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.854+0000 7f982a399b80 20 client.25181 populate_metadata read hostname 'smithi119' 2023-12-07T10:19:29.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.854+0000 7f982a399b80 1 --2- 172.21.15.119:0/530716301 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x556f7ca9f4e0 0x556f7ca9b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:29.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.854+0000 7f982a399b80 4 client.25181 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-07T10:19:29.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.854+0000 7f982a399b80 1 -- 172.21.15.119:0/530716301 --> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556f7c949f80 con 0x556f7ca9f4e0 2023-12-07T10:19:29.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.854+0000 7f980f7fe700 1 --2- 172.21.15.119:0/530716301 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x556f7ca9f4e0 0x556f7ca9b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:29.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.855+0000 7f97ef7fe700 10 client.25181 ms_handle_connect on v2:172.21.15.159:6834/2406380792 2023-12-07T10:19:29.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.855+0000 7f980f7fe700 1 --2- 172.21.15.119:0/530716301 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x556f7ca9f4e0 0x556f7ca9b8d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=26b12c33d8d5cc5f server_cookie=20263736594d3be2 in_seq=0 out_seq=0 2023-12-07T10:19:29.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.855+0000 7f97ef7fe700 1 -- 172.21.15.119:0/530716301 <== mds.3 v2:172.21.15.159:6834/2406380792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x556f7c949f80 con 0x556f7ca9f4e0 2023-12-07T10:19:29.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.855+0000 7f97ef7fe700 10 client.25181 handle_command_reply: tid=0 2023-12-07T10:19:29.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.855+0000 7f982a399b80 1 client.25181 shutdown 2023-12-07T10:19:29.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.855+0000 7f982a399b80 1 -- 172.21.15.119:0/530716301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f7ca2f130 msgr2=0x556f7ca3fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:29.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.855+0000 7f982a399b80 1 --2- 172.21.15.119:0/530716301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f7ca2f130 0x556f7ca3fc50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x556f7c983a30 tx=0x7f980001b000 comp rx=0 tx=0).stop 2023-12-07T10:19:29.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.855+0000 7f982a399b80 1 -- 172.21.15.119:0/530716301 shutdown_connections 2023-12-07T10:19:29.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.855+0000 7f982a399b80 1 --2- 172.21.15.119:0/530716301 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x556f7ca9f4e0 0x556f7ca9b8d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.855+0000 7f982a399b80 1 --2- 172.21.15.119:0/530716301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f7ca30980 0x556f7ca3c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.855+0000 7f97ef7fe700 0 client.25181 ms_handle_reset on v2:172.21.15.159:6834/2406380792 2023-12-07T10:19:29.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.855+0000 7f982a399b80 1 --2- 172.21.15.119:0/530716301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f7ca2f130 0x556f7ca3fc50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.855+0000 7f982a399b80 1 --2- 172.21.15.119:0/530716301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f7ca2fa90 0x556f7ca40190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.855+0000 7f982a399b80 1 -- 172.21.15.119:0/530716301 >> 172.21.15.119:0/530716301 conn(0x556f7c95e310 msgr2=0x556f7ca338f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:29.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.855+0000 7f982a399b80 1 -- 172.21.15.119:0/530716301 shutdown_connections 2023-12-07T10:19:29.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.855+0000 7f982a399b80 1 -- 172.21.15.119:0/530716301 wait complete. 2023-12-07T10:19:29.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.856+0000 7f982a399b80 20 client.25181 trim_cache size 0 max 16384 2023-12-07T10:19:29.862 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:19:29.862 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "a4c642c6-94e8-11ee-95a2-87774f69a715", 2023-12-07T10:19:29.862 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 3, 2023-12-07T10:19:29.862 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 24505, 2023-12-07T10:19:29.862 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-12-07T10:19:29.862 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-12-07T10:19:29.862 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-12-07T10:19:29.862 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 78.117676219000003, 2023-12-07T10:19:29.862 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 29, 2023-12-07T10:19:29.862 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 93, 2023-12-07T10:19:29.863 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 91, 2023-12-07T10:19:29.863 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 131.89460025100001 2023-12-07T10:19:29.863 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:19:29.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.856+0000 7f98169ff700 1 -- 172.21.15.119:0/1552544191 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f97f80a1ba0 msgr2=0x7f97f80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:29.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.856+0000 7f98169ff700 1 --2- 172.21.15.119:0/1552544191 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f97f80a1ba0 0x7f97f80a4060 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9800013760 tx=0x7f9800013b20 comp rx=0 tx=0).stop 2023-12-07T10:19:29.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.856+0000 7f98169ff700 1 -- 172.21.15.119:0/1552544191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9810104950 msgr2=0x7f9810199c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:29.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.856+0000 7f98169ff700 1 --2- 172.21.15.119:0/1552544191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9810104950 0x7f9810199c90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9804009810 tx=0x7f980400b790 comp rx=0 tx=0).stop 2023-12-07T10:19:29.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.856+0000 7f98169ff700 1 -- 172.21.15.119:0/1552544191 shutdown_connections 2023-12-07T10:19:29.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.856+0000 7f98169ff700 1 --2- 172.21.15.119:0/1552544191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9810107280 0x7f98101a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.856+0000 7f98169ff700 1 --2- 172.21.15.119:0/1552544191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9810102020 0x7f9810199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.856+0000 7f98169ff700 1 --2- 172.21.15.119:0/1552544191 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f97f80a1ba0 0x7f97f80a4060 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.856+0000 7f98169ff700 1 --2- 172.21.15.119:0/1552544191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9810104950 0x7f9810199c90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:29.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.856+0000 7f98169ff700 1 -- 172.21.15.119:0/1552544191 >> 172.21.15.119:0/1552544191 conn(0x7f98100fb9b0 msgr2=0x7f98100fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:29.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.856+0000 7f98169ff700 1 -- 172.21.15.119:0/1552544191 shutdown_connections 2023-12-07T10:19:29.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:29.856+0000 7f98169ff700 1 -- 172.21.15.119:0/1552544191 wait complete. 2023-12-07T10:19:29.876 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-07T10:19:30.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.074+0000 7f40841ff700 1 -- 172.21.15.119:0/1754333814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f407c101550 msgr2=0x7f407c101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.074+0000 7f40841ff700 1 --2- 172.21.15.119:0/1754333814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f407c101550 0x7f407c101970 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f40780099c0 tx=0x7f4078009cd0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.074+0000 7f40841ff700 1 -- 172.21.15.119:0/1754333814 shutdown_connections 2023-12-07T10:19:30.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.074+0000 7f40841ff700 1 --2- 172.21.15.119:0/1754333814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f407c1030e0 0x7f407c109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.074+0000 7f40841ff700 1 --2- 172.21.15.119:0/1754333814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f407c101550 0x7f407c101970 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.074+0000 7f40841ff700 1 --2- 172.21.15.119:0/1754333814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f407c102740 0x7f407c102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.074+0000 7f40841ff700 1 -- 172.21.15.119:0/1754333814 >> 172.21.15.119:0/1754333814 conn(0x7f407c0fcae0 msgr2=0x7f407c0fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:30.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.074+0000 7f40841ff700 1 -- 172.21.15.119:0/1754333814 shutdown_connections 2023-12-07T10:19:30.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.074+0000 7f40841ff700 1 -- 172.21.15.119:0/1754333814 wait complete. 2023-12-07T10:19:30.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.075+0000 7f40841ff700 1 Processor -- start 2023-12-07T10:19:30.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.075+0000 7f40841ff700 1 -- start start 2023-12-07T10:19:30.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.075+0000 7f40841ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f407c101550 0x7f407c1974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.075+0000 7f40841ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f407c102740 0x7f407c197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.075+0000 7f40841ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f407c1030e0 0x7f407c19ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.075+0000 7f40841ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407c19f010 con 0x7f407c1030e0 2023-12-07T10:19:30.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.075+0000 7f40841ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407c19f150 con 0x7f407c102740 2023-12-07T10:19:30.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.075+0000 7f40841ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407c19f2e0 con 0x7f407c101550 2023-12-07T10:19:30.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.076+0000 7f408279c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f407c1030e0 0x7f407c19ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.076+0000 7f408279c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f407c1030e0 0x7f407c19ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52094/0 (socket says 172.21.15.119:52094) 2023-12-07T10:19:30.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.076+0000 7f408279c700 1 -- 172.21.15.119:0/2644582228 learned_addr learned my addr 172.21.15.119:0/2644582228 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:30.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.076+0000 7f4081f9b700 1 --2- 172.21.15.119:0/2644582228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f407c101550 0x7f407c1974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.076+0000 7f408179a700 1 --2- 172.21.15.119:0/2644582228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f407c102740 0x7f407c197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.076+0000 7f408279c700 1 -- 172.21.15.119:0/2644582228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f407c101550 msgr2=0x7f407c1974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.076+0000 7f408279c700 1 --2- 172.21.15.119:0/2644582228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f407c101550 0x7f407c1974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.076+0000 7f408279c700 1 -- 172.21.15.119:0/2644582228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f407c102740 msgr2=0x7f407c197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.076+0000 7f408279c700 1 --2- 172.21.15.119:0/2644582228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f407c102740 0x7f407c197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.076+0000 7f408279c700 1 -- 172.21.15.119:0/2644582228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40780095f0 con 0x7f407c1030e0 2023-12-07T10:19:30.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.076+0000 7f408279c700 1 --2- 172.21.15.119:0/2644582228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f407c1030e0 0x7f407c19ea40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4074007ee0 tx=0x7f407400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:30.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.076+0000 7f4072ffd700 1 -- 172.21.15.119:0/2644582228 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4074017070 con 0x7f407c1030e0 2023-12-07T10:19:30.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.076+0000 7f4072ffd700 1 -- 172.21.15.119:0/2644582228 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f407400f040 con 0x7f407c1030e0 2023-12-07T10:19:30.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.076+0000 7f4072ffd700 1 -- 172.21.15.119:0/2644582228 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4074005580 con 0x7f407c1030e0 2023-12-07T10:19:30.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.076+0000 7f40841ff700 1 -- 172.21.15.119:0/2644582228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f407c19f5c0 con 0x7f407c1030e0 2023-12-07T10:19:30.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.077+0000 7f40841ff700 1 -- 172.21.15.119:0/2644582228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f407c19fa80 con 0x7f407c1030e0 2023-12-07T10:19:30.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.079+0000 7f4072ffd700 1 -- 172.21.15.119:0/2644582228 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f407400db20 con 0x7f407c1030e0 2023-12-07T10:19:30.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.081+0000 7f4097b04b80 1 Processor -- start 2023-12-07T10:19:30.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.081+0000 7f4072ffd700 1 --2- 172.21.15.119:0/2644582228 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f40680a1ba0 0x7f40680a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.081+0000 7f4097b04b80 1 -- start start 2023-12-07T10:19:30.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.081+0000 7f4072ffd700 1 -- 172.21.15.119:0/2644582228 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f40740cd200 con 0x7f407c1030e0 2023-12-07T10:19:30.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.081+0000 7f408179a700 1 --2- 172.21.15.119:0/2644582228 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f40680a1ba0 0x7f40680a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.081+0000 7f4097b04b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567d037d130 0x5567d037d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.081+0000 7f4097b04b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567d037da90 0x5567d037e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.081+0000 7f4097b04b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d037e980 0x5567d037ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.081+0000 7f4097b04b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d02f1770 con 0x5567d037da90 2023-12-07T10:19:30.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.081+0000 7f4097b04b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d02d8750 con 0x5567d037d130 2023-12-07T10:19:30.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.081+0000 7f4097b04b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d02d6070 con 0x5567d037e980 2023-12-07T10:19:30.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.082+0000 7f4081f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567d037d130 0x5567d037d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.082+0000 7f408179a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567d037da90 0x5567d037e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.082+0000 7f4081f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567d037d130 0x5567d037d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:41132/0 (socket says 172.21.15.119:41132) 2023-12-07T10:19:30.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.082+0000 7f4081f9b700 1 -- 172.21.15.119:0/516182918 learned_addr learned my addr 172.21.15.119:0/516182918 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:30.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.082+0000 7f408279c700 1 --2- 172.21.15.119:0/516182918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d037e980 0x5567d037ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.082+0000 7f4081f9b700 1 -- 172.21.15.119:0/516182918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d037e980 msgr2=0x5567d037ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:30.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.082+0000 7f4081f9b700 1 --2- 172.21.15.119:0/516182918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d037e980 0x5567d037ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.082+0000 7f4081f9b700 1 -- 172.21.15.119:0/516182918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567d037da90 msgr2=0x5567d037e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:30.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.082+0000 7f4081f9b700 1 --2- 172.21.15.119:0/516182918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567d037da90 0x5567d037e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.082+0000 7f4081f9b700 1 -- 172.21.15.119:0/516182918 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567d014e5e0 con 0x5567d037d130 2023-12-07T10:19:30.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.082+0000 7f408279c700 1 --2- 172.21.15.119:0/516182918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d037e980 0x5567d037ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:19:30.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.082+0000 7f408179a700 1 --2- 172.21.15.119:0/2644582228 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f40680a1ba0 0x7f40680a4060 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f406c013570 tx=0x7f406c013930 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:30.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.082+0000 7f4081f9b700 1 --2- 172.21.15.119:0/516182918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567d037d130 0x5567d037d550 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4078006010 tx=0x7f4078005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87537890d4d5219f server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:30.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.082+0000 7f4067fff700 1 -- 172.21.15.119:0/516182918 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407800ba10 con 0x5567d037d130 2023-12-07T10:19:30.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.083+0000 7f4067fff700 1 -- 172.21.15.119:0/516182918 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f407801e040 con 0x5567d037d130 2023-12-07T10:19:30.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.083+0000 7f4067fff700 1 -- 172.21.15.119:0/516182918 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4078003b40 con 0x5567d037d130 2023-12-07T10:19:30.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.083+0000 7f4097b04b80 1 -- 172.21.15.119:0/516182918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567d037d130 msgr2=0x5567d037d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.083+0000 7f4097b04b80 1 --2- 172.21.15.119:0/516182918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567d037d130 0x5567d037d550 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4078006010 tx=0x7f4078005c70 comp rx=0 tx=0).stop 2023-12-07T10:19:30.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.083+0000 7f4097b04b80 1 -- 172.21.15.119:0/516182918 shutdown_connections 2023-12-07T10:19:30.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.083+0000 7f4097b04b80 1 --2- 172.21.15.119:0/516182918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d037e980 0x5567d037ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.083+0000 7f4097b04b80 1 --2- 172.21.15.119:0/516182918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567d037d130 0x5567d037d550 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.083+0000 7f4097b04b80 1 --2- 172.21.15.119:0/516182918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567d037da90 0x5567d037e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.083+0000 7f4097b04b80 1 -- 172.21.15.119:0/516182918 >> 172.21.15.119:0/516182918 conn(0x5567d02ac310 msgr2=0x5567d0379b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:30.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.083+0000 7f4097b04b80 1 -- 172.21.15.119:0/516182918 shutdown_connections 2023-12-07T10:19:30.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.083+0000 7f4097b04b80 1 -- 172.21.15.119:0/516182918 wait complete. 2023-12-07T10:19:30.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.084+0000 7f4097b04b80 1 -- start start 2023-12-07T10:19:30.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.084+0000 7f4097b04b80 1 Processor -- start 2023-12-07T10:19:30.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.084+0000 7f4097b04b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d037d130 0x5567d038b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.084+0000 7f4097b04b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567d037da90 0x5567d038b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.084+0000 7f4097b04b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567d037e980 0x5567d0392930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.085+0000 7f4097b04b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d02d1960 con 0x5567d037e980 2023-12-07T10:19:30.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.085+0000 7f4097b04b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d028b510 con 0x5567d037da90 2023-12-07T10:19:30.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.085+0000 7f4097b04b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d028b1b0 con 0x5567d037d130 2023-12-07T10:19:30.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.085+0000 7f4097b04b80 10 client.0 _ll_register_callbacks cb 0x5567d02728f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:19:30.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.085+0000 7f408279c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567d037e980 0x5567d0392930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.085+0000 7f408279c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567d037e980 0x5567d0392930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52100/0 (socket says 172.21.15.119:52100) 2023-12-07T10:19:30.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.085+0000 7f408279c700 1 -- 172.21.15.119:0/1193467843 learned_addr learned my addr 172.21.15.119:0/1193467843 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:30.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.085+0000 7f408179a700 1 --2- 172.21.15.119:0/1193467843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567d037da90 0x5567d038b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.085+0000 7f4081f9b700 1 --2- 172.21.15.119:0/1193467843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d037d130 0x5567d038b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.085+0000 7f408279c700 1 -- 172.21.15.119:0/1193467843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d037d130 msgr2=0x5567d038b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:30.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.085+0000 7f408279c700 1 --2- 172.21.15.119:0/1193467843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d037d130 0x5567d038b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.085+0000 7f408279c700 1 -- 172.21.15.119:0/1193467843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567d037da90 msgr2=0x5567d038b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:30.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.085+0000 7f408279c700 1 --2- 172.21.15.119:0/1193467843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567d037da90 0x5567d038b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.085+0000 7f408279c700 1 -- 172.21.15.119:0/1193467843 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f407c10a2e0 con 0x5567d037e980 2023-12-07T10:19:30.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.085+0000 7f408179a700 1 --2- 172.21.15.119:0/1193467843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567d037da90 0x5567d038b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:30.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.085+0000 7f408279c700 1 --2- 172.21.15.119:0/1193467843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567d037e980 0x5567d0392930 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4074099b10 tx=0x7f407409a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9acfc0588f7d4d6c server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:30.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.085+0000 7f40657fa700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:19:30.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.086+0000 7f40657fa700 1 -- 172.21.15.119:0/1193467843 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407409df60 con 0x5567d037e980 2023-12-07T10:19:30.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.086+0000 7f4097b04b80 1 -- 172.21.15.119:0/1193467843 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5567d0370ab0 con 0x5567d037e980 2023-12-07T10:19:30.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.086+0000 7f40657fa700 1 -- 172.21.15.119:0/1193467843 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f40740a21a0 con 0x5567d037e980 2023-12-07T10:19:30.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.086+0000 7f40657fa700 1 -- 172.21.15.119:0/1193467843 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40740caa90 con 0x5567d037e980 2023-12-07T10:19:30.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.086+0000 7f4097b04b80 10 client.15168 fetch_fsmap learned FSMap version 29 2023-12-07T10:19:30.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.086+0000 7f4097b04b80 1 -- 172.21.15.119:0/1193467843 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5567d00fad90 con 0x5567d037e980 2023-12-07T10:19:30.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.086+0000 7f40657fa700 1 -- 172.21.15.119:0/1193467843 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4074063080 con 0x5567d037e980 2023-12-07T10:19:30.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.086+0000 7f40657fa700 1 -- 172.21.15.119:0/1193467843 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f4074099300 con 0x5567d037e980 2023-12-07T10:19:30.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.086+0000 7f4097b04b80 10 client.15168 fetch_fsmap finished waiting for FSMap version 29 2023-12-07T10:19:30.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.086+0000 7f4097b04b80 10 client.15168 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-07T10:19:30.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.086+0000 7f4097b04b80 20 client.15168 populate_metadata read hostname 'smithi119' 2023-12-07T10:19:30.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.086+0000 7f4097b04b80 1 --2- 172.21.15.119:0/1193467843 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x5567d0397020 0x5567d0395b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.086+0000 7f4097b04b80 4 client.15168 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:19:30.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.086+0000 7f4097b04b80 1 -- 172.21.15.119:0/1193467843 --> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567d034c750 con 0x5567d0397020 2023-12-07T10:19:30.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.087+0000 7f4081f9b700 1 --2- 172.21.15.119:0/1193467843 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x5567d0397020 0x5567d0395b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.087+0000 7f4081f9b700 1 --2- 172.21.15.119:0/1193467843 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x5567d0397020 0x5567d0395b30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6efdbd346d2c48c5 server_cookie=929ea903125bc5ce in_seq=0 out_seq=0 2023-12-07T10:19:30.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.087+0000 7f40657fa700 10 client.15168 ms_handle_connect on v2:172.21.15.159:6836/3106097520 2023-12-07T10:19:30.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.088+0000 7f40657fa700 1 -- 172.21.15.119:0/1193467843 <== mds.2 v2:172.21.15.159:6836/3106097520 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5567d034c750 con 0x5567d0397020 2023-12-07T10:19:30.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.088+0000 7f40657fa700 10 client.15168 handle_command_reply: tid=0 2023-12-07T10:19:30.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.088+0000 7f4097b04b80 1 client.15168 shutdown 2023-12-07T10:19:30.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.088+0000 7f4097b04b80 1 -- 172.21.15.119:0/1193467843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567d037e980 msgr2=0x5567d0392930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.088+0000 7f4097b04b80 1 --2- 172.21.15.119:0/1193467843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567d037e980 0x5567d0392930 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4074099b10 tx=0x7f407409a580 comp rx=0 tx=0).stop 2023-12-07T10:19:30.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.089+0000 7f4097b04b80 1 -- 172.21.15.119:0/1193467843 shutdown_connections 2023-12-07T10:19:30.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.089+0000 7f4097b04b80 1 --2- 172.21.15.119:0/1193467843 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x5567d0397020 0x5567d0395b30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.089+0000 7f4097b04b80 1 --2- 172.21.15.119:0/1193467843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567d037e980 0x5567d0392930 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.089+0000 7f4097b04b80 1 --2- 172.21.15.119:0/1193467843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d037d130 0x5567d038b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.089+0000 7f4097b04b80 1 --2- 172.21.15.119:0/1193467843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567d037da90 0x5567d038b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.089+0000 7f40657fa700 0 client.15168 ms_handle_reset on v2:172.21.15.159:6836/3106097520 2023-12-07T10:19:30.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.089+0000 7f4097b04b80 1 -- 172.21.15.119:0/1193467843 >> 172.21.15.119:0/1193467843 conn(0x5567d02ac310 msgr2=0x5567d0378990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:30.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.089+0000 7f4097b04b80 1 -- 172.21.15.119:0/1193467843 shutdown_connections 2023-12-07T10:19:30.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.089+0000 7f4097b04b80 1 -- 172.21.15.119:0/1193467843 wait complete. 2023-12-07T10:19:30.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.089+0000 7f4097b04b80 20 client.15168 trim_cache size 0 max 16384 2023-12-07T10:19:30.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.107+0000 7f4097b04b80 1 Processor -- start 2023-12-07T10:19:30.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.107+0000 7f4097b04b80 1 -- start start 2023-12-07T10:19:30.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.107+0000 7f4097b04b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567d037d130 0x5567d038a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.107+0000 7f4097b04b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567d037da90 0x5567d03928f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.107+0000 7f4097b04b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d037e980 0x5567d0379570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.107+0000 7f4097b04b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d01508d0 con 0x5567d037da90 2023-12-07T10:19:30.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.107+0000 7f4097b04b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d0274830 con 0x5567d037d130 2023-12-07T10:19:30.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.107+0000 7f4097b04b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d015cde0 con 0x5567d037e980 2023-12-07T10:19:30.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.108+0000 7f408179a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567d037da90 0x5567d03928f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.108+0000 7f408179a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567d037da90 0x5567d03928f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52112/0 (socket says 172.21.15.119:52112) 2023-12-07T10:19:30.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.108+0000 7f408179a700 1 -- 172.21.15.119:0/2756984624 learned_addr learned my addr 172.21.15.119:0/2756984624 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:30.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.108+0000 7f408279c700 1 --2- 172.21.15.119:0/2756984624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d037e980 0x5567d0379570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.108+0000 7f4081f9b700 1 --2- 172.21.15.119:0/2756984624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567d037d130 0x5567d038a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.108+0000 7f408179a700 1 -- 172.21.15.119:0/2756984624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d037e980 msgr2=0x5567d0379570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:30.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.108+0000 7f408179a700 1 --2- 172.21.15.119:0/2756984624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d037e980 0x5567d0379570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.108+0000 7f408179a700 1 -- 172.21.15.119:0/2756984624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567d037d130 msgr2=0x5567d038a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:30.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.108+0000 7f408179a700 1 --2- 172.21.15.119:0/2756984624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567d037d130 0x5567d038a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.108+0000 7f408179a700 1 -- 172.21.15.119:0/2756984624 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567d00e3c10 con 0x5567d037da90 2023-12-07T10:19:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.108+0000 7f408179a700 1 --2- 172.21.15.119:0/2756984624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567d037da90 0x5567d03928f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f406c004610 tx=0x7f406c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8437c74ec336b4 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.108+0000 7f4070ff9700 1 -- 172.21.15.119:0/2756984624 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406c018040 con 0x5567d037da90 2023-12-07T10:19:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.109+0000 7f4070ff9700 1 -- 172.21.15.119:0/2756984624 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f406c012230 con 0x5567d037da90 2023-12-07T10:19:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.109+0000 7f4070ff9700 1 -- 172.21.15.119:0/2756984624 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406c0054e0 con 0x5567d037da90 2023-12-07T10:19:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.109+0000 7f4097b04b80 1 -- 172.21.15.119:0/2756984624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567d037da90 msgr2=0x5567d03928f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.109+0000 7f4097b04b80 1 --2- 172.21.15.119:0/2756984624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567d037da90 0x5567d03928f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f406c004610 tx=0x7f406c01f000 comp rx=0 tx=0).stop 2023-12-07T10:19:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.109+0000 7f4097b04b80 1 -- 172.21.15.119:0/2756984624 shutdown_connections 2023-12-07T10:19:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.109+0000 7f4097b04b80 1 --2- 172.21.15.119:0/2756984624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d037e980 0x5567d0379570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.109+0000 7f4097b04b80 1 --2- 172.21.15.119:0/2756984624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567d037d130 0x5567d038a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.109+0000 7f4097b04b80 1 --2- 172.21.15.119:0/2756984624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567d037da90 0x5567d03928f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.109+0000 7f4097b04b80 1 -- 172.21.15.119:0/2756984624 >> 172.21.15.119:0/2756984624 conn(0x5567d02ac310 msgr2=0x5567d0384f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.109+0000 7f4097b04b80 1 -- 172.21.15.119:0/2756984624 shutdown_connections 2023-12-07T10:19:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.109+0000 7f4097b04b80 1 -- 172.21.15.119:0/2756984624 wait complete. 2023-12-07T10:19:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.110+0000 7f4097b04b80 1 -- start start 2023-12-07T10:19:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.110+0000 7f4097b04b80 1 Processor -- start 2023-12-07T10:19:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.110+0000 7f4097b04b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d037d130 0x5567d038ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.110+0000 7f4097b04b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567d037da90 0x5567d038efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.110+0000 7f4097b04b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567d037e980 0x5567d0391e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.110+0000 7f4097b04b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d03923d0 con 0x5567d037e980 2023-12-07T10:19:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.110+0000 7f4097b04b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d0392540 con 0x5567d037da90 2023-12-07T10:19:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.110+0000 7f4097b04b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d0392700 con 0x5567d037d130 2023-12-07T10:19:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.110+0000 7f4097b04b80 10 client.0 _ll_register_callbacks cb 0x5567d02728f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:19:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.110+0000 7f4081f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d037d130 0x5567d038ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.110+0000 7f4081f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d037d130 0x5567d038ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:33030/0 (socket says 172.21.15.119:33030) 2023-12-07T10:19:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.110+0000 7f4081f9b700 1 -- 172.21.15.119:0/1273590729 learned_addr learned my addr 172.21.15.119:0/1273590729 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:30.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.111+0000 7f408179a700 1 --2- 172.21.15.119:0/1273590729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567d037da90 0x5567d038efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.111+0000 7f408279c700 1 --2- 172.21.15.119:0/1273590729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567d037e980 0x5567d0391e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.111+0000 7f4081f9b700 1 -- 172.21.15.119:0/1273590729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567d037da90 msgr2=0x5567d038efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:30.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.111+0000 7f4081f9b700 1 --2- 172.21.15.119:0/1273590729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567d037da90 0x5567d038efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.111+0000 7f4081f9b700 1 -- 172.21.15.119:0/1273590729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567d037e980 msgr2=0x5567d0391e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:30.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.111+0000 7f4081f9b700 1 --2- 172.21.15.119:0/1273590729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567d037e980 0x5567d0391e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.111+0000 7f4081f9b700 1 -- 172.21.15.119:0/1273590729 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f406c004350 con 0x5567d037d130 2023-12-07T10:19:30.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.111+0000 7f4081f9b700 1 --2- 172.21.15.119:0/1273590729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d037d130 0x5567d038ea70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4078006010 tx=0x7f4078009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42bff8348781e7be server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:30.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.111+0000 7f40657fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:19:30.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.111+0000 7f40657fa700 1 -- 172.21.15.119:0/1273590729 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40780052c0 con 0x5567d037d130 2023-12-07T10:19:30.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.111+0000 7f4097b04b80 1 -- 172.21.15.119:0/1273590729 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f407801e040 con 0x5567d037d130 2023-12-07T10:19:30.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.111+0000 7f40657fa700 1 -- 172.21.15.119:0/1273590729 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f4078005420 con 0x5567d037d130 2023-12-07T10:19:30.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.112+0000 7f40657fa700 1 -- 172.21.15.119:0/1273590729 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40780055e0 con 0x5567d037d130 2023-12-07T10:19:30.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.112+0000 7f4097b04b80 10 client.25183 fetch_fsmap learned FSMap version 29 2023-12-07T10:19:30.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.112+0000 7f4097b04b80 1 -- 172.21.15.119:0/1273590729 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5567d0121200 con 0x5567d037d130 2023-12-07T10:19:30.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.112+0000 7f40657fa700 1 -- 172.21.15.119:0/1273590729 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f407801e040 con 0x5567d037d130 2023-12-07T10:19:30.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.112+0000 7f40657fa700 1 -- 172.21.15.119:0/1273590729 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f407802f3b0 con 0x5567d037d130 2023-12-07T10:19:30.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.112+0000 7f4097b04b80 10 client.25183 fetch_fsmap finished waiting for FSMap version 29 2023-12-07T10:19:30.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.112+0000 7f4097b04b80 10 client.25183 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-07T10:19:30.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.112+0000 7f4097b04b80 20 client.25183 populate_metadata read hostname 'smithi119' 2023-12-07T10:19:30.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.112+0000 7f4097b04b80 1 --2- 172.21.15.119:0/1273590729 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x5567d03f1850 0x5567d03edcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.112+0000 7f4097b04b80 4 client.25183 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-07T10:19:30.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.112+0000 7f4097b04b80 1 -- 172.21.15.119:0/1273590729 --> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5567d0297f80 con 0x5567d03f1850 2023-12-07T10:19:30.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.112+0000 7f408179a700 1 --2- 172.21.15.119:0/1273590729 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x5567d03f1850 0x5567d03edcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.113+0000 7f408179a700 1 --2- 172.21.15.119:0/1273590729 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x5567d03f1850 0x5567d03edcc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=df956a1d16813adc server_cookie=147c881b25a57f78 in_seq=0 out_seq=0 2023-12-07T10:19:30.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.113+0000 7f40657fa700 10 client.25183 ms_handle_connect on v2:172.21.15.159:6836/3106097520 2023-12-07T10:19:30.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.113+0000 7f40657fa700 1 -- 172.21.15.119:0/1273590729 <== mds.2 v2:172.21.15.159:6836/3106097520 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5567d0297f80 con 0x5567d03f1850 2023-12-07T10:19:30.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.113+0000 7f40657fa700 10 client.25183 handle_command_reply: tid=0 2023-12-07T10:19:30.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.113+0000 7f4097b04b80 1 client.25183 shutdown 2023-12-07T10:19:30.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.113+0000 7f4097b04b80 1 -- 172.21.15.119:0/1273590729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d037d130 msgr2=0x5567d038ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.113+0000 7f4097b04b80 1 --2- 172.21.15.119:0/1273590729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d037d130 0x5567d038ea70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4078006010 tx=0x7f4078009310 comp rx=0 tx=0).stop 2023-12-07T10:19:30.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.113+0000 7f4097b04b80 1 -- 172.21.15.119:0/1273590729 shutdown_connections 2023-12-07T10:19:30.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.113+0000 7f4097b04b80 1 --2- 172.21.15.119:0/1273590729 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x5567d03f1850 0x5567d03edcc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.113+0000 7f4097b04b80 1 --2- 172.21.15.119:0/1273590729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567d037e980 0x5567d0391e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.113+0000 7f4097b04b80 1 --2- 172.21.15.119:0/1273590729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d037d130 0x5567d038ea70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.113+0000 7f40657fa700 0 client.25183 ms_handle_reset on v2:172.21.15.159:6836/3106097520 2023-12-07T10:19:30.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.113+0000 7f4097b04b80 1 --2- 172.21.15.119:0/1273590729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5567d037da90 0x5567d038efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.113+0000 7f4097b04b80 1 -- 172.21.15.119:0/1273590729 >> 172.21.15.119:0/1273590729 conn(0x5567d02ac310 msgr2=0x5567d0381920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:30.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.114+0000 7f4097b04b80 1 -- 172.21.15.119:0/1273590729 shutdown_connections 2023-12-07T10:19:30.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.114+0000 7f4097b04b80 1 -- 172.21.15.119:0/1273590729 wait complete. 2023-12-07T10:19:30.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.114+0000 7f4097b04b80 20 client.25183 trim_cache size 0 max 16384 2023-12-07T10:19:30.120 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:19:30.120 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "a4c642c6-94e8-11ee-95a2-87774f69a715", 2023-12-07T10:19:30.120 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 2, 2023-12-07T10:19:30.120 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 24517, 2023-12-07T10:19:30.120 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-12-07T10:19:30.120 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-12-07T10:19:30.120 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-12-07T10:19:30.120 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 79.382185792000001, 2023-12-07T10:19:30.120 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 29, 2023-12-07T10:19:30.120 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 93, 2023-12-07T10:19:30.120 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 91, 2023-12-07T10:19:30.120 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 129.83733647299999 2023-12-07T10:19:30.120 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:19:30.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.114+0000 7f40841ff700 1 -- 172.21.15.119:0/2644582228 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f40680a1ba0 msgr2=0x7f40680a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.114+0000 7f40841ff700 1 --2- 172.21.15.119:0/2644582228 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f40680a1ba0 0x7f40680a4060 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f406c013570 tx=0x7f406c013930 comp rx=0 tx=0).stop 2023-12-07T10:19:30.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.114+0000 7f40841ff700 1 -- 172.21.15.119:0/2644582228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f407c1030e0 msgr2=0x7f407c19ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.114+0000 7f40841ff700 1 --2- 172.21.15.119:0/2644582228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f407c1030e0 0x7f407c19ea40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4074007ee0 tx=0x7f407400d3b0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.114+0000 7f40841ff700 1 -- 172.21.15.119:0/2644582228 shutdown_connections 2023-12-07T10:19:30.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.114+0000 7f40841ff700 1 --2- 172.21.15.119:0/2644582228 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f40680a1ba0 0x7f40680a4060 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.114+0000 7f40841ff700 1 --2- 172.21.15.119:0/2644582228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f407c1030e0 0x7f407c19ea40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.114+0000 7f40841ff700 1 --2- 172.21.15.119:0/2644582228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f407c101550 0x7f407c1974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.114+0000 7f40841ff700 1 --2- 172.21.15.119:0/2644582228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f407c102740 0x7f407c197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.114+0000 7f40841ff700 1 -- 172.21.15.119:0/2644582228 >> 172.21.15.119:0/2644582228 conn(0x7f407c0fcae0 msgr2=0x7f407c105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:30.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.115+0000 7f40841ff700 1 -- 172.21.15.119:0/2644582228 shutdown_connections 2023-12-07T10:19:30.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.115+0000 7f40841ff700 1 -- 172.21.15.119:0/2644582228 wait complete. 2023-12-07T10:19:30.132 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:29 smithi119 ceph-mon[97908]: pgmap v337: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-07T10:19:30.132 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:29 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1851376352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:19:30.134 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2023-12-07T10:19:30.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:29 smithi159 ceph-mon[134726]: pgmap v337: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-07T10:19:30.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:29 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1851376352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:19:30.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:29 smithi153 ceph-mon[133625]: pgmap v337: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-07T10:19:30.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:29 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1851376352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:19:30.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.349+0000 7fe236fff700 1 -- 172.21.15.119:0/1676634540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe230102730 msgr2=0x7fe230102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.349+0000 7fe236fff700 1 --2- 172.21.15.119:0/1676634540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe230102730 0x7fe230102b90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009990 tx=0x7fe21c009ca0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.349+0000 7fe236fff700 1 -- 172.21.15.119:0/1676634540 shutdown_connections 2023-12-07T10:19:30.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.349+0000 7fe236fff700 1 --2- 172.21.15.119:0/1676634540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2301030d0 0x7fe230109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.349+0000 7fe236fff700 1 --2- 172.21.15.119:0/1676634540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe230101540 0x7fe230101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.349+0000 7fe236fff700 1 --2- 172.21.15.119:0/1676634540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe230102730 0x7fe230102b90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.349+0000 7fe236fff700 1 -- 172.21.15.119:0/1676634540 >> 172.21.15.119:0/1676634540 conn(0x7fe2300fcab0 msgr2=0x7fe2300feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:30.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.349+0000 7fe236fff700 1 -- 172.21.15.119:0/1676634540 shutdown_connections 2023-12-07T10:19:30.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.349+0000 7fe236fff700 1 -- 172.21.15.119:0/1676634540 wait complete. 2023-12-07T10:19:30.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.350+0000 7fe236fff700 1 Processor -- start 2023-12-07T10:19:30.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.350+0000 7fe236fff700 1 -- start start 2023-12-07T10:19:30.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.350+0000 7fe236fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe230101540 0x7fe2301996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.351+0000 7fe236fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe230102730 0x7fe230199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.351+0000 7fe236fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2301030d0 0x7fe2301a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.351+0000 7fe236fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2301a11d0 con 0x7fe2301030d0 2023-12-07T10:19:30.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.351+0000 7fe236fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2301a1310 con 0x7fe230101540 2023-12-07T10:19:30.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.351+0000 7fe236fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2301a14a0 con 0x7fe230102730 2023-12-07T10:19:30.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.351+0000 7fe23559c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2301030d0 0x7fe2301a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.351+0000 7fe23559c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2301030d0 0x7fe2301a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52150/0 (socket says 172.21.15.119:52150) 2023-12-07T10:19:30.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.351+0000 7fe23559c700 1 -- 172.21.15.119:0/3196234741 learned_addr learned my addr 172.21.15.119:0/3196234741 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:30.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.351+0000 7fe234d9b700 1 --2- 172.21.15.119:0/3196234741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe230101540 0x7fe2301996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.351+0000 7fe22ffff700 1 --2- 172.21.15.119:0/3196234741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe230102730 0x7fe230199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.351+0000 7fe23559c700 1 -- 172.21.15.119:0/3196234741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe230102730 msgr2=0x7fe230199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.351+0000 7fe23559c700 1 --2- 172.21.15.119:0/3196234741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe230102730 0x7fe230199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.351+0000 7fe23559c700 1 -- 172.21.15.119:0/3196234741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe230101540 msgr2=0x7fe2301996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.351+0000 7fe23559c700 1 --2- 172.21.15.119:0/3196234741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe230101540 0x7fe2301996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.351+0000 7fe23559c700 1 -- 172.21.15.119:0/3196234741 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe224009520 con 0x7fe2301030d0 2023-12-07T10:19:30.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.352+0000 7fe23559c700 1 --2- 172.21.15.119:0/3196234741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2301030d0 0x7fe2301a0c00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe22800ef30 tx=0x7fe22800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:30.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.352+0000 7fe22dffb700 1 -- 172.21.15.119:0/3196234741 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe228018070 con 0x7fe2301030d0 2023-12-07T10:19:30.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.352+0000 7fe236fff700 1 -- 172.21.15.119:0/3196234741 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21c0095f0 con 0x7fe2301030d0 2023-12-07T10:19:30.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.352+0000 7fe22dffb700 1 -- 172.21.15.119:0/3196234741 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe22800f040 con 0x7fe2301030d0 2023-12-07T10:19:30.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.352+0000 7fe22dffb700 1 -- 172.21.15.119:0/3196234741 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22800cd00 con 0x7fe2301030d0 2023-12-07T10:19:30.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.352+0000 7fe236fff700 1 -- 172.21.15.119:0/3196234741 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2301a1a50 con 0x7fe2301030d0 2023-12-07T10:19:30.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.355+0000 7fe22dffb700 1 -- 172.21.15.119:0/3196234741 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7fe228003a00 con 0x7fe2301030d0 2023-12-07T10:19:30.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.356+0000 7fe24a981b80 1 Processor -- start 2023-12-07T10:19:30.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.356+0000 7fe22dffb700 1 --2- 172.21.15.119:0/3196234741 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe2200a1ba0 0x7fe2200a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.356+0000 7fe24a981b80 1 -- start start 2023-12-07T10:19:30.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.356+0000 7fe22dffb700 1 -- 172.21.15.119:0/3196234741 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fe2280cd290 con 0x7fe2301030d0 2023-12-07T10:19:30.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.356+0000 7fe24a981b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361130 0x55e5cf361550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.356+0000 7fe22ffff700 1 --2- 172.21.15.119:0/3196234741 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe2200a1ba0 0x7fe2200a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.356+0000 7fe24a981b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf361a90 0x55e5cf3623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.357+0000 7fe24a981b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf362980 0x55e5cf362e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.357+0000 7fe24a981b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5cf2d5770 con 0x55e5cf361130 2023-12-07T10:19:30.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.357+0000 7fe24a981b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5cf2bc750 con 0x55e5cf362980 2023-12-07T10:19:30.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.357+0000 7fe24a981b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5cf2ba070 con 0x55e5cf361a90 2023-12-07T10:19:30.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.357+0000 7fe234d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361130 0x55e5cf361550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.357+0000 7fe22ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf361a90 0x55e5cf3623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.357+0000 7fe234d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361130 0x55e5cf361550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52162/0 (socket says 172.21.15.119:52162) 2023-12-07T10:19:30.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.357+0000 7fe234d9b700 1 -- 172.21.15.119:0/278121831 learned_addr learned my addr 172.21.15.119:0/278121831 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:30.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.357+0000 7fe23559c700 1 --2- 172.21.15.119:0/278121831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf362980 0x55e5cf362e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.357+0000 7fe234d9b700 1 -- 172.21.15.119:0/278121831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf361a90 msgr2=0x55e5cf3623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:30.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.357+0000 7fe234d9b700 1 --2- 172.21.15.119:0/278121831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf361a90 0x55e5cf3623a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.357+0000 7fe234d9b700 1 -- 172.21.15.119:0/278121831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf362980 msgr2=0x55e5cf362e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:30.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.357+0000 7fe234d9b700 1 --2- 172.21.15.119:0/278121831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf362980 0x55e5cf362e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.357+0000 7fe22ffff700 1 --2- 172.21.15.119:0/3196234741 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe2200a1ba0 0x7fe2200a4060 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00b2e0 tx=0x7fe21c00bbf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:30.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.357+0000 7fe234d9b700 1 -- 172.21.15.119:0/278121831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5cf1325e0 con 0x55e5cf361130 2023-12-07T10:19:30.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.357+0000 7fe23559c700 1 --2- 172.21.15.119:0/278121831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf362980 0x55e5cf362e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:30.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.358+0000 7fe234d9b700 1 --2- 172.21.15.119:0/278121831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361130 0x55e5cf361550 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe224009eb0 tx=0x7fe224011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2deacdba6d5505 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:30.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.358+0000 7fe21affd700 1 -- 172.21.15.119:0/278121831 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe224019630 con 0x55e5cf361130 2023-12-07T10:19:30.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.358+0000 7fe21affd700 1 -- 172.21.15.119:0/278121831 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe224019c50 con 0x55e5cf361130 2023-12-07T10:19:30.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.358+0000 7fe21affd700 1 -- 172.21.15.119:0/278121831 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22400ed10 con 0x55e5cf361130 2023-12-07T10:19:30.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.359+0000 7fe24a981b80 1 -- 172.21.15.119:0/278121831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361130 msgr2=0x55e5cf361550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.359+0000 7fe24a981b80 1 --2- 172.21.15.119:0/278121831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361130 0x55e5cf361550 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe224009eb0 tx=0x7fe224011720 comp rx=0 tx=0).stop 2023-12-07T10:19:30.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.359+0000 7fe24a981b80 1 -- 172.21.15.119:0/278121831 shutdown_connections 2023-12-07T10:19:30.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.359+0000 7fe24a981b80 1 --2- 172.21.15.119:0/278121831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf362980 0x55e5cf362e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.359+0000 7fe24a981b80 1 --2- 172.21.15.119:0/278121831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361130 0x55e5cf361550 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.359+0000 7fe24a981b80 1 --2- 172.21.15.119:0/278121831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf361a90 0x55e5cf3623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.359+0000 7fe24a981b80 1 -- 172.21.15.119:0/278121831 >> 172.21.15.119:0/278121831 conn(0x55e5cf290310 msgr2=0x55e5cf35db00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:30.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.359+0000 7fe24a981b80 1 -- 172.21.15.119:0/278121831 shutdown_connections 2023-12-07T10:19:30.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.359+0000 7fe24a981b80 1 -- 172.21.15.119:0/278121831 wait complete. 2023-12-07T10:19:30.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.359+0000 7fe24a981b80 1 -- start start 2023-12-07T10:19:30.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.360+0000 7fe24a981b80 1 Processor -- start 2023-12-07T10:19:30.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.360+0000 7fe24a981b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361130 0x55e5cf36af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.360+0000 7fe24a981b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf361a90 0x55e5cf36b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.360+0000 7fe24a981b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf362980 0x55e5cf3724c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.360+0000 7fe24a981b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5cf2b5960 con 0x55e5cf361130 2023-12-07T10:19:30.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.360+0000 7fe24a981b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5cf26f510 con 0x55e5cf361a90 2023-12-07T10:19:30.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.360+0000 7fe24a981b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5cf26f1b0 con 0x55e5cf362980 2023-12-07T10:19:30.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.361+0000 7fe24a981b80 10 client.0 _ll_register_callbacks cb 0x55e5cf2568f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:19:30.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.361+0000 7fe234d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361130 0x55e5cf36af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.361+0000 7fe234d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361130 0x55e5cf36af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52168/0 (socket says 172.21.15.119:52168) 2023-12-07T10:19:30.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.361+0000 7fe234d9b700 1 -- 172.21.15.119:0/308707376 learned_addr learned my addr 172.21.15.119:0/308707376 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:30.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.361+0000 7fe22ffff700 1 --2- 172.21.15.119:0/308707376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf361a90 0x55e5cf36b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.361+0000 7fe23559c700 1 --2- 172.21.15.119:0/308707376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf362980 0x55e5cf3724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.361+0000 7fe234d9b700 1 -- 172.21.15.119:0/308707376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf362980 msgr2=0x55e5cf3724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:30.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.361+0000 7fe234d9b700 1 --2- 172.21.15.119:0/308707376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf362980 0x55e5cf3724c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.361+0000 7fe234d9b700 1 -- 172.21.15.119:0/308707376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf361a90 msgr2=0x55e5cf36b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:30.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.361+0000 7fe234d9b700 1 --2- 172.21.15.119:0/308707376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf361a90 0x55e5cf36b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.361+0000 7fe234d9b700 1 -- 172.21.15.119:0/308707376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2301a1a50 con 0x55e5cf361130 2023-12-07T10:19:30.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.361+0000 7fe234d9b700 1 --2- 172.21.15.119:0/308707376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361130 0x55e5cf36af40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe224009840 tx=0x7fe22400e810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11981889eac03fd server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:30.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.361+0000 7fe20ffff700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:19:30.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.361+0000 7fe20ffff700 1 -- 172.21.15.119:0/308707376 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe224004230 con 0x55e5cf361130 2023-12-07T10:19:30.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.362+0000 7fe20ffff700 1 -- 172.21.15.119:0/308707376 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fe22400ee80 con 0x55e5cf361130 2023-12-07T10:19:30.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.362+0000 7fe24a981b80 1 -- 172.21.15.119:0/308707376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e5cf354ab0 con 0x55e5cf361130 2023-12-07T10:19:30.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.362+0000 7fe20ffff700 1 -- 172.21.15.119:0/308707376 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe224004390 con 0x55e5cf361130 2023-12-07T10:19:30.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.362+0000 7fe24a981b80 10 client.15186 fetch_fsmap learned FSMap version 29 2023-12-07T10:19:30.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.362+0000 7fe24a981b80 1 -- 172.21.15.119:0/308707376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fe224009ae0 con 0x55e5cf361130 2023-12-07T10:19:30.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.362+0000 7fe20ffff700 1 -- 172.21.15.119:0/308707376 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e5cf354ab0 con 0x55e5cf361130 2023-12-07T10:19:30.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.362+0000 7fe20ffff700 1 -- 172.21.15.119:0/308707376 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fe22402c060 con 0x55e5cf361130 2023-12-07T10:19:30.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.362+0000 7fe24a981b80 10 client.15186 fetch_fsmap finished waiting for FSMap version 29 2023-12-07T10:19:30.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.362+0000 7fe24a981b80 10 client.15186 resolve_mds: resolved name 'g' to daemon mds.g 2023-12-07T10:19:30.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.362+0000 7fe24a981b80 20 client.15186 populate_metadata read hostname 'smithi119' 2023-12-07T10:19:30.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.362+0000 7fe24a981b80 1 --2- 172.21.15.119:0/308707376 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55e5cf376a40 0x55e5cf375550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.362+0000 7fe24a981b80 4 client.15186 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:19:30.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.362+0000 7fe24a981b80 1 -- 172.21.15.119:0/308707376 --> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5cf330750 con 0x55e5cf376a40 2023-12-07T10:19:30.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.363+0000 7fe22ffff700 1 --2- 172.21.15.119:0/308707376 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55e5cf376a40 0x55e5cf375550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.363+0000 7fe20ffff700 10 client.15186 ms_handle_connect on v2:172.21.15.119:6838/1920762602 2023-12-07T10:19:30.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.363+0000 7fe22ffff700 1 --2- 172.21.15.119:0/308707376 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55e5cf376a40 0x55e5cf375550 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e979800329a44b7d server_cookie=5ab76f2c16eb4c04 in_seq=0 out_seq=0 2023-12-07T10:19:30.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.364+0000 7fe20ffff700 1 -- 172.21.15.119:0/308707376 <== mds.1 v2:172.21.15.119:6838/1920762602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e5cf330750 con 0x55e5cf376a40 2023-12-07T10:19:30.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.364+0000 7fe20ffff700 10 client.15186 handle_command_reply: tid=0 2023-12-07T10:19:30.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.364+0000 7fe24a981b80 1 client.15186 shutdown 2023-12-07T10:19:30.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.364+0000 7fe24a981b80 1 -- 172.21.15.119:0/308707376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361130 msgr2=0x55e5cf36af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.364+0000 7fe24a981b80 1 --2- 172.21.15.119:0/308707376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361130 0x55e5cf36af40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe224009840 tx=0x7fe22400e810 comp rx=0 tx=0).stop 2023-12-07T10:19:30.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.364+0000 7fe24a981b80 1 -- 172.21.15.119:0/308707376 shutdown_connections 2023-12-07T10:19:30.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.364+0000 7fe24a981b80 1 --2- 172.21.15.119:0/308707376 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55e5cf376a40 0x55e5cf375550 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.364+0000 7fe24a981b80 1 --2- 172.21.15.119:0/308707376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf362980 0x55e5cf3724c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.364+0000 7fe24a981b80 1 --2- 172.21.15.119:0/308707376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361130 0x55e5cf36af40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.364+0000 7fe24a981b80 1 --2- 172.21.15.119:0/308707376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf361a90 0x55e5cf36b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.364+0000 7fe24a981b80 1 -- 172.21.15.119:0/308707376 >> 172.21.15.119:0/308707376 conn(0x55e5cf290310 msgr2=0x55e5cf35c990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:30.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.364+0000 7fe20ffff700 0 client.15186 ms_handle_reset on v2:172.21.15.119:6838/1920762602 2023-12-07T10:19:30.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.364+0000 7fe24a981b80 1 -- 172.21.15.119:0/308707376 shutdown_connections 2023-12-07T10:19:30.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.364+0000 7fe24a981b80 1 -- 172.21.15.119:0/308707376 wait complete. 2023-12-07T10:19:30.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.365+0000 7fe24a981b80 20 client.15186 trim_cache size 0 max 16384 2023-12-07T10:19:30.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.383+0000 7fe24a981b80 1 Processor -- start 2023-12-07T10:19:30.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.383+0000 7fe24a981b80 1 -- start start 2023-12-07T10:19:30.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.383+0000 7fe24a981b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf361130 0x55e5cf3658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.383+0000 7fe24a981b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361a90 0x55e5cf365df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.383+0000 7fe24a981b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf362980 0x55e5cf372480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.383+0000 7fe24a981b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5cf1348d0 con 0x55e5cf361a90 2023-12-07T10:19:30.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.383+0000 7fe24a981b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5cf140de0 con 0x55e5cf362980 2023-12-07T10:19:30.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.383+0000 7fe24a981b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5cf251360 con 0x55e5cf361130 2023-12-07T10:19:30.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.384+0000 7fe22ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361a90 0x55e5cf365df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.384+0000 7fe22ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361a90 0x55e5cf365df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52170/0 (socket says 172.21.15.119:52170) 2023-12-07T10:19:30.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.384+0000 7fe22ffff700 1 -- 172.21.15.119:0/3758976957 learned_addr learned my addr 172.21.15.119:0/3758976957 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:30.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.384+0000 7fe23559c700 1 --2- 172.21.15.119:0/3758976957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf362980 0x55e5cf372480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.384+0000 7fe234d9b700 1 --2- 172.21.15.119:0/3758976957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf361130 0x55e5cf3658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.384+0000 7fe22ffff700 1 -- 172.21.15.119:0/3758976957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf361130 msgr2=0x55e5cf3658b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:30.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.384+0000 7fe22ffff700 1 --2- 172.21.15.119:0/3758976957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf361130 0x55e5cf3658b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.384+0000 7fe22ffff700 1 -- 172.21.15.119:0/3758976957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf362980 msgr2=0x55e5cf372480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:30.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.384+0000 7fe22ffff700 1 --2- 172.21.15.119:0/3758976957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf362980 0x55e5cf372480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.384+0000 7fe22ffff700 1 -- 172.21.15.119:0/3758976957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5cf0ded90 con 0x55e5cf361a90 2023-12-07T10:19:30.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.384+0000 7fe22ffff700 1 --2- 172.21.15.119:0/3758976957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361a90 0x55e5cf365df0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009990 tx=0x7fe21c027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4558604648c560 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:30.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.385+0000 7fe21b7fe700 1 -- 172.21.15.119:0/3758976957 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c01d040 con 0x55e5cf361a90 2023-12-07T10:19:30.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.385+0000 7fe21b7fe700 1 -- 172.21.15.119:0/3758976957 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe21c004da0 con 0x55e5cf361a90 2023-12-07T10:19:30.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.385+0000 7fe21b7fe700 1 -- 172.21.15.119:0/3758976957 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c0358a0 con 0x55e5cf361a90 2023-12-07T10:19:30.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.385+0000 7fe24a981b80 1 -- 172.21.15.119:0/3758976957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361a90 msgr2=0x55e5cf365df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.385+0000 7fe24a981b80 1 --2- 172.21.15.119:0/3758976957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361a90 0x55e5cf365df0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009990 tx=0x7fe21c027000 comp rx=0 tx=0).stop 2023-12-07T10:19:30.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.385+0000 7fe24a981b80 1 -- 172.21.15.119:0/3758976957 shutdown_connections 2023-12-07T10:19:30.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.385+0000 7fe24a981b80 1 --2- 172.21.15.119:0/3758976957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf362980 0x55e5cf372480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.385+0000 7fe24a981b80 1 --2- 172.21.15.119:0/3758976957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf361130 0x55e5cf3658b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.385+0000 7fe24a981b80 1 --2- 172.21.15.119:0/3758976957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361a90 0x55e5cf365df0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.385+0000 7fe24a981b80 1 -- 172.21.15.119:0/3758976957 >> 172.21.15.119:0/3758976957 conn(0x55e5cf290310 msgr2=0x55e5cf36a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:30.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.385+0000 7fe24a981b80 1 -- 172.21.15.119:0/3758976957 shutdown_connections 2023-12-07T10:19:30.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.385+0000 7fe24a981b80 1 -- 172.21.15.119:0/3758976957 wait complete. 2023-12-07T10:19:30.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.386+0000 7fe24a981b80 1 -- start start 2023-12-07T10:19:30.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.386+0000 7fe24a981b80 1 Processor -- start 2023-12-07T10:19:30.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.386+0000 7fe24a981b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361130 0x55e5cf36cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.386+0000 7fe24a981b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf361a90 0x55e5cf3723d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.386+0000 7fe24a981b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf362980 0x55e5cf36d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.386+0000 7fe24a981b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5cf36db20 con 0x55e5cf361130 2023-12-07T10:19:30.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.386+0000 7fe24a981b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5cf3ccb30 con 0x55e5cf361a90 2023-12-07T10:19:30.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.386+0000 7fe24a981b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5cf3ccc70 con 0x55e5cf362980 2023-12-07T10:19:30.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.386+0000 7fe24a981b80 10 client.0 _ll_register_callbacks cb 0x55e5cf2568f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:19:30.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.387+0000 7fe234d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361130 0x55e5cf36cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.387+0000 7fe22ffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf361a90 0x55e5cf3723d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.387+0000 7fe22ffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf361a90 0x55e5cf3723d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:41236/0 (socket says 172.21.15.119:41236) 2023-12-07T10:19:30.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.387+0000 7fe22ffff700 1 -- 172.21.15.119:0/1444930056 learned_addr learned my addr 172.21.15.119:0/1444930056 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:30.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.387+0000 7fe23559c700 1 --2- 172.21.15.119:0/1444930056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf362980 0x55e5cf36d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.387+0000 7fe22ffff700 1 -- 172.21.15.119:0/1444930056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf362980 msgr2=0x55e5cf36d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:30.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.387+0000 7fe22ffff700 1 --2- 172.21.15.119:0/1444930056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf362980 0x55e5cf36d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.387+0000 7fe22ffff700 1 -- 172.21.15.119:0/1444930056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361130 msgr2=0x55e5cf36cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:30.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.387+0000 7fe22ffff700 1 --2- 172.21.15.119:0/1444930056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361130 0x55e5cf36cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.387+0000 7fe22ffff700 1 -- 172.21.15.119:0/1444930056 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe21c020040 con 0x55e5cf361a90 2023-12-07T10:19:30.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.387+0000 7fe234d9b700 1 --2- 172.21.15.119:0/1444930056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361130 0x55e5cf36cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:30.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.387+0000 7fe22ffff700 1 --2- 172.21.15.119:0/1444930056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf361a90 0x55e5cf3723d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009ce0 tx=0x7fe21c00b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b72a895db6a3da3 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:30.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.387+0000 7fe20ffff700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:19:30.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.387+0000 7fe20ffff700 1 -- 172.21.15.119:0/1444930056 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c01d030 con 0x55e5cf361a90 2023-12-07T10:19:30.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.387+0000 7fe24a981b80 1 -- 172.21.15.119:0/1444930056 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe224019c50 con 0x55e5cf361a90 2023-12-07T10:19:30.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.388+0000 7fe20ffff700 1 -- 172.21.15.119:0/1444930056 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fe21c0351a0 con 0x55e5cf361a90 2023-12-07T10:19:30.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.388+0000 7fe20ffff700 1 -- 172.21.15.119:0/1444930056 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe21c0355e0 con 0x55e5cf361a90 2023-12-07T10:19:30.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.388+0000 7fe24a981b80 10 client.25217 fetch_fsmap learned FSMap version 29 2023-12-07T10:19:30.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.388+0000 7fe24a981b80 1 -- 172.21.15.119:0/1444930056 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e5cf0c7c10 con 0x55e5cf361a90 2023-12-07T10:19:30.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.388+0000 7fe20ffff700 1 -- 172.21.15.119:0/1444930056 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe224019c50 con 0x55e5cf361a90 2023-12-07T10:19:30.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.388+0000 7fe20ffff700 1 -- 172.21.15.119:0/1444930056 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fe21c03e070 con 0x55e5cf361a90 2023-12-07T10:19:30.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.388+0000 7fe24a981b80 10 client.25217 fetch_fsmap finished waiting for FSMap version 29 2023-12-07T10:19:30.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.388+0000 7fe24a981b80 10 client.25217 resolve_mds: resolved name 'g' to daemon mds.g 2023-12-07T10:19:30.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.388+0000 7fe24a981b80 20 client.25217 populate_metadata read hostname 'smithi119' 2023-12-07T10:19:30.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.388+0000 7fe24a981b80 1 --2- 172.21.15.119:0/1444930056 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55e5cf3d1ba0 0x55e5cf3ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.388+0000 7fe24a981b80 4 client.25217 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-07T10:19:30.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.388+0000 7fe24a981b80 1 -- 172.21.15.119:0/1444930056 --> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e5cf27bf80 con 0x55e5cf3d1ba0 2023-12-07T10:19:30.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.389+0000 7fe234d9b700 1 --2- 172.21.15.119:0/1444930056 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55e5cf3d1ba0 0x55e5cf3ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.389+0000 7fe234d9b700 1 --2- 172.21.15.119:0/1444930056 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55e5cf3d1ba0 0x55e5cf3ce050 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3cb1a217714ba2d9 server_cookie=92f9bc8a47adc1d7 in_seq=0 out_seq=0 2023-12-07T10:19:30.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.389+0000 7fe20ffff700 10 client.25217 ms_handle_connect on v2:172.21.15.119:6838/1920762602 2023-12-07T10:19:30.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.389+0000 7fe20ffff700 1 -- 172.21.15.119:0/1444930056 <== mds.1 v2:172.21.15.119:6838/1920762602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55e5cf27bf80 con 0x55e5cf3d1ba0 2023-12-07T10:19:30.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.389+0000 7fe20ffff700 10 client.25217 handle_command_reply: tid=0 2023-12-07T10:19:30.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.389+0000 7fe24a981b80 1 client.25217 shutdown 2023-12-07T10:19:30.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.389+0000 7fe24a981b80 1 -- 172.21.15.119:0/1444930056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf361a90 msgr2=0x55e5cf3723d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.389+0000 7fe24a981b80 1 --2- 172.21.15.119:0/1444930056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf361a90 0x55e5cf3723d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009ce0 tx=0x7fe21c00b850 comp rx=0 tx=0).stop 2023-12-07T10:19:30.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.389+0000 7fe24a981b80 1 -- 172.21.15.119:0/1444930056 shutdown_connections 2023-12-07T10:19:30.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.389+0000 7fe24a981b80 1 --2- 172.21.15.119:0/1444930056 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55e5cf3d1ba0 0x55e5cf3ce050 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.389+0000 7fe24a981b80 1 --2- 172.21.15.119:0/1444930056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf362980 0x55e5cf36d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.389+0000 7fe24a981b80 1 --2- 172.21.15.119:0/1444930056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf361130 0x55e5cf36cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.389+0000 7fe24a981b80 1 --2- 172.21.15.119:0/1444930056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf361a90 0x55e5cf3723d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.389+0000 7fe20ffff700 0 client.25217 ms_handle_reset on v2:172.21.15.119:6838/1920762602 2023-12-07T10:19:30.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.389+0000 7fe24a981b80 1 -- 172.21.15.119:0/1444930056 >> 172.21.15.119:0/1444930056 conn(0x55e5cf290310 msgr2=0x55e5cf36a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:30.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.390+0000 7fe24a981b80 1 -- 172.21.15.119:0/1444930056 shutdown_connections 2023-12-07T10:19:30.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.390+0000 7fe24a981b80 1 -- 172.21.15.119:0/1444930056 wait complete. 2023-12-07T10:19:30.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.390+0000 7fe24a981b80 20 client.25217 trim_cache size 0 max 16384 2023-12-07T10:19:30.400 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:19:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "a4c642c6-94e8-11ee-95a2-87774f69a715", 2023-12-07T10:19:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 1, 2023-12-07T10:19:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 24542, 2023-12-07T10:19:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-12-07T10:19:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-12-07T10:19:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-12-07T10:19:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 80.499549150999997, 2023-12-07T10:19:30.401 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 29, 2023-12-07T10:19:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 93, 2023-12-07T10:19:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 91, 2023-12-07T10:19:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 122.347373078 2023-12-07T10:19:30.402 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:19:30.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.390+0000 7fe236fff700 1 -- 172.21.15.119:0/3196234741 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe2200a1ba0 msgr2=0x7fe2200a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.390+0000 7fe236fff700 1 --2- 172.21.15.119:0/3196234741 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe2200a1ba0 0x7fe2200a4060 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00b2e0 tx=0x7fe21c00bbf0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.390+0000 7fe236fff700 1 -- 172.21.15.119:0/3196234741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2301030d0 msgr2=0x7fe2301a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.390+0000 7fe236fff700 1 --2- 172.21.15.119:0/3196234741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2301030d0 0x7fe2301a0c00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe22800ef30 tx=0x7fe22800c3a0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.390+0000 7fe236fff700 1 -- 172.21.15.119:0/3196234741 shutdown_connections 2023-12-07T10:19:30.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.390+0000 7fe236fff700 1 --2- 172.21.15.119:0/3196234741 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe2200a1ba0 0x7fe2200a4060 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.390+0000 7fe236fff700 1 --2- 172.21.15.119:0/3196234741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2301030d0 0x7fe2301a0c00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.390+0000 7fe236fff700 1 --2- 172.21.15.119:0/3196234741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe230101540 0x7fe2301996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.390+0000 7fe236fff700 1 --2- 172.21.15.119:0/3196234741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe230102730 0x7fe230199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.390+0000 7fe236fff700 1 -- 172.21.15.119:0/3196234741 >> 172.21.15.119:0/3196234741 conn(0x7fe2300fcab0 msgr2=0x7fe230107f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:30.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.391+0000 7fe236fff700 1 -- 172.21.15.119:0/3196234741 shutdown_connections 2023-12-07T10:19:30.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.391+0000 7fe236fff700 1 -- 172.21.15.119:0/3196234741 wait complete. 2023-12-07T10:19:30.410 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-07T10:19:30.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.606+0000 7f89c3dff700 1 -- 172.21.15.119:0/3966499972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89bc101540 msgr2=0x7f89bc101960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.606+0000 7f89c3dff700 1 --2- 172.21.15.119:0/3966499972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89bc101540 0x7f89bc101960 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f89b80099c0 tx=0x7f89b8009cd0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.606+0000 7f89c3dff700 1 -- 172.21.15.119:0/3966499972 shutdown_connections 2023-12-07T10:19:30.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.606+0000 7f89c3dff700 1 --2- 172.21.15.119:0/3966499972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89bc1030d0 0x7f89bc109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.606+0000 7f89c3dff700 1 --2- 172.21.15.119:0/3966499972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89bc101540 0x7f89bc101960 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.606+0000 7f89c3dff700 1 --2- 172.21.15.119:0/3966499972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89bc102730 0x7f89bc102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.606+0000 7f89c3dff700 1 -- 172.21.15.119:0/3966499972 >> 172.21.15.119:0/3966499972 conn(0x7f89bc0fcab0 msgr2=0x7f89bc0feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:30.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.606+0000 7f89c3dff700 1 -- 172.21.15.119:0/3966499972 shutdown_connections 2023-12-07T10:19:30.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.606+0000 7f89c3dff700 1 -- 172.21.15.119:0/3966499972 wait complete. 2023-12-07T10:19:30.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.607+0000 7f89c3dff700 1 Processor -- start 2023-12-07T10:19:30.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.608+0000 7f89c3dff700 1 -- start start 2023-12-07T10:19:30.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.608+0000 7f89c3dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89bc101540 0x7f89bc075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.608+0000 7f89c3dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89bc102730 0x7f89bc073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.608+0000 7f89c3dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89bc1030d0 0x7f89bc073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.608+0000 7f89c3dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc0746e0 con 0x7f89bc101540 2023-12-07T10:19:30.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.608+0000 7f89c3dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc074850 con 0x7f89bc1030d0 2023-12-07T10:19:30.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.608+0000 7f89c3dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc1a1530 con 0x7f89bc102730 2023-12-07T10:19:30.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.608+0000 7f89c1b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89bc101540 0x7f89bc075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.608+0000 7f89c239c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89bc1030d0 0x7f89bc073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.608+0000 7f89c1b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89bc101540 0x7f89bc075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52196/0 (socket says 172.21.15.119:52196) 2023-12-07T10:19:30.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.608+0000 7f89c1b9b700 1 -- 172.21.15.119:0/2710954173 learned_addr learned my addr 172.21.15.119:0/2710954173 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:30.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.608+0000 7f89c139a700 1 --2- 172.21.15.119:0/2710954173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89bc102730 0x7f89bc073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.609+0000 7f89c1b9b700 1 -- 172.21.15.119:0/2710954173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89bc102730 msgr2=0x7f89bc073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.609+0000 7f89c1b9b700 1 --2- 172.21.15.119:0/2710954173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89bc102730 0x7f89bc073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.609+0000 7f89c1b9b700 1 -- 172.21.15.119:0/2710954173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89bc1030d0 msgr2=0x7f89bc073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.609+0000 7f89c1b9b700 1 --2- 172.21.15.119:0/2710954173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89bc1030d0 0x7f89bc073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.609+0000 7f89c1b9b700 1 -- 172.21.15.119:0/2710954173 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b80095f0 con 0x7f89bc101540 2023-12-07T10:19:30.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.609+0000 7f89c139a700 1 --2- 172.21.15.119:0/2710954173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89bc102730 0x7f89bc073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:30.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.609+0000 7f89c239c700 1 --2- 172.21.15.119:0/2710954173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89bc1030d0 0x7f89bc073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:30.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.609+0000 7f89c1b9b700 1 --2- 172.21.15.119:0/2710954173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89bc101540 0x7f89bc075370 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f89b8006010 tx=0x7f89b800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:30.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.609+0000 7f89b2ffd700 1 -- 172.21.15.119:0/2710954173 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b801d070 con 0x7f89bc101540 2023-12-07T10:19:30.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.609+0000 7f89b2ffd700 1 -- 172.21.15.119:0/2710954173 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89b8005020 con 0x7f89bc101540 2023-12-07T10:19:30.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.609+0000 7f89c3dff700 1 -- 172.21.15.119:0/2710954173 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89bc1a1760 con 0x7f89bc101540 2023-12-07T10:19:30.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.609+0000 7f89b2ffd700 1 -- 172.21.15.119:0/2710954173 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b8004b10 con 0x7f89bc101540 2023-12-07T10:19:30.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.609+0000 7f89c3dff700 1 -- 172.21.15.119:0/2710954173 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89bc1a1b20 con 0x7f89bc101540 2023-12-07T10:19:30.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.612+0000 7f89b2ffd700 1 -- 172.21.15.119:0/2710954173 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 136535+0+0 (secure 0 0 0) 0x7f89b8005190 con 0x7f89bc101540 2023-12-07T10:19:30.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.613+0000 7f89d77ccb80 1 Processor -- start 2023-12-07T10:19:30.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.614+0000 7f89d77ccb80 1 -- start start 2023-12-07T10:19:30.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.614+0000 7f89b2ffd700 1 --2- 172.21.15.119:0/2710954173 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f89a80a1c20 0x7f89a80a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.614+0000 7f89b2ffd700 1 -- 172.21.15.119:0/2710954173 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f89b80ce450 con 0x7f89bc101540 2023-12-07T10:19:30.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.614+0000 7f89d77ccb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cca9e6130 0x562cca9e6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.614+0000 7f89d77ccb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cca9e6a90 0x562cca9e73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.614+0000 7f89d77ccb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cca9e7980 0x562cca9e7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.614+0000 7f89d77ccb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cca95a770 con 0x562cca9e7980 2023-12-07T10:19:30.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.614+0000 7f89d77ccb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cca941750 con 0x562cca9e6130 2023-12-07T10:19:30.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.614+0000 7f89d77ccb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cca93f070 con 0x562cca9e6a90 2023-12-07T10:19:30.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.614+0000 7f89c239c700 1 --2- 172.21.15.119:0/2710954173 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f89a80a1c20 0x7f89a80a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.614+0000 7f89c1b9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cca9e6130 0x562cca9e6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.614+0000 7f89c1b9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cca9e6130 0x562cca9e6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:41254/0 (socket says 172.21.15.119:41254) 2023-12-07T10:19:30.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.614+0000 7f89c1b9b700 1 -- 172.21.15.119:0/133263906 learned_addr learned my addr 172.21.15.119:0/133263906 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:30.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.614+0000 7f89c239c700 1 --2- 172.21.15.119:0/133263906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cca9e7980 0x562cca9e7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.614+0000 7f89c139a700 1 --2- 172.21.15.119:0/133263906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cca9e6a90 0x562cca9e73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.615+0000 7f89c1b9b700 1 -- 172.21.15.119:0/133263906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cca9e6a90 msgr2=0x562cca9e73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:30.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.615+0000 7f89c1b9b700 1 --2- 172.21.15.119:0/133263906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cca9e6a90 0x562cca9e73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.615+0000 7f89c1b9b700 1 -- 172.21.15.119:0/133263906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cca9e7980 msgr2=0x562cca9e7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:30.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.615+0000 7f89c1b9b700 1 --2- 172.21.15.119:0/133263906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cca9e7980 0x562cca9e7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.615+0000 7f89c1b9b700 1 -- 172.21.15.119:0/133263906 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cca7b75e0 con 0x562cca9e6130 2023-12-07T10:19:30.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.615+0000 7f89c1b9b700 1 --2- 172.21.15.119:0/133263906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cca9e6130 0x562cca9e6550 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f89b809adf0 tx=0x7f89b8097560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb3d21ec080af32e server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:30.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.615+0000 7f89c239c700 1 --2- 172.21.15.119:0/2710954173 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f89a80a1c20 0x7f89a80a40e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f89b4009fd0 tx=0x7f89b4009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:30.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.615+0000 7f89a7fff700 1 -- 172.21.15.119:0/133263906 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b809f240 con 0x562cca9e6130 2023-12-07T10:19:30.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.615+0000 7f89a7fff700 1 -- 172.21.15.119:0/133263906 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89b80a33f0 con 0x562cca9e6130 2023-12-07T10:19:30.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.615+0000 7f89a7fff700 1 -- 172.21.15.119:0/133263906 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b80ce2d0 con 0x562cca9e6130 2023-12-07T10:19:30.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.616+0000 7f89d77ccb80 1 -- 172.21.15.119:0/133263906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cca9e6130 msgr2=0x562cca9e6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.616+0000 7f89d77ccb80 1 --2- 172.21.15.119:0/133263906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cca9e6130 0x562cca9e6550 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f89b809adf0 tx=0x7f89b8097560 comp rx=0 tx=0).stop 2023-12-07T10:19:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.616+0000 7f89d77ccb80 1 -- 172.21.15.119:0/133263906 shutdown_connections 2023-12-07T10:19:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.616+0000 7f89d77ccb80 1 --2- 172.21.15.119:0/133263906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cca9e7980 0x562cca9e7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.616+0000 7f89d77ccb80 1 --2- 172.21.15.119:0/133263906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cca9e6130 0x562cca9e6550 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.616+0000 7f89d77ccb80 1 --2- 172.21.15.119:0/133263906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cca9e6a90 0x562cca9e73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.616+0000 7f89d77ccb80 1 -- 172.21.15.119:0/133263906 >> 172.21.15.119:0/133263906 conn(0x562cca915310 msgr2=0x562cca9e2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.616+0000 7f89d77ccb80 1 -- 172.21.15.119:0/133263906 shutdown_connections 2023-12-07T10:19:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.616+0000 7f89d77ccb80 1 -- 172.21.15.119:0/133263906 wait complete. 2023-12-07T10:19:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.617+0000 7f89d77ccb80 1 -- start start 2023-12-07T10:19:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.617+0000 7f89d77ccb80 1 Processor -- start 2023-12-07T10:19:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.617+0000 7f89d77ccb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cca9e6130 0x562cca9efc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.617+0000 7f89d77ccb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cca9e6a90 0x562cca9f0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.617+0000 7f89d77ccb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cca9e7980 0x562cca9f71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.617+0000 7f89d77ccb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cca93a960 con 0x562cca9e6a90 2023-12-07T10:19:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.617+0000 7f89d77ccb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cca8f4510 con 0x562cca9e6130 2023-12-07T10:19:30.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.617+0000 7f89d77ccb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cca8f41b0 con 0x562cca9e7980 2023-12-07T10:19:30.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.617+0000 7f89d77ccb80 10 client.0 _ll_register_callbacks cb 0x562cca8db8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:19:30.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.618+0000 7f89c139a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cca9e6a90 0x562cca9f0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.618+0000 7f89c1b9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cca9e6130 0x562cca9efc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.618+0000 7f89c139a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cca9e6a90 0x562cca9f0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52214/0 (socket says 172.21.15.119:52214) 2023-12-07T10:19:30.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.618+0000 7f89c139a700 1 -- 172.21.15.119:0/4025443715 learned_addr learned my addr 172.21.15.119:0/4025443715 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:30.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.618+0000 7f89c239c700 1 --2- 172.21.15.119:0/4025443715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cca9e7980 0x562cca9f71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.618+0000 7f89c139a700 1 -- 172.21.15.119:0/4025443715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cca9e7980 msgr2=0x562cca9f71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:30.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.618+0000 7f89c139a700 1 --2- 172.21.15.119:0/4025443715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cca9e7980 0x562cca9f71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.618+0000 7f89c139a700 1 -- 172.21.15.119:0/4025443715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cca9e6130 msgr2=0x562cca9efc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:30.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.618+0000 7f89c139a700 1 --2- 172.21.15.119:0/4025443715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cca9e6130 0x562cca9efc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.618+0000 7f89c139a700 1 -- 172.21.15.119:0/4025443715 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89bc1a1b20 con 0x562cca9e6a90 2023-12-07T10:19:30.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.618+0000 7f89c239c700 1 --2- 172.21.15.119:0/4025443715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cca9e7980 0x562cca9f71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:19:30.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.618+0000 7f89c1b9b700 1 --2- 172.21.15.119:0/4025443715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cca9e6130 0x562cca9efc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.618+0000 7f89c139a700 1 --2- 172.21.15.119:0/4025443715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cca9e6a90 0x562cca9f0180 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f89ac011d10 tx=0x7f89ac00f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6be7242c7a9b10 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.618+0000 7f89a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:19:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.618+0000 7f89a57fa700 1 -- 172.21.15.119:0/4025443715 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac016070 con 0x562cca9e6a90 2023-12-07T10:19:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.619+0000 7f89d77ccb80 1 -- 172.21.15.119:0/4025443715 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562cca9d9ab0 con 0x562cca9e6a90 2023-12-07T10:19:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.619+0000 7f89a57fa700 1 -- 172.21.15.119:0/4025443715 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f89ac004050 con 0x562cca9e6a90 2023-12-07T10:19:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.619+0000 7f89a57fa700 1 -- 172.21.15.119:0/4025443715 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89ac004210 con 0x562cca9e6a90 2023-12-07T10:19:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.619+0000 7f89d77ccb80 10 client.15204 fetch_fsmap learned FSMap version 29 2023-12-07T10:19:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.619+0000 7f89d77ccb80 1 -- 172.21.15.119:0/4025443715 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562cca763d90 con 0x562cca9e6a90 2023-12-07T10:19:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.619+0000 7f89a57fa700 1 -- 172.21.15.119:0/4025443715 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562cca9d9ab0 con 0x562cca9e6a90 2023-12-07T10:19:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.619+0000 7f89a57fa700 1 -- 172.21.15.119:0/4025443715 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f89ac019370 con 0x562cca9e6a90 2023-12-07T10:19:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.619+0000 7f89d77ccb80 10 client.15204 fetch_fsmap finished waiting for FSMap version 29 2023-12-07T10:19:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.619+0000 7f89d77ccb80 10 client.15204 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-07T10:19:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.619+0000 7f89d77ccb80 20 client.15204 populate_metadata read hostname 'smithi119' 2023-12-07T10:19:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.619+0000 7f89d77ccb80 1 --2- 172.21.15.119:0/4025443715 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562cca9fb910 0x562cca9fa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.619+0000 7f89d77ccb80 4 client.15204 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:19:30.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.619+0000 7f89d77ccb80 1 -- 172.21.15.119:0/4025443715 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562cca9b5750 con 0x562cca9fb910 2023-12-07T10:19:30.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.620+0000 7f89c1b9b700 1 --2- 172.21.15.119:0/4025443715 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562cca9fb910 0x562cca9fa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.620+0000 7f89c1b9b700 1 --2- 172.21.15.119:0/4025443715 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562cca9fb910 0x562cca9fa3c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99463c2f3f1c1cff server_cookie=14d490594b4fdd64 in_seq=0 out_seq=0 2023-12-07T10:19:30.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.620+0000 7f89a57fa700 10 client.15204 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:19:30.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.621+0000 7f89a57fa700 1 -- 172.21.15.119:0/4025443715 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562cca9b5750 con 0x562cca9fb910 2023-12-07T10:19:30.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.621+0000 7f89a57fa700 10 client.15204 handle_command_reply: tid=0 2023-12-07T10:19:30.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.621+0000 7f89d77ccb80 1 client.15204 shutdown 2023-12-07T10:19:30.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.621+0000 7f89d77ccb80 1 -- 172.21.15.119:0/4025443715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cca9e6a90 msgr2=0x562cca9f0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.621+0000 7f89d77ccb80 1 --2- 172.21.15.119:0/4025443715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cca9e6a90 0x562cca9f0180 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f89ac011d10 tx=0x7f89ac00f6a0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.621+0000 7f89d77ccb80 1 -- 172.21.15.119:0/4025443715 shutdown_connections 2023-12-07T10:19:30.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.621+0000 7f89d77ccb80 1 --2- 172.21.15.119:0/4025443715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cca9e7980 0x562cca9f71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.621+0000 7f89d77ccb80 1 --2- 172.21.15.119:0/4025443715 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562cca9fb910 0x562cca9fa3c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.621+0000 7f89d77ccb80 1 --2- 172.21.15.119:0/4025443715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cca9e6130 0x562cca9efc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.621+0000 7f89d77ccb80 1 --2- 172.21.15.119:0/4025443715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cca9e6a90 0x562cca9f0180 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.621+0000 7f89a57fa700 0 client.15204 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:19:30.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.621+0000 7f89d77ccb80 1 -- 172.21.15.119:0/4025443715 >> 172.21.15.119:0/4025443715 conn(0x562cca915310 msgr2=0x562cca9e1990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:30.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.621+0000 7f89d77ccb80 1 -- 172.21.15.119:0/4025443715 shutdown_connections 2023-12-07T10:19:30.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.621+0000 7f89d77ccb80 1 -- 172.21.15.119:0/4025443715 wait complete. 2023-12-07T10:19:30.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.622+0000 7f89d77ccb80 20 client.15204 trim_cache size 0 max 16384 2023-12-07T10:19:30.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.640+0000 7f89d77ccb80 1 Processor -- start 2023-12-07T10:19:30.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.640+0000 7f89d77ccb80 1 -- start start 2023-12-07T10:19:30.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.640+0000 7f89d77ccb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cca9e6130 0x562cca9e7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.640+0000 7f89d77ccb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cca9e6a90 0x562cca9f7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.640+0000 7f89d77ccb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cca9e7980 0x562cca9fa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.640+0000 7f89d77ccb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cca7b98d0 con 0x562cca9e6a90 2023-12-07T10:19:30.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.640+0000 7f89d77ccb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cca936a80 con 0x562cca9e6130 2023-12-07T10:19:30.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.640+0000 7f89d77ccb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cca8dd830 con 0x562cca9e7980 2023-12-07T10:19:30.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.641+0000 7f89c139a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cca9e6a90 0x562cca9f7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.641+0000 7f89c139a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cca9e6a90 0x562cca9f7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52230/0 (socket says 172.21.15.119:52230) 2023-12-07T10:19:30.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.641+0000 7f89c139a700 1 -- 172.21.15.119:0/2440811786 learned_addr learned my addr 172.21.15.119:0/2440811786 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:30.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.641+0000 7f89c1b9b700 1 --2- 172.21.15.119:0/2440811786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cca9e6130 0x562cca9e7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.641+0000 7f89c239c700 1 --2- 172.21.15.119:0/2440811786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cca9e7980 0x562cca9fa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.641+0000 7f89c139a700 1 -- 172.21.15.119:0/2440811786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cca9e7980 msgr2=0x562cca9fa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:30.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.641+0000 7f89c139a700 1 --2- 172.21.15.119:0/2440811786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cca9e7980 0x562cca9fa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.641+0000 7f89c139a700 1 -- 172.21.15.119:0/2440811786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cca9e6130 msgr2=0x562cca9e7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:30.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.641+0000 7f89c139a700 1 --2- 172.21.15.119:0/2440811786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cca9e6130 0x562cca9e7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.641+0000 7f89c139a700 1 -- 172.21.15.119:0/2440811786 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cca74cc10 con 0x562cca9e6a90 2023-12-07T10:19:30.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.641+0000 7f89c239c700 1 --2- 172.21.15.119:0/2440811786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cca9e7980 0x562cca9fa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:19:30.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.641+0000 7f89c139a700 1 --2- 172.21.15.119:0/2440811786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cca9e6a90 0x562cca9f7180 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009eb0 tx=0x7f89ac009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caed3cd48eb7b391 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:30.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.641+0000 7f89b0bff700 1 -- 172.21.15.119:0/2440811786 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac016070 con 0x562cca9e6a90 2023-12-07T10:19:30.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.641+0000 7f89b0bff700 1 -- 172.21.15.119:0/2440811786 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89ac019430 con 0x562cca9e6a90 2023-12-07T10:19:30.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.641+0000 7f89b0bff700 1 -- 172.21.15.119:0/2440811786 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac003680 con 0x562cca9e6a90 2023-12-07T10:19:30.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.642+0000 7f89d77ccb80 1 -- 172.21.15.119:0/2440811786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cca9e6a90 msgr2=0x562cca9f7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.642+0000 7f89d77ccb80 1 --2- 172.21.15.119:0/2440811786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cca9e6a90 0x562cca9f7180 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009eb0 tx=0x7f89ac009e20 comp rx=0 tx=0).stop 2023-12-07T10:19:30.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.642+0000 7f89d77ccb80 1 -- 172.21.15.119:0/2440811786 shutdown_connections 2023-12-07T10:19:30.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.642+0000 7f89d77ccb80 1 --2- 172.21.15.119:0/2440811786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cca9e7980 0x562cca9fa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.642+0000 7f89d77ccb80 1 --2- 172.21.15.119:0/2440811786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cca9e6130 0x562cca9e7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.642+0000 7f89d77ccb80 1 --2- 172.21.15.119:0/2440811786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cca9e6a90 0x562cca9f7180 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.642+0000 7f89d77ccb80 1 -- 172.21.15.119:0/2440811786 >> 172.21.15.119:0/2440811786 conn(0x562cca915310 msgr2=0x562cca9e2570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:30.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.642+0000 7f89d77ccb80 1 -- 172.21.15.119:0/2440811786 shutdown_connections 2023-12-07T10:19:30.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.642+0000 7f89d77ccb80 1 -- 172.21.15.119:0/2440811786 wait complete. 2023-12-07T10:19:30.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.642+0000 7f89d77ccb80 1 -- start start 2023-12-07T10:19:30.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.643+0000 7f89d77ccb80 1 Processor -- start 2023-12-07T10:19:30.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.643+0000 7f89d77ccb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cca9e6130 0x562cca9f7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.643+0000 7f89d77ccb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cca9e6a90 0x562cca9f3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.643+0000 7f89d77ccb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cca9e7980 0x562cca9f4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.643+0000 7f89d77ccb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ccaa51a00 con 0x562cca9e6130 2023-12-07T10:19:30.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.643+0000 7f89d77ccb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ccaa51b40 con 0x562cca9e6a90 2023-12-07T10:19:30.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.643+0000 7f89d77ccb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ccaa51cd0 con 0x562cca9e7980 2023-12-07T10:19:30.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.643+0000 7f89d77ccb80 10 client.0 _ll_register_callbacks cb 0x562cca8db8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:19:30.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.643+0000 7f89c139a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cca9e6a90 0x562cca9f3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.643+0000 7f89c1b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cca9e6130 0x562cca9f7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.643+0000 7f89c139a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cca9e6a90 0x562cca9f3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:41272/0 (socket says 172.21.15.119:41272) 2023-12-07T10:19:30.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.643+0000 7f89c139a700 1 -- 172.21.15.119:0/906047567 learned_addr learned my addr 172.21.15.119:0/906047567 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:30.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.644+0000 7f89c239c700 1 --2- 172.21.15.119:0/906047567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cca9e7980 0x562cca9f4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.644+0000 7f89c139a700 1 -- 172.21.15.119:0/906047567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cca9e7980 msgr2=0x562cca9f4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:30.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.644+0000 7f89c139a700 1 --2- 172.21.15.119:0/906047567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cca9e7980 0x562cca9f4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.644+0000 7f89c139a700 1 -- 172.21.15.119:0/906047567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cca9e6130 msgr2=0x562cca9f7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:30.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.644+0000 7f89c139a700 1 --2- 172.21.15.119:0/906047567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cca9e6130 0x562cca9f7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.644+0000 7f89c139a700 1 -- 172.21.15.119:0/906047567 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562cca763d90 con 0x562cca9e6a90 2023-12-07T10:19:30.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.644+0000 7f89c1b9b700 1 --2- 172.21.15.119:0/906047567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cca9e6130 0x562cca9f7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:30.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.644+0000 7f89c139a700 1 --2- 172.21.15.119:0/906047567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cca9e6a90 0x562cca9f3cf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f89ac014020 tx=0x7f89ac019750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7d80670afc0796f server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:30.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.644+0000 7f89a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:19:30.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.644+0000 7f89a57fa700 1 -- 172.21.15.119:0/906047567 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac016040 con 0x562cca9e6a90 2023-12-07T10:19:30.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.644+0000 7f89d77ccb80 1 -- 172.21.15.119:0/906047567 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89b80a33f0 con 0x562cca9e6a90 2023-12-07T10:19:30.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.644+0000 7f89a57fa700 1 -- 172.21.15.119:0/906047567 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f89ac000e40 con 0x562cca9e6a90 2023-12-07T10:19:30.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.644+0000 7f89a57fa700 1 -- 172.21.15.119:0/906047567 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89ac019e20 con 0x562cca9e6a90 2023-12-07T10:19:30.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.644+0000 7f89d77ccb80 10 client.25235 fetch_fsmap learned FSMap version 29 2023-12-07T10:19:30.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.645+0000 7f89d77ccb80 1 -- 172.21.15.119:0/906047567 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562cca78a200 con 0x562cca9e6a90 2023-12-07T10:19:30.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.645+0000 7f89a57fa700 1 -- 172.21.15.119:0/906047567 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89b80a33f0 con 0x562cca9e6a90 2023-12-07T10:19:30.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.645+0000 7f89a57fa700 1 -- 172.21.15.119:0/906047567 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f89ac022040 con 0x562cca9e6a90 2023-12-07T10:19:30.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.645+0000 7f89d77ccb80 10 client.25235 fetch_fsmap finished waiting for FSMap version 29 2023-12-07T10:19:30.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.645+0000 7f89d77ccb80 10 client.25235 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-07T10:19:30.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.645+0000 7f89d77ccb80 20 client.25235 populate_metadata read hostname 'smithi119' 2023-12-07T10:19:30.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.645+0000 7f89d77ccb80 1 --2- 172.21.15.119:0/906047567 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562ccaa56eb0 0x562ccaa53300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:30.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.645+0000 7f89d77ccb80 4 client.25235 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-07T10:19:30.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.645+0000 7f89d77ccb80 1 -- 172.21.15.119:0/906047567 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562cca900f80 con 0x562ccaa56eb0 2023-12-07T10:19:30.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.645+0000 7f89c1b9b700 1 --2- 172.21.15.119:0/906047567 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562ccaa56eb0 0x562ccaa53300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:30.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.646+0000 7f89c1b9b700 1 --2- 172.21.15.119:0/906047567 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562ccaa56eb0 0x562ccaa53300 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0f682b2cc272a0c server_cookie=793e249158aecb2b in_seq=0 out_seq=0 2023-12-07T10:19:30.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.646+0000 7f89a57fa700 10 client.25235 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:19:30.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.646+0000 7f89a57fa700 1 -- 172.21.15.119:0/906047567 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x562cca900f80 con 0x562ccaa56eb0 2023-12-07T10:19:30.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.646+0000 7f89a57fa700 10 client.25235 handle_command_reply: tid=0 2023-12-07T10:19:30.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.646+0000 7f89d77ccb80 1 client.25235 shutdown 2023-12-07T10:19:30.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.646+0000 7f89d77ccb80 1 -- 172.21.15.119:0/906047567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cca9e6a90 msgr2=0x562cca9f3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.646+0000 7f89d77ccb80 1 --2- 172.21.15.119:0/906047567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cca9e6a90 0x562cca9f3cf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f89ac014020 tx=0x7f89ac019750 comp rx=0 tx=0).stop 2023-12-07T10:19:30.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.646+0000 7f89d77ccb80 1 -- 172.21.15.119:0/906047567 shutdown_connections 2023-12-07T10:19:30.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.646+0000 7f89d77ccb80 1 --2- 172.21.15.119:0/906047567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cca9e7980 0x562cca9f4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.646+0000 7f89d77ccb80 1 --2- 172.21.15.119:0/906047567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cca9e6130 0x562cca9f7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.646+0000 7f89d77ccb80 1 --2- 172.21.15.119:0/906047567 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562ccaa56eb0 0x562ccaa53300 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.646+0000 7f89d77ccb80 1 --2- 172.21.15.119:0/906047567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cca9e6a90 0x562cca9f3cf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.646+0000 7f89a57fa700 0 client.25235 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:19:30.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.646+0000 7f89d77ccb80 1 -- 172.21.15.119:0/906047567 >> 172.21.15.119:0/906047567 conn(0x562cca915310 msgr2=0x562cca9ef9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:30.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.646+0000 7f89d77ccb80 1 -- 172.21.15.119:0/906047567 shutdown_connections 2023-12-07T10:19:30.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.646+0000 7f89d77ccb80 1 -- 172.21.15.119:0/906047567 wait complete. 2023-12-07T10:19:30.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.647+0000 7f89d77ccb80 20 client.25235 trim_cache size 0 max 16384 2023-12-07T10:19:30.654 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:19:30.654 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "a4c642c6-94e8-11ee-95a2-87774f69a715", 2023-12-07T10:19:30.654 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-12-07T10:19:30.654 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 24566, 2023-12-07T10:19:30.654 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-12-07T10:19:30.654 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-12-07T10:19:30.654 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-12-07T10:19:30.654 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 108.895298634, 2023-12-07T10:19:30.654 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 29, 2023-12-07T10:19:30.655 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 93, 2023-12-07T10:19:30.655 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 91, 2023-12-07T10:19:30.655 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 117.107815559 2023-12-07T10:19:30.655 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:19:30.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.647+0000 7f89c3dff700 1 -- 172.21.15.119:0/2710954173 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f89a80a1c20 msgr2=0x7f89a80a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.647+0000 7f89c3dff700 1 --2- 172.21.15.119:0/2710954173 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f89a80a1c20 0x7f89a80a40e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f89b4009fd0 tx=0x7f89b4009310 comp rx=0 tx=0).stop 2023-12-07T10:19:30.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.647+0000 7f89c3dff700 1 -- 172.21.15.119:0/2710954173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89bc101540 msgr2=0x7f89bc075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:30.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.647+0000 7f89c3dff700 1 --2- 172.21.15.119:0/2710954173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89bc101540 0x7f89bc075370 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f89b8006010 tx=0x7f89b800f690 comp rx=0 tx=0).stop 2023-12-07T10:19:30.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.647+0000 7f89c3dff700 1 -- 172.21.15.119:0/2710954173 shutdown_connections 2023-12-07T10:19:30.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.647+0000 7f89c3dff700 1 --2- 172.21.15.119:0/2710954173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f89bc1030d0 0x7f89bc073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.647+0000 7f89c3dff700 1 --2- 172.21.15.119:0/2710954173 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f89a80a1c20 0x7f89a80a40e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.647+0000 7f89c3dff700 1 --2- 172.21.15.119:0/2710954173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89bc101540 0x7f89bc075370 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.647+0000 7f89c3dff700 1 --2- 172.21.15.119:0/2710954173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89bc102730 0x7f89bc073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:30.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.647+0000 7f89c3dff700 1 -- 172.21.15.119:0/2710954173 >> 172.21.15.119:0/2710954173 conn(0x7f89bc0fcab0 msgr2=0x7f89bc105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:30.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.647+0000 7f89c3dff700 1 -- 172.21.15.119:0/2710954173 shutdown_connections 2023-12-07T10:19:30.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:30.647+0000 7f89c3dff700 1 -- 172.21.15.119:0/2710954173 wait complete. 2023-12-07T10:19:30.668 INFO:tasks.cephadm:Setup complete, yielding 2023-12-07T10:19:30.668 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-12-07T10:19:30.678 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi119.front.sepia.ceph.com 2023-12-07T10:19:30.678 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- bash -c 'ceph orch status' 2023-12-07T10:19:31.479 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:19:32.093 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:31 smithi119 ceph-mon[97908]: pgmap v338: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-07T10:19:32.093 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:31 smithi119 ceph-mon[97908]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-12-07T10:19:32.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:31 smithi159 ceph-mon[134726]: pgmap v338: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-07T10:19:32.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:31 smithi159 ceph-mon[134726]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-12-07T10:19:32.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:31 smithi153 ceph-mon[133625]: pgmap v338: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-07T10:19:32.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:31 smithi153 ceph-mon[133625]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-12-07T10:19:32.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.996+0000 7f1c1248c700 1 -- 172.21.15.119:0/582889338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0c103200 msgr2=0x7f1c0c109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:32.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.996+0000 7f1c1248c700 1 --2- 172.21.15.119:0/582889338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0c103200 0x7f1c0c109ab0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1c0400b0d0 tx=0x7f1c0400b3e0 comp rx=0 tx=0).stop 2023-12-07T10:19:32.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.996+0000 7f1c1248c700 1 -- 172.21.15.119:0/582889338 shutdown_connections 2023-12-07T10:19:32.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.996+0000 7f1c1248c700 1 --2- 172.21.15.119:0/582889338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0c103200 0x7f1c0c109ab0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:32.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.996+0000 7f1c1248c700 1 --2- 172.21.15.119:0/582889338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c0c101660 0x7f1c0c101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:32.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.996+0000 7f1c1248c700 1 --2- 172.21.15.119:0/582889338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c0c102860 0x7f1c0c102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:32.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.996+0000 7f1c1248c700 1 -- 172.21.15.119:0/582889338 >> 172.21.15.119:0/582889338 conn(0x7f1c0c0fcc20 msgr2=0x7f1c0c0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:32.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.996+0000 7f1c1248c700 1 -- 172.21.15.119:0/582889338 shutdown_connections 2023-12-07T10:19:32.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.996+0000 7f1c1248c700 1 -- 172.21.15.119:0/582889338 wait complete. 2023-12-07T10:19:33.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.997+0000 7f1c1248c700 1 Processor -- start 2023-12-07T10:19:33.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.998+0000 7f1c1248c700 1 -- start start 2023-12-07T10:19:33.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.998+0000 7f1c1248c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c0c101660 0x7f1c0c199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:33.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.998+0000 7f1c1248c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0c102860 0x7f1c0c199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:33.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.998+0000 7f1c1248c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c0c103200 0x7f1c0c1a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:33.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.998+0000 7f1c1248c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c1a13b0 con 0x7f1c0c102860 2023-12-07T10:19:33.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.998+0000 7f1c1248c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c1a14f0 con 0x7f1c0c103200 2023-12-07T10:19:33.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.998+0000 7f1c1248c700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c1a1680 con 0x7f1c0c101660 2023-12-07T10:19:33.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.998+0000 7f1c0b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0c102860 0x7f1c0c199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:33.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.998+0000 7f1c0bfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c0c101660 0x7f1c0c199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:33.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.998+0000 7f1c0b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0c102860 0x7f1c0c199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60632/0 (socket says 172.21.15.119:60632) 2023-12-07T10:19:33.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.998+0000 7f1c0b7fe700 1 -- 172.21.15.119:0/1095018443 learned_addr learned my addr 172.21.15.119:0/1095018443 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:33.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.998+0000 7f1c10a29700 1 --2- 172.21.15.119:0/1095018443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c0c103200 0x7f1c0c1a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:33.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.999+0000 7f1c0b7fe700 1 -- 172.21.15.119:0/1095018443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c0c101660 msgr2=0x7f1c0c199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:33.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.999+0000 7f1c0b7fe700 1 --2- 172.21.15.119:0/1095018443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c0c101660 0x7f1c0c199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:33.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.999+0000 7f1c0b7fe700 1 -- 172.21.15.119:0/1095018443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c0c103200 msgr2=0x7f1c0c1a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:33.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.999+0000 7f1c0b7fe700 1 --2- 172.21.15.119:0/1095018443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c0c103200 0x7f1c0c1a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:33.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.999+0000 7f1c0b7fe700 1 -- 172.21.15.119:0/1095018443 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c04009d00 con 0x7f1c0c102860 2023-12-07T10:19:33.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.999+0000 7f1c0bfff700 1 --2- 172.21.15.119:0/1095018443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c0c101660 0x7f1c0c199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:33.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.999+0000 7f1c10a29700 1 --2- 172.21.15.119:0/1095018443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c0c103200 0x7f1c0c1a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:33.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.999+0000 7f1c0b7fe700 1 --2- 172.21.15.119:0/1095018443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0c102860 0x7f1c0c199db0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000e9d0 tx=0x7f1c0000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:33.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.999+0000 7f1c097fa700 1 -- 172.21.15.119:0/1095018443 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00009790 con 0x7f1c0c102860 2023-12-07T10:19:33.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.999+0000 7f1c097fa700 1 -- 172.21.15.119:0/1095018443 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c0000ee70 con 0x7f1c0c102860 2023-12-07T10:19:33.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.999+0000 7f1c1248c700 1 -- 172.21.15.119:0/1095018443 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c0c1a1960 con 0x7f1c0c102860 2023-12-07T10:19:33.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.999+0000 7f1c097fa700 1 -- 172.21.15.119:0/1095018443 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00010ea0 con 0x7f1c0c102860 2023-12-07T10:19:33.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:32.999+0000 7f1c1248c700 1 -- 172.21.15.119:0/1095018443 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c0c1a1e60 con 0x7f1c0c102860 2023-12-07T10:19:33.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:33.000+0000 7f1c1248c700 1 -- 172.21.15.119:0/1095018443 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c0c04f2d0 con 0x7f1c0c102860 2023-12-07T10:19:33.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:33.002+0000 7f1c097fa700 1 -- 172.21.15.119:0/1095018443 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f1c0000cbd0 con 0x7f1c0c102860 2023-12-07T10:19:33.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:33.003+0000 7f1c097fa700 1 --2- 172.21.15.119:0/1095018443 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1bf40a1b10 0x7f1bf40a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:33.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:33.004+0000 7f1c097fa700 1 -- 172.21.15.119:0/1095018443 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f1c00014070 con 0x7f1c0c102860 2023-12-07T10:19:33.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:33.004+0000 7f1c0bfff700 1 --2- 172.21.15.119:0/1095018443 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1bf40a1b10 0x7f1bf40a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:33.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:33.004+0000 7f1c0bfff700 1 --2- 172.21.15.119:0/1095018443 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1bf40a1b10 0x7f1bf40a3fd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c1026c0 tx=0x7f1bfc007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:33.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:33.008+0000 7f1c097fa700 1 -- 172.21.15.119:0/1095018443 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1c000522f0 con 0x7f1c0c102860 2023-12-07T10:19:33.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:33.225+0000 7f1c1248c700 1 -- 172.21.15.119:0/1095018443 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c0c074f60 con 0x7f1bf40a1b10 2023-12-07T10:19:33.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:33.226+0000 7f1c097fa700 1 -- 172.21.15.119:0/1095018443 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f1c0c074f60 con 0x7f1bf40a1b10 2023-12-07T10:19:33.228 INFO:teuthology.orchestra.run.smithi119.stdout:Backend: cephadm 2023-12-07T10:19:33.228 INFO:teuthology.orchestra.run.smithi119.stdout:Available: Yes 2023-12-07T10:19:33.228 INFO:teuthology.orchestra.run.smithi119.stdout:Paused: No 2023-12-07T10:19:33.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:33.229+0000 7f1c1248c700 1 -- 172.21.15.119:0/1095018443 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1bf40a1b10 msgr2=0x7f1bf40a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:33.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:33.229+0000 7f1c1248c700 1 --2- 172.21.15.119:0/1095018443 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1bf40a1b10 0x7f1bf40a3fd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c1026c0 tx=0x7f1bfc007400 comp rx=0 tx=0).stop 2023-12-07T10:19:33.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:33.229+0000 7f1c1248c700 1 -- 172.21.15.119:0/1095018443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0c102860 msgr2=0x7f1c0c199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:33.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:33.229+0000 7f1c1248c700 1 --2- 172.21.15.119:0/1095018443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0c102860 0x7f1c0c199db0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000e9d0 tx=0x7f1c0000ece0 comp rx=0 tx=0).stop 2023-12-07T10:19:33.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:33.229+0000 7f1c1248c700 1 -- 172.21.15.119:0/1095018443 shutdown_connections 2023-12-07T10:19:33.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:33.229+0000 7f1c1248c700 1 --2- 172.21.15.119:0/1095018443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c0c103200 0x7f1c0c1a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:33.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:33.229+0000 7f1c1248c700 1 --2- 172.21.15.119:0/1095018443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c0c101660 0x7f1c0c199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:33.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:33.229+0000 7f1c1248c700 1 --2- 172.21.15.119:0/1095018443 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1bf40a1b10 0x7f1bf40a3fd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:33.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:33.229+0000 7f1c1248c700 1 --2- 172.21.15.119:0/1095018443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0c102860 0x7f1c0c199db0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:33.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:33.229+0000 7f1c1248c700 1 -- 172.21.15.119:0/1095018443 >> 172.21.15.119:0/1095018443 conn(0x7f1c0c0fcc20 msgr2=0x7f1c0c073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:33.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:33.229+0000 7f1c1248c700 1 -- 172.21.15.119:0/1095018443 shutdown_connections 2023-12-07T10:19:33.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:33.229+0000 7f1c1248c700 1 -- 172.21.15.119:0/1095018443 wait complete. 2023-12-07T10:19:33.785 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- bash -c 'ceph orch ps' 2023-12-07T10:19:34.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:33 smithi119 ceph-mon[97908]: pgmap v339: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-12-07T10:19:34.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:33 smithi159 ceph-mon[134726]: pgmap v339: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-12-07T10:19:34.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:33 smithi153 ceph-mon[133625]: pgmap v339: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-12-07T10:19:34.563 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:19:35.109 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:34 smithi119 ceph-mon[97908]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:19:35.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:34 smithi159 ceph-mon[134726]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:19:35.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:34 smithi153 ceph-mon[133625]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:19:36.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.103+0000 7f8e6f8a4700 1 -- 172.21.15.119:0/3557228164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e68102860 msgr2=0x7f8e68102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:36.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.103+0000 7f8e6f8a4700 1 --2- 172.21.15.119:0/3557228164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e68102860 0x7f8e68102cc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8e580099c0 tx=0x7f8e58009cd0 comp rx=0 tx=0).stop 2023-12-07T10:19:36.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.103+0000 7f8e6f8a4700 1 -- 172.21.15.119:0/3557228164 shutdown_connections 2023-12-07T10:19:36.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.103+0000 7f8e6f8a4700 1 --2- 172.21.15.119:0/3557228164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e68103200 0x7f8e68109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:36.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.103+0000 7f8e6f8a4700 1 --2- 172.21.15.119:0/3557228164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e68101660 0x7f8e68101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:36.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.103+0000 7f8e6f8a4700 1 --2- 172.21.15.119:0/3557228164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e68102860 0x7f8e68102cc0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:36.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.103+0000 7f8e6f8a4700 1 -- 172.21.15.119:0/3557228164 >> 172.21.15.119:0/3557228164 conn(0x7f8e680fcc20 msgr2=0x7f8e680ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:36.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.103+0000 7f8e6f8a4700 1 -- 172.21.15.119:0/3557228164 shutdown_connections 2023-12-07T10:19:36.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.103+0000 7f8e6f8a4700 1 -- 172.21.15.119:0/3557228164 wait complete. 2023-12-07T10:19:36.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.104+0000 7f8e6f8a4700 1 Processor -- start 2023-12-07T10:19:36.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.104+0000 7f8e6f8a4700 1 -- start start 2023-12-07T10:19:36.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.104+0000 7f8e6f8a4700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e68101660 0x7f8e681997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:36.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.104+0000 7f8e6f8a4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e68102860 0x7f8e68199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:36.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.104+0000 7f8e6f8a4700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e68103200 0x7f8e681a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:36.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.104+0000 7f8e6f8a4700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e681a1300 con 0x7f8e68103200 2023-12-07T10:19:36.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.104+0000 7f8e6f8a4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e681a1440 con 0x7f8e68102860 2023-12-07T10:19:36.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.104+0000 7f8e6f8a4700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e681a15d0 con 0x7f8e68101660 2023-12-07T10:19:36.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.105+0000 7f8e6d640700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e68101660 0x7f8e681997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:36.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.105+0000 7f8e6d640700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e68101660 0x7f8e681997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:60262/0 (socket says 172.21.15.119:60262) 2023-12-07T10:19:36.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.105+0000 7f8e6d640700 1 -- 172.21.15.119:0/1436254143 learned_addr learned my addr 172.21.15.119:0/1436254143 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:36.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.105+0000 7f8e6de41700 1 --2- 172.21.15.119:0/1436254143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e68103200 0x7f8e681a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:36.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.105+0000 7f8e6ce3f700 1 --2- 172.21.15.119:0/1436254143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e68102860 0x7f8e68199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:36.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.105+0000 7f8e6d640700 1 -- 172.21.15.119:0/1436254143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e68102860 msgr2=0x7f8e68199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:36.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.105+0000 7f8e6d640700 1 --2- 172.21.15.119:0/1436254143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e68102860 0x7f8e68199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:36.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.105+0000 7f8e6d640700 1 -- 172.21.15.119:0/1436254143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e68103200 msgr2=0x7f8e681a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:36.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.105+0000 7f8e6d640700 1 --2- 172.21.15.119:0/1436254143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e68103200 0x7f8e681a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:36.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.105+0000 7f8e6d640700 1 -- 172.21.15.119:0/1436254143 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e580095f0 con 0x7f8e68101660 2023-12-07T10:19:36.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.105+0000 7f8e6d640700 1 --2- 172.21.15.119:0/1436254143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e68101660 0x7f8e681997c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8e6400e9a0 tx=0x7f8e6400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:36.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.106+0000 7f8e5e7fc700 1 -- 172.21.15.119:0/1436254143 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e64009790 con 0x7f8e68101660 2023-12-07T10:19:36.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.106+0000 7f8e6f8a4700 1 -- 172.21.15.119:0/1436254143 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e681a18b0 con 0x7f8e68101660 2023-12-07T10:19:36.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.106+0000 7f8e5e7fc700 1 -- 172.21.15.119:0/1436254143 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e6400c810 con 0x7f8e68101660 2023-12-07T10:19:36.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.106+0000 7f8e5e7fc700 1 -- 172.21.15.119:0/1436254143 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e64003c70 con 0x7f8e68101660 2023-12-07T10:19:36.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.106+0000 7f8e6f8a4700 1 -- 172.21.15.119:0/1436254143 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e681a1db0 con 0x7f8e68101660 2023-12-07T10:19:36.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.107+0000 7f8e6f8a4700 1 -- 172.21.15.119:0/1436254143 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e6804f2d0 con 0x7f8e68101660 2023-12-07T10:19:36.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.109+0000 7f8e5e7fc700 1 -- 172.21.15.119:0/1436254143 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f8e6400cc80 con 0x7f8e68101660 2023-12-07T10:19:36.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.110+0000 7f8e5e7fc700 1 --2- 172.21.15.119:0/1436254143 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8e540a1b10 0x7f8e540a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:36.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.110+0000 7f8e5e7fc700 1 -- 172.21.15.119:0/1436254143 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f8e64014070 con 0x7f8e68101660 2023-12-07T10:19:36.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.110+0000 7f8e6ce3f700 1 --2- 172.21.15.119:0/1436254143 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8e540a1b10 0x7f8e540a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:36.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.111+0000 7f8e6ce3f700 1 --2- 172.21.15.119:0/1436254143 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8e540a1b10 0x7f8e540a3fd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8e58006010 tx=0x7f8e5800b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:36.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.115+0000 7f8e5e7fc700 1 -- 172.21.15.119:0/1436254143 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8e64096b20 con 0x7f8e68101660 2023-12-07T10:19:36.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:35 smithi119 ceph-mon[97908]: pgmap v340: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-07T10:19:36.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:35 smithi159 ceph-mon[134726]: pgmap v340: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-07T10:19:36.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:35 smithi153 ceph-mon[133625]: pgmap v340: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-07T10:19:36.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.322+0000 7f8e6f8a4700 1 -- 172.21.15.119:0/1436254143 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e68109770 con 0x7f8e540a1b10 2023-12-07T10:19:36.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.336+0000 7f8e5e7fc700 1 -- 172.21.15.119:0/1436254143 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4030 (secure 0 0 0) 0x7f8e68109770 con 0x7f8e540a1b10 2023-12-07T10:19:36.338 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-12-07T10:19:36.338 INFO:teuthology.orchestra.run.smithi119.stdout:mds.a smithi119 running (2m) 38s ago 2m 16.3M - 19.0.0-18-g3bd0bfa4 29737aea1903 192ab5811644 2023-12-07T10:19:36.338 INFO:teuthology.orchestra.run.smithi119.stdout:mds.b smithi153 running (2m) 51s ago 2m 33.6M - 19.0.0-18-g3bd0bfa4 29737aea1903 828dc654c757 2023-12-07T10:19:36.338 INFO:teuthology.orchestra.run.smithi119.stdout:mds.c smithi159 running (2m) 51s ago 2m 18.9M - 19.0.0-18-g3bd0bfa4 29737aea1903 64599de510fc 2023-12-07T10:19:36.338 INFO:teuthology.orchestra.run.smithi119.stdout:mds.d smithi119 running (2m) 38s ago 2m 13.9M - 19.0.0-18-g3bd0bfa4 29737aea1903 cfc4c1daf627 2023-12-07T10:19:36.338 INFO:teuthology.orchestra.run.smithi119.stdout:mds.e smithi153 running (2m) 51s ago 2m 20.1M - 19.0.0-18-g3bd0bfa4 29737aea1903 19ed7a677328 2023-12-07T10:19:36.338 INFO:teuthology.orchestra.run.smithi119.stdout:mds.f smithi159 running (2m) 51s ago 2m 19.9M - 19.0.0-18-g3bd0bfa4 29737aea1903 6be0ddf23a18 2023-12-07T10:19:36.338 INFO:teuthology.orchestra.run.smithi119.stdout:mds.g smithi119 running (2m) 38s ago 2m 18.9M - 19.0.0-18-g3bd0bfa4 29737aea1903 e6b70bca1e3e 2023-12-07T10:19:36.338 INFO:teuthology.orchestra.run.smithi119.stdout:mds.h smithi153 running (2m) 51s ago 2m 17.0M - 19.0.0-18-g3bd0bfa4 29737aea1903 b7582912e5e5 2023-12-07T10:19:36.338 INFO:teuthology.orchestra.run.smithi119.stdout:mds.i smithi159 running (2m) 51s ago 2m 21.7M - 19.0.0-18-g3bd0bfa4 29737aea1903 bb11777dcfde 2023-12-07T10:19:36.338 INFO:teuthology.orchestra.run.smithi119.stdout:mds.j smithi119 running (2m) 38s ago 2m 13.8M - 19.0.0-18-g3bd0bfa4 29737aea1903 77c6708926be 2023-12-07T10:19:36.338 INFO:teuthology.orchestra.run.smithi119.stdout:mds.k smithi153 running (2m) 51s ago 2m 17.7M - 19.0.0-18-g3bd0bfa4 29737aea1903 b447bfee8ee4 2023-12-07T10:19:36.338 INFO:teuthology.orchestra.run.smithi119.stdout:mds.l smithi159 running (2m) 51s ago 2m 17.6M - 19.0.0-18-g3bd0bfa4 29737aea1903 19f143714d13 2023-12-07T10:19:36.339 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.x smithi119 *:9283,8765 running (9m) 38s ago 9m 531M - 19.0.0-18-g3bd0bfa4 29737aea1903 9b1f99d02597 2023-12-07T10:19:36.339 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.y smithi153 *:8443,8765 running (7m) 51s ago 7m 450M - 19.0.0-18-g3bd0bfa4 29737aea1903 78ec1f418011 2023-12-07T10:19:36.339 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.z smithi159 *:8443,8765 running (7m) 51s ago 7m 450M - 19.0.0-18-g3bd0bfa4 29737aea1903 d5c6b8816696 2023-12-07T10:19:36.339 INFO:teuthology.orchestra.run.smithi119.stdout:mon.a smithi119 running (9m) 38s ago 9m 96.8M 2048M 19.0.0-18-g3bd0bfa4 29737aea1903 f719e80f5e13 2023-12-07T10:19:36.339 INFO:teuthology.orchestra.run.smithi119.stdout:mon.b smithi153 running (7m) 51s ago 7m 66.0M 2048M 19.0.0-18-g3bd0bfa4 29737aea1903 c83140c64b80 2023-12-07T10:19:36.339 INFO:teuthology.orchestra.run.smithi119.stdout:mon.c smithi159 running (7m) 51s ago 7m 67.7M 2048M 19.0.0-18-g3bd0bfa4 29737aea1903 472f244eb51f 2023-12-07T10:19:36.339 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (7m) 38s ago 7m 72.3M 4317M 19.0.0-18-g3bd0bfa4 29737aea1903 09454e5f9d24 2023-12-07T10:19:36.339 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi153 running (6m) 51s ago 6m 79.3M 4292M 19.0.0-18-g3bd0bfa4 29737aea1903 89edd22411ab 2023-12-07T10:19:36.339 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi159 running (6m) 51s ago 6m 79.0M 4292M 19.0.0-18-g3bd0bfa4 29737aea1903 415588a2c2ed 2023-12-07T10:19:36.339 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi119 running (5m) 38s ago 5m 67.4M 4317M 19.0.0-18-g3bd0bfa4 29737aea1903 d439b866c079 2023-12-07T10:19:36.339 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi153 running (5m) 51s ago 5m 64.7M 4292M 19.0.0-18-g3bd0bfa4 29737aea1903 255183aaeaab 2023-12-07T10:19:36.339 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi159 running (5m) 51s ago 5m 72.4M 4292M 19.0.0-18-g3bd0bfa4 29737aea1903 578decf38e99 2023-12-07T10:19:36.339 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6 smithi119 running (4m) 38s ago 4m 61.6M 4317M 19.0.0-18-g3bd0bfa4 29737aea1903 952ed719f755 2023-12-07T10:19:36.339 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7 smithi153 running (4m) 51s ago 4m 66.0M 4292M 19.0.0-18-g3bd0bfa4 29737aea1903 6f15287e5dbe 2023-12-07T10:19:36.340 INFO:teuthology.orchestra.run.smithi119.stdout:osd.8 smithi159 running (4m) 51s ago 4m 64.3M 4292M 19.0.0-18-g3bd0bfa4 29737aea1903 0448a409d975 2023-12-07T10:19:36.340 INFO:teuthology.orchestra.run.smithi119.stdout:osd.9 smithi119 running (3m) 38s ago 3m 60.8M 4317M 19.0.0-18-g3bd0bfa4 29737aea1903 f2e47a8b9a59 2023-12-07T10:19:36.340 INFO:teuthology.orchestra.run.smithi119.stdout:osd.10 smithi153 running (3m) 51s ago 3m 64.0M 4292M 19.0.0-18-g3bd0bfa4 29737aea1903 31e1b76921eb 2023-12-07T10:19:36.340 INFO:teuthology.orchestra.run.smithi119.stdout:osd.11 smithi159 running (2m) 51s ago 2m 59.2M 4292M 19.0.0-18-g3bd0bfa4 29737aea1903 dd4928472580 2023-12-07T10:19:36.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.338+0000 7f8e6f8a4700 1 -- 172.21.15.119:0/1436254143 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8e540a1b10 msgr2=0x7f8e540a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:36.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.338+0000 7f8e6f8a4700 1 --2- 172.21.15.119:0/1436254143 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8e540a1b10 0x7f8e540a3fd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8e58006010 tx=0x7f8e5800b540 comp rx=0 tx=0).stop 2023-12-07T10:19:36.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.338+0000 7f8e6f8a4700 1 -- 172.21.15.119:0/1436254143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e68101660 msgr2=0x7f8e681997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:36.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.338+0000 7f8e6f8a4700 1 --2- 172.21.15.119:0/1436254143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e68101660 0x7f8e681997c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8e6400e9a0 tx=0x7f8e6400ecb0 comp rx=0 tx=0).stop 2023-12-07T10:19:36.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.338+0000 7f8e6f8a4700 1 -- 172.21.15.119:0/1436254143 shutdown_connections 2023-12-07T10:19:36.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.338+0000 7f8e6f8a4700 1 --2- 172.21.15.119:0/1436254143 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f8e540a1b10 0x7f8e540a3fd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:36.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.338+0000 7f8e6f8a4700 1 --2- 172.21.15.119:0/1436254143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e68103200 0x7f8e681a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:36.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.338+0000 7f8e6f8a4700 1 --2- 172.21.15.119:0/1436254143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e68101660 0x7f8e681997c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:36.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.338+0000 7f8e6f8a4700 1 --2- 172.21.15.119:0/1436254143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e68102860 0x7f8e68199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:36.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.338+0000 7f8e6f8a4700 1 -- 172.21.15.119:0/1436254143 >> 172.21.15.119:0/1436254143 conn(0x7f8e680fcc20 msgr2=0x7f8e68108050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:36.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.339+0000 7f8e6f8a4700 1 -- 172.21.15.119:0/1436254143 shutdown_connections 2023-12-07T10:19:36.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:36.339+0000 7f8e6f8a4700 1 -- 172.21.15.119:0/1436254143 wait complete. 2023-12-07T10:19:36.864 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- bash -c 'ceph orch ls' 2023-12-07T10:19:37.638 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:19:38.199 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:37 smithi119 ceph-mon[97908]: from='client.25231 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:19:38.200 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:37 smithi119 ceph-mon[97908]: pgmap v341: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2023-12-07T10:19:38.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:37 smithi159 ceph-mon[134726]: from='client.25231 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:19:38.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:37 smithi159 ceph-mon[134726]: pgmap v341: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2023-12-07T10:19:38.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:37 smithi153 ceph-mon[133625]: from='client.25231 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:19:38.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:37 smithi153 ceph-mon[133625]: pgmap v341: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2023-12-07T10:19:39.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.191+0000 7f1f08fd4700 1 -- 172.21.15.119:0/2260776364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f04103200 msgr2=0x7f1f04109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:39.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.191+0000 7f1f08fd4700 1 --2- 172.21.15.119:0/2260776364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f04103200 0x7f1f04109ab0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00b0d0 tx=0x7f1efc00b3e0 comp rx=0 tx=0).stop 2023-12-07T10:19:39.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.191+0000 7f1f08fd4700 1 -- 172.21.15.119:0/2260776364 shutdown_connections 2023-12-07T10:19:39.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.191+0000 7f1f08fd4700 1 --2- 172.21.15.119:0/2260776364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f04103200 0x7f1f04109ab0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:39.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.191+0000 7f1f08fd4700 1 --2- 172.21.15.119:0/2260776364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f04101660 0x7f1f04101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:39.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.191+0000 7f1f08fd4700 1 --2- 172.21.15.119:0/2260776364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f04102860 0x7f1f04102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:39.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.191+0000 7f1f08fd4700 1 -- 172.21.15.119:0/2260776364 >> 172.21.15.119:0/2260776364 conn(0x7f1f040fcc20 msgr2=0x7f1f040ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:39.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.192+0000 7f1f08fd4700 1 -- 172.21.15.119:0/2260776364 shutdown_connections 2023-12-07T10:19:39.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.192+0000 7f1f08fd4700 1 -- 172.21.15.119:0/2260776364 wait complete. 2023-12-07T10:19:39.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.193+0000 7f1f08fd4700 1 Processor -- start 2023-12-07T10:19:39.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.193+0000 7f1f08fd4700 1 -- start start 2023-12-07T10:19:39.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.193+0000 7f1f08fd4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f04101660 0x7f1f04075360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:39.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.193+0000 7f1f08fd4700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f04102860 0x7f1f04073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:39.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.193+0000 7f1f08fd4700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f04103200 0x7f1f04073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:39.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.193+0000 7f1f08fd4700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f040746e0 con 0x7f1f04103200 2023-12-07T10:19:39.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.193+0000 7f1f08fd4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f04074850 con 0x7f1f04101660 2023-12-07T10:19:39.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.193+0000 7f1f08fd4700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f041a16d0 con 0x7f1f04102860 2023-12-07T10:19:39.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.193+0000 7f1f02d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f04103200 0x7f1f04073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:39.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.193+0000 7f1f02d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f04103200 0x7f1f04073f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60678/0 (socket says 172.21.15.119:60678) 2023-12-07T10:19:39.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.193+0000 7f1f02d9d700 1 -- 172.21.15.119:0/853632888 learned_addr learned my addr 172.21.15.119:0/853632888 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:39.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.193+0000 7f1f01d9b700 1 --2- 172.21.15.119:0/853632888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f04102860 0x7f1f04073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:39.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.194+0000 7f1f0259c700 1 --2- 172.21.15.119:0/853632888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f04101660 0x7f1f04075360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:39.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.194+0000 7f1f02d9d700 1 -- 172.21.15.119:0/853632888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f04102860 msgr2=0x7f1f04073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:39.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.194+0000 7f1f02d9d700 1 --2- 172.21.15.119:0/853632888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f04102860 0x7f1f04073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:39.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.194+0000 7f1f02d9d700 1 -- 172.21.15.119:0/853632888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f04101660 msgr2=0x7f1f04075360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:39.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.194+0000 7f1f02d9d700 1 --2- 172.21.15.119:0/853632888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f04101660 0x7f1f04075360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:39.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.194+0000 7f1f02d9d700 1 -- 172.21.15.119:0/853632888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc009d00 con 0x7f1f04103200 2023-12-07T10:19:39.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.194+0000 7f1f02d9d700 1 --2- 172.21.15.119:0/853632888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f04103200 0x7f1f04073f40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1efc007bd0 tx=0x7f1efc009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:39.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.194+0000 7f1ef37fe700 1 -- 172.21.15.119:0/853632888 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc00e040 con 0x7f1f04103200 2023-12-07T10:19:39.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.194+0000 7f1ef37fe700 1 -- 172.21.15.119:0/853632888 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1efc01dd50 con 0x7f1f04103200 2023-12-07T10:19:39.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.194+0000 7f1f08fd4700 1 -- 172.21.15.119:0/853632888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f041a1900 con 0x7f1f04103200 2023-12-07T10:19:39.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.194+0000 7f1ef37fe700 1 -- 172.21.15.119:0/853632888 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc0123f0 con 0x7f1f04103200 2023-12-07T10:19:39.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.195+0000 7f1f08fd4700 1 -- 172.21.15.119:0/853632888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f041a1d20 con 0x7f1f04103200 2023-12-07T10:19:39.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.196+0000 7f1f08fd4700 1 -- 172.21.15.119:0/853632888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f04108a20 con 0x7f1f04103200 2023-12-07T10:19:39.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.197+0000 7f1ef37fe700 1 -- 172.21.15.119:0/853632888 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f1efc01b550 con 0x7f1f04103200 2023-12-07T10:19:39.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.199+0000 7f1ef37fe700 1 --2- 172.21.15.119:0/853632888 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1eec0a1b10 0x7f1eec0a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:39.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.200+0000 7f1ef37fe700 1 -- 172.21.15.119:0/853632888 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f1efc0cf610 con 0x7f1f04103200 2023-12-07T10:19:39.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.200+0000 7f1f0259c700 1 --2- 172.21.15.119:0/853632888 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1eec0a1b10 0x7f1eec0a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:39.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.200+0000 7f1f0259c700 1 --2- 172.21.15.119:0/853632888 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1eec0a1b10 0x7f1eec0a3fd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1f041026c0 tx=0x7f1ef4007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:39.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.204+0000 7f1ef37fe700 1 -- 172.21.15.119:0/853632888 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1efc0986f0 con 0x7f1f04103200 2023-12-07T10:19:39.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.414+0000 7f1f08fd4700 1 -- 172.21.15.119:0/853632888 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f04061a50 con 0x7f1eec0a1b10 2023-12-07T10:19:39.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.420+0000 7f1ef37fe700 1 -- 172.21.15.119:0/853632888 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f1f04061a50 con 0x7f1eec0a1b10 2023-12-07T10:19:39.421 INFO:teuthology.orchestra.run.smithi119.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-12-07T10:19:39.422 INFO:teuthology.orchestra.run.smithi119.stdout:mds.all 12/12 54s ago 2m smithi119=a;smithi119=d;smithi119=g;smithi119=j;smithi153=b;smithi153=e;smithi153=h;smithi153=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l;count:12 2023-12-07T10:19:39.422 INFO:teuthology.orchestra.run.smithi119.stdout:mgr 3/3 54s ago 7m smithi119=x;smithi153=y;smithi159=z;count:3 2023-12-07T10:19:39.422 INFO:teuthology.orchestra.run.smithi119.stdout:mon 3/3 54s ago 7m smithi119:172.21.15.119=a;smithi153:172.21.15.153=b;smithi159:172.21.15.159=c;count:3 2023-12-07T10:19:39.422 INFO:teuthology.orchestra.run.smithi119.stdout:osd 12 54s ago - 2023-12-07T10:19:39.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.422+0000 7f1f08fd4700 1 -- 172.21.15.119:0/853632888 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1eec0a1b10 msgr2=0x7f1eec0a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:39.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.422+0000 7f1f08fd4700 1 --2- 172.21.15.119:0/853632888 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1eec0a1b10 0x7f1eec0a3fd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1f041026c0 tx=0x7f1ef4007400 comp rx=0 tx=0).stop 2023-12-07T10:19:39.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.422+0000 7f1f08fd4700 1 -- 172.21.15.119:0/853632888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f04103200 msgr2=0x7f1f04073f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:39.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.422+0000 7f1f08fd4700 1 --2- 172.21.15.119:0/853632888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f04103200 0x7f1f04073f40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1efc007bd0 tx=0x7f1efc009440 comp rx=0 tx=0).stop 2023-12-07T10:19:39.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.422+0000 7f1f08fd4700 1 -- 172.21.15.119:0/853632888 shutdown_connections 2023-12-07T10:19:39.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.422+0000 7f1f08fd4700 1 --2- 172.21.15.119:0/853632888 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1eec0a1b10 0x7f1eec0a3fd0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:39.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.422+0000 7f1f08fd4700 1 --2- 172.21.15.119:0/853632888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f04103200 0x7f1f04073f40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:39.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.422+0000 7f1f08fd4700 1 --2- 172.21.15.119:0/853632888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f04101660 0x7f1f04075360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:39.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.422+0000 7f1f08fd4700 1 --2- 172.21.15.119:0/853632888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f04102860 0x7f1f04073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:39.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.422+0000 7f1f08fd4700 1 -- 172.21.15.119:0/853632888 >> 172.21.15.119:0/853632888 conn(0x7f1f040fcc20 msgr2=0x7f1f04105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:39.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.422+0000 7f1f08fd4700 1 -- 172.21.15.119:0/853632888 shutdown_connections 2023-12-07T10:19:39.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:39.423+0000 7f1f08fd4700 1 -- 172.21.15.119:0/853632888 wait complete. 2023-12-07T10:19:40.042 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- bash -c 'ceph orch host ls' 2023-12-07T10:19:40.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:39 smithi119 ceph-mon[97908]: pgmap v342: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2023-12-07T10:19:40.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:39 smithi159 ceph-mon[134726]: pgmap v342: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2023-12-07T10:19:40.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:39 smithi153 ceph-mon[133625]: pgmap v342: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2023-12-07T10:19:40.813 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:19:41.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:40 smithi119 ceph-mon[97908]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:19:41.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:40 smithi159 ceph-mon[134726]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:19:41.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:40 smithi153 ceph-mon[133625]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:19:42.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:41 smithi119 ceph-mon[97908]: pgmap v343: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2023-12-07T10:19:42.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:41 smithi159 ceph-mon[134726]: pgmap v343: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2023-12-07T10:19:42.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:41 smithi153 ceph-mon[133625]: pgmap v343: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2023-12-07T10:19:42.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.336+0000 7fe8efc94700 1 -- 172.21.15.119:0/2924197444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8e8103200 msgr2=0x7fe8e8109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:42.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.336+0000 7fe8efc94700 1 --2- 172.21.15.119:0/2924197444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8e8103200 0x7fe8e8109ab0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000b0d0 tx=0x7fe8e000b3e0 comp rx=0 tx=0).stop 2023-12-07T10:19:42.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.336+0000 7fe8efc94700 1 -- 172.21.15.119:0/2924197444 shutdown_connections 2023-12-07T10:19:42.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.336+0000 7fe8efc94700 1 --2- 172.21.15.119:0/2924197444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8e8103200 0x7fe8e8109ab0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:42.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.336+0000 7fe8efc94700 1 --2- 172.21.15.119:0/2924197444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e8101660 0x7fe8e8101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:42.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.336+0000 7fe8efc94700 1 --2- 172.21.15.119:0/2924197444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8e8102860 0x7fe8e8102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:42.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.336+0000 7fe8efc94700 1 -- 172.21.15.119:0/2924197444 >> 172.21.15.119:0/2924197444 conn(0x7fe8e80fcc20 msgr2=0x7fe8e80ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:42.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.336+0000 7fe8efc94700 1 -- 172.21.15.119:0/2924197444 shutdown_connections 2023-12-07T10:19:42.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.336+0000 7fe8efc94700 1 -- 172.21.15.119:0/2924197444 wait complete. 2023-12-07T10:19:42.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.337+0000 7fe8efc94700 1 Processor -- start 2023-12-07T10:19:42.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.337+0000 7fe8efc94700 1 -- start start 2023-12-07T10:19:42.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.337+0000 7fe8efc94700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8e8101660 0x7fe8e8199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:42.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.337+0000 7fe8efc94700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8e8102860 0x7fe8e8199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:42.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.338+0000 7fe8efc94700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e8103200 0x7fe8e81a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:42.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.338+0000 7fe8efc94700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e81a13b0 con 0x7fe8e8101660 2023-12-07T10:19:42.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.338+0000 7fe8efc94700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e81a14f0 con 0x7fe8e8102860 2023-12-07T10:19:42.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.338+0000 7fe8efc94700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e81a1680 con 0x7fe8e8103200 2023-12-07T10:19:42.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.338+0000 7fe8eda30700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8e8101660 0x7fe8e8199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:42.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.338+0000 7fe8eda30700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8e8101660 0x7fe8e8199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60690/0 (socket says 172.21.15.119:60690) 2023-12-07T10:19:42.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.338+0000 7fe8eda30700 1 -- 172.21.15.119:0/2489038716 learned_addr learned my addr 172.21.15.119:0/2489038716 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:42.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.338+0000 7fe8ed22f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8e8102860 0x7fe8e8199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:42.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.338+0000 7fe8ee231700 1 --2- 172.21.15.119:0/2489038716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e8103200 0x7fe8e81a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:42.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.338+0000 7fe8eda30700 1 -- 172.21.15.119:0/2489038716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e8103200 msgr2=0x7fe8e81a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:42.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.338+0000 7fe8eda30700 1 --2- 172.21.15.119:0/2489038716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e8103200 0x7fe8e81a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:42.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.338+0000 7fe8eda30700 1 -- 172.21.15.119:0/2489038716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8e8102860 msgr2=0x7fe8e8199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:42.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.338+0000 7fe8eda30700 1 --2- 172.21.15.119:0/2489038716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8e8102860 0x7fe8e8199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:42.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.338+0000 7fe8eda30700 1 -- 172.21.15.119:0/2489038716 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e0009d00 con 0x7fe8e8101660 2023-12-07T10:19:42.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.339+0000 7fe8eda30700 1 --2- 172.21.15.119:0/2489038716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8e8101660 0x7fe8e8199870 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400d9b0 tx=0x7fe8e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:42.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.339+0000 7fe8deffd700 1 -- 172.21.15.119:0/2489038716 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4004020 con 0x7fe8e8101660 2023-12-07T10:19:42.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.339+0000 7fe8deffd700 1 -- 172.21.15.119:0/2489038716 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8e4009730 con 0x7fe8e8101660 2023-12-07T10:19:42.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.339+0000 7fe8deffd700 1 -- 172.21.15.119:0/2489038716 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e40039e0 con 0x7fe8e8101660 2023-12-07T10:19:42.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.339+0000 7fe8efc94700 1 -- 172.21.15.119:0/2489038716 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e81a1960 con 0x7fe8e8101660 2023-12-07T10:19:42.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.339+0000 7fe8efc94700 1 -- 172.21.15.119:0/2489038716 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e81a1e10 con 0x7fe8e8101660 2023-12-07T10:19:42.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.340+0000 7fe8efc94700 1 -- 172.21.15.119:0/2489038716 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e804f2d0 con 0x7fe8e8101660 2023-12-07T10:19:42.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.342+0000 7fe8deffd700 1 -- 172.21.15.119:0/2489038716 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fe8e40098a0 con 0x7fe8e8101660 2023-12-07T10:19:42.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.343+0000 7fe8deffd700 1 --2- 172.21.15.119:0/2489038716 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe8d40a1b90 0x7fe8d40a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:42.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.343+0000 7fe8deffd700 1 -- 172.21.15.119:0/2489038716 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fe8e40cd520 con 0x7fe8e8101660 2023-12-07T10:19:42.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.343+0000 7fe8ed22f700 1 --2- 172.21.15.119:0/2489038716 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe8d40a1b90 0x7fe8d40a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:42.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.344+0000 7fe8ed22f700 1 --2- 172.21.15.119:0/2489038716 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe8d40a1b90 0x7fe8d40a4050 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8005950 tx=0x7fe8d800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:42.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.347+0000 7fe8deffd700 1 -- 172.21.15.119:0/2489038716 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe8e4096570 con 0x7fe8e8101660 2023-12-07T10:19:42.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.552+0000 7fe8efc94700 1 -- 172.21.15.119:0/2489038716 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8e8074f60 con 0x7fe8d40a1b90 2023-12-07T10:19:42.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.554+0000 7fe8deffd700 1 -- 172.21.15.119:0/2489038716 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fe8e8074f60 con 0x7fe8d40a1b90 2023-12-07T10:19:42.556 INFO:teuthology.orchestra.run.smithi119.stdout:HOST ADDR LABELS STATUS 2023-12-07T10:19:42.556 INFO:teuthology.orchestra.run.smithi119.stdout:smithi119 172.21.15.119 2023-12-07T10:19:42.556 INFO:teuthology.orchestra.run.smithi119.stdout:smithi153 172.21.15.153 2023-12-07T10:19:42.556 INFO:teuthology.orchestra.run.smithi119.stdout:smithi159 172.21.15.159 2023-12-07T10:19:42.556 INFO:teuthology.orchestra.run.smithi119.stdout:3 hosts in cluster 2023-12-07T10:19:42.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.556+0000 7fe8efc94700 1 -- 172.21.15.119:0/2489038716 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe8d40a1b90 msgr2=0x7fe8d40a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:42.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.556+0000 7fe8efc94700 1 --2- 172.21.15.119:0/2489038716 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe8d40a1b90 0x7fe8d40a4050 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8005950 tx=0x7fe8d800b410 comp rx=0 tx=0).stop 2023-12-07T10:19:42.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.556+0000 7fe8efc94700 1 -- 172.21.15.119:0/2489038716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8e8101660 msgr2=0x7fe8e8199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:42.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.556+0000 7fe8efc94700 1 --2- 172.21.15.119:0/2489038716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8e8101660 0x7fe8e8199870 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400d9b0 tx=0x7fe8e400dcc0 comp rx=0 tx=0).stop 2023-12-07T10:19:42.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.556+0000 7fe8efc94700 1 -- 172.21.15.119:0/2489038716 shutdown_connections 2023-12-07T10:19:42.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.556+0000 7fe8efc94700 1 --2- 172.21.15.119:0/2489038716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e8103200 0x7fe8e81a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:42.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.556+0000 7fe8efc94700 1 --2- 172.21.15.119:0/2489038716 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe8d40a1b90 0x7fe8d40a4050 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:42.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.556+0000 7fe8efc94700 1 --2- 172.21.15.119:0/2489038716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8e8101660 0x7fe8e8199870 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:42.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.556+0000 7fe8efc94700 1 --2- 172.21.15.119:0/2489038716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8e8102860 0x7fe8e8199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:42.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.556+0000 7fe8efc94700 1 -- 172.21.15.119:0/2489038716 >> 172.21.15.119:0/2489038716 conn(0x7fe8e80fcc20 msgr2=0x7fe8e8073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:42.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.557+0000 7fe8efc94700 1 -- 172.21.15.119:0/2489038716 shutdown_connections 2023-12-07T10:19:42.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:42.557+0000 7fe8efc94700 1 -- 172.21.15.119:0/2489038716 wait complete. 2023-12-07T10:19:43.016 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- bash -c 'ceph orch device ls' 2023-12-07T10:19:43.791 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:19:44.221 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:43 smithi119 ceph-mon[97908]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:19:44.222 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:43 smithi119 ceph-mon[97908]: pgmap v344: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2023-12-07T10:19:44.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:43 smithi159 ceph-mon[134726]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:19:44.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:43 smithi159 ceph-mon[134726]: pgmap v344: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2023-12-07T10:19:44.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:43 smithi153 ceph-mon[133625]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:19:44.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:43 smithi153 ceph-mon[133625]: pgmap v344: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2023-12-07T10:19:45.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.302+0000 7f029823c700 1 -- 172.21.15.119:0/1315902639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0290102860 msgr2=0x7f0290102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:45.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.302+0000 7f029823c700 1 --2- 172.21.15.119:0/1315902639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0290102860 0x7f0290102cc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f02800099c0 tx=0x7f0280009cd0 comp rx=0 tx=0).stop 2023-12-07T10:19:45.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.303+0000 7f029823c700 1 -- 172.21.15.119:0/1315902639 shutdown_connections 2023-12-07T10:19:45.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.303+0000 7f029823c700 1 --2- 172.21.15.119:0/1315902639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0290103200 0x7f0290109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:45.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.303+0000 7f029823c700 1 --2- 172.21.15.119:0/1315902639 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0290101660 0x7f0290101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:45.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.303+0000 7f029823c700 1 --2- 172.21.15.119:0/1315902639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0290102860 0x7f0290102cc0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:45.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.303+0000 7f029823c700 1 -- 172.21.15.119:0/1315902639 >> 172.21.15.119:0/1315902639 conn(0x7f02900fcc20 msgr2=0x7f02900ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:45.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.303+0000 7f029823c700 1 -- 172.21.15.119:0/1315902639 shutdown_connections 2023-12-07T10:19:45.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.303+0000 7f029823c700 1 -- 172.21.15.119:0/1315902639 wait complete. 2023-12-07T10:19:45.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.304+0000 7f029823c700 1 Processor -- start 2023-12-07T10:19:45.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.304+0000 7f029823c700 1 -- start start 2023-12-07T10:19:45.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.304+0000 7f029823c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0290101660 0x7f0290199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:45.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.304+0000 7f029823c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0290102860 0x7f0290199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:45.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.304+0000 7f029823c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0290103200 0x7f02901a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:45.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.304+0000 7f029823c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02901a1290 con 0x7f0290103200 2023-12-07T10:19:45.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.304+0000 7f029823c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02901a13d0 con 0x7f0290102860 2023-12-07T10:19:45.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.304+0000 7f029823c700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02901a1560 con 0x7f0290101660 2023-12-07T10:19:45.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.305+0000 7f02967d9700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0290103200 0x7f02901a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:45.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.305+0000 7f02967d9700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0290103200 0x7f02901a0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33216/0 (socket says 172.21.15.119:33216) 2023-12-07T10:19:45.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.305+0000 7f02957d7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0290102860 0x7f0290199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:45.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.305+0000 7f02967d9700 1 -- 172.21.15.119:0/2019525809 learned_addr learned my addr 172.21.15.119:0/2019525809 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:45.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.305+0000 7f0295fd8700 1 --2- 172.21.15.119:0/2019525809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0290101660 0x7f0290199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:45.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.305+0000 7f02967d9700 1 -- 172.21.15.119:0/2019525809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0290101660 msgr2=0x7f0290199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:45.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.305+0000 7f02967d9700 1 --2- 172.21.15.119:0/2019525809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0290101660 0x7f0290199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:45.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.305+0000 7f02967d9700 1 -- 172.21.15.119:0/2019525809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0290102860 msgr2=0x7f0290199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:45.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.305+0000 7f02967d9700 1 --2- 172.21.15.119:0/2019525809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0290102860 0x7f0290199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:45.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.305+0000 7f02967d9700 1 -- 172.21.15.119:0/2019525809 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02800095f0 con 0x7f0290103200 2023-12-07T10:19:45.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.305+0000 7f0295fd8700 1 --2- 172.21.15.119:0/2019525809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0290101660 0x7f0290199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:45.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.305+0000 7f02967d9700 1 --2- 172.21.15.119:0/2019525809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0290103200 0x7f02901a0cc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0288007f10 tx=0x7f028800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:45.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.306+0000 7f0286ffd700 1 -- 172.21.15.119:0/2019525809 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028800dd20 con 0x7f0290103200 2023-12-07T10:19:45.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.306+0000 7f0286ffd700 1 -- 172.21.15.119:0/2019525809 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f028800f040 con 0x7f0290103200 2023-12-07T10:19:45.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.306+0000 7f029823c700 1 -- 172.21.15.119:0/2019525809 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02901a1840 con 0x7f0290103200 2023-12-07T10:19:45.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.306+0000 7f0286ffd700 1 -- 172.21.15.119:0/2019525809 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02880048a0 con 0x7f0290103200 2023-12-07T10:19:45.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.306+0000 7f029823c700 1 -- 172.21.15.119:0/2019525809 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02901a1d40 con 0x7f0290103200 2023-12-07T10:19:45.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.307+0000 7f029823c700 1 -- 172.21.15.119:0/2019525809 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f029004f2d0 con 0x7f0290103200 2023-12-07T10:19:45.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.309+0000 7f0286ffd700 1 -- 172.21.15.119:0/2019525809 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f0288003a00 con 0x7f0290103200 2023-12-07T10:19:45.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.310+0000 7f0286ffd700 1 --2- 172.21.15.119:0/2019525809 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f027c0a1b90 0x7f027c0a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:45.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.310+0000 7f0286ffd700 1 -- 172.21.15.119:0/2019525809 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f02880cd910 con 0x7f0290103200 2023-12-07T10:19:45.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.310+0000 7f0295fd8700 1 --2- 172.21.15.119:0/2019525809 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f027c0a1b90 0x7f027c0a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:45.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.311+0000 7f0295fd8700 1 --2- 172.21.15.119:0/2019525809 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f027c0a1b90 0x7f027c0a4050 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f02901026c0 tx=0x7f028c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:45.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.312+0000 7f0286ffd700 1 -- 172.21.15.119:0/2019525809 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0288096960 con 0x7f0290103200 2023-12-07T10:19:45.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.519+0000 7f029823c700 1 -- 172.21.15.119:0/2019525809 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0290074f60 con 0x7f027c0a1b90 2023-12-07T10:19:45.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.521+0000 7f0286ffd700 1 -- 172.21.15.119:0/2019525809 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f0290074f60 con 0x7f027c0a1b90 2023-12-07T10:19:45.522 INFO:teuthology.orchestra.run.smithi119.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-12-07T10:19:45.522 INFO:teuthology.orchestra.run.smithi119.stdout:smithi119 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400Y1400BGN 372G No 3m ago Has a FileSystem, LVM detected 2023-12-07T10:19:45.523 INFO:teuthology.orchestra.run.smithi119.stdout:smithi153 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300MV400BGN 372G No 3m ago Has a FileSystem, LVM detected 2023-12-07T10:19:45.523 INFO:teuthology.orchestra.run.smithi119.stdout:smithi159 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GH400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-12-07T10:19:45.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.523+0000 7f029823c700 1 -- 172.21.15.119:0/2019525809 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f027c0a1b90 msgr2=0x7f027c0a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:45.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.523+0000 7f029823c700 1 --2- 172.21.15.119:0/2019525809 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f027c0a1b90 0x7f027c0a4050 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f02901026c0 tx=0x7f028c009260 comp rx=0 tx=0).stop 2023-12-07T10:19:45.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.523+0000 7f029823c700 1 -- 172.21.15.119:0/2019525809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0290103200 msgr2=0x7f02901a0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:45.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.523+0000 7f029823c700 1 --2- 172.21.15.119:0/2019525809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0290103200 0x7f02901a0cc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0288007f10 tx=0x7f028800d3b0 comp rx=0 tx=0).stop 2023-12-07T10:19:45.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.523+0000 7f029823c700 1 -- 172.21.15.119:0/2019525809 shutdown_connections 2023-12-07T10:19:45.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.523+0000 7f029823c700 1 --2- 172.21.15.119:0/2019525809 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f027c0a1b90 0x7f027c0a4050 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:45.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.523+0000 7f029823c700 1 --2- 172.21.15.119:0/2019525809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0290103200 0x7f02901a0cc0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:45.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.523+0000 7f029823c700 1 --2- 172.21.15.119:0/2019525809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0290101660 0x7f0290199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:45.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.523+0000 7f029823c700 1 --2- 172.21.15.119:0/2019525809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0290102860 0x7f0290199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:45.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.523+0000 7f029823c700 1 -- 172.21.15.119:0/2019525809 >> 172.21.15.119:0/2019525809 conn(0x7f02900fcc20 msgr2=0x7f0290073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:45.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.523+0000 7f029823c700 1 -- 172.21.15.119:0/2019525809 shutdown_connections 2023-12-07T10:19:45.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:45.523+0000 7f029823c700 1 -- 172.21.15.119:0/2019525809 wait complete. 2023-12-07T10:19:45.994 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-12-07T10:19:46.005 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi119.front.sepia.ceph.com 2023-12-07T10:19:46.005 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- bash -c 'ceph fs dump' 2023-12-07T10:19:46.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:45 smithi119 ceph-mon[97908]: pgmap v345: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2023-12-07T10:19:46.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:45 smithi159 ceph-mon[134726]: pgmap v345: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2023-12-07T10:19:46.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:45 smithi153 ceph-mon[133625]: pgmap v345: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2023-12-07T10:19:46.774 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:19:47.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:46 smithi119 ceph-mon[97908]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:19:47.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:46 smithi159 ceph-mon[134726]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:19:47.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:46 smithi153 ceph-mon[133625]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:19:48.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:47 smithi119 ceph-mon[97908]: pgmap v346: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 10 op/s 2023-12-07T10:19:48.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:47 smithi159 ceph-mon[134726]: pgmap v346: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 10 op/s 2023-12-07T10:19:48.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:47 smithi153 ceph-mon[133625]: pgmap v346: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 10 op/s 2023-12-07T10:19:48.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.310+0000 7ff20d717700 1 -- 172.21.15.119:0/2776960448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff208102860 msgr2=0x7ff208102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:48.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.310+0000 7ff20d717700 1 --2- 172.21.15.119:0/2776960448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff208102860 0x7ff208102cc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc0099c0 tx=0x7ff1fc009cd0 comp rx=0 tx=0).stop 2023-12-07T10:19:48.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.310+0000 7ff20d717700 1 -- 172.21.15.119:0/2776960448 shutdown_connections 2023-12-07T10:19:48.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.310+0000 7ff20d717700 1 --2- 172.21.15.119:0/2776960448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff208103200 0x7ff208109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:48.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.310+0000 7ff20d717700 1 --2- 172.21.15.119:0/2776960448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff208101660 0x7ff208101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:48.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.310+0000 7ff20d717700 1 --2- 172.21.15.119:0/2776960448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff208102860 0x7ff208102cc0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:48.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.310+0000 7ff20d717700 1 -- 172.21.15.119:0/2776960448 >> 172.21.15.119:0/2776960448 conn(0x7ff2080fcc20 msgr2=0x7ff2080ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:48.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.310+0000 7ff20d717700 1 -- 172.21.15.119:0/2776960448 shutdown_connections 2023-12-07T10:19:48.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.310+0000 7ff20d717700 1 -- 172.21.15.119:0/2776960448 wait complete. 2023-12-07T10:19:48.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.311+0000 7ff20d717700 1 Processor -- start 2023-12-07T10:19:48.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.311+0000 7ff20d717700 1 -- start start 2023-12-07T10:19:48.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.311+0000 7ff20d717700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff208101660 0x7ff2081997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:48.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.311+0000 7ff20d717700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff208102860 0x7ff208199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:48.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.311+0000 7ff20d717700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff208103200 0x7ff2081a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:48.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.312+0000 7ff20d717700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2081a12e0 con 0x7ff208101660 2023-12-07T10:19:48.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.312+0000 7ff20d717700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2081a1420 con 0x7ff208102860 2023-12-07T10:19:48.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.312+0000 7ff20d717700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2081a15b0 con 0x7ff208103200 2023-12-07T10:19:48.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.312+0000 7ff206ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff208101660 0x7ff2081997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:48.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.312+0000 7ff206ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff208101660 0x7ff2081997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33234/0 (socket says 172.21.15.119:33234) 2023-12-07T10:19:48.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.312+0000 7ff206ffd700 1 -- 172.21.15.119:0/2453737436 learned_addr learned my addr 172.21.15.119:0/2453737436 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:48.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.312+0000 7ff2077fe700 1 --2- 172.21.15.119:0/2453737436 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff208103200 0x7ff2081a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:48.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.312+0000 7ff2067fc700 1 --2- 172.21.15.119:0/2453737436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff208102860 0x7ff208199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:48.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.312+0000 7ff206ffd700 1 -- 172.21.15.119:0/2453737436 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff208103200 msgr2=0x7ff2081a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:48.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.312+0000 7ff206ffd700 1 --2- 172.21.15.119:0/2453737436 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff208103200 0x7ff2081a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:48.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.312+0000 7ff206ffd700 1 -- 172.21.15.119:0/2453737436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff208102860 msgr2=0x7ff208199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:48.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.312+0000 7ff206ffd700 1 --2- 172.21.15.119:0/2453737436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff208102860 0x7ff208199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:48.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.312+0000 7ff206ffd700 1 -- 172.21.15.119:0/2453737436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1fc0095f0 con 0x7ff208101660 2023-12-07T10:19:48.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.312+0000 7ff206ffd700 1 --2- 172.21.15.119:0/2453737436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff208101660 0x7ff2081997a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff1f800e9a0 tx=0x7ff1f800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:48.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.313+0000 7ff1effff700 1 -- 172.21.15.119:0/2453737436 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8009790 con 0x7ff208101660 2023-12-07T10:19:48.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.313+0000 7ff1effff700 1 -- 172.21.15.119:0/2453737436 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1f800c810 con 0x7ff208101660 2023-12-07T10:19:48.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.313+0000 7ff20d717700 1 -- 172.21.15.119:0/2453737436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2081a1890 con 0x7ff208101660 2023-12-07T10:19:48.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.313+0000 7ff1effff700 1 -- 172.21.15.119:0/2453737436 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8003c70 con 0x7ff208101660 2023-12-07T10:19:48.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.313+0000 7ff20d717700 1 -- 172.21.15.119:0/2453737436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2081a1de0 con 0x7ff208101660 2023-12-07T10:19:48.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.314+0000 7ff20d717700 1 -- 172.21.15.119:0/2453737436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff20804f2d0 con 0x7ff208101660 2023-12-07T10:19:48.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.316+0000 7ff1effff700 1 -- 172.21.15.119:0/2453737436 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7ff1f800cc80 con 0x7ff208101660 2023-12-07T10:19:48.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.317+0000 7ff1effff700 1 --2- 172.21.15.119:0/2453737436 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff1f00a1b90 0x7ff1f00a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:48.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.317+0000 7ff1effff700 1 -- 172.21.15.119:0/2453737436 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7ff1f8014070 con 0x7ff208101660 2023-12-07T10:19:48.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.317+0000 7ff2067fc700 1 --2- 172.21.15.119:0/2453737436 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff1f00a1b90 0x7ff1f00a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:48.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.318+0000 7ff2067fc700 1 --2- 172.21.15.119:0/2453737436 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff1f00a1b90 0x7ff1f00a4050 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc006010 tx=0x7ff1fc00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:48.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.321+0000 7ff1effff700 1 -- 172.21.15.119:0/2453737436 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff1f8096ae0 con 0x7ff208101660 2023-12-07T10:19:48.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.563+0000 7ff20d717700 1 -- 172.21.15.119:0/2453737436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff2081a2190 con 0x7ff208101660 2023-12-07T10:19:48.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.563+0000 7ff1effff700 1 -- 172.21.15.119:0/2453737436 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+3000 (secure 0 0 0) 0x7ff1f809a100 con 0x7ff208101660 2023-12-07T10:19:48.565 INFO:teuthology.orchestra.run.smithi119.stdout:e29 2023-12-07T10:19:48.565 INFO:teuthology.orchestra.run.smithi119.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-12-07T10:19:48.566 INFO:teuthology.orchestra.run.smithi119.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-12-07T10:19:48.566 INFO:teuthology.orchestra.run.smithi119.stdout:legacy client fscid: 1 2023-12-07T10:19:48.566 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:19:48.566 INFO:teuthology.orchestra.run.smithi119.stdout:Filesystem 'cephfs' (1) 2023-12-07T10:19:48.566 INFO:teuthology.orchestra.run.smithi119.stdout:fs_name cephfs 2023-12-07T10:19:48.566 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 29 2023-12-07T10:19:48.566 INFO:teuthology.orchestra.run.smithi119.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-12-07T10:19:48.566 INFO:teuthology.orchestra.run.smithi119.stdout:created 2023-12-07T10:17:41.278311+0000 2023-12-07T10:19:48.566 INFO:teuthology.orchestra.run.smithi119.stdout:modified 2023-12-07T10:18:18.543356+0000 2023-12-07T10:19:48.566 INFO:teuthology.orchestra.run.smithi119.stdout:tableserver 0 2023-12-07T10:19:48.566 INFO:teuthology.orchestra.run.smithi119.stdout:root 0 2023-12-07T10:19:48.566 INFO:teuthology.orchestra.run.smithi119.stdout:session_timeout 300 2023-12-07T10:19:48.566 INFO:teuthology.orchestra.run.smithi119.stdout:session_autoclose 300 2023-12-07T10:19:48.566 INFO:teuthology.orchestra.run.smithi119.stdout:max_file_size 1099511627776 2023-12-07T10:19:48.567 INFO:teuthology.orchestra.run.smithi119.stdout:max_xattr_size 65536 2023-12-07T10:19:48.567 INFO:teuthology.orchestra.run.smithi119.stdout:required_client_features {} 2023-12-07T10:19:48.567 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure 0 2023-12-07T10:19:48.567 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure_osd_epoch 0 2023-12-07T10:19:48.567 INFO:teuthology.orchestra.run.smithi119.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-12-07T10:19:48.567 INFO:teuthology.orchestra.run.smithi119.stdout:max_mds 5 2023-12-07T10:19:48.567 INFO:teuthology.orchestra.run.smithi119.stdout:in 0,1,2,3,4 2023-12-07T10:19:48.567 INFO:teuthology.orchestra.run.smithi119.stdout:up {0=24566,1=24542,2=24517,3=24505,4=24488} 2023-12-07T10:19:48.567 INFO:teuthology.orchestra.run.smithi119.stdout:failed 2023-12-07T10:19:48.567 INFO:teuthology.orchestra.run.smithi119.stdout:damaged 2023-12-07T10:19:48.567 INFO:teuthology.orchestra.run.smithi119.stdout:stopped 2023-12-07T10:19:48.567 INFO:teuthology.orchestra.run.smithi119.stdout:data_pools [3,4] 2023-12-07T10:19:48.567 INFO:teuthology.orchestra.run.smithi119.stdout:metadata_pool 2 2023-12-07T10:19:48.567 INFO:teuthology.orchestra.run.smithi119.stdout:inline_data disabled 2023-12-07T10:19:48.567 INFO:teuthology.orchestra.run.smithi119.stdout:balancer 2023-12-07T10:19:48.568 INFO:teuthology.orchestra.run.smithi119.stdout:bal_rank_mask -1 2023-12-07T10:19:48.568 INFO:teuthology.orchestra.run.smithi119.stdout:standby_count_wanted 0 2023-12-07T10:19:48.568 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.b{0:24566} state up:active seq 4 export targets 1 addr [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] compat {c=[1],r=[1],i=[fff]}] 2023-12-07T10:19:48.568 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.i{0:24538} state up:standby-replay seq 1 addr [v2:172.21.15.159:6838/3984962900,v1:172.21.15.159:6839/3984962900] compat {c=[1],r=[1],i=[fff]}] 2023-12-07T10:19:48.568 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.g{1:24542} state up:active seq 12 addr [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] compat {c=[1],r=[1],i=[fff]}] 2023-12-07T10:19:48.568 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.h{1:24518} state up:standby-replay seq 1 addr [v2:172.21.15.153:6836/2925503246,v1:172.21.15.153:6837/2925503246] compat {c=[1],r=[1],i=[fff]}] 2023-12-07T10:19:48.568 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.c{2:24517} state up:active seq 14 addr [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] compat {c=[1],r=[1],i=[fff]}] 2023-12-07T10:19:48.568 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.k{2:24452} state up:standby-replay seq 1 addr [v2:172.21.15.153:6832/209059922,v1:172.21.15.153:6833/209059922] compat {c=[1],r=[1],i=[fff]}] 2023-12-07T10:19:48.568 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.f{3:24505} state up:active seq 15 addr [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] compat {c=[1],r=[1],i=[fff]}] 2023-12-07T10:19:48.568 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.l{3:24445} state up:standby-replay seq 1 addr [v2:172.21.15.159:6832/935461464,v1:172.21.15.159:6833/935461464] compat {c=[1],r=[1],i=[fff]}] 2023-12-07T10:19:48.568 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.e{4:24488} state up:active seq 16 addr [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] compat {c=[1],r=[1],i=[fff]}] 2023-12-07T10:19:48.568 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.a{4:14535} state up:standby-replay seq 1 addr [v2:172.21.15.119:6840/1791998028,v1:172.21.15.119:6841/1791998028] compat {c=[1],r=[1],i=[fff]}] 2023-12-07T10:19:48.568 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:19:48.568 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:19:48.568 INFO:teuthology.orchestra.run.smithi119.stdout:Standby daemons: 2023-12-07T10:19:48.568 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:19:48.568 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.j{-1:14457} state up:standby seq 1 addr [v2:172.21.15.119:6834/1556173472,v1:172.21.15.119:6835/1556173472] compat {c=[1],r=[1],i=[fff]}] 2023-12-07T10:19:48.568 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.d{-1:14469} state up:standby seq 1 addr [v2:172.21.15.119:6836/184007634,v1:172.21.15.119:6837/184007634] compat {c=[1],r=[1],i=[fff]}] 2023-12-07T10:19:48.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.566+0000 7ff20d717700 1 -- 172.21.15.119:0/2453737436 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff1f00a1b90 msgr2=0x7ff1f00a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:48.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.566+0000 7ff20d717700 1 --2- 172.21.15.119:0/2453737436 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff1f00a1b90 0x7ff1f00a4050 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc006010 tx=0x7ff1fc00b540 comp rx=0 tx=0).stop 2023-12-07T10:19:48.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.566+0000 7ff20d717700 1 -- 172.21.15.119:0/2453737436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff208101660 msgr2=0x7ff2081997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:48.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.566+0000 7ff20d717700 1 --2- 172.21.15.119:0/2453737436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff208101660 0x7ff2081997a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff1f800e9a0 tx=0x7ff1f800ecb0 comp rx=0 tx=0).stop 2023-12-07T10:19:48.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.566+0000 7ff20d717700 1 -- 172.21.15.119:0/2453737436 shutdown_connections 2023-12-07T10:19:48.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.566+0000 7ff20d717700 1 --2- 172.21.15.119:0/2453737436 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff208103200 0x7ff2081a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:48.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.566+0000 7ff20d717700 1 --2- 172.21.15.119:0/2453737436 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff1f00a1b90 0x7ff1f00a4050 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:48.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.566+0000 7ff20d717700 1 --2- 172.21.15.119:0/2453737436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff208101660 0x7ff2081997a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:48.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.566+0000 7ff20d717700 1 --2- 172.21.15.119:0/2453737436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff208102860 0x7ff208199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:48.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.566+0000 7ff20d717700 1 -- 172.21.15.119:0/2453737436 >> 172.21.15.119:0/2453737436 conn(0x7ff2080fcc20 msgr2=0x7ff208108000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:48.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.566+0000 7ff20d717700 1 -- 172.21.15.119:0/2453737436 shutdown_connections 2023-12-07T10:19:48.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:48.566+0000 7ff20d717700 1 -- 172.21.15.119:0/2453737436 wait complete. 2023-12-07T10:19:48.570 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 29 2023-12-07T10:19:49.077 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- bash -c 'ceph osd dump' 2023-12-07T10:19:49.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:48 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2453737436' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-12-07T10:19:49.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:48 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2453737436' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-12-07T10:19:49.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:48 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2453737436' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-12-07T10:19:49.848 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:19:50.262 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:49 smithi119 ceph-mon[97908]: pgmap v347: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-07T10:19:50.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:49 smithi159 ceph-mon[134726]: pgmap v347: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-07T10:19:50.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:49 smithi153 ceph-mon[133625]: pgmap v347: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-07T10:19:51.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.425+0000 7f05b3c2d700 1 -- 172.21.15.119:0/1934596595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05ac103200 msgr2=0x7f05ac109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:51.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.425+0000 7f05b3c2d700 1 --2- 172.21.15.119:0/1934596595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05ac103200 0x7f05ac109ab0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f05a400b0d0 tx=0x7f05a400b3e0 comp rx=0 tx=0).stop 2023-12-07T10:19:51.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.425+0000 7f05b3c2d700 1 -- 172.21.15.119:0/1934596595 shutdown_connections 2023-12-07T10:19:51.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.425+0000 7f05b3c2d700 1 --2- 172.21.15.119:0/1934596595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05ac103200 0x7f05ac109ab0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:51.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.425+0000 7f05b3c2d700 1 --2- 172.21.15.119:0/1934596595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05ac101660 0x7f05ac101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:51.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.425+0000 7f05b3c2d700 1 --2- 172.21.15.119:0/1934596595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05ac102860 0x7f05ac102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:51.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.425+0000 7f05b3c2d700 1 -- 172.21.15.119:0/1934596595 >> 172.21.15.119:0/1934596595 conn(0x7f05ac0fcc20 msgr2=0x7f05ac0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:51.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.425+0000 7f05b3c2d700 1 -- 172.21.15.119:0/1934596595 shutdown_connections 2023-12-07T10:19:51.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.425+0000 7f05b3c2d700 1 -- 172.21.15.119:0/1934596595 wait complete. 2023-12-07T10:19:51.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.427+0000 7f05b3c2d700 1 Processor -- start 2023-12-07T10:19:51.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.427+0000 7f05b3c2d700 1 -- start start 2023-12-07T10:19:51.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.427+0000 7f05b3c2d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05ac101660 0x7f05ac199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:51.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.427+0000 7f05b3c2d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05ac102860 0x7f05ac199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:51.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.427+0000 7f05b3c2d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05ac103200 0x7f05ac1a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:51.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.427+0000 7f05b3c2d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac1a13b0 con 0x7f05ac103200 2023-12-07T10:19:51.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.427+0000 7f05b3c2d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac1a14f0 con 0x7f05ac102860 2023-12-07T10:19:51.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.427+0000 7f05b3c2d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac1a1680 con 0x7f05ac101660 2023-12-07T10:19:51.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.427+0000 7f05b21ca700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05ac103200 0x7f05ac1a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:51.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.427+0000 7f05b21ca700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05ac103200 0x7f05ac1a0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33260/0 (socket says 172.21.15.119:33260) 2023-12-07T10:19:51.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.427+0000 7f05b21ca700 1 -- 172.21.15.119:0/4088625684 learned_addr learned my addr 172.21.15.119:0/4088625684 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:51.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.427+0000 7f05b19c9700 1 --2- 172.21.15.119:0/4088625684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05ac101660 0x7f05ac199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:51.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.427+0000 7f05b11c8700 1 --2- 172.21.15.119:0/4088625684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05ac102860 0x7f05ac199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:51.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.428+0000 7f05b21ca700 1 -- 172.21.15.119:0/4088625684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05ac101660 msgr2=0x7f05ac199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:51.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.428+0000 7f05b21ca700 1 --2- 172.21.15.119:0/4088625684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05ac101660 0x7f05ac199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:51.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.428+0000 7f05b21ca700 1 -- 172.21.15.119:0/4088625684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05ac102860 msgr2=0x7f05ac199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:51.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.428+0000 7f05b21ca700 1 --2- 172.21.15.119:0/4088625684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05ac102860 0x7f05ac199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:51.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.428+0000 7f05b21ca700 1 -- 172.21.15.119:0/4088625684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a4009d00 con 0x7f05ac103200 2023-12-07T10:19:51.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.428+0000 7f05b21ca700 1 --2- 172.21.15.119:0/4088625684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05ac103200 0x7f05ac1a0de0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f05a4008e40 tx=0x7f05a4008e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:51.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.428+0000 7f05a2ffd700 1 -- 172.21.15.119:0/4088625684 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a400e040 con 0x7f05ac103200 2023-12-07T10:19:51.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.428+0000 7f05a2ffd700 1 -- 172.21.15.119:0/4088625684 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05a401d930 con 0x7f05ac103200 2023-12-07T10:19:51.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.428+0000 7f05a2ffd700 1 -- 172.21.15.119:0/4088625684 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a401b9f0 con 0x7f05ac103200 2023-12-07T10:19:51.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.428+0000 7f05b3c2d700 1 -- 172.21.15.119:0/4088625684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ac1a1900 con 0x7f05ac103200 2023-12-07T10:19:51.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.429+0000 7f05b3c2d700 1 -- 172.21.15.119:0/4088625684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ac1a1d70 con 0x7f05ac103200 2023-12-07T10:19:51.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.430+0000 7f05b3c2d700 1 -- 172.21.15.119:0/4088625684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ac04f2d0 con 0x7f05ac103200 2023-12-07T10:19:51.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.432+0000 7f05a2ffd700 1 -- 172.21.15.119:0/4088625684 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f05a401daa0 con 0x7f05ac103200 2023-12-07T10:19:51.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.433+0000 7f05a2ffd700 1 --2- 172.21.15.119:0/4088625684 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f05980a1b90 0x7f05980a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:51.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.433+0000 7f05a2ffd700 1 -- 172.21.15.119:0/4088625684 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f05a40cec70 con 0x7f05ac103200 2023-12-07T10:19:51.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.433+0000 7f05b19c9700 1 --2- 172.21.15.119:0/4088625684 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f05980a1b90 0x7f05980a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:51.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.434+0000 7f05b19c9700 1 --2- 172.21.15.119:0/4088625684 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f05980a1b90 0x7f05980a4050 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f05ac1026c0 tx=0x7f05a8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:51.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.437+0000 7f05a2ffd700 1 -- 172.21.15.119:0/4088625684 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f05a4097d50 con 0x7f05ac103200 2023-12-07T10:19:51.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.635+0000 7f05b3c2d700 1 -- 172.21.15.119:0/4088625684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f05ac10cec0 con 0x7f05ac103200 2023-12-07T10:19:51.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.637+0000 7f05a2ffd700 1 -- 172.21.15.119:0/4088625684 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v93) v1 ==== 56+0+5712 (secure 0 0 0) 0x7f05a409b370 con 0x7f05ac103200 2023-12-07T10:19:51.638 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 93 2023-12-07T10:19:51.638 INFO:teuthology.orchestra.run.smithi119.stdout:fsid a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:19:51.638 INFO:teuthology.orchestra.run.smithi119.stdout:created 2023-12-07T10:09:59.945863+0000 2023-12-07T10:19:51.638 INFO:teuthology.orchestra.run.smithi119.stdout:modified 2023-12-07T10:19:02.800626+0000 2023-12-07T10:19:51.639 INFO:teuthology.orchestra.run.smithi119.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-12-07T10:19:51.639 INFO:teuthology.orchestra.run.smithi119.stdout:crush_version 27 2023-12-07T10:19:51.639 INFO:teuthology.orchestra.run.smithi119.stdout:full_ratio 0.95 2023-12-07T10:19:51.639 INFO:teuthology.orchestra.run.smithi119.stdout:backfillfull_ratio 0.9 2023-12-07T10:19:51.639 INFO:teuthology.orchestra.run.smithi119.stdout:nearfull_ratio 0.85 2023-12-07T10:19:51.639 INFO:teuthology.orchestra.run.smithi119.stdout:require_min_compat_client luminous 2023-12-07T10:19:51.639 INFO:teuthology.orchestra.run.smithi119.stdout:min_compat_client luminous 2023-12-07T10:19:51.639 INFO:teuthology.orchestra.run.smithi119.stdout:require_osd_release reef 2023-12-07T10:19:51.639 INFO:teuthology.orchestra.run.smithi119.stdout:stretch_mode_enabled false 2023-12-07T10:19:51.639 INFO:teuthology.orchestra.run.smithi119.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-12-07T10:19:51.639 INFO:teuthology.orchestra.run.smithi119.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-12-07T10:19:51.639 INFO:teuthology.orchestra.run.smithi119.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-12-07T10:19:51.639 INFO:teuthology.orchestra.run.smithi119.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-12-07T10:19:51.639 INFO:teuthology.orchestra.run.smithi119.stdout:max_osd 12 2023-12-07T10:19:51.639 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 up in weight 1 up_from 8 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6802/2141950133,v1:172.21.15.119:6803/2141950133] [v2:172.21.15.119:6804/2141950133,v1:172.21.15.119:6805/2141950133] exists,up 56b5818e-e625-40f1-8838-3ddcf2d34ea7 2023-12-07T10:19:51.639 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 up in weight 1 up_from 13 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6800/3390875370,v1:172.21.15.153:6801/3390875370] [v2:172.21.15.153:6802/3390875370,v1:172.21.15.153:6803/3390875370] exists,up 36489a6a-5278-4802-9525-9871521722e6 2023-12-07T10:19:51.639 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 up in weight 1 up_from 18 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6800/708101637,v1:172.21.15.159:6801/708101637] [v2:172.21.15.159:6802/708101637,v1:172.21.15.159:6803/708101637] exists,up a937ad26-5e9b-4c8a-b65a-626a8feda6fa 2023-12-07T10:19:51.640 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 up in weight 1 up_from 26 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6810/3228047257,v1:172.21.15.119:6811/3228047257] [v2:172.21.15.119:6812/3228047257,v1:172.21.15.119:6813/3228047257] exists,up 26ef8300-aa07-48f6-b31d-8b0f2217ffcc 2023-12-07T10:19:51.640 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 up in weight 1 up_from 34 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6808/4021998299,v1:172.21.15.153:6809/4021998299] [v2:172.21.15.153:6810/4021998299,v1:172.21.15.153:6811/4021998299] exists,up 000ee4b9-9896-4901-8cc3-c85fdfc8cbaf 2023-12-07T10:19:51.640 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 up in weight 1 up_from 39 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6808/1832540768,v1:172.21.15.159:6809/1832540768] [v2:172.21.15.159:6810/1832540768,v1:172.21.15.159:6811/1832540768] exists,up 8b2fc167-262d-474f-89f3-47bd29424421 2023-12-07T10:19:51.640 INFO:teuthology.orchestra.run.smithi119.stdout:osd.6 up in weight 1 up_from 47 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6818/1162436840,v1:172.21.15.119:6819/1162436840] [v2:172.21.15.119:6820/1162436840,v1:172.21.15.119:6821/1162436840] exists,up f9aa6749-46cf-4b2a-8114-cbd034219d30 2023-12-07T10:19:51.640 INFO:teuthology.orchestra.run.smithi119.stdout:osd.7 up in weight 1 up_from 52 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6816/3254653457,v1:172.21.15.153:6817/3254653457] [v2:172.21.15.153:6818/3254653457,v1:172.21.15.153:6819/3254653457] exists,up 39216361-e1c4-4fef-86cb-a4707656b547 2023-12-07T10:19:51.640 INFO:teuthology.orchestra.run.smithi119.stdout:osd.8 up in weight 1 up_from 59 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6816/2600946380,v1:172.21.15.159:6817/2600946380] [v2:172.21.15.159:6818/2600946380,v1:172.21.15.159:6819/2600946380] exists,up cceb6ebc-7c02-48ea-9d16-b7a016dd449b 2023-12-07T10:19:51.640 INFO:teuthology.orchestra.run.smithi119.stdout:osd.9 up in weight 1 up_from 67 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6826/1940737285,v1:172.21.15.119:6827/1940737285] [v2:172.21.15.119:6828/1940737285,v1:172.21.15.119:6829/1940737285] exists,up f337a2e4-e065-4a9f-91d9-e3615de2d6f8 2023-12-07T10:19:51.640 INFO:teuthology.orchestra.run.smithi119.stdout:osd.10 up in weight 1 up_from 72 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.153:6824/663834864,v1:172.21.15.153:6825/663834864] [v2:172.21.15.153:6826/663834864,v1:172.21.15.153:6827/663834864] exists,up 3a3bc347-85f4-4878-9797-d9da96137e56 2023-12-07T10:19:51.640 INFO:teuthology.orchestra.run.smithi119.stdout:osd.11 up in weight 1 up_from 80 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6824/1262216348,v1:172.21.15.159:6825/1262216348] [v2:172.21.15.159:6826/1262216348,v1:172.21.15.159:6827/1262216348] exists,up a0fc5909-279d-4172-977c-2dc2da0a7e34 2023-12-07T10:19:51.640 INFO:teuthology.orchestra.run.smithi119.stdout:pg_upmap_items 2.5 [8,11] 2023-12-07T10:19:51.640 INFO:teuthology.orchestra.run.smithi119.stdout:pg_upmap_items 2.6 [8,11] 2023-12-07T10:19:51.640 INFO:teuthology.orchestra.run.smithi119.stdout:pg_upmap_items 2.10 [2,4] 2023-12-07T10:19:51.640 INFO:teuthology.orchestra.run.smithi119.stdout:pg_upmap_items 2.1d [2,1] 2023-12-07T10:19:51.640 INFO:teuthology.orchestra.run.smithi119.stdout:pg_upmap_items 3.e [4,8] 2023-12-07T10:19:51.640 INFO:teuthology.orchestra.run.smithi119.stdout:pg_upmap_items 3.17 [4,6,5,7] 2023-12-07T10:19:51.640 INFO:teuthology.orchestra.run.smithi119.stdout:pg_upmap_items 3.2d [3,9] 2023-12-07T10:19:51.641 INFO:teuthology.orchestra.run.smithi119.stdout:pg_upmap_items 3.38 [3,9] 2023-12-07T10:19:51.641 INFO:teuthology.orchestra.run.smithi119.stdout:pg_upmap_items 3.3c [4,9] 2023-12-07T10:19:51.641 INFO:teuthology.orchestra.run.smithi119.stdout:pg_upmap_items 4.a [5,8] 2023-12-07T10:19:51.641 INFO:teuthology.orchestra.run.smithi119.stdout:pg_upmap_items 4.f [9,8] 2023-12-07T10:19:51.641 INFO:teuthology.orchestra.run.smithi119.stdout:pg_upmap_items 4.17 [9,8] 2023-12-07T10:19:51.641 INFO:teuthology.orchestra.run.smithi119.stdout:pg_upmap_items 4.1b [9,10] 2023-12-07T10:19:51.641 INFO:teuthology.orchestra.run.smithi119.stdout:blocklist 172.21.15.119:0/1628639392 expires 2023-12-08T10:11:00.585565+0000 2023-12-07T10:19:51.641 INFO:teuthology.orchestra.run.smithi119.stdout:blocklist 172.21.15.119:6801/1355617847 expires 2023-12-08T10:11:00.585565+0000 2023-12-07T10:19:51.641 INFO:teuthology.orchestra.run.smithi119.stdout:blocklist 172.21.15.119:6800/3515858087 expires 2023-12-08T10:10:32.177930+0000 2023-12-07T10:19:51.641 INFO:teuthology.orchestra.run.smithi119.stdout:blocklist 172.21.15.119:0/362307836 expires 2023-12-08T10:10:32.177930+0000 2023-12-07T10:19:51.641 INFO:teuthology.orchestra.run.smithi119.stdout:blocklist 172.21.15.119:0/128964558 expires 2023-12-08T10:11:00.585565+0000 2023-12-07T10:19:51.641 INFO:teuthology.orchestra.run.smithi119.stdout:blocklist 172.21.15.119:0/2278813554 expires 2023-12-08T10:10:32.177930+0000 2023-12-07T10:19:51.641 INFO:teuthology.orchestra.run.smithi119.stdout:blocklist 172.21.15.119:6801/3515858087 expires 2023-12-08T10:10:32.177930+0000 2023-12-07T10:19:51.641 INFO:teuthology.orchestra.run.smithi119.stdout:blocklist 172.21.15.119:0/1411933614 expires 2023-12-08T10:11:00.585565+0000 2023-12-07T10:19:51.641 INFO:teuthology.orchestra.run.smithi119.stdout:blocklist 172.21.15.119:0/1685698277 expires 2023-12-08T10:10:32.177930+0000 2023-12-07T10:19:51.642 INFO:teuthology.orchestra.run.smithi119.stdout:blocklist 172.21.15.119:6800/1355617847 expires 2023-12-08T10:11:00.585565+0000 2023-12-07T10:19:51.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.639+0000 7f05b3c2d700 1 -- 172.21.15.119:0/4088625684 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f05980a1b90 msgr2=0x7f05980a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:51.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.639+0000 7f05b3c2d700 1 --2- 172.21.15.119:0/4088625684 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f05980a1b90 0x7f05980a4050 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f05ac1026c0 tx=0x7f05a8009260 comp rx=0 tx=0).stop 2023-12-07T10:19:51.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.639+0000 7f05b3c2d700 1 -- 172.21.15.119:0/4088625684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05ac103200 msgr2=0x7f05ac1a0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:51.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.639+0000 7f05b3c2d700 1 --2- 172.21.15.119:0/4088625684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05ac103200 0x7f05ac1a0de0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f05a4008e40 tx=0x7f05a4008e70 comp rx=0 tx=0).stop 2023-12-07T10:19:51.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.639+0000 7f05b3c2d700 1 -- 172.21.15.119:0/4088625684 shutdown_connections 2023-12-07T10:19:51.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.639+0000 7f05b3c2d700 1 --2- 172.21.15.119:0/4088625684 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f05980a1b90 0x7f05980a4050 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:51.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.639+0000 7f05b3c2d700 1 --2- 172.21.15.119:0/4088625684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05ac103200 0x7f05ac1a0de0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:51.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.639+0000 7f05b3c2d700 1 --2- 172.21.15.119:0/4088625684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05ac101660 0x7f05ac199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:51.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.639+0000 7f05b3c2d700 1 --2- 172.21.15.119:0/4088625684 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05ac102860 0x7f05ac199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:51.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.639+0000 7f05b3c2d700 1 -- 172.21.15.119:0/4088625684 >> 172.21.15.119:0/4088625684 conn(0x7f05ac0fcc20 msgr2=0x7f05ac073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:51.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.639+0000 7f05b3c2d700 1 -- 172.21.15.119:0/4088625684 shutdown_connections 2023-12-07T10:19:51.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:51.639+0000 7f05b3c2d700 1 -- 172.21.15.119:0/4088625684 wait complete. 2023-12-07T10:19:52.150 INFO:teuthology.run_tasks:Running task fs.ready... 2023-12-07T10:19:52.164 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-07T10:19:52.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:51 smithi159 ceph-mon[134726]: pgmap v348: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-07T10:19:52.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:51 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/4088625684' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-12-07T10:19:52.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:51 smithi153 ceph-mon[133625]: pgmap v348: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-07T10:19:52.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:51 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/4088625684' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-12-07T10:19:52.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.364+0000 7fba547ff700 1 -- 172.21.15.119:0/3984470138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba4c107210 msgr2=0x7fba4c109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:52.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.364+0000 7fba547ff700 1 --2- 172.21.15.119:0/3984470138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba4c107210 0x7fba4c109680 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fba4400b0a0 tx=0x7fba4400b3b0 comp rx=0 tx=0).stop 2023-12-07T10:19:52.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.364+0000 7fba547ff700 1 -- 172.21.15.119:0/3984470138 shutdown_connections 2023-12-07T10:19:52.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.364+0000 7fba547ff700 1 --2- 172.21.15.119:0/3984470138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba4c107210 0x7fba4c109680 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:52.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.364+0000 7fba547ff700 1 --2- 172.21.15.119:0/3984470138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba4c101fb0 0x7fba4c1043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:52.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.364+0000 7fba547ff700 1 --2- 172.21.15.119:0/3984470138 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba4c1048e0 0x7fba4c106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:52.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.364+0000 7fba547ff700 1 -- 172.21.15.119:0/3984470138 >> 172.21.15.119:0/3984470138 conn(0x7fba4c0fb9c0 msgr2=0x7fba4c0fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:52.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.365+0000 7fba547ff700 1 -- 172.21.15.119:0/3984470138 shutdown_connections 2023-12-07T10:19:52.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.365+0000 7fba547ff700 1 -- 172.21.15.119:0/3984470138 wait complete. 2023-12-07T10:19:52.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.365+0000 7fba547ff700 1 Processor -- start 2023-12-07T10:19:52.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.366+0000 7fba547ff700 1 -- start start 2023-12-07T10:19:52.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.366+0000 7fba547ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba4c101fb0 0x7fba4c1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:52.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.366+0000 7fba547ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba4c1048e0 0x7fba4c199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:52.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.366+0000 7fba547ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba4c107210 0x7fba4c1a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:52.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.366+0000 7fba547ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba4c1a12f0 con 0x7fba4c1048e0 2023-12-07T10:19:52.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.366+0000 7fba547ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba4c1a1430 con 0x7fba4c107210 2023-12-07T10:19:52.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.366+0000 7fba547ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba4c1a15c0 con 0x7fba4c101fb0 2023-12-07T10:19:52.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.366+0000 7fba51d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba4c1048e0 0x7fba4c199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:52.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.366+0000 7fba5259b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba4c101fb0 0x7fba4c1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:52.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.366+0000 7fba51d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba4c1048e0 0x7fba4c199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33270/0 (socket says 172.21.15.119:33270) 2023-12-07T10:19:52.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.366+0000 7fba51d9a700 1 -- 172.21.15.119:0/3768020622 learned_addr learned my addr 172.21.15.119:0/3768020622 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:52.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.366+0000 7fba52d9c700 1 --2- 172.21.15.119:0/3768020622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba4c107210 0x7fba4c1a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:52.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.367+0000 7fba51d9a700 1 -- 172.21.15.119:0/3768020622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba4c101fb0 msgr2=0x7fba4c1997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:52.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.367+0000 7fba51d9a700 1 --2- 172.21.15.119:0/3768020622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba4c101fb0 0x7fba4c1997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:52.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.367+0000 7fba51d9a700 1 -- 172.21.15.119:0/3768020622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba4c107210 msgr2=0x7fba4c1a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:52.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.367+0000 7fba51d9a700 1 --2- 172.21.15.119:0/3768020622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba4c107210 0x7fba4c1a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:52.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.367+0000 7fba51d9a700 1 -- 172.21.15.119:0/3768020622 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba44009d00 con 0x7fba4c1048e0 2023-12-07T10:19:52.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.367+0000 7fba51d9a700 1 --2- 172.21.15.119:0/3768020622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba4c1048e0 0x7fba4c199d00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fba3c00e9a0 tx=0x7fba3c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:52.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.367+0000 7fba437fe700 1 -- 172.21.15.119:0/3768020622 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c009790 con 0x7fba4c1048e0 2023-12-07T10:19:52.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.367+0000 7fba547ff700 1 -- 172.21.15.119:0/3768020622 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba4c1a18a0 con 0x7fba4c1048e0 2023-12-07T10:19:52.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.367+0000 7fba437fe700 1 -- 172.21.15.119:0/3768020622 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba3c010430 con 0x7fba4c1048e0 2023-12-07T10:19:52.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.367+0000 7fba437fe700 1 -- 172.21.15.119:0/3768020622 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c018680 con 0x7fba4c1048e0 2023-12-07T10:19:52.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.367+0000 7fba547ff700 1 -- 172.21.15.119:0/3768020622 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba4c1a1db0 con 0x7fba4c1048e0 2023-12-07T10:19:52.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.368+0000 7fba547ff700 1 -- 172.21.15.119:0/3768020622 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba4c108d90 con 0x7fba4c1048e0 2023-12-07T10:19:52.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.370+0000 7fba437fe700 1 -- 172.21.15.119:0/3768020622 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fba3c0105a0 con 0x7fba4c1048e0 2023-12-07T10:19:52.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.371+0000 7fba437fe700 1 --2- 172.21.15.119:0/3768020622 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fba380a1b10 0x7fba380a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:52.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.371+0000 7fba437fe700 1 -- 172.21.15.119:0/3768020622 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fba3c014070 con 0x7fba4c1048e0 2023-12-07T10:19:52.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.372+0000 7fba5259b700 1 --2- 172.21.15.119:0/3768020622 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fba380a1b10 0x7fba380a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:52.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.372+0000 7fba5259b700 1 --2- 172.21.15.119:0/3768020622 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fba380a1b10 0x7fba380a3fd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fba48005950 tx=0x7fba4800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:52.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.375+0000 7fba437fe700 1 -- 172.21.15.119:0/3768020622 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fba3c0522f0 con 0x7fba4c1048e0 2023-12-07T10:19:52.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:51 smithi119 ceph-mon[97908]: pgmap v348: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-07T10:19:52.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:51 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/4088625684' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-12-07T10:19:52.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.576+0000 7fba547ff700 1 -- 172.21.15.119:0/3768020622 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba4c068350 con 0x7fba4c1048e0 2023-12-07T10:19:52.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.578+0000 7fba437fe700 1 -- 172.21.15.119:0/3768020622 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24715 (secure 0 0 0) 0x7fba3c096b70 con 0x7fba4c1048e0 2023-12-07T10:19:52.579 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:19:52.580 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":93,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","created":"2023-12-07T10:09:59.945863+0000","modified":"2023-12-07T10:19:02.800626+0000","last_up_change":"2023-12-07T10:16:50.929427+0000","last_in_change":"2023-12-07T10:16:33.558158+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T10:13:25.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-07T10:17:39.347813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-07T10:17:40.714501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-07T10:17:44.257760+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"56b5818e-e625-40f1-8838-3ddcf2d34ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6803","nonce":2141950133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6805","nonce":2141950133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6809","nonce":2141950133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6807","nonce":2141950133}]},"public_addr":"172.21.15.119:6803/2141950133","cluster_addr":"172.21.15.119:6805/2141950133","heartbeat_back_addr":"172.21.15.119:6809/2141950133","heartbeat_front_addr":"172.21.15.119:6807/2141950133","state":["exists","up"]},{"osd":1,"uuid":"36489a6a-5278-4802-9525-9871521722e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6801","nonce":3390875370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6803","nonce":3390875370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6807","nonce":3390875370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6805","nonce":3390875370}]},"public_addr":"172.21.15.153:6801/3390875370","cluster_addr":"172.21.15.153:6803/3390875370","heartbeat_back_addr":"172.21.15.153:6807/3390875370","heartbeat_front_addr":"172.21.15.153:6805/3390875370","state":["exists","up"]},{"osd":2,"uuid":"a937ad26-5e9b-4c8a-b65a-626a8feda6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6801","nonce":708101637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6803","nonce":708101637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6807","nonce":708101637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6805","nonce":708101637}]},"public_addr":"172.21.15.159:6801/708101637","cluster_addr":"172.21.15.159:6803/708101637","heartbeat_back_addr":"172.21.15.159:6807/708101637","heartbeat_front_addr":"172.21.15.159:6805/708101637","state":["exists","up"]},{"osd":3,"uuid":"26ef8300-aa07-48f6-b31d-8b0f2217ffcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6811","nonce":3228047257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6813","nonce":3228047257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6817","nonce":3228047257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6815","nonce":3228047257}]},"public_addr":"172.21.15.119:6811/3228047257","cluster_addr":"172.21.15.119:6813/3228047257","heartbeat_back_addr":"172.21.15.119:6817/3228047257","heartbeat_front_addr":"172.21.15.119:6815/3228047257","state":["exists","up"]},{"osd":4,"uuid":"000ee4b9-9896-4901-8cc3-c85fdfc8cbaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6809","nonce":4021998299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6811","nonce":4021998299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6815","nonce":4021998299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6813","nonce":4021998299}]},"public_addr":"172.21.15.153:6809/4021998299","cluster_addr":"172.21.15.153:6811/4021998299","heartbeat_back_addr":"172.21.15.153:6815/4021998299","heartbeat_front_addr":"172.21.15.153:6813/4021998299","state":["exists","up"]},{"osd":5,"uuid":"8b2fc167-262d-474f-89f3-47bd29424421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6809","nonce":1832540768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6811","nonce":1832540768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6815","nonce":1832540768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6813","nonce":1832540768}]},"public_addr":"172.21.15.159:6809/1832540768","cluster_addr":"172.21.15.159:6811/1832540768","heartbeat_back_addr":"172.21.15.159:6815/1832540768","heartbeat_front_addr":"172.21.15.159:6813/1832540768","state":["exists","up"]},{"osd":6,"uuid":"f9aa6749-46cf-4b2a-8114-cbd034219d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162436840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162436840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6825","nonce":1162436840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162436840}]},"public_addr":"172.21.15.119:6819/1162436840","cluster_addr":"172.21.15.119:6821/1162436840","heartbeat_back_addr":"172.21.15.119:6825/1162436840","heartbeat_front_addr":"172.21.15.119:6823/1162436840","state":["exists","up"]},{"osd":7,"uuid":"39216361-e1c4-4fef-86cb-a4707656b547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6817","nonce":3254653457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6819","nonce":3254653457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6823","nonce":3254653457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6821","nonce":3254653457}]},"public_addr":"172.21.15.153:6817/3254653457","cluster_addr":"172.21.15.153:6819/3254653457","heartbeat_back_addr":"172.21.15.153:6823/3254653457","heartbeat_front_addr":"172.21.15.153:6821/3254653457","state":["exists","up"]},{"osd":8,"uuid":"cceb6ebc-7c02-48ea-9d16-b7a016dd449b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6817","nonce":2600946380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6819","nonce":2600946380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6823","nonce":2600946380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6821","nonce":2600946380}]},"public_addr":"172.21.15.159:6817/2600946380","cluster_addr":"172.21.15.159:6819/2600946380","heartbeat_back_addr":"172.21.15.159:6823/2600946380","heartbeat_front_addr":"172.21.15.159:6821/2600946380","state":["exists","up"]},{"osd":9,"uuid":"f337a2e4-e065-4a9f-91d9-e3615de2d6f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6827","nonce":1940737285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6829","nonce":1940737285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6833","nonce":1940737285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6831","nonce":1940737285}]},"public_addr":"172.21.15.119:6827/1940737285","cluster_addr":"172.21.15.119:6829/1940737285","heartbeat_back_addr":"172.21.15.119:6833/1940737285","heartbeat_front_addr":"172.21.15.119:6831/1940737285","state":["exists","up"]},{"osd":10,"uuid":"3a3bc347-85f4-4878-9797-d9da96137e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6825","nonce":663834864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6827","nonce":663834864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6831","nonce":663834864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6829","nonce":663834864}]},"public_addr":"172.21.15.153:6825/663834864","cluster_addr":"172.21.15.153:6827/663834864","heartbeat_back_addr":"172.21.15.153:6831/663834864","heartbeat_front_addr":"172.21.15.153:6829/663834864","state":["exists","up"]},{"osd":11,"uuid":"a0fc5909-279d-4172-977c-2dc2da0a7e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6825","nonce":1262216348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6827","nonce":1262216348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6831","nonce":1262216348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6829","nonce":1262216348}]},"public_addr":"172.21.15.159:6825/1262216348","cluster_addr":"172.21.15.159:6827/1262216348","heartbeat_back_addr":"172.21.15.159:6831/1262216348","heartbeat_front_addr":"172.21.15.159:6829/1262216348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:37.488109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:59.543060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:21.381585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:47.112729+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:09.732150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:30.893729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:56.678710+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:19.004256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:40.388818+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:06.391734+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:28.394053+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:49.634348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1628639392":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6801/1355617847":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6800/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/362307836":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/128964558":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/2278813554":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/1411933614":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/1685698277":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6800/1355617847":"2023-12-08T10:11:00.585565+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T10:19:52.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.580+0000 7fba547ff700 1 -- 172.21.15.119:0/3768020622 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fba380a1b10 msgr2=0x7fba380a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:52.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.580+0000 7fba547ff700 1 --2- 172.21.15.119:0/3768020622 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fba380a1b10 0x7fba380a3fd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fba48005950 tx=0x7fba4800b410 comp rx=0 tx=0).stop 2023-12-07T10:19:52.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.580+0000 7fba547ff700 1 -- 172.21.15.119:0/3768020622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba4c1048e0 msgr2=0x7fba4c199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:52.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.580+0000 7fba547ff700 1 --2- 172.21.15.119:0/3768020622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba4c1048e0 0x7fba4c199d00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fba3c00e9a0 tx=0x7fba3c00ecb0 comp rx=0 tx=0).stop 2023-12-07T10:19:52.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.580+0000 7fba547ff700 1 -- 172.21.15.119:0/3768020622 shutdown_connections 2023-12-07T10:19:52.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.580+0000 7fba547ff700 1 --2- 172.21.15.119:0/3768020622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba4c107210 0x7fba4c1a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:52.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.580+0000 7fba547ff700 1 --2- 172.21.15.119:0/3768020622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba4c101fb0 0x7fba4c1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:52.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.580+0000 7fba547ff700 1 --2- 172.21.15.119:0/3768020622 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fba380a1b10 0x7fba380a3fd0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:52.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.580+0000 7fba547ff700 1 --2- 172.21.15.119:0/3768020622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba4c1048e0 0x7fba4c199d00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:52.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.580+0000 7fba547ff700 1 -- 172.21.15.119:0/3768020622 >> 172.21.15.119:0/3768020622 conn(0x7fba4c0fb9c0 msgr2=0x7fba4c101720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:52.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.580+0000 7fba547ff700 1 -- 172.21.15.119:0/3768020622 shutdown_connections 2023-12-07T10:19:52.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.580+0000 7fba547ff700 1 -- 172.21.15.119:0/3768020622 wait complete. 2023-12-07T10:19:52.601 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-07T10:13:25.194366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-07T10:17:39.347813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-07T10:17:40.714501+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-07T10:17:44.257760+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-07T10:19:52.601 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-07T10:19:52.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.800+0000 7f22283ff700 1 -- 172.21.15.119:0/627216193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22201027b0 msgr2=0x7f2220102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:52.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.800+0000 7f22283ff700 1 --2- 172.21.15.119:0/627216193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22201027b0 0x7f2220102c10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f22100099c0 tx=0x7f2210009cd0 comp rx=0 tx=0).stop 2023-12-07T10:19:52.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.800+0000 7f22283ff700 1 -- 172.21.15.119:0/627216193 shutdown_connections 2023-12-07T10:19:52.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.800+0000 7f22283ff700 1 --2- 172.21.15.119:0/627216193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2220103150 0x7f2220109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:52.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.800+0000 7f22283ff700 1 --2- 172.21.15.119:0/627216193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22201015c0 0x7f22201019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:52.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.800+0000 7f22283ff700 1 --2- 172.21.15.119:0/627216193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22201027b0 0x7f2220102c10 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:52.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.800+0000 7f22283ff700 1 -- 172.21.15.119:0/627216193 >> 172.21.15.119:0/627216193 conn(0x7f22200fcb10 msgr2=0x7f22200fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:52.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.801+0000 7f22283ff700 1 -- 172.21.15.119:0/627216193 shutdown_connections 2023-12-07T10:19:52.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.801+0000 7f22283ff700 1 -- 172.21.15.119:0/627216193 wait complete. 2023-12-07T10:19:52.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.801+0000 7f22283ff700 1 Processor -- start 2023-12-07T10:19:52.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.802+0000 7f22283ff700 1 -- start start 2023-12-07T10:19:52.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.802+0000 7f22283ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22201015c0 0x7f2220197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:52.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.802+0000 7f22283ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22201027b0 0x7f2220197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:52.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.802+0000 7f22283ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2220103150 0x7f222019ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:52.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.802+0000 7f22283ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222019f060 con 0x7f2220103150 2023-12-07T10:19:52.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.802+0000 7f22283ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222019f1a0 con 0x7f22201015c0 2023-12-07T10:19:52.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.802+0000 7f22283ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222019f330 con 0x7f22201027b0 2023-12-07T10:19:52.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.802+0000 7f222699c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2220103150 0x7f222019ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:52.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.802+0000 7f222599a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22201027b0 0x7f2220197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:52.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.802+0000 7f222599a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22201027b0 0x7f2220197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:53378/0 (socket says 172.21.15.119:53378) 2023-12-07T10:19:52.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.802+0000 7f222599a700 1 -- 172.21.15.119:0/3955883543 learned_addr learned my addr 172.21.15.119:0/3955883543 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:52.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.802+0000 7f222619b700 1 --2- 172.21.15.119:0/3955883543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22201015c0 0x7f2220197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:52.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.803+0000 7f222699c700 1 -- 172.21.15.119:0/3955883543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22201027b0 msgr2=0x7f2220197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:52.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.803+0000 7f222699c700 1 --2- 172.21.15.119:0/3955883543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22201027b0 0x7f2220197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:52.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.803+0000 7f222699c700 1 -- 172.21.15.119:0/3955883543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22201015c0 msgr2=0x7f2220197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:52.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.803+0000 7f222699c700 1 --2- 172.21.15.119:0/3955883543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22201015c0 0x7f2220197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:52.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.803+0000 7f222699c700 1 -- 172.21.15.119:0/3955883543 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22100095f0 con 0x7f2220103150 2023-12-07T10:19:52.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.803+0000 7f222699c700 1 --2- 172.21.15.119:0/3955883543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2220103150 0x7f222019ea90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f221800e970 tx=0x7f221800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:52.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.803+0000 7f22177fe700 1 -- 172.21.15.119:0/3955883543 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2218004220 con 0x7f2220103150 2023-12-07T10:19:52.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.803+0000 7f22177fe700 1 -- 172.21.15.119:0/3955883543 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2218004cf0 con 0x7f2220103150 2023-12-07T10:19:52.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.803+0000 7f22283ff700 1 -- 172.21.15.119:0/3955883543 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f222019f610 con 0x7f2220103150 2023-12-07T10:19:52.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.803+0000 7f22177fe700 1 -- 172.21.15.119:0/3955883543 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2218005660 con 0x7f2220103150 2023-12-07T10:19:52.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.803+0000 7f22283ff700 1 -- 172.21.15.119:0/3955883543 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222019fb20 con 0x7f2220103150 2023-12-07T10:19:52.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.804+0000 7f22283ff700 1 -- 172.21.15.119:0/3955883543 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2220108a60 con 0x7f2220103150 2023-12-07T10:19:52.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.806+0000 7f22177fe700 1 -- 172.21.15.119:0/3955883543 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f2218004750 con 0x7f2220103150 2023-12-07T10:19:52.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.807+0000 7f22177fe700 1 --2- 172.21.15.119:0/3955883543 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f220c0a1c10 0x7f220c0a40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:52.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.807+0000 7f22177fe700 1 -- 172.21.15.119:0/3955883543 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f2218014070 con 0x7f2220103150 2023-12-07T10:19:52.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.808+0000 7f222619b700 1 --2- 172.21.15.119:0/3955883543 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f220c0a1c10 0x7f220c0a40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:52.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.808+0000 7f222619b700 1 --2- 172.21.15.119:0/3955883543 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f220c0a1c10 0x7f220c0a40d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2220102610 tx=0x7f221c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:52.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:52.811+0000 7f22177fe700 1 -- 172.21.15.119:0/3955883543 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f22180966f0 con 0x7f2220103150 2023-12-07T10:19:53.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.013+0000 7f22283ff700 1 -- 172.21.15.119:0/3955883543 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2220108440 con 0x7f2220103150 2023-12-07T10:19:53.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.014+0000 7f22177fe700 1 -- 172.21.15.119:0/3955883543 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2218004e60 con 0x7f2220103150 2023-12-07T10:19:53.015 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-12-07T10:19:53.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.016+0000 7f22283ff700 1 -- 172.21.15.119:0/3955883543 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f220c0a1c10 msgr2=0x7f220c0a40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:53.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.016+0000 7f22283ff700 1 --2- 172.21.15.119:0/3955883543 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f220c0a1c10 0x7f220c0a40d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2220102610 tx=0x7f221c00b040 comp rx=0 tx=0).stop 2023-12-07T10:19:53.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.016+0000 7f22283ff700 1 -- 172.21.15.119:0/3955883543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2220103150 msgr2=0x7f222019ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:53.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.016+0000 7f22283ff700 1 --2- 172.21.15.119:0/3955883543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2220103150 0x7f222019ea90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f221800e970 tx=0x7f221800ec80 comp rx=0 tx=0).stop 2023-12-07T10:19:53.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.016+0000 7f22283ff700 1 -- 172.21.15.119:0/3955883543 shutdown_connections 2023-12-07T10:19:53.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.016+0000 7f22283ff700 1 --2- 172.21.15.119:0/3955883543 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f220c0a1c10 0x7f220c0a40d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:53.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.016+0000 7f22283ff700 1 --2- 172.21.15.119:0/3955883543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2220103150 0x7f222019ea90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:53.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.016+0000 7f22283ff700 1 --2- 172.21.15.119:0/3955883543 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22201015c0 0x7f2220197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:53.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.016+0000 7f22283ff700 1 --2- 172.21.15.119:0/3955883543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22201027b0 0x7f2220197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:53.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.016+0000 7f22283ff700 1 -- 172.21.15.119:0/3955883543 >> 172.21.15.119:0/3955883543 conn(0x7f22200fcb10 msgr2=0x7f2220105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:53.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.017+0000 7f22283ff700 1 -- 172.21.15.119:0/3955883543 shutdown_connections 2023-12-07T10:19:53.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.017+0000 7f22283ff700 1 -- 172.21.15.119:0/3955883543 wait complete. 2023-12-07T10:19:53.036 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-07T10:19:53.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.235+0000 7f70bb7ff700 1 -- 172.21.15.119:0/3891385597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b4103150 msgr2=0x7f70b4109a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:53.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.235+0000 7f70bb7ff700 1 --2- 172.21.15.119:0/3891385597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b4103150 0x7f70b4109a00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f70ac00b0d0 tx=0x7f70ac00b3e0 comp rx=0 tx=0).stop 2023-12-07T10:19:53.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.235+0000 7f70bb7ff700 1 -- 172.21.15.119:0/3891385597 shutdown_connections 2023-12-07T10:19:53.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.235+0000 7f70bb7ff700 1 --2- 172.21.15.119:0/3891385597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b4103150 0x7f70b4109a00 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:53.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.235+0000 7f70bb7ff700 1 --2- 172.21.15.119:0/3891385597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70b41015c0 0x7f70b41019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:53.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.235+0000 7f70bb7ff700 1 --2- 172.21.15.119:0/3891385597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70b41027b0 0x7f70b4102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:53.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.235+0000 7f70bb7ff700 1 -- 172.21.15.119:0/3891385597 >> 172.21.15.119:0/3891385597 conn(0x7f70b40fcb10 msgr2=0x7f70b40fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:53.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.236+0000 7f70bb7ff700 1 -- 172.21.15.119:0/3891385597 shutdown_connections 2023-12-07T10:19:53.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.236+0000 7f70bb7ff700 1 -- 172.21.15.119:0/3891385597 wait complete. 2023-12-07T10:19:53.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.236+0000 7f70bb7ff700 1 Processor -- start 2023-12-07T10:19:53.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.237+0000 7f70bb7ff700 1 -- start start 2023-12-07T10:19:53.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.237+0000 7f70bb7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70b41015c0 0x7f70b4197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:53.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.237+0000 7f70bb7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70b41027b0 0x7f70b4197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:53.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.237+0000 7f70bb7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b4103150 0x7f70b419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:53.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.237+0000 7f70bb7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b419f190 con 0x7f70b4103150 2023-12-07T10:19:53.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.237+0000 7f70bb7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b419f2d0 con 0x7f70b41027b0 2023-12-07T10:19:53.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.237+0000 7f70bb7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b419f460 con 0x7f70b41015c0 2023-12-07T10:19:53.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.237+0000 7f70b9d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b4103150 0x7f70b419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:53.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.237+0000 7f70b9d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b4103150 0x7f70b419ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50858/0 (socket says 172.21.15.119:50858) 2023-12-07T10:19:53.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.237+0000 7f70b9d9c700 1 -- 172.21.15.119:0/585543184 learned_addr learned my addr 172.21.15.119:0/585543184 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:53.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.237+0000 7f70b959b700 1 --2- 172.21.15.119:0/585543184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70b41015c0 0x7f70b4197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:53.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.237+0000 7f70b8d9a700 1 --2- 172.21.15.119:0/585543184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70b41027b0 0x7f70b4197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:53.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.238+0000 7f70b9d9c700 1 -- 172.21.15.119:0/585543184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70b41015c0 msgr2=0x7f70b4197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:53.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.238+0000 7f70b9d9c700 1 --2- 172.21.15.119:0/585543184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70b41015c0 0x7f70b4197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:53.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.238+0000 7f70b9d9c700 1 -- 172.21.15.119:0/585543184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70b41027b0 msgr2=0x7f70b4197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:53.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.238+0000 7f70b9d9c700 1 --2- 172.21.15.119:0/585543184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70b41027b0 0x7f70b4197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:53.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.238+0000 7f70b9d9c700 1 -- 172.21.15.119:0/585543184 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ac009d00 con 0x7f70b4103150 2023-12-07T10:19:53.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.238+0000 7f70b9d9c700 1 --2- 172.21.15.119:0/585543184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b4103150 0x7f70b419ebc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f70ac007b60 tx=0x7f70ac008e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:53.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.238+0000 7f70aa7fc700 1 -- 172.21.15.119:0/585543184 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac00e040 con 0x7f70b4103150 2023-12-07T10:19:53.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.238+0000 7f70aa7fc700 1 -- 172.21.15.119:0/585543184 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70ac01d930 con 0x7f70b4103150 2023-12-07T10:19:53.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.238+0000 7f70bb7ff700 1 -- 172.21.15.119:0/585543184 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b419f6e0 con 0x7f70b4103150 2023-12-07T10:19:53.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.238+0000 7f70aa7fc700 1 -- 172.21.15.119:0/585543184 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac01b9f0 con 0x7f70b4103150 2023-12-07T10:19:53.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.238+0000 7f70bb7ff700 1 -- 172.21.15.119:0/585543184 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b419fb60 con 0x7f70b4103150 2023-12-07T10:19:53.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.239+0000 7f70bb7ff700 1 -- 172.21.15.119:0/585543184 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b4108970 con 0x7f70b4103150 2023-12-07T10:19:53.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.241+0000 7f70aa7fc700 1 -- 172.21.15.119:0/585543184 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f70ac01daa0 con 0x7f70b4103150 2023-12-07T10:19:53.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.243+0000 7f70aa7fc700 1 --2- 172.21.15.119:0/585543184 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f70a00a1b10 0x7f70a00a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:53.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.244+0000 7f70aa7fc700 1 -- 172.21.15.119:0/585543184 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f70ac0cecd0 con 0x7f70b4103150 2023-12-07T10:19:53.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.244+0000 7f70b959b700 1 --2- 172.21.15.119:0/585543184 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f70a00a1b10 0x7f70a00a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:53.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.244+0000 7f70b959b700 1 --2- 172.21.15.119:0/585543184 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f70a00a1b10 0x7f70a00a3fd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f70b4102610 tx=0x7f70b0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:53.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.247+0000 7f70aa7fc700 1 -- 172.21.15.119:0/585543184 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f70ac097e10 con 0x7f70b4103150 2023-12-07T10:19:53.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:52 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3768020622' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:19:53.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:52 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3768020622' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:19:53.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.448+0000 7f70bb7ff700 1 -- 172.21.15.119:0/585543184 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f70b4108350 con 0x7f70b4103150 2023-12-07T10:19:53.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.448+0000 7f70aa7fc700 1 -- 172.21.15.119:0/585543184 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f70ac09b430 con 0x7f70b4103150 2023-12-07T10:19:53.450 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-12-07T10:19:53.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.451+0000 7f70bb7ff700 1 -- 172.21.15.119:0/585543184 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f70a00a1b10 msgr2=0x7f70a00a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:53.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.451+0000 7f70bb7ff700 1 --2- 172.21.15.119:0/585543184 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f70a00a1b10 0x7f70a00a3fd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f70b4102610 tx=0x7f70b0009260 comp rx=0 tx=0).stop 2023-12-07T10:19:53.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.451+0000 7f70bb7ff700 1 -- 172.21.15.119:0/585543184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b4103150 msgr2=0x7f70b419ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:53.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.451+0000 7f70bb7ff700 1 --2- 172.21.15.119:0/585543184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b4103150 0x7f70b419ebc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f70ac007b60 tx=0x7f70ac008e40 comp rx=0 tx=0).stop 2023-12-07T10:19:53.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.451+0000 7f70bb7ff700 1 -- 172.21.15.119:0/585543184 shutdown_connections 2023-12-07T10:19:53.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.451+0000 7f70bb7ff700 1 --2- 172.21.15.119:0/585543184 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f70a00a1b10 0x7f70a00a3fd0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:53.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.451+0000 7f70bb7ff700 1 --2- 172.21.15.119:0/585543184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b4103150 0x7f70b419ebc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:53.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.451+0000 7f70bb7ff700 1 --2- 172.21.15.119:0/585543184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70b41015c0 0x7f70b4197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:53.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.451+0000 7f70bb7ff700 1 --2- 172.21.15.119:0/585543184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70b41027b0 0x7f70b4197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:53.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.451+0000 7f70bb7ff700 1 -- 172.21.15.119:0/585543184 >> 172.21.15.119:0/585543184 conn(0x7f70b40fcb10 msgr2=0x7f70b4105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:53.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.451+0000 7f70bb7ff700 1 -- 172.21.15.119:0/585543184 shutdown_connections 2023-12-07T10:19:53.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.451+0000 7f70bb7ff700 1 -- 172.21.15.119:0/585543184 wait complete. 2023-12-07T10:19:53.469 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:52 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3768020622' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:19:53.471 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-07T10:19:53.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.669+0000 7efcab3ff700 1 -- 172.21.15.119:0/1345675739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efca41015c0 msgr2=0x7efca41019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:53.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.669+0000 7efcab3ff700 1 --2- 172.21.15.119:0/1345675739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efca41015c0 0x7efca41019e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efca00099c0 tx=0x7efca0009cd0 comp rx=0 tx=0).stop 2023-12-07T10:19:53.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.669+0000 7efcab3ff700 1 -- 172.21.15.119:0/1345675739 shutdown_connections 2023-12-07T10:19:53.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.669+0000 7efcab3ff700 1 --2- 172.21.15.119:0/1345675739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efca4103150 0x7efca4109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:53.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.669+0000 7efcab3ff700 1 --2- 172.21.15.119:0/1345675739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efca41015c0 0x7efca41019e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:53.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.669+0000 7efcab3ff700 1 --2- 172.21.15.119:0/1345675739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efca41027b0 0x7efca4102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:53.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.669+0000 7efcab3ff700 1 -- 172.21.15.119:0/1345675739 >> 172.21.15.119:0/1345675739 conn(0x7efca40fcb10 msgr2=0x7efca40fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:53.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.669+0000 7efcab3ff700 1 -- 172.21.15.119:0/1345675739 shutdown_connections 2023-12-07T10:19:53.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.669+0000 7efcab3ff700 1 -- 172.21.15.119:0/1345675739 wait complete. 2023-12-07T10:19:53.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.670+0000 7efcab3ff700 1 Processor -- start 2023-12-07T10:19:53.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.670+0000 7efcab3ff700 1 -- start start 2023-12-07T10:19:53.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.670+0000 7efcab3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efca41015c0 0x7efca41997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:53.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.671+0000 7efcab3ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efca41027b0 0x7efca4199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:53.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.671+0000 7efcab3ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efca4103150 0x7efca41a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:53.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.671+0000 7efcab3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca41a12d0 con 0x7efca41015c0 2023-12-07T10:19:53.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.671+0000 7efcab3ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca41a1410 con 0x7efca4103150 2023-12-07T10:19:53.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.671+0000 7efcab3ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca41a15a0 con 0x7efca41027b0 2023-12-07T10:19:53.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.671+0000 7efca919b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efca41015c0 0x7efca41997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:53.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.671+0000 7efca919b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efca41015c0 0x7efca41997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50874/0 (socket says 172.21.15.119:50874) 2023-12-07T10:19:53.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.671+0000 7efca919b700 1 -- 172.21.15.119:0/3872992145 learned_addr learned my addr 172.21.15.119:0/3872992145 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:53.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.671+0000 7efca899a700 1 --2- 172.21.15.119:0/3872992145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efca41027b0 0x7efca4199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:53.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.671+0000 7efca999c700 1 --2- 172.21.15.119:0/3872992145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efca4103150 0x7efca41a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:53.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.671+0000 7efca919b700 1 -- 172.21.15.119:0/3872992145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efca41027b0 msgr2=0x7efca4199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:53.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.671+0000 7efca919b700 1 --2- 172.21.15.119:0/3872992145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efca41027b0 0x7efca4199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:53.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.671+0000 7efca919b700 1 -- 172.21.15.119:0/3872992145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efca4103150 msgr2=0x7efca41a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:53.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.671+0000 7efca919b700 1 --2- 172.21.15.119:0/3872992145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efca4103150 0x7efca41a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:53.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.671+0000 7efca919b700 1 -- 172.21.15.119:0/3872992145 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca00095f0 con 0x7efca41015c0 2023-12-07T10:19:53.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.671+0000 7efca919b700 1 --2- 172.21.15.119:0/3872992145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efca41015c0 0x7efca41997f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efca0006010 tx=0x7efca000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:53.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.672+0000 7efc9a7fc700 1 -- 172.21.15.119:0/3872992145 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca001d070 con 0x7efca41015c0 2023-12-07T10:19:53.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.672+0000 7efcab3ff700 1 -- 172.21.15.119:0/3872992145 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca41a1820 con 0x7efca41015c0 2023-12-07T10:19:53.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.672+0000 7efc9a7fc700 1 -- 172.21.15.119:0/3872992145 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efca0005020 con 0x7efca41015c0 2023-12-07T10:19:53.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.672+0000 7efc9a7fc700 1 -- 172.21.15.119:0/3872992145 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca0004b50 con 0x7efca41015c0 2023-12-07T10:19:53.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.672+0000 7efcab3ff700 1 -- 172.21.15.119:0/3872992145 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca41a1be0 con 0x7efca41015c0 2023-12-07T10:19:53.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.673+0000 7efcab3ff700 1 -- 172.21.15.119:0/3872992145 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca40683c0 con 0x7efca41015c0 2023-12-07T10:19:53.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.675+0000 7efc9a7fc700 1 -- 172.21.15.119:0/3872992145 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7efca0005190 con 0x7efca41015c0 2023-12-07T10:19:53.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.676+0000 7efc9a7fc700 1 --2- 172.21.15.119:0/3872992145 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7efc900a1b10 0x7efc900a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:53.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.676+0000 7efc9a7fc700 1 -- 172.21.15.119:0/3872992145 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7efca00ce3d0 con 0x7efca41015c0 2023-12-07T10:19:53.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.676+0000 7efca899a700 1 --2- 172.21.15.119:0/3872992145 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7efc900a1b10 0x7efc900a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:53.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.677+0000 7efca899a700 1 --2- 172.21.15.119:0/3872992145 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7efc900a1b10 0x7efc900a3fd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efc94009c60 tx=0x7efc94009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:53.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.680+0000 7efc9a7fc700 1 -- 172.21.15.119:0/3872992145 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7efca0097510 con 0x7efca41015c0 2023-12-07T10:19:53.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.889+0000 7efcab3ff700 1 -- 172.21.15.119:0/3872992145 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efca41a1fa0 con 0x7efca41015c0 2023-12-07T10:19:53.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.889+0000 7efc9a7fc700 1 -- 172.21.15.119:0/3872992145 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7efca009ab30 con 0x7efca41015c0 2023-12-07T10:19:53.891 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-12-07T10:19:53.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.891+0000 7efcab3ff700 1 -- 172.21.15.119:0/3872992145 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7efc900a1b10 msgr2=0x7efc900a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:53.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.891+0000 7efcab3ff700 1 --2- 172.21.15.119:0/3872992145 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7efc900a1b10 0x7efc900a3fd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efc94009c60 tx=0x7efc94009260 comp rx=0 tx=0).stop 2023-12-07T10:19:53.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.891+0000 7efcab3ff700 1 -- 172.21.15.119:0/3872992145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efca41015c0 msgr2=0x7efca41997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:53.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.891+0000 7efcab3ff700 1 --2- 172.21.15.119:0/3872992145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efca41015c0 0x7efca41997f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efca0006010 tx=0x7efca000f690 comp rx=0 tx=0).stop 2023-12-07T10:19:53.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.891+0000 7efcab3ff700 1 -- 172.21.15.119:0/3872992145 shutdown_connections 2023-12-07T10:19:53.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.891+0000 7efcab3ff700 1 --2- 172.21.15.119:0/3872992145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efca4103150 0x7efca41a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:53.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.891+0000 7efcab3ff700 1 --2- 172.21.15.119:0/3872992145 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7efc900a1b10 0x7efc900a3fd0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:53.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.891+0000 7efcab3ff700 1 --2- 172.21.15.119:0/3872992145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efca41015c0 0x7efca41997f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:53.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.891+0000 7efcab3ff700 1 --2- 172.21.15.119:0/3872992145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efca41027b0 0x7efca4199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:53.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.891+0000 7efcab3ff700 1 -- 172.21.15.119:0/3872992145 >> 172.21.15.119:0/3872992145 conn(0x7efca40fcb10 msgr2=0x7efca4107fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:53.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.892+0000 7efcab3ff700 1 -- 172.21.15.119:0/3872992145 shutdown_connections 2023-12-07T10:19:53.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:53.892+0000 7efcab3ff700 1 -- 172.21.15.119:0/3872992145 wait complete. 2023-12-07T10:19:53.912 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-07T10:19:54.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.113+0000 7f40119ff700 1 -- 172.21.15.119:0/3070045082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f400c103150 msgr2=0x7f400c109a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:54.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.113+0000 7f40119ff700 1 --2- 172.21.15.119:0/3070045082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f400c103150 0x7f400c109a00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f400400b0d0 tx=0x7f400400b3e0 comp rx=0 tx=0).stop 2023-12-07T10:19:54.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.113+0000 7f40119ff700 1 -- 172.21.15.119:0/3070045082 shutdown_connections 2023-12-07T10:19:54.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.113+0000 7f40119ff700 1 --2- 172.21.15.119:0/3070045082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f400c103150 0x7f400c109a00 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:54.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.113+0000 7f40119ff700 1 --2- 172.21.15.119:0/3070045082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f400c1015c0 0x7f400c1019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:54.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.113+0000 7f40119ff700 1 --2- 172.21.15.119:0/3070045082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f400c1027b0 0x7f400c102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:54.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.113+0000 7f40119ff700 1 -- 172.21.15.119:0/3070045082 >> 172.21.15.119:0/3070045082 conn(0x7f400c0fcb10 msgr2=0x7f400c0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:54.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.114+0000 7f40119ff700 1 -- 172.21.15.119:0/3070045082 shutdown_connections 2023-12-07T10:19:54.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.114+0000 7f40119ff700 1 -- 172.21.15.119:0/3070045082 wait complete. 2023-12-07T10:19:54.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.115+0000 7f40119ff700 1 Processor -- start 2023-12-07T10:19:54.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.115+0000 7f40119ff700 1 -- start start 2023-12-07T10:19:54.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.115+0000 7f40119ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f400c1015c0 0x7f400c199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:54.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.115+0000 7f40119ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f400c1027b0 0x7f400c199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:54.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.115+0000 7f40119ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f400c103150 0x7f400c1a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:54.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.115+0000 7f40119ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c1a13c0 con 0x7f400c103150 2023-12-07T10:19:54.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.115+0000 7f40119ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c1a1500 con 0x7f400c1027b0 2023-12-07T10:19:54.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.115+0000 7f40119ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c1a1690 con 0x7f400c1015c0 2023-12-07T10:19:54.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.115+0000 7f400affd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f400c1015c0 0x7f400c199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:54.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.115+0000 7f400b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f400c103150 0x7f400c1a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:54.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.115+0000 7f400a7fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f400c1027b0 0x7f400c199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:54.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.115+0000 7f400affd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f400c1015c0 0x7f400c199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:53436/0 (socket says 172.21.15.119:53436) 2023-12-07T10:19:54.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.115+0000 7f400affd700 1 -- 172.21.15.119:0/8176276 learned_addr learned my addr 172.21.15.119:0/8176276 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:54.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.116+0000 7f400affd700 1 -- 172.21.15.119:0/8176276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f400c1027b0 msgr2=0x7f400c199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:54.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.116+0000 7f400affd700 1 --2- 172.21.15.119:0/8176276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f400c1027b0 0x7f400c199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:54.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.116+0000 7f400affd700 1 -- 172.21.15.119:0/8176276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f400c103150 msgr2=0x7f400c1a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:54.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.116+0000 7f400affd700 1 --2- 172.21.15.119:0/8176276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f400c103150 0x7f400c1a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:54.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.116+0000 7f400affd700 1 -- 172.21.15.119:0/8176276 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4004009d00 con 0x7f400c1015c0 2023-12-07T10:19:54.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.116+0000 7f400b7fe700 1 --2- 172.21.15.119:0/8176276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f400c103150 0x7f400c1a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:54.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.116+0000 7f400a7fc700 1 --2- 172.21.15.119:0/8176276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f400c1027b0 0x7f400c199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:54.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.116+0000 7f400affd700 1 --2- 172.21.15.119:0/8176276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f400c1015c0 0x7f400c199890 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00e970 tx=0x7f3ffc00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:54.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.116+0000 7f40109fd700 1 -- 172.21.15.119:0/8176276 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc009790 con 0x7f400c1015c0 2023-12-07T10:19:54.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.116+0000 7f40109fd700 1 -- 172.21.15.119:0/8176276 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ffc00c7e0 con 0x7f400c1015c0 2023-12-07T10:19:54.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.116+0000 7f40119ff700 1 -- 172.21.15.119:0/8176276 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f400c1a1970 con 0x7f400c1015c0 2023-12-07T10:19:54.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.116+0000 7f40109fd700 1 -- 172.21.15.119:0/8176276 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc003c70 con 0x7f400c1015c0 2023-12-07T10:19:54.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.116+0000 7f40119ff700 1 -- 172.21.15.119:0/8176276 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f400c1a1ed0 con 0x7f400c1015c0 2023-12-07T10:19:54.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.117+0000 7f40119ff700 1 -- 172.21.15.119:0/8176276 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f400c0683c0 con 0x7f400c1015c0 2023-12-07T10:19:54.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.119+0000 7f40109fd700 1 -- 172.21.15.119:0/8176276 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f3ffc00cc50 con 0x7f400c1015c0 2023-12-07T10:19:54.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.121+0000 7f40109fd700 1 --2- 172.21.15.119:0/8176276 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3ff40a1b90 0x7f3ff40a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:54.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.121+0000 7f40109fd700 1 -- 172.21.15.119:0/8176276 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f3ffc014070 con 0x7f400c1015c0 2023-12-07T10:19:54.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.121+0000 7f400a7fc700 1 --2- 172.21.15.119:0/8176276 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3ff40a1b90 0x7f3ff40a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:54.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.121+0000 7f400a7fc700 1 --2- 172.21.15.119:0/8176276 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3ff40a1b90 0x7f3ff40a4050 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4000011490 tx=0x7f4000009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:54.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.123+0000 7f40109fd700 1 -- 172.21.15.119:0/8176276 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3ffc096ad0 con 0x7f400c1015c0 2023-12-07T10:19:54.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:53 smithi119 ceph-mon[97908]: pgmap v349: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-07T10:19:54.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:53 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3955883543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:19:54.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:53 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/585543184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:19:54.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:53 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3872992145' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:19:54.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:53 smithi159 ceph-mon[134726]: pgmap v349: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-07T10:19:54.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:53 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3955883543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:19:54.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:53 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/585543184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:19:54.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:53 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3872992145' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:19:54.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:53 smithi153 ceph-mon[133625]: pgmap v349: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-07T10:19:54.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:53 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3955883543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:19:54.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:53 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/585543184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:19:54.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:53 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3872992145' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:19:54.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.324+0000 7f40119ff700 1 -- 172.21.15.119:0/8176276 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f400c1a2170 con 0x7f400c1015c0 2023-12-07T10:19:54.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.325+0000 7f40109fd700 1 -- 172.21.15.119:0/8176276 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3ffc09a0f0 con 0x7f400c1015c0 2023-12-07T10:19:54.327 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-12-07T10:19:54.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.328+0000 7f40119ff700 1 -- 172.21.15.119:0/8176276 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3ff40a1b90 msgr2=0x7f3ff40a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:54.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.328+0000 7f40119ff700 1 --2- 172.21.15.119:0/8176276 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3ff40a1b90 0x7f3ff40a4050 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4000011490 tx=0x7f4000009260 comp rx=0 tx=0).stop 2023-12-07T10:19:54.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.328+0000 7f40119ff700 1 -- 172.21.15.119:0/8176276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f400c1015c0 msgr2=0x7f400c199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:54.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.328+0000 7f40119ff700 1 --2- 172.21.15.119:0/8176276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f400c1015c0 0x7f400c199890 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00e970 tx=0x7f3ffc00ec80 comp rx=0 tx=0).stop 2023-12-07T10:19:54.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.328+0000 7f40119ff700 1 -- 172.21.15.119:0/8176276 shutdown_connections 2023-12-07T10:19:54.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.328+0000 7f40119ff700 1 --2- 172.21.15.119:0/8176276 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3ff40a1b90 0x7f3ff40a4050 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:54.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.328+0000 7f40119ff700 1 --2- 172.21.15.119:0/8176276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f400c103150 0x7f400c1a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:54.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.328+0000 7f40119ff700 1 --2- 172.21.15.119:0/8176276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f400c1015c0 0x7f400c199890 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:54.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.328+0000 7f40119ff700 1 --2- 172.21.15.119:0/8176276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f400c1027b0 0x7f400c199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:54.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.328+0000 7f40119ff700 1 -- 172.21.15.119:0/8176276 >> 172.21.15.119:0/8176276 conn(0x7f400c0fcb10 msgr2=0x7f400c073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:54.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.328+0000 7f40119ff700 1 -- 172.21.15.119:0/8176276 shutdown_connections 2023-12-07T10:19:54.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.328+0000 7f40119ff700 1 -- 172.21.15.119:0/8176276 wait complete. 2023-12-07T10:19:54.348 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:19:54.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.543+0000 7f6a9c9ff700 1 -- 172.21.15.119:0/2451605224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a98102750 msgr2=0x7f6a98102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:54.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.543+0000 7f6a9c9ff700 1 --2- 172.21.15.119:0/2451605224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a98102750 0x7f6a98102bb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6a800099c0 tx=0x7f6a80009cd0 comp rx=0 tx=0).stop 2023-12-07T10:19:54.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.543+0000 7f6a9c9ff700 1 -- 172.21.15.119:0/2451605224 shutdown_connections 2023-12-07T10:19:54.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.543+0000 7f6a9c9ff700 1 --2- 172.21.15.119:0/2451605224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a981030f0 0x7f6a981099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:54.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.543+0000 7f6a9c9ff700 1 --2- 172.21.15.119:0/2451605224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a98101560 0x7f6a98101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:54.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.543+0000 7f6a9c9ff700 1 --2- 172.21.15.119:0/2451605224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a98102750 0x7f6a98102bb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:54.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.543+0000 7f6a9c9ff700 1 -- 172.21.15.119:0/2451605224 >> 172.21.15.119:0/2451605224 conn(0x7f6a980fcab0 msgr2=0x7f6a980fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:54.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.543+0000 7f6a9c9ff700 1 -- 172.21.15.119:0/2451605224 shutdown_connections 2023-12-07T10:19:54.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.543+0000 7f6a9c9ff700 1 -- 172.21.15.119:0/2451605224 wait complete. 2023-12-07T10:19:54.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.544+0000 7f6a9c9ff700 1 Processor -- start 2023-12-07T10:19:54.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.545+0000 7f6a9c9ff700 1 -- start start 2023-12-07T10:19:54.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.545+0000 7f6a9c9ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a98101560 0x7f6a981996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:54.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.545+0000 7f6a9c9ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a98102750 0x7f6a98199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:54.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.545+0000 7f6a9c9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a981030f0 0x7f6a981a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:54.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.545+0000 7f6a9c9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a981a11f0 con 0x7f6a981030f0 2023-12-07T10:19:54.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.545+0000 7f6a9c9ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a981a1330 con 0x7f6a98101560 2023-12-07T10:19:54.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.545+0000 7f6a9c9ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a981a14c0 con 0x7f6a98102750 2023-12-07T10:19:54.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.545+0000 7f6a96d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a981030f0 0x7f6a981a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:54.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.545+0000 7f6a9659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a98101560 0x7f6a981996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:54.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.545+0000 7f6a96d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a981030f0 0x7f6a981a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50912/0 (socket says 172.21.15.119:50912) 2023-12-07T10:19:54.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.545+0000 7f6a96d9d700 1 -- 172.21.15.119:0/2336569439 learned_addr learned my addr 172.21.15.119:0/2336569439 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:54.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.545+0000 7f6a8ffff700 1 --2- 172.21.15.119:0/2336569439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a98102750 0x7f6a98199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:54.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.546+0000 7f6a96d9d700 1 -- 172.21.15.119:0/2336569439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a98102750 msgr2=0x7f6a98199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:54.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.546+0000 7f6a96d9d700 1 --2- 172.21.15.119:0/2336569439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a98102750 0x7f6a98199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:54.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.546+0000 7f6a96d9d700 1 -- 172.21.15.119:0/2336569439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a98101560 msgr2=0x7f6a981996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:54.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.546+0000 7f6a96d9d700 1 --2- 172.21.15.119:0/2336569439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a98101560 0x7f6a981996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:54.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.546+0000 7f6a96d9d700 1 -- 172.21.15.119:0/2336569439 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a90009c40 con 0x7f6a981030f0 2023-12-07T10:19:54.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.546+0000 7f6a96d9d700 1 --2- 172.21.15.119:0/2336569439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a981030f0 0x7f6a981a0c20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000c370 tx=0x7f6a9000c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:54.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.546+0000 7f6a8f7fe700 1 -- 172.21.15.119:0/2336569439 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a90017070 con 0x7f6a981030f0 2023-12-07T10:19:54.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.546+0000 7f6a8f7fe700 1 -- 172.21.15.119:0/2336569439 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a9000f040 con 0x7f6a981030f0 2023-12-07T10:19:54.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.546+0000 7f6a9c9ff700 1 -- 172.21.15.119:0/2336569439 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a800095f0 con 0x7f6a981030f0 2023-12-07T10:19:54.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.546+0000 7f6a8f7fe700 1 -- 172.21.15.119:0/2336569439 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9000e050 con 0x7f6a981030f0 2023-12-07T10:19:54.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.546+0000 7f6a9c9ff700 1 -- 172.21.15.119:0/2336569439 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a981a1a70 con 0x7f6a981030f0 2023-12-07T10:19:54.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.547+0000 7f6a9c9ff700 1 -- 172.21.15.119:0/2336569439 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a98068350 con 0x7f6a981030f0 2023-12-07T10:19:54.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.549+0000 7f6a8f7fe700 1 -- 172.21.15.119:0/2336569439 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f6a9000cb60 con 0x7f6a981030f0 2023-12-07T10:19:54.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.551+0000 7f6a8f7fe700 1 --2- 172.21.15.119:0/2336569439 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6a840a1b10 0x7f6a840a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:54.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.551+0000 7f6a8f7fe700 1 -- 172.21.15.119:0/2336569439 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f6a900cd300 con 0x7f6a981030f0 2023-12-07T10:19:54.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.551+0000 7f6a9659c700 1 --2- 172.21.15.119:0/2336569439 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6a840a1b10 0x7f6a840a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:54.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.551+0000 7f6a9659c700 1 --2- 172.21.15.119:0/2336569439 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6a840a1b10 0x7f6a840a3fd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6a981025b0 tx=0x7f6a8800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:54.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.554+0000 7f6a8f7fe700 1 -- 172.21.15.119:0/2336569439 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f6a90096350 con 0x7f6a981030f0 2023-12-07T10:19:54.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.800+0000 7f6a9c9ff700 1 -- 172.21.15.119:0/2336569439 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a9810cd80 con 0x7f6a981030f0 2023-12-07T10:19:54.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.801+0000 7f6a8f7fe700 1 -- 172.21.15.119:0/2336569439 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11385 (secure 0 0 0) 0x7f6a90099970 con 0x7f6a981030f0 2023-12-07T10:19:54.802 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:19:54.802 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-07T10:17:41.278311+0000","modified":"2023-12-07T10:18:18.543356+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24517,"mds_3":24505,"mds_4":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14535":{"gid":14535,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6841/1791998028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1791998028},{"type":"v1","addr":"172.21.15.119:6841","nonce":1791998028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24505":{"gid":24505,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/2925503246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2925503246},{"type":"v1","addr":"172.21.15.153:6837","nonce":2925503246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6839/3984962900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3984962900},{"type":"v1","addr":"172.21.15.159:6839","nonce":3984962900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.119:6839/1920762602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1920762602},{"type":"v1","addr":"172.21.15.119:6839","nonce":1920762602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/566825184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":566825184},{"type":"v1","addr":"172.21.15.153:6839","nonce":566825184}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-07T10:19:54.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.803+0000 7f6a9c9ff700 1 -- 172.21.15.119:0/2336569439 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6a840a1b10 msgr2=0x7f6a840a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:54.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.803+0000 7f6a9c9ff700 1 --2- 172.21.15.119:0/2336569439 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6a840a1b10 0x7f6a840a3fd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6a981025b0 tx=0x7f6a8800b410 comp rx=0 tx=0).stop 2023-12-07T10:19:54.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.803+0000 7f6a9c9ff700 1 -- 172.21.15.119:0/2336569439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a981030f0 msgr2=0x7f6a981a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:54.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.803+0000 7f6a9c9ff700 1 --2- 172.21.15.119:0/2336569439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a981030f0 0x7f6a981a0c20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000c370 tx=0x7f6a9000c680 comp rx=0 tx=0).stop 2023-12-07T10:19:54.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.803+0000 7f6a9c9ff700 1 -- 172.21.15.119:0/2336569439 shutdown_connections 2023-12-07T10:19:54.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.803+0000 7f6a9c9ff700 1 --2- 172.21.15.119:0/2336569439 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6a840a1b10 0x7f6a840a3fd0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:54.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.803+0000 7f6a9c9ff700 1 --2- 172.21.15.119:0/2336569439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a981030f0 0x7f6a981a0c20 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:54.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.803+0000 7f6a9c9ff700 1 --2- 172.21.15.119:0/2336569439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a98101560 0x7f6a981996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:54.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.803+0000 7f6a9c9ff700 1 --2- 172.21.15.119:0/2336569439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a98102750 0x7f6a98199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:54.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.803+0000 7f6a9c9ff700 1 -- 172.21.15.119:0/2336569439 >> 172.21.15.119:0/2336569439 conn(0x7f6a980fcab0 msgr2=0x7f6a98073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:54.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.803+0000 7f6a9c9ff700 1 -- 172.21.15.119:0/2336569439 shutdown_connections 2023-12-07T10:19:54.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:54.803+0000 7f6a9c9ff700 1 -- 172.21.15.119:0/2336569439 wait complete. 2023-12-07T10:19:54.806 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 29 2023-12-07T10:19:54.824 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-07T10:19:55.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.023+0000 7f5e211ff700 1 -- 172.21.15.119:0/4067026957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e1c1030f0 msgr2=0x7f5e1c1099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:55.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.023+0000 7f5e211ff700 1 --2- 172.21.15.119:0/4067026957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e1c1030f0 0x7f5e1c1099a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400b0d0 tx=0x7f5e1400b3e0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.023+0000 7f5e211ff700 1 -- 172.21.15.119:0/4067026957 shutdown_connections 2023-12-07T10:19:55.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.023+0000 7f5e211ff700 1 --2- 172.21.15.119:0/4067026957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e1c1030f0 0x7f5e1c1099a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.023+0000 7f5e211ff700 1 --2- 172.21.15.119:0/4067026957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e1c101560 0x7f5e1c101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.023+0000 7f5e211ff700 1 --2- 172.21.15.119:0/4067026957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e1c102750 0x7f5e1c102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.023+0000 7f5e211ff700 1 -- 172.21.15.119:0/4067026957 >> 172.21.15.119:0/4067026957 conn(0x7f5e1c0fcab0 msgr2=0x7f5e1c0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:55.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.023+0000 7f5e211ff700 1 -- 172.21.15.119:0/4067026957 shutdown_connections 2023-12-07T10:19:55.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.023+0000 7f5e211ff700 1 -- 172.21.15.119:0/4067026957 wait complete. 2023-12-07T10:19:55.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.024+0000 7f5e211ff700 1 Processor -- start 2023-12-07T10:19:55.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.024+0000 7f5e211ff700 1 -- start start 2023-12-07T10:19:55.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.024+0000 7f5e211ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e1c101560 0x7f5e1c1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:55.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.024+0000 7f5e211ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e1c102750 0x7f5e1c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:55.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.024+0000 7f5e211ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e1c1030f0 0x7f5e1c19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:55.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.024+0000 7f5e211ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c19f110 con 0x7f5e1c1030f0 2023-12-07T10:19:55.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.024+0000 7f5e211ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c19f250 con 0x7f5e1c101560 2023-12-07T10:19:55.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.024+0000 7f5e211ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c19f3e0 con 0x7f5e1c102750 2023-12-07T10:19:55.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.025+0000 7f5e1b59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e1c1030f0 0x7f5e1c19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:55.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.025+0000 7f5e1b59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e1c1030f0 0x7f5e1c19eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50932/0 (socket says 172.21.15.119:50932) 2023-12-07T10:19:55.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.025+0000 7f5e1b59e700 1 -- 172.21.15.119:0/375869703 learned_addr learned my addr 172.21.15.119:0/375869703 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:55.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.025+0000 7f5e1ad9d700 1 --2- 172.21.15.119:0/375869703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e1c101560 0x7f5e1c1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:55.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.025+0000 7f5e1a59c700 1 --2- 172.21.15.119:0/375869703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e1c102750 0x7f5e1c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:55.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.025+0000 7f5e1b59e700 1 -- 172.21.15.119:0/375869703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e1c102750 msgr2=0x7f5e1c197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:55.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.025+0000 7f5e1b59e700 1 --2- 172.21.15.119:0/375869703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e1c102750 0x7f5e1c197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.025+0000 7f5e1b59e700 1 -- 172.21.15.119:0/375869703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e1c101560 msgr2=0x7f5e1c1975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:55.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.025+0000 7f5e1b59e700 1 --2- 172.21.15.119:0/375869703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e1c101560 0x7f5e1c1975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.025+0000 7f5e1b59e700 1 -- 172.21.15.119:0/375869703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e14009d00 con 0x7f5e1c1030f0 2023-12-07T10:19:55.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.025+0000 7f5e1b59e700 1 --2- 172.21.15.119:0/375869703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e1c1030f0 0x7f5e1c19eb40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5e14007b60 tx=0x7f5e14008e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:55.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.026+0000 7f5e03fff700 1 -- 172.21.15.119:0/375869703 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1400e040 con 0x7f5e1c1030f0 2023-12-07T10:19:55.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.026+0000 7f5e211ff700 1 -- 172.21.15.119:0/375869703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e1c19f660 con 0x7f5e1c1030f0 2023-12-07T10:19:55.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.026+0000 7f5e03fff700 1 -- 172.21.15.119:0/375869703 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e1401d930 con 0x7f5e1c1030f0 2023-12-07T10:19:55.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.026+0000 7f5e03fff700 1 -- 172.21.15.119:0/375869703 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1401b9f0 con 0x7f5e1c1030f0 2023-12-07T10:19:55.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.026+0000 7f5e211ff700 1 -- 172.21.15.119:0/375869703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e1c19fae0 con 0x7f5e1c1030f0 2023-12-07T10:19:55.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.027+0000 7f5e211ff700 1 -- 172.21.15.119:0/375869703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e1c108910 con 0x7f5e1c1030f0 2023-12-07T10:19:55.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.029+0000 7f5e03fff700 1 -- 172.21.15.119:0/375869703 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f5e1401daa0 con 0x7f5e1c1030f0 2023-12-07T10:19:55.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.030+0000 7f5e03fff700 1 --2- 172.21.15.119:0/375869703 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5e040a1b10 0x7f5e040a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:55.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.030+0000 7f5e03fff700 1 -- 172.21.15.119:0/375869703 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f5e140cedd0 con 0x7f5e1c1030f0 2023-12-07T10:19:55.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.030+0000 7f5e1ad9d700 1 --2- 172.21.15.119:0/375869703 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5e040a1b10 0x7f5e040a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:55.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.031+0000 7f5e1ad9d700 1 --2- 172.21.15.119:0/375869703 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5e040a1b10 0x7f5e040a3fd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c1025b0 tx=0x7f5e0c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:55.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.034+0000 7f5e03fff700 1 -- 172.21.15.119:0/375869703 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5e14097eb0 con 0x7f5e1c1030f0 2023-12-07T10:19:55.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:55 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/8176276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:19:55.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:55 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2336569439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:19:55.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.237+0000 7f5e211ff700 1 -- 172.21.15.119:0/375869703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e1c10cdd0 con 0x7f5e1c1030f0 2023-12-07T10:19:55.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.239+0000 7f5e03fff700 1 -- 172.21.15.119:0/375869703 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24715 (secure 0 0 0) 0x7f5e1409b4d0 con 0x7f5e1c1030f0 2023-12-07T10:19:55.241 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:19:55.241 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":93,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","created":"2023-12-07T10:09:59.945863+0000","modified":"2023-12-07T10:19:02.800626+0000","last_up_change":"2023-12-07T10:16:50.929427+0000","last_in_change":"2023-12-07T10:16:33.558158+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T10:13:25.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-07T10:17:39.347813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-07T10:17:40.714501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-07T10:17:44.257760+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"56b5818e-e625-40f1-8838-3ddcf2d34ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6803","nonce":2141950133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6805","nonce":2141950133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6809","nonce":2141950133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6807","nonce":2141950133}]},"public_addr":"172.21.15.119:6803/2141950133","cluster_addr":"172.21.15.119:6805/2141950133","heartbeat_back_addr":"172.21.15.119:6809/2141950133","heartbeat_front_addr":"172.21.15.119:6807/2141950133","state":["exists","up"]},{"osd":1,"uuid":"36489a6a-5278-4802-9525-9871521722e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6801","nonce":3390875370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6803","nonce":3390875370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6807","nonce":3390875370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6805","nonce":3390875370}]},"public_addr":"172.21.15.153:6801/3390875370","cluster_addr":"172.21.15.153:6803/3390875370","heartbeat_back_addr":"172.21.15.153:6807/3390875370","heartbeat_front_addr":"172.21.15.153:6805/3390875370","state":["exists","up"]},{"osd":2,"uuid":"a937ad26-5e9b-4c8a-b65a-626a8feda6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6801","nonce":708101637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6803","nonce":708101637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6807","nonce":708101637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6805","nonce":708101637}]},"public_addr":"172.21.15.159:6801/708101637","cluster_addr":"172.21.15.159:6803/708101637","heartbeat_back_addr":"172.21.15.159:6807/708101637","heartbeat_front_addr":"172.21.15.159:6805/708101637","state":["exists","up"]},{"osd":3,"uuid":"26ef8300-aa07-48f6-b31d-8b0f2217ffcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6811","nonce":3228047257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6813","nonce":3228047257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6817","nonce":3228047257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6815","nonce":3228047257}]},"public_addr":"172.21.15.119:6811/3228047257","cluster_addr":"172.21.15.119:6813/3228047257","heartbeat_back_addr":"172.21.15.119:6817/3228047257","heartbeat_front_addr":"172.21.15.119:6815/3228047257","state":["exists","up"]},{"osd":4,"uuid":"000ee4b9-9896-4901-8cc3-c85fdfc8cbaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6809","nonce":4021998299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6811","nonce":4021998299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6815","nonce":4021998299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6813","nonce":4021998299}]},"public_addr":"172.21.15.153:6809/4021998299","cluster_addr":"172.21.15.153:6811/4021998299","heartbeat_back_addr":"172.21.15.153:6815/4021998299","heartbeat_front_addr":"172.21.15.153:6813/4021998299","state":["exists","up"]},{"osd":5,"uuid":"8b2fc167-262d-474f-89f3-47bd29424421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6809","nonce":1832540768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6811","nonce":1832540768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6815","nonce":1832540768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6813","nonce":1832540768}]},"public_addr":"172.21.15.159:6809/1832540768","cluster_addr":"172.21.15.159:6811/1832540768","heartbeat_back_addr":"172.21.15.159:6815/1832540768","heartbeat_front_addr":"172.21.15.159:6813/1832540768","state":["exists","up"]},{"osd":6,"uuid":"f9aa6749-46cf-4b2a-8114-cbd034219d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162436840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162436840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6825","nonce":1162436840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162436840}]},"public_addr":"172.21.15.119:6819/1162436840","cluster_addr":"172.21.15.119:6821/1162436840","heartbeat_back_addr":"172.21.15.119:6825/1162436840","heartbeat_front_addr":"172.21.15.119:6823/1162436840","state":["exists","up"]},{"osd":7,"uuid":"39216361-e1c4-4fef-86cb-a4707656b547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6817","nonce":3254653457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6819","nonce":3254653457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6823","nonce":3254653457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6821","nonce":3254653457}]},"public_addr":"172.21.15.153:6817/3254653457","cluster_addr":"172.21.15.153:6819/3254653457","heartbeat_back_addr":"172.21.15.153:6823/3254653457","heartbeat_front_addr":"172.21.15.153:6821/3254653457","state":["exists","up"]},{"osd":8,"uuid":"cceb6ebc-7c02-48ea-9d16-b7a016dd449b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6817","nonce":2600946380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6819","nonce":2600946380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6823","nonce":2600946380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6821","nonce":2600946380}]},"public_addr":"172.21.15.159:6817/2600946380","cluster_addr":"172.21.15.159:6819/2600946380","heartbeat_back_addr":"172.21.15.159:6823/2600946380","heartbeat_front_addr":"172.21.15.159:6821/2600946380","state":["exists","up"]},{"osd":9,"uuid":"f337a2e4-e065-4a9f-91d9-e3615de2d6f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6827","nonce":1940737285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6829","nonce":1940737285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6833","nonce":1940737285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6831","nonce":1940737285}]},"public_addr":"172.21.15.119:6827/1940737285","cluster_addr":"172.21.15.119:6829/1940737285","heartbeat_back_addr":"172.21.15.119:6833/1940737285","heartbeat_front_addr":"172.21.15.119:6831/1940737285","state":["exists","up"]},{"osd":10,"uuid":"3a3bc347-85f4-4878-9797-d9da96137e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6825","nonce":663834864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6827","nonce":663834864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6831","nonce":663834864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6829","nonce":663834864}]},"public_addr":"172.21.15.153:6825/663834864","cluster_addr":"172.21.15.153:6827/663834864","heartbeat_back_addr":"172.21.15.153:6831/663834864","heartbeat_front_addr":"172.21.15.153:6829/663834864","state":["exists","up"]},{"osd":11,"uuid":"a0fc5909-279d-4172-977c-2dc2da0a7e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6825","nonce":1262216348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6827","nonce":1262216348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6831","nonce":1262216348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6829","nonce":1262216348}]},"public_addr":"172.21.15.159:6825/1262216348","cluster_addr":"172.21.15.159:6827/1262216348","heartbeat_back_addr":"172.21.15.159:6831/1262216348","heartbeat_front_addr":"172.21.15.159:6829/1262216348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:37.488109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:59.543060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:21.381585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:47.112729+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:09.732150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:30.893729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:56.678710+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:19.004256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:40.388818+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:06.391734+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:28.394053+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:49.634348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1628639392":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6801/1355617847":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6800/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/362307836":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/128964558":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/2278813554":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/1411933614":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/1685698277":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6800/1355617847":"2023-12-08T10:11:00.585565+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T10:19:55.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.242+0000 7f5e211ff700 1 -- 172.21.15.119:0/375869703 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5e040a1b10 msgr2=0x7f5e040a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:55.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.242+0000 7f5e211ff700 1 --2- 172.21.15.119:0/375869703 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5e040a1b10 0x7f5e040a3fd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c1025b0 tx=0x7f5e0c009260 comp rx=0 tx=0).stop 2023-12-07T10:19:55.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.242+0000 7f5e211ff700 1 -- 172.21.15.119:0/375869703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e1c1030f0 msgr2=0x7f5e1c19eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:55.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.242+0000 7f5e211ff700 1 --2- 172.21.15.119:0/375869703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e1c1030f0 0x7f5e1c19eb40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5e14007b60 tx=0x7f5e14008e90 comp rx=0 tx=0).stop 2023-12-07T10:19:55.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.242+0000 7f5e211ff700 1 -- 172.21.15.119:0/375869703 shutdown_connections 2023-12-07T10:19:55.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.242+0000 7f5e211ff700 1 --2- 172.21.15.119:0/375869703 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5e040a1b10 0x7f5e040a3fd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.242+0000 7f5e211ff700 1 --2- 172.21.15.119:0/375869703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e1c1030f0 0x7f5e1c19eb40 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.242+0000 7f5e211ff700 1 --2- 172.21.15.119:0/375869703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e1c101560 0x7f5e1c1975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.242+0000 7f5e211ff700 1 --2- 172.21.15.119:0/375869703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e1c102750 0x7f5e1c197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.242+0000 7f5e211ff700 1 -- 172.21.15.119:0/375869703 >> 172.21.15.119:0/375869703 conn(0x7f5e1c0fcab0 msgr2=0x7f5e1c105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:55.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.242+0000 7f5e211ff700 1 -- 172.21.15.119:0/375869703 shutdown_connections 2023-12-07T10:19:55.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.242+0000 7f5e211ff700 1 -- 172.21.15.119:0/375869703 wait complete. 2023-12-07T10:19:55.263 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-07T10:13:25.194366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-07T10:17:39.347813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-07T10:17:40.714501+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-07T10:17:44.257760+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-07T10:19:55.263 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-07T10:19:55.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:55 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/8176276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:19:55.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:55 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2336569439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:19:55.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:55 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/8176276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:19:55.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:55 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2336569439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:19:55.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.462+0000 7fc8f1dff700 1 -- 172.21.15.119:0/248335234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8ec1015f0 msgr2=0x7fc8ec101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:55.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.462+0000 7fc8f1dff700 1 --2- 172.21.15.119:0/248335234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8ec1015f0 0x7fc8ec101a10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0099c0 tx=0x7fc8dc009cd0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.462+0000 7fc8f1dff700 1 -- 172.21.15.119:0/248335234 shutdown_connections 2023-12-07T10:19:55.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.462+0000 7fc8f1dff700 1 --2- 172.21.15.119:0/248335234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8ec103180 0x7fc8ec109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.462+0000 7fc8f1dff700 1 --2- 172.21.15.119:0/248335234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8ec1015f0 0x7fc8ec101a10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.462+0000 7fc8f1dff700 1 --2- 172.21.15.119:0/248335234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8ec1027e0 0x7fc8ec102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.462+0000 7fc8f1dff700 1 -- 172.21.15.119:0/248335234 >> 172.21.15.119:0/248335234 conn(0x7fc8ec0fcb60 msgr2=0x7fc8ec0fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:55.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.462+0000 7fc8f1dff700 1 -- 172.21.15.119:0/248335234 shutdown_connections 2023-12-07T10:19:55.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.462+0000 7fc8f1dff700 1 -- 172.21.15.119:0/248335234 wait complete. 2023-12-07T10:19:55.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.463+0000 7fc8f1dff700 1 Processor -- start 2023-12-07T10:19:55.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.464+0000 7fc8f1dff700 1 -- start start 2023-12-07T10:19:55.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.464+0000 7fc8f1dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8ec1015f0 0x7fc8ec199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:55.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.464+0000 7fc8f1dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8ec1027e0 0x7fc8ec199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:55.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.464+0000 7fc8f1dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8ec103180 0x7fc8ec1a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:55.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.464+0000 7fc8f1dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec1a12b0 con 0x7fc8ec103180 2023-12-07T10:19:55.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.464+0000 7fc8f1dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec1a13f0 con 0x7fc8ec1027e0 2023-12-07T10:19:55.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.464+0000 7fc8f1dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec1a1580 con 0x7fc8ec1015f0 2023-12-07T10:19:55.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.464+0000 7fc8ebfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8ec103180 0x7fc8ec1a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:55.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.464+0000 7fc8ebfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8ec103180 0x7fc8ec1a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50956/0 (socket says 172.21.15.119:50956) 2023-12-07T10:19:55.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.464+0000 7fc8ebfff700 1 -- 172.21.15.119:0/802474210 learned_addr learned my addr 172.21.15.119:0/802474210 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:55.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.464+0000 7fc8eaffd700 1 --2- 172.21.15.119:0/802474210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8ec1027e0 0x7fc8ec199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:55.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.464+0000 7fc8eb7fe700 1 --2- 172.21.15.119:0/802474210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8ec1015f0 0x7fc8ec199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:55.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.465+0000 7fc8ebfff700 1 -- 172.21.15.119:0/802474210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8ec1015f0 msgr2=0x7fc8ec199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:55.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.465+0000 7fc8ebfff700 1 --2- 172.21.15.119:0/802474210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8ec1015f0 0x7fc8ec199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.465+0000 7fc8ebfff700 1 -- 172.21.15.119:0/802474210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8ec1027e0 msgr2=0x7fc8ec199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:55.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.465+0000 7fc8ebfff700 1 --2- 172.21.15.119:0/802474210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8ec1027e0 0x7fc8ec199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.465+0000 7fc8ebfff700 1 -- 172.21.15.119:0/802474210 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8dc0095f0 con 0x7fc8ec103180 2023-12-07T10:19:55.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.465+0000 7fc8ebfff700 1 --2- 172.21.15.119:0/802474210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8ec103180 0x7fc8ec1a0ce0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4007ee0 tx=0x7fc8e400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:55.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.465+0000 7fc8e8ff9700 1 -- 172.21.15.119:0/802474210 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4017070 con 0x7fc8ec103180 2023-12-07T10:19:55.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.465+0000 7fc8e8ff9700 1 -- 172.21.15.119:0/802474210 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8e400f040 con 0x7fc8ec103180 2023-12-07T10:19:55.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.465+0000 7fc8f1dff700 1 -- 172.21.15.119:0/802474210 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8ec1a1860 con 0x7fc8ec103180 2023-12-07T10:19:55.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.465+0000 7fc8e8ff9700 1 -- 172.21.15.119:0/802474210 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4005580 con 0x7fc8ec103180 2023-12-07T10:19:55.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.465+0000 7fc8f1dff700 1 -- 172.21.15.119:0/802474210 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ec1a1d70 con 0x7fc8ec103180 2023-12-07T10:19:55.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.466+0000 7fc8f1dff700 1 -- 172.21.15.119:0/802474210 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8ec0683c0 con 0x7fc8ec103180 2023-12-07T10:19:55.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.468+0000 7fc8e8ff9700 1 -- 172.21.15.119:0/802474210 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fc8e400db20 con 0x7fc8ec103180 2023-12-07T10:19:55.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.470+0000 7fc8e8ff9700 1 --2- 172.21.15.119:0/802474210 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc8d40a1b90 0x7fc8d40a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:55.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.470+0000 7fc8e8ff9700 1 -- 172.21.15.119:0/802474210 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fc8e40ccfe0 con 0x7fc8ec103180 2023-12-07T10:19:55.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.470+0000 7fc8eb7fe700 1 --2- 172.21.15.119:0/802474210 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc8d40a1b90 0x7fc8d40a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:55.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.471+0000 7fc8eb7fe700 1 --2- 172.21.15.119:0/802474210 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc8d40a1b90 0x7fc8d40a4050 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc003730 tx=0x7fc8dc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:55.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.474+0000 7fc8e8ff9700 1 -- 172.21.15.119:0/802474210 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc8e4096070 con 0x7fc8ec103180 2023-12-07T10:19:55.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.674+0000 7fc8f1dff700 1 -- 172.21.15.119:0/802474210 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc8ec1a2060 con 0x7fc8ec103180 2023-12-07T10:19:55.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.674+0000 7fc8e8ff9700 1 -- 172.21.15.119:0/802474210 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc8e4099690 con 0x7fc8ec103180 2023-12-07T10:19:55.678 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-12-07T10:19:55.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.676+0000 7fc8f1dff700 1 -- 172.21.15.119:0/802474210 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc8d40a1b90 msgr2=0x7fc8d40a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:55.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.676+0000 7fc8f1dff700 1 --2- 172.21.15.119:0/802474210 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc8d40a1b90 0x7fc8d40a4050 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc003730 tx=0x7fc8dc000bc0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.676+0000 7fc8f1dff700 1 -- 172.21.15.119:0/802474210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8ec103180 msgr2=0x7fc8ec1a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:55.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.676+0000 7fc8f1dff700 1 --2- 172.21.15.119:0/802474210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8ec103180 0x7fc8ec1a0ce0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4007ee0 tx=0x7fc8e400d3b0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.676+0000 7fc8f1dff700 1 -- 172.21.15.119:0/802474210 shutdown_connections 2023-12-07T10:19:55.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.676+0000 7fc8f1dff700 1 --2- 172.21.15.119:0/802474210 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fc8d40a1b90 0x7fc8d40a4050 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.677+0000 7fc8f1dff700 1 --2- 172.21.15.119:0/802474210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8ec103180 0x7fc8ec1a0ce0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.677+0000 7fc8f1dff700 1 --2- 172.21.15.119:0/802474210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8ec1015f0 0x7fc8ec199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.677+0000 7fc8f1dff700 1 --2- 172.21.15.119:0/802474210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8ec1027e0 0x7fc8ec199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.677+0000 7fc8f1dff700 1 -- 172.21.15.119:0/802474210 >> 172.21.15.119:0/802474210 conn(0x7fc8ec0fcb60 msgr2=0x7fc8ec073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:55.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.677+0000 7fc8f1dff700 1 -- 172.21.15.119:0/802474210 shutdown_connections 2023-12-07T10:19:55.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.677+0000 7fc8f1dff700 1 -- 172.21.15.119:0/802474210 wait complete. 2023-12-07T10:19:55.697 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-07T10:19:55.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.897+0000 7f424b5ff700 1 -- 172.21.15.119:0/4194334828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42441015c0 msgr2=0x7f42441019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:55.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.897+0000 7f424b5ff700 1 --2- 172.21.15.119:0/4194334828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42441015c0 0x7f42441019e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f42400099c0 tx=0x7f4240009cd0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.897+0000 7f424b5ff700 1 -- 172.21.15.119:0/4194334828 shutdown_connections 2023-12-07T10:19:55.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.897+0000 7f424b5ff700 1 --2- 172.21.15.119:0/4194334828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4244103150 0x7f4244109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.897+0000 7f424b5ff700 1 --2- 172.21.15.119:0/4194334828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42441015c0 0x7f42441019e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.897+0000 7f424b5ff700 1 --2- 172.21.15.119:0/4194334828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42441027b0 0x7f4244102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.897+0000 7f424b5ff700 1 -- 172.21.15.119:0/4194334828 >> 172.21.15.119:0/4194334828 conn(0x7f42440fcb10 msgr2=0x7f42440fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:55.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.897+0000 7f424b5ff700 1 -- 172.21.15.119:0/4194334828 shutdown_connections 2023-12-07T10:19:55.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.897+0000 7f424b5ff700 1 -- 172.21.15.119:0/4194334828 wait complete. 2023-12-07T10:19:55.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.898+0000 7f424b5ff700 1 Processor -- start 2023-12-07T10:19:55.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.898+0000 7f424b5ff700 1 -- start start 2023-12-07T10:19:55.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.898+0000 7f424b5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42441015c0 0x7f4244199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:55.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.899+0000 7f424b5ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42441027b0 0x7f4244199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:55.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.899+0000 7f424b5ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4244103150 0x7f42441a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:55.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.899+0000 7f424b5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42441a1290 con 0x7f42441015c0 2023-12-07T10:19:55.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.899+0000 7f424b5ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42441a13d0 con 0x7f42441027b0 2023-12-07T10:19:55.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.899+0000 7f424b5ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42441a1560 con 0x7f4244103150 2023-12-07T10:19:55.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.899+0000 7f424939b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42441015c0 0x7f4244199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:55.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.899+0000 7f424939b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42441015c0 0x7f4244199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50972/0 (socket says 172.21.15.119:50972) 2023-12-07T10:19:55.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.899+0000 7f4249b9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4244103150 0x7f42441a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:55.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.899+0000 7f4248b9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42441027b0 0x7f4244199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:55.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.899+0000 7f424939b700 1 -- 172.21.15.119:0/3802569577 learned_addr learned my addr 172.21.15.119:0/3802569577 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:55.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.899+0000 7f424939b700 1 -- 172.21.15.119:0/3802569577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4244103150 msgr2=0x7f42441a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:55.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.899+0000 7f424939b700 1 --2- 172.21.15.119:0/3802569577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4244103150 0x7f42441a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.899+0000 7f424939b700 1 -- 172.21.15.119:0/3802569577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42441027b0 msgr2=0x7f4244199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:55.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.899+0000 7f424939b700 1 --2- 172.21.15.119:0/3802569577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42441027b0 0x7f4244199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:55.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.899+0000 7f424939b700 1 -- 172.21.15.119:0/3802569577 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42400095f0 con 0x7f42441015c0 2023-12-07T10:19:55.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.899+0000 7f4248b9a700 1 --2- 172.21.15.119:0/3802569577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42441027b0 0x7f4244199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:55.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.899+0000 7f424939b700 1 --2- 172.21.15.119:0/3802569577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42441015c0 0x7f4244199760 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4240006010 tx=0x7f4240005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:55.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.900+0000 7f423a7fc700 1 -- 172.21.15.119:0/3802569577 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424001d070 con 0x7f42441015c0 2023-12-07T10:19:55.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.900+0000 7f423a7fc700 1 -- 172.21.15.119:0/3802569577 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4240003f90 con 0x7f42441015c0 2023-12-07T10:19:55.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.900+0000 7f424b5ff700 1 -- 172.21.15.119:0/3802569577 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42441a17e0 con 0x7f42441015c0 2023-12-07T10:19:55.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.900+0000 7f424b5ff700 1 -- 172.21.15.119:0/3802569577 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42441a1c00 con 0x7f42441015c0 2023-12-07T10:19:55.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.900+0000 7f423a7fc700 1 -- 172.21.15.119:0/3802569577 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4240022c50 con 0x7f42441015c0 2023-12-07T10:19:55.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.901+0000 7f424b5ff700 1 -- 172.21.15.119:0/3802569577 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42440683c0 con 0x7f42441015c0 2023-12-07T10:19:55.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.903+0000 7f423a7fc700 1 -- 172.21.15.119:0/3802569577 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f4240022470 con 0x7f42441015c0 2023-12-07T10:19:55.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.905+0000 7f423a7fc700 1 --2- 172.21.15.119:0/3802569577 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f42300a1b10 0x7f42300a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:55.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.905+0000 7f423a7fc700 1 -- 172.21.15.119:0/3802569577 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f42400cea60 con 0x7f42441015c0 2023-12-07T10:19:55.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.905+0000 7f4248b9a700 1 --2- 172.21.15.119:0/3802569577 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f42300a1b10 0x7f42300a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:55.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.906+0000 7f4248b9a700 1 --2- 172.21.15.119:0/3802569577 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f42300a1b10 0x7f42300a3fd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4234010420 tx=0x7f42340073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:55.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:55.909+0000 7f423a7fc700 1 -- 172.21.15.119:0/3802569577 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f424005d2f0 con 0x7f42441015c0 2023-12-07T10:19:56.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.113+0000 7f424b5ff700 1 -- 172.21.15.119:0/3802569577 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f42441a1f30 con 0x7f42441015c0 2023-12-07T10:19:56.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.113+0000 7f423a7fc700 1 -- 172.21.15.119:0/3802569577 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4240097b40 con 0x7f42441015c0 2023-12-07T10:19:56.115 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-12-07T10:19:56.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.116+0000 7f424b5ff700 1 -- 172.21.15.119:0/3802569577 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f42300a1b10 msgr2=0x7f42300a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:56.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.116+0000 7f424b5ff700 1 --2- 172.21.15.119:0/3802569577 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f42300a1b10 0x7f42300a3fd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4234010420 tx=0x7f42340073d0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.116+0000 7f424b5ff700 1 -- 172.21.15.119:0/3802569577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42441015c0 msgr2=0x7f4244199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:56.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.116+0000 7f424b5ff700 1 --2- 172.21.15.119:0/3802569577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42441015c0 0x7f4244199760 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4240006010 tx=0x7f4240005740 comp rx=0 tx=0).stop 2023-12-07T10:19:56.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.116+0000 7f424b5ff700 1 -- 172.21.15.119:0/3802569577 shutdown_connections 2023-12-07T10:19:56.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.116+0000 7f424b5ff700 1 --2- 172.21.15.119:0/3802569577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4244103150 0x7f42441a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.116+0000 7f424b5ff700 1 --2- 172.21.15.119:0/3802569577 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f42300a1b10 0x7f42300a3fd0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.116+0000 7f424b5ff700 1 --2- 172.21.15.119:0/3802569577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42441015c0 0x7f4244199760 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.116+0000 7f424b5ff700 1 --2- 172.21.15.119:0/3802569577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42441027b0 0x7f4244199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.116+0000 7f424b5ff700 1 -- 172.21.15.119:0/3802569577 >> 172.21.15.119:0/3802569577 conn(0x7f42440fcb10 msgr2=0x7f4244073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:56.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.116+0000 7f424b5ff700 1 -- 172.21.15.119:0/3802569577 shutdown_connections 2023-12-07T10:19:56.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.116+0000 7f424b5ff700 1 -- 172.21.15.119:0/3802569577 wait complete. 2023-12-07T10:19:56.136 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-07T10:19:56.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:56 smithi159 ceph-mon[134726]: pgmap v350: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-12-07T10:19:56.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:56 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/375869703' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:19:56.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:56 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/802474210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:19:56.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:56 smithi153 ceph-mon[133625]: pgmap v350: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-12-07T10:19:56.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:56 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/375869703' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:19:56.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:56 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/802474210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:19:56.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.334+0000 7f929c3ff700 1 -- 172.21.15.119:0/3506977495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92941015c0 msgr2=0x7f92941019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:56.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.334+0000 7f929c3ff700 1 --2- 172.21.15.119:0/3506977495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92941015c0 0x7f92941019e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f92900099c0 tx=0x7f9290009cd0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.334+0000 7f929c3ff700 1 -- 172.21.15.119:0/3506977495 shutdown_connections 2023-12-07T10:19:56.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.334+0000 7f929c3ff700 1 --2- 172.21.15.119:0/3506977495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9294103150 0x7f9294109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.334+0000 7f929c3ff700 1 --2- 172.21.15.119:0/3506977495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92941015c0 0x7f92941019e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.334+0000 7f929c3ff700 1 --2- 172.21.15.119:0/3506977495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92941027b0 0x7f9294102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.334+0000 7f929c3ff700 1 -- 172.21.15.119:0/3506977495 >> 172.21.15.119:0/3506977495 conn(0x7f92940fcb10 msgr2=0x7f92940fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:56.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.334+0000 7f929c3ff700 1 -- 172.21.15.119:0/3506977495 shutdown_connections 2023-12-07T10:19:56.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.334+0000 7f929c3ff700 1 -- 172.21.15.119:0/3506977495 wait complete. 2023-12-07T10:19:56.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.335+0000 7f929c3ff700 1 Processor -- start 2023-12-07T10:19:56.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.335+0000 7f929c3ff700 1 -- start start 2023-12-07T10:19:56.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.335+0000 7f929c3ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92941015c0 0x7f9294199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:56.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.335+0000 7f929c3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92941027b0 0x7f9294199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:56.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.335+0000 7f929c3ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9294103150 0x7f92941a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:56.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.335+0000 7f929c3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92941a12c0 con 0x7f92941027b0 2023-12-07T10:19:56.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.335+0000 7f929c3ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92941a1400 con 0x7f92941015c0 2023-12-07T10:19:56.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.335+0000 7f929c3ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92941a1590 con 0x7f9294103150 2023-12-07T10:19:56.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.336+0000 7f929999a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92941027b0 0x7f9294199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:56.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.336+0000 7f929999a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92941027b0 0x7f9294199cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50992/0 (socket says 172.21.15.119:50992) 2023-12-07T10:19:56.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.336+0000 7f929999a700 1 -- 172.21.15.119:0/3760175748 learned_addr learned my addr 172.21.15.119:0/3760175748 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:56.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.336+0000 7f929a19b700 1 --2- 172.21.15.119:0/3760175748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92941015c0 0x7f9294199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:56.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.336+0000 7f929a99c700 1 --2- 172.21.15.119:0/3760175748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9294103150 0x7f92941a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:56.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.336+0000 7f929999a700 1 -- 172.21.15.119:0/3760175748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9294103150 msgr2=0x7f92941a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:56.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.336+0000 7f929999a700 1 --2- 172.21.15.119:0/3760175748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9294103150 0x7f92941a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.336+0000 7f929999a700 1 -- 172.21.15.119:0/3760175748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92941015c0 msgr2=0x7f9294199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:56.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.336+0000 7f929999a700 1 --2- 172.21.15.119:0/3760175748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92941015c0 0x7f9294199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.336+0000 7f929999a700 1 -- 172.21.15.119:0/3760175748 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92900095f0 con 0x7f92941027b0 2023-12-07T10:19:56.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.336+0000 7f929999a700 1 --2- 172.21.15.119:0/3760175748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92941027b0 0x7f9294199cd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f928400e9a0 tx=0x7f928400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:56.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.337+0000 7f928b7fe700 1 -- 172.21.15.119:0/3760175748 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9284009790 con 0x7f92941027b0 2023-12-07T10:19:56.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.337+0000 7f928b7fe700 1 -- 172.21.15.119:0/3760175748 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9284010430 con 0x7f92941027b0 2023-12-07T10:19:56.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.337+0000 7f929c3ff700 1 -- 172.21.15.119:0/3760175748 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92941a1870 con 0x7f92941027b0 2023-12-07T10:19:56.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.337+0000 7f928b7fe700 1 -- 172.21.15.119:0/3760175748 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9284018680 con 0x7f92941027b0 2023-12-07T10:19:56.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.337+0000 7f929c3ff700 1 -- 172.21.15.119:0/3760175748 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92941a1dd0 con 0x7f92941027b0 2023-12-07T10:19:56.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.338+0000 7f929c3ff700 1 -- 172.21.15.119:0/3760175748 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92940683c0 con 0x7f92941027b0 2023-12-07T10:19:56.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.340+0000 7f928b7fe700 1 -- 172.21.15.119:0/3760175748 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f92840105a0 con 0x7f92941027b0 2023-12-07T10:19:56.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.341+0000 7f928b7fe700 1 --2- 172.21.15.119:0/3760175748 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f92800a1b90 0x7f92800a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:56.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.341+0000 7f928b7fe700 1 -- 172.21.15.119:0/3760175748 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f9284014070 con 0x7f92941027b0 2023-12-07T10:19:56.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.341+0000 7f929a19b700 1 --2- 172.21.15.119:0/3760175748 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f92800a1b90 0x7f92800a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:56.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.342+0000 7f929a19b700 1 --2- 172.21.15.119:0/3760175748 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f92800a1b90 0x7f92800a4050 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9290009990 tx=0x7f9290000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:56.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.343+0000 7f928b7fe700 1 -- 172.21.15.119:0/3760175748 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9284096db0 con 0x7f92941027b0 2023-12-07T10:19:56.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:56 smithi119 ceph-mon[97908]: pgmap v350: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-12-07T10:19:56.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:56 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/375869703' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:19:56.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:56 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/802474210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:19:56.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.547+0000 7f929c3ff700 1 -- 172.21.15.119:0/3760175748 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f929410cfd0 con 0x7f92941027b0 2023-12-07T10:19:56.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.547+0000 7f928b7fe700 1 -- 172.21.15.119:0/3760175748 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9284026020 con 0x7f92941027b0 2023-12-07T10:19:56.549 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-12-07T10:19:56.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.550+0000 7f929c3ff700 1 -- 172.21.15.119:0/3760175748 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f92800a1b90 msgr2=0x7f92800a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:56.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.550+0000 7f929c3ff700 1 --2- 172.21.15.119:0/3760175748 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f92800a1b90 0x7f92800a4050 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9290009990 tx=0x7f9290000bc0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.550+0000 7f929c3ff700 1 -- 172.21.15.119:0/3760175748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92941027b0 msgr2=0x7f9294199cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:56.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.550+0000 7f929c3ff700 1 --2- 172.21.15.119:0/3760175748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92941027b0 0x7f9294199cd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f928400e9a0 tx=0x7f928400ecb0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.550+0000 7f929c3ff700 1 -- 172.21.15.119:0/3760175748 shutdown_connections 2023-12-07T10:19:56.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.550+0000 7f929c3ff700 1 --2- 172.21.15.119:0/3760175748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9294103150 0x7f92941a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.550+0000 7f929c3ff700 1 --2- 172.21.15.119:0/3760175748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92941015c0 0x7f9294199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.550+0000 7f929c3ff700 1 --2- 172.21.15.119:0/3760175748 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f92800a1b90 0x7f92800a4050 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.550+0000 7f929c3ff700 1 --2- 172.21.15.119:0/3760175748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92941027b0 0x7f9294199cd0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.550+0000 7f929c3ff700 1 -- 172.21.15.119:0/3760175748 >> 172.21.15.119:0/3760175748 conn(0x7f92940fcb10 msgr2=0x7f9294107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:56.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.550+0000 7f929c3ff700 1 -- 172.21.15.119:0/3760175748 shutdown_connections 2023-12-07T10:19:56.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.550+0000 7f929c3ff700 1 -- 172.21.15.119:0/3760175748 wait complete. 2023-12-07T10:19:56.570 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-07T10:19:56.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.771+0000 7f518279d700 1 -- 172.21.15.119:0/3262484872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f517c075650 msgr2=0x7f517c073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:56.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.771+0000 7f518279d700 1 --2- 172.21.15.119:0/3262484872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f517c075650 0x7f517c073aa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5178009990 tx=0x7f5178009ca0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.771+0000 7f518279d700 1 -- 172.21.15.119:0/3262484872 shutdown_connections 2023-12-07T10:19:56.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.771+0000 7f518279d700 1 --2- 172.21.15.119:0/3262484872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f517c10da30 0x7f517c10fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.771+0000 7f518279d700 1 --2- 172.21.15.119:0/3262484872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f517c075650 0x7f517c073aa0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.771+0000 7f518279d700 1 --2- 172.21.15.119:0/3262484872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f517c074070 0x7f517c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.771+0000 7f518279d700 1 -- 172.21.15.119:0/3262484872 >> 172.21.15.119:0/3262484872 conn(0x7f517c0fc9f0 msgr2=0x7f517c0fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:56.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.772+0000 7f518279d700 1 -- 172.21.15.119:0/3262484872 shutdown_connections 2023-12-07T10:19:56.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.772+0000 7f518279d700 1 -- 172.21.15.119:0/3262484872 wait complete. 2023-12-07T10:19:56.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.773+0000 7f518279d700 1 Processor -- start 2023-12-07T10:19:56.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.773+0000 7f518279d700 1 -- start start 2023-12-07T10:19:56.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.773+0000 7f518279d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f517c074070 0x7f517c19d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:56.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.774+0000 7f518279d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f517c075650 0x7f517c19df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:56.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.774+0000 7f518279d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f517c10da30 0x7f517c1a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:56.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.774+0000 7f518279d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c1a54a0 con 0x7f517c074070 2023-12-07T10:19:56.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.774+0000 7f518279d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c1a55e0 con 0x7f517c075650 2023-12-07T10:19:56.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.774+0000 7f518279d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c1a5770 con 0x7f517c10da30 2023-12-07T10:19:56.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.774+0000 7f518179b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f517c074070 0x7f517c19d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:56.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.774+0000 7f518179b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f517c074070 0x7f517c19d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51004/0 (socket says 172.21.15.119:51004) 2023-12-07T10:19:56.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.774+0000 7f518179b700 1 -- 172.21.15.119:0/1186105028 learned_addr learned my addr 172.21.15.119:0/1186105028 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:56.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.774+0000 7f5181f9c700 1 --2- 172.21.15.119:0/1186105028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f517c10da30 0x7f517c1a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:56.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.774+0000 7f5180f9a700 1 --2- 172.21.15.119:0/1186105028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f517c075650 0x7f517c19df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:56.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.774+0000 7f518179b700 1 -- 172.21.15.119:0/1186105028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f517c10da30 msgr2=0x7f517c1a4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:56.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.774+0000 7f518179b700 1 --2- 172.21.15.119:0/1186105028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f517c10da30 0x7f517c1a4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.774+0000 7f518179b700 1 -- 172.21.15.119:0/1186105028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f517c075650 msgr2=0x7f517c19df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:56.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.774+0000 7f518179b700 1 --2- 172.21.15.119:0/1186105028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f517c075650 0x7f517c19df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.774+0000 7f518179b700 1 -- 172.21.15.119:0/1186105028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51780095f0 con 0x7f517c074070 2023-12-07T10:19:56.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.774+0000 7f518179b700 1 --2- 172.21.15.119:0/1186105028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f517c074070 0x7f517c19d9c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f517800b680 tx=0x7f5178009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:56.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.775+0000 7f51727fc700 1 -- 172.21.15.119:0/1186105028 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517801d070 con 0x7f517c074070 2023-12-07T10:19:56.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.775+0000 7f51727fc700 1 -- 172.21.15.119:0/1186105028 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5178022470 con 0x7f517c074070 2023-12-07T10:19:56.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.775+0000 7f51727fc700 1 -- 172.21.15.119:0/1186105028 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178017650 con 0x7f517c074070 2023-12-07T10:19:56.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.775+0000 7f518279d700 1 -- 172.21.15.119:0/1186105028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517c1a59f0 con 0x7f517c074070 2023-12-07T10:19:56.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.775+0000 7f518279d700 1 -- 172.21.15.119:0/1186105028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517c1a5dc0 con 0x7f517c074070 2023-12-07T10:19:56.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.776+0000 7f518279d700 1 -- 172.21.15.119:0/1186105028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f517c103c90 con 0x7f517c074070 2023-12-07T10:19:56.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.778+0000 7f51727fc700 1 -- 172.21.15.119:0/1186105028 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f51780225e0 con 0x7f517c074070 2023-12-07T10:19:56.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.779+0000 7f51727fc700 1 --2- 172.21.15.119:0/1186105028 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f51680a1b90 0x7f51680a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:56.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.779+0000 7f51727fc700 1 -- 172.21.15.119:0/1186105028 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f51780ceb20 con 0x7f517c074070 2023-12-07T10:19:56.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.779+0000 7f5180f9a700 1 --2- 172.21.15.119:0/1186105028 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f51680a1b90 0x7f51680a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:56.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.780+0000 7f5180f9a700 1 --2- 172.21.15.119:0/1186105028 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f51680a1b90 0x7f51680a4050 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f517c075370 tx=0x7f516c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:56.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.783+0000 7f51727fc700 1 -- 172.21.15.119:0/1186105028 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5178097c60 con 0x7f517c074070 2023-12-07T10:19:56.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.988+0000 7f518279d700 1 -- 172.21.15.119:0/1186105028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f517c103670 con 0x7f517c074070 2023-12-07T10:19:56.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.989+0000 7f51727fc700 1 -- 172.21.15.119:0/1186105028 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f517809b280 con 0x7f517c074070 2023-12-07T10:19:56.990 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-12-07T10:19:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.991+0000 7f518279d700 1 -- 172.21.15.119:0/1186105028 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f51680a1b90 msgr2=0x7f51680a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.991+0000 7f518279d700 1 --2- 172.21.15.119:0/1186105028 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f51680a1b90 0x7f51680a4050 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f517c075370 tx=0x7f516c009260 comp rx=0 tx=0).stop 2023-12-07T10:19:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.991+0000 7f518279d700 1 -- 172.21.15.119:0/1186105028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f517c074070 msgr2=0x7f517c19d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.991+0000 7f518279d700 1 --2- 172.21.15.119:0/1186105028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f517c074070 0x7f517c19d9c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f517800b680 tx=0x7f5178009580 comp rx=0 tx=0).stop 2023-12-07T10:19:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.991+0000 7f518279d700 1 -- 172.21.15.119:0/1186105028 shutdown_connections 2023-12-07T10:19:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.991+0000 7f518279d700 1 --2- 172.21.15.119:0/1186105028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f517c10da30 0x7f517c1a4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.991+0000 7f518279d700 1 --2- 172.21.15.119:0/1186105028 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f51680a1b90 0x7f51680a4050 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.991+0000 7f518279d700 1 --2- 172.21.15.119:0/1186105028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f517c074070 0x7f517c19d9c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.991+0000 7f518279d700 1 --2- 172.21.15.119:0/1186105028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f517c075650 0x7f517c19df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:56.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.991+0000 7f518279d700 1 -- 172.21.15.119:0/1186105028 >> 172.21.15.119:0/1186105028 conn(0x7f517c0fc9f0 msgr2=0x7f517c100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:56.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.991+0000 7f518279d700 1 -- 172.21.15.119:0/1186105028 shutdown_connections 2023-12-07T10:19:56.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:56.991+0000 7f518279d700 1 -- 172.21.15.119:0/1186105028 wait complete. 2023-12-07T10:19:57.012 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:19:57.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.208+0000 7f1986dff700 1 -- 172.21.15.119:0/4233248227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1980101560 msgr2=0x7f1980101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:57.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.208+0000 7f1986dff700 1 --2- 172.21.15.119:0/4233248227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1980101560 0x7f1980101980 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1974009990 tx=0x7f1974009ca0 comp rx=0 tx=0).stop 2023-12-07T10:19:57.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.208+0000 7f1986dff700 1 -- 172.21.15.119:0/4233248227 shutdown_connections 2023-12-07T10:19:57.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.208+0000 7f1986dff700 1 --2- 172.21.15.119:0/4233248227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19801030f0 0x7f19801099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:57.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.208+0000 7f1986dff700 1 --2- 172.21.15.119:0/4233248227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1980101560 0x7f1980101980 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:57.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.208+0000 7f1986dff700 1 --2- 172.21.15.119:0/4233248227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1980102750 0x7f1980102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:57.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.208+0000 7f1986dff700 1 -- 172.21.15.119:0/4233248227 >> 172.21.15.119:0/4233248227 conn(0x7f19800fcab0 msgr2=0x7f19800fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:57.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.208+0000 7f1986dff700 1 -- 172.21.15.119:0/4233248227 shutdown_connections 2023-12-07T10:19:57.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.208+0000 7f1986dff700 1 -- 172.21.15.119:0/4233248227 wait complete. 2023-12-07T10:19:57.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.209+0000 7f1986dff700 1 Processor -- start 2023-12-07T10:19:57.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.209+0000 7f1986dff700 1 -- start start 2023-12-07T10:19:57.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.209+0000 7f1986dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1980101560 0x7f1980075320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:57.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.209+0000 7f1986dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1980102750 0x7f1980073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:57.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.209+0000 7f1986dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19801030f0 0x7f1980073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:57.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.209+0000 7f1986dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198019b3b0 con 0x7f1980101560 2023-12-07T10:19:57.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.209+0000 7f1986dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1980074480 con 0x7f19801030f0 2023-12-07T10:19:57.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.209+0000 7f1986dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1980074640 con 0x7f1980102750 2023-12-07T10:19:57.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.210+0000 7f1984b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1980101560 0x7f1980075320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:57.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.210+0000 7f1984b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1980101560 0x7f1980075320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51016/0 (socket says 172.21.15.119:51016) 2023-12-07T10:19:57.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.210+0000 7f1984b9b700 1 -- 172.21.15.119:0/391335128 learned_addr learned my addr 172.21.15.119:0/391335128 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:57.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.210+0000 7f197ffff700 1 --2- 172.21.15.119:0/391335128 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1980102750 0x7f1980073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:57.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.210+0000 7f198539c700 1 --2- 172.21.15.119:0/391335128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19801030f0 0x7f1980073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:57.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.210+0000 7f1984b9b700 1 -- 172.21.15.119:0/391335128 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1980102750 msgr2=0x7f1980073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:57.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.210+0000 7f1984b9b700 1 --2- 172.21.15.119:0/391335128 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1980102750 0x7f1980073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:57.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.210+0000 7f1984b9b700 1 -- 172.21.15.119:0/391335128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19801030f0 msgr2=0x7f1980073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:57.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.210+0000 7f1984b9b700 1 --2- 172.21.15.119:0/391335128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19801030f0 0x7f1980073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:57.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.210+0000 7f1984b9b700 1 -- 172.21.15.119:0/391335128 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c009520 con 0x7f1980101560 2023-12-07T10:19:57.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.210+0000 7f197ffff700 1 --2- 172.21.15.119:0/391335128 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1980102750 0x7f1980073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:19:57.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.210+0000 7f198539c700 1 --2- 172.21.15.119:0/391335128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19801030f0 0x7f1980073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:19:57.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.210+0000 7f1984b9b700 1 --2- 172.21.15.119:0/391335128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1980101560 0x7f1980075320 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1974000c00 tx=0x7f1974005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:57.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.211+0000 7f197dffb700 1 -- 172.21.15.119:0/391335128 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197401d070 con 0x7f1980101560 2023-12-07T10:19:57.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.211+0000 7f1986dff700 1 -- 172.21.15.119:0/391335128 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19740095f0 con 0x7f1980101560 2023-12-07T10:19:57.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.211+0000 7f197dffb700 1 -- 172.21.15.119:0/391335128 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f197400fbc0 con 0x7f1980101560 2023-12-07T10:19:57.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.211+0000 7f1986dff700 1 -- 172.21.15.119:0/391335128 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19801a5fe0 con 0x7f1980101560 2023-12-07T10:19:57.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.211+0000 7f197dffb700 1 -- 172.21.15.119:0/391335128 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19740175f0 con 0x7f1980101560 2023-12-07T10:19:57.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.212+0000 7f1986dff700 1 -- 172.21.15.119:0/391335128 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1980108a80 con 0x7f1980101560 2023-12-07T10:19:57.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.214+0000 7f197dffb700 1 -- 172.21.15.119:0/391335128 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f1974021410 con 0x7f1980101560 2023-12-07T10:19:57.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.216+0000 7f197dffb700 1 --2- 172.21.15.119:0/391335128 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f19700a1b10 0x7f19700a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:57.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.216+0000 7f197dffb700 1 -- 172.21.15.119:0/391335128 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f19740ced40 con 0x7f1980101560 2023-12-07T10:19:57.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.216+0000 7f197ffff700 1 --2- 172.21.15.119:0/391335128 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f19700a1b10 0x7f19700a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:57.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.217+0000 7f197ffff700 1 --2- 172.21.15.119:0/391335128 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f19700a1b10 0x7f19700a3fd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f196c009e40 tx=0x7f196c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:57.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.220+0000 7f197dffb700 1 -- 172.21.15.119:0/391335128 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f197405c2f0 con 0x7f1980101560 2023-12-07T10:19:57.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:57 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3802569577' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:19:57.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:57 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3760175748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:19:57.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:57 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1186105028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:19:57.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:57 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3802569577' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:19:57.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:57 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3760175748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:19:57.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:57 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1186105028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:19:57.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.469+0000 7f1986dff700 1 -- 172.21.15.119:0/391335128 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1980068350 con 0x7f1980101560 2023-12-07T10:19:57.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.470+0000 7f197dffb700 1 -- 172.21.15.119:0/391335128 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11385 (secure 0 0 0) 0x7f1974097e80 con 0x7f1980101560 2023-12-07T10:19:57.472 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:19:57.472 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-07T10:17:41.278311+0000","modified":"2023-12-07T10:18:18.543356+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24517,"mds_3":24505,"mds_4":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14535":{"gid":14535,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6841/1791998028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1791998028},{"type":"v1","addr":"172.21.15.119:6841","nonce":1791998028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24505":{"gid":24505,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/2925503246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2925503246},{"type":"v1","addr":"172.21.15.153:6837","nonce":2925503246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6839/3984962900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3984962900},{"type":"v1","addr":"172.21.15.159:6839","nonce":3984962900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.119:6839/1920762602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1920762602},{"type":"v1","addr":"172.21.15.119:6839","nonce":1920762602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/566825184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":566825184},{"type":"v1","addr":"172.21.15.153:6839","nonce":566825184}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-07T10:19:57.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.473+0000 7f1986dff700 1 -- 172.21.15.119:0/391335128 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f19700a1b10 msgr2=0x7f19700a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:57.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.473+0000 7f1986dff700 1 --2- 172.21.15.119:0/391335128 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f19700a1b10 0x7f19700a3fd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f196c009e40 tx=0x7f196c009260 comp rx=0 tx=0).stop 2023-12-07T10:19:57.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.473+0000 7f1986dff700 1 -- 172.21.15.119:0/391335128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1980101560 msgr2=0x7f1980075320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:57.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.473+0000 7f1986dff700 1 --2- 172.21.15.119:0/391335128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1980101560 0x7f1980075320 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1974000c00 tx=0x7f1974005ef0 comp rx=0 tx=0).stop 2023-12-07T10:19:57.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.473+0000 7f1986dff700 1 -- 172.21.15.119:0/391335128 shutdown_connections 2023-12-07T10:19:57.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.473+0000 7f1986dff700 1 --2- 172.21.15.119:0/391335128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19801030f0 0x7f1980073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:57.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.473+0000 7f1986dff700 1 --2- 172.21.15.119:0/391335128 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f19700a1b10 0x7f19700a3fd0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:57.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.473+0000 7f1986dff700 1 --2- 172.21.15.119:0/391335128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1980101560 0x7f1980075320 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:57.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.473+0000 7f1986dff700 1 --2- 172.21.15.119:0/391335128 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1980102750 0x7f1980073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:57.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.473+0000 7f1986dff700 1 -- 172.21.15.119:0/391335128 >> 172.21.15.119:0/391335128 conn(0x7f19800fcab0 msgr2=0x7f1980105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:57.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.473+0000 7f1986dff700 1 -- 172.21.15.119:0/391335128 shutdown_connections 2023-12-07T10:19:57.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.473+0000 7f1986dff700 1 -- 172.21.15.119:0/391335128 wait complete. 2023-12-07T10:19:57.475 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 29 2023-12-07T10:19:57.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:57 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3802569577' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:19:57.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:57 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3760175748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:19:57.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:57 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1186105028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:19:57.493 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-07T10:19:57.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.692+0000 7f2e55bff700 1 -- 172.21.15.119:0/2627444750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e50075650 msgr2=0x7f2e50073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:57.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.692+0000 7f2e55bff700 1 --2- 172.21.15.119:0/2627444750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e50075650 0x7f2e50073aa0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2e40009990 tx=0x7f2e40009ca0 comp rx=0 tx=0).stop 2023-12-07T10:19:57.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.692+0000 7f2e55bff700 1 -- 172.21.15.119:0/2627444750 shutdown_connections 2023-12-07T10:19:57.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.692+0000 7f2e55bff700 1 --2- 172.21.15.119:0/2627444750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e50109420 0x7f2e5010b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:57.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.692+0000 7f2e55bff700 1 --2- 172.21.15.119:0/2627444750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e50075650 0x7f2e50073aa0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:57.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.692+0000 7f2e55bff700 1 --2- 172.21.15.119:0/2627444750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e50074070 0x7f2e500744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:57.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.692+0000 7f2e55bff700 1 -- 172.21.15.119:0/2627444750 >> 172.21.15.119:0/2627444750 conn(0x7f2e500fcab0 msgr2=0x7f2e500fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:57.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.692+0000 7f2e55bff700 1 -- 172.21.15.119:0/2627444750 shutdown_connections 2023-12-07T10:19:57.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.692+0000 7f2e55bff700 1 -- 172.21.15.119:0/2627444750 wait complete. 2023-12-07T10:19:57.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.693+0000 7f2e55bff700 1 Processor -- start 2023-12-07T10:19:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.693+0000 7f2e55bff700 1 -- start start 2023-12-07T10:19:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.694+0000 7f2e55bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e50074070 0x7f2e50199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.694+0000 7f2e55bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e50075650 0x7f2e50199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.694+0000 7f2e55bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e50109420 0x7f2e501a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.694+0000 7f2e55bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e501a1270 con 0x7f2e50074070 2023-12-07T10:19:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.694+0000 7f2e55bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e501a13b0 con 0x7f2e50109420 2023-12-07T10:19:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.694+0000 7f2e55bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e501a1540 con 0x7f2e50075650 2023-12-07T10:19:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.694+0000 7f2e4f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e50074070 0x7f2e50199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.694+0000 7f2e4effd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e50075650 0x7f2e50199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.694+0000 7f2e4f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e50074070 0x7f2e50199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51042/0 (socket says 172.21.15.119:51042) 2023-12-07T10:19:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.694+0000 7f2e4f7fe700 1 -- 172.21.15.119:0/2877126870 learned_addr learned my addr 172.21.15.119:0/2877126870 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.694+0000 7f2e4ffff700 1 --2- 172.21.15.119:0/2877126870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e50109420 0x7f2e501a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.694+0000 7f2e4f7fe700 1 -- 172.21.15.119:0/2877126870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e50075650 msgr2=0x7f2e50199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:57.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.694+0000 7f2e4f7fe700 1 --2- 172.21.15.119:0/2877126870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e50075650 0x7f2e50199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:57.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.694+0000 7f2e4f7fe700 1 -- 172.21.15.119:0/2877126870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e50109420 msgr2=0x7f2e501a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:57.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.694+0000 7f2e4f7fe700 1 --2- 172.21.15.119:0/2877126870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e50109420 0x7f2e501a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:57.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.694+0000 7f2e4f7fe700 1 -- 172.21.15.119:0/2877126870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e400095f0 con 0x7f2e50074070 2023-12-07T10:19:57.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.695+0000 7f2e4f7fe700 1 --2- 172.21.15.119:0/2877126870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e50074070 0x7f2e50199790 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2e4000b680 tx=0x7f2e40009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:57.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.695+0000 7f2e4cff9700 1 -- 172.21.15.119:0/2877126870 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4001d070 con 0x7f2e50074070 2023-12-07T10:19:57.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.695+0000 7f2e4cff9700 1 -- 172.21.15.119:0/2877126870 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e40022470 con 0x7f2e50074070 2023-12-07T10:19:57.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.695+0000 7f2e55bff700 1 -- 172.21.15.119:0/2877126870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e501a17c0 con 0x7f2e50074070 2023-12-07T10:19:57.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.695+0000 7f2e4cff9700 1 -- 172.21.15.119:0/2877126870 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e40017650 con 0x7f2e50074070 2023-12-07T10:19:57.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.695+0000 7f2e55bff700 1 -- 172.21.15.119:0/2877126870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e501a1be0 con 0x7f2e50074070 2023-12-07T10:19:57.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.696+0000 7f2e55bff700 1 -- 172.21.15.119:0/2877126870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e501081e0 con 0x7f2e50074070 2023-12-07T10:19:57.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.698+0000 7f2e4cff9700 1 -- 172.21.15.119:0/2877126870 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f2e400225e0 con 0x7f2e50074070 2023-12-07T10:19:57.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.699+0000 7f2e4cff9700 1 --2- 172.21.15.119:0/2877126870 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2e380a1b10 0x7f2e380a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:57.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.699+0000 7f2e4cff9700 1 -- 172.21.15.119:0/2877126870 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f2e400cf1c0 con 0x7f2e50074070 2023-12-07T10:19:57.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.699+0000 7f2e4effd700 1 --2- 172.21.15.119:0/2877126870 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2e380a1b10 0x7f2e380a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:57.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.700+0000 7f2e4effd700 1 --2- 172.21.15.119:0/2877126870 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2e380a1b10 0x7f2e380a3fd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2e50075370 tx=0x7f2e44009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:57.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.702+0000 7f2e4cff9700 1 -- 172.21.15.119:0/2877126870 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2e400982a0 con 0x7f2e50074070 2023-12-07T10:19:57.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.909+0000 7f2e55bff700 1 -- 172.21.15.119:0/2877126870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e50068350 con 0x7f2e50074070 2023-12-07T10:19:57.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.911+0000 7f2e4cff9700 1 -- 172.21.15.119:0/2877126870 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24715 (secure 0 0 0) 0x7f2e4009b8c0 con 0x7f2e50074070 2023-12-07T10:19:57.912 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:19:57.913 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":93,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","created":"2023-12-07T10:09:59.945863+0000","modified":"2023-12-07T10:19:02.800626+0000","last_up_change":"2023-12-07T10:16:50.929427+0000","last_in_change":"2023-12-07T10:16:33.558158+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T10:13:25.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-07T10:17:39.347813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-07T10:17:40.714501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-07T10:17:44.257760+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"56b5818e-e625-40f1-8838-3ddcf2d34ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6803","nonce":2141950133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6805","nonce":2141950133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6809","nonce":2141950133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6807","nonce":2141950133}]},"public_addr":"172.21.15.119:6803/2141950133","cluster_addr":"172.21.15.119:6805/2141950133","heartbeat_back_addr":"172.21.15.119:6809/2141950133","heartbeat_front_addr":"172.21.15.119:6807/2141950133","state":["exists","up"]},{"osd":1,"uuid":"36489a6a-5278-4802-9525-9871521722e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6801","nonce":3390875370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6803","nonce":3390875370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6807","nonce":3390875370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6805","nonce":3390875370}]},"public_addr":"172.21.15.153:6801/3390875370","cluster_addr":"172.21.15.153:6803/3390875370","heartbeat_back_addr":"172.21.15.153:6807/3390875370","heartbeat_front_addr":"172.21.15.153:6805/3390875370","state":["exists","up"]},{"osd":2,"uuid":"a937ad26-5e9b-4c8a-b65a-626a8feda6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6801","nonce":708101637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6803","nonce":708101637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6807","nonce":708101637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6805","nonce":708101637}]},"public_addr":"172.21.15.159:6801/708101637","cluster_addr":"172.21.15.159:6803/708101637","heartbeat_back_addr":"172.21.15.159:6807/708101637","heartbeat_front_addr":"172.21.15.159:6805/708101637","state":["exists","up"]},{"osd":3,"uuid":"26ef8300-aa07-48f6-b31d-8b0f2217ffcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6811","nonce":3228047257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6813","nonce":3228047257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6817","nonce":3228047257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6815","nonce":3228047257}]},"public_addr":"172.21.15.119:6811/3228047257","cluster_addr":"172.21.15.119:6813/3228047257","heartbeat_back_addr":"172.21.15.119:6817/3228047257","heartbeat_front_addr":"172.21.15.119:6815/3228047257","state":["exists","up"]},{"osd":4,"uuid":"000ee4b9-9896-4901-8cc3-c85fdfc8cbaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6809","nonce":4021998299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6811","nonce":4021998299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6815","nonce":4021998299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6813","nonce":4021998299}]},"public_addr":"172.21.15.153:6809/4021998299","cluster_addr":"172.21.15.153:6811/4021998299","heartbeat_back_addr":"172.21.15.153:6815/4021998299","heartbeat_front_addr":"172.21.15.153:6813/4021998299","state":["exists","up"]},{"osd":5,"uuid":"8b2fc167-262d-474f-89f3-47bd29424421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6809","nonce":1832540768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6811","nonce":1832540768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6815","nonce":1832540768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6813","nonce":1832540768}]},"public_addr":"172.21.15.159:6809/1832540768","cluster_addr":"172.21.15.159:6811/1832540768","heartbeat_back_addr":"172.21.15.159:6815/1832540768","heartbeat_front_addr":"172.21.15.159:6813/1832540768","state":["exists","up"]},{"osd":6,"uuid":"f9aa6749-46cf-4b2a-8114-cbd034219d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162436840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162436840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6825","nonce":1162436840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162436840}]},"public_addr":"172.21.15.119:6819/1162436840","cluster_addr":"172.21.15.119:6821/1162436840","heartbeat_back_addr":"172.21.15.119:6825/1162436840","heartbeat_front_addr":"172.21.15.119:6823/1162436840","state":["exists","up"]},{"osd":7,"uuid":"39216361-e1c4-4fef-86cb-a4707656b547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6817","nonce":3254653457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6819","nonce":3254653457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6823","nonce":3254653457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6821","nonce":3254653457}]},"public_addr":"172.21.15.153:6817/3254653457","cluster_addr":"172.21.15.153:6819/3254653457","heartbeat_back_addr":"172.21.15.153:6823/3254653457","heartbeat_front_addr":"172.21.15.153:6821/3254653457","state":["exists","up"]},{"osd":8,"uuid":"cceb6ebc-7c02-48ea-9d16-b7a016dd449b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6817","nonce":2600946380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6819","nonce":2600946380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6823","nonce":2600946380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6821","nonce":2600946380}]},"public_addr":"172.21.15.159:6817/2600946380","cluster_addr":"172.21.15.159:6819/2600946380","heartbeat_back_addr":"172.21.15.159:6823/2600946380","heartbeat_front_addr":"172.21.15.159:6821/2600946380","state":["exists","up"]},{"osd":9,"uuid":"f337a2e4-e065-4a9f-91d9-e3615de2d6f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6827","nonce":1940737285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6829","nonce":1940737285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6833","nonce":1940737285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6831","nonce":1940737285}]},"public_addr":"172.21.15.119:6827/1940737285","cluster_addr":"172.21.15.119:6829/1940737285","heartbeat_back_addr":"172.21.15.119:6833/1940737285","heartbeat_front_addr":"172.21.15.119:6831/1940737285","state":["exists","up"]},{"osd":10,"uuid":"3a3bc347-85f4-4878-9797-d9da96137e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6825","nonce":663834864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6827","nonce":663834864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6831","nonce":663834864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6829","nonce":663834864}]},"public_addr":"172.21.15.153:6825/663834864","cluster_addr":"172.21.15.153:6827/663834864","heartbeat_back_addr":"172.21.15.153:6831/663834864","heartbeat_front_addr":"172.21.15.153:6829/663834864","state":["exists","up"]},{"osd":11,"uuid":"a0fc5909-279d-4172-977c-2dc2da0a7e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6825","nonce":1262216348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6827","nonce":1262216348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6831","nonce":1262216348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6829","nonce":1262216348}]},"public_addr":"172.21.15.159:6825/1262216348","cluster_addr":"172.21.15.159:6827/1262216348","heartbeat_back_addr":"172.21.15.159:6831/1262216348","heartbeat_front_addr":"172.21.15.159:6829/1262216348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:37.488109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:59.543060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:21.381585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:47.112729+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:09.732150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:30.893729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:56.678710+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:19.004256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:40.388818+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:06.391734+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:28.394053+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:49.634348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1628639392":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6801/1355617847":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6800/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/362307836":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/128964558":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/2278813554":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/1411933614":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/1685698277":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6800/1355617847":"2023-12-08T10:11:00.585565+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T10:19:57.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.913+0000 7f2e55bff700 1 -- 172.21.15.119:0/2877126870 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2e380a1b10 msgr2=0x7f2e380a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:57.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.913+0000 7f2e55bff700 1 --2- 172.21.15.119:0/2877126870 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2e380a1b10 0x7f2e380a3fd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2e50075370 tx=0x7f2e44009260 comp rx=0 tx=0).stop 2023-12-07T10:19:57.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.913+0000 7f2e55bff700 1 -- 172.21.15.119:0/2877126870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e50074070 msgr2=0x7f2e50199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:57.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.913+0000 7f2e55bff700 1 --2- 172.21.15.119:0/2877126870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e50074070 0x7f2e50199790 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2e4000b680 tx=0x7f2e40009580 comp rx=0 tx=0).stop 2023-12-07T10:19:57.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.913+0000 7f2e55bff700 1 -- 172.21.15.119:0/2877126870 shutdown_connections 2023-12-07T10:19:57.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.913+0000 7f2e55bff700 1 --2- 172.21.15.119:0/2877126870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e50109420 0x7f2e501a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:57.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.913+0000 7f2e55bff700 1 --2- 172.21.15.119:0/2877126870 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f2e380a1b10 0x7f2e380a3fd0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:57.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.913+0000 7f2e55bff700 1 --2- 172.21.15.119:0/2877126870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e50074070 0x7f2e50199790 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:57.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.913+0000 7f2e55bff700 1 --2- 172.21.15.119:0/2877126870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e50075650 0x7f2e50199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:57.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.913+0000 7f2e55bff700 1 -- 172.21.15.119:0/2877126870 >> 172.21.15.119:0/2877126870 conn(0x7f2e500fcab0 msgr2=0x7f2e50105320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:57.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.914+0000 7f2e55bff700 1 -- 172.21.15.119:0/2877126870 shutdown_connections 2023-12-07T10:19:57.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:57.914+0000 7f2e55bff700 1 -- 172.21.15.119:0/2877126870 wait complete. 2023-12-07T10:19:57.935 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-12-07T10:17:41.278311+0000', 'modified': '2023-12-07T10:18:18.543356+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24566, 'mds_1': 24542, 'mds_2': 24517, 'mds_3': 24505, 'mds_4': 24488}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14535': {'gid': 14535, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.119:6841/1791998028', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6840', 'nonce': 1791998028}, {'type': 'v1', 'addr': '172.21.15.119:6841', 'nonce': 1791998028}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24445': {'gid': 24445, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6833/935461464', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6832', 'nonce': 935461464}, {'type': 'v1', 'addr': '172.21.15.159:6833', 'nonce': 935461464}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24452': {'gid': 24452, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.153:6833/209059922', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6832', 'nonce': 209059922}, {'type': 'v1', 'addr': '172.21.15.153:6833', 'nonce': 209059922}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24488': {'gid': 24488, 'name': 'e', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.153:6835/1295649464', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6834', 'nonce': 1295649464}, {'type': 'v1', 'addr': '172.21.15.153:6835', 'nonce': 1295649464}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24505': {'gid': 24505, 'name': 'f', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.159:6835/2406380792', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 2406380792}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 2406380792}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24517': {'gid': 24517, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.159:6837/3106097520', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6836', 'nonce': 3106097520}, {'type': 'v1', 'addr': '172.21.15.159:6837', 'nonce': 3106097520}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24518': {'gid': 24518, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.153:6837/2925503246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6836', 'nonce': 2925503246}, {'type': 'v1', 'addr': '172.21.15.153:6837', 'nonce': 2925503246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24538': {'gid': 24538, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6839/3984962900', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6838', 'nonce': 3984962900}, {'type': 'v1', 'addr': '172.21.15.159:6839', 'nonce': 3984962900}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24542': {'gid': 24542, 'name': 'g', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.119:6839/1920762602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6838', 'nonce': 1920762602}, {'type': 'v1', 'addr': '172.21.15.119:6839', 'nonce': 1920762602}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24566': {'gid': 24566, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.153:6839/566825184', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6838', 'nonce': 566825184}, {'type': 'v1', 'addr': '172.21.15.153:6839', 'nonce': 566825184}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-07T10:19:57.935 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-12-07T10:19:57.935 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-12-07T10:19:58.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.132+0000 7f982c5ff700 1 -- 172.21.15.119:0/1406995553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9824102730 msgr2=0x7f9824102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.132+0000 7f982c5ff700 1 --2- 172.21.15.119:0/1406995553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9824102730 0x7f9824102b90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f98140099c0 tx=0x7f9814009cd0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.132+0000 7f982c5ff700 1 -- 172.21.15.119:0/1406995553 shutdown_connections 2023-12-07T10:19:58.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.132+0000 7f982c5ff700 1 --2- 172.21.15.119:0/1406995553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f98241030d0 0x7f9824109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.132+0000 7f982c5ff700 1 --2- 172.21.15.119:0/1406995553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9824101540 0x7f9824101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.132+0000 7f982c5ff700 1 --2- 172.21.15.119:0/1406995553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9824102730 0x7f9824102b90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.132+0000 7f982c5ff700 1 -- 172.21.15.119:0/1406995553 >> 172.21.15.119:0/1406995553 conn(0x7f98240fcab0 msgr2=0x7f98240feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:58.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.132+0000 7f982c5ff700 1 -- 172.21.15.119:0/1406995553 shutdown_connections 2023-12-07T10:19:58.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.132+0000 7f982c5ff700 1 -- 172.21.15.119:0/1406995553 wait complete. 2023-12-07T10:19:58.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.133+0000 7f982c5ff700 1 Processor -- start 2023-12-07T10:19:58.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.133+0000 7f982c5ff700 1 -- start start 2023-12-07T10:19:58.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.134+0000 7f982c5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9824101540 0x7f98241974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.134+0000 7f982c5ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9824102730 0x7f98241979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.134+0000 7f982c5ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98241030d0 0x7f982419ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.134+0000 7f982c5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f982419efd0 con 0x7f9824101540 2023-12-07T10:19:58.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.134+0000 7f982c5ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f982419f110 con 0x7f98241030d0 2023-12-07T10:19:58.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.134+0000 7f982c5ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f982419f2a0 con 0x7f9824102730 2023-12-07T10:19:58.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.134+0000 7f982a39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9824101540 0x7f98241974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.134+0000 7f982a39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9824101540 0x7f98241974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51060/0 (socket says 172.21.15.119:51060) 2023-12-07T10:19:58.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.134+0000 7f982a39b700 1 -- 172.21.15.119:0/1375105307 learned_addr learned my addr 172.21.15.119:0/1375105307 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:58.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.134+0000 7f9829b9a700 1 --2- 172.21.15.119:0/1375105307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9824102730 0x7f98241979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.134+0000 7f982ab9c700 1 --2- 172.21.15.119:0/1375105307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98241030d0 0x7f982419ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.134+0000 7f982a39b700 1 -- 172.21.15.119:0/1375105307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9824102730 msgr2=0x7f98241979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.134+0000 7f982a39b700 1 --2- 172.21.15.119:0/1375105307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9824102730 0x7f98241979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.134+0000 7f982a39b700 1 -- 172.21.15.119:0/1375105307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98241030d0 msgr2=0x7f982419ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.134+0000 7f982a39b700 1 --2- 172.21.15.119:0/1375105307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98241030d0 0x7f982419ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.134+0000 7f982a39b700 1 -- 172.21.15.119:0/1375105307 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98140095f0 con 0x7f9824101540 2023-12-07T10:19:58.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.134+0000 7f982ab9c700 1 --2- 172.21.15.119:0/1375105307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98241030d0 0x7f982419ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:19:58.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.134+0000 7f982a39b700 1 --2- 172.21.15.119:0/1375105307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9824101540 0x7f98241974a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f982000e970 tx=0x7f982000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:58.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.135+0000 7f981b7fe700 1 -- 172.21.15.119:0/1375105307 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9820009790 con 0x7f9824101540 2023-12-07T10:19:58.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.135+0000 7f981b7fe700 1 -- 172.21.15.119:0/1375105307 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f982000eec0 con 0x7f9824101540 2023-12-07T10:19:58.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.135+0000 7f981b7fe700 1 -- 172.21.15.119:0/1375105307 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9820003cf0 con 0x7f9824101540 2023-12-07T10:19:58.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.135+0000 7f982c5ff700 1 -- 172.21.15.119:0/1375105307 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f982419f580 con 0x7f9824101540 2023-12-07T10:19:58.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.135+0000 7f982c5ff700 1 -- 172.21.15.119:0/1375105307 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f982419fa90 con 0x7f9824101540 2023-12-07T10:19:58.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.138+0000 7f981b7fe700 1 -- 172.21.15.119:0/1375105307 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f982000ccd0 con 0x7f9824101540 2023-12-07T10:19:58.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.139+0000 7f983ffd5b80 1 Processor -- start 2023-12-07T10:19:58.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.139+0000 7f981b7fe700 1 --2- 172.21.15.119:0/1375105307 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f98100a1b90 0x7f98100a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.139+0000 7f981b7fe700 1 -- 172.21.15.119:0/1375105307 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f9820014070 con 0x7f9824101540 2023-12-07T10:19:58.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.139+0000 7f983ffd5b80 1 -- start start 2023-12-07T10:19:58.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.140+0000 7f982ab9c700 1 --2- 172.21.15.119:0/1375105307 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f98100a1b90 0x7f98100a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.140+0000 7f983ffd5b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639b13f3130 0x5639b13f3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.140+0000 7f983ffd5b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3a90 0x5639b13f43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.140+0000 7f983ffd5b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639b13f4980 0x5639b13f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.140+0000 7f983ffd5b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b1367770 con 0x5639b13f3a90 2023-12-07T10:19:58.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.140+0000 7f983ffd5b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b134e750 con 0x5639b13f4980 2023-12-07T10:19:58.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.140+0000 7f983ffd5b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b134c070 con 0x5639b13f3130 2023-12-07T10:19:58.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.140+0000 7f9829b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3a90 0x5639b13f43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.140+0000 7f9829b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3a90 0x5639b13f43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51064/0 (socket says 172.21.15.119:51064) 2023-12-07T10:19:58.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.140+0000 7f9829b9a700 1 -- 172.21.15.119:0/1828679009 learned_addr learned my addr 172.21.15.119:0/1828679009 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:58.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.140+0000 7f982a39b700 1 --2- 172.21.15.119:0/1828679009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639b13f3130 0x5639b13f3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.140+0000 7f982ab9c700 1 --2- 172.21.15.119:0/1375105307 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f98100a1b90 0x7f98100a4050 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f981c0093f0 tx=0x7f981c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:58.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.140+0000 7f982ab9c700 1 --2- 172.21.15.119:0/1828679009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639b13f4980 0x5639b13f4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.141+0000 7f9829b9a700 1 -- 172.21.15.119:0/1828679009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639b13f3130 msgr2=0x5639b13f3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.141+0000 7f9829b9a700 1 --2- 172.21.15.119:0/1828679009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639b13f3130 0x5639b13f3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.141+0000 7f9829b9a700 1 -- 172.21.15.119:0/1828679009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639b13f4980 msgr2=0x5639b13f4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.141+0000 7f9829b9a700 1 --2- 172.21.15.119:0/1828679009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639b13f4980 0x5639b13f4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.141+0000 7f9829b9a700 1 -- 172.21.15.119:0/1828679009 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639b11c45e0 con 0x5639b13f3a90 2023-12-07T10:19:58.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.141+0000 7f982a39b700 1 --2- 172.21.15.119:0/1828679009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639b13f3130 0x5639b13f3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:58.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.141+0000 7f9829b9a700 1 --2- 172.21.15.119:0/1828679009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3a90 0x5639b13f43a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9814000c00 tx=0x7f9814005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b824ae317fc078e2 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:58.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.141+0000 7f9818bfe700 1 -- 172.21.15.119:0/1828679009 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981400ba10 con 0x5639b13f3a90 2023-12-07T10:19:58.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.141+0000 7f9818bfe700 1 -- 172.21.15.119:0/1828679009 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f981401e040 con 0x5639b13f3a90 2023-12-07T10:19:58.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.141+0000 7f9818bfe700 1 -- 172.21.15.119:0/1828679009 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9814003b50 con 0x5639b13f3a90 2023-12-07T10:19:58.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.142+0000 7f983ffd5b80 1 -- 172.21.15.119:0/1828679009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3a90 msgr2=0x5639b13f43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.142+0000 7f983ffd5b80 1 --2- 172.21.15.119:0/1828679009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3a90 0x5639b13f43a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9814000c00 tx=0x7f9814005bc0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.142+0000 7f983ffd5b80 1 -- 172.21.15.119:0/1828679009 shutdown_connections 2023-12-07T10:19:58.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.142+0000 7f983ffd5b80 1 --2- 172.21.15.119:0/1828679009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639b13f4980 0x5639b13f4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.142+0000 7f983ffd5b80 1 --2- 172.21.15.119:0/1828679009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639b13f3130 0x5639b13f3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.142+0000 7f983ffd5b80 1 --2- 172.21.15.119:0/1828679009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3a90 0x5639b13f43a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.142+0000 7f983ffd5b80 1 -- 172.21.15.119:0/1828679009 >> 172.21.15.119:0/1828679009 conn(0x5639b1322310 msgr2=0x5639b13efb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:58.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.142+0000 7f983ffd5b80 1 -- 172.21.15.119:0/1828679009 shutdown_connections 2023-12-07T10:19:58.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.142+0000 7f983ffd5b80 1 -- 172.21.15.119:0/1828679009 wait complete. 2023-12-07T10:19:58.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.142+0000 7f983ffd5b80 1 -- start start 2023-12-07T10:19:58.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.143+0000 7f983ffd5b80 1 Processor -- start 2023-12-07T10:19:58.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.143+0000 7f983ffd5b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639b13f3130 0x5639b13fcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.143+0000 7f983ffd5b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3a90 0x5639b13fd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.143+0000 7f983ffd5b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639b13f4980 0x5639b14044b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.143+0000 7f983ffd5b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b1347960 con 0x5639b13f3a90 2023-12-07T10:19:58.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.143+0000 7f983ffd5b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b1301510 con 0x5639b13f4980 2023-12-07T10:19:58.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.143+0000 7f983ffd5b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b13011b0 con 0x5639b13f3130 2023-12-07T10:19:58.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.143+0000 7f983ffd5b80 10 client.0 _ll_register_callbacks cb 0x5639b12e88f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:19:58.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.143+0000 7f9829b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3a90 0x5639b13fd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.143+0000 7f9829b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3a90 0x5639b13fd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51066/0 (socket says 172.21.15.119:51066) 2023-12-07T10:19:58.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.144+0000 7f9829b9a700 1 -- 172.21.15.119:0/3264010018 learned_addr learned my addr 172.21.15.119:0/3264010018 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:58.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.144+0000 7f982ab9c700 1 --2- 172.21.15.119:0/3264010018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639b13f4980 0x5639b14044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.144+0000 7f982a39b700 1 --2- 172.21.15.119:0/3264010018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639b13f3130 0x5639b13fcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.144+0000 7f982a39b700 1 -- 172.21.15.119:0/3264010018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639b13f4980 msgr2=0x5639b14044b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.144+0000 7f982a39b700 1 --2- 172.21.15.119:0/3264010018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639b13f4980 0x5639b14044b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.144+0000 7f982a39b700 1 -- 172.21.15.119:0/3264010018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3a90 msgr2=0x5639b13fd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.144+0000 7f982a39b700 1 --2- 172.21.15.119:0/3264010018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3a90 0x5639b13fd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.144+0000 7f982a39b700 1 -- 172.21.15.119:0/3264010018 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f982419fa90 con 0x5639b13f3130 2023-12-07T10:19:58.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.144+0000 7f982ab9c700 1 --2- 172.21.15.119:0/3264010018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639b13f4980 0x5639b14044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:58.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.144+0000 7f9829b9a700 1 --2- 172.21.15.119:0/3264010018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3a90 0x5639b13fd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:58.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.144+0000 7f982a39b700 1 --2- 172.21.15.119:0/3264010018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639b13f3130 0x5639b13fcf30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f982009a570 tx=0x7f982009aff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1de309d451a8f53c server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:58.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.144+0000 7f980dffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:19:58.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.144+0000 7f980dffb700 1 -- 172.21.15.119:0/3264010018 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98200b2070 con 0x5639b13f3130 2023-12-07T10:19:58.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.144+0000 7f983ffd5b80 1 -- 172.21.15.119:0/3264010018 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639b13e6ab0 con 0x5639b13f3130 2023-12-07T10:19:58.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.144+0000 7f980dffb700 1 -- 172.21.15.119:0/3264010018 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f982009e9c0 con 0x5639b13f3130 2023-12-07T10:19:58.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.145+0000 7f980dffb700 1 -- 172.21.15.119:0/3264010018 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98200a2c00 con 0x5639b13f3130 2023-12-07T10:19:58.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.145+0000 7f983ffd5b80 10 client.25351 fetch_fsmap learned FSMap version 29 2023-12-07T10:19:58.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.145+0000 7f983ffd5b80 1 -- 172.21.15.119:0/3264010018 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f982410a2d0 con 0x5639b13f3130 2023-12-07T10:19:58.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.145+0000 7f980dffb700 1 -- 172.21.15.119:0/3264010018 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f982007d020 con 0x5639b13f3130 2023-12-07T10:19:58.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.145+0000 7f980dffb700 1 -- 172.21.15.119:0/3264010018 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f9820099d60 con 0x5639b13f3130 2023-12-07T10:19:58.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.145+0000 7f983ffd5b80 10 client.25351 fetch_fsmap finished waiting for FSMap version 29 2023-12-07T10:19:58.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.145+0000 7f983ffd5b80 10 client.25351 resolve_mds: resolved name 'e' to daemon mds.e 2023-12-07T10:19:58.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.145+0000 7f983ffd5b80 20 client.25351 populate_metadata read hostname 'smithi119' 2023-12-07T10:19:58.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.145+0000 7f983ffd5b80 1 --2- 172.21.15.119:0/3264010018 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x5639b1408b90 0x5639b14076b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.145+0000 7f983ffd5b80 4 client.25351 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:19:58.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.145+0000 7f983ffd5b80 1 -- 172.21.15.119:0/3264010018 --> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639b13c2750 con 0x5639b1408b90 2023-12-07T10:19:58.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.146+0000 7f9829b9a700 1 --2- 172.21.15.119:0/3264010018 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x5639b1408b90 0x5639b14076b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.146+0000 7f9829b9a700 1 --2- 172.21.15.119:0/3264010018 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x5639b1408b90 0x5639b14076b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ca625f9ad75f53e4 server_cookie=e820800d3aef6aff in_seq=0 out_seq=0 2023-12-07T10:19:58.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.146+0000 7f980dffb700 10 client.25351 ms_handle_connect on v2:172.21.15.153:6834/1295649464 2023-12-07T10:19:58.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.147+0000 7f980dffb700 1 -- 172.21.15.119:0/3264010018 <== mds.4 v2:172.21.15.153:6834/1295649464 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5639b13c2750 con 0x5639b1408b90 2023-12-07T10:19:58.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.147+0000 7f980dffb700 10 client.25351 handle_command_reply: tid=0 2023-12-07T10:19:58.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.147+0000 7f983ffd5b80 1 client.25351 shutdown 2023-12-07T10:19:58.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.147+0000 7f983ffd5b80 1 -- 172.21.15.119:0/3264010018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639b13f3130 msgr2=0x5639b13fcf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.147+0000 7f983ffd5b80 1 --2- 172.21.15.119:0/3264010018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639b13f3130 0x5639b13fcf30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f982009a570 tx=0x7f982009aff0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.147+0000 7f983ffd5b80 1 -- 172.21.15.119:0/3264010018 shutdown_connections 2023-12-07T10:19:58.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.147+0000 7f983ffd5b80 1 --2- 172.21.15.119:0/3264010018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639b13f4980 0x5639b14044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.147+0000 7f983ffd5b80 1 --2- 172.21.15.119:0/3264010018 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x5639b1408b90 0x5639b14076b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.147+0000 7f983ffd5b80 1 --2- 172.21.15.119:0/3264010018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639b13f3130 0x5639b13fcf30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.147+0000 7f983ffd5b80 1 --2- 172.21.15.119:0/3264010018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3a90 0x5639b13fd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.147+0000 7f980dffb700 0 client.25351 ms_handle_reset on v2:172.21.15.153:6834/1295649464 2023-12-07T10:19:58.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.147+0000 7f983ffd5b80 1 -- 172.21.15.119:0/3264010018 >> 172.21.15.119:0/3264010018 conn(0x5639b1322310 msgr2=0x5639b13ee990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:58.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.147+0000 7f983ffd5b80 1 -- 172.21.15.119:0/3264010018 shutdown_connections 2023-12-07T10:19:58.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.147+0000 7f983ffd5b80 1 -- 172.21.15.119:0/3264010018 wait complete. 2023-12-07T10:19:58.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.148+0000 7f983ffd5b80 20 client.25351 trim_cache size 0 max 16384 2023-12-07T10:19:58.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.166+0000 7f983ffd5b80 1 Processor -- start 2023-12-07T10:19:58.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.166+0000 7f983ffd5b80 1 -- start start 2023-12-07T10:19:58.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.166+0000 7f983ffd5b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3130 0x5639b13f78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.166+0000 7f983ffd5b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639b13f3a90 0x5639b1404470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.166+0000 7f983ffd5b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639b13f4980 0x5639b13f7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.166+0000 7f983ffd5b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b11c68d0 con 0x5639b13f3130 2023-12-07T10:19:58.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.166+0000 7f983ffd5b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b13fde30 con 0x5639b13f4980 2023-12-07T10:19:58.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.166+0000 7f983ffd5b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b13c2750 con 0x5639b13f3a90 2023-12-07T10:19:58.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.167+0000 7f982a39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3130 0x5639b13f78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.167+0000 7f982ab9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639b13f4980 0x5639b13f7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.167+0000 7f9829b9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639b13f3a90 0x5639b1404470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.167+0000 7f982a39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3130 0x5639b13f78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51074/0 (socket says 172.21.15.119:51074) 2023-12-07T10:19:58.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.167+0000 7f982a39b700 1 -- 172.21.15.119:0/3545987132 learned_addr learned my addr 172.21.15.119:0/3545987132 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:58.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.167+0000 7f982a39b700 1 -- 172.21.15.119:0/3545987132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639b13f3a90 msgr2=0x5639b1404470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.167+0000 7f982a39b700 1 --2- 172.21.15.119:0/3545987132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639b13f3a90 0x5639b1404470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.167+0000 7f982a39b700 1 -- 172.21.15.119:0/3545987132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639b13f4980 msgr2=0x5639b13f7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.167+0000 7f982a39b700 1 --2- 172.21.15.119:0/3545987132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639b13f4980 0x5639b13f7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.167+0000 7f982a39b700 1 -- 172.21.15.119:0/3545987132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639b1170d90 con 0x5639b13f3130 2023-12-07T10:19:58.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.167+0000 7f982ab9c700 1 --2- 172.21.15.119:0/3545987132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639b13f4980 0x5639b13f7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:58.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.167+0000 7f982a39b700 1 --2- 172.21.15.119:0/3545987132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3130 0x5639b13f78f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9820005950 tx=0x7f9820000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506f4373afaef2fd server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:58.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.167+0000 7f98193ff700 1 -- 172.21.15.119:0/3545987132 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98200ba0b0 con 0x5639b13f3130 2023-12-07T10:19:58.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.167+0000 7f98193ff700 1 -- 172.21.15.119:0/3545987132 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5639b13e6ab0 con 0x5639b13f3130 2023-12-07T10:19:58.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.167+0000 7f98193ff700 1 -- 172.21.15.119:0/3545987132 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9820099d60 con 0x5639b13f3130 2023-12-07T10:19:58.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.168+0000 7f983ffd5b80 1 -- 172.21.15.119:0/3545987132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3130 msgr2=0x5639b13f78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.168+0000 7f983ffd5b80 1 --2- 172.21.15.119:0/3545987132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3130 0x5639b13f78f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9820005950 tx=0x7f9820000bc0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.168+0000 7f983ffd5b80 1 -- 172.21.15.119:0/3545987132 shutdown_connections 2023-12-07T10:19:58.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.168+0000 7f983ffd5b80 1 --2- 172.21.15.119:0/3545987132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639b13f4980 0x5639b13f7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.168+0000 7f983ffd5b80 1 --2- 172.21.15.119:0/3545987132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3130 0x5639b13f78f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.168+0000 7f983ffd5b80 1 --2- 172.21.15.119:0/3545987132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639b13f3a90 0x5639b1404470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.168+0000 7f983ffd5b80 1 -- 172.21.15.119:0/3545987132 >> 172.21.15.119:0/3545987132 conn(0x5639b1322310 msgr2=0x5639b13fbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:58.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.168+0000 7f983ffd5b80 1 -- 172.21.15.119:0/3545987132 shutdown_connections 2023-12-07T10:19:58.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.168+0000 7f983ffd5b80 1 -- 172.21.15.119:0/3545987132 wait complete. 2023-12-07T10:19:58.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.169+0000 7f983ffd5b80 1 -- start start 2023-12-07T10:19:58.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.169+0000 7f983ffd5b80 1 Processor -- start 2023-12-07T10:19:58.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.169+0000 7f983ffd5b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3130 0x5639b1403d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.169+0000 7f983ffd5b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639b13f3a90 0x5639b1404270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.169+0000 7f983ffd5b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639b13f4980 0x5639b1400820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.169+0000 7f983ffd5b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b1400e40 con 0x5639b13f3130 2023-12-07T10:19:58.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.169+0000 7f983ffd5b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b1400fb0 con 0x5639b13f3a90 2023-12-07T10:19:58.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.169+0000 7f983ffd5b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b1401170 con 0x5639b13f4980 2023-12-07T10:19:58.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.169+0000 7f983ffd5b80 10 client.0 _ll_register_callbacks cb 0x5639b12e88f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:19:58.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.169+0000 7f982a39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3130 0x5639b1403d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.169+0000 7f982a39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3130 0x5639b1403d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51076/0 (socket says 172.21.15.119:51076) 2023-12-07T10:19:58.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.169+0000 7f982a39b700 1 -- 172.21.15.119:0/1853474187 learned_addr learned my addr 172.21.15.119:0/1853474187 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:58.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.170+0000 7f9829b9a700 1 --2- 172.21.15.119:0/1853474187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639b13f3a90 0x5639b1404270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.170+0000 7f982ab9c700 1 --2- 172.21.15.119:0/1853474187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639b13f4980 0x5639b1400820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.170+0000 7f982a39b700 1 -- 172.21.15.119:0/1853474187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639b13f4980 msgr2=0x5639b1400820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.170+0000 7f982a39b700 1 --2- 172.21.15.119:0/1853474187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639b13f4980 0x5639b1400820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.170+0000 7f982a39b700 1 -- 172.21.15.119:0/1853474187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639b13f3a90 msgr2=0x5639b1404270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.170+0000 7f982a39b700 1 --2- 172.21.15.119:0/1853474187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639b13f3a90 0x5639b1404270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.170+0000 7f982a39b700 1 -- 172.21.15.119:0/1853474187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f981c015750 con 0x5639b13f3130 2023-12-07T10:19:58.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.170+0000 7f982a39b700 1 --2- 172.21.15.119:0/1853474187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3130 0x5639b1403d30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9820096d70 tx=0x7f9820096ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7a3eb6e7719e17 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:58.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.170+0000 7f980dffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:19:58.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.170+0000 7f980dffb700 1 -- 172.21.15.119:0/1853474187 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98200b2070 con 0x5639b13f3130 2023-12-07T10:19:58.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.170+0000 7f980dffb700 1 -- 172.21.15.119:0/1853474187 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f982007e080 con 0x5639b13f3130 2023-12-07T10:19:58.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.170+0000 7f983ffd5b80 1 -- 172.21.15.119:0/1853474187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f981401e040 con 0x5639b13f3130 2023-12-07T10:19:58.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.170+0000 7f980dffb700 1 -- 172.21.15.119:0/1853474187 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f982009e9c0 con 0x5639b13f3130 2023-12-07T10:19:58.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.170+0000 7f983ffd5b80 10 client.15348 fetch_fsmap learned FSMap version 29 2023-12-07T10:19:58.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.170+0000 7f983ffd5b80 1 -- 172.21.15.119:0/1853474187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f982410a2d0 con 0x5639b13f3130 2023-12-07T10:19:58.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.171+0000 7f980dffb700 1 -- 172.21.15.119:0/1853474187 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f981401e040 con 0x5639b13f3130 2023-12-07T10:19:58.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.171+0000 7f980dffb700 1 -- 172.21.15.119:0/1853474187 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f98200a8070 con 0x5639b13f3130 2023-12-07T10:19:58.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.171+0000 7f983ffd5b80 10 client.15348 fetch_fsmap finished waiting for FSMap version 29 2023-12-07T10:19:58.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.171+0000 7f983ffd5b80 10 client.15348 resolve_mds: resolved name 'e' to daemon mds.e 2023-12-07T10:19:58.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.171+0000 7f983ffd5b80 20 client.15348 populate_metadata read hostname 'smithi119' 2023-12-07T10:19:58.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.171+0000 7f983ffd5b80 1 --2- 172.21.15.119:0/1853474187 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x5639b14635e0 0x5639b145fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.171+0000 7f983ffd5b80 4 client.15348 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-07T10:19:58.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.171+0000 7f983ffd5b80 1 -- 172.21.15.119:0/1853474187 --> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5639b130df80 con 0x5639b14635e0 2023-12-07T10:19:58.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.171+0000 7f9829b9a700 1 --2- 172.21.15.119:0/1853474187 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x5639b14635e0 0x5639b145fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.172+0000 7f9829b9a700 1 --2- 172.21.15.119:0/1853474187 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x5639b14635e0 0x5639b145fb00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=55a800993ae01755 server_cookie=544302aea5bf2d6 in_seq=0 out_seq=0 2023-12-07T10:19:58.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.172+0000 7f980dffb700 10 client.15348 ms_handle_connect on v2:172.21.15.153:6834/1295649464 2023-12-07T10:19:58.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.172+0000 7f980dffb700 1 -- 172.21.15.119:0/1853474187 <== mds.4 v2:172.21.15.153:6834/1295649464 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x5639b130df80 con 0x5639b14635e0 2023-12-07T10:19:58.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.172+0000 7f980dffb700 10 client.15348 handle_command_reply: tid=0 2023-12-07T10:19:58.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.172+0000 7f983ffd5b80 1 client.15348 shutdown 2023-12-07T10:19:58.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.172+0000 7f983ffd5b80 1 -- 172.21.15.119:0/1853474187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3130 msgr2=0x5639b1403d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.172+0000 7f983ffd5b80 1 --2- 172.21.15.119:0/1853474187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3130 0x5639b1403d30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9820096d70 tx=0x7f9820096ce0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.172+0000 7f983ffd5b80 1 -- 172.21.15.119:0/1853474187 shutdown_connections 2023-12-07T10:19:58.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.172+0000 7f983ffd5b80 1 --2- 172.21.15.119:0/1853474187 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x5639b14635e0 0x5639b145fb00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.172+0000 7f983ffd5b80 1 --2- 172.21.15.119:0/1853474187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639b13f4980 0x5639b1400820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.172+0000 7f983ffd5b80 1 --2- 172.21.15.119:0/1853474187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639b13f3130 0x5639b1403d30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.172+0000 7f983ffd5b80 1 --2- 172.21.15.119:0/1853474187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5639b13f3a90 0x5639b1404270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.172+0000 7f980dffb700 0 client.15348 ms_handle_reset on v2:172.21.15.153:6834/1295649464 2023-12-07T10:19:58.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.172+0000 7f983ffd5b80 1 -- 172.21.15.119:0/1853474187 >> 172.21.15.119:0/1853474187 conn(0x5639b1322310 msgr2=0x5639b13f78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:58.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.172+0000 7f983ffd5b80 1 -- 172.21.15.119:0/1853474187 shutdown_connections 2023-12-07T10:19:58.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.172+0000 7f983ffd5b80 1 -- 172.21.15.119:0/1853474187 wait complete. 2023-12-07T10:19:58.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.172+0000 7f983ffd5b80 20 client.15348 trim_cache size 0 max 16384 2023-12-07T10:19:58.180 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:19:58.180 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "a4c642c6-94e8-11ee-95a2-87774f69a715", 2023-12-07T10:19:58.180 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 4, 2023-12-07T10:19:58.181 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 24488, 2023-12-07T10:19:58.181 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-12-07T10:19:58.181 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-12-07T10:19:58.181 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-12-07T10:19:58.181 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 105.427675534, 2023-12-07T10:19:58.181 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 29, 2023-12-07T10:19:58.181 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 93, 2023-12-07T10:19:58.181 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 91, 2023-12-07T10:19:58.181 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 162.531567889 2023-12-07T10:19:58.181 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:19:58.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.173+0000 7f982c5ff700 1 -- 172.21.15.119:0/1375105307 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f98100a1b90 msgr2=0x7f98100a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.173+0000 7f982c5ff700 1 --2- 172.21.15.119:0/1375105307 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f98100a1b90 0x7f98100a4050 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f981c0093f0 tx=0x7f981c00b040 comp rx=0 tx=0).stop 2023-12-07T10:19:58.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.173+0000 7f982c5ff700 1 -- 172.21.15.119:0/1375105307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9824101540 msgr2=0x7f98241974a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.173+0000 7f982c5ff700 1 --2- 172.21.15.119:0/1375105307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9824101540 0x7f98241974a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f982000e970 tx=0x7f982000ec80 comp rx=0 tx=0).stop 2023-12-07T10:19:58.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.173+0000 7f982c5ff700 1 -- 172.21.15.119:0/1375105307 shutdown_connections 2023-12-07T10:19:58.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.173+0000 7f982c5ff700 1 --2- 172.21.15.119:0/1375105307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98241030d0 0x7f982419ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.173+0000 7f982c5ff700 1 --2- 172.21.15.119:0/1375105307 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f98100a1b90 0x7f98100a4050 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.173+0000 7f982c5ff700 1 --2- 172.21.15.119:0/1375105307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9824101540 0x7f98241974a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.173+0000 7f982c5ff700 1 --2- 172.21.15.119:0/1375105307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9824102730 0x7f98241979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.173+0000 7f982c5ff700 1 -- 172.21.15.119:0/1375105307 >> 172.21.15.119:0/1375105307 conn(0x7f98240fcab0 msgr2=0x7f9824105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:58.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.173+0000 7f982c5ff700 1 -- 172.21.15.119:0/1375105307 shutdown_connections 2023-12-07T10:19:58.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.173+0000 7f982c5ff700 1 -- 172.21.15.119:0/1375105307 wait complete. 2023-12-07T10:19:58.191 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:58 smithi119 ceph-mon[97908]: pgmap v351: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-12-07T10:19:58.191 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:58 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/391335128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:19:58.191 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:58 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2877126870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:19:58.193 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-12-07T10:19:58.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:58 smithi159 ceph-mon[134726]: pgmap v351: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-12-07T10:19:58.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:58 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/391335128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:19:58.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:58 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2877126870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:19:58.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:58 smithi153 ceph-mon[133625]: pgmap v351: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-12-07T10:19:58.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:58 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/391335128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:19:58.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:58 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2877126870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:19:58.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.388+0000 7f5dfdfff700 1 -- 172.21.15.119:0/3777527913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df81030d0 msgr2=0x7f5df8109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.388+0000 7f5dfdfff700 1 --2- 172.21.15.119:0/3777527913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df81030d0 0x7f5df8109980 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5df000b0d0 tx=0x7f5df000b3e0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.388+0000 7f5dfdfff700 1 -- 172.21.15.119:0/3777527913 shutdown_connections 2023-12-07T10:19:58.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.388+0000 7f5dfdfff700 1 --2- 172.21.15.119:0/3777527913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df81030d0 0x7f5df8109980 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.388+0000 7f5dfdfff700 1 --2- 172.21.15.119:0/3777527913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5df8101540 0x7f5df8101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.388+0000 7f5dfdfff700 1 --2- 172.21.15.119:0/3777527913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5df8102730 0x7f5df8102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.388+0000 7f5dfdfff700 1 -- 172.21.15.119:0/3777527913 >> 172.21.15.119:0/3777527913 conn(0x7f5df80fcab0 msgr2=0x7f5df80feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:58.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.388+0000 7f5dfdfff700 1 -- 172.21.15.119:0/3777527913 shutdown_connections 2023-12-07T10:19:58.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.388+0000 7f5dfdfff700 1 -- 172.21.15.119:0/3777527913 wait complete. 2023-12-07T10:19:58.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.389+0000 7f5dfdfff700 1 Processor -- start 2023-12-07T10:19:58.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.389+0000 7f5dfdfff700 1 -- start start 2023-12-07T10:19:58.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.389+0000 7f5dfdfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5df8101540 0x7f5df81997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.389+0000 7f5dfdfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df8102730 0x7f5df8199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.389+0000 7f5dfdfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5df81030d0 0x7f5df81a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.389+0000 7f5dfdfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df81a1320 con 0x7f5df8102730 2023-12-07T10:19:58.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.389+0000 7f5dfdfff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df81a1460 con 0x7f5df81030d0 2023-12-07T10:19:58.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.389+0000 7f5dfdfff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df81a15f0 con 0x7f5df8101540 2023-12-07T10:19:58.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.389+0000 7f5df6ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df8102730 0x7f5df8199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.390+0000 7f5df6ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df8102730 0x7f5df8199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51096/0 (socket says 172.21.15.119:51096) 2023-12-07T10:19:58.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.390+0000 7f5df6ffd700 1 -- 172.21.15.119:0/835592251 learned_addr learned my addr 172.21.15.119:0/835592251 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:58.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.390+0000 7f5df77fe700 1 --2- 172.21.15.119:0/835592251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5df8101540 0x7f5df81997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.390+0000 7f5df7fff700 1 --2- 172.21.15.119:0/835592251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5df81030d0 0x7f5df81a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.390+0000 7f5df6ffd700 1 -- 172.21.15.119:0/835592251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5df8101540 msgr2=0x7f5df81997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.390+0000 7f5df6ffd700 1 --2- 172.21.15.119:0/835592251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5df8101540 0x7f5df81997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.390+0000 7f5df6ffd700 1 -- 172.21.15.119:0/835592251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5df81030d0 msgr2=0x7f5df81a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.390+0000 7f5df6ffd700 1 --2- 172.21.15.119:0/835592251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5df81030d0 0x7f5df81a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.390+0000 7f5df6ffd700 1 -- 172.21.15.119:0/835592251 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df0009d00 con 0x7f5df8102730 2023-12-07T10:19:58.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.390+0000 7f5df6ffd700 1 --2- 172.21.15.119:0/835592251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df8102730 0x7f5df8199d30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00d980 tx=0x7f5dec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:58.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.390+0000 7f5df4ff9700 1 -- 172.21.15.119:0/835592251 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec004220 con 0x7f5df8102730 2023-12-07T10:19:58.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.390+0000 7f5df4ff9700 1 -- 172.21.15.119:0/835592251 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dec007640 con 0x7f5df8102730 2023-12-07T10:19:58.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.390+0000 7f5dfdfff700 1 -- 172.21.15.119:0/835592251 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df81a18d0 con 0x7f5df8102730 2023-12-07T10:19:58.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.390+0000 7f5df4ff9700 1 -- 172.21.15.119:0/835592251 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec005690 con 0x7f5df8102730 2023-12-07T10:19:58.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.391+0000 7f5dfdfff700 1 -- 172.21.15.119:0/835592251 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df81a1de0 con 0x7f5df8102730 2023-12-07T10:19:58.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.395+0000 7f5df4ff9700 1 -- 172.21.15.119:0/835592251 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f5dec004750 con 0x7f5df8102730 2023-12-07T10:19:58.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.396+0000 7f5e1199bb80 1 Processor -- start 2023-12-07T10:19:58.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.396+0000 7f5e1199bb80 1 -- start start 2023-12-07T10:19:58.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.396+0000 7f5e1199bb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c096f3e130 0x55c096f3e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.396+0000 7f5e1199bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c096f3ea90 0x55c096f3f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.396+0000 7f5e1199bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3f980 0x55c096f3fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.396+0000 7f5e1199bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c096eb2770 con 0x55c096f3f980 2023-12-07T10:19:58.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.396+0000 7f5e1199bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c096e99750 con 0x55c096f3ea90 2023-12-07T10:19:58.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.396+0000 7f5e1199bb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c096e97070 con 0x55c096f3e130 2023-12-07T10:19:58.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.396+0000 7f5df4ff9700 1 --2- 172.21.15.119:0/835592251 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5de00a1b10 0x7f5de00a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.397+0000 7f5df4ff9700 1 -- 172.21.15.119:0/835592251 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f5dec0cd610 con 0x7f5df8102730 2023-12-07T10:19:58.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.397+0000 7f5df6ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3f980 0x55c096f3fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.397+0000 7f5df7fff700 1 --2- 172.21.15.119:0/835592251 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5de00a1b10 0x7f5de00a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.397+0000 7f5df77fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c096f3ea90 0x55c096f3f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.397+0000 7f5df6ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3f980 0x55c096f3fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51100/0 (socket says 172.21.15.119:51100) 2023-12-07T10:19:58.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.397+0000 7f5df6ffd700 1 -- 172.21.15.119:0/422652645 learned_addr learned my addr 172.21.15.119:0/422652645 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:58.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.397+0000 7f5df7fff700 1 --2- 172.21.15.119:0/422652645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c096f3e130 0x55c096f3e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.397+0000 7f5df6ffd700 1 -- 172.21.15.119:0/422652645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c096f3e130 msgr2=0x55c096f3e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.397+0000 7f5df6ffd700 1 --2- 172.21.15.119:0/422652645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c096f3e130 0x55c096f3e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.397+0000 7f5df6ffd700 1 -- 172.21.15.119:0/422652645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c096f3ea90 msgr2=0x55c096f3f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.397+0000 7f5df6ffd700 1 --2- 172.21.15.119:0/422652645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c096f3ea90 0x55c096f3f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.397+0000 7f5df6ffd700 1 -- 172.21.15.119:0/422652645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c096d0f5e0 con 0x55c096f3f980 2023-12-07T10:19:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.397+0000 7f5df6ffd700 1 --2- 172.21.15.119:0/422652645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3f980 0x55c096f3fe00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5dec017000 tx=0x7f5dec094f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d0deeb58734eb5 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.397+0000 7f5df7fff700 1 --2- 172.21.15.119:0/835592251 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5de00a1b10 0x7f5de00a3fd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009fd0 tx=0x7f5df000cf20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.397+0000 7f5dddffb700 1 -- 172.21.15.119:0/422652645 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec09aa80 con 0x55c096f3f980 2023-12-07T10:19:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.398+0000 7f5dddffb700 1 -- 172.21.15.119:0/422652645 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dec09b470 con 0x55c096f3f980 2023-12-07T10:19:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.398+0000 7f5dddffb700 1 -- 172.21.15.119:0/422652645 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec09e4d0 con 0x55c096f3f980 2023-12-07T10:19:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.398+0000 7f5e1199bb80 1 -- 172.21.15.119:0/422652645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3f980 msgr2=0x55c096f3fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.398+0000 7f5e1199bb80 1 --2- 172.21.15.119:0/422652645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3f980 0x55c096f3fe00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5dec017000 tx=0x7f5dec094f90 comp rx=0 tx=0).stop 2023-12-07T10:19:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.398+0000 7f5e1199bb80 1 -- 172.21.15.119:0/422652645 shutdown_connections 2023-12-07T10:19:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.398+0000 7f5e1199bb80 1 --2- 172.21.15.119:0/422652645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3f980 0x55c096f3fe00 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.398+0000 7f5e1199bb80 1 --2- 172.21.15.119:0/422652645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c096f3e130 0x55c096f3e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.398+0000 7f5e1199bb80 1 --2- 172.21.15.119:0/422652645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c096f3ea90 0x55c096f3f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.398+0000 7f5e1199bb80 1 -- 172.21.15.119:0/422652645 >> 172.21.15.119:0/422652645 conn(0x55c096e6d310 msgr2=0x55c096f3ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.398+0000 7f5e1199bb80 1 -- 172.21.15.119:0/422652645 shutdown_connections 2023-12-07T10:19:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.398+0000 7f5e1199bb80 1 -- 172.21.15.119:0/422652645 wait complete. 2023-12-07T10:19:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.399+0000 7f5e1199bb80 1 -- start start 2023-12-07T10:19:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.399+0000 7f5e1199bb80 1 Processor -- start 2023-12-07T10:19:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.399+0000 7f5e1199bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3e130 0x55c096f48000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.399+0000 7f5e1199bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c096f3ea90 0x55c096f48540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.400+0000 7f5e1199bb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c096f3f980 0x55c096f4f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.400+0000 7f5e1199bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c096e92960 con 0x55c096f3e130 2023-12-07T10:19:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.400+0000 7f5e1199bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c096e4c510 con 0x55c096f3ea90 2023-12-07T10:19:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.400+0000 7f5e1199bb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c096e4c1b0 con 0x55c096f3f980 2023-12-07T10:19:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.400+0000 7f5e1199bb80 10 client.0 _ll_register_callbacks cb 0x55c096e338f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:19:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.400+0000 7f5df77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3e130 0x55c096f48000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.400+0000 7f5df77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3e130 0x55c096f48000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51116/0 (socket says 172.21.15.119:51116) 2023-12-07T10:19:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.400+0000 7f5df77fe700 1 -- 172.21.15.119:0/1005400448 learned_addr learned my addr 172.21.15.119:0/1005400448 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.400+0000 7f5df6ffd700 1 --2- 172.21.15.119:0/1005400448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c096f3ea90 0x55c096f48540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.400+0000 7f5df7fff700 1 --2- 172.21.15.119:0/1005400448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c096f3f980 0x55c096f4f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.400+0000 7f5df77fe700 1 -- 172.21.15.119:0/1005400448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c096f3f980 msgr2=0x55c096f4f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.400+0000 7f5df77fe700 1 --2- 172.21.15.119:0/1005400448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c096f3f980 0x55c096f4f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.400+0000 7f5df77fe700 1 -- 172.21.15.119:0/1005400448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c096f3ea90 msgr2=0x55c096f48540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.400+0000 7f5df77fe700 1 --2- 172.21.15.119:0/1005400448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c096f3ea90 0x55c096f48540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.400+0000 7f5df77fe700 1 -- 172.21.15.119:0/1005400448 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df81a1de0 con 0x55c096f3e130 2023-12-07T10:19:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.400+0000 7f5df6ffd700 1 --2- 172.21.15.119:0/1005400448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c096f3ea90 0x55c096f48540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:19:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.400+0000 7f5df77fe700 1 --2- 172.21.15.119:0/1005400448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3e130 0x55c096f48000 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5de8007f00 tx=0x7f5de8011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=814c1e7b539c10b9 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.400+0000 7f5dd6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:19:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.401+0000 7f5dd6ffd700 1 -- 172.21.15.119:0/1005400448 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8016070 con 0x55c096f3e130 2023-12-07T10:19:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.401+0000 7f5e1199bb80 1 -- 172.21.15.119:0/1005400448 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c096f31ab0 con 0x55c096f3e130 2023-12-07T10:19:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.401+0000 7f5dd6ffd700 1 -- 172.21.15.119:0/1005400448 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f5de8010700 con 0x55c096f3e130 2023-12-07T10:19:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.401+0000 7f5dd6ffd700 1 -- 172.21.15.119:0/1005400448 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5de80108c0 con 0x55c096f3e130 2023-12-07T10:19:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.401+0000 7f5e1199bb80 10 client.15360 fetch_fsmap learned FSMap version 29 2023-12-07T10:19:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.401+0000 7f5e1199bb80 1 -- 172.21.15.119:0/1005400448 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c096cbbd90 con 0x55c096f3e130 2023-12-07T10:19:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.401+0000 7f5dd6ffd700 1 -- 172.21.15.119:0/1005400448 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c096f31ab0 con 0x55c096f3e130 2023-12-07T10:19:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.401+0000 7f5dd6ffd700 1 -- 172.21.15.119:0/1005400448 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f5de801bbb0 con 0x55c096f3e130 2023-12-07T10:19:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.401+0000 7f5e1199bb80 10 client.15360 fetch_fsmap finished waiting for FSMap version 29 2023-12-07T10:19:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.401+0000 7f5e1199bb80 10 client.15360 resolve_mds: resolved name 'f' to daemon mds.f 2023-12-07T10:19:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.402+0000 7f5e1199bb80 20 client.15360 populate_metadata read hostname 'smithi119' 2023-12-07T10:19:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.402+0000 7f5e1199bb80 1 --2- 172.21.15.119:0/1005400448 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x55c096f53d90 0x55c096f527d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.402+0000 7f5e1199bb80 4 client.15360 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:19:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.402+0000 7f5e1199bb80 1 -- 172.21.15.119:0/1005400448 --> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c096f0d750 con 0x55c096f53d90 2023-12-07T10:19:58.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.402+0000 7f5df6ffd700 1 --2- 172.21.15.119:0/1005400448 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x55c096f53d90 0x55c096f527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.403+0000 7f5df6ffd700 1 --2- 172.21.15.119:0/1005400448 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x55c096f53d90 0x55c096f527d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d458e536df00d134 server_cookie=1e198d144694a5f4 in_seq=0 out_seq=0 2023-12-07T10:19:58.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.403+0000 7f5dd6ffd700 10 client.15360 ms_handle_connect on v2:172.21.15.159:6834/2406380792 2023-12-07T10:19:58.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.403+0000 7f5dd6ffd700 1 -- 172.21.15.119:0/1005400448 <== mds.3 v2:172.21.15.159:6834/2406380792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c096f0d750 con 0x55c096f53d90 2023-12-07T10:19:58.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.403+0000 7f5dd6ffd700 10 client.15360 handle_command_reply: tid=0 2023-12-07T10:19:58.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.404+0000 7f5e1199bb80 1 client.15360 shutdown 2023-12-07T10:19:58.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.404+0000 7f5e1199bb80 1 -- 172.21.15.119:0/1005400448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3e130 msgr2=0x55c096f48000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.404+0000 7f5e1199bb80 1 --2- 172.21.15.119:0/1005400448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3e130 0x55c096f48000 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5de8007f00 tx=0x7f5de8011720 comp rx=0 tx=0).stop 2023-12-07T10:19:58.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.404+0000 7f5e1199bb80 1 -- 172.21.15.119:0/1005400448 shutdown_connections 2023-12-07T10:19:58.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.404+0000 7f5e1199bb80 1 --2- 172.21.15.119:0/1005400448 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x55c096f53d90 0x55c096f527d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.404+0000 7f5e1199bb80 1 --2- 172.21.15.119:0/1005400448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c096f3f980 0x55c096f4f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.404+0000 7f5e1199bb80 1 --2- 172.21.15.119:0/1005400448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3e130 0x55c096f48000 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.404+0000 7f5dd6ffd700 0 client.15360 ms_handle_reset on v2:172.21.15.159:6834/2406380792 2023-12-07T10:19:58.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.404+0000 7f5e1199bb80 1 --2- 172.21.15.119:0/1005400448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c096f3ea90 0x55c096f48540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.404+0000 7f5e1199bb80 1 -- 172.21.15.119:0/1005400448 >> 172.21.15.119:0/1005400448 conn(0x55c096e6d310 msgr2=0x55c096f39990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:58.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.404+0000 7f5e1199bb80 1 -- 172.21.15.119:0/1005400448 shutdown_connections 2023-12-07T10:19:58.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.404+0000 7f5e1199bb80 1 -- 172.21.15.119:0/1005400448 wait complete. 2023-12-07T10:19:58.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.404+0000 7f5e1199bb80 20 client.15360 trim_cache size 0 max 16384 2023-12-07T10:19:58.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.423+0000 7f5e1199bb80 1 Processor -- start 2023-12-07T10:19:58.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.423+0000 7f5e1199bb80 1 -- start start 2023-12-07T10:19:58.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.423+0000 7f5e1199bb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c096f3e130 0x55c096f3fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.423+0000 7f5e1199bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c096f3ea90 0x55c096f4f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.423+0000 7f5e1199bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3f980 0x55c096f471e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.423+0000 7f5e1199bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c096d118d0 con 0x55c096f3f980 2023-12-07T10:19:58.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.423+0000 7f5e1199bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c096f3f800 con 0x55c096f3ea90 2023-12-07T10:19:58.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.423+0000 7f5e1199bb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c096f50700 con 0x55c096f3e130 2023-12-07T10:19:58.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.423+0000 7f5df7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3f980 0x55c096f471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.423+0000 7f5df7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3f980 0x55c096f471e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51126/0 (socket says 172.21.15.119:51126) 2023-12-07T10:19:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.423+0000 7f5df77fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c096f3e130 0x55c096f3fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.423+0000 7f5df7fff700 1 -- 172.21.15.119:0/2380233991 learned_addr learned my addr 172.21.15.119:0/2380233991 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.423+0000 7f5df6ffd700 1 --2- 172.21.15.119:0/2380233991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c096f3ea90 0x55c096f4f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.424+0000 7f5df7fff700 1 -- 172.21.15.119:0/2380233991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c096f3e130 msgr2=0x55c096f3fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.424+0000 7f5df7fff700 1 --2- 172.21.15.119:0/2380233991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c096f3e130 0x55c096f3fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.424+0000 7f5df7fff700 1 -- 172.21.15.119:0/2380233991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c096f3ea90 msgr2=0x55c096f4f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.424+0000 7f5df7fff700 1 --2- 172.21.15.119:0/2380233991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c096f3ea90 0x55c096f4f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.424+0000 7f5df7fff700 1 -- 172.21.15.119:0/2380233991 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c096ca4c10 con 0x55c096f3f980 2023-12-07T10:19:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.424+0000 7f5df6ffd700 1 --2- 172.21.15.119:0/2380233991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c096f3ea90 0x55c096f4f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:19:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.424+0000 7f5df77fe700 1 --2- 172.21.15.119:0/2380233991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c096f3e130 0x55c096f3fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.424+0000 7f5df7fff700 1 --2- 172.21.15.119:0/2380233991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3f980 0x55c096f471e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5df0007810 tx=0x7f5df000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343f5b3f860297bf server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.424+0000 7f5dde7fc700 1 -- 172.21.15.119:0/2380233991 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df000e070 con 0x55c096f3f980 2023-12-07T10:19:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.424+0000 7f5dde7fc700 1 -- 172.21.15.119:0/2380233991 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5df0009390 con 0x55c096f3f980 2023-12-07T10:19:58.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.424+0000 7f5dde7fc700 1 -- 172.21.15.119:0/2380233991 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df0025070 con 0x55c096f3f980 2023-12-07T10:19:58.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.425+0000 7f5e1199bb80 1 -- 172.21.15.119:0/2380233991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3f980 msgr2=0x55c096f471e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.425+0000 7f5e1199bb80 1 --2- 172.21.15.119:0/2380233991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3f980 0x55c096f471e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5df0007810 tx=0x7f5df000e000 comp rx=0 tx=0).stop 2023-12-07T10:19:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.425+0000 7f5e1199bb80 1 -- 172.21.15.119:0/2380233991 shutdown_connections 2023-12-07T10:19:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.425+0000 7f5e1199bb80 1 --2- 172.21.15.119:0/2380233991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3f980 0x55c096f471e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.425+0000 7f5e1199bb80 1 --2- 172.21.15.119:0/2380233991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c096f3e130 0x55c096f3fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.425+0000 7f5e1199bb80 1 --2- 172.21.15.119:0/2380233991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c096f3ea90 0x55c096f4f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.425+0000 7f5e1199bb80 1 -- 172.21.15.119:0/2380233991 >> 172.21.15.119:0/2380233991 conn(0x55c096e6d310 msgr2=0x55c096f45ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.425+0000 7f5e1199bb80 1 -- 172.21.15.119:0/2380233991 shutdown_connections 2023-12-07T10:19:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.425+0000 7f5e1199bb80 1 -- 172.21.15.119:0/2380233991 wait complete. 2023-12-07T10:19:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.426+0000 7f5e1199bb80 1 -- start start 2023-12-07T10:19:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.426+0000 7f5e1199bb80 1 Processor -- start 2023-12-07T10:19:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.426+0000 7f5e1199bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c096f3e130 0x55c096f4b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.427+0000 7f5e1199bb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c096f3ea90 0x55c096f4bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.427+0000 7f5e1199bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3f980 0x55c096f4ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.427+0000 7f5e1199bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c096f4f160 con 0x55c096f3f980 2023-12-07T10:19:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.427+0000 7f5e1199bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c096f4f2d0 con 0x55c096f3e130 2023-12-07T10:19:58.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.427+0000 7f5e1199bb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c096f4f490 con 0x55c096f3ea90 2023-12-07T10:19:58.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.427+0000 7f5e1199bb80 10 client.0 _ll_register_callbacks cb 0x55c096e338f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:19:58.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.427+0000 7f5df7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3f980 0x55c096f4ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.427+0000 7f5df6ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c096f3ea90 0x55c096f4bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.427+0000 7f5df77fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c096f3e130 0x55c096f4b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.427+0000 7f5df7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3f980 0x55c096f4ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51142/0 (socket says 172.21.15.119:51142) 2023-12-07T10:19:58.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.427+0000 7f5df6ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c096f3ea90 0x55c096f4bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:53676/0 (socket says 172.21.15.119:53676) 2023-12-07T10:19:58.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.427+0000 7f5df7fff700 1 -- 172.21.15.119:0/2390303054 learned_addr learned my addr 172.21.15.119:0/2390303054 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:58.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.427+0000 7f5df7fff700 1 -- 172.21.15.119:0/2390303054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c096f3ea90 msgr2=0x55c096f4bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.427+0000 7f5df7fff700 1 --2- 172.21.15.119:0/2390303054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c096f3ea90 0x55c096f4bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.427+0000 7f5df7fff700 1 -- 172.21.15.119:0/2390303054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c096f3e130 msgr2=0x55c096f4b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.427+0000 7f5df7fff700 1 --2- 172.21.15.119:0/2390303054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c096f3e130 0x55c096f4b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.427+0000 7f5df7fff700 1 -- 172.21.15.119:0/2390303054 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c096cbbd90 con 0x55c096f3f980 2023-12-07T10:19:58.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.427+0000 7f5df77fe700 1 --2- 172.21.15.119:0/2390303054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c096f3e130 0x55c096f4b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:58.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.427+0000 7f5df6ffd700 1 --2- 172.21.15.119:0/2390303054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c096f3ea90 0x55c096f4bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:58.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.428+0000 7f5df7fff700 1 --2- 172.21.15.119:0/2390303054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3f980 0x55c096f4ec20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009ac0 tx=0x7f5df0020b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=227df9f86f58678 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:58.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.428+0000 7f5dd6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:19:58.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.428+0000 7f5dd6ffd700 1 -- 172.21.15.119:0/2390303054 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df0025030 con 0x55c096f3f980 2023-12-07T10:19:58.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.428+0000 7f5dd6ffd700 1 -- 172.21.15.119:0/2390303054 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f5df0034850 con 0x55c096f3f980 2023-12-07T10:19:58.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.428+0000 7f5e1199bb80 1 -- 172.21.15.119:0/2390303054 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5dec09b470 con 0x55c096f3f980 2023-12-07T10:19:58.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.428+0000 7f5dd6ffd700 1 -- 172.21.15.119:0/2390303054 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5df0020e90 con 0x55c096f3f980 2023-12-07T10:19:58.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.428+0000 7f5e1199bb80 10 client.15366 fetch_fsmap learned FSMap version 29 2023-12-07T10:19:58.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.428+0000 7f5e1199bb80 1 -- 172.21.15.119:0/2390303054 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f5df810a2d0 con 0x55c096f3f980 2023-12-07T10:19:58.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.428+0000 7f5dd6ffd700 1 -- 172.21.15.119:0/2390303054 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dec09b470 con 0x55c096f3f980 2023-12-07T10:19:58.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.428+0000 7f5dd6ffd700 1 -- 172.21.15.119:0/2390303054 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f5df003e070 con 0x55c096f3f980 2023-12-07T10:19:58.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.428+0000 7f5e1199bb80 10 client.15366 fetch_fsmap finished waiting for FSMap version 29 2023-12-07T10:19:58.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.428+0000 7f5e1199bb80 10 client.15366 resolve_mds: resolved name 'f' to daemon mds.f 2023-12-07T10:19:58.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.428+0000 7f5e1199bb80 20 client.15366 populate_metadata read hostname 'smithi119' 2023-12-07T10:19:58.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.429+0000 7f5e1199bb80 1 --2- 172.21.15.119:0/2390303054 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x55c096fae700 0x55c096faab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.429+0000 7f5e1199bb80 4 client.15366 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-07T10:19:58.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.429+0000 7f5e1199bb80 1 -- 172.21.15.119:0/2390303054 --> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c096e58f80 con 0x55c096fae700 2023-12-07T10:19:58.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.429+0000 7f5df77fe700 1 --2- 172.21.15.119:0/2390303054 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x55c096fae700 0x55c096faab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.429+0000 7f5dd6ffd700 10 client.15366 ms_handle_connect on v2:172.21.15.159:6834/2406380792 2023-12-07T10:19:58.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.429+0000 7f5df77fe700 1 --2- 172.21.15.119:0/2390303054 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x55c096fae700 0x55c096faab70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=596d7f55e1d13330 server_cookie=bf340dc4681a599e in_seq=0 out_seq=0 2023-12-07T10:19:58.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.430+0000 7f5dd6ffd700 1 -- 172.21.15.119:0/2390303054 <== mds.3 v2:172.21.15.159:6834/2406380792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x55c096e58f80 con 0x55c096fae700 2023-12-07T10:19:58.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.430+0000 7f5dd6ffd700 10 client.15366 handle_command_reply: tid=0 2023-12-07T10:19:58.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.430+0000 7f5e1199bb80 1 client.15366 shutdown 2023-12-07T10:19:58.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.430+0000 7f5e1199bb80 1 -- 172.21.15.119:0/2390303054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3f980 msgr2=0x55c096f4ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.430+0000 7f5e1199bb80 1 --2- 172.21.15.119:0/2390303054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3f980 0x55c096f4ec20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009ac0 tx=0x7f5df0020b40 comp rx=0 tx=0).stop 2023-12-07T10:19:58.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.430+0000 7f5e1199bb80 1 -- 172.21.15.119:0/2390303054 shutdown_connections 2023-12-07T10:19:58.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.430+0000 7f5e1199bb80 1 --2- 172.21.15.119:0/2390303054 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x55c096fae700 0x55c096faab70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.430+0000 7f5e1199bb80 1 --2- 172.21.15.119:0/2390303054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c096f3f980 0x55c096f4ec20 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.430+0000 7f5e1199bb80 1 --2- 172.21.15.119:0/2390303054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c096f3e130 0x55c096f4b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.430+0000 7f5dd6ffd700 0 client.15366 ms_handle_reset on v2:172.21.15.159:6834/2406380792 2023-12-07T10:19:58.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.430+0000 7f5e1199bb80 1 --2- 172.21.15.119:0/2390303054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c096f3ea90 0x55c096f4bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.430+0000 7f5e1199bb80 1 -- 172.21.15.119:0/2390303054 >> 172.21.15.119:0/2390303054 conn(0x55c096e6d310 msgr2=0x55c096f471e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:58.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.430+0000 7f5e1199bb80 1 -- 172.21.15.119:0/2390303054 shutdown_connections 2023-12-07T10:19:58.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.430+0000 7f5e1199bb80 1 -- 172.21.15.119:0/2390303054 wait complete. 2023-12-07T10:19:58.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.430+0000 7f5e1199bb80 20 client.15366 trim_cache size 0 max 16384 2023-12-07T10:19:58.437 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:19:58.437 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "a4c642c6-94e8-11ee-95a2-87774f69a715", 2023-12-07T10:19:58.437 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 3, 2023-12-07T10:19:58.437 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 24505, 2023-12-07T10:19:58.438 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-12-07T10:19:58.438 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-12-07T10:19:58.438 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-12-07T10:19:58.438 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 106.693269937, 2023-12-07T10:19:58.438 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 29, 2023-12-07T10:19:58.438 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 93, 2023-12-07T10:19:58.438 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 91, 2023-12-07T10:19:58.438 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 160.470193964 2023-12-07T10:19:58.438 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:19:58.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.431+0000 7f5dfdfff700 1 -- 172.21.15.119:0/835592251 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5de00a1b10 msgr2=0x7f5de00a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.431+0000 7f5dfdfff700 1 --2- 172.21.15.119:0/835592251 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5de00a1b10 0x7f5de00a3fd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009fd0 tx=0x7f5df000cf20 comp rx=0 tx=0).stop 2023-12-07T10:19:58.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.431+0000 7f5dfdfff700 1 -- 172.21.15.119:0/835592251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df8102730 msgr2=0x7f5df8199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.431+0000 7f5dfdfff700 1 --2- 172.21.15.119:0/835592251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df8102730 0x7f5df8199d30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00d980 tx=0x7f5dec00dc90 comp rx=0 tx=0).stop 2023-12-07T10:19:58.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.431+0000 7f5dfdfff700 1 -- 172.21.15.119:0/835592251 shutdown_connections 2023-12-07T10:19:58.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.431+0000 7f5dfdfff700 1 --2- 172.21.15.119:0/835592251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5df81030d0 0x7f5df81a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.431+0000 7f5dfdfff700 1 --2- 172.21.15.119:0/835592251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5df8101540 0x7f5df81997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.431+0000 7f5dfdfff700 1 --2- 172.21.15.119:0/835592251 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5de00a1b10 0x7f5de00a3fd0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.431+0000 7f5dfdfff700 1 --2- 172.21.15.119:0/835592251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df8102730 0x7f5df8199d30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.431+0000 7f5dfdfff700 1 -- 172.21.15.119:0/835592251 >> 172.21.15.119:0/835592251 conn(0x7f5df80fcab0 msgr2=0x7f5df8073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:58.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.431+0000 7f5dfdfff700 1 -- 172.21.15.119:0/835592251 shutdown_connections 2023-12-07T10:19:58.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.431+0000 7f5dfdfff700 1 -- 172.21.15.119:0/835592251 wait complete. 2023-12-07T10:19:58.452 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-07T10:19:58.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.647+0000 7f68b75ff700 1 -- 172.21.15.119:0/4062394810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b0102730 msgr2=0x7f68b0102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.647+0000 7f68b75ff700 1 --2- 172.21.15.119:0/4062394810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b0102730 0x7f68b0102b90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f68a00099c0 tx=0x7f68a0009cd0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.647+0000 7f68b75ff700 1 -- 172.21.15.119:0/4062394810 shutdown_connections 2023-12-07T10:19:58.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.647+0000 7f68b75ff700 1 --2- 172.21.15.119:0/4062394810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68b01030d0 0x7f68b0109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.647+0000 7f68b75ff700 1 --2- 172.21.15.119:0/4062394810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68b0101540 0x7f68b0101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.647+0000 7f68b75ff700 1 --2- 172.21.15.119:0/4062394810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b0102730 0x7f68b0102b90 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.647+0000 7f68b75ff700 1 -- 172.21.15.119:0/4062394810 >> 172.21.15.119:0/4062394810 conn(0x7f68b00fcab0 msgr2=0x7f68b00feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:58.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.647+0000 7f68b75ff700 1 -- 172.21.15.119:0/4062394810 shutdown_connections 2023-12-07T10:19:58.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.647+0000 7f68b75ff700 1 -- 172.21.15.119:0/4062394810 wait complete. 2023-12-07T10:19:58.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.648+0000 7f68b75ff700 1 Processor -- start 2023-12-07T10:19:58.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.648+0000 7f68b75ff700 1 -- start start 2023-12-07T10:19:58.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.649+0000 7f68b75ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68b0101540 0x7f68b01996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.649+0000 7f68b75ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b0102730 0x7f68b0199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.649+0000 7f68b75ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68b01030d0 0x7f68b01a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.649+0000 7f68b75ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b01a11b0 con 0x7f68b0102730 2023-12-07T10:19:58.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.649+0000 7f68b75ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b01a12f0 con 0x7f68b01030d0 2023-12-07T10:19:58.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.649+0000 7f68b75ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b01a1480 con 0x7f68b0101540 2023-12-07T10:19:58.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.649+0000 7f68b4b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b0102730 0x7f68b0199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.649+0000 7f68b4b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b0102730 0x7f68b0199c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51170/0 (socket says 172.21.15.119:51170) 2023-12-07T10:19:58.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.649+0000 7f68b4b9a700 1 -- 172.21.15.119:0/9112927 learned_addr learned my addr 172.21.15.119:0/9112927 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:58.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.649+0000 7f68b539b700 1 --2- 172.21.15.119:0/9112927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68b0101540 0x7f68b01996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.649+0000 7f68b5b9c700 1 --2- 172.21.15.119:0/9112927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68b01030d0 0x7f68b01a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.649+0000 7f68b4b9a700 1 -- 172.21.15.119:0/9112927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68b0101540 msgr2=0x7f68b01996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.649+0000 7f68b4b9a700 1 --2- 172.21.15.119:0/9112927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68b0101540 0x7f68b01996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.649+0000 7f68b4b9a700 1 -- 172.21.15.119:0/9112927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68b01030d0 msgr2=0x7f68b01a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.649+0000 7f68b4b9a700 1 --2- 172.21.15.119:0/9112927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68b01030d0 0x7f68b01a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.649+0000 7f68b4b9a700 1 -- 172.21.15.119:0/9112927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a00095f0 con 0x7f68b0102730 2023-12-07T10:19:58.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.649+0000 7f68b539b700 1 --2- 172.21.15.119:0/9112927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68b0101540 0x7f68b01996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:19:58.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.650+0000 7f68b4b9a700 1 --2- 172.21.15.119:0/9112927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b0102730 0x7f68b0199c10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f68a0006010 tx=0x7f68a000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:58.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.650+0000 7f68a67fc700 1 -- 172.21.15.119:0/9112927 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a001d070 con 0x7f68b0102730 2023-12-07T10:19:58.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.650+0000 7f68a67fc700 1 -- 172.21.15.119:0/9112927 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68a0005030 con 0x7f68b0102730 2023-12-07T10:19:58.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.650+0000 7f68b75ff700 1 -- 172.21.15.119:0/9112927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b01a1700 con 0x7f68b0102730 2023-12-07T10:19:58.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.650+0000 7f68a67fc700 1 -- 172.21.15.119:0/9112927 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a0004bb0 con 0x7f68b0102730 2023-12-07T10:19:58.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.650+0000 7f68b75ff700 1 -- 172.21.15.119:0/9112927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b01a1b80 con 0x7f68b0102730 2023-12-07T10:19:58.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.653+0000 7f68a67fc700 1 -- 172.21.15.119:0/9112927 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f68a00051a0 con 0x7f68b0102730 2023-12-07T10:19:58.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.654+0000 7f68cafc1b80 1 Processor -- start 2023-12-07T10:19:58.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.654+0000 7f68cafc1b80 1 -- start start 2023-12-07T10:19:58.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.655+0000 7f68a67fc700 1 --2- 172.21.15.119:0/9112927 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f689c0a1b90 0x7f689c0a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.655+0000 7f68cafc1b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea53c3130 0x555ea53c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.655+0000 7f68cafc1b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ea53c3a90 0x555ea53c43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.655+0000 7f68cafc1b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ea53c4980 0x555ea53c4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.655+0000 7f68cafc1b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea5337770 con 0x555ea53c4980 2023-12-07T10:19:58.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.655+0000 7f68cafc1b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea531e750 con 0x555ea53c3a90 2023-12-07T10:19:58.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.655+0000 7f68cafc1b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea531c070 con 0x555ea53c3130 2023-12-07T10:19:58.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.655+0000 7f68a67fc700 1 -- 172.21.15.119:0/9112927 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f68a00ce590 con 0x7f68b0102730 2023-12-07T10:19:58.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.655+0000 7f68b539b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea53c3130 0x555ea53c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.655+0000 7f68b4b9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ea53c3a90 0x555ea53c43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.655+0000 7f68b4b9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ea53c3a90 0x555ea53c43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:52964/0 (socket says 172.21.15.119:52964) 2023-12-07T10:19:58.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.655+0000 7f68b539b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea53c3130 0x555ea53c3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:53720/0 (socket says 172.21.15.119:53720) 2023-12-07T10:19:58.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.655+0000 7f68b4b9a700 1 -- 172.21.15.119:0/2029090879 learned_addr learned my addr 172.21.15.119:0/2029090879 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:58.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.655+0000 7f68b5b9c700 1 --2- 172.21.15.119:0/9112927 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f689c0a1b90 0x7f689c0a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.655+0000 7f68b5b9c700 1 --2- 172.21.15.119:0/2029090879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ea53c4980 0x555ea53c4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.655+0000 7f68b4b9a700 1 -- 172.21.15.119:0/2029090879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea53c3130 msgr2=0x555ea53c3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.655+0000 7f68b4b9a700 1 --2- 172.21.15.119:0/2029090879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea53c3130 0x555ea53c3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.655+0000 7f68b4b9a700 1 -- 172.21.15.119:0/2029090879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ea53c4980 msgr2=0x555ea53c4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.655+0000 7f68b4b9a700 1 --2- 172.21.15.119:0/2029090879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ea53c4980 0x555ea53c4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.655+0000 7f68b4b9a700 1 -- 172.21.15.119:0/2029090879 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ea51945e0 con 0x555ea53c3a90 2023-12-07T10:19:58.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.656+0000 7f68b4b9a700 1 --2- 172.21.15.119:0/2029090879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ea53c3a90 0x555ea53c43a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f68a00a3160 tx=0x7f68a009b290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3bc93d1f42afcda server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:58.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.656+0000 7f68b5b9c700 1 --2- 172.21.15.119:0/9112927 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f689c0a1b90 0x7f689c0a4050 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x555ea53c4160 tx=0x7f68a8009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:58.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.656+0000 7f689b7fe700 1 -- 172.21.15.119:0/2029090879 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a007ef20 con 0x555ea53c3a90 2023-12-07T10:19:58.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.656+0000 7f689b7fe700 1 -- 172.21.15.119:0/2029090879 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68a007e070 con 0x555ea53c3a90 2023-12-07T10:19:58.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.656+0000 7f689b7fe700 1 -- 172.21.15.119:0/2029090879 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a0099540 con 0x555ea53c3a90 2023-12-07T10:19:58.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.657+0000 7f68cafc1b80 1 -- 172.21.15.119:0/2029090879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ea53c3a90 msgr2=0x555ea53c43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.657+0000 7f68cafc1b80 1 --2- 172.21.15.119:0/2029090879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ea53c3a90 0x555ea53c43a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f68a00a3160 tx=0x7f68a009b290 comp rx=0 tx=0).stop 2023-12-07T10:19:58.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.657+0000 7f68cafc1b80 1 -- 172.21.15.119:0/2029090879 shutdown_connections 2023-12-07T10:19:58.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.657+0000 7f68cafc1b80 1 --2- 172.21.15.119:0/2029090879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ea53c4980 0x555ea53c4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.657+0000 7f68cafc1b80 1 --2- 172.21.15.119:0/2029090879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea53c3130 0x555ea53c3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.657+0000 7f68cafc1b80 1 --2- 172.21.15.119:0/2029090879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ea53c3a90 0x555ea53c43a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.657+0000 7f68cafc1b80 1 -- 172.21.15.119:0/2029090879 >> 172.21.15.119:0/2029090879 conn(0x555ea52f2310 msgr2=0x555ea53bfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:58.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.657+0000 7f68cafc1b80 1 -- 172.21.15.119:0/2029090879 shutdown_connections 2023-12-07T10:19:58.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.657+0000 7f68cafc1b80 1 -- 172.21.15.119:0/2029090879 wait complete. 2023-12-07T10:19:58.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.657+0000 7f68cafc1b80 1 -- start start 2023-12-07T10:19:58.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.658+0000 7f68cafc1b80 1 Processor -- start 2023-12-07T10:19:58.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.658+0000 7f68cafc1b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ea53c3130 0x555ea53ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.658+0000 7f68cafc1b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea53c3a90 0x555ea53cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.658+0000 7f68cafc1b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ea53c4980 0x555ea53d4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.658+0000 7f68cafc1b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea5317960 con 0x555ea53c3130 2023-12-07T10:19:58.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.658+0000 7f68cafc1b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea52d1510 con 0x555ea53c4980 2023-12-07T10:19:58.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.658+0000 7f68cafc1b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea52d11b0 con 0x555ea53c3a90 2023-12-07T10:19:58.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.658+0000 7f68cafc1b80 10 client.0 _ll_register_callbacks cb 0x555ea52b88f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:19:58.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.658+0000 7f68b4b9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea53c3a90 0x555ea53cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.658+0000 7f68b539b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ea53c3130 0x555ea53ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.658+0000 7f68b4b9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea53c3a90 0x555ea53cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:53736/0 (socket says 172.21.15.119:53736) 2023-12-07T10:19:58.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.658+0000 7f68b4b9a700 1 -- 172.21.15.119:0/1886752377 learned_addr learned my addr 172.21.15.119:0/1886752377 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:58.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.659+0000 7f68b5b9c700 1 --2- 172.21.15.119:0/1886752377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ea53c4980 0x555ea53d4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.659+0000 7f68b4b9a700 1 -- 172.21.15.119:0/1886752377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ea53c4980 msgr2=0x555ea53d4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.659+0000 7f68b4b9a700 1 --2- 172.21.15.119:0/1886752377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ea53c4980 0x555ea53d4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.659+0000 7f68b4b9a700 1 -- 172.21.15.119:0/1886752377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ea53c3130 msgr2=0x555ea53ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.659+0000 7f68b4b9a700 1 --2- 172.21.15.119:0/1886752377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ea53c3130 0x555ea53ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.659+0000 7f68b4b9a700 1 -- 172.21.15.119:0/1886752377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b01a1b80 con 0x555ea53c3a90 2023-12-07T10:19:58.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.659+0000 7f68b4b9a700 1 --2- 172.21.15.119:0/1886752377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea53c3a90 0x555ea53cd140 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f68a000b580 tx=0x7f68a009f570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfa4511efb0521ba server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:58.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.659+0000 7f6898ff9700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:19:58.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.659+0000 7f6898ff9700 1 -- 172.21.15.119:0/1886752377 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a009ba90 con 0x555ea53c3a90 2023-12-07T10:19:58.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.659+0000 7f68cafc1b80 1 -- 172.21.15.119:0/1886752377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ea53b6ab0 con 0x555ea53c3a90 2023-12-07T10:19:58.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.659+0000 7f6898ff9700 1 -- 172.21.15.119:0/1886752377 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f68a009b5f0 con 0x555ea53c3a90 2023-12-07T10:19:58.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.660+0000 7f6898ff9700 1 -- 172.21.15.119:0/1886752377 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68a00b1050 con 0x555ea53c3a90 2023-12-07T10:19:58.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.660+0000 7f68cafc1b80 10 client.25387 fetch_fsmap learned FSMap version 29 2023-12-07T10:19:58.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.660+0000 7f68cafc1b80 1 -- 172.21.15.119:0/1886752377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555ea5140d90 con 0x555ea53c3a90 2023-12-07T10:19:58.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.660+0000 7f6898ff9700 1 -- 172.21.15.119:0/1886752377 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555ea53b6ab0 con 0x555ea53c3a90 2023-12-07T10:19:58.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.660+0000 7f6898ff9700 1 -- 172.21.15.119:0/1886752377 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f68a00a3200 con 0x555ea53c3a90 2023-12-07T10:19:58.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.660+0000 7f68cafc1b80 10 client.25387 fetch_fsmap finished waiting for FSMap version 29 2023-12-07T10:19:58.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.660+0000 7f68cafc1b80 10 client.25387 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-07T10:19:58.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.660+0000 7f68cafc1b80 20 client.25387 populate_metadata read hostname 'smithi119' 2023-12-07T10:19:58.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.660+0000 7f68cafc1b80 1 --2- 172.21.15.119:0/1886752377 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x555ea53d8340 0x555ea53d7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.660+0000 7f68cafc1b80 4 client.25387 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:19:58.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.660+0000 7f68cafc1b80 1 -- 172.21.15.119:0/1886752377 --> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ea5392750 con 0x555ea53d8340 2023-12-07T10:19:58.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.660+0000 7f68b539b700 1 --2- 172.21.15.119:0/1886752377 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x555ea53d8340 0x555ea53d7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.661+0000 7f6898ff9700 10 client.25387 ms_handle_connect on v2:172.21.15.159:6836/3106097520 2023-12-07T10:19:58.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.661+0000 7f68b539b700 1 --2- 172.21.15.119:0/1886752377 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x555ea53d8340 0x555ea53d7150 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=52e389b2653f4c5 server_cookie=1c39cd11bca5dbcf in_seq=0 out_seq=0 2023-12-07T10:19:58.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.662+0000 7f6898ff9700 1 -- 172.21.15.119:0/1886752377 <== mds.2 v2:172.21.15.159:6836/3106097520 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555ea5392750 con 0x555ea53d8340 2023-12-07T10:19:58.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.662+0000 7f6898ff9700 10 client.25387 handle_command_reply: tid=0 2023-12-07T10:19:58.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.662+0000 7f68cafc1b80 1 client.25387 shutdown 2023-12-07T10:19:58.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.662+0000 7f68cafc1b80 1 -- 172.21.15.119:0/1886752377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea53c3a90 msgr2=0x555ea53cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.662+0000 7f68cafc1b80 1 --2- 172.21.15.119:0/1886752377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea53c3a90 0x555ea53cd140 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f68a000b580 tx=0x7f68a009f570 comp rx=0 tx=0).stop 2023-12-07T10:19:58.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.662+0000 7f68cafc1b80 1 -- 172.21.15.119:0/1886752377 shutdown_connections 2023-12-07T10:19:58.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.662+0000 7f68cafc1b80 1 --2- 172.21.15.119:0/1886752377 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x555ea53d8340 0x555ea53d7150 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.662+0000 7f68cafc1b80 1 --2- 172.21.15.119:0/1886752377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ea53c4980 0x555ea53d4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.662+0000 7f68cafc1b80 1 --2- 172.21.15.119:0/1886752377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ea53c3130 0x555ea53ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.662+0000 7f68cafc1b80 1 --2- 172.21.15.119:0/1886752377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea53c3a90 0x555ea53cd140 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.662+0000 7f6898ff9700 0 client.25387 ms_handle_reset on v2:172.21.15.159:6836/3106097520 2023-12-07T10:19:58.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.662+0000 7f68cafc1b80 1 -- 172.21.15.119:0/1886752377 >> 172.21.15.119:0/1886752377 conn(0x555ea52f2310 msgr2=0x555ea53be990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:58.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.662+0000 7f68cafc1b80 1 -- 172.21.15.119:0/1886752377 shutdown_connections 2023-12-07T10:19:58.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.662+0000 7f68cafc1b80 1 -- 172.21.15.119:0/1886752377 wait complete. 2023-12-07T10:19:58.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.662+0000 7f68cafc1b80 20 client.25387 trim_cache size 0 max 16384 2023-12-07T10:19:58.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.681+0000 7f68cafc1b80 1 Processor -- start 2023-12-07T10:19:58.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.681+0000 7f68cafc1b80 1 -- start start 2023-12-07T10:19:58.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.681+0000 7f68cafc1b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ea53c3130 0x555ea53bf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.681+0000 7f68cafc1b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ea53c3a90 0x555ea53d4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.681+0000 7f68cafc1b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea53c4980 0x555ea53e0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.681+0000 7f68cafc1b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea51968d0 con 0x555ea53c3130 2023-12-07T10:19:58.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.681+0000 7f68cafc1b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea5392750 con 0x555ea53c3a90 2023-12-07T10:19:58.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.681+0000 7f68cafc1b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea5313a50 con 0x555ea53c4980 2023-12-07T10:19:58.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.682+0000 7f68b539b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ea53c3130 0x555ea53bf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.682+0000 7f68b539b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ea53c3130 0x555ea53bf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51206/0 (socket says 172.21.15.119:51206) 2023-12-07T10:19:58.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.682+0000 7f68b539b700 1 -- 172.21.15.119:0/3035835905 learned_addr learned my addr 172.21.15.119:0/3035835905 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:58.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.682+0000 7f68b5b9c700 1 --2- 172.21.15.119:0/3035835905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea53c4980 0x555ea53e0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.682+0000 7f68b4b9a700 1 --2- 172.21.15.119:0/3035835905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ea53c3a90 0x555ea53d4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.682+0000 7f68b539b700 1 -- 172.21.15.119:0/3035835905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea53c4980 msgr2=0x555ea53e0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.682+0000 7f68b539b700 1 --2- 172.21.15.119:0/3035835905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea53c4980 0x555ea53e0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.682+0000 7f68b539b700 1 -- 172.21.15.119:0/3035835905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ea53c3a90 msgr2=0x555ea53d4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.682+0000 7f68b539b700 1 --2- 172.21.15.119:0/3035835905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ea53c3a90 0x555ea53d4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.682+0000 7f68b539b700 1 -- 172.21.15.119:0/3035835905 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ea5129c10 con 0x555ea53c3130 2023-12-07T10:19:58.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.682+0000 7f68b539b700 1 --2- 172.21.15.119:0/3035835905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ea53c3130 0x555ea53bf6b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x555ea52b9f50 tx=0x7f68ac009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1807df88fbb15d server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:58.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.682+0000 7f689bfff700 1 -- 172.21.15.119:0/3035835905 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac017030 con 0x555ea53c3130 2023-12-07T10:19:58.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.683+0000 7f689bfff700 1 -- 172.21.15.119:0/3035835905 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68ac018040 con 0x555ea53c3130 2023-12-07T10:19:58.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.683+0000 7f689bfff700 1 -- 172.21.15.119:0/3035835905 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac005400 con 0x555ea53c3130 2023-12-07T10:19:58.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.683+0000 7f68cafc1b80 1 -- 172.21.15.119:0/3035835905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ea53c3130 msgr2=0x555ea53bf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.683+0000 7f68cafc1b80 1 --2- 172.21.15.119:0/3035835905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ea53c3130 0x555ea53bf6b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x555ea52b9f50 tx=0x7f68ac009260 comp rx=0 tx=0).stop 2023-12-07T10:19:58.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.683+0000 7f68cafc1b80 1 -- 172.21.15.119:0/3035835905 shutdown_connections 2023-12-07T10:19:58.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.683+0000 7f68cafc1b80 1 --2- 172.21.15.119:0/3035835905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea53c4980 0x555ea53e0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.683+0000 7f68cafc1b80 1 --2- 172.21.15.119:0/3035835905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ea53c3130 0x555ea53bf6b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.683+0000 7f68cafc1b80 1 --2- 172.21.15.119:0/3035835905 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ea53c3a90 0x555ea53d4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.683+0000 7f68cafc1b80 1 -- 172.21.15.119:0/3035835905 >> 172.21.15.119:0/3035835905 conn(0x555ea52f2310 msgr2=0x555ea53cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:58.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.683+0000 7f68cafc1b80 1 -- 172.21.15.119:0/3035835905 shutdown_connections 2023-12-07T10:19:58.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.683+0000 7f68cafc1b80 1 -- 172.21.15.119:0/3035835905 wait complete. 2023-12-07T10:19:58.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.684+0000 7f68cafc1b80 1 -- start start 2023-12-07T10:19:58.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.684+0000 7f68cafc1b80 1 Processor -- start 2023-12-07T10:19:58.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.684+0000 7f68cafc1b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ea53c3130 0x555ea53cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.684+0000 7f68cafc1b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ea53c3a90 0x555ea53d4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.684+0000 7f68cafc1b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea53c4980 0x555ea53cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.684+0000 7f68cafc1b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea53cf8a0 con 0x555ea53c3130 2023-12-07T10:19:58.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.684+0000 7f68cafc1b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea542e460 con 0x555ea53c3a90 2023-12-07T10:19:58.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.684+0000 7f68cafc1b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea542e620 con 0x555ea53c4980 2023-12-07T10:19:58.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.684+0000 7f68cafc1b80 10 client.0 _ll_register_callbacks cb 0x555ea52b88f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:19:58.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.685+0000 7f68b4b9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ea53c3a90 0x555ea53d4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.685+0000 7f68b539b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ea53c3130 0x555ea53cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.685+0000 7f68b4b9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ea53c3a90 0x555ea53d4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:52992/0 (socket says 172.21.15.119:52992) 2023-12-07T10:19:58.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.685+0000 7f68b5b9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea53c4980 0x555ea53cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.685+0000 7f68b4b9a700 1 -- 172.21.15.119:0/1851043551 learned_addr learned my addr 172.21.15.119:0/1851043551 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:58.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.685+0000 7f68b4b9a700 1 -- 172.21.15.119:0/1851043551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea53c4980 msgr2=0x555ea53cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.685+0000 7f68b4b9a700 1 --2- 172.21.15.119:0/1851043551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea53c4980 0x555ea53cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.685+0000 7f68b4b9a700 1 -- 172.21.15.119:0/1851043551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ea53c3130 msgr2=0x555ea53cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.685+0000 7f68b4b9a700 1 --2- 172.21.15.119:0/1851043551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ea53c3130 0x555ea53cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.685+0000 7f68b4b9a700 1 -- 172.21.15.119:0/1851043551 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ac012450 con 0x555ea53c3a90 2023-12-07T10:19:58.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.685+0000 7f68b539b700 1 --2- 172.21.15.119:0/1851043551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ea53c3130 0x555ea53cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:58.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.685+0000 7f68b4b9a700 1 --2- 172.21.15.119:0/1851043551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ea53c3a90 0x555ea53d4150 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f68a009b090 tx=0x7f68a00036f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b4e28616341ee58 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:58.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.685+0000 7f6898ff9700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:19:58.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.685+0000 7f6898ff9700 1 -- 172.21.15.119:0/1851043551 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a00b1070 con 0x555ea53c3a90 2023-12-07T10:19:58.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.685+0000 7f68cafc1b80 1 -- 172.21.15.119:0/1851043551 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f68a007e070 con 0x555ea53c3a90 2023-12-07T10:19:58.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.685+0000 7f6898ff9700 1 -- 172.21.15.119:0/1851043551 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f68a009ba90 con 0x555ea53c3a90 2023-12-07T10:19:58.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.686+0000 7f6898ff9700 1 -- 172.21.15.119:0/1851043551 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68a00a3700 con 0x555ea53c3a90 2023-12-07T10:19:58.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.686+0000 7f68cafc1b80 10 client.25409 fetch_fsmap learned FSMap version 29 2023-12-07T10:19:58.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.686+0000 7f68cafc1b80 1 -- 172.21.15.119:0/1851043551 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555ea5167200 con 0x555ea53c3a90 2023-12-07T10:19:58.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.686+0000 7f6898ff9700 1 -- 172.21.15.119:0/1851043551 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68a007e070 con 0x555ea53c3a90 2023-12-07T10:19:58.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.686+0000 7f6898ff9700 1 -- 172.21.15.119:0/1851043551 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f68a009a980 con 0x555ea53c3a90 2023-12-07T10:19:58.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.686+0000 7f68cafc1b80 10 client.25409 fetch_fsmap finished waiting for FSMap version 29 2023-12-07T10:19:58.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.686+0000 7f68cafc1b80 10 client.25409 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-07T10:19:58.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.686+0000 7f68cafc1b80 20 client.25409 populate_metadata read hostname 'smithi119' 2023-12-07T10:19:58.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.686+0000 7f68cafc1b80 1 --2- 172.21.15.119:0/1851043551 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x555ea5433550 0x555ea542f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.686+0000 7f68cafc1b80 4 client.25409 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-07T10:19:58.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.686+0000 7f68cafc1b80 1 -- 172.21.15.119:0/1851043551 --> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555ea52ddf80 con 0x555ea5433550 2023-12-07T10:19:58.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.686+0000 7f68b539b700 1 --2- 172.21.15.119:0/1851043551 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x555ea5433550 0x555ea542f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.687+0000 7f68b539b700 1 --2- 172.21.15.119:0/1851043551 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x555ea5433550 0x555ea542f990 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d2a01a02cf9a7668 server_cookie=20549d3c24ebcbce in_seq=0 out_seq=0 2023-12-07T10:19:58.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.687+0000 7f6898ff9700 10 client.25409 ms_handle_connect on v2:172.21.15.159:6836/3106097520 2023-12-07T10:19:58.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.687+0000 7f6898ff9700 1 -- 172.21.15.119:0/1851043551 <== mds.2 v2:172.21.15.159:6836/3106097520 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x555ea52ddf80 con 0x555ea5433550 2023-12-07T10:19:58.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.687+0000 7f6898ff9700 10 client.25409 handle_command_reply: tid=0 2023-12-07T10:19:58.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.687+0000 7f68cafc1b80 1 client.25409 shutdown 2023-12-07T10:19:58.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.687+0000 7f68cafc1b80 1 -- 172.21.15.119:0/1851043551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ea53c3a90 msgr2=0x555ea53d4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.687+0000 7f68cafc1b80 1 --2- 172.21.15.119:0/1851043551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ea53c3a90 0x555ea53d4150 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f68a009b090 tx=0x7f68a00036f0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.687+0000 7f68cafc1b80 1 -- 172.21.15.119:0/1851043551 shutdown_connections 2023-12-07T10:19:58.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.687+0000 7f68cafc1b80 1 --2- 172.21.15.119:0/1851043551 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x555ea5433550 0x555ea542f990 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.687+0000 7f68cafc1b80 1 --2- 172.21.15.119:0/1851043551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea53c4980 0x555ea53cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.687+0000 7f68cafc1b80 1 --2- 172.21.15.119:0/1851043551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ea53c3130 0x555ea53cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.687+0000 7f68cafc1b80 1 --2- 172.21.15.119:0/1851043551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ea53c3a90 0x555ea53d4150 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.687+0000 7f68cafc1b80 1 -- 172.21.15.119:0/1851043551 >> 172.21.15.119:0/1851043551 conn(0x555ea52f2310 msgr2=0x555ea53e0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:58.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.687+0000 7f6898ff9700 0 client.25409 ms_handle_reset on v2:172.21.15.159:6836/3106097520 2023-12-07T10:19:58.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.687+0000 7f68cafc1b80 1 -- 172.21.15.119:0/1851043551 shutdown_connections 2023-12-07T10:19:58.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.687+0000 7f68cafc1b80 1 -- 172.21.15.119:0/1851043551 wait complete. 2023-12-07T10:19:58.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.688+0000 7f68cafc1b80 20 client.25409 trim_cache size 0 max 16384 2023-12-07T10:19:58.694 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:19:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "a4c642c6-94e8-11ee-95a2-87774f69a715", 2023-12-07T10:19:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 2, 2023-12-07T10:19:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 24517, 2023-12-07T10:19:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-12-07T10:19:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-12-07T10:19:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-12-07T10:19:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 107.95705848, 2023-12-07T10:19:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 29, 2023-12-07T10:19:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 93, 2023-12-07T10:19:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 91, 2023-12-07T10:19:58.695 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 158.41220824300001 2023-12-07T10:19:58.695 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:19:58.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.688+0000 7f68b75ff700 1 -- 172.21.15.119:0/9112927 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f689c0a1b90 msgr2=0x7f689c0a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.688+0000 7f68b75ff700 1 --2- 172.21.15.119:0/9112927 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f689c0a1b90 0x7f689c0a4050 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x555ea53c4160 tx=0x7f68a8009920 comp rx=0 tx=0).stop 2023-12-07T10:19:58.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.688+0000 7f68b75ff700 1 -- 172.21.15.119:0/9112927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b0102730 msgr2=0x7f68b0199c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.688+0000 7f68b75ff700 1 --2- 172.21.15.119:0/9112927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b0102730 0x7f68b0199c10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f68a0006010 tx=0x7f68a000f690 comp rx=0 tx=0).stop 2023-12-07T10:19:58.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.688+0000 7f68b75ff700 1 -- 172.21.15.119:0/9112927 shutdown_connections 2023-12-07T10:19:58.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.688+0000 7f68b75ff700 1 --2- 172.21.15.119:0/9112927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68b01030d0 0x7f68b01a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.688+0000 7f68b75ff700 1 --2- 172.21.15.119:0/9112927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68b0101540 0x7f68b01996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.688+0000 7f68b75ff700 1 --2- 172.21.15.119:0/9112927 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f689c0a1b90 0x7f689c0a4050 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.688+0000 7f68b75ff700 1 --2- 172.21.15.119:0/9112927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b0102730 0x7f68b0199c10 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.688+0000 7f68b75ff700 1 -- 172.21.15.119:0/9112927 >> 172.21.15.119:0/9112927 conn(0x7f68b00fcab0 msgr2=0x7f68b0073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:58.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.688+0000 7f68b75ff700 1 -- 172.21.15.119:0/9112927 shutdown_connections 2023-12-07T10:19:58.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.688+0000 7f68b75ff700 1 -- 172.21.15.119:0/9112927 wait complete. 2023-12-07T10:19:58.709 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2023-12-07T10:19:58.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.906+0000 7f4978bff700 1 -- 172.21.15.119:0/3783565926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4974101550 msgr2=0x7f4974101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.906+0000 7f4978bff700 1 --2- 172.21.15.119:0/3783565926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4974101550 0x7f4974101970 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f49640098f0 tx=0x7f4964009c00 comp rx=0 tx=0).stop 2023-12-07T10:19:58.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.906+0000 7f4978bff700 1 -- 172.21.15.119:0/3783565926 shutdown_connections 2023-12-07T10:19:58.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.906+0000 7f4978bff700 1 --2- 172.21.15.119:0/3783565926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49741030e0 0x7f4974109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.906+0000 7f4978bff700 1 --2- 172.21.15.119:0/3783565926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4974101550 0x7f4974101970 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.906+0000 7f4978bff700 1 --2- 172.21.15.119:0/3783565926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4974102740 0x7f4974102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.906+0000 7f4978bff700 1 -- 172.21.15.119:0/3783565926 >> 172.21.15.119:0/3783565926 conn(0x7f49740fcae0 msgr2=0x7f49740fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:58.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.906+0000 7f4978bff700 1 -- 172.21.15.119:0/3783565926 shutdown_connections 2023-12-07T10:19:58.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.906+0000 7f4978bff700 1 -- 172.21.15.119:0/3783565926 wait complete. 2023-12-07T10:19:58.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.907+0000 7f4978bff700 1 Processor -- start 2023-12-07T10:19:58.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.907+0000 7f4978bff700 1 -- start start 2023-12-07T10:19:58.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.907+0000 7f4978bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4974101550 0x7f4974197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.907+0000 7f4978bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4974102740 0x7f4974197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.907+0000 7f4978bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49741030e0 0x7f497419ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.907+0000 7f4978bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497419f060 con 0x7f4974102740 2023-12-07T10:19:58.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.907+0000 7f4978bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497419f1a0 con 0x7f49741030e0 2023-12-07T10:19:58.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.907+0000 7f4978bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497419f330 con 0x7f4974101550 2023-12-07T10:19:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.907+0000 7f4971d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4974102740 0x7f4974197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.908+0000 7f4971d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4974102740 0x7f4974197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51224/0 (socket says 172.21.15.119:51224) 2023-12-07T10:19:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.908+0000 7f4971d9b700 1 -- 172.21.15.119:0/2158921196 learned_addr learned my addr 172.21.15.119:0/2158921196 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.908+0000 7f497259c700 1 --2- 172.21.15.119:0/2158921196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4974101550 0x7f4974197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.908+0000 7f4972d9d700 1 --2- 172.21.15.119:0/2158921196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49741030e0 0x7f497419ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.908+0000 7f4971d9b700 1 -- 172.21.15.119:0/2158921196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4974101550 msgr2=0x7f4974197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.908+0000 7f4971d9b700 1 --2- 172.21.15.119:0/2158921196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4974101550 0x7f4974197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.908+0000 7f4971d9b700 1 -- 172.21.15.119:0/2158921196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49741030e0 msgr2=0x7f497419ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.908+0000 7f4971d9b700 1 --2- 172.21.15.119:0/2158921196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49741030e0 0x7f497419ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.908+0000 7f4971d9b700 1 -- 172.21.15.119:0/2158921196 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496c009d00 con 0x7f4974102740 2023-12-07T10:19:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.908+0000 7f4972d9d700 1 --2- 172.21.15.119:0/2158921196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49741030e0 0x7f497419ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:19:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.908+0000 7f4971d9b700 1 --2- 172.21.15.119:0/2158921196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4974102740 0x7f4974197ac0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f496800e9a0 tx=0x7f496800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.908+0000 7f49637fe700 1 -- 172.21.15.119:0/2158921196 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968009790 con 0x7f4974102740 2023-12-07T10:19:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.908+0000 7f49637fe700 1 -- 172.21.15.119:0/2158921196 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f496800cbd0 con 0x7f4974102740 2023-12-07T10:19:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.908+0000 7f4978bff700 1 -- 172.21.15.119:0/2158921196 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4964009520 con 0x7f4974102740 2023-12-07T10:19:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.908+0000 7f49637fe700 1 -- 172.21.15.119:0/2158921196 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968018680 con 0x7f4974102740 2023-12-07T10:19:58.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.909+0000 7f4978bff700 1 -- 172.21.15.119:0/2158921196 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f497419f990 con 0x7f4974102740 2023-12-07T10:19:58.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.911+0000 7f49637fe700 1 -- 172.21.15.119:0/2158921196 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f496800cd40 con 0x7f4974102740 2023-12-07T10:19:58.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.913+0000 7f498c60cb80 1 Processor -- start 2023-12-07T10:19:58.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.913+0000 7f49637fe700 1 --2- 172.21.15.119:0/2158921196 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f495c0a1b10 0x7f495c0a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.913+0000 7f49637fe700 1 -- 172.21.15.119:0/2158921196 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f4968014070 con 0x7f4974102740 2023-12-07T10:19:58.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.913+0000 7f498c60cb80 1 -- start start 2023-12-07T10:19:58.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.913+0000 7f497259c700 1 --2- 172.21.15.119:0/2158921196 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f495c0a1b10 0x7f495c0a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.913+0000 7f498c60cb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cdb7350540 0x55cdb7350960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.913+0000 7f498c60cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdb7350ea0 0x55cdb73517b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.913+0000 7f498c60cb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cdb7351d90 0x55cdb7352210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.913+0000 7f498c60cb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb72c0770 con 0x55cdb7350540 2023-12-07T10:19:58.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.913+0000 7f498c60cb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb72a7750 con 0x55cdb7351d90 2023-12-07T10:19:58.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.913+0000 7f498c60cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb72a5070 con 0x55cdb7350ea0 2023-12-07T10:19:58.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.914+0000 7f497259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cdb7350540 0x55cdb7350960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.914+0000 7f497259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cdb7350540 0x55cdb7350960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51240/0 (socket says 172.21.15.119:51240) 2023-12-07T10:19:58.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.914+0000 7f497259c700 1 -- 172.21.15.119:0/3909883536 learned_addr learned my addr 172.21.15.119:0/3909883536 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:58.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.914+0000 7f4972d9d700 1 --2- 172.21.15.119:0/3909883536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cdb7351d90 0x55cdb7352210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.914+0000 7f4971d9b700 1 --2- 172.21.15.119:0/3909883536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdb7350ea0 0x55cdb73517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.914+0000 7f497259c700 1 --2- 172.21.15.119:0/2158921196 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f495c0a1b10 0x7f495c0a3fd0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f496400c010 tx=0x7f496400f780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:58.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.914+0000 7f4972d9d700 1 -- 172.21.15.119:0/3909883536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdb7350ea0 msgr2=0x55cdb73517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.914+0000 7f4972d9d700 1 --2- 172.21.15.119:0/3909883536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdb7350ea0 0x55cdb73517b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.914+0000 7f4972d9d700 1 -- 172.21.15.119:0/3909883536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cdb7350540 msgr2=0x55cdb7350960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.914+0000 7f4972d9d700 1 --2- 172.21.15.119:0/3909883536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cdb7350540 0x55cdb7350960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.914+0000 7f4972d9d700 1 -- 172.21.15.119:0/3909883536 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdb711d5e0 con 0x55cdb7351d90 2023-12-07T10:19:58.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.914+0000 7f497259c700 1 --2- 172.21.15.119:0/3909883536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cdb7350540 0x55cdb7350960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:58.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.915+0000 7f4972d9d700 1 --2- 172.21.15.119:0/3909883536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cdb7351d90 0x55cdb7352210 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f496c0094d0 tx=0x7f496c0099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=838b09adac26e720 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:58.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.915+0000 7f4960ff9700 1 -- 172.21.15.119:0/3909883536 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c00e070 con 0x55cdb7351d90 2023-12-07T10:19:58.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.915+0000 7f4960ff9700 1 -- 172.21.15.119:0/3909883536 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f496c00f040 con 0x55cdb7351d90 2023-12-07T10:19:58.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.915+0000 7f4960ff9700 1 -- 172.21.15.119:0/3909883536 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c0149e0 con 0x55cdb7351d90 2023-12-07T10:19:58.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.916+0000 7f498c60cb80 1 -- 172.21.15.119:0/3909883536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cdb7351d90 msgr2=0x55cdb7352210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.916+0000 7f498c60cb80 1 --2- 172.21.15.119:0/3909883536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cdb7351d90 0x55cdb7352210 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f496c0094d0 tx=0x7f496c0099e0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.916+0000 7f498c60cb80 1 -- 172.21.15.119:0/3909883536 shutdown_connections 2023-12-07T10:19:58.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.916+0000 7f498c60cb80 1 --2- 172.21.15.119:0/3909883536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cdb7351d90 0x55cdb7352210 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.916+0000 7f498c60cb80 1 --2- 172.21.15.119:0/3909883536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cdb7350540 0x55cdb7350960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.916+0000 7f498c60cb80 1 --2- 172.21.15.119:0/3909883536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdb7350ea0 0x55cdb73517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.916+0000 7f498c60cb80 1 -- 172.21.15.119:0/3909883536 >> 172.21.15.119:0/3909883536 conn(0x55cdb727b310 msgr2=0x55cdb7348b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:58.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.916+0000 7f498c60cb80 1 -- 172.21.15.119:0/3909883536 shutdown_connections 2023-12-07T10:19:58.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.916+0000 7f498c60cb80 1 -- 172.21.15.119:0/3909883536 wait complete. 2023-12-07T10:19:58.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.916+0000 7f498c60cb80 1 -- start start 2023-12-07T10:19:58.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.917+0000 7f498c60cb80 1 Processor -- start 2023-12-07T10:19:58.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.917+0000 7f498c60cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdb7350540 0x55cdb735a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.917+0000 7f498c60cb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cdb7350ea0 0x55cdb735a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.917+0000 7f498c60cb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cdb7351d90 0x55cdb73618a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.917+0000 7f498c60cb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb72a0960 con 0x55cdb7351d90 2023-12-07T10:19:58.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.917+0000 7f498c60cb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb725a510 con 0x55cdb7350ea0 2023-12-07T10:19:58.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.917+0000 7f498c60cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb725a1b0 con 0x55cdb7350540 2023-12-07T10:19:58.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.917+0000 7f498c60cb80 10 client.0 _ll_register_callbacks cb 0x55cdb72418f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:19:58.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.917+0000 7f497259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdb7350540 0x55cdb735a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.917+0000 7f4972d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cdb7351d90 0x55cdb73618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.917+0000 7f497259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdb7350540 0x55cdb735a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:53822/0 (socket says 172.21.15.119:53822) 2023-12-07T10:19:58.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.917+0000 7f497259c700 1 -- 172.21.15.119:0/3645396078 learned_addr learned my addr 172.21.15.119:0/3645396078 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:58.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.917+0000 7f4971d9b700 1 --2- 172.21.15.119:0/3645396078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cdb7350ea0 0x55cdb735a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.918+0000 7f497259c700 1 -- 172.21.15.119:0/3645396078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cdb7350ea0 msgr2=0x55cdb735a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.918+0000 7f497259c700 1 --2- 172.21.15.119:0/3645396078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cdb7350ea0 0x55cdb735a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.918+0000 7f497259c700 1 -- 172.21.15.119:0/3645396078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cdb7351d90 msgr2=0x55cdb73618a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.918+0000 7f497259c700 1 --2- 172.21.15.119:0/3645396078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cdb7351d90 0x55cdb73618a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.918+0000 7f497259c700 1 -- 172.21.15.119:0/3645396078 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f497410a2e0 con 0x55cdb7350540 2023-12-07T10:19:58.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.918+0000 7f497259c700 1 --2- 172.21.15.119:0/3645396078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdb7350540 0x55cdb735a320 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4964000c00 tx=0x7f4964009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fe0be37d84e5899 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:58.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.918+0000 7f4959ffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:19:58.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.918+0000 7f4959ffb700 1 -- 172.21.15.119:0/3645396078 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4964005300 con 0x55cdb7350540 2023-12-07T10:19:58.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.918+0000 7f498c60cb80 1 -- 172.21.15.119:0/3645396078 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cdb733fab0 con 0x55cdb7350540 2023-12-07T10:19:58.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.918+0000 7f4959ffb700 1 -- 172.21.15.119:0/3645396078 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f4964005460 con 0x55cdb7350540 2023-12-07T10:19:58.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.919+0000 7f4959ffb700 1 -- 172.21.15.119:0/3645396078 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4964005620 con 0x55cdb7350540 2023-12-07T10:19:58.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.919+0000 7f498c60cb80 10 client.25405 fetch_fsmap learned FSMap version 29 2023-12-07T10:19:58.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.919+0000 7f498c60cb80 1 -- 172.21.15.119:0/3645396078 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f496c0091b0 con 0x55cdb7350540 2023-12-07T10:19:58.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.919+0000 7f4959ffb700 1 -- 172.21.15.119:0/3645396078 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cdb733fab0 con 0x55cdb7350540 2023-12-07T10:19:58.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.919+0000 7f4959ffb700 1 -- 172.21.15.119:0/3645396078 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f4964027610 con 0x55cdb7350540 2023-12-07T10:19:58.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.919+0000 7f498c60cb80 10 client.25405 fetch_fsmap finished waiting for FSMap version 29 2023-12-07T10:19:58.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.919+0000 7f498c60cb80 10 client.25405 resolve_mds: resolved name 'g' to daemon mds.g 2023-12-07T10:19:58.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.919+0000 7f498c60cb80 20 client.25405 populate_metadata read hostname 'smithi119' 2023-12-07T10:19:58.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.919+0000 7f498c60cb80 1 --2- 172.21.15.119:0/3645396078 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55cdb7365fd0 0x55cdb7364a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.919+0000 7f498c60cb80 4 client.25405 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:19:58.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.919+0000 7f498c60cb80 1 -- 172.21.15.119:0/3645396078 --> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cdb731b750 con 0x55cdb7365fd0 2023-12-07T10:19:58.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.919+0000 7f4971d9b700 1 --2- 172.21.15.119:0/3645396078 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55cdb7365fd0 0x55cdb7364a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.920+0000 7f4971d9b700 1 --2- 172.21.15.119:0/3645396078 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55cdb7365fd0 0x55cdb7364a90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4ed4934b90d38dc8 server_cookie=101eac8fdb2be3d0 in_seq=0 out_seq=0 2023-12-07T10:19:58.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.920+0000 7f4959ffb700 10 client.25405 ms_handle_connect on v2:172.21.15.119:6838/1920762602 2023-12-07T10:19:58.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.920+0000 7f4959ffb700 1 -- 172.21.15.119:0/3645396078 <== mds.1 v2:172.21.15.119:6838/1920762602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cdb731b750 con 0x55cdb7365fd0 2023-12-07T10:19:58.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.920+0000 7f4959ffb700 10 client.25405 handle_command_reply: tid=0 2023-12-07T10:19:58.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.921+0000 7f498c60cb80 1 client.25405 shutdown 2023-12-07T10:19:58.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.921+0000 7f498c60cb80 1 -- 172.21.15.119:0/3645396078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdb7350540 msgr2=0x55cdb735a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.921+0000 7f498c60cb80 1 --2- 172.21.15.119:0/3645396078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdb7350540 0x55cdb735a320 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4964000c00 tx=0x7f4964009c00 comp rx=0 tx=0).stop 2023-12-07T10:19:58.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.921+0000 7f498c60cb80 1 -- 172.21.15.119:0/3645396078 shutdown_connections 2023-12-07T10:19:58.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.921+0000 7f498c60cb80 1 --2- 172.21.15.119:0/3645396078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cdb7351d90 0x55cdb73618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.921+0000 7f498c60cb80 1 --2- 172.21.15.119:0/3645396078 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55cdb7365fd0 0x55cdb7364a90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.921+0000 7f498c60cb80 1 --2- 172.21.15.119:0/3645396078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdb7350540 0x55cdb735a320 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.921+0000 7f498c60cb80 1 --2- 172.21.15.119:0/3645396078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cdb7350ea0 0x55cdb735a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.921+0000 7f4959ffb700 0 client.25405 ms_handle_reset on v2:172.21.15.119:6838/1920762602 2023-12-07T10:19:58.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.921+0000 7f498c60cb80 1 -- 172.21.15.119:0/3645396078 >> 172.21.15.119:0/3645396078 conn(0x55cdb727b310 msgr2=0x55cdb7347990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:58.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.921+0000 7f498c60cb80 1 -- 172.21.15.119:0/3645396078 shutdown_connections 2023-12-07T10:19:58.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.921+0000 7f498c60cb80 1 -- 172.21.15.119:0/3645396078 wait complete. 2023-12-07T10:19:58.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.921+0000 7f498c60cb80 20 client.25405 trim_cache size 0 max 16384 2023-12-07T10:19:58.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.940+0000 7f498c60cb80 1 Processor -- start 2023-12-07T10:19:58.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.940+0000 7f498c60cb80 1 -- start start 2023-12-07T10:19:58.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.940+0000 7f498c60cb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cdb7350540 0x55cdb7348570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.940+0000 7f498c60cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdb7350ea0 0x55cdb7348ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.940+0000 7f498c60cb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cdb7351d90 0x55cdb7361860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.940+0000 7f498c60cb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb711f8d0 con 0x55cdb7350540 2023-12-07T10:19:58.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.940+0000 7f498c60cb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb723c360 con 0x55cdb7351d90 2023-12-07T10:19:58.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.940+0000 7f498c60cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb7343700 con 0x55cdb7350ea0 2023-12-07T10:19:58.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.940+0000 7f497259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cdb7350540 0x55cdb7348570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.940+0000 7f497259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cdb7350540 0x55cdb7348570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51270/0 (socket says 172.21.15.119:51270) 2023-12-07T10:19:58.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.940+0000 7f497259c700 1 -- 172.21.15.119:0/49991682 learned_addr learned my addr 172.21.15.119:0/49991682 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:58.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.940+0000 7f4972d9d700 1 --2- 172.21.15.119:0/49991682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cdb7351d90 0x55cdb7361860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.940+0000 7f4971d9b700 1 --2- 172.21.15.119:0/49991682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdb7350ea0 0x55cdb7348ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.940+0000 7f497259c700 1 -- 172.21.15.119:0/49991682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdb7350ea0 msgr2=0x55cdb7348ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.941+0000 7f497259c700 1 --2- 172.21.15.119:0/49991682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdb7350ea0 0x55cdb7348ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.941+0000 7f497259c700 1 -- 172.21.15.119:0/49991682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cdb7351d90 msgr2=0x55cdb7361860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.941+0000 7f497259c700 1 --2- 172.21.15.119:0/49991682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cdb7351d90 0x55cdb7361860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.941+0000 7f497259c700 1 -- 172.21.15.119:0/49991682 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdb70c9d90 con 0x55cdb7350540 2023-12-07T10:19:58.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.941+0000 7f4972d9d700 1 --2- 172.21.15.119:0/49991682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cdb7351d90 0x55cdb7361860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:19:58.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.941+0000 7f4971d9b700 1 --2- 172.21.15.119:0/49991682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdb7350ea0 0x55cdb7348ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:19:58.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.941+0000 7f497259c700 1 --2- 172.21.15.119:0/49991682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cdb7350540 0x55cdb7348570 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4964005e40 tx=0x7f4964005ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8562e07ea777f46e server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:58.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.941+0000 7f49617fa700 1 -- 172.21.15.119:0/49991682 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4964027d90 con 0x55cdb7350540 2023-12-07T10:19:58.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.941+0000 7f49617fa700 1 -- 172.21.15.119:0/49991682 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4964005100 con 0x55cdb7350540 2023-12-07T10:19:58.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.941+0000 7f49617fa700 1 -- 172.21.15.119:0/49991682 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4964003aa0 con 0x55cdb7350540 2023-12-07T10:19:58.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.942+0000 7f498c60cb80 1 -- 172.21.15.119:0/49991682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cdb7350540 msgr2=0x55cdb7348570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.942+0000 7f498c60cb80 1 --2- 172.21.15.119:0/49991682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cdb7350540 0x55cdb7348570 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4964005e40 tx=0x7f4964005ba0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.942+0000 7f498c60cb80 1 -- 172.21.15.119:0/49991682 shutdown_connections 2023-12-07T10:19:58.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.942+0000 7f498c60cb80 1 --2- 172.21.15.119:0/49991682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cdb7351d90 0x55cdb7361860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.942+0000 7f498c60cb80 1 --2- 172.21.15.119:0/49991682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cdb7350540 0x55cdb7348570 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.942+0000 7f498c60cb80 1 --2- 172.21.15.119:0/49991682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdb7350ea0 0x55cdb7348ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.942+0000 7f498c60cb80 1 -- 172.21.15.119:0/49991682 >> 172.21.15.119:0/49991682 conn(0x55cdb727b310 msgr2=0x55cdb7359500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:58.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.942+0000 7f498c60cb80 1 -- 172.21.15.119:0/49991682 shutdown_connections 2023-12-07T10:19:58.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.942+0000 7f498c60cb80 1 -- 172.21.15.119:0/49991682 wait complete. 2023-12-07T10:19:58.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.942+0000 7f498c60cb80 1 -- start start 2023-12-07T10:19:58.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.942+0000 7f498c60cb80 1 Processor -- start 2023-12-07T10:19:58.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.943+0000 7f498c60cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdb7350540 0x55cdb735b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.943+0000 7f498c60cb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cdb7350ea0 0x55cdb735bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.943+0000 7f498c60cb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cdb7351d90 0x55cdb7360c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.943+0000 7f498c60cb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb73611b0 con 0x55cdb7350ea0 2023-12-07T10:19:58.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.943+0000 7f498c60cb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb7361320 con 0x55cdb7351d90 2023-12-07T10:19:58.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.943+0000 7f498c60cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb73614e0 con 0x55cdb7350540 2023-12-07T10:19:58.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.943+0000 7f498c60cb80 10 client.0 _ll_register_callbacks cb 0x55cdb72418f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:19:58.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.943+0000 7f4971d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cdb7350ea0 0x55cdb735bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.943+0000 7f4971d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cdb7350ea0 0x55cdb735bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51278/0 (socket says 172.21.15.119:51278) 2023-12-07T10:19:58.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.943+0000 7f4971d9b700 1 -- 172.21.15.119:0/443788312 learned_addr learned my addr 172.21.15.119:0/443788312 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:58.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.943+0000 7f497259c700 1 --2- 172.21.15.119:0/443788312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdb7350540 0x55cdb735b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.943+0000 7f4972d9d700 1 --2- 172.21.15.119:0/443788312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cdb7351d90 0x55cdb7360c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.943+0000 7f4971d9b700 1 -- 172.21.15.119:0/443788312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdb7350540 msgr2=0x55cdb735b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.943+0000 7f4971d9b700 1 --2- 172.21.15.119:0/443788312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdb7350540 0x55cdb735b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.943+0000 7f4971d9b700 1 -- 172.21.15.119:0/443788312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cdb7351d90 msgr2=0x55cdb7360c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:58.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.943+0000 7f4971d9b700 1 --2- 172.21.15.119:0/443788312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cdb7351d90 0x55cdb7360c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.943+0000 7f4971d9b700 1 -- 172.21.15.119:0/443788312 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f496c0091b0 con 0x55cdb7350ea0 2023-12-07T10:19:58.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.944+0000 7f4971d9b700 1 --2- 172.21.15.119:0/443788312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cdb7350ea0 0x55cdb735bf40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f49680cb340 tx=0x7f49680ba040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da96cc725a39214e server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:58.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.944+0000 7f4959ffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:19:58.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.944+0000 7f4959ffb700 1 -- 172.21.15.119:0/443788312 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49680a2cb0 con 0x55cdb7350ea0 2023-12-07T10:19:58.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.944+0000 7f498c60cb80 1 -- 172.21.15.119:0/443788312 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f496c00f040 con 0x55cdb7350ea0 2023-12-07T10:19:58.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.944+0000 7f4959ffb700 1 -- 172.21.15.119:0/443788312 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f4968098cc0 con 0x55cdb7350ea0 2023-12-07T10:19:58.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.944+0000 7f4959ffb700 1 -- 172.21.15.119:0/443788312 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49680850b0 con 0x55cdb7350ea0 2023-12-07T10:19:58.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.944+0000 7f498c60cb80 10 client.15402 fetch_fsmap learned FSMap version 29 2023-12-07T10:19:58.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.944+0000 7f498c60cb80 1 -- 172.21.15.119:0/443788312 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55cdb70b2c10 con 0x55cdb7350ea0 2023-12-07T10:19:58.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.944+0000 7f4959ffb700 1 -- 172.21.15.119:0/443788312 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f496807d020 con 0x55cdb7350ea0 2023-12-07T10:19:58.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.944+0000 7f4959ffb700 1 -- 172.21.15.119:0/443788312 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f4968099ea0 con 0x55cdb7350ea0 2023-12-07T10:19:58.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.944+0000 7f498c60cb80 10 client.15402 fetch_fsmap finished waiting for FSMap version 29 2023-12-07T10:19:58.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.945+0000 7f498c60cb80 10 client.15402 resolve_mds: resolved name 'g' to daemon mds.g 2023-12-07T10:19:58.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.945+0000 7f498c60cb80 20 client.15402 populate_metadata read hostname 'smithi119' 2023-12-07T10:19:58.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.945+0000 7f498c60cb80 1 --2- 172.21.15.119:0/443788312 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55cdb73c0b30 0x55cdb73bcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:58.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.945+0000 7f498c60cb80 4 client.15402 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-07T10:19:58.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.945+0000 7f498c60cb80 1 -- 172.21.15.119:0/443788312 --> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cdb7266f80 con 0x55cdb73c0b30 2023-12-07T10:19:58.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.945+0000 7f497259c700 1 --2- 172.21.15.119:0/443788312 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55cdb73c0b30 0x55cdb73bcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:58.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.945+0000 7f497259c700 1 --2- 172.21.15.119:0/443788312 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55cdb73c0b30 0x55cdb73bcf80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c0aedf45aed070b9 server_cookie=dd6aff8141b0938b in_seq=0 out_seq=0 2023-12-07T10:19:58.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.945+0000 7f4959ffb700 10 client.15402 ms_handle_connect on v2:172.21.15.119:6838/1920762602 2023-12-07T10:19:58.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.945+0000 7f4959ffb700 1 -- 172.21.15.119:0/443788312 <== mds.1 v2:172.21.15.119:6838/1920762602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55cdb7266f80 con 0x55cdb73c0b30 2023-12-07T10:19:58.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.945+0000 7f4959ffb700 10 client.15402 handle_command_reply: tid=0 2023-12-07T10:19:58.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.946+0000 7f498c60cb80 1 client.15402 shutdown 2023-12-07T10:19:58.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.946+0000 7f498c60cb80 1 -- 172.21.15.119:0/443788312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cdb7350ea0 msgr2=0x55cdb735bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.946+0000 7f498c60cb80 1 --2- 172.21.15.119:0/443788312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cdb7350ea0 0x55cdb735bf40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f49680cb340 tx=0x7f49680ba040 comp rx=0 tx=0).stop 2023-12-07T10:19:58.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.946+0000 7f498c60cb80 1 -- 172.21.15.119:0/443788312 shutdown_connections 2023-12-07T10:19:58.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.946+0000 7f498c60cb80 1 --2- 172.21.15.119:0/443788312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cdb7351d90 0x55cdb7360c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.946+0000 7f498c60cb80 1 --2- 172.21.15.119:0/443788312 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55cdb73c0b30 0x55cdb73bcf80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.946+0000 7f498c60cb80 1 --2- 172.21.15.119:0/443788312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdb7350540 0x55cdb735b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.946+0000 7f498c60cb80 1 --2- 172.21.15.119:0/443788312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cdb7350ea0 0x55cdb735bf40 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.946+0000 7f4959ffb700 0 client.15402 ms_handle_reset on v2:172.21.15.119:6838/1920762602 2023-12-07T10:19:58.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.946+0000 7f498c60cb80 1 -- 172.21.15.119:0/443788312 >> 172.21.15.119:0/443788312 conn(0x55cdb727b310 msgr2=0x55cdb7359500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:58.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.946+0000 7f498c60cb80 1 -- 172.21.15.119:0/443788312 shutdown_connections 2023-12-07T10:19:58.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.946+0000 7f498c60cb80 1 -- 172.21.15.119:0/443788312 wait complete. 2023-12-07T10:19:58.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.946+0000 7f498c60cb80 20 client.15402 trim_cache size 0 max 16384 2023-12-07T10:19:58.953 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:19:58.953 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "a4c642c6-94e8-11ee-95a2-87774f69a715", 2023-12-07T10:19:58.953 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 1, 2023-12-07T10:19:58.953 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 24542, 2023-12-07T10:19:58.953 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-12-07T10:19:58.953 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-12-07T10:19:58.953 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-12-07T10:19:58.953 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 109.056773643, 2023-12-07T10:19:58.954 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 29, 2023-12-07T10:19:58.954 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 93, 2023-12-07T10:19:58.954 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 91, 2023-12-07T10:19:58.954 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 150.90459660400001 2023-12-07T10:19:58.954 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:19:58.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.947+0000 7f4978bff700 1 -- 172.21.15.119:0/2158921196 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f495c0a1b10 msgr2=0x7f495c0a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.947+0000 7f4978bff700 1 --2- 172.21.15.119:0/2158921196 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f495c0a1b10 0x7f495c0a3fd0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f496400c010 tx=0x7f496400f780 comp rx=0 tx=0).stop 2023-12-07T10:19:58.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.947+0000 7f4978bff700 1 -- 172.21.15.119:0/2158921196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4974102740 msgr2=0x7f4974197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:58.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.947+0000 7f4978bff700 1 --2- 172.21.15.119:0/2158921196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4974102740 0x7f4974197ac0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f496800e9a0 tx=0x7f496800ecb0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.947+0000 7f4978bff700 1 -- 172.21.15.119:0/2158921196 shutdown_connections 2023-12-07T10:19:58.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.947+0000 7f4978bff700 1 --2- 172.21.15.119:0/2158921196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49741030e0 0x7f497419ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.947+0000 7f4978bff700 1 --2- 172.21.15.119:0/2158921196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4974101550 0x7f4974197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.947+0000 7f4978bff700 1 --2- 172.21.15.119:0/2158921196 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f495c0a1b10 0x7f495c0a3fd0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.947+0000 7f4978bff700 1 --2- 172.21.15.119:0/2158921196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4974102740 0x7f4974197ac0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:58.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.947+0000 7f4978bff700 1 -- 172.21.15.119:0/2158921196 >> 172.21.15.119:0/2158921196 conn(0x7f49740fcae0 msgr2=0x7f4974105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:58.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.947+0000 7f4978bff700 1 -- 172.21.15.119:0/2158921196 shutdown_connections 2023-12-07T10:19:58.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:58.947+0000 7f4978bff700 1 -- 172.21.15.119:0/2158921196 wait complete. 2023-12-07T10:19:58.968 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-07T10:19:59.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.182+0000 7f40a3dff700 1 -- 172.21.15.119:0/2382070084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f409c1048f0 msgr2=0x7f409c106ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:59.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.182+0000 7f40a3dff700 1 --2- 172.21.15.119:0/2382070084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f409c1048f0 0x7f409c106ce0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f408c009990 tx=0x7f408c009ca0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.182+0000 7f40a3dff700 1 -- 172.21.15.119:0/2382070084 shutdown_connections 2023-12-07T10:19:59.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.182+0000 7f40a3dff700 1 --2- 172.21.15.119:0/2382070084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f409c107220 0x7f409c109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.182+0000 7f40a3dff700 1 --2- 172.21.15.119:0/2382070084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f409c101fc0 0x7f409c1043b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.182+0000 7f40a3dff700 1 --2- 172.21.15.119:0/2382070084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f409c1048f0 0x7f409c106ce0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.182+0000 7f40a3dff700 1 -- 172.21.15.119:0/2382070084 >> 172.21.15.119:0/2382070084 conn(0x7f409c0fb9b0 msgr2=0x7f409c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:59.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.182+0000 7f40a3dff700 1 -- 172.21.15.119:0/2382070084 shutdown_connections 2023-12-07T10:19:59.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.182+0000 7f40a3dff700 1 -- 172.21.15.119:0/2382070084 wait complete. 2023-12-07T10:19:59.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.183+0000 7f40a3dff700 1 Processor -- start 2023-12-07T10:19:59.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.183+0000 7f40a3dff700 1 -- start start 2023-12-07T10:19:59.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.184+0000 7f40a3dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f409c101fc0 0x7f409c1018e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:59.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.184+0000 7f40a3dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f409c1048f0 0x7f409c0fff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:59.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.184+0000 7f40a3dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f409c107220 0x7f409c1004b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:59.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.184+0000 7f40a3dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c100c50 con 0x7f409c101fc0 2023-12-07T10:19:59.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.184+0000 7f40a3dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c100dc0 con 0x7f409c107220 2023-12-07T10:19:59.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.184+0000 7f40a3dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c1a1500 con 0x7f409c1048f0 2023-12-07T10:19:59.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.184+0000 7f40a1b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f409c101fc0 0x7f409c1018e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:59.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.184+0000 7f40a239c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f409c107220 0x7f409c1004b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:59.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.184+0000 7f40a139a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f409c1048f0 0x7f409c0fff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:59.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.184+0000 7f40a239c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f409c107220 0x7f409c1004b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:53084/0 (socket says 172.21.15.119:53084) 2023-12-07T10:19:59.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.184+0000 7f40a139a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f409c1048f0 0x7f409c0fff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:53852/0 (socket says 172.21.15.119:53852) 2023-12-07T10:19:59.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.184+0000 7f40a239c700 1 -- 172.21.15.119:0/668564 learned_addr learned my addr 172.21.15.119:0/668564 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:59.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.184+0000 7f40a239c700 1 -- 172.21.15.119:0/668564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f409c1048f0 msgr2=0x7f409c0fff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:59.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.184+0000 7f40a239c700 1 --2- 172.21.15.119:0/668564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f409c1048f0 0x7f409c0fff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.184+0000 7f40a239c700 1 -- 172.21.15.119:0/668564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f409c101fc0 msgr2=0x7f409c1018e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:59.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.184+0000 7f40a239c700 1 --2- 172.21.15.119:0/668564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f409c101fc0 0x7f409c1018e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.185+0000 7f40a239c700 1 -- 172.21.15.119:0/668564 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408c0095f0 con 0x7f409c107220 2023-12-07T10:19:59.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.185+0000 7f40a239c700 1 --2- 172.21.15.119:0/668564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f409c107220 0x7f409c1004b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f409400ef30 tx=0x7f409400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:59.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.185+0000 7f4092ffd700 1 -- 172.21.15.119:0/668564 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4094018070 con 0x7f409c107220 2023-12-07T10:19:59.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.185+0000 7f4092ffd700 1 -- 172.21.15.119:0/668564 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f409400f040 con 0x7f409c107220 2023-12-07T10:19:59.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.185+0000 7f40a3dff700 1 -- 172.21.15.119:0/668564 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f409c1a1730 con 0x7f409c107220 2023-12-07T10:19:59.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.185+0000 7f4092ffd700 1 -- 172.21.15.119:0/668564 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409400cca0 con 0x7f409c107220 2023-12-07T10:19:59.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.185+0000 7f40a3dff700 1 -- 172.21.15.119:0/668564 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409c1a1c40 con 0x7f409c107220 2023-12-07T10:19:59.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.189+0000 7f4092ffd700 1 -- 172.21.15.119:0/668564 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f4094003a00 con 0x7f409c107220 2023-12-07T10:19:59.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.190+0000 7f40b77cbb80 1 Processor -- start 2023-12-07T10:19:59.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.191+0000 7f40b77cbb80 1 -- start start 2023-12-07T10:19:59.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.191+0000 7f40b77cbb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5ab3c4130 0x55e5ab3c4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:59.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.191+0000 7f4092ffd700 1 --2- 172.21.15.119:0/668564 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f40880a1b90 0x7f40880a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:59.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.191+0000 7f40b77cbb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5ab3c4a90 0x55e5ab3c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:59.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.191+0000 7f40b77cbb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5ab3c5980 0x55e5ab3c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:59.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.191+0000 7f40b77cbb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5ab338770 con 0x55e5ab3c5980 2023-12-07T10:19:59.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.191+0000 7f40b77cbb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5ab31f750 con 0x55e5ab3c4a90 2023-12-07T10:19:59.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.191+0000 7f40b77cbb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5ab31d070 con 0x55e5ab3c4130 2023-12-07T10:19:59.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.191+0000 7f4092ffd700 1 -- 172.21.15.119:0/668564 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f40940cd370 con 0x7f409c107220 2023-12-07T10:19:59.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.191+0000 7f40a139a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5ab3c4130 0x55e5ab3c4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:59.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.191+0000 7f40a239c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5ab3c5980 0x55e5ab3c5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:59.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.191+0000 7f40a139a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5ab3c4130 0x55e5ab3c4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:53860/0 (socket says 172.21.15.119:53860) 2023-12-07T10:19:59.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.191+0000 7f40a139a700 1 -- 172.21.15.119:0/815533312 learned_addr learned my addr 172.21.15.119:0/815533312 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:59.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.191+0000 7f40a1b9b700 1 --2- 172.21.15.119:0/668564 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f40880a1b90 0x7f40880a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:59.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.191+0000 7f40a139a700 1 --2- 172.21.15.119:0/815533312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5ab3c4a90 0x55e5ab3c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:59.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.192+0000 7f40a239c700 1 -- 172.21.15.119:0/815533312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5ab3c4130 msgr2=0x55e5ab3c4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:59.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.192+0000 7f40a239c700 1 --2- 172.21.15.119:0/815533312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5ab3c4130 0x55e5ab3c4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.192+0000 7f40a239c700 1 -- 172.21.15.119:0/815533312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5ab3c4a90 msgr2=0x55e5ab3c53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.192+0000 7f40a239c700 1 --2- 172.21.15.119:0/815533312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5ab3c4a90 0x55e5ab3c53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.192+0000 7f40a239c700 1 -- 172.21.15.119:0/815533312 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5ab1955e0 con 0x55e5ab3c5980 2023-12-07T10:19:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.192+0000 7f40a139a700 1 --2- 172.21.15.119:0/815533312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5ab3c4a90 0x55e5ab3c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:19:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.192+0000 7f40a239c700 1 --2- 172.21.15.119:0/815533312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5ab3c5980 0x55e5ab3c5e00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f409400ef00 tx=0x7f4094099fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f2ba96f7a7ed16 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.192+0000 7f40a1b9b700 1 --2- 172.21.15.119:0/668564 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f40880a1b90 0x7f40880a4050 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4098009c60 tx=0x7f4098009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.192+0000 7f4087fff700 1 -- 172.21.15.119:0/815533312 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409407dd00 con 0x55e5ab3c5980 2023-12-07T10:19:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.192+0000 7f4087fff700 1 -- 172.21.15.119:0/815533312 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4094004ad0 con 0x55e5ab3c5980 2023-12-07T10:19:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.192+0000 7f4087fff700 1 -- 172.21.15.119:0/815533312 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40940a2470 con 0x55e5ab3c5980 2023-12-07T10:19:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.193+0000 7f40b77cbb80 1 -- 172.21.15.119:0/815533312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5ab3c5980 msgr2=0x55e5ab3c5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.193+0000 7f40b77cbb80 1 --2- 172.21.15.119:0/815533312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5ab3c5980 0x55e5ab3c5e00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f409400ef00 tx=0x7f4094099fc0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.193+0000 7f40b77cbb80 1 -- 172.21.15.119:0/815533312 shutdown_connections 2023-12-07T10:19:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.193+0000 7f40b77cbb80 1 --2- 172.21.15.119:0/815533312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5ab3c5980 0x55e5ab3c5e00 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.193+0000 7f40b77cbb80 1 --2- 172.21.15.119:0/815533312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5ab3c4130 0x55e5ab3c4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.193+0000 7f40b77cbb80 1 --2- 172.21.15.119:0/815533312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5ab3c4a90 0x55e5ab3c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.193+0000 7f40b77cbb80 1 -- 172.21.15.119:0/815533312 >> 172.21.15.119:0/815533312 conn(0x55e5ab2f3310 msgr2=0x55e5ab3c0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:59.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.193+0000 7f40b77cbb80 1 -- 172.21.15.119:0/815533312 shutdown_connections 2023-12-07T10:19:59.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.193+0000 7f40b77cbb80 1 -- 172.21.15.119:0/815533312 wait complete. 2023-12-07T10:19:59.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.194+0000 7f40b77cbb80 1 -- start start 2023-12-07T10:19:59.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.194+0000 7f40b77cbb80 1 Processor -- start 2023-12-07T10:19:59.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.194+0000 7f40b77cbb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5ab3c4130 0x55e5ab3cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:59.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.194+0000 7f40b77cbb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5ab3c4a90 0x55e5ab3ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:59.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.194+0000 7f40b77cbb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5ab3c5980 0x55e5ab3d54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:59.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.194+0000 7f40b77cbb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5ab318960 con 0x55e5ab3c5980 2023-12-07T10:19:59.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.194+0000 7f40b77cbb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5ab2d2510 con 0x55e5ab3c4a90 2023-12-07T10:19:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.194+0000 7f40b77cbb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5ab2d21b0 con 0x55e5ab3c4130 2023-12-07T10:19:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.194+0000 7f40b77cbb80 10 client.0 _ll_register_callbacks cb 0x55e5ab2b98f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:19:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.195+0000 7f40a1b9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5ab3c4130 0x55e5ab3cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.195+0000 7f40a1b9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5ab3c4130 0x55e5ab3cdf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:53862/0 (socket says 172.21.15.119:53862) 2023-12-07T10:19:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.195+0000 7f40a1b9b700 1 -- 172.21.15.119:0/994997713 learned_addr learned my addr 172.21.15.119:0/994997713 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.195+0000 7f40a239c700 1 --2- 172.21.15.119:0/994997713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5ab3c5980 0x55e5ab3d54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.195+0000 7f40a139a700 1 --2- 172.21.15.119:0/994997713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5ab3c4a90 0x55e5ab3ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.195+0000 7f40a1b9b700 1 -- 172.21.15.119:0/994997713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5ab3c4a90 msgr2=0x55e5ab3ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.195+0000 7f40a1b9b700 1 --2- 172.21.15.119:0/994997713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5ab3c4a90 0x55e5ab3ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.195+0000 7f40a1b9b700 1 -- 172.21.15.119:0/994997713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5ab3c5980 msgr2=0x55e5ab3d54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.195+0000 7f40a1b9b700 1 --2- 172.21.15.119:0/994997713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5ab3c5980 0x55e5ab3d54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.195+0000 7f40a1b9b700 1 -- 172.21.15.119:0/994997713 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409c1a1c40 con 0x55e5ab3c4130 2023-12-07T10:19:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.195+0000 7f40a1b9b700 1 --2- 172.21.15.119:0/994997713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5ab3c4130 0x55e5ab3cdf60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f409800eae0 tx=0x7f40980047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0be6d78891310d1 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.195+0000 7f40857fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:19:59.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.196+0000 7f40857fa700 1 -- 172.21.15.119:0/994997713 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409801c9b0 con 0x55e5ab3c4130 2023-12-07T10:19:59.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.196+0000 7f40b77cbb80 1 -- 172.21.15.119:0/994997713 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e5ab3b7ab0 con 0x55e5ab3c4130 2023-12-07T10:19:59.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.196+0000 7f40857fa700 1 -- 172.21.15.119:0/994997713 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f4098023070 con 0x55e5ab3c4130 2023-12-07T10:19:59.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.196+0000 7f40857fa700 1 -- 172.21.15.119:0/994997713 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f409801cb10 con 0x55e5ab3c4130 2023-12-07T10:19:59.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.196+0000 7f40b77cbb80 10 client.25423 fetch_fsmap learned FSMap version 29 2023-12-07T10:19:59.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.196+0000 7f40b77cbb80 1 -- 172.21.15.119:0/994997713 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e5ab141d90 con 0x55e5ab3c4130 2023-12-07T10:19:59.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.196+0000 7f40857fa700 1 -- 172.21.15.119:0/994997713 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e5ab3b7ab0 con 0x55e5ab3c4130 2023-12-07T10:19:59.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.196+0000 7f40857fa700 1 -- 172.21.15.119:0/994997713 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f409801d410 con 0x55e5ab3c4130 2023-12-07T10:19:59.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.196+0000 7f40b77cbb80 10 client.25423 fetch_fsmap finished waiting for FSMap version 29 2023-12-07T10:19:59.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.196+0000 7f40b77cbb80 10 client.25423 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-07T10:19:59.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.196+0000 7f40b77cbb80 20 client.25423 populate_metadata read hostname 'smithi119' 2023-12-07T10:19:59.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.196+0000 7f40b77cbb80 1 --2- 172.21.15.119:0/994997713 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55e5ab3d9c10 0x55e5ab3d86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:59.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.196+0000 7f40b77cbb80 4 client.25423 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:19:59.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.196+0000 7f40b77cbb80 1 -- 172.21.15.119:0/994997713 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5ab393750 con 0x55e5ab3d9c10 2023-12-07T10:19:59.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.197+0000 7f40a139a700 1 --2- 172.21.15.119:0/994997713 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55e5ab3d9c10 0x55e5ab3d86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:59.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.197+0000 7f40857fa700 10 client.25423 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:19:59.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.197+0000 7f40a139a700 1 --2- 172.21.15.119:0/994997713 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55e5ab3d9c10 0x55e5ab3d86e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28a8efbe338433ed server_cookie=87a90770b5ad0eb7 in_seq=0 out_seq=0 2023-12-07T10:19:59.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.198+0000 7f40857fa700 1 -- 172.21.15.119:0/994997713 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e5ab393750 con 0x55e5ab3d9c10 2023-12-07T10:19:59.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.198+0000 7f40857fa700 10 client.25423 handle_command_reply: tid=0 2023-12-07T10:19:59.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.198+0000 7f40b77cbb80 1 client.25423 shutdown 2023-12-07T10:19:59.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.198+0000 7f40b77cbb80 1 -- 172.21.15.119:0/994997713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5ab3c4130 msgr2=0x55e5ab3cdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:59.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.198+0000 7f40b77cbb80 1 --2- 172.21.15.119:0/994997713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5ab3c4130 0x55e5ab3cdf60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f409800eae0 tx=0x7f40980047c0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.198+0000 7f40b77cbb80 1 -- 172.21.15.119:0/994997713 shutdown_connections 2023-12-07T10:19:59.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.198+0000 7f40b77cbb80 1 --2- 172.21.15.119:0/994997713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5ab3c5980 0x55e5ab3d54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.198+0000 7f40b77cbb80 1 --2- 172.21.15.119:0/994997713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5ab3c4130 0x55e5ab3cdf60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.198+0000 7f40b77cbb80 1 --2- 172.21.15.119:0/994997713 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55e5ab3d9c10 0x55e5ab3d86e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.198+0000 7f40b77cbb80 1 --2- 172.21.15.119:0/994997713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5ab3c4a90 0x55e5ab3ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.198+0000 7f40b77cbb80 1 -- 172.21.15.119:0/994997713 >> 172.21.15.119:0/994997713 conn(0x55e5ab2f3310 msgr2=0x55e5ab3bf990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:59.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.198+0000 7f40857fa700 0 client.25423 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:19:59.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.198+0000 7f40b77cbb80 1 -- 172.21.15.119:0/994997713 shutdown_connections 2023-12-07T10:19:59.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.199+0000 7f40b77cbb80 1 -- 172.21.15.119:0/994997713 wait complete. 2023-12-07T10:19:59.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.199+0000 7f40b77cbb80 20 client.25423 trim_cache size 0 max 16384 2023-12-07T10:19:59.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.217+0000 7f40b77cbb80 1 Processor -- start 2023-12-07T10:19:59.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.217+0000 7f40b77cbb80 1 -- start start 2023-12-07T10:19:59.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.218+0000 7f40b77cbb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5ab3c4130 0x55e5ab3cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:59.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.218+0000 7f40b77cbb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5ab3c4a90 0x55e5ab3d5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:59.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.218+0000 7f40b77cbb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5ab3c5980 0x55e5ab3c8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:59.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.218+0000 7f40b77cbb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5ab1978d0 con 0x55e5ab3c4a90 2023-12-07T10:19:59.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.218+0000 7f40b77cbb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5ab2bdb80 con 0x55e5ab3c4130 2023-12-07T10:19:59.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.218+0000 7f40b77cbb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5ab1a3de0 con 0x55e5ab3c5980 2023-12-07T10:19:59.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.218+0000 7f40a139a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5ab3c4a90 0x55e5ab3d5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:59.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.218+0000 7f40a239c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5ab3c5980 0x55e5ab3c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:59.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.218+0000 7f40a239c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5ab3c5980 0x55e5ab3c8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:53874/0 (socket says 172.21.15.119:53874) 2023-12-07T10:19:59.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.218+0000 7f40a239c700 1 -- 172.21.15.119:0/4254281651 learned_addr learned my addr 172.21.15.119:0/4254281651 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:59.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.218+0000 7f40a1b9b700 1 --2- 172.21.15.119:0/4254281651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5ab3c4130 0x55e5ab3cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:59.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.218+0000 7f40a239c700 1 -- 172.21.15.119:0/4254281651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5ab3c4130 msgr2=0x55e5ab3cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:59.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.218+0000 7f40a239c700 1 --2- 172.21.15.119:0/4254281651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5ab3c4130 0x55e5ab3cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.218+0000 7f40a239c700 1 -- 172.21.15.119:0/4254281651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5ab3c4a90 msgr2=0x55e5ab3d5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:59.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.218+0000 7f40a239c700 1 --2- 172.21.15.119:0/4254281651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5ab3c4a90 0x55e5ab3d5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.218+0000 7f40a239c700 1 -- 172.21.15.119:0/4254281651 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5ab12ac10 con 0x55e5ab3c5980 2023-12-07T10:19:59.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.219+0000 7f40a239c700 1 --2- 172.21.15.119:0/4254281651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5ab3c5980 0x55e5ab3c8990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4094099e70 tx=0x7f40940098a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6528f5fd33e07e6 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:59.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.219+0000 7f4090bff700 1 -- 172.21.15.119:0/4254281651 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409409a870 con 0x55e5ab3c5980 2023-12-07T10:19:59.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.219+0000 7f4090bff700 1 -- 172.21.15.119:0/4254281651 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40940a2470 con 0x55e5ab3c5980 2023-12-07T10:19:59.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.219+0000 7f4090bff700 1 -- 172.21.15.119:0/4254281651 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409400ce90 con 0x55e5ab3c5980 2023-12-07T10:19:59.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.219+0000 7f40b77cbb80 1 -- 172.21.15.119:0/4254281651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5ab3c5980 msgr2=0x55e5ab3c8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:59.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.220+0000 7f40b77cbb80 1 --2- 172.21.15.119:0/4254281651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5ab3c5980 0x55e5ab3c8990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4094099e70 tx=0x7f40940098a0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.220+0000 7f40b77cbb80 1 -- 172.21.15.119:0/4254281651 shutdown_connections 2023-12-07T10:19:59.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.220+0000 7f40b77cbb80 1 --2- 172.21.15.119:0/4254281651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5ab3c5980 0x55e5ab3c8990 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.220+0000 7f40b77cbb80 1 --2- 172.21.15.119:0/4254281651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5ab3c4130 0x55e5ab3cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.220+0000 7f40b77cbb80 1 --2- 172.21.15.119:0/4254281651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5ab3c4a90 0x55e5ab3d5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.220+0000 7f40b77cbb80 1 -- 172.21.15.119:0/4254281651 >> 172.21.15.119:0/4254281651 conn(0x55e5ab2f3310 msgr2=0x55e5ab3cbf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:59.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.220+0000 7f40b77cbb80 1 -- 172.21.15.119:0/4254281651 shutdown_connections 2023-12-07T10:19:59.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.220+0000 7f40b77cbb80 1 -- 172.21.15.119:0/4254281651 wait complete. 2023-12-07T10:19:59.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.220+0000 7f40b77cbb80 1 -- start start 2023-12-07T10:19:59.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.220+0000 7f40b77cbb80 1 Processor -- start 2023-12-07T10:19:59.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.221+0000 7f40b77cbb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5ab3c4130 0x55e5ab3d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:59.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.221+0000 7f40b77cbb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5ab3c4a90 0x55e5ab3d1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:59.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.221+0000 7f40b77cbb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5ab3c5980 0x55e5ab3d48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:59.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.221+0000 7f40b77cbb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5ab3d4e00 con 0x55e5ab3c4a90 2023-12-07T10:19:59.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.221+0000 7f40b77cbb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5ab3d4f70 con 0x55e5ab3c5980 2023-12-07T10:19:59.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.221+0000 7f40b77cbb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5ab3d5130 con 0x55e5ab3c4130 2023-12-07T10:19:59.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.221+0000 7f40b77cbb80 10 client.0 _ll_register_callbacks cb 0x55e5ab2b98f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:19:59.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.221+0000 7f40a139a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5ab3c4a90 0x55e5ab3d1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:59.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.221+0000 7f40a1b9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5ab3c4130 0x55e5ab3d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:59.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.221+0000 7f40a239c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5ab3c5980 0x55e5ab3d48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:59.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.221+0000 7f40a139a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5ab3c4a90 0x55e5ab3d1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51334/0 (socket says 172.21.15.119:51334) 2023-12-07T10:19:59.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.221+0000 7f40a139a700 1 -- 172.21.15.119:0/451643928 learned_addr learned my addr 172.21.15.119:0/451643928 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:19:59.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.221+0000 7f40a1b9b700 1 -- 172.21.15.119:0/451643928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5ab3c5980 msgr2=0x55e5ab3d48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:59.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.221+0000 7f40a1b9b700 1 --2- 172.21.15.119:0/451643928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5ab3c5980 0x55e5ab3d48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.221+0000 7f40a1b9b700 1 -- 172.21.15.119:0/451643928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5ab3c4a90 msgr2=0x55e5ab3d1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:19:59.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.221+0000 7f40a1b9b700 1 --2- 172.21.15.119:0/451643928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5ab3c4a90 0x55e5ab3d1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.222+0000 7f40a1b9b700 1 -- 172.21.15.119:0/451643928 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5ab1955e0 con 0x55e5ab3c4130 2023-12-07T10:19:59.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.222+0000 7f40a239c700 1 --2- 172.21.15.119:0/451643928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5ab3c5980 0x55e5ab3d48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:19:59.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.222+0000 7f40a1b9b700 1 --2- 172.21.15.119:0/451643928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5ab3c4130 0x55e5ab3d1400 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f409801d7a0 tx=0x7f409801d6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=302dbf6ad62dcaa6 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:19:59.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.222+0000 7f40857fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:19:59.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.222+0000 7f40857fa700 1 -- 172.21.15.119:0/451643928 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409801dbd0 con 0x55e5ab3c4130 2023-12-07T10:19:59.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.222+0000 7f40b77cbb80 1 -- 172.21.15.119:0/451643928 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4094004ad0 con 0x55e5ab3c4130 2023-12-07T10:19:59.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.222+0000 7f40857fa700 1 -- 172.21.15.119:0/451643928 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f4098023040 con 0x55e5ab3c4130 2023-12-07T10:19:59.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.222+0000 7f40857fa700 1 -- 172.21.15.119:0/451643928 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4098004250 con 0x55e5ab3c4130 2023-12-07T10:19:59.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.222+0000 7f40b77cbb80 10 client.25429 fetch_fsmap learned FSMap version 29 2023-12-07T10:19:59.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.222+0000 7f40b77cbb80 1 -- 172.21.15.119:0/451643928 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e5ab168200 con 0x55e5ab3c4130 2023-12-07T10:19:59.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.222+0000 7f40857fa700 1 -- 172.21.15.119:0/451643928 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4094004ad0 con 0x55e5ab3c4130 2023-12-07T10:19:59.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.223+0000 7f40857fa700 1 -- 172.21.15.119:0/451643928 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f409801c680 con 0x55e5ab3c4130 2023-12-07T10:19:59.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.223+0000 7f40b77cbb80 10 client.25429 fetch_fsmap finished waiting for FSMap version 29 2023-12-07T10:19:59.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.223+0000 7f40b77cbb80 10 client.25429 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-07T10:19:59.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.223+0000 7f40b77cbb80 20 client.25429 populate_metadata read hostname 'smithi119' 2023-12-07T10:19:59.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.223+0000 7f40b77cbb80 1 --2- 172.21.15.119:0/451643928 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55e5ab4346b0 0x55e5ab430b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:19:59.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.223+0000 7f40b77cbb80 4 client.25429 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-07T10:19:59.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.223+0000 7f40b77cbb80 1 -- 172.21.15.119:0/451643928 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e5ab2def80 con 0x55e5ab4346b0 2023-12-07T10:19:59.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.223+0000 7f40a139a700 1 --2- 172.21.15.119:0/451643928 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55e5ab4346b0 0x55e5ab430b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:19:59.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.224+0000 7f40857fa700 10 client.25429 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:19:59.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.224+0000 7f40a139a700 1 --2- 172.21.15.119:0/451643928 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55e5ab4346b0 0x55e5ab430b00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2690cbc73e3f21 server_cookie=49b6b1c34bc77d93 in_seq=0 out_seq=0 2023-12-07T10:19:59.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.224+0000 7f40857fa700 1 -- 172.21.15.119:0/451643928 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55e5ab2def80 con 0x55e5ab4346b0 2023-12-07T10:19:59.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.224+0000 7f40857fa700 10 client.25429 handle_command_reply: tid=0 2023-12-07T10:19:59.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.224+0000 7f40b77cbb80 1 client.25429 shutdown 2023-12-07T10:19:59.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.224+0000 7f40b77cbb80 1 -- 172.21.15.119:0/451643928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5ab3c4130 msgr2=0x55e5ab3d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:59.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.224+0000 7f40b77cbb80 1 --2- 172.21.15.119:0/451643928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5ab3c4130 0x55e5ab3d1400 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f409801d7a0 tx=0x7f409801d6c0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.224+0000 7f40b77cbb80 1 -- 172.21.15.119:0/451643928 shutdown_connections 2023-12-07T10:19:59.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.224+0000 7f40b77cbb80 1 --2- 172.21.15.119:0/451643928 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55e5ab4346b0 0x55e5ab430b00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.224+0000 7f40b77cbb80 1 --2- 172.21.15.119:0/451643928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5ab3c5980 0x55e5ab3d48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.224+0000 7f40b77cbb80 1 --2- 172.21.15.119:0/451643928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5ab3c4130 0x55e5ab3d1400 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.224+0000 7f40b77cbb80 1 --2- 172.21.15.119:0/451643928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5ab3c4a90 0x55e5ab3d1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.224+0000 7f40857fa700 0 client.25429 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:19:59.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.224+0000 7f40b77cbb80 1 -- 172.21.15.119:0/451643928 >> 172.21.15.119:0/451643928 conn(0x55e5ab2f3310 msgr2=0x55e5ab3cbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:59.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.224+0000 7f40b77cbb80 1 -- 172.21.15.119:0/451643928 shutdown_connections 2023-12-07T10:19:59.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.225+0000 7f40b77cbb80 1 -- 172.21.15.119:0/451643928 wait complete. 2023-12-07T10:19:59.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.225+0000 7f40b77cbb80 20 client.25429 trim_cache size 0 max 16384 2023-12-07T10:19:59.242 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:19:59.242 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "a4c642c6-94e8-11ee-95a2-87774f69a715", 2023-12-07T10:19:59.242 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-12-07T10:19:59.242 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 24566, 2023-12-07T10:19:59.242 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-12-07T10:19:59.242 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-12-07T10:19:59.242 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-12-07T10:19:59.243 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 137.474312161, 2023-12-07T10:19:59.243 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 29, 2023-12-07T10:19:59.243 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 93, 2023-12-07T10:19:59.243 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 91, 2023-12-07T10:19:59.243 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 145.68683008400001 2023-12-07T10:19:59.243 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:19:59.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.225+0000 7f40a3dff700 1 -- 172.21.15.119:0/668564 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f40880a1b90 msgr2=0x7f40880a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:59.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.225+0000 7f40a3dff700 1 --2- 172.21.15.119:0/668564 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f40880a1b90 0x7f40880a4050 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4098009c60 tx=0x7f4098009260 comp rx=0 tx=0).stop 2023-12-07T10:19:59.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.225+0000 7f40a3dff700 1 -- 172.21.15.119:0/668564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f409c107220 msgr2=0x7f409c1004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:19:59.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.225+0000 7f40a3dff700 1 --2- 172.21.15.119:0/668564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f409c107220 0x7f409c1004b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f409400ef30 tx=0x7f409400c3a0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.225+0000 7f40a3dff700 1 -- 172.21.15.119:0/668564 shutdown_connections 2023-12-07T10:19:59.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.225+0000 7f40a3dff700 1 --2- 172.21.15.119:0/668564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f409c107220 0x7f409c1004b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.225+0000 7f40a3dff700 1 --2- 172.21.15.119:0/668564 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f40880a1b90 0x7f40880a4050 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.225+0000 7f40a3dff700 1 --2- 172.21.15.119:0/668564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f409c101fc0 0x7f409c1018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.225+0000 7f40a3dff700 1 --2- 172.21.15.119:0/668564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f409c1048f0 0x7f409c0fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:19:59.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.225+0000 7f40a3dff700 1 -- 172.21.15.119:0/668564 >> 172.21.15.119:0/668564 conn(0x7f409c0fb9b0 msgr2=0x7f409c103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:19:59.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.226+0000 7f40a3dff700 1 -- 172.21.15.119:0/668564 shutdown_connections 2023-12-07T10:19:59.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:19:59.226+0000 7f40a3dff700 1 -- 172.21.15.119:0/668564 wait complete. 2023-12-07T10:19:59.245 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:19:59 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:19:59.247 INFO:teuthology.run_tasks:Running task kclient... 2023-12-07T10:19:59.261 INFO:tasks.kclient:Mounting kernel clients... 2023-12-07T10:19:59.261 INFO:tasks.kclient:config is {'client.0': {}} 2023-12-07T10:19:59.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:19:59.261 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-12-07T10:19:59.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:19:59 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:19:59.310 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-12-07T10:19:59.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:19:59.312 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-07T10:19:59.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:19:59.389 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link delete ceph-brx 2023-12-07T10:19:59.504 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2023-12-07T10:19:59.504 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-07T10:19:59.504 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-07T10:19:59.504 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph fs ls 2023-12-07T10:19:59.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:19:59 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:20:00.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:00 smithi153 ceph-mon[133625]: pgmap v352: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-07T10:20:00.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:00 smithi153 ceph-mon[133625]: overall HEALTH_OK 2023-12-07T10:20:00.364 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:20:00.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:00 smithi119 ceph-mon[97908]: pgmap v352: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-07T10:20:00.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:00 smithi119 ceph-mon[97908]: overall HEALTH_OK 2023-12-07T10:20:00.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:00 smithi159 ceph-mon[134726]: pgmap v352: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-07T10:20:00.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:00 smithi159 ceph-mon[134726]: overall HEALTH_OK 2023-12-07T10:20:02.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.121+0000 7f97e8ca3700 1 -- 172.21.15.119:0/1202886683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97e4074160 msgr2=0x7f97e410db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:02.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.121+0000 7f97e8ca3700 1 --2- 172.21.15.119:0/1202886683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97e4074160 0x7f97e410db30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f97d8009990 tx=0x7f97d8009ca0 comp rx=0 tx=0).stop 2023-12-07T10:20:02.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.121+0000 7f97e8ca3700 1 -- 172.21.15.119:0/1202886683 shutdown_connections 2023-12-07T10:20:02.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.121+0000 7f97e8ca3700 1 --2- 172.21.15.119:0/1202886683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97e410e070 0x7f97e4110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:02.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.121+0000 7f97e8ca3700 1 --2- 172.21.15.119:0/1202886683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97e4073840 0x7f97e4073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:02.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.121+0000 7f97e8ca3700 1 --2- 172.21.15.119:0/1202886683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97e4074160 0x7f97e410db30 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:02.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.121+0000 7f97e8ca3700 1 -- 172.21.15.119:0/1202886683 >> 172.21.15.119:0/1202886683 conn(0x7f97e4078800 msgr2=0x7f97e4078c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:02.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.122+0000 7f97e8ca3700 1 -- 172.21.15.119:0/1202886683 shutdown_connections 2023-12-07T10:20:02.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.122+0000 7f97e8ca3700 1 -- 172.21.15.119:0/1202886683 wait complete. 2023-12-07T10:20:02.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.123+0000 7f97e8ca3700 1 Processor -- start 2023-12-07T10:20:02.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.123+0000 7f97e8ca3700 1 -- start start 2023-12-07T10:20:02.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.124+0000 7f97e8ca3700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97e4073840 0x7f97e419f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:02.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.124+0000 7f97e8ca3700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97e4074160 0x7f97e419ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:02.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.124+0000 7f97e8ca3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97e410e070 0x7f97e4199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:02.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.124+0000 7f97e8ca3700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e419a190 con 0x7f97e4073840 2023-12-07T10:20:02.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.124+0000 7f97e8ca3700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e419a300 con 0x7f97e410e070 2023-12-07T10:20:02.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.124+0000 7f97e8ca3700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e419a4c0 con 0x7f97e4074160 2023-12-07T10:20:02.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.124+0000 7f97e259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97e4073840 0x7f97e419f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:02.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.124+0000 7f97e1d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97e4074160 0x7f97e419ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:02.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.124+0000 7f97e259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97e4073840 0x7f97e419f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51346/0 (socket says 172.21.15.119:51346) 2023-12-07T10:20:02.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.124+0000 7f97e259c700 1 -- 172.21.15.119:0/346000749 learned_addr learned my addr 172.21.15.119:0/346000749 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:02.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.124+0000 7f97e2d9d700 1 --2- 172.21.15.119:0/346000749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97e410e070 0x7f97e4199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:02.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.125+0000 7f97e259c700 1 -- 172.21.15.119:0/346000749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97e4074160 msgr2=0x7f97e419ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:02.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.125+0000 7f97e259c700 1 --2- 172.21.15.119:0/346000749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97e4074160 0x7f97e419ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:02.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.125+0000 7f97e259c700 1 -- 172.21.15.119:0/346000749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97e410e070 msgr2=0x7f97e4199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:02.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.125+0000 7f97e259c700 1 --2- 172.21.15.119:0/346000749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97e410e070 0x7f97e4199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:02.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.125+0000 7f97e259c700 1 -- 172.21.15.119:0/346000749 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d80095f0 con 0x7f97e4073840 2023-12-07T10:20:02.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.125+0000 7f97e1d9b700 1 --2- 172.21.15.119:0/346000749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97e4074160 0x7f97e419ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:02.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.125+0000 7f97e259c700 1 --2- 172.21.15.119:0/346000749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97e4073840 0x7f97e419f9f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f97d400e9a0 tx=0x7f97d400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:02.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.125+0000 7f97d37fe700 1 -- 172.21.15.119:0/346000749 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d4009790 con 0x7f97e4073840 2023-12-07T10:20:02.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.125+0000 7f97d37fe700 1 -- 172.21.15.119:0/346000749 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97d4010430 con 0x7f97e4073840 2023-12-07T10:20:02.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.125+0000 7f97e8ca3700 1 -- 172.21.15.119:0/346000749 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e419a7a0 con 0x7f97e4073840 2023-12-07T10:20:02.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.125+0000 7f97d37fe700 1 -- 172.21.15.119:0/346000749 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d4018680 con 0x7f97e4073840 2023-12-07T10:20:02.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.125+0000 7f97e8ca3700 1 -- 172.21.15.119:0/346000749 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e41a69e0 con 0x7f97e4073840 2023-12-07T10:20:02.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.127+0000 7f97e8ca3700 1 -- 172.21.15.119:0/346000749 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e4109280 con 0x7f97e4073840 2023-12-07T10:20:02.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.130+0000 7f97d37fe700 1 -- 172.21.15.119:0/346000749 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f97d40105a0 con 0x7f97e4073840 2023-12-07T10:20:02.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.132+0000 7f97d37fe700 1 --2- 172.21.15.119:0/346000749 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f97cc0a1b90 0x7f97cc0a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:02.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.132+0000 7f97d37fe700 1 -- 172.21.15.119:0/346000749 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f97d4014070 con 0x7f97e4073840 2023-12-07T10:20:02.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.132+0000 7f97e1d9b700 1 --2- 172.21.15.119:0/346000749 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f97cc0a1b90 0x7f97cc0a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:02.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.133+0000 7f97e1d9b700 1 --2- 172.21.15.119:0/346000749 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f97cc0a1b90 0x7f97cc0a4050 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f97e419acf0 tx=0x7f97d800b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:02.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.136+0000 7f97d37fe700 1 -- 172.21.15.119:0/346000749 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f97d4096b10 con 0x7f97e4073840 2023-12-07T10:20:02.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:02 smithi119 ceph-mon[97908]: pgmap v353: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-07T10:20:02.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:02 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:20:02.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:02 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:20:02.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:02 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:20:02.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:02 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:20:02.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:02 smithi153 ceph-mon[133625]: pgmap v353: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-07T10:20:02.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:02 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:20:02.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:02 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:20:02.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:02 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:20:02.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:02 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:20:02.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.396+0000 7f97e8ca3700 1 -- 172.21.15.119:0/346000749 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f97e419aff0 con 0x7f97e4073840 2023-12-07T10:20:02.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.396+0000 7f97d37fe700 1 -- 172.21.15.119:0/346000749 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) v1 ==== 53+0+88 (secure 0 0 0) 0x7f97d409a130 con 0x7f97e4073840 2023-12-07T10:20:02.399 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-12-07T10:20:02.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.400+0000 7f97e8ca3700 1 -- 172.21.15.119:0/346000749 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f97cc0a1b90 msgr2=0x7f97cc0a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:02.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.400+0000 7f97e8ca3700 1 --2- 172.21.15.119:0/346000749 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f97cc0a1b90 0x7f97cc0a4050 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f97e419acf0 tx=0x7f97d800b260 comp rx=0 tx=0).stop 2023-12-07T10:20:02.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.400+0000 7f97e8ca3700 1 -- 172.21.15.119:0/346000749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97e4073840 msgr2=0x7f97e419f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:02.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.400+0000 7f97e8ca3700 1 --2- 172.21.15.119:0/346000749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97e4073840 0x7f97e419f9f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f97d400e9a0 tx=0x7f97d400ecb0 comp rx=0 tx=0).stop 2023-12-07T10:20:02.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.400+0000 7f97e8ca3700 1 -- 172.21.15.119:0/346000749 shutdown_connections 2023-12-07T10:20:02.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.400+0000 7f97e8ca3700 1 --2- 172.21.15.119:0/346000749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97e410e070 0x7f97e4199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:02.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.400+0000 7f97e8ca3700 1 --2- 172.21.15.119:0/346000749 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f97cc0a1b90 0x7f97cc0a4050 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:02.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.400+0000 7f97e8ca3700 1 --2- 172.21.15.119:0/346000749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97e4073840 0x7f97e419f9f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:02.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.400+0000 7f97e8ca3700 1 --2- 172.21.15.119:0/346000749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97e4074160 0x7f97e419ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:02.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.400+0000 7f97e8ca3700 1 -- 172.21.15.119:0/346000749 >> 172.21.15.119:0/346000749 conn(0x7f97e4078800 msgr2=0x7f97e40fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:02.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.400+0000 7f97e8ca3700 1 -- 172.21.15.119:0/346000749 shutdown_connections 2023-12-07T10:20:02.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:02.400+0000 7f97e8ca3700 1 -- 172.21.15.119:0/346000749 wait complete. 2023-12-07T10:20:02.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:02 smithi159 ceph-mon[134726]: pgmap v353: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-07T10:20:02.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:02 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:20:02.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:02 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:20:02.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:02 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:20:02.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:02 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:20:02.869 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-07T10:20:02.869 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-07T10:20:02.869 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-12-07T10:20:02.869 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-07T10:20:02.869 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-07T10:20:02.869 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-07T10:20:02.869 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-07T10:20:02.869 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-07T10:20:02.869 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-07T10:20:02.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:20:02.870 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-12-07T10:20:02.915 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-07T10:20:02.915 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-07T10:20:02.916 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-12-07T10:20:02.916 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-12-07T10:20:02.916 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-12-07T10:20:02.916 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-12-07T10:20:02.916 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-07T10:20:02.916 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-12-07T10:20:02.916 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-12-07T10:20:02.916 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-07T10:20:02.916 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-12-07T10:20:02.916 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-12-07T10:20:02.916 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-07T10:20:02.916 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-12-07T10:20:02.916 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-07T10:20:02.916 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-12-07T10:20:02.916 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-07T10:20:02.917 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 41609sec preferred_lft 41609sec 2023-12-07T10:20:02.917 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::9e03:31a5:7c84:6136/64 scope link noprefixroute 2023-12-07T10:20:02.917 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-12-07T10:20:02.918 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-07T10:20:02.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:20:02.918 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-12-07T10:20:02.918 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-12-07T10:20:02.918 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add name ceph-brx type bridge 2023-12-07T10:20:02.918 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip addr flush dev ceph-brx 2023-12-07T10:20:02.918 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set ceph-brx up 2023-12-07T10:20:02.919 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-07T10:20:02.919 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-12-07T10:20:03.075 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:03 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/346000749' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-07T10:20:03.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:20:03.176 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-07T10:20:03.247 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-12-07T10:20:03.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:20:03.251 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-12-07T10:20:03.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:03 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/346000749' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-07T10:20:03.298 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-12-07T10:20:03.298 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-07T10:20:03.298 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-07T10:20:03.298 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-07T10:20:03.298 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-07T10:20:03.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:20:03.299 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-12-07T10:20:03.299 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-12-07T10:20:03.300 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-07T10:20:03.300 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-07T10:20:03.300 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-07T10:20:03.300 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-12-07T10:20:03.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:20:03.473 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-12-07T10:20:03.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:03 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/346000749' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-07T10:20:03.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:20:03.516 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list-id 2023-12-07T10:20:03.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:20:03.605 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-12-07T10:20:03.605 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-12-07T10:20:03.605 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-07T10:20:03.605 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-12-07T10:20:03.605 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-12-07T10:20:03.750 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-12-07T10:20:03.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:20:03.750 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-12-07T10:20:03.750 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-12-07T10:20:03.750 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-12-07T10:20:03.750 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-07T10:20:03.750 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-12-07T10:20:03.750 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-12-07T10:20:03.750 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-12-07T10:20:03.751 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-12-07T10:20:04.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:20:04.003 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-12-07T10:20:04.003 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-12-07T10:20:04.003 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2023-12-07T10:20:04.003 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set dev brx.0 master ceph-brx 2023-12-07T10:20:04.003 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-12-07T10:20:04.179 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-07T10:20:04.179 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-07T10:20:04.223 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-07T10:20:04.223 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-07T10:20:04.223 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-07T10:20:04.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:04 smithi119 ceph-mon[97908]: pgmap v354: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-07T10:20:04.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:04 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:20:04.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:04 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:20:04.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:04 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:20:04.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:04 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:20:04.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:04 smithi119 ceph-mon[97908]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:20:04.271 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-12-07T10:20:04.271 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-12-07T10:20:04.272 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69 2023-12-07T10:20:04.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:20:04.272 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2023-12-07T10:20:04.417 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2023-12-07T10:20:04.418 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2023-12-07T10:20:04.418 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2023-12-07T10:20:04.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:04 smithi159 ceph-mon[134726]: pgmap v354: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-07T10:20:04.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:04 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:20:04.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:04 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:20:04.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:04 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:20:04.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:04 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:20:04.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:04 smithi159 ceph-mon[134726]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:20:04.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:04 smithi153 ceph-mon[133625]: pgmap v354: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-07T10:20:04.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:04 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:20:04.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:04 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:20:04.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:04 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:20:04.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:04 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:20:04.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:04 smithi153 ceph-mon[133625]: from='mgr.14150 172.21.15.119:0/3440670956' entity='mgr.x' 2023-12-07T10:20:04.864 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2023-12-07T10:20:04.864 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2023-12-07T10:20:04.864 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2023-12-07T10:20:04.864 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.153:3300,172.21.15.159:3300" 2023-12-07T10:20:04.864 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.119:3300,172.21.15.153:3300,172.21.15.159:3300:/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69 2023-12-07T10:20:04.864 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,key=0,fsid=a4c642c6-94e8-11ee-95a2-87774f69a715" will pass to kernel 2023-12-07T10:20:04.870 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-07T10:20:04.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:20:04.870 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-07T10:20:04.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:20:04.941 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-07T10:20:05.027 INFO:teuthology.orchestra.run.smithi119.stdout:client25454 2023-12-07T10:20:05.027 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-12-07T10:20:05.027 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-07T10:20:05.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:20:05.071 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-07T10:20:05.197 INFO:teuthology.orchestra.run.smithi119.stdout:client25454 2023-12-07T10:20:05.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:20:05.198 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-07T10:20:05.284 INFO:teuthology.orchestra.run.smithi119.stdout:client25454 2023-12-07T10:20:05.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:20:05.284 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715.client25454/status') 2023-12-07T10:20:05.392 INFO:teuthology.orchestra.run.smithi119.stdout:instance: client.25454 (3)172.21.15.119:0/1640894208 2023-12-07T10:20:05.392 INFO:teuthology.orchestra.run.smithi119.stdout:blocklisted: false 2023-12-07T10:20:05.392 INFO:teuthology.orchestra.run.smithi119.stderr:0+1 records in 2023-12-07T10:20:05.392 INFO:teuthology.orchestra.run.smithi119.stderr:0+1 records out 2023-12-07T10:20:05.392 INFO:teuthology.orchestra.run.smithi119.stderr:72 bytes copied, 4.9076e-05 s, 1.5 MB/s 2023-12-07T10:20:05.395 INFO:teuthology.run_tasks:Running task check-counter... 2023-12-07T10:20:05.406 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-12-07T10:20:05.429 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-07T10:20:05.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.627+0000 7fed801ff700 1 -- 172.21.15.119:0/1777054835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed78101560 msgr2=0x7fed78101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:05.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.627+0000 7fed801ff700 1 --2- 172.21.15.119:0/1777054835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed78101560 0x7fed78101980 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fed740099c0 tx=0x7fed74009cd0 comp rx=0 tx=0).stop 2023-12-07T10:20:05.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.627+0000 7fed801ff700 1 -- 172.21.15.119:0/1777054835 shutdown_connections 2023-12-07T10:20:05.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.627+0000 7fed801ff700 1 --2- 172.21.15.119:0/1777054835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed781030f0 0x7fed781099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:05.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.627+0000 7fed801ff700 1 --2- 172.21.15.119:0/1777054835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed78101560 0x7fed78101980 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:05.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.627+0000 7fed801ff700 1 --2- 172.21.15.119:0/1777054835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed78102750 0x7fed78102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:05.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.627+0000 7fed801ff700 1 -- 172.21.15.119:0/1777054835 >> 172.21.15.119:0/1777054835 conn(0x7fed780fcab0 msgr2=0x7fed780fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:05.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.627+0000 7fed801ff700 1 -- 172.21.15.119:0/1777054835 shutdown_connections 2023-12-07T10:20:05.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.627+0000 7fed801ff700 1 -- 172.21.15.119:0/1777054835 wait complete. 2023-12-07T10:20:05.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.628+0000 7fed801ff700 1 Processor -- start 2023-12-07T10:20:05.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.628+0000 7fed801ff700 1 -- start start 2023-12-07T10:20:05.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.628+0000 7fed801ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed78101560 0x7fed78075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:05.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.628+0000 7fed801ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed78102750 0x7fed78073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:05.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.628+0000 7fed801ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed781030f0 0x7fed78073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:05.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.628+0000 7fed801ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed780746e0 con 0x7fed78101560 2023-12-07T10:20:05.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.628+0000 7fed801ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed78074850 con 0x7fed78102750 2023-12-07T10:20:05.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.628+0000 7fed801ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed781a1570 con 0x7fed781030f0 2023-12-07T10:20:05.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.629+0000 7fed7df9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed78101560 0x7fed78075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:05.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.629+0000 7fed7df9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed78101560 0x7fed78075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47128/0 (socket says 172.21.15.119:47128) 2023-12-07T10:20:05.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.629+0000 7fed7df9b700 1 -- 172.21.15.119:0/1245416183 learned_addr learned my addr 172.21.15.119:0/1245416183 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:05.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.629+0000 7fed7d79a700 1 --2- 172.21.15.119:0/1245416183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed78102750 0x7fed78073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:05.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.629+0000 7fed7e79c700 1 --2- 172.21.15.119:0/1245416183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed781030f0 0x7fed78073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:05.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.629+0000 7fed7df9b700 1 -- 172.21.15.119:0/1245416183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed781030f0 msgr2=0x7fed78073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:05.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.629+0000 7fed7df9b700 1 --2- 172.21.15.119:0/1245416183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed781030f0 0x7fed78073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:05.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.629+0000 7fed7df9b700 1 -- 172.21.15.119:0/1245416183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed78102750 msgr2=0x7fed78073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:05.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.629+0000 7fed7df9b700 1 --2- 172.21.15.119:0/1245416183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed78102750 0x7fed78073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:05.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.629+0000 7fed7df9b700 1 -- 172.21.15.119:0/1245416183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed740095f0 con 0x7fed78101560 2023-12-07T10:20:05.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.629+0000 7fed7df9b700 1 --2- 172.21.15.119:0/1245416183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed78101560 0x7fed78075370 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fed74003730 tx=0x7fed7400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:05.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.629+0000 7fed6effd700 1 -- 172.21.15.119:0/1245416183 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7401d070 con 0x7fed78101560 2023-12-07T10:20:05.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.630+0000 7fed6effd700 1 -- 172.21.15.119:0/1245416183 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed74005510 con 0x7fed78101560 2023-12-07T10:20:05.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.630+0000 7fed801ff700 1 -- 172.21.15.119:0/1245416183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed781a17a0 con 0x7fed78101560 2023-12-07T10:20:05.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.630+0000 7fed6effd700 1 -- 172.21.15.119:0/1245416183 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed74004ba0 con 0x7fed78101560 2023-12-07T10:20:05.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.630+0000 7fed801ff700 1 -- 172.21.15.119:0/1245416183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed781a1bc0 con 0x7fed78101560 2023-12-07T10:20:05.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.631+0000 7fed801ff700 1 -- 172.21.15.119:0/1245416183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed78108a30 con 0x7fed78101560 2023-12-07T10:20:05.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.633+0000 7fed6effd700 1 -- 172.21.15.119:0/1245416183 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fed74004d00 con 0x7fed78101560 2023-12-07T10:20:05.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.634+0000 7fed6effd700 1 --2- 172.21.15.119:0/1245416183 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fed640a1b90 0x7fed640a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:05.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.634+0000 7fed6effd700 1 -- 172.21.15.119:0/1245416183 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fed740ce740 con 0x7fed78101560 2023-12-07T10:20:05.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.634+0000 7fed7d79a700 1 --2- 172.21.15.119:0/1245416183 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fed640a1b90 0x7fed640a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:05.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.635+0000 7fed7d79a700 1 --2- 172.21.15.119:0/1245416183 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fed640a1b90 0x7fed640a4050 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fed68009bf0 tx=0x7fed68009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:05.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.638+0000 7fed6effd700 1 -- 172.21.15.119:0/1245416183 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fed74097820 con 0x7fed78101560 2023-12-07T10:20:05.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.844+0000 7fed801ff700 1 -- 172.21.15.119:0/1245416183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed78068350 con 0x7fed78101560 2023-12-07T10:20:05.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.845+0000 7fed6effd700 1 -- 172.21.15.119:0/1245416183 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24715 (secure 0 0 0) 0x7fed7409ae40 con 0x7fed78101560 2023-12-07T10:20:05.847 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:20:05.847 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":93,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","created":"2023-12-07T10:09:59.945863+0000","modified":"2023-12-07T10:19:02.800626+0000","last_up_change":"2023-12-07T10:16:50.929427+0000","last_in_change":"2023-12-07T10:16:33.558158+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T10:13:25.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-07T10:17:39.347813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-07T10:17:40.714501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-07T10:17:44.257760+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"56b5818e-e625-40f1-8838-3ddcf2d34ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6803","nonce":2141950133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6805","nonce":2141950133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6809","nonce":2141950133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6807","nonce":2141950133}]},"public_addr":"172.21.15.119:6803/2141950133","cluster_addr":"172.21.15.119:6805/2141950133","heartbeat_back_addr":"172.21.15.119:6809/2141950133","heartbeat_front_addr":"172.21.15.119:6807/2141950133","state":["exists","up"]},{"osd":1,"uuid":"36489a6a-5278-4802-9525-9871521722e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6801","nonce":3390875370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6803","nonce":3390875370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6807","nonce":3390875370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6805","nonce":3390875370}]},"public_addr":"172.21.15.153:6801/3390875370","cluster_addr":"172.21.15.153:6803/3390875370","heartbeat_back_addr":"172.21.15.153:6807/3390875370","heartbeat_front_addr":"172.21.15.153:6805/3390875370","state":["exists","up"]},{"osd":2,"uuid":"a937ad26-5e9b-4c8a-b65a-626a8feda6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6801","nonce":708101637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6803","nonce":708101637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6807","nonce":708101637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6805","nonce":708101637}]},"public_addr":"172.21.15.159:6801/708101637","cluster_addr":"172.21.15.159:6803/708101637","heartbeat_back_addr":"172.21.15.159:6807/708101637","heartbeat_front_addr":"172.21.15.159:6805/708101637","state":["exists","up"]},{"osd":3,"uuid":"26ef8300-aa07-48f6-b31d-8b0f2217ffcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6811","nonce":3228047257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6813","nonce":3228047257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6817","nonce":3228047257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6815","nonce":3228047257}]},"public_addr":"172.21.15.119:6811/3228047257","cluster_addr":"172.21.15.119:6813/3228047257","heartbeat_back_addr":"172.21.15.119:6817/3228047257","heartbeat_front_addr":"172.21.15.119:6815/3228047257","state":["exists","up"]},{"osd":4,"uuid":"000ee4b9-9896-4901-8cc3-c85fdfc8cbaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6809","nonce":4021998299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6811","nonce":4021998299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6815","nonce":4021998299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6813","nonce":4021998299}]},"public_addr":"172.21.15.153:6809/4021998299","cluster_addr":"172.21.15.153:6811/4021998299","heartbeat_back_addr":"172.21.15.153:6815/4021998299","heartbeat_front_addr":"172.21.15.153:6813/4021998299","state":["exists","up"]},{"osd":5,"uuid":"8b2fc167-262d-474f-89f3-47bd29424421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6809","nonce":1832540768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6811","nonce":1832540768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6815","nonce":1832540768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6813","nonce":1832540768}]},"public_addr":"172.21.15.159:6809/1832540768","cluster_addr":"172.21.15.159:6811/1832540768","heartbeat_back_addr":"172.21.15.159:6815/1832540768","heartbeat_front_addr":"172.21.15.159:6813/1832540768","state":["exists","up"]},{"osd":6,"uuid":"f9aa6749-46cf-4b2a-8114-cbd034219d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162436840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162436840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6825","nonce":1162436840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162436840}]},"public_addr":"172.21.15.119:6819/1162436840","cluster_addr":"172.21.15.119:6821/1162436840","heartbeat_back_addr":"172.21.15.119:6825/1162436840","heartbeat_front_addr":"172.21.15.119:6823/1162436840","state":["exists","up"]},{"osd":7,"uuid":"39216361-e1c4-4fef-86cb-a4707656b547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6817","nonce":3254653457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6819","nonce":3254653457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6823","nonce":3254653457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6821","nonce":3254653457}]},"public_addr":"172.21.15.153:6817/3254653457","cluster_addr":"172.21.15.153:6819/3254653457","heartbeat_back_addr":"172.21.15.153:6823/3254653457","heartbeat_front_addr":"172.21.15.153:6821/3254653457","state":["exists","up"]},{"osd":8,"uuid":"cceb6ebc-7c02-48ea-9d16-b7a016dd449b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6817","nonce":2600946380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6819","nonce":2600946380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6823","nonce":2600946380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6821","nonce":2600946380}]},"public_addr":"172.21.15.159:6817/2600946380","cluster_addr":"172.21.15.159:6819/2600946380","heartbeat_back_addr":"172.21.15.159:6823/2600946380","heartbeat_front_addr":"172.21.15.159:6821/2600946380","state":["exists","up"]},{"osd":9,"uuid":"f337a2e4-e065-4a9f-91d9-e3615de2d6f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6827","nonce":1940737285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6829","nonce":1940737285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6833","nonce":1940737285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6831","nonce":1940737285}]},"public_addr":"172.21.15.119:6827/1940737285","cluster_addr":"172.21.15.119:6829/1940737285","heartbeat_back_addr":"172.21.15.119:6833/1940737285","heartbeat_front_addr":"172.21.15.119:6831/1940737285","state":["exists","up"]},{"osd":10,"uuid":"3a3bc347-85f4-4878-9797-d9da96137e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6825","nonce":663834864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6827","nonce":663834864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6831","nonce":663834864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6829","nonce":663834864}]},"public_addr":"172.21.15.153:6825/663834864","cluster_addr":"172.21.15.153:6827/663834864","heartbeat_back_addr":"172.21.15.153:6831/663834864","heartbeat_front_addr":"172.21.15.153:6829/663834864","state":["exists","up"]},{"osd":11,"uuid":"a0fc5909-279d-4172-977c-2dc2da0a7e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6825","nonce":1262216348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6827","nonce":1262216348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6831","nonce":1262216348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6829","nonce":1262216348}]},"public_addr":"172.21.15.159:6825/1262216348","cluster_addr":"172.21.15.159:6827/1262216348","heartbeat_back_addr":"172.21.15.159:6831/1262216348","heartbeat_front_addr":"172.21.15.159:6829/1262216348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:37.488109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:59.543060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:21.381585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:47.112729+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:09.732150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:30.893729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:56.678710+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:19.004256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:40.388818+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:06.391734+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:28.394053+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:49.634348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1628639392":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6801/1355617847":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6800/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/362307836":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/128964558":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/2278813554":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/1411933614":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/1685698277":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6800/1355617847":"2023-12-08T10:11:00.585565+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T10:20:05.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.848+0000 7fed801ff700 1 -- 172.21.15.119:0/1245416183 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fed640a1b90 msgr2=0x7fed640a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:05.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.848+0000 7fed801ff700 1 --2- 172.21.15.119:0/1245416183 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fed640a1b90 0x7fed640a4050 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fed68009bf0 tx=0x7fed68009260 comp rx=0 tx=0).stop 2023-12-07T10:20:05.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.848+0000 7fed801ff700 1 -- 172.21.15.119:0/1245416183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed78101560 msgr2=0x7fed78075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:05.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.848+0000 7fed801ff700 1 --2- 172.21.15.119:0/1245416183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed78101560 0x7fed78075370 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fed74003730 tx=0x7fed7400f740 comp rx=0 tx=0).stop 2023-12-07T10:20:05.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.848+0000 7fed801ff700 1 -- 172.21.15.119:0/1245416183 shutdown_connections 2023-12-07T10:20:05.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.848+0000 7fed801ff700 1 --2- 172.21.15.119:0/1245416183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed781030f0 0x7fed78073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:05.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.848+0000 7fed801ff700 1 --2- 172.21.15.119:0/1245416183 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fed640a1b90 0x7fed640a4050 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:05.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.848+0000 7fed801ff700 1 --2- 172.21.15.119:0/1245416183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed78101560 0x7fed78075370 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:05.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.848+0000 7fed801ff700 1 --2- 172.21.15.119:0/1245416183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed78102750 0x7fed78073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:05.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.848+0000 7fed801ff700 1 -- 172.21.15.119:0/1245416183 >> 172.21.15.119:0/1245416183 conn(0x7fed780fcab0 msgr2=0x7fed78105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:05.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.848+0000 7fed801ff700 1 -- 172.21.15.119:0/1245416183 shutdown_connections 2023-12-07T10:20:05.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:05.848+0000 7fed801ff700 1 -- 172.21.15.119:0/1245416183 wait complete. 2023-12-07T10:20:05.869 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-07T10:13:25.194366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-07T10:17:39.347813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-07T10:17:40.714501+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-07T10:17:44.257760+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-07T10:20:05.869 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-07T10:20:06.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.069+0000 7f65eb9ff700 1 -- 172.21.15.119:0/30213772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65e41015c0 msgr2=0x7f65e41019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:06.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.069+0000 7f65eb9ff700 1 --2- 172.21.15.119:0/30213772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65e41015c0 0x7f65e41019e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f65e00099c0 tx=0x7f65e0009cd0 comp rx=0 tx=0).stop 2023-12-07T10:20:06.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.069+0000 7f65eb9ff700 1 -- 172.21.15.119:0/30213772 shutdown_connections 2023-12-07T10:20:06.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.069+0000 7f65eb9ff700 1 --2- 172.21.15.119:0/30213772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65e4103150 0x7f65e4109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:06.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.069+0000 7f65eb9ff700 1 --2- 172.21.15.119:0/30213772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65e41015c0 0x7f65e41019e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:06.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.069+0000 7f65eb9ff700 1 --2- 172.21.15.119:0/30213772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65e41027b0 0x7f65e4102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:06.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.069+0000 7f65eb9ff700 1 -- 172.21.15.119:0/30213772 >> 172.21.15.119:0/30213772 conn(0x7f65e40fcb10 msgr2=0x7f65e40fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:06.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.069+0000 7f65eb9ff700 1 -- 172.21.15.119:0/30213772 shutdown_connections 2023-12-07T10:20:06.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.069+0000 7f65eb9ff700 1 -- 172.21.15.119:0/30213772 wait complete. 2023-12-07T10:20:06.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.070+0000 7f65eb9ff700 1 Processor -- start 2023-12-07T10:20:06.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.070+0000 7f65eb9ff700 1 -- start start 2023-12-07T10:20:06.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.070+0000 7f65eb9ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65e41015c0 0x7f65e4199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:06.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.070+0000 7f65eb9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65e41027b0 0x7f65e4199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:06.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.070+0000 7f65eb9ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65e4103150 0x7f65e41a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:06.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.070+0000 7f65eb9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e41a12b0 con 0x7f65e41027b0 2023-12-07T10:20:06.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.070+0000 7f65eb9ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e41a13f0 con 0x7f65e4103150 2023-12-07T10:20:06.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.070+0000 7f65eb9ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e41a1580 con 0x7f65e41015c0 2023-12-07T10:20:06.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.070+0000 7f65e8f9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65e41027b0 0x7f65e4199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:06.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.070+0000 7f65e8f9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65e41027b0 0x7f65e4199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47142/0 (socket says 172.21.15.119:47142) 2023-12-07T10:20:06.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.070+0000 7f65e8f9a700 1 -- 172.21.15.119:0/2370646050 learned_addr learned my addr 172.21.15.119:0/2370646050 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:06.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.071+0000 7f65e979b700 1 --2- 172.21.15.119:0/2370646050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65e41015c0 0x7f65e4199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:06.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.071+0000 7f65e9f9c700 1 --2- 172.21.15.119:0/2370646050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65e4103150 0x7f65e41a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:06.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.071+0000 7f65e8f9a700 1 -- 172.21.15.119:0/2370646050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65e41015c0 msgr2=0x7f65e4199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:06.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.071+0000 7f65e8f9a700 1 --2- 172.21.15.119:0/2370646050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65e41015c0 0x7f65e4199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:06.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.071+0000 7f65e8f9a700 1 -- 172.21.15.119:0/2370646050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65e4103150 msgr2=0x7f65e41a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:06.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.071+0000 7f65e8f9a700 1 --2- 172.21.15.119:0/2370646050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65e4103150 0x7f65e41a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:06.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.071+0000 7f65e8f9a700 1 -- 172.21.15.119:0/2370646050 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e00095f0 con 0x7f65e41027b0 2023-12-07T10:20:06.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.071+0000 7f65e8f9a700 1 --2- 172.21.15.119:0/2370646050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65e41027b0 0x7f65e4199cc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f65d400d980 tx=0x7f65d400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:06.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.071+0000 7f65da7fc700 1 -- 172.21.15.119:0/2370646050 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d4004220 con 0x7f65e41027b0 2023-12-07T10:20:06.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.071+0000 7f65da7fc700 1 -- 172.21.15.119:0/2370646050 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65d4007640 con 0x7f65e41027b0 2023-12-07T10:20:06.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.071+0000 7f65eb9ff700 1 -- 172.21.15.119:0/2370646050 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e41a1860 con 0x7f65e41027b0 2023-12-07T10:20:06.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.071+0000 7f65da7fc700 1 -- 172.21.15.119:0/2370646050 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d4005690 con 0x7f65e41027b0 2023-12-07T10:20:06.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.071+0000 7f65eb9ff700 1 -- 172.21.15.119:0/2370646050 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e41a1dc0 con 0x7f65e41027b0 2023-12-07T10:20:06.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.072+0000 7f65eb9ff700 1 -- 172.21.15.119:0/2370646050 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e40683c0 con 0x7f65e41027b0 2023-12-07T10:20:06.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.074+0000 7f65da7fc700 1 -- 172.21.15.119:0/2370646050 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f65d4004750 con 0x7f65e41027b0 2023-12-07T10:20:06.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.076+0000 7f65da7fc700 1 --2- 172.21.15.119:0/2370646050 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f65d00a1b90 0x7f65d00a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:06.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.076+0000 7f65da7fc700 1 -- 172.21.15.119:0/2370646050 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f65d40cd700 con 0x7f65e41027b0 2023-12-07T10:20:06.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.076+0000 7f65e979b700 1 --2- 172.21.15.119:0/2370646050 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f65d00a1b90 0x7f65d00a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:06.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.076+0000 7f65e979b700 1 --2- 172.21.15.119:0/2370646050 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f65d00a1b90 0x7f65d00a4050 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f65e0003730 tx=0x7f65e0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:06.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.080+0000 7f65da7fc700 1 -- 172.21.15.119:0/2370646050 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f65d4096750 con 0x7f65e41027b0 2023-12-07T10:20:06.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.282+0000 7f65eb9ff700 1 -- 172.21.15.119:0/2370646050 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f65e41a20b0 con 0x7f65e41027b0 2023-12-07T10:20:06.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.283+0000 7f65da7fc700 1 -- 172.21.15.119:0/2370646050 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f65d40160a0 con 0x7f65e41027b0 2023-12-07T10:20:06.284 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-12-07T10:20:06.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.285+0000 7f65eb9ff700 1 -- 172.21.15.119:0/2370646050 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f65d00a1b90 msgr2=0x7f65d00a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:06.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.285+0000 7f65eb9ff700 1 --2- 172.21.15.119:0/2370646050 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f65d00a1b90 0x7f65d00a4050 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f65e0003730 tx=0x7f65e0000bc0 comp rx=0 tx=0).stop 2023-12-07T10:20:06.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.285+0000 7f65eb9ff700 1 -- 172.21.15.119:0/2370646050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65e41027b0 msgr2=0x7f65e4199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:06.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.285+0000 7f65eb9ff700 1 --2- 172.21.15.119:0/2370646050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65e41027b0 0x7f65e4199cc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f65d400d980 tx=0x7f65d400dc90 comp rx=0 tx=0).stop 2023-12-07T10:20:06.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.285+0000 7f65eb9ff700 1 -- 172.21.15.119:0/2370646050 shutdown_connections 2023-12-07T10:20:06.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.285+0000 7f65eb9ff700 1 --2- 172.21.15.119:0/2370646050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65e4103150 0x7f65e41a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:06.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.285+0000 7f65eb9ff700 1 --2- 172.21.15.119:0/2370646050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65e41015c0 0x7f65e4199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:06.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.285+0000 7f65eb9ff700 1 --2- 172.21.15.119:0/2370646050 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f65d00a1b90 0x7f65d00a4050 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:06.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.285+0000 7f65eb9ff700 1 --2- 172.21.15.119:0/2370646050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65e41027b0 0x7f65e4199cc0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:06.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.285+0000 7f65eb9ff700 1 -- 172.21.15.119:0/2370646050 >> 172.21.15.119:0/2370646050 conn(0x7f65e40fcb10 msgr2=0x7f65e4107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:06.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.285+0000 7f65eb9ff700 1 -- 172.21.15.119:0/2370646050 shutdown_connections 2023-12-07T10:20:06.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.285+0000 7f65eb9ff700 1 -- 172.21.15.119:0/2370646050 wait complete. 2023-12-07T10:20:06.306 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-07T10:20:06.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:06 smithi119 ceph-mon[97908]: pgmap v355: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-12-07T10:20:06.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:06 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1245416183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:20:06.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.504+0000 7f0a985ff700 1 -- 172.21.15.119:0/3633418419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a901015c0 msgr2=0x7f0a901019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:06.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.504+0000 7f0a985ff700 1 --2- 172.21.15.119:0/3633418419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a901015c0 0x7f0a901019e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c009920 tx=0x7f0a8c009c30 comp rx=0 tx=0).stop 2023-12-07T10:20:06.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.504+0000 7f0a985ff700 1 -- 172.21.15.119:0/3633418419 shutdown_connections 2023-12-07T10:20:06.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.504+0000 7f0a985ff700 1 --2- 172.21.15.119:0/3633418419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a90103150 0x7f0a90109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:06.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.504+0000 7f0a985ff700 1 --2- 172.21.15.119:0/3633418419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a901015c0 0x7f0a901019e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:06.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.504+0000 7f0a985ff700 1 --2- 172.21.15.119:0/3633418419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a901027b0 0x7f0a90102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:06.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.504+0000 7f0a985ff700 1 -- 172.21.15.119:0/3633418419 >> 172.21.15.119:0/3633418419 conn(0x7f0a900fcb10 msgr2=0x7f0a900fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:06.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.504+0000 7f0a985ff700 1 -- 172.21.15.119:0/3633418419 shutdown_connections 2023-12-07T10:20:06.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.504+0000 7f0a985ff700 1 -- 172.21.15.119:0/3633418419 wait complete. 2023-12-07T10:20:06.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.505+0000 7f0a985ff700 1 Processor -- start 2023-12-07T10:20:06.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.506+0000 7f0a985ff700 1 -- start start 2023-12-07T10:20:06.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.506+0000 7f0a985ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a901015c0 0x7f0a90197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:06.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.506+0000 7f0a985ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a901027b0 0x7f0a90197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:06.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.506+0000 7f0a985ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a90103150 0x7f0a9019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:06.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.506+0000 7f0a985ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9019f180 con 0x7f0a90103150 2023-12-07T10:20:06.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.506+0000 7f0a985ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9019f2c0 con 0x7f0a901015c0 2023-12-07T10:20:06.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.506+0000 7f0a985ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9019f450 con 0x7f0a901027b0 2023-12-07T10:20:06.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.506+0000 7f0a96b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a90103150 0x7f0a9019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:06.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.506+0000 7f0a9639b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a901015c0 0x7f0a90197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:06.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.506+0000 7f0a96b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a90103150 0x7f0a9019ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47162/0 (socket says 172.21.15.119:47162) 2023-12-07T10:20:06.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.507+0000 7f0a96b9c700 1 -- 172.21.15.119:0/1183283183 learned_addr learned my addr 172.21.15.119:0/1183283183 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:06.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.507+0000 7f0a95b9a700 1 --2- 172.21.15.119:0/1183283183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a901027b0 0x7f0a90197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:06.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.507+0000 7f0a96b9c700 1 -- 172.21.15.119:0/1183283183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a901027b0 msgr2=0x7f0a90197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:06.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.507+0000 7f0a96b9c700 1 --2- 172.21.15.119:0/1183283183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a901027b0 0x7f0a90197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:06.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.507+0000 7f0a96b9c700 1 -- 172.21.15.119:0/1183283183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a901015c0 msgr2=0x7f0a90197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:06.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.507+0000 7f0a96b9c700 1 --2- 172.21.15.119:0/1183283183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a901015c0 0x7f0a90197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:06.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.507+0000 7f0a96b9c700 1 -- 172.21.15.119:0/1183283183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8c009520 con 0x7f0a90103150 2023-12-07T10:20:06.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.507+0000 7f0a96b9c700 1 --2- 172.21.15.119:0/1183283183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a90103150 0x7f0a9019ebb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0a88007ee0 tx=0x7f0a8800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:06.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.507+0000 7f0a877fe700 1 -- 172.21.15.119:0/1183283183 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8800ddd0 con 0x7f0a90103150 2023-12-07T10:20:06.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.507+0000 7f0a985ff700 1 -- 172.21.15.119:0/1183283183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a900738a0 con 0x7f0a90103150 2023-12-07T10:20:06.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.507+0000 7f0a877fe700 1 -- 172.21.15.119:0/1183283183 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a8800f040 con 0x7f0a90103150 2023-12-07T10:20:06.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.507+0000 7f0a877fe700 1 -- 172.21.15.119:0/1183283183 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a88004800 con 0x7f0a90103150 2023-12-07T10:20:06.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.507+0000 7f0a985ff700 1 -- 172.21.15.119:0/1183283183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a90073db0 con 0x7f0a90103150 2023-12-07T10:20:06.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.508+0000 7f0a985ff700 1 -- 172.21.15.119:0/1183283183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a90108ab0 con 0x7f0a90103150 2023-12-07T10:20:06.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.510+0000 7f0a877fe700 1 -- 172.21.15.119:0/1183283183 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f0a88003a00 con 0x7f0a90103150 2023-12-07T10:20:06.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:06 smithi159 ceph-mon[134726]: pgmap v355: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-12-07T10:20:06.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:06 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1245416183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:20:06.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.511+0000 7f0a877fe700 1 --2- 172.21.15.119:0/1183283183 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0a7c0a1b90 0x7f0a7c0a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:06.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.512+0000 7f0a877fe700 1 -- 172.21.15.119:0/1183283183 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f0a880cd0e0 con 0x7f0a90103150 2023-12-07T10:20:06.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.512+0000 7f0a9639b700 1 --2- 172.21.15.119:0/1183283183 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0a7c0a1b90 0x7f0a7c0a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:06.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.512+0000 7f0a9639b700 1 --2- 172.21.15.119:0/1183283183 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0a7c0a1b90 0x7f0a7c0a4050 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c0098f0 tx=0x7f0a8c01d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:06.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.515+0000 7f0a877fe700 1 -- 172.21.15.119:0/1183283183 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0a8804f2f0 con 0x7f0a90103150 2023-12-07T10:20:06.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:06 smithi153 ceph-mon[133625]: pgmap v355: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-12-07T10:20:06.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:06 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1245416183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:20:06.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.719+0000 7f0a985ff700 1 -- 172.21.15.119:0/1183283183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0a90108490 con 0x7f0a90103150 2023-12-07T10:20:06.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.719+0000 7f0a877fe700 1 -- 172.21.15.119:0/1183283183 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0a88096190 con 0x7f0a90103150 2023-12-07T10:20:06.721 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-12-07T10:20:06.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.722+0000 7f0a985ff700 1 -- 172.21.15.119:0/1183283183 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0a7c0a1b90 msgr2=0x7f0a7c0a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:06.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.722+0000 7f0a985ff700 1 --2- 172.21.15.119:0/1183283183 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0a7c0a1b90 0x7f0a7c0a4050 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c0098f0 tx=0x7f0a8c01d040 comp rx=0 tx=0).stop 2023-12-07T10:20:06.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.722+0000 7f0a985ff700 1 -- 172.21.15.119:0/1183283183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a90103150 msgr2=0x7f0a9019ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:06.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.722+0000 7f0a985ff700 1 --2- 172.21.15.119:0/1183283183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a90103150 0x7f0a9019ebb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0a88007ee0 tx=0x7f0a8800d3b0 comp rx=0 tx=0).stop 2023-12-07T10:20:06.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.722+0000 7f0a985ff700 1 -- 172.21.15.119:0/1183283183 shutdown_connections 2023-12-07T10:20:06.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.722+0000 7f0a985ff700 1 --2- 172.21.15.119:0/1183283183 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0a7c0a1b90 0x7f0a7c0a4050 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:06.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.722+0000 7f0a985ff700 1 --2- 172.21.15.119:0/1183283183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a90103150 0x7f0a9019ebb0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:06.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.722+0000 7f0a985ff700 1 --2- 172.21.15.119:0/1183283183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a901015c0 0x7f0a90197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:06.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.722+0000 7f0a985ff700 1 --2- 172.21.15.119:0/1183283183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a901027b0 0x7f0a90197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:06.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.722+0000 7f0a985ff700 1 -- 172.21.15.119:0/1183283183 >> 172.21.15.119:0/1183283183 conn(0x7f0a900fcb10 msgr2=0x7f0a90105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:06.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.722+0000 7f0a985ff700 1 -- 172.21.15.119:0/1183283183 shutdown_connections 2023-12-07T10:20:06.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.722+0000 7f0a985ff700 1 -- 172.21.15.119:0/1183283183 wait complete. 2023-12-07T10:20:06.742 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-07T10:20:06.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.943+0000 7f57ab5ff700 1 -- 172.21.15.119:0/2685858605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a4107330 msgr2=0x7f57a4109720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:06.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.943+0000 7f57ab5ff700 1 --2- 172.21.15.119:0/2685858605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a4107330 0x7f57a4109720 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f579c009f90 tx=0x7f579c00b330 comp rx=0 tx=0).stop 2023-12-07T10:20:06.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.943+0000 7f57ab5ff700 1 -- 172.21.15.119:0/2685858605 shutdown_connections 2023-12-07T10:20:06.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.943+0000 7f57ab5ff700 1 --2- 172.21.15.119:0/2685858605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a4107330 0x7f57a4109720 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:06.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.943+0000 7f57ab5ff700 1 --2- 172.21.15.119:0/2685858605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57a41020d0 0x7f57a41044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:06.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.943+0000 7f57ab5ff700 1 --2- 172.21.15.119:0/2685858605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57a4104a00 0x7f57a4106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:06.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.943+0000 7f57ab5ff700 1 -- 172.21.15.119:0/2685858605 >> 172.21.15.119:0/2685858605 conn(0x7f57a40fba20 msgr2=0x7f57a40fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:06.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.943+0000 7f57ab5ff700 1 -- 172.21.15.119:0/2685858605 shutdown_connections 2023-12-07T10:20:06.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.943+0000 7f57ab5ff700 1 -- 172.21.15.119:0/2685858605 wait complete. 2023-12-07T10:20:06.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.944+0000 7f57ab5ff700 1 Processor -- start 2023-12-07T10:20:06.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.944+0000 7f57ab5ff700 1 -- start start 2023-12-07T10:20:06.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.944+0000 7f57ab5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a41020d0 0x7f57a4199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:06.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.944+0000 7f57ab5ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57a4104a00 0x7f57a4199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:06.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.944+0000 7f57ab5ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57a4107330 0x7f57a41a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:06.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.944+0000 7f57ab5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a41a1390 con 0x7f57a41020d0 2023-12-07T10:20:06.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.944+0000 7f57ab5ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a41a14d0 con 0x7f57a4104a00 2023-12-07T10:20:06.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.944+0000 7f57ab5ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a41a1660 con 0x7f57a4107330 2023-12-07T10:20:06.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.945+0000 7f57a939b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a41020d0 0x7f57a4199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:06.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.945+0000 7f57a939b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a41020d0 0x7f57a4199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47184/0 (socket says 172.21.15.119:47184) 2023-12-07T10:20:06.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.945+0000 7f57a939b700 1 -- 172.21.15.119:0/2341008095 learned_addr learned my addr 172.21.15.119:0/2341008095 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:06.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.945+0000 7f57a9b9c700 1 --2- 172.21.15.119:0/2341008095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57a4107330 0x7f57a41a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:06.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.945+0000 7f57a8b9a700 1 --2- 172.21.15.119:0/2341008095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57a4104a00 0x7f57a4199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:06.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.945+0000 7f57a9b9c700 1 -- 172.21.15.119:0/2341008095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57a4104a00 msgr2=0x7f57a4199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:06.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.945+0000 7f57a9b9c700 1 --2- 172.21.15.119:0/2341008095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57a4104a00 0x7f57a4199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:06.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.945+0000 7f57a9b9c700 1 -- 172.21.15.119:0/2341008095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a41020d0 msgr2=0x7f57a4199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:06.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.945+0000 7f57a9b9c700 1 --2- 172.21.15.119:0/2341008095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a41020d0 0x7f57a4199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:06.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.945+0000 7f57a9b9c700 1 -- 172.21.15.119:0/2341008095 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c009c40 con 0x7f57a4107330 2023-12-07T10:20:06.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.945+0000 7f57a8b9a700 1 --2- 172.21.15.119:0/2341008095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57a4104a00 0x7f57a4199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:06.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.945+0000 7f57a9b9c700 1 --2- 172.21.15.119:0/2341008095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57a4107330 0x7f57a41a0dc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f579c00b6c0 tx=0x7f579c004c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:06.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.946+0000 7f579a7fc700 1 -- 172.21.15.119:0/2341008095 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c00e040 con 0x7f57a4107330 2023-12-07T10:20:06.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.946+0000 7f57ab5ff700 1 -- 172.21.15.119:0/2341008095 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a41a18e0 con 0x7f57a4107330 2023-12-07T10:20:06.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.946+0000 7f579a7fc700 1 -- 172.21.15.119:0/2341008095 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f579c00bbf0 con 0x7f57a4107330 2023-12-07T10:20:06.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.946+0000 7f579a7fc700 1 -- 172.21.15.119:0/2341008095 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c005040 con 0x7f57a4107330 2023-12-07T10:20:06.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.946+0000 7f57ab5ff700 1 -- 172.21.15.119:0/2341008095 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a41a1db0 con 0x7f57a4107330 2023-12-07T10:20:06.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.947+0000 7f57ab5ff700 1 -- 172.21.15.119:0/2341008095 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a4108eb0 con 0x7f57a4107330 2023-12-07T10:20:06.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.949+0000 7f579a7fc700 1 -- 172.21.15.119:0/2341008095 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f579c01d470 con 0x7f57a4107330 2023-12-07T10:20:06.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.950+0000 7f579a7fc700 1 --2- 172.21.15.119:0/2341008095 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f57900a1b10 0x7f57900a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:06.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.951+0000 7f579a7fc700 1 -- 172.21.15.119:0/2341008095 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f579c0cefe0 con 0x7f57a4107330 2023-12-07T10:20:06.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.951+0000 7f57a939b700 1 --2- 172.21.15.119:0/2341008095 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f57900a1b10 0x7f57900a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:06.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.951+0000 7f57a939b700 1 --2- 172.21.15.119:0/2341008095 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f57900a1b10 0x7f57900a3fd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f57a0009d20 tx=0x7f57a0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:06.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:06.953+0000 7f579a7fc700 1 -- 172.21.15.119:0/2341008095 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f579c098220 con 0x7f57a4107330 2023-12-07T10:20:07.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.153+0000 7f57ab5ff700 1 -- 172.21.15.119:0/2341008095 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f57a40683c0 con 0x7f57a4107330 2023-12-07T10:20:07.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.154+0000 7f579a7fc700 1 -- 172.21.15.119:0/2341008095 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f579c09b840 con 0x7f57a4107330 2023-12-07T10:20:07.155 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-12-07T10:20:07.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.156+0000 7f57ab5ff700 1 -- 172.21.15.119:0/2341008095 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f57900a1b10 msgr2=0x7f57900a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:07.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.156+0000 7f57ab5ff700 1 --2- 172.21.15.119:0/2341008095 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f57900a1b10 0x7f57900a3fd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f57a0009d20 tx=0x7f57a0009260 comp rx=0 tx=0).stop 2023-12-07T10:20:07.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.156+0000 7f57ab5ff700 1 -- 172.21.15.119:0/2341008095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57a4107330 msgr2=0x7f57a41a0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:07.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.156+0000 7f57ab5ff700 1 --2- 172.21.15.119:0/2341008095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57a4107330 0x7f57a41a0dc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f579c00b6c0 tx=0x7f579c004c40 comp rx=0 tx=0).stop 2023-12-07T10:20:07.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.156+0000 7f57ab5ff700 1 -- 172.21.15.119:0/2341008095 shutdown_connections 2023-12-07T10:20:07.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.156+0000 7f57ab5ff700 1 --2- 172.21.15.119:0/2341008095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57a4107330 0x7f57a41a0dc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:07.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.156+0000 7f57ab5ff700 1 --2- 172.21.15.119:0/2341008095 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f57900a1b10 0x7f57900a3fd0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:07.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.156+0000 7f57ab5ff700 1 --2- 172.21.15.119:0/2341008095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a41020d0 0x7f57a4199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:07.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.156+0000 7f57ab5ff700 1 --2- 172.21.15.119:0/2341008095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f57a4104a00 0x7f57a4199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:07.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.156+0000 7f57ab5ff700 1 -- 172.21.15.119:0/2341008095 >> 172.21.15.119:0/2341008095 conn(0x7f57a40fba20 msgr2=0x7f57a41017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:07.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.156+0000 7f57ab5ff700 1 -- 172.21.15.119:0/2341008095 shutdown_connections 2023-12-07T10:20:07.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.156+0000 7f57ab5ff700 1 -- 172.21.15.119:0/2341008095 wait complete. 2023-12-07T10:20:07.177 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-07T10:20:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.376+0000 7faa945ff700 1 -- 172.21.15.119:0/4152245154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa8c1037f0 msgr2=0x7faa8c103c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.376+0000 7faa945ff700 1 --2- 172.21.15.119:0/4152245154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa8c1037f0 0x7faa8c103c10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7faa880099c0 tx=0x7faa88009cd0 comp rx=0 tx=0).stop 2023-12-07T10:20:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.376+0000 7faa945ff700 1 -- 172.21.15.119:0/4152245154 shutdown_connections 2023-12-07T10:20:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.376+0000 7faa945ff700 1 --2- 172.21.15.119:0/4152245154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa8c105380 0x7faa8c10bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.376+0000 7faa945ff700 1 --2- 172.21.15.119:0/4152245154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa8c1037f0 0x7faa8c103c10 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.376+0000 7faa945ff700 1 --2- 172.21.15.119:0/4152245154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8c1049e0 0x7faa8c104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.376+0000 7faa945ff700 1 -- 172.21.15.119:0/4152245154 >> 172.21.15.119:0/4152245154 conn(0x7faa8c0fed60 msgr2=0x7faa8c1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.376+0000 7faa945ff700 1 -- 172.21.15.119:0/4152245154 shutdown_connections 2023-12-07T10:20:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.376+0000 7faa945ff700 1 -- 172.21.15.119:0/4152245154 wait complete. 2023-12-07T10:20:07.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.377+0000 7faa945ff700 1 Processor -- start 2023-12-07T10:20:07.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.377+0000 7faa945ff700 1 -- start start 2023-12-07T10:20:07.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.377+0000 7faa945ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa8c1037f0 0x7faa8c199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:07.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.377+0000 7faa945ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8c1049e0 0x7faa8c199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:07.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.377+0000 7faa945ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa8c105380 0x7faa8c1a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:07.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.377+0000 7faa945ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c1a12a0 con 0x7faa8c105380 2023-12-07T10:20:07.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.377+0000 7faa945ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c1a13e0 con 0x7faa8c1049e0 2023-12-07T10:20:07.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.377+0000 7faa945ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c1a1570 con 0x7faa8c1037f0 2023-12-07T10:20:07.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.378+0000 7faa9239b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa8c1037f0 0x7faa8c199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:07.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.378+0000 7faa92b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa8c105380 0x7faa8c1a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:07.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.378+0000 7faa9239b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa8c1037f0 0x7faa8c199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:58074/0 (socket says 172.21.15.119:58074) 2023-12-07T10:20:07.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.378+0000 7faa9239b700 1 -- 172.21.15.119:0/2382153837 learned_addr learned my addr 172.21.15.119:0/2382153837 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:07.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.378+0000 7faa91b9a700 1 --2- 172.21.15.119:0/2382153837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8c1049e0 0x7faa8c199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:07.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.378+0000 7faa92b9c700 1 -- 172.21.15.119:0/2382153837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa8c1037f0 msgr2=0x7faa8c199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:07.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.378+0000 7faa92b9c700 1 --2- 172.21.15.119:0/2382153837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa8c1037f0 0x7faa8c199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:07.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.378+0000 7faa92b9c700 1 -- 172.21.15.119:0/2382153837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8c1049e0 msgr2=0x7faa8c199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:07.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.378+0000 7faa92b9c700 1 --2- 172.21.15.119:0/2382153837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8c1049e0 0x7faa8c199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:07.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.378+0000 7faa92b9c700 1 -- 172.21.15.119:0/2382153837 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa880095f0 con 0x7faa8c105380 2023-12-07T10:20:07.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.378+0000 7faa92b9c700 1 --2- 172.21.15.119:0/2382153837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa8c105380 0x7faa8c1a0cd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7faa8400efc0 tx=0x7faa8400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:07.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.378+0000 7faa837fe700 1 -- 172.21.15.119:0/2382153837 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa84018070 con 0x7faa8c105380 2023-12-07T10:20:07.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.378+0000 7faa837fe700 1 -- 172.21.15.119:0/2382153837 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa8400f040 con 0x7faa8c105380 2023-12-07T10:20:07.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.378+0000 7faa837fe700 1 -- 172.21.15.119:0/2382153837 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8400cd40 con 0x7faa8c105380 2023-12-07T10:20:07.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.378+0000 7faa945ff700 1 -- 172.21.15.119:0/2382153837 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa8c1a17f0 con 0x7faa8c105380 2023-12-07T10:20:07.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.379+0000 7faa945ff700 1 -- 172.21.15.119:0/2382153837 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa8c1a1cb0 con 0x7faa8c105380 2023-12-07T10:20:07.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.380+0000 7faa945ff700 1 -- 172.21.15.119:0/2382153837 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa8c10ac60 con 0x7faa8c105380 2023-12-07T10:20:07.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.382+0000 7faa837fe700 1 -- 172.21.15.119:0/2382153837 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7faa84003a00 con 0x7faa8c105380 2023-12-07T10:20:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.383+0000 7faa837fe700 1 --2- 172.21.15.119:0/2382153837 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7faa780a1b90 0x7faa780a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.383+0000 7faa837fe700 1 -- 172.21.15.119:0/2382153837 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7faa840cddc0 con 0x7faa8c105380 2023-12-07T10:20:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.383+0000 7faa9239b700 1 --2- 172.21.15.119:0/2382153837 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7faa780a1b90 0x7faa780a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:07.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.384+0000 7faa9239b700 1 --2- 172.21.15.119:0/2382153837 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7faa780a1b90 0x7faa780a4050 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faa88000c00 tx=0x7faa88005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:07.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.385+0000 7faa837fe700 1 -- 172.21.15.119:0/2382153837 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7faa84096ea0 con 0x7faa8c105380 2023-12-07T10:20:07.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:07 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2370646050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:20:07.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:07 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1183283183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:20:07.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:07 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2341008095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:20:07.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:07 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2370646050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:20:07.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:07 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1183283183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:20:07.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:07 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2341008095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:20:07.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:07 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2370646050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:20:07.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:07 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1183283183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:20:07.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:07 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2341008095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:20:07.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.586+0000 7faa945ff700 1 -- 172.21.15.119:0/2382153837 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7faa8c10f050 con 0x7faa8c105380 2023-12-07T10:20:07.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.586+0000 7faa837fe700 1 -- 172.21.15.119:0/2382153837 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7faa8409a4c0 con 0x7faa8c105380 2023-12-07T10:20:07.588 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-12-07T10:20:07.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.589+0000 7faa945ff700 1 -- 172.21.15.119:0/2382153837 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7faa780a1b90 msgr2=0x7faa780a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:07.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.589+0000 7faa945ff700 1 --2- 172.21.15.119:0/2382153837 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7faa780a1b90 0x7faa780a4050 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faa88000c00 tx=0x7faa88005a90 comp rx=0 tx=0).stop 2023-12-07T10:20:07.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.589+0000 7faa945ff700 1 -- 172.21.15.119:0/2382153837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa8c105380 msgr2=0x7faa8c1a0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:07.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.589+0000 7faa945ff700 1 --2- 172.21.15.119:0/2382153837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa8c105380 0x7faa8c1a0cd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7faa8400efc0 tx=0x7faa8400c3a0 comp rx=0 tx=0).stop 2023-12-07T10:20:07.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.589+0000 7faa945ff700 1 -- 172.21.15.119:0/2382153837 shutdown_connections 2023-12-07T10:20:07.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.589+0000 7faa945ff700 1 --2- 172.21.15.119:0/2382153837 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7faa780a1b90 0x7faa780a4050 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:07.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.589+0000 7faa945ff700 1 --2- 172.21.15.119:0/2382153837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa8c105380 0x7faa8c1a0cd0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:07.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.589+0000 7faa945ff700 1 --2- 172.21.15.119:0/2382153837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa8c1037f0 0x7faa8c199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:07.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.589+0000 7faa945ff700 1 --2- 172.21.15.119:0/2382153837 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8c1049e0 0x7faa8c199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:07.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.589+0000 7faa945ff700 1 -- 172.21.15.119:0/2382153837 >> 172.21.15.119:0/2382153837 conn(0x7faa8c0fed60 msgr2=0x7faa8c107da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:07.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.589+0000 7faa945ff700 1 -- 172.21.15.119:0/2382153837 shutdown_connections 2023-12-07T10:20:07.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.589+0000 7faa945ff700 1 -- 172.21.15.119:0/2382153837 wait complete. 2023-12-07T10:20:07.609 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-07T10:20:07.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.806+0000 7f3d0ddff700 1 -- 172.21.15.119:0/376720781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d081030f0 msgr2=0x7f3d081099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:07.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.806+0000 7f3d0ddff700 1 --2- 172.21.15.119:0/376720781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d081030f0 0x7f3d081099a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000b0d0 tx=0x7f3d0000b3e0 comp rx=0 tx=0).stop 2023-12-07T10:20:07.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.806+0000 7f3d0ddff700 1 -- 172.21.15.119:0/376720781 shutdown_connections 2023-12-07T10:20:07.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.806+0000 7f3d0ddff700 1 --2- 172.21.15.119:0/376720781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d081030f0 0x7f3d081099a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:07.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.806+0000 7f3d0ddff700 1 --2- 172.21.15.119:0/376720781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d08101560 0x7f3d08101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:07.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.806+0000 7f3d0ddff700 1 --2- 172.21.15.119:0/376720781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d08102750 0x7f3d08102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:07.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.806+0000 7f3d0ddff700 1 -- 172.21.15.119:0/376720781 >> 172.21.15.119:0/376720781 conn(0x7f3d080fcab0 msgr2=0x7f3d080fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:07.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.806+0000 7f3d0ddff700 1 -- 172.21.15.119:0/376720781 shutdown_connections 2023-12-07T10:20:07.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.806+0000 7f3d0ddff700 1 -- 172.21.15.119:0/376720781 wait complete. 2023-12-07T10:20:07.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.807+0000 7f3d0ddff700 1 Processor -- start 2023-12-07T10:20:07.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.807+0000 7f3d0ddff700 1 -- start start 2023-12-07T10:20:07.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.807+0000 7f3d0ddff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d08101560 0x7f3d081975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:07.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.807+0000 7f3d0ddff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d08102750 0x7f3d08197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:07.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.807+0000 7f3d0ddff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d081030f0 0x7f3d0819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:07.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.807+0000 7f3d0ddff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0819f110 con 0x7f3d08102750 2023-12-07T10:20:07.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.807+0000 7f3d0ddff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0819f250 con 0x7f3d08101560 2023-12-07T10:20:07.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.807+0000 7f3d0ddff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0819f3e0 con 0x7f3d081030f0 2023-12-07T10:20:07.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.807+0000 7f3d077fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d08101560 0x7f3d081975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:07.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.807+0000 7f3d06ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d08102750 0x7f3d08197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:07.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.808+0000 7f3d077fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d08101560 0x7f3d081975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:37470/0 (socket says 172.21.15.119:37470) 2023-12-07T10:20:07.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.808+0000 7f3d077fe700 1 -- 172.21.15.119:0/3422785942 learned_addr learned my addr 172.21.15.119:0/3422785942 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:07.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.808+0000 7f3d07fff700 1 --2- 172.21.15.119:0/3422785942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d081030f0 0x7f3d0819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:07.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.808+0000 7f3d06ffd700 1 -- 172.21.15.119:0/3422785942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d081030f0 msgr2=0x7f3d0819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:07.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.808+0000 7f3d06ffd700 1 --2- 172.21.15.119:0/3422785942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d081030f0 0x7f3d0819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:07.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.808+0000 7f3d06ffd700 1 -- 172.21.15.119:0/3422785942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d08101560 msgr2=0x7f3d081975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:07.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.808+0000 7f3d06ffd700 1 --2- 172.21.15.119:0/3422785942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d08101560 0x7f3d081975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:07.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.808+0000 7f3d06ffd700 1 -- 172.21.15.119:0/3422785942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d00009d00 con 0x7f3d08102750 2023-12-07T10:20:07.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.808+0000 7f3d077fe700 1 --2- 172.21.15.119:0/3422785942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d08101560 0x7f3d081975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:07.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.808+0000 7f3d06ffd700 1 --2- 172.21.15.119:0/3422785942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d08102750 0x7f3d08197b20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00e9f0 tx=0x7f3cfc00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:07.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.808+0000 7f3d04ff9700 1 -- 172.21.15.119:0/3422785942 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc00eee0 con 0x7f3d08102750 2023-12-07T10:20:07.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.808+0000 7f3d04ff9700 1 -- 172.21.15.119:0/3422785942 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cfc00cbd0 con 0x7f3d08102750 2023-12-07T10:20:07.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.808+0000 7f3d0ddff700 1 -- 172.21.15.119:0/3422785942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d0819f6c0 con 0x7f3d08102750 2023-12-07T10:20:07.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.808+0000 7f3d04ff9700 1 -- 172.21.15.119:0/3422785942 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc018680 con 0x7f3d08102750 2023-12-07T10:20:07.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.809+0000 7f3d0ddff700 1 -- 172.21.15.119:0/3422785942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0819fbd0 con 0x7f3d08102750 2023-12-07T10:20:07.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.810+0000 7f3d0ddff700 1 -- 172.21.15.119:0/3422785942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d08108910 con 0x7f3d08102750 2023-12-07T10:20:07.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.811+0000 7f3d04ff9700 1 -- 172.21.15.119:0/3422785942 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f3cfc00cd40 con 0x7f3d08102750 2023-12-07T10:20:07.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.813+0000 7f3d04ff9700 1 --2- 172.21.15.119:0/3422785942 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3cf00a1b10 0x7f3cf00a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:07.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.813+0000 7f3d04ff9700 1 -- 172.21.15.119:0/3422785942 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f3cfc014070 con 0x7f3d08102750 2023-12-07T10:20:07.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.813+0000 7f3d077fe700 1 --2- 172.21.15.119:0/3422785942 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3cf00a1b10 0x7f3cf00a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:07.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.814+0000 7f3d077fe700 1 --2- 172.21.15.119:0/3422785942 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3cf00a1b10 0x7f3cf00a3fd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8005950 tx=0x7f3cf8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:07.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:07.816+0000 7f3d04ff9700 1 -- 172.21.15.119:0/3422785942 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3cfc096b50 con 0x7f3d08102750 2023-12-07T10:20:08.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.022+0000 7f3d0ddff700 1 -- 172.21.15.119:0/3422785942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d08068350 con 0x7f3d08102750 2023-12-07T10:20:08.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.023+0000 7f3d04ff9700 1 -- 172.21.15.119:0/3422785942 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24715 (secure 0 0 0) 0x7f3cfc09a170 con 0x7f3d08102750 2023-12-07T10:20:08.025 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:20:08.025 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":93,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","created":"2023-12-07T10:09:59.945863+0000","modified":"2023-12-07T10:19:02.800626+0000","last_up_change":"2023-12-07T10:16:50.929427+0000","last_in_change":"2023-12-07T10:16:33.558158+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T10:13:25.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-07T10:17:39.347813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-07T10:17:40.714501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-07T10:17:44.257760+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"56b5818e-e625-40f1-8838-3ddcf2d34ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6803","nonce":2141950133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6805","nonce":2141950133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6809","nonce":2141950133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6807","nonce":2141950133}]},"public_addr":"172.21.15.119:6803/2141950133","cluster_addr":"172.21.15.119:6805/2141950133","heartbeat_back_addr":"172.21.15.119:6809/2141950133","heartbeat_front_addr":"172.21.15.119:6807/2141950133","state":["exists","up"]},{"osd":1,"uuid":"36489a6a-5278-4802-9525-9871521722e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6801","nonce":3390875370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6803","nonce":3390875370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6807","nonce":3390875370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6805","nonce":3390875370}]},"public_addr":"172.21.15.153:6801/3390875370","cluster_addr":"172.21.15.153:6803/3390875370","heartbeat_back_addr":"172.21.15.153:6807/3390875370","heartbeat_front_addr":"172.21.15.153:6805/3390875370","state":["exists","up"]},{"osd":2,"uuid":"a937ad26-5e9b-4c8a-b65a-626a8feda6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6801","nonce":708101637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6803","nonce":708101637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6807","nonce":708101637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6805","nonce":708101637}]},"public_addr":"172.21.15.159:6801/708101637","cluster_addr":"172.21.15.159:6803/708101637","heartbeat_back_addr":"172.21.15.159:6807/708101637","heartbeat_front_addr":"172.21.15.159:6805/708101637","state":["exists","up"]},{"osd":3,"uuid":"26ef8300-aa07-48f6-b31d-8b0f2217ffcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6811","nonce":3228047257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6813","nonce":3228047257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6817","nonce":3228047257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6815","nonce":3228047257}]},"public_addr":"172.21.15.119:6811/3228047257","cluster_addr":"172.21.15.119:6813/3228047257","heartbeat_back_addr":"172.21.15.119:6817/3228047257","heartbeat_front_addr":"172.21.15.119:6815/3228047257","state":["exists","up"]},{"osd":4,"uuid":"000ee4b9-9896-4901-8cc3-c85fdfc8cbaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6809","nonce":4021998299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6811","nonce":4021998299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6815","nonce":4021998299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6813","nonce":4021998299}]},"public_addr":"172.21.15.153:6809/4021998299","cluster_addr":"172.21.15.153:6811/4021998299","heartbeat_back_addr":"172.21.15.153:6815/4021998299","heartbeat_front_addr":"172.21.15.153:6813/4021998299","state":["exists","up"]},{"osd":5,"uuid":"8b2fc167-262d-474f-89f3-47bd29424421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6809","nonce":1832540768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6811","nonce":1832540768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6815","nonce":1832540768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6813","nonce":1832540768}]},"public_addr":"172.21.15.159:6809/1832540768","cluster_addr":"172.21.15.159:6811/1832540768","heartbeat_back_addr":"172.21.15.159:6815/1832540768","heartbeat_front_addr":"172.21.15.159:6813/1832540768","state":["exists","up"]},{"osd":6,"uuid":"f9aa6749-46cf-4b2a-8114-cbd034219d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162436840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162436840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6825","nonce":1162436840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162436840}]},"public_addr":"172.21.15.119:6819/1162436840","cluster_addr":"172.21.15.119:6821/1162436840","heartbeat_back_addr":"172.21.15.119:6825/1162436840","heartbeat_front_addr":"172.21.15.119:6823/1162436840","state":["exists","up"]},{"osd":7,"uuid":"39216361-e1c4-4fef-86cb-a4707656b547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6817","nonce":3254653457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6819","nonce":3254653457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6823","nonce":3254653457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6821","nonce":3254653457}]},"public_addr":"172.21.15.153:6817/3254653457","cluster_addr":"172.21.15.153:6819/3254653457","heartbeat_back_addr":"172.21.15.153:6823/3254653457","heartbeat_front_addr":"172.21.15.153:6821/3254653457","state":["exists","up"]},{"osd":8,"uuid":"cceb6ebc-7c02-48ea-9d16-b7a016dd449b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6817","nonce":2600946380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6819","nonce":2600946380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6823","nonce":2600946380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6821","nonce":2600946380}]},"public_addr":"172.21.15.159:6817/2600946380","cluster_addr":"172.21.15.159:6819/2600946380","heartbeat_back_addr":"172.21.15.159:6823/2600946380","heartbeat_front_addr":"172.21.15.159:6821/2600946380","state":["exists","up"]},{"osd":9,"uuid":"f337a2e4-e065-4a9f-91d9-e3615de2d6f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6827","nonce":1940737285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6829","nonce":1940737285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6833","nonce":1940737285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6831","nonce":1940737285}]},"public_addr":"172.21.15.119:6827/1940737285","cluster_addr":"172.21.15.119:6829/1940737285","heartbeat_back_addr":"172.21.15.119:6833/1940737285","heartbeat_front_addr":"172.21.15.119:6831/1940737285","state":["exists","up"]},{"osd":10,"uuid":"3a3bc347-85f4-4878-9797-d9da96137e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6825","nonce":663834864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6827","nonce":663834864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6831","nonce":663834864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6829","nonce":663834864}]},"public_addr":"172.21.15.153:6825/663834864","cluster_addr":"172.21.15.153:6827/663834864","heartbeat_back_addr":"172.21.15.153:6831/663834864","heartbeat_front_addr":"172.21.15.153:6829/663834864","state":["exists","up"]},{"osd":11,"uuid":"a0fc5909-279d-4172-977c-2dc2da0a7e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6825","nonce":1262216348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6827","nonce":1262216348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6831","nonce":1262216348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6829","nonce":1262216348}]},"public_addr":"172.21.15.159:6825/1262216348","cluster_addr":"172.21.15.159:6827/1262216348","heartbeat_back_addr":"172.21.15.159:6831/1262216348","heartbeat_front_addr":"172.21.15.159:6829/1262216348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:37.488109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:59.543060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:21.381585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:47.112729+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:09.732150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:30.893729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:56.678710+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:19.004256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:40.388818+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:06.391734+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:28.394053+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:49.634348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1628639392":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6801/1355617847":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6800/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/362307836":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/128964558":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/2278813554":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/1411933614":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/1685698277":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6800/1355617847":"2023-12-08T10:11:00.585565+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T10:20:08.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.026+0000 7f3d0ddff700 1 -- 172.21.15.119:0/3422785942 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3cf00a1b10 msgr2=0x7f3cf00a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:08.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.026+0000 7f3d0ddff700 1 --2- 172.21.15.119:0/3422785942 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3cf00a1b10 0x7f3cf00a3fd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8005950 tx=0x7f3cf8009260 comp rx=0 tx=0).stop 2023-12-07T10:20:08.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.026+0000 7f3d0ddff700 1 -- 172.21.15.119:0/3422785942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d08102750 msgr2=0x7f3d08197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:08.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.026+0000 7f3d0ddff700 1 --2- 172.21.15.119:0/3422785942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d08102750 0x7f3d08197b20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00e9f0 tx=0x7f3cfc00ed00 comp rx=0 tx=0).stop 2023-12-07T10:20:08.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.026+0000 7f3d0ddff700 1 -- 172.21.15.119:0/3422785942 shutdown_connections 2023-12-07T10:20:08.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.026+0000 7f3d0ddff700 1 --2- 172.21.15.119:0/3422785942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d081030f0 0x7f3d0819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.026+0000 7f3d0ddff700 1 --2- 172.21.15.119:0/3422785942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d08101560 0x7f3d081975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.026+0000 7f3d0ddff700 1 --2- 172.21.15.119:0/3422785942 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f3cf00a1b10 0x7f3cf00a3fd0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.026+0000 7f3d0ddff700 1 --2- 172.21.15.119:0/3422785942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d08102750 0x7f3d08197b20 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.026+0000 7f3d0ddff700 1 -- 172.21.15.119:0/3422785942 >> 172.21.15.119:0/3422785942 conn(0x7f3d080fcab0 msgr2=0x7f3d08105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:08.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.026+0000 7f3d0ddff700 1 -- 172.21.15.119:0/3422785942 shutdown_connections 2023-12-07T10:20:08.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.026+0000 7f3d0ddff700 1 -- 172.21.15.119:0/3422785942 wait complete. 2023-12-07T10:20:08.047 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-07T10:13:25.194366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-07T10:17:39.347813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-07T10:17:40.714501+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-07T10:17:44.257760+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-07T10:20:08.048 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-07T10:20:08.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.249+0000 7f5faf9ff700 1 -- 172.21.15.119:0/3015347903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fa81015c0 msgr2=0x7f5fa81019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:08.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.249+0000 7f5faf9ff700 1 --2- 172.21.15.119:0/3015347903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fa81015c0 0x7f5fa81019e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40099c0 tx=0x7f5fa4009cd0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.249+0000 7f5faf9ff700 1 -- 172.21.15.119:0/3015347903 shutdown_connections 2023-12-07T10:20:08.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.249+0000 7f5faf9ff700 1 --2- 172.21.15.119:0/3015347903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fa8103150 0x7f5fa8109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.249+0000 7f5faf9ff700 1 --2- 172.21.15.119:0/3015347903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fa81015c0 0x7f5fa81019e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.249+0000 7f5faf9ff700 1 --2- 172.21.15.119:0/3015347903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fa81027b0 0x7f5fa8102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.249+0000 7f5faf9ff700 1 -- 172.21.15.119:0/3015347903 >> 172.21.15.119:0/3015347903 conn(0x7f5fa80fcb10 msgr2=0x7f5fa80fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:08.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.249+0000 7f5faf9ff700 1 -- 172.21.15.119:0/3015347903 shutdown_connections 2023-12-07T10:20:08.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.249+0000 7f5faf9ff700 1 -- 172.21.15.119:0/3015347903 wait complete. 2023-12-07T10:20:08.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.250+0000 7f5faf9ff700 1 Processor -- start 2023-12-07T10:20:08.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.250+0000 7f5faf9ff700 1 -- start start 2023-12-07T10:20:08.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.250+0000 7f5faf9ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fa81015c0 0x7f5fa8199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:08.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.251+0000 7f5faf9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fa81027b0 0x7f5fa8199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:08.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.251+0000 7f5faf9ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fa8103150 0x7f5fa81a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:08.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.251+0000 7f5faf9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa81a1290 con 0x7f5fa81027b0 2023-12-07T10:20:08.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.251+0000 7f5faf9ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa81a13d0 con 0x7f5fa8103150 2023-12-07T10:20:08.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.251+0000 7f5faf9ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa81a1560 con 0x7f5fa81015c0 2023-12-07T10:20:08.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.251+0000 7f5facf9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fa81027b0 0x7f5fa8199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:08.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.251+0000 7f5fad79b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fa81015c0 0x7f5fa8199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:08.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.251+0000 7f5facf9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fa81027b0 0x7f5fa8199ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47252/0 (socket says 172.21.15.119:47252) 2023-12-07T10:20:08.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.251+0000 7f5facf9a700 1 -- 172.21.15.119:0/3985192163 learned_addr learned my addr 172.21.15.119:0/3985192163 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:08.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.251+0000 7f5fadf9c700 1 --2- 172.21.15.119:0/3985192163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fa8103150 0x7f5fa81a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:08.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.251+0000 7f5facf9a700 1 -- 172.21.15.119:0/3985192163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fa81015c0 msgr2=0x7f5fa8199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:08.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.251+0000 7f5facf9a700 1 --2- 172.21.15.119:0/3985192163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fa81015c0 0x7f5fa8199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.251+0000 7f5facf9a700 1 -- 172.21.15.119:0/3985192163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fa8103150 msgr2=0x7f5fa81a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:08.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.251+0000 7f5facf9a700 1 --2- 172.21.15.119:0/3985192163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fa8103150 0x7f5fa81a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.251+0000 7f5facf9a700 1 -- 172.21.15.119:0/3985192163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f98009520 con 0x7f5fa81027b0 2023-12-07T10:20:08.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.251+0000 7f5fadf9c700 1 --2- 172.21.15.119:0/3985192163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fa8103150 0x7f5fa81a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:20:08.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.251+0000 7f5facf9a700 1 --2- 172.21.15.119:0/3985192163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fa81027b0 0x7f5fa8199ca0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800eb30 tx=0x7f5f9800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:08.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.252+0000 7f5f9e7fc700 1 -- 172.21.15.119:0/3985192163 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f98009880 con 0x7f5fa81027b0 2023-12-07T10:20:08.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.252+0000 7f5f9e7fc700 1 -- 172.21.15.119:0/3985192163 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f980044e0 con 0x7f5fa81027b0 2023-12-07T10:20:08.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.252+0000 7f5faf9ff700 1 -- 172.21.15.119:0/3985192163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa40095f0 con 0x7f5fa81027b0 2023-12-07T10:20:08.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.252+0000 7f5f9e7fc700 1 -- 172.21.15.119:0/3985192163 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f98003e90 con 0x7f5fa81027b0 2023-12-07T10:20:08.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.252+0000 7f5faf9ff700 1 -- 172.21.15.119:0/3985192163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa81a1c10 con 0x7f5fa81027b0 2023-12-07T10:20:08.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.253+0000 7f5faf9ff700 1 -- 172.21.15.119:0/3985192163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa80683c0 con 0x7f5fa81027b0 2023-12-07T10:20:08.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.255+0000 7f5f9e7fc700 1 -- 172.21.15.119:0/3985192163 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f5f98004000 con 0x7f5fa81027b0 2023-12-07T10:20:08.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.257+0000 7f5f9e7fc700 1 --2- 172.21.15.119:0/3985192163 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5f940a1b90 0x7f5f940a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:08.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.257+0000 7f5f9e7fc700 1 -- 172.21.15.119:0/3985192163 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f5f98014070 con 0x7f5fa81027b0 2023-12-07T10:20:08.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.257+0000 7f5fad79b700 1 --2- 172.21.15.119:0/3985192163 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5f940a1b90 0x7f5f940a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:08.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.258+0000 7f5fad79b700 1 --2- 172.21.15.119:0/3985192163 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5f940a1b90 0x7f5f940a4050 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009990 tx=0x7f5fa4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:08.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.261+0000 7f5f9e7fc700 1 -- 172.21.15.119:0/3985192163 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5f98096db0 con 0x7f5fa81027b0 2023-12-07T10:20:08.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.458+0000 7f5faf9ff700 1 -- 172.21.15.119:0/3985192163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5fa81a1f60 con 0x7f5fa81027b0 2023-12-07T10:20:08.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.459+0000 7f5f9e7fc700 1 -- 172.21.15.119:0/3985192163 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f98083e60 con 0x7f5fa81027b0 2023-12-07T10:20:08.460 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-12-07T10:20:08.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.461+0000 7f5faf9ff700 1 -- 172.21.15.119:0/3985192163 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5f940a1b90 msgr2=0x7f5f940a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:08.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.462+0000 7f5faf9ff700 1 --2- 172.21.15.119:0/3985192163 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5f940a1b90 0x7f5f940a4050 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009990 tx=0x7f5fa4000bc0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.462+0000 7f5faf9ff700 1 -- 172.21.15.119:0/3985192163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fa81027b0 msgr2=0x7f5fa8199ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:08.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.462+0000 7f5faf9ff700 1 --2- 172.21.15.119:0/3985192163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fa81027b0 0x7f5fa8199ca0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800eb30 tx=0x7f5f9800ee40 comp rx=0 tx=0).stop 2023-12-07T10:20:08.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.462+0000 7f5faf9ff700 1 -- 172.21.15.119:0/3985192163 shutdown_connections 2023-12-07T10:20:08.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.462+0000 7f5faf9ff700 1 --2- 172.21.15.119:0/3985192163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fa8103150 0x7f5fa81a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.462+0000 7f5faf9ff700 1 --2- 172.21.15.119:0/3985192163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fa81015c0 0x7f5fa8199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.462+0000 7f5faf9ff700 1 --2- 172.21.15.119:0/3985192163 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f5f940a1b90 0x7f5f940a4050 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.462+0000 7f5faf9ff700 1 --2- 172.21.15.119:0/3985192163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fa81027b0 0x7f5fa8199ca0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.462+0000 7f5faf9ff700 1 -- 172.21.15.119:0/3985192163 >> 172.21.15.119:0/3985192163 conn(0x7f5fa80fcb10 msgr2=0x7f5fa8107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:08.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.462+0000 7f5faf9ff700 1 -- 172.21.15.119:0/3985192163 shutdown_connections 2023-12-07T10:20:08.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.462+0000 7f5faf9ff700 1 -- 172.21.15.119:0/3985192163 wait complete. 2023-12-07T10:20:08.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:08 smithi119 ceph-mon[97908]: pgmap v356: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-07T10:20:08.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:08 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2382153837' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:20:08.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:08 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3422785942' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:20:08.482 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-07T10:20:08.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:08 smithi159 ceph-mon[134726]: pgmap v356: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-07T10:20:08.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:08 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2382153837' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:20:08.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:08 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3422785942' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:20:08.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:08 smithi153 ceph-mon[133625]: pgmap v356: 129 pgs: 129 active+clean; 657 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-07T10:20:08.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:08 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2382153837' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:20:08.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:08 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3422785942' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:20:08.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.681+0000 7fe4ce3ff700 1 -- 172.21.15.119:0/3403629615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c81015c0 msgr2=0x7fe4c81019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:08.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.681+0000 7fe4ce3ff700 1 --2- 172.21.15.119:0/3403629615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c81015c0 0x7fe4c81019e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe4b80099c0 tx=0x7fe4b8009cd0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.681+0000 7fe4ce3ff700 1 -- 172.21.15.119:0/3403629615 shutdown_connections 2023-12-07T10:20:08.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.681+0000 7fe4ce3ff700 1 --2- 172.21.15.119:0/3403629615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4c8103150 0x7fe4c8109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.681+0000 7fe4ce3ff700 1 --2- 172.21.15.119:0/3403629615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c81015c0 0x7fe4c81019e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.681+0000 7fe4ce3ff700 1 --2- 172.21.15.119:0/3403629615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4c81027b0 0x7fe4c8102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.681+0000 7fe4ce3ff700 1 -- 172.21.15.119:0/3403629615 >> 172.21.15.119:0/3403629615 conn(0x7fe4c80fcb10 msgr2=0x7fe4c80fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:08.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.681+0000 7fe4ce3ff700 1 -- 172.21.15.119:0/3403629615 shutdown_connections 2023-12-07T10:20:08.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.681+0000 7fe4ce3ff700 1 -- 172.21.15.119:0/3403629615 wait complete. 2023-12-07T10:20:08.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.682+0000 7fe4ce3ff700 1 Processor -- start 2023-12-07T10:20:08.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.682+0000 7fe4ce3ff700 1 -- start start 2023-12-07T10:20:08.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.682+0000 7fe4ce3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c81015c0 0x7fe4c819dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:08.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.682+0000 7fe4ce3ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4c81027b0 0x7fe4c819e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:08.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.682+0000 7fe4ce3ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4c8103150 0x7fe4c81a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:08.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.682+0000 7fe4ce3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c81a56e0 con 0x7fe4c81015c0 2023-12-07T10:20:08.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.682+0000 7fe4ce3ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c81a5820 con 0x7fe4c8103150 2023-12-07T10:20:08.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.682+0000 7fe4ce3ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c81a59b0 con 0x7fe4c81027b0 2023-12-07T10:20:08.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.683+0000 7fe4c7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c81015c0 0x7fe4c819dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:08.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.683+0000 7fe4c7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c81015c0 0x7fe4c819dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47276/0 (socket says 172.21.15.119:47276) 2023-12-07T10:20:08.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.683+0000 7fe4c7fff700 1 -- 172.21.15.119:0/1305274787 learned_addr learned my addr 172.21.15.119:0/1305274787 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:08.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.683+0000 7fe4c77fe700 1 --2- 172.21.15.119:0/1305274787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4c81027b0 0x7fe4c819e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:08.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.683+0000 7fe4cc99c700 1 --2- 172.21.15.119:0/1305274787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4c8103150 0x7fe4c81a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:08.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.683+0000 7fe4c7fff700 1 -- 172.21.15.119:0/1305274787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4c81027b0 msgr2=0x7fe4c819e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:08.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.683+0000 7fe4c7fff700 1 --2- 172.21.15.119:0/1305274787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4c81027b0 0x7fe4c819e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.683+0000 7fe4c7fff700 1 -- 172.21.15.119:0/1305274787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4c8103150 msgr2=0x7fe4c81a5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:08.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.683+0000 7fe4c7fff700 1 --2- 172.21.15.119:0/1305274787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4c8103150 0x7fe4c81a5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.683+0000 7fe4c7fff700 1 -- 172.21.15.119:0/1305274787 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b80095f0 con 0x7fe4c81015c0 2023-12-07T10:20:08.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.683+0000 7fe4cc99c700 1 --2- 172.21.15.119:0/1305274787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4c8103150 0x7fe4c81a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:20:08.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.683+0000 7fe4c77fe700 1 --2- 172.21.15.119:0/1305274787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4c81027b0 0x7fe4c819e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:20:08.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.683+0000 7fe4c7fff700 1 --2- 172.21.15.119:0/1305274787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c81015c0 0x7fe4c819dbb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8003730 tx=0x7fe4b800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:08.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.684+0000 7fe4c57fa700 1 -- 172.21.15.119:0/1305274787 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b801d070 con 0x7fe4c81015c0 2023-12-07T10:20:08.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.684+0000 7fe4c57fa700 1 -- 172.21.15.119:0/1305274787 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4b800bda0 con 0x7fe4c81015c0 2023-12-07T10:20:08.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.684+0000 7fe4ce3ff700 1 -- 172.21.15.119:0/1305274787 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c81a5c30 con 0x7fe4c81015c0 2023-12-07T10:20:08.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.684+0000 7fe4c57fa700 1 -- 172.21.15.119:0/1305274787 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8017650 con 0x7fe4c81015c0 2023-12-07T10:20:08.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.684+0000 7fe4ce3ff700 1 -- 172.21.15.119:0/1305274787 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c81a60b0 con 0x7fe4c81015c0 2023-12-07T10:20:08.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.685+0000 7fe4ce3ff700 1 -- 172.21.15.119:0/1305274787 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c80683c0 con 0x7fe4c81015c0 2023-12-07T10:20:08.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.686+0000 7fe4c57fa700 1 -- 172.21.15.119:0/1305274787 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fe4b8022470 con 0x7fe4c81015c0 2023-12-07T10:20:08.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.688+0000 7fe4c57fa700 1 --2- 172.21.15.119:0/1305274787 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe4b00a1b90 0x7fe4b00a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:08.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.688+0000 7fe4c57fa700 1 -- 172.21.15.119:0/1305274787 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fe4b80cebf0 con 0x7fe4c81015c0 2023-12-07T10:20:08.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.688+0000 7fe4c77fe700 1 --2- 172.21.15.119:0/1305274787 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe4b00a1b90 0x7fe4b00a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:08.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.689+0000 7fe4c77fe700 1 --2- 172.21.15.119:0/1305274787 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe4b00a1b90 0x7fe4b00a4050 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc005950 tx=0x7fe4bc0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:08.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.691+0000 7fe4c57fa700 1 -- 172.21.15.119:0/1305274787 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe4b8097cd0 con 0x7fe4c81015c0 2023-12-07T10:20:08.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.899+0000 7fe4ce3ff700 1 -- 172.21.15.119:0/1305274787 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe4c81a6340 con 0x7fe4c81015c0 2023-12-07T10:20:08.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.900+0000 7fe4c57fa700 1 -- 172.21.15.119:0/1305274787 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe4b809b2f0 con 0x7fe4c81015c0 2023-12-07T10:20:08.901 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-12-07T10:20:08.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.902+0000 7fe4ce3ff700 1 -- 172.21.15.119:0/1305274787 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe4b00a1b90 msgr2=0x7fe4b00a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:08.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.902+0000 7fe4ce3ff700 1 --2- 172.21.15.119:0/1305274787 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe4b00a1b90 0x7fe4b00a4050 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc005950 tx=0x7fe4bc0058e0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.902+0000 7fe4ce3ff700 1 -- 172.21.15.119:0/1305274787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c81015c0 msgr2=0x7fe4c819dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:08.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.902+0000 7fe4ce3ff700 1 --2- 172.21.15.119:0/1305274787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c81015c0 0x7fe4c819dbb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8003730 tx=0x7fe4b800fab0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.903+0000 7fe4ce3ff700 1 -- 172.21.15.119:0/1305274787 shutdown_connections 2023-12-07T10:20:08.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.903+0000 7fe4ce3ff700 1 --2- 172.21.15.119:0/1305274787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4c8103150 0x7fe4c81a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.903+0000 7fe4ce3ff700 1 --2- 172.21.15.119:0/1305274787 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fe4b00a1b90 0x7fe4b00a4050 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.903+0000 7fe4ce3ff700 1 --2- 172.21.15.119:0/1305274787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c81015c0 0x7fe4c819dbb0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.903+0000 7fe4ce3ff700 1 --2- 172.21.15.119:0/1305274787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4c81027b0 0x7fe4c819e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:08.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.903+0000 7fe4ce3ff700 1 -- 172.21.15.119:0/1305274787 >> 172.21.15.119:0/1305274787 conn(0x7fe4c80fcb10 msgr2=0x7fe4c8073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:08.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.903+0000 7fe4ce3ff700 1 -- 172.21.15.119:0/1305274787 shutdown_connections 2023-12-07T10:20:08.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:08.903+0000 7fe4ce3ff700 1 -- 172.21.15.119:0/1305274787 wait complete. 2023-12-07T10:20:08.923 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-07T10:20:09.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.122+0000 7f0e2b1ff700 1 -- 172.21.15.119:0/4140396953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e241015c0 msgr2=0x7f0e241019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:09.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.122+0000 7f0e2b1ff700 1 --2- 172.21.15.119:0/4140396953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e241015c0 0x7f0e241019e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009990 tx=0x7f0e18009ca0 comp rx=0 tx=0).stop 2023-12-07T10:20:09.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.122+0000 7f0e2b1ff700 1 -- 172.21.15.119:0/4140396953 shutdown_connections 2023-12-07T10:20:09.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.122+0000 7f0e2b1ff700 1 --2- 172.21.15.119:0/4140396953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e24103150 0x7f0e24109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:09.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.122+0000 7f0e2b1ff700 1 --2- 172.21.15.119:0/4140396953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e241015c0 0x7f0e241019e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:09.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.122+0000 7f0e2b1ff700 1 --2- 172.21.15.119:0/4140396953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e241027b0 0x7f0e24102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:09.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.122+0000 7f0e2b1ff700 1 -- 172.21.15.119:0/4140396953 >> 172.21.15.119:0/4140396953 conn(0x7f0e240fcb10 msgr2=0x7f0e240fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:09.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.122+0000 7f0e2b1ff700 1 -- 172.21.15.119:0/4140396953 shutdown_connections 2023-12-07T10:20:09.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.122+0000 7f0e2b1ff700 1 -- 172.21.15.119:0/4140396953 wait complete. 2023-12-07T10:20:09.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.123+0000 7f0e2b1ff700 1 Processor -- start 2023-12-07T10:20:09.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.123+0000 7f0e2b1ff700 1 -- start start 2023-12-07T10:20:09.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.124+0000 7f0e2b1ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e241015c0 0x7f0e241975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:09.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.124+0000 7f0e2b1ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e241027b0 0x7f0e24197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:09.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.124+0000 7f0e2b1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e24103150 0x7f0e2419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:09.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.124+0000 7f0e2b1ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2419f0c0 con 0x7f0e241027b0 2023-12-07T10:20:09.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.124+0000 7f0e2b1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2419f200 con 0x7f0e24103150 2023-12-07T10:20:09.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.124+0000 7f0e2b1ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2419f390 con 0x7f0e241015c0 2023-12-07T10:20:09.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.124+0000 7f0e23fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e241027b0 0x7f0e24197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:09.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.124+0000 7f0e28f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e241015c0 0x7f0e241975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:09.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.124+0000 7f0e28f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e241015c0 0x7f0e241975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:58184/0 (socket says 172.21.15.119:58184) 2023-12-07T10:20:09.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.124+0000 7f0e28f9b700 1 -- 172.21.15.119:0/2528522730 learned_addr learned my addr 172.21.15.119:0/2528522730 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:09.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.124+0000 7f0e2979c700 1 --2- 172.21.15.119:0/2528522730 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e24103150 0x7f0e2419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:09.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.125+0000 7f0e23fff700 1 -- 172.21.15.119:0/2528522730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e241015c0 msgr2=0x7f0e241975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:09.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.125+0000 7f0e23fff700 1 --2- 172.21.15.119:0/2528522730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e241015c0 0x7f0e241975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:09.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.125+0000 7f0e23fff700 1 -- 172.21.15.119:0/2528522730 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e24103150 msgr2=0x7f0e2419eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:09.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.125+0000 7f0e23fff700 1 --2- 172.21.15.119:0/2528522730 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e24103150 0x7f0e2419eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:09.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.125+0000 7f0e23fff700 1 -- 172.21.15.119:0/2528522730 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e180095f0 con 0x7f0e241027b0 2023-12-07T10:20:09.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.125+0000 7f0e28f9b700 1 --2- 172.21.15.119:0/2528522730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e241015c0 0x7f0e241975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:09.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.125+0000 7f0e23fff700 1 --2- 172.21.15.119:0/2528522730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e241027b0 0x7f0e24197b20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000e9f0 tx=0x7f0e1000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:09.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.125+0000 7f0e21ffb700 1 -- 172.21.15.119:0/2528522730 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1000c9c0 con 0x7f0e241027b0 2023-12-07T10:20:09.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.125+0000 7f0e2b1ff700 1 -- 172.21.15.119:0/2528522730 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e2419f670 con 0x7f0e241027b0 2023-12-07T10:20:09.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.125+0000 7f0e21ffb700 1 -- 172.21.15.119:0/2528522730 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e10010430 con 0x7f0e241027b0 2023-12-07T10:20:09.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.125+0000 7f0e21ffb700 1 -- 172.21.15.119:0/2528522730 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e10018680 con 0x7f0e241027b0 2023-12-07T10:20:09.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.125+0000 7f0e2b1ff700 1 -- 172.21.15.119:0/2528522730 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2419fbd0 con 0x7f0e241027b0 2023-12-07T10:20:09.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.126+0000 7f0e2b1ff700 1 -- 172.21.15.119:0/2528522730 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e24108ae0 con 0x7f0e241027b0 2023-12-07T10:20:09.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.128+0000 7f0e21ffb700 1 -- 172.21.15.119:0/2528522730 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f0e10010860 con 0x7f0e241027b0 2023-12-07T10:20:09.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.130+0000 7f0e21ffb700 1 --2- 172.21.15.119:0/2528522730 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0e140a1b90 0x7f0e140a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:09.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.130+0000 7f0e21ffb700 1 -- 172.21.15.119:0/2528522730 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f0e10014070 con 0x7f0e241027b0 2023-12-07T10:20:09.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.130+0000 7f0e28f9b700 1 --2- 172.21.15.119:0/2528522730 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0e140a1b90 0x7f0e140a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:09.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.130+0000 7f0e28f9b700 1 --2- 172.21.15.119:0/2528522730 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0e140a1b90 0x7f0e140a4050 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800b2e0 tx=0x7f0e18005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:09.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.133+0000 7f0e21ffb700 1 -- 172.21.15.119:0/2528522730 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0e100522f0 con 0x7f0e241027b0 2023-12-07T10:20:09.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.341+0000 7f0e2b1ff700 1 -- 172.21.15.119:0/2528522730 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0e241084c0 con 0x7f0e241027b0 2023-12-07T10:20:09.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.341+0000 7f0e21ffb700 1 -- 172.21.15.119:0/2528522730 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0e10096a60 con 0x7f0e241027b0 2023-12-07T10:20:09.343 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-12-07T10:20:09.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.343+0000 7f0e2b1ff700 1 -- 172.21.15.119:0/2528522730 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0e140a1b90 msgr2=0x7f0e140a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:09.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.343+0000 7f0e2b1ff700 1 --2- 172.21.15.119:0/2528522730 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0e140a1b90 0x7f0e140a4050 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800b2e0 tx=0x7f0e18005fd0 comp rx=0 tx=0).stop 2023-12-07T10:20:09.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.343+0000 7f0e2b1ff700 1 -- 172.21.15.119:0/2528522730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e241027b0 msgr2=0x7f0e24197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:09.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.343+0000 7f0e2b1ff700 1 --2- 172.21.15.119:0/2528522730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e241027b0 0x7f0e24197b20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000e9f0 tx=0x7f0e1000ed00 comp rx=0 tx=0).stop 2023-12-07T10:20:09.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.344+0000 7f0e2b1ff700 1 -- 172.21.15.119:0/2528522730 shutdown_connections 2023-12-07T10:20:09.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.344+0000 7f0e2b1ff700 1 --2- 172.21.15.119:0/2528522730 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e24103150 0x7f0e2419eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:09.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.344+0000 7f0e2b1ff700 1 --2- 172.21.15.119:0/2528522730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e241015c0 0x7f0e241975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:09.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.344+0000 7f0e2b1ff700 1 --2- 172.21.15.119:0/2528522730 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f0e140a1b90 0x7f0e140a4050 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:09.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.344+0000 7f0e2b1ff700 1 --2- 172.21.15.119:0/2528522730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e241027b0 0x7f0e24197b20 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:09.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.344+0000 7f0e2b1ff700 1 -- 172.21.15.119:0/2528522730 >> 172.21.15.119:0/2528522730 conn(0x7f0e240fcb10 msgr2=0x7f0e24105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:09.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.344+0000 7f0e2b1ff700 1 -- 172.21.15.119:0/2528522730 shutdown_connections 2023-12-07T10:20:09.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.344+0000 7f0e2b1ff700 1 -- 172.21.15.119:0/2528522730 wait complete. 2023-12-07T10:20:09.364 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-07T10:20:09.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:09 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3985192163' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:20:09.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:09 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1305274787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:20:09.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:09 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3985192163' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:20:09.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:09 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1305274787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:20:09.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:09 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3985192163' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:20:09.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:09 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1305274787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:20:09.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.564+0000 7f83173ff700 1 -- 172.21.15.119:0/4099530295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8310074070 msgr2=0x7f83100744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:09.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.564+0000 7f83173ff700 1 --2- 172.21.15.119:0/4099530295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8310074070 0x7f83100744f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8300009990 tx=0x7f8300009ca0 comp rx=0 tx=0).stop 2023-12-07T10:20:09.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.564+0000 7f83173ff700 1 -- 172.21.15.119:0/4099530295 shutdown_connections 2023-12-07T10:20:09.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.564+0000 7f83173ff700 1 --2- 172.21.15.119:0/4099530295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8310109480 0x7f831010b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:09.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.564+0000 7f83173ff700 1 --2- 172.21.15.119:0/4099530295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8310075650 0x7f8310073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:09.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.564+0000 7f83173ff700 1 --2- 172.21.15.119:0/4099530295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8310074070 0x7f83100744f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:09.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.564+0000 7f83173ff700 1 -- 172.21.15.119:0/4099530295 >> 172.21.15.119:0/4099530295 conn(0x7f83100fcb10 msgr2=0x7f83100fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:09.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.564+0000 7f83173ff700 1 -- 172.21.15.119:0/4099530295 shutdown_connections 2023-12-07T10:20:09.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.564+0000 7f83173ff700 1 -- 172.21.15.119:0/4099530295 wait complete. 2023-12-07T10:20:09.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.565+0000 7f83173ff700 1 Processor -- start 2023-12-07T10:20:09.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.565+0000 7f83173ff700 1 -- start start 2023-12-07T10:20:09.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.566+0000 7f83173ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8310074070 0x7f83101997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:09.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.566+0000 7f83173ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8310075650 0x7f8310199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:09.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.566+0000 7f83173ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8310109480 0x7f83101a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:09.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.566+0000 7f83173ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83101a12a0 con 0x7f8310109480 2023-12-07T10:20:09.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.566+0000 7f83173ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83101a13e0 con 0x7f8310074070 2023-12-07T10:20:09.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.566+0000 7f83173ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83101a1570 con 0x7f8310075650 2023-12-07T10:20:09.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.566+0000 7f831599c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8310109480 0x7f83101a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:09.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.566+0000 7f831599c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8310109480 0x7f83101a0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47316/0 (socket says 172.21.15.119:47316) 2023-12-07T10:20:09.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.566+0000 7f831599c700 1 -- 172.21.15.119:0/145373448 learned_addr learned my addr 172.21.15.119:0/145373448 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:09.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.566+0000 7f831499a700 1 --2- 172.21.15.119:0/145373448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8310075650 0x7f8310199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:09.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.566+0000 7f831519b700 1 --2- 172.21.15.119:0/145373448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8310074070 0x7f83101997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:09.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.566+0000 7f831599c700 1 -- 172.21.15.119:0/145373448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8310075650 msgr2=0x7f8310199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:09.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.566+0000 7f831599c700 1 --2- 172.21.15.119:0/145373448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8310075650 0x7f8310199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:09.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.566+0000 7f831599c700 1 -- 172.21.15.119:0/145373448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8310074070 msgr2=0x7f83101997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:09.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.566+0000 7f831599c700 1 --2- 172.21.15.119:0/145373448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8310074070 0x7f83101997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:09.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.566+0000 7f831599c700 1 -- 172.21.15.119:0/145373448 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83000095f0 con 0x7f8310109480 2023-12-07T10:20:09.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.566+0000 7f831519b700 1 --2- 172.21.15.119:0/145373448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8310074070 0x7f83101997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:20:09.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.567+0000 7f831599c700 1 --2- 172.21.15.119:0/145373448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8310109480 0x7f83101a0cd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f830800ef30 tx=0x7f830800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:09.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.567+0000 7f83067fc700 1 -- 172.21.15.119:0/145373448 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8308018070 con 0x7f8310109480 2023-12-07T10:20:09.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.567+0000 7f83067fc700 1 -- 172.21.15.119:0/145373448 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f830800f040 con 0x7f8310109480 2023-12-07T10:20:09.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.567+0000 7f83173ff700 1 -- 172.21.15.119:0/145373448 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83101a17f0 con 0x7f8310109480 2023-12-07T10:20:09.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.567+0000 7f83067fc700 1 -- 172.21.15.119:0/145373448 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830800cdb0 con 0x7f8310109480 2023-12-07T10:20:09.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.567+0000 7f83173ff700 1 -- 172.21.15.119:0/145373448 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83101a1cb0 con 0x7f8310109480 2023-12-07T10:20:09.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.568+0000 7f83173ff700 1 -- 172.21.15.119:0/145373448 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8310108240 con 0x7f8310109480 2023-12-07T10:20:09.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.570+0000 7f83067fc700 1 -- 172.21.15.119:0/145373448 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f8308003a00 con 0x7f8310109480 2023-12-07T10:20:09.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.571+0000 7f83067fc700 1 --2- 172.21.15.119:0/145373448 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f82fc0a1b10 0x7f82fc0a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:09.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.571+0000 7f83067fc700 1 -- 172.21.15.119:0/145373448 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f83080cd500 con 0x7f8310109480 2023-12-07T10:20:09.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.571+0000 7f831519b700 1 --2- 172.21.15.119:0/145373448 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f82fc0a1b10 0x7f82fc0a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:09.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.572+0000 7f831519b700 1 --2- 172.21.15.119:0/145373448 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f82fc0a1b10 0x7f82fc0a3fd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f830c009cb0 tx=0x7f830c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:09.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.575+0000 7f83067fc700 1 -- 172.21.15.119:0/145373448 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8308096640 con 0x7f8310109480 2023-12-07T10:20:09.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.779+0000 7f83173ff700 1 -- 172.21.15.119:0/145373448 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f831010f000 con 0x7f8310109480 2023-12-07T10:20:09.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.779+0000 7f83067fc700 1 -- 172.21.15.119:0/145373448 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8308099c60 con 0x7f8310109480 2023-12-07T10:20:09.781 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-12-07T10:20:09.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.782+0000 7f83173ff700 1 -- 172.21.15.119:0/145373448 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f82fc0a1b10 msgr2=0x7f82fc0a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:09.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.782+0000 7f83173ff700 1 --2- 172.21.15.119:0/145373448 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f82fc0a1b10 0x7f82fc0a3fd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f830c009cb0 tx=0x7f830c009260 comp rx=0 tx=0).stop 2023-12-07T10:20:09.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.782+0000 7f83173ff700 1 -- 172.21.15.119:0/145373448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8310109480 msgr2=0x7f83101a0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:09.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.782+0000 7f83173ff700 1 --2- 172.21.15.119:0/145373448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8310109480 0x7f83101a0cd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f830800ef30 tx=0x7f830800c3a0 comp rx=0 tx=0).stop 2023-12-07T10:20:09.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.782+0000 7f83173ff700 1 -- 172.21.15.119:0/145373448 shutdown_connections 2023-12-07T10:20:09.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.782+0000 7f83173ff700 1 --2- 172.21.15.119:0/145373448 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f82fc0a1b10 0x7f82fc0a3fd0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:09.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.782+0000 7f83173ff700 1 --2- 172.21.15.119:0/145373448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8310109480 0x7f83101a0cd0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:09.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.782+0000 7f83173ff700 1 --2- 172.21.15.119:0/145373448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8310074070 0x7f83101997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:09.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.782+0000 7f83173ff700 1 --2- 172.21.15.119:0/145373448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8310075650 0x7f8310199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:09.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.782+0000 7f83173ff700 1 -- 172.21.15.119:0/145373448 >> 172.21.15.119:0/145373448 conn(0x7f83100fcb10 msgr2=0x7f8310105380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:09.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.782+0000 7f83173ff700 1 -- 172.21.15.119:0/145373448 shutdown_connections 2023-12-07T10:20:09.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.782+0000 7f83173ff700 1 -- 172.21.15.119:0/145373448 wait complete. 2023-12-07T10:20:09.803 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-12-07T10:20:09.803 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:20:10.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.999+0000 7fa9027ff700 1 -- 172.21.15.119:0/912976032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8fc1030f0 msgr2=0x7fa8fc1099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:10.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.999+0000 7fa9027ff700 1 --2- 172.21.15.119:0/912976032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8fc1030f0 0x7fa8fc1099a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa8f400b0a0 tx=0x7fa8f400b3b0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.999+0000 7fa9027ff700 1 -- 172.21.15.119:0/912976032 shutdown_connections 2023-12-07T10:20:10.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.999+0000 7fa9027ff700 1 --2- 172.21.15.119:0/912976032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8fc1030f0 0x7fa8fc1099a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.999+0000 7fa9027ff700 1 --2- 172.21.15.119:0/912976032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8fc101560 0x7fa8fc101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.999+0000 7fa9027ff700 1 --2- 172.21.15.119:0/912976032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8fc102750 0x7fa8fc102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.999+0000 7fa9027ff700 1 -- 172.21.15.119:0/912976032 >> 172.21.15.119:0/912976032 conn(0x7fa8fc0fcab0 msgr2=0x7fa8fc0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:10.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.999+0000 7fa9027ff700 1 -- 172.21.15.119:0/912976032 shutdown_connections 2023-12-07T10:20:10.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:09.999+0000 7fa9027ff700 1 -- 172.21.15.119:0/912976032 wait complete. 2023-12-07T10:20:10.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.000+0000 7fa9027ff700 1 Processor -- start 2023-12-07T10:20:10.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.000+0000 7fa9027ff700 1 -- start start 2023-12-07T10:20:10.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.001+0000 7fa9027ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8fc101560 0x7fa8fc1975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:10.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.001+0000 7fa9027ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8fc102750 0x7fa8fc197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:10.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.001+0000 7fa9027ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8fc1030f0 0x7fa8fc19eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:10.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.001+0000 7fa9027ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8fc19f100 con 0x7fa8fc1030f0 2023-12-07T10:20:10.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.001+0000 7fa9027ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8fc19f240 con 0x7fa8fc101560 2023-12-07T10:20:10.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.001+0000 7fa9027ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8fc19f3d0 con 0x7fa8fc102750 2023-12-07T10:20:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.001+0000 7fa900d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8fc1030f0 0x7fa8fc19eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.001+0000 7fa8fbfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8fc101560 0x7fa8fc1975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.001+0000 7fa900d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8fc1030f0 0x7fa8fc19eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47326/0 (socket says 172.21.15.119:47326) 2023-12-07T10:20:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.001+0000 7fa900d9c700 1 -- 172.21.15.119:0/2753226587 learned_addr learned my addr 172.21.15.119:0/2753226587 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.001+0000 7fa8fb7fe700 1 --2- 172.21.15.119:0/2753226587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8fc102750 0x7fa8fc197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.001+0000 7fa900d9c700 1 -- 172.21.15.119:0/2753226587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8fc102750 msgr2=0x7fa8fc197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.001+0000 7fa900d9c700 1 --2- 172.21.15.119:0/2753226587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8fc102750 0x7fa8fc197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.001+0000 7fa900d9c700 1 -- 172.21.15.119:0/2753226587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8fc101560 msgr2=0x7fa8fc1975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.001+0000 7fa900d9c700 1 --2- 172.21.15.119:0/2753226587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8fc101560 0x7fa8fc1975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.001+0000 7fa900d9c700 1 -- 172.21.15.119:0/2753226587 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f4009d00 con 0x7fa8fc1030f0 2023-12-07T10:20:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.002+0000 7fa900d9c700 1 --2- 172.21.15.119:0/2753226587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8fc1030f0 0x7fa8fc19eb30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4007a90 tx=0x7fa8f400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.002+0000 7fa8f97fa700 1 -- 172.21.15.119:0/2753226587 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f400e040 con 0x7fa8fc1030f0 2023-12-07T10:20:10.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.002+0000 7fa9027ff700 1 -- 172.21.15.119:0/2753226587 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8fc19f650 con 0x7fa8fc1030f0 2023-12-07T10:20:10.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.002+0000 7fa8f97fa700 1 -- 172.21.15.119:0/2753226587 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8f4004e60 con 0x7fa8fc1030f0 2023-12-07T10:20:10.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.002+0000 7fa8f97fa700 1 -- 172.21.15.119:0/2753226587 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4005240 con 0x7fa8fc1030f0 2023-12-07T10:20:10.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.002+0000 7fa9027ff700 1 -- 172.21.15.119:0/2753226587 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8fc19fb20 con 0x7fa8fc1030f0 2023-12-07T10:20:10.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.003+0000 7fa9027ff700 1 -- 172.21.15.119:0/2753226587 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8fc108910 con 0x7fa8fc1030f0 2023-12-07T10:20:10.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.005+0000 7fa8f97fa700 1 -- 172.21.15.119:0/2753226587 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fa8f401d470 con 0x7fa8fc1030f0 2023-12-07T10:20:10.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.006+0000 7fa8f97fa700 1 --2- 172.21.15.119:0/2753226587 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa8e40a1b10 0x7fa8e40a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:10.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.006+0000 7fa8f97fa700 1 -- 172.21.15.119:0/2753226587 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fa8f40cead0 con 0x7fa8fc1030f0 2023-12-07T10:20:10.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.006+0000 7fa8fbfff700 1 --2- 172.21.15.119:0/2753226587 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa8e40a1b10 0x7fa8e40a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:10.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.007+0000 7fa8fbfff700 1 --2- 172.21.15.119:0/2753226587 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa8e40a1b10 0x7fa8e40a3fd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc1025b0 tx=0x7fa8ec009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:10.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.010+0000 7fa8f97fa700 1 -- 172.21.15.119:0/2753226587 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa8f4097bb0 con 0x7fa8fc1030f0 2023-12-07T10:20:10.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.256+0000 7fa9027ff700 1 -- 172.21.15.119:0/2753226587 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8fc068350 con 0x7fa8fc1030f0 2023-12-07T10:20:10.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.257+0000 7fa8f97fa700 1 -- 172.21.15.119:0/2753226587 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11385 (secure 0 0 0) 0x7fa8f409b1d0 con 0x7fa8fc1030f0 2023-12-07T10:20:10.258 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:20:10.259 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-07T10:17:41.278311+0000","modified":"2023-12-07T10:18:18.543356+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24517,"mds_3":24505,"mds_4":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14535":{"gid":14535,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6841/1791998028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1791998028},{"type":"v1","addr":"172.21.15.119:6841","nonce":1791998028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24505":{"gid":24505,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/2925503246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2925503246},{"type":"v1","addr":"172.21.15.153:6837","nonce":2925503246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6839/3984962900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3984962900},{"type":"v1","addr":"172.21.15.159:6839","nonce":3984962900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.119:6839/1920762602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1920762602},{"type":"v1","addr":"172.21.15.119:6839","nonce":1920762602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/566825184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":566825184},{"type":"v1","addr":"172.21.15.153:6839","nonce":566825184}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-07T10:20:10.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.259+0000 7fa9027ff700 1 -- 172.21.15.119:0/2753226587 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa8e40a1b10 msgr2=0x7fa8e40a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:10.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.259+0000 7fa9027ff700 1 --2- 172.21.15.119:0/2753226587 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa8e40a1b10 0x7fa8e40a3fd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc1025b0 tx=0x7fa8ec009260 comp rx=0 tx=0).stop 2023-12-07T10:20:10.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.259+0000 7fa9027ff700 1 -- 172.21.15.119:0/2753226587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8fc1030f0 msgr2=0x7fa8fc19eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:10.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.259+0000 7fa9027ff700 1 --2- 172.21.15.119:0/2753226587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8fc1030f0 0x7fa8fc19eb30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4007a90 tx=0x7fa8f400bce0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.260+0000 7fa9027ff700 1 -- 172.21.15.119:0/2753226587 shutdown_connections 2023-12-07T10:20:10.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.260+0000 7fa9027ff700 1 --2- 172.21.15.119:0/2753226587 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa8e40a1b10 0x7fa8e40a3fd0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.260+0000 7fa9027ff700 1 --2- 172.21.15.119:0/2753226587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8fc1030f0 0x7fa8fc19eb30 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.260+0000 7fa9027ff700 1 --2- 172.21.15.119:0/2753226587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8fc101560 0x7fa8fc1975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.260+0000 7fa9027ff700 1 --2- 172.21.15.119:0/2753226587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8fc102750 0x7fa8fc197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.260+0000 7fa9027ff700 1 -- 172.21.15.119:0/2753226587 >> 172.21.15.119:0/2753226587 conn(0x7fa8fc0fcab0 msgr2=0x7fa8fc105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:10.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.260+0000 7fa9027ff700 1 -- 172.21.15.119:0/2753226587 shutdown_connections 2023-12-07T10:20:10.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.260+0000 7fa9027ff700 1 -- 172.21.15.119:0/2753226587 wait complete. 2023-12-07T10:20:10.263 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 29 2023-12-07T10:20:10.281 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-12-07T10:20:10.281 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-12-07T10:20:10.281 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-07T10:20:10.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:10 smithi119 ceph-mon[97908]: pgmap v357: 129 pgs: 129 active+clean; 660 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2023-12-07T10:20:10.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:10 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2528522730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:20:10.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:10 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/145373448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:20:10.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.478+0000 7fae193ff700 1 -- 172.21.15.119:0/948446854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae141030f0 msgr2=0x7fae141099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:10.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.478+0000 7fae193ff700 1 --2- 172.21.15.119:0/948446854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae141030f0 0x7fae141099a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00b0d0 tx=0x7fae0c00b3e0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.478+0000 7fae193ff700 1 -- 172.21.15.119:0/948446854 shutdown_connections 2023-12-07T10:20:10.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.478+0000 7fae193ff700 1 --2- 172.21.15.119:0/948446854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae141030f0 0x7fae141099a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.478+0000 7fae193ff700 1 --2- 172.21.15.119:0/948446854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae14101560 0x7fae14101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.478+0000 7fae193ff700 1 --2- 172.21.15.119:0/948446854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae14102750 0x7fae14102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.478+0000 7fae193ff700 1 -- 172.21.15.119:0/948446854 >> 172.21.15.119:0/948446854 conn(0x7fae140fcab0 msgr2=0x7fae140fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:10.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.479+0000 7fae193ff700 1 -- 172.21.15.119:0/948446854 shutdown_connections 2023-12-07T10:20:10.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.479+0000 7fae193ff700 1 -- 172.21.15.119:0/948446854 wait complete. 2023-12-07T10:20:10.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.480+0000 7fae193ff700 1 Processor -- start 2023-12-07T10:20:10.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.480+0000 7fae193ff700 1 -- start start 2023-12-07T10:20:10.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.481+0000 7fae193ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae14101560 0x7fae14197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:10.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.481+0000 7fae193ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae14102750 0x7fae14197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:10.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.481+0000 7fae193ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae141030f0 0x7fae1419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:10.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.481+0000 7fae193ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1419f130 con 0x7fae141030f0 2023-12-07T10:20:10.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.481+0000 7fae193ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1419f270 con 0x7fae14101560 2023-12-07T10:20:10.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.481+0000 7fae193ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1419f400 con 0x7fae14102750 2023-12-07T10:20:10.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.481+0000 7fae137fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae141030f0 0x7fae1419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:10.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.481+0000 7fae137fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae141030f0 0x7fae1419eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47348/0 (socket says 172.21.15.119:47348) 2023-12-07T10:20:10.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.481+0000 7fae137fe700 1 -- 172.21.15.119:0/1983220538 learned_addr learned my addr 172.21.15.119:0/1983220538 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:10.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.481+0000 7fae12ffd700 1 --2- 172.21.15.119:0/1983220538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae14101560 0x7fae14197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:10.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.481+0000 7fae127fc700 1 --2- 172.21.15.119:0/1983220538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae14102750 0x7fae14197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:10.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.481+0000 7fae137fe700 1 -- 172.21.15.119:0/1983220538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae14102750 msgr2=0x7fae14197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:10.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.481+0000 7fae137fe700 1 --2- 172.21.15.119:0/1983220538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae14102750 0x7fae14197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.481+0000 7fae137fe700 1 -- 172.21.15.119:0/1983220538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae14101560 msgr2=0x7fae14197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:10.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.481+0000 7fae137fe700 1 --2- 172.21.15.119:0/1983220538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae14101560 0x7fae14197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.481+0000 7fae137fe700 1 -- 172.21.15.119:0/1983220538 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0c009d00 con 0x7fae141030f0 2023-12-07T10:20:10.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.482+0000 7fae137fe700 1 --2- 172.21.15.119:0/1983220538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae141030f0 0x7fae1419eb60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fae0c007b60 tx=0x7fae0c008e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:10.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.482+0000 7fadfbfff700 1 -- 172.21.15.119:0/1983220538 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c00e040 con 0x7fae141030f0 2023-12-07T10:20:10.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.482+0000 7fadfbfff700 1 -- 172.21.15.119:0/1983220538 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae0c01d930 con 0x7fae141030f0 2023-12-07T10:20:10.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.482+0000 7fadfbfff700 1 -- 172.21.15.119:0/1983220538 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c01b9f0 con 0x7fae141030f0 2023-12-07T10:20:10.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.482+0000 7fae193ff700 1 -- 172.21.15.119:0/1983220538 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1419f680 con 0x7fae141030f0 2023-12-07T10:20:10.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.482+0000 7fae193ff700 1 -- 172.21.15.119:0/1983220538 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae1419fab0 con 0x7fae141030f0 2023-12-07T10:20:10.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.483+0000 7fae193ff700 1 -- 172.21.15.119:0/1983220538 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae14108910 con 0x7fae141030f0 2023-12-07T10:20:10.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.485+0000 7fadfbfff700 1 -- 172.21.15.119:0/1983220538 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fae0c01daa0 con 0x7fae141030f0 2023-12-07T10:20:10.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.486+0000 7fadfbfff700 1 --2- 172.21.15.119:0/1983220538 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fadfc0a1b90 0x7fadfc0a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:10.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.486+0000 7fadfbfff700 1 -- 172.21.15.119:0/1983220538 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fae0c0cecd0 con 0x7fae141030f0 2023-12-07T10:20:10.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.486+0000 7fae12ffd700 1 --2- 172.21.15.119:0/1983220538 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fadfc0a1b90 0x7fadfc0a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:10.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.487+0000 7fae12ffd700 1 --2- 172.21.15.119:0/1983220538 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fadfc0a1b90 0x7fadfc0a4050 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fae141025b0 tx=0x7fae04007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:10.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.490+0000 7fadfbfff700 1 -- 172.21.15.119:0/1983220538 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fae0c097e10 con 0x7fae141030f0 2023-12-07T10:20:10.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:10 smithi159 ceph-mon[134726]: pgmap v357: 129 pgs: 129 active+clean; 660 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2023-12-07T10:20:10.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:10 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2528522730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:20:10.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:10 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/145373448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:20:10.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:10 smithi153 ceph-mon[133625]: pgmap v357: 129 pgs: 129 active+clean; 660 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2023-12-07T10:20:10.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:10 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2528522730' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:20:10.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:10 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/145373448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:20:10.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.689+0000 7fae193ff700 1 -- 172.21.15.119:0/1983220538 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fae1402d460 con 0x7fadfc0a1b90 2023-12-07T10:20:10.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.702+0000 7fadfbfff700 1 -- 172.21.15.119:0/1983220538 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427020 (secure 0 0 0) 0x7fae1402d460 con 0x7fadfc0a1b90 2023-12-07T10:20:10.709 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:20:10.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.707+0000 7fae193ff700 1 -- 172.21.15.119:0/1983220538 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fadfc0a1b90 msgr2=0x7fadfc0a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:10.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.707+0000 7fae193ff700 1 --2- 172.21.15.119:0/1983220538 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fadfc0a1b90 0x7fadfc0a4050 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fae141025b0 tx=0x7fae04007400 comp rx=0 tx=0).stop 2023-12-07T10:20:10.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.707+0000 7fae193ff700 1 -- 172.21.15.119:0/1983220538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae141030f0 msgr2=0x7fae1419eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:10.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.707+0000 7fae193ff700 1 --2- 172.21.15.119:0/1983220538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae141030f0 0x7fae1419eb60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fae0c007b60 tx=0x7fae0c008e90 comp rx=0 tx=0).stop 2023-12-07T10:20:10.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.707+0000 7fae193ff700 1 -- 172.21.15.119:0/1983220538 shutdown_connections 2023-12-07T10:20:10.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.707+0000 7fae193ff700 1 --2- 172.21.15.119:0/1983220538 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fadfc0a1b90 0x7fadfc0a4050 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.707+0000 7fae193ff700 1 --2- 172.21.15.119:0/1983220538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae141030f0 0x7fae1419eb60 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.707+0000 7fae193ff700 1 --2- 172.21.15.119:0/1983220538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae14101560 0x7fae14197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.707+0000 7fae193ff700 1 --2- 172.21.15.119:0/1983220538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae14102750 0x7fae14197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.707+0000 7fae193ff700 1 -- 172.21.15.119:0/1983220538 >> 172.21.15.119:0/1983220538 conn(0x7fae140fcab0 msgr2=0x7fae14105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:10.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.707+0000 7fae193ff700 1 -- 172.21.15.119:0/1983220538 shutdown_connections 2023-12-07T10:20:10.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.707+0000 7fae193ff700 1 -- 172.21.15.119:0/1983220538 wait complete. 2023-12-07T10:20:10.713 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-12-07T10:20:10.728 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2023-12-07T10:20:08.708844+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":675582,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1318,"num_read_kb":800,"num_write":442,"num_write_kb":3106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":865,"ondisk_log_size":865,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338732,"kb_used_data":14540,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603092,"statfs":{"total":1151940427776,"available":1151593566208,"internally_reserved":0,"allocated":14888960,"data_stored":8188727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":331855161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2792,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":57,"num_write":5,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003364"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.734136+0000","last_change":"2023-12-07T10:17:41.614206+0000","last_active":"2023-12-07T10:19:30.734136+0000","last_peered":"2023-12-07T10:19:30.734136+0000","last_clean":"2023-12-07T10:19:30.734136+0000","last_became_active":"2023-12-07T10:17:41.613553+0000","last_became_peered":"2023-12-07T10:17:41.613553+0000","last_unstale":"2023-12-07T10:19:30.734136+0000","last_undegraded":"2023-12-07T10:19:30.734136+0000","last_fullsized":"2023-12-07T10:19:30.734136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:10:22.198414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.729545+0000","last_change":"2023-12-07T10:17:41.616917+0000","last_active":"2023-12-07T10:19:30.729545+0000","last_peered":"2023-12-07T10:19:30.729545+0000","last_clean":"2023-12-07T10:19:30.729545+0000","last_became_active":"2023-12-07T10:17:41.615345+0000","last_became_peered":"2023-12-07T10:17:41.615345+0000","last_unstale":"2023-12-07T10:19:30.729545+0000","last_undegraded":"2023-12-07T10:19:30.729545+0000","last_fullsized":"2023-12-07T10:19:30.729545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:22:25.566738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.733461+0000","last_change":"2023-12-07T10:17:41.619767+0000","last_active":"2023-12-07T10:19:30.733461+0000","last_peered":"2023-12-07T10:19:30.733461+0000","last_clean":"2023-12-07T10:19:30.733461+0000","last_became_active":"2023-12-07T10:17:41.619362+0000","last_became_peered":"2023-12-07T10:17:41.619362+0000","last_unstale":"2023-12-07T10:19:30.733461+0000","last_undegraded":"2023-12-07T10:19:30.733461+0000","last_fullsized":"2023-12-07T10:19:30.733461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:12:55.458520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.730527+0000","last_change":"2023-12-07T10:18:01.861930+0000","last_active":"2023-12-07T10:19:30.730527+0000","last_peered":"2023-12-07T10:19:30.730527+0000","last_clean":"2023-12-07T10:19:30.730527+0000","last_became_active":"2023-12-07T10:18:01.861570+0000","last_became_peered":"2023-12-07T10:18:01.861570+0000","last_unstale":"2023-12-07T10:19:30.730527+0000","last_undegraded":"2023-12-07T10:19:30.730527+0000","last_fullsized":"2023-12-07T10:19:30.730527+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:29:45.234261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.736864+0000","last_change":"2023-12-07T10:17:41.613890+0000","last_active":"2023-12-07T10:19:30.736864+0000","last_peered":"2023-12-07T10:19:30.736864+0000","last_clean":"2023-12-07T10:19:30.736864+0000","last_became_active":"2023-12-07T10:17:41.613073+0000","last_became_peered":"2023-12-07T10:17:41.613073+0000","last_unstale":"2023-12-07T10:19:30.736864+0000","last_undegraded":"2023-12-07T10:19:30.736864+0000","last_fullsized":"2023-12-07T10:19:30.736864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:17:14.339002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.721832+0000","last_change":"2023-12-07T10:17:41.609585+0000","last_active":"2023-12-07T10:19:30.721832+0000","last_peered":"2023-12-07T10:19:30.721832+0000","last_clean":"2023-12-07T10:19:30.721832+0000","last_became_active":"2023-12-07T10:17:41.609256+0000","last_became_peered":"2023-12-07T10:17:41.609256+0000","last_unstale":"2023-12-07T10:19:30.721832+0000","last_undegraded":"2023-12-07T10:19:30.721832+0000","last_fullsized":"2023-12-07T10:19:30.721832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:20:15.103208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.720115+0000","last_change":"2023-12-07T10:17:41.606138+0000","last_active":"2023-12-07T10:19:30.720115+0000","last_peered":"2023-12-07T10:19:30.720115+0000","last_clean":"2023-12-07T10:19:30.720115+0000","last_became_active":"2023-12-07T10:17:41.605818+0000","last_became_peered":"2023-12-07T10:17:41.605818+0000","last_unstale":"2023-12-07T10:19:30.720115+0000","last_undegraded":"2023-12-07T10:19:30.720115+0000","last_fullsized":"2023-12-07T10:19:30.720115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:38:39.244077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.725926+0000","last_change":"2023-12-07T10:18:02.255959+0000","last_active":"2023-12-07T10:19:30.725926+0000","last_peered":"2023-12-07T10:19:30.725926+0000","last_clean":"2023-12-07T10:19:30.725926+0000","last_became_active":"2023-12-07T10:18:02.255560+0000","last_became_peered":"2023-12-07T10:18:02.255560+0000","last_unstale":"2023-12-07T10:19:30.725926+0000","last_undegraded":"2023-12-07T10:19:30.725926+0000","last_fullsized":"2023-12-07T10:19:30.725926+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:05:49.065538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.733953+0000","last_change":"2023-12-07T10:17:41.610225+0000","last_active":"2023-12-07T10:19:30.733953+0000","last_peered":"2023-12-07T10:19:30.733953+0000","last_clean":"2023-12-07T10:19:30.733953+0000","last_became_active":"2023-12-07T10:17:41.609875+0000","last_became_peered":"2023-12-07T10:17:41.609875+0000","last_unstale":"2023-12-07T10:19:30.733953+0000","last_undegraded":"2023-12-07T10:19:30.733953+0000","last_fullsized":"2023-12-07T10:19:30.733953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:00:43.136176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.726358+0000","last_change":"2023-12-07T10:17:41.617335+0000","last_active":"2023-12-07T10:19:30.726358+0000","last_peered":"2023-12-07T10:19:30.726358+0000","last_clean":"2023-12-07T10:19:30.726358+0000","last_became_active":"2023-12-07T10:17:41.617005+0000","last_became_peered":"2023-12-07T10:17:41.617005+0000","last_unstale":"2023-12-07T10:19:30.726358+0000","last_undegraded":"2023-12-07T10:19:30.726358+0000","last_fullsized":"2023-12-07T10:19:30.726358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:43:36.855465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.719323+0000","last_change":"2023-12-07T10:17:41.608104+0000","last_active":"2023-12-07T10:19:30.719323+0000","last_peered":"2023-12-07T10:19:30.719323+0000","last_clean":"2023-12-07T10:19:30.719323+0000","last_became_active":"2023-12-07T10:17:41.607769+0000","last_became_peered":"2023-12-07T10:17:41.607769+0000","last_unstale":"2023-12-07T10:19:30.719323+0000","last_undegraded":"2023-12-07T10:19:30.719323+0000","last_fullsized":"2023-12-07T10:19:30.719323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:47:04.853044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.726603+0000","last_change":"2023-12-07T10:17:41.611585+0000","last_active":"2023-12-07T10:19:30.726603+0000","last_peered":"2023-12-07T10:19:30.726603+0000","last_clean":"2023-12-07T10:19:30.726603+0000","last_became_active":"2023-12-07T10:17:41.611202+0000","last_became_peered":"2023-12-07T10:17:41.611202+0000","last_unstale":"2023-12-07T10:19:30.726603+0000","last_undegraded":"2023-12-07T10:19:30.726603+0000","last_fullsized":"2023-12-07T10:19:30.726603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:17:01.649424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.739580+0000","last_change":"2023-12-07T10:17:41.611116+0000","last_active":"2023-12-07T10:19:30.739580+0000","last_peered":"2023-12-07T10:19:30.739580+0000","last_clean":"2023-12-07T10:19:30.739580+0000","last_became_active":"2023-12-07T10:17:41.610791+0000","last_became_peered":"2023-12-07T10:17:41.610791+0000","last_unstale":"2023-12-07T10:19:30.739580+0000","last_undegraded":"2023-12-07T10:19:30.739580+0000","last_fullsized":"2023-12-07T10:19:30.739580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:48:08.822235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.729464+0000","last_change":"2023-12-07T10:17:41.617447+0000","last_active":"2023-12-07T10:19:30.729464+0000","last_peered":"2023-12-07T10:19:30.729464+0000","last_clean":"2023-12-07T10:19:30.729464+0000","last_became_active":"2023-12-07T10:17:41.612638+0000","last_became_peered":"2023-12-07T10:17:41.612638+0000","last_unstale":"2023-12-07T10:19:30.729464+0000","last_undegraded":"2023-12-07T10:19:30.729464+0000","last_fullsized":"2023-12-07T10:19:30.729464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:11:30.384367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.729222+0000","last_change":"2023-12-07T10:17:41.611607+0000","last_active":"2023-12-07T10:19:30.729222+0000","last_peered":"2023-12-07T10:19:30.729222+0000","last_clean":"2023-12-07T10:19:30.729222+0000","last_became_active":"2023-12-07T10:17:41.611331+0000","last_became_peered":"2023-12-07T10:17:41.611331+0000","last_unstale":"2023-12-07T10:19:30.729222+0000","last_undegraded":"2023-12-07T10:19:30.729222+0000","last_fullsized":"2023-12-07T10:19:30.729222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:17:53.049922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.734247+0000","last_change":"2023-12-07T10:17:41.610528+0000","last_active":"2023-12-07T10:19:30.734247+0000","last_peered":"2023-12-07T10:19:30.734247+0000","last_clean":"2023-12-07T10:19:30.734247+0000","last_became_active":"2023-12-07T10:17:41.610216+0000","last_became_peered":"2023-12-07T10:17:41.610216+0000","last_unstale":"2023-12-07T10:19:30.734247+0000","last_undegraded":"2023-12-07T10:19:30.734247+0000","last_fullsized":"2023-12-07T10:19:30.734247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:11:36.749537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.734059+0000","last_change":"2023-12-07T10:17:41.613967+0000","last_active":"2023-12-07T10:19:30.734059+0000","last_peered":"2023-12-07T10:19:30.734059+0000","last_clean":"2023-12-07T10:19:30.734059+0000","last_became_active":"2023-12-07T10:17:41.613116+0000","last_became_peered":"2023-12-07T10:17:41.613116+0000","last_unstale":"2023-12-07T10:19:30.734059+0000","last_undegraded":"2023-12-07T10:19:30.734059+0000","last_fullsized":"2023-12-07T10:19:30.734059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:46:32.444025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.730539+0000","last_change":"2023-12-07T10:17:41.605478+0000","last_active":"2023-12-07T10:19:30.730539+0000","last_peered":"2023-12-07T10:19:30.730539+0000","last_clean":"2023-12-07T10:19:30.730539+0000","last_became_active":"2023-12-07T10:17:41.605110+0000","last_became_peered":"2023-12-07T10:17:41.605110+0000","last_unstale":"2023-12-07T10:19:30.730539+0000","last_undegraded":"2023-12-07T10:19:30.730539+0000","last_fullsized":"2023-12-07T10:19:30.730539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:53:56.410196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.734012+0000","last_change":"2023-12-07T10:18:02.256377+0000","last_active":"2023-12-07T10:19:30.734012+0000","last_peered":"2023-12-07T10:19:30.734012+0000","last_clean":"2023-12-07T10:19:30.734012+0000","last_became_active":"2023-12-07T10:18:02.255987+0000","last_became_peered":"2023-12-07T10:18:02.255987+0000","last_unstale":"2023-12-07T10:19:30.734012+0000","last_undegraded":"2023-12-07T10:19:30.734012+0000","last_fullsized":"2023-12-07T10:19:30.734012+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:23:34.909187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.809857+0000","last_change":"2023-12-07T10:17:45.799925+0000","last_active":"2023-12-07T10:19:01.809857+0000","last_peered":"2023-12-07T10:19:01.809857+0000","last_clean":"2023-12-07T10:19:01.809857+0000","last_became_active":"2023-12-07T10:17:45.799505+0000","last_became_peered":"2023-12-07T10:17:45.799505+0000","last_unstale":"2023-12-07T10:19:01.809857+0000","last_undegraded":"2023-12-07T10:19:01.809857+0000","last_fullsized":"2023-12-07T10:19:01.809857+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:52:51.470333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'13","reported_seq":147,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:20:02.921049+0000","last_change":"2023-12-07T10:17:40.755066+0000","last_active":"2023-12-07T10:20:02.921049+0000","last_peered":"2023-12-07T10:20:02.921049+0000","last_clean":"2023-12-07T10:20:02.921049+0000","last_became_active":"2023-12-07T10:17:40.754709+0000","last_became_peered":"2023-12-07T10:17:40.754709+0000","last_unstale":"2023-12-07T10:20:02.921049+0000","last_undegraded":"2023-12-07T10:20:02.921049+0000","last_fullsized":"2023-12-07T10:20:02.921049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:18:31.148592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.724533+0000","last_change":"2023-12-07T10:17:41.619107+0000","last_active":"2023-12-07T10:19:30.724533+0000","last_peered":"2023-12-07T10:19:30.724533+0000","last_clean":"2023-12-07T10:19:30.724533+0000","last_became_active":"2023-12-07T10:17:41.617057+0000","last_became_peered":"2023-12-07T10:17:41.617057+0000","last_unstale":"2023-12-07T10:19:30.724533+0000","last_undegraded":"2023-12-07T10:19:30.724533+0000","last_fullsized":"2023-12-07T10:19:30.724533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:53:16.402933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.804762+0000","last_change":"2023-12-07T10:17:45.797582+0000","last_active":"2023-12-07T10:19:01.804762+0000","last_peered":"2023-12-07T10:19:01.804762+0000","last_clean":"2023-12-07T10:19:01.804762+0000","last_became_active":"2023-12-07T10:17:45.796358+0000","last_became_peered":"2023-12-07T10:17:45.796358+0000","last_unstale":"2023-12-07T10:19:01.804762+0000","last_undegraded":"2023-12-07T10:19:01.804762+0000","last_fullsized":"2023-12-07T10:19:01.804762+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:53:19.551459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.214318+0000","last_change":"2023-12-07T10:17:40.762155+0000","last_active":"2023-12-07T10:19:03.214318+0000","last_peered":"2023-12-07T10:19:03.214318+0000","last_clean":"2023-12-07T10:19:03.214318+0000","last_became_active":"2023-12-07T10:17:40.761776+0000","last_became_peered":"2023-12-07T10:17:40.761776+0000","last_unstale":"2023-12-07T10:19:03.214318+0000","last_undegraded":"2023-12-07T10:19:03.214318+0000","last_fullsized":"2023-12-07T10:19:03.214318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:15:06.096331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.734387+0000","last_change":"2023-12-07T10:17:41.614999+0000","last_active":"2023-12-07T10:19:30.734387+0000","last_peered":"2023-12-07T10:19:30.734387+0000","last_clean":"2023-12-07T10:19:30.734387+0000","last_became_active":"2023-12-07T10:17:41.614673+0000","last_became_peered":"2023-12-07T10:17:41.614673+0000","last_unstale":"2023-12-07T10:19:30.734387+0000","last_undegraded":"2023-12-07T10:19:30.734387+0000","last_fullsized":"2023-12-07T10:19:30.734387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:34:45.378843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.215249+0000","last_change":"2023-12-07T10:19:03.215249+0000","last_active":"2023-12-07T10:19:03.215249+0000","last_peered":"2023-12-07T10:19:03.215249+0000","last_clean":"2023-12-07T10:19:03.215249+0000","last_became_active":"2023-12-07T10:19:03.214647+0000","last_became_peered":"2023-12-07T10:19:03.214647+0000","last_unstale":"2023-12-07T10:19:03.215249+0000","last_undegraded":"2023-12-07T10:19:03.215249+0000","last_fullsized":"2023-12-07T10:19:03.215249+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:41:01.893661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808809+0000","last_change":"2023-12-07T10:18:01.993128+0000","last_active":"2023-12-07T10:19:01.808809+0000","last_peered":"2023-12-07T10:19:01.808809+0000","last_clean":"2023-12-07T10:19:01.808809+0000","last_became_active":"2023-12-07T10:18:01.990244+0000","last_became_peered":"2023-12-07T10:18:01.990244+0000","last_unstale":"2023-12-07T10:19:01.808809+0000","last_undegraded":"2023-12-07T10:19:01.808809+0000","last_fullsized":"2023-12-07T10:19:01.808809+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:42:53.778363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.735930+0000","last_change":"2023-12-07T10:17:41.612903+0000","last_active":"2023-12-07T10:19:30.735930+0000","last_peered":"2023-12-07T10:19:30.735930+0000","last_clean":"2023-12-07T10:19:30.735930+0000","last_became_active":"2023-12-07T10:17:41.611968+0000","last_became_peered":"2023-12-07T10:17:41.611968+0000","last_unstale":"2023-12-07T10:19:30.735930+0000","last_undegraded":"2023-12-07T10:19:30.735930+0000","last_fullsized":"2023-12-07T10:19:30.735930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:34:08.442828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.804826+0000","last_change":"2023-12-07T10:17:45.803858+0000","last_active":"2023-12-07T10:19:01.804826+0000","last_peered":"2023-12-07T10:19:01.804826+0000","last_clean":"2023-12-07T10:19:01.804826+0000","last_became_active":"2023-12-07T10:17:45.803481+0000","last_became_peered":"2023-12-07T10:17:45.803481+0000","last_unstale":"2023-12-07T10:19:01.804826+0000","last_undegraded":"2023-12-07T10:19:01.804826+0000","last_fullsized":"2023-12-07T10:19:01.804826+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:08:03.728554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808719+0000","last_change":"2023-12-07T10:17:40.751866+0000","last_active":"2023-12-07T10:19:01.808719+0000","last_peered":"2023-12-07T10:19:01.808719+0000","last_clean":"2023-12-07T10:19:01.808719+0000","last_became_active":"2023-12-07T10:17:40.751492+0000","last_became_peered":"2023-12-07T10:17:40.751492+0000","last_unstale":"2023-12-07T10:19:01.808719+0000","last_undegraded":"2023-12-07T10:19:01.808719+0000","last_fullsized":"2023-12-07T10:19:01.808719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:42:53.839651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.728912+0000","last_change":"2023-12-07T10:17:41.611326+0000","last_active":"2023-12-07T10:19:30.728912+0000","last_peered":"2023-12-07T10:19:30.728912+0000","last_clean":"2023-12-07T10:19:30.728912+0000","last_became_active":"2023-12-07T10:17:41.610863+0000","last_became_peered":"2023-12-07T10:17:41.610863+0000","last_unstale":"2023-12-07T10:19:30.728912+0000","last_undegraded":"2023-12-07T10:19:30.728912+0000","last_fullsized":"2023-12-07T10:19:30.728912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:07:55.801356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810238+0000","last_change":"2023-12-07T10:17:45.791409+0000","last_active":"2023-12-07T10:19:01.810238+0000","last_peered":"2023-12-07T10:19:01.810238+0000","last_clean":"2023-12-07T10:19:01.810238+0000","last_became_active":"2023-12-07T10:17:45.791022+0000","last_became_peered":"2023-12-07T10:17:45.791022+0000","last_unstale":"2023-12-07T10:19:01.810238+0000","last_undegraded":"2023-12-07T10:19:01.810238+0000","last_fullsized":"2023-12-07T10:19:01.810238+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:57:14.617409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'12","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:20:07.916152+0000","last_change":"2023-12-07T10:17:40.745318+0000","last_active":"2023-12-07T10:20:07.916152+0000","last_peered":"2023-12-07T10:20:07.916152+0000","last_clean":"2023-12-07T10:20:07.916152+0000","last_became_active":"2023-12-07T10:17:40.745008+0000","last_became_peered":"2023-12-07T10:17:40.745008+0000","last_unstale":"2023-12-07T10:20:07.916152+0000","last_undegraded":"2023-12-07T10:20:07.916152+0000","last_fullsized":"2023-12-07T10:20:07.916152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:42:28.503421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13625,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":16,"num_write":9,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.724884+0000","last_change":"2023-12-07T10:17:41.605121+0000","last_active":"2023-12-07T10:19:30.724884+0000","last_peered":"2023-12-07T10:19:30.724884+0000","last_clean":"2023-12-07T10:19:30.724884+0000","last_became_active":"2023-12-07T10:17:41.604770+0000","last_became_peered":"2023-12-07T10:17:41.604770+0000","last_unstale":"2023-12-07T10:19:30.724884+0000","last_undegraded":"2023-12-07T10:19:30.724884+0000","last_fullsized":"2023-12-07T10:19:30.724884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:51:02.151155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810286+0000","last_change":"2023-12-07T10:17:45.809039+0000","last_active":"2023-12-07T10:19:01.810286+0000","last_peered":"2023-12-07T10:19:01.810286+0000","last_clean":"2023-12-07T10:19:01.810286+0000","last_became_active":"2023-12-07T10:17:45.808622+0000","last_became_peered":"2023-12-07T10:17:45.808622+0000","last_unstale":"2023-12-07T10:19:01.810286+0000","last_undegraded":"2023-12-07T10:19:01.810286+0000","last_fullsized":"2023-12-07T10:19:01.810286+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:16:02.835931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802888+0000","last_change":"2023-12-07T10:17:40.757876+0000","last_active":"2023-12-07T10:19:01.802888+0000","last_peered":"2023-12-07T10:19:01.802888+0000","last_clean":"2023-12-07T10:19:01.802888+0000","last_became_active":"2023-12-07T10:17:40.757613+0000","last_became_peered":"2023-12-07T10:17:40.757613+0000","last_unstale":"2023-12-07T10:19:01.802888+0000","last_undegraded":"2023-12-07T10:19:01.802888+0000","last_fullsized":"2023-12-07T10:19:01.802888+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:30:40.423092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.732539+0000","last_change":"2023-12-07T10:17:41.611575+0000","last_active":"2023-12-07T10:19:30.732539+0000","last_peered":"2023-12-07T10:19:30.732539+0000","last_clean":"2023-12-07T10:19:30.732539+0000","last_became_active":"2023-12-07T10:17:41.611042+0000","last_became_peered":"2023-12-07T10:17:41.611042+0000","last_unstale":"2023-12-07T10:19:30.732539+0000","last_undegraded":"2023-12-07T10:19:30.732539+0000","last_fullsized":"2023-12-07T10:19:30.732539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:54:05.589792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.214206+0000","last_change":"2023-12-07T10:17:45.796640+0000","last_active":"2023-12-07T10:19:03.214206+0000","last_peered":"2023-12-07T10:19:03.214206+0000","last_clean":"2023-12-07T10:19:03.214206+0000","last_became_active":"2023-12-07T10:17:45.796183+0000","last_became_peered":"2023-12-07T10:17:45.796183+0000","last_unstale":"2023-12-07T10:19:03.214206+0000","last_undegraded":"2023-12-07T10:19:03.214206+0000","last_fullsized":"2023-12-07T10:19:03.214206+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:41:56.890662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.839561+0000","last_change":"2023-12-07T10:17:40.760283+0000","last_active":"2023-12-07T10:19:02.839561+0000","last_peered":"2023-12-07T10:19:02.839561+0000","last_clean":"2023-12-07T10:19:02.839561+0000","last_became_active":"2023-12-07T10:17:40.759398+0000","last_became_peered":"2023-12-07T10:17:40.759398+0000","last_unstale":"2023-12-07T10:19:02.839561+0000","last_undegraded":"2023-12-07T10:19:02.839561+0000","last_fullsized":"2023-12-07T10:19:02.839561+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:34:37.365490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.736136+0000","last_change":"2023-12-07T10:17:41.604150+0000","last_active":"2023-12-07T10:19:30.736136+0000","last_peered":"2023-12-07T10:19:30.736136+0000","last_clean":"2023-12-07T10:19:30.736136+0000","last_became_active":"2023-12-07T10:17:41.603688+0000","last_became_peered":"2023-12-07T10:17:41.603688+0000","last_unstale":"2023-12-07T10:19:30.736136+0000","last_undegraded":"2023-12-07T10:19:30.736136+0000","last_fullsized":"2023-12-07T10:19:30.736136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:52:43.735609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.383739+0000","last_change":"2023-12-07T10:17:45.806569+0000","last_active":"2023-12-07T10:19:03.383739+0000","last_peered":"2023-12-07T10:19:03.383739+0000","last_clean":"2023-12-07T10:19:03.383739+0000","last_became_active":"2023-12-07T10:17:45.806099+0000","last_became_peered":"2023-12-07T10:17:45.806099+0000","last_unstale":"2023-12-07T10:19:03.383739+0000","last_undegraded":"2023-12-07T10:19:03.383739+0000","last_fullsized":"2023-12-07T10:19:03.383739+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:02:19.977670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.818060+0000","last_change":"2023-12-07T10:17:40.744388+0000","last_active":"2023-12-07T10:19:02.818060+0000","last_peered":"2023-12-07T10:19:02.818060+0000","last_clean":"2023-12-07T10:19:02.818060+0000","last_became_active":"2023-12-07T10:17:40.743887+0000","last_became_peered":"2023-12-07T10:17:40.743887+0000","last_unstale":"2023-12-07T10:19:02.818060+0000","last_undegraded":"2023-12-07T10:19:02.818060+0000","last_fullsized":"2023-12-07T10:19:02.818060+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:18:39.102099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.730736+0000","last_change":"2023-12-07T10:17:41.595616+0000","last_active":"2023-12-07T10:19:30.730736+0000","last_peered":"2023-12-07T10:19:30.730736+0000","last_clean":"2023-12-07T10:19:30.730736+0000","last_became_active":"2023-12-07T10:17:41.595256+0000","last_became_peered":"2023-12-07T10:17:41.595256+0000","last_unstale":"2023-12-07T10:19:30.730736+0000","last_undegraded":"2023-12-07T10:19:30.730736+0000","last_fullsized":"2023-12-07T10:19:30.730736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:08:00.156043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.802753+0000","last_change":"2023-12-07T10:17:45.794523+0000","last_active":"2023-12-07T10:19:01.802753+0000","last_peered":"2023-12-07T10:19:01.802753+0000","last_clean":"2023-12-07T10:19:01.802753+0000","last_became_active":"2023-12-07T10:17:45.794139+0000","last_became_peered":"2023-12-07T10:17:45.794139+0000","last_unstale":"2023-12-07T10:19:01.802753+0000","last_undegraded":"2023-12-07T10:19:01.802753+0000","last_fullsized":"2023-12-07T10:19:01.802753+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:43:24.767627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'6","reported_seq":255,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:20:04.923876+0000","last_change":"2023-12-07T10:17:40.759620+0000","last_active":"2023-12-07T10:20:04.923876+0000","last_peered":"2023-12-07T10:20:04.923876+0000","last_clean":"2023-12-07T10:20:04.923876+0000","last_became_active":"2023-12-07T10:17:40.759372+0000","last_became_peered":"2023-12-07T10:17:40.759372+0000","last_unstale":"2023-12-07T10:20:04.923876+0000","last_undegraded":"2023-12-07T10:20:04.923876+0000","last_fullsized":"2023-12-07T10:20:04.923876+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:46:14.459409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":227,"num_read_kb":227,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.731231+0000","last_change":"2023-12-07T10:17:41.601152+0000","last_active":"2023-12-07T10:19:30.731231+0000","last_peered":"2023-12-07T10:19:30.731231+0000","last_clean":"2023-12-07T10:19:30.731231+0000","last_became_active":"2023-12-07T10:17:41.600147+0000","last_became_peered":"2023-12-07T10:17:41.600147+0000","last_unstale":"2023-12-07T10:19:30.731231+0000","last_undegraded":"2023-12-07T10:19:30.731231+0000","last_fullsized":"2023-12-07T10:19:30.731231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:40:46.671489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807019+0000","last_change":"2023-12-07T10:17:45.796229+0000","last_active":"2023-12-07T10:19:01.807019+0000","last_peered":"2023-12-07T10:19:01.807019+0000","last_clean":"2023-12-07T10:19:01.807019+0000","last_became_active":"2023-12-07T10:17:45.795582+0000","last_became_peered":"2023-12-07T10:17:45.795582+0000","last_unstale":"2023-12-07T10:19:01.807019+0000","last_undegraded":"2023-12-07T10:19:01.807019+0000","last_fullsized":"2023-12-07T10:19:01.807019+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:29:32.853975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.809294+0000","last_change":"2023-12-07T10:17:40.748353+0000","last_active":"2023-12-07T10:19:02.809294+0000","last_peered":"2023-12-07T10:19:02.809294+0000","last_clean":"2023-12-07T10:19:02.809294+0000","last_became_active":"2023-12-07T10:17:40.748050+0000","last_became_peered":"2023-12-07T10:17:40.748050+0000","last_unstale":"2023-12-07T10:19:02.809294+0000","last_undegraded":"2023-12-07T10:19:02.809294+0000","last_fullsized":"2023-12-07T10:19:02.809294+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:18:15.298534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.735883+0000","last_change":"2023-12-07T10:18:02.256728+0000","last_active":"2023-12-07T10:19:30.735883+0000","last_peered":"2023-12-07T10:19:30.735883+0000","last_clean":"2023-12-07T10:19:30.735883+0000","last_became_active":"2023-12-07T10:18:02.256271+0000","last_became_peered":"2023-12-07T10:18:02.256271+0000","last_unstale":"2023-12-07T10:19:30.735883+0000","last_undegraded":"2023-12-07T10:19:30.735883+0000","last_fullsized":"2023-12-07T10:19:30.735883+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:27:00.198934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.809834+0000","last_change":"2023-12-07T10:17:45.796903+0000","last_active":"2023-12-07T10:19:02.809834+0000","last_peered":"2023-12-07T10:19:02.809834+0000","last_clean":"2023-12-07T10:19:02.809834+0000","last_became_active":"2023-12-07T10:17:45.796454+0000","last_became_peered":"2023-12-07T10:17:45.796454+0000","last_unstale":"2023-12-07T10:19:02.809834+0000","last_undegraded":"2023-12-07T10:19:02.809834+0000","last_fullsized":"2023-12-07T10:19:02.809834+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:46:36.955396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.818962+0000","last_change":"2023-12-07T10:17:40.756045+0000","last_active":"2023-12-07T10:19:02.818962+0000","last_peered":"2023-12-07T10:19:02.818962+0000","last_clean":"2023-12-07T10:19:02.818962+0000","last_became_active":"2023-12-07T10:17:40.755763+0000","last_became_peered":"2023-12-07T10:17:40.755763+0000","last_unstale":"2023-12-07T10:19:02.818962+0000","last_undegraded":"2023-12-07T10:19:02.818962+0000","last_fullsized":"2023-12-07T10:19:02.818962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:01:08.009076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.736894+0000","last_change":"2023-12-07T10:17:41.611178+0000","last_active":"2023-12-07T10:19:30.736894+0000","last_peered":"2023-12-07T10:19:30.736894+0000","last_clean":"2023-12-07T10:19:30.736894+0000","last_became_active":"2023-12-07T10:17:41.610750+0000","last_became_peered":"2023-12-07T10:17:41.610750+0000","last_unstale":"2023-12-07T10:19:30.736894+0000","last_undegraded":"2023-12-07T10:19:30.736894+0000","last_fullsized":"2023-12-07T10:19:30.736894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:56:30.763523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.807197+0000","last_change":"2023-12-07T10:17:45.800183+0000","last_active":"2023-12-07T10:19:01.807197+0000","last_peered":"2023-12-07T10:19:01.807197+0000","last_clean":"2023-12-07T10:19:01.807197+0000","last_became_active":"2023-12-07T10:17:45.798598+0000","last_became_peered":"2023-12-07T10:17:45.798598+0000","last_unstale":"2023-12-07T10:19:01.807197+0000","last_undegraded":"2023-12-07T10:19:01.807197+0000","last_fullsized":"2023-12-07T10:19:01.807197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:06:22.162951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'39","reported_seq":300,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:20:04.911054+0000","last_change":"2023-12-07T10:17:40.752481+0000","last_active":"2023-12-07T10:20:04.911054+0000","last_peered":"2023-12-07T10:20:04.911054+0000","last_clean":"2023-12-07T10:20:04.911054+0000","last_became_active":"2023-12-07T10:17:40.752158+0000","last_became_peered":"2023-12-07T10:17:40.752158+0000","last_unstale":"2023-12-07T10:20:04.911054+0000","last_undegraded":"2023-12-07T10:20:04.911054+0000","last_fullsized":"2023-12-07T10:20:04.911054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:43:27.963014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65014,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":239,"num_read_kb":183,"num_write":41,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.725283+0000","last_change":"2023-12-07T10:17:41.614107+0000","last_active":"2023-12-07T10:19:30.725283+0000","last_peered":"2023-12-07T10:19:30.725283+0000","last_clean":"2023-12-07T10:19:30.725283+0000","last_became_active":"2023-12-07T10:17:41.613743+0000","last_became_peered":"2023-12-07T10:17:41.613743+0000","last_unstale":"2023-12-07T10:19:30.725283+0000","last_undegraded":"2023-12-07T10:19:30.725283+0000","last_fullsized":"2023-12-07T10:19:30.725283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:49:03.200387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808863+0000","last_change":"2023-12-07T10:17:45.804282+0000","last_active":"2023-12-07T10:19:01.808863+0000","last_peered":"2023-12-07T10:19:01.808863+0000","last_clean":"2023-12-07T10:19:01.808863+0000","last_became_active":"2023-12-07T10:17:45.803794+0000","last_became_peered":"2023-12-07T10:17:45.803794+0000","last_unstale":"2023-12-07T10:19:01.808863+0000","last_undegraded":"2023-12-07T10:19:01.808863+0000","last_fullsized":"2023-12-07T10:19:01.808863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:31:26.887565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.840418+0000","last_change":"2023-12-07T10:17:40.747204+0000","last_active":"2023-12-07T10:19:02.840418+0000","last_peered":"2023-12-07T10:19:02.840418+0000","last_clean":"2023-12-07T10:19:02.840418+0000","last_became_active":"2023-12-07T10:17:40.746916+0000","last_became_peered":"2023-12-07T10:17:40.746916+0000","last_unstale":"2023-12-07T10:19:02.840418+0000","last_undegraded":"2023-12-07T10:19:02.840418+0000","last_fullsized":"2023-12-07T10:19:02.840418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:53:49.228320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.736158+0000","last_change":"2023-12-07T10:17:41.606895+0000","last_active":"2023-12-07T10:19:30.736158+0000","last_peered":"2023-12-07T10:19:30.736158+0000","last_clean":"2023-12-07T10:19:30.736158+0000","last_became_active":"2023-12-07T10:17:41.606596+0000","last_became_peered":"2023-12-07T10:17:41.606596+0000","last_unstale":"2023-12-07T10:19:30.736158+0000","last_undegraded":"2023-12-07T10:19:30.736158+0000","last_fullsized":"2023-12-07T10:19:30.736158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:34:25.166906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.214106+0000","last_change":"2023-12-07T10:17:45.807101+0000","last_active":"2023-12-07T10:19:03.214106+0000","last_peered":"2023-12-07T10:19:03.214106+0000","last_clean":"2023-12-07T10:19:03.214106+0000","last_became_active":"2023-12-07T10:17:45.806770+0000","last_became_peered":"2023-12-07T10:17:45.806770+0000","last_unstale":"2023-12-07T10:19:03.214106+0000","last_undegraded":"2023-12-07T10:19:03.214106+0000","last_fullsized":"2023-12-07T10:19:03.214106+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:39:32.522968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.818391+0000","last_change":"2023-12-07T10:17:40.754856+0000","last_active":"2023-12-07T10:19:02.818391+0000","last_peered":"2023-12-07T10:19:02.818391+0000","last_clean":"2023-12-07T10:19:02.818391+0000","last_became_active":"2023-12-07T10:17:40.754488+0000","last_became_peered":"2023-12-07T10:17:40.754488+0000","last_unstale":"2023-12-07T10:19:02.818391+0000","last_undegraded":"2023-12-07T10:19:02.818391+0000","last_fullsized":"2023-12-07T10:19:02.818391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:44:50.960216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.737528+0000","last_change":"2023-12-07T10:17:41.606963+0000","last_active":"2023-12-07T10:19:30.737528+0000","last_peered":"2023-12-07T10:19:30.737528+0000","last_clean":"2023-12-07T10:19:30.737528+0000","last_became_active":"2023-12-07T10:17:41.606408+0000","last_became_peered":"2023-12-07T10:17:41.606408+0000","last_unstale":"2023-12-07T10:19:30.737528+0000","last_undegraded":"2023-12-07T10:19:30.737528+0000","last_fullsized":"2023-12-07T10:19:30.737528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:25:47.906649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.384918+0000","last_change":"2023-12-07T10:19:03.384918+0000","last_active":"2023-12-07T10:19:03.384918+0000","last_peered":"2023-12-07T10:19:03.384918+0000","last_clean":"2023-12-07T10:19:03.384918+0000","last_became_active":"2023-12-07T10:19:03.384613+0000","last_became_peered":"2023-12-07T10:19:03.384613+0000","last_unstale":"2023-12-07T10:19:03.384918+0000","last_undegraded":"2023-12-07T10:19:03.384918+0000","last_fullsized":"2023-12-07T10:19:03.384918+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:07:29.385953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"93'7","reported_seq":145,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:20:04.924559+0000","last_change":"2023-12-07T10:17:40.747494+0000","last_active":"2023-12-07T10:20:04.924559+0000","last_peered":"2023-12-07T10:20:04.924559+0000","last_clean":"2023-12-07T10:20:04.924559+0000","last_became_active":"2023-12-07T10:17:40.747157+0000","last_became_peered":"2023-12-07T10:17:40.747157+0000","last_unstale":"2023-12-07T10:20:04.924559+0000","last_undegraded":"2023-12-07T10:20:04.924559+0000","last_fullsized":"2023-12-07T10:20:04.924559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:45:24.395418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1117,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.727117+0000","last_change":"2023-12-07T10:17:41.609495+0000","last_active":"2023-12-07T10:19:30.727117+0000","last_peered":"2023-12-07T10:19:30.727117+0000","last_clean":"2023-12-07T10:19:30.727117+0000","last_became_active":"2023-12-07T10:17:41.609181+0000","last_became_peered":"2023-12-07T10:17:41.609181+0000","last_unstale":"2023-12-07T10:19:30.727117+0000","last_undegraded":"2023-12-07T10:19:30.727117+0000","last_fullsized":"2023-12-07T10:19:30.727117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:20:33.473824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813855+0000","last_change":"2023-12-07T10:17:45.808862+0000","last_active":"2023-12-07T10:19:01.813855+0000","last_peered":"2023-12-07T10:19:01.813855+0000","last_clean":"2023-12-07T10:19:01.813855+0000","last_became_active":"2023-12-07T10:17:45.808461+0000","last_became_peered":"2023-12-07T10:17:45.808461+0000","last_unstale":"2023-12-07T10:19:01.813855+0000","last_undegraded":"2023-12-07T10:19:01.813855+0000","last_fullsized":"2023-12-07T10:19:01.813855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:09:12.598126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.810181+0000","last_change":"2023-12-07T10:18:02.263709+0000","last_active":"2023-12-07T10:19:02.810181+0000","last_peered":"2023-12-07T10:19:02.810181+0000","last_clean":"2023-12-07T10:19:02.810181+0000","last_became_active":"2023-12-07T10:18:02.255788+0000","last_became_peered":"2023-12-07T10:18:02.255788+0000","last_unstale":"2023-12-07T10:19:02.810181+0000","last_undegraded":"2023-12-07T10:19:02.810181+0000","last_fullsized":"2023-12-07T10:19:02.810181+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:12:04.976226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.727135+0000","last_change":"2023-12-07T10:17:41.610509+0000","last_active":"2023-12-07T10:19:30.727135+0000","last_peered":"2023-12-07T10:19:30.727135+0000","last_clean":"2023-12-07T10:19:30.727135+0000","last_became_active":"2023-12-07T10:17:41.610121+0000","last_became_peered":"2023-12-07T10:17:41.610121+0000","last_unstale":"2023-12-07T10:19:30.727135+0000","last_undegraded":"2023-12-07T10:19:30.727135+0000","last_fullsized":"2023-12-07T10:19:30.727135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:53:17.365880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.818151+0000","last_change":"2023-12-07T10:17:45.795818+0000","last_active":"2023-12-07T10:19:02.818151+0000","last_peered":"2023-12-07T10:19:02.818151+0000","last_clean":"2023-12-07T10:19:02.818151+0000","last_became_active":"2023-12-07T10:17:45.795477+0000","last_became_peered":"2023-12-07T10:17:45.795477+0000","last_unstale":"2023-12-07T10:19:02.818151+0000","last_undegraded":"2023-12-07T10:19:02.818151+0000","last_fullsized":"2023-12-07T10:19:02.818151+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:17:21.494654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.811207+0000","last_change":"2023-12-07T10:17:40.751791+0000","last_active":"2023-12-07T10:19:02.811207+0000","last_peered":"2023-12-07T10:19:02.811207+0000","last_clean":"2023-12-07T10:19:02.811207+0000","last_became_active":"2023-12-07T10:17:40.751502+0000","last_became_peered":"2023-12-07T10:17:40.751502+0000","last_unstale":"2023-12-07T10:19:02.811207+0000","last_undegraded":"2023-12-07T10:19:02.811207+0000","last_fullsized":"2023-12-07T10:19:02.811207+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:15:52.319265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.723353+0000","last_change":"2023-12-07T10:18:02.256247+0000","last_active":"2023-12-07T10:19:30.723353+0000","last_peered":"2023-12-07T10:19:30.723353+0000","last_clean":"2023-12-07T10:19:30.723353+0000","last_became_active":"2023-12-07T10:18:02.255732+0000","last_became_peered":"2023-12-07T10:18:02.255732+0000","last_unstale":"2023-12-07T10:19:30.723353+0000","last_undegraded":"2023-12-07T10:19:30.723353+0000","last_fullsized":"2023-12-07T10:19:30.723353+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:00:06.516804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.810760+0000","last_change":"2023-12-07T10:17:45.791966+0000","last_active":"2023-12-07T10:19:02.810760+0000","last_peered":"2023-12-07T10:19:02.810760+0000","last_clean":"2023-12-07T10:19:02.810760+0000","last_became_active":"2023-12-07T10:17:45.791613+0000","last_became_peered":"2023-12-07T10:17:45.791613+0000","last_unstale":"2023-12-07T10:19:02.810760+0000","last_undegraded":"2023-12-07T10:19:02.810760+0000","last_fullsized":"2023-12-07T10:19:02.810760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:13:02.291289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.808445+0000","last_change":"2023-12-07T10:17:40.762896+0000","last_active":"2023-12-07T10:19:02.808445+0000","last_peered":"2023-12-07T10:19:02.808445+0000","last_clean":"2023-12-07T10:19:02.808445+0000","last_became_active":"2023-12-07T10:17:40.762427+0000","last_became_peered":"2023-12-07T10:17:40.762427+0000","last_unstale":"2023-12-07T10:19:02.808445+0000","last_undegraded":"2023-12-07T10:19:02.808445+0000","last_fullsized":"2023-12-07T10:19:02.808445+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:56:35.589548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.732249+0000","last_change":"2023-12-07T10:17:41.610354+0000","last_active":"2023-12-07T10:19:30.732249+0000","last_peered":"2023-12-07T10:19:30.732249+0000","last_clean":"2023-12-07T10:19:30.732249+0000","last_became_active":"2023-12-07T10:17:41.609888+0000","last_became_peered":"2023-12-07T10:17:41.609888+0000","last_unstale":"2023-12-07T10:19:30.732249+0000","last_undegraded":"2023-12-07T10:19:30.732249+0000","last_fullsized":"2023-12-07T10:19:30.732249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:48:19.976949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.839882+0000","last_change":"2023-12-07T10:17:45.807324+0000","last_active":"2023-12-07T10:19:02.839882+0000","last_peered":"2023-12-07T10:19:02.839882+0000","last_clean":"2023-12-07T10:19:02.839882+0000","last_became_active":"2023-12-07T10:17:45.806036+0000","last_became_peered":"2023-12-07T10:17:45.806036+0000","last_unstale":"2023-12-07T10:19:02.839882+0000","last_undegraded":"2023-12-07T10:19:02.839882+0000","last_fullsized":"2023-12-07T10:19:02.839882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:54:36.847992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'5","reported_seq":140,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:20:03.923120+0000","last_change":"2023-12-07T10:17:40.754247+0000","last_active":"2023-12-07T10:20:03.923120+0000","last_peered":"2023-12-07T10:20:03.923120+0000","last_clean":"2023-12-07T10:20:03.923120+0000","last_became_active":"2023-12-07T10:17:40.753964+0000","last_became_peered":"2023-12-07T10:17:40.753964+0000","last_unstale":"2023-12-07T10:20:03.923120+0000","last_undegraded":"2023-12-07T10:20:03.923120+0000","last_fullsized":"2023-12-07T10:20:03.923120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:22:04.832399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.723617+0000","last_change":"2023-12-07T10:17:41.610516+0000","last_active":"2023-12-07T10:19:30.723617+0000","last_peered":"2023-12-07T10:19:30.723617+0000","last_clean":"2023-12-07T10:19:30.723617+0000","last_became_active":"2023-12-07T10:17:41.610184+0000","last_became_peered":"2023-12-07T10:17:41.610184+0000","last_unstale":"2023-12-07T10:19:30.723617+0000","last_undegraded":"2023-12-07T10:19:30.723617+0000","last_fullsized":"2023-12-07T10:19:30.723617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:53:43.100723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.840220+0000","last_change":"2023-12-07T10:17:45.804624+0000","last_active":"2023-12-07T10:19:02.840220+0000","last_peered":"2023-12-07T10:19:02.840220+0000","last_clean":"2023-12-07T10:19:02.840220+0000","last_became_active":"2023-12-07T10:17:45.803667+0000","last_became_peered":"2023-12-07T10:17:45.803667+0000","last_unstale":"2023-12-07T10:19:02.840220+0000","last_undegraded":"2023-12-07T10:19:02.840220+0000","last_fullsized":"2023-12-07T10:19:02.840220+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:55:09.642795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.726585+0000","last_change":"2023-12-07T10:17:41.618433+0000","last_active":"2023-12-07T10:19:30.726585+0000","last_peered":"2023-12-07T10:19:30.726585+0000","last_clean":"2023-12-07T10:19:30.726585+0000","last_became_active":"2023-12-07T10:17:41.617942+0000","last_became_peered":"2023-12-07T10:17:41.617942+0000","last_unstale":"2023-12-07T10:19:30.726585+0000","last_undegraded":"2023-12-07T10:19:30.726585+0000","last_fullsized":"2023-12-07T10:19:30.726585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:26:48.166157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":144,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:20:06.926556+0000","last_change":"2023-12-07T10:17:40.761907+0000","last_active":"2023-12-07T10:20:06.926556+0000","last_peered":"2023-12-07T10:20:06.926556+0000","last_clean":"2023-12-07T10:20:06.926556+0000","last_became_active":"2023-12-07T10:17:40.761564+0000","last_became_peered":"2023-12-07T10:17:40.761564+0000","last_unstale":"2023-12-07T10:20:06.926556+0000","last_undegraded":"2023-12-07T10:20:06.926556+0000","last_fullsized":"2023-12-07T10:20:06.926556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:57:22.304352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.809474+0000","last_change":"2023-12-07T10:17:45.792684+0000","last_active":"2023-12-07T10:19:02.809474+0000","last_peered":"2023-12-07T10:19:02.809474+0000","last_clean":"2023-12-07T10:19:02.809474+0000","last_became_active":"2023-12-07T10:17:45.792172+0000","last_became_peered":"2023-12-07T10:17:45.792172+0000","last_unstale":"2023-12-07T10:19:02.809474+0000","last_undegraded":"2023-12-07T10:19:02.809474+0000","last_fullsized":"2023-12-07T10:19:02.809474+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:24:31.084727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.737013+0000","last_change":"2023-12-07T10:17:41.612582+0000","last_active":"2023-12-07T10:19:30.737013+0000","last_peered":"2023-12-07T10:19:30.737013+0000","last_clean":"2023-12-07T10:19:30.737013+0000","last_became_active":"2023-12-07T10:17:41.612244+0000","last_became_peered":"2023-12-07T10:17:41.612244+0000","last_unstale":"2023-12-07T10:19:30.737013+0000","last_undegraded":"2023-12-07T10:19:30.737013+0000","last_fullsized":"2023-12-07T10:19:30.737013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:41:53.506037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.809740+0000","last_change":"2023-12-07T10:17:40.748565+0000","last_active":"2023-12-07T10:19:02.809740+0000","last_peered":"2023-12-07T10:19:02.809740+0000","last_clean":"2023-12-07T10:19:02.809740+0000","last_became_active":"2023-12-07T10:17:40.748250+0000","last_became_peered":"2023-12-07T10:17:40.748250+0000","last_unstale":"2023-12-07T10:19:02.809740+0000","last_undegraded":"2023-12-07T10:19:02.809740+0000","last_fullsized":"2023-12-07T10:19:02.809740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:26:28.202440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":329,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.808386+0000","last_change":"2023-12-07T10:16:33.482173+0000","last_active":"2023-12-07T10:19:02.808386+0000","last_peered":"2023-12-07T10:19:02.808386+0000","last_clean":"2023-12-07T10:19:02.808386+0000","last_became_active":"2023-12-07T10:16:33.481699+0000","last_became_peered":"2023-12-07T10:16:33.481699+0000","last_unstale":"2023-12-07T10:19:02.808386+0000","last_undegraded":"2023-12-07T10:19:02.808386+0000","last_fullsized":"2023-12-07T10:19:02.808386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:13:25.388258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:13:25.388258+0000","last_clean_scrub_stamp":"2023-12-07T10:13:25.388258+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:57:09.583677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.818400+0000","last_change":"2023-12-07T10:17:45.806289+0000","last_active":"2023-12-07T10:19:02.818400+0000","last_peered":"2023-12-07T10:19:02.818400+0000","last_clean":"2023-12-07T10:19:02.818400+0000","last_became_active":"2023-12-07T10:17:45.805954+0000","last_became_peered":"2023-12-07T10:17:45.805954+0000","last_unstale":"2023-12-07T10:19:02.818400+0000","last_undegraded":"2023-12-07T10:19:02.818400+0000","last_fullsized":"2023-12-07T10:19:02.818400+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:21:09.955380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.727877+0000","last_change":"2023-12-07T10:17:41.612322+0000","last_active":"2023-12-07T10:19:30.727877+0000","last_peered":"2023-12-07T10:19:30.727877+0000","last_clean":"2023-12-07T10:19:30.727877+0000","last_became_active":"2023-12-07T10:17:41.612039+0000","last_became_peered":"2023-12-07T10:17:41.612039+0000","last_unstale":"2023-12-07T10:19:30.727877+0000","last_undegraded":"2023-12-07T10:19:30.727877+0000","last_fullsized":"2023-12-07T10:19:30.727877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:40:49.883742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.808419+0000","last_change":"2023-12-07T10:17:40.749694+0000","last_active":"2023-12-07T10:19:02.808419+0000","last_peered":"2023-12-07T10:19:02.808419+0000","last_clean":"2023-12-07T10:19:02.808419+0000","last_became_active":"2023-12-07T10:17:40.749393+0000","last_became_peered":"2023-12-07T10:17:40.749393+0000","last_unstale":"2023-12-07T10:19:02.808419+0000","last_undegraded":"2023-12-07T10:19:02.808419+0000","last_fullsized":"2023-12-07T10:19:02.808419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:15:36.949062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.214038+0000","last_change":"2023-12-07T10:17:45.805769+0000","last_active":"2023-12-07T10:19:03.214038+0000","last_peered":"2023-12-07T10:19:03.214038+0000","last_clean":"2023-12-07T10:19:03.214038+0000","last_became_active":"2023-12-07T10:17:45.805346+0000","last_became_peered":"2023-12-07T10:17:45.805346+0000","last_unstale":"2023-12-07T10:19:03.214038+0000","last_undegraded":"2023-12-07T10:19:03.214038+0000","last_fullsized":"2023-12-07T10:19:03.214038+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:47:47.781933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.214136+0000","last_change":"2023-12-07T10:18:01.862558+0000","last_active":"2023-12-07T10:19:03.214136+0000","last_peered":"2023-12-07T10:19:03.214136+0000","last_clean":"2023-12-07T10:19:03.214136+0000","last_became_active":"2023-12-07T10:18:01.862063+0000","last_became_peered":"2023-12-07T10:18:01.862063+0000","last_unstale":"2023-12-07T10:19:03.214136+0000","last_undegraded":"2023-12-07T10:19:03.214136+0000","last_fullsized":"2023-12-07T10:19:03.214136+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:25:32.652160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.733695+0000","last_change":"2023-12-07T10:17:41.617199+0000","last_active":"2023-12-07T10:19:30.733695+0000","last_peered":"2023-12-07T10:19:30.733695+0000","last_clean":"2023-12-07T10:19:30.733695+0000","last_became_active":"2023-12-07T10:17:41.616901+0000","last_became_peered":"2023-12-07T10:17:41.616901+0000","last_unstale":"2023-12-07T10:19:30.733695+0000","last_undegraded":"2023-12-07T10:19:30.733695+0000","last_fullsized":"2023-12-07T10:19:30.733695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:23:59.433861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.213749+0000","last_change":"2023-12-07T10:17:45.790914+0000","last_active":"2023-12-07T10:19:03.213749+0000","last_peered":"2023-12-07T10:19:03.213749+0000","last_clean":"2023-12-07T10:19:03.213749+0000","last_became_active":"2023-12-07T10:17:45.790428+0000","last_became_peered":"2023-12-07T10:17:45.790428+0000","last_unstale":"2023-12-07T10:19:03.213749+0000","last_undegraded":"2023-12-07T10:19:03.213749+0000","last_fullsized":"2023-12-07T10:19:03.213749+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:03:57.516300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'11","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:33.545539+0000","last_change":"2023-12-07T10:18:01.863765+0000","last_active":"2023-12-07T10:19:33.545539+0000","last_peered":"2023-12-07T10:19:33.545539+0000","last_clean":"2023-12-07T10:19:33.545539+0000","last_became_active":"2023-12-07T10:18:01.858926+0000","last_became_peered":"2023-12-07T10:18:01.858926+0000","last_unstale":"2023-12-07T10:19:33.545539+0000","last_undegraded":"2023-12-07T10:19:33.545539+0000","last_fullsized":"2023-12-07T10:19:33.545539+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:44:19.663592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.733950+0000","last_change":"2023-12-07T10:17:41.608513+0000","last_active":"2023-12-07T10:19:30.733950+0000","last_peered":"2023-12-07T10:19:30.733950+0000","last_clean":"2023-12-07T10:19:30.733950+0000","last_became_active":"2023-12-07T10:17:41.605049+0000","last_became_peered":"2023-12-07T10:17:41.605049+0000","last_unstale":"2023-12-07T10:19:30.733950+0000","last_undegraded":"2023-12-07T10:19:30.733950+0000","last_fullsized":"2023-12-07T10:19:30.733950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:33:00.730827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.840029+0000","last_change":"2023-12-07T10:17:45.792583+0000","last_active":"2023-12-07T10:19:02.840029+0000","last_peered":"2023-12-07T10:19:02.840029+0000","last_clean":"2023-12-07T10:19:02.840029+0000","last_became_active":"2023-12-07T10:17:45.792162+0000","last_became_peered":"2023-12-07T10:17:45.792162+0000","last_unstale":"2023-12-07T10:19:02.840029+0000","last_undegraded":"2023-12-07T10:19:02.840029+0000","last_fullsized":"2023-12-07T10:19:02.840029+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:47:30.901477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.808789+0000","last_change":"2023-12-07T10:17:40.750826+0000","last_active":"2023-12-07T10:19:02.808789+0000","last_peered":"2023-12-07T10:19:02.808789+0000","last_clean":"2023-12-07T10:19:02.808789+0000","last_became_active":"2023-12-07T10:17:40.750519+0000","last_became_peered":"2023-12-07T10:17:40.750519+0000","last_unstale":"2023-12-07T10:19:02.808789+0000","last_undegraded":"2023-12-07T10:19:02.808789+0000","last_fullsized":"2023-12-07T10:19:02.808789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:21:01.008897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.735624+0000","last_change":"2023-12-07T10:17:41.609488+0000","last_active":"2023-12-07T10:19:30.735624+0000","last_peered":"2023-12-07T10:19:30.735624+0000","last_clean":"2023-12-07T10:19:30.735624+0000","last_became_active":"2023-12-07T10:17:41.609061+0000","last_became_peered":"2023-12-07T10:17:41.609061+0000","last_unstale":"2023-12-07T10:19:30.735624+0000","last_undegraded":"2023-12-07T10:19:30.735624+0000","last_fullsized":"2023-12-07T10:19:30.735624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:39:32.729502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.808632+0000","last_change":"2023-12-07T10:17:45.797573+0000","last_active":"2023-12-07T10:19:02.808632+0000","last_peered":"2023-12-07T10:19:02.808632+0000","last_clean":"2023-12-07T10:19:02.808632+0000","last_became_active":"2023-12-07T10:17:45.797119+0000","last_became_peered":"2023-12-07T10:17:45.797119+0000","last_unstale":"2023-12-07T10:19:02.808632+0000","last_undegraded":"2023-12-07T10:19:02.808632+0000","last_fullsized":"2023-12-07T10:19:02.808632+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:17:27.100470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.810618+0000","last_change":"2023-12-07T10:17:40.753596+0000","last_active":"2023-12-07T10:19:02.810618+0000","last_peered":"2023-12-07T10:19:02.810618+0000","last_clean":"2023-12-07T10:19:02.810618+0000","last_became_active":"2023-12-07T10:17:40.753251+0000","last_became_peered":"2023-12-07T10:17:40.753251+0000","last_unstale":"2023-12-07T10:19:02.810618+0000","last_undegraded":"2023-12-07T10:19:02.810618+0000","last_fullsized":"2023-12-07T10:19:02.810618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:08:30.868794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.725751+0000","last_change":"2023-12-07T10:17:41.599689+0000","last_active":"2023-12-07T10:19:30.725751+0000","last_peered":"2023-12-07T10:19:30.725751+0000","last_clean":"2023-12-07T10:19:30.725751+0000","last_became_active":"2023-12-07T10:17:41.599358+0000","last_became_peered":"2023-12-07T10:17:41.599358+0000","last_unstale":"2023-12-07T10:19:30.725751+0000","last_undegraded":"2023-12-07T10:19:30.725751+0000","last_fullsized":"2023-12-07T10:19:30.725751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:44:19.742560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.215169+0000","last_change":"2023-12-07T10:19:03.215169+0000","last_active":"2023-12-07T10:19:03.215169+0000","last_peered":"2023-12-07T10:19:03.215169+0000","last_clean":"2023-12-07T10:19:03.215169+0000","last_became_active":"2023-12-07T10:19:03.214754+0000","last_became_peered":"2023-12-07T10:19:03.214754+0000","last_unstale":"2023-12-07T10:19:03.215169+0000","last_undegraded":"2023-12-07T10:19:03.215169+0000","last_fullsized":"2023-12-07T10:19:03.215169+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:51:41.461131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.213651+0000","last_change":"2023-12-07T10:17:40.758663+0000","last_active":"2023-12-07T10:19:03.213651+0000","last_peered":"2023-12-07T10:19:03.213651+0000","last_clean":"2023-12-07T10:19:03.213651+0000","last_became_active":"2023-12-07T10:17:40.753611+0000","last_became_peered":"2023-12-07T10:17:40.753611+0000","last_unstale":"2023-12-07T10:19:03.213651+0000","last_undegraded":"2023-12-07T10:19:03.213651+0000","last_fullsized":"2023-12-07T10:19:03.213651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:00:59.256468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.730928+0000","last_change":"2023-12-07T10:17:41.616546+0000","last_active":"2023-12-07T10:19:30.730928+0000","last_peered":"2023-12-07T10:19:30.730928+0000","last_clean":"2023-12-07T10:19:30.730928+0000","last_became_active":"2023-12-07T10:17:41.615780+0000","last_became_peered":"2023-12-07T10:17:41.615780+0000","last_unstale":"2023-12-07T10:19:30.730928+0000","last_undegraded":"2023-12-07T10:19:30.730928+0000","last_fullsized":"2023-12-07T10:19:30.730928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:22:32.455784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.809460+0000","last_change":"2023-12-07T10:17:45.797550+0000","last_active":"2023-12-07T10:19:02.809460+0000","last_peered":"2023-12-07T10:19:02.809460+0000","last_clean":"2023-12-07T10:19:02.809460+0000","last_became_active":"2023-12-07T10:17:45.797140+0000","last_became_peered":"2023-12-07T10:17:45.797140+0000","last_unstale":"2023-12-07T10:19:02.809460+0000","last_undegraded":"2023-12-07T10:19:02.809460+0000","last_fullsized":"2023-12-07T10:19:02.809460+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:53:44.984904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"93'5","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:33.049544+0000","last_change":"2023-12-07T10:17:40.748410+0000","last_active":"2023-12-07T10:19:33.049544+0000","last_peered":"2023-12-07T10:19:33.049544+0000","last_clean":"2023-12-07T10:19:33.049544+0000","last_became_active":"2023-12-07T10:17:40.748069+0000","last_became_peered":"2023-12-07T10:17:40.748069+0000","last_unstale":"2023-12-07T10:19:33.049544+0000","last_undegraded":"2023-12-07T10:19:33.049544+0000","last_fullsized":"2023-12-07T10:19:33.049544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:51:48.278974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.734791+0000","last_change":"2023-12-07T10:17:41.609248+0000","last_active":"2023-12-07T10:19:30.734791+0000","last_peered":"2023-12-07T10:19:30.734791+0000","last_clean":"2023-12-07T10:19:30.734791+0000","last_became_active":"2023-12-07T10:17:41.608943+0000","last_became_peered":"2023-12-07T10:17:41.608943+0000","last_unstale":"2023-12-07T10:19:30.734791+0000","last_undegraded":"2023-12-07T10:19:30.734791+0000","last_fullsized":"2023-12-07T10:19:30.734791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:29:32.322273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.808615+0000","last_change":"2023-12-07T10:17:45.794855+0000","last_active":"2023-12-07T10:19:02.808615+0000","last_peered":"2023-12-07T10:19:02.808615+0000","last_clean":"2023-12-07T10:19:02.808615+0000","last_became_active":"2023-12-07T10:17:45.794410+0000","last_became_peered":"2023-12-07T10:17:45.794410+0000","last_unstale":"2023-12-07T10:19:02.808615+0000","last_undegraded":"2023-12-07T10:19:02.808615+0000","last_fullsized":"2023-12-07T10:19:02.808615+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:57:25.118036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.810614+0000","last_change":"2023-12-07T10:17:40.750471+0000","last_active":"2023-12-07T10:19:02.810614+0000","last_peered":"2023-12-07T10:19:02.810614+0000","last_clean":"2023-12-07T10:19:02.810614+0000","last_became_active":"2023-12-07T10:17:40.750176+0000","last_became_peered":"2023-12-07T10:17:40.750176+0000","last_unstale":"2023-12-07T10:19:02.810614+0000","last_undegraded":"2023-12-07T10:19:02.810614+0000","last_fullsized":"2023-12-07T10:19:02.810614+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:04:22.305343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.734958+0000","last_change":"2023-12-07T10:17:41.614087+0000","last_active":"2023-12-07T10:19:30.734958+0000","last_peered":"2023-12-07T10:19:30.734958+0000","last_clean":"2023-12-07T10:19:30.734958+0000","last_became_active":"2023-12-07T10:17:41.607689+0000","last_became_peered":"2023-12-07T10:17:41.607689+0000","last_unstale":"2023-12-07T10:19:30.734958+0000","last_undegraded":"2023-12-07T10:19:30.734958+0000","last_fullsized":"2023-12-07T10:19:30.734958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:50:10.316085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.841410+0000","last_change":"2023-12-07T10:19:02.841001+0000","last_active":"2023-12-07T10:19:02.841410+0000","last_peered":"2023-12-07T10:19:02.841410+0000","last_clean":"2023-12-07T10:19:02.841410+0000","last_became_active":"2023-12-07T10:19:02.840595+0000","last_became_peered":"2023-12-07T10:19:02.840595+0000","last_unstale":"2023-12-07T10:19:02.841410+0000","last_undegraded":"2023-12-07T10:19:02.841410+0000","last_fullsized":"2023-12-07T10:19:02.841410+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:18:35.590140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'5","reported_seq":140,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:20:02.921658+0000","last_change":"2023-12-07T10:17:40.753645+0000","last_active":"2023-12-07T10:20:02.921658+0000","last_peered":"2023-12-07T10:20:02.921658+0000","last_clean":"2023-12-07T10:20:02.921658+0000","last_became_active":"2023-12-07T10:17:40.753326+0000","last_became_peered":"2023-12-07T10:17:40.753326+0000","last_unstale":"2023-12-07T10:20:02.921658+0000","last_undegraded":"2023-12-07T10:20:02.921658+0000","last_fullsized":"2023-12-07T10:20:02.921658+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:43:03.617591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.719211+0000","last_change":"2023-12-07T10:17:41.607968+0000","last_active":"2023-12-07T10:19:30.719211+0000","last_peered":"2023-12-07T10:19:30.719211+0000","last_clean":"2023-12-07T10:19:30.719211+0000","last_became_active":"2023-12-07T10:17:41.607621+0000","last_became_peered":"2023-12-07T10:17:41.607621+0000","last_unstale":"2023-12-07T10:19:30.719211+0000","last_undegraded":"2023-12-07T10:19:30.719211+0000","last_fullsized":"2023-12-07T10:19:30.719211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:55:10.224738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.383569+0000","last_change":"2023-12-07T10:17:45.793678+0000","last_active":"2023-12-07T10:19:03.383569+0000","last_peered":"2023-12-07T10:19:03.383569+0000","last_clean":"2023-12-07T10:19:03.383569+0000","last_became_active":"2023-12-07T10:17:45.793283+0000","last_became_peered":"2023-12-07T10:17:45.793283+0000","last_unstale":"2023-12-07T10:19:03.383569+0000","last_undegraded":"2023-12-07T10:19:03.383569+0000","last_fullsized":"2023-12-07T10:19:03.383569+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:33:56.501107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.818650+0000","last_change":"2023-12-07T10:17:40.762069+0000","last_active":"2023-12-07T10:19:02.818650+0000","last_peered":"2023-12-07T10:19:02.818650+0000","last_clean":"2023-12-07T10:19:02.818650+0000","last_became_active":"2023-12-07T10:17:40.761752+0000","last_became_peered":"2023-12-07T10:17:40.761752+0000","last_unstale":"2023-12-07T10:19:02.818650+0000","last_undegraded":"2023-12-07T10:19:02.818650+0000","last_fullsized":"2023-12-07T10:19:02.818650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:57:55.736144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.731611+0000","last_change":"2023-12-07T10:17:41.607347+0000","last_active":"2023-12-07T10:19:30.731611+0000","last_peered":"2023-12-07T10:19:30.731611+0000","last_clean":"2023-12-07T10:19:30.731611+0000","last_became_active":"2023-12-07T10:17:41.606618+0000","last_became_peered":"2023-12-07T10:17:41.606618+0000","last_unstale":"2023-12-07T10:19:30.731611+0000","last_undegraded":"2023-12-07T10:19:30.731611+0000","last_fullsized":"2023-12-07T10:19:30.731611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:53:34.748130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.817908+0000","last_change":"2023-12-07T10:17:45.791459+0000","last_active":"2023-12-07T10:19:02.817908+0000","last_peered":"2023-12-07T10:19:02.817908+0000","last_clean":"2023-12-07T10:19:02.817908+0000","last_became_active":"2023-12-07T10:17:45.791070+0000","last_became_peered":"2023-12-07T10:17:45.791070+0000","last_unstale":"2023-12-07T10:19:02.817908+0000","last_undegraded":"2023-12-07T10:19:02.817908+0000","last_fullsized":"2023-12-07T10:19:02.817908+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:46:35.985990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.840125+0000","last_change":"2023-12-07T10:17:40.762726+0000","last_active":"2023-12-07T10:19:02.840125+0000","last_peered":"2023-12-07T10:19:02.840125+0000","last_clean":"2023-12-07T10:19:02.840125+0000","last_became_active":"2023-12-07T10:17:40.762378+0000","last_became_peered":"2023-12-07T10:17:40.762378+0000","last_unstale":"2023-12-07T10:19:02.840125+0000","last_undegraded":"2023-12-07T10:19:02.840125+0000","last_fullsized":"2023-12-07T10:19:02.840125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:42:06.193834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.718804+0000","last_change":"2023-12-07T10:17:41.602356+0000","last_active":"2023-12-07T10:19:30.718804+0000","last_peered":"2023-12-07T10:19:30.718804+0000","last_clean":"2023-12-07T10:19:30.718804+0000","last_became_active":"2023-12-07T10:17:41.602025+0000","last_became_peered":"2023-12-07T10:17:41.602025+0000","last_unstale":"2023-12-07T10:19:30.718804+0000","last_undegraded":"2023-12-07T10:19:30.718804+0000","last_fullsized":"2023-12-07T10:19:30.718804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:33:04.994510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.728025+0000","last_change":"2023-12-07T10:17:41.616766+0000","last_active":"2023-12-07T10:19:30.728025+0000","last_peered":"2023-12-07T10:19:30.728025+0000","last_clean":"2023-12-07T10:19:30.728025+0000","last_became_active":"2023-12-07T10:17:41.616221+0000","last_became_peered":"2023-12-07T10:17:41.616221+0000","last_unstale":"2023-12-07T10:19:30.728025+0000","last_undegraded":"2023-12-07T10:19:30.728025+0000","last_fullsized":"2023-12-07T10:19:30.728025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:16:00.393772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.715723+0000","last_change":"2023-12-07T10:17:41.613646+0000","last_active":"2023-12-07T10:19:30.715723+0000","last_peered":"2023-12-07T10:19:30.715723+0000","last_clean":"2023-12-07T10:19:30.715723+0000","last_became_active":"2023-12-07T10:17:41.613257+0000","last_became_peered":"2023-12-07T10:17:41.613257+0000","last_unstale":"2023-12-07T10:19:30.715723+0000","last_undegraded":"2023-12-07T10:19:30.715723+0000","last_fullsized":"2023-12-07T10:19:30.715723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:10:33.416955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.723232+0000","last_change":"2023-12-07T10:17:41.620017+0000","last_active":"2023-12-07T10:19:30.723232+0000","last_peered":"2023-12-07T10:19:30.723232+0000","last_clean":"2023-12-07T10:19:30.723232+0000","last_became_active":"2023-12-07T10:17:41.619647+0000","last_became_peered":"2023-12-07T10:17:41.619647+0000","last_unstale":"2023-12-07T10:19:30.723232+0000","last_undegraded":"2023-12-07T10:19:30.723232+0000","last_fullsized":"2023-12-07T10:19:30.723232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:36:42.891160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.732929+0000","last_change":"2023-12-07T10:17:41.610803+0000","last_active":"2023-12-07T10:19:30.732929+0000","last_peered":"2023-12-07T10:19:30.732929+0000","last_clean":"2023-12-07T10:19:30.732929+0000","last_became_active":"2023-12-07T10:17:41.610388+0000","last_became_peered":"2023-12-07T10:17:41.610388+0000","last_unstale":"2023-12-07T10:19:30.732929+0000","last_undegraded":"2023-12-07T10:19:30.732929+0000","last_fullsized":"2023-12-07T10:19:30.732929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:33:25.787281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.737405+0000","last_change":"2023-12-07T10:17:41.615916+0000","last_active":"2023-12-07T10:19:30.737405+0000","last_peered":"2023-12-07T10:19:30.737405+0000","last_clean":"2023-12-07T10:19:30.737405+0000","last_became_active":"2023-12-07T10:17:41.615591+0000","last_became_peered":"2023-12-07T10:17:41.615591+0000","last_unstale":"2023-12-07T10:19:30.737405+0000","last_undegraded":"2023-12-07T10:19:30.737405+0000","last_fullsized":"2023-12-07T10:19:30.737405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:40:45.541571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.721231+0000","last_change":"2023-12-07T10:17:41.614735+0000","last_active":"2023-12-07T10:19:30.721231+0000","last_peered":"2023-12-07T10:19:30.721231+0000","last_clean":"2023-12-07T10:19:30.721231+0000","last_became_active":"2023-12-07T10:17:41.614388+0000","last_became_peered":"2023-12-07T10:17:41.614388+0000","last_unstale":"2023-12-07T10:19:30.721231+0000","last_undegraded":"2023-12-07T10:19:30.721231+0000","last_fullsized":"2023-12-07T10:19:30.721231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:10:07.196841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.721750+0000","last_change":"2023-12-07T10:17:41.620059+0000","last_active":"2023-12-07T10:19:30.721750+0000","last_peered":"2023-12-07T10:19:30.721750+0000","last_clean":"2023-12-07T10:19:30.721750+0000","last_became_active":"2023-12-07T10:17:41.619770+0000","last_became_peered":"2023-12-07T10:17:41.619770+0000","last_unstale":"2023-12-07T10:19:30.721750+0000","last_undegraded":"2023-12-07T10:19:30.721750+0000","last_fullsized":"2023-12-07T10:19:30.721750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:13:43.421776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.723274+0000","last_change":"2023-12-07T10:17:41.609720+0000","last_active":"2023-12-07T10:19:30.723274+0000","last_peered":"2023-12-07T10:19:30.723274+0000","last_clean":"2023-12-07T10:19:30.723274+0000","last_became_active":"2023-12-07T10:17:41.609417+0000","last_became_peered":"2023-12-07T10:17:41.609417+0000","last_unstale":"2023-12-07T10:19:30.723274+0000","last_undegraded":"2023-12-07T10:19:30.723274+0000","last_fullsized":"2023-12-07T10:19:30.723274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:48:47.768754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.732262+0000","last_change":"2023-12-07T10:17:41.620952+0000","last_active":"2023-12-07T10:19:30.732262+0000","last_peered":"2023-12-07T10:19:30.732262+0000","last_clean":"2023-12-07T10:19:30.732262+0000","last_became_active":"2023-12-07T10:17:41.620560+0000","last_became_peered":"2023-12-07T10:17:41.620560+0000","last_unstale":"2023-12-07T10:19:30.732262+0000","last_undegraded":"2023-12-07T10:19:30.732262+0000","last_fullsized":"2023-12-07T10:19:30.732262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:51:37.699511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.733197+0000","last_change":"2023-12-07T10:17:41.611908+0000","last_active":"2023-12-07T10:19:30.733197+0000","last_peered":"2023-12-07T10:19:30.733197+0000","last_clean":"2023-12-07T10:19:30.733197+0000","last_became_active":"2023-12-07T10:17:41.610433+0000","last_became_peered":"2023-12-07T10:17:41.610433+0000","last_unstale":"2023-12-07T10:19:30.733197+0000","last_undegraded":"2023-12-07T10:19:30.733197+0000","last_fullsized":"2023-12-07T10:19:30.733197+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:42:53.906221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.739825+0000","last_change":"2023-12-07T10:17:41.604116+0000","last_active":"2023-12-07T10:19:30.739825+0000","last_peered":"2023-12-07T10:19:30.739825+0000","last_clean":"2023-12-07T10:19:30.739825+0000","last_became_active":"2023-12-07T10:17:41.603880+0000","last_became_peered":"2023-12-07T10:17:41.603880+0000","last_unstale":"2023-12-07T10:19:30.739825+0000","last_undegraded":"2023-12-07T10:19:30.739825+0000","last_fullsized":"2023-12-07T10:19:30.739825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:56:27.343169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.739711+0000","last_change":"2023-12-07T10:17:41.610497+0000","last_active":"2023-12-07T10:19:30.739711+0000","last_peered":"2023-12-07T10:19:30.739711+0000","last_clean":"2023-12-07T10:19:30.739711+0000","last_became_active":"2023-12-07T10:17:41.609956+0000","last_became_peered":"2023-12-07T10:17:41.609956+0000","last_unstale":"2023-12-07T10:19:30.739711+0000","last_undegraded":"2023-12-07T10:19:30.739711+0000","last_fullsized":"2023-12-07T10:19:30.739711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:04:03.967377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.728017+0000","last_change":"2023-12-07T10:17:41.612284+0000","last_active":"2023-12-07T10:19:30.728017+0000","last_peered":"2023-12-07T10:19:30.728017+0000","last_clean":"2023-12-07T10:19:30.728017+0000","last_became_active":"2023-12-07T10:17:41.611917+0000","last_became_peered":"2023-12-07T10:17:41.611917+0000","last_unstale":"2023-12-07T10:19:30.728017+0000","last_undegraded":"2023-12-07T10:19:30.728017+0000","last_fullsized":"2023-12-07T10:19:30.728017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:02:27.355883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":84968,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1182,"num_read_kb":681,"num_write":259,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":606208,"data_stored":249944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383721,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":516906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.41499999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78500000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.41699999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79000000000000004,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.505}]},{"osd":1,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.63700000000000001,"15min":0.61099999999999999},"min":{"1min":0.52100000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85599999999999998,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.626},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.439,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.48399999999999999,"15min":0.45800000000000002},"min":{"1min":0.35699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.75800000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.48499999999999999,"15min":0.46200000000000002},"min":{"1min":0.33000000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.75900000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.35199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68799999999999994,"15min":0.67500000000000004},"min":{"1min":0.60099999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.80600000000000005,"5min":1.046,"15min":1.046},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63400000000000001,"15min":0.628},"min":{"1min":0.504,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86099999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.52200000000000002}]},{"osd":5,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.58599999999999997,"15min":0.56299999999999994},"min":{"1min":0.42499999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.93899999999999995,"5min":1.034,"15min":1.034},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60599999999999998,"15min":0.58499999999999996},"min":{"1min":0.441,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.95299999999999996,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.46800000000000003}]},{"osd":6,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83499999999999996,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.438},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73199999999999998,"15min":0.71899999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.90800000000000003,"5min":1.363,"15min":1.363},"last":0.55000000000000004}]},{"osd":7,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.77100000000000002,"15min":0.754},"min":{"1min":0.65400000000000003,"5min":0.501,"15min":0.501},"max":{"1min":0.90300000000000002,"5min":1.25,"15min":1.25},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.71599999999999997,"15min":0.69899999999999995},"min":{"1min":0.54800000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94999999999999996,"5min":1.085,"15min":1.085},"last":0.57199999999999995}]},{"osd":8,"last update":"Thu Dec 7 10:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.68799999999999994,"15min":0.69399999999999995},"min":{"1min":0.47999999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.878,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77400000000000002,"15min":0.77700000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.876,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.66500000000000004}]},{"osd":9,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.79900000000000004,"15min":0.77300000000000002},"min":{"1min":0.66700000000000004,"5min":0.496,"15min":0.496},"max":{"1min":0.97699999999999998,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.752},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.80200000000000005,"15min":0.78000000000000003},"min":{"1min":0.69999999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0049999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.53700000000000003}]},{"osd":10,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.76600000000000001,"15min":0.746},"min":{"1min":0.64300000000000002,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.92500000000000004,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.80700000000000005,"15min":0.78700000000000003},"min":{"1min":0.69299999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.024,"5min":1.411,"15min":1.411},"last":0.79300000000000004}]}]},{"osd":10,"up_from":72,"seq":309237645357,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28724,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716428,"statfs":{"total":95995035648,"available":95965622272,"internally_reserved":0,"allocated":1748992,"data_stored":1168822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.71099999999999997,"15min":0.71599999999999997},"min":{"1min":0.42399999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.3130000000000002,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.625,"15min":0.62},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.47999999999999998}]},{"osd":1,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.49099999999999999,"15min":0.49299999999999999},"min":{"1min":0.311,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":0.308},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.48899999999999999,"15min":0.49299999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.33600000000000002}]},{"osd":2,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63800000000000001,"15min":0.64600000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79600000000000004,"5min":1.008,"15min":1.008},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.41599999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78900000000000003,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.36699999999999999}]},{"osd":3,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.44700000000000001,"5min":0.437,"15min":0.437},"max":{"1min":2.6859999999999999,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.74399999999999999,"15min":0.70999999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":8.2590000000000003,"5min":8.2590000000000003,"15min":8.2590000000000003},"last":0.46000000000000002}]},{"osd":4,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.56200000000000006,"15min":0.57799999999999996},"min":{"1min":0.35499999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.56999999999999995,"15min":0.58599999999999997},"min":{"1min":0.36399999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82499999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.40500000000000003}]},{"osd":5,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.70099999999999996,"15min":0.71299999999999997},"min":{"1min":0.46800000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.97299999999999998,"5min":1.22,"15min":1.22},"last":0.441},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72699999999999998,"15min":0.72799999999999998},"min":{"1min":0.42499999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88200000000000001,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.49299999999999999}]},{"osd":6,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":0.81899999999999995,"15min":0.77500000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":6.5049999999999999,"5min":6.5049999999999999,"15min":6.5049999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.76800000000000002,"15min":0.78000000000000003},"min":{"1min":0.45900000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.98199999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.50800000000000001}]},{"osd":7,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.63900000000000001}]},{"osd":8,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.77800000000000002,"15min":0.79900000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0209999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.80700000000000005,"15min":0.83399999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93500000000000005,"5min":1.165,"15min":1.165},"last":0.54000000000000004}]},{"osd":9,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.82599999999999996,"15min":0.83699999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.872,"5min":0.85199999999999998,"15min":0.86599999999999999},"min":{"1min":0.56399999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":0.59999999999999998}]},{"osd":11,"last update":"Thu Dec 7 10:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.80600000000000005,"15min":0.81799999999999995},"min":{"1min":0.56000000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.92700000000000005,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.81000000000000005,"15min":0.81799999999999995},"min":{"1min":0.58599999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.98299999999999998,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.57699999999999996}]}]},{"osd":9,"up_from":67,"seq":287762808882,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":520820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.56599999999999995,"15min":0.60799999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.626,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57199999999999995,"15min":0.58499999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.42799999999999999}]},{"osd":1,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.66900000000000004,"15min":0.69899999999999995},"min":{"1min":0.45300000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.70899999999999996,"5min":1.899,"15min":1.899},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.745,"15min":0.78300000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.85799999999999998,"5min":1.409,"15min":1.409},"last":0.46400000000000002}]},{"osd":2,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":0.89700000000000002,"15min":0.85099999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":11.782999999999999,"5min":11.782999999999999,"15min":11.782999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.82999999999999996,"15min":0.88400000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.94499999999999995,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":0.50600000000000001}]},{"osd":3,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.68400000000000005,"15min":0.73099999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91400000000000003,"5min":1.47,"15min":1.47},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.753,"15min":0.83299999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.032,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.73799999999999999}]},{"osd":4,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":0.879,"15min":0.85599999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":11.808999999999999,"5min":11.808999999999999,"15min":11.808999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.82999999999999996,"15min":0.88500000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.88,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.56899999999999995}]},{"osd":5,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.87,"15min":0.93600000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.91800000000000004,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.3109999999999999,"5min":0.93100000000000005,"15min":0.90400000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":11.863,"5min":11.863,"15min":11.863},"last":0.58099999999999996}]},{"osd":6,"last update":"Thu Dec 7 10:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":0.749}]},{"osd":7,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.879,"15min":0.93999999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86099999999999999,"5min":1.5,"15min":1.5},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.94899999999999995,"15min":1.0289999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.96899999999999997,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":0.84499999999999997}]},{"osd":8,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.032,"15min":1},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":11.831,"5min":11.831,"15min":11.831},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.96999999999999997,"15min":1.048},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.006,"5min":2.6539999999999999,"15min":2.6539999999999999},"last":0.86899999999999999}]},{"osd":10,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.96099999999999997,"15min":1.032},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.97499999999999998,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.88300000000000001,"15min":0.94999999999999996},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.82899999999999996,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.628}]},{"osd":11,"last update":"Thu Dec 7 10:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.82599999999999996,"15min":0.83099999999999996},"min":{"1min":0.59199999999999997,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.95999999999999996,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.61699999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.84999999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.66400000000000003}]}]},{"osd":8,"up_from":59,"seq":253403070520,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":516990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.38700000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.75600000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56699999999999995,"15min":0.56499999999999995},"min":{"1min":0.40999999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76500000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.44900000000000001}]},{"osd":1,"last update":"Thu Dec 7 10:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61199999999999999,"15min":0.58399999999999996},"min":{"1min":0.42799999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.96399999999999997,"5min":1,"15min":1},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.62,"15min":0.58899999999999997},"min":{"1min":0.50700000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77700000000000002,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.61699999999999999}]},{"osd":2,"last update":"Thu Dec 7 10:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.47699999999999998,"15min":0.44900000000000001},"min":{"1min":0.34699999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.88400000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.495},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47299999999999998,"15min":0.441},"min":{"1min":0.29399999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.65500000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.53600000000000003}]},{"osd":3,"last update":"Thu Dec 7 10:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.37,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73199999999999998,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.61399999999999999,"15min":0.59199999999999997},"min":{"1min":0.52300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81599999999999995,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.81599999999999995}]},{"osd":4,"last update":"Thu Dec 7 10:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.61699999999999999,"15min":0.59299999999999997},"min":{"1min":0.47099999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.66500000000000004,"15min":0.64100000000000001},"min":{"1min":0.56399999999999995,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92000000000000004,"5min":1.079,"15min":1.079},"last":0.59599999999999997}]},{"osd":5,"last update":"Thu Dec 7 10:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55900000000000005,"15min":0.52300000000000002},"min":{"1min":0.434,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.68000000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.59099999999999997,"15min":0.56100000000000005},"min":{"1min":0.45100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.82899999999999996,"5min":1.024,"15min":1.024},"last":0.74199999999999999}]},{"osd":6,"last update":"Thu Dec 7 10:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.69199999999999995,"15min":0.66200000000000003},"min":{"1min":0.46899999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.877,"5min":1.046,"15min":1.046},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.70999999999999996,"15min":0.67100000000000004},"min":{"1min":0.501,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93000000000000005,"5min":1.022,"15min":1.022},"last":0.75800000000000001}]},{"osd":7,"last update":"Thu Dec 7 10:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.71299999999999997,"15min":0.66300000000000003},"min":{"1min":0.64600000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94499999999999995,"5min":1.099,"15min":1.099},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.72099999999999997,"15min":0.68500000000000005},"min":{"1min":0.624,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.69899999999999995}]},{"osd":9,"last update":"Thu Dec 7 10:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.73899999999999999,"15min":0.72099999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.76900000000000002,"15min":0.74299999999999999},"min":{"1min":0.62,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.84499999999999997}]},{"osd":10,"last update":"Thu Dec 7 10:19:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.76500000000000001,"15min":0.73399999999999999},"min":{"1min":0.57599999999999996,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0009999999999999,"5min":1.069,"15min":1.069},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.873,"5min":0.77300000000000002,"15min":0.73799999999999999},"min":{"1min":0.64300000000000002,"5min":0.503,"15min":0.503},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.77100000000000002}]},{"osd":11,"last update":"Thu Dec 7 10:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.79700000000000004,"15min":0.79200000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.74199999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0149999999999999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.86699999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299452,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1679360,"data_stored":1124241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.58199999999999996,"15min":0.55200000000000005},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.5369999999999999,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56599999999999995,"15min":0.56000000000000005},"min":{"1min":0.39700000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.36899999999999999}]},{"osd":1,"last update":"Thu Dec 7 10:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.28899999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.64200000000000002,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45700000000000002,"15min":0.45500000000000002},"min":{"1min":0.33500000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.68899999999999995,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.32800000000000001}]},{"osd":2,"last update":"Thu Dec 7 10:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.57999999999999996},"min":{"1min":0.315,"5min":0.314,"15min":0.314},"max":{"1min":0.68000000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58599999999999997,"15min":0.57399999999999995},"min":{"1min":0.49299999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71099999999999997,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.432}]},{"osd":3,"last update":"Thu Dec 7 10:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.374,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.69399999999999995,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63700000000000001,"15min":0.62},"min":{"1min":0.42799999999999999,"5min":0.38,"15min":0.38},"max":{"1min":2.4380000000000002,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":0.57499999999999996}]},{"osd":4,"last update":"Thu Dec 7 10:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51900000000000002,"15min":0.51600000000000001},"min":{"1min":0.38400000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71599999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.50600000000000001,"15min":0.52700000000000002},"min":{"1min":0.35799999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.58799999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.34799999999999998}]},{"osd":5,"last update":"Thu Dec 7 10:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63900000000000001,"15min":0.64400000000000002},"min":{"1min":0.47499999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.81299999999999994,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.44600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81299999999999994,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.47899999999999998}]},{"osd":6,"last update":"Thu Dec 7 10:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.68000000000000005,"15min":0.68500000000000005},"min":{"1min":0.50800000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85299999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68999999999999995,"15min":0.69299999999999995},"min":{"1min":0.47599999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.347,"5min":2.347,"15min":2.347},"last":0.628}]},{"osd":8,"last update":"Thu Dec 7 10:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70599999999999996,"15min":0.69799999999999995},"min":{"1min":0.55900000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.83399999999999996,"5min":1.012,"15min":1.012},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72599999999999998,"15min":0.72899999999999998},"min":{"1min":0.64800000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92200000000000004,"5min":1.026,"15min":1.026},"last":0.49199999999999999}]},{"osd":9,"last update":"Thu Dec 7 10:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73599999999999999,"15min":0.74199999999999999},"min":{"1min":0.55700000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94399999999999995,"5min":1.075,"15min":1.075},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.75800000000000001,"15min":0.755},"min":{"1min":0.56399999999999995,"5min":0.505,"15min":0.505},"max":{"1min":2.1989999999999998,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":0.63600000000000001}]},{"osd":10,"last update":"Thu Dec 7 10:19:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.752,"15min":0.76200000000000001},"min":{"1min":0.54500000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.83399999999999996,"5min":1.03,"15min":1.03},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73699999999999999,"15min":0.74099999999999999},"min":{"1min":0.58299999999999996,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.82499999999999996,"5min":1.002,"15min":1.002},"last":0.40899999999999997}]},{"osd":11,"last update":"Thu Dec 7 10:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.78900000000000003,"15min":0.78500000000000003},"min":{"1min":0.71799999999999997,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":0.94999999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.77000000000000002,"15min":0.76500000000000001},"min":{"1min":0.70199999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.90900000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.51800000000000002}]}]},{"osd":6,"up_from":47,"seq":201863462977,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":512228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.5,"15min":0.48199999999999998},"min":{"1min":0.374,"5min":0.316,"15min":0.316},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.53000000000000003,"15min":0.48799999999999999},"min":{"1min":0.35399999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.58299999999999996}]},{"osd":1,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.69299999999999995,"15min":0.67100000000000004},"min":{"1min":0.56399999999999995,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":3.077,"5min":3.077,"15min":3.077},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.70599999999999996,"15min":0.67300000000000004},"min":{"1min":0.55300000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.113,"5min":2.113,"15min":2.113},"last":0.76800000000000002}]},{"osd":2,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.71999999999999997,"15min":0.68200000000000005},"min":{"1min":0.51400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.42299999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":3.153,"5min":3.153,"15min":3.153},"last":0.56299999999999994}]},{"osd":3,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":3.1200000000000001,"5min":3.1200000000000001,"15min":3.1200000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.67600000000000005,"15min":0.65100000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.42499999999999999}]},{"osd":4,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.77900000000000003,"15min":0.76300000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":3.2629999999999999,"5min":3.2629999999999999,"15min":3.2629999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.76000000000000001,"15min":0.748},"min":{"1min":0.56799999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.68899999999999995}]},{"osd":5,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.79600000000000004,"15min":0.80200000000000005},"min":{"1min":0.628,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.81200000000000006,"15min":0.79600000000000004},"min":{"1min":0.55200000000000005,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.5369999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.79400000000000004}]},{"osd":7,"last update":"Thu Dec 7 10:19:25 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":0.876,"15min":0.81699999999999995},"min":{"1min":0.496,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":6.9809999999999999,"5min":6.9809999999999999,"15min":6.9809999999999999},"last":0.745},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.81599999999999995,"15min":0.79200000000000004},"min":{"1min":0.65500000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.73199999999999998}]},{"osd":8,"last update":"Thu Dec 7 10:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.84599999999999997,"15min":0.86399999999999999},"min":{"1min":0.68700000000000006,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.90400000000000003,"5min":2.569,"15min":2.569},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.85299999999999998,"15min":0.88300000000000001},"min":{"1min":0.56000000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.93600000000000005,"5min":3.1850000000000001,"15min":3.1850000000000001},"last":0.66100000000000003}]},{"osd":9,"last update":"Thu Dec 7 10:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.872,"15min":0.874},"min":{"1min":0.66000000000000003,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.88100000000000001,"15min":0.86099999999999999},"min":{"1min":0.63300000000000001,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.6600000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":0.52500000000000002}]},{"osd":10,"last update":"Thu Dec 7 10:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.88200000000000001,"15min":0.85699999999999998},"min":{"1min":0.55100000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":3.2280000000000002,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.89400000000000002,"15min":0.88600000000000001},"min":{"1min":0.76500000000000001,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.82899999999999996}]},{"osd":11,"last update":"Thu Dec 7 10:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.96399999999999997,"15min":0.98399999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":0.98399999999999999,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.86499999999999999,"5min":1.0329999999999999,"15min":1.0609999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.069,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.71899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":578272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64300000000000002,"15min":0.57099999999999995},"min":{"1min":0.38200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.93100000000000005,"5min":6.8120000000000003,"15min":6.8120000000000003},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58699999999999997,"15min":0.55100000000000005},"min":{"1min":0.44600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.81999999999999995,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.53400000000000003}]},{"osd":2,"last update":"Thu Dec 7 10:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.64900000000000002,"15min":0.56100000000000005},"min":{"1min":0.64900000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.95099999999999996,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.60499999999999998,"15min":0.52500000000000002},"min":{"1min":0.58999999999999997,"5min":0.35799999999999998,"15min":0.35699999999999998},"max":{"1min":0.89500000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.84499999999999997}]},{"osd":3,"last update":"Thu Dec 7 10:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.63800000000000001,"15min":0.57399999999999995},"min":{"1min":0.54000000000000004,"5min":0.378,"15min":0.378},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.621},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57099999999999995,"15min":0.53300000000000003},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58699999999999997}]},{"osd":4,"last update":"Thu Dec 7 10:19:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.46600000000000003,"15min":0.40600000000000003},"min":{"1min":0.36199999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.51700000000000002,"15min":0.434},"min":{"1min":0.40400000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.88500000000000001,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.55800000000000005}]},{"osd":5,"last update":"Thu Dec 7 10:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.71499999999999997,"15min":0.60699999999999998},"min":{"1min":0.627,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94699999999999995,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.65700000000000003,"15min":0.57899999999999996},"min":{"1min":0.625,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.90700000000000003}]},{"osd":6,"last update":"Thu Dec 7 10:19:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.71799999999999997,"15min":0.67600000000000005},"min":{"1min":0.54000000000000004,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.101,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65700000000000003,"15min":0.64300000000000002},"min":{"1min":0.49199999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.83099999999999996,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.72299999999999998}]},{"osd":7,"last update":"Thu Dec 7 10:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56399999999999995,"15min":0.53100000000000003},"min":{"1min":0.433,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79500000000000004,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.70799999999999996,"15min":0.64600000000000002},"min":{"1min":0.59399999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.99099999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.98899999999999999}]},{"osd":8,"last update":"Thu Dec 7 10:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.81200000000000006,"15min":0.748},"min":{"1min":0.77200000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":1.016},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.70099999999999996,"15min":0.65500000000000003},"min":{"1min":0.60899999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.86899999999999999}]},{"osd":9,"last update":"Thu Dec 7 10:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.76900000000000002,"15min":0.75600000000000001},"min":{"1min":0.55900000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.96699999999999997,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.86299999999999999,"15min":0.85199999999999998},"min":{"1min":0.63300000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0660000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":1.0649999999999999}]},{"osd":10,"last update":"Thu Dec 7 10:19:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.748,"15min":0.73999999999999999},"min":{"1min":0.51100000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0389999999999999,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.77900000000000003,"15min":0.75800000000000001},"min":{"1min":0.54600000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0760000000000001,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.96499999999999997}]},{"osd":11,"last update":"Thu Dec 7 10:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.94799999999999995,"15min":0.93999999999999995},"min":{"1min":0.85599999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":1.04},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.86299999999999999,"15min":0.84899999999999998},"min":{"1min":0.78600000000000003,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.0029999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":513684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Dec 7 10:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.628,"15min":0.58199999999999996},"min":{"1min":0.40899999999999997,"5min":0.375,"15min":0.375},"max":{"1min":2.3929999999999998,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.628},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.61099999999999999,"15min":0.57099999999999995},"min":{"1min":0.39500000000000002,"5min":0.34399999999999997,"15min":0.34300000000000003},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.73599999999999999}]},{"osd":2,"last update":"Thu Dec 7 10:19:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64700000000000002,"15min":0.59999999999999998},"min":{"1min":0.437,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.97699999999999998,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66600000000000004,"15min":0.61199999999999999},"min":{"1min":0.45600000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.147,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.83999999999999997}]},{"osd":3,"last update":"Thu Dec 7 10:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.79200000000000004,"15min":0.82999999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.80500000000000005,"5min":3.484,"15min":3.484},"last":0.752},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.74199999999999999,"15min":0.77200000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81799999999999995,"5min":3.1379999999999999,"15min":3.1379999999999999},"last":0.67500000000000004}]},{"osd":4,"last update":"Thu Dec 7 10:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.64900000000000002,"15min":0.58299999999999996},"min":{"1min":0.47299999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":3.0049999999999999,"5min":3.0049999999999999,"15min":3.0049999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.64700000000000002,"15min":0.61299999999999999},"min":{"1min":0.43099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":0.78400000000000003}]},{"osd":5,"last update":"Thu Dec 7 10:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.63900000000000001,"15min":0.624},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91000000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.626,"15min":0.624},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85399999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.628}]},{"osd":6,"last update":"Thu Dec 7 10:19:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63600000000000001,"15min":0.628},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":2.5,"5min":2.5,"15min":2.5},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.67200000000000004,"15min":0.64400000000000002},"min":{"1min":0.38100000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.452,"5min":3.452,"15min":3.452},"last":0.71799999999999997}]},{"osd":7,"last update":"Thu Dec 7 10:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.753,"15min":0.75900000000000001},"min":{"1min":0.56299999999999994,"5min":0.496,"15min":0.496},"max":{"1min":0.90300000000000002,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.83099999999999996,"15min":0.85399999999999998},"min":{"1min":0.57199999999999995,"5min":0.435,"15min":0.435},"max":{"1min":0.95099999999999996,"5min":3.0430000000000001,"15min":3.0430000000000001},"last":0.76400000000000001}]},{"osd":8,"last update":"Thu Dec 7 10:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.79400000000000004,"15min":0.79800000000000004},"min":{"1min":0.53500000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.89400000000000002,"5min":3.75,"15min":3.75},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79500000000000004,"15min":0.80700000000000005},"min":{"1min":0.59199999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0249999999999999,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.84899999999999998}]},{"osd":9,"last update":"Thu Dec 7 10:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.84299999999999997,"15min":0.85999999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.4180000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.49199999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":3.7050000000000001,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.77300000000000002}]},{"osd":10,"last update":"Thu Dec 7 10:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80500000000000005,"15min":0.78100000000000003},"min":{"1min":0.56699999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1259999999999999,"5min":3.0979999999999999,"15min":3.0979999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85499999999999998,"15min":0.84799999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2669999999999999,"5min":2.383,"15min":2.383},"last":0.878}]},{"osd":11,"last update":"Thu Dec 7 10:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.83999999999999997,"15min":0.81599999999999995},"min":{"1min":0.64600000000000002,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.89500000000000002,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.82199999999999995,"15min":0.80200000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.079,"5min":3.1749999999999998,"15min":3.1749999999999998},"last":0.82899999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":588591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56599999999999995,"15min":0.51600000000000001},"min":{"1min":0.36799999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57199999999999995,"15min":0.52700000000000002},"min":{"1min":0.39200000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.86499999999999999,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.52000000000000002}]},{"osd":1,"last update":"Thu Dec 7 10:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.62,"15min":0.54100000000000004},"min":{"1min":0.41699999999999998,"5min":0.38900000000000001,"15min":0.34999999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.748},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.59499999999999997,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.70799999999999996}]},{"osd":3,"last update":"Thu Dec 7 10:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.628,"15min":0.56599999999999995},"min":{"1min":0.42599999999999999,"5min":0.41399999999999998,"15min":0.35799999999999998},"max":{"1min":0.92800000000000005,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.625,"15min":0.57099999999999995},"min":{"1min":0.44400000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86199999999999999,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.76500000000000001}]},{"osd":4,"last update":"Thu Dec 7 10:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71999999999999997,"15min":0.71099999999999997},"min":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.78400000000000003,"15min":0.78000000000000003},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.64600000000000002}]},{"osd":5,"last update":"Thu Dec 7 10:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.51200000000000001,"15min":0.45400000000000001},"min":{"1min":0.438,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.38},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.52600000000000002,"15min":0.47999999999999998},"min":{"1min":0.41999999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.80400000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.46899999999999997}]},{"osd":6,"last update":"Thu Dec 7 10:19:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68000000000000005,"15min":0.67000000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.69199999999999995,"15min":0.66000000000000003},"min":{"1min":0.57699999999999996,"5min":0.5,"15min":0.5},"max":{"1min":0.96699999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.57799999999999996}]},{"osd":7,"last update":"Thu Dec 7 10:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.73799999999999999,"15min":0.73399999999999999},"min":{"1min":0.53400000000000003,"5min":0.499,"15min":0.499},"max":{"1min":0.93100000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.51200000000000001,"5min":0.501,"15min":0.501},"max":{"1min":0.88800000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.73299999999999998}]},{"osd":8,"last update":"Thu Dec 7 10:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.65700000000000003,"15min":0.61499999999999999},"min":{"1min":0.52000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.70099999999999996,"15min":0.67100000000000004},"min":{"1min":0.56699999999999995,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.018,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.53600000000000003}]},{"osd":9,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79000000000000004,"15min":0.78400000000000003},"min":{"1min":0.64000000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.038,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.873,"5min":0.83899999999999997,"15min":0.83399999999999996},"min":{"1min":0.65500000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0680000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.77700000000000002}]},{"osd":10,"last update":"Thu Dec 7 10:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.79500000000000004,"15min":0.76200000000000001},"min":{"1min":0.69399999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0389999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.80200000000000005,"15min":0.77400000000000002},"min":{"1min":0.68500000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0489999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.82399999999999995}]},{"osd":11,"last update":"Thu Dec 7 10:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.873,"15min":0.873},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0129999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.873,"5min":0.876,"15min":0.876},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.087,"5min":1.129,"15min":1.129},"last":0.85699999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149774,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":511605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46100000000000002,"15min":0.40699999999999997},"min":{"1min":0.32200000000000001,"5min":0.28199999999999997,"15min":0.22500000000000001},"max":{"1min":0.79500000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.47699999999999998,"15min":0.41599999999999998},"min":{"1min":0.29599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.64500000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.378}]},{"osd":1,"last update":"Thu Dec 7 10:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71999999999999997,"15min":0.59799999999999998},"min":{"1min":0.55700000000000005,"5min":0.47299999999999998,"15min":0.438},"max":{"1min":0.84899999999999998,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.504},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66600000000000004,"15min":0.57099999999999995},"min":{"1min":0.49399999999999999,"5min":0.41899999999999998,"15min":0.39900000000000002},"max":{"1min":0.81399999999999995,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.61799999999999999}]},{"osd":2,"last update":"Thu Dec 7 10:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.442,"5min":0.39500000000000002,"15min":0.38300000000000001},"max":{"1min":0.76300000000000001,"5min":1.087,"15min":1.087},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.71599999999999997,"15min":0.63},"min":{"1min":0.57799999999999996,"5min":0.44,"15min":0.34699999999999998},"max":{"1min":0.89800000000000002,"5min":1.177,"15min":1.177},"last":0.752}]},{"osd":4,"last update":"Thu Dec 7 10:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71199999999999997,"15min":0.629},"min":{"1min":0.50800000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.032,"5min":1.208,"15min":1.208},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65000000000000002,"15min":0.59599999999999997},"min":{"1min":0.46500000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.874,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.58199999999999996}]},{"osd":5,"last update":"Thu Dec 7 10:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.63400000000000001,"15min":0.61399999999999999},"min":{"1min":0.54000000000000004,"5min":0.372,"15min":0.372},"max":{"1min":0.877,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.72599999999999998,"15min":0.69999999999999996},"min":{"1min":0.49299999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0760000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.72799999999999998}]},{"osd":6,"last update":"Thu Dec 7 10:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65300000000000002,"15min":0.64200000000000002},"min":{"1min":0.40999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":1.179,"5min":1.282,"15min":1.282},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.60299999999999998,"15min":0.61599999999999999},"min":{"1min":0.40699999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.86599999999999999,"5min":1.292,"15min":1.292},"last":0.46999999999999997}]},{"osd":7,"last update":"Thu Dec 7 10:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.79500000000000004,"15min":0.77200000000000002},"min":{"1min":0.61199999999999999,"5min":0.498,"15min":0.498},"max":{"1min":1.1140000000000001,"5min":1.323,"15min":1.323},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95299999999999996,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.55600000000000005}]},{"osd":8,"last update":"Thu Dec 7 10:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.872}]},{"osd":9,"last update":"Thu Dec 7 10:19:15 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.86299999999999999,"15min":0.88100000000000001},"min":{"1min":0.58699999999999997,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1479999999999999,"5min":1.242,"15min":1.242},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.78500000000000003,"15min":0.80500000000000005},"min":{"1min":0.53000000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0009999999999999,"5min":1.149,"15min":1.149},"last":0.53200000000000003}]},{"osd":10,"last update":"Thu Dec 7 10:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.76600000000000001,"15min":0.78600000000000003},"min":{"1min":0.53900000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.89000000000000001,"5min":1.335,"15min":1.335},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.90600000000000003,"15min":0.92300000000000004},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.59599999999999997}]},{"osd":11,"last update":"Thu Dec 7 10:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.873,"15min":0.871},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.77500000000000002,"15min":0.77400000000000002},"min":{"1min":0.63300000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.68600000000000005}]}]},{"osd":4,"up_from":34,"seq":146028888138,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":516943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.52100000000000002,"15min":0.49099999999999999},"min":{"1min":0.41499999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76500000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Dec 7 10:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.41999999999999998,"15min":0.39300000000000002},"min":{"1min":0.28199999999999997,"5min":0.215,"15min":0.215},"max":{"1min":0.58499999999999996,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.43099999999999999,"15min":0.39500000000000002},"min":{"1min":0.32800000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.61399999999999999,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.60099999999999998}]},{"osd":2,"last update":"Thu Dec 7 10:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.61899999999999999,"15min":0.56999999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85899999999999999,"5min":1.323,"15min":1.323},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60199999999999998,"15min":0.54900000000000004},"min":{"1min":0.47599999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79500000000000004,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.66200000000000003}]},{"osd":3,"last update":"Thu Dec 7 10:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60299999999999998,"15min":0.55000000000000004},"min":{"1min":0.45800000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.873,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.57299999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83799999999999997,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.79700000000000004}]},{"osd":5,"last update":"Thu Dec 7 10:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64500000000000002,"15min":0.59699999999999998},"min":{"1min":0.47699999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.85499999999999998,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63500000000000001,"15min":0.58999999999999997},"min":{"1min":0.48799999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.876,"5min":1.258,"15min":1.258},"last":0.69099999999999995}]},{"osd":6,"last update":"Thu Dec 7 10:19:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65300000000000002,"15min":0.65100000000000002},"min":{"1min":0.434,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90000000000000002,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65200000000000002,"15min":0.63800000000000001},"min":{"1min":0.49399999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85199999999999998,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.73499999999999999}]},{"osd":7,"last update":"Thu Dec 7 10:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56899999999999995,"15min":0.52400000000000002},"min":{"1min":0.40799999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.79800000000000004,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60599999999999998,"15min":0.56200000000000006},"min":{"1min":0.40100000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.96099999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.90300000000000002}]},{"osd":8,"last update":"Thu Dec 7 10:19:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.745,"15min":0.68999999999999995},"min":{"1min":0.58499999999999996,"5min":0.432,"15min":0.432},"max":{"1min":1.026,"5min":1.381,"15min":1.381},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.73599999999999999,"15min":0.68999999999999995},"min":{"1min":0.53700000000000003,"5min":0.502,"15min":0.502},"max":{"1min":1.0389999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.78300000000000003}]},{"osd":9,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.76100000000000001,"15min":0.76700000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.93500000000000005,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78000000000000003,"15min":0.76100000000000001},"min":{"1min":0.59399999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0289999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.996}]},{"osd":10,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.55100000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.94899999999999995,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.752,"15min":0.73999999999999999},"min":{"1min":0.53800000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97499999999999998,"5min":1.038,"15min":1.038},"last":0.83999999999999997}]},{"osd":11,"last update":"Thu Dec 7 10:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.876,"15min":0.876},"min":{"1min":0.68200000000000005,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.079,"5min":1.222,"15min":1.222},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82399999999999995,"15min":0.82299999999999995},"min":{"1min":0.67100000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0129999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.95299999999999996}]}]},{"osd":5,"up_from":39,"seq":167503724614,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1691648,"data_stored":1119625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.52100000000000002},"min":{"1min":0.44,"5min":0.318,"15min":0.318},"max":{"1min":0.76300000000000001,"5min":2.105,"15min":2.105},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55000000000000004,"15min":0.52300000000000002},"min":{"1min":0.35599999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.78100000000000003,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.42499999999999999}]},{"osd":1,"last update":"Thu Dec 7 10:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.58599999999999997,"15min":0.52400000000000002},"min":{"1min":0.38600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77500000000000002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.625},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.621,"15min":0.55100000000000005},"min":{"1min":0.33100000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.81299999999999994,"5min":3.0070000000000001,"15min":3.0070000000000001},"last":0.56000000000000005}]},{"osd":2,"last update":"Thu Dec 7 10:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.438,"15min":0.39000000000000001},"min":{"1min":0.30299999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.57799999999999996,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47299999999999998,"15min":0.40500000000000003},"min":{"1min":0.36499999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.68200000000000005,"5min":1.851,"15min":1.851},"last":0.495}]},{"osd":3,"last update":"Thu Dec 7 10:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59099999999999997,"15min":0.54300000000000004},"min":{"1min":0.35199999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82399999999999995,"5min":1.093,"15min":1.093},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60099999999999998,"15min":0.55900000000000005},"min":{"1min":0.437,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80300000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Dec 7 10:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65900000000000003,"15min":0.58199999999999996},"min":{"1min":0.51900000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.874,"5min":1.722,"15min":1.722},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64000000000000001,"15min":0.58599999999999997},"min":{"1min":0.47099999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.92000000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60899999999999999}]},{"osd":6,"last update":"Thu Dec 7 10:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.66000000000000003,"15min":0.59999999999999998},"min":{"1min":0.52200000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90200000000000002,"5min":1.272,"15min":1.272},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75,"15min":0.65000000000000002},"min":{"1min":0.623,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90100000000000002,"5min":5.9720000000000004,"15min":5.9720000000000004},"last":0.72699999999999998}]},{"osd":7,"last update":"Thu Dec 7 10:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67800000000000005,"15min":0.63200000000000001},"min":{"1min":0.47099999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88100000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67900000000000005,"15min":0.64300000000000002},"min":{"1min":0.499,"5min":0.437,"15min":0.437},"max":{"1min":0.83299999999999996,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.86799999999999999}]},{"osd":8,"last update":"Thu Dec 7 10:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.67600000000000005,"15min":0.63600000000000001},"min":{"1min":0.54300000000000004,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.96799999999999997,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.64100000000000001,"15min":0.60699999999999998},"min":{"1min":0.56100000000000005,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85299999999999998,"5min":1.135,"15min":1.135},"last":0.70999999999999996}]},{"osd":9,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.75700000000000001},"min":{"1min":0.56200000000000006,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.98499999999999999,"5min":4.532,"15min":4.532},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77300000000000002,"15min":0.76600000000000001},"min":{"1min":0.55000000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.01,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.75}]},{"osd":10,"last update":"Thu Dec 7 10:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.84399999999999997,"15min":0.83999999999999997},"min":{"1min":0.58199999999999996,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.026,"5min":2.927,"15min":2.927},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.86199999999999999,"15min":0.81100000000000005},"min":{"1min":0.59099999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.94899999999999995,"5min":5.7709999999999999,"15min":5.7709999999999999},"last":0.80900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":77085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":15823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":14506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":67279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-07T10:20:10.736 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-07T10:20:10.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.965+0000 7f4e6d7ff700 1 -- 172.21.15.119:0/1656278949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e68074070 msgr2=0x7f4e680744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:10.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.965+0000 7f4e6d7ff700 1 --2- 172.21.15.119:0/1656278949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e68074070 0x7f4e680744f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009990 tx=0x7f4e50009ca0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.965+0000 7f4e6d7ff700 1 -- 172.21.15.119:0/1656278949 shutdown_connections 2023-12-07T10:20:10.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.965+0000 7f4e6d7ff700 1 --2- 172.21.15.119:0/1656278949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e68109420 0x7f4e6810b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.965+0000 7f4e6d7ff700 1 --2- 172.21.15.119:0/1656278949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e68075650 0x7f4e68073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.965+0000 7f4e6d7ff700 1 --2- 172.21.15.119:0/1656278949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e68074070 0x7f4e680744f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.965+0000 7f4e6d7ff700 1 -- 172.21.15.119:0/1656278949 >> 172.21.15.119:0/1656278949 conn(0x7f4e680fcab0 msgr2=0x7f4e680fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:10.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.965+0000 7f4e6d7ff700 1 -- 172.21.15.119:0/1656278949 shutdown_connections 2023-12-07T10:20:10.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.965+0000 7f4e6d7ff700 1 -- 172.21.15.119:0/1656278949 wait complete. 2023-12-07T10:20:10.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.966+0000 7f4e6d7ff700 1 Processor -- start 2023-12-07T10:20:10.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.966+0000 7f4e6d7ff700 1 -- start start 2023-12-07T10:20:10.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.966+0000 7f4e6d7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e68074070 0x7f4e68199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:10.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.966+0000 7f4e6d7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e68075650 0x7f4e68199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:10.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.966+0000 7f4e6d7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e68109420 0x7f4e681a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:10.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.966+0000 7f4e6d7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e681a1330 con 0x7f4e68109420 2023-12-07T10:20:10.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.966+0000 7f4e6d7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e681a1470 con 0x7f4e68075650 2023-12-07T10:20:10.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.966+0000 7f4e6d7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e681a1600 con 0x7f4e68074070 2023-12-07T10:20:10.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.966+0000 7f4e66ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e68074070 0x7f4e68199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:10.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.966+0000 7f4e66ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e68074070 0x7f4e68199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:58268/0 (socket says 172.21.15.119:58268) 2023-12-07T10:20:10.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.966+0000 7f4e66ffd700 1 -- 172.21.15.119:0/699052094 learned_addr learned my addr 172.21.15.119:0/699052094 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:10.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.966+0000 7f4e5ffff700 1 --2- 172.21.15.119:0/699052094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e68075650 0x7f4e68199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:10.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.967+0000 7f4e677fe700 1 --2- 172.21.15.119:0/699052094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e68109420 0x7f4e681a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:10.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.967+0000 7f4e66ffd700 1 -- 172.21.15.119:0/699052094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e68075650 msgr2=0x7f4e68199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:10.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.967+0000 7f4e66ffd700 1 --2- 172.21.15.119:0/699052094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e68075650 0x7f4e68199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.967+0000 7f4e66ffd700 1 -- 172.21.15.119:0/699052094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e68109420 msgr2=0x7f4e681a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:10.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.967+0000 7f4e66ffd700 1 --2- 172.21.15.119:0/699052094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e68109420 0x7f4e681a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:10.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.967+0000 7f4e66ffd700 1 -- 172.21.15.119:0/699052094 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e500095f0 con 0x7f4e68074070 2023-12-07T10:20:10.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.967+0000 7f4e5ffff700 1 --2- 172.21.15.119:0/699052094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e68075650 0x7f4e68199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:10.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.967+0000 7f4e66ffd700 1 --2- 172.21.15.119:0/699052094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e68074070 0x7f4e68199850 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800d980 tx=0x7f4e5800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:10.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.967+0000 7f4e64ff9700 1 -- 172.21.15.119:0/699052094 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e58004220 con 0x7f4e68074070 2023-12-07T10:20:10.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.968+0000 7f4e6d7ff700 1 -- 172.21.15.119:0/699052094 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e681a18e0 con 0x7f4e68074070 2023-12-07T10:20:10.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.968+0000 7f4e64ff9700 1 -- 172.21.15.119:0/699052094 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e58007640 con 0x7f4e68074070 2023-12-07T10:20:10.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.968+0000 7f4e64ff9700 1 -- 172.21.15.119:0/699052094 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e58005690 con 0x7f4e68074070 2023-12-07T10:20:10.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.968+0000 7f4e6d7ff700 1 -- 172.21.15.119:0/699052094 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e681a1e40 con 0x7f4e68074070 2023-12-07T10:20:10.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.969+0000 7f4e6d7ff700 1 -- 172.21.15.119:0/699052094 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e681081e0 con 0x7f4e68074070 2023-12-07T10:20:10.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.971+0000 7f4e64ff9700 1 -- 172.21.15.119:0/699052094 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f4e58004750 con 0x7f4e68074070 2023-12-07T10:20:10.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.972+0000 7f4e64ff9700 1 --2- 172.21.15.119:0/699052094 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4e540a1b90 0x7f4e540a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:10.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.972+0000 7f4e64ff9700 1 -- 172.21.15.119:0/699052094 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f4e580cdc30 con 0x7f4e68074070 2023-12-07T10:20:10.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.973+0000 7f4e5ffff700 1 --2- 172.21.15.119:0/699052094 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4e540a1b90 0x7f4e540a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:10.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.973+0000 7f4e5ffff700 1 --2- 172.21.15.119:0/699052094 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4e540a1b90 0x7f4e540a4050 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4e50003730 tx=0x7f4e50005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:10.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:10.975+0000 7f4e64ff9700 1 -- 172.21.15.119:0/699052094 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4e58096c80 con 0x7f4e68074070 2023-12-07T10:20:11.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.175+0000 7f4e6d7ff700 1 -- 172.21.15.119:0/699052094 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4e68061a50 con 0x7f4e540a1b90 2023-12-07T10:20:11.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.188+0000 7f4e64ff9700 1 -- 172.21.15.119:0/699052094 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427005 (secure 0 0 0) 0x7f4e68061a50 con 0x7f4e540a1b90 2023-12-07T10:20:11.195 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:20:11.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.193+0000 7f4e6d7ff700 1 -- 172.21.15.119:0/699052094 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4e540a1b90 msgr2=0x7f4e540a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:11.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.193+0000 7f4e6d7ff700 1 --2- 172.21.15.119:0/699052094 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4e540a1b90 0x7f4e540a4050 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4e50003730 tx=0x7f4e50005fb0 comp rx=0 tx=0).stop 2023-12-07T10:20:11.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.193+0000 7f4e6d7ff700 1 -- 172.21.15.119:0/699052094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e68074070 msgr2=0x7f4e68199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:11.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.193+0000 7f4e6d7ff700 1 --2- 172.21.15.119:0/699052094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e68074070 0x7f4e68199850 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800d980 tx=0x7f4e5800dc90 comp rx=0 tx=0).stop 2023-12-07T10:20:11.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.194+0000 7f4e6d7ff700 1 -- 172.21.15.119:0/699052094 shutdown_connections 2023-12-07T10:20:11.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.194+0000 7f4e6d7ff700 1 --2- 172.21.15.119:0/699052094 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f4e540a1b90 0x7f4e540a4050 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:11.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.194+0000 7f4e6d7ff700 1 --2- 172.21.15.119:0/699052094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e68109420 0x7f4e681a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:11.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.194+0000 7f4e6d7ff700 1 --2- 172.21.15.119:0/699052094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e68074070 0x7f4e68199850 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:11.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.194+0000 7f4e6d7ff700 1 --2- 172.21.15.119:0/699052094 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e68075650 0x7f4e68199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:11.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.194+0000 7f4e6d7ff700 1 -- 172.21.15.119:0/699052094 >> 172.21.15.119:0/699052094 conn(0x7f4e680fcab0 msgr2=0x7f4e68105320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:11.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.194+0000 7f4e6d7ff700 1 -- 172.21.15.119:0/699052094 shutdown_connections 2023-12-07T10:20:11.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.194+0000 7f4e6d7ff700 1 -- 172.21.15.119:0/699052094 wait complete. 2023-12-07T10:20:11.199 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-12-07T10:20:11.215 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2023-12-07T10:20:10.709360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":675582,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1353,"num_read_kb":820,"num_write":444,"num_write_kb":3107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338740,"kb_used_data":14548,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603084,"statfs":{"total":1151940427776,"available":1151593558016,"internally_reserved":0,"allocated":14897152,"data_stored":8193687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":331855161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2792,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":137,"num_read_kb":72,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003485"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.734136+0000","last_change":"2023-12-07T10:17:41.614206+0000","last_active":"2023-12-07T10:19:30.734136+0000","last_peered":"2023-12-07T10:19:30.734136+0000","last_clean":"2023-12-07T10:19:30.734136+0000","last_became_active":"2023-12-07T10:17:41.613553+0000","last_became_peered":"2023-12-07T10:17:41.613553+0000","last_unstale":"2023-12-07T10:19:30.734136+0000","last_undegraded":"2023-12-07T10:19:30.734136+0000","last_fullsized":"2023-12-07T10:19:30.734136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:10:22.198414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.729545+0000","last_change":"2023-12-07T10:17:41.616917+0000","last_active":"2023-12-07T10:19:30.729545+0000","last_peered":"2023-12-07T10:19:30.729545+0000","last_clean":"2023-12-07T10:19:30.729545+0000","last_became_active":"2023-12-07T10:17:41.615345+0000","last_became_peered":"2023-12-07T10:17:41.615345+0000","last_unstale":"2023-12-07T10:19:30.729545+0000","last_undegraded":"2023-12-07T10:19:30.729545+0000","last_fullsized":"2023-12-07T10:19:30.729545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:22:25.566738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.733461+0000","last_change":"2023-12-07T10:17:41.619767+0000","last_active":"2023-12-07T10:19:30.733461+0000","last_peered":"2023-12-07T10:19:30.733461+0000","last_clean":"2023-12-07T10:19:30.733461+0000","last_became_active":"2023-12-07T10:17:41.619362+0000","last_became_peered":"2023-12-07T10:17:41.619362+0000","last_unstale":"2023-12-07T10:19:30.733461+0000","last_undegraded":"2023-12-07T10:19:30.733461+0000","last_fullsized":"2023-12-07T10:19:30.733461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:12:55.458520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.730527+0000","last_change":"2023-12-07T10:18:01.861930+0000","last_active":"2023-12-07T10:19:30.730527+0000","last_peered":"2023-12-07T10:19:30.730527+0000","last_clean":"2023-12-07T10:19:30.730527+0000","last_became_active":"2023-12-07T10:18:01.861570+0000","last_became_peered":"2023-12-07T10:18:01.861570+0000","last_unstale":"2023-12-07T10:19:30.730527+0000","last_undegraded":"2023-12-07T10:19:30.730527+0000","last_fullsized":"2023-12-07T10:19:30.730527+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:29:45.234261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.736864+0000","last_change":"2023-12-07T10:17:41.613890+0000","last_active":"2023-12-07T10:19:30.736864+0000","last_peered":"2023-12-07T10:19:30.736864+0000","last_clean":"2023-12-07T10:19:30.736864+0000","last_became_active":"2023-12-07T10:17:41.613073+0000","last_became_peered":"2023-12-07T10:17:41.613073+0000","last_unstale":"2023-12-07T10:19:30.736864+0000","last_undegraded":"2023-12-07T10:19:30.736864+0000","last_fullsized":"2023-12-07T10:19:30.736864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:17:14.339002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.721832+0000","last_change":"2023-12-07T10:17:41.609585+0000","last_active":"2023-12-07T10:19:30.721832+0000","last_peered":"2023-12-07T10:19:30.721832+0000","last_clean":"2023-12-07T10:19:30.721832+0000","last_became_active":"2023-12-07T10:17:41.609256+0000","last_became_peered":"2023-12-07T10:17:41.609256+0000","last_unstale":"2023-12-07T10:19:30.721832+0000","last_undegraded":"2023-12-07T10:19:30.721832+0000","last_fullsized":"2023-12-07T10:19:30.721832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:20:15.103208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.720115+0000","last_change":"2023-12-07T10:17:41.606138+0000","last_active":"2023-12-07T10:19:30.720115+0000","last_peered":"2023-12-07T10:19:30.720115+0000","last_clean":"2023-12-07T10:19:30.720115+0000","last_became_active":"2023-12-07T10:17:41.605818+0000","last_became_peered":"2023-12-07T10:17:41.605818+0000","last_unstale":"2023-12-07T10:19:30.720115+0000","last_undegraded":"2023-12-07T10:19:30.720115+0000","last_fullsized":"2023-12-07T10:19:30.720115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:38:39.244077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.725926+0000","last_change":"2023-12-07T10:18:02.255959+0000","last_active":"2023-12-07T10:19:30.725926+0000","last_peered":"2023-12-07T10:19:30.725926+0000","last_clean":"2023-12-07T10:19:30.725926+0000","last_became_active":"2023-12-07T10:18:02.255560+0000","last_became_peered":"2023-12-07T10:18:02.255560+0000","last_unstale":"2023-12-07T10:19:30.725926+0000","last_undegraded":"2023-12-07T10:19:30.725926+0000","last_fullsized":"2023-12-07T10:19:30.725926+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:05:49.065538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.733953+0000","last_change":"2023-12-07T10:17:41.610225+0000","last_active":"2023-12-07T10:19:30.733953+0000","last_peered":"2023-12-07T10:19:30.733953+0000","last_clean":"2023-12-07T10:19:30.733953+0000","last_became_active":"2023-12-07T10:17:41.609875+0000","last_became_peered":"2023-12-07T10:17:41.609875+0000","last_unstale":"2023-12-07T10:19:30.733953+0000","last_undegraded":"2023-12-07T10:19:30.733953+0000","last_fullsized":"2023-12-07T10:19:30.733953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:00:43.136176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.726466+0000","last_change":"2023-12-07T10:17:41.617335+0000","last_active":"2023-12-07T10:19:30.726466+0000","last_peered":"2023-12-07T10:19:30.726466+0000","last_clean":"2023-12-07T10:19:30.726466+0000","last_became_active":"2023-12-07T10:17:41.617005+0000","last_became_peered":"2023-12-07T10:17:41.617005+0000","last_unstale":"2023-12-07T10:19:30.726466+0000","last_undegraded":"2023-12-07T10:19:30.726466+0000","last_fullsized":"2023-12-07T10:19:30.726466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:43:36.855465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.721509+0000","last_change":"2023-12-07T10:17:41.608104+0000","last_active":"2023-12-07T10:19:30.721509+0000","last_peered":"2023-12-07T10:19:30.721509+0000","last_clean":"2023-12-07T10:19:30.721509+0000","last_became_active":"2023-12-07T10:17:41.607769+0000","last_became_peered":"2023-12-07T10:17:41.607769+0000","last_unstale":"2023-12-07T10:19:30.721509+0000","last_undegraded":"2023-12-07T10:19:30.721509+0000","last_fullsized":"2023-12-07T10:19:30.721509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:47:04.853044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.726603+0000","last_change":"2023-12-07T10:17:41.611585+0000","last_active":"2023-12-07T10:19:30.726603+0000","last_peered":"2023-12-07T10:19:30.726603+0000","last_clean":"2023-12-07T10:19:30.726603+0000","last_became_active":"2023-12-07T10:17:41.611202+0000","last_became_peered":"2023-12-07T10:17:41.611202+0000","last_unstale":"2023-12-07T10:19:30.726603+0000","last_undegraded":"2023-12-07T10:19:30.726603+0000","last_fullsized":"2023-12-07T10:19:30.726603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:17:01.649424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.739580+0000","last_change":"2023-12-07T10:17:41.611116+0000","last_active":"2023-12-07T10:19:30.739580+0000","last_peered":"2023-12-07T10:19:30.739580+0000","last_clean":"2023-12-07T10:19:30.739580+0000","last_became_active":"2023-12-07T10:17:41.610791+0000","last_became_peered":"2023-12-07T10:17:41.610791+0000","last_unstale":"2023-12-07T10:19:30.739580+0000","last_undegraded":"2023-12-07T10:19:30.739580+0000","last_fullsized":"2023-12-07T10:19:30.739580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:48:08.822235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.729464+0000","last_change":"2023-12-07T10:17:41.617447+0000","last_active":"2023-12-07T10:19:30.729464+0000","last_peered":"2023-12-07T10:19:30.729464+0000","last_clean":"2023-12-07T10:19:30.729464+0000","last_became_active":"2023-12-07T10:17:41.612638+0000","last_became_peered":"2023-12-07T10:17:41.612638+0000","last_unstale":"2023-12-07T10:19:30.729464+0000","last_undegraded":"2023-12-07T10:19:30.729464+0000","last_fullsized":"2023-12-07T10:19:30.729464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:11:30.384367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.729222+0000","last_change":"2023-12-07T10:17:41.611607+0000","last_active":"2023-12-07T10:19:30.729222+0000","last_peered":"2023-12-07T10:19:30.729222+0000","last_clean":"2023-12-07T10:19:30.729222+0000","last_became_active":"2023-12-07T10:17:41.611331+0000","last_became_peered":"2023-12-07T10:17:41.611331+0000","last_unstale":"2023-12-07T10:19:30.729222+0000","last_undegraded":"2023-12-07T10:19:30.729222+0000","last_fullsized":"2023-12-07T10:19:30.729222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:17:53.049922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.734247+0000","last_change":"2023-12-07T10:17:41.610528+0000","last_active":"2023-12-07T10:19:30.734247+0000","last_peered":"2023-12-07T10:19:30.734247+0000","last_clean":"2023-12-07T10:19:30.734247+0000","last_became_active":"2023-12-07T10:17:41.610216+0000","last_became_peered":"2023-12-07T10:17:41.610216+0000","last_unstale":"2023-12-07T10:19:30.734247+0000","last_undegraded":"2023-12-07T10:19:30.734247+0000","last_fullsized":"2023-12-07T10:19:30.734247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:11:36.749537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.734059+0000","last_change":"2023-12-07T10:17:41.613967+0000","last_active":"2023-12-07T10:19:30.734059+0000","last_peered":"2023-12-07T10:19:30.734059+0000","last_clean":"2023-12-07T10:19:30.734059+0000","last_became_active":"2023-12-07T10:17:41.613116+0000","last_became_peered":"2023-12-07T10:17:41.613116+0000","last_unstale":"2023-12-07T10:19:30.734059+0000","last_undegraded":"2023-12-07T10:19:30.734059+0000","last_fullsized":"2023-12-07T10:19:30.734059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:46:32.444025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.730539+0000","last_change":"2023-12-07T10:17:41.605478+0000","last_active":"2023-12-07T10:19:30.730539+0000","last_peered":"2023-12-07T10:19:30.730539+0000","last_clean":"2023-12-07T10:19:30.730539+0000","last_became_active":"2023-12-07T10:17:41.605110+0000","last_became_peered":"2023-12-07T10:17:41.605110+0000","last_unstale":"2023-12-07T10:19:30.730539+0000","last_undegraded":"2023-12-07T10:19:30.730539+0000","last_fullsized":"2023-12-07T10:19:30.730539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:53:56.410196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.734012+0000","last_change":"2023-12-07T10:18:02.256377+0000","last_active":"2023-12-07T10:19:30.734012+0000","last_peered":"2023-12-07T10:19:30.734012+0000","last_clean":"2023-12-07T10:19:30.734012+0000","last_became_active":"2023-12-07T10:18:02.255987+0000","last_became_peered":"2023-12-07T10:18:02.255987+0000","last_unstale":"2023-12-07T10:19:30.734012+0000","last_undegraded":"2023-12-07T10:19:30.734012+0000","last_fullsized":"2023-12-07T10:19:30.734012+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:23:34.909187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.809857+0000","last_change":"2023-12-07T10:17:45.799925+0000","last_active":"2023-12-07T10:19:01.809857+0000","last_peered":"2023-12-07T10:19:01.809857+0000","last_clean":"2023-12-07T10:19:01.809857+0000","last_became_active":"2023-12-07T10:17:45.799505+0000","last_became_peered":"2023-12-07T10:17:45.799505+0000","last_unstale":"2023-12-07T10:19:01.809857+0000","last_undegraded":"2023-12-07T10:19:01.809857+0000","last_fullsized":"2023-12-07T10:19:01.809857+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:52:51.470333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'13","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:20:07.927725+0000","last_change":"2023-12-07T10:17:40.755066+0000","last_active":"2023-12-07T10:20:07.927725+0000","last_peered":"2023-12-07T10:20:07.927725+0000","last_clean":"2023-12-07T10:20:07.927725+0000","last_became_active":"2023-12-07T10:17:40.754709+0000","last_became_peered":"2023-12-07T10:17:40.754709+0000","last_unstale":"2023-12-07T10:20:07.927725+0000","last_undegraded":"2023-12-07T10:20:07.927725+0000","last_fullsized":"2023-12-07T10:20:07.927725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:18:31.148592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.724533+0000","last_change":"2023-12-07T10:17:41.619107+0000","last_active":"2023-12-07T10:19:30.724533+0000","last_peered":"2023-12-07T10:19:30.724533+0000","last_clean":"2023-12-07T10:19:30.724533+0000","last_became_active":"2023-12-07T10:17:41.617057+0000","last_became_peered":"2023-12-07T10:17:41.617057+0000","last_unstale":"2023-12-07T10:19:30.724533+0000","last_undegraded":"2023-12-07T10:19:30.724533+0000","last_fullsized":"2023-12-07T10:19:30.724533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:53:16.402933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.804762+0000","last_change":"2023-12-07T10:17:45.797582+0000","last_active":"2023-12-07T10:19:01.804762+0000","last_peered":"2023-12-07T10:19:01.804762+0000","last_clean":"2023-12-07T10:19:01.804762+0000","last_became_active":"2023-12-07T10:17:45.796358+0000","last_became_peered":"2023-12-07T10:17:45.796358+0000","last_unstale":"2023-12-07T10:19:01.804762+0000","last_undegraded":"2023-12-07T10:19:01.804762+0000","last_fullsized":"2023-12-07T10:19:01.804762+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:53:19.551459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.214318+0000","last_change":"2023-12-07T10:17:40.762155+0000","last_active":"2023-12-07T10:19:03.214318+0000","last_peered":"2023-12-07T10:19:03.214318+0000","last_clean":"2023-12-07T10:19:03.214318+0000","last_became_active":"2023-12-07T10:17:40.761776+0000","last_became_peered":"2023-12-07T10:17:40.761776+0000","last_unstale":"2023-12-07T10:19:03.214318+0000","last_undegraded":"2023-12-07T10:19:03.214318+0000","last_fullsized":"2023-12-07T10:19:03.214318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:15:06.096331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.734387+0000","last_change":"2023-12-07T10:17:41.614999+0000","last_active":"2023-12-07T10:19:30.734387+0000","last_peered":"2023-12-07T10:19:30.734387+0000","last_clean":"2023-12-07T10:19:30.734387+0000","last_became_active":"2023-12-07T10:17:41.614673+0000","last_became_peered":"2023-12-07T10:17:41.614673+0000","last_unstale":"2023-12-07T10:19:30.734387+0000","last_undegraded":"2023-12-07T10:19:30.734387+0000","last_fullsized":"2023-12-07T10:19:30.734387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:34:45.378843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.215249+0000","last_change":"2023-12-07T10:19:03.215249+0000","last_active":"2023-12-07T10:19:03.215249+0000","last_peered":"2023-12-07T10:19:03.215249+0000","last_clean":"2023-12-07T10:19:03.215249+0000","last_became_active":"2023-12-07T10:19:03.214647+0000","last_became_peered":"2023-12-07T10:19:03.214647+0000","last_unstale":"2023-12-07T10:19:03.215249+0000","last_undegraded":"2023-12-07T10:19:03.215249+0000","last_fullsized":"2023-12-07T10:19:03.215249+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:41:01.893661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808809+0000","last_change":"2023-12-07T10:18:01.993128+0000","last_active":"2023-12-07T10:19:01.808809+0000","last_peered":"2023-12-07T10:19:01.808809+0000","last_clean":"2023-12-07T10:19:01.808809+0000","last_became_active":"2023-12-07T10:18:01.990244+0000","last_became_peered":"2023-12-07T10:18:01.990244+0000","last_unstale":"2023-12-07T10:19:01.808809+0000","last_undegraded":"2023-12-07T10:19:01.808809+0000","last_fullsized":"2023-12-07T10:19:01.808809+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:42:53.778363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.735930+0000","last_change":"2023-12-07T10:17:41.612903+0000","last_active":"2023-12-07T10:19:30.735930+0000","last_peered":"2023-12-07T10:19:30.735930+0000","last_clean":"2023-12-07T10:19:30.735930+0000","last_became_active":"2023-12-07T10:17:41.611968+0000","last_became_peered":"2023-12-07T10:17:41.611968+0000","last_unstale":"2023-12-07T10:19:30.735930+0000","last_undegraded":"2023-12-07T10:19:30.735930+0000","last_fullsized":"2023-12-07T10:19:30.735930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:34:08.442828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.804826+0000","last_change":"2023-12-07T10:17:45.803858+0000","last_active":"2023-12-07T10:19:01.804826+0000","last_peered":"2023-12-07T10:19:01.804826+0000","last_clean":"2023-12-07T10:19:01.804826+0000","last_became_active":"2023-12-07T10:17:45.803481+0000","last_became_peered":"2023-12-07T10:17:45.803481+0000","last_unstale":"2023-12-07T10:19:01.804826+0000","last_undegraded":"2023-12-07T10:19:01.804826+0000","last_fullsized":"2023-12-07T10:19:01.804826+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:08:03.728554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808719+0000","last_change":"2023-12-07T10:17:40.751866+0000","last_active":"2023-12-07T10:19:01.808719+0000","last_peered":"2023-12-07T10:19:01.808719+0000","last_clean":"2023-12-07T10:19:01.808719+0000","last_became_active":"2023-12-07T10:17:40.751492+0000","last_became_peered":"2023-12-07T10:17:40.751492+0000","last_unstale":"2023-12-07T10:19:01.808719+0000","last_undegraded":"2023-12-07T10:19:01.808719+0000","last_fullsized":"2023-12-07T10:19:01.808719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:42:53.839651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.728912+0000","last_change":"2023-12-07T10:17:41.611326+0000","last_active":"2023-12-07T10:19:30.728912+0000","last_peered":"2023-12-07T10:19:30.728912+0000","last_clean":"2023-12-07T10:19:30.728912+0000","last_became_active":"2023-12-07T10:17:41.610863+0000","last_became_peered":"2023-12-07T10:17:41.610863+0000","last_unstale":"2023-12-07T10:19:30.728912+0000","last_undegraded":"2023-12-07T10:19:30.728912+0000","last_fullsized":"2023-12-07T10:19:30.728912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:07:55.801356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810238+0000","last_change":"2023-12-07T10:17:45.791409+0000","last_active":"2023-12-07T10:19:01.810238+0000","last_peered":"2023-12-07T10:19:01.810238+0000","last_clean":"2023-12-07T10:19:01.810238+0000","last_became_active":"2023-12-07T10:17:45.791022+0000","last_became_peered":"2023-12-07T10:17:45.791022+0000","last_unstale":"2023-12-07T10:19:01.810238+0000","last_undegraded":"2023-12-07T10:19:01.810238+0000","last_fullsized":"2023-12-07T10:19:01.810238+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:57:14.617409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'12","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:20:07.916152+0000","last_change":"2023-12-07T10:17:40.745318+0000","last_active":"2023-12-07T10:20:07.916152+0000","last_peered":"2023-12-07T10:20:07.916152+0000","last_clean":"2023-12-07T10:20:07.916152+0000","last_became_active":"2023-12-07T10:17:40.745008+0000","last_became_peered":"2023-12-07T10:17:40.745008+0000","last_unstale":"2023-12-07T10:20:07.916152+0000","last_undegraded":"2023-12-07T10:20:07.916152+0000","last_fullsized":"2023-12-07T10:20:07.916152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:42:28.503421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13625,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":16,"num_write":9,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.724884+0000","last_change":"2023-12-07T10:17:41.605121+0000","last_active":"2023-12-07T10:19:30.724884+0000","last_peered":"2023-12-07T10:19:30.724884+0000","last_clean":"2023-12-07T10:19:30.724884+0000","last_became_active":"2023-12-07T10:17:41.604770+0000","last_became_peered":"2023-12-07T10:17:41.604770+0000","last_unstale":"2023-12-07T10:19:30.724884+0000","last_undegraded":"2023-12-07T10:19:30.724884+0000","last_fullsized":"2023-12-07T10:19:30.724884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:51:02.151155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810286+0000","last_change":"2023-12-07T10:17:45.809039+0000","last_active":"2023-12-07T10:19:01.810286+0000","last_peered":"2023-12-07T10:19:01.810286+0000","last_clean":"2023-12-07T10:19:01.810286+0000","last_became_active":"2023-12-07T10:17:45.808622+0000","last_became_peered":"2023-12-07T10:17:45.808622+0000","last_unstale":"2023-12-07T10:19:01.810286+0000","last_undegraded":"2023-12-07T10:19:01.810286+0000","last_fullsized":"2023-12-07T10:19:01.810286+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:16:02.835931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.815605+0000","last_change":"2023-12-07T10:17:40.757876+0000","last_active":"2023-12-07T10:19:02.815605+0000","last_peered":"2023-12-07T10:19:02.815605+0000","last_clean":"2023-12-07T10:19:02.815605+0000","last_became_active":"2023-12-07T10:17:40.757613+0000","last_became_peered":"2023-12-07T10:17:40.757613+0000","last_unstale":"2023-12-07T10:19:02.815605+0000","last_undegraded":"2023-12-07T10:19:02.815605+0000","last_fullsized":"2023-12-07T10:19:02.815605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:30:40.423092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.732539+0000","last_change":"2023-12-07T10:17:41.611575+0000","last_active":"2023-12-07T10:19:30.732539+0000","last_peered":"2023-12-07T10:19:30.732539+0000","last_clean":"2023-12-07T10:19:30.732539+0000","last_became_active":"2023-12-07T10:17:41.611042+0000","last_became_peered":"2023-12-07T10:17:41.611042+0000","last_unstale":"2023-12-07T10:19:30.732539+0000","last_undegraded":"2023-12-07T10:19:30.732539+0000","last_fullsized":"2023-12-07T10:19:30.732539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:54:05.589792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.214206+0000","last_change":"2023-12-07T10:17:45.796640+0000","last_active":"2023-12-07T10:19:03.214206+0000","last_peered":"2023-12-07T10:19:03.214206+0000","last_clean":"2023-12-07T10:19:03.214206+0000","last_became_active":"2023-12-07T10:17:45.796183+0000","last_became_peered":"2023-12-07T10:17:45.796183+0000","last_unstale":"2023-12-07T10:19:03.214206+0000","last_undegraded":"2023-12-07T10:19:03.214206+0000","last_fullsized":"2023-12-07T10:19:03.214206+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:41:56.890662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.839561+0000","last_change":"2023-12-07T10:17:40.760283+0000","last_active":"2023-12-07T10:19:02.839561+0000","last_peered":"2023-12-07T10:19:02.839561+0000","last_clean":"2023-12-07T10:19:02.839561+0000","last_became_active":"2023-12-07T10:17:40.759398+0000","last_became_peered":"2023-12-07T10:17:40.759398+0000","last_unstale":"2023-12-07T10:19:02.839561+0000","last_undegraded":"2023-12-07T10:19:02.839561+0000","last_fullsized":"2023-12-07T10:19:02.839561+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:34:37.365490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.736248+0000","last_change":"2023-12-07T10:17:41.604150+0000","last_active":"2023-12-07T10:19:30.736248+0000","last_peered":"2023-12-07T10:19:30.736248+0000","last_clean":"2023-12-07T10:19:30.736248+0000","last_became_active":"2023-12-07T10:17:41.603688+0000","last_became_peered":"2023-12-07T10:17:41.603688+0000","last_unstale":"2023-12-07T10:19:30.736248+0000","last_undegraded":"2023-12-07T10:19:30.736248+0000","last_fullsized":"2023-12-07T10:19:30.736248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:52:43.735609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.383739+0000","last_change":"2023-12-07T10:17:45.806569+0000","last_active":"2023-12-07T10:19:03.383739+0000","last_peered":"2023-12-07T10:19:03.383739+0000","last_clean":"2023-12-07T10:19:03.383739+0000","last_became_active":"2023-12-07T10:17:45.806099+0000","last_became_peered":"2023-12-07T10:17:45.806099+0000","last_unstale":"2023-12-07T10:19:03.383739+0000","last_undegraded":"2023-12-07T10:19:03.383739+0000","last_fullsized":"2023-12-07T10:19:03.383739+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:02:19.977670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.818060+0000","last_change":"2023-12-07T10:17:40.744388+0000","last_active":"2023-12-07T10:19:02.818060+0000","last_peered":"2023-12-07T10:19:02.818060+0000","last_clean":"2023-12-07T10:19:02.818060+0000","last_became_active":"2023-12-07T10:17:40.743887+0000","last_became_peered":"2023-12-07T10:17:40.743887+0000","last_unstale":"2023-12-07T10:19:02.818060+0000","last_undegraded":"2023-12-07T10:19:02.818060+0000","last_fullsized":"2023-12-07T10:19:02.818060+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:18:39.102099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.733087+0000","last_change":"2023-12-07T10:17:41.595616+0000","last_active":"2023-12-07T10:19:30.733087+0000","last_peered":"2023-12-07T10:19:30.733087+0000","last_clean":"2023-12-07T10:19:30.733087+0000","last_became_active":"2023-12-07T10:17:41.595256+0000","last_became_peered":"2023-12-07T10:17:41.595256+0000","last_unstale":"2023-12-07T10:19:30.733087+0000","last_undegraded":"2023-12-07T10:19:30.733087+0000","last_fullsized":"2023-12-07T10:19:30.733087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:08:00.156043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.213892+0000","last_change":"2023-12-07T10:17:45.794523+0000","last_active":"2023-12-07T10:19:03.213892+0000","last_peered":"2023-12-07T10:19:03.213892+0000","last_clean":"2023-12-07T10:19:03.213892+0000","last_became_active":"2023-12-07T10:17:45.794139+0000","last_became_peered":"2023-12-07T10:17:45.794139+0000","last_unstale":"2023-12-07T10:19:03.213892+0000","last_undegraded":"2023-12-07T10:19:03.213892+0000","last_fullsized":"2023-12-07T10:19:03.213892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:43:24.767627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'6","reported_seq":265,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:20:09.919230+0000","last_change":"2023-12-07T10:17:40.759620+0000","last_active":"2023-12-07T10:20:09.919230+0000","last_peered":"2023-12-07T10:20:09.919230+0000","last_clean":"2023-12-07T10:20:09.919230+0000","last_became_active":"2023-12-07T10:17:40.759372+0000","last_became_peered":"2023-12-07T10:17:40.759372+0000","last_unstale":"2023-12-07T10:20:09.919230+0000","last_undegraded":"2023-12-07T10:20:09.919230+0000","last_fullsized":"2023-12-07T10:20:09.919230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:46:14.459409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":237,"num_read_kb":237,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.731231+0000","last_change":"2023-12-07T10:17:41.601152+0000","last_active":"2023-12-07T10:19:30.731231+0000","last_peered":"2023-12-07T10:19:30.731231+0000","last_clean":"2023-12-07T10:19:30.731231+0000","last_became_active":"2023-12-07T10:17:41.600147+0000","last_became_peered":"2023-12-07T10:17:41.600147+0000","last_unstale":"2023-12-07T10:19:30.731231+0000","last_undegraded":"2023-12-07T10:19:30.731231+0000","last_fullsized":"2023-12-07T10:19:30.731231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:40:46.671489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.808262+0000","last_change":"2023-12-07T10:17:45.796229+0000","last_active":"2023-12-07T10:19:02.808262+0000","last_peered":"2023-12-07T10:19:02.808262+0000","last_clean":"2023-12-07T10:19:02.808262+0000","last_became_active":"2023-12-07T10:17:45.795582+0000","last_became_peered":"2023-12-07T10:17:45.795582+0000","last_unstale":"2023-12-07T10:19:02.808262+0000","last_undegraded":"2023-12-07T10:19:02.808262+0000","last_fullsized":"2023-12-07T10:19:02.808262+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:29:32.853975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.809294+0000","last_change":"2023-12-07T10:17:40.748353+0000","last_active":"2023-12-07T10:19:02.809294+0000","last_peered":"2023-12-07T10:19:02.809294+0000","last_clean":"2023-12-07T10:19:02.809294+0000","last_became_active":"2023-12-07T10:17:40.748050+0000","last_became_peered":"2023-12-07T10:17:40.748050+0000","last_unstale":"2023-12-07T10:19:02.809294+0000","last_undegraded":"2023-12-07T10:19:02.809294+0000","last_fullsized":"2023-12-07T10:19:02.809294+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:18:15.298534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.736790+0000","last_change":"2023-12-07T10:18:02.256728+0000","last_active":"2023-12-07T10:19:30.736790+0000","last_peered":"2023-12-07T10:19:30.736790+0000","last_clean":"2023-12-07T10:19:30.736790+0000","last_became_active":"2023-12-07T10:18:02.256271+0000","last_became_peered":"2023-12-07T10:18:02.256271+0000","last_unstale":"2023-12-07T10:19:30.736790+0000","last_undegraded":"2023-12-07T10:19:30.736790+0000","last_fullsized":"2023-12-07T10:19:30.736790+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:27:00.198934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.809834+0000","last_change":"2023-12-07T10:17:45.796903+0000","last_active":"2023-12-07T10:19:02.809834+0000","last_peered":"2023-12-07T10:19:02.809834+0000","last_clean":"2023-12-07T10:19:02.809834+0000","last_became_active":"2023-12-07T10:17:45.796454+0000","last_became_peered":"2023-12-07T10:17:45.796454+0000","last_unstale":"2023-12-07T10:19:02.809834+0000","last_undegraded":"2023-12-07T10:19:02.809834+0000","last_fullsized":"2023-12-07T10:19:02.809834+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:46:36.955396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.818962+0000","last_change":"2023-12-07T10:17:40.756045+0000","last_active":"2023-12-07T10:19:02.818962+0000","last_peered":"2023-12-07T10:19:02.818962+0000","last_clean":"2023-12-07T10:19:02.818962+0000","last_became_active":"2023-12-07T10:17:40.755763+0000","last_became_peered":"2023-12-07T10:17:40.755763+0000","last_unstale":"2023-12-07T10:19:02.818962+0000","last_undegraded":"2023-12-07T10:19:02.818962+0000","last_fullsized":"2023-12-07T10:19:02.818962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:01:08.009076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.736894+0000","last_change":"2023-12-07T10:17:41.611178+0000","last_active":"2023-12-07T10:19:30.736894+0000","last_peered":"2023-12-07T10:19:30.736894+0000","last_clean":"2023-12-07T10:19:30.736894+0000","last_became_active":"2023-12-07T10:17:41.610750+0000","last_became_peered":"2023-12-07T10:17:41.610750+0000","last_unstale":"2023-12-07T10:19:30.736894+0000","last_undegraded":"2023-12-07T10:19:30.736894+0000","last_fullsized":"2023-12-07T10:19:30.736894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:56:30.763523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.808289+0000","last_change":"2023-12-07T10:17:45.800183+0000","last_active":"2023-12-07T10:19:02.808289+0000","last_peered":"2023-12-07T10:19:02.808289+0000","last_clean":"2023-12-07T10:19:02.808289+0000","last_became_active":"2023-12-07T10:17:45.798598+0000","last_became_peered":"2023-12-07T10:17:45.798598+0000","last_unstale":"2023-12-07T10:19:02.808289+0000","last_undegraded":"2023-12-07T10:19:02.808289+0000","last_fullsized":"2023-12-07T10:19:02.808289+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:06:22.162951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'39","reported_seq":310,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:20:09.919018+0000","last_change":"2023-12-07T10:17:40.752481+0000","last_active":"2023-12-07T10:20:09.919018+0000","last_peered":"2023-12-07T10:20:09.919018+0000","last_clean":"2023-12-07T10:20:09.919018+0000","last_became_active":"2023-12-07T10:17:40.752158+0000","last_became_peered":"2023-12-07T10:17:40.752158+0000","last_unstale":"2023-12-07T10:20:09.919018+0000","last_undegraded":"2023-12-07T10:20:09.919018+0000","last_fullsized":"2023-12-07T10:20:09.919018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:43:27.963014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65014,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":249,"num_read_kb":188,"num_write":41,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.725283+0000","last_change":"2023-12-07T10:17:41.614107+0000","last_active":"2023-12-07T10:19:30.725283+0000","last_peered":"2023-12-07T10:19:30.725283+0000","last_clean":"2023-12-07T10:19:30.725283+0000","last_became_active":"2023-12-07T10:17:41.613743+0000","last_became_peered":"2023-12-07T10:17:41.613743+0000","last_unstale":"2023-12-07T10:19:30.725283+0000","last_undegraded":"2023-12-07T10:19:30.725283+0000","last_fullsized":"2023-12-07T10:19:30.725283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:49:03.200387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808863+0000","last_change":"2023-12-07T10:17:45.804282+0000","last_active":"2023-12-07T10:19:01.808863+0000","last_peered":"2023-12-07T10:19:01.808863+0000","last_clean":"2023-12-07T10:19:01.808863+0000","last_became_active":"2023-12-07T10:17:45.803794+0000","last_became_peered":"2023-12-07T10:17:45.803794+0000","last_unstale":"2023-12-07T10:19:01.808863+0000","last_undegraded":"2023-12-07T10:19:01.808863+0000","last_fullsized":"2023-12-07T10:19:01.808863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:31:26.887565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.840418+0000","last_change":"2023-12-07T10:17:40.747204+0000","last_active":"2023-12-07T10:19:02.840418+0000","last_peered":"2023-12-07T10:19:02.840418+0000","last_clean":"2023-12-07T10:19:02.840418+0000","last_became_active":"2023-12-07T10:17:40.746916+0000","last_became_peered":"2023-12-07T10:17:40.746916+0000","last_unstale":"2023-12-07T10:19:02.840418+0000","last_undegraded":"2023-12-07T10:19:02.840418+0000","last_fullsized":"2023-12-07T10:19:02.840418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:53:49.228320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.736158+0000","last_change":"2023-12-07T10:17:41.606895+0000","last_active":"2023-12-07T10:19:30.736158+0000","last_peered":"2023-12-07T10:19:30.736158+0000","last_clean":"2023-12-07T10:19:30.736158+0000","last_became_active":"2023-12-07T10:17:41.606596+0000","last_became_peered":"2023-12-07T10:17:41.606596+0000","last_unstale":"2023-12-07T10:19:30.736158+0000","last_undegraded":"2023-12-07T10:19:30.736158+0000","last_fullsized":"2023-12-07T10:19:30.736158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:34:25.166906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.214106+0000","last_change":"2023-12-07T10:17:45.807101+0000","last_active":"2023-12-07T10:19:03.214106+0000","last_peered":"2023-12-07T10:19:03.214106+0000","last_clean":"2023-12-07T10:19:03.214106+0000","last_became_active":"2023-12-07T10:17:45.806770+0000","last_became_peered":"2023-12-07T10:17:45.806770+0000","last_unstale":"2023-12-07T10:19:03.214106+0000","last_undegraded":"2023-12-07T10:19:03.214106+0000","last_fullsized":"2023-12-07T10:19:03.214106+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:39:32.522968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.818391+0000","last_change":"2023-12-07T10:17:40.754856+0000","last_active":"2023-12-07T10:19:02.818391+0000","last_peered":"2023-12-07T10:19:02.818391+0000","last_clean":"2023-12-07T10:19:02.818391+0000","last_became_active":"2023-12-07T10:17:40.754488+0000","last_became_peered":"2023-12-07T10:17:40.754488+0000","last_unstale":"2023-12-07T10:19:02.818391+0000","last_undegraded":"2023-12-07T10:19:02.818391+0000","last_fullsized":"2023-12-07T10:19:02.818391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:44:50.960216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.737528+0000","last_change":"2023-12-07T10:17:41.606963+0000","last_active":"2023-12-07T10:19:30.737528+0000","last_peered":"2023-12-07T10:19:30.737528+0000","last_clean":"2023-12-07T10:19:30.737528+0000","last_became_active":"2023-12-07T10:17:41.606408+0000","last_became_peered":"2023-12-07T10:17:41.606408+0000","last_unstale":"2023-12-07T10:19:30.737528+0000","last_undegraded":"2023-12-07T10:19:30.737528+0000","last_fullsized":"2023-12-07T10:19:30.737528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:25:47.906649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.384918+0000","last_change":"2023-12-07T10:19:03.384918+0000","last_active":"2023-12-07T10:19:03.384918+0000","last_peered":"2023-12-07T10:19:03.384918+0000","last_clean":"2023-12-07T10:19:03.384918+0000","last_became_active":"2023-12-07T10:19:03.384613+0000","last_became_peered":"2023-12-07T10:19:03.384613+0000","last_unstale":"2023-12-07T10:19:03.384918+0000","last_undegraded":"2023-12-07T10:19:03.384918+0000","last_fullsized":"2023-12-07T10:19:03.384918+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:07:29.385953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"93'7","reported_seq":145,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:20:04.924559+0000","last_change":"2023-12-07T10:17:40.747494+0000","last_active":"2023-12-07T10:20:04.924559+0000","last_peered":"2023-12-07T10:20:04.924559+0000","last_clean":"2023-12-07T10:20:04.924559+0000","last_became_active":"2023-12-07T10:17:40.747157+0000","last_became_peered":"2023-12-07T10:17:40.747157+0000","last_unstale":"2023-12-07T10:20:04.924559+0000","last_undegraded":"2023-12-07T10:20:04.924559+0000","last_fullsized":"2023-12-07T10:20:04.924559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:45:24.395418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1117,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.727117+0000","last_change":"2023-12-07T10:17:41.609495+0000","last_active":"2023-12-07T10:19:30.727117+0000","last_peered":"2023-12-07T10:19:30.727117+0000","last_clean":"2023-12-07T10:19:30.727117+0000","last_became_active":"2023-12-07T10:17:41.609181+0000","last_became_peered":"2023-12-07T10:17:41.609181+0000","last_unstale":"2023-12-07T10:19:30.727117+0000","last_undegraded":"2023-12-07T10:19:30.727117+0000","last_fullsized":"2023-12-07T10:19:30.727117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:20:33.473824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813855+0000","last_change":"2023-12-07T10:17:45.808862+0000","last_active":"2023-12-07T10:19:01.813855+0000","last_peered":"2023-12-07T10:19:01.813855+0000","last_clean":"2023-12-07T10:19:01.813855+0000","last_became_active":"2023-12-07T10:17:45.808461+0000","last_became_peered":"2023-12-07T10:17:45.808461+0000","last_unstale":"2023-12-07T10:19:01.813855+0000","last_undegraded":"2023-12-07T10:19:01.813855+0000","last_fullsized":"2023-12-07T10:19:01.813855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:09:12.598126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.810181+0000","last_change":"2023-12-07T10:18:02.263709+0000","last_active":"2023-12-07T10:19:02.810181+0000","last_peered":"2023-12-07T10:19:02.810181+0000","last_clean":"2023-12-07T10:19:02.810181+0000","last_became_active":"2023-12-07T10:18:02.255788+0000","last_became_peered":"2023-12-07T10:18:02.255788+0000","last_unstale":"2023-12-07T10:19:02.810181+0000","last_undegraded":"2023-12-07T10:19:02.810181+0000","last_fullsized":"2023-12-07T10:19:02.810181+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:12:04.976226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.727135+0000","last_change":"2023-12-07T10:17:41.610509+0000","last_active":"2023-12-07T10:19:30.727135+0000","last_peered":"2023-12-07T10:19:30.727135+0000","last_clean":"2023-12-07T10:19:30.727135+0000","last_became_active":"2023-12-07T10:17:41.610121+0000","last_became_peered":"2023-12-07T10:17:41.610121+0000","last_unstale":"2023-12-07T10:19:30.727135+0000","last_undegraded":"2023-12-07T10:19:30.727135+0000","last_fullsized":"2023-12-07T10:19:30.727135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:53:17.365880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.818151+0000","last_change":"2023-12-07T10:17:45.795818+0000","last_active":"2023-12-07T10:19:02.818151+0000","last_peered":"2023-12-07T10:19:02.818151+0000","last_clean":"2023-12-07T10:19:02.818151+0000","last_became_active":"2023-12-07T10:17:45.795477+0000","last_became_peered":"2023-12-07T10:17:45.795477+0000","last_unstale":"2023-12-07T10:19:02.818151+0000","last_undegraded":"2023-12-07T10:19:02.818151+0000","last_fullsized":"2023-12-07T10:19:02.818151+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:17:21.494654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.811207+0000","last_change":"2023-12-07T10:17:40.751791+0000","last_active":"2023-12-07T10:19:02.811207+0000","last_peered":"2023-12-07T10:19:02.811207+0000","last_clean":"2023-12-07T10:19:02.811207+0000","last_became_active":"2023-12-07T10:17:40.751502+0000","last_became_peered":"2023-12-07T10:17:40.751502+0000","last_unstale":"2023-12-07T10:19:02.811207+0000","last_undegraded":"2023-12-07T10:19:02.811207+0000","last_fullsized":"2023-12-07T10:19:02.811207+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:15:52.319265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.723353+0000","last_change":"2023-12-07T10:18:02.256247+0000","last_active":"2023-12-07T10:19:30.723353+0000","last_peered":"2023-12-07T10:19:30.723353+0000","last_clean":"2023-12-07T10:19:30.723353+0000","last_became_active":"2023-12-07T10:18:02.255732+0000","last_became_peered":"2023-12-07T10:18:02.255732+0000","last_unstale":"2023-12-07T10:19:30.723353+0000","last_undegraded":"2023-12-07T10:19:30.723353+0000","last_fullsized":"2023-12-07T10:19:30.723353+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:00:06.516804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.810760+0000","last_change":"2023-12-07T10:17:45.791966+0000","last_active":"2023-12-07T10:19:02.810760+0000","last_peered":"2023-12-07T10:19:02.810760+0000","last_clean":"2023-12-07T10:19:02.810760+0000","last_became_active":"2023-12-07T10:17:45.791613+0000","last_became_peered":"2023-12-07T10:17:45.791613+0000","last_unstale":"2023-12-07T10:19:02.810760+0000","last_undegraded":"2023-12-07T10:19:02.810760+0000","last_fullsized":"2023-12-07T10:19:02.810760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:13:02.291289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.808445+0000","last_change":"2023-12-07T10:17:40.762896+0000","last_active":"2023-12-07T10:19:02.808445+0000","last_peered":"2023-12-07T10:19:02.808445+0000","last_clean":"2023-12-07T10:19:02.808445+0000","last_became_active":"2023-12-07T10:17:40.762427+0000","last_became_peered":"2023-12-07T10:17:40.762427+0000","last_unstale":"2023-12-07T10:19:02.808445+0000","last_undegraded":"2023-12-07T10:19:02.808445+0000","last_fullsized":"2023-12-07T10:19:02.808445+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:56:35.589548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.732249+0000","last_change":"2023-12-07T10:17:41.610354+0000","last_active":"2023-12-07T10:19:30.732249+0000","last_peered":"2023-12-07T10:19:30.732249+0000","last_clean":"2023-12-07T10:19:30.732249+0000","last_became_active":"2023-12-07T10:17:41.609888+0000","last_became_peered":"2023-12-07T10:17:41.609888+0000","last_unstale":"2023-12-07T10:19:30.732249+0000","last_undegraded":"2023-12-07T10:19:30.732249+0000","last_fullsized":"2023-12-07T10:19:30.732249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:48:19.976949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.839882+0000","last_change":"2023-12-07T10:17:45.807324+0000","last_active":"2023-12-07T10:19:02.839882+0000","last_peered":"2023-12-07T10:19:02.839882+0000","last_clean":"2023-12-07T10:19:02.839882+0000","last_became_active":"2023-12-07T10:17:45.806036+0000","last_became_peered":"2023-12-07T10:17:45.806036+0000","last_unstale":"2023-12-07T10:19:02.839882+0000","last_undegraded":"2023-12-07T10:19:02.839882+0000","last_fullsized":"2023-12-07T10:19:02.839882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:54:36.847992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'5","reported_seq":145,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:20:08.929550+0000","last_change":"2023-12-07T10:17:40.754247+0000","last_active":"2023-12-07T10:20:08.929550+0000","last_peered":"2023-12-07T10:20:08.929550+0000","last_clean":"2023-12-07T10:20:08.929550+0000","last_became_active":"2023-12-07T10:17:40.753964+0000","last_became_peered":"2023-12-07T10:17:40.753964+0000","last_unstale":"2023-12-07T10:20:08.929550+0000","last_undegraded":"2023-12-07T10:20:08.929550+0000","last_fullsized":"2023-12-07T10:20:08.929550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:22:04.832399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.723617+0000","last_change":"2023-12-07T10:17:41.610516+0000","last_active":"2023-12-07T10:19:30.723617+0000","last_peered":"2023-12-07T10:19:30.723617+0000","last_clean":"2023-12-07T10:19:30.723617+0000","last_became_active":"2023-12-07T10:17:41.610184+0000","last_became_peered":"2023-12-07T10:17:41.610184+0000","last_unstale":"2023-12-07T10:19:30.723617+0000","last_undegraded":"2023-12-07T10:19:30.723617+0000","last_fullsized":"2023-12-07T10:19:30.723617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:53:43.100723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.840220+0000","last_change":"2023-12-07T10:17:45.804624+0000","last_active":"2023-12-07T10:19:02.840220+0000","last_peered":"2023-12-07T10:19:02.840220+0000","last_clean":"2023-12-07T10:19:02.840220+0000","last_became_active":"2023-12-07T10:17:45.803667+0000","last_became_peered":"2023-12-07T10:17:45.803667+0000","last_unstale":"2023-12-07T10:19:02.840220+0000","last_undegraded":"2023-12-07T10:19:02.840220+0000","last_fullsized":"2023-12-07T10:19:02.840220+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:55:09.642795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.726585+0000","last_change":"2023-12-07T10:17:41.618433+0000","last_active":"2023-12-07T10:19:30.726585+0000","last_peered":"2023-12-07T10:19:30.726585+0000","last_clean":"2023-12-07T10:19:30.726585+0000","last_became_active":"2023-12-07T10:17:41.617942+0000","last_became_peered":"2023-12-07T10:17:41.617942+0000","last_unstale":"2023-12-07T10:19:30.726585+0000","last_undegraded":"2023-12-07T10:19:30.726585+0000","last_fullsized":"2023-12-07T10:19:30.726585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:26:48.166157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":144,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:20:06.926556+0000","last_change":"2023-12-07T10:17:40.761907+0000","last_active":"2023-12-07T10:20:06.926556+0000","last_peered":"2023-12-07T10:20:06.926556+0000","last_clean":"2023-12-07T10:20:06.926556+0000","last_became_active":"2023-12-07T10:17:40.761564+0000","last_became_peered":"2023-12-07T10:17:40.761564+0000","last_unstale":"2023-12-07T10:20:06.926556+0000","last_undegraded":"2023-12-07T10:20:06.926556+0000","last_fullsized":"2023-12-07T10:20:06.926556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:57:22.304352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.809474+0000","last_change":"2023-12-07T10:17:45.792684+0000","last_active":"2023-12-07T10:19:02.809474+0000","last_peered":"2023-12-07T10:19:02.809474+0000","last_clean":"2023-12-07T10:19:02.809474+0000","last_became_active":"2023-12-07T10:17:45.792172+0000","last_became_peered":"2023-12-07T10:17:45.792172+0000","last_unstale":"2023-12-07T10:19:02.809474+0000","last_undegraded":"2023-12-07T10:19:02.809474+0000","last_fullsized":"2023-12-07T10:19:02.809474+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:24:31.084727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.737013+0000","last_change":"2023-12-07T10:17:41.612582+0000","last_active":"2023-12-07T10:19:30.737013+0000","last_peered":"2023-12-07T10:19:30.737013+0000","last_clean":"2023-12-07T10:19:30.737013+0000","last_became_active":"2023-12-07T10:17:41.612244+0000","last_became_peered":"2023-12-07T10:17:41.612244+0000","last_unstale":"2023-12-07T10:19:30.737013+0000","last_undegraded":"2023-12-07T10:19:30.737013+0000","last_fullsized":"2023-12-07T10:19:30.737013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:41:53.506037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.809740+0000","last_change":"2023-12-07T10:17:40.748565+0000","last_active":"2023-12-07T10:19:02.809740+0000","last_peered":"2023-12-07T10:19:02.809740+0000","last_clean":"2023-12-07T10:19:02.809740+0000","last_became_active":"2023-12-07T10:17:40.748250+0000","last_became_peered":"2023-12-07T10:17:40.748250+0000","last_unstale":"2023-12-07T10:19:02.809740+0000","last_undegraded":"2023-12-07T10:19:02.809740+0000","last_fullsized":"2023-12-07T10:19:02.809740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:26:28.202440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":329,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.808386+0000","last_change":"2023-12-07T10:16:33.482173+0000","last_active":"2023-12-07T10:19:02.808386+0000","last_peered":"2023-12-07T10:19:02.808386+0000","last_clean":"2023-12-07T10:19:02.808386+0000","last_became_active":"2023-12-07T10:16:33.481699+0000","last_became_peered":"2023-12-07T10:16:33.481699+0000","last_unstale":"2023-12-07T10:19:02.808386+0000","last_undegraded":"2023-12-07T10:19:02.808386+0000","last_fullsized":"2023-12-07T10:19:02.808386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:13:25.388258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:13:25.388258+0000","last_clean_scrub_stamp":"2023-12-07T10:13:25.388258+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:57:09.583677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.818400+0000","last_change":"2023-12-07T10:17:45.806289+0000","last_active":"2023-12-07T10:19:02.818400+0000","last_peered":"2023-12-07T10:19:02.818400+0000","last_clean":"2023-12-07T10:19:02.818400+0000","last_became_active":"2023-12-07T10:17:45.805954+0000","last_became_peered":"2023-12-07T10:17:45.805954+0000","last_unstale":"2023-12-07T10:19:02.818400+0000","last_undegraded":"2023-12-07T10:19:02.818400+0000","last_fullsized":"2023-12-07T10:19:02.818400+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:21:09.955380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.727877+0000","last_change":"2023-12-07T10:17:41.612322+0000","last_active":"2023-12-07T10:19:30.727877+0000","last_peered":"2023-12-07T10:19:30.727877+0000","last_clean":"2023-12-07T10:19:30.727877+0000","last_became_active":"2023-12-07T10:17:41.612039+0000","last_became_peered":"2023-12-07T10:17:41.612039+0000","last_unstale":"2023-12-07T10:19:30.727877+0000","last_undegraded":"2023-12-07T10:19:30.727877+0000","last_fullsized":"2023-12-07T10:19:30.727877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:40:49.883742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.808419+0000","last_change":"2023-12-07T10:17:40.749694+0000","last_active":"2023-12-07T10:19:02.808419+0000","last_peered":"2023-12-07T10:19:02.808419+0000","last_clean":"2023-12-07T10:19:02.808419+0000","last_became_active":"2023-12-07T10:17:40.749393+0000","last_became_peered":"2023-12-07T10:17:40.749393+0000","last_unstale":"2023-12-07T10:19:02.808419+0000","last_undegraded":"2023-12-07T10:19:02.808419+0000","last_fullsized":"2023-12-07T10:19:02.808419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:15:36.949062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.214038+0000","last_change":"2023-12-07T10:17:45.805769+0000","last_active":"2023-12-07T10:19:03.214038+0000","last_peered":"2023-12-07T10:19:03.214038+0000","last_clean":"2023-12-07T10:19:03.214038+0000","last_became_active":"2023-12-07T10:17:45.805346+0000","last_became_peered":"2023-12-07T10:17:45.805346+0000","last_unstale":"2023-12-07T10:19:03.214038+0000","last_undegraded":"2023-12-07T10:19:03.214038+0000","last_fullsized":"2023-12-07T10:19:03.214038+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:47:47.781933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.214136+0000","last_change":"2023-12-07T10:18:01.862558+0000","last_active":"2023-12-07T10:19:03.214136+0000","last_peered":"2023-12-07T10:19:03.214136+0000","last_clean":"2023-12-07T10:19:03.214136+0000","last_became_active":"2023-12-07T10:18:01.862063+0000","last_became_peered":"2023-12-07T10:18:01.862063+0000","last_unstale":"2023-12-07T10:19:03.214136+0000","last_undegraded":"2023-12-07T10:19:03.214136+0000","last_fullsized":"2023-12-07T10:19:03.214136+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:25:32.652160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.733695+0000","last_change":"2023-12-07T10:17:41.617199+0000","last_active":"2023-12-07T10:19:30.733695+0000","last_peered":"2023-12-07T10:19:30.733695+0000","last_clean":"2023-12-07T10:19:30.733695+0000","last_became_active":"2023-12-07T10:17:41.616901+0000","last_became_peered":"2023-12-07T10:17:41.616901+0000","last_unstale":"2023-12-07T10:19:30.733695+0000","last_undegraded":"2023-12-07T10:19:30.733695+0000","last_fullsized":"2023-12-07T10:19:30.733695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:23:59.433861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.213749+0000","last_change":"2023-12-07T10:17:45.790914+0000","last_active":"2023-12-07T10:19:03.213749+0000","last_peered":"2023-12-07T10:19:03.213749+0000","last_clean":"2023-12-07T10:19:03.213749+0000","last_became_active":"2023-12-07T10:17:45.790428+0000","last_became_peered":"2023-12-07T10:17:45.790428+0000","last_unstale":"2023-12-07T10:19:03.213749+0000","last_undegraded":"2023-12-07T10:19:03.213749+0000","last_fullsized":"2023-12-07T10:19:03.213749+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:03:57.516300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'11","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:33.545539+0000","last_change":"2023-12-07T10:18:01.863765+0000","last_active":"2023-12-07T10:19:33.545539+0000","last_peered":"2023-12-07T10:19:33.545539+0000","last_clean":"2023-12-07T10:19:33.545539+0000","last_became_active":"2023-12-07T10:18:01.858926+0000","last_became_peered":"2023-12-07T10:18:01.858926+0000","last_unstale":"2023-12-07T10:19:33.545539+0000","last_undegraded":"2023-12-07T10:19:33.545539+0000","last_fullsized":"2023-12-07T10:19:33.545539+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:44:19.663592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.733950+0000","last_change":"2023-12-07T10:17:41.608513+0000","last_active":"2023-12-07T10:19:30.733950+0000","last_peered":"2023-12-07T10:19:30.733950+0000","last_clean":"2023-12-07T10:19:30.733950+0000","last_became_active":"2023-12-07T10:17:41.605049+0000","last_became_peered":"2023-12-07T10:17:41.605049+0000","last_unstale":"2023-12-07T10:19:30.733950+0000","last_undegraded":"2023-12-07T10:19:30.733950+0000","last_fullsized":"2023-12-07T10:19:30.733950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:33:00.730827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.840029+0000","last_change":"2023-12-07T10:17:45.792583+0000","last_active":"2023-12-07T10:19:02.840029+0000","last_peered":"2023-12-07T10:19:02.840029+0000","last_clean":"2023-12-07T10:19:02.840029+0000","last_became_active":"2023-12-07T10:17:45.792162+0000","last_became_peered":"2023-12-07T10:17:45.792162+0000","last_unstale":"2023-12-07T10:19:02.840029+0000","last_undegraded":"2023-12-07T10:19:02.840029+0000","last_fullsized":"2023-12-07T10:19:02.840029+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:47:30.901477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.808789+0000","last_change":"2023-12-07T10:17:40.750826+0000","last_active":"2023-12-07T10:19:02.808789+0000","last_peered":"2023-12-07T10:19:02.808789+0000","last_clean":"2023-12-07T10:19:02.808789+0000","last_became_active":"2023-12-07T10:17:40.750519+0000","last_became_peered":"2023-12-07T10:17:40.750519+0000","last_unstale":"2023-12-07T10:19:02.808789+0000","last_undegraded":"2023-12-07T10:19:02.808789+0000","last_fullsized":"2023-12-07T10:19:02.808789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:21:01.008897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.735624+0000","last_change":"2023-12-07T10:17:41.609488+0000","last_active":"2023-12-07T10:19:30.735624+0000","last_peered":"2023-12-07T10:19:30.735624+0000","last_clean":"2023-12-07T10:19:30.735624+0000","last_became_active":"2023-12-07T10:17:41.609061+0000","last_became_peered":"2023-12-07T10:17:41.609061+0000","last_unstale":"2023-12-07T10:19:30.735624+0000","last_undegraded":"2023-12-07T10:19:30.735624+0000","last_fullsized":"2023-12-07T10:19:30.735624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:39:32.729502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.808632+0000","last_change":"2023-12-07T10:17:45.797573+0000","last_active":"2023-12-07T10:19:02.808632+0000","last_peered":"2023-12-07T10:19:02.808632+0000","last_clean":"2023-12-07T10:19:02.808632+0000","last_became_active":"2023-12-07T10:17:45.797119+0000","last_became_peered":"2023-12-07T10:17:45.797119+0000","last_unstale":"2023-12-07T10:19:02.808632+0000","last_undegraded":"2023-12-07T10:19:02.808632+0000","last_fullsized":"2023-12-07T10:19:02.808632+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:17:27.100470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.810618+0000","last_change":"2023-12-07T10:17:40.753596+0000","last_active":"2023-12-07T10:19:02.810618+0000","last_peered":"2023-12-07T10:19:02.810618+0000","last_clean":"2023-12-07T10:19:02.810618+0000","last_became_active":"2023-12-07T10:17:40.753251+0000","last_became_peered":"2023-12-07T10:17:40.753251+0000","last_unstale":"2023-12-07T10:19:02.810618+0000","last_undegraded":"2023-12-07T10:19:02.810618+0000","last_fullsized":"2023-12-07T10:19:02.810618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:08:30.868794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.725751+0000","last_change":"2023-12-07T10:17:41.599689+0000","last_active":"2023-12-07T10:19:30.725751+0000","last_peered":"2023-12-07T10:19:30.725751+0000","last_clean":"2023-12-07T10:19:30.725751+0000","last_became_active":"2023-12-07T10:17:41.599358+0000","last_became_peered":"2023-12-07T10:17:41.599358+0000","last_unstale":"2023-12-07T10:19:30.725751+0000","last_undegraded":"2023-12-07T10:19:30.725751+0000","last_fullsized":"2023-12-07T10:19:30.725751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:44:19.742560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.215169+0000","last_change":"2023-12-07T10:19:03.215169+0000","last_active":"2023-12-07T10:19:03.215169+0000","last_peered":"2023-12-07T10:19:03.215169+0000","last_clean":"2023-12-07T10:19:03.215169+0000","last_became_active":"2023-12-07T10:19:03.214754+0000","last_became_peered":"2023-12-07T10:19:03.214754+0000","last_unstale":"2023-12-07T10:19:03.215169+0000","last_undegraded":"2023-12-07T10:19:03.215169+0000","last_fullsized":"2023-12-07T10:19:03.215169+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:51:41.461131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.213651+0000","last_change":"2023-12-07T10:17:40.758663+0000","last_active":"2023-12-07T10:19:03.213651+0000","last_peered":"2023-12-07T10:19:03.213651+0000","last_clean":"2023-12-07T10:19:03.213651+0000","last_became_active":"2023-12-07T10:17:40.753611+0000","last_became_peered":"2023-12-07T10:17:40.753611+0000","last_unstale":"2023-12-07T10:19:03.213651+0000","last_undegraded":"2023-12-07T10:19:03.213651+0000","last_fullsized":"2023-12-07T10:19:03.213651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:00:59.256468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.730928+0000","last_change":"2023-12-07T10:17:41.616546+0000","last_active":"2023-12-07T10:19:30.730928+0000","last_peered":"2023-12-07T10:19:30.730928+0000","last_clean":"2023-12-07T10:19:30.730928+0000","last_became_active":"2023-12-07T10:17:41.615780+0000","last_became_peered":"2023-12-07T10:17:41.615780+0000","last_unstale":"2023-12-07T10:19:30.730928+0000","last_undegraded":"2023-12-07T10:19:30.730928+0000","last_fullsized":"2023-12-07T10:19:30.730928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:22:32.455784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.809460+0000","last_change":"2023-12-07T10:17:45.797550+0000","last_active":"2023-12-07T10:19:02.809460+0000","last_peered":"2023-12-07T10:19:02.809460+0000","last_clean":"2023-12-07T10:19:02.809460+0000","last_became_active":"2023-12-07T10:17:45.797140+0000","last_became_peered":"2023-12-07T10:17:45.797140+0000","last_unstale":"2023-12-07T10:19:02.809460+0000","last_undegraded":"2023-12-07T10:19:02.809460+0000","last_fullsized":"2023-12-07T10:19:02.809460+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:53:44.984904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"93'6","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:54.075606+0000","last_change":"2023-12-07T10:17:40.748410+0000","last_active":"2023-12-07T10:19:54.075606+0000","last_peered":"2023-12-07T10:19:54.075606+0000","last_clean":"2023-12-07T10:19:54.075606+0000","last_became_active":"2023-12-07T10:17:40.748069+0000","last_became_peered":"2023-12-07T10:17:40.748069+0000","last_unstale":"2023-12-07T10:19:54.075606+0000","last_undegraded":"2023-12-07T10:19:54.075606+0000","last_fullsized":"2023-12-07T10:19:54.075606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:51:48.278974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.734791+0000","last_change":"2023-12-07T10:17:41.609248+0000","last_active":"2023-12-07T10:19:30.734791+0000","last_peered":"2023-12-07T10:19:30.734791+0000","last_clean":"2023-12-07T10:19:30.734791+0000","last_became_active":"2023-12-07T10:17:41.608943+0000","last_became_peered":"2023-12-07T10:17:41.608943+0000","last_unstale":"2023-12-07T10:19:30.734791+0000","last_undegraded":"2023-12-07T10:19:30.734791+0000","last_fullsized":"2023-12-07T10:19:30.734791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:29:32.322273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.808615+0000","last_change":"2023-12-07T10:17:45.794855+0000","last_active":"2023-12-07T10:19:02.808615+0000","last_peered":"2023-12-07T10:19:02.808615+0000","last_clean":"2023-12-07T10:19:02.808615+0000","last_became_active":"2023-12-07T10:17:45.794410+0000","last_became_peered":"2023-12-07T10:17:45.794410+0000","last_unstale":"2023-12-07T10:19:02.808615+0000","last_undegraded":"2023-12-07T10:19:02.808615+0000","last_fullsized":"2023-12-07T10:19:02.808615+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:57:25.118036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.810614+0000","last_change":"2023-12-07T10:17:40.750471+0000","last_active":"2023-12-07T10:19:02.810614+0000","last_peered":"2023-12-07T10:19:02.810614+0000","last_clean":"2023-12-07T10:19:02.810614+0000","last_became_active":"2023-12-07T10:17:40.750176+0000","last_became_peered":"2023-12-07T10:17:40.750176+0000","last_unstale":"2023-12-07T10:19:02.810614+0000","last_undegraded":"2023-12-07T10:19:02.810614+0000","last_fullsized":"2023-12-07T10:19:02.810614+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:04:22.305343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.734958+0000","last_change":"2023-12-07T10:17:41.614087+0000","last_active":"2023-12-07T10:19:30.734958+0000","last_peered":"2023-12-07T10:19:30.734958+0000","last_clean":"2023-12-07T10:19:30.734958+0000","last_became_active":"2023-12-07T10:17:41.607689+0000","last_became_peered":"2023-12-07T10:17:41.607689+0000","last_unstale":"2023-12-07T10:19:30.734958+0000","last_undegraded":"2023-12-07T10:19:30.734958+0000","last_fullsized":"2023-12-07T10:19:30.734958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:50:10.316085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.841410+0000","last_change":"2023-12-07T10:19:02.841001+0000","last_active":"2023-12-07T10:19:02.841410+0000","last_peered":"2023-12-07T10:19:02.841410+0000","last_clean":"2023-12-07T10:19:02.841410+0000","last_became_active":"2023-12-07T10:19:02.840595+0000","last_became_peered":"2023-12-07T10:19:02.840595+0000","last_unstale":"2023-12-07T10:19:02.841410+0000","last_undegraded":"2023-12-07T10:19:02.841410+0000","last_fullsized":"2023-12-07T10:19:02.841410+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:18:35.590140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'5","reported_seq":145,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:20:07.928786+0000","last_change":"2023-12-07T10:17:40.753645+0000","last_active":"2023-12-07T10:20:07.928786+0000","last_peered":"2023-12-07T10:20:07.928786+0000","last_clean":"2023-12-07T10:20:07.928786+0000","last_became_active":"2023-12-07T10:17:40.753326+0000","last_became_peered":"2023-12-07T10:17:40.753326+0000","last_unstale":"2023-12-07T10:20:07.928786+0000","last_undegraded":"2023-12-07T10:20:07.928786+0000","last_fullsized":"2023-12-07T10:20:07.928786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:43:03.617591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.719211+0000","last_change":"2023-12-07T10:17:41.607968+0000","last_active":"2023-12-07T10:19:30.719211+0000","last_peered":"2023-12-07T10:19:30.719211+0000","last_clean":"2023-12-07T10:19:30.719211+0000","last_became_active":"2023-12-07T10:17:41.607621+0000","last_became_peered":"2023-12-07T10:17:41.607621+0000","last_unstale":"2023-12-07T10:19:30.719211+0000","last_undegraded":"2023-12-07T10:19:30.719211+0000","last_fullsized":"2023-12-07T10:19:30.719211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:55:10.224738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.383569+0000","last_change":"2023-12-07T10:17:45.793678+0000","last_active":"2023-12-07T10:19:03.383569+0000","last_peered":"2023-12-07T10:19:03.383569+0000","last_clean":"2023-12-07T10:19:03.383569+0000","last_became_active":"2023-12-07T10:17:45.793283+0000","last_became_peered":"2023-12-07T10:17:45.793283+0000","last_unstale":"2023-12-07T10:19:03.383569+0000","last_undegraded":"2023-12-07T10:19:03.383569+0000","last_fullsized":"2023-12-07T10:19:03.383569+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:33:56.501107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.818650+0000","last_change":"2023-12-07T10:17:40.762069+0000","last_active":"2023-12-07T10:19:02.818650+0000","last_peered":"2023-12-07T10:19:02.818650+0000","last_clean":"2023-12-07T10:19:02.818650+0000","last_became_active":"2023-12-07T10:17:40.761752+0000","last_became_peered":"2023-12-07T10:17:40.761752+0000","last_unstale":"2023-12-07T10:19:02.818650+0000","last_undegraded":"2023-12-07T10:19:02.818650+0000","last_fullsized":"2023-12-07T10:19:02.818650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:57:55.736144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.731611+0000","last_change":"2023-12-07T10:17:41.607347+0000","last_active":"2023-12-07T10:19:30.731611+0000","last_peered":"2023-12-07T10:19:30.731611+0000","last_clean":"2023-12-07T10:19:30.731611+0000","last_became_active":"2023-12-07T10:17:41.606618+0000","last_became_peered":"2023-12-07T10:17:41.606618+0000","last_unstale":"2023-12-07T10:19:30.731611+0000","last_undegraded":"2023-12-07T10:19:30.731611+0000","last_fullsized":"2023-12-07T10:19:30.731611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:53:34.748130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.817908+0000","last_change":"2023-12-07T10:17:45.791459+0000","last_active":"2023-12-07T10:19:02.817908+0000","last_peered":"2023-12-07T10:19:02.817908+0000","last_clean":"2023-12-07T10:19:02.817908+0000","last_became_active":"2023-12-07T10:17:45.791070+0000","last_became_peered":"2023-12-07T10:17:45.791070+0000","last_unstale":"2023-12-07T10:19:02.817908+0000","last_undegraded":"2023-12-07T10:19:02.817908+0000","last_fullsized":"2023-12-07T10:19:02.817908+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:46:35.985990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.840125+0000","last_change":"2023-12-07T10:17:40.762726+0000","last_active":"2023-12-07T10:19:02.840125+0000","last_peered":"2023-12-07T10:19:02.840125+0000","last_clean":"2023-12-07T10:19:02.840125+0000","last_became_active":"2023-12-07T10:17:40.762378+0000","last_became_peered":"2023-12-07T10:17:40.762378+0000","last_unstale":"2023-12-07T10:19:02.840125+0000","last_undegraded":"2023-12-07T10:19:02.840125+0000","last_fullsized":"2023-12-07T10:19:02.840125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:42:06.193834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.718804+0000","last_change":"2023-12-07T10:17:41.602356+0000","last_active":"2023-12-07T10:19:30.718804+0000","last_peered":"2023-12-07T10:19:30.718804+0000","last_clean":"2023-12-07T10:19:30.718804+0000","last_became_active":"2023-12-07T10:17:41.602025+0000","last_became_peered":"2023-12-07T10:17:41.602025+0000","last_unstale":"2023-12-07T10:19:30.718804+0000","last_undegraded":"2023-12-07T10:19:30.718804+0000","last_fullsized":"2023-12-07T10:19:30.718804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:33:04.994510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.728025+0000","last_change":"2023-12-07T10:17:41.616766+0000","last_active":"2023-12-07T10:19:30.728025+0000","last_peered":"2023-12-07T10:19:30.728025+0000","last_clean":"2023-12-07T10:19:30.728025+0000","last_became_active":"2023-12-07T10:17:41.616221+0000","last_became_peered":"2023-12-07T10:17:41.616221+0000","last_unstale":"2023-12-07T10:19:30.728025+0000","last_undegraded":"2023-12-07T10:19:30.728025+0000","last_fullsized":"2023-12-07T10:19:30.728025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:16:00.393772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.715723+0000","last_change":"2023-12-07T10:17:41.613646+0000","last_active":"2023-12-07T10:19:30.715723+0000","last_peered":"2023-12-07T10:19:30.715723+0000","last_clean":"2023-12-07T10:19:30.715723+0000","last_became_active":"2023-12-07T10:17:41.613257+0000","last_became_peered":"2023-12-07T10:17:41.613257+0000","last_unstale":"2023-12-07T10:19:30.715723+0000","last_undegraded":"2023-12-07T10:19:30.715723+0000","last_fullsized":"2023-12-07T10:19:30.715723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:10:33.416955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.723232+0000","last_change":"2023-12-07T10:17:41.620017+0000","last_active":"2023-12-07T10:19:30.723232+0000","last_peered":"2023-12-07T10:19:30.723232+0000","last_clean":"2023-12-07T10:19:30.723232+0000","last_became_active":"2023-12-07T10:17:41.619647+0000","last_became_peered":"2023-12-07T10:17:41.619647+0000","last_unstale":"2023-12-07T10:19:30.723232+0000","last_undegraded":"2023-12-07T10:19:30.723232+0000","last_fullsized":"2023-12-07T10:19:30.723232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:36:42.891160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.732929+0000","last_change":"2023-12-07T10:17:41.610803+0000","last_active":"2023-12-07T10:19:30.732929+0000","last_peered":"2023-12-07T10:19:30.732929+0000","last_clean":"2023-12-07T10:19:30.732929+0000","last_became_active":"2023-12-07T10:17:41.610388+0000","last_became_peered":"2023-12-07T10:17:41.610388+0000","last_unstale":"2023-12-07T10:19:30.732929+0000","last_undegraded":"2023-12-07T10:19:30.732929+0000","last_fullsized":"2023-12-07T10:19:30.732929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:33:25.787281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.737405+0000","last_change":"2023-12-07T10:17:41.615916+0000","last_active":"2023-12-07T10:19:30.737405+0000","last_peered":"2023-12-07T10:19:30.737405+0000","last_clean":"2023-12-07T10:19:30.737405+0000","last_became_active":"2023-12-07T10:17:41.615591+0000","last_became_peered":"2023-12-07T10:17:41.615591+0000","last_unstale":"2023-12-07T10:19:30.737405+0000","last_undegraded":"2023-12-07T10:19:30.737405+0000","last_fullsized":"2023-12-07T10:19:30.737405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:40:45.541571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.721231+0000","last_change":"2023-12-07T10:17:41.614735+0000","last_active":"2023-12-07T10:19:30.721231+0000","last_peered":"2023-12-07T10:19:30.721231+0000","last_clean":"2023-12-07T10:19:30.721231+0000","last_became_active":"2023-12-07T10:17:41.614388+0000","last_became_peered":"2023-12-07T10:17:41.614388+0000","last_unstale":"2023-12-07T10:19:30.721231+0000","last_undegraded":"2023-12-07T10:19:30.721231+0000","last_fullsized":"2023-12-07T10:19:30.721231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:10:07.196841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.721750+0000","last_change":"2023-12-07T10:17:41.620059+0000","last_active":"2023-12-07T10:19:30.721750+0000","last_peered":"2023-12-07T10:19:30.721750+0000","last_clean":"2023-12-07T10:19:30.721750+0000","last_became_active":"2023-12-07T10:17:41.619770+0000","last_became_peered":"2023-12-07T10:17:41.619770+0000","last_unstale":"2023-12-07T10:19:30.721750+0000","last_undegraded":"2023-12-07T10:19:30.721750+0000","last_fullsized":"2023-12-07T10:19:30.721750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:13:43.421776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.723274+0000","last_change":"2023-12-07T10:17:41.609720+0000","last_active":"2023-12-07T10:19:30.723274+0000","last_peered":"2023-12-07T10:19:30.723274+0000","last_clean":"2023-12-07T10:19:30.723274+0000","last_became_active":"2023-12-07T10:17:41.609417+0000","last_became_peered":"2023-12-07T10:17:41.609417+0000","last_unstale":"2023-12-07T10:19:30.723274+0000","last_undegraded":"2023-12-07T10:19:30.723274+0000","last_fullsized":"2023-12-07T10:19:30.723274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:48:47.768754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.732262+0000","last_change":"2023-12-07T10:17:41.620952+0000","last_active":"2023-12-07T10:19:30.732262+0000","last_peered":"2023-12-07T10:19:30.732262+0000","last_clean":"2023-12-07T10:19:30.732262+0000","last_became_active":"2023-12-07T10:17:41.620560+0000","last_became_peered":"2023-12-07T10:17:41.620560+0000","last_unstale":"2023-12-07T10:19:30.732262+0000","last_undegraded":"2023-12-07T10:19:30.732262+0000","last_fullsized":"2023-12-07T10:19:30.732262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:51:37.699511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.733197+0000","last_change":"2023-12-07T10:17:41.611908+0000","last_active":"2023-12-07T10:19:30.733197+0000","last_peered":"2023-12-07T10:19:30.733197+0000","last_clean":"2023-12-07T10:19:30.733197+0000","last_became_active":"2023-12-07T10:17:41.610433+0000","last_became_peered":"2023-12-07T10:17:41.610433+0000","last_unstale":"2023-12-07T10:19:30.733197+0000","last_undegraded":"2023-12-07T10:19:30.733197+0000","last_fullsized":"2023-12-07T10:19:30.733197+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:42:53.906221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.739825+0000","last_change":"2023-12-07T10:17:41.604116+0000","last_active":"2023-12-07T10:19:30.739825+0000","last_peered":"2023-12-07T10:19:30.739825+0000","last_clean":"2023-12-07T10:19:30.739825+0000","last_became_active":"2023-12-07T10:17:41.603880+0000","last_became_peered":"2023-12-07T10:17:41.603880+0000","last_unstale":"2023-12-07T10:19:30.739825+0000","last_undegraded":"2023-12-07T10:19:30.739825+0000","last_fullsized":"2023-12-07T10:19:30.739825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:56:27.343169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.739711+0000","last_change":"2023-12-07T10:17:41.610497+0000","last_active":"2023-12-07T10:19:30.739711+0000","last_peered":"2023-12-07T10:19:30.739711+0000","last_clean":"2023-12-07T10:19:30.739711+0000","last_became_active":"2023-12-07T10:17:41.609956+0000","last_became_peered":"2023-12-07T10:17:41.609956+0000","last_unstale":"2023-12-07T10:19:30.739711+0000","last_undegraded":"2023-12-07T10:19:30.739711+0000","last_fullsized":"2023-12-07T10:19:30.739711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:04:03.967377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.728017+0000","last_change":"2023-12-07T10:17:41.612284+0000","last_active":"2023-12-07T10:19:30.728017+0000","last_peered":"2023-12-07T10:19:30.728017+0000","last_clean":"2023-12-07T10:19:30.728017+0000","last_became_active":"2023-12-07T10:17:41.611917+0000","last_became_peered":"2023-12-07T10:17:41.611917+0000","last_unstale":"2023-12-07T10:19:30.728017+0000","last_undegraded":"2023-12-07T10:19:30.728017+0000","last_fullsized":"2023-12-07T10:19:30.728017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:02:27.355883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":84968,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1217,"num_read_kb":701,"num_write":261,"num_write_kb":167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":614400,"data_stored":254904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383721,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":516906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.41499999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78500000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.41699999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79000000000000004,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.505}]},{"osd":1,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.63700000000000001,"15min":0.61099999999999999},"min":{"1min":0.52100000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85599999999999998,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.626},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.439,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.48399999999999999,"15min":0.45800000000000002},"min":{"1min":0.35699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.75800000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.48499999999999999,"15min":0.46200000000000002},"min":{"1min":0.33000000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.75900000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.35199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68799999999999994,"15min":0.67500000000000004},"min":{"1min":0.60099999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.80600000000000005,"5min":1.046,"15min":1.046},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63400000000000001,"15min":0.628},"min":{"1min":0.504,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86099999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.52200000000000002}]},{"osd":5,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.58599999999999997,"15min":0.56299999999999994},"min":{"1min":0.42499999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.93899999999999995,"5min":1.034,"15min":1.034},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60599999999999998,"15min":0.58499999999999996},"min":{"1min":0.441,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.95299999999999996,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.46800000000000003}]},{"osd":6,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83499999999999996,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.438},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73199999999999998,"15min":0.71899999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.90800000000000003,"5min":1.363,"15min":1.363},"last":0.55000000000000004}]},{"osd":7,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.77100000000000002,"15min":0.754},"min":{"1min":0.65400000000000003,"5min":0.501,"15min":0.501},"max":{"1min":0.90300000000000002,"5min":1.25,"15min":1.25},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.71599999999999997,"15min":0.69899999999999995},"min":{"1min":0.54800000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94999999999999996,"5min":1.085,"15min":1.085},"last":0.57199999999999995}]},{"osd":8,"last update":"Thu Dec 7 10:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.68799999999999994,"15min":0.69399999999999995},"min":{"1min":0.47999999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.878,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77400000000000002,"15min":0.77700000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.876,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.66500000000000004}]},{"osd":9,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.79900000000000004,"15min":0.77300000000000002},"min":{"1min":0.66700000000000004,"5min":0.496,"15min":0.496},"max":{"1min":0.97699999999999998,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.752},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.80200000000000005,"15min":0.78000000000000003},"min":{"1min":0.69999999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0049999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.53700000000000003}]},{"osd":10,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.76600000000000001,"15min":0.746},"min":{"1min":0.64300000000000002,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.92500000000000004,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.80700000000000005,"15min":0.78700000000000003},"min":{"1min":0.69299999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.024,"5min":1.411,"15min":1.411},"last":0.79300000000000004}]}]},{"osd":10,"up_from":72,"seq":309237645358,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28724,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716428,"statfs":{"total":95995035648,"available":95965622272,"internally_reserved":0,"allocated":1748992,"data_stored":1168822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.71099999999999997,"15min":0.71599999999999997},"min":{"1min":0.42399999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.3130000000000002,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.625,"15min":0.62},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.59899999999999998}]},{"osd":1,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.49099999999999999,"15min":0.49299999999999999},"min":{"1min":0.311,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.48899999999999999,"15min":0.49299999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.63700000000000001}]},{"osd":2,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63800000000000001,"15min":0.64600000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79600000000000004,"5min":1.008,"15min":1.008},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.41599999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78900000000000003,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.78900000000000003}]},{"osd":3,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.44700000000000001,"5min":0.437,"15min":0.437},"max":{"1min":2.6859999999999999,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.74399999999999999,"15min":0.70999999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":8.2590000000000003,"5min":8.2590000000000003,"15min":8.2590000000000003},"last":0.73899999999999999}]},{"osd":4,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.56200000000000006,"15min":0.57799999999999996},"min":{"1min":0.35499999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.56999999999999995,"15min":0.58599999999999997},"min":{"1min":0.36399999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82499999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.67400000000000004}]},{"osd":5,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.70099999999999996,"15min":0.71299999999999997},"min":{"1min":0.46800000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.97299999999999998,"5min":1.22,"15min":1.22},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72699999999999998,"15min":0.72799999999999998},"min":{"1min":0.42499999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88200000000000001,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.85299999999999998}]},{"osd":6,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":0.81899999999999995,"15min":0.77500000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":6.5049999999999999,"5min":6.5049999999999999,"15min":6.5049999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.76800000000000002,"15min":0.78000000000000003},"min":{"1min":0.45900000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.98199999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.84099999999999997}]},{"osd":7,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.63900000000000001}]},{"osd":8,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.77800000000000002,"15min":0.79900000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0209999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.80700000000000005,"15min":0.83399999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93500000000000005,"5min":1.165,"15min":1.165},"last":0.91600000000000004}]},{"osd":9,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.82599999999999996,"15min":0.83699999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.872,"5min":0.85199999999999998,"15min":0.86599999999999999},"min":{"1min":0.56399999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":0.95199999999999996}]},{"osd":11,"last update":"Thu Dec 7 10:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.80600000000000005,"15min":0.81799999999999995},"min":{"1min":0.56000000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.92700000000000005,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.81000000000000005,"15min":0.81799999999999995},"min":{"1min":0.58599999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.98299999999999998,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.97799999999999998}]}]},{"osd":9,"up_from":67,"seq":287762808883,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":520820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.56599999999999995,"15min":0.60799999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.626,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57199999999999995,"15min":0.58499999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.66900000000000004}]},{"osd":1,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.66900000000000004,"15min":0.69899999999999995},"min":{"1min":0.45300000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.70899999999999996,"5min":1.899,"15min":1.899},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.745,"15min":0.78300000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.85799999999999998,"5min":1.409,"15min":1.409},"last":0.58599999999999997}]},{"osd":2,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":0.89700000000000002,"15min":0.85099999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":11.782999999999999,"5min":11.782999999999999,"15min":11.782999999999999},"last":0.629},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.82999999999999996,"15min":0.88400000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.94499999999999995,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.68400000000000005,"15min":0.73099999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91400000000000003,"5min":1.47,"15min":1.47},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.753,"15min":0.83299999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.032,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.42799999999999999}]},{"osd":4,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":0.879,"15min":0.85599999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":11.808999999999999,"5min":11.808999999999999,"15min":11.808999999999999},"last":0.502},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.82999999999999996,"15min":0.88500000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.88,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.53900000000000003}]},{"osd":5,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.87,"15min":0.93600000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.91800000000000004,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":1.3109999999999999,"5min":0.93100000000000005,"15min":0.90400000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":11.863,"5min":11.863,"15min":11.863},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Dec 7 10:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":0.749}]},{"osd":7,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.879,"15min":0.93999999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86099999999999999,"5min":1.5,"15min":1.5},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.94899999999999995,"15min":1.0289999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.96899999999999997,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":0.72499999999999998}]},{"osd":8,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.032,"15min":1},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":11.831,"5min":11.831,"15min":11.831},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.96999999999999997,"15min":1.048},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.006,"5min":2.6539999999999999,"15min":2.6539999999999999},"last":0.75}]},{"osd":10,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.96099999999999997,"15min":1.032},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.97499999999999998,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.88300000000000001,"15min":0.94999999999999996},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.82899999999999996,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.73899999999999999}]},{"osd":11,"last update":"Thu Dec 7 10:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.82599999999999996,"15min":0.83099999999999996},"min":{"1min":0.59199999999999997,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.95999999999999996,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.61699999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.84999999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68600000000000005}]}]},{"osd":8,"up_from":59,"seq":253403070520,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":516990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.38700000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.75600000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56699999999999995,"15min":0.56499999999999995},"min":{"1min":0.40999999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76500000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.44900000000000001}]},{"osd":1,"last update":"Thu Dec 7 10:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61199999999999999,"15min":0.58399999999999996},"min":{"1min":0.42799999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.96399999999999997,"5min":1,"15min":1},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.62,"15min":0.58899999999999997},"min":{"1min":0.50700000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77700000000000002,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.61699999999999999}]},{"osd":2,"last update":"Thu Dec 7 10:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.47699999999999998,"15min":0.44900000000000001},"min":{"1min":0.34699999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.88400000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.495},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47299999999999998,"15min":0.441},"min":{"1min":0.29399999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.65500000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.53600000000000003}]},{"osd":3,"last update":"Thu Dec 7 10:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.37,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73199999999999998,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.61399999999999999,"15min":0.59199999999999997},"min":{"1min":0.52300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81599999999999995,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.81599999999999995}]},{"osd":4,"last update":"Thu Dec 7 10:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.61699999999999999,"15min":0.59299999999999997},"min":{"1min":0.47099999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.66500000000000004,"15min":0.64100000000000001},"min":{"1min":0.56399999999999995,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92000000000000004,"5min":1.079,"15min":1.079},"last":0.59599999999999997}]},{"osd":5,"last update":"Thu Dec 7 10:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55900000000000005,"15min":0.52300000000000002},"min":{"1min":0.434,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.68000000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.59099999999999997,"15min":0.56100000000000005},"min":{"1min":0.45100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.82899999999999996,"5min":1.024,"15min":1.024},"last":0.74199999999999999}]},{"osd":6,"last update":"Thu Dec 7 10:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.69199999999999995,"15min":0.66200000000000003},"min":{"1min":0.46899999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.877,"5min":1.046,"15min":1.046},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.70999999999999996,"15min":0.67100000000000004},"min":{"1min":0.501,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93000000000000005,"5min":1.022,"15min":1.022},"last":0.75800000000000001}]},{"osd":7,"last update":"Thu Dec 7 10:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.71299999999999997,"15min":0.66300000000000003},"min":{"1min":0.64600000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94499999999999995,"5min":1.099,"15min":1.099},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.72099999999999997,"15min":0.68500000000000005},"min":{"1min":0.624,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.69899999999999995}]},{"osd":9,"last update":"Thu Dec 7 10:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.73899999999999999,"15min":0.72099999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.76900000000000002,"15min":0.74299999999999999},"min":{"1min":0.62,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.84499999999999997}]},{"osd":10,"last update":"Thu Dec 7 10:19:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.76500000000000001,"15min":0.73399999999999999},"min":{"1min":0.57599999999999996,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0009999999999999,"5min":1.069,"15min":1.069},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.873,"5min":0.77300000000000002,"15min":0.73799999999999999},"min":{"1min":0.64300000000000002,"5min":0.503,"15min":0.503},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.77100000000000002}]},{"osd":11,"last update":"Thu Dec 7 10:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.79700000000000004,"15min":0.79200000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.74199999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0149999999999999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.86699999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299453,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1683456,"data_stored":1126409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.58199999999999996,"15min":0.55200000000000005},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.5369999999999999,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56599999999999995,"15min":0.56000000000000005},"min":{"1min":0.39700000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.36899999999999999}]},{"osd":1,"last update":"Thu Dec 7 10:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.28899999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.64200000000000002,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.438},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45700000000000002,"15min":0.45500000000000002},"min":{"1min":0.33500000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.68899999999999995,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.54800000000000004}]},{"osd":2,"last update":"Thu Dec 7 10:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.57999999999999996},"min":{"1min":0.315,"5min":0.314,"15min":0.314},"max":{"1min":0.68000000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58599999999999997,"15min":0.57399999999999995},"min":{"1min":0.49299999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71099999999999997,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.68200000000000005}]},{"osd":3,"last update":"Thu Dec 7 10:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.374,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.69399999999999995,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63700000000000001,"15min":0.62},"min":{"1min":0.42799999999999999,"5min":0.38,"15min":0.38},"max":{"1min":2.4380000000000002,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":0.627}]},{"osd":4,"last update":"Thu Dec 7 10:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51900000000000002,"15min":0.51600000000000001},"min":{"1min":0.38400000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71599999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.50600000000000001,"15min":0.52700000000000002},"min":{"1min":0.35799999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.58799999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51400000000000001}]},{"osd":5,"last update":"Thu Dec 7 10:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63900000000000001,"15min":0.64400000000000002},"min":{"1min":0.47499999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.81299999999999994,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.44600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81299999999999994,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.85099999999999998}]},{"osd":6,"last update":"Thu Dec 7 10:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.68000000000000005,"15min":0.68500000000000005},"min":{"1min":0.50800000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85299999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68999999999999995,"15min":0.69299999999999995},"min":{"1min":0.47599999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.347,"5min":2.347,"15min":2.347},"last":0.56999999999999995}]},{"osd":8,"last update":"Thu Dec 7 10:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70599999999999996,"15min":0.69799999999999995},"min":{"1min":0.55900000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.83399999999999996,"5min":1.012,"15min":1.012},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72599999999999998,"15min":0.72899999999999998},"min":{"1min":0.64800000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92200000000000004,"5min":1.026,"15min":1.026},"last":0.87}]},{"osd":9,"last update":"Thu Dec 7 10:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73599999999999999,"15min":0.74199999999999999},"min":{"1min":0.55700000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94399999999999995,"5min":1.075,"15min":1.075},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.75800000000000001,"15min":0.755},"min":{"1min":0.56399999999999995,"5min":0.505,"15min":0.505},"max":{"1min":2.1989999999999998,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":0.64800000000000002}]},{"osd":10,"last update":"Thu Dec 7 10:19:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.752,"15min":0.76200000000000001},"min":{"1min":0.54500000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.83399999999999996,"5min":1.03,"15min":1.03},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73699999999999999,"15min":0.74099999999999999},"min":{"1min":0.58299999999999996,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.82499999999999996,"5min":1.002,"15min":1.002},"last":0.77800000000000002}]},{"osd":11,"last update":"Thu Dec 7 10:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.78900000000000003,"15min":0.78500000000000003},"min":{"1min":0.71799999999999997,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":0.94999999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.77000000000000002,"15min":0.76500000000000001},"min":{"1min":0.70199999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.90900000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.89200000000000002}]}]},{"osd":6,"up_from":47,"seq":201863462977,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":512228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.5,"15min":0.48199999999999998},"min":{"1min":0.374,"5min":0.316,"15min":0.316},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.53000000000000003,"15min":0.48799999999999999},"min":{"1min":0.35399999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.58299999999999996}]},{"osd":1,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.69299999999999995,"15min":0.67100000000000004},"min":{"1min":0.56399999999999995,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":3.077,"5min":3.077,"15min":3.077},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.70599999999999996,"15min":0.67300000000000004},"min":{"1min":0.55300000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.113,"5min":2.113,"15min":2.113},"last":0.76800000000000002}]},{"osd":2,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.71999999999999997,"15min":0.68200000000000005},"min":{"1min":0.51400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.42299999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":3.153,"5min":3.153,"15min":3.153},"last":0.56299999999999994}]},{"osd":3,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":3.1200000000000001,"5min":3.1200000000000001,"15min":3.1200000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.67600000000000005,"15min":0.65100000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.42499999999999999}]},{"osd":4,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.77900000000000003,"15min":0.76300000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":3.2629999999999999,"5min":3.2629999999999999,"15min":3.2629999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.76000000000000001,"15min":0.748},"min":{"1min":0.56799999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.68899999999999995}]},{"osd":5,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.79600000000000004,"15min":0.80200000000000005},"min":{"1min":0.628,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.81200000000000006,"15min":0.79600000000000004},"min":{"1min":0.55200000000000005,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.5369999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.79400000000000004}]},{"osd":7,"last update":"Thu Dec 7 10:19:25 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":0.876,"15min":0.81699999999999995},"min":{"1min":0.496,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":6.9809999999999999,"5min":6.9809999999999999,"15min":6.9809999999999999},"last":0.745},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.81599999999999995,"15min":0.79200000000000004},"min":{"1min":0.65500000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.73199999999999998}]},{"osd":8,"last update":"Thu Dec 7 10:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.84599999999999997,"15min":0.86399999999999999},"min":{"1min":0.68700000000000006,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.90400000000000003,"5min":2.569,"15min":2.569},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.85299999999999998,"15min":0.88300000000000001},"min":{"1min":0.56000000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.93600000000000005,"5min":3.1850000000000001,"15min":3.1850000000000001},"last":0.66100000000000003}]},{"osd":9,"last update":"Thu Dec 7 10:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.872,"15min":0.874},"min":{"1min":0.66000000000000003,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.88100000000000001,"15min":0.86099999999999999},"min":{"1min":0.63300000000000001,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.6600000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":0.52500000000000002}]},{"osd":10,"last update":"Thu Dec 7 10:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.88200000000000001,"15min":0.85699999999999998},"min":{"1min":0.55100000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":3.2280000000000002,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.89400000000000002,"15min":0.88600000000000001},"min":{"1min":0.76500000000000001,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.82899999999999996}]},{"osd":11,"last update":"Thu Dec 7 10:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.96399999999999997,"15min":0.98399999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":0.98399999999999999,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.86499999999999999,"5min":1.0329999999999999,"15min":1.0609999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.069,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.71899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":578272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64300000000000002,"15min":0.57099999999999995},"min":{"1min":0.38200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.93100000000000005,"5min":6.8120000000000003,"15min":6.8120000000000003},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58699999999999997,"15min":0.55100000000000005},"min":{"1min":0.44600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.81999999999999995,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.53400000000000003}]},{"osd":2,"last update":"Thu Dec 7 10:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.64900000000000002,"15min":0.56100000000000005},"min":{"1min":0.64900000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.95099999999999996,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.60499999999999998,"15min":0.52500000000000002},"min":{"1min":0.58999999999999997,"5min":0.35799999999999998,"15min":0.35699999999999998},"max":{"1min":0.89500000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.84499999999999997}]},{"osd":3,"last update":"Thu Dec 7 10:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.63800000000000001,"15min":0.57399999999999995},"min":{"1min":0.54000000000000004,"5min":0.378,"15min":0.378},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.621},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57099999999999995,"15min":0.53300000000000003},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58699999999999997}]},{"osd":4,"last update":"Thu Dec 7 10:19:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.46600000000000003,"15min":0.40600000000000003},"min":{"1min":0.36199999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.51700000000000002,"15min":0.434},"min":{"1min":0.40400000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.88500000000000001,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.55800000000000005}]},{"osd":5,"last update":"Thu Dec 7 10:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.71499999999999997,"15min":0.60699999999999998},"min":{"1min":0.627,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94699999999999995,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.65700000000000003,"15min":0.57899999999999996},"min":{"1min":0.625,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.90700000000000003}]},{"osd":6,"last update":"Thu Dec 7 10:19:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.71799999999999997,"15min":0.67600000000000005},"min":{"1min":0.54000000000000004,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.101,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65700000000000003,"15min":0.64300000000000002},"min":{"1min":0.49199999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.83099999999999996,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.72299999999999998}]},{"osd":7,"last update":"Thu Dec 7 10:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56399999999999995,"15min":0.53100000000000003},"min":{"1min":0.433,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79500000000000004,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.70799999999999996,"15min":0.64600000000000002},"min":{"1min":0.59399999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.99099999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.98899999999999999}]},{"osd":8,"last update":"Thu Dec 7 10:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.81200000000000006,"15min":0.748},"min":{"1min":0.77200000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":1.016},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.70099999999999996,"15min":0.65500000000000003},"min":{"1min":0.60899999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.86899999999999999}]},{"osd":9,"last update":"Thu Dec 7 10:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.76900000000000002,"15min":0.75600000000000001},"min":{"1min":0.55900000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.96699999999999997,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.86299999999999999,"15min":0.85199999999999998},"min":{"1min":0.63300000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0660000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":1.0649999999999999}]},{"osd":10,"last update":"Thu Dec 7 10:19:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.748,"15min":0.73999999999999999},"min":{"1min":0.51100000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0389999999999999,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.77900000000000003,"15min":0.75800000000000001},"min":{"1min":0.54600000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0760000000000001,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.96499999999999997}]},{"osd":11,"last update":"Thu Dec 7 10:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.94799999999999995,"15min":0.93999999999999995},"min":{"1min":0.85599999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":1.04},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.86299999999999999,"15min":0.84899999999999998},"min":{"1min":0.78600000000000003,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.0029999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":513684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Dec 7 10:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.628,"15min":0.58199999999999996},"min":{"1min":0.40899999999999997,"5min":0.375,"15min":0.375},"max":{"1min":2.3929999999999998,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.61099999999999999,"15min":0.57099999999999995},"min":{"1min":0.39500000000000002,"5min":0.34399999999999997,"15min":0.34300000000000003},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.622}]},{"osd":2,"last update":"Thu Dec 7 10:19:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64700000000000002,"15min":0.59999999999999998},"min":{"1min":0.437,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.97699999999999998,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66600000000000004,"15min":0.61199999999999999},"min":{"1min":0.45600000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.147,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.70199999999999996}]},{"osd":3,"last update":"Thu Dec 7 10:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.79200000000000004,"15min":0.82999999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.80500000000000005,"5min":3.484,"15min":3.484},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.74199999999999999,"15min":0.77200000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81799999999999995,"5min":3.1379999999999999,"15min":3.1379999999999999},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Dec 7 10:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.64900000000000002,"15min":0.58299999999999996},"min":{"1min":0.47299999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":3.0049999999999999,"5min":3.0049999999999999,"15min":3.0049999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.64700000000000002,"15min":0.61299999999999999},"min":{"1min":0.43099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":0.59799999999999998}]},{"osd":5,"last update":"Thu Dec 7 10:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.63900000000000001,"15min":0.624},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91000000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.626,"15min":0.624},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85399999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.628}]},{"osd":6,"last update":"Thu Dec 7 10:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63600000000000001,"15min":0.628},"min":{"1min":0.39300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.042,"5min":2.5,"15min":2.5},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.66700000000000004,"15min":0.64300000000000002},"min":{"1min":0.378,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.90700000000000003,"5min":3.452,"15min":3.452},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Dec 7 10:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.753,"15min":0.75900000000000001},"min":{"1min":0.56299999999999994,"5min":0.496,"15min":0.496},"max":{"1min":0.90300000000000002,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.83099999999999996,"15min":0.85399999999999998},"min":{"1min":0.57199999999999995,"5min":0.435,"15min":0.435},"max":{"1min":0.95099999999999996,"5min":3.0430000000000001,"15min":3.0430000000000001},"last":0.74199999999999999}]},{"osd":8,"last update":"Thu Dec 7 10:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.79400000000000004,"15min":0.79800000000000004},"min":{"1min":0.53500000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.89400000000000002,"5min":3.75,"15min":3.75},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79500000000000004,"15min":0.80700000000000005},"min":{"1min":0.59199999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0249999999999999,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.85099999999999998}]},{"osd":9,"last update":"Thu Dec 7 10:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.84299999999999997,"15min":0.85999999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.4180000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.49199999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":3.7050000000000001,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.86499999999999999}]},{"osd":10,"last update":"Thu Dec 7 10:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80500000000000005,"15min":0.78100000000000003},"min":{"1min":0.56699999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1259999999999999,"5min":3.0979999999999999,"15min":3.0979999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85499999999999998,"15min":0.84799999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2669999999999999,"5min":2.383,"15min":2.383},"last":0.82799999999999996}]},{"osd":11,"last update":"Thu Dec 7 10:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.83999999999999997,"15min":0.81599999999999995},"min":{"1min":0.64600000000000002,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.89500000000000002,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.82199999999999995,"15min":0.80200000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.079,"5min":3.1749999999999998,"15min":3.1749999999999998},"last":0.83999999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1150976,"data_stored":591383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56599999999999995,"15min":0.51600000000000001},"min":{"1min":0.36799999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57199999999999995,"15min":0.52700000000000002},"min":{"1min":0.39200000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.86499999999999999,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.66700000000000004}]},{"osd":1,"last update":"Thu Dec 7 10:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.62,"15min":0.54100000000000004},"min":{"1min":0.41699999999999998,"5min":0.38900000000000001,"15min":0.34999999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.75},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.59499999999999997,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.77500000000000002}]},{"osd":3,"last update":"Thu Dec 7 10:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.628,"15min":0.56599999999999995},"min":{"1min":0.42599999999999999,"5min":0.41399999999999998,"15min":0.35799999999999998},"max":{"1min":0.92800000000000005,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.625,"15min":0.57099999999999995},"min":{"1min":0.44400000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86199999999999999,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.55100000000000005}]},{"osd":4,"last update":"Thu Dec 7 10:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71999999999999997,"15min":0.71099999999999997},"min":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.78400000000000003,"15min":0.78000000000000003},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.83199999999999996}]},{"osd":5,"last update":"Thu Dec 7 10:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.51200000000000001,"15min":0.45400000000000001},"min":{"1min":0.438,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.502},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.52600000000000002,"15min":0.47999999999999998},"min":{"1min":0.41999999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.80400000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.59499999999999997}]},{"osd":6,"last update":"Thu Dec 7 10:19:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68000000000000005,"15min":0.67000000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.69199999999999995,"15min":0.66000000000000003},"min":{"1min":0.57699999999999996,"5min":0.5,"15min":0.5},"max":{"1min":0.96699999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.72699999999999998}]},{"osd":7,"last update":"Thu Dec 7 10:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.73799999999999999,"15min":0.73399999999999999},"min":{"1min":0.53400000000000003,"5min":0.499,"15min":0.499},"max":{"1min":0.93100000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.51200000000000001,"5min":0.501,"15min":0.501},"max":{"1min":0.88800000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.85299999999999998}]},{"osd":8,"last update":"Thu Dec 7 10:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.65700000000000003,"15min":0.61499999999999999},"min":{"1min":0.52000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.70099999999999996,"15min":0.67100000000000004},"min":{"1min":0.56699999999999995,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.018,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.92300000000000004}]},{"osd":9,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79000000000000004,"15min":0.78400000000000003},"min":{"1min":0.64000000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.038,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.88},{"interface":"front","average":{"1min":0.873,"5min":0.83899999999999997,"15min":0.83399999999999996},"min":{"1min":0.65500000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0680000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.93300000000000005}]},{"osd":10,"last update":"Thu Dec 7 10:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.79500000000000004,"15min":0.76200000000000001},"min":{"1min":0.69399999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0389999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.80200000000000005,"15min":0.77400000000000002},"min":{"1min":0.68500000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0489999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.98199999999999998}]},{"osd":11,"last update":"Thu Dec 7 10:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.873,"15min":0.873},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0129999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.873,"5min":0.876,"15min":0.876},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.087,"5min":1.129,"15min":1.129},"last":0.94999999999999996}]}]},{"osd":3,"up_from":26,"seq":111669149775,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":511605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46100000000000002,"15min":0.40699999999999997},"min":{"1min":0.32200000000000001,"5min":0.28199999999999997,"15min":0.22500000000000001},"max":{"1min":0.79500000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.434,"5min":0.47699999999999998,"15min":0.41599999999999998},"min":{"1min":0.29599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.64500000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.42199999999999999}]},{"osd":1,"last update":"Thu Dec 7 10:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71999999999999997,"15min":0.59799999999999998},"min":{"1min":0.55700000000000005,"5min":0.47299999999999998,"15min":0.438},"max":{"1min":0.84899999999999998,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66600000000000004,"15min":0.57099999999999995},"min":{"1min":0.49399999999999999,"5min":0.41899999999999998,"15min":0.39900000000000002},"max":{"1min":0.81399999999999995,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.628}]},{"osd":2,"last update":"Thu Dec 7 10:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.442,"5min":0.39500000000000002,"15min":0.38300000000000001},"max":{"1min":0.76300000000000001,"5min":1.087,"15min":1.087},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.71599999999999997,"15min":0.63},"min":{"1min":0.57799999999999996,"5min":0.44,"15min":0.34699999999999998},"max":{"1min":0.89800000000000002,"5min":1.177,"15min":1.177},"last":0.57699999999999996}]},{"osd":4,"last update":"Thu Dec 7 10:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71199999999999997,"15min":0.629},"min":{"1min":0.50800000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.032,"5min":1.208,"15min":1.208},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65000000000000002,"15min":0.59599999999999997},"min":{"1min":0.46500000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.874,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.66700000000000004}]},{"osd":5,"last update":"Thu Dec 7 10:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.63400000000000001,"15min":0.61399999999999999},"min":{"1min":0.54000000000000004,"5min":0.372,"15min":0.372},"max":{"1min":0.877,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.72599999999999998,"15min":0.69999999999999996},"min":{"1min":0.49299999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0760000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.70699999999999996}]},{"osd":6,"last update":"Thu Dec 7 10:20:05 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.65100000000000002,"15min":0.64100000000000001},"min":{"1min":0.42099999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.99099999999999999,"5min":1.282,"15min":1.282},"last":0.748},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.58499999999999996,"15min":0.60999999999999999},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83399999999999996,"5min":1.292,"15min":1.292},"last":0.501}]},{"osd":7,"last update":"Thu Dec 7 10:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.79500000000000004,"15min":0.77200000000000002},"min":{"1min":0.61199999999999999,"5min":0.498,"15min":0.498},"max":{"1min":1.1140000000000001,"5min":1.323,"15min":1.323},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95299999999999996,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.73699999999999999}]},{"osd":8,"last update":"Thu Dec 7 10:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.872}]},{"osd":9,"last update":"Thu Dec 7 10:19:15 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.86299999999999999,"15min":0.88100000000000001},"min":{"1min":0.58699999999999997,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1479999999999999,"5min":1.242,"15min":1.242},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.78500000000000003,"15min":0.80500000000000005},"min":{"1min":0.53000000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0009999999999999,"5min":1.149,"15min":1.149},"last":0.80700000000000005}]},{"osd":10,"last update":"Thu Dec 7 10:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.76600000000000001,"15min":0.78600000000000003},"min":{"1min":0.53900000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.89000000000000001,"5min":1.335,"15min":1.335},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.90600000000000003,"15min":0.92300000000000004},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.91400000000000003}]},{"osd":11,"last update":"Thu Dec 7 10:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.873,"15min":0.871},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.77500000000000002,"15min":0.77400000000000002},"min":{"1min":0.63300000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.84099999999999997}]}]},{"osd":4,"up_from":34,"seq":146028888138,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":516943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.52100000000000002,"15min":0.49099999999999999},"min":{"1min":0.41499999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76500000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Dec 7 10:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.41999999999999998,"15min":0.39300000000000002},"min":{"1min":0.28199999999999997,"5min":0.215,"15min":0.215},"max":{"1min":0.58499999999999996,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.43099999999999999,"15min":0.39500000000000002},"min":{"1min":0.32800000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.61399999999999999,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.60099999999999998}]},{"osd":2,"last update":"Thu Dec 7 10:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.61899999999999999,"15min":0.56999999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85899999999999999,"5min":1.323,"15min":1.323},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60199999999999998,"15min":0.54900000000000004},"min":{"1min":0.47599999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79500000000000004,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.66200000000000003}]},{"osd":3,"last update":"Thu Dec 7 10:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60299999999999998,"15min":0.55000000000000004},"min":{"1min":0.45800000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.873,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.57299999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83799999999999997,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.79700000000000004}]},{"osd":5,"last update":"Thu Dec 7 10:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64500000000000002,"15min":0.59699999999999998},"min":{"1min":0.47699999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.85499999999999998,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63500000000000001,"15min":0.58999999999999997},"min":{"1min":0.48799999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.876,"5min":1.258,"15min":1.258},"last":0.69099999999999995}]},{"osd":6,"last update":"Thu Dec 7 10:19:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65300000000000002,"15min":0.65100000000000002},"min":{"1min":0.434,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90000000000000002,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65200000000000002,"15min":0.63800000000000001},"min":{"1min":0.49399999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85199999999999998,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.73499999999999999}]},{"osd":7,"last update":"Thu Dec 7 10:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56899999999999995,"15min":0.52400000000000002},"min":{"1min":0.40799999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.79800000000000004,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60599999999999998,"15min":0.56200000000000006},"min":{"1min":0.40100000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.96099999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.90300000000000002}]},{"osd":8,"last update":"Thu Dec 7 10:19:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.745,"15min":0.68999999999999995},"min":{"1min":0.58499999999999996,"5min":0.432,"15min":0.432},"max":{"1min":1.026,"5min":1.381,"15min":1.381},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.73599999999999999,"15min":0.68999999999999995},"min":{"1min":0.53700000000000003,"5min":0.502,"15min":0.502},"max":{"1min":1.0389999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.78300000000000003}]},{"osd":9,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.76100000000000001,"15min":0.76700000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.93500000000000005,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78000000000000003,"15min":0.76100000000000001},"min":{"1min":0.59399999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0289999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.996}]},{"osd":10,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.55100000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.94899999999999995,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.752,"15min":0.73999999999999999},"min":{"1min":0.53800000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97499999999999998,"5min":1.038,"15min":1.038},"last":0.83999999999999997}]},{"osd":11,"last update":"Thu Dec 7 10:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.876,"15min":0.876},"min":{"1min":0.68200000000000005,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.079,"5min":1.222,"15min":1.222},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82399999999999995,"15min":0.82299999999999995},"min":{"1min":0.67100000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0129999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.95299999999999996}]}]},{"osd":5,"up_from":39,"seq":167503724614,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1691648,"data_stored":1119625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.52100000000000002},"min":{"1min":0.44,"5min":0.318,"15min":0.318},"max":{"1min":0.76300000000000001,"5min":2.105,"15min":2.105},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55000000000000004,"15min":0.52300000000000002},"min":{"1min":0.35599999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.78100000000000003,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.42499999999999999}]},{"osd":1,"last update":"Thu Dec 7 10:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.58599999999999997,"15min":0.52400000000000002},"min":{"1min":0.38600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77500000000000002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.625},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.621,"15min":0.55100000000000005},"min":{"1min":0.33100000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.81299999999999994,"5min":3.0070000000000001,"15min":3.0070000000000001},"last":0.56000000000000005}]},{"osd":2,"last update":"Thu Dec 7 10:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.438,"15min":0.39000000000000001},"min":{"1min":0.30299999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.57799999999999996,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47299999999999998,"15min":0.40500000000000003},"min":{"1min":0.36499999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.68200000000000005,"5min":1.851,"15min":1.851},"last":0.495}]},{"osd":3,"last update":"Thu Dec 7 10:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59099999999999997,"15min":0.54300000000000004},"min":{"1min":0.35199999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82399999999999995,"5min":1.093,"15min":1.093},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60099999999999998,"15min":0.55900000000000005},"min":{"1min":0.437,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80300000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Dec 7 10:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65900000000000003,"15min":0.58199999999999996},"min":{"1min":0.51900000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.874,"5min":1.722,"15min":1.722},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64000000000000001,"15min":0.58599999999999997},"min":{"1min":0.47099999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.92000000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60899999999999999}]},{"osd":6,"last update":"Thu Dec 7 10:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.66000000000000003,"15min":0.59999999999999998},"min":{"1min":0.52200000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90200000000000002,"5min":1.272,"15min":1.272},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75,"15min":0.65000000000000002},"min":{"1min":0.623,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90100000000000002,"5min":5.9720000000000004,"15min":5.9720000000000004},"last":0.72699999999999998}]},{"osd":7,"last update":"Thu Dec 7 10:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67800000000000005,"15min":0.63200000000000001},"min":{"1min":0.47099999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88100000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67900000000000005,"15min":0.64300000000000002},"min":{"1min":0.499,"5min":0.437,"15min":0.437},"max":{"1min":0.83299999999999996,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.86799999999999999}]},{"osd":8,"last update":"Thu Dec 7 10:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.67600000000000005,"15min":0.63600000000000001},"min":{"1min":0.54300000000000004,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.96799999999999997,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.64100000000000001,"15min":0.60699999999999998},"min":{"1min":0.56100000000000005,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85299999999999998,"5min":1.135,"15min":1.135},"last":0.70999999999999996}]},{"osd":9,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.75700000000000001},"min":{"1min":0.56200000000000006,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.98499999999999999,"5min":4.532,"15min":4.532},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77300000000000002,"15min":0.76600000000000001},"min":{"1min":0.55000000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.01,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.75}]},{"osd":10,"last update":"Thu Dec 7 10:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.84399999999999997,"15min":0.83999999999999997},"min":{"1min":0.58199999999999996,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.026,"5min":2.927,"15min":2.927},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.86199999999999999,"15min":0.81100000000000005},"min":{"1min":0.59099999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.94899999999999995,"5min":5.7709999999999999,"15min":5.7709999999999999},"last":0.80900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":79877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":15823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":16674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":67279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-07T10:20:11.221 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-12-07T10:20:11.222 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-07T10:20:11.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.450+0000 7f11e69ff700 1 -- 172.21.15.119:0/2718356952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11e0101560 msgr2=0x7f11e0101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:11.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.450+0000 7f11e69ff700 1 --2- 172.21.15.119:0/2718356952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11e0101560 0x7f11e0101980 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f11d0009990 tx=0x7f11d0009ca0 comp rx=0 tx=0).stop 2023-12-07T10:20:11.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.450+0000 7f11e69ff700 1 -- 172.21.15.119:0/2718356952 shutdown_connections 2023-12-07T10:20:11.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.450+0000 7f11e69ff700 1 --2- 172.21.15.119:0/2718356952 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e01030f0 0x7f11e01099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:11.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.450+0000 7f11e69ff700 1 --2- 172.21.15.119:0/2718356952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11e0101560 0x7f11e0101980 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:11.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.450+0000 7f11e69ff700 1 --2- 172.21.15.119:0/2718356952 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11e0102750 0x7f11e0102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:11.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.450+0000 7f11e69ff700 1 -- 172.21.15.119:0/2718356952 >> 172.21.15.119:0/2718356952 conn(0x7f11e00fcab0 msgr2=0x7f11e00fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:11.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.450+0000 7f11e69ff700 1 -- 172.21.15.119:0/2718356952 shutdown_connections 2023-12-07T10:20:11.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.450+0000 7f11e69ff700 1 -- 172.21.15.119:0/2718356952 wait complete. 2023-12-07T10:20:11.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.451+0000 7f11e69ff700 1 Processor -- start 2023-12-07T10:20:11.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.451+0000 7f11e69ff700 1 -- start start 2023-12-07T10:20:11.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.451+0000 7f11e69ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11e0101560 0x7f11e0075320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:11.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.451+0000 7f11e69ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e0102750 0x7f11e0073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:11.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.451+0000 7f11e69ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11e01030f0 0x7f11e0073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:11.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.451+0000 7f11e69ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e0074640 con 0x7f11e0101560 2023-12-07T10:20:11.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.451+0000 7f11e69ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e00747b0 con 0x7f11e0102750 2023-12-07T10:20:11.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.451+0000 7f11e69ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e01a1600 con 0x7f11e01030f0 2023-12-07T10:20:11.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.451+0000 7f11dffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11e0101560 0x7f11e0075320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:11.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.452+0000 7f11dffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11e0101560 0x7f11e0075320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47386/0 (socket says 172.21.15.119:47386) 2023-12-07T10:20:11.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.452+0000 7f11dffff700 1 -- 172.21.15.119:0/478864471 learned_addr learned my addr 172.21.15.119:0/478864471 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:11.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.452+0000 7f11d7fff700 1 --2- 172.21.15.119:0/478864471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e0102750 0x7f11e0073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:11.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.452+0000 7f11e4f9c700 1 --2- 172.21.15.119:0/478864471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11e01030f0 0x7f11e0073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:11.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.452+0000 7f11dffff700 1 -- 172.21.15.119:0/478864471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11e01030f0 msgr2=0x7f11e0073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:11.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.452+0000 7f11dffff700 1 --2- 172.21.15.119:0/478864471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11e01030f0 0x7f11e0073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:11.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.452+0000 7f11dffff700 1 -- 172.21.15.119:0/478864471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e0102750 msgr2=0x7f11e0073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:11.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.452+0000 7f11dffff700 1 --2- 172.21.15.119:0/478864471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e0102750 0x7f11e0073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:11.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.452+0000 7f11dffff700 1 -- 172.21.15.119:0/478864471 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c8009520 con 0x7f11e0101560 2023-12-07T10:20:11.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.452+0000 7f11e4f9c700 1 --2- 172.21.15.119:0/478864471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11e01030f0 0x7f11e0073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:20:11.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.452+0000 7f11dffff700 1 --2- 172.21.15.119:0/478864471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11e0101560 0x7f11e0075320 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f11d0000c00 tx=0x7f11d0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:11.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.452+0000 7f11ddffb700 1 -- 172.21.15.119:0/478864471 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d001d070 con 0x7f11e0101560 2023-12-07T10:20:11.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.452+0000 7f11ddffb700 1 -- 172.21.15.119:0/478864471 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11d0021410 con 0x7f11e0101560 2023-12-07T10:20:11.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.452+0000 7f11e69ff700 1 -- 172.21.15.119:0/478864471 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d00095f0 con 0x7f11e0101560 2023-12-07T10:20:11.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.452+0000 7f11ddffb700 1 -- 172.21.15.119:0/478864471 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d0017650 con 0x7f11e0101560 2023-12-07T10:20:11.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.453+0000 7f11e69ff700 1 -- 172.21.15.119:0/478864471 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e01a1a90 con 0x7f11e0101560 2023-12-07T10:20:11.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.453+0000 7f11e69ff700 1 -- 172.21.15.119:0/478864471 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e0108aa0 con 0x7f11e0101560 2023-12-07T10:20:11.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.455+0000 7f11ddffb700 1 -- 172.21.15.119:0/478864471 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f11d0021580 con 0x7f11e0101560 2023-12-07T10:20:11.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.457+0000 7f11ddffb700 1 --2- 172.21.15.119:0/478864471 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f11cc0a1b10 0x7f11cc0a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:11.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.457+0000 7f11ddffb700 1 -- 172.21.15.119:0/478864471 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f11d00cecc0 con 0x7f11e0101560 2023-12-07T10:20:11.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.457+0000 7f11d7fff700 1 --2- 172.21.15.119:0/478864471 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f11cc0a1b10 0x7f11cc0a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:11.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.457+0000 7f11d7fff700 1 --2- 172.21.15.119:0/478864471 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f11cc0a1b10 0x7f11cc0a3fd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009df0 tx=0x7f11c8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:11.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.461+0000 7f11ddffb700 1 -- 172.21.15.119:0/478864471 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f11d00d5080 con 0x7f11e0101560 2023-12-07T10:20:11.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:11 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2753226587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:20:11.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:11 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2753226587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:20:11.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:11 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2753226587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:20:11.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.659+0000 7f11e69ff700 1 -- 172.21.15.119:0/478864471 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f11e0061a50 con 0x7f11cc0a1b10 2023-12-07T10:20:11.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.671+0000 7f11ddffb700 1 -- 172.21.15.119:0/478864471 <== mgr.14150 v2:172.21.15.119:6800/1344846367 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427005 (secure 0 0 0) 0x7f11e0061a50 con 0x7f11cc0a1b10 2023-12-07T10:20:11.679 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:20:11.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.676+0000 7f11e69ff700 1 -- 172.21.15.119:0/478864471 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f11cc0a1b10 msgr2=0x7f11cc0a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:11.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.676+0000 7f11e69ff700 1 --2- 172.21.15.119:0/478864471 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f11cc0a1b10 0x7f11cc0a3fd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009df0 tx=0x7f11c8009260 comp rx=0 tx=0).stop 2023-12-07T10:20:11.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.676+0000 7f11e69ff700 1 -- 172.21.15.119:0/478864471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11e0101560 msgr2=0x7f11e0075320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:11.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.676+0000 7f11e69ff700 1 --2- 172.21.15.119:0/478864471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11e0101560 0x7f11e0075320 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f11d0000c00 tx=0x7f11d0009580 comp rx=0 tx=0).stop 2023-12-07T10:20:11.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.676+0000 7f11e69ff700 1 -- 172.21.15.119:0/478864471 shutdown_connections 2023-12-07T10:20:11.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.676+0000 7f11e69ff700 1 --2- 172.21.15.119:0/478864471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11e01030f0 0x7f11e0073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:11.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.676+0000 7f11e69ff700 1 --2- 172.21.15.119:0/478864471 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f11cc0a1b10 0x7f11cc0a3fd0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:11.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.676+0000 7f11e69ff700 1 --2- 172.21.15.119:0/478864471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11e0101560 0x7f11e0075320 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:11.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.676+0000 7f11e69ff700 1 --2- 172.21.15.119:0/478864471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e0102750 0x7f11e0073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:11.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.676+0000 7f11e69ff700 1 -- 172.21.15.119:0/478864471 >> 172.21.15.119:0/478864471 conn(0x7f11e00fcab0 msgr2=0x7f11e0105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:11.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.676+0000 7f11e69ff700 1 -- 172.21.15.119:0/478864471 shutdown_connections 2023-12-07T10:20:11.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.676+0000 7f11e69ff700 1 -- 172.21.15.119:0/478864471 wait complete. 2023-12-07T10:20:11.684 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-12-07T10:20:11.697 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2023-12-07T10:20:10.709360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":675582,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1353,"num_read_kb":820,"num_write":444,"num_write_kb":3107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338740,"kb_used_data":14548,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603084,"statfs":{"total":1151940427776,"available":1151593558016,"internally_reserved":0,"allocated":14897152,"data_stored":8193687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":331855161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2792,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":137,"num_read_kb":72,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003485"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.734136+0000","last_change":"2023-12-07T10:17:41.614206+0000","last_active":"2023-12-07T10:19:30.734136+0000","last_peered":"2023-12-07T10:19:30.734136+0000","last_clean":"2023-12-07T10:19:30.734136+0000","last_became_active":"2023-12-07T10:17:41.613553+0000","last_became_peered":"2023-12-07T10:17:41.613553+0000","last_unstale":"2023-12-07T10:19:30.734136+0000","last_undegraded":"2023-12-07T10:19:30.734136+0000","last_fullsized":"2023-12-07T10:19:30.734136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:10:22.198414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.729545+0000","last_change":"2023-12-07T10:17:41.616917+0000","last_active":"2023-12-07T10:19:30.729545+0000","last_peered":"2023-12-07T10:19:30.729545+0000","last_clean":"2023-12-07T10:19:30.729545+0000","last_became_active":"2023-12-07T10:17:41.615345+0000","last_became_peered":"2023-12-07T10:17:41.615345+0000","last_unstale":"2023-12-07T10:19:30.729545+0000","last_undegraded":"2023-12-07T10:19:30.729545+0000","last_fullsized":"2023-12-07T10:19:30.729545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:22:25.566738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.733461+0000","last_change":"2023-12-07T10:17:41.619767+0000","last_active":"2023-12-07T10:19:30.733461+0000","last_peered":"2023-12-07T10:19:30.733461+0000","last_clean":"2023-12-07T10:19:30.733461+0000","last_became_active":"2023-12-07T10:17:41.619362+0000","last_became_peered":"2023-12-07T10:17:41.619362+0000","last_unstale":"2023-12-07T10:19:30.733461+0000","last_undegraded":"2023-12-07T10:19:30.733461+0000","last_fullsized":"2023-12-07T10:19:30.733461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:12:55.458520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.730527+0000","last_change":"2023-12-07T10:18:01.861930+0000","last_active":"2023-12-07T10:19:30.730527+0000","last_peered":"2023-12-07T10:19:30.730527+0000","last_clean":"2023-12-07T10:19:30.730527+0000","last_became_active":"2023-12-07T10:18:01.861570+0000","last_became_peered":"2023-12-07T10:18:01.861570+0000","last_unstale":"2023-12-07T10:19:30.730527+0000","last_undegraded":"2023-12-07T10:19:30.730527+0000","last_fullsized":"2023-12-07T10:19:30.730527+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:29:45.234261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.736864+0000","last_change":"2023-12-07T10:17:41.613890+0000","last_active":"2023-12-07T10:19:30.736864+0000","last_peered":"2023-12-07T10:19:30.736864+0000","last_clean":"2023-12-07T10:19:30.736864+0000","last_became_active":"2023-12-07T10:17:41.613073+0000","last_became_peered":"2023-12-07T10:17:41.613073+0000","last_unstale":"2023-12-07T10:19:30.736864+0000","last_undegraded":"2023-12-07T10:19:30.736864+0000","last_fullsized":"2023-12-07T10:19:30.736864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:17:14.339002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.721832+0000","last_change":"2023-12-07T10:17:41.609585+0000","last_active":"2023-12-07T10:19:30.721832+0000","last_peered":"2023-12-07T10:19:30.721832+0000","last_clean":"2023-12-07T10:19:30.721832+0000","last_became_active":"2023-12-07T10:17:41.609256+0000","last_became_peered":"2023-12-07T10:17:41.609256+0000","last_unstale":"2023-12-07T10:19:30.721832+0000","last_undegraded":"2023-12-07T10:19:30.721832+0000","last_fullsized":"2023-12-07T10:19:30.721832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:20:15.103208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.720115+0000","last_change":"2023-12-07T10:17:41.606138+0000","last_active":"2023-12-07T10:19:30.720115+0000","last_peered":"2023-12-07T10:19:30.720115+0000","last_clean":"2023-12-07T10:19:30.720115+0000","last_became_active":"2023-12-07T10:17:41.605818+0000","last_became_peered":"2023-12-07T10:17:41.605818+0000","last_unstale":"2023-12-07T10:19:30.720115+0000","last_undegraded":"2023-12-07T10:19:30.720115+0000","last_fullsized":"2023-12-07T10:19:30.720115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:38:39.244077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.725926+0000","last_change":"2023-12-07T10:18:02.255959+0000","last_active":"2023-12-07T10:19:30.725926+0000","last_peered":"2023-12-07T10:19:30.725926+0000","last_clean":"2023-12-07T10:19:30.725926+0000","last_became_active":"2023-12-07T10:18:02.255560+0000","last_became_peered":"2023-12-07T10:18:02.255560+0000","last_unstale":"2023-12-07T10:19:30.725926+0000","last_undegraded":"2023-12-07T10:19:30.725926+0000","last_fullsized":"2023-12-07T10:19:30.725926+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:05:49.065538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.733953+0000","last_change":"2023-12-07T10:17:41.610225+0000","last_active":"2023-12-07T10:19:30.733953+0000","last_peered":"2023-12-07T10:19:30.733953+0000","last_clean":"2023-12-07T10:19:30.733953+0000","last_became_active":"2023-12-07T10:17:41.609875+0000","last_became_peered":"2023-12-07T10:17:41.609875+0000","last_unstale":"2023-12-07T10:19:30.733953+0000","last_undegraded":"2023-12-07T10:19:30.733953+0000","last_fullsized":"2023-12-07T10:19:30.733953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:00:43.136176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.726466+0000","last_change":"2023-12-07T10:17:41.617335+0000","last_active":"2023-12-07T10:19:30.726466+0000","last_peered":"2023-12-07T10:19:30.726466+0000","last_clean":"2023-12-07T10:19:30.726466+0000","last_became_active":"2023-12-07T10:17:41.617005+0000","last_became_peered":"2023-12-07T10:17:41.617005+0000","last_unstale":"2023-12-07T10:19:30.726466+0000","last_undegraded":"2023-12-07T10:19:30.726466+0000","last_fullsized":"2023-12-07T10:19:30.726466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:43:36.855465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.721509+0000","last_change":"2023-12-07T10:17:41.608104+0000","last_active":"2023-12-07T10:19:30.721509+0000","last_peered":"2023-12-07T10:19:30.721509+0000","last_clean":"2023-12-07T10:19:30.721509+0000","last_became_active":"2023-12-07T10:17:41.607769+0000","last_became_peered":"2023-12-07T10:17:41.607769+0000","last_unstale":"2023-12-07T10:19:30.721509+0000","last_undegraded":"2023-12-07T10:19:30.721509+0000","last_fullsized":"2023-12-07T10:19:30.721509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:47:04.853044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.726603+0000","last_change":"2023-12-07T10:17:41.611585+0000","last_active":"2023-12-07T10:19:30.726603+0000","last_peered":"2023-12-07T10:19:30.726603+0000","last_clean":"2023-12-07T10:19:30.726603+0000","last_became_active":"2023-12-07T10:17:41.611202+0000","last_became_peered":"2023-12-07T10:17:41.611202+0000","last_unstale":"2023-12-07T10:19:30.726603+0000","last_undegraded":"2023-12-07T10:19:30.726603+0000","last_fullsized":"2023-12-07T10:19:30.726603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:17:01.649424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.739580+0000","last_change":"2023-12-07T10:17:41.611116+0000","last_active":"2023-12-07T10:19:30.739580+0000","last_peered":"2023-12-07T10:19:30.739580+0000","last_clean":"2023-12-07T10:19:30.739580+0000","last_became_active":"2023-12-07T10:17:41.610791+0000","last_became_peered":"2023-12-07T10:17:41.610791+0000","last_unstale":"2023-12-07T10:19:30.739580+0000","last_undegraded":"2023-12-07T10:19:30.739580+0000","last_fullsized":"2023-12-07T10:19:30.739580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:48:08.822235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.729464+0000","last_change":"2023-12-07T10:17:41.617447+0000","last_active":"2023-12-07T10:19:30.729464+0000","last_peered":"2023-12-07T10:19:30.729464+0000","last_clean":"2023-12-07T10:19:30.729464+0000","last_became_active":"2023-12-07T10:17:41.612638+0000","last_became_peered":"2023-12-07T10:17:41.612638+0000","last_unstale":"2023-12-07T10:19:30.729464+0000","last_undegraded":"2023-12-07T10:19:30.729464+0000","last_fullsized":"2023-12-07T10:19:30.729464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:11:30.384367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.729222+0000","last_change":"2023-12-07T10:17:41.611607+0000","last_active":"2023-12-07T10:19:30.729222+0000","last_peered":"2023-12-07T10:19:30.729222+0000","last_clean":"2023-12-07T10:19:30.729222+0000","last_became_active":"2023-12-07T10:17:41.611331+0000","last_became_peered":"2023-12-07T10:17:41.611331+0000","last_unstale":"2023-12-07T10:19:30.729222+0000","last_undegraded":"2023-12-07T10:19:30.729222+0000","last_fullsized":"2023-12-07T10:19:30.729222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:17:53.049922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.734247+0000","last_change":"2023-12-07T10:17:41.610528+0000","last_active":"2023-12-07T10:19:30.734247+0000","last_peered":"2023-12-07T10:19:30.734247+0000","last_clean":"2023-12-07T10:19:30.734247+0000","last_became_active":"2023-12-07T10:17:41.610216+0000","last_became_peered":"2023-12-07T10:17:41.610216+0000","last_unstale":"2023-12-07T10:19:30.734247+0000","last_undegraded":"2023-12-07T10:19:30.734247+0000","last_fullsized":"2023-12-07T10:19:30.734247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:11:36.749537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.734059+0000","last_change":"2023-12-07T10:17:41.613967+0000","last_active":"2023-12-07T10:19:30.734059+0000","last_peered":"2023-12-07T10:19:30.734059+0000","last_clean":"2023-12-07T10:19:30.734059+0000","last_became_active":"2023-12-07T10:17:41.613116+0000","last_became_peered":"2023-12-07T10:17:41.613116+0000","last_unstale":"2023-12-07T10:19:30.734059+0000","last_undegraded":"2023-12-07T10:19:30.734059+0000","last_fullsized":"2023-12-07T10:19:30.734059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:46:32.444025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.730539+0000","last_change":"2023-12-07T10:17:41.605478+0000","last_active":"2023-12-07T10:19:30.730539+0000","last_peered":"2023-12-07T10:19:30.730539+0000","last_clean":"2023-12-07T10:19:30.730539+0000","last_became_active":"2023-12-07T10:17:41.605110+0000","last_became_peered":"2023-12-07T10:17:41.605110+0000","last_unstale":"2023-12-07T10:19:30.730539+0000","last_undegraded":"2023-12-07T10:19:30.730539+0000","last_fullsized":"2023-12-07T10:19:30.730539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:53:56.410196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.734012+0000","last_change":"2023-12-07T10:18:02.256377+0000","last_active":"2023-12-07T10:19:30.734012+0000","last_peered":"2023-12-07T10:19:30.734012+0000","last_clean":"2023-12-07T10:19:30.734012+0000","last_became_active":"2023-12-07T10:18:02.255987+0000","last_became_peered":"2023-12-07T10:18:02.255987+0000","last_unstale":"2023-12-07T10:19:30.734012+0000","last_undegraded":"2023-12-07T10:19:30.734012+0000","last_fullsized":"2023-12-07T10:19:30.734012+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:23:34.909187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.809857+0000","last_change":"2023-12-07T10:17:45.799925+0000","last_active":"2023-12-07T10:19:01.809857+0000","last_peered":"2023-12-07T10:19:01.809857+0000","last_clean":"2023-12-07T10:19:01.809857+0000","last_became_active":"2023-12-07T10:17:45.799505+0000","last_became_peered":"2023-12-07T10:17:45.799505+0000","last_unstale":"2023-12-07T10:19:01.809857+0000","last_undegraded":"2023-12-07T10:19:01.809857+0000","last_fullsized":"2023-12-07T10:19:01.809857+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:52:51.470333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'13","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:20:07.927725+0000","last_change":"2023-12-07T10:17:40.755066+0000","last_active":"2023-12-07T10:20:07.927725+0000","last_peered":"2023-12-07T10:20:07.927725+0000","last_clean":"2023-12-07T10:20:07.927725+0000","last_became_active":"2023-12-07T10:17:40.754709+0000","last_became_peered":"2023-12-07T10:17:40.754709+0000","last_unstale":"2023-12-07T10:20:07.927725+0000","last_undegraded":"2023-12-07T10:20:07.927725+0000","last_fullsized":"2023-12-07T10:20:07.927725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:18:31.148592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.724533+0000","last_change":"2023-12-07T10:17:41.619107+0000","last_active":"2023-12-07T10:19:30.724533+0000","last_peered":"2023-12-07T10:19:30.724533+0000","last_clean":"2023-12-07T10:19:30.724533+0000","last_became_active":"2023-12-07T10:17:41.617057+0000","last_became_peered":"2023-12-07T10:17:41.617057+0000","last_unstale":"2023-12-07T10:19:30.724533+0000","last_undegraded":"2023-12-07T10:19:30.724533+0000","last_fullsized":"2023-12-07T10:19:30.724533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:53:16.402933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.804762+0000","last_change":"2023-12-07T10:17:45.797582+0000","last_active":"2023-12-07T10:19:01.804762+0000","last_peered":"2023-12-07T10:19:01.804762+0000","last_clean":"2023-12-07T10:19:01.804762+0000","last_became_active":"2023-12-07T10:17:45.796358+0000","last_became_peered":"2023-12-07T10:17:45.796358+0000","last_unstale":"2023-12-07T10:19:01.804762+0000","last_undegraded":"2023-12-07T10:19:01.804762+0000","last_fullsized":"2023-12-07T10:19:01.804762+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:53:19.551459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.214318+0000","last_change":"2023-12-07T10:17:40.762155+0000","last_active":"2023-12-07T10:19:03.214318+0000","last_peered":"2023-12-07T10:19:03.214318+0000","last_clean":"2023-12-07T10:19:03.214318+0000","last_became_active":"2023-12-07T10:17:40.761776+0000","last_became_peered":"2023-12-07T10:17:40.761776+0000","last_unstale":"2023-12-07T10:19:03.214318+0000","last_undegraded":"2023-12-07T10:19:03.214318+0000","last_fullsized":"2023-12-07T10:19:03.214318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:15:06.096331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.734387+0000","last_change":"2023-12-07T10:17:41.614999+0000","last_active":"2023-12-07T10:19:30.734387+0000","last_peered":"2023-12-07T10:19:30.734387+0000","last_clean":"2023-12-07T10:19:30.734387+0000","last_became_active":"2023-12-07T10:17:41.614673+0000","last_became_peered":"2023-12-07T10:17:41.614673+0000","last_unstale":"2023-12-07T10:19:30.734387+0000","last_undegraded":"2023-12-07T10:19:30.734387+0000","last_fullsized":"2023-12-07T10:19:30.734387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:34:45.378843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.215249+0000","last_change":"2023-12-07T10:19:03.215249+0000","last_active":"2023-12-07T10:19:03.215249+0000","last_peered":"2023-12-07T10:19:03.215249+0000","last_clean":"2023-12-07T10:19:03.215249+0000","last_became_active":"2023-12-07T10:19:03.214647+0000","last_became_peered":"2023-12-07T10:19:03.214647+0000","last_unstale":"2023-12-07T10:19:03.215249+0000","last_undegraded":"2023-12-07T10:19:03.215249+0000","last_fullsized":"2023-12-07T10:19:03.215249+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:41:01.893661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808809+0000","last_change":"2023-12-07T10:18:01.993128+0000","last_active":"2023-12-07T10:19:01.808809+0000","last_peered":"2023-12-07T10:19:01.808809+0000","last_clean":"2023-12-07T10:19:01.808809+0000","last_became_active":"2023-12-07T10:18:01.990244+0000","last_became_peered":"2023-12-07T10:18:01.990244+0000","last_unstale":"2023-12-07T10:19:01.808809+0000","last_undegraded":"2023-12-07T10:19:01.808809+0000","last_fullsized":"2023-12-07T10:19:01.808809+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:42:53.778363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.735930+0000","last_change":"2023-12-07T10:17:41.612903+0000","last_active":"2023-12-07T10:19:30.735930+0000","last_peered":"2023-12-07T10:19:30.735930+0000","last_clean":"2023-12-07T10:19:30.735930+0000","last_became_active":"2023-12-07T10:17:41.611968+0000","last_became_peered":"2023-12-07T10:17:41.611968+0000","last_unstale":"2023-12-07T10:19:30.735930+0000","last_undegraded":"2023-12-07T10:19:30.735930+0000","last_fullsized":"2023-12-07T10:19:30.735930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:34:08.442828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.804826+0000","last_change":"2023-12-07T10:17:45.803858+0000","last_active":"2023-12-07T10:19:01.804826+0000","last_peered":"2023-12-07T10:19:01.804826+0000","last_clean":"2023-12-07T10:19:01.804826+0000","last_became_active":"2023-12-07T10:17:45.803481+0000","last_became_peered":"2023-12-07T10:17:45.803481+0000","last_unstale":"2023-12-07T10:19:01.804826+0000","last_undegraded":"2023-12-07T10:19:01.804826+0000","last_fullsized":"2023-12-07T10:19:01.804826+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:08:03.728554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808719+0000","last_change":"2023-12-07T10:17:40.751866+0000","last_active":"2023-12-07T10:19:01.808719+0000","last_peered":"2023-12-07T10:19:01.808719+0000","last_clean":"2023-12-07T10:19:01.808719+0000","last_became_active":"2023-12-07T10:17:40.751492+0000","last_became_peered":"2023-12-07T10:17:40.751492+0000","last_unstale":"2023-12-07T10:19:01.808719+0000","last_undegraded":"2023-12-07T10:19:01.808719+0000","last_fullsized":"2023-12-07T10:19:01.808719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:42:53.839651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.728912+0000","last_change":"2023-12-07T10:17:41.611326+0000","last_active":"2023-12-07T10:19:30.728912+0000","last_peered":"2023-12-07T10:19:30.728912+0000","last_clean":"2023-12-07T10:19:30.728912+0000","last_became_active":"2023-12-07T10:17:41.610863+0000","last_became_peered":"2023-12-07T10:17:41.610863+0000","last_unstale":"2023-12-07T10:19:30.728912+0000","last_undegraded":"2023-12-07T10:19:30.728912+0000","last_fullsized":"2023-12-07T10:19:30.728912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:07:55.801356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810238+0000","last_change":"2023-12-07T10:17:45.791409+0000","last_active":"2023-12-07T10:19:01.810238+0000","last_peered":"2023-12-07T10:19:01.810238+0000","last_clean":"2023-12-07T10:19:01.810238+0000","last_became_active":"2023-12-07T10:17:45.791022+0000","last_became_peered":"2023-12-07T10:17:45.791022+0000","last_unstale":"2023-12-07T10:19:01.810238+0000","last_undegraded":"2023-12-07T10:19:01.810238+0000","last_fullsized":"2023-12-07T10:19:01.810238+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:57:14.617409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'12","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:20:07.916152+0000","last_change":"2023-12-07T10:17:40.745318+0000","last_active":"2023-12-07T10:20:07.916152+0000","last_peered":"2023-12-07T10:20:07.916152+0000","last_clean":"2023-12-07T10:20:07.916152+0000","last_became_active":"2023-12-07T10:17:40.745008+0000","last_became_peered":"2023-12-07T10:17:40.745008+0000","last_unstale":"2023-12-07T10:20:07.916152+0000","last_undegraded":"2023-12-07T10:20:07.916152+0000","last_fullsized":"2023-12-07T10:20:07.916152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:42:28.503421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13625,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":16,"num_write":9,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.724884+0000","last_change":"2023-12-07T10:17:41.605121+0000","last_active":"2023-12-07T10:19:30.724884+0000","last_peered":"2023-12-07T10:19:30.724884+0000","last_clean":"2023-12-07T10:19:30.724884+0000","last_became_active":"2023-12-07T10:17:41.604770+0000","last_became_peered":"2023-12-07T10:17:41.604770+0000","last_unstale":"2023-12-07T10:19:30.724884+0000","last_undegraded":"2023-12-07T10:19:30.724884+0000","last_fullsized":"2023-12-07T10:19:30.724884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:51:02.151155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.810286+0000","last_change":"2023-12-07T10:17:45.809039+0000","last_active":"2023-12-07T10:19:01.810286+0000","last_peered":"2023-12-07T10:19:01.810286+0000","last_clean":"2023-12-07T10:19:01.810286+0000","last_became_active":"2023-12-07T10:17:45.808622+0000","last_became_peered":"2023-12-07T10:17:45.808622+0000","last_unstale":"2023-12-07T10:19:01.810286+0000","last_undegraded":"2023-12-07T10:19:01.810286+0000","last_fullsized":"2023-12-07T10:19:01.810286+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:16:02.835931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.815605+0000","last_change":"2023-12-07T10:17:40.757876+0000","last_active":"2023-12-07T10:19:02.815605+0000","last_peered":"2023-12-07T10:19:02.815605+0000","last_clean":"2023-12-07T10:19:02.815605+0000","last_became_active":"2023-12-07T10:17:40.757613+0000","last_became_peered":"2023-12-07T10:17:40.757613+0000","last_unstale":"2023-12-07T10:19:02.815605+0000","last_undegraded":"2023-12-07T10:19:02.815605+0000","last_fullsized":"2023-12-07T10:19:02.815605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:30:40.423092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.732539+0000","last_change":"2023-12-07T10:17:41.611575+0000","last_active":"2023-12-07T10:19:30.732539+0000","last_peered":"2023-12-07T10:19:30.732539+0000","last_clean":"2023-12-07T10:19:30.732539+0000","last_became_active":"2023-12-07T10:17:41.611042+0000","last_became_peered":"2023-12-07T10:17:41.611042+0000","last_unstale":"2023-12-07T10:19:30.732539+0000","last_undegraded":"2023-12-07T10:19:30.732539+0000","last_fullsized":"2023-12-07T10:19:30.732539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:54:05.589792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.214206+0000","last_change":"2023-12-07T10:17:45.796640+0000","last_active":"2023-12-07T10:19:03.214206+0000","last_peered":"2023-12-07T10:19:03.214206+0000","last_clean":"2023-12-07T10:19:03.214206+0000","last_became_active":"2023-12-07T10:17:45.796183+0000","last_became_peered":"2023-12-07T10:17:45.796183+0000","last_unstale":"2023-12-07T10:19:03.214206+0000","last_undegraded":"2023-12-07T10:19:03.214206+0000","last_fullsized":"2023-12-07T10:19:03.214206+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:41:56.890662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.839561+0000","last_change":"2023-12-07T10:17:40.760283+0000","last_active":"2023-12-07T10:19:02.839561+0000","last_peered":"2023-12-07T10:19:02.839561+0000","last_clean":"2023-12-07T10:19:02.839561+0000","last_became_active":"2023-12-07T10:17:40.759398+0000","last_became_peered":"2023-12-07T10:17:40.759398+0000","last_unstale":"2023-12-07T10:19:02.839561+0000","last_undegraded":"2023-12-07T10:19:02.839561+0000","last_fullsized":"2023-12-07T10:19:02.839561+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:34:37.365490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.736248+0000","last_change":"2023-12-07T10:17:41.604150+0000","last_active":"2023-12-07T10:19:30.736248+0000","last_peered":"2023-12-07T10:19:30.736248+0000","last_clean":"2023-12-07T10:19:30.736248+0000","last_became_active":"2023-12-07T10:17:41.603688+0000","last_became_peered":"2023-12-07T10:17:41.603688+0000","last_unstale":"2023-12-07T10:19:30.736248+0000","last_undegraded":"2023-12-07T10:19:30.736248+0000","last_fullsized":"2023-12-07T10:19:30.736248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:52:43.735609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.383739+0000","last_change":"2023-12-07T10:17:45.806569+0000","last_active":"2023-12-07T10:19:03.383739+0000","last_peered":"2023-12-07T10:19:03.383739+0000","last_clean":"2023-12-07T10:19:03.383739+0000","last_became_active":"2023-12-07T10:17:45.806099+0000","last_became_peered":"2023-12-07T10:17:45.806099+0000","last_unstale":"2023-12-07T10:19:03.383739+0000","last_undegraded":"2023-12-07T10:19:03.383739+0000","last_fullsized":"2023-12-07T10:19:03.383739+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:02:19.977670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.818060+0000","last_change":"2023-12-07T10:17:40.744388+0000","last_active":"2023-12-07T10:19:02.818060+0000","last_peered":"2023-12-07T10:19:02.818060+0000","last_clean":"2023-12-07T10:19:02.818060+0000","last_became_active":"2023-12-07T10:17:40.743887+0000","last_became_peered":"2023-12-07T10:17:40.743887+0000","last_unstale":"2023-12-07T10:19:02.818060+0000","last_undegraded":"2023-12-07T10:19:02.818060+0000","last_fullsized":"2023-12-07T10:19:02.818060+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:18:39.102099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.733087+0000","last_change":"2023-12-07T10:17:41.595616+0000","last_active":"2023-12-07T10:19:30.733087+0000","last_peered":"2023-12-07T10:19:30.733087+0000","last_clean":"2023-12-07T10:19:30.733087+0000","last_became_active":"2023-12-07T10:17:41.595256+0000","last_became_peered":"2023-12-07T10:17:41.595256+0000","last_unstale":"2023-12-07T10:19:30.733087+0000","last_undegraded":"2023-12-07T10:19:30.733087+0000","last_fullsized":"2023-12-07T10:19:30.733087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:08:00.156043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.213892+0000","last_change":"2023-12-07T10:17:45.794523+0000","last_active":"2023-12-07T10:19:03.213892+0000","last_peered":"2023-12-07T10:19:03.213892+0000","last_clean":"2023-12-07T10:19:03.213892+0000","last_became_active":"2023-12-07T10:17:45.794139+0000","last_became_peered":"2023-12-07T10:17:45.794139+0000","last_unstale":"2023-12-07T10:19:03.213892+0000","last_undegraded":"2023-12-07T10:19:03.213892+0000","last_fullsized":"2023-12-07T10:19:03.213892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:43:24.767627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'6","reported_seq":265,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:20:09.919230+0000","last_change":"2023-12-07T10:17:40.759620+0000","last_active":"2023-12-07T10:20:09.919230+0000","last_peered":"2023-12-07T10:20:09.919230+0000","last_clean":"2023-12-07T10:20:09.919230+0000","last_became_active":"2023-12-07T10:17:40.759372+0000","last_became_peered":"2023-12-07T10:17:40.759372+0000","last_unstale":"2023-12-07T10:20:09.919230+0000","last_undegraded":"2023-12-07T10:20:09.919230+0000","last_fullsized":"2023-12-07T10:20:09.919230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:46:14.459409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":237,"num_read_kb":237,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.731231+0000","last_change":"2023-12-07T10:17:41.601152+0000","last_active":"2023-12-07T10:19:30.731231+0000","last_peered":"2023-12-07T10:19:30.731231+0000","last_clean":"2023-12-07T10:19:30.731231+0000","last_became_active":"2023-12-07T10:17:41.600147+0000","last_became_peered":"2023-12-07T10:17:41.600147+0000","last_unstale":"2023-12-07T10:19:30.731231+0000","last_undegraded":"2023-12-07T10:19:30.731231+0000","last_fullsized":"2023-12-07T10:19:30.731231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:40:46.671489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.808262+0000","last_change":"2023-12-07T10:17:45.796229+0000","last_active":"2023-12-07T10:19:02.808262+0000","last_peered":"2023-12-07T10:19:02.808262+0000","last_clean":"2023-12-07T10:19:02.808262+0000","last_became_active":"2023-12-07T10:17:45.795582+0000","last_became_peered":"2023-12-07T10:17:45.795582+0000","last_unstale":"2023-12-07T10:19:02.808262+0000","last_undegraded":"2023-12-07T10:19:02.808262+0000","last_fullsized":"2023-12-07T10:19:02.808262+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:29:32.853975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.809294+0000","last_change":"2023-12-07T10:17:40.748353+0000","last_active":"2023-12-07T10:19:02.809294+0000","last_peered":"2023-12-07T10:19:02.809294+0000","last_clean":"2023-12-07T10:19:02.809294+0000","last_became_active":"2023-12-07T10:17:40.748050+0000","last_became_peered":"2023-12-07T10:17:40.748050+0000","last_unstale":"2023-12-07T10:19:02.809294+0000","last_undegraded":"2023-12-07T10:19:02.809294+0000","last_fullsized":"2023-12-07T10:19:02.809294+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:18:15.298534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.736790+0000","last_change":"2023-12-07T10:18:02.256728+0000","last_active":"2023-12-07T10:19:30.736790+0000","last_peered":"2023-12-07T10:19:30.736790+0000","last_clean":"2023-12-07T10:19:30.736790+0000","last_became_active":"2023-12-07T10:18:02.256271+0000","last_became_peered":"2023-12-07T10:18:02.256271+0000","last_unstale":"2023-12-07T10:19:30.736790+0000","last_undegraded":"2023-12-07T10:19:30.736790+0000","last_fullsized":"2023-12-07T10:19:30.736790+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:27:00.198934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.809834+0000","last_change":"2023-12-07T10:17:45.796903+0000","last_active":"2023-12-07T10:19:02.809834+0000","last_peered":"2023-12-07T10:19:02.809834+0000","last_clean":"2023-12-07T10:19:02.809834+0000","last_became_active":"2023-12-07T10:17:45.796454+0000","last_became_peered":"2023-12-07T10:17:45.796454+0000","last_unstale":"2023-12-07T10:19:02.809834+0000","last_undegraded":"2023-12-07T10:19:02.809834+0000","last_fullsized":"2023-12-07T10:19:02.809834+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:46:36.955396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.818962+0000","last_change":"2023-12-07T10:17:40.756045+0000","last_active":"2023-12-07T10:19:02.818962+0000","last_peered":"2023-12-07T10:19:02.818962+0000","last_clean":"2023-12-07T10:19:02.818962+0000","last_became_active":"2023-12-07T10:17:40.755763+0000","last_became_peered":"2023-12-07T10:17:40.755763+0000","last_unstale":"2023-12-07T10:19:02.818962+0000","last_undegraded":"2023-12-07T10:19:02.818962+0000","last_fullsized":"2023-12-07T10:19:02.818962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:01:08.009076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.736894+0000","last_change":"2023-12-07T10:17:41.611178+0000","last_active":"2023-12-07T10:19:30.736894+0000","last_peered":"2023-12-07T10:19:30.736894+0000","last_clean":"2023-12-07T10:19:30.736894+0000","last_became_active":"2023-12-07T10:17:41.610750+0000","last_became_peered":"2023-12-07T10:17:41.610750+0000","last_unstale":"2023-12-07T10:19:30.736894+0000","last_undegraded":"2023-12-07T10:19:30.736894+0000","last_fullsized":"2023-12-07T10:19:30.736894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:56:30.763523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.808289+0000","last_change":"2023-12-07T10:17:45.800183+0000","last_active":"2023-12-07T10:19:02.808289+0000","last_peered":"2023-12-07T10:19:02.808289+0000","last_clean":"2023-12-07T10:19:02.808289+0000","last_became_active":"2023-12-07T10:17:45.798598+0000","last_became_peered":"2023-12-07T10:17:45.798598+0000","last_unstale":"2023-12-07T10:19:02.808289+0000","last_undegraded":"2023-12-07T10:19:02.808289+0000","last_fullsized":"2023-12-07T10:19:02.808289+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:06:22.162951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'39","reported_seq":310,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:20:09.919018+0000","last_change":"2023-12-07T10:17:40.752481+0000","last_active":"2023-12-07T10:20:09.919018+0000","last_peered":"2023-12-07T10:20:09.919018+0000","last_clean":"2023-12-07T10:20:09.919018+0000","last_became_active":"2023-12-07T10:17:40.752158+0000","last_became_peered":"2023-12-07T10:17:40.752158+0000","last_unstale":"2023-12-07T10:20:09.919018+0000","last_undegraded":"2023-12-07T10:20:09.919018+0000","last_fullsized":"2023-12-07T10:20:09.919018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:43:27.963014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65014,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":249,"num_read_kb":188,"num_write":41,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.725283+0000","last_change":"2023-12-07T10:17:41.614107+0000","last_active":"2023-12-07T10:19:30.725283+0000","last_peered":"2023-12-07T10:19:30.725283+0000","last_clean":"2023-12-07T10:19:30.725283+0000","last_became_active":"2023-12-07T10:17:41.613743+0000","last_became_peered":"2023-12-07T10:17:41.613743+0000","last_unstale":"2023-12-07T10:19:30.725283+0000","last_undegraded":"2023-12-07T10:19:30.725283+0000","last_fullsized":"2023-12-07T10:19:30.725283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:49:03.200387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.808863+0000","last_change":"2023-12-07T10:17:45.804282+0000","last_active":"2023-12-07T10:19:01.808863+0000","last_peered":"2023-12-07T10:19:01.808863+0000","last_clean":"2023-12-07T10:19:01.808863+0000","last_became_active":"2023-12-07T10:17:45.803794+0000","last_became_peered":"2023-12-07T10:17:45.803794+0000","last_unstale":"2023-12-07T10:19:01.808863+0000","last_undegraded":"2023-12-07T10:19:01.808863+0000","last_fullsized":"2023-12-07T10:19:01.808863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:31:26.887565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.840418+0000","last_change":"2023-12-07T10:17:40.747204+0000","last_active":"2023-12-07T10:19:02.840418+0000","last_peered":"2023-12-07T10:19:02.840418+0000","last_clean":"2023-12-07T10:19:02.840418+0000","last_became_active":"2023-12-07T10:17:40.746916+0000","last_became_peered":"2023-12-07T10:17:40.746916+0000","last_unstale":"2023-12-07T10:19:02.840418+0000","last_undegraded":"2023-12-07T10:19:02.840418+0000","last_fullsized":"2023-12-07T10:19:02.840418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:53:49.228320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.736158+0000","last_change":"2023-12-07T10:17:41.606895+0000","last_active":"2023-12-07T10:19:30.736158+0000","last_peered":"2023-12-07T10:19:30.736158+0000","last_clean":"2023-12-07T10:19:30.736158+0000","last_became_active":"2023-12-07T10:17:41.606596+0000","last_became_peered":"2023-12-07T10:17:41.606596+0000","last_unstale":"2023-12-07T10:19:30.736158+0000","last_undegraded":"2023-12-07T10:19:30.736158+0000","last_fullsized":"2023-12-07T10:19:30.736158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:34:25.166906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.214106+0000","last_change":"2023-12-07T10:17:45.807101+0000","last_active":"2023-12-07T10:19:03.214106+0000","last_peered":"2023-12-07T10:19:03.214106+0000","last_clean":"2023-12-07T10:19:03.214106+0000","last_became_active":"2023-12-07T10:17:45.806770+0000","last_became_peered":"2023-12-07T10:17:45.806770+0000","last_unstale":"2023-12-07T10:19:03.214106+0000","last_undegraded":"2023-12-07T10:19:03.214106+0000","last_fullsized":"2023-12-07T10:19:03.214106+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:39:32.522968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.818391+0000","last_change":"2023-12-07T10:17:40.754856+0000","last_active":"2023-12-07T10:19:02.818391+0000","last_peered":"2023-12-07T10:19:02.818391+0000","last_clean":"2023-12-07T10:19:02.818391+0000","last_became_active":"2023-12-07T10:17:40.754488+0000","last_became_peered":"2023-12-07T10:17:40.754488+0000","last_unstale":"2023-12-07T10:19:02.818391+0000","last_undegraded":"2023-12-07T10:19:02.818391+0000","last_fullsized":"2023-12-07T10:19:02.818391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:44:50.960216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.737528+0000","last_change":"2023-12-07T10:17:41.606963+0000","last_active":"2023-12-07T10:19:30.737528+0000","last_peered":"2023-12-07T10:19:30.737528+0000","last_clean":"2023-12-07T10:19:30.737528+0000","last_became_active":"2023-12-07T10:17:41.606408+0000","last_became_peered":"2023-12-07T10:17:41.606408+0000","last_unstale":"2023-12-07T10:19:30.737528+0000","last_undegraded":"2023-12-07T10:19:30.737528+0000","last_fullsized":"2023-12-07T10:19:30.737528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:25:47.906649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.384918+0000","last_change":"2023-12-07T10:19:03.384918+0000","last_active":"2023-12-07T10:19:03.384918+0000","last_peered":"2023-12-07T10:19:03.384918+0000","last_clean":"2023-12-07T10:19:03.384918+0000","last_became_active":"2023-12-07T10:19:03.384613+0000","last_became_peered":"2023-12-07T10:19:03.384613+0000","last_unstale":"2023-12-07T10:19:03.384918+0000","last_undegraded":"2023-12-07T10:19:03.384918+0000","last_fullsized":"2023-12-07T10:19:03.384918+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:07:29.385953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"93'7","reported_seq":145,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:20:04.924559+0000","last_change":"2023-12-07T10:17:40.747494+0000","last_active":"2023-12-07T10:20:04.924559+0000","last_peered":"2023-12-07T10:20:04.924559+0000","last_clean":"2023-12-07T10:20:04.924559+0000","last_became_active":"2023-12-07T10:17:40.747157+0000","last_became_peered":"2023-12-07T10:17:40.747157+0000","last_unstale":"2023-12-07T10:20:04.924559+0000","last_undegraded":"2023-12-07T10:20:04.924559+0000","last_fullsized":"2023-12-07T10:20:04.924559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:45:24.395418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1117,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.727117+0000","last_change":"2023-12-07T10:17:41.609495+0000","last_active":"2023-12-07T10:19:30.727117+0000","last_peered":"2023-12-07T10:19:30.727117+0000","last_clean":"2023-12-07T10:19:30.727117+0000","last_became_active":"2023-12-07T10:17:41.609181+0000","last_became_peered":"2023-12-07T10:17:41.609181+0000","last_unstale":"2023-12-07T10:19:30.727117+0000","last_undegraded":"2023-12-07T10:19:30.727117+0000","last_fullsized":"2023-12-07T10:19:30.727117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:20:33.473824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-07T10:19:01.813855+0000","last_change":"2023-12-07T10:17:45.808862+0000","last_active":"2023-12-07T10:19:01.813855+0000","last_peered":"2023-12-07T10:19:01.813855+0000","last_clean":"2023-12-07T10:19:01.813855+0000","last_became_active":"2023-12-07T10:17:45.808461+0000","last_became_peered":"2023-12-07T10:17:45.808461+0000","last_unstale":"2023-12-07T10:19:01.813855+0000","last_undegraded":"2023-12-07T10:19:01.813855+0000","last_fullsized":"2023-12-07T10:19:01.813855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:09:12.598126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.810181+0000","last_change":"2023-12-07T10:18:02.263709+0000","last_active":"2023-12-07T10:19:02.810181+0000","last_peered":"2023-12-07T10:19:02.810181+0000","last_clean":"2023-12-07T10:19:02.810181+0000","last_became_active":"2023-12-07T10:18:02.255788+0000","last_became_peered":"2023-12-07T10:18:02.255788+0000","last_unstale":"2023-12-07T10:19:02.810181+0000","last_undegraded":"2023-12-07T10:19:02.810181+0000","last_fullsized":"2023-12-07T10:19:02.810181+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:12:04.976226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.727135+0000","last_change":"2023-12-07T10:17:41.610509+0000","last_active":"2023-12-07T10:19:30.727135+0000","last_peered":"2023-12-07T10:19:30.727135+0000","last_clean":"2023-12-07T10:19:30.727135+0000","last_became_active":"2023-12-07T10:17:41.610121+0000","last_became_peered":"2023-12-07T10:17:41.610121+0000","last_unstale":"2023-12-07T10:19:30.727135+0000","last_undegraded":"2023-12-07T10:19:30.727135+0000","last_fullsized":"2023-12-07T10:19:30.727135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:53:17.365880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.818151+0000","last_change":"2023-12-07T10:17:45.795818+0000","last_active":"2023-12-07T10:19:02.818151+0000","last_peered":"2023-12-07T10:19:02.818151+0000","last_clean":"2023-12-07T10:19:02.818151+0000","last_became_active":"2023-12-07T10:17:45.795477+0000","last_became_peered":"2023-12-07T10:17:45.795477+0000","last_unstale":"2023-12-07T10:19:02.818151+0000","last_undegraded":"2023-12-07T10:19:02.818151+0000","last_fullsized":"2023-12-07T10:19:02.818151+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:17:21.494654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.811207+0000","last_change":"2023-12-07T10:17:40.751791+0000","last_active":"2023-12-07T10:19:02.811207+0000","last_peered":"2023-12-07T10:19:02.811207+0000","last_clean":"2023-12-07T10:19:02.811207+0000","last_became_active":"2023-12-07T10:17:40.751502+0000","last_became_peered":"2023-12-07T10:17:40.751502+0000","last_unstale":"2023-12-07T10:19:02.811207+0000","last_undegraded":"2023-12-07T10:19:02.811207+0000","last_fullsized":"2023-12-07T10:19:02.811207+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:15:52.319265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.723353+0000","last_change":"2023-12-07T10:18:02.256247+0000","last_active":"2023-12-07T10:19:30.723353+0000","last_peered":"2023-12-07T10:19:30.723353+0000","last_clean":"2023-12-07T10:19:30.723353+0000","last_became_active":"2023-12-07T10:18:02.255732+0000","last_became_peered":"2023-12-07T10:18:02.255732+0000","last_unstale":"2023-12-07T10:19:30.723353+0000","last_undegraded":"2023-12-07T10:19:30.723353+0000","last_fullsized":"2023-12-07T10:19:30.723353+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:00:06.516804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.810760+0000","last_change":"2023-12-07T10:17:45.791966+0000","last_active":"2023-12-07T10:19:02.810760+0000","last_peered":"2023-12-07T10:19:02.810760+0000","last_clean":"2023-12-07T10:19:02.810760+0000","last_became_active":"2023-12-07T10:17:45.791613+0000","last_became_peered":"2023-12-07T10:17:45.791613+0000","last_unstale":"2023-12-07T10:19:02.810760+0000","last_undegraded":"2023-12-07T10:19:02.810760+0000","last_fullsized":"2023-12-07T10:19:02.810760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:13:02.291289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.808445+0000","last_change":"2023-12-07T10:17:40.762896+0000","last_active":"2023-12-07T10:19:02.808445+0000","last_peered":"2023-12-07T10:19:02.808445+0000","last_clean":"2023-12-07T10:19:02.808445+0000","last_became_active":"2023-12-07T10:17:40.762427+0000","last_became_peered":"2023-12-07T10:17:40.762427+0000","last_unstale":"2023-12-07T10:19:02.808445+0000","last_undegraded":"2023-12-07T10:19:02.808445+0000","last_fullsized":"2023-12-07T10:19:02.808445+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:56:35.589548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.732249+0000","last_change":"2023-12-07T10:17:41.610354+0000","last_active":"2023-12-07T10:19:30.732249+0000","last_peered":"2023-12-07T10:19:30.732249+0000","last_clean":"2023-12-07T10:19:30.732249+0000","last_became_active":"2023-12-07T10:17:41.609888+0000","last_became_peered":"2023-12-07T10:17:41.609888+0000","last_unstale":"2023-12-07T10:19:30.732249+0000","last_undegraded":"2023-12-07T10:19:30.732249+0000","last_fullsized":"2023-12-07T10:19:30.732249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:48:19.976949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.839882+0000","last_change":"2023-12-07T10:17:45.807324+0000","last_active":"2023-12-07T10:19:02.839882+0000","last_peered":"2023-12-07T10:19:02.839882+0000","last_clean":"2023-12-07T10:19:02.839882+0000","last_became_active":"2023-12-07T10:17:45.806036+0000","last_became_peered":"2023-12-07T10:17:45.806036+0000","last_unstale":"2023-12-07T10:19:02.839882+0000","last_undegraded":"2023-12-07T10:19:02.839882+0000","last_fullsized":"2023-12-07T10:19:02.839882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:54:36.847992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'5","reported_seq":145,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:20:08.929550+0000","last_change":"2023-12-07T10:17:40.754247+0000","last_active":"2023-12-07T10:20:08.929550+0000","last_peered":"2023-12-07T10:20:08.929550+0000","last_clean":"2023-12-07T10:20:08.929550+0000","last_became_active":"2023-12-07T10:17:40.753964+0000","last_became_peered":"2023-12-07T10:17:40.753964+0000","last_unstale":"2023-12-07T10:20:08.929550+0000","last_undegraded":"2023-12-07T10:20:08.929550+0000","last_fullsized":"2023-12-07T10:20:08.929550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:22:04.832399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.723617+0000","last_change":"2023-12-07T10:17:41.610516+0000","last_active":"2023-12-07T10:19:30.723617+0000","last_peered":"2023-12-07T10:19:30.723617+0000","last_clean":"2023-12-07T10:19:30.723617+0000","last_became_active":"2023-12-07T10:17:41.610184+0000","last_became_peered":"2023-12-07T10:17:41.610184+0000","last_unstale":"2023-12-07T10:19:30.723617+0000","last_undegraded":"2023-12-07T10:19:30.723617+0000","last_fullsized":"2023-12-07T10:19:30.723617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:53:43.100723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.840220+0000","last_change":"2023-12-07T10:17:45.804624+0000","last_active":"2023-12-07T10:19:02.840220+0000","last_peered":"2023-12-07T10:19:02.840220+0000","last_clean":"2023-12-07T10:19:02.840220+0000","last_became_active":"2023-12-07T10:17:45.803667+0000","last_became_peered":"2023-12-07T10:17:45.803667+0000","last_unstale":"2023-12-07T10:19:02.840220+0000","last_undegraded":"2023-12-07T10:19:02.840220+0000","last_fullsized":"2023-12-07T10:19:02.840220+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:55:09.642795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.726585+0000","last_change":"2023-12-07T10:17:41.618433+0000","last_active":"2023-12-07T10:19:30.726585+0000","last_peered":"2023-12-07T10:19:30.726585+0000","last_clean":"2023-12-07T10:19:30.726585+0000","last_became_active":"2023-12-07T10:17:41.617942+0000","last_became_peered":"2023-12-07T10:17:41.617942+0000","last_unstale":"2023-12-07T10:19:30.726585+0000","last_undegraded":"2023-12-07T10:19:30.726585+0000","last_fullsized":"2023-12-07T10:19:30.726585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:26:48.166157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":144,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:20:06.926556+0000","last_change":"2023-12-07T10:17:40.761907+0000","last_active":"2023-12-07T10:20:06.926556+0000","last_peered":"2023-12-07T10:20:06.926556+0000","last_clean":"2023-12-07T10:20:06.926556+0000","last_became_active":"2023-12-07T10:17:40.761564+0000","last_became_peered":"2023-12-07T10:17:40.761564+0000","last_unstale":"2023-12-07T10:20:06.926556+0000","last_undegraded":"2023-12-07T10:20:06.926556+0000","last_fullsized":"2023-12-07T10:20:06.926556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:57:22.304352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.809474+0000","last_change":"2023-12-07T10:17:45.792684+0000","last_active":"2023-12-07T10:19:02.809474+0000","last_peered":"2023-12-07T10:19:02.809474+0000","last_clean":"2023-12-07T10:19:02.809474+0000","last_became_active":"2023-12-07T10:17:45.792172+0000","last_became_peered":"2023-12-07T10:17:45.792172+0000","last_unstale":"2023-12-07T10:19:02.809474+0000","last_undegraded":"2023-12-07T10:19:02.809474+0000","last_fullsized":"2023-12-07T10:19:02.809474+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:24:31.084727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.737013+0000","last_change":"2023-12-07T10:17:41.612582+0000","last_active":"2023-12-07T10:19:30.737013+0000","last_peered":"2023-12-07T10:19:30.737013+0000","last_clean":"2023-12-07T10:19:30.737013+0000","last_became_active":"2023-12-07T10:17:41.612244+0000","last_became_peered":"2023-12-07T10:17:41.612244+0000","last_unstale":"2023-12-07T10:19:30.737013+0000","last_undegraded":"2023-12-07T10:19:30.737013+0000","last_fullsized":"2023-12-07T10:19:30.737013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:41:53.506037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.809740+0000","last_change":"2023-12-07T10:17:40.748565+0000","last_active":"2023-12-07T10:19:02.809740+0000","last_peered":"2023-12-07T10:19:02.809740+0000","last_clean":"2023-12-07T10:19:02.809740+0000","last_became_active":"2023-12-07T10:17:40.748250+0000","last_became_peered":"2023-12-07T10:17:40.748250+0000","last_unstale":"2023-12-07T10:19:02.809740+0000","last_undegraded":"2023-12-07T10:19:02.809740+0000","last_fullsized":"2023-12-07T10:19:02.809740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:26:28.202440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":329,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.808386+0000","last_change":"2023-12-07T10:16:33.482173+0000","last_active":"2023-12-07T10:19:02.808386+0000","last_peered":"2023-12-07T10:19:02.808386+0000","last_clean":"2023-12-07T10:19:02.808386+0000","last_became_active":"2023-12-07T10:16:33.481699+0000","last_became_peered":"2023-12-07T10:16:33.481699+0000","last_unstale":"2023-12-07T10:19:02.808386+0000","last_undegraded":"2023-12-07T10:19:02.808386+0000","last_fullsized":"2023-12-07T10:19:02.808386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:13:25.388258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:13:25.388258+0000","last_clean_scrub_stamp":"2023-12-07T10:13:25.388258+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:57:09.583677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.818400+0000","last_change":"2023-12-07T10:17:45.806289+0000","last_active":"2023-12-07T10:19:02.818400+0000","last_peered":"2023-12-07T10:19:02.818400+0000","last_clean":"2023-12-07T10:19:02.818400+0000","last_became_active":"2023-12-07T10:17:45.805954+0000","last_became_peered":"2023-12-07T10:17:45.805954+0000","last_unstale":"2023-12-07T10:19:02.818400+0000","last_undegraded":"2023-12-07T10:19:02.818400+0000","last_fullsized":"2023-12-07T10:19:02.818400+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:21:09.955380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.727877+0000","last_change":"2023-12-07T10:17:41.612322+0000","last_active":"2023-12-07T10:19:30.727877+0000","last_peered":"2023-12-07T10:19:30.727877+0000","last_clean":"2023-12-07T10:19:30.727877+0000","last_became_active":"2023-12-07T10:17:41.612039+0000","last_became_peered":"2023-12-07T10:17:41.612039+0000","last_unstale":"2023-12-07T10:19:30.727877+0000","last_undegraded":"2023-12-07T10:19:30.727877+0000","last_fullsized":"2023-12-07T10:19:30.727877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:40:49.883742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.808419+0000","last_change":"2023-12-07T10:17:40.749694+0000","last_active":"2023-12-07T10:19:02.808419+0000","last_peered":"2023-12-07T10:19:02.808419+0000","last_clean":"2023-12-07T10:19:02.808419+0000","last_became_active":"2023-12-07T10:17:40.749393+0000","last_became_peered":"2023-12-07T10:17:40.749393+0000","last_unstale":"2023-12-07T10:19:02.808419+0000","last_undegraded":"2023-12-07T10:19:02.808419+0000","last_fullsized":"2023-12-07T10:19:02.808419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:15:36.949062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.214038+0000","last_change":"2023-12-07T10:17:45.805769+0000","last_active":"2023-12-07T10:19:03.214038+0000","last_peered":"2023-12-07T10:19:03.214038+0000","last_clean":"2023-12-07T10:19:03.214038+0000","last_became_active":"2023-12-07T10:17:45.805346+0000","last_became_peered":"2023-12-07T10:17:45.805346+0000","last_unstale":"2023-12-07T10:19:03.214038+0000","last_undegraded":"2023-12-07T10:19:03.214038+0000","last_fullsized":"2023-12-07T10:19:03.214038+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:47:47.781933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.214136+0000","last_change":"2023-12-07T10:18:01.862558+0000","last_active":"2023-12-07T10:19:03.214136+0000","last_peered":"2023-12-07T10:19:03.214136+0000","last_clean":"2023-12-07T10:19:03.214136+0000","last_became_active":"2023-12-07T10:18:01.862063+0000","last_became_peered":"2023-12-07T10:18:01.862063+0000","last_unstale":"2023-12-07T10:19:03.214136+0000","last_undegraded":"2023-12-07T10:19:03.214136+0000","last_fullsized":"2023-12-07T10:19:03.214136+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:25:32.652160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.733695+0000","last_change":"2023-12-07T10:17:41.617199+0000","last_active":"2023-12-07T10:19:30.733695+0000","last_peered":"2023-12-07T10:19:30.733695+0000","last_clean":"2023-12-07T10:19:30.733695+0000","last_became_active":"2023-12-07T10:17:41.616901+0000","last_became_peered":"2023-12-07T10:17:41.616901+0000","last_unstale":"2023-12-07T10:19:30.733695+0000","last_undegraded":"2023-12-07T10:19:30.733695+0000","last_fullsized":"2023-12-07T10:19:30.733695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:23:59.433861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.213749+0000","last_change":"2023-12-07T10:17:45.790914+0000","last_active":"2023-12-07T10:19:03.213749+0000","last_peered":"2023-12-07T10:19:03.213749+0000","last_clean":"2023-12-07T10:19:03.213749+0000","last_became_active":"2023-12-07T10:17:45.790428+0000","last_became_peered":"2023-12-07T10:17:45.790428+0000","last_unstale":"2023-12-07T10:19:03.213749+0000","last_undegraded":"2023-12-07T10:19:03.213749+0000","last_fullsized":"2023-12-07T10:19:03.213749+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:03:57.516300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'11","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:33.545539+0000","last_change":"2023-12-07T10:18:01.863765+0000","last_active":"2023-12-07T10:19:33.545539+0000","last_peered":"2023-12-07T10:19:33.545539+0000","last_clean":"2023-12-07T10:19:33.545539+0000","last_became_active":"2023-12-07T10:18:01.858926+0000","last_became_peered":"2023-12-07T10:18:01.858926+0000","last_unstale":"2023-12-07T10:19:33.545539+0000","last_undegraded":"2023-12-07T10:19:33.545539+0000","last_fullsized":"2023-12-07T10:19:33.545539+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:44:19.663592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.733950+0000","last_change":"2023-12-07T10:17:41.608513+0000","last_active":"2023-12-07T10:19:30.733950+0000","last_peered":"2023-12-07T10:19:30.733950+0000","last_clean":"2023-12-07T10:19:30.733950+0000","last_became_active":"2023-12-07T10:17:41.605049+0000","last_became_peered":"2023-12-07T10:17:41.605049+0000","last_unstale":"2023-12-07T10:19:30.733950+0000","last_undegraded":"2023-12-07T10:19:30.733950+0000","last_fullsized":"2023-12-07T10:19:30.733950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:33:00.730827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.840029+0000","last_change":"2023-12-07T10:17:45.792583+0000","last_active":"2023-12-07T10:19:02.840029+0000","last_peered":"2023-12-07T10:19:02.840029+0000","last_clean":"2023-12-07T10:19:02.840029+0000","last_became_active":"2023-12-07T10:17:45.792162+0000","last_became_peered":"2023-12-07T10:17:45.792162+0000","last_unstale":"2023-12-07T10:19:02.840029+0000","last_undegraded":"2023-12-07T10:19:02.840029+0000","last_fullsized":"2023-12-07T10:19:02.840029+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:47:30.901477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.808789+0000","last_change":"2023-12-07T10:17:40.750826+0000","last_active":"2023-12-07T10:19:02.808789+0000","last_peered":"2023-12-07T10:19:02.808789+0000","last_clean":"2023-12-07T10:19:02.808789+0000","last_became_active":"2023-12-07T10:17:40.750519+0000","last_became_peered":"2023-12-07T10:17:40.750519+0000","last_unstale":"2023-12-07T10:19:02.808789+0000","last_undegraded":"2023-12-07T10:19:02.808789+0000","last_fullsized":"2023-12-07T10:19:02.808789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:21:01.008897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.735624+0000","last_change":"2023-12-07T10:17:41.609488+0000","last_active":"2023-12-07T10:19:30.735624+0000","last_peered":"2023-12-07T10:19:30.735624+0000","last_clean":"2023-12-07T10:19:30.735624+0000","last_became_active":"2023-12-07T10:17:41.609061+0000","last_became_peered":"2023-12-07T10:17:41.609061+0000","last_unstale":"2023-12-07T10:19:30.735624+0000","last_undegraded":"2023-12-07T10:19:30.735624+0000","last_fullsized":"2023-12-07T10:19:30.735624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:39:32.729502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.808632+0000","last_change":"2023-12-07T10:17:45.797573+0000","last_active":"2023-12-07T10:19:02.808632+0000","last_peered":"2023-12-07T10:19:02.808632+0000","last_clean":"2023-12-07T10:19:02.808632+0000","last_became_active":"2023-12-07T10:17:45.797119+0000","last_became_peered":"2023-12-07T10:17:45.797119+0000","last_unstale":"2023-12-07T10:19:02.808632+0000","last_undegraded":"2023-12-07T10:19:02.808632+0000","last_fullsized":"2023-12-07T10:19:02.808632+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:17:27.100470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.810618+0000","last_change":"2023-12-07T10:17:40.753596+0000","last_active":"2023-12-07T10:19:02.810618+0000","last_peered":"2023-12-07T10:19:02.810618+0000","last_clean":"2023-12-07T10:19:02.810618+0000","last_became_active":"2023-12-07T10:17:40.753251+0000","last_became_peered":"2023-12-07T10:17:40.753251+0000","last_unstale":"2023-12-07T10:19:02.810618+0000","last_undegraded":"2023-12-07T10:19:02.810618+0000","last_fullsized":"2023-12-07T10:19:02.810618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:08:30.868794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.725751+0000","last_change":"2023-12-07T10:17:41.599689+0000","last_active":"2023-12-07T10:19:30.725751+0000","last_peered":"2023-12-07T10:19:30.725751+0000","last_clean":"2023-12-07T10:19:30.725751+0000","last_became_active":"2023-12-07T10:17:41.599358+0000","last_became_peered":"2023-12-07T10:17:41.599358+0000","last_unstale":"2023-12-07T10:19:30.725751+0000","last_undegraded":"2023-12-07T10:19:30.725751+0000","last_fullsized":"2023-12-07T10:19:30.725751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T13:44:19.742560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.215169+0000","last_change":"2023-12-07T10:19:03.215169+0000","last_active":"2023-12-07T10:19:03.215169+0000","last_peered":"2023-12-07T10:19:03.215169+0000","last_clean":"2023-12-07T10:19:03.215169+0000","last_became_active":"2023-12-07T10:19:03.214754+0000","last_became_peered":"2023-12-07T10:19:03.214754+0000","last_unstale":"2023-12-07T10:19:03.215169+0000","last_undegraded":"2023-12-07T10:19:03.215169+0000","last_fullsized":"2023-12-07T10:19:03.215169+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:51:41.461131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.213651+0000","last_change":"2023-12-07T10:17:40.758663+0000","last_active":"2023-12-07T10:19:03.213651+0000","last_peered":"2023-12-07T10:19:03.213651+0000","last_clean":"2023-12-07T10:19:03.213651+0000","last_became_active":"2023-12-07T10:17:40.753611+0000","last_became_peered":"2023-12-07T10:17:40.753611+0000","last_unstale":"2023-12-07T10:19:03.213651+0000","last_undegraded":"2023-12-07T10:19:03.213651+0000","last_fullsized":"2023-12-07T10:19:03.213651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:00:59.256468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.730928+0000","last_change":"2023-12-07T10:17:41.616546+0000","last_active":"2023-12-07T10:19:30.730928+0000","last_peered":"2023-12-07T10:19:30.730928+0000","last_clean":"2023-12-07T10:19:30.730928+0000","last_became_active":"2023-12-07T10:17:41.615780+0000","last_became_peered":"2023-12-07T10:17:41.615780+0000","last_unstale":"2023-12-07T10:19:30.730928+0000","last_undegraded":"2023-12-07T10:19:30.730928+0000","last_fullsized":"2023-12-07T10:19:30.730928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:22:32.455784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.809460+0000","last_change":"2023-12-07T10:17:45.797550+0000","last_active":"2023-12-07T10:19:02.809460+0000","last_peered":"2023-12-07T10:19:02.809460+0000","last_clean":"2023-12-07T10:19:02.809460+0000","last_became_active":"2023-12-07T10:17:45.797140+0000","last_became_peered":"2023-12-07T10:17:45.797140+0000","last_unstale":"2023-12-07T10:19:02.809460+0000","last_undegraded":"2023-12-07T10:19:02.809460+0000","last_fullsized":"2023-12-07T10:19:02.809460+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:53:44.984904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"93'6","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:54.075606+0000","last_change":"2023-12-07T10:17:40.748410+0000","last_active":"2023-12-07T10:19:54.075606+0000","last_peered":"2023-12-07T10:19:54.075606+0000","last_clean":"2023-12-07T10:19:54.075606+0000","last_became_active":"2023-12-07T10:17:40.748069+0000","last_became_peered":"2023-12-07T10:17:40.748069+0000","last_unstale":"2023-12-07T10:19:54.075606+0000","last_undegraded":"2023-12-07T10:19:54.075606+0000","last_fullsized":"2023-12-07T10:19:54.075606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:51:48.278974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.734791+0000","last_change":"2023-12-07T10:17:41.609248+0000","last_active":"2023-12-07T10:19:30.734791+0000","last_peered":"2023-12-07T10:19:30.734791+0000","last_clean":"2023-12-07T10:19:30.734791+0000","last_became_active":"2023-12-07T10:17:41.608943+0000","last_became_peered":"2023-12-07T10:17:41.608943+0000","last_unstale":"2023-12-07T10:19:30.734791+0000","last_undegraded":"2023-12-07T10:19:30.734791+0000","last_fullsized":"2023-12-07T10:19:30.734791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T16:29:32.322273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.808615+0000","last_change":"2023-12-07T10:17:45.794855+0000","last_active":"2023-12-07T10:19:02.808615+0000","last_peered":"2023-12-07T10:19:02.808615+0000","last_clean":"2023-12-07T10:19:02.808615+0000","last_became_active":"2023-12-07T10:17:45.794410+0000","last_became_peered":"2023-12-07T10:17:45.794410+0000","last_unstale":"2023-12-07T10:19:02.808615+0000","last_undegraded":"2023-12-07T10:19:02.808615+0000","last_fullsized":"2023-12-07T10:19:02.808615+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:57:25.118036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.810614+0000","last_change":"2023-12-07T10:17:40.750471+0000","last_active":"2023-12-07T10:19:02.810614+0000","last_peered":"2023-12-07T10:19:02.810614+0000","last_clean":"2023-12-07T10:19:02.810614+0000","last_became_active":"2023-12-07T10:17:40.750176+0000","last_became_peered":"2023-12-07T10:17:40.750176+0000","last_unstale":"2023-12-07T10:19:02.810614+0000","last_undegraded":"2023-12-07T10:19:02.810614+0000","last_fullsized":"2023-12-07T10:19:02.810614+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T17:04:22.305343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.734958+0000","last_change":"2023-12-07T10:17:41.614087+0000","last_active":"2023-12-07T10:19:30.734958+0000","last_peered":"2023-12-07T10:19:30.734958+0000","last_clean":"2023-12-07T10:19:30.734958+0000","last_became_active":"2023-12-07T10:17:41.607689+0000","last_became_peered":"2023-12-07T10:17:41.607689+0000","last_unstale":"2023-12-07T10:19:30.734958+0000","last_undegraded":"2023-12-07T10:19:30.734958+0000","last_fullsized":"2023-12-07T10:19:30.734958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:50:10.316085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.841410+0000","last_change":"2023-12-07T10:19:02.841001+0000","last_active":"2023-12-07T10:19:02.841410+0000","last_peered":"2023-12-07T10:19:02.841410+0000","last_clean":"2023-12-07T10:19:02.841410+0000","last_became_active":"2023-12-07T10:19:02.840595+0000","last_became_peered":"2023-12-07T10:19:02.840595+0000","last_unstale":"2023-12-07T10:19:02.841410+0000","last_undegraded":"2023-12-07T10:19:02.841410+0000","last_fullsized":"2023-12-07T10:19:02.841410+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:18:35.590140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'5","reported_seq":145,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:20:07.928786+0000","last_change":"2023-12-07T10:17:40.753645+0000","last_active":"2023-12-07T10:20:07.928786+0000","last_peered":"2023-12-07T10:20:07.928786+0000","last_clean":"2023-12-07T10:20:07.928786+0000","last_became_active":"2023-12-07T10:17:40.753326+0000","last_became_peered":"2023-12-07T10:17:40.753326+0000","last_unstale":"2023-12-07T10:20:07.928786+0000","last_undegraded":"2023-12-07T10:20:07.928786+0000","last_fullsized":"2023-12-07T10:20:07.928786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:43:03.617591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.719211+0000","last_change":"2023-12-07T10:17:41.607968+0000","last_active":"2023-12-07T10:19:30.719211+0000","last_peered":"2023-12-07T10:19:30.719211+0000","last_clean":"2023-12-07T10:19:30.719211+0000","last_became_active":"2023-12-07T10:17:41.607621+0000","last_became_peered":"2023-12-07T10:17:41.607621+0000","last_unstale":"2023-12-07T10:19:30.719211+0000","last_undegraded":"2023-12-07T10:19:30.719211+0000","last_fullsized":"2023-12-07T10:19:30.719211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:55:10.224738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:03.383569+0000","last_change":"2023-12-07T10:17:45.793678+0000","last_active":"2023-12-07T10:19:03.383569+0000","last_peered":"2023-12-07T10:19:03.383569+0000","last_clean":"2023-12-07T10:19:03.383569+0000","last_became_active":"2023-12-07T10:17:45.793283+0000","last_became_peered":"2023-12-07T10:17:45.793283+0000","last_unstale":"2023-12-07T10:19:03.383569+0000","last_undegraded":"2023-12-07T10:19:03.383569+0000","last_fullsized":"2023-12-07T10:19:03.383569+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:33:56.501107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.818650+0000","last_change":"2023-12-07T10:17:40.762069+0000","last_active":"2023-12-07T10:19:02.818650+0000","last_peered":"2023-12-07T10:19:02.818650+0000","last_clean":"2023-12-07T10:19:02.818650+0000","last_became_active":"2023-12-07T10:17:40.761752+0000","last_became_peered":"2023-12-07T10:17:40.761752+0000","last_unstale":"2023-12-07T10:19:02.818650+0000","last_undegraded":"2023-12-07T10:19:02.818650+0000","last_fullsized":"2023-12-07T10:19:02.818650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T11:57:55.736144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.731611+0000","last_change":"2023-12-07T10:17:41.607347+0000","last_active":"2023-12-07T10:19:30.731611+0000","last_peered":"2023-12-07T10:19:30.731611+0000","last_clean":"2023-12-07T10:19:30.731611+0000","last_became_active":"2023-12-07T10:17:41.606618+0000","last_became_peered":"2023-12-07T10:17:41.606618+0000","last_unstale":"2023-12-07T10:19:30.731611+0000","last_undegraded":"2023-12-07T10:19:30.731611+0000","last_fullsized":"2023-12-07T10:19:30.731611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:53:34.748130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.817908+0000","last_change":"2023-12-07T10:17:45.791459+0000","last_active":"2023-12-07T10:19:02.817908+0000","last_peered":"2023-12-07T10:19:02.817908+0000","last_clean":"2023-12-07T10:19:02.817908+0000","last_became_active":"2023-12-07T10:17:45.791070+0000","last_became_peered":"2023-12-07T10:17:45.791070+0000","last_unstale":"2023-12-07T10:19:02.817908+0000","last_undegraded":"2023-12-07T10:19:02.817908+0000","last_fullsized":"2023-12-07T10:19:02.817908+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:44.766631+0000","last_clean_scrub_stamp":"2023-12-07T10:17:44.766631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:46:35.985990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:02.840125+0000","last_change":"2023-12-07T10:17:40.762726+0000","last_active":"2023-12-07T10:19:02.840125+0000","last_peered":"2023-12-07T10:19:02.840125+0000","last_clean":"2023-12-07T10:19:02.840125+0000","last_became_active":"2023-12-07T10:17:40.762378+0000","last_became_peered":"2023-12-07T10:17:40.762378+0000","last_unstale":"2023-12-07T10:19:02.840125+0000","last_undegraded":"2023-12-07T10:19:02.840125+0000","last_fullsized":"2023-12-07T10:19:02.840125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:39.717837+0000","last_clean_scrub_stamp":"2023-12-07T10:17:39.717837+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:42:06.193834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.718804+0000","last_change":"2023-12-07T10:17:41.602356+0000","last_active":"2023-12-07T10:19:30.718804+0000","last_peered":"2023-12-07T10:19:30.718804+0000","last_clean":"2023-12-07T10:19:30.718804+0000","last_became_active":"2023-12-07T10:17:41.602025+0000","last_became_peered":"2023-12-07T10:17:41.602025+0000","last_unstale":"2023-12-07T10:19:30.718804+0000","last_undegraded":"2023-12-07T10:19:30.718804+0000","last_fullsized":"2023-12-07T10:19:30.718804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T19:33:04.994510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.728025+0000","last_change":"2023-12-07T10:17:41.616766+0000","last_active":"2023-12-07T10:19:30.728025+0000","last_peered":"2023-12-07T10:19:30.728025+0000","last_clean":"2023-12-07T10:19:30.728025+0000","last_became_active":"2023-12-07T10:17:41.616221+0000","last_became_peered":"2023-12-07T10:17:41.616221+0000","last_unstale":"2023-12-07T10:19:30.728025+0000","last_undegraded":"2023-12-07T10:19:30.728025+0000","last_fullsized":"2023-12-07T10:19:30.728025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T12:16:00.393772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.715723+0000","last_change":"2023-12-07T10:17:41.613646+0000","last_active":"2023-12-07T10:19:30.715723+0000","last_peered":"2023-12-07T10:19:30.715723+0000","last_clean":"2023-12-07T10:19:30.715723+0000","last_became_active":"2023-12-07T10:17:41.613257+0000","last_became_peered":"2023-12-07T10:17:41.613257+0000","last_unstale":"2023-12-07T10:19:30.715723+0000","last_undegraded":"2023-12-07T10:19:30.715723+0000","last_fullsized":"2023-12-07T10:19:30.715723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T15:10:33.416955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.723232+0000","last_change":"2023-12-07T10:17:41.620017+0000","last_active":"2023-12-07T10:19:30.723232+0000","last_peered":"2023-12-07T10:19:30.723232+0000","last_clean":"2023-12-07T10:19:30.723232+0000","last_became_active":"2023-12-07T10:17:41.619647+0000","last_became_peered":"2023-12-07T10:17:41.619647+0000","last_unstale":"2023-12-07T10:19:30.723232+0000","last_undegraded":"2023-12-07T10:19:30.723232+0000","last_fullsized":"2023-12-07T10:19:30.723232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:36:42.891160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.732929+0000","last_change":"2023-12-07T10:17:41.610803+0000","last_active":"2023-12-07T10:19:30.732929+0000","last_peered":"2023-12-07T10:19:30.732929+0000","last_clean":"2023-12-07T10:19:30.732929+0000","last_became_active":"2023-12-07T10:17:41.610388+0000","last_became_peered":"2023-12-07T10:17:41.610388+0000","last_unstale":"2023-12-07T10:19:30.732929+0000","last_undegraded":"2023-12-07T10:19:30.732929+0000","last_fullsized":"2023-12-07T10:19:30.732929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T10:33:25.787281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.737405+0000","last_change":"2023-12-07T10:17:41.615916+0000","last_active":"2023-12-07T10:19:30.737405+0000","last_peered":"2023-12-07T10:19:30.737405+0000","last_clean":"2023-12-07T10:19:30.737405+0000","last_became_active":"2023-12-07T10:17:41.615591+0000","last_became_peered":"2023-12-07T10:17:41.615591+0000","last_unstale":"2023-12-07T10:19:30.737405+0000","last_undegraded":"2023-12-07T10:19:30.737405+0000","last_fullsized":"2023-12-07T10:19:30.737405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:40:45.541571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.721231+0000","last_change":"2023-12-07T10:17:41.614735+0000","last_active":"2023-12-07T10:19:30.721231+0000","last_peered":"2023-12-07T10:19:30.721231+0000","last_clean":"2023-12-07T10:19:30.721231+0000","last_became_active":"2023-12-07T10:17:41.614388+0000","last_became_peered":"2023-12-07T10:17:41.614388+0000","last_unstale":"2023-12-07T10:19:30.721231+0000","last_undegraded":"2023-12-07T10:19:30.721231+0000","last_fullsized":"2023-12-07T10:19:30.721231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:10:07.196841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.721750+0000","last_change":"2023-12-07T10:17:41.620059+0000","last_active":"2023-12-07T10:19:30.721750+0000","last_peered":"2023-12-07T10:19:30.721750+0000","last_clean":"2023-12-07T10:19:30.721750+0000","last_became_active":"2023-12-07T10:17:41.619770+0000","last_became_peered":"2023-12-07T10:17:41.619770+0000","last_unstale":"2023-12-07T10:19:30.721750+0000","last_undegraded":"2023-12-07T10:19:30.721750+0000","last_fullsized":"2023-12-07T10:19:30.721750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T22:13:43.421776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.723274+0000","last_change":"2023-12-07T10:17:41.609720+0000","last_active":"2023-12-07T10:19:30.723274+0000","last_peered":"2023-12-07T10:19:30.723274+0000","last_clean":"2023-12-07T10:19:30.723274+0000","last_became_active":"2023-12-07T10:17:41.609417+0000","last_became_peered":"2023-12-07T10:17:41.609417+0000","last_unstale":"2023-12-07T10:19:30.723274+0000","last_undegraded":"2023-12-07T10:19:30.723274+0000","last_fullsized":"2023-12-07T10:19:30.723274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:48:47.768754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.732262+0000","last_change":"2023-12-07T10:17:41.620952+0000","last_active":"2023-12-07T10:19:30.732262+0000","last_peered":"2023-12-07T10:19:30.732262+0000","last_clean":"2023-12-07T10:19:30.732262+0000","last_became_active":"2023-12-07T10:17:41.620560+0000","last_became_peered":"2023-12-07T10:17:41.620560+0000","last_unstale":"2023-12-07T10:19:30.732262+0000","last_undegraded":"2023-12-07T10:19:30.732262+0000","last_fullsized":"2023-12-07T10:19:30.732262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T18:51:37.699511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.733197+0000","last_change":"2023-12-07T10:17:41.611908+0000","last_active":"2023-12-07T10:19:30.733197+0000","last_peered":"2023-12-07T10:19:30.733197+0000","last_clean":"2023-12-07T10:19:30.733197+0000","last_became_active":"2023-12-07T10:17:41.610433+0000","last_became_peered":"2023-12-07T10:17:41.610433+0000","last_unstale":"2023-12-07T10:19:30.733197+0000","last_undegraded":"2023-12-07T10:19:30.733197+0000","last_fullsized":"2023-12-07T10:19:30.733197+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:42:53.906221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.739825+0000","last_change":"2023-12-07T10:17:41.604116+0000","last_active":"2023-12-07T10:19:30.739825+0000","last_peered":"2023-12-07T10:19:30.739825+0000","last_clean":"2023-12-07T10:19:30.739825+0000","last_became_active":"2023-12-07T10:17:41.603880+0000","last_became_peered":"2023-12-07T10:17:41.603880+0000","last_unstale":"2023-12-07T10:19:30.739825+0000","last_undegraded":"2023-12-07T10:19:30.739825+0000","last_fullsized":"2023-12-07T10:19:30.739825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T14:56:27.343169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.739711+0000","last_change":"2023-12-07T10:17:41.610497+0000","last_active":"2023-12-07T10:19:30.739711+0000","last_peered":"2023-12-07T10:19:30.739711+0000","last_clean":"2023-12-07T10:19:30.739711+0000","last_became_active":"2023-12-07T10:17:41.609956+0000","last_became_peered":"2023-12-07T10:17:41.609956+0000","last_unstale":"2023-12-07T10:19:30.739711+0000","last_undegraded":"2023-12-07T10:19:30.739711+0000","last_fullsized":"2023-12-07T10:19:30.739711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T21:04:03.967377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-07T10:19:30.728017+0000","last_change":"2023-12-07T10:17:41.612284+0000","last_active":"2023-12-07T10:19:30.728017+0000","last_peered":"2023-12-07T10:19:30.728017+0000","last_clean":"2023-12-07T10:19:30.728017+0000","last_became_active":"2023-12-07T10:17:41.611917+0000","last_became_peered":"2023-12-07T10:17:41.611917+0000","last_unstale":"2023-12-07T10:19:30.728017+0000","last_undegraded":"2023-12-07T10:19:30.728017+0000","last_fullsized":"2023-12-07T10:19:30.728017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-07T10:17:40.726243+0000","last_clean_scrub_stamp":"2023-12-07T10:17:40.726243+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T20:02:27.355883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":84968,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1217,"num_read_kb":701,"num_write":261,"num_write_kb":167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":614400,"data_stored":254904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383721,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":516906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.41499999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78500000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.41699999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79000000000000004,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.505}]},{"osd":1,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.63700000000000001,"15min":0.61099999999999999},"min":{"1min":0.52100000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85599999999999998,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.626},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.439,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.48399999999999999,"15min":0.45800000000000002},"min":{"1min":0.35699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.75800000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.48499999999999999,"15min":0.46200000000000002},"min":{"1min":0.33000000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.75900000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.35199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68799999999999994,"15min":0.67500000000000004},"min":{"1min":0.60099999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.80600000000000005,"5min":1.046,"15min":1.046},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63400000000000001,"15min":0.628},"min":{"1min":0.504,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86099999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.52200000000000002}]},{"osd":5,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.58599999999999997,"15min":0.56299999999999994},"min":{"1min":0.42499999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.93899999999999995,"5min":1.034,"15min":1.034},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60599999999999998,"15min":0.58499999999999996},"min":{"1min":0.441,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.95299999999999996,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.46800000000000003}]},{"osd":6,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83499999999999996,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.438},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73199999999999998,"15min":0.71899999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.90800000000000003,"5min":1.363,"15min":1.363},"last":0.55000000000000004}]},{"osd":7,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.77100000000000002,"15min":0.754},"min":{"1min":0.65400000000000003,"5min":0.501,"15min":0.501},"max":{"1min":0.90300000000000002,"5min":1.25,"15min":1.25},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.71599999999999997,"15min":0.69899999999999995},"min":{"1min":0.54800000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94999999999999996,"5min":1.085,"15min":1.085},"last":0.57199999999999995}]},{"osd":8,"last update":"Thu Dec 7 10:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.68799999999999994,"15min":0.69399999999999995},"min":{"1min":0.47999999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.878,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77400000000000002,"15min":0.77700000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.876,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.66500000000000004}]},{"osd":9,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.79900000000000004,"15min":0.77300000000000002},"min":{"1min":0.66700000000000004,"5min":0.496,"15min":0.496},"max":{"1min":0.97699999999999998,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.752},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.80200000000000005,"15min":0.78000000000000003},"min":{"1min":0.69999999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0049999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.53700000000000003}]},{"osd":10,"last update":"Thu Dec 7 10:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.76600000000000001,"15min":0.746},"min":{"1min":0.64300000000000002,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.92500000000000004,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.80700000000000005,"15min":0.78700000000000003},"min":{"1min":0.69299999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.024,"5min":1.411,"15min":1.411},"last":0.79300000000000004}]}]},{"osd":10,"up_from":72,"seq":309237645358,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28724,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716428,"statfs":{"total":95995035648,"available":95965622272,"internally_reserved":0,"allocated":1748992,"data_stored":1168822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.71099999999999997,"15min":0.71599999999999997},"min":{"1min":0.42399999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.3130000000000002,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.625,"15min":0.62},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.59899999999999998}]},{"osd":1,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.49099999999999999,"15min":0.49299999999999999},"min":{"1min":0.311,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.48899999999999999,"15min":0.49299999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.63700000000000001}]},{"osd":2,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63800000000000001,"15min":0.64600000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79600000000000004,"5min":1.008,"15min":1.008},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.41599999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78900000000000003,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.78900000000000003}]},{"osd":3,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.44700000000000001,"5min":0.437,"15min":0.437},"max":{"1min":2.6859999999999999,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.74399999999999999,"15min":0.70999999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":8.2590000000000003,"5min":8.2590000000000003,"15min":8.2590000000000003},"last":0.73899999999999999}]},{"osd":4,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.56200000000000006,"15min":0.57799999999999996},"min":{"1min":0.35499999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.56999999999999995,"15min":0.58599999999999997},"min":{"1min":0.36399999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82499999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.67400000000000004}]},{"osd":5,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.70099999999999996,"15min":0.71299999999999997},"min":{"1min":0.46800000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.97299999999999998,"5min":1.22,"15min":1.22},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72699999999999998,"15min":0.72799999999999998},"min":{"1min":0.42499999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88200000000000001,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.85299999999999998}]},{"osd":6,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":0.81899999999999995,"15min":0.77500000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":6.5049999999999999,"5min":6.5049999999999999,"15min":6.5049999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.76800000000000002,"15min":0.78000000000000003},"min":{"1min":0.45900000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.98199999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.84099999999999997}]},{"osd":7,"last update":"Thu Dec 7 10:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.63900000000000001}]},{"osd":8,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.77800000000000002,"15min":0.79900000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0209999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.80700000000000005,"15min":0.83399999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93500000000000005,"5min":1.165,"15min":1.165},"last":0.91600000000000004}]},{"osd":9,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.82599999999999996,"15min":0.83699999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.872,"5min":0.85199999999999998,"15min":0.86599999999999999},"min":{"1min":0.56399999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":0.95199999999999996}]},{"osd":11,"last update":"Thu Dec 7 10:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.80600000000000005,"15min":0.81799999999999995},"min":{"1min":0.56000000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.92700000000000005,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.81000000000000005,"15min":0.81799999999999995},"min":{"1min":0.58599999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.98299999999999998,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.97799999999999998}]}]},{"osd":9,"up_from":67,"seq":287762808883,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":520820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.56599999999999995,"15min":0.60799999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.626,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57199999999999995,"15min":0.58499999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.66900000000000004}]},{"osd":1,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.66900000000000004,"15min":0.69899999999999995},"min":{"1min":0.45300000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.70899999999999996,"5min":1.899,"15min":1.899},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.745,"15min":0.78300000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.85799999999999998,"5min":1.409,"15min":1.409},"last":0.58599999999999997}]},{"osd":2,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":0.89700000000000002,"15min":0.85099999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":11.782999999999999,"5min":11.782999999999999,"15min":11.782999999999999},"last":0.629},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.82999999999999996,"15min":0.88400000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.94499999999999995,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.68400000000000005,"15min":0.73099999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91400000000000003,"5min":1.47,"15min":1.47},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.753,"15min":0.83299999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.032,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.42799999999999999}]},{"osd":4,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":0.879,"15min":0.85599999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":11.808999999999999,"5min":11.808999999999999,"15min":11.808999999999999},"last":0.502},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.82999999999999996,"15min":0.88500000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.88,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.53900000000000003}]},{"osd":5,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.87,"15min":0.93600000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.91800000000000004,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":1.3109999999999999,"5min":0.93100000000000005,"15min":0.90400000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":11.863,"5min":11.863,"15min":11.863},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Dec 7 10:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":0.749}]},{"osd":7,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.879,"15min":0.93999999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.86099999999999999,"5min":1.5,"15min":1.5},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.94899999999999995,"15min":1.0289999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.96899999999999997,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":0.72499999999999998}]},{"osd":8,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.032,"15min":1},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":11.831,"5min":11.831,"15min":11.831},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.96999999999999997,"15min":1.048},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.006,"5min":2.6539999999999999,"15min":2.6539999999999999},"last":0.75}]},{"osd":10,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.96099999999999997,"15min":1.032},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.97499999999999998,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.88300000000000001,"15min":0.94999999999999996},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.82899999999999996,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.73899999999999999}]},{"osd":11,"last update":"Thu Dec 7 10:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.82599999999999996,"15min":0.83099999999999996},"min":{"1min":0.59199999999999997,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.95999999999999996,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.61699999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.84999999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68600000000000005}]}]},{"osd":8,"up_from":59,"seq":253403070520,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":516990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.38700000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.75600000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56699999999999995,"15min":0.56499999999999995},"min":{"1min":0.40999999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76500000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.44900000000000001}]},{"osd":1,"last update":"Thu Dec 7 10:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61199999999999999,"15min":0.58399999999999996},"min":{"1min":0.42799999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.96399999999999997,"5min":1,"15min":1},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.62,"15min":0.58899999999999997},"min":{"1min":0.50700000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77700000000000002,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.61699999999999999}]},{"osd":2,"last update":"Thu Dec 7 10:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.47699999999999998,"15min":0.44900000000000001},"min":{"1min":0.34699999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.88400000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.495},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47299999999999998,"15min":0.441},"min":{"1min":0.29399999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.65500000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.53600000000000003}]},{"osd":3,"last update":"Thu Dec 7 10:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.37,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73199999999999998,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.61399999999999999,"15min":0.59199999999999997},"min":{"1min":0.52300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81599999999999995,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.81599999999999995}]},{"osd":4,"last update":"Thu Dec 7 10:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.61699999999999999,"15min":0.59299999999999997},"min":{"1min":0.47099999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.66500000000000004,"15min":0.64100000000000001},"min":{"1min":0.56399999999999995,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92000000000000004,"5min":1.079,"15min":1.079},"last":0.59599999999999997}]},{"osd":5,"last update":"Thu Dec 7 10:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55900000000000005,"15min":0.52300000000000002},"min":{"1min":0.434,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.68000000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.59099999999999997,"15min":0.56100000000000005},"min":{"1min":0.45100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.82899999999999996,"5min":1.024,"15min":1.024},"last":0.74199999999999999}]},{"osd":6,"last update":"Thu Dec 7 10:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.69199999999999995,"15min":0.66200000000000003},"min":{"1min":0.46899999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.877,"5min":1.046,"15min":1.046},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.70999999999999996,"15min":0.67100000000000004},"min":{"1min":0.501,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93000000000000005,"5min":1.022,"15min":1.022},"last":0.75800000000000001}]},{"osd":7,"last update":"Thu Dec 7 10:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.71299999999999997,"15min":0.66300000000000003},"min":{"1min":0.64600000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94499999999999995,"5min":1.099,"15min":1.099},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.72099999999999997,"15min":0.68500000000000005},"min":{"1min":0.624,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.69899999999999995}]},{"osd":9,"last update":"Thu Dec 7 10:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.73899999999999999,"15min":0.72099999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.76900000000000002,"15min":0.74299999999999999},"min":{"1min":0.62,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.84499999999999997}]},{"osd":10,"last update":"Thu Dec 7 10:19:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.76500000000000001,"15min":0.73399999999999999},"min":{"1min":0.57599999999999996,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0009999999999999,"5min":1.069,"15min":1.069},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.873,"5min":0.77300000000000002,"15min":0.73799999999999999},"min":{"1min":0.64300000000000002,"5min":0.503,"15min":0.503},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.77100000000000002}]},{"osd":11,"last update":"Thu Dec 7 10:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.79700000000000004,"15min":0.79200000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.74199999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0149999999999999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.86699999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299453,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1683456,"data_stored":1126409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.58199999999999996,"15min":0.55200000000000005},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.5369999999999999,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56599999999999995,"15min":0.56000000000000005},"min":{"1min":0.39700000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.36899999999999999}]},{"osd":1,"last update":"Thu Dec 7 10:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.28899999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.64200000000000002,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.438},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45700000000000002,"15min":0.45500000000000002},"min":{"1min":0.33500000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.68899999999999995,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.54800000000000004}]},{"osd":2,"last update":"Thu Dec 7 10:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.57999999999999996},"min":{"1min":0.315,"5min":0.314,"15min":0.314},"max":{"1min":0.68000000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58599999999999997,"15min":0.57399999999999995},"min":{"1min":0.49299999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71099999999999997,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.68200000000000005}]},{"osd":3,"last update":"Thu Dec 7 10:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.374,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.69399999999999995,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63700000000000001,"15min":0.62},"min":{"1min":0.42799999999999999,"5min":0.38,"15min":0.38},"max":{"1min":2.4380000000000002,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":0.627}]},{"osd":4,"last update":"Thu Dec 7 10:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51900000000000002,"15min":0.51600000000000001},"min":{"1min":0.38400000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71599999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.50600000000000001,"15min":0.52700000000000002},"min":{"1min":0.35799999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.58799999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51400000000000001}]},{"osd":5,"last update":"Thu Dec 7 10:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63900000000000001,"15min":0.64400000000000002},"min":{"1min":0.47499999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.81299999999999994,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.44600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81299999999999994,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.85099999999999998}]},{"osd":6,"last update":"Thu Dec 7 10:19:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.68000000000000005,"15min":0.68500000000000005},"min":{"1min":0.50800000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85299999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68999999999999995,"15min":0.69299999999999995},"min":{"1min":0.47599999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.347,"5min":2.347,"15min":2.347},"last":0.56999999999999995}]},{"osd":8,"last update":"Thu Dec 7 10:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70599999999999996,"15min":0.69799999999999995},"min":{"1min":0.55900000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.83399999999999996,"5min":1.012,"15min":1.012},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72599999999999998,"15min":0.72899999999999998},"min":{"1min":0.64800000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92200000000000004,"5min":1.026,"15min":1.026},"last":0.87}]},{"osd":9,"last update":"Thu Dec 7 10:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73599999999999999,"15min":0.74199999999999999},"min":{"1min":0.55700000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94399999999999995,"5min":1.075,"15min":1.075},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.75800000000000001,"15min":0.755},"min":{"1min":0.56399999999999995,"5min":0.505,"15min":0.505},"max":{"1min":2.1989999999999998,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":0.64800000000000002}]},{"osd":10,"last update":"Thu Dec 7 10:19:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.752,"15min":0.76200000000000001},"min":{"1min":0.54500000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.83399999999999996,"5min":1.03,"15min":1.03},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73699999999999999,"15min":0.74099999999999999},"min":{"1min":0.58299999999999996,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.82499999999999996,"5min":1.002,"15min":1.002},"last":0.77800000000000002}]},{"osd":11,"last update":"Thu Dec 7 10:19:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.78900000000000003,"15min":0.78500000000000003},"min":{"1min":0.71799999999999997,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":0.94999999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.77000000000000002,"15min":0.76500000000000001},"min":{"1min":0.70199999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.90900000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.89200000000000002}]}]},{"osd":6,"up_from":47,"seq":201863462977,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":512228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.5,"15min":0.48199999999999998},"min":{"1min":0.374,"5min":0.316,"15min":0.316},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.53000000000000003,"15min":0.48799999999999999},"min":{"1min":0.35399999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.58299999999999996}]},{"osd":1,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.69299999999999995,"15min":0.67100000000000004},"min":{"1min":0.56399999999999995,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":3.077,"5min":3.077,"15min":3.077},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.70599999999999996,"15min":0.67300000000000004},"min":{"1min":0.55300000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.113,"5min":2.113,"15min":2.113},"last":0.76800000000000002}]},{"osd":2,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.71999999999999997,"15min":0.68200000000000005},"min":{"1min":0.51400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.42299999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":3.153,"5min":3.153,"15min":3.153},"last":0.56299999999999994}]},{"osd":3,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":3.1200000000000001,"5min":3.1200000000000001,"15min":3.1200000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.67600000000000005,"15min":0.65100000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.42499999999999999}]},{"osd":4,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.77900000000000003,"15min":0.76300000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":3.2629999999999999,"5min":3.2629999999999999,"15min":3.2629999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.76000000000000001,"15min":0.748},"min":{"1min":0.56799999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.68899999999999995}]},{"osd":5,"last update":"Thu Dec 7 10:19:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.79600000000000004,"15min":0.80200000000000005},"min":{"1min":0.628,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.81200000000000006,"15min":0.79600000000000004},"min":{"1min":0.55200000000000005,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.5369999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.79400000000000004}]},{"osd":7,"last update":"Thu Dec 7 10:19:25 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":0.876,"15min":0.81699999999999995},"min":{"1min":0.496,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":6.9809999999999999,"5min":6.9809999999999999,"15min":6.9809999999999999},"last":0.745},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.81599999999999995,"15min":0.79200000000000004},"min":{"1min":0.65500000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.73199999999999998}]},{"osd":8,"last update":"Thu Dec 7 10:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.84599999999999997,"15min":0.86399999999999999},"min":{"1min":0.68700000000000006,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.90400000000000003,"5min":2.569,"15min":2.569},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.85299999999999998,"15min":0.88300000000000001},"min":{"1min":0.56000000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.93600000000000005,"5min":3.1850000000000001,"15min":3.1850000000000001},"last":0.66100000000000003}]},{"osd":9,"last update":"Thu Dec 7 10:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.872,"15min":0.874},"min":{"1min":0.66000000000000003,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.88100000000000001,"15min":0.86099999999999999},"min":{"1min":0.63300000000000001,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.6600000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":0.52500000000000002}]},{"osd":10,"last update":"Thu Dec 7 10:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.88200000000000001,"15min":0.85699999999999998},"min":{"1min":0.55100000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":3.2280000000000002,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.89400000000000002,"15min":0.88600000000000001},"min":{"1min":0.76500000000000001,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.82899999999999996}]},{"osd":11,"last update":"Thu Dec 7 10:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.96399999999999997,"15min":0.98399999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":0.98399999999999999,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.86499999999999999,"5min":1.0329999999999999,"15min":1.0609999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.069,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":0.71899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":578272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64300000000000002,"15min":0.57099999999999995},"min":{"1min":0.38200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.93100000000000005,"5min":6.8120000000000003,"15min":6.8120000000000003},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58699999999999997,"15min":0.55100000000000005},"min":{"1min":0.44600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.81999999999999995,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.53400000000000003}]},{"osd":2,"last update":"Thu Dec 7 10:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.64900000000000002,"15min":0.56100000000000005},"min":{"1min":0.64900000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.95099999999999996,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.60499999999999998,"15min":0.52500000000000002},"min":{"1min":0.58999999999999997,"5min":0.35799999999999998,"15min":0.35699999999999998},"max":{"1min":0.89500000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.84499999999999997}]},{"osd":3,"last update":"Thu Dec 7 10:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.63800000000000001,"15min":0.57399999999999995},"min":{"1min":0.54000000000000004,"5min":0.378,"15min":0.378},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.621},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57099999999999995,"15min":0.53300000000000003},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58699999999999997}]},{"osd":4,"last update":"Thu Dec 7 10:19:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.46600000000000003,"15min":0.40600000000000003},"min":{"1min":0.36199999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.51700000000000002,"15min":0.434},"min":{"1min":0.40400000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.88500000000000001,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.55800000000000005}]},{"osd":5,"last update":"Thu Dec 7 10:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.71499999999999997,"15min":0.60699999999999998},"min":{"1min":0.627,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94699999999999995,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.65700000000000003,"15min":0.57899999999999996},"min":{"1min":0.625,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.90700000000000003}]},{"osd":6,"last update":"Thu Dec 7 10:19:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.71799999999999997,"15min":0.67600000000000005},"min":{"1min":0.54000000000000004,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.101,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65700000000000003,"15min":0.64300000000000002},"min":{"1min":0.49199999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.83099999999999996,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.72299999999999998}]},{"osd":7,"last update":"Thu Dec 7 10:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56399999999999995,"15min":0.53100000000000003},"min":{"1min":0.433,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79500000000000004,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.70799999999999996,"15min":0.64600000000000002},"min":{"1min":0.59399999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.99099999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.98899999999999999}]},{"osd":8,"last update":"Thu Dec 7 10:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.81200000000000006,"15min":0.748},"min":{"1min":0.77200000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":1.016},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.70099999999999996,"15min":0.65500000000000003},"min":{"1min":0.60899999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.86899999999999999}]},{"osd":9,"last update":"Thu Dec 7 10:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.76900000000000002,"15min":0.75600000000000001},"min":{"1min":0.55900000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.96699999999999997,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.86299999999999999,"15min":0.85199999999999998},"min":{"1min":0.63300000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0660000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":1.0649999999999999}]},{"osd":10,"last update":"Thu Dec 7 10:19:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.748,"15min":0.73999999999999999},"min":{"1min":0.51100000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0389999999999999,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.77900000000000003,"15min":0.75800000000000001},"min":{"1min":0.54600000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0760000000000001,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.96499999999999997}]},{"osd":11,"last update":"Thu Dec 7 10:19:48 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.94799999999999995,"15min":0.93999999999999995},"min":{"1min":0.85599999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":1.04},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.86299999999999999,"15min":0.84899999999999998},"min":{"1min":0.78600000000000003,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.0029999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":513684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Dec 7 10:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.628,"15min":0.58199999999999996},"min":{"1min":0.40899999999999997,"5min":0.375,"15min":0.375},"max":{"1min":2.3929999999999998,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.61099999999999999,"15min":0.57099999999999995},"min":{"1min":0.39500000000000002,"5min":0.34399999999999997,"15min":0.34300000000000003},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.622}]},{"osd":2,"last update":"Thu Dec 7 10:19:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64700000000000002,"15min":0.59999999999999998},"min":{"1min":0.437,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.97699999999999998,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66600000000000004,"15min":0.61199999999999999},"min":{"1min":0.45600000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.147,"5min":3.4140000000000001,"15min":3.4140000000000001},"last":0.70199999999999996}]},{"osd":3,"last update":"Thu Dec 7 10:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.79200000000000004,"15min":0.82999999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.80500000000000005,"5min":3.484,"15min":3.484},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.74199999999999999,"15min":0.77200000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81799999999999995,"5min":3.1379999999999999,"15min":3.1379999999999999},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Dec 7 10:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.64900000000000002,"15min":0.58299999999999996},"min":{"1min":0.47299999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":3.0049999999999999,"5min":3.0049999999999999,"15min":3.0049999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.64700000000000002,"15min":0.61299999999999999},"min":{"1min":0.43099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":0.59799999999999998}]},{"osd":5,"last update":"Thu Dec 7 10:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.63900000000000001,"15min":0.624},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91000000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.626,"15min":0.624},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85399999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.628}]},{"osd":6,"last update":"Thu Dec 7 10:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63600000000000001,"15min":0.628},"min":{"1min":0.39300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.042,"5min":2.5,"15min":2.5},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.66700000000000004,"15min":0.64300000000000002},"min":{"1min":0.378,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.90700000000000003,"5min":3.452,"15min":3.452},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Dec 7 10:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.753,"15min":0.75900000000000001},"min":{"1min":0.56299999999999994,"5min":0.496,"15min":0.496},"max":{"1min":0.90300000000000002,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.83099999999999996,"15min":0.85399999999999998},"min":{"1min":0.57199999999999995,"5min":0.435,"15min":0.435},"max":{"1min":0.95099999999999996,"5min":3.0430000000000001,"15min":3.0430000000000001},"last":0.74199999999999999}]},{"osd":8,"last update":"Thu Dec 7 10:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.79400000000000004,"15min":0.79800000000000004},"min":{"1min":0.53500000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.89400000000000002,"5min":3.75,"15min":3.75},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79500000000000004,"15min":0.80700000000000005},"min":{"1min":0.59199999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0249999999999999,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.85099999999999998}]},{"osd":9,"last update":"Thu Dec 7 10:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.84299999999999997,"15min":0.85999999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.4180000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.49199999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":3.7050000000000001,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.86499999999999999}]},{"osd":10,"last update":"Thu Dec 7 10:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80500000000000005,"15min":0.78100000000000003},"min":{"1min":0.56699999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1259999999999999,"5min":3.0979999999999999,"15min":3.0979999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85499999999999998,"15min":0.84799999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2669999999999999,"5min":2.383,"15min":2.383},"last":0.82799999999999996}]},{"osd":11,"last update":"Thu Dec 7 10:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.83999999999999997,"15min":0.81599999999999995},"min":{"1min":0.64600000000000002,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.89500000000000002,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.82199999999999995,"15min":0.80200000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.079,"5min":3.1749999999999998,"15min":3.1749999999999998},"last":0.83999999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1150976,"data_stored":591383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56599999999999995,"15min":0.51600000000000001},"min":{"1min":0.36799999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57199999999999995,"15min":0.52700000000000002},"min":{"1min":0.39200000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.86499999999999999,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.66700000000000004}]},{"osd":1,"last update":"Thu Dec 7 10:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.62,"15min":0.54100000000000004},"min":{"1min":0.41699999999999998,"5min":0.38900000000000001,"15min":0.34999999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.75},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.59499999999999997,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.77500000000000002}]},{"osd":3,"last update":"Thu Dec 7 10:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.628,"15min":0.56599999999999995},"min":{"1min":0.42599999999999999,"5min":0.41399999999999998,"15min":0.35799999999999998},"max":{"1min":0.92800000000000005,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.625,"15min":0.57099999999999995},"min":{"1min":0.44400000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86199999999999999,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.55100000000000005}]},{"osd":4,"last update":"Thu Dec 7 10:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71999999999999997,"15min":0.71099999999999997},"min":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.78400000000000003,"15min":0.78000000000000003},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.83199999999999996}]},{"osd":5,"last update":"Thu Dec 7 10:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.51200000000000001,"15min":0.45400000000000001},"min":{"1min":0.438,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.502},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.52600000000000002,"15min":0.47999999999999998},"min":{"1min":0.41999999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.80400000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.59499999999999997}]},{"osd":6,"last update":"Thu Dec 7 10:19:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68000000000000005,"15min":0.67000000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.69199999999999995,"15min":0.66000000000000003},"min":{"1min":0.57699999999999996,"5min":0.5,"15min":0.5},"max":{"1min":0.96699999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.72699999999999998}]},{"osd":7,"last update":"Thu Dec 7 10:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.73799999999999999,"15min":0.73399999999999999},"min":{"1min":0.53400000000000003,"5min":0.499,"15min":0.499},"max":{"1min":0.93100000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.51200000000000001,"5min":0.501,"15min":0.501},"max":{"1min":0.88800000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.85299999999999998}]},{"osd":8,"last update":"Thu Dec 7 10:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.65700000000000003,"15min":0.61499999999999999},"min":{"1min":0.52000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.70099999999999996,"15min":0.67100000000000004},"min":{"1min":0.56699999999999995,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.018,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.92300000000000004}]},{"osd":9,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79000000000000004,"15min":0.78400000000000003},"min":{"1min":0.64000000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.038,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.88},{"interface":"front","average":{"1min":0.873,"5min":0.83899999999999997,"15min":0.83399999999999996},"min":{"1min":0.65500000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0680000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.93300000000000005}]},{"osd":10,"last update":"Thu Dec 7 10:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.79500000000000004,"15min":0.76200000000000001},"min":{"1min":0.69399999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0389999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.80200000000000005,"15min":0.77400000000000002},"min":{"1min":0.68500000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0489999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.98199999999999998}]},{"osd":11,"last update":"Thu Dec 7 10:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.873,"15min":0.873},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0129999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.873,"5min":0.876,"15min":0.876},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.087,"5min":1.129,"15min":1.129},"last":0.94999999999999996}]}]},{"osd":3,"up_from":26,"seq":111669149775,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":511605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46100000000000002,"15min":0.40699999999999997},"min":{"1min":0.32200000000000001,"5min":0.28199999999999997,"15min":0.22500000000000001},"max":{"1min":0.79500000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.434,"5min":0.47699999999999998,"15min":0.41599999999999998},"min":{"1min":0.29599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.64500000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.42199999999999999}]},{"osd":1,"last update":"Thu Dec 7 10:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71999999999999997,"15min":0.59799999999999998},"min":{"1min":0.55700000000000005,"5min":0.47299999999999998,"15min":0.438},"max":{"1min":0.84899999999999998,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66600000000000004,"15min":0.57099999999999995},"min":{"1min":0.49399999999999999,"5min":0.41899999999999998,"15min":0.39900000000000002},"max":{"1min":0.81399999999999995,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.628}]},{"osd":2,"last update":"Thu Dec 7 10:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.442,"5min":0.39500000000000002,"15min":0.38300000000000001},"max":{"1min":0.76300000000000001,"5min":1.087,"15min":1.087},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.71599999999999997,"15min":0.63},"min":{"1min":0.57799999999999996,"5min":0.44,"15min":0.34699999999999998},"max":{"1min":0.89800000000000002,"5min":1.177,"15min":1.177},"last":0.57699999999999996}]},{"osd":4,"last update":"Thu Dec 7 10:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71199999999999997,"15min":0.629},"min":{"1min":0.50800000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.032,"5min":1.208,"15min":1.208},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65000000000000002,"15min":0.59599999999999997},"min":{"1min":0.46500000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.874,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.66700000000000004}]},{"osd":5,"last update":"Thu Dec 7 10:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.63400000000000001,"15min":0.61399999999999999},"min":{"1min":0.54000000000000004,"5min":0.372,"15min":0.372},"max":{"1min":0.877,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.72599999999999998,"15min":0.69999999999999996},"min":{"1min":0.49299999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0760000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.70699999999999996}]},{"osd":6,"last update":"Thu Dec 7 10:20:05 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.65100000000000002,"15min":0.64100000000000001},"min":{"1min":0.42099999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.99099999999999999,"5min":1.282,"15min":1.282},"last":0.748},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.58499999999999996,"15min":0.60999999999999999},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83399999999999996,"5min":1.292,"15min":1.292},"last":0.501}]},{"osd":7,"last update":"Thu Dec 7 10:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.79500000000000004,"15min":0.77200000000000002},"min":{"1min":0.61199999999999999,"5min":0.498,"15min":0.498},"max":{"1min":1.1140000000000001,"5min":1.323,"15min":1.323},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95299999999999996,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.73699999999999999}]},{"osd":8,"last update":"Thu Dec 7 10:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.872}]},{"osd":9,"last update":"Thu Dec 7 10:19:15 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.86299999999999999,"15min":0.88100000000000001},"min":{"1min":0.58699999999999997,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1479999999999999,"5min":1.242,"15min":1.242},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.78500000000000003,"15min":0.80500000000000005},"min":{"1min":0.53000000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0009999999999999,"5min":1.149,"15min":1.149},"last":0.80700000000000005}]},{"osd":10,"last update":"Thu Dec 7 10:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.76600000000000001,"15min":0.78600000000000003},"min":{"1min":0.53900000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.89000000000000001,"5min":1.335,"15min":1.335},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.90600000000000003,"15min":0.92300000000000004},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.91400000000000003}]},{"osd":11,"last update":"Thu Dec 7 10:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.873,"15min":0.871},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.77500000000000002,"15min":0.77400000000000002},"min":{"1min":0.63300000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.84099999999999997}]}]},{"osd":4,"up_from":34,"seq":146028888138,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":516943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.52100000000000002,"15min":0.49099999999999999},"min":{"1min":0.41499999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76500000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Dec 7 10:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.41999999999999998,"15min":0.39300000000000002},"min":{"1min":0.28199999999999997,"5min":0.215,"15min":0.215},"max":{"1min":0.58499999999999996,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.43099999999999999,"15min":0.39500000000000002},"min":{"1min":0.32800000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.61399999999999999,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.60099999999999998}]},{"osd":2,"last update":"Thu Dec 7 10:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.61899999999999999,"15min":0.56999999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85899999999999999,"5min":1.323,"15min":1.323},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60199999999999998,"15min":0.54900000000000004},"min":{"1min":0.47599999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79500000000000004,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.66200000000000003}]},{"osd":3,"last update":"Thu Dec 7 10:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60299999999999998,"15min":0.55000000000000004},"min":{"1min":0.45800000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.873,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.57299999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83799999999999997,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.79700000000000004}]},{"osd":5,"last update":"Thu Dec 7 10:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64500000000000002,"15min":0.59699999999999998},"min":{"1min":0.47699999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.85499999999999998,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63500000000000001,"15min":0.58999999999999997},"min":{"1min":0.48799999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.876,"5min":1.258,"15min":1.258},"last":0.69099999999999995}]},{"osd":6,"last update":"Thu Dec 7 10:19:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65300000000000002,"15min":0.65100000000000002},"min":{"1min":0.434,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90000000000000002,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65200000000000002,"15min":0.63800000000000001},"min":{"1min":0.49399999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85199999999999998,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.73499999999999999}]},{"osd":7,"last update":"Thu Dec 7 10:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56899999999999995,"15min":0.52400000000000002},"min":{"1min":0.40799999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.79800000000000004,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60599999999999998,"15min":0.56200000000000006},"min":{"1min":0.40100000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.96099999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.90300000000000002}]},{"osd":8,"last update":"Thu Dec 7 10:19:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.745,"15min":0.68999999999999995},"min":{"1min":0.58499999999999996,"5min":0.432,"15min":0.432},"max":{"1min":1.026,"5min":1.381,"15min":1.381},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.73599999999999999,"15min":0.68999999999999995},"min":{"1min":0.53700000000000003,"5min":0.502,"15min":0.502},"max":{"1min":1.0389999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.78300000000000003}]},{"osd":9,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.76100000000000001,"15min":0.76700000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.93500000000000005,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78000000000000003,"15min":0.76100000000000001},"min":{"1min":0.59399999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0289999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.996}]},{"osd":10,"last update":"Thu Dec 7 10:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.55100000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.94899999999999995,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.752,"15min":0.73999999999999999},"min":{"1min":0.53800000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97499999999999998,"5min":1.038,"15min":1.038},"last":0.83999999999999997}]},{"osd":11,"last update":"Thu Dec 7 10:19:44 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.876,"15min":0.876},"min":{"1min":0.68200000000000005,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.079,"5min":1.222,"15min":1.222},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82399999999999995,"15min":0.82299999999999995},"min":{"1min":0.67100000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0129999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.95299999999999996}]}]},{"osd":5,"up_from":39,"seq":167503724614,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1691648,"data_stored":1119625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 7 10:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.52100000000000002},"min":{"1min":0.44,"5min":0.318,"15min":0.318},"max":{"1min":0.76300000000000001,"5min":2.105,"15min":2.105},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55000000000000004,"15min":0.52300000000000002},"min":{"1min":0.35599999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.78100000000000003,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.42499999999999999}]},{"osd":1,"last update":"Thu Dec 7 10:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.58599999999999997,"15min":0.52400000000000002},"min":{"1min":0.38600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77500000000000002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.625},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.621,"15min":0.55100000000000005},"min":{"1min":0.33100000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.81299999999999994,"5min":3.0070000000000001,"15min":3.0070000000000001},"last":0.56000000000000005}]},{"osd":2,"last update":"Thu Dec 7 10:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.438,"15min":0.39000000000000001},"min":{"1min":0.30299999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.57799999999999996,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47299999999999998,"15min":0.40500000000000003},"min":{"1min":0.36499999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.68200000000000005,"5min":1.851,"15min":1.851},"last":0.495}]},{"osd":3,"last update":"Thu Dec 7 10:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59099999999999997,"15min":0.54300000000000004},"min":{"1min":0.35199999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82399999999999995,"5min":1.093,"15min":1.093},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60099999999999998,"15min":0.55900000000000005},"min":{"1min":0.437,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80300000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Dec 7 10:19:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65900000000000003,"15min":0.58199999999999996},"min":{"1min":0.51900000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.874,"5min":1.722,"15min":1.722},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64000000000000001,"15min":0.58599999999999997},"min":{"1min":0.47099999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.92000000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60899999999999999}]},{"osd":6,"last update":"Thu Dec 7 10:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.66000000000000003,"15min":0.59999999999999998},"min":{"1min":0.52200000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90200000000000002,"5min":1.272,"15min":1.272},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75,"15min":0.65000000000000002},"min":{"1min":0.623,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90100000000000002,"5min":5.9720000000000004,"15min":5.9720000000000004},"last":0.72699999999999998}]},{"osd":7,"last update":"Thu Dec 7 10:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67800000000000005,"15min":0.63200000000000001},"min":{"1min":0.47099999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88100000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67900000000000005,"15min":0.64300000000000002},"min":{"1min":0.499,"5min":0.437,"15min":0.437},"max":{"1min":0.83299999999999996,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.86799999999999999}]},{"osd":8,"last update":"Thu Dec 7 10:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.67600000000000005,"15min":0.63600000000000001},"min":{"1min":0.54300000000000004,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.96799999999999997,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.64100000000000001,"15min":0.60699999999999998},"min":{"1min":0.56100000000000005,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85299999999999998,"5min":1.135,"15min":1.135},"last":0.70999999999999996}]},{"osd":9,"last update":"Thu Dec 7 10:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.75700000000000001},"min":{"1min":0.56200000000000006,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.98499999999999999,"5min":4.532,"15min":4.532},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77300000000000002,"15min":0.76600000000000001},"min":{"1min":0.55000000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.01,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.75}]},{"osd":10,"last update":"Thu Dec 7 10:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.84399999999999997,"15min":0.83999999999999997},"min":{"1min":0.58199999999999996,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.026,"5min":2.927,"15min":2.927},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.86199999999999999,"15min":0.81100000000000005},"min":{"1min":0.59099999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.94899999999999995,"5min":5.7709999999999999,"15min":5.7709999999999999},"last":0.80900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":79877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":15823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":16674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":67279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-07T10:20:11.745 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-07T10:20:11.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.942+0000 7fda1bbff700 1 -- 172.21.15.119:0/3083651566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda14101140 msgr2=0x7fda141015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:11.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.942+0000 7fda1bbff700 1 --2- 172.21.15.119:0/3083651566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda14101140 0x7fda141015c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fda0c009f90 tx=0x7fda0c00b330 comp rx=0 tx=0).stop 2023-12-07T10:20:11.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.942+0000 7fda1bbff700 1 -- 172.21.15.119:0/3083651566 shutdown_connections 2023-12-07T10:20:11.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.942+0000 7fda1bbff700 1 --2- 172.21.15.119:0/3083651566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda14101140 0x7fda141015c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:11.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.942+0000 7fda1bbff700 1 --2- 172.21.15.119:0/3083651566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda140ffe20 0x7fda14100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:11.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.942+0000 7fda1bbff700 1 --2- 172.21.15.119:0/3083651566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda14100780 0x7fda14100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:11.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.942+0000 7fda1bbff700 1 -- 172.21.15.119:0/3083651566 >> 172.21.15.119:0/3083651566 conn(0x7fda140fb9c0 msgr2=0x7fda140fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:11.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.943+0000 7fda1bbff700 1 -- 172.21.15.119:0/3083651566 shutdown_connections 2023-12-07T10:20:11.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.943+0000 7fda1bbff700 1 -- 172.21.15.119:0/3083651566 wait complete. 2023-12-07T10:20:11.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.943+0000 7fda1bbff700 1 Processor -- start 2023-12-07T10:20:11.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.944+0000 7fda1bbff700 1 -- start start 2023-12-07T10:20:11.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.944+0000 7fda1bbff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda140ffe20 0x7fda14199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:11.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.944+0000 7fda1bbff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda14100780 0x7fda14199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:11.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.944+0000 7fda1bbff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda14101140 0x7fda141a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:11.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.944+0000 7fda1bbff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda141a1280 con 0x7fda14100780 2023-12-07T10:20:11.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.944+0000 7fda1bbff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda141a13c0 con 0x7fda140ffe20 2023-12-07T10:20:11.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.944+0000 7fda1bbff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda141a1550 con 0x7fda14101140 2023-12-07T10:20:11.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.944+0000 7fda1919a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda14100780 0x7fda14199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:11.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.944+0000 7fda1919a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda14100780 0x7fda14199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47406/0 (socket says 172.21.15.119:47406) 2023-12-07T10:20:11.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.944+0000 7fda1919a700 1 -- 172.21.15.119:0/3812667202 learned_addr learned my addr 172.21.15.119:0/3812667202 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:11.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.944+0000 7fda1999b700 1 --2- 172.21.15.119:0/3812667202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda140ffe20 0x7fda14199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:11.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.944+0000 7fda1a19c700 1 --2- 172.21.15.119:0/3812667202 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda14101140 0x7fda141a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:11.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.945+0000 7fda1919a700 1 -- 172.21.15.119:0/3812667202 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda14101140 msgr2=0x7fda141a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:11.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.945+0000 7fda1919a700 1 --2- 172.21.15.119:0/3812667202 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda14101140 0x7fda141a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:11.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.945+0000 7fda1919a700 1 -- 172.21.15.119:0/3812667202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda140ffe20 msgr2=0x7fda14199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:11.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.945+0000 7fda1919a700 1 --2- 172.21.15.119:0/3812667202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda140ffe20 0x7fda14199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:11.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.945+0000 7fda1919a700 1 -- 172.21.15.119:0/3812667202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0c009c40 con 0x7fda14100780 2023-12-07T10:20:11.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.945+0000 7fda1999b700 1 --2- 172.21.15.119:0/3812667202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda140ffe20 0x7fda14199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:11.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.945+0000 7fda1919a700 1 --2- 172.21.15.119:0/3812667202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda14100780 0x7fda14199c90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fda0400e9a0 tx=0x7fda0400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:11.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.945+0000 7fda0affd700 1 -- 172.21.15.119:0/3812667202 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda04009790 con 0x7fda14100780 2023-12-07T10:20:11.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.945+0000 7fda0affd700 1 -- 172.21.15.119:0/3812667202 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda0400ce80 con 0x7fda14100780 2023-12-07T10:20:11.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.945+0000 7fda1bbff700 1 -- 172.21.15.119:0/3812667202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda141a1830 con 0x7fda14100780 2023-12-07T10:20:11.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.945+0000 7fda0affd700 1 -- 172.21.15.119:0/3812667202 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda04018680 con 0x7fda14100780 2023-12-07T10:20:11.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.945+0000 7fda1bbff700 1 -- 172.21.15.119:0/3812667202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda141a1d40 con 0x7fda14100780 2023-12-07T10:20:11.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.946+0000 7fda1bbff700 1 -- 172.21.15.119:0/3812667202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda1410ad90 con 0x7fda14100780 2023-12-07T10:20:11.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.948+0000 7fda0affd700 1 -- 172.21.15.119:0/3812667202 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fda04010430 con 0x7fda14100780 2023-12-07T10:20:11.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.949+0000 7fda0affd700 1 --2- 172.21.15.119:0/3812667202 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fda000a1b10 0x7fda000a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:11.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.949+0000 7fda0affd700 1 -- 172.21.15.119:0/3812667202 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fda04014070 con 0x7fda14100780 2023-12-07T10:20:11.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.950+0000 7fda1999b700 1 --2- 172.21.15.119:0/3812667202 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fda000a1b10 0x7fda000a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:11.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.950+0000 7fda1999b700 1 --2- 172.21.15.119:0/3812667202 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fda000a1b10 0x7fda000a3fd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fda10005fd0 tx=0x7fda10009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:11.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:11.953+0000 7fda0affd700 1 -- 172.21.15.119:0/3812667202 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fda040522f0 con 0x7fda14100780 2023-12-07T10:20:12.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.155+0000 7fda1bbff700 1 -- 172.21.15.119:0/3812667202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda14068350 con 0x7fda14100780 2023-12-07T10:20:12.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.157+0000 7fda0affd700 1 -- 172.21.15.119:0/3812667202 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24715 (secure 0 0 0) 0x7fda0409a620 con 0x7fda14100780 2023-12-07T10:20:12.159 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:20:12.159 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":93,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","created":"2023-12-07T10:09:59.945863+0000","modified":"2023-12-07T10:19:02.800626+0000","last_up_change":"2023-12-07T10:16:50.929427+0000","last_in_change":"2023-12-07T10:16:33.558158+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T10:13:25.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-07T10:17:39.347813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-07T10:17:40.714501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-07T10:17:44.257760+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"56b5818e-e625-40f1-8838-3ddcf2d34ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6803","nonce":2141950133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6805","nonce":2141950133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6809","nonce":2141950133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6807","nonce":2141950133}]},"public_addr":"172.21.15.119:6803/2141950133","cluster_addr":"172.21.15.119:6805/2141950133","heartbeat_back_addr":"172.21.15.119:6809/2141950133","heartbeat_front_addr":"172.21.15.119:6807/2141950133","state":["exists","up"]},{"osd":1,"uuid":"36489a6a-5278-4802-9525-9871521722e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6801","nonce":3390875370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6803","nonce":3390875370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6807","nonce":3390875370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6805","nonce":3390875370}]},"public_addr":"172.21.15.153:6801/3390875370","cluster_addr":"172.21.15.153:6803/3390875370","heartbeat_back_addr":"172.21.15.153:6807/3390875370","heartbeat_front_addr":"172.21.15.153:6805/3390875370","state":["exists","up"]},{"osd":2,"uuid":"a937ad26-5e9b-4c8a-b65a-626a8feda6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6801","nonce":708101637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6803","nonce":708101637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6807","nonce":708101637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6805","nonce":708101637}]},"public_addr":"172.21.15.159:6801/708101637","cluster_addr":"172.21.15.159:6803/708101637","heartbeat_back_addr":"172.21.15.159:6807/708101637","heartbeat_front_addr":"172.21.15.159:6805/708101637","state":["exists","up"]},{"osd":3,"uuid":"26ef8300-aa07-48f6-b31d-8b0f2217ffcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6811","nonce":3228047257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6813","nonce":3228047257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6817","nonce":3228047257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6815","nonce":3228047257}]},"public_addr":"172.21.15.119:6811/3228047257","cluster_addr":"172.21.15.119:6813/3228047257","heartbeat_back_addr":"172.21.15.119:6817/3228047257","heartbeat_front_addr":"172.21.15.119:6815/3228047257","state":["exists","up"]},{"osd":4,"uuid":"000ee4b9-9896-4901-8cc3-c85fdfc8cbaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6809","nonce":4021998299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6811","nonce":4021998299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6815","nonce":4021998299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6813","nonce":4021998299}]},"public_addr":"172.21.15.153:6809/4021998299","cluster_addr":"172.21.15.153:6811/4021998299","heartbeat_back_addr":"172.21.15.153:6815/4021998299","heartbeat_front_addr":"172.21.15.153:6813/4021998299","state":["exists","up"]},{"osd":5,"uuid":"8b2fc167-262d-474f-89f3-47bd29424421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6809","nonce":1832540768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6811","nonce":1832540768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6815","nonce":1832540768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6813","nonce":1832540768}]},"public_addr":"172.21.15.159:6809/1832540768","cluster_addr":"172.21.15.159:6811/1832540768","heartbeat_back_addr":"172.21.15.159:6815/1832540768","heartbeat_front_addr":"172.21.15.159:6813/1832540768","state":["exists","up"]},{"osd":6,"uuid":"f9aa6749-46cf-4b2a-8114-cbd034219d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162436840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162436840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6825","nonce":1162436840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162436840}]},"public_addr":"172.21.15.119:6819/1162436840","cluster_addr":"172.21.15.119:6821/1162436840","heartbeat_back_addr":"172.21.15.119:6825/1162436840","heartbeat_front_addr":"172.21.15.119:6823/1162436840","state":["exists","up"]},{"osd":7,"uuid":"39216361-e1c4-4fef-86cb-a4707656b547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6817","nonce":3254653457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6819","nonce":3254653457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6823","nonce":3254653457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6821","nonce":3254653457}]},"public_addr":"172.21.15.153:6817/3254653457","cluster_addr":"172.21.15.153:6819/3254653457","heartbeat_back_addr":"172.21.15.153:6823/3254653457","heartbeat_front_addr":"172.21.15.153:6821/3254653457","state":["exists","up"]},{"osd":8,"uuid":"cceb6ebc-7c02-48ea-9d16-b7a016dd449b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6817","nonce":2600946380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6819","nonce":2600946380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6823","nonce":2600946380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6821","nonce":2600946380}]},"public_addr":"172.21.15.159:6817/2600946380","cluster_addr":"172.21.15.159:6819/2600946380","heartbeat_back_addr":"172.21.15.159:6823/2600946380","heartbeat_front_addr":"172.21.15.159:6821/2600946380","state":["exists","up"]},{"osd":9,"uuid":"f337a2e4-e065-4a9f-91d9-e3615de2d6f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6827","nonce":1940737285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6829","nonce":1940737285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6833","nonce":1940737285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6831","nonce":1940737285}]},"public_addr":"172.21.15.119:6827/1940737285","cluster_addr":"172.21.15.119:6829/1940737285","heartbeat_back_addr":"172.21.15.119:6833/1940737285","heartbeat_front_addr":"172.21.15.119:6831/1940737285","state":["exists","up"]},{"osd":10,"uuid":"3a3bc347-85f4-4878-9797-d9da96137e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6825","nonce":663834864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6827","nonce":663834864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6831","nonce":663834864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6829","nonce":663834864}]},"public_addr":"172.21.15.153:6825/663834864","cluster_addr":"172.21.15.153:6827/663834864","heartbeat_back_addr":"172.21.15.153:6831/663834864","heartbeat_front_addr":"172.21.15.153:6829/663834864","state":["exists","up"]},{"osd":11,"uuid":"a0fc5909-279d-4172-977c-2dc2da0a7e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6825","nonce":1262216348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6827","nonce":1262216348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6831","nonce":1262216348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6829","nonce":1262216348}]},"public_addr":"172.21.15.159:6825/1262216348","cluster_addr":"172.21.15.159:6827/1262216348","heartbeat_back_addr":"172.21.15.159:6831/1262216348","heartbeat_front_addr":"172.21.15.159:6829/1262216348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:37.488109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:59.543060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:21.381585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:47.112729+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:09.732150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:30.893729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:56.678710+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:19.004256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:40.388818+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:06.391734+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:28.394053+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:49.634348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1628639392":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6801/1355617847":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6800/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/362307836":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/128964558":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/2278813554":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/1411933614":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/1685698277":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6800/1355617847":"2023-12-08T10:11:00.585565+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T10:20:12.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.160+0000 7fda1bbff700 1 -- 172.21.15.119:0/3812667202 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fda000a1b10 msgr2=0x7fda000a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:12.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.160+0000 7fda1bbff700 1 --2- 172.21.15.119:0/3812667202 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fda000a1b10 0x7fda000a3fd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fda10005fd0 tx=0x7fda10009310 comp rx=0 tx=0).stop 2023-12-07T10:20:12.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.160+0000 7fda1bbff700 1 -- 172.21.15.119:0/3812667202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda14100780 msgr2=0x7fda14199c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:12.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.160+0000 7fda1bbff700 1 --2- 172.21.15.119:0/3812667202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda14100780 0x7fda14199c90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fda0400e9a0 tx=0x7fda0400ecb0 comp rx=0 tx=0).stop 2023-12-07T10:20:12.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.160+0000 7fda1bbff700 1 -- 172.21.15.119:0/3812667202 shutdown_connections 2023-12-07T10:20:12.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.160+0000 7fda1bbff700 1 --2- 172.21.15.119:0/3812667202 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda14101140 0x7fda141a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:12.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.160+0000 7fda1bbff700 1 --2- 172.21.15.119:0/3812667202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda140ffe20 0x7fda14199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:12.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.160+0000 7fda1bbff700 1 --2- 172.21.15.119:0/3812667202 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fda000a1b10 0x7fda000a3fd0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:12.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.160+0000 7fda1bbff700 1 --2- 172.21.15.119:0/3812667202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda14100780 0x7fda14199c90 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:12.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.160+0000 7fda1bbff700 1 -- 172.21.15.119:0/3812667202 >> 172.21.15.119:0/3812667202 conn(0x7fda140fb9c0 msgr2=0x7fda140fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:12.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.160+0000 7fda1bbff700 1 -- 172.21.15.119:0/3812667202 shutdown_connections 2023-12-07T10:20:12.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.160+0000 7fda1bbff700 1 -- 172.21.15.119:0/3812667202 wait complete. 2023-12-07T10:20:12.181 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-07T10:13:25.194366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-07T10:17:39.347813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-07T10:17:40.714501+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-07T10:17:44.257760+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-07T10:20:12.182 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-07T10:20:12.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.381+0000 7f604dfff700 1 -- 172.21.15.119:0/4024594392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60481020d0 msgr2=0x7f60481044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:12.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.381+0000 7f604dfff700 1 --2- 172.21.15.119:0/4024594392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60481020d0 0x7f60481044c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f60380098f0 tx=0x7f6038009c00 comp rx=0 tx=0).stop 2023-12-07T10:20:12.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.381+0000 7f604dfff700 1 -- 172.21.15.119:0/4024594392 shutdown_connections 2023-12-07T10:20:12.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.381+0000 7f604dfff700 1 --2- 172.21.15.119:0/4024594392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6048107330 0x7f6048109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:12.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.381+0000 7f604dfff700 1 --2- 172.21.15.119:0/4024594392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60481020d0 0x7f60481044c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:12.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.381+0000 7f604dfff700 1 --2- 172.21.15.119:0/4024594392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6048104a00 0x7f6048106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:12.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.381+0000 7f604dfff700 1 -- 172.21.15.119:0/4024594392 >> 172.21.15.119:0/4024594392 conn(0x7f60480fba20 msgr2=0x7f60480fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:12.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.381+0000 7f604dfff700 1 -- 172.21.15.119:0/4024594392 shutdown_connections 2023-12-07T10:20:12.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.381+0000 7f604dfff700 1 -- 172.21.15.119:0/4024594392 wait complete. 2023-12-07T10:20:12.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.382+0000 7f604dfff700 1 Processor -- start 2023-12-07T10:20:12.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.382+0000 7f604dfff700 1 -- start start 2023-12-07T10:20:12.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.382+0000 7f604dfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60481020d0 0x7f60481997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:12.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.382+0000 7f604dfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6048104a00 0x7f6048199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:12.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.382+0000 7f604dfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6048107330 0x7f60481a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:12.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.382+0000 7f604dfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60481a12f0 con 0x7f60481020d0 2023-12-07T10:20:12.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.382+0000 7f604dfff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60481a1430 con 0x7f6048104a00 2023-12-07T10:20:12.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.382+0000 7f604dfff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60481a15c0 con 0x7f6048107330 2023-12-07T10:20:12.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.382+0000 7f60477fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60481020d0 0x7f60481997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:12.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.382+0000 7f6046ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6048104a00 0x7f6048199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:12.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.382+0000 7f60477fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60481020d0 0x7f60481997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47422/0 (socket says 172.21.15.119:47422) 2023-12-07T10:20:12.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.382+0000 7f60477fe700 1 -- 172.21.15.119:0/2586042750 learned_addr learned my addr 172.21.15.119:0/2586042750 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:12.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.383+0000 7f6047fff700 1 --2- 172.21.15.119:0/2586042750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6048107330 0x7f60481a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:12.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.383+0000 7f6046ffd700 1 -- 172.21.15.119:0/2586042750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6048107330 msgr2=0x7f60481a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:12.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.383+0000 7f6046ffd700 1 --2- 172.21.15.119:0/2586042750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6048107330 0x7f60481a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:12.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.383+0000 7f6046ffd700 1 -- 172.21.15.119:0/2586042750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60481020d0 msgr2=0x7f60481997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:12.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.383+0000 7f6046ffd700 1 --2- 172.21.15.119:0/2586042750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60481020d0 0x7f60481997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:12.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.383+0000 7f6046ffd700 1 -- 172.21.15.119:0/2586042750 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6040009d00 con 0x7f6048104a00 2023-12-07T10:20:12.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.383+0000 7f6046ffd700 1 --2- 172.21.15.119:0/2586042750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6048104a00 0x7f6048199d00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f603c00e9f0 tx=0x7f603c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:12.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.383+0000 7f6044ff9700 1 -- 172.21.15.119:0/2586042750 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c00c9c0 con 0x7f6048104a00 2023-12-07T10:20:12.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.383+0000 7f6044ff9700 1 -- 172.21.15.119:0/2586042750 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f603c010430 con 0x7f6048104a00 2023-12-07T10:20:12.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.383+0000 7f604dfff700 1 -- 172.21.15.119:0/2586042750 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6038009520 con 0x7f6048104a00 2023-12-07T10:20:12.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.383+0000 7f6044ff9700 1 -- 172.21.15.119:0/2586042750 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c018680 con 0x7f6048104a00 2023-12-07T10:20:12.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.384+0000 7f604dfff700 1 -- 172.21.15.119:0/2586042750 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60481a1c20 con 0x7f6048104a00 2023-12-07T10:20:12.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.384+0000 7f604dfff700 1 -- 172.21.15.119:0/2586042750 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60480683c0 con 0x7f6048104a00 2023-12-07T10:20:12.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.386+0000 7f6044ff9700 1 -- 172.21.15.119:0/2586042750 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f603c010860 con 0x7f6048104a00 2023-12-07T10:20:12.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.388+0000 7f6044ff9700 1 --2- 172.21.15.119:0/2586042750 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f60300a1b10 0x7f60300a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:12.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.388+0000 7f6044ff9700 1 -- 172.21.15.119:0/2586042750 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f603c014070 con 0x7f6048104a00 2023-12-07T10:20:12.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.388+0000 7f60477fe700 1 --2- 172.21.15.119:0/2586042750 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f60300a1b10 0x7f60300a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:12.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.389+0000 7f60477fe700 1 --2- 172.21.15.119:0/2586042750 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f60300a1b10 0x7f60300a3fd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6038009fd0 tx=0x7f6038019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:12.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.390+0000 7f6044ff9700 1 -- 172.21.15.119:0/2586042750 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f603c096ca0 con 0x7f6048104a00 2023-12-07T10:20:12.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:12 smithi119 ceph-mon[97908]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:20:12.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:12 smithi119 ceph-mon[97908]: pgmap v358: 129 pgs: 129 active+clean; 660 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2023-12-07T10:20:12.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:12 smithi119 ceph-mon[97908]: from='client.25513 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:20:12.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:12 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3812667202' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:20:12.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:12 smithi159 ceph-mon[134726]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:20:12.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:12 smithi159 ceph-mon[134726]: pgmap v358: 129 pgs: 129 active+clean; 660 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2023-12-07T10:20:12.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:12 smithi159 ceph-mon[134726]: from='client.25513 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:20:12.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:12 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3812667202' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:20:12.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:12 smithi153 ceph-mon[133625]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:20:12.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:12 smithi153 ceph-mon[133625]: pgmap v358: 129 pgs: 129 active+clean; 660 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2023-12-07T10:20:12.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:12 smithi153 ceph-mon[133625]: from='client.25513 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:20:12.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:12 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3812667202' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:20:12.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.596+0000 7f604dfff700 1 -- 172.21.15.119:0/2586042750 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f604810cf90 con 0x7f6048104a00 2023-12-07T10:20:12.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.597+0000 7f6044ff9700 1 -- 172.21.15.119:0/2586042750 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f603c0b8e60 con 0x7f6048104a00 2023-12-07T10:20:12.598 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-12-07T10:20:12.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.599+0000 7f604dfff700 1 -- 172.21.15.119:0/2586042750 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f60300a1b10 msgr2=0x7f60300a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:12.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.599+0000 7f604dfff700 1 --2- 172.21.15.119:0/2586042750 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f60300a1b10 0x7f60300a3fd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6038009fd0 tx=0x7f6038019040 comp rx=0 tx=0).stop 2023-12-07T10:20:12.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.599+0000 7f604dfff700 1 -- 172.21.15.119:0/2586042750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6048104a00 msgr2=0x7f6048199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:12.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.599+0000 7f604dfff700 1 --2- 172.21.15.119:0/2586042750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6048104a00 0x7f6048199d00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f603c00e9f0 tx=0x7f603c00ed00 comp rx=0 tx=0).stop 2023-12-07T10:20:12.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.599+0000 7f604dfff700 1 -- 172.21.15.119:0/2586042750 shutdown_connections 2023-12-07T10:20:12.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.599+0000 7f604dfff700 1 --2- 172.21.15.119:0/2586042750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6048107330 0x7f60481a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:12.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.599+0000 7f604dfff700 1 --2- 172.21.15.119:0/2586042750 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f60300a1b10 0x7f60300a3fd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:12.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.599+0000 7f604dfff700 1 --2- 172.21.15.119:0/2586042750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60481020d0 0x7f60481997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:12.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.599+0000 7f604dfff700 1 --2- 172.21.15.119:0/2586042750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6048104a00 0x7f6048199d00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:12.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.599+0000 7f604dfff700 1 -- 172.21.15.119:0/2586042750 >> 172.21.15.119:0/2586042750 conn(0x7f60480fba20 msgr2=0x7f6048100160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:12.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.599+0000 7f604dfff700 1 -- 172.21.15.119:0/2586042750 shutdown_connections 2023-12-07T10:20:12.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.599+0000 7f604dfff700 1 -- 172.21.15.119:0/2586042750 wait complete. 2023-12-07T10:20:12.620 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-07T10:20:12.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.822+0000 7f01fdf9d700 1 -- 172.21.15.119:0/3891574137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f8101460 msgr2=0x7f01f8101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:12.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.822+0000 7f01fdf9d700 1 --2- 172.21.15.119:0/3891574137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f8101460 0x7f01f8101880 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f01ec0099c0 tx=0x7f01ec009cd0 comp rx=0 tx=0).stop 2023-12-07T10:20:12.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.822+0000 7f01fdf9d700 1 -- 172.21.15.119:0/3891574137 shutdown_connections 2023-12-07T10:20:12.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.822+0000 7f01fdf9d700 1 --2- 172.21.15.119:0/3891574137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01f8102ff0 0x7f01f81098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:12.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.822+0000 7f01fdf9d700 1 --2- 172.21.15.119:0/3891574137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f8101460 0x7f01f8101880 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:12.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.822+0000 7f01fdf9d700 1 --2- 172.21.15.119:0/3891574137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01f8102650 0x7f01f8102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:12.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.822+0000 7f01fdf9d700 1 -- 172.21.15.119:0/3891574137 >> 172.21.15.119:0/3891574137 conn(0x7f01f80fc9f0 msgr2=0x7f01f80fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:12.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.822+0000 7f01fdf9d700 1 -- 172.21.15.119:0/3891574137 shutdown_connections 2023-12-07T10:20:12.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.822+0000 7f01fdf9d700 1 -- 172.21.15.119:0/3891574137 wait complete. 2023-12-07T10:20:12.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.823+0000 7f01fdf9d700 1 Processor -- start 2023-12-07T10:20:12.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.824+0000 7f01fdf9d700 1 -- start start 2023-12-07T10:20:12.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.824+0000 7f01fdf9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f8101460 0x7f01f8199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:12.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.824+0000 7f01fdf9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01f8102650 0x7f01f8199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:12.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.824+0000 7f01fdf9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01f8102ff0 0x7f01f81a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:12.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.824+0000 7f01fdf9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f81a1170 con 0x7f01f8101460 2023-12-07T10:20:12.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.824+0000 7f01fdf9d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f81a12b0 con 0x7f01f8102650 2023-12-07T10:20:12.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.824+0000 7f01fdf9d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f81a1440 con 0x7f01f8102ff0 2023-12-07T10:20:12.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.824+0000 7f01fcf9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f8101460 0x7f01f8199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:12.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.824+0000 7f01fcf9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f8101460 0x7f01f8199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39190/0 (socket says 172.21.15.119:39190) 2023-12-07T10:20:12.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.824+0000 7f01fcf9b700 1 -- 172.21.15.119:0/691671912 learned_addr learned my addr 172.21.15.119:0/691671912 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:12.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.824+0000 7f01fd79c700 1 --2- 172.21.15.119:0/691671912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01f8102ff0 0x7f01f81a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:12.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.824+0000 7f01f7fff700 1 --2- 172.21.15.119:0/691671912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01f8102650 0x7f01f8199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:12.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.825+0000 7f01fcf9b700 1 -- 172.21.15.119:0/691671912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01f8102ff0 msgr2=0x7f01f81a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:12.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.825+0000 7f01fcf9b700 1 --2- 172.21.15.119:0/691671912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01f8102ff0 0x7f01f81a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:12.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.825+0000 7f01fcf9b700 1 -- 172.21.15.119:0/691671912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01f8102650 msgr2=0x7f01f8199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:12.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.825+0000 7f01fcf9b700 1 --2- 172.21.15.119:0/691671912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01f8102650 0x7f01f8199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:12.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.825+0000 7f01fcf9b700 1 -- 172.21.15.119:0/691671912 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec0095f0 con 0x7f01f8101460 2023-12-07T10:20:12.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.825+0000 7f01fcf9b700 1 --2- 172.21.15.119:0/691671912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f8101460 0x7f01f8199690 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f01ec006010 tx=0x7f01ec00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:12.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.825+0000 7f01f5ffb700 1 -- 172.21.15.119:0/691671912 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec01d070 con 0x7f01f8101460 2023-12-07T10:20:12.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.825+0000 7f01f5ffb700 1 -- 172.21.15.119:0/691671912 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01ec005020 con 0x7f01f8101460 2023-12-07T10:20:12.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.825+0000 7f01fdf9d700 1 -- 172.21.15.119:0/691671912 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f81a16c0 con 0x7f01f8101460 2023-12-07T10:20:12.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.825+0000 7f01f5ffb700 1 -- 172.21.15.119:0/691671912 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec004b10 con 0x7f01f8101460 2023-12-07T10:20:12.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.825+0000 7f01fdf9d700 1 -- 172.21.15.119:0/691671912 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f81a1ae0 con 0x7f01f8101460 2023-12-07T10:20:12.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.826+0000 7f01fdf9d700 1 -- 172.21.15.119:0/691671912 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f8068200 con 0x7f01f8101460 2023-12-07T10:20:12.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.828+0000 7f01f5ffb700 1 -- 172.21.15.119:0/691671912 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f01ec005190 con 0x7f01f8101460 2023-12-07T10:20:12.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.830+0000 7f01f5ffb700 1 --2- 172.21.15.119:0/691671912 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f01e80a1b90 0x7f01e80a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:12.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.830+0000 7f01f5ffb700 1 -- 172.21.15.119:0/691671912 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f01ec0ce2d0 con 0x7f01f8101460 2023-12-07T10:20:12.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.830+0000 7f01f7fff700 1 --2- 172.21.15.119:0/691671912 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f01e80a1b90 0x7f01e80a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:12.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.830+0000 7f01f7fff700 1 --2- 172.21.15.119:0/691671912 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f01e80a1b90 0x7f01e80a4050 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009c60 tx=0x7f01e4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:12.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:12.834+0000 7f01f5ffb700 1 -- 172.21.15.119:0/691671912 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f01ec0973b0 con 0x7f01f8101460 2023-12-07T10:20:13.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.037+0000 7f01fdf9d700 1 -- 172.21.15.119:0/691671912 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f01f81a1d60 con 0x7f01f8101460 2023-12-07T10:20:13.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.038+0000 7f01f5ffb700 1 -- 172.21.15.119:0/691671912 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f01ec09a9d0 con 0x7f01f8101460 2023-12-07T10:20:13.040 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-12-07T10:20:13.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.040+0000 7f01fdf9d700 1 -- 172.21.15.119:0/691671912 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f01e80a1b90 msgr2=0x7f01e80a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:13.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.040+0000 7f01fdf9d700 1 --2- 172.21.15.119:0/691671912 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f01e80a1b90 0x7f01e80a4050 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009c60 tx=0x7f01e4009260 comp rx=0 tx=0).stop 2023-12-07T10:20:13.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.040+0000 7f01fdf9d700 1 -- 172.21.15.119:0/691671912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f8101460 msgr2=0x7f01f8199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:13.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.040+0000 7f01fdf9d700 1 --2- 172.21.15.119:0/691671912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f8101460 0x7f01f8199690 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f01ec006010 tx=0x7f01ec00f690 comp rx=0 tx=0).stop 2023-12-07T10:20:13.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.040+0000 7f01fdf9d700 1 -- 172.21.15.119:0/691671912 shutdown_connections 2023-12-07T10:20:13.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.040+0000 7f01fdf9d700 1 --2- 172.21.15.119:0/691671912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01f8102ff0 0x7f01f81a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.040+0000 7f01fdf9d700 1 --2- 172.21.15.119:0/691671912 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f01e80a1b90 0x7f01e80a4050 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.040+0000 7f01fdf9d700 1 --2- 172.21.15.119:0/691671912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f8101460 0x7f01f8199690 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.040+0000 7f01fdf9d700 1 --2- 172.21.15.119:0/691671912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01f8102650 0x7f01f8199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.040+0000 7f01fdf9d700 1 -- 172.21.15.119:0/691671912 >> 172.21.15.119:0/691671912 conn(0x7f01f80fc9f0 msgr2=0x7f01f8073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:13.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.040+0000 7f01fdf9d700 1 -- 172.21.15.119:0/691671912 shutdown_connections 2023-12-07T10:20:13.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.041+0000 7f01fdf9d700 1 -- 172.21.15.119:0/691671912 wait complete. 2023-12-07T10:20:13.061 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-07T10:20:13.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.263+0000 7fdf081ff700 1 -- 172.21.15.119:0/3869783993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf001015c0 msgr2=0x7fdf001019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:13.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.263+0000 7fdf081ff700 1 --2- 172.21.15.119:0/3869783993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf001015c0 0x7fdf001019e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdefc0099c0 tx=0x7fdefc009cd0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.264+0000 7fdf081ff700 1 -- 172.21.15.119:0/3869783993 shutdown_connections 2023-12-07T10:20:13.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.264+0000 7fdf081ff700 1 --2- 172.21.15.119:0/3869783993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf00103150 0x7fdf00109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.264+0000 7fdf081ff700 1 --2- 172.21.15.119:0/3869783993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf001015c0 0x7fdf001019e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.264+0000 7fdf081ff700 1 --2- 172.21.15.119:0/3869783993 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf001027b0 0x7fdf00102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.264+0000 7fdf081ff700 1 -- 172.21.15.119:0/3869783993 >> 172.21.15.119:0/3869783993 conn(0x7fdf000fcb50 msgr2=0x7fdf000fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:13.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.264+0000 7fdf081ff700 1 -- 172.21.15.119:0/3869783993 shutdown_connections 2023-12-07T10:20:13.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.264+0000 7fdf081ff700 1 -- 172.21.15.119:0/3869783993 wait complete. 2023-12-07T10:20:13.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.265+0000 7fdf081ff700 1 Processor -- start 2023-12-07T10:20:13.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.265+0000 7fdf081ff700 1 -- start start 2023-12-07T10:20:13.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.265+0000 7fdf081ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf001015c0 0x7fdf001997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:13.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.265+0000 7fdf081ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf001027b0 0x7fdf00199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:13.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.265+0000 7fdf081ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf00103150 0x7fdf001a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:13.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.265+0000 7fdf081ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf001a12e0 con 0x7fdf00103150 2023-12-07T10:20:13.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.265+0000 7fdf081ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf001a1420 con 0x7fdf001027b0 2023-12-07T10:20:13.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.265+0000 7fdf081ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf001a15b0 con 0x7fdf001015c0 2023-12-07T10:20:13.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.265+0000 7fdf0679c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf00103150 0x7fdf001a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:13.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.265+0000 7fdf05f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf001015c0 0x7fdf001997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:13.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.265+0000 7fdf0679c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf00103150 0x7fdf001a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39198/0 (socket says 172.21.15.119:39198) 2023-12-07T10:20:13.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.265+0000 7fdf0679c700 1 -- 172.21.15.119:0/1217000359 learned_addr learned my addr 172.21.15.119:0/1217000359 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:13.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.265+0000 7fdf0579a700 1 --2- 172.21.15.119:0/1217000359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf001027b0 0x7fdf00199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:13.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.266+0000 7fdf0679c700 1 -- 172.21.15.119:0/1217000359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf001015c0 msgr2=0x7fdf001997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:13.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.266+0000 7fdf0679c700 1 --2- 172.21.15.119:0/1217000359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf001015c0 0x7fdf001997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.266+0000 7fdf0679c700 1 -- 172.21.15.119:0/1217000359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf001027b0 msgr2=0x7fdf00199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:13.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.266+0000 7fdf0679c700 1 --2- 172.21.15.119:0/1217000359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf001027b0 0x7fdf00199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.266+0000 7fdf0679c700 1 -- 172.21.15.119:0/1217000359 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc0095f0 con 0x7fdf00103150 2023-12-07T10:20:13.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.266+0000 7fdf0679c700 1 --2- 172.21.15.119:0/1217000359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf00103150 0x7fdf001a0d10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdef800ef30 tx=0x7fdef800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:13.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.266+0000 7fdef6ffd700 1 -- 172.21.15.119:0/1217000359 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef8018070 con 0x7fdf00103150 2023-12-07T10:20:13.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.266+0000 7fdef6ffd700 1 -- 172.21.15.119:0/1217000359 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdef800f040 con 0x7fdf00103150 2023-12-07T10:20:13.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.266+0000 7fdf081ff700 1 -- 172.21.15.119:0/1217000359 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf001a1830 con 0x7fdf00103150 2023-12-07T10:20:13.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.266+0000 7fdef6ffd700 1 -- 172.21.15.119:0/1217000359 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef800cd00 con 0x7fdf00103150 2023-12-07T10:20:13.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.266+0000 7fdf081ff700 1 -- 172.21.15.119:0/1217000359 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf001a1d40 con 0x7fdf00103150 2023-12-07T10:20:13.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.267+0000 7fdf081ff700 1 -- 172.21.15.119:0/1217000359 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf00068810 con 0x7fdf00103150 2023-12-07T10:20:13.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.269+0000 7fdef6ffd700 1 -- 172.21.15.119:0/1217000359 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fdef8003a00 con 0x7fdf00103150 2023-12-07T10:20:13.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.272+0000 7fdef6ffd700 1 --2- 172.21.15.119:0/1217000359 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fdeec0a1b90 0x7fdeec0a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:13.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.272+0000 7fdef6ffd700 1 -- 172.21.15.119:0/1217000359 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fdef80cd490 con 0x7fdf00103150 2023-12-07T10:20:13.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.272+0000 7fdf05f9b700 1 --2- 172.21.15.119:0/1217000359 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fdeec0a1b90 0x7fdeec0a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:13.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.272+0000 7fdf05f9b700 1 --2- 172.21.15.119:0/1217000359 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fdeec0a1b90 0x7fdeec0a4050 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdefc003730 tx=0x7fdefc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:13.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.276+0000 7fdef6ffd700 1 -- 172.21.15.119:0/1217000359 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fdef8096570 con 0x7fdf00103150 2023-12-07T10:20:13.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:13 smithi119 ceph-mon[97908]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:20:13.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:13 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2586042750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:20:13.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:13 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/691671912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:20:13.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.479+0000 7fdf081ff700 1 -- 172.21.15.119:0/1217000359 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdf001a2030 con 0x7fdf00103150 2023-12-07T10:20:13.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.480+0000 7fdef6ffd700 1 -- 172.21.15.119:0/1217000359 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdef8099b90 con 0x7fdf00103150 2023-12-07T10:20:13.481 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-12-07T10:20:13.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.482+0000 7fdf081ff700 1 -- 172.21.15.119:0/1217000359 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fdeec0a1b90 msgr2=0x7fdeec0a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:13.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.482+0000 7fdf081ff700 1 --2- 172.21.15.119:0/1217000359 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fdeec0a1b90 0x7fdeec0a4050 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdefc003730 tx=0x7fdefc000bc0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.482+0000 7fdf081ff700 1 -- 172.21.15.119:0/1217000359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf00103150 msgr2=0x7fdf001a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:13.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.482+0000 7fdf081ff700 1 --2- 172.21.15.119:0/1217000359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf00103150 0x7fdf001a0d10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdef800ef30 tx=0x7fdef800c3a0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.483+0000 7fdf081ff700 1 -- 172.21.15.119:0/1217000359 shutdown_connections 2023-12-07T10:20:13.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.483+0000 7fdf081ff700 1 --2- 172.21.15.119:0/1217000359 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fdeec0a1b90 0x7fdeec0a4050 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.483+0000 7fdf081ff700 1 --2- 172.21.15.119:0/1217000359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf00103150 0x7fdf001a0d10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.483+0000 7fdf081ff700 1 --2- 172.21.15.119:0/1217000359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf001015c0 0x7fdf001997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.483+0000 7fdf081ff700 1 --2- 172.21.15.119:0/1217000359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf001027b0 0x7fdf00199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.483+0000 7fdf081ff700 1 -- 172.21.15.119:0/1217000359 >> 172.21.15.119:0/1217000359 conn(0x7fdf000fcb50 msgr2=0x7fdf00073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:13.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.483+0000 7fdf081ff700 1 -- 172.21.15.119:0/1217000359 shutdown_connections 2023-12-07T10:20:13.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.483+0000 7fdf081ff700 1 -- 172.21.15.119:0/1217000359 wait complete. 2023-12-07T10:20:13.503 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-07T10:20:13.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:13 smithi159 ceph-mon[134726]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:20:13.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:13 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2586042750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:20:13.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:13 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/691671912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:20:13.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:13 smithi153 ceph-mon[133625]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-07T10:20:13.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:13 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2586042750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:20:13.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:13 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/691671912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:20:13.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.706+0000 7ff6a53ff700 1 -- 172.21.15.119:0/588106910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6a0104960 msgr2=0x7ff6a0106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:13.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.706+0000 7ff6a53ff700 1 --2- 172.21.15.119:0/588106910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6a0104960 0x7ff6a0106d50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff688009990 tx=0x7ff688009ca0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.706+0000 7ff6a53ff700 1 -- 172.21.15.119:0/588106910 shutdown_connections 2023-12-07T10:20:13.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.706+0000 7ff6a53ff700 1 --2- 172.21.15.119:0/588106910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6a0107290 0x7ff6a0109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.706+0000 7ff6a53ff700 1 --2- 172.21.15.119:0/588106910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff6a0102030 0x7ff6a0104420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.706+0000 7ff6a53ff700 1 --2- 172.21.15.119:0/588106910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6a0104960 0x7ff6a0106d50 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.706+0000 7ff6a53ff700 1 -- 172.21.15.119:0/588106910 >> 172.21.15.119:0/588106910 conn(0x7ff6a00fba20 msgr2=0x7ff6a00fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:13.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.706+0000 7ff6a53ff700 1 -- 172.21.15.119:0/588106910 shutdown_connections 2023-12-07T10:20:13.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.706+0000 7ff6a53ff700 1 -- 172.21.15.119:0/588106910 wait complete. 2023-12-07T10:20:13.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.707+0000 7ff6a53ff700 1 Processor -- start 2023-12-07T10:20:13.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.707+0000 7ff6a53ff700 1 -- start start 2023-12-07T10:20:13.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.708+0000 7ff6a53ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff6a0102030 0x7ff6a0101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:13.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.708+0000 7ff6a53ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6a0104960 0x7ff6a00fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:13.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.708+0000 7ff6a53ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6a0107290 0x7ff6a0100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:13.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.708+0000 7ff6a53ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a0100cc0 con 0x7ff6a0107290 2023-12-07T10:20:13.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.708+0000 7ff6a53ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a0100e30 con 0x7ff6a0104960 2023-12-07T10:20:13.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.708+0000 7ff6a53ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a01a15a0 con 0x7ff6a0102030 2023-12-07T10:20:13.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.708+0000 7ff69f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6a0107290 0x7ff6a0100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:13.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.708+0000 7ff69f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6a0107290 0x7ff6a0100520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39228/0 (socket says 172.21.15.119:39228) 2023-12-07T10:20:13.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.708+0000 7ff69f7fe700 1 -- 172.21.15.119:0/2319005423 learned_addr learned my addr 172.21.15.119:0/2319005423 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:13.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.708+0000 7ff69effd700 1 --2- 172.21.15.119:0/2319005423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff6a0102030 0x7ff6a0101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:13.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.708+0000 7ff697fff700 1 --2- 172.21.15.119:0/2319005423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6a0104960 0x7ff6a00fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:13.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.708+0000 7ff69f7fe700 1 -- 172.21.15.119:0/2319005423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff6a0102030 msgr2=0x7ff6a0101950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:13.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.708+0000 7ff69f7fe700 1 --2- 172.21.15.119:0/2319005423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff6a0102030 0x7ff6a0101950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.708+0000 7ff69f7fe700 1 -- 172.21.15.119:0/2319005423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6a0104960 msgr2=0x7ff6a00fffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:13.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.708+0000 7ff69f7fe700 1 --2- 172.21.15.119:0/2319005423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6a0104960 0x7ff6a00fffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.708+0000 7ff69f7fe700 1 -- 172.21.15.119:0/2319005423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff690009520 con 0x7ff6a0107290 2023-12-07T10:20:13.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.709+0000 7ff69f7fe700 1 --2- 172.21.15.119:0/2319005423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6a0107290 0x7ff6a0100520 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff69800ef30 tx=0x7ff69800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:13.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.709+0000 7ff69cff9700 1 -- 172.21.15.119:0/2319005423 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff698018070 con 0x7ff6a0107290 2023-12-07T10:20:13.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.709+0000 7ff69cff9700 1 -- 172.21.15.119:0/2319005423 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff69800f040 con 0x7ff6a0107290 2023-12-07T10:20:13.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.709+0000 7ff69cff9700 1 -- 172.21.15.119:0/2319005423 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69800cca0 con 0x7ff6a0107290 2023-12-07T10:20:13.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.709+0000 7ff6a53ff700 1 -- 172.21.15.119:0/2319005423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6880095f0 con 0x7ff6a0107290 2023-12-07T10:20:13.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.709+0000 7ff6a53ff700 1 -- 172.21.15.119:0/2319005423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a01a1b00 con 0x7ff6a0107290 2023-12-07T10:20:13.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.710+0000 7ff6a53ff700 1 -- 172.21.15.119:0/2319005423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a00fd5a0 con 0x7ff6a0107290 2023-12-07T10:20:13.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.712+0000 7ff69cff9700 1 -- 172.21.15.119:0/2319005423 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7ff698003a00 con 0x7ff6a0107290 2023-12-07T10:20:13.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.713+0000 7ff69cff9700 1 --2- 172.21.15.119:0/2319005423 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff68c0a1b10 0x7ff68c0a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:13.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.713+0000 7ff69cff9700 1 -- 172.21.15.119:0/2319005423 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7ff6980cd480 con 0x7ff6a0107290 2023-12-07T10:20:13.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.713+0000 7ff69effd700 1 --2- 172.21.15.119:0/2319005423 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff68c0a1b10 0x7ff68c0a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:13.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.714+0000 7ff69effd700 1 --2- 172.21.15.119:0/2319005423 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff68c0a1b10 0x7ff68c0a3fd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff690009df0 tx=0x7ff690009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:13.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.717+0000 7ff69cff9700 1 -- 172.21.15.119:0/2319005423 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff6980965c0 con 0x7ff6a0107290 2023-12-07T10:20:13.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.923+0000 7ff6a53ff700 1 -- 172.21.15.119:0/2319005423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff6a00fcf80 con 0x7ff6a0107290 2023-12-07T10:20:13.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.923+0000 7ff69cff9700 1 -- 172.21.15.119:0/2319005423 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff698099be0 con 0x7ff6a0107290 2023-12-07T10:20:13.925 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-12-07T10:20:13.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.926+0000 7ff6a53ff700 1 -- 172.21.15.119:0/2319005423 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff68c0a1b10 msgr2=0x7ff68c0a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:13.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.926+0000 7ff6a53ff700 1 --2- 172.21.15.119:0/2319005423 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff68c0a1b10 0x7ff68c0a3fd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff690009df0 tx=0x7ff690009260 comp rx=0 tx=0).stop 2023-12-07T10:20:13.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.926+0000 7ff6a53ff700 1 -- 172.21.15.119:0/2319005423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6a0107290 msgr2=0x7ff6a0100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:13.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.926+0000 7ff6a53ff700 1 --2- 172.21.15.119:0/2319005423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6a0107290 0x7ff6a0100520 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff69800ef30 tx=0x7ff69800c3a0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.926+0000 7ff6a53ff700 1 -- 172.21.15.119:0/2319005423 shutdown_connections 2023-12-07T10:20:13.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.926+0000 7ff6a53ff700 1 --2- 172.21.15.119:0/2319005423 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff68c0a1b10 0x7ff68c0a3fd0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.926+0000 7ff6a53ff700 1 --2- 172.21.15.119:0/2319005423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6a0107290 0x7ff6a0100520 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.926+0000 7ff6a53ff700 1 --2- 172.21.15.119:0/2319005423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff6a0102030 0x7ff6a0101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.926+0000 7ff6a53ff700 1 --2- 172.21.15.119:0/2319005423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6a0104960 0x7ff6a00fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:13.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.926+0000 7ff6a53ff700 1 -- 172.21.15.119:0/2319005423 >> 172.21.15.119:0/2319005423 conn(0x7ff6a00fba20 msgr2=0x7ff6a0103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:13.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.926+0000 7ff6a53ff700 1 -- 172.21.15.119:0/2319005423 shutdown_connections 2023-12-07T10:20:13.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:13.926+0000 7ff6a53ff700 1 -- 172.21.15.119:0/2319005423 wait complete. 2023-12-07T10:20:13.946 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:20:14.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.140+0000 7fa072fff700 1 -- 172.21.15.119:0/618592618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa06c102750 msgr2=0x7fa06c102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:14.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.140+0000 7fa072fff700 1 --2- 172.21.15.119:0/618592618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa06c102750 0x7fa06c102bb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa0580099c0 tx=0x7fa058009cd0 comp rx=0 tx=0).stop 2023-12-07T10:20:14.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.140+0000 7fa072fff700 1 -- 172.21.15.119:0/618592618 shutdown_connections 2023-12-07T10:20:14.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.140+0000 7fa072fff700 1 --2- 172.21.15.119:0/618592618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa06c1030f0 0x7fa06c1099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:14.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.140+0000 7fa072fff700 1 --2- 172.21.15.119:0/618592618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa06c101560 0x7fa06c101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:14.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.140+0000 7fa072fff700 1 --2- 172.21.15.119:0/618592618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa06c102750 0x7fa06c102bb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:14.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.140+0000 7fa072fff700 1 -- 172.21.15.119:0/618592618 >> 172.21.15.119:0/618592618 conn(0x7fa06c0fcab0 msgr2=0x7fa06c0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:14.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.140+0000 7fa072fff700 1 -- 172.21.15.119:0/618592618 shutdown_connections 2023-12-07T10:20:14.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.140+0000 7fa072fff700 1 -- 172.21.15.119:0/618592618 wait complete. 2023-12-07T10:20:14.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.141+0000 7fa072fff700 1 Processor -- start 2023-12-07T10:20:14.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.141+0000 7fa072fff700 1 -- start start 2023-12-07T10:20:14.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.142+0000 7fa072fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa06c101560 0x7fa06c199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:14.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.142+0000 7fa072fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa06c102750 0x7fa06c199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:14.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.142+0000 7fa072fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa06c1030f0 0x7fa06c1a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:14.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.142+0000 7fa072fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c1a1230 con 0x7fa06c101560 2023-12-07T10:20:14.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.142+0000 7fa072fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c1a1370 con 0x7fa06c1030f0 2023-12-07T10:20:14.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.142+0000 7fa072fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c1a1500 con 0x7fa06c102750 2023-12-07T10:20:14.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.142+0000 7fa070d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa06c101560 0x7fa06c199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:14.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.142+0000 7fa070d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa06c101560 0x7fa06c199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39248/0 (socket says 172.21.15.119:39248) 2023-12-07T10:20:14.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.142+0000 7fa070d9b700 1 -- 172.21.15.119:0/1306722163 learned_addr learned my addr 172.21.15.119:0/1306722163 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:14.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.142+0000 7fa06bfff700 1 --2- 172.21.15.119:0/1306722163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa06c102750 0x7fa06c199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:14.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.142+0000 7fa07159c700 1 --2- 172.21.15.119:0/1306722163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa06c1030f0 0x7fa06c1a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:14.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.142+0000 7fa070d9b700 1 -- 172.21.15.119:0/1306722163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa06c102750 msgr2=0x7fa06c199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:14.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.142+0000 7fa070d9b700 1 --2- 172.21.15.119:0/1306722163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa06c102750 0x7fa06c199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:14.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.142+0000 7fa070d9b700 1 -- 172.21.15.119:0/1306722163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa06c1030f0 msgr2=0x7fa06c1a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:14.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.142+0000 7fa070d9b700 1 --2- 172.21.15.119:0/1306722163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa06c1030f0 0x7fa06c1a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:14.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.142+0000 7fa070d9b700 1 -- 172.21.15.119:0/1306722163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0580095f0 con 0x7fa06c101560 2023-12-07T10:20:14.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.142+0000 7fa07159c700 1 --2- 172.21.15.119:0/1306722163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa06c1030f0 0x7fa06c1a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:20:14.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.143+0000 7fa070d9b700 1 --2- 172.21.15.119:0/1306722163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa06c101560 0x7fa06c199750 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa06000d980 tx=0x7fa06000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:14.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.143+0000 7fa069ffb700 1 -- 172.21.15.119:0/1306722163 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06000b950 con 0x7fa06c101560 2023-12-07T10:20:14.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.143+0000 7fa069ffb700 1 -- 172.21.15.119:0/1306722163 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa060007620 con 0x7fa06c101560 2023-12-07T10:20:14.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.143+0000 7fa069ffb700 1 -- 172.21.15.119:0/1306722163 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06000f3c0 con 0x7fa06c101560 2023-12-07T10:20:14.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.143+0000 7fa072fff700 1 -- 172.21.15.119:0/1306722163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06c1a17e0 con 0x7fa06c101560 2023-12-07T10:20:14.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.143+0000 7fa072fff700 1 -- 172.21.15.119:0/1306722163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa06c1a1cf0 con 0x7fa06c101560 2023-12-07T10:20:14.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.144+0000 7fa072fff700 1 -- 172.21.15.119:0/1306722163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa06c068350 con 0x7fa06c101560 2023-12-07T10:20:14.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.146+0000 7fa069ffb700 1 -- 172.21.15.119:0/1306722163 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fa060010640 con 0x7fa06c101560 2023-12-07T10:20:14.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.147+0000 7fa069ffb700 1 --2- 172.21.15.119:0/1306722163 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa05c0a1b90 0x7fa05c0a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:14.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.147+0000 7fa069ffb700 1 -- 172.21.15.119:0/1306722163 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fa0600cd790 con 0x7fa06c101560 2023-12-07T10:20:14.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.147+0000 7fa06bfff700 1 --2- 172.21.15.119:0/1306722163 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa05c0a1b90 0x7fa05c0a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:14.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.148+0000 7fa06bfff700 1 --2- 172.21.15.119:0/1306722163 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa05c0a1b90 0x7fa05c0a4050 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa058006010 tx=0x7fa05800b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:14.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.151+0000 7fa069ffb700 1 -- 172.21.15.119:0/1306722163 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa0600967e0 con 0x7fa06c101560 2023-12-07T10:20:14.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.400+0000 7fa072fff700 1 -- 172.21.15.119:0/1306722163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa06c10cff0 con 0x7fa06c101560 2023-12-07T10:20:14.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.401+0000 7fa069ffb700 1 -- 172.21.15.119:0/1306722163 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11385 (secure 0 0 0) 0x7fa060099e00 con 0x7fa06c101560 2023-12-07T10:20:14.403 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:20:14.403 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-07T10:17:41.278311+0000","modified":"2023-12-07T10:18:18.543356+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24517,"mds_3":24505,"mds_4":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14535":{"gid":14535,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6841/1791998028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1791998028},{"type":"v1","addr":"172.21.15.119:6841","nonce":1791998028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24505":{"gid":24505,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/2925503246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2925503246},{"type":"v1","addr":"172.21.15.153:6837","nonce":2925503246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6839/3984962900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3984962900},{"type":"v1","addr":"172.21.15.159:6839","nonce":3984962900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.119:6839/1920762602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1920762602},{"type":"v1","addr":"172.21.15.119:6839","nonce":1920762602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/566825184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":566825184},{"type":"v1","addr":"172.21.15.153:6839","nonce":566825184}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-07T10:20:14.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.404+0000 7fa072fff700 1 -- 172.21.15.119:0/1306722163 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa05c0a1b90 msgr2=0x7fa05c0a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:14.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.404+0000 7fa072fff700 1 --2- 172.21.15.119:0/1306722163 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa05c0a1b90 0x7fa05c0a4050 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa058006010 tx=0x7fa05800b540 comp rx=0 tx=0).stop 2023-12-07T10:20:14.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.404+0000 7fa072fff700 1 -- 172.21.15.119:0/1306722163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa06c101560 msgr2=0x7fa06c199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:14.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.404+0000 7fa072fff700 1 --2- 172.21.15.119:0/1306722163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa06c101560 0x7fa06c199750 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa06000d980 tx=0x7fa06000dc90 comp rx=0 tx=0).stop 2023-12-07T10:20:14.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.404+0000 7fa072fff700 1 -- 172.21.15.119:0/1306722163 shutdown_connections 2023-12-07T10:20:14.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.404+0000 7fa072fff700 1 --2- 172.21.15.119:0/1306722163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa06c1030f0 0x7fa06c1a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:14.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.404+0000 7fa072fff700 1 --2- 172.21.15.119:0/1306722163 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fa05c0a1b90 0x7fa05c0a4050 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:14.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.404+0000 7fa072fff700 1 --2- 172.21.15.119:0/1306722163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa06c101560 0x7fa06c199750 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:14.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.404+0000 7fa072fff700 1 --2- 172.21.15.119:0/1306722163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa06c102750 0x7fa06c199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:14.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.404+0000 7fa072fff700 1 -- 172.21.15.119:0/1306722163 >> 172.21.15.119:0/1306722163 conn(0x7fa06c0fcab0 msgr2=0x7fa06c073840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:14.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.404+0000 7fa072fff700 1 -- 172.21.15.119:0/1306722163 shutdown_connections 2023-12-07T10:20:14.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.404+0000 7fa072fff700 1 -- 172.21.15.119:0/1306722163 wait complete. 2023-12-07T10:20:14.407 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 29 2023-12-07T10:20:14.424 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-07T10:20:14.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:14 smithi119 ceph-mon[97908]: pgmap v359: 129 pgs: 129 active+clean; 660 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 9 op/s 2023-12-07T10:20:14.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:14 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1217000359' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:20:14.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:14 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2319005423' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:20:14.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:14 smithi159 ceph-mon[134726]: pgmap v359: 129 pgs: 129 active+clean; 660 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 9 op/s 2023-12-07T10:20:14.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:14 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1217000359' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:20:14.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:14 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2319005423' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:20:14.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:14 smithi153 ceph-mon[133625]: pgmap v359: 129 pgs: 129 active+clean; 660 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 9 op/s 2023-12-07T10:20:14.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:14 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1217000359' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:20:14.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:14 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2319005423' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:20:14.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.620+0000 7fd9cc7ff700 1 -- 172.21.15.119:0/3322378023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9c410db30 msgr2=0x7fd9c410ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:14.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.620+0000 7fd9cc7ff700 1 --2- 172.21.15.119:0/3322378023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9c410db30 0x7fd9c410ff20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc00b0a0 tx=0x7fd9bc00b3b0 comp rx=0 tx=0).stop 2023-12-07T10:20:14.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.620+0000 7fd9cc7ff700 1 -- 172.21.15.119:0/3322378023 shutdown_connections 2023-12-07T10:20:14.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.620+0000 7fd9cc7ff700 1 --2- 172.21.15.119:0/3322378023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9c410db30 0x7fd9c410ff20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:14.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.620+0000 7fd9cc7ff700 1 --2- 172.21.15.119:0/3322378023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9c4075650 0x7fd9c4073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:14.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.620+0000 7fd9cc7ff700 1 --2- 172.21.15.119:0/3322378023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9c4074070 0x7fd9c40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:14.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.620+0000 7fd9cc7ff700 1 -- 172.21.15.119:0/3322378023 >> 172.21.15.119:0/3322378023 conn(0x7fd9c40fcab0 msgr2=0x7fd9c40fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:14.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.620+0000 7fd9cc7ff700 1 -- 172.21.15.119:0/3322378023 shutdown_connections 2023-12-07T10:20:14.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.620+0000 7fd9cc7ff700 1 -- 172.21.15.119:0/3322378023 wait complete. 2023-12-07T10:20:14.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.621+0000 7fd9cc7ff700 1 Processor -- start 2023-12-07T10:20:14.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.621+0000 7fd9cc7ff700 1 -- start start 2023-12-07T10:20:14.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.621+0000 7fd9cc7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9c4074070 0x7fd9c419dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:14.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.621+0000 7fd9cc7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9c4075650 0x7fd9c419e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:14.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.622+0000 7fd9cc7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9c410db30 0x7fd9c41a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:14.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.622+0000 7fd9cc7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c41a5710 con 0x7fd9c4074070 2023-12-07T10:20:14.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.622+0000 7fd9cc7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c41a5850 con 0x7fd9c4075650 2023-12-07T10:20:14.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.622+0000 7fd9cc7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c41a59e0 con 0x7fd9c410db30 2023-12-07T10:20:14.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.622+0000 7fd9ca59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9c4074070 0x7fd9c419dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:14.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.622+0000 7fd9ca59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9c4074070 0x7fd9c419dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39266/0 (socket says 172.21.15.119:39266) 2023-12-07T10:20:14.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.622+0000 7fd9ca59b700 1 -- 172.21.15.119:0/1026003140 learned_addr learned my addr 172.21.15.119:0/1026003140 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:14.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.622+0000 7fd9c9d9a700 1 --2- 172.21.15.119:0/1026003140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9c4075650 0x7fd9c419e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:14.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.622+0000 7fd9cad9c700 1 --2- 172.21.15.119:0/1026003140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9c410db30 0x7fd9c41a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:14.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.622+0000 7fd9ca59b700 1 -- 172.21.15.119:0/1026003140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9c410db30 msgr2=0x7fd9c41a5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:14.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.622+0000 7fd9ca59b700 1 --2- 172.21.15.119:0/1026003140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9c410db30 0x7fd9c41a5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:14.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.622+0000 7fd9ca59b700 1 -- 172.21.15.119:0/1026003140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9c4075650 msgr2=0x7fd9c419e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:14.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.622+0000 7fd9ca59b700 1 --2- 172.21.15.119:0/1026003140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9c4075650 0x7fd9c419e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:14.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.622+0000 7fd9ca59b700 1 -- 172.21.15.119:0/1026003140 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9bc009d00 con 0x7fd9c4074070 2023-12-07T10:20:14.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.622+0000 7fd9cad9c700 1 --2- 172.21.15.119:0/1026003140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9c410db30 0x7fd9c41a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:20:14.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.622+0000 7fd9ca59b700 1 --2- 172.21.15.119:0/1026003140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9c4074070 0x7fd9c419dbe0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000e9a0 tx=0x7fd9c000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:14.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.623+0000 7fd9bb7fe700 1 -- 172.21.15.119:0/1026003140 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0009790 con 0x7fd9c4074070 2023-12-07T10:20:14.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.623+0000 7fd9cc7ff700 1 -- 172.21.15.119:0/1026003140 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c41a5cc0 con 0x7fd9c4074070 2023-12-07T10:20:14.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.623+0000 7fd9bb7fe700 1 -- 172.21.15.119:0/1026003140 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9c000cbd0 con 0x7fd9c4074070 2023-12-07T10:20:14.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.623+0000 7fd9bb7fe700 1 -- 172.21.15.119:0/1026003140 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0018680 con 0x7fd9c4074070 2023-12-07T10:20:14.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.623+0000 7fd9cc7ff700 1 -- 172.21.15.119:0/1026003140 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c41a61d0 con 0x7fd9c4074070 2023-12-07T10:20:14.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.624+0000 7fd9cc7ff700 1 -- 172.21.15.119:0/1026003140 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c4103d90 con 0x7fd9c4074070 2023-12-07T10:20:14.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.626+0000 7fd9bb7fe700 1 -- 172.21.15.119:0/1026003140 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7fd9c000cd40 con 0x7fd9c4074070 2023-12-07T10:20:14.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.627+0000 7fd9bb7fe700 1 --2- 172.21.15.119:0/1026003140 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd9b00a1b10 0x7fd9b00a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:14.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.627+0000 7fd9bb7fe700 1 -- 172.21.15.119:0/1026003140 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7fd9c0014070 con 0x7fd9c4074070 2023-12-07T10:20:14.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.627+0000 7fd9c9d9a700 1 --2- 172.21.15.119:0/1026003140 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd9b00a1b10 0x7fd9b00a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:14.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.628+0000 7fd9c9d9a700 1 --2- 172.21.15.119:0/1026003140 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd9b00a1b10 0x7fd9b00a3fd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4075370 tx=0x7fd9b4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:14.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.631+0000 7fd9bb7fe700 1 -- 172.21.15.119:0/1026003140 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd9c0096ce0 con 0x7fd9c4074070 2023-12-07T10:20:14.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.835+0000 7fd9cc7ff700 1 -- 172.21.15.119:0/1026003140 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9c4068350 con 0x7fd9c4074070 2023-12-07T10:20:14.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.837+0000 7fd9bb7fe700 1 -- 172.21.15.119:0/1026003140 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24715 (secure 0 0 0) 0x7fd9c009a300 con 0x7fd9c4074070 2023-12-07T10:20:14.839 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:20:14.839 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":93,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","created":"2023-12-07T10:09:59.945863+0000","modified":"2023-12-07T10:19:02.800626+0000","last_up_change":"2023-12-07T10:16:50.929427+0000","last_in_change":"2023-12-07T10:16:33.558158+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T10:13:25.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-07T10:17:39.347813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-07T10:17:40.714501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-07T10:17:44.257760+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"56b5818e-e625-40f1-8838-3ddcf2d34ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6803","nonce":2141950133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6805","nonce":2141950133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6809","nonce":2141950133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6807","nonce":2141950133}]},"public_addr":"172.21.15.119:6803/2141950133","cluster_addr":"172.21.15.119:6805/2141950133","heartbeat_back_addr":"172.21.15.119:6809/2141950133","heartbeat_front_addr":"172.21.15.119:6807/2141950133","state":["exists","up"]},{"osd":1,"uuid":"36489a6a-5278-4802-9525-9871521722e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6801","nonce":3390875370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6803","nonce":3390875370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6807","nonce":3390875370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6805","nonce":3390875370}]},"public_addr":"172.21.15.153:6801/3390875370","cluster_addr":"172.21.15.153:6803/3390875370","heartbeat_back_addr":"172.21.15.153:6807/3390875370","heartbeat_front_addr":"172.21.15.153:6805/3390875370","state":["exists","up"]},{"osd":2,"uuid":"a937ad26-5e9b-4c8a-b65a-626a8feda6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6801","nonce":708101637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6803","nonce":708101637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6807","nonce":708101637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6805","nonce":708101637}]},"public_addr":"172.21.15.159:6801/708101637","cluster_addr":"172.21.15.159:6803/708101637","heartbeat_back_addr":"172.21.15.159:6807/708101637","heartbeat_front_addr":"172.21.15.159:6805/708101637","state":["exists","up"]},{"osd":3,"uuid":"26ef8300-aa07-48f6-b31d-8b0f2217ffcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6811","nonce":3228047257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6813","nonce":3228047257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6817","nonce":3228047257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6815","nonce":3228047257}]},"public_addr":"172.21.15.119:6811/3228047257","cluster_addr":"172.21.15.119:6813/3228047257","heartbeat_back_addr":"172.21.15.119:6817/3228047257","heartbeat_front_addr":"172.21.15.119:6815/3228047257","state":["exists","up"]},{"osd":4,"uuid":"000ee4b9-9896-4901-8cc3-c85fdfc8cbaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6809","nonce":4021998299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6811","nonce":4021998299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6815","nonce":4021998299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6813","nonce":4021998299}]},"public_addr":"172.21.15.153:6809/4021998299","cluster_addr":"172.21.15.153:6811/4021998299","heartbeat_back_addr":"172.21.15.153:6815/4021998299","heartbeat_front_addr":"172.21.15.153:6813/4021998299","state":["exists","up"]},{"osd":5,"uuid":"8b2fc167-262d-474f-89f3-47bd29424421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6809","nonce":1832540768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6811","nonce":1832540768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6815","nonce":1832540768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6813","nonce":1832540768}]},"public_addr":"172.21.15.159:6809/1832540768","cluster_addr":"172.21.15.159:6811/1832540768","heartbeat_back_addr":"172.21.15.159:6815/1832540768","heartbeat_front_addr":"172.21.15.159:6813/1832540768","state":["exists","up"]},{"osd":6,"uuid":"f9aa6749-46cf-4b2a-8114-cbd034219d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162436840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162436840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6825","nonce":1162436840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162436840}]},"public_addr":"172.21.15.119:6819/1162436840","cluster_addr":"172.21.15.119:6821/1162436840","heartbeat_back_addr":"172.21.15.119:6825/1162436840","heartbeat_front_addr":"172.21.15.119:6823/1162436840","state":["exists","up"]},{"osd":7,"uuid":"39216361-e1c4-4fef-86cb-a4707656b547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6817","nonce":3254653457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6819","nonce":3254653457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6823","nonce":3254653457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6821","nonce":3254653457}]},"public_addr":"172.21.15.153:6817/3254653457","cluster_addr":"172.21.15.153:6819/3254653457","heartbeat_back_addr":"172.21.15.153:6823/3254653457","heartbeat_front_addr":"172.21.15.153:6821/3254653457","state":["exists","up"]},{"osd":8,"uuid":"cceb6ebc-7c02-48ea-9d16-b7a016dd449b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6817","nonce":2600946380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6819","nonce":2600946380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6823","nonce":2600946380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6821","nonce":2600946380}]},"public_addr":"172.21.15.159:6817/2600946380","cluster_addr":"172.21.15.159:6819/2600946380","heartbeat_back_addr":"172.21.15.159:6823/2600946380","heartbeat_front_addr":"172.21.15.159:6821/2600946380","state":["exists","up"]},{"osd":9,"uuid":"f337a2e4-e065-4a9f-91d9-e3615de2d6f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6827","nonce":1940737285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6829","nonce":1940737285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6833","nonce":1940737285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6831","nonce":1940737285}]},"public_addr":"172.21.15.119:6827/1940737285","cluster_addr":"172.21.15.119:6829/1940737285","heartbeat_back_addr":"172.21.15.119:6833/1940737285","heartbeat_front_addr":"172.21.15.119:6831/1940737285","state":["exists","up"]},{"osd":10,"uuid":"3a3bc347-85f4-4878-9797-d9da96137e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6825","nonce":663834864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6827","nonce":663834864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6831","nonce":663834864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6829","nonce":663834864}]},"public_addr":"172.21.15.153:6825/663834864","cluster_addr":"172.21.15.153:6827/663834864","heartbeat_back_addr":"172.21.15.153:6831/663834864","heartbeat_front_addr":"172.21.15.153:6829/663834864","state":["exists","up"]},{"osd":11,"uuid":"a0fc5909-279d-4172-977c-2dc2da0a7e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6825","nonce":1262216348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6827","nonce":1262216348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6831","nonce":1262216348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6829","nonce":1262216348}]},"public_addr":"172.21.15.159:6825/1262216348","cluster_addr":"172.21.15.159:6827/1262216348","heartbeat_back_addr":"172.21.15.159:6831/1262216348","heartbeat_front_addr":"172.21.15.159:6829/1262216348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:37.488109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:59.543060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:21.381585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:47.112729+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:09.732150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:30.893729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:56.678710+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:19.004256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:40.388818+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:06.391734+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:28.394053+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:49.634348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1628639392":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6801/1355617847":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6800/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/362307836":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/128964558":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/2278813554":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/1411933614":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/1685698277":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6800/1355617847":"2023-12-08T10:11:00.585565+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T10:20:14.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.840+0000 7fd9cc7ff700 1 -- 172.21.15.119:0/1026003140 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd9b00a1b10 msgr2=0x7fd9b00a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:14.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.840+0000 7fd9cc7ff700 1 --2- 172.21.15.119:0/1026003140 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd9b00a1b10 0x7fd9b00a3fd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4075370 tx=0x7fd9b4009260 comp rx=0 tx=0).stop 2023-12-07T10:20:14.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.840+0000 7fd9cc7ff700 1 -- 172.21.15.119:0/1026003140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9c4074070 msgr2=0x7fd9c419dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:14.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.840+0000 7fd9cc7ff700 1 --2- 172.21.15.119:0/1026003140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9c4074070 0x7fd9c419dbe0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000e9a0 tx=0x7fd9c000ed60 comp rx=0 tx=0).stop 2023-12-07T10:20:14.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.840+0000 7fd9cc7ff700 1 -- 172.21.15.119:0/1026003140 shutdown_connections 2023-12-07T10:20:14.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.840+0000 7fd9cc7ff700 1 --2- 172.21.15.119:0/1026003140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9c410db30 0x7fd9c41a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:14.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.840+0000 7fd9cc7ff700 1 --2- 172.21.15.119:0/1026003140 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7fd9b00a1b10 0x7fd9b00a3fd0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:14.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.840+0000 7fd9cc7ff700 1 --2- 172.21.15.119:0/1026003140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9c4074070 0x7fd9c419dbe0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:14.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.840+0000 7fd9cc7ff700 1 --2- 172.21.15.119:0/1026003140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9c4075650 0x7fd9c419e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:14.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.840+0000 7fd9cc7ff700 1 -- 172.21.15.119:0/1026003140 >> 172.21.15.119:0/1026003140 conn(0x7fd9c40fcab0 msgr2=0x7fd9c4100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:14.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.840+0000 7fd9cc7ff700 1 -- 172.21.15.119:0/1026003140 shutdown_connections 2023-12-07T10:20:14.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:14.840+0000 7fd9cc7ff700 1 -- 172.21.15.119:0/1026003140 wait complete. 2023-12-07T10:20:14.861 DEBUG:tasks.fwd_scrub:Yielding 2023-12-07T10:20:14.861 INFO:teuthology.run_tasks:Running task exec... 2023-12-07T10:20:14.873 INFO:teuthology.task.exec:Executing custom commands... 2023-12-07T10:20:14.873 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi119.front.sepia.ceph.com 2023-12-07T10:20:14.873 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-12-07T10:20:14.874 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-12-07T10:20:14.874 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-07T10:20:14.874 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-07T10:20:15.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.070+0000 7ff9d0b9d700 1 -- 172.21.15.119:0/3443218413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9cc102590 msgr2=0x7ff9cc104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:15.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.070+0000 7ff9d0b9d700 1 --2- 172.21.15.119:0/3443218413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9cc102590 0x7ff9cc104a20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009990 tx=0x7ff9c0009ca0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.070+0000 7ff9d0b9d700 1 -- 172.21.15.119:0/3443218413 shutdown_connections 2023-12-07T10:20:15.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.070+0000 7ff9d0b9d700 1 --2- 172.21.15.119:0/3443218413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9cc104f60 0x7ff9cc1053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.070+0000 7ff9d0b9d700 1 --2- 172.21.15.119:0/3443218413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9cc0ffc20 0x7ff9cc102050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.070+0000 7ff9d0b9d700 1 --2- 172.21.15.119:0/3443218413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9cc102590 0x7ff9cc104a20 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.070+0000 7ff9d0b9d700 1 -- 172.21.15.119:0/3443218413 >> 172.21.15.119:0/3443218413 conn(0x7ff9cc0fb820 msgr2=0x7ff9cc0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:15.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.070+0000 7ff9d0b9d700 1 -- 172.21.15.119:0/3443218413 shutdown_connections 2023-12-07T10:20:15.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.070+0000 7ff9d0b9d700 1 -- 172.21.15.119:0/3443218413 wait complete. 2023-12-07T10:20:15.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.071+0000 7ff9d0b9d700 1 Processor -- start 2023-12-07T10:20:15.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.071+0000 7ff9d0b9d700 1 -- start start 2023-12-07T10:20:15.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.071+0000 7ff9d0b9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9cc0ffc20 0x7ff9cc19d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.071+0000 7ff9d0b9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9cc102590 0x7ff9cc19de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.071+0000 7ff9d0b9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9cc104f60 0x7ff9cc1a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.071+0000 7ff9d0b9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc1a5470 con 0x7ff9cc102590 2023-12-07T10:20:15.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.071+0000 7ff9d0b9d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc1a55b0 con 0x7ff9cc0ffc20 2023-12-07T10:20:15.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.071+0000 7ff9d0b9d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc1a5740 con 0x7ff9cc104f60 2023-12-07T10:20:15.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.072+0000 7ff9caffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9cc102590 0x7ff9cc19de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.072+0000 7ff9caffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9cc102590 0x7ff9cc19de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39298/0 (socket says 172.21.15.119:39298) 2023-12-07T10:20:15.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.072+0000 7ff9caffd700 1 -- 172.21.15.119:0/18588951 learned_addr learned my addr 172.21.15.119:0/18588951 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:15.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.072+0000 7ff9cb7fe700 1 --2- 172.21.15.119:0/18588951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9cc0ffc20 0x7ff9cc19d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.072+0000 7ff9cbfff700 1 --2- 172.21.15.119:0/18588951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9cc104f60 0x7ff9cc1a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.072+0000 7ff9caffd700 1 -- 172.21.15.119:0/18588951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9cc104f60 msgr2=0x7ff9cc1a4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:15.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.072+0000 7ff9caffd700 1 --2- 172.21.15.119:0/18588951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9cc104f60 0x7ff9cc1a4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.072+0000 7ff9caffd700 1 -- 172.21.15.119:0/18588951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9cc0ffc20 msgr2=0x7ff9cc19d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:15.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.072+0000 7ff9caffd700 1 --2- 172.21.15.119:0/18588951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9cc0ffc20 0x7ff9cc19d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.072+0000 7ff9caffd700 1 -- 172.21.15.119:0/18588951 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c00095f0 con 0x7ff9cc102590 2023-12-07T10:20:15.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.072+0000 7ff9caffd700 1 --2- 172.21.15.119:0/18588951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9cc102590 0x7ff9cc19de80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00098e0 tx=0x7ff9c0005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:15.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.073+0000 7ff9c8ff9700 1 -- 172.21.15.119:0/18588951 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c001d070 con 0x7ff9cc102590 2023-12-07T10:20:15.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.073+0000 7ff9d0b9d700 1 -- 172.21.15.119:0/18588951 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9cc1a59c0 con 0x7ff9cc102590 2023-12-07T10:20:15.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.073+0000 7ff9c8ff9700 1 -- 172.21.15.119:0/18588951 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9c0022470 con 0x7ff9cc102590 2023-12-07T10:20:15.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.073+0000 7ff9d0b9d700 1 -- 172.21.15.119:0/18588951 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9cc1a5df0 con 0x7ff9cc102590 2023-12-07T10:20:15.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.073+0000 7ff9c8ff9700 1 -- 172.21.15.119:0/18588951 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c00175f0 con 0x7ff9cc102590 2023-12-07T10:20:15.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.074+0000 7ff9d0b9d700 1 -- 172.21.15.119:0/18588951 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9cc0fd3a0 con 0x7ff9cc102590 2023-12-07T10:20:15.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.076+0000 7ff9c8ff9700 1 -- 172.21.15.119:0/18588951 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7ff9c00225e0 con 0x7ff9cc102590 2023-12-07T10:20:15.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.077+0000 7ff9c8ff9700 1 --2- 172.21.15.119:0/18588951 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff9b40a1b10 0x7ff9b40a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.077+0000 7ff9c8ff9700 1 -- 172.21.15.119:0/18588951 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7ff9c00cedb0 con 0x7ff9cc102590 2023-12-07T10:20:15.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.077+0000 7ff9cb7fe700 1 --2- 172.21.15.119:0/18588951 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff9b40a1b10 0x7ff9b40a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.078+0000 7ff9cb7fe700 1 --2- 172.21.15.119:0/18588951 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff9b40a1b10 0x7ff9b40a3fd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009c60 tx=0x7ff9bc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:15.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.081+0000 7ff9c8ff9700 1 -- 172.21.15.119:0/18588951 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff9c0097e90 con 0x7ff9cc102590 2023-12-07T10:20:15.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.082+0000 7f6a383ff700 1 -- 172.21.15.119:0/1806347655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a301015c0 msgr2=0x7f6a301019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:15.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.082+0000 7f6a383ff700 1 --2- 172.21.15.119:0/1806347655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a301015c0 0x7f6a301019e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00b0d0 tx=0x7f6a2c00b3e0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.082+0000 7f6a383ff700 1 -- 172.21.15.119:0/1806347655 shutdown_connections 2023-12-07T10:20:15.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.082+0000 7f6a383ff700 1 --2- 172.21.15.119:0/1806347655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a30103150 0x7f6a30109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.082+0000 7f6a383ff700 1 --2- 172.21.15.119:0/1806347655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a301015c0 0x7f6a301019e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.082+0000 7f6a383ff700 1 --2- 172.21.15.119:0/1806347655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a301027b0 0x7f6a30102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.082+0000 7f6a383ff700 1 -- 172.21.15.119:0/1806347655 >> 172.21.15.119:0/1806347655 conn(0x7f6a300fcb10 msgr2=0x7f6a300fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:15.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.082+0000 7f6a383ff700 1 -- 172.21.15.119:0/1806347655 shutdown_connections 2023-12-07T10:20:15.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.082+0000 7f6a383ff700 1 -- 172.21.15.119:0/1806347655 wait complete. 2023-12-07T10:20:15.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.083+0000 7f6a383ff700 1 Processor -- start 2023-12-07T10:20:15.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.083+0000 7f6a383ff700 1 -- start start 2023-12-07T10:20:15.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.084+0000 7f6a383ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a301015c0 0x7f6a30075320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.084+0000 7f6a383ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a301027b0 0x7f6a30073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.084+0000 7f6a383ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a30103150 0x7f6a30073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.084+0000 7f6a383ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a30074640 con 0x7f6a30103150 2023-12-07T10:20:15.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.084+0000 7f6a383ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a300747b0 con 0x7f6a301027b0 2023-12-07T10:20:15.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.084+0000 7f6a383ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a301a17a0 con 0x7f6a301015c0 2023-12-07T10:20:15.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.084+0000 7f6a3699c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a30103150 0x7f6a30073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.084+0000 7f6a3699c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a30103150 0x7f6a30073f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39306/0 (socket says 172.21.15.119:39306) 2023-12-07T10:20:15.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.084+0000 7f6a3699c700 1 -- 172.21.15.119:0/1161486247 learned_addr learned my addr 172.21.15.119:0/1161486247 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:15.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.084+0000 7f6a3599a700 1 --2- 172.21.15.119:0/1161486247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a301027b0 0x7f6a30073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.084+0000 7f6a3619b700 1 --2- 172.21.15.119:0/1161486247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a301015c0 0x7f6a30075320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.084+0000 7f6a3699c700 1 -- 172.21.15.119:0/1161486247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a301015c0 msgr2=0x7f6a30075320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:15.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.084+0000 7f6a3699c700 1 --2- 172.21.15.119:0/1161486247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a301015c0 0x7f6a30075320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.084+0000 7f6a3699c700 1 -- 172.21.15.119:0/1161486247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a301027b0 msgr2=0x7f6a30073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:15.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.084+0000 7f6a3699c700 1 --2- 172.21.15.119:0/1161486247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a301027b0 0x7f6a30073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.084+0000 7f6a3699c700 1 -- 172.21.15.119:0/1161486247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2c009d00 con 0x7f6a30103150 2023-12-07T10:20:15.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.085+0000 7f6a3699c700 1 --2- 172.21.15.119:0/1161486247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a30103150 0x7f6a30073f40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6a2800d980 tx=0x7f6a2800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:15.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.085+0000 7f6a277fe700 1 -- 172.21.15.119:0/1161486247 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a28004220 con 0x7f6a30103150 2023-12-07T10:20:15.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.085+0000 7f6a383ff700 1 -- 172.21.15.119:0/1161486247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a301a1a00 con 0x7f6a30103150 2023-12-07T10:20:15.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.085+0000 7f6a277fe700 1 -- 172.21.15.119:0/1161486247 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a280075e0 con 0x7f6a30103150 2023-12-07T10:20:15.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.085+0000 7f6a277fe700 1 -- 172.21.15.119:0/1161486247 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a28005650 con 0x7f6a30103150 2023-12-07T10:20:15.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.085+0000 7f6a383ff700 1 -- 172.21.15.119:0/1161486247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a301a1ec0 con 0x7f6a30103150 2023-12-07T10:20:15.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.089+0000 7f6a277fe700 1 -- 172.21.15.119:0/1161486247 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f6a28004750 con 0x7f6a30103150 2023-12-07T10:20:15.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.090+0000 7f6a4bc69b80 1 Processor -- start 2023-12-07T10:20:15.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.090+0000 7f6a4bc69b80 1 -- start start 2023-12-07T10:20:15.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.091+0000 7f6a4bc69b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582c4a6b200 0x5582c4a6b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.091+0000 7f6a4bc69b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582c4a6bb60 0x5582c4a6c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.091+0000 7f6a4bc69b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582c4a6ca50 0x5582c4a6ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.091+0000 7f6a4bc69b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c49df840 con 0x5582c4a6ca50 2023-12-07T10:20:15.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.091+0000 7f6a4bc69b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c49c6820 con 0x5582c4a6bb60 2023-12-07T10:20:15.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.091+0000 7f6a4bc69b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c49c4140 con 0x5582c4a6b200 2023-12-07T10:20:15.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.091+0000 7f6a277fe700 1 --2- 172.21.15.119:0/1161486247 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6a1c0a1b10 0x7f6a1c0a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.091+0000 7f6a3619b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582c4a6bb60 0x5582c4a6c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.091+0000 7f6a3599a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582c4a6b200 0x5582c4a6b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.091+0000 7f6a3619b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582c4a6bb60 0x5582c4a6c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:51448/0 (socket says 172.21.15.119:51448) 2023-12-07T10:20:15.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.091+0000 7f6a3619b700 1 -- 172.21.15.119:0/1090725208 learned_addr learned my addr 172.21.15.119:0/1090725208 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:15.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.091+0000 7f6a3599a700 1 --2- 172.21.15.119:0/1090725208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582c4a6ca50 0x5582c4a6ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.091+0000 7f6a277fe700 1 -- 172.21.15.119:0/1161486247 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f6a280cd690 con 0x7f6a30103150 2023-12-07T10:20:15.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.092+0000 7f6a3699c700 1 --2- 172.21.15.119:0/1161486247 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6a1c0a1b10 0x7f6a1c0a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.092+0000 7f6a3619b700 1 -- 172.21.15.119:0/1090725208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582c4a6b200 msgr2=0x5582c4a6b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:15.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.092+0000 7f6a3619b700 1 --2- 172.21.15.119:0/1090725208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582c4a6b200 0x5582c4a6b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.092+0000 7f6a3619b700 1 -- 172.21.15.119:0/1090725208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582c4a6ca50 msgr2=0x5582c4a6ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:15.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.092+0000 7f6a3619b700 1 --2- 172.21.15.119:0/1090725208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582c4a6ca50 0x5582c4a6ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.092+0000 7f6a3619b700 1 -- 172.21.15.119:0/1090725208 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582c483c6b0 con 0x5582c4a6bb60 2023-12-07T10:20:15.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.092+0000 7f6a3599a700 1 --2- 172.21.15.119:0/1090725208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582c4a6ca50 0x5582c4a6ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:20:15.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.092+0000 7f6a3619b700 1 --2- 172.21.15.119:0/1090725208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582c4a6bb60 0x5582c4a6c470 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6a30102610 tx=0x7f6a2c009c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a8177f1e9cddecb server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:15.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.092+0000 7f6a3699c700 1 --2- 172.21.15.119:0/1161486247 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6a1c0a1b10 0x7f6a1c0a3fd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6a28096e70 tx=0x7f6a280050b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:15.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.092+0000 7f6a24bfe700 1 -- 172.21.15.119:0/1090725208 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c00e040 con 0x5582c4a6bb60 2023-12-07T10:20:15.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.092+0000 7f6a24bfe700 1 -- 172.21.15.119:0/1090725208 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a2c00baa0 con 0x5582c4a6bb60 2023-12-07T10:20:15.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.092+0000 7f6a24bfe700 1 -- 172.21.15.119:0/1090725208 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c005070 con 0x5582c4a6bb60 2023-12-07T10:20:15.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.093+0000 7f6a4bc69b80 1 -- 172.21.15.119:0/1090725208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582c4a6bb60 msgr2=0x5582c4a6c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:15.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.093+0000 7f6a4bc69b80 1 --2- 172.21.15.119:0/1090725208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582c4a6bb60 0x5582c4a6c470 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6a30102610 tx=0x7f6a2c009c90 comp rx=0 tx=0).stop 2023-12-07T10:20:15.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.093+0000 7f6a4bc69b80 1 -- 172.21.15.119:0/1090725208 shutdown_connections 2023-12-07T10:20:15.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.093+0000 7f6a4bc69b80 1 --2- 172.21.15.119:0/1090725208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582c4a6ca50 0x5582c4a6ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.093+0000 7f6a4bc69b80 1 --2- 172.21.15.119:0/1090725208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582c4a6b200 0x5582c4a6b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.093+0000 7f6a4bc69b80 1 --2- 172.21.15.119:0/1090725208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582c4a6bb60 0x5582c4a6c470 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.093+0000 7f6a4bc69b80 1 -- 172.21.15.119:0/1090725208 >> 172.21.15.119:0/1090725208 conn(0x5582c499a3e0 msgr2=0x5582c4a67bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:15.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.093+0000 7f6a4bc69b80 1 -- 172.21.15.119:0/1090725208 shutdown_connections 2023-12-07T10:20:15.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.093+0000 7f6a4bc69b80 1 -- 172.21.15.119:0/1090725208 wait complete. 2023-12-07T10:20:15.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.094+0000 7f6a4bc69b80 1 -- start start 2023-12-07T10:20:15.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.094+0000 7f6a4bc69b80 1 Processor -- start 2023-12-07T10:20:15.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.094+0000 7f6a4bc69b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582c4a6b200 0x5582c4a75010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.094+0000 7f6a4bc69b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582c4a6bb60 0x5582c4a75550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.094+0000 7f6a4bc69b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582c4a6ca50 0x5582c4a7c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.094+0000 7f6a4bc69b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c49bfa30 con 0x5582c4a6bb60 2023-12-07T10:20:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.094+0000 7f6a4bc69b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c49795e0 con 0x5582c4a6ca50 2023-12-07T10:20:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.094+0000 7f6a4bc69b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c4979280 con 0x5582c4a6b200 2023-12-07T10:20:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.094+0000 7f6a4bc69b80 10 client.0 _ll_register_callbacks cb 0x5582c49609c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:20:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.095+0000 7f6a3619b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582c4a6bb60 0x5582c4a75550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.095+0000 7f6a3619b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582c4a6bb60 0x5582c4a75550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39320/0 (socket says 172.21.15.119:39320) 2023-12-07T10:20:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.095+0000 7f6a3619b700 1 -- 172.21.15.119:0/1826359887 learned_addr learned my addr 172.21.15.119:0/1826359887 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.095+0000 7f6a3599a700 1 --2- 172.21.15.119:0/1826359887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582c4a6ca50 0x5582c4a7c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.095+0000 7f6a3599a700 1 --2- 172.21.15.119:0/1826359887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582c4a6b200 0x5582c4a75010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.095+0000 7f6a3619b700 1 -- 172.21.15.119:0/1826359887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582c4a6b200 msgr2=0x5582c4a75010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.095+0000 7f6a3619b700 1 --2- 172.21.15.119:0/1826359887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582c4a6b200 0x5582c4a75010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.095+0000 7f6a3619b700 1 -- 172.21.15.119:0/1826359887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582c4a6ca50 msgr2=0x5582c4a7c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.095+0000 7f6a3619b700 1 --2- 172.21.15.119:0/1826359887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582c4a6ca50 0x5582c4a7c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.095+0000 7f6a3619b700 1 -- 172.21.15.119:0/1826359887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a3010a350 con 0x5582c4a6bb60 2023-12-07T10:20:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.095+0000 7f6a3599a700 1 --2- 172.21.15.119:0/1826359887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582c4a6b200 0x5582c4a75010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:20:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.095+0000 7f6a3619b700 1 --2- 172.21.15.119:0/1826359887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582c4a6bb60 0x5582c4a75550 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00b770 tx=0x7f6a2c006080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687132369d6c1aa1 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.095+0000 7f6a19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:20:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.096+0000 7f6a19ffb700 1 -- 172.21.15.119:0/1826359887 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c00e040 con 0x5582c4a6bb60 2023-12-07T10:20:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.096+0000 7f6a4bc69b80 1 -- 172.21.15.119:0/1826359887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5582c4a5eb80 con 0x5582c4a6bb60 2023-12-07T10:20:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.096+0000 7f6a19ffb700 1 -- 172.21.15.119:0/1826359887 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f6a2c0050c0 con 0x5582c4a6bb60 2023-12-07T10:20:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.096+0000 7f6a19ffb700 1 -- 172.21.15.119:0/1826359887 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a2c005280 con 0x5582c4a6bb60 2023-12-07T10:20:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.096+0000 7f6a4bc69b80 10 client.15570 fetch_fsmap learned FSMap version 29 2023-12-07T10:20:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.096+0000 7f6a4bc69b80 1 -- 172.21.15.119:0/1826359887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5582c47e8e60 con 0x5582c4a6bb60 2023-12-07T10:20:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.096+0000 7f6a19ffb700 1 -- 172.21.15.119:0/1826359887 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5582c4a5eb80 con 0x5582c4a6bb60 2023-12-07T10:20:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.096+0000 7f6a19ffb700 1 -- 172.21.15.119:0/1826359887 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f6a2c00b100 con 0x5582c4a6bb60 2023-12-07T10:20:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.097+0000 7f6a4bc69b80 10 client.15570 fetch_fsmap finished waiting for FSMap version 29 2023-12-07T10:20:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.097+0000 7f6a4bc69b80 10 client.15570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:20:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.097+0000 7f6a4bc69b80 20 client.15570 populate_metadata read hostname 'smithi119' 2023-12-07T10:20:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.097+0000 7f6a4bc69b80 1 --2- 172.21.15.119:0/1826359887 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5582c4a80a80 0x5582c4a82ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.097+0000 7f6a4bc69b80 4 client.15570 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:20:15.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.097+0000 7f6a4bc69b80 1 -- 172.21.15.119:0/1826359887 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582c4a3a820 con 0x5582c4a80a80 2023-12-07T10:20:15.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.097+0000 7f6a3599a700 1 --2- 172.21.15.119:0/1826359887 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5582c4a80a80 0x5582c4a82ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.098+0000 7f6a19ffb700 10 client.15570 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:15.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.098+0000 7f6a3599a700 1 --2- 172.21.15.119:0/1826359887 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5582c4a80a80 0x5582c4a82ea0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6493f36ec88d731 server_cookie=ae3290fa3ad09623 in_seq=0 out_seq=0 2023-12-07T10:20:15.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.098+0000 7f6a19ffb700 1 -- 172.21.15.119:0/1826359887 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5582c4a3a820 con 0x5582c4a80a80 2023-12-07T10:20:15.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.098+0000 7f6a19ffb700 10 client.15570 handle_command_reply: tid=0 2023-12-07T10:20:15.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.098+0000 7f6a4bc69b80 1 client.15570 shutdown 2023-12-07T10:20:15.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.098+0000 7f6a4bc69b80 1 -- 172.21.15.119:0/1826359887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582c4a6bb60 msgr2=0x5582c4a75550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:15.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.098+0000 7f6a4bc69b80 1 --2- 172.21.15.119:0/1826359887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582c4a6bb60 0x5582c4a75550 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00b770 tx=0x7f6a2c006080 comp rx=0 tx=0).stop 2023-12-07T10:20:15.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.099+0000 7f6a4bc69b80 1 -- 172.21.15.119:0/1826359887 shutdown_connections 2023-12-07T10:20:15.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.099+0000 7f6a4bc69b80 1 --2- 172.21.15.119:0/1826359887 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5582c4a80a80 0x5582c4a82ea0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.099+0000 7f6a4bc69b80 1 --2- 172.21.15.119:0/1826359887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582c4a6ca50 0x5582c4a7c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.099+0000 7f6a4bc69b80 1 --2- 172.21.15.119:0/1826359887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582c4a6b200 0x5582c4a75010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.099+0000 7f6a19ffb700 0 client.15570 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:15.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.099+0000 7f6a4bc69b80 1 --2- 172.21.15.119:0/1826359887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582c4a6bb60 0x5582c4a75550 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.099+0000 7f6a4bc69b80 1 -- 172.21.15.119:0/1826359887 >> 172.21.15.119:0/1826359887 conn(0x5582c499a3e0 msgr2=0x5582c4a66a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:15.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.099+0000 7f6a4bc69b80 1 -- 172.21.15.119:0/1826359887 shutdown_connections 2023-12-07T10:20:15.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.099+0000 7f6a4bc69b80 1 -- 172.21.15.119:0/1826359887 wait complete. 2023-12-07T10:20:15.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.099+0000 7f6a4bc69b80 20 client.15570 trim_cache size 0 max 16384 2023-12-07T10:20:15.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.116+0000 7f6a4bc69b80 1 Processor -- start 2023-12-07T10:20:15.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.117+0000 7f6a4bc69b80 1 -- start start 2023-12-07T10:20:15.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.117+0000 7f6a4bc69b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582c4a6b200 0x5582c4a675a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.117+0000 7f6a4bc69b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582c4a6bb60 0x5582c4a7c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.117+0000 7f6a4bc69b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582c4a6ca50 0x5582c4a67ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.117+0000 7f6a4bc69b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c483e9a0 con 0x5582c4a6bb60 2023-12-07T10:20:15.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.117+0000 7f6a4bc69b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c4a6ada0 con 0x5582c4a6b200 2023-12-07T10:20:15.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.117+0000 7f6a4bc69b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c4a672a0 con 0x5582c4a6ca50 2023-12-07T10:20:15.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.117+0000 7f6a3619b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582c4a6bb60 0x5582c4a7c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.117+0000 7f6a3619b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582c4a6bb60 0x5582c4a7c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39326/0 (socket says 172.21.15.119:39326) 2023-12-07T10:20:15.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.117+0000 7f6a3599a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582c4a6ca50 0x5582c4a67ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.117+0000 7f6a3619b700 1 -- 172.21.15.119:0/688887256 learned_addr learned my addr 172.21.15.119:0/688887256 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:15.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.117+0000 7f6a3599a700 1 --2- 172.21.15.119:0/688887256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582c4a6b200 0x5582c4a675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.118+0000 7f6a3619b700 1 -- 172.21.15.119:0/688887256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582c4a6ca50 msgr2=0x5582c4a67ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:15.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.118+0000 7f6a3619b700 1 --2- 172.21.15.119:0/688887256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582c4a6ca50 0x5582c4a67ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.118+0000 7f6a3619b700 1 -- 172.21.15.119:0/688887256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582c4a6b200 msgr2=0x5582c4a675a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:15.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.118+0000 7f6a3619b700 1 --2- 172.21.15.119:0/688887256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582c4a6b200 0x5582c4a675a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.118+0000 7f6a3619b700 1 -- 172.21.15.119:0/688887256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582c47d1ce0 con 0x5582c4a6bb60 2023-12-07T10:20:15.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.118+0000 7f6a3619b700 1 --2- 172.21.15.119:0/688887256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582c4a6bb60 0x5582c4a7c500 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c025ea0 tx=0x7f6a2c009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd68214f8929258c server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:15.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.118+0000 7f6a253ff700 1 -- 172.21.15.119:0/688887256 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c00e040 con 0x5582c4a6bb60 2023-12-07T10:20:15.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.118+0000 7f6a253ff700 1 -- 172.21.15.119:0/688887256 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a2c011390 con 0x5582c4a6bb60 2023-12-07T10:20:15.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.118+0000 7f6a253ff700 1 -- 172.21.15.119:0/688887256 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c005110 con 0x5582c4a6bb60 2023-12-07T10:20:15.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.119+0000 7f6a4bc69b80 1 -- 172.21.15.119:0/688887256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582c4a6bb60 msgr2=0x5582c4a7c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:15.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.119+0000 7f6a4bc69b80 1 --2- 172.21.15.119:0/688887256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582c4a6bb60 0x5582c4a7c500 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c025ea0 tx=0x7f6a2c009c90 comp rx=0 tx=0).stop 2023-12-07T10:20:15.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.119+0000 7f6a4bc69b80 1 -- 172.21.15.119:0/688887256 shutdown_connections 2023-12-07T10:20:15.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.119+0000 7f6a4bc69b80 1 --2- 172.21.15.119:0/688887256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582c4a6ca50 0x5582c4a67ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.119+0000 7f6a4bc69b80 1 --2- 172.21.15.119:0/688887256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582c4a6b200 0x5582c4a675a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.119+0000 7f6a4bc69b80 1 --2- 172.21.15.119:0/688887256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582c4a6bb60 0x5582c4a7c500 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.119+0000 7f6a4bc69b80 1 -- 172.21.15.119:0/688887256 >> 172.21.15.119:0/688887256 conn(0x5582c499a3e0 msgr2=0x5582c4a75010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:15.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.119+0000 7f6a4bc69b80 1 -- 172.21.15.119:0/688887256 shutdown_connections 2023-12-07T10:20:15.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.119+0000 7f6a4bc69b80 1 -- 172.21.15.119:0/688887256 wait complete. 2023-12-07T10:20:15.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.119+0000 7f6a4bc69b80 1 -- start start 2023-12-07T10:20:15.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.120+0000 7f6a4bc69b80 1 Processor -- start 2023-12-07T10:20:15.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.120+0000 7f6a4bc69b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582c4a6b200 0x5582c4a7c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.120+0000 7f6a4bc69b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582c4a6bb60 0x5582c4a7a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.120+0000 7f6a4bc69b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582c4a6ca50 0x5582c4a7acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.120+0000 7f6a4bc69b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c4a7b1f0 con 0x5582c4a6ca50 2023-12-07T10:20:15.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.120+0000 7f6a4bc69b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c4a7b360 con 0x5582c4a6b200 2023-12-07T10:20:15.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.120+0000 7f6a4bc69b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582c4a7b520 con 0x5582c4a6bb60 2023-12-07T10:20:15.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.120+0000 7f6a4bc69b80 10 client.0 _ll_register_callbacks cb 0x5582c49609c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:20:15.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.120+0000 7f6a3599a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582c4a6ca50 0x5582c4a7acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.120+0000 7f6a3599a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582c4a6b200 0x5582c4a7c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.120+0000 7f6a3619b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582c4a6bb60 0x5582c4a7a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.120+0000 7f6a3599a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582c4a6b200 0x5582c4a7c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:51470/0 (socket says 172.21.15.119:51470) 2023-12-07T10:20:15.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.120+0000 7f6a3599a700 1 -- 172.21.15.119:0/1105356420 learned_addr learned my addr 172.21.15.119:0/1105356420 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:15.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.121+0000 7f6a3599a700 1 -- 172.21.15.119:0/1105356420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582c4a6bb60 msgr2=0x5582c4a7a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:15.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.121+0000 7f6a3599a700 1 --2- 172.21.15.119:0/1105356420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582c4a6bb60 0x5582c4a7a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.121+0000 7f6a3599a700 1 -- 172.21.15.119:0/1105356420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582c4a6ca50 msgr2=0x5582c4a7acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:15.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.121+0000 7f6a3599a700 1 --2- 172.21.15.119:0/1105356420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582c4a6ca50 0x5582c4a7acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.121+0000 7f6a3599a700 1 -- 172.21.15.119:0/1105356420 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582c47e8e60 con 0x5582c4a6b200 2023-12-07T10:20:15.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.121+0000 7f6a3619b700 1 --2- 172.21.15.119:0/1105356420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582c4a6bb60 0x5582c4a7a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:15.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.121+0000 7f6a3599a700 1 --2- 172.21.15.119:0/1105356420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582c4a6b200 0x5582c4a7c110 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009340 tx=0x7f6a20004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46456024d041c6ba server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:15.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.121+0000 7f6a19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:20:15.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.121+0000 7f6a19ffb700 1 -- 172.21.15.119:0/1105356420 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a20016040 con 0x5582c4a6b200 2023-12-07T10:20:15.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.121+0000 7f6a4bc69b80 1 -- 172.21.15.119:0/1105356420 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a2c00baa0 con 0x5582c4a6b200 2023-12-07T10:20:15.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.121+0000 7f6a19ffb700 1 -- 172.21.15.119:0/1105356420 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f6a200240d0 con 0x5582c4a6b200 2023-12-07T10:20:15.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.121+0000 7f6a19ffb700 1 -- 172.21.15.119:0/1105356420 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a20024290 con 0x5582c4a6b200 2023-12-07T10:20:15.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.121+0000 7f6a4bc69b80 10 client.25604 fetch_fsmap learned FSMap version 29 2023-12-07T10:20:15.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.122+0000 7f6a4bc69b80 1 -- 172.21.15.119:0/1105356420 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5582c480f2d0 con 0x5582c4a6b200 2023-12-07T10:20:15.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.122+0000 7f6a19ffb700 1 -- 172.21.15.119:0/1105356420 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a20036080 con 0x5582c4a6b200 2023-12-07T10:20:15.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.122+0000 7f6a19ffb700 1 -- 172.21.15.119:0/1105356420 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f6a20003be0 con 0x5582c4a6b200 2023-12-07T10:20:15.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.122+0000 7f6a4bc69b80 10 client.25604 fetch_fsmap finished waiting for FSMap version 29 2023-12-07T10:20:15.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.122+0000 7f6a4bc69b80 10 client.25604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:20:15.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.122+0000 7f6a4bc69b80 20 client.25604 populate_metadata read hostname 'smithi119' 2023-12-07T10:20:15.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.122+0000 7f6a4bc69b80 1 --2- 172.21.15.119:0/1105356420 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5582c4ad86a0 0x5582c4adaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.122+0000 7f6a4bc69b80 4 client.25604 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-07T10:20:15.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.122+0000 7f6a4bc69b80 1 -- 172.21.15.119:0/1105356420 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5582c4986050 con 0x5582c4ad86a0 2023-12-07T10:20:15.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.122+0000 7f6a3619b700 1 --2- 172.21.15.119:0/1105356420 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5582c4ad86a0 0x5582c4adaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.123+0000 7f6a3619b700 1 --2- 172.21.15.119:0/1105356420 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5582c4ad86a0 0x5582c4adaac0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6326df7b0d9a3881 server_cookie=8b79738f77403097 in_seq=0 out_seq=0 2023-12-07T10:20:15.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.123+0000 7f6a19ffb700 10 client.25604 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:15.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.124+0000 7f6a19ffb700 1 -- 172.21.15.119:0/1105356420 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5582c4986050 con 0x5582c4ad86a0 2023-12-07T10:20:15.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.124+0000 7f6a19ffb700 10 client.25604 handle_command_reply: tid=0 2023-12-07T10:20:15.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.124+0000 7f6a4bc69b80 1 client.25604 shutdown 2023-12-07T10:20:15.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.124+0000 7f6a4bc69b80 1 -- 172.21.15.119:0/1105356420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582c4a6b200 msgr2=0x5582c4a7c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:15.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.124+0000 7f6a4bc69b80 1 --2- 172.21.15.119:0/1105356420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582c4a6b200 0x5582c4a7c110 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009340 tx=0x7f6a20004750 comp rx=0 tx=0).stop 2023-12-07T10:20:15.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.124+0000 7f6a4bc69b80 1 -- 172.21.15.119:0/1105356420 shutdown_connections 2023-12-07T10:20:15.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.124+0000 7f6a4bc69b80 1 --2- 172.21.15.119:0/1105356420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582c4a6ca50 0x5582c4a7acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.124+0000 7f6a4bc69b80 1 --2- 172.21.15.119:0/1105356420 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5582c4ad86a0 0x5582c4adaac0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.124+0000 7f6a4bc69b80 1 --2- 172.21.15.119:0/1105356420 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582c4a6b200 0x5582c4a7c110 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.124+0000 7f6a4bc69b80 1 --2- 172.21.15.119:0/1105356420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582c4a6bb60 0x5582c4a7a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.124+0000 7f6a4bc69b80 1 -- 172.21.15.119:0/1105356420 >> 172.21.15.119:0/1105356420 conn(0x5582c499a3e0 msgr2=0x5582c4a67510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:15.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.124+0000 7f6a19ffb700 0 client.25604 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:15.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.124+0000 7f6a4bc69b80 1 -- 172.21.15.119:0/1105356420 shutdown_connections 2023-12-07T10:20:15.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.125+0000 7f6a4bc69b80 1 -- 172.21.15.119:0/1105356420 wait complete. 2023-12-07T10:20:15.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.125+0000 7f6a4bc69b80 20 client.25604 trim_cache size 0 max 16384 2023-12-07T10:20:15.132 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:20:15.132 INFO:teuthology.orchestra.run.smithi119.stdout: "return_code": 0, 2023-12-07T10:20:15.132 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_tag": "6c28b114-ced4-46ba-9f90-666af450944c", 2023-12-07T10:20:15.132 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": "asynchronous" 2023-12-07T10:20:15.132 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:20:15.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.125+0000 7f6a383ff700 1 -- 172.21.15.119:0/1161486247 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6a1c0a1b10 msgr2=0x7f6a1c0a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:15.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.125+0000 7f6a383ff700 1 --2- 172.21.15.119:0/1161486247 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6a1c0a1b10 0x7f6a1c0a3fd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6a28096e70 tx=0x7f6a280050b0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.125+0000 7f6a383ff700 1 -- 172.21.15.119:0/1161486247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a30103150 msgr2=0x7f6a30073f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:15.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.125+0000 7f6a383ff700 1 --2- 172.21.15.119:0/1161486247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a30103150 0x7f6a30073f40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6a2800d980 tx=0x7f6a2800dc90 comp rx=0 tx=0).stop 2023-12-07T10:20:15.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.126+0000 7f6a383ff700 1 -- 172.21.15.119:0/1161486247 shutdown_connections 2023-12-07T10:20:15.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.126+0000 7f6a383ff700 1 --2- 172.21.15.119:0/1161486247 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f6a1c0a1b10 0x7f6a1c0a3fd0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.126+0000 7f6a383ff700 1 --2- 172.21.15.119:0/1161486247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a30103150 0x7f6a30073f40 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.126+0000 7f6a383ff700 1 --2- 172.21.15.119:0/1161486247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a301015c0 0x7f6a30075320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.126+0000 7f6a383ff700 1 --2- 172.21.15.119:0/1161486247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a301027b0 0x7f6a30073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.126+0000 7f6a383ff700 1 -- 172.21.15.119:0/1161486247 >> 172.21.15.119:0/1161486247 conn(0x7f6a300fcb10 msgr2=0x7f6a30105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:15.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.126+0000 7f6a383ff700 1 -- 172.21.15.119:0/1161486247 shutdown_connections 2023-12-07T10:20:15.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.126+0000 7f6a383ff700 1 -- 172.21.15.119:0/1161486247 wait complete. 2023-12-07T10:20:15.146 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-07T10:20:15.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.349+0000 7f1625dff700 1 -- 172.21.15.119:0/1834117276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1620103150 msgr2=0x7f1620109a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:15.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.349+0000 7f1625dff700 1 --2- 172.21.15.119:0/1834117276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1620103150 0x7f1620109a00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f161800b0d0 tx=0x7f161800b3e0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.350+0000 7f1625dff700 1 -- 172.21.15.119:0/1834117276 shutdown_connections 2023-12-07T10:20:15.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.350+0000 7f1625dff700 1 --2- 172.21.15.119:0/1834117276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1620103150 0x7f1620109a00 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.350+0000 7f1625dff700 1 --2- 172.21.15.119:0/1834117276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16201015c0 0x7f16201019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.350+0000 7f1625dff700 1 --2- 172.21.15.119:0/1834117276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16201027b0 0x7f1620102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.350+0000 7f1625dff700 1 -- 172.21.15.119:0/1834117276 >> 172.21.15.119:0/1834117276 conn(0x7f16200fcb10 msgr2=0x7f16200fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:15.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.350+0000 7f1625dff700 1 -- 172.21.15.119:0/1834117276 shutdown_connections 2023-12-07T10:20:15.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.350+0000 7f1625dff700 1 -- 172.21.15.119:0/1834117276 wait complete. 2023-12-07T10:20:15.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.351+0000 7f1625dff700 1 Processor -- start 2023-12-07T10:20:15.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.351+0000 7f1625dff700 1 -- start start 2023-12-07T10:20:15.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.352+0000 7f1625dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16201015c0 0x7f1620197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.352+0000 7f1625dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16201027b0 0x7f1620197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.352+0000 7f1625dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1620103150 0x7f162019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.352+0000 7f1625dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162019f190 con 0x7f16201015c0 2023-12-07T10:20:15.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.352+0000 7f1625dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162019f2d0 con 0x7f16201027b0 2023-12-07T10:20:15.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.352+0000 7f1625dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162019f460 con 0x7f1620103150 2023-12-07T10:20:15.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.352+0000 7f161f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16201015c0 0x7f1620197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.352+0000 7f161f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16201015c0 0x7f1620197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39346/0 (socket says 172.21.15.119:39346) 2023-12-07T10:20:15.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.352+0000 7f161f7fe700 1 -- 172.21.15.119:0/1641205881 learned_addr learned my addr 172.21.15.119:0/1641205881 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:15.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.352+0000 7f161effd700 1 --2- 172.21.15.119:0/1641205881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16201027b0 0x7f1620197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.352+0000 7f161ffff700 1 --2- 172.21.15.119:0/1641205881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1620103150 0x7f162019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.353+0000 7f161f7fe700 1 -- 172.21.15.119:0/1641205881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1620103150 msgr2=0x7f162019ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:15.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.353+0000 7f161f7fe700 1 --2- 172.21.15.119:0/1641205881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1620103150 0x7f162019ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.353+0000 7f161f7fe700 1 -- 172.21.15.119:0/1641205881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16201027b0 msgr2=0x7f1620197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:15.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.353+0000 7f161f7fe700 1 --2- 172.21.15.119:0/1641205881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16201027b0 0x7f1620197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.353+0000 7f161f7fe700 1 -- 172.21.15.119:0/1641205881 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1618009d00 con 0x7f16201015c0 2023-12-07T10:20:15.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.353+0000 7f161f7fe700 1 --2- 172.21.15.119:0/1641205881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16201015c0 0x7f1620197660 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f161000e970 tx=0x7f161000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:15.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.353+0000 7f161cff9700 1 -- 172.21.15.119:0/1641205881 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1610009790 con 0x7f16201015c0 2023-12-07T10:20:15.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.353+0000 7f161cff9700 1 -- 172.21.15.119:0/1641205881 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f161000c7e0 con 0x7f16201015c0 2023-12-07T10:20:15.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.353+0000 7f1625dff700 1 -- 172.21.15.119:0/1641205881 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f162019f740 con 0x7f16201015c0 2023-12-07T10:20:15.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.353+0000 7f161cff9700 1 -- 172.21.15.119:0/1641205881 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1610003c70 con 0x7f16201015c0 2023-12-07T10:20:15.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.353+0000 7f1625dff700 1 -- 172.21.15.119:0/1641205881 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f162019fca0 con 0x7f16201015c0 2023-12-07T10:20:15.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.357+0000 7f1639740b80 1 Processor -- start 2023-12-07T10:20:15.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.357+0000 7f1639740b80 1 -- start start 2023-12-07T10:20:15.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.358+0000 7f1639740b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608d69a3180 0x5608d69a35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.358+0000 7f1639740b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608d69a3ae0 0x5608d69a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.358+0000 7f1639740b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a49d0 0x5608d69a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.358+0000 7f1639740b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d69177c0 con 0x5608d69a49d0 2023-12-07T10:20:15.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.358+0000 7f1639740b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d68fe7a0 con 0x5608d69a3ae0 2023-12-07T10:20:15.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.358+0000 7f1639740b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d68fc0c0 con 0x5608d69a3180 2023-12-07T10:20:15.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.358+0000 7f161cff9700 1 -- 172.21.15.119:0/1641205881 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 136484+0+0 (secure 0 0 0) 0x7f161000cc50 con 0x7f16201015c0 2023-12-07T10:20:15.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.358+0000 7f161effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a49d0 0x5608d69a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.358+0000 7f161ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608d69a3180 0x5608d69a35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.358+0000 7f161effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a49d0 0x5608d69a4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39348/0 (socket says 172.21.15.119:39348) 2023-12-07T10:20:15.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.358+0000 7f161ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608d69a3180 0x5608d69a35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:45820/0 (socket says 172.21.15.119:45820) 2023-12-07T10:20:15.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.358+0000 7f161effd700 1 -- 172.21.15.119:0/1880090666 learned_addr learned my addr 172.21.15.119:0/1880090666 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:15.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.358+0000 7f161f7fe700 1 --2- 172.21.15.119:0/1880090666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608d69a3ae0 0x5608d69a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.359+0000 7f161effd700 1 -- 172.21.15.119:0/1880090666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608d69a3180 msgr2=0x5608d69a35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:15.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.359+0000 7f161effd700 1 --2- 172.21.15.119:0/1880090666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608d69a3180 0x5608d69a35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.359+0000 7f161effd700 1 -- 172.21.15.119:0/1880090666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608d69a3ae0 msgr2=0x5608d69a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:15.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.359+0000 7f161effd700 1 --2- 172.21.15.119:0/1880090666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608d69a3ae0 0x5608d69a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.359+0000 7f161effd700 1 -- 172.21.15.119:0/1880090666 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608d6774630 con 0x5608d69a49d0 2023-12-07T10:20:15.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.359+0000 7f161effd700 1 --2- 172.21.15.119:0/1880090666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a49d0 0x5608d69a4e50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1614007cf0 tx=0x7f16140103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac91705e4e97955 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:15.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.359+0000 7f1605ffb700 1 -- 172.21.15.119:0/1880090666 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614010d10 con 0x5608d69a49d0 2023-12-07T10:20:15.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.359+0000 7f1605ffb700 1 -- 172.21.15.119:0/1880090666 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1614015070 con 0x5608d69a49d0 2023-12-07T10:20:15.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.359+0000 7f1605ffb700 1 -- 172.21.15.119:0/1880090666 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16140195f0 con 0x5608d69a49d0 2023-12-07T10:20:15.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.360+0000 7f161cff9700 1 --2- 172.21.15.119:0/1641205881 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f16080a1b90 0x7f16080a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.360+0000 7f1639740b80 1 -- 172.21.15.119:0/1880090666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a49d0 msgr2=0x5608d69a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:15.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.360+0000 7f161cff9700 1 -- 172.21.15.119:0/1641205881 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f1610014070 con 0x7f16201015c0 2023-12-07T10:20:15.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.360+0000 7f1639740b80 1 --2- 172.21.15.119:0/1880090666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a49d0 0x5608d69a4e50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1614007cf0 tx=0x7f16140103d0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.360+0000 7f161ffff700 1 --2- 172.21.15.119:0/1641205881 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f16080a1b90 0x7f16080a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.360+0000 7f1639740b80 1 -- 172.21.15.119:0/1880090666 shutdown_connections 2023-12-07T10:20:15.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.360+0000 7f1639740b80 1 --2- 172.21.15.119:0/1880090666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a49d0 0x5608d69a4e50 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.360+0000 7f1639740b80 1 --2- 172.21.15.119:0/1880090666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608d69a3180 0x5608d69a35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.360+0000 7f1639740b80 1 --2- 172.21.15.119:0/1880090666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608d69a3ae0 0x5608d69a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.360+0000 7f1639740b80 1 -- 172.21.15.119:0/1880090666 >> 172.21.15.119:0/1880090666 conn(0x5608d68d2360 msgr2=0x5608d699fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:15.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.360+0000 7f1639740b80 1 -- 172.21.15.119:0/1880090666 shutdown_connections 2023-12-07T10:20:15.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.361+0000 7f1639740b80 1 -- 172.21.15.119:0/1880090666 wait complete. 2023-12-07T10:20:15.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.361+0000 7f161ffff700 1 --2- 172.21.15.119:0/1641205881 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f16080a1b90 0x7f16080a4050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1618009fd0 tx=0x7f161800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:15.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.361+0000 7f1639740b80 1 -- start start 2023-12-07T10:20:15.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.361+0000 7f1639740b80 1 Processor -- start 2023-12-07T10:20:15.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.362+0000 7f1639740b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608d69a3180 0x5608d69b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.362+0000 7f1639740b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608d69a3ae0 0x5608d69b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.362+0000 7f1639740b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a49d0 0x5608d69b8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.362+0000 7f1639740b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d68f79b0 con 0x5608d69a49d0 2023-12-07T10:20:15.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.362+0000 7f1639740b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d68b1560 con 0x5608d69a3ae0 2023-12-07T10:20:15.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.362+0000 7f1639740b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d68b1200 con 0x5608d69a3180 2023-12-07T10:20:15.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.362+0000 7f1639740b80 10 client.0 _ll_register_callbacks cb 0x5608d6898940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:20:15.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.362+0000 7f161ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a49d0 0x5608d69b8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.362+0000 7f161f7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608d69a3180 0x5608d69b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.362+0000 7f161effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608d69a3ae0 0x5608d69b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.362+0000 7f161ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a49d0 0x5608d69b8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39356/0 (socket says 172.21.15.119:39356) 2023-12-07T10:20:15.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.362+0000 7f161ffff700 1 -- 172.21.15.119:0/1847544804 learned_addr learned my addr 172.21.15.119:0/1847544804 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:15.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.363+0000 7f161f7fe700 1 -- 172.21.15.119:0/1847544804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608d69a3ae0 msgr2=0x5608d69b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:15.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.363+0000 7f161f7fe700 1 --2- 172.21.15.119:0/1847544804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608d69a3ae0 0x5608d69b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.363+0000 7f161f7fe700 1 -- 172.21.15.119:0/1847544804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a49d0 msgr2=0x5608d69b8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:15.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.363+0000 7f161f7fe700 1 --2- 172.21.15.119:0/1847544804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a49d0 0x5608d69b8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.363+0000 7f161f7fe700 1 -- 172.21.15.119:0/1847544804 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1614007920 con 0x5608d69a3180 2023-12-07T10:20:15.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.363+0000 7f161effd700 1 --2- 172.21.15.119:0/1847544804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608d69a3ae0 0x5608d69b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:15.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.363+0000 7f161f7fe700 1 --2- 172.21.15.119:0/1847544804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608d69a3180 0x5608d69b13f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f161000efd0 tx=0x7f161009a690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70dc0f44bb6e334f server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:15.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.363+0000 7f15feffd700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:20:15.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.363+0000 7f15feffd700 1 -- 172.21.15.119:0/1847544804 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1610003dd0 con 0x5608d69a3180 2023-12-07T10:20:15.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.363+0000 7f1639740b80 1 -- 172.21.15.119:0/1847544804 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608d6996b00 con 0x5608d69a3180 2023-12-07T10:20:15.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.363+0000 7f15feffd700 1 -- 172.21.15.119:0/1847544804 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f161009aec0 con 0x5608d69a3180 2023-12-07T10:20:15.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.363+0000 7f15feffd700 1 -- 172.21.15.119:0/1847544804 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f161009a7c0 con 0x5608d69a3180 2023-12-07T10:20:15.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.363+0000 7f1639740b80 10 client.25591 fetch_fsmap learned FSMap version 29 2023-12-07T10:20:15.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.363+0000 7f1639740b80 1 -- 172.21.15.119:0/1847544804 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5608d6720de0 con 0x5608d69a3180 2023-12-07T10:20:15.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.364+0000 7f15feffd700 1 -- 172.21.15.119:0/1847544804 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5608d6996b00 con 0x5608d69a3180 2023-12-07T10:20:15.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.364+0000 7f15feffd700 1 -- 172.21.15.119:0/1847544804 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f1610099cb0 con 0x5608d69a3180 2023-12-07T10:20:15.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.364+0000 7f1639740b80 10 client.25591 fetch_fsmap finished waiting for FSMap version 29 2023-12-07T10:20:15.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.364+0000 7f1639740b80 10 client.25591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:20:15.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.364+0000 7f1639740b80 20 client.25591 populate_metadata read hostname 'smithi119' 2023-12-07T10:20:15.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.364+0000 7f1639740b80 1 --2- 172.21.15.119:0/1847544804 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5608d69bd210 0x5608d69bf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.364+0000 7f1639740b80 4 client.25591 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:20:15.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.364+0000 7f1639740b80 1 -- 172.21.15.119:0/1847544804 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608d69727a0 con 0x5608d69bd210 2023-12-07T10:20:15.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.364+0000 7f161effd700 1 --2- 172.21.15.119:0/1847544804 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5608d69bd210 0x5608d69bf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.365+0000 7f161effd700 1 --2- 172.21.15.119:0/1847544804 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5608d69bd210 0x5608d69bf630 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f63c4d4e0a676 server_cookie=fb11f63d443679f3 in_seq=0 out_seq=0 2023-12-07T10:20:15.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.365+0000 7f15feffd700 10 client.25591 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:15.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.365+0000 7ff9d0b9d700 1 -- 172.21.15.119:0/18588951 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7ff9cc1131a0 con 0x7ff9cc102590 2023-12-07T10:20:15.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.366+0000 7f15feffd700 1 -- 172.21.15.119:0/1847544804 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5608d69727a0 con 0x5608d69bd210 2023-12-07T10:20:15.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.366+0000 7f15feffd700 10 client.25591 handle_command_reply: tid=0 2023-12-07T10:20:15.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.366+0000 7f1639740b80 1 client.25591 shutdown 2023-12-07T10:20:15.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.366+0000 7f1639740b80 1 -- 172.21.15.119:0/1847544804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608d69a3180 msgr2=0x5608d69b13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:15.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.366+0000 7f1639740b80 1 --2- 172.21.15.119:0/1847544804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608d69a3180 0x5608d69b13f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f161000efd0 tx=0x7f161009a690 comp rx=0 tx=0).stop 2023-12-07T10:20:15.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.366+0000 7f1639740b80 1 -- 172.21.15.119:0/1847544804 shutdown_connections 2023-12-07T10:20:15.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.366+0000 7f1639740b80 1 --2- 172.21.15.119:0/1847544804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a49d0 0x5608d69b8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.366+0000 7f1639740b80 1 --2- 172.21.15.119:0/1847544804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608d69a3180 0x5608d69b13f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.366+0000 7f1639740b80 1 --2- 172.21.15.119:0/1847544804 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5608d69bd210 0x5608d69bf630 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.366+0000 7f1639740b80 1 --2- 172.21.15.119:0/1847544804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608d69a3ae0 0x5608d69b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.366+0000 7f15feffd700 0 client.25591 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:15.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.366+0000 7f1639740b80 1 -- 172.21.15.119:0/1847544804 >> 172.21.15.119:0/1847544804 conn(0x5608d68d2360 msgr2=0x5608d699e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:15.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.366+0000 7f1639740b80 1 -- 172.21.15.119:0/1847544804 shutdown_connections 2023-12-07T10:20:15.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.366+0000 7f1639740b80 1 -- 172.21.15.119:0/1847544804 wait complete. 2023-12-07T10:20:15.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.366+0000 7f1639740b80 20 client.25591 trim_cache size 0 max 16384 2023-12-07T10:20:15.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.384+0000 7f1639740b80 1 Processor -- start 2023-12-07T10:20:15.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.384+0000 7f1639740b80 1 -- start start 2023-12-07T10:20:15.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.384+0000 7f1639740b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608d69a3180 0x5608d69a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.384+0000 7f1639740b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608d69a3ae0 0x5608d69b8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.384+0000 7f1639740b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a49d0 0x5608d69a78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.384+0000 7f1639740b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d6776920 con 0x5608d69a49d0 2023-12-07T10:20:15.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.384+0000 7f1639740b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d699f550 con 0x5608d69a3ae0 2023-12-07T10:20:15.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.384+0000 7f1639740b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d69727d0 con 0x5608d69a3180 2023-12-07T10:20:15.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.384+0000 7f161ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a49d0 0x5608d69a78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.384+0000 7f161ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a49d0 0x5608d69a78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39370/0 (socket says 172.21.15.119:39370) 2023-12-07T10:20:15.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.384+0000 7f161ffff700 1 -- 172.21.15.119:0/2974198012 learned_addr learned my addr 172.21.15.119:0/2974198012 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:15.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.384+0000 7f161f7fe700 1 --2- 172.21.15.119:0/2974198012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608d69a3180 0x5608d69a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.385+0000 7f161effd700 1 --2- 172.21.15.119:0/2974198012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608d69a3ae0 0x5608d69b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.385+0000 7f161ffff700 1 -- 172.21.15.119:0/2974198012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608d69a3180 msgr2=0x5608d69a4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:15.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.385+0000 7f161ffff700 1 --2- 172.21.15.119:0/2974198012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608d69a3180 0x5608d69a4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.385+0000 7f161ffff700 1 -- 172.21.15.119:0/2974198012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608d69a3ae0 msgr2=0x5608d69b8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:15.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.385+0000 7f161ffff700 1 --2- 172.21.15.119:0/2974198012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608d69a3ae0 0x5608d69b8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.385+0000 7f161ffff700 1 -- 172.21.15.119:0/2974198012 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608d6709c60 con 0x5608d69a49d0 2023-12-07T10:20:15.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.385+0000 7f161ffff700 1 --2- 172.21.15.119:0/2974198012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a49d0 0x5608d69a78a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f16180062a0 tx=0x7f1618020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2efb193d1280361f server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:15.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.385+0000 7f16067fc700 1 -- 172.21.15.119:0/2974198012 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161802c070 con 0x5608d69a49d0 2023-12-07T10:20:15.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.385+0000 7f16067fc700 1 -- 172.21.15.119:0/2974198012 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16180077b0 con 0x5608d69a49d0 2023-12-07T10:20:15.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.385+0000 7f16067fc700 1 -- 172.21.15.119:0/2974198012 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161800e040 con 0x5608d69a49d0 2023-12-07T10:20:15.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.386+0000 7f1639740b80 1 -- 172.21.15.119:0/2974198012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a49d0 msgr2=0x5608d69a78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:15.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.386+0000 7f1639740b80 1 --2- 172.21.15.119:0/2974198012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a49d0 0x5608d69a78a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f16180062a0 tx=0x7f1618020000 comp rx=0 tx=0).stop 2023-12-07T10:20:15.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.386+0000 7f1639740b80 1 -- 172.21.15.119:0/2974198012 shutdown_connections 2023-12-07T10:20:15.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.386+0000 7f1639740b80 1 --2- 172.21.15.119:0/2974198012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a49d0 0x5608d69a78a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.386+0000 7f1639740b80 1 --2- 172.21.15.119:0/2974198012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608d69a3180 0x5608d69a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.386+0000 7f1639740b80 1 --2- 172.21.15.119:0/2974198012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608d69a3ae0 0x5608d69b8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.386+0000 7f1639740b80 1 -- 172.21.15.119:0/2974198012 >> 172.21.15.119:0/2974198012 conn(0x5608d68d2360 msgr2=0x5608d69aaec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:15.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.386+0000 7f1639740b80 1 -- 172.21.15.119:0/2974198012 shutdown_connections 2023-12-07T10:20:15.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.386+0000 7f1639740b80 1 -- 172.21.15.119:0/2974198012 wait complete. 2023-12-07T10:20:15.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.386+0000 7f1639740b80 1 -- start start 2023-12-07T10:20:15.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.387+0000 7f1639740b80 1 Processor -- start 2023-12-07T10:20:15.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.387+0000 7f1639740b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608d69a3180 0x5608d69b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.387+0000 7f1639740b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a3ae0 0x5608d69b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.387+0000 7f1639740b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608d69a49d0 0x5608d69b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.387+0000 7f1639740b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d69b8030 con 0x5608d69a3ae0 2023-12-07T10:20:15.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.387+0000 7f1639740b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d69b81a0 con 0x5608d69a3180 2023-12-07T10:20:15.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.387+0000 7f1639740b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d69b8360 con 0x5608d69a49d0 2023-12-07T10:20:15.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.387+0000 7f1639740b80 10 client.0 _ll_register_callbacks cb 0x5608d6898940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:20:15.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.387+0000 7f161effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a3ae0 0x5608d69b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.387+0000 7f161effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a3ae0 0x5608d69b4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39382/0 (socket says 172.21.15.119:39382) 2023-12-07T10:20:15.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.387+0000 7f161ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608d69a49d0 0x5608d69b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.387+0000 7f161effd700 1 -- 172.21.15.119:0/3329915624 learned_addr learned my addr 172.21.15.119:0/3329915624 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:15.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.387+0000 7f161f7fe700 1 --2- 172.21.15.119:0/3329915624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608d69a3180 0x5608d69b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.388+0000 7f161effd700 1 -- 172.21.15.119:0/3329915624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608d69a49d0 msgr2=0x5608d69b7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:15.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.388+0000 7f161effd700 1 --2- 172.21.15.119:0/3329915624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608d69a49d0 0x5608d69b7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.388+0000 7f161effd700 1 -- 172.21.15.119:0/3329915624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608d69a3180 msgr2=0x5608d69b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:15.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.388+0000 7f161effd700 1 --2- 172.21.15.119:0/3329915624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608d69a3180 0x5608d69b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.388+0000 7f161effd700 1 -- 172.21.15.119:0/3329915624 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f162010a350 con 0x5608d69a3ae0 2023-12-07T10:20:15.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.388+0000 7f161effd700 1 --2- 172.21.15.119:0/3329915624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a3ae0 0x5608d69b4ad0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1614007430 tx=0x7f1614005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ff0fa0eabcb070 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:15.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.388+0000 7f15feffd700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:20:15.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.388+0000 7f15feffd700 1 -- 172.21.15.119:0/3329915624 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614010d90 con 0x5608d69a3ae0 2023-12-07T10:20:15.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.388+0000 7f1639740b80 1 -- 172.21.15.119:0/3329915624 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1614015070 con 0x5608d69a3ae0 2023-12-07T10:20:15.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.388+0000 7f15feffd700 1 -- 172.21.15.119:0/3329915624 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f1614005110 con 0x5608d69a3ae0 2023-12-07T10:20:15.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.388+0000 7f15feffd700 1 -- 172.21.15.119:0/3329915624 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1614005310 con 0x5608d69a3ae0 2023-12-07T10:20:15.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.388+0000 7f1639740b80 10 client.15594 fetch_fsmap learned FSMap version 29 2023-12-07T10:20:15.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.388+0000 7f1639740b80 1 -- 172.21.15.119:0/3329915624 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5608d6747250 con 0x5608d69a3ae0 2023-12-07T10:20:15.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.389+0000 7f15feffd700 1 -- 172.21.15.119:0/3329915624 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1614015070 con 0x5608d69a3ae0 2023-12-07T10:20:15.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.389+0000 7f15feffd700 1 -- 172.21.15.119:0/3329915624 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f1614003680 con 0x5608d69a3ae0 2023-12-07T10:20:15.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.389+0000 7f1639740b80 10 client.15594 fetch_fsmap finished waiting for FSMap version 29 2023-12-07T10:20:15.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.389+0000 7f1639740b80 10 client.15594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:20:15.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.389+0000 7f1639740b80 20 client.15594 populate_metadata read hostname 'smithi119' 2023-12-07T10:20:15.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.389+0000 7f1639740b80 1 --2- 172.21.15.119:0/3329915624 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5608d6a145e0 0x5608d6a16a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:15.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.389+0000 7f1639740b80 4 client.15594 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-12-07T10:20:15.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.389+0000 7f1639740b80 1 -- 172.21.15.119:0/3329915624 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5608d68bdfd0 con 0x5608d6a145e0 2023-12-07T10:20:15.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.389+0000 7f161f7fe700 1 --2- 172.21.15.119:0/3329915624 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5608d6a145e0 0x5608d6a16a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:15.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.390+0000 7f161f7fe700 1 --2- 172.21.15.119:0/3329915624 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5608d6a145e0 0x5608d6a16a00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17754e494ef89776 server_cookie=8b498231e0e192df in_seq=0 out_seq=0 2023-12-07T10:20:15.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.390+0000 7f15feffd700 10 client.15594 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:15.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.390+0000 7f15feffd700 1 -- 172.21.15.119:0/3329915624 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5608d68bdfd0 con 0x5608d6a145e0 2023-12-07T10:20:15.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.390+0000 7f15feffd700 10 client.15594 handle_command_reply: tid=0 2023-12-07T10:20:15.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.390+0000 7f1639740b80 1 client.15594 shutdown 2023-12-07T10:20:15.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.390+0000 7f1639740b80 1 -- 172.21.15.119:0/3329915624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a3ae0 msgr2=0x5608d69b4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:15.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.390+0000 7f1639740b80 1 --2- 172.21.15.119:0/3329915624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a3ae0 0x5608d69b4ad0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1614007430 tx=0x7f1614005b00 comp rx=0 tx=0).stop 2023-12-07T10:20:15.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.390+0000 7f1639740b80 1 -- 172.21.15.119:0/3329915624 shutdown_connections 2023-12-07T10:20:15.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.390+0000 7f1639740b80 1 --2- 172.21.15.119:0/3329915624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608d69a49d0 0x5608d69b7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.390+0000 7f1639740b80 1 --2- 172.21.15.119:0/3329915624 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5608d6a145e0 0x5608d6a16a00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.390+0000 7f1639740b80 1 --2- 172.21.15.119:0/3329915624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5608d69a3180 0x5608d69b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.390+0000 7f1639740b80 1 --2- 172.21.15.119:0/3329915624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608d69a3ae0 0x5608d69b4ad0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.390+0000 7f1639740b80 1 -- 172.21.15.119:0/3329915624 >> 172.21.15.119:0/3329915624 conn(0x5608d68d2360 msgr2=0x5608d69a78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:15.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.390+0000 7f15feffd700 0 client.15594 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:15.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.390+0000 7f1639740b80 1 -- 172.21.15.119:0/3329915624 shutdown_connections 2023-12-07T10:20:15.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.390+0000 7f1639740b80 1 -- 172.21.15.119:0/3329915624 wait complete. 2023-12-07T10:20:15.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.390+0000 7f1639740b80 20 client.15594 trim_cache size 0 max 16384 2023-12-07T10:20:15.397 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:20:15.397 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-07T10:20:15.397 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": { 2023-12-07T10:20:15.397 INFO:teuthology.orchestra.run.smithi119.stdout: "6c28b114-ced4-46ba-9f90-666af450944c": { 2023-12-07T10:20:15.397 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/", 2023-12-07T10:20:15.397 INFO:teuthology.orchestra.run.smithi119.stdout: "tag": "6c28b114-ced4-46ba-9f90-666af450944c", 2023-12-07T10:20:15.397 INFO:teuthology.orchestra.run.smithi119.stdout: "options": "recursive,force" 2023-12-07T10:20:15.397 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:20:15.397 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:20:15.397 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:20:15.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.391+0000 7f1625dff700 1 -- 172.21.15.119:0/1641205881 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f16080a1b90 msgr2=0x7f16080a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:15.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.391+0000 7f1625dff700 1 --2- 172.21.15.119:0/1641205881 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f16080a1b90 0x7f16080a4050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1618009fd0 tx=0x7f161800b040 comp rx=0 tx=0).stop 2023-12-07T10:20:15.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.391+0000 7f1625dff700 1 -- 172.21.15.119:0/1641205881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16201015c0 msgr2=0x7f1620197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:15.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.391+0000 7f1625dff700 1 --2- 172.21.15.119:0/1641205881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16201015c0 0x7f1620197660 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f161000e970 tx=0x7f161000ec80 comp rx=0 tx=0).stop 2023-12-07T10:20:15.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.391+0000 7f1625dff700 1 -- 172.21.15.119:0/1641205881 shutdown_connections 2023-12-07T10:20:15.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.391+0000 7f1625dff700 1 --2- 172.21.15.119:0/1641205881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1620103150 0x7f162019ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.391+0000 7f1625dff700 1 --2- 172.21.15.119:0/1641205881 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f16080a1b90 0x7f16080a4050 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.391+0000 7f1625dff700 1 --2- 172.21.15.119:0/1641205881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16201015c0 0x7f1620197660 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.391+0000 7f1625dff700 1 --2- 172.21.15.119:0/1641205881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16201027b0 0x7f1620197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:15.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.391+0000 7f1625dff700 1 -- 172.21.15.119:0/1641205881 >> 172.21.15.119:0/1641205881 conn(0x7f16200fcb10 msgr2=0x7f1620105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:15.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.391+0000 7f1625dff700 1 -- 172.21.15.119:0/1641205881 shutdown_connections 2023-12-07T10:20:15.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:15.391+0000 7f1625dff700 1 -- 172.21.15.119:0/1641205881 wait complete. 2023-12-07T10:20:15.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:15 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1306722163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:20:15.407 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:15 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1026003140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:20:15.410 INFO:tasks.cephfs.filesystem:scrub status for tag:6c28b114-ced4-46ba-9f90-666af450944c - {'path': '/', 'tag': '6c28b114-ced4-46ba-9f90-666af450944c', 'options': 'recursive,force'} 2023-12-07T10:20:15.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:15 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1306722163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:20:15.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:15 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1026003140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:20:15.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:15 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1306722163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:20:15.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:15 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1026003140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:20:16.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.219+0000 7ff9c8ff9700 1 -- 172.21.15.119:0/18588951 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7ff9c009b4b0 con 0x7ff9cc102590 2023-12-07T10:20:16.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.223+0000 7ff9d0b9d700 1 -- 172.21.15.119:0/18588951 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff9b40a1b10 msgr2=0x7ff9b40a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:16.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.223+0000 7ff9d0b9d700 1 --2- 172.21.15.119:0/18588951 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff9b40a1b10 0x7ff9b40a3fd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009c60 tx=0x7ff9bc009260 comp rx=0 tx=0).stop 2023-12-07T10:20:16.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.223+0000 7ff9d0b9d700 1 -- 172.21.15.119:0/18588951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9cc102590 msgr2=0x7ff9cc19de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:16.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.223+0000 7ff9d0b9d700 1 --2- 172.21.15.119:0/18588951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9cc102590 0x7ff9cc19de80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00098e0 tx=0x7ff9c0005720 comp rx=0 tx=0).stop 2023-12-07T10:20:16.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.223+0000 7ff9d0b9d700 1 -- 172.21.15.119:0/18588951 shutdown_connections 2023-12-07T10:20:16.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.223+0000 7ff9d0b9d700 1 --2- 172.21.15.119:0/18588951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9cc104f60 0x7ff9cc1a4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:16.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.223+0000 7ff9d0b9d700 1 --2- 172.21.15.119:0/18588951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9cc0ffc20 0x7ff9cc19d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:16.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.223+0000 7ff9d0b9d700 1 --2- 172.21.15.119:0/18588951 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7ff9b40a1b10 0x7ff9b40a3fd0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:16.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.223+0000 7ff9d0b9d700 1 --2- 172.21.15.119:0/18588951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9cc102590 0x7ff9cc19de80 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:16.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.223+0000 7ff9d0b9d700 1 -- 172.21.15.119:0/18588951 >> 172.21.15.119:0/18588951 conn(0x7ff9cc0fb820 msgr2=0x7ff9cc1009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:16.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.225+0000 7ff9d0b9d700 1 -- 172.21.15.119:0/18588951 shutdown_connections 2023-12-07T10:20:16.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.225+0000 7ff9d0b9d700 1 -- 172.21.15.119:0/18588951 wait complete. 2023-12-07T10:20:16.253 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-12-07T10:20:16.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.456+0000 7f1c4ebff700 1 -- 172.21.15.119:0/2949447603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c48075650 msgr2=0x7f1c48073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:16.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.456+0000 7f1c4ebff700 1 --2- 172.21.15.119:0/2949447603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c48075650 0x7f1c48073aa0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009990 tx=0x7f1c3c009ca0 comp rx=0 tx=0).stop 2023-12-07T10:20:16.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.456+0000 7f1c4ebff700 1 -- 172.21.15.119:0/2949447603 shutdown_connections 2023-12-07T10:20:16.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.456+0000 7f1c4ebff700 1 --2- 172.21.15.119:0/2949447603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c4810db50 0x7f1c4810ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:16.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.456+0000 7f1c4ebff700 1 --2- 172.21.15.119:0/2949447603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c48075650 0x7f1c48073aa0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:16.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.456+0000 7f1c4ebff700 1 --2- 172.21.15.119:0/2949447603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c48074070 0x7f1c480744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:16.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.456+0000 7f1c4ebff700 1 -- 172.21.15.119:0/2949447603 >> 172.21.15.119:0/2949447603 conn(0x7f1c480fcb10 msgr2=0x7f1c480fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:16.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.456+0000 7f1c4ebff700 1 -- 172.21.15.119:0/2949447603 shutdown_connections 2023-12-07T10:20:16.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.457+0000 7f1c4ebff700 1 -- 172.21.15.119:0/2949447603 wait complete. 2023-12-07T10:20:16.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.458+0000 7f1c4ebff700 1 Processor -- start 2023-12-07T10:20:16.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.458+0000 7f1c4ebff700 1 -- start start 2023-12-07T10:20:16.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.458+0000 7f1c4ebff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c48074070 0x7f1c4819da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:16.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.458+0000 7f1c4ebff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c48075650 0x7f1c4819dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:16.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.458+0000 7f1c4ebff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c4810db50 0x7f1c481a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:16.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.458+0000 7f1c4ebff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c481a55c0 con 0x7f1c48074070 2023-12-07T10:20:16.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.458+0000 7f1c4ebff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c481a5700 con 0x7f1c48075650 2023-12-07T10:20:16.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.458+0000 7f1c4ebff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c481a5890 con 0x7f1c4810db50 2023-12-07T10:20:16.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.458+0000 7f1c4c99b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c48074070 0x7f1c4819da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:16.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.458+0000 7f1c4c99b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c48074070 0x7f1c4819da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39418/0 (socket says 172.21.15.119:39418) 2023-12-07T10:20:16.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.458+0000 7f1c4c99b700 1 -- 172.21.15.119:0/2038064176 learned_addr learned my addr 172.21.15.119:0/2038064176 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:16.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.458+0000 7f1c47fff700 1 --2- 172.21.15.119:0/2038064176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c48075650 0x7f1c4819dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:16.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.458+0000 7f1c4d19c700 1 --2- 172.21.15.119:0/2038064176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c4810db50 0x7f1c481a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:16.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.459+0000 7f1c4c99b700 1 -- 172.21.15.119:0/2038064176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c4810db50 msgr2=0x7f1c481a4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:16.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.459+0000 7f1c4c99b700 1 --2- 172.21.15.119:0/2038064176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c4810db50 0x7f1c481a4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:16.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.459+0000 7f1c4c99b700 1 -- 172.21.15.119:0/2038064176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c48075650 msgr2=0x7f1c4819dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:16.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.459+0000 7f1c4c99b700 1 --2- 172.21.15.119:0/2038064176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c48075650 0x7f1c4819dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:16.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.459+0000 7f1c4c99b700 1 -- 172.21.15.119:0/2038064176 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3c0095f0 con 0x7f1c48074070 2023-12-07T10:20:16.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.459+0000 7f1c4c99b700 1 --2- 172.21.15.119:0/2038064176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c48074070 0x7f1c4819da90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00b680 tx=0x7f1c3c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:16.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.459+0000 7f1c45ffb700 1 -- 172.21.15.119:0/2038064176 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c01d070 con 0x7f1c48074070 2023-12-07T10:20:16.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.459+0000 7f1c4ebff700 1 -- 172.21.15.119:0/2038064176 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c481a5b10 con 0x7f1c48074070 2023-12-07T10:20:16.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.459+0000 7f1c45ffb700 1 -- 172.21.15.119:0/2038064176 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c3c022470 con 0x7f1c48074070 2023-12-07T10:20:16.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.459+0000 7f1c45ffb700 1 -- 172.21.15.119:0/2038064176 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c017650 con 0x7f1c48074070 2023-12-07T10:20:16.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.459+0000 7f1c4ebff700 1 -- 172.21.15.119:0/2038064176 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c481a5f90 con 0x7f1c48074070 2023-12-07T10:20:16.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.461+0000 7f1c4ebff700 1 -- 172.21.15.119:0/2038064176 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c48103db0 con 0x7f1c48074070 2023-12-07T10:20:16.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.462+0000 7f1c45ffb700 1 -- 172.21.15.119:0/2038064176 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 136501+0+0 (secure 0 0 0) 0x7f1c3c0225e0 con 0x7f1c48074070 2023-12-07T10:20:16.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.464+0000 7f1c45ffb700 1 --2- 172.21.15.119:0/2038064176 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1c380a1b60 0x7f1c380a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:16.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.464+0000 7f1c47fff700 1 -- 172.21.15.119:0/2038064176 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1c380a1b60 msgr2=0x7f1c380a4020 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1344846367 2023-12-07T10:20:16.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.464+0000 7f1c45ffb700 1 -- 172.21.15.119:0/2038064176 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f1c3c0cec70 con 0x7f1c48074070 2023-12-07T10:20:16.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.464+0000 7f1c47fff700 1 --2- 172.21.15.119:0/2038064176 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1c380a1b60 0x7f1c380a4020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-07T10:20:16.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.472+0000 7f1c45ffb700 1 -- 172.21.15.119:0/2038064176 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1c3c097d50 con 0x7f1c48074070 2023-12-07T10:20:16.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:16 smithi119 ceph-mon[97908]: pgmap v360: 129 pgs: 129 active+clean; 660 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 682 B/s wr, 10 op/s 2023-12-07T10:20:16.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:16 smithi119 ceph-mon[97908]: scrub queued for path: / 2023-12-07T10:20:16.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:16 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:20:16.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:16 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:20:16.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:16 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:20:16.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:16 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:20:16.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:16 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:20:16.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:16 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/18588951' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-12-07T10:20:16.479 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:16 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: ignoring --setuser ceph since I am not root 2023-12-07T10:20:16.479 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:16 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: ignoring --setgroup ceph since I am not root 2023-12-07T10:20:16.483 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:16 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: ignoring --setuser ceph since I am not root 2023-12-07T10:20:16.483 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:16 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: ignoring --setgroup ceph since I am not root 2023-12-07T10:20:16.483 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:16 smithi159 ceph-mgr[136859]: -- 172.21.15.159:0/600465731 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ea500bc1a0 con 0x55ea50092c00 2023-12-07T10:20:16.483 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:16 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:20:16.400+0000 7f313211c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-07T10:20:16.483 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:16 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:20:16.481+0000 7f313211c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-07T10:20:16.484 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:16 smithi159 ceph-mon[134726]: pgmap v360: 129 pgs: 129 active+clean; 660 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 682 B/s wr, 10 op/s 2023-12-07T10:20:16.484 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:16 smithi159 ceph-mon[134726]: scrub queued for path: / 2023-12-07T10:20:16.484 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:16 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:20:16.484 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:16 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:20:16.484 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:16 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:20:16.484 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:16 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:20:16.484 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:16 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:20:16.484 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:16 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/18588951' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-12-07T10:20:16.485 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:16 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: ignoring --setuser ceph since I am not root 2023-12-07T10:20:16.485 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:16 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: ignoring --setgroup ceph since I am not root 2023-12-07T10:20:16.485 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:16 smithi153 ceph-mgr[135104]: -- 172.21.15.153:0/3031237672 <== mon.2 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5607f58281a0 con 0x5607f57fe800 2023-12-07T10:20:16.485 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:16 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:20:16.402+0000 7f872515d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-07T10:20:16.485 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:16 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:20:16.482+0000 7f872515d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-07T10:20:16.485 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:16 smithi153 ceph-mon[133625]: pgmap v360: 129 pgs: 129 active+clean; 660 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 682 B/s wr, 10 op/s 2023-12-07T10:20:16.485 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:16 smithi153 ceph-mon[133625]: scrub queued for path: / 2023-12-07T10:20:16.485 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:16 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:20:16.485 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:16 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:20:16.485 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:16 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:20:16.485 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:16 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:20:16.485 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:16 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:20:16.486 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:16 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/18588951' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-12-07T10:20:16.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.665+0000 7f1c47fff700 1 -- 172.21.15.119:0/2038064176 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1c380a1b60 msgr2=0x7f1c380a4020 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1344846367 2023-12-07T10:20:16.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.665+0000 7f1c47fff700 1 --2- 172.21.15.119:0/2038064176 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1c380a1b60 0x7f1c380a4020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-07T10:20:16.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.681+0000 7f1c4ebff700 1 -- 172.21.15.119:0/2038064176 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f1c48068490 con 0x7f1c48074070 2023-12-07T10:20:16.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.685+0000 7f1c45ffb700 1 -- 172.21.15.119:0/2038064176 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1c3c09b370 con 0x7f1c48074070 2023-12-07T10:20:16.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.688+0000 7f1c4ebff700 1 -- 172.21.15.119:0/2038064176 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1c380a1b60 msgr2=0x7f1c380a4020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-07T10:20:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.688+0000 7f1c4ebff700 1 --2- 172.21.15.119:0/2038064176 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1c380a1b60 0x7f1c380a4020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.688+0000 7f1c4ebff700 1 -- 172.21.15.119:0/2038064176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c48074070 msgr2=0x7f1c4819da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.688+0000 7f1c4ebff700 1 --2- 172.21.15.119:0/2038064176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c48074070 0x7f1c4819da90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00b680 tx=0x7f1c3c009580 comp rx=0 tx=0).stop 2023-12-07T10:20:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.688+0000 7f1c4ebff700 1 -- 172.21.15.119:0/2038064176 shutdown_connections 2023-12-07T10:20:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.688+0000 7f1c4ebff700 1 --2- 172.21.15.119:0/2038064176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c4810db50 0x7f1c481a4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.688+0000 7f1c4ebff700 1 --2- 172.21.15.119:0/2038064176 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f1c380a1b60 0x7f1c380a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:16.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.688+0000 7f1c4ebff700 1 --2- 172.21.15.119:0/2038064176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c48074070 0x7f1c4819da90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:16.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.688+0000 7f1c4ebff700 1 --2- 172.21.15.119:0/2038064176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c48075650 0x7f1c4819dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:16.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.688+0000 7f1c4ebff700 1 -- 172.21.15.119:0/2038064176 >> 172.21.15.119:0/2038064176 conn(0x7f1c480fcb10 msgr2=0x7f1c48100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:16.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.689+0000 7f1c4ebff700 1 -- 172.21.15.119:0/2038064176 shutdown_connections 2023-12-07T10:20:16.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.689+0000 7f1c4ebff700 1 -- 172.21.15.119:0/2038064176 wait complete. 2023-12-07T10:20:16.710 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-12-07T10:20:16.763 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:16 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:20:16.572+0000 7f313211c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-07T10:20:16.763 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:16 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:20:16.665+0000 7f313211c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-07T10:20:16.771 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:16 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:16.514+0000 7fb69bae3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-07T10:20:16.771 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:16 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:16.621+0000 7fb69bae3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-07T10:20:16.771 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:16 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:16.741+0000 7fb69bae3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-07T10:20:16.779 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:16 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:20:16.575+0000 7f872515d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-07T10:20:16.780 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:16 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:20:16.668+0000 7f872515d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-07T10:20:16.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.907+0000 7f018edff700 1 -- 172.21.15.119:0/3325106933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0188101580 msgr2=0x7f01881019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:16.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.907+0000 7f018edff700 1 --2- 172.21.15.119:0/3325106933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0188101580 0x7f01881019a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f017c009920 tx=0x7f017c009c30 comp rx=0 tx=0).stop 2023-12-07T10:20:16.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.907+0000 7f018edff700 1 -- 172.21.15.119:0/3325106933 shutdown_connections 2023-12-07T10:20:16.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.907+0000 7f018edff700 1 --2- 172.21.15.119:0/3325106933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0188103110 0x7f01881099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:16.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.907+0000 7f018edff700 1 --2- 172.21.15.119:0/3325106933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0188101580 0x7f01881019a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:16.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.907+0000 7f018edff700 1 --2- 172.21.15.119:0/3325106933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0188102770 0x7f0188102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:16.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.907+0000 7f018edff700 1 -- 172.21.15.119:0/3325106933 >> 172.21.15.119:0/3325106933 conn(0x7f01880fcb10 msgr2=0x7f01880fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:16.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.908+0000 7f018edff700 1 -- 172.21.15.119:0/3325106933 shutdown_connections 2023-12-07T10:20:16.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.908+0000 7f018edff700 1 -- 172.21.15.119:0/3325106933 wait complete. 2023-12-07T10:20:16.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.909+0000 7f018edff700 1 Processor -- start 2023-12-07T10:20:16.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.909+0000 7f018edff700 1 -- start start 2023-12-07T10:20:16.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.909+0000 7f018edff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0188101580 0x7f0188199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:16.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.909+0000 7f018edff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0188102770 0x7f0188199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:16.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.909+0000 7f018edff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0188103110 0x7f01881a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:16.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.909+0000 7f018edff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01881a1360 con 0x7f0188102770 2023-12-07T10:20:16.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.909+0000 7f018edff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01881a14a0 con 0x7f0188103110 2023-12-07T10:20:16.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.909+0000 7f018edff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01881a1630 con 0x7f0188101580 2023-12-07T10:20:16.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.909+0000 7f018cb9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0188101580 0x7f0188199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:16.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.909+0000 7f018d39c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0188103110 0x7f01881a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:16.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.909+0000 7f018cb9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0188101580 0x7f0188199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:45892/0 (socket says 172.21.15.119:45892) 2023-12-07T10:20:16.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.909+0000 7f018cb9b700 1 -- 172.21.15.119:0/280773223 learned_addr learned my addr 172.21.15.119:0/280773223 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:16.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.909+0000 7f0187fff700 1 --2- 172.21.15.119:0/280773223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0188102770 0x7f0188199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:16.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.910+0000 7f018cb9b700 1 -- 172.21.15.119:0/280773223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0188103110 msgr2=0x7f01881a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:16.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.910+0000 7f018cb9b700 1 --2- 172.21.15.119:0/280773223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0188103110 0x7f01881a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:16.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.910+0000 7f018cb9b700 1 -- 172.21.15.119:0/280773223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0188102770 msgr2=0x7f0188199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:16.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.910+0000 7f018cb9b700 1 --2- 172.21.15.119:0/280773223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0188102770 0x7f0188199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:16.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.910+0000 7f018cb9b700 1 -- 172.21.15.119:0/280773223 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f017c009520 con 0x7f0188101580 2023-12-07T10:20:16.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.910+0000 7f0187fff700 1 --2- 172.21.15.119:0/280773223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0188102770 0x7f0188199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:20:16.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.910+0000 7f018cb9b700 1 --2- 172.21.15.119:0/280773223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0188101580 0x7f0188199830 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f017c009230 tx=0x7f017c00b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:16.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.910+0000 7f0185ffb700 1 -- 172.21.15.119:0/280773223 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c01c070 con 0x7f0188101580 2023-12-07T10:20:16.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.910+0000 7f0185ffb700 1 -- 172.21.15.119:0/280773223 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f017c00f7d0 con 0x7f0188101580 2023-12-07T10:20:16.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.910+0000 7f018edff700 1 -- 172.21.15.119:0/280773223 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01881a18b0 con 0x7f0188101580 2023-12-07T10:20:16.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.910+0000 7f0185ffb700 1 -- 172.21.15.119:0/280773223 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c004a90 con 0x7f0188101580 2023-12-07T10:20:16.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.910+0000 7f018edff700 1 -- 172.21.15.119:0/280773223 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01881a1cd0 con 0x7f0188101580 2023-12-07T10:20:16.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.911+0000 7f018edff700 1 -- 172.21.15.119:0/280773223 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0188068490 con 0x7f0188101580 2023-12-07T10:20:16.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.913+0000 7f0185ffb700 1 -- 172.21.15.119:0/280773223 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 19) v1 ==== 136501+0+0 (secure 0 0 0) 0x7f017c020070 con 0x7f0188101580 2023-12-07T10:20:16.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.914+0000 7f0185ffb700 1 --2- 172.21.15.119:0/280773223 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f01780a1be0 0x7f01780a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:16.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.915+0000 7f0187fff700 1 -- 172.21.15.119:0/280773223 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f01780a1be0 msgr2=0x7f01780a40a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1344846367 2023-12-07T10:20:16.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.915+0000 7f0185ffb700 1 -- 172.21.15.119:0/280773223 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f017c0ce770 con 0x7f0188101580 2023-12-07T10:20:16.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.915+0000 7f0187fff700 1 --2- 172.21.15.119:0/280773223 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f01780a1be0 0x7f01780a40a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-07T10:20:16.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:16.917+0000 7f0185ffb700 1 -- 172.21.15.119:0/280773223 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f017c097850 con 0x7f0188101580 2023-12-07T10:20:17.101 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:16 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:16.869+0000 7fb69bae3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-07T10:20:17.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.116+0000 7f0187fff700 1 -- 172.21.15.119:0/280773223 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f01780a1be0 msgr2=0x7f01780a40a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1344846367 2023-12-07T10:20:17.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.116+0000 7f0187fff700 1 --2- 172.21.15.119:0/280773223 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f01780a1be0 0x7f01780a40a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-07T10:20:17.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.126+0000 7f018edff700 1 -- 172.21.15.119:0/280773223 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f018810d050 con 0x7f0188101580 2023-12-07T10:20:17.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.132+0000 7f0185ffb700 1 -- 172.21.15.119:0/280773223 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f017c0270c0 con 0x7f0188101580 2023-12-07T10:20:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.134+0000 7f018edff700 1 -- 172.21.15.119:0/280773223 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f01780a1be0 msgr2=0x7f01780a40a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-07T10:20:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.134+0000 7f018edff700 1 --2- 172.21.15.119:0/280773223 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f01780a1be0 0x7f01780a40a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.134+0000 7f018edff700 1 -- 172.21.15.119:0/280773223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0188101580 msgr2=0x7f0188199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.134+0000 7f018edff700 1 --2- 172.21.15.119:0/280773223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0188101580 0x7f0188199830 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f017c009230 tx=0x7f017c00b790 comp rx=0 tx=0).stop 2023-12-07T10:20:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.134+0000 7f018edff700 1 -- 172.21.15.119:0/280773223 shutdown_connections 2023-12-07T10:20:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.135+0000 7f018edff700 1 --2- 172.21.15.119:0/280773223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0188103110 0x7f01881a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.135+0000 7f018edff700 1 --2- 172.21.15.119:0/280773223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0188101580 0x7f0188199830 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:17.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.135+0000 7f018edff700 1 --2- 172.21.15.119:0/280773223 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f01780a1be0 0x7f01780a40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:17.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.135+0000 7f018edff700 1 --2- 172.21.15.119:0/280773223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0188102770 0x7f0188199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:17.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.135+0000 7f018edff700 1 -- 172.21.15.119:0/280773223 >> 172.21.15.119:0/280773223 conn(0x7f01880fcb10 msgr2=0x7f0188107fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:17.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.135+0000 7f018edff700 1 -- 172.21.15.119:0/280773223 shutdown_connections 2023-12-07T10:20:17.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.135+0000 7f018edff700 1 -- 172.21.15.119:0/280773223 wait complete. 2023-12-07T10:20:17.156 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2023-12-07T10:20:17.202 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:16 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:20:16.835+0000 7f313211c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-07T10:20:17.209 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:16 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:20:16.840+0000 7f872515d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-07T10:20:17.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.354+0000 7f49deb9d700 1 -- 172.21.15.119:0/2513490025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49d8101e30 msgr2=0x7f49d8104220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:17.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.354+0000 7f49deb9d700 1 --2- 172.21.15.119:0/2513490025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49d8101e30 0x7f49d8104220 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f49d4009990 tx=0x7f49d4009ca0 comp rx=0 tx=0).stop 2023-12-07T10:20:17.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.354+0000 7f49deb9d700 1 -- 172.21.15.119:0/2513490025 shutdown_connections 2023-12-07T10:20:17.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.354+0000 7f49deb9d700 1 --2- 172.21.15.119:0/2513490025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49d8107090 0x7f49d8109500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:17.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.354+0000 7f49deb9d700 1 --2- 172.21.15.119:0/2513490025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49d8101e30 0x7f49d8104220 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:17.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.354+0000 7f49deb9d700 1 --2- 172.21.15.119:0/2513490025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49d8104760 0x7f49d8106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:17.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.354+0000 7f49deb9d700 1 -- 172.21.15.119:0/2513490025 >> 172.21.15.119:0/2513490025 conn(0x7f49d80fb820 msgr2=0x7f49d80fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:17.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.354+0000 7f49deb9d700 1 -- 172.21.15.119:0/2513490025 shutdown_connections 2023-12-07T10:20:17.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.354+0000 7f49deb9d700 1 -- 172.21.15.119:0/2513490025 wait complete. 2023-12-07T10:20:17.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.356+0000 7f49deb9d700 1 Processor -- start 2023-12-07T10:20:17.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.356+0000 7f49deb9d700 1 -- start start 2023-12-07T10:20:17.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.356+0000 7f49deb9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49d8101e30 0x7f49d81994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:17.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.356+0000 7f49deb9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49d8104760 0x7f49d8199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:17.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.356+0000 7f49deb9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49d8107090 0x7f49d81a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:17.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.356+0000 7f49deb9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d81a1020 con 0x7f49d8101e30 2023-12-07T10:20:17.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.356+0000 7f49deb9d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d81a1160 con 0x7f49d8104760 2023-12-07T10:20:17.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.356+0000 7f49deb9d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d81a12f0 con 0x7f49d8107090 2023-12-07T10:20:17.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.357+0000 7f49dd39a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49d8104760 0x7f49d8199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:17.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.357+0000 7f49de39c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49d8107090 0x7f49d81a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:17.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.357+0000 7f49dd39a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49d8104760 0x7f49d8199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:51598/0 (socket says 172.21.15.119:51598) 2023-12-07T10:20:17.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.357+0000 7f49dd39a700 1 -- 172.21.15.119:0/1753978060 learned_addr learned my addr 172.21.15.119:0/1753978060 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:17.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.357+0000 7f49ddb9b700 1 --2- 172.21.15.119:0/1753978060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49d8101e30 0x7f49d81994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:17.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.357+0000 7f49dd39a700 1 -- 172.21.15.119:0/1753978060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49d8107090 msgr2=0x7f49d81a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:17.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.357+0000 7f49dd39a700 1 --2- 172.21.15.119:0/1753978060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49d8107090 0x7f49d81a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:17.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.357+0000 7f49dd39a700 1 -- 172.21.15.119:0/1753978060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49d8101e30 msgr2=0x7f49d81994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:17.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.357+0000 7f49dd39a700 1 --2- 172.21.15.119:0/1753978060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49d8101e30 0x7f49d81994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:17.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.357+0000 7f49dd39a700 1 -- 172.21.15.119:0/1753978060 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d40095f0 con 0x7f49d8104760 2023-12-07T10:20:17.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.357+0000 7f49de39c700 1 --2- 172.21.15.119:0/1753978060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49d8107090 0x7f49d81a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:17.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.357+0000 7f49ddb9b700 1 --2- 172.21.15.119:0/1753978060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49d8101e30 0x7f49d81994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:17.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.357+0000 7f49dd39a700 1 --2- 172.21.15.119:0/1753978060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49d8104760 0x7f49d8199a30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f49c800e9a0 tx=0x7f49c800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:17.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.358+0000 7f49ceffd700 1 -- 172.21.15.119:0/1753978060 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c8009790 con 0x7f49d8104760 2023-12-07T10:20:17.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.358+0000 7f49ceffd700 1 -- 172.21.15.119:0/1753978060 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49c8010430 con 0x7f49d8104760 2023-12-07T10:20:17.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.358+0000 7f49deb9d700 1 -- 172.21.15.119:0/1753978060 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d81a15d0 con 0x7f49d8104760 2023-12-07T10:20:17.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.358+0000 7f49ceffd700 1 -- 172.21.15.119:0/1753978060 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c8018680 con 0x7f49d8104760 2023-12-07T10:20:17.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.358+0000 7f49deb9d700 1 -- 172.21.15.119:0/1753978060 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d81a1ae0 con 0x7f49d8104760 2023-12-07T10:20:17.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.359+0000 7f49deb9d700 1 -- 172.21.15.119:0/1753978060 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d8108c90 con 0x7f49d8104760 2023-12-07T10:20:17.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.362+0000 7f49ceffd700 1 -- 172.21.15.119:0/1753978060 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 136501+0+0 (secure 0 0 0) 0x7f49c80105a0 con 0x7f49d8104760 2023-12-07T10:20:17.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.364+0000 7f49ceffd700 1 --2- 172.21.15.119:0/1753978060 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f49c40a1b60 0x7f49c40a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:17.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.364+0000 7f49ddb9b700 1 -- 172.21.15.119:0/1753978060 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f49c40a1b60 msgr2=0x7f49c40a4020 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1344846367 2023-12-07T10:20:17.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.364+0000 7f49ceffd700 1 -- 172.21.15.119:0/1753978060 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9115+0+0 (secure 0 0 0) 0x7f49c8014070 con 0x7f49d8104760 2023-12-07T10:20:17.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.364+0000 7f49ddb9b700 1 --2- 172.21.15.119:0/1753978060 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f49c40a1b60 0x7f49c40a4020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-07T10:20:17.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.368+0000 7f49ceffd700 1 -- 172.21.15.119:0/1753978060 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f49c80522f0 con 0x7f49d8104760 2023-12-07T10:20:17.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:17 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/18588951' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-12-07T10:20:17.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:17 smithi119 ceph-mon[97908]: mgrmap e19: x(active, since 9m), standbys: z, y 2023-12-07T10:20:17.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:17 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2038064176' entity='client.admin' 2023-12-07T10:20:17.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:17 smithi119 ceph-mon[97908]: from='client.? ' entity='client.admin' 2023-12-07T10:20:17.479 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:17 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:17.099+0000 7fb69bae3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-07T10:20:17.512 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:17 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:20:17.200+0000 7f313211c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-07T10:20:17.513 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:17 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:20:17.332+0000 7f313211c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-07T10:20:17.513 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:17 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:20:17.422+0000 7f313211c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-07T10:20:17.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:17 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/18588951' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-12-07T10:20:17.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:17 smithi159 ceph-mon[134726]: mgrmap e19: x(active, since 9m), standbys: z, y 2023-12-07T10:20:17.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:17 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2038064176' entity='client.admin' 2023-12-07T10:20:17.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:17 smithi159 ceph-mon[134726]: from='client.? ' entity='client.admin' 2023-12-07T10:20:17.527 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:17 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/18588951' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-12-07T10:20:17.527 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:17 smithi153 ceph-mon[133625]: mgrmap e19: x(active, since 9m), standbys: z, y 2023-12-07T10:20:17.527 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:17 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2038064176' entity='client.admin' 2023-12-07T10:20:17.527 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:17 smithi153 ceph-mon[133625]: from='client.? ' entity='client.admin' 2023-12-07T10:20:17.528 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:17 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:20:17.207+0000 7f872515d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-07T10:20:17.528 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:17 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:20:17.340+0000 7f872515d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-07T10:20:17.528 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:17 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:20:17.430+0000 7f872515d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-07T10:20:17.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.565+0000 7f49ddb9b700 1 -- 172.21.15.119:0/1753978060 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f49c40a1b60 msgr2=0x7f49c40a4020 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1344846367 2023-12-07T10:20:17.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.565+0000 7f49ddb9b700 1 --2- 172.21.15.119:0/1753978060 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f49c40a1b60 0x7f49c40a4020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-07T10:20:17.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.605+0000 7f49deb9d700 1 -- 172.21.15.119:0/1753978060 --> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f49d8061a50 con 0x7f49c40a1b60 2023-12-07T10:20:17.779 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:17 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:20:17.525+0000 7f872515d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-07T10:20:17.870 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:17 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:17.578+0000 7fb69bae3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-07T10:20:17.870 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:17 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:17.747+0000 7fb69bae3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-07T10:20:17.943 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:17 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:20:17.515+0000 7f313211c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-07T10:20:17.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.966+0000 7f49ddb9b700 1 -- 172.21.15.119:0/1753978060 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f49c40a1b60 msgr2=0x7f49c40a4020 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1344846367 2023-12-07T10:20:17.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:17.966+0000 7f49ddb9b700 1 --2- 172.21.15.119:0/1753978060 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f49c40a1b60 0x7f49c40a4020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-07T10:20:18.228 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:17 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:17.868+0000 7fb69bae3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-07T10:20:18.229 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:17 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:17.993+0000 7fb69bae3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-07T10:20:18.262 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:17 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:20:17.940+0000 7f313211c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-07T10:20:18.279 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:17 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:20:17.953+0000 7f872515d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-07T10:20:18.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:18.767+0000 7f49ddb9b700 1 -- 172.21.15.119:0/1753978060 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f49c40a1b60 msgr2=0x7f49c40a4020 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1344846367 2023-12-07T10:20:18.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:18.767+0000 7f49ddb9b700 1 --2- 172.21.15.119:0/1753978060 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f49c40a1b60 0x7f49c40a4020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-07T10:20:18.978 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:18 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:18.528+0000 7fb69bae3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-07T10:20:19.012 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:18 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:20:18.600+0000 7f313211c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-07T10:20:19.013 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:18 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:20:18.682+0000 7f313211c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-07T10:20:19.029 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:18 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:20:18.617+0000 7f872515d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-07T10:20:19.029 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:18 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:20:18.700+0000 7f872515d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-07T10:20:19.728 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:19 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:20:19.462+0000 7f313211c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-07T10:20:19.728 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:19 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:20:19.553+0000 7f313211c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-07T10:20:19.728 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:19 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:19.376+0000 7fb69bae3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-07T10:20:19.728 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:19 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:19.482+0000 7fb69bae3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-07T10:20:19.749 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:19 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:20:19.482+0000 7f872515d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-07T10:20:19.749 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:19 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:20:19.573+0000 7f872515d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-07T10:20:20.013 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:19 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:20:19.726+0000 7f313211c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-07T10:20:20.029 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:19 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:20:19.747+0000 7f872515d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-07T10:20:20.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:20.369+0000 7f49ddb9b700 1 -- 172.21.15.119:0/1753978060 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f49c40a1b60 msgr2=0x7f49c40a4020 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1344846367 2023-12-07T10:20:20.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:20.369+0000 7f49ddb9b700 1 --2- 172.21.15.119:0/1753978060 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f49c40a1b60 0x7f49c40a4020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-07T10:20:20.512 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:20 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:20:20.146+0000 7f313211c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-07T10:20:20.513 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:20 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:20:20.234+0000 7f313211c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-07T10:20:20.513 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:20 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:20:20.383+0000 7f313211c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-07T10:20:20.529 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:20 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:20:20.168+0000 7f872515d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-07T10:20:20.530 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:20 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:20:20.255+0000 7f872515d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-07T10:20:20.530 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:20 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:20:20.404+0000 7f872515d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-07T10:20:20.728 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:20 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:20.443+0000 7fb69bae3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-07T10:20:20.728 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:20 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:20.563+0000 7fb69bae3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-07T10:20:21.013 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:20 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:20:20.648+0000 7f313211c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-07T10:20:21.029 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:20 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:20:20.668+0000 7f872515d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-07T10:20:21.228 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:20 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:20.792+0000 7fb69bae3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-07T10:20:21.636 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:21 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:21.319+0000 7fb69bae3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-07T10:20:21.636 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:21 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:21.436+0000 7fb69bae3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-07T10:20:21.684 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:21 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:20:21.354+0000 7f313211c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-07T10:20:21.684 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:21 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:20:21.485+0000 7f313211c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-07T10:20:21.708 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:21 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:20:21.377+0000 7f872515d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-07T10:20:21.708 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:21 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:20:21.508+0000 7f872515d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-07T10:20:21.978 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:21 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:21.634+0000 7fb69bae3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-07T10:20:22.012 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:21 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:20:21.682+0000 7f313211c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-07T10:20:22.012 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:21 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:20:21.797+0000 7f313211c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-07T10:20:22.029 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:21 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:20:21.706+0000 7f872515d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-07T10:20:22.029 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:21 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:20:21.819+0000 7f872515d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-07T10:20:22.478 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:21 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:21.983+0000 7fb69bae3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-07T10:20:22.512 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:22 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:20:22.015+0000 7f313211c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-07T10:20:22.529 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:22 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:20:22.041+0000 7f872515d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-07T10:20:22.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:22.953+0000 7f49ceffd700 1 -- 172.21.15.119:0/1753978060 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mgrmap(e 20) v1 ==== 136501+0+0 (secure 0 0 0) 0x7f49c8010880 con 0x7f49d8104760 2023-12-07T10:20:23.228 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:22 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:22.913+0000 7fb69bae3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-07T10:20:23.229 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:23 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:23.091+0000 7fb69bae3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-07T10:20:23.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:22 smithi119 ceph-mon[97908]: Standby manager daemon z restarted 2023-12-07T10:20:23.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:22 smithi119 ceph-mon[97908]: Standby manager daemon z started 2023-12-07T10:20:23.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:22 smithi119 ceph-mon[97908]: from='mgr.? 172.21.15.159:0/4021195753' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-07T10:20:23.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:22 smithi119 ceph-mon[97908]: from='mgr.? 172.21.15.159:0/4021195753' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-07T10:20:23.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:22 smithi119 ceph-mon[97908]: from='mgr.? 172.21.15.159:0/4021195753' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-07T10:20:23.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:22 smithi119 ceph-mon[97908]: from='mgr.? 172.21.15.159:0/4021195753' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-07T10:20:23.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:22 smithi119 ceph-mon[97908]: Standby manager daemon y restarted 2023-12-07T10:20:23.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:22 smithi119 ceph-mon[97908]: Standby manager daemon y started 2023-12-07T10:20:23.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:22 smithi119 ceph-mon[97908]: from='mgr.? 172.21.15.153:0/4137898917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-07T10:20:23.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:22 smithi119 ceph-mon[97908]: from='mgr.? 172.21.15.153:0/4137898917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-07T10:20:23.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:22 smithi119 ceph-mon[97908]: from='mgr.? 172.21.15.153:0/4137898917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-07T10:20:23.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:22 smithi119 ceph-mon[97908]: from='mgr.? 172.21.15.153:0/4137898917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-07T10:20:23.262 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:20:22 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z[136834]: 2023-12-07T10:20:22.877+0000 7f313211c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-07T10:20:23.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:22 smithi159 ceph-mon[134726]: Standby manager daemon z restarted 2023-12-07T10:20:23.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:22 smithi159 ceph-mon[134726]: Standby manager daemon z started 2023-12-07T10:20:23.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:22 smithi159 ceph-mon[134726]: from='mgr.? 172.21.15.159:0/4021195753' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-07T10:20:23.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:22 smithi159 ceph-mon[134726]: from='mgr.? 172.21.15.159:0/4021195753' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-07T10:20:23.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:22 smithi159 ceph-mon[134726]: from='mgr.? 172.21.15.159:0/4021195753' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-07T10:20:23.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:22 smithi159 ceph-mon[134726]: from='mgr.? 172.21.15.159:0/4021195753' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-07T10:20:23.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:22 smithi159 ceph-mon[134726]: Standby manager daemon y restarted 2023-12-07T10:20:23.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:22 smithi159 ceph-mon[134726]: Standby manager daemon y started 2023-12-07T10:20:23.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:22 smithi159 ceph-mon[134726]: from='mgr.? 172.21.15.153:0/4137898917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-07T10:20:23.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:22 smithi159 ceph-mon[134726]: from='mgr.? 172.21.15.153:0/4137898917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-07T10:20:23.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:22 smithi159 ceph-mon[134726]: from='mgr.? 172.21.15.153:0/4137898917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-07T10:20:23.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:22 smithi159 ceph-mon[134726]: from='mgr.? 172.21.15.153:0/4137898917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-07T10:20:23.279 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:20:22 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y[135079]: 2023-12-07T10:20:22.905+0000 7f872515d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-07T10:20:23.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:22 smithi153 ceph-mon[133625]: Standby manager daemon z restarted 2023-12-07T10:20:23.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:22 smithi153 ceph-mon[133625]: Standby manager daemon z started 2023-12-07T10:20:23.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:22 smithi153 ceph-mon[133625]: from='mgr.? 172.21.15.159:0/4021195753' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-07T10:20:23.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:22 smithi153 ceph-mon[133625]: from='mgr.? 172.21.15.159:0/4021195753' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-07T10:20:23.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:22 smithi153 ceph-mon[133625]: from='mgr.? 172.21.15.159:0/4021195753' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-07T10:20:23.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:22 smithi153 ceph-mon[133625]: from='mgr.? 172.21.15.159:0/4021195753' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-07T10:20:23.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:22 smithi153 ceph-mon[133625]: Standby manager daemon y restarted 2023-12-07T10:20:23.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:22 smithi153 ceph-mon[133625]: Standby manager daemon y started 2023-12-07T10:20:23.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:22 smithi153 ceph-mon[133625]: from='mgr.? 172.21.15.153:0/4137898917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-07T10:20:23.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:22 smithi153 ceph-mon[133625]: from='mgr.? 172.21.15.153:0/4137898917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-07T10:20:23.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:22 smithi153 ceph-mon[133625]: from='mgr.? 172.21.15.153:0/4137898917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-07T10:20:23.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:22 smithi153 ceph-mon[133625]: from='mgr.? 172.21.15.153:0/4137898917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-07T10:20:23.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:23.572+0000 7f49ddb9b700 1 -- 172.21.15.119:0/1753978060 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f49c40a1b60 msgr2=0x7f49c40a4020 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1344846367 2023-12-07T10:20:23.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:23.572+0000 7f49ddb9b700 1 --2- 172.21.15.119:0/1753978060 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f49c40a1b60 0x7f49c40a4020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-07T10:20:23.728 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:23 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:23.347+0000 7fb69bae3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-07T10:20:23.729 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:23 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:23.494+0000 7fb69bae3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-07T10:20:24.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:23 smithi119 ceph-mon[97908]: mgrmap e20: x(active, since 9m), standbys: z, y 2023-12-07T10:20:24.229 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:23 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:23.776+0000 7fb69bae3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-07T10:20:24.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:23 smithi159 ceph-mon[134726]: mgrmap e20: x(active, since 9m), standbys: z, y 2023-12-07T10:20:24.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:23 smithi153 ceph-mon[133625]: mgrmap e20: x(active, since 9m), standbys: z, y 2023-12-07T10:20:24.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:24.871+0000 7f49ceffd700 1 -- 172.21.15.119:0/1753978060 <== mon.2 v2:172.21.15.153:3300/0 8 ==== mgrmap(e 21) v1 ==== 136168+0+0 (secure 0 0 0) 0x7f49c80cb330 con 0x7f49d8104760 2023-12-07T10:20:24.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:24.871+0000 7f49ceffd700 1 -- 172.21.15.119:0/1753978060 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f49c40a1b60 msgr2=0x7f49c40a4020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-07T10:20:24.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:24.871+0000 7f49ceffd700 1 --2- 172.21.15.119:0/1753978060 >> [v2:172.21.15.119:6800/1344846367,v1:172.21.15.119:6801/1344846367] conn(0x7f49c40a1b60 0x7f49c40a4020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:25.130 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: Active manager daemon x restarted 2023-12-07T10:20:25.130 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: Activating manager daemon x 2023-12-07T10:20:25.130 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: osdmap e94: 12 total, 12 up, 12 in 2023-12-07T10:20:25.130 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: mgrmap e21: x(active, starting, since 0.0118194s), standbys: z, y 2023-12-07T10:20:25.131 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T10:20:25.131 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:20:25.131 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:20:25.131 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-07T10:20:25.131 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-07T10:20:25.131 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-07T10:20:25.131 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-07T10:20:25.131 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-07T10:20:25.131 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-07T10:20:25.131 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-07T10:20:25.131 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-07T10:20:25.131 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-07T10:20:25.131 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-07T10:20:25.131 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-07T10:20:25.131 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-07T10:20:25.132 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-07T10:20:25.132 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-07T10:20:25.132 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-07T10:20:25.132 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T10:20:25.132 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T10:20:25.132 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T10:20:25.132 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T10:20:25.132 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T10:20:25.132 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:24.853+0000 7fb69bae3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-07T10:20:25.132 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:25 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:25.083+0000 7fb5fb36a700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-07T10:20:25.132 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:25 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:25.083+0000 7fb5fb36a700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-07T10:20:25.133 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:25 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:25.083+0000 7fb5fb36a700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-07T10:20:25.133 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:25 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:25.083+0000 7fb5fb36a700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-07T10:20:25.133 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:25 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:25.083+0000 7fb5fb36a700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-07T10:20:25.133 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:25 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:25.085+0000 7fb5fdb6f700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-07T10:20:25.133 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:25 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:25.085+0000 7fb5fdb6f700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-07T10:20:25.133 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:25 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:25.085+0000 7fb5fdb6f700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-07T10:20:25.133 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:25 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:25.085+0000 7fb5fdb6f700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-07T10:20:25.133 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:20:25 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:20:25.085+0000 7fb5fdb6f700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-07T10:20:25.133 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T10:20:25.133 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T10:20:25.133 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T10:20:25.133 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-07T10:20:25.134 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-07T10:20:25.134 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-07T10:20:25.134 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-07T10:20:25.134 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-07T10:20:25.134 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-07T10:20:25.134 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-07T10:20:25.134 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: Manager daemon x is now available 2023-12-07T10:20:25.134 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:20:25.134 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:24 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:20:25.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: Active manager daemon x restarted 2023-12-07T10:20:25.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: Activating manager daemon x 2023-12-07T10:20:25.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: osdmap e94: 12 total, 12 up, 12 in 2023-12-07T10:20:25.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: mgrmap e21: x(active, starting, since 0.0118194s), standbys: z, y 2023-12-07T10:20:25.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T10:20:25.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:20:25.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:20:25.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-07T10:20:25.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-07T10:20:25.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-07T10:20:25.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-07T10:20:25.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-07T10:20:25.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-07T10:20:25.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-07T10:20:25.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-07T10:20:25.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-07T10:20:25.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-07T10:20:25.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-07T10:20:25.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-07T10:20:25.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-07T10:20:25.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-07T10:20:25.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-07T10:20:25.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T10:20:25.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T10:20:25.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T10:20:25.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T10:20:25.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T10:20:25.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T10:20:25.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T10:20:25.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T10:20:25.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-07T10:20:25.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-07T10:20:25.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-07T10:20:25.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-07T10:20:25.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-07T10:20:25.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-07T10:20:25.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-07T10:20:25.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: Manager daemon x is now available 2023-12-07T10:20:25.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:20:25.266 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:24 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:20:25.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: Active manager daemon x restarted 2023-12-07T10:20:25.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: Activating manager daemon x 2023-12-07T10:20:25.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: osdmap e94: 12 total, 12 up, 12 in 2023-12-07T10:20:25.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: mgrmap e21: x(active, starting, since 0.0118194s), standbys: z, y 2023-12-07T10:20:25.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-07T10:20:25.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-07T10:20:25.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-07T10:20:25.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-07T10:20:25.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-07T10:20:25.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-07T10:20:25.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-07T10:20:25.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-07T10:20:25.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-07T10:20:25.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-07T10:20:25.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-07T10:20:25.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-07T10:20:25.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-07T10:20:25.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-07T10:20:25.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-07T10:20:25.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-07T10:20:25.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-07T10:20:25.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-07T10:20:25.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-07T10:20:25.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-07T10:20:25.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-07T10:20:25.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-07T10:20:25.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-07T10:20:25.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-07T10:20:25.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-07T10:20:25.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-07T10:20:25.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-07T10:20:25.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-07T10:20:25.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-07T10:20:25.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-07T10:20:25.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-07T10:20:25.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-07T10:20:25.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-07T10:20:25.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: Manager daemon x is now available 2023-12-07T10:20:25.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:20:25.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:24 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:20:25.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:25.875+0000 7f49ceffd700 1 -- 172.21.15.119:0/1753978060 <== mon.2 v2:172.21.15.153:3300/0 9 ==== mgrmap(e 22) v1 ==== 136295+0+0 (secure 0 0 0) 0x7f49c80ae2e0 con 0x7f49d8104760 2023-12-07T10:20:25.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:25.876+0000 7f49ceffd700 1 --2- 172.21.15.119:0/1753978060 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f49c401b560 0x7f49c401b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:25.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:25.876+0000 7f49ceffd700 1 -- 172.21.15.119:0/1753978060 --> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f49d8061a50 con 0x7f49c401b560 2023-12-07T10:20:25.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:25.878+0000 7f49ddb9b700 1 --2- 172.21.15.119:0/1753978060 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f49c401b560 0x7f49c401b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:25.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:25.881+0000 7f49ddb9b700 1 --2- 172.21.15.119:0/1753978060 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f49c401b560 0x7f49c401b940 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f49d400b2e0 tx=0x7f49d4005fd0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:25.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:25.948+0000 7f49ceffd700 1 -- 172.21.15.119:0/1753978060 <== mgr.15612 v2:172.21.15.119:6800/3763427845 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f49d8061a50 con 0x7f49c401b560 2023-12-07T10:20:25.950 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule set for path / 2023-12-07T10:20:25.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:25.952+0000 7f49deb9d700 1 -- 172.21.15.119:0/1753978060 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f49c401b560 msgr2=0x7f49c401b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:25.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:25.952+0000 7f49deb9d700 1 --2- 172.21.15.119:0/1753978060 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f49c401b560 0x7f49c401b940 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f49d400b2e0 tx=0x7f49d4005fd0 comp rx=0 tx=0).stop 2023-12-07T10:20:25.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:25.952+0000 7f49deb9d700 1 -- 172.21.15.119:0/1753978060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49d8104760 msgr2=0x7f49d8199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:25.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:25.952+0000 7f49deb9d700 1 --2- 172.21.15.119:0/1753978060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49d8104760 0x7f49d8199a30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f49c800e9a0 tx=0x7f49c800ecb0 comp rx=0 tx=0).stop 2023-12-07T10:20:25.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:25.952+0000 7f49de39c700 1 -- 172.21.15.119:0/1753978060 reap_dead start 2023-12-07T10:20:25.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:25.952+0000 7f49deb9d700 1 -- 172.21.15.119:0/1753978060 shutdown_connections 2023-12-07T10:20:25.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:25.952+0000 7f49deb9d700 1 -- 172.21.15.119:0/1753978060 >> 172.21.15.119:0/1753978060 conn(0x7f49d80fb820 msgr2=0x7f49d81016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:25.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:25.952+0000 7f49deb9d700 1 -- 172.21.15.119:0/1753978060 shutdown_connections 2023-12-07T10:20:25.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:25.952+0000 7f49deb9d700 1 -- 172.21.15.119:0/1753978060 wait complete. 2023-12-07T10:20:25.977 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2023-12-07T10:20:26.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.185+0000 7f8d4599d700 1 -- 172.21.15.119:0/2533446048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d40102f60 msgr2=0x7f8d40109810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:26.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.185+0000 7f8d4599d700 1 --2- 172.21.15.119:0/2533446048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d40102f60 0x7f8d40109810 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800b0d0 tx=0x7f8d3800b3e0 comp rx=0 tx=0).stop 2023-12-07T10:20:26.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.186+0000 7f8d4599d700 1 -- 172.21.15.119:0/2533446048 shutdown_connections 2023-12-07T10:20:26.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.186+0000 7f8d4599d700 1 --2- 172.21.15.119:0/2533446048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d40102f60 0x7f8d40109810 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:26.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.186+0000 7f8d4599d700 1 --2- 172.21.15.119:0/2533446048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d401013d0 0x7f8d401017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:26.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.186+0000 7f8d4599d700 1 --2- 172.21.15.119:0/2533446048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d401025c0 0x7f8d40102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:26.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.186+0000 7f8d4599d700 1 -- 172.21.15.119:0/2533446048 >> 172.21.15.119:0/2533446048 conn(0x7f8d400fc940 msgr2=0x7f8d400fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:26.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.186+0000 7f8d4599d700 1 -- 172.21.15.119:0/2533446048 shutdown_connections 2023-12-07T10:20:26.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.186+0000 7f8d4599d700 1 -- 172.21.15.119:0/2533446048 wait complete. 2023-12-07T10:20:26.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.187+0000 7f8d4599d700 1 Processor -- start 2023-12-07T10:20:26.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.187+0000 7f8d4599d700 1 -- start start 2023-12-07T10:20:26.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.188+0000 7f8d4599d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d401013d0 0x7f8d40197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:26.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.188+0000 7f8d4599d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d401025c0 0x7f8d40197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:26.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.188+0000 7f8d4599d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d40102f60 0x7f8d4019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:26.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.188+0000 7f8d4599d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4019ef40 con 0x7f8d401013d0 2023-12-07T10:20:26.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.188+0000 7f8d4599d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4019f080 con 0x7f8d401025c0 2023-12-07T10:20:26.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.188+0000 7f8d4599d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4019f210 con 0x7f8d40102f60 2023-12-07T10:20:26.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.188+0000 7f8d4499b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d401013d0 0x7f8d40197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:26.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.188+0000 7f8d4499b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d401013d0 0x7f8d40197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34782/0 (socket says 172.21.15.119:34782) 2023-12-07T10:20:26.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.188+0000 7f8d4499b700 1 -- 172.21.15.119:0/2435738875 learned_addr learned my addr 172.21.15.119:0/2435738875 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:26.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.188+0000 7f8d4519c700 1 --2- 172.21.15.119:0/2435738875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d40102f60 0x7f8d4019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:26.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.188+0000 7f8d3ffff700 1 --2- 172.21.15.119:0/2435738875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d401025c0 0x7f8d40197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:26.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.188+0000 7f8d4499b700 1 -- 172.21.15.119:0/2435738875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d40102f60 msgr2=0x7f8d4019e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:26.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.188+0000 7f8d4499b700 1 --2- 172.21.15.119:0/2435738875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d40102f60 0x7f8d4019e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:26.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.188+0000 7f8d4499b700 1 -- 172.21.15.119:0/2435738875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d401025c0 msgr2=0x7f8d40197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:26.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.188+0000 7f8d4499b700 1 --2- 172.21.15.119:0/2435738875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d401025c0 0x7f8d40197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:26.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.188+0000 7f8d4499b700 1 -- 172.21.15.119:0/2435738875 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d38009d00 con 0x7f8d401013d0 2023-12-07T10:20:26.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.188+0000 7f8d4519c700 1 --2- 172.21.15.119:0/2435738875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d40102f60 0x7f8d4019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:20:26.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.189+0000 7f8d4499b700 1 --2- 172.21.15.119:0/2435738875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d401013d0 0x7f8d40197410 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400e970 tx=0x7f8d3400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:26.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.189+0000 7f8d3dffb700 1 -- 172.21.15.119:0/2435738875 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d34009790 con 0x7f8d401013d0 2023-12-07T10:20:26.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.189+0000 7f8d3dffb700 1 -- 172.21.15.119:0/2435738875 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d3400eec0 con 0x7f8d401013d0 2023-12-07T10:20:26.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.189+0000 7f8d4599d700 1 -- 172.21.15.119:0/2435738875 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d4019f4f0 con 0x7f8d401013d0 2023-12-07T10:20:26.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.189+0000 7f8d3dffb700 1 -- 172.21.15.119:0/2435738875 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d34003cf0 con 0x7f8d401013d0 2023-12-07T10:20:26.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.189+0000 7f8d4599d700 1 -- 172.21.15.119:0/2435738875 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4019fa00 con 0x7f8d401013d0 2023-12-07T10:20:26.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.190+0000 7f8d4599d700 1 -- 172.21.15.119:0/2435738875 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d40108780 con 0x7f8d401013d0 2023-12-07T10:20:26.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.192+0000 7f8d3dffb700 1 -- 172.21.15.119:0/2435738875 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 22) v1 ==== 136295+0+0 (secure 0 0 0) 0x7f8d3400ccd0 con 0x7f8d401013d0 2023-12-07T10:20:26.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.194+0000 7f8d3dffb700 1 --2- 172.21.15.119:0/2435738875 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8d300a1910 0x7f8d300a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:26.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.194+0000 7f8d3dffb700 1 -- 172.21.15.119:0/2435738875 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f8d34014070 con 0x7f8d401013d0 2023-12-07T10:20:26.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.194+0000 7f8d3ffff700 1 --2- 172.21.15.119:0/2435738875 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8d300a1910 0x7f8d300a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:26.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.194+0000 7f8d3ffff700 1 --2- 172.21.15.119:0/2435738875 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8d300a1910 0x7f8d300a3dd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009c60 tx=0x7f8d2c009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:26.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.198+0000 7f8d3dffb700 1 -- 172.21.15.119:0/2435738875 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8d34096b20 con 0x7f8d401013d0 2023-12-07T10:20:26.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:25 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-07T10:20:26.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:25 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-07T10:20:26.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:25 smithi119 ceph-mon[97908]: [07/Dec/2023:10:20:25] ENGINE Bus STARTING 2023-12-07T10:20:26.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:25 smithi119 ceph-mon[97908]: mgrmap e22: x(active, since 1.02046s), standbys: z, y 2023-12-07T10:20:26.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:25 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-07T10:20:26.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:25 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-07T10:20:26.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:25 smithi159 ceph-mon[134726]: [07/Dec/2023:10:20:25] ENGINE Bus STARTING 2023-12-07T10:20:26.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:25 smithi159 ceph-mon[134726]: mgrmap e22: x(active, since 1.02046s), standbys: z, y 2023-12-07T10:20:26.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:25 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-07T10:20:26.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:25 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-07T10:20:26.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:25 smithi153 ceph-mon[133625]: [07/Dec/2023:10:20:25] ENGINE Bus STARTING 2023-12-07T10:20:26.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:25 smithi153 ceph-mon[133625]: mgrmap e22: x(active, since 1.02046s), standbys: z, y 2023-12-07T10:20:26.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.435+0000 7f8d4599d700 1 -- 172.21.15.119:0/2435738875 --> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d4002d3c0 con 0x7f8d300a1910 2023-12-07T10:20:26.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.489+0000 7f8d3dffb700 1 -- 172.21.15.119:0/2435738875 <== mgr.15612 v2:172.21.15.119:6800/3763427845 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f8d4002d3c0 con 0x7f8d300a1910 2023-12-07T10:20:26.491 INFO:teuthology.orchestra.run.smithi119.stdout:Retention added to path / 2023-12-07T10:20:26.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.492+0000 7f8d4599d700 1 -- 172.21.15.119:0/2435738875 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8d300a1910 msgr2=0x7f8d300a3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:26.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.492+0000 7f8d4599d700 1 --2- 172.21.15.119:0/2435738875 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8d300a1910 0x7f8d300a3dd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009c60 tx=0x7f8d2c009260 comp rx=0 tx=0).stop 2023-12-07T10:20:26.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.492+0000 7f8d4599d700 1 -- 172.21.15.119:0/2435738875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d401013d0 msgr2=0x7f8d40197410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:26.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.492+0000 7f8d4599d700 1 --2- 172.21.15.119:0/2435738875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d401013d0 0x7f8d40197410 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400e970 tx=0x7f8d3400ec80 comp rx=0 tx=0).stop 2023-12-07T10:20:26.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.492+0000 7f8d4599d700 1 -- 172.21.15.119:0/2435738875 shutdown_connections 2023-12-07T10:20:26.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.492+0000 7f8d4599d700 1 --2- 172.21.15.119:0/2435738875 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8d300a1910 0x7f8d300a3dd0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:26.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.492+0000 7f8d4599d700 1 --2- 172.21.15.119:0/2435738875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d40102f60 0x7f8d4019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:26.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.492+0000 7f8d4599d700 1 --2- 172.21.15.119:0/2435738875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d401013d0 0x7f8d40197410 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:26.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.492+0000 7f8d4599d700 1 --2- 172.21.15.119:0/2435738875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d401025c0 0x7f8d40197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:26.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.492+0000 7f8d4599d700 1 -- 172.21.15.119:0/2435738875 >> 172.21.15.119:0/2435738875 conn(0x7f8d400fc940 msgr2=0x7f8d401058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:26.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.492+0000 7f8d4599d700 1 -- 172.21.15.119:0/2435738875 shutdown_connections 2023-12-07T10:20:26.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.492+0000 7f8d4599d700 1 -- 172.21.15.119:0/2435738875 wait complete. 2023-12-07T10:20:26.513 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-12-07T10:20:26.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.714+0000 7f472139d700 1 -- 172.21.15.119:0/2220011110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f471c101e40 msgr2=0x7f471c1086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:26.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.714+0000 7f472139d700 1 --2- 172.21.15.119:0/2220011110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f471c101e40 0x7f471c1086f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4714009fc0 tx=0x7f471400b330 comp rx=0 tx=0).stop 2023-12-07T10:20:26.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.714+0000 7f472139d700 1 -- 172.21.15.119:0/2220011110 shutdown_connections 2023-12-07T10:20:26.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.714+0000 7f472139d700 1 --2- 172.21.15.119:0/2220011110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f471c101e40 0x7f471c1086f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:26.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.714+0000 7f472139d700 1 --2- 172.21.15.119:0/2220011110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f471c1002b0 0x7f471c1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:26.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.714+0000 7f472139d700 1 --2- 172.21.15.119:0/2220011110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f471c1014a0 0x7f471c101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:26.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.714+0000 7f472139d700 1 -- 172.21.15.119:0/2220011110 >> 172.21.15.119:0/2220011110 conn(0x7f471c0fb820 msgr2=0x7f471c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:26.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.714+0000 7f472139d700 1 -- 172.21.15.119:0/2220011110 shutdown_connections 2023-12-07T10:20:26.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.715+0000 7f472139d700 1 -- 172.21.15.119:0/2220011110 wait complete. 2023-12-07T10:20:26.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.715+0000 7f472139d700 1 Processor -- start 2023-12-07T10:20:26.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.716+0000 7f472139d700 1 -- start start 2023-12-07T10:20:26.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.716+0000 7f472139d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f471c1002b0 0x7f471c199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:26.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.716+0000 7f472139d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f471c1014a0 0x7f471c199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.716+0000 7f472139d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f471c101e40 0x7f471c1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.716+0000 7f472139d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471c1a1150 con 0x7f471c1014a0 2023-12-07T10:20:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.716+0000 7f472139d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471c1a1290 con 0x7f471c1002b0 2023-12-07T10:20:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.716+0000 7f472139d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471c1a1420 con 0x7f471c101e40 2023-12-07T10:20:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.716+0000 7f471b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f471c1014a0 0x7f471c199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.716+0000 7f471b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f471c1014a0 0x7f471c199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34804/0 (socket says 172.21.15.119:34804) 2023-12-07T10:20:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.716+0000 7f471b7fe700 1 -- 172.21.15.119:0/1581503195 learned_addr learned my addr 172.21.15.119:0/1581503195 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.716+0000 7f4720b9c700 1 --2- 172.21.15.119:0/1581503195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f471c101e40 0x7f471c1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.716+0000 7f471bfff700 1 --2- 172.21.15.119:0/1581503195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f471c1002b0 0x7f471c199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.717+0000 7f4720b9c700 1 -- 172.21.15.119:0/1581503195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f471c1002b0 msgr2=0x7f471c199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.717+0000 7f4720b9c700 1 --2- 172.21.15.119:0/1581503195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f471c1002b0 0x7f471c199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.717+0000 7f4720b9c700 1 -- 172.21.15.119:0/1581503195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f471c1014a0 msgr2=0x7f471c199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.717+0000 7f4720b9c700 1 --2- 172.21.15.119:0/1581503195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f471c1014a0 0x7f471c199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:26.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.717+0000 7f4720b9c700 1 -- 172.21.15.119:0/1581503195 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4714009c40 con 0x7f471c101e40 2023-12-07T10:20:26.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.717+0000 7f4720b9c700 1 --2- 172.21.15.119:0/1581503195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f471c101e40 0x7f471c1a0b80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f471400b6c0 tx=0x7f4714007730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:26.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.717+0000 7f47197fa700 1 -- 172.21.15.119:0/1581503195 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471400e040 con 0x7f471c101e40 2023-12-07T10:20:26.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.717+0000 7f472139d700 1 -- 172.21.15.119:0/1581503195 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f471c1a16a0 con 0x7f471c101e40 2023-12-07T10:20:26.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.717+0000 7f47197fa700 1 -- 172.21.15.119:0/1581503195 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f471401d470 con 0x7f471c101e40 2023-12-07T10:20:26.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.717+0000 7f47197fa700 1 -- 172.21.15.119:0/1581503195 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47140052a0 con 0x7f471c101e40 2023-12-07T10:20:26.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.717+0000 7f472139d700 1 -- 172.21.15.119:0/1581503195 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f471c1a1b20 con 0x7f471c101e40 2023-12-07T10:20:26.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.718+0000 7f472139d700 1 -- 172.21.15.119:0/1581503195 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f471c04f2d0 con 0x7f471c101e40 2023-12-07T10:20:26.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.720+0000 7f47197fa700 1 -- 172.21.15.119:0/1581503195 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 22) v1 ==== 136295+0+0 (secure 0 0 0) 0x7f471401d850 con 0x7f471c101e40 2023-12-07T10:20:26.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.721+0000 7f47197fa700 1 --2- 172.21.15.119:0/1581503195 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f47040a1910 0x7f47040a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:26.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.721+0000 7f47197fa700 1 -- 172.21.15.119:0/1581503195 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f47140ce9f0 con 0x7f471c101e40 2023-12-07T10:20:26.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.722+0000 7f471bfff700 1 --2- 172.21.15.119:0/1581503195 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f47040a1910 0x7f47040a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:26.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.722+0000 7f471bfff700 1 --2- 172.21.15.119:0/1581503195 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f47040a1910 0x7f47040a3dd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f471c101300 tx=0x7f470c009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:26.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.724+0000 7f47197fa700 1 -- 172.21.15.119:0/1581503195 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4714097ce0 con 0x7f471c101e40 2023-12-07T10:20:26.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.977+0000 7f47197fa700 1 -- 172.21.15.119:0/1581503195 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mgrmap(e 23) v1 ==== 136604+0+0 (secure 0 0 0) 0x7f47140980b0 con 0x7f471c101e40 2023-12-07T10:20:26.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.977+0000 7f472139d700 1 -- 172.21.15.119:0/1581503195 --> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f471c10a760 con 0x7f47040a1910 2023-12-07T10:20:26.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.987+0000 7f47197fa700 1 -- 172.21.15.119:0/1581503195 <== mgr.15612 v2:172.21.15.119:6800/3763427845 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7f471c10a760 con 0x7f47040a1910 2023-12-07T10:20:26.989 INFO:teuthology.orchestra.run.smithi119.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2023-12-07T00:00:00", "created": "2023-12-07T10:20:25", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-12-07T10:20:26.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.991+0000 7f472139d700 1 -- 172.21.15.119:0/1581503195 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f47040a1910 msgr2=0x7f47040a3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:26.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.991+0000 7f472139d700 1 --2- 172.21.15.119:0/1581503195 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f47040a1910 0x7f47040a3dd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f471c101300 tx=0x7f470c009260 comp rx=0 tx=0).stop 2023-12-07T10:20:26.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.991+0000 7f472139d700 1 -- 172.21.15.119:0/1581503195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f471c101e40 msgr2=0x7f471c1a0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:26.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.991+0000 7f472139d700 1 --2- 172.21.15.119:0/1581503195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f471c101e40 0x7f471c1a0b80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f471400b6c0 tx=0x7f4714007730 comp rx=0 tx=0).stop 2023-12-07T10:20:26.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.991+0000 7f472139d700 1 -- 172.21.15.119:0/1581503195 shutdown_connections 2023-12-07T10:20:26.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.991+0000 7f472139d700 1 --2- 172.21.15.119:0/1581503195 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f47040a1910 0x7f47040a3dd0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:26.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.991+0000 7f472139d700 1 --2- 172.21.15.119:0/1581503195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f471c101e40 0x7f471c1a0b80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:26.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.991+0000 7f472139d700 1 --2- 172.21.15.119:0/1581503195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f471c1002b0 0x7f471c199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:26.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.991+0000 7f472139d700 1 --2- 172.21.15.119:0/1581503195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f471c1014a0 0x7f471c199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:26.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.991+0000 7f472139d700 1 -- 172.21.15.119:0/1581503195 >> 172.21.15.119:0/1581503195 conn(0x7f471c0fb820 msgr2=0x7f471c109040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:26.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.991+0000 7f472139d700 1 -- 172.21.15.119:0/1581503195 shutdown_connections 2023-12-07T10:20:26.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:26.991+0000 7f472139d700 1 -- 172.21.15.119:0/1581503195 wait complete. 2023-12-07T10:20:27.014 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-12-07T10:20:27.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.215+0000 7fa14abff700 1 -- 172.21.15.119:0/29266396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa144074070 msgr2=0x7fa1440744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:27.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.215+0000 7fa14abff700 1 --2- 172.21.15.119:0/29266396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa144074070 0x7fa1440744f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa1300098f0 tx=0x7fa130009c00 comp rx=0 tx=0).stop 2023-12-07T10:20:27.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.216+0000 7fa14abff700 1 -- 172.21.15.119:0/29266396 shutdown_connections 2023-12-07T10:20:27.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.216+0000 7fa14abff700 1 --2- 172.21.15.119:0/29266396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa144109440 0x7fa14410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:27.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.216+0000 7fa14abff700 1 --2- 172.21.15.119:0/29266396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa144075650 0x7fa144073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:27.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.216+0000 7fa14abff700 1 --2- 172.21.15.119:0/29266396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa144074070 0x7fa1440744f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:27.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.216+0000 7fa14abff700 1 -- 172.21.15.119:0/29266396 >> 172.21.15.119:0/29266396 conn(0x7fa1440fcb10 msgr2=0x7fa1440fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:27.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.216+0000 7fa14abff700 1 -- 172.21.15.119:0/29266396 shutdown_connections 2023-12-07T10:20:27.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.216+0000 7fa14abff700 1 -- 172.21.15.119:0/29266396 wait complete. 2023-12-07T10:20:27.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.217+0000 7fa14abff700 1 Processor -- start 2023-12-07T10:20:27.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.217+0000 7fa14abff700 1 -- start start 2023-12-07T10:20:27.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.217+0000 7fa14abff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa144074070 0x7fa144199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:27.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.217+0000 7fa14abff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa144075650 0x7fa144199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:27.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.217+0000 7fa14abff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa144109440 0x7fa1441a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:27.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.217+0000 7fa14abff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1441a1290 con 0x7fa144109440 2023-12-07T10:20:27.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.217+0000 7fa14abff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1441a13d0 con 0x7fa144075650 2023-12-07T10:20:27.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.217+0000 7fa14abff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1441a1560 con 0x7fa144074070 2023-12-07T10:20:27.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.218+0000 7fa143fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa144075650 0x7fa144199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:27.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.218+0000 7fa14899b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa144074070 0x7fa144199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:27.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.218+0000 7fa14899b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa144074070 0x7fa144199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:43720/0 (socket says 172.21.15.119:43720) 2023-12-07T10:20:27.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.218+0000 7fa14899b700 1 -- 172.21.15.119:0/1273302 learned_addr learned my addr 172.21.15.119:0/1273302 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:27.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.218+0000 7fa143fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa144075650 0x7fa144199ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:58982/0 (socket says 172.21.15.119:58982) 2023-12-07T10:20:27.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.218+0000 7fa14919c700 1 --2- 172.21.15.119:0/1273302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa144109440 0x7fa1441a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:27.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.218+0000 7fa143fff700 1 -- 172.21.15.119:0/1273302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa144074070 msgr2=0x7fa144199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:27.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.218+0000 7fa143fff700 1 --2- 172.21.15.119:0/1273302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa144074070 0x7fa144199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:27.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.218+0000 7fa143fff700 1 -- 172.21.15.119:0/1273302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa144109440 msgr2=0x7fa1441a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:27.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.218+0000 7fa143fff700 1 --2- 172.21.15.119:0/1273302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa144109440 0x7fa1441a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:27.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.218+0000 7fa143fff700 1 -- 172.21.15.119:0/1273302 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa130009520 con 0x7fa144075650 2023-12-07T10:20:27.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.218+0000 7fa143fff700 1 --2- 172.21.15.119:0/1273302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa144075650 0x7fa144199ca0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa130009230 tx=0x7fa13000b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:27.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.218+0000 7fa141ffb700 1 -- 172.21.15.119:0/1273302 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13001c070 con 0x7fa144075650 2023-12-07T10:20:27.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.219+0000 7fa14abff700 1 -- 172.21.15.119:0/1273302 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1441a17e0 con 0x7fa144075650 2023-12-07T10:20:27.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.219+0000 7fa141ffb700 1 -- 172.21.15.119:0/1273302 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa13000f7d0 con 0x7fa144075650 2023-12-07T10:20:27.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.219+0000 7fa141ffb700 1 -- 172.21.15.119:0/1273302 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1300038c0 con 0x7fa144075650 2023-12-07T10:20:27.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.219+0000 7fa14abff700 1 -- 172.21.15.119:0/1273302 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1441a1c60 con 0x7fa144075650 2023-12-07T10:20:27.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.220+0000 7fa14abff700 1 -- 172.21.15.119:0/1273302 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa144108200 con 0x7fa144075650 2023-12-07T10:20:27.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.222+0000 7fa141ffb700 1 -- 172.21.15.119:0/1273302 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 23) v1 ==== 136604+0+0 (secure 0 0 0) 0x7fa130020070 con 0x7fa144075650 2023-12-07T10:20:27.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.223+0000 7fa141ffb700 1 --2- 172.21.15.119:0/1273302 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa1340a1d00 0x7fa1340a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:27.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.223+0000 7fa141ffb700 1 -- 172.21.15.119:0/1273302 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fa1300ce540 con 0x7fa144075650 2023-12-07T10:20:27.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.223+0000 7fa14899b700 1 --2- 172.21.15.119:0/1273302 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa1340a1d00 0x7fa1340a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:27.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.224+0000 7fa14899b700 1 --2- 172.21.15.119:0/1273302 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa1340a1d00 0x7fa1340a41c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa138009d20 tx=0x7fa138009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:27.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.228+0000 7fa141ffb700 1 -- 172.21.15.119:0/1273302 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa130097360 con 0x7fa144075650 2023-12-07T10:20:27.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:26 smithi159 ceph-mon[134726]: [07/Dec/2023:10:20:25] ENGINE Serving on https://172.21.15.119:7150 2023-12-07T10:20:27.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:26 smithi159 ceph-mon[134726]: [07/Dec/2023:10:20:25] ENGINE Serving on http://172.21.15.119:8765 2023-12-07T10:20:27.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:26 smithi159 ceph-mon[134726]: [07/Dec/2023:10:20:25] ENGINE Bus STARTED 2023-12-07T10:20:27.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:26 smithi159 ceph-mon[134726]: from='client.25655 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:20:27.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:26 smithi159 ceph-mon[134726]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:20:27.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:26 smithi153 ceph-mon[133625]: [07/Dec/2023:10:20:25] ENGINE Serving on https://172.21.15.119:7150 2023-12-07T10:20:27.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:26 smithi153 ceph-mon[133625]: [07/Dec/2023:10:20:25] ENGINE Serving on http://172.21.15.119:8765 2023-12-07T10:20:27.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:26 smithi153 ceph-mon[133625]: [07/Dec/2023:10:20:25] ENGINE Bus STARTED 2023-12-07T10:20:27.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:26 smithi153 ceph-mon[133625]: from='client.25655 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:20:27.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:26 smithi153 ceph-mon[133625]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:20:27.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:26 smithi119 ceph-mon[97908]: [07/Dec/2023:10:20:25] ENGINE Serving on https://172.21.15.119:7150 2023-12-07T10:20:27.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:26 smithi119 ceph-mon[97908]: [07/Dec/2023:10:20:25] ENGINE Serving on http://172.21.15.119:8765 2023-12-07T10:20:27.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:26 smithi119 ceph-mon[97908]: [07/Dec/2023:10:20:25] ENGINE Bus STARTED 2023-12-07T10:20:27.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:26 smithi119 ceph-mon[97908]: from='client.25655 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:20:27.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:26 smithi119 ceph-mon[97908]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:20:27.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.484+0000 7fa14abff700 1 -- 172.21.15.119:0/1273302 --> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fa144061a50 con 0x7fa1340a1d00 2023-12-07T10:20:27.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.492+0000 7fa141ffb700 1 -- 172.21.15.119:0/1273302 <== mgr.15612 v2:172.21.15.119:6800/3763427845 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fa144061a50 con 0x7fa1340a1d00 2023-12-07T10:20:27.493 INFO:teuthology.orchestra.run.smithi119.stdout:/ 1m 6m3h 2023-12-07T10:20:27.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.494+0000 7fa14abff700 1 -- 172.21.15.119:0/1273302 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa1340a1d00 msgr2=0x7fa1340a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:27.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.494+0000 7fa14abff700 1 --2- 172.21.15.119:0/1273302 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa1340a1d00 0x7fa1340a41c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa138009d20 tx=0x7fa138009260 comp rx=0 tx=0).stop 2023-12-07T10:20:27.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.494+0000 7fa14abff700 1 -- 172.21.15.119:0/1273302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa144075650 msgr2=0x7fa144199ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:27.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.494+0000 7fa14abff700 1 --2- 172.21.15.119:0/1273302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa144075650 0x7fa144199ca0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa130009230 tx=0x7fa13000b790 comp rx=0 tx=0).stop 2023-12-07T10:20:27.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.494+0000 7fa14abff700 1 -- 172.21.15.119:0/1273302 shutdown_connections 2023-12-07T10:20:27.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.494+0000 7fa14abff700 1 --2- 172.21.15.119:0/1273302 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa1340a1d00 0x7fa1340a41c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:27.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.494+0000 7fa14abff700 1 --2- 172.21.15.119:0/1273302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa144109440 0x7fa1441a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:27.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.494+0000 7fa14abff700 1 --2- 172.21.15.119:0/1273302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa144074070 0x7fa144199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:27.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.494+0000 7fa14abff700 1 --2- 172.21.15.119:0/1273302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa144075650 0x7fa144199ca0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:27.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.494+0000 7fa14abff700 1 -- 172.21.15.119:0/1273302 >> 172.21.15.119:0/1273302 conn(0x7fa1440fcb10 msgr2=0x7fa144105340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:27.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.494+0000 7fa14abff700 1 -- 172.21.15.119:0/1273302 shutdown_connections 2023-12-07T10:20:27.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:27.494+0000 7fa14abff700 1 -- 172.21.15.119:0/1273302 wait complete. 2023-12-07T10:20:27.514 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-12-07T10:20:27.586 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-12-07T10:20:27.596 INFO:teuthology.run_tasks:Running task background_exec... 2023-12-07T10:20:27.661 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi119.front.sepia.ceph.com 2023-12-07T10:20:27.680 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-12-07T10:20:27.685 INFO:teuthology.run_tasks:Running task workunit... 2023-12-07T10:20:27.705 INFO:tasks.workunit:Pulling workunits from ref 3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 2023-12-07T10:20:27.705 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-12-07T10:20:27.705 INFO:tasks.workunit:timeout=3h 2023-12-07T10:20:27.706 INFO:tasks.workunit:cleanup=True 2023-12-07T10:20:27.706 DEBUG:teuthology.orchestra.run.smithi119:> stat -- /home/ubuntu/cephtest/mnt.0 2023-12-07T10:20:27.751 INFO:teuthology.orchestra.run.smithi119.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-12-07T10:20:27.751 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-12-07T10:20:27.751 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 79h/121d Inode: 1099511628283 Links: 2 2023-12-07T10:20:27.751 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-07T10:20:27.751 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:cephfs_t:s0 2023-12-07T10:20:27.751 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2023-12-07 10:18:13.206616505 +0000 2023-12-07T10:20:27.751 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2023-12-07 10:18:13.206616505 +0000 2023-12-07T10:20:27.751 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2023-12-07 10:20:04.934831767 +0000 2023-12-07T10:20:27.751 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2023-12-07 10:18:13.206616505 +0000 2023-12-07T10:20:27.751 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-12-07T10:20:27.752 DEBUG:teuthology.orchestra.run.smithi119:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-12-07T10:20:27.877 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 2023-12-07T10:20:27.923 INFO:tasks.workunit.client.0.smithi119.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-12-07T10:20:28.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:27 smithi119 ceph-mon[97908]: from='client.15666 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:20:28.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:27 smithi119 ceph-mon[97908]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:20:28.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:27 smithi119 ceph-mon[97908]: mgrmap e23: x(active, since 2s), standbys: z, y 2023-12-07T10:20:28.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:27 smithi119 ceph-mon[97908]: from='client.25678 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:20:28.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:27 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:28.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:27 smithi159 ceph-mon[134726]: from='client.15666 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:20:28.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:27 smithi159 ceph-mon[134726]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:20:28.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:27 smithi159 ceph-mon[134726]: mgrmap e23: x(active, since 2s), standbys: z, y 2023-12-07T10:20:28.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:27 smithi159 ceph-mon[134726]: from='client.25678 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:20:28.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:27 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:28.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:27 smithi153 ceph-mon[133625]: from='client.15666 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:20:28.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:27 smithi153 ceph-mon[133625]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:20:28.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:27 smithi153 ceph-mon[133625]: mgrmap e23: x(active, since 2s), standbys: z, y 2023-12-07T10:20:28.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:27 smithi153 ceph-mon[133625]: from='client.25678 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:20:28.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:27 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:29.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:28 smithi159 ceph-mon[134726]: from='client.25712 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:20:29.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:28 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:29.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:28 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:29.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:28 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:29.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:28 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:20:29.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:28 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:29.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:28 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:29.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:28 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:29.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:28 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:29.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:28 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:29.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:28 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:29.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:28 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:29.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:28 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:29.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:28 smithi153 ceph-mon[133625]: from='client.25712 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:20:29.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:28 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:29.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:28 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:29.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:28 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:29.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:28 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:20:29.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:28 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:29.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:28 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:29.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:28 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:29.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:28 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:29.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:28 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:29.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:28 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:29.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:28 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:29.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:28 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:29.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:28 smithi119 ceph-mon[97908]: from='client.25712 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:20:29.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:28 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:29.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:28 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:29.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:28 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:29.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:28 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:20:29.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:28 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:29.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:28 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:29.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:28 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:29.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:28 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:29.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:28 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:29.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:28 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:29.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:28 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:29.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:28 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:30.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:29 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:20:30.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:29 smithi159 ceph-mon[134726]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:20:30.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:29 smithi159 ceph-mon[134726]: Adjusting osd_memory_target on smithi153 to 196.2M 2023-12-07T10:20:30.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:29 smithi159 ceph-mon[134726]: Unable to set osd_memory_target on smithi153 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-12-07T10:20:30.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:29 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:30.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:29 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:30.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:29 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:30.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:29 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:30.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:29 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:30.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:29 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:30.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:29 smithi159 ceph-mon[134726]: Adjusting osd_memory_target on smithi159 to 196.2M 2023-12-07T10:20:30.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:29 smithi159 ceph-mon[134726]: Unable to set osd_memory_target on smithi159 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-12-07T10:20:30.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:29 smithi159 ceph-mon[134726]: mgrmap e24: x(active, since 4s), standbys: z, y 2023-12-07T10:20:30.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:29 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:20:30.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:29 smithi153 ceph-mon[133625]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:20:30.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:29 smithi153 ceph-mon[133625]: Adjusting osd_memory_target on smithi153 to 196.2M 2023-12-07T10:20:30.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:29 smithi153 ceph-mon[133625]: Unable to set osd_memory_target on smithi153 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-12-07T10:20:30.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:29 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:30.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:29 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:30.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:29 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:30.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:29 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:30.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:29 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:30.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:29 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:30.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:29 smithi153 ceph-mon[133625]: Adjusting osd_memory_target on smithi159 to 196.2M 2023-12-07T10:20:30.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:29 smithi153 ceph-mon[133625]: Unable to set osd_memory_target on smithi159 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-12-07T10:20:30.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:29 smithi153 ceph-mon[133625]: mgrmap e24: x(active, since 4s), standbys: z, y 2023-12-07T10:20:30.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:29 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:20:30.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:29 smithi119 ceph-mon[97908]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:20:30.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:29 smithi119 ceph-mon[97908]: Adjusting osd_memory_target on smithi153 to 196.2M 2023-12-07T10:20:30.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:29 smithi119 ceph-mon[97908]: Unable to set osd_memory_target on smithi153 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-12-07T10:20:30.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:29 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:30.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:29 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:30.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:29 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:30.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:29 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:30.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:29 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:30.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:29 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:30.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:29 smithi119 ceph-mon[97908]: Adjusting osd_memory_target on smithi159 to 196.2M 2023-12-07T10:20:30.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:29 smithi119 ceph-mon[97908]: Unable to set osd_memory_target on smithi159 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-12-07T10:20:30.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:29 smithi119 ceph-mon[97908]: mgrmap e24: x(active, since 4s), standbys: z, y 2023-12-07T10:20:31.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:31 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:31.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:31 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:31.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:31 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:31.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:31 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:31.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:31 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:31.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:31 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:31.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:31 smithi119 ceph-mon[97908]: Adjusting osd_memory_target on smithi119 to 221.9M 2023-12-07T10:20:31.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:31 smithi119 ceph-mon[97908]: Unable to set osd_memory_target on smithi119 to 232752128: error parsing value: Value '232752128' is below minimum 939524096 2023-12-07T10:20:31.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:31 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:20:31.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:31 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:20:31.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:31 smithi119 ceph-mon[97908]: Updating smithi119:/etc/ceph/ceph.conf 2023-12-07T10:20:31.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:31 smithi119 ceph-mon[97908]: Updating smithi153:/etc/ceph/ceph.conf 2023-12-07T10:20:31.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:31 smithi119 ceph-mon[97908]: Updating smithi159:/etc/ceph/ceph.conf 2023-12-07T10:20:31.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:31 smithi119 ceph-mon[97908]: Updating smithi153:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:20:31.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:31 smithi119 ceph-mon[97908]: Updating smithi159:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:20:31.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:31 smithi119 ceph-mon[97908]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:20:31.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:31 smithi119 ceph-mon[97908]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2023-12-07T10:20:31.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:31 smithi119 ceph-mon[97908]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2023-12-07T10:20:31.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:31 smithi119 ceph-mon[97908]: Updating smithi119:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:20:31.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:31 smithi119 ceph-mon[97908]: Updating smithi153:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-07T10:20:31.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:31 smithi119 ceph-mon[97908]: Updating smithi159:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-07T10:20:31.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:31 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:31.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:31 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:31.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:31 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:31.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:31 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:31.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:31 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:31.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:31 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:31.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:31 smithi159 ceph-mon[134726]: Adjusting osd_memory_target on smithi119 to 221.9M 2023-12-07T10:20:31.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:31 smithi159 ceph-mon[134726]: Unable to set osd_memory_target on smithi119 to 232752128: error parsing value: Value '232752128' is below minimum 939524096 2023-12-07T10:20:31.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:31 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:20:31.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:31 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:20:31.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:31 smithi159 ceph-mon[134726]: Updating smithi119:/etc/ceph/ceph.conf 2023-12-07T10:20:31.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:31 smithi159 ceph-mon[134726]: Updating smithi153:/etc/ceph/ceph.conf 2023-12-07T10:20:31.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:31 smithi159 ceph-mon[134726]: Updating smithi159:/etc/ceph/ceph.conf 2023-12-07T10:20:31.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:31 smithi159 ceph-mon[134726]: Updating smithi153:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:20:31.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:31 smithi159 ceph-mon[134726]: Updating smithi159:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:20:31.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:31 smithi159 ceph-mon[134726]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:20:31.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:31 smithi159 ceph-mon[134726]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2023-12-07T10:20:31.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:31 smithi159 ceph-mon[134726]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2023-12-07T10:20:31.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:31 smithi159 ceph-mon[134726]: Updating smithi119:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:20:31.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:31 smithi159 ceph-mon[134726]: Updating smithi153:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-07T10:20:31.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:31 smithi159 ceph-mon[134726]: Updating smithi159:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-07T10:20:31.779 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:31 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:31.779 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:31 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:31.779 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:31 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:31.779 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:31 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:31.779 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:31 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:31.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:31 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:20:31.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:31 smithi153 ceph-mon[133625]: Adjusting osd_memory_target on smithi119 to 221.9M 2023-12-07T10:20:31.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:31 smithi153 ceph-mon[133625]: Unable to set osd_memory_target on smithi119 to 232752128: error parsing value: Value '232752128' is below minimum 939524096 2023-12-07T10:20:31.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:31 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:20:31.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:31 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:20:31.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:31 smithi153 ceph-mon[133625]: Updating smithi119:/etc/ceph/ceph.conf 2023-12-07T10:20:31.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:31 smithi153 ceph-mon[133625]: Updating smithi153:/etc/ceph/ceph.conf 2023-12-07T10:20:31.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:31 smithi153 ceph-mon[133625]: Updating smithi159:/etc/ceph/ceph.conf 2023-12-07T10:20:31.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:31 smithi153 ceph-mon[133625]: Updating smithi153:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:20:31.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:31 smithi153 ceph-mon[133625]: Updating smithi159:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:20:31.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:31 smithi153 ceph-mon[133625]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:20:31.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:31 smithi153 ceph-mon[133625]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2023-12-07T10:20:31.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:31 smithi153 ceph-mon[133625]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2023-12-07T10:20:31.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:31 smithi153 ceph-mon[133625]: Updating smithi119:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-07T10:20:31.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:31 smithi153 ceph-mon[133625]: Updating smithi153:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-07T10:20:31.781 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:31 smithi153 ceph-mon[133625]: Updating smithi159:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-07T10:20:32.728 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:32 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:32.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:32 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:32.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:32 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:32.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:32 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:32.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:32 smithi119 ceph-mon[97908]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2023-12-07T10:20:32.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:32 smithi119 ceph-mon[97908]: Updating smithi119:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-07T10:20:32.762 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:32 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:32.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:32 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:32.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:32 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:32.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:32 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:32.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:32 smithi159 ceph-mon[134726]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2023-12-07T10:20:32.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:32 smithi159 ceph-mon[134726]: Updating smithi119:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-07T10:20:32.779 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:32 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:32.779 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:32 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:32.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:32 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:32.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:32 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:32.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:32 smithi153 ceph-mon[133625]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2023-12-07T10:20:32.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:32 smithi153 ceph-mon[133625]: Updating smithi119:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-07T10:20:33.978 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:33 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:33.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:33 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:33.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:33 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:33.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:33 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:20:33.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:33 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:20:33.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:33 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:20:33.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:33 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:33.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:33 smithi119 ceph-mon[97908]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 132 KiB/s wr, 72 op/s 2023-12-07T10:20:34.012 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:33 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:34.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:33 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:34.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:33 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:34.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:33 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:20:34.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:33 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:20:34.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:33 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:20:34.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:33 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:34.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:33 smithi159 ceph-mon[134726]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 132 KiB/s wr, 72 op/s 2023-12-07T10:20:34.029 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:33 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:34.029 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:33 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:34.029 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:33 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:34.029 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:33 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:20:34.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:33 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:20:34.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:33 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:20:34.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:33 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:20:34.030 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:33 smithi153 ceph-mon[133625]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 132 KiB/s wr, 72 op/s 2023-12-07T10:20:35.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:34 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:20:35.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:34 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:20:35.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:34 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:20:36.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:35 smithi119 ceph-mon[97908]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 103 KiB/s wr, 55 op/s 2023-12-07T10:20:36.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:35 smithi159 ceph-mon[134726]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 103 KiB/s wr, 55 op/s 2023-12-07T10:20:36.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:35 smithi153 ceph-mon[133625]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 103 KiB/s wr, 55 op/s 2023-12-07T10:20:38.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:37 smithi119 ceph-mon[97908]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 50 op/s 2023-12-07T10:20:38.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:37 smithi159 ceph-mon[134726]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 50 op/s 2023-12-07T10:20:38.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:37 smithi153 ceph-mon[133625]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 50 op/s 2023-12-07T10:20:40.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:39 smithi119 ceph-mon[97908]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2023-12-07T10:20:40.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:39 smithi159 ceph-mon[134726]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2023-12-07T10:20:40.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:39 smithi153 ceph-mon[133625]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2023-12-07T10:20:40.937 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.935+0000 7fc8c61ff700 1 -- 172.21.15.119:0/774583005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c01075c0 msgr2=0x7fc8c01079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:40.937 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.935+0000 7fc8c61ff700 1 --2- 172.21.15.119:0/774583005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c01075c0 0x7fc8c01079a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00098f0 tx=0x7fc8b0009c00 comp rx=0 tx=0).stop 2023-12-07T10:20:40.937 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.935+0000 7fc8c61ff700 1 -- 172.21.15.119:0/774583005 shutdown_connections 2023-12-07T10:20:40.937 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.935+0000 7fc8c61ff700 1 --2- 172.21.15.119:0/774583005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8c0101ec0 0x7fc8c010e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.938 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.935+0000 7fc8c61ff700 1 --2- 172.21.15.119:0/774583005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c01075c0 0x7fc8c01079a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.938 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.935+0000 7fc8c61ff700 1 --2- 172.21.15.119:0/774583005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8c0101500 0x7fc8c0101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.938 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.935+0000 7fc8c61ff700 1 -- 172.21.15.119:0/774583005 >> 172.21.15.119:0/774583005 conn(0x7fc8c00fcfb0 msgr2=0x7fc8c00ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:40.938 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.935+0000 7fc8c61ff700 1 -- 172.21.15.119:0/774583005 shutdown_connections 2023-12-07T10:20:40.938 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.935+0000 7fc8c61ff700 1 -- 172.21.15.119:0/774583005 wait complete. 2023-12-07T10:20:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.936+0000 7fc8c61ff700 1 Processor -- start 2023-12-07T10:20:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.936+0000 7fc8c61ff700 1 -- start start 2023-12-07T10:20:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.936+0000 7fc8c61ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c0101500 0x7fc8c019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.936+0000 7fc8c61ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8c0101ec0 0x7fc8c019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.936+0000 7fc8c61ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8c01075c0 0x7fc8c0197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.936+0000 7fc8c61ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c0197e00 con 0x7fc8c0101ec0 2023-12-07T10:20:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.936+0000 7fc8c61ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c0197f70 con 0x7fc8c01075c0 2023-12-07T10:20:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.936+0000 7fc8c61ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c0198130 con 0x7fc8c0101500 2023-12-07T10:20:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.937+0000 7fc8beffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8c0101ec0 0x7fc8c019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.937+0000 7fc8bf7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c0101500 0x7fc8c019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.937+0000 7fc8bf7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c0101500 0x7fc8c019d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:44434/0 (socket says 172.21.15.119:44434) 2023-12-07T10:20:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.937+0000 7fc8bf7fe700 1 -- 172.21.15.119:0/605151066 learned_addr learned my addr 172.21.15.119:0/605151066 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.937+0000 7fc8bffff700 1 --2- 172.21.15.119:0/605151066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8c01075c0 0x7fc8c0197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.937+0000 7fc8beffd700 1 -- 172.21.15.119:0/605151066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c0101500 msgr2=0x7fc8c019d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.937+0000 7fc8beffd700 1 --2- 172.21.15.119:0/605151066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c0101500 0x7fc8c019d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.937+0000 7fc8beffd700 1 -- 172.21.15.119:0/605151066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8c01075c0 msgr2=0x7fc8c0197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.937+0000 7fc8beffd700 1 --2- 172.21.15.119:0/605151066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8c01075c0 0x7fc8c0197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.937+0000 7fc8beffd700 1 -- 172.21.15.119:0/605151066 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b8009d00 con 0x7fc8c0101ec0 2023-12-07T10:20:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.937+0000 7fc8bffff700 1 --2- 172.21.15.119:0/605151066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8c01075c0 0x7fc8c0197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:20:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.937+0000 7fc8bf7fe700 1 --2- 172.21.15.119:0/605151066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c0101500 0x7fc8c019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.937+0000 7fc8beffd700 1 --2- 172.21.15.119:0/605151066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8c0101ec0 0x7fc8c019dba0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400e9f0 tx=0x7fc8b400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.937+0000 7fc8bcff9700 1 -- 172.21.15.119:0/605151066 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b400eee0 con 0x7fc8c0101ec0 2023-12-07T10:20:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.937+0000 7fc8c61ff700 1 -- 172.21.15.119:0/605151066 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b0009520 con 0x7fc8c0101ec0 2023-12-07T10:20:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.937+0000 7fc8bcff9700 1 -- 172.21.15.119:0/605151066 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8b400cbd0 con 0x7fc8c0101ec0 2023-12-07T10:20:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.938+0000 7fc8c61ff700 1 -- 172.21.15.119:0/605151066 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c01a4470 con 0x7fc8c0101ec0 2023-12-07T10:20:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.938+0000 7fc8bcff9700 1 -- 172.21.15.119:0/605151066 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4018620 con 0x7fc8c0101ec0 2023-12-07T10:20:40.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.941+0000 7fc8bcff9700 1 -- 172.21.15.119:0/605151066 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 24) v1 ==== 136660+0+0 (secure 0 0 0) 0x7fc8b400cd40 con 0x7fc8c0101ec0 2023-12-07T10:20:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.942+0000 7fc8bcff9700 1 --2- 172.21.15.119:0/605151066 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fc8a80a1d10 0x7fc8a80a41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.942+0000 7fc8bcff9700 1 -- 172.21.15.119:0/605151066 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fc8b4014070 con 0x7fc8c0101ec0 2023-12-07T10:20:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.942+0000 7fc8d9c18b80 1 Processor -- start 2023-12-07T10:20:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.942+0000 7fc8bf7fe700 1 --2- 172.21.15.119:0/605151066 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fc8a80a1d10 0x7fc8a80a41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.942+0000 7fc8d9c18b80 1 -- start start 2023-12-07T10:20:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.943+0000 7fc8d9c18b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aa8270a60 0x555aa8270e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.943+0000 7fc8d9c18b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555aa8271380 0x555aa826b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.943+0000 7fc8d9c18b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa826bad0 0x555aa826bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.943+0000 7fc8d9c18b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa81df760 con 0x555aa8271380 2023-12-07T10:20:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.943+0000 7fc8d9c18b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa81c6740 con 0x555aa826bad0 2023-12-07T10:20:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.943+0000 7fc8d9c18b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa81c4060 con 0x555aa8270a60 2023-12-07T10:20:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.943+0000 7fc8bf7fe700 1 --2- 172.21.15.119:0/605151066 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fc8a80a1d10 0x7fc8a80a41d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0011040 tx=0x7fc8b0019040 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.943+0000 7fc8beffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555aa8271380 0x555aa826b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.943+0000 7fc8bf7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aa8270a60 0x555aa8270e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.943+0000 7fc8bf7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aa8270a60 0x555aa8270e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:44438/0 (socket says 172.21.15.119:44438) 2023-12-07T10:20:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.943+0000 7fc8beffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555aa8271380 0x555aa826b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39880/0 (socket says 172.21.15.119:39880) 2023-12-07T10:20:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.943+0000 7fc8bffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa826bad0 0x555aa826bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.943+0000 7fc8bf7fe700 1 -- 172.21.15.119:0/2513569443 learned_addr learned my addr 172.21.15.119:0/2513569443 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.944+0000 7fc8beffd700 1 -- 172.21.15.119:0/2513569443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aa8270a60 msgr2=0x555aa8270e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.944+0000 7fc8beffd700 1 --2- 172.21.15.119:0/2513569443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aa8270a60 0x555aa8270e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.944+0000 7fc8beffd700 1 -- 172.21.15.119:0/2513569443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa826bad0 msgr2=0x555aa826bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.944+0000 7fc8beffd700 1 --2- 172.21.15.119:0/2513569443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa826bad0 0x555aa826bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.944+0000 7fc8beffd700 1 -- 172.21.15.119:0/2513569443 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555aa803c5d0 con 0x555aa8271380 2023-12-07T10:20:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.944+0000 7fc8bffff700 1 --2- 172.21.15.119:0/2513569443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa826bad0 0x555aa826bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.944+0000 7fc8beffd700 1 --2- 172.21.15.119:0/2513569443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555aa8271380 0x555aa826b540 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc8b40cb340 tx=0x7fc8b409a4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f661918d2652cc server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.944+0000 7fc8a5ffb700 1 -- 172.21.15.119:0/2513569443 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b409afe0 con 0x555aa8271380 2023-12-07T10:20:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.944+0000 7fc8a5ffb700 1 -- 172.21.15.119:0/2513569443 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8b409a8e0 con 0x555aa8271380 2023-12-07T10:20:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.944+0000 7fc8a5ffb700 1 -- 172.21.15.119:0/2513569443 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b409ada0 con 0x555aa8271380 2023-12-07T10:20:40.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.945+0000 7fc8d9c18b80 1 -- 172.21.15.119:0/2513569443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555aa8271380 msgr2=0x555aa826b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:40.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.945+0000 7fc8d9c18b80 1 --2- 172.21.15.119:0/2513569443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555aa8271380 0x555aa826b540 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc8b40cb340 tx=0x7fc8b409a4a0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.945+0000 7fc8d9c18b80 1 -- 172.21.15.119:0/2513569443 shutdown_connections 2023-12-07T10:20:40.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.945+0000 7fc8d9c18b80 1 --2- 172.21.15.119:0/2513569443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa826bad0 0x555aa826bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.945+0000 7fc8d9c18b80 1 --2- 172.21.15.119:0/2513569443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aa8270a60 0x555aa8270e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.945+0000 7fc8d9c18b80 1 --2- 172.21.15.119:0/2513569443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555aa8271380 0x555aa826b540 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.945+0000 7fc8d9c18b80 1 -- 172.21.15.119:0/2513569443 >> 172.21.15.119:0/2513569443 conn(0x555aa819a300 msgr2=0x555aa8265b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:40.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.945+0000 7fc8d9c18b80 1 -- 172.21.15.119:0/2513569443 shutdown_connections 2023-12-07T10:20:40.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.945+0000 7fc8d9c18b80 1 -- 172.21.15.119:0/2513569443 wait complete. 2023-12-07T10:20:40.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.945+0000 7fc8d9c18b80 1 -- start start 2023-12-07T10:20:40.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.946+0000 7fc8d9c18b80 1 Processor -- start 2023-12-07T10:20:40.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.946+0000 7fc8d9c18b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa826bad0 0x555aa8274710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.946+0000 7fc8d9c18b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555aa8270a60 0x555aa8274c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.946+0000 7fc8d9c18b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aa8271380 0x555aa8275190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.946+0000 7fc8d9c18b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa81bf950 con 0x555aa8270a60 2023-12-07T10:20:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.946+0000 7fc8d9c18b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa8179500 con 0x555aa826bad0 2023-12-07T10:20:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.946+0000 7fc8d9c18b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa81791a0 con 0x555aa8271380 2023-12-07T10:20:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.946+0000 7fc8d9c18b80 10 client.0 _ll_register_callbacks cb 0x555aa81608e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:20:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.946+0000 7fc8bf7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa826bad0 0x555aa8274710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.946+0000 7fc8beffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555aa8270a60 0x555aa8274c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.946+0000 7fc8bf7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa826bad0 0x555aa8274710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:60890/0 (socket says 172.21.15.119:60890) 2023-12-07T10:20:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.946+0000 7fc8bf7fe700 1 -- 172.21.15.119:0/556478933 learned_addr learned my addr 172.21.15.119:0/556478933 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.947+0000 7fc8bffff700 1 --2- 172.21.15.119:0/556478933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aa8271380 0x555aa8275190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.947+0000 7fc8beffd700 1 -- 172.21.15.119:0/556478933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aa8271380 msgr2=0x555aa8275190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.947+0000 7fc8beffd700 1 --2- 172.21.15.119:0/556478933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aa8271380 0x555aa8275190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.947+0000 7fc8beffd700 1 -- 172.21.15.119:0/556478933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa826bad0 msgr2=0x555aa8274710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.947+0000 7fc8beffd700 1 --2- 172.21.15.119:0/556478933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa826bad0 0x555aa8274710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.947+0000 7fc8beffd700 1 -- 172.21.15.119:0/556478933 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b8009190 con 0x555aa8270a60 2023-12-07T10:20:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.947+0000 7fc8beffd700 1 --2- 172.21.15.119:0/556478933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555aa8270a60 0x555aa8274c50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc8b40a2e30 tx=0x7fc8b409a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f51e938d806ed70 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.947+0000 7fc89effd700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:20:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.947+0000 7fc89effd700 1 -- 172.21.15.119:0/556478933 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b40a2c70 con 0x555aa8270a60 2023-12-07T10:20:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.947+0000 7fc89effd700 1 -- 172.21.15.119:0/556478933 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fc8b40ba0f0 con 0x555aa8270a60 2023-12-07T10:20:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.947+0000 7fc8d9c18b80 1 -- 172.21.15.119:0/556478933 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555aa825e6f0 con 0x555aa8270a60 2023-12-07T10:20:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.947+0000 7fc89effd700 1 -- 172.21.15.119:0/556478933 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8b40ba2b0 con 0x555aa8270a60 2023-12-07T10:20:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.947+0000 7fc8d9c18b80 10 client.15696 fetch_fsmap learned FSMap version 30 2023-12-07T10:20:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.947+0000 7fc8d9c18b80 1 -- 172.21.15.119:0/556478933 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fc8c01a4470 con 0x555aa8270a60 2023-12-07T10:20:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.948+0000 7fc89effd700 1 -- 172.21.15.119:0/556478933 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8b4064080 con 0x555aa8270a60 2023-12-07T10:20:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.948+0000 7fc89effd700 1 -- 172.21.15.119:0/556478933 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc8b40cbb50 con 0x555aa8270a60 2023-12-07T10:20:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.948+0000 7fc8d9c18b80 10 client.15696 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:20:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.948+0000 7fc8d9c18b80 10 client.15696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:20:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.948+0000 7fc8d9c18b80 20 client.15696 populate_metadata read hostname 'smithi119' 2023-12-07T10:20:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.948+0000 7fc8d9c18b80 1 --2- 172.21.15.119:0/556478933 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555aa8284b90 0x555aa8286fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.948+0000 7fc8d9c18b80 4 client.15696 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:20:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.948+0000 7fc8d9c18b80 1 -- 172.21.15.119:0/556478933 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555aa823a740 con 0x555aa8284b90 2023-12-07T10:20:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.948+0000 7fc8bf7fe700 1 --2- 172.21.15.119:0/556478933 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555aa8284b90 0x555aa8286fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.949+0000 7fc8bf7fe700 1 --2- 172.21.15.119:0/556478933 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555aa8284b90 0x555aa8286fb0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3291dd0142425e1b server_cookie=e4cb309d9ca062d9 in_seq=0 out_seq=0 2023-12-07T10:20:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.949+0000 7fc89effd700 10 client.15696 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.950+0000 7fc89effd700 1 -- 172.21.15.119:0/556478933 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555aa823a740 con 0x555aa8284b90 2023-12-07T10:20:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.950+0000 7fc89effd700 10 client.15696 handle_command_reply: tid=0 2023-12-07T10:20:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.950+0000 7fc8d9c18b80 1 client.15696 shutdown 2023-12-07T10:20:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.950+0000 7fc8d9c18b80 1 -- 172.21.15.119:0/556478933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555aa8270a60 msgr2=0x555aa8274c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.950+0000 7fc8d9c18b80 1 --2- 172.21.15.119:0/556478933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555aa8270a60 0x555aa8274c50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc8b40a2e30 tx=0x7fc8b409a180 comp rx=0 tx=0).stop 2023-12-07T10:20:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.950+0000 7fc8d9c18b80 1 -- 172.21.15.119:0/556478933 shutdown_connections 2023-12-07T10:20:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.950+0000 7fc8d9c18b80 1 --2- 172.21.15.119:0/556478933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aa8271380 0x555aa8275190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.950+0000 7fc8d9c18b80 1 --2- 172.21.15.119:0/556478933 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555aa8284b90 0x555aa8286fb0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.950+0000 7fc8d9c18b80 1 --2- 172.21.15.119:0/556478933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa826bad0 0x555aa8274710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.950+0000 7fc8d9c18b80 1 --2- 172.21.15.119:0/556478933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555aa8270a60 0x555aa8274c50 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.950+0000 7fc89effd700 0 client.15696 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.950+0000 7fc8d9c18b80 1 -- 172.21.15.119:0/556478933 >> 172.21.15.119:0/556478933 conn(0x555aa819a300 msgr2=0x555aa8260980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.950+0000 7fc8d9c18b80 1 -- 172.21.15.119:0/556478933 shutdown_connections 2023-12-07T10:20:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.950+0000 7fc8d9c18b80 1 -- 172.21.15.119:0/556478933 wait complete. 2023-12-07T10:20:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.950+0000 7fc8d9c18b80 20 client.15696 trim_cache size 0 max 16384 2023-12-07T10:20:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.965+0000 7fc8d9c18b80 1 Processor -- start 2023-12-07T10:20:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.966+0000 7fc8d9c18b80 1 -- start start 2023-12-07T10:20:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.966+0000 7fc8d9c18b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aa826bad0 0x555aa8270e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.966+0000 7fc8d9c18b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555aa8270a60 0x555aa826b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.966+0000 7fc8d9c18b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa8271380 0x555aa826df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.966+0000 7fc8d9c18b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa803e8c0 con 0x555aa8270a60 2023-12-07T10:20:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.966+0000 7fc8d9c18b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa8266ea0 con 0x555aa8271380 2023-12-07T10:20:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.966+0000 7fc8d9c18b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa82778d0 con 0x555aa826bad0 2023-12-07T10:20:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.966+0000 7fc8bf7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aa826bad0 0x555aa8270e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.966+0000 7fc8bf7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aa826bad0 0x555aa8270e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:44456/0 (socket says 172.21.15.119:44456) 2023-12-07T10:20:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.966+0000 7fc8bf7fe700 1 -- 172.21.15.119:0/4194041290 learned_addr learned my addr 172.21.15.119:0/4194041290 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.966+0000 7fc8bffff700 1 --2- 172.21.15.119:0/4194041290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa8271380 0x555aa826df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.966+0000 7fc8beffd700 1 --2- 172.21.15.119:0/4194041290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555aa8270a60 0x555aa826b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.967+0000 7fc8bf7fe700 1 -- 172.21.15.119:0/4194041290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa8271380 msgr2=0x555aa826df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.967+0000 7fc8bf7fe700 1 --2- 172.21.15.119:0/4194041290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa8271380 0x555aa826df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.967+0000 7fc8bf7fe700 1 -- 172.21.15.119:0/4194041290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555aa8270a60 msgr2=0x555aa826b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.967+0000 7fc8bf7fe700 1 --2- 172.21.15.119:0/4194041290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555aa8270a60 0x555aa826b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.967+0000 7fc8bf7fe700 1 -- 172.21.15.119:0/4194041290 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555aa7fe8d80 con 0x555aa826bad0 2023-12-07T10:20:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.967+0000 7fc8bf7fe700 1 --2- 172.21.15.119:0/4194041290 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aa826bad0 0x555aa8270e40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0009810 tx=0x7fc8b0023000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=374a68910f0528ed server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.967+0000 7fc8a67fc700 1 -- 172.21.15.119:0/4194041290 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b00050b0 con 0x555aa826bad0 2023-12-07T10:20:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.967+0000 7fc8a67fc700 1 -- 172.21.15.119:0/4194041290 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8b001d040 con 0x555aa826bad0 2023-12-07T10:20:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.967+0000 7fc8a67fc700 1 -- 172.21.15.119:0/4194041290 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0003d50 con 0x555aa826bad0 2023-12-07T10:20:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.968+0000 7fc8d9c18b80 1 -- 172.21.15.119:0/4194041290 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aa826bad0 msgr2=0x555aa8270e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.968+0000 7fc8d9c18b80 1 --2- 172.21.15.119:0/4194041290 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aa826bad0 0x555aa8270e40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0009810 tx=0x7fc8b0023000 comp rx=0 tx=0).stop 2023-12-07T10:20:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.968+0000 7fc8d9c18b80 1 -- 172.21.15.119:0/4194041290 shutdown_connections 2023-12-07T10:20:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.968+0000 7fc8d9c18b80 1 --2- 172.21.15.119:0/4194041290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa8271380 0x555aa826df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.968+0000 7fc8d9c18b80 1 --2- 172.21.15.119:0/4194041290 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aa826bad0 0x555aa8270e40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.968+0000 7fc8d9c18b80 1 --2- 172.21.15.119:0/4194041290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555aa8270a60 0x555aa826b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.968+0000 7fc8d9c18b80 1 -- 172.21.15.119:0/4194041290 >> 172.21.15.119:0/4194041290 conn(0x555aa819a300 msgr2=0x555aa825f960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.968+0000 7fc8d9c18b80 1 -- 172.21.15.119:0/4194041290 shutdown_connections 2023-12-07T10:20:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.968+0000 7fc8d9c18b80 1 -- 172.21.15.119:0/4194041290 wait complete. 2023-12-07T10:20:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.968+0000 7fc8d9c18b80 1 -- start start 2023-12-07T10:20:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.968+0000 7fc8d9c18b80 1 Processor -- start 2023-12-07T10:20:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.969+0000 7fc8d9c18b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555aa826bad0 0x555aa827c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.969+0000 7fc8d9c18b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa8270a60 0x555aa8277e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.969+0000 7fc8d9c18b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aa8271380 0x555aa8278430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.969+0000 7fc8d9c18b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa8278b10 con 0x555aa826bad0 2023-12-07T10:20:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.969+0000 7fc8d9c18b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa8278c50 con 0x555aa8270a60 2023-12-07T10:20:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.969+0000 7fc8d9c18b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aa82d3030 con 0x555aa8271380 2023-12-07T10:20:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.969+0000 7fc8d9c18b80 10 client.0 _ll_register_callbacks cb 0x555aa81608e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:20:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.969+0000 7fc8bffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aa8271380 0x555aa8278430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.969+0000 7fc8bf7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555aa826bad0 0x555aa827c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.969+0000 7fc8bffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aa8271380 0x555aa8278430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:44458/0 (socket says 172.21.15.119:44458) 2023-12-07T10:20:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.969+0000 7fc8bffff700 1 -- 172.21.15.119:0/4167160951 learned_addr learned my addr 172.21.15.119:0/4167160951 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.969+0000 7fc8beffd700 1 --2- 172.21.15.119:0/4167160951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa8270a60 0x555aa8277e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.969+0000 7fc8bf7fe700 1 -- 172.21.15.119:0/4167160951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aa8271380 msgr2=0x555aa8278430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.969+0000 7fc8bf7fe700 1 --2- 172.21.15.119:0/4167160951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aa8271380 0x555aa8278430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.969+0000 7fc8bf7fe700 1 -- 172.21.15.119:0/4167160951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa8270a60 msgr2=0x555aa8277e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.969+0000 7fc8bf7fe700 1 --2- 172.21.15.119:0/4167160951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa8270a60 0x555aa8277e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.969+0000 7fc8bf7fe700 1 -- 172.21.15.119:0/4167160951 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b0028040 con 0x555aa826bad0 2023-12-07T10:20:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.969+0000 7fc8bffff700 1 --2- 172.21.15.119:0/4167160951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aa8271380 0x555aa8278430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.970+0000 7fc8bf7fe700 1 --2- 172.21.15.119:0/4167160951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555aa826bad0 0x555aa827c1d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00098c0 tx=0x7fc8b000f700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a3d088d5132a1c server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.970+0000 7fc89effd700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:20:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.970+0000 7fc89effd700 1 -- 172.21.15.119:0/4167160951 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b00039f0 con 0x555aa826bad0 2023-12-07T10:20:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.970+0000 7fc89effd700 1 -- 172.21.15.119:0/4167160951 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fc8b0003b50 con 0x555aa826bad0 2023-12-07T10:20:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.970+0000 7fc8d9c18b80 1 -- 172.21.15.119:0/4167160951 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8b409a8e0 con 0x555aa826bad0 2023-12-07T10:20:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.970+0000 7fc89effd700 1 -- 172.21.15.119:0/4167160951 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8b0003d10 con 0x555aa826bad0 2023-12-07T10:20:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.970+0000 7fc8d9c18b80 10 client.15702 fetch_fsmap learned FSMap version 30 2023-12-07T10:20:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.970+0000 7fc8d9c18b80 1 -- 172.21.15.119:0/4167160951 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555aa7fd1c00 con 0x555aa826bad0 2023-12-07T10:20:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.970+0000 7fc89effd700 1 -- 172.21.15.119:0/4167160951 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8b409a8e0 con 0x555aa826bad0 2023-12-07T10:20:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.971+0000 7fc89effd700 1 -- 172.21.15.119:0/4167160951 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc8b00436b0 con 0x555aa826bad0 2023-12-07T10:20:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.971+0000 7fc8d9c18b80 10 client.15702 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:20:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.971+0000 7fc8d9c18b80 10 client.15702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:20:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.971+0000 7fc8d9c18b80 20 client.15702 populate_metadata read hostname 'smithi119' 2023-12-07T10:20:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.971+0000 7fc8d9c18b80 1 --2- 172.21.15.119:0/4167160951 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555aa82e40d0 0x555aa82e64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.971+0000 7fc8d9c18b80 4 client.15702 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:20:40.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.971+0000 7fc8d9c18b80 1 -- 172.21.15.119:0/4167160951 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555aa8185f70 con 0x555aa82e40d0 2023-12-07T10:20:40.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.971+0000 7fc8beffd700 1 --2- 172.21.15.119:0/4167160951 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555aa82e40d0 0x555aa82e64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:40.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.971+0000 7fc89effd700 10 client.15702 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:40.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.971+0000 7fc8beffd700 1 --2- 172.21.15.119:0/4167160951 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555aa82e40d0 0x555aa82e64f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f9adef302db7f5f server_cookie=864f72167a5ad351 in_seq=0 out_seq=0 2023-12-07T10:20:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.981+0000 7fc89effd700 1 -- 172.21.15.119:0/4167160951 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555aa8185f70 con 0x555aa82e40d0 2023-12-07T10:20:40.984 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.981+0000 7fc89effd700 10 client.15702 handle_command_reply: tid=0 2023-12-07T10:20:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.981+0000 7fc8d9c18b80 1 client.15702 shutdown 2023-12-07T10:20:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.982+0000 7fc8d9c18b80 1 -- 172.21.15.119:0/4167160951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555aa826bad0 msgr2=0x555aa827c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.982+0000 7fc8d9c18b80 1 --2- 172.21.15.119:0/4167160951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555aa826bad0 0x555aa827c1d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00098c0 tx=0x7fc8b000f700 comp rx=0 tx=0).stop 2023-12-07T10:20:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.982+0000 7fc8d9c18b80 1 -- 172.21.15.119:0/4167160951 shutdown_connections 2023-12-07T10:20:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.982+0000 7fc8d9c18b80 1 --2- 172.21.15.119:0/4167160951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aa8271380 0x555aa8278430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.982+0000 7fc8d9c18b80 1 --2- 172.21.15.119:0/4167160951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555aa826bad0 0x555aa827c1d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.982+0000 7fc8d9c18b80 1 --2- 172.21.15.119:0/4167160951 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555aa82e40d0 0x555aa82e64f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.982+0000 7fc8d9c18b80 1 --2- 172.21.15.119:0/4167160951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555aa8270a60 0x555aa8277e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.982+0000 7fc89effd700 0 client.15702 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.982+0000 7fc8d9c18b80 1 -- 172.21.15.119:0/4167160951 >> 172.21.15.119:0/4167160951 conn(0x555aa819a300 msgr2=0x555aa82659b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.982+0000 7fc8d9c18b80 1 -- 172.21.15.119:0/4167160951 shutdown_connections 2023-12-07T10:20:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.982+0000 7fc8d9c18b80 1 -- 172.21.15.119:0/4167160951 wait complete. 2023-12-07T10:20:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.982+0000 7fc8d9c18b80 20 client.15702 trim_cache size 0 max 16384 2023-12-07T10:20:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:20:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:20:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:20:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:20:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.983+0000 7fc8c61ff700 1 -- 172.21.15.119:0/605151066 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fc8a80a1d10 msgr2=0x7fc8a80a41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.983+0000 7fc8c61ff700 1 --2- 172.21.15.119:0/605151066 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fc8a80a1d10 0x7fc8a80a41d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0011040 tx=0x7fc8b0019040 comp rx=0 tx=0).stop 2023-12-07T10:20:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.983+0000 7fc8c61ff700 1 -- 172.21.15.119:0/605151066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8c0101ec0 msgr2=0x7fc8c019dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.983+0000 7fc8c61ff700 1 --2- 172.21.15.119:0/605151066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8c0101ec0 0x7fc8c019dba0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400e9f0 tx=0x7fc8b400ed00 comp rx=0 tx=0).stop 2023-12-07T10:20:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.983+0000 7fc8c61ff700 1 -- 172.21.15.119:0/605151066 shutdown_connections 2023-12-07T10:20:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.983+0000 7fc8c61ff700 1 --2- 172.21.15.119:0/605151066 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fc8a80a1d10 0x7fc8a80a41d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.983+0000 7fc8c61ff700 1 --2- 172.21.15.119:0/605151066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8c01075c0 0x7fc8c0197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.983+0000 7fc8c61ff700 1 --2- 172.21.15.119:0/605151066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c0101500 0x7fc8c019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.983+0000 7fc8c61ff700 1 --2- 172.21.15.119:0/605151066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8c0101ec0 0x7fc8c019dba0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.983+0000 7fc8c61ff700 1 -- 172.21.15.119:0/605151066 >> 172.21.15.119:0/605151066 conn(0x7fc8c00fcfb0 msgr2=0x7fc8c00fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.983+0000 7fc8c61ff700 1 -- 172.21.15.119:0/605151066 shutdown_connections 2023-12-07T10:20:40.988 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:40.983+0000 7fc8c61ff700 1 -- 172.21.15.119:0/605151066 wait complete. 2023-12-07T10:20:42.029 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:41 smithi153 ceph-mon[133625]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2023-12-07T10:20:42.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:41 smithi119 ceph-mon[97908]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2023-12-07T10:20:42.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:41 smithi159 ceph-mon[134726]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2023-12-07T10:20:44.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:43 smithi119 ceph-mon[97908]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2023-12-07T10:20:44.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:43 smithi159 ceph-mon[134726]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2023-12-07T10:20:44.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:43 smithi153 ceph-mon[133625]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2023-12-07T10:20:45.410 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-07T10:20:45.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.622+0000 7f010d39d700 1 -- 172.21.15.119:0/1637663086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01081013d0 msgr2=0x7f0108101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:45.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.622+0000 7f010d39d700 1 --2- 172.21.15.119:0/1637663086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01081013d0 0x7f0108101850 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f00fc009990 tx=0x7f00fc009ca0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.622+0000 7f010d39d700 1 -- 172.21.15.119:0/1637663086 shutdown_connections 2023-12-07T10:20:45.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.622+0000 7f010d39d700 1 --2- 172.21.15.119:0/1637663086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0108101d90 0x7f010810e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.622+0000 7f010d39d700 1 --2- 172.21.15.119:0/1637663086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0108107470 0x7f0108107850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.622+0000 7f010d39d700 1 --2- 172.21.15.119:0/1637663086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01081013d0 0x7f0108101850 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.622+0000 7f010d39d700 1 -- 172.21.15.119:0/1637663086 >> 172.21.15.119:0/1637663086 conn(0x7f01080fcee0 msgr2=0x7f01080ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:45.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.622+0000 7f010d39d700 1 -- 172.21.15.119:0/1637663086 shutdown_connections 2023-12-07T10:20:45.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.622+0000 7f010d39d700 1 -- 172.21.15.119:0/1637663086 wait complete. 2023-12-07T10:20:45.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.623+0000 7f010d39d700 1 Processor -- start 2023-12-07T10:20:45.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.623+0000 7f010d39d700 1 -- start start 2023-12-07T10:20:45.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.623+0000 7f010d39d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01081013d0 0x7f010819d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:45.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.623+0000 7f010d39d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0108101d90 0x7f010819da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:45.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.624+0000 7f010d39d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0108107470 0x7f0108197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:45.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.624+0000 7f010d39d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0108197c90 con 0x7f01081013d0 2023-12-07T10:20:45.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.624+0000 7f010d39d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0108197e00 con 0x7f0108107470 2023-12-07T10:20:45.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.624+0000 7f010d39d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0108197fc0 con 0x7f0108101d90 2023-12-07T10:20:45.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.624+0000 7f0107fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01081013d0 0x7f010819d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:45.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.624+0000 7f0107fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01081013d0 0x7f010819d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37450/0 (socket says 172.21.15.119:37450) 2023-12-07T10:20:45.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.624+0000 7f0107fff700 1 -- 172.21.15.119:0/499787671 learned_addr learned my addr 172.21.15.119:0/499787671 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:45.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.624+0000 7f010cb9c700 1 --2- 172.21.15.119:0/499787671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0108107470 0x7f0108197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:45.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.624+0000 7f01077fe700 1 --2- 172.21.15.119:0/499787671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0108101d90 0x7f010819da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:45.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.624+0000 7f0107fff700 1 -- 172.21.15.119:0/499787671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0108101d90 msgr2=0x7f010819da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:45.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.624+0000 7f0107fff700 1 --2- 172.21.15.119:0/499787671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0108101d90 0x7f010819da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.624+0000 7f0107fff700 1 -- 172.21.15.119:0/499787671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0108107470 msgr2=0x7f0108197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:45.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.624+0000 7f0107fff700 1 --2- 172.21.15.119:0/499787671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0108107470 0x7f0108197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.624+0000 7f0107fff700 1 -- 172.21.15.119:0/499787671 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00fc0095f0 con 0x7f01081013d0 2023-12-07T10:20:45.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.624+0000 7f0107fff700 1 --2- 172.21.15.119:0/499787671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01081013d0 0x7f010819d500 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f00f800e9a0 tx=0x7f00f800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:45.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.625+0000 7f01057fa700 1 -- 172.21.15.119:0/499787671 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f8009790 con 0x7f01081013d0 2023-12-07T10:20:45.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.625+0000 7f010d39d700 1 -- 172.21.15.119:0/499787671 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01081982a0 con 0x7f01081013d0 2023-12-07T10:20:45.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.625+0000 7f01057fa700 1 -- 172.21.15.119:0/499787671 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00f8010430 con 0x7f01081013d0 2023-12-07T10:20:45.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.625+0000 7f010d39d700 1 -- 172.21.15.119:0/499787671 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01081a44f0 con 0x7f01081013d0 2023-12-07T10:20:45.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.625+0000 7f01057fa700 1 -- 172.21.15.119:0/499787671 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f8018620 con 0x7f01081013d0 2023-12-07T10:20:45.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.628+0000 7f01057fa700 1 -- 172.21.15.119:0/499787671 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 24) v1 ==== 136660+0+0 (secure 0 0 0) 0x7f00f80105a0 con 0x7f01081013d0 2023-12-07T10:20:45.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.629+0000 7f0121f87b80 1 Processor -- start 2023-12-07T10:20:45.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.629+0000 7f0121f87b80 1 -- start start 2023-12-07T10:20:45.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.629+0000 7f01057fa700 1 --2- 172.21.15.119:0/499787671 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f00f00a1d10 0x7f00f00a41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:45.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.630+0000 7f0121f87b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c06dec4620 0x55c06dec4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:45.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.630+0000 7f0121f87b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c06debec00 0x55c06debf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:45.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.630+0000 7f01057fa700 1 -- 172.21.15.119:0/499787671 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f00f8014070 con 0x7f01081013d0 2023-12-07T10:20:45.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.630+0000 7f0121f87b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06debf5c0 0x55c06decb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:45.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.630+0000 7f0121f87b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06de2e7c0 con 0x55c06debf5c0 2023-12-07T10:20:45.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.630+0000 7f0121f87b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06de157a0 con 0x55c06debec00 2023-12-07T10:20:45.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.630+0000 7f0121f87b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06de130c0 con 0x55c06dec4620 2023-12-07T10:20:45.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.630+0000 7f010cb9c700 1 --2- 172.21.15.119:0/499787671 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f00f00a1d10 0x7f00f00a41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:45.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.630+0000 7f0107fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c06dec4620 0x55c06dec4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:45.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.630+0000 7f0107fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c06dec4620 0x55c06dec4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:43572/0 (socket says 172.21.15.119:43572) 2023-12-07T10:20:45.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.630+0000 7f0107fff700 1 -- 172.21.15.119:0/1042480721 learned_addr learned my addr 172.21.15.119:0/1042480721 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:45.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.630+0000 7f01077fe700 1 --2- 172.21.15.119:0/1042480721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c06debec00 0x55c06debf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:45.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.630+0000 7f010cb9c700 1 --2- 172.21.15.119:0/1042480721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06debf5c0 0x55c06decb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:45.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.630+0000 7f0107fff700 1 -- 172.21.15.119:0/1042480721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c06debec00 msgr2=0x55c06debf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:45.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.630+0000 7f0107fff700 1 --2- 172.21.15.119:0/1042480721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c06debec00 0x55c06debf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.630+0000 7f0107fff700 1 -- 172.21.15.119:0/1042480721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06debf5c0 msgr2=0x55c06decb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:45.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.630+0000 7f0107fff700 1 --2- 172.21.15.119:0/1042480721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06debf5c0 0x55c06decb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.630+0000 7f0107fff700 1 -- 172.21.15.119:0/1042480721 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c06dc8b630 con 0x55c06dec4620 2023-12-07T10:20:45.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.631+0000 7f0107fff700 1 --2- 172.21.15.119:0/1042480721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c06dec4620 0x55c06dec4a00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f00f800e970 tx=0x7f00f809a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fcd3b8856cac34a server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:45.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.631+0000 7f010cb9c700 1 --2- 172.21.15.119:0/499787671 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f00f00a1d10 0x7f00f00a41d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f010000f870 tx=0x7f010000fc30 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:45.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.631+0000 7f00ee7fc700 1 -- 172.21.15.119:0/1042480721 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f80850f0 con 0x55c06dec4620 2023-12-07T10:20:45.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.631+0000 7f00ee7fc700 1 -- 172.21.15.119:0/1042480721 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00f809a880 con 0x55c06dec4620 2023-12-07T10:20:45.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.631+0000 7f00ee7fc700 1 -- 172.21.15.119:0/1042480721 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f809aae0 con 0x55c06dec4620 2023-12-07T10:20:45.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.632+0000 7f0121f87b80 1 -- 172.21.15.119:0/1042480721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c06dec4620 msgr2=0x55c06dec4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:45.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.632+0000 7f0121f87b80 1 --2- 172.21.15.119:0/1042480721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c06dec4620 0x55c06dec4a00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f00f800e970 tx=0x7f00f809a1e0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.632+0000 7f0121f87b80 1 -- 172.21.15.119:0/1042480721 shutdown_connections 2023-12-07T10:20:45.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.632+0000 7f0121f87b80 1 --2- 172.21.15.119:0/1042480721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06debf5c0 0x55c06decb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.632+0000 7f0121f87b80 1 --2- 172.21.15.119:0/1042480721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c06dec4620 0x55c06dec4a00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.632+0000 7f0121f87b80 1 --2- 172.21.15.119:0/1042480721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c06debec00 0x55c06debf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.632+0000 7f0121f87b80 1 -- 172.21.15.119:0/1042480721 >> 172.21.15.119:0/1042480721 conn(0x55c06dde9360 msgr2=0x55c06deb5170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:45.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.632+0000 7f0121f87b80 1 -- 172.21.15.119:0/1042480721 shutdown_connections 2023-12-07T10:20:45.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.632+0000 7f0121f87b80 1 -- 172.21.15.119:0/1042480721 wait complete. 2023-12-07T10:20:45.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.632+0000 7f0121f87b80 1 -- start start 2023-12-07T10:20:45.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.633+0000 7f0121f87b80 1 Processor -- start 2023-12-07T10:20:45.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.633+0000 7f0121f87b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06debec00 0x55c06decb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:45.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.633+0000 7f0121f87b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c06debf5c0 0x55c06dec8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:45.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.633+0000 7f0121f87b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c06dec4620 0x55c06dec8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:45.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.633+0000 7f0121f87b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06de0e9b0 con 0x55c06debec00 2023-12-07T10:20:45.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.633+0000 7f0121f87b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06ddc8560 con 0x55c06dec4620 2023-12-07T10:20:45.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.633+0000 7f0121f87b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06ddc8200 con 0x55c06debf5c0 2023-12-07T10:20:45.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.633+0000 7f0121f87b80 10 client.0 _ll_register_callbacks cb 0x55c06ddaf940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:20:45.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.633+0000 7f01077fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c06debf5c0 0x55c06dec8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:45.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.633+0000 7f010cb9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c06dec4620 0x55c06dec8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:45.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.633+0000 7f01077fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c06debf5c0 0x55c06dec8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:43586/0 (socket says 172.21.15.119:43586) 2023-12-07T10:20:45.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.633+0000 7f01077fe700 1 -- 172.21.15.119:0/1158961725 learned_addr learned my addr 172.21.15.119:0/1158961725 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:45.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.633+0000 7f0107fff700 1 --2- 172.21.15.119:0/1158961725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06debec00 0x55c06decb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:45.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.634+0000 7f010cb9c700 1 -- 172.21.15.119:0/1158961725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c06debf5c0 msgr2=0x55c06dec8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:45.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.634+0000 7f010cb9c700 1 --2- 172.21.15.119:0/1158961725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c06debf5c0 0x55c06dec8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.634+0000 7f010cb9c700 1 -- 172.21.15.119:0/1158961725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06debec00 msgr2=0x55c06decb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:45.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.634+0000 7f010cb9c700 1 --2- 172.21.15.119:0/1158961725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06debec00 0x55c06decb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.634+0000 7f010cb9c700 1 -- 172.21.15.119:0/1158961725 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01081a44f0 con 0x55c06dec4620 2023-12-07T10:20:45.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.634+0000 7f010cb9c700 1 --2- 172.21.15.119:0/1158961725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c06dec4620 0x55c06dec8b10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0100018a80 tx=0x7f0100004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5bfd34c329707d4 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:45.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.634+0000 7f00e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:20:45.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.634+0000 7f00e77fe700 1 -- 172.21.15.119:0/1158961725 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010000e040 con 0x55c06dec4620 2023-12-07T10:20:45.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.634+0000 7f0121f87b80 1 -- 172.21.15.119:0/1158961725 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c06dead9a0 con 0x55c06dec4620 2023-12-07T10:20:45.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.634+0000 7f00e77fe700 1 -- 172.21.15.119:0/1158961725 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f0100004bb0 con 0x55c06dec4620 2023-12-07T10:20:45.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.635+0000 7f00e77fe700 1 -- 172.21.15.119:0/1158961725 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0100004d70 con 0x55c06dec4620 2023-12-07T10:20:45.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.635+0000 7f0121f87b80 10 client.25742 fetch_fsmap learned FSMap version 30 2023-12-07T10:20:45.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.635+0000 7f0121f87b80 1 -- 172.21.15.119:0/1158961725 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c06dc37de0 con 0x55c06dec4620 2023-12-07T10:20:45.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.635+0000 7f00e77fe700 1 -- 172.21.15.119:0/1158961725 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c06dead9a0 con 0x55c06dec4620 2023-12-07T10:20:45.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.635+0000 7f00e77fe700 1 -- 172.21.15.119:0/1158961725 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0100025850 con 0x55c06dec4620 2023-12-07T10:20:45.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.635+0000 7f0121f87b80 10 client.25742 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:20:45.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.635+0000 7f0121f87b80 10 client.25742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:20:45.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.635+0000 7f0121f87b80 20 client.25742 populate_metadata read hostname 'smithi119' 2023-12-07T10:20:45.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.635+0000 7f0121f87b80 1 --2- 172.21.15.119:0/1158961725 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c06ded8af0 0x55c06dedaf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:45.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.635+0000 7f0121f87b80 4 client.25742 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:20:45.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.635+0000 7f0121f87b80 1 -- 172.21.15.119:0/1158961725 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c06de897a0 con 0x55c06ded8af0 2023-12-07T10:20:45.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.635+0000 7f0107fff700 1 --2- 172.21.15.119:0/1158961725 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c06ded8af0 0x55c06dedaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:45.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.636+0000 7f00e77fe700 10 client.25742 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:45.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.636+0000 7f0107fff700 1 --2- 172.21.15.119:0/1158961725 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c06ded8af0 0x55c06dedaf10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16ba0ec447572b44 server_cookie=30138090606dcfe2 in_seq=0 out_seq=0 2023-12-07T10:20:45.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.636+0000 7f00e77fe700 1 -- 172.21.15.119:0/1158961725 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c06de897a0 con 0x55c06ded8af0 2023-12-07T10:20:45.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.637+0000 7f00e77fe700 10 client.25742 handle_command_reply: tid=0 2023-12-07T10:20:45.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.637+0000 7f0121f87b80 1 client.25742 shutdown 2023-12-07T10:20:45.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.637+0000 7f0121f87b80 1 -- 172.21.15.119:0/1158961725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c06dec4620 msgr2=0x55c06dec8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:45.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.637+0000 7f0121f87b80 1 --2- 172.21.15.119:0/1158961725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c06dec4620 0x55c06dec8b10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0100018a80 tx=0x7f0100004b40 comp rx=0 tx=0).stop 2023-12-07T10:20:45.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.637+0000 7f0121f87b80 1 -- 172.21.15.119:0/1158961725 shutdown_connections 2023-12-07T10:20:45.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.637+0000 7f0121f87b80 1 --2- 172.21.15.119:0/1158961725 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c06ded8af0 0x55c06dedaf10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.637+0000 7f0121f87b80 1 --2- 172.21.15.119:0/1158961725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c06dec4620 0x55c06dec8b10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.637+0000 7f00e77fe700 0 client.25742 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:45.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.637+0000 7f0121f87b80 1 --2- 172.21.15.119:0/1158961725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06debec00 0x55c06decb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.637+0000 7f0121f87b80 1 --2- 172.21.15.119:0/1158961725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c06debf5c0 0x55c06dec8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.637+0000 7f0121f87b80 1 -- 172.21.15.119:0/1158961725 >> 172.21.15.119:0/1158961725 conn(0x55c06dde9360 msgr2=0x55c06deb97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:45.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.637+0000 7f0121f87b80 1 -- 172.21.15.119:0/1158961725 shutdown_connections 2023-12-07T10:20:45.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.637+0000 7f0121f87b80 1 -- 172.21.15.119:0/1158961725 wait complete. 2023-12-07T10:20:45.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.637+0000 7f0121f87b80 20 client.25742 trim_cache size 0 max 16384 2023-12-07T10:20:45.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.655+0000 7f0121f87b80 1 Processor -- start 2023-12-07T10:20:45.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.655+0000 7f0121f87b80 1 -- start start 2023-12-07T10:20:45.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.655+0000 7f0121f87b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c06debec00 0x55c06deba170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:45.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.655+0000 7f0121f87b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c06debf5c0 0x55c06deb5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:45.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.655+0000 7f0121f87b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06dec4620 0x55c06dedf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:45.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.655+0000 7f0121f87b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06dc8d920 con 0x55c06dec4620 2023-12-07T10:20:45.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.655+0000 7f0121f87b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06deb5650 con 0x55c06debf5c0 2023-12-07T10:20:45.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.655+0000 7f0121f87b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06deb5d70 con 0x55c06debec00 2023-12-07T10:20:45.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.656+0000 7f010cb9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06dec4620 0x55c06dedf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:45.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.656+0000 7f01077fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c06debf5c0 0x55c06deb5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:45.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.656+0000 7f0107fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c06debec00 0x55c06deba170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:45.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.656+0000 7f01077fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c06debf5c0 0x55c06deb5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:37798/0 (socket says 172.21.15.119:37798) 2023-12-07T10:20:45.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.656+0000 7f0107fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c06debec00 0x55c06deba170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:43600/0 (socket says 172.21.15.119:43600) 2023-12-07T10:20:45.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.656+0000 7f01077fe700 1 -- 172.21.15.119:0/3069092795 learned_addr learned my addr 172.21.15.119:0/3069092795 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:45.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.656+0000 7f01077fe700 1 -- 172.21.15.119:0/3069092795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c06debec00 msgr2=0x55c06deba170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:45.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.656+0000 7f01077fe700 1 --2- 172.21.15.119:0/3069092795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c06debec00 0x55c06deba170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.656+0000 7f01077fe700 1 -- 172.21.15.119:0/3069092795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06dec4620 msgr2=0x55c06dedf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:45.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.656+0000 7f01077fe700 1 --2- 172.21.15.119:0/3069092795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06dec4620 0x55c06dedf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.656+0000 7f01077fe700 1 -- 172.21.15.119:0/3069092795 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c06dc20c60 con 0x55c06debf5c0 2023-12-07T10:20:45.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.656+0000 7f0107fff700 1 --2- 172.21.15.119:0/3069092795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c06debec00 0x55c06deba170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:45.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.656+0000 7f010cb9c700 1 --2- 172.21.15.119:0/3069092795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06dec4620 0x55c06dedf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:45.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.656+0000 7f01077fe700 1 --2- 172.21.15.119:0/3069092795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c06debf5c0 0x55c06deb5110 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f00fc005ca0 tx=0x7f00fc005960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c70d5658dd0d9bbf server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:45.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.657+0000 7f00eeffd700 1 -- 172.21.15.119:0/3069092795 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc01d030 con 0x55c06debf5c0 2023-12-07T10:20:45.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.657+0000 7f00eeffd700 1 -- 172.21.15.119:0/3069092795 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00fc01e040 con 0x55c06debf5c0 2023-12-07T10:20:45.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.657+0000 7f00eeffd700 1 -- 172.21.15.119:0/3069092795 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc004760 con 0x55c06debf5c0 2023-12-07T10:20:45.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.657+0000 7f0121f87b80 1 -- 172.21.15.119:0/3069092795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c06debf5c0 msgr2=0x55c06deb5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:45.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.657+0000 7f0121f87b80 1 --2- 172.21.15.119:0/3069092795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c06debf5c0 0x55c06deb5110 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f00fc005ca0 tx=0x7f00fc005960 comp rx=0 tx=0).stop 2023-12-07T10:20:45.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.657+0000 7f0121f87b80 1 -- 172.21.15.119:0/3069092795 shutdown_connections 2023-12-07T10:20:45.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.657+0000 7f0121f87b80 1 --2- 172.21.15.119:0/3069092795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06dec4620 0x55c06dedf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.657+0000 7f0121f87b80 1 --2- 172.21.15.119:0/3069092795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c06debec00 0x55c06deba170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.657+0000 7f0121f87b80 1 --2- 172.21.15.119:0/3069092795 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c06debf5c0 0x55c06deb5110 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.657+0000 7f0121f87b80 1 -- 172.21.15.119:0/3069092795 >> 172.21.15.119:0/3069092795 conn(0x55c06dde9360 msgr2=0x55c06decaf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:45.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.657+0000 7f0121f87b80 1 -- 172.21.15.119:0/3069092795 shutdown_connections 2023-12-07T10:20:45.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.657+0000 7f0121f87b80 1 -- 172.21.15.119:0/3069092795 wait complete. 2023-12-07T10:20:45.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.658+0000 7f0121f87b80 1 -- start start 2023-12-07T10:20:45.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.658+0000 7f0121f87b80 1 Processor -- start 2023-12-07T10:20:45.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.658+0000 7f0121f87b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06debec00 0x55c06deaf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:45.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.658+0000 7f0121f87b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c06debf5c0 0x55c06deaf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:45.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.658+0000 7f0121f87b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c06dec4620 0x55c06deceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:45.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.658+0000 7f0121f87b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06deb0090 con 0x55c06debec00 2023-12-07T10:20:45.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.658+0000 7f0121f87b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06deb0200 con 0x55c06debf5c0 2023-12-07T10:20:45.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.658+0000 7f0121f87b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c06decf020 con 0x55c06dec4620 2023-12-07T10:20:45.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.658+0000 7f0121f87b80 10 client.0 _ll_register_callbacks cb 0x55c06ddaf940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:20:45.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.658+0000 7f0107fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06debec00 0x55c06deaf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:45.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.658+0000 7f01077fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c06debf5c0 0x55c06deaf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:45.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.658+0000 7f0107fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06debec00 0x55c06deaf4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37490/0 (socket says 172.21.15.119:37490) 2023-12-07T10:20:45.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.658+0000 7f0107fff700 1 -- 172.21.15.119:0/1130189929 learned_addr learned my addr 172.21.15.119:0/1130189929 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:45.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.659+0000 7f010cb9c700 1 --2- 172.21.15.119:0/1130189929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c06dec4620 0x55c06deceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:45.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.659+0000 7f0107fff700 1 -- 172.21.15.119:0/1130189929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c06dec4620 msgr2=0x55c06deceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:45.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.659+0000 7f0107fff700 1 --2- 172.21.15.119:0/1130189929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c06dec4620 0x55c06deceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.659+0000 7f0107fff700 1 -- 172.21.15.119:0/1130189929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c06debf5c0 msgr2=0x55c06deaf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:45.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.659+0000 7f0107fff700 1 --2- 172.21.15.119:0/1130189929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c06debf5c0 0x55c06deaf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.659+0000 7f0107fff700 1 -- 172.21.15.119:0/1130189929 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c06dc8b630 con 0x55c06debec00 2023-12-07T10:20:45.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.659+0000 7f01077fe700 1 --2- 172.21.15.119:0/1130189929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c06debf5c0 0x55c06deaf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:45.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.659+0000 7f0107fff700 1 --2- 172.21.15.119:0/1130189929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06debec00 0x55c06deaf4b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f00f80cb370 tx=0x7f00f809a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e2f9ac61623d3a server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:45.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.659+0000 7f00e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:20:45.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.659+0000 7f00e77fe700 1 -- 172.21.15.119:0/1130189929 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f80a2c10 con 0x55c06debec00 2023-12-07T10:20:45.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.659+0000 7f00e77fe700 1 -- 172.21.15.119:0/1130189929 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f00f809e9d0 con 0x55c06debec00 2023-12-07T10:20:45.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.659+0000 7f0121f87b80 1 -- 172.21.15.119:0/1130189929 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00f809a880 con 0x55c06debec00 2023-12-07T10:20:45.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.659+0000 7f00e77fe700 1 -- 172.21.15.119:0/1130189929 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00f8010880 con 0x55c06debec00 2023-12-07T10:20:45.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.660+0000 7f0121f87b80 10 client.15720 fetch_fsmap learned FSMap version 30 2023-12-07T10:20:45.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.660+0000 7f0121f87b80 1 -- 172.21.15.119:0/1130189929 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c06dc37de0 con 0x55c06debec00 2023-12-07T10:20:45.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.660+0000 7f00e77fe700 1 -- 172.21.15.119:0/1130189929 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00f809a880 con 0x55c06debec00 2023-12-07T10:20:45.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.660+0000 7f00e77fe700 1 -- 172.21.15.119:0/1130189929 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f00f8099de0 con 0x55c06debec00 2023-12-07T10:20:45.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.660+0000 7f0121f87b80 10 client.15720 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:20:45.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.660+0000 7f0121f87b80 10 client.15720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:20:45.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.660+0000 7f0121f87b80 20 client.15720 populate_metadata read hostname 'smithi119' 2023-12-07T10:20:45.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.660+0000 7f0121f87b80 1 --2- 172.21.15.119:0/1130189929 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c06df2faa0 0x55c06df31ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:45.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.660+0000 7f0121f87b80 4 client.15720 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-12-07T10:20:45.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.660+0000 7f0121f87b80 1 -- 172.21.15.119:0/1130189929 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c06ddd4fd0 con 0x55c06df2faa0 2023-12-07T10:20:45.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.660+0000 7f01077fe700 1 --2- 172.21.15.119:0/1130189929 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c06df2faa0 0x55c06df31ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:45.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.661+0000 7f01077fe700 1 --2- 172.21.15.119:0/1130189929 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c06df2faa0 0x55c06df31ec0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89bdabc5dfbc5211 server_cookie=373fda8ea54d164f in_seq=0 out_seq=0 2023-12-07T10:20:45.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.661+0000 7f00e77fe700 10 client.15720 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:45.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.661+0000 7f00e77fe700 1 -- 172.21.15.119:0/1130189929 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c06ddd4fd0 con 0x55c06df2faa0 2023-12-07T10:20:45.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.661+0000 7f00e77fe700 10 client.15720 handle_command_reply: tid=0 2023-12-07T10:20:45.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.661+0000 7f0121f87b80 1 client.15720 shutdown 2023-12-07T10:20:45.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.661+0000 7f0121f87b80 1 -- 172.21.15.119:0/1130189929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06debec00 msgr2=0x55c06deaf4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:45.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.661+0000 7f0121f87b80 1 --2- 172.21.15.119:0/1130189929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06debec00 0x55c06deaf4b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f00f80cb370 tx=0x7f00f809a250 comp rx=0 tx=0).stop 2023-12-07T10:20:45.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.661+0000 7f0121f87b80 1 -- 172.21.15.119:0/1130189929 shutdown_connections 2023-12-07T10:20:45.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.661+0000 7f0121f87b80 1 --2- 172.21.15.119:0/1130189929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c06dec4620 0x55c06deceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.661+0000 7f0121f87b80 1 --2- 172.21.15.119:0/1130189929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06debec00 0x55c06deaf4b0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.661+0000 7f0121f87b80 1 --2- 172.21.15.119:0/1130189929 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c06df2faa0 0x55c06df31ec0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.661+0000 7f0121f87b80 1 --2- 172.21.15.119:0/1130189929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c06debf5c0 0x55c06deaf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.661+0000 7f00e77fe700 0 client.15720 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:45.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.661+0000 7f0121f87b80 1 -- 172.21.15.119:0/1130189929 >> 172.21.15.119:0/1130189929 conn(0x55c06dde9360 msgr2=0x55c06dec2830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:45.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.661+0000 7f0121f87b80 1 -- 172.21.15.119:0/1130189929 shutdown_connections 2023-12-07T10:20:45.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.661+0000 7f0121f87b80 1 -- 172.21.15.119:0/1130189929 wait complete. 2023-12-07T10:20:45.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.662+0000 7f0121f87b80 20 client.15720 trim_cache size 0 max 16384 2023-12-07T10:20:45.671 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:20:45.671 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "no active scrubs running", 2023-12-07T10:20:45.672 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": {} 2023-12-07T10:20:45.672 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:20:45.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.662+0000 7f010d39d700 1 -- 172.21.15.119:0/499787671 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f00f00a1d10 msgr2=0x7f00f00a41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:45.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.662+0000 7f010d39d700 1 --2- 172.21.15.119:0/499787671 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f00f00a1d10 0x7f00f00a41d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f010000f870 tx=0x7f010000fc30 comp rx=0 tx=0).stop 2023-12-07T10:20:45.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.662+0000 7f010d39d700 1 -- 172.21.15.119:0/499787671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01081013d0 msgr2=0x7f010819d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:45.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.662+0000 7f010d39d700 1 --2- 172.21.15.119:0/499787671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01081013d0 0x7f010819d500 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f00f800e9a0 tx=0x7f00f800ecb0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.662+0000 7f010d39d700 1 -- 172.21.15.119:0/499787671 shutdown_connections 2023-12-07T10:20:45.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.662+0000 7f010d39d700 1 --2- 172.21.15.119:0/499787671 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f00f00a1d10 0x7f00f00a41d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.662+0000 7f010d39d700 1 --2- 172.21.15.119:0/499787671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0108107470 0x7f0108197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.662+0000 7f010d39d700 1 --2- 172.21.15.119:0/499787671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01081013d0 0x7f010819d500 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.662+0000 7f010d39d700 1 --2- 172.21.15.119:0/499787671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0108101d90 0x7f010819da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.662+0000 7f010d39d700 1 -- 172.21.15.119:0/499787671 >> 172.21.15.119:0/499787671 conn(0x7f01080fcee0 msgr2=0x7f01080fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:45.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.662+0000 7f010d39d700 1 -- 172.21.15.119:0/499787671 shutdown_connections 2023-12-07T10:20:45.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.662+0000 7f010d39d700 1 -- 172.21.15.119:0/499787671 wait complete. 2023-12-07T10:20:45.683 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-07T10:20:45.683 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:20:45.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.881+0000 7ff1a1bff700 1 -- 172.21.15.119:0/2607258114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff19c1014e0 msgr2=0x7ff19c101960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:45.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.881+0000 7ff1a1bff700 1 --2- 172.21.15.119:0/2607258114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff19c1014e0 0x7ff19c101960 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff1900099c0 tx=0x7ff190009cd0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.881+0000 7ff1a1bff700 1 -- 172.21.15.119:0/2607258114 shutdown_connections 2023-12-07T10:20:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.881+0000 7ff1a1bff700 1 --2- 172.21.15.119:0/2607258114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff19c101ea0 0x7ff19c10e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.881+0000 7ff1a1bff700 1 --2- 172.21.15.119:0/2607258114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff19c1075a0 0x7ff19c107980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.881+0000 7ff1a1bff700 1 --2- 172.21.15.119:0/2607258114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff19c1014e0 0x7ff19c101960 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.881+0000 7ff1a1bff700 1 -- 172.21.15.119:0/2607258114 >> 172.21.15.119:0/2607258114 conn(0x7ff19c0fcf90 msgr2=0x7ff19c0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.881+0000 7ff1a1bff700 1 -- 172.21.15.119:0/2607258114 shutdown_connections 2023-12-07T10:20:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.881+0000 7ff1a1bff700 1 -- 172.21.15.119:0/2607258114 wait complete. 2023-12-07T10:20:45.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.882+0000 7ff1a1bff700 1 Processor -- start 2023-12-07T10:20:45.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.882+0000 7ff1a1bff700 1 -- start start 2023-12-07T10:20:45.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.882+0000 7ff1a1bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff19c1014e0 0x7ff19c19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:45.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.882+0000 7ff1a1bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff19c101ea0 0x7ff19c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:45.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.882+0000 7ff1a1bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff19c1075a0 0x7ff19c199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:45.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.882+0000 7ff1a1bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c199f90 con 0x7ff19c1075a0 2023-12-07T10:20:45.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.882+0000 7ff1a1bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c19a100 con 0x7ff19c1014e0 2023-12-07T10:20:45.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.882+0000 7ff1a1bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c19a2c0 con 0x7ff19c101ea0 2023-12-07T10:20:45.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.883+0000 7ff19affd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff19c101ea0 0x7ff19c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:45.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.883+0000 7ff19b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff19c1014e0 0x7ff19c19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:45.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.883+0000 7ff19affd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff19c101ea0 0x7ff19c19fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:43636/0 (socket says 172.21.15.119:43636) 2023-12-07T10:20:45.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.883+0000 7ff19affd700 1 -- 172.21.15.119:0/2116017470 learned_addr learned my addr 172.21.15.119:0/2116017470 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:45.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.883+0000 7ff19bfff700 1 --2- 172.21.15.119:0/2116017470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff19c1075a0 0x7ff19c199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:45.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.883+0000 7ff19b7fe700 1 -- 172.21.15.119:0/2116017470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff19c101ea0 msgr2=0x7ff19c19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:45.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.883+0000 7ff19b7fe700 1 --2- 172.21.15.119:0/2116017470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff19c101ea0 0x7ff19c19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.883+0000 7ff19b7fe700 1 -- 172.21.15.119:0/2116017470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff19c1075a0 msgr2=0x7ff19c199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:45.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.883+0000 7ff19b7fe700 1 --2- 172.21.15.119:0/2116017470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff19c1075a0 0x7ff19c199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:45.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.883+0000 7ff19b7fe700 1 -- 172.21.15.119:0/2116017470 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1900095f0 con 0x7ff19c1014e0 2023-12-07T10:20:45.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.883+0000 7ff19affd700 1 --2- 172.21.15.119:0/2116017470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff19c101ea0 0x7ff19c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:45.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.883+0000 7ff19bfff700 1 --2- 172.21.15.119:0/2116017470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff19c1075a0 0x7ff19c199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:45.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.883+0000 7ff19b7fe700 1 --2- 172.21.15.119:0/2116017470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff19c1014e0 0x7ff19c19f800 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00d950 tx=0x7ff18c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:45.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.884+0000 7ff198ff9700 1 -- 172.21.15.119:0/2116017470 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c00f890 con 0x7ff19c1014e0 2023-12-07T10:20:45.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.884+0000 7ff198ff9700 1 -- 172.21.15.119:0/2116017470 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff18c010460 con 0x7ff19c1014e0 2023-12-07T10:20:45.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.884+0000 7ff198ff9700 1 -- 172.21.15.119:0/2116017470 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c00e610 con 0x7ff19c1014e0 2023-12-07T10:20:45.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.884+0000 7ff1a1bff700 1 -- 172.21.15.119:0/2116017470 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19c19a5a0 con 0x7ff19c1014e0 2023-12-07T10:20:45.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.884+0000 7ff1a1bff700 1 -- 172.21.15.119:0/2116017470 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19c1a67f0 con 0x7ff19c1014e0 2023-12-07T10:20:45.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.885+0000 7ff1a1bff700 1 -- 172.21.15.119:0/2116017470 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff19c102c00 con 0x7ff19c1014e0 2023-12-07T10:20:45.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.887+0000 7ff198ff9700 1 -- 172.21.15.119:0/2116017470 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 24) v1 ==== 136660+0+0 (secure 0 0 0) 0x7ff18c0105d0 con 0x7ff19c1014e0 2023-12-07T10:20:45.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.888+0000 7ff198ff9700 1 --2- 172.21.15.119:0/2116017470 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff1840a1d90 0x7ff1840a4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:45.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.889+0000 7ff198ff9700 1 -- 172.21.15.119:0/2116017470 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7ff18c0cdb70 con 0x7ff19c1014e0 2023-12-07T10:20:45.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.889+0000 7ff19affd700 1 --2- 172.21.15.119:0/2116017470 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff1840a1d90 0x7ff1840a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:45.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.889+0000 7ff19affd700 1 --2- 172.21.15.119:0/2116017470 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff1840a1d90 0x7ff1840a4250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff19c19aaf0 tx=0x7ff190005ab0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:45.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:45.891+0000 7ff198ff9700 1 -- 172.21.15.119:0/2116017470 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff18c096790 con 0x7ff19c1014e0 2023-12-07T10:20:46.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.140+0000 7ff1a1bff700 1 -- 172.21.15.119:0/2116017470 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff19c068350 con 0x7ff19c1014e0 2023-12-07T10:20:46.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.141+0000 7ff198ff9700 1 -- 172.21.15.119:0/2116017470 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11384 (secure 0 0 0) 0x7ff18c020020 con 0x7ff19c1014e0 2023-12-07T10:20:46.143 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:20:46.144 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-07T10:17:41.278311+0000","modified":"2023-12-07T10:20:33.545611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24517,"mds_3":24505,"mds_4":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14535":{"gid":14535,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6841/1791998028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1791998028},{"type":"v1","addr":"172.21.15.119:6841","nonce":1791998028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24505":{"gid":24505,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/2925503246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2925503246},{"type":"v1","addr":"172.21.15.153:6837","nonce":2925503246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6839/3984962900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3984962900},{"type":"v1","addr":"172.21.15.159:6839","nonce":3984962900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.119:6839/1920762602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1920762602},{"type":"v1","addr":"172.21.15.119:6839","nonce":1920762602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/566825184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":566825184},{"type":"v1","addr":"172.21.15.153:6839","nonce":566825184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-07T10:20:46.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.144+0000 7ff1a1bff700 1 -- 172.21.15.119:0/2116017470 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff1840a1d90 msgr2=0x7ff1840a4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.144+0000 7ff1a1bff700 1 --2- 172.21.15.119:0/2116017470 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff1840a1d90 0x7ff1840a4250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff19c19aaf0 tx=0x7ff190005ab0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.144+0000 7ff1a1bff700 1 -- 172.21.15.119:0/2116017470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff19c1014e0 msgr2=0x7ff19c19f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.144+0000 7ff1a1bff700 1 --2- 172.21.15.119:0/2116017470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff19c1014e0 0x7ff19c19f800 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00d950 tx=0x7ff18c00dc60 comp rx=0 tx=0).stop 2023-12-07T10:20:46.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.144+0000 7ff1a1bff700 1 -- 172.21.15.119:0/2116017470 shutdown_connections 2023-12-07T10:20:46.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.144+0000 7ff1a1bff700 1 --2- 172.21.15.119:0/2116017470 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff1840a1d90 0x7ff1840a4250 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.144+0000 7ff1a1bff700 1 --2- 172.21.15.119:0/2116017470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff19c1075a0 0x7ff19c199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.144+0000 7ff1a1bff700 1 --2- 172.21.15.119:0/2116017470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff19c1014e0 0x7ff19c19f800 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.144+0000 7ff1a1bff700 1 --2- 172.21.15.119:0/2116017470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff19c101ea0 0x7ff19c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.144+0000 7ff1a1bff700 1 -- 172.21.15.119:0/2116017470 >> 172.21.15.119:0/2116017470 conn(0x7ff19c0fcf90 msgr2=0x7ff19c0fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:46.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.144+0000 7ff1a1bff700 1 -- 172.21.15.119:0/2116017470 shutdown_connections 2023-12-07T10:20:46.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.145+0000 7ff1a1bff700 1 -- 172.21.15.119:0/2116017470 wait complete. 2023-12-07T10:20:46.147 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 30 2023-12-07T10:20:46.162 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:45 smithi119 ceph-mon[97908]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 8 op/s 2023-12-07T10:20:46.164 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-12-07T10:20:46.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:45 smithi159 ceph-mon[134726]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 8 op/s 2023-12-07T10:20:46.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:45 smithi153 ceph-mon[133625]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 8 op/s 2023-12-07T10:20:46.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.363+0000 7f0e073ff700 1 -- 172.21.15.119:0/3443228274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e00107610 msgr2=0x7f0e001079f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.363+0000 7f0e073ff700 1 --2- 172.21.15.119:0/3443228274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e00107610 0x7f0e001079f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc0099c0 tx=0x7f0dfc009cd0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.363+0000 7f0e073ff700 1 -- 172.21.15.119:0/3443228274 shutdown_connections 2023-12-07T10:20:46.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.363+0000 7f0e073ff700 1 --2- 172.21.15.119:0/3443228274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e00101f10 0x7f0e0010e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.363+0000 7f0e073ff700 1 --2- 172.21.15.119:0/3443228274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e00107610 0x7f0e001079f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.363+0000 7f0e073ff700 1 --2- 172.21.15.119:0/3443228274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e00101550 0x7f0e001019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.363+0000 7f0e073ff700 1 -- 172.21.15.119:0/3443228274 >> 172.21.15.119:0/3443228274 conn(0x7f0e000fd020 msgr2=0x7f0e000ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:46.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.363+0000 7f0e073ff700 1 -- 172.21.15.119:0/3443228274 shutdown_connections 2023-12-07T10:20:46.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.363+0000 7f0e073ff700 1 -- 172.21.15.119:0/3443228274 wait complete. 2023-12-07T10:20:46.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.364+0000 7f0e073ff700 1 Processor -- start 2023-12-07T10:20:46.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.364+0000 7f0e073ff700 1 -- start start 2023-12-07T10:20:46.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.365+0000 7f0e073ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e00101550 0x7f0e0019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.365+0000 7f0e073ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e00101f10 0x7f0e0019fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.365+0000 7f0e073ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e00107610 0x7f0e00199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.365+0000 7f0e073ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e00199fe0 con 0x7f0e00101550 2023-12-07T10:20:46.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.365+0000 7f0e073ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0019a150 con 0x7f0e00101f10 2023-12-07T10:20:46.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.365+0000 7f0e073ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0019a310 con 0x7f0e00107610 2023-12-07T10:20:46.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.365+0000 7f0e0519b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e00101550 0x7f0e0019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.365+0000 7f0e0519b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e00101550 0x7f0e0019f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37522/0 (socket says 172.21.15.119:37522) 2023-12-07T10:20:46.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.365+0000 7f0e0519b700 1 -- 172.21.15.119:0/3369037685 learned_addr learned my addr 172.21.15.119:0/3369037685 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:46.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.365+0000 7f0e0599c700 1 --2- 172.21.15.119:0/3369037685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e00107610 0x7f0e00199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.365+0000 7f0e0499a700 1 --2- 172.21.15.119:0/3369037685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e00101f10 0x7f0e0019fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.365+0000 7f0e0519b700 1 -- 172.21.15.119:0/3369037685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e00107610 msgr2=0x7f0e00199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.365+0000 7f0e0519b700 1 --2- 172.21.15.119:0/3369037685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e00107610 0x7f0e00199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.365+0000 7f0e0519b700 1 -- 172.21.15.119:0/3369037685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e00101f10 msgr2=0x7f0e0019fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.365+0000 7f0e0519b700 1 --2- 172.21.15.119:0/3369037685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e00101f10 0x7f0e0019fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.365+0000 7f0e0519b700 1 -- 172.21.15.119:0/3369037685 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dfc0095f0 con 0x7f0e00101550 2023-12-07T10:20:46.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.365+0000 7f0e0499a700 1 --2- 172.21.15.119:0/3369037685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e00101f10 0x7f0e0019fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:20:46.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.366+0000 7f0e0519b700 1 --2- 172.21.15.119:0/3369037685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e00101550 0x7f0e0019f850 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc00b580 tx=0x7f0dfc00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:46.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.366+0000 7f0df67fc700 1 -- 172.21.15.119:0/3369037685 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc01c070 con 0x7f0e00101550 2023-12-07T10:20:46.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.366+0000 7f0df67fc700 1 -- 172.21.15.119:0/3369037685 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0dfc00fdf0 con 0x7f0e00101550 2023-12-07T10:20:46.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.366+0000 7f0e073ff700 1 -- 172.21.15.119:0/3369037685 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e0019a590 con 0x7f0e00101550 2023-12-07T10:20:46.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.366+0000 7f0df67fc700 1 -- 172.21.15.119:0/3369037685 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc022c40 con 0x7f0e00101550 2023-12-07T10:20:46.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.366+0000 7f0e073ff700 1 -- 172.21.15.119:0/3369037685 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e001a6840 con 0x7f0e00101550 2023-12-07T10:20:46.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.369+0000 7f0df67fc700 1 -- 172.21.15.119:0/3369037685 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 24) v1 ==== 136660+0+0 (secure 0 0 0) 0x7f0dfc020070 con 0x7f0e00101550 2023-12-07T10:20:46.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.370+0000 7f0e1adb5b80 1 Processor -- start 2023-12-07T10:20:46.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.371+0000 7f0e1adb5b80 1 -- start start 2023-12-07T10:20:46.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.371+0000 7f0df67fc700 1 --2- 172.21.15.119:0/3369037685 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0dec0a1d10 0x7f0dec0a41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.371+0000 7f0e1adb5b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b79721fb40 0x55b79721ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.371+0000 7f0e1adb5b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b797220460 0x55b797223c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.371+0000 7f0e1adb5b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b797224150 0x55b797224570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.371+0000 7f0e1adb5b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7971937c0 con 0x55b79721fb40 2023-12-07T10:20:46.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.371+0000 7f0e1adb5b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b79717a7a0 con 0x55b797220460 2023-12-07T10:20:46.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.371+0000 7f0e1adb5b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7971780c0 con 0x55b797224150 2023-12-07T10:20:46.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.371+0000 7f0df67fc700 1 -- 172.21.15.119:0/3369037685 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f0dfc0cecd0 con 0x7f0e00101550 2023-12-07T10:20:46.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.371+0000 7f0e0599c700 1 --2- 172.21.15.119:0/3369037685 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0dec0a1d10 0x7f0dec0a41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.371+0000 7f0e0519b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b79721fb40 0x55b79721ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.371+0000 7f0e0499a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b797220460 0x55b797223c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.371+0000 7f0e0519b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b79721fb40 0x55b79721ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37538/0 (socket says 172.21.15.119:37538) 2023-12-07T10:20:46.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.371+0000 7f0e0519b700 1 -- 172.21.15.119:0/1648293141 learned_addr learned my addr 172.21.15.119:0/1648293141 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:46.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.371+0000 7f0e0599c700 1 --2- 172.21.15.119:0/1648293141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b797224150 0x55b797224570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.372+0000 7f0e0519b700 1 -- 172.21.15.119:0/1648293141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b797224150 msgr2=0x55b797224570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:46.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.372+0000 7f0e0519b700 1 --2- 172.21.15.119:0/1648293141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b797224150 0x55b797224570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.372+0000 7f0e0519b700 1 -- 172.21.15.119:0/1648293141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b797220460 msgr2=0x55b797223c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:46.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.372+0000 7f0e0519b700 1 --2- 172.21.15.119:0/1648293141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b797220460 0x55b797223c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.372+0000 7f0e0519b700 1 -- 172.21.15.119:0/1648293141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b796ff0630 con 0x55b79721fb40 2023-12-07T10:20:46.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.372+0000 7f0e0499a700 1 --2- 172.21.15.119:0/1648293141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b797220460 0x55b797223c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:46.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.372+0000 7f0e0519b700 1 --2- 172.21.15.119:0/1648293141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b79721fb40 0x55b79721ff20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc097ae0 tx=0x7f0dfc09b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579ccd0e90431228 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:46.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.372+0000 7f0e0599c700 1 --2- 172.21.15.119:0/3369037685 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0dec0a1d10 0x7f0dec0a41d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009ec0 tx=0x7f0df80092f0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:46.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.372+0000 7f0deb7fe700 1 -- 172.21.15.119:0/1648293141 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc09af40 con 0x55b79721fb40 2023-12-07T10:20:46.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.372+0000 7f0deb7fe700 1 -- 172.21.15.119:0/1648293141 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0dfc01e2d0 con 0x55b79721fb40 2023-12-07T10:20:46.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.372+0000 7f0deb7fe700 1 -- 172.21.15.119:0/1648293141 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc09f730 con 0x55b79721fb40 2023-12-07T10:20:46.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.373+0000 7f0e1adb5b80 1 -- 172.21.15.119:0/1648293141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b79721fb40 msgr2=0x55b79721ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.373+0000 7f0e1adb5b80 1 --2- 172.21.15.119:0/1648293141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b79721fb40 0x55b79721ff20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc097ae0 tx=0x7f0dfc09b3b0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.373+0000 7f0e1adb5b80 1 -- 172.21.15.119:0/1648293141 shutdown_connections 2023-12-07T10:20:46.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.373+0000 7f0e1adb5b80 1 --2- 172.21.15.119:0/1648293141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b797224150 0x55b797224570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.373+0000 7f0e1adb5b80 1 --2- 172.21.15.119:0/1648293141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b79721fb40 0x55b79721ff20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.373+0000 7f0e1adb5b80 1 --2- 172.21.15.119:0/1648293141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b797220460 0x55b797223c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.373+0000 7f0e1adb5b80 1 -- 172.21.15.119:0/1648293141 >> 172.21.15.119:0/1648293141 conn(0x55b79714e360 msgr2=0x55b79721a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:46.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.373+0000 7f0e1adb5b80 1 -- 172.21.15.119:0/1648293141 shutdown_connections 2023-12-07T10:20:46.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.373+0000 7f0e1adb5b80 1 -- 172.21.15.119:0/1648293141 wait complete. 2023-12-07T10:20:46.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.374+0000 7f0e1adb5b80 1 -- start start 2023-12-07T10:20:46.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.374+0000 7f0e1adb5b80 1 Processor -- start 2023-12-07T10:20:46.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.374+0000 7f0e1adb5b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b79721fb40 0x55b79722b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.374+0000 7f0e1adb5b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b797220460 0x55b797228c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.374+0000 7f0e1adb5b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b797224150 0x55b797229280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.374+0000 7f0e1adb5b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7971739b0 con 0x55b797224150 2023-12-07T10:20:46.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.374+0000 7f0e1adb5b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b79712d560 con 0x55b79721fb40 2023-12-07T10:20:46.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.374+0000 7f0e1adb5b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b79712d200 con 0x55b797220460 2023-12-07T10:20:46.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.374+0000 7f0e1adb5b80 10 client.0 _ll_register_callbacks cb 0x55b797114940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:20:46.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.374+0000 7f0e0599c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b797224150 0x55b797229280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.374+0000 7f0e0519b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b79721fb40 0x55b79722b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.375+0000 7f0e0499a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b797220460 0x55b797228c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.375+0000 7f0e0599c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b797224150 0x55b797229280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37552/0 (socket says 172.21.15.119:37552) 2023-12-07T10:20:46.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.375+0000 7f0e0599c700 1 -- 172.21.15.119:0/737738809 learned_addr learned my addr 172.21.15.119:0/737738809 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:46.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.375+0000 7f0e0519b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b79721fb40 0x55b79722b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:37852/0 (socket says 172.21.15.119:37852) 2023-12-07T10:20:46.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.375+0000 7f0e0599c700 1 -- 172.21.15.119:0/737738809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b797220460 msgr2=0x55b797228c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:46.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.375+0000 7f0e0599c700 1 --2- 172.21.15.119:0/737738809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b797220460 0x55b797228c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.375+0000 7f0e0599c700 1 -- 172.21.15.119:0/737738809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b79721fb40 msgr2=0x55b79722b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:46.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.375+0000 7f0e0599c700 1 --2- 172.21.15.119:0/737738809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b79721fb40 0x55b79722b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.375+0000 7f0e0599c700 1 -- 172.21.15.119:0/737738809 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df00098f0 con 0x55b797224150 2023-12-07T10:20:46.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.375+0000 7f0e0499a700 1 --2- 172.21.15.119:0/737738809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b797220460 0x55b797228c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:46.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.375+0000 7f0e0519b700 1 --2- 172.21.15.119:0/737738809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b79721fb40 0x55b79722b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:46.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.375+0000 7f0e0599c700 1 --2- 172.21.15.119:0/737738809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b797224150 0x55b797229280 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0df801aeb0 tx=0x7f0df800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a283cf006d0b972 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:46.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.375+0000 7f0de8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:20:46.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.375+0000 7f0de8ff9700 1 -- 172.21.15.119:0/737738809 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df800e070 con 0x55b797224150 2023-12-07T10:20:46.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.375+0000 7f0e1adb5b80 1 -- 172.21.15.119:0/737738809 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b797212b00 con 0x55b797224150 2023-12-07T10:20:46.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.375+0000 7f0de8ff9700 1 -- 172.21.15.119:0/737738809 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f0df8021040 con 0x55b797224150 2023-12-07T10:20:46.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.376+0000 7f0de8ff9700 1 -- 172.21.15.119:0/737738809 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0df801aee0 con 0x55b797224150 2023-12-07T10:20:46.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.376+0000 7f0e1adb5b80 10 client.15738 fetch_fsmap learned FSMap version 30 2023-12-07T10:20:46.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.376+0000 7f0e1adb5b80 1 -- 172.21.15.119:0/737738809 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f0e001a6840 con 0x55b797224150 2023-12-07T10:20:46.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.376+0000 7f0de8ff9700 1 -- 172.21.15.119:0/737738809 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b797212b00 con 0x55b797224150 2023-12-07T10:20:46.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.376+0000 7f0de8ff9700 1 -- 172.21.15.119:0/737738809 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0df80254a0 con 0x55b797224150 2023-12-07T10:20:46.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.376+0000 7f0e1adb5b80 10 client.15738 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:20:46.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.376+0000 7f0e1adb5b80 10 client.15738 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2023-12-07T10:20:46.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.376+0000 7f0e1adb5b80 20 client.15738 populate_metadata read hostname 'smithi119' 2023-12-07T10:20:46.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.376+0000 7f0e1adb5b80 1 --2- 172.21.15.119:0/737738809 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x55b7972390e0 0x55b79723b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.376+0000 7f0e1adb5b80 4 client.15738 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:20:46.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.376+0000 7f0e1adb5b80 1 -- 172.21.15.119:0/737738809 --> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b7971ee7a0 con 0x55b7972390e0 2023-12-07T10:20:46.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.377+0000 7f0e0519b700 1 --2- 172.21.15.119:0/737738809 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x55b7972390e0 0x55b79723b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.377+0000 7f0de8ff9700 10 client.15738 ms_handle_connect on v2:172.21.15.153:6834/1295649464 2023-12-07T10:20:46.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.377+0000 7f0e0519b700 1 --2- 172.21.15.119:0/737738809 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x55b7972390e0 0x55b79723b500 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a8b23fd3fa32a983 server_cookie=ce16ef3d4f11042a in_seq=0 out_seq=0 2023-12-07T10:20:46.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.378+0000 7f0de8ff9700 1 -- 172.21.15.119:0/737738809 <== mds.4 v2:172.21.15.153:6834/1295649464 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b7971ee7a0 con 0x55b7972390e0 2023-12-07T10:20:46.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.378+0000 7f0de8ff9700 10 client.15738 handle_command_reply: tid=0 2023-12-07T10:20:46.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.378+0000 7f0e1adb5b80 1 client.15738 shutdown 2023-12-07T10:20:46.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.378+0000 7f0e1adb5b80 1 -- 172.21.15.119:0/737738809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b797224150 msgr2=0x55b797229280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.378+0000 7f0e1adb5b80 1 --2- 172.21.15.119:0/737738809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b797224150 0x55b797229280 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0df801aeb0 tx=0x7f0df800b040 comp rx=0 tx=0).stop 2023-12-07T10:20:46.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.378+0000 7f0e1adb5b80 1 -- 172.21.15.119:0/737738809 shutdown_connections 2023-12-07T10:20:46.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.378+0000 7f0e1adb5b80 1 --2- 172.21.15.119:0/737738809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b797224150 0x55b797229280 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.378+0000 7f0e1adb5b80 1 --2- 172.21.15.119:0/737738809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b79721fb40 0x55b79722b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.378+0000 7f0e1adb5b80 1 --2- 172.21.15.119:0/737738809 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x55b7972390e0 0x55b79723b500 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.378+0000 7f0e1adb5b80 1 --2- 172.21.15.119:0/737738809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b797220460 0x55b797228c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.378+0000 7f0e1adb5b80 1 -- 172.21.15.119:0/737738809 >> 172.21.15.119:0/737738809 conn(0x55b79714e360 msgr2=0x55b797214d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:46.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.378+0000 7f0de8ff9700 0 client.15738 ms_handle_reset on v2:172.21.15.153:6834/1295649464 2023-12-07T10:20:46.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.378+0000 7f0e1adb5b80 1 -- 172.21.15.119:0/737738809 shutdown_connections 2023-12-07T10:20:46.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.378+0000 7f0e1adb5b80 1 -- 172.21.15.119:0/737738809 wait complete. 2023-12-07T10:20:46.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.378+0000 7f0e1adb5b80 20 client.15738 trim_cache size 0 max 16384 2023-12-07T10:20:46.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.392+0000 7f0e1adb5b80 1 Processor -- start 2023-12-07T10:20:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.392+0000 7f0e1adb5b80 1 -- start start 2023-12-07T10:20:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.392+0000 7f0e1adb5b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b79721fb40 0x55b797229170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.392+0000 7f0e1adb5b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b797220460 0x55b797230000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.392+0000 7f0e1adb5b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b797224150 0x55b797230540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.392+0000 7f0e1adb5b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b796ff2920 con 0x55b797224150 2023-12-07T10:20:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.392+0000 7f0e1adb5b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b797212ae0 con 0x55b79721fb40 2023-12-07T10:20:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.392+0000 7f0e1adb5b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b797222e40 con 0x55b797220460 2023-12-07T10:20:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.392+0000 7f0e0599c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b797224150 0x55b797230540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.392+0000 7f0e0519b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b79721fb40 0x55b797229170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.392+0000 7f0e0599c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b797224150 0x55b797230540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37558/0 (socket says 172.21.15.119:37558) 2023-12-07T10:20:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.393+0000 7f0e0599c700 1 -- 172.21.15.119:0/3677642077 learned_addr learned my addr 172.21.15.119:0/3677642077 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:46.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.393+0000 7f0e0499a700 1 --2- 172.21.15.119:0/3677642077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b797220460 0x55b797230000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.393+0000 7f0e0599c700 1 -- 172.21.15.119:0/3677642077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b797220460 msgr2=0x55b797230000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:46.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.393+0000 7f0e0599c700 1 --2- 172.21.15.119:0/3677642077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b797220460 0x55b797230000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.393+0000 7f0e0599c700 1 -- 172.21.15.119:0/3677642077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b79721fb40 msgr2=0x55b797229170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:46.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.393+0000 7f0e0599c700 1 --2- 172.21.15.119:0/3677642077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b79721fb40 0x55b797229170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.393+0000 7f0e0599c700 1 -- 172.21.15.119:0/3677642077 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b796f9cde0 con 0x55b797224150 2023-12-07T10:20:46.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.393+0000 7f0e0519b700 1 --2- 172.21.15.119:0/3677642077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b79721fb40 0x55b797229170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:46.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.393+0000 7f0e0599c700 1 --2- 172.21.15.119:0/3677642077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b797224150 0x55b797230540 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0df801a770 tx=0x7f0df801aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b776f1a0e62d687 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:46.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.393+0000 7f0debfff700 1 -- 172.21.15.119:0/3677642077 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df800e040 con 0x55b797224150 2023-12-07T10:20:46.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.393+0000 7f0debfff700 1 -- 172.21.15.119:0/3677642077 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0df800f040 con 0x55b797224150 2023-12-07T10:20:46.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.393+0000 7f0debfff700 1 -- 172.21.15.119:0/3677642077 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df8006000 con 0x55b797224150 2023-12-07T10:20:46.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.394+0000 7f0e1adb5b80 1 -- 172.21.15.119:0/3677642077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b797224150 msgr2=0x55b797230540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.394+0000 7f0e1adb5b80 1 --2- 172.21.15.119:0/3677642077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b797224150 0x55b797230540 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0df801a770 tx=0x7f0df801aee0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.394+0000 7f0e1adb5b80 1 -- 172.21.15.119:0/3677642077 shutdown_connections 2023-12-07T10:20:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.394+0000 7f0e1adb5b80 1 --2- 172.21.15.119:0/3677642077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b797224150 0x55b797230540 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.394+0000 7f0e1adb5b80 1 --2- 172.21.15.119:0/3677642077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b79721fb40 0x55b797229170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.394+0000 7f0e1adb5b80 1 --2- 172.21.15.119:0/3677642077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b797220460 0x55b797230000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.394+0000 7f0e1adb5b80 1 -- 172.21.15.119:0/3677642077 >> 172.21.15.119:0/3677642077 conn(0x55b79714e360 msgr2=0x55b797224530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.394+0000 7f0e1adb5b80 1 -- 172.21.15.119:0/3677642077 shutdown_connections 2023-12-07T10:20:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.394+0000 7f0e1adb5b80 1 -- 172.21.15.119:0/3677642077 wait complete. 2023-12-07T10:20:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.394+0000 7f0e1adb5b80 1 -- start start 2023-12-07T10:20:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.395+0000 7f0e1adb5b80 1 Processor -- start 2023-12-07T10:20:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.395+0000 7f0e1adb5b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b79721fb40 0x55b797230200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.395+0000 7f0e1adb5b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b797220460 0x55b797224f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.395+0000 7f0e1adb5b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b797224150 0x55b7972254c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.395+0000 7f0e1adb5b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b797225a90 con 0x55b79721fb40 2023-12-07T10:20:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.395+0000 7f0e1adb5b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b797225c00 con 0x55b797224150 2023-12-07T10:20:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.395+0000 7f0e1adb5b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b797225dc0 con 0x55b797220460 2023-12-07T10:20:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.395+0000 7f0e1adb5b80 10 client.0 _ll_register_callbacks cb 0x55b797114940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:20:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.395+0000 7f0e0519b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b79721fb40 0x55b797230200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.395+0000 7f0e0519b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b79721fb40 0x55b797230200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37564/0 (socket says 172.21.15.119:37564) 2023-12-07T10:20:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.395+0000 7f0e0519b700 1 -- 172.21.15.119:0/2211406345 learned_addr learned my addr 172.21.15.119:0/2211406345 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.395+0000 7f0e0499a700 1 --2- 172.21.15.119:0/2211406345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b797220460 0x55b797224f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.395+0000 7f0e0599c700 1 --2- 172.21.15.119:0/2211406345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b797224150 0x55b7972254c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.396+0000 7f0e0519b700 1 -- 172.21.15.119:0/2211406345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b797220460 msgr2=0x55b797224f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.396+0000 7f0e0519b700 1 --2- 172.21.15.119:0/2211406345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b797220460 0x55b797224f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.396+0000 7f0e0519b700 1 -- 172.21.15.119:0/2211406345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b797224150 msgr2=0x55b7972254c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.396+0000 7f0e0519b700 1 --2- 172.21.15.119:0/2211406345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b797224150 0x55b7972254c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.396+0000 7f0e0519b700 1 -- 172.21.15.119:0/2211406345 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b796ff0630 con 0x55b79721fb40 2023-12-07T10:20:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.396+0000 7f0e0499a700 1 --2- 172.21.15.119:0/2211406345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b797220460 0x55b797224f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:20:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.396+0000 7f0e0519b700 1 --2- 172.21.15.119:0/2211406345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b79721fb40 0x55b797230200 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc09b2e0 tx=0x7f0dfc005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cffa11b71566651 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.396+0000 7f0de8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:20:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.396+0000 7f0de8ff9700 1 -- 172.21.15.119:0/2211406345 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc066c40 con 0x55b79721fb40 2023-12-07T10:20:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.396+0000 7f0e1adb5b80 1 -- 172.21.15.119:0/2211406345 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0dfc01e2d0 con 0x55b79721fb40 2023-12-07T10:20:46.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.396+0000 7f0de8ff9700 1 -- 172.21.15.119:0/2211406345 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f0dfc09bce0 con 0x55b79721fb40 2023-12-07T10:20:46.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.396+0000 7f0de8ff9700 1 -- 172.21.15.119:0/2211406345 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0dfc0a38e0 con 0x55b79721fb40 2023-12-07T10:20:46.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.396+0000 7f0e1adb5b80 10 client.15744 fetch_fsmap learned FSMap version 30 2023-12-07T10:20:46.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.396+0000 7f0e1adb5b80 1 -- 172.21.15.119:0/2211406345 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f0e001a6840 con 0x55b79721fb40 2023-12-07T10:20:46.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.396+0000 7f0de8ff9700 1 -- 172.21.15.119:0/2211406345 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0dfc01e2d0 con 0x55b79721fb40 2023-12-07T10:20:46.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.397+0000 7f0de8ff9700 1 -- 172.21.15.119:0/2211406345 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0dfc0a3450 con 0x55b79721fb40 2023-12-07T10:20:46.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.397+0000 7f0e1adb5b80 10 client.15744 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:20:46.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.397+0000 7f0e1adb5b80 10 client.15744 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2023-12-07T10:20:46.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.397+0000 7f0e1adb5b80 20 client.15744 populate_metadata read hostname 'smithi119' 2023-12-07T10:20:46.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.397+0000 7f0e1adb5b80 1 --2- 172.21.15.119:0/2211406345 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x55b79728f860 0x55b797291c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.397+0000 7f0e1adb5b80 4 client.15744 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:20:46.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.397+0000 7f0e1adb5b80 1 -- 172.21.15.119:0/2211406345 --> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b797139fd0 con 0x55b79728f860 2023-12-07T10:20:46.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.397+0000 7f0e0499a700 1 --2- 172.21.15.119:0/2211406345 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x55b79728f860 0x55b797291c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.397+0000 7f0e0499a700 1 --2- 172.21.15.119:0/2211406345 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x55b79728f860 0x55b797291c50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=596791f0579fa353 server_cookie=610f4932e0e1abe6 in_seq=0 out_seq=0 2023-12-07T10:20:46.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.397+0000 7f0de8ff9700 10 client.15744 ms_handle_connect on v2:172.21.15.153:6834/1295649464 2023-12-07T10:20:46.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.398+0000 7f0de8ff9700 1 -- 172.21.15.119:0/2211406345 <== mds.4 v2:172.21.15.153:6834/1295649464 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b797139fd0 con 0x55b79728f860 2023-12-07T10:20:46.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.398+0000 7f0de8ff9700 10 client.15744 handle_command_reply: tid=0 2023-12-07T10:20:46.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.398+0000 7f0e1adb5b80 1 client.15744 shutdown 2023-12-07T10:20:46.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.398+0000 7f0e1adb5b80 1 -- 172.21.15.119:0/2211406345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b79721fb40 msgr2=0x55b797230200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.398+0000 7f0e1adb5b80 1 --2- 172.21.15.119:0/2211406345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b79721fb40 0x55b797230200 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc09b2e0 tx=0x7f0dfc005ef0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.398+0000 7f0e1adb5b80 1 -- 172.21.15.119:0/2211406345 shutdown_connections 2023-12-07T10:20:46.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.398+0000 7f0e1adb5b80 1 --2- 172.21.15.119:0/2211406345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b797224150 0x55b7972254c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.398+0000 7f0e1adb5b80 1 --2- 172.21.15.119:0/2211406345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b79721fb40 0x55b797230200 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.398+0000 7f0e1adb5b80 1 --2- 172.21.15.119:0/2211406345 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x55b79728f860 0x55b797291c50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.398+0000 7f0e1adb5b80 1 --2- 172.21.15.119:0/2211406345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b797220460 0x55b797224f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.398+0000 7f0de8ff9700 0 client.15744 ms_handle_reset on v2:172.21.15.153:6834/1295649464 2023-12-07T10:20:46.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.398+0000 7f0e1adb5b80 1 -- 172.21.15.119:0/2211406345 >> 172.21.15.119:0/2211406345 conn(0x55b79714e360 msgr2=0x55b7972232d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:46.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.398+0000 7f0e1adb5b80 1 -- 172.21.15.119:0/2211406345 shutdown_connections 2023-12-07T10:20:46.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.398+0000 7f0e1adb5b80 1 -- 172.21.15.119:0/2211406345 wait complete. 2023-12-07T10:20:46.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.398+0000 7f0e1adb5b80 20 client.15744 trim_cache size 0 max 16384 2023-12-07T10:20:46.405 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:20:46.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.399+0000 7f0e073ff700 1 -- 172.21.15.119:0/3369037685 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0dec0a1d10 msgr2=0x7f0dec0a41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.399+0000 7f0e073ff700 1 --2- 172.21.15.119:0/3369037685 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0dec0a1d10 0x7f0dec0a41d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009ec0 tx=0x7f0df80092f0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.399+0000 7f0e073ff700 1 -- 172.21.15.119:0/3369037685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e00101550 msgr2=0x7f0e0019f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.399+0000 7f0e073ff700 1 --2- 172.21.15.119:0/3369037685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e00101550 0x7f0e0019f850 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc00b580 tx=0x7f0dfc00bfd0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.399+0000 7f0e073ff700 1 -- 172.21.15.119:0/3369037685 shutdown_connections 2023-12-07T10:20:46.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.399+0000 7f0e073ff700 1 --2- 172.21.15.119:0/3369037685 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0dec0a1d10 0x7f0dec0a41d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.399+0000 7f0e073ff700 1 --2- 172.21.15.119:0/3369037685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e00107610 0x7f0e00199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.399+0000 7f0e073ff700 1 --2- 172.21.15.119:0/3369037685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e00101550 0x7f0e0019f850 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.399+0000 7f0e073ff700 1 --2- 172.21.15.119:0/3369037685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e00101f10 0x7f0e0019fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.399+0000 7f0e073ff700 1 -- 172.21.15.119:0/3369037685 >> 172.21.15.119:0/3369037685 conn(0x7f0e000fd020 msgr2=0x7f0e000fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:46.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.399+0000 7f0e073ff700 1 -- 172.21.15.119:0/3369037685 shutdown_connections 2023-12-07T10:20:46.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.399+0000 7f0e073ff700 1 -- 172.21.15.119:0/3369037685 wait complete. 2023-12-07T10:20:46.418 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-12-07T10:20:46.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.617+0000 7fe4e15ff700 1 -- 172.21.15.119:0/2539734801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4dc101560 msgr2=0x7fe4dc1019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.617+0000 7fe4e15ff700 1 --2- 172.21.15.119:0/2539734801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4dc101560 0x7fe4dc1019e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00098f0 tx=0x7fe4d0009c00 comp rx=0 tx=0).stop 2023-12-07T10:20:46.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.617+0000 7fe4e15ff700 1 -- 172.21.15.119:0/2539734801 shutdown_connections 2023-12-07T10:20:46.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.617+0000 7fe4e15ff700 1 --2- 172.21.15.119:0/2539734801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4dc101f20 0x7fe4dc10e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.617+0000 7fe4e15ff700 1 --2- 172.21.15.119:0/2539734801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4dc107620 0x7fe4dc107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.617+0000 7fe4e15ff700 1 --2- 172.21.15.119:0/2539734801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4dc101560 0x7fe4dc1019e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.617+0000 7fe4e15ff700 1 -- 172.21.15.119:0/2539734801 >> 172.21.15.119:0/2539734801 conn(0x7fe4dc0fcfd0 msgr2=0x7fe4dc0ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:46.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.617+0000 7fe4e15ff700 1 -- 172.21.15.119:0/2539734801 shutdown_connections 2023-12-07T10:20:46.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.617+0000 7fe4e15ff700 1 -- 172.21.15.119:0/2539734801 wait complete. 2023-12-07T10:20:46.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.619+0000 7fe4e15ff700 1 Processor -- start 2023-12-07T10:20:46.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.619+0000 7fe4e15ff700 1 -- start start 2023-12-07T10:20:46.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.619+0000 7fe4e15ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4dc101560 0x7fe4dc073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.619+0000 7fe4e15ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4dc101f20 0x7fe4dc073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.619+0000 7fe4e15ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4dc107620 0x7fe4dc074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.619+0000 7fe4e15ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc1a5d40 con 0x7fe4dc101f20 2023-12-07T10:20:46.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.619+0000 7fe4e15ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc1a5eb0 con 0x7fe4dc101560 2023-12-07T10:20:46.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.619+0000 7fe4e15ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc1a6070 con 0x7fe4dc107620 2023-12-07T10:20:46.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.620+0000 7fe4da7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4dc101f20 0x7fe4dc073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.620+0000 7fe4da7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4dc101f20 0x7fe4dc073e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37580/0 (socket says 172.21.15.119:37580) 2023-12-07T10:20:46.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.620+0000 7fe4daffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4dc101560 0x7fe4dc073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.620+0000 7fe4da7fc700 1 -- 172.21.15.119:0/93394874 learned_addr learned my addr 172.21.15.119:0/93394874 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:46.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.620+0000 7fe4db7fe700 1 --2- 172.21.15.119:0/93394874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4dc107620 0x7fe4dc074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.620+0000 7fe4da7fc700 1 -- 172.21.15.119:0/93394874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4dc107620 msgr2=0x7fe4dc074390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.620+0000 7fe4da7fc700 1 --2- 172.21.15.119:0/93394874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4dc107620 0x7fe4dc074390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.620+0000 7fe4da7fc700 1 -- 172.21.15.119:0/93394874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4dc101560 msgr2=0x7fe4dc073910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.620+0000 7fe4da7fc700 1 --2- 172.21.15.119:0/93394874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4dc101560 0x7fe4dc073910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.620+0000 7fe4da7fc700 1 -- 172.21.15.119:0/93394874 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4cc0095f0 con 0x7fe4dc101f20 2023-12-07T10:20:46.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.620+0000 7fe4da7fc700 1 --2- 172.21.15.119:0/93394874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4dc101f20 0x7fe4dc073e50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000fa00 tx=0x7fe4d000fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:46.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.621+0000 7fe4c3fff700 1 -- 172.21.15.119:0/93394874 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d001c070 con 0x7fe4dc101f20 2023-12-07T10:20:46.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.621+0000 7fe4c3fff700 1 -- 172.21.15.119:0/93394874 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4d000baa0 con 0x7fe4dc101f20 2023-12-07T10:20:46.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.621+0000 7fe4e15ff700 1 -- 172.21.15.119:0/93394874 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d0009520 con 0x7fe4dc101f20 2023-12-07T10:20:46.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.621+0000 7fe4c3fff700 1 -- 172.21.15.119:0/93394874 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0004930 con 0x7fe4dc101f20 2023-12-07T10:20:46.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.621+0000 7fe4e15ff700 1 -- 172.21.15.119:0/93394874 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4dc1a6580 con 0x7fe4dc101f20 2023-12-07T10:20:46.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.625+0000 7fe4c3fff700 1 -- 172.21.15.119:0/93394874 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 24) v1 ==== 136660+0+0 (secure 0 0 0) 0x7fe4d0020070 con 0x7fe4dc101f20 2023-12-07T10:20:46.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.626+0000 7fe4f4f0bb80 1 Processor -- start 2023-12-07T10:20:46.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.626+0000 7fe4f4f0bb80 1 -- start start 2023-12-07T10:20:46.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.626+0000 7fe4f4f0bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581f36beb40 0x5581f36bef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.627+0000 7fe4f4f0bb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581f36bf460 0x5581f36c2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.627+0000 7fe4f4f0bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581f36c3150 0x5581f36c3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.627+0000 7fe4f4f0bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f36327c0 con 0x5581f36c3150 2023-12-07T10:20:46.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.627+0000 7fe4f4f0bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f36197a0 con 0x5581f36beb40 2023-12-07T10:20:46.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.627+0000 7fe4f4f0bb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f36170c0 con 0x5581f36bf460 2023-12-07T10:20:46.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.627+0000 7fe4da7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581f36c3150 0x5581f36c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.627+0000 7fe4db7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581f36beb40 0x5581f36bef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.627+0000 7fe4da7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581f36c3150 0x5581f36c3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37592/0 (socket says 172.21.15.119:37592) 2023-12-07T10:20:46.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.627+0000 7fe4da7fc700 1 -- 172.21.15.119:0/19766168 learned_addr learned my addr 172.21.15.119:0/19766168 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:46.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.627+0000 7fe4c3fff700 1 --2- 172.21.15.119:0/93394874 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe4c40a1d10 0x7fe4c40a41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.627+0000 7fe4daffd700 1 --2- 172.21.15.119:0/19766168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581f36bf460 0x5581f36c2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.627+0000 7fe4c3fff700 1 -- 172.21.15.119:0/93394874 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fe4d00cea30 con 0x7fe4dc101f20 2023-12-07T10:20:46.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.627+0000 7fe4db7fe700 1 --2- 172.21.15.119:0/93394874 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe4c40a1d10 0x7fe4c40a41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.627+0000 7fe4da7fc700 1 -- 172.21.15.119:0/19766168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581f36bf460 msgr2=0x5581f36c2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:46.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.627+0000 7fe4da7fc700 1 --2- 172.21.15.119:0/19766168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581f36bf460 0x5581f36c2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.627+0000 7fe4da7fc700 1 -- 172.21.15.119:0/19766168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581f36beb40 msgr2=0x5581f36bef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:46.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.627+0000 7fe4da7fc700 1 --2- 172.21.15.119:0/19766168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581f36beb40 0x5581f36bef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.627+0000 7fe4da7fc700 1 -- 172.21.15.119:0/19766168 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581f348f630 con 0x5581f36c3150 2023-12-07T10:20:46.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.628+0000 7fe4da7fc700 1 --2- 172.21.15.119:0/19766168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581f36c3150 0x5581f36c3570 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00d1fd0 tx=0x7fe4d00094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67bc98cb7eb706a server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:46.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.628+0000 7fe4db7fe700 1 --2- 172.21.15.119:0/93394874 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe4c40a1d10 0x7fe4c40a41d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009ec0 tx=0x7fe4d4004b40 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:46.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.628+0000 7fe4c17fa700 1 -- 172.21.15.119:0/19766168 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d009b110 con 0x5581f36c3150 2023-12-07T10:20:46.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.628+0000 7fe4c17fa700 1 -- 172.21.15.119:0/19766168 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4d009ae80 con 0x5581f36c3150 2023-12-07T10:20:46.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.628+0000 7fe4c17fa700 1 -- 172.21.15.119:0/19766168 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d00a3640 con 0x5581f36c3150 2023-12-07T10:20:46.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.629+0000 7fe4f4f0bb80 1 -- 172.21.15.119:0/19766168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581f36c3150 msgr2=0x5581f36c3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.629+0000 7fe4f4f0bb80 1 --2- 172.21.15.119:0/19766168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581f36c3150 0x5581f36c3570 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00d1fd0 tx=0x7fe4d00094b0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.629+0000 7fe4f4f0bb80 1 -- 172.21.15.119:0/19766168 shutdown_connections 2023-12-07T10:20:46.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.629+0000 7fe4f4f0bb80 1 --2- 172.21.15.119:0/19766168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581f36c3150 0x5581f36c3570 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.629+0000 7fe4f4f0bb80 1 --2- 172.21.15.119:0/19766168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581f36beb40 0x5581f36bef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.629+0000 7fe4f4f0bb80 1 --2- 172.21.15.119:0/19766168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581f36bf460 0x5581f36c2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.629+0000 7fe4f4f0bb80 1 -- 172.21.15.119:0/19766168 >> 172.21.15.119:0/19766168 conn(0x5581f35ed360 msgr2=0x5581f36b9130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:46.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.629+0000 7fe4f4f0bb80 1 -- 172.21.15.119:0/19766168 shutdown_connections 2023-12-07T10:20:46.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.629+0000 7fe4f4f0bb80 1 -- 172.21.15.119:0/19766168 wait complete. 2023-12-07T10:20:46.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.629+0000 7fe4f4f0bb80 1 -- start start 2023-12-07T10:20:46.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.630+0000 7fe4f4f0bb80 1 Processor -- start 2023-12-07T10:20:46.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.630+0000 7fe4f4f0bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581f36beb40 0x5581f36ca930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.630+0000 7fe4f4f0bb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581f36bf460 0x5581f36c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.630+0000 7fe4f4f0bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581f36c3150 0x5581f36c8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.630+0000 7fe4f4f0bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f36129b0 con 0x5581f36beb40 2023-12-07T10:20:46.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.630+0000 7fe4f4f0bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f35cc560 con 0x5581f36c3150 2023-12-07T10:20:46.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.630+0000 7fe4f4f0bb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f35cc200 con 0x5581f36bf460 2023-12-07T10:20:46.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.630+0000 7fe4f4f0bb80 10 client.0 _ll_register_callbacks cb 0x5581f35b3940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:20:46.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.630+0000 7fe4daffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581f36beb40 0x5581f36ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.630+0000 7fe4daffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581f36beb40 0x5581f36ca930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37606/0 (socket says 172.21.15.119:37606) 2023-12-07T10:20:46.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.630+0000 7fe4daffd700 1 -- 172.21.15.119:0/560157148 learned_addr learned my addr 172.21.15.119:0/560157148 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:46.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.630+0000 7fe4db7fe700 1 --2- 172.21.15.119:0/560157148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581f36c3150 0x5581f36c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.630+0000 7fe4da7fc700 1 --2- 172.21.15.119:0/560157148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581f36bf460 0x5581f36c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.631+0000 7fe4daffd700 1 -- 172.21.15.119:0/560157148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581f36bf460 msgr2=0x5581f36c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:46.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.631+0000 7fe4daffd700 1 --2- 172.21.15.119:0/560157148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581f36bf460 0x5581f36c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.631+0000 7fe4daffd700 1 -- 172.21.15.119:0/560157148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581f36c3150 msgr2=0x5581f36c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:46.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.631+0000 7fe4daffd700 1 --2- 172.21.15.119:0/560157148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581f36c3150 0x5581f36c8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.631+0000 7fe4daffd700 1 -- 172.21.15.119:0/560157148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4dc1a6580 con 0x5581f36beb40 2023-12-07T10:20:46.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.631+0000 7fe4daffd700 1 --2- 172.21.15.119:0/560157148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581f36beb40 0x5581f36ca930 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc005d10 tx=0x7fe4cc012ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db47bb249783791d server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:46.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.631+0000 7fe4beff5700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:20:46.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.631+0000 7fe4beff5700 1 -- 172.21.15.119:0/560157148 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc017070 con 0x5581f36beb40 2023-12-07T10:20:46.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.631+0000 7fe4f4f0bb80 1 -- 172.21.15.119:0/560157148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581f36b1b00 con 0x5581f36beb40 2023-12-07T10:20:46.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.631+0000 7fe4beff5700 1 -- 172.21.15.119:0/560157148 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fe4cc00f7d0 con 0x5581f36beb40 2023-12-07T10:20:46.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.631+0000 7fe4beff5700 1 -- 172.21.15.119:0/560157148 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4cc00f990 con 0x5581f36beb40 2023-12-07T10:20:46.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.631+0000 7fe4f4f0bb80 10 client.15756 fetch_fsmap learned FSMap version 30 2023-12-07T10:20:46.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.631+0000 7fe4f4f0bb80 1 -- 172.21.15.119:0/560157148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5581f343bde0 con 0x5581f36beb40 2023-12-07T10:20:46.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.632+0000 7fe4beff5700 1 -- 172.21.15.119:0/560157148 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5581f36b1b00 con 0x5581f36beb40 2023-12-07T10:20:46.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.632+0000 7fe4beff5700 1 -- 172.21.15.119:0/560157148 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe4cc010b50 con 0x5581f36beb40 2023-12-07T10:20:46.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.632+0000 7fe4f4f0bb80 10 client.15756 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:20:46.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.632+0000 7fe4f4f0bb80 10 client.15756 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-12-07T10:20:46.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.632+0000 7fe4f4f0bb80 20 client.15756 populate_metadata read hostname 'smithi119' 2023-12-07T10:20:46.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.632+0000 7fe4f4f0bb80 1 --2- 172.21.15.119:0/560157148 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x5581f36d8000 0x5581f36da420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.632+0000 7fe4f4f0bb80 4 client.15756 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:20:46.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.632+0000 7fe4f4f0bb80 1 -- 172.21.15.119:0/560157148 --> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581f368d7a0 con 0x5581f36d8000 2023-12-07T10:20:46.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.632+0000 7fe4da7fc700 1 --2- 172.21.15.119:0/560157148 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x5581f36d8000 0x5581f36da420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.633+0000 7fe4beff5700 10 client.15756 ms_handle_connect on v2:172.21.15.159:6834/2406380792 2023-12-07T10:20:46.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.633+0000 7fe4da7fc700 1 --2- 172.21.15.119:0/560157148 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x5581f36d8000 0x5581f36da420 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e309b9631ab0faa3 server_cookie=d836c5bf87099620 in_seq=0 out_seq=0 2023-12-07T10:20:46.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.633+0000 7fe4beff5700 1 -- 172.21.15.119:0/560157148 <== mds.3 v2:172.21.15.159:6834/2406380792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5581f368d7a0 con 0x5581f36d8000 2023-12-07T10:20:46.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.633+0000 7fe4beff5700 10 client.15756 handle_command_reply: tid=0 2023-12-07T10:20:46.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.633+0000 7fe4f4f0bb80 1 client.15756 shutdown 2023-12-07T10:20:46.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.634+0000 7fe4f4f0bb80 1 -- 172.21.15.119:0/560157148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581f36beb40 msgr2=0x5581f36ca930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.634+0000 7fe4f4f0bb80 1 --2- 172.21.15.119:0/560157148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581f36beb40 0x5581f36ca930 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc005d10 tx=0x7fe4cc012ce0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.634+0000 7fe4f4f0bb80 1 -- 172.21.15.119:0/560157148 shutdown_connections 2023-12-07T10:20:46.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.634+0000 7fe4f4f0bb80 1 --2- 172.21.15.119:0/560157148 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x5581f36d8000 0x5581f36da420 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.634+0000 7fe4f4f0bb80 1 --2- 172.21.15.119:0/560157148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581f36c3150 0x5581f36c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.634+0000 7fe4f4f0bb80 1 --2- 172.21.15.119:0/560157148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581f36beb40 0x5581f36ca930 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.634+0000 7fe4beff5700 0 client.15756 ms_handle_reset on v2:172.21.15.159:6834/2406380792 2023-12-07T10:20:46.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.634+0000 7fe4f4f0bb80 1 --2- 172.21.15.119:0/560157148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581f36bf460 0x5581f36c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.634+0000 7fe4f4f0bb80 1 -- 172.21.15.119:0/560157148 >> 172.21.15.119:0/560157148 conn(0x5581f35ed360 msgr2=0x5581f36c4620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:46.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.634+0000 7fe4f4f0bb80 1 -- 172.21.15.119:0/560157148 shutdown_connections 2023-12-07T10:20:46.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.634+0000 7fe4f4f0bb80 1 -- 172.21.15.119:0/560157148 wait complete. 2023-12-07T10:20:46.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.634+0000 7fe4f4f0bb80 20 client.15756 trim_cache size 0 max 16384 2023-12-07T10:20:46.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.647+0000 7fe4f4f0bb80 1 Processor -- start 2023-12-07T10:20:46.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.648+0000 7fe4f4f0bb80 1 -- start start 2023-12-07T10:20:46.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.648+0000 7fe4f4f0bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581f36beb40 0x5581f36dedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.648+0000 7fe4f4f0bb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581f36bf460 0x5581f36cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.648+0000 7fe4f4f0bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581f36c3150 0x5581f36cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.648+0000 7fe4f4f0bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f3491920 con 0x5581f36beb40 2023-12-07T10:20:46.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.648+0000 7fe4f4f0bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f36b5800 con 0x5581f36c3150 2023-12-07T10:20:46.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.648+0000 7fe4f4f0bb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f36ba7d0 con 0x5581f36bf460 2023-12-07T10:20:46.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.648+0000 7fe4daffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581f36beb40 0x5581f36dedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.648+0000 7fe4daffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581f36beb40 0x5581f36dedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37610/0 (socket says 172.21.15.119:37610) 2023-12-07T10:20:46.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.648+0000 7fe4daffd700 1 -- 172.21.15.119:0/3304516894 learned_addr learned my addr 172.21.15.119:0/3304516894 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:46.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.648+0000 7fe4da7fc700 1 --2- 172.21.15.119:0/3304516894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581f36bf460 0x5581f36cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.649+0000 7fe4db7fe700 1 --2- 172.21.15.119:0/3304516894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581f36c3150 0x5581f36cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.649+0000 7fe4daffd700 1 -- 172.21.15.119:0/3304516894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581f36bf460 msgr2=0x5581f36cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:46.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.649+0000 7fe4daffd700 1 --2- 172.21.15.119:0/3304516894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581f36bf460 0x5581f36cefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.649+0000 7fe4daffd700 1 -- 172.21.15.119:0/3304516894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581f36c3150 msgr2=0x5581f36cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:46.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.649+0000 7fe4daffd700 1 --2- 172.21.15.119:0/3304516894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581f36c3150 0x5581f36cf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.649+0000 7fe4daffd700 1 -- 172.21.15.119:0/3304516894 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581f3424c60 con 0x5581f36beb40 2023-12-07T10:20:46.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.649+0000 7fe4daffd700 1 --2- 172.21.15.119:0/3304516894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581f36beb40 0x5581f36dedc0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc000c00 tx=0x7fe4cc0099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506b6be8b74683cb server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:46.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.649+0000 7fe4c1ffb700 1 -- 172.21.15.119:0/3304516894 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc017070 con 0x5581f36beb40 2023-12-07T10:20:46.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.649+0000 7fe4c1ffb700 1 -- 172.21.15.119:0/3304516894 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4cc018040 con 0x5581f36beb40 2023-12-07T10:20:46.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.649+0000 7fe4c1ffb700 1 -- 172.21.15.119:0/3304516894 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc005040 con 0x5581f36beb40 2023-12-07T10:20:46.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.650+0000 7fe4f4f0bb80 1 -- 172.21.15.119:0/3304516894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581f36beb40 msgr2=0x5581f36dedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.650+0000 7fe4f4f0bb80 1 --2- 172.21.15.119:0/3304516894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581f36beb40 0x5581f36dedc0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc000c00 tx=0x7fe4cc0099f0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.650+0000 7fe4f4f0bb80 1 -- 172.21.15.119:0/3304516894 shutdown_connections 2023-12-07T10:20:46.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.650+0000 7fe4f4f0bb80 1 --2- 172.21.15.119:0/3304516894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581f36c3150 0x5581f36cf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.650+0000 7fe4f4f0bb80 1 --2- 172.21.15.119:0/3304516894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581f36beb40 0x5581f36dedc0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.650+0000 7fe4f4f0bb80 1 --2- 172.21.15.119:0/3304516894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581f36bf460 0x5581f36cefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.650+0000 7fe4f4f0bb80 1 -- 172.21.15.119:0/3304516894 >> 172.21.15.119:0/3304516894 conn(0x5581f35ed360 msgr2=0x5581f36b3490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:46.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.650+0000 7fe4f4f0bb80 1 -- 172.21.15.119:0/3304516894 shutdown_connections 2023-12-07T10:20:46.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.650+0000 7fe4f4f0bb80 1 -- 172.21.15.119:0/3304516894 wait complete. 2023-12-07T10:20:46.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.650+0000 7fe4f4f0bb80 1 -- start start 2023-12-07T10:20:46.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.651+0000 7fe4f4f0bb80 1 Processor -- start 2023-12-07T10:20:46.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.651+0000 7fe4f4f0bb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581f36beb40 0x5581f36ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.651+0000 7fe4f4f0bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581f36bf460 0x5581f36cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.651+0000 7fe4f4f0bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581f36c3150 0x5581f36cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.651+0000 7fe4f4f0bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f372d540 con 0x5581f36c3150 2023-12-07T10:20:46.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.651+0000 7fe4f4f0bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f372d680 con 0x5581f36bf460 2023-12-07T10:20:46.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.651+0000 7fe4f4f0bb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581f372d7c0 con 0x5581f36beb40 2023-12-07T10:20:46.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.651+0000 7fe4f4f0bb80 10 client.0 _ll_register_callbacks cb 0x5581f35b3940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:20:46.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.651+0000 7fe4da7fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581f36bf460 0x5581f36cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.652+0000 7fe4da7fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581f36bf460 0x5581f36cd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:37926/0 (socket says 172.21.15.119:37926) 2023-12-07T10:20:46.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.652+0000 7fe4da7fc700 1 -- 172.21.15.119:0/264107206 learned_addr learned my addr 172.21.15.119:0/264107206 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:46.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.652+0000 7fe4db7fe700 1 --2- 172.21.15.119:0/264107206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581f36c3150 0x5581f36cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.652+0000 7fe4daffd700 1 --2- 172.21.15.119:0/264107206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581f36beb40 0x5581f36ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.652+0000 7fe4da7fc700 1 -- 172.21.15.119:0/264107206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581f36beb40 msgr2=0x5581f36ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:46.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.652+0000 7fe4da7fc700 1 --2- 172.21.15.119:0/264107206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581f36beb40 0x5581f36ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.652+0000 7fe4da7fc700 1 -- 172.21.15.119:0/264107206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581f36c3150 msgr2=0x5581f36cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:46.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.652+0000 7fe4da7fc700 1 --2- 172.21.15.119:0/264107206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581f36c3150 0x5581f36cda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.652+0000 7fe4da7fc700 1 -- 172.21.15.119:0/264107206 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581f343bde0 con 0x5581f36bf460 2023-12-07T10:20:46.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.652+0000 7fe4da7fc700 1 --2- 172.21.15.119:0/264107206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581f36bf460 0x5581f36cd4d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe4d009b2d0 tx=0x7fe4d00054e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ac5ebde24ea7df0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:46.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.652+0000 7fe4beff5700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:20:46.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.652+0000 7fe4beff5700 1 -- 172.21.15.119:0/264107206 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d00b1020 con 0x5581f36bf460 2023-12-07T10:20:46.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.652+0000 7fe4f4f0bb80 1 -- 172.21.15.119:0/264107206 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4d009ae80 con 0x5581f36bf460 2023-12-07T10:20:46.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.652+0000 7fe4beff5700 1 -- 172.21.15.119:0/264107206 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fe4d009ba40 con 0x5581f36bf460 2023-12-07T10:20:46.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.652+0000 7fe4beff5700 1 -- 172.21.15.119:0/264107206 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4d00a3640 con 0x5581f36bf460 2023-12-07T10:20:46.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.652+0000 7fe4f4f0bb80 10 client.25790 fetch_fsmap learned FSMap version 30 2023-12-07T10:20:46.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.652+0000 7fe4f4f0bb80 1 -- 172.21.15.119:0/264107206 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5581f3462250 con 0x5581f36bf460 2023-12-07T10:20:46.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.653+0000 7fe4beff5700 1 -- 172.21.15.119:0/264107206 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4d009ae80 con 0x5581f36bf460 2023-12-07T10:20:46.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.653+0000 7fe4beff5700 1 -- 172.21.15.119:0/264107206 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe4d0005140 con 0x5581f36bf460 2023-12-07T10:20:46.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.653+0000 7fe4f4f0bb80 10 client.25790 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:20:46.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.653+0000 7fe4f4f0bb80 10 client.25790 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-12-07T10:20:46.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.653+0000 7fe4f4f0bb80 20 client.25790 populate_metadata read hostname 'smithi119' 2023-12-07T10:20:46.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.653+0000 7fe4f4f0bb80 1 --2- 172.21.15.119:0/264107206 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x5581f372ec50 0x5581f3731070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.653+0000 7fe4f4f0bb80 4 client.25790 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:20:46.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.653+0000 7fe4f4f0bb80 1 -- 172.21.15.119:0/264107206 --> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5581f35d8fd0 con 0x5581f372ec50 2023-12-07T10:20:46.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.653+0000 7fe4daffd700 1 --2- 172.21.15.119:0/264107206 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x5581f372ec50 0x5581f3731070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.654+0000 7fe4daffd700 1 --2- 172.21.15.119:0/264107206 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x5581f372ec50 0x5581f3731070 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=57bce4208557f37d server_cookie=c624802574239713 in_seq=0 out_seq=0 2023-12-07T10:20:46.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.654+0000 7fe4beff5700 10 client.25790 ms_handle_connect on v2:172.21.15.159:6834/2406380792 2023-12-07T10:20:46.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.654+0000 7fe4beff5700 1 -- 172.21.15.119:0/264107206 <== mds.3 v2:172.21.15.159:6834/2406380792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5581f35d8fd0 con 0x5581f372ec50 2023-12-07T10:20:46.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.654+0000 7fe4beff5700 10 client.25790 handle_command_reply: tid=0 2023-12-07T10:20:46.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.654+0000 7fe4f4f0bb80 1 client.25790 shutdown 2023-12-07T10:20:46.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.654+0000 7fe4f4f0bb80 1 -- 172.21.15.119:0/264107206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581f36bf460 msgr2=0x5581f36cd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.654+0000 7fe4f4f0bb80 1 --2- 172.21.15.119:0/264107206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581f36bf460 0x5581f36cd4d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe4d009b2d0 tx=0x7fe4d00054e0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.654+0000 7fe4f4f0bb80 1 -- 172.21.15.119:0/264107206 shutdown_connections 2023-12-07T10:20:46.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.654+0000 7fe4f4f0bb80 1 --2- 172.21.15.119:0/264107206 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x5581f372ec50 0x5581f3731070 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.654+0000 7fe4f4f0bb80 1 --2- 172.21.15.119:0/264107206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581f36c3150 0x5581f36cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.654+0000 7fe4f4f0bb80 1 --2- 172.21.15.119:0/264107206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581f36beb40 0x5581f36ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.654+0000 7fe4beff5700 0 client.25790 ms_handle_reset on v2:172.21.15.159:6834/2406380792 2023-12-07T10:20:46.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.654+0000 7fe4f4f0bb80 1 --2- 172.21.15.119:0/264107206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581f36bf460 0x5581f36cd4d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.654+0000 7fe4f4f0bb80 1 -- 172.21.15.119:0/264107206 >> 172.21.15.119:0/264107206 conn(0x5581f35ed360 msgr2=0x5581f36dedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:46.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.655+0000 7fe4f4f0bb80 1 -- 172.21.15.119:0/264107206 shutdown_connections 2023-12-07T10:20:46.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.655+0000 7fe4f4f0bb80 1 -- 172.21.15.119:0/264107206 wait complete. 2023-12-07T10:20:46.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.655+0000 7fe4f4f0bb80 20 client.25790 trim_cache size 0 max 16384 2023-12-07T10:20:46.661 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:20:46.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.655+0000 7fe4e15ff700 1 -- 172.21.15.119:0/93394874 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe4c40a1d10 msgr2=0x7fe4c40a41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.655+0000 7fe4e15ff700 1 --2- 172.21.15.119:0/93394874 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe4c40a1d10 0x7fe4c40a41d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009ec0 tx=0x7fe4d4004b40 comp rx=0 tx=0).stop 2023-12-07T10:20:46.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.655+0000 7fe4e15ff700 1 -- 172.21.15.119:0/93394874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4dc101f20 msgr2=0x7fe4dc073e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.655+0000 7fe4e15ff700 1 --2- 172.21.15.119:0/93394874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4dc101f20 0x7fe4dc073e50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000fa00 tx=0x7fe4d000fa30 comp rx=0 tx=0).stop 2023-12-07T10:20:46.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.655+0000 7fe4e15ff700 1 -- 172.21.15.119:0/93394874 shutdown_connections 2023-12-07T10:20:46.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.655+0000 7fe4e15ff700 1 --2- 172.21.15.119:0/93394874 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe4c40a1d10 0x7fe4c40a41d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.655+0000 7fe4e15ff700 1 --2- 172.21.15.119:0/93394874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4dc107620 0x7fe4dc074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.655+0000 7fe4e15ff700 1 --2- 172.21.15.119:0/93394874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4dc101560 0x7fe4dc073910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.655+0000 7fe4e15ff700 1 --2- 172.21.15.119:0/93394874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4dc101f20 0x7fe4dc073e50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.655+0000 7fe4e15ff700 1 -- 172.21.15.119:0/93394874 >> 172.21.15.119:0/93394874 conn(0x7fe4dc0fcfd0 msgr2=0x7fe4dc0fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:46.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.655+0000 7fe4e15ff700 1 -- 172.21.15.119:0/93394874 shutdown_connections 2023-12-07T10:20:46.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.656+0000 7fe4e15ff700 1 -- 172.21.15.119:0/93394874 wait complete. 2023-12-07T10:20:46.675 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-07T10:20:46.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.873+0000 7f1df49ff700 1 -- 172.21.15.119:0/2712754385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1df0107620 msgr2=0x7f1df0107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.873+0000 7f1df49ff700 1 --2- 172.21.15.119:0/2712754385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1df0107620 0x7f1df0107a00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1de00099c0 tx=0x7f1de0009cd0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.873+0000 7f1df49ff700 1 -- 172.21.15.119:0/2712754385 shutdown_connections 2023-12-07T10:20:46.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.873+0000 7f1df49ff700 1 --2- 172.21.15.119:0/2712754385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1df0101f20 0x7f1df010e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.873+0000 7f1df49ff700 1 --2- 172.21.15.119:0/2712754385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1df0107620 0x7f1df0107a00 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.873+0000 7f1df49ff700 1 --2- 172.21.15.119:0/2712754385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1df0101560 0x7f1df01019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.873+0000 7f1df49ff700 1 -- 172.21.15.119:0/2712754385 >> 172.21.15.119:0/2712754385 conn(0x7f1df00fcfd0 msgr2=0x7f1df00ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:46.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.873+0000 7f1df49ff700 1 -- 172.21.15.119:0/2712754385 shutdown_connections 2023-12-07T10:20:46.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.873+0000 7f1df49ff700 1 -- 172.21.15.119:0/2712754385 wait complete. 2023-12-07T10:20:46.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.874+0000 7f1df49ff700 1 Processor -- start 2023-12-07T10:20:46.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.874+0000 7f1df49ff700 1 -- start start 2023-12-07T10:20:46.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.874+0000 7f1df49ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1df0101560 0x7f1df019f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.874+0000 7f1df49ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1df0101f20 0x7f1df019ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.874+0000 7f1df49ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1df0107620 0x7f1df0199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.874+0000 7f1df49ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df019a150 con 0x7f1df0101560 2023-12-07T10:20:46.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.874+0000 7f1df49ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df019a2c0 con 0x7f1df0107620 2023-12-07T10:20:46.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.874+0000 7f1df49ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df019a480 con 0x7f1df0101f20 2023-12-07T10:20:46.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.875+0000 7f1dee59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1df0101560 0x7f1df019f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.875+0000 7f1dee59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1df0101560 0x7f1df019f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37640/0 (socket says 172.21.15.119:37640) 2023-12-07T10:20:46.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.875+0000 7f1dee59c700 1 -- 172.21.15.119:0/3918463933 learned_addr learned my addr 172.21.15.119:0/3918463933 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:46.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.875+0000 7f1dedd9b700 1 --2- 172.21.15.119:0/3918463933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1df0101f20 0x7f1df019ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.875+0000 7f1deed9d700 1 --2- 172.21.15.119:0/3918463933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1df0107620 0x7f1df0199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.875+0000 7f1dee59c700 1 -- 172.21.15.119:0/3918463933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1df0101f20 msgr2=0x7f1df019ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.875+0000 7f1dee59c700 1 --2- 172.21.15.119:0/3918463933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1df0101f20 0x7f1df019ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.875+0000 7f1dee59c700 1 -- 172.21.15.119:0/3918463933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1df0107620 msgr2=0x7f1df0199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.875+0000 7f1dee59c700 1 --2- 172.21.15.119:0/3918463933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1df0107620 0x7f1df0199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.875+0000 7f1dee59c700 1 -- 172.21.15.119:0/3918463933 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de00095f0 con 0x7f1df0101560 2023-12-07T10:20:46.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.875+0000 7f1dee59c700 1 --2- 172.21.15.119:0/3918463933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1df0101560 0x7f1df019f9c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1de000b580 tx=0x7f1de000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:46.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.876+0000 7f1ddf7fe700 1 -- 172.21.15.119:0/3918463933 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de001c070 con 0x7f1df0101560 2023-12-07T10:20:46.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.876+0000 7f1ddf7fe700 1 -- 172.21.15.119:0/3918463933 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1de000fe30 con 0x7f1df0101560 2023-12-07T10:20:46.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.876+0000 7f1df49ff700 1 -- 172.21.15.119:0/3918463933 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df019a700 con 0x7f1df0101560 2023-12-07T10:20:46.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.876+0000 7f1ddf7fe700 1 -- 172.21.15.119:0/3918463933 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de0022d40 con 0x7f1df0101560 2023-12-07T10:20:46.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.876+0000 7f1df49ff700 1 -- 172.21.15.119:0/3918463933 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df01a69b0 con 0x7f1df0101560 2023-12-07T10:20:46.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.880+0000 7f1ddf7fe700 1 -- 172.21.15.119:0/3918463933 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 24) v1 ==== 136660+0+0 (secure 0 0 0) 0x7f1de0020070 con 0x7f1df0101560 2023-12-07T10:20:46.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.881+0000 7f1e08364b80 1 Processor -- start 2023-12-07T10:20:46.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.881+0000 7f1e08364b80 1 -- start start 2023-12-07T10:20:46.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.881+0000 7f1e08364b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472683b40 0x561472683f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.881+0000 7f1ddf7fe700 1 --2- 172.21.15.119:0/3918463933 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f1dd80a1d10 0x7f1dd80a41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.881+0000 7f1e08364b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561472684460 0x561472687c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.881+0000 7f1e08364b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561472688150 0x561472688570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.881+0000 7f1e08364b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614725f77c0 con 0x561472683b40 2023-12-07T10:20:46.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.881+0000 7f1e08364b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614725de7a0 con 0x561472688150 2023-12-07T10:20:46.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.881+0000 7f1e08364b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614725dc0c0 con 0x561472684460 2023-12-07T10:20:46.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.882+0000 7f1ddf7fe700 1 -- 172.21.15.119:0/3918463933 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f1de00ceaf0 con 0x7f1df0101560 2023-12-07T10:20:46.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.882+0000 7f1dee59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472683b40 0x561472683f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.882+0000 7f1dee59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472683b40 0x561472683f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37650/0 (socket says 172.21.15.119:37650) 2023-12-07T10:20:46.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.882+0000 7f1dee59c700 1 -- 172.21.15.119:0/1986271922 learned_addr learned my addr 172.21.15.119:0/1986271922 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:46.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.882+0000 7f1dedd9b700 1 --2- 172.21.15.119:0/1986271922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561472684460 0x561472687c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.882+0000 7f1deed9d700 1 --2- 172.21.15.119:0/3918463933 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f1dd80a1d10 0x7f1dd80a41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.882+0000 7f1deed9d700 1 --2- 172.21.15.119:0/1986271922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561472688150 0x561472688570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.882+0000 7f1dee59c700 1 -- 172.21.15.119:0/1986271922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561472684460 msgr2=0x561472687c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:46.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.882+0000 7f1dee59c700 1 --2- 172.21.15.119:0/1986271922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561472684460 0x561472687c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.882+0000 7f1dee59c700 1 -- 172.21.15.119:0/1986271922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561472688150 msgr2=0x561472688570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:46.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.882+0000 7f1dee59c700 1 --2- 172.21.15.119:0/1986271922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561472688150 0x561472688570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.882+0000 7f1dee59c700 1 -- 172.21.15.119:0/1986271922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561472454630 con 0x561472683b40 2023-12-07T10:20:46.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.882+0000 7f1deed9d700 1 --2- 172.21.15.119:0/1986271922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561472688150 0x561472688570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:20:46.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.882+0000 7f1dee59c700 1 --2- 172.21.15.119:0/1986271922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472683b40 0x561472683f20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009fd0 tx=0x7f1de0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4183d8b5683a748 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:46.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.883+0000 7f1ddcff9700 1 -- 172.21.15.119:0/1986271922 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de009bb00 con 0x561472683b40 2023-12-07T10:20:46.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.883+0000 7f1deed9d700 1 --2- 172.21.15.119:0/3918463933 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f1dd80a1d10 0x7f1dd80a41d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1de8009e20 tx=0x7f1de8009050 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:46.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.883+0000 7f1ddcff9700 1 -- 172.21.15.119:0/1986271922 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1de009b400 con 0x561472683b40 2023-12-07T10:20:46.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.883+0000 7f1ddcff9700 1 -- 172.21.15.119:0/1986271922 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de0022680 con 0x561472683b40 2023-12-07T10:20:46.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.883+0000 7f1e08364b80 1 -- 172.21.15.119:0/1986271922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472683b40 msgr2=0x561472683f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.883+0000 7f1e08364b80 1 --2- 172.21.15.119:0/1986271922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472683b40 0x561472683f20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009fd0 tx=0x7f1de0005c00 comp rx=0 tx=0).stop 2023-12-07T10:20:46.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.883+0000 7f1e08364b80 1 -- 172.21.15.119:0/1986271922 shutdown_connections 2023-12-07T10:20:46.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.883+0000 7f1e08364b80 1 --2- 172.21.15.119:0/1986271922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561472688150 0x561472688570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.883+0000 7f1e08364b80 1 --2- 172.21.15.119:0/1986271922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472683b40 0x561472683f20 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.883+0000 7f1e08364b80 1 --2- 172.21.15.119:0/1986271922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561472684460 0x561472687c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.884+0000 7f1e08364b80 1 -- 172.21.15.119:0/1986271922 >> 172.21.15.119:0/1986271922 conn(0x5614725b2360 msgr2=0x56147267e130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:46.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.884+0000 7f1e08364b80 1 -- 172.21.15.119:0/1986271922 shutdown_connections 2023-12-07T10:20:46.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.884+0000 7f1e08364b80 1 -- 172.21.15.119:0/1986271922 wait complete. 2023-12-07T10:20:46.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.884+0000 7f1e08364b80 1 -- start start 2023-12-07T10:20:46.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.884+0000 7f1e08364b80 1 Processor -- start 2023-12-07T10:20:46.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.885+0000 7f1e08364b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472683b40 0x56147268f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.885+0000 7f1e08364b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561472684460 0x56147268cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.885+0000 7f1e08364b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561472688150 0x56147268d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.885+0000 7f1e08364b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614725d79b0 con 0x561472683b40 2023-12-07T10:20:46.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.885+0000 7f1e08364b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x561472591560 con 0x561472688150 2023-12-07T10:20:46.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.885+0000 7f1e08364b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561472591200 con 0x561472684460 2023-12-07T10:20:46.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.885+0000 7f1e08364b80 10 client.0 _ll_register_callbacks cb 0x561472578940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:20:46.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.885+0000 7f1dee59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472683b40 0x56147268f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.885+0000 7f1dee59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472683b40 0x56147268f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37654/0 (socket says 172.21.15.119:37654) 2023-12-07T10:20:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.885+0000 7f1dee59c700 1 -- 172.21.15.119:0/1378235904 learned_addr learned my addr 172.21.15.119:0/1378235904 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.885+0000 7f1dedd9b700 1 --2- 172.21.15.119:0/1378235904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561472684460 0x56147268cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.885+0000 7f1deed9d700 1 --2- 172.21.15.119:0/1378235904 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561472688150 0x56147268d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.885+0000 7f1dee59c700 1 -- 172.21.15.119:0/1378235904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561472684460 msgr2=0x56147268cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.885+0000 7f1dee59c700 1 --2- 172.21.15.119:0/1378235904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561472684460 0x56147268cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.885+0000 7f1dee59c700 1 -- 172.21.15.119:0/1378235904 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561472688150 msgr2=0x56147268d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.885+0000 7f1dee59c700 1 --2- 172.21.15.119:0/1378235904 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561472688150 0x56147268d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.885+0000 7f1dee59c700 1 -- 172.21.15.119:0/1378235904 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df01a69b0 con 0x561472683b40 2023-12-07T10:20:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.886+0000 7f1dee59c700 1 --2- 172.21.15.119:0/1378235904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472683b40 0x56147268f900 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1de0097900 tx=0x7f1de0005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1aa5bed889d2a server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.886+0000 7f1dd5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:20:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.886+0000 7f1dd5ffb700 1 -- 172.21.15.119:0/1378235904 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de00a3700 con 0x561472683b40 2023-12-07T10:20:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.886+0000 7f1dd5ffb700 1 -- 172.21.15.119:0/1378235904 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f1de009f550 con 0x561472683b40 2023-12-07T10:20:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.886+0000 7f1e08364b80 1 -- 172.21.15.119:0/1378235904 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561472676b00 con 0x561472683b40 2023-12-07T10:20:46.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.886+0000 7f1dd5ffb700 1 -- 172.21.15.119:0/1378235904 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1de009b870 con 0x561472683b40 2023-12-07T10:20:46.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.886+0000 7f1e08364b80 10 client.15774 fetch_fsmap learned FSMap version 30 2023-12-07T10:20:46.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.886+0000 7f1e08364b80 1 -- 172.21.15.119:0/1378235904 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561472400de0 con 0x561472683b40 2023-12-07T10:20:46.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.886+0000 7f1dd5ffb700 1 -- 172.21.15.119:0/1378235904 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561472676b00 con 0x561472683b40 2023-12-07T10:20:46.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.886+0000 7f1dd5ffb700 1 -- 172.21.15.119:0/1378235904 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1de00cc080 con 0x561472683b40 2023-12-07T10:20:46.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.886+0000 7f1e08364b80 10 client.15774 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:20:46.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.886+0000 7f1e08364b80 10 client.15774 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-07T10:20:46.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.886+0000 7f1e08364b80 20 client.15774 populate_metadata read hostname 'smithi119' 2023-12-07T10:20:46.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.886+0000 7f1e08364b80 1 --2- 172.21.15.119:0/1378235904 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x56147269ca40 0x56147269ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.886+0000 7f1e08364b80 4 client.15774 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:20:46.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.886+0000 7f1e08364b80 1 -- 172.21.15.119:0/1378235904 --> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614726527a0 con 0x56147269ca40 2023-12-07T10:20:46.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.887+0000 7f1dedd9b700 1 --2- 172.21.15.119:0/1378235904 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x56147269ca40 0x56147269ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.887+0000 7f1dedd9b700 1 --2- 172.21.15.119:0/1378235904 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x56147269ca40 0x56147269ee30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=72a3e3a480deb8e3 server_cookie=4453171a44d84832 in_seq=0 out_seq=0 2023-12-07T10:20:46.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.887+0000 7f1dd5ffb700 10 client.15774 ms_handle_connect on v2:172.21.15.159:6836/3106097520 2023-12-07T10:20:46.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.888+0000 7f1dd5ffb700 1 -- 172.21.15.119:0/1378235904 <== mds.2 v2:172.21.15.159:6836/3106097520 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5614726527a0 con 0x56147269ca40 2023-12-07T10:20:46.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.888+0000 7f1dd5ffb700 10 client.15774 handle_command_reply: tid=0 2023-12-07T10:20:46.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.888+0000 7f1e08364b80 1 client.15774 shutdown 2023-12-07T10:20:46.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.888+0000 7f1e08364b80 1 -- 172.21.15.119:0/1378235904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472683b40 msgr2=0x56147268f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.888+0000 7f1e08364b80 1 --2- 172.21.15.119:0/1378235904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472683b40 0x56147268f900 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1de0097900 tx=0x7f1de0005b00 comp rx=0 tx=0).stop 2023-12-07T10:20:46.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.888+0000 7f1e08364b80 1 -- 172.21.15.119:0/1378235904 shutdown_connections 2023-12-07T10:20:46.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.888+0000 7f1e08364b80 1 --2- 172.21.15.119:0/1378235904 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x56147269ca40 0x56147269ee30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.888+0000 7f1e08364b80 1 --2- 172.21.15.119:0/1378235904 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561472688150 0x56147268d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.888+0000 7f1e08364b80 1 --2- 172.21.15.119:0/1378235904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472683b40 0x56147268f900 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.888+0000 7f1dd5ffb700 0 client.15774 ms_handle_reset on v2:172.21.15.159:6836/3106097520 2023-12-07T10:20:46.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.889+0000 7f1e08364b80 1 --2- 172.21.15.119:0/1378235904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561472684460 0x56147268cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.889+0000 7f1e08364b80 1 -- 172.21.15.119:0/1378235904 >> 172.21.15.119:0/1378235904 conn(0x5614725b2360 msgr2=0x561472678d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:46.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.889+0000 7f1e08364b80 1 -- 172.21.15.119:0/1378235904 shutdown_connections 2023-12-07T10:20:46.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.889+0000 7f1e08364b80 1 -- 172.21.15.119:0/1378235904 wait complete. 2023-12-07T10:20:46.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.889+0000 7f1e08364b80 20 client.15774 trim_cache size 0 max 16384 2023-12-07T10:20:46.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.902+0000 7f1e08364b80 1 Processor -- start 2023-12-07T10:20:46.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.902+0000 7f1e08364b80 1 -- start start 2023-12-07T10:20:46.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.902+0000 7f1e08364b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472683b40 0x5614726872d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.902+0000 7f1e08364b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561472684460 0x561472687810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.902+0000 7f1e08364b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561472688150 0x561472685b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.902+0000 7f1e08364b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561472456920 con 0x561472683b40 2023-12-07T10:20:46.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.903+0000 7f1e08364b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147268e1f0 con 0x561472684460 2023-12-07T10:20:46.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.903+0000 7f1e08364b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561472676b10 con 0x561472688150 2023-12-07T10:20:46.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.903+0000 7f1dee59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472683b40 0x5614726872d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.903+0000 7f1dee59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472683b40 0x5614726872d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37656/0 (socket says 172.21.15.119:37656) 2023-12-07T10:20:46.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.903+0000 7f1dee59c700 1 -- 172.21.15.119:0/1651524351 learned_addr learned my addr 172.21.15.119:0/1651524351 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:46.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.903+0000 7f1dedd9b700 1 --2- 172.21.15.119:0/1651524351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561472684460 0x561472687810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.903+0000 7f1deed9d700 1 --2- 172.21.15.119:0/1651524351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561472688150 0x561472685b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.903+0000 7f1dee59c700 1 -- 172.21.15.119:0/1651524351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561472688150 msgr2=0x561472685b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:46.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.903+0000 7f1dee59c700 1 --2- 172.21.15.119:0/1651524351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561472688150 0x561472685b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.903+0000 7f1dee59c700 1 -- 172.21.15.119:0/1651524351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561472684460 msgr2=0x561472687810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:46.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.903+0000 7f1dee59c700 1 --2- 172.21.15.119:0/1651524351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561472684460 0x561472687810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.903+0000 7f1dee59c700 1 -- 172.21.15.119:0/1651524351 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614723e9c60 con 0x561472683b40 2023-12-07T10:20:46.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.903+0000 7f1dedd9b700 1 --2- 172.21.15.119:0/1651524351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561472684460 0x561472687810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:20:46.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.903+0000 7f1dee59c700 1 --2- 172.21.15.119:0/1651524351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472683b40 0x5614726872d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009990 tx=0x7f1de0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f231b2f373bc83 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:46.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.903+0000 7f1ddd7fa700 1 -- 172.21.15.119:0/1651524351 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de00a3700 con 0x561472683b40 2023-12-07T10:20:46.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.904+0000 7f1ddd7fa700 1 -- 172.21.15.119:0/1651524351 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1de00cbd10 con 0x561472683b40 2023-12-07T10:20:46.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.904+0000 7f1ddd7fa700 1 -- 172.21.15.119:0/1651524351 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de00a3270 con 0x561472683b40 2023-12-07T10:20:46.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.904+0000 7f1e08364b80 1 -- 172.21.15.119:0/1651524351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472683b40 msgr2=0x5614726872d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.904+0000 7f1e08364b80 1 --2- 172.21.15.119:0/1651524351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472683b40 0x5614726872d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009990 tx=0x7f1de0009310 comp rx=0 tx=0).stop 2023-12-07T10:20:46.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.904+0000 7f1e08364b80 1 -- 172.21.15.119:0/1651524351 shutdown_connections 2023-12-07T10:20:46.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.904+0000 7f1e08364b80 1 --2- 172.21.15.119:0/1651524351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561472688150 0x561472685b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.904+0000 7f1e08364b80 1 --2- 172.21.15.119:0/1651524351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472683b40 0x5614726872d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.904+0000 7f1e08364b80 1 --2- 172.21.15.119:0/1651524351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561472684460 0x561472687810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.904+0000 7f1e08364b80 1 -- 172.21.15.119:0/1651524351 >> 172.21.15.119:0/1651524351 conn(0x5614725b2360 msgr2=0x56147268f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:46.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.904+0000 7f1e08364b80 1 -- 172.21.15.119:0/1651524351 shutdown_connections 2023-12-07T10:20:46.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.904+0000 7f1e08364b80 1 -- 172.21.15.119:0/1651524351 wait complete. 2023-12-07T10:20:46.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.905+0000 7f1e08364b80 1 -- start start 2023-12-07T10:20:46.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.905+0000 7f1e08364b80 1 Processor -- start 2023-12-07T10:20:46.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.905+0000 7f1e08364b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561472683b40 0x56147268e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.905+0000 7f1e08364b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472684460 0x56147268ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.905+0000 7f1e08364b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561472688150 0x561472682940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.905+0000 7f1e08364b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614726946f0 con 0x561472684460 2023-12-07T10:20:46.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.905+0000 7f1e08364b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x561472694860 con 0x561472688150 2023-12-07T10:20:46.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.905+0000 7f1e08364b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561472682e80 con 0x561472683b40 2023-12-07T10:20:46.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.905+0000 7f1e08364b80 10 client.0 _ll_register_callbacks cb 0x561472578940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:20:46.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.906+0000 7f1dedd9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472684460 0x56147268ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.906+0000 7f1dedd9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472684460 0x56147268ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37668/0 (socket says 172.21.15.119:37668) 2023-12-07T10:20:46.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.906+0000 7f1dedd9b700 1 -- 172.21.15.119:0/3846630950 learned_addr learned my addr 172.21.15.119:0/3846630950 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:46.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.906+0000 7f1dee59c700 1 --2- 172.21.15.119:0/3846630950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561472683b40 0x56147268e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.906+0000 7f1deed9d700 1 --2- 172.21.15.119:0/3846630950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561472688150 0x561472682940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.906+0000 7f1dedd9b700 1 -- 172.21.15.119:0/3846630950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561472683b40 msgr2=0x56147268e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:46.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.906+0000 7f1dedd9b700 1 --2- 172.21.15.119:0/3846630950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561472683b40 0x56147268e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.906+0000 7f1dedd9b700 1 -- 172.21.15.119:0/3846630950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561472688150 msgr2=0x561472682940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:46.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.906+0000 7f1dedd9b700 1 --2- 172.21.15.119:0/3846630950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561472688150 0x561472682940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.906+0000 7f1dedd9b700 1 -- 172.21.15.119:0/3846630950 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561472400de0 con 0x561472684460 2023-12-07T10:20:46.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.906+0000 7f1deed9d700 1 --2- 172.21.15.119:0/3846630950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561472688150 0x561472682940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:20:46.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.906+0000 7f1dedd9b700 1 --2- 172.21.15.119:0/3846630950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472684460 0x56147268ed10 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x56147268e1c0 tx=0x7f1de4004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ea0de4bcc48840 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:46.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.906+0000 7f1dd5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:20:46.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.906+0000 7f1dd5ffb700 1 -- 172.21.15.119:0/3846630950 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de4017030 con 0x561472684460 2023-12-07T10:20:46.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.906+0000 7f1dd5ffb700 1 -- 172.21.15.119:0/3846630950 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f1de4003bf0 con 0x561472684460 2023-12-07T10:20:46.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.906+0000 7f1e08364b80 1 -- 172.21.15.119:0/3846630950 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56147259dfd0 con 0x561472684460 2023-12-07T10:20:46.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.907+0000 7f1dd5ffb700 1 -- 172.21.15.119:0/3846630950 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1de4003db0 con 0x561472684460 2023-12-07T10:20:46.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.907+0000 7f1e08364b80 10 client.15780 fetch_fsmap learned FSMap version 30 2023-12-07T10:20:46.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.907+0000 7f1e08364b80 1 -- 172.21.15.119:0/3846630950 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561472427250 con 0x561472684460 2023-12-07T10:20:46.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.907+0000 7f1dd5ffb700 1 -- 172.21.15.119:0/3846630950 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56147259dfd0 con 0x561472684460 2023-12-07T10:20:46.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.907+0000 7f1dd5ffb700 1 -- 172.21.15.119:0/3846630950 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1de40047c0 con 0x561472684460 2023-12-07T10:20:46.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.907+0000 7f1e08364b80 10 client.15780 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:20:46.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.907+0000 7f1e08364b80 10 client.15780 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-07T10:20:46.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.907+0000 7f1e08364b80 20 client.15780 populate_metadata read hostname 'smithi119' 2023-12-07T10:20:46.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.907+0000 7f1e08364b80 1 --2- 172.21.15.119:0/3846630950 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x5614726f3ca0 0x5614726f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:46.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.907+0000 7f1e08364b80 4 client.15780 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:20:46.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.907+0000 7f1e08364b80 1 -- 172.21.15.119:0/3846630950 --> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561472590490 con 0x5614726f3ca0 2023-12-07T10:20:46.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.908+0000 7f1dee59c700 1 --2- 172.21.15.119:0/3846630950 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x5614726f3ca0 0x5614726f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:46.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.908+0000 7f1dd5ffb700 10 client.15780 ms_handle_connect on v2:172.21.15.159:6836/3106097520 2023-12-07T10:20:46.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.908+0000 7f1dee59c700 1 --2- 172.21.15.119:0/3846630950 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x5614726f3ca0 0x5614726f60c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=293d4ac348fa12f7 server_cookie=d8796e3b3685be3d in_seq=0 out_seq=0 2023-12-07T10:20:46.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.908+0000 7f1dd5ffb700 1 -- 172.21.15.119:0/3846630950 <== mds.2 v2:172.21.15.159:6836/3106097520 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f1de0064020 con 0x5614726f3ca0 2023-12-07T10:20:46.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.908+0000 7f1dd5ffb700 10 client.15780 handle_command_reply: tid=0 2023-12-07T10:20:46.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.908+0000 7f1e08364b80 1 client.15780 shutdown 2023-12-07T10:20:46.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.908+0000 7f1e08364b80 1 -- 172.21.15.119:0/3846630950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472684460 msgr2=0x56147268ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.908+0000 7f1e08364b80 1 --2- 172.21.15.119:0/3846630950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472684460 0x56147268ed10 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x56147268e1c0 tx=0x7f1de4004320 comp rx=0 tx=0).stop 2023-12-07T10:20:46.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.908+0000 7f1e08364b80 1 -- 172.21.15.119:0/3846630950 shutdown_connections 2023-12-07T10:20:46.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.908+0000 7f1e08364b80 1 --2- 172.21.15.119:0/3846630950 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x5614726f3ca0 0x5614726f60c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.908+0000 7f1e08364b80 1 --2- 172.21.15.119:0/3846630950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561472688150 0x561472682940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.908+0000 7f1e08364b80 1 --2- 172.21.15.119:0/3846630950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561472683b40 0x56147268e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.908+0000 7f1dd5ffb700 0 client.15780 ms_handle_reset on v2:172.21.15.159:6836/3106097520 2023-12-07T10:20:46.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.908+0000 7f1e08364b80 1 --2- 172.21.15.119:0/3846630950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561472684460 0x56147268ed10 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.908+0000 7f1e08364b80 1 -- 172.21.15.119:0/3846630950 >> 172.21.15.119:0/3846630950 conn(0x5614725b2360 msgr2=0x561472698ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:46.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.909+0000 7f1e08364b80 1 -- 172.21.15.119:0/3846630950 shutdown_connections 2023-12-07T10:20:46.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.909+0000 7f1e08364b80 1 -- 172.21.15.119:0/3846630950 wait complete. 2023-12-07T10:20:46.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.909+0000 7f1e08364b80 20 client.15780 trim_cache size 0 max 16384 2023-12-07T10:20:46.915 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:20:46.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.909+0000 7f1df49ff700 1 -- 172.21.15.119:0/3918463933 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f1dd80a1d10 msgr2=0x7f1dd80a41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.909+0000 7f1df49ff700 1 --2- 172.21.15.119:0/3918463933 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f1dd80a1d10 0x7f1dd80a41d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1de8009e20 tx=0x7f1de8009050 comp rx=0 tx=0).stop 2023-12-07T10:20:46.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.909+0000 7f1df49ff700 1 -- 172.21.15.119:0/3918463933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1df0101560 msgr2=0x7f1df019f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:46.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.909+0000 7f1df49ff700 1 --2- 172.21.15.119:0/3918463933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1df0101560 0x7f1df019f9c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1de000b580 tx=0x7f1de000f690 comp rx=0 tx=0).stop 2023-12-07T10:20:46.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.909+0000 7f1df49ff700 1 -- 172.21.15.119:0/3918463933 shutdown_connections 2023-12-07T10:20:46.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.909+0000 7f1df49ff700 1 --2- 172.21.15.119:0/3918463933 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f1dd80a1d10 0x7f1dd80a41d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.909+0000 7f1df49ff700 1 --2- 172.21.15.119:0/3918463933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1df0107620 0x7f1df0199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.909+0000 7f1df49ff700 1 --2- 172.21.15.119:0/3918463933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1df0101560 0x7f1df019f9c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.909+0000 7f1df49ff700 1 --2- 172.21.15.119:0/3918463933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1df0101f20 0x7f1df019ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:46.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.909+0000 7f1df49ff700 1 -- 172.21.15.119:0/3918463933 >> 172.21.15.119:0/3918463933 conn(0x7f1df00fcfd0 msgr2=0x7f1df00fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:46.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.910+0000 7f1df49ff700 1 -- 172.21.15.119:0/3918463933 shutdown_connections 2023-12-07T10:20:46.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:46.910+0000 7f1df49ff700 1 -- 172.21.15.119:0/3918463933 wait complete. 2023-12-07T10:20:46.929 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-07T10:20:47.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.130+0000 7f04e07ff700 1 -- 172.21.15.119:0/2857252745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d8107620 msgr2=0x7f04d8107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:47.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.130+0000 7f04e07ff700 1 --2- 172.21.15.119:0/2857252745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d8107620 0x7f04d8107a00 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f04d40099c0 tx=0x7f04d4009cd0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.130+0000 7f04e07ff700 1 -- 172.21.15.119:0/2857252745 shutdown_connections 2023-12-07T10:20:47.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.130+0000 7f04e07ff700 1 --2- 172.21.15.119:0/2857252745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04d8101f20 0x7f04d810e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.130+0000 7f04e07ff700 1 --2- 172.21.15.119:0/2857252745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d8107620 0x7f04d8107a00 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.130+0000 7f04e07ff700 1 --2- 172.21.15.119:0/2857252745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04d8101560 0x7f04d81019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.130+0000 7f04e07ff700 1 -- 172.21.15.119:0/2857252745 >> 172.21.15.119:0/2857252745 conn(0x7f04d80fcfd0 msgr2=0x7f04d80ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:47.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.130+0000 7f04e07ff700 1 -- 172.21.15.119:0/2857252745 shutdown_connections 2023-12-07T10:20:47.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.130+0000 7f04e07ff700 1 -- 172.21.15.119:0/2857252745 wait complete. 2023-12-07T10:20:47.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.131+0000 7f04e07ff700 1 Processor -- start 2023-12-07T10:20:47.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.131+0000 7f04e07ff700 1 -- start start 2023-12-07T10:20:47.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.132+0000 7f04e07ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d8101560 0x7f04d819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.132+0000 7f04e07ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04d8101f20 0x7f04d819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.132+0000 7f04e07ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04d8107620 0x7f04d8197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.132+0000 7f04e07ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d8197e00 con 0x7f04d8101560 2023-12-07T10:20:47.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.132+0000 7f04e07ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d8197f70 con 0x7f04d8107620 2023-12-07T10:20:47.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.132+0000 7f04e07ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d8198130 con 0x7f04d8101f20 2023-12-07T10:20:47.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.132+0000 7f04de59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d8101560 0x7f04d819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.132+0000 7f04de59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d8101560 0x7f04d819d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37690/0 (socket says 172.21.15.119:37690) 2023-12-07T10:20:47.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.132+0000 7f04de59b700 1 -- 172.21.15.119:0/1146851401 learned_addr learned my addr 172.21.15.119:0/1146851401 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:47.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.132+0000 7f04ddd9a700 1 --2- 172.21.15.119:0/1146851401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04d8101f20 0x7f04d819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.132+0000 7f04ded9c700 1 --2- 172.21.15.119:0/1146851401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04d8107620 0x7f04d8197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.132+0000 7f04de59b700 1 -- 172.21.15.119:0/1146851401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04d8101f20 msgr2=0x7f04d819dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:47.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.132+0000 7f04de59b700 1 --2- 172.21.15.119:0/1146851401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04d8101f20 0x7f04d819dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.132+0000 7f04de59b700 1 -- 172.21.15.119:0/1146851401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04d8107620 msgr2=0x7f04d8197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:47.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.132+0000 7f04de59b700 1 --2- 172.21.15.119:0/1146851401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04d8107620 0x7f04d8197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.132+0000 7f04de59b700 1 -- 172.21.15.119:0/1146851401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d40095f0 con 0x7f04d8101560 2023-12-07T10:20:47.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.133+0000 7f04de59b700 1 --2- 172.21.15.119:0/1146851401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d8101560 0x7f04d819d660 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f04d400b580 tx=0x7f04d400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:47.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.133+0000 7f04cf7fe700 1 -- 172.21.15.119:0/1146851401 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d401c070 con 0x7f04d8101560 2023-12-07T10:20:47.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.133+0000 7f04cf7fe700 1 -- 172.21.15.119:0/1146851401 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04d400fd40 con 0x7f04d8101560 2023-12-07T10:20:47.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.133+0000 7f04e07ff700 1 -- 172.21.15.119:0/1146851401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d81983b0 con 0x7f04d8101560 2023-12-07T10:20:47.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.133+0000 7f04cf7fe700 1 -- 172.21.15.119:0/1146851401 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d4022b80 con 0x7f04d8101560 2023-12-07T10:20:47.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.133+0000 7f04e07ff700 1 -- 172.21.15.119:0/1146851401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d81a4650 con 0x7f04d8101560 2023-12-07T10:20:47.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.136+0000 7f04cf7fe700 1 -- 172.21.15.119:0/1146851401 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 24) v1 ==== 136660+0+0 (secure 0 0 0) 0x7f04d4020070 con 0x7f04d8101560 2023-12-07T10:20:47.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.137+0000 7f04cf7fe700 1 --2- 172.21.15.119:0/1146851401 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f04c40a1d90 0x7f04c40a4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.137+0000 7f04cf7fe700 1 -- 172.21.15.119:0/1146851401 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f04d40ceba0 con 0x7f04d8101560 2023-12-07T10:20:47.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.137+0000 7f04f4191b80 1 Processor -- start 2023-12-07T10:20:47.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.137+0000 7f04ddd9a700 1 --2- 172.21.15.119:0/1146851401 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f04c40a1d90 0x7f04c40a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.138+0000 7f04f4191b80 1 -- start start 2023-12-07T10:20:47.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.138+0000 7f04ddd9a700 1 --2- 172.21.15.119:0/1146851401 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f04c40a1d90 0x7f04c40a4250 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f04d8198bb0 tx=0x7f04c8009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:47.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.138+0000 7f04f4191b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777925f50 0x55c777926330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.138+0000 7f04f4191b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c777926870 0x55c77792a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.138+0000 7f04f4191b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c77792a560 0x55c77792a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.138+0000 7f04f4191b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7778957c0 con 0x55c777925f50 2023-12-07T10:20:47.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.138+0000 7f04f4191b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77787c7a0 con 0x55c777926870 2023-12-07T10:20:47.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.138+0000 7f04f4191b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77787a0c0 con 0x55c77792a560 2023-12-07T10:20:47.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.138+0000 7f04de59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777925f50 0x55c777926330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.138+0000 7f04de59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777925f50 0x55c777926330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37698/0 (socket says 172.21.15.119:37698) 2023-12-07T10:20:47.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.138+0000 7f04de59b700 1 -- 172.21.15.119:0/3169883278 learned_addr learned my addr 172.21.15.119:0/3169883278 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:47.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.138+0000 7f04ddd9a700 1 --2- 172.21.15.119:0/3169883278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c777926870 0x55c77792a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.138+0000 7f04ded9c700 1 --2- 172.21.15.119:0/3169883278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c77792a560 0x55c77792a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.139+0000 7f04de59b700 1 -- 172.21.15.119:0/3169883278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c77792a560 msgr2=0x55c77792a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:47.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.139+0000 7f04de59b700 1 --2- 172.21.15.119:0/3169883278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c77792a560 0x55c77792a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.139+0000 7f04de59b700 1 -- 172.21.15.119:0/3169883278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c777926870 msgr2=0x55c77792a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:47.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.139+0000 7f04de59b700 1 --2- 172.21.15.119:0/3169883278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c777926870 0x55c77792a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.139+0000 7f04de59b700 1 -- 172.21.15.119:0/3169883278 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7776f2630 con 0x55c777925f50 2023-12-07T10:20:47.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.139+0000 7f04de59b700 1 --2- 172.21.15.119:0/3169883278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777925f50 0x55c777926330 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009990 tx=0x7f04d409b370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f861450795daf7 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:47.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.139+0000 7f04ccff9700 1 -- 172.21.15.119:0/3169883278 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d409bbb0 con 0x55c777925f50 2023-12-07T10:20:47.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.139+0000 7f04ccff9700 1 -- 172.21.15.119:0/3169883278 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04d409b4b0 con 0x55c777925f50 2023-12-07T10:20:47.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.139+0000 7f04ccff9700 1 -- 172.21.15.119:0/3169883278 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d409b970 con 0x55c777925f50 2023-12-07T10:20:47.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.140+0000 7f04f4191b80 1 -- 172.21.15.119:0/3169883278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777925f50 msgr2=0x55c777926330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:47.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.140+0000 7f04f4191b80 1 --2- 172.21.15.119:0/3169883278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777925f50 0x55c777926330 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009990 tx=0x7f04d409b370 comp rx=0 tx=0).stop 2023-12-07T10:20:47.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.140+0000 7f04f4191b80 1 -- 172.21.15.119:0/3169883278 shutdown_connections 2023-12-07T10:20:47.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.140+0000 7f04f4191b80 1 --2- 172.21.15.119:0/3169883278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c77792a560 0x55c77792a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.140+0000 7f04f4191b80 1 --2- 172.21.15.119:0/3169883278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777925f50 0x55c777926330 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.140+0000 7f04f4191b80 1 --2- 172.21.15.119:0/3169883278 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c777926870 0x55c77792a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.140+0000 7f04f4191b80 1 -- 172.21.15.119:0/3169883278 >> 172.21.15.119:0/3169883278 conn(0x55c777850360 msgr2=0x55c77791c130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:47.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.140+0000 7f04f4191b80 1 -- 172.21.15.119:0/3169883278 shutdown_connections 2023-12-07T10:20:47.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.140+0000 7f04f4191b80 1 -- 172.21.15.119:0/3169883278 wait complete. 2023-12-07T10:20:47.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.141+0000 7f04f4191b80 1 -- start start 2023-12-07T10:20:47.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.141+0000 7f04f4191b80 1 Processor -- start 2023-12-07T10:20:47.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.142+0000 7f04f4191b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c777925f50 0x55c777931cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.142+0000 7f04f4191b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777926870 0x55c77792f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.142+0000 7f04f4191b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c77792a560 0x55c77792f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.142+0000 7f04f4191b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7778759b0 con 0x55c777926870 2023-12-07T10:20:47.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.142+0000 7f04f4191b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77782f560 con 0x55c77792a560 2023-12-07T10:20:47.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.142+0000 7f04f4191b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77782f200 con 0x55c777925f50 2023-12-07T10:20:47.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.142+0000 7f04f4191b80 10 client.0 _ll_register_callbacks cb 0x55c777816940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:20:47.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.142+0000 7f04ddd9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777926870 0x55c77792f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.142+0000 7f04ddd9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777926870 0x55c77792f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37714/0 (socket says 172.21.15.119:37714) 2023-12-07T10:20:47.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.142+0000 7f04ddd9a700 1 -- 172.21.15.119:0/1178958873 learned_addr learned my addr 172.21.15.119:0/1178958873 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:47.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.142+0000 7f04de59b700 1 --2- 172.21.15.119:0/1178958873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c777925f50 0x55c777931cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.142+0000 7f04ded9c700 1 --2- 172.21.15.119:0/1178958873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c77792a560 0x55c77792f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.142+0000 7f04ddd9a700 1 -- 172.21.15.119:0/1178958873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c777925f50 msgr2=0x55c777931cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:47.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.142+0000 7f04ddd9a700 1 --2- 172.21.15.119:0/1178958873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c777925f50 0x55c777931cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.142+0000 7f04ddd9a700 1 -- 172.21.15.119:0/1178958873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c77792a560 msgr2=0x55c77792f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:47.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.142+0000 7f04ddd9a700 1 --2- 172.21.15.119:0/1178958873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c77792a560 0x55c77792f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.142+0000 7f04ddd9a700 1 -- 172.21.15.119:0/1178958873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d81a4650 con 0x55c777926870 2023-12-07T10:20:47.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.142+0000 7f04ded9c700 1 --2- 172.21.15.119:0/1178958873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c77792a560 0x55c77792f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:20:47.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.143+0000 7f04ddd9a700 1 --2- 172.21.15.119:0/1178958873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777926870 0x55c77792f030 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f04c8011fa0 tx=0x7f04c80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb5539c2a35af14 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:47.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.143+0000 7f04c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:20:47.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.143+0000 7f04c1ffb700 1 -- 172.21.15.119:0/1178958873 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c8022070 con 0x55c777926870 2023-12-07T10:20:47.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.143+0000 7f04f4191b80 1 -- 172.21.15.119:0/1178958873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c777914b00 con 0x55c777926870 2023-12-07T10:20:47.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.143+0000 7f04c1ffb700 1 -- 172.21.15.119:0/1178958873 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f04c8003d70 con 0x55c777926870 2023-12-07T10:20:47.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.143+0000 7f04c1ffb700 1 -- 172.21.15.119:0/1178958873 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04c8004580 con 0x55c777926870 2023-12-07T10:20:47.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.143+0000 7f04f4191b80 10 client.15792 fetch_fsmap learned FSMap version 30 2023-12-07T10:20:47.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.143+0000 7f04f4191b80 1 -- 172.21.15.119:0/1178958873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c77769ede0 con 0x55c777926870 2023-12-07T10:20:47.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.143+0000 7f04c1ffb700 1 -- 172.21.15.119:0/1178958873 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c777914b00 con 0x55c777926870 2023-12-07T10:20:47.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.144+0000 7f04c1ffb700 1 -- 172.21.15.119:0/1178958873 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f04c801d670 con 0x55c777926870 2023-12-07T10:20:47.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.144+0000 7f04f4191b80 10 client.15792 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:20:47.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.144+0000 7f04f4191b80 10 client.15792 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2023-12-07T10:20:47.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.144+0000 7f04f4191b80 20 client.15792 populate_metadata read hostname 'smithi119' 2023-12-07T10:20:47.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.144+0000 7f04f4191b80 1 --2- 172.21.15.119:0/1178958873 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55c77793f390 0x55c7779417b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.144+0000 7f04f4191b80 4 client.15792 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:20:47.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.144+0000 7f04f4191b80 1 -- 172.21.15.119:0/1178958873 --> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7778f07a0 con 0x55c77793f390 2023-12-07T10:20:47.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.144+0000 7f04de59b700 1 --2- 172.21.15.119:0/1178958873 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55c77793f390 0x55c7779417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.144+0000 7f04c1ffb700 10 client.15792 ms_handle_connect on v2:172.21.15.119:6838/1920762602 2023-12-07T10:20:47.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.144+0000 7f04de59b700 1 --2- 172.21.15.119:0/1178958873 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55c77793f390 0x55c7779417b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9ab75fd68f4a603 server_cookie=a39b7235d66318f9 in_seq=0 out_seq=0 2023-12-07T10:20:47.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.145+0000 7f04c1ffb700 1 -- 172.21.15.119:0/1178958873 <== mds.1 v2:172.21.15.119:6838/1920762602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c7778f07a0 con 0x55c77793f390 2023-12-07T10:20:47.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.145+0000 7f04c1ffb700 10 client.15792 handle_command_reply: tid=0 2023-12-07T10:20:47.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.145+0000 7f04f4191b80 1 client.15792 shutdown 2023-12-07T10:20:47.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.145+0000 7f04f4191b80 1 -- 172.21.15.119:0/1178958873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777926870 msgr2=0x55c77792f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:47.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.145+0000 7f04f4191b80 1 --2- 172.21.15.119:0/1178958873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777926870 0x55c77792f030 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f04c8011fa0 tx=0x7f04c80047c0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.145+0000 7f04f4191b80 1 -- 172.21.15.119:0/1178958873 shutdown_connections 2023-12-07T10:20:47.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.145+0000 7f04f4191b80 1 --2- 172.21.15.119:0/1178958873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c77792a560 0x55c77792f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.145+0000 7f04f4191b80 1 --2- 172.21.15.119:0/1178958873 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55c77793f390 0x55c7779417b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.145+0000 7f04f4191b80 1 --2- 172.21.15.119:0/1178958873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c777925f50 0x55c777931cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.145+0000 7f04f4191b80 1 --2- 172.21.15.119:0/1178958873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777926870 0x55c77792f030 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.145+0000 7f04f4191b80 1 -- 172.21.15.119:0/1178958873 >> 172.21.15.119:0/1178958873 conn(0x55c777850360 msgr2=0x55c777916d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:47.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.145+0000 7f04c1ffb700 0 client.15792 ms_handle_reset on v2:172.21.15.119:6838/1920762602 2023-12-07T10:20:47.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.146+0000 7f04f4191b80 1 -- 172.21.15.119:0/1178958873 shutdown_connections 2023-12-07T10:20:47.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.146+0000 7f04f4191b80 1 -- 172.21.15.119:0/1178958873 wait complete. 2023-12-07T10:20:47.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.146+0000 7f04f4191b80 20 client.15792 trim_cache size 0 max 16384 2023-12-07T10:20:47.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.159+0000 7f04f4191b80 1 Processor -- start 2023-12-07T10:20:47.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.159+0000 7f04f4191b80 1 -- start start 2023-12-07T10:20:47.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.160+0000 7f04f4191b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c777925f50 0x55c777946110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.160+0000 7f04f4191b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777926870 0x55c77792e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.160+0000 7f04f4191b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c77792a560 0x55c77792eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.160+0000 7f04f4191b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7776f4920 con 0x55c777926870 2023-12-07T10:20:47.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.160+0000 7f04f4191b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77791dbe0 con 0x55c777925f50 2023-12-07T10:20:47.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.160+0000 7f04f4191b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77792aff0 con 0x55c77792a560 2023-12-07T10:20:47.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.160+0000 7f04ddd9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777926870 0x55c77792e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.160+0000 7f04ddd9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777926870 0x55c77792e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37716/0 (socket says 172.21.15.119:37716) 2023-12-07T10:20:47.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.160+0000 7f04ddd9a700 1 -- 172.21.15.119:0/1275972597 learned_addr learned my addr 172.21.15.119:0/1275972597 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:47.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.160+0000 7f04ded9c700 1 --2- 172.21.15.119:0/1275972597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c77792a560 0x55c77792eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.160+0000 7f04de59b700 1 --2- 172.21.15.119:0/1275972597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c777925f50 0x55c777946110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.160+0000 7f04ddd9a700 1 -- 172.21.15.119:0/1275972597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c77792a560 msgr2=0x55c77792eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:47.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.160+0000 7f04ddd9a700 1 --2- 172.21.15.119:0/1275972597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c77792a560 0x55c77792eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.160+0000 7f04ddd9a700 1 -- 172.21.15.119:0/1275972597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c777925f50 msgr2=0x55c777946110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:47.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.160+0000 7f04ddd9a700 1 --2- 172.21.15.119:0/1275972597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c777925f50 0x55c777946110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.160+0000 7f04ddd9a700 1 -- 172.21.15.119:0/1275972597 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c777687c60 con 0x55c777926870 2023-12-07T10:20:47.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.160+0000 7f04ddd9a700 1 --2- 172.21.15.119:0/1275972597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777926870 0x55c77792e570 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f04c8011850 tx=0x7f04c8009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b769b4fa1f94ed server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:47.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.161+0000 7f04cd7fa700 1 -- 172.21.15.119:0/1275972597 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c8022040 con 0x55c777926870 2023-12-07T10:20:47.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.161+0000 7f04cd7fa700 1 -- 172.21.15.119:0/1275972597 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04c8034480 con 0x55c777926870 2023-12-07T10:20:47.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.161+0000 7f04cd7fa700 1 -- 172.21.15.119:0/1275972597 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c8027680 con 0x55c777926870 2023-12-07T10:20:47.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.161+0000 7f04f4191b80 1 -- 172.21.15.119:0/1275972597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777926870 msgr2=0x55c77792e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:47.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.161+0000 7f04f4191b80 1 --2- 172.21.15.119:0/1275972597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777926870 0x55c77792e570 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f04c8011850 tx=0x7f04c8009370 comp rx=0 tx=0).stop 2023-12-07T10:20:47.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.161+0000 7f04f4191b80 1 -- 172.21.15.119:0/1275972597 shutdown_connections 2023-12-07T10:20:47.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.161+0000 7f04f4191b80 1 --2- 172.21.15.119:0/1275972597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c77792a560 0x55c77792eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.161+0000 7f04f4191b80 1 --2- 172.21.15.119:0/1275972597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c777925f50 0x55c777946110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.161+0000 7f04f4191b80 1 --2- 172.21.15.119:0/1275972597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777926870 0x55c77792e570 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.161+0000 7f04f4191b80 1 -- 172.21.15.119:0/1275972597 >> 172.21.15.119:0/1275972597 conn(0x55c777850360 msgr2=0x55c77793aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:47.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.162+0000 7f04f4191b80 1 -- 172.21.15.119:0/1275972597 shutdown_connections 2023-12-07T10:20:47.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.162+0000 7f04f4191b80 1 -- 172.21.15.119:0/1275972597 wait complete. 2023-12-07T10:20:47.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.162+0000 7f04f4191b80 1 -- start start 2023-12-07T10:20:47.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.162+0000 7f04f4191b80 1 Processor -- start 2023-12-07T10:20:47.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.163+0000 7f04f4191b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777925f50 0x55c7779325e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.163+0000 7f04f4191b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c777926870 0x55c777932b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.163+0000 7f04f4191b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c77792a560 0x55c777994880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.163+0000 7f04f4191b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c777933290 con 0x55c777925f50 2023-12-07T10:20:47.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.163+0000 7f04f4191b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c777994dc0 con 0x55c777926870 2023-12-07T10:20:47.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.163+0000 7f04f4191b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c777994f80 con 0x55c77792a560 2023-12-07T10:20:47.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.163+0000 7f04f4191b80 10 client.0 _ll_register_callbacks cb 0x55c777816940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:20:47.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.163+0000 7f04de59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777925f50 0x55c7779325e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.163+0000 7f04de59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777925f50 0x55c7779325e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37722/0 (socket says 172.21.15.119:37722) 2023-12-07T10:20:47.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.163+0000 7f04de59b700 1 -- 172.21.15.119:0/956379920 learned_addr learned my addr 172.21.15.119:0/956379920 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:47.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.163+0000 7f04ded9c700 1 --2- 172.21.15.119:0/956379920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c77792a560 0x55c777994880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.163+0000 7f04ddd9a700 1 --2- 172.21.15.119:0/956379920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c777926870 0x55c777932b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.164+0000 7f04de59b700 1 -- 172.21.15.119:0/956379920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c77792a560 msgr2=0x55c777994880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:47.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.164+0000 7f04de59b700 1 --2- 172.21.15.119:0/956379920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c77792a560 0x55c777994880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.164+0000 7f04de59b700 1 -- 172.21.15.119:0/956379920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c777926870 msgr2=0x55c777932b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:47.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.164+0000 7f04de59b700 1 --2- 172.21.15.119:0/956379920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c777926870 0x55c777932b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.164+0000 7f04de59b700 1 -- 172.21.15.119:0/956379920 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c802c040 con 0x55c777925f50 2023-12-07T10:20:47.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.164+0000 7f04de59b700 1 --2- 172.21.15.119:0/956379920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777925f50 0x55c7779325e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009d70 tx=0x7f04d4005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e1900b01ecff0a server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:47.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.164+0000 7f04c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:20:47.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.164+0000 7f04c1ffb700 1 -- 172.21.15.119:0/956379920 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d407b020 con 0x55c777925f50 2023-12-07T10:20:47.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.164+0000 7f04c1ffb700 1 -- 172.21.15.119:0/956379920 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f04d40a37b0 con 0x55c777925f50 2023-12-07T10:20:47.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.164+0000 7f04f4191b80 1 -- 172.21.15.119:0/956379920 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04d409b4b0 con 0x55c777925f50 2023-12-07T10:20:47.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.164+0000 7f04c1ffb700 1 -- 172.21.15.119:0/956379920 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04d407fa90 con 0x55c777925f50 2023-12-07T10:20:47.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.164+0000 7f04f4191b80 10 client.15798 fetch_fsmap learned FSMap version 30 2023-12-07T10:20:47.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.164+0000 7f04f4191b80 1 -- 172.21.15.119:0/956379920 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c7776c5250 con 0x55c777925f50 2023-12-07T10:20:47.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.164+0000 7f04c1ffb700 1 -- 172.21.15.119:0/956379920 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04d409b4b0 con 0x55c777925f50 2023-12-07T10:20:47.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.165+0000 7f04c1ffb700 1 -- 172.21.15.119:0/956379920 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f04d409b830 con 0x55c777925f50 2023-12-07T10:20:47.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.165+0000 7f04f4191b80 10 client.15798 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:20:47.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.165+0000 7f04f4191b80 10 client.15798 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2023-12-07T10:20:47.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.165+0000 7f04f4191b80 20 client.15798 populate_metadata read hostname 'smithi119' 2023-12-07T10:20:47.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.165+0000 7f04f4191b80 1 --2- 172.21.15.119:0/956379920 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55c777996390 0x55c7779987b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.165+0000 7f04f4191b80 4 client.15798 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:20:47.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.165+0000 7f04f4191b80 1 -- 172.21.15.119:0/956379920 --> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c77783bfd0 con 0x55c777996390 2023-12-07T10:20:47.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.165+0000 7f04ddd9a700 1 --2- 172.21.15.119:0/956379920 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55c777996390 0x55c7779987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.166+0000 7f04c1ffb700 10 client.15798 ms_handle_connect on v2:172.21.15.119:6838/1920762602 2023-12-07T10:20:47.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.166+0000 7f04ddd9a700 1 --2- 172.21.15.119:0/956379920 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55c777996390 0x55c7779987b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c79a5777ad62cc46 server_cookie=d23c4f8ea22852d4 in_seq=0 out_seq=0 2023-12-07T10:20:47.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.166+0000 7f04c1ffb700 1 -- 172.21.15.119:0/956379920 <== mds.1 v2:172.21.15.119:6838/1920762602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c77783bfd0 con 0x55c777996390 2023-12-07T10:20:47.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.166+0000 7f04c1ffb700 10 client.15798 handle_command_reply: tid=0 2023-12-07T10:20:47.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.166+0000 7f04f4191b80 1 client.15798 shutdown 2023-12-07T10:20:47.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.166+0000 7f04f4191b80 1 -- 172.21.15.119:0/956379920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777925f50 msgr2=0x55c7779325e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:47.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.166+0000 7f04f4191b80 1 --2- 172.21.15.119:0/956379920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777925f50 0x55c7779325e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009d70 tx=0x7f04d4005a90 comp rx=0 tx=0).stop 2023-12-07T10:20:47.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.166+0000 7f04f4191b80 1 -- 172.21.15.119:0/956379920 shutdown_connections 2023-12-07T10:20:47.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.166+0000 7f04f4191b80 1 --2- 172.21.15.119:0/956379920 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55c777996390 0x55c7779987b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.166+0000 7f04f4191b80 1 --2- 172.21.15.119:0/956379920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c77792a560 0x55c777994880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.166+0000 7f04f4191b80 1 --2- 172.21.15.119:0/956379920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c777925f50 0x55c7779325e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.166+0000 7f04f4191b80 1 --2- 172.21.15.119:0/956379920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c777926870 0x55c777932b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.166+0000 7f04c1ffb700 0 client.15798 ms_handle_reset on v2:172.21.15.119:6838/1920762602 2023-12-07T10:20:47.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.166+0000 7f04f4191b80 1 -- 172.21.15.119:0/956379920 >> 172.21.15.119:0/956379920 conn(0x55c777850360 msgr2=0x55c7779296a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:47.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.166+0000 7f04f4191b80 1 -- 172.21.15.119:0/956379920 shutdown_connections 2023-12-07T10:20:47.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.166+0000 7f04f4191b80 1 -- 172.21.15.119:0/956379920 wait complete. 2023-12-07T10:20:47.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.166+0000 7f04f4191b80 20 client.15798 trim_cache size 0 max 16384 2023-12-07T10:20:47.172 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:20:47.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.167+0000 7f04e07ff700 1 -- 172.21.15.119:0/1146851401 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f04c40a1d90 msgr2=0x7f04c40a4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:47.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.167+0000 7f04e07ff700 1 --2- 172.21.15.119:0/1146851401 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f04c40a1d90 0x7f04c40a4250 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f04d8198bb0 tx=0x7f04c8009260 comp rx=0 tx=0).stop 2023-12-07T10:20:47.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.167+0000 7f04e07ff700 1 -- 172.21.15.119:0/1146851401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d8101560 msgr2=0x7f04d819d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:47.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.167+0000 7f04e07ff700 1 --2- 172.21.15.119:0/1146851401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d8101560 0x7f04d819d660 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f04d400b580 tx=0x7f04d400bfd0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.167+0000 7f04e07ff700 1 -- 172.21.15.119:0/1146851401 shutdown_connections 2023-12-07T10:20:47.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.167+0000 7f04e07ff700 1 --2- 172.21.15.119:0/1146851401 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f04c40a1d90 0x7f04c40a4250 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.167+0000 7f04e07ff700 1 --2- 172.21.15.119:0/1146851401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f04d8107620 0x7f04d8197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.167+0000 7f04e07ff700 1 --2- 172.21.15.119:0/1146851401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d8101560 0x7f04d819d660 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.167+0000 7f04e07ff700 1 --2- 172.21.15.119:0/1146851401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04d8101f20 0x7f04d819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.167+0000 7f04e07ff700 1 -- 172.21.15.119:0/1146851401 >> 172.21.15.119:0/1146851401 conn(0x7f04d80fcfd0 msgr2=0x7f04d80fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:47.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.167+0000 7f04e07ff700 1 -- 172.21.15.119:0/1146851401 shutdown_connections 2023-12-07T10:20:47.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.167+0000 7f04e07ff700 1 -- 172.21.15.119:0/1146851401 wait complete. 2023-12-07T10:20:47.184 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:46 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2116017470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:20:47.186 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-07T10:20:47.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:46 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2116017470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:20:47.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:46 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2116017470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:20:47.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.385+0000 7f5afc7ff700 1 -- 172.21.15.119:0/4288286394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5af4101560 msgr2=0x7f5af41019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:47.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.385+0000 7f5afc7ff700 1 --2- 172.21.15.119:0/4288286394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5af4101560 0x7f5af41019e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40099c0 tx=0x7f5ae4009cd0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.385+0000 7f5afc7ff700 1 -- 172.21.15.119:0/4288286394 shutdown_connections 2023-12-07T10:20:47.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.385+0000 7f5afc7ff700 1 --2- 172.21.15.119:0/4288286394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5af4101f20 0x7f5af410e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.385+0000 7f5afc7ff700 1 --2- 172.21.15.119:0/4288286394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5af4107620 0x7f5af4107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.385+0000 7f5afc7ff700 1 --2- 172.21.15.119:0/4288286394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5af4101560 0x7f5af41019e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.385+0000 7f5afc7ff700 1 -- 172.21.15.119:0/4288286394 >> 172.21.15.119:0/4288286394 conn(0x7f5af40fcfd0 msgr2=0x7f5af40ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:47.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.385+0000 7f5afc7ff700 1 -- 172.21.15.119:0/4288286394 shutdown_connections 2023-12-07T10:20:47.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.386+0000 7f5afc7ff700 1 -- 172.21.15.119:0/4288286394 wait complete. 2023-12-07T10:20:47.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.387+0000 7f5afc7ff700 1 Processor -- start 2023-12-07T10:20:47.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.387+0000 7f5afc7ff700 1 -- start start 2023-12-07T10:20:47.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.388+0000 7f5afc7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5af4101560 0x7f5af419d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.388+0000 7f5afc7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5af4101f20 0x7f5af419dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.388+0000 7f5afc7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5af4107620 0x7f5af41977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.388+0000 7f5afc7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af4197e40 con 0x7f5af4101f20 2023-12-07T10:20:47.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.388+0000 7f5afc7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af4197fb0 con 0x7f5af4107620 2023-12-07T10:20:47.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.388+0000 7f5afc7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af4198170 con 0x7f5af4101560 2023-12-07T10:20:47.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.388+0000 7f5afa59b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5af4101560 0x7f5af419d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.388+0000 7f5afa59b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5af4101560 0x7f5af419d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:43842/0 (socket says 172.21.15.119:43842) 2023-12-07T10:20:47.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.388+0000 7f5afa59b700 1 -- 172.21.15.119:0/147891559 learned_addr learned my addr 172.21.15.119:0/147891559 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:47.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.388+0000 7f5af9d9a700 1 --2- 172.21.15.119:0/147891559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5af4101f20 0x7f5af419dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.388+0000 7f5afad9c700 1 --2- 172.21.15.119:0/147891559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5af4107620 0x7f5af41977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.388+0000 7f5afa59b700 1 -- 172.21.15.119:0/147891559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5af4107620 msgr2=0x7f5af41977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:47.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.388+0000 7f5afa59b700 1 --2- 172.21.15.119:0/147891559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5af4107620 0x7f5af41977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.388+0000 7f5afa59b700 1 -- 172.21.15.119:0/147891559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5af4101f20 msgr2=0x7f5af419dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:47.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.388+0000 7f5afa59b700 1 --2- 172.21.15.119:0/147891559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5af4101f20 0x7f5af419dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.388+0000 7f5afa59b700 1 -- 172.21.15.119:0/147891559 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae40095f0 con 0x7f5af4101560 2023-12-07T10:20:47.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.389+0000 7f5afa59b700 1 --2- 172.21.15.119:0/147891559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5af4101560 0x7f5af419d6a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5af000e9a0 tx=0x7f5af000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:47.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.389+0000 7f5aeb7fe700 1 -- 172.21.15.119:0/147891559 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0009790 con 0x7f5af4101560 2023-12-07T10:20:47.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.389+0000 7f5afc7ff700 1 -- 172.21.15.119:0/147891559 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af4198450 con 0x7f5af4101560 2023-12-07T10:20:47.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.389+0000 7f5aeb7fe700 1 -- 172.21.15.119:0/147891559 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5af0010430 con 0x7f5af4101560 2023-12-07T10:20:47.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.389+0000 7f5aeb7fe700 1 -- 172.21.15.119:0/147891559 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0018680 con 0x7f5af4101560 2023-12-07T10:20:47.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.389+0000 7f5afc7ff700 1 -- 172.21.15.119:0/147891559 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af41a4690 con 0x7f5af4101560 2023-12-07T10:20:47.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.394+0000 7f5aeb7fe700 1 -- 172.21.15.119:0/147891559 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 24) v1 ==== 136660+0+0 (secure 0 0 0) 0x7f5af00105a0 con 0x7f5af4101560 2023-12-07T10:20:47.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.394+0000 7f5b1021eb80 1 Processor -- start 2023-12-07T10:20:47.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.394+0000 7f5b1021eb80 1 -- start start 2023-12-07T10:20:47.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.395+0000 7f5b1021eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a81d419b40 0x55a81d419f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.395+0000 7f5b1021eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a81d41a460 0x55a81d41dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.395+0000 7f5b1021eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a81d41e150 0x55a81d41e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.395+0000 7f5b1021eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81d38d7c0 con 0x55a81d41a460 2023-12-07T10:20:47.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.395+0000 7f5b1021eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81d3747a0 con 0x55a81d41e150 2023-12-07T10:20:47.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.395+0000 7f5b1021eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81d3720c0 con 0x55a81d419b40 2023-12-07T10:20:47.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.395+0000 7f5afa59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a81d41a460 0x55a81d41dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.395+0000 7f5afa59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a81d41a460 0x55a81d41dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37760/0 (socket says 172.21.15.119:37760) 2023-12-07T10:20:47.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.395+0000 7f5afa59b700 1 -- 172.21.15.119:0/369518306 learned_addr learned my addr 172.21.15.119:0/369518306 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:47.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.395+0000 7f5afad9c700 1 --2- 172.21.15.119:0/369518306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a81d419b40 0x55a81d419f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.395+0000 7f5aeb7fe700 1 --2- 172.21.15.119:0/147891559 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5ae00a1d10 0x7f5ae00a41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.395+0000 7f5af9d9a700 1 --2- 172.21.15.119:0/369518306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a81d41e150 0x55a81d41e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.395+0000 7f5aeb7fe700 1 -- 172.21.15.119:0/147891559 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f5af0014070 con 0x7f5af4101560 2023-12-07T10:20:47.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.395+0000 7f5afa59b700 1 -- 172.21.15.119:0/369518306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a81d419b40 msgr2=0x55a81d419f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:47.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.395+0000 7f5afad9c700 1 --2- 172.21.15.119:0/147891559 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5ae00a1d10 0x7f5ae00a41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.395+0000 7f5afa59b700 1 --2- 172.21.15.119:0/369518306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a81d419b40 0x55a81d419f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.396+0000 7f5afa59b700 1 -- 172.21.15.119:0/369518306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a81d41e150 msgr2=0x55a81d41e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:47.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.396+0000 7f5afa59b700 1 --2- 172.21.15.119:0/369518306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a81d41e150 0x55a81d41e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.396+0000 7f5afa59b700 1 -- 172.21.15.119:0/369518306 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a81d1ea630 con 0x55a81d41a460 2023-12-07T10:20:47.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.396+0000 7f5afa59b700 1 --2- 172.21.15.119:0/369518306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a81d41a460 0x55a81d41dc10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5af00cb2f0 tx=0x7f5af0010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1d3ff5e230c538 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:47.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.396+0000 7f5ae8ff9700 1 -- 172.21.15.119:0/369518306 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af009a460 con 0x55a81d41a460 2023-12-07T10:20:47.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.396+0000 7f5ae8ff9700 1 -- 172.21.15.119:0/369518306 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5af009a280 con 0x55a81d41a460 2023-12-07T10:20:47.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.396+0000 7f5ae8ff9700 1 -- 172.21.15.119:0/369518306 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af00a2d40 con 0x55a81d41a460 2023-12-07T10:20:47.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.397+0000 7f5b1021eb80 1 -- 172.21.15.119:0/369518306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a81d41a460 msgr2=0x55a81d41dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:47.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.397+0000 7f5afad9c700 1 --2- 172.21.15.119:0/147891559 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5ae00a1d10 0x7f5ae00a41d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009ec0 tx=0x7f5aec004b40 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:47.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.397+0000 7f5b1021eb80 1 --2- 172.21.15.119:0/369518306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a81d41a460 0x55a81d41dc10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5af00cb2f0 tx=0x7f5af0010c80 comp rx=0 tx=0).stop 2023-12-07T10:20:47.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.397+0000 7f5b1021eb80 1 -- 172.21.15.119:0/369518306 shutdown_connections 2023-12-07T10:20:47.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.397+0000 7f5b1021eb80 1 --2- 172.21.15.119:0/369518306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a81d41e150 0x55a81d41e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.397+0000 7f5b1021eb80 1 --2- 172.21.15.119:0/369518306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a81d419b40 0x55a81d419f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.397+0000 7f5b1021eb80 1 --2- 172.21.15.119:0/369518306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a81d41a460 0x55a81d41dc10 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.397+0000 7f5b1021eb80 1 -- 172.21.15.119:0/369518306 >> 172.21.15.119:0/369518306 conn(0x55a81d348360 msgr2=0x55a81d414130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:47.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.397+0000 7f5b1021eb80 1 -- 172.21.15.119:0/369518306 shutdown_connections 2023-12-07T10:20:47.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.397+0000 7f5b1021eb80 1 -- 172.21.15.119:0/369518306 wait complete. 2023-12-07T10:20:47.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.397+0000 7f5b1021eb80 1 -- start start 2023-12-07T10:20:47.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.398+0000 7f5b1021eb80 1 Processor -- start 2023-12-07T10:20:47.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.398+0000 7f5b1021eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a81d419b40 0x55a81d425900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.398+0000 7f5b1021eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a81d41a460 0x55a81d422cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.398+0000 7f5b1021eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a81d41e150 0x55a81d423400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.398+0000 7f5b1021eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81d36d9b0 con 0x55a81d41a460 2023-12-07T10:20:47.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.398+0000 7f5b1021eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81d327560 con 0x55a81d419b40 2023-12-07T10:20:47.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.398+0000 7f5b1021eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81d327200 con 0x55a81d41e150 2023-12-07T10:20:47.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.398+0000 7f5b1021eb80 10 client.0 _ll_register_callbacks cb 0x55a81d30e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:20:47.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.398+0000 7f5af9d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a81d41a460 0x55a81d422cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.398+0000 7f5afa59b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a81d419b40 0x55a81d425900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.398+0000 7f5af9d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a81d41a460 0x55a81d422cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37772/0 (socket says 172.21.15.119:37772) 2023-12-07T10:20:47.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.399+0000 7f5af9d9a700 1 -- 172.21.15.119:0/3949702812 learned_addr learned my addr 172.21.15.119:0/3949702812 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:47.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.399+0000 7f5afad9c700 1 --2- 172.21.15.119:0/3949702812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a81d41e150 0x55a81d423400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.399+0000 7f5af9d9a700 1 -- 172.21.15.119:0/3949702812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a81d41e150 msgr2=0x55a81d423400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:47.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.399+0000 7f5af9d9a700 1 --2- 172.21.15.119:0/3949702812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a81d41e150 0x55a81d423400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.399+0000 7f5af9d9a700 1 -- 172.21.15.119:0/3949702812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a81d419b40 msgr2=0x55a81d425900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:47.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.399+0000 7f5af9d9a700 1 --2- 172.21.15.119:0/3949702812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a81d419b40 0x55a81d425900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.399+0000 7f5af9d9a700 1 -- 172.21.15.119:0/3949702812 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af41a4690 con 0x55a81d41a460 2023-12-07T10:20:47.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.399+0000 7f5afa59b700 1 --2- 172.21.15.119:0/3949702812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a81d419b40 0x55a81d425900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:47.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.399+0000 7f5af9d9a700 1 --2- 172.21.15.119:0/3949702812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a81d41a460 0x55a81d422cc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40099c0 tx=0x7f5ae4005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40f53a6243f89fe server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:47.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.399+0000 7f5addffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:20:47.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.399+0000 7f5addffb700 1 -- 172.21.15.119:0/3949702812 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae401d030 con 0x55a81d41a460 2023-12-07T10:20:47.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.399+0000 7f5addffb700 1 -- 172.21.15.119:0/3949702812 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f5ae400bba0 con 0x55a81d41a460 2023-12-07T10:20:47.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.399+0000 7f5b1021eb80 1 -- 172.21.15.119:0/3949702812 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a81d40cb00 con 0x55a81d41a460 2023-12-07T10:20:47.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.400+0000 7f5addffb700 1 -- 172.21.15.119:0/3949702812 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ae400bd60 con 0x55a81d41a460 2023-12-07T10:20:47.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.400+0000 7f5b1021eb80 10 client.15810 fetch_fsmap learned FSMap version 30 2023-12-07T10:20:47.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.400+0000 7f5b1021eb80 1 -- 172.21.15.119:0/3949702812 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a81d196de0 con 0x55a81d41a460 2023-12-07T10:20:47.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.400+0000 7f5addffb700 1 -- 172.21.15.119:0/3949702812 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a81d40cb00 con 0x55a81d41a460 2023-12-07T10:20:47.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.400+0000 7f5addffb700 1 -- 172.21.15.119:0/3949702812 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5ae4024770 con 0x55a81d41a460 2023-12-07T10:20:47.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.400+0000 7f5b1021eb80 10 client.15810 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:20:47.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.400+0000 7f5b1021eb80 10 client.15810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:20:47.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.400+0000 7f5b1021eb80 20 client.15810 populate_metadata read hostname 'smithi119' 2023-12-07T10:20:47.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.400+0000 7f5b1021eb80 1 --2- 172.21.15.119:0/3949702812 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a81d4376a0 0x55a81d439ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.400+0000 7f5b1021eb80 4 client.15810 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:20:47.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.400+0000 7f5b1021eb80 1 -- 172.21.15.119:0/3949702812 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a81d3e87a0 con 0x55a81d4376a0 2023-12-07T10:20:47.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.401+0000 7f5afa59b700 1 --2- 172.21.15.119:0/3949702812 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a81d4376a0 0x55a81d439ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.401+0000 7f5afa59b700 1 --2- 172.21.15.119:0/3949702812 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a81d4376a0 0x55a81d439ac0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b868c43ceabb58b server_cookie=66a94ee0d04f5ff1 in_seq=0 out_seq=0 2023-12-07T10:20:47.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.401+0000 7f5addffb700 10 client.15810 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:47.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.402+0000 7f5addffb700 1 -- 172.21.15.119:0/3949702812 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a81d3e87a0 con 0x55a81d4376a0 2023-12-07T10:20:47.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.402+0000 7f5addffb700 10 client.15810 handle_command_reply: tid=0 2023-12-07T10:20:47.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.402+0000 7f5b1021eb80 1 client.15810 shutdown 2023-12-07T10:20:47.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.402+0000 7f5b1021eb80 1 -- 172.21.15.119:0/3949702812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a81d41a460 msgr2=0x55a81d422cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:47.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.402+0000 7f5b1021eb80 1 --2- 172.21.15.119:0/3949702812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a81d41a460 0x55a81d422cc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40099c0 tx=0x7f5ae4005790 comp rx=0 tx=0).stop 2023-12-07T10:20:47.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.402+0000 7f5b1021eb80 1 -- 172.21.15.119:0/3949702812 shutdown_connections 2023-12-07T10:20:47.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.402+0000 7f5b1021eb80 1 --2- 172.21.15.119:0/3949702812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a81d41e150 0x55a81d423400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.402+0000 7f5b1021eb80 1 --2- 172.21.15.119:0/3949702812 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a81d4376a0 0x55a81d439ac0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.402+0000 7f5b1021eb80 1 --2- 172.21.15.119:0/3949702812 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a81d419b40 0x55a81d425900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.402+0000 7f5b1021eb80 1 --2- 172.21.15.119:0/3949702812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a81d41a460 0x55a81d422cc0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.402+0000 7f5b1021eb80 1 -- 172.21.15.119:0/3949702812 >> 172.21.15.119:0/3949702812 conn(0x55a81d348360 msgr2=0x55a81d41f620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:47.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.402+0000 7f5addffb700 0 client.15810 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:47.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.402+0000 7f5b1021eb80 1 -- 172.21.15.119:0/3949702812 shutdown_connections 2023-12-07T10:20:47.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.402+0000 7f5b1021eb80 1 -- 172.21.15.119:0/3949702812 wait complete. 2023-12-07T10:20:47.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.403+0000 7f5b1021eb80 20 client.15810 trim_cache size 0 max 16384 2023-12-07T10:20:47.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.416+0000 7f5b1021eb80 1 Processor -- start 2023-12-07T10:20:47.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.416+0000 7f5b1021eb80 1 -- start start 2023-12-07T10:20:47.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.416+0000 7f5b1021eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a81d419b40 0x55a81d42e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.416+0000 7f5b1021eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a81d41a460 0x55a81d42ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.416+0000 7f5b1021eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a81d41e150 0x55a81d41c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.417+0000 7f5b1021eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81d1ec920 con 0x55a81d41a460 2023-12-07T10:20:47.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.417+0000 7f5b1021eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81d425e50 con 0x55a81d419b40 2023-12-07T10:20:47.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.417+0000 7f5b1021eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81d424010 con 0x55a81d41e150 2023-12-07T10:20:47.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.417+0000 7f5af9d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a81d41a460 0x55a81d42ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.417+0000 7f5afa59b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a81d419b40 0x55a81d42e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.417+0000 7f5afa59b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a81d419b40 0x55a81d42e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:38064/0 (socket says 172.21.15.119:38064) 2023-12-07T10:20:47.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.417+0000 7f5afa59b700 1 -- 172.21.15.119:0/3798093030 learned_addr learned my addr 172.21.15.119:0/3798093030 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:47.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.417+0000 7f5afad9c700 1 --2- 172.21.15.119:0/3798093030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a81d41e150 0x55a81d41c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.417+0000 7f5afa59b700 1 -- 172.21.15.119:0/3798093030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a81d41e150 msgr2=0x55a81d41c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:47.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.417+0000 7f5afa59b700 1 --2- 172.21.15.119:0/3798093030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a81d41e150 0x55a81d41c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.417+0000 7f5afa59b700 1 -- 172.21.15.119:0/3798093030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a81d41a460 msgr2=0x55a81d42ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:47.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.417+0000 7f5afa59b700 1 --2- 172.21.15.119:0/3798093030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a81d41a460 0x55a81d42ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.417+0000 7f5afa59b700 1 -- 172.21.15.119:0/3798093030 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a81d17fc60 con 0x55a81d419b40 2023-12-07T10:20:47.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.417+0000 7f5afa59b700 1 --2- 172.21.15.119:0/3798093030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a81d419b40 0x55a81d42e6f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5af0096e20 tx=0x7f5af00cd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2231d28ce125687c server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:47.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.418+0000 7f5ae97fa700 1 -- 172.21.15.119:0/3798093030 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0096c60 con 0x55a81d419b40 2023-12-07T10:20:47.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.418+0000 7f5ae97fa700 1 -- 172.21.15.119:0/3798093030 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5af009b020 con 0x55a81d419b40 2023-12-07T10:20:47.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.418+0000 7f5ae97fa700 1 -- 172.21.15.119:0/3798093030 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af00ba2d0 con 0x55a81d419b40 2023-12-07T10:20:47.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.418+0000 7f5b1021eb80 1 -- 172.21.15.119:0/3798093030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a81d419b40 msgr2=0x55a81d42e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:47.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.418+0000 7f5b1021eb80 1 --2- 172.21.15.119:0/3798093030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a81d419b40 0x55a81d42e6f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5af0096e20 tx=0x7f5af00cd040 comp rx=0 tx=0).stop 2023-12-07T10:20:47.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.418+0000 7f5b1021eb80 1 -- 172.21.15.119:0/3798093030 shutdown_connections 2023-12-07T10:20:47.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.418+0000 7f5b1021eb80 1 --2- 172.21.15.119:0/3798093030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a81d41e150 0x55a81d41c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.418+0000 7f5b1021eb80 1 --2- 172.21.15.119:0/3798093030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a81d419b40 0x55a81d42e6f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.418+0000 7f5b1021eb80 1 --2- 172.21.15.119:0/3798093030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a81d41a460 0x55a81d42ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.418+0000 7f5b1021eb80 1 -- 172.21.15.119:0/3798093030 >> 172.21.15.119:0/3798093030 conn(0x55a81d348360 msgr2=0x55a81d43e440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:47.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.418+0000 7f5b1021eb80 1 -- 172.21.15.119:0/3798093030 shutdown_connections 2023-12-07T10:20:47.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.418+0000 7f5b1021eb80 1 -- 172.21.15.119:0/3798093030 wait complete. 2023-12-07T10:20:47.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.419+0000 7f5b1021eb80 1 -- start start 2023-12-07T10:20:47.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.419+0000 7f5b1021eb80 1 Processor -- start 2023-12-07T10:20:47.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.419+0000 7f5b1021eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a81d419b40 0x55a81d42ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.419+0000 7f5b1021eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a81d41a460 0x55a81d42c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.419+0000 7f5b1021eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a81d41e150 0x55a81d42ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.419+0000 7f5b1021eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81d42d4f0 con 0x55a81d41e150 2023-12-07T10:20:47.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.419+0000 7f5b1021eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81d48cbc0 con 0x55a81d419b40 2023-12-07T10:20:47.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.419+0000 7f5b1021eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81d48cd00 con 0x55a81d41a460 2023-12-07T10:20:47.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.419+0000 7f5b1021eb80 10 client.0 _ll_register_callbacks cb 0x55a81d30e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:20:47.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.420+0000 7f5afad9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a81d41e150 0x55a81d42ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.420+0000 7f5afad9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a81d41e150 0x55a81d42ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37788/0 (socket says 172.21.15.119:37788) 2023-12-07T10:20:47.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.420+0000 7f5afad9c700 1 -- 172.21.15.119:0/2847091233 learned_addr learned my addr 172.21.15.119:0/2847091233 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:47.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.420+0000 7f5af9d9a700 1 --2- 172.21.15.119:0/2847091233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a81d41a460 0x55a81d42c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.420+0000 7f5afa59b700 1 --2- 172.21.15.119:0/2847091233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a81d419b40 0x55a81d42ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.420+0000 7f5afad9c700 1 -- 172.21.15.119:0/2847091233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a81d41a460 msgr2=0x55a81d42c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:47.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.420+0000 7f5afad9c700 1 --2- 172.21.15.119:0/2847091233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a81d41a460 0x55a81d42c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.420+0000 7f5afad9c700 1 -- 172.21.15.119:0/2847091233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a81d419b40 msgr2=0x55a81d42ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:47.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.420+0000 7f5afad9c700 1 --2- 172.21.15.119:0/2847091233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a81d419b40 0x55a81d42ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.420+0000 7f5afad9c700 1 -- 172.21.15.119:0/2847091233 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a81d1ea630 con 0x55a81d41e150 2023-12-07T10:20:47.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.420+0000 7f5afa59b700 1 --2- 172.21.15.119:0/2847091233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a81d419b40 0x55a81d42ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:47.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.420+0000 7f5afad9c700 1 --2- 172.21.15.119:0/2847091233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a81d41e150 0x55a81d42ce10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5aec013780 tx=0x7f5aec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850769763f1e321b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:47.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.420+0000 7f5addffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:20:47.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.420+0000 7f5addffb700 1 -- 172.21.15.119:0/2847091233 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec02c070 con 0x55a81d41e150 2023-12-07T10:20:47.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.420+0000 7f5b1021eb80 1 -- 172.21.15.119:0/2847091233 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5af009a280 con 0x55a81d41e150 2023-12-07T10:20:47.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.421+0000 7f5addffb700 1 -- 172.21.15.119:0/2847091233 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f5aec004bb0 con 0x55a81d41e150 2023-12-07T10:20:47.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.421+0000 7f5addffb700 1 -- 172.21.15.119:0/2847091233 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5aec021070 con 0x55a81d41e150 2023-12-07T10:20:47.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.421+0000 7f5b1021eb80 10 client.15816 fetch_fsmap learned FSMap version 30 2023-12-07T10:20:47.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.421+0000 7f5b1021eb80 1 -- 172.21.15.119:0/2847091233 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a81d1bd250 con 0x55a81d41e150 2023-12-07T10:20:47.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.421+0000 7f5addffb700 1 -- 172.21.15.119:0/2847091233 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5af009a280 con 0x55a81d41e150 2023-12-07T10:20:47.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.421+0000 7f5addffb700 1 -- 172.21.15.119:0/2847091233 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5aec02f020 con 0x55a81d41e150 2023-12-07T10:20:47.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.421+0000 7f5b1021eb80 10 client.15816 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:20:47.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.421+0000 7f5b1021eb80 10 client.15816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:20:47.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.421+0000 7f5b1021eb80 20 client.15816 populate_metadata read hostname 'smithi119' 2023-12-07T10:20:47.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.421+0000 7f5b1021eb80 1 --2- 172.21.15.119:0/2847091233 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a81d48e520 0x55a81d490940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:47.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.421+0000 7f5b1021eb80 4 client.15816 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:20:47.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.421+0000 7f5b1021eb80 1 -- 172.21.15.119:0/2847091233 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a81d333fd0 con 0x55a81d48e520 2023-12-07T10:20:47.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.422+0000 7f5afa59b700 1 --2- 172.21.15.119:0/2847091233 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a81d48e520 0x55a81d490940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:47.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.422+0000 7f5afa59b700 1 --2- 172.21.15.119:0/2847091233 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a81d48e520 0x55a81d490940 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d506b85c844c0b3d server_cookie=d85bae3f74ae4b46 in_seq=0 out_seq=0 2023-12-07T10:20:47.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.422+0000 7f5addffb700 10 client.15816 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:47.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.422+0000 7f5addffb700 1 -- 172.21.15.119:0/2847091233 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a81d333fd0 con 0x55a81d48e520 2023-12-07T10:20:47.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.422+0000 7f5addffb700 10 client.15816 handle_command_reply: tid=0 2023-12-07T10:20:47.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.422+0000 7f5b1021eb80 1 client.15816 shutdown 2023-12-07T10:20:47.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.422+0000 7f5b1021eb80 1 -- 172.21.15.119:0/2847091233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a81d41e150 msgr2=0x55a81d42ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:47.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.422+0000 7f5b1021eb80 1 --2- 172.21.15.119:0/2847091233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a81d41e150 0x55a81d42ce10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5aec013780 tx=0x7f5aec004ad0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.422+0000 7f5b1021eb80 1 -- 172.21.15.119:0/2847091233 shutdown_connections 2023-12-07T10:20:47.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.422+0000 7f5b1021eb80 1 --2- 172.21.15.119:0/2847091233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a81d41e150 0x55a81d42ce10 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.422+0000 7f5b1021eb80 1 --2- 172.21.15.119:0/2847091233 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a81d48e520 0x55a81d490940 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.422+0000 7f5b1021eb80 1 --2- 172.21.15.119:0/2847091233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a81d419b40 0x55a81d42ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.422+0000 7f5b1021eb80 1 --2- 172.21.15.119:0/2847091233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a81d41a460 0x55a81d42c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.422+0000 7f5addffb700 0 client.15816 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:47.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.422+0000 7f5b1021eb80 1 -- 172.21.15.119:0/2847091233 >> 172.21.15.119:0/2847091233 conn(0x55a81d348360 msgr2=0x55a81d41c370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:47.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.423+0000 7f5b1021eb80 1 -- 172.21.15.119:0/2847091233 shutdown_connections 2023-12-07T10:20:47.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.423+0000 7f5b1021eb80 1 -- 172.21.15.119:0/2847091233 wait complete. 2023-12-07T10:20:47.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.423+0000 7f5b1021eb80 20 client.15816 trim_cache size 0 max 16384 2023-12-07T10:20:47.429 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:20:47.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.423+0000 7f5afc7ff700 1 -- 172.21.15.119:0/147891559 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5ae00a1d10 msgr2=0x7f5ae00a41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:47.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.423+0000 7f5afc7ff700 1 --2- 172.21.15.119:0/147891559 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5ae00a1d10 0x7f5ae00a41d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009ec0 tx=0x7f5aec004b40 comp rx=0 tx=0).stop 2023-12-07T10:20:47.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.423+0000 7f5afc7ff700 1 -- 172.21.15.119:0/147891559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5af4101560 msgr2=0x7f5af419d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:47.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.423+0000 7f5afc7ff700 1 --2- 172.21.15.119:0/147891559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5af4101560 0x7f5af419d6a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5af000e9a0 tx=0x7f5af000ecb0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.423+0000 7f5afc7ff700 1 -- 172.21.15.119:0/147891559 shutdown_connections 2023-12-07T10:20:47.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.423+0000 7f5afc7ff700 1 --2- 172.21.15.119:0/147891559 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5ae00a1d10 0x7f5ae00a41d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.423+0000 7f5afc7ff700 1 --2- 172.21.15.119:0/147891559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5af4107620 0x7f5af41977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.423+0000 7f5afc7ff700 1 --2- 172.21.15.119:0/147891559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5af4101560 0x7f5af419d6a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.423+0000 7f5afc7ff700 1 --2- 172.21.15.119:0/147891559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5af4101f20 0x7f5af419dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:47.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.423+0000 7f5afc7ff700 1 -- 172.21.15.119:0/147891559 >> 172.21.15.119:0/147891559 conn(0x7f5af40fcfd0 msgr2=0x7f5af40fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:47.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.424+0000 7f5afc7ff700 1 -- 172.21.15.119:0/147891559 shutdown_connections 2023-12-07T10:20:47.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:47.424+0000 7f5afc7ff700 1 -- 172.21.15.119:0/147891559 wait complete. 2023-12-07T10:20:48.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:47 smithi119 ceph-mon[97908]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 15 op/s 2023-12-07T10:20:48.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:47 smithi159 ceph-mon[134726]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 15 op/s 2023-12-07T10:20:48.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:47 smithi153 ceph-mon[133625]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 15 op/s 2023-12-07T10:20:48.443 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-07T10:20:48.443 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-07T10:20:48.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.654+0000 7f1c5efff700 1 -- 172.21.15.119:0/2833009290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c58101560 msgr2=0x7f1c581019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:48.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.654+0000 7f1c5efff700 1 --2- 172.21.15.119:0/2833009290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c58101560 0x7f1c581019e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1c440099c0 tx=0x7f1c44009cd0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.654+0000 7f1c5efff700 1 -- 172.21.15.119:0/2833009290 shutdown_connections 2023-12-07T10:20:48.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.654+0000 7f1c5efff700 1 --2- 172.21.15.119:0/2833009290 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c58101f20 0x7f1c5810e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.654+0000 7f1c5efff700 1 --2- 172.21.15.119:0/2833009290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c58107620 0x7f1c58107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.654+0000 7f1c5efff700 1 --2- 172.21.15.119:0/2833009290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c58101560 0x7f1c581019e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.654+0000 7f1c5efff700 1 -- 172.21.15.119:0/2833009290 >> 172.21.15.119:0/2833009290 conn(0x7f1c580fcfd0 msgr2=0x7f1c580ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:48.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.654+0000 7f1c5efff700 1 -- 172.21.15.119:0/2833009290 shutdown_connections 2023-12-07T10:20:48.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.654+0000 7f1c5efff700 1 -- 172.21.15.119:0/2833009290 wait complete. 2023-12-07T10:20:48.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.655+0000 7f1c5efff700 1 Processor -- start 2023-12-07T10:20:48.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.655+0000 7f1c5efff700 1 -- start start 2023-12-07T10:20:48.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.656+0000 7f1c5efff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c58101560 0x7f1c5819f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.656+0000 7f1c5efff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c58101f20 0x7f1c5819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.656+0000 7f1c5efff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c58107620 0x7f1c58199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.656+0000 7f1c5efff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c58199fc0 con 0x7f1c58101f20 2023-12-07T10:20:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.656+0000 7f1c5efff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5819a130 con 0x7f1c58101560 2023-12-07T10:20:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.656+0000 7f1c5efff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5819a2f0 con 0x7f1c58107620 2023-12-07T10:20:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.656+0000 7f1c57fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c58101f20 0x7f1c5819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.656+0000 7f1c57fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c58101f20 0x7f1c5819fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37802/0 (socket says 172.21.15.119:37802) 2023-12-07T10:20:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.656+0000 7f1c57fff700 1 -- 172.21.15.119:0/3808557561 learned_addr learned my addr 172.21.15.119:0/3808557561 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.656+0000 7f1c5d59c700 1 --2- 172.21.15.119:0/3808557561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c58107620 0x7f1c58199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.656+0000 7f1c5cd9b700 1 --2- 172.21.15.119:0/3808557561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c58101560 0x7f1c5819f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.656+0000 7f1c57fff700 1 -- 172.21.15.119:0/3808557561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c58107620 msgr2=0x7f1c58199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.656+0000 7f1c57fff700 1 --2- 172.21.15.119:0/3808557561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c58107620 0x7f1c58199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.656+0000 7f1c57fff700 1 -- 172.21.15.119:0/3808557561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c58101560 msgr2=0x7f1c5819f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.656+0000 7f1c57fff700 1 --2- 172.21.15.119:0/3808557561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c58101560 0x7f1c5819f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.656+0000 7f1c57fff700 1 -- 172.21.15.119:0/3808557561 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c440095f0 con 0x7f1c58101f20 2023-12-07T10:20:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.657+0000 7f1c57fff700 1 --2- 172.21.15.119:0/3808557561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c58101f20 0x7f1c5819fd70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400f690 tx=0x7f1c4400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:48.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.657+0000 7f1c55ffb700 1 -- 172.21.15.119:0/3808557561 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4401c070 con 0x7f1c58101f20 2023-12-07T10:20:48.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.657+0000 7f1c5efff700 1 -- 172.21.15.119:0/3808557561 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5819a570 con 0x7f1c58101f20 2023-12-07T10:20:48.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.657+0000 7f1c55ffb700 1 -- 172.21.15.119:0/3808557561 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c4400fcf0 con 0x7f1c58101f20 2023-12-07T10:20:48.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.657+0000 7f1c55ffb700 1 -- 172.21.15.119:0/3808557561 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c44022c60 con 0x7f1c58101f20 2023-12-07T10:20:48.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.657+0000 7f1c5efff700 1 -- 172.21.15.119:0/3808557561 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c581a6820 con 0x7f1c58101f20 2023-12-07T10:20:48.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.660+0000 7f1c55ffb700 1 -- 172.21.15.119:0/3808557561 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 24) v1 ==== 136660+0+0 (secure 0 0 0) 0x7f1c44020070 con 0x7f1c58101f20 2023-12-07T10:20:48.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.661+0000 7f1c72992b80 1 Processor -- start 2023-12-07T10:20:48.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.662+0000 7f1c72992b80 1 -- start start 2023-12-07T10:20:48.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.662+0000 7f1c55ffb700 1 --2- 172.21.15.119:0/3808557561 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f1c480a1d10 0x7f1c480a41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.662+0000 7f1c72992b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c53602ebc0 0x55c53602efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.662+0000 7f1c72992b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c53602f4e0 0x55c536032c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.662+0000 7f1c55ffb700 1 -- 172.21.15.119:0/3808557561 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f1c440cece0 con 0x7f1c58101f20 2023-12-07T10:20:48.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.662+0000 7f1c72992b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5360331d0 0x55c5360335f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.662+0000 7f1c72992b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c535fa2840 con 0x55c5360331d0 2023-12-07T10:20:48.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.662+0000 7f1c72992b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c535f89820 con 0x55c53602ebc0 2023-12-07T10:20:48.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.662+0000 7f1c72992b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c535f87140 con 0x55c53602f4e0 2023-12-07T10:20:48.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.662+0000 7f1c5d59c700 1 --2- 172.21.15.119:0/3808557561 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f1c480a1d10 0x7f1c480a41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.662+0000 7f1c5cd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c53602ebc0 0x55c53602efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.662+0000 7f1c5cd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c53602ebc0 0x55c53602efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:38102/0 (socket says 172.21.15.119:38102) 2023-12-07T10:20:48.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.662+0000 7f1c5cd9b700 1 -- 172.21.15.119:0/2347937494 learned_addr learned my addr 172.21.15.119:0/2347937494 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:48.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.662+0000 7f1c5d59c700 1 --2- 172.21.15.119:0/2347937494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5360331d0 0x55c5360335f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.662+0000 7f1c57fff700 1 --2- 172.21.15.119:0/2347937494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c53602f4e0 0x55c536032c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.663+0000 7f1c5cd9b700 1 -- 172.21.15.119:0/2347937494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c53602f4e0 msgr2=0x55c536032c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:48.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.663+0000 7f1c5cd9b700 1 --2- 172.21.15.119:0/2347937494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c53602f4e0 0x55c536032c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.663+0000 7f1c5cd9b700 1 -- 172.21.15.119:0/2347937494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5360331d0 msgr2=0x55c5360335f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:48.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.663+0000 7f1c5cd9b700 1 --2- 172.21.15.119:0/2347937494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5360331d0 0x55c5360335f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.663+0000 7f1c5cd9b700 1 -- 172.21.15.119:0/2347937494 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c535dff6b0 con 0x55c53602ebc0 2023-12-07T10:20:48.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.663+0000 7f1c5d59c700 1 --2- 172.21.15.119:0/3808557561 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f1c480a1d10 0x7f1c480a41d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009ec0 tx=0x7f1c50009920 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:48.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.663+0000 7f1c5cd9b700 1 --2- 172.21.15.119:0/2347937494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c53602ebc0 0x55c53602efa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c009d80 tx=0x7f1c4c011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce28a0fee2246aab server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:48.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.663+0000 7f1c42ffd700 1 -- 172.21.15.119:0/2347937494 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c004230 con 0x55c53602ebc0 2023-12-07T10:20:48.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.663+0000 7f1c42ffd700 1 -- 172.21.15.119:0/2347937494 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c4c004390 con 0x55c53602ebc0 2023-12-07T10:20:48.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.663+0000 7f1c42ffd700 1 -- 172.21.15.119:0/2347937494 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c019860 con 0x55c53602ebc0 2023-12-07T10:20:48.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.664+0000 7f1c72992b80 1 -- 172.21.15.119:0/2347937494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c53602ebc0 msgr2=0x55c53602efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:48.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.664+0000 7f1c72992b80 1 --2- 172.21.15.119:0/2347937494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c53602ebc0 0x55c53602efa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c009d80 tx=0x7f1c4c011720 comp rx=0 tx=0).stop 2023-12-07T10:20:48.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.664+0000 7f1c72992b80 1 -- 172.21.15.119:0/2347937494 shutdown_connections 2023-12-07T10:20:48.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.664+0000 7f1c72992b80 1 --2- 172.21.15.119:0/2347937494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5360331d0 0x55c5360335f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.664+0000 7f1c72992b80 1 --2- 172.21.15.119:0/2347937494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c53602ebc0 0x55c53602efa0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.664+0000 7f1c72992b80 1 --2- 172.21.15.119:0/2347937494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c53602f4e0 0x55c536032c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.664+0000 7f1c72992b80 1 -- 172.21.15.119:0/2347937494 >> 172.21.15.119:0/2347937494 conn(0x55c535f5d3e0 msgr2=0x55c5360291b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:48.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.664+0000 7f1c72992b80 1 -- 172.21.15.119:0/2347937494 shutdown_connections 2023-12-07T10:20:48.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.664+0000 7f1c72992b80 1 -- 172.21.15.119:0/2347937494 wait complete. 2023-12-07T10:20:48.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.665+0000 7f1c72992b80 1 -- start start 2023-12-07T10:20:48.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.665+0000 7f1c72992b80 1 Processor -- start 2023-12-07T10:20:48.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.666+0000 7f1c72992b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c53602ebc0 0x55c53603a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.666+0000 7f1c72992b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c53602f4e0 0x55c536037c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.666+0000 7f1c72992b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5360331d0 0x55c536038380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.666+0000 7f1c72992b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c535f82a30 con 0x55c53602ebc0 2023-12-07T10:20:48.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.666+0000 7f1c72992b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c535f3c5e0 con 0x55c53602f4e0 2023-12-07T10:20:48.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.666+0000 7f1c72992b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c535f3c280 con 0x55c5360331d0 2023-12-07T10:20:48.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.666+0000 7f1c72992b80 10 client.0 _ll_register_callbacks cb 0x55c535f239c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:20:48.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.666+0000 7f1c5cd9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c53602ebc0 0x55c53603a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.666+0000 7f1c5cd9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c53602ebc0 0x55c53603a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37828/0 (socket says 172.21.15.119:37828) 2023-12-07T10:20:48.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.666+0000 7f1c5cd9b700 1 -- 172.21.15.119:0/817875449 learned_addr learned my addr 172.21.15.119:0/817875449 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:48.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.666+0000 7f1c57fff700 1 --2- 172.21.15.119:0/817875449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c53602f4e0 0x55c536037c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.666+0000 7f1c5d59c700 1 --2- 172.21.15.119:0/817875449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5360331d0 0x55c536038380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.666+0000 7f1c5cd9b700 1 -- 172.21.15.119:0/817875449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5360331d0 msgr2=0x55c536038380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:48.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.666+0000 7f1c5cd9b700 1 --2- 172.21.15.119:0/817875449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5360331d0 0x55c536038380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.666+0000 7f1c5cd9b700 1 -- 172.21.15.119:0/817875449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c53602f4e0 msgr2=0x55c536037c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:48.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.666+0000 7f1c5cd9b700 1 --2- 172.21.15.119:0/817875449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c53602f4e0 0x55c536037c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.666+0000 7f1c5cd9b700 1 -- 172.21.15.119:0/817875449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4c0099e0 con 0x55c53602ebc0 2023-12-07T10:20:48.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.667+0000 7f1c5cd9b700 1 --2- 172.21.15.119:0/817875449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c53602ebc0 0x55c53603a950 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c000c00 tx=0x7f1c4c003930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268116eeb00d573c server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:48.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.667+0000 7f1c37fff700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:20:48.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.667+0000 7f1c37fff700 1 -- 172.21.15.119:0/817875449 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c005450 con 0x55c53602ebc0 2023-12-07T10:20:48.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.667+0000 7f1c72992b80 1 -- 172.21.15.119:0/817875449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c536021b80 con 0x55c53602ebc0 2023-12-07T10:20:48.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.667+0000 7f1c37fff700 1 -- 172.21.15.119:0/817875449 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f1c4c004500 con 0x55c53602ebc0 2023-12-07T10:20:48.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.667+0000 7f1c37fff700 1 -- 172.21.15.119:0/817875449 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c4c009ed0 con 0x55c53602ebc0 2023-12-07T10:20:48.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.667+0000 7f1c72992b80 10 client.15828 fetch_fsmap learned FSMap version 30 2023-12-07T10:20:48.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.667+0000 7f1c72992b80 1 -- 172.21.15.119:0/817875449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c535dabe60 con 0x55c53602ebc0 2023-12-07T10:20:48.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.668+0000 7f1c37fff700 1 -- 172.21.15.119:0/817875449 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c536021b80 con 0x55c53602ebc0 2023-12-07T10:20:48.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.668+0000 7f1c37fff700 1 -- 172.21.15.119:0/817875449 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1c4c021040 con 0x55c53602ebc0 2023-12-07T10:20:48.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.668+0000 7f1c72992b80 10 client.15828 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:20:48.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.668+0000 7f1c72992b80 10 client.15828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:20:48.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.668+0000 7f1c72992b80 20 client.15828 populate_metadata read hostname 'smithi119' 2023-12-07T10:20:48.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.668+0000 7f1c72992b80 1 --2- 172.21.15.119:0/817875449 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c536048350 0x55c53604a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.668+0000 7f1c72992b80 4 client.15828 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:20:48.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.668+0000 7f1c72992b80 1 -- 172.21.15.119:0/817875449 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c535ffd820 con 0x55c536048350 2023-12-07T10:20:48.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.668+0000 7f1c57fff700 1 --2- 172.21.15.119:0/817875449 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c536048350 0x55c53604a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.669+0000 7f1c57fff700 1 --2- 172.21.15.119:0/817875449 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c536048350 0x55c53604a770 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46b7ec100ef7bbc3 server_cookie=253f78013ec1dccb in_seq=0 out_seq=0 2023-12-07T10:20:48.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.669+0000 7f1c37fff700 10 client.15828 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:48.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.669+0000 7f1c37fff700 1 -- 172.21.15.119:0/817875449 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c535ffd820 con 0x55c536048350 2023-12-07T10:20:48.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.669+0000 7f1c37fff700 10 client.15828 handle_command_reply: tid=0 2023-12-07T10:20:48.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.669+0000 7f1c72992b80 1 client.15828 shutdown 2023-12-07T10:20:48.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.670+0000 7f1c72992b80 1 -- 172.21.15.119:0/817875449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c53602ebc0 msgr2=0x55c53603a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:48.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.670+0000 7f1c72992b80 1 --2- 172.21.15.119:0/817875449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c53602ebc0 0x55c53603a950 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c000c00 tx=0x7f1c4c003930 comp rx=0 tx=0).stop 2023-12-07T10:20:48.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.670+0000 7f1c72992b80 1 -- 172.21.15.119:0/817875449 shutdown_connections 2023-12-07T10:20:48.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.670+0000 7f1c72992b80 1 --2- 172.21.15.119:0/817875449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5360331d0 0x55c536038380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.670+0000 7f1c72992b80 1 --2- 172.21.15.119:0/817875449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c53602ebc0 0x55c53603a950 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.670+0000 7f1c72992b80 1 --2- 172.21.15.119:0/817875449 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c536048350 0x55c53604a770 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.670+0000 7f1c72992b80 1 --2- 172.21.15.119:0/817875449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c53602f4e0 0x55c536037c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.670+0000 7f1c72992b80 1 -- 172.21.15.119:0/817875449 >> 172.21.15.119:0/817875449 conn(0x55c535f5d3e0 msgr2=0x55c536029170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:48.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.670+0000 7f1c37fff700 0 client.15828 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:48.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.670+0000 7f1c72992b80 1 -- 172.21.15.119:0/817875449 shutdown_connections 2023-12-07T10:20:48.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.670+0000 7f1c72992b80 1 -- 172.21.15.119:0/817875449 wait complete. 2023-12-07T10:20:48.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.670+0000 7f1c72992b80 20 client.15828 trim_cache size 0 max 16384 2023-12-07T10:20:48.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.687+0000 7f1c72992b80 1 Processor -- start 2023-12-07T10:20:48.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.687+0000 7f1c72992b80 1 -- start start 2023-12-07T10:20:48.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.688+0000 7f1c72992b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c53602ebc0 0x55c53603f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.688+0000 7f1c72992b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c53602f4e0 0x55c53603f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.688+0000 7f1c72992b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5360331d0 0x55c536036510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.688+0000 7f1c72992b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c535e019a0 con 0x55c53602f4e0 2023-12-07T10:20:48.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.688+0000 7f1c72992b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c536038bf0 con 0x55c5360331d0 2023-12-07T10:20:48.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.688+0000 7f1c72992b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c536029a70 con 0x55c53602ebc0 2023-12-07T10:20:48.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.688+0000 7f1c57fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c53602f4e0 0x55c53603f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.688+0000 7f1c57fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c53602f4e0 0x55c53603f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37844/0 (socket says 172.21.15.119:37844) 2023-12-07T10:20:48.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.688+0000 7f1c57fff700 1 -- 172.21.15.119:0/2259638520 learned_addr learned my addr 172.21.15.119:0/2259638520 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:48.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.688+0000 7f1c5cd9b700 1 --2- 172.21.15.119:0/2259638520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c53602ebc0 0x55c53603f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.688+0000 7f1c5d59c700 1 --2- 172.21.15.119:0/2259638520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5360331d0 0x55c536036510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.688+0000 7f1c57fff700 1 -- 172.21.15.119:0/2259638520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c53602ebc0 msgr2=0x55c53603f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:48.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.688+0000 7f1c57fff700 1 --2- 172.21.15.119:0/2259638520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c53602ebc0 0x55c53603f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.688+0000 7f1c57fff700 1 -- 172.21.15.119:0/2259638520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5360331d0 msgr2=0x55c536036510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:48.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.688+0000 7f1c57fff700 1 --2- 172.21.15.119:0/2259638520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5360331d0 0x55c536036510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.688+0000 7f1c57fff700 1 -- 172.21.15.119:0/2259638520 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c535d94ce0 con 0x55c53602f4e0 2023-12-07T10:20:48.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.688+0000 7f1c57fff700 1 --2- 172.21.15.119:0/2259638520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c53602f4e0 0x55c53603f910 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1c44029000 tx=0x7f1c44009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a3e4e78d83ea52 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:48.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.689+0000 7f1c437fe700 1 -- 172.21.15.119:0/2259638520 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c44062070 con 0x55c53602f4e0 2023-12-07T10:20:48.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.689+0000 7f1c437fe700 1 -- 172.21.15.119:0/2259638520 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c4409bcf0 con 0x55c53602f4e0 2023-12-07T10:20:48.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.689+0000 7f1c437fe700 1 -- 172.21.15.119:0/2259638520 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c44066a90 con 0x55c53602f4e0 2023-12-07T10:20:48.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.689+0000 7f1c72992b80 1 -- 172.21.15.119:0/2259638520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c53602f4e0 msgr2=0x55c53603f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:48.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.689+0000 7f1c72992b80 1 --2- 172.21.15.119:0/2259638520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c53602f4e0 0x55c53603f910 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1c44029000 tx=0x7f1c44009310 comp rx=0 tx=0).stop 2023-12-07T10:20:48.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.689+0000 7f1c72992b80 1 -- 172.21.15.119:0/2259638520 shutdown_connections 2023-12-07T10:20:48.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.689+0000 7f1c72992b80 1 --2- 172.21.15.119:0/2259638520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5360331d0 0x55c536036510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.689+0000 7f1c72992b80 1 --2- 172.21.15.119:0/2259638520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c53602ebc0 0x55c53603f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.689+0000 7f1c72992b80 1 --2- 172.21.15.119:0/2259638520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c53602f4e0 0x55c53603f910 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.689+0000 7f1c72992b80 1 -- 172.21.15.119:0/2259638520 >> 172.21.15.119:0/2259638520 conn(0x55c535f5d3e0 msgr2=0x55c5360322c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:48.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.689+0000 7f1c72992b80 1 -- 172.21.15.119:0/2259638520 shutdown_connections 2023-12-07T10:20:48.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.689+0000 7f1c72992b80 1 -- 172.21.15.119:0/2259638520 wait complete. 2023-12-07T10:20:48.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.690+0000 7f1c72992b80 1 -- start start 2023-12-07T10:20:48.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.690+0000 7f1c72992b80 1 Processor -- start 2023-12-07T10:20:48.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.690+0000 7f1c72992b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c53602ebc0 0x55c53603e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.690+0000 7f1c72992b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c53602f4e0 0x55c53603e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.690+0000 7f1c72992b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5360331d0 0x55c53609e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.690+0000 7f1c72992b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c53603ee90 con 0x55c53602ebc0 2023-12-07T10:20:48.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.690+0000 7f1c72992b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c53603efd0 con 0x55c53602f4e0 2023-12-07T10:20:48.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.690+0000 7f1c72992b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c53609ede0 con 0x55c5360331d0 2023-12-07T10:20:48.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.690+0000 7f1c72992b80 10 client.0 _ll_register_callbacks cb 0x55c535f239c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:20:48.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.691+0000 7f1c5cd9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c53602ebc0 0x55c53603e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.691+0000 7f1c5cd9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c53602ebc0 0x55c53603e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37856/0 (socket says 172.21.15.119:37856) 2023-12-07T10:20:48.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.691+0000 7f1c5cd9b700 1 -- 172.21.15.119:0/3467817160 learned_addr learned my addr 172.21.15.119:0/3467817160 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:48.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.691+0000 7f1c57fff700 1 --2- 172.21.15.119:0/3467817160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c53602f4e0 0x55c53603e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.691+0000 7f1c5d59c700 1 --2- 172.21.15.119:0/3467817160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5360331d0 0x55c53609e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.691+0000 7f1c5cd9b700 1 -- 172.21.15.119:0/3467817160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5360331d0 msgr2=0x55c53609e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:48.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.691+0000 7f1c5cd9b700 1 --2- 172.21.15.119:0/3467817160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5360331d0 0x55c53609e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.691+0000 7f1c5cd9b700 1 -- 172.21.15.119:0/3467817160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c53602f4e0 msgr2=0x55c53603e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:48.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.691+0000 7f1c5cd9b700 1 --2- 172.21.15.119:0/3467817160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c53602f4e0 0x55c53603e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.691+0000 7f1c5cd9b700 1 -- 172.21.15.119:0/3467817160 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c581a6820 con 0x55c53602ebc0 2023-12-07T10:20:48.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.691+0000 7f1c57fff700 1 --2- 172.21.15.119:0/3467817160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c53602f4e0 0x55c53603e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:48.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.691+0000 7f1c5cd9b700 1 --2- 172.21.15.119:0/3467817160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c53602ebc0 0x55c53603e1a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c009cd0 tx=0x7f1c4c019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7d5bdd98408c9e server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:48.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.691+0000 7f1c37fff700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:20:48.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.692+0000 7f1c37fff700 1 -- 172.21.15.119:0/3467817160 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c009ed0 con 0x55c53602ebc0 2023-12-07T10:20:48.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.692+0000 7f1c72992b80 1 -- 172.21.15.119:0/3467817160 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c4c004390 con 0x55c53602ebc0 2023-12-07T10:20:48.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.692+0000 7f1c37fff700 1 -- 172.21.15.119:0/3467817160 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f1c4c005250 con 0x55c53602ebc0 2023-12-07T10:20:48.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.692+0000 7f1c37fff700 1 -- 172.21.15.119:0/3467817160 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c4c019700 con 0x55c53602ebc0 2023-12-07T10:20:48.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.692+0000 7f1c72992b80 10 client.15834 fetch_fsmap learned FSMap version 30 2023-12-07T10:20:48.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.692+0000 7f1c72992b80 1 -- 172.21.15.119:0/3467817160 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c535dd22d0 con 0x55c53602ebc0 2023-12-07T10:20:48.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.692+0000 7f1c37fff700 1 -- 172.21.15.119:0/3467817160 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c4c004390 con 0x55c53602ebc0 2023-12-07T10:20:48.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.692+0000 7f1c37fff700 1 -- 172.21.15.119:0/3467817160 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1c4c011790 con 0x55c53602ebc0 2023-12-07T10:20:48.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.692+0000 7f1c72992b80 10 client.15834 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:20:48.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.692+0000 7f1c72992b80 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:20:48.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.692+0000 7f1c72992b80 20 client.15834 populate_metadata read hostname 'smithi119' 2023-12-07T10:20:48.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.692+0000 7f1c72992b80 1 --2- 172.21.15.119:0/3467817160 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c5360a0260 0x55c5360a2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.692+0000 7f1c72992b80 4 client.15834 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-07T10:20:48.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.692+0000 7f1c72992b80 1 -- 172.21.15.119:0/3467817160 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c535f49050 con 0x55c5360a0260 2023-12-07T10:20:48.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.693+0000 7f1c57fff700 1 --2- 172.21.15.119:0/3467817160 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c5360a0260 0x55c5360a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.693+0000 7f1c37fff700 10 client.15834 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:48.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.693+0000 7f1c57fff700 1 --2- 172.21.15.119:0/3467817160 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c5360a0260 0x55c5360a2680 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c76a9d373347f3f0 server_cookie=dd011eb6f50df915 in_seq=0 out_seq=0 2023-12-07T10:20:48.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.694+0000 7f1c37fff700 1 -- 172.21.15.119:0/3467817160 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c535f49050 con 0x55c5360a0260 2023-12-07T10:20:48.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.694+0000 7f1c37fff700 10 client.15834 handle_command_reply: tid=0 2023-12-07T10:20:48.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.694+0000 7f1c72992b80 1 client.15834 shutdown 2023-12-07T10:20:48.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.695+0000 7f1c72992b80 1 -- 172.21.15.119:0/3467817160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c53602ebc0 msgr2=0x55c53603e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:48.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.695+0000 7f1c72992b80 1 --2- 172.21.15.119:0/3467817160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c53602ebc0 0x55c53603e1a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c009cd0 tx=0x7f1c4c019690 comp rx=0 tx=0).stop 2023-12-07T10:20:48.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.695+0000 7f1c72992b80 1 -- 172.21.15.119:0/3467817160 shutdown_connections 2023-12-07T10:20:48.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.695+0000 7f1c72992b80 1 --2- 172.21.15.119:0/3467817160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5360331d0 0x55c53609e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.695+0000 7f1c72992b80 1 --2- 172.21.15.119:0/3467817160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c53602ebc0 0x55c53603e1a0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.695+0000 7f1c72992b80 1 --2- 172.21.15.119:0/3467817160 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c5360a0260 0x55c5360a2680 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.695+0000 7f1c72992b80 1 --2- 172.21.15.119:0/3467817160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c53602f4e0 0x55c53603e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.695+0000 7f1c72992b80 1 -- 172.21.15.119:0/3467817160 >> 172.21.15.119:0/3467817160 conn(0x55c535f5d3e0 msgr2=0x55c5360364d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:48.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.695+0000 7f1c37fff700 0 client.15834 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:48.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.695+0000 7f1c72992b80 1 -- 172.21.15.119:0/3467817160 shutdown_connections 2023-12-07T10:20:48.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.695+0000 7f1c72992b80 1 -- 172.21.15.119:0/3467817160 wait complete. 2023-12-07T10:20:48.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.695+0000 7f1c72992b80 20 client.15834 trim_cache size 0 max 16384 2023-12-07T10:20:48.701 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:20:48.701 INFO:teuthology.orchestra.run.smithi119.stdout: "return_code": 0, 2023-12-07T10:20:48.701 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_tag": "0f1c1e61-0615-4d7b-90c2-c8e09f02e124", 2023-12-07T10:20:48.701 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": "asynchronous" 2023-12-07T10:20:48.701 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:20:48.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.696+0000 7f1c5efff700 1 -- 172.21.15.119:0/3808557561 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f1c480a1d10 msgr2=0x7f1c480a41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:48.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.696+0000 7f1c5efff700 1 --2- 172.21.15.119:0/3808557561 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f1c480a1d10 0x7f1c480a41d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009ec0 tx=0x7f1c50009920 comp rx=0 tx=0).stop 2023-12-07T10:20:48.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.696+0000 7f1c5efff700 1 -- 172.21.15.119:0/3808557561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c58101f20 msgr2=0x7f1c5819fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:48.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.696+0000 7f1c5efff700 1 --2- 172.21.15.119:0/3808557561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c58101f20 0x7f1c5819fd70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400f690 tx=0x7f1c4400bfd0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.696+0000 7f1c5efff700 1 -- 172.21.15.119:0/3808557561 shutdown_connections 2023-12-07T10:20:48.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.696+0000 7f1c5efff700 1 --2- 172.21.15.119:0/3808557561 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f1c480a1d10 0x7f1c480a41d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.696+0000 7f1c5efff700 1 --2- 172.21.15.119:0/3808557561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c58107620 0x7f1c58199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.696+0000 7f1c5efff700 1 --2- 172.21.15.119:0/3808557561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c58101560 0x7f1c5819f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.696+0000 7f1c5efff700 1 --2- 172.21.15.119:0/3808557561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c58101f20 0x7f1c5819fd70 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.696+0000 7f1c5efff700 1 -- 172.21.15.119:0/3808557561 >> 172.21.15.119:0/3808557561 conn(0x7f1c580fcfd0 msgr2=0x7f1c5810d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:48.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.696+0000 7f1c5efff700 1 -- 172.21.15.119:0/3808557561 shutdown_connections 2023-12-07T10:20:48.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.696+0000 7f1c5efff700 1 -- 172.21.15.119:0/3808557561 wait complete. 2023-12-07T10:20:48.716 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-07T10:20:48.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.916+0000 7fe7277ff700 1 -- 172.21.15.119:0/2130734615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7201074e0 msgr2=0x7fe7201078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:48.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.916+0000 7fe7277ff700 1 --2- 172.21.15.119:0/2130734615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7201074e0 0x7fe7201078c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009990 tx=0x7fe71c009ca0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.917+0000 7fe7277ff700 1 -- 172.21.15.119:0/2130734615 shutdown_connections 2023-12-07T10:20:48.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.917+0000 7fe7277ff700 1 --2- 172.21.15.119:0/2130734615 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe720101e00 0x7fe72010e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.917+0000 7fe7277ff700 1 --2- 172.21.15.119:0/2130734615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7201074e0 0x7fe7201078c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.917+0000 7fe7277ff700 1 --2- 172.21.15.119:0/2130734615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe720101440 0x7fe7201018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.917+0000 7fe7277ff700 1 -- 172.21.15.119:0/2130734615 >> 172.21.15.119:0/2130734615 conn(0x7fe7200fcf10 msgr2=0x7fe7200ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:48.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.917+0000 7fe7277ff700 1 -- 172.21.15.119:0/2130734615 shutdown_connections 2023-12-07T10:20:48.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.917+0000 7fe7277ff700 1 -- 172.21.15.119:0/2130734615 wait complete. 2023-12-07T10:20:48.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.918+0000 7fe7277ff700 1 Processor -- start 2023-12-07T10:20:48.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.918+0000 7fe7277ff700 1 -- start start 2023-12-07T10:20:48.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.918+0000 7fe7277ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe720101440 0x7fe72019d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.918+0000 7fe7277ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe720101e00 0x7fe72019db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.918+0000 7fe7277ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7201074e0 0x7fe720197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.918+0000 7fe7277ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe720197e60 con 0x7fe720101e00 2023-12-07T10:20:48.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.918+0000 7fe7277ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe720197fd0 con 0x7fe720101440 2023-12-07T10:20:48.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.918+0000 7fe7277ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe720198190 con 0x7fe7201074e0 2023-12-07T10:20:48.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.918+0000 7fe72559b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe720101440 0x7fe72019d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.918+0000 7fe72559b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe720101440 0x7fe72019d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:38158/0 (socket says 172.21.15.119:38158) 2023-12-07T10:20:48.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.918+0000 7fe72559b700 1 -- 172.21.15.119:0/4177081713 learned_addr learned my addr 172.21.15.119:0/4177081713 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:48.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.918+0000 7fe724d9a700 1 --2- 172.21.15.119:0/4177081713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe720101e00 0x7fe72019db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.919+0000 7fe725d9c700 1 --2- 172.21.15.119:0/4177081713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7201074e0 0x7fe720197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.919+0000 7fe724d9a700 1 -- 172.21.15.119:0/4177081713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7201074e0 msgr2=0x7fe720197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:48.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.919+0000 7fe724d9a700 1 --2- 172.21.15.119:0/4177081713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7201074e0 0x7fe720197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.919+0000 7fe724d9a700 1 -- 172.21.15.119:0/4177081713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe720101440 msgr2=0x7fe72019d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:48.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.919+0000 7fe724d9a700 1 --2- 172.21.15.119:0/4177081713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe720101440 0x7fe72019d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.919+0000 7fe724d9a700 1 -- 172.21.15.119:0/4177081713 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71c0095f0 con 0x7fe720101e00 2023-12-07T10:20:48.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.919+0000 7fe725d9c700 1 --2- 172.21.15.119:0/4177081713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7201074e0 0x7fe720197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:20:48.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.919+0000 7fe724d9a700 1 --2- 172.21.15.119:0/4177081713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe720101e00 0x7fe72019db30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe71000e9f0 tx=0x7fe71000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:48.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.919+0000 7fe7167fc700 1 -- 172.21.15.119:0/4177081713 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71000eee0 con 0x7fe720101e00 2023-12-07T10:20:48.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.919+0000 7fe7167fc700 1 -- 172.21.15.119:0/4177081713 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe71000cbd0 con 0x7fe720101e00 2023-12-07T10:20:48.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.919+0000 7fe7167fc700 1 -- 172.21.15.119:0/4177081713 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe710018680 con 0x7fe720101e00 2023-12-07T10:20:48.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.919+0000 7fe7277ff700 1 -- 172.21.15.119:0/4177081713 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe720198470 con 0x7fe720101e00 2023-12-07T10:20:48.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.919+0000 7fe7277ff700 1 -- 172.21.15.119:0/4177081713 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7201a4570 con 0x7fe720101e00 2023-12-07T10:20:48.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.923+0000 7fe7167fc700 1 -- 172.21.15.119:0/4177081713 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 24) v1 ==== 136660+0+0 (secure 0 0 0) 0x7fe71000cd40 con 0x7fe720101e00 2023-12-07T10:20:48.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.924+0000 7fe73b0cab80 1 Processor -- start 2023-12-07T10:20:48.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.924+0000 7fe7167fc700 1 --2- 172.21.15.119:0/4177081713 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe70c0a1d80 0x7fe70c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.924+0000 7fe7167fc700 1 -- 172.21.15.119:0/4177081713 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fe710014070 con 0x7fe720101e00 2023-12-07T10:20:48.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.924+0000 7fe73b0cab80 1 -- start start 2023-12-07T10:20:48.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.924+0000 7fe725d9c700 1 --2- 172.21.15.119:0/4177081713 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe70c0a1d80 0x7fe70c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.924+0000 7fe73b0cab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 0x56511ead8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.924+0000 7fe73b0cab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511ead9460 0x56511eadcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.924+0000 7fe73b0cab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56511eadd150 0x56511eadd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.925+0000 7fe73b0cab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56511ea4c7c0 con 0x56511ead8b40 2023-12-07T10:20:48.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.925+0000 7fe73b0cab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56511ea337a0 con 0x56511eadd150 2023-12-07T10:20:48.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.925+0000 7fe73b0cab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56511ea310c0 con 0x56511ead9460 2023-12-07T10:20:48.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.925+0000 7fe72559b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 0x56511ead8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.925+0000 7fe72559b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 0x56511ead8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37888/0 (socket says 172.21.15.119:37888) 2023-12-07T10:20:48.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.925+0000 7fe72559b700 1 -- 172.21.15.119:0/2451205169 learned_addr learned my addr 172.21.15.119:0/2451205169 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:48.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.925+0000 7fe724d9a700 1 --2- 172.21.15.119:0/2451205169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511ead9460 0x56511eadcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.925+0000 7fe72559b700 1 -- 172.21.15.119:0/2451205169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511ead9460 msgr2=0x56511eadcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:48.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.925+0000 7fe72559b700 1 --2- 172.21.15.119:0/2451205169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511ead9460 0x56511eadcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.925+0000 7fe72559b700 1 -- 172.21.15.119:0/2451205169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56511eadd150 msgr2=0x56511eadd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:48.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.925+0000 7fe72559b700 1 --2- 172.21.15.119:0/2451205169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56511eadd150 0x56511eadd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.925+0000 7fe72559b700 1 -- 172.21.15.119:0/2451205169 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56511e8a9630 con 0x56511ead8b40 2023-12-07T10:20:48.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.925+0000 7fe725d9c700 1 --2- 172.21.15.119:0/4177081713 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe70c0a1d80 0x7fe70c0a4240 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe72019e280 tx=0x7fe71800b040 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:48.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.925+0000 7fe72559b700 1 --2- 172.21.15.119:0/2451205169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 0x56511ead8f20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe71c00b640 tx=0x7fe71c00b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08ac2d3a9d1e1a0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:48.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.926+0000 7fe70b7fe700 1 -- 172.21.15.119:0/2451205169 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c01d070 con 0x56511ead8b40 2023-12-07T10:20:48.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.926+0000 7fe70b7fe700 1 -- 172.21.15.119:0/2451205169 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe71c01e040 con 0x56511ead8b40 2023-12-07T10:20:48.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.926+0000 7fe70b7fe700 1 -- 172.21.15.119:0/2451205169 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c004920 con 0x56511ead8b40 2023-12-07T10:20:48.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.926+0000 7fe73b0cab80 1 -- 172.21.15.119:0/2451205169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 msgr2=0x56511ead8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:48.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.926+0000 7fe73b0cab80 1 --2- 172.21.15.119:0/2451205169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 0x56511ead8f20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe71c00b640 tx=0x7fe71c00b670 comp rx=0 tx=0).stop 2023-12-07T10:20:48.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.926+0000 7fe73b0cab80 1 -- 172.21.15.119:0/2451205169 shutdown_connections 2023-12-07T10:20:48.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.926+0000 7fe73b0cab80 1 --2- 172.21.15.119:0/2451205169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56511eadd150 0x56511eadd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.926+0000 7fe73b0cab80 1 --2- 172.21.15.119:0/2451205169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 0x56511ead8f20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.926+0000 7fe73b0cab80 1 --2- 172.21.15.119:0/2451205169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511ead9460 0x56511eadcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.926+0000 7fe73b0cab80 1 -- 172.21.15.119:0/2451205169 >> 172.21.15.119:0/2451205169 conn(0x56511ea07360 msgr2=0x56511ead3130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:48.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.927+0000 7fe73b0cab80 1 -- 172.21.15.119:0/2451205169 shutdown_connections 2023-12-07T10:20:48.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.927+0000 7fe73b0cab80 1 -- 172.21.15.119:0/2451205169 wait complete. 2023-12-07T10:20:48.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.927+0000 7fe73b0cab80 1 -- start start 2023-12-07T10:20:48.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.927+0000 7fe73b0cab80 1 Processor -- start 2023-12-07T10:20:48.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.928+0000 7fe73b0cab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 0x56511eaec170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.928+0000 7fe73b0cab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56511ead9460 0x56511eaec6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.928+0000 7fe73b0cab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511eadd150 0x56511eae6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.928+0000 7fe73b0cab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56511ea2c9b0 con 0x56511ead8b40 2023-12-07T10:20:48.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.928+0000 7fe73b0cab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56511e9e6560 con 0x56511ead9460 2023-12-07T10:20:48.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.928+0000 7fe73b0cab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56511e9e6200 con 0x56511eadd150 2023-12-07T10:20:48.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.928+0000 7fe73b0cab80 10 client.0 _ll_register_callbacks cb 0x56511e9cd940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:20:48.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.928+0000 7fe72559b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 0x56511eaec170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.928+0000 7fe72559b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 0x56511eaec170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37894/0 (socket says 172.21.15.119:37894) 2023-12-07T10:20:48.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.928+0000 7fe72559b700 1 -- 172.21.15.119:0/2476102129 learned_addr learned my addr 172.21.15.119:0/2476102129 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:48.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.928+0000 7fe725d9c700 1 --2- 172.21.15.119:0/2476102129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511eadd150 0x56511eae6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.928+0000 7fe724d9a700 1 --2- 172.21.15.119:0/2476102129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56511ead9460 0x56511eaec6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.928+0000 7fe72559b700 1 -- 172.21.15.119:0/2476102129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511eadd150 msgr2=0x56511eae6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:48.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.928+0000 7fe72559b700 1 --2- 172.21.15.119:0/2476102129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511eadd150 0x56511eae6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.928+0000 7fe72559b700 1 -- 172.21.15.119:0/2476102129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56511ead9460 msgr2=0x56511eaec6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:48.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.928+0000 7fe72559b700 1 --2- 172.21.15.119:0/2476102129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56511ead9460 0x56511eaec6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.928+0000 7fe72559b700 1 -- 172.21.15.119:0/2476102129 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe72010ec10 con 0x56511ead8b40 2023-12-07T10:20:48.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.929+0000 7fe72559b700 1 --2- 172.21.15.119:0/2476102129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 0x56511eaec170 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe71c004540 tx=0x7fe71c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e7317e4f29c933 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:48.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.929+0000 7fe708ff9700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:20:48.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.929+0000 7fe708ff9700 1 -- 172.21.15.119:0/2476102129 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c032070 con 0x56511ead8b40 2023-12-07T10:20:48.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.929+0000 7fe708ff9700 1 -- 172.21.15.119:0/2476102129 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fe71c00ba20 con 0x56511ead8b40 2023-12-07T10:20:48.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.929+0000 7fe73b0cab80 1 -- 172.21.15.119:0/2476102129 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56511eacbb00 con 0x56511ead8b40 2023-12-07T10:20:48.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.929+0000 7fe708ff9700 1 -- 172.21.15.119:0/2476102129 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe71c01d070 con 0x56511ead8b40 2023-12-07T10:20:48.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.929+0000 7fe73b0cab80 10 client.15846 fetch_fsmap learned FSMap version 30 2023-12-07T10:20:48.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.929+0000 7fe73b0cab80 1 -- 172.21.15.119:0/2476102129 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56511e855de0 con 0x56511ead8b40 2023-12-07T10:20:48.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.929+0000 7fe708ff9700 1 -- 172.21.15.119:0/2476102129 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56511eacbb00 con 0x56511ead8b40 2023-12-07T10:20:48.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.930+0000 7fe708ff9700 1 -- 172.21.15.119:0/2476102129 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe71c035020 con 0x56511ead8b40 2023-12-07T10:20:48.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.930+0000 7fe73b0cab80 10 client.15846 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:20:48.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.930+0000 7fe73b0cab80 10 client.15846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:20:48.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.930+0000 7fe73b0cab80 20 client.15846 populate_metadata read hostname 'smithi119' 2023-12-07T10:20:48.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.930+0000 7fe73b0cab80 1 --2- 172.21.15.119:0/2476102129 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56511eaf6280 0x56511eaf8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.930+0000 7fe73b0cab80 4 client.15846 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:20:48.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.930+0000 7fe73b0cab80 1 -- 172.21.15.119:0/2476102129 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56511eaa77a0 con 0x56511eaf6280 2023-12-07T10:20:48.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.930+0000 7fe724d9a700 1 --2- 172.21.15.119:0/2476102129 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56511eaf6280 0x56511eaf8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.931+0000 7fe708ff9700 10 client.15846 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:48.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.931+0000 7fe724d9a700 1 --2- 172.21.15.119:0/2476102129 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56511eaf6280 0x56511eaf8670 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1768f11177e64d9 server_cookie=b5fb566eb7e490f2 in_seq=0 out_seq=0 2023-12-07T10:20:48.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.931+0000 7fe708ff9700 1 -- 172.21.15.119:0/2476102129 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56511eaa77a0 con 0x56511eaf6280 2023-12-07T10:20:48.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.931+0000 7fe708ff9700 10 client.15846 handle_command_reply: tid=0 2023-12-07T10:20:48.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.931+0000 7fe73b0cab80 1 client.15846 shutdown 2023-12-07T10:20:48.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.931+0000 7fe73b0cab80 1 -- 172.21.15.119:0/2476102129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 msgr2=0x56511eaec170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:48.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.931+0000 7fe73b0cab80 1 --2- 172.21.15.119:0/2476102129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 0x56511eaec170 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe71c004540 tx=0x7fe71c00be00 comp rx=0 tx=0).stop 2023-12-07T10:20:48.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.932+0000 7fe73b0cab80 1 -- 172.21.15.119:0/2476102129 shutdown_connections 2023-12-07T10:20:48.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.932+0000 7fe73b0cab80 1 --2- 172.21.15.119:0/2476102129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511eadd150 0x56511eae6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.932+0000 7fe73b0cab80 1 --2- 172.21.15.119:0/2476102129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 0x56511eaec170 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.932+0000 7fe73b0cab80 1 --2- 172.21.15.119:0/2476102129 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56511eaf6280 0x56511eaf8670 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.932+0000 7fe73b0cab80 1 --2- 172.21.15.119:0/2476102129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56511ead9460 0x56511eaec6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.932+0000 7fe708ff9700 0 client.15846 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:48.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.932+0000 7fe73b0cab80 1 -- 172.21.15.119:0/2476102129 >> 172.21.15.119:0/2476102129 conn(0x56511ea07360 msgr2=0x56511eacdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:48.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.932+0000 7fe73b0cab80 1 -- 172.21.15.119:0/2476102129 shutdown_connections 2023-12-07T10:20:48.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.932+0000 7fe73b0cab80 1 -- 172.21.15.119:0/2476102129 wait complete. 2023-12-07T10:20:48.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.932+0000 7fe73b0cab80 20 client.15846 trim_cache size 0 max 16384 2023-12-07T10:20:48.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.952+0000 7fe73b0cab80 1 Processor -- start 2023-12-07T10:20:48.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.952+0000 7fe73b0cab80 1 -- start start 2023-12-07T10:20:48.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.952+0000 7fe73b0cab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 0x56511ead3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.952+0000 7fe73b0cab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56511ead9460 0x56511ead3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.952+0000 7fe73b0cab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511eadd150 0x56511eae5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.952+0000 7fe73b0cab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56511e8ab920 con 0x56511ead8b40 2023-12-07T10:20:48.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.952+0000 7fe73b0cab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56511ead4020 con 0x56511ead9460 2023-12-07T10:20:48.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.952+0000 7fe73b0cab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56511eae6f70 con 0x56511eadd150 2023-12-07T10:20:48.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.952+0000 7fe72559b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 0x56511ead3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.952+0000 7fe72559b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 0x56511ead3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37908/0 (socket says 172.21.15.119:37908) 2023-12-07T10:20:48.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.952+0000 7fe72559b700 1 -- 172.21.15.119:0/3611241737 learned_addr learned my addr 172.21.15.119:0/3611241737 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:48.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.952+0000 7fe724d9a700 1 --2- 172.21.15.119:0/3611241737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56511ead9460 0x56511ead3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.952+0000 7fe725d9c700 1 --2- 172.21.15.119:0/3611241737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511eadd150 0x56511eae5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.953+0000 7fe72559b700 1 -- 172.21.15.119:0/3611241737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511eadd150 msgr2=0x56511eae5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:48.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.953+0000 7fe72559b700 1 --2- 172.21.15.119:0/3611241737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511eadd150 0x56511eae5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.953+0000 7fe72559b700 1 -- 172.21.15.119:0/3611241737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56511ead9460 msgr2=0x56511ead3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:48.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.953+0000 7fe72559b700 1 --2- 172.21.15.119:0/3611241737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56511ead9460 0x56511ead3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.953+0000 7fe72559b700 1 -- 172.21.15.119:0/3611241737 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56511e83ec60 con 0x56511ead8b40 2023-12-07T10:20:48.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.953+0000 7fe72559b700 1 --2- 172.21.15.119:0/3611241737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 0x56511ead3030 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0094e0 tx=0x7fe71c00f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3460eb9ebcd5c5 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:48.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.953+0000 7fe70bfff700 1 -- 172.21.15.119:0/3611241737 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c01d070 con 0x56511ead8b40 2023-12-07T10:20:48.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.953+0000 7fe70bfff700 1 -- 172.21.15.119:0/3611241737 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe71c005bc0 con 0x56511ead8b40 2023-12-07T10:20:48.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.953+0000 7fe70bfff700 1 -- 172.21.15.119:0/3611241737 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c032040 con 0x56511ead8b40 2023-12-07T10:20:48.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.954+0000 7fe73b0cab80 1 -- 172.21.15.119:0/3611241737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 msgr2=0x56511ead3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:48.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.954+0000 7fe73b0cab80 1 --2- 172.21.15.119:0/3611241737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 0x56511ead3030 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0094e0 tx=0x7fe71c00f460 comp rx=0 tx=0).stop 2023-12-07T10:20:48.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.954+0000 7fe73b0cab80 1 -- 172.21.15.119:0/3611241737 shutdown_connections 2023-12-07T10:20:48.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.954+0000 7fe73b0cab80 1 --2- 172.21.15.119:0/3611241737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511eadd150 0x56511eae5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.954+0000 7fe73b0cab80 1 --2- 172.21.15.119:0/3611241737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 0x56511ead3030 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.954+0000 7fe73b0cab80 1 --2- 172.21.15.119:0/3611241737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56511ead9460 0x56511ead3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.954+0000 7fe73b0cab80 1 -- 172.21.15.119:0/3611241737 >> 172.21.15.119:0/3611241737 conn(0x56511ea07360 msgr2=0x56511eae41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:48.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.954+0000 7fe73b0cab80 1 -- 172.21.15.119:0/3611241737 shutdown_connections 2023-12-07T10:20:48.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.954+0000 7fe73b0cab80 1 -- 172.21.15.119:0/3611241737 wait complete. 2023-12-07T10:20:48.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.954+0000 7fe73b0cab80 1 -- start start 2023-12-07T10:20:48.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.955+0000 7fe73b0cab80 1 Processor -- start 2023-12-07T10:20:48.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.955+0000 7fe73b0cab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 0x56511eaf29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.955+0000 7fe73b0cab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56511ead9460 0x56511eaf0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.955+0000 7fe73b0cab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511eadd150 0x56511eaf1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.955+0000 7fe73b0cab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56511eaf1940 con 0x56511ead8b40 2023-12-07T10:20:48.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.955+0000 7fe73b0cab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56511eaf1ab0 con 0x56511ead9460 2023-12-07T10:20:48.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.955+0000 7fe73b0cab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56511eaf1c70 con 0x56511eadd150 2023-12-07T10:20:48.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.955+0000 7fe73b0cab80 10 client.0 _ll_register_callbacks cb 0x56511e9cd940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:20:48.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.955+0000 7fe72559b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 0x56511eaf29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.955+0000 7fe72559b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 0x56511eaf29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37910/0 (socket says 172.21.15.119:37910) 2023-12-07T10:20:48.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.955+0000 7fe724d9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56511ead9460 0x56511eaf0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.955+0000 7fe725d9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511eadd150 0x56511eaf1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.955+0000 7fe72559b700 1 -- 172.21.15.119:0/1929251860 learned_addr learned my addr 172.21.15.119:0/1929251860 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:48.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.956+0000 7fe72559b700 1 -- 172.21.15.119:0/1929251860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511eadd150 msgr2=0x56511eaf1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:48.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.956+0000 7fe72559b700 1 --2- 172.21.15.119:0/1929251860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511eadd150 0x56511eaf1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.956+0000 7fe72559b700 1 -- 172.21.15.119:0/1929251860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56511ead9460 msgr2=0x56511eaf0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:48.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.956+0000 7fe72559b700 1 --2- 172.21.15.119:0/1929251860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56511ead9460 0x56511eaf0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.956+0000 7fe72559b700 1 -- 172.21.15.119:0/1929251860 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56511e855de0 con 0x56511ead8b40 2023-12-07T10:20:48.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.956+0000 7fe72559b700 1 --2- 172.21.15.119:0/1929251860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 0x56511eaf29f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe71c005ce0 tx=0x7fe71c0057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e0da93aedc028c server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:48.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.956+0000 7fe708ff9700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:20:48.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.956+0000 7fe708ff9700 1 -- 172.21.15.119:0/1929251860 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c01d070 con 0x56511ead8b40 2023-12-07T10:20:48.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.956+0000 7fe73b0cab80 1 -- 172.21.15.119:0/1929251860 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56511e9f2fd0 con 0x56511ead8b40 2023-12-07T10:20:48.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.956+0000 7fe708ff9700 1 -- 172.21.15.119:0/1929251860 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fe71c00f890 con 0x56511ead8b40 2023-12-07T10:20:48.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.956+0000 7fe708ff9700 1 -- 172.21.15.119:0/1929251860 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe71c032040 con 0x56511ead8b40 2023-12-07T10:20:48.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.956+0000 7fe73b0cab80 10 client.15852 fetch_fsmap learned FSMap version 30 2023-12-07T10:20:48.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.956+0000 7fe73b0cab80 1 -- 172.21.15.119:0/1929251860 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56511e87c250 con 0x56511ead8b40 2023-12-07T10:20:48.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.957+0000 7fe708ff9700 1 -- 172.21.15.119:0/1929251860 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56511e9f2fd0 con 0x56511ead8b40 2023-12-07T10:20:48.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.957+0000 7fe708ff9700 1 -- 172.21.15.119:0/1929251860 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe71c005040 con 0x56511ead8b40 2023-12-07T10:20:48.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.957+0000 7fe73b0cab80 10 client.15852 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:20:48.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.957+0000 7fe73b0cab80 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:20:48.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.957+0000 7fe73b0cab80 20 client.15852 populate_metadata read hostname 'smithi119' 2023-12-07T10:20:48.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.957+0000 7fe73b0cab80 1 --2- 172.21.15.119:0/1929251860 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56511eb4d930 0x56511eb4fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:48.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.957+0000 7fe73b0cab80 4 client.15852 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-12-07T10:20:48.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.957+0000 7fe73b0cab80 1 -- 172.21.15.119:0/1929251860 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56511e9e5490 con 0x56511eb4d930 2023-12-07T10:20:48.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.957+0000 7fe724d9a700 1 --2- 172.21.15.119:0/1929251860 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56511eb4d930 0x56511eb4fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:48.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.958+0000 7fe708ff9700 10 client.15852 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:48.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.958+0000 7fe724d9a700 1 --2- 172.21.15.119:0/1929251860 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56511eb4d930 0x56511eb4fd50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f99b367493c347c server_cookie=55d82f6370befbea in_seq=0 out_seq=0 2023-12-07T10:20:48.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.958+0000 7fe708ff9700 1 -- 172.21.15.119:0/1929251860 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56511e9e5490 con 0x56511eb4d930 2023-12-07T10:20:48.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.958+0000 7fe708ff9700 10 client.15852 handle_command_reply: tid=0 2023-12-07T10:20:48.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.958+0000 7fe73b0cab80 1 client.15852 shutdown 2023-12-07T10:20:48.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.958+0000 7fe73b0cab80 1 -- 172.21.15.119:0/1929251860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 msgr2=0x56511eaf29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:48.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.958+0000 7fe73b0cab80 1 --2- 172.21.15.119:0/1929251860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 0x56511eaf29f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe71c005ce0 tx=0x7fe71c0057e0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.958+0000 7fe73b0cab80 1 -- 172.21.15.119:0/1929251860 shutdown_connections 2023-12-07T10:20:48.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.958+0000 7fe73b0cab80 1 --2- 172.21.15.119:0/1929251860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511eadd150 0x56511eaf1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.958+0000 7fe73b0cab80 1 --2- 172.21.15.119:0/1929251860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56511ead8b40 0x56511eaf29f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.958+0000 7fe73b0cab80 1 --2- 172.21.15.119:0/1929251860 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56511eb4d930 0x56511eb4fd50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.958+0000 7fe73b0cab80 1 --2- 172.21.15.119:0/1929251860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56511ead9460 0x56511eaf0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.958+0000 7fe708ff9700 0 client.15852 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:48.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.958+0000 7fe73b0cab80 1 -- 172.21.15.119:0/1929251860 >> 172.21.15.119:0/1929251860 conn(0x56511ea07360 msgr2=0x56511ead3030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:48.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.958+0000 7fe73b0cab80 1 -- 172.21.15.119:0/1929251860 shutdown_connections 2023-12-07T10:20:48.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.958+0000 7fe73b0cab80 1 -- 172.21.15.119:0/1929251860 wait complete. 2023-12-07T10:20:48.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.959+0000 7fe73b0cab80 20 client.15852 trim_cache size 0 max 16384 2023-12-07T10:20:48.965 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:20:48.965 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-07T10:20:48.965 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": { 2023-12-07T10:20:48.966 INFO:teuthology.orchestra.run.smithi119.stdout: "0f1c1e61-0615-4d7b-90c2-c8e09f02e124": { 2023-12-07T10:20:48.966 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/", 2023-12-07T10:20:48.966 INFO:teuthology.orchestra.run.smithi119.stdout: "tag": "0f1c1e61-0615-4d7b-90c2-c8e09f02e124", 2023-12-07T10:20:48.966 INFO:teuthology.orchestra.run.smithi119.stdout: "options": "recursive,force" 2023-12-07T10:20:48.966 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:20:48.966 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:20:48.966 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:20:48.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.959+0000 7fe7277ff700 1 -- 172.21.15.119:0/4177081713 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe70c0a1d80 msgr2=0x7fe70c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:48.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.959+0000 7fe7277ff700 1 --2- 172.21.15.119:0/4177081713 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe70c0a1d80 0x7fe70c0a4240 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe72019e280 tx=0x7fe71800b040 comp rx=0 tx=0).stop 2023-12-07T10:20:48.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.959+0000 7fe7277ff700 1 -- 172.21.15.119:0/4177081713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe720101e00 msgr2=0x7fe72019db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:48.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.959+0000 7fe7277ff700 1 --2- 172.21.15.119:0/4177081713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe720101e00 0x7fe72019db30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe71000e9f0 tx=0x7fe71000ed00 comp rx=0 tx=0).stop 2023-12-07T10:20:48.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.959+0000 7fe7277ff700 1 -- 172.21.15.119:0/4177081713 shutdown_connections 2023-12-07T10:20:48.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.959+0000 7fe7277ff700 1 --2- 172.21.15.119:0/4177081713 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe70c0a1d80 0x7fe70c0a4240 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.959+0000 7fe7277ff700 1 --2- 172.21.15.119:0/4177081713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7201074e0 0x7fe720197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.959+0000 7fe7277ff700 1 --2- 172.21.15.119:0/4177081713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe720101440 0x7fe72019d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.959+0000 7fe7277ff700 1 --2- 172.21.15.119:0/4177081713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe720101e00 0x7fe72019db30 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:48.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.959+0000 7fe7277ff700 1 -- 172.21.15.119:0/4177081713 >> 172.21.15.119:0/4177081713 conn(0x7fe7200fcf10 msgr2=0x7fe7200fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:48.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.959+0000 7fe7277ff700 1 -- 172.21.15.119:0/4177081713 shutdown_connections 2023-12-07T10:20:48.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:20:48.959+0000 7fe7277ff700 1 -- 172.21.15.119:0/4177081713 wait complete. 2023-12-07T10:20:48.978 INFO:tasks.cephfs.filesystem:scrub status for tag:0f1c1e61-0615-4d7b-90c2-c8e09f02e124 - {'path': '/', 'tag': '0f1c1e61-0615-4d7b-90c2-c8e09f02e124', 'options': 'recursive,force'} 2023-12-07T10:20:50.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:49 smithi119 ceph-mon[97908]: scrub queued for path: / 2023-12-07T10:20:50.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:49 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:20:50.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:49 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:20:50.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:49 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:20:50.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:49 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:20:50.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:49 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:20:50.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:49 smithi119 ceph-mon[97908]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-12-07T10:20:50.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:49 smithi159 ceph-mon[134726]: scrub queued for path: / 2023-12-07T10:20:50.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:49 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:20:50.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:49 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:20:50.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:49 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:20:50.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:49 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:20:50.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:49 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:20:50.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:49 smithi159 ceph-mon[134726]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-12-07T10:20:50.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:49 smithi153 ceph-mon[133625]: scrub queued for path: / 2023-12-07T10:20:50.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:49 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:20:50.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:49 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:20:50.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:49 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:20:50.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:49 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:20:50.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:49 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:20:50.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:49 smithi153 ceph-mon[133625]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-12-07T10:20:52.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:51 smithi119 ceph-mon[97908]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-12-07T10:20:52.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:51 smithi159 ceph-mon[134726]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-12-07T10:20:52.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:51 smithi153 ceph-mon[133625]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-12-07T10:20:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.141+0000 7f75e47ff700 1 -- 172.21.15.119:0/74814997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75dc1075c0 msgr2=0x7f75dc1079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.141+0000 7f75e47ff700 1 --2- 172.21.15.119:0/74814997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75dc1075c0 0x7f75dc1079a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f75d80099c0 tx=0x7f75d8009cd0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.141+0000 7f75e47ff700 1 -- 172.21.15.119:0/74814997 shutdown_connections 2023-12-07T10:20:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.141+0000 7f75e47ff700 1 --2- 172.21.15.119:0/74814997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75dc101ec0 0x7f75dc10e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.141+0000 7f75e47ff700 1 --2- 172.21.15.119:0/74814997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75dc1075c0 0x7f75dc1079a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.141+0000 7f75e47ff700 1 --2- 172.21.15.119:0/74814997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75dc101500 0x7f75dc101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.141+0000 7f75e47ff700 1 -- 172.21.15.119:0/74814997 >> 172.21.15.119:0/74814997 conn(0x7f75dc0fcfb0 msgr2=0x7f75dc0ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.141+0000 7f75e47ff700 1 -- 172.21.15.119:0/74814997 shutdown_connections 2023-12-07T10:20:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.141+0000 7f75e47ff700 1 -- 172.21.15.119:0/74814997 wait complete. 2023-12-07T10:20:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.142+0000 7f75e47ff700 1 Processor -- start 2023-12-07T10:20:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.142+0000 7f75e47ff700 1 -- start start 2023-12-07T10:20:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.143+0000 7f75e47ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75dc101500 0x7f75dc073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.143+0000 7f75e47ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75dc101ec0 0x7f75dc073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.143+0000 7f75e47ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75dc1075c0 0x7f75dc074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.143+0000 7f75e47ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc1a5c70 con 0x7f75dc1075c0 2023-12-07T10:20:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.143+0000 7f75e47ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc1a5de0 con 0x7f75dc101500 2023-12-07T10:20:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.143+0000 7f75e47ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc1a5fa0 con 0x7f75dc101ec0 2023-12-07T10:20:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.143+0000 7f75e1d9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75dc101ec0 0x7f75dc073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.143+0000 7f75e2d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75dc1075c0 0x7f75dc074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.143+0000 7f75e1d9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75dc101ec0 0x7f75dc073e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:46756/0 (socket says 172.21.15.119:46756) 2023-12-07T10:20:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.143+0000 7f75e1d9a700 1 -- 172.21.15.119:0/1661955455 learned_addr learned my addr 172.21.15.119:0/1661955455 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.143+0000 7f75e259b700 1 --2- 172.21.15.119:0/1661955455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75dc101500 0x7f75dc073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.143+0000 7f75e2d9c700 1 -- 172.21.15.119:0/1661955455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75dc101ec0 msgr2=0x7f75dc073e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.143+0000 7f75e2d9c700 1 --2- 172.21.15.119:0/1661955455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75dc101ec0 0x7f75dc073e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.143+0000 7f75e2d9c700 1 -- 172.21.15.119:0/1661955455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75dc101500 msgr2=0x7f75dc073910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.143+0000 7f75e2d9c700 1 --2- 172.21.15.119:0/1661955455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75dc101500 0x7f75dc073910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.143+0000 7f75e2d9c700 1 -- 172.21.15.119:0/1661955455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d80095f0 con 0x7f75dc1075c0 2023-12-07T10:20:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.143+0000 7f75e1d9a700 1 --2- 172.21.15.119:0/1661955455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75dc101ec0 0x7f75dc073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.144+0000 7f75e2d9c700 1 --2- 172.21.15.119:0/1661955455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75dc1075c0 0x7f75dc074390 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f75d400efc0 tx=0x7f75d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.144+0000 7f75d37fe700 1 -- 172.21.15.119:0/1661955455 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d4018070 con 0x7f75dc1075c0 2023-12-07T10:20:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.144+0000 7f75d37fe700 1 -- 172.21.15.119:0/1661955455 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75d400f040 con 0x7f75dc1075c0 2023-12-07T10:20:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.144+0000 7f75e47ff700 1 -- 172.21.15.119:0/1661955455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75dc1a6220 con 0x7f75dc1075c0 2023-12-07T10:20:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.144+0000 7f75d37fe700 1 -- 172.21.15.119:0/1661955455 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d400cd40 con 0x7f75dc1075c0 2023-12-07T10:20:54.149 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.144+0000 7f75e47ff700 1 -- 172.21.15.119:0/1661955455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75dc1a6730 con 0x7f75dc1075c0 2023-12-07T10:20:54.149 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.148+0000 7f75d37fe700 1 -- 172.21.15.119:0/1661955455 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 24) v1 ==== 136660+0+0 (secure 0 0 0) 0x7f75d4003a00 con 0x7f75dc1075c0 2023-12-07T10:20:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.149+0000 7f75f816fb80 1 Processor -- start 2023-12-07T10:20:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.150+0000 7f75f816fb80 1 -- start start 2023-12-07T10:20:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.150+0000 7f75d37fe700 1 --2- 172.21.15.119:0/1661955455 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f75c80a1d90 0x7f75c80a4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.150+0000 7f75f816fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d5e65ea60 0x560d5e65ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.150+0000 7f75f816fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5e65f380 0x560d5e659540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.150+0000 7f75f816fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d5e659ad0 0x560d5e659f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.150+0000 7f75f816fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5e5cd760 con 0x560d5e65f380 2023-12-07T10:20:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.150+0000 7f75f816fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5e5b4740 con 0x560d5e65ea60 2023-12-07T10:20:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.150+0000 7f75f816fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5e5b2060 con 0x560d5e659ad0 2023-12-07T10:20:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.150+0000 7f75d37fe700 1 -- 172.21.15.119:0/1661955455 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f75d40cda40 con 0x7f75dc1075c0 2023-12-07T10:20:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.150+0000 7f75e1d9a700 1 --2- 172.21.15.119:0/1661955455 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f75c80a1d90 0x7f75c80a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.150+0000 7f75e2d9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d5e659ad0 0x560d5e659f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.150+0000 7f75e259b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d5e65ea60 0x560d5e65ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.151+0000 7f75e1d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5e65f380 0x560d5e659540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.151+0000 7f75e259b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d5e65ea60 0x560d5e65ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:53142/0 (socket says 172.21.15.119:53142) 2023-12-07T10:20:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.151+0000 7f75e259b700 1 -- 172.21.15.119:0/4083142141 learned_addr learned my addr 172.21.15.119:0/4083142141 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.151+0000 7f75e259b700 1 -- 172.21.15.119:0/4083142141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d5e659ad0 msgr2=0x560d5e659f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.151+0000 7f75e259b700 1 --2- 172.21.15.119:0/4083142141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d5e659ad0 0x560d5e659f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.151+0000 7f75e259b700 1 -- 172.21.15.119:0/4083142141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5e65f380 msgr2=0x560d5e659540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.151+0000 7f75e259b700 1 --2- 172.21.15.119:0/4083142141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5e65f380 0x560d5e659540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.151+0000 7f75e259b700 1 -- 172.21.15.119:0/4083142141 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d5e42a5d0 con 0x560d5e65ea60 2023-12-07T10:20:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.151+0000 7f75e2d9c700 1 --2- 172.21.15.119:0/4083142141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d5e659ad0 0x560d5e659f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.151+0000 7f75e1d9a700 1 --2- 172.21.15.119:0/1661955455 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f75c80a1d90 0x7f75c80a4250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0096b0 tx=0x7f75cc0114d0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.151+0000 7f75e259b700 1 --2- 172.21.15.119:0/4083142141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d5e65ea60 0x560d5e65ee40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f75d800b580 tx=0x7f75d8005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce12b6102496347e server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.152+0000 7f75d0ff9700 1 -- 172.21.15.119:0/4083142141 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d801c050 con 0x560d5e65ea60 2023-12-07T10:20:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.152+0000 7f75d0ff9700 1 -- 172.21.15.119:0/4083142141 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75d801d040 con 0x560d5e65ea60 2023-12-07T10:20:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.152+0000 7f75d0ff9700 1 -- 172.21.15.119:0/4083142141 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d8003680 con 0x560d5e65ea60 2023-12-07T10:20:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.152+0000 7f75f816fb80 1 -- 172.21.15.119:0/4083142141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d5e65ea60 msgr2=0x560d5e65ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.152+0000 7f75f816fb80 1 --2- 172.21.15.119:0/4083142141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d5e65ea60 0x560d5e65ee40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f75d800b580 tx=0x7f75d8005fb0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.152+0000 7f75f816fb80 1 -- 172.21.15.119:0/4083142141 shutdown_connections 2023-12-07T10:20:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.152+0000 7f75f816fb80 1 --2- 172.21.15.119:0/4083142141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d5e659ad0 0x560d5e659f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.152+0000 7f75f816fb80 1 --2- 172.21.15.119:0/4083142141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d5e65ea60 0x560d5e65ee40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.152+0000 7f75f816fb80 1 --2- 172.21.15.119:0/4083142141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5e65f380 0x560d5e659540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.152+0000 7f75f816fb80 1 -- 172.21.15.119:0/4083142141 >> 172.21.15.119:0/4083142141 conn(0x560d5e588300 msgr2=0x560d5e653b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.153+0000 7f75f816fb80 1 -- 172.21.15.119:0/4083142141 shutdown_connections 2023-12-07T10:20:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.153+0000 7f75f816fb80 1 -- 172.21.15.119:0/4083142141 wait complete. 2023-12-07T10:20:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.153+0000 7f75f816fb80 1 -- start start 2023-12-07T10:20:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.153+0000 7f75f816fb80 1 Processor -- start 2023-12-07T10:20:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.154+0000 7f75f816fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d5e659ad0 0x560d5e665310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.154+0000 7f75f816fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5e65ea60 0x560d5e662600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.154+0000 7f75f816fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d5e65f380 0x560d5e662d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.154+0000 7f75f816fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5e5ad950 con 0x560d5e65ea60 2023-12-07T10:20:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.154+0000 7f75f816fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5e567500 con 0x560d5e65f380 2023-12-07T10:20:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.154+0000 7f75f816fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5e5671a0 con 0x560d5e659ad0 2023-12-07T10:20:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.154+0000 7f75f816fb80 10 client.0 _ll_register_callbacks cb 0x560d5e54e8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:20:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.154+0000 7f75e259b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d5e659ad0 0x560d5e665310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.154+0000 7f75e1d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5e65ea60 0x560d5e662600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.154+0000 7f75e2d9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d5e65f380 0x560d5e662d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.154+0000 7f75e259b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d5e659ad0 0x560d5e665310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:46770/0 (socket says 172.21.15.119:46770) 2023-12-07T10:20:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.154+0000 7f75e259b700 1 -- 172.21.15.119:0/1614063377 learned_addr learned my addr 172.21.15.119:0/1614063377 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.155+0000 7f75e1d9a700 1 -- 172.21.15.119:0/1614063377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d5e659ad0 msgr2=0x560d5e665310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.155+0000 7f75e1d9a700 1 --2- 172.21.15.119:0/1614063377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d5e659ad0 0x560d5e665310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.155+0000 7f75e1d9a700 1 -- 172.21.15.119:0/1614063377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d5e65f380 msgr2=0x560d5e662d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.155+0000 7f75e1d9a700 1 --2- 172.21.15.119:0/1614063377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d5e65f380 0x560d5e662d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.155+0000 7f75e1d9a700 1 -- 172.21.15.119:0/1614063377 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75dc1a6730 con 0x560d5e65ea60 2023-12-07T10:20:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.155+0000 7f75e2d9c700 1 --2- 172.21.15.119:0/1614063377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d5e65f380 0x560d5e662d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.155+0000 7f75e259b700 1 --2- 172.21.15.119:0/1614063377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d5e659ad0 0x560d5e665310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.155+0000 7f75e1d9a700 1 --2- 172.21.15.119:0/1614063377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5e65ea60 0x560d5e662600 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x560d5e655890 tx=0x7f75cc01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed89ed023e4b631 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.155+0000 7f75c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:20:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.155+0000 7f75c5ffb700 1 -- 172.21.15.119:0/1614063377 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc016040 con 0x560d5e65ea60 2023-12-07T10:20:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.155+0000 7f75c5ffb700 1 -- 172.21.15.119:0/1614063377 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f75cc01ed50 con 0x560d5e65ea60 2023-12-07T10:20:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.155+0000 7f75f816fb80 1 -- 172.21.15.119:0/1614063377 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d5e64c6f0 con 0x560d5e65ea60 2023-12-07T10:20:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.155+0000 7f75c5ffb700 1 -- 172.21.15.119:0/1614063377 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75cc02a410 con 0x560d5e65ea60 2023-12-07T10:20:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.155+0000 7f75f816fb80 10 client.15864 fetch_fsmap learned FSMap version 30 2023-12-07T10:20:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.155+0000 7f75f816fb80 1 -- 172.21.15.119:0/1614063377 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f75dc10ec80 con 0x560d5e65ea60 2023-12-07T10:20:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.156+0000 7f75c5ffb700 1 -- 172.21.15.119:0/1614063377 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560d5e64c6f0 con 0x560d5e65ea60 2023-12-07T10:20:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.156+0000 7f75c5ffb700 1 -- 172.21.15.119:0/1614063377 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f75cc0205b0 con 0x560d5e65ea60 2023-12-07T10:20:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.156+0000 7f75f816fb80 10 client.15864 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:20:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.156+0000 7f75f816fb80 10 client.15864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:20:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.156+0000 7f75f816fb80 20 client.15864 populate_metadata read hostname 'smithi119' 2023-12-07T10:20:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.156+0000 7f75f816fb80 1 --2- 172.21.15.119:0/1614063377 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560d5e672e00 0x560d5e675220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.156+0000 7f75f816fb80 4 client.15864 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:20:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.156+0000 7f75f816fb80 1 -- 172.21.15.119:0/1614063377 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d5e628740 con 0x560d5e672e00 2023-12-07T10:20:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.156+0000 7f75e259b700 1 --2- 172.21.15.119:0/1614063377 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560d5e672e00 0x560d5e675220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.157+0000 7f75c5ffb700 10 client.15864 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.157+0000 7f75e259b700 1 --2- 172.21.15.119:0/1614063377 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560d5e672e00 0x560d5e675220 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab49435200a32897 server_cookie=7684319267960da9 in_seq=0 out_seq=0 2023-12-07T10:20:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.158+0000 7f75c5ffb700 1 -- 172.21.15.119:0/1614063377 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560d5e628740 con 0x560d5e672e00 2023-12-07T10:20:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.158+0000 7f75c5ffb700 10 client.15864 handle_command_reply: tid=0 2023-12-07T10:20:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.158+0000 7f75f816fb80 1 client.15864 shutdown 2023-12-07T10:20:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.158+0000 7f75f816fb80 1 -- 172.21.15.119:0/1614063377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5e65ea60 msgr2=0x560d5e662600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.158+0000 7f75f816fb80 1 --2- 172.21.15.119:0/1614063377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5e65ea60 0x560d5e662600 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x560d5e655890 tx=0x7f75cc01d000 comp rx=0 tx=0).stop 2023-12-07T10:20:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.158+0000 7f75f816fb80 1 -- 172.21.15.119:0/1614063377 shutdown_connections 2023-12-07T10:20:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.158+0000 7f75f816fb80 1 --2- 172.21.15.119:0/1614063377 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560d5e672e00 0x560d5e675220 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.158+0000 7f75f816fb80 1 --2- 172.21.15.119:0/1614063377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d5e65f380 0x560d5e662d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.158+0000 7f75f816fb80 1 --2- 172.21.15.119:0/1614063377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d5e659ad0 0x560d5e665310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.158+0000 7f75c5ffb700 0 client.15864 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.158+0000 7f75f816fb80 1 --2- 172.21.15.119:0/1614063377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5e65ea60 0x560d5e662600 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.158+0000 7f75f816fb80 1 -- 172.21.15.119:0/1614063377 >> 172.21.15.119:0/1614063377 conn(0x560d5e588300 msgr2=0x560d5e64e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.158+0000 7f75f816fb80 1 -- 172.21.15.119:0/1614063377 shutdown_connections 2023-12-07T10:20:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.158+0000 7f75f816fb80 1 -- 172.21.15.119:0/1614063377 wait complete. 2023-12-07T10:20:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.158+0000 7f75f816fb80 20 client.15864 trim_cache size 0 max 16384 2023-12-07T10:20:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.173+0000 7f75f816fb80 1 Processor -- start 2023-12-07T10:20:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.173+0000 7f75f816fb80 1 -- start start 2023-12-07T10:20:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.174+0000 7f75f816fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d5e659ad0 0x560d5e659540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.174+0000 7f75f816fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d5e65ea60 0x560d5e655250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.174+0000 7f75f816fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5e65f380 0x560d5e669d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.174+0000 7f75f816fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5e42c8c0 con 0x560d5e65f380 2023-12-07T10:20:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.174+0000 7f75f816fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5e64c6c0 con 0x560d5e659ad0 2023-12-07T10:20:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.174+0000 7f75f816fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5e65f210 con 0x560d5e65ea60 2023-12-07T10:20:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.174+0000 7f75e2d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5e65f380 0x560d5e669d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.174+0000 7f75e1d9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d5e65ea60 0x560d5e655250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.174+0000 7f75e259b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d5e659ad0 0x560d5e659540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.174+0000 7f75e1d9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d5e65ea60 0x560d5e655250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:46776/0 (socket says 172.21.15.119:46776) 2023-12-07T10:20:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.174+0000 7f75e1d9a700 1 -- 172.21.15.119:0/2573585637 learned_addr learned my addr 172.21.15.119:0/2573585637 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.174+0000 7f75e1d9a700 1 -- 172.21.15.119:0/2573585637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d5e659ad0 msgr2=0x560d5e659540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.174+0000 7f75e1d9a700 1 --2- 172.21.15.119:0/2573585637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d5e659ad0 0x560d5e659540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.174+0000 7f75e1d9a700 1 -- 172.21.15.119:0/2573585637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5e65f380 msgr2=0x560d5e669d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.174+0000 7f75e1d9a700 1 --2- 172.21.15.119:0/2573585637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5e65f380 0x560d5e669d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.174+0000 7f75e1d9a700 1 -- 172.21.15.119:0/2573585637 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d5e3d6d80 con 0x560d5e65ea60 2023-12-07T10:20:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.175+0000 7f75e2d9c700 1 --2- 172.21.15.119:0/2573585637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5e65f380 0x560d5e669d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:20:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.175+0000 7f75e1d9a700 1 --2- 172.21.15.119:0/2573585637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d5e65ea60 0x560d5e655250 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0209e0 tx=0x7f75cc00fa30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab839495e917865 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.175+0000 7f75d17fa700 1 -- 172.21.15.119:0/2573585637 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc03e070 con 0x560d5e65ea60 2023-12-07T10:20:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.175+0000 7f75d17fa700 1 -- 172.21.15.119:0/2573585637 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75cc01e9c0 con 0x560d5e65ea60 2023-12-07T10:20:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.175+0000 7f75d17fa700 1 -- 172.21.15.119:0/2573585637 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc016070 con 0x560d5e65ea60 2023-12-07T10:20:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.176+0000 7f75f816fb80 1 -- 172.21.15.119:0/2573585637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d5e65ea60 msgr2=0x560d5e655250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.176+0000 7f75f816fb80 1 --2- 172.21.15.119:0/2573585637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d5e65ea60 0x560d5e655250 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0209e0 tx=0x7f75cc00fa30 comp rx=0 tx=0).stop 2023-12-07T10:20:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.176+0000 7f75f816fb80 1 -- 172.21.15.119:0/2573585637 shutdown_connections 2023-12-07T10:20:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.176+0000 7f75f816fb80 1 --2- 172.21.15.119:0/2573585637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5e65f380 0x560d5e669d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.176+0000 7f75f816fb80 1 --2- 172.21.15.119:0/2573585637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d5e659ad0 0x560d5e659540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.176+0000 7f75f816fb80 1 --2- 172.21.15.119:0/2573585637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d5e65ea60 0x560d5e655250 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.176+0000 7f75f816fb80 1 -- 172.21.15.119:0/2573585637 >> 172.21.15.119:0/2573585637 conn(0x560d5e588300 msgr2=0x560d5e679bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.176+0000 7f75f816fb80 1 -- 172.21.15.119:0/2573585637 shutdown_connections 2023-12-07T10:20:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.176+0000 7f75f816fb80 1 -- 172.21.15.119:0/2573585637 wait complete. 2023-12-07T10:20:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.176+0000 7f75f816fb80 1 -- start start 2023-12-07T10:20:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.176+0000 7f75f816fb80 1 Processor -- start 2023-12-07T10:20:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.177+0000 7f75f816fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5e659ad0 0x560d5e668f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.177+0000 7f75f816fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d5e65ea60 0x560d5e669460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.177+0000 7f75f816fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d5e65f380 0x560d5e6699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.177+0000 7f75f816fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5e6c8340 con 0x560d5e659ad0 2023-12-07T10:20:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.177+0000 7f75f816fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5e6c8480 con 0x560d5e65f380 2023-12-07T10:20:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.177+0000 7f75f816fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5e6c85c0 con 0x560d5e65ea60 2023-12-07T10:20:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.177+0000 7f75f816fb80 10 client.0 _ll_register_callbacks cb 0x560d5e54e8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:20:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.177+0000 7f75e259b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5e659ad0 0x560d5e668f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.177+0000 7f75e1d9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d5e65ea60 0x560d5e669460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.177+0000 7f75e259b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5e659ad0 0x560d5e668f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60572/0 (socket says 172.21.15.119:60572) 2023-12-07T10:20:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.177+0000 7f75e259b700 1 -- 172.21.15.119:0/4001664671 learned_addr learned my addr 172.21.15.119:0/4001664671 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:20:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.177+0000 7f75e2d9c700 1 --2- 172.21.15.119:0/4001664671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d5e65f380 0x560d5e6699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.177+0000 7f75e259b700 1 -- 172.21.15.119:0/4001664671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d5e65ea60 msgr2=0x560d5e669460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.177+0000 7f75e259b700 1 --2- 172.21.15.119:0/4001664671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d5e65ea60 0x560d5e669460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.177+0000 7f75e259b700 1 -- 172.21.15.119:0/4001664671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d5e65f380 msgr2=0x560d5e6699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:20:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.177+0000 7f75e259b700 1 --2- 172.21.15.119:0/4001664671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d5e65f380 0x560d5e6699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.177+0000 7f75e259b700 1 -- 172.21.15.119:0/4001664671 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75dc1a6220 con 0x560d5e659ad0 2023-12-07T10:20:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.178+0000 7f75e259b700 1 --2- 172.21.15.119:0/4001664671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5e659ad0 0x560d5e668f20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f75d8009350 tx=0x7f75d800b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a860f10a47825e7 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:20:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.178+0000 7f75c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:20:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.178+0000 7f75c5ffb700 1 -- 172.21.15.119:0/4001664671 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d80337a0 con 0x560d5e659ad0 2023-12-07T10:20:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.178+0000 7f75c5ffb700 1 -- 172.21.15.119:0/4001664671 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f75d802f040 con 0x560d5e659ad0 2023-12-07T10:20:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.178+0000 7f75f816fb80 1 -- 172.21.15.119:0/4001664671 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d5e573f70 con 0x560d5e659ad0 2023-12-07T10:20:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.178+0000 7f75c5ffb700 1 -- 172.21.15.119:0/4001664671 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75d800bea0 con 0x560d5e659ad0 2023-12-07T10:20:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.178+0000 7f75f816fb80 10 client.15870 fetch_fsmap learned FSMap version 30 2023-12-07T10:20:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.178+0000 7f75f816fb80 1 -- 172.21.15.119:0/4001664671 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f75dc10ec80 con 0x560d5e659ad0 2023-12-07T10:20:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.178+0000 7f75c5ffb700 1 -- 172.21.15.119:0/4001664671 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560d5e573f70 con 0x560d5e659ad0 2023-12-07T10:20:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.179+0000 7f75c5ffb700 1 -- 172.21.15.119:0/4001664671 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f75d8003a00 con 0x560d5e659ad0 2023-12-07T10:20:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.179+0000 7f75f816fb80 10 client.15870 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:20:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.179+0000 7f75f816fb80 10 client.15870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:20:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.179+0000 7f75f816fb80 20 client.15870 populate_metadata read hostname 'smithi119' 2023-12-07T10:20:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.179+0000 7f75f816fb80 1 --2- 172.21.15.119:0/4001664671 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560d5e6c9950 0x560d5e6cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:20:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.179+0000 7f75f816fb80 4 client.15870 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:20:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.179+0000 7f75f816fb80 1 -- 172.21.15.119:0/4001664671 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560d5e566430 con 0x560d5e6c9950 2023-12-07T10:20:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.179+0000 7f75e1d9a700 1 --2- 172.21.15.119:0/4001664671 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560d5e6c9950 0x560d5e6cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:20:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.180+0000 7f75e1d9a700 1 --2- 172.21.15.119:0/4001664671 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560d5e6c9950 0x560d5e6cbd70 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32caa54be12a635e server_cookie=726e59838bb5e822 in_seq=0 out_seq=0 2023-12-07T10:20:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.180+0000 7f75c5ffb700 10 client.15870 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.184+0000 7f75c5ffb700 1 -- 172.21.15.119:0/4001664671 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560d5e566430 con 0x560d5e6c9950 2023-12-07T10:20:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.184+0000 7f75c5ffb700 10 client.15870 handle_command_reply: tid=0 2023-12-07T10:20:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.184+0000 7f75f816fb80 1 client.15870 shutdown 2023-12-07T10:20:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.184+0000 7f75f816fb80 1 -- 172.21.15.119:0/4001664671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5e659ad0 msgr2=0x560d5e668f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.184+0000 7f75f816fb80 1 --2- 172.21.15.119:0/4001664671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5e659ad0 0x560d5e668f20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f75d8009350 tx=0x7f75d800b670 comp rx=0 tx=0).stop 2023-12-07T10:20:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.184+0000 7f75f816fb80 1 -- 172.21.15.119:0/4001664671 shutdown_connections 2023-12-07T10:20:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.184+0000 7f75f816fb80 1 --2- 172.21.15.119:0/4001664671 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560d5e6c9950 0x560d5e6cbd70 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.184+0000 7f75f816fb80 1 --2- 172.21.15.119:0/4001664671 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560d5e65f380 0x560d5e6699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.184+0000 7f75c5ffb700 0 client.15870 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:20:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.184+0000 7f75f816fb80 1 --2- 172.21.15.119:0/4001664671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560d5e659ad0 0x560d5e668f20 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.184+0000 7f75f816fb80 1 --2- 172.21.15.119:0/4001664671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d5e65ea60 0x560d5e669460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.184+0000 7f75f816fb80 1 -- 172.21.15.119:0/4001664671 >> 172.21.15.119:0/4001664671 conn(0x560d5e588300 msgr2=0x560d5e65c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.184+0000 7f75f816fb80 1 -- 172.21.15.119:0/4001664671 shutdown_connections 2023-12-07T10:20:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.184+0000 7f75f816fb80 1 -- 172.21.15.119:0/4001664671 wait complete. 2023-12-07T10:20:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.184+0000 7f75f816fb80 20 client.15870 trim_cache size 0 max 16384 2023-12-07T10:20:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:20:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:20:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:20:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:20:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.185+0000 7f75e47ff700 1 -- 172.21.15.119:0/1661955455 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f75c80a1d90 msgr2=0x7f75c80a4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.185+0000 7f75e47ff700 1 --2- 172.21.15.119:0/1661955455 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f75c80a1d90 0x7f75c80a4250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0096b0 tx=0x7f75cc0114d0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.185+0000 7f75e47ff700 1 -- 172.21.15.119:0/1661955455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75dc1075c0 msgr2=0x7f75dc074390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:20:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.185+0000 7f75e47ff700 1 --2- 172.21.15.119:0/1661955455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75dc1075c0 0x7f75dc074390 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f75d400efc0 tx=0x7f75d400c3a0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.185+0000 7f75e47ff700 1 -- 172.21.15.119:0/1661955455 shutdown_connections 2023-12-07T10:20:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.185+0000 7f75e47ff700 1 --2- 172.21.15.119:0/1661955455 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f75c80a1d90 0x7f75c80a4250 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.185+0000 7f75e47ff700 1 --2- 172.21.15.119:0/1661955455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75dc1075c0 0x7f75dc074390 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.185+0000 7f75e47ff700 1 --2- 172.21.15.119:0/1661955455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75dc101500 0x7f75dc073910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.185+0000 7f75e47ff700 1 --2- 172.21.15.119:0/1661955455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75dc101ec0 0x7f75dc073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:20:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.185+0000 7f75e47ff700 1 -- 172.21.15.119:0/1661955455 >> 172.21.15.119:0/1661955455 conn(0x7f75dc0fcfb0 msgr2=0x7f75dc0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:20:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.185+0000 7f75e47ff700 1 -- 172.21.15.119:0/1661955455 shutdown_connections 2023-12-07T10:20:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:20:54.185+0000 7f75e47ff700 1 -- 172.21.15.119:0/1661955455 wait complete. 2023-12-07T10:20:54.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:53 smithi119 ceph-mon[97908]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.6 KiB/s wr, 16 op/s 2023-12-07T10:20:54.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:53 smithi159 ceph-mon[134726]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.6 KiB/s wr, 16 op/s 2023-12-07T10:20:54.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:53 smithi153 ceph-mon[133625]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.6 KiB/s wr, 16 op/s 2023-12-07T10:20:56.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:55 smithi119 ceph-mon[97908]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-12-07T10:20:56.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:55 smithi159 ceph-mon[134726]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-12-07T10:20:56.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:55 smithi153 ceph-mon[133625]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-12-07T10:20:58.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:57 smithi119 ceph-mon[97908]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.1 KiB/s wr, 16 op/s 2023-12-07T10:20:58.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:57 smithi159 ceph-mon[134726]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.1 KiB/s wr, 16 op/s 2023-12-07T10:20:58.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:57 smithi153 ceph-mon[133625]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.1 KiB/s wr, 16 op/s 2023-12-07T10:20:59.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:58 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:20:59.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:58 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:20:59.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:58 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:21:00.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:59 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:21:00.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:20:59 smithi119 ceph-mon[97908]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 9 op/s 2023-12-07T10:21:00.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:59 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:21:00.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:20:59 smithi159 ceph-mon[134726]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 9 op/s 2023-12-07T10:21:00.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:59 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:21:00.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:20:59 smithi153 ceph-mon[133625]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 9 op/s 2023-12-07T10:21:00.978 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:21:00 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:21:00.502+0000 7fb5dc7ad700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-07T10:21:00.979 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:21:00 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:21:00.502+0000 7fb5dc7ad700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-07T10:21:00.979 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:21:00 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:21:00.502+0000 7fb5dc7ad700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-07T10:21:00.979 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:21:00 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:21:00.502+0000 7fb5dc7ad700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-07T10:21:00.979 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:21:00 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:21:00.502+0000 7fb5dc7ad700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-07T10:21:02.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:01 smithi119 ceph-mon[97908]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2023-12-07T10:21:02.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:01 smithi159 ceph-mon[134726]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2023-12-07T10:21:02.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:01 smithi153 ceph-mon[133625]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2023-12-07T10:21:03.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:02 smithi119 ceph-mon[97908]: mgrmap e25: x(active, since 36s), standbys: z, y 2023-12-07T10:21:03.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:02 smithi159 ceph-mon[134726]: mgrmap e25: x(active, since 36s), standbys: z, y 2023-12-07T10:21:03.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:02 smithi153 ceph-mon[133625]: mgrmap e25: x(active, since 36s), standbys: z, y 2023-12-07T10:21:04.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:03 smithi119 ceph-mon[97908]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2023-12-07T10:21:04.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:03 smithi159 ceph-mon[134726]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2023-12-07T10:21:04.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:03 smithi153 ceph-mon[133625]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2023-12-07T10:21:06.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:05 smithi119 ceph-mon[97908]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-12-07T10:21:06.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:05 smithi159 ceph-mon[134726]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-12-07T10:21:06.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:05 smithi153 ceph-mon[133625]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-12-07T10:21:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.333+0000 7fd2d37ff700 1 -- 172.21.15.119:0/2134076008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2cc10dd40 msgr2=0x7fd2cc1101b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.333+0000 7fd2d37ff700 1 --2- 172.21.15.119:0/2134076008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2cc10dd40 0x7fd2cc1101b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4009f90 tx=0x7fd2c400b330 comp rx=0 tx=0).stop 2023-12-07T10:21:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.333+0000 7fd2d37ff700 1 -- 172.21.15.119:0/2134076008 shutdown_connections 2023-12-07T10:21:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.333+0000 7fd2d37ff700 1 --2- 172.21.15.119:0/2134076008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2cc10dd40 0x7fd2cc1101b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.333+0000 7fd2d37ff700 1 --2- 172.21.15.119:0/2134076008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2cc073a40 0x7fd2cc073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.333+0000 7fd2d37ff700 1 --2- 172.21.15.119:0/2134076008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2cc074360 0x7fd2cc10d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.333+0000 7fd2d37ff700 1 -- 172.21.15.119:0/2134076008 >> 172.21.15.119:0/2134076008 conn(0x7fd2cc0fd010 msgr2=0x7fd2cc0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.333+0000 7fd2d37ff700 1 -- 172.21.15.119:0/2134076008 shutdown_connections 2023-12-07T10:21:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.333+0000 7fd2d37ff700 1 -- 172.21.15.119:0/2134076008 wait complete. 2023-12-07T10:21:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.334+0000 7fd2d37ff700 1 Processor -- start 2023-12-07T10:21:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.335+0000 7fd2d37ff700 1 -- start start 2023-12-07T10:21:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.335+0000 7fd2d37ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2cc073a40 0x7fd2cc199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.335+0000 7fd2d37ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2cc074360 0x7fd2cc19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.335+0000 7fd2d37ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2cc10dd40 0x7fd2cc19d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.335+0000 7fd2d37ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc19db30 con 0x7fd2cc10dd40 2023-12-07T10:21:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.335+0000 7fd2d37ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc19dca0 con 0x7fd2cc073a40 2023-12-07T10:21:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.335+0000 7fd2d37ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc19de60 con 0x7fd2cc074360 2023-12-07T10:21:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.335+0000 7fd2d159b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2cc073a40 0x7fd2cc199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.335+0000 7fd2d0d9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2cc074360 0x7fd2cc19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.335+0000 7fd2d159b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2cc073a40 0x7fd2cc199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:36350/0 (socket says 172.21.15.119:36350) 2023-12-07T10:21:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.335+0000 7fd2d0d9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2cc074360 0x7fd2cc19a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:46384/0 (socket says 172.21.15.119:46384) 2023-12-07T10:21:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.335+0000 7fd2d159b700 1 -- 172.21.15.119:0/744563378 learned_addr learned my addr 172.21.15.119:0/744563378 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.335+0000 7fd2d1d9c700 1 --2- 172.21.15.119:0/744563378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2cc10dd40 0x7fd2cc19d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.336+0000 7fd2d159b700 1 -- 172.21.15.119:0/744563378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2cc074360 msgr2=0x7fd2cc19a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.336+0000 7fd2d159b700 1 --2- 172.21.15.119:0/744563378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2cc074360 0x7fd2cc19a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.336+0000 7fd2d159b700 1 -- 172.21.15.119:0/744563378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2cc10dd40 msgr2=0x7fd2cc19d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.336+0000 7fd2d159b700 1 --2- 172.21.15.119:0/744563378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2cc10dd40 0x7fd2cc19d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.336+0000 7fd2d159b700 1 -- 172.21.15.119:0/744563378 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c80095f0 con 0x7fd2cc073a40 2023-12-07T10:21:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.336+0000 7fd2d0d9a700 1 --2- 172.21.15.119:0/744563378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2cc074360 0x7fd2cc19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.336+0000 7fd2d1d9c700 1 --2- 172.21.15.119:0/744563378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2cc10dd40 0x7fd2cc19d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.336+0000 7fd2d159b700 1 --2- 172.21.15.119:0/744563378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2cc073a40 0x7fd2cc199d00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800ec10 tx=0x7fd2c800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.336+0000 7fd2c27fc700 1 -- 172.21.15.119:0/744563378 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c800cc90 con 0x7fd2cc073a40 2023-12-07T10:21:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.336+0000 7fd2c27fc700 1 -- 172.21.15.119:0/744563378 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2c80044e0 con 0x7fd2cc073a40 2023-12-07T10:21:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.336+0000 7fd2d37ff700 1 -- 172.21.15.119:0/744563378 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c4009c40 con 0x7fd2cc073a40 2023-12-07T10:21:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.336+0000 7fd2c27fc700 1 -- 172.21.15.119:0/744563378 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8010430 con 0x7fd2cc073a40 2023-12-07T10:21:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.336+0000 7fd2d37ff700 1 -- 172.21.15.119:0/744563378 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2cc1a68c0 con 0x7fd2cc073a40 2023-12-07T10:21:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.339+0000 7fd2c27fc700 1 -- 172.21.15.119:0/744563378 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 25) v1 ==== 136717+0+0 (secure 0 0 0) 0x7fd2c8003fb0 con 0x7fd2cc073a40 2023-12-07T10:21:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.341+0000 7fd2e7145b80 1 Processor -- start 2023-12-07T10:21:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.341+0000 7fd2e7145b80 1 -- start start 2023-12-07T10:21:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.341+0000 7fd2c27fc700 1 --2- 172.21.15.119:0/744563378 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd2b80a1e20 0x7fd2b80a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.341+0000 7fd2e7145b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e3c83a60 0x55c2e3c83e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.341+0000 7fd2c27fc700 1 -- 172.21.15.119:0/744563378 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fd2c8014070 con 0x7fd2cc073a40 2023-12-07T10:21:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.341+0000 7fd2e7145b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2e3c84380 0x55c2e3c7e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.341+0000 7fd2e7145b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2e3c7ead0 0x55c2e3c7ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.341+0000 7fd2e7145b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e3bf2760 con 0x55c2e3c83a60 2023-12-07T10:21:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.341+0000 7fd2e7145b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e3bd9740 con 0x55c2e3c84380 2023-12-07T10:21:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.341+0000 7fd2e7145b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e3bd7060 con 0x55c2e3c7ead0 2023-12-07T10:21:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.342+0000 7fd2d1d9c700 1 --2- 172.21.15.119:0/744563378 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd2b80a1e20 0x7fd2b80a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.342+0000 7fd2d159b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e3c83a60 0x55c2e3c83e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.342+0000 7fd2d159b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e3c83a60 0x55c2e3c83e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39328/0 (socket says 172.21.15.119:39328) 2023-12-07T10:21:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.342+0000 7fd2d159b700 1 -- 172.21.15.119:0/1212761813 learned_addr learned my addr 172.21.15.119:0/1212761813 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.342+0000 7fd2d1d9c700 1 --2- 172.21.15.119:0/1212761813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2e3c7ead0 0x55c2e3c7ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.342+0000 7fd2d0d9a700 1 --2- 172.21.15.119:0/1212761813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2e3c84380 0x55c2e3c7e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.343+0000 7fd2d0d9a700 1 -- 172.21.15.119:0/1212761813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2e3c7ead0 msgr2=0x55c2e3c7ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.343+0000 7fd2d0d9a700 1 --2- 172.21.15.119:0/1212761813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2e3c7ead0 0x55c2e3c7ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.343+0000 7fd2d0d9a700 1 -- 172.21.15.119:0/1212761813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e3c83a60 msgr2=0x55c2e3c83e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.343+0000 7fd2d0d9a700 1 --2- 172.21.15.119:0/1212761813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e3c83a60 0x55c2e3c83e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.343+0000 7fd2d0d9a700 1 -- 172.21.15.119:0/1212761813 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2e3a4f5d0 con 0x55c2e3c84380 2023-12-07T10:21:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.343+0000 7fd2d159b700 1 --2- 172.21.15.119:0/1212761813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e3c83a60 0x55c2e3c83e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.343+0000 7fd2d1d9c700 1 --2- 172.21.15.119:0/744563378 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd2b80a1e20 0x7fd2b80a42e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4009790 tx=0x7fd2c4003be0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.343+0000 7fd2d0d9a700 1 --2- 172.21.15.119:0/1212761813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2e3c84380 0x55c2e3c7e540 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc19b2b0 tx=0x7fd2bc011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c236a40819beda04 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.343+0000 7fd2b77fe700 1 -- 172.21.15.119:0/1212761813 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc004230 con 0x55c2e3c84380 2023-12-07T10:21:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.343+0000 7fd2b77fe700 1 -- 172.21.15.119:0/1212761813 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2bc004cf0 con 0x55c2e3c84380 2023-12-07T10:21:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.343+0000 7fd2b77fe700 1 -- 172.21.15.119:0/1212761813 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc005720 con 0x55c2e3c84380 2023-12-07T10:21:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.344+0000 7fd2e7145b80 1 -- 172.21.15.119:0/1212761813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2e3c84380 msgr2=0x55c2e3c7e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.344+0000 7fd2e7145b80 1 --2- 172.21.15.119:0/1212761813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2e3c84380 0x55c2e3c7e540 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc19b2b0 tx=0x7fd2bc011720 comp rx=0 tx=0).stop 2023-12-07T10:21:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.344+0000 7fd2e7145b80 1 -- 172.21.15.119:0/1212761813 shutdown_connections 2023-12-07T10:21:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.344+0000 7fd2e7145b80 1 --2- 172.21.15.119:0/1212761813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2e3c7ead0 0x55c2e3c7ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.344+0000 7fd2e7145b80 1 --2- 172.21.15.119:0/1212761813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e3c83a60 0x55c2e3c83e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.344+0000 7fd2e7145b80 1 --2- 172.21.15.119:0/1212761813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2e3c84380 0x55c2e3c7e540 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.344+0000 7fd2e7145b80 1 -- 172.21.15.119:0/1212761813 >> 172.21.15.119:0/1212761813 conn(0x55c2e3bad300 msgr2=0x55c2e3c78b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.344+0000 7fd2e7145b80 1 -- 172.21.15.119:0/1212761813 shutdown_connections 2023-12-07T10:21:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.344+0000 7fd2e7145b80 1 -- 172.21.15.119:0/1212761813 wait complete. 2023-12-07T10:21:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.345+0000 7fd2e7145b80 1 -- start start 2023-12-07T10:21:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.345+0000 7fd2e7145b80 1 Processor -- start 2023-12-07T10:21:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.345+0000 7fd2e7145b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2e3c7ead0 0x55c2e3c87720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.345+0000 7fd2e7145b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e3c83a60 0x55c2e3c87c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.345+0000 7fd2e7145b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2e3c84380 0x55c2e3c881a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.345+0000 7fd2e7145b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e3bd2950 con 0x55c2e3c83a60 2023-12-07T10:21:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.345+0000 7fd2e7145b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e3b8c500 con 0x55c2e3c7ead0 2023-12-07T10:21:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.345+0000 7fd2e7145b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e3b8c1a0 con 0x55c2e3c84380 2023-12-07T10:21:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.345+0000 7fd2e7145b80 10 client.0 _ll_register_callbacks cb 0x55c2e3b738e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.346+0000 7fd2d159b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2e3c7ead0 0x55c2e3c87720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.346+0000 7fd2d159b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2e3c7ead0 0x55c2e3c87720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:36370/0 (socket says 172.21.15.119:36370) 2023-12-07T10:21:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.346+0000 7fd2d159b700 1 -- 172.21.15.119:0/1888117287 learned_addr learned my addr 172.21.15.119:0/1888117287 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.346+0000 7fd2d1d9c700 1 --2- 172.21.15.119:0/1888117287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2e3c84380 0x55c2e3c881a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.346+0000 7fd2d0d9a700 1 --2- 172.21.15.119:0/1888117287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e3c83a60 0x55c2e3c87c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.346+0000 7fd2d159b700 1 -- 172.21.15.119:0/1888117287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2e3c84380 msgr2=0x55c2e3c881a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.346+0000 7fd2d159b700 1 --2- 172.21.15.119:0/1888117287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2e3c84380 0x55c2e3c881a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.346+0000 7fd2d159b700 1 -- 172.21.15.119:0/1888117287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e3c83a60 msgr2=0x55c2e3c87c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.346+0000 7fd2d159b700 1 --2- 172.21.15.119:0/1888117287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e3c83a60 0x55c2e3c87c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.346+0000 7fd2d159b700 1 -- 172.21.15.119:0/1888117287 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2cc1a68c0 con 0x55c2e3c7ead0 2023-12-07T10:21:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.346+0000 7fd2d159b700 1 --2- 172.21.15.119:0/1888117287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2e3c7ead0 0x55c2e3c87720 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd2c809ac40 tx=0x7fd2c8097060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b483a70f8ca6df4 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.346+0000 7fd2b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:21:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.347+0000 7fd2b4ff9700 1 -- 172.21.15.119:0/1888117287 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c809b3b0 con 0x55c2e3c7ead0 2023-12-07T10:21:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.347+0000 7fd2e7145b80 1 -- 172.21.15.119:0/1888117287 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2e3c716f0 con 0x55c2e3c7ead0 2023-12-07T10:21:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.347+0000 7fd2b4ff9700 1 -- 172.21.15.119:0/1888117287 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fd2c809acb0 con 0x55c2e3c7ead0 2023-12-07T10:21:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.347+0000 7fd2b4ff9700 1 -- 172.21.15.119:0/1888117287 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2c809ee00 con 0x55c2e3c7ead0 2023-12-07T10:21:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.347+0000 7fd2e7145b80 10 client.25907 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.347+0000 7fd2e7145b80 1 -- 172.21.15.119:0/1888117287 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fd2bc009c90 con 0x55c2e3c7ead0 2023-12-07T10:21:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.347+0000 7fd2b4ff9700 1 -- 172.21.15.119:0/1888117287 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c2e3c716f0 con 0x55c2e3c7ead0 2023-12-07T10:21:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.347+0000 7fd2b4ff9700 1 -- 172.21.15.119:0/1888117287 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd2c809a1a0 con 0x55c2e3c7ead0 2023-12-07T10:21:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.348+0000 7fd2e7145b80 10 client.25907 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.348+0000 7fd2e7145b80 10 client.25907 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:21:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.348+0000 7fd2e7145b80 20 client.25907 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.348+0000 7fd2e7145b80 1 --2- 172.21.15.119:0/1888117287 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c2e3c97fa0 0x55c2e3c9a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.348+0000 7fd2e7145b80 4 client.25907 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:21:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.348+0000 7fd2e7145b80 1 -- 172.21.15.119:0/1888117287 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2e3c4d740 con 0x55c2e3c97fa0 2023-12-07T10:21:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.348+0000 7fd2d0d9a700 1 --2- 172.21.15.119:0/1888117287 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c2e3c97fa0 0x55c2e3c9a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.349+0000 7fd2b4ff9700 10 client.25907 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.349+0000 7fd2d0d9a700 1 --2- 172.21.15.119:0/1888117287 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c2e3c97fa0 0x55c2e3c9a3c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f13796882c1ce4e0 server_cookie=99ed326f5f620c2a in_seq=0 out_seq=0 2023-12-07T10:21:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.349+0000 7fd2b4ff9700 1 -- 172.21.15.119:0/1888117287 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c2e3c4d740 con 0x55c2e3c97fa0 2023-12-07T10:21:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.349+0000 7fd2b4ff9700 10 client.25907 handle_command_reply: tid=0 2023-12-07T10:21:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.349+0000 7fd2e7145b80 1 client.25907 shutdown 2023-12-07T10:21:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.350+0000 7fd2e7145b80 1 -- 172.21.15.119:0/1888117287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2e3c7ead0 msgr2=0x55c2e3c87720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.350+0000 7fd2e7145b80 1 --2- 172.21.15.119:0/1888117287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2e3c7ead0 0x55c2e3c87720 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd2c809ac40 tx=0x7fd2c8097060 comp rx=0 tx=0).stop 2023-12-07T10:21:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.350+0000 7fd2e7145b80 1 -- 172.21.15.119:0/1888117287 shutdown_connections 2023-12-07T10:21:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.350+0000 7fd2e7145b80 1 --2- 172.21.15.119:0/1888117287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2e3c84380 0x55c2e3c881a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.350+0000 7fd2e7145b80 1 --2- 172.21.15.119:0/1888117287 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c2e3c97fa0 0x55c2e3c9a3c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.350+0000 7fd2e7145b80 1 --2- 172.21.15.119:0/1888117287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2e3c7ead0 0x55c2e3c87720 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.350+0000 7fd2b4ff9700 0 client.25907 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.350+0000 7fd2e7145b80 1 --2- 172.21.15.119:0/1888117287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e3c83a60 0x55c2e3c87c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.350+0000 7fd2e7145b80 1 -- 172.21.15.119:0/1888117287 >> 172.21.15.119:0/1888117287 conn(0x55c2e3bad300 msgr2=0x55c2e3c73980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.350+0000 7fd2e7145b80 1 -- 172.21.15.119:0/1888117287 shutdown_connections 2023-12-07T10:21:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.350+0000 7fd2e7145b80 1 -- 172.21.15.119:0/1888117287 wait complete. 2023-12-07T10:21:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.350+0000 7fd2e7145b80 20 client.25907 trim_cache size 0 max 16384 2023-12-07T10:21:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.366+0000 7fd2e7145b80 1 Processor -- start 2023-12-07T10:21:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.366+0000 7fd2e7145b80 1 -- start start 2023-12-07T10:21:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.366+0000 7fd2e7145b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2e3c7ead0 0x55c2e3c7e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.366+0000 7fd2e7145b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e3c83a60 0x55c2e3c8edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.366+0000 7fd2e7145b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2e3c84380 0x55c2e3c8f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.366+0000 7fd2e7145b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e3a518c0 con 0x55c2e3c83a60 2023-12-07T10:21:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.366+0000 7fd2e7145b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e3c7b030 con 0x55c2e3c7ead0 2023-12-07T10:21:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.366+0000 7fd2e7145b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e3c7a6a0 con 0x55c2e3c84380 2023-12-07T10:21:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.367+0000 7fd2d0d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e3c83a60 0x55c2e3c8edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.367+0000 7fd2d0d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e3c83a60 0x55c2e3c8edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39344/0 (socket says 172.21.15.119:39344) 2023-12-07T10:21:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.367+0000 7fd2d1d9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2e3c84380 0x55c2e3c8f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.367+0000 7fd2d0d9a700 1 -- 172.21.15.119:0/1543677943 learned_addr learned my addr 172.21.15.119:0/1543677943 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.367+0000 7fd2d159b700 1 --2- 172.21.15.119:0/1543677943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2e3c7ead0 0x55c2e3c7e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.367+0000 7fd2d1d9c700 1 -- 172.21.15.119:0/1543677943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2e3c7ead0 msgr2=0x55c2e3c7e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.367+0000 7fd2d1d9c700 1 --2- 172.21.15.119:0/1543677943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2e3c7ead0 0x55c2e3c7e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.367+0000 7fd2d1d9c700 1 -- 172.21.15.119:0/1543677943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e3c83a60 msgr2=0x55c2e3c8edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.367+0000 7fd2d1d9c700 1 --2- 172.21.15.119:0/1543677943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e3c83a60 0x55c2e3c8edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.367+0000 7fd2d1d9c700 1 -- 172.21.15.119:0/1543677943 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2e39fbd80 con 0x55c2e3c84380 2023-12-07T10:21:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.367+0000 7fd2d159b700 1 --2- 172.21.15.119:0/1543677943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2e3c7ead0 0x55c2e3c7e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.367+0000 7fd2d1d9c700 1 --2- 172.21.15.119:0/1543677943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2e3c84380 0x55c2e3c8f310 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4025000 tx=0x7fd2c4023000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d237f55be6be68b7 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.368+0000 7fd2b7fff700 1 -- 172.21.15.119:0/1543677943 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c400e040 con 0x55c2e3c84380 2023-12-07T10:21:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.368+0000 7fd2b7fff700 1 -- 172.21.15.119:0/1543677943 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2c400be50 con 0x55c2e3c84380 2023-12-07T10:21:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.368+0000 7fd2b7fff700 1 -- 172.21.15.119:0/1543677943 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4005950 con 0x55c2e3c84380 2023-12-07T10:21:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.368+0000 7fd2e7145b80 1 -- 172.21.15.119:0/1543677943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2e3c84380 msgr2=0x55c2e3c8f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.368+0000 7fd2e7145b80 1 --2- 172.21.15.119:0/1543677943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2e3c84380 0x55c2e3c8f310 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4025000 tx=0x7fd2c4023000 comp rx=0 tx=0).stop 2023-12-07T10:21:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.368+0000 7fd2e7145b80 1 -- 172.21.15.119:0/1543677943 shutdown_connections 2023-12-07T10:21:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.368+0000 7fd2e7145b80 1 --2- 172.21.15.119:0/1543677943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2e3c84380 0x55c2e3c8f310 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.368+0000 7fd2e7145b80 1 --2- 172.21.15.119:0/1543677943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2e3c7ead0 0x55c2e3c7e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.368+0000 7fd2e7145b80 1 --2- 172.21.15.119:0/1543677943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e3c83a60 0x55c2e3c8edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.368+0000 7fd2e7145b80 1 -- 172.21.15.119:0/1543677943 >> 172.21.15.119:0/1543677943 conn(0x55c2e3bad300 msgr2=0x55c2e3c86010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.369+0000 7fd2e7145b80 1 -- 172.21.15.119:0/1543677943 shutdown_connections 2023-12-07T10:21:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.369+0000 7fd2e7145b80 1 -- 172.21.15.119:0/1543677943 wait complete. 2023-12-07T10:21:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.369+0000 7fd2e7145b80 1 -- start start 2023-12-07T10:21:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.369+0000 7fd2e7145b80 1 Processor -- start 2023-12-07T10:21:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.370+0000 7fd2e7145b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2e3c7ead0 0x55c2e3c8eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.370+0000 7fd2e7145b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e3c83a60 0x55c2e3c8f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.370+0000 7fd2e7145b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2e3c84380 0x55c2e3c8d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.370+0000 7fd2e7145b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e3c8d950 con 0x55c2e3c83a60 2023-12-07T10:21:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.370+0000 7fd2e7145b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e3c8dac0 con 0x55c2e3c7ead0 2023-12-07T10:21:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.370+0000 7fd2e7145b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e3c8dc80 con 0x55c2e3c84380 2023-12-07T10:21:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.370+0000 7fd2e7145b80 10 client.0 _ll_register_callbacks cb 0x55c2e3b738e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.370+0000 7fd2d0d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e3c83a60 0x55c2e3c8f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.370+0000 7fd2d0d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e3c83a60 0x55c2e3c8f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39350/0 (socket says 172.21.15.119:39350) 2023-12-07T10:21:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.370+0000 7fd2d0d9a700 1 -- 172.21.15.119:0/2569773174 learned_addr learned my addr 172.21.15.119:0/2569773174 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.370+0000 7fd2d1d9c700 1 --2- 172.21.15.119:0/2569773174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2e3c84380 0x55c2e3c8d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.370+0000 7fd2d159b700 1 --2- 172.21.15.119:0/2569773174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2e3c7ead0 0x55c2e3c8eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.370+0000 7fd2d0d9a700 1 -- 172.21.15.119:0/2569773174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2e3c84380 msgr2=0x55c2e3c8d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.370+0000 7fd2d0d9a700 1 --2- 172.21.15.119:0/2569773174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2e3c84380 0x55c2e3c8d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.370+0000 7fd2d0d9a700 1 -- 172.21.15.119:0/2569773174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2e3c7ead0 msgr2=0x55c2e3c8eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.370+0000 7fd2d0d9a700 1 --2- 172.21.15.119:0/2569773174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2e3c7ead0 0x55c2e3c8eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.370+0000 7fd2d0d9a700 1 -- 172.21.15.119:0/2569773174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2bc009c90 con 0x55c2e3c83a60 2023-12-07T10:21:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.371+0000 7fd2d0d9a700 1 --2- 172.21.15.119:0/2569773174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e3c83a60 0x55c2e3c8f080 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc19b2b0 tx=0x7fd2bc0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9044c414e4cd41 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.371+0000 7fd2b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:21:07.376 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.371+0000 7fd2b4ff9700 1 -- 172.21.15.119:0/2569773174 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc032070 con 0x55c2e3c83a60 2023-12-07T10:21:07.376 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.371+0000 7fd2e7145b80 1 -- 172.21.15.119:0/2569773174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2bc004cf0 con 0x55c2e3c83a60 2023-12-07T10:21:07.376 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.371+0000 7fd2b4ff9700 1 -- 172.21.15.119:0/2569773174 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fd2bc02b070 con 0x55c2e3c83a60 2023-12-07T10:21:07.376 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.371+0000 7fd2b4ff9700 1 -- 172.21.15.119:0/2569773174 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2bc005260 con 0x55c2e3c83a60 2023-12-07T10:21:07.377 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.371+0000 7fd2e7145b80 10 client.15894 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:07.377 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.371+0000 7fd2e7145b80 1 -- 172.21.15.119:0/2569773174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fd2c4026070 con 0x55c2e3c83a60 2023-12-07T10:21:07.377 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.371+0000 7fd2b4ff9700 1 -- 172.21.15.119:0/2569773174 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2bc004cf0 con 0x55c2e3c83a60 2023-12-07T10:21:07.377 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.372+0000 7fd2b4ff9700 1 -- 172.21.15.119:0/2569773174 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd2bc035020 con 0x55c2e3c83a60 2023-12-07T10:21:07.377 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.372+0000 7fd2e7145b80 10 client.15894 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:07.377 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.372+0000 7fd2e7145b80 10 client.15894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:21:07.377 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.372+0000 7fd2e7145b80 20 client.15894 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:07.377 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.372+0000 7fd2e7145b80 1 --2- 172.21.15.119:0/2569773174 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c2e3cef750 0x55c2e3cf1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:07.377 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.372+0000 7fd2e7145b80 4 client.15894 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:21:07.377 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.372+0000 7fd2e7145b80 1 -- 172.21.15.119:0/2569773174 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c2e3b98f70 con 0x55c2e3cef750 2023-12-07T10:21:07.377 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.372+0000 7fd2d159b700 1 --2- 172.21.15.119:0/2569773174 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c2e3cef750 0x55c2e3cf1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:07.377 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.373+0000 7fd2d159b700 1 --2- 172.21.15.119:0/2569773174 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c2e3cef750 0x55c2e3cf1b70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7b76893558a2cb7 server_cookie=7c78ce484c7c8261 in_seq=0 out_seq=0 2023-12-07T10:21:07.378 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.373+0000 7fd2b4ff9700 10 client.15894 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.379+0000 7fd2b4ff9700 1 -- 172.21.15.119:0/2569773174 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c2e3b98f70 con 0x55c2e3cef750 2023-12-07T10:21:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.379+0000 7fd2b4ff9700 10 client.15894 handle_command_reply: tid=0 2023-12-07T10:21:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.379+0000 7fd2e7145b80 1 client.15894 shutdown 2023-12-07T10:21:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.379+0000 7fd2e7145b80 1 -- 172.21.15.119:0/2569773174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e3c83a60 msgr2=0x55c2e3c8f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:07.382 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.379+0000 7fd2e7145b80 1 --2- 172.21.15.119:0/2569773174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e3c83a60 0x55c2e3c8f080 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc19b2b0 tx=0x7fd2bc0050b0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.382 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.379+0000 7fd2e7145b80 1 -- 172.21.15.119:0/2569773174 shutdown_connections 2023-12-07T10:21:07.382 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.379+0000 7fd2e7145b80 1 --2- 172.21.15.119:0/2569773174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2e3c84380 0x55c2e3c8d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.382 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.379+0000 7fd2e7145b80 1 --2- 172.21.15.119:0/2569773174 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c2e3cef750 0x55c2e3cf1b70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.382 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.379+0000 7fd2e7145b80 1 --2- 172.21.15.119:0/2569773174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2e3c7ead0 0x55c2e3c8eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.382 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.379+0000 7fd2e7145b80 1 --2- 172.21.15.119:0/2569773174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2e3c83a60 0x55c2e3c8f080 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.382 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.379+0000 7fd2b4ff9700 0 client.15894 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:07.382 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.379+0000 7fd2e7145b80 1 -- 172.21.15.119:0/2569773174 >> 172.21.15.119:0/2569773174 conn(0x55c2e3bad300 msgr2=0x55c2e3c730f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:07.382 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.380+0000 7fd2e7145b80 1 -- 172.21.15.119:0/2569773174 shutdown_connections 2023-12-07T10:21:07.382 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.380+0000 7fd2e7145b80 1 -- 172.21.15.119:0/2569773174 wait complete. 2023-12-07T10:21:07.382 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.380+0000 7fd2e7145b80 20 client.15894 trim_cache size 0 max 16384 2023-12-07T10:21:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:21:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:21:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:21:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:21:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.380+0000 7fd2d37ff700 1 -- 172.21.15.119:0/744563378 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd2b80a1e20 msgr2=0x7fd2b80a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.380+0000 7fd2d37ff700 1 --2- 172.21.15.119:0/744563378 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd2b80a1e20 0x7fd2b80a42e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4009790 tx=0x7fd2c4003be0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.380+0000 7fd2d37ff700 1 -- 172.21.15.119:0/744563378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2cc073a40 msgr2=0x7fd2cc199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.380+0000 7fd2d37ff700 1 --2- 172.21.15.119:0/744563378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2cc073a40 0x7fd2cc199d00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800ec10 tx=0x7fd2c800ef20 comp rx=0 tx=0).stop 2023-12-07T10:21:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.380+0000 7fd2d37ff700 1 -- 172.21.15.119:0/744563378 shutdown_connections 2023-12-07T10:21:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.381+0000 7fd2d37ff700 1 --2- 172.21.15.119:0/744563378 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd2b80a1e20 0x7fd2b80a42e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.381+0000 7fd2d37ff700 1 --2- 172.21.15.119:0/744563378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2cc10dd40 0x7fd2cc19d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.381+0000 7fd2d37ff700 1 --2- 172.21.15.119:0/744563378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2cc073a40 0x7fd2cc199d00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.381+0000 7fd2d37ff700 1 --2- 172.21.15.119:0/744563378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2cc074360 0x7fd2cc19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.381+0000 7fd2d37ff700 1 -- 172.21.15.119:0/744563378 >> 172.21.15.119:0/744563378 conn(0x7fd2cc0fd010 msgr2=0x7fd2cc0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.381+0000 7fd2d37ff700 1 -- 172.21.15.119:0/744563378 shutdown_connections 2023-12-07T10:21:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:07.381+0000 7fd2d37ff700 1 -- 172.21.15.119:0/744563378 wait complete. 2023-12-07T10:21:08.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:07 smithi119 ceph-mon[97908]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2023-12-07T10:21:08.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:07 smithi159 ceph-mon[134726]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2023-12-07T10:21:08.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:07 smithi153 ceph-mon[133625]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2023-12-07T10:21:10.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:09 smithi119 ceph-mon[97908]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-12-07T10:21:10.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:09 smithi159 ceph-mon[134726]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-12-07T10:21:10.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:09 smithi153 ceph-mon[133625]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-12-07T10:21:12.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:11 smithi119 ceph-mon[97908]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2023-12-07T10:21:12.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:11 smithi159 ceph-mon[134726]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2023-12-07T10:21:12.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:11 smithi153 ceph-mon[133625]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2023-12-07T10:21:14.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:13 smithi119 ceph-mon[97908]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 25 op/s 2023-12-07T10:21:14.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:13 smithi159 ceph-mon[134726]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 25 op/s 2023-12-07T10:21:14.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:13 smithi153 ceph-mon[133625]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 25 op/s 2023-12-07T10:21:16.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:15 smithi119 ceph-mon[97908]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2023-12-07T10:21:16.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:15 smithi159 ceph-mon[134726]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2023-12-07T10:21:16.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:15 smithi153 ceph-mon[133625]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2023-12-07T10:21:18.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:17 smithi119 ceph-mon[97908]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2023-12-07T10:21:18.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:17 smithi159 ceph-mon[134726]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2023-12-07T10:21:18.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:17 smithi153 ceph-mon[133625]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2023-12-07T10:21:18.980 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-07T10:21:19.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.243+0000 7f26bedff700 1 -- 172.21.15.119:0/3893778827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26b8101560 msgr2=0x7f26b81019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:19.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.243+0000 7f26bedff700 1 --2- 172.21.15.119:0/3893778827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26b8101560 0x7f26b81019e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009990 tx=0x7f26a4009ca0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.243+0000 7f26bedff700 1 -- 172.21.15.119:0/3893778827 shutdown_connections 2023-12-07T10:21:19.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.243+0000 7f26bedff700 1 --2- 172.21.15.119:0/3893778827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b8101f20 0x7f26b810e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.243+0000 7f26bedff700 1 --2- 172.21.15.119:0/3893778827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26b8107620 0x7f26b8107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.243+0000 7f26bedff700 1 --2- 172.21.15.119:0/3893778827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26b8101560 0x7f26b81019e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.243+0000 7f26bedff700 1 -- 172.21.15.119:0/3893778827 >> 172.21.15.119:0/3893778827 conn(0x7f26b80fcfd0 msgr2=0x7f26b80ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:19.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.243+0000 7f26bedff700 1 -- 172.21.15.119:0/3893778827 shutdown_connections 2023-12-07T10:21:19.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.243+0000 7f26bedff700 1 -- 172.21.15.119:0/3893778827 wait complete. 2023-12-07T10:21:19.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.245+0000 7f26bedff700 1 Processor -- start 2023-12-07T10:21:19.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.245+0000 7f26bedff700 1 -- start start 2023-12-07T10:21:19.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.245+0000 7f26bedff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b8101560 0x7f26b819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:19.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.245+0000 7f26bedff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26b8101f20 0x7f26b819dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:19.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.245+0000 7f26bedff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26b8107620 0x7f26b81977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:19.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.245+0000 7f26bedff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b8197e40 con 0x7f26b8101560 2023-12-07T10:21:19.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.245+0000 7f26bedff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b8197fb0 con 0x7f26b8107620 2023-12-07T10:21:19.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.245+0000 7f26bedff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b8198170 con 0x7f26b8101f20 2023-12-07T10:21:19.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.246+0000 7f26bcb9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b8101560 0x7f26b819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:19.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.246+0000 7f26bcb9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b8101560 0x7f26b819d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37706/0 (socket says 172.21.15.119:37706) 2023-12-07T10:21:19.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.246+0000 7f26bcb9b700 1 -- 172.21.15.119:0/1808817510 learned_addr learned my addr 172.21.15.119:0/1808817510 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:19.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.246+0000 7f26bd39c700 1 --2- 172.21.15.119:0/1808817510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26b8107620 0x7f26b81977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:19.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.246+0000 7f26b7fff700 1 --2- 172.21.15.119:0/1808817510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26b8101f20 0x7f26b819dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:19.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.246+0000 7f26bcb9b700 1 -- 172.21.15.119:0/1808817510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26b8101f20 msgr2=0x7f26b819dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:19.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.246+0000 7f26bcb9b700 1 --2- 172.21.15.119:0/1808817510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26b8101f20 0x7f26b819dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.246+0000 7f26bcb9b700 1 -- 172.21.15.119:0/1808817510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26b8107620 msgr2=0x7f26b81977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:19.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.246+0000 7f26bcb9b700 1 --2- 172.21.15.119:0/1808817510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26b8107620 0x7f26b81977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.246+0000 7f26bcb9b700 1 -- 172.21.15.119:0/1808817510 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a40095f0 con 0x7f26b8101560 2023-12-07T10:21:19.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.246+0000 7f26bd39c700 1 --2- 172.21.15.119:0/1808817510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26b8107620 0x7f26b81977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:19.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.247+0000 7f26bcb9b700 1 --2- 172.21.15.119:0/1808817510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b8101560 0x7f26b819d6a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00e9a0 tx=0x7f26ac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:19.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.247+0000 7f26b5ffb700 1 -- 172.21.15.119:0/1808817510 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac009790 con 0x7f26b8101560 2023-12-07T10:21:19.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.247+0000 7f26bedff700 1 -- 172.21.15.119:0/1808817510 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b8198450 con 0x7f26b8101560 2023-12-07T10:21:19.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.247+0000 7f26b5ffb700 1 -- 172.21.15.119:0/1808817510 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26ac010430 con 0x7f26b8101560 2023-12-07T10:21:19.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.247+0000 7f26b5ffb700 1 -- 172.21.15.119:0/1808817510 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac018680 con 0x7f26b8101560 2023-12-07T10:21:19.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.247+0000 7f26bedff700 1 -- 172.21.15.119:0/1808817510 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b81a4690 con 0x7f26b8101560 2023-12-07T10:21:19.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.252+0000 7f26b5ffb700 1 -- 172.21.15.119:0/1808817510 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 25) v1 ==== 136717+0+0 (secure 0 0 0) 0x7f26ac0105a0 con 0x7f26b8101560 2023-12-07T10:21:19.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.253+0000 7f26d2795b80 1 Processor -- start 2023-12-07T10:21:19.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.254+0000 7f26d2795b80 1 -- start start 2023-12-07T10:21:19.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.254+0000 7f26d2795b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f55956bb40 0x55f55956bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:19.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.254+0000 7f26d2795b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f55956c460 0x55f559579670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:19.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.254+0000 7f26d2795b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f559579bb0 0x55f55957a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:19.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.254+0000 7f26d2795b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5594df7c0 con 0x55f55956c460 2023-12-07T10:21:19.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.254+0000 7f26d2795b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5594c67a0 con 0x55f55956bb40 2023-12-07T10:21:19.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.254+0000 7f26d2795b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5594c40c0 con 0x55f559579bb0 2023-12-07T10:21:19.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.255+0000 7f26bd39c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f55956bb40 0x55f55956bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:19.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.255+0000 7f26bd39c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f55956bb40 0x55f55956bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:47482/0 (socket says 172.21.15.119:47482) 2023-12-07T10:21:19.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.255+0000 7f26bd39c700 1 -- 172.21.15.119:0/2788005977 learned_addr learned my addr 172.21.15.119:0/2788005977 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:19.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.255+0000 7f26b5ffb700 1 --2- 172.21.15.119:0/1808817510 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f26a80a1da0 0x7f26a80a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:19.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.255+0000 7f26bcb9b700 1 --2- 172.21.15.119:0/2788005977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f55956c460 0x55f559579670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:19.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.255+0000 7f26b7fff700 1 --2- 172.21.15.119:0/2788005977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f559579bb0 0x55f55957a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:19.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.255+0000 7f26bd39c700 1 -- 172.21.15.119:0/2788005977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f559579bb0 msgr2=0x55f55957a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:19.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.255+0000 7f26bd39c700 1 --2- 172.21.15.119:0/2788005977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f559579bb0 0x55f55957a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.255+0000 7f26bd39c700 1 -- 172.21.15.119:0/2788005977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f55956c460 msgr2=0x55f559579670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:19.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.255+0000 7f26bd39c700 1 --2- 172.21.15.119:0/2788005977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f55956c460 0x55f559579670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.255+0000 7f26bd39c700 1 -- 172.21.15.119:0/2788005977 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f55933c630 con 0x55f55956bb40 2023-12-07T10:21:19.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.255+0000 7f26bd39c700 1 --2- 172.21.15.119:0/1808817510 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f26a80a1da0 0x7f26a80a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:19.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.256+0000 7f26bd39c700 1 --2- 172.21.15.119:0/2788005977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f55956bb40 0x55f55956bf20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f26b0003f90 tx=0x7f26b0004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3821a8de732568b1 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:19.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.256+0000 7f26b5ffb700 1 -- 172.21.15.119:0/1808817510 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f26ac014070 con 0x7f26b8101560 2023-12-07T10:21:19.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.256+0000 7f26a2ffd700 1 -- 172.21.15.119:0/2788005977 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b000e4f0 con 0x55f55956bb40 2023-12-07T10:21:19.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.256+0000 7f26bd39c700 1 --2- 172.21.15.119:0/1808817510 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f26a80a1da0 0x7f26a80a4260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f26b0022000 tx=0x7f26b001f000 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:19.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.257+0000 7f26a2ffd700 1 -- 172.21.15.119:0/2788005977 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26b000f040 con 0x55f55956bb40 2023-12-07T10:21:19.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.257+0000 7f26a2ffd700 1 -- 172.21.15.119:0/2788005977 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b00187c0 con 0x55f55956bb40 2023-12-07T10:21:19.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.257+0000 7f26d2795b80 1 -- 172.21.15.119:0/2788005977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f55956bb40 msgr2=0x55f55956bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:19.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.257+0000 7f26d2795b80 1 --2- 172.21.15.119:0/2788005977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f55956bb40 0x55f55956bf20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f26b0003f90 tx=0x7f26b0004320 comp rx=0 tx=0).stop 2023-12-07T10:21:19.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.257+0000 7f26d2795b80 1 -- 172.21.15.119:0/2788005977 shutdown_connections 2023-12-07T10:21:19.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.257+0000 7f26d2795b80 1 --2- 172.21.15.119:0/2788005977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f559579bb0 0x55f55957a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.257+0000 7f26d2795b80 1 --2- 172.21.15.119:0/2788005977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f55956bb40 0x55f55956bf20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.257+0000 7f26d2795b80 1 --2- 172.21.15.119:0/2788005977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f55956c460 0x55f559579670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.257+0000 7f26d2795b80 1 -- 172.21.15.119:0/2788005977 >> 172.21.15.119:0/2788005977 conn(0x55f55949a360 msgr2=0x55f559566130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:19.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.257+0000 7f26d2795b80 1 -- 172.21.15.119:0/2788005977 shutdown_connections 2023-12-07T10:21:19.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.258+0000 7f26d2795b80 1 -- 172.21.15.119:0/2788005977 wait complete. 2023-12-07T10:21:19.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.258+0000 7f26d2795b80 1 -- start start 2023-12-07T10:21:19.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.258+0000 7f26d2795b80 1 Processor -- start 2023-12-07T10:21:19.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.259+0000 7f26d2795b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f55956bb40 0x55f559580140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:19.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.259+0000 7f26d2795b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f55956c460 0x55f55957d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:19.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.259+0000 7f26d2795b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f559579bb0 0x55f55957dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:19.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.259+0000 7f26d2795b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5594bf9b0 con 0x55f55956c460 2023-12-07T10:21:19.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.259+0000 7f26d2795b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f559479560 con 0x55f55956bb40 2023-12-07T10:21:19.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.259+0000 7f26d2795b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f559479200 con 0x55f559579bb0 2023-12-07T10:21:19.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.259+0000 7f26d2795b80 10 client.0 _ll_register_callbacks cb 0x55f559460940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:19.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.259+0000 7f26bcb9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f55956bb40 0x55f559580140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:19.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.259+0000 7f26bcb9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f55956bb40 0x55f559580140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:47494/0 (socket says 172.21.15.119:47494) 2023-12-07T10:21:19.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.259+0000 7f26bcb9b700 1 -- 172.21.15.119:0/2622302705 learned_addr learned my addr 172.21.15.119:0/2622302705 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:19.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.259+0000 7f26bd39c700 1 --2- 172.21.15.119:0/2622302705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f559579bb0 0x55f55957dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:19.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.259+0000 7f26b7fff700 1 --2- 172.21.15.119:0/2622302705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f55956c460 0x55f55957d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:19.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.260+0000 7f26bcb9b700 1 -- 172.21.15.119:0/2622302705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f559579bb0 msgr2=0x55f55957dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:19.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.260+0000 7f26bcb9b700 1 --2- 172.21.15.119:0/2622302705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f559579bb0 0x55f55957dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.260+0000 7f26bcb9b700 1 -- 172.21.15.119:0/2622302705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f55956c460 msgr2=0x55f55957d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:19.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.260+0000 7f26bcb9b700 1 --2- 172.21.15.119:0/2622302705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f55956c460 0x55f55957d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.260+0000 7f26bcb9b700 1 -- 172.21.15.119:0/2622302705 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b0003bb0 con 0x55f55956bb40 2023-12-07T10:21:19.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.260+0000 7f26b7fff700 1 --2- 172.21.15.119:0/2622302705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f55956c460 0x55f55957d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:21:19.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.260+0000 7f26bd39c700 1 --2- 172.21.15.119:0/2622302705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f559579bb0 0x55f55957dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:21:19.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.260+0000 7f26bcb9b700 1 --2- 172.21.15.119:0/2622302705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f55956bb40 0x55f559580140 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00c2d0 tx=0x7f26ac0ba390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8380f6d101c9a5f server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:19.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.260+0000 7f2697fff700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:21:19.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.260+0000 7f2697fff700 1 -- 172.21.15.119:0/2622302705 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac09eab0 con 0x55f55956bb40 2023-12-07T10:21:19.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.260+0000 7f2697fff700 1 -- 172.21.15.119:0/2622302705 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f26ac0a2cf0 con 0x55f55956bb40 2023-12-07T10:21:19.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.260+0000 7f26d2795b80 1 -- 172.21.15.119:0/2622302705 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f55955eb00 con 0x55f55956bb40 2023-12-07T10:21:19.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.260+0000 7f2697fff700 1 -- 172.21.15.119:0/2622302705 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26ac0b3250 con 0x55f55956bb40 2023-12-07T10:21:19.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.260+0000 7f26d2795b80 10 client.25925 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:19.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.260+0000 7f26d2795b80 1 -- 172.21.15.119:0/2622302705 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f5592e8de0 con 0x55f55956bb40 2023-12-07T10:21:19.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.261+0000 7f2697fff700 1 -- 172.21.15.119:0/2622302705 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f55955eb00 con 0x55f55956bb40 2023-12-07T10:21:19.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.261+0000 7f2697fff700 1 -- 172.21.15.119:0/2622302705 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f26ac0a2860 con 0x55f55956bb40 2023-12-07T10:21:19.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.261+0000 7f26d2795b80 10 client.25925 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:19.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.261+0000 7f26d2795b80 10 client.25925 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:21:19.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.261+0000 7f26d2795b80 20 client.25925 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:19.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.261+0000 7f26d2795b80 1 --2- 172.21.15.119:0/2622302705 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f55958db30 0x55f55958ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:19.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.261+0000 7f26d2795b80 4 client.25925 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:21:19.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.261+0000 7f26d2795b80 1 -- 172.21.15.119:0/2622302705 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f55953a7a0 con 0x55f55958db30 2023-12-07T10:21:19.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.262+0000 7f26b7fff700 1 --2- 172.21.15.119:0/2622302705 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f55958db30 0x55f55958ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:19.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.262+0000 7f2697fff700 10 client.25925 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:19.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.262+0000 7f26b7fff700 1 --2- 172.21.15.119:0/2622302705 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f55958db30 0x55f55958ff50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93153a3460ee1317 server_cookie=fb7be5589830aad3 in_seq=0 out_seq=0 2023-12-07T10:21:19.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.263+0000 7f2697fff700 1 -- 172.21.15.119:0/2622302705 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f55953a7a0 con 0x55f55958db30 2023-12-07T10:21:19.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.263+0000 7f2697fff700 10 client.25925 handle_command_reply: tid=0 2023-12-07T10:21:19.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.263+0000 7f26d2795b80 1 client.25925 shutdown 2023-12-07T10:21:19.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.263+0000 7f26d2795b80 1 -- 172.21.15.119:0/2622302705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f55956bb40 msgr2=0x55f559580140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:19.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.263+0000 7f26d2795b80 1 --2- 172.21.15.119:0/2622302705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f55956bb40 0x55f559580140 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00c2d0 tx=0x7f26ac0ba390 comp rx=0 tx=0).stop 2023-12-07T10:21:19.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.263+0000 7f26d2795b80 1 -- 172.21.15.119:0/2622302705 shutdown_connections 2023-12-07T10:21:19.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.263+0000 7f26d2795b80 1 --2- 172.21.15.119:0/2622302705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f559579bb0 0x55f55957dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.263+0000 7f26d2795b80 1 --2- 172.21.15.119:0/2622302705 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f55958db30 0x55f55958ff50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.263+0000 7f26d2795b80 1 --2- 172.21.15.119:0/2622302705 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f55956bb40 0x55f559580140 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.263+0000 7f26d2795b80 1 --2- 172.21.15.119:0/2622302705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f55956c460 0x55f55957d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.263+0000 7f26d2795b80 1 -- 172.21.15.119:0/2622302705 >> 172.21.15.119:0/2622302705 conn(0x55f55949a360 msgr2=0x55f559560d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:19.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.263+0000 7f2697fff700 0 client.25925 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:19.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.264+0000 7f26d2795b80 1 -- 172.21.15.119:0/2622302705 shutdown_connections 2023-12-07T10:21:19.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.264+0000 7f26d2795b80 1 -- 172.21.15.119:0/2622302705 wait complete. 2023-12-07T10:21:19.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.264+0000 7f26d2795b80 20 client.25925 trim_cache size 0 max 16384 2023-12-07T10:21:19.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.292+0000 7f26d2795b80 1 Processor -- start 2023-12-07T10:21:19.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.292+0000 7f26d2795b80 1 -- start start 2023-12-07T10:21:19.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.292+0000 7f26d2795b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f55956bb40 0x55f559584b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:19.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.292+0000 7f26d2795b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f55956c460 0x55f559585080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:19.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.293+0000 7f26d2795b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f559579bb0 0x55f559577c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:19.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.293+0000 7f26d2795b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55933e920 con 0x55f55956c460 2023-12-07T10:21:19.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.293+0000 7f26d2795b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f559567a00 con 0x55f55956bb40 2023-12-07T10:21:19.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.293+0000 7f26d2795b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5595677d0 con 0x55f559579bb0 2023-12-07T10:21:19.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.293+0000 7f26b7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f55956c460 0x55f559585080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:19.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.293+0000 7f26b7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f55956c460 0x55f559585080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37738/0 (socket says 172.21.15.119:37738) 2023-12-07T10:21:19.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.293+0000 7f26b7fff700 1 -- 172.21.15.119:0/1703430255 learned_addr learned my addr 172.21.15.119:0/1703430255 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:19.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.293+0000 7f26bcb9b700 1 --2- 172.21.15.119:0/1703430255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f55956bb40 0x55f559584b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:19.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.293+0000 7f26bd39c700 1 --2- 172.21.15.119:0/1703430255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f559579bb0 0x55f559577c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:19.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.293+0000 7f26b7fff700 1 -- 172.21.15.119:0/1703430255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f559579bb0 msgr2=0x55f559577c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:19.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.293+0000 7f26b7fff700 1 --2- 172.21.15.119:0/1703430255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f559579bb0 0x55f559577c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.293+0000 7f26b7fff700 1 -- 172.21.15.119:0/1703430255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f55956bb40 msgr2=0x55f559584b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:19.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.293+0000 7f26b7fff700 1 --2- 172.21.15.119:0/1703430255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f55956bb40 0x55f559584b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.293+0000 7f26b7fff700 1 -- 172.21.15.119:0/1703430255 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5592d1c60 con 0x55f55956c460 2023-12-07T10:21:19.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.293+0000 7f26bcb9b700 1 --2- 172.21.15.119:0/1703430255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f55956bb40 0x55f559584b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:19.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.294+0000 7f26b7fff700 1 --2- 172.21.15.119:0/1703430255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f55956c460 0x55f559585080 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f26a40098e0 tx=0x7f26a4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9adb0da4db85e6f server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:19.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.294+0000 7f26a37fe700 1 -- 172.21.15.119:0/1703430255 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a401d070 con 0x55f55956c460 2023-12-07T10:21:19.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.294+0000 7f26a37fe700 1 -- 172.21.15.119:0/1703430255 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26a401e040 con 0x55f55956c460 2023-12-07T10:21:19.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.294+0000 7f26a37fe700 1 -- 172.21.15.119:0/1703430255 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a400f460 con 0x55f55956c460 2023-12-07T10:21:19.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.295+0000 7f26d2795b80 1 -- 172.21.15.119:0/1703430255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f55956c460 msgr2=0x55f559585080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:19.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.295+0000 7f26d2795b80 1 --2- 172.21.15.119:0/1703430255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f55956c460 0x55f559585080 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f26a40098e0 tx=0x7f26a4009310 comp rx=0 tx=0).stop 2023-12-07T10:21:19.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.295+0000 7f26d2795b80 1 -- 172.21.15.119:0/1703430255 shutdown_connections 2023-12-07T10:21:19.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.295+0000 7f26d2795b80 1 --2- 172.21.15.119:0/1703430255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f559579bb0 0x55f559577c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.295+0000 7f26d2795b80 1 --2- 172.21.15.119:0/1703430255 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f55956bb40 0x55f559584b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.295+0000 7f26d2795b80 1 --2- 172.21.15.119:0/1703430255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f55956c460 0x55f559585080 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.295+0000 7f26d2795b80 1 -- 172.21.15.119:0/1703430255 >> 172.21.15.119:0/1703430255 conn(0x55f55949a360 msgr2=0x55f559577110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:19.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.295+0000 7f26d2795b80 1 -- 172.21.15.119:0/1703430255 shutdown_connections 2023-12-07T10:21:19.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.295+0000 7f26d2795b80 1 -- 172.21.15.119:0/1703430255 wait complete. 2023-12-07T10:21:19.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.295+0000 7f26d2795b80 1 -- start start 2023-12-07T10:21:19.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.296+0000 7f26d2795b80 1 Processor -- start 2023-12-07T10:21:19.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.296+0000 7f26d2795b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f55956bb40 0x55f559581110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:19.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.296+0000 7f26d2795b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f55956c460 0x55f559581650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:19.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.296+0000 7f26d2795b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f559579bb0 0x55f5595dc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:19.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.296+0000 7f26d2795b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5595dc710 con 0x55f55956c460 2023-12-07T10:21:19.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.296+0000 7f26d2795b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5595dc880 con 0x55f559579bb0 2023-12-07T10:21:19.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.296+0000 7f26d2795b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5595dca40 con 0x55f55956bb40 2023-12-07T10:21:19.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.296+0000 7f26d2795b80 10 client.0 _ll_register_callbacks cb 0x55f559460940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:19.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.297+0000 7f26bcb9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f55956bb40 0x55f559581110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:19.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.297+0000 7f26bcb9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f55956bb40 0x55f559581110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:51612/0 (socket says 172.21.15.119:51612) 2023-12-07T10:21:19.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.297+0000 7f26bcb9b700 1 -- 172.21.15.119:0/32475286 learned_addr learned my addr 172.21.15.119:0/32475286 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:19.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.297+0000 7f26bd39c700 1 --2- 172.21.15.119:0/32475286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f559579bb0 0x55f5595dc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:19.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.297+0000 7f26b7fff700 1 --2- 172.21.15.119:0/32475286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f55956c460 0x55f559581650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:19.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.297+0000 7f26bcb9b700 1 -- 172.21.15.119:0/32475286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f559579bb0 msgr2=0x55f5595dc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:19.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.297+0000 7f26bcb9b700 1 --2- 172.21.15.119:0/32475286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f559579bb0 0x55f5595dc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.297+0000 7f26bcb9b700 1 -- 172.21.15.119:0/32475286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f55956c460 msgr2=0x55f559581650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:19.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.297+0000 7f26bcb9b700 1 --2- 172.21.15.119:0/32475286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f55956c460 0x55f559581650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.297+0000 7f26bcb9b700 1 -- 172.21.15.119:0/32475286 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5592e8de0 con 0x55f55956bb40 2023-12-07T10:21:19.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.298+0000 7f26bd39c700 1 --2- 172.21.15.119:0/32475286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f559579bb0 0x55f5595dc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:19.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.298+0000 7f26bcb9b700 1 --2- 172.21.15.119:0/32475286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f55956bb40 0x55f559581110 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f26ac097380 tx=0x7f26ac09b1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2eba079133a61624 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:19.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.298+0000 7f2697fff700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:21:19.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.298+0000 7f2697fff700 1 -- 172.21.15.119:0/32475286 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac0b3250 con 0x55f55956bb40 2023-12-07T10:21:19.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.298+0000 7f26d2795b80 1 -- 172.21.15.119:0/32475286 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26b000f040 con 0x55f55956bb40 2023-12-07T10:21:19.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.298+0000 7f2697fff700 1 -- 172.21.15.119:0/32475286 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f26ac0a2cf0 con 0x55f55956bb40 2023-12-07T10:21:19.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.298+0000 7f2697fff700 1 -- 172.21.15.119:0/32475286 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26ac0cb320 con 0x55f55956bb40 2023-12-07T10:21:19.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.298+0000 7f26d2795b80 10 client.25918 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:19.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.298+0000 7f26d2795b80 1 -- 172.21.15.119:0/32475286 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f26b810ece0 con 0x55f55956bb40 2023-12-07T10:21:19.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.298+0000 7f2697fff700 1 -- 172.21.15.119:0/32475286 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26b000f040 con 0x55f55956bb40 2023-12-07T10:21:19.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.299+0000 7f2697fff700 1 -- 172.21.15.119:0/32475286 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f26ac099a50 con 0x55f55956bb40 2023-12-07T10:21:19.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.299+0000 7f26d2795b80 10 client.25918 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:19.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.299+0000 7f26d2795b80 10 client.25918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:21:19.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.299+0000 7f26d2795b80 20 client.25918 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:19.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.299+0000 7f26d2795b80 1 --2- 172.21.15.119:0/32475286 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f5595e4fe0 0x55f5595e7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:19.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.299+0000 7f26d2795b80 4 client.25918 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-12-07T10:21:19.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.299+0000 7f26d2795b80 1 -- 172.21.15.119:0/32475286 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f559485fd0 con 0x55f5595e4fe0 2023-12-07T10:21:19.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.299+0000 7f26b7fff700 1 --2- 172.21.15.119:0/32475286 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f5595e4fe0 0x55f5595e7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:19.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.300+0000 7f26b7fff700 1 --2- 172.21.15.119:0/32475286 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f5595e4fe0 0x55f5595e7400 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef26197ea012e182 server_cookie=4eb435daf7a7f9a2 in_seq=0 out_seq=0 2023-12-07T10:21:19.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.300+0000 7f2697fff700 10 client.25918 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:19.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.300+0000 7f2697fff700 1 -- 172.21.15.119:0/32475286 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f559485fd0 con 0x55f5595e4fe0 2023-12-07T10:21:19.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.300+0000 7f2697fff700 10 client.25918 handle_command_reply: tid=0 2023-12-07T10:21:19.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.300+0000 7f26d2795b80 1 client.25918 shutdown 2023-12-07T10:21:19.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.300+0000 7f26d2795b80 1 -- 172.21.15.119:0/32475286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f55956bb40 msgr2=0x55f559581110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:19.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.300+0000 7f26d2795b80 1 --2- 172.21.15.119:0/32475286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f55956bb40 0x55f559581110 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f26ac097380 tx=0x7f26ac09b1d0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.300+0000 7f26d2795b80 1 -- 172.21.15.119:0/32475286 shutdown_connections 2023-12-07T10:21:19.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.300+0000 7f26d2795b80 1 --2- 172.21.15.119:0/32475286 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f5595e4fe0 0x55f5595e7400 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.301+0000 7f26d2795b80 1 --2- 172.21.15.119:0/32475286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f559579bb0 0x55f5595dc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.301+0000 7f26d2795b80 1 --2- 172.21.15.119:0/32475286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f55956bb40 0x55f559581110 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.301+0000 7f26d2795b80 1 --2- 172.21.15.119:0/32475286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f55956c460 0x55f559581650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.301+0000 7f2697fff700 0 client.25918 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:19.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.301+0000 7f26d2795b80 1 -- 172.21.15.119:0/32475286 >> 172.21.15.119:0/32475286 conn(0x55f55949a360 msgr2=0x55f55957bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:19.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.301+0000 7f26d2795b80 1 -- 172.21.15.119:0/32475286 shutdown_connections 2023-12-07T10:21:19.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.301+0000 7f26d2795b80 1 -- 172.21.15.119:0/32475286 wait complete. 2023-12-07T10:21:19.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.301+0000 7f26d2795b80 20 client.25918 trim_cache size 0 max 16384 2023-12-07T10:21:19.307 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:21:19.307 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "no active scrubs running", 2023-12-07T10:21:19.307 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": {} 2023-12-07T10:21:19.307 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:21:19.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.302+0000 7f26bedff700 1 -- 172.21.15.119:0/1808817510 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f26a80a1da0 msgr2=0x7f26a80a4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:19.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.302+0000 7f26bedff700 1 --2- 172.21.15.119:0/1808817510 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f26a80a1da0 0x7f26a80a4260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f26b0022000 tx=0x7f26b001f000 comp rx=0 tx=0).stop 2023-12-07T10:21:19.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.302+0000 7f26bedff700 1 -- 172.21.15.119:0/1808817510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b8101560 msgr2=0x7f26b819d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:19.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.302+0000 7f26bedff700 1 --2- 172.21.15.119:0/1808817510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b8101560 0x7f26b819d6a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00e9a0 tx=0x7f26ac00ecb0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.302+0000 7f26bedff700 1 -- 172.21.15.119:0/1808817510 shutdown_connections 2023-12-07T10:21:19.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.302+0000 7f26bedff700 1 --2- 172.21.15.119:0/1808817510 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f26a80a1da0 0x7f26a80a4260 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.302+0000 7f26bedff700 1 --2- 172.21.15.119:0/1808817510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26b8107620 0x7f26b81977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.302+0000 7f26bedff700 1 --2- 172.21.15.119:0/1808817510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26b8101560 0x7f26b819d6a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.302+0000 7f26bedff700 1 --2- 172.21.15.119:0/1808817510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26b8101f20 0x7f26b819dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.302+0000 7f26bedff700 1 -- 172.21.15.119:0/1808817510 >> 172.21.15.119:0/1808817510 conn(0x7f26b80fcfd0 msgr2=0x7f26b80fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:19.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.302+0000 7f26bedff700 1 -- 172.21.15.119:0/1808817510 shutdown_connections 2023-12-07T10:21:19.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.302+0000 7f26bedff700 1 -- 172.21.15.119:0/1808817510 wait complete. 2023-12-07T10:21:19.327 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-07T10:21:19.328 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:21:19.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.535+0000 7fcd20b9d700 1 -- 172.21.15.119:0/3727949618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd1c1021b0 msgr2=0x7fcd1c068650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:19.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.535+0000 7fcd20b9d700 1 --2- 172.21.15.119:0/3727949618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd1c1021b0 0x7fcd1c068650 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009990 tx=0x7fcd0c009ca0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.535+0000 7fcd20b9d700 1 -- 172.21.15.119:0/3727949618 shutdown_connections 2023-12-07T10:21:19.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.535+0000 7fcd20b9d700 1 --2- 172.21.15.119:0/3727949618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd1c10db80 0x7fcd1c10ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.535+0000 7fcd20b9d700 1 --2- 172.21.15.119:0/3727949618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd1c1021b0 0x7fcd1c068650 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.535+0000 7fcd20b9d700 1 --2- 172.21.15.119:0/3727949618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd1c068b90 0x7fcd1c069010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.535+0000 7fcd20b9d700 1 -- 172.21.15.119:0/3727949618 >> 172.21.15.119:0/3727949618 conn(0x7fcd1c075d00 msgr2=0x7fcd1c076110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:19.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.535+0000 7fcd20b9d700 1 -- 172.21.15.119:0/3727949618 shutdown_connections 2023-12-07T10:21:19.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.535+0000 7fcd20b9d700 1 -- 172.21.15.119:0/3727949618 wait complete. 2023-12-07T10:21:19.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.537+0000 7fcd20b9d700 1 Processor -- start 2023-12-07T10:21:19.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.537+0000 7fcd20b9d700 1 -- start start 2023-12-07T10:21:19.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.537+0000 7fcd20b9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd1c068b90 0x7fcd1c102760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:19.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.537+0000 7fcd20b9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd1c1021b0 0x7fcd1c102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:19.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.537+0000 7fcd20b9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd1c10db80 0x7fcd1c1031e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:19.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.537+0000 7fcd20b9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c1a9ff0 con 0x7fcd1c10db80 2023-12-07T10:21:19.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.538+0000 7fcd1b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd1c068b90 0x7fcd1c102760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:19.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.538+0000 7fcd1b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd1c068b90 0x7fcd1c102760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:47506/0 (socket says 172.21.15.119:47506) 2023-12-07T10:21:19.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.538+0000 7fcd1b7fe700 1 -- 172.21.15.119:0/458100681 learned_addr learned my addr 172.21.15.119:0/458100681 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:19.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.538+0000 7fcd1affd700 1 --2- 172.21.15.119:0/458100681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd1c1021b0 0x7fcd1c102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:19.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.538+0000 7fcd20b9d700 1 -- 172.21.15.119:0/458100681 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c199700 con 0x7fcd1c068b90 2023-12-07T10:21:19.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.538+0000 7fcd20b9d700 1 -- 172.21.15.119:0/458100681 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c199870 con 0x7fcd1c1021b0 2023-12-07T10:21:19.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.538+0000 7fcd1bfff700 1 --2- 172.21.15.119:0/458100681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd1c10db80 0x7fcd1c1031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:19.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.538+0000 7fcd1affd700 1 -- 172.21.15.119:0/458100681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd1c068b90 msgr2=0x7fcd1c102760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:19.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.538+0000 7fcd1affd700 1 --2- 172.21.15.119:0/458100681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd1c068b90 0x7fcd1c102760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.538+0000 7fcd1affd700 1 -- 172.21.15.119:0/458100681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd1c10db80 msgr2=0x7fcd1c1031e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:19.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.538+0000 7fcd1affd700 1 --2- 172.21.15.119:0/458100681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd1c10db80 0x7fcd1c1031e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.538+0000 7fcd1affd700 1 -- 172.21.15.119:0/458100681 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0c0095f0 con 0x7fcd1c1021b0 2023-12-07T10:21:19.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.538+0000 7fcd1b7fe700 1 --2- 172.21.15.119:0/458100681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd1c068b90 0x7fcd1c102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:19.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.539+0000 7fcd1affd700 1 --2- 172.21.15.119:0/458100681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd1c1021b0 0x7fcd1c102ca0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000e7e0 tx=0x7fcd1000eaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:19.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.539+0000 7fcd18ff9700 1 -- 172.21.15.119:0/458100681 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10004220 con 0x7fcd1c1021b0 2023-12-07T10:21:19.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.539+0000 7fcd20b9d700 1 -- 172.21.15.119:0/458100681 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd1c199b50 con 0x7fcd1c1021b0 2023-12-07T10:21:19.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.539+0000 7fcd18ff9700 1 -- 172.21.15.119:0/458100681 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd10004cf0 con 0x7fcd1c1021b0 2023-12-07T10:21:19.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.539+0000 7fcd18ff9700 1 -- 172.21.15.119:0/458100681 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd100056e0 con 0x7fcd1c1021b0 2023-12-07T10:21:19.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.539+0000 7fcd20b9d700 1 -- 172.21.15.119:0/458100681 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1c19a010 con 0x7fcd1c1021b0 2023-12-07T10:21:19.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.540+0000 7fcd20b9d700 1 -- 172.21.15.119:0/458100681 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd1c1007d0 con 0x7fcd1c1021b0 2023-12-07T10:21:19.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.544+0000 7fcd18ff9700 1 -- 172.21.15.119:0/458100681 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) v1 ==== 136717+0+0 (secure 0 0 0) 0x7fcd10004e60 con 0x7fcd1c1021b0 2023-12-07T10:21:19.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.545+0000 7fcd18ff9700 1 --2- 172.21.15.119:0/458100681 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fcd040a1da0 0x7fcd040a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:19.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.545+0000 7fcd18ff9700 1 -- 172.21.15.119:0/458100681 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fcd10014070 con 0x7fcd1c1021b0 2023-12-07T10:21:19.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.545+0000 7fcd1b7fe700 1 --2- 172.21.15.119:0/458100681 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fcd040a1da0 0x7fcd040a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:19.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.546+0000 7fcd1b7fe700 1 --2- 172.21.15.119:0/458100681 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fcd040a1da0 0x7fcd040a4260 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c006010 tx=0x7fcd0c00b280 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:19.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.549+0000 7fcd18ff9700 1 -- 172.21.15.119:0/458100681 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fcd100968b0 con 0x7fcd1c1021b0 2023-12-07T10:21:19.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.872+0000 7fcd20b9d700 1 -- 172.21.15.119:0/458100681 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd1c068200 con 0x7fcd1c1021b0 2023-12-07T10:21:19.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.873+0000 7fcd18ff9700 1 -- 172.21.15.119:0/458100681 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11384 (secure 0 0 0) 0x7fcd10099ed0 con 0x7fcd1c1021b0 2023-12-07T10:21:19.875 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:21:19.876 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-07T10:17:41.278311+0000","modified":"2023-12-07T10:20:33.545611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24517,"mds_3":24505,"mds_4":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14535":{"gid":14535,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6841/1791998028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1791998028},{"type":"v1","addr":"172.21.15.119:6841","nonce":1791998028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24505":{"gid":24505,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/2925503246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2925503246},{"type":"v1","addr":"172.21.15.153:6837","nonce":2925503246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6839/3984962900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3984962900},{"type":"v1","addr":"172.21.15.159:6839","nonce":3984962900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.119:6839/1920762602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1920762602},{"type":"v1","addr":"172.21.15.119:6839","nonce":1920762602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/566825184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":566825184},{"type":"v1","addr":"172.21.15.153:6839","nonce":566825184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-07T10:21:19.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.876+0000 7fcd20b9d700 1 -- 172.21.15.119:0/458100681 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fcd040a1da0 msgr2=0x7fcd040a4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:19.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.876+0000 7fcd20b9d700 1 --2- 172.21.15.119:0/458100681 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fcd040a1da0 0x7fcd040a4260 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c006010 tx=0x7fcd0c00b280 comp rx=0 tx=0).stop 2023-12-07T10:21:19.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.876+0000 7fcd20b9d700 1 -- 172.21.15.119:0/458100681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd1c1021b0 msgr2=0x7fcd1c102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:19.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.876+0000 7fcd20b9d700 1 --2- 172.21.15.119:0/458100681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd1c1021b0 0x7fcd1c102ca0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000e7e0 tx=0x7fcd1000eaf0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.876+0000 7fcd20b9d700 1 -- 172.21.15.119:0/458100681 shutdown_connections 2023-12-07T10:21:19.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.876+0000 7fcd20b9d700 1 --2- 172.21.15.119:0/458100681 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fcd040a1da0 0x7fcd040a4260 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.876+0000 7fcd20b9d700 1 --2- 172.21.15.119:0/458100681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd1c10db80 0x7fcd1c1031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.876+0000 7fcd20b9d700 1 --2- 172.21.15.119:0/458100681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd1c068b90 0x7fcd1c102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.876+0000 7fcd20b9d700 1 --2- 172.21.15.119:0/458100681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd1c1021b0 0x7fcd1c102ca0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:19.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.876+0000 7fcd20b9d700 1 -- 172.21.15.119:0/458100681 >> 172.21.15.119:0/458100681 conn(0x7fcd1c075d00 msgr2=0x7fcd1c0ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:19.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.876+0000 7fcd20b9d700 1 -- 172.21.15.119:0/458100681 shutdown_connections 2023-12-07T10:21:19.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:19.876+0000 7fcd20b9d700 1 -- 172.21.15.119:0/458100681 wait complete. 2023-12-07T10:21:19.879 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 30 2023-12-07T10:21:19.905 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-12-07T10:21:20.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.144+0000 7ff9c319d700 1 -- 172.21.15.119:0/4024225372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9bc1013d0 msgr2=0x7ff9bc101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.144+0000 7ff9c319d700 1 --2- 172.21.15.119:0/4024225372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9bc1013d0 0x7ff9bc101850 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009990 tx=0x7ff9ac009ca0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.145+0000 7ff9c319d700 1 -- 172.21.15.119:0/4024225372 shutdown_connections 2023-12-07T10:21:20.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.145+0000 7ff9c319d700 1 --2- 172.21.15.119:0/4024225372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9bc101d90 0x7ff9bc10e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.145+0000 7ff9c319d700 1 --2- 172.21.15.119:0/4024225372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9bc107470 0x7ff9bc107850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.145+0000 7ff9c319d700 1 --2- 172.21.15.119:0/4024225372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9bc1013d0 0x7ff9bc101850 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.145+0000 7ff9c319d700 1 -- 172.21.15.119:0/4024225372 >> 172.21.15.119:0/4024225372 conn(0x7ff9bc0fcee0 msgr2=0x7ff9bc0ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:20.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.145+0000 7ff9c319d700 1 -- 172.21.15.119:0/4024225372 shutdown_connections 2023-12-07T10:21:20.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.145+0000 7ff9c319d700 1 -- 172.21.15.119:0/4024225372 wait complete. 2023-12-07T10:21:20.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.146+0000 7ff9c319d700 1 Processor -- start 2023-12-07T10:21:20.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.146+0000 7ff9c319d700 1 -- start start 2023-12-07T10:21:20.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.146+0000 7ff9c319d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9bc1013d0 0x7ff9bc199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.146+0000 7ff9c319d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9bc101d90 0x7ff9bc199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.146+0000 7ff9c319d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9bc107470 0x7ff9bc19e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.146+0000 7ff9c319d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc19e940 con 0x7ff9bc107470 2023-12-07T10:21:20.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.146+0000 7ff9c319d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc19eab0 con 0x7ff9bc101d90 2023-12-07T10:21:20.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.146+0000 7ff9c319d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc19ec70 con 0x7ff9bc1013d0 2023-12-07T10:21:20.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.147+0000 7ff9c219b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9bc1013d0 0x7ff9bc199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.147+0000 7ff9c219b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9bc1013d0 0x7ff9bc199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:51656/0 (socket says 172.21.15.119:51656) 2023-12-07T10:21:20.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.147+0000 7ff9c219b700 1 -- 172.21.15.119:0/3282797009 learned_addr learned my addr 172.21.15.119:0/3282797009 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:20.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.147+0000 7ff9c199a700 1 --2- 172.21.15.119:0/3282797009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9bc101d90 0x7ff9bc199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.147+0000 7ff9c299c700 1 --2- 172.21.15.119:0/3282797009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9bc107470 0x7ff9bc19e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.147+0000 7ff9c219b700 1 -- 172.21.15.119:0/3282797009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9bc101d90 msgr2=0x7ff9bc199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.147+0000 7ff9c219b700 1 --2- 172.21.15.119:0/3282797009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9bc101d90 0x7ff9bc199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.147+0000 7ff9c219b700 1 -- 172.21.15.119:0/3282797009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9bc107470 msgr2=0x7ff9bc19e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.147+0000 7ff9c219b700 1 --2- 172.21.15.119:0/3282797009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9bc107470 0x7ff9bc19e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.147+0000 7ff9c219b700 1 -- 172.21.15.119:0/3282797009 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ac0095f0 con 0x7ff9bc1013d0 2023-12-07T10:21:20.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.147+0000 7ff9c219b700 1 --2- 172.21.15.119:0/3282797009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9bc1013d0 0x7ff9bc199ab0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800e9a0 tx=0x7ff9b800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:20.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.148+0000 7ff9b37fe700 1 -- 172.21.15.119:0/3282797009 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b800ca10 con 0x7ff9bc1013d0 2023-12-07T10:21:20.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.148+0000 7ff9b37fe700 1 -- 172.21.15.119:0/3282797009 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9b800cb70 con 0x7ff9bc1013d0 2023-12-07T10:21:20.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.148+0000 7ff9b37fe700 1 -- 172.21.15.119:0/3282797009 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8018680 con 0x7ff9bc1013d0 2023-12-07T10:21:20.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.148+0000 7ff9c319d700 1 -- 172.21.15.119:0/3282797009 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9bc19ef50 con 0x7ff9bc1013d0 2023-12-07T10:21:20.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.148+0000 7ff9c319d700 1 -- 172.21.15.119:0/3282797009 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9bc1a6800 con 0x7ff9bc1013d0 2023-12-07T10:21:20.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.153+0000 7ff9b37fe700 1 -- 172.21.15.119:0/3282797009 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) v1 ==== 136717+0+0 (secure 0 0 0) 0x7ff9b800cce0 con 0x7ff9bc1013d0 2023-12-07T10:21:20.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.154+0000 7ff9b37fe700 1 --2- 172.21.15.119:0/3282797009 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff9a80a1da0 0x7ff9a80a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.154+0000 7ff9d7d54b80 1 Processor -- start 2023-12-07T10:21:20.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.154+0000 7ff9b37fe700 1 -- 172.21.15.119:0/3282797009 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7ff9b8014070 con 0x7ff9bc1013d0 2023-12-07T10:21:20.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.154+0000 7ff9d7d54b80 1 -- start start 2023-12-07T10:21:20.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.154+0000 7ff9c199a700 1 --2- 172.21.15.119:0/3282797009 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff9a80a1da0 0x7ff9a80a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.155+0000 7ff9d7d54b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56372c4fca20 0x56372c4fce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.155+0000 7ff9d7d54b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372c4f7000 0x56372c4f7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.155+0000 7ff9d7d54b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56372c4f79c0 0x56372c5037b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.155+0000 7ff9d7d54b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372c4627c0 con 0x56372c4f7000 2023-12-07T10:21:20.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.155+0000 7ff9d7d54b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372c4497a0 con 0x56372c4fca20 2023-12-07T10:21:20.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.155+0000 7ff9d7d54b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372c4470c0 con 0x56372c4f79c0 2023-12-07T10:21:20.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.155+0000 7ff9c219b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56372c4fca20 0x56372c4fce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.155+0000 7ff9c219b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56372c4fca20 0x56372c4fce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:47522/0 (socket says 172.21.15.119:47522) 2023-12-07T10:21:20.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.155+0000 7ff9c219b700 1 -- 172.21.15.119:0/427419724 learned_addr learned my addr 172.21.15.119:0/427419724 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:20.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.155+0000 7ff9c299c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56372c4f79c0 0x56372c5037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.155+0000 7ff9c199a700 1 --2- 172.21.15.119:0/427419724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372c4f7000 0x56372c4f7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.156+0000 7ff9c219b700 1 -- 172.21.15.119:0/427419724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56372c4f79c0 msgr2=0x56372c5037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.156+0000 7ff9c219b700 1 --2- 172.21.15.119:0/427419724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56372c4f79c0 0x56372c5037b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.156+0000 7ff9c219b700 1 -- 172.21.15.119:0/427419724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372c4f7000 msgr2=0x56372c4f7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.156+0000 7ff9c219b700 1 --2- 172.21.15.119:0/427419724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372c4f7000 0x56372c4f7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.156+0000 7ff9c219b700 1 -- 172.21.15.119:0/427419724 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56372c2bf630 con 0x56372c4fca20 2023-12-07T10:21:20.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.156+0000 7ff9c199a700 1 --2- 172.21.15.119:0/3282797009 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff9a80a1da0 0x7ff9a80a4260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc19afb0 tx=0x7ff9ac005060 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:20.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.156+0000 7ff9c219b700 1 --2- 172.21.15.119:0/427419724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56372c4fca20 0x56372c4fce00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800e970 tx=0x7ff9b809a540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2500534c5cb8a138 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:20.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.156+0000 7ff9b0bfe700 1 -- 172.21.15.119:0/427419724 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b809a910 con 0x56372c4fca20 2023-12-07T10:21:20.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.156+0000 7ff9b0bfe700 1 -- 172.21.15.119:0/427419724 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9b809ea60 con 0x56372c4fca20 2023-12-07T10:21:20.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.156+0000 7ff9b0bfe700 1 -- 172.21.15.119:0/427419724 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b809a060 con 0x56372c4fca20 2023-12-07T10:21:20.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.157+0000 7ff9d7d54b80 1 -- 172.21.15.119:0/427419724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56372c4fca20 msgr2=0x56372c4fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.157+0000 7ff9d7d54b80 1 --2- 172.21.15.119:0/427419724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56372c4fca20 0x56372c4fce00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800e970 tx=0x7ff9b809a540 comp rx=0 tx=0).stop 2023-12-07T10:21:20.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.157+0000 7ff9d7d54b80 1 -- 172.21.15.119:0/427419724 shutdown_connections 2023-12-07T10:21:20.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.157+0000 7ff9d7d54b80 1 --2- 172.21.15.119:0/427419724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56372c4f79c0 0x56372c5037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.157+0000 7ff9d7d54b80 1 --2- 172.21.15.119:0/427419724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56372c4fca20 0x56372c4fce00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.157+0000 7ff9d7d54b80 1 --2- 172.21.15.119:0/427419724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372c4f7000 0x56372c4f7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.157+0000 7ff9d7d54b80 1 -- 172.21.15.119:0/427419724 >> 172.21.15.119:0/427419724 conn(0x56372c41d360 msgr2=0x56372c4e9170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:20.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.157+0000 7ff9d7d54b80 1 -- 172.21.15.119:0/427419724 shutdown_connections 2023-12-07T10:21:20.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.157+0000 7ff9d7d54b80 1 -- 172.21.15.119:0/427419724 wait complete. 2023-12-07T10:21:20.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.158+0000 7ff9d7d54b80 1 -- start start 2023-12-07T10:21:20.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.158+0000 7ff9d7d54b80 1 Processor -- start 2023-12-07T10:21:20.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.159+0000 7ff9d7d54b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56372c4f7000 0x56372c5035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.159+0000 7ff9d7d54b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56372c4f79c0 0x56372c5008e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.159+0000 7ff9d7d54b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372c4fca20 0x56372c500e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.159+0000 7ff9d7d54b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372c4429b0 con 0x56372c4fca20 2023-12-07T10:21:20.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.159+0000 7ff9d7d54b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372c3fc560 con 0x56372c4f7000 2023-12-07T10:21:20.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.159+0000 7ff9d7d54b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372c3fc200 con 0x56372c4f79c0 2023-12-07T10:21:20.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.159+0000 7ff9c219b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56372c4f7000 0x56372c5035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.159+0000 7ff9c219b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56372c4f7000 0x56372c5035a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:47532/0 (socket says 172.21.15.119:47532) 2023-12-07T10:21:20.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.159+0000 7ff9c219b700 1 -- 172.21.15.119:0/3767376808 learned_addr learned my addr 172.21.15.119:0/3767376808 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:20.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.159+0000 7ff9d7d54b80 10 client.0 _ll_register_callbacks cb 0x56372c3e3940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:20.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.159+0000 7ff9c299c700 1 --2- 172.21.15.119:0/3767376808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372c4fca20 0x56372c500e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.159+0000 7ff9c219b700 1 -- 172.21.15.119:0/3767376808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56372c4f79c0 msgr2=0x56372c5008e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.159+0000 7ff9c199a700 1 --2- 172.21.15.119:0/3767376808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56372c4f79c0 0x56372c5008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.159+0000 7ff9c219b700 1 --2- 172.21.15.119:0/3767376808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56372c4f79c0 0x56372c5008e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.159+0000 7ff9c219b700 1 -- 172.21.15.119:0/3767376808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372c4fca20 msgr2=0x56372c500e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.159+0000 7ff9c219b700 1 --2- 172.21.15.119:0/3767376808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372c4fca20 0x56372c500e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.159+0000 7ff9c219b700 1 -- 172.21.15.119:0/3767376808 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9bc1a6800 con 0x56372c4f7000 2023-12-07T10:21:20.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.159+0000 7ff9c199a700 1 --2- 172.21.15.119:0/3767376808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56372c4f79c0 0x56372c5008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:21:20.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.160+0000 7ff9c299c700 1 --2- 172.21.15.119:0/3767376808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372c4fca20 0x56372c500e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:21:20.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.160+0000 7ff9c219b700 1 --2- 172.21.15.119:0/3767376808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56372c4f7000 0x56372c5035a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009230 tx=0x7ff9b80cb330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=743fc8d2be35c57b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:20.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.160+0000 7ff9a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:21:20.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.160+0000 7ff9a5ffb700 1 -- 172.21.15.119:0/3767376808 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b80ba240 con 0x56372c4f7000 2023-12-07T10:21:20.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.160+0000 7ff9d7d54b80 1 -- 172.21.15.119:0/3767376808 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56372c4e19a0 con 0x56372c4f7000 2023-12-07T10:21:20.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.160+0000 7ff9a5ffb700 1 -- 172.21.15.119:0/3767376808 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7ff9b80a2810 con 0x56372c4f7000 2023-12-07T10:21:20.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.160+0000 7ff9a5ffb700 1 -- 172.21.15.119:0/3767376808 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9b80a29d0 con 0x56372c4f7000 2023-12-07T10:21:20.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.160+0000 7ff9d7d54b80 10 client.25949 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:20.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.160+0000 7ff9d7d54b80 1 -- 172.21.15.119:0/3767376808 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56372c26bde0 con 0x56372c4f7000 2023-12-07T10:21:20.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.161+0000 7ff9a5ffb700 1 -- 172.21.15.119:0/3767376808 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56372c4e19a0 con 0x56372c4f7000 2023-12-07T10:21:20.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.161+0000 7ff9a5ffb700 1 -- 172.21.15.119:0/3767376808 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff9b809f7a0 con 0x56372c4f7000 2023-12-07T10:21:20.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.161+0000 7ff9d7d54b80 10 client.25949 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:20.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.161+0000 7ff9d7d54b80 10 client.25949 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2023-12-07T10:21:20.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.161+0000 7ff9d7d54b80 20 client.25949 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:20.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.161+0000 7ff9d7d54b80 1 --2- 172.21.15.119:0/3767376808 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x56372c5109f0 0x56372c512e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.161+0000 7ff9d7d54b80 4 client.25949 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:21:20.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.161+0000 7ff9d7d54b80 1 -- 172.21.15.119:0/3767376808 --> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56372c4bd7a0 con 0x56372c5109f0 2023-12-07T10:21:20.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.161+0000 7ff9c199a700 1 --2- 172.21.15.119:0/3767376808 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x56372c5109f0 0x56372c512e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.162+0000 7ff9a5ffb700 10 client.25949 ms_handle_connect on v2:172.21.15.153:6834/1295649464 2023-12-07T10:21:20.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.162+0000 7ff9c199a700 1 --2- 172.21.15.119:0/3767376808 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x56372c5109f0 0x56372c512e10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d8c89dddd2c9ab07 server_cookie=6b0db91af7b208dc in_seq=0 out_seq=0 2023-12-07T10:21:20.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.163+0000 7ff9a5ffb700 1 -- 172.21.15.119:0/3767376808 <== mds.4 v2:172.21.15.153:6834/1295649464 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56372c4bd7a0 con 0x56372c5109f0 2023-12-07T10:21:20.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.163+0000 7ff9a5ffb700 10 client.25949 handle_command_reply: tid=0 2023-12-07T10:21:20.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.163+0000 7ff9d7d54b80 1 client.25949 shutdown 2023-12-07T10:21:20.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.163+0000 7ff9d7d54b80 1 -- 172.21.15.119:0/3767376808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56372c4f7000 msgr2=0x56372c5035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.163+0000 7ff9d7d54b80 1 --2- 172.21.15.119:0/3767376808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56372c4f7000 0x56372c5035a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009230 tx=0x7ff9b80cb330 comp rx=0 tx=0).stop 2023-12-07T10:21:20.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.163+0000 7ff9d7d54b80 1 -- 172.21.15.119:0/3767376808 shutdown_connections 2023-12-07T10:21:20.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.163+0000 7ff9d7d54b80 1 --2- 172.21.15.119:0/3767376808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372c4fca20 0x56372c500e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.163+0000 7ff9d7d54b80 1 --2- 172.21.15.119:0/3767376808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56372c4f7000 0x56372c5035a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.163+0000 7ff9d7d54b80 1 --2- 172.21.15.119:0/3767376808 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x56372c5109f0 0x56372c512e10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.163+0000 7ff9d7d54b80 1 --2- 172.21.15.119:0/3767376808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56372c4f79c0 0x56372c5008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.163+0000 7ff9d7d54b80 1 -- 172.21.15.119:0/3767376808 >> 172.21.15.119:0/3767376808 conn(0x56372c41d360 msgr2=0x56372c4ed7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:20.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.163+0000 7ff9a5ffb700 0 client.25949 ms_handle_reset on v2:172.21.15.153:6834/1295649464 2023-12-07T10:21:20.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.163+0000 7ff9d7d54b80 1 -- 172.21.15.119:0/3767376808 shutdown_connections 2023-12-07T10:21:20.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.163+0000 7ff9d7d54b80 1 -- 172.21.15.119:0/3767376808 wait complete. 2023-12-07T10:21:20.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.164+0000 7ff9d7d54b80 20 client.25949 trim_cache size 0 max 16384 2023-12-07T10:21:20.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.177+0000 7ff9d7d54b80 1 Processor -- start 2023-12-07T10:21:20.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.177+0000 7ff9d7d54b80 1 -- start start 2023-12-07T10:21:20.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.178+0000 7ff9d7d54b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56372c4f7000 0x56372c4f73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.178+0000 7ff9d7d54b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56372c4f79c0 0x56372c5032d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.178+0000 7ff9d7d54b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372c4fca20 0x56372c507ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.178+0000 7ff9d7d54b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372c2c1920 con 0x56372c4fca20 2023-12-07T10:21:20.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.178+0000 7ff9d7d54b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372c4eaf50 con 0x56372c4f79c0 2023-12-07T10:21:20.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.178+0000 7ff9d7d54b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372c518050 con 0x56372c4f7000 2023-12-07T10:21:20.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.178+0000 7ff9c299c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372c4fca20 0x56372c507ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.178+0000 7ff9c219b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56372c4f7000 0x56372c4f73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.178+0000 7ff9c219b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56372c4f7000 0x56372c4f73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:51688/0 (socket says 172.21.15.119:51688) 2023-12-07T10:21:20.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.178+0000 7ff9c299c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372c4fca20 0x56372c507ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37810/0 (socket says 172.21.15.119:37810) 2023-12-07T10:21:20.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.178+0000 7ff9c219b700 1 -- 172.21.15.119:0/1180468721 learned_addr learned my addr 172.21.15.119:0/1180468721 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:20.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.178+0000 7ff9c199a700 1 --2- 172.21.15.119:0/1180468721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56372c4f79c0 0x56372c5032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.179+0000 7ff9c219b700 1 -- 172.21.15.119:0/1180468721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56372c4f79c0 msgr2=0x56372c5032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.179+0000 7ff9c219b700 1 --2- 172.21.15.119:0/1180468721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56372c4f79c0 0x56372c5032d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.179+0000 7ff9c219b700 1 -- 172.21.15.119:0/1180468721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372c4fca20 msgr2=0x56372c507ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.179+0000 7ff9c219b700 1 --2- 172.21.15.119:0/1180468721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372c4fca20 0x56372c507ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.179+0000 7ff9c219b700 1 -- 172.21.15.119:0/1180468721 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56372c254c60 con 0x56372c4f7000 2023-12-07T10:21:20.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.179+0000 7ff9c299c700 1 --2- 172.21.15.119:0/1180468721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372c4fca20 0x56372c507ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:20.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.179+0000 7ff9c219b700 1 --2- 172.21.15.119:0/1180468721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56372c4f7000 0x56372c4f73e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff9b80096b0 tx=0x7ff9b8098ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4fe36a267747e5 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:20.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.179+0000 7ff9b13ff700 1 -- 172.21.15.119:0/1180468721 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b80a2ca0 con 0x56372c4f7000 2023-12-07T10:21:20.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.179+0000 7ff9b13ff700 1 -- 172.21.15.119:0/1180468721 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9b809f7a0 con 0x56372c4f7000 2023-12-07T10:21:20.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.179+0000 7ff9b13ff700 1 -- 172.21.15.119:0/1180468721 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8096700 con 0x56372c4f7000 2023-12-07T10:21:20.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.180+0000 7ff9d7d54b80 1 -- 172.21.15.119:0/1180468721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56372c4f7000 msgr2=0x56372c4f73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.180+0000 7ff9d7d54b80 1 --2- 172.21.15.119:0/1180468721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56372c4f7000 0x56372c4f73e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff9b80096b0 tx=0x7ff9b8098ec0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.180+0000 7ff9d7d54b80 1 -- 172.21.15.119:0/1180468721 shutdown_connections 2023-12-07T10:21:20.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.180+0000 7ff9d7d54b80 1 --2- 172.21.15.119:0/1180468721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372c4fca20 0x56372c507ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.180+0000 7ff9d7d54b80 1 --2- 172.21.15.119:0/1180468721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56372c4f7000 0x56372c4f73e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.180+0000 7ff9d7d54b80 1 --2- 172.21.15.119:0/1180468721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56372c4f79c0 0x56372c5032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.180+0000 7ff9d7d54b80 1 -- 172.21.15.119:0/1180468721 >> 172.21.15.119:0/1180468721 conn(0x56372c41d360 msgr2=0x56372c4e90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:20.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.180+0000 7ff9d7d54b80 1 -- 172.21.15.119:0/1180468721 shutdown_connections 2023-12-07T10:21:20.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.180+0000 7ff9d7d54b80 1 -- 172.21.15.119:0/1180468721 wait complete. 2023-12-07T10:21:20.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.181+0000 7ff9d7d54b80 1 -- start start 2023-12-07T10:21:20.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.181+0000 7ff9d7d54b80 1 Processor -- start 2023-12-07T10:21:20.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.182+0000 7ff9d7d54b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56372c4f7000 0x56372c5080c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.182+0000 7ff9d7d54b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56372c4f79c0 0x56372c505eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.182+0000 7ff9d7d54b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372c4fca20 0x56372c5063f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.182+0000 7ff9d7d54b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372c5069a0 con 0x56372c4fca20 2023-12-07T10:21:20.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.182+0000 7ff9d7d54b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372c506b10 con 0x56372c4f79c0 2023-12-07T10:21:20.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.182+0000 7ff9d7d54b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56372c506cd0 con 0x56372c4f7000 2023-12-07T10:21:20.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.182+0000 7ff9d7d54b80 10 client.0 _ll_register_callbacks cb 0x56372c3e3940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:20.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.182+0000 7ff9c219b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56372c4f7000 0x56372c5080c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.182+0000 7ff9c219b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56372c4f7000 0x56372c5080c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:51694/0 (socket says 172.21.15.119:51694) 2023-12-07T10:21:20.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.182+0000 7ff9c219b700 1 -- 172.21.15.119:0/4162433097 learned_addr learned my addr 172.21.15.119:0/4162433097 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:20.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.182+0000 7ff9c299c700 1 --2- 172.21.15.119:0/4162433097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372c4fca20 0x56372c5063f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.182+0000 7ff9c199a700 1 --2- 172.21.15.119:0/4162433097 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56372c4f79c0 0x56372c505eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.183+0000 7ff9c219b700 1 -- 172.21.15.119:0/4162433097 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56372c4f79c0 msgr2=0x56372c505eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.183+0000 7ff9c219b700 1 --2- 172.21.15.119:0/4162433097 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56372c4f79c0 0x56372c505eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.183+0000 7ff9c219b700 1 -- 172.21.15.119:0/4162433097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372c4fca20 msgr2=0x56372c5063f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.183+0000 7ff9c219b700 1 --2- 172.21.15.119:0/4162433097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372c4fca20 0x56372c5063f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.183+0000 7ff9c219b700 1 -- 172.21.15.119:0/4162433097 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b4009150 con 0x56372c4f7000 2023-12-07T10:21:20.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.183+0000 7ff9c219b700 1 --2- 172.21.15.119:0/4162433097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56372c4f7000 0x56372c5080c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8000c00 tx=0x7ff9b809a450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db0b7100ac259e77 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:20.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.183+0000 7ff9a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:21:20.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.183+0000 7ff9a5ffb700 1 -- 172.21.15.119:0/4162433097 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b80a2ca0 con 0x56372c4f7000 2023-12-07T10:21:20.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.183+0000 7ff9a5ffb700 1 -- 172.21.15.119:0/4162433097 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7ff9b80a2a20 con 0x56372c4f7000 2023-12-07T10:21:20.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.183+0000 7ff9d7d54b80 1 -- 172.21.15.119:0/4162433097 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9b809ea60 con 0x56372c4f7000 2023-12-07T10:21:20.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.183+0000 7ff9a5ffb700 1 -- 172.21.15.119:0/4162433097 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9b8096700 con 0x56372c4f7000 2023-12-07T10:21:20.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.183+0000 7ff9d7d54b80 10 client.25939 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:20.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.184+0000 7ff9d7d54b80 1 -- 172.21.15.119:0/4162433097 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56372c26bde0 con 0x56372c4f7000 2023-12-07T10:21:20.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.184+0000 7ff9a5ffb700 1 -- 172.21.15.119:0/4162433097 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9b809ea60 con 0x56372c4f7000 2023-12-07T10:21:20.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.184+0000 7ff9a5ffb700 1 -- 172.21.15.119:0/4162433097 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff9b80187e0 con 0x56372c4f7000 2023-12-07T10:21:20.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.184+0000 7ff9d7d54b80 10 client.25939 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:20.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.184+0000 7ff9d7d54b80 10 client.25939 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2023-12-07T10:21:20.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.184+0000 7ff9d7d54b80 20 client.25939 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:20.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.184+0000 7ff9d7d54b80 1 --2- 172.21.15.119:0/4162433097 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x56372c566fb0 0x56372c5693a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.184+0000 7ff9d7d54b80 4 client.25939 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:21:20.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.184+0000 7ff9d7d54b80 1 -- 172.21.15.119:0/4162433097 --> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56372c408fd0 con 0x56372c566fb0 2023-12-07T10:21:20.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.184+0000 7ff9c199a700 1 --2- 172.21.15.119:0/4162433097 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x56372c566fb0 0x56372c5693a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.185+0000 7ff9c199a700 1 --2- 172.21.15.119:0/4162433097 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x56372c566fb0 0x56372c5693a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2c8198232e509411 server_cookie=2bfc2505bdd53481 in_seq=0 out_seq=0 2023-12-07T10:21:20.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.185+0000 7ff9a5ffb700 10 client.25939 ms_handle_connect on v2:172.21.15.153:6834/1295649464 2023-12-07T10:21:20.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.185+0000 7ff9a5ffb700 1 -- 172.21.15.119:0/4162433097 <== mds.4 v2:172.21.15.153:6834/1295649464 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56372c408fd0 con 0x56372c566fb0 2023-12-07T10:21:20.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.185+0000 7ff9a5ffb700 10 client.25939 handle_command_reply: tid=0 2023-12-07T10:21:20.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.185+0000 7ff9d7d54b80 1 client.25939 shutdown 2023-12-07T10:21:20.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.185+0000 7ff9d7d54b80 1 -- 172.21.15.119:0/4162433097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56372c4f7000 msgr2=0x56372c5080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.185+0000 7ff9d7d54b80 1 --2- 172.21.15.119:0/4162433097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56372c4f7000 0x56372c5080c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8000c00 tx=0x7ff9b809a450 comp rx=0 tx=0).stop 2023-12-07T10:21:20.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.186+0000 7ff9d7d54b80 1 -- 172.21.15.119:0/4162433097 shutdown_connections 2023-12-07T10:21:20.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.186+0000 7ff9d7d54b80 1 --2- 172.21.15.119:0/4162433097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372c4fca20 0x56372c5063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.186+0000 7ff9d7d54b80 1 --2- 172.21.15.119:0/4162433097 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x56372c566fb0 0x56372c5693a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.186+0000 7ff9d7d54b80 1 --2- 172.21.15.119:0/4162433097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56372c4f7000 0x56372c5080c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.186+0000 7ff9a5ffb700 0 client.25939 ms_handle_reset on v2:172.21.15.153:6834/1295649464 2023-12-07T10:21:20.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.186+0000 7ff9d7d54b80 1 --2- 172.21.15.119:0/4162433097 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56372c4f79c0 0x56372c505eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.186+0000 7ff9d7d54b80 1 -- 172.21.15.119:0/4162433097 >> 172.21.15.119:0/4162433097 conn(0x56372c41d360 msgr2=0x56372c4faa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:20.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.186+0000 7ff9d7d54b80 1 -- 172.21.15.119:0/4162433097 shutdown_connections 2023-12-07T10:21:20.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.186+0000 7ff9d7d54b80 1 -- 172.21.15.119:0/4162433097 wait complete. 2023-12-07T10:21:20.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.186+0000 7ff9d7d54b80 20 client.25939 trim_cache size 0 max 16384 2023-12-07T10:21:20.191 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:21:20.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.187+0000 7ff9c319d700 1 -- 172.21.15.119:0/3282797009 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff9a80a1da0 msgr2=0x7ff9a80a4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.187+0000 7ff9c319d700 1 --2- 172.21.15.119:0/3282797009 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff9a80a1da0 0x7ff9a80a4260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc19afb0 tx=0x7ff9ac005060 comp rx=0 tx=0).stop 2023-12-07T10:21:20.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.187+0000 7ff9c319d700 1 -- 172.21.15.119:0/3282797009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9bc1013d0 msgr2=0x7ff9bc199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.187+0000 7ff9c319d700 1 --2- 172.21.15.119:0/3282797009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9bc1013d0 0x7ff9bc199ab0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800e9a0 tx=0x7ff9b800ecb0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.187+0000 7ff9c319d700 1 -- 172.21.15.119:0/3282797009 shutdown_connections 2023-12-07T10:21:20.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.187+0000 7ff9c319d700 1 --2- 172.21.15.119:0/3282797009 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff9a80a1da0 0x7ff9a80a4260 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.187+0000 7ff9c319d700 1 --2- 172.21.15.119:0/3282797009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9bc107470 0x7ff9bc19e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.187+0000 7ff9c319d700 1 --2- 172.21.15.119:0/3282797009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9bc1013d0 0x7ff9bc199ab0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.187+0000 7ff9c319d700 1 --2- 172.21.15.119:0/3282797009 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9bc101d90 0x7ff9bc199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.187+0000 7ff9c319d700 1 -- 172.21.15.119:0/3282797009 >> 172.21.15.119:0/3282797009 conn(0x7ff9bc0fcee0 msgr2=0x7ff9bc0fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:20.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.187+0000 7ff9c319d700 1 -- 172.21.15.119:0/3282797009 shutdown_connections 2023-12-07T10:21:20.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.187+0000 7ff9c319d700 1 -- 172.21.15.119:0/3282797009 wait complete. 2023-12-07T10:21:20.207 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:19 smithi119 ceph-mon[97908]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2023-12-07T10:21:20.207 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:19 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/458100681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:21:20.210 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-12-07T10:21:20.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:19 smithi159 ceph-mon[134726]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2023-12-07T10:21:20.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:19 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/458100681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:21:20.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:19 smithi153 ceph-mon[133625]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2023-12-07T10:21:20.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:19 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/458100681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:21:20.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.443+0000 7fb29f7ff700 1 -- 172.21.15.119:0/1403538461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb298100460 msgr2=0x7fb2981008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.443+0000 7fb29f7ff700 1 --2- 172.21.15.119:0/1403538461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb298100460 0x7fb2981008e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb28c009990 tx=0x7fb28c009ca0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.443+0000 7fb29f7ff700 1 -- 172.21.15.119:0/1403538461 shutdown_connections 2023-12-07T10:21:20.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.443+0000 7fb29f7ff700 1 --2- 172.21.15.119:0/1403538461 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb298100e20 0x7fb29810d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.443+0000 7fb29f7ff700 1 --2- 172.21.15.119:0/1403538461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2981053d0 0x7fb2981057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.443+0000 7fb29f7ff700 1 --2- 172.21.15.119:0/1403538461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb298100460 0x7fb2981008e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.443+0000 7fb29f7ff700 1 -- 172.21.15.119:0/1403538461 >> 172.21.15.119:0/1403538461 conn(0x7fb298077610 msgr2=0x7fb298077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:20.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.443+0000 7fb29f7ff700 1 -- 172.21.15.119:0/1403538461 shutdown_connections 2023-12-07T10:21:20.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.443+0000 7fb29f7ff700 1 -- 172.21.15.119:0/1403538461 wait complete. 2023-12-07T10:21:20.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.444+0000 7fb29f7ff700 1 Processor -- start 2023-12-07T10:21:20.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.445+0000 7fb29f7ff700 1 -- start start 2023-12-07T10:21:20.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.445+0000 7fb29f7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb298100460 0x7fb2981a2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.445+0000 7fb29f7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb298100e20 0x7fb2981a31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.445+0000 7fb29f7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2981053d0 0x7fb29810c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.445+0000 7fb29f7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29810ca80 con 0x7fb298100e20 2023-12-07T10:21:20.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.445+0000 7fb29f7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29810cbf0 con 0x7fb298100460 2023-12-07T10:21:20.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.445+0000 7fb29f7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29810cdb0 con 0x7fb2981053d0 2023-12-07T10:21:20.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.446+0000 7fb29cd9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb298100e20 0x7fb2981a31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.446+0000 7fb29cd9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb298100e20 0x7fb2981a31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37858/0 (socket says 172.21.15.119:37858) 2023-12-07T10:21:20.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.446+0000 7fb29cd9a700 1 -- 172.21.15.119:0/1081030707 learned_addr learned my addr 172.21.15.119:0/1081030707 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:20.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.446+0000 7fb29d59b700 1 --2- 172.21.15.119:0/1081030707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb298100460 0x7fb2981a2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.446+0000 7fb29dd9c700 1 --2- 172.21.15.119:0/1081030707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2981053d0 0x7fb29810c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.446+0000 7fb29cd9a700 1 -- 172.21.15.119:0/1081030707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2981053d0 msgr2=0x7fb29810c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.446+0000 7fb29cd9a700 1 --2- 172.21.15.119:0/1081030707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2981053d0 0x7fb29810c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.446+0000 7fb29cd9a700 1 -- 172.21.15.119:0/1081030707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb298100460 msgr2=0x7fb2981a2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.446+0000 7fb29cd9a700 1 --2- 172.21.15.119:0/1081030707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb298100460 0x7fb2981a2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.446+0000 7fb29cd9a700 1 -- 172.21.15.119:0/1081030707 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb294009520 con 0x7fb298100e20 2023-12-07T10:21:20.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.446+0000 7fb29d59b700 1 --2- 172.21.15.119:0/1081030707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb298100460 0x7fb2981a2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:20.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.446+0000 7fb29dd9c700 1 --2- 172.21.15.119:0/1081030707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2981053d0 0x7fb29810c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:21:20.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.447+0000 7fb29cd9a700 1 --2- 172.21.15.119:0/1081030707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb298100e20 0x7fb2981a31c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb28c005910 tx=0x7fb28c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:20.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.447+0000 7fb28a7fc700 1 -- 172.21.15.119:0/1081030707 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c01d070 con 0x7fb298100e20 2023-12-07T10:21:20.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.447+0000 7fb29f7ff700 1 -- 172.21.15.119:0/1081030707 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb28c0095f0 con 0x7fb298100e20 2023-12-07T10:21:20.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.447+0000 7fb28a7fc700 1 -- 172.21.15.119:0/1081030707 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb28c022470 con 0x7fb298100e20 2023-12-07T10:21:20.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.447+0000 7fb28a7fc700 1 -- 172.21.15.119:0/1081030707 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c017650 con 0x7fb298100e20 2023-12-07T10:21:20.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.448+0000 7fb29f7ff700 1 -- 172.21.15.119:0/1081030707 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29810d1c0 con 0x7fb298100e20 2023-12-07T10:21:20.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.451+0000 7fb28a7fc700 1 -- 172.21.15.119:0/1081030707 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 25) v1 ==== 136717+0+0 (secure 0 0 0) 0x7fb28c0225e0 con 0x7fb298100e20 2023-12-07T10:21:20.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.452+0000 7fb28a7fc700 1 --2- 172.21.15.119:0/1081030707 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb2840a1e20 0x7fb2840a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.452+0000 7fb28a7fc700 1 -- 172.21.15.119:0/1081030707 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fb28c0cefb0 con 0x7fb298100e20 2023-12-07T10:21:20.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.455+0000 7fb29d59b700 1 --2- 172.21.15.119:0/1081030707 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb2840a1e20 0x7fb2840a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.455+0000 7fb2b3138b80 1 Processor -- start 2023-12-07T10:21:20.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.455+0000 7fb2b3138b80 1 -- start start 2023-12-07T10:21:20.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.455+0000 7fb29d59b700 1 --2- 172.21.15.119:0/1081030707 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb2840a1e20 0x7fb2840a42e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb294011630 tx=0x7fb294009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:20.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.456+0000 7fb2b3138b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619bae7bb40 0x5619bae7bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.456+0000 7fb2b3138b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619bae7c460 0x5619bae7fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.456+0000 7fb2b3138b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bae80150 0x5619bae80570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.456+0000 7fb2b3138b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619badef7c0 con 0x5619bae80150 2023-12-07T10:21:20.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.456+0000 7fb2b3138b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619badd67a0 con 0x5619bae7bb40 2023-12-07T10:21:20.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.456+0000 7fb2b3138b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619badd40c0 con 0x5619bae7c460 2023-12-07T10:21:20.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.456+0000 7fb29cd9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619bae7c460 0x5619bae7fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.456+0000 7fb29cd9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619bae7c460 0x5619bae7fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:51730/0 (socket says 172.21.15.119:51730) 2023-12-07T10:21:20.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.456+0000 7fb29cd9a700 1 -- 172.21.15.119:0/139709678 learned_addr learned my addr 172.21.15.119:0/139709678 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:20.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.456+0000 7fb29cd9a700 1 -- 172.21.15.119:0/139709678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619bae7bb40 msgr2=0x5619bae7bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.456+0000 7fb29cd9a700 1 --2- 172.21.15.119:0/139709678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619bae7bb40 0x5619bae7bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.456+0000 7fb29cd9a700 1 -- 172.21.15.119:0/139709678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bae80150 msgr2=0x5619bae80570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.456+0000 7fb29dd9c700 1 --2- 172.21.15.119:0/139709678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bae80150 0x5619bae80570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.457+0000 7fb29cd9a700 1 --2- 172.21.15.119:0/139709678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bae80150 0x5619bae80570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.457+0000 7fb29cd9a700 1 -- 172.21.15.119:0/139709678 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619bac4c630 con 0x5619bae7c460 2023-12-07T10:21:20.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.457+0000 7fb29dd9c700 1 --2- 172.21.15.119:0/139709678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bae80150 0x5619bae80570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:21:20.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.457+0000 7fb29cd9a700 1 --2- 172.21.15.119:0/139709678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619bae7c460 0x5619bae7fc10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb28c097d40 tx=0x7fb28c009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5ad02028c0daa5c server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:20.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.457+0000 7fb2837fe700 1 -- 172.21.15.119:0/139709678 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c09b1a0 con 0x5619bae7c460 2023-12-07T10:21:20.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.457+0000 7fb2837fe700 1 -- 172.21.15.119:0/139709678 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb28c09bf40 con 0x5619bae7c460 2023-12-07T10:21:20.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.457+0000 7fb2837fe700 1 -- 172.21.15.119:0/139709678 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c084be0 con 0x5619bae7c460 2023-12-07T10:21:20.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.458+0000 7fb2b3138b80 1 -- 172.21.15.119:0/139709678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619bae7c460 msgr2=0x5619bae7fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.458+0000 7fb2b3138b80 1 --2- 172.21.15.119:0/139709678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619bae7c460 0x5619bae7fc10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb28c097d40 tx=0x7fb28c009ca0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.458+0000 7fb2b3138b80 1 -- 172.21.15.119:0/139709678 shutdown_connections 2023-12-07T10:21:20.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.458+0000 7fb2b3138b80 1 --2- 172.21.15.119:0/139709678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bae80150 0x5619bae80570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.458+0000 7fb2b3138b80 1 --2- 172.21.15.119:0/139709678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619bae7bb40 0x5619bae7bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.458+0000 7fb2b3138b80 1 --2- 172.21.15.119:0/139709678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619bae7c460 0x5619bae7fc10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.458+0000 7fb2b3138b80 1 -- 172.21.15.119:0/139709678 >> 172.21.15.119:0/139709678 conn(0x5619badaa360 msgr2=0x5619bae76130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:20.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.459+0000 7fb2b3138b80 1 -- 172.21.15.119:0/139709678 shutdown_connections 2023-12-07T10:21:20.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.459+0000 7fb2b3138b80 1 -- 172.21.15.119:0/139709678 wait complete. 2023-12-07T10:21:20.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.459+0000 7fb2b3138b80 1 -- start start 2023-12-07T10:21:20.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.460+0000 7fb2b3138b80 1 Processor -- start 2023-12-07T10:21:20.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.460+0000 7fb2b3138b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619bae7bb40 0x5619bae84ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.460+0000 7fb2b3138b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bae7c460 0x5619bae85220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.460+0000 7fb2b3138b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619bae80150 0x5619bae85760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.460+0000 7fb2b3138b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619badcf9b0 con 0x5619bae7c460 2023-12-07T10:21:20.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.460+0000 7fb2b3138b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bad89560 con 0x5619bae80150 2023-12-07T10:21:20.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.460+0000 7fb2b3138b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bad89200 con 0x5619bae7bb40 2023-12-07T10:21:20.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.460+0000 7fb2b3138b80 10 client.0 _ll_register_callbacks cb 0x5619bad70940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:20.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.461+0000 7fb29d59b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619bae7bb40 0x5619bae84ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.461+0000 7fb29d59b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619bae7bb40 0x5619bae84ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:51738/0 (socket says 172.21.15.119:51738) 2023-12-07T10:21:20.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.461+0000 7fb29d59b700 1 -- 172.21.15.119:0/2439358509 learned_addr learned my addr 172.21.15.119:0/2439358509 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:20.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.461+0000 7fb29cd9a700 1 --2- 172.21.15.119:0/2439358509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bae7c460 0x5619bae85220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.461+0000 7fb29d59b700 1 -- 172.21.15.119:0/2439358509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619bae80150 msgr2=0x5619bae85760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-07T10:21:20.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.461+0000 7fb29d59b700 1 --2- 172.21.15.119:0/2439358509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619bae80150 0x5619bae85760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.461+0000 7fb29d59b700 1 -- 172.21.15.119:0/2439358509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bae7c460 msgr2=0x5619bae85220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.461+0000 7fb29d59b700 1 --2- 172.21.15.119:0/2439358509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bae7c460 0x5619bae85220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.461+0000 7fb29d59b700 1 -- 172.21.15.119:0/2439358509 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29810d1c0 con 0x5619bae7bb40 2023-12-07T10:21:20.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.461+0000 7fb29d59b700 1 --2- 172.21.15.119:0/2439358509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619bae7bb40 0x5619bae84ce0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb29401da50 tx=0x7fb294004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c5b71437d64a6c0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:20.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.462+0000 7fb280ff9700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:21:20.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.462+0000 7fb280ff9700 1 -- 172.21.15.119:0/2439358509 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29400ebd0 con 0x5619bae7bb40 2023-12-07T10:21:20.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.462+0000 7fb280ff9700 1 -- 172.21.15.119:0/2439358509 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fb29400ed30 con 0x5619bae7bb40 2023-12-07T10:21:20.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.462+0000 7fb2b3138b80 1 -- 172.21.15.119:0/2439358509 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5619bae6eb00 con 0x5619bae7bb40 2023-12-07T10:21:20.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.462+0000 7fb280ff9700 1 -- 172.21.15.119:0/2439358509 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb294027470 con 0x5619bae7bb40 2023-12-07T10:21:20.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.462+0000 7fb2b3138b80 10 client.25948 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:20.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.462+0000 7fb2b3138b80 1 -- 172.21.15.119:0/2439358509 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5619babf8de0 con 0x5619bae7bb40 2023-12-07T10:21:20.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.463+0000 7fb280ff9700 1 -- 172.21.15.119:0/2439358509 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5619bae6eb00 con 0x5619bae7bb40 2023-12-07T10:21:20.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.463+0000 7fb280ff9700 1 -- 172.21.15.119:0/2439358509 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb29401e680 con 0x5619bae7bb40 2023-12-07T10:21:20.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.463+0000 7fb2b3138b80 10 client.25948 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:20.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.463+0000 7fb2b3138b80 10 client.25948 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-12-07T10:21:20.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.463+0000 7fb2b3138b80 20 client.25948 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:20.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.463+0000 7fb2b3138b80 1 --2- 172.21.15.119:0/2439358509 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x5619bae950e0 0x5619bae97500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.463+0000 7fb2b3138b80 4 client.25948 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:21:20.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.463+0000 7fb2b3138b80 1 -- 172.21.15.119:0/2439358509 --> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619bae4a7a0 con 0x5619bae950e0 2023-12-07T10:21:20.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.464+0000 7fb29cd9a700 1 --2- 172.21.15.119:0/2439358509 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x5619bae950e0 0x5619bae97500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.464+0000 7fb29cd9a700 1 --2- 172.21.15.119:0/2439358509 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x5619bae950e0 0x5619bae97500 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6a5e63660c27a2d6 server_cookie=e72f4de2888b2a3f in_seq=0 out_seq=0 2023-12-07T10:21:20.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.464+0000 7fb280ff9700 10 client.25948 ms_handle_connect on v2:172.21.15.159:6834/2406380792 2023-12-07T10:21:20.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.465+0000 7fb280ff9700 1 -- 172.21.15.119:0/2439358509 <== mds.3 v2:172.21.15.159:6834/2406380792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5619bae4a7a0 con 0x5619bae950e0 2023-12-07T10:21:20.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.465+0000 7fb280ff9700 10 client.25948 handle_command_reply: tid=0 2023-12-07T10:21:20.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.465+0000 7fb2b3138b80 1 client.25948 shutdown 2023-12-07T10:21:20.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.466+0000 7fb2b3138b80 1 -- 172.21.15.119:0/2439358509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619bae7bb40 msgr2=0x5619bae84ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.466+0000 7fb2b3138b80 1 --2- 172.21.15.119:0/2439358509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619bae7bb40 0x5619bae84ce0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb29401da50 tx=0x7fb294004750 comp rx=0 tx=0).stop 2023-12-07T10:21:20.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.466+0000 7fb2b3138b80 1 -- 172.21.15.119:0/2439358509 shutdown_connections 2023-12-07T10:21:20.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.466+0000 7fb2b3138b80 1 --2- 172.21.15.119:0/2439358509 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x5619bae950e0 0x5619bae97500 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.466+0000 7fb2b3138b80 1 --2- 172.21.15.119:0/2439358509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619bae80150 0x5619bae85760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.466+0000 7fb2b3138b80 1 --2- 172.21.15.119:0/2439358509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619bae7bb40 0x5619bae84ce0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.466+0000 7fb2b3138b80 1 --2- 172.21.15.119:0/2439358509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bae7c460 0x5619bae85220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.466+0000 7fb2b3138b80 1 -- 172.21.15.119:0/2439358509 >> 172.21.15.119:0/2439358509 conn(0x5619badaa360 msgr2=0x5619bae81620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:20.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.466+0000 7fb280ff9700 0 client.25948 ms_handle_reset on v2:172.21.15.159:6834/2406380792 2023-12-07T10:21:20.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.466+0000 7fb2b3138b80 1 -- 172.21.15.119:0/2439358509 shutdown_connections 2023-12-07T10:21:20.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.466+0000 7fb2b3138b80 1 -- 172.21.15.119:0/2439358509 wait complete. 2023-12-07T10:21:20.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.466+0000 7fb2b3138b80 20 client.25948 trim_cache size 0 max 16384 2023-12-07T10:21:20.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.480+0000 7fb2b3138b80 1 Processor -- start 2023-12-07T10:21:20.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.481+0000 7fb2b3138b80 1 -- start start 2023-12-07T10:21:20.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.481+0000 7fb2b3138b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bae7bb40 0x5619bae91120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.481+0000 7fb2b3138b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619bae7c460 0x5619bae85580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.481+0000 7fb2b3138b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619bae80150 0x5619bae8c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.481+0000 7fb2b3138b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bac4e920 con 0x5619bae7bb40 2023-12-07T10:21:20.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.481+0000 7fb2b3138b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bae72b60 con 0x5619bae80150 2023-12-07T10:21:20.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.481+0000 7fb2b3138b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bae9c750 con 0x5619bae7c460 2023-12-07T10:21:20.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.481+0000 7fb29cd9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619bae7c460 0x5619bae85580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.481+0000 7fb29cd9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619bae7c460 0x5619bae85580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:51748/0 (socket says 172.21.15.119:51748) 2023-12-07T10:21:20.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.481+0000 7fb29cd9a700 1 -- 172.21.15.119:0/1258613621 learned_addr learned my addr 172.21.15.119:0/1258613621 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:20.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.481+0000 7fb29dd9c700 1 --2- 172.21.15.119:0/1258613621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619bae80150 0x5619bae8c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.482+0000 7fb29d59b700 1 --2- 172.21.15.119:0/1258613621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bae7bb40 0x5619bae91120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.482+0000 7fb29dd9c700 1 -- 172.21.15.119:0/1258613621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619bae7c460 msgr2=0x5619bae85580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.482+0000 7fb29dd9c700 1 --2- 172.21.15.119:0/1258613621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619bae7c460 0x5619bae85580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.482+0000 7fb29dd9c700 1 -- 172.21.15.119:0/1258613621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bae7bb40 msgr2=0x5619bae91120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.482+0000 7fb29dd9c700 1 --2- 172.21.15.119:0/1258613621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bae7bb40 0x5619bae91120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.482+0000 7fb29dd9c700 1 -- 172.21.15.119:0/1258613621 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619babe1c60 con 0x5619bae80150 2023-12-07T10:21:20.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.482+0000 7fb29d59b700 1 --2- 172.21.15.119:0/1258613621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bae7bb40 0x5619bae91120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:21:20.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.483+0000 7fb29dd9c700 1 --2- 172.21.15.119:0/1258613621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619bae80150 0x5619bae8c010 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb290009fd0 tx=0x7fb29000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5388a87293b4a4e1 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:20.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.483+0000 7fb283fff700 1 -- 172.21.15.119:0/1258613621 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb290007c40 con 0x5619bae80150 2023-12-07T10:21:20.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.483+0000 7fb283fff700 1 -- 172.21.15.119:0/1258613621 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb290007da0 con 0x5619bae80150 2023-12-07T10:21:20.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.483+0000 7fb283fff700 1 -- 172.21.15.119:0/1258613621 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb290014740 con 0x5619bae80150 2023-12-07T10:21:20.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.484+0000 7fb2b3138b80 1 -- 172.21.15.119:0/1258613621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619bae80150 msgr2=0x5619bae8c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.484+0000 7fb2b3138b80 1 --2- 172.21.15.119:0/1258613621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619bae80150 0x5619bae8c010 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb290009fd0 tx=0x7fb29000b040 comp rx=0 tx=0).stop 2023-12-07T10:21:20.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.484+0000 7fb2b3138b80 1 -- 172.21.15.119:0/1258613621 shutdown_connections 2023-12-07T10:21:20.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.484+0000 7fb2b3138b80 1 --2- 172.21.15.119:0/1258613621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619bae80150 0x5619bae8c010 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.484+0000 7fb2b3138b80 1 --2- 172.21.15.119:0/1258613621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bae7bb40 0x5619bae91120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.484+0000 7fb2b3138b80 1 --2- 172.21.15.119:0/1258613621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619bae7c460 0x5619bae85580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.484+0000 7fb2b3138b80 1 -- 172.21.15.119:0/1258613621 >> 172.21.15.119:0/1258613621 conn(0x5619badaa360 msgr2=0x5619bae75fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:20.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.484+0000 7fb2b3138b80 1 -- 172.21.15.119:0/1258613621 shutdown_connections 2023-12-07T10:21:20.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.484+0000 7fb2b3138b80 1 -- 172.21.15.119:0/1258613621 wait complete. 2023-12-07T10:21:20.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.485+0000 7fb2b3138b80 1 -- start start 2023-12-07T10:21:20.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.485+0000 7fb2b3138b80 1 Processor -- start 2023-12-07T10:21:20.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.486+0000 7fb2b3138b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bae7bb40 0x5619baee6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.486+0000 7fb2b3138b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619bae7c460 0x5619baee6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.486+0000 7fb2b3138b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619bae80150 0x5619bae8a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.486+0000 7fb2b3138b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bae8a9f0 con 0x5619bae7bb40 2023-12-07T10:21:20.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.486+0000 7fb2b3138b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bae8ab60 con 0x5619bae80150 2023-12-07T10:21:20.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.486+0000 7fb2b3138b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619bae8ad20 con 0x5619bae7c460 2023-12-07T10:21:20.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.486+0000 7fb2b3138b80 10 client.0 _ll_register_callbacks cb 0x5619bad70940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:20.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.486+0000 7fb29d59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bae7bb40 0x5619baee6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.486+0000 7fb29d59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bae7bb40 0x5619baee6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37880/0 (socket says 172.21.15.119:37880) 2023-12-07T10:21:20.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.486+0000 7fb29d59b700 1 -- 172.21.15.119:0/222375789 learned_addr learned my addr 172.21.15.119:0/222375789 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:20.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.487+0000 7fb29cd9a700 1 --2- 172.21.15.119:0/222375789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619bae7c460 0x5619baee6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.487+0000 7fb29d59b700 1 -- 172.21.15.119:0/222375789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619bae7c460 msgr2=0x5619baee6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.487+0000 7fb29d59b700 1 --2- 172.21.15.119:0/222375789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619bae7c460 0x5619baee6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.487+0000 7fb29d59b700 1 -- 172.21.15.119:0/222375789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619bae80150 msgr2=0x5619bae8a2b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-07T10:21:20.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.487+0000 7fb29d59b700 1 --2- 172.21.15.119:0/222375789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619bae80150 0x5619bae8a2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.487+0000 7fb29d59b700 1 -- 172.21.15.119:0/222375789 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb290011040 con 0x5619bae7bb40 2023-12-07T10:21:20.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.487+0000 7fb29d59b700 1 --2- 172.21.15.119:0/222375789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bae7bb40 0x5619baee6a00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb294000c00 tx=0x7fb2940114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b1eaa55c81f133 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:20.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.487+0000 7fb280ff9700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:21:20.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.487+0000 7fb280ff9700 1 -- 172.21.15.119:0/222375789 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294011d80 con 0x5619bae7bb40 2023-12-07T10:21:20.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.487+0000 7fb280ff9700 1 -- 172.21.15.119:0/222375789 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fb29400ebe0 con 0x5619bae7bb40 2023-12-07T10:21:20.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.488+0000 7fb2b3138b80 1 -- 172.21.15.119:0/222375789 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb28c09bf40 con 0x5619bae7bb40 2023-12-07T10:21:20.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.488+0000 7fb280ff9700 1 -- 172.21.15.119:0/222375789 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb29400eda0 con 0x5619bae7bb40 2023-12-07T10:21:20.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.488+0000 7fb2b3138b80 10 client.15951 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:20.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.488+0000 7fb2b3138b80 1 -- 172.21.15.119:0/222375789 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5619bac1f250 con 0x5619bae7bb40 2023-12-07T10:21:20.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.488+0000 7fb280ff9700 1 -- 172.21.15.119:0/222375789 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb28c09bf40 con 0x5619bae7bb40 2023-12-07T10:21:20.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.488+0000 7fb280ff9700 1 -- 172.21.15.119:0/222375789 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb294004850 con 0x5619bae7bb40 2023-12-07T10:21:20.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.488+0000 7fb2b3138b80 10 client.15951 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:20.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.488+0000 7fb2b3138b80 10 client.15951 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-12-07T10:21:20.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.488+0000 7fb2b3138b80 20 client.15951 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:20.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.489+0000 7fb2b3138b80 1 --2- 172.21.15.119:0/222375789 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x5619baefd830 0x5619baeffc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.489+0000 7fb2b3138b80 4 client.15951 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:21:20.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.489+0000 7fb2b3138b80 1 -- 172.21.15.119:0/222375789 --> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5619bad95fd0 con 0x5619baefd830 2023-12-07T10:21:20.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.489+0000 7fb29cd9a700 1 --2- 172.21.15.119:0/222375789 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x5619baefd830 0x5619baeffc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.489+0000 7fb280ff9700 10 client.15951 ms_handle_connect on v2:172.21.15.159:6834/2406380792 2023-12-07T10:21:20.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.489+0000 7fb29cd9a700 1 --2- 172.21.15.119:0/222375789 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x5619baefd830 0x5619baeffc50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2ad58e24297831a4 server_cookie=5ef821e56d13a4da in_seq=0 out_seq=0 2023-12-07T10:21:20.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.490+0000 7fb280ff9700 1 -- 172.21.15.119:0/222375789 <== mds.3 v2:172.21.15.159:6834/2406380792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5619bad95fd0 con 0x5619baefd830 2023-12-07T10:21:20.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.490+0000 7fb280ff9700 10 client.15951 handle_command_reply: tid=0 2023-12-07T10:21:20.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.490+0000 7fb2b3138b80 1 client.15951 shutdown 2023-12-07T10:21:20.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.490+0000 7fb2b3138b80 1 -- 172.21.15.119:0/222375789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bae7bb40 msgr2=0x5619baee6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.490+0000 7fb2b3138b80 1 --2- 172.21.15.119:0/222375789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bae7bb40 0x5619baee6a00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb294000c00 tx=0x7fb2940114b0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.490+0000 7fb2b3138b80 1 -- 172.21.15.119:0/222375789 shutdown_connections 2023-12-07T10:21:20.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.490+0000 7fb2b3138b80 1 --2- 172.21.15.119:0/222375789 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x5619baefd830 0x5619baeffc50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.490+0000 7fb2b3138b80 1 --2- 172.21.15.119:0/222375789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619bae80150 0x5619bae8a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.490+0000 7fb2b3138b80 1 --2- 172.21.15.119:0/222375789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619bae7bb40 0x5619baee6a00 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.490+0000 7fb280ff9700 0 client.15951 ms_handle_reset on v2:172.21.15.159:6834/2406380792 2023-12-07T10:21:20.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.490+0000 7fb2b3138b80 1 --2- 172.21.15.119:0/222375789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619bae7c460 0x5619baee6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.490+0000 7fb2b3138b80 1 -- 172.21.15.119:0/222375789 >> 172.21.15.119:0/222375789 conn(0x5619badaa360 msgr2=0x5619bae855c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:20.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.490+0000 7fb2b3138b80 1 -- 172.21.15.119:0/222375789 shutdown_connections 2023-12-07T10:21:20.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.490+0000 7fb2b3138b80 1 -- 172.21.15.119:0/222375789 wait complete. 2023-12-07T10:21:20.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.491+0000 7fb2b3138b80 20 client.15951 trim_cache size 0 max 16384 2023-12-07T10:21:20.494 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:21:20.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.491+0000 7fb29f7ff700 1 -- 172.21.15.119:0/1081030707 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb2840a1e20 msgr2=0x7fb2840a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.491+0000 7fb29f7ff700 1 --2- 172.21.15.119:0/1081030707 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb2840a1e20 0x7fb2840a42e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb294011630 tx=0x7fb294009260 comp rx=0 tx=0).stop 2023-12-07T10:21:20.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.491+0000 7fb29f7ff700 1 -- 172.21.15.119:0/1081030707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb298100e20 msgr2=0x7fb2981a31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.491+0000 7fb29f7ff700 1 --2- 172.21.15.119:0/1081030707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb298100e20 0x7fb2981a31c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb28c005910 tx=0x7fb28c004730 comp rx=0 tx=0).stop 2023-12-07T10:21:20.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.491+0000 7fb29f7ff700 1 -- 172.21.15.119:0/1081030707 shutdown_connections 2023-12-07T10:21:20.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.491+0000 7fb29f7ff700 1 --2- 172.21.15.119:0/1081030707 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb2840a1e20 0x7fb2840a42e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.491+0000 7fb29f7ff700 1 --2- 172.21.15.119:0/1081030707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2981053d0 0x7fb29810c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.491+0000 7fb29f7ff700 1 --2- 172.21.15.119:0/1081030707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb298100460 0x7fb2981a2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.491+0000 7fb29f7ff700 1 --2- 172.21.15.119:0/1081030707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb298100e20 0x7fb2981a31c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.491+0000 7fb29f7ff700 1 -- 172.21.15.119:0/1081030707 >> 172.21.15.119:0/1081030707 conn(0x7fb298077610 msgr2=0x7fb2980fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:20.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.492+0000 7fb29f7ff700 1 -- 172.21.15.119:0/1081030707 shutdown_connections 2023-12-07T10:21:20.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.492+0000 7fb29f7ff700 1 -- 172.21.15.119:0/1081030707 wait complete. 2023-12-07T10:21:20.519 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-07T10:21:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.566+0000 7f51fb7ff700 1 -- 172.21.15.119:0/1898453401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51f41124f0 msgr2=0x7f51f41148e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.566+0000 7f51fb7ff700 1 --2- 172.21.15.119:0/1898453401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51f41124f0 0x7f51f41148e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00b0a0 tx=0x7f51ec00b3b0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.566+0000 7f51fb7ff700 1 -- 172.21.15.119:0/1898453401 shutdown_connections 2023-12-07T10:21:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.566+0000 7f51fb7ff700 1 --2- 172.21.15.119:0/1898453401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51f41124f0 0x7f51f41148e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.566+0000 7f51fb7ff700 1 --2- 172.21.15.119:0/1898453401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51f40739b0 0x7f51f4073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.566+0000 7f51fb7ff700 1 --2- 172.21.15.119:0/1898453401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51f40742d0 0x7f51f4111fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.566+0000 7f51fb7ff700 1 -- 172.21.15.119:0/1898453401 >> 172.21.15.119:0/1898453401 conn(0x7f51f40fcfb0 msgr2=0x7f51f40ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.566+0000 7f51fb7ff700 1 -- 172.21.15.119:0/1898453401 shutdown_connections 2023-12-07T10:21:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.566+0000 7f51fb7ff700 1 -- 172.21.15.119:0/1898453401 wait complete. 2023-12-07T10:21:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.568+0000 7f51fb7ff700 1 Processor -- start 2023-12-07T10:21:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.568+0000 7f51fb7ff700 1 -- start start 2023-12-07T10:21:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.568+0000 7f51fb7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51f40739b0 0x7f51f419e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.568+0000 7f51fb7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51f40742d0 0x7f51f419e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.568+0000 7f51fb7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51f41124f0 0x7f51f41a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.568+0000 7f51fb7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f41a3030 con 0x7f51f40739b0 2023-12-07T10:21:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.568+0000 7f51fb7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f41a31a0 con 0x7f51f40742d0 2023-12-07T10:21:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.568+0000 7f51fb7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f41a3360 con 0x7f51f41124f0 2023-12-07T10:21:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.569+0000 7f51f959b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51f40739b0 0x7f51f419e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.569+0000 7f51f959b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51f40739b0 0x7f51f419e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37890/0 (socket says 172.21.15.119:37890) 2023-12-07T10:21:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.569+0000 7f51f959b700 1 -- 172.21.15.119:0/2627770465 learned_addr learned my addr 172.21.15.119:0/2627770465 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.569+0000 7f51f9d9c700 1 --2- 172.21.15.119:0/2627770465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51f41124f0 0x7f51f41a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.569+0000 7f51f8d9a700 1 --2- 172.21.15.119:0/2627770465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51f40742d0 0x7f51f419e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.569+0000 7f51f9d9c700 1 -- 172.21.15.119:0/2627770465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51f40742d0 msgr2=0x7f51f419e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.569+0000 7f51f9d9c700 1 --2- 172.21.15.119:0/2627770465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51f40742d0 0x7f51f419e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.569+0000 7f51f9d9c700 1 -- 172.21.15.119:0/2627770465 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51f40739b0 msgr2=0x7f51f419e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.569+0000 7f51f9d9c700 1 --2- 172.21.15.119:0/2627770465 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51f40739b0 0x7f51f419e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.569+0000 7f51f9d9c700 1 -- 172.21.15.119:0/2627770465 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec009d00 con 0x7f51f41124f0 2023-12-07T10:21:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.570+0000 7f51f8d9a700 1 --2- 172.21.15.119:0/2627770465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51f40742d0 0x7f51f419e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:21:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.570+0000 7f51f9d9c700 1 --2- 172.21.15.119:0/2627770465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51f41124f0 0x7f51f41a29f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00b070 tx=0x7f51ec007930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.570+0000 7f51ea7fc700 1 -- 172.21.15.119:0/2627770465 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec00e040 con 0x7f51f41124f0 2023-12-07T10:21:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.570+0000 7f51ea7fc700 1 -- 172.21.15.119:0/2627770465 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51ec004c50 con 0x7f51f41124f0 2023-12-07T10:21:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.570+0000 7f51ea7fc700 1 -- 172.21.15.119:0/2627770465 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec005040 con 0x7f51f41124f0 2023-12-07T10:21:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.570+0000 7f51fb7ff700 1 -- 172.21.15.119:0/2627770465 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f41a35e0 con 0x7f51f41124f0 2023-12-07T10:21:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.570+0000 7f51fb7ff700 1 -- 172.21.15.119:0/2627770465 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f41aae80 con 0x7f51f41124f0 2023-12-07T10:21:20.576 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.574+0000 7f51ea7fc700 1 -- 172.21.15.119:0/2627770465 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) v1 ==== 136717+0+0 (secure 0 0 0) 0x7f51ec01d470 con 0x7f51f41124f0 2023-12-07T10:21:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.576+0000 7f51ea7fc700 1 --2- 172.21.15.119:0/2627770465 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f51e00a1e20 0x7f51e00a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.576+0000 7f520f08db80 1 Processor -- start 2023-12-07T10:21:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.576+0000 7f51ea7fc700 1 -- 172.21.15.119:0/2627770465 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f51ec0cf390 con 0x7f51f41124f0 2023-12-07T10:21:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.577+0000 7f520f08db80 1 -- start start 2023-12-07T10:21:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.577+0000 7f51f959b700 1 --2- 172.21.15.119:0/2627770465 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f51e00a1e20 0x7f51e00a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.577+0000 7f520f08db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cd9ebaa60 0x558cd9ebae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.577+0000 7f520f08db80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558cd9ebb380 0x558cd9eb5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.577+0000 7f520f08db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd9eb5ad0 0x558cd9eb5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.577+0000 7f520f08db80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd9e29760 con 0x558cd9ebb380 2023-12-07T10:21:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.577+0000 7f520f08db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd9e10740 con 0x558cd9ebaa60 2023-12-07T10:21:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.577+0000 7f520f08db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd9e0e060 con 0x558cd9eb5ad0 2023-12-07T10:21:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.578+0000 7f51f8d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558cd9ebb380 0x558cd9eb5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.578+0000 7f51f8d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558cd9ebb380 0x558cd9eb5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37898/0 (socket says 172.21.15.119:37898) 2023-12-07T10:21:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.578+0000 7f51f8d9a700 1 -- 172.21.15.119:0/506131518 learned_addr learned my addr 172.21.15.119:0/506131518 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.578+0000 7f51f9d9c700 1 --2- 172.21.15.119:0/506131518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd9eb5ad0 0x558cd9eb5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.578+0000 7f51f8d9a700 1 -- 172.21.15.119:0/506131518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd9eb5ad0 msgr2=0x558cd9eb5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.578+0000 7f51f8d9a700 1 --2- 172.21.15.119:0/506131518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd9eb5ad0 0x558cd9eb5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.578+0000 7f51f8d9a700 1 -- 172.21.15.119:0/506131518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cd9ebaa60 msgr2=0x558cd9ebae40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-07T10:21:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.578+0000 7f51f8d9a700 1 --2- 172.21.15.119:0/506131518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cd9ebaa60 0x558cd9ebae40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.578+0000 7f51f8d9a700 1 -- 172.21.15.119:0/506131518 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cd9c865d0 con 0x558cd9ebb380 2023-12-07T10:21:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.579+0000 7f51f9d9c700 1 --2- 172.21.15.119:0/506131518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd9eb5ad0 0x558cd9eb5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:21:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.579+0000 7f51f959b700 1 --2- 172.21.15.119:0/2627770465 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f51e00a1e20 0x7f51e00a42e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f51f0007ce0 tx=0x7f51f0007480 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.579+0000 7f51f8d9a700 1 --2- 172.21.15.119:0/506131518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558cd9ebb380 0x558cd9eb5540 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009d60 tx=0x7f51e40106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d879d190e97e63a server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.579+0000 7f51df7fe700 1 -- 172.21.15.119:0/506131518 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e4010ec0 con 0x558cd9ebb380 2023-12-07T10:21:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.579+0000 7f51df7fe700 1 -- 172.21.15.119:0/506131518 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51e4015070 con 0x558cd9ebb380 2023-12-07T10:21:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.579+0000 7f51df7fe700 1 -- 172.21.15.119:0/506131518 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e40117a0 con 0x558cd9ebb380 2023-12-07T10:21:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.580+0000 7f520f08db80 1 -- 172.21.15.119:0/506131518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558cd9ebb380 msgr2=0x558cd9eb5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.580+0000 7f520f08db80 1 --2- 172.21.15.119:0/506131518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558cd9ebb380 0x558cd9eb5540 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009d60 tx=0x7f51e40106b0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.580+0000 7f520f08db80 1 -- 172.21.15.119:0/506131518 shutdown_connections 2023-12-07T10:21:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.580+0000 7f520f08db80 1 --2- 172.21.15.119:0/506131518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd9eb5ad0 0x558cd9eb5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.580+0000 7f520f08db80 1 --2- 172.21.15.119:0/506131518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cd9ebaa60 0x558cd9ebae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.580+0000 7f520f08db80 1 --2- 172.21.15.119:0/506131518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558cd9ebb380 0x558cd9eb5540 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.580+0000 7f520f08db80 1 -- 172.21.15.119:0/506131518 >> 172.21.15.119:0/506131518 conn(0x558cd9de4300 msgr2=0x558cd9eafb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.581+0000 7f520f08db80 1 -- 172.21.15.119:0/506131518 shutdown_connections 2023-12-07T10:21:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.581+0000 7f520f08db80 1 -- 172.21.15.119:0/506131518 wait complete. 2023-12-07T10:21:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.581+0000 7f520f08db80 1 -- start start 2023-12-07T10:21:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.582+0000 7f520f08db80 1 Processor -- start 2023-12-07T10:21:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.582+0000 7f520f08db80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558cd9eb5ad0 0x558cd9ec1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.582+0000 7f520f08db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cd9ebaa60 0x558cd9ebe710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.583+0000 7f520f08db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd9ebb380 0x558cd9ebee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.583+0000 7f520f08db80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd9e09950 con 0x558cd9eb5ad0 2023-12-07T10:21:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.583+0000 7f520f08db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd9dc3500 con 0x558cd9ebaa60 2023-12-07T10:21:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.583+0000 7f520f08db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd9dc31a0 con 0x558cd9ebb380 2023-12-07T10:21:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.583+0000 7f520f08db80 10 client.0 _ll_register_callbacks cb 0x558cd9daa8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.583+0000 7f51f8d9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cd9ebaa60 0x558cd9ebe710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.583+0000 7f51f8d9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cd9ebaa60 0x558cd9ebe710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:47642/0 (socket says 172.21.15.119:47642) 2023-12-07T10:21:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.583+0000 7f51f8d9a700 1 -- 172.21.15.119:0/328515071 learned_addr learned my addr 172.21.15.119:0/328515071 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.583+0000 7f51f8d9a700 1 -- 172.21.15.119:0/328515071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd9ebb380 msgr2=0x558cd9ebee50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-07T10:21:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.583+0000 7f51f8d9a700 1 --2- 172.21.15.119:0/328515071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd9ebb380 0x558cd9ebee50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.583+0000 7f51f8d9a700 1 -- 172.21.15.119:0/328515071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558cd9eb5ad0 msgr2=0x558cd9ec1390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-07T10:21:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.584+0000 7f51f959b700 1 --2- 172.21.15.119:0/328515071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558cd9eb5ad0 0x558cd9ec1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.584+0000 7f51f8d9a700 1 --2- 172.21.15.119:0/328515071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558cd9eb5ad0 0x558cd9ec1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.584+0000 7f51f8d9a700 1 -- 172.21.15.119:0/328515071 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e4009990 con 0x558cd9ebaa60 2023-12-07T10:21:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.584+0000 7f51f959b700 1 --2- 172.21.15.119:0/328515071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558cd9eb5ad0 0x558cd9ec1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:21:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.584+0000 7f51f8d9a700 1 --2- 172.21.15.119:0/328515071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cd9ebaa60 0x558cd9ebe710 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009d30 tx=0x7f51e401c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e4efad62d32b36f server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.584+0000 7f51dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:21:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.584+0000 7f51dcff9700 1 -- 172.21.15.119:0/328515071 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e4020070 con 0x558cd9ebaa60 2023-12-07T10:21:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.585+0000 7f51dcff9700 1 -- 172.21.15.119:0/328515071 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f51e4017070 con 0x558cd9ebaa60 2023-12-07T10:21:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.585+0000 7f520f08db80 1 -- 172.21.15.119:0/328515071 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558cd9ea86f0 con 0x558cd9ebaa60 2023-12-07T10:21:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.585+0000 7f51dcff9700 1 -- 172.21.15.119:0/328515071 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51e4009eb0 con 0x558cd9ebaa60 2023-12-07T10:21:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.585+0000 7f520f08db80 10 client.25973 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.585+0000 7f520f08db80 1 -- 172.21.15.119:0/328515071 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558cd9c32d80 con 0x558cd9ebaa60 2023-12-07T10:21:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.586+0000 7f51dcff9700 1 -- 172.21.15.119:0/328515071 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558cd9ea86f0 con 0x558cd9ebaa60 2023-12-07T10:21:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.586+0000 7f51dcff9700 1 -- 172.21.15.119:0/328515071 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f51e4011440 con 0x558cd9ebaa60 2023-12-07T10:21:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.586+0000 7f520f08db80 10 client.25973 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.586+0000 7f520f08db80 10 client.25973 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:21:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.586+0000 7f520f08db80 20 client.25973 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.586+0000 7f520f08db80 1 --2- 172.21.15.119:0/328515071 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558cd9edfc70 0x558cd9ee2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.586+0000 7f520f08db80 4 client.25973 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:21:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.586+0000 7f520f08db80 1 -- 172.21.15.119:0/328515071 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558cd9e84740 con 0x558cd9edfc70 2023-12-07T10:21:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.587+0000 7f51f959b700 1 --2- 172.21.15.119:0/328515071 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558cd9edfc70 0x558cd9ee2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.587+0000 7f51f959b700 1 --2- 172.21.15.119:0/328515071 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558cd9edfc70 0x558cd9ee2090 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cb30b792aa13f2f server_cookie=67653ca4b14d675a in_seq=0 out_seq=0 2023-12-07T10:21:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.587+0000 7f51dcff9700 10 client.25973 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.588+0000 7f51dcff9700 1 -- 172.21.15.119:0/328515071 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558cd9e84740 con 0x558cd9edfc70 2023-12-07T10:21:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.588+0000 7f51dcff9700 10 client.25973 handle_command_reply: tid=0 2023-12-07T10:21:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.588+0000 7f520f08db80 1 client.25973 shutdown 2023-12-07T10:21:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.589+0000 7f520f08db80 1 -- 172.21.15.119:0/328515071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cd9ebaa60 msgr2=0x558cd9ebe710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.589+0000 7f520f08db80 1 --2- 172.21.15.119:0/328515071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cd9ebaa60 0x558cd9ebe710 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009d30 tx=0x7f51e401c760 comp rx=0 tx=0).stop 2023-12-07T10:21:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.589+0000 7f520f08db80 1 -- 172.21.15.119:0/328515071 shutdown_connections 2023-12-07T10:21:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.589+0000 7f520f08db80 1 --2- 172.21.15.119:0/328515071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd9ebb380 0x558cd9ebee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.589+0000 7f520f08db80 1 --2- 172.21.15.119:0/328515071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558cd9eb5ad0 0x558cd9ec1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.589+0000 7f520f08db80 1 --2- 172.21.15.119:0/328515071 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558cd9edfc70 0x558cd9ee2090 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.589+0000 7f520f08db80 1 --2- 172.21.15.119:0/328515071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cd9ebaa60 0x558cd9ebe710 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.589+0000 7f520f08db80 1 -- 172.21.15.119:0/328515071 >> 172.21.15.119:0/328515071 conn(0x558cd9de4300 msgr2=0x558cd9eaa980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.589+0000 7f51dcff9700 0 client.25973 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.589+0000 7f520f08db80 1 -- 172.21.15.119:0/328515071 shutdown_connections 2023-12-07T10:21:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.589+0000 7f520f08db80 1 -- 172.21.15.119:0/328515071 wait complete. 2023-12-07T10:21:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.590+0000 7f520f08db80 20 client.25973 trim_cache size 0 max 16384 2023-12-07T10:21:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.618+0000 7f520f08db80 1 Processor -- start 2023-12-07T10:21:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.618+0000 7f520f08db80 1 -- start start 2023-12-07T10:21:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.618+0000 7f520f08db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cd9eb5ad0 0x558cd9edbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.618+0000 7f520f08db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd9ebaa60 0x558cd9eb5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.618+0000 7f520f08db80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558cd9ebb380 0x558cd9ebe030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.618+0000 7f520f08db80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd9c888c0 con 0x558cd9ebb380 2023-12-07T10:21:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.618+0000 7f520f08db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd9ea86c0 con 0x558cd9eb5ad0 2023-12-07T10:21:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.618+0000 7f520f08db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd9eb14a0 con 0x558cd9ebaa60 2023-12-07T10:21:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.619+0000 7f51f959b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cd9eb5ad0 0x558cd9edbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.619+0000 7f51f959b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cd9eb5ad0 0x558cd9edbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:47646/0 (socket says 172.21.15.119:47646) 2023-12-07T10:21:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.619+0000 7f51f959b700 1 -- 172.21.15.119:0/624601228 learned_addr learned my addr 172.21.15.119:0/624601228 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.619+0000 7f51f8d9a700 1 --2- 172.21.15.119:0/624601228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd9ebaa60 0x558cd9eb5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.619+0000 7f51f9d9c700 1 --2- 172.21.15.119:0/624601228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558cd9ebb380 0x558cd9ebe030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.619+0000 7f51f959b700 1 -- 172.21.15.119:0/624601228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd9ebaa60 msgr2=0x558cd9eb5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.619+0000 7f51f959b700 1 --2- 172.21.15.119:0/624601228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd9ebaa60 0x558cd9eb5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.619+0000 7f51f959b700 1 -- 172.21.15.119:0/624601228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558cd9ebb380 msgr2=0x558cd9ebe030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.619+0000 7f51f959b700 1 --2- 172.21.15.119:0/624601228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558cd9ebb380 0x558cd9ebe030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.619+0000 7f51f959b700 1 -- 172.21.15.119:0/624601228 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cd9c1bc00 con 0x558cd9eb5ad0 2023-12-07T10:21:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.620+0000 7f51f959b700 1 --2- 172.21.15.119:0/624601228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cd9eb5ad0 0x558cd9edbec0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f51f0010e70 tx=0x7f51f0013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99d61acc879ccdf4 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.620+0000 7f51dffff700 1 -- 172.21.15.119:0/624601228 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f0023040 con 0x558cd9eb5ad0 2023-12-07T10:21:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.620+0000 7f51dffff700 1 -- 172.21.15.119:0/624601228 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51f0015070 con 0x558cd9eb5ad0 2023-12-07T10:21:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.620+0000 7f51dffff700 1 -- 172.21.15.119:0/624601228 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f001e680 con 0x558cd9eb5ad0 2023-12-07T10:21:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.621+0000 7f520f08db80 1 -- 172.21.15.119:0/624601228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cd9eb5ad0 msgr2=0x558cd9edbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.621+0000 7f520f08db80 1 --2- 172.21.15.119:0/624601228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cd9eb5ad0 0x558cd9edbec0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f51f0010e70 tx=0x7f51f0013000 comp rx=0 tx=0).stop 2023-12-07T10:21:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.622+0000 7f520f08db80 1 -- 172.21.15.119:0/624601228 shutdown_connections 2023-12-07T10:21:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.622+0000 7f520f08db80 1 --2- 172.21.15.119:0/624601228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558cd9ebb380 0x558cd9ebe030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.622+0000 7f520f08db80 1 --2- 172.21.15.119:0/624601228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cd9eb5ad0 0x558cd9edbec0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.622+0000 7f520f08db80 1 --2- 172.21.15.119:0/624601228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd9ebaa60 0x558cd9eb5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.622+0000 7f520f08db80 1 -- 172.21.15.119:0/624601228 >> 172.21.15.119:0/624601228 conn(0x558cd9de4300 msgr2=0x558cd9eafb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.622+0000 7f520f08db80 1 -- 172.21.15.119:0/624601228 shutdown_connections 2023-12-07T10:21:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.622+0000 7f520f08db80 1 -- 172.21.15.119:0/624601228 wait complete. 2023-12-07T10:21:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.622+0000 7f520f08db80 1 -- start start 2023-12-07T10:21:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.623+0000 7f520f08db80 1 Processor -- start 2023-12-07T10:21:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.623+0000 7f520f08db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd9eb5ad0 0x558cd9ec3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.623+0000 7f520f08db80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558cd9ebaa60 0x558cd9ec4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.623+0000 7f520f08db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cd9ebb380 0x558cd9ec4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.623+0000 7f520f08db80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd9f35190 con 0x558cd9ebaa60 2023-12-07T10:21:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.623+0000 7f520f08db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd9f352d0 con 0x558cd9ebb380 2023-12-07T10:21:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.623+0000 7f520f08db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd9f35410 con 0x558cd9eb5ad0 2023-12-07T10:21:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.623+0000 7f520f08db80 10 client.0 _ll_register_callbacks cb 0x558cd9daa8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.624+0000 7f51f8d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558cd9ebaa60 0x558cd9ec4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.624+0000 7f51f959b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd9eb5ad0 0x558cd9ec3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.624+0000 7f51f8d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558cd9ebaa60 0x558cd9ec4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37914/0 (socket says 172.21.15.119:37914) 2023-12-07T10:21:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.624+0000 7f51f8d9a700 1 -- 172.21.15.119:0/3324044637 learned_addr learned my addr 172.21.15.119:0/3324044637 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.624+0000 7f51f9d9c700 1 --2- 172.21.15.119:0/3324044637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cd9ebb380 0x558cd9ec4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.625+0000 7f51f8d9a700 1 -- 172.21.15.119:0/3324044637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd9eb5ad0 msgr2=0x558cd9ec3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.625+0000 7f51f8d9a700 1 --2- 172.21.15.119:0/3324044637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd9eb5ad0 0x558cd9ec3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.625+0000 7f51f8d9a700 1 -- 172.21.15.119:0/3324044637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cd9ebb380 msgr2=0x558cd9ec4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.625+0000 7f51f8d9a700 1 --2- 172.21.15.119:0/3324044637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cd9ebb380 0x558cd9ec4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.625+0000 7f51f8d9a700 1 -- 172.21.15.119:0/3324044637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f0010ce0 con 0x558cd9ebaa60 2023-12-07T10:21:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.625+0000 7f51f9d9c700 1 --2- 172.21.15.119:0/3324044637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cd9ebb380 0x558cd9ec4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.625+0000 7f51f8d9a700 1 --2- 172.21.15.119:0/3324044637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558cd9ebaa60 0x558cd9ec4320 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f51e4010b30 tx=0x7f51e401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33181f821d43eaf server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.625+0000 7f51dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:21:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.625+0000 7f51dcff9700 1 -- 172.21.15.119:0/3324044637 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e4020070 con 0x558cd9ebaa60 2023-12-07T10:21:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.626+0000 7f51dcff9700 1 -- 172.21.15.119:0/3324044637 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f51e4017040 con 0x558cd9ebaa60 2023-12-07T10:21:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.626+0000 7f520f08db80 1 -- 172.21.15.119:0/3324044637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51e4015070 con 0x558cd9ebaa60 2023-12-07T10:21:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.626+0000 7f51dcff9700 1 -- 172.21.15.119:0/3324044637 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51e401cd80 con 0x558cd9ebaa60 2023-12-07T10:21:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.626+0000 7f520f08db80 10 client.15966 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.626+0000 7f520f08db80 1 -- 172.21.15.119:0/3324044637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558cd9c591f0 con 0x558cd9ebaa60 2023-12-07T10:21:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.626+0000 7f51dcff9700 1 -- 172.21.15.119:0/3324044637 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51e4015070 con 0x558cd9ebaa60 2023-12-07T10:21:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.627+0000 7f51dcff9700 1 -- 172.21.15.119:0/3324044637 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f51e4003b40 con 0x558cd9ebaa60 2023-12-07T10:21:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.627+0000 7f520f08db80 10 client.15966 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.627+0000 7f520f08db80 10 client.15966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:21:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.627+0000 7f520f08db80 20 client.15966 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.627+0000 7f520f08db80 1 --2- 172.21.15.119:0/3324044637 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558cd9f36950 0x558cd9f38d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.627+0000 7f520f08db80 4 client.15966 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:21:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.627+0000 7f520f08db80 1 -- 172.21.15.119:0/3324044637 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558cd9dcff70 con 0x558cd9f36950 2023-12-07T10:21:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.627+0000 7f51f959b700 1 --2- 172.21.15.119:0/3324044637 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558cd9f36950 0x558cd9f38d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.628+0000 7f51f959b700 1 --2- 172.21.15.119:0/3324044637 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558cd9f36950 0x558cd9f38d70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f6dce40c64a4a46 server_cookie=8ed11eff4b521265 in_seq=0 out_seq=0 2023-12-07T10:21:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.628+0000 7f51dcff9700 10 client.15966 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.636+0000 7f51dcff9700 1 -- 172.21.15.119:0/3324044637 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558cd9dcff70 con 0x558cd9f36950 2023-12-07T10:21:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.636+0000 7f51dcff9700 10 client.15966 handle_command_reply: tid=0 2023-12-07T10:21:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.636+0000 7f520f08db80 1 client.15966 shutdown 2023-12-07T10:21:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.636+0000 7f520f08db80 1 -- 172.21.15.119:0/3324044637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558cd9ebaa60 msgr2=0x558cd9ec4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.636+0000 7f520f08db80 1 --2- 172.21.15.119:0/3324044637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558cd9ebaa60 0x558cd9ec4320 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f51e4010b30 tx=0x7f51e401cab0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.636+0000 7f520f08db80 1 -- 172.21.15.119:0/3324044637 shutdown_connections 2023-12-07T10:21:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.636+0000 7f520f08db80 1 --2- 172.21.15.119:0/3324044637 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558cd9f36950 0x558cd9f38d70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.637+0000 7f520f08db80 1 --2- 172.21.15.119:0/3324044637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558cd9ebb380 0x558cd9ec4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.637+0000 7f520f08db80 1 --2- 172.21.15.119:0/3324044637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cd9eb5ad0 0x558cd9ec3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.637+0000 7f520f08db80 1 --2- 172.21.15.119:0/3324044637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558cd9ebaa60 0x558cd9ec4320 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.637+0000 7f520f08db80 1 -- 172.21.15.119:0/3324044637 >> 172.21.15.119:0/3324044637 conn(0x558cd9de4300 msgr2=0x558cd9eb8960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.637+0000 7f51dcff9700 0 client.15966 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.637+0000 7f520f08db80 1 -- 172.21.15.119:0/3324044637 shutdown_connections 2023-12-07T10:21:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.637+0000 7f520f08db80 1 -- 172.21.15.119:0/3324044637 wait complete. 2023-12-07T10:21:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.637+0000 7f520f08db80 20 client.15966 trim_cache size 0 max 16384 2023-12-07T10:21:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:21:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:21:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:21:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:21:20.641 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.638+0000 7f51fb7ff700 1 -- 172.21.15.119:0/2627770465 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f51e00a1e20 msgr2=0x7f51e00a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.638+0000 7f51fb7ff700 1 --2- 172.21.15.119:0/2627770465 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f51e00a1e20 0x7f51e00a42e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f51f0007ce0 tx=0x7f51f0007480 comp rx=0 tx=0).stop 2023-12-07T10:21:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.638+0000 7f51fb7ff700 1 -- 172.21.15.119:0/2627770465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51f41124f0 msgr2=0x7f51f41a29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.638+0000 7f51fb7ff700 1 --2- 172.21.15.119:0/2627770465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51f41124f0 0x7f51f41a29f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00b070 tx=0x7f51ec007930 comp rx=0 tx=0).stop 2023-12-07T10:21:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.638+0000 7f51fb7ff700 1 -- 172.21.15.119:0/2627770465 shutdown_connections 2023-12-07T10:21:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.638+0000 7f51fb7ff700 1 --2- 172.21.15.119:0/2627770465 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f51e00a1e20 0x7f51e00a42e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.638+0000 7f51fb7ff700 1 --2- 172.21.15.119:0/2627770465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51f41124f0 0x7f51f41a29f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.638+0000 7f51fb7ff700 1 --2- 172.21.15.119:0/2627770465 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51f40739b0 0x7f51f419e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.638+0000 7f51fb7ff700 1 --2- 172.21.15.119:0/2627770465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51f40742d0 0x7f51f419e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.638+0000 7f51fb7ff700 1 -- 172.21.15.119:0/2627770465 >> 172.21.15.119:0/2627770465 conn(0x7f51f40fcfb0 msgr2=0x7f51f40fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.638+0000 7f51fb7ff700 1 -- 172.21.15.119:0/2627770465 shutdown_connections 2023-12-07T10:21:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:20.638+0000 7f51fb7ff700 1 -- 172.21.15.119:0/2627770465 wait complete. 2023-12-07T10:21:20.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.770+0000 7f96ae7ff700 1 -- 172.21.15.119:0/113275726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96a8107620 msgr2=0x7f96a8107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.770+0000 7f96ae7ff700 1 --2- 172.21.15.119:0/113275726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96a8107620 0x7f96a8107a00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9698009920 tx=0x7f9698009c30 comp rx=0 tx=0).stop 2023-12-07T10:21:20.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.770+0000 7f96ae7ff700 1 -- 172.21.15.119:0/113275726 shutdown_connections 2023-12-07T10:21:20.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.770+0000 7f96ae7ff700 1 --2- 172.21.15.119:0/113275726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96a8101f20 0x7f96a810e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.770+0000 7f96ae7ff700 1 --2- 172.21.15.119:0/113275726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96a8107620 0x7f96a8107a00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.770+0000 7f96ae7ff700 1 --2- 172.21.15.119:0/113275726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96a8101560 0x7f96a81019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.770+0000 7f96ae7ff700 1 -- 172.21.15.119:0/113275726 >> 172.21.15.119:0/113275726 conn(0x7f96a80fcfd0 msgr2=0x7f96a80ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:20.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.770+0000 7f96ae7ff700 1 -- 172.21.15.119:0/113275726 shutdown_connections 2023-12-07T10:21:20.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.770+0000 7f96ae7ff700 1 -- 172.21.15.119:0/113275726 wait complete. 2023-12-07T10:21:20.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.772+0000 7f96ae7ff700 1 Processor -- start 2023-12-07T10:21:20.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.772+0000 7f96ae7ff700 1 -- start start 2023-12-07T10:21:20.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.772+0000 7f96ae7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96a8101560 0x7f96a8073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.772+0000 7f96ae7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96a8101f20 0x7f96a8073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.772+0000 7f96ae7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96a8107620 0x7f96a80743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.772+0000 7f96ae7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a81a5ef0 con 0x7f96a8101f20 2023-12-07T10:21:20.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.772+0000 7f96ae7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a81a6060 con 0x7f96a8101560 2023-12-07T10:21:20.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.772+0000 7f96ae7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a81a6220 con 0x7f96a8107620 2023-12-07T10:21:20.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.773+0000 7f96a7fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96a8101560 0x7f96a8073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.773+0000 7f96a77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96a8101f20 0x7f96a8073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.773+0000 7f96a7fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96a8101560 0x7f96a8073910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:47674/0 (socket says 172.21.15.119:47674) 2023-12-07T10:21:20.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.773+0000 7f96a7fff700 1 -- 172.21.15.119:0/3523287098 learned_addr learned my addr 172.21.15.119:0/3523287098 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:20.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.773+0000 7f96acd9c700 1 --2- 172.21.15.119:0/3523287098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96a8107620 0x7f96a80743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.773+0000 7f96a7fff700 1 -- 172.21.15.119:0/3523287098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96a8107620 msgr2=0x7f96a80743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.773+0000 7f96a7fff700 1 --2- 172.21.15.119:0/3523287098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96a8107620 0x7f96a80743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.773+0000 7f96a7fff700 1 -- 172.21.15.119:0/3523287098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96a8101f20 msgr2=0x7f96a8073e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.773+0000 7f96a7fff700 1 --2- 172.21.15.119:0/3523287098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96a8101f20 0x7f96a8073e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.773+0000 7f96a7fff700 1 -- 172.21.15.119:0/3523287098 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9698009520 con 0x7f96a8101560 2023-12-07T10:21:20.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.773+0000 7f96a7fff700 1 --2- 172.21.15.119:0/3523287098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96a8101560 0x7f96a8073910 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9698000c00 tx=0x7f969800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:20.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.774+0000 7f96a57fa700 1 -- 172.21.15.119:0/3523287098 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969801c070 con 0x7f96a8101560 2023-12-07T10:21:20.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.774+0000 7f96a57fa700 1 -- 172.21.15.119:0/3523287098 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f969800ba20 con 0x7f96a8101560 2023-12-07T10:21:20.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.774+0000 7f96a57fa700 1 -- 172.21.15.119:0/3523287098 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96980038e0 con 0x7f96a8101560 2023-12-07T10:21:20.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.774+0000 7f96ae7ff700 1 -- 172.21.15.119:0/3523287098 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a81a64a0 con 0x7f96a8101560 2023-12-07T10:21:20.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.774+0000 7f96ae7ff700 1 -- 172.21.15.119:0/3523287098 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a81a6810 con 0x7f96a8101560 2023-12-07T10:21:20.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.779+0000 7f96a57fa700 1 -- 172.21.15.119:0/3523287098 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 25) v1 ==== 136717+0+0 (secure 0 0 0) 0x7f9698020070 con 0x7f96a8101560 2023-12-07T10:21:20.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.779+0000 7f96c221eb80 1 Processor -- start 2023-12-07T10:21:20.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.779+0000 7f96c221eb80 1 -- start start 2023-12-07T10:21:20.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.780+0000 7f96c221eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555904ef2b40 0x555904ef2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.780+0000 7f96c221eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555904ef3460 0x555904ef6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.780+0000 7f96c221eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555904ef7150 0x555904ef7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.780+0000 7f96c221eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555904e667c0 con 0x555904ef2b40 2023-12-07T10:21:20.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.780+0000 7f96c221eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555904e4d7a0 con 0x555904ef7150 2023-12-07T10:21:20.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.780+0000 7f96c221eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555904e4b0c0 con 0x555904ef3460 2023-12-07T10:21:20.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.780+0000 7f96a7fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555904ef3460 0x555904ef6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.780+0000 7f96a77fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555904ef7150 0x555904ef7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.780+0000 7f96acd9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555904ef2b40 0x555904ef2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.780+0000 7f96a77fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555904ef7150 0x555904ef7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:47682/0 (socket says 172.21.15.119:47682) 2023-12-07T10:21:20.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.780+0000 7f96acd9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555904ef2b40 0x555904ef2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37928/0 (socket says 172.21.15.119:37928) 2023-12-07T10:21:20.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.780+0000 7f96a77fe700 1 -- 172.21.15.119:0/1016037463 learned_addr learned my addr 172.21.15.119:0/1016037463 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:20.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.781+0000 7f96a77fe700 1 -- 172.21.15.119:0/1016037463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555904ef3460 msgr2=0x555904ef6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.781+0000 7f96a77fe700 1 --2- 172.21.15.119:0/1016037463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555904ef3460 0x555904ef6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.781+0000 7f96a77fe700 1 -- 172.21.15.119:0/1016037463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555904ef2b40 msgr2=0x555904ef2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.781+0000 7f96a77fe700 1 --2- 172.21.15.119:0/1016037463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555904ef2b40 0x555904ef2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.781+0000 7f96a77fe700 1 -- 172.21.15.119:0/1016037463 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555904cc3630 con 0x555904ef7150 2023-12-07T10:21:20.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.781+0000 7f96a57fa700 1 --2- 172.21.15.119:0/3523287098 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f96900a1da0 0x7f96900a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.781+0000 7f96a77fe700 1 --2- 172.21.15.119:0/1016037463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555904ef7150 0x555904ef7570 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f969c009cf0 tx=0x7f969c011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f20636ff2406623 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:20.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.781+0000 7f96a57fa700 1 -- 172.21.15.119:0/3523287098 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f96980ce920 con 0x7f96a8101560 2023-12-07T10:21:20.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.781+0000 7f96acd9c700 1 --2- 172.21.15.119:0/3523287098 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f96900a1da0 0x7f96900a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.782+0000 7f96acd9c700 1 --2- 172.21.15.119:0/3523287098 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f96900a1da0 0x7f96900a4260 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f96a8075a20 tx=0x7f96a000b040 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:20.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.784+0000 7f968e7fc700 1 -- 172.21.15.119:0/1016037463 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c011d80 con 0x555904ef7150 2023-12-07T10:21:20.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.784+0000 7f968e7fc700 1 -- 172.21.15.119:0/1016037463 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f969c004510 con 0x555904ef7150 2023-12-07T10:21:20.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.784+0000 7f968e7fc700 1 -- 172.21.15.119:0/1016037463 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c005280 con 0x555904ef7150 2023-12-07T10:21:20.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.785+0000 7f96c221eb80 1 -- 172.21.15.119:0/1016037463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555904ef7150 msgr2=0x555904ef7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.785+0000 7f96c221eb80 1 --2- 172.21.15.119:0/1016037463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555904ef7150 0x555904ef7570 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f969c009cf0 tx=0x7f969c011440 comp rx=0 tx=0).stop 2023-12-07T10:21:20.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.785+0000 7f96c221eb80 1 -- 172.21.15.119:0/1016037463 shutdown_connections 2023-12-07T10:21:20.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.785+0000 7f96c221eb80 1 --2- 172.21.15.119:0/1016037463 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555904ef7150 0x555904ef7570 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.785+0000 7f96c221eb80 1 --2- 172.21.15.119:0/1016037463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555904ef2b40 0x555904ef2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.785+0000 7f96c221eb80 1 --2- 172.21.15.119:0/1016037463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555904ef3460 0x555904ef6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.785+0000 7f96c221eb80 1 -- 172.21.15.119:0/1016037463 >> 172.21.15.119:0/1016037463 conn(0x555904e21360 msgr2=0x555904eed130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:20.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.785+0000 7f96c221eb80 1 -- 172.21.15.119:0/1016037463 shutdown_connections 2023-12-07T10:21:20.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.785+0000 7f96c221eb80 1 -- 172.21.15.119:0/1016037463 wait complete. 2023-12-07T10:21:20.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.786+0000 7f96c221eb80 1 -- start start 2023-12-07T10:21:20.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.786+0000 7f96c221eb80 1 Processor -- start 2023-12-07T10:21:20.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.786+0000 7f96c221eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555904ef2b40 0x555904efbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.786+0000 7f96c221eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555904ef3460 0x555904efc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.786+0000 7f96c221eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555904ef7150 0x555904efc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.786+0000 7f96c221eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555904e469b0 con 0x555904ef3460 2023-12-07T10:21:20.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.786+0000 7f96c221eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555904e00560 con 0x555904ef7150 2023-12-07T10:21:20.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.786+0000 7f96c221eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555904e00200 con 0x555904ef2b40 2023-12-07T10:21:20.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.786+0000 7f96c221eb80 10 client.0 _ll_register_callbacks cb 0x555904de7940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:20.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.787+0000 7f96a7fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555904ef2b40 0x555904efbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.787+0000 7f96a7fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555904ef2b40 0x555904efbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:51850/0 (socket says 172.21.15.119:51850) 2023-12-07T10:21:20.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.787+0000 7f96a7fff700 1 -- 172.21.15.119:0/77923442 learned_addr learned my addr 172.21.15.119:0/77923442 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:20.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.787+0000 7f96a77fe700 1 --2- 172.21.15.119:0/77923442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555904ef3460 0x555904efc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.787+0000 7f96acd9c700 1 --2- 172.21.15.119:0/77923442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555904ef7150 0x555904efc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.787+0000 7f96a7fff700 1 -- 172.21.15.119:0/77923442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555904ef7150 msgr2=0x555904efc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.787+0000 7f96a7fff700 1 --2- 172.21.15.119:0/77923442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555904ef7150 0x555904efc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.787+0000 7f96a7fff700 1 -- 172.21.15.119:0/77923442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555904ef3460 msgr2=0x555904efc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.787+0000 7f96a7fff700 1 --2- 172.21.15.119:0/77923442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555904ef3460 0x555904efc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.787+0000 7f96a7fff700 1 -- 172.21.15.119:0/77923442 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969c009920 con 0x555904ef2b40 2023-12-07T10:21:20.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.787+0000 7f96a7fff700 1 --2- 172.21.15.119:0/77923442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555904ef2b40 0x555904efbcc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9698097650 tx=0x7f9698005310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eef9fe1a014e88c8 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:20.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.787+0000 7f96877fe700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:21:20.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.787+0000 7f96877fe700 1 -- 172.21.15.119:0/77923442 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96980d1ea0 con 0x555904ef2b40 2023-12-07T10:21:20.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.787+0000 7f96877fe700 1 -- 172.21.15.119:0/77923442 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f969809b850 con 0x555904ef2b40 2023-12-07T10:21:20.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.787+0000 7f96c221eb80 1 -- 172.21.15.119:0/77923442 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555904ee5b00 con 0x555904ef2b40 2023-12-07T10:21:20.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.788+0000 7f96877fe700 1 -- 172.21.15.119:0/77923442 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f969809af20 con 0x555904ef2b40 2023-12-07T10:21:20.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.788+0000 7f96c221eb80 10 client.25978 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:20.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.788+0000 7f96c221eb80 1 -- 172.21.15.119:0/77923442 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555904c6fde0 con 0x555904ef2b40 2023-12-07T10:21:20.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.788+0000 7f96877fe700 1 -- 172.21.15.119:0/77923442 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555904ee5b00 con 0x555904ef2b40 2023-12-07T10:21:20.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.788+0000 7f96877fe700 1 -- 172.21.15.119:0/77923442 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f96980a2fc0 con 0x555904ef2b40 2023-12-07T10:21:20.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.788+0000 7f96c221eb80 10 client.25978 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:20.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.788+0000 7f96c221eb80 10 client.25978 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-07T10:21:20.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.788+0000 7f96c221eb80 20 client.25978 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:20.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.788+0000 7f96c221eb80 1 --2- 172.21.15.119:0/77923442 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x555904f0c0d0 0x555904f0e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.788+0000 7f96c221eb80 4 client.25978 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:21:20.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.788+0000 7f96c221eb80 1 -- 172.21.15.119:0/77923442 --> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555904ec17a0 con 0x555904f0c0d0 2023-12-07T10:21:20.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.789+0000 7f96a77fe700 1 --2- 172.21.15.119:0/77923442 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x555904f0c0d0 0x555904f0e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.789+0000 7f96a77fe700 1 --2- 172.21.15.119:0/77923442 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x555904f0c0d0 0x555904f0e4c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=43fc179226efd289 server_cookie=b8aeff466f39a882 in_seq=0 out_seq=0 2023-12-07T10:21:20.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.789+0000 7f96877fe700 10 client.25978 ms_handle_connect on v2:172.21.15.159:6836/3106097520 2023-12-07T10:21:20.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.790+0000 7f96877fe700 1 -- 172.21.15.119:0/77923442 <== mds.2 v2:172.21.15.159:6836/3106097520 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555904ec17a0 con 0x555904f0c0d0 2023-12-07T10:21:20.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.790+0000 7f96877fe700 10 client.25978 handle_command_reply: tid=0 2023-12-07T10:21:20.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.790+0000 7f96c221eb80 1 client.25978 shutdown 2023-12-07T10:21:20.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.790+0000 7f96c221eb80 1 -- 172.21.15.119:0/77923442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555904ef2b40 msgr2=0x555904efbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.790+0000 7f96c221eb80 1 --2- 172.21.15.119:0/77923442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555904ef2b40 0x555904efbcc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9698097650 tx=0x7f9698005310 comp rx=0 tx=0).stop 2023-12-07T10:21:20.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.790+0000 7f96c221eb80 1 -- 172.21.15.119:0/77923442 shutdown_connections 2023-12-07T10:21:20.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.790+0000 7f96c221eb80 1 --2- 172.21.15.119:0/77923442 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x555904f0c0d0 0x555904f0e4c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.790+0000 7f96c221eb80 1 --2- 172.21.15.119:0/77923442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555904ef7150 0x555904efc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.790+0000 7f96877fe700 0 client.25978 ms_handle_reset on v2:172.21.15.159:6836/3106097520 2023-12-07T10:21:20.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.790+0000 7f96c221eb80 1 --2- 172.21.15.119:0/77923442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555904ef2b40 0x555904efbcc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.790+0000 7f96c221eb80 1 --2- 172.21.15.119:0/77923442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555904ef3460 0x555904efc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.790+0000 7f96c221eb80 1 -- 172.21.15.119:0/77923442 >> 172.21.15.119:0/77923442 conn(0x555904e21360 msgr2=0x555904ef8620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:20.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.790+0000 7f96c221eb80 1 -- 172.21.15.119:0/77923442 shutdown_connections 2023-12-07T10:21:20.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.790+0000 7f96c221eb80 1 -- 172.21.15.119:0/77923442 wait complete. 2023-12-07T10:21:20.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.790+0000 7f96c221eb80 20 client.25978 trim_cache size 0 max 16384 2023-12-07T10:21:20.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.804+0000 7f96c221eb80 1 Processor -- start 2023-12-07T10:21:20.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.804+0000 7f96c221eb80 1 -- start start 2023-12-07T10:21:20.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.804+0000 7f96c221eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555904ef2b40 0x555904f08470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.804+0000 7f96c221eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555904ef3460 0x555904efc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.804+0000 7f96c221eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555904ef7150 0x555904f12da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.804+0000 7f96c221eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555904cc5920 con 0x555904ef3460 2023-12-07T10:21:20.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.804+0000 7f96c221eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555904eed9c0 con 0x555904ef7150 2023-12-07T10:21:20.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.804+0000 7f96c221eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555904eedbf0 con 0x555904ef2b40 2023-12-07T10:21:20.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.805+0000 7f96a77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555904ef3460 0x555904efc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.805+0000 7f96a77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555904ef3460 0x555904efc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37956/0 (socket says 172.21.15.119:37956) 2023-12-07T10:21:20.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.805+0000 7f96a77fe700 1 -- 172.21.15.119:0/3990997277 learned_addr learned my addr 172.21.15.119:0/3990997277 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:20.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.805+0000 7f96a7fff700 1 --2- 172.21.15.119:0/3990997277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555904ef2b40 0x555904f08470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.805+0000 7f96acd9c700 1 --2- 172.21.15.119:0/3990997277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555904ef7150 0x555904f12da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.805+0000 7f96a77fe700 1 -- 172.21.15.119:0/3990997277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555904ef2b40 msgr2=0x555904f08470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.805+0000 7f96a77fe700 1 --2- 172.21.15.119:0/3990997277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555904ef2b40 0x555904f08470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.805+0000 7f96a77fe700 1 -- 172.21.15.119:0/3990997277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555904ef7150 msgr2=0x555904f12da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.805+0000 7f96a77fe700 1 --2- 172.21.15.119:0/3990997277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555904ef7150 0x555904f12da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.805+0000 7f96a77fe700 1 -- 172.21.15.119:0/3990997277 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555904c58c60 con 0x555904ef3460 2023-12-07T10:21:20.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.805+0000 7f96a77fe700 1 --2- 172.21.15.119:0/3990997277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555904ef3460 0x555904efc560 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f969c009cf0 tx=0x7f969c005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5613cc67038903e server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:20.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.806+0000 7f968effd700 1 -- 172.21.15.119:0/3990997277 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c029070 con 0x555904ef3460 2023-12-07T10:21:20.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.806+0000 7f968effd700 1 -- 172.21.15.119:0/3990997277 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f969c004ba0 con 0x555904ef3460 2023-12-07T10:21:20.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.806+0000 7f968effd700 1 -- 172.21.15.119:0/3990997277 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c005270 con 0x555904ef3460 2023-12-07T10:21:20.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.806+0000 7f96c221eb80 1 -- 172.21.15.119:0/3990997277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555904ef3460 msgr2=0x555904efc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.806+0000 7f96c221eb80 1 --2- 172.21.15.119:0/3990997277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555904ef3460 0x555904efc560 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f969c009cf0 tx=0x7f969c005d00 comp rx=0 tx=0).stop 2023-12-07T10:21:20.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.806+0000 7f96c221eb80 1 -- 172.21.15.119:0/3990997277 shutdown_connections 2023-12-07T10:21:20.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.806+0000 7f96c221eb80 1 --2- 172.21.15.119:0/3990997277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555904ef7150 0x555904f12da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.806+0000 7f96c221eb80 1 --2- 172.21.15.119:0/3990997277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555904ef2b40 0x555904f08470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.806+0000 7f96c221eb80 1 --2- 172.21.15.119:0/3990997277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555904ef3460 0x555904efc560 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.806+0000 7f96c221eb80 1 -- 172.21.15.119:0/3990997277 >> 172.21.15.119:0/3990997277 conn(0x555904e21360 msgr2=0x555904ef7530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:20.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.807+0000 7f96c221eb80 1 -- 172.21.15.119:0/3990997277 shutdown_connections 2023-12-07T10:21:20.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.807+0000 7f96c221eb80 1 -- 172.21.15.119:0/3990997277 wait complete. 2023-12-07T10:21:20.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.807+0000 7f96c221eb80 1 -- start start 2023-12-07T10:21:20.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.807+0000 7f96c221eb80 1 Processor -- start 2023-12-07T10:21:20.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.808+0000 7f96c221eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555904ef2b40 0x555904f01380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.808+0000 7f96c221eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555904ef3460 0x555904f018c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.808+0000 7f96c221eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555904ef7150 0x555904f61520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.808+0000 7f96c221eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555904f02030 con 0x555904ef3460 2023-12-07T10:21:20.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.808+0000 7f96c221eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555904f61a60 con 0x555904ef7150 2023-12-07T10:21:20.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.808+0000 7f96c221eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555904f61c20 con 0x555904ef2b40 2023-12-07T10:21:20.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.808+0000 7f96c221eb80 10 client.0 _ll_register_callbacks cb 0x555904de7940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:20.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.808+0000 7f96a77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555904ef3460 0x555904f018c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.808+0000 7f96a77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555904ef3460 0x555904f018c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37966/0 (socket says 172.21.15.119:37966) 2023-12-07T10:21:20.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.808+0000 7f96a77fe700 1 -- 172.21.15.119:0/2218362291 learned_addr learned my addr 172.21.15.119:0/2218362291 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:20.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.808+0000 7f96a7fff700 1 --2- 172.21.15.119:0/2218362291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555904ef2b40 0x555904f01380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.808+0000 7f96acd9c700 1 --2- 172.21.15.119:0/2218362291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555904ef7150 0x555904f61520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.808+0000 7f96a7fff700 1 -- 172.21.15.119:0/2218362291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555904ef7150 msgr2=0x555904f61520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.808+0000 7f96a7fff700 1 --2- 172.21.15.119:0/2218362291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555904ef7150 0x555904f61520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.808+0000 7f96a7fff700 1 -- 172.21.15.119:0/2218362291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555904ef3460 msgr2=0x555904f018c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:20.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.808+0000 7f96a7fff700 1 --2- 172.21.15.119:0/2218362291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555904ef3460 0x555904f018c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.808+0000 7f96a7fff700 1 -- 172.21.15.119:0/2218362291 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969c01d040 con 0x555904ef2b40 2023-12-07T10:21:20.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.809+0000 7f96a7fff700 1 --2- 172.21.15.119:0/2218362291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555904ef2b40 0x555904f01380 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f96980ceca0 tx=0x7f9698005380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e051c84b3125fee9 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:20.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.809+0000 7f96877fe700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:21:20.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.809+0000 7f96877fe700 1 -- 172.21.15.119:0/2218362291 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96980d4060 con 0x555904ef2b40 2023-12-07T10:21:20.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.809+0000 7f96877fe700 1 -- 172.21.15.119:0/2218362291 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f969809b850 con 0x555904ef2b40 2023-12-07T10:21:20.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.809+0000 7f96c221eb80 1 -- 172.21.15.119:0/2218362291 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f969c004510 con 0x555904ef2b40 2023-12-07T10:21:20.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.809+0000 7f96877fe700 1 -- 172.21.15.119:0/2218362291 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96980aeea0 con 0x555904ef2b40 2023-12-07T10:21:20.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.809+0000 7f96c221eb80 10 client.25984 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:20.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.809+0000 7f96c221eb80 1 -- 172.21.15.119:0/2218362291 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555904c96250 con 0x555904ef2b40 2023-12-07T10:21:20.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.809+0000 7f96877fe700 1 -- 172.21.15.119:0/2218362291 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f969c004510 con 0x555904ef2b40 2023-12-07T10:21:20.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.809+0000 7f96877fe700 1 -- 172.21.15.119:0/2218362291 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f96980cbe20 con 0x555904ef2b40 2023-12-07T10:21:20.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.809+0000 7f96c221eb80 10 client.25984 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:20.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.809+0000 7f96c221eb80 10 client.25984 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-07T10:21:20.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.809+0000 7f96c221eb80 20 client.25984 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:20.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.809+0000 7f96c221eb80 1 --2- 172.21.15.119:0/2218362291 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x555904f62f10 0x555904f65300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:20.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.809+0000 7f96c221eb80 4 client.25984 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:21:20.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.809+0000 7f96c221eb80 1 -- 172.21.15.119:0/2218362291 --> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555904e0cfd0 con 0x555904f62f10 2023-12-07T10:21:20.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.810+0000 7f96a77fe700 1 --2- 172.21.15.119:0/2218362291 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x555904f62f10 0x555904f65300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:20.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.810+0000 7f96a77fe700 1 --2- 172.21.15.119:0/2218362291 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x555904f62f10 0x555904f65300 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f3a7c8df4a50cbb server_cookie=d9aa372e0573ed0e in_seq=0 out_seq=0 2023-12-07T10:21:20.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.810+0000 7f96877fe700 10 client.25984 ms_handle_connect on v2:172.21.15.159:6836/3106097520 2023-12-07T10:21:20.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.811+0000 7f96877fe700 1 -- 172.21.15.119:0/2218362291 <== mds.2 v2:172.21.15.159:6836/3106097520 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555904e0cfd0 con 0x555904f62f10 2023-12-07T10:21:20.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.811+0000 7f96877fe700 10 client.25984 handle_command_reply: tid=0 2023-12-07T10:21:20.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.811+0000 7f96c221eb80 1 client.25984 shutdown 2023-12-07T10:21:20.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.811+0000 7f96c221eb80 1 -- 172.21.15.119:0/2218362291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555904ef2b40 msgr2=0x555904f01380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.811+0000 7f96c221eb80 1 --2- 172.21.15.119:0/2218362291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555904ef2b40 0x555904f01380 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f96980ceca0 tx=0x7f9698005380 comp rx=0 tx=0).stop 2023-12-07T10:21:20.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.811+0000 7f96c221eb80 1 -- 172.21.15.119:0/2218362291 shutdown_connections 2023-12-07T10:21:20.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.811+0000 7f96c221eb80 1 --2- 172.21.15.119:0/2218362291 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x555904f62f10 0x555904f65300 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.811+0000 7f96c221eb80 1 --2- 172.21.15.119:0/2218362291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555904ef7150 0x555904f61520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.811+0000 7f96877fe700 0 client.25984 ms_handle_reset on v2:172.21.15.159:6836/3106097520 2023-12-07T10:21:20.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.811+0000 7f96c221eb80 1 --2- 172.21.15.119:0/2218362291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555904ef2b40 0x555904f01380 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.811+0000 7f96c221eb80 1 --2- 172.21.15.119:0/2218362291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555904ef3460 0x555904f018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.811+0000 7f96c221eb80 1 -- 172.21.15.119:0/2218362291 >> 172.21.15.119:0/2218362291 conn(0x555904e21360 msgr2=0x555904f12da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:20.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.811+0000 7f96c221eb80 1 -- 172.21.15.119:0/2218362291 shutdown_connections 2023-12-07T10:21:20.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.811+0000 7f96c221eb80 1 -- 172.21.15.119:0/2218362291 wait complete. 2023-12-07T10:21:20.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.811+0000 7f96c221eb80 20 client.25984 trim_cache size 0 max 16384 2023-12-07T10:21:20.819 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:21:20.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.812+0000 7f96ae7ff700 1 -- 172.21.15.119:0/3523287098 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f96900a1da0 msgr2=0x7f96900a4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.812+0000 7f96ae7ff700 1 --2- 172.21.15.119:0/3523287098 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f96900a1da0 0x7f96900a4260 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f96a8075a20 tx=0x7f96a000b040 comp rx=0 tx=0).stop 2023-12-07T10:21:20.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.812+0000 7f96ae7ff700 1 -- 172.21.15.119:0/3523287098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96a8101560 msgr2=0x7f96a8073910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:20.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.812+0000 7f96ae7ff700 1 --2- 172.21.15.119:0/3523287098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96a8101560 0x7f96a8073910 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9698000c00 tx=0x7f969800fa00 comp rx=0 tx=0).stop 2023-12-07T10:21:20.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.812+0000 7f96ae7ff700 1 -- 172.21.15.119:0/3523287098 shutdown_connections 2023-12-07T10:21:20.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.812+0000 7f96ae7ff700 1 --2- 172.21.15.119:0/3523287098 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f96900a1da0 0x7f96900a4260 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.812+0000 7f96ae7ff700 1 --2- 172.21.15.119:0/3523287098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96a8107620 0x7f96a80743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.812+0000 7f96ae7ff700 1 --2- 172.21.15.119:0/3523287098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96a8101560 0x7f96a8073910 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.812+0000 7f96ae7ff700 1 --2- 172.21.15.119:0/3523287098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96a8101f20 0x7f96a8073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:20.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.812+0000 7f96ae7ff700 1 -- 172.21.15.119:0/3523287098 >> 172.21.15.119:0/3523287098 conn(0x7f96a80fcfd0 msgr2=0x7f96a80fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:20.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.812+0000 7f96ae7ff700 1 -- 172.21.15.119:0/3523287098 shutdown_connections 2023-12-07T10:21:20.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:20.812+0000 7f96ae7ff700 1 -- 172.21.15.119:0/3523287098 wait complete. 2023-12-07T10:21:20.833 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-07T10:21:21.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.087+0000 7f622719d700 1 -- 172.21.15.119:0/2264953453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62201013d0 msgr2=0x7f6220101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:21.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.087+0000 7f622719d700 1 --2- 172.21.15.119:0/2264953453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62201013d0 0x7f6220101850 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6210009990 tx=0x7f6210009ca0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.087+0000 7f622719d700 1 -- 172.21.15.119:0/2264953453 shutdown_connections 2023-12-07T10:21:21.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.087+0000 7f622719d700 1 --2- 172.21.15.119:0/2264953453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6220101d90 0x7f622010e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.087+0000 7f622719d700 1 --2- 172.21.15.119:0/2264953453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6220107470 0x7f6220107850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.087+0000 7f622719d700 1 --2- 172.21.15.119:0/2264953453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62201013d0 0x7f6220101850 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.087+0000 7f622719d700 1 -- 172.21.15.119:0/2264953453 >> 172.21.15.119:0/2264953453 conn(0x7f62200fcee0 msgr2=0x7f62200ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:21.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.088+0000 7f622719d700 1 -- 172.21.15.119:0/2264953453 shutdown_connections 2023-12-07T10:21:21.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.088+0000 7f622719d700 1 -- 172.21.15.119:0/2264953453 wait complete. 2023-12-07T10:21:21.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.089+0000 7f622719d700 1 Processor -- start 2023-12-07T10:21:21.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.089+0000 7f622719d700 1 -- start start 2023-12-07T10:21:21.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.090+0000 7f622719d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62201013d0 0x7f6220199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.090+0000 7f622719d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6220101d90 0x7f6220199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.090+0000 7f622719d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6220107470 0x7f622019e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.090+0000 7f622719d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622019e940 con 0x7f6220101d90 2023-12-07T10:21:21.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.090+0000 7f622719d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622019eab0 con 0x7f62201013d0 2023-12-07T10:21:21.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.090+0000 7f622719d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622019ec70 con 0x7f6220107470 2023-12-07T10:21:21.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.090+0000 7f622619b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62201013d0 0x7f6220199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.090+0000 7f622619b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62201013d0 0x7f6220199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:47732/0 (socket says 172.21.15.119:47732) 2023-12-07T10:21:21.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.090+0000 7f622699c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6220107470 0x7f622019e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.090+0000 7f622619b700 1 -- 172.21.15.119:0/2216940874 learned_addr learned my addr 172.21.15.119:0/2216940874 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:21.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.090+0000 7f622599a700 1 --2- 172.21.15.119:0/2216940874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6220101d90 0x7f6220199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.091+0000 7f622619b700 1 -- 172.21.15.119:0/2216940874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6220107470 msgr2=0x7f622019e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:21.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.091+0000 7f622619b700 1 --2- 172.21.15.119:0/2216940874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6220107470 0x7f622019e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.091+0000 7f622619b700 1 -- 172.21.15.119:0/2216940874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6220101d90 msgr2=0x7f6220199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:21.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.091+0000 7f622619b700 1 --2- 172.21.15.119:0/2216940874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6220101d90 0x7f6220199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.091+0000 7f622619b700 1 -- 172.21.15.119:0/2216940874 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62100095f0 con 0x7f62201013d0 2023-12-07T10:21:21.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.091+0000 7f622699c700 1 --2- 172.21.15.119:0/2216940874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6220107470 0x7f622019e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:21.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.091+0000 7f622599a700 1 --2- 172.21.15.119:0/2216940874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6220101d90 0x7f6220199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:21.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.091+0000 7f622619b700 1 --2- 172.21.15.119:0/2216940874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62201013d0 0x7f6220199a60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f621c00e9a0 tx=0x7f621c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:21.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.092+0000 7f62177fe700 1 -- 172.21.15.119:0/2216940874 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c009790 con 0x7f62201013d0 2023-12-07T10:21:21.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.092+0000 7f62177fe700 1 -- 172.21.15.119:0/2216940874 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f621c010430 con 0x7f62201013d0 2023-12-07T10:21:21.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.092+0000 7f62177fe700 1 -- 172.21.15.119:0/2216940874 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c018680 con 0x7f62201013d0 2023-12-07T10:21:21.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.092+0000 7f622719d700 1 -- 172.21.15.119:0/2216940874 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f622019ef50 con 0x7f62201013d0 2023-12-07T10:21:21.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.092+0000 7f622719d700 1 -- 172.21.15.119:0/2216940874 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62201a6800 con 0x7f62201013d0 2023-12-07T10:21:21.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.097+0000 7f62177fe700 1 -- 172.21.15.119:0/2216940874 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 25) v1 ==== 136717+0+0 (secure 0 0 0) 0x7f621c0105a0 con 0x7f62201013d0 2023-12-07T10:21:21.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.098+0000 7f623bc9fb80 1 Processor -- start 2023-12-07T10:21:21.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.098+0000 7f623bc9fb80 1 -- start start 2023-12-07T10:21:21.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.099+0000 7f623bc9fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5637d5b711f0 0x5637d5b715d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.099+0000 7f623bc9fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637d5b71b10 0x5637d5b75610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.099+0000 7f62177fe700 1 --2- 172.21.15.119:0/2216940874 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f620c0a1e20 0x7f620c0a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.099+0000 7f623bc9fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637d5b75b50 0x5637d5b75fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.099+0000 7f623bc9fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d5adc7c0 con 0x5637d5b75b50 2023-12-07T10:21:21.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.099+0000 7f623bc9fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d5ac37a0 con 0x5637d5b71b10 2023-12-07T10:21:21.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.099+0000 7f623bc9fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d5ac10c0 con 0x5637d5b711f0 2023-12-07T10:21:21.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.099+0000 7f62177fe700 1 -- 172.21.15.119:0/2216940874 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f621c014070 con 0x7f62201013d0 2023-12-07T10:21:21.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.099+0000 7f622599a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637d5b71b10 0x5637d5b75610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.099+0000 7f622599a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637d5b71b10 0x5637d5b75610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:47748/0 (socket says 172.21.15.119:47748) 2023-12-07T10:21:21.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.099+0000 7f622599a700 1 -- 172.21.15.119:0/2105928880 learned_addr learned my addr 172.21.15.119:0/2105928880 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:21.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.099+0000 7f622699c700 1 --2- 172.21.15.119:0/2105928880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5637d5b711f0 0x5637d5b715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.099+0000 7f622699c700 1 --2- 172.21.15.119:0/2105928880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637d5b75b50 0x5637d5b75fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.100+0000 7f622619b700 1 --2- 172.21.15.119:0/2216940874 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f620c0a1e20 0x7f620c0a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.100+0000 7f622699c700 1 -- 172.21.15.119:0/2105928880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637d5b71b10 msgr2=0x5637d5b75610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:21.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.100+0000 7f622699c700 1 --2- 172.21.15.119:0/2105928880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637d5b71b10 0x5637d5b75610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.100+0000 7f622699c700 1 -- 172.21.15.119:0/2105928880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637d5b75b50 msgr2=0x5637d5b75fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:21.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.100+0000 7f622699c700 1 --2- 172.21.15.119:0/2105928880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637d5b75b50 0x5637d5b75fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.100+0000 7f622699c700 1 -- 172.21.15.119:0/2105928880 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637d5939630 con 0x5637d5b711f0 2023-12-07T10:21:21.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.100+0000 7f622619b700 1 --2- 172.21.15.119:0/2216940874 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f620c0a1e20 0x7f620c0a42e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f621c0cb350 tx=0x7f621c010b80 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:21.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.100+0000 7f622699c700 1 --2- 172.21.15.119:0/2105928880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5637d5b711f0 0x5637d5b715d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6218008b90 tx=0x7f6218009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7f8ec85af000cb3 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:21.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.101+0000 7f6214bfe700 1 -- 172.21.15.119:0/2105928880 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6218008ea0 con 0x5637d5b711f0 2023-12-07T10:21:21.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.101+0000 7f6214bfe700 1 -- 172.21.15.119:0/2105928880 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f621800f040 con 0x5637d5b711f0 2023-12-07T10:21:21.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.101+0000 7f6214bfe700 1 -- 172.21.15.119:0/2105928880 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6218003df0 con 0x5637d5b711f0 2023-12-07T10:21:21.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.101+0000 7f623bc9fb80 1 -- 172.21.15.119:0/2105928880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5637d5b711f0 msgr2=0x5637d5b715d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:21.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.101+0000 7f623bc9fb80 1 --2- 172.21.15.119:0/2105928880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5637d5b711f0 0x5637d5b715d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6218008b90 tx=0x7f6218009920 comp rx=0 tx=0).stop 2023-12-07T10:21:21.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.101+0000 7f623bc9fb80 1 -- 172.21.15.119:0/2105928880 shutdown_connections 2023-12-07T10:21:21.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.101+0000 7f623bc9fb80 1 --2- 172.21.15.119:0/2105928880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637d5b75b50 0x5637d5b75fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.101+0000 7f623bc9fb80 1 --2- 172.21.15.119:0/2105928880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5637d5b711f0 0x5637d5b715d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.101+0000 7f623bc9fb80 1 --2- 172.21.15.119:0/2105928880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637d5b71b10 0x5637d5b75610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.101+0000 7f623bc9fb80 1 -- 172.21.15.119:0/2105928880 >> 172.21.15.119:0/2105928880 conn(0x5637d5a97360 msgr2=0x5637d5b63170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:21.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.102+0000 7f623bc9fb80 1 -- 172.21.15.119:0/2105928880 shutdown_connections 2023-12-07T10:21:21.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.102+0000 7f623bc9fb80 1 -- 172.21.15.119:0/2105928880 wait complete. 2023-12-07T10:21:21.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.102+0000 7f623bc9fb80 1 -- start start 2023-12-07T10:21:21.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.102+0000 7f623bc9fb80 1 Processor -- start 2023-12-07T10:21:21.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.103+0000 7f623bc9fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637d5b711f0 0x5637d5b7a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.103+0000 7f623bc9fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5637d5b71b10 0x5637d5b7ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.103+0000 7f623bc9fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637d5b75b50 0x5637d5b7b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.103+0000 7f623bc9fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d5abc9b0 con 0x5637d5b75b50 2023-12-07T10:21:21.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.103+0000 7f623bc9fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d5a76560 con 0x5637d5b711f0 2023-12-07T10:21:21.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.103+0000 7f623bc9fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d5a76200 con 0x5637d5b71b10 2023-12-07T10:21:21.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.103+0000 7f623bc9fb80 10 client.0 _ll_register_callbacks cb 0x5637d5a5d940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:21.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.104+0000 7f622599a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5637d5b71b10 0x5637d5b7ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.104+0000 7f622599a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5637d5b71b10 0x5637d5b7ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:51884/0 (socket says 172.21.15.119:51884) 2023-12-07T10:21:21.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.104+0000 7f622599a700 1 -- 172.21.15.119:0/770073753 learned_addr learned my addr 172.21.15.119:0/770073753 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:21.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.104+0000 7f622699c700 1 --2- 172.21.15.119:0/770073753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637d5b75b50 0x5637d5b7b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.104+0000 7f622699c700 1 --2- 172.21.15.119:0/770073753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637d5b711f0 0x5637d5b7a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.104+0000 7f622599a700 1 -- 172.21.15.119:0/770073753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637d5b711f0 msgr2=0x5637d5b7a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:21.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.104+0000 7f622599a700 1 --2- 172.21.15.119:0/770073753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637d5b711f0 0x5637d5b7a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.104+0000 7f622599a700 1 -- 172.21.15.119:0/770073753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637d5b75b50 msgr2=0x5637d5b7b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:21.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.104+0000 7f622599a700 1 --2- 172.21.15.119:0/770073753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637d5b75b50 0x5637d5b7b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.104+0000 7f622599a700 1 -- 172.21.15.119:0/770073753 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6218008870 con 0x5637d5b71b10 2023-12-07T10:21:21.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.104+0000 7f622599a700 1 --2- 172.21.15.119:0/770073753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5637d5b71b10 0x5637d5b7ab40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6210006010 tx=0x7f6210005300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=773b9132728692a5 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:21.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.104+0000 7f6209ffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:21:21.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.105+0000 7f6209ffb700 1 -- 172.21.15.119:0/770073753 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621001d040 con 0x5637d5b71b10 2023-12-07T10:21:21.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.105+0000 7f6209ffb700 1 -- 172.21.15.119:0/770073753 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f62100046f0 con 0x5637d5b71b10 2023-12-07T10:21:21.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.105+0000 7f623bc9fb80 1 -- 172.21.15.119:0/770073753 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637d5b5b9a0 con 0x5637d5b71b10 2023-12-07T10:21:21.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.105+0000 7f6209ffb700 1 -- 172.21.15.119:0/770073753 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62100048b0 con 0x5637d5b71b10 2023-12-07T10:21:21.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.105+0000 7f623bc9fb80 10 client.25996 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:21.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.105+0000 7f623bc9fb80 1 -- 172.21.15.119:0/770073753 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5637d58e5de0 con 0x5637d5b71b10 2023-12-07T10:21:21.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.105+0000 7f6209ffb700 1 -- 172.21.15.119:0/770073753 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5637d5b5b9a0 con 0x5637d5b71b10 2023-12-07T10:21:21.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.106+0000 7f6209ffb700 1 -- 172.21.15.119:0/770073753 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f621000fab0 con 0x5637d5b71b10 2023-12-07T10:21:21.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.106+0000 7f623bc9fb80 10 client.25996 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:21.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.106+0000 7f623bc9fb80 10 client.25996 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2023-12-07T10:21:21.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.106+0000 7f623bc9fb80 20 client.25996 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:21.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.106+0000 7f623bc9fb80 1 --2- 172.21.15.119:0/770073753 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x5637d5b6c840 0x5637d5b6ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.106+0000 7f623bc9fb80 4 client.25996 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:21:21.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.106+0000 7f623bc9fb80 1 -- 172.21.15.119:0/770073753 --> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637d5b377a0 con 0x5637d5b6c840 2023-12-07T10:21:21.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.107+0000 7f622699c700 1 --2- 172.21.15.119:0/770073753 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x5637d5b6c840 0x5637d5b6ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.107+0000 7f622699c700 1 --2- 172.21.15.119:0/770073753 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x5637d5b6c840 0x5637d5b6ec60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9f08b4b66d000eed server_cookie=cbe1f7d0112473bf in_seq=0 out_seq=0 2023-12-07T10:21:21.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.107+0000 7f6209ffb700 10 client.25996 ms_handle_connect on v2:172.21.15.119:6838/1920762602 2023-12-07T10:21:21.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.108+0000 7f6209ffb700 1 -- 172.21.15.119:0/770073753 <== mds.1 v2:172.21.15.119:6838/1920762602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5637d5b377a0 con 0x5637d5b6c840 2023-12-07T10:21:21.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.108+0000 7f6209ffb700 10 client.25996 handle_command_reply: tid=0 2023-12-07T10:21:21.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.108+0000 7f623bc9fb80 1 client.25996 shutdown 2023-12-07T10:21:21.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.109+0000 7f623bc9fb80 1 -- 172.21.15.119:0/770073753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5637d5b71b10 msgr2=0x5637d5b7ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:21.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.109+0000 7f623bc9fb80 1 --2- 172.21.15.119:0/770073753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5637d5b71b10 0x5637d5b7ab40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6210006010 tx=0x7f6210005300 comp rx=0 tx=0).stop 2023-12-07T10:21:21.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.109+0000 7f623bc9fb80 1 -- 172.21.15.119:0/770073753 shutdown_connections 2023-12-07T10:21:21.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.109+0000 7f623bc9fb80 1 --2- 172.21.15.119:0/770073753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637d5b75b50 0x5637d5b7b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.109+0000 7f623bc9fb80 1 --2- 172.21.15.119:0/770073753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637d5b711f0 0x5637d5b7a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.109+0000 7f623bc9fb80 1 --2- 172.21.15.119:0/770073753 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x5637d5b6c840 0x5637d5b6ec60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.109+0000 7f623bc9fb80 1 --2- 172.21.15.119:0/770073753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5637d5b71b10 0x5637d5b7ab40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.109+0000 7f623bc9fb80 1 -- 172.21.15.119:0/770073753 >> 172.21.15.119:0/770073753 conn(0x5637d5a97360 msgr2=0x5637d5b677e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:21.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.109+0000 7f6209ffb700 0 client.25996 ms_handle_reset on v2:172.21.15.119:6838/1920762602 2023-12-07T10:21:21.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.109+0000 7f623bc9fb80 1 -- 172.21.15.119:0/770073753 shutdown_connections 2023-12-07T10:21:21.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.109+0000 7f623bc9fb80 1 -- 172.21.15.119:0/770073753 wait complete. 2023-12-07T10:21:21.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.109+0000 7f623bc9fb80 20 client.25996 trim_cache size 0 max 16384 2023-12-07T10:21:21.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.123+0000 7f623bc9fb80 1 Processor -- start 2023-12-07T10:21:21.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.123+0000 7f623bc9fb80 1 -- start start 2023-12-07T10:21:21.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.123+0000 7f623bc9fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637d5b711f0 0x5637d5b73440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.123+0000 7f623bc9fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637d5b71b10 0x5637d5b70570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.123+0000 7f623bc9fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5637d5b75b50 0x5637d5b70ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.123+0000 7f623bc9fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d593b920 con 0x5637d5b71b10 2023-12-07T10:21:21.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.123+0000 7f623bc9fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d5b5b980 con 0x5637d5b711f0 2023-12-07T10:21:21.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.123+0000 7f623bc9fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d5b78b70 con 0x5637d5b75b50 2023-12-07T10:21:21.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.124+0000 7f622599a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637d5b71b10 0x5637d5b70570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.124+0000 7f622599a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637d5b71b10 0x5637d5b70570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38016/0 (socket says 172.21.15.119:38016) 2023-12-07T10:21:21.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.124+0000 7f622599a700 1 -- 172.21.15.119:0/3684806002 learned_addr learned my addr 172.21.15.119:0/3684806002 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:21.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.124+0000 7f622699c700 1 --2- 172.21.15.119:0/3684806002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637d5b711f0 0x5637d5b73440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.124+0000 7f622699c700 1 --2- 172.21.15.119:0/3684806002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5637d5b75b50 0x5637d5b70ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.124+0000 7f622599a700 1 -- 172.21.15.119:0/3684806002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5637d5b75b50 msgr2=0x5637d5b70ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:21.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.124+0000 7f622599a700 1 --2- 172.21.15.119:0/3684806002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5637d5b75b50 0x5637d5b70ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.124+0000 7f622599a700 1 -- 172.21.15.119:0/3684806002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637d5b711f0 msgr2=0x5637d5b73440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:21.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.124+0000 7f622599a700 1 --2- 172.21.15.119:0/3684806002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637d5b711f0 0x5637d5b73440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.124+0000 7f622599a700 1 -- 172.21.15.119:0/3684806002 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637d58cec60 con 0x5637d5b71b10 2023-12-07T10:21:21.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.124+0000 7f622699c700 1 --2- 172.21.15.119:0/3684806002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637d5b711f0 0x5637d5b73440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:21:21.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.125+0000 7f622599a700 1 --2- 172.21.15.119:0/3684806002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637d5b71b10 0x5637d5b70570 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6210009990 tx=0x7f6210009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6395aa41df6469d3 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:21.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.125+0000 7f62153ff700 1 -- 172.21.15.119:0/3684806002 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6210039030 con 0x5637d5b71b10 2023-12-07T10:21:21.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.125+0000 7f62153ff700 1 -- 172.21.15.119:0/3684806002 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f621001e040 con 0x5637d5b71b10 2023-12-07T10:21:21.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.125+0000 7f62153ff700 1 -- 172.21.15.119:0/3684806002 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621001d070 con 0x5637d5b71b10 2023-12-07T10:21:21.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.126+0000 7f623bc9fb80 1 -- 172.21.15.119:0/3684806002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637d5b71b10 msgr2=0x5637d5b70570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:21.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.126+0000 7f623bc9fb80 1 --2- 172.21.15.119:0/3684806002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637d5b71b10 0x5637d5b70570 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6210009990 tx=0x7f6210009310 comp rx=0 tx=0).stop 2023-12-07T10:21:21.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.126+0000 7f623bc9fb80 1 -- 172.21.15.119:0/3684806002 shutdown_connections 2023-12-07T10:21:21.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.126+0000 7f623bc9fb80 1 --2- 172.21.15.119:0/3684806002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5637d5b75b50 0x5637d5b70ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.126+0000 7f623bc9fb80 1 --2- 172.21.15.119:0/3684806002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637d5b711f0 0x5637d5b73440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.126+0000 7f623bc9fb80 1 --2- 172.21.15.119:0/3684806002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637d5b71b10 0x5637d5b70570 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.126+0000 7f623bc9fb80 1 -- 172.21.15.119:0/3684806002 >> 172.21.15.119:0/3684806002 conn(0x5637d5a97360 msgr2=0x5637d5b9bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:21.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.126+0000 7f623bc9fb80 1 -- 172.21.15.119:0/3684806002 shutdown_connections 2023-12-07T10:21:21.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.126+0000 7f623bc9fb80 1 -- 172.21.15.119:0/3684806002 wait complete. 2023-12-07T10:21:21.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.127+0000 7f623bc9fb80 1 -- start start 2023-12-07T10:21:21.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.127+0000 7f623bc9fb80 1 Processor -- start 2023-12-07T10:21:21.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.128+0000 7f623bc9fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637d5b711f0 0x5637d5b81ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.128+0000 7f623bc9fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5637d5b75b50 0x5637d5b70570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.128+0000 7f623bc9fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637d5b70ab0 0x5637d5b67ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.128+0000 7f623bc9fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d5b82510 con 0x5637d5b711f0 2023-12-07T10:21:21.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.128+0000 7f623bc9fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d5b68400 con 0x5637d5b70ab0 2023-12-07T10:21:21.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.128+0000 7f623bc9fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d5b685c0 con 0x5637d5b75b50 2023-12-07T10:21:21.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.128+0000 7f623bc9fb80 10 client.0 _ll_register_callbacks cb 0x5637d5a5d940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:21.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.128+0000 7f622699c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637d5b711f0 0x5637d5b81ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.129+0000 7f622699c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637d5b711f0 0x5637d5b81ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38018/0 (socket says 172.21.15.119:38018) 2023-12-07T10:21:21.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.129+0000 7f622699c700 1 -- 172.21.15.119:0/119708702 learned_addr learned my addr 172.21.15.119:0/119708702 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:21.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.129+0000 7f622699c700 1 --2- 172.21.15.119:0/119708702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637d5b70ab0 0x5637d5b67ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.129+0000 7f622599a700 1 --2- 172.21.15.119:0/119708702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5637d5b75b50 0x5637d5b70570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.129+0000 7f622699c700 1 -- 172.21.15.119:0/119708702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5637d5b75b50 msgr2=0x5637d5b70570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:21.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.129+0000 7f622699c700 1 --2- 172.21.15.119:0/119708702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5637d5b75b50 0x5637d5b70570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.129+0000 7f622699c700 1 -- 172.21.15.119:0/119708702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637d5b70ab0 msgr2=0x5637d5b67ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:21.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.129+0000 7f622699c700 1 --2- 172.21.15.119:0/119708702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637d5b70ab0 0x5637d5b67ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.129+0000 7f622699c700 1 -- 172.21.15.119:0/119708702 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637d58e5de0 con 0x5637d5b711f0 2023-12-07T10:21:21.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.129+0000 7f622699c700 1 --2- 172.21.15.119:0/119708702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637d5b711f0 0x5637d5b81ef0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6218008b60 tx=0x7f6218009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c530c2baf3c155d0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:21.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.129+0000 7f6209ffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:21:21.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.130+0000 7f6209ffb700 1 -- 172.21.15.119:0/119708702 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621800ea10 con 0x5637d5b711f0 2023-12-07T10:21:21.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.130+0000 7f6209ffb700 1 -- 172.21.15.119:0/119708702 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f621800eb70 con 0x5637d5b711f0 2023-12-07T10:21:21.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.130+0000 7f623bc9fb80 1 -- 172.21.15.119:0/119708702 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f621800f040 con 0x5637d5b711f0 2023-12-07T10:21:21.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.130+0000 7f6209ffb700 1 -- 172.21.15.119:0/119708702 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f621800ed30 con 0x5637d5b711f0 2023-12-07T10:21:21.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.130+0000 7f623bc9fb80 10 client.15999 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:21.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.130+0000 7f623bc9fb80 1 -- 172.21.15.119:0/119708702 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5637d590c250 con 0x5637d5b711f0 2023-12-07T10:21:21.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.130+0000 7f6209ffb700 1 -- 172.21.15.119:0/119708702 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f621800f040 con 0x5637d5b711f0 2023-12-07T10:21:21.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.130+0000 7f6209ffb700 1 -- 172.21.15.119:0/119708702 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6218027450 con 0x5637d5b711f0 2023-12-07T10:21:21.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.131+0000 7f623bc9fb80 10 client.15999 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:21.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.131+0000 7f623bc9fb80 10 client.15999 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2023-12-07T10:21:21.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.131+0000 7f623bc9fb80 20 client.15999 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:21.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.131+0000 7f623bc9fb80 1 --2- 172.21.15.119:0/119708702 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x5637d5bea230 0x5637d5bec6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.131+0000 7f623bc9fb80 4 client.15999 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:21:21.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.131+0000 7f623bc9fb80 1 -- 172.21.15.119:0/119708702 --> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5637d5a82fd0 con 0x5637d5bea230 2023-12-07T10:21:21.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.131+0000 7f622599a700 1 --2- 172.21.15.119:0/119708702 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x5637d5bea230 0x5637d5bec6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.131+0000 7f622599a700 1 --2- 172.21.15.119:0/119708702 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x5637d5bea230 0x5637d5bec6f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c350ff9368357253 server_cookie=814d599cf5394ce9 in_seq=0 out_seq=0 2023-12-07T10:21:21.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.132+0000 7f6209ffb700 10 client.15999 ms_handle_connect on v2:172.21.15.119:6838/1920762602 2023-12-07T10:21:21.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.132+0000 7f6209ffb700 1 -- 172.21.15.119:0/119708702 <== mds.1 v2:172.21.15.119:6838/1920762602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5637d5a82fd0 con 0x5637d5bea230 2023-12-07T10:21:21.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.132+0000 7f6209ffb700 10 client.15999 handle_command_reply: tid=0 2023-12-07T10:21:21.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.132+0000 7f623bc9fb80 1 client.15999 shutdown 2023-12-07T10:21:21.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.132+0000 7f623bc9fb80 1 -- 172.21.15.119:0/119708702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637d5b711f0 msgr2=0x5637d5b81ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:21.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.132+0000 7f623bc9fb80 1 --2- 172.21.15.119:0/119708702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637d5b711f0 0x5637d5b81ef0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6218008b60 tx=0x7f6218009920 comp rx=0 tx=0).stop 2023-12-07T10:21:21.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.132+0000 7f623bc9fb80 1 -- 172.21.15.119:0/119708702 shutdown_connections 2023-12-07T10:21:21.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.132+0000 7f623bc9fb80 1 --2- 172.21.15.119:0/119708702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5637d5b70ab0 0x5637d5b67ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.132+0000 7f623bc9fb80 1 --2- 172.21.15.119:0/119708702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637d5b711f0 0x5637d5b81ef0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.132+0000 7f623bc9fb80 1 --2- 172.21.15.119:0/119708702 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x5637d5bea230 0x5637d5bec6f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.132+0000 7f623bc9fb80 1 --2- 172.21.15.119:0/119708702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5637d5b75b50 0x5637d5b70570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.132+0000 7f6209ffb700 0 client.15999 ms_handle_reset on v2:172.21.15.119:6838/1920762602 2023-12-07T10:21:21.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.132+0000 7f623bc9fb80 1 -- 172.21.15.119:0/119708702 >> 172.21.15.119:0/119708702 conn(0x5637d5a97360 msgr2=0x5637d5b649b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:21.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.133+0000 7f623bc9fb80 1 -- 172.21.15.119:0/119708702 shutdown_connections 2023-12-07T10:21:21.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.133+0000 7f623bc9fb80 1 -- 172.21.15.119:0/119708702 wait complete. 2023-12-07T10:21:21.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.133+0000 7f623bc9fb80 20 client.15999 trim_cache size 0 max 16384 2023-12-07T10:21:21.137 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:21:21.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.134+0000 7f622719d700 1 -- 172.21.15.119:0/2216940874 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f620c0a1e20 msgr2=0x7f620c0a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:21.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.134+0000 7f622719d700 1 --2- 172.21.15.119:0/2216940874 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f620c0a1e20 0x7f620c0a42e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f621c0cb350 tx=0x7f621c010b80 comp rx=0 tx=0).stop 2023-12-07T10:21:21.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.134+0000 7f622719d700 1 -- 172.21.15.119:0/2216940874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62201013d0 msgr2=0x7f6220199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:21.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.134+0000 7f622719d700 1 --2- 172.21.15.119:0/2216940874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62201013d0 0x7f6220199a60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f621c00e9a0 tx=0x7f621c00ecb0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.134+0000 7f622719d700 1 -- 172.21.15.119:0/2216940874 shutdown_connections 2023-12-07T10:21:21.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.134+0000 7f622719d700 1 --2- 172.21.15.119:0/2216940874 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f620c0a1e20 0x7f620c0a42e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.134+0000 7f622719d700 1 --2- 172.21.15.119:0/2216940874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6220107470 0x7f622019e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.134+0000 7f622719d700 1 --2- 172.21.15.119:0/2216940874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62201013d0 0x7f6220199a60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.134+0000 7f622719d700 1 --2- 172.21.15.119:0/2216940874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6220101d90 0x7f6220199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.134+0000 7f622719d700 1 -- 172.21.15.119:0/2216940874 >> 172.21.15.119:0/2216940874 conn(0x7f62200fcee0 msgr2=0x7f622010cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:21.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.134+0000 7f622719d700 1 -- 172.21.15.119:0/2216940874 shutdown_connections 2023-12-07T10:21:21.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.134+0000 7f622719d700 1 -- 172.21.15.119:0/2216940874 wait complete. 2023-12-07T10:21:21.156 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-07T10:21:21.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.408+0000 7f2461dff700 1 -- 172.21.15.119:0/3317998095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f245c101fa0 msgr2=0x7f245c102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:21.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.408+0000 7f2461dff700 1 --2- 172.21.15.119:0/3317998095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f245c101fa0 0x7f245c102420 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f24500099c0 tx=0x7f2450009cd0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.408+0000 7f2461dff700 1 -- 172.21.15.119:0/3317998095 shutdown_connections 2023-12-07T10:21:21.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.408+0000 7f2461dff700 1 --2- 172.21.15.119:0/3317998095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f245c102960 0x7f245c10d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.408+0000 7f2461dff700 1 --2- 172.21.15.119:0/3317998095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f245c100820 0x7f245c100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.408+0000 7f2461dff700 1 --2- 172.21.15.119:0/3317998095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f245c101fa0 0x7f245c102420 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.408+0000 7f2461dff700 1 -- 172.21.15.119:0/3317998095 >> 172.21.15.119:0/3317998095 conn(0x7f245c077610 msgr2=0x7f245c077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:21.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.408+0000 7f2461dff700 1 -- 172.21.15.119:0/3317998095 shutdown_connections 2023-12-07T10:21:21.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.409+0000 7f2461dff700 1 -- 172.21.15.119:0/3317998095 wait complete. 2023-12-07T10:21:21.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.410+0000 7f2461dff700 1 Processor -- start 2023-12-07T10:21:21.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.410+0000 7f2461dff700 1 -- start start 2023-12-07T10:21:21.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.411+0000 7f2461dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f245c100820 0x7f245c199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.411+0000 7f2461dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f245c101fa0 0x7f245c19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.411+0000 7f2461dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f245c102960 0x7f245c19d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.411+0000 7f2461dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245c19db50 con 0x7f245c101fa0 2023-12-07T10:21:21.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.411+0000 7f2461dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245c19dcc0 con 0x7f245c102960 2023-12-07T10:21:21.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.411+0000 7f2461dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245c19de80 con 0x7f245c100820 2023-12-07T10:21:21.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.411+0000 7f245b7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f245c100820 0x7f245c199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.411+0000 7f245b7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f245c100820 0x7f245c199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:51936/0 (socket says 172.21.15.119:51936) 2023-12-07T10:21:21.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.411+0000 7f245b7fe700 1 -- 172.21.15.119:0/177424116 learned_addr learned my addr 172.21.15.119:0/177424116 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:21.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.411+0000 7f245affd700 1 --2- 172.21.15.119:0/177424116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f245c101fa0 0x7f245c19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.411+0000 7f245bfff700 1 --2- 172.21.15.119:0/177424116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f245c102960 0x7f245c19d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.412+0000 7f245b7fe700 1 -- 172.21.15.119:0/177424116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f245c102960 msgr2=0x7f245c19d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:21.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.412+0000 7f245b7fe700 1 --2- 172.21.15.119:0/177424116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f245c102960 0x7f245c19d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.412+0000 7f245b7fe700 1 -- 172.21.15.119:0/177424116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f245c101fa0 msgr2=0x7f245c19a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:21.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.412+0000 7f245b7fe700 1 --2- 172.21.15.119:0/177424116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f245c101fa0 0x7f245c19a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.412+0000 7f245b7fe700 1 -- 172.21.15.119:0/177424116 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24500095f0 con 0x7f245c100820 2023-12-07T10:21:21.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.412+0000 7f245b7fe700 1 --2- 172.21.15.119:0/177424116 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f245c100820 0x7f245c199d20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f244c00e9a0 tx=0x7f244c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:21.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.412+0000 7f2458ff9700 1 -- 172.21.15.119:0/177424116 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c009790 con 0x7f245c100820 2023-12-07T10:21:21.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.412+0000 7f2458ff9700 1 -- 172.21.15.119:0/177424116 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f244c010430 con 0x7f245c100820 2023-12-07T10:21:21.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.412+0000 7f2461dff700 1 -- 172.21.15.119:0/177424116 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f245c19e160 con 0x7f245c100820 2023-12-07T10:21:21.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.412+0000 7f2458ff9700 1 -- 172.21.15.119:0/177424116 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c018680 con 0x7f245c100820 2023-12-07T10:21:21.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.412+0000 7f2461dff700 1 -- 172.21.15.119:0/177424116 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f245c1a6ac0 con 0x7f245c100820 2023-12-07T10:21:21.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.417+0000 7f2458ff9700 1 -- 172.21.15.119:0/177424116 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) v1 ==== 136717+0+0 (secure 0 0 0) 0x7f244c0105a0 con 0x7f245c100820 2023-12-07T10:21:21.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.419+0000 7f2475692b80 1 Processor -- start 2023-12-07T10:21:21.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.419+0000 7f2475692b80 1 -- start start 2023-12-07T10:21:21.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.419+0000 7f2475692b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa9c20b40 0x560aa9c20f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.419+0000 7f2475692b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560aa9c21460 0x560aa9c24c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.419+0000 7f2475692b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560aa9c25150 0x560aa9c25570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.419+0000 7f2475692b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa9b947c0 con 0x560aa9c20b40 2023-12-07T10:21:21.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.419+0000 7f2475692b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa9b7b7a0 con 0x560aa9c25150 2023-12-07T10:21:21.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.419+0000 7f2475692b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa9b790c0 con 0x560aa9c21460 2023-12-07T10:21:21.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.419+0000 7f2458ff9700 1 --2- 172.21.15.119:0/177424116 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f24440a1da0 0x7f24440a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.420+0000 7f2458ff9700 1 -- 172.21.15.119:0/177424116 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f244c014070 con 0x7f245c100820 2023-12-07T10:21:21.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.420+0000 7f245b7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560aa9c21460 0x560aa9c24c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.420+0000 7f245bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa9c20b40 0x560aa9c20f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.420+0000 7f245b7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560aa9c21460 0x560aa9c24c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:51948/0 (socket says 172.21.15.119:51948) 2023-12-07T10:21:21.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.420+0000 7f245b7fe700 1 -- 172.21.15.119:0/695207408 learned_addr learned my addr 172.21.15.119:0/695207408 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:21.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.420+0000 7f245bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa9c20b40 0x560aa9c20f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38048/0 (socket says 172.21.15.119:38048) 2023-12-07T10:21:21.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.420+0000 7f245affd700 1 --2- 172.21.15.119:0/695207408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560aa9c25150 0x560aa9c25570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.420+0000 7f245bfff700 1 --2- 172.21.15.119:0/177424116 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f24440a1da0 0x7f24440a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.420+0000 7f245b7fe700 1 -- 172.21.15.119:0/695207408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560aa9c25150 msgr2=0x560aa9c25570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:21.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.420+0000 7f245b7fe700 1 --2- 172.21.15.119:0/695207408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560aa9c25150 0x560aa9c25570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.420+0000 7f245b7fe700 1 -- 172.21.15.119:0/695207408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa9c20b40 msgr2=0x560aa9c20f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:21.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.420+0000 7f245b7fe700 1 --2- 172.21.15.119:0/695207408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa9c20b40 0x560aa9c20f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.420+0000 7f245b7fe700 1 -- 172.21.15.119:0/695207408 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560aa99f1630 con 0x560aa9c21460 2023-12-07T10:21:21.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.421+0000 7f245affd700 1 --2- 172.21.15.119:0/695207408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560aa9c25150 0x560aa9c25570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:21:21.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.421+0000 7f245b7fe700 1 --2- 172.21.15.119:0/695207408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560aa9c21460 0x560aa9c24c10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f244c09a660 tx=0x7f244c09a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caca559ea8503bd1 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:21.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.421+0000 7f2441ffb700 1 -- 172.21.15.119:0/695207408 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c09a960 con 0x560aa9c21460 2023-12-07T10:21:21.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.421+0000 7f2441ffb700 1 -- 172.21.15.119:0/695207408 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f244c09eab0 con 0x560aa9c21460 2023-12-07T10:21:21.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.421+0000 7f245bfff700 1 --2- 172.21.15.119:0/177424116 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f24440a1da0 0x7f24440a4260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2454007fd0 tx=0x7f2454007d60 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:21.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.421+0000 7f2441ffb700 1 -- 172.21.15.119:0/695207408 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c0bd740 con 0x560aa9c21460 2023-12-07T10:21:21.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.422+0000 7f2475692b80 1 -- 172.21.15.119:0/695207408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560aa9c21460 msgr2=0x560aa9c24c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:21.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.422+0000 7f2475692b80 1 --2- 172.21.15.119:0/695207408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560aa9c21460 0x560aa9c24c10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f244c09a660 tx=0x7f244c09a340 comp rx=0 tx=0).stop 2023-12-07T10:21:21.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.422+0000 7f2475692b80 1 -- 172.21.15.119:0/695207408 shutdown_connections 2023-12-07T10:21:21.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.422+0000 7f2475692b80 1 --2- 172.21.15.119:0/695207408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560aa9c25150 0x560aa9c25570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.422+0000 7f2475692b80 1 --2- 172.21.15.119:0/695207408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa9c20b40 0x560aa9c20f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.422+0000 7f2475692b80 1 --2- 172.21.15.119:0/695207408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560aa9c21460 0x560aa9c24c10 secure :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f244c09a660 tx=0x7f244c09a340 comp rx=0 tx=0).stop 2023-12-07T10:21:21.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.422+0000 7f2475692b80 1 -- 172.21.15.119:0/695207408 >> 172.21.15.119:0/695207408 conn(0x560aa9b4f360 msgr2=0x560aa9c1b130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:21.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.422+0000 7f2475692b80 1 -- 172.21.15.119:0/695207408 shutdown_connections 2023-12-07T10:21:21.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.422+0000 7f2475692b80 1 -- 172.21.15.119:0/695207408 wait complete. 2023-12-07T10:21:21.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.423+0000 7f2475692b80 1 -- start start 2023-12-07T10:21:21.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.423+0000 7f2475692b80 1 Processor -- start 2023-12-07T10:21:21.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.423+0000 7f2475692b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560aa9c20b40 0x560aa9c29de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.423+0000 7f2475692b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa9c25150 0x560aa9c2a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.423+0000 7f2475692b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560aa9c2a860 0x560aa9c36430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.423+0000 7f2475692b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa9b749b0 con 0x560aa9c25150 2023-12-07T10:21:21.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.423+0000 7f2475692b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa9b2e560 con 0x560aa9c20b40 2023-12-07T10:21:21.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.423+0000 7f2475692b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa9b2e200 con 0x560aa9c2a860 2023-12-07T10:21:21.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.424+0000 7f2475692b80 10 client.0 _ll_register_callbacks cb 0x560aa9b15940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:21.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.424+0000 7f245b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560aa9c20b40 0x560aa9c29de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.424+0000 7f245b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560aa9c20b40 0x560aa9c29de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:47806/0 (socket says 172.21.15.119:47806) 2023-12-07T10:21:21.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.424+0000 7f245b7fe700 1 -- 172.21.15.119:0/2678766714 learned_addr learned my addr 172.21.15.119:0/2678766714 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:21.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.424+0000 7f245affd700 1 --2- 172.21.15.119:0/2678766714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa9c25150 0x560aa9c2a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.424+0000 7f245bfff700 1 --2- 172.21.15.119:0/2678766714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560aa9c2a860 0x560aa9c36430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.424+0000 7f245b7fe700 1 -- 172.21.15.119:0/2678766714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560aa9c2a860 msgr2=0x560aa9c36430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:21.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.424+0000 7f245b7fe700 1 --2- 172.21.15.119:0/2678766714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560aa9c2a860 0x560aa9c36430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.424+0000 7f245b7fe700 1 -- 172.21.15.119:0/2678766714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa9c25150 msgr2=0x560aa9c2a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:21.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.424+0000 7f245b7fe700 1 --2- 172.21.15.119:0/2678766714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa9c25150 0x560aa9c2a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.424+0000 7f245b7fe700 1 -- 172.21.15.119:0/2678766714 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f245c1a6ac0 con 0x560aa9c20b40 2023-12-07T10:21:21.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.424+0000 7f245b7fe700 1 --2- 172.21.15.119:0/2678766714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560aa9c20b40 0x560aa9c29de0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f244c00e970 tx=0x7f244c010c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b3c418d14f8741f server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:21.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.424+0000 7f243affd700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:21:21.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.425+0000 7f243affd700 1 -- 172.21.15.119:0/2678766714 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c09a960 con 0x560aa9c20b40 2023-12-07T10:21:21.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.425+0000 7f2475692b80 1 -- 172.21.15.119:0/2678766714 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560aa9c13b00 con 0x560aa9c20b40 2023-12-07T10:21:21.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.425+0000 7f243affd700 1 -- 172.21.15.119:0/2678766714 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f244c0ba120 con 0x560aa9c20b40 2023-12-07T10:21:21.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.425+0000 7f243affd700 1 -- 172.21.15.119:0/2678766714 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f244c0b3200 con 0x560aa9c20b40 2023-12-07T10:21:21.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.425+0000 7f2475692b80 10 client.26024 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:21.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.425+0000 7f2475692b80 1 -- 172.21.15.119:0/2678766714 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560aa999dde0 con 0x560aa9c20b40 2023-12-07T10:21:21.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.425+0000 7f243affd700 1 -- 172.21.15.119:0/2678766714 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560aa9c13b00 con 0x560aa9c20b40 2023-12-07T10:21:21.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.426+0000 7f243affd700 1 -- 172.21.15.119:0/2678766714 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f244c0bd6a0 con 0x560aa9c20b40 2023-12-07T10:21:21.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.426+0000 7f2475692b80 10 client.26024 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:21.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.426+0000 7f2475692b80 10 client.26024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:21:21.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.426+0000 7f2475692b80 20 client.26024 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:21.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.426+0000 7f2475692b80 1 --2- 172.21.15.119:0/2678766714 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560aa9c3a560 0x560aa9c3c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.426+0000 7f2475692b80 4 client.26024 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:21:21.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.426+0000 7f2475692b80 1 -- 172.21.15.119:0/2678766714 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560aa9bef7a0 con 0x560aa9c3a560 2023-12-07T10:21:21.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.426+0000 7f245affd700 1 --2- 172.21.15.119:0/2678766714 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560aa9c3a560 0x560aa9c3c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.427+0000 7f243affd700 10 client.26024 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:21.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.427+0000 7f245affd700 1 --2- 172.21.15.119:0/2678766714 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560aa9c3a560 0x560aa9c3c950 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a17059ed2895ff8 server_cookie=72814a31425a91c in_seq=0 out_seq=0 2023-12-07T10:21:21.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.428+0000 7f243affd700 1 -- 172.21.15.119:0/2678766714 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560aa9bef7a0 con 0x560aa9c3a560 2023-12-07T10:21:21.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.428+0000 7f243affd700 10 client.26024 handle_command_reply: tid=0 2023-12-07T10:21:21.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.428+0000 7f2475692b80 1 client.26024 shutdown 2023-12-07T10:21:21.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.428+0000 7f2475692b80 1 -- 172.21.15.119:0/2678766714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560aa9c20b40 msgr2=0x560aa9c29de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:21.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.428+0000 7f2475692b80 1 --2- 172.21.15.119:0/2678766714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560aa9c20b40 0x560aa9c29de0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f244c00e970 tx=0x7f244c010c80 comp rx=0 tx=0).stop 2023-12-07T10:21:21.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.428+0000 7f2475692b80 1 -- 172.21.15.119:0/2678766714 shutdown_connections 2023-12-07T10:21:21.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.428+0000 7f2475692b80 1 --2- 172.21.15.119:0/2678766714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560aa9c2a860 0x560aa9c36430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.428+0000 7f2475692b80 1 --2- 172.21.15.119:0/2678766714 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560aa9c3a560 0x560aa9c3c950 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.428+0000 7f2475692b80 1 --2- 172.21.15.119:0/2678766714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560aa9c20b40 0x560aa9c29de0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.428+0000 7f2475692b80 1 --2- 172.21.15.119:0/2678766714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa9c25150 0x560aa9c2a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.428+0000 7f2475692b80 1 -- 172.21.15.119:0/2678766714 >> 172.21.15.119:0/2678766714 conn(0x560aa9b4f360 msgr2=0x560aa9c15d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:21.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.428+0000 7f243affd700 0 client.26024 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:21.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.428+0000 7f2475692b80 1 -- 172.21.15.119:0/2678766714 shutdown_connections 2023-12-07T10:21:21.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.429+0000 7f2475692b80 1 -- 172.21.15.119:0/2678766714 wait complete. 2023-12-07T10:21:21.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.429+0000 7f2475692b80 20 client.26024 trim_cache size 0 max 16384 2023-12-07T10:21:21.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.442+0000 7f2475692b80 1 Processor -- start 2023-12-07T10:21:21.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.442+0000 7f2475692b80 1 -- start start 2023-12-07T10:21:21.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.443+0000 7f2475692b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560aa9c20b40 0x560aa9c31370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.443+0000 7f2475692b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa9c25150 0x560aa9c318b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.443+0000 7f2475692b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560aa9c2a860 0x560aa9c35e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.443+0000 7f2475692b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa99f3920 con 0x560aa9c25150 2023-12-07T10:21:21.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.443+0000 7f2475692b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa9c24f40 con 0x560aa9c2a860 2023-12-07T10:21:21.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.443+0000 7f2475692b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa9c41af0 con 0x560aa9c20b40 2023-12-07T10:21:21.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.443+0000 7f245b7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560aa9c20b40 0x560aa9c31370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.443+0000 7f245affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa9c25150 0x560aa9c318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.443+0000 7f245b7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560aa9c20b40 0x560aa9c31370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:51966/0 (socket says 172.21.15.119:51966) 2023-12-07T10:21:21.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.443+0000 7f245b7fe700 1 -- 172.21.15.119:0/2491260877 learned_addr learned my addr 172.21.15.119:0/2491260877 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:21.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.443+0000 7f245bfff700 1 --2- 172.21.15.119:0/2491260877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560aa9c2a860 0x560aa9c35e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.444+0000 7f245b7fe700 1 -- 172.21.15.119:0/2491260877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560aa9c2a860 msgr2=0x560aa9c35e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:21.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.444+0000 7f245b7fe700 1 --2- 172.21.15.119:0/2491260877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560aa9c2a860 0x560aa9c35e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.444+0000 7f245b7fe700 1 -- 172.21.15.119:0/2491260877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa9c25150 msgr2=0x560aa9c318b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:21.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.444+0000 7f245b7fe700 1 --2- 172.21.15.119:0/2491260877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa9c25150 0x560aa9c318b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.444+0000 7f245b7fe700 1 -- 172.21.15.119:0/2491260877 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560aa9986c60 con 0x560aa9c20b40 2023-12-07T10:21:21.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.444+0000 7f245b7fe700 1 --2- 172.21.15.119:0/2491260877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560aa9c20b40 0x560aa9c31370 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f244c0973d0 tx=0x7f244c09a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc7e90a7e0a0b4c server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:21.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.444+0000 7f24427fc700 1 -- 172.21.15.119:0/2491260877 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c0a2b10 con 0x560aa9c20b40 2023-12-07T10:21:21.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.444+0000 7f24427fc700 1 -- 172.21.15.119:0/2491260877 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f244c0b3200 con 0x560aa9c20b40 2023-12-07T10:21:21.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.444+0000 7f24427fc700 1 -- 172.21.15.119:0/2491260877 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c099e50 con 0x560aa9c20b40 2023-12-07T10:21:21.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.445+0000 7f2475692b80 1 -- 172.21.15.119:0/2491260877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560aa9c20b40 msgr2=0x560aa9c31370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:21.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.445+0000 7f2475692b80 1 --2- 172.21.15.119:0/2491260877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560aa9c20b40 0x560aa9c31370 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f244c0973d0 tx=0x7f244c09a340 comp rx=0 tx=0).stop 2023-12-07T10:21:21.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.445+0000 7f2475692b80 1 -- 172.21.15.119:0/2491260877 shutdown_connections 2023-12-07T10:21:21.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.445+0000 7f2475692b80 1 --2- 172.21.15.119:0/2491260877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560aa9c2a860 0x560aa9c35e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.445+0000 7f2475692b80 1 --2- 172.21.15.119:0/2491260877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560aa9c20b40 0x560aa9c31370 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.445+0000 7f2475692b80 1 --2- 172.21.15.119:0/2491260877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa9c25150 0x560aa9c318b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.445+0000 7f2475692b80 1 -- 172.21.15.119:0/2491260877 >> 172.21.15.119:0/2491260877 conn(0x560aa9b4f360 msgr2=0x560aa9c243d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:21.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.445+0000 7f2475692b80 1 -- 172.21.15.119:0/2491260877 shutdown_connections 2023-12-07T10:21:21.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.445+0000 7f2475692b80 1 -- 172.21.15.119:0/2491260877 wait complete. 2023-12-07T10:21:21.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.446+0000 7f2475692b80 1 -- start start 2023-12-07T10:21:21.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.446+0000 7f2475692b80 1 Processor -- start 2023-12-07T10:21:21.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.446+0000 7f2475692b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560aa9c20b40 0x560aa9c31650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.446+0000 7f2475692b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa9c25150 0x560aa9c26210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.446+0000 7f2475692b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560aa9c2a860 0x560aa9c26750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.446+0000 7f2475692b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa9c26e60 con 0x560aa9c25150 2023-12-07T10:21:21.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.446+0000 7f2475692b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa9c26fd0 con 0x560aa9c20b40 2023-12-07T10:21:21.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.446+0000 7f2475692b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560aa9c27190 con 0x560aa9c2a860 2023-12-07T10:21:21.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.446+0000 7f2475692b80 10 client.0 _ll_register_callbacks cb 0x560aa9b15940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:21.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.447+0000 7f245b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560aa9c20b40 0x560aa9c31650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.447+0000 7f245b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560aa9c20b40 0x560aa9c31650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:47838/0 (socket says 172.21.15.119:47838) 2023-12-07T10:21:21.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.447+0000 7f245b7fe700 1 -- 172.21.15.119:0/1966460400 learned_addr learned my addr 172.21.15.119:0/1966460400 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:21.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.447+0000 7f245affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa9c25150 0x560aa9c26210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.447+0000 7f245bfff700 1 --2- 172.21.15.119:0/1966460400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560aa9c2a860 0x560aa9c26750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.447+0000 7f245b7fe700 1 -- 172.21.15.119:0/1966460400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560aa9c2a860 msgr2=0x560aa9c26750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:21.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.447+0000 7f245b7fe700 1 --2- 172.21.15.119:0/1966460400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560aa9c2a860 0x560aa9c26750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.447+0000 7f245b7fe700 1 -- 172.21.15.119:0/1966460400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa9c25150 msgr2=0x560aa9c26210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:21.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.447+0000 7f245b7fe700 1 --2- 172.21.15.119:0/1966460400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa9c25150 0x560aa9c26210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.447+0000 7f245b7fe700 1 -- 172.21.15.119:0/1966460400 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560aa999dde0 con 0x560aa9c20b40 2023-12-07T10:21:21.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.448+0000 7f245b7fe700 1 --2- 172.21.15.119:0/1966460400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560aa9c20b40 0x560aa9c31650 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f244c0cb580 tx=0x7f244c00c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5afd8c2e0c3ed75 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:21.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.448+0000 7f243affd700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:21:21.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.448+0000 7f243affd700 1 -- 172.21.15.119:0/1966460400 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c0bd850 con 0x560aa9c20b40 2023-12-07T10:21:21.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.448+0000 7f2475692b80 1 -- 172.21.15.119:0/1966460400 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560aa9b3afd0 con 0x560aa9c20b40 2023-12-07T10:21:21.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.448+0000 7f243affd700 1 -- 172.21.15.119:0/1966460400 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f244c09a960 con 0x560aa9c20b40 2023-12-07T10:21:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.448+0000 7f243affd700 1 -- 172.21.15.119:0/1966460400 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f244c09a730 con 0x560aa9c20b40 2023-12-07T10:21:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.448+0000 7f2475692b80 10 client.26030 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.448+0000 7f2475692b80 1 -- 172.21.15.119:0/1966460400 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560aa99c4250 con 0x560aa9c20b40 2023-12-07T10:21:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.448+0000 7f243affd700 1 -- 172.21.15.119:0/1966460400 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560aa9b3afd0 con 0x560aa9c20b40 2023-12-07T10:21:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.449+0000 7f243affd700 1 -- 172.21.15.119:0/1966460400 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f244c096750 con 0x560aa9c20b40 2023-12-07T10:21:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.449+0000 7f2475692b80 10 client.26030 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.449+0000 7f2475692b80 10 client.26030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:21:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.449+0000 7f2475692b80 20 client.26030 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.449+0000 7f2475692b80 1 --2- 172.21.15.119:0/1966460400 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560aa9c90e90 0x560aa9c93280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.449+0000 7f2475692b80 4 client.26030 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:21:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.449+0000 7f2475692b80 1 -- 172.21.15.119:0/1966460400 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560aa9b2d490 con 0x560aa9c90e90 2023-12-07T10:21:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.449+0000 7f245affd700 1 --2- 172.21.15.119:0/1966460400 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560aa9c90e90 0x560aa9c93280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.450+0000 7f243affd700 10 client.26030 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:21.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.450+0000 7f245affd700 1 --2- 172.21.15.119:0/1966460400 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560aa9c90e90 0x560aa9c93280 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93ca4031375245f3 server_cookie=15202d091258f6ec in_seq=0 out_seq=0 2023-12-07T10:21:21.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.450+0000 7f243affd700 1 -- 172.21.15.119:0/1966460400 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560aa9b2d490 con 0x560aa9c90e90 2023-12-07T10:21:21.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.450+0000 7f243affd700 10 client.26030 handle_command_reply: tid=0 2023-12-07T10:21:21.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.450+0000 7f2475692b80 1 client.26030 shutdown 2023-12-07T10:21:21.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.450+0000 7f2475692b80 1 -- 172.21.15.119:0/1966460400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560aa9c20b40 msgr2=0x560aa9c31650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:21.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.450+0000 7f2475692b80 1 --2- 172.21.15.119:0/1966460400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560aa9c20b40 0x560aa9c31650 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f244c0cb580 tx=0x7f244c00c400 comp rx=0 tx=0).stop 2023-12-07T10:21:21.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.450+0000 7f2475692b80 1 -- 172.21.15.119:0/1966460400 shutdown_connections 2023-12-07T10:21:21.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.450+0000 7f2475692b80 1 --2- 172.21.15.119:0/1966460400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560aa9c2a860 0x560aa9c26750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.450+0000 7f2475692b80 1 --2- 172.21.15.119:0/1966460400 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560aa9c90e90 0x560aa9c93280 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.451+0000 7f2475692b80 1 --2- 172.21.15.119:0/1966460400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560aa9c20b40 0x560aa9c31650 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.451+0000 7f2475692b80 1 --2- 172.21.15.119:0/1966460400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa9c25150 0x560aa9c26210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.451+0000 7f2475692b80 1 -- 172.21.15.119:0/1966460400 >> 172.21.15.119:0/1966460400 conn(0x560aa9b4f360 msgr2=0x560aa9c35e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:21.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.451+0000 7f243affd700 0 client.26030 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:21.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.451+0000 7f2475692b80 1 -- 172.21.15.119:0/1966460400 shutdown_connections 2023-12-07T10:21:21.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.451+0000 7f2475692b80 1 -- 172.21.15.119:0/1966460400 wait complete. 2023-12-07T10:21:21.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.451+0000 7f2475692b80 20 client.26030 trim_cache size 0 max 16384 2023-12-07T10:21:21.456 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:21:21.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.452+0000 7f2461dff700 1 -- 172.21.15.119:0/177424116 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f24440a1da0 msgr2=0x7f24440a4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:21.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.452+0000 7f2461dff700 1 --2- 172.21.15.119:0/177424116 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f24440a1da0 0x7f24440a4260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2454007fd0 tx=0x7f2454007d60 comp rx=0 tx=0).stop 2023-12-07T10:21:21.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.452+0000 7f2461dff700 1 -- 172.21.15.119:0/177424116 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f245c100820 msgr2=0x7f245c199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:21.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.452+0000 7f2461dff700 1 --2- 172.21.15.119:0/177424116 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f245c100820 0x7f245c199d20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f244c00e9a0 tx=0x7f244c00ecb0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.452+0000 7f2461dff700 1 -- 172.21.15.119:0/177424116 shutdown_connections 2023-12-07T10:21:21.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.452+0000 7f2461dff700 1 --2- 172.21.15.119:0/177424116 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f24440a1da0 0x7f24440a4260 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.452+0000 7f2461dff700 1 --2- 172.21.15.119:0/177424116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f245c102960 0x7f245c19d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.452+0000 7f2461dff700 1 --2- 172.21.15.119:0/177424116 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f245c100820 0x7f245c199d20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.452+0000 7f2461dff700 1 --2- 172.21.15.119:0/177424116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f245c101fa0 0x7f245c19a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:21.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.452+0000 7f2461dff700 1 -- 172.21.15.119:0/177424116 >> 172.21.15.119:0/177424116 conn(0x7f245c077610 msgr2=0x7f245c1030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:21.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.452+0000 7f2461dff700 1 -- 172.21.15.119:0/177424116 shutdown_connections 2023-12-07T10:21:21.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:21.452+0000 7f2461dff700 1 -- 172.21.15.119:0/177424116 wait complete. 2023-12-07T10:21:22.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:21 smithi119 ceph-mon[97908]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2023-12-07T10:21:22.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:21 smithi159 ceph-mon[134726]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2023-12-07T10:21:22.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:21 smithi153 ceph-mon[133625]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2023-12-07T10:21:22.481 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-07T10:21:22.481 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-07T10:21:22.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.732+0000 7f1d8759e700 1 -- 172.21.15.119:0/2646493125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d88102ed0 msgr2=0x7f1d8810b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:22.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.732+0000 7f1d8759e700 1 --2- 172.21.15.119:0/2646493125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d88102ed0 0x7f1d8810b3b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c009990 tx=0x7f1d7c009ca0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.732+0000 7f1d8759e700 1 -- 172.21.15.119:0/2646493125 shutdown_connections 2023-12-07T10:21:22.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.732+0000 7f1d8759e700 1 --2- 172.21.15.119:0/2646493125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d8810b9d0 0x7f1d8810de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.732+0000 7f1d8759e700 1 --2- 172.21.15.119:0/2646493125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d881025b0 0x7f1d88102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.732+0000 7f1d8759e700 1 --2- 172.21.15.119:0/2646493125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d88102ed0 0x7f1d8810b3b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.732+0000 7f1d8759e700 1 -- 172.21.15.119:0/2646493125 >> 172.21.15.119:0/2646493125 conn(0x7f1d880fbdd0 msgr2=0x7f1d880fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:22.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.732+0000 7f1d8759e700 1 -- 172.21.15.119:0/2646493125 shutdown_connections 2023-12-07T10:21:22.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.732+0000 7f1d8759e700 1 -- 172.21.15.119:0/2646493125 wait complete. 2023-12-07T10:21:22.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.733+0000 7f1d8759e700 1 Processor -- start 2023-12-07T10:21:22.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.734+0000 7f1d8759e700 1 -- start start 2023-12-07T10:21:22.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.734+0000 7f1d8759e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d881025b0 0x7f1d8819d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:22.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.734+0000 7f1d8759e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d88102ed0 0x7f1d8819db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:22.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.734+0000 7f1d8759e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d8810b9d0 0x7f1d88197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:22.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.734+0000 7f1d8759e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d88197d80 con 0x7f1d88102ed0 2023-12-07T10:21:22.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.734+0000 7f1d8759e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d88197ef0 con 0x7f1d8810b9d0 2023-12-07T10:21:22.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.734+0000 7f1d8759e700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d881980b0 con 0x7f1d881025b0 2023-12-07T10:21:22.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.734+0000 7f1d85d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d88102ed0 0x7f1d8819db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:22.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.734+0000 7f1d8659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d881025b0 0x7f1d8819d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:22.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.734+0000 7f1d85d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d88102ed0 0x7f1d8819db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38088/0 (socket says 172.21.15.119:38088) 2023-12-07T10:21:22.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.734+0000 7f1d85d9b700 1 -- 172.21.15.119:0/1769721597 learned_addr learned my addr 172.21.15.119:0/1769721597 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:22.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.735+0000 7f1d86d9d700 1 --2- 172.21.15.119:0/1769721597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d8810b9d0 0x7f1d88197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:22.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.735+0000 7f1d8659c700 1 -- 172.21.15.119:0/1769721597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d8810b9d0 msgr2=0x7f1d88197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:22.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.735+0000 7f1d8659c700 1 --2- 172.21.15.119:0/1769721597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d8810b9d0 0x7f1d88197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.735+0000 7f1d8659c700 1 -- 172.21.15.119:0/1769721597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d88102ed0 msgr2=0x7f1d8819db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:22.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.735+0000 7f1d8659c700 1 --2- 172.21.15.119:0/1769721597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d88102ed0 0x7f1d8819db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.735+0000 7f1d8659c700 1 -- 172.21.15.119:0/1769721597 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c0095f0 con 0x7f1d881025b0 2023-12-07T10:21:22.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.735+0000 7f1d8659c700 1 --2- 172.21.15.119:0/1769721597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d881025b0 0x7f1d8819d5f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1d78009f20 tx=0x7f1d7800eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:22.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.735+0000 7f1d777fe700 1 -- 172.21.15.119:0/1769721597 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d78013d00 con 0x7f1d881025b0 2023-12-07T10:21:22.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.735+0000 7f1d8759e700 1 -- 172.21.15.119:0/1769721597 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d88198390 con 0x7f1d881025b0 2023-12-07T10:21:22.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.736+0000 7f1d777fe700 1 -- 172.21.15.119:0/1769721597 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d78013e60 con 0x7f1d881025b0 2023-12-07T10:21:22.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.736+0000 7f1d777fe700 1 -- 172.21.15.119:0/1769721597 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7800c8f0 con 0x7f1d881025b0 2023-12-07T10:21:22.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.736+0000 7f1d8759e700 1 -- 172.21.15.119:0/1769721597 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d88100310 con 0x7f1d881025b0 2023-12-07T10:21:22.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.740+0000 7f1d777fe700 1 -- 172.21.15.119:0/1769721597 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) v1 ==== 136717+0+0 (secure 0 0 0) 0x7f1d7801bb70 con 0x7f1d881025b0 2023-12-07T10:21:22.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.742+0000 7f1da06dfb80 1 Processor -- start 2023-12-07T10:21:22.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.742+0000 7f1da06dfb80 1 -- start start 2023-12-07T10:21:22.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.742+0000 7f1d777fe700 1 --2- 172.21.15.119:0/1769721597 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f1d700a1da0 0x7f1d700a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:22.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.742+0000 7f1da06dfb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c8707270 0x5603c8707650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:22.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.742+0000 7f1d777fe700 1 -- 172.21.15.119:0/1769721597 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f1d780d2020 con 0x7f1d881025b0 2023-12-07T10:21:22.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.742+0000 7f1da06dfb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603c8707b90 0x5603c870b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:22.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.742+0000 7f1da06dfb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c870bbd0 0x5603c870c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:22.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.743+0000 7f1d85d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603c8707b90 0x5603c870b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:22.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.743+0000 7f1d8659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c8707270 0x5603c8707650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:22.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.743+0000 7f1d85d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603c8707b90 0x5603c870b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:47858/0 (socket says 172.21.15.119:47858) 2023-12-07T10:21:22.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.743+0000 7f1d8659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c8707270 0x5603c8707650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38100/0 (socket says 172.21.15.119:38100) 2023-12-07T10:21:22.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.743+0000 7f1d86d9d700 1 --2- 172.21.15.119:0/1769721597 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f1d700a1da0 0x7f1d700a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:22.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.743+0000 7f1d85d9b700 1 -- 172.21.15.119:0/1301739316 learned_addr learned my addr 172.21.15.119:0/1301739316 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:22.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.743+0000 7f1da06dfb80 1 -- 172.21.15.119:0/1301739316 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c8672840 con 0x5603c8707270 2023-12-07T10:21:22.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.743+0000 7f1da06dfb80 1 -- 172.21.15.119:0/1301739316 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c8659820 con 0x5603c8707b90 2023-12-07T10:21:22.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.743+0000 7f1da06dfb80 1 -- 172.21.15.119:0/1301739316 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c8657140 con 0x5603c870bbd0 2023-12-07T10:21:22.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.743+0000 7f1d85d9b700 1 -- 172.21.15.119:0/1301739316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c870bbd0 msgr2=0x5603c870c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:22.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.743+0000 7f1d85d9b700 1 --2- 172.21.15.119:0/1301739316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c870bbd0 0x5603c870c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.743+0000 7f1d85d9b700 1 -- 172.21.15.119:0/1301739316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c8707270 msgr2=0x5603c8707650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:22.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.743+0000 7f1d85d9b700 1 --2- 172.21.15.119:0/1301739316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c8707270 0x5603c8707650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.743+0000 7f1d85d9b700 1 -- 172.21.15.119:0/1301739316 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603c84cf6b0 con 0x5603c8707b90 2023-12-07T10:21:22.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.744+0000 7f1d86d9d700 1 --2- 172.21.15.119:0/1769721597 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f1d700a1da0 0x7f1d700a4260 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000f540 tx=0x7f1d80009920 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:22.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.744+0000 7f1d85d9b700 1 --2- 172.21.15.119:0/1301739316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603c8707b90 0x5603c870b690 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c005f50 tx=0x7f1d7c00b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57137c3024301770 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:22.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.744+0000 7f1d74ff9700 1 -- 172.21.15.119:0/1301739316 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c01d070 con 0x5603c8707b90 2023-12-07T10:21:22.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.744+0000 7f1d74ff9700 1 -- 172.21.15.119:0/1301739316 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d7c01e040 con 0x5603c8707b90 2023-12-07T10:21:22.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.744+0000 7f1d74ff9700 1 -- 172.21.15.119:0/1301739316 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c0048c0 con 0x5603c8707b90 2023-12-07T10:21:22.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.745+0000 7f1da06dfb80 1 -- 172.21.15.119:0/1301739316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603c8707b90 msgr2=0x5603c870b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:22.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.745+0000 7f1da06dfb80 1 --2- 172.21.15.119:0/1301739316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603c8707b90 0x5603c870b690 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c005f50 tx=0x7f1d7c00b610 comp rx=0 tx=0).stop 2023-12-07T10:21:22.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.745+0000 7f1da06dfb80 1 -- 172.21.15.119:0/1301739316 shutdown_connections 2023-12-07T10:21:22.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.745+0000 7f1da06dfb80 1 --2- 172.21.15.119:0/1301739316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c870bbd0 0x5603c870c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.745+0000 7f1da06dfb80 1 --2- 172.21.15.119:0/1301739316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c8707270 0x5603c8707650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.745+0000 7f1da06dfb80 1 --2- 172.21.15.119:0/1301739316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603c8707b90 0x5603c870b690 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.745+0000 7f1da06dfb80 1 -- 172.21.15.119:0/1301739316 >> 172.21.15.119:0/1301739316 conn(0x5603c862d3e0 msgr2=0x5603c86f91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:22.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.745+0000 7f1da06dfb80 1 -- 172.21.15.119:0/1301739316 shutdown_connections 2023-12-07T10:21:22.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.745+0000 7f1da06dfb80 1 -- 172.21.15.119:0/1301739316 wait complete. 2023-12-07T10:21:22.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.746+0000 7f1da06dfb80 1 -- start start 2023-12-07T10:21:22.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.746+0000 7f1da06dfb80 1 Processor -- start 2023-12-07T10:21:22.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.747+0000 7f1da06dfb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8707270 0x5603c87132c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:22.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.747+0000 7f1da06dfb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603c8707b90 0x5603c87105b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:22.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.747+0000 7f1da06dfb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c870bbd0 0x5603c8710bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:22.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.747+0000 7f1da06dfb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c8652a30 con 0x5603c870bbd0 2023-12-07T10:21:22.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.747+0000 7f1da06dfb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c860c5e0 con 0x5603c8707b90 2023-12-07T10:21:22.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.747+0000 7f1da06dfb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c860c280 con 0x5603c8707270 2023-12-07T10:21:22.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.747+0000 7f1da06dfb80 10 client.0 _ll_register_callbacks cb 0x5603c85f39c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:22.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.748+0000 7f1d86d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c870bbd0 0x5603c8710bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:22.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.748+0000 7f1d8659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8707270 0x5603c87132c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:22.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.748+0000 7f1d86d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c870bbd0 0x5603c8710bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38112/0 (socket says 172.21.15.119:38112) 2023-12-07T10:21:22.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.748+0000 7f1d8659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8707270 0x5603c87132c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:52018/0 (socket says 172.21.15.119:52018) 2023-12-07T10:21:22.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.748+0000 7f1d86d9d700 1 -- 172.21.15.119:0/763832365 learned_addr learned my addr 172.21.15.119:0/763832365 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:22.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.748+0000 7f1d85d9b700 1 --2- 172.21.15.119:0/763832365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603c8707b90 0x5603c87105b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:22.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.748+0000 7f1d86d9d700 1 -- 172.21.15.119:0/763832365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8707270 msgr2=0x5603c87132c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:22.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.748+0000 7f1d86d9d700 1 --2- 172.21.15.119:0/763832365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8707270 0x5603c87132c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.748+0000 7f1d86d9d700 1 -- 172.21.15.119:0/763832365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603c8707b90 msgr2=0x5603c87105b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:22.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.748+0000 7f1d86d9d700 1 --2- 172.21.15.119:0/763832365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603c8707b90 0x5603c87105b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.748+0000 7f1d86d9d700 1 -- 172.21.15.119:0/763832365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d8810eb50 con 0x5603c870bbd0 2023-12-07T10:21:22.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.749+0000 7f1d86d9d700 1 --2- 172.21.15.119:0/763832365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c870bbd0 0x5603c8710bc0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1d80017dc0 tx=0x7f1d80004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a1047e8ddbda42 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:22.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.749+0000 7f1d6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:21:22.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.749+0000 7f1d6dffb700 1 -- 172.21.15.119:0/763832365 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8000e040 con 0x5603c870bbd0 2023-12-07T10:21:22.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.749+0000 7f1da06dfb80 1 -- 172.21.15.119:0/763832365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5603c86f1a20 con 0x5603c870bbd0 2023-12-07T10:21:22.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.749+0000 7f1d6dffb700 1 -- 172.21.15.119:0/763832365 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f1d80019af0 con 0x5603c870bbd0 2023-12-07T10:21:22.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.750+0000 7f1d6dffb700 1 -- 172.21.15.119:0/763832365 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d80017ea0 con 0x5603c870bbd0 2023-12-07T10:21:22.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.750+0000 7f1da06dfb80 10 client.16029 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:22.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.750+0000 7f1da06dfb80 1 -- 172.21.15.119:0/763832365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5603c847be60 con 0x5603c870bbd0 2023-12-07T10:21:22.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.750+0000 7f1d6dffb700 1 -- 172.21.15.119:0/763832365 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5603c86f1a20 con 0x5603c870bbd0 2023-12-07T10:21:22.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.750+0000 7f1d6dffb700 1 -- 172.21.15.119:0/763832365 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1d800163d0 con 0x5603c870bbd0 2023-12-07T10:21:22.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.750+0000 7f1da06dfb80 10 client.16029 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:22.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.750+0000 7f1da06dfb80 10 client.16029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:21:22.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.750+0000 7f1da06dfb80 20 client.16029 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:22.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.750+0000 7f1da06dfb80 1 --2- 172.21.15.119:0/763832365 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5603c8729580 0x5603c872b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:22.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.750+0000 7f1da06dfb80 4 client.16029 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:21:22.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.750+0000 7f1da06dfb80 1 -- 172.21.15.119:0/763832365 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5603c86cd820 con 0x5603c8729580 2023-12-07T10:21:22.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.751+0000 7f1d8659c700 1 --2- 172.21.15.119:0/763832365 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5603c8729580 0x5603c872b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:22.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.751+0000 7f1d8659c700 1 --2- 172.21.15.119:0/763832365 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5603c8729580 0x5603c872b9a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=158a08dbdc7146f server_cookie=d849815522719190 in_seq=0 out_seq=0 2023-12-07T10:21:22.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.751+0000 7f1d6dffb700 10 client.16029 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:22.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.752+0000 7f1d6dffb700 1 -- 172.21.15.119:0/763832365 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5603c86cd820 con 0x5603c8729580 2023-12-07T10:21:22.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.752+0000 7f1d6dffb700 10 client.16029 handle_command_reply: tid=0 2023-12-07T10:21:22.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.752+0000 7f1da06dfb80 1 client.16029 shutdown 2023-12-07T10:21:22.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.752+0000 7f1da06dfb80 1 -- 172.21.15.119:0/763832365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c870bbd0 msgr2=0x5603c8710bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:22.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.752+0000 7f1da06dfb80 1 --2- 172.21.15.119:0/763832365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c870bbd0 0x5603c8710bc0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1d80017dc0 tx=0x7f1d80004b40 comp rx=0 tx=0).stop 2023-12-07T10:21:22.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.753+0000 7f1da06dfb80 1 -- 172.21.15.119:0/763832365 shutdown_connections 2023-12-07T10:21:22.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.753+0000 7f1da06dfb80 1 --2- 172.21.15.119:0/763832365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c870bbd0 0x5603c8710bc0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.753+0000 7f1da06dfb80 1 --2- 172.21.15.119:0/763832365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8707270 0x5603c87132c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.753+0000 7f1da06dfb80 1 --2- 172.21.15.119:0/763832365 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5603c8729580 0x5603c872b9a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.753+0000 7f1da06dfb80 1 --2- 172.21.15.119:0/763832365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603c8707b90 0x5603c87105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.753+0000 7f1da06dfb80 1 -- 172.21.15.119:0/763832365 >> 172.21.15.119:0/763832365 conn(0x5603c862d3e0 msgr2=0x5603c870c780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:22.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.753+0000 7f1d6dffb700 0 client.16029 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:22.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.753+0000 7f1da06dfb80 1 -- 172.21.15.119:0/763832365 shutdown_connections 2023-12-07T10:21:22.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.753+0000 7f1da06dfb80 1 -- 172.21.15.119:0/763832365 wait complete. 2023-12-07T10:21:22.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.753+0000 7f1da06dfb80 20 client.16029 trim_cache size 0 max 16384 2023-12-07T10:21:22.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.786+0000 7f1da06dfb80 1 Processor -- start 2023-12-07T10:21:22.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.786+0000 7f1da06dfb80 1 -- start start 2023-12-07T10:21:22.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.787+0000 7f1da06dfb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8707270 0x5603c870e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:22.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.787+0000 7f1da06dfb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c8707b90 0x5603c870e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:22.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.787+0000 7f1da06dfb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603c870bbd0 0x5603c8702d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:22.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.787+0000 7f1da06dfb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c84d19a0 con 0x5603c8707b90 2023-12-07T10:21:22.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.787+0000 7f1da06dfb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c86f8f00 con 0x5603c870bbd0 2023-12-07T10:21:22.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.787+0000 7f1da06dfb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c86f5bf0 con 0x5603c8707270 2023-12-07T10:21:22.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.787+0000 7f1d8659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8707270 0x5603c870e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:22.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.787+0000 7f1d8659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8707270 0x5603c870e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:52030/0 (socket says 172.21.15.119:52030) 2023-12-07T10:21:22.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.787+0000 7f1d8659c700 1 -- 172.21.15.119:0/1611579687 learned_addr learned my addr 172.21.15.119:0/1611579687 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:22.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.787+0000 7f1d85d9b700 1 --2- 172.21.15.119:0/1611579687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c8707b90 0x5603c870e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:22.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.788+0000 7f1d86d9d700 1 --2- 172.21.15.119:0/1611579687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603c870bbd0 0x5603c8702d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:22.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.788+0000 7f1d8659c700 1 -- 172.21.15.119:0/1611579687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603c870bbd0 msgr2=0x5603c8702d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:22.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.788+0000 7f1d8659c700 1 --2- 172.21.15.119:0/1611579687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603c870bbd0 0x5603c8702d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.788+0000 7f1d8659c700 1 -- 172.21.15.119:0/1611579687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c8707b90 msgr2=0x5603c870e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:22.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.788+0000 7f1d8659c700 1 --2- 172.21.15.119:0/1611579687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c8707b90 0x5603c870e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.788+0000 7f1d8659c700 1 -- 172.21.15.119:0/1611579687 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603c8464ce0 con 0x5603c8707270 2023-12-07T10:21:22.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.788+0000 7f1d8659c700 1 --2- 172.21.15.119:0/1611579687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8707270 0x5603c870e040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1d7809e630 tx=0x7f1d78009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7843f6d58af65de server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:22.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.788+0000 7f1d757fa700 1 -- 172.21.15.119:0/1611579687 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d780a6cc0 con 0x5603c8707270 2023-12-07T10:21:22.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.788+0000 7f1d757fa700 1 -- 172.21.15.119:0/1611579687 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d7809ccd0 con 0x5603c8707270 2023-12-07T10:21:22.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.788+0000 7f1d757fa700 1 -- 172.21.15.119:0/1611579687 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7809e060 con 0x5603c8707270 2023-12-07T10:21:22.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.789+0000 7f1da06dfb80 1 -- 172.21.15.119:0/1611579687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8707270 msgr2=0x5603c870e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:22.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.789+0000 7f1da06dfb80 1 --2- 172.21.15.119:0/1611579687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8707270 0x5603c870e040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1d7809e630 tx=0x7f1d78009790 comp rx=0 tx=0).stop 2023-12-07T10:21:22.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.789+0000 7f1da06dfb80 1 -- 172.21.15.119:0/1611579687 shutdown_connections 2023-12-07T10:21:22.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.789+0000 7f1da06dfb80 1 --2- 172.21.15.119:0/1611579687 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603c870bbd0 0x5603c8702d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.789+0000 7f1da06dfb80 1 --2- 172.21.15.119:0/1611579687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8707270 0x5603c870e040 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.789+0000 7f1da06dfb80 1 --2- 172.21.15.119:0/1611579687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c8707b90 0x5603c870e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.789+0000 7f1da06dfb80 1 -- 172.21.15.119:0/1611579687 >> 172.21.15.119:0/1611579687 conn(0x5603c862d3e0 msgr2=0x5603c8730300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:22.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.789+0000 7f1da06dfb80 1 -- 172.21.15.119:0/1611579687 shutdown_connections 2023-12-07T10:21:22.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.790+0000 7f1da06dfb80 1 -- 172.21.15.119:0/1611579687 wait complete. 2023-12-07T10:21:22.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.790+0000 7f1da06dfb80 1 -- start start 2023-12-07T10:21:22.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.790+0000 7f1da06dfb80 1 Processor -- start 2023-12-07T10:21:22.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.791+0000 7f1da06dfb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8707270 0x5603c8703010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:22.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.791+0000 7f1da06dfb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603c8707b90 0x5603c86fd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:22.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.791+0000 7f1da06dfb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c870bbd0 0x5603c86fdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:22.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.791+0000 7f1da06dfb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c86fe410 con 0x5603c870bbd0 2023-12-07T10:21:22.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.791+0000 7f1da06dfb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c86fe580 con 0x5603c8707b90 2023-12-07T10:21:22.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.791+0000 7f1da06dfb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c877fa80 con 0x5603c8707270 2023-12-07T10:21:22.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.791+0000 7f1da06dfb80 10 client.0 _ll_register_callbacks cb 0x5603c85f39c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:22.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.791+0000 7f1d8659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8707270 0x5603c8703010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:22.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.791+0000 7f1d8659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8707270 0x5603c8703010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:52046/0 (socket says 172.21.15.119:52046) 2023-12-07T10:21:22.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.791+0000 7f1d8659c700 1 -- 172.21.15.119:0/273579022 learned_addr learned my addr 172.21.15.119:0/273579022 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:22.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.792+0000 7f1d86d9d700 1 --2- 172.21.15.119:0/273579022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c870bbd0 0x5603c86fdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:22.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.792+0000 7f1d85d9b700 1 --2- 172.21.15.119:0/273579022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603c8707b90 0x5603c86fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:22.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.792+0000 7f1d8659c700 1 -- 172.21.15.119:0/273579022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603c8707b90 msgr2=0x5603c86fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:22.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.792+0000 7f1d8659c700 1 --2- 172.21.15.119:0/273579022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603c8707b90 0x5603c86fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.792+0000 7f1d8659c700 1 -- 172.21.15.119:0/273579022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c870bbd0 msgr2=0x5603c86fdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:22.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.792+0000 7f1d8659c700 1 --2- 172.21.15.119:0/273579022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c870bbd0 0x5603c86fdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.792+0000 7f1d8659c700 1 -- 172.21.15.119:0/273579022 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d88100310 con 0x5603c8707270 2023-12-07T10:21:22.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.793+0000 7f1d8659c700 1 --2- 172.21.15.119:0/273579022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8707270 0x5603c8703010 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1d78009f50 tx=0x7f1d7809e780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=758b21742c5809fe server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:22.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.793+0000 7f1d6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:21:22.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.793+0000 7f1d6dffb700 1 -- 172.21.15.119:0/273579022 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7809e020 con 0x5603c8707270 2023-12-07T10:21:22.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.793+0000 7f1d6dffb700 1 -- 172.21.15.119:0/273579022 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f1d7809eb90 con 0x5603c8707270 2023-12-07T10:21:22.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.793+0000 7f1da06dfb80 1 -- 172.21.15.119:0/273579022 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d7c01e040 con 0x5603c8707270 2023-12-07T10:21:22.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.793+0000 7f1d6dffb700 1 -- 172.21.15.119:0/273579022 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d780b7220 con 0x5603c8707270 2023-12-07T10:21:22.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.793+0000 7f1da06dfb80 10 client.26035 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:22.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.793+0000 7f1da06dfb80 1 -- 172.21.15.119:0/273579022 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5603c84a22d0 con 0x5603c8707270 2023-12-07T10:21:22.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.794+0000 7f1d6dffb700 1 -- 172.21.15.119:0/273579022 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d7c01e040 con 0x5603c8707270 2023-12-07T10:21:22.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.794+0000 7f1d6dffb700 1 -- 172.21.15.119:0/273579022 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1d780cf660 con 0x5603c8707270 2023-12-07T10:21:22.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.794+0000 7f1da06dfb80 10 client.26035 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:22.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.794+0000 7f1da06dfb80 10 client.26035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:21:22.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.794+0000 7f1da06dfb80 20 client.26035 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:22.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.794+0000 7f1da06dfb80 1 --2- 172.21.15.119:0/273579022 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5603c8780c90 0x5603c87830b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:22.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.794+0000 7f1da06dfb80 4 client.26035 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-07T10:21:22.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.794+0000 7f1da06dfb80 1 -- 172.21.15.119:0/273579022 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5603c8619050 con 0x5603c8780c90 2023-12-07T10:21:22.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.794+0000 7f1d85d9b700 1 --2- 172.21.15.119:0/273579022 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5603c8780c90 0x5603c87830b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:22.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.795+0000 7f1d6dffb700 10 client.26035 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:22.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.795+0000 7f1d85d9b700 1 --2- 172.21.15.119:0/273579022 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5603c8780c90 0x5603c87830b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3fccd48e42277ea server_cookie=c45671dd7d449c69 in_seq=0 out_seq=0 2023-12-07T10:21:22.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.796+0000 7f1d6dffb700 1 -- 172.21.15.119:0/273579022 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5603c8619050 con 0x5603c8780c90 2023-12-07T10:21:22.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.796+0000 7f1d6dffb700 10 client.26035 handle_command_reply: tid=0 2023-12-07T10:21:22.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.796+0000 7f1da06dfb80 1 client.26035 shutdown 2023-12-07T10:21:22.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.796+0000 7f1da06dfb80 1 -- 172.21.15.119:0/273579022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8707270 msgr2=0x5603c8703010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:22.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.796+0000 7f1da06dfb80 1 --2- 172.21.15.119:0/273579022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8707270 0x5603c8703010 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1d78009f50 tx=0x7f1d7809e780 comp rx=0 tx=0).stop 2023-12-07T10:21:22.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.797+0000 7f1da06dfb80 1 -- 172.21.15.119:0/273579022 shutdown_connections 2023-12-07T10:21:22.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.797+0000 7f1da06dfb80 1 --2- 172.21.15.119:0/273579022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c870bbd0 0x5603c86fdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.797+0000 7f1da06dfb80 1 --2- 172.21.15.119:0/273579022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8707270 0x5603c8703010 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.797+0000 7f1da06dfb80 1 --2- 172.21.15.119:0/273579022 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5603c8780c90 0x5603c87830b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.797+0000 7f1d6dffb700 0 client.26035 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:22.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.797+0000 7f1da06dfb80 1 --2- 172.21.15.119:0/273579022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5603c8707b90 0x5603c86fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.797+0000 7f1da06dfb80 1 -- 172.21.15.119:0/273579022 >> 172.21.15.119:0/273579022 conn(0x5603c862d3e0 msgr2=0x5603c8709d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:22.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.797+0000 7f1da06dfb80 1 -- 172.21.15.119:0/273579022 shutdown_connections 2023-12-07T10:21:22.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.797+0000 7f1da06dfb80 1 -- 172.21.15.119:0/273579022 wait complete. 2023-12-07T10:21:22.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.798+0000 7f1da06dfb80 20 client.26035 trim_cache size 0 max 16384 2023-12-07T10:21:22.801 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:21:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: "return_code": 0, 2023-12-07T10:21:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_tag": "d40af9ad-1552-469f-8433-ce686cd66e4c", 2023-12-07T10:21:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": "asynchronous" 2023-12-07T10:21:22.801 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:21:22.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.798+0000 7f1d8759e700 1 -- 172.21.15.119:0/1769721597 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f1d700a1da0 msgr2=0x7f1d700a4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:22.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.798+0000 7f1d8759e700 1 --2- 172.21.15.119:0/1769721597 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f1d700a1da0 0x7f1d700a4260 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000f540 tx=0x7f1d80009920 comp rx=0 tx=0).stop 2023-12-07T10:21:22.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.798+0000 7f1d8759e700 1 -- 172.21.15.119:0/1769721597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d881025b0 msgr2=0x7f1d8819d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:22.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.798+0000 7f1d8759e700 1 --2- 172.21.15.119:0/1769721597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d881025b0 0x7f1d8819d5f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1d78009f20 tx=0x7f1d7800eca0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.799+0000 7f1d8759e700 1 -- 172.21.15.119:0/1769721597 shutdown_connections 2023-12-07T10:21:22.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.799+0000 7f1d8759e700 1 --2- 172.21.15.119:0/1769721597 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f1d700a1da0 0x7f1d700a4260 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.799+0000 7f1d8759e700 1 --2- 172.21.15.119:0/1769721597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d8810b9d0 0x7f1d88197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.799+0000 7f1d8759e700 1 --2- 172.21.15.119:0/1769721597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d881025b0 0x7f1d8819d5f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.799+0000 7f1d8759e700 1 --2- 172.21.15.119:0/1769721597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d88102ed0 0x7f1d8819db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:22.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.799+0000 7f1d8759e700 1 -- 172.21.15.119:0/1769721597 >> 172.21.15.119:0/1769721597 conn(0x7f1d880fbdd0 msgr2=0x7f1d880fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:22.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.799+0000 7f1d8759e700 1 -- 172.21.15.119:0/1769721597 shutdown_connections 2023-12-07T10:21:22.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:22.799+0000 7f1d8759e700 1 -- 172.21.15.119:0/1769721597 wait complete. 2023-12-07T10:21:22.822 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-07T10:21:23.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.081+0000 7f4a57fff700 1 -- 172.21.15.119:0/3522690568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a58107470 msgr2=0x7f4a58107850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:23.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.081+0000 7f4a57fff700 1 --2- 172.21.15.119:0/3522690568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a58107470 0x7f4a58107850 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4a48009990 tx=0x7f4a48009ca0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.081+0000 7f4a57fff700 1 -- 172.21.15.119:0/3522690568 shutdown_connections 2023-12-07T10:21:23.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.081+0000 7f4a57fff700 1 --2- 172.21.15.119:0/3522690568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a58101d90 0x7f4a5810e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.081+0000 7f4a57fff700 1 --2- 172.21.15.119:0/3522690568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a58107470 0x7f4a58107850 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.081+0000 7f4a57fff700 1 --2- 172.21.15.119:0/3522690568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a581013d0 0x7f4a58101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.081+0000 7f4a57fff700 1 -- 172.21.15.119:0/3522690568 >> 172.21.15.119:0/3522690568 conn(0x7f4a580fcee0 msgr2=0x7f4a580ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:23.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.081+0000 7f4a57fff700 1 -- 172.21.15.119:0/3522690568 shutdown_connections 2023-12-07T10:21:23.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.081+0000 7f4a57fff700 1 -- 172.21.15.119:0/3522690568 wait complete. 2023-12-07T10:21:23.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.082+0000 7f4a57fff700 1 Processor -- start 2023-12-07T10:21:23.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.082+0000 7f4a57fff700 1 -- start start 2023-12-07T10:21:23.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.083+0000 7f4a57fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a581013d0 0x7f4a5819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:23.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.083+0000 7f4a57fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a58101d90 0x7f4a5819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:23.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.083+0000 7f4a57fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a58107470 0x7f4a58199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:23.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.083+0000 7f4a57fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a58199e80 con 0x7f4a58101d90 2023-12-07T10:21:23.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.083+0000 7f4a57fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a58199ff0 con 0x7f4a581013d0 2023-12-07T10:21:23.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.083+0000 7f4a57fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5819a1b0 con 0x7f4a58107470 2023-12-07T10:21:23.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.083+0000 7f4a567fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a58101d90 0x7f4a5819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:23.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.083+0000 7f4a577fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a58107470 0x7f4a58199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:23.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.083+0000 7f4a577fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a58107470 0x7f4a58199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:35212/0 (socket says 172.21.15.119:35212) 2023-12-07T10:21:23.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.083+0000 7f4a577fe700 1 -- 172.21.15.119:0/948961386 learned_addr learned my addr 172.21.15.119:0/948961386 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:23.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.083+0000 7f4a56ffd700 1 --2- 172.21.15.119:0/948961386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a581013d0 0x7f4a5819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:23.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.084+0000 7f4a56ffd700 1 -- 172.21.15.119:0/948961386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a58107470 msgr2=0x7f4a58199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:23.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.084+0000 7f4a56ffd700 1 --2- 172.21.15.119:0/948961386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a58107470 0x7f4a58199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.084+0000 7f4a56ffd700 1 -- 172.21.15.119:0/948961386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a58101d90 msgr2=0x7f4a5819fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:23.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.084+0000 7f4a56ffd700 1 --2- 172.21.15.119:0/948961386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a58101d90 0x7f4a5819fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.084+0000 7f4a56ffd700 1 -- 172.21.15.119:0/948961386 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a480095f0 con 0x7f4a581013d0 2023-12-07T10:21:23.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.084+0000 7f4a567fc700 1 --2- 172.21.15.119:0/948961386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a58101d90 0x7f4a5819fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:23.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.084+0000 7f4a56ffd700 1 --2- 172.21.15.119:0/948961386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a581013d0 0x7f4a5819f6e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4a480059d0 tx=0x7f4a48004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:23.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.085+0000 7f4a3ffff700 1 -- 172.21.15.119:0/948961386 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4801d070 con 0x7f4a581013d0 2023-12-07T10:21:23.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.085+0000 7f4a57fff700 1 -- 172.21.15.119:0/948961386 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5819a430 con 0x7f4a581013d0 2023-12-07T10:21:23.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.085+0000 7f4a3ffff700 1 -- 172.21.15.119:0/948961386 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a48022470 con 0x7f4a581013d0 2023-12-07T10:21:23.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.085+0000 7f4a3ffff700 1 -- 172.21.15.119:0/948961386 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a48017650 con 0x7f4a581013d0 2023-12-07T10:21:23.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.085+0000 7f4a57fff700 1 -- 172.21.15.119:0/948961386 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a581a66d0 con 0x7f4a581013d0 2023-12-07T10:21:23.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.089+0000 7f4a71036b80 1 Processor -- start 2023-12-07T10:21:23.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.089+0000 7f4a71036b80 1 -- start start 2023-12-07T10:21:23.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.089+0000 7f4a3ffff700 1 -- 172.21.15.119:0/948961386 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 25) v1 ==== 136717+0+0 (secure 0 0 0) 0x7f4a480225e0 con 0x7f4a581013d0 2023-12-07T10:21:23.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.090+0000 7f4a71036b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559354d5b1f0 0x559354d5b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:23.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.090+0000 7f4a71036b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559354d5bb10 0x559354d5f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:23.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.090+0000 7f4a71036b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559354d5fb50 0x559354d5ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:23.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.090+0000 7f4a71036b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559354cc67c0 con 0x559354d5fb50 2023-12-07T10:21:23.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.090+0000 7f4a71036b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x559354cad7a0 con 0x559354d5bb10 2023-12-07T10:21:23.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.090+0000 7f4a71036b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559354cab0c0 con 0x559354d5b1f0 2023-12-07T10:21:23.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.090+0000 7f4a577fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559354d5b1f0 0x559354d5b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:23.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.090+0000 7f4a577fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559354d5b1f0 0x559354d5b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:35226/0 (socket says 172.21.15.119:35226) 2023-12-07T10:21:23.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.090+0000 7f4a577fe700 1 -- 172.21.15.119:0/2722601719 learned_addr learned my addr 172.21.15.119:0/2722601719 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:23.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.090+0000 7f4a56ffd700 1 --2- 172.21.15.119:0/2722601719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559354d5bb10 0x559354d5f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:23.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.090+0000 7f4a567fc700 1 --2- 172.21.15.119:0/2722601719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559354d5fb50 0x559354d5ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:23.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.091+0000 7f4a56ffd700 1 -- 172.21.15.119:0/2722601719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559354d5b1f0 msgr2=0x559354d5b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:23.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.091+0000 7f4a56ffd700 1 --2- 172.21.15.119:0/2722601719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559354d5b1f0 0x559354d5b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.091+0000 7f4a56ffd700 1 -- 172.21.15.119:0/2722601719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559354d5fb50 msgr2=0x559354d5ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:23.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.091+0000 7f4a56ffd700 1 --2- 172.21.15.119:0/2722601719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559354d5fb50 0x559354d5ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.091+0000 7f4a56ffd700 1 -- 172.21.15.119:0/2722601719 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559354b23630 con 0x559354d5bb10 2023-12-07T10:21:23.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.091+0000 7f4a56ffd700 1 --2- 172.21.15.119:0/2722601719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559354d5bb10 0x559354d5f610 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4a480cc2a0 tx=0x7f4a480d73c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82d2eb50bf648881 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:23.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.092+0000 7f4a3ffff700 1 --2- 172.21.15.119:0/948961386 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4a400a1da0 0x7f4a400a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:23.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.092+0000 7f4a3d7fa700 1 -- 172.21.15.119:0/2722601719 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a480d12e0 con 0x559354d5bb10 2023-12-07T10:21:23.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.092+0000 7f4a3d7fa700 1 -- 172.21.15.119:0/2722601719 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a48097bf0 con 0x559354d5bb10 2023-12-07T10:21:23.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.092+0000 7f4a3d7fa700 1 -- 172.21.15.119:0/2722601719 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4809fa00 con 0x559354d5bb10 2023-12-07T10:21:23.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.092+0000 7f4a3ffff700 1 -- 172.21.15.119:0/948961386 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4a480cf020 con 0x7f4a581013d0 2023-12-07T10:21:23.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.092+0000 7f4a577fe700 1 --2- 172.21.15.119:0/948961386 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4a400a1da0 0x7f4a400a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:23.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.093+0000 7f4a71036b80 1 -- 172.21.15.119:0/2722601719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559354d5bb10 msgr2=0x559354d5f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:23.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.093+0000 7f4a71036b80 1 --2- 172.21.15.119:0/2722601719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559354d5bb10 0x559354d5f610 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4a480cc2a0 tx=0x7f4a480d73c0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.093+0000 7f4a577fe700 1 --2- 172.21.15.119:0/948961386 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4a400a1da0 0x7f4a400a4260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009fd0 tx=0x7f4a5000b040 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:23.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.093+0000 7f4a71036b80 1 -- 172.21.15.119:0/2722601719 shutdown_connections 2023-12-07T10:21:23.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.093+0000 7f4a71036b80 1 --2- 172.21.15.119:0/2722601719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559354d5fb50 0x559354d5ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.093+0000 7f4a71036b80 1 --2- 172.21.15.119:0/2722601719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559354d5b1f0 0x559354d5b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.093+0000 7f4a71036b80 1 --2- 172.21.15.119:0/2722601719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559354d5bb10 0x559354d5f610 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.093+0000 7f4a71036b80 1 -- 172.21.15.119:0/2722601719 >> 172.21.15.119:0/2722601719 conn(0x559354c81360 msgr2=0x559354d4d170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:23.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.093+0000 7f4a71036b80 1 -- 172.21.15.119:0/2722601719 shutdown_connections 2023-12-07T10:21:23.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.093+0000 7f4a71036b80 1 -- 172.21.15.119:0/2722601719 wait complete. 2023-12-07T10:21:23.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.094+0000 7f4a71036b80 1 -- start start 2023-12-07T10:21:23.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.094+0000 7f4a71036b80 1 Processor -- start 2023-12-07T10:21:23.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.095+0000 7f4a71036b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559354d5b1f0 0x559354d67170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:23.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.095+0000 7f4a71036b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559354d5bb10 0x559354d64500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:23.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.095+0000 7f4a71036b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559354d5fb50 0x559354d64a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:23.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.095+0000 7f4a71036b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559354ca69b0 con 0x559354d5fb50 2023-12-07T10:21:23.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.095+0000 7f4a71036b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x559354c60560 con 0x559354d5bb10 2023-12-07T10:21:23.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.095+0000 7f4a71036b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559354c60200 con 0x559354d5b1f0 2023-12-07T10:21:23.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.095+0000 7f4a71036b80 10 client.0 _ll_register_callbacks cb 0x559354c47940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:23.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.095+0000 7f4a577fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559354d5fb50 0x559354d64a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:23.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.095+0000 7f4a567fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559354d5bb10 0x559354d64500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:23.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.095+0000 7f4a567fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559354d5bb10 0x559354d64500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:38072/0 (socket says 172.21.15.119:38072) 2023-12-07T10:21:23.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.095+0000 7f4a567fc700 1 -- 172.21.15.119:0/105019991 learned_addr learned my addr 172.21.15.119:0/105019991 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:23.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.095+0000 7f4a56ffd700 1 --2- 172.21.15.119:0/105019991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559354d5b1f0 0x559354d67170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:23.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.096+0000 7f4a567fc700 1 -- 172.21.15.119:0/105019991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559354d5b1f0 msgr2=0x559354d67170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:23.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.096+0000 7f4a567fc700 1 --2- 172.21.15.119:0/105019991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559354d5b1f0 0x559354d67170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.096+0000 7f4a567fc700 1 -- 172.21.15.119:0/105019991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559354d5fb50 msgr2=0x559354d64a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:23.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.096+0000 7f4a567fc700 1 --2- 172.21.15.119:0/105019991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559354d5fb50 0x559354d64a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.096+0000 7f4a567fc700 1 -- 172.21.15.119:0/105019991 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a581a66d0 con 0x559354d5bb10 2023-12-07T10:21:23.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.096+0000 7f4a567fc700 1 --2- 172.21.15.119:0/105019991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559354d5bb10 0x559354d64500 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c012d20 tx=0x7f4a4c00f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9f9676b42066178 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:23.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.096+0000 7f4a3aff5700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:21:23.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.096+0000 7f4a3aff5700 1 -- 172.21.15.119:0/105019991 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c017070 con 0x559354d5bb10 2023-12-07T10:21:23.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.096+0000 7f4a71036b80 1 -- 172.21.15.119:0/105019991 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559354d459a0 con 0x559354d5bb10 2023-12-07T10:21:23.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.096+0000 7f4a3aff5700 1 -- 172.21.15.119:0/105019991 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4a4c00fb80 con 0x559354d5bb10 2023-12-07T10:21:23.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.096+0000 7f4a3aff5700 1 -- 172.21.15.119:0/105019991 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a4c00fd80 con 0x559354d5bb10 2023-12-07T10:21:23.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.097+0000 7f4a71036b80 10 client.26060 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:23.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.097+0000 7f4a71036b80 1 -- 172.21.15.119:0/105019991 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559354acfde0 con 0x559354d5bb10 2023-12-07T10:21:23.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.097+0000 7f4a3aff5700 1 -- 172.21.15.119:0/105019991 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559354d459a0 con 0x559354d5bb10 2023-12-07T10:21:23.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.097+0000 7f4a3aff5700 1 -- 172.21.15.119:0/105019991 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4a4c003b30 con 0x559354d5bb10 2023-12-07T10:21:23.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.097+0000 7f4a71036b80 10 client.26060 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:23.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.097+0000 7f4a71036b80 10 client.26060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:21:23.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.097+0000 7f4a71036b80 20 client.26060 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:23.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.097+0000 7f4a71036b80 1 --2- 172.21.15.119:0/105019991 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x559354d74a80 0x559354d76ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:23.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.098+0000 7f4a71036b80 4 client.26060 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:21:23.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.098+0000 7f4a71036b80 1 -- 172.21.15.119:0/105019991 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559354d217a0 con 0x559354d74a80 2023-12-07T10:21:23.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.098+0000 7f4a56ffd700 1 --2- 172.21.15.119:0/105019991 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x559354d74a80 0x559354d76ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:23.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.099+0000 7f4a3aff5700 10 client.26060 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:23.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.099+0000 7f4a56ffd700 1 --2- 172.21.15.119:0/105019991 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x559354d74a80 0x559354d76ea0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed35d1a89bdb99a9 server_cookie=af0ec91c2b3236d9 in_seq=0 out_seq=0 2023-12-07T10:21:23.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.099+0000 7f4a3aff5700 1 -- 172.21.15.119:0/105019991 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559354d217a0 con 0x559354d74a80 2023-12-07T10:21:23.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.099+0000 7f4a3aff5700 10 client.26060 handle_command_reply: tid=0 2023-12-07T10:21:23.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.099+0000 7f4a71036b80 1 client.26060 shutdown 2023-12-07T10:21:23.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.100+0000 7f4a71036b80 1 -- 172.21.15.119:0/105019991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559354d5bb10 msgr2=0x559354d64500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:23.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.100+0000 7f4a71036b80 1 --2- 172.21.15.119:0/105019991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559354d5bb10 0x559354d64500 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c012d20 tx=0x7f4a4c00f6a0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.100+0000 7f4a71036b80 1 -- 172.21.15.119:0/105019991 shutdown_connections 2023-12-07T10:21:23.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.100+0000 7f4a71036b80 1 --2- 172.21.15.119:0/105019991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559354d5fb50 0x559354d64a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.100+0000 7f4a71036b80 1 --2- 172.21.15.119:0/105019991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559354d5b1f0 0x559354d67170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.100+0000 7f4a71036b80 1 --2- 172.21.15.119:0/105019991 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x559354d74a80 0x559354d76ea0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.100+0000 7f4a71036b80 1 --2- 172.21.15.119:0/105019991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559354d5bb10 0x559354d64500 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.100+0000 7f4a71036b80 1 -- 172.21.15.119:0/105019991 >> 172.21.15.119:0/105019991 conn(0x559354c81360 msgr2=0x559354d517e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:23.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.100+0000 7f4a3aff5700 0 client.26060 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:23.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.100+0000 7f4a71036b80 1 -- 172.21.15.119:0/105019991 shutdown_connections 2023-12-07T10:21:23.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.100+0000 7f4a71036b80 1 -- 172.21.15.119:0/105019991 wait complete. 2023-12-07T10:21:23.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.100+0000 7f4a71036b80 20 client.26060 trim_cache size 0 max 16384 2023-12-07T10:21:23.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.118+0000 7f4a71036b80 1 Processor -- start 2023-12-07T10:21:23.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.118+0000 7f4a71036b80 1 -- start start 2023-12-07T10:21:23.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.119+0000 7f4a71036b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559354d5b1f0 0x559354d70560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:23.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.119+0000 7f4a71036b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559354d5bb10 0x559354d7b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:23.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.119+0000 7f4a71036b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559354d5fb50 0x559354d5dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:23.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.119+0000 7f4a71036b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559354b25920 con 0x559354d5fb50 2023-12-07T10:21:23.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.119+0000 7f4a71036b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x559354d67520 con 0x559354d5b1f0 2023-12-07T10:21:23.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.119+0000 7f4a71036b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559354d49890 con 0x559354d5bb10 2023-12-07T10:21:23.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.119+0000 7f4a577fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559354d5fb50 0x559354d5dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:23.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.119+0000 7f4a56ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559354d5b1f0 0x559354d70560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:23.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.119+0000 7f4a567fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559354d5bb10 0x559354d7b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:23.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.119+0000 7f4a56ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559354d5b1f0 0x559354d70560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:38084/0 (socket says 172.21.15.119:38084) 2023-12-07T10:21:23.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.119+0000 7f4a56ffd700 1 -- 172.21.15.119:0/721611613 learned_addr learned my addr 172.21.15.119:0/721611613 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:23.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.120+0000 7f4a567fc700 1 -- 172.21.15.119:0/721611613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559354d5b1f0 msgr2=0x559354d70560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:23.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.120+0000 7f4a567fc700 1 --2- 172.21.15.119:0/721611613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559354d5b1f0 0x559354d70560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.120+0000 7f4a567fc700 1 -- 172.21.15.119:0/721611613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559354d5fb50 msgr2=0x559354d5dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:23.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.120+0000 7f4a567fc700 1 --2- 172.21.15.119:0/721611613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559354d5fb50 0x559354d5dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.120+0000 7f4a567fc700 1 -- 172.21.15.119:0/721611613 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559354ab8c60 con 0x559354d5bb10 2023-12-07T10:21:23.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.120+0000 7f4a567fc700 1 --2- 172.21.15.119:0/721611613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559354d5bb10 0x559354d7b800 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c000c00 tx=0x7f4a4c009400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edc63e122d14ed3c server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:23.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.120+0000 7f4a3dffb700 1 -- 172.21.15.119:0/721611613 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c017070 con 0x559354d5bb10 2023-12-07T10:21:23.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.120+0000 7f4a3dffb700 1 -- 172.21.15.119:0/721611613 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a4c0106a0 con 0x559354d5bb10 2023-12-07T10:21:23.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.120+0000 7f4a3dffb700 1 -- 172.21.15.119:0/721611613 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c005110 con 0x559354d5bb10 2023-12-07T10:21:23.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.121+0000 7f4a71036b80 1 -- 172.21.15.119:0/721611613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559354d5bb10 msgr2=0x559354d7b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:23.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.121+0000 7f4a71036b80 1 --2- 172.21.15.119:0/721611613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559354d5bb10 0x559354d7b800 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c000c00 tx=0x7f4a4c009400 comp rx=0 tx=0).stop 2023-12-07T10:21:23.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.121+0000 7f4a71036b80 1 -- 172.21.15.119:0/721611613 shutdown_connections 2023-12-07T10:21:23.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.121+0000 7f4a71036b80 1 --2- 172.21.15.119:0/721611613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559354d5fb50 0x559354d5dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.121+0000 7f4a71036b80 1 --2- 172.21.15.119:0/721611613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559354d5b1f0 0x559354d70560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.121+0000 7f4a71036b80 1 --2- 172.21.15.119:0/721611613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559354d5bb10 0x559354d7b800 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.121+0000 7f4a71036b80 1 -- 172.21.15.119:0/721611613 >> 172.21.15.119:0/721611613 conn(0x559354c81360 msgr2=0x559354d4ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:23.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.121+0000 7f4a71036b80 1 -- 172.21.15.119:0/721611613 shutdown_connections 2023-12-07T10:21:23.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.121+0000 7f4a71036b80 1 -- 172.21.15.119:0/721611613 wait complete. 2023-12-07T10:21:23.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.122+0000 7f4a71036b80 1 -- start start 2023-12-07T10:21:23.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.122+0000 7f4a71036b80 1 Processor -- start 2023-12-07T10:21:23.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.123+0000 7f4a71036b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559354d5b1f0 0x559354d6bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:23.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.123+0000 7f4a71036b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559354d5bb10 0x559354d66250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:23.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.123+0000 7f4a71036b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559354d5fb50 0x559354d63a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:23.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.123+0000 7f4a71036b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559354d66b60 con 0x559354d5bb10 2023-12-07T10:21:23.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.123+0000 7f4a71036b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x559354d66cd0 con 0x559354d5b1f0 2023-12-07T10:21:23.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.123+0000 7f4a71036b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559354d63f80 con 0x559354d5fb50 2023-12-07T10:21:23.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.123+0000 7f4a71036b80 10 client.0 _ll_register_callbacks cb 0x559354c47940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:23.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.123+0000 7f4a577fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559354d5fb50 0x559354d63a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:23.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.123+0000 7f4a567fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559354d5bb10 0x559354d66250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:23.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.123+0000 7f4a56ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559354d5b1f0 0x559354d6bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:23.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.123+0000 7f4a567fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559354d5bb10 0x559354d66250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54508/0 (socket says 172.21.15.119:54508) 2023-12-07T10:21:23.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.123+0000 7f4a567fc700 1 -- 172.21.15.119:0/2890396937 learned_addr learned my addr 172.21.15.119:0/2890396937 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:23.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.124+0000 7f4a567fc700 1 -- 172.21.15.119:0/2890396937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559354d5fb50 msgr2=0x559354d63a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:23.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.124+0000 7f4a567fc700 1 --2- 172.21.15.119:0/2890396937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559354d5fb50 0x559354d63a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.124+0000 7f4a567fc700 1 -- 172.21.15.119:0/2890396937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559354d5b1f0 msgr2=0x559354d6bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:23.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.124+0000 7f4a567fc700 1 --2- 172.21.15.119:0/2890396937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559354d5b1f0 0x559354d6bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.124+0000 7f4a567fc700 1 -- 172.21.15.119:0/2890396937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4c021040 con 0x559354d5bb10 2023-12-07T10:21:23.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.124+0000 7f4a577fe700 1 --2- 172.21.15.119:0/2890396937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559354d5fb50 0x559354d63a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:23.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.124+0000 7f4a56ffd700 1 --2- 172.21.15.119:0/2890396937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559354d5b1f0 0x559354d6bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:23.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.124+0000 7f4a567fc700 1 --2- 172.21.15.119:0/2890396937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559354d5bb10 0x559354d66250 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c003f40 tx=0x7f4a4c00f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5d9ea30f3e9b19 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:23.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.124+0000 7f4a3aff5700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:21:23.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.125+0000 7f4a3aff5700 1 -- 172.21.15.119:0/2890396937 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c017030 con 0x559354d5bb10 2023-12-07T10:21:23.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.125+0000 7f4a3aff5700 1 -- 172.21.15.119:0/2890396937 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4a4c000e40 con 0x559354d5bb10 2023-12-07T10:21:23.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.125+0000 7f4a71036b80 1 -- 172.21.15.119:0/2890396937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a48097bf0 con 0x559354d5bb10 2023-12-07T10:21:23.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.125+0000 7f4a3aff5700 1 -- 172.21.15.119:0/2890396937 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a4c010b60 con 0x559354d5bb10 2023-12-07T10:21:23.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.125+0000 7f4a71036b80 10 client.16053 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:23.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.125+0000 7f4a71036b80 1 -- 172.21.15.119:0/2890396937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559354af6250 con 0x559354d5bb10 2023-12-07T10:21:23.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.126+0000 7f4a3aff5700 1 -- 172.21.15.119:0/2890396937 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a48097bf0 con 0x559354d5bb10 2023-12-07T10:21:23.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.126+0000 7f4a3aff5700 1 -- 172.21.15.119:0/2890396937 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4a4c003790 con 0x559354d5bb10 2023-12-07T10:21:23.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.126+0000 7f4a71036b80 10 client.16053 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:23.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.126+0000 7f4a71036b80 10 client.16053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:21:23.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.126+0000 7f4a71036b80 20 client.16053 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:23.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.126+0000 7f4a71036b80 1 --2- 172.21.15.119:0/2890396937 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x559354dcb890 0x559354dcdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:23.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.126+0000 7f4a71036b80 4 client.16053 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-12-07T10:21:23.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.126+0000 7f4a71036b80 1 -- 172.21.15.119:0/2890396937 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559354c6cfd0 con 0x559354dcb890 2023-12-07T10:21:23.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.126+0000 7f4a56ffd700 1 --2- 172.21.15.119:0/2890396937 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x559354dcb890 0x559354dcdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:23.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.127+0000 7f4a56ffd700 1 --2- 172.21.15.119:0/2890396937 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x559354dcb890 0x559354dcdcb0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93e1a72937c6f1bb server_cookie=9cd0467c684881e2 in_seq=0 out_seq=0 2023-12-07T10:21:23.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.127+0000 7f4a3aff5700 10 client.16053 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:23.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.127+0000 7f4a3aff5700 1 -- 172.21.15.119:0/2890396937 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559354c6cfd0 con 0x559354dcb890 2023-12-07T10:21:23.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.127+0000 7f4a3aff5700 10 client.16053 handle_command_reply: tid=0 2023-12-07T10:21:23.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.127+0000 7f4a71036b80 1 client.16053 shutdown 2023-12-07T10:21:23.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.127+0000 7f4a71036b80 1 -- 172.21.15.119:0/2890396937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559354d5bb10 msgr2=0x559354d66250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:23.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.127+0000 7f4a71036b80 1 --2- 172.21.15.119:0/2890396937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559354d5bb10 0x559354d66250 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c003f40 tx=0x7f4a4c00f6a0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.128+0000 7f4a71036b80 1 -- 172.21.15.119:0/2890396937 shutdown_connections 2023-12-07T10:21:23.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.128+0000 7f4a71036b80 1 --2- 172.21.15.119:0/2890396937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559354d5fb50 0x559354d63a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.128+0000 7f4a71036b80 1 --2- 172.21.15.119:0/2890396937 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x559354dcb890 0x559354dcdcb0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.128+0000 7f4a71036b80 1 --2- 172.21.15.119:0/2890396937 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559354d5b1f0 0x559354d6bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.128+0000 7f4a71036b80 1 --2- 172.21.15.119:0/2890396937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559354d5bb10 0x559354d66250 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.128+0000 7f4a71036b80 1 -- 172.21.15.119:0/2890396937 >> 172.21.15.119:0/2890396937 conn(0x559354c81360 msgr2=0x559354d5dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:23.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.128+0000 7f4a3aff5700 0 client.16053 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:23.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.128+0000 7f4a71036b80 1 -- 172.21.15.119:0/2890396937 shutdown_connections 2023-12-07T10:21:23.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.128+0000 7f4a71036b80 1 -- 172.21.15.119:0/2890396937 wait complete. 2023-12-07T10:21:23.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.128+0000 7f4a71036b80 20 client.16053 trim_cache size 0 max 16384 2023-12-07T10:21:23.134 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:21:23.134 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-07T10:21:23.134 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": { 2023-12-07T10:21:23.134 INFO:teuthology.orchestra.run.smithi119.stdout: "d40af9ad-1552-469f-8433-ce686cd66e4c": { 2023-12-07T10:21:23.134 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/", 2023-12-07T10:21:23.135 INFO:teuthology.orchestra.run.smithi119.stdout: "tag": "d40af9ad-1552-469f-8433-ce686cd66e4c", 2023-12-07T10:21:23.135 INFO:teuthology.orchestra.run.smithi119.stdout: "options": "recursive,force" 2023-12-07T10:21:23.135 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:21:23.135 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:21:23.135 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:21:23.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.129+0000 7f4a57fff700 1 -- 172.21.15.119:0/948961386 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4a400a1da0 msgr2=0x7f4a400a4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:23.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.129+0000 7f4a57fff700 1 --2- 172.21.15.119:0/948961386 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4a400a1da0 0x7f4a400a4260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009fd0 tx=0x7f4a5000b040 comp rx=0 tx=0).stop 2023-12-07T10:21:23.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.129+0000 7f4a57fff700 1 -- 172.21.15.119:0/948961386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a581013d0 msgr2=0x7f4a5819f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:23.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.129+0000 7f4a57fff700 1 --2- 172.21.15.119:0/948961386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a581013d0 0x7f4a5819f6e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4a480059d0 tx=0x7f4a48004730 comp rx=0 tx=0).stop 2023-12-07T10:21:23.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.129+0000 7f4a57fff700 1 -- 172.21.15.119:0/948961386 shutdown_connections 2023-12-07T10:21:23.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.129+0000 7f4a57fff700 1 --2- 172.21.15.119:0/948961386 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4a400a1da0 0x7f4a400a4260 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.129+0000 7f4a57fff700 1 --2- 172.21.15.119:0/948961386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a58107470 0x7f4a58199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.129+0000 7f4a57fff700 1 --2- 172.21.15.119:0/948961386 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a581013d0 0x7f4a5819f6e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.129+0000 7f4a57fff700 1 --2- 172.21.15.119:0/948961386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a58101d90 0x7f4a5819fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:23.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.129+0000 7f4a57fff700 1 -- 172.21.15.119:0/948961386 >> 172.21.15.119:0/948961386 conn(0x7f4a580fcee0 msgr2=0x7f4a5810cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:23.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.129+0000 7f4a57fff700 1 -- 172.21.15.119:0/948961386 shutdown_connections 2023-12-07T10:21:23.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:23.129+0000 7f4a57fff700 1 -- 172.21.15.119:0/948961386 wait complete. 2023-12-07T10:21:23.155 INFO:tasks.cephfs.filesystem:scrub status for tag:d40af9ad-1552-469f-8433-ce686cd66e4c - {'path': '/', 'tag': 'd40af9ad-1552-469f-8433-ce686cd66e4c', 'options': 'recursive,force'} 2023-12-07T10:21:24.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:23 smithi119 ceph-mon[97908]: scrub queued for path: / 2023-12-07T10:21:24.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:23 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:21:24.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:23 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:21:24.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:23 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:21:24.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:23 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:21:24.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:23 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:21:24.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:23 smithi119 ceph-mon[97908]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-12-07T10:21:24.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:23 smithi159 ceph-mon[134726]: scrub queued for path: / 2023-12-07T10:21:24.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:23 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:21:24.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:23 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:21:24.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:23 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:21:24.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:23 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:21:24.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:23 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:21:24.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:23 smithi159 ceph-mon[134726]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-12-07T10:21:24.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:23 smithi153 ceph-mon[133625]: scrub queued for path: / 2023-12-07T10:21:24.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:23 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:21:24.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:23 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:21:24.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:23 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:21:24.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:23 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:21:24.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:23 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:21:24.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:23 smithi153 ceph-mon[133625]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-12-07T10:21:26.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:25 smithi119 ceph-mon[97908]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2023-12-07T10:21:26.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:25 smithi159 ceph-mon[134726]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2023-12-07T10:21:26.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:25 smithi153 ceph-mon[133625]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2023-12-07T10:21:28.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:27 smithi119 ceph-mon[97908]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2023-12-07T10:21:28.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:27 smithi159 ceph-mon[134726]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2023-12-07T10:21:28.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:27 smithi153 ceph-mon[133625]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2023-12-07T10:21:29.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:28 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:21:29.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:28 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:21:29.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:28 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:21:30.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:29 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:21:30.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:29 smithi119 ceph-mon[97908]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-12-07T10:21:30.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:29 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:21:30.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:29 smithi159 ceph-mon[134726]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-12-07T10:21:30.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:29 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:21:30.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:29 smithi153 ceph-mon[133625]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-12-07T10:21:32.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:31 smithi119 ceph-mon[97908]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2023-12-07T10:21:32.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:31 smithi159 ceph-mon[134726]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2023-12-07T10:21:32.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:31 smithi153 ceph-mon[133625]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2023-12-07T10:21:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.881+0000 7f48ac1ff700 1 -- 172.21.15.119:0/889907147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48a410ddc0 msgr2=0x7f48a41101b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.881+0000 7f48ac1ff700 1 --2- 172.21.15.119:0/889907147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48a410ddc0 0x7f48a41101b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f489c00b0a0 tx=0x7f489c00b3b0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.881+0000 7f48ac1ff700 1 -- 172.21.15.119:0/889907147 shutdown_connections 2023-12-07T10:21:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.881+0000 7f48ac1ff700 1 --2- 172.21.15.119:0/889907147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48a410ddc0 0x7f48a41101b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.881+0000 7f48ac1ff700 1 --2- 172.21.15.119:0/889907147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48a40739c0 0x7f48a4073da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.881+0000 7f48ac1ff700 1 --2- 172.21.15.119:0/889907147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a40742e0 0x7f48a410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.881+0000 7f48ac1ff700 1 -- 172.21.15.119:0/889907147 >> 172.21.15.119:0/889907147 conn(0x7f48a40fcfd0 msgr2=0x7f48a40ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.881+0000 7f48ac1ff700 1 -- 172.21.15.119:0/889907147 shutdown_connections 2023-12-07T10:21:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.881+0000 7f48ac1ff700 1 -- 172.21.15.119:0/889907147 wait complete. 2023-12-07T10:21:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.883+0000 7f48ac1ff700 1 Processor -- start 2023-12-07T10:21:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.883+0000 7f48ac1ff700 1 -- start start 2023-12-07T10:21:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.884+0000 7f48ac1ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a40739c0 0x7f48a419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.884+0000 7f48ac1ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48a40742e0 0x7f48a419fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.884+0000 7f48ac1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48a410ddc0 0x7f48a41999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.884+0000 7f48ac1ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a419a040 con 0x7f48a40739c0 2023-12-07T10:21:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.884+0000 7f48ac1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a419a1b0 con 0x7f48a410ddc0 2023-12-07T10:21:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.884+0000 7f48ac1ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a419a370 con 0x7f48a40742e0 2023-12-07T10:21:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.884+0000 7f48a979a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48a40742e0 0x7f48a419fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.884+0000 7f48a979a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48a40742e0 0x7f48a419fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:53932/0 (socket says 172.21.15.119:53932) 2023-12-07T10:21:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.884+0000 7f48a979a700 1 -- 172.21.15.119:0/1668578639 learned_addr learned my addr 172.21.15.119:0/1668578639 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.885+0000 7f48a9f9b700 1 --2- 172.21.15.119:0/1668578639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a40739c0 0x7f48a419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.885+0000 7f48a979a700 1 -- 172.21.15.119:0/1668578639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48a410ddc0 msgr2=0x7f48a41999e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-07T10:21:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.885+0000 7f48a979a700 1 --2- 172.21.15.119:0/1668578639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48a410ddc0 0x7f48a41999e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.885+0000 7f48a979a700 1 -- 172.21.15.119:0/1668578639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a40739c0 msgr2=0x7f48a419f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.885+0000 7f48a979a700 1 --2- 172.21.15.119:0/1668578639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a40739c0 0x7f48a419f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.885+0000 7f48a979a700 1 -- 172.21.15.119:0/1668578639 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489c009d00 con 0x7f48a40742e0 2023-12-07T10:21:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.885+0000 7f48a9f9b700 1 --2- 172.21.15.119:0/1668578639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a40739c0 0x7f48a419f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:21:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.885+0000 7f48a979a700 1 --2- 172.21.15.119:0/1668578639 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48a40742e0 0x7f48a419fdf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f489400e9a0 tx=0x7f489400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.886+0000 7f489affd700 1 -- 172.21.15.119:0/1668578639 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489400cbd0 con 0x7f48a40742e0 2023-12-07T10:21:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.886+0000 7f489affd700 1 -- 172.21.15.119:0/1668578639 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f489400cd30 con 0x7f48a40742e0 2023-12-07T10:21:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.886+0000 7f489affd700 1 -- 172.21.15.119:0/1668578639 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4894018680 con 0x7f48a40742e0 2023-12-07T10:21:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.886+0000 7f48ac1ff700 1 -- 172.21.15.119:0/1668578639 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a419a650 con 0x7f48a40742e0 2023-12-07T10:21:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.886+0000 7f48ac1ff700 1 -- 172.21.15.119:0/1668578639 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a41015f0 con 0x7f48a40742e0 2023-12-07T10:21:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.892+0000 7f489affd700 1 -- 172.21.15.119:0/1668578639 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) v1 ==== 136717+0+0 (secure 0 0 0) 0x7f4894005070 con 0x7f48a40742e0 2023-12-07T10:21:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.893+0000 7f48bfb00b80 1 Processor -- start 2023-12-07T10:21:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.893+0000 7f48bfb00b80 1 -- start start 2023-12-07T10:21:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.894+0000 7f48bfb00b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629b98fca60 0x5629b98fce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.894+0000 7f48bfb00b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629b98fd380 0x5629b98f7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.894+0000 7f48bfb00b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b98f7ad0 0x5629b98f7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.894+0000 7f48bfb00b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b986b760 con 0x5629b98f7ad0 2023-12-07T10:21:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.894+0000 7f48bfb00b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b9852740 con 0x5629b98fd380 2023-12-07T10:21:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.894+0000 7f48bfb00b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b9850060 con 0x5629b98fca60 2023-12-07T10:21:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.894+0000 7f48aa79c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629b98fca60 0x5629b98fce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.894+0000 7f48aa79c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629b98fca60 0x5629b98fce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:53944/0 (socket says 172.21.15.119:53944) 2023-12-07T10:21:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.894+0000 7f48aa79c700 1 -- 172.21.15.119:0/393603498 learned_addr learned my addr 172.21.15.119:0/393603498 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.895+0000 7f48aa79c700 1 -- 172.21.15.119:0/393603498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629b98fd380 msgr2=0x5629b98f7540 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-07T10:21:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.895+0000 7f48aa79c700 1 --2- 172.21.15.119:0/393603498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629b98fd380 0x5629b98f7540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.895+0000 7f48aa79c700 1 -- 172.21.15.119:0/393603498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b98f7ad0 msgr2=0x5629b98f7f30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-07T10:21:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.895+0000 7f48a979a700 1 --2- 172.21.15.119:0/393603498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b98f7ad0 0x5629b98f7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.895+0000 7f48aa79c700 1 --2- 172.21.15.119:0/393603498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b98f7ad0 0x5629b98f7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.895+0000 7f48aa79c700 1 -- 172.21.15.119:0/393603498 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629b96c85d0 con 0x5629b98fca60 2023-12-07T10:21:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.895+0000 7f489affd700 1 --2- 172.21.15.119:0/1668578639 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f48900a1ea0 0x7f48900a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.895+0000 7f48a979a700 1 --2- 172.21.15.119:0/393603498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b98f7ad0 0x5629b98f7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:21:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.895+0000 7f489affd700 1 -- 172.21.15.119:0/1668578639 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4894014070 con 0x7f48a40742e0 2023-12-07T10:21:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.896+0000 7f48a9f9b700 1 --2- 172.21.15.119:0/1668578639 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f48900a1ea0 0x7f48900a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.896+0000 7f48aa79c700 1 --2- 172.21.15.119:0/393603498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629b98fca60 0x5629b98fce40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f489c009c60 tx=0x7f489c009c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de59a2fd24544a58 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.897+0000 7f488ffff700 1 -- 172.21.15.119:0/393603498 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c00e070 con 0x5629b98fca60 2023-12-07T10:21:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.897+0000 7f488ffff700 1 -- 172.21.15.119:0/393603498 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f489c007e20 con 0x5629b98fca60 2023-12-07T10:21:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.897+0000 7f48a9f9b700 1 --2- 172.21.15.119:0/1668578639 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f48900a1ea0 0x7f48900a4360 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x5629b98f3890 tx=0x7f48a0005f50 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.897+0000 7f488ffff700 1 -- 172.21.15.119:0/393603498 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c007940 con 0x5629b98fca60 2023-12-07T10:21:33.899 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.898+0000 7f48bfb00b80 1 -- 172.21.15.119:0/393603498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629b98fca60 msgr2=0x5629b98fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:33.899 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.898+0000 7f48bfb00b80 1 --2- 172.21.15.119:0/393603498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629b98fca60 0x5629b98fce40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f489c009c60 tx=0x7f489c009c90 comp rx=0 tx=0).stop 2023-12-07T10:21:33.899 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.898+0000 7f48bfb00b80 1 -- 172.21.15.119:0/393603498 shutdown_connections 2023-12-07T10:21:33.899 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.898+0000 7f48bfb00b80 1 --2- 172.21.15.119:0/393603498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b98f7ad0 0x5629b98f7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.899 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.898+0000 7f48bfb00b80 1 --2- 172.21.15.119:0/393603498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629b98fca60 0x5629b98fce40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.899 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.898+0000 7f48bfb00b80 1 --2- 172.21.15.119:0/393603498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629b98fd380 0x5629b98f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.899 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.898+0000 7f48bfb00b80 1 -- 172.21.15.119:0/393603498 >> 172.21.15.119:0/393603498 conn(0x5629b9826300 msgr2=0x5629b98f1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.898+0000 7f48bfb00b80 1 -- 172.21.15.119:0/393603498 shutdown_connections 2023-12-07T10:21:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.898+0000 7f48bfb00b80 1 -- 172.21.15.119:0/393603498 wait complete. 2023-12-07T10:21:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.899+0000 7f48bfb00b80 1 -- start start 2023-12-07T10:21:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.899+0000 7f48bfb00b80 1 Processor -- start 2023-12-07T10:21:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.900+0000 7f48bfb00b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629b98f7ad0 0x5629b9900750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.900+0000 7f48bfb00b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b98fca60 0x5629b9900c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.900+0000 7f48bfb00b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629b98fd380 0x5629b99011d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.900+0000 7f48bfb00b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b984b950 con 0x5629b98fca60 2023-12-07T10:21:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.900+0000 7f48bfb00b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b9805500 con 0x5629b98f7ad0 2023-12-07T10:21:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.900+0000 7f48bfb00b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b98051a0 con 0x5629b98fd380 2023-12-07T10:21:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.900+0000 7f48bfb00b80 10 client.0 _ll_register_callbacks cb 0x5629b97ec8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.901+0000 7f48a9f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629b98f7ad0 0x5629b9900750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.901+0000 7f48a9f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629b98f7ad0 0x5629b9900750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:43110/0 (socket says 172.21.15.119:43110) 2023-12-07T10:21:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.901+0000 7f48a9f9b700 1 -- 172.21.15.119:0/2984465796 learned_addr learned my addr 172.21.15.119:0/2984465796 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.901+0000 7f48aa79c700 1 --2- 172.21.15.119:0/2984465796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629b98fd380 0x5629b99011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.901+0000 7f48a979a700 1 --2- 172.21.15.119:0/2984465796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b98fca60 0x5629b9900c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.901+0000 7f48aa79c700 1 -- 172.21.15.119:0/2984465796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629b98f7ad0 msgr2=0x5629b9900750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.901+0000 7f48aa79c700 1 --2- 172.21.15.119:0/2984465796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629b98f7ad0 0x5629b9900750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.901+0000 7f48aa79c700 1 -- 172.21.15.119:0/2984465796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b98fca60 msgr2=0x5629b9900c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.901+0000 7f48aa79c700 1 --2- 172.21.15.119:0/2984465796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b98fca60 0x5629b9900c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.901+0000 7f48aa79c700 1 -- 172.21.15.119:0/2984465796 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a4110e70 con 0x5629b98fd380 2023-12-07T10:21:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.901+0000 7f48a979a700 1 --2- 172.21.15.119:0/2984465796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b98fca60 0x5629b9900c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:21:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.902+0000 7f48aa79c700 1 --2- 172.21.15.119:0/2984465796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629b98fd380 0x5629b99011d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f489c00b700 tx=0x7f489c00bb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5395fed9eac6955a server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.902+0000 7f488d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:21:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.902+0000 7f488d7fa700 1 -- 172.21.15.119:0/2984465796 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c02a070 con 0x5629b98fd380 2023-12-07T10:21:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.902+0000 7f488d7fa700 1 -- 172.21.15.119:0/2984465796 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f489c007750 con 0x5629b98fd380 2023-12-07T10:21:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.902+0000 7f48bfb00b80 1 -- 172.21.15.119:0/2984465796 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629b98ea6f0 con 0x5629b98fd380 2023-12-07T10:21:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.903+0000 7f488d7fa700 1 -- 172.21.15.119:0/2984465796 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f489c00e040 con 0x5629b98fd380 2023-12-07T10:21:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.903+0000 7f48bfb00b80 10 client.26065 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.903+0000 7f48bfb00b80 1 -- 172.21.15.119:0/2984465796 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5629b9674d80 con 0x5629b98fd380 2023-12-07T10:21:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.904+0000 7f488d7fa700 1 -- 172.21.15.119:0/2984465796 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5629b98ea6f0 con 0x5629b98fd380 2023-12-07T10:21:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.904+0000 7f488d7fa700 1 -- 172.21.15.119:0/2984465796 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f489c02d030 con 0x5629b98fd380 2023-12-07T10:21:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.904+0000 7f48bfb00b80 10 client.26065 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.904+0000 7f48bfb00b80 10 client.26065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:21:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.904+0000 7f48bfb00b80 20 client.26065 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.904+0000 7f48bfb00b80 1 --2- 172.21.15.119:0/2984465796 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5629b9910900 0x5629b9912cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.904+0000 7f48bfb00b80 4 client.26065 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:21:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.904+0000 7f48bfb00b80 1 -- 172.21.15.119:0/2984465796 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629b98c6740 con 0x5629b9910900 2023-12-07T10:21:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.905+0000 7f48a9f9b700 1 --2- 172.21.15.119:0/2984465796 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5629b9910900 0x5629b9912cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.905+0000 7f48a9f9b700 1 --2- 172.21.15.119:0/2984465796 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5629b9910900 0x5629b9912cf0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=895679fef36f1921 server_cookie=77e6fce3dc2f8924 in_seq=0 out_seq=0 2023-12-07T10:21:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.905+0000 7f488d7fa700 10 client.26065 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.906+0000 7f488d7fa700 1 -- 172.21.15.119:0/2984465796 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5629b98c6740 con 0x5629b9910900 2023-12-07T10:21:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.907+0000 7f488d7fa700 10 client.26065 handle_command_reply: tid=0 2023-12-07T10:21:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.907+0000 7f48bfb00b80 1 client.26065 shutdown 2023-12-07T10:21:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.907+0000 7f48bfb00b80 1 -- 172.21.15.119:0/2984465796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629b98fd380 msgr2=0x5629b99011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.907+0000 7f48bfb00b80 1 --2- 172.21.15.119:0/2984465796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629b98fd380 0x5629b99011d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f489c00b700 tx=0x7f489c00bb30 comp rx=0 tx=0).stop 2023-12-07T10:21:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.907+0000 7f48bfb00b80 1 -- 172.21.15.119:0/2984465796 shutdown_connections 2023-12-07T10:21:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.907+0000 7f48bfb00b80 1 --2- 172.21.15.119:0/2984465796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629b98fd380 0x5629b99011d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.907+0000 7f48bfb00b80 1 --2- 172.21.15.119:0/2984465796 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5629b9910900 0x5629b9912cf0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.907+0000 7f488d7fa700 0 client.26065 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.907+0000 7f48bfb00b80 1 --2- 172.21.15.119:0/2984465796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629b98f7ad0 0x5629b9900750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.907+0000 7f48bfb00b80 1 --2- 172.21.15.119:0/2984465796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b98fca60 0x5629b9900c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.907+0000 7f48bfb00b80 1 -- 172.21.15.119:0/2984465796 >> 172.21.15.119:0/2984465796 conn(0x5629b9826300 msgr2=0x5629b98ec980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.908+0000 7f48bfb00b80 1 -- 172.21.15.119:0/2984465796 shutdown_connections 2023-12-07T10:21:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.908+0000 7f48bfb00b80 1 -- 172.21.15.119:0/2984465796 wait complete. 2023-12-07T10:21:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.908+0000 7f48bfb00b80 20 client.26065 trim_cache size 0 max 16384 2023-12-07T10:21:33.939 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.938+0000 7f48bfb00b80 1 Processor -- start 2023-12-07T10:21:33.939 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.938+0000 7f48bfb00b80 1 -- start start 2023-12-07T10:21:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.938+0000 7f48bfb00b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629b98f7ad0 0x5629b98fce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.938+0000 7f48bfb00b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b98fca60 0x5629b9907d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.938+0000 7f48bfb00b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629b98fd380 0x5629b99082b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.939+0000 7f48bfb00b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b96ca8c0 con 0x5629b98fca60 2023-12-07T10:21:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.939+0000 7f48bfb00b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b990d940 con 0x5629b98fd380 2023-12-07T10:21:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.939+0000 7f48bfb00b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b9917eb0 con 0x5629b98f7ad0 2023-12-07T10:21:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.939+0000 7f48a979a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b98fca60 0x5629b9907d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.939+0000 7f48a979a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b98fca60 0x5629b9907d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45734/0 (socket says 172.21.15.119:45734) 2023-12-07T10:21:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.939+0000 7f48a979a700 1 -- 172.21.15.119:0/2886679778 learned_addr learned my addr 172.21.15.119:0/2886679778 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.939+0000 7f48aa79c700 1 --2- 172.21.15.119:0/2886679778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629b98fd380 0x5629b99082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.939+0000 7f48a9f9b700 1 --2- 172.21.15.119:0/2886679778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629b98f7ad0 0x5629b98fce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.940+0000 7f48aa79c700 1 -- 172.21.15.119:0/2886679778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629b98f7ad0 msgr2=0x5629b98fce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.940+0000 7f48aa79c700 1 --2- 172.21.15.119:0/2886679778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629b98f7ad0 0x5629b98fce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.940+0000 7f48aa79c700 1 -- 172.21.15.119:0/2886679778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b98fca60 msgr2=0x5629b9907d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.940+0000 7f48aa79c700 1 --2- 172.21.15.119:0/2886679778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b98fca60 0x5629b9907d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.940+0000 7f48aa79c700 1 -- 172.21.15.119:0/2886679778 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629b965dc00 con 0x5629b98fd380 2023-12-07T10:21:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.940+0000 7f48aa79c700 1 --2- 172.21.15.119:0/2886679778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629b98fd380 0x5629b99082b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f489c009fd0 tx=0x7f489c008fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63abbd8331a07f59 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.940+0000 7f4898ff9700 1 -- 172.21.15.119:0/2886679778 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c02a070 con 0x5629b98fd380 2023-12-07T10:21:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.940+0000 7f4898ff9700 1 -- 172.21.15.119:0/2886679778 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f489c00be90 con 0x5629b98fd380 2023-12-07T10:21:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.940+0000 7f4898ff9700 1 -- 172.21.15.119:0/2886679778 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c00e040 con 0x5629b98fd380 2023-12-07T10:21:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.941+0000 7f48bfb00b80 1 -- 172.21.15.119:0/2886679778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629b98fd380 msgr2=0x5629b99082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.941+0000 7f48bfb00b80 1 --2- 172.21.15.119:0/2886679778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629b98fd380 0x5629b99082b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f489c009fd0 tx=0x7f489c008fa0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.941+0000 7f48bfb00b80 1 -- 172.21.15.119:0/2886679778 shutdown_connections 2023-12-07T10:21:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.941+0000 7f48bfb00b80 1 --2- 172.21.15.119:0/2886679778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629b98fd380 0x5629b99082b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.941+0000 7f48bfb00b80 1 --2- 172.21.15.119:0/2886679778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629b98f7ad0 0x5629b98fce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.941+0000 7f48bfb00b80 1 --2- 172.21.15.119:0/2886679778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b98fca60 0x5629b9907d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.941+0000 7f48bfb00b80 1 -- 172.21.15.119:0/2886679778 >> 172.21.15.119:0/2886679778 conn(0x5629b9826300 msgr2=0x5629b98fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.942+0000 7f48bfb00b80 1 -- 172.21.15.119:0/2886679778 shutdown_connections 2023-12-07T10:21:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.942+0000 7f48bfb00b80 1 -- 172.21.15.119:0/2886679778 wait complete. 2023-12-07T10:21:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.942+0000 7f48bfb00b80 1 -- start start 2023-12-07T10:21:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.943+0000 7f48bfb00b80 1 Processor -- start 2023-12-07T10:21:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.943+0000 7f48bfb00b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b98f7ad0 0x5629b9905d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.943+0000 7f48bfb00b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629b98fca60 0x5629b9906290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.943+0000 7f48bfb00b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629b98fd380 0x5629b99067d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.943+0000 7f48bfb00b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b9965da0 con 0x5629b98f7ad0 2023-12-07T10:21:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.943+0000 7f48bfb00b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b9965f10 con 0x5629b98fd380 2023-12-07T10:21:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.943+0000 7f48bfb00b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b99660d0 con 0x5629b98fca60 2023-12-07T10:21:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.943+0000 7f48bfb00b80 10 client.0 _ll_register_callbacks cb 0x5629b97ec8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.944+0000 7f48a979a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629b98fca60 0x5629b9906290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.944+0000 7f48a979a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629b98fca60 0x5629b9906290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:53964/0 (socket says 172.21.15.119:53964) 2023-12-07T10:21:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.944+0000 7f48a979a700 1 -- 172.21.15.119:0/945622388 learned_addr learned my addr 172.21.15.119:0/945622388 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.944+0000 7f48aa79c700 1 --2- 172.21.15.119:0/945622388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629b98fd380 0x5629b99067d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.944+0000 7f48a9f9b700 1 --2- 172.21.15.119:0/945622388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b98f7ad0 0x5629b9905d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.944+0000 7f48a979a700 1 -- 172.21.15.119:0/945622388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629b98fd380 msgr2=0x5629b99067d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.944+0000 7f48a979a700 1 --2- 172.21.15.119:0/945622388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629b98fd380 0x5629b99067d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.944+0000 7f48a979a700 1 -- 172.21.15.119:0/945622388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b98f7ad0 msgr2=0x5629b9905d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.945+0000 7f48a979a700 1 --2- 172.21.15.119:0/945622388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b98f7ad0 0x5629b9905d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.945+0000 7f48a979a700 1 -- 172.21.15.119:0/945622388 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629b9674d80 con 0x5629b98fca60 2023-12-07T10:21:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.945+0000 7f48a9f9b700 1 --2- 172.21.15.119:0/945622388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b98f7ad0 0x5629b9905d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:21:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.945+0000 7f48a979a700 1 --2- 172.21.15.119:0/945622388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629b98fca60 0x5629b9906290 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4894010fd0 tx=0x7f489409a320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93ac6e8f6c6dc700 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:33.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.945+0000 7f488d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:21:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.945+0000 7f488d7fa700 1 -- 172.21.15.119:0/945622388 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489409a9c0 con 0x5629b98fca60 2023-12-07T10:21:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.945+0000 7f488d7fa700 1 -- 172.21.15.119:0/945622388 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f489409eb10 con 0x5629b98fca60 2023-12-07T10:21:33.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.946+0000 7f48bfb00b80 1 -- 172.21.15.119:0/945622388 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f489c007e20 con 0x5629b98fca60 2023-12-07T10:21:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.953+0000 7f488d7fa700 1 -- 172.21.15.119:0/945622388 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48940b3250 con 0x5629b98fca60 2023-12-07T10:21:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.954+0000 7f48bfb00b80 10 client.26071 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.954+0000 7f48bfb00b80 1 -- 172.21.15.119:0/945622388 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5629b969b1f0 con 0x5629b98fca60 2023-12-07T10:21:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.954+0000 7f488d7fa700 1 -- 172.21.15.119:0/945622388 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f489c007e20 con 0x5629b98fca60 2023-12-07T10:21:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.954+0000 7f488d7fa700 1 -- 172.21.15.119:0/945622388 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f48940cb7b0 con 0x5629b98fca60 2023-12-07T10:21:33.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.954+0000 7f48bfb00b80 10 client.26071 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:33.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.954+0000 7f48bfb00b80 10 client.26071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:21:33.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.954+0000 7f48bfb00b80 20 client.26071 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:33.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.954+0000 7f48bfb00b80 1 --2- 172.21.15.119:0/945622388 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5629b9967a80 0x5629b9969e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:33.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.954+0000 7f48bfb00b80 4 client.26071 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:21:33.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.954+0000 7f48bfb00b80 1 -- 172.21.15.119:0/945622388 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5629b9811f70 con 0x5629b9967a80 2023-12-07T10:21:33.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.955+0000 7f48a9f9b700 1 --2- 172.21.15.119:0/945622388 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5629b9967a80 0x5629b9969e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.956+0000 7f48a9f9b700 1 --2- 172.21.15.119:0/945622388 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5629b9967a80 0x5629b9969e70 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af96b86ae1f2d13b server_cookie=681609345d15f80a in_seq=0 out_seq=0 2023-12-07T10:21:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.956+0000 7f488d7fa700 10 client.26071 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.959+0000 7f488d7fa700 1 -- 172.21.15.119:0/945622388 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5629b9811f70 con 0x5629b9967a80 2023-12-07T10:21:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.959+0000 7f488d7fa700 10 client.26071 handle_command_reply: tid=0 2023-12-07T10:21:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.960+0000 7f48bfb00b80 1 client.26071 shutdown 2023-12-07T10:21:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.960+0000 7f48bfb00b80 1 -- 172.21.15.119:0/945622388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629b98fca60 msgr2=0x5629b9906290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.960+0000 7f48bfb00b80 1 --2- 172.21.15.119:0/945622388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629b98fca60 0x5629b9906290 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4894010fd0 tx=0x7f489409a320 comp rx=0 tx=0).stop 2023-12-07T10:21:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.961+0000 7f48bfb00b80 1 -- 172.21.15.119:0/945622388 shutdown_connections 2023-12-07T10:21:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.961+0000 7f48bfb00b80 1 --2- 172.21.15.119:0/945622388 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5629b9967a80 0x5629b9969e70 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.961+0000 7f48bfb00b80 1 --2- 172.21.15.119:0/945622388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5629b98fd380 0x5629b99067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.961+0000 7f48bfb00b80 1 --2- 172.21.15.119:0/945622388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629b98f7ad0 0x5629b9905d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.961+0000 7f48bfb00b80 1 --2- 172.21.15.119:0/945622388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629b98fca60 0x5629b9906290 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.961+0000 7f48bfb00b80 1 -- 172.21.15.119:0/945622388 >> 172.21.15.119:0/945622388 conn(0x5629b9826300 msgr2=0x5629b98fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.961+0000 7f488d7fa700 0 client.26071 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.961+0000 7f48bfb00b80 1 -- 172.21.15.119:0/945622388 shutdown_connections 2023-12-07T10:21:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.961+0000 7f48bfb00b80 1 -- 172.21.15.119:0/945622388 wait complete. 2023-12-07T10:21:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.962+0000 7f48bfb00b80 20 client.26071 trim_cache size 0 max 16384 2023-12-07T10:21:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:21:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:21:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:21:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:21:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.962+0000 7f48ac1ff700 1 -- 172.21.15.119:0/1668578639 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f48900a1ea0 msgr2=0x7f48900a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.962+0000 7f48ac1ff700 1 --2- 172.21.15.119:0/1668578639 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f48900a1ea0 0x7f48900a4360 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x5629b98f3890 tx=0x7f48a0005f50 comp rx=0 tx=0).stop 2023-12-07T10:21:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.963+0000 7f48ac1ff700 1 -- 172.21.15.119:0/1668578639 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48a40742e0 msgr2=0x7f48a419fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.963+0000 7f48ac1ff700 1 --2- 172.21.15.119:0/1668578639 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48a40742e0 0x7f48a419fdf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f489400e9a0 tx=0x7f489400ed60 comp rx=0 tx=0).stop 2023-12-07T10:21:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.963+0000 7f48ac1ff700 1 -- 172.21.15.119:0/1668578639 shutdown_connections 2023-12-07T10:21:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.963+0000 7f48ac1ff700 1 --2- 172.21.15.119:0/1668578639 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f48900a1ea0 0x7f48900a4360 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.963+0000 7f48ac1ff700 1 --2- 172.21.15.119:0/1668578639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48a410ddc0 0x7f48a41999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.963+0000 7f48ac1ff700 1 --2- 172.21.15.119:0/1668578639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a40739c0 0x7f48a419f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.963+0000 7f48ac1ff700 1 --2- 172.21.15.119:0/1668578639 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48a40742e0 0x7f48a419fdf0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.963+0000 7f48ac1ff700 1 -- 172.21.15.119:0/1668578639 >> 172.21.15.119:0/1668578639 conn(0x7f48a40fcfd0 msgr2=0x7f48a40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.965+0000 7f48ac1ff700 1 -- 172.21.15.119:0/1668578639 shutdown_connections 2023-12-07T10:21:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:33.965+0000 7f48ac1ff700 1 -- 172.21.15.119:0/1668578639 wait complete. 2023-12-07T10:21:34.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:33 smithi119 ceph-mon[97908]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 10 op/s 2023-12-07T10:21:34.299 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:33 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:21:34.300 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:33 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:21:34.300 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:33 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:21:34.300 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:33 smithi153 ceph-mon[133625]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 10 op/s 2023-12-07T10:21:34.300 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:33 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:21:34.300 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:33 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:21:34.300 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:33 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:21:34.301 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:33 smithi159 ceph-mon[134726]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 10 op/s 2023-12-07T10:21:34.301 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:33 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:21:34.301 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:33 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:21:34.301 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:33 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:21:35.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:35 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:21:35.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:35 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:21:35.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:35 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:21:35.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:35 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:21:35.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:35 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:21:35.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:35 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:21:36.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:36 smithi119 ceph-mon[97908]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2023-12-07T10:21:36.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:36 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:21:36.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:36 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:21:36.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:36 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:21:36.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:36 smithi159 ceph-mon[134726]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2023-12-07T10:21:36.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:36 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:21:36.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:36 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:21:36.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:36 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:21:36.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:36 smithi153 ceph-mon[133625]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2023-12-07T10:21:36.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:36 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:21:36.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:36 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:21:36.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:36 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:21:38.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:38 smithi119 ceph-mon[97908]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 682 B/s wr, 13 op/s 2023-12-07T10:21:38.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:38 smithi159 ceph-mon[134726]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 682 B/s wr, 13 op/s 2023-12-07T10:21:38.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:38 smithi153 ceph-mon[133625]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 682 B/s wr, 13 op/s 2023-12-07T10:21:40.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:40 smithi119 ceph-mon[97908]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2023-12-07T10:21:40.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:40 smithi159 ceph-mon[134726]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2023-12-07T10:21:40.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:40 smithi153 ceph-mon[133625]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2023-12-07T10:21:42.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:41 smithi119 ceph-mon[97908]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2023-12-07T10:21:42.239 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:41 smithi159 ceph-mon[134726]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2023-12-07T10:21:42.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:41 smithi153 ceph-mon[133625]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2023-12-07T10:21:44.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:43 smithi119 ceph-mon[97908]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2023-12-07T10:21:44.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:43 smithi159 ceph-mon[134726]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2023-12-07T10:21:44.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:43 smithi153 ceph-mon[133625]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2023-12-07T10:21:46.044 INFO:tasks.workunit.client.0.smithi119.stderr:Updating files: 64% (7876/12123) Updating files: 65% (7880/12123) Updating files: 66% (8002/12123) Updating files: 67% (8123/12123) Updating files: 68% (8244/12123) Updating files: 69% (8365/12123) Updating files: 70% (8487/12123) Updating files: 71% (8608/12123) Updating files: 72% (8729/12123) Updating files: 73% (8850/12123) Updating files: 74% (8972/12123) Updating files: 75% (9093/12123) Updating files: 76% (9214/12123) Updating files: 77% (9335/12123) Updating files: 78% (9456/12123) Updating files: 79% (9578/12123) Updating files: 80% (9699/12123) Updating files: 81% (9820/12123) Updating files: 82% (9941/12123) Updating files: 83% (10063/12123) Updating files: 84% (10184/12123) Updating files: 85% (10305/12123) Updating files: 86% (10426/12123) Updating files: 87% (10548/12123) Updating files: 88% (10669/12123) Updating files: 89% (10790/12123) Updating files: 90% (10911/12123) Updating files: 91% (11032/12123) Updating files: 92% (11154/12123) Updating files: 93% (11275/12123) Updating files: 94% (11396/12123) Updating files: 95% (11517/12123) Updating files: 96% (11639/12123) Updating files: 97% (11760/12123) Updating files: 98% (11881/12123) Updating files: 99% (12002/12123) Updating files: 100% (12123/12123) Updating files: 100% (12123/12123), done. 2023-12-07T10:21:46.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:45 smithi119 ceph-mon[97908]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2023-12-07T10:21:46.238 INFO:tasks.workunit.client.0.smithi119.stderr:Note: switching to '3bd0bfa458b4399e7266c56d8215a2cf3a6091a0'. 2023-12-07T10:21:46.238 INFO:tasks.workunit.client.0.smithi119.stderr: 2023-12-07T10:21:46.238 INFO:tasks.workunit.client.0.smithi119.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-12-07T10:21:46.238 INFO:tasks.workunit.client.0.smithi119.stderr:changes and commit them, and you can discard any commits you make in this 2023-12-07T10:21:46.238 INFO:tasks.workunit.client.0.smithi119.stderr:state without impacting any branches by switching back to a branch. 2023-12-07T10:21:46.239 INFO:tasks.workunit.client.0.smithi119.stderr: 2023-12-07T10:21:46.239 INFO:tasks.workunit.client.0.smithi119.stderr:If you want to create a new branch to retain commits you create, you may 2023-12-07T10:21:46.239 INFO:tasks.workunit.client.0.smithi119.stderr:do so (now or later) by using -c with the switch command. Example: 2023-12-07T10:21:46.239 INFO:tasks.workunit.client.0.smithi119.stderr: 2023-12-07T10:21:46.239 INFO:tasks.workunit.client.0.smithi119.stderr: git switch -c 2023-12-07T10:21:46.239 INFO:tasks.workunit.client.0.smithi119.stderr: 2023-12-07T10:21:46.239 INFO:tasks.workunit.client.0.smithi119.stderr:Or undo this operation with: 2023-12-07T10:21:46.239 INFO:tasks.workunit.client.0.smithi119.stderr: 2023-12-07T10:21:46.239 INFO:tasks.workunit.client.0.smithi119.stderr: git switch - 2023-12-07T10:21:46.239 INFO:tasks.workunit.client.0.smithi119.stderr: 2023-12-07T10:21:46.239 INFO:tasks.workunit.client.0.smithi119.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-12-07T10:21:46.239 INFO:tasks.workunit.client.0.smithi119.stderr: 2023-12-07T10:21:46.239 INFO:tasks.workunit.client.0.smithi119.stderr:HEAD is now at 3bd0bfa458b Revert "rgw/lua: print luarocks config during reload" 2023-12-07T10:21:46.245 DEBUG:teuthology.orchestra.run.smithi119:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-12-07T10:21:46.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:45 smithi159 ceph-mon[134726]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2023-12-07T10:21:46.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:45 smithi153 ceph-mon[133625]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2023-12-07T10:21:46.334 INFO:tasks.workunit.client.0.smithi119.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-12-07T10:21:46.339 INFO:tasks.workunit.client.0.smithi119.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-12-07T10:21:46.339 INFO:tasks.workunit.client.0.smithi119.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-12-07T10:21:46.984 INFO:tasks.workunit.client.0.smithi119.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-12-07T10:21:47.120 INFO:tasks.workunit.client.0.smithi119.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-12-07T10:21:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.125+0000 7fcd739ff700 1 -- 172.21.15.119:0/2731170473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd6c0742d0 msgr2=0x7fcd6c111fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.125+0000 7fcd739ff700 1 --2- 172.21.15.119:0/2731170473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd6c0742d0 0x7fcd6c111fb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009990 tx=0x7fcd5c009ca0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.125+0000 7fcd739ff700 1 -- 172.21.15.119:0/2731170473 shutdown_connections 2023-12-07T10:21:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.125+0000 7fcd739ff700 1 --2- 172.21.15.119:0/2731170473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd6c1124f0 0x7fcd6c1148e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.125+0000 7fcd739ff700 1 --2- 172.21.15.119:0/2731170473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd6c0739b0 0x7fcd6c073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.125+0000 7fcd739ff700 1 --2- 172.21.15.119:0/2731170473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd6c0742d0 0x7fcd6c111fb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.125+0000 7fcd739ff700 1 -- 172.21.15.119:0/2731170473 >> 172.21.15.119:0/2731170473 conn(0x7fcd6c0fcfb0 msgr2=0x7fcd6c0ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.125+0000 7fcd739ff700 1 -- 172.21.15.119:0/2731170473 shutdown_connections 2023-12-07T10:21:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.125+0000 7fcd739ff700 1 -- 172.21.15.119:0/2731170473 wait complete. 2023-12-07T10:21:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.126+0000 7fcd739ff700 1 Processor -- start 2023-12-07T10:21:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.126+0000 7fcd739ff700 1 -- start start 2023-12-07T10:21:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.126+0000 7fcd739ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd6c0739b0 0x7fcd6c19dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.126+0000 7fcd739ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd6c0742d0 0x7fcd6c19e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.126+0000 7fcd739ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd6c1124f0 0x7fcd6c1a2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.126+0000 7fcd739ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c1a2ec0 con 0x7fcd6c0742d0 2023-12-07T10:21:47.128 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.126+0000 7fcd739ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c1a3030 con 0x7fcd6c0739b0 2023-12-07T10:21:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.126+0000 7fcd739ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c1a31f0 con 0x7fcd6c1124f0 2023-12-07T10:21:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.127+0000 7fcd70f9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd6c0742d0 0x7fcd6c19e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.127+0000 7fcd70f9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd6c0742d0 0x7fcd6c19e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54038/0 (socket says 172.21.15.119:54038) 2023-12-07T10:21:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.127+0000 7fcd70f9a700 1 -- 172.21.15.119:0/1379225211 learned_addr learned my addr 172.21.15.119:0/1379225211 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.127+0000 7fcd7179b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd6c0739b0 0x7fcd6c19dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.127+0000 7fcd71f9c700 1 --2- 172.21.15.119:0/1379225211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd6c1124f0 0x7fcd6c1a2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.127+0000 7fcd70f9a700 1 -- 172.21.15.119:0/1379225211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd6c1124f0 msgr2=0x7fcd6c1a2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.127+0000 7fcd70f9a700 1 --2- 172.21.15.119:0/1379225211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd6c1124f0 0x7fcd6c1a2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.127+0000 7fcd70f9a700 1 -- 172.21.15.119:0/1379225211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd6c0739b0 msgr2=0x7fcd6c19dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.127+0000 7fcd70f9a700 1 --2- 172.21.15.119:0/1379225211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd6c0739b0 0x7fcd6c19dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.127+0000 7fcd70f9a700 1 -- 172.21.15.119:0/1379225211 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd64009c40 con 0x7fcd6c0742d0 2023-12-07T10:21:47.129 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.127+0000 7fcd70f9a700 1 --2- 172.21.15.119:0/1379225211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd6c0742d0 0x7fcd6c19e530 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c005770 tx=0x7fcd5c0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.127+0000 7fcd627fc700 1 -- 172.21.15.119:0/1379225211 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c01d070 con 0x7fcd6c0742d0 2023-12-07T10:21:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.128+0000 7fcd739ff700 1 -- 172.21.15.119:0/1379225211 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5c0095f0 con 0x7fcd6c0742d0 2023-12-07T10:21:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.128+0000 7fcd627fc700 1 -- 172.21.15.119:0/1379225211 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd5c021410 con 0x7fcd6c0742d0 2023-12-07T10:21:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.128+0000 7fcd739ff700 1 -- 172.21.15.119:0/1379225211 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd6c1aab30 con 0x7fcd6c0742d0 2023-12-07T10:21:47.130 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.128+0000 7fcd627fc700 1 -- 172.21.15.119:0/1379225211 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c0175f0 con 0x7fcd6c0742d0 2023-12-07T10:21:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.131+0000 7fcd627fc700 1 -- 172.21.15.119:0/1379225211 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 25) v1 ==== 136717+0+0 (secure 0 0 0) 0x7fcd5c0217f0 con 0x7fcd6c0742d0 2023-12-07T10:21:47.133 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.132+0000 7fcd872bdb80 1 Processor -- start 2023-12-07T10:21:47.133 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.132+0000 7fcd872bdb80 1 -- start start 2023-12-07T10:21:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.133+0000 7fcd872bdb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98eaa60 0x5582e98eae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.133+0000 7fcd872bdb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582e98eb380 0x5582e98e5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.133+0000 7fcd872bdb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582e98e5ad0 0x5582e98e5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.133+0000 7fcd872bdb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e9859760 con 0x5582e98eaa60 2023-12-07T10:21:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.133+0000 7fcd872bdb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e9840740 con 0x5582e98eb380 2023-12-07T10:21:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.133+0000 7fcd872bdb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e983e060 con 0x5582e98e5ad0 2023-12-07T10:21:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.133+0000 7fcd71f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98eaa60 0x5582e98eae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.133+0000 7fcd71f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98eaa60 0x5582e98eae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54052/0 (socket says 172.21.15.119:54052) 2023-12-07T10:21:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.133+0000 7fcd71f9c700 1 -- 172.21.15.119:0/1545596035 learned_addr learned my addr 172.21.15.119:0/1545596035 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.133+0000 7fcd7179b700 1 --2- 172.21.15.119:0/1545596035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582e98eb380 0x5582e98e5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.133+0000 7fcd70f9a700 1 --2- 172.21.15.119:0/1545596035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582e98e5ad0 0x5582e98e5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.133+0000 7fcd627fc700 1 --2- 172.21.15.119:0/1379225211 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fcd580a1da0 0x7fcd580a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.133+0000 7fcd7179b700 1 -- 172.21.15.119:0/1545596035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582e98e5ad0 msgr2=0x5582e98e5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.133+0000 7fcd7179b700 1 --2- 172.21.15.119:0/1545596035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582e98e5ad0 0x5582e98e5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.133+0000 7fcd7179b700 1 -- 172.21.15.119:0/1545596035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98eaa60 msgr2=0x5582e98eae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.133+0000 7fcd7179b700 1 --2- 172.21.15.119:0/1545596035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98eaa60 0x5582e98eae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.133+0000 7fcd7179b700 1 -- 172.21.15.119:0/1545596035 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582e96b65d0 con 0x5582e98eb380 2023-12-07T10:21:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.133+0000 7fcd70f9a700 1 --2- 172.21.15.119:0/1545596035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582e98e5ad0 0x5582e98e5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:21:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.134+0000 7fcd627fc700 1 -- 172.21.15.119:0/1379225211 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fcd5c0cefd0 con 0x7fcd6c0742d0 2023-12-07T10:21:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.134+0000 7fcd71f9c700 1 --2- 172.21.15.119:0/1379225211 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fcd580a1da0 0x7fcd580a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.134+0000 7fcd7179b700 1 --2- 172.21.15.119:0/1545596035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582e98eb380 0x5582e98e5540 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009cb0 tx=0x7fcd68011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef3bb3e634547a03 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.134+0000 7fcd577fe700 1 -- 172.21.15.119:0/1545596035 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68011d20 con 0x5582e98eb380 2023-12-07T10:21:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.134+0000 7fcd577fe700 1 -- 172.21.15.119:0/1545596035 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd68011e80 con 0x5582e98eb380 2023-12-07T10:21:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.134+0000 7fcd71f9c700 1 --2- 172.21.15.119:0/1379225211 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fcd580a1da0 0x7fcd580a4260 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcd640094b0 tx=0x7fcd64004ad0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.134+0000 7fcd577fe700 1 -- 172.21.15.119:0/1545596035 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68005280 con 0x5582e98eb380 2023-12-07T10:21:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.135+0000 7fcd872bdb80 1 -- 172.21.15.119:0/1545596035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582e98eb380 msgr2=0x5582e98e5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.135+0000 7fcd872bdb80 1 --2- 172.21.15.119:0/1545596035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582e98eb380 0x5582e98e5540 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009cb0 tx=0x7fcd68011440 comp rx=0 tx=0).stop 2023-12-07T10:21:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.135+0000 7fcd872bdb80 1 -- 172.21.15.119:0/1545596035 shutdown_connections 2023-12-07T10:21:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.135+0000 7fcd872bdb80 1 --2- 172.21.15.119:0/1545596035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582e98e5ad0 0x5582e98e5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.135+0000 7fcd872bdb80 1 --2- 172.21.15.119:0/1545596035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98eaa60 0x5582e98eae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.135+0000 7fcd872bdb80 1 --2- 172.21.15.119:0/1545596035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582e98eb380 0x5582e98e5540 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.135+0000 7fcd872bdb80 1 -- 172.21.15.119:0/1545596035 >> 172.21.15.119:0/1545596035 conn(0x5582e9814300 msgr2=0x5582e98dfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.135+0000 7fcd872bdb80 1 -- 172.21.15.119:0/1545596035 shutdown_connections 2023-12-07T10:21:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.135+0000 7fcd872bdb80 1 -- 172.21.15.119:0/1545596035 wait complete. 2023-12-07T10:21:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.136+0000 7fcd872bdb80 1 -- start start 2023-12-07T10:21:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.136+0000 7fcd872bdb80 1 Processor -- start 2023-12-07T10:21:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.136+0000 7fcd872bdb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98e5ad0 0x5582e98ee6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.136+0000 7fcd872bdb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582e98eaa60 0x5582e98eec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.136+0000 7fcd872bdb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582e98eb380 0x5582e98ef150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.137+0000 7fcd872bdb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e9839950 con 0x5582e98e5ad0 2023-12-07T10:21:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.137+0000 7fcd872bdb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e97f3500 con 0x5582e98eb380 2023-12-07T10:21:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.137+0000 7fcd872bdb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e97f31a0 con 0x5582e98eaa60 2023-12-07T10:21:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.137+0000 7fcd872bdb80 10 client.0 _ll_register_callbacks cb 0x5582e97da8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.137+0000 7fcd7179b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98e5ad0 0x5582e98ee6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.137+0000 7fcd7179b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98e5ad0 0x5582e98ee6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54054/0 (socket says 172.21.15.119:54054) 2023-12-07T10:21:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.137+0000 7fcd7179b700 1 -- 172.21.15.119:0/3598025153 learned_addr learned my addr 172.21.15.119:0/3598025153 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.137+0000 7fcd70f9a700 1 --2- 172.21.15.119:0/3598025153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582e98eaa60 0x5582e98eec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.137+0000 7fcd7179b700 1 -- 172.21.15.119:0/3598025153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582e98eaa60 msgr2=0x5582e98eec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.137+0000 7fcd7179b700 1 --2- 172.21.15.119:0/3598025153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582e98eaa60 0x5582e98eec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.137+0000 7fcd7179b700 1 -- 172.21.15.119:0/3598025153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582e98eb380 msgr2=0x5582e98ef150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.137+0000 7fcd7179b700 1 --2- 172.21.15.119:0/3598025153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582e98eb380 0x5582e98ef150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.137+0000 7fcd7179b700 1 -- 172.21.15.119:0/3598025153 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd680098e0 con 0x5582e98e5ad0 2023-12-07T10:21:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.137+0000 7fcd70f9a700 1 --2- 172.21.15.119:0/3598025153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582e98eaa60 0x5582e98eec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:21:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.138+0000 7fcd7179b700 1 --2- 172.21.15.119:0/3598025153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98e5ad0 0x5582e98ee6d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fcd680097c0 tx=0x7fcd68011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a17b8816e4f943 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.138+0000 7fcd54ff9700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:21:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.138+0000 7fcd54ff9700 1 -- 172.21.15.119:0/3598025153 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68004050 con 0x5582e98e5ad0 2023-12-07T10:21:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.138+0000 7fcd54ff9700 1 -- 172.21.15.119:0/3598025153 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fcd680041b0 con 0x5582e98e5ad0 2023-12-07T10:21:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.138+0000 7fcd872bdb80 1 -- 172.21.15.119:0/3598025153 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5582e98b4740 con 0x5582e98e5ad0 2023-12-07T10:21:47.142 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.138+0000 7fcd54ff9700 1 -- 172.21.15.119:0/3598025153 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd68004370 con 0x5582e98e5ad0 2023-12-07T10:21:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.138+0000 7fcd872bdb80 10 client.16080 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.138+0000 7fcd872bdb80 1 -- 172.21.15.119:0/3598025153 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5582e9662d80 con 0x5582e98e5ad0 2023-12-07T10:21:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.138+0000 7fcd54ff9700 1 -- 172.21.15.119:0/3598025153 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5582e98b4740 con 0x5582e98e5ad0 2023-12-07T10:21:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.139+0000 7fcd54ff9700 1 -- 172.21.15.119:0/3598025153 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcd680050b0 con 0x5582e98e5ad0 2023-12-07T10:21:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.139+0000 7fcd872bdb80 10 client.16080 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.139+0000 7fcd872bdb80 10 client.16080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:21:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.139+0000 7fcd872bdb80 20 client.16080 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.139+0000 7fcd872bdb80 1 --2- 172.21.15.119:0/3598025153 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5582e98fed80 0x5582e9901170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.139+0000 7fcd872bdb80 4 client.16080 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:21:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.139+0000 7fcd872bdb80 1 -- 172.21.15.119:0/3598025153 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582e97fff70 con 0x5582e98fed80 2023-12-07T10:21:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.139+0000 7fcd70f9a700 1 --2- 172.21.15.119:0/3598025153 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5582e98fed80 0x5582e9901170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.140+0000 7fcd54ff9700 10 client.16080 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.140+0000 7fcd70f9a700 1 --2- 172.21.15.119:0/3598025153 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5582e98fed80 0x5582e9901170 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa40c549fbb1b839 server_cookie=9d79d0265abc4bb4 in_seq=0 out_seq=0 2023-12-07T10:21:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.140+0000 7fcd54ff9700 1 -- 172.21.15.119:0/3598025153 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5582e97fff70 con 0x5582e98fed80 2023-12-07T10:21:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.141+0000 7fcd54ff9700 10 client.16080 handle_command_reply: tid=0 2023-12-07T10:21:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.141+0000 7fcd872bdb80 1 client.16080 shutdown 2023-12-07T10:21:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.141+0000 7fcd872bdb80 1 -- 172.21.15.119:0/3598025153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98e5ad0 msgr2=0x5582e98ee6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.141+0000 7fcd872bdb80 1 --2- 172.21.15.119:0/3598025153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98e5ad0 0x5582e98ee6d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fcd680097c0 tx=0x7fcd68011bd0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.141+0000 7fcd872bdb80 1 -- 172.21.15.119:0/3598025153 shutdown_connections 2023-12-07T10:21:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.141+0000 7fcd872bdb80 1 --2- 172.21.15.119:0/3598025153 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5582e98fed80 0x5582e9901170 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.141+0000 7fcd872bdb80 1 --2- 172.21.15.119:0/3598025153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582e98eb380 0x5582e98ef150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.141+0000 7fcd872bdb80 1 --2- 172.21.15.119:0/3598025153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98e5ad0 0x5582e98ee6d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.141+0000 7fcd872bdb80 1 --2- 172.21.15.119:0/3598025153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582e98eaa60 0x5582e98eec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.141+0000 7fcd872bdb80 1 -- 172.21.15.119:0/3598025153 >> 172.21.15.119:0/3598025153 conn(0x5582e9814300 msgr2=0x5582e98da980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.141+0000 7fcd54ff9700 0 client.16080 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.141+0000 7fcd872bdb80 1 -- 172.21.15.119:0/3598025153 shutdown_connections 2023-12-07T10:21:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.141+0000 7fcd872bdb80 1 -- 172.21.15.119:0/3598025153 wait complete. 2023-12-07T10:21:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.141+0000 7fcd872bdb80 20 client.16080 trim_cache size 0 max 16384 2023-12-07T10:21:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.170+0000 7fcd872bdb80 1 Processor -- start 2023-12-07T10:21:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.171+0000 7fcd872bdb80 1 -- start start 2023-12-07T10:21:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.171+0000 7fcd872bdb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582e98e5ad0 0x5582e98e5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.171+0000 7fcd872bdb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582e98eaa60 0x5582e98eef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.171+0000 7fcd872bdb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98eb380 0x5582e98ecfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.171+0000 7fcd872bdb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e96b88c0 con 0x5582e98eb380 2023-12-07T10:21:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.171+0000 7fcd872bdb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e98efca0 con 0x5582e98e5ad0 2023-12-07T10:21:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.171+0000 7fcd872bdb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e9906380 con 0x5582e98eaa60 2023-12-07T10:21:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.171+0000 7fcd71f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98eb380 0x5582e98ecfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.171+0000 7fcd71f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98eb380 0x5582e98ecfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54062/0 (socket says 172.21.15.119:54062) 2023-12-07T10:21:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.171+0000 7fcd71f9c700 1 -- 172.21.15.119:0/4147080537 learned_addr learned my addr 172.21.15.119:0/4147080537 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.171+0000 7fcd70f9a700 1 --2- 172.21.15.119:0/4147080537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582e98eaa60 0x5582e98eef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.171+0000 7fcd7179b700 1 --2- 172.21.15.119:0/4147080537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582e98e5ad0 0x5582e98e5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.172+0000 7fcd71f9c700 1 -- 172.21.15.119:0/4147080537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582e98eaa60 msgr2=0x5582e98eef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.172+0000 7fcd71f9c700 1 --2- 172.21.15.119:0/4147080537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582e98eaa60 0x5582e98eef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.172+0000 7fcd71f9c700 1 -- 172.21.15.119:0/4147080537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582e98e5ad0 msgr2=0x5582e98e5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.172+0000 7fcd71f9c700 1 --2- 172.21.15.119:0/4147080537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582e98e5ad0 0x5582e98e5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.172+0000 7fcd71f9c700 1 -- 172.21.15.119:0/4147080537 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582e964bc00 con 0x5582e98eb380 2023-12-07T10:21:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.172+0000 7fcd71f9c700 1 --2- 172.21.15.119:0/4147080537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98eb380 0x5582e98ecfc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009fd0 tx=0x7fcd64004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd284b964c700f7 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.172+0000 7fcd57fff700 1 -- 172.21.15.119:0/4147080537 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6400e040 con 0x5582e98eb380 2023-12-07T10:21:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.172+0000 7fcd57fff700 1 -- 172.21.15.119:0/4147080537 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd6400f040 con 0x5582e98eb380 2023-12-07T10:21:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.172+0000 7fcd57fff700 1 -- 172.21.15.119:0/4147080537 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6400e6f0 con 0x5582e98eb380 2023-12-07T10:21:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.173+0000 7fcd872bdb80 1 -- 172.21.15.119:0/4147080537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98eb380 msgr2=0x5582e98ecfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.173+0000 7fcd872bdb80 1 --2- 172.21.15.119:0/4147080537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98eb380 0x5582e98ecfc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009fd0 tx=0x7fcd64004b40 comp rx=0 tx=0).stop 2023-12-07T10:21:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.173+0000 7fcd872bdb80 1 -- 172.21.15.119:0/4147080537 shutdown_connections 2023-12-07T10:21:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.173+0000 7fcd872bdb80 1 --2- 172.21.15.119:0/4147080537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98eb380 0x5582e98ecfc0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.173+0000 7fcd872bdb80 1 --2- 172.21.15.119:0/4147080537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582e98e5ad0 0x5582e98e5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.173+0000 7fcd872bdb80 1 --2- 172.21.15.119:0/4147080537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582e98eaa60 0x5582e98eef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.173+0000 7fcd872bdb80 1 -- 172.21.15.119:0/4147080537 >> 172.21.15.119:0/4147080537 conn(0x5582e9814300 msgr2=0x5582e98e88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.173+0000 7fcd872bdb80 1 -- 172.21.15.119:0/4147080537 shutdown_connections 2023-12-07T10:21:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.173+0000 7fcd872bdb80 1 -- 172.21.15.119:0/4147080537 wait complete. 2023-12-07T10:21:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.174+0000 7fcd872bdb80 1 -- start start 2023-12-07T10:21:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.174+0000 7fcd872bdb80 1 Processor -- start 2023-12-07T10:21:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.174+0000 7fcd872bdb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98e5ad0 0x5582e98f3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.174+0000 7fcd872bdb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582e98eaa60 0x5582e98f4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.174+0000 7fcd872bdb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582e98eb380 0x5582e9954270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.174+0000 7fcd872bdb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e99547b0 con 0x5582e98e5ad0 2023-12-07T10:21:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.174+0000 7fcd872bdb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e99548f0 con 0x5582e98eb380 2023-12-07T10:21:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.174+0000 7fcd872bdb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e9954a30 con 0x5582e98eaa60 2023-12-07T10:21:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.174+0000 7fcd872bdb80 10 client.0 _ll_register_callbacks cb 0x5582e97da8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.175+0000 7fcd7179b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98e5ad0 0x5582e98f3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.175+0000 7fcd7179b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98e5ad0 0x5582e98f3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54068/0 (socket says 172.21.15.119:54068) 2023-12-07T10:21:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.175+0000 7fcd7179b700 1 -- 172.21.15.119:0/2708776928 learned_addr learned my addr 172.21.15.119:0/2708776928 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.175+0000 7fcd70f9a700 1 --2- 172.21.15.119:0/2708776928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582e98eaa60 0x5582e98f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.175+0000 7fcd71f9c700 1 --2- 172.21.15.119:0/2708776928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582e98eb380 0x5582e9954270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.175+0000 7fcd7179b700 1 -- 172.21.15.119:0/2708776928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582e98eaa60 msgr2=0x5582e98f4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.175+0000 7fcd7179b700 1 --2- 172.21.15.119:0/2708776928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582e98eaa60 0x5582e98f4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.175+0000 7fcd7179b700 1 -- 172.21.15.119:0/2708776928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582e98eb380 msgr2=0x5582e9954270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.175+0000 7fcd7179b700 1 --2- 172.21.15.119:0/2708776928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582e98eb380 0x5582e9954270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.175+0000 7fcd7179b700 1 -- 172.21.15.119:0/2708776928 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd64017440 con 0x5582e98e5ad0 2023-12-07T10:21:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.175+0000 7fcd70f9a700 1 --2- 172.21.15.119:0/2708776928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582e98eaa60 0x5582e98f4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.175+0000 7fcd7179b700 1 --2- 172.21.15.119:0/2708776928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98e5ad0 0x5582e98f3fc0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fcd68001eb0 tx=0x7fcd68005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf33819c4039b01e server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.175+0000 7fcd54ff9700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:21:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.176+0000 7fcd54ff9700 1 -- 172.21.15.119:0/2708776928 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68009260 con 0x5582e98e5ad0 2023-12-07T10:21:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.176+0000 7fcd872bdb80 1 -- 172.21.15.119:0/2708776928 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5582e98d86f0 con 0x5582e98e5ad0 2023-12-07T10:21:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.176+0000 7fcd54ff9700 1 -- 172.21.15.119:0/2708776928 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fcd68000e40 con 0x5582e98e5ad0 2023-12-07T10:21:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.176+0000 7fcd54ff9700 1 -- 172.21.15.119:0/2708776928 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd68005630 con 0x5582e98e5ad0 2023-12-07T10:21:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.176+0000 7fcd872bdb80 10 client.16086 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.176+0000 7fcd872bdb80 1 -- 172.21.15.119:0/2708776928 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5582e96891f0 con 0x5582e98e5ad0 2023-12-07T10:21:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.176+0000 7fcd54ff9700 1 -- 172.21.15.119:0/2708776928 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5582e98d86f0 con 0x5582e98e5ad0 2023-12-07T10:21:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.176+0000 7fcd54ff9700 1 -- 172.21.15.119:0/2708776928 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcd680037a0 con 0x5582e98e5ad0 2023-12-07T10:21:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.176+0000 7fcd872bdb80 10 client.16086 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.176+0000 7fcd872bdb80 10 client.16086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:21:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.176+0000 7fcd872bdb80 20 client.16086 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.176+0000 7fcd872bdb80 1 --2- 172.21.15.119:0/2708776928 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5582e9955fa0 0x5582e99583c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:47.181 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.176+0000 7fcd872bdb80 4 client.16086 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:21:47.181 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.176+0000 7fcd872bdb80 1 -- 172.21.15.119:0/2708776928 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5582e97f2430 con 0x5582e9955fa0 2023-12-07T10:21:47.181 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.177+0000 7fcd70f9a700 1 --2- 172.21.15.119:0/2708776928 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5582e9955fa0 0x5582e99583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:47.181 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.177+0000 7fcd54ff9700 10 client.16086 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:47.181 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.177+0000 7fcd70f9a700 1 --2- 172.21.15.119:0/2708776928 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5582e9955fa0 0x5582e99583c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cca2b437e9442295 server_cookie=59210085a6d01ac2 in_seq=0 out_seq=0 2023-12-07T10:21:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.182+0000 7fcd54ff9700 1 -- 172.21.15.119:0/2708776928 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5582e97f2430 con 0x5582e9955fa0 2023-12-07T10:21:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.182+0000 7fcd54ff9700 10 client.16086 handle_command_reply: tid=0 2023-12-07T10:21:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.182+0000 7fcd872bdb80 1 client.16086 shutdown 2023-12-07T10:21:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.182+0000 7fcd872bdb80 1 -- 172.21.15.119:0/2708776928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98e5ad0 msgr2=0x5582e98f3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.182+0000 7fcd872bdb80 1 --2- 172.21.15.119:0/2708776928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98e5ad0 0x5582e98f3fc0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fcd68001eb0 tx=0x7fcd68005810 comp rx=0 tx=0).stop 2023-12-07T10:21:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.182+0000 7fcd872bdb80 1 -- 172.21.15.119:0/2708776928 shutdown_connections 2023-12-07T10:21:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.182+0000 7fcd872bdb80 1 --2- 172.21.15.119:0/2708776928 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5582e9955fa0 0x5582e99583c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.182+0000 7fcd872bdb80 1 --2- 172.21.15.119:0/2708776928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5582e98eb380 0x5582e9954270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.182+0000 7fcd872bdb80 1 --2- 172.21.15.119:0/2708776928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e98e5ad0 0x5582e98f3fc0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.182+0000 7fcd872bdb80 1 --2- 172.21.15.119:0/2708776928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582e98eaa60 0x5582e98f4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.182+0000 7fcd54ff9700 0 client.16086 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.182+0000 7fcd872bdb80 1 -- 172.21.15.119:0/2708776928 >> 172.21.15.119:0/2708776928 conn(0x5582e9814300 msgr2=0x5582e98eefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.182+0000 7fcd872bdb80 1 -- 172.21.15.119:0/2708776928 shutdown_connections 2023-12-07T10:21:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.182+0000 7fcd872bdb80 1 -- 172.21.15.119:0/2708776928 wait complete. 2023-12-07T10:21:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.183+0000 7fcd872bdb80 20 client.16086 trim_cache size 0 max 16384 2023-12-07T10:21:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:21:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:21:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:21:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:21:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.183+0000 7fcd739ff700 1 -- 172.21.15.119:0/1379225211 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fcd580a1da0 msgr2=0x7fcd580a4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.183+0000 7fcd739ff700 1 --2- 172.21.15.119:0/1379225211 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fcd580a1da0 0x7fcd580a4260 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcd640094b0 tx=0x7fcd64004ad0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.183+0000 7fcd739ff700 1 -- 172.21.15.119:0/1379225211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd6c0742d0 msgr2=0x7fcd6c19e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.183+0000 7fcd739ff700 1 --2- 172.21.15.119:0/1379225211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd6c0742d0 0x7fcd6c19e530 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c005770 tx=0x7fcd5c0057a0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.183+0000 7fcd739ff700 1 -- 172.21.15.119:0/1379225211 shutdown_connections 2023-12-07T10:21:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.183+0000 7fcd739ff700 1 --2- 172.21.15.119:0/1379225211 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fcd580a1da0 0x7fcd580a4260 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.183+0000 7fcd739ff700 1 --2- 172.21.15.119:0/1379225211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd6c1124f0 0x7fcd6c1a2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.183+0000 7fcd739ff700 1 --2- 172.21.15.119:0/1379225211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd6c0739b0 0x7fcd6c19dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.183+0000 7fcd739ff700 1 --2- 172.21.15.119:0/1379225211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd6c0742d0 0x7fcd6c19e530 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.183+0000 7fcd739ff700 1 -- 172.21.15.119:0/1379225211 >> 172.21.15.119:0/1379225211 conn(0x7fcd6c0fcfb0 msgr2=0x7fcd6c0fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.183+0000 7fcd739ff700 1 -- 172.21.15.119:0/1379225211 shutdown_connections 2023-12-07T10:21:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:21:47.183+0000 7fcd739ff700 1 -- 172.21.15.119:0/1379225211 wait complete. 2023-12-07T10:21:47.203 INFO:tasks.workunit.client.0.smithi119.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-12-07T10:21:47.207 INFO:tasks.workunit.client.0.smithi119.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-12-07T10:21:47.207 INFO:tasks.workunit.client.0.smithi119.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-12-07T10:21:47.271 INFO:tasks.workunit.client.0.smithi119.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-12-07T10:21:47.278 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-12-07T10:21:47.278 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-12-07T10:21:47.321 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2023-12-07T10:21:47.322 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2023-12-07T10:21:47.322 DEBUG:teuthology.orchestra.run.smithi119:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2023-12-07T10:21:47.418 INFO:tasks.workunit.client.0.smithi119.stderr:+ dbench 1 2023-12-07T10:21:47.436 INFO:tasks.workunit.client.0.smithi119.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2023-12-07T10:21:47.436 INFO:tasks.workunit.client.0.smithi119.stdout: 2023-12-07T10:21:47.436 INFO:tasks.workunit.client.0.smithi119.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2023-12-07T10:21:47.437 INFO:tasks.workunit.client.0.smithi119.stdout:0 of 1 processes prepared for launch 0 sec 2023-12-07T10:21:47.538 INFO:tasks.workunit.client.0.smithi119.stdout:1 of 1 processes prepared for launch 0 sec 2023-12-07T10:21:47.538 INFO:tasks.workunit.client.0.smithi119.stdout:releasing clients 2023-12-07T10:21:48.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:47 smithi119 ceph-mon[97908]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2023-12-07T10:21:48.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:47 smithi159 ceph-mon[134726]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2023-12-07T10:21:48.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:47 smithi153 ceph-mon[133625]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2023-12-07T10:21:48.538 INFO:tasks.workunit.client.0.smithi119.stdout: 1 3209 34.27 MB/sec warmup 1 sec latency 6.682 ms 2023-12-07T10:21:49.538 INFO:tasks.workunit.client.0.smithi119.stdout: 1 6049 23.92 MB/sec warmup 2 sec latency 6.506 ms 2023-12-07T10:21:50.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:49 smithi119 ceph-mon[97908]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-07T10:21:50.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:49 smithi159 ceph-mon[134726]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-07T10:21:50.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:49 smithi153 ceph-mon[133625]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-07T10:21:50.538 INFO:tasks.workunit.client.0.smithi119.stdout: 1 8555 21.42 MB/sec warmup 3 sec latency 13.858 ms 2023-12-07T10:21:51.542 INFO:tasks.workunit.client.0.smithi119.stdout: 1 11454 19.70 MB/sec warmup 4 sec latency 10.154 ms 2023-12-07T10:21:52.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:51 smithi119 ceph-mon[97908]: pgmap v46: 129 pgs: 129 active+clean; 6.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 505 KiB/s wr, 71 op/s 2023-12-07T10:21:52.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:51 smithi159 ceph-mon[134726]: pgmap v46: 129 pgs: 129 active+clean; 6.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 505 KiB/s wr, 71 op/s 2023-12-07T10:21:52.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:51 smithi153 ceph-mon[133625]: pgmap v46: 129 pgs: 129 active+clean; 6.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 505 KiB/s wr, 71 op/s 2023-12-07T10:21:52.538 INFO:tasks.workunit.client.0.smithi119.stdout: 1 14646 19.48 MB/sec warmup 5 sec latency 18.507 ms 2023-12-07T10:21:53.156 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-07T10:21:53.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.414+0000 7f7ac73ff700 1 -- 172.21.15.119:0/1131287304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ac0068a50 msgr2=0x7f7ac0068e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:53.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.414+0000 7f7ac73ff700 1 --2- 172.21.15.119:0/1131287304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ac0068a50 0x7f7ac0068e30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009990 tx=0x7f7abc009ca0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.414+0000 7f7ac73ff700 1 -- 172.21.15.119:0/1131287304 shutdown_connections 2023-12-07T10:21:53.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.414+0000 7f7ac73ff700 1 --2- 172.21.15.119:0/1131287304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ac0106ad0 0x7f7ac010e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.414+0000 7f7ac73ff700 1 --2- 172.21.15.119:0/1131287304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ac0068a50 0x7f7ac0068e30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.414+0000 7f7ac73ff700 1 --2- 172.21.15.119:0/1131287304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ac0106110 0x7f7ac0106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.414+0000 7f7ac73ff700 1 -- 172.21.15.119:0/1131287304 >> 172.21.15.119:0/1131287304 conn(0x7f7ac0077610 msgr2=0x7f7ac0077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:53.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.414+0000 7f7ac73ff700 1 -- 172.21.15.119:0/1131287304 shutdown_connections 2023-12-07T10:21:53.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.414+0000 7f7ac73ff700 1 -- 172.21.15.119:0/1131287304 wait complete. 2023-12-07T10:21:53.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.415+0000 7f7ac73ff700 1 Processor -- start 2023-12-07T10:21:53.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.415+0000 7f7ac73ff700 1 -- start start 2023-12-07T10:21:53.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.415+0000 7f7ac73ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ac0068a50 0x7f7ac0199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:53.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.416+0000 7f7ac73ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ac0106110 0x7f7ac019a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:53.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.416+0000 7f7ac73ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ac0106ad0 0x7f7ac019e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:53.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.416+0000 7f7ac73ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac019eb40 con 0x7f7ac0106ad0 2023-12-07T10:21:53.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.416+0000 7f7ac73ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac019ecb0 con 0x7f7ac0106110 2023-12-07T10:21:53.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.416+0000 7f7ac73ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac019ee70 con 0x7f7ac0068a50 2023-12-07T10:21:53.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.416+0000 7f7ac519b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ac0068a50 0x7f7ac0199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:53.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.416+0000 7f7ac519b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ac0068a50 0x7f7ac0199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:39820/0 (socket says 172.21.15.119:39820) 2023-12-07T10:21:53.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.416+0000 7f7ac519b700 1 -- 172.21.15.119:0/1019920379 learned_addr learned my addr 172.21.15.119:0/1019920379 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:53.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.416+0000 7f7ac499a700 1 --2- 172.21.15.119:0/1019920379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ac0106110 0x7f7ac019a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:53.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.416+0000 7f7ac599c700 1 --2- 172.21.15.119:0/1019920379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ac0106ad0 0x7f7ac019e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:53.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.416+0000 7f7ac519b700 1 -- 172.21.15.119:0/1019920379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ac0106110 msgr2=0x7f7ac019a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:53.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.416+0000 7f7ac519b700 1 --2- 172.21.15.119:0/1019920379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ac0106110 0x7f7ac019a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.416+0000 7f7ac519b700 1 -- 172.21.15.119:0/1019920379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ac0106ad0 msgr2=0x7f7ac019e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:53.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.416+0000 7f7ac519b700 1 --2- 172.21.15.119:0/1019920379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ac0106ad0 0x7f7ac019e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.416+0000 7f7ac519b700 1 -- 172.21.15.119:0/1019920379 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7abc0095f0 con 0x7f7ac0068a50 2023-12-07T10:21:53.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.416+0000 7f7ac599c700 1 --2- 172.21.15.119:0/1019920379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ac0106ad0 0x7f7ac019e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:21:53.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.416+0000 7f7ac519b700 1 --2- 172.21.15.119:0/1019920379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ac0068a50 0x7f7ac0199db0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7abc000c00 tx=0x7f7abc005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:53.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.417+0000 7f7ab67fc700 1 -- 172.21.15.119:0/1019920379 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc01d070 con 0x7f7ac0068a50 2023-12-07T10:21:53.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.417+0000 7f7ab67fc700 1 -- 172.21.15.119:0/1019920379 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7abc022470 con 0x7f7ac0068a50 2023-12-07T10:21:53.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.417+0000 7f7ac73ff700 1 -- 172.21.15.119:0/1019920379 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ac019f0f0 con 0x7f7ac0068a50 2023-12-07T10:21:53.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.417+0000 7f7ab67fc700 1 -- 172.21.15.119:0/1019920379 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc017650 con 0x7f7ac0068a50 2023-12-07T10:21:53.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.417+0000 7f7ac73ff700 1 -- 172.21.15.119:0/1019920379 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac01a69b0 con 0x7f7ac0068a50 2023-12-07T10:21:53.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.420+0000 7f7ab67fc700 1 -- 172.21.15.119:0/1019920379 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) v1 ==== 136717+0+0 (secure 0 0 0) 0x7f7abc0225e0 con 0x7f7ac0068a50 2023-12-07T10:21:53.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.422+0000 7f7adac4db80 1 Processor -- start 2023-12-07T10:21:53.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.422+0000 7f7ab67fc700 1 --2- 172.21.15.119:0/1019920379 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f7aac0a1e20 0x7f7aac0a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:53.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.422+0000 7f7adac4db80 1 -- start start 2023-12-07T10:21:53.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.422+0000 7f7ab67fc700 1 -- 172.21.15.119:0/1019920379 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f7abc0cf230 con 0x7f7ac0068a50 2023-12-07T10:21:53.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.422+0000 7f7adac4db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e549f5b40 0x561e549f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:53.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.422+0000 7f7adac4db80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e549f6460 0x561e549f9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:53.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.423+0000 7f7adac4db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e549fa150 0x561e549fa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:53.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.423+0000 7f7adac4db80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e549697c0 con 0x561e549f6460 2023-12-07T10:21:53.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.423+0000 7f7adac4db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e549507a0 con 0x561e549f5b40 2023-12-07T10:21:53.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.423+0000 7f7adac4db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e5494e0c0 con 0x561e549fa150 2023-12-07T10:21:53.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.423+0000 7f7ac599c700 1 --2- 172.21.15.119:0/1019920379 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f7aac0a1e20 0x7f7aac0a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:53.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.423+0000 7f7ac519b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e549f5b40 0x561e549f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:53.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.423+0000 7f7ac519b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e549f5b40 0x561e549f5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45448/0 (socket says 172.21.15.119:45448) 2023-12-07T10:21:53.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.423+0000 7f7ac519b700 1 -- 172.21.15.119:0/417066344 learned_addr learned my addr 172.21.15.119:0/417066344 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:53.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.423+0000 7f7ac499a700 1 --2- 172.21.15.119:0/417066344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e549f6460 0x561e549f9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:53.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.423+0000 7f7ac519b700 1 -- 172.21.15.119:0/417066344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e549fa150 msgr2=0x561e549fa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:53.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.423+0000 7f7ac519b700 1 --2- 172.21.15.119:0/417066344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e549fa150 0x561e549fa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.423+0000 7f7ac519b700 1 -- 172.21.15.119:0/417066344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e549f6460 msgr2=0x561e549f9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:53.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.423+0000 7f7ac519b700 1 --2- 172.21.15.119:0/417066344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e549f6460 0x561e549f9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.423+0000 7f7ac519b700 1 -- 172.21.15.119:0/417066344 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e547c6630 con 0x561e549f5b40 2023-12-07T10:21:53.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.423+0000 7f7ac599c700 1 --2- 172.21.15.119:0/1019920379 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f7aac0a1e20 0x7f7aac0a42e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009470 tx=0x7f7ab8009920 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:53.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.423+0000 7f7ac519b700 1 --2- 172.21.15.119:0/417066344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e549f5b40 0x561e549f5f20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7abc0984d0 tx=0x7f7abc009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e2ea98a9a4190fb server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:53.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.424+0000 7f7aab7fe700 1 -- 172.21.15.119:0/417066344 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc09c160 con 0x561e549f5b40 2023-12-07T10:21:53.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.424+0000 7f7aab7fe700 1 -- 172.21.15.119:0/417066344 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7abc09ba60 con 0x561e549f5b40 2023-12-07T10:21:53.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.424+0000 7f7aab7fe700 1 -- 172.21.15.119:0/417066344 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc0860f0 con 0x561e549f5b40 2023-12-07T10:21:53.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.424+0000 7f7adac4db80 1 -- 172.21.15.119:0/417066344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e549f5b40 msgr2=0x561e549f5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:53.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.424+0000 7f7adac4db80 1 --2- 172.21.15.119:0/417066344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e549f5b40 0x561e549f5f20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7abc0984d0 tx=0x7f7abc009ca0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.425+0000 7f7adac4db80 1 -- 172.21.15.119:0/417066344 shutdown_connections 2023-12-07T10:21:53.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.425+0000 7f7adac4db80 1 --2- 172.21.15.119:0/417066344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e549fa150 0x561e549fa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.425+0000 7f7adac4db80 1 --2- 172.21.15.119:0/417066344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e549f5b40 0x561e549f5f20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.425+0000 7f7adac4db80 1 --2- 172.21.15.119:0/417066344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e549f6460 0x561e549f9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.425+0000 7f7adac4db80 1 -- 172.21.15.119:0/417066344 >> 172.21.15.119:0/417066344 conn(0x561e54924360 msgr2=0x561e549f0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:53.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.425+0000 7f7adac4db80 1 -- 172.21.15.119:0/417066344 shutdown_connections 2023-12-07T10:21:53.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.425+0000 7f7adac4db80 1 -- 172.21.15.119:0/417066344 wait complete. 2023-12-07T10:21:53.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.425+0000 7f7adac4db80 1 -- start start 2023-12-07T10:21:53.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.425+0000 7f7adac4db80 1 Processor -- start 2023-12-07T10:21:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.426+0000 7f7adac4db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e549f5b40 0x561e54a01860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.426+0000 7f7adac4db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e549f6460 0x561e549febf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.426+0000 7f7adac4db80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e549fa150 0x561e549ff200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.426+0000 7f7adac4db80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e549499b0 con 0x561e549fa150 2023-12-07T10:21:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.426+0000 7f7adac4db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e54903560 con 0x561e549f5b40 2023-12-07T10:21:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.426+0000 7f7adac4db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e54903200 con 0x561e549f6460 2023-12-07T10:21:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.426+0000 7f7adac4db80 10 client.0 _ll_register_callbacks cb 0x561e548ea940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.427+0000 7f7ac599c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e549fa150 0x561e549ff200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.427+0000 7f7ac599c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e549fa150 0x561e549ff200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34498/0 (socket says 172.21.15.119:34498) 2023-12-07T10:21:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.427+0000 7f7ac499a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e549f6460 0x561e549febf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.427+0000 7f7ac599c700 1 -- 172.21.15.119:0/3605134012 learned_addr learned my addr 172.21.15.119:0/3605134012 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.427+0000 7f7ac519b700 1 --2- 172.21.15.119:0/3605134012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e549f5b40 0x561e54a01860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:53.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.427+0000 7f7ac599c700 1 -- 172.21.15.119:0/3605134012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e549f6460 msgr2=0x561e549febf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:53.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.427+0000 7f7ac599c700 1 --2- 172.21.15.119:0/3605134012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e549f6460 0x561e549febf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.427+0000 7f7ac599c700 1 -- 172.21.15.119:0/3605134012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e549f5b40 msgr2=0x561e54a01860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:53.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.427+0000 7f7ac599c700 1 --2- 172.21.15.119:0/3605134012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e549f5b40 0x561e54a01860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.427+0000 7f7ac599c700 1 -- 172.21.15.119:0/3605134012 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac01a69b0 con 0x561e549fa150 2023-12-07T10:21:53.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.427+0000 7f7ac499a700 1 --2- 172.21.15.119:0/3605134012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e549f6460 0x561e549febf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:53.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.427+0000 7f7ac599c700 1 --2- 172.21.15.119:0/3605134012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e549fa150 0x561e549ff200 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8018e10 tx=0x7f7ab8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29036b061ac1af1a server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:53.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.427+0000 7f7aa8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:21:53.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.428+0000 7f7aa8ff9700 1 -- 172.21.15.119:0/3605134012 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab801d070 con 0x561e549fa150 2023-12-07T10:21:53.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.428+0000 7f7adac4db80 1 -- 172.21.15.119:0/3605134012 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e549e8b00 con 0x561e549fa150 2023-12-07T10:21:53.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.428+0000 7f7aa8ff9700 1 -- 172.21.15.119:0/3605134012 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f7ab8018e40 con 0x561e549fa150 2023-12-07T10:21:53.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.428+0000 7f7aa8ff9700 1 -- 172.21.15.119:0/3605134012 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ab800e050 con 0x561e549fa150 2023-12-07T10:21:53.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.428+0000 7f7adac4db80 10 client.16098 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:53.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.428+0000 7f7adac4db80 1 -- 172.21.15.119:0/3605134012 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561e54772de0 con 0x561e549fa150 2023-12-07T10:21:53.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.428+0000 7f7aa8ff9700 1 -- 172.21.15.119:0/3605134012 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561e549e8b00 con 0x561e549fa150 2023-12-07T10:21:53.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.429+0000 7f7aa8ff9700 1 -- 172.21.15.119:0/3605134012 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7ab8021c90 con 0x561e549fa150 2023-12-07T10:21:53.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.429+0000 7f7adac4db80 10 client.16098 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:53.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.429+0000 7f7adac4db80 10 client.16098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:21:53.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.429+0000 7f7adac4db80 20 client.16098 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:53.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.429+0000 7f7adac4db80 1 --2- 172.21.15.119:0/3605134012 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x561e54a0eff0 0x561e54a11410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:53.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.429+0000 7f7adac4db80 4 client.16098 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:21:53.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.429+0000 7f7adac4db80 1 -- 172.21.15.119:0/3605134012 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e549c47a0 con 0x561e54a0eff0 2023-12-07T10:21:53.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.429+0000 7f7ac519b700 1 --2- 172.21.15.119:0/3605134012 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x561e54a0eff0 0x561e54a11410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:53.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.430+0000 7f7ac519b700 1 --2- 172.21.15.119:0/3605134012 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x561e54a0eff0 0x561e54a11410 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1d23a3b3a946dd server_cookie=9de57933915d128a in_seq=0 out_seq=0 2023-12-07T10:21:53.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.430+0000 7f7aa8ff9700 10 client.16098 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:53.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.430+0000 7f7aa8ff9700 1 -- 172.21.15.119:0/3605134012 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561e549c47a0 con 0x561e54a0eff0 2023-12-07T10:21:53.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.430+0000 7f7aa8ff9700 10 client.16098 handle_command_reply: tid=0 2023-12-07T10:21:53.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.430+0000 7f7adac4db80 1 client.16098 shutdown 2023-12-07T10:21:53.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.431+0000 7f7adac4db80 1 -- 172.21.15.119:0/3605134012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e549fa150 msgr2=0x561e549ff200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:53.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.431+0000 7f7adac4db80 1 --2- 172.21.15.119:0/3605134012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e549fa150 0x561e549ff200 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8018e10 tx=0x7f7ab8004b40 comp rx=0 tx=0).stop 2023-12-07T10:21:53.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.431+0000 7f7adac4db80 1 -- 172.21.15.119:0/3605134012 shutdown_connections 2023-12-07T10:21:53.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.431+0000 7f7adac4db80 1 --2- 172.21.15.119:0/3605134012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e549fa150 0x561e549ff200 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.431+0000 7f7adac4db80 1 --2- 172.21.15.119:0/3605134012 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x561e54a0eff0 0x561e54a11410 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.431+0000 7f7adac4db80 1 --2- 172.21.15.119:0/3605134012 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e549f5b40 0x561e54a01860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.431+0000 7f7adac4db80 1 --2- 172.21.15.119:0/3605134012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e549f6460 0x561e549febf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.431+0000 7f7adac4db80 1 -- 172.21.15.119:0/3605134012 >> 172.21.15.119:0/3605134012 conn(0x561e54924360 msgr2=0x561e549ead20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:53.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.431+0000 7f7aa8ff9700 0 client.16098 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:53.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.431+0000 7f7adac4db80 1 -- 172.21.15.119:0/3605134012 shutdown_connections 2023-12-07T10:21:53.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.431+0000 7f7adac4db80 1 -- 172.21.15.119:0/3605134012 wait complete. 2023-12-07T10:21:53.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.431+0000 7f7adac4db80 20 client.16098 trim_cache size 0 max 16384 2023-12-07T10:21:53.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.456+0000 7f7adac4db80 1 Processor -- start 2023-12-07T10:21:53.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.456+0000 7f7adac4db80 1 -- start start 2023-12-07T10:21:53.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.456+0000 7f7adac4db80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e549f5b40 0x561e549ff140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:53.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.456+0000 7f7adac4db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e549f6460 0x561e54a06000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:53.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.456+0000 7f7adac4db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e549fa150 0x561e54a06540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:53.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.456+0000 7f7adac4db80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e549faa50 con 0x561e549f5b40 2023-12-07T10:21:53.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.456+0000 7f7adac4db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e547c8920 con 0x561e549f6460 2023-12-07T10:21:53.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.456+0000 7f7adac4db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e549e8ae0 con 0x561e549fa150 2023-12-07T10:21:53.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.457+0000 7f7ac519b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e549f5b40 0x561e549ff140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:53.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.457+0000 7f7ac499a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e549f6460 0x561e54a06000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:53.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.457+0000 7f7ac599c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e549fa150 0x561e54a06540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:53.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.457+0000 7f7ac499a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e549f6460 0x561e54a06000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45470/0 (socket says 172.21.15.119:45470) 2023-12-07T10:21:53.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.457+0000 7f7ac499a700 1 -- 172.21.15.119:0/2442718771 learned_addr learned my addr 172.21.15.119:0/2442718771 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:53.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.457+0000 7f7ac599c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e549fa150 0x561e54a06540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:39846/0 (socket says 172.21.15.119:39846) 2023-12-07T10:21:53.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.457+0000 7f7ac599c700 1 -- 172.21.15.119:0/2442718771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e549f6460 msgr2=0x561e54a06000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:53.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.457+0000 7f7ac599c700 1 --2- 172.21.15.119:0/2442718771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e549f6460 0x561e54a06000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.457+0000 7f7ac599c700 1 -- 172.21.15.119:0/2442718771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e549f5b40 msgr2=0x561e549ff140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:53.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.457+0000 7f7ac599c700 1 --2- 172.21.15.119:0/2442718771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e549f5b40 0x561e549ff140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.457+0000 7f7ac599c700 1 -- 172.21.15.119:0/2442718771 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e549ea040 con 0x561e549fa150 2023-12-07T10:21:53.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.457+0000 7f7ac499a700 1 --2- 172.21.15.119:0/2442718771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e549f6460 0x561e54a06000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:53.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.457+0000 7f7ac599c700 1 --2- 172.21.15.119:0/2442718771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e549fa150 0x561e54a06540 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8018e10 tx=0x7f7ab8007f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3421f2bc7108a11 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:53.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.457+0000 7f7aabfff700 1 -- 172.21.15.119:0/2442718771 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab801d070 con 0x561e549fa150 2023-12-07T10:21:53.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.457+0000 7f7aabfff700 1 -- 172.21.15.119:0/2442718771 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ab800f040 con 0x561e549fa150 2023-12-07T10:21:53.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.457+0000 7f7aabfff700 1 -- 172.21.15.119:0/2442718771 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab800e040 con 0x561e549fa150 2023-12-07T10:21:53.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.458+0000 7f7adac4db80 1 -- 172.21.15.119:0/2442718771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e549fa150 msgr2=0x561e54a06540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:53.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.458+0000 7f7adac4db80 1 --2- 172.21.15.119:0/2442718771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e549fa150 0x561e54a06540 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8018e10 tx=0x7f7ab8007f80 comp rx=0 tx=0).stop 2023-12-07T10:21:53.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.458+0000 7f7adac4db80 1 -- 172.21.15.119:0/2442718771 shutdown_connections 2023-12-07T10:21:53.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.458+0000 7f7adac4db80 1 --2- 172.21.15.119:0/2442718771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e549fa150 0x561e54a06540 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.458+0000 7f7adac4db80 1 --2- 172.21.15.119:0/2442718771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e549f5b40 0x561e549ff140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.458+0000 7f7adac4db80 1 --2- 172.21.15.119:0/2442718771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e549f6460 0x561e54a06000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.458+0000 7f7adac4db80 1 -- 172.21.15.119:0/2442718771 >> 172.21.15.119:0/2442718771 conn(0x561e54924360 msgr2=0x561e54a01520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:53.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.458+0000 7f7adac4db80 1 -- 172.21.15.119:0/2442718771 shutdown_connections 2023-12-07T10:21:53.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.458+0000 7f7adac4db80 1 -- 172.21.15.119:0/2442718771 wait complete. 2023-12-07T10:21:53.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.459+0000 7f7adac4db80 1 -- start start 2023-12-07T10:21:53.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.459+0000 7f7adac4db80 1 Processor -- start 2023-12-07T10:21:53.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.459+0000 7f7adac4db80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e549f5b40 0x561e54a064f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:53.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.459+0000 7f7adac4db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e549f6460 0x561e549fc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:53.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.459+0000 7f7adac4db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e549fa150 0x561e549fc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:53.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.459+0000 7f7adac4db80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e549faf80 con 0x561e549f5b40 2023-12-07T10:21:53.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.459+0000 7f7adac4db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e549fb0f0 con 0x561e549fa150 2023-12-07T10:21:53.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.459+0000 7f7adac4db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e549fb2b0 con 0x561e549f6460 2023-12-07T10:21:53.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.459+0000 7f7adac4db80 10 client.0 _ll_register_callbacks cb 0x561e548ea940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:53.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.460+0000 7f7ac499a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e549f6460 0x561e549fc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:53.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.460+0000 7f7ac599c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e549fa150 0x561e549fc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:53.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.460+0000 7f7ac599c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e549fa150 0x561e549fc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45482/0 (socket says 172.21.15.119:45482) 2023-12-07T10:21:53.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.460+0000 7f7ac599c700 1 -- 172.21.15.119:0/1463836620 learned_addr learned my addr 172.21.15.119:0/1463836620 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:53.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.460+0000 7f7ac519b700 1 --2- 172.21.15.119:0/1463836620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e549f5b40 0x561e54a064f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:53.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.460+0000 7f7ac599c700 1 -- 172.21.15.119:0/1463836620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e549f6460 msgr2=0x561e549fc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:53.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.460+0000 7f7ac599c700 1 --2- 172.21.15.119:0/1463836620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e549f6460 0x561e549fc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.460+0000 7f7ac599c700 1 -- 172.21.15.119:0/1463836620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e549f5b40 msgr2=0x561e54a064f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:53.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.460+0000 7f7ac599c700 1 --2- 172.21.15.119:0/1463836620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e549f5b40 0x561e54a064f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.460+0000 7f7ac599c700 1 -- 172.21.15.119:0/1463836620 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab0007ab0 con 0x561e549fa150 2023-12-07T10:21:53.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.460+0000 7f7ac499a700 1 --2- 172.21.15.119:0/1463836620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e549f6460 0x561e549fc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:21:53.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.460+0000 7f7ac519b700 1 --2- 172.21.15.119:0/1463836620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e549f5b40 0x561e54a064f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:53.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.460+0000 7f7ac599c700 1 --2- 172.21.15.119:0/1463836620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e549fa150 0x561e549fc7b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8015470 tx=0x7f7ab8007ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65f275d98a660e30 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:53.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.460+0000 7f7aa8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:21:53.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.461+0000 7f7aa8ff9700 1 -- 172.21.15.119:0/1463836620 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab801d030 con 0x561e549fa150 2023-12-07T10:21:53.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.461+0000 7f7adac4db80 1 -- 172.21.15.119:0/1463836620 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7abc09ba60 con 0x561e549fa150 2023-12-07T10:21:53.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.461+0000 7f7aa8ff9700 1 -- 172.21.15.119:0/1463836620 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f7ab801e070 con 0x561e549fa150 2023-12-07T10:21:53.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.461+0000 7f7aa8ff9700 1 -- 172.21.15.119:0/1463836620 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ab8004b20 con 0x561e549fa150 2023-12-07T10:21:53.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.461+0000 7f7adac4db80 10 client.26108 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:53.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.461+0000 7f7adac4db80 1 -- 172.21.15.119:0/1463836620 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561e54772de0 con 0x561e549fa150 2023-12-07T10:21:53.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.462+0000 7f7aa8ff9700 1 -- 172.21.15.119:0/1463836620 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7abc09ba60 con 0x561e549fa150 2023-12-07T10:21:53.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.462+0000 7f7aa8ff9700 1 -- 172.21.15.119:0/1463836620 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7ab8029040 con 0x561e549fa150 2023-12-07T10:21:53.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.462+0000 7f7adac4db80 10 client.26108 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:53.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.462+0000 7f7adac4db80 10 client.26108 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:21:53.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.462+0000 7f7adac4db80 20 client.26108 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:53.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.462+0000 7f7adac4db80 1 --2- 172.21.15.119:0/1463836620 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x561e54a65cb0 0x561e54a680d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:53.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.462+0000 7f7adac4db80 4 client.26108 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-12-07T10:21:53.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.462+0000 7f7adac4db80 1 -- 172.21.15.119:0/1463836620 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561e5490ffd0 con 0x561e54a65cb0 2023-12-07T10:21:53.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.462+0000 7f7ac519b700 1 --2- 172.21.15.119:0/1463836620 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x561e54a65cb0 0x561e54a680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:53.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.463+0000 7f7aa8ff9700 10 client.26108 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:53.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.463+0000 7f7ac519b700 1 --2- 172.21.15.119:0/1463836620 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x561e54a65cb0 0x561e54a680d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9601a8443605d9 server_cookie=32f038402639ba0c in_seq=0 out_seq=0 2023-12-07T10:21:53.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.463+0000 7f7aa8ff9700 1 -- 172.21.15.119:0/1463836620 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561e5490ffd0 con 0x561e54a65cb0 2023-12-07T10:21:53.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.463+0000 7f7aa8ff9700 10 client.26108 handle_command_reply: tid=0 2023-12-07T10:21:53.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.463+0000 7f7adac4db80 1 client.26108 shutdown 2023-12-07T10:21:53.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.463+0000 7f7adac4db80 1 -- 172.21.15.119:0/1463836620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e549fa150 msgr2=0x561e549fc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:53.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.463+0000 7f7adac4db80 1 --2- 172.21.15.119:0/1463836620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e549fa150 0x561e549fc7b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8015470 tx=0x7f7ab8007ee0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.463+0000 7f7adac4db80 1 -- 172.21.15.119:0/1463836620 shutdown_connections 2023-12-07T10:21:53.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.463+0000 7f7adac4db80 1 --2- 172.21.15.119:0/1463836620 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x561e54a65cb0 0x561e54a680d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.463+0000 7f7adac4db80 1 --2- 172.21.15.119:0/1463836620 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561e549fa150 0x561e549fc7b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.463+0000 7f7adac4db80 1 --2- 172.21.15.119:0/1463836620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e549f5b40 0x561e54a064f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.463+0000 7f7aa8ff9700 0 client.26108 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:53.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.463+0000 7f7adac4db80 1 --2- 172.21.15.119:0/1463836620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e549f6460 0x561e549fc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.463+0000 7f7adac4db80 1 -- 172.21.15.119:0/1463836620 >> 172.21.15.119:0/1463836620 conn(0x561e54924360 msgr2=0x561e54a0aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:53.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.463+0000 7f7adac4db80 1 -- 172.21.15.119:0/1463836620 shutdown_connections 2023-12-07T10:21:53.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.463+0000 7f7adac4db80 1 -- 172.21.15.119:0/1463836620 wait complete. 2023-12-07T10:21:53.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.464+0000 7f7adac4db80 20 client.26108 trim_cache size 0 max 16384 2023-12-07T10:21:53.474 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:21:53.474 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "no active scrubs running", 2023-12-07T10:21:53.474 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": {} 2023-12-07T10:21:53.475 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:21:53.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.464+0000 7f7ac73ff700 1 -- 172.21.15.119:0/1019920379 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f7aac0a1e20 msgr2=0x7f7aac0a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:53.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.464+0000 7f7ac73ff700 1 --2- 172.21.15.119:0/1019920379 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f7aac0a1e20 0x7f7aac0a42e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009470 tx=0x7f7ab8009920 comp rx=0 tx=0).stop 2023-12-07T10:21:53.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.464+0000 7f7ac73ff700 1 -- 172.21.15.119:0/1019920379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ac0068a50 msgr2=0x7f7ac0199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:53.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.464+0000 7f7ac73ff700 1 --2- 172.21.15.119:0/1019920379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ac0068a50 0x7f7ac0199db0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7abc000c00 tx=0x7f7abc005740 comp rx=0 tx=0).stop 2023-12-07T10:21:53.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.464+0000 7f7ac73ff700 1 -- 172.21.15.119:0/1019920379 shutdown_connections 2023-12-07T10:21:53.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.464+0000 7f7ac73ff700 1 --2- 172.21.15.119:0/1019920379 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f7aac0a1e20 0x7f7aac0a42e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.464+0000 7f7ac73ff700 1 --2- 172.21.15.119:0/1019920379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ac0106ad0 0x7f7ac019e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.464+0000 7f7ac73ff700 1 --2- 172.21.15.119:0/1019920379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ac0068a50 0x7f7ac0199db0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.464+0000 7f7ac73ff700 1 --2- 172.21.15.119:0/1019920379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ac0106110 0x7f7ac019a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.464+0000 7f7ac73ff700 1 -- 172.21.15.119:0/1019920379 >> 172.21.15.119:0/1019920379 conn(0x7f7ac0077610 msgr2=0x7f7ac00fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:53.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.465+0000 7f7ac73ff700 1 -- 172.21.15.119:0/1019920379 shutdown_connections 2023-12-07T10:21:53.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.465+0000 7f7ac73ff700 1 -- 172.21.15.119:0/1019920379 wait complete. 2023-12-07T10:21:53.487 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-07T10:21:53.487 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:21:53.538 INFO:tasks.workunit.client.0.smithi119.stdout: 1 17587 18.66 MB/sec warmup 6 sec latency 27.643 ms 2023-12-07T10:21:53.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.717+0000 7f5286dff700 1 -- 172.21.15.119:0/1154140781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5280107560 msgr2=0x7f5280107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:53.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.717+0000 7f5286dff700 1 --2- 172.21.15.119:0/1154140781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5280107560 0x7f5280107940 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5274009990 tx=0x7f5274009ca0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.717+0000 7f5286dff700 1 -- 172.21.15.119:0/1154140781 shutdown_connections 2023-12-07T10:21:53.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.717+0000 7f5286dff700 1 --2- 172.21.15.119:0/1154140781 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f528006a4f0 0x7f528010e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.717+0000 7f5286dff700 1 --2- 172.21.15.119:0/1154140781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5280107560 0x7f5280107940 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.717+0000 7f5286dff700 1 --2- 172.21.15.119:0/1154140781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5280069b30 0x7f5280069fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.717+0000 7f5286dff700 1 -- 172.21.15.119:0/1154140781 >> 172.21.15.119:0/1154140781 conn(0x7f5280077530 msgr2=0x7f5280077940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:53.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.717+0000 7f5286dff700 1 -- 172.21.15.119:0/1154140781 shutdown_connections 2023-12-07T10:21:53.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.717+0000 7f5286dff700 1 -- 172.21.15.119:0/1154140781 wait complete. 2023-12-07T10:21:53.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.719+0000 7f5286dff700 1 Processor -- start 2023-12-07T10:21:53.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.719+0000 7f5286dff700 1 -- start start 2023-12-07T10:21:53.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.719+0000 7f5286dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5280069b30 0x7f528019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:53.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.719+0000 7f5286dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f528006a4f0 0x7f528019fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:53.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.719+0000 7f5286dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5280107560 0x7f5280199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:53.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.719+0000 7f5286dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5280199fe0 con 0x7f528006a4f0 2023-12-07T10:21:53.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.719+0000 7f5286dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528019a150 con 0x7f5280069b30 2023-12-07T10:21:53.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.719+0000 7f5286dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528019a310 con 0x7f5280107560 2023-12-07T10:21:53.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.720+0000 7f527ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f528006a4f0 0x7f528019fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:53.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.720+0000 7f528539c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5280107560 0x7f5280199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:53.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.720+0000 7f527ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f528006a4f0 0x7f528019fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34530/0 (socket says 172.21.15.119:34530) 2023-12-07T10:21:53.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.720+0000 7f528539c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5280107560 0x7f5280199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:39870/0 (socket says 172.21.15.119:39870) 2023-12-07T10:21:53.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.720+0000 7f527ffff700 1 -- 172.21.15.119:0/1997861941 learned_addr learned my addr 172.21.15.119:0/1997861941 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:53.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.720+0000 7f5284b9b700 1 --2- 172.21.15.119:0/1997861941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5280069b30 0x7f528019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:53.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.720+0000 7f528539c700 1 -- 172.21.15.119:0/1997861941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5280069b30 msgr2=0x7f528019f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:53.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.720+0000 7f528539c700 1 --2- 172.21.15.119:0/1997861941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5280069b30 0x7f528019f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.720+0000 7f528539c700 1 -- 172.21.15.119:0/1997861941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f528006a4f0 msgr2=0x7f528019fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:53.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.720+0000 7f528539c700 1 --2- 172.21.15.119:0/1997861941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f528006a4f0 0x7f528019fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:53.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.720+0000 7f528539c700 1 -- 172.21.15.119:0/1997861941 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526c009520 con 0x7f5280107560 2023-12-07T10:21:53.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.720+0000 7f527ffff700 1 --2- 172.21.15.119:0/1997861941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f528006a4f0 0x7f528019fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:53.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.720+0000 7f528539c700 1 --2- 172.21.15.119:0/1997861941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5280107560 0x7f5280199980 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f527800efc0 tx=0x7f527800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:53.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.721+0000 7f527dffb700 1 -- 172.21.15.119:0/1997861941 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5278018070 con 0x7f5280107560 2023-12-07T10:21:53.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.721+0000 7f527dffb700 1 -- 172.21.15.119:0/1997861941 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f527800f040 con 0x7f5280107560 2023-12-07T10:21:53.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.721+0000 7f527dffb700 1 -- 172.21.15.119:0/1997861941 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527800cd40 con 0x7f5280107560 2023-12-07T10:21:53.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.721+0000 7f5286dff700 1 -- 172.21.15.119:0/1997861941 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52740095f0 con 0x7f5280107560 2023-12-07T10:21:53.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.721+0000 7f5286dff700 1 -- 172.21.15.119:0/1997861941 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52801a6660 con 0x7f5280107560 2023-12-07T10:21:53.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.722+0000 7f5286dff700 1 -- 172.21.15.119:0/1997861941 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5280102c10 con 0x7f5280107560 2023-12-07T10:21:53.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.724+0000 7f527dffb700 1 -- 172.21.15.119:0/1997861941 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) v1 ==== 136717+0+0 (secure 0 0 0) 0x7f5278003a00 con 0x7f5280107560 2023-12-07T10:21:53.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.725+0000 7f527dffb700 1 --2- 172.21.15.119:0/1997861941 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f52700a1e20 0x7f52700a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:53.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.725+0000 7f527dffb700 1 -- 172.21.15.119:0/1997861941 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f52780cdac0 con 0x7f5280107560 2023-12-07T10:21:53.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.725+0000 7f5284b9b700 1 --2- 172.21.15.119:0/1997861941 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f52700a1e20 0x7f52700a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:53.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.726+0000 7f5284b9b700 1 --2- 172.21.15.119:0/1997861941 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f52700a1e20 0x7f52700a42e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5274006010 tx=0x7f527400b260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:53.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:53.730+0000 7f527dffb700 1 -- 172.21.15.119:0/1997861941 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5278096630 con 0x7f5280107560 2023-12-07T10:21:54.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.019+0000 7f5286dff700 1 -- 172.21.15.119:0/1997861941 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52800687a0 con 0x7f5280107560 2023-12-07T10:21:54.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.020+0000 7f527dffb700 1 -- 172.21.15.119:0/1997861941 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11384 (secure 0 0 0) 0x7f5278099c50 con 0x7f5280107560 2023-12-07T10:21:54.022 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:21:54.022 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-07T10:17:41.278311+0000","modified":"2023-12-07T10:20:33.545611+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24517,"mds_3":24505,"mds_4":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14535":{"gid":14535,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6841/1791998028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1791998028},{"type":"v1","addr":"172.21.15.119:6841","nonce":1791998028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24505":{"gid":24505,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/2925503246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2925503246},{"type":"v1","addr":"172.21.15.153:6837","nonce":2925503246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6839/3984962900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3984962900},{"type":"v1","addr":"172.21.15.159:6839","nonce":3984962900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.119:6839/1920762602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1920762602},{"type":"v1","addr":"172.21.15.119:6839","nonce":1920762602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/566825184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":566825184},{"type":"v1","addr":"172.21.15.153:6839","nonce":566825184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-07T10:21:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.023+0000 7f5286dff700 1 -- 172.21.15.119:0/1997861941 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f52700a1e20 msgr2=0x7f52700a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.023+0000 7f5286dff700 1 --2- 172.21.15.119:0/1997861941 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f52700a1e20 0x7f52700a42e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5274006010 tx=0x7f527400b260 comp rx=0 tx=0).stop 2023-12-07T10:21:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.023+0000 7f5286dff700 1 -- 172.21.15.119:0/1997861941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5280107560 msgr2=0x7f5280199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.023+0000 7f5286dff700 1 --2- 172.21.15.119:0/1997861941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5280107560 0x7f5280199980 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f527800efc0 tx=0x7f527800c3a0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.023+0000 7f5286dff700 1 -- 172.21.15.119:0/1997861941 shutdown_connections 2023-12-07T10:21:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.023+0000 7f5286dff700 1 --2- 172.21.15.119:0/1997861941 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f52700a1e20 0x7f52700a42e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.023+0000 7f5286dff700 1 --2- 172.21.15.119:0/1997861941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5280107560 0x7f5280199980 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.023+0000 7f5286dff700 1 --2- 172.21.15.119:0/1997861941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5280069b30 0x7f528019f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.023+0000 7f5286dff700 1 --2- 172.21.15.119:0/1997861941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f528006a4f0 0x7f528019fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.023+0000 7f5286dff700 1 -- 172.21.15.119:0/1997861941 >> 172.21.15.119:0/1997861941 conn(0x7f5280077530 msgr2=0x7f52800fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.023+0000 7f5286dff700 1 -- 172.21.15.119:0/1997861941 shutdown_connections 2023-12-07T10:21:54.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.023+0000 7f5286dff700 1 -- 172.21.15.119:0/1997861941 wait complete. 2023-12-07T10:21:54.026 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 30 2023-12-07T10:21:54.046 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-12-07T10:21:54.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:53 smithi119 ceph-mon[97908]: pgmap v47: 129 pgs: 129 active+clean; 6.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 505 KiB/s wr, 67 op/s 2023-12-07T10:21:54.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:53 smithi159 ceph-mon[134726]: pgmap v47: 129 pgs: 129 active+clean; 6.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 505 KiB/s wr, 67 op/s 2023-12-07T10:21:54.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.276+0000 7f9519bff700 1 -- 172.21.15.119:0/3120331741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9514068a50 msgr2=0x7f9514068e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.276+0000 7f9519bff700 1 --2- 172.21.15.119:0/3120331741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9514068a50 0x7f9514068e30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9504009990 tx=0x7f9504009ca0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.276+0000 7f9519bff700 1 -- 172.21.15.119:0/3120331741 shutdown_connections 2023-12-07T10:21:54.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.276+0000 7f9519bff700 1 --2- 172.21.15.119:0/3120331741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9514106ad0 0x7f951410e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.276+0000 7f9519bff700 1 --2- 172.21.15.119:0/3120331741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9514068a50 0x7f9514068e30 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.276+0000 7f9519bff700 1 --2- 172.21.15.119:0/3120331741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9514106110 0x7f9514106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.276+0000 7f9519bff700 1 -- 172.21.15.119:0/3120331741 >> 172.21.15.119:0/3120331741 conn(0x7f9514077610 msgr2=0x7f9514077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:54.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.276+0000 7f9519bff700 1 -- 172.21.15.119:0/3120331741 shutdown_connections 2023-12-07T10:21:54.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.276+0000 7f9519bff700 1 -- 172.21.15.119:0/3120331741 wait complete. 2023-12-07T10:21:54.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.277+0000 7f9519bff700 1 Processor -- start 2023-12-07T10:21:54.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.277+0000 7f9519bff700 1 -- start start 2023-12-07T10:21:54.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.277+0000 7f9519bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9514068a50 0x7f951419f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.277+0000 7f9519bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9514106110 0x7f951419fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.277+0000 7f9519bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9514106ad0 0x7f9514199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.277+0000 7f9519bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951419a0e0 con 0x7f9514106ad0 2023-12-07T10:21:54.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.277+0000 7f9519bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951419a250 con 0x7f9514106110 2023-12-07T10:21:54.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.277+0000 7f9519bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951419a410 con 0x7f9514068a50 2023-12-07T10:21:54.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:53 smithi153 ceph-mon[133625]: pgmap v47: 129 pgs: 129 active+clean; 6.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 505 KiB/s wr, 67 op/s 2023-12-07T10:21:54.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.278+0000 7f95137fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9514068a50 0x7f951419f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.278+0000 7f95137fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9514068a50 0x7f951419f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:39894/0 (socket says 172.21.15.119:39894) 2023-12-07T10:21:54.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.278+0000 7f95137fe700 1 -- 172.21.15.119:0/2677650132 learned_addr learned my addr 172.21.15.119:0/2677650132 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:54.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.278+0000 7f9513fff700 1 --2- 172.21.15.119:0/2677650132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9514106ad0 0x7f9514199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.278+0000 7f950bfff700 1 --2- 172.21.15.119:0/2677650132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9514106110 0x7f951419fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.278+0000 7f95137fe700 1 -- 172.21.15.119:0/2677650132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9514106110 msgr2=0x7f951419fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.278+0000 7f95137fe700 1 --2- 172.21.15.119:0/2677650132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9514106110 0x7f951419fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.278+0000 7f95137fe700 1 -- 172.21.15.119:0/2677650132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9514106ad0 msgr2=0x7f9514199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.278+0000 7f95137fe700 1 --2- 172.21.15.119:0/2677650132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9514106ad0 0x7f9514199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.278+0000 7f95137fe700 1 -- 172.21.15.119:0/2677650132 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95040095f0 con 0x7f9514068a50 2023-12-07T10:21:54.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.278+0000 7f95137fe700 1 --2- 172.21.15.119:0/2677650132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9514068a50 0x7f951419f950 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9504005910 tx=0x7f9504004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:54.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.279+0000 7f95117fa700 1 -- 172.21.15.119:0/2677650132 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950401d070 con 0x7f9514068a50 2023-12-07T10:21:54.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.279+0000 7f95117fa700 1 -- 172.21.15.119:0/2677650132 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9504022470 con 0x7f9514068a50 2023-12-07T10:21:54.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.279+0000 7f9519bff700 1 -- 172.21.15.119:0/2677650132 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951419a690 con 0x7f9514068a50 2023-12-07T10:21:54.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.279+0000 7f95117fa700 1 -- 172.21.15.119:0/2677650132 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9504017650 con 0x7f9514068a50 2023-12-07T10:21:54.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.279+0000 7f9519bff700 1 -- 172.21.15.119:0/2677650132 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95141a6940 con 0x7f9514068a50 2023-12-07T10:21:54.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.283+0000 7f952d4c7b80 1 Processor -- start 2023-12-07T10:21:54.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.283+0000 7f952d4c7b80 1 -- start start 2023-12-07T10:21:54.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.284+0000 7f95117fa700 1 -- 172.21.15.119:0/2677650132 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) v1 ==== 136717+0+0 (secure 0 0 0) 0x7f95040225e0 con 0x7f9514068a50 2023-12-07T10:21:54.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.284+0000 7f952d4c7b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621f4873350 0x5621f4873730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.284+0000 7f952d4c7b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621f4873c70 0x5621f4877420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.284+0000 7f952d4c7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621f4877960 0x5621f4877d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.284+0000 7f952d4c7b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621f47de7c0 con 0x5621f4873c70 2023-12-07T10:21:54.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.284+0000 7f952d4c7b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621f47c57a0 con 0x5621f4873350 2023-12-07T10:21:54.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.284+0000 7f952d4c7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621f47c30c0 con 0x5621f4877960 2023-12-07T10:21:54.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.284+0000 7f9513fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621f4873350 0x5621f4873730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.284+0000 7f950bfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621f4877960 0x5621f4877d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.284+0000 7f9513fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621f4873350 0x5621f4873730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45532/0 (socket says 172.21.15.119:45532) 2023-12-07T10:21:54.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.284+0000 7f9513fff700 1 -- 172.21.15.119:0/2085820157 learned_addr learned my addr 172.21.15.119:0/2085820157 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:54.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.285+0000 7f95137fe700 1 --2- 172.21.15.119:0/2085820157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621f4873c70 0x5621f4877420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.285+0000 7f950bfff700 1 -- 172.21.15.119:0/2085820157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621f4873350 msgr2=0x5621f4873730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:54.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.285+0000 7f950bfff700 1 --2- 172.21.15.119:0/2085820157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621f4873350 0x5621f4873730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.285+0000 7f950bfff700 1 -- 172.21.15.119:0/2085820157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621f4873c70 msgr2=0x5621f4877420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:54.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.285+0000 7f950bfff700 1 --2- 172.21.15.119:0/2085820157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621f4873c70 0x5621f4877420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.285+0000 7f950bfff700 1 -- 172.21.15.119:0/2085820157 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621f463b630 con 0x5621f4877960 2023-12-07T10:21:54.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.285+0000 7f95137fe700 1 --2- 172.21.15.119:0/2085820157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621f4873c70 0x5621f4877420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:21:54.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.285+0000 7f950bfff700 1 --2- 172.21.15.119:0/2085820157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621f4877960 0x5621f4877d80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f94fc003f10 tx=0x7f94fc009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89df8c21bf55ff26 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:54.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.286+0000 7f9509ffb700 1 -- 172.21.15.119:0/2085820157 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94fc019070 con 0x5621f4877960 2023-12-07T10:21:54.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.286+0000 7f9509ffb700 1 -- 172.21.15.119:0/2085820157 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94fc00f040 con 0x5621f4877960 2023-12-07T10:21:54.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.286+0000 7f9509ffb700 1 -- 172.21.15.119:0/2085820157 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94fc007ae0 con 0x5621f4877960 2023-12-07T10:21:54.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.286+0000 7f95117fa700 1 --2- 172.21.15.119:0/2677650132 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f95000a1da0 0x7f95000a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.286+0000 7f952d4c7b80 1 -- 172.21.15.119:0/2085820157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621f4877960 msgr2=0x5621f4877d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.286+0000 7f952d4c7b80 1 --2- 172.21.15.119:0/2085820157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621f4877960 0x5621f4877d80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f94fc003f10 tx=0x7f94fc009c20 comp rx=0 tx=0).stop 2023-12-07T10:21:54.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.287+0000 7f95117fa700 1 -- 172.21.15.119:0/2677650132 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f95040cf2d0 con 0x7f9514068a50 2023-12-07T10:21:54.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.287+0000 7f952d4c7b80 1 -- 172.21.15.119:0/2085820157 shutdown_connections 2023-12-07T10:21:54.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.287+0000 7f952d4c7b80 1 --2- 172.21.15.119:0/2085820157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621f4877960 0x5621f4877d80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.287+0000 7f952d4c7b80 1 --2- 172.21.15.119:0/2085820157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621f4873350 0x5621f4873730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.287+0000 7f9513fff700 1 --2- 172.21.15.119:0/2677650132 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f95000a1da0 0x7f95000a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.287+0000 7f952d4c7b80 1 --2- 172.21.15.119:0/2085820157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621f4873c70 0x5621f4877420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.287+0000 7f952d4c7b80 1 -- 172.21.15.119:0/2085820157 >> 172.21.15.119:0/2085820157 conn(0x5621f4799360 msgr2=0x5621f4865130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:54.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.287+0000 7f952d4c7b80 1 -- 172.21.15.119:0/2085820157 shutdown_connections 2023-12-07T10:21:54.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.287+0000 7f952d4c7b80 1 -- 172.21.15.119:0/2085820157 wait complete. 2023-12-07T10:21:54.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.287+0000 7f9513fff700 1 --2- 172.21.15.119:0/2677650132 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f95000a1da0 0x7f95000a4260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f950c009f00 tx=0x7f950c009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:54.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.288+0000 7f952d4c7b80 1 -- start start 2023-12-07T10:21:54.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.288+0000 7f952d4c7b80 1 Processor -- start 2023-12-07T10:21:54.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.288+0000 7f952d4c7b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621f4873350 0x5621f487f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.289+0000 7f952d4c7b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621f4873c70 0x5621f487c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.289+0000 7f952d4c7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621f4877960 0x5621f487c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.289+0000 7f952d4c7b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621f47be9b0 con 0x5621f4873c70 2023-12-07T10:21:54.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.289+0000 7f952d4c7b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621f4778560 con 0x5621f4873350 2023-12-07T10:21:54.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.289+0000 7f952d4c7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621f4778200 con 0x5621f4877960 2023-12-07T10:21:54.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.289+0000 7f952d4c7b80 10 client.0 _ll_register_callbacks cb 0x5621f475f940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:54.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.289+0000 7f95137fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621f4873350 0x5621f487f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.289+0000 7f950bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621f4873c70 0x5621f487c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.289+0000 7f95137fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621f4873350 0x5621f487f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45540/0 (socket says 172.21.15.119:45540) 2023-12-07T10:21:54.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.289+0000 7f950bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621f4873c70 0x5621f487c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34558/0 (socket says 172.21.15.119:34558) 2023-12-07T10:21:54.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.289+0000 7f95137fe700 1 -- 172.21.15.119:0/461336546 learned_addr learned my addr 172.21.15.119:0/461336546 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:54.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.289+0000 7f9513fff700 1 --2- 172.21.15.119:0/461336546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621f4877960 0x5621f487c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.289+0000 7f950bfff700 1 -- 172.21.15.119:0/461336546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621f4877960 msgr2=0x5621f487c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:54.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.289+0000 7f950bfff700 1 --2- 172.21.15.119:0/461336546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621f4877960 0x5621f487c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.289+0000 7f950bfff700 1 -- 172.21.15.119:0/461336546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621f4873350 msgr2=0x5621f487f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:54.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.289+0000 7f950bfff700 1 --2- 172.21.15.119:0/461336546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621f4873350 0x5621f487f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.289+0000 7f950bfff700 1 -- 172.21.15.119:0/461336546 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94fc003b70 con 0x5621f4873c70 2023-12-07T10:21:54.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.290+0000 7f950bfff700 1 --2- 172.21.15.119:0/461336546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621f4873c70 0x5621f487c320 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f94fc000f80 tx=0x7f94fc0046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2003a7291fd46f7 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:54.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.290+0000 7f94f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:21:54.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.290+0000 7f94f2ffd700 1 -- 172.21.15.119:0/461336546 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94fc023070 con 0x5621f4873c70 2023-12-07T10:21:54.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.290+0000 7f94f2ffd700 1 -- 172.21.15.119:0/461336546 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f94fc007850 con 0x5621f4873c70 2023-12-07T10:21:54.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.290+0000 7f952d4c7b80 1 -- 172.21.15.119:0/461336546 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5621f485db00 con 0x5621f4873c70 2023-12-07T10:21:54.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.290+0000 7f94f2ffd700 1 -- 172.21.15.119:0/461336546 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94fc019040 con 0x5621f4873c70 2023-12-07T10:21:54.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.290+0000 7f952d4c7b80 10 client.16122 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:54.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.290+0000 7f952d4c7b80 1 -- 172.21.15.119:0/461336546 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5621f45e7de0 con 0x5621f4873c70 2023-12-07T10:21:54.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.291+0000 7f94f2ffd700 1 -- 172.21.15.119:0/461336546 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5621f485db00 con 0x5621f4873c70 2023-12-07T10:21:54.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.291+0000 7f94f2ffd700 1 -- 172.21.15.119:0/461336546 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f94fc026020 con 0x5621f4873c70 2023-12-07T10:21:54.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.291+0000 7f952d4c7b80 10 client.16122 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:54.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.291+0000 7f952d4c7b80 10 client.16122 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2023-12-07T10:21:54.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.291+0000 7f952d4c7b80 20 client.16122 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:54.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.291+0000 7f952d4c7b80 1 --2- 172.21.15.119:0/461336546 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x5621f488ca10 0x5621f488ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.291+0000 7f952d4c7b80 4 client.16122 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:21:54.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.291+0000 7f952d4c7b80 1 -- 172.21.15.119:0/461336546 --> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5621f48397a0 con 0x5621f488ca10 2023-12-07T10:21:54.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.291+0000 7f95137fe700 1 --2- 172.21.15.119:0/461336546 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x5621f488ca10 0x5621f488ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.292+0000 7f95137fe700 1 --2- 172.21.15.119:0/461336546 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x5621f488ca10 0x5621f488ee30 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b7423e835f2625ae server_cookie=fa7d7a221dc8c88d in_seq=0 out_seq=0 2023-12-07T10:21:54.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.292+0000 7f94f2ffd700 10 client.16122 ms_handle_connect on v2:172.21.15.153:6834/1295649464 2023-12-07T10:21:54.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.292+0000 7f94f2ffd700 1 -- 172.21.15.119:0/461336546 <== mds.4 v2:172.21.15.153:6834/1295649464 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5621f48397a0 con 0x5621f488ca10 2023-12-07T10:21:54.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.292+0000 7f94f2ffd700 10 client.16122 handle_command_reply: tid=0 2023-12-07T10:21:54.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.292+0000 7f952d4c7b80 1 client.16122 shutdown 2023-12-07T10:21:54.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.293+0000 7f952d4c7b80 1 -- 172.21.15.119:0/461336546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621f4873c70 msgr2=0x5621f487c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.293+0000 7f952d4c7b80 1 --2- 172.21.15.119:0/461336546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621f4873c70 0x5621f487c320 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f94fc000f80 tx=0x7f94fc0046a0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.293+0000 7f952d4c7b80 1 -- 172.21.15.119:0/461336546 shutdown_connections 2023-12-07T10:21:54.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.293+0000 7f952d4c7b80 1 --2- 172.21.15.119:0/461336546 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x5621f488ca10 0x5621f488ee30 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.293+0000 7f952d4c7b80 1 --2- 172.21.15.119:0/461336546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621f4877960 0x5621f487c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.293+0000 7f952d4c7b80 1 --2- 172.21.15.119:0/461336546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621f4873350 0x5621f487f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.293+0000 7f952d4c7b80 1 --2- 172.21.15.119:0/461336546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621f4873c70 0x5621f487c320 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.293+0000 7f952d4c7b80 1 -- 172.21.15.119:0/461336546 >> 172.21.15.119:0/461336546 conn(0x5621f4799360 msgr2=0x5621f4878e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:54.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.293+0000 7f94f2ffd700 0 client.16122 ms_handle_reset on v2:172.21.15.153:6834/1295649464 2023-12-07T10:21:54.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.293+0000 7f952d4c7b80 1 -- 172.21.15.119:0/461336546 shutdown_connections 2023-12-07T10:21:54.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.293+0000 7f952d4c7b80 1 -- 172.21.15.119:0/461336546 wait complete. 2023-12-07T10:21:54.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.293+0000 7f952d4c7b80 20 client.16122 trim_cache size 0 max 16384 2023-12-07T10:21:54.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.310+0000 7f952d4c7b80 1 Processor -- start 2023-12-07T10:21:54.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.310+0000 7f952d4c7b80 1 -- start start 2023-12-07T10:21:54.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.310+0000 7f952d4c7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621f4873350 0x5621f485fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.310+0000 7f952d4c7b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621f4873c70 0x5621f4893790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.310+0000 7f952d4c7b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621f4877960 0x5621f4883bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.310+0000 7f952d4c7b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621f463d920 con 0x5621f4877960 2023-12-07T10:21:54.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.310+0000 7f952d4c7b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621f485dae0 con 0x5621f4873c70 2023-12-07T10:21:54.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.310+0000 7f952d4c7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621f4876310 con 0x5621f4873350 2023-12-07T10:21:54.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.311+0000 7f95137fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621f4873350 0x5621f485fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.311+0000 7f950bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621f4873c70 0x5621f4893790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.311+0000 7f95137fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621f4873350 0x5621f485fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:39934/0 (socket says 172.21.15.119:39934) 2023-12-07T10:21:54.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.311+0000 7f950bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621f4873c70 0x5621f4893790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45542/0 (socket says 172.21.15.119:45542) 2023-12-07T10:21:54.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.311+0000 7f95137fe700 1 -- 172.21.15.119:0/3288635883 learned_addr learned my addr 172.21.15.119:0/3288635883 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:54.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.311+0000 7f9513fff700 1 --2- 172.21.15.119:0/3288635883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621f4877960 0x5621f4883bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.311+0000 7f95137fe700 1 -- 172.21.15.119:0/3288635883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621f4873c70 msgr2=0x5621f4893790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:54.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.311+0000 7f95137fe700 1 --2- 172.21.15.119:0/3288635883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621f4873c70 0x5621f4893790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.311+0000 7f95137fe700 1 -- 172.21.15.119:0/3288635883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621f4877960 msgr2=0x5621f4883bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:54.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.311+0000 7f95137fe700 1 --2- 172.21.15.119:0/3288635883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621f4877960 0x5621f4883bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.311+0000 7f95137fe700 1 -- 172.21.15.119:0/3288635883 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621f45d0c60 con 0x5621f4873350 2023-12-07T10:21:54.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.311+0000 7f950bfff700 1 --2- 172.21.15.119:0/3288635883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621f4873c70 0x5621f4893790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:54.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.311+0000 7f95137fe700 1 --2- 172.21.15.119:0/3288635883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621f4873350 0x5621f485fc70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9504098060 tx=0x7f95040bb040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64d8adf807acb39 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:54.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.311+0000 7f950a7fc700 1 -- 172.21.15.119:0/3288635883 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950409b930 con 0x5621f4873350 2023-12-07T10:21:54.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.312+0000 7f950a7fc700 1 -- 172.21.15.119:0/3288635883 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f950409ce90 con 0x5621f4873350 2023-12-07T10:21:54.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.312+0000 7f950a7fc700 1 -- 172.21.15.119:0/3288635883 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950409fcb0 con 0x5621f4873350 2023-12-07T10:21:54.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.312+0000 7f952d4c7b80 1 -- 172.21.15.119:0/3288635883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621f4873350 msgr2=0x5621f485fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.312+0000 7f952d4c7b80 1 --2- 172.21.15.119:0/3288635883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621f4873350 0x5621f485fc70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9504098060 tx=0x7f95040bb040 comp rx=0 tx=0).stop 2023-12-07T10:21:54.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.312+0000 7f952d4c7b80 1 -- 172.21.15.119:0/3288635883 shutdown_connections 2023-12-07T10:21:54.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.312+0000 7f952d4c7b80 1 --2- 172.21.15.119:0/3288635883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621f4877960 0x5621f4883bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.312+0000 7f952d4c7b80 1 --2- 172.21.15.119:0/3288635883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621f4873350 0x5621f485fc70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.312+0000 7f952d4c7b80 1 --2- 172.21.15.119:0/3288635883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621f4873c70 0x5621f4893790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.312+0000 7f952d4c7b80 1 -- 172.21.15.119:0/3288635883 >> 172.21.15.119:0/3288635883 conn(0x5621f4799360 msgr2=0x5621f487ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:54.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.312+0000 7f952d4c7b80 1 -- 172.21.15.119:0/3288635883 shutdown_connections 2023-12-07T10:21:54.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.312+0000 7f952d4c7b80 1 -- 172.21.15.119:0/3288635883 wait complete. 2023-12-07T10:21:54.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.313+0000 7f952d4c7b80 1 -- start start 2023-12-07T10:21:54.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.313+0000 7f952d4c7b80 1 Processor -- start 2023-12-07T10:21:54.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.313+0000 7f952d4c7b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621f4873350 0x5621f4884020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.313+0000 7f952d4c7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621f4873c70 0x5621f4881ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.314+0000 7f952d4c7b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621f4877960 0x5621f48821b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.314+0000 7f952d4c7b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621f4882890 con 0x5621f4877960 2023-12-07T10:21:54.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.314+0000 7f952d4c7b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621f4882a00 con 0x5621f4873350 2023-12-07T10:21:54.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.314+0000 7f952d4c7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621f48e1f40 con 0x5621f4873c70 2023-12-07T10:21:54.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.314+0000 7f952d4c7b80 10 client.0 _ll_register_callbacks cb 0x5621f475f940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:54.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.314+0000 7f950bfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621f4873c70 0x5621f4881ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.314+0000 7f95137fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621f4873350 0x5621f4884020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.314+0000 7f95137fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621f4873350 0x5621f4884020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45554/0 (socket says 172.21.15.119:45554) 2023-12-07T10:21:54.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.314+0000 7f95137fe700 1 -- 172.21.15.119:0/1384491422 learned_addr learned my addr 172.21.15.119:0/1384491422 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:54.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.314+0000 7f9513fff700 1 --2- 172.21.15.119:0/1384491422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621f4877960 0x5621f48821b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.314+0000 7f95137fe700 1 -- 172.21.15.119:0/1384491422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621f4873c70 msgr2=0x5621f4881ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:54.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.314+0000 7f95137fe700 1 --2- 172.21.15.119:0/1384491422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621f4873c70 0x5621f4881ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.314+0000 7f95137fe700 1 -- 172.21.15.119:0/1384491422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621f4877960 msgr2=0x5621f48821b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:54.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.314+0000 7f95137fe700 1 --2- 172.21.15.119:0/1384491422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621f4877960 0x5621f48821b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.314+0000 7f95137fe700 1 -- 172.21.15.119:0/1384491422 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621f45e7de0 con 0x5621f4873350 2023-12-07T10:21:54.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.314+0000 7f950bfff700 1 --2- 172.21.15.119:0/1384491422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621f4873c70 0x5621f4881ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:54.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.314+0000 7f95137fe700 1 --2- 172.21.15.119:0/1384491422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621f4873350 0x5621f4884020 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9504022c40 tx=0x7f9504009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39790ac58d80ebd7 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:54.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.314+0000 7f94f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:21:54.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.315+0000 7f94f2ffd700 1 -- 172.21.15.119:0/1384491422 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950409c260 con 0x5621f4873350 2023-12-07T10:21:54.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.315+0000 7f952d4c7b80 1 -- 172.21.15.119:0/1384491422 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94fc00f040 con 0x5621f4873350 2023-12-07T10:21:54.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.315+0000 7f94f2ffd700 1 -- 172.21.15.119:0/1384491422 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f95040a3e60 con 0x5621f4873350 2023-12-07T10:21:54.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.315+0000 7f94f2ffd700 1 -- 172.21.15.119:0/1384491422 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f950409fcb0 con 0x5621f4873350 2023-12-07T10:21:54.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.315+0000 7f952d4c7b80 10 client.26132 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:54.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.315+0000 7f952d4c7b80 1 -- 172.21.15.119:0/1384491422 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f95141a6940 con 0x5621f4873350 2023-12-07T10:21:54.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.315+0000 7f94f2ffd700 1 -- 172.21.15.119:0/1384491422 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94fc00f040 con 0x5621f4873350 2023-12-07T10:21:54.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.315+0000 7f94f2ffd700 1 -- 172.21.15.119:0/1384491422 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f950409b0e0 con 0x5621f4873350 2023-12-07T10:21:54.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.316+0000 7f952d4c7b80 10 client.26132 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:54.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.316+0000 7f952d4c7b80 10 client.26132 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2023-12-07T10:21:54.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.316+0000 7f952d4c7b80 20 client.26132 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:54.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.316+0000 7f952d4c7b80 1 --2- 172.21.15.119:0/1384491422 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x5621f48e32f0 0x5621f48e5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.316+0000 7f952d4c7b80 4 client.26132 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:21:54.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.316+0000 7f952d4c7b80 1 -- 172.21.15.119:0/1384491422 --> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5621f4784fd0 con 0x5621f48e32f0 2023-12-07T10:21:54.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.316+0000 7f950bfff700 1 --2- 172.21.15.119:0/1384491422 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x5621f48e32f0 0x5621f48e5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.316+0000 7f94f2ffd700 10 client.26132 ms_handle_connect on v2:172.21.15.153:6834/1295649464 2023-12-07T10:21:54.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.316+0000 7f950bfff700 1 --2- 172.21.15.119:0/1384491422 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x5621f48e32f0 0x5621f48e5710 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=95174276ffbe7fdf server_cookie=ad269457cb126697 in_seq=0 out_seq=0 2023-12-07T10:21:54.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.317+0000 7f94f2ffd700 1 -- 172.21.15.119:0/1384491422 <== mds.4 v2:172.21.15.153:6834/1295649464 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5621f4784fd0 con 0x5621f48e32f0 2023-12-07T10:21:54.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.317+0000 7f94f2ffd700 10 client.26132 handle_command_reply: tid=0 2023-12-07T10:21:54.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.317+0000 7f952d4c7b80 1 client.26132 shutdown 2023-12-07T10:21:54.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.317+0000 7f952d4c7b80 1 -- 172.21.15.119:0/1384491422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621f4873350 msgr2=0x5621f4884020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.317+0000 7f952d4c7b80 1 --2- 172.21.15.119:0/1384491422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621f4873350 0x5621f4884020 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9504022c40 tx=0x7f9504009ca0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.317+0000 7f952d4c7b80 1 -- 172.21.15.119:0/1384491422 shutdown_connections 2023-12-07T10:21:54.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.317+0000 7f952d4c7b80 1 --2- 172.21.15.119:0/1384491422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5621f4877960 0x5621f48821b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.317+0000 7f952d4c7b80 1 --2- 172.21.15.119:0/1384491422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5621f4873350 0x5621f4884020 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.317+0000 7f952d4c7b80 1 --2- 172.21.15.119:0/1384491422 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x5621f48e32f0 0x5621f48e5710 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.317+0000 7f94f2ffd700 0 client.26132 ms_handle_reset on v2:172.21.15.153:6834/1295649464 2023-12-07T10:21:54.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.317+0000 7f952d4c7b80 1 --2- 172.21.15.119:0/1384491422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621f4873c70 0x5621f4881ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.317+0000 7f952d4c7b80 1 -- 172.21.15.119:0/1384491422 >> 172.21.15.119:0/1384491422 conn(0x5621f4799360 msgr2=0x5621f4893790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:54.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.318+0000 7f952d4c7b80 1 -- 172.21.15.119:0/1384491422 shutdown_connections 2023-12-07T10:21:54.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.318+0000 7f952d4c7b80 1 -- 172.21.15.119:0/1384491422 wait complete. 2023-12-07T10:21:54.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.318+0000 7f952d4c7b80 20 client.26132 trim_cache size 0 max 16384 2023-12-07T10:21:54.323 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:21:54.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.318+0000 7f9519bff700 1 -- 172.21.15.119:0/2677650132 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f95000a1da0 msgr2=0x7f95000a4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.318+0000 7f9519bff700 1 --2- 172.21.15.119:0/2677650132 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f95000a1da0 0x7f95000a4260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f950c009f00 tx=0x7f950c009260 comp rx=0 tx=0).stop 2023-12-07T10:21:54.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.319+0000 7f9519bff700 1 -- 172.21.15.119:0/2677650132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9514068a50 msgr2=0x7f951419f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.319+0000 7f9519bff700 1 --2- 172.21.15.119:0/2677650132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9514068a50 0x7f951419f950 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9504005910 tx=0x7f9504004730 comp rx=0 tx=0).stop 2023-12-07T10:21:54.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.319+0000 7f9519bff700 1 -- 172.21.15.119:0/2677650132 shutdown_connections 2023-12-07T10:21:54.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.319+0000 7f9519bff700 1 --2- 172.21.15.119:0/2677650132 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f95000a1da0 0x7f95000a4260 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.319+0000 7f9519bff700 1 --2- 172.21.15.119:0/2677650132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9514106ad0 0x7f9514199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.319+0000 7f9519bff700 1 --2- 172.21.15.119:0/2677650132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9514068a50 0x7f951419f950 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.319+0000 7f9519bff700 1 --2- 172.21.15.119:0/2677650132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9514106110 0x7f951419fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.319+0000 7f9519bff700 1 -- 172.21.15.119:0/2677650132 >> 172.21.15.119:0/2677650132 conn(0x7f9514077610 msgr2=0x7f951410cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:54.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.319+0000 7f9519bff700 1 -- 172.21.15.119:0/2677650132 shutdown_connections 2023-12-07T10:21:54.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.319+0000 7f9519bff700 1 -- 172.21.15.119:0/2677650132 wait complete. 2023-12-07T10:21:54.341 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-12-07T10:21:54.538 INFO:tasks.workunit.client.0.smithi119.stdout: 1 20153 17.65 MB/sec warmup 7 sec latency 8.648 ms 2023-12-07T10:21:54.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.572+0000 7f77debff700 1 -- 172.21.15.119:0/3327526586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d8107640 msgr2=0x7f77d8107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.572+0000 7f77debff700 1 --2- 172.21.15.119:0/3327526586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d8107640 0x7f77d8107a20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009990 tx=0x7f77d4009ca0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.572+0000 7f77debff700 1 -- 172.21.15.119:0/3327526586 shutdown_connections 2023-12-07T10:21:54.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.572+0000 7f77debff700 1 --2- 172.21.15.119:0/3327526586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77d8101f40 0x7f77d810e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.572+0000 7f77debff700 1 --2- 172.21.15.119:0/3327526586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d8107640 0x7f77d8107a20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.572+0000 7f77debff700 1 --2- 172.21.15.119:0/3327526586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77d8101580 0x7f77d8101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.572+0000 7f77debff700 1 -- 172.21.15.119:0/3327526586 >> 172.21.15.119:0/3327526586 conn(0x7f77d80fd050 msgr2=0x7f77d80ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:54.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.572+0000 7f77debff700 1 -- 172.21.15.119:0/3327526586 shutdown_connections 2023-12-07T10:21:54.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.572+0000 7f77debff700 1 -- 172.21.15.119:0/3327526586 wait complete. 2023-12-07T10:21:54.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.574+0000 7f77debff700 1 Processor -- start 2023-12-07T10:21:54.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.574+0000 7f77debff700 1 -- start start 2023-12-07T10:21:54.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.574+0000 7f77debff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77d8101580 0x7f77d8073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.574+0000 7f77debff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77d8101f40 0x7f77d8073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.574+0000 7f77debff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d8107640 0x7f77d8074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.574+0000 7f77debff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d81a5d50 con 0x7f77d8101f40 2023-12-07T10:21:54.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.574+0000 7f77debff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d81a5ec0 con 0x7f77d8101580 2023-12-07T10:21:54.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.574+0000 7f77debff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d81a6080 con 0x7f77d8107640 2023-12-07T10:21:54.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.575+0000 7f77dc99b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77d8101580 0x7f77d8073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.575+0000 7f77dc99b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77d8101580 0x7f77d8073910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45570/0 (socket says 172.21.15.119:45570) 2023-12-07T10:21:54.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.575+0000 7f77dc99b700 1 -- 172.21.15.119:0/1751080493 learned_addr learned my addr 172.21.15.119:0/1751080493 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:54.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.575+0000 7f77cffff700 1 --2- 172.21.15.119:0/1751080493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77d8101f40 0x7f77d8073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.575+0000 7f77dd19c700 1 --2- 172.21.15.119:0/1751080493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d8107640 0x7f77d8074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.575+0000 7f77dc99b700 1 -- 172.21.15.119:0/1751080493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d8107640 msgr2=0x7f77d8074390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.575+0000 7f77dc99b700 1 --2- 172.21.15.119:0/1751080493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d8107640 0x7f77d8074390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.575+0000 7f77dc99b700 1 -- 172.21.15.119:0/1751080493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77d8101f40 msgr2=0x7f77d8073e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.575+0000 7f77dc99b700 1 --2- 172.21.15.119:0/1751080493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77d8101f40 0x7f77d8073e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.575+0000 7f77dc99b700 1 -- 172.21.15.119:0/1751080493 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d40095f0 con 0x7f77d8101580 2023-12-07T10:21:54.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.575+0000 7f77dd19c700 1 --2- 172.21.15.119:0/1751080493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d8107640 0x7f77d8074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:21:54.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.575+0000 7f77dc99b700 1 --2- 172.21.15.119:0/1751080493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77d8101580 0x7f77d8073910 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f77d4000c00 tx=0x7f77d4005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:54.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.576+0000 7f77cdffb700 1 -- 172.21.15.119:0/1751080493 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d401d070 con 0x7f77d8101580 2023-12-07T10:21:54.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.576+0000 7f77debff700 1 -- 172.21.15.119:0/1751080493 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d81a6300 con 0x7f77d8101580 2023-12-07T10:21:54.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.576+0000 7f77debff700 1 -- 172.21.15.119:0/1751080493 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d81a6780 con 0x7f77d8101580 2023-12-07T10:21:54.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.576+0000 7f77cdffb700 1 -- 172.21.15.119:0/1751080493 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77d4022470 con 0x7f77d8101580 2023-12-07T10:21:54.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.576+0000 7f77cdffb700 1 -- 172.21.15.119:0/1751080493 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d4017610 con 0x7f77d8101580 2023-12-07T10:21:54.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.580+0000 7f77cdffb700 1 -- 172.21.15.119:0/1751080493 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 25) v1 ==== 136717+0+0 (secure 0 0 0) 0x7f77d4017770 con 0x7f77d8101580 2023-12-07T10:21:54.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.581+0000 7f77f24f4b80 1 Processor -- start 2023-12-07T10:21:54.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.581+0000 7f77f24f4b80 1 -- start start 2023-12-07T10:21:54.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.581+0000 7f77f24f4b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5c5afeb40 0x55f5c5afef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.581+0000 7f77f24f4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5aff460 0x55f5c5b02c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.581+0000 7f77f24f4b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5c5b03150 0x55f5c5b03570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.581+0000 7f77f24f4b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c5a727c0 con 0x55f5c5b03150 2023-12-07T10:21:54.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.581+0000 7f77f24f4b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c5a597a0 con 0x55f5c5afeb40 2023-12-07T10:21:54.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.581+0000 7f77f24f4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c5a570c0 con 0x55f5c5aff460 2023-12-07T10:21:54.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.581+0000 7f77dd19c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5c5afeb40 0x55f5c5afef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.581+0000 7f77dd19c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5c5afeb40 0x55f5c5afef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45580/0 (socket says 172.21.15.119:45580) 2023-12-07T10:21:54.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.581+0000 7f77dd19c700 1 -- 172.21.15.119:0/3117056989 learned_addr learned my addr 172.21.15.119:0/3117056989 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:54.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.582+0000 7f77dc99b700 1 --2- 172.21.15.119:0/3117056989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5aff460 0x55f5c5b02c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.582+0000 7f77cffff700 1 --2- 172.21.15.119:0/3117056989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5c5b03150 0x55f5c5b03570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.582+0000 7f77dd19c700 1 -- 172.21.15.119:0/3117056989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5aff460 msgr2=0x55f5c5b02c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:54.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.582+0000 7f77dd19c700 1 --2- 172.21.15.119:0/3117056989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5aff460 0x55f5c5b02c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.582+0000 7f77dd19c700 1 -- 172.21.15.119:0/3117056989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5c5b03150 msgr2=0x55f5c5b03570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:54.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.582+0000 7f77dd19c700 1 --2- 172.21.15.119:0/3117056989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5c5b03150 0x55f5c5b03570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.582+0000 7f77dd19c700 1 -- 172.21.15.119:0/3117056989 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5c58cf630 con 0x55f5c5afeb40 2023-12-07T10:21:54.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.582+0000 7f77dc99b700 1 --2- 172.21.15.119:0/3117056989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5aff460 0x55f5c5b02c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:21:54.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.582+0000 7f77cdffb700 1 --2- 172.21.15.119:0/1751080493 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f77c00a1e20 0x7f77c00a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.582+0000 7f77dd19c700 1 --2- 172.21.15.119:0/3117056989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5c5afeb40 0x55f5c5afef20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f77d0009440 tx=0x7f77d0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b07b34a18f8e9d66 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:54.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.582+0000 7f77cdffb700 1 -- 172.21.15.119:0/1751080493 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f77d40ceff0 con 0x7f77d8101580 2023-12-07T10:21:54.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.582+0000 7f77dc99b700 1 --2- 172.21.15.119:0/1751080493 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f77c00a1e20 0x7f77c00a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.582+0000 7f77beffd700 1 -- 172.21.15.119:0/3117056989 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d000e070 con 0x55f5c5afeb40 2023-12-07T10:21:54.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.583+0000 7f77beffd700 1 -- 172.21.15.119:0/3117056989 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77d000f040 con 0x55f5c5afeb40 2023-12-07T10:21:54.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.583+0000 7f77beffd700 1 -- 172.21.15.119:0/3117056989 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d0014ac0 con 0x55f5c5afeb40 2023-12-07T10:21:54.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.583+0000 7f77dc99b700 1 --2- 172.21.15.119:0/1751080493 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f77c00a1e20 0x7f77c00a42e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f77d40cc2b0 tx=0x7f77d4009ca0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:54.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.583+0000 7f77f24f4b80 1 -- 172.21.15.119:0/3117056989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5c5afeb40 msgr2=0x55f5c5afef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.583+0000 7f77f24f4b80 1 --2- 172.21.15.119:0/3117056989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5c5afeb40 0x55f5c5afef20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f77d0009440 tx=0x7f77d0009920 comp rx=0 tx=0).stop 2023-12-07T10:21:54.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.584+0000 7f77f24f4b80 1 -- 172.21.15.119:0/3117056989 shutdown_connections 2023-12-07T10:21:54.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.584+0000 7f77f24f4b80 1 --2- 172.21.15.119:0/3117056989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5c5b03150 0x55f5c5b03570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.584+0000 7f77f24f4b80 1 --2- 172.21.15.119:0/3117056989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5c5afeb40 0x55f5c5afef20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.584+0000 7f77f24f4b80 1 --2- 172.21.15.119:0/3117056989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5aff460 0x55f5c5b02c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.584+0000 7f77f24f4b80 1 -- 172.21.15.119:0/3117056989 >> 172.21.15.119:0/3117056989 conn(0x55f5c5a2d360 msgr2=0x55f5c5af9130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:54.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.584+0000 7f77f24f4b80 1 -- 172.21.15.119:0/3117056989 shutdown_connections 2023-12-07T10:21:54.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.584+0000 7f77f24f4b80 1 -- 172.21.15.119:0/3117056989 wait complete. 2023-12-07T10:21:54.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.584+0000 7f77f24f4b80 1 -- start start 2023-12-07T10:21:54.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.584+0000 7f77f24f4b80 1 Processor -- start 2023-12-07T10:21:54.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.585+0000 7f77f24f4b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5c5afeb40 0x55f5c5b0a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.585+0000 7f77f24f4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5aff460 0x55f5c5b07ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.585+0000 7f77f24f4b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5c5b03150 0x55f5c5b08050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.585+0000 7f77f24f4b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c5a529b0 con 0x55f5c5b03150 2023-12-07T10:21:54.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.585+0000 7f77f24f4b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c5a0c560 con 0x55f5c5afeb40 2023-12-07T10:21:54.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.585+0000 7f77f24f4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c5a0c200 con 0x55f5c5aff460 2023-12-07T10:21:54.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.585+0000 7f77f24f4b80 10 client.0 _ll_register_callbacks cb 0x55f5c59f3940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:54.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.585+0000 7f77cffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5aff460 0x55f5c5b07ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.585+0000 7f77dd19c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5c5afeb40 0x55f5c5b0a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.585+0000 7f77cffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5aff460 0x55f5c5b07ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:39984/0 (socket says 172.21.15.119:39984) 2023-12-07T10:21:54.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.585+0000 7f77cffff700 1 -- 172.21.15.119:0/444119336 learned_addr learned my addr 172.21.15.119:0/444119336 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:54.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.585+0000 7f77dd19c700 1 --2- 172.21.15.119:0/444119336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5c5b03150 0x55f5c5b08050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.586+0000 7f77cffff700 1 -- 172.21.15.119:0/444119336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5c5afeb40 msgr2=0x55f5c5b0a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:54.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.586+0000 7f77cffff700 1 --2- 172.21.15.119:0/444119336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5c5afeb40 0x55f5c5b0a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.586+0000 7f77cffff700 1 -- 172.21.15.119:0/444119336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5c5b03150 msgr2=0x55f5c5b08050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:54.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.586+0000 7f77cffff700 1 --2- 172.21.15.119:0/444119336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5c5b03150 0x55f5c5b08050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.586+0000 7f77cffff700 1 -- 172.21.15.119:0/444119336 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d00090f0 con 0x55f5c5aff460 2023-12-07T10:21:54.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.586+0000 7f77cffff700 1 --2- 172.21.15.119:0/444119336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5aff460 0x55f5c5b07ab0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009fd0 tx=0x7f77c8011a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3fb0f4f3f6b8569 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:54.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.586+0000 7f77b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:21:54.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.586+0000 7f77b7fff700 1 -- 172.21.15.119:0/444119336 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8016070 con 0x55f5c5aff460 2023-12-07T10:21:54.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.586+0000 7f77f24f4b80 1 -- 172.21.15.119:0/444119336 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5c5af1b00 con 0x55f5c5aff460 2023-12-07T10:21:54.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.586+0000 7f77b7fff700 1 -- 172.21.15.119:0/444119336 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f77c800f690 con 0x55f5c5aff460 2023-12-07T10:21:54.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.586+0000 7f77b7fff700 1 -- 172.21.15.119:0/444119336 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77c800f850 con 0x55f5c5aff460 2023-12-07T10:21:54.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.587+0000 7f77f24f4b80 10 client.26140 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:54.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.587+0000 7f77f24f4b80 1 -- 172.21.15.119:0/444119336 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f5c587bde0 con 0x55f5c5aff460 2023-12-07T10:21:54.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.587+0000 7f77b7fff700 1 -- 172.21.15.119:0/444119336 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f5c5af1b00 con 0x55f5c5aff460 2023-12-07T10:21:54.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.587+0000 7f77b7fff700 1 -- 172.21.15.119:0/444119336 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f77c801abd0 con 0x55f5c5aff460 2023-12-07T10:21:54.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.587+0000 7f77f24f4b80 10 client.26140 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:54.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.587+0000 7f77f24f4b80 10 client.26140 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-12-07T10:21:54.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.587+0000 7f77f24f4b80 20 client.26140 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:54.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.587+0000 7f77f24f4b80 1 --2- 172.21.15.119:0/444119336 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x55f5c5b184a0 0x55f5c5b1a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.587+0000 7f77f24f4b80 4 client.26140 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:21:54.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.587+0000 7f77f24f4b80 1 -- 172.21.15.119:0/444119336 --> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5c5acd7a0 con 0x55f5c5b184a0 2023-12-07T10:21:54.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.587+0000 7f77dd19c700 1 --2- 172.21.15.119:0/444119336 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x55f5c5b184a0 0x55f5c5b1a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.588+0000 7f77dd19c700 1 --2- 172.21.15.119:0/444119336 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x55f5c5b184a0 0x55f5c5b1a8c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=67d0d7f960e4cd5b server_cookie=f27544d29821672f in_seq=0 out_seq=0 2023-12-07T10:21:54.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.588+0000 7f77b7fff700 10 client.26140 ms_handle_connect on v2:172.21.15.159:6834/2406380792 2023-12-07T10:21:54.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.589+0000 7f77b7fff700 1 -- 172.21.15.119:0/444119336 <== mds.3 v2:172.21.15.159:6834/2406380792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f5c5acd7a0 con 0x55f5c5b184a0 2023-12-07T10:21:54.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.589+0000 7f77b7fff700 10 client.26140 handle_command_reply: tid=0 2023-12-07T10:21:54.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.589+0000 7f77f24f4b80 1 client.26140 shutdown 2023-12-07T10:21:54.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.589+0000 7f77f24f4b80 1 -- 172.21.15.119:0/444119336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5aff460 msgr2=0x55f5c5b07ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.589+0000 7f77f24f4b80 1 --2- 172.21.15.119:0/444119336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5aff460 0x55f5c5b07ab0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009fd0 tx=0x7f77c8011a10 comp rx=0 tx=0).stop 2023-12-07T10:21:54.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.589+0000 7f77f24f4b80 1 -- 172.21.15.119:0/444119336 shutdown_connections 2023-12-07T10:21:54.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.589+0000 7f77f24f4b80 1 --2- 172.21.15.119:0/444119336 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x55f5c5b184a0 0x55f5c5b1a8c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.589+0000 7f77f24f4b80 1 --2- 172.21.15.119:0/444119336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5c5b03150 0x55f5c5b08050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.589+0000 7f77f24f4b80 1 --2- 172.21.15.119:0/444119336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5c5afeb40 0x55f5c5b0a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.589+0000 7f77f24f4b80 1 --2- 172.21.15.119:0/444119336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5aff460 0x55f5c5b07ab0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.589+0000 7f77b7fff700 0 client.26140 ms_handle_reset on v2:172.21.15.159:6834/2406380792 2023-12-07T10:21:54.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.589+0000 7f77f24f4b80 1 -- 172.21.15.119:0/444119336 >> 172.21.15.119:0/444119336 conn(0x55f5c5a2d360 msgr2=0x55f5c5af90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:54.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.589+0000 7f77f24f4b80 1 -- 172.21.15.119:0/444119336 shutdown_connections 2023-12-07T10:21:54.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.589+0000 7f77f24f4b80 1 -- 172.21.15.119:0/444119336 wait complete. 2023-12-07T10:21:54.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.589+0000 7f77f24f4b80 20 client.26140 trim_cache size 0 max 16384 2023-12-07T10:21:54.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.605+0000 7f77f24f4b80 1 Processor -- start 2023-12-07T10:21:54.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.605+0000 7f77f24f4b80 1 -- start start 2023-12-07T10:21:54.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.605+0000 7f77f24f4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5afeb40 0x55f5c5b08000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.605+0000 7f77f24f4b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5c5aff460 0x55f5c5b00b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.605+0000 7f77f24f4b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5c5b03150 0x55f5c5b13e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.605+0000 7f77f24f4b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c58d1920 con 0x55f5c5b03150 2023-12-07T10:21:54.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.605+0000 7f77f24f4b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c5af9c50 con 0x55f5c5aff460 2023-12-07T10:21:54.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.605+0000 7f77f24f4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c5b01cb0 con 0x55f5c5afeb40 2023-12-07T10:21:54.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.606+0000 7f77cffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5c5aff460 0x55f5c5b00b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.606+0000 7f77dd19c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5afeb40 0x55f5c5b08000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.606+0000 7f77cffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5c5aff460 0x55f5c5b00b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45594/0 (socket says 172.21.15.119:45594) 2023-12-07T10:21:54.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.606+0000 7f77dd19c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5afeb40 0x55f5c5b08000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:39994/0 (socket says 172.21.15.119:39994) 2023-12-07T10:21:54.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.606+0000 7f77cffff700 1 -- 172.21.15.119:0/843638335 learned_addr learned my addr 172.21.15.119:0/843638335 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:54.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.606+0000 7f77dd19c700 1 --2- 172.21.15.119:0/843638335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5c5b03150 0x55f5c5b13e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.606+0000 7f77cffff700 1 -- 172.21.15.119:0/843638335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5afeb40 msgr2=0x55f5c5b08000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:54.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.606+0000 7f77cffff700 1 --2- 172.21.15.119:0/843638335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5afeb40 0x55f5c5b08000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.606+0000 7f77cffff700 1 -- 172.21.15.119:0/843638335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5c5b03150 msgr2=0x55f5c5b13e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:54.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.606+0000 7f77cffff700 1 --2- 172.21.15.119:0/843638335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5c5b03150 0x55f5c5b13e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.606+0000 7f77cffff700 1 -- 172.21.15.119:0/843638335 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5c5b05810 con 0x55f5c5aff460 2023-12-07T10:21:54.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.606+0000 7f77dd19c700 1 --2- 172.21.15.119:0/843638335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5afeb40 0x55f5c5b08000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:54.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.606+0000 7f77cffff700 1 --2- 172.21.15.119:0/843638335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5c5aff460 0x55f5c5b00b80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f77c801af30 tx=0x7f77c8005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef97346f218dbcfb server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:54.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.607+0000 7f77bf7fe700 1 -- 172.21.15.119:0/843638335 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8016070 con 0x55f5c5aff460 2023-12-07T10:21:54.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.607+0000 7f77bf7fe700 1 -- 172.21.15.119:0/843638335 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77c8019d40 con 0x55f5c5aff460 2023-12-07T10:21:54.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.607+0000 7f77bf7fe700 1 -- 172.21.15.119:0/843638335 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8005620 con 0x55f5c5aff460 2023-12-07T10:21:54.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.607+0000 7f77f24f4b80 1 -- 172.21.15.119:0/843638335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5c5aff460 msgr2=0x55f5c5b00b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.607+0000 7f77f24f4b80 1 --2- 172.21.15.119:0/843638335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5c5aff460 0x55f5c5b00b80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f77c801af30 tx=0x7f77c8005e40 comp rx=0 tx=0).stop 2023-12-07T10:21:54.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.608+0000 7f77f24f4b80 1 -- 172.21.15.119:0/843638335 shutdown_connections 2023-12-07T10:21:54.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.608+0000 7f77f24f4b80 1 --2- 172.21.15.119:0/843638335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5c5b03150 0x55f5c5b13e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.608+0000 7f77f24f4b80 1 --2- 172.21.15.119:0/843638335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5afeb40 0x55f5c5b08000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.608+0000 7f77f24f4b80 1 --2- 172.21.15.119:0/843638335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5c5aff460 0x55f5c5b00b80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.608+0000 7f77f24f4b80 1 -- 172.21.15.119:0/843638335 >> 172.21.15.119:0/843638335 conn(0x55f5c5a2d360 msgr2=0x55f5c5b1f220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:54.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.608+0000 7f77f24f4b80 1 -- 172.21.15.119:0/843638335 shutdown_connections 2023-12-07T10:21:54.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.608+0000 7f77f24f4b80 1 -- 172.21.15.119:0/843638335 wait complete. 2023-12-07T10:21:54.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.608+0000 7f77f24f4b80 1 -- start start 2023-12-07T10:21:54.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.609+0000 7f77f24f4b80 1 Processor -- start 2023-12-07T10:21:54.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.609+0000 7f77f24f4b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5c5afeb40 0x55f5c5b0f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.609+0000 7f77f24f4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5aff460 0x55f5c5b0d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.609+0000 7f77f24f4b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5c5b03150 0x55f5c5b0d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.609+0000 7f77f24f4b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c5b0e100 con 0x55f5c5afeb40 2023-12-07T10:21:54.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.609+0000 7f77f24f4b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c5b6d9a0 con 0x55f5c5b03150 2023-12-07T10:21:54.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.609+0000 7f77f24f4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c5b6dae0 con 0x55f5c5aff460 2023-12-07T10:21:54.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.609+0000 7f77f24f4b80 10 client.0 _ll_register_callbacks cb 0x55f5c59f3940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:54.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.609+0000 7f77cffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5aff460 0x55f5c5b0d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.609+0000 7f77cffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5aff460 0x55f5c5b0d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:40010/0 (socket says 172.21.15.119:40010) 2023-12-07T10:21:54.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.609+0000 7f77cffff700 1 -- 172.21.15.119:0/811111364 learned_addr learned my addr 172.21.15.119:0/811111364 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:54.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.609+0000 7f77dd19c700 1 --2- 172.21.15.119:0/811111364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5c5afeb40 0x55f5c5b0f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.610+0000 7f77dd19c700 1 --2- 172.21.15.119:0/811111364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5c5b03150 0x55f5c5b0d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.610+0000 7f77cffff700 1 -- 172.21.15.119:0/811111364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5c5b03150 msgr2=0x55f5c5b0d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:54.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.610+0000 7f77cffff700 1 --2- 172.21.15.119:0/811111364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5c5b03150 0x55f5c5b0d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.610+0000 7f77cffff700 1 -- 172.21.15.119:0/811111364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5c5afeb40 msgr2=0x55f5c5b0f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:54.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.610+0000 7f77cffff700 1 --2- 172.21.15.119:0/811111364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5c5afeb40 0x55f5c5b0f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.610+0000 7f77cffff700 1 -- 172.21.15.119:0/811111364 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d002c040 con 0x55f5c5aff460 2023-12-07T10:21:54.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.610+0000 7f77dd19c700 1 --2- 172.21.15.119:0/811111364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5c5b03150 0x55f5c5b0d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:21:54.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.610+0000 7f77cffff700 1 --2- 172.21.15.119:0/811111364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5aff460 0x55f5c5b0d390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f77c80193b0 tx=0x7f77c8005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7abaf28aa4290351 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:54.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.610+0000 7f77b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:21:54.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.610+0000 7f77b7fff700 1 -- 172.21.15.119:0/811111364 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8016040 con 0x55f5c5aff460 2023-12-07T10:21:54.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.610+0000 7f77f24f4b80 1 -- 172.21.15.119:0/811111364 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77d000f040 con 0x55f5c5aff460 2023-12-07T10:21:54.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.610+0000 7f77b7fff700 1 -- 172.21.15.119:0/811111364 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f77c8000e40 con 0x55f5c5aff460 2023-12-07T10:21:54.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.610+0000 7f77b7fff700 1 -- 172.21.15.119:0/811111364 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77c8011220 con 0x55f5c5aff460 2023-12-07T10:21:54.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.610+0000 7f77f24f4b80 10 client.26146 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:54.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.611+0000 7f77f24f4b80 1 -- 172.21.15.119:0/811111364 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f5c587bde0 con 0x55f5c5aff460 2023-12-07T10:21:54.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.611+0000 7f77b7fff700 1 -- 172.21.15.119:0/811111364 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77d000f040 con 0x55f5c5aff460 2023-12-07T10:21:54.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.611+0000 7f77b7fff700 1 -- 172.21.15.119:0/811111364 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f77c8011a80 con 0x55f5c5aff460 2023-12-07T10:21:54.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.611+0000 7f77f24f4b80 10 client.26146 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:54.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.611+0000 7f77f24f4b80 10 client.26146 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-12-07T10:21:54.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.611+0000 7f77f24f4b80 20 client.26146 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:54.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.611+0000 7f77f24f4b80 1 --2- 172.21.15.119:0/811111364 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x55f5c5b6ef60 0x55f5c5b71380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.611+0000 7f77f24f4b80 4 client.26146 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:21:54.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.611+0000 7f77f24f4b80 1 -- 172.21.15.119:0/811111364 --> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f5c5a18fd0 con 0x55f5c5b6ef60 2023-12-07T10:21:54.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.611+0000 7f77dd19c700 1 --2- 172.21.15.119:0/811111364 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x55f5c5b6ef60 0x55f5c5b71380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.612+0000 7f77b7fff700 10 client.26146 ms_handle_connect on v2:172.21.15.159:6834/2406380792 2023-12-07T10:21:54.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.612+0000 7f77dd19c700 1 --2- 172.21.15.119:0/811111364 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x55f5c5b6ef60 0x55f5c5b71380 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=97b623f28f0de2c2 server_cookie=4247d6bfeba95c70 in_seq=0 out_seq=0 2023-12-07T10:21:54.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.612+0000 7f77b7fff700 1 -- 172.21.15.119:0/811111364 <== mds.3 v2:172.21.15.159:6834/2406380792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f5c5a18fd0 con 0x55f5c5b6ef60 2023-12-07T10:21:54.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.612+0000 7f77b7fff700 10 client.26146 handle_command_reply: tid=0 2023-12-07T10:21:54.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.612+0000 7f77f24f4b80 1 client.26146 shutdown 2023-12-07T10:21:54.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.612+0000 7f77f24f4b80 1 -- 172.21.15.119:0/811111364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5aff460 msgr2=0x55f5c5b0d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.612+0000 7f77f24f4b80 1 --2- 172.21.15.119:0/811111364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5aff460 0x55f5c5b0d390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f77c80193b0 tx=0x7f77c8005fa0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.612+0000 7f77f24f4b80 1 -- 172.21.15.119:0/811111364 shutdown_connections 2023-12-07T10:21:54.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.612+0000 7f77f24f4b80 1 --2- 172.21.15.119:0/811111364 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x55f5c5b6ef60 0x55f5c5b71380 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.612+0000 7f77f24f4b80 1 --2- 172.21.15.119:0/811111364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f5c5b03150 0x55f5c5b0d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.612+0000 7f77f24f4b80 1 --2- 172.21.15.119:0/811111364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5c5afeb40 0x55f5c5b0f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.612+0000 7f77b7fff700 0 client.26146 ms_handle_reset on v2:172.21.15.159:6834/2406380792 2023-12-07T10:21:54.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.612+0000 7f77f24f4b80 1 --2- 172.21.15.119:0/811111364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5c5aff460 0x55f5c5b0d390 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.612+0000 7f77f24f4b80 1 -- 172.21.15.119:0/811111364 >> 172.21.15.119:0/811111364 conn(0x55f5c5a2d360 msgr2=0x55f5c5b02170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:54.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.613+0000 7f77f24f4b80 1 -- 172.21.15.119:0/811111364 shutdown_connections 2023-12-07T10:21:54.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.613+0000 7f77f24f4b80 1 -- 172.21.15.119:0/811111364 wait complete. 2023-12-07T10:21:54.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.613+0000 7f77f24f4b80 20 client.26146 trim_cache size 0 max 16384 2023-12-07T10:21:54.618 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:21:54.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.613+0000 7f77debff700 1 -- 172.21.15.119:0/1751080493 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f77c00a1e20 msgr2=0x7f77c00a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.613+0000 7f77debff700 1 --2- 172.21.15.119:0/1751080493 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f77c00a1e20 0x7f77c00a42e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f77d40cc2b0 tx=0x7f77d4009ca0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.613+0000 7f77debff700 1 -- 172.21.15.119:0/1751080493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77d8101580 msgr2=0x7f77d8073910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.613+0000 7f77debff700 1 --2- 172.21.15.119:0/1751080493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77d8101580 0x7f77d8073910 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f77d4000c00 tx=0x7f77d4005740 comp rx=0 tx=0).stop 2023-12-07T10:21:54.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.613+0000 7f77debff700 1 -- 172.21.15.119:0/1751080493 shutdown_connections 2023-12-07T10:21:54.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.613+0000 7f77debff700 1 --2- 172.21.15.119:0/1751080493 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f77c00a1e20 0x7f77c00a42e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.613+0000 7f77debff700 1 --2- 172.21.15.119:0/1751080493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d8107640 0x7f77d8074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.613+0000 7f77debff700 1 --2- 172.21.15.119:0/1751080493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77d8101580 0x7f77d8073910 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.613+0000 7f77debff700 1 --2- 172.21.15.119:0/1751080493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77d8101f40 0x7f77d8073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.613+0000 7f77debff700 1 -- 172.21.15.119:0/1751080493 >> 172.21.15.119:0/1751080493 conn(0x7f77d80fd050 msgr2=0x7f77d80fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:54.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.614+0000 7f77debff700 1 -- 172.21.15.119:0/1751080493 shutdown_connections 2023-12-07T10:21:54.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.614+0000 7f77debff700 1 -- 172.21.15.119:0/1751080493 wait complete. 2023-12-07T10:21:54.634 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-07T10:21:54.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.872+0000 7fbc325ff700 1 -- 172.21.15.119:0/1558257279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc2c100460 msgr2=0x7fbc2c1008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.872+0000 7fbc325ff700 1 --2- 172.21.15.119:0/1558257279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc2c100460 0x7fbc2c1008e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbc20009990 tx=0x7fbc20009ca0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.872+0000 7fbc325ff700 1 -- 172.21.15.119:0/1558257279 shutdown_connections 2023-12-07T10:21:54.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.872+0000 7fbc325ff700 1 --2- 172.21.15.119:0/1558257279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc2c100e20 0x7fbc2c10d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.872+0000 7fbc325ff700 1 --2- 172.21.15.119:0/1558257279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc2c1053d0 0x7fbc2c1057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.872+0000 7fbc325ff700 1 --2- 172.21.15.119:0/1558257279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc2c100460 0x7fbc2c1008e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.872+0000 7fbc325ff700 1 -- 172.21.15.119:0/1558257279 >> 172.21.15.119:0/1558257279 conn(0x7fbc2c077610 msgr2=0x7fbc2c077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:54.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.872+0000 7fbc325ff700 1 -- 172.21.15.119:0/1558257279 shutdown_connections 2023-12-07T10:21:54.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.872+0000 7fbc325ff700 1 -- 172.21.15.119:0/1558257279 wait complete. 2023-12-07T10:21:54.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.873+0000 7fbc325ff700 1 Processor -- start 2023-12-07T10:21:54.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.873+0000 7fbc325ff700 1 -- start start 2023-12-07T10:21:54.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.874+0000 7fbc325ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc2c100460 0x7fbc2c0fded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.874+0000 7fbc325ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc2c100e20 0x7fbc2c0fe410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.874+0000 7fbc325ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc2c1053d0 0x7fbc2c0fe950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.874+0000 7fbc325ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c1a5db0 con 0x7fbc2c100460 2023-12-07T10:21:54.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.874+0000 7fbc325ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c1a5f20 con 0x7fbc2c1053d0 2023-12-07T10:21:54.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.874+0000 7fbc325ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c1a60e0 con 0x7fbc2c100e20 2023-12-07T10:21:54.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.874+0000 7fbc2b7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc2c100e20 0x7fbc2c0fe410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.874+0000 7fbc2b7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc2c100e20 0x7fbc2c0fe410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:40028/0 (socket says 172.21.15.119:40028) 2023-12-07T10:21:54.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.874+0000 7fbc2b7fe700 1 -- 172.21.15.119:0/1543450727 learned_addr learned my addr 172.21.15.119:0/1543450727 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:54.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.874+0000 7fbc2bfff700 1 --2- 172.21.15.119:0/1543450727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc2c100460 0x7fbc2c0fded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.874+0000 7fbc30b9c700 1 --2- 172.21.15.119:0/1543450727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc2c1053d0 0x7fbc2c0fe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.874+0000 7fbc2b7fe700 1 -- 172.21.15.119:0/1543450727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc2c1053d0 msgr2=0x7fbc2c0fe950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.874+0000 7fbc2b7fe700 1 --2- 172.21.15.119:0/1543450727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc2c1053d0 0x7fbc2c0fe950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.874+0000 7fbc2b7fe700 1 -- 172.21.15.119:0/1543450727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc2c100460 msgr2=0x7fbc2c0fded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.874+0000 7fbc2b7fe700 1 --2- 172.21.15.119:0/1543450727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc2c100460 0x7fbc2c0fded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.874+0000 7fbc2b7fe700 1 -- 172.21.15.119:0/1543450727 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1c009520 con 0x7fbc2c100e20 2023-12-07T10:21:54.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.875+0000 7fbc2b7fe700 1 --2- 172.21.15.119:0/1543450727 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc2c100e20 0x7fbc2c0fe410 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbc20000c00 tx=0x7fbc20005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:54.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.875+0000 7fbc297fa700 1 -- 172.21.15.119:0/1543450727 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2001d070 con 0x7fbc2c100e20 2023-12-07T10:21:54.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.875+0000 7fbc297fa700 1 -- 172.21.15.119:0/1543450727 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc20022470 con 0x7fbc2c100e20 2023-12-07T10:21:54.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.875+0000 7fbc325ff700 1 -- 172.21.15.119:0/1543450727 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc200095f0 con 0x7fbc2c100e20 2023-12-07T10:21:54.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.875+0000 7fbc297fa700 1 -- 172.21.15.119:0/1543450727 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc20017650 con 0x7fbc2c100e20 2023-12-07T10:21:54.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.875+0000 7fbc325ff700 1 -- 172.21.15.119:0/1543450727 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc2c1a6680 con 0x7fbc2c100e20 2023-12-07T10:21:54.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.879+0000 7fbc297fa700 1 -- 172.21.15.119:0/1543450727 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) v1 ==== 136717+0+0 (secure 0 0 0) 0x7fbc200225e0 con 0x7fbc2c100e20 2023-12-07T10:21:54.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.880+0000 7fbc4601fb80 1 Processor -- start 2023-12-07T10:21:54.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.880+0000 7fbc4601fb80 1 -- start start 2023-12-07T10:21:54.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.880+0000 7fbc4601fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559379ef7b40 0x559379ef7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.880+0000 7fbc4601fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559379ef8460 0x559379efbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.881+0000 7fbc4601fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559379efc150 0x559379efc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.881+0000 7fbc4601fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559379e6b7c0 con 0x559379efc150 2023-12-07T10:21:54.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.881+0000 7fbc4601fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x559379e527a0 con 0x559379ef7b40 2023-12-07T10:21:54.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.881+0000 7fbc4601fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559379e500c0 con 0x559379ef8460 2023-12-07T10:21:54.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.881+0000 7fbc30b9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559379ef7b40 0x559379ef7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.881+0000 7fbc30b9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559379ef7b40 0x559379ef7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45638/0 (socket says 172.21.15.119:45638) 2023-12-07T10:21:54.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.881+0000 7fbc30b9c700 1 -- 172.21.15.119:0/4053102670 learned_addr learned my addr 172.21.15.119:0/4053102670 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:54.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.881+0000 7fbc2b7fe700 1 --2- 172.21.15.119:0/4053102670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559379efc150 0x559379efc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.881+0000 7fbc30b9c700 1 -- 172.21.15.119:0/4053102670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559379ef8460 msgr2=0x559379efbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:54.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.881+0000 7fbc30b9c700 1 --2- 172.21.15.119:0/4053102670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559379ef8460 0x559379efbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.881+0000 7fbc30b9c700 1 -- 172.21.15.119:0/4053102670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559379efc150 msgr2=0x559379efc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:54.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.881+0000 7fbc30b9c700 1 --2- 172.21.15.119:0/4053102670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559379efc150 0x559379efc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.881+0000 7fbc30b9c700 1 -- 172.21.15.119:0/4053102670 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559379cc8630 con 0x559379ef7b40 2023-12-07T10:21:54.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.881+0000 7fbc297fa700 1 --2- 172.21.15.119:0/1543450727 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fbc140a1da0 0x7fbc140a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.881+0000 7fbc297fa700 1 -- 172.21.15.119:0/1543450727 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fbc200cf4b0 con 0x7fbc2c100e20 2023-12-07T10:21:54.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.882+0000 7fbc30b9c700 1 --2- 172.21.15.119:0/4053102670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559379ef7b40 0x559379ef7f20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbc24011670 tx=0x7fbc24009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a55e784124dabc9b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:54.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.882+0000 7fbc2bfff700 1 --2- 172.21.15.119:0/1543450727 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fbc140a1da0 0x7fbc140a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.882+0000 7fbc127fc700 1 -- 172.21.15.119:0/4053102670 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2400e040 con 0x559379ef7b40 2023-12-07T10:21:54.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.882+0000 7fbc127fc700 1 -- 172.21.15.119:0/4053102670 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc2400f040 con 0x559379ef7b40 2023-12-07T10:21:54.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.882+0000 7fbc127fc700 1 -- 172.21.15.119:0/4053102670 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc240048d0 con 0x559379ef7b40 2023-12-07T10:21:54.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.883+0000 7fbc4601fb80 1 -- 172.21.15.119:0/4053102670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559379ef7b40 msgr2=0x559379ef7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.883+0000 7fbc2bfff700 1 --2- 172.21.15.119:0/1543450727 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fbc140a1da0 0x7fbc140a4260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c009e60 tx=0x7fbc1c009900 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:54.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.883+0000 7fbc4601fb80 1 --2- 172.21.15.119:0/4053102670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559379ef7b40 0x559379ef7f20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbc24011670 tx=0x7fbc24009920 comp rx=0 tx=0).stop 2023-12-07T10:21:54.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.883+0000 7fbc4601fb80 1 -- 172.21.15.119:0/4053102670 shutdown_connections 2023-12-07T10:21:54.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.883+0000 7fbc4601fb80 1 --2- 172.21.15.119:0/4053102670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559379efc150 0x559379efc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.883+0000 7fbc4601fb80 1 --2- 172.21.15.119:0/4053102670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559379ef7b40 0x559379ef7f20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.883+0000 7fbc4601fb80 1 --2- 172.21.15.119:0/4053102670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559379ef8460 0x559379efbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.883+0000 7fbc4601fb80 1 -- 172.21.15.119:0/4053102670 >> 172.21.15.119:0/4053102670 conn(0x559379e26360 msgr2=0x559379ef2130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:54.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.883+0000 7fbc4601fb80 1 -- 172.21.15.119:0/4053102670 shutdown_connections 2023-12-07T10:21:54.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.883+0000 7fbc4601fb80 1 -- 172.21.15.119:0/4053102670 wait complete. 2023-12-07T10:21:54.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.884+0000 7fbc4601fb80 1 -- start start 2023-12-07T10:21:54.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.884+0000 7fbc4601fb80 1 Processor -- start 2023-12-07T10:21:54.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.884+0000 7fbc4601fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559379ef7b40 0x559379f00bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.885+0000 7fbc4601fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559379ef8460 0x559379f01130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.885+0000 7fbc4601fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559379efc150 0x559379f01670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.885+0000 7fbc4601fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559379e4b9b0 con 0x559379ef8460 2023-12-07T10:21:54.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.885+0000 7fbc4601fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x559379e05560 con 0x559379efc150 2023-12-07T10:21:54.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.885+0000 7fbc4601fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559379e05200 con 0x559379ef7b40 2023-12-07T10:21:54.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.885+0000 7fbc4601fb80 10 client.0 _ll_register_callbacks cb 0x559379dec940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:54.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.885+0000 7fbc2b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559379ef8460 0x559379f01130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.885+0000 7fbc2b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559379ef8460 0x559379f01130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34692/0 (socket says 172.21.15.119:34692) 2023-12-07T10:21:54.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.885+0000 7fbc2b7fe700 1 -- 172.21.15.119:0/3362233936 learned_addr learned my addr 172.21.15.119:0/3362233936 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:54.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.885+0000 7fbc2bfff700 1 --2- 172.21.15.119:0/3362233936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559379ef7b40 0x559379f00bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.885+0000 7fbc30b9c700 1 --2- 172.21.15.119:0/3362233936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559379efc150 0x559379f01670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.885+0000 7fbc2bfff700 1 -- 172.21.15.119:0/3362233936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559379efc150 msgr2=0x559379f01670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:54.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.885+0000 7fbc2bfff700 1 --2- 172.21.15.119:0/3362233936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559379efc150 0x559379f01670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.885+0000 7fbc2bfff700 1 -- 172.21.15.119:0/3362233936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559379ef8460 msgr2=0x559379f01130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:54.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.885+0000 7fbc2bfff700 1 --2- 172.21.15.119:0/3362233936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559379ef8460 0x559379f01130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.885+0000 7fbc2bfff700 1 -- 172.21.15.119:0/3362233936 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc24011350 con 0x559379ef7b40 2023-12-07T10:21:54.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.885+0000 7fbc2b7fe700 1 --2- 172.21.15.119:0/3362233936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559379ef8460 0x559379f01130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:54.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.886+0000 7fbc2bfff700 1 --2- 172.21.15.119:0/3362233936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559379ef7b40 0x559379f00bf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00f820 tx=0x7fbc1c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28d36969fc0e63d1 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:54.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.886+0000 7fbc0b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:21:54.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.886+0000 7fbc0b7fe700 1 -- 172.21.15.119:0/3362233936 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c01e630 con 0x559379ef7b40 2023-12-07T10:21:54.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.886+0000 7fbc4601fb80 1 -- 172.21.15.119:0/3362233936 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559379eeab00 con 0x559379ef7b40 2023-12-07T10:21:54.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.886+0000 7fbc0b7fe700 1 -- 172.21.15.119:0/3362233936 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fbc1c01e790 con 0x559379ef7b40 2023-12-07T10:21:54.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.886+0000 7fbc0b7fe700 1 -- 172.21.15.119:0/3362233936 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc1c01e950 con 0x559379ef7b40 2023-12-07T10:21:54.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.886+0000 7fbc4601fb80 10 client.26155 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:54.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.886+0000 7fbc4601fb80 1 -- 172.21.15.119:0/3362233936 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559379c74de0 con 0x559379ef7b40 2023-12-07T10:21:54.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.887+0000 7fbc0b7fe700 1 -- 172.21.15.119:0/3362233936 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559379eeab00 con 0x559379ef7b40 2023-12-07T10:21:54.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.887+0000 7fbc0b7fe700 1 -- 172.21.15.119:0/3362233936 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbc1c024020 con 0x559379ef7b40 2023-12-07T10:21:54.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.887+0000 7fbc4601fb80 10 client.26155 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:54.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.887+0000 7fbc4601fb80 10 client.26155 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-07T10:21:54.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.887+0000 7fbc4601fb80 20 client.26155 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:54.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.887+0000 7fbc4601fb80 1 --2- 172.21.15.119:0/3362233936 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x559379f11390 0x559379f137b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.887+0000 7fbc4601fb80 4 client.26155 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:21:54.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.887+0000 7fbc4601fb80 1 -- 172.21.15.119:0/3362233936 --> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559379ec67a0 con 0x559379f11390 2023-12-07T10:21:54.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.887+0000 7fbc2b7fe700 1 --2- 172.21.15.119:0/3362233936 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x559379f11390 0x559379f137b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.888+0000 7fbc0b7fe700 10 client.26155 ms_handle_connect on v2:172.21.15.159:6836/3106097520 2023-12-07T10:21:54.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.888+0000 7fbc2b7fe700 1 --2- 172.21.15.119:0/3362233936 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x559379f11390 0x559379f137b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=56f056fa9dfcb3af server_cookie=9681c6afea3433b in_seq=0 out_seq=0 2023-12-07T10:21:54.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.888+0000 7fbc0b7fe700 1 -- 172.21.15.119:0/3362233936 <== mds.2 v2:172.21.15.159:6836/3106097520 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559379ec67a0 con 0x559379f11390 2023-12-07T10:21:54.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.888+0000 7fbc0b7fe700 10 client.26155 handle_command_reply: tid=0 2023-12-07T10:21:54.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.889+0000 7fbc4601fb80 1 client.26155 shutdown 2023-12-07T10:21:54.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.889+0000 7fbc4601fb80 1 -- 172.21.15.119:0/3362233936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559379ef7b40 msgr2=0x559379f00bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.889+0000 7fbc4601fb80 1 --2- 172.21.15.119:0/3362233936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559379ef7b40 0x559379f00bf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00f820 tx=0x7fbc1c009260 comp rx=0 tx=0).stop 2023-12-07T10:21:54.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.889+0000 7fbc4601fb80 1 -- 172.21.15.119:0/3362233936 shutdown_connections 2023-12-07T10:21:54.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.889+0000 7fbc4601fb80 1 --2- 172.21.15.119:0/3362233936 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x559379f11390 0x559379f137b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.889+0000 7fbc4601fb80 1 --2- 172.21.15.119:0/3362233936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559379efc150 0x559379f01670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.889+0000 7fbc0b7fe700 0 client.26155 ms_handle_reset on v2:172.21.15.159:6836/3106097520 2023-12-07T10:21:54.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.889+0000 7fbc4601fb80 1 --2- 172.21.15.119:0/3362233936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559379ef7b40 0x559379f00bf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.889+0000 7fbc4601fb80 1 --2- 172.21.15.119:0/3362233936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559379ef8460 0x559379f01130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.889+0000 7fbc4601fb80 1 -- 172.21.15.119:0/3362233936 >> 172.21.15.119:0/3362233936 conn(0x559379e26360 msgr2=0x559379eecd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:54.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.889+0000 7fbc4601fb80 1 -- 172.21.15.119:0/3362233936 shutdown_connections 2023-12-07T10:21:54.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.889+0000 7fbc4601fb80 1 -- 172.21.15.119:0/3362233936 wait complete. 2023-12-07T10:21:54.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.889+0000 7fbc4601fb80 20 client.26155 trim_cache size 0 max 16384 2023-12-07T10:21:54.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.907+0000 7fbc4601fb80 1 Processor -- start 2023-12-07T10:21:54.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.907+0000 7fbc4601fb80 1 -- start start 2023-12-07T10:21:54.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.908+0000 7fbc4601fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559379ef7b40 0x559379f0ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.908+0000 7fbc4601fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559379ef8460 0x559379efa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.908+0000 7fbc4601fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559379efc150 0x559379f014b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.908+0000 7fbc4601fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559379cca920 con 0x559379ef7b40 2023-12-07T10:21:54.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.908+0000 7fbc4601fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x559379eeeaf0 con 0x559379efc150 2023-12-07T10:21:54.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.908+0000 7fbc4601fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559379f189c0 con 0x559379ef8460 2023-12-07T10:21:54.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.908+0000 7fbc2b7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559379ef8460 0x559379efa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.908+0000 7fbc2b7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559379ef8460 0x559379efa7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:40056/0 (socket says 172.21.15.119:40056) 2023-12-07T10:21:54.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.908+0000 7fbc2b7fe700 1 -- 172.21.15.119:0/418106874 learned_addr learned my addr 172.21.15.119:0/418106874 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:54.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.908+0000 7fbc2bfff700 1 --2- 172.21.15.119:0/418106874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559379ef7b40 0x559379f0ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.908+0000 7fbc30b9c700 1 --2- 172.21.15.119:0/418106874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559379efc150 0x559379f014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.908+0000 7fbc2b7fe700 1 -- 172.21.15.119:0/418106874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559379efc150 msgr2=0x559379f014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:54.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.908+0000 7fbc2b7fe700 1 --2- 172.21.15.119:0/418106874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559379efc150 0x559379f014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.908+0000 7fbc2b7fe700 1 -- 172.21.15.119:0/418106874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559379ef7b40 msgr2=0x559379f0ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:54.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.908+0000 7fbc2b7fe700 1 --2- 172.21.15.119:0/418106874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559379ef7b40 0x559379f0ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.908+0000 7fbc2b7fe700 1 -- 172.21.15.119:0/418106874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559379c5dc60 con 0x559379ef8460 2023-12-07T10:21:54.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.909+0000 7fbc2b7fe700 1 --2- 172.21.15.119:0/418106874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559379ef8460 0x559379efa7d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbc2009bb00 tx=0x7fbc2001a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90cb3523d6f12879 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:54.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.909+0000 7fbc12ffd700 1 -- 172.21.15.119:0/418106874 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2009e030 con 0x559379ef8460 2023-12-07T10:21:54.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.909+0000 7fbc12ffd700 1 -- 172.21.15.119:0/418106874 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc20098140 con 0x559379ef8460 2023-12-07T10:21:54.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.909+0000 7fbc12ffd700 1 -- 172.21.15.119:0/418106874 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2009b760 con 0x559379ef8460 2023-12-07T10:21:54.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.910+0000 7fbc4601fb80 1 -- 172.21.15.119:0/418106874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559379ef8460 msgr2=0x559379efa7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.910+0000 7fbc4601fb80 1 --2- 172.21.15.119:0/418106874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559379ef8460 0x559379efa7d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbc2009bb00 tx=0x7fbc2001a040 comp rx=0 tx=0).stop 2023-12-07T10:21:54.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.910+0000 7fbc4601fb80 1 -- 172.21.15.119:0/418106874 shutdown_connections 2023-12-07T10:21:54.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.910+0000 7fbc4601fb80 1 --2- 172.21.15.119:0/418106874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559379efc150 0x559379f014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.910+0000 7fbc4601fb80 1 --2- 172.21.15.119:0/418106874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559379ef7b40 0x559379f0ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.910+0000 7fbc4601fb80 1 --2- 172.21.15.119:0/418106874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559379ef8460 0x559379efa7d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.910+0000 7fbc4601fb80 1 -- 172.21.15.119:0/418106874 >> 172.21.15.119:0/418106874 conn(0x559379e26360 msgr2=0x559379efc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:54.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.910+0000 7fbc4601fb80 1 -- 172.21.15.119:0/418106874 shutdown_connections 2023-12-07T10:21:54.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.910+0000 7fbc4601fb80 1 -- 172.21.15.119:0/418106874 wait complete. 2023-12-07T10:21:54.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.910+0000 7fbc4601fb80 1 -- start start 2023-12-07T10:21:54.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.911+0000 7fbc4601fb80 1 Processor -- start 2023-12-07T10:21:54.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.911+0000 7fbc4601fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559379ef7b40 0x559379f08550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.911+0000 7fbc4601fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559379ef8460 0x559379f04320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.911+0000 7fbc4601fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559379efc150 0x559379f04860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.911+0000 7fbc4601fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559379f04da0 con 0x559379ef8460 2023-12-07T10:21:54.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.911+0000 7fbc4601fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x559379f04f10 con 0x559379ef7b40 2023-12-07T10:21:54.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.911+0000 7fbc4601fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559379f050d0 con 0x559379efc150 2023-12-07T10:21:54.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.911+0000 7fbc4601fb80 10 client.0 _ll_register_callbacks cb 0x559379dec940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:54.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.911+0000 7fbc2bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559379ef7b40 0x559379f08550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.911+0000 7fbc2bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559379ef7b40 0x559379f08550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45662/0 (socket says 172.21.15.119:45662) 2023-12-07T10:21:54.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.911+0000 7fbc2b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559379ef8460 0x559379f04320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.911+0000 7fbc2bfff700 1 -- 172.21.15.119:0/993644018 learned_addr learned my addr 172.21.15.119:0/993644018 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:54.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.911+0000 7fbc30b9c700 1 --2- 172.21.15.119:0/993644018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559379efc150 0x559379f04860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.912+0000 7fbc2bfff700 1 -- 172.21.15.119:0/993644018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559379efc150 msgr2=0x559379f04860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:54.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.912+0000 7fbc2bfff700 1 --2- 172.21.15.119:0/993644018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559379efc150 0x559379f04860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.912+0000 7fbc2bfff700 1 -- 172.21.15.119:0/993644018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559379ef8460 msgr2=0x559379f04320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:54.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.912+0000 7fbc2bfff700 1 --2- 172.21.15.119:0/993644018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559379ef8460 0x559379f04320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.912+0000 7fbc2bfff700 1 -- 172.21.15.119:0/993644018 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc2c1a6680 con 0x559379ef7b40 2023-12-07T10:21:54.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.912+0000 7fbc2b7fe700 1 --2- 172.21.15.119:0/993644018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559379ef8460 0x559379f04320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:54.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.912+0000 7fbc2bfff700 1 --2- 172.21.15.119:0/993644018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559379ef7b40 0x559379f08550 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c026d40 tx=0x7fbc1c009cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dfbd74109f31dd8 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:54.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.912+0000 7fbc0b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:21:54.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.912+0000 7fbc0b7fe700 1 -- 172.21.15.119:0/993644018 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c005650 con 0x559379ef7b40 2023-12-07T10:21:54.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.912+0000 7fbc0b7fe700 1 -- 172.21.15.119:0/993644018 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fbc1c026610 con 0x559379ef7b40 2023-12-07T10:21:54.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.912+0000 7fbc4601fb80 1 -- 172.21.15.119:0/993644018 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc2400f040 con 0x559379ef7b40 2023-12-07T10:21:54.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.913+0000 7fbc0b7fe700 1 -- 172.21.15.119:0/993644018 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc1c0267d0 con 0x559379ef7b40 2023-12-07T10:21:54.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.913+0000 7fbc4601fb80 10 client.26165 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:54.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.913+0000 7fbc4601fb80 1 -- 172.21.15.119:0/993644018 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559379c9b250 con 0x559379ef7b40 2023-12-07T10:21:54.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.913+0000 7fbc0b7fe700 1 -- 172.21.15.119:0/993644018 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc2400f040 con 0x559379ef7b40 2023-12-07T10:21:54.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.913+0000 7fbc0b7fe700 1 -- 172.21.15.119:0/993644018 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbc1c00f850 con 0x559379ef7b40 2023-12-07T10:21:54.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.913+0000 7fbc4601fb80 10 client.26165 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:54.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.913+0000 7fbc4601fb80 10 client.26165 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-07T10:21:54.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.913+0000 7fbc4601fb80 20 client.26165 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:54.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.913+0000 7fbc4601fb80 1 --2- 172.21.15.119:0/993644018 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x559379f67cc0 0x559379f6a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:54.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.913+0000 7fbc4601fb80 4 client.26165 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:21:54.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.913+0000 7fbc4601fb80 1 -- 172.21.15.119:0/993644018 --> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559379e11fd0 con 0x559379f67cc0 2023-12-07T10:21:54.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.914+0000 7fbc2b7fe700 1 --2- 172.21.15.119:0/993644018 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x559379f67cc0 0x559379f6a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:54.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.914+0000 7fbc0b7fe700 10 client.26165 ms_handle_connect on v2:172.21.15.159:6836/3106097520 2023-12-07T10:21:54.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.914+0000 7fbc2b7fe700 1 --2- 172.21.15.119:0/993644018 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x559379f67cc0 0x559379f6a0e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1aa65e7ed672aab3 server_cookie=d44c65f5f645c55d in_seq=0 out_seq=0 2023-12-07T10:21:54.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.914+0000 7fbc0b7fe700 1 -- 172.21.15.119:0/993644018 <== mds.2 v2:172.21.15.159:6836/3106097520 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559379e11fd0 con 0x559379f67cc0 2023-12-07T10:21:54.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.914+0000 7fbc0b7fe700 10 client.26165 handle_command_reply: tid=0 2023-12-07T10:21:54.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.914+0000 7fbc4601fb80 1 client.26165 shutdown 2023-12-07T10:21:54.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.915+0000 7fbc4601fb80 1 -- 172.21.15.119:0/993644018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559379ef7b40 msgr2=0x559379f08550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.915+0000 7fbc4601fb80 1 --2- 172.21.15.119:0/993644018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559379ef7b40 0x559379f08550 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c026d40 tx=0x7fbc1c009cb0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.915+0000 7fbc4601fb80 1 -- 172.21.15.119:0/993644018 shutdown_connections 2023-12-07T10:21:54.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.915+0000 7fbc4601fb80 1 --2- 172.21.15.119:0/993644018 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x559379f67cc0 0x559379f6a0e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.915+0000 7fbc4601fb80 1 --2- 172.21.15.119:0/993644018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559379efc150 0x559379f04860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.915+0000 7fbc4601fb80 1 --2- 172.21.15.119:0/993644018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559379ef7b40 0x559379f08550 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.915+0000 7fbc0b7fe700 0 client.26165 ms_handle_reset on v2:172.21.15.159:6836/3106097520 2023-12-07T10:21:54.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.915+0000 7fbc4601fb80 1 --2- 172.21.15.119:0/993644018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559379ef8460 0x559379f04320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.915+0000 7fbc4601fb80 1 -- 172.21.15.119:0/993644018 >> 172.21.15.119:0/993644018 conn(0x559379e26360 msgr2=0x559379efb310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:54.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.915+0000 7fbc4601fb80 1 -- 172.21.15.119:0/993644018 shutdown_connections 2023-12-07T10:21:54.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.915+0000 7fbc4601fb80 1 -- 172.21.15.119:0/993644018 wait complete. 2023-12-07T10:21:54.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.915+0000 7fbc4601fb80 20 client.26165 trim_cache size 0 max 16384 2023-12-07T10:21:54.920 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:21:54.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.916+0000 7fbc325ff700 1 -- 172.21.15.119:0/1543450727 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fbc140a1da0 msgr2=0x7fbc140a4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.916+0000 7fbc325ff700 1 --2- 172.21.15.119:0/1543450727 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fbc140a1da0 0x7fbc140a4260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c009e60 tx=0x7fbc1c009900 comp rx=0 tx=0).stop 2023-12-07T10:21:54.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.916+0000 7fbc325ff700 1 -- 172.21.15.119:0/1543450727 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc2c100e20 msgr2=0x7fbc2c0fe410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:54.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.916+0000 7fbc325ff700 1 --2- 172.21.15.119:0/1543450727 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc2c100e20 0x7fbc2c0fe410 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbc20000c00 tx=0x7fbc20005740 comp rx=0 tx=0).stop 2023-12-07T10:21:54.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.916+0000 7fbc325ff700 1 -- 172.21.15.119:0/1543450727 shutdown_connections 2023-12-07T10:21:54.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.916+0000 7fbc325ff700 1 --2- 172.21.15.119:0/1543450727 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fbc140a1da0 0x7fbc140a4260 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.916+0000 7fbc325ff700 1 --2- 172.21.15.119:0/1543450727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc2c1053d0 0x7fbc2c0fe950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.916+0000 7fbc325ff700 1 --2- 172.21.15.119:0/1543450727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc2c100460 0x7fbc2c0fded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.916+0000 7fbc325ff700 1 --2- 172.21.15.119:0/1543450727 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc2c100e20 0x7fbc2c0fe410 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:54.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.916+0000 7fbc325ff700 1 -- 172.21.15.119:0/1543450727 >> 172.21.15.119:0/1543450727 conn(0x7fbc2c077610 msgr2=0x7fbc2c068a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:54.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.916+0000 7fbc325ff700 1 -- 172.21.15.119:0/1543450727 shutdown_connections 2023-12-07T10:21:54.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:54.916+0000 7fbc325ff700 1 -- 172.21.15.119:0/1543450727 wait complete. 2023-12-07T10:21:54.938 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-07T10:21:55.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.196+0000 7f730f59e700 1 -- 172.21.15.119:0/4003201725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f731010b9e0 msgr2=0x7f731010de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:55.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.196+0000 7f730f59e700 1 --2- 172.21.15.119:0/4003201725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f731010b9e0 0x7f731010de80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7308009f90 tx=0x7f730800b330 comp rx=0 tx=0).stop 2023-12-07T10:21:55.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.196+0000 7f730f59e700 1 -- 172.21.15.119:0/4003201725 shutdown_connections 2023-12-07T10:21:55.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.196+0000 7f730f59e700 1 --2- 172.21.15.119:0/4003201725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f731010b9e0 0x7f731010de80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.196+0000 7f730f59e700 1 --2- 172.21.15.119:0/4003201725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7310102610 0x7f73101029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.196+0000 7f730f59e700 1 --2- 172.21.15.119:0/4003201725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7310102f30 0x7f731010b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.196+0000 7f730f59e700 1 -- 172.21.15.119:0/4003201725 >> 172.21.15.119:0/4003201725 conn(0x7f73100fbdd0 msgr2=0x7f73100fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:55.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.196+0000 7f730f59e700 1 -- 172.21.15.119:0/4003201725 shutdown_connections 2023-12-07T10:21:55.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.196+0000 7f730f59e700 1 -- 172.21.15.119:0/4003201725 wait complete. 2023-12-07T10:21:55.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.197+0000 7f730f59e700 1 Processor -- start 2023-12-07T10:21:55.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.197+0000 7f730f59e700 1 -- start start 2023-12-07T10:21:55.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.198+0000 7f730f59e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7310102610 0x7f7310100260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.198+0000 7f730f59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7310102f30 0x7f73101007a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.198+0000 7f730f59e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f731010b9e0 0x7f7310100ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.198+0000 7f730f59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73101a5da0 con 0x7f7310102f30 2023-12-07T10:21:55.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.198+0000 7f730f59e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73101a5f10 con 0x7f7310102610 2023-12-07T10:21:55.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.198+0000 7f730f59e700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73101a60d0 con 0x7f731010b9e0 2023-12-07T10:21:55.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.198+0000 7f730e59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7310102610 0x7f7310100260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.198+0000 7f730e59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7310102610 0x7f7310100260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45680/0 (socket says 172.21.15.119:45680) 2023-12-07T10:21:55.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.198+0000 7f730e59c700 1 -- 172.21.15.119:0/1972848689 learned_addr learned my addr 172.21.15.119:0/1972848689 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:55.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.198+0000 7f730dd9b700 1 --2- 172.21.15.119:0/1972848689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7310102f30 0x7f73101007a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.198+0000 7f730ed9d700 1 --2- 172.21.15.119:0/1972848689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f731010b9e0 0x7f7310100ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.199+0000 7f730e59c700 1 -- 172.21.15.119:0/1972848689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f731010b9e0 msgr2=0x7f7310100ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:55.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.199+0000 7f730e59c700 1 --2- 172.21.15.119:0/1972848689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f731010b9e0 0x7f7310100ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.199+0000 7f730e59c700 1 -- 172.21.15.119:0/1972848689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7310102f30 msgr2=0x7f73101007a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:55.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.199+0000 7f730e59c700 1 --2- 172.21.15.119:0/1972848689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7310102f30 0x7f73101007a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.199+0000 7f730e59c700 1 -- 172.21.15.119:0/1972848689 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73000095f0 con 0x7f7310102610 2023-12-07T10:21:55.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.199+0000 7f730e59c700 1 --2- 172.21.15.119:0/1972848689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7310102610 0x7f7310100260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f730000ec10 tx=0x7f730000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:55.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.199+0000 7f72ff7fe700 1 -- 172.21.15.119:0/1972848689 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730000cc90 con 0x7f7310102610 2023-12-07T10:21:55.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.199+0000 7f730f59e700 1 -- 172.21.15.119:0/1972848689 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7308009c40 con 0x7f7310102610 2023-12-07T10:21:55.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.199+0000 7f72ff7fe700 1 -- 172.21.15.119:0/1972848689 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73000044e0 con 0x7f7310102610 2023-12-07T10:21:55.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.199+0000 7f72ff7fe700 1 -- 172.21.15.119:0/1972848689 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7300010430 con 0x7f7310102610 2023-12-07T10:21:55.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.199+0000 7f730f59e700 1 -- 172.21.15.119:0/1972848689 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73101a6780 con 0x7f7310102610 2023-12-07T10:21:55.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.203+0000 7f72ff7fe700 1 -- 172.21.15.119:0/1972848689 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 25) v1 ==== 136717+0+0 (secure 0 0 0) 0x7f7300003fb0 con 0x7f7310102610 2023-12-07T10:21:55.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.204+0000 7f732838cb80 1 Processor -- start 2023-12-07T10:21:55.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.205+0000 7f732838cb80 1 -- start start 2023-12-07T10:21:55.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.205+0000 7f72ff7fe700 1 --2- 172.21.15.119:0/1972848689 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f72f80a1ea0 0x7f72f80a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.205+0000 7f732838cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557556e79e0 0x5557556e7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.205+0000 7f72ff7fe700 1 -- 172.21.15.119:0/1972848689 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f7300014070 con 0x7f7310102610 2023-12-07T10:21:55.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.205+0000 7f732838cb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557556e8300 0x5557556ebda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.205+0000 7f732838cb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557556ec2e0 0x5557556ec740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.205+0000 7f732838cb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55575565b7c0 con 0x5557556ec2e0 2023-12-07T10:21:55.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.205+0000 7f732838cb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557556427a0 con 0x5557556e8300 2023-12-07T10:21:55.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.205+0000 7f732838cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557556400c0 con 0x5557556e79e0 2023-12-07T10:21:55.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.205+0000 7f730ed9d700 1 --2- 172.21.15.119:0/1972848689 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f72f80a1ea0 0x7f72f80a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.206+0000 7f730e59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557556e79e0 0x5557556e7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.206+0000 7f730dd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557556e8300 0x5557556ebda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.206+0000 7f730e59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557556e79e0 0x5557556e7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:40098/0 (socket says 172.21.15.119:40098) 2023-12-07T10:21:55.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.206+0000 7f730e59c700 1 -- 172.21.15.119:0/3029068841 learned_addr learned my addr 172.21.15.119:0/3029068841 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:55.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.206+0000 7f730ed9d700 1 --2- 172.21.15.119:0/3029068841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557556ec2e0 0x5557556ec740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.206+0000 7f730e59c700 1 -- 172.21.15.119:0/3029068841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557556e8300 msgr2=0x5557556ebda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:55.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.206+0000 7f730e59c700 1 --2- 172.21.15.119:0/3029068841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557556e8300 0x5557556ebda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.206+0000 7f730e59c700 1 -- 172.21.15.119:0/3029068841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557556ec2e0 msgr2=0x5557556ec740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:55.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.206+0000 7f730e59c700 1 --2- 172.21.15.119:0/3029068841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557556ec2e0 0x5557556ec740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.206+0000 7f730e59c700 1 -- 172.21.15.119:0/3029068841 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557554b8630 con 0x5557556e79e0 2023-12-07T10:21:55.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.206+0000 7f730dd9b700 1 --2- 172.21.15.119:0/3029068841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557556e8300 0x5557556ebda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:55.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.206+0000 7f730e59c700 1 --2- 172.21.15.119:0/3029068841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557556e79e0 0x5557556e7dc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f73000a2b10 tx=0x7f730009ab50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c82f0ad3f026fb1c server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:55.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.206+0000 7f72fcff9700 1 -- 172.21.15.119:0/3029068841 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730009b3b0 con 0x5557556e79e0 2023-12-07T10:21:55.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.206+0000 7f72fcff9700 1 -- 172.21.15.119:0/3029068841 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f730009acb0 con 0x5557556e79e0 2023-12-07T10:21:55.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.206+0000 7f72fcff9700 1 -- 172.21.15.119:0/3029068841 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73000ce160 con 0x5557556e79e0 2023-12-07T10:21:55.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.207+0000 7f732838cb80 1 -- 172.21.15.119:0/3029068841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557556e79e0 msgr2=0x5557556e7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:55.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.207+0000 7f730ed9d700 1 --2- 172.21.15.119:0/1972848689 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f72f80a1ea0 0x7f72f80a4360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7308009550 tx=0x7f7308009700 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:55.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.207+0000 7f732838cb80 1 --2- 172.21.15.119:0/3029068841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557556e79e0 0x5557556e7dc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f73000a2b10 tx=0x7f730009ab50 comp rx=0 tx=0).stop 2023-12-07T10:21:55.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.207+0000 7f732838cb80 1 -- 172.21.15.119:0/3029068841 shutdown_connections 2023-12-07T10:21:55.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.207+0000 7f732838cb80 1 --2- 172.21.15.119:0/3029068841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557556ec2e0 0x5557556ec740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.207+0000 7f732838cb80 1 --2- 172.21.15.119:0/3029068841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557556e79e0 0x5557556e7dc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.207+0000 7f732838cb80 1 --2- 172.21.15.119:0/3029068841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557556e8300 0x5557556ebda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.207+0000 7f732838cb80 1 -- 172.21.15.119:0/3029068841 >> 172.21.15.119:0/3029068841 conn(0x555755616360 msgr2=0x5557556e2170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:55.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.207+0000 7f732838cb80 1 -- 172.21.15.119:0/3029068841 shutdown_connections 2023-12-07T10:21:55.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.208+0000 7f732838cb80 1 -- 172.21.15.119:0/3029068841 wait complete. 2023-12-07T10:21:55.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.208+0000 7f732838cb80 1 -- start start 2023-12-07T10:21:55.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.208+0000 7f732838cb80 1 Processor -- start 2023-12-07T10:21:55.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.209+0000 7f732838cb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557556e79e0 0x5557556f3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.209+0000 7f732838cb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557556e8300 0x5557556f0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.209+0000 7f732838cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557556ec2e0 0x5557556f14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.209+0000 7f732838cb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55575563b9b0 con 0x5557556e79e0 2023-12-07T10:21:55.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.209+0000 7f732838cb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557555f5560 con 0x5557556e8300 2023-12-07T10:21:55.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.209+0000 7f732838cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557555f5200 con 0x5557556ec2e0 2023-12-07T10:21:55.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.209+0000 7f732838cb80 10 client.0 _ll_register_callbacks cb 0x5557555dc940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:55.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.209+0000 7f730dd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557556e8300 0x5557556f0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.210+0000 7f730dd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557556e8300 0x5557556f0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45692/0 (socket says 172.21.15.119:45692) 2023-12-07T10:21:55.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.210+0000 7f730dd9b700 1 -- 172.21.15.119:0/473865990 learned_addr learned my addr 172.21.15.119:0/473865990 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:55.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.210+0000 7f730ed9d700 1 --2- 172.21.15.119:0/473865990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557556ec2e0 0x5557556f14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.210+0000 7f730e59c700 1 --2- 172.21.15.119:0/473865990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557556e79e0 0x5557556f3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.210+0000 7f730dd9b700 1 -- 172.21.15.119:0/473865990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557556ec2e0 msgr2=0x5557556f14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:55.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.210+0000 7f730dd9b700 1 --2- 172.21.15.119:0/473865990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557556ec2e0 0x5557556f14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.210+0000 7f730dd9b700 1 -- 172.21.15.119:0/473865990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557556e79e0 msgr2=0x5557556f3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:55.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.210+0000 7f730dd9b700 1 --2- 172.21.15.119:0/473865990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557556e79e0 0x5557556f3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.210+0000 7f730dd9b700 1 -- 172.21.15.119:0/473865990 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73040098e0 con 0x5557556e8300 2023-12-07T10:21:55.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.210+0000 7f730dd9b700 1 --2- 172.21.15.119:0/473865990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557556e8300 0x5557556f0d80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7304011870 tx=0x7f7304011b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be9eed3aed5894c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:55.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.210+0000 7f72f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:21:55.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.211+0000 7f72f5ffb700 1 -- 172.21.15.119:0/473865990 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7304016070 con 0x5557556e8300 2023-12-07T10:21:55.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.211+0000 7f732838cb80 1 -- 172.21.15.119:0/473865990 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5557556da9a0 con 0x5557556e8300 2023-12-07T10:21:55.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.211+0000 7f72f5ffb700 1 -- 172.21.15.119:0/473865990 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f7304004300 con 0x5557556e8300 2023-12-07T10:21:55.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.211+0000 7f72f5ffb700 1 -- 172.21.15.119:0/473865990 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73040044c0 con 0x5557556e8300 2023-12-07T10:21:55.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.211+0000 7f732838cb80 10 client.26177 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:55.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.211+0000 7f732838cb80 1 -- 172.21.15.119:0/473865990 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f73101a6780 con 0x5557556e8300 2023-12-07T10:21:55.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.211+0000 7f72f5ffb700 1 -- 172.21.15.119:0/473865990 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5557556da9a0 con 0x5557556e8300 2023-12-07T10:21:55.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.211+0000 7f72f5ffb700 1 -- 172.21.15.119:0/473865990 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f730400f940 con 0x5557556e8300 2023-12-07T10:21:55.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.211+0000 7f732838cb80 10 client.26177 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:55.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.211+0000 7f732838cb80 10 client.26177 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2023-12-07T10:21:55.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.211+0000 7f732838cb80 20 client.26177 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:55.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.212+0000 7f732838cb80 1 --2- 172.21.15.119:0/473865990 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x555755701560 0x555755703980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.212+0000 7f732838cb80 4 client.26177 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:21:55.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.212+0000 7f732838cb80 1 -- 172.21.15.119:0/473865990 --> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5557556b67a0 con 0x555755701560 2023-12-07T10:21:55.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.212+0000 7f730e59c700 1 --2- 172.21.15.119:0/473865990 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x555755701560 0x555755703980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.212+0000 7f730e59c700 1 --2- 172.21.15.119:0/473865990 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x555755701560 0x555755703980 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dce8463e3719bca7 server_cookie=e0dc33d2ae2f6b22 in_seq=0 out_seq=0 2023-12-07T10:21:55.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.212+0000 7f72f5ffb700 10 client.26177 ms_handle_connect on v2:172.21.15.119:6838/1920762602 2023-12-07T10:21:55.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.213+0000 7f72f5ffb700 1 -- 172.21.15.119:0/473865990 <== mds.1 v2:172.21.15.119:6838/1920762602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5557556b67a0 con 0x555755701560 2023-12-07T10:21:55.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.213+0000 7f72f5ffb700 10 client.26177 handle_command_reply: tid=0 2023-12-07T10:21:55.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.213+0000 7f732838cb80 1 client.26177 shutdown 2023-12-07T10:21:55.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.213+0000 7f732838cb80 1 -- 172.21.15.119:0/473865990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557556e8300 msgr2=0x5557556f0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:55.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.213+0000 7f732838cb80 1 --2- 172.21.15.119:0/473865990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557556e8300 0x5557556f0d80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7304011870 tx=0x7f7304011b80 comp rx=0 tx=0).stop 2023-12-07T10:21:55.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.214+0000 7f732838cb80 1 -- 172.21.15.119:0/473865990 shutdown_connections 2023-12-07T10:21:55.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.214+0000 7f732838cb80 1 --2- 172.21.15.119:0/473865990 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x555755701560 0x555755703980 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.214+0000 7f732838cb80 1 --2- 172.21.15.119:0/473865990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557556ec2e0 0x5557556f14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.214+0000 7f72f5ffb700 0 client.26177 ms_handle_reset on v2:172.21.15.119:6838/1920762602 2023-12-07T10:21:55.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.214+0000 7f732838cb80 1 --2- 172.21.15.119:0/473865990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557556e79e0 0x5557556f3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.214+0000 7f732838cb80 1 --2- 172.21.15.119:0/473865990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557556e8300 0x5557556f0d80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.214+0000 7f732838cb80 1 -- 172.21.15.119:0/473865990 >> 172.21.15.119:0/473865990 conn(0x555755616360 msgr2=0x5557556e67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:55.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.214+0000 7f732838cb80 1 -- 172.21.15.119:0/473865990 shutdown_connections 2023-12-07T10:21:55.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.214+0000 7f732838cb80 1 -- 172.21.15.119:0/473865990 wait complete. 2023-12-07T10:21:55.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.214+0000 7f732838cb80 20 client.26177 trim_cache size 0 max 16384 2023-12-07T10:21:55.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:54 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1997861941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:21:55.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.229+0000 7f732838cb80 1 Processor -- start 2023-12-07T10:21:55.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.230+0000 7f732838cb80 1 -- start start 2023-12-07T10:21:55.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.230+0000 7f732838cb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557556e79e0 0x5557556f14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.230+0000 7f732838cb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557556e8300 0x5557556ec6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.230+0000 7f732838cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557556ec2e0 0x5557556e67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.230+0000 7f732838cb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557554ba920 con 0x5557556e8300 2023-12-07T10:21:55.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.230+0000 7f732838cb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557556de5f0 con 0x5557556e79e0 2023-12-07T10:21:55.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.230+0000 7f732838cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557556fd370 con 0x5557556ec2e0 2023-12-07T10:21:55.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.230+0000 7f730e59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557556e79e0 0x5557556f14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.230+0000 7f730e59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557556e79e0 0x5557556f14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45698/0 (socket says 172.21.15.119:45698) 2023-12-07T10:21:55.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.230+0000 7f730e59c700 1 -- 172.21.15.119:0/3162693412 learned_addr learned my addr 172.21.15.119:0/3162693412 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:55.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.230+0000 7f730dd9b700 1 --2- 172.21.15.119:0/3162693412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557556e8300 0x5557556ec6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.230+0000 7f730ed9d700 1 --2- 172.21.15.119:0/3162693412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557556ec2e0 0x5557556e67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.231+0000 7f730e59c700 1 -- 172.21.15.119:0/3162693412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557556ec2e0 msgr2=0x5557556e67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:55.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.231+0000 7f730e59c700 1 --2- 172.21.15.119:0/3162693412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557556ec2e0 0x5557556e67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.231+0000 7f730e59c700 1 -- 172.21.15.119:0/3162693412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557556e8300 msgr2=0x5557556ec6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:55.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.231+0000 7f730e59c700 1 --2- 172.21.15.119:0/3162693412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557556e8300 0x5557556ec6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.231+0000 7f730e59c700 1 -- 172.21.15.119:0/3162693412 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555755464de0 con 0x5557556e79e0 2023-12-07T10:21:55.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.231+0000 7f730ed9d700 1 --2- 172.21.15.119:0/3162693412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557556ec2e0 0x5557556e67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:21:55.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.231+0000 7f730e59c700 1 --2- 172.21.15.119:0/3162693412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557556e79e0 0x5557556f14c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7300097720 tx=0x7f7300003a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=594003908e555b6a server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:55.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.231+0000 7f72fd7fa700 1 -- 172.21.15.119:0/3162693412 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730009ee80 con 0x5557556e79e0 2023-12-07T10:21:55.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.231+0000 7f72fd7fa700 1 -- 172.21.15.119:0/3162693412 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73000a3040 con 0x5557556e79e0 2023-12-07T10:21:55.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.231+0000 7f72fd7fa700 1 -- 172.21.15.119:0/3162693412 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730009b170 con 0x5557556e79e0 2023-12-07T10:21:55.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.232+0000 7f732838cb80 1 -- 172.21.15.119:0/3162693412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557556e79e0 msgr2=0x5557556f14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:55.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.232+0000 7f732838cb80 1 --2- 172.21.15.119:0/3162693412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557556e79e0 0x5557556f14c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7300097720 tx=0x7f7300003a10 comp rx=0 tx=0).stop 2023-12-07T10:21:55.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.232+0000 7f732838cb80 1 -- 172.21.15.119:0/3162693412 shutdown_connections 2023-12-07T10:21:55.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.232+0000 7f732838cb80 1 --2- 172.21.15.119:0/3162693412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557556ec2e0 0x5557556e67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.232+0000 7f732838cb80 1 --2- 172.21.15.119:0/3162693412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557556e79e0 0x5557556f14c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.232+0000 7f732838cb80 1 --2- 172.21.15.119:0/3162693412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557556e8300 0x5557556ec6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.232+0000 7f732838cb80 1 -- 172.21.15.119:0/3162693412 >> 172.21.15.119:0/3162693412 conn(0x555755616360 msgr2=0x5557556e3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:55.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.232+0000 7f732838cb80 1 -- 172.21.15.119:0/3162693412 shutdown_connections 2023-12-07T10:21:55.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.232+0000 7f732838cb80 1 -- 172.21.15.119:0/3162693412 wait complete. 2023-12-07T10:21:55.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.233+0000 7f732838cb80 1 -- start start 2023-12-07T10:21:55.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.233+0000 7f732838cb80 1 Processor -- start 2023-12-07T10:21:55.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.234+0000 7f732838cb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557556e79e0 0x5557556f88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.234+0000 7f732838cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557556e8300 0x5557556f64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.234+0000 7f732838cb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557556ec2e0 0x5557556f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.234+0000 7f732838cb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557556f7110 con 0x5557556ec2e0 2023-12-07T10:21:55.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.234+0000 7f732838cb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557556f7280 con 0x5557556e79e0 2023-12-07T10:21:55.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.234+0000 7f732838cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555755756a60 con 0x5557556e8300 2023-12-07T10:21:55.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.234+0000 7f732838cb80 10 client.0 _ll_register_callbacks cb 0x5557555dc940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:55.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.234+0000 7f730e59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557556e79e0 0x5557556f88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.234+0000 7f730dd9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557556e8300 0x5557556f64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.234+0000 7f730dd9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557556e8300 0x5557556f64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:40120/0 (socket says 172.21.15.119:40120) 2023-12-07T10:21:55.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.234+0000 7f730dd9b700 1 -- 172.21.15.119:0/3293222847 learned_addr learned my addr 172.21.15.119:0/3293222847 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:55.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.234+0000 7f730ed9d700 1 --2- 172.21.15.119:0/3293222847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557556ec2e0 0x5557556f6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.234+0000 7f730e59c700 1 -- 172.21.15.119:0/3293222847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557556e8300 msgr2=0x5557556f64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:55.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.234+0000 7f730e59c700 1 --2- 172.21.15.119:0/3293222847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557556e8300 0x5557556f64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.234+0000 7f730e59c700 1 -- 172.21.15.119:0/3293222847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557556ec2e0 msgr2=0x5557556f6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:55.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.234+0000 7f730e59c700 1 --2- 172.21.15.119:0/3293222847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557556ec2e0 0x5557556f6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.234+0000 7f730e59c700 1 -- 172.21.15.119:0/3293222847 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557554b8630 con 0x5557556e79e0 2023-12-07T10:21:55.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.235+0000 7f730e59c700 1 --2- 172.21.15.119:0/3293222847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557556e79e0 0x5557556f88b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f730009e040 tx=0x7f730009ef30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7346e7bc9b16307a server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:55.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.235+0000 7f72f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:21:55.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.235+0000 7f72f5ffb700 1 -- 172.21.15.119:0/3293222847 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7300004320 con 0x5557556e79e0 2023-12-07T10:21:55.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.235+0000 7f72f5ffb700 1 -- 172.21.15.119:0/3293222847 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f730009a1a0 con 0x5557556e79e0 2023-12-07T10:21:55.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.235+0000 7f732838cb80 1 -- 172.21.15.119:0/3293222847 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f730009acb0 con 0x5557556e79e0 2023-12-07T10:21:55.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.235+0000 7f72f5ffb700 1 -- 172.21.15.119:0/3293222847 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f730009a360 con 0x5557556e79e0 2023-12-07T10:21:55.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.235+0000 7f732838cb80 10 client.26183 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:55.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.235+0000 7f732838cb80 1 -- 172.21.15.119:0/3293222847 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55575544dc60 con 0x5557556e79e0 2023-12-07T10:21:55.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.235+0000 7f72f5ffb700 1 -- 172.21.15.119:0/3293222847 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f730009acb0 con 0x5557556e79e0 2023-12-07T10:21:55.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.235+0000 7f72f5ffb700 1 -- 172.21.15.119:0/3293222847 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f73000b3230 con 0x5557556e79e0 2023-12-07T10:21:55.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.235+0000 7f732838cb80 10 client.26183 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:55.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.235+0000 7f732838cb80 10 client.26183 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2023-12-07T10:21:55.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.235+0000 7f732838cb80 20 client.26183 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:55.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.235+0000 7f732838cb80 1 --2- 172.21.15.119:0/3293222847 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x555755757d10 0x55575575a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.236+0000 7f732838cb80 4 client.26183 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:21:55.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.236+0000 7f732838cb80 1 -- 172.21.15.119:0/3293222847 --> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555755601fd0 con 0x555755757d10 2023-12-07T10:21:55.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.236+0000 7f730dd9b700 1 --2- 172.21.15.119:0/3293222847 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x555755757d10 0x55575575a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.236+0000 7f730dd9b700 1 --2- 172.21.15.119:0/3293222847 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x555755757d10 0x55575575a100 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=642f28f41798618a server_cookie=6cce3724f9d5d94 in_seq=0 out_seq=0 2023-12-07T10:21:55.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.236+0000 7f72f5ffb700 10 client.26183 ms_handle_connect on v2:172.21.15.119:6838/1920762602 2023-12-07T10:21:55.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.236+0000 7f72f5ffb700 1 -- 172.21.15.119:0/3293222847 <== mds.1 v2:172.21.15.119:6838/1920762602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555755601fd0 con 0x555755757d10 2023-12-07T10:21:55.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.236+0000 7f72f5ffb700 10 client.26183 handle_command_reply: tid=0 2023-12-07T10:21:55.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.236+0000 7f732838cb80 1 client.26183 shutdown 2023-12-07T10:21:55.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.237+0000 7f732838cb80 1 -- 172.21.15.119:0/3293222847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557556e79e0 msgr2=0x5557556f88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:55.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.237+0000 7f732838cb80 1 --2- 172.21.15.119:0/3293222847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557556e79e0 0x5557556f88b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f730009e040 tx=0x7f730009ef30 comp rx=0 tx=0).stop 2023-12-07T10:21:55.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.237+0000 7f732838cb80 1 -- 172.21.15.119:0/3293222847 shutdown_connections 2023-12-07T10:21:55.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.237+0000 7f732838cb80 1 --2- 172.21.15.119:0/3293222847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557556ec2e0 0x5557556f6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.237+0000 7f732838cb80 1 --2- 172.21.15.119:0/3293222847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5557556e79e0 0x5557556f88b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.237+0000 7f732838cb80 1 --2- 172.21.15.119:0/3293222847 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x555755757d10 0x55575575a100 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.237+0000 7f732838cb80 1 --2- 172.21.15.119:0/3293222847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557556e8300 0x5557556f64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.237+0000 7f72f5ffb700 0 client.26183 ms_handle_reset on v2:172.21.15.119:6838/1920762602 2023-12-07T10:21:55.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.237+0000 7f732838cb80 1 -- 172.21.15.119:0/3293222847 >> 172.21.15.119:0/3293222847 conn(0x555755616360 msgr2=0x5557557082e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:55.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.237+0000 7f732838cb80 1 -- 172.21.15.119:0/3293222847 shutdown_connections 2023-12-07T10:21:55.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.237+0000 7f732838cb80 1 -- 172.21.15.119:0/3293222847 wait complete. 2023-12-07T10:21:55.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.237+0000 7f732838cb80 20 client.26183 trim_cache size 0 max 16384 2023-12-07T10:21:55.243 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:21:55.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.238+0000 7f730f59e700 1 -- 172.21.15.119:0/1972848689 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f72f80a1ea0 msgr2=0x7f72f80a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:55.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.238+0000 7f730f59e700 1 --2- 172.21.15.119:0/1972848689 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f72f80a1ea0 0x7f72f80a4360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7308009550 tx=0x7f7308009700 comp rx=0 tx=0).stop 2023-12-07T10:21:55.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.238+0000 7f730f59e700 1 -- 172.21.15.119:0/1972848689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7310102610 msgr2=0x7f7310100260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:55.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.238+0000 7f730f59e700 1 --2- 172.21.15.119:0/1972848689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7310102610 0x7f7310100260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f730000ec10 tx=0x7f730000ef20 comp rx=0 tx=0).stop 2023-12-07T10:21:55.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.238+0000 7f730f59e700 1 -- 172.21.15.119:0/1972848689 shutdown_connections 2023-12-07T10:21:55.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.238+0000 7f730f59e700 1 --2- 172.21.15.119:0/1972848689 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f72f80a1ea0 0x7f72f80a4360 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.238+0000 7f730f59e700 1 --2- 172.21.15.119:0/1972848689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f731010b9e0 0x7f7310100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.238+0000 7f730f59e700 1 --2- 172.21.15.119:0/1972848689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7310102610 0x7f7310100260 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.238+0000 7f730f59e700 1 --2- 172.21.15.119:0/1972848689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7310102f30 0x7f73101007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.238+0000 7f730f59e700 1 -- 172.21.15.119:0/1972848689 >> 172.21.15.119:0/1972848689 conn(0x7f73100fbdd0 msgr2=0x7f73100fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:55.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.238+0000 7f730f59e700 1 -- 172.21.15.119:0/1972848689 shutdown_connections 2023-12-07T10:21:55.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.238+0000 7f730f59e700 1 -- 172.21.15.119:0/1972848689 wait complete. 2023-12-07T10:21:55.258 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-07T10:21:55.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:54 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1997861941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:21:55.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:54 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1997861941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:21:55.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.478+0000 7f50231ff700 1 -- 172.21.15.119:0/3890899160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f501c101560 msgr2=0x7f501c1019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:55.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.478+0000 7f50231ff700 1 --2- 172.21.15.119:0/3890899160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f501c101560 0x7f501c1019e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5008009990 tx=0x7f5008009ca0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.478+0000 7f50231ff700 1 -- 172.21.15.119:0/3890899160 shutdown_connections 2023-12-07T10:21:55.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.478+0000 7f50231ff700 1 --2- 172.21.15.119:0/3890899160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f501c101f20 0x7f501c10e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.478+0000 7f50231ff700 1 --2- 172.21.15.119:0/3890899160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f501c107620 0x7f501c107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.478+0000 7f50231ff700 1 --2- 172.21.15.119:0/3890899160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f501c101560 0x7f501c1019e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.478+0000 7f50231ff700 1 -- 172.21.15.119:0/3890899160 >> 172.21.15.119:0/3890899160 conn(0x7f501c0fcfd0 msgr2=0x7f501c0ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:55.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.478+0000 7f50231ff700 1 -- 172.21.15.119:0/3890899160 shutdown_connections 2023-12-07T10:21:55.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.478+0000 7f50231ff700 1 -- 172.21.15.119:0/3890899160 wait complete. 2023-12-07T10:21:55.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.479+0000 7f50231ff700 1 Processor -- start 2023-12-07T10:21:55.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.480+0000 7f50231ff700 1 -- start start 2023-12-07T10:21:55.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.480+0000 7f50231ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f501c101560 0x7f501c19f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.480+0000 7f50231ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f501c101f20 0x7f501c19fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.480+0000 7f50231ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f501c107620 0x7f501c1999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.480+0000 7f50231ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501c19a030 con 0x7f501c101f20 2023-12-07T10:21:55.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.480+0000 7f50231ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501c19a1a0 con 0x7f501c101560 2023-12-07T10:21:55.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.480+0000 7f50231ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501c19a360 con 0x7f501c107620 2023-12-07T10:21:55.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.480+0000 7f502179c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f501c107620 0x7f501c1999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.480+0000 7f501bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f501c101f20 0x7f501c19fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.480+0000 7f502179c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f501c107620 0x7f501c1999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:40132/0 (socket says 172.21.15.119:40132) 2023-12-07T10:21:55.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.480+0000 7f502179c700 1 -- 172.21.15.119:0/2332391253 learned_addr learned my addr 172.21.15.119:0/2332391253 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:55.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.480+0000 7f5020f9b700 1 --2- 172.21.15.119:0/2332391253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f501c101560 0x7f501c19f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.481+0000 7f501bfff700 1 -- 172.21.15.119:0/2332391253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f501c107620 msgr2=0x7f501c1999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:55.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.481+0000 7f501bfff700 1 --2- 172.21.15.119:0/2332391253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f501c107620 0x7f501c1999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.481+0000 7f501bfff700 1 -- 172.21.15.119:0/2332391253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f501c101560 msgr2=0x7f501c19f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:55.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.481+0000 7f501bfff700 1 --2- 172.21.15.119:0/2332391253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f501c101560 0x7f501c19f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.481+0000 7f501bfff700 1 -- 172.21.15.119:0/2332391253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50080095f0 con 0x7f501c101f20 2023-12-07T10:21:55.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.481+0000 7f502179c700 1 --2- 172.21.15.119:0/2332391253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f501c107620 0x7f501c1999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:55.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.481+0000 7f5020f9b700 1 --2- 172.21.15.119:0/2332391253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f501c101560 0x7f501c19f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:55.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.481+0000 7f501bfff700 1 --2- 172.21.15.119:0/2332391253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f501c101f20 0x7f501c19fde0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5008000c00 tx=0x7f5008004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:55.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.481+0000 7f5019ffb700 1 -- 172.21.15.119:0/2332391253 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500801d070 con 0x7f501c101f20 2023-12-07T10:21:55.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.481+0000 7f5019ffb700 1 -- 172.21.15.119:0/2332391253 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5008021410 con 0x7f501c101f20 2023-12-07T10:21:55.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.482+0000 7f5019ffb700 1 -- 172.21.15.119:0/2332391253 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5008017650 con 0x7f501c101f20 2023-12-07T10:21:55.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.482+0000 7f50231ff700 1 -- 172.21.15.119:0/2332391253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f501c19a5e0 con 0x7f501c101f20 2023-12-07T10:21:55.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.482+0000 7f50231ff700 1 -- 172.21.15.119:0/2332391253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f501c108320 con 0x7f501c101f20 2023-12-07T10:21:55.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.486+0000 7f5019ffb700 1 -- 172.21.15.119:0/2332391253 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 25) v1 ==== 136717+0+0 (secure 0 0 0) 0x7f5008021580 con 0x7f501c101f20 2023-12-07T10:21:55.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.487+0000 7f5036b1cb80 1 Processor -- start 2023-12-07T10:21:55.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.488+0000 7f5036b1cb80 1 -- start start 2023-12-07T10:21:55.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.488+0000 7f5036b1cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3014dfb40 0x55a3014dff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.488+0000 7f5036b1cb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3014e0460 0x55a3014e3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.488+0000 7f5036b1cb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3014e4150 0x55a3014e4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.488+0000 7f5036b1cb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3014537c0 con 0x55a3014e4150 2023-12-07T10:21:55.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.488+0000 7f5036b1cb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a30143a7a0 con 0x55a3014e0460 2023-12-07T10:21:55.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.488+0000 7f5036b1cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3014380c0 con 0x55a3014dfb40 2023-12-07T10:21:55.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.488+0000 7f502179c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3014dfb40 0x55a3014dff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.488+0000 7f5020f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3014e0460 0x55a3014e3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.488+0000 7f5019ffb700 1 --2- 172.21.15.119:0/2332391253 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f500c0a1e20 0x7f500c0a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.488+0000 7f502179c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3014dfb40 0x55a3014dff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:40138/0 (socket says 172.21.15.119:40138) 2023-12-07T10:21:55.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.488+0000 7f502179c700 1 -- 172.21.15.119:0/2529487037 learned_addr learned my addr 172.21.15.119:0/2529487037 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:55.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.488+0000 7f501bfff700 1 --2- 172.21.15.119:0/2529487037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3014e4150 0x55a3014e4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.489+0000 7f5019ffb700 1 -- 172.21.15.119:0/2332391253 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f50080cf110 con 0x7f501c101f20 2023-12-07T10:21:55.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.489+0000 7f502179c700 1 --2- 172.21.15.119:0/2332391253 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f500c0a1e20 0x7f500c0a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.489+0000 7f5020f9b700 1 -- 172.21.15.119:0/2529487037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3014dfb40 msgr2=0x55a3014dff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:55.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.489+0000 7f5020f9b700 1 --2- 172.21.15.119:0/2529487037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3014dfb40 0x55a3014dff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.489+0000 7f5020f9b700 1 -- 172.21.15.119:0/2529487037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3014e4150 msgr2=0x55a3014e4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:55.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.489+0000 7f5020f9b700 1 --2- 172.21.15.119:0/2529487037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3014e4150 0x55a3014e4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.489+0000 7f5020f9b700 1 -- 172.21.15.119:0/2529487037 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3012b0630 con 0x55a3014e0460 2023-12-07T10:21:55.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.489+0000 7f501bfff700 1 --2- 172.21.15.119:0/2529487037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3014e4150 0x55a3014e4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:55.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.490+0000 7f5020f9b700 1 --2- 172.21.15.119:0/2529487037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3014e0460 0x55a3014e3c10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f50100114f0 tx=0x7f5010011800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=408ad39c326b47e5 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:55.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.490+0000 7f502179c700 1 --2- 172.21.15.119:0/2332391253 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f500c0a1e20 0x7f500c0a42e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5014007ea0 tx=0x7f5014004b40 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:55.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.490+0000 7f5006ffd700 1 -- 172.21.15.119:0/2529487037 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5010004230 con 0x55a3014e0460 2023-12-07T10:21:55.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.490+0000 7f5006ffd700 1 -- 172.21.15.119:0/2529487037 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5010004390 con 0x55a3014e0460 2023-12-07T10:21:55.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.490+0000 7f5006ffd700 1 -- 172.21.15.119:0/2529487037 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50100198e0 con 0x55a3014e0460 2023-12-07T10:21:55.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.491+0000 7f5036b1cb80 1 -- 172.21.15.119:0/2529487037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3014e0460 msgr2=0x55a3014e3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:55.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.491+0000 7f5036b1cb80 1 --2- 172.21.15.119:0/2529487037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3014e0460 0x55a3014e3c10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f50100114f0 tx=0x7f5010011800 comp rx=0 tx=0).stop 2023-12-07T10:21:55.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.491+0000 7f5036b1cb80 1 -- 172.21.15.119:0/2529487037 shutdown_connections 2023-12-07T10:21:55.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.491+0000 7f5036b1cb80 1 --2- 172.21.15.119:0/2529487037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3014e4150 0x55a3014e4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.491+0000 7f5036b1cb80 1 --2- 172.21.15.119:0/2529487037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3014dfb40 0x55a3014dff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.491+0000 7f5036b1cb80 1 --2- 172.21.15.119:0/2529487037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3014e0460 0x55a3014e3c10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.491+0000 7f5036b1cb80 1 -- 172.21.15.119:0/2529487037 >> 172.21.15.119:0/2529487037 conn(0x55a30140e360 msgr2=0x55a3014da130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:55.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.491+0000 7f5036b1cb80 1 -- 172.21.15.119:0/2529487037 shutdown_connections 2023-12-07T10:21:55.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.491+0000 7f5036b1cb80 1 -- 172.21.15.119:0/2529487037 wait complete. 2023-12-07T10:21:55.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.491+0000 7f5036b1cb80 1 -- start start 2023-12-07T10:21:55.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.492+0000 7f5036b1cb80 1 Processor -- start 2023-12-07T10:21:55.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.492+0000 7f5036b1cb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3014dfb40 0x55a3014e8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.493+0000 7f5036b1cb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3014e0460 0x55a3014e91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.493+0000 7f5036b1cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3014e4150 0x55a3014e9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.493+0000 7f5036b1cb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3014339b0 con 0x55a3014e0460 2023-12-07T10:21:55.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.493+0000 7f5036b1cb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3013ed560 con 0x55a3014dfb40 2023-12-07T10:21:55.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.493+0000 7f5036b1cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3013ed200 con 0x55a3014e4150 2023-12-07T10:21:55.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.493+0000 7f5036b1cb80 10 client.0 _ll_register_callbacks cb 0x55a3013d4940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:55.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.493+0000 7f502179c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3014e4150 0x55a3014e9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.493+0000 7f5020f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3014dfb40 0x55a3014e8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.493+0000 7f502179c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3014e4150 0x55a3014e9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:40140/0 (socket says 172.21.15.119:40140) 2023-12-07T10:21:55.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.493+0000 7f5020f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3014dfb40 0x55a3014e8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45742/0 (socket says 172.21.15.119:45742) 2023-12-07T10:21:55.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.493+0000 7f502179c700 1 -- 172.21.15.119:0/1377212451 learned_addr learned my addr 172.21.15.119:0/1377212451 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:55.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.493+0000 7f501bfff700 1 --2- 172.21.15.119:0/1377212451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3014e0460 0x55a3014e91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.493+0000 7f502179c700 1 -- 172.21.15.119:0/1377212451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3014dfb40 msgr2=0x55a3014e8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:55.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.493+0000 7f502179c700 1 --2- 172.21.15.119:0/1377212451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3014dfb40 0x55a3014e8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.493+0000 7f502179c700 1 -- 172.21.15.119:0/1377212451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3014e0460 msgr2=0x55a3014e91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:55.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.493+0000 7f502179c700 1 --2- 172.21.15.119:0/1377212451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3014e0460 0x55a3014e91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.493+0000 7f502179c700 1 -- 172.21.15.119:0/1377212451 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f501c108320 con 0x55a3014e4150 2023-12-07T10:21:55.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.493+0000 7f5020f9b700 1 --2- 172.21.15.119:0/1377212451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3014dfb40 0x55a3014e8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:55.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.494+0000 7f502179c700 1 --2- 172.21.15.119:0/1377212451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3014e4150 0x55a3014e9710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5014009c10 tx=0x7f5014004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4ce083011c2aab5 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:55.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.494+0000 7f4ffbfff700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:21:55.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.494+0000 7f4ffbfff700 1 -- 172.21.15.119:0/1377212451 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5014004db0 con 0x55a3014e4150 2023-12-07T10:21:55.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.494+0000 7f5036b1cb80 1 -- 172.21.15.119:0/1377212451 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3014d2b00 con 0x55a3014e4150 2023-12-07T10:21:55.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.494+0000 7f4ffbfff700 1 -- 172.21.15.119:0/1377212451 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f5014020040 con 0x55a3014e4150 2023-12-07T10:21:55.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.494+0000 7f4ffbfff700 1 -- 172.21.15.119:0/1377212451 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f501400ee90 con 0x55a3014e4150 2023-12-07T10:21:55.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.494+0000 7f5036b1cb80 10 client.26188 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:55.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.494+0000 7f5036b1cb80 1 -- 172.21.15.119:0/1377212451 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f5010009c90 con 0x55a3014e4150 2023-12-07T10:21:55.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.495+0000 7f4ffbfff700 1 -- 172.21.15.119:0/1377212451 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f501402e080 con 0x55a3014e4150 2023-12-07T10:21:55.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.495+0000 7f4ffbfff700 1 -- 172.21.15.119:0/1377212451 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5014004020 con 0x55a3014e4150 2023-12-07T10:21:55.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.495+0000 7f5036b1cb80 10 client.26188 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:55.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.495+0000 7f5036b1cb80 10 client.26188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:21:55.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.495+0000 7f5036b1cb80 20 client.26188 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:55.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.495+0000 7f5036b1cb80 1 --2- 172.21.15.119:0/1377212451 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a3014f94b0 0x55a3014fb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.495+0000 7f5036b1cb80 4 client.26188 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:21:55.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.495+0000 7f5036b1cb80 1 -- 172.21.15.119:0/1377212451 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3014ae7a0 con 0x55a3014f94b0 2023-12-07T10:21:55.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.495+0000 7f5020f9b700 1 --2- 172.21.15.119:0/1377212451 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a3014f94b0 0x55a3014fb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.496+0000 7f5020f9b700 1 --2- 172.21.15.119:0/1377212451 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a3014f94b0 0x55a3014fb8d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b07fb2885d483731 server_cookie=5b8dfdb29fdf5341 in_seq=0 out_seq=0 2023-12-07T10:21:55.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.496+0000 7f4ffbfff700 10 client.26188 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:55.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.496+0000 7f4ffbfff700 1 -- 172.21.15.119:0/1377212451 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a3014ae7a0 con 0x55a3014f94b0 2023-12-07T10:21:55.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.496+0000 7f4ffbfff700 10 client.26188 handle_command_reply: tid=0 2023-12-07T10:21:55.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.496+0000 7f5036b1cb80 1 client.26188 shutdown 2023-12-07T10:21:55.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.497+0000 7f5036b1cb80 1 -- 172.21.15.119:0/1377212451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3014e4150 msgr2=0x55a3014e9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:55.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.497+0000 7f5036b1cb80 1 --2- 172.21.15.119:0/1377212451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3014e4150 0x55a3014e9710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5014009c10 tx=0x7f5014004ad0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.497+0000 7f5036b1cb80 1 -- 172.21.15.119:0/1377212451 shutdown_connections 2023-12-07T10:21:55.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.497+0000 7f5036b1cb80 1 --2- 172.21.15.119:0/1377212451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3014e4150 0x55a3014e9710 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.497+0000 7f5036b1cb80 1 --2- 172.21.15.119:0/1377212451 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a3014f94b0 0x55a3014fb8d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.497+0000 7f5036b1cb80 1 --2- 172.21.15.119:0/1377212451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3014dfb40 0x55a3014e8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.497+0000 7f5036b1cb80 1 --2- 172.21.15.119:0/1377212451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3014e0460 0x55a3014e91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.497+0000 7f5036b1cb80 1 -- 172.21.15.119:0/1377212451 >> 172.21.15.119:0/1377212451 conn(0x55a30140e360 msgr2=0x55a3014d4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:55.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.497+0000 7f4ffbfff700 0 client.26188 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:55.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.497+0000 7f5036b1cb80 1 -- 172.21.15.119:0/1377212451 shutdown_connections 2023-12-07T10:21:55.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.497+0000 7f5036b1cb80 1 -- 172.21.15.119:0/1377212451 wait complete. 2023-12-07T10:21:55.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.497+0000 7f5036b1cb80 20 client.26188 trim_cache size 0 max 16384 2023-12-07T10:21:55.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.514+0000 7f5036b1cb80 1 Processor -- start 2023-12-07T10:21:55.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.515+0000 7f5036b1cb80 1 -- start start 2023-12-07T10:21:55.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.515+0000 7f5036b1cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3014dfb40 0x55a3014e2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.515+0000 7f5036b1cb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3014e0460 0x55a3014f0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.515+0000 7f5036b1cb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3014e4150 0x55a3014f0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.515+0000 7f5036b1cb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3012b2920 con 0x55a3014e0460 2023-12-07T10:21:55.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.515+0000 7f5036b1cb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a301500ae0 con 0x55a3014e4150 2023-12-07T10:21:55.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.515+0000 7f5036b1cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3014f91e0 con 0x55a3014dfb40 2023-12-07T10:21:55.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.515+0000 7f5020f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3014dfb40 0x55a3014e2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.515+0000 7f5020f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3014dfb40 0x55a3014e2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:40146/0 (socket says 172.21.15.119:40146) 2023-12-07T10:21:55.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.515+0000 7f5020f9b700 1 -- 172.21.15.119:0/3177479419 learned_addr learned my addr 172.21.15.119:0/3177479419 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:55.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.516+0000 7f502179c700 1 --2- 172.21.15.119:0/3177479419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3014e4150 0x55a3014f0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.516+0000 7f501bfff700 1 --2- 172.21.15.119:0/3177479419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3014e0460 0x55a3014f0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.516+0000 7f5020f9b700 1 -- 172.21.15.119:0/3177479419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3014e4150 msgr2=0x55a3014f0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:55.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.516+0000 7f5020f9b700 1 --2- 172.21.15.119:0/3177479419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3014e4150 0x55a3014f0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.516+0000 7f5020f9b700 1 -- 172.21.15.119:0/3177479419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3014e0460 msgr2=0x55a3014f0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:55.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.516+0000 7f5020f9b700 1 --2- 172.21.15.119:0/3177479419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3014e0460 0x55a3014f0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.516+0000 7f5020f9b700 1 -- 172.21.15.119:0/3177479419 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a30125cde0 con 0x55a3014dfb40 2023-12-07T10:21:55.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.516+0000 7f501bfff700 1 --2- 172.21.15.119:0/3177479419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3014e0460 0x55a3014f0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:21:55.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.516+0000 7f5020f9b700 1 --2- 172.21.15.119:0/3177479419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3014dfb40 0x55a3014e2790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5010011c00 tx=0x7f5010011800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e7a361719037346 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:55.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.516+0000 7f50077fe700 1 -- 172.21.15.119:0/3177479419 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501002ad00 con 0x55a3014dfb40 2023-12-07T10:21:55.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.517+0000 7f50077fe700 1 -- 172.21.15.119:0/3177479419 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f501002ae60 con 0x55a3014dfb40 2023-12-07T10:21:55.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.517+0000 7f50077fe700 1 -- 172.21.15.119:0/3177479419 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5010005620 con 0x55a3014dfb40 2023-12-07T10:21:55.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.517+0000 7f5036b1cb80 1 -- 172.21.15.119:0/3177479419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3014dfb40 msgr2=0x55a3014e2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:55.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.517+0000 7f5036b1cb80 1 --2- 172.21.15.119:0/3177479419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3014dfb40 0x55a3014e2790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5010011c00 tx=0x7f5010011800 comp rx=0 tx=0).stop 2023-12-07T10:21:55.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.517+0000 7f5036b1cb80 1 -- 172.21.15.119:0/3177479419 shutdown_connections 2023-12-07T10:21:55.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.517+0000 7f5036b1cb80 1 --2- 172.21.15.119:0/3177479419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3014e4150 0x55a3014f0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.518+0000 7f5036b1cb80 1 --2- 172.21.15.119:0/3177479419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3014dfb40 0x55a3014e2790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.518+0000 7f5036b1cb80 1 --2- 172.21.15.119:0/3177479419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3014e0460 0x55a3014f0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.518+0000 7f5036b1cb80 1 -- 172.21.15.119:0/3177479419 >> 172.21.15.119:0/3177479419 conn(0x55a30140e360 msgr2=0x55a3014e4530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:55.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.518+0000 7f5036b1cb80 1 -- 172.21.15.119:0/3177479419 shutdown_connections 2023-12-07T10:21:55.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.518+0000 7f5036b1cb80 1 -- 172.21.15.119:0/3177479419 wait complete. 2023-12-07T10:21:55.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.518+0000 7f5036b1cb80 1 -- start start 2023-12-07T10:21:55.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.518+0000 7f5036b1cb80 1 Processor -- start 2023-12-07T10:21:55.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.519+0000 7f5036b1cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3014dfb40 0x55a3014ec2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.519+0000 7f5036b1cb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3014e0460 0x55a3014ec830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.519+0000 7f5036b1cb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3014e4150 0x55a30154e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.519+0000 7f5036b1cb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3014ed140 con 0x55a3014e0460 2023-12-07T10:21:55.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.519+0000 7f5036b1cb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a30154eee0 con 0x55a3014e4150 2023-12-07T10:21:55.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.519+0000 7f5036b1cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a30154f020 con 0x55a3014dfb40 2023-12-07T10:21:55.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.519+0000 7f5036b1cb80 10 client.0 _ll_register_callbacks cb 0x55a3013d4940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:55.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.519+0000 7f5020f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3014dfb40 0x55a3014ec2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.519+0000 7f5020f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3014dfb40 0x55a3014ec2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:40162/0 (socket says 172.21.15.119:40162) 2023-12-07T10:21:55.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.520+0000 7f5020f9b700 1 -- 172.21.15.119:0/2446328966 learned_addr learned my addr 172.21.15.119:0/2446328966 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:55.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.520+0000 7f501bfff700 1 --2- 172.21.15.119:0/2446328966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3014e0460 0x55a3014ec830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.520+0000 7f502179c700 1 --2- 172.21.15.119:0/2446328966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3014e4150 0x55a30154e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.520+0000 7f502179c700 1 -- 172.21.15.119:0/2446328966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3014dfb40 msgr2=0x55a3014ec2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:55.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.520+0000 7f502179c700 1 --2- 172.21.15.119:0/2446328966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3014dfb40 0x55a3014ec2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.520+0000 7f502179c700 1 -- 172.21.15.119:0/2446328966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3014e0460 msgr2=0x55a3014ec830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:55.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.520+0000 7f502179c700 1 --2- 172.21.15.119:0/2446328966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3014e0460 0x55a3014ec830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.520+0000 7f502179c700 1 -- 172.21.15.119:0/2446328966 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3012b0630 con 0x55a3014e4150 2023-12-07T10:21:55.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.520+0000 7f501bfff700 1 --2- 172.21.15.119:0/2446328966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3014e0460 0x55a3014ec830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:55.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.521+0000 7f502179c700 1 --2- 172.21.15.119:0/2446328966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3014e4150 0x55a30154e9a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f50140077a0 tx=0x7f5014003c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=390ce71597fb8a88 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:55.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.521+0000 7f4ffbfff700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:21:55.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.521+0000 7f4ffbfff700 1 -- 172.21.15.119:0/2446328966 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5014004db0 con 0x55a3014e4150 2023-12-07T10:21:55.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.521+0000 7f4ffbfff700 1 -- 172.21.15.119:0/2446328966 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f501400ee70 con 0x55a3014e4150 2023-12-07T10:21:55.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.521+0000 7f5036b1cb80 1 -- 172.21.15.119:0/2446328966 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5010004390 con 0x55a3014e4150 2023-12-07T10:21:55.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.522+0000 7f4ffbfff700 1 -- 172.21.15.119:0/2446328966 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5014009920 con 0x55a3014e4150 2023-12-07T10:21:55.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.522+0000 7f5036b1cb80 10 client.26201 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:55.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.522+0000 7f5036b1cb80 1 -- 172.21.15.119:0/2446328966 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a301245c60 con 0x55a3014e4150 2023-12-07T10:21:55.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.522+0000 7f4ffbfff700 1 -- 172.21.15.119:0/2446328966 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5010004390 con 0x55a3014e4150 2023-12-07T10:21:55.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.522+0000 7f4ffbfff700 1 -- 172.21.15.119:0/2446328966 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5014024c50 con 0x55a3014e4150 2023-12-07T10:21:55.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.522+0000 7f5036b1cb80 10 client.26201 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:55.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.522+0000 7f5036b1cb80 10 client.26201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:21:55.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.522+0000 7f5036b1cb80 20 client.26201 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:55.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.522+0000 7f5036b1cb80 1 --2- 172.21.15.119:0/2446328966 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a301550780 0x55a301552ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:55.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.522+0000 7f5036b1cb80 4 client.26201 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:21:55.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.522+0000 7f5036b1cb80 1 -- 172.21.15.119:0/2446328966 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a3013f9fd0 con 0x55a301550780 2023-12-07T10:21:55.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.523+0000 7f5020f9b700 1 --2- 172.21.15.119:0/2446328966 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a301550780 0x55a301552ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:55.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.524+0000 7f5020f9b700 1 --2- 172.21.15.119:0/2446328966 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a301550780 0x55a301552ba0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99d9ea097bc2ec9a server_cookie=43628955e6c3beee in_seq=0 out_seq=0 2023-12-07T10:21:55.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.524+0000 7f4ffbfff700 10 client.26201 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:55.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.524+0000 7f4ffbfff700 1 -- 172.21.15.119:0/2446328966 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a3013f9fd0 con 0x55a301550780 2023-12-07T10:21:55.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.524+0000 7f4ffbfff700 10 client.26201 handle_command_reply: tid=0 2023-12-07T10:21:55.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.524+0000 7f5036b1cb80 1 client.26201 shutdown 2023-12-07T10:21:55.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.524+0000 7f5036b1cb80 1 -- 172.21.15.119:0/2446328966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3014e4150 msgr2=0x55a30154e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:55.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.524+0000 7f5036b1cb80 1 --2- 172.21.15.119:0/2446328966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3014e4150 0x55a30154e9a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f50140077a0 tx=0x7f5014003c30 comp rx=0 tx=0).stop 2023-12-07T10:21:55.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.524+0000 7f5036b1cb80 1 -- 172.21.15.119:0/2446328966 shutdown_connections 2023-12-07T10:21:55.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.524+0000 7f5036b1cb80 1 --2- 172.21.15.119:0/2446328966 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a301550780 0x55a301552ba0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.524+0000 7f5036b1cb80 1 --2- 172.21.15.119:0/2446328966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a3014e4150 0x55a30154e9a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.524+0000 7f5036b1cb80 1 --2- 172.21.15.119:0/2446328966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3014dfb40 0x55a3014ec2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.524+0000 7f5036b1cb80 1 --2- 172.21.15.119:0/2446328966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a3014e0460 0x55a3014ec830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.524+0000 7f5036b1cb80 1 -- 172.21.15.119:0/2446328966 >> 172.21.15.119:0/2446328966 conn(0x55a30140e360 msgr2=0x55a3014d9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:55.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.524+0000 7f4ffbfff700 0 client.26201 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:55.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.524+0000 7f5036b1cb80 1 -- 172.21.15.119:0/2446328966 shutdown_connections 2023-12-07T10:21:55.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.524+0000 7f5036b1cb80 1 -- 172.21.15.119:0/2446328966 wait complete. 2023-12-07T10:21:55.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.525+0000 7f5036b1cb80 20 client.26201 trim_cache size 0 max 16384 2023-12-07T10:21:55.527 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:21:55.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.525+0000 7f50231ff700 1 -- 172.21.15.119:0/2332391253 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f500c0a1e20 msgr2=0x7f500c0a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:55.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.525+0000 7f50231ff700 1 --2- 172.21.15.119:0/2332391253 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f500c0a1e20 0x7f500c0a42e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5014007ea0 tx=0x7f5014004b40 comp rx=0 tx=0).stop 2023-12-07T10:21:55.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.525+0000 7f50231ff700 1 -- 172.21.15.119:0/2332391253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f501c101f20 msgr2=0x7f501c19fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:55.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.525+0000 7f50231ff700 1 --2- 172.21.15.119:0/2332391253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f501c101f20 0x7f501c19fde0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5008000c00 tx=0x7f5008004730 comp rx=0 tx=0).stop 2023-12-07T10:21:55.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.526+0000 7f50231ff700 1 -- 172.21.15.119:0/2332391253 shutdown_connections 2023-12-07T10:21:55.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.526+0000 7f50231ff700 1 --2- 172.21.15.119:0/2332391253 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f500c0a1e20 0x7f500c0a42e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.526+0000 7f50231ff700 1 --2- 172.21.15.119:0/2332391253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f501c107620 0x7f501c1999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.526+0000 7f50231ff700 1 --2- 172.21.15.119:0/2332391253 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f501c101560 0x7f501c19f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.526+0000 7f50231ff700 1 --2- 172.21.15.119:0/2332391253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f501c101f20 0x7f501c19fde0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:55.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.526+0000 7f50231ff700 1 -- 172.21.15.119:0/2332391253 >> 172.21.15.119:0/2332391253 conn(0x7f501c0fcfd0 msgr2=0x7f501c0fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:55.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.526+0000 7f50231ff700 1 -- 172.21.15.119:0/2332391253 shutdown_connections 2023-12-07T10:21:55.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:55.526+0000 7f50231ff700 1 -- 172.21.15.119:0/2332391253 wait complete. 2023-12-07T10:21:55.538 INFO:tasks.workunit.client.0.smithi119.stdout: 1 22114 17.02 MB/sec warmup 8 sec latency 18.176 ms 2023-12-07T10:21:56.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:55 smithi119 ceph-mon[97908]: pgmap v48: 129 pgs: 129 active+clean; 6.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 505 KiB/s wr, 67 op/s 2023-12-07T10:21:56.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:55 smithi159 ceph-mon[134726]: pgmap v48: 129 pgs: 129 active+clean; 6.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 505 KiB/s wr, 67 op/s 2023-12-07T10:21:56.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:55 smithi153 ceph-mon[133625]: pgmap v48: 129 pgs: 129 active+clean; 6.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 505 KiB/s wr, 67 op/s 2023-12-07T10:21:56.539 INFO:tasks.workunit.client.0.smithi119.stdout: 1 25274 17.20 MB/sec warmup 9 sec latency 6.967 ms 2023-12-07T10:21:56.548 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-07T10:21:56.548 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-07T10:21:56.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.779+0000 7f963f59e700 1 -- 172.21.15.119:0/1211009857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9640107470 msgr2=0x7f9640107850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:56.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.779+0000 7f963f59e700 1 --2- 172.21.15.119:0/1211009857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9640107470 0x7f9640107850 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9630009990 tx=0x7f9630009ca0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.779+0000 7f963f59e700 1 -- 172.21.15.119:0/1211009857 shutdown_connections 2023-12-07T10:21:56.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.779+0000 7f963f59e700 1 --2- 172.21.15.119:0/1211009857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9640101d90 0x7f964010e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.779+0000 7f963f59e700 1 --2- 172.21.15.119:0/1211009857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9640107470 0x7f9640107850 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.779+0000 7f963f59e700 1 --2- 172.21.15.119:0/1211009857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96401013d0 0x7f9640101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.779+0000 7f963f59e700 1 -- 172.21.15.119:0/1211009857 >> 172.21.15.119:0/1211009857 conn(0x7f96400fcee0 msgr2=0x7f96400ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:56.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.779+0000 7f963f59e700 1 -- 172.21.15.119:0/1211009857 shutdown_connections 2023-12-07T10:21:56.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.780+0000 7f963f59e700 1 -- 172.21.15.119:0/1211009857 wait complete. 2023-12-07T10:21:56.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.780+0000 7f963f59e700 1 Processor -- start 2023-12-07T10:21:56.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.781+0000 7f963f59e700 1 -- start start 2023-12-07T10:21:56.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.781+0000 7f963f59e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96401013d0 0x7f9640199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:56.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.781+0000 7f963f59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9640101d90 0x7f9640199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:56.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.781+0000 7f963f59e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9640107470 0x7f964019e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:56.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.781+0000 7f963f59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964019e900 con 0x7f9640101d90 2023-12-07T10:21:56.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.781+0000 7f963f59e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964019ea70 con 0x7f9640107470 2023-12-07T10:21:56.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.781+0000 7f963f59e700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964019ec30 con 0x7f96401013d0 2023-12-07T10:21:56.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.781+0000 7f963e59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96401013d0 0x7f9640199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:56.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.781+0000 7f963dd9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9640101d90 0x7f9640199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:56.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.781+0000 7f963e59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96401013d0 0x7f9640199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:40184/0 (socket says 172.21.15.119:40184) 2023-12-07T10:21:56.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.781+0000 7f963e59c700 1 -- 172.21.15.119:0/862900827 learned_addr learned my addr 172.21.15.119:0/862900827 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:56.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.781+0000 7f963ed9d700 1 --2- 172.21.15.119:0/862900827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9640107470 0x7f964019e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:56.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.782+0000 7f963e59c700 1 -- 172.21.15.119:0/862900827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9640107470 msgr2=0x7f964019e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:56.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.782+0000 7f963e59c700 1 --2- 172.21.15.119:0/862900827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9640107470 0x7f964019e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.782+0000 7f963e59c700 1 -- 172.21.15.119:0/862900827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9640101d90 msgr2=0x7f9640199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:56.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.782+0000 7f963e59c700 1 --2- 172.21.15.119:0/862900827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9640101d90 0x7f9640199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.782+0000 7f963e59c700 1 -- 172.21.15.119:0/862900827 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96300095f0 con 0x7f96401013d0 2023-12-07T10:21:56.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.782+0000 7f963e59c700 1 --2- 172.21.15.119:0/862900827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96401013d0 0x7f9640199a20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9630005910 tx=0x7f9630004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:56.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.782+0000 7f962f7fe700 1 -- 172.21.15.119:0/862900827 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963001d070 con 0x7f96401013d0 2023-12-07T10:21:56.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.782+0000 7f962f7fe700 1 -- 172.21.15.119:0/862900827 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9630022470 con 0x7f96401013d0 2023-12-07T10:21:56.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.782+0000 7f963f59e700 1 -- 172.21.15.119:0/862900827 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964019eeb0 con 0x7f96401013d0 2023-12-07T10:21:56.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.782+0000 7f962f7fe700 1 -- 172.21.15.119:0/862900827 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9630017650 con 0x7f96401013d0 2023-12-07T10:21:56.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.783+0000 7f963f59e700 1 -- 172.21.15.119:0/862900827 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96401a67c0 con 0x7f96401013d0 2023-12-07T10:21:56.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.786+0000 7f962f7fe700 1 -- 172.21.15.119:0/862900827 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) v1 ==== 136717+0+0 (secure 0 0 0) 0x7f96300225e0 con 0x7f96401013d0 2023-12-07T10:21:56.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.788+0000 7f962f7fe700 1 --2- 172.21.15.119:0/862900827 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f96280a1e20 0x7f96280a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:56.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.788+0000 7f962f7fe700 1 -- 172.21.15.119:0/862900827 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f96300cf1d0 con 0x7f96401013d0 2023-12-07T10:21:56.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.788+0000 7f963dd9b700 1 --2- 172.21.15.119:0/862900827 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f96280a1e20 0x7f96280a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:56.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.788+0000 7f9658687b80 1 Processor -- start 2023-12-07T10:21:56.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.789+0000 7f9658687b80 1 -- start start 2023-12-07T10:21:56.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.789+0000 7f963dd9b700 1 --2- 172.21.15.119:0/862900827 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f96280a1e20 0x7f96280a42e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f964019af70 tx=0x7f9634009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:56.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.789+0000 7f9658687b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4dca01a70 0x55a4dca01e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:56.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.789+0000 7f9658687b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4dca02390 0x55a4dca05e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:56.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.789+0000 7f9658687b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4dca063d0 0x55a4dca06830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:56.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.789+0000 7f9658687b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4dc964840 con 0x55a4dca063d0 2023-12-07T10:21:56.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.789+0000 7f9658687b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4dc94b820 con 0x55a4dca01a70 2023-12-07T10:21:56.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.789+0000 7f9658687b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4dc949140 con 0x55a4dca02390 2023-12-07T10:21:56.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.789+0000 7f963e59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4dca01a70 0x55a4dca01e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:56.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.790+0000 7f963e59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4dca01a70 0x55a4dca01e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45778/0 (socket says 172.21.15.119:45778) 2023-12-07T10:21:56.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.790+0000 7f963e59c700 1 -- 172.21.15.119:0/2761774911 learned_addr learned my addr 172.21.15.119:0/2761774911 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:56.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.790+0000 7f963dd9b700 1 --2- 172.21.15.119:0/2761774911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4dca02390 0x55a4dca05e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:56.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.790+0000 7f963e59c700 1 -- 172.21.15.119:0/2761774911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4dca02390 msgr2=0x55a4dca05e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:56.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.790+0000 7f963e59c700 1 --2- 172.21.15.119:0/2761774911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4dca02390 0x55a4dca05e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.790+0000 7f963e59c700 1 -- 172.21.15.119:0/2761774911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4dca063d0 msgr2=0x55a4dca06830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:56.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.790+0000 7f963e59c700 1 --2- 172.21.15.119:0/2761774911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4dca063d0 0x55a4dca06830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.790+0000 7f963e59c700 1 -- 172.21.15.119:0/2761774911 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4dc7c16b0 con 0x55a4dca01a70 2023-12-07T10:21:56.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.790+0000 7f963e59c700 1 --2- 172.21.15.119:0/2761774911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4dca01a70 0x55a4dca01e50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9630006010 tx=0x7f9630022f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5253a5d01a08b92b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:56.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.790+0000 7f962cff9700 1 -- 172.21.15.119:0/2761774911 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963009c220 con 0x55a4dca01a70 2023-12-07T10:21:56.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.790+0000 7f962cff9700 1 -- 172.21.15.119:0/2761774911 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f963009bb20 con 0x55a4dca01a70 2023-12-07T10:21:56.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.790+0000 7f962cff9700 1 -- 172.21.15.119:0/2761774911 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963009b270 con 0x55a4dca01a70 2023-12-07T10:21:56.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.791+0000 7f9658687b80 1 -- 172.21.15.119:0/2761774911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4dca01a70 msgr2=0x55a4dca01e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:56.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.791+0000 7f9658687b80 1 --2- 172.21.15.119:0/2761774911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4dca01a70 0x55a4dca01e50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9630006010 tx=0x7f9630022f90 comp rx=0 tx=0).stop 2023-12-07T10:21:56.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.791+0000 7f9658687b80 1 -- 172.21.15.119:0/2761774911 shutdown_connections 2023-12-07T10:21:56.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.791+0000 7f9658687b80 1 --2- 172.21.15.119:0/2761774911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4dca063d0 0x55a4dca06830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.791+0000 7f9658687b80 1 --2- 172.21.15.119:0/2761774911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4dca01a70 0x55a4dca01e50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.791+0000 7f9658687b80 1 --2- 172.21.15.119:0/2761774911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4dca02390 0x55a4dca05e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.791+0000 7f9658687b80 1 -- 172.21.15.119:0/2761774911 >> 172.21.15.119:0/2761774911 conn(0x55a4dc91f3e0 msgr2=0x55a4dc9eb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:56.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.791+0000 7f9658687b80 1 -- 172.21.15.119:0/2761774911 shutdown_connections 2023-12-07T10:21:56.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.791+0000 7f9658687b80 1 -- 172.21.15.119:0/2761774911 wait complete. 2023-12-07T10:21:56.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.792+0000 7f9658687b80 1 -- start start 2023-12-07T10:21:56.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.792+0000 7f9658687b80 1 Processor -- start 2023-12-07T10:21:56.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.792+0000 7f9658687b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4dca01a70 0x55a4dca0db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:56.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.792+0000 7f9658687b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4dca02390 0x55a4dca0ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:56.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.793+0000 7f9658687b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4dca063d0 0x55a4dca0b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:56.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.793+0000 7f9658687b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4dc944a30 con 0x55a4dca02390 2023-12-07T10:21:56.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.793+0000 7f9658687b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4dc8fe5e0 con 0x55a4dca01a70 2023-12-07T10:21:56.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.793+0000 7f9658687b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4dc8fe280 con 0x55a4dca063d0 2023-12-07T10:21:56.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.793+0000 7f9658687b80 10 client.0 _ll_register_callbacks cb 0x55a4dc8e59c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:56.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.793+0000 7f963e59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4dca01a70 0x55a4dca0db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:56.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.793+0000 7f963dd9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4dca02390 0x55a4dca0ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:56.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.793+0000 7f963e59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4dca01a70 0x55a4dca0db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45788/0 (socket says 172.21.15.119:45788) 2023-12-07T10:21:56.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.793+0000 7f963e59c700 1 -- 172.21.15.119:0/4080779416 learned_addr learned my addr 172.21.15.119:0/4080779416 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:56.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.793+0000 7f963ed9d700 1 --2- 172.21.15.119:0/4080779416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4dca063d0 0x55a4dca0b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:56.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.793+0000 7f963e59c700 1 -- 172.21.15.119:0/4080779416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4dca063d0 msgr2=0x55a4dca0b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:56.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.793+0000 7f963e59c700 1 --2- 172.21.15.119:0/4080779416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4dca063d0 0x55a4dca0b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.793+0000 7f963e59c700 1 -- 172.21.15.119:0/4080779416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4dca02390 msgr2=0x55a4dca0ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:56.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.793+0000 7f963e59c700 1 --2- 172.21.15.119:0/4080779416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4dca02390 0x55a4dca0ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.793+0000 7f963e59c700 1 -- 172.21.15.119:0/4080779416 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96401a67c0 con 0x55a4dca01a70 2023-12-07T10:21:56.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.793+0000 7f963ed9d700 1 --2- 172.21.15.119:0/4080779416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4dca063d0 0x55a4dca0b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:21:56.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.794+0000 7f963e59c700 1 --2- 172.21.15.119:0/4080779416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4dca01a70 0x55a4dca0db80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f963009bab0 tx=0x7f9630009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38906bd7c0ef71c2 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:56.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.794+0000 7f9625ffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:21:56.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.794+0000 7f9625ffb700 1 -- 172.21.15.119:0/4080779416 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963007e070 con 0x55a4dca01a70 2023-12-07T10:21:56.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.794+0000 7f9625ffb700 1 -- 172.21.15.119:0/4080779416 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f963009fc70 con 0x55a4dca01a70 2023-12-07T10:21:56.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.794+0000 7f9658687b80 1 -- 172.21.15.119:0/4080779416 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a4dc9e3a20 con 0x55a4dca01a70 2023-12-07T10:21:56.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.794+0000 7f9625ffb700 1 -- 172.21.15.119:0/4080779416 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f963009c220 con 0x55a4dca01a70 2023-12-07T10:21:56.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.794+0000 7f9658687b80 10 client.26213 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:56.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.794+0000 7f9658687b80 1 -- 172.21.15.119:0/4080779416 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a4dc76de60 con 0x55a4dca01a70 2023-12-07T10:21:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.794+0000 7f9625ffb700 1 -- 172.21.15.119:0/4080779416 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96300a7020 con 0x55a4dca01a70 2023-12-07T10:21:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.794+0000 7f9625ffb700 1 -- 172.21.15.119:0/4080779416 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9630081020 con 0x55a4dca01a70 2023-12-07T10:21:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.794+0000 7f9658687b80 10 client.26213 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.794+0000 7f9658687b80 10 client.26213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:21:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.795+0000 7f9658687b80 20 client.26213 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.795+0000 7f9658687b80 1 --2- 172.21.15.119:0/4080779416 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a4dca1ada0 0x55a4dca1d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.795+0000 7f9658687b80 4 client.26213 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:21:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.795+0000 7f9658687b80 1 -- 172.21.15.119:0/4080779416 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a4dc9e3a20 con 0x55a4dca1ada0 2023-12-07T10:21:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.795+0000 7f963dd9b700 1 --2- 172.21.15.119:0/4080779416 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a4dca1ada0 0x55a4dca1d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.795+0000 7f963dd9b700 1 --2- 172.21.15.119:0/4080779416 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a4dca1ada0 0x55a4dca1d190 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eca42755e79825bb server_cookie=43444e0a65aef87c in_seq=0 out_seq=0 2023-12-07T10:21:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.795+0000 7f9625ffb700 10 client.26213 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.796+0000 7f9625ffb700 1 -- 172.21.15.119:0/4080779416 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a4dc9e3a20 con 0x55a4dca1ada0 2023-12-07T10:21:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.796+0000 7f9625ffb700 10 client.26213 handle_command_reply: tid=0 2023-12-07T10:21:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.796+0000 7f9658687b80 1 client.26213 shutdown 2023-12-07T10:21:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.796+0000 7f9658687b80 1 -- 172.21.15.119:0/4080779416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4dca01a70 msgr2=0x55a4dca0db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.796+0000 7f9658687b80 1 --2- 172.21.15.119:0/4080779416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4dca01a70 0x55a4dca0db80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f963009bab0 tx=0x7f9630009310 comp rx=0 tx=0).stop 2023-12-07T10:21:56.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.796+0000 7f9658687b80 1 -- 172.21.15.119:0/4080779416 shutdown_connections 2023-12-07T10:21:56.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.796+0000 7f9658687b80 1 --2- 172.21.15.119:0/4080779416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4dca063d0 0x55a4dca0b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.796+0000 7f9658687b80 1 --2- 172.21.15.119:0/4080779416 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a4dca1ada0 0x55a4dca1d190 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.796+0000 7f9658687b80 1 --2- 172.21.15.119:0/4080779416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4dca01a70 0x55a4dca0db80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.796+0000 7f9658687b80 1 --2- 172.21.15.119:0/4080779416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4dca02390 0x55a4dca0ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.797+0000 7f9658687b80 1 -- 172.21.15.119:0/4080779416 >> 172.21.15.119:0/4080779416 conn(0x55a4dc91f3e0 msgr2=0x55a4dc9eb190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:56.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.797+0000 7f9625ffb700 0 client.26213 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:56.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.797+0000 7f9658687b80 1 -- 172.21.15.119:0/4080779416 shutdown_connections 2023-12-07T10:21:56.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.797+0000 7f9658687b80 1 -- 172.21.15.119:0/4080779416 wait complete. 2023-12-07T10:21:56.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.797+0000 7f9658687b80 20 client.26213 trim_cache size 0 max 16384 2023-12-07T10:21:56.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.816+0000 7f9658687b80 1 Processor -- start 2023-12-07T10:21:56.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.816+0000 7f9658687b80 1 -- start start 2023-12-07T10:21:56.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.816+0000 7f9658687b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4dca01a70 0x55a4dca087b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:56.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.816+0000 7f9658687b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4dca02390 0x55a4dca12160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:56.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.816+0000 7f9658687b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4dca063d0 0x55a4dca126a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:56.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.816+0000 7f9658687b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4dc7c39a0 con 0x55a4dca01a70 2023-12-07T10:21:56.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.816+0000 7f9658687b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4dca0bc10 con 0x55a4dca02390 2023-12-07T10:21:56.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.816+0000 7f9658687b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4dca223b0 con 0x55a4dca063d0 2023-12-07T10:21:56.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.817+0000 7f963dd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4dca02390 0x55a4dca12160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:56.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.817+0000 7f963dd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4dca02390 0x55a4dca12160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45800/0 (socket says 172.21.15.119:45800) 2023-12-07T10:21:56.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.817+0000 7f963dd9b700 1 -- 172.21.15.119:0/2457370439 learned_addr learned my addr 172.21.15.119:0/2457370439 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:56.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.817+0000 7f963e59c700 1 --2- 172.21.15.119:0/2457370439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4dca01a70 0x55a4dca087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:56.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.817+0000 7f963ed9d700 1 --2- 172.21.15.119:0/2457370439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4dca063d0 0x55a4dca126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:56.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.817+0000 7f963dd9b700 1 -- 172.21.15.119:0/2457370439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4dca063d0 msgr2=0x55a4dca126a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:56.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.817+0000 7f963dd9b700 1 --2- 172.21.15.119:0/2457370439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4dca063d0 0x55a4dca126a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.817+0000 7f963dd9b700 1 -- 172.21.15.119:0/2457370439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4dca01a70 msgr2=0x55a4dca087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:56.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.817+0000 7f963dd9b700 1 --2- 172.21.15.119:0/2457370439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4dca01a70 0x55a4dca087b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.817+0000 7f963dd9b700 1 -- 172.21.15.119:0/2457370439 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4dca16ee0 con 0x55a4dca02390 2023-12-07T10:21:56.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.817+0000 7f963e59c700 1 --2- 172.21.15.119:0/2457370439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4dca01a70 0x55a4dca087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:21:56.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.817+0000 7f963dd9b700 1 --2- 172.21.15.119:0/2457370439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4dca02390 0x55a4dca12160 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f96340094f0 tx=0x7f9634014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2c07edc0c92fa52 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:56.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.818+0000 7f962d7fa700 1 -- 172.21.15.119:0/2457370439 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9634023040 con 0x55a4dca02390 2023-12-07T10:21:56.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.818+0000 7f962d7fa700 1 -- 172.21.15.119:0/2457370439 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9634011c10 con 0x55a4dca02390 2023-12-07T10:21:56.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.818+0000 7f962d7fa700 1 -- 172.21.15.119:0/2457370439 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9634027a90 con 0x55a4dca02390 2023-12-07T10:21:56.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.818+0000 7f9658687b80 1 -- 172.21.15.119:0/2457370439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4dca02390 msgr2=0x55a4dca12160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:56.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.818+0000 7f9658687b80 1 --2- 172.21.15.119:0/2457370439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4dca02390 0x55a4dca12160 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f96340094f0 tx=0x7f9634014000 comp rx=0 tx=0).stop 2023-12-07T10:21:56.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.818+0000 7f9658687b80 1 -- 172.21.15.119:0/2457370439 shutdown_connections 2023-12-07T10:21:56.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.818+0000 7f9658687b80 1 --2- 172.21.15.119:0/2457370439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4dca063d0 0x55a4dca126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.818+0000 7f9658687b80 1 --2- 172.21.15.119:0/2457370439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4dca01a70 0x55a4dca087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.818+0000 7f9658687b80 1 --2- 172.21.15.119:0/2457370439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4dca02390 0x55a4dca12160 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.818+0000 7f9658687b80 1 -- 172.21.15.119:0/2457370439 >> 172.21.15.119:0/2457370439 conn(0x55a4dc91f3e0 msgr2=0x55a4dca044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:56.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.819+0000 7f9658687b80 1 -- 172.21.15.119:0/2457370439 shutdown_connections 2023-12-07T10:21:56.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.819+0000 7f9658687b80 1 -- 172.21.15.119:0/2457370439 wait complete. 2023-12-07T10:21:56.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.819+0000 7f9658687b80 1 -- start start 2023-12-07T10:21:56.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.819+0000 7f9658687b80 1 Processor -- start 2023-12-07T10:21:56.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.820+0000 7f9658687b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4dca01a70 0x55a4dca123e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:56.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.820+0000 7f9658687b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4dca02390 0x55a4dca0e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:56.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.820+0000 7f9658687b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4dca063d0 0x55a4dca0ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:56.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.820+0000 7f9658687b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4dca0f320 con 0x55a4dca01a70 2023-12-07T10:21:56.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.820+0000 7f9658687b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4dca0f490 con 0x55a4dca063d0 2023-12-07T10:21:56.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.820+0000 7f9658687b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4dca0f650 con 0x55a4dca02390 2023-12-07T10:21:56.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.820+0000 7f9658687b80 10 client.0 _ll_register_callbacks cb 0x55a4dc8e59c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:56.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.820+0000 7f963dd9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4dca02390 0x55a4dca0e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:56.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.820+0000 7f963dd9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4dca02390 0x55a4dca0e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:40222/0 (socket says 172.21.15.119:40222) 2023-12-07T10:21:56.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.820+0000 7f963dd9b700 1 -- 172.21.15.119:0/2346879530 learned_addr learned my addr 172.21.15.119:0/2346879530 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:56.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.820+0000 7f963ed9d700 1 --2- 172.21.15.119:0/2346879530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4dca063d0 0x55a4dca0ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:56.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.820+0000 7f963e59c700 1 --2- 172.21.15.119:0/2346879530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4dca01a70 0x55a4dca123e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:56.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.821+0000 7f963dd9b700 1 -- 172.21.15.119:0/2346879530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4dca063d0 msgr2=0x55a4dca0ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:56.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.821+0000 7f963dd9b700 1 --2- 172.21.15.119:0/2346879530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4dca063d0 0x55a4dca0ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.821+0000 7f963dd9b700 1 -- 172.21.15.119:0/2346879530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4dca01a70 msgr2=0x55a4dca123e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:56.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.821+0000 7f963dd9b700 1 --2- 172.21.15.119:0/2346879530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4dca01a70 0x55a4dca123e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.821+0000 7f963dd9b700 1 -- 172.21.15.119:0/2346879530 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9634017070 con 0x55a4dca02390 2023-12-07T10:21:56.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.821+0000 7f963dd9b700 1 --2- 172.21.15.119:0/2346879530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4dca02390 0x55a4dca0e800 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f96340097c0 tx=0x7f9634011c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e75e677eb0436587 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:56.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.821+0000 7f9625ffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:21:56.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.821+0000 7f9625ffb700 1 -- 172.21.15.119:0/2346879530 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9634023070 con 0x55a4dca02390 2023-12-07T10:21:56.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.821+0000 7f9625ffb700 1 -- 172.21.15.119:0/2346879530 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9634003c90 con 0x55a4dca02390 2023-12-07T10:21:56.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.821+0000 7f9658687b80 1 -- 172.21.15.119:0/2346879530 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a4dc9bf820 con 0x55a4dca02390 2023-12-07T10:21:56.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.821+0000 7f9625ffb700 1 -- 172.21.15.119:0/2346879530 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9634003880 con 0x55a4dca02390 2023-12-07T10:21:56.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.821+0000 7f9658687b80 10 client.26212 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:56.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.821+0000 7f9658687b80 1 -- 172.21.15.119:0/2346879530 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a4dc756ce0 con 0x55a4dca02390 2023-12-07T10:21:56.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.822+0000 7f9625ffb700 1 -- 172.21.15.119:0/2346879530 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a4dc9bf820 con 0x55a4dca02390 2023-12-07T10:21:56.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.822+0000 7f9625ffb700 1 -- 172.21.15.119:0/2346879530 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9634004af0 con 0x55a4dca02390 2023-12-07T10:21:56.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.822+0000 7f9658687b80 10 client.26212 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:56.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.822+0000 7f9658687b80 10 client.26212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:21:56.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.822+0000 7f9658687b80 20 client.26212 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:56.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.822+0000 7f9658687b80 1 --2- 172.21.15.119:0/2346879530 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a4dca838d0 0x55a4dca85cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:56.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.822+0000 7f9658687b80 4 client.26212 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-07T10:21:56.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.822+0000 7f9658687b80 1 -- 172.21.15.119:0/2346879530 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a4dc90b050 con 0x55a4dca838d0 2023-12-07T10:21:56.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.822+0000 7f963e59c700 1 --2- 172.21.15.119:0/2346879530 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a4dca838d0 0x55a4dca85cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:56.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.823+0000 7f963e59c700 1 --2- 172.21.15.119:0/2346879530 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a4dca838d0 0x55a4dca85cf0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3898e18e179d9a66 server_cookie=66bdf92f696f07dd in_seq=0 out_seq=0 2023-12-07T10:21:56.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.823+0000 7f9625ffb700 10 client.26212 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:56.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.824+0000 7f9625ffb700 1 -- 172.21.15.119:0/2346879530 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a4dc90b050 con 0x55a4dca838d0 2023-12-07T10:21:56.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.824+0000 7f9625ffb700 10 client.26212 handle_command_reply: tid=0 2023-12-07T10:21:56.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.824+0000 7f9658687b80 1 client.26212 shutdown 2023-12-07T10:21:56.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.824+0000 7f9658687b80 1 -- 172.21.15.119:0/2346879530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4dca02390 msgr2=0x55a4dca0e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:56.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.824+0000 7f9658687b80 1 --2- 172.21.15.119:0/2346879530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4dca02390 0x55a4dca0e800 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f96340097c0 tx=0x7f9634011c10 comp rx=0 tx=0).stop 2023-12-07T10:21:56.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.825+0000 7f9658687b80 1 -- 172.21.15.119:0/2346879530 shutdown_connections 2023-12-07T10:21:56.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.825+0000 7f9658687b80 1 --2- 172.21.15.119:0/2346879530 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a4dca838d0 0x55a4dca85cf0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.825+0000 7f9658687b80 1 --2- 172.21.15.119:0/2346879530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a4dca063d0 0x55a4dca0ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.825+0000 7f9658687b80 1 --2- 172.21.15.119:0/2346879530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4dca01a70 0x55a4dca123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.825+0000 7f9658687b80 1 --2- 172.21.15.119:0/2346879530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4dca02390 0x55a4dca0e800 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.825+0000 7f9625ffb700 0 client.26212 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:56.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.825+0000 7f9658687b80 1 -- 172.21.15.119:0/2346879530 >> 172.21.15.119:0/2346879530 conn(0x55a4dc91f3e0 msgr2=0x55a4dca087c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:56.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.825+0000 7f9658687b80 1 -- 172.21.15.119:0/2346879530 shutdown_connections 2023-12-07T10:21:56.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.825+0000 7f9658687b80 1 -- 172.21.15.119:0/2346879530 wait complete. 2023-12-07T10:21:56.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.825+0000 7f9658687b80 20 client.26212 trim_cache size 0 max 16384 2023-12-07T10:21:56.828 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:21:56.828 INFO:teuthology.orchestra.run.smithi119.stdout: "return_code": 0, 2023-12-07T10:21:56.828 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_tag": "4e857eee-9284-44c1-9aed-0fee3db97193", 2023-12-07T10:21:56.828 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": "asynchronous" 2023-12-07T10:21:56.828 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:21:56.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.826+0000 7f963f59e700 1 -- 172.21.15.119:0/862900827 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f96280a1e20 msgr2=0x7f96280a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:56.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.826+0000 7f963f59e700 1 --2- 172.21.15.119:0/862900827 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f96280a1e20 0x7f96280a42e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f964019af70 tx=0x7f9634009260 comp rx=0 tx=0).stop 2023-12-07T10:21:56.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.826+0000 7f963f59e700 1 -- 172.21.15.119:0/862900827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96401013d0 msgr2=0x7f9640199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:56.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.826+0000 7f963f59e700 1 --2- 172.21.15.119:0/862900827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96401013d0 0x7f9640199a20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9630005910 tx=0x7f9630004730 comp rx=0 tx=0).stop 2023-12-07T10:21:56.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.826+0000 7f963f59e700 1 -- 172.21.15.119:0/862900827 shutdown_connections 2023-12-07T10:21:56.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.826+0000 7f963f59e700 1 --2- 172.21.15.119:0/862900827 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f96280a1e20 0x7f96280a42e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.826+0000 7f963f59e700 1 --2- 172.21.15.119:0/862900827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9640107470 0x7f964019e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.826+0000 7f963f59e700 1 --2- 172.21.15.119:0/862900827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96401013d0 0x7f9640199a20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.826+0000 7f963f59e700 1 --2- 172.21.15.119:0/862900827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9640101d90 0x7f9640199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:56.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.826+0000 7f963f59e700 1 -- 172.21.15.119:0/862900827 >> 172.21.15.119:0/862900827 conn(0x7f96400fcee0 msgr2=0x7f96400fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:56.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.826+0000 7f963f59e700 1 -- 172.21.15.119:0/862900827 shutdown_connections 2023-12-07T10:21:56.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:56.827+0000 7f963f59e700 1 -- 172.21.15.119:0/862900827 wait complete. 2023-12-07T10:21:56.850 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-07T10:21:57.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.080+0000 7f63e19ff700 1 -- 172.21.15.119:0/1371128588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63dc074300 msgr2=0x7f63dc10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:57.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.080+0000 7f63e19ff700 1 --2- 172.21.15.119:0/1371128588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63dc074300 0x7f63dc10d880 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009990 tx=0x7f63d0009ca0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.080+0000 7f63e19ff700 1 -- 172.21.15.119:0/1371128588 shutdown_connections 2023-12-07T10:21:57.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.080+0000 7f63e19ff700 1 --2- 172.21.15.119:0/1371128588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63dc10ddc0 0x7f63dc1101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.080+0000 7f63e19ff700 1 --2- 172.21.15.119:0/1371128588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63dc0739e0 0x7f63dc073dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.080+0000 7f63e19ff700 1 --2- 172.21.15.119:0/1371128588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63dc074300 0x7f63dc10d880 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.080+0000 7f63e19ff700 1 -- 172.21.15.119:0/1371128588 >> 172.21.15.119:0/1371128588 conn(0x7f63dc0fcff0 msgr2=0x7f63dc0ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:57.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.080+0000 7f63e19ff700 1 -- 172.21.15.119:0/1371128588 shutdown_connections 2023-12-07T10:21:57.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.080+0000 7f63e19ff700 1 -- 172.21.15.119:0/1371128588 wait complete. 2023-12-07T10:21:57.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.081+0000 7f63e19ff700 1 Processor -- start 2023-12-07T10:21:57.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.081+0000 7f63e19ff700 1 -- start start 2023-12-07T10:21:57.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.082+0000 7f63e19ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63dc0739e0 0x7f63dc19fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:57.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.082+0000 7f63e19ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63dc074300 0x7f63dc19ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:57.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.082+0000 7f63e19ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63dc10ddc0 0x7f63dc199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:57.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.082+0000 7f63e19ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc19a290 con 0x7f63dc0739e0 2023-12-07T10:21:57.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.082+0000 7f63e19ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc19a400 con 0x7f63dc10ddc0 2023-12-07T10:21:57.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.082+0000 7f63e19ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc19a5c0 con 0x7f63dc074300 2023-12-07T10:21:57.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.082+0000 7f63da7fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63dc074300 0x7f63dc19ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:57.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.082+0000 7f63da7fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63dc074300 0x7f63dc19ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:40252/0 (socket says 172.21.15.119:40252) 2023-12-07T10:21:57.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.082+0000 7f63da7fc700 1 -- 172.21.15.119:0/4236832849 learned_addr learned my addr 172.21.15.119:0/4236832849 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:57.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.082+0000 7f63db7fe700 1 --2- 172.21.15.119:0/4236832849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63dc10ddc0 0x7f63dc199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:57.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.082+0000 7f63daffd700 1 --2- 172.21.15.119:0/4236832849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63dc0739e0 0x7f63dc19fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:57.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.083+0000 7f63da7fc700 1 -- 172.21.15.119:0/4236832849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63dc10ddc0 msgr2=0x7f63dc199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:57.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.083+0000 7f63da7fc700 1 --2- 172.21.15.119:0/4236832849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63dc10ddc0 0x7f63dc199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.083+0000 7f63da7fc700 1 -- 172.21.15.119:0/4236832849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63dc0739e0 msgr2=0x7f63dc19fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:57.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.083+0000 7f63da7fc700 1 --2- 172.21.15.119:0/4236832849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63dc0739e0 0x7f63dc19fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.083+0000 7f63da7fc700 1 -- 172.21.15.119:0/4236832849 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63cc009520 con 0x7f63dc074300 2023-12-07T10:21:57.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.083+0000 7f63da7fc700 1 --2- 172.21.15.119:0/4236832849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63dc074300 0x7f63dc19ff60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f63d0005910 tx=0x7f63d0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:57.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.083+0000 7f63e09fd700 1 -- 172.21.15.119:0/4236832849 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d001d070 con 0x7f63dc074300 2023-12-07T10:21:57.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.083+0000 7f63e09fd700 1 -- 172.21.15.119:0/4236832849 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63d0022470 con 0x7f63dc074300 2023-12-07T10:21:57.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.083+0000 7f63e19ff700 1 -- 172.21.15.119:0/4236832849 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d4009c40 con 0x7f63dc074300 2023-12-07T10:21:57.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.083+0000 7f63e09fd700 1 -- 172.21.15.119:0/4236832849 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d0017650 con 0x7f63dc074300 2023-12-07T10:21:57.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.083+0000 7f63e19ff700 1 -- 172.21.15.119:0/4236832849 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d00095f0 con 0x7f63dc074300 2023-12-07T10:21:57.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.087+0000 7f63e09fd700 1 -- 172.21.15.119:0/4236832849 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f63d00225e0 con 0x7f63dc074300 2023-12-07T10:21:57.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.088+0000 7f63e09fd700 1 --2- 172.21.15.119:0/4236832849 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f63c40a1c80 0x7f63c40a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:57.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.088+0000 7f63e09fd700 1 -- 172.21.15.119:0/4236832849 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f63d00cee50 con 0x7f63dc074300 2023-12-07T10:21:57.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.088+0000 7f63daffd700 1 --2- 172.21.15.119:0/4236832849 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f63c40a1c80 0x7f63c40a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:57.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.089+0000 7f63daffd700 1 --2- 172.21.15.119:0/4236832849 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f63c40a1c80 0x7f63c40a4140 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009df0 tx=0x7f63cc009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:57.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.089+0000 7f63f5280b80 1 Processor -- start 2023-12-07T10:21:57.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.090+0000 7f63f5280b80 1 -- start start 2023-12-07T10:21:57.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.090+0000 7f63f5280b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b24b40 0x560757b24f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:57.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.090+0000 7f63f5280b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560757b25460 0x560757b28c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:57.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.090+0000 7f63f5280b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560757b29150 0x560757b29570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:57.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.090+0000 7f63f5280b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560757a987c0 con 0x560757b29150 2023-12-07T10:21:57.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.090+0000 7f63f5280b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x560757a7f7a0 con 0x560757b24b40 2023-12-07T10:21:57.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.090+0000 7f63f5280b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560757a7d0c0 con 0x560757b25460 2023-12-07T10:21:57.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.090+0000 7f63daffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b24b40 0x560757b24f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:57.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.090+0000 7f63daffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b24b40 0x560757b24f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45832/0 (socket says 172.21.15.119:45832) 2023-12-07T10:21:57.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.090+0000 7f63daffd700 1 -- 172.21.15.119:0/1774576454 learned_addr learned my addr 172.21.15.119:0/1774576454 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:57.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.090+0000 7f63da7fc700 1 --2- 172.21.15.119:0/1774576454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560757b25460 0x560757b28c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:57.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.090+0000 7f63db7fe700 1 --2- 172.21.15.119:0/1774576454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560757b29150 0x560757b29570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:57.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.091+0000 7f63daffd700 1 -- 172.21.15.119:0/1774576454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560757b25460 msgr2=0x560757b28c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:57.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.091+0000 7f63daffd700 1 --2- 172.21.15.119:0/1774576454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560757b25460 0x560757b28c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.091+0000 7f63daffd700 1 -- 172.21.15.119:0/1774576454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560757b29150 msgr2=0x560757b29570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:57.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.091+0000 7f63daffd700 1 --2- 172.21.15.119:0/1774576454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560757b29150 0x560757b29570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.091+0000 7f63daffd700 1 -- 172.21.15.119:0/1774576454 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607578f5630 con 0x560757b24b40 2023-12-07T10:21:57.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.091+0000 7f63daffd700 1 --2- 172.21.15.119:0/1774576454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b24b40 0x560757b24f20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f63cc0094f0 tx=0x7f63cc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b55cfd0902a1b1 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:57.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.091+0000 7f63c1ffb700 1 -- 172.21.15.119:0/1774576454 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc026630 con 0x560757b24b40 2023-12-07T10:21:57.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.091+0000 7f63c1ffb700 1 -- 172.21.15.119:0/1774576454 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63cc021070 con 0x560757b24b40 2023-12-07T10:21:57.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.091+0000 7f63c1ffb700 1 -- 172.21.15.119:0/1774576454 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc01d6e0 con 0x560757b24b40 2023-12-07T10:21:57.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.092+0000 7f63f5280b80 1 -- 172.21.15.119:0/1774576454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b24b40 msgr2=0x560757b24f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:57.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.092+0000 7f63f5280b80 1 --2- 172.21.15.119:0/1774576454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b24b40 0x560757b24f20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f63cc0094f0 tx=0x7f63cc014000 comp rx=0 tx=0).stop 2023-12-07T10:21:57.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.092+0000 7f63f5280b80 1 -- 172.21.15.119:0/1774576454 shutdown_connections 2023-12-07T10:21:57.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.092+0000 7f63f5280b80 1 --2- 172.21.15.119:0/1774576454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560757b29150 0x560757b29570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.092+0000 7f63f5280b80 1 --2- 172.21.15.119:0/1774576454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b24b40 0x560757b24f20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.092+0000 7f63f5280b80 1 --2- 172.21.15.119:0/1774576454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560757b25460 0x560757b28c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.092+0000 7f63f5280b80 1 -- 172.21.15.119:0/1774576454 >> 172.21.15.119:0/1774576454 conn(0x560757a53360 msgr2=0x560757b1f130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:57.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.092+0000 7f63f5280b80 1 -- 172.21.15.119:0/1774576454 shutdown_connections 2023-12-07T10:21:57.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.092+0000 7f63f5280b80 1 -- 172.21.15.119:0/1774576454 wait complete. 2023-12-07T10:21:57.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.093+0000 7f63f5280b80 1 -- start start 2023-12-07T10:21:57.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.093+0000 7f63f5280b80 1 Processor -- start 2023-12-07T10:21:57.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.093+0000 7f63f5280b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560757b24b40 0x560757b2dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:57.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.093+0000 7f63f5280b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560757b25460 0x560757b2e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:57.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.094+0000 7f63f5280b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b29150 0x560757b2e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:57.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.094+0000 7f63f5280b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560757a789b0 con 0x560757b24b40 2023-12-07T10:21:57.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.094+0000 7f63f5280b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x560757a32560 con 0x560757b29150 2023-12-07T10:21:57.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.094+0000 7f63f5280b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560757a32200 con 0x560757b25460 2023-12-07T10:21:57.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.094+0000 7f63f5280b80 10 client.0 _ll_register_callbacks cb 0x560757a19940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:57.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.094+0000 7f63daffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560757b24b40 0x560757b2dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:57.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.094+0000 7f63da7fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560757b25460 0x560757b2e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:57.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.094+0000 7f63daffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560757b24b40 0x560757b2dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34956/0 (socket says 172.21.15.119:34956) 2023-12-07T10:21:57.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.094+0000 7f63da7fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560757b25460 0x560757b2e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:40264/0 (socket says 172.21.15.119:40264) 2023-12-07T10:21:57.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.094+0000 7f63daffd700 1 -- 172.21.15.119:0/4236407614 learned_addr learned my addr 172.21.15.119:0/4236407614 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:57.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.094+0000 7f63db7fe700 1 --2- 172.21.15.119:0/4236407614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b29150 0x560757b2e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:57.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.094+0000 7f63daffd700 1 -- 172.21.15.119:0/4236407614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560757b25460 msgr2=0x560757b2e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:57.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.094+0000 7f63daffd700 1 --2- 172.21.15.119:0/4236407614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560757b25460 0x560757b2e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.094+0000 7f63daffd700 1 -- 172.21.15.119:0/4236407614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b29150 msgr2=0x560757b2e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:57.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.094+0000 7f63daffd700 1 --2- 172.21.15.119:0/4236407614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b29150 0x560757b2e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.094+0000 7f63daffd700 1 -- 172.21.15.119:0/4236407614 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63cc017070 con 0x560757b24b40 2023-12-07T10:21:57.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.094+0000 7f63db7fe700 1 --2- 172.21.15.119:0/4236407614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b29150 0x560757b2e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:21:57.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.095+0000 7f63daffd700 1 --2- 172.21.15.119:0/4236407614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560757b24b40 0x560757b2dcd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00ec00 tx=0x7f63cc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a336528c79f476b8 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:57.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.095+0000 7f63baffd700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:21:57.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.095+0000 7f63baffd700 1 -- 172.21.15.119:0/4236407614 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc01c620 con 0x560757b24b40 2023-12-07T10:21:57.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.095+0000 7f63f5280b80 1 -- 172.21.15.119:0/4236407614 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560757b17b00 con 0x560757b24b40 2023-12-07T10:21:57.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.095+0000 7f63baffd700 1 -- 172.21.15.119:0/4236407614 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f63cc01c780 con 0x560757b24b40 2023-12-07T10:21:57.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.095+0000 7f63baffd700 1 -- 172.21.15.119:0/4236407614 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63cc01c940 con 0x560757b24b40 2023-12-07T10:21:57.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.095+0000 7f63f5280b80 10 client.16227 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:57.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.095+0000 7f63f5280b80 1 -- 172.21.15.119:0/4236407614 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5607578a1de0 con 0x560757b24b40 2023-12-07T10:21:57.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.096+0000 7f63baffd700 1 -- 172.21.15.119:0/4236407614 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560757b17b00 con 0x560757b24b40 2023-12-07T10:21:57.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.096+0000 7f63baffd700 1 -- 172.21.15.119:0/4236407614 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f63cc003b10 con 0x560757b24b40 2023-12-07T10:21:57.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.096+0000 7f63f5280b80 10 client.16227 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:57.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.096+0000 7f63f5280b80 10 client.16227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:21:57.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.096+0000 7f63f5280b80 20 client.16227 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:57.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.096+0000 7f63f5280b80 1 --2- 172.21.15.119:0/4236407614 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560757b3e380 0x560757b407a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:57.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.096+0000 7f63f5280b80 4 client.16227 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:21:57.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.096+0000 7f63f5280b80 1 -- 172.21.15.119:0/4236407614 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560757af37a0 con 0x560757b3e380 2023-12-07T10:21:57.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.096+0000 7f63da7fc700 1 --2- 172.21.15.119:0/4236407614 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560757b3e380 0x560757b407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:57.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.097+0000 7f63baffd700 10 client.16227 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:57.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.097+0000 7f63da7fc700 1 --2- 172.21.15.119:0/4236407614 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560757b3e380 0x560757b407a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bd5e4391f9eda55 server_cookie=2ad5bc9ec14e508d in_seq=0 out_seq=0 2023-12-07T10:21:57.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.098+0000 7f63baffd700 1 -- 172.21.15.119:0/4236407614 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560757af37a0 con 0x560757b3e380 2023-12-07T10:21:57.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.098+0000 7f63baffd700 10 client.16227 handle_command_reply: tid=0 2023-12-07T10:21:57.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.098+0000 7f63f5280b80 1 client.16227 shutdown 2023-12-07T10:21:57.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.098+0000 7f63f5280b80 1 -- 172.21.15.119:0/4236407614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560757b24b40 msgr2=0x560757b2dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:57.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.098+0000 7f63f5280b80 1 --2- 172.21.15.119:0/4236407614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560757b24b40 0x560757b2dcd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00ec00 tx=0x7f63cc009c30 comp rx=0 tx=0).stop 2023-12-07T10:21:57.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.098+0000 7f63f5280b80 1 -- 172.21.15.119:0/4236407614 shutdown_connections 2023-12-07T10:21:57.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.098+0000 7f63f5280b80 1 --2- 172.21.15.119:0/4236407614 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560757b3e380 0x560757b407a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.098+0000 7f63f5280b80 1 --2- 172.21.15.119:0/4236407614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b29150 0x560757b2e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.098+0000 7f63f5280b80 1 --2- 172.21.15.119:0/4236407614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560757b24b40 0x560757b2dcd0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.098+0000 7f63f5280b80 1 --2- 172.21.15.119:0/4236407614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560757b25460 0x560757b2e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.098+0000 7f63f5280b80 1 -- 172.21.15.119:0/4236407614 >> 172.21.15.119:0/4236407614 conn(0x560757a53360 msgr2=0x560757b19d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:57.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.098+0000 7f63baffd700 0 client.16227 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:57.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.098+0000 7f63f5280b80 1 -- 172.21.15.119:0/4236407614 shutdown_connections 2023-12-07T10:21:57.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.098+0000 7f63f5280b80 1 -- 172.21.15.119:0/4236407614 wait complete. 2023-12-07T10:21:57.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.099+0000 7f63f5280b80 20 client.16227 trim_cache size 0 max 16384 2023-12-07T10:21:57.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.119+0000 7f63f5280b80 1 Processor -- start 2023-12-07T10:21:57.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.119+0000 7f63f5280b80 1 -- start start 2023-12-07T10:21:57.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.119+0000 7f63f5280b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560757b24b40 0x560757b2e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:57.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.119+0000 7f63f5280b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560757b25460 0x560757b450c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:57.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.119+0000 7f63f5280b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b29150 0x560757b26b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:57.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.119+0000 7f63f5280b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607578f7920 con 0x560757b24b40 2023-12-07T10:21:57.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.119+0000 7f63f5280b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x560757b31130 con 0x560757b29150 2023-12-07T10:21:57.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.119+0000 7f63f5280b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560757b20800 con 0x560757b25460 2023-12-07T10:21:57.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.119+0000 7f63da7fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560757b25460 0x560757b450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:57.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.119+0000 7f63daffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560757b24b40 0x560757b2e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:57.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.119+0000 7f63db7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b29150 0x560757b26b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:57.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.120+0000 7f63db7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b29150 0x560757b26b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45858/0 (socket says 172.21.15.119:45858) 2023-12-07T10:21:57.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.120+0000 7f63da7fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560757b25460 0x560757b450c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:40268/0 (socket says 172.21.15.119:40268) 2023-12-07T10:21:57.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.120+0000 7f63db7fe700 1 -- 172.21.15.119:0/77903852 learned_addr learned my addr 172.21.15.119:0/77903852 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:57.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.120+0000 7f63db7fe700 1 -- 172.21.15.119:0/77903852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560757b25460 msgr2=0x560757b450c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:57.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.120+0000 7f63db7fe700 1 --2- 172.21.15.119:0/77903852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560757b25460 0x560757b450c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.120+0000 7f63db7fe700 1 -- 172.21.15.119:0/77903852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560757b24b40 msgr2=0x560757b2e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:57.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.120+0000 7f63db7fe700 1 --2- 172.21.15.119:0/77903852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560757b24b40 0x560757b2e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.120+0000 7f63db7fe700 1 -- 172.21.15.119:0/77903852 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56075788ac60 con 0x560757b29150 2023-12-07T10:21:57.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.120+0000 7f63daffd700 1 --2- 172.21.15.119:0/77903852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560757b24b40 0x560757b2e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:21:57.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.120+0000 7f63db7fe700 1 --2- 172.21.15.119:0/77903852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b29150 0x560757b26b80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f63d4007670 tx=0x7f63d400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=934fec6bfb17fdde server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:57.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.120+0000 7f63c27fc700 1 -- 172.21.15.119:0/77903852 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d400e040 con 0x560757b29150 2023-12-07T10:21:57.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.121+0000 7f63c27fc700 1 -- 172.21.15.119:0/77903852 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63d400f040 con 0x560757b29150 2023-12-07T10:21:57.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.121+0000 7f63c27fc700 1 -- 172.21.15.119:0/77903852 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d401e9b0 con 0x560757b29150 2023-12-07T10:21:57.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.121+0000 7f63f5280b80 1 -- 172.21.15.119:0/77903852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b29150 msgr2=0x560757b26b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:57.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.121+0000 7f63f5280b80 1 --2- 172.21.15.119:0/77903852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b29150 0x560757b26b80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f63d4007670 tx=0x7f63d400b040 comp rx=0 tx=0).stop 2023-12-07T10:21:57.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.121+0000 7f63f5280b80 1 -- 172.21.15.119:0/77903852 shutdown_connections 2023-12-07T10:21:57.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.121+0000 7f63f5280b80 1 --2- 172.21.15.119:0/77903852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b29150 0x560757b26b80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.121+0000 7f63f5280b80 1 --2- 172.21.15.119:0/77903852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560757b24b40 0x560757b2e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.121+0000 7f63f5280b80 1 --2- 172.21.15.119:0/77903852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560757b25460 0x560757b450c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.121+0000 7f63f5280b80 1 -- 172.21.15.119:0/77903852 >> 172.21.15.119:0/77903852 conn(0x560757a53360 msgr2=0x560757b27870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:57.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.122+0000 7f63f5280b80 1 -- 172.21.15.119:0/77903852 shutdown_connections 2023-12-07T10:21:57.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.122+0000 7f63f5280b80 1 -- 172.21.15.119:0/77903852 wait complete. 2023-12-07T10:21:57.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.122+0000 7f63f5280b80 1 -- start start 2023-12-07T10:21:57.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.123+0000 7f63f5280b80 1 Processor -- start 2023-12-07T10:21:57.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.123+0000 7f63f5280b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b24b40 0x560757b350a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:57.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.123+0000 7f63f5280b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560757b25460 0x560757b355e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:57.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.123+0000 7f63f5280b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560757b29150 0x560757b33710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:57.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.123+0000 7f63f5280b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560757b33d30 con 0x560757b29150 2023-12-07T10:21:57.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.123+0000 7f63f5280b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x560757b33ea0 con 0x560757b24b40 2023-12-07T10:21:57.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.123+0000 7f63f5280b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560757b34060 con 0x560757b25460 2023-12-07T10:21:57.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.123+0000 7f63f5280b80 10 client.0 _ll_register_callbacks cb 0x560757a19940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:21:57.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.123+0000 7f63daffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b24b40 0x560757b350a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:57.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.123+0000 7f63daffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b24b40 0x560757b350a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45864/0 (socket says 172.21.15.119:45864) 2023-12-07T10:21:57.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.123+0000 7f63daffd700 1 -- 172.21.15.119:0/1314731751 learned_addr learned my addr 172.21.15.119:0/1314731751 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:21:57.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.124+0000 7f63da7fc700 1 --2- 172.21.15.119:0/1314731751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560757b25460 0x560757b355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:57.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.124+0000 7f63db7fe700 1 --2- 172.21.15.119:0/1314731751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560757b29150 0x560757b33710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:57.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.124+0000 7f63daffd700 1 -- 172.21.15.119:0/1314731751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560757b25460 msgr2=0x560757b355e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:57.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.124+0000 7f63daffd700 1 --2- 172.21.15.119:0/1314731751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560757b25460 0x560757b355e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.124+0000 7f63daffd700 1 -- 172.21.15.119:0/1314731751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560757b29150 msgr2=0x560757b33710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:21:57.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.124+0000 7f63daffd700 1 --2- 172.21.15.119:0/1314731751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560757b29150 0x560757b33710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.124+0000 7f63daffd700 1 -- 172.21.15.119:0/1314731751 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5607578a1de0 con 0x560757b24b40 2023-12-07T10:21:57.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.124+0000 7f63daffd700 1 --2- 172.21.15.119:0/1314731751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b24b40 0x560757b350a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f63cc01cf20 tx=0x7f63cc00e3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68c96d4cdd29b2fd server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:21:57.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.124+0000 7f63baffd700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:21:57.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.124+0000 7f63baffd700 1 -- 172.21.15.119:0/1314731751 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc009340 con 0x560757b24b40 2023-12-07T10:21:57.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.124+0000 7f63f5280b80 1 -- 172.21.15.119:0/1314731751 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63cc021070 con 0x560757b24b40 2023-12-07T10:21:57.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.124+0000 7f63baffd700 1 -- 172.21.15.119:0/1314731751 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f63cc026b10 con 0x560757b24b40 2023-12-07T10:21:57.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.124+0000 7f63baffd700 1 -- 172.21.15.119:0/1314731751 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63cc00ec30 con 0x560757b24b40 2023-12-07T10:21:57.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.125+0000 7f63f5280b80 10 client.26237 fetch_fsmap learned FSMap version 30 2023-12-07T10:21:57.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.125+0000 7f63f5280b80 1 -- 172.21.15.119:0/1314731751 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f63d40093a0 con 0x560757b24b40 2023-12-07T10:21:57.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.125+0000 7f63baffd700 1 -- 172.21.15.119:0/1314731751 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63cc021070 con 0x560757b24b40 2023-12-07T10:21:57.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.125+0000 7f63baffd700 1 -- 172.21.15.119:0/1314731751 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f63cc036020 con 0x560757b24b40 2023-12-07T10:21:57.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.125+0000 7f63f5280b80 10 client.26237 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:21:57.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.125+0000 7f63f5280b80 10 client.26237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:21:57.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.125+0000 7f63f5280b80 20 client.26237 populate_metadata read hostname 'smithi119' 2023-12-07T10:21:57.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.125+0000 7f63f5280b80 1 --2- 172.21.15.119:0/1314731751 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560757b958c0 0x560757b97ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:21:57.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.125+0000 7f63f5280b80 4 client.26237 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-12-07T10:21:57.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.125+0000 7f63f5280b80 1 -- 172.21.15.119:0/1314731751 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560757a3efd0 con 0x560757b958c0 2023-12-07T10:21:57.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.125+0000 7f63da7fc700 1 --2- 172.21.15.119:0/1314731751 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560757b958c0 0x560757b97ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:21:57.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.126+0000 7f63da7fc700 1 --2- 172.21.15.119:0/1314731751 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560757b958c0 0x560757b97ce0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=967689bba6c16a3a server_cookie=f04a520f7d7d6f98 in_seq=0 out_seq=0 2023-12-07T10:21:57.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.126+0000 7f63baffd700 10 client.26237 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:57.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.126+0000 7f63baffd700 1 -- 172.21.15.119:0/1314731751 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560757a3efd0 con 0x560757b958c0 2023-12-07T10:21:57.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.126+0000 7f63baffd700 10 client.26237 handle_command_reply: tid=0 2023-12-07T10:21:57.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.126+0000 7f63f5280b80 1 client.26237 shutdown 2023-12-07T10:21:57.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.126+0000 7f63f5280b80 1 -- 172.21.15.119:0/1314731751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b24b40 msgr2=0x560757b350a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:57.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.126+0000 7f63f5280b80 1 --2- 172.21.15.119:0/1314731751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b24b40 0x560757b350a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f63cc01cf20 tx=0x7f63cc00e3b0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.126+0000 7f63f5280b80 1 -- 172.21.15.119:0/1314731751 shutdown_connections 2023-12-07T10:21:57.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.126+0000 7f63f5280b80 1 --2- 172.21.15.119:0/1314731751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560757b29150 0x560757b33710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.126+0000 7f63f5280b80 1 --2- 172.21.15.119:0/1314731751 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560757b958c0 0x560757b97ce0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.126+0000 7f63f5280b80 1 --2- 172.21.15.119:0/1314731751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560757b24b40 0x560757b350a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.126+0000 7f63f5280b80 1 --2- 172.21.15.119:0/1314731751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560757b25460 0x560757b355e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.126+0000 7f63baffd700 0 client.26237 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:21:57.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.126+0000 7f63f5280b80 1 -- 172.21.15.119:0/1314731751 >> 172.21.15.119:0/1314731751 conn(0x560757a53360 msgr2=0x560757b450c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:57.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.127+0000 7f63f5280b80 1 -- 172.21.15.119:0/1314731751 shutdown_connections 2023-12-07T10:21:57.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.127+0000 7f63f5280b80 1 -- 172.21.15.119:0/1314731751 wait complete. 2023-12-07T10:21:57.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.127+0000 7f63f5280b80 20 client.26237 trim_cache size 0 max 16384 2023-12-07T10:21:57.131 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:21:57.132 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-07T10:21:57.132 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": { 2023-12-07T10:21:57.132 INFO:teuthology.orchestra.run.smithi119.stdout: "4e857eee-9284-44c1-9aed-0fee3db97193": { 2023-12-07T10:21:57.132 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/", 2023-12-07T10:21:57.132 INFO:teuthology.orchestra.run.smithi119.stdout: "tag": "4e857eee-9284-44c1-9aed-0fee3db97193", 2023-12-07T10:21:57.132 INFO:teuthology.orchestra.run.smithi119.stdout: "options": "recursive,force" 2023-12-07T10:21:57.132 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:21:57.132 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:21:57.132 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:21:57.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.127+0000 7f63e19ff700 1 -- 172.21.15.119:0/4236832849 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f63c40a1c80 msgr2=0x7f63c40a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:57.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.127+0000 7f63e19ff700 1 --2- 172.21.15.119:0/4236832849 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f63c40a1c80 0x7f63c40a4140 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009df0 tx=0x7f63cc009260 comp rx=0 tx=0).stop 2023-12-07T10:21:57.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.127+0000 7f63e19ff700 1 -- 172.21.15.119:0/4236832849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63dc074300 msgr2=0x7f63dc19ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:21:57.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.127+0000 7f63e19ff700 1 --2- 172.21.15.119:0/4236832849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63dc074300 0x7f63dc19ff60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f63d0005910 tx=0x7f63d0004730 comp rx=0 tx=0).stop 2023-12-07T10:21:57.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.128+0000 7f63e19ff700 1 -- 172.21.15.119:0/4236832849 shutdown_connections 2023-12-07T10:21:57.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.128+0000 7f63e19ff700 1 --2- 172.21.15.119:0/4236832849 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f63c40a1c80 0x7f63c40a4140 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.128+0000 7f63e19ff700 1 --2- 172.21.15.119:0/4236832849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63dc10ddc0 0x7f63dc199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.128+0000 7f63e19ff700 1 --2- 172.21.15.119:0/4236832849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63dc0739e0 0x7f63dc19fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.128+0000 7f63e19ff700 1 --2- 172.21.15.119:0/4236832849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63dc074300 0x7f63dc19ff60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:21:57.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.128+0000 7f63e19ff700 1 -- 172.21.15.119:0/4236832849 >> 172.21.15.119:0/4236832849 conn(0x7f63dc0fcff0 msgr2=0x7f63dc0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:21:57.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.128+0000 7f63e19ff700 1 -- 172.21.15.119:0/4236832849 shutdown_connections 2023-12-07T10:21:57.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:21:57.128+0000 7f63e19ff700 1 -- 172.21.15.119:0/4236832849 wait complete. 2023-12-07T10:21:57.150 INFO:tasks.cephfs.filesystem:scrub status for tag:4e857eee-9284-44c1-9aed-0fee3db97193 - {'path': '/', 'tag': '4e857eee-9284-44c1-9aed-0fee3db97193', 'options': 'recursive,force'} 2023-12-07T10:21:57.539 INFO:tasks.workunit.client.0.smithi119.stdout: 1 28167 16.92 MB/sec warmup 10 sec latency 10.212 ms 2023-12-07T10:21:58.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:57 smithi119 ceph-mon[97908]: scrub queued for path: / 2023-12-07T10:21:58.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:57 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:21:58.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:57 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:21:58.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:57 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:21:58.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:57 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:21:58.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:57 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:21:58.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:57 smithi119 ceph-mon[97908]: pgmap v49: 129 pgs: 129 active+clean; 12 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 1.1 MiB/s wr, 155 op/s 2023-12-07T10:21:58.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:57 smithi119 ceph-mon[97908]: mgrmap e26: x(active, since 92s), standbys: z, y 2023-12-07T10:21:58.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:57 smithi159 ceph-mon[134726]: scrub queued for path: / 2023-12-07T10:21:58.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:57 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:21:58.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:57 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:21:58.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:57 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:21:58.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:57 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:21:58.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:57 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:21:58.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:57 smithi159 ceph-mon[134726]: pgmap v49: 129 pgs: 129 active+clean; 12 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 1.1 MiB/s wr, 155 op/s 2023-12-07T10:21:58.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:57 smithi159 ceph-mon[134726]: mgrmap e26: x(active, since 92s), standbys: z, y 2023-12-07T10:21:58.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:57 smithi153 ceph-mon[133625]: scrub queued for path: / 2023-12-07T10:21:58.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:57 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:21:58.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:57 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:21:58.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:57 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:21:58.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:57 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:21:58.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:57 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:21:58.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:57 smithi153 ceph-mon[133625]: pgmap v49: 129 pgs: 129 active+clean; 12 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 1.1 MiB/s wr, 155 op/s 2023-12-07T10:21:58.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:57 smithi153 ceph-mon[133625]: mgrmap e26: x(active, since 92s), standbys: z, y 2023-12-07T10:21:58.539 INFO:tasks.workunit.client.0.smithi119.stdout: 1 30373 16.31 MB/sec warmup 11 sec latency 18.702 ms 2023-12-07T10:21:59.539 INFO:tasks.workunit.client.0.smithi119.stdout: 1 32878 16.15 MB/sec warmup 12 sec latency 6.600 ms 2023-12-07T10:22:00.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:21:59 smithi119 ceph-mon[97908]: pgmap v50: 129 pgs: 129 active+clean; 12 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.1 MiB/s wr, 150 op/s 2023-12-07T10:22:00.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:21:59 smithi159 ceph-mon[134726]: pgmap v50: 129 pgs: 129 active+clean; 12 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.1 MiB/s wr, 150 op/s 2023-12-07T10:22:00.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:21:59 smithi153 ceph-mon[133625]: pgmap v50: 129 pgs: 129 active+clean; 12 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.1 MiB/s wr, 150 op/s 2023-12-07T10:22:00.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.348+0000 7f2d943ff700 1 -- 172.21.15.119:0/634810641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d8c106a80 msgr2=0x7f2d8c10e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:00.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.348+0000 7f2d943ff700 1 --2- 172.21.15.119:0/634810641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d8c106a80 0x7f2d8c10e360 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400b0a0 tx=0x7f2d8400b3b0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.348+0000 7f2d943ff700 1 -- 172.21.15.119:0/634810641 shutdown_connections 2023-12-07T10:22:00.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.348+0000 7f2d943ff700 1 --2- 172.21.15.119:0/634810641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d8c106a80 0x7f2d8c10e360 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.348+0000 7f2d943ff700 1 --2- 172.21.15.119:0/634810641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d8c068ce0 0x7f2d8c0690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.348+0000 7f2d943ff700 1 --2- 172.21.15.119:0/634810641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c1060c0 0x7f2d8c106540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.348+0000 7f2d943ff700 1 -- 172.21.15.119:0/634810641 >> 172.21.15.119:0/634810641 conn(0x7f2d8c0775b0 msgr2=0x7f2d8c0779c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.348+0000 7f2d943ff700 1 -- 172.21.15.119:0/634810641 shutdown_connections 2023-12-07T10:22:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.348+0000 7f2d943ff700 1 -- 172.21.15.119:0/634810641 wait complete. 2023-12-07T10:22:00.352 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.349+0000 7f2d943ff700 1 Processor -- start 2023-12-07T10:22:00.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.349+0000 7f2d943ff700 1 -- start start 2023-12-07T10:22:00.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.350+0000 7f2d943ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d8c068ce0 0x7f2d8c19f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:00.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.350+0000 7f2d943ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c1060c0 0x7f2d8c19ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:00.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.350+0000 7f2d943ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d8c106a80 0x7f2d8c199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:00.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.350+0000 7f2d943ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c19a1c0 con 0x7f2d8c1060c0 2023-12-07T10:22:00.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.350+0000 7f2d943ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c19a330 con 0x7f2d8c068ce0 2023-12-07T10:22:00.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.350+0000 7f2d943ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c19a4f0 con 0x7f2d8c106a80 2023-12-07T10:22:00.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.350+0000 7f2d9199a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c1060c0 0x7f2d8c19ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:00.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.350+0000 7f2d9299c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d8c106a80 0x7f2d8c199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:00.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.350+0000 7f2d9199a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c1060c0 0x7f2d8c19ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34998/0 (socket says 172.21.15.119:34998) 2023-12-07T10:22:00.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.350+0000 7f2d9199a700 1 -- 172.21.15.119:0/889981490 learned_addr learned my addr 172.21.15.119:0/889981490 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:00.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.350+0000 7f2d9219b700 1 --2- 172.21.15.119:0/889981490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d8c068ce0 0x7f2d8c19f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:00.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.350+0000 7f2d9299c700 1 -- 172.21.15.119:0/889981490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d8c068ce0 msgr2=0x7f2d8c19f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:00.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.350+0000 7f2d9299c700 1 --2- 172.21.15.119:0/889981490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d8c068ce0 0x7f2d8c19f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.351+0000 7f2d9299c700 1 -- 172.21.15.119:0/889981490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c1060c0 msgr2=0x7f2d8c19ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:00.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.351+0000 7f2d9299c700 1 --2- 172.21.15.119:0/889981490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c1060c0 0x7f2d8c19ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.351+0000 7f2d9299c700 1 -- 172.21.15.119:0/889981490 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d84009d00 con 0x7f2d8c106a80 2023-12-07T10:22:00.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.351+0000 7f2d9299c700 1 --2- 172.21.15.119:0/889981490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d8c106a80 0x7f2d8c199b60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400b070 tx=0x7f2d84007860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:00.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.351+0000 7f2d837fe700 1 -- 172.21.15.119:0/889981490 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8400e040 con 0x7f2d8c106a80 2023-12-07T10:22:00.355 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.351+0000 7f2d837fe700 1 -- 172.21.15.119:0/889981490 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d8401d470 con 0x7f2d8c106a80 2023-12-07T10:22:00.355 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.351+0000 7f2d837fe700 1 -- 172.21.15.119:0/889981490 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84005280 con 0x7f2d8c106a80 2023-12-07T10:22:00.355 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.351+0000 7f2d943ff700 1 -- 172.21.15.119:0/889981490 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8c19a770 con 0x7f2d8c106a80 2023-12-07T10:22:00.355 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.351+0000 7f2d943ff700 1 -- 172.21.15.119:0/889981490 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8c1a69d0 con 0x7f2d8c106a80 2023-12-07T10:22:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.355+0000 7f2d837fe700 1 -- 172.21.15.119:0/889981490 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f2d8401d5e0 con 0x7f2d8c106a80 2023-12-07T10:22:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.356+0000 7f2da7dbbb80 1 Processor -- start 2023-12-07T10:22:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.357+0000 7f2da7dbbb80 1 -- start start 2023-12-07T10:22:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.357+0000 7f2da7dbbb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf54310a60 0x55cf54310e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.357+0000 7f2da7dbbb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf54311380 0x55cf5430b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.357+0000 7f2da7dbbb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf5430bad0 0x55cf5430bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.357+0000 7f2da7dbbb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5427f760 con 0x55cf5430bad0 2023-12-07T10:22:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.357+0000 7f2da7dbbb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf54266740 con 0x55cf54310a60 2023-12-07T10:22:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.357+0000 7f2da7dbbb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf54264060 con 0x55cf54311380 2023-12-07T10:22:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.357+0000 7f2d9199a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf54310a60 0x55cf54310e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.357+0000 7f2d9199a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf54310a60 0x55cf54310e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45898/0 (socket says 172.21.15.119:45898) 2023-12-07T10:22:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.357+0000 7f2d9199a700 1 -- 172.21.15.119:0/1874342297 learned_addr learned my addr 172.21.15.119:0/1874342297 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.357+0000 7f2d9199a700 1 --2- 172.21.15.119:0/1874342297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf5430bad0 0x55cf5430bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.357+0000 7f2d9219b700 1 --2- 172.21.15.119:0/1874342297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf54311380 0x55cf5430b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.358+0000 7f2d837fe700 1 --2- 172.21.15.119:0/889981490 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f2d780a1d00 0x7f2d780a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.358+0000 7f2d837fe700 1 -- 172.21.15.119:0/889981490 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f2d840ceee0 con 0x7f2d8c106a80 2023-12-07T10:22:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.358+0000 7f2d9219b700 1 -- 172.21.15.119:0/1874342297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf54310a60 msgr2=0x55cf54310e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.358+0000 7f2d9219b700 1 --2- 172.21.15.119:0/1874342297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf54310a60 0x55cf54310e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.358+0000 7f2d9219b700 1 -- 172.21.15.119:0/1874342297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf5430bad0 msgr2=0x55cf5430bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.358+0000 7f2d9219b700 1 --2- 172.21.15.119:0/1874342297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf5430bad0 0x55cf5430bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.358+0000 7f2d9219b700 1 -- 172.21.15.119:0/1874342297 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf540dc5d0 con 0x55cf54311380 2023-12-07T10:22:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.358+0000 7f2d9199a700 1 --2- 172.21.15.119:0/1874342297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf54310a60 0x55cf54310e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.358+0000 7f2d9299c700 1 --2- 172.21.15.119:0/889981490 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f2d780a1d00 0x7f2d780a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.358+0000 7f2d9219b700 1 --2- 172.21.15.119:0/1874342297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf54311380 0x55cf5430b540 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009cb0 tx=0x7f2d88011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c17c3292fbcb2e73 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.358+0000 7f2d80bfe700 1 -- 172.21.15.119:0/1874342297 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d88011d20 con 0x55cf54311380 2023-12-07T10:22:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.359+0000 7f2d9299c700 1 --2- 172.21.15.119:0/889981490 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f2d780a1d00 0x7f2d780a41c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2d840983d0 tx=0x7f2d8400b3b0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.359+0000 7f2d80bfe700 1 -- 172.21.15.119:0/1874342297 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d88011e80 con 0x55cf54311380 2023-12-07T10:22:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.359+0000 7f2d80bfe700 1 -- 172.21.15.119:0/1874342297 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d88005280 con 0x55cf54311380 2023-12-07T10:22:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.359+0000 7f2da7dbbb80 1 -- 172.21.15.119:0/1874342297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf54311380 msgr2=0x55cf5430b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.359+0000 7f2da7dbbb80 1 --2- 172.21.15.119:0/1874342297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf54311380 0x55cf5430b540 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009cb0 tx=0x7f2d88011440 comp rx=0 tx=0).stop 2023-12-07T10:22:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.359+0000 7f2da7dbbb80 1 -- 172.21.15.119:0/1874342297 shutdown_connections 2023-12-07T10:22:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.359+0000 7f2da7dbbb80 1 --2- 172.21.15.119:0/1874342297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf5430bad0 0x55cf5430bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.359+0000 7f2da7dbbb80 1 --2- 172.21.15.119:0/1874342297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf54310a60 0x55cf54310e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.359+0000 7f2da7dbbb80 1 --2- 172.21.15.119:0/1874342297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf54311380 0x55cf5430b540 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.359+0000 7f2da7dbbb80 1 -- 172.21.15.119:0/1874342297 >> 172.21.15.119:0/1874342297 conn(0x55cf5423a300 msgr2=0x55cf54305b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.360+0000 7f2da7dbbb80 1 -- 172.21.15.119:0/1874342297 shutdown_connections 2023-12-07T10:22:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.360+0000 7f2da7dbbb80 1 -- 172.21.15.119:0/1874342297 wait complete. 2023-12-07T10:22:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.360+0000 7f2da7dbbb80 1 -- start start 2023-12-07T10:22:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.360+0000 7f2da7dbbb80 1 Processor -- start 2023-12-07T10:22:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.361+0000 7f2da7dbbb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf5430bad0 0x55cf54314800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.361+0000 7f2da7dbbb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf54310a60 0x55cf54314d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.361+0000 7f2da7dbbb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf54311380 0x55cf54315280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.361+0000 7f2da7dbbb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5425f950 con 0x55cf54311380 2023-12-07T10:22:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.361+0000 7f2da7dbbb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf54219500 con 0x55cf54310a60 2023-12-07T10:22:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.361+0000 7f2da7dbbb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf542191a0 con 0x55cf5430bad0 2023-12-07T10:22:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.361+0000 7f2da7dbbb80 10 client.0 _ll_register_callbacks cb 0x55cf542008e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:22:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.361+0000 7f2d9199a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf5430bad0 0x55cf54314800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.361+0000 7f2d9219b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf54310a60 0x55cf54314d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.361+0000 7f2d9219b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf54310a60 0x55cf54314d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45908/0 (socket says 172.21.15.119:45908) 2023-12-07T10:22:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.361+0000 7f2d9219b700 1 -- 172.21.15.119:0/4068158307 learned_addr learned my addr 172.21.15.119:0/4068158307 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.361+0000 7f2d9199a700 1 --2- 172.21.15.119:0/4068158307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf54311380 0x55cf54315280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.362+0000 7f2d9199a700 1 -- 172.21.15.119:0/4068158307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf54310a60 msgr2=0x55cf54314d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.362+0000 7f2d9199a700 1 --2- 172.21.15.119:0/4068158307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf54310a60 0x55cf54314d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.362+0000 7f2d9199a700 1 -- 172.21.15.119:0/4068158307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf54311380 msgr2=0x55cf54315280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.362+0000 7f2d9199a700 1 --2- 172.21.15.119:0/4068158307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf54311380 0x55cf54315280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.362+0000 7f2d9199a700 1 -- 172.21.15.119:0/4068158307 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7c01b040 con 0x55cf5430bad0 2023-12-07T10:22:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.362+0000 7f2d9219b700 1 --2- 172.21.15.119:0/4068158307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf54310a60 0x55cf54314d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.362+0000 7f2d9199a700 1 --2- 172.21.15.119:0/4068158307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf5430bad0 0x55cf54314800 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x55cf54307670 tx=0x7f2d7c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b0018ac7aa816c5 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.362+0000 7f2d75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:22:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.362+0000 7f2d75ffb700 1 -- 172.21.15.119:0/4068158307 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c004830 con 0x55cf5430bad0 2023-12-07T10:22:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.362+0000 7f2da7dbbb80 1 -- 172.21.15.119:0/4068158307 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cf542da740 con 0x55cf5430bad0 2023-12-07T10:22:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.362+0000 7f2d75ffb700 1 -- 172.21.15.119:0/4068158307 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f2d7c004990 con 0x55cf5430bad0 2023-12-07T10:22:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.362+0000 7f2d75ffb700 1 -- 172.21.15.119:0/4068158307 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d7c004b50 con 0x55cf5430bad0 2023-12-07T10:22:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.362+0000 7f2da7dbbb80 10 client.26242 fetch_fsmap learned FSMap version 30 2023-12-07T10:22:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.362+0000 7f2da7dbbb80 1 -- 172.21.15.119:0/4068158307 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f2d880098e0 con 0x55cf5430bad0 2023-12-07T10:22:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.363+0000 7f2d75ffb700 1 -- 172.21.15.119:0/4068158307 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cf542da740 con 0x55cf5430bad0 2023-12-07T10:22:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.363+0000 7f2d75ffb700 1 -- 172.21.15.119:0/4068158307 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2d7c0215a0 con 0x55cf5430bad0 2023-12-07T10:22:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.363+0000 7f2da7dbbb80 10 client.26242 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:22:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.363+0000 7f2da7dbbb80 10 client.26242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:22:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.363+0000 7f2da7dbbb80 20 client.26242 populate_metadata read hostname 'smithi119' 2023-12-07T10:22:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.363+0000 7f2da7dbbb80 1 --2- 172.21.15.119:0/4068158307 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55cf54324eb0 0x55cf543272d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.363+0000 7f2da7dbbb80 4 client.26242 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:22:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.363+0000 7f2da7dbbb80 1 -- 172.21.15.119:0/4068158307 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cf54225f70 con 0x55cf54324eb0 2023-12-07T10:22:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.364+0000 7f2d9219b700 1 --2- 172.21.15.119:0/4068158307 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55cf54324eb0 0x55cf543272d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.364+0000 7f2d75ffb700 10 client.26242 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.364+0000 7f2d9219b700 1 --2- 172.21.15.119:0/4068158307 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55cf54324eb0 0x55cf543272d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=295efc62909c09b5 server_cookie=a0b93c15bf4d10c4 in_seq=0 out_seq=0 2023-12-07T10:22:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.365+0000 7f2d75ffb700 1 -- 172.21.15.119:0/4068158307 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cf54225f70 con 0x55cf54324eb0 2023-12-07T10:22:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.365+0000 7f2d75ffb700 10 client.26242 handle_command_reply: tid=0 2023-12-07T10:22:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.365+0000 7f2da7dbbb80 1 client.26242 shutdown 2023-12-07T10:22:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.365+0000 7f2da7dbbb80 1 -- 172.21.15.119:0/4068158307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf5430bad0 msgr2=0x55cf54314800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.365+0000 7f2da7dbbb80 1 --2- 172.21.15.119:0/4068158307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf5430bad0 0x55cf54314800 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x55cf54307670 tx=0x7f2d7c0047c0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.365+0000 7f2da7dbbb80 1 -- 172.21.15.119:0/4068158307 shutdown_connections 2023-12-07T10:22:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.365+0000 7f2da7dbbb80 1 --2- 172.21.15.119:0/4068158307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf54311380 0x55cf54315280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.365+0000 7f2da7dbbb80 1 --2- 172.21.15.119:0/4068158307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf5430bad0 0x55cf54314800 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.365+0000 7f2da7dbbb80 1 --2- 172.21.15.119:0/4068158307 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55cf54324eb0 0x55cf543272d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.365+0000 7f2da7dbbb80 1 --2- 172.21.15.119:0/4068158307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf54310a60 0x55cf54314d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.365+0000 7f2d75ffb700 0 client.26242 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.365+0000 7f2da7dbbb80 1 -- 172.21.15.119:0/4068158307 >> 172.21.15.119:0/4068158307 conn(0x55cf5423a300 msgr2=0x55cf54300980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.365+0000 7f2da7dbbb80 1 -- 172.21.15.119:0/4068158307 shutdown_connections 2023-12-07T10:22:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.365+0000 7f2da7dbbb80 1 -- 172.21.15.119:0/4068158307 wait complete. 2023-12-07T10:22:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.366+0000 7f2da7dbbb80 20 client.26242 trim_cache size 0 max 16384 2023-12-07T10:22:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.384+0000 7f2da7dbbb80 1 Processor -- start 2023-12-07T10:22:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.384+0000 7f2da7dbbb80 1 -- start start 2023-12-07T10:22:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.385+0000 7f2da7dbbb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf5430bad0 0x55cf543140c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.385+0000 7f2da7dbbb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf54310a60 0x55cf5430f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.385+0000 7f2da7dbbb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf54311380 0x55cf5430fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.385+0000 7f2da7dbbb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf540de8c0 con 0x55cf54310a60 2023-12-07T10:22:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.385+0000 7f2da7dbbb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf54306480 con 0x55cf54311380 2023-12-07T10:22:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.385+0000 7f2da7dbbb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5430f510 con 0x55cf5430bad0 2023-12-07T10:22:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.385+0000 7f2d9219b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf54310a60 0x55cf5430f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.385+0000 7f2d9219b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf54310a60 0x55cf5430f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35024/0 (socket says 172.21.15.119:35024) 2023-12-07T10:22:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.385+0000 7f2d9199a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf5430bad0 0x55cf543140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.385+0000 7f2d9219b700 1 -- 172.21.15.119:0/4224220746 learned_addr learned my addr 172.21.15.119:0/4224220746 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.385+0000 7f2d9199a700 1 --2- 172.21.15.119:0/4224220746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf54311380 0x55cf5430fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.385+0000 7f2d9219b700 1 -- 172.21.15.119:0/4224220746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf5430bad0 msgr2=0x55cf543140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.385+0000 7f2d9219b700 1 --2- 172.21.15.119:0/4224220746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf5430bad0 0x55cf543140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.385+0000 7f2d9219b700 1 -- 172.21.15.119:0/4224220746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf54311380 msgr2=0x55cf5430fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.385+0000 7f2d9219b700 1 --2- 172.21.15.119:0/4224220746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf54311380 0x55cf5430fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.386+0000 7f2d9219b700 1 -- 172.21.15.119:0/4224220746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf54088d80 con 0x55cf54310a60 2023-12-07T10:22:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.386+0000 7f2d9199a700 1 --2- 172.21.15.119:0/4224220746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf54311380 0x55cf5430fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:22:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.386+0000 7f2d9219b700 1 --2- 172.21.15.119:0/4224220746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf54310a60 0x55cf5430f920 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009c80 tx=0x7f2d88011880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13542f0faa96fa0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.386+0000 7f2d813ff700 1 -- 172.21.15.119:0/4224220746 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d880048d0 con 0x55cf54310a60 2023-12-07T10:22:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.386+0000 7f2d813ff700 1 -- 172.21.15.119:0/4224220746 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d88004500 con 0x55cf54310a60 2023-12-07T10:22:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.386+0000 7f2d813ff700 1 -- 172.21.15.119:0/4224220746 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d88028920 con 0x55cf54310a60 2023-12-07T10:22:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.387+0000 7f2da7dbbb80 1 -- 172.21.15.119:0/4224220746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf54310a60 msgr2=0x55cf5430f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.387+0000 7f2da7dbbb80 1 --2- 172.21.15.119:0/4224220746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf54310a60 0x55cf5430f920 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009c80 tx=0x7f2d88011880 comp rx=0 tx=0).stop 2023-12-07T10:22:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.387+0000 7f2da7dbbb80 1 -- 172.21.15.119:0/4224220746 shutdown_connections 2023-12-07T10:22:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.387+0000 7f2da7dbbb80 1 --2- 172.21.15.119:0/4224220746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf54311380 0x55cf5430fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.387+0000 7f2da7dbbb80 1 --2- 172.21.15.119:0/4224220746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf5430bad0 0x55cf543140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.387+0000 7f2da7dbbb80 1 --2- 172.21.15.119:0/4224220746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf54310a60 0x55cf5430f920 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.387+0000 7f2da7dbbb80 1 -- 172.21.15.119:0/4224220746 >> 172.21.15.119:0/4224220746 conn(0x55cf5423a300 msgr2=0x55cf5430e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.387+0000 7f2da7dbbb80 1 -- 172.21.15.119:0/4224220746 shutdown_connections 2023-12-07T10:22:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.387+0000 7f2da7dbbb80 1 -- 172.21.15.119:0/4224220746 wait complete. 2023-12-07T10:22:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.388+0000 7f2da7dbbb80 1 -- start start 2023-12-07T10:22:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.388+0000 7f2da7dbbb80 1 Processor -- start 2023-12-07T10:22:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.389+0000 7f2da7dbbb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf5430bad0 0x55cf54319660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.389+0000 7f2da7dbbb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf54310a60 0x55cf54319ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.389+0000 7f2da7dbbb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf54311380 0x55cf5431a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.389+0000 7f2da7dbbb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5431c6c0 con 0x55cf5430bad0 2023-12-07T10:22:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.389+0000 7f2da7dbbb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5437a390 con 0x55cf54311380 2023-12-07T10:22:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.389+0000 7f2da7dbbb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf5437a4d0 con 0x55cf54310a60 2023-12-07T10:22:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.389+0000 7f2da7dbbb80 10 client.0 _ll_register_callbacks cb 0x55cf542008e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:22:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.389+0000 7f2d9199a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf5430bad0 0x55cf54319660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.389+0000 7f2d9219b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf54310a60 0x55cf54319ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.389+0000 7f2d9199a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf5430bad0 0x55cf54319660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35034/0 (socket says 172.21.15.119:35034) 2023-12-07T10:22:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.389+0000 7f2d9199a700 1 -- 172.21.15.119:0/3157217122 learned_addr learned my addr 172.21.15.119:0/3157217122 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.389+0000 7f2d9199a700 1 --2- 172.21.15.119:0/3157217122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf54311380 0x55cf5431a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.389+0000 7f2d9219b700 1 -- 172.21.15.119:0/3157217122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf54311380 msgr2=0x55cf5431a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.389+0000 7f2d9219b700 1 --2- 172.21.15.119:0/3157217122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf54311380 0x55cf5431a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.389+0000 7f2d9219b700 1 -- 172.21.15.119:0/3157217122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf5430bad0 msgr2=0x55cf54319660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.389+0000 7f2d9219b700 1 --2- 172.21.15.119:0/3157217122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf5430bad0 0x55cf54319660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.389+0000 7f2d9219b700 1 -- 172.21.15.119:0/3157217122 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8801d040 con 0x55cf54310a60 2023-12-07T10:22:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.390+0000 7f2d9219b700 1 --2- 172.21.15.119:0/3157217122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf54310a60 0x55cf54319ba0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009bd0 tx=0x7f2d88028260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=512c156e05b6d5a8 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.390+0000 7f2d75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:22:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.390+0000 7f2d75ffb700 1 -- 172.21.15.119:0/3157217122 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d88004a10 con 0x55cf54310a60 2023-12-07T10:22:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.390+0000 7f2da7dbbb80 1 -- 172.21.15.119:0/3157217122 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cf542fe6f0 con 0x55cf54310a60 2023-12-07T10:22:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.390+0000 7f2d75ffb700 1 -- 172.21.15.119:0/3157217122 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f2d88012e40 con 0x55cf54310a60 2023-12-07T10:22:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.390+0000 7f2d75ffb700 1 -- 172.21.15.119:0/3157217122 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d88004b70 con 0x55cf54310a60 2023-12-07T10:22:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.390+0000 7f2da7dbbb80 10 client.26248 fetch_fsmap learned FSMap version 30 2023-12-07T10:22:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.390+0000 7f2da7dbbb80 1 -- 172.21.15.119:0/3157217122 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55cf54071c00 con 0x55cf54310a60 2023-12-07T10:22:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.390+0000 7f2d75ffb700 1 -- 172.21.15.119:0/3157217122 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cf542fe6f0 con 0x55cf54310a60 2023-12-07T10:22:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.391+0000 7f2d75ffb700 1 -- 172.21.15.119:0/3157217122 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2d88003a90 con 0x55cf54310a60 2023-12-07T10:22:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.391+0000 7f2da7dbbb80 10 client.26248 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:22:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.391+0000 7f2da7dbbb80 10 client.26248 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:22:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.391+0000 7f2da7dbbb80 20 client.26248 populate_metadata read hostname 'smithi119' 2023-12-07T10:22:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.391+0000 7f2da7dbbb80 1 --2- 172.21.15.119:0/3157217122 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55cf5437b940 0x55cf5437dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.391+0000 7f2da7dbbb80 4 client.26248 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:22:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.391+0000 7f2da7dbbb80 1 -- 172.21.15.119:0/3157217122 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cf54218430 con 0x55cf5437b940 2023-12-07T10:22:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.391+0000 7f2d9199a700 1 --2- 172.21.15.119:0/3157217122 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55cf5437b940 0x55cf5437dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.392+0000 7f2d9199a700 1 --2- 172.21.15.119:0/3157217122 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55cf5437b940 0x55cf5437dd30 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b9c4bba35f01a73 server_cookie=f111228773a31b14 in_seq=0 out_seq=0 2023-12-07T10:22:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.392+0000 7f2d75ffb700 10 client.26248 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.399+0000 7f2d75ffb700 1 -- 172.21.15.119:0/3157217122 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cf54218430 con 0x55cf5437b940 2023-12-07T10:22:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.399+0000 7f2d75ffb700 10 client.26248 handle_command_reply: tid=0 2023-12-07T10:22:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.399+0000 7f2da7dbbb80 1 client.26248 shutdown 2023-12-07T10:22:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.399+0000 7f2da7dbbb80 1 -- 172.21.15.119:0/3157217122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf54310a60 msgr2=0x55cf54319ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.399+0000 7f2da7dbbb80 1 --2- 172.21.15.119:0/3157217122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf54310a60 0x55cf54319ba0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009bd0 tx=0x7f2d88028260 comp rx=0 tx=0).stop 2023-12-07T10:22:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.399+0000 7f2da7dbbb80 1 -- 172.21.15.119:0/3157217122 shutdown_connections 2023-12-07T10:22:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.399+0000 7f2da7dbbb80 1 --2- 172.21.15.119:0/3157217122 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55cf5437b940 0x55cf5437dd30 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.399+0000 7f2da7dbbb80 1 --2- 172.21.15.119:0/3157217122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cf54311380 0x55cf5431a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.399+0000 7f2da7dbbb80 1 --2- 172.21.15.119:0/3157217122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cf5430bad0 0x55cf54319660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.399+0000 7f2d75ffb700 0 client.26248 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.399+0000 7f2da7dbbb80 1 --2- 172.21.15.119:0/3157217122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf54310a60 0x55cf54319ba0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.399+0000 7f2da7dbbb80 1 -- 172.21.15.119:0/3157217122 >> 172.21.15.119:0/3157217122 conn(0x55cf5423a300 msgr2=0x55cf543150c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.399+0000 7f2da7dbbb80 1 -- 172.21.15.119:0/3157217122 shutdown_connections 2023-12-07T10:22:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.399+0000 7f2da7dbbb80 1 -- 172.21.15.119:0/3157217122 wait complete. 2023-12-07T10:22:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.400+0000 7f2da7dbbb80 20 client.26248 trim_cache size 0 max 16384 2023-12-07T10:22:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:22:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:22:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:22:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:22:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.400+0000 7f2d943ff700 1 -- 172.21.15.119:0/889981490 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f2d780a1d00 msgr2=0x7f2d780a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.400+0000 7f2d943ff700 1 --2- 172.21.15.119:0/889981490 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f2d780a1d00 0x7f2d780a41c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2d840983d0 tx=0x7f2d8400b3b0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.400+0000 7f2d943ff700 1 -- 172.21.15.119:0/889981490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d8c106a80 msgr2=0x7f2d8c199b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.400+0000 7f2d943ff700 1 --2- 172.21.15.119:0/889981490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d8c106a80 0x7f2d8c199b60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400b070 tx=0x7f2d84007860 comp rx=0 tx=0).stop 2023-12-07T10:22:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.400+0000 7f2d943ff700 1 -- 172.21.15.119:0/889981490 shutdown_connections 2023-12-07T10:22:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.400+0000 7f2d943ff700 1 --2- 172.21.15.119:0/889981490 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f2d780a1d00 0x7f2d780a41c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.400+0000 7f2d943ff700 1 --2- 172.21.15.119:0/889981490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d8c106a80 0x7f2d8c199b60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.400+0000 7f2d943ff700 1 --2- 172.21.15.119:0/889981490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d8c068ce0 0x7f2d8c19f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.400+0000 7f2d943ff700 1 --2- 172.21.15.119:0/889981490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c1060c0 0x7f2d8c19ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.400+0000 7f2d943ff700 1 -- 172.21.15.119:0/889981490 >> 172.21.15.119:0/889981490 conn(0x7f2d8c0775b0 msgr2=0x7f2d8c10c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.401+0000 7f2d943ff700 1 -- 172.21.15.119:0/889981490 shutdown_connections 2023-12-07T10:22:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:00.401+0000 7f2d943ff700 1 -- 172.21.15.119:0/889981490 wait complete. 2023-12-07T10:22:00.539 INFO:tasks.workunit.client.0.smithi119.stdout: 1 35909 16.32 MB/sec warmup 13 sec latency 7.965 ms 2023-12-07T10:22:01.539 INFO:tasks.workunit.client.0.smithi119.stdout: 1 36543 15.29 MB/sec warmup 14 sec latency 57.370 ms 2023-12-07T10:22:02.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:01 smithi119 ceph-mon[97908]: pgmap v51: 129 pgs: 129 active+clean; 19 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 1.8 MiB/s wr, 247 op/s 2023-12-07T10:22:02.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:01 smithi159 ceph-mon[134726]: pgmap v51: 129 pgs: 129 active+clean; 19 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 1.8 MiB/s wr, 247 op/s 2023-12-07T10:22:02.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:01 smithi153 ceph-mon[133625]: pgmap v51: 129 pgs: 129 active+clean; 19 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 1.8 MiB/s wr, 247 op/s 2023-12-07T10:22:02.539 INFO:tasks.workunit.client.0.smithi119.stdout: 1 39454 15.42 MB/sec warmup 15 sec latency 43.501 ms 2023-12-07T10:22:03.539 INFO:tasks.workunit.client.0.smithi119.stdout: 1 42565 15.55 MB/sec warmup 16 sec latency 6.868 ms 2023-12-07T10:22:04.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:03 smithi119 ceph-mon[97908]: pgmap v52: 129 pgs: 129 active+clean; 19 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.3 MiB/s wr, 184 op/s 2023-12-07T10:22:04.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:03 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:22:04.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:03 smithi159 ceph-mon[134726]: pgmap v52: 129 pgs: 129 active+clean; 19 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.3 MiB/s wr, 184 op/s 2023-12-07T10:22:04.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:03 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:22:04.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:03 smithi153 ceph-mon[133625]: pgmap v52: 129 pgs: 129 active+clean; 19 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.3 MiB/s wr, 184 op/s 2023-12-07T10:22:04.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:03 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:22:04.539 INFO:tasks.workunit.client.0.smithi119.stdout: 1 44688 15.07 MB/sec warmup 17 sec latency 21.025 ms 2023-12-07T10:22:05.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:04 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:22:05.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:04 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:22:05.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:04 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:22:05.539 INFO:tasks.workunit.client.0.smithi119.stdout: 1 47062 15.02 MB/sec warmup 18 sec latency 6.899 ms 2023-12-07T10:22:06.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:05 smithi119 ceph-mon[97908]: pgmap v53: 129 pgs: 129 active+clean; 19 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.3 MiB/s wr, 184 op/s 2023-12-07T10:22:06.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:05 smithi159 ceph-mon[134726]: pgmap v53: 129 pgs: 129 active+clean; 19 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.3 MiB/s wr, 184 op/s 2023-12-07T10:22:06.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:05 smithi153 ceph-mon[133625]: pgmap v53: 129 pgs: 129 active+clean; 19 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.3 MiB/s wr, 184 op/s 2023-12-07T10:22:06.539 INFO:tasks.workunit.client.0.smithi119.stdout: 1 50110 15.19 MB/sec warmup 19 sec latency 7.343 ms 2023-12-07T10:22:07.539 INFO:tasks.workunit.client.0.smithi119.stdout: 1 52132 14.79 MB/sec warmup 20 sec latency 142.375 ms 2023-12-07T10:22:08.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:07 smithi119 ceph-mon[97908]: pgmap v54: 129 pgs: 129 active+clean; 26 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 279 op/s 2023-12-07T10:22:08.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:07 smithi159 ceph-mon[134726]: pgmap v54: 129 pgs: 129 active+clean; 26 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 279 op/s 2023-12-07T10:22:08.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:07 smithi153 ceph-mon[133625]: pgmap v54: 129 pgs: 129 active+clean; 26 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 279 op/s 2023-12-07T10:22:08.539 INFO:tasks.workunit.client.0.smithi119.stdout: 1 54017 14.68 MB/sec warmup 21 sec latency 94.262 ms 2023-12-07T10:22:09.539 INFO:tasks.workunit.client.0.smithi119.stdout: 1 56889 14.80 MB/sec warmup 22 sec latency 26.393 ms 2023-12-07T10:22:10.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:09 smithi119 ceph-mon[97908]: pgmap v55: 129 pgs: 129 active+clean; 26 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.4 MiB/s wr, 192 op/s 2023-12-07T10:22:10.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:09 smithi159 ceph-mon[134726]: pgmap v55: 129 pgs: 129 active+clean; 26 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.4 MiB/s wr, 192 op/s 2023-12-07T10:22:10.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:09 smithi153 ceph-mon[133625]: pgmap v55: 129 pgs: 129 active+clean; 26 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.4 MiB/s wr, 192 op/s 2023-12-07T10:22:10.539 INFO:tasks.workunit.client.0.smithi119.stdout: 1 59134 14.52 MB/sec warmup 23 sec latency 13.659 ms 2023-12-07T10:22:11.539 INFO:tasks.workunit.client.0.smithi119.stdout: 1 61083 14.43 MB/sec warmup 24 sec latency 15.692 ms 2023-12-07T10:22:12.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:11 smithi119 ceph-mon[97908]: pgmap v56: 129 pgs: 129 active+clean; 50 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 288 op/s 2023-12-07T10:22:12.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:11 smithi159 ceph-mon[134726]: pgmap v56: 129 pgs: 129 active+clean; 50 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 288 op/s 2023-12-07T10:22:12.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:11 smithi153 ceph-mon[133625]: pgmap v56: 129 pgs: 129 active+clean; 50 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 288 op/s 2023-12-07T10:22:12.539 INFO:tasks.workunit.client.0.smithi119.stdout: 1 64010 14.55 MB/sec warmup 25 sec latency 70.007 ms 2023-12-07T10:22:13.544 INFO:tasks.workunit.client.0.smithi119.stdout: 1 65574 14.24 MB/sec warmup 26 sec latency 101.955 ms 2023-12-07T10:22:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.675+0000 7f2fd4dff700 1 -- 172.21.15.119:0/1200245063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fd0068940 msgr2=0x7f2fd0068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.675+0000 7f2fd4dff700 1 --2- 172.21.15.119:0/1200245063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fd0068940 0x7f2fd0068dc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009f90 tx=0x7f2fc800b330 comp rx=0 tx=0).stop 2023-12-07T10:22:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.675+0000 7f2fd4dff700 1 -- 172.21.15.119:0/1200245063 shutdown_connections 2023-12-07T10:22:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.675+0000 7f2fd4dff700 1 --2- 172.21.15.119:0/1200245063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fd0068940 0x7f2fd0068dc0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.675+0000 7f2fd4dff700 1 --2- 172.21.15.119:0/1200245063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fd01064a0 0x7f2fd0103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.675+0000 7f2fd4dff700 1 --2- 172.21.15.119:0/1200245063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fd0104460 0x7f2fd01048e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.675+0000 7f2fd4dff700 1 -- 172.21.15.119:0/1200245063 >> 172.21.15.119:0/1200245063 conn(0x7f2fd00775b0 msgr2=0x7f2fd00779c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.675+0000 7f2fd4dff700 1 -- 172.21.15.119:0/1200245063 shutdown_connections 2023-12-07T10:22:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.675+0000 7f2fd4dff700 1 -- 172.21.15.119:0/1200245063 wait complete. 2023-12-07T10:22:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.677+0000 7f2fd4dff700 1 Processor -- start 2023-12-07T10:22:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.677+0000 7f2fd4dff700 1 -- start start 2023-12-07T10:22:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.678+0000 7f2fd4dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fd0068940 0x7f2fd019e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.678+0000 7f2fd4dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fd0104460 0x7f2fd019e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:13.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.678+0000 7f2fd4dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fd01064a0 0x7f2fd01a2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:13.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.678+0000 7f2fd4dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd01a2f70 con 0x7f2fd01064a0 2023-12-07T10:22:13.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.678+0000 7f2fd4dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd01a30e0 con 0x7f2fd0104460 2023-12-07T10:22:13.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.678+0000 7f2fd4dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd01a32a0 con 0x7f2fd0068940 2023-12-07T10:22:13.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.679+0000 7f2fce59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fd0068940 0x7f2fd019e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:13.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.679+0000 7f2fcdd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fd0104460 0x7f2fd019e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:13.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.679+0000 7f2fce59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fd0068940 0x7f2fd019e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:58540/0 (socket says 172.21.15.119:58540) 2023-12-07T10:22:13.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.679+0000 7f2fce59c700 1 -- 172.21.15.119:0/3515859951 learned_addr learned my addr 172.21.15.119:0/3515859951 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:13.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.679+0000 7f2fcdd9b700 1 -- 172.21.15.119:0/3515859951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fd0068940 msgr2=0x7f2fd019e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:13.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.679+0000 7f2fcdd9b700 1 --2- 172.21.15.119:0/3515859951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fd0068940 0x7f2fd019e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.679+0000 7f2fcdd9b700 1 -- 172.21.15.119:0/3515859951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fd01064a0 msgr2=0x7f2fd01a2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:13.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.679+0000 7f2fcdd9b700 1 --2- 172.21.15.119:0/3515859951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fd01064a0 0x7f2fd01a2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.679+0000 7f2fcdd9b700 1 -- 172.21.15.119:0/3515859951 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc8009c40 con 0x7f2fd0104460 2023-12-07T10:22:13.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.679+0000 7f2fcdd9b700 1 --2- 172.21.15.119:0/3515859951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fd0104460 0x7f2fd019e5e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000e9c0 tx=0x7f2fc000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:13.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.680+0000 7f2fbf7fe700 1 -- 172.21.15.119:0/3515859951 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc000c9c0 con 0x7f2fd0104460 2023-12-07T10:22:13.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.680+0000 7f2fbf7fe700 1 -- 172.21.15.119:0/3515859951 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fc0010430 con 0x7f2fd0104460 2023-12-07T10:22:13.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.680+0000 7f2fbf7fe700 1 -- 172.21.15.119:0/3515859951 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0018680 con 0x7f2fd0104460 2023-12-07T10:22:13.683 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.680+0000 7f2fd4dff700 1 -- 172.21.15.119:0/3515859951 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd01a3580 con 0x7f2fd0104460 2023-12-07T10:22:13.684 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.683+0000 7f2fd4dff700 1 -- 172.21.15.119:0/3515859951 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd01aadc0 con 0x7f2fd0104460 2023-12-07T10:22:13.691 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.690+0000 7f2fe8775b80 1 Processor -- start 2023-12-07T10:22:13.692 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.690+0000 7f2fbf7fe700 1 -- 172.21.15.119:0/3515859951 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f2fc00105a0 con 0x7f2fd0104460 2023-12-07T10:22:13.692 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.691+0000 7f2fe8775b80 1 -- start start 2023-12-07T10:22:13.694 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.692+0000 7f2fe8775b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd635a4a60 0x55fd635a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:13.694 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.692+0000 7f2fe8775b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd635a5380 0x55fd6359f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:13.694 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.692+0000 7f2fe8775b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd6359fad0 0x55fd6359ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:13.694 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.693+0000 7f2fe8775b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd63513760 con 0x55fd635a4a60 2023-12-07T10:22:13.694 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.693+0000 7f2fe8775b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd634fa740 con 0x55fd6359fad0 2023-12-07T10:22:13.694 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.693+0000 7f2fe8775b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd634f8060 con 0x55fd635a5380 2023-12-07T10:22:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.693+0000 7f2fbf7fe700 1 --2- 172.21.15.119:0/3515859951 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f2fb80a1dd0 0x7f2fb80a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.693+0000 7f2fced9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd635a4a60 0x55fd635a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.693+0000 7f2fbf7fe700 1 -- 172.21.15.119:0/3515859951 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f2fc0014070 con 0x7f2fd0104460 2023-12-07T10:22:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.694+0000 7f2fce59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd635a5380 0x55fd6359f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.694+0000 7f2fced9d700 1 --2- 172.21.15.119:0/3515859951 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f2fb80a1dd0 0x7f2fb80a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.694+0000 7f2fce59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd635a5380 0x55fd6359f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:58546/0 (socket says 172.21.15.119:58546) 2023-12-07T10:22:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.694+0000 7f2fce59c700 1 -- 172.21.15.119:0/2663958702 learned_addr learned my addr 172.21.15.119:0/2663958702 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.694+0000 7f2fcdd9b700 1 --2- 172.21.15.119:0/2663958702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd6359fad0 0x55fd6359ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.694+0000 7f2fce59c700 1 -- 172.21.15.119:0/2663958702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd6359fad0 msgr2=0x55fd6359ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.694+0000 7f2fce59c700 1 --2- 172.21.15.119:0/2663958702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd6359fad0 0x55fd6359ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.694+0000 7f2fce59c700 1 -- 172.21.15.119:0/2663958702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd635a4a60 msgr2=0x55fd635a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.694+0000 7f2fce59c700 1 --2- 172.21.15.119:0/2663958702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd635a4a60 0x55fd635a4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.694+0000 7f2fce59c700 1 -- 172.21.15.119:0/2663958702 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd633705d0 con 0x55fd635a5380 2023-12-07T10:22:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.695+0000 7f2fce59c700 1 --2- 172.21.15.119:0/2663958702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd635a5380 0x55fd6359f540 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009dd0 tx=0x7f2fc4011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=455134b49772e760 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.695+0000 7f2fced9d700 1 --2- 172.21.15.119:0/3515859951 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f2fb80a1dd0 0x7f2fb80a4290 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009c10 tx=0x7f2fc8009ba0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.695+0000 7f2fbcff9700 1 -- 172.21.15.119:0/2663958702 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4004a30 con 0x55fd635a5380 2023-12-07T10:22:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.696+0000 7f2fbcff9700 1 -- 172.21.15.119:0/2663958702 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fc40054e0 con 0x55fd635a5380 2023-12-07T10:22:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.696+0000 7f2fbcff9700 1 -- 172.21.15.119:0/2663958702 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4004a30 con 0x55fd635a5380 2023-12-07T10:22:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.697+0000 7f2fe8775b80 1 -- 172.21.15.119:0/2663958702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd635a5380 msgr2=0x55fd6359f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.697+0000 7f2fe8775b80 1 --2- 172.21.15.119:0/2663958702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd635a5380 0x55fd6359f540 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009dd0 tx=0x7f2fc4011720 comp rx=0 tx=0).stop 2023-12-07T10:22:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.697+0000 7f2fe8775b80 1 -- 172.21.15.119:0/2663958702 shutdown_connections 2023-12-07T10:22:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.697+0000 7f2fe8775b80 1 --2- 172.21.15.119:0/2663958702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd6359fad0 0x55fd6359ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.697+0000 7f2fe8775b80 1 --2- 172.21.15.119:0/2663958702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd635a4a60 0x55fd635a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.697+0000 7f2fe8775b80 1 --2- 172.21.15.119:0/2663958702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd635a5380 0x55fd6359f540 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.697+0000 7f2fe8775b80 1 -- 172.21.15.119:0/2663958702 >> 172.21.15.119:0/2663958702 conn(0x55fd634ce300 msgr2=0x55fd63599b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:13.699 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.697+0000 7f2fe8775b80 1 -- 172.21.15.119:0/2663958702 shutdown_connections 2023-12-07T10:22:13.699 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.697+0000 7f2fe8775b80 1 -- 172.21.15.119:0/2663958702 wait complete. 2023-12-07T10:22:13.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.698+0000 7f2fe8775b80 1 -- start start 2023-12-07T10:22:13.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.699+0000 7f2fe8775b80 1 Processor -- start 2023-12-07T10:22:13.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.699+0000 7f2fe8775b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd6359fad0 0x55fd635ab360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.700+0000 7f2fe8775b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd635a4a60 0x55fd635a8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.700+0000 7f2fe8775b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd635a5380 0x55fd635a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.700+0000 7f2fe8775b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd634f3950 con 0x55fd635a5380 2023-12-07T10:22:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.700+0000 7f2fe8775b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd634ad500 con 0x55fd6359fad0 2023-12-07T10:22:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.700+0000 7f2fe8775b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd634ad1a0 con 0x55fd635a4a60 2023-12-07T10:22:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.700+0000 7f2fe8775b80 10 client.0 _ll_register_callbacks cb 0x55fd634948e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:22:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.700+0000 7f2fcdd9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd635a4a60 0x55fd635a8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.700+0000 7f2fce59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd6359fad0 0x55fd635ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.700+0000 7f2fce59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd6359fad0 0x55fd635ab360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:47818/0 (socket says 172.21.15.119:47818) 2023-12-07T10:22:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.700+0000 7f2fce59c700 1 -- 172.21.15.119:0/1633157023 learned_addr learned my addr 172.21.15.119:0/1633157023 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.701+0000 7f2fced9d700 1 --2- 172.21.15.119:0/1633157023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd635a5380 0x55fd635a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.701+0000 7f2fce59c700 1 -- 172.21.15.119:0/1633157023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd635a4a60 msgr2=0x55fd635a8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.701+0000 7f2fce59c700 1 --2- 172.21.15.119:0/1633157023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd635a4a60 0x55fd635a8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.701+0000 7f2fce59c700 1 -- 172.21.15.119:0/1633157023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd635a5380 msgr2=0x55fd635a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.701+0000 7f2fce59c700 1 --2- 172.21.15.119:0/1633157023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd635a5380 0x55fd635a8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.701+0000 7f2fce59c700 1 -- 172.21.15.119:0/1633157023 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc4009a30 con 0x55fd6359fad0 2023-12-07T10:22:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.702+0000 7f2fb5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:22:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.702+0000 7f2fce59c700 1 --2- 172.21.15.119:0/1633157023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd6359fad0 0x55fd635ab360 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40050b0 tx=0x7f2fc40050e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46bbcfad0d90de29 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.702+0000 7f2fb5ffb700 1 -- 172.21.15.119:0/1633157023 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4004450 con 0x55fd6359fad0 2023-12-07T10:22:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.702+0000 7f2fe8775b80 1 -- 172.21.15.119:0/1633157023 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd635926f0 con 0x55fd6359fad0 2023-12-07T10:22:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.702+0000 7f2fb5ffb700 1 -- 172.21.15.119:0/1633157023 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f2fc4005ce0 con 0x55fd6359fad0 2023-12-07T10:22:13.704 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.703+0000 7f2fb5ffb700 1 -- 172.21.15.119:0/1633157023 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fc4004450 con 0x55fd6359fad0 2023-12-07T10:22:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.704+0000 7f2fe8775b80 10 client.26267 fetch_fsmap learned FSMap version 30 2023-12-07T10:22:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.704+0000 7f2fe8775b80 1 -- 172.21.15.119:0/1633157023 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55fd6331cd80 con 0x55fd6359fad0 2023-12-07T10:22:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.704+0000 7f2fb5ffb700 1 -- 172.21.15.119:0/1633157023 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fd635926f0 con 0x55fd6359fad0 2023-12-07T10:22:13.706 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.706+0000 7f2fb5ffb700 1 -- 172.21.15.119:0/1633157023 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2fc4022630 con 0x55fd6359fad0 2023-12-07T10:22:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.706+0000 7f2fe8775b80 10 client.26267 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:22:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.706+0000 7f2fe8775b80 10 client.26267 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:22:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.706+0000 7f2fe8775b80 20 client.26267 populate_metadata read hostname 'smithi119' 2023-12-07T10:22:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.706+0000 7f2fe8775b80 1 --2- 172.21.15.119:0/1633157023 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55fd635b8af0 0x55fd635baf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.706+0000 7f2fe8775b80 4 client.26267 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:22:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.706+0000 7f2fe8775b80 1 -- 172.21.15.119:0/1633157023 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd6356e740 con 0x55fd635b8af0 2023-12-07T10:22:13.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.707+0000 7f2fcdd9b700 1 --2- 172.21.15.119:0/1633157023 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55fd635b8af0 0x55fd635baf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.708+0000 7f2fb5ffb700 10 client.26267 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.708+0000 7f2fcdd9b700 1 --2- 172.21.15.119:0/1633157023 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55fd635b8af0 0x55fd635baf10 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6375c75805739b server_cookie=a3aecb7242a4da0a in_seq=0 out_seq=0 2023-12-07T10:22:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.708+0000 7f2fb5ffb700 1 -- 172.21.15.119:0/1633157023 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fd6356e740 con 0x55fd635b8af0 2023-12-07T10:22:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.708+0000 7f2fb5ffb700 10 client.26267 handle_command_reply: tid=0 2023-12-07T10:22:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.708+0000 7f2fe8775b80 1 client.26267 shutdown 2023-12-07T10:22:13.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.711+0000 7f2fe8775b80 1 -- 172.21.15.119:0/1633157023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd6359fad0 msgr2=0x55fd635ab360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:13.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.711+0000 7f2fe8775b80 1 --2- 172.21.15.119:0/1633157023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd6359fad0 0x55fd635ab360 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40050b0 tx=0x7f2fc40050e0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.711+0000 7f2fe8775b80 1 -- 172.21.15.119:0/1633157023 shutdown_connections 2023-12-07T10:22:13.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.711+0000 7f2fe8775b80 1 --2- 172.21.15.119:0/1633157023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd635a5380 0x55fd635a8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.711+0000 7f2fe8775b80 1 --2- 172.21.15.119:0/1633157023 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55fd635b8af0 0x55fd635baf10 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.711+0000 7f2fe8775b80 1 --2- 172.21.15.119:0/1633157023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd6359fad0 0x55fd635ab360 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.711+0000 7f2fe8775b80 1 --2- 172.21.15.119:0/1633157023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd635a4a60 0x55fd635a8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.711+0000 7f2fb5ffb700 0 client.26267 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:13.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.711+0000 7f2fe8775b80 1 -- 172.21.15.119:0/1633157023 >> 172.21.15.119:0/1633157023 conn(0x55fd634ce300 msgr2=0x55fd63594980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:13.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.712+0000 7f2fe8775b80 1 -- 172.21.15.119:0/1633157023 shutdown_connections 2023-12-07T10:22:13.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.712+0000 7f2fe8775b80 1 -- 172.21.15.119:0/1633157023 wait complete. 2023-12-07T10:22:13.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.712+0000 7f2fe8775b80 20 client.26267 trim_cache size 0 max 16384 2023-12-07T10:22:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.741+0000 7f2fe8775b80 1 Processor -- start 2023-12-07T10:22:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.741+0000 7f2fe8775b80 1 -- start start 2023-12-07T10:22:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.742+0000 7f2fe8775b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd635a4a60 0x55fd635afe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.742+0000 7f2fe8775b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd635a5380 0x55fd635b0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.742+0000 7f2fe8775b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd635bfef0 0x55fd635a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.742+0000 7f2fe8775b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd633728c0 con 0x55fd635bfef0 2023-12-07T10:22:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.742+0000 7f2fe8775b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6359aa90 con 0x55fd635a5380 2023-12-07T10:22:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.742+0000 7f2fe8775b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd635926f0 con 0x55fd635a4a60 2023-12-07T10:22:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.742+0000 7f2fcdd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd635a5380 0x55fd635b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.742+0000 7f2fce59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd635a4a60 0x55fd635afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.742+0000 7f2fce59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd635a4a60 0x55fd635afe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:58570/0 (socket says 172.21.15.119:58570) 2023-12-07T10:22:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.742+0000 7f2fce59c700 1 -- 172.21.15.119:0/3728304568 learned_addr learned my addr 172.21.15.119:0/3728304568 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.743+0000 7f2fce59c700 1 -- 172.21.15.119:0/3728304568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd635a5380 msgr2=0x55fd635b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.743+0000 7f2fce59c700 1 --2- 172.21.15.119:0/3728304568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd635a5380 0x55fd635b0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.743+0000 7f2fce59c700 1 -- 172.21.15.119:0/3728304568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd635bfef0 msgr2=0x55fd635a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.743+0000 7f2fce59c700 1 --2- 172.21.15.119:0/3728304568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd635bfef0 0x55fd635a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.743+0000 7f2fce59c700 1 -- 172.21.15.119:0/3728304568 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd63305c00 con 0x55fd635a4a60 2023-12-07T10:22:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.743+0000 7f2fce59c700 1 --2- 172.21.15.119:0/3728304568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd635a4a60 0x55fd635afe30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4004c20 tx=0x7f2fc4004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a34f7b28e103d38 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.743+0000 7f2fbd7fa700 1 -- 172.21.15.119:0/3728304568 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4005c80 con 0x55fd635a4a60 2023-12-07T10:22:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.743+0000 7f2fbd7fa700 1 -- 172.21.15.119:0/3728304568 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fc4005200 con 0x55fd635a4a60 2023-12-07T10:22:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.743+0000 7f2fbd7fa700 1 -- 172.21.15.119:0/3728304568 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc40038e0 con 0x55fd635a4a60 2023-12-07T10:22:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.744+0000 7f2fe8775b80 1 -- 172.21.15.119:0/3728304568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd635a4a60 msgr2=0x55fd635afe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.744+0000 7f2fe8775b80 1 --2- 172.21.15.119:0/3728304568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd635a4a60 0x55fd635afe30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4004c20 tx=0x7f2fc4004d00 comp rx=0 tx=0).stop 2023-12-07T10:22:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.744+0000 7f2fe8775b80 1 -- 172.21.15.119:0/3728304568 shutdown_connections 2023-12-07T10:22:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.744+0000 7f2fe8775b80 1 --2- 172.21.15.119:0/3728304568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd635bfef0 0x55fd635a7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.744+0000 7f2fe8775b80 1 --2- 172.21.15.119:0/3728304568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd635a4a60 0x55fd635afe30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.744+0000 7f2fe8775b80 1 --2- 172.21.15.119:0/3728304568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd635a5380 0x55fd635b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.744+0000 7f2fe8775b80 1 -- 172.21.15.119:0/3728304568 >> 172.21.15.119:0/3728304568 conn(0x55fd634ce300 msgr2=0x55fd635b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.744+0000 7f2fe8775b80 1 -- 172.21.15.119:0/3728304568 shutdown_connections 2023-12-07T10:22:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.745+0000 7f2fe8775b80 1 -- 172.21.15.119:0/3728304568 wait complete. 2023-12-07T10:22:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.745+0000 7f2fe8775b80 1 -- start start 2023-12-07T10:22:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.745+0000 7f2fe8775b80 1 Processor -- start 2023-12-07T10:22:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.746+0000 7f2fe8775b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd635a4a60 0x55fd635b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.746+0000 7f2fe8775b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd635a5380 0x55fd635ae520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.746+0000 7f2fe8775b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd635aea60 0x55fd635aeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.746+0000 7f2fe8775b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6360f000 con 0x55fd635a4a60 2023-12-07T10:22:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.746+0000 7f2fe8775b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6360f140 con 0x55fd635aea60 2023-12-07T10:22:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.746+0000 7f2fe8775b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6360f280 con 0x55fd635a5380 2023-12-07T10:22:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.746+0000 7f2fe8775b80 10 client.0 _ll_register_callbacks cb 0x55fd634948e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:22:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.746+0000 7f2fce59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd635a4a60 0x55fd635b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.747+0000 7f2fcdd9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd635a5380 0x55fd635ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.747+0000 7f2fcdd9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd635a5380 0x55fd635ae520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:58580/0 (socket says 172.21.15.119:58580) 2023-12-07T10:22:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.747+0000 7f2fcdd9b700 1 -- 172.21.15.119:0/2566587297 learned_addr learned my addr 172.21.15.119:0/2566587297 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.747+0000 7f2fced9d700 1 --2- 172.21.15.119:0/2566587297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd635aea60 0x55fd635aeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.747+0000 7f2fcdd9b700 1 -- 172.21.15.119:0/2566587297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd635aea60 msgr2=0x55fd635aeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.747+0000 7f2fcdd9b700 1 --2- 172.21.15.119:0/2566587297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd635aea60 0x55fd635aeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.747+0000 7f2fcdd9b700 1 -- 172.21.15.119:0/2566587297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd635a4a60 msgr2=0x55fd635b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.747+0000 7f2fcdd9b700 1 --2- 172.21.15.119:0/2566587297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd635a4a60 0x55fd635b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.747+0000 7f2fcdd9b700 1 -- 172.21.15.119:0/2566587297 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd6331cd80 con 0x55fd635a5380 2023-12-07T10:22:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.748+0000 7f2fcdd9b700 1 --2- 172.21.15.119:0/2566587297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd635a5380 0x55fd635ae520 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0096b30 tx=0x7f2fc00cd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de13794028b0826e server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.748+0000 7f2fb5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:22:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.748+0000 7f2fb5ffb700 1 -- 172.21.15.119:0/2566587297 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc009e780 con 0x55fd635a5380 2023-12-07T10:22:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.748+0000 7f2fb5ffb700 1 -- 172.21.15.119:0/2566587297 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f2fc00a29c0 con 0x55fd635a5380 2023-12-07T10:22:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.748+0000 7f2fe8775b80 1 -- 172.21.15.119:0/2566587297 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2fc40054e0 con 0x55fd635a5380 2023-12-07T10:22:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.748+0000 7f2fb5ffb700 1 -- 172.21.15.119:0/2566587297 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fc0010880 con 0x55fd635a5380 2023-12-07T10:22:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.748+0000 7f2fe8775b80 10 client.26266 fetch_fsmap learned FSMap version 30 2023-12-07T10:22:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.748+0000 7f2fe8775b80 1 -- 172.21.15.119:0/2566587297 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55fd633431f0 con 0x55fd635a5380 2023-12-07T10:22:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.748+0000 7f2fb5ffb700 1 -- 172.21.15.119:0/2566587297 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fc40054e0 con 0x55fd635a5380 2023-12-07T10:22:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.750+0000 7f2fb5ffb700 1 -- 172.21.15.119:0/2566587297 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2fc0099b90 con 0x55fd635a5380 2023-12-07T10:22:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.750+0000 7f2fe8775b80 10 client.26266 fetch_fsmap finished waiting for FSMap version 30 2023-12-07T10:22:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.750+0000 7f2fe8775b80 10 client.26266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:22:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.750+0000 7f2fe8775b80 20 client.26266 populate_metadata read hostname 'smithi119' 2023-12-07T10:22:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.750+0000 7f2fe8775b80 1 --2- 172.21.15.119:0/2566587297 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55fd63610a90 0x55fd63612eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.750+0000 7f2fe8775b80 4 client.26266 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:22:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.750+0000 7f2fe8775b80 1 -- 172.21.15.119:0/2566587297 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fd634b9f70 con 0x55fd63610a90 2023-12-07T10:22:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.750+0000 7f2fce59c700 1 --2- 172.21.15.119:0/2566587297 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55fd63610a90 0x55fd63612eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.751+0000 7f2fce59c700 1 --2- 172.21.15.119:0/2566587297 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55fd63610a90 0x55fd63612eb0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38be13f32b281c64 server_cookie=f2ff1b8fc8dc7304 in_seq=0 out_seq=0 2023-12-07T10:22:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.751+0000 7f2fb5ffb700 10 client.26266 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.762+0000 7f2fb5ffb700 1 -- 172.21.15.119:0/2566587297 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fd634b9f70 con 0x55fd63610a90 2023-12-07T10:22:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.762+0000 7f2fb5ffb700 10 client.26266 handle_command_reply: tid=0 2023-12-07T10:22:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.762+0000 7f2fe8775b80 1 client.26266 shutdown 2023-12-07T10:22:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.762+0000 7f2fe8775b80 1 -- 172.21.15.119:0/2566587297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd635a5380 msgr2=0x55fd635ae520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.762+0000 7f2fe8775b80 1 --2- 172.21.15.119:0/2566587297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd635a5380 0x55fd635ae520 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0096b30 tx=0x7f2fc00cd040 comp rx=0 tx=0).stop 2023-12-07T10:22:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.763+0000 7f2fe8775b80 1 -- 172.21.15.119:0/2566587297 shutdown_connections 2023-12-07T10:22:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.763+0000 7f2fe8775b80 1 --2- 172.21.15.119:0/2566587297 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55fd63610a90 0x55fd63612eb0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.763+0000 7f2fe8775b80 1 --2- 172.21.15.119:0/2566587297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fd635aea60 0x55fd635aeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.763+0000 7f2fb5ffb700 0 client.26266 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.763+0000 7f2fe8775b80 1 --2- 172.21.15.119:0/2566587297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd635a4a60 0x55fd635b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.763+0000 7f2fe8775b80 1 --2- 172.21.15.119:0/2566587297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd635a5380 0x55fd635ae520 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.763+0000 7f2fe8775b80 1 -- 172.21.15.119:0/2566587297 >> 172.21.15.119:0/2566587297 conn(0x55fd634ce300 msgr2=0x55fd635a38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.763+0000 7f2fe8775b80 1 -- 172.21.15.119:0/2566587297 shutdown_connections 2023-12-07T10:22:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.763+0000 7f2fe8775b80 1 -- 172.21.15.119:0/2566587297 wait complete. 2023-12-07T10:22:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.763+0000 7f2fe8775b80 20 client.26266 trim_cache size 0 max 16384 2023-12-07T10:22:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:22:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:22:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:22:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:22:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.764+0000 7f2fd4dff700 1 -- 172.21.15.119:0/3515859951 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f2fb80a1dd0 msgr2=0x7f2fb80a4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.764+0000 7f2fd4dff700 1 --2- 172.21.15.119:0/3515859951 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f2fb80a1dd0 0x7f2fb80a4290 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009c10 tx=0x7f2fc8009ba0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.764+0000 7f2fd4dff700 1 -- 172.21.15.119:0/3515859951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fd0104460 msgr2=0x7f2fd019e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.764+0000 7f2fd4dff700 1 --2- 172.21.15.119:0/3515859951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fd0104460 0x7f2fd019e5e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000e9c0 tx=0x7f2fc000ecd0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.764+0000 7f2fd4dff700 1 -- 172.21.15.119:0/3515859951 shutdown_connections 2023-12-07T10:22:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.764+0000 7f2fd4dff700 1 --2- 172.21.15.119:0/3515859951 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f2fb80a1dd0 0x7f2fb80a4290 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.764+0000 7f2fd4dff700 1 --2- 172.21.15.119:0/3515859951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fd01064a0 0x7f2fd01a2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.764+0000 7f2fd4dff700 1 --2- 172.21.15.119:0/3515859951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fd0068940 0x7f2fd019e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.764+0000 7f2fd4dff700 1 --2- 172.21.15.119:0/3515859951 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fd0104460 0x7f2fd019e5e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.764+0000 7f2fd4dff700 1 -- 172.21.15.119:0/3515859951 >> 172.21.15.119:0/3515859951 conn(0x7f2fd00775b0 msgr2=0x7f2fd00fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.764+0000 7f2fd4dff700 1 -- 172.21.15.119:0/3515859951 shutdown_connections 2023-12-07T10:22:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:13.764+0000 7f2fd4dff700 1 -- 172.21.15.119:0/3515859951 wait complete. 2023-12-07T10:22:14.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:13 smithi119 ceph-mon[97908]: pgmap v57: 129 pgs: 129 active+clean; 50 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 3.2 MiB/s wr, 191 op/s 2023-12-07T10:22:14.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:13 smithi159 ceph-mon[134726]: pgmap v57: 129 pgs: 129 active+clean; 50 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 3.2 MiB/s wr, 191 op/s 2023-12-07T10:22:14.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:13 smithi153 ceph-mon[133625]: pgmap v57: 129 pgs: 129 active+clean; 50 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 3.2 MiB/s wr, 191 op/s 2023-12-07T10:22:14.544 INFO:tasks.workunit.client.0.smithi119.stdout: 1 66429 13.82 MB/sec warmup 27 sec latency 196.409 ms 2023-12-07T10:22:15.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:14 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:22:15.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:14 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:22:15.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:14 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:22:15.544 INFO:tasks.workunit.client.0.smithi119.stdout: 1 67629 13.71 MB/sec warmup 28 sec latency 457.057 ms 2023-12-07T10:22:16.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:15 smithi119 ceph-mon[97908]: pgmap v58: 129 pgs: 129 active+clean; 50 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 3.2 MiB/s wr, 191 op/s 2023-12-07T10:22:16.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:15 smithi159 ceph-mon[134726]: pgmap v58: 129 pgs: 129 active+clean; 50 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 3.2 MiB/s wr, 191 op/s 2023-12-07T10:22:16.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:15 smithi153 ceph-mon[133625]: pgmap v58: 129 pgs: 129 active+clean; 50 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 3.2 MiB/s wr, 191 op/s 2023-12-07T10:22:16.544 INFO:tasks.workunit.client.0.smithi119.stdout: 1 67629 13.24 MB/sec warmup 29 sec latency 1457.142 ms 2023-12-07T10:22:17.544 INFO:tasks.workunit.client.0.smithi119.stdout: 1 67629 12.80 MB/sec warmup 30 sec latency 2457.251 ms 2023-12-07T10:22:18.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:17 smithi119 ceph-mon[97908]: pgmap v59: 129 pgs: 129 active+clean; 51 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.9 MiB/s wr, 362 op/s 2023-12-07T10:22:18.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:17 smithi159 ceph-mon[134726]: pgmap v59: 129 pgs: 129 active+clean; 51 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.9 MiB/s wr, 362 op/s 2023-12-07T10:22:18.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:17 smithi153 ceph-mon[133625]: pgmap v59: 129 pgs: 129 active+clean; 51 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.9 MiB/s wr, 362 op/s 2023-12-07T10:22:18.544 INFO:tasks.workunit.client.0.smithi119.stdout: 1 67629 12.38 MB/sec warmup 31 sec latency 3457.337 ms 2023-12-07T10:22:19.544 INFO:tasks.workunit.client.0.smithi119.stdout: 1 67658 12.00 MB/sec warmup 32 sec latency 3475.343 ms 2023-12-07T10:22:20.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:19 smithi159 ceph-mon[134726]: pgmap v60: 129 pgs: 129 active+clean; 51 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.3 MiB/s wr, 267 op/s 2023-12-07T10:22:20.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:19 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:22:20.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:19 smithi153 ceph-mon[133625]: pgmap v60: 129 pgs: 129 active+clean; 51 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.3 MiB/s wr, 267 op/s 2023-12-07T10:22:20.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:19 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:22:20.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:19 smithi119 ceph-mon[97908]: pgmap v60: 129 pgs: 129 active+clean; 51 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.3 MiB/s wr, 267 op/s 2023-12-07T10:22:20.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:19 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:22:20.544 INFO:tasks.workunit.client.0.smithi119.stdout: 1 67658 11.63 MB/sec warmup 33 sec latency 1931.254 ms 2023-12-07T10:22:21.545 INFO:tasks.workunit.client.0.smithi119.stdout: 1 67658 11.29 MB/sec warmup 34 sec latency 2931.388 ms 2023-12-07T10:22:22.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:21 smithi119 ceph-mon[97908]: pgmap v61: 129 pgs: 129 active+clean; 51 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.3 MiB/s wr, 276 op/s 2023-12-07T10:22:22.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:21 smithi159 ceph-mon[134726]: pgmap v61: 129 pgs: 129 active+clean; 51 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.3 MiB/s wr, 276 op/s 2023-12-07T10:22:22.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:21 smithi153 ceph-mon[133625]: pgmap v61: 129 pgs: 129 active+clean; 51 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.3 MiB/s wr, 276 op/s 2023-12-07T10:22:22.545 INFO:tasks.workunit.client.0.smithi119.stdout: 1 67658 10.97 MB/sec warmup 35 sec latency 3931.567 ms 2023-12-07T10:22:23.545 INFO:tasks.workunit.client.0.smithi119.stdout: 1 67662 10.66 MB/sec warmup 36 sec latency 4455.226 ms 2023-12-07T10:22:24.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:23 smithi119 ceph-mon[97908]: pgmap v62: 129 pgs: 129 active+clean; 51 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 3.8 MiB/s wr, 180 op/s 2023-12-07T10:22:24.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:23 smithi159 ceph-mon[134726]: pgmap v62: 129 pgs: 129 active+clean; 51 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 3.8 MiB/s wr, 180 op/s 2023-12-07T10:22:24.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:23 smithi153 ceph-mon[133625]: pgmap v62: 129 pgs: 129 active+clean; 51 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 3.8 MiB/s wr, 180 op/s 2023-12-07T10:22:24.545 INFO:tasks.workunit.client.0.smithi119.stdout: 1 67745 10.38 MB/sec warmup 37 sec latency 849.487 ms 2023-12-07T10:22:25.545 INFO:tasks.workunit.client.0.smithi119.stdout: 1 67745 10.10 MB/sec warmup 38 sec latency 1849.603 ms 2023-12-07T10:22:26.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:25 smithi119 ceph-mon[97908]: pgmap v63: 129 pgs: 129 active+clean; 51 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 3.8 MiB/s wr, 180 op/s 2023-12-07T10:22:26.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:25 smithi159 ceph-mon[134726]: pgmap v63: 129 pgs: 129 active+clean; 51 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 3.8 MiB/s wr, 180 op/s 2023-12-07T10:22:26.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:25 smithi153 ceph-mon[133625]: pgmap v63: 129 pgs: 129 active+clean; 51 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 3.8 MiB/s wr, 180 op/s 2023-12-07T10:22:26.545 INFO:tasks.workunit.client.0.smithi119.stdout: 1 67745 9.84 MB/sec warmup 39 sec latency 2849.761 ms 2023-12-07T10:22:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.930+0000 7f278bdff700 1 -- 172.21.15.119:0/533755711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2784101ee0 msgr2=0x7f278410e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.930+0000 7f278bdff700 1 --2- 172.21.15.119:0/533755711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2784101ee0 0x7f278410e3a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f277c00b0d0 tx=0x7f277c00b3e0 comp rx=0 tx=0).stop 2023-12-07T10:22:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.930+0000 7f278bdff700 1 -- 172.21.15.119:0/533755711 shutdown_connections 2023-12-07T10:22:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.930+0000 7f278bdff700 1 --2- 172.21.15.119:0/533755711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2784101ee0 0x7f278410e3a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.930+0000 7f278bdff700 1 --2- 172.21.15.119:0/533755711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27841075e0 0x7f27841079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.930+0000 7f278bdff700 1 --2- 172.21.15.119:0/533755711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2784101520 0x7f27841019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.930+0000 7f278bdff700 1 -- 172.21.15.119:0/533755711 >> 172.21.15.119:0/533755711 conn(0x7f27840fd010 msgr2=0x7f27840ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.930+0000 7f278bdff700 1 -- 172.21.15.119:0/533755711 shutdown_connections 2023-12-07T10:22:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.930+0000 7f278bdff700 1 -- 172.21.15.119:0/533755711 wait complete. 2023-12-07T10:22:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.931+0000 7f278bdff700 1 Processor -- start 2023-12-07T10:22:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.931+0000 7f278bdff700 1 -- start start 2023-12-07T10:22:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.932+0000 7f278bdff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2784101520 0x7f278419d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.932+0000 7f278bdff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2784101ee0 0x7f278419dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.932+0000 7f278bdff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27841075e0 0x7f2784197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.932+0000 7f278bdff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2784197ea0 con 0x7f2784101ee0 2023-12-07T10:22:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.932+0000 7f278bdff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2784198010 con 0x7f27841075e0 2023-12-07T10:22:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.932+0000 7f278bdff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27841981d0 con 0x7f2784101520 2023-12-07T10:22:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.932+0000 7f278939a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2784101ee0 0x7f278419dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.932+0000 7f278939a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2784101ee0 0x7f278419dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44738/0 (socket says 172.21.15.119:44738) 2023-12-07T10:22:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.932+0000 7f278939a700 1 -- 172.21.15.119:0/918136927 learned_addr learned my addr 172.21.15.119:0/918136927 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.932+0000 7f2789b9b700 1 --2- 172.21.15.119:0/918136927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2784101520 0x7f278419d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.932+0000 7f278a39c700 1 --2- 172.21.15.119:0/918136927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27841075e0 0x7f2784197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.932+0000 7f278939a700 1 -- 172.21.15.119:0/918136927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2784101520 msgr2=0x7f278419d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.932+0000 7f278939a700 1 --2- 172.21.15.119:0/918136927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2784101520 0x7f278419d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.932+0000 7f278939a700 1 -- 172.21.15.119:0/918136927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27841075e0 msgr2=0x7f2784197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.932+0000 7f278939a700 1 --2- 172.21.15.119:0/918136927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27841075e0 0x7f2784197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.932+0000 7f278939a700 1 -- 172.21.15.119:0/918136927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277c009d00 con 0x7f2784101ee0 2023-12-07T10:22:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.933+0000 7f278939a700 1 --2- 172.21.15.119:0/918136927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2784101ee0 0x7f278419dc40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f277400e9a0 tx=0x7f277400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.933+0000 7f277affd700 1 -- 172.21.15.119:0/918136927 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2774009790 con 0x7f2784101ee0 2023-12-07T10:22:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.933+0000 7f277affd700 1 -- 172.21.15.119:0/918136927 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2774010430 con 0x7f2784101ee0 2023-12-07T10:22:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.933+0000 7f278bdff700 1 -- 172.21.15.119:0/918136927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27841984b0 con 0x7f2784101ee0 2023-12-07T10:22:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.933+0000 7f277affd700 1 -- 172.21.15.119:0/918136927 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2774018680 con 0x7f2784101ee0 2023-12-07T10:22:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.933+0000 7f278bdff700 1 -- 172.21.15.119:0/918136927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27841a46f0 con 0x7f2784101ee0 2023-12-07T10:22:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.936+0000 7f277affd700 1 -- 172.21.15.119:0/918136927 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f27740105a0 con 0x7f2784101ee0 2023-12-07T10:22:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.937+0000 7f279f820b80 1 Processor -- start 2023-12-07T10:22:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.938+0000 7f279f820b80 1 -- start start 2023-12-07T10:22:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.938+0000 7f277affd700 1 --2- 172.21.15.119:0/918136927 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f27700a1dd0 0x7f27700a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.938+0000 7f279f820b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55860e83ea60 0x55860e83ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.938+0000 7f279f820b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55860e83f380 0x55860e839540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.938+0000 7f279f820b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55860e839ad0 0x55860e839f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.938+0000 7f279f820b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55860e7ad760 con 0x55860e83f380 2023-12-07T10:22:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.938+0000 7f279f820b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55860e794740 con 0x55860e83ea60 2023-12-07T10:22:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.938+0000 7f279f820b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55860e792060 con 0x55860e839ad0 2023-12-07T10:22:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.938+0000 7f277affd700 1 -- 172.21.15.119:0/918136927 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f2774014070 con 0x7f2784101ee0 2023-12-07T10:22:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.938+0000 7f278a39c700 1 --2- 172.21.15.119:0/918136927 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f27700a1dd0 0x7f27700a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.938+0000 7f2789b9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55860e83ea60 0x55860e83ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.938+0000 7f278939a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55860e83f380 0x55860e839540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.938+0000 7f2789b9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55860e83ea60 0x55860e83ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42312/0 (socket says 172.21.15.119:42312) 2023-12-07T10:22:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.938+0000 7f2789b9b700 1 -- 172.21.15.119:0/1104211403 learned_addr learned my addr 172.21.15.119:0/1104211403 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.939+0000 7f278a39c700 1 --2- 172.21.15.119:0/1104211403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55860e839ad0 0x55860e839f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.939+0000 7f2789b9b700 1 -- 172.21.15.119:0/1104211403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55860e839ad0 msgr2=0x55860e839f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.939+0000 7f2789b9b700 1 --2- 172.21.15.119:0/1104211403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55860e839ad0 0x55860e839f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.939+0000 7f2789b9b700 1 -- 172.21.15.119:0/1104211403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55860e83f380 msgr2=0x55860e839540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.939+0000 7f2789b9b700 1 --2- 172.21.15.119:0/1104211403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55860e83f380 0x55860e839540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.939+0000 7f2789b9b700 1 -- 172.21.15.119:0/1104211403 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55860e60a5d0 con 0x55860e83ea60 2023-12-07T10:22:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.939+0000 7f278939a700 1 --2- 172.21.15.119:0/1104211403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55860e83f380 0x55860e839540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.939+0000 7f278a39c700 1 --2- 172.21.15.119:0/918136927 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f27700a1dd0 0x7f27700a4290 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f277c009cd0 tx=0x7f277c008bf0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.939+0000 7f2789b9b700 1 --2- 172.21.15.119:0/1104211403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55860e83ea60 0x55860e83ee40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2780007c70 tx=0x7f2780007f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc81a3fc54666b0a server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.939+0000 7f276ffff700 1 -- 172.21.15.119:0/1104211403 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2780017070 con 0x55860e83ea60 2023-12-07T10:22:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.939+0000 7f276ffff700 1 -- 172.21.15.119:0/1104211403 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f278001b880 con 0x55860e83ea60 2023-12-07T10:22:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.939+0000 7f276ffff700 1 -- 172.21.15.119:0/1104211403 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278000d820 con 0x55860e83ea60 2023-12-07T10:22:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.940+0000 7f279f820b80 1 -- 172.21.15.119:0/1104211403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55860e83ea60 msgr2=0x55860e83ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.940+0000 7f279f820b80 1 --2- 172.21.15.119:0/1104211403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55860e83ea60 0x55860e83ee40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2780007c70 tx=0x7f2780007f80 comp rx=0 tx=0).stop 2023-12-07T10:22:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.940+0000 7f279f820b80 1 -- 172.21.15.119:0/1104211403 shutdown_connections 2023-12-07T10:22:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.940+0000 7f279f820b80 1 --2- 172.21.15.119:0/1104211403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55860e839ad0 0x55860e839f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.940+0000 7f279f820b80 1 --2- 172.21.15.119:0/1104211403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55860e83ea60 0x55860e83ee40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.940+0000 7f279f820b80 1 --2- 172.21.15.119:0/1104211403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55860e83f380 0x55860e839540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.940+0000 7f279f820b80 1 -- 172.21.15.119:0/1104211403 >> 172.21.15.119:0/1104211403 conn(0x55860e768300 msgr2=0x55860e833b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.940+0000 7f279f820b80 1 -- 172.21.15.119:0/1104211403 shutdown_connections 2023-12-07T10:22:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.940+0000 7f279f820b80 1 -- 172.21.15.119:0/1104211403 wait complete. 2023-12-07T10:22:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.941+0000 7f279f820b80 1 -- start start 2023-12-07T10:22:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.941+0000 7f279f820b80 1 Processor -- start 2023-12-07T10:22:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.941+0000 7f279f820b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55860e839ad0 0x55860e8453b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.941+0000 7f279f820b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55860e83ea60 0x55860e842730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.941+0000 7f279f820b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55860e83f380 0x55860e842e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.941+0000 7f279f820b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55860e78d950 con 0x55860e83f380 2023-12-07T10:22:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.942+0000 7f279f820b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55860e747500 con 0x55860e83ea60 2023-12-07T10:22:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.942+0000 7f279f820b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55860e7471a0 con 0x55860e839ad0 2023-12-07T10:22:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.942+0000 7f279f820b80 10 client.0 _ll_register_callbacks cb 0x55860e72e8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:22:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.942+0000 7f278a39c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55860e83f380 0x55860e842e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.942+0000 7f278a39c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55860e83f380 0x55860e842e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44756/0 (socket says 172.21.15.119:44756) 2023-12-07T10:22:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.942+0000 7f2789b9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55860e839ad0 0x55860e8453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.942+0000 7f278a39c700 1 -- 172.21.15.119:0/1511435304 learned_addr learned my addr 172.21.15.119:0/1511435304 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.942+0000 7f278939a700 1 --2- 172.21.15.119:0/1511435304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55860e83ea60 0x55860e842730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.942+0000 7f278a39c700 1 -- 172.21.15.119:0/1511435304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55860e839ad0 msgr2=0x55860e8453b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.942+0000 7f278a39c700 1 --2- 172.21.15.119:0/1511435304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55860e839ad0 0x55860e8453b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.942+0000 7f278a39c700 1 -- 172.21.15.119:0/1511435304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55860e83ea60 msgr2=0x55860e842730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.942+0000 7f278a39c700 1 --2- 172.21.15.119:0/1511435304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55860e83ea60 0x55860e842730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.942+0000 7f278a39c700 1 -- 172.21.15.119:0/1511435304 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27841a46f0 con 0x55860e83f380 2023-12-07T10:22:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.942+0000 7f2789b9b700 1 --2- 172.21.15.119:0/1511435304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55860e839ad0 0x55860e8453b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.942+0000 7f278a39c700 1 --2- 172.21.15.119:0/1511435304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55860e83f380 0x55860e842e50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f277c00b0d0 tx=0x7f277c00b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8855db475832dba server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.942+0000 7f276d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:22:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.943+0000 7f276d7fa700 1 -- 172.21.15.119:0/1511435304 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c00e040 con 0x55860e83f380 2023-12-07T10:22:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.943+0000 7f279f820b80 1 -- 172.21.15.119:0/1511435304 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55860e82c6f0 con 0x55860e83f380 2023-12-07T10:22:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.943+0000 7f276d7fa700 1 -- 172.21.15.119:0/1511435304 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f277c01f650 con 0x55860e83f380 2023-12-07T10:22:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.943+0000 7f276d7fa700 1 -- 172.21.15.119:0/1511435304 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f277c007940 con 0x55860e83f380 2023-12-07T10:22:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.943+0000 7f279f820b80 10 client.16272 fetch_fsmap learned FSMap version 32 2023-12-07T10:22:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.943+0000 7f279f820b80 1 -- 172.21.15.119:0/1511435304 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f2780007920 con 0x55860e83f380 2023-12-07T10:22:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.943+0000 7f276d7fa700 1 -- 172.21.15.119:0/1511435304 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55860e82c6f0 con 0x55860e83f380 2023-12-07T10:22:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.943+0000 7f276d7fa700 1 -- 172.21.15.119:0/1511435304 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f277c02d820 con 0x55860e83f380 2023-12-07T10:22:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.943+0000 7f279f820b80 10 client.16272 fetch_fsmap finished waiting for FSMap version 32 2023-12-07T10:22:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.943+0000 7f279f820b80 10 client.16272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:22:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.944+0000 7f279f820b80 20 client.16272 populate_metadata read hostname 'smithi119' 2023-12-07T10:22:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.944+0000 7f279f820b80 1 --2- 172.21.15.119:0/1511435304 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55860e852eb0 0x55860e8552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.944+0000 7f279f820b80 4 client.16272 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:22:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.944+0000 7f279f820b80 1 -- 172.21.15.119:0/1511435304 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55860e808740 con 0x55860e852eb0 2023-12-07T10:22:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.944+0000 7f2789b9b700 1 --2- 172.21.15.119:0/1511435304 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55860e852eb0 0x55860e8552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:26.949 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.944+0000 7f2789b9b700 1 --2- 172.21.15.119:0/1511435304 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55860e852eb0 0x55860e8552d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a3a0188038eceee server_cookie=60415944439344a2 in_seq=0 out_seq=0 2023-12-07T10:22:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.944+0000 7f276d7fa700 10 client.16272 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.945+0000 7f276d7fa700 1 -- 172.21.15.119:0/1511435304 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55860e808740 con 0x55860e852eb0 2023-12-07T10:22:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.945+0000 7f276d7fa700 10 client.16272 handle_command_reply: tid=0 2023-12-07T10:22:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.945+0000 7f279f820b80 1 client.16272 shutdown 2023-12-07T10:22:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.945+0000 7f279f820b80 1 -- 172.21.15.119:0/1511435304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55860e83f380 msgr2=0x55860e842e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.945+0000 7f279f820b80 1 --2- 172.21.15.119:0/1511435304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55860e83f380 0x55860e842e50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f277c00b0d0 tx=0x7f277c00b3e0 comp rx=0 tx=0).stop 2023-12-07T10:22:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.946+0000 7f279f820b80 1 -- 172.21.15.119:0/1511435304 shutdown_connections 2023-12-07T10:22:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.946+0000 7f279f820b80 1 --2- 172.21.15.119:0/1511435304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55860e83f380 0x55860e842e50 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.946+0000 7f279f820b80 1 --2- 172.21.15.119:0/1511435304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55860e839ad0 0x55860e8453b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.946+0000 7f279f820b80 1 --2- 172.21.15.119:0/1511435304 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55860e852eb0 0x55860e8552d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.946+0000 7f279f820b80 1 --2- 172.21.15.119:0/1511435304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55860e83ea60 0x55860e842730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.946+0000 7f276d7fa700 0 client.16272 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.946+0000 7f279f820b80 1 -- 172.21.15.119:0/1511435304 >> 172.21.15.119:0/1511435304 conn(0x55860e768300 msgr2=0x55860e82e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.946+0000 7f279f820b80 1 -- 172.21.15.119:0/1511435304 shutdown_connections 2023-12-07T10:22:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.946+0000 7f279f820b80 1 -- 172.21.15.119:0/1511435304 wait complete. 2023-12-07T10:22:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.946+0000 7f279f820b80 20 client.16272 trim_cache size 0 max 16384 2023-12-07T10:22:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.961+0000 7f279f820b80 1 Processor -- start 2023-12-07T10:22:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.961+0000 7f279f820b80 1 -- start start 2023-12-07T10:22:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.961+0000 7f279f820b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55860e839ad0 0x55860e841ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.961+0000 7f279f820b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55860e83ea60 0x55860e849dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.961+0000 7f279f820b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55860e83f380 0x55860e84a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.961+0000 7f279f820b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55860e60c8c0 con 0x55860e83f380 2023-12-07T10:22:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.961+0000 7f279f820b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55860e843c60 con 0x55860e83ea60 2023-12-07T10:22:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.961+0000 7f279f820b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55860e835a60 con 0x55860e839ad0 2023-12-07T10:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.962+0000 7f2789b9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55860e839ad0 0x55860e841ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.962+0000 7f278939a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55860e83ea60 0x55860e849dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.962+0000 7f2789b9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55860e839ad0 0x55860e841ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:46390/0 (socket says 172.21.15.119:46390) 2023-12-07T10:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.962+0000 7f2789b9b700 1 -- 172.21.15.119:0/2743181621 learned_addr learned my addr 172.21.15.119:0/2743181621 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.962+0000 7f278a39c700 1 --2- 172.21.15.119:0/2743181621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55860e83f380 0x55860e84a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.962+0000 7f278939a700 1 -- 172.21.15.119:0/2743181621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55860e839ad0 msgr2=0x55860e841ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.962+0000 7f278939a700 1 --2- 172.21.15.119:0/2743181621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55860e839ad0 0x55860e841ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.962+0000 7f278939a700 1 -- 172.21.15.119:0/2743181621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55860e83f380 msgr2=0x55860e84a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.962+0000 7f278939a700 1 --2- 172.21.15.119:0/2743181621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55860e83f380 0x55860e84a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.962+0000 7f278939a700 1 -- 172.21.15.119:0/2743181621 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55860e5b6d80 con 0x55860e83ea60 2023-12-07T10:22:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.962+0000 7f2789b9b700 1 --2- 172.21.15.119:0/2743181621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55860e839ad0 0x55860e841ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.962+0000 7f278939a700 1 --2- 172.21.15.119:0/2743181621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55860e83ea60 0x55860e849dd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f27740cb1d0 tx=0x7f277401a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5ce0ae6fd262373 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.963+0000 7f2778bff700 1 -- 172.21.15.119:0/2743181621 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27740b2070 con 0x55860e83ea60 2023-12-07T10:22:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.963+0000 7f2778bff700 1 -- 172.21.15.119:0/2743181621 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f277409e970 con 0x55860e83ea60 2023-12-07T10:22:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.963+0000 7f2778bff700 1 -- 172.21.15.119:0/2743181621 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277409aa80 con 0x55860e83ea60 2023-12-07T10:22:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.963+0000 7f279f820b80 1 -- 172.21.15.119:0/2743181621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55860e83ea60 msgr2=0x55860e849dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.963+0000 7f279f820b80 1 --2- 172.21.15.119:0/2743181621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55860e83ea60 0x55860e849dd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f27740cb1d0 tx=0x7f277401a040 comp rx=0 tx=0).stop 2023-12-07T10:22:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.963+0000 7f279f820b80 1 -- 172.21.15.119:0/2743181621 shutdown_connections 2023-12-07T10:22:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.963+0000 7f279f820b80 1 --2- 172.21.15.119:0/2743181621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55860e83f380 0x55860e84a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.963+0000 7f279f820b80 1 --2- 172.21.15.119:0/2743181621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55860e839ad0 0x55860e841ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.963+0000 7f279f820b80 1 --2- 172.21.15.119:0/2743181621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55860e83ea60 0x55860e849dd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.963+0000 7f279f820b80 1 -- 172.21.15.119:0/2743181621 >> 172.21.15.119:0/2743181621 conn(0x55860e768300 msgr2=0x55860e859c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.964+0000 7f279f820b80 1 -- 172.21.15.119:0/2743181621 shutdown_connections 2023-12-07T10:22:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.964+0000 7f279f820b80 1 -- 172.21.15.119:0/2743181621 wait complete. 2023-12-07T10:22:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.964+0000 7f279f820b80 1 -- start start 2023-12-07T10:22:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.964+0000 7f279f820b80 1 Processor -- start 2023-12-07T10:22:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.965+0000 7f279f820b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55860e839ad0 0x55860e847db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.965+0000 7f279f820b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55860e83ea60 0x55860e8482f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.965+0000 7f279f820b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55860e83f380 0x55860e848830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.965+0000 7f279f820b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55860e8a8470 con 0x55860e839ad0 2023-12-07T10:22:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.965+0000 7f279f820b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55860e8a85b0 con 0x55860e83f380 2023-12-07T10:22:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.965+0000 7f279f820b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55860e8a8770 con 0x55860e83ea60 2023-12-07T10:22:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.965+0000 7f279f820b80 10 client.0 _ll_register_callbacks cb 0x55860e72e8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:22:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.965+0000 7f2789b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55860e839ad0 0x55860e847db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.965+0000 7f278939a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55860e83ea60 0x55860e8482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.965+0000 7f2789b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55860e839ad0 0x55860e847db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44774/0 (socket says 172.21.15.119:44774) 2023-12-07T10:22:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.965+0000 7f2789b9b700 1 -- 172.21.15.119:0/614097159 learned_addr learned my addr 172.21.15.119:0/614097159 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.965+0000 7f278a39c700 1 --2- 172.21.15.119:0/614097159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55860e83f380 0x55860e848830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.966+0000 7f278939a700 1 -- 172.21.15.119:0/614097159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55860e83f380 msgr2=0x55860e848830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.966+0000 7f278939a700 1 --2- 172.21.15.119:0/614097159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55860e83f380 0x55860e848830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.966+0000 7f278939a700 1 -- 172.21.15.119:0/614097159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55860e839ad0 msgr2=0x55860e847db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.966+0000 7f278939a700 1 --2- 172.21.15.119:0/614097159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55860e839ad0 0x55860e847db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.966+0000 7f278939a700 1 -- 172.21.15.119:0/614097159 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2780020040 con 0x55860e83ea60 2023-12-07T10:22:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.966+0000 7f278939a700 1 --2- 172.21.15.119:0/614097159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55860e83ea60 0x55860e8482f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f27740cb1d0 tx=0x7f2774096c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d273d84b1d2e4f0a server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.966+0000 7f276d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:22:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.967+0000 7f276d7fa700 1 -- 172.21.15.119:0/614097159 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27740b2070 con 0x55860e83ea60 2023-12-07T10:22:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.967+0000 7f276d7fa700 1 -- 172.21.15.119:0/614097159 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f2774010880 con 0x55860e83ea60 2023-12-07T10:22:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.967+0000 7f279f820b80 1 -- 172.21.15.119:0/614097159 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f278001b880 con 0x55860e83ea60 2023-12-07T10:22:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.967+0000 7f276d7fa700 1 -- 172.21.15.119:0/614097159 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27740ba0b0 con 0x55860e83ea60 2023-12-07T10:22:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.967+0000 7f279f820b80 10 client.26284 fetch_fsmap learned FSMap version 32 2023-12-07T10:22:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.967+0000 7f279f820b80 1 -- 172.21.15.119:0/614097159 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f27841984b0 con 0x55860e83ea60 2023-12-07T10:22:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.967+0000 7f276d7fa700 1 -- 172.21.15.119:0/614097159 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f278001b880 con 0x55860e83ea60 2023-12-07T10:22:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.967+0000 7f276d7fa700 1 -- 172.21.15.119:0/614097159 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f27740ae1f0 con 0x55860e83ea60 2023-12-07T10:22:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.967+0000 7f279f820b80 10 client.26284 fetch_fsmap finished waiting for FSMap version 32 2023-12-07T10:22:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.967+0000 7f279f820b80 10 client.26284 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:22:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.967+0000 7f279f820b80 20 client.26284 populate_metadata read hostname 'smithi119' 2023-12-07T10:22:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.967+0000 7f279f820b80 1 --2- 172.21.15.119:0/614097159 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55860e8a9bf0 0x55860e8ac010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.967+0000 7f279f820b80 4 client.26284 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:22:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.967+0000 7f279f820b80 1 -- 172.21.15.119:0/614097159 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55860e753f70 con 0x55860e8a9bf0 2023-12-07T10:22:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.968+0000 7f2789b9b700 1 --2- 172.21.15.119:0/614097159 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55860e8a9bf0 0x55860e8ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.968+0000 7f2789b9b700 1 --2- 172.21.15.119:0/614097159 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55860e8a9bf0 0x55860e8ac010 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90b0166eee63a90b server_cookie=980ad2017c9b6a6 in_seq=0 out_seq=0 2023-12-07T10:22:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:26.968+0000 7f276d7fa700 10 client.26284 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.000+0000 7f276d7fa700 1 -- 172.21.15.119:0/614097159 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55860e753f70 con 0x55860e8a9bf0 2023-12-07T10:22:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.000+0000 7f276d7fa700 10 client.26284 handle_command_reply: tid=0 2023-12-07T10:22:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.000+0000 7f279f820b80 1 client.26284 shutdown 2023-12-07T10:22:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.001+0000 7f279f820b80 1 -- 172.21.15.119:0/614097159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55860e83ea60 msgr2=0x55860e8482f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.001+0000 7f279f820b80 1 --2- 172.21.15.119:0/614097159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55860e83ea60 0x55860e8482f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f27740cb1d0 tx=0x7f2774096c10 comp rx=0 tx=0).stop 2023-12-07T10:22:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.001+0000 7f279f820b80 1 -- 172.21.15.119:0/614097159 shutdown_connections 2023-12-07T10:22:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.001+0000 7f279f820b80 1 --2- 172.21.15.119:0/614097159 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55860e8a9bf0 0x55860e8ac010 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.001+0000 7f279f820b80 1 --2- 172.21.15.119:0/614097159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55860e83f380 0x55860e848830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.001+0000 7f279f820b80 1 --2- 172.21.15.119:0/614097159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55860e839ad0 0x55860e847db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.001+0000 7f279f820b80 1 --2- 172.21.15.119:0/614097159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55860e83ea60 0x55860e8482f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.001+0000 7f279f820b80 1 -- 172.21.15.119:0/614097159 >> 172.21.15.119:0/614097159 conn(0x55860e768300 msgr2=0x55860e83c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.001+0000 7f276d7fa700 0 client.26284 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.001+0000 7f279f820b80 1 -- 172.21.15.119:0/614097159 shutdown_connections 2023-12-07T10:22:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.001+0000 7f279f820b80 1 -- 172.21.15.119:0/614097159 wait complete. 2023-12-07T10:22:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.001+0000 7f279f820b80 20 client.26284 trim_cache size 0 max 16384 2023-12-07T10:22:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:22:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:22:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:22:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:22:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.002+0000 7f278bdff700 1 -- 172.21.15.119:0/918136927 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f27700a1dd0 msgr2=0x7f27700a4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.002+0000 7f278bdff700 1 --2- 172.21.15.119:0/918136927 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f27700a1dd0 0x7f27700a4290 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f277c009cd0 tx=0x7f277c008bf0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.002+0000 7f278bdff700 1 -- 172.21.15.119:0/918136927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2784101ee0 msgr2=0x7f278419dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.002+0000 7f278bdff700 1 --2- 172.21.15.119:0/918136927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2784101ee0 0x7f278419dc40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f277400e9a0 tx=0x7f277400ecb0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.002+0000 7f278bdff700 1 -- 172.21.15.119:0/918136927 shutdown_connections 2023-12-07T10:22:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.002+0000 7f278bdff700 1 --2- 172.21.15.119:0/918136927 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f27700a1dd0 0x7f27700a4290 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.002+0000 7f278bdff700 1 --2- 172.21.15.119:0/918136927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27841075e0 0x7f2784197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.002+0000 7f278bdff700 1 --2- 172.21.15.119:0/918136927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2784101520 0x7f278419d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.002+0000 7f278bdff700 1 --2- 172.21.15.119:0/918136927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2784101ee0 0x7f278419dc40 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.002+0000 7f278bdff700 1 -- 172.21.15.119:0/918136927 >> 172.21.15.119:0/918136927 conn(0x7f27840fd010 msgr2=0x7f27840fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.002+0000 7f278bdff700 1 -- 172.21.15.119:0/918136927 shutdown_connections 2023-12-07T10:22:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:27.002+0000 7f278bdff700 1 -- 172.21.15.119:0/918136927 wait complete. 2023-12-07T10:22:27.151 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-07T10:22:27.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.352+0000 7f9cc5bff700 1 -- 172.21.15.119:0/1975450835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cc010dfb0 msgr2=0x7f9cc01103a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:27.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.352+0000 7f9cc5bff700 1 --2- 172.21.15.119:0/1975450835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cc010dfb0 0x7f9cc01103a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8009f90 tx=0x7f9cb800b330 comp rx=0 tx=0).stop 2023-12-07T10:22:27.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.352+0000 7f9cc5bff700 1 -- 172.21.15.119:0/1975450835 shutdown_connections 2023-12-07T10:22:27.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.352+0000 7f9cc5bff700 1 --2- 172.21.15.119:0/1975450835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cc010dfb0 0x7f9cc01103a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.352+0000 7f9cc5bff700 1 --2- 172.21.15.119:0/1975450835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cc0100f40 0x7f9cc0101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.352+0000 7f9cc5bff700 1 --2- 172.21.15.119:0/1975450835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cc0101860 0x7f9cc010da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.352+0000 7f9cc5bff700 1 -- 172.21.15.119:0/1975450835 >> 172.21.15.119:0/1975450835 conn(0x7f9cc0077610 msgr2=0x7f9cc0077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:27.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.352+0000 7f9cc5bff700 1 -- 172.21.15.119:0/1975450835 shutdown_connections 2023-12-07T10:22:27.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.352+0000 7f9cc5bff700 1 -- 172.21.15.119:0/1975450835 wait complete. 2023-12-07T10:22:27.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.353+0000 7f9cc5bff700 1 Processor -- start 2023-12-07T10:22:27.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.353+0000 7f9cc5bff700 1 -- start start 2023-12-07T10:22:27.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.354+0000 7f9cc5bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cc0100f40 0x7f9cc0199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:27.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.354+0000 7f9cc5bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cc0101860 0x7f9cc019a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:27.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.354+0000 7f9cc5bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cc010dfb0 0x7f9cc019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:27.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.354+0000 7f9cc5bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc019ecf0 con 0x7f9cc010dfb0 2023-12-07T10:22:27.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.354+0000 7f9cc5bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc019ee60 con 0x7f9cc0101860 2023-12-07T10:22:27.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.354+0000 7f9cc5bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc019f020 con 0x7f9cc0100f40 2023-12-07T10:22:27.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.354+0000 7f9cbffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cc010dfb0 0x7f9cc019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:27.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.354+0000 7f9cbf7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cc0100f40 0x7f9cc0199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:27.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.354+0000 7f9cbffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cc010dfb0 0x7f9cc019e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44790/0 (socket says 172.21.15.119:44790) 2023-12-07T10:22:27.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.354+0000 7f9cbffff700 1 -- 172.21.15.119:0/797543573 learned_addr learned my addr 172.21.15.119:0/797543573 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:27.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.354+0000 7f9cbf7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cc0100f40 0x7f9cc0199dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:46408/0 (socket says 172.21.15.119:46408) 2023-12-07T10:22:27.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.354+0000 7f9cbeffd700 1 --2- 172.21.15.119:0/797543573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cc0101860 0x7f9cc019a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:27.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.354+0000 7f9cbffff700 1 -- 172.21.15.119:0/797543573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cc0100f40 msgr2=0x7f9cc0199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:27.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.354+0000 7f9cbffff700 1 --2- 172.21.15.119:0/797543573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cc0100f40 0x7f9cc0199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.354+0000 7f9cbffff700 1 -- 172.21.15.119:0/797543573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cc0101860 msgr2=0x7f9cc019a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:27.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.354+0000 7f9cbffff700 1 --2- 172.21.15.119:0/797543573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cc0101860 0x7f9cc019a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.355+0000 7f9cbffff700 1 -- 172.21.15.119:0/797543573 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb00095f0 con 0x7f9cc010dfb0 2023-12-07T10:22:27.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.355+0000 7f9cbffff700 1 --2- 172.21.15.119:0/797543573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cc010dfb0 0x7f9cc019e6b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8009f60 tx=0x7f9cb8004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:27.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.355+0000 7f9cbcff9700 1 -- 172.21.15.119:0/797543573 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb800e040 con 0x7f9cc010dfb0 2023-12-07T10:22:27.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.355+0000 7f9cbcff9700 1 -- 172.21.15.119:0/797543573 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cb8007e60 con 0x7f9cc010dfb0 2023-12-07T10:22:27.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.355+0000 7f9cc5bff700 1 -- 172.21.15.119:0/797543573 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb8009c40 con 0x7f9cc010dfb0 2023-12-07T10:22:27.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.355+0000 7f9cbcff9700 1 -- 172.21.15.119:0/797543573 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb801be60 con 0x7f9cc010dfb0 2023-12-07T10:22:27.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.355+0000 7f9cc5bff700 1 -- 172.21.15.119:0/797543573 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc01a6960 con 0x7f9cc010dfb0 2023-12-07T10:22:27.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.358+0000 7f9cbcff9700 1 -- 172.21.15.119:0/797543573 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f9cb801b4e0 con 0x7f9cc010dfb0 2023-12-07T10:22:27.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.359+0000 7f9cd95feb80 1 Processor -- start 2023-12-07T10:22:27.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.359+0000 7f9cbcff9700 1 --2- 172.21.15.119:0/797543573 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9ca80a1c80 0x7f9ca80a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:27.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.359+0000 7f9cd95feb80 1 -- start start 2023-12-07T10:22:27.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.360+0000 7f9cbcff9700 1 -- 172.21.15.119:0/797543573 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9cb80cef50 con 0x7f9cc010dfb0 2023-12-07T10:22:27.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.360+0000 7f9cbeffd700 1 --2- 172.21.15.119:0/797543573 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9ca80a1c80 0x7f9ca80a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:27.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.360+0000 7f9cd95feb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f71a91b40 0x562f71a91f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:27.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.360+0000 7f9cd95feb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a92460 0x562f71a95c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:27.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.360+0000 7f9cd95feb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f71a96150 0x562f71a96570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:27.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.360+0000 7f9cd95feb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f71a057c0 con 0x562f71a92460 2023-12-07T10:22:27.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.360+0000 7f9cd95feb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f719ec7a0 con 0x562f71a96150 2023-12-07T10:22:27.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.360+0000 7f9cd95feb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f719ea0c0 con 0x562f71a91b40 2023-12-07T10:22:27.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.360+0000 7f9cbeffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a92460 0x562f71a95c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:27.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.360+0000 7f9cbf7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f71a91b40 0x562f71a91f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:27.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.360+0000 7f9cbf7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f71a91b40 0x562f71a91f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:46414/0 (socket says 172.21.15.119:46414) 2023-12-07T10:22:27.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.360+0000 7f9cbeffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a92460 0x562f71a95c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44802/0 (socket says 172.21.15.119:44802) 2023-12-07T10:22:27.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.360+0000 7f9cbf7fe700 1 -- 172.21.15.119:0/4121120729 learned_addr learned my addr 172.21.15.119:0/4121120729 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:27.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.360+0000 7f9cbffff700 1 --2- 172.21.15.119:0/4121120729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f71a96150 0x562f71a96570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:27.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.361+0000 7f9cbffff700 1 -- 172.21.15.119:0/4121120729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f71a91b40 msgr2=0x562f71a91f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:27.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.361+0000 7f9cbffff700 1 --2- 172.21.15.119:0/4121120729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f71a91b40 0x562f71a91f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.361+0000 7f9cbffff700 1 -- 172.21.15.119:0/4121120729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a92460 msgr2=0x562f71a95c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:27.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.361+0000 7f9cbffff700 1 --2- 172.21.15.119:0/4121120729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a92460 0x562f71a95c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.361+0000 7f9cbffff700 1 -- 172.21.15.119:0/4121120729 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f71862630 con 0x562f71a96150 2023-12-07T10:22:27.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.361+0000 7f9cbf7fe700 1 --2- 172.21.15.119:0/4121120729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f71a91b40 0x562f71a91f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:27.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.361+0000 7f9cbeffd700 1 --2- 172.21.15.119:0/4121120729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a92460 0x562f71a95c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:27.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.361+0000 7f9cbeffd700 1 --2- 172.21.15.119:0/797543573 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9ca80a1c80 0x7f9ca80a4140 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9cc019b320 tx=0x7f9cb4013840 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:27.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.361+0000 7f9cbffff700 1 --2- 172.21.15.119:0/4121120729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f71a96150 0x562f71a96570 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80cc2f0 tx=0x7f9cb809b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba4aa7a70d61b78d server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:27.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.361+0000 7f9ca5ffb700 1 -- 172.21.15.119:0/4121120729 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb809c0d0 con 0x562f71a96150 2023-12-07T10:22:27.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.361+0000 7f9ca5ffb700 1 -- 172.21.15.119:0/4121120729 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cb809b9d0 con 0x562f71a96150 2023-12-07T10:22:27.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.361+0000 7f9ca5ffb700 1 -- 172.21.15.119:0/4121120729 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb80bb0b0 con 0x562f71a96150 2023-12-07T10:22:27.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.362+0000 7f9cd95feb80 1 -- 172.21.15.119:0/4121120729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f71a96150 msgr2=0x562f71a96570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:27.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.362+0000 7f9cd95feb80 1 --2- 172.21.15.119:0/4121120729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f71a96150 0x562f71a96570 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80cc2f0 tx=0x7f9cb809b820 comp rx=0 tx=0).stop 2023-12-07T10:22:27.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.362+0000 7f9cd95feb80 1 -- 172.21.15.119:0/4121120729 shutdown_connections 2023-12-07T10:22:27.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.362+0000 7f9cd95feb80 1 --2- 172.21.15.119:0/4121120729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f71a96150 0x562f71a96570 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.362+0000 7f9cd95feb80 1 --2- 172.21.15.119:0/4121120729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f71a91b40 0x562f71a91f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.362+0000 7f9cd95feb80 1 --2- 172.21.15.119:0/4121120729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a92460 0x562f71a95c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.362+0000 7f9cd95feb80 1 -- 172.21.15.119:0/4121120729 >> 172.21.15.119:0/4121120729 conn(0x562f719c0360 msgr2=0x562f71a8c130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:27.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.362+0000 7f9cd95feb80 1 -- 172.21.15.119:0/4121120729 shutdown_connections 2023-12-07T10:22:27.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.362+0000 7f9cd95feb80 1 -- 172.21.15.119:0/4121120729 wait complete. 2023-12-07T10:22:27.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.362+0000 7f9cd95feb80 1 -- start start 2023-12-07T10:22:27.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.363+0000 7f9cd95feb80 1 Processor -- start 2023-12-07T10:22:27.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.363+0000 7f9cd95feb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f71a91b40 0x562f71a9acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:27.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.363+0000 7f9cd95feb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a92460 0x562f71a9b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:27.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.363+0000 7f9cd95feb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f71a96150 0x562f71a9b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:27.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.363+0000 7f9cd95feb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f719e59b0 con 0x562f71a92460 2023-12-07T10:22:27.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.363+0000 7f9cd95feb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f7199f560 con 0x562f71a91b40 2023-12-07T10:22:27.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.363+0000 7f9cd95feb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f7199f200 con 0x562f71a96150 2023-12-07T10:22:27.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.363+0000 7f9cd95feb80 10 client.0 _ll_register_callbacks cb 0x562f71986940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:22:27.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.363+0000 7f9cbeffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a92460 0x562f71a9b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:27.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.363+0000 7f9cbeffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a92460 0x562f71a9b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44804/0 (socket says 172.21.15.119:44804) 2023-12-07T10:22:27.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.363+0000 7f9cbeffd700 1 -- 172.21.15.119:0/372398033 learned_addr learned my addr 172.21.15.119:0/372398033 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:27.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.364+0000 7f9cbffff700 1 --2- 172.21.15.119:0/372398033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f71a96150 0x562f71a9b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:27.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.364+0000 7f9cbf7fe700 1 --2- 172.21.15.119:0/372398033 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f71a91b40 0x562f71a9acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:27.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.364+0000 7f9cbeffd700 1 -- 172.21.15.119:0/372398033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f71a96150 msgr2=0x562f71a9b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:27.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.364+0000 7f9cbeffd700 1 --2- 172.21.15.119:0/372398033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f71a96150 0x562f71a9b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.364+0000 7f9cbeffd700 1 -- 172.21.15.119:0/372398033 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f71a91b40 msgr2=0x562f71a9acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:27.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.364+0000 7f9cbeffd700 1 --2- 172.21.15.119:0/372398033 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f71a91b40 0x562f71a9acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.364+0000 7f9cbeffd700 1 -- 172.21.15.119:0/372398033 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb4020430 con 0x562f71a92460 2023-12-07T10:22:27.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.364+0000 7f9cbffff700 1 --2- 172.21.15.119:0/372398033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f71a96150 0x562f71a9b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:22:27.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.364+0000 7f9cbf7fe700 1 --2- 172.21.15.119:0/372398033 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f71a91b40 0x562f71a9acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:22:27.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.364+0000 7f9cbeffd700 1 --2- 172.21.15.119:0/372398033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a92460 0x562f71a9b230 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4020a10 tx=0x7f9cb40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9693d49a2c88ea29 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:27.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.364+0000 7f9c9effd700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:22:27.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.364+0000 7f9c9effd700 1 -- 172.21.15.119:0/372398033 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4018040 con 0x562f71a92460 2023-12-07T10:22:27.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.364+0000 7f9cd95feb80 1 -- 172.21.15.119:0/372398033 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f71a84b00 con 0x562f71a92460 2023-12-07T10:22:27.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.364+0000 7f9c9effd700 1 -- 172.21.15.119:0/372398033 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9cb4010bd0 con 0x562f71a92460 2023-12-07T10:22:27.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.365+0000 7f9c9effd700 1 -- 172.21.15.119:0/372398033 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cb4010d90 con 0x562f71a92460 2023-12-07T10:22:27.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.365+0000 7f9cd95feb80 10 client.16290 fetch_fsmap learned FSMap version 32 2023-12-07T10:22:27.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.365+0000 7f9cd95feb80 1 -- 172.21.15.119:0/372398033 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f9cc01a6960 con 0x562f71a92460 2023-12-07T10:22:27.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.365+0000 7f9c9effd700 1 -- 172.21.15.119:0/372398033 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562f71a84b00 con 0x562f71a92460 2023-12-07T10:22:27.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.365+0000 7f9c9effd700 1 -- 172.21.15.119:0/372398033 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f9cb4026030 con 0x562f71a92460 2023-12-07T10:22:27.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.365+0000 7f9cd95feb80 10 client.16290 fetch_fsmap finished waiting for FSMap version 32 2023-12-07T10:22:27.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.365+0000 7f9cd95feb80 10 client.16290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:22:27.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.365+0000 7f9cd95feb80 20 client.16290 populate_metadata read hostname 'smithi119' 2023-12-07T10:22:27.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.365+0000 7f9cd95feb80 1 --2- 172.21.15.119:0/372398033 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562f71aab200 0x562f71aad620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:27.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.365+0000 7f9cd95feb80 4 client.16290 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:22:27.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.365+0000 7f9cd95feb80 1 -- 172.21.15.119:0/372398033 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f71a607a0 con 0x562f71aab200 2023-12-07T10:22:27.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.366+0000 7f9cbf7fe700 1 --2- 172.21.15.119:0/372398033 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562f71aab200 0x562f71aad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:27.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.366+0000 7f9c9effd700 10 client.16290 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:27.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.366+0000 7f9cbf7fe700 1 --2- 172.21.15.119:0/372398033 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562f71aab200 0x562f71aad620 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a4880f312f4bda server_cookie=bbce3b2bd1e76c8d in_seq=0 out_seq=0 2023-12-07T10:22:27.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.367+0000 7f9c9effd700 1 -- 172.21.15.119:0/372398033 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562f71a607a0 con 0x562f71aab200 2023-12-07T10:22:27.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.367+0000 7f9c9effd700 10 client.16290 handle_command_reply: tid=0 2023-12-07T10:22:27.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.367+0000 7f9cd95feb80 1 client.16290 shutdown 2023-12-07T10:22:27.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.367+0000 7f9cd95feb80 1 -- 172.21.15.119:0/372398033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a92460 msgr2=0x562f71a9b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:27.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.367+0000 7f9cd95feb80 1 --2- 172.21.15.119:0/372398033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a92460 0x562f71a9b230 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4020a10 tx=0x7f9cb40047c0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.367+0000 7f9cd95feb80 1 -- 172.21.15.119:0/372398033 shutdown_connections 2023-12-07T10:22:27.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.367+0000 7f9cd95feb80 1 --2- 172.21.15.119:0/372398033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f71a96150 0x562f71a9b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.367+0000 7f9cd95feb80 1 --2- 172.21.15.119:0/372398033 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562f71aab200 0x562f71aad620 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.367+0000 7f9cd95feb80 1 --2- 172.21.15.119:0/372398033 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f71a91b40 0x562f71a9acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.367+0000 7f9cd95feb80 1 --2- 172.21.15.119:0/372398033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a92460 0x562f71a9b230 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.367+0000 7f9c9effd700 0 client.16290 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:27.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.367+0000 7f9cd95feb80 1 -- 172.21.15.119:0/372398033 >> 172.21.15.119:0/372398033 conn(0x562f719c0360 msgr2=0x562f71a97620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:27.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.367+0000 7f9cd95feb80 1 -- 172.21.15.119:0/372398033 shutdown_connections 2023-12-07T10:22:27.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.367+0000 7f9cd95feb80 1 -- 172.21.15.119:0/372398033 wait complete. 2023-12-07T10:22:27.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.367+0000 7f9cd95feb80 20 client.16290 trim_cache size 0 max 16384 2023-12-07T10:22:27.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.385+0000 7f9cd95feb80 1 Processor -- start 2023-12-07T10:22:27.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.385+0000 7f9cd95feb80 1 -- start start 2023-12-07T10:22:27.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.385+0000 7f9cd95feb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a91b40 0x562f71a9b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:27.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.385+0000 7f9cd95feb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f71a92460 0x562f71a943b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:27.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.385+0000 7f9cd95feb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f71a96150 0x562f71a948f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:27.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.385+0000 7f9cd95feb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f71864920 con 0x562f71a91b40 2023-12-07T10:22:27.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.385+0000 7f9cd95feb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f71a9c2a0 con 0x562f71a96150 2023-12-07T10:22:27.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.385+0000 7f9cd95feb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f71a84ae0 con 0x562f71a92460 2023-12-07T10:22:27.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.385+0000 7f9cbf7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a91b40 0x562f71a9b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:27.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.386+0000 7f9cbf7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a91b40 0x562f71a9b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44818/0 (socket says 172.21.15.119:44818) 2023-12-07T10:22:27.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.386+0000 7f9cbf7fe700 1 -- 172.21.15.119:0/4196518697 learned_addr learned my addr 172.21.15.119:0/4196518697 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:27.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.386+0000 7f9cbffff700 1 --2- 172.21.15.119:0/4196518697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f71a96150 0x562f71a948f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:27.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.386+0000 7f9cbeffd700 1 --2- 172.21.15.119:0/4196518697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f71a92460 0x562f71a943b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:27.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.386+0000 7f9cbf7fe700 1 -- 172.21.15.119:0/4196518697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f71a92460 msgr2=0x562f71a943b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:27.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.386+0000 7f9cbf7fe700 1 --2- 172.21.15.119:0/4196518697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f71a92460 0x562f71a943b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.386+0000 7f9cbf7fe700 1 -- 172.21.15.119:0/4196518697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f71a96150 msgr2=0x562f71a948f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:27.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.386+0000 7f9cbf7fe700 1 --2- 172.21.15.119:0/4196518697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f71a96150 0x562f71a948f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.386+0000 7f9cbf7fe700 1 -- 172.21.15.119:0/4196518697 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f7180ede0 con 0x562f71a91b40 2023-12-07T10:22:27.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.386+0000 7f9cbf7fe700 1 --2- 172.21.15.119:0/4196518697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a91b40 0x562f71a9b600 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x562f71aadf30 tx=0x7f9cb0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a8f355255e2bae server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:27.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.386+0000 7f9ca67fc700 1 -- 172.21.15.119:0/4196518697 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0003be0 con 0x562f71a91b40 2023-12-07T10:22:27.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.386+0000 7f9ca67fc700 1 -- 172.21.15.119:0/4196518697 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cb0018040 con 0x562f71a91b40 2023-12-07T10:22:27.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.386+0000 7f9ca67fc700 1 -- 172.21.15.119:0/4196518697 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0010b30 con 0x562f71a91b40 2023-12-07T10:22:27.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.387+0000 7f9cd95feb80 1 -- 172.21.15.119:0/4196518697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a91b40 msgr2=0x562f71a9b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:27.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.387+0000 7f9cd95feb80 1 --2- 172.21.15.119:0/4196518697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a91b40 0x562f71a9b600 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x562f71aadf30 tx=0x7f9cb0004750 comp rx=0 tx=0).stop 2023-12-07T10:22:27.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.387+0000 7f9cd95feb80 1 -- 172.21.15.119:0/4196518697 shutdown_connections 2023-12-07T10:22:27.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.387+0000 7f9cd95feb80 1 --2- 172.21.15.119:0/4196518697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f71a96150 0x562f71a948f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.387+0000 7f9cd95feb80 1 --2- 172.21.15.119:0/4196518697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a91b40 0x562f71a9b600 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.387+0000 7f9cd95feb80 1 --2- 172.21.15.119:0/4196518697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f71a92460 0x562f71a943b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.387+0000 7f9cd95feb80 1 -- 172.21.15.119:0/4196518697 >> 172.21.15.119:0/4196518697 conn(0x562f719c0360 msgr2=0x562f71ab1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:27.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.387+0000 7f9cd95feb80 1 -- 172.21.15.119:0/4196518697 shutdown_connections 2023-12-07T10:22:27.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.387+0000 7f9cd95feb80 1 -- 172.21.15.119:0/4196518697 wait complete. 2023-12-07T10:22:27.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.387+0000 7f9cd95feb80 1 -- start start 2023-12-07T10:22:27.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.388+0000 7f9cd95feb80 1 Processor -- start 2023-12-07T10:22:27.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.388+0000 7f9cd95feb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f71a91b40 0x562f71aa2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:27.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.388+0000 7f9cd95feb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f71a92460 0x562f71aa0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:27.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.388+0000 7f9cd95feb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a96150 0x562f71aa0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:27.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.388+0000 7f9cd95feb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f71aa2a20 con 0x562f71a96150 2023-12-07T10:22:27.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.388+0000 7f9cd95feb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f71aa0f80 con 0x562f71a91b40 2023-12-07T10:22:27.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.388+0000 7f9cd95feb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f71aa10f0 con 0x562f71a92460 2023-12-07T10:22:27.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.388+0000 7f9cd95feb80 10 client.0 _ll_register_callbacks cb 0x562f71986940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:22:27.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.388+0000 7f9cbffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a96150 0x562f71aa0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:27.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.388+0000 7f9cbf7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f71a91b40 0x562f71aa2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:27.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.388+0000 7f9cbffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a96150 0x562f71aa0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44834/0 (socket says 172.21.15.119:44834) 2023-12-07T10:22:27.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.388+0000 7f9cbffff700 1 -- 172.21.15.119:0/1193999150 learned_addr learned my addr 172.21.15.119:0/1193999150 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:27.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.388+0000 7f9cbeffd700 1 --2- 172.21.15.119:0/1193999150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f71a92460 0x562f71aa0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:27.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.389+0000 7f9cbffff700 1 -- 172.21.15.119:0/1193999150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f71a92460 msgr2=0x562f71aa0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:27.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.389+0000 7f9cbffff700 1 --2- 172.21.15.119:0/1193999150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f71a92460 0x562f71aa0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.389+0000 7f9cbffff700 1 -- 172.21.15.119:0/1193999150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f71a91b40 msgr2=0x562f71aa2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:27.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.389+0000 7f9cbffff700 1 --2- 172.21.15.119:0/1193999150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f71a91b40 0x562f71aa2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.389+0000 7f9cbffff700 1 -- 172.21.15.119:0/1193999150 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb0004040 con 0x562f71a96150 2023-12-07T10:22:27.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.389+0000 7f9cbf7fe700 1 --2- 172.21.15.119:0/1193999150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f71a91b40 0x562f71aa2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:27.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.389+0000 7f9cbffff700 1 --2- 172.21.15.119:0/1193999150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a96150 0x562f71aa0a40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80a37a0 tx=0x7f9cb8009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b048ca1a347ea32 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:27.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.389+0000 7f9c9effd700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:22:27.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.389+0000 7f9c9effd700 1 -- 172.21.15.119:0/1193999150 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb80bb2b0 con 0x562f71a96150 2023-12-07T10:22:27.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.389+0000 7f9cd95feb80 1 -- 172.21.15.119:0/1193999150 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cb809b9d0 con 0x562f71a96150 2023-12-07T10:22:27.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.389+0000 7f9c9effd700 1 -- 172.21.15.119:0/1193999150 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9cb80b9e40 con 0x562f71a96150 2023-12-07T10:22:27.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.389+0000 7f9c9effd700 1 -- 172.21.15.119:0/1193999150 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cb80b4240 con 0x562f71a96150 2023-12-07T10:22:27.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.389+0000 7f9cd95feb80 10 client.16296 fetch_fsmap learned FSMap version 32 2023-12-07T10:22:27.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.389+0000 7f9cd95feb80 1 -- 172.21.15.119:0/1193999150 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562f717f7c60 con 0x562f71a96150 2023-12-07T10:22:27.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.390+0000 7f9c9effd700 1 -- 172.21.15.119:0/1193999150 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cb809b9d0 con 0x562f71a96150 2023-12-07T10:22:27.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.390+0000 7f9c9effd700 1 -- 172.21.15.119:0/1193999150 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f9cb809aec0 con 0x562f71a96150 2023-12-07T10:22:27.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.390+0000 7f9cd95feb80 10 client.16296 fetch_fsmap finished waiting for FSMap version 32 2023-12-07T10:22:27.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.390+0000 7f9cd95feb80 10 client.16296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:22:27.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.390+0000 7f9cd95feb80 20 client.16296 populate_metadata read hostname 'smithi119' 2023-12-07T10:22:27.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.390+0000 7f9cd95feb80 1 --2- 172.21.15.119:0/1193999150 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562f71b02820 0x562f71b04c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:27.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.390+0000 7f9cd95feb80 4 client.16296 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-12-07T10:22:27.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.390+0000 7f9cd95feb80 1 -- 172.21.15.119:0/1193999150 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562f719abfd0 con 0x562f71b02820 2023-12-07T10:22:27.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.390+0000 7f9cbf7fe700 1 --2- 172.21.15.119:0/1193999150 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562f71b02820 0x562f71b04c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:27.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.391+0000 7f9c9effd700 10 client.16296 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:27.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.391+0000 7f9cbf7fe700 1 --2- 172.21.15.119:0/1193999150 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562f71b02820 0x562f71b04c40 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c67bde14c2760b82 server_cookie=5ecc9686182bbe38 in_seq=0 out_seq=0 2023-12-07T10:22:27.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.391+0000 7f9c9effd700 1 -- 172.21.15.119:0/1193999150 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562f719abfd0 con 0x562f71b02820 2023-12-07T10:22:27.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.391+0000 7f9c9effd700 10 client.16296 handle_command_reply: tid=0 2023-12-07T10:22:27.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.391+0000 7f9cd95feb80 1 client.16296 shutdown 2023-12-07T10:22:27.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.391+0000 7f9cd95feb80 1 -- 172.21.15.119:0/1193999150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a96150 msgr2=0x562f71aa0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:27.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.391+0000 7f9cd95feb80 1 --2- 172.21.15.119:0/1193999150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a96150 0x562f71aa0a40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80a37a0 tx=0x7f9cb8009990 comp rx=0 tx=0).stop 2023-12-07T10:22:27.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.391+0000 7f9cd95feb80 1 -- 172.21.15.119:0/1193999150 shutdown_connections 2023-12-07T10:22:27.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.391+0000 7f9cd95feb80 1 --2- 172.21.15.119:0/1193999150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f71a96150 0x562f71aa0a40 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.391+0000 7f9cd95feb80 1 --2- 172.21.15.119:0/1193999150 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562f71b02820 0x562f71b04c40 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.391+0000 7f9cd95feb80 1 --2- 172.21.15.119:0/1193999150 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f71a91b40 0x562f71aa2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.391+0000 7f9c9effd700 0 client.16296 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:27.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.391+0000 7f9cd95feb80 1 --2- 172.21.15.119:0/1193999150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f71a92460 0x562f71aa0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.391+0000 7f9cd95feb80 1 -- 172.21.15.119:0/1193999150 >> 172.21.15.119:0/1193999150 conn(0x562f719c0360 msgr2=0x562f71a9b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:27.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.391+0000 7f9cd95feb80 1 -- 172.21.15.119:0/1193999150 shutdown_connections 2023-12-07T10:22:27.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.392+0000 7f9cd95feb80 1 -- 172.21.15.119:0/1193999150 wait complete. 2023-12-07T10:22:27.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.392+0000 7f9cd95feb80 20 client.16296 trim_cache size 0 max 16384 2023-12-07T10:22:27.399 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:22:27.399 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "no active scrubs running", 2023-12-07T10:22:27.399 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": {} 2023-12-07T10:22:27.399 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:22:27.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.392+0000 7f9cc5bff700 1 -- 172.21.15.119:0/797543573 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9ca80a1c80 msgr2=0x7f9ca80a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:27.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.392+0000 7f9cc5bff700 1 --2- 172.21.15.119:0/797543573 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9ca80a1c80 0x7f9ca80a4140 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9cc019b320 tx=0x7f9cb4013840 comp rx=0 tx=0).stop 2023-12-07T10:22:27.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.392+0000 7f9cc5bff700 1 -- 172.21.15.119:0/797543573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cc010dfb0 msgr2=0x7f9cc019e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:27.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.392+0000 7f9cc5bff700 1 --2- 172.21.15.119:0/797543573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cc010dfb0 0x7f9cc019e6b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8009f60 tx=0x7f9cb8004ed0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.392+0000 7f9cc5bff700 1 -- 172.21.15.119:0/797543573 shutdown_connections 2023-12-07T10:22:27.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.392+0000 7f9cc5bff700 1 --2- 172.21.15.119:0/797543573 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9ca80a1c80 0x7f9ca80a4140 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.392+0000 7f9cc5bff700 1 --2- 172.21.15.119:0/797543573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cc010dfb0 0x7f9cc019e6b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.392+0000 7f9cc5bff700 1 --2- 172.21.15.119:0/797543573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cc0100f40 0x7f9cc0199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.392+0000 7f9cc5bff700 1 --2- 172.21.15.119:0/797543573 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cc0101860 0x7f9cc019a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.392+0000 7f9cc5bff700 1 -- 172.21.15.119:0/797543573 >> 172.21.15.119:0/797543573 conn(0x7f9cc0077610 msgr2=0x7f9cc010f670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:27.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.392+0000 7f9cc5bff700 1 -- 172.21.15.119:0/797543573 shutdown_connections 2023-12-07T10:22:27.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.392+0000 7f9cc5bff700 1 -- 172.21.15.119:0/797543573 wait complete. 2023-12-07T10:22:27.411 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-07T10:22:27.411 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:22:27.545 INFO:tasks.workunit.client.0.smithi119.stdout: 1 67793 9.61 MB/sec warmup 40 sec latency 3299.478 ms 2023-12-07T10:22:27.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.608+0000 7fde203ff700 1 -- 172.21.15.119:0/410493826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde181014e0 msgr2=0x7fde18101960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:27.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.608+0000 7fde203ff700 1 --2- 172.21.15.119:0/410493826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde181014e0 0x7fde18101960 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fde08009990 tx=0x7fde08009ca0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.608+0000 7fde203ff700 1 -- 172.21.15.119:0/410493826 shutdown_connections 2023-12-07T10:22:27.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.608+0000 7fde203ff700 1 --2- 172.21.15.119:0/410493826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde18101ea0 0x7fde1810e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.608+0000 7fde203ff700 1 --2- 172.21.15.119:0/410493826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde181075a0 0x7fde18107980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.608+0000 7fde203ff700 1 --2- 172.21.15.119:0/410493826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde181014e0 0x7fde18101960 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.608+0000 7fde203ff700 1 -- 172.21.15.119:0/410493826 >> 172.21.15.119:0/410493826 conn(0x7fde180fcff0 msgr2=0x7fde180ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:27.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.608+0000 7fde203ff700 1 -- 172.21.15.119:0/410493826 shutdown_connections 2023-12-07T10:22:27.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.608+0000 7fde203ff700 1 -- 172.21.15.119:0/410493826 wait complete. 2023-12-07T10:22:27.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.610+0000 7fde203ff700 1 Processor -- start 2023-12-07T10:22:27.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.610+0000 7fde203ff700 1 -- start start 2023-12-07T10:22:27.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.610+0000 7fde203ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde181014e0 0x7fde1819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:27.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.610+0000 7fde203ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde18101ea0 0x7fde1819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:27.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.610+0000 7fde203ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde181075a0 0x7fde18199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:27.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.610+0000 7fde203ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde18199f80 con 0x7fde181075a0 2023-12-07T10:22:27.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.610+0000 7fde203ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1819a0f0 con 0x7fde18101ea0 2023-12-07T10:22:27.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.610+0000 7fde203ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1819a2b0 con 0x7fde181014e0 2023-12-07T10:22:27.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.611+0000 7fde1e99c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde181075a0 0x7fde18199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:27.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.611+0000 7fde1e99c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde181075a0 0x7fde18199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44852/0 (socket says 172.21.15.119:44852) 2023-12-07T10:22:27.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.611+0000 7fde1e99c700 1 -- 172.21.15.119:0/1832563783 learned_addr learned my addr 172.21.15.119:0/1832563783 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:27.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.611+0000 7fde1d99a700 1 --2- 172.21.15.119:0/1832563783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde18101ea0 0x7fde1819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:27.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.611+0000 7fde1e19b700 1 --2- 172.21.15.119:0/1832563783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde181014e0 0x7fde1819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:27.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.611+0000 7fde1e99c700 1 -- 172.21.15.119:0/1832563783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde181014e0 msgr2=0x7fde1819f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:27.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.611+0000 7fde1e99c700 1 --2- 172.21.15.119:0/1832563783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde181014e0 0x7fde1819f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.611+0000 7fde1e99c700 1 -- 172.21.15.119:0/1832563783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde18101ea0 msgr2=0x7fde1819fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:27.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.611+0000 7fde1e99c700 1 --2- 172.21.15.119:0/1832563783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde18101ea0 0x7fde1819fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.611+0000 7fde1e99c700 1 -- 172.21.15.119:0/1832563783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde080095f0 con 0x7fde181075a0 2023-12-07T10:22:27.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.611+0000 7fde1e99c700 1 --2- 172.21.15.119:0/1832563783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde181075a0 0x7fde18199920 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fde1000ef30 tx=0x7fde1000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:27.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.612+0000 7fde0f7fe700 1 -- 172.21.15.119:0/1832563783 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde10018070 con 0x7fde181075a0 2023-12-07T10:22:27.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.612+0000 7fde0f7fe700 1 -- 172.21.15.119:0/1832563783 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde1000f040 con 0x7fde181075a0 2023-12-07T10:22:27.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.612+0000 7fde203ff700 1 -- 172.21.15.119:0/1832563783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1819a530 con 0x7fde181075a0 2023-12-07T10:22:27.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.612+0000 7fde0f7fe700 1 -- 172.21.15.119:0/1832563783 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1000cd00 con 0x7fde181075a0 2023-12-07T10:22:27.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.612+0000 7fde203ff700 1 -- 172.21.15.119:0/1832563783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde181a67e0 con 0x7fde181075a0 2023-12-07T10:22:27.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.613+0000 7fde203ff700 1 -- 172.21.15.119:0/1832563783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde18102c00 con 0x7fde181075a0 2023-12-07T10:22:27.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.615+0000 7fde0f7fe700 1 -- 172.21.15.119:0/1832563783 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fde10003a00 con 0x7fde181075a0 2023-12-07T10:22:27.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.616+0000 7fde0f7fe700 1 --2- 172.21.15.119:0/1832563783 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fde040a1c80 0x7fde040a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:27.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.616+0000 7fde0f7fe700 1 -- 172.21.15.119:0/1832563783 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fde100cd8d0 con 0x7fde181075a0 2023-12-07T10:22:27.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.616+0000 7fde1e19b700 1 --2- 172.21.15.119:0/1832563783 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fde040a1c80 0x7fde040a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:27.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.617+0000 7fde1e19b700 1 --2- 172.21.15.119:0/1832563783 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fde040a1c80 0x7fde040a4140 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fde14009c60 tx=0x7fde14009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:27.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.621+0000 7fde0f7fe700 1 -- 172.21.15.119:0/1832563783 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fde10096690 con 0x7fde181075a0 2023-12-07T10:22:27.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.870+0000 7fde203ff700 1 -- 172.21.15.119:0/1832563783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde18068320 con 0x7fde181075a0 2023-12-07T10:22:27.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.870+0000 7fde0f7fe700 1 -- 172.21.15.119:0/1832563783 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11391 (secure 0 0 0) 0x7fde10099cb0 con 0x7fde181075a0 2023-12-07T10:22:27.872 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:22:27.872 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-07T10:17:41.278311+0000","modified":"2023-12-07T10:22:18.053011+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24517,"mds_3":24505,"mds_4":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14535":{"gid":14535,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6841/1791998028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1791998028},{"type":"v1","addr":"172.21.15.119:6841","nonce":1791998028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24505":{"gid":24505,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/2925503246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2925503246},{"type":"v1","addr":"172.21.15.153:6837","nonce":2925503246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6839/3984962900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3984962900},{"type":"v1","addr":"172.21.15.159:6839","nonce":3984962900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.119:6839/1920762602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1920762602},{"type":"v1","addr":"172.21.15.119:6839","nonce":1920762602}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/566825184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":566825184},{"type":"v1","addr":"172.21.15.153:6839","nonce":566825184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-07T10:22:27.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.873+0000 7fde203ff700 1 -- 172.21.15.119:0/1832563783 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fde040a1c80 msgr2=0x7fde040a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:27.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.873+0000 7fde203ff700 1 --2- 172.21.15.119:0/1832563783 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fde040a1c80 0x7fde040a4140 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fde14009c60 tx=0x7fde14009260 comp rx=0 tx=0).stop 2023-12-07T10:22:27.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.873+0000 7fde203ff700 1 -- 172.21.15.119:0/1832563783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde181075a0 msgr2=0x7fde18199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:27.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.873+0000 7fde203ff700 1 --2- 172.21.15.119:0/1832563783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde181075a0 0x7fde18199920 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fde1000ef30 tx=0x7fde1000c3a0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.873+0000 7fde203ff700 1 -- 172.21.15.119:0/1832563783 shutdown_connections 2023-12-07T10:22:27.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.873+0000 7fde203ff700 1 --2- 172.21.15.119:0/1832563783 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fde040a1c80 0x7fde040a4140 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.873+0000 7fde203ff700 1 --2- 172.21.15.119:0/1832563783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde181075a0 0x7fde18199920 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.873+0000 7fde203ff700 1 --2- 172.21.15.119:0/1832563783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde181014e0 0x7fde1819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.873+0000 7fde203ff700 1 --2- 172.21.15.119:0/1832563783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde18101ea0 0x7fde1819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:27.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.873+0000 7fde203ff700 1 -- 172.21.15.119:0/1832563783 >> 172.21.15.119:0/1832563783 conn(0x7fde180fcff0 msgr2=0x7fde180fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:27.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.873+0000 7fde203ff700 1 -- 172.21.15.119:0/1832563783 shutdown_connections 2023-12-07T10:22:27.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:27.873+0000 7fde203ff700 1 -- 172.21.15.119:0/1832563783 wait complete. 2023-12-07T10:22:27.875 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 32 2023-12-07T10:22:27.893 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-12-07T10:22:28.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.093+0000 7f0ea7bff700 1 -- 172.21.15.119:0/3828068906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea0107620 msgr2=0x7f0ea0107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.093+0000 7f0ea7bff700 1 --2- 172.21.15.119:0/3828068906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea0107620 0x7f0ea0107a00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0099c0 tx=0x7f0e9c009cd0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.093+0000 7f0ea7bff700 1 -- 172.21.15.119:0/3828068906 shutdown_connections 2023-12-07T10:22:28.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.093+0000 7f0ea7bff700 1 --2- 172.21.15.119:0/3828068906 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ea0101f20 0x7f0ea010e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.093+0000 7f0ea7bff700 1 --2- 172.21.15.119:0/3828068906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea0107620 0x7f0ea0107a00 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.093+0000 7f0ea7bff700 1 --2- 172.21.15.119:0/3828068906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ea0101560 0x7f0ea01019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.093+0000 7f0ea7bff700 1 -- 172.21.15.119:0/3828068906 >> 172.21.15.119:0/3828068906 conn(0x7f0ea00fcfd0 msgr2=0x7f0ea00ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:28.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.093+0000 7f0ea7bff700 1 -- 172.21.15.119:0/3828068906 shutdown_connections 2023-12-07T10:22:28.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.093+0000 7f0ea7bff700 1 -- 172.21.15.119:0/3828068906 wait complete. 2023-12-07T10:22:28.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.094+0000 7f0ea7bff700 1 Processor -- start 2023-12-07T10:22:28.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.094+0000 7f0ea7bff700 1 -- start start 2023-12-07T10:22:28.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.094+0000 7f0ea7bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ea0101560 0x7f0ea019d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.094+0000 7f0ea7bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea0101f20 0x7f0ea019dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.094+0000 7f0ea7bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ea0107620 0x7f0ea0197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.094+0000 7f0ea7bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea0197ec0 con 0x7f0ea0101f20 2023-12-07T10:22:28.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.094+0000 7f0ea7bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea0198030 con 0x7f0ea0107620 2023-12-07T10:22:28.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.094+0000 7f0ea7bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea01981f0 con 0x7f0ea0101560 2023-12-07T10:22:28.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.095+0000 7f0ea519a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea0101f20 0x7f0ea019dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.095+0000 7f0ea599b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ea0101560 0x7f0ea019d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.095+0000 7f0ea519a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea0101f20 0x7f0ea019dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44866/0 (socket says 172.21.15.119:44866) 2023-12-07T10:22:28.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.095+0000 7f0ea519a700 1 -- 172.21.15.119:0/1747366890 learned_addr learned my addr 172.21.15.119:0/1747366890 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:28.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.095+0000 7f0ea619c700 1 --2- 172.21.15.119:0/1747366890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ea0107620 0x7f0ea0197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.095+0000 7f0ea519a700 1 -- 172.21.15.119:0/1747366890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ea0101560 msgr2=0x7f0ea019d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.095+0000 7f0ea519a700 1 --2- 172.21.15.119:0/1747366890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ea0101560 0x7f0ea019d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.095+0000 7f0ea519a700 1 -- 172.21.15.119:0/1747366890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ea0107620 msgr2=0x7f0ea0197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.095+0000 7f0ea519a700 1 --2- 172.21.15.119:0/1747366890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ea0107620 0x7f0ea0197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.095+0000 7f0ea519a700 1 -- 172.21.15.119:0/1747366890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e9c0095f0 con 0x7f0ea0101f20 2023-12-07T10:22:28.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.095+0000 7f0ea519a700 1 --2- 172.21.15.119:0/1747366890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea0101f20 0x7f0ea019dc60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0e9000e9a0 tx=0x7f0e9000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:28.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.095+0000 7f0e96ffd700 1 -- 172.21.15.119:0/1747366890 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e90009790 con 0x7f0ea0101f20 2023-12-07T10:22:28.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.095+0000 7f0ea7bff700 1 -- 172.21.15.119:0/1747366890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea01984d0 con 0x7f0ea0101f20 2023-12-07T10:22:28.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.095+0000 7f0e96ffd700 1 -- 172.21.15.119:0/1747366890 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e9000cbd0 con 0x7f0ea0101f20 2023-12-07T10:22:28.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.095+0000 7f0e96ffd700 1 -- 172.21.15.119:0/1747366890 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e90018680 con 0x7f0ea0101f20 2023-12-07T10:22:28.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.096+0000 7f0ea7bff700 1 -- 172.21.15.119:0/1747366890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea01a4710 con 0x7f0ea0101f20 2023-12-07T10:22:28.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.099+0000 7f0e96ffd700 1 -- 172.21.15.119:0/1747366890 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f0e9000cd40 con 0x7f0ea0101f20 2023-12-07T10:22:28.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.101+0000 7f0ebb5ffb80 1 Processor -- start 2023-12-07T10:22:28.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.101+0000 7f0e96ffd700 1 --2- 172.21.15.119:0/1747366890 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0e8c0a1c80 0x7f0e8c0a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.101+0000 7f0ebb5ffb80 1 -- start start 2023-12-07T10:22:28.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.101+0000 7f0e96ffd700 1 -- 172.21.15.119:0/1747366890 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f0e90014070 con 0x7f0ea0101f20 2023-12-07T10:22:28.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.101+0000 7f0ea619c700 1 --2- 172.21.15.119:0/1747366890 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0e8c0a1c80 0x7f0e8c0a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.101+0000 7f0ebb5ffb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ed6152f50 0x555ed6153330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.101+0000 7f0ebb5ffb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ed6153870 0x555ed6157020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.101+0000 7f0ebb5ffb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ed6157560 0x555ed6157980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.101+0000 7f0ebb5ffb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ed60c27c0 con 0x555ed6152f50 2023-12-07T10:22:28.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.101+0000 7f0ebb5ffb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ed60a97a0 con 0x555ed6157560 2023-12-07T10:22:28.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.101+0000 7f0ebb5ffb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ed60a70c0 con 0x555ed6153870 2023-12-07T10:22:28.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.102+0000 7f0ea599b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ed6152f50 0x555ed6153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.102+0000 7f0ea599b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ed6152f50 0x555ed6153330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44878/0 (socket says 172.21.15.119:44878) 2023-12-07T10:22:28.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.102+0000 7f0ea599b700 1 -- 172.21.15.119:0/2333357100 learned_addr learned my addr 172.21.15.119:0/2333357100 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:28.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.102+0000 7f0ea519a700 1 --2- 172.21.15.119:0/2333357100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ed6153870 0x555ed6157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.102+0000 7f0ea619c700 1 --2- 172.21.15.119:0/2333357100 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ed6157560 0x555ed6157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.102+0000 7f0ea619c700 1 --2- 172.21.15.119:0/1747366890 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0e8c0a1c80 0x7f0e8c0a4140 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0ea019e420 tx=0x7f0e98009040 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:28.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.102+0000 7f0ea599b700 1 -- 172.21.15.119:0/2333357100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ed6153870 msgr2=0x555ed6157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.102+0000 7f0ea599b700 1 --2- 172.21.15.119:0/2333357100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ed6153870 0x555ed6157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.102+0000 7f0ea599b700 1 -- 172.21.15.119:0/2333357100 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ed6157560 msgr2=0x555ed6157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.102+0000 7f0ea599b700 1 --2- 172.21.15.119:0/2333357100 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ed6157560 0x555ed6157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.102+0000 7f0ea599b700 1 -- 172.21.15.119:0/2333357100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ed5f1f630 con 0x555ed6152f50 2023-12-07T10:22:28.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.102+0000 7f0ea519a700 1 --2- 172.21.15.119:0/2333357100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ed6153870 0x555ed6157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:22:28.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.102+0000 7f0ea599b700 1 --2- 172.21.15.119:0/2333357100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ed6152f50 0x555ed6153330 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c015000 tx=0x7f0e9c005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17be56f0c382ceac server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:28.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.103+0000 7f0e8bfff700 1 -- 172.21.15.119:0/2333357100 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c01c050 con 0x555ed6152f50 2023-12-07T10:22:28.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.103+0000 7f0e8bfff700 1 -- 172.21.15.119:0/2333357100 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e9c01d040 con 0x555ed6152f50 2023-12-07T10:22:28.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.103+0000 7f0e8bfff700 1 -- 172.21.15.119:0/2333357100 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c0038c0 con 0x555ed6152f50 2023-12-07T10:22:28.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.103+0000 7f0ebb5ffb80 1 -- 172.21.15.119:0/2333357100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ed6152f50 msgr2=0x555ed6153330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.103+0000 7f0ebb5ffb80 1 --2- 172.21.15.119:0/2333357100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ed6152f50 0x555ed6153330 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c015000 tx=0x7f0e9c005fb0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.103+0000 7f0ebb5ffb80 1 -- 172.21.15.119:0/2333357100 shutdown_connections 2023-12-07T10:22:28.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.103+0000 7f0ebb5ffb80 1 --2- 172.21.15.119:0/2333357100 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ed6157560 0x555ed6157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.103+0000 7f0ebb5ffb80 1 --2- 172.21.15.119:0/2333357100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ed6152f50 0x555ed6153330 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.103+0000 7f0ebb5ffb80 1 --2- 172.21.15.119:0/2333357100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ed6153870 0x555ed6157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.103+0000 7f0ebb5ffb80 1 -- 172.21.15.119:0/2333357100 >> 172.21.15.119:0/2333357100 conn(0x555ed607d360 msgr2=0x555ed6149130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:28.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.103+0000 7f0ebb5ffb80 1 -- 172.21.15.119:0/2333357100 shutdown_connections 2023-12-07T10:22:28.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.104+0000 7f0ebb5ffb80 1 -- 172.21.15.119:0/2333357100 wait complete. 2023-12-07T10:22:28.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.104+0000 7f0ebb5ffb80 1 -- start start 2023-12-07T10:22:28.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.104+0000 7f0ebb5ffb80 1 Processor -- start 2023-12-07T10:22:28.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.105+0000 7f0ebb5ffb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ed6152f50 0x555ed615eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.105+0000 7f0ebb5ffb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ed6153870 0x555ed615bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.105+0000 7f0ebb5ffb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ed6157560 0x555ed615c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.105+0000 7f0ebb5ffb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ed60a29b0 con 0x555ed6157560 2023-12-07T10:22:28.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.105+0000 7f0ebb5ffb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ed605c560 con 0x555ed6153870 2023-12-07T10:22:28.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.105+0000 7f0ebb5ffb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ed605c200 con 0x555ed6152f50 2023-12-07T10:22:28.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.105+0000 7f0ebb5ffb80 10 client.0 _ll_register_callbacks cb 0x555ed6043940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:22:28.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.105+0000 7f0ea619c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ed6157560 0x555ed615c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.105+0000 7f0ea619c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ed6157560 0x555ed615c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44886/0 (socket says 172.21.15.119:44886) 2023-12-07T10:22:28.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.105+0000 7f0ea599b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ed6152f50 0x555ed615eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.105+0000 7f0ea619c700 1 -- 172.21.15.119:0/1783179633 learned_addr learned my addr 172.21.15.119:0/1783179633 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:28.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.105+0000 7f0ea519a700 1 --2- 172.21.15.119:0/1783179633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ed6153870 0x555ed615bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.105+0000 7f0ea619c700 1 -- 172.21.15.119:0/1783179633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ed6152f50 msgr2=0x555ed615eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.105+0000 7f0ea619c700 1 --2- 172.21.15.119:0/1783179633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ed6152f50 0x555ed615eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.105+0000 7f0ea619c700 1 -- 172.21.15.119:0/1783179633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ed6153870 msgr2=0x555ed615bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.105+0000 7f0ea619c700 1 --2- 172.21.15.119:0/1783179633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ed6153870 0x555ed615bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.105+0000 7f0ea619c700 1 -- 172.21.15.119:0/1783179633 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea010ece0 con 0x555ed6157560 2023-12-07T10:22:28.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.106+0000 7f0ea619c700 1 --2- 172.21.15.119:0/1783179633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ed6157560 0x555ed615c450 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0e980169d0 tx=0x7f0e98004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a7cff703ef4040 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:28.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.106+0000 7f0e897fa700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:22:28.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.106+0000 7f0e897fa700 1 -- 172.21.15.119:0/1783179633 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e98025070 con 0x555ed6157560 2023-12-07T10:22:28.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.106+0000 7f0e897fa700 1 -- 172.21.15.119:0/1783179633 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f0e98003a00 con 0x555ed6157560 2023-12-07T10:22:28.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.106+0000 7f0ebb5ffb80 1 -- 172.21.15.119:0/1783179633 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ed6141b00 con 0x555ed6157560 2023-12-07T10:22:28.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.106+0000 7f0e897fa700 1 -- 172.21.15.119:0/1783179633 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e98003bc0 con 0x555ed6157560 2023-12-07T10:22:28.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.106+0000 7f0ebb5ffb80 10 client.16314 fetch_fsmap learned FSMap version 32 2023-12-07T10:22:28.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.106+0000 7f0ebb5ffb80 1 -- 172.21.15.119:0/1783179633 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555ed5ecbde0 con 0x555ed6157560 2023-12-07T10:22:28.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.106+0000 7f0e897fa700 1 -- 172.21.15.119:0/1783179633 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555ed6141b00 con 0x555ed6157560 2023-12-07T10:22:28.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.107+0000 7f0e897fa700 1 -- 172.21.15.119:0/1783179633 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f0e98028020 con 0x555ed6157560 2023-12-07T10:22:28.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.107+0000 7f0ebb5ffb80 10 client.16314 fetch_fsmap finished waiting for FSMap version 32 2023-12-07T10:22:28.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.107+0000 7f0ebb5ffb80 10 client.16314 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2023-12-07T10:22:28.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.107+0000 7f0ebb5ffb80 20 client.16314 populate_metadata read hostname 'smithi119' 2023-12-07T10:22:28.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.107+0000 7f0ebb5ffb80 1 --2- 172.21.15.119:0/1783179633 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x555ed616c750 0x555ed616eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.107+0000 7f0ebb5ffb80 4 client.16314 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:22:28.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.107+0000 7f0ebb5ffb80 1 -- 172.21.15.119:0/1783179633 --> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ed611d7a0 con 0x555ed616c750 2023-12-07T10:22:28.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.107+0000 7f0ea599b700 1 --2- 172.21.15.119:0/1783179633 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x555ed616c750 0x555ed616eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.108+0000 7f0e897fa700 10 client.16314 ms_handle_connect on v2:172.21.15.153:6834/1295649464 2023-12-07T10:22:28.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.108+0000 7f0ea599b700 1 --2- 172.21.15.119:0/1783179633 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x555ed616c750 0x555ed616eb70 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a9034e1e8bf0eadf server_cookie=519f21cc064c834a in_seq=0 out_seq=0 2023-12-07T10:22:28.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.109+0000 7f0e897fa700 1 -- 172.21.15.119:0/1783179633 <== mds.4 v2:172.21.15.153:6834/1295649464 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555ed611d7a0 con 0x555ed616c750 2023-12-07T10:22:28.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.109+0000 7f0e897fa700 10 client.16314 handle_command_reply: tid=0 2023-12-07T10:22:28.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.109+0000 7f0ebb5ffb80 1 client.16314 shutdown 2023-12-07T10:22:28.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.109+0000 7f0ebb5ffb80 1 -- 172.21.15.119:0/1783179633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ed6157560 msgr2=0x555ed615c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.109+0000 7f0ebb5ffb80 1 --2- 172.21.15.119:0/1783179633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ed6157560 0x555ed615c450 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0e980169d0 tx=0x7f0e98004ad0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.109+0000 7f0ebb5ffb80 1 -- 172.21.15.119:0/1783179633 shutdown_connections 2023-12-07T10:22:28.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.109+0000 7f0ebb5ffb80 1 --2- 172.21.15.119:0/1783179633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ed6157560 0x555ed615c450 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.109+0000 7f0ebb5ffb80 1 --2- 172.21.15.119:0/1783179633 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x555ed616c750 0x555ed616eb70 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.109+0000 7f0ebb5ffb80 1 --2- 172.21.15.119:0/1783179633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ed6152f50 0x555ed615eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.109+0000 7f0ebb5ffb80 1 --2- 172.21.15.119:0/1783179633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ed6153870 0x555ed615bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.109+0000 7f0e897fa700 0 client.16314 ms_handle_reset on v2:172.21.15.153:6834/1295649464 2023-12-07T10:22:28.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.109+0000 7f0ebb5ffb80 1 -- 172.21.15.119:0/1783179633 >> 172.21.15.119:0/1783179633 conn(0x555ed607d360 msgr2=0x555ed6143d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:28.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.109+0000 7f0ebb5ffb80 1 -- 172.21.15.119:0/1783179633 shutdown_connections 2023-12-07T10:22:28.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.109+0000 7f0ebb5ffb80 1 -- 172.21.15.119:0/1783179633 wait complete. 2023-12-07T10:22:28.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.109+0000 7f0ebb5ffb80 20 client.16314 trim_cache size 0 max 16384 2023-12-07T10:22:28.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.123+0000 7f0ebb5ffb80 1 Processor -- start 2023-12-07T10:22:28.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.123+0000 7f0ebb5ffb80 1 -- start start 2023-12-07T10:22:28.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.123+0000 7f0ebb5ffb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ed6152f50 0x555ed61734d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.123+0000 7f0ebb5ffb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ed6153870 0x555ed6163700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.123+0000 7f0ebb5ffb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ed6157560 0x555ed6163c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.123+0000 7f0ebb5ffb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ed5f21920 con 0x555ed6152f50 2023-12-07T10:22:28.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.123+0000 7f0ebb5ffb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ed6141b10 con 0x555ed6153870 2023-12-07T10:22:28.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.123+0000 7f0ebb5ffb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ed615bcc0 con 0x555ed6157560 2023-12-07T10:22:28.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.123+0000 7f0ea519a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ed6153870 0x555ed6163700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.123+0000 7f0ea599b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ed6152f50 0x555ed61734d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.123+0000 7f0ea519a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ed6153870 0x555ed6163700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42448/0 (socket says 172.21.15.119:42448) 2023-12-07T10:22:28.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.123+0000 7f0ea519a700 1 -- 172.21.15.119:0/3416548069 learned_addr learned my addr 172.21.15.119:0/3416548069 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:28.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.124+0000 7f0ea619c700 1 --2- 172.21.15.119:0/3416548069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ed6157560 0x555ed6163c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.124+0000 7f0ea519a700 1 -- 172.21.15.119:0/3416548069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ed6157560 msgr2=0x555ed6163c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.124+0000 7f0ea519a700 1 --2- 172.21.15.119:0/3416548069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ed6157560 0x555ed6163c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.124+0000 7f0ea519a700 1 -- 172.21.15.119:0/3416548069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ed6152f50 msgr2=0x555ed61734d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.124+0000 7f0ea519a700 1 --2- 172.21.15.119:0/3416548069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ed6152f50 0x555ed61734d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.124+0000 7f0ea519a700 1 -- 172.21.15.119:0/3416548069 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ed5eb4c60 con 0x555ed6153870 2023-12-07T10:22:28.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.124+0000 7f0ea599b700 1 --2- 172.21.15.119:0/3416548069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ed6152f50 0x555ed61734d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:28.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.124+0000 7f0ea619c700 1 --2- 172.21.15.119:0/3416548069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ed6157560 0x555ed6163c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:22:28.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.124+0000 7f0ea519a700 1 --2- 172.21.15.119:0/3416548069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ed6153870 0x555ed6163700 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0e900cb400 tx=0x7f0e900ba040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c2fc46c9752ed13 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:28.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.124+0000 7f0e94bff700 1 -- 172.21.15.119:0/3416548069 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9009eb40 con 0x555ed6153870 2023-12-07T10:22:28.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.124+0000 7f0e94bff700 1 -- 172.21.15.119:0/3416548069 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e900a2d80 con 0x555ed6153870 2023-12-07T10:22:28.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.124+0000 7f0e94bff700 1 -- 172.21.15.119:0/3416548069 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9009a140 con 0x555ed6153870 2023-12-07T10:22:28.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.125+0000 7f0ebb5ffb80 1 -- 172.21.15.119:0/3416548069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ed6153870 msgr2=0x555ed6163700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.125+0000 7f0ebb5ffb80 1 --2- 172.21.15.119:0/3416548069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ed6153870 0x555ed6163700 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0e900cb400 tx=0x7f0e900ba040 comp rx=0 tx=0).stop 2023-12-07T10:22:28.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.125+0000 7f0ebb5ffb80 1 -- 172.21.15.119:0/3416548069 shutdown_connections 2023-12-07T10:22:28.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.125+0000 7f0ebb5ffb80 1 --2- 172.21.15.119:0/3416548069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ed6157560 0x555ed6163c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.125+0000 7f0ebb5ffb80 1 --2- 172.21.15.119:0/3416548069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ed6152f50 0x555ed61734d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.125+0000 7f0ebb5ffb80 1 --2- 172.21.15.119:0/3416548069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ed6153870 0x555ed6163700 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.125+0000 7f0ebb5ffb80 1 -- 172.21.15.119:0/3416548069 >> 172.21.15.119:0/3416548069 conn(0x555ed607d360 msgr2=0x555ed6155ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:28.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.125+0000 7f0ebb5ffb80 1 -- 172.21.15.119:0/3416548069 shutdown_connections 2023-12-07T10:22:28.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.125+0000 7f0ebb5ffb80 1 -- 172.21.15.119:0/3416548069 wait complete. 2023-12-07T10:22:28.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.125+0000 7f0ebb5ffb80 1 -- start start 2023-12-07T10:22:28.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.125+0000 7f0ebb5ffb80 1 Processor -- start 2023-12-07T10:22:28.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.126+0000 7f0ebb5ffb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ed6152f50 0x555ed6163860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.126+0000 7f0ebb5ffb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ed6153870 0x555ed6161d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.126+0000 7f0ebb5ffb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ed6157560 0x555ed6162240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.126+0000 7f0ebb5ffb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ed6162780 con 0x555ed6157560 2023-12-07T10:22:28.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.126+0000 7f0ebb5ffb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ed61628f0 con 0x555ed6152f50 2023-12-07T10:22:28.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.126+0000 7f0ebb5ffb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ed6162a60 con 0x555ed6153870 2023-12-07T10:22:28.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.126+0000 7f0ebb5ffb80 10 client.0 _ll_register_callbacks cb 0x555ed6043940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:22:28.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.126+0000 7f0ea599b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ed6152f50 0x555ed6163860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.126+0000 7f0ea619c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ed6157560 0x555ed6162240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.126+0000 7f0ea599b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ed6152f50 0x555ed6163860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42462/0 (socket says 172.21.15.119:42462) 2023-12-07T10:22:28.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.126+0000 7f0ea599b700 1 -- 172.21.15.119:0/2587175877 learned_addr learned my addr 172.21.15.119:0/2587175877 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:28.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.126+0000 7f0ea519a700 1 --2- 172.21.15.119:0/2587175877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ed6153870 0x555ed6161d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.127+0000 7f0ea619c700 1 -- 172.21.15.119:0/2587175877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ed6153870 msgr2=0x555ed6161d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.127+0000 7f0ea619c700 1 --2- 172.21.15.119:0/2587175877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ed6153870 0x555ed6161d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.127+0000 7f0ea619c700 1 -- 172.21.15.119:0/2587175877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ed6152f50 msgr2=0x555ed6163860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.127+0000 7f0ea619c700 1 --2- 172.21.15.119:0/2587175877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ed6152f50 0x555ed6163860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.127+0000 7f0ea619c700 1 -- 172.21.15.119:0/2587175877 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ed5f1f630 con 0x555ed6157560 2023-12-07T10:22:28.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.127+0000 7f0ea599b700 1 --2- 172.21.15.119:0/2587175877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ed6152f50 0x555ed6163860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:28.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.127+0000 7f0ea619c700 1 --2- 172.21.15.119:0/2587175877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ed6157560 0x555ed6162240 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x555ed615d280 tx=0x7f0e98009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31de57afb124ddae server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:28.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.127+0000 7f0e897fa700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:22:28.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.127+0000 7f0e897fa700 1 -- 172.21.15.119:0/2587175877 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e98025070 con 0x555ed6157560 2023-12-07T10:22:28.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.127+0000 7f0ebb5ffb80 1 -- 172.21.15.119:0/2587175877 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e9c01d040 con 0x555ed6157560 2023-12-07T10:22:28.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.127+0000 7f0e897fa700 1 -- 172.21.15.119:0/2587175877 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f0e98004560 con 0x555ed6157560 2023-12-07T10:22:28.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.127+0000 7f0e897fa700 1 -- 172.21.15.119:0/2587175877 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e98003a00 con 0x555ed6157560 2023-12-07T10:22:28.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.127+0000 7f0ebb5ffb80 10 client.16320 fetch_fsmap learned FSMap version 32 2023-12-07T10:22:28.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.127+0000 7f0ebb5ffb80 1 -- 172.21.15.119:0/2587175877 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f0ea01a4710 con 0x555ed6157560 2023-12-07T10:22:28.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.127+0000 7f0e897fa700 1 -- 172.21.15.119:0/2587175877 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e9c01d040 con 0x555ed6157560 2023-12-07T10:22:28.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.128+0000 7f0e897fa700 1 -- 172.21.15.119:0/2587175877 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f0e98028050 con 0x555ed6157560 2023-12-07T10:22:28.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.128+0000 7f0ebb5ffb80 10 client.16320 fetch_fsmap finished waiting for FSMap version 32 2023-12-07T10:22:28.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.128+0000 7f0ebb5ffb80 10 client.16320 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2023-12-07T10:22:28.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.128+0000 7f0ebb5ffb80 20 client.16320 populate_metadata read hostname 'smithi119' 2023-12-07T10:22:28.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.128+0000 7f0ebb5ffb80 1 --2- 172.21.15.119:0/2587175877 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x555ed61c4200 0x555ed61c6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.128+0000 7f0ebb5ffb80 4 client.16320 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:22:28.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.128+0000 7f0ebb5ffb80 1 -- 172.21.15.119:0/2587175877 --> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555ed6068fd0 con 0x555ed61c4200 2023-12-07T10:22:28.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.128+0000 7f0ea599b700 1 --2- 172.21.15.119:0/2587175877 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x555ed61c4200 0x555ed61c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.129+0000 7f0ea599b700 1 --2- 172.21.15.119:0/2587175877 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x555ed61c4200 0x555ed61c6620 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=62a2fa70e945a370 server_cookie=428888b1196b6102 in_seq=0 out_seq=0 2023-12-07T10:22:28.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.129+0000 7f0e897fa700 10 client.16320 ms_handle_connect on v2:172.21.15.153:6834/1295649464 2023-12-07T10:22:28.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.129+0000 7f0e897fa700 1 -- 172.21.15.119:0/2587175877 <== mds.4 v2:172.21.15.153:6834/1295649464 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555ed6068fd0 con 0x555ed61c4200 2023-12-07T10:22:28.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.129+0000 7f0e897fa700 10 client.16320 handle_command_reply: tid=0 2023-12-07T10:22:28.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.129+0000 7f0ebb5ffb80 1 client.16320 shutdown 2023-12-07T10:22:28.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.129+0000 7f0ebb5ffb80 1 -- 172.21.15.119:0/2587175877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ed6157560 msgr2=0x555ed6162240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.129+0000 7f0ebb5ffb80 1 --2- 172.21.15.119:0/2587175877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ed6157560 0x555ed6162240 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x555ed615d280 tx=0x7f0e98009940 comp rx=0 tx=0).stop 2023-12-07T10:22:28.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.129+0000 7f0ebb5ffb80 1 -- 172.21.15.119:0/2587175877 shutdown_connections 2023-12-07T10:22:28.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.129+0000 7f0ebb5ffb80 1 --2- 172.21.15.119:0/2587175877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555ed6157560 0x555ed6162240 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.129+0000 7f0ebb5ffb80 1 --2- 172.21.15.119:0/2587175877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555ed6152f50 0x555ed6163860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.129+0000 7f0ebb5ffb80 1 --2- 172.21.15.119:0/2587175877 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x555ed61c4200 0x555ed61c6620 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.129+0000 7f0ebb5ffb80 1 --2- 172.21.15.119:0/2587175877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ed6153870 0x555ed6161d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.129+0000 7f0ebb5ffb80 1 -- 172.21.15.119:0/2587175877 >> 172.21.15.119:0/2587175877 conn(0x555ed607d360 msgr2=0x555ed6156620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:28.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.129+0000 7f0e897fa700 0 client.16320 ms_handle_reset on v2:172.21.15.153:6834/1295649464 2023-12-07T10:22:28.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.129+0000 7f0ebb5ffb80 1 -- 172.21.15.119:0/2587175877 shutdown_connections 2023-12-07T10:22:28.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.129+0000 7f0ebb5ffb80 1 -- 172.21.15.119:0/2587175877 wait complete. 2023-12-07T10:22:28.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.130+0000 7f0ebb5ffb80 20 client.16320 trim_cache size 0 max 16384 2023-12-07T10:22:28.141 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:22:28.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.130+0000 7f0ea7bff700 1 -- 172.21.15.119:0/1747366890 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0e8c0a1c80 msgr2=0x7f0e8c0a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.130+0000 7f0ea7bff700 1 --2- 172.21.15.119:0/1747366890 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0e8c0a1c80 0x7f0e8c0a4140 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0ea019e420 tx=0x7f0e98009040 comp rx=0 tx=0).stop 2023-12-07T10:22:28.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.130+0000 7f0ea7bff700 1 -- 172.21.15.119:0/1747366890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea0101f20 msgr2=0x7f0ea019dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.130+0000 7f0ea7bff700 1 --2- 172.21.15.119:0/1747366890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea0101f20 0x7f0ea019dc60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0e9000e9a0 tx=0x7f0e9000ecb0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.130+0000 7f0ea7bff700 1 -- 172.21.15.119:0/1747366890 shutdown_connections 2023-12-07T10:22:28.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.130+0000 7f0ea7bff700 1 --2- 172.21.15.119:0/1747366890 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0e8c0a1c80 0x7f0e8c0a4140 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.130+0000 7f0ea7bff700 1 --2- 172.21.15.119:0/1747366890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ea0107620 0x7f0ea0197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.130+0000 7f0ea7bff700 1 --2- 172.21.15.119:0/1747366890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ea0101560 0x7f0ea019d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.130+0000 7f0ea7bff700 1 --2- 172.21.15.119:0/1747366890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea0101f20 0x7f0ea019dc60 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.130+0000 7f0ea7bff700 1 -- 172.21.15.119:0/1747366890 >> 172.21.15.119:0/1747366890 conn(0x7f0ea00fcfd0 msgr2=0x7f0ea00fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:28.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.130+0000 7f0ea7bff700 1 -- 172.21.15.119:0/1747366890 shutdown_connections 2023-12-07T10:22:28.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.130+0000 7f0ea7bff700 1 -- 172.21.15.119:0/1747366890 wait complete. 2023-12-07T10:22:28.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:27 smithi119 ceph-mon[97908]: pgmap v64: 129 pgs: 129 active+clean; 52 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 3.8 MiB/s wr, 189 op/s 2023-12-07T10:22:28.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:27 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1832563783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:22:28.149 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-12-07T10:22:28.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:27 smithi159 ceph-mon[134726]: pgmap v64: 129 pgs: 129 active+clean; 52 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 3.8 MiB/s wr, 189 op/s 2023-12-07T10:22:28.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:27 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1832563783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:22:28.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:27 smithi153 ceph-mon[133625]: pgmap v64: 129 pgs: 129 active+clean; 52 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 3.8 MiB/s wr, 189 op/s 2023-12-07T10:22:28.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:27 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1832563783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:22:28.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.347+0000 7f4357dff700 1 -- 172.21.15.119:0/4267704569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f435010dd80 msgr2=0x7f43501101f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.347+0000 7f4357dff700 1 --2- 172.21.15.119:0/4267704569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f435010dd80 0x7f43501101f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f434800b0a0 tx=0x7f434800b3b0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.347+0000 7f4357dff700 1 -- 172.21.15.119:0/4267704569 shutdown_connections 2023-12-07T10:22:28.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.347+0000 7f4357dff700 1 --2- 172.21.15.119:0/4267704569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f435010dd80 0x7f43501101f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.347+0000 7f4357dff700 1 --2- 172.21.15.119:0/4267704569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4350073a40 0x7f4350073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.347+0000 7f4357dff700 1 --2- 172.21.15.119:0/4267704569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4350074360 0x7f435010d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.347+0000 7f4357dff700 1 -- 172.21.15.119:0/4267704569 >> 172.21.15.119:0/4267704569 conn(0x7f43500fd010 msgr2=0x7f43500ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:28.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.347+0000 7f4357dff700 1 -- 172.21.15.119:0/4267704569 shutdown_connections 2023-12-07T10:22:28.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.348+0000 7f4357dff700 1 -- 172.21.15.119:0/4267704569 wait complete. 2023-12-07T10:22:28.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.348+0000 7f4357dff700 1 Processor -- start 2023-12-07T10:22:28.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.349+0000 7f4357dff700 1 -- start start 2023-12-07T10:22:28.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.349+0000 7f4357dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4350073a40 0x7f4350199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.349+0000 7f4357dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4350074360 0x7f435019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.349+0000 7f4357dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f435010dd80 0x7f435019d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.349+0000 7f4357dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435019db40 con 0x7f435010dd80 2023-12-07T10:22:28.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.349+0000 7f4357dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435019dcb0 con 0x7f4350074360 2023-12-07T10:22:28.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.349+0000 7f4357dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435019de70 con 0x7f4350073a40 2023-12-07T10:22:28.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.349+0000 7f435639c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f435010dd80 0x7f435019d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.349+0000 7f435639c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f435010dd80 0x7f435019d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44918/0 (socket says 172.21.15.119:44918) 2023-12-07T10:22:28.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.349+0000 7f435639c700 1 -- 172.21.15.119:0/4015878453 learned_addr learned my addr 172.21.15.119:0/4015878453 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:28.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.349+0000 7f4355b9b700 1 --2- 172.21.15.119:0/4015878453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4350073a40 0x7f4350199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.349+0000 7f435539a700 1 --2- 172.21.15.119:0/4015878453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4350074360 0x7f435019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.350+0000 7f435639c700 1 -- 172.21.15.119:0/4015878453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4350073a40 msgr2=0x7f4350199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.350+0000 7f435639c700 1 --2- 172.21.15.119:0/4015878453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4350073a40 0x7f4350199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.350+0000 7f435639c700 1 -- 172.21.15.119:0/4015878453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4350074360 msgr2=0x7f435019a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.350+0000 7f435639c700 1 --2- 172.21.15.119:0/4015878453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4350074360 0x7f435019a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.350+0000 7f435639c700 1 -- 172.21.15.119:0/4015878453 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4348009d00 con 0x7f435010dd80 2023-12-07T10:22:28.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.350+0000 7f435639c700 1 --2- 172.21.15.119:0/4015878453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f435010dd80 0x7f435019d540 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f434800b070 tx=0x7f43480078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:28.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.350+0000 7f4346ffd700 1 -- 172.21.15.119:0/4015878453 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434800e040 con 0x7f435010dd80 2023-12-07T10:22:28.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.350+0000 7f4346ffd700 1 -- 172.21.15.119:0/4015878453 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f434801d470 con 0x7f435010dd80 2023-12-07T10:22:28.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.350+0000 7f4346ffd700 1 -- 172.21.15.119:0/4015878453 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4348005280 con 0x7f435010dd80 2023-12-07T10:22:28.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.350+0000 7f4357dff700 1 -- 172.21.15.119:0/4015878453 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f435019e0f0 con 0x7f435010dd80 2023-12-07T10:22:28.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.350+0000 7f4357dff700 1 -- 172.21.15.119:0/4015878453 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43501a6ab0 con 0x7f435010dd80 2023-12-07T10:22:28.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.353+0000 7f4346ffd700 1 -- 172.21.15.119:0/4015878453 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f434801d5e0 con 0x7f435010dd80 2023-12-07T10:22:28.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.355+0000 7f4346ffd700 1 --2- 172.21.15.119:0/4015878453 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f433c0a1d00 0x7f433c0a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.355+0000 7f436b629b80 1 Processor -- start 2023-12-07T10:22:28.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.355+0000 7f4346ffd700 1 -- 172.21.15.119:0/4015878453 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f43480ced00 con 0x7f435010dd80 2023-12-07T10:22:28.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.355+0000 7f4355b9b700 1 --2- 172.21.15.119:0/4015878453 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f433c0a1d00 0x7f433c0a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.355+0000 7f436b629b80 1 -- start start 2023-12-07T10:22:28.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.355+0000 7f436b629b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56427a9b5b40 0x56427a9b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.355+0000 7f436b629b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56427a9b6460 0x56427a9b9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.355+0000 7f436b629b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9ba150 0x56427a9ba570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.355+0000 7f436b629b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56427a9297c0 con 0x56427a9ba150 2023-12-07T10:22:28.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.355+0000 7f436b629b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56427a9107a0 con 0x56427a9b6460 2023-12-07T10:22:28.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.355+0000 7f436b629b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56427a90e0c0 con 0x56427a9b5b40 2023-12-07T10:22:28.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.356+0000 7f4355b9b700 1 --2- 172.21.15.119:0/4015878453 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f433c0a1d00 0x7f433c0a41c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f434c009c60 tx=0x7f434c009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:28.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.356+0000 7f435639c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9ba150 0x56427a9ba570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.356+0000 7f435639c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9ba150 0x56427a9ba570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44924/0 (socket says 172.21.15.119:44924) 2023-12-07T10:22:28.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.356+0000 7f435639c700 1 -- 172.21.15.119:0/1660260238 learned_addr learned my addr 172.21.15.119:0/1660260238 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:28.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.356+0000 7f4355b9b700 1 --2- 172.21.15.119:0/1660260238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56427a9b5b40 0x56427a9b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.356+0000 7f435539a700 1 --2- 172.21.15.119:0/1660260238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56427a9b6460 0x56427a9b9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.356+0000 7f435639c700 1 -- 172.21.15.119:0/1660260238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56427a9b5b40 msgr2=0x56427a9b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.356+0000 7f435639c700 1 --2- 172.21.15.119:0/1660260238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56427a9b5b40 0x56427a9b5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.356+0000 7f435639c700 1 -- 172.21.15.119:0/1660260238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56427a9b6460 msgr2=0x56427a9b9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.356+0000 7f435639c700 1 --2- 172.21.15.119:0/1660260238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56427a9b6460 0x56427a9b9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.356+0000 7f435639c700 1 -- 172.21.15.119:0/1660260238 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56427a786630 con 0x56427a9ba150 2023-12-07T10:22:28.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.356+0000 7f4355b9b700 1 --2- 172.21.15.119:0/1660260238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56427a9b5b40 0x56427a9b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:22:28.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.356+0000 7f435639c700 1 --2- 172.21.15.119:0/1660260238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9ba150 0x56427a9ba570 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f43480a3550 tx=0x7f434809b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b2ae58f6df9e36 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:28.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.356+0000 7f433bfff700 1 -- 172.21.15.119:0/1660260238 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434809b0e0 con 0x56427a9ba150 2023-12-07T10:22:28.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.357+0000 7f433bfff700 1 -- 172.21.15.119:0/1660260238 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f434809be80 con 0x56427a9ba150 2023-12-07T10:22:28.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.357+0000 7f433bfff700 1 -- 172.21.15.119:0/1660260238 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4348098020 con 0x56427a9ba150 2023-12-07T10:22:28.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.357+0000 7f436b629b80 1 -- 172.21.15.119:0/1660260238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9ba150 msgr2=0x56427a9ba570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.357+0000 7f436b629b80 1 --2- 172.21.15.119:0/1660260238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9ba150 0x56427a9ba570 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f43480a3550 tx=0x7f434809b5d0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.357+0000 7f436b629b80 1 -- 172.21.15.119:0/1660260238 shutdown_connections 2023-12-07T10:22:28.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.357+0000 7f436b629b80 1 --2- 172.21.15.119:0/1660260238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9ba150 0x56427a9ba570 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.357+0000 7f436b629b80 1 --2- 172.21.15.119:0/1660260238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56427a9b5b40 0x56427a9b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.357+0000 7f436b629b80 1 --2- 172.21.15.119:0/1660260238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56427a9b6460 0x56427a9b9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.357+0000 7f436b629b80 1 -- 172.21.15.119:0/1660260238 >> 172.21.15.119:0/1660260238 conn(0x56427a8e4360 msgr2=0x56427a9b0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:28.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.357+0000 7f436b629b80 1 -- 172.21.15.119:0/1660260238 shutdown_connections 2023-12-07T10:22:28.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.357+0000 7f436b629b80 1 -- 172.21.15.119:0/1660260238 wait complete. 2023-12-07T10:22:28.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.358+0000 7f436b629b80 1 -- start start 2023-12-07T10:22:28.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.358+0000 7f436b629b80 1 Processor -- start 2023-12-07T10:22:28.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.359+0000 7f436b629b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56427a9b5b40 0x56427a9c18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.359+0000 7f436b629b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9b6460 0x56427a9bebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.359+0000 7f436b629b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56427a9ba150 0x56427a9bf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.359+0000 7f436b629b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56427a9099b0 con 0x56427a9b6460 2023-12-07T10:22:28.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.359+0000 7f436b629b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56427a8c3560 con 0x56427a9b5b40 2023-12-07T10:22:28.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.359+0000 7f436b629b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56427a8c3200 con 0x56427a9ba150 2023-12-07T10:22:28.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.359+0000 7f436b629b80 10 client.0 _ll_register_callbacks cb 0x56427a8aa940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:22:28.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.359+0000 7f4355b9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56427a9b5b40 0x56427a9c18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.359+0000 7f435539a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9b6460 0x56427a9bebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.359+0000 7f4355b9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56427a9b5b40 0x56427a9c18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42496/0 (socket says 172.21.15.119:42496) 2023-12-07T10:22:28.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.359+0000 7f4355b9b700 1 -- 172.21.15.119:0/3709805313 learned_addr learned my addr 172.21.15.119:0/3709805313 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:28.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.359+0000 7f435639c700 1 --2- 172.21.15.119:0/3709805313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56427a9ba150 0x56427a9bf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.360+0000 7f435539a700 1 -- 172.21.15.119:0/3709805313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56427a9ba150 msgr2=0x56427a9bf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.360+0000 7f435539a700 1 --2- 172.21.15.119:0/3709805313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56427a9ba150 0x56427a9bf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.360+0000 7f435539a700 1 -- 172.21.15.119:0/3709805313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56427a9b5b40 msgr2=0x56427a9c18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.360+0000 7f435539a700 1 --2- 172.21.15.119:0/3709805313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56427a9b5b40 0x56427a9c18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.360+0000 7f435539a700 1 -- 172.21.15.119:0/3709805313 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43501a6ab0 con 0x56427a9b6460 2023-12-07T10:22:28.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.360+0000 7f435539a700 1 --2- 172.21.15.119:0/3709805313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9b6460 0x56427a9bebc0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f435019b2c0 tx=0x7f4340012960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c2085a765d7106 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:28.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.360+0000 7f43397fa700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:22:28.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.360+0000 7f43397fa700 1 -- 172.21.15.119:0/3709805313 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4340017070 con 0x56427a9b6460 2023-12-07T10:22:28.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.360+0000 7f436b629b80 1 -- 172.21.15.119:0/3709805313 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56427a9a8b00 con 0x56427a9b6460 2023-12-07T10:22:28.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.360+0000 7f43397fa700 1 -- 172.21.15.119:0/3709805313 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4340010430 con 0x56427a9b6460 2023-12-07T10:22:28.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.361+0000 7f43397fa700 1 -- 172.21.15.119:0/3709805313 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4340021600 con 0x56427a9b6460 2023-12-07T10:22:28.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.361+0000 7f436b629b80 10 client.16332 fetch_fsmap learned FSMap version 32 2023-12-07T10:22:28.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.361+0000 7f436b629b80 1 -- 172.21.15.119:0/3709805313 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56427a732de0 con 0x56427a9b6460 2023-12-07T10:22:28.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.361+0000 7f43397fa700 1 -- 172.21.15.119:0/3709805313 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56427a9a8b00 con 0x56427a9b6460 2023-12-07T10:22:28.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.361+0000 7f43397fa700 1 -- 172.21.15.119:0/3709805313 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f4340010680 con 0x56427a9b6460 2023-12-07T10:22:28.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.361+0000 7f436b629b80 10 client.16332 fetch_fsmap finished waiting for FSMap version 32 2023-12-07T10:22:28.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.361+0000 7f436b629b80 10 client.16332 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-12-07T10:22:28.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.361+0000 7f436b629b80 20 client.16332 populate_metadata read hostname 'smithi119' 2023-12-07T10:22:28.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.361+0000 7f436b629b80 1 --2- 172.21.15.119:0/3709805313 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x56427a9cef30 0x56427a9d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.361+0000 7f436b629b80 4 client.16332 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:22:28.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.361+0000 7f436b629b80 1 -- 172.21.15.119:0/3709805313 --> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56427a9847a0 con 0x56427a9cef30 2023-12-07T10:22:28.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.361+0000 7f4355b9b700 1 --2- 172.21.15.119:0/3709805313 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x56427a9cef30 0x56427a9d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.362+0000 7f4355b9b700 1 --2- 172.21.15.119:0/3709805313 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x56427a9cef30 0x56427a9d1350 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=55993c88a846b75b server_cookie=f409ea981ded6465 in_seq=0 out_seq=0 2023-12-07T10:22:28.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.362+0000 7f43397fa700 10 client.16332 ms_handle_connect on v2:172.21.15.159:6834/2406380792 2023-12-07T10:22:28.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.363+0000 7f43397fa700 1 -- 172.21.15.119:0/3709805313 <== mds.3 v2:172.21.15.159:6834/2406380792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56427a9847a0 con 0x56427a9cef30 2023-12-07T10:22:28.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.363+0000 7f43397fa700 10 client.16332 handle_command_reply: tid=0 2023-12-07T10:22:28.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.363+0000 7f436b629b80 1 client.16332 shutdown 2023-12-07T10:22:28.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.363+0000 7f436b629b80 1 -- 172.21.15.119:0/3709805313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9b6460 msgr2=0x56427a9bebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.363+0000 7f436b629b80 1 --2- 172.21.15.119:0/3709805313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9b6460 0x56427a9bebc0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f435019b2c0 tx=0x7f4340012960 comp rx=0 tx=0).stop 2023-12-07T10:22:28.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.363+0000 7f436b629b80 1 -- 172.21.15.119:0/3709805313 shutdown_connections 2023-12-07T10:22:28.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.363+0000 7f436b629b80 1 --2- 172.21.15.119:0/3709805313 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x56427a9cef30 0x56427a9d1350 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.363+0000 7f436b629b80 1 --2- 172.21.15.119:0/3709805313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56427a9ba150 0x56427a9bf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.363+0000 7f436b629b80 1 --2- 172.21.15.119:0/3709805313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56427a9b5b40 0x56427a9c18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.363+0000 7f436b629b80 1 --2- 172.21.15.119:0/3709805313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9b6460 0x56427a9bebc0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.363+0000 7f436b629b80 1 -- 172.21.15.119:0/3709805313 >> 172.21.15.119:0/3709805313 conn(0x56427a8e4360 msgr2=0x56427a9bb620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:28.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.363+0000 7f43397fa700 0 client.16332 ms_handle_reset on v2:172.21.15.159:6834/2406380792 2023-12-07T10:22:28.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.363+0000 7f436b629b80 1 -- 172.21.15.119:0/3709805313 shutdown_connections 2023-12-07T10:22:28.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.363+0000 7f436b629b80 1 -- 172.21.15.119:0/3709805313 wait complete. 2023-12-07T10:22:28.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.364+0000 7f436b629b80 20 client.16332 trim_cache size 0 max 16384 2023-12-07T10:22:28.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.377+0000 7f436b629b80 1 Processor -- start 2023-12-07T10:22:28.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.377+0000 7f436b629b80 1 -- start start 2023-12-07T10:22:28.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.377+0000 7f436b629b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56427a9b5b40 0x56427a9d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.378+0000 7f436b629b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9b6460 0x56427a9caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.378+0000 7f436b629b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56427a9ba150 0x56427a9cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.378+0000 7f436b629b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56427a9baa50 con 0x56427a9b6460 2023-12-07T10:22:28.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.378+0000 7f436b629b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56427a788920 con 0x56427a9ba150 2023-12-07T10:22:28.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.378+0000 7f436b629b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56427a9b1300 con 0x56427a9b5b40 2023-12-07T10:22:28.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.378+0000 7f435539a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9b6460 0x56427a9caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.378+0000 7f435539a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9b6460 0x56427a9caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44940/0 (socket says 172.21.15.119:44940) 2023-12-07T10:22:28.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.378+0000 7f435539a700 1 -- 172.21.15.119:0/2608391809 learned_addr learned my addr 172.21.15.119:0/2608391809 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:28.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.378+0000 7f4355b9b700 1 --2- 172.21.15.119:0/2608391809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56427a9b5b40 0x56427a9d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.378+0000 7f435639c700 1 --2- 172.21.15.119:0/2608391809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56427a9ba150 0x56427a9cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.378+0000 7f435539a700 1 -- 172.21.15.119:0/2608391809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56427a9b5b40 msgr2=0x56427a9d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.378+0000 7f435539a700 1 --2- 172.21.15.119:0/2608391809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56427a9b5b40 0x56427a9d5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.378+0000 7f435539a700 1 -- 172.21.15.119:0/2608391809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56427a9ba150 msgr2=0x56427a9cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.378+0000 7f435539a700 1 --2- 172.21.15.119:0/2608391809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56427a9ba150 0x56427a9cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.378+0000 7f435539a700 1 -- 172.21.15.119:0/2608391809 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56427a71bc60 con 0x56427a9b6460 2023-12-07T10:22:28.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.378+0000 7f435539a700 1 --2- 172.21.15.119:0/2608391809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9b6460 0x56427a9caa60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4340009340 tx=0x7f4340012d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e32ffccd3c655 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:28.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.379+0000 7f4344bff700 1 -- 172.21.15.119:0/2608391809 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4340017040 con 0x56427a9b6460 2023-12-07T10:22:28.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.379+0000 7f4344bff700 1 -- 172.21.15.119:0/2608391809 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4340005040 con 0x56427a9b6460 2023-12-07T10:22:28.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.379+0000 7f4344bff700 1 -- 172.21.15.119:0/2608391809 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43400038c0 con 0x56427a9b6460 2023-12-07T10:22:28.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.379+0000 7f436b629b80 1 -- 172.21.15.119:0/2608391809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9b6460 msgr2=0x56427a9caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.379+0000 7f436b629b80 1 --2- 172.21.15.119:0/2608391809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9b6460 0x56427a9caa60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4340009340 tx=0x7f4340012d30 comp rx=0 tx=0).stop 2023-12-07T10:22:28.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.379+0000 7f436b629b80 1 -- 172.21.15.119:0/2608391809 shutdown_connections 2023-12-07T10:22:28.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.379+0000 7f436b629b80 1 --2- 172.21.15.119:0/2608391809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56427a9ba150 0x56427a9cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.379+0000 7f436b629b80 1 --2- 172.21.15.119:0/2608391809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56427a9b5b40 0x56427a9d5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.379+0000 7f436b629b80 1 --2- 172.21.15.119:0/2608391809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9b6460 0x56427a9caa60 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.379+0000 7f436b629b80 1 -- 172.21.15.119:0/2608391809 >> 172.21.15.119:0/2608391809 conn(0x56427a8e4360 msgr2=0x56427a9c1590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:28.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.379+0000 7f436b629b80 1 -- 172.21.15.119:0/2608391809 shutdown_connections 2023-12-07T10:22:28.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.379+0000 7f436b629b80 1 -- 172.21.15.119:0/2608391809 wait complete. 2023-12-07T10:22:28.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.380+0000 7f436b629b80 1 -- start start 2023-12-07T10:22:28.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.380+0000 7f436b629b80 1 Processor -- start 2023-12-07T10:22:28.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.381+0000 7f436b629b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9b5b40 0x56427a9c4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.381+0000 7f436b629b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56427a9b6460 0x56427a9c4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.381+0000 7f436b629b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56427a9ba150 0x56427aa24430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.381+0000 7f436b629b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56427aa24970 con 0x56427a9b5b40 2023-12-07T10:22:28.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.381+0000 7f436b629b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56427aa24ab0 con 0x56427a9b6460 2023-12-07T10:22:28.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.381+0000 7f436b629b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56427aa24c70 con 0x56427a9ba150 2023-12-07T10:22:28.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.381+0000 7f436b629b80 10 client.0 _ll_register_callbacks cb 0x56427a8aa940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:22:28.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.381+0000 7f4355b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9b5b40 0x56427a9c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.381+0000 7f4355b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9b5b40 0x56427a9c4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44954/0 (socket says 172.21.15.119:44954) 2023-12-07T10:22:28.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.381+0000 7f4355b9b700 1 -- 172.21.15.119:0/1490949266 learned_addr learned my addr 172.21.15.119:0/1490949266 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:28.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.381+0000 7f435539a700 1 --2- 172.21.15.119:0/1490949266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56427a9b6460 0x56427a9c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.381+0000 7f435639c700 1 --2- 172.21.15.119:0/1490949266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56427a9ba150 0x56427aa24430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.382+0000 7f4355b9b700 1 -- 172.21.15.119:0/1490949266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56427a9ba150 msgr2=0x56427aa24430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.382+0000 7f4355b9b700 1 --2- 172.21.15.119:0/1490949266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56427a9ba150 0x56427aa24430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.382+0000 7f4355b9b700 1 -- 172.21.15.119:0/1490949266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56427a9b6460 msgr2=0x56427a9c4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.382+0000 7f4355b9b700 1 --2- 172.21.15.119:0/1490949266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56427a9b6460 0x56427a9c4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.382+0000 7f4355b9b700 1 -- 172.21.15.119:0/1490949266 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56427a732de0 con 0x56427a9b5b40 2023-12-07T10:22:28.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.382+0000 7f435539a700 1 --2- 172.21.15.119:0/1490949266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56427a9b6460 0x56427a9c4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:28.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.382+0000 7f4355b9b700 1 --2- 172.21.15.119:0/1490949266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9b5b40 0x56427a9c4240 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f434c013010 tx=0x7f434c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a628f3c5fc9780dd server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:28.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.382+0000 7f43397fa700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:22:28.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.382+0000 7f43397fa700 1 -- 172.21.15.119:0/1490949266 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c023040 con 0x56427a9b5b40 2023-12-07T10:22:28.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.382+0000 7f43397fa700 1 -- 172.21.15.119:0/1490949266 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f434c004a30 con 0x56427a9b5b40 2023-12-07T10:22:28.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.382+0000 7f436b629b80 1 -- 172.21.15.119:0/1490949266 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f434809be80 con 0x56427a9b5b40 2023-12-07T10:22:28.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.382+0000 7f43397fa700 1 -- 172.21.15.119:0/1490949266 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f434c004bf0 con 0x56427a9b5b40 2023-12-07T10:22:28.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.382+0000 7f436b629b80 10 client.16338 fetch_fsmap learned FSMap version 32 2023-12-07T10:22:28.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.382+0000 7f436b629b80 1 -- 172.21.15.119:0/1490949266 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56427a759250 con 0x56427a9b5b40 2023-12-07T10:22:28.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.383+0000 7f43397fa700 1 -- 172.21.15.119:0/1490949266 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f434809be80 con 0x56427a9b5b40 2023-12-07T10:22:28.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.383+0000 7f43397fa700 1 -- 172.21.15.119:0/1490949266 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f434c0316d0 con 0x56427a9b5b40 2023-12-07T10:22:28.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.383+0000 7f436b629b80 10 client.16338 fetch_fsmap finished waiting for FSMap version 32 2023-12-07T10:22:28.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.383+0000 7f436b629b80 10 client.16338 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-12-07T10:22:28.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.383+0000 7f436b629b80 20 client.16338 populate_metadata read hostname 'smithi119' 2023-12-07T10:22:28.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.383+0000 7f436b629b80 1 --2- 172.21.15.119:0/1490949266 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x56427aa264c0 0x56427aa288e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.383+0000 7f436b629b80 4 client.16338 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:22:28.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.383+0000 7f436b629b80 1 -- 172.21.15.119:0/1490949266 --> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56427a8cffd0 con 0x56427aa264c0 2023-12-07T10:22:28.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.383+0000 7f435539a700 1 --2- 172.21.15.119:0/1490949266 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x56427aa264c0 0x56427aa288e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.384+0000 7f435539a700 1 --2- 172.21.15.119:0/1490949266 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x56427aa264c0 0x56427aa288e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=42446aeb2a7e7380 server_cookie=4505d12eedbad1fc in_seq=0 out_seq=0 2023-12-07T10:22:28.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.384+0000 7f43397fa700 10 client.16338 ms_handle_connect on v2:172.21.15.159:6834/2406380792 2023-12-07T10:22:28.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.384+0000 7f43397fa700 1 -- 172.21.15.119:0/1490949266 <== mds.3 v2:172.21.15.159:6834/2406380792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56427a8cffd0 con 0x56427aa264c0 2023-12-07T10:22:28.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.384+0000 7f43397fa700 10 client.16338 handle_command_reply: tid=0 2023-12-07T10:22:28.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.384+0000 7f436b629b80 1 client.16338 shutdown 2023-12-07T10:22:28.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.384+0000 7f436b629b80 1 -- 172.21.15.119:0/1490949266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9b5b40 msgr2=0x56427a9c4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.384+0000 7f436b629b80 1 --2- 172.21.15.119:0/1490949266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9b5b40 0x56427a9c4240 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f434c013010 tx=0x7f434c0047c0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.384+0000 7f436b629b80 1 -- 172.21.15.119:0/1490949266 shutdown_connections 2023-12-07T10:22:28.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.384+0000 7f436b629b80 1 --2- 172.21.15.119:0/1490949266 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x56427aa264c0 0x56427aa288e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.384+0000 7f436b629b80 1 --2- 172.21.15.119:0/1490949266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56427a9ba150 0x56427aa24430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.384+0000 7f436b629b80 1 --2- 172.21.15.119:0/1490949266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56427a9b5b40 0x56427a9c4240 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.384+0000 7f43397fa700 0 client.16338 ms_handle_reset on v2:172.21.15.159:6834/2406380792 2023-12-07T10:22:28.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.384+0000 7f436b629b80 1 --2- 172.21.15.119:0/1490949266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56427a9b6460 0x56427a9c4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.384+0000 7f436b629b80 1 -- 172.21.15.119:0/1490949266 >> 172.21.15.119:0/1490949266 conn(0x56427a8e4360 msgr2=0x56427a9d5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:28.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.384+0000 7f436b629b80 1 -- 172.21.15.119:0/1490949266 shutdown_connections 2023-12-07T10:22:28.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.384+0000 7f436b629b80 1 -- 172.21.15.119:0/1490949266 wait complete. 2023-12-07T10:22:28.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.384+0000 7f436b629b80 20 client.16338 trim_cache size 0 max 16384 2023-12-07T10:22:28.390 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:22:28.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.385+0000 7f4357dff700 1 -- 172.21.15.119:0/4015878453 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f433c0a1d00 msgr2=0x7f433c0a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.385+0000 7f4357dff700 1 --2- 172.21.15.119:0/4015878453 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f433c0a1d00 0x7f433c0a41c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f434c009c60 tx=0x7f434c009260 comp rx=0 tx=0).stop 2023-12-07T10:22:28.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.385+0000 7f4357dff700 1 -- 172.21.15.119:0/4015878453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f435010dd80 msgr2=0x7f435019d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.385+0000 7f4357dff700 1 --2- 172.21.15.119:0/4015878453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f435010dd80 0x7f435019d540 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f434800b070 tx=0x7f43480078c0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.385+0000 7f4357dff700 1 -- 172.21.15.119:0/4015878453 shutdown_connections 2023-12-07T10:22:28.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.385+0000 7f4357dff700 1 --2- 172.21.15.119:0/4015878453 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f433c0a1d00 0x7f433c0a41c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.385+0000 7f4357dff700 1 --2- 172.21.15.119:0/4015878453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f435010dd80 0x7f435019d540 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.385+0000 7f4357dff700 1 --2- 172.21.15.119:0/4015878453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4350073a40 0x7f4350199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.385+0000 7f4357dff700 1 --2- 172.21.15.119:0/4015878453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4350074360 0x7f435019a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.385+0000 7f4357dff700 1 -- 172.21.15.119:0/4015878453 >> 172.21.15.119:0/4015878453 conn(0x7f43500fd010 msgr2=0x7f43500fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:28.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.385+0000 7f4357dff700 1 -- 172.21.15.119:0/4015878453 shutdown_connections 2023-12-07T10:22:28.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.385+0000 7f4357dff700 1 -- 172.21.15.119:0/4015878453 wait complete. 2023-12-07T10:22:28.404 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-07T10:22:28.545 INFO:tasks.workunit.client.0.smithi119.stdout: 1 67793 9.37 MB/sec warmup 41 sec latency 1499.267 ms 2023-12-07T10:22:28.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.609+0000 7fa0805ff700 1 -- 172.21.15.119:0/1610822608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa078107620 msgr2=0x7fa078107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.609+0000 7fa0805ff700 1 --2- 172.21.15.119:0/1610822608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa078107620 0x7fa078107a00 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa0740099c0 tx=0x7fa074009cd0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.609+0000 7fa0805ff700 1 -- 172.21.15.119:0/1610822608 shutdown_connections 2023-12-07T10:22:28.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.609+0000 7fa0805ff700 1 --2- 172.21.15.119:0/1610822608 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa078101f20 0x7fa07810e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.609+0000 7fa0805ff700 1 --2- 172.21.15.119:0/1610822608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa078107620 0x7fa078107a00 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.609+0000 7fa0805ff700 1 --2- 172.21.15.119:0/1610822608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa078101560 0x7fa0781019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.609+0000 7fa0805ff700 1 -- 172.21.15.119:0/1610822608 >> 172.21.15.119:0/1610822608 conn(0x7fa0780fcfd0 msgr2=0x7fa0780ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:28.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.609+0000 7fa0805ff700 1 -- 172.21.15.119:0/1610822608 shutdown_connections 2023-12-07T10:22:28.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.609+0000 7fa0805ff700 1 -- 172.21.15.119:0/1610822608 wait complete. 2023-12-07T10:22:28.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.610+0000 7fa0805ff700 1 Processor -- start 2023-12-07T10:22:28.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.610+0000 7fa0805ff700 1 -- start start 2023-12-07T10:22:28.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.611+0000 7fa0805ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa078101560 0x7fa07819f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.611+0000 7fa0805ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa078101f20 0x7fa07819fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.611+0000 7fa0805ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa078107620 0x7fa078199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.611+0000 7fa0805ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07819a0d0 con 0x7fa078101560 2023-12-07T10:22:28.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.611+0000 7fa0805ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07819a240 con 0x7fa078101f20 2023-12-07T10:22:28.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.611+0000 7fa0805ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07819a400 con 0x7fa078107620 2023-12-07T10:22:28.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.611+0000 7fa07eb9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa078107620 0x7fa078199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.611+0000 7fa07db9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa078101f20 0x7fa07819fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.611+0000 7fa07e39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa078101560 0x7fa07819f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.611+0000 7fa07eb9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa078107620 0x7fa078199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:46586/0 (socket says 172.21.15.119:46586) 2023-12-07T10:22:28.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.611+0000 7fa07eb9c700 1 -- 172.21.15.119:0/487315329 learned_addr learned my addr 172.21.15.119:0/487315329 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:28.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.611+0000 7fa07eb9c700 1 -- 172.21.15.119:0/487315329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa078101f20 msgr2=0x7fa07819fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.611+0000 7fa07eb9c700 1 --2- 172.21.15.119:0/487315329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa078101f20 0x7fa07819fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.611+0000 7fa07eb9c700 1 -- 172.21.15.119:0/487315329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa078101560 msgr2=0x7fa07819f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.611+0000 7fa07eb9c700 1 --2- 172.21.15.119:0/487315329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa078101560 0x7fa07819f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.611+0000 7fa07eb9c700 1 -- 172.21.15.119:0/487315329 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa070009c40 con 0x7fa078107620 2023-12-07T10:22:28.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.611+0000 7fa07e39b700 1 --2- 172.21.15.119:0/487315329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa078101560 0x7fa07819f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:28.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.611+0000 7fa07db9a700 1 --2- 172.21.15.119:0/487315329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa078101f20 0x7fa07819fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:28.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.612+0000 7fa07eb9c700 1 --2- 172.21.15.119:0/487315329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa078107620 0x7fa078199a70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa07000c370 tx=0x7fa07000c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:28.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.612+0000 7fa06f7fe700 1 -- 172.21.15.119:0/487315329 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa070017070 con 0x7fa078107620 2023-12-07T10:22:28.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.612+0000 7fa06f7fe700 1 -- 172.21.15.119:0/487315329 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa07000f040 con 0x7fa078107620 2023-12-07T10:22:28.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.612+0000 7fa0805ff700 1 -- 172.21.15.119:0/487315329 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0740095f0 con 0x7fa078107620 2023-12-07T10:22:28.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.612+0000 7fa06f7fe700 1 -- 172.21.15.119:0/487315329 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07000e050 con 0x7fa078107620 2023-12-07T10:22:28.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.612+0000 7fa0805ff700 1 -- 172.21.15.119:0/487315329 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0781a6750 con 0x7fa078107620 2023-12-07T10:22:28.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.615+0000 7fa06f7fe700 1 -- 172.21.15.119:0/487315329 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fa07000cb60 con 0x7fa078107620 2023-12-07T10:22:28.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.617+0000 7fa093eeeb80 1 Processor -- start 2023-12-07T10:22:28.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.617+0000 7fa093eeeb80 1 -- start start 2023-12-07T10:22:28.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.618+0000 7fa093eeeb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564091a0ab40 0x564091a0af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.618+0000 7fa06f7fe700 1 --2- 172.21.15.119:0/487315329 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa0640a1d00 0x7fa0640a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.618+0000 7fa093eeeb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564091a0b460 0x564091a0ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.618+0000 7fa093eeeb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564091a0f150 0x564091a0f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.618+0000 7fa093eeeb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56409197e7c0 con 0x564091a0ab40 2023-12-07T10:22:28.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.618+0000 7fa093eeeb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640919657a0 con 0x564091a0f150 2023-12-07T10:22:28.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.618+0000 7fa093eeeb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640919630c0 con 0x564091a0b460 2023-12-07T10:22:28.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.618+0000 7fa06f7fe700 1 -- 172.21.15.119:0/487315329 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fa0700cd620 con 0x7fa078107620 2023-12-07T10:22:28.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.618+0000 7fa07e39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564091a0ab40 0x564091a0af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.618+0000 7fa07e39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564091a0ab40 0x564091a0af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44978/0 (socket says 172.21.15.119:44978) 2023-12-07T10:22:28.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.618+0000 7fa07db9a700 1 --2- 172.21.15.119:0/487315329 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa0640a1d00 0x7fa0640a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.618+0000 7fa07e39b700 1 -- 172.21.15.119:0/135433583 learned_addr learned my addr 172.21.15.119:0/135433583 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:28.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.618+0000 7fa07eb9c700 1 --2- 172.21.15.119:0/135433583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564091a0f150 0x564091a0f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.618+0000 7fa07db9a700 1 --2- 172.21.15.119:0/135433583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564091a0b460 0x564091a0ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.619+0000 7fa07eb9c700 1 -- 172.21.15.119:0/135433583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564091a0b460 msgr2=0x564091a0ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.619+0000 7fa07eb9c700 1 --2- 172.21.15.119:0/135433583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564091a0b460 0x564091a0ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.619+0000 7fa07eb9c700 1 -- 172.21.15.119:0/135433583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564091a0ab40 msgr2=0x564091a0af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.619+0000 7fa07eb9c700 1 --2- 172.21.15.119:0/135433583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564091a0ab40 0x564091a0af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.619+0000 7fa07eb9c700 1 -- 172.21.15.119:0/135433583 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640917db630 con 0x564091a0f150 2023-12-07T10:22:28.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.619+0000 7fa07e39b700 1 --2- 172.21.15.119:0/135433583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564091a0ab40 0x564091a0af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:28.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.619+0000 7fa07db9a700 1 --2- 172.21.15.119:0/487315329 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa0640a1d00 0x7fa0640a41c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa07819ac30 tx=0x7fa068013c90 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:28.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.619+0000 7fa07eb9c700 1 --2- 172.21.15.119:0/135433583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564091a0f150 0x564091a0f570 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa0700ca9c0 tx=0x7fa070099b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58cf42f5bb2e9ece server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:28.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.619+0000 7fa06cbfe700 1 -- 172.21.15.119:0/135433583 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07001cad0 con 0x564091a0f150 2023-12-07T10:22:28.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.619+0000 7fa06cbfe700 1 -- 172.21.15.119:0/135433583 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0700092e0 con 0x564091a0f150 2023-12-07T10:22:28.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.619+0000 7fa06cbfe700 1 -- 172.21.15.119:0/135433583 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa070098220 con 0x564091a0f150 2023-12-07T10:22:28.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.620+0000 7fa093eeeb80 1 -- 172.21.15.119:0/135433583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564091a0f150 msgr2=0x564091a0f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.620+0000 7fa093eeeb80 1 --2- 172.21.15.119:0/135433583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564091a0f150 0x564091a0f570 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa0700ca9c0 tx=0x7fa070099b50 comp rx=0 tx=0).stop 2023-12-07T10:22:28.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.620+0000 7fa093eeeb80 1 -- 172.21.15.119:0/135433583 shutdown_connections 2023-12-07T10:22:28.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.620+0000 7fa093eeeb80 1 --2- 172.21.15.119:0/135433583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564091a0f150 0x564091a0f570 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.620+0000 7fa093eeeb80 1 --2- 172.21.15.119:0/135433583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564091a0ab40 0x564091a0af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.620+0000 7fa093eeeb80 1 --2- 172.21.15.119:0/135433583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564091a0b460 0x564091a0ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.620+0000 7fa093eeeb80 1 -- 172.21.15.119:0/135433583 >> 172.21.15.119:0/135433583 conn(0x564091939360 msgr2=0x564091a05130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:28.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.620+0000 7fa093eeeb80 1 -- 172.21.15.119:0/135433583 shutdown_connections 2023-12-07T10:22:28.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.620+0000 7fa093eeeb80 1 -- 172.21.15.119:0/135433583 wait complete. 2023-12-07T10:22:28.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.621+0000 7fa093eeeb80 1 -- start start 2023-12-07T10:22:28.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.621+0000 7fa093eeeb80 1 Processor -- start 2023-12-07T10:22:28.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.621+0000 7fa093eeeb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564091a0ab40 0x564091a16940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.621+0000 7fa093eeeb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564091a0b460 0x564091a13d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.621+0000 7fa093eeeb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564091a0f150 0x564091a14420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.621+0000 7fa093eeeb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56409195e9b0 con 0x564091a0b460 2023-12-07T10:22:28.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.621+0000 7fa093eeeb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x564091918560 con 0x564091a0ab40 2023-12-07T10:22:28.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.621+0000 7fa093eeeb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564091918200 con 0x564091a0f150 2023-12-07T10:22:28.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.622+0000 7fa093eeeb80 10 client.0 _ll_register_callbacks cb 0x5640918ff940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:22:28.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.622+0000 7fa07e39b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564091a0ab40 0x564091a16940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.622+0000 7fa07e39b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564091a0ab40 0x564091a16940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42540/0 (socket says 172.21.15.119:42540) 2023-12-07T10:22:28.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.622+0000 7fa07e39b700 1 -- 172.21.15.119:0/1159790102 learned_addr learned my addr 172.21.15.119:0/1159790102 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:28.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.622+0000 7fa07db9a700 1 --2- 172.21.15.119:0/1159790102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564091a0b460 0x564091a13d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.622+0000 7fa07eb9c700 1 --2- 172.21.15.119:0/1159790102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564091a0f150 0x564091a14420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.622+0000 7fa07db9a700 1 -- 172.21.15.119:0/1159790102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564091a0f150 msgr2=0x564091a14420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.622+0000 7fa07db9a700 1 --2- 172.21.15.119:0/1159790102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564091a0f150 0x564091a14420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.622+0000 7fa07db9a700 1 -- 172.21.15.119:0/1159790102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564091a0ab40 msgr2=0x564091a16940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.622+0000 7fa07db9a700 1 --2- 172.21.15.119:0/1159790102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564091a0ab40 0x564091a16940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.622+0000 7fa07db9a700 1 -- 172.21.15.119:0/1159790102 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa07810ece0 con 0x564091a0b460 2023-12-07T10:22:28.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.622+0000 7fa07db9a700 1 --2- 172.21.15.119:0/1159790102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564091a0b460 0x564091a13d30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa068009fd0 tx=0x7fa0680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac42284fa627dc server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:28.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.622+0000 7fa061ffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:22:28.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.623+0000 7fa061ffb700 1 -- 172.21.15.119:0/1159790102 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0680122b0 con 0x564091a0b460 2023-12-07T10:22:28.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.623+0000 7fa093eeeb80 1 -- 172.21.15.119:0/1159790102 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640919fdb00 con 0x564091a0b460 2023-12-07T10:22:28.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.623+0000 7fa061ffb700 1 -- 172.21.15.119:0/1159790102 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fa068003890 con 0x564091a0b460 2023-12-07T10:22:28.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.623+0000 7fa061ffb700 1 -- 172.21.15.119:0/1159790102 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa068003a50 con 0x564091a0b460 2023-12-07T10:22:28.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.623+0000 7fa093eeeb80 10 client.16350 fetch_fsmap learned FSMap version 32 2023-12-07T10:22:28.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.623+0000 7fa093eeeb80 1 -- 172.21.15.119:0/1159790102 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fa0781a6750 con 0x564091a0b460 2023-12-07T10:22:28.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.623+0000 7fa061ffb700 1 -- 172.21.15.119:0/1159790102 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5640919fdb00 con 0x564091a0b460 2023-12-07T10:22:28.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.623+0000 7fa061ffb700 1 -- 172.21.15.119:0/1159790102 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa068005020 con 0x564091a0b460 2023-12-07T10:22:28.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.623+0000 7fa093eeeb80 10 client.16350 fetch_fsmap finished waiting for FSMap version 32 2023-12-07T10:22:28.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.623+0000 7fa093eeeb80 10 client.16350 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-07T10:22:28.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.624+0000 7fa093eeeb80 20 client.16350 populate_metadata read hostname 'smithi119' 2023-12-07T10:22:28.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.624+0000 7fa093eeeb80 1 --2- 172.21.15.119:0/1159790102 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x564091a243f0 0x564091a26810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.624+0000 7fa093eeeb80 4 client.16350 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:22:28.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.624+0000 7fa093eeeb80 1 -- 172.21.15.119:0/1159790102 --> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5640919d97a0 con 0x564091a243f0 2023-12-07T10:22:28.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.624+0000 7fa07e39b700 1 --2- 172.21.15.119:0/1159790102 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x564091a243f0 0x564091a26810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.625+0000 7fa07e39b700 1 --2- 172.21.15.119:0/1159790102 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x564091a243f0 0x564091a26810 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7f345dfc913dae9 server_cookie=ed771ef8994ce9b in_seq=0 out_seq=0 2023-12-07T10:22:28.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.625+0000 7fa061ffb700 10 client.16350 ms_handle_connect on v2:172.21.15.159:6836/3106097520 2023-12-07T10:22:28.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.625+0000 7fa061ffb700 1 -- 172.21.15.119:0/1159790102 <== mds.2 v2:172.21.15.159:6836/3106097520 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5640919d97a0 con 0x564091a243f0 2023-12-07T10:22:28.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.625+0000 7fa061ffb700 10 client.16350 handle_command_reply: tid=0 2023-12-07T10:22:28.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.626+0000 7fa093eeeb80 1 client.16350 shutdown 2023-12-07T10:22:28.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.626+0000 7fa093eeeb80 1 -- 172.21.15.119:0/1159790102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564091a0b460 msgr2=0x564091a13d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.626+0000 7fa093eeeb80 1 --2- 172.21.15.119:0/1159790102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564091a0b460 0x564091a13d30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa068009fd0 tx=0x7fa0680047c0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.626+0000 7fa093eeeb80 1 -- 172.21.15.119:0/1159790102 shutdown_connections 2023-12-07T10:22:28.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.626+0000 7fa093eeeb80 1 --2- 172.21.15.119:0/1159790102 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x564091a243f0 0x564091a26810 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.626+0000 7fa093eeeb80 1 --2- 172.21.15.119:0/1159790102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564091a0f150 0x564091a14420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.626+0000 7fa093eeeb80 1 --2- 172.21.15.119:0/1159790102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564091a0ab40 0x564091a16940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.626+0000 7fa061ffb700 0 client.16350 ms_handle_reset on v2:172.21.15.159:6836/3106097520 2023-12-07T10:22:28.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.626+0000 7fa093eeeb80 1 --2- 172.21.15.119:0/1159790102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564091a0b460 0x564091a13d30 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.626+0000 7fa093eeeb80 1 -- 172.21.15.119:0/1159790102 >> 172.21.15.119:0/1159790102 conn(0x564091939360 msgr2=0x5640919ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:28.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.626+0000 7fa093eeeb80 1 -- 172.21.15.119:0/1159790102 shutdown_connections 2023-12-07T10:22:28.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.626+0000 7fa093eeeb80 1 -- 172.21.15.119:0/1159790102 wait complete. 2023-12-07T10:22:28.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.626+0000 7fa093eeeb80 20 client.16350 trim_cache size 0 max 16384 2023-12-07T10:22:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.641+0000 7fa093eeeb80 1 Processor -- start 2023-12-07T10:22:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.641+0000 7fa093eeeb80 1 -- start start 2023-12-07T10:22:28.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.641+0000 7fa093eeeb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564091a0ab40 0x564091a0db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.641+0000 7fa093eeeb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564091a0b460 0x564091a135f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.641+0000 7fa093eeeb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564091a0f150 0x564091a0e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.641+0000 7fa093eeeb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640917dd920 con 0x564091a0ab40 2023-12-07T10:22:28.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.641+0000 7fa093eeeb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x564091a0b170 con 0x564091a0f150 2023-12-07T10:22:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.641+0000 7fa093eeeb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564091a063e0 con 0x564091a0b460 2023-12-07T10:22:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.642+0000 7fa07e39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564091a0ab40 0x564091a0db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.642+0000 7fa07e39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564091a0ab40 0x564091a0db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45000/0 (socket says 172.21.15.119:45000) 2023-12-07T10:22:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.642+0000 7fa07e39b700 1 -- 172.21.15.119:0/406110218 learned_addr learned my addr 172.21.15.119:0/406110218 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.642+0000 7fa07eb9c700 1 --2- 172.21.15.119:0/406110218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564091a0f150 0x564091a0e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.642+0000 7fa07db9a700 1 --2- 172.21.15.119:0/406110218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564091a0b460 0x564091a135f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.642+0000 7fa07e39b700 1 -- 172.21.15.119:0/406110218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564091a0b460 msgr2=0x564091a135f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.642+0000 7fa07e39b700 1 --2- 172.21.15.119:0/406110218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564091a0b460 0x564091a135f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.642+0000 7fa07e39b700 1 -- 172.21.15.119:0/406110218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564091a0f150 msgr2=0x564091a0e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.642+0000 7fa07e39b700 1 --2- 172.21.15.119:0/406110218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564091a0f150 0x564091a0e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.642+0000 7fa07e39b700 1 -- 172.21.15.119:0/406110218 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564091787de0 con 0x564091a0ab40 2023-12-07T10:22:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.642+0000 7fa07e39b700 1 --2- 172.21.15.119:0/406110218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564091a0ab40 0x564091a0db80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa07400b580 tx=0x7fa074005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448cfb447555fc37 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.642+0000 7fa06d3ff700 1 -- 172.21.15.119:0/406110218 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07401c040 con 0x564091a0ab40 2023-12-07T10:22:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.642+0000 7fa06d3ff700 1 -- 172.21.15.119:0/406110218 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa07401d040 con 0x564091a0ab40 2023-12-07T10:22:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.643+0000 7fa06d3ff700 1 -- 172.21.15.119:0/406110218 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0740280b0 con 0x564091a0ab40 2023-12-07T10:22:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.643+0000 7fa093eeeb80 1 -- 172.21.15.119:0/406110218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564091a0ab40 msgr2=0x564091a0db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.643+0000 7fa093eeeb80 1 --2- 172.21.15.119:0/406110218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564091a0ab40 0x564091a0db80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa07400b580 tx=0x7fa074005040 comp rx=0 tx=0).stop 2023-12-07T10:22:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.643+0000 7fa093eeeb80 1 -- 172.21.15.119:0/406110218 shutdown_connections 2023-12-07T10:22:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.643+0000 7fa093eeeb80 1 --2- 172.21.15.119:0/406110218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564091a0f150 0x564091a0e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.643+0000 7fa093eeeb80 1 --2- 172.21.15.119:0/406110218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564091a0ab40 0x564091a0db80 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.643+0000 7fa093eeeb80 1 --2- 172.21.15.119:0/406110218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564091a0b460 0x564091a135f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.643+0000 7fa093eeeb80 1 -- 172.21.15.119:0/406110218 >> 172.21.15.119:0/406110218 conn(0x564091939360 msgr2=0x564091a2b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.643+0000 7fa093eeeb80 1 -- 172.21.15.119:0/406110218 shutdown_connections 2023-12-07T10:22:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.643+0000 7fa093eeeb80 1 -- 172.21.15.119:0/406110218 wait complete. 2023-12-07T10:22:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.644+0000 7fa093eeeb80 1 -- start start 2023-12-07T10:22:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.644+0000 7fa093eeeb80 1 Processor -- start 2023-12-07T10:22:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.644+0000 7fa093eeeb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564091a0ab40 0x564091a1ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.644+0000 7fa093eeeb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564091a0b460 0x564091a1b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.644+0000 7fa093eeeb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564091a0f150 0x564091a798f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.645+0000 7fa093eeeb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564091a1b850 con 0x564091a0f150 2023-12-07T10:22:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.645+0000 7fa093eeeb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x564091a79e60 con 0x564091a0b460 2023-12-07T10:22:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.645+0000 7fa093eeeb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564091a7a020 con 0x564091a0ab40 2023-12-07T10:22:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.645+0000 7fa093eeeb80 10 client.0 _ll_register_callbacks cb 0x5640918ff940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:22:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.645+0000 7fa07db9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564091a0b460 0x564091a1b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.645+0000 7fa07e39b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564091a0ab40 0x564091a1ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.645+0000 7fa07db9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564091a0b460 0x564091a1b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:11725/0 (socket says 172.21.15.119:42558) 2023-12-07T10:22:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.645+0000 7fa07db9a700 1 -- 172.21.15.119:0/3451810551 learned_addr learned my addr 172.21.15.119:0/3451810551 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.645+0000 7fa07eb9c700 1 --2- 172.21.15.119:0/3451810551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564091a0f150 0x564091a798f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.645+0000 7fa07db9a700 1 -- 172.21.15.119:0/3451810551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564091a0ab40 msgr2=0x564091a1ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.645+0000 7fa07db9a700 1 --2- 172.21.15.119:0/3451810551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564091a0ab40 0x564091a1ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.645+0000 7fa07db9a700 1 -- 172.21.15.119:0/3451810551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564091a0f150 msgr2=0x564091a798f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.645+0000 7fa07db9a700 1 --2- 172.21.15.119:0/3451810551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564091a0f150 0x564091a798f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.645+0000 7fa07db9a700 1 -- 172.21.15.119:0/3451810551 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa07400bc80 con 0x564091a0b460 2023-12-07T10:22:28.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.645+0000 7fa07db9a700 1 --2- 172.21.15.119:0/3451810551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564091a0b460 0x564091a1b0b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x564091a06ad0 tx=0x7fa068009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cc1a7ddc8e33a73 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:28.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.645+0000 7fa061ffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:22:28.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.646+0000 7fa061ffb700 1 -- 172.21.15.119:0/3451810551 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa068003890 con 0x564091a0b460 2023-12-07T10:22:28.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.646+0000 7fa061ffb700 1 -- 172.21.15.119:0/3451810551 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fa0680039f0 con 0x564091a0b460 2023-12-07T10:22:28.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.646+0000 7fa093eeeb80 1 -- 172.21.15.119:0/3451810551 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0700092e0 con 0x564091a0b460 2023-12-07T10:22:28.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.646+0000 7fa061ffb700 1 -- 172.21.15.119:0/3451810551 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa068003bf0 con 0x564091a0b460 2023-12-07T10:22:28.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.646+0000 7fa093eeeb80 10 client.26369 fetch_fsmap learned FSMap version 32 2023-12-07T10:22:28.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.646+0000 7fa093eeeb80 1 -- 172.21.15.119:0/3451810551 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564091770c60 con 0x564091a0b460 2023-12-07T10:22:28.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.646+0000 7fa061ffb700 1 -- 172.21.15.119:0/3451810551 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0700092e0 con 0x564091a0b460 2023-12-07T10:22:28.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.646+0000 7fa061ffb700 1 -- 172.21.15.119:0/3451810551 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa068004e60 con 0x564091a0b460 2023-12-07T10:22:28.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.646+0000 7fa093eeeb80 10 client.26369 fetch_fsmap finished waiting for FSMap version 32 2023-12-07T10:22:28.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.646+0000 7fa093eeeb80 10 client.26369 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-07T10:22:28.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.646+0000 7fa093eeeb80 20 client.26369 populate_metadata read hostname 'smithi119' 2023-12-07T10:22:28.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.647+0000 7fa093eeeb80 1 --2- 172.21.15.119:0/3451810551 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x564091a7b920 0x564091a7dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.647+0000 7fa093eeeb80 4 client.26369 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:22:28.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.647+0000 7fa093eeeb80 1 -- 172.21.15.119:0/3451810551 --> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564091924fd0 con 0x564091a7b920 2023-12-07T10:22:28.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.647+0000 7fa07e39b700 1 --2- 172.21.15.119:0/3451810551 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x564091a7b920 0x564091a7dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.647+0000 7fa07e39b700 1 --2- 172.21.15.119:0/3451810551 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x564091a7b920 0x564091a7dd40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c50acd577c5a219 server_cookie=12f3b5f3308e4655 in_seq=0 out_seq=0 2023-12-07T10:22:28.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.647+0000 7fa061ffb700 10 client.26369 ms_handle_connect on v2:172.21.15.159:6836/3106097520 2023-12-07T10:22:28.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.648+0000 7fa061ffb700 1 -- 172.21.15.119:0/3451810551 <== mds.2 v2:172.21.15.159:6836/3106097520 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564091924fd0 con 0x564091a7b920 2023-12-07T10:22:28.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.648+0000 7fa061ffb700 10 client.26369 handle_command_reply: tid=0 2023-12-07T10:22:28.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.648+0000 7fa093eeeb80 1 client.26369 shutdown 2023-12-07T10:22:28.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.648+0000 7fa093eeeb80 1 -- 172.21.15.119:0/3451810551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564091a0b460 msgr2=0x564091a1b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.648+0000 7fa093eeeb80 1 --2- 172.21.15.119:0/3451810551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564091a0b460 0x564091a1b0b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x564091a06ad0 tx=0x7fa068009370 comp rx=0 tx=0).stop 2023-12-07T10:22:28.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.648+0000 7fa093eeeb80 1 -- 172.21.15.119:0/3451810551 shutdown_connections 2023-12-07T10:22:28.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.648+0000 7fa093eeeb80 1 --2- 172.21.15.119:0/3451810551 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x564091a7b920 0x564091a7dd40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.648+0000 7fa093eeeb80 1 --2- 172.21.15.119:0/3451810551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564091a0f150 0x564091a798f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.648+0000 7fa093eeeb80 1 --2- 172.21.15.119:0/3451810551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564091a0ab40 0x564091a1ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.648+0000 7fa061ffb700 0 client.26369 ms_handle_reset on v2:172.21.15.159:6836/3106097520 2023-12-07T10:22:28.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.648+0000 7fa093eeeb80 1 --2- 172.21.15.119:0/3451810551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564091a0b460 0x564091a1b0b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.648+0000 7fa093eeeb80 1 -- 172.21.15.119:0/3451810551 >> 172.21.15.119:0/3451810551 conn(0x564091939360 msgr2=0x564091a0e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:28.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.648+0000 7fa093eeeb80 1 -- 172.21.15.119:0/3451810551 shutdown_connections 2023-12-07T10:22:28.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.648+0000 7fa093eeeb80 1 -- 172.21.15.119:0/3451810551 wait complete. 2023-12-07T10:22:28.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.648+0000 7fa093eeeb80 20 client.26369 trim_cache size 0 max 16384 2023-12-07T10:22:28.654 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:22:28.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.649+0000 7fa0805ff700 1 -- 172.21.15.119:0/487315329 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa0640a1d00 msgr2=0x7fa0640a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.649+0000 7fa0805ff700 1 --2- 172.21.15.119:0/487315329 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa0640a1d00 0x7fa0640a41c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa07819ac30 tx=0x7fa068013c90 comp rx=0 tx=0).stop 2023-12-07T10:22:28.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.649+0000 7fa0805ff700 1 -- 172.21.15.119:0/487315329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa078107620 msgr2=0x7fa078199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.649+0000 7fa0805ff700 1 --2- 172.21.15.119:0/487315329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa078107620 0x7fa078199a70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa07000c370 tx=0x7fa07000c680 comp rx=0 tx=0).stop 2023-12-07T10:22:28.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.649+0000 7fa0805ff700 1 -- 172.21.15.119:0/487315329 shutdown_connections 2023-12-07T10:22:28.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.649+0000 7fa0805ff700 1 --2- 172.21.15.119:0/487315329 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa0640a1d00 0x7fa0640a41c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.649+0000 7fa0805ff700 1 --2- 172.21.15.119:0/487315329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa078107620 0x7fa078199a70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.649+0000 7fa0805ff700 1 --2- 172.21.15.119:0/487315329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa078101560 0x7fa07819f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.649+0000 7fa0805ff700 1 --2- 172.21.15.119:0/487315329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa078101f20 0x7fa07819fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.649+0000 7fa0805ff700 1 -- 172.21.15.119:0/487315329 >> 172.21.15.119:0/487315329 conn(0x7fa0780fcfd0 msgr2=0x7fa0780fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:28.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.649+0000 7fa0805ff700 1 -- 172.21.15.119:0/487315329 shutdown_connections 2023-12-07T10:22:28.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.649+0000 7fa0805ff700 1 -- 172.21.15.119:0/487315329 wait complete. 2023-12-07T10:22:28.669 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-07T10:22:28.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.885+0000 7f17d09fe700 1 -- 172.21.15.119:0/1804330047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17cc1013e0 msgr2=0x7f17cc101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.885+0000 7f17d09fe700 1 --2- 172.21.15.119:0/1804330047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17cc1013e0 0x7f17cc101860 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009990 tx=0x7f17c0009ca0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.885+0000 7f17d09fe700 1 -- 172.21.15.119:0/1804330047 shutdown_connections 2023-12-07T10:22:28.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.885+0000 7f17d09fe700 1 --2- 172.21.15.119:0/1804330047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17cc101da0 0x7f17cc10e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.885+0000 7f17d09fe700 1 --2- 172.21.15.119:0/1804330047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc107480 0x7f17cc107860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.885+0000 7f17d09fe700 1 --2- 172.21.15.119:0/1804330047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17cc1013e0 0x7f17cc101860 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.885+0000 7f17d09fe700 1 -- 172.21.15.119:0/1804330047 >> 172.21.15.119:0/1804330047 conn(0x7f17cc0fcf10 msgr2=0x7f17cc0ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:28.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.886+0000 7f17d09fe700 1 -- 172.21.15.119:0/1804330047 shutdown_connections 2023-12-07T10:22:28.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.886+0000 7f17d09fe700 1 -- 172.21.15.119:0/1804330047 wait complete. 2023-12-07T10:22:28.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.887+0000 7f17d09fe700 1 Processor -- start 2023-12-07T10:22:28.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.887+0000 7f17d09fe700 1 -- start start 2023-12-07T10:22:28.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.888+0000 7f17d09fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17cc1013e0 0x7f17cc19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.888+0000 7f17d09fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc101da0 0x7f17cc19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.888+0000 7f17d09fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17cc107480 0x7f17cc1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.888+0000 7f17d09fe700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc199f30 con 0x7f17cc101da0 2023-12-07T10:22:28.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.888+0000 7f17d09fe700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc19a0a0 con 0x7f17cc107480 2023-12-07T10:22:28.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.888+0000 7f17d09fe700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc19a260 con 0x7f17cc1013e0 2023-12-07T10:22:28.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.888+0000 7f17cb59e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17cc107480 0x7f17cc1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.888+0000 7f17ca59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc101da0 0x7f17cc19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.888+0000 7f17cb59e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17cc107480 0x7f17cc1998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42578/0 (socket says 172.21.15.119:42578) 2023-12-07T10:22:28.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.888+0000 7f17cb59e700 1 -- 172.21.15.119:0/2278948571 learned_addr learned my addr 172.21.15.119:0/2278948571 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:28.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.888+0000 7f17cad9d700 1 --2- 172.21.15.119:0/2278948571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17cc1013e0 0x7f17cc19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.889+0000 7f17cb59e700 1 -- 172.21.15.119:0/2278948571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17cc1013e0 msgr2=0x7f17cc19f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.889+0000 7f17cb59e700 1 --2- 172.21.15.119:0/2278948571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17cc1013e0 0x7f17cc19f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.889+0000 7f17cb59e700 1 -- 172.21.15.119:0/2278948571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc101da0 msgr2=0x7f17cc19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.889+0000 7f17cb59e700 1 --2- 172.21.15.119:0/2278948571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc101da0 0x7f17cc19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.889+0000 7f17cb59e700 1 -- 172.21.15.119:0/2278948571 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c00095f0 con 0x7f17cc107480 2023-12-07T10:22:28.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.890+0000 7f17cb59e700 1 --2- 172.21.15.119:0/2278948571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17cc107480 0x7f17cc1998d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f17c400ef30 tx=0x7f17c400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:28.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.890+0000 7f17b3fff700 1 -- 172.21.15.119:0/2278948571 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c4018070 con 0x7f17cc107480 2023-12-07T10:22:28.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.891+0000 7f17b3fff700 1 -- 172.21.15.119:0/2278948571 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17c400f040 con 0x7f17cc107480 2023-12-07T10:22:28.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.891+0000 7f17b3fff700 1 -- 172.21.15.119:0/2278948571 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c400cd00 con 0x7f17cc107480 2023-12-07T10:22:28.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.891+0000 7f17d09fe700 1 -- 172.21.15.119:0/2278948571 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17cc19a4e0 con 0x7f17cc107480 2023-12-07T10:22:28.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.891+0000 7f17d09fe700 1 -- 172.21.15.119:0/2278948571 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17cc06b210 con 0x7f17cc107480 2023-12-07T10:22:28.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.901+0000 7f17b3fff700 1 -- 172.21.15.119:0/2278948571 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f17c4003a00 con 0x7f17cc107480 2023-12-07T10:22:28.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.901+0000 7f17e4b47b80 1 Processor -- start 2023-12-07T10:22:28.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.902+0000 7f17e4b47b80 1 -- start start 2023-12-07T10:22:28.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.902+0000 7f17e4b47b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616e5c3c9e0 0x5616e5c3cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.902+0000 7f17e4b47b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3d300 0x5616e5c40da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.902+0000 7f17e4b47b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e5c412e0 0x5616e5c41740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.902+0000 7f17e4b47b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e5bb07c0 con 0x5616e5c412e0 2023-12-07T10:22:28.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.902+0000 7f17e4b47b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e5b977a0 con 0x5616e5c3d300 2023-12-07T10:22:28.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.902+0000 7f17e4b47b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e5b950c0 con 0x5616e5c3c9e0 2023-12-07T10:22:28.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.902+0000 7f17b3fff700 1 --2- 172.21.15.119:0/2278948571 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f17b40a1d00 0x7f17b40a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.903+0000 7f17cad9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3d300 0x5616e5c40da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.903+0000 7f17cad9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3d300 0x5616e5c40da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42588/0 (socket says 172.21.15.119:42588) 2023-12-07T10:22:28.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.903+0000 7f17cad9d700 1 -- 172.21.15.119:0/1885498260 learned_addr learned my addr 172.21.15.119:0/1885498260 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:28.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.903+0000 7f17ca59c700 1 --2- 172.21.15.119:0/1885498260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616e5c3c9e0 0x5616e5c3cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.903+0000 7f17ca59c700 1 --2- 172.21.15.119:0/1885498260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e5c412e0 0x5616e5c41740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.903+0000 7f17b3fff700 1 -- 172.21.15.119:0/2278948571 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f17c40cd820 con 0x7f17cc107480 2023-12-07T10:22:28.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.903+0000 7f17cb59e700 1 --2- 172.21.15.119:0/2278948571 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f17b40a1d00 0x7f17b40a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.903+0000 7f17cad9d700 1 -- 172.21.15.119:0/1885498260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616e5c3c9e0 msgr2=0x5616e5c3cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.903+0000 7f17cad9d700 1 --2- 172.21.15.119:0/1885498260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616e5c3c9e0 0x5616e5c3cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.903+0000 7f17cad9d700 1 -- 172.21.15.119:0/1885498260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e5c412e0 msgr2=0x5616e5c41740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.903+0000 7f17cad9d700 1 --2- 172.21.15.119:0/1885498260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e5c412e0 0x5616e5c41740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.903+0000 7f17cad9d700 1 -- 172.21.15.119:0/1885498260 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616e5a0d630 con 0x5616e5c3d300 2023-12-07T10:22:28.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.903+0000 7f17cad9d700 1 --2- 172.21.15.119:0/1885498260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3d300 0x5616e5c40da0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00bcf0 tx=0x7f17bc0103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=567e1bd908b5770b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:28.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.904+0000 7f17b17fa700 1 -- 172.21.15.119:0/1885498260 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc010c50 con 0x5616e5c3d300 2023-12-07T10:22:28.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.904+0000 7f17b17fa700 1 -- 172.21.15.119:0/1885498260 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17bc015070 con 0x5616e5c3d300 2023-12-07T10:22:28.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.904+0000 7f17b17fa700 1 -- 172.21.15.119:0/1885498260 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc0056a0 con 0x5616e5c3d300 2023-12-07T10:22:28.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.904+0000 7f17e4b47b80 1 -- 172.21.15.119:0/1885498260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3d300 msgr2=0x5616e5c40da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.904+0000 7f17cb59e700 1 --2- 172.21.15.119:0/2278948571 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f17b40a1d00 0x7f17b40a41c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f17c4096d70 tx=0x7f17c4004dc0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:28.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.904+0000 7f17e4b47b80 1 --2- 172.21.15.119:0/1885498260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3d300 0x5616e5c40da0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00bcf0 tx=0x7f17bc0103d0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.905+0000 7f17e4b47b80 1 -- 172.21.15.119:0/1885498260 shutdown_connections 2023-12-07T10:22:28.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.905+0000 7f17e4b47b80 1 --2- 172.21.15.119:0/1885498260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e5c412e0 0x5616e5c41740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.905+0000 7f17e4b47b80 1 --2- 172.21.15.119:0/1885498260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616e5c3c9e0 0x5616e5c3cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.905+0000 7f17e4b47b80 1 --2- 172.21.15.119:0/1885498260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3d300 0x5616e5c40da0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.905+0000 7f17e4b47b80 1 -- 172.21.15.119:0/1885498260 >> 172.21.15.119:0/1885498260 conn(0x5616e5b6b360 msgr2=0x5616e5c37170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:28.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.905+0000 7f17e4b47b80 1 -- 172.21.15.119:0/1885498260 shutdown_connections 2023-12-07T10:22:28.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.905+0000 7f17e4b47b80 1 -- 172.21.15.119:0/1885498260 wait complete. 2023-12-07T10:22:28.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.905+0000 7f17e4b47b80 1 -- start start 2023-12-07T10:22:28.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.906+0000 7f17e4b47b80 1 Processor -- start 2023-12-07T10:22:28.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.906+0000 7f17e4b47b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616e5c3c9e0 0x5616e5c48ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.906+0000 7f17e4b47b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3d300 0x5616e5c45df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.906+0000 7f17e4b47b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e5c412e0 0x5616e5c46530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.906+0000 7f17e4b47b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e5b909b0 con 0x5616e5c412e0 2023-12-07T10:22:28.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.906+0000 7f17e4b47b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e5b4a560 con 0x5616e5c3d300 2023-12-07T10:22:28.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.906+0000 7f17e4b47b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e5b4a200 con 0x5616e5c3c9e0 2023-12-07T10:22:28.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.906+0000 7f17e4b47b80 10 client.0 _ll_register_callbacks cb 0x5616e5b31940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:22:28.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.907+0000 7f17ca59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e5c412e0 0x5616e5c46530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.907+0000 7f17cad9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3d300 0x5616e5c45df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.907+0000 7f17cad9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3d300 0x5616e5c45df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42602/0 (socket says 172.21.15.119:42602) 2023-12-07T10:22:28.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.907+0000 7f17cad9d700 1 -- 172.21.15.119:0/1480521000 learned_addr learned my addr 172.21.15.119:0/1480521000 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:28.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.907+0000 7f17cad9d700 1 -- 172.21.15.119:0/1480521000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616e5c3c9e0 msgr2=0x5616e5c48ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.907+0000 7f17cad9d700 1 --2- 172.21.15.119:0/1480521000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616e5c3c9e0 0x5616e5c48ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.907+0000 7f17cad9d700 1 -- 172.21.15.119:0/1480521000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e5c412e0 msgr2=0x5616e5c46530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.907+0000 7f17cad9d700 1 --2- 172.21.15.119:0/1480521000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e5c412e0 0x5616e5c46530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.907+0000 7f17cad9d700 1 -- 172.21.15.119:0/1480521000 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17bc00b920 con 0x5616e5c3d300 2023-12-07T10:22:28.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.907+0000 7f17ca59c700 1 --2- 172.21.15.119:0/1480521000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e5c412e0 0x5616e5c46530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:28.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.908+0000 7f17cad9d700 1 --2- 172.21.15.119:0/1480521000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3d300 0x5616e5c45df0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f17bc010c90 tx=0x7f17bc010cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=245078ee2d256758 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:28.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.908+0000 7f17aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:22:28.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.908+0000 7f17aeff5700 1 -- 172.21.15.119:0/1480521000 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc005650 con 0x5616e5c3d300 2023-12-07T10:22:28.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.908+0000 7f17aeff5700 1 -- 172.21.15.119:0/1480521000 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f17bc017070 con 0x5616e5c3d300 2023-12-07T10:22:28.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.908+0000 7f17e4b47b80 1 -- 172.21.15.119:0/1480521000 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5616e5c2f9a0 con 0x5616e5c3d300 2023-12-07T10:22:28.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.908+0000 7f17aeff5700 1 -- 172.21.15.119:0/1480521000 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17bc003680 con 0x5616e5c3d300 2023-12-07T10:22:28.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.908+0000 7f17e4b47b80 10 client.26381 fetch_fsmap learned FSMap version 32 2023-12-07T10:22:28.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.908+0000 7f17e4b47b80 1 -- 172.21.15.119:0/1480521000 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5616e59b9de0 con 0x5616e5c3d300 2023-12-07T10:22:28.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.909+0000 7f17aeff5700 1 -- 172.21.15.119:0/1480521000 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5616e5c2f9a0 con 0x5616e5c3d300 2023-12-07T10:22:28.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.909+0000 7f17aeff5700 1 -- 172.21.15.119:0/1480521000 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f17bc0257b0 con 0x5616e5c3d300 2023-12-07T10:22:28.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.909+0000 7f17e4b47b80 10 client.26381 fetch_fsmap finished waiting for FSMap version 32 2023-12-07T10:22:28.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.909+0000 7f17e4b47b80 10 client.26381 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2023-12-07T10:22:28.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.909+0000 7f17e4b47b80 20 client.26381 populate_metadata read hostname 'smithi119' 2023-12-07T10:22:28.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.909+0000 7f17e4b47b80 1 --2- 172.21.15.119:0/1480521000 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x5616e5c564e0 0x5616e5c58900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.909+0000 7f17e4b47b80 4 client.26381 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:22:28.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.909+0000 7f17e4b47b80 1 -- 172.21.15.119:0/1480521000 --> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616e5c0b7a0 con 0x5616e5c564e0 2023-12-07T10:22:28.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.909+0000 7f17ca59c700 1 --2- 172.21.15.119:0/1480521000 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x5616e5c564e0 0x5616e5c58900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.910+0000 7f17ca59c700 1 --2- 172.21.15.119:0/1480521000 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x5616e5c564e0 0x5616e5c58900 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dc2811f424c8c07e server_cookie=b87aa6bb16c8ad79 in_seq=0 out_seq=0 2023-12-07T10:22:28.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.910+0000 7f17aeff5700 10 client.26381 ms_handle_connect on v2:172.21.15.119:6838/1920762602 2023-12-07T10:22:28.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.911+0000 7f17aeff5700 1 -- 172.21.15.119:0/1480521000 <== mds.1 v2:172.21.15.119:6838/1920762602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5616e5c0b7a0 con 0x5616e5c564e0 2023-12-07T10:22:28.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.911+0000 7f17aeff5700 10 client.26381 handle_command_reply: tid=0 2023-12-07T10:22:28.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.911+0000 7f17e4b47b80 1 client.26381 shutdown 2023-12-07T10:22:28.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.911+0000 7f17e4b47b80 1 -- 172.21.15.119:0/1480521000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3d300 msgr2=0x5616e5c45df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.911+0000 7f17e4b47b80 1 --2- 172.21.15.119:0/1480521000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3d300 0x5616e5c45df0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f17bc010c90 tx=0x7f17bc010cc0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.911+0000 7f17e4b47b80 1 -- 172.21.15.119:0/1480521000 shutdown_connections 2023-12-07T10:22:28.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.911+0000 7f17e4b47b80 1 --2- 172.21.15.119:0/1480521000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e5c412e0 0x5616e5c46530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.911+0000 7f17e4b47b80 1 --2- 172.21.15.119:0/1480521000 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x5616e5c564e0 0x5616e5c58900 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.911+0000 7f17e4b47b80 1 --2- 172.21.15.119:0/1480521000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616e5c3c9e0 0x5616e5c48ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.911+0000 7f17e4b47b80 1 --2- 172.21.15.119:0/1480521000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3d300 0x5616e5c45df0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.911+0000 7f17e4b47b80 1 -- 172.21.15.119:0/1480521000 >> 172.21.15.119:0/1480521000 conn(0x5616e5b6b360 msgr2=0x5616e5c3b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:28.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.911+0000 7f17aeff5700 0 client.26381 ms_handle_reset on v2:172.21.15.119:6838/1920762602 2023-12-07T10:22:28.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.911+0000 7f17e4b47b80 1 -- 172.21.15.119:0/1480521000 shutdown_connections 2023-12-07T10:22:28.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.911+0000 7f17e4b47b80 1 -- 172.21.15.119:0/1480521000 wait complete. 2023-12-07T10:22:28.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.911+0000 7f17e4b47b80 20 client.26381 trim_cache size 0 max 16384 2023-12-07T10:22:28.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.926+0000 7f17e4b47b80 1 Processor -- start 2023-12-07T10:22:28.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.926+0000 7f17e4b47b80 1 -- start start 2023-12-07T10:22:28.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.927+0000 7f17e4b47b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616e5c3c9e0 0x5616e5c426d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.927+0000 7f17e4b47b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3d300 0x5616e5c5d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.927+0000 7f17e4b47b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e5c412e0 0x5616e5c389b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.927+0000 7f17e4b47b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e5a0f920 con 0x5616e5c412e0 2023-12-07T10:22:28.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.927+0000 7f17e4b47b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e5c2f970 con 0x5616e5c3d300 2023-12-07T10:22:28.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.927+0000 7f17e4b47b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e5c42c40 con 0x5616e5c3c9e0 2023-12-07T10:22:28.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.927+0000 7f17ca59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616e5c3c9e0 0x5616e5c426d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.927+0000 7f17cad9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3d300 0x5616e5c5d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.927+0000 7f17cad9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3d300 0x5616e5c5d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42606/0 (socket says 172.21.15.119:42606) 2023-12-07T10:22:28.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.927+0000 7f17cad9d700 1 -- 172.21.15.119:0/1378600462 learned_addr learned my addr 172.21.15.119:0/1378600462 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:28.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.927+0000 7f17ca59c700 1 --2- 172.21.15.119:0/1378600462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e5c412e0 0x5616e5c389b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.928+0000 7f17cad9d700 1 -- 172.21.15.119:0/1378600462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616e5c3c9e0 msgr2=0x5616e5c426d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.928+0000 7f17cad9d700 1 --2- 172.21.15.119:0/1378600462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616e5c3c9e0 0x5616e5c426d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.928+0000 7f17cad9d700 1 -- 172.21.15.119:0/1378600462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e5c412e0 msgr2=0x5616e5c389b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.928+0000 7f17cad9d700 1 --2- 172.21.15.119:0/1378600462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e5c412e0 0x5616e5c389b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.928+0000 7f17cad9d700 1 -- 172.21.15.119:0/1378600462 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616e59a2c60 con 0x5616e5c3d300 2023-12-07T10:22:28.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.928+0000 7f17cad9d700 1 --2- 172.21.15.119:0/1378600462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3d300 0x5616e5c5d220 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f17bc025b70 tx=0x7f17bc0103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fb0e8c7d0425608 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:28.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.928+0000 7f17b1ffb700 1 -- 172.21.15.119:0/1378600462 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc005650 con 0x5616e5c3d300 2023-12-07T10:22:28.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.928+0000 7f17b1ffb700 1 -- 172.21.15.119:0/1378600462 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17bc020040 con 0x5616e5c3d300 2023-12-07T10:22:28.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.928+0000 7f17b1ffb700 1 -- 172.21.15.119:0/1378600462 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc003bf0 con 0x5616e5c3d300 2023-12-07T10:22:28.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.929+0000 7f17e4b47b80 1 -- 172.21.15.119:0/1378600462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3d300 msgr2=0x5616e5c5d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.929+0000 7f17e4b47b80 1 --2- 172.21.15.119:0/1378600462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3d300 0x5616e5c5d220 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f17bc025b70 tx=0x7f17bc0103d0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.929+0000 7f17e4b47b80 1 -- 172.21.15.119:0/1378600462 shutdown_connections 2023-12-07T10:22:28.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.929+0000 7f17e4b47b80 1 --2- 172.21.15.119:0/1378600462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e5c412e0 0x5616e5c389b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.929+0000 7f17e4b47b80 1 --2- 172.21.15.119:0/1378600462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616e5c3c9e0 0x5616e5c426d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.929+0000 7f17e4b47b80 1 --2- 172.21.15.119:0/1378600462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3d300 0x5616e5c5d220 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.929+0000 7f17e4b47b80 1 -- 172.21.15.119:0/1378600462 >> 172.21.15.119:0/1378600462 conn(0x5616e5b6b360 msgr2=0x5616e5c3bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:28.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.929+0000 7f17e4b47b80 1 -- 172.21.15.119:0/1378600462 shutdown_connections 2023-12-07T10:22:28.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.929+0000 7f17e4b47b80 1 -- 172.21.15.119:0/1378600462 wait complete. 2023-12-07T10:22:28.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.930+0000 7f17e4b47b80 1 -- start start 2023-12-07T10:22:28.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.930+0000 7f17e4b47b80 1 Processor -- start 2023-12-07T10:22:28.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.930+0000 7f17e4b47b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3c9e0 0x5616e5c4d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.930+0000 7f17e4b47b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e5c3d300 0x5616e5c4da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.930+0000 7f17e4b47b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616e5c412e0 0x5616e5c445b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.930+0000 7f17e4b47b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e5c44cc0 con 0x5616e5c3d300 2023-12-07T10:22:28.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.930+0000 7f17e4b47b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e5c44e30 con 0x5616e5c3c9e0 2023-12-07T10:22:28.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.930+0000 7f17e4b47b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e5c44ff0 con 0x5616e5c412e0 2023-12-07T10:22:28.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.930+0000 7f17e4b47b80 10 client.0 _ll_register_callbacks cb 0x5616e5b31940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:22:28.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.930+0000 7f17ca59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3c9e0 0x5616e5c4d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.931+0000 7f17ca59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3c9e0 0x5616e5c4d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42608/0 (socket says 172.21.15.119:42608) 2023-12-07T10:22:28.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.931+0000 7f17ca59c700 1 -- 172.21.15.119:0/2101300748 learned_addr learned my addr 172.21.15.119:0/2101300748 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:28.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.931+0000 7f17ca59c700 1 --2- 172.21.15.119:0/2101300748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616e5c412e0 0x5616e5c445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.931+0000 7f17cad9d700 1 --2- 172.21.15.119:0/2101300748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e5c3d300 0x5616e5c4da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.931+0000 7f17ca59c700 1 -- 172.21.15.119:0/2101300748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616e5c412e0 msgr2=0x5616e5c445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.931+0000 7f17ca59c700 1 --2- 172.21.15.119:0/2101300748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616e5c412e0 0x5616e5c445b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.931+0000 7f17ca59c700 1 -- 172.21.15.119:0/2101300748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e5c3d300 msgr2=0x5616e5c4da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:28.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.931+0000 7f17ca59c700 1 --2- 172.21.15.119:0/2101300748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e5c3d300 0x5616e5c4da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.931+0000 7f17ca59c700 1 -- 172.21.15.119:0/2101300748 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616e59b9de0 con 0x5616e5c3c9e0 2023-12-07T10:22:28.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.931+0000 7f17ca59c700 1 --2- 172.21.15.119:0/2101300748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3c9e0 0x5616e5c4d500 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f17c00098e0 tx=0x7f17c0009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7b8c9adaaa11ff1 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:28.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.931+0000 7f17aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:22:28.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.931+0000 7f17aeff5700 1 -- 172.21.15.119:0/2101300748 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c001d040 con 0x5616e5c3c9e0 2023-12-07T10:22:28.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.931+0000 7f17e4b47b80 1 -- 172.21.15.119:0/2101300748 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17bc015070 con 0x5616e5c3c9e0 2023-12-07T10:22:28.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.931+0000 7f17aeff5700 1 -- 172.21.15.119:0/2101300748 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f17c000fc70 con 0x5616e5c3c9e0 2023-12-07T10:22:28.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.932+0000 7f17aeff5700 1 -- 172.21.15.119:0/2101300748 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17c000fe30 con 0x5616e5c3c9e0 2023-12-07T10:22:28.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.932+0000 7f17e4b47b80 10 client.26387 fetch_fsmap learned FSMap version 32 2023-12-07T10:22:28.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.932+0000 7f17e4b47b80 1 -- 172.21.15.119:0/2101300748 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5616e59e0250 con 0x5616e5c3c9e0 2023-12-07T10:22:28.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.932+0000 7f17aeff5700 1 -- 172.21.15.119:0/2101300748 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17bc015070 con 0x5616e5c3c9e0 2023-12-07T10:22:28.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.932+0000 7f17aeff5700 1 -- 172.21.15.119:0/2101300748 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f17c002a570 con 0x5616e5c3c9e0 2023-12-07T10:22:28.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.932+0000 7f17e4b47b80 10 client.26387 fetch_fsmap finished waiting for FSMap version 32 2023-12-07T10:22:28.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.932+0000 7f17e4b47b80 10 client.26387 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2023-12-07T10:22:28.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.932+0000 7f17e4b47b80 20 client.26387 populate_metadata read hostname 'smithi119' 2023-12-07T10:22:28.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.932+0000 7f17e4b47b80 1 --2- 172.21.15.119:0/2101300748 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x5616e5cacf40 0x5616e5caf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:28.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.932+0000 7f17e4b47b80 4 client.26387 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:22:28.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.932+0000 7f17e4b47b80 1 -- 172.21.15.119:0/2101300748 --> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5616e5b56fd0 con 0x5616e5cacf40 2023-12-07T10:22:28.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.932+0000 7f17cad9d700 1 --2- 172.21.15.119:0/2101300748 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x5616e5cacf40 0x5616e5caf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:28.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.933+0000 7f17cad9d700 1 --2- 172.21.15.119:0/2101300748 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x5616e5cacf40 0x5616e5caf360 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f5147c43d386852f server_cookie=a5ed6169ef7a1e08 in_seq=0 out_seq=0 2023-12-07T10:22:28.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.933+0000 7f17aeff5700 10 client.26387 ms_handle_connect on v2:172.21.15.119:6838/1920762602 2023-12-07T10:22:28.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.933+0000 7f17aeff5700 1 -- 172.21.15.119:0/2101300748 <== mds.1 v2:172.21.15.119:6838/1920762602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5616e5b56fd0 con 0x5616e5cacf40 2023-12-07T10:22:28.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.933+0000 7f17aeff5700 10 client.26387 handle_command_reply: tid=0 2023-12-07T10:22:28.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.933+0000 7f17e4b47b80 1 client.26387 shutdown 2023-12-07T10:22:28.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.933+0000 7f17e4b47b80 1 -- 172.21.15.119:0/2101300748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3c9e0 msgr2=0x5616e5c4d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.933+0000 7f17e4b47b80 1 --2- 172.21.15.119:0/2101300748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3c9e0 0x5616e5c4d500 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f17c00098e0 tx=0x7f17c0009ca0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.933+0000 7f17e4b47b80 1 -- 172.21.15.119:0/2101300748 shutdown_connections 2023-12-07T10:22:28.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.933+0000 7f17e4b47b80 1 --2- 172.21.15.119:0/2101300748 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x5616e5cacf40 0x5616e5caf360 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.933+0000 7f17e4b47b80 1 --2- 172.21.15.119:0/2101300748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616e5c412e0 0x5616e5c445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.933+0000 7f17e4b47b80 1 --2- 172.21.15.119:0/2101300748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616e5c3c9e0 0x5616e5c4d500 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.933+0000 7f17e4b47b80 1 --2- 172.21.15.119:0/2101300748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e5c3d300 0x5616e5c4da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.933+0000 7f17aeff5700 0 client.26387 ms_handle_reset on v2:172.21.15.119:6838/1920762602 2023-12-07T10:22:28.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.933+0000 7f17e4b47b80 1 -- 172.21.15.119:0/2101300748 >> 172.21.15.119:0/2101300748 conn(0x5616e5b6b360 msgr2=0x5616e5c389b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:28.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.934+0000 7f17e4b47b80 1 -- 172.21.15.119:0/2101300748 shutdown_connections 2023-12-07T10:22:28.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.934+0000 7f17e4b47b80 1 -- 172.21.15.119:0/2101300748 wait complete. 2023-12-07T10:22:28.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.934+0000 7f17e4b47b80 20 client.26387 trim_cache size 0 max 16384 2023-12-07T10:22:28.938 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:22:28.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.934+0000 7f17d09fe700 1 -- 172.21.15.119:0/2278948571 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f17b40a1d00 msgr2=0x7f17b40a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.934+0000 7f17d09fe700 1 --2- 172.21.15.119:0/2278948571 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f17b40a1d00 0x7f17b40a41c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f17c4096d70 tx=0x7f17c4004dc0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.934+0000 7f17d09fe700 1 -- 172.21.15.119:0/2278948571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17cc107480 msgr2=0x7f17cc1998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:28.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.934+0000 7f17d09fe700 1 --2- 172.21.15.119:0/2278948571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17cc107480 0x7f17cc1998d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f17c400ef30 tx=0x7f17c400c3a0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.935+0000 7f17d09fe700 1 -- 172.21.15.119:0/2278948571 shutdown_connections 2023-12-07T10:22:28.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.935+0000 7f17d09fe700 1 --2- 172.21.15.119:0/2278948571 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f17b40a1d00 0x7f17b40a41c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.935+0000 7f17d09fe700 1 --2- 172.21.15.119:0/2278948571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17cc107480 0x7f17cc1998d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.935+0000 7f17d09fe700 1 --2- 172.21.15.119:0/2278948571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17cc1013e0 0x7f17cc19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.935+0000 7f17d09fe700 1 --2- 172.21.15.119:0/2278948571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc101da0 0x7f17cc19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:28.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.935+0000 7f17d09fe700 1 -- 172.21.15.119:0/2278948571 >> 172.21.15.119:0/2278948571 conn(0x7f17cc0fcf10 msgr2=0x7f17cc10cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:28.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.935+0000 7f17d09fe700 1 -- 172.21.15.119:0/2278948571 shutdown_connections 2023-12-07T10:22:28.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:28.935+0000 7f17d09fe700 1 -- 172.21.15.119:0/2278948571 wait complete. 2023-12-07T10:22:28.959 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-07T10:22:29.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.184+0000 7f4d0d5ff700 1 -- 172.21.15.119:0/1102158084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d08106ad0 msgr2=0x7f4d0810e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:29.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.184+0000 7f4d0d5ff700 1 --2- 172.21.15.119:0/1102158084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d08106ad0 0x7f4d0810e3b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4d0000b0a0 tx=0x7f4d0000b3b0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.184+0000 7f4d0d5ff700 1 -- 172.21.15.119:0/1102158084 shutdown_connections 2023-12-07T10:22:29.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.184+0000 7f4d0d5ff700 1 --2- 172.21.15.119:0/1102158084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d08106ad0 0x7f4d0810e3b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.184+0000 7f4d0d5ff700 1 --2- 172.21.15.119:0/1102158084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d08068a50 0x7f4d08068e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.184+0000 7f4d0d5ff700 1 --2- 172.21.15.119:0/1102158084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d08106110 0x7f4d08106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.184+0000 7f4d0d5ff700 1 -- 172.21.15.119:0/1102158084 >> 172.21.15.119:0/1102158084 conn(0x7f4d08077610 msgr2=0x7f4d08077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:29.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.184+0000 7f4d0d5ff700 1 -- 172.21.15.119:0/1102158084 shutdown_connections 2023-12-07T10:22:29.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.184+0000 7f4d0d5ff700 1 -- 172.21.15.119:0/1102158084 wait complete. 2023-12-07T10:22:29.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.185+0000 7f4d0d5ff700 1 Processor -- start 2023-12-07T10:22:29.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.185+0000 7f4d0d5ff700 1 -- start start 2023-12-07T10:22:29.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.186+0000 7f4d0d5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d08068a50 0x7f4d0819f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:29.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.186+0000 7f4d0d5ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d08106110 0x7f4d0819fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:29.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.186+0000 7f4d0d5ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d08106ad0 0x7f4d08199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:29.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.186+0000 7f4d0d5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0819a190 con 0x7f4d08068a50 2023-12-07T10:22:29.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.186+0000 7f4d0d5ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0819a300 con 0x7f4d08106ad0 2023-12-07T10:22:29.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.186+0000 7f4d0d5ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0819a4c0 con 0x7f4d08106110 2023-12-07T10:22:29.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.186+0000 7f4d06ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d08068a50 0x7f4d0819f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:29.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.186+0000 7f4cfe7fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d08106110 0x7f4d0819fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:29.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.186+0000 7f4d077fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d08106ad0 0x7f4d08199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:29.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.186+0000 7f4cfe7fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d08106110 0x7f4d0819fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:46718/0 (socket says 172.21.15.119:46718) 2023-12-07T10:22:29.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.186+0000 7f4cfe7fc700 1 -- 172.21.15.119:0/3433884769 learned_addr learned my addr 172.21.15.119:0/3433884769 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:29.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.186+0000 7f4cfe7fc700 1 -- 172.21.15.119:0/3433884769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d08106ad0 msgr2=0x7f4d08199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:29.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.186+0000 7f4cfe7fc700 1 --2- 172.21.15.119:0/3433884769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d08106ad0 0x7f4d08199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.186+0000 7f4cfe7fc700 1 -- 172.21.15.119:0/3433884769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d08068a50 msgr2=0x7f4d0819f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:29.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.186+0000 7f4cfe7fc700 1 --2- 172.21.15.119:0/3433884769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d08068a50 0x7f4d0819f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.186+0000 7f4cfe7fc700 1 -- 172.21.15.119:0/3433884769 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d00009d00 con 0x7f4d08106110 2023-12-07T10:22:29.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.186+0000 7f4d077fe700 1 --2- 172.21.15.119:0/3433884769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d08106ad0 0x7f4d08199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:29.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.187+0000 7f4cfe7fc700 1 --2- 172.21.15.119:0/3433884769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d08106110 0x7f4d0819fee0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000e9a0 tx=0x7f4cf000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:29.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.187+0000 7f4d04ff9700 1 -- 172.21.15.119:0/3433884769 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0009790 con 0x7f4d08106110 2023-12-07T10:22:29.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.187+0000 7f4d04ff9700 1 -- 172.21.15.119:0/3433884769 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cf0010430 con 0x7f4d08106110 2023-12-07T10:22:29.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.187+0000 7f4d0d5ff700 1 -- 172.21.15.119:0/3433884769 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d0819a7a0 con 0x7f4d08106110 2023-12-07T10:22:29.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.187+0000 7f4d04ff9700 1 -- 172.21.15.119:0/3433884769 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0018680 con 0x7f4d08106110 2023-12-07T10:22:29.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.187+0000 7f4d0d5ff700 1 -- 172.21.15.119:0/3433884769 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d081a6a40 con 0x7f4d08106110 2023-12-07T10:22:29.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.190+0000 7f4d04ff9700 1 -- 172.21.15.119:0/3433884769 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f4cf00105a0 con 0x7f4d08106110 2023-12-07T10:22:29.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.192+0000 7f4d20f6fb80 1 Processor -- start 2023-12-07T10:22:29.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.192+0000 7f4d20f6fb80 1 -- start start 2023-12-07T10:22:29.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.192+0000 7f4d20f6fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614dd0b7b40 0x5614dd0b7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:29.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.192+0000 7f4d04ff9700 1 --2- 172.21.15.119:0/3433884769 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4cf40a1d00 0x7f4cf40a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:29.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.192+0000 7f4d20f6fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614dd0b8460 0x5614dd0bbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:29.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.192+0000 7f4d20f6fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dd0bc150 0x5614dd0bc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:29.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.192+0000 7f4d20f6fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614dd02b7c0 con 0x5614dd0b7b40 2023-12-07T10:22:29.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.192+0000 7f4d20f6fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614dd0127a0 con 0x5614dd0bc150 2023-12-07T10:22:29.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.192+0000 7f4d20f6fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614dd0100c0 con 0x5614dd0b8460 2023-12-07T10:22:29.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.192+0000 7f4d077fe700 1 --2- 172.21.15.119:0/3433884769 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4cf40a1d00 0x7f4cf40a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:29.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.193+0000 7f4cfe7fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614dd0b8460 0x5614dd0bbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:29.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.193+0000 7f4cfe7fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614dd0b8460 0x5614dd0bbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:46722/0 (socket says 172.21.15.119:46722) 2023-12-07T10:22:29.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.193+0000 7f4cfe7fc700 1 -- 172.21.15.119:0/1557560232 learned_addr learned my addr 172.21.15.119:0/1557560232 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:29.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.193+0000 7f4d077fe700 1 --2- 172.21.15.119:0/1557560232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dd0bc150 0x5614dd0bc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:29.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.193+0000 7f4d04ff9700 1 -- 172.21.15.119:0/3433884769 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4cf0014070 con 0x7f4d08106110 2023-12-07T10:22:29.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.193+0000 7f4d077fe700 1 --2- 172.21.15.119:0/3433884769 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4cf40a1d00 0x7f4cf40a41c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4d00009040 tx=0x7f4d0000bc60 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:29.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.193+0000 7f4d06ffd700 1 --2- 172.21.15.119:0/1557560232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614dd0b7b40 0x5614dd0b7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:29.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.193+0000 7f4d077fe700 1 -- 172.21.15.119:0/1557560232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614dd0b8460 msgr2=0x5614dd0bbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:29.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.193+0000 7f4d077fe700 1 --2- 172.21.15.119:0/1557560232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614dd0b8460 0x5614dd0bbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.193+0000 7f4d077fe700 1 -- 172.21.15.119:0/1557560232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614dd0b7b40 msgr2=0x5614dd0b7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:29.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.193+0000 7f4d077fe700 1 --2- 172.21.15.119:0/1557560232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614dd0b7b40 0x5614dd0b7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.193+0000 7f4d077fe700 1 -- 172.21.15.119:0/1557560232 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614dce88630 con 0x5614dd0bc150 2023-12-07T10:22:29.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.193+0000 7f4d06ffd700 1 --2- 172.21.15.119:0/1557560232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614dd0b7b40 0x5614dd0b7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:22:29.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.194+0000 7f4d077fe700 1 --2- 172.21.15.119:0/1557560232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dd0bc150 0x5614dd0bc570 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4d0000b070 tx=0x7f4d00028000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f02fa375ce6e6a9e server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:29.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.194+0000 7f4cfd7fa700 1 -- 172.21.15.119:0/1557560232 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0000e040 con 0x5614dd0bc150 2023-12-07T10:22:29.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.194+0000 7f4cfd7fa700 1 -- 172.21.15.119:0/1557560232 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d000076e0 con 0x5614dd0bc150 2023-12-07T10:22:29.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.194+0000 7f4cfd7fa700 1 -- 172.21.15.119:0/1557560232 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0001e6c0 con 0x5614dd0bc150 2023-12-07T10:22:29.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.195+0000 7f4d20f6fb80 1 -- 172.21.15.119:0/1557560232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dd0bc150 msgr2=0x5614dd0bc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:29.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.195+0000 7f4d20f6fb80 1 --2- 172.21.15.119:0/1557560232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dd0bc150 0x5614dd0bc570 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4d0000b070 tx=0x7f4d00028000 comp rx=0 tx=0).stop 2023-12-07T10:22:29.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.195+0000 7f4d20f6fb80 1 -- 172.21.15.119:0/1557560232 shutdown_connections 2023-12-07T10:22:29.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.195+0000 7f4d20f6fb80 1 --2- 172.21.15.119:0/1557560232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dd0bc150 0x5614dd0bc570 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.195+0000 7f4d20f6fb80 1 --2- 172.21.15.119:0/1557560232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614dd0b7b40 0x5614dd0b7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.195+0000 7f4d20f6fb80 1 --2- 172.21.15.119:0/1557560232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614dd0b8460 0x5614dd0bbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.195+0000 7f4d20f6fb80 1 -- 172.21.15.119:0/1557560232 >> 172.21.15.119:0/1557560232 conn(0x5614dcfe6360 msgr2=0x5614dd0b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:29.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.195+0000 7f4d20f6fb80 1 -- 172.21.15.119:0/1557560232 shutdown_connections 2023-12-07T10:22:29.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.195+0000 7f4d20f6fb80 1 -- 172.21.15.119:0/1557560232 wait complete. 2023-12-07T10:22:29.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.195+0000 7f4d20f6fb80 1 -- start start 2023-12-07T10:22:29.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.196+0000 7f4d20f6fb80 1 Processor -- start 2023-12-07T10:22:29.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.196+0000 7f4d20f6fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dd0b7b40 0x5614dd0c3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:29.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.196+0000 7f4d20f6fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614dd0b8460 0x5614dd0c0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:29.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.196+0000 7f4d20f6fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614dd0bc150 0x5614dd0c13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:29.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.196+0000 7f4d20f6fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614dd00b9b0 con 0x5614dd0b8460 2023-12-07T10:22:29.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.196+0000 7f4d20f6fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614dcfc5560 con 0x5614dd0b7b40 2023-12-07T10:22:29.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.196+0000 7f4d20f6fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614dcfc5200 con 0x5614dd0bc150 2023-12-07T10:22:29.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.196+0000 7f4d20f6fb80 10 client.0 _ll_register_callbacks cb 0x5614dcfac940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:22:29.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.196+0000 7f4d06ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dd0b7b40 0x5614dd0c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:29.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.196+0000 7f4cfe7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614dd0b8460 0x5614dd0c0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:29.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.196+0000 7f4d06ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dd0b7b40 0x5614dd0c3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42654/0 (socket says 172.21.15.119:42654) 2023-12-07T10:22:29.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.197+0000 7f4d06ffd700 1 -- 172.21.15.119:0/237924524 learned_addr learned my addr 172.21.15.119:0/237924524 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:29.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.197+0000 7f4d077fe700 1 --2- 172.21.15.119:0/237924524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614dd0bc150 0x5614dd0c13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:29.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.197+0000 7f4d06ffd700 1 -- 172.21.15.119:0/237924524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614dd0bc150 msgr2=0x5614dd0c13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:29.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.197+0000 7f4d06ffd700 1 --2- 172.21.15.119:0/237924524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614dd0bc150 0x5614dd0c13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.197+0000 7f4d06ffd700 1 -- 172.21.15.119:0/237924524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614dd0b8460 msgr2=0x5614dd0c0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:29.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.197+0000 7f4d06ffd700 1 --2- 172.21.15.119:0/237924524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614dd0b8460 0x5614dd0c0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.197+0000 7f4d06ffd700 1 -- 172.21.15.119:0/237924524 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d00027070 con 0x5614dd0b7b40 2023-12-07T10:22:29.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.197+0000 7f4cfe7fc700 1 --2- 172.21.15.119:0/237924524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614dd0b8460 0x5614dd0c0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:29.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.197+0000 7f4d06ffd700 1 --2- 172.21.15.119:0/237924524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dd0b7b40 0x5614dd0c3900 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8013780 tx=0x7f4cf8013a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=386ad6b57ac1ae22 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:29.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.197+0000 7f4cee7fc700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:22:29.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.197+0000 7f4cee7fc700 1 -- 172.21.15.119:0/237924524 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf80117f0 con 0x5614dd0b7b40 2023-12-07T10:22:29.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.197+0000 7f4d20f6fb80 1 -- 172.21.15.119:0/237924524 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614dd0aab00 con 0x5614dd0b7b40 2023-12-07T10:22:29.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.197+0000 7f4cee7fc700 1 -- 172.21.15.119:0/237924524 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4cf8011950 con 0x5614dd0b7b40 2023-12-07T10:22:29.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.198+0000 7f4cee7fc700 1 -- 172.21.15.119:0/237924524 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cf8011b10 con 0x5614dd0b7b40 2023-12-07T10:22:29.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.198+0000 7f4d20f6fb80 10 client.26399 fetch_fsmap learned FSMap version 32 2023-12-07T10:22:29.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.198+0000 7f4d20f6fb80 1 -- 172.21.15.119:0/237924524 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5614dce34de0 con 0x5614dd0b7b40 2023-12-07T10:22:29.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.198+0000 7f4cee7fc700 1 -- 172.21.15.119:0/237924524 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5614dd0aab00 con 0x5614dd0b7b40 2023-12-07T10:22:29.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.198+0000 7f4cee7fc700 1 -- 172.21.15.119:0/237924524 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f4cf800f3c0 con 0x5614dd0b7b40 2023-12-07T10:22:29.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.198+0000 7f4d20f6fb80 10 client.26399 fetch_fsmap finished waiting for FSMap version 32 2023-12-07T10:22:29.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.198+0000 7f4d20f6fb80 10 client.26399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:22:29.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.198+0000 7f4d20f6fb80 20 client.26399 populate_metadata read hostname 'smithi119' 2023-12-07T10:22:29.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.198+0000 7f4d20f6fb80 1 --2- 172.21.15.119:0/237924524 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5614dd0d11d0 0x5614dd0d35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:29.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.198+0000 7f4d20f6fb80 4 client.26399 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:22:29.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.198+0000 7f4d20f6fb80 1 -- 172.21.15.119:0/237924524 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614dd0867a0 con 0x5614dd0d11d0 2023-12-07T10:22:29.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.199+0000 7f4cfe7fc700 1 --2- 172.21.15.119:0/237924524 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5614dd0d11d0 0x5614dd0d35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:29.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.199+0000 7f4cfe7fc700 1 --2- 172.21.15.119:0/237924524 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5614dd0d11d0 0x5614dd0d35f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66042d752079b3b5 server_cookie=23c4491d4827a15e in_seq=0 out_seq=0 2023-12-07T10:22:29.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.199+0000 7f4cee7fc700 10 client.26399 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:29.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.200+0000 7f4cee7fc700 1 -- 172.21.15.119:0/237924524 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5614dd0867a0 con 0x5614dd0d11d0 2023-12-07T10:22:29.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.200+0000 7f4cee7fc700 10 client.26399 handle_command_reply: tid=0 2023-12-07T10:22:29.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.200+0000 7f4d20f6fb80 1 client.26399 shutdown 2023-12-07T10:22:29.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.200+0000 7f4d20f6fb80 1 -- 172.21.15.119:0/237924524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dd0b7b40 msgr2=0x5614dd0c3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:29.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.200+0000 7f4d20f6fb80 1 --2- 172.21.15.119:0/237924524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dd0b7b40 0x5614dd0c3900 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8013780 tx=0x7f4cf8013a90 comp rx=0 tx=0).stop 2023-12-07T10:22:29.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.200+0000 7f4d20f6fb80 1 -- 172.21.15.119:0/237924524 shutdown_connections 2023-12-07T10:22:29.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.200+0000 7f4d20f6fb80 1 --2- 172.21.15.119:0/237924524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614dd0bc150 0x5614dd0c13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.200+0000 7f4d20f6fb80 1 --2- 172.21.15.119:0/237924524 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5614dd0d11d0 0x5614dd0d35f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.200+0000 7f4d20f6fb80 1 --2- 172.21.15.119:0/237924524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dd0b7b40 0x5614dd0c3900 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.200+0000 7f4cee7fc700 0 client.26399 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:29.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.200+0000 7f4d20f6fb80 1 --2- 172.21.15.119:0/237924524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614dd0b8460 0x5614dd0c0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.200+0000 7f4d20f6fb80 1 -- 172.21.15.119:0/237924524 >> 172.21.15.119:0/237924524 conn(0x5614dcfe6360 msgr2=0x5614dd0acd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:29.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.200+0000 7f4d20f6fb80 1 -- 172.21.15.119:0/237924524 shutdown_connections 2023-12-07T10:22:29.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.200+0000 7f4d20f6fb80 1 -- 172.21.15.119:0/237924524 wait complete. 2023-12-07T10:22:29.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.200+0000 7f4d20f6fb80 20 client.26399 trim_cache size 0 max 16384 2023-12-07T10:22:29.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.216+0000 7f4d20f6fb80 1 Processor -- start 2023-12-07T10:22:29.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.216+0000 7f4d20f6fb80 1 -- start start 2023-12-07T10:22:29.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.216+0000 7f4d20f6fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614dd0b7b40 0x5614dd0d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:29.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.216+0000 7f4d20f6fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614dd0b8460 0x5614dd0c35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:29.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.216+0000 7f4d20f6fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dd0bc150 0x5614dd0b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:29.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.217+0000 7f4d20f6fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614dce8a920 con 0x5614dd0b7b40 2023-12-07T10:22:29.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.217+0000 7f4d20f6fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614dd0d88b0 con 0x5614dd0bc150 2023-12-07T10:22:29.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.217+0000 7f4d20f6fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614dd0aeb70 con 0x5614dd0b8460 2023-12-07T10:22:29.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.217+0000 7f4d06ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614dd0b7b40 0x5614dd0d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:29.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.217+0000 7f4d06ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614dd0b7b40 0x5614dd0d35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45102/0 (socket says 172.21.15.119:45102) 2023-12-07T10:22:29.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.217+0000 7f4d077fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dd0bc150 0x5614dd0b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:29.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.217+0000 7f4d06ffd700 1 -- 172.21.15.119:0/1701345495 learned_addr learned my addr 172.21.15.119:0/1701345495 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:29.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.217+0000 7f4cfe7fc700 1 --2- 172.21.15.119:0/1701345495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614dd0b8460 0x5614dd0c35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:29.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.217+0000 7f4d06ffd700 1 -- 172.21.15.119:0/1701345495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614dd0b8460 msgr2=0x5614dd0c35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:29.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.217+0000 7f4d06ffd700 1 --2- 172.21.15.119:0/1701345495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614dd0b8460 0x5614dd0c35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.217+0000 7f4d06ffd700 1 -- 172.21.15.119:0/1701345495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dd0bc150 msgr2=0x5614dd0b9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:29.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.217+0000 7f4d06ffd700 1 --2- 172.21.15.119:0/1701345495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dd0bc150 0x5614dd0b9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.217+0000 7f4d06ffd700 1 -- 172.21.15.119:0/1701345495 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614dce1dc60 con 0x5614dd0b7b40 2023-12-07T10:22:29.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.217+0000 7f4d077fe700 1 --2- 172.21.15.119:0/1701345495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dd0bc150 0x5614dd0b9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:29.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.217+0000 7f4d06ffd700 1 --2- 172.21.15.119:0/1701345495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614dd0b7b40 0x5614dd0d35c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8001eb0 tx=0x7f4cf8009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=421aaf5f437598dd server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:29.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.218+0000 7f4cfdffb700 1 -- 172.21.15.119:0/1701345495 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8005ce0 con 0x5614dd0b7b40 2023-12-07T10:22:29.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.218+0000 7f4cfdffb700 1 -- 172.21.15.119:0/1701345495 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cf8013e90 con 0x5614dd0b7b40 2023-12-07T10:22:29.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.218+0000 7f4cfdffb700 1 -- 172.21.15.119:0/1701345495 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8011a50 con 0x5614dd0b7b40 2023-12-07T10:22:29.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.218+0000 7f4d20f6fb80 1 -- 172.21.15.119:0/1701345495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614dd0b7b40 msgr2=0x5614dd0d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:29.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.218+0000 7f4d20f6fb80 1 --2- 172.21.15.119:0/1701345495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614dd0b7b40 0x5614dd0d35c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8001eb0 tx=0x7f4cf8009960 comp rx=0 tx=0).stop 2023-12-07T10:22:29.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.218+0000 7f4d20f6fb80 1 -- 172.21.15.119:0/1701345495 shutdown_connections 2023-12-07T10:22:29.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.218+0000 7f4d20f6fb80 1 --2- 172.21.15.119:0/1701345495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dd0bc150 0x5614dd0b9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.218+0000 7f4d20f6fb80 1 --2- 172.21.15.119:0/1701345495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614dd0b7b40 0x5614dd0d35c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.218+0000 7f4d20f6fb80 1 --2- 172.21.15.119:0/1701345495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614dd0b8460 0x5614dd0c35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.218+0000 7f4d20f6fb80 1 -- 172.21.15.119:0/1701345495 >> 172.21.15.119:0/1701345495 conn(0x5614dcfe6360 msgr2=0x5614dd0ba7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:29.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.219+0000 7f4d20f6fb80 1 -- 172.21.15.119:0/1701345495 shutdown_connections 2023-12-07T10:22:29.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.219+0000 7f4d20f6fb80 1 -- 172.21.15.119:0/1701345495 wait complete. 2023-12-07T10:22:29.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.219+0000 7f4d20f6fb80 1 -- start start 2023-12-07T10:22:29.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.219+0000 7f4d20f6fb80 1 Processor -- start 2023-12-07T10:22:29.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.220+0000 7f4d20f6fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614dd0b7b40 0x5614dd0c42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:29.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.220+0000 7f4d20f6fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614dd0b8460 0x5614dd0c4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:29.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.220+0000 7f4d20f6fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dd0bc150 0x5614dd0c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:29.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.220+0000 7f4d20f6fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614dd0c8b90 con 0x5614dd0b7b40 2023-12-07T10:22:29.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.220+0000 7f4d20f6fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614dd12ef10 con 0x5614dd0bc150 2023-12-07T10:22:29.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.220+0000 7f4d20f6fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614dd12f050 con 0x5614dd0b8460 2023-12-07T10:22:29.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.220+0000 7f4d20f6fb80 10 client.0 _ll_register_callbacks cb 0x5614dcfac940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:22:29.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.220+0000 7f4d077fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dd0bc150 0x5614dd0c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:29.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.220+0000 7f4d077fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dd0bc150 0x5614dd0c4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42668/0 (socket says 172.21.15.119:42668) 2023-12-07T10:22:29.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.220+0000 7f4d077fe700 1 -- 172.21.15.119:0/2869185871 learned_addr learned my addr 172.21.15.119:0/2869185871 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:29.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.220+0000 7f4d06ffd700 1 --2- 172.21.15.119:0/2869185871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614dd0b7b40 0x5614dd0c42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:29.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.220+0000 7f4cfe7fc700 1 --2- 172.21.15.119:0/2869185871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614dd0b8460 0x5614dd0c4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:29.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.220+0000 7f4d077fe700 1 -- 172.21.15.119:0/2869185871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614dd0b8460 msgr2=0x5614dd0c4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:29.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.220+0000 7f4d077fe700 1 --2- 172.21.15.119:0/2869185871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614dd0b8460 0x5614dd0c4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.220+0000 7f4d077fe700 1 -- 172.21.15.119:0/2869185871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614dd0b7b40 msgr2=0x5614dd0c42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:29.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.220+0000 7f4d077fe700 1 --2- 172.21.15.119:0/2869185871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614dd0b7b40 0x5614dd0c42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.220+0000 7f4d077fe700 1 -- 172.21.15.119:0/2869185871 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614dce88630 con 0x5614dd0bc150 2023-12-07T10:22:29.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.221+0000 7f4d06ffd700 1 --2- 172.21.15.119:0/2869185871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614dd0b7b40 0x5614dd0c42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:29.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.221+0000 7f4cfe7fc700 1 --2- 172.21.15.119:0/2869185871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614dd0b8460 0x5614dd0c4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:29.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.221+0000 7f4d077fe700 1 --2- 172.21.15.119:0/2869185871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dd0bc150 0x5614dd0c4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4d0002bd50 tx=0x7f4d000061a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edb5e9a2cd27b816 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:29.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.221+0000 7f4cee7fc700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:22:29.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.221+0000 7f4cee7fc700 1 -- 172.21.15.119:0/2869185871 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0000e040 con 0x5614dd0bc150 2023-12-07T10:22:29.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.221+0000 7f4d20f6fb80 1 -- 172.21.15.119:0/2869185871 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d000076e0 con 0x5614dd0bc150 2023-12-07T10:22:29.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.221+0000 7f4cee7fc700 1 -- 172.21.15.119:0/2869185871 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4d00007850 con 0x5614dd0bc150 2023-12-07T10:22:29.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.221+0000 7f4cee7fc700 1 -- 172.21.15.119:0/2869185871 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d0002ba90 con 0x5614dd0bc150 2023-12-07T10:22:29.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.221+0000 7f4d20f6fb80 10 client.26405 fetch_fsmap learned FSMap version 32 2023-12-07T10:22:29.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.221+0000 7f4d20f6fb80 1 -- 172.21.15.119:0/2869185871 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5614dce34de0 con 0x5614dd0bc150 2023-12-07T10:22:29.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.222+0000 7f4cee7fc700 1 -- 172.21.15.119:0/2869185871 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d000076e0 con 0x5614dd0bc150 2023-12-07T10:22:29.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.222+0000 7f4cee7fc700 1 -- 172.21.15.119:0/2869185871 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f4d00040080 con 0x5614dd0bc150 2023-12-07T10:22:29.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.222+0000 7f4d20f6fb80 10 client.26405 fetch_fsmap finished waiting for FSMap version 32 2023-12-07T10:22:29.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.222+0000 7f4d20f6fb80 10 client.26405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:22:29.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.222+0000 7f4d20f6fb80 20 client.26405 populate_metadata read hostname 'smithi119' 2023-12-07T10:22:29.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.222+0000 7f4d20f6fb80 1 --2- 172.21.15.119:0/2869185871 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5614dd130490 0x5614dd1328b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:29.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.222+0000 7f4d20f6fb80 4 client.26405 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:22:29.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.222+0000 7f4d20f6fb80 1 -- 172.21.15.119:0/2869185871 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5614dcfd1fd0 con 0x5614dd130490 2023-12-07T10:22:29.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.222+0000 7f4d06ffd700 1 --2- 172.21.15.119:0/2869185871 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5614dd130490 0x5614dd1328b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:29.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.223+0000 7f4d06ffd700 1 --2- 172.21.15.119:0/2869185871 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5614dd130490 0x5614dd1328b0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b49a1f9312becad server_cookie=d1faac2af8932d4e in_seq=0 out_seq=0 2023-12-07T10:22:29.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.223+0000 7f4cee7fc700 10 client.26405 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:29.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.223+0000 7f4cee7fc700 1 -- 172.21.15.119:0/2869185871 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5614dcfd1fd0 con 0x5614dd130490 2023-12-07T10:22:29.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.223+0000 7f4cee7fc700 10 client.26405 handle_command_reply: tid=0 2023-12-07T10:22:29.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.223+0000 7f4d20f6fb80 1 client.26405 shutdown 2023-12-07T10:22:29.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.223+0000 7f4d20f6fb80 1 -- 172.21.15.119:0/2869185871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dd0bc150 msgr2=0x5614dd0c4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:29.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.223+0000 7f4d20f6fb80 1 --2- 172.21.15.119:0/2869185871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dd0bc150 0x5614dd0c4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4d0002bd50 tx=0x7f4d000061a0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.223+0000 7f4d20f6fb80 1 -- 172.21.15.119:0/2869185871 shutdown_connections 2023-12-07T10:22:29.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.223+0000 7f4d20f6fb80 1 --2- 172.21.15.119:0/2869185871 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5614dd130490 0x5614dd1328b0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.223+0000 7f4d20f6fb80 1 --2- 172.21.15.119:0/2869185871 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5614dd0bc150 0x5614dd0c4d40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.223+0000 7f4d20f6fb80 1 --2- 172.21.15.119:0/2869185871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5614dd0b7b40 0x5614dd0c42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.223+0000 7f4cee7fc700 0 client.26405 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:29.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.223+0000 7f4d20f6fb80 1 --2- 172.21.15.119:0/2869185871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614dd0b8460 0x5614dd0c4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.223+0000 7f4d20f6fb80 1 -- 172.21.15.119:0/2869185871 >> 172.21.15.119:0/2869185871 conn(0x5614dcfe6360 msgr2=0x5614dd0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:29.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.223+0000 7f4d20f6fb80 1 -- 172.21.15.119:0/2869185871 shutdown_connections 2023-12-07T10:22:29.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.223+0000 7f4d20f6fb80 1 -- 172.21.15.119:0/2869185871 wait complete. 2023-12-07T10:22:29.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.223+0000 7f4d20f6fb80 20 client.26405 trim_cache size 0 max 16384 2023-12-07T10:22:29.228 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:22:29.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.224+0000 7f4d0d5ff700 1 -- 172.21.15.119:0/3433884769 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4cf40a1d00 msgr2=0x7f4cf40a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:29.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.224+0000 7f4d0d5ff700 1 --2- 172.21.15.119:0/3433884769 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4cf40a1d00 0x7f4cf40a41c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4d00009040 tx=0x7f4d0000bc60 comp rx=0 tx=0).stop 2023-12-07T10:22:29.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.224+0000 7f4d0d5ff700 1 -- 172.21.15.119:0/3433884769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d08106110 msgr2=0x7f4d0819fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:29.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.224+0000 7f4d0d5ff700 1 --2- 172.21.15.119:0/3433884769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d08106110 0x7f4d0819fee0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000e9a0 tx=0x7f4cf000ecb0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.224+0000 7f4d0d5ff700 1 -- 172.21.15.119:0/3433884769 shutdown_connections 2023-12-07T10:22:29.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.224+0000 7f4d0d5ff700 1 --2- 172.21.15.119:0/3433884769 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4cf40a1d00 0x7f4cf40a41c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.224+0000 7f4d0d5ff700 1 --2- 172.21.15.119:0/3433884769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d08106ad0 0x7f4d08199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.224+0000 7f4d0d5ff700 1 --2- 172.21.15.119:0/3433884769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d08068a50 0x7f4d0819f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.224+0000 7f4d0d5ff700 1 --2- 172.21.15.119:0/3433884769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d08106110 0x7f4d0819fee0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:29.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.224+0000 7f4d0d5ff700 1 -- 172.21.15.119:0/3433884769 >> 172.21.15.119:0/3433884769 conn(0x7f4d08077610 msgr2=0x7f4d080fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:29.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.224+0000 7f4d0d5ff700 1 -- 172.21.15.119:0/3433884769 shutdown_connections 2023-12-07T10:22:29.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:29.224+0000 7f4d0d5ff700 1 -- 172.21.15.119:0/3433884769 wait complete. 2023-12-07T10:22:29.546 INFO:tasks.workunit.client.0.smithi119.stdout: 1 69755 9.31 MB/sec warmup 42 sec latency 1512.998 ms 2023-12-07T10:22:30.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:29 smithi119 ceph-mon[97908]: pgmap v65: 129 pgs: 129 active+clean; 52 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 45 KiB/s wr, 18 op/s 2023-12-07T10:22:30.246 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-07T10:22:30.246 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-07T10:22:30.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:29 smithi159 ceph-mon[134726]: pgmap v65: 129 pgs: 129 active+clean; 52 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 45 KiB/s wr, 18 op/s 2023-12-07T10:22:30.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:29 smithi153 ceph-mon[133625]: pgmap v65: 129 pgs: 129 active+clean; 52 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 45 KiB/s wr, 18 op/s 2023-12-07T10:22:30.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.460+0000 7f7db83ff700 1 -- 172.21.15.119:0/504897624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db0107610 msgr2=0x7f7db01079f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:30.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.460+0000 7f7db83ff700 1 --2- 172.21.15.119:0/504897624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db0107610 0x7f7db01079f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7dac0099c0 tx=0x7f7dac009cd0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.460+0000 7f7db83ff700 1 -- 172.21.15.119:0/504897624 shutdown_connections 2023-12-07T10:22:30.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.460+0000 7f7db83ff700 1 --2- 172.21.15.119:0/504897624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db0101f10 0x7f7db010e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.460+0000 7f7db83ff700 1 --2- 172.21.15.119:0/504897624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db0107610 0x7f7db01079f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.460+0000 7f7db83ff700 1 --2- 172.21.15.119:0/504897624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7db0101550 0x7f7db01019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.460+0000 7f7db83ff700 1 -- 172.21.15.119:0/504897624 >> 172.21.15.119:0/504897624 conn(0x7f7db00fd020 msgr2=0x7f7db00ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:30.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.460+0000 7f7db83ff700 1 -- 172.21.15.119:0/504897624 shutdown_connections 2023-12-07T10:22:30.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.461+0000 7f7db83ff700 1 -- 172.21.15.119:0/504897624 wait complete. 2023-12-07T10:22:30.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.461+0000 7f7db83ff700 1 Processor -- start 2023-12-07T10:22:30.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.462+0000 7f7db83ff700 1 -- start start 2023-12-07T10:22:30.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.462+0000 7f7db83ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7db0101550 0x7f7db019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.462+0000 7f7db83ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db0101f10 0x7f7db019fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.462+0000 7f7db83ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db0107610 0x7f7db0199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.462+0000 7f7db83ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db0199fe0 con 0x7f7db0101f10 2023-12-07T10:22:30.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.462+0000 7f7db83ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db019a150 con 0x7f7db0107610 2023-12-07T10:22:30.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.462+0000 7f7db83ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db019a310 con 0x7f7db0101550 2023-12-07T10:22:30.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.462+0000 7f7db619b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7db0101550 0x7f7db019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.462+0000 7f7db599a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db0101f10 0x7f7db019fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.462+0000 7f7db619b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7db0101550 0x7f7db019f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:46766/0 (socket says 172.21.15.119:46766) 2023-12-07T10:22:30.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.462+0000 7f7db619b700 1 -- 172.21.15.119:0/3122459569 learned_addr learned my addr 172.21.15.119:0/3122459569 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:30.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.462+0000 7f7db699c700 1 --2- 172.21.15.119:0/3122459569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db0107610 0x7f7db0199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.463+0000 7f7db619b700 1 -- 172.21.15.119:0/3122459569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db0107610 msgr2=0x7f7db0199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:30.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.463+0000 7f7db619b700 1 --2- 172.21.15.119:0/3122459569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db0107610 0x7f7db0199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.463+0000 7f7db619b700 1 -- 172.21.15.119:0/3122459569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db0101f10 msgr2=0x7f7db019fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:30.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.463+0000 7f7db619b700 1 --2- 172.21.15.119:0/3122459569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db0101f10 0x7f7db019fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.463+0000 7f7db619b700 1 -- 172.21.15.119:0/3122459569 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dac0095f0 con 0x7f7db0101550 2023-12-07T10:22:30.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.463+0000 7f7db599a700 1 --2- 172.21.15.119:0/3122459569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db0101f10 0x7f7db019fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:30.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.463+0000 7f7db619b700 1 --2- 172.21.15.119:0/3122459569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7db0101550 0x7f7db019f850 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00b9c0 tx=0x7f7dac009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:30.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.463+0000 7f7da77fe700 1 -- 172.21.15.119:0/3122459569 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac01d070 con 0x7f7db0101550 2023-12-07T10:22:30.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.463+0000 7f7db83ff700 1 -- 172.21.15.119:0/3122459569 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db019a590 con 0x7f7db0101550 2023-12-07T10:22:30.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.463+0000 7f7da77fe700 1 -- 172.21.15.119:0/3122459569 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dac00bdb0 con 0x7f7db0101550 2023-12-07T10:22:30.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.463+0000 7f7da77fe700 1 -- 172.21.15.119:0/3122459569 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac00f990 con 0x7f7db0101550 2023-12-07T10:22:30.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.464+0000 7f7db83ff700 1 -- 172.21.15.119:0/3122459569 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db01a6840 con 0x7f7db0101550 2023-12-07T10:22:30.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.467+0000 7f7da77fe700 1 -- 172.21.15.119:0/3122459569 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f7dac003720 con 0x7f7db0101550 2023-12-07T10:22:30.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.468+0000 7f7da77fe700 1 --2- 172.21.15.119:0/3122459569 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f7d9c0a1c80 0x7f7d9c0a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.469+0000 7f7dcbc87b80 1 Processor -- start 2023-12-07T10:22:30.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.469+0000 7f7da77fe700 1 -- 172.21.15.119:0/3122459569 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f7dac0ce9d0 con 0x7f7db0101550 2023-12-07T10:22:30.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.469+0000 7f7dcbc87b80 1 -- start start 2023-12-07T10:22:30.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.469+0000 7f7db599a700 1 --2- 172.21.15.119:0/3122459569 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f7d9c0a1c80 0x7f7d9c0a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.469+0000 7f7dcbc87b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aab2e763d0 0x55aab2e767b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.469+0000 7f7dcbc87b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aab2e76cf0 0x55aab2e7a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.469+0000 7f7dcbc87b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab2e7a9e0 0x55aab2e7ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.469+0000 7f7dcbc87b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab2de1840 con 0x55aab2e7a9e0 2023-12-07T10:22:30.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.469+0000 7f7dcbc87b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab2dc8820 con 0x55aab2e763d0 2023-12-07T10:22:30.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.469+0000 7f7dcbc87b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab2dc6140 con 0x55aab2e76cf0 2023-12-07T10:22:30.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.469+0000 7f7db619b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aab2e763d0 0x55aab2e767b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.470+0000 7f7db619b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aab2e763d0 0x55aab2e767b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42692/0 (socket says 172.21.15.119:42692) 2023-12-07T10:22:30.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.470+0000 7f7db619b700 1 -- 172.21.15.119:0/2818299041 learned_addr learned my addr 172.21.15.119:0/2818299041 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:30.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.470+0000 7f7db699c700 1 --2- 172.21.15.119:0/2818299041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab2e7a9e0 0x55aab2e7ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.470+0000 7f7db599a700 1 --2- 172.21.15.119:0/2818299041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aab2e76cf0 0x55aab2e7a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.470+0000 7f7db599a700 1 --2- 172.21.15.119:0/3122459569 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f7d9c0a1c80 0x7f7d9c0a4140 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7db019ab40 tx=0x7f7da0009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:30.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.470+0000 7f7db619b700 1 -- 172.21.15.119:0/2818299041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aab2e76cf0 msgr2=0x55aab2e7a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:30.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.470+0000 7f7db619b700 1 --2- 172.21.15.119:0/2818299041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aab2e76cf0 0x55aab2e7a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.470+0000 7f7db619b700 1 -- 172.21.15.119:0/2818299041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab2e7a9e0 msgr2=0x55aab2e7ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:30.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.470+0000 7f7db619b700 1 --2- 172.21.15.119:0/2818299041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab2e7a9e0 0x55aab2e7ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.470+0000 7f7db619b700 1 -- 172.21.15.119:0/2818299041 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aab2c3e6b0 con 0x55aab2e763d0 2023-12-07T10:22:30.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.470+0000 7f7db619b700 1 --2- 172.21.15.119:0/2818299041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aab2e763d0 0x55aab2e767b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7dac097ec0 tx=0x7f7dac09b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6aae9cb730aca792 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:30.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.470+0000 7f7da4bfe700 1 -- 172.21.15.119:0/2818299041 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac09f5a0 con 0x55aab2e763d0 2023-12-07T10:22:30.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.470+0000 7f7da4bfe700 1 -- 172.21.15.119:0/2818299041 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dac0a3750 con 0x55aab2e763d0 2023-12-07T10:22:30.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.470+0000 7f7da4bfe700 1 -- 172.21.15.119:0/2818299041 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac09a940 con 0x55aab2e763d0 2023-12-07T10:22:30.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.471+0000 7f7dcbc87b80 1 -- 172.21.15.119:0/2818299041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aab2e763d0 msgr2=0x55aab2e767b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:30.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.471+0000 7f7dcbc87b80 1 --2- 172.21.15.119:0/2818299041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aab2e763d0 0x55aab2e767b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7dac097ec0 tx=0x7f7dac09b330 comp rx=0 tx=0).stop 2023-12-07T10:22:30.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.471+0000 7f7dcbc87b80 1 -- 172.21.15.119:0/2818299041 shutdown_connections 2023-12-07T10:22:30.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.471+0000 7f7dcbc87b80 1 --2- 172.21.15.119:0/2818299041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab2e7a9e0 0x55aab2e7ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.471+0000 7f7dcbc87b80 1 --2- 172.21.15.119:0/2818299041 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aab2e763d0 0x55aab2e767b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.471+0000 7f7dcbc87b80 1 --2- 172.21.15.119:0/2818299041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aab2e76cf0 0x55aab2e7a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.471+0000 7f7dcbc87b80 1 -- 172.21.15.119:0/2818299041 >> 172.21.15.119:0/2818299041 conn(0x55aab2d9c3e0 msgr2=0x55aab2e681b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:30.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.471+0000 7f7dcbc87b80 1 -- 172.21.15.119:0/2818299041 shutdown_connections 2023-12-07T10:22:30.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.471+0000 7f7dcbc87b80 1 -- 172.21.15.119:0/2818299041 wait complete. 2023-12-07T10:22:30.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.472+0000 7f7dcbc87b80 1 -- start start 2023-12-07T10:22:30.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.472+0000 7f7dcbc87b80 1 Processor -- start 2023-12-07T10:22:30.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.473+0000 7f7dcbc87b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab2e763d0 0x55aab2e82170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.473+0000 7f7dcbc87b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aab2e76cf0 0x55aab2e7f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.473+0000 7f7dcbc87b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aab2e7a9e0 0x55aab2e7fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.473+0000 7f7dcbc87b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab2dc1a30 con 0x55aab2e763d0 2023-12-07T10:22:30.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.473+0000 7f7dcbc87b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab2d7b5e0 con 0x55aab2e7a9e0 2023-12-07T10:22:30.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.473+0000 7f7dcbc87b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab2d7b280 con 0x55aab2e76cf0 2023-12-07T10:22:30.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.473+0000 7f7dcbc87b80 10 client.0 _ll_register_callbacks cb 0x55aab2d629c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:22:30.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.473+0000 7f7db599a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aab2e76cf0 0x55aab2e7f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.473+0000 7f7db619b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab2e763d0 0x55aab2e82170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.473+0000 7f7db599a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aab2e76cf0 0x55aab2e7f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:46788/0 (socket says 172.21.15.119:46788) 2023-12-07T10:22:30.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.473+0000 7f7db599a700 1 -- 172.21.15.119:0/2873942266 learned_addr learned my addr 172.21.15.119:0/2873942266 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:30.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.473+0000 7f7db699c700 1 --2- 172.21.15.119:0/2873942266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aab2e7a9e0 0x55aab2e7fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.473+0000 7f7db599a700 1 -- 172.21.15.119:0/2873942266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aab2e7a9e0 msgr2=0x55aab2e7fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:30.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.473+0000 7f7db599a700 1 --2- 172.21.15.119:0/2873942266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aab2e7a9e0 0x55aab2e7fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.473+0000 7f7db599a700 1 -- 172.21.15.119:0/2873942266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab2e763d0 msgr2=0x55aab2e82170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:30.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.473+0000 7f7db599a700 1 --2- 172.21.15.119:0/2873942266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab2e763d0 0x55aab2e82170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.473+0000 7f7db599a700 1 -- 172.21.15.119:0/2873942266 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db01a6840 con 0x55aab2e76cf0 2023-12-07T10:22:30.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.474+0000 7f7db599a700 1 --2- 172.21.15.119:0/2873942266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aab2e76cf0 0x55aab2e7f3f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009fd0 tx=0x7f7da00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34a1d11504c4e85b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:30.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.474+0000 7f7d99ffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:22:30.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.474+0000 7f7d99ffb700 1 -- 172.21.15.119:0/2873942266 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da0021070 con 0x55aab2e76cf0 2023-12-07T10:22:30.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.474+0000 7f7d99ffb700 1 -- 172.21.15.119:0/2873942266 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f7da001c440 con 0x55aab2e76cf0 2023-12-07T10:22:30.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.474+0000 7f7dcbc87b80 1 -- 172.21.15.119:0/2873942266 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aab2e60b80 con 0x55aab2e76cf0 2023-12-07T10:22:30.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.474+0000 7f7d99ffb700 1 -- 172.21.15.119:0/2873942266 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7da001c600 con 0x55aab2e76cf0 2023-12-07T10:22:30.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.474+0000 7f7dcbc87b80 10 client.26407 fetch_fsmap learned FSMap version 32 2023-12-07T10:22:30.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.474+0000 7f7dcbc87b80 1 -- 172.21.15.119:0/2873942266 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55aab2beae60 con 0x55aab2e76cf0 2023-12-07T10:22:30.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.474+0000 7f7d99ffb700 1 -- 172.21.15.119:0/2873942266 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aab2e60b80 con 0x55aab2e76cf0 2023-12-07T10:22:30.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.475+0000 7f7d99ffb700 1 -- 172.21.15.119:0/2873942266 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f7da001b9d0 con 0x55aab2e76cf0 2023-12-07T10:22:30.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.475+0000 7f7dcbc87b80 10 client.26407 fetch_fsmap finished waiting for FSMap version 32 2023-12-07T10:22:30.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.475+0000 7f7dcbc87b80 10 client.26407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:22:30.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.475+0000 7f7dcbc87b80 20 client.26407 populate_metadata read hostname 'smithi119' 2023-12-07T10:22:30.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.475+0000 7f7dcbc87b80 1 --2- 172.21.15.119:0/2873942266 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55aab2e8f850 0x55aab2e91c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.475+0000 7f7dcbc87b80 4 client.26407 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:22:30.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.475+0000 7f7dcbc87b80 1 -- 172.21.15.119:0/2873942266 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aab2e3c820 con 0x55aab2e8f850 2023-12-07T10:22:30.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.475+0000 7f7db619b700 1 --2- 172.21.15.119:0/2873942266 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55aab2e8f850 0x55aab2e91c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.475+0000 7f7d99ffb700 10 client.26407 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:30.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.475+0000 7f7db619b700 1 --2- 172.21.15.119:0/2873942266 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55aab2e8f850 0x55aab2e91c70 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81dbb920f94382e server_cookie=1e2585c36a4c1596 in_seq=0 out_seq=0 2023-12-07T10:22:30.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.476+0000 7f7d99ffb700 1 -- 172.21.15.119:0/2873942266 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aab2e3c820 con 0x55aab2e8f850 2023-12-07T10:22:30.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.476+0000 7f7d99ffb700 10 client.26407 handle_command_reply: tid=0 2023-12-07T10:22:30.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.476+0000 7f7dcbc87b80 1 client.26407 shutdown 2023-12-07T10:22:30.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.476+0000 7f7dcbc87b80 1 -- 172.21.15.119:0/2873942266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aab2e76cf0 msgr2=0x55aab2e7f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:30.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.476+0000 7f7dcbc87b80 1 --2- 172.21.15.119:0/2873942266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aab2e76cf0 0x55aab2e7f3f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009fd0 tx=0x7f7da00047c0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.476+0000 7f7dcbc87b80 1 -- 172.21.15.119:0/2873942266 shutdown_connections 2023-12-07T10:22:30.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.476+0000 7f7dcbc87b80 1 --2- 172.21.15.119:0/2873942266 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55aab2e8f850 0x55aab2e91c70 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.476+0000 7f7dcbc87b80 1 --2- 172.21.15.119:0/2873942266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aab2e7a9e0 0x55aab2e7fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.476+0000 7f7dcbc87b80 1 --2- 172.21.15.119:0/2873942266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab2e763d0 0x55aab2e82170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.476+0000 7f7dcbc87b80 1 --2- 172.21.15.119:0/2873942266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aab2e76cf0 0x55aab2e7f3f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.476+0000 7f7d99ffb700 0 client.26407 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:30.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.476+0000 7f7dcbc87b80 1 -- 172.21.15.119:0/2873942266 >> 172.21.15.119:0/2873942266 conn(0x55aab2d9c3e0 msgr2=0x55aab2e62da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:30.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.476+0000 7f7dcbc87b80 1 -- 172.21.15.119:0/2873942266 shutdown_connections 2023-12-07T10:22:30.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.477+0000 7f7dcbc87b80 1 -- 172.21.15.119:0/2873942266 wait complete. 2023-12-07T10:22:30.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.477+0000 7f7dcbc87b80 20 client.26407 trim_cache size 0 max 16384 2023-12-07T10:22:30.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.497+0000 7f7dcbc87b80 1 Processor -- start 2023-12-07T10:22:30.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.497+0000 7f7dcbc87b80 1 -- start start 2023-12-07T10:22:30.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.497+0000 7f7dcbc87b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab2e763d0 0x55aab2e7f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.497+0000 7f7dcbc87b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aab2e76cf0 0x55aab2e78410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.497+0000 7f7dcbc87b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aab2e7a9e0 0x55aab2e78950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.497+0000 7f7dcbc87b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab2c409a0 con 0x55aab2e763d0 2023-12-07T10:22:30.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.497+0000 7f7dcbc87b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab2e80a90 con 0x55aab2e7a9e0 2023-12-07T10:22:30.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.497+0000 7f7dcbc87b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab2e80340 con 0x55aab2e76cf0 2023-12-07T10:22:30.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.498+0000 7f7db699c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aab2e7a9e0 0x55aab2e78950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.498+0000 7f7db619b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab2e763d0 0x55aab2e7f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.498+0000 7f7db699c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aab2e7a9e0 0x55aab2e78950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42710/0 (socket says 172.21.15.119:42710) 2023-12-07T10:22:30.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.498+0000 7f7db699c700 1 -- 172.21.15.119:0/2178866531 learned_addr learned my addr 172.21.15.119:0/2178866531 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:30.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.498+0000 7f7db599a700 1 --2- 172.21.15.119:0/2178866531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aab2e76cf0 0x55aab2e78410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.498+0000 7f7db699c700 1 -- 172.21.15.119:0/2178866531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aab2e76cf0 msgr2=0x55aab2e78410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:30.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.498+0000 7f7db699c700 1 --2- 172.21.15.119:0/2178866531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aab2e76cf0 0x55aab2e78410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.498+0000 7f7db699c700 1 -- 172.21.15.119:0/2178866531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab2e763d0 msgr2=0x55aab2e7f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:30.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.498+0000 7f7db699c700 1 --2- 172.21.15.119:0/2178866531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab2e763d0 0x55aab2e7f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.498+0000 7f7db699c700 1 -- 172.21.15.119:0/2178866531 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aab2bd3ce0 con 0x55aab2e7a9e0 2023-12-07T10:22:30.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.498+0000 7f7db599a700 1 --2- 172.21.15.119:0/2178866531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aab2e76cf0 0x55aab2e78410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:30.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.498+0000 7f7db619b700 1 --2- 172.21.15.119:0/2178866531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab2e763d0 0x55aab2e7f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:30.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.498+0000 7f7db699c700 1 --2- 172.21.15.119:0/2178866531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aab2e7a9e0 0x55aab2e78950 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7da80119d0 tx=0x7f7da8009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afab04bfcae060e2 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:30.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.499+0000 7f7da53ff700 1 -- 172.21.15.119:0/2178866531 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da800e040 con 0x55aab2e7a9e0 2023-12-07T10:22:30.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.499+0000 7f7da53ff700 1 -- 172.21.15.119:0/2178866531 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7da800f040 con 0x55aab2e7a9e0 2023-12-07T10:22:30.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.499+0000 7f7da53ff700 1 -- 172.21.15.119:0/2178866531 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da8007af0 con 0x55aab2e7a9e0 2023-12-07T10:22:30.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.499+0000 7f7dcbc87b80 1 -- 172.21.15.119:0/2178866531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aab2e7a9e0 msgr2=0x55aab2e78950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:30.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.499+0000 7f7dcbc87b80 1 --2- 172.21.15.119:0/2178866531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aab2e7a9e0 0x55aab2e78950 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7da80119d0 tx=0x7f7da8009920 comp rx=0 tx=0).stop 2023-12-07T10:22:30.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.500+0000 7f7dcbc87b80 1 -- 172.21.15.119:0/2178866531 shutdown_connections 2023-12-07T10:22:30.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.500+0000 7f7dcbc87b80 1 --2- 172.21.15.119:0/2178866531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aab2e7a9e0 0x55aab2e78950 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.500+0000 7f7dcbc87b80 1 --2- 172.21.15.119:0/2178866531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab2e763d0 0x55aab2e7f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.500+0000 7f7dcbc87b80 1 --2- 172.21.15.119:0/2178866531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aab2e76cf0 0x55aab2e78410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.500+0000 7f7dcbc87b80 1 -- 172.21.15.119:0/2178866531 >> 172.21.15.119:0/2178866531 conn(0x55aab2d9c3e0 msgr2=0x55aab2e7ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:30.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.500+0000 7f7dcbc87b80 1 -- 172.21.15.119:0/2178866531 shutdown_connections 2023-12-07T10:22:30.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.500+0000 7f7dcbc87b80 1 -- 172.21.15.119:0/2178866531 wait complete. 2023-12-07T10:22:30.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.500+0000 7f7dcbc87b80 1 -- start start 2023-12-07T10:22:30.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.500+0000 7f7dcbc87b80 1 Processor -- start 2023-12-07T10:22:30.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.501+0000 7f7dcbc87b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aab2e763d0 0x55aab2e84820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.501+0000 7f7dcbc87b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab2e76cf0 0x55aab2e84d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.501+0000 7f7dcbc87b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aab2e7a9e0 0x55aab2ee6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.501+0000 7f7dcbc87b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab2e854d0 con 0x55aab2e76cf0 2023-12-07T10:22:30.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.501+0000 7f7dcbc87b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab2e85640 con 0x55aab2e763d0 2023-12-07T10:22:30.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.501+0000 7f7dcbc87b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab2ee72a0 con 0x55aab2e7a9e0 2023-12-07T10:22:30.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.501+0000 7f7dcbc87b80 10 client.0 _ll_register_callbacks cb 0x55aab2d629c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:22:30.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.501+0000 7f7db619b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aab2e763d0 0x55aab2e84820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.501+0000 7f7db619b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aab2e763d0 0x55aab2e84820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42722/0 (socket says 172.21.15.119:42722) 2023-12-07T10:22:30.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.501+0000 7f7db619b700 1 -- 172.21.15.119:0/3955985978 learned_addr learned my addr 172.21.15.119:0/3955985978 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:30.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.501+0000 7f7db699c700 1 --2- 172.21.15.119:0/3955985978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aab2e7a9e0 0x55aab2ee6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.501+0000 7f7db599a700 1 --2- 172.21.15.119:0/3955985978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab2e76cf0 0x55aab2e84d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.501+0000 7f7db619b700 1 -- 172.21.15.119:0/3955985978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aab2e7a9e0 msgr2=0x55aab2ee6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:30.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.501+0000 7f7db619b700 1 --2- 172.21.15.119:0/3955985978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aab2e7a9e0 0x55aab2ee6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.501+0000 7f7db619b700 1 -- 172.21.15.119:0/3955985978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab2e76cf0 msgr2=0x55aab2e84d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:30.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.501+0000 7f7db619b700 1 --2- 172.21.15.119:0/3955985978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab2e76cf0 0x55aab2e84d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.501+0000 7f7db619b700 1 -- 172.21.15.119:0/3955985978 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aab2c3e6b0 con 0x55aab2e763d0 2023-12-07T10:22:30.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.501+0000 7f7db599a700 1 --2- 172.21.15.119:0/3955985978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab2e76cf0 0x55aab2e84d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:22:30.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.502+0000 7f7db619b700 1 --2- 172.21.15.119:0/3955985978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aab2e763d0 0x55aab2e84820 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7dac09b3e0 tx=0x7f7dac097160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e53a2b8c071cf7bc server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:30.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.502+0000 7f7d99ffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:22:30.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.502+0000 7f7d99ffb700 1 -- 172.21.15.119:0/3955985978 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac09f330 con 0x55aab2e763d0 2023-12-07T10:22:30.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.502+0000 7f7d99ffb700 1 -- 172.21.15.119:0/3955985978 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f7dac09a940 con 0x55aab2e763d0 2023-12-07T10:22:30.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.502+0000 7f7dcbc87b80 1 -- 172.21.15.119:0/3955985978 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7dac0a3750 con 0x55aab2e763d0 2023-12-07T10:22:30.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.502+0000 7f7d99ffb700 1 -- 172.21.15.119:0/3955985978 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dac07c070 con 0x55aab2e763d0 2023-12-07T10:22:30.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.502+0000 7f7dcbc87b80 10 client.26423 fetch_fsmap learned FSMap version 32 2023-12-07T10:22:30.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.502+0000 7f7dcbc87b80 1 -- 172.21.15.119:0/3955985978 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55aab2beae60 con 0x55aab2e763d0 2023-12-07T10:22:30.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.502+0000 7f7d99ffb700 1 -- 172.21.15.119:0/3955985978 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dac0a3750 con 0x55aab2e763d0 2023-12-07T10:22:30.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.502+0000 7f7d99ffb700 1 -- 172.21.15.119:0/3955985978 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f7dac0cbff0 con 0x55aab2e763d0 2023-12-07T10:22:30.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.502+0000 7f7dcbc87b80 10 client.26423 fetch_fsmap finished waiting for FSMap version 32 2023-12-07T10:22:30.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.502+0000 7f7dcbc87b80 10 client.26423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:22:30.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.502+0000 7f7dcbc87b80 20 client.26423 populate_metadata read hostname 'smithi119' 2023-12-07T10:22:30.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.502+0000 7f7dcbc87b80 1 --2- 172.21.15.119:0/3955985978 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55aab2ee86a0 0x55aab2eeaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.502+0000 7f7dcbc87b80 4 client.26423 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-07T10:22:30.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.502+0000 7f7dcbc87b80 1 -- 172.21.15.119:0/3955985978 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55aab2d88050 con 0x55aab2ee86a0 2023-12-07T10:22:30.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.503+0000 7f7db599a700 1 --2- 172.21.15.119:0/3955985978 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55aab2ee86a0 0x55aab2eeaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.503+0000 7f7db599a700 1 --2- 172.21.15.119:0/3955985978 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55aab2ee86a0 0x55aab2eeaa90 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84824c73facbc8ff server_cookie=b61aa6bb4708ecaf in_seq=0 out_seq=0 2023-12-07T10:22:30.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.503+0000 7f7d99ffb700 10 client.26423 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:30.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.504+0000 7f7d99ffb700 1 -- 172.21.15.119:0/3955985978 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55aab2d88050 con 0x55aab2ee86a0 2023-12-07T10:22:30.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.504+0000 7f7d99ffb700 10 client.26423 handle_command_reply: tid=0 2023-12-07T10:22:30.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.504+0000 7f7dcbc87b80 1 client.26423 shutdown 2023-12-07T10:22:30.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.504+0000 7f7dcbc87b80 1 -- 172.21.15.119:0/3955985978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aab2e763d0 msgr2=0x55aab2e84820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:30.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.504+0000 7f7dcbc87b80 1 --2- 172.21.15.119:0/3955985978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aab2e763d0 0x55aab2e84820 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7dac09b3e0 tx=0x7f7dac097160 comp rx=0 tx=0).stop 2023-12-07T10:22:30.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.504+0000 7f7dcbc87b80 1 -- 172.21.15.119:0/3955985978 shutdown_connections 2023-12-07T10:22:30.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.504+0000 7f7dcbc87b80 1 --2- 172.21.15.119:0/3955985978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aab2e7a9e0 0x55aab2ee6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.504+0000 7f7dcbc87b80 1 --2- 172.21.15.119:0/3955985978 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55aab2ee86a0 0x55aab2eeaa90 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.504+0000 7f7dcbc87b80 1 --2- 172.21.15.119:0/3955985978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55aab2e763d0 0x55aab2e84820 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.504+0000 7f7dcbc87b80 1 --2- 172.21.15.119:0/3955985978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab2e76cf0 0x55aab2e84d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.504+0000 7f7dcbc87b80 1 -- 172.21.15.119:0/3955985978 >> 172.21.15.119:0/3955985978 conn(0x55aab2d9c3e0 msgr2=0x55aab2e7dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:30.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.504+0000 7f7d99ffb700 0 client.26423 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:30.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.505+0000 7f7dcbc87b80 1 -- 172.21.15.119:0/3955985978 shutdown_connections 2023-12-07T10:22:30.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.505+0000 7f7dcbc87b80 1 -- 172.21.15.119:0/3955985978 wait complete. 2023-12-07T10:22:30.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.505+0000 7f7dcbc87b80 20 client.26423 trim_cache size 0 max 16384 2023-12-07T10:22:30.512 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:22:30.512 INFO:teuthology.orchestra.run.smithi119.stdout: "return_code": 0, 2023-12-07T10:22:30.512 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_tag": "7486ba01-e158-40fd-9e83-d5b9b52c671e", 2023-12-07T10:22:30.512 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": "asynchronous" 2023-12-07T10:22:30.512 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:22:30.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.505+0000 7f7db83ff700 1 -- 172.21.15.119:0/3122459569 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f7d9c0a1c80 msgr2=0x7f7d9c0a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:30.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.505+0000 7f7db83ff700 1 --2- 172.21.15.119:0/3122459569 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f7d9c0a1c80 0x7f7d9c0a4140 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7db019ab40 tx=0x7f7da0009260 comp rx=0 tx=0).stop 2023-12-07T10:22:30.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.505+0000 7f7db83ff700 1 -- 172.21.15.119:0/3122459569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7db0101550 msgr2=0x7f7db019f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:30.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.505+0000 7f7db83ff700 1 --2- 172.21.15.119:0/3122459569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7db0101550 0x7f7db019f850 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00b9c0 tx=0x7f7dac009580 comp rx=0 tx=0).stop 2023-12-07T10:22:30.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.506+0000 7f7db83ff700 1 -- 172.21.15.119:0/3122459569 shutdown_connections 2023-12-07T10:22:30.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.506+0000 7f7db83ff700 1 --2- 172.21.15.119:0/3122459569 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f7d9c0a1c80 0x7f7d9c0a4140 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.506+0000 7f7db83ff700 1 --2- 172.21.15.119:0/3122459569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db0107610 0x7f7db0199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.506+0000 7f7db83ff700 1 --2- 172.21.15.119:0/3122459569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7db0101550 0x7f7db019f850 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.506+0000 7f7db83ff700 1 --2- 172.21.15.119:0/3122459569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db0101f10 0x7f7db019fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.506+0000 7f7db83ff700 1 -- 172.21.15.119:0/3122459569 >> 172.21.15.119:0/3122459569 conn(0x7f7db00fd020 msgr2=0x7f7db00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:30.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.506+0000 7f7db83ff700 1 -- 172.21.15.119:0/3122459569 shutdown_connections 2023-12-07T10:22:30.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.506+0000 7f7db83ff700 1 -- 172.21.15.119:0/3122459569 wait complete. 2023-12-07T10:22:30.532 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-07T10:22:30.545 INFO:tasks.workunit.client.0.smithi119.stdout: 1 71360 9.38 MB/sec warmup 43 sec latency 25.739 ms 2023-12-07T10:22:30.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.742+0000 7f9f71fff700 1 -- 172.21.15.119:0/2993296969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f6c107610 msgr2=0x7f9f6c1079f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:30.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.742+0000 7f9f71fff700 1 --2- 172.21.15.119:0/2993296969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f6c107610 0x7f9f6c1079f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c0099c0 tx=0x7f9f5c009cd0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.742+0000 7f9f71fff700 1 -- 172.21.15.119:0/2993296969 shutdown_connections 2023-12-07T10:22:30.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.742+0000 7f9f71fff700 1 --2- 172.21.15.119:0/2993296969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f6c101f10 0x7f9f6c10e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.742+0000 7f9f71fff700 1 --2- 172.21.15.119:0/2993296969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f6c107610 0x7f9f6c1079f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.742+0000 7f9f71fff700 1 --2- 172.21.15.119:0/2993296969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f6c101550 0x7f9f6c1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.742+0000 7f9f71fff700 1 -- 172.21.15.119:0/2993296969 >> 172.21.15.119:0/2993296969 conn(0x7f9f6c0fd020 msgr2=0x7f9f6c0ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:30.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.742+0000 7f9f71fff700 1 -- 172.21.15.119:0/2993296969 shutdown_connections 2023-12-07T10:22:30.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.742+0000 7f9f71fff700 1 -- 172.21.15.119:0/2993296969 wait complete. 2023-12-07T10:22:30.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.743+0000 7f9f71fff700 1 Processor -- start 2023-12-07T10:22:30.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.743+0000 7f9f71fff700 1 -- start start 2023-12-07T10:22:30.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.744+0000 7f9f71fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f6c101550 0x7f9f6c1083e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.744+0000 7f9f71fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f6c101f10 0x7f9f6c108920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.744+0000 7f9f71fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f6c107610 0x7f9f6c108e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.744+0000 7f9f71fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6c1aa1a0 con 0x7f9f6c107610 2023-12-07T10:22:30.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.744+0000 7f9f71fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6c1aa2e0 con 0x7f9f6c101550 2023-12-07T10:22:30.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.744+0000 7f9f71fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6c1aa450 con 0x7f9f6c101f10 2023-12-07T10:22:30.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.744+0000 7f9f6b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f6c101550 0x7f9f6c1083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.744+0000 7f9f6b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f6c101550 0x7f9f6c1083e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42732/0 (socket says 172.21.15.119:42732) 2023-12-07T10:22:30.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.744+0000 7f9f6b7fe700 1 -- 172.21.15.119:0/901611174 learned_addr learned my addr 172.21.15.119:0/901611174 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:30.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.744+0000 7f9f6bfff700 1 --2- 172.21.15.119:0/901611174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f6c107610 0x7f9f6c108e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.744+0000 7f9f6affd700 1 --2- 172.21.15.119:0/901611174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f6c101f10 0x7f9f6c108920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.744+0000 7f9f6b7fe700 1 -- 172.21.15.119:0/901611174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f6c101f10 msgr2=0x7f9f6c108920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:30.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.744+0000 7f9f6b7fe700 1 --2- 172.21.15.119:0/901611174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f6c101f10 0x7f9f6c108920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.744+0000 7f9f6b7fe700 1 -- 172.21.15.119:0/901611174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f6c107610 msgr2=0x7f9f6c108e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:30.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.744+0000 7f9f6b7fe700 1 --2- 172.21.15.119:0/901611174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f6c107610 0x7f9f6c108e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.744+0000 7f9f6b7fe700 1 -- 172.21.15.119:0/901611174 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f60009520 con 0x7f9f6c101550 2023-12-07T10:22:30.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.745+0000 7f9f6b7fe700 1 --2- 172.21.15.119:0/901611174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f6c101550 0x7f9f6c1083e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c005b40 tx=0x7f9f5c00b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:30.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.745+0000 7f9f68ff9700 1 -- 172.21.15.119:0/901611174 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c01d070 con 0x7f9f6c101550 2023-12-07T10:22:30.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.745+0000 7f9f68ff9700 1 -- 172.21.15.119:0/901611174 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f5c00be00 con 0x7f9f6c101550 2023-12-07T10:22:30.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.745+0000 7f9f71fff700 1 -- 172.21.15.119:0/901611174 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5c0095f0 con 0x7f9f6c101550 2023-12-07T10:22:30.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.745+0000 7f9f68ff9700 1 -- 172.21.15.119:0/901611174 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c00fb00 con 0x7f9f6c101550 2023-12-07T10:22:30.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.745+0000 7f9f71fff700 1 -- 172.21.15.119:0/901611174 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f6c1aa9c0 con 0x7f9f6c101550 2023-12-07T10:22:30.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.748+0000 7f9f68ff9700 1 -- 172.21.15.119:0/901611174 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f9f5c02b440 con 0x7f9f6c101550 2023-12-07T10:22:30.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.750+0000 7f9f8592ab80 1 Processor -- start 2023-12-07T10:22:30.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.750+0000 7f9f8592ab80 1 -- start start 2023-12-07T10:22:30.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.750+0000 7f9f8592ab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644170b8b40 0x5644170b8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.751+0000 7f9f8592ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644170b9460 0x5644170bcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.751+0000 7f9f8592ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644170bd150 0x5644170bd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.751+0000 7f9f68ff9700 1 --2- 172.21.15.119:0/901611174 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9f540a1d00 0x7f9f540a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.751+0000 7f9f8592ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56441702c7c0 con 0x5644170b9460 2023-12-07T10:22:30.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.751+0000 7f9f8592ab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644170137a0 con 0x5644170b8b40 2023-12-07T10:22:30.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.751+0000 7f9f8592ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644170110c0 con 0x5644170bd150 2023-12-07T10:22:30.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.751+0000 7f9f68ff9700 1 -- 172.21.15.119:0/901611174 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9f5c0ce910 con 0x7f9f6c101550 2023-12-07T10:22:30.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.751+0000 7f9f6b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644170b9460 0x5644170bcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.751+0000 7f9f6affd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644170bd150 0x5644170bd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.751+0000 7f9f6bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644170b8b40 0x5644170b8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.751+0000 7f9f6b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644170b9460 0x5644170bcc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45166/0 (socket says 172.21.15.119:45166) 2023-12-07T10:22:30.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.751+0000 7f9f6b7fe700 1 -- 172.21.15.119:0/1622117883 learned_addr learned my addr 172.21.15.119:0/1622117883 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:30.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.751+0000 7f9f6bfff700 1 --2- 172.21.15.119:0/901611174 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9f540a1d00 0x7f9f540a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.751+0000 7f9f6affd700 1 -- 172.21.15.119:0/1622117883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644170b8b40 msgr2=0x5644170b8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:30.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.751+0000 7f9f6affd700 1 --2- 172.21.15.119:0/1622117883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644170b8b40 0x5644170b8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.751+0000 7f9f6affd700 1 -- 172.21.15.119:0/1622117883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644170b9460 msgr2=0x5644170bcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:30.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.751+0000 7f9f6affd700 1 --2- 172.21.15.119:0/1622117883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644170b9460 0x5644170bcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.751+0000 7f9f6affd700 1 -- 172.21.15.119:0/1622117883 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564416e89630 con 0x5644170bd150 2023-12-07T10:22:30.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.752+0000 7f9f6affd700 1 --2- 172.21.15.119:0/1622117883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644170bd150 0x5644170bd570 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9f60009e40 tx=0x7f9f60011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=278e3ddc38c62177 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:30.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.752+0000 7f9f6bfff700 1 --2- 172.21.15.119:0/901611174 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9f540a1d00 0x7f9f540a41c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9f64009ec0 tx=0x7f9f6400b040 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:30.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.752+0000 7f9f51ffb700 1 -- 172.21.15.119:0/1622117883 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f60019630 con 0x5644170bd150 2023-12-07T10:22:30.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.752+0000 7f9f51ffb700 1 -- 172.21.15.119:0/1622117883 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f60019c50 con 0x5644170bd150 2023-12-07T10:22:30.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.752+0000 7f9f51ffb700 1 -- 172.21.15.119:0/1622117883 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6000ed10 con 0x5644170bd150 2023-12-07T10:22:30.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.753+0000 7f9f8592ab80 1 -- 172.21.15.119:0/1622117883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644170bd150 msgr2=0x5644170bd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:30.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.753+0000 7f9f8592ab80 1 --2- 172.21.15.119:0/1622117883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644170bd150 0x5644170bd570 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9f60009e40 tx=0x7f9f60011720 comp rx=0 tx=0).stop 2023-12-07T10:22:30.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.753+0000 7f9f8592ab80 1 -- 172.21.15.119:0/1622117883 shutdown_connections 2023-12-07T10:22:30.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.753+0000 7f9f8592ab80 1 --2- 172.21.15.119:0/1622117883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644170bd150 0x5644170bd570 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.753+0000 7f9f8592ab80 1 --2- 172.21.15.119:0/1622117883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644170b8b40 0x5644170b8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.753+0000 7f9f8592ab80 1 --2- 172.21.15.119:0/1622117883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644170b9460 0x5644170bcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.753+0000 7f9f8592ab80 1 -- 172.21.15.119:0/1622117883 >> 172.21.15.119:0/1622117883 conn(0x564416fe7360 msgr2=0x5644170b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:30.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.753+0000 7f9f8592ab80 1 -- 172.21.15.119:0/1622117883 shutdown_connections 2023-12-07T10:22:30.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.753+0000 7f9f8592ab80 1 -- 172.21.15.119:0/1622117883 wait complete. 2023-12-07T10:22:30.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.753+0000 7f9f8592ab80 1 -- start start 2023-12-07T10:22:30.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.754+0000 7f9f8592ab80 1 Processor -- start 2023-12-07T10:22:30.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.754+0000 7f9f8592ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644170b8b40 0x5644170c4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.754+0000 7f9f8592ab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644170b9460 0x5644170c1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.754+0000 7f9f8592ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644170bd150 0x5644170c2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.754+0000 7f9f8592ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56441700c9b0 con 0x5644170b8b40 2023-12-07T10:22:30.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.754+0000 7f9f8592ab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x564416fc6560 con 0x5644170b9460 2023-12-07T10:22:30.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.754+0000 7f9f8592ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564416fc6200 con 0x5644170bd150 2023-12-07T10:22:30.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.754+0000 7f9f8592ab80 10 client.0 _ll_register_callbacks cb 0x564416fad940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:22:30.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.754+0000 7f9f6affd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644170b9460 0x5644170c1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.754+0000 7f9f6affd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644170b9460 0x5644170c1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42738/0 (socket says 172.21.15.119:42738) 2023-12-07T10:22:30.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.754+0000 7f9f6affd700 1 -- 172.21.15.119:0/3383893275 learned_addr learned my addr 172.21.15.119:0/3383893275 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:30.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.754+0000 7f9f6b7fe700 1 --2- 172.21.15.119:0/3383893275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644170b8b40 0x5644170c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.755+0000 7f9f6bfff700 1 --2- 172.21.15.119:0/3383893275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644170bd150 0x5644170c2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.755+0000 7f9f6affd700 1 -- 172.21.15.119:0/3383893275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644170bd150 msgr2=0x5644170c2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:30.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.755+0000 7f9f6affd700 1 --2- 172.21.15.119:0/3383893275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644170bd150 0x5644170c2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.755+0000 7f9f6affd700 1 -- 172.21.15.119:0/3383893275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644170b8b40 msgr2=0x5644170c4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:30.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.755+0000 7f9f6affd700 1 --2- 172.21.15.119:0/3383893275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644170b8b40 0x5644170c4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.755+0000 7f9f6affd700 1 -- 172.21.15.119:0/3383893275 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f60009a70 con 0x5644170b9460 2023-12-07T10:22:30.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.755+0000 7f9f6b7fe700 1 --2- 172.21.15.119:0/3383893275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644170b8b40 0x5644170c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:30.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.755+0000 7f9f6affd700 1 --2- 172.21.15.119:0/3383893275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644170b9460 0x5644170c1c20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9f60011a70 tx=0x7f9f6000ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afa5c272186d4c50 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:30.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.755+0000 7f9f4affd700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:22:30.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.755+0000 7f9f4affd700 1 -- 172.21.15.119:0/3383893275 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f60004250 con 0x5644170b9460 2023-12-07T10:22:30.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.755+0000 7f9f8592ab80 1 -- 172.21.15.119:0/3383893275 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5644170abb00 con 0x5644170b9460 2023-12-07T10:22:30.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.755+0000 7f9f4affd700 1 -- 172.21.15.119:0/3383893275 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9f600043b0 con 0x5644170b9460 2023-12-07T10:22:30.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.756+0000 7f9f4affd700 1 -- 172.21.15.119:0/3383893275 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f60004570 con 0x5644170b9460 2023-12-07T10:22:30.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.756+0000 7f9f8592ab80 10 client.26435 fetch_fsmap learned FSMap version 32 2023-12-07T10:22:30.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.756+0000 7f9f8592ab80 1 -- 172.21.15.119:0/3383893275 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564416e35de0 con 0x5644170b9460 2023-12-07T10:22:30.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.756+0000 7f9f4affd700 1 -- 172.21.15.119:0/3383893275 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5644170abb00 con 0x5644170b9460 2023-12-07T10:22:30.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.756+0000 7f9f4affd700 1 -- 172.21.15.119:0/3383893275 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f9f60028020 con 0x5644170b9460 2023-12-07T10:22:30.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.756+0000 7f9f8592ab80 10 client.26435 fetch_fsmap finished waiting for FSMap version 32 2023-12-07T10:22:30.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.756+0000 7f9f8592ab80 10 client.26435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:22:30.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.756+0000 7f9f8592ab80 20 client.26435 populate_metadata read hostname 'smithi119' 2023-12-07T10:22:30.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.756+0000 7f9f8592ab80 1 --2- 172.21.15.119:0/3383893275 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5644170d2180 0x5644170d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.756+0000 7f9f8592ab80 4 client.26435 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:22:30.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.756+0000 7f9f8592ab80 1 -- 172.21.15.119:0/3383893275 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5644170877a0 con 0x5644170d2180 2023-12-07T10:22:30.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.756+0000 7f9f6b7fe700 1 --2- 172.21.15.119:0/3383893275 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5644170d2180 0x5644170d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.757+0000 7f9f6b7fe700 1 --2- 172.21.15.119:0/3383893275 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5644170d2180 0x5644170d45a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=744a53b3a653d29 server_cookie=d11dcf60b3bfba62 in_seq=0 out_seq=0 2023-12-07T10:22:30.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.757+0000 7f9f4affd700 10 client.26435 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:30.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.757+0000 7f9f4affd700 1 -- 172.21.15.119:0/3383893275 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5644170877a0 con 0x5644170d2180 2023-12-07T10:22:30.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.757+0000 7f9f4affd700 10 client.26435 handle_command_reply: tid=0 2023-12-07T10:22:30.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.757+0000 7f9f8592ab80 1 client.26435 shutdown 2023-12-07T10:22:30.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.758+0000 7f9f8592ab80 1 -- 172.21.15.119:0/3383893275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644170b9460 msgr2=0x5644170c1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:30.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.758+0000 7f9f8592ab80 1 --2- 172.21.15.119:0/3383893275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644170b9460 0x5644170c1c20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9f60011a70 tx=0x7f9f6000ec40 comp rx=0 tx=0).stop 2023-12-07T10:22:30.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.758+0000 7f9f8592ab80 1 -- 172.21.15.119:0/3383893275 shutdown_connections 2023-12-07T10:22:30.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.758+0000 7f9f8592ab80 1 --2- 172.21.15.119:0/3383893275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644170bd150 0x5644170c2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.758+0000 7f9f8592ab80 1 --2- 172.21.15.119:0/3383893275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644170b8b40 0x5644170c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.758+0000 7f9f8592ab80 1 --2- 172.21.15.119:0/3383893275 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5644170d2180 0x5644170d45a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.758+0000 7f9f8592ab80 1 --2- 172.21.15.119:0/3383893275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644170b9460 0x5644170c1c20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.758+0000 7f9f4affd700 0 client.26435 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:30.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.758+0000 7f9f8592ab80 1 -- 172.21.15.119:0/3383893275 >> 172.21.15.119:0/3383893275 conn(0x564416fe7360 msgr2=0x5644170be620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:30.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.758+0000 7f9f8592ab80 1 -- 172.21.15.119:0/3383893275 shutdown_connections 2023-12-07T10:22:30.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.758+0000 7f9f8592ab80 1 -- 172.21.15.119:0/3383893275 wait complete. 2023-12-07T10:22:30.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.758+0000 7f9f8592ab80 20 client.26435 trim_cache size 0 max 16384 2023-12-07T10:22:30.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.780+0000 7f9f8592ab80 1 Processor -- start 2023-12-07T10:22:30.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.780+0000 7f9f8592ab80 1 -- start start 2023-12-07T10:22:30.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.780+0000 7f9f8592ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644170b8b40 0x5644170c4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.780+0000 7f9f8592ab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644170b9460 0x5644170ad490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.780+0000 7f9f8592ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644170bd150 0x5644170d8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.780+0000 7f9f8592ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564416e8b920 con 0x5644170b8b40 2023-12-07T10:22:30.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.780+0000 7f9f8592ab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644170c2830 con 0x5644170b9460 2023-12-07T10:22:30.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.780+0000 7f9f8592ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644170afaf0 con 0x5644170bd150 2023-12-07T10:22:30.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.781+0000 7f9f6affd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644170b9460 0x5644170ad490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.781+0000 7f9f6b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644170b8b40 0x5644170c4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.781+0000 7f9f6affd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644170b9460 0x5644170ad490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42748/0 (socket says 172.21.15.119:42748) 2023-12-07T10:22:30.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.781+0000 7f9f6bfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644170bd150 0x5644170d8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.781+0000 7f9f6affd700 1 -- 172.21.15.119:0/135646259 learned_addr learned my addr 172.21.15.119:0/135646259 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:30.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.781+0000 7f9f6affd700 1 -- 172.21.15.119:0/135646259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644170bd150 msgr2=0x5644170d8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:30.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.781+0000 7f9f6affd700 1 --2- 172.21.15.119:0/135646259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644170bd150 0x5644170d8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.781+0000 7f9f6affd700 1 -- 172.21.15.119:0/135646259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644170b8b40 msgr2=0x5644170c4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:30.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.781+0000 7f9f6affd700 1 --2- 172.21.15.119:0/135646259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644170b8b40 0x5644170c4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.781+0000 7f9f6affd700 1 -- 172.21.15.119:0/135646259 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564416e1ec60 con 0x5644170b9460 2023-12-07T10:22:30.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.781+0000 7f9f6b7fe700 1 --2- 172.21.15.119:0/135646259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644170b8b40 0x5644170c4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:30.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.781+0000 7f9f6affd700 1 --2- 172.21.15.119:0/135646259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644170b9460 0x5644170ad490 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9f60009d60 tx=0x7f9f60011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be5212b2834d8f82 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:30.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.782+0000 7f9f527fc700 1 -- 172.21.15.119:0/135646259 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6000ecb0 con 0x5644170b9460 2023-12-07T10:22:30.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.782+0000 7f9f527fc700 1 -- 172.21.15.119:0/135646259 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f6000ee10 con 0x5644170b9460 2023-12-07T10:22:30.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.782+0000 7f9f527fc700 1 -- 172.21.15.119:0/135646259 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f60019810 con 0x5644170b9460 2023-12-07T10:22:30.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.782+0000 7f9f8592ab80 1 -- 172.21.15.119:0/135646259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644170b9460 msgr2=0x5644170ad490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:30.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.782+0000 7f9f8592ab80 1 --2- 172.21.15.119:0/135646259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644170b9460 0x5644170ad490 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9f60009d60 tx=0x7f9f60011720 comp rx=0 tx=0).stop 2023-12-07T10:22:30.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.783+0000 7f9f8592ab80 1 -- 172.21.15.119:0/135646259 shutdown_connections 2023-12-07T10:22:30.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.783+0000 7f9f8592ab80 1 --2- 172.21.15.119:0/135646259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644170bd150 0x5644170d8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.783+0000 7f9f8592ab80 1 --2- 172.21.15.119:0/135646259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644170b8b40 0x5644170c4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.783+0000 7f9f8592ab80 1 --2- 172.21.15.119:0/135646259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644170b9460 0x5644170ad490 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.783+0000 7f9f8592ab80 1 -- 172.21.15.119:0/135646259 >> 172.21.15.119:0/135646259 conn(0x564416fe7360 msgr2=0x5644170b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:30.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.783+0000 7f9f8592ab80 1 -- 172.21.15.119:0/135646259 shutdown_connections 2023-12-07T10:22:30.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.783+0000 7f9f8592ab80 1 -- 172.21.15.119:0/135646259 wait complete. 2023-12-07T10:22:30.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.783+0000 7f9f8592ab80 1 -- start start 2023-12-07T10:22:30.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.783+0000 7f9f8592ab80 1 Processor -- start 2023-12-07T10:22:30.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.784+0000 7f9f8592ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644170b8b40 0x5644170c9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.784+0000 7f9f8592ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644170b9460 0x5644170c7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.784+0000 7f9f8592ab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644170bd150 0x5644170c7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.784+0000 7f9f8592ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644170c7d80 con 0x5644170b8b40 2023-12-07T10:22:30.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.784+0000 7f9f8592ab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644170c7ef0 con 0x5644170bd150 2023-12-07T10:22:30.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.784+0000 7f9f8592ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644170c80b0 con 0x5644170b9460 2023-12-07T10:22:30.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.784+0000 7f9f8592ab80 10 client.0 _ll_register_callbacks cb 0x564416fad940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:22:30.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.784+0000 7f9f6b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644170b8b40 0x5644170c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.784+0000 7f9f6b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644170b8b40 0x5644170c9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45200/0 (socket says 172.21.15.119:45200) 2023-12-07T10:22:30.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.784+0000 7f9f6b7fe700 1 -- 172.21.15.119:0/3131502134 learned_addr learned my addr 172.21.15.119:0/3131502134 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:30.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.784+0000 7f9f6bfff700 1 --2- 172.21.15.119:0/3131502134 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644170bd150 0x5644170c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.784+0000 7f9f6affd700 1 --2- 172.21.15.119:0/3131502134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644170b9460 0x5644170c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.785+0000 7f9f6b7fe700 1 -- 172.21.15.119:0/3131502134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644170b9460 msgr2=0x5644170c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:30.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.785+0000 7f9f6b7fe700 1 --2- 172.21.15.119:0/3131502134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644170b9460 0x5644170c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.785+0000 7f9f6b7fe700 1 -- 172.21.15.119:0/3131502134 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644170bd150 msgr2=0x5644170c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:30.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.785+0000 7f9f6b7fe700 1 --2- 172.21.15.119:0/3131502134 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644170bd150 0x5644170c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.785+0000 7f9f6b7fe700 1 -- 172.21.15.119:0/3131502134 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f6c1aa9c0 con 0x5644170b8b40 2023-12-07T10:22:30.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.785+0000 7f9f6b7fe700 1 --2- 172.21.15.119:0/3131502134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644170b8b40 0x5644170c9510 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c000c00 tx=0x7f9f5c0cc210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd02d471b5b7c09 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:30.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.785+0000 7f9f4affd700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:22:30.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.785+0000 7f9f4affd700 1 -- 172.21.15.119:0/3131502134 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c09b390 con 0x5644170b8b40 2023-12-07T10:22:30.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.785+0000 7f9f4affd700 1 -- 172.21.15.119:0/3131502134 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9f5c09f4e0 con 0x5644170b8b40 2023-12-07T10:22:30.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.785+0000 7f9f8592ab80 1 -- 172.21.15.119:0/3131502134 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f60019c50 con 0x5644170b8b40 2023-12-07T10:22:30.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.785+0000 7f9f4affd700 1 -- 172.21.15.119:0/3131502134 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f5c0a3690 con 0x5644170b8b40 2023-12-07T10:22:30.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.785+0000 7f9f8592ab80 10 client.16425 fetch_fsmap learned FSMap version 32 2023-12-07T10:22:30.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.785+0000 7f9f8592ab80 1 -- 172.21.15.119:0/3131502134 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564416e35de0 con 0x5644170b8b40 2023-12-07T10:22:30.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.785+0000 7f9f4affd700 1 -- 172.21.15.119:0/3131502134 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f60019c50 con 0x5644170b8b40 2023-12-07T10:22:30.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.786+0000 7f9f4affd700 1 -- 172.21.15.119:0/3131502134 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f9f5c09a880 con 0x5644170b8b40 2023-12-07T10:22:30.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.786+0000 7f9f8592ab80 10 client.16425 fetch_fsmap finished waiting for FSMap version 32 2023-12-07T10:22:30.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.786+0000 7f9f8592ab80 10 client.16425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:22:30.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.786+0000 7f9f8592ab80 20 client.16425 populate_metadata read hostname 'smithi119' 2023-12-07T10:22:30.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.786+0000 7f9f8592ab80 1 --2- 172.21.15.119:0/3131502134 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5644171288f0 0x56441712ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:30.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.786+0000 7f9f8592ab80 4 client.16425 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-12-07T10:22:30.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.786+0000 7f9f8592ab80 1 -- 172.21.15.119:0/3131502134 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564416fd2fd0 con 0x5644171288f0 2023-12-07T10:22:30.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.786+0000 7f9f6affd700 1 --2- 172.21.15.119:0/3131502134 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5644171288f0 0x56441712ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:30.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.786+0000 7f9f4affd700 10 client.16425 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:30.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.786+0000 7f9f6affd700 1 --2- 172.21.15.119:0/3131502134 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5644171288f0 0x56441712ad10 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24012cd102468fa5 server_cookie=f92ded9087db6f00 in_seq=0 out_seq=0 2023-12-07T10:22:30.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.787+0000 7f9f4affd700 1 -- 172.21.15.119:0/3131502134 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564416fd2fd0 con 0x5644171288f0 2023-12-07T10:22:30.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.787+0000 7f9f4affd700 10 client.16425 handle_command_reply: tid=0 2023-12-07T10:22:30.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.787+0000 7f9f8592ab80 1 client.16425 shutdown 2023-12-07T10:22:30.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.787+0000 7f9f8592ab80 1 -- 172.21.15.119:0/3131502134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644170b8b40 msgr2=0x5644170c9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:30.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.787+0000 7f9f8592ab80 1 --2- 172.21.15.119:0/3131502134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644170b8b40 0x5644170c9510 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c000c00 tx=0x7f9f5c0cc210 comp rx=0 tx=0).stop 2023-12-07T10:22:30.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.787+0000 7f9f8592ab80 1 -- 172.21.15.119:0/3131502134 shutdown_connections 2023-12-07T10:22:30.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.787+0000 7f9f8592ab80 1 --2- 172.21.15.119:0/3131502134 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5644171288f0 0x56441712ad10 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.787+0000 7f9f8592ab80 1 --2- 172.21.15.119:0/3131502134 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5644170bd150 0x5644170c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.787+0000 7f9f8592ab80 1 --2- 172.21.15.119:0/3131502134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644170b8b40 0x5644170c9510 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.787+0000 7f9f8592ab80 1 --2- 172.21.15.119:0/3131502134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644170b9460 0x5644170c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.787+0000 7f9f8592ab80 1 -- 172.21.15.119:0/3131502134 >> 172.21.15.119:0/3131502134 conn(0x564416fe7360 msgr2=0x5644170ad490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:30.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.787+0000 7f9f4affd700 0 client.16425 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:30.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.787+0000 7f9f8592ab80 1 -- 172.21.15.119:0/3131502134 shutdown_connections 2023-12-07T10:22:30.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.787+0000 7f9f8592ab80 1 -- 172.21.15.119:0/3131502134 wait complete. 2023-12-07T10:22:30.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.787+0000 7f9f8592ab80 20 client.16425 trim_cache size 0 max 16384 2023-12-07T10:22:30.792 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:22:30.792 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-07T10:22:30.792 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": { 2023-12-07T10:22:30.792 INFO:teuthology.orchestra.run.smithi119.stdout: "7486ba01-e158-40fd-9e83-d5b9b52c671e": { 2023-12-07T10:22:30.792 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/", 2023-12-07T10:22:30.792 INFO:teuthology.orchestra.run.smithi119.stdout: "tag": "7486ba01-e158-40fd-9e83-d5b9b52c671e", 2023-12-07T10:22:30.792 INFO:teuthology.orchestra.run.smithi119.stdout: "options": "recursive,force" 2023-12-07T10:22:30.792 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:22:30.792 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:22:30.792 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:22:30.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.788+0000 7f9f71fff700 1 -- 172.21.15.119:0/901611174 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9f540a1d00 msgr2=0x7f9f540a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:30.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.788+0000 7f9f71fff700 1 --2- 172.21.15.119:0/901611174 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9f540a1d00 0x7f9f540a41c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9f64009ec0 tx=0x7f9f6400b040 comp rx=0 tx=0).stop 2023-12-07T10:22:30.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.788+0000 7f9f71fff700 1 -- 172.21.15.119:0/901611174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f6c101550 msgr2=0x7f9f6c1083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:30.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.788+0000 7f9f71fff700 1 --2- 172.21.15.119:0/901611174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f6c101550 0x7f9f6c1083e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c005b40 tx=0x7f9f5c00b9c0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.788+0000 7f9f71fff700 1 -- 172.21.15.119:0/901611174 shutdown_connections 2023-12-07T10:22:30.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.788+0000 7f9f71fff700 1 --2- 172.21.15.119:0/901611174 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9f540a1d00 0x7f9f540a41c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.788+0000 7f9f71fff700 1 --2- 172.21.15.119:0/901611174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f6c107610 0x7f9f6c108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.788+0000 7f9f71fff700 1 --2- 172.21.15.119:0/901611174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f6c101550 0x7f9f6c1083e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.788+0000 7f9f71fff700 1 --2- 172.21.15.119:0/901611174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f6c101f10 0x7f9f6c108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:30.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.788+0000 7f9f71fff700 1 -- 172.21.15.119:0/901611174 >> 172.21.15.119:0/901611174 conn(0x7f9f6c0fd020 msgr2=0x7f9f6c0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:30.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.788+0000 7f9f71fff700 1 -- 172.21.15.119:0/901611174 shutdown_connections 2023-12-07T10:22:30.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:22:30.788+0000 7f9f71fff700 1 -- 172.21.15.119:0/901611174 wait complete. 2023-12-07T10:22:30.809 INFO:tasks.cephfs.filesystem:scrub status for tag:7486ba01-e158-40fd-9e83-d5b9b52c671e - {'path': '/', 'tag': '7486ba01-e158-40fd-9e83-d5b9b52c671e', 'options': 'recursive,force'} 2023-12-07T10:22:31.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:30 smithi119 ceph-mon[97908]: scrub queued for path: / 2023-12-07T10:22:31.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:30 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:22:31.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:30 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:22:31.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:30 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:22:31.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:30 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:22:31.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:30 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:22:31.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:30 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:22:31.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:30 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:22:31.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:30 smithi159 ceph-mon[134726]: scrub queued for path: / 2023-12-07T10:22:31.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:30 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:22:31.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:30 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:22:31.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:30 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:22:31.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:30 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:22:31.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:30 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:22:31.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:30 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:22:31.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:30 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:22:31.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:30 smithi153 ceph-mon[133625]: scrub queued for path: / 2023-12-07T10:22:31.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:30 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:22:31.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:30 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:22:31.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:30 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:22:31.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:30 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:22:31.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:30 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:22:31.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:30 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:22:31.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:30 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:22:31.545 INFO:tasks.workunit.client.0.smithi119.stdout: 1 73451 9.33 MB/sec warmup 44 sec latency 13.586 ms 2023-12-07T10:22:32.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:31 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:22:32.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:31 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:22:32.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:31 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:22:32.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:31 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:22:32.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:31 smithi119 ceph-mon[97908]: pgmap v66: 129 pgs: 129 active+clean; 54 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 307 KiB/s wr, 80 op/s 2023-12-07T10:22:32.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:31 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:22:32.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:31 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:22:32.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:31 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:22:32.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:31 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:22:32.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:31 smithi159 ceph-mon[134726]: pgmap v66: 129 pgs: 129 active+clean; 54 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 307 KiB/s wr, 80 op/s 2023-12-07T10:22:32.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:31 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:22:32.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:31 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:22:32.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:31 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:22:32.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:31 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:22:32.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:31 smithi153 ceph-mon[133625]: pgmap v66: 129 pgs: 129 active+clean; 54 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 307 KiB/s wr, 80 op/s 2023-12-07T10:22:32.545 INFO:tasks.workunit.client.0.smithi119.stdout: 1 74994 9.39 MB/sec warmup 45 sec latency 11.421 ms 2023-12-07T10:22:33.545 INFO:tasks.workunit.client.0.smithi119.stdout: 1 77102 9.36 MB/sec warmup 46 sec latency 12.905 ms 2023-12-07T10:22:34.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:33 smithi119 ceph-mon[97908]: pgmap v67: 129 pgs: 129 active+clean; 54 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 279 KiB/s wr, 70 op/s 2023-12-07T10:22:34.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:33 smithi159 ceph-mon[134726]: pgmap v67: 129 pgs: 129 active+clean; 54 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 279 KiB/s wr, 70 op/s 2023-12-07T10:22:34.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:33 smithi153 ceph-mon[133625]: pgmap v67: 129 pgs: 129 active+clean; 54 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 279 KiB/s wr, 70 op/s 2023-12-07T10:22:34.545 INFO:tasks.workunit.client.0.smithi119.stdout: 1 78573 9.40 MB/sec warmup 47 sec latency 20.457 ms 2023-12-07T10:22:35.545 INFO:tasks.workunit.client.0.smithi119.stdout: 1 80854 9.43 MB/sec warmup 48 sec latency 12.557 ms 2023-12-07T10:22:36.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:35 smithi119 ceph-mon[97908]: pgmap v68: 129 pgs: 129 active+clean; 54 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 279 KiB/s wr, 70 op/s 2023-12-07T10:22:36.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:35 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:22:36.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:35 smithi159 ceph-mon[134726]: pgmap v68: 129 pgs: 129 active+clean; 54 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 279 KiB/s wr, 70 op/s 2023-12-07T10:22:36.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:35 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:22:36.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:35 smithi153 ceph-mon[133625]: pgmap v68: 129 pgs: 129 active+clean; 54 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 279 KiB/s wr, 70 op/s 2023-12-07T10:22:36.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:35 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:22:36.545 INFO:tasks.workunit.client.0.smithi119.stdout: 1 82158 9.40 MB/sec warmup 49 sec latency 16.340 ms 2023-12-07T10:22:37.546 INFO:tasks.workunit.client.0.smithi119.stdout: 1 84231 9.38 MB/sec warmup 50 sec latency 15.882 ms 2023-12-07T10:22:37.728 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:37 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:22:37.728 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:37 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:22:37.728 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:37 smithi119 ceph-mon[97908]: pgmap v69: 129 pgs: 129 active+clean; 59 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 835 KiB/s wr, 164 op/s 2023-12-07T10:22:37.728 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:37 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:22:37.728 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:37 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:22:37.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:37 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:22:37.762 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:37 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:22:37.762 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:37 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:22:37.762 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:37 smithi159 ceph-mon[134726]: pgmap v69: 129 pgs: 129 active+clean; 59 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 835 KiB/s wr, 164 op/s 2023-12-07T10:22:37.762 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:37 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:22:37.762 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:37 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:22:37.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:37 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:22:37.779 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:37 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:22:37.779 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:37 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:22:37.779 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:37 smithi153 ceph-mon[133625]: pgmap v69: 129 pgs: 129 active+clean; 59 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 835 KiB/s wr, 164 op/s 2023-12-07T10:22:37.779 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:37 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:22:37.779 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:37 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:22:37.779 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:37 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:22:38.546 INFO:tasks.workunit.client.0.smithi119.stdout: 1 85713 9.41 MB/sec warmup 51 sec latency 10.996 ms 2023-12-07T10:22:39.546 INFO:tasks.workunit.client.0.smithi119.stdout: 1 87937 9.44 MB/sec warmup 52 sec latency 15.245 ms 2023-12-07T10:22:39.978 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:39 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:22:39.978 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:39 smithi119 ceph-mon[97908]: pgmap v70: 129 pgs: 129 active+clean; 59 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 819 KiB/s wr, 155 op/s 2023-12-07T10:22:40.012 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:39 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:22:40.012 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:39 smithi159 ceph-mon[134726]: pgmap v70: 129 pgs: 129 active+clean; 59 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 819 KiB/s wr, 155 op/s 2023-12-07T10:22:40.029 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:39 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:22:40.029 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:39 smithi153 ceph-mon[133625]: pgmap v70: 129 pgs: 129 active+clean; 59 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 819 KiB/s wr, 155 op/s 2023-12-07T10:22:40.187 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.185+0000 7fa1437ff700 1 -- 172.21.15.119:0/2042975839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c109720 msgr2=0x7fa13c109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:40.187 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.185+0000 7fa1437ff700 1 --2- 172.21.15.119:0/2042975839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c109720 0x7fa13c109b00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa138013ef0 tx=0x7fa1380097f0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.187 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.186+0000 7fa1437ff700 1 -- 172.21.15.119:0/2042975839 shutdown_connections 2023-12-07T10:22:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.186+0000 7fa1437ff700 1 --2- 172.21.15.119:0/2042975839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa13c1040d0 0x7fa13c1105e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.186+0000 7fa1437ff700 1 --2- 172.21.15.119:0/2042975839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c109720 0x7fa13c109b00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.186+0000 7fa1437ff700 1 --2- 172.21.15.119:0/2042975839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa13c103710 0x7fa13c103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.186+0000 7fa1437ff700 1 -- 172.21.15.119:0/2042975839 >> 172.21.15.119:0/2042975839 conn(0x7fa13c0ff1c0 msgr2=0x7fa13c1015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.186+0000 7fa1437ff700 1 -- 172.21.15.119:0/2042975839 shutdown_connections 2023-12-07T10:22:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.186+0000 7fa1437ff700 1 -- 172.21.15.119:0/2042975839 wait complete. 2023-12-07T10:22:40.191 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.187+0000 7fa1437ff700 1 Processor -- start 2023-12-07T10:22:40.191 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.187+0000 7fa1437ff700 1 -- start start 2023-12-07T10:22:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.188+0000 7fa1437ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c103710 0x7fa13c1961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.188+0000 7fa1437ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa13c1040d0 0x7fa13c196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.188+0000 7fa1437ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa13c109720 0x7fa13c19b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.188+0000 7fa1437ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c19b940 con 0x7fa13c109720 2023-12-07T10:22:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.188+0000 7fa1437ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c19bab0 con 0x7fa13c103710 2023-12-07T10:22:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.188+0000 7fa1437ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c19bc70 con 0x7fa13c1040d0 2023-12-07T10:22:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.188+0000 7fa140d9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa13c1040d0 0x7fa13c196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.188+0000 7fa14159b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c103710 0x7fa13c1961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.188+0000 7fa14159b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c103710 0x7fa13c1961c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:46072/0 (socket says 172.21.15.119:46072) 2023-12-07T10:22:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.188+0000 7fa14159b700 1 -- 172.21.15.119:0/3763690130 learned_addr learned my addr 172.21.15.119:0/3763690130 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.188+0000 7fa140d9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa13c1040d0 0x7fa13c196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:44022/0 (socket says 172.21.15.119:44022) 2023-12-07T10:22:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.188+0000 7fa141d9c700 1 --2- 172.21.15.119:0/3763690130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa13c109720 0x7fa13c19b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.188+0000 7fa140d9a700 1 -- 172.21.15.119:0/3763690130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c103710 msgr2=0x7fa13c1961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.188+0000 7fa140d9a700 1 --2- 172.21.15.119:0/3763690130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c103710 0x7fa13c1961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.188+0000 7fa140d9a700 1 -- 172.21.15.119:0/3763690130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa13c109720 msgr2=0x7fa13c19b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.188+0000 7fa140d9a700 1 --2- 172.21.15.119:0/3763690130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa13c109720 0x7fa13c19b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.188+0000 7fa140d9a700 1 -- 172.21.15.119:0/3763690130 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa138013b50 con 0x7fa13c1040d0 2023-12-07T10:22:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.189+0000 7fa140d9a700 1 --2- 172.21.15.119:0/3763690130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa13c1040d0 0x7fa13c196700 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa12c00e9f0 tx=0x7fa12c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.189+0000 7fa1327fc700 1 -- 172.21.15.119:0/3763690130 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c00c9c0 con 0x7fa13c1040d0 2023-12-07T10:22:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.189+0000 7fa1437ff700 1 -- 172.21.15.119:0/3763690130 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13c19bef0 con 0x7fa13c1040d0 2023-12-07T10:22:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.189+0000 7fa1327fc700 1 -- 172.21.15.119:0/3763690130 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa12c010430 con 0x7fa13c1040d0 2023-12-07T10:22:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.189+0000 7fa1327fc700 1 -- 172.21.15.119:0/3763690130 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c018680 con 0x7fa13c1040d0 2023-12-07T10:22:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.189+0000 7fa1437ff700 1 -- 172.21.15.119:0/3763690130 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13c1a2500 con 0x7fa13c1040d0 2023-12-07T10:22:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.192+0000 7fa1327fc700 1 -- 172.21.15.119:0/3763690130 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fa12c010860 con 0x7fa13c1040d0 2023-12-07T10:22:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.194+0000 7fa1571b2b80 1 Processor -- start 2023-12-07T10:22:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.194+0000 7fa1327fc700 1 --2- 172.21.15.119:0/3763690130 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa1280a1d40 0x7fa1280a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.194+0000 7fa1571b2b80 1 -- start start 2023-12-07T10:22:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.194+0000 7fa1327fc700 1 -- 172.21.15.119:0/3763690130 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fa12c014070 con 0x7fa13c1040d0 2023-12-07T10:22:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.194+0000 7fa141d9c700 1 --2- 172.21.15.119:0/3763690130 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa1280a1d40 0x7fa1280a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.195+0000 7fa1571b2b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558640e7aa60 0x558640e7ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.195+0000 7fa1571b2b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558640e7b380 0x558640e75540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.195+0000 7fa1571b2b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558640e75ad0 0x558640e75f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.195+0000 7fa1571b2b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558640de9760 con 0x558640e75ad0 2023-12-07T10:22:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.195+0000 7fa1571b2b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558640dd0740 con 0x558640e7b380 2023-12-07T10:22:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.195+0000 7fa1571b2b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558640dce060 con 0x558640e7aa60 2023-12-07T10:22:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.195+0000 7fa14159b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558640e7aa60 0x558640e7ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.195+0000 7fa14159b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558640e7aa60 0x558640e7ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:44032/0 (socket says 172.21.15.119:44032) 2023-12-07T10:22:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.195+0000 7fa14159b700 1 -- 172.21.15.119:0/952985367 learned_addr learned my addr 172.21.15.119:0/952985367 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.195+0000 7fa140d9a700 1 --2- 172.21.15.119:0/952985367 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558640e7b380 0x558640e75540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.195+0000 7fa14159b700 1 -- 172.21.15.119:0/952985367 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558640e7b380 msgr2=0x558640e75540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.195+0000 7fa14159b700 1 --2- 172.21.15.119:0/952985367 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558640e7b380 0x558640e75540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.195+0000 7fa14159b700 1 -- 172.21.15.119:0/952985367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558640e75ad0 msgr2=0x558640e75f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.195+0000 7fa14159b700 1 --2- 172.21.15.119:0/952985367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558640e75ad0 0x558640e75f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.195+0000 7fa14159b700 1 -- 172.21.15.119:0/952985367 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558640c465d0 con 0x558640e7aa60 2023-12-07T10:22:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.195+0000 7fa140d9a700 1 --2- 172.21.15.119:0/952985367 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558640e7b380 0x558640e75540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.195+0000 7fa141d9c700 1 --2- 172.21.15.119:0/3763690130 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa1280a1d40 0x7fa1280a4200 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa13400f590 tx=0x7fa134009920 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.196+0000 7fa14159b700 1 --2- 172.21.15.119:0/952985367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558640e7aa60 0x558640e7ae40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa138005c10 tx=0x7fa138005c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d0f5511def77eb9 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.196+0000 7fa1277fe700 1 -- 172.21.15.119:0/952985367 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa138003c70 con 0x558640e7aa60 2023-12-07T10:22:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.196+0000 7fa1277fe700 1 -- 172.21.15.119:0/952985367 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1380038c0 con 0x558640e7aa60 2023-12-07T10:22:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.196+0000 7fa1277fe700 1 -- 172.21.15.119:0/952985367 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa138013170 con 0x558640e7aa60 2023-12-07T10:22:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.197+0000 7fa1571b2b80 1 -- 172.21.15.119:0/952985367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558640e7aa60 msgr2=0x558640e7ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.197+0000 7fa1571b2b80 1 --2- 172.21.15.119:0/952985367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558640e7aa60 0x558640e7ae40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa138005c10 tx=0x7fa138005c40 comp rx=0 tx=0).stop 2023-12-07T10:22:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.197+0000 7fa1571b2b80 1 -- 172.21.15.119:0/952985367 shutdown_connections 2023-12-07T10:22:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.197+0000 7fa1571b2b80 1 --2- 172.21.15.119:0/952985367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558640e75ad0 0x558640e75f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.197+0000 7fa1571b2b80 1 --2- 172.21.15.119:0/952985367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558640e7aa60 0x558640e7ae40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.197+0000 7fa1571b2b80 1 --2- 172.21.15.119:0/952985367 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558640e7b380 0x558640e75540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.197+0000 7fa1571b2b80 1 -- 172.21.15.119:0/952985367 >> 172.21.15.119:0/952985367 conn(0x558640da4300 msgr2=0x558640e6fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.197+0000 7fa1571b2b80 1 -- 172.21.15.119:0/952985367 shutdown_connections 2023-12-07T10:22:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.197+0000 7fa1571b2b80 1 -- 172.21.15.119:0/952985367 wait complete. 2023-12-07T10:22:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.197+0000 7fa1571b2b80 1 -- start start 2023-12-07T10:22:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.198+0000 7fa1571b2b80 1 Processor -- start 2023-12-07T10:22:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.198+0000 7fa1571b2b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558640e75ad0 0x558640e7e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.198+0000 7fa1571b2b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558640e7aa60 0x558640e7ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.198+0000 7fa1571b2b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558640e7b380 0x558640e7f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.198+0000 7fa1571b2b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558640dc9950 con 0x558640e7b380 2023-12-07T10:22:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.198+0000 7fa1571b2b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558640d83500 con 0x558640e7aa60 2023-12-07T10:22:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.198+0000 7fa1571b2b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558640d831a0 con 0x558640e75ad0 2023-12-07T10:22:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.198+0000 7fa1571b2b80 10 client.0 _ll_register_callbacks cb 0x558640d6a8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:22:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.198+0000 7fa14159b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558640e75ad0 0x558640e7e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.198+0000 7fa14159b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558640e75ad0 0x558640e7e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:44044/0 (socket says 172.21.15.119:44044) 2023-12-07T10:22:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.198+0000 7fa14159b700 1 -- 172.21.15.119:0/4172861048 learned_addr learned my addr 172.21.15.119:0/4172861048 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.199+0000 7fa141d9c700 1 --2- 172.21.15.119:0/4172861048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558640e7b380 0x558640e7f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.199+0000 7fa140d9a700 1 --2- 172.21.15.119:0/4172861048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558640e7aa60 0x558640e7ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.199+0000 7fa14159b700 1 -- 172.21.15.119:0/4172861048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558640e7aa60 msgr2=0x558640e7ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.199+0000 7fa14159b700 1 --2- 172.21.15.119:0/4172861048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558640e7aa60 0x558640e7ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.199+0000 7fa14159b700 1 -- 172.21.15.119:0/4172861048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558640e7b380 msgr2=0x558640e7f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.199+0000 7fa14159b700 1 --2- 172.21.15.119:0/4172861048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558640e7b380 0x558640e7f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.199+0000 7fa14159b700 1 -- 172.21.15.119:0/4172861048 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13c1a2500 con 0x558640e75ad0 2023-12-07T10:22:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.199+0000 7fa140d9a700 1 --2- 172.21.15.119:0/4172861048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558640e7aa60 0x558640e7ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:22:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.199+0000 7fa14159b700 1 --2- 172.21.15.119:0/4172861048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558640e75ad0 0x558640e7e6c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa138005320 tx=0x7fa138012dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99ca01f0d3654bf5 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.199+0000 7fa124ff9700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:22:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.199+0000 7fa124ff9700 1 -- 172.21.15.119:0/4172861048 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa138003c70 con 0x558640e75ad0 2023-12-07T10:22:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.199+0000 7fa124ff9700 1 -- 172.21.15.119:0/4172861048 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fa138003dd0 con 0x558640e75ad0 2023-12-07T10:22:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.199+0000 7fa1571b2b80 1 -- 172.21.15.119:0/4172861048 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558640e44740 con 0x558640e75ad0 2023-12-07T10:22:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.200+0000 7fa124ff9700 1 -- 172.21.15.119:0/4172861048 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1380038c0 con 0x558640e75ad0 2023-12-07T10:22:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.200+0000 7fa1571b2b80 10 client.26443 fetch_fsmap learned FSMap version 33 2023-12-07T10:22:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.200+0000 7fa1571b2b80 1 -- 172.21.15.119:0/4172861048 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fa13c110ee0 con 0x558640e75ad0 2023-12-07T10:22:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.200+0000 7fa124ff9700 1 -- 172.21.15.119:0/4172861048 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558640e44740 con 0x558640e75ad0 2023-12-07T10:22:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.200+0000 7fa124ff9700 1 -- 172.21.15.119:0/4172861048 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 33) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fa13802b020 con 0x558640e75ad0 2023-12-07T10:22:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.200+0000 7fa1571b2b80 10 client.26443 fetch_fsmap finished waiting for FSMap version 33 2023-12-07T10:22:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.200+0000 7fa1571b2b80 10 client.26443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:22:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.200+0000 7fa1571b2b80 20 client.26443 populate_metadata read hostname 'smithi119' 2023-12-07T10:22:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.200+0000 7fa1571b2b80 1 --2- 172.21.15.119:0/4172861048 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558640e8e8d0 0x558640e90cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.200+0000 7fa1571b2b80 4 client.26443 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:22:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.200+0000 7fa1571b2b80 1 -- 172.21.15.119:0/4172861048 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558640d8ff70 con 0x558640e8e8d0 2023-12-07T10:22:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.200+0000 7fa140d9a700 1 --2- 172.21.15.119:0/4172861048 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558640e8e8d0 0x558640e90cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.201+0000 7fa140d9a700 1 --2- 172.21.15.119:0/4172861048 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558640e8e8d0 0x558640e90cc0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac1b4886b602881b server_cookie=66112e7acbc1cfd9 in_seq=0 out_seq=0 2023-12-07T10:22:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.201+0000 7fa124ff9700 10 client.26443 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.202+0000 7fa124ff9700 1 -- 172.21.15.119:0/4172861048 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558640d8ff70 con 0x558640e8e8d0 2023-12-07T10:22:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.202+0000 7fa124ff9700 10 client.26443 handle_command_reply: tid=0 2023-12-07T10:22:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.202+0000 7fa1571b2b80 1 client.26443 shutdown 2023-12-07T10:22:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.202+0000 7fa1571b2b80 1 -- 172.21.15.119:0/4172861048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558640e75ad0 msgr2=0x558640e7e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.202+0000 7fa1571b2b80 1 --2- 172.21.15.119:0/4172861048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558640e75ad0 0x558640e7e6c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa138005320 tx=0x7fa138012dc0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.202+0000 7fa1571b2b80 1 -- 172.21.15.119:0/4172861048 shutdown_connections 2023-12-07T10:22:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.202+0000 7fa1571b2b80 1 --2- 172.21.15.119:0/4172861048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558640e7b380 0x558640e7f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.202+0000 7fa1571b2b80 1 --2- 172.21.15.119:0/4172861048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558640e75ad0 0x558640e7e6c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.202+0000 7fa1571b2b80 1 --2- 172.21.15.119:0/4172861048 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558640e8e8d0 0x558640e90cc0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.202+0000 7fa1571b2b80 1 --2- 172.21.15.119:0/4172861048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558640e7aa60 0x558640e7ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.202+0000 7fa1571b2b80 1 -- 172.21.15.119:0/4172861048 >> 172.21.15.119:0/4172861048 conn(0x558640da4300 msgr2=0x558640e6a980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.202+0000 7fa124ff9700 0 client.26443 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.202+0000 7fa1571b2b80 1 -- 172.21.15.119:0/4172861048 shutdown_connections 2023-12-07T10:22:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.202+0000 7fa1571b2b80 1 -- 172.21.15.119:0/4172861048 wait complete. 2023-12-07T10:22:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.203+0000 7fa1571b2b80 20 client.26443 trim_cache size 0 max 16384 2023-12-07T10:22:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.220+0000 7fa1571b2b80 1 Processor -- start 2023-12-07T10:22:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.220+0000 7fa1571b2b80 1 -- start start 2023-12-07T10:22:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.220+0000 7fa1571b2b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558640e75ad0 0x558640e75540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.220+0000 7fa1571b2b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558640e7aa60 0x558640e7ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.220+0000 7fa1571b2b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558640e7b380 0x558640e78920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.220+0000 7fa1571b2b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558640c488c0 con 0x558640e7b380 2023-12-07T10:22:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.220+0000 7fa1571b2b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558640e7fce0 con 0x558640e75ad0 2023-12-07T10:22:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.220+0000 7fa1571b2b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558640e70270 con 0x558640e7aa60 2023-12-07T10:22:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.221+0000 7fa14159b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558640e75ad0 0x558640e75540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.221+0000 7fa14159b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558640e75ad0 0x558640e75540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:46096/0 (socket says 172.21.15.119:46096) 2023-12-07T10:22:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.221+0000 7fa140d9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558640e7aa60 0x558640e7ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.221+0000 7fa14159b700 1 -- 172.21.15.119:0/3348912399 learned_addr learned my addr 172.21.15.119:0/3348912399 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.221+0000 7fa141d9c700 1 --2- 172.21.15.119:0/3348912399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558640e7b380 0x558640e78920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.221+0000 7fa14159b700 1 -- 172.21.15.119:0/3348912399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558640e7aa60 msgr2=0x558640e7ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.221+0000 7fa14159b700 1 --2- 172.21.15.119:0/3348912399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558640e7aa60 0x558640e7ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.221+0000 7fa14159b700 1 -- 172.21.15.119:0/3348912399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558640e7b380 msgr2=0x558640e78920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.221+0000 7fa14159b700 1 --2- 172.21.15.119:0/3348912399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558640e7b380 0x558640e78920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.221+0000 7fa14159b700 1 -- 172.21.15.119:0/3348912399 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558640bf2d80 con 0x558640e75ad0 2023-12-07T10:22:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.221+0000 7fa141d9c700 1 --2- 172.21.15.119:0/3348912399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558640e7b380 0x558640e78920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:22:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.221+0000 7fa14159b700 1 --2- 172.21.15.119:0/3348912399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558640e75ad0 0x558640e75540 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa1380053f0 tx=0x7fa138005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96e328c15b8732c0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.222+0000 7fa127fff700 1 -- 172.21.15.119:0/3348912399 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1380038c0 con 0x558640e75ad0 2023-12-07T10:22:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.222+0000 7fa127fff700 1 -- 172.21.15.119:0/3348912399 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa138003680 con 0x558640e75ad0 2023-12-07T10:22:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.222+0000 7fa127fff700 1 -- 172.21.15.119:0/3348912399 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1380135c0 con 0x558640e75ad0 2023-12-07T10:22:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.222+0000 7fa1571b2b80 1 -- 172.21.15.119:0/3348912399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558640e75ad0 msgr2=0x558640e75540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.222+0000 7fa1571b2b80 1 --2- 172.21.15.119:0/3348912399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558640e75ad0 0x558640e75540 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa1380053f0 tx=0x7fa138005fb0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.223+0000 7fa1571b2b80 1 -- 172.21.15.119:0/3348912399 shutdown_connections 2023-12-07T10:22:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.223+0000 7fa1571b2b80 1 --2- 172.21.15.119:0/3348912399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558640e7b380 0x558640e78920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.223+0000 7fa1571b2b80 1 --2- 172.21.15.119:0/3348912399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558640e75ad0 0x558640e75540 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.223+0000 7fa1571b2b80 1 --2- 172.21.15.119:0/3348912399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558640e7aa60 0x558640e7ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.223+0000 7fa1571b2b80 1 -- 172.21.15.119:0/3348912399 >> 172.21.15.119:0/3348912399 conn(0x558640da4300 msgr2=0x558640e95660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.223+0000 7fa1571b2b80 1 -- 172.21.15.119:0/3348912399 shutdown_connections 2023-12-07T10:22:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.223+0000 7fa1571b2b80 1 -- 172.21.15.119:0/3348912399 wait complete. 2023-12-07T10:22:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.223+0000 7fa1571b2b80 1 -- start start 2023-12-07T10:22:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.223+0000 7fa1571b2b80 1 Processor -- start 2023-12-07T10:22:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.224+0000 7fa1571b2b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558640e75ad0 0x558640e86040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.224+0000 7fa1571b2b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558640e7aa60 0x558640e83d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.224+0000 7fa1571b2b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558640e7b380 0x558640e84290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.224+0000 7fa1571b2b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558640e84860 con 0x558640e75ad0 2023-12-07T10:22:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.224+0000 7fa1571b2b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558640e849d0 con 0x558640e7aa60 2023-12-07T10:22:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.224+0000 7fa1571b2b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558640e84b90 con 0x558640e7b380 2023-12-07T10:22:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.224+0000 7fa1571b2b80 10 client.0 _ll_register_callbacks cb 0x558640d6a8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:22:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.224+0000 7fa14159b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558640e75ad0 0x558640e86040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.224+0000 7fa14159b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558640e75ad0 0x558640e86040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60640/0 (socket says 172.21.15.119:60640) 2023-12-07T10:22:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.224+0000 7fa14159b700 1 -- 172.21.15.119:0/41921525 learned_addr learned my addr 172.21.15.119:0/41921525 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.224+0000 7fa140d9a700 1 --2- 172.21.15.119:0/41921525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558640e7aa60 0x558640e83d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.224+0000 7fa141d9c700 1 --2- 172.21.15.119:0/41921525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558640e7b380 0x558640e84290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.225+0000 7fa14159b700 1 -- 172.21.15.119:0/41921525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558640e7b380 msgr2=0x558640e84290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.225+0000 7fa14159b700 1 --2- 172.21.15.119:0/41921525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558640e7b380 0x558640e84290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.225+0000 7fa14159b700 1 -- 172.21.15.119:0/41921525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558640e7aa60 msgr2=0x558640e83d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.225+0000 7fa14159b700 1 --2- 172.21.15.119:0/41921525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558640e7aa60 0x558640e83d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.225+0000 7fa14159b700 1 -- 172.21.15.119:0/41921525 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13c110ee0 con 0x558640e75ad0 2023-12-07T10:22:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.225+0000 7fa140d9a700 1 --2- 172.21.15.119:0/41921525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558640e7aa60 0x558640e83d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.225+0000 7fa14159b700 1 --2- 172.21.15.119:0/41921525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558640e75ad0 0x558640e86040 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa1380036a0 tx=0x7fa1380097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e78fe07f8f849b1 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.225+0000 7fa124ff9700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:22:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.225+0000 7fa124ff9700 1 -- 172.21.15.119:0/41921525 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa138012ee0 con 0x558640e75ad0 2023-12-07T10:22:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.225+0000 7fa1571b2b80 1 -- 172.21.15.119:0/41921525 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558640d82430 con 0x558640e75ad0 2023-12-07T10:22:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.225+0000 7fa124ff9700 1 -- 172.21.15.119:0/41921525 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fa138000e40 con 0x558640e75ad0 2023-12-07T10:22:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.226+0000 7fa124ff9700 1 -- 172.21.15.119:0/41921525 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa138004e80 con 0x558640e75ad0 2023-12-07T10:22:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.226+0000 7fa1571b2b80 10 client.16437 fetch_fsmap learned FSMap version 33 2023-12-07T10:22:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.226+0000 7fa1571b2b80 1 -- 172.21.15.119:0/41921525 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558640bdbc00 con 0x558640e75ad0 2023-12-07T10:22:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.226+0000 7fa124ff9700 1 -- 172.21.15.119:0/41921525 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558640d82430 con 0x558640e75ad0 2023-12-07T10:22:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.226+0000 7fa124ff9700 1 -- 172.21.15.119:0/41921525 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 33) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fa138053070 con 0x558640e75ad0 2023-12-07T10:22:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.226+0000 7fa1571b2b80 10 client.16437 fetch_fsmap finished waiting for FSMap version 33 2023-12-07T10:22:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.226+0000 7fa1571b2b80 10 client.16437 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:22:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.226+0000 7fa1571b2b80 20 client.16437 populate_metadata read hostname 'smithi119' 2023-12-07T10:22:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.226+0000 7fa1571b2b80 1 --2- 172.21.15.119:0/41921525 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558640ee4ee0 0x558640ee7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.226+0000 7fa1571b2b80 4 client.16437 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:22:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.226+0000 7fa1571b2b80 1 -- 172.21.15.119:0/41921525 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558640c13f10 con 0x558640ee4ee0 2023-12-07T10:22:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.227+0000 7fa140d9a700 1 --2- 172.21.15.119:0/41921525 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558640ee4ee0 0x558640ee7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.227+0000 7fa140d9a700 1 --2- 172.21.15.119:0/41921525 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558640ee4ee0 0x558640ee7300 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64d639d24584dc61 server_cookie=ed42bc0dbb8909e8 in_seq=0 out_seq=0 2023-12-07T10:22:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.227+0000 7fa124ff9700 10 client.16437 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.263+0000 7fa124ff9700 1 -- 172.21.15.119:0/41921525 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558640c13f10 con 0x558640ee4ee0 2023-12-07T10:22:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.263+0000 7fa124ff9700 10 client.16437 handle_command_reply: tid=0 2023-12-07T10:22:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.263+0000 7fa1571b2b80 1 client.16437 shutdown 2023-12-07T10:22:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.264+0000 7fa1571b2b80 1 -- 172.21.15.119:0/41921525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558640e75ad0 msgr2=0x558640e86040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.264+0000 7fa1571b2b80 1 --2- 172.21.15.119:0/41921525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558640e75ad0 0x558640e86040 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa1380036a0 tx=0x7fa1380097f0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.264+0000 7fa1571b2b80 1 -- 172.21.15.119:0/41921525 shutdown_connections 2023-12-07T10:22:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.264+0000 7fa1571b2b80 1 --2- 172.21.15.119:0/41921525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558640e7b380 0x558640e84290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.264+0000 7fa1571b2b80 1 --2- 172.21.15.119:0/41921525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558640e75ad0 0x558640e86040 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.264+0000 7fa1571b2b80 1 --2- 172.21.15.119:0/41921525 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558640ee4ee0 0x558640ee7300 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.264+0000 7fa1571b2b80 1 --2- 172.21.15.119:0/41921525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558640e7aa60 0x558640e83d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.264+0000 7fa1571b2b80 1 -- 172.21.15.119:0/41921525 >> 172.21.15.119:0/41921525 conn(0x558640da4300 msgr2=0x558640e78920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.264+0000 7fa124ff9700 0 client.16437 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.264+0000 7fa1571b2b80 1 -- 172.21.15.119:0/41921525 shutdown_connections 2023-12-07T10:22:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.264+0000 7fa1571b2b80 1 -- 172.21.15.119:0/41921525 wait complete. 2023-12-07T10:22:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.264+0000 7fa1571b2b80 20 client.16437 trim_cache size 0 max 16384 2023-12-07T10:22:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:22:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:22:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:22:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:22:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.265+0000 7fa1437ff700 1 -- 172.21.15.119:0/3763690130 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa1280a1d40 msgr2=0x7fa1280a4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.265+0000 7fa1437ff700 1 --2- 172.21.15.119:0/3763690130 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa1280a1d40 0x7fa1280a4200 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa13400f590 tx=0x7fa134009920 comp rx=0 tx=0).stop 2023-12-07T10:22:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.265+0000 7fa1437ff700 1 -- 172.21.15.119:0/3763690130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa13c1040d0 msgr2=0x7fa13c196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.265+0000 7fa1437ff700 1 --2- 172.21.15.119:0/3763690130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa13c1040d0 0x7fa13c196700 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa12c00e9f0 tx=0x7fa12c00ed00 comp rx=0 tx=0).stop 2023-12-07T10:22:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.265+0000 7fa1437ff700 1 -- 172.21.15.119:0/3763690130 shutdown_connections 2023-12-07T10:22:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.265+0000 7fa1437ff700 1 --2- 172.21.15.119:0/3763690130 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa1280a1d40 0x7fa1280a4200 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.265+0000 7fa1437ff700 1 --2- 172.21.15.119:0/3763690130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa13c109720 0x7fa13c19b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.265+0000 7fa1437ff700 1 --2- 172.21.15.119:0/3763690130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c103710 0x7fa13c1961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.265+0000 7fa1437ff700 1 --2- 172.21.15.119:0/3763690130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa13c1040d0 0x7fa13c196700 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.265+0000 7fa1437ff700 1 -- 172.21.15.119:0/3763690130 >> 172.21.15.119:0/3763690130 conn(0x7fa13c0ff1c0 msgr2=0x7fa13c105210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.265+0000 7fa1437ff700 1 -- 172.21.15.119:0/3763690130 shutdown_connections 2023-12-07T10:22:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:40.265+0000 7fa1437ff700 1 -- 172.21.15.119:0/3763690130 wait complete. 2023-12-07T10:22:40.546 INFO:tasks.workunit.client.0.smithi119.stdout: 1 89188 9.42 MB/sec warmup 53 sec latency 33.023 ms 2023-12-07T10:22:41.546 INFO:tasks.workunit.client.0.smithi119.stdout: 1 91500 9.45 MB/sec warmup 54 sec latency 12.996 ms 2023-12-07T10:22:42.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:41 smithi119 ceph-mon[97908]: pgmap v71: 129 pgs: 129 active+clean; 65 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 270 op/s 2023-12-07T10:22:42.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:41 smithi159 ceph-mon[134726]: pgmap v71: 129 pgs: 129 active+clean; 65 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 270 op/s 2023-12-07T10:22:42.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:41 smithi153 ceph-mon[133625]: pgmap v71: 129 pgs: 129 active+clean; 65 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 270 op/s 2023-12-07T10:22:42.546 INFO:tasks.workunit.client.0.smithi119.stdout: 1 92871 9.42 MB/sec warmup 55 sec latency 10.833 ms 2023-12-07T10:22:43.546 INFO:tasks.workunit.client.0.smithi119.stdout: 1 95201 9.48 MB/sec warmup 56 sec latency 10.606 ms 2023-12-07T10:22:44.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:43 smithi119 ceph-mon[97908]: pgmap v72: 129 pgs: 129 active+clean; 65 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.4 MiB/s wr, 209 op/s 2023-12-07T10:22:44.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:43 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:22:44.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:43 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:22:44.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:43 smithi159 ceph-mon[134726]: pgmap v72: 129 pgs: 129 active+clean; 65 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.4 MiB/s wr, 209 op/s 2023-12-07T10:22:44.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:43 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:22:44.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:43 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:22:44.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:43 smithi153 ceph-mon[133625]: pgmap v72: 129 pgs: 129 active+clean; 65 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.4 MiB/s wr, 209 op/s 2023-12-07T10:22:44.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:43 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:22:44.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:43 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:22:44.546 INFO:tasks.workunit.client.0.smithi119.stdout: 1 96539 9.43 MB/sec warmup 57 sec latency 12.146 ms 2023-12-07T10:22:45.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:44 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:22:45.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:44 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:22:45.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:44 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:22:45.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:44 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:22:45.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:44 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:22:45.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:44 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:22:45.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:44 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:22:45.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:44 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:22:45.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:44 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:22:45.546 INFO:tasks.workunit.client.0.smithi119.stdout: 1 98375 9.40 MB/sec warmup 58 sec latency 107.938 ms 2023-12-07T10:22:46.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:45 smithi159 ceph-mon[134726]: pgmap v73: 129 pgs: 129 active+clean; 65 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.4 MiB/s wr, 209 op/s 2023-12-07T10:22:46.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:45 smithi153 ceph-mon[133625]: pgmap v73: 129 pgs: 129 active+clean; 65 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.4 MiB/s wr, 209 op/s 2023-12-07T10:22:46.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:45 smithi119 ceph-mon[97908]: pgmap v73: 129 pgs: 129 active+clean; 65 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.4 MiB/s wr, 209 op/s 2023-12-07T10:22:46.546 INFO:tasks.workunit.client.0.smithi119.stdout: 1 99867 9.43 MB/sec warmup 59 sec latency 11.661 ms 2023-12-07T10:22:47.546 INFO:tasks.workunit.client.0.smithi119.stdout: 1 102119 9.46 MB/sec warmup 60 sec latency 12.610 ms 2023-12-07T10:22:48.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:47 smithi159 ceph-mon[134726]: pgmap v74: 129 pgs: 129 active+clean; 68 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 326 op/s 2023-12-07T10:22:48.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:47 smithi153 ceph-mon[133625]: pgmap v74: 129 pgs: 129 active+clean; 68 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 326 op/s 2023-12-07T10:22:48.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:47 smithi119 ceph-mon[97908]: pgmap v74: 129 pgs: 129 active+clean; 68 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 326 op/s 2023-12-07T10:22:48.546 INFO:tasks.workunit.client.0.smithi119.stdout: 1 103517 9.44 MB/sec warmup 61 sec latency 10.977 ms 2023-12-07T10:22:49.546 INFO:tasks.workunit.client.0.smithi119.stdout: 1 105839 9.48 MB/sec warmup 62 sec latency 16.379 ms 2023-12-07T10:22:50.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:49 smithi159 ceph-mon[134726]: pgmap v75: 129 pgs: 129 active+clean; 68 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.9 MiB/s wr, 232 op/s 2023-12-07T10:22:50.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:49 smithi153 ceph-mon[133625]: pgmap v75: 129 pgs: 129 active+clean; 68 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.9 MiB/s wr, 232 op/s 2023-12-07T10:22:50.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:49 smithi119 ceph-mon[97908]: pgmap v75: 129 pgs: 129 active+clean; 68 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.9 MiB/s wr, 232 op/s 2023-12-07T10:22:50.546 INFO:tasks.workunit.client.0.smithi119.stdout: 1 107136 9.44 MB/sec warmup 63 sec latency 17.721 ms 2023-12-07T10:22:51.546 INFO:tasks.workunit.client.0.smithi119.stdout: 1 109444 9.49 MB/sec warmup 64 sec latency 15.586 ms 2023-12-07T10:22:52.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:51 smithi159 ceph-mon[134726]: pgmap v76: 129 pgs: 129 active+clean; 74 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 328 op/s 2023-12-07T10:22:52.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:51 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:22:52.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:51 smithi153 ceph-mon[133625]: pgmap v76: 129 pgs: 129 active+clean; 74 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 328 op/s 2023-12-07T10:22:52.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:51 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:22:52.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:51 smithi119 ceph-mon[97908]: pgmap v76: 129 pgs: 129 active+clean; 74 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 328 op/s 2023-12-07T10:22:52.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:51 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:22:52.546 INFO:tasks.workunit.client.0.smithi119.stdout: 1 109802 9.37 MB/sec warmup 65 sec latency 876.239 ms 2023-12-07T10:22:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.422+0000 7f8c15bff700 1 -- 172.21.15.119:0/4062842467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c101075c0 msgr2=0x7f8c101079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.422+0000 7f8c15bff700 1 --2- 172.21.15.119:0/4062842467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c101075c0 0x7f8c101079a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8c000099c0 tx=0x7f8c00009cd0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.423+0000 7f8c15bff700 1 -- 172.21.15.119:0/4062842467 shutdown_connections 2023-12-07T10:22:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.423+0000 7f8c15bff700 1 --2- 172.21.15.119:0/4062842467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c10101ec0 0x7f8c1010e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.423+0000 7f8c15bff700 1 --2- 172.21.15.119:0/4062842467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c101075c0 0x7f8c101079a0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.423+0000 7f8c15bff700 1 --2- 172.21.15.119:0/4062842467 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c10101500 0x7f8c10101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.423+0000 7f8c15bff700 1 -- 172.21.15.119:0/4062842467 >> 172.21.15.119:0/4062842467 conn(0x7f8c100fcfb0 msgr2=0x7f8c100ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.423+0000 7f8c15bff700 1 -- 172.21.15.119:0/4062842467 shutdown_connections 2023-12-07T10:22:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.423+0000 7f8c15bff700 1 -- 172.21.15.119:0/4062842467 wait complete. 2023-12-07T10:22:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.424+0000 7f8c15bff700 1 Processor -- start 2023-12-07T10:22:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.424+0000 7f8c15bff700 1 -- start start 2023-12-07T10:22:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.425+0000 7f8c15bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c10101500 0x7f8c1019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.425+0000 7f8c15bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c10101ec0 0x7f8c1019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.425+0000 7f8c15bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c101075c0 0x7f8c10199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.425+0000 7f8c15bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c10199fc0 con 0x7f8c10101500 2023-12-07T10:22:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.425+0000 7f8c15bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1019a130 con 0x7f8c101075c0 2023-12-07T10:22:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.425+0000 7f8c15bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1019a2f0 con 0x7f8c10101ec0 2023-12-07T10:22:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.425+0000 7f8c0f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c10101500 0x7f8c1019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.425+0000 7f8c0effd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c10101ec0 0x7f8c1019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.425+0000 7f8c0f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c10101500 0x7f8c1019f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57924/0 (socket says 172.21.15.119:57924) 2023-12-07T10:22:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.425+0000 7f8c0f7fe700 1 -- 172.21.15.119:0/2754743422 learned_addr learned my addr 172.21.15.119:0/2754743422 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.425+0000 7f8c0ffff700 1 --2- 172.21.15.119:0/2754743422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c101075c0 0x7f8c10199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.425+0000 7f8c0f7fe700 1 -- 172.21.15.119:0/2754743422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c10101ec0 msgr2=0x7f8c1019fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.425+0000 7f8c0f7fe700 1 --2- 172.21.15.119:0/2754743422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c10101ec0 0x7f8c1019fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.426+0000 7f8c0f7fe700 1 -- 172.21.15.119:0/2754743422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c101075c0 msgr2=0x7f8c10199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.426+0000 7f8c0f7fe700 1 --2- 172.21.15.119:0/2754743422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c101075c0 0x7f8c10199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.426+0000 7f8c0f7fe700 1 -- 172.21.15.119:0/2754743422 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c000095f0 con 0x7f8c10101500 2023-12-07T10:22:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.426+0000 7f8c0f7fe700 1 --2- 172.21.15.119:0/2754743422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c10101500 0x7f8c1019f830 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000b580 tx=0x7f8c0000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.426+0000 7f8c0cff9700 1 -- 172.21.15.119:0/2754743422 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0001c070 con 0x7f8c10101500 2023-12-07T10:22:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.426+0000 7f8c0cff9700 1 -- 172.21.15.119:0/2754743422 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c0000fd80 con 0x7f8c10101500 2023-12-07T10:22:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.426+0000 7f8c15bff700 1 -- 172.21.15.119:0/2754743422 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1019a570 con 0x7f8c10101500 2023-12-07T10:22:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.426+0000 7f8c0cff9700 1 -- 172.21.15.119:0/2754743422 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00022c40 con 0x7f8c10101500 2023-12-07T10:22:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.426+0000 7f8c15bff700 1 -- 172.21.15.119:0/2754743422 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c101a6820 con 0x7f8c10101500 2023-12-07T10:22:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.429+0000 7f8c0cff9700 1 -- 172.21.15.119:0/2754743422 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f8c00020070 con 0x7f8c10101500 2023-12-07T10:22:53.431 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.430+0000 7f8c29594b80 1 Processor -- start 2023-12-07T10:22:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.430+0000 7f8c0cff9700 1 --2- 172.21.15.119:0/2754743422 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8bf80a1c80 0x7f8bf80a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.431+0000 7f8c0cff9700 1 -- 172.21.15.119:0/2754743422 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f8c000ceae0 con 0x7f8c10101500 2023-12-07T10:22:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.431+0000 7f8c29594b80 1 -- start start 2023-12-07T10:22:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.431+0000 7f8c0effd700 1 --2- 172.21.15.119:0/2754743422 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8bf80a1c80 0x7f8bf80a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.431+0000 7f8c29594b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916ee1da60 0x55916ee1de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.431+0000 7f8c29594b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916ee1e380 0x55916ee18540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.431+0000 7f8c29594b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee18ad0 0x55916ee18f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.431+0000 7f8c29594b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55916ed8c760 con 0x55916ee18ad0 2023-12-07T10:22:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.431+0000 7f8c29594b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55916ed73740 con 0x55916ee1da60 2023-12-07T10:22:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.431+0000 7f8c29594b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55916ed71060 con 0x55916ee1e380 2023-12-07T10:22:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.431+0000 7f8c0f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916ee1da60 0x55916ee1de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.431+0000 7f8c0effd700 1 --2- 172.21.15.119:0/2754743422 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8bf80a1c80 0x7f8bf80a4140 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8c1019ab20 tx=0x7f8c04009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.431+0000 7f8c0ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee18ad0 0x55916ee18f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.431+0000 7f8c0f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916ee1da60 0x55916ee1de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:41654/0 (socket says 172.21.15.119:41654) 2023-12-07T10:22:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.432+0000 7f8c0f7fe700 1 -- 172.21.15.119:0/672256168 learned_addr learned my addr 172.21.15.119:0/672256168 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.432+0000 7f8c0effd700 1 --2- 172.21.15.119:0/672256168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916ee1e380 0x55916ee18540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.432+0000 7f8c0f7fe700 1 -- 172.21.15.119:0/672256168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916ee1e380 msgr2=0x55916ee18540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.432+0000 7f8c0f7fe700 1 --2- 172.21.15.119:0/672256168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916ee1e380 0x55916ee18540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.432+0000 7f8c0f7fe700 1 -- 172.21.15.119:0/672256168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee18ad0 msgr2=0x55916ee18f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.432+0000 7f8c0f7fe700 1 --2- 172.21.15.119:0/672256168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee18ad0 0x55916ee18f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.432+0000 7f8c0f7fe700 1 -- 172.21.15.119:0/672256168 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55916ebe95d0 con 0x55916ee1da60 2023-12-07T10:22:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.432+0000 7f8c0ffff700 1 --2- 172.21.15.119:0/672256168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee18ad0 0x55916ee18f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:22:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.432+0000 7f8c0f7fe700 1 --2- 172.21.15.119:0/672256168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916ee1da60 0x55916ee1de40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8c00097a60 tx=0x7f8c0009b120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e87c036329986439 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.432+0000 7f8bf5ffb700 1 -- 172.21.15.119:0/672256168 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0009b560 con 0x55916ee1da60 2023-12-07T10:22:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.432+0000 7f8bf5ffb700 1 -- 172.21.15.119:0/672256168 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c0009f6b0 con 0x55916ee1da60 2023-12-07T10:22:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.432+0000 7f8bf5ffb700 1 -- 172.21.15.119:0/672256168 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0009aa50 con 0x55916ee1da60 2023-12-07T10:22:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.433+0000 7f8c29594b80 1 -- 172.21.15.119:0/672256168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916ee1da60 msgr2=0x55916ee1de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.433+0000 7f8c29594b80 1 --2- 172.21.15.119:0/672256168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916ee1da60 0x55916ee1de40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8c00097a60 tx=0x7f8c0009b120 comp rx=0 tx=0).stop 2023-12-07T10:22:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.433+0000 7f8c29594b80 1 -- 172.21.15.119:0/672256168 shutdown_connections 2023-12-07T10:22:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.433+0000 7f8c29594b80 1 --2- 172.21.15.119:0/672256168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee18ad0 0x55916ee18f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.433+0000 7f8c29594b80 1 --2- 172.21.15.119:0/672256168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916ee1da60 0x55916ee1de40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.433+0000 7f8c29594b80 1 --2- 172.21.15.119:0/672256168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916ee1e380 0x55916ee18540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.433+0000 7f8c29594b80 1 -- 172.21.15.119:0/672256168 >> 172.21.15.119:0/672256168 conn(0x55916ed47300 msgr2=0x55916ee12b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.433+0000 7f8c29594b80 1 -- 172.21.15.119:0/672256168 shutdown_connections 2023-12-07T10:22:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.433+0000 7f8c29594b80 1 -- 172.21.15.119:0/672256168 wait complete. 2023-12-07T10:22:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.434+0000 7f8c29594b80 1 -- start start 2023-12-07T10:22:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.434+0000 7f8c29594b80 1 Processor -- start 2023-12-07T10:22:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.434+0000 7f8c29594b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee18ad0 0x55916ee21730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.434+0000 7f8c29594b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916ee1da60 0x55916ee21c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.434+0000 7f8c29594b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916ee1e380 0x55916ee221b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.435+0000 7f8c29594b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55916ed6c950 con 0x55916ee18ad0 2023-12-07T10:22:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.435+0000 7f8c29594b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55916ed26500 con 0x55916ee1e380 2023-12-07T10:22:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.435+0000 7f8c29594b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55916ed261a0 con 0x55916ee1da60 2023-12-07T10:22:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.435+0000 7f8c29594b80 10 client.0 _ll_register_callbacks cb 0x55916ed0d8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:22:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.435+0000 7f8c0f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee18ad0 0x55916ee21730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.435+0000 7f8c0f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee18ad0 0x55916ee21730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57946/0 (socket says 172.21.15.119:57946) 2023-12-07T10:22:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.435+0000 7f8c0f7fe700 1 -- 172.21.15.119:0/1805526683 learned_addr learned my addr 172.21.15.119:0/1805526683 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.435+0000 7f8c0effd700 1 --2- 172.21.15.119:0/1805526683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916ee1da60 0x55916ee21c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.435+0000 7f8c0ffff700 1 --2- 172.21.15.119:0/1805526683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916ee1e380 0x55916ee221b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.435+0000 7f8c0f7fe700 1 -- 172.21.15.119:0/1805526683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916ee1da60 msgr2=0x55916ee21c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.435+0000 7f8c0f7fe700 1 --2- 172.21.15.119:0/1805526683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916ee1da60 0x55916ee21c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.435+0000 7f8c0f7fe700 1 -- 172.21.15.119:0/1805526683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916ee1e380 msgr2=0x55916ee221b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.435+0000 7f8c0f7fe700 1 --2- 172.21.15.119:0/1805526683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916ee1e380 0x55916ee221b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.435+0000 7f8c0f7fe700 1 -- 172.21.15.119:0/1805526683 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c08003b50 con 0x55916ee18ad0 2023-12-07T10:22:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.435+0000 7f8c0f7fe700 1 --2- 172.21.15.119:0/1805526683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee18ad0 0x55916ee21730 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8c00000c00 tx=0x7f8c00022650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7919da4bbc2fc26f server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.435+0000 7f8beeffd700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:22:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.436+0000 7f8beeffd700 1 -- 172.21.15.119:0/1805526683 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0009b560 con 0x55916ee18ad0 2023-12-07T10:22:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.436+0000 7f8beeffd700 1 -- 172.21.15.119:0/1805526683 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f8c00099900 con 0x55916ee18ad0 2023-12-07T10:22:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.436+0000 7f8c29594b80 1 -- 172.21.15.119:0/1805526683 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55916ee0b6f0 con 0x55916ee18ad0 2023-12-07T10:22:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.436+0000 7f8beeffd700 1 -- 172.21.15.119:0/1805526683 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c0009ac60 con 0x55916ee18ad0 2023-12-07T10:22:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.436+0000 7f8c29594b80 10 client.16449 fetch_fsmap learned FSMap version 35 2023-12-07T10:22:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.436+0000 7f8c29594b80 1 -- 172.21.15.119:0/1805526683 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7f8c101a6820 con 0x55916ee18ad0 2023-12-07T10:22:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.436+0000 7f8beeffd700 1 -- 172.21.15.119:0/1805526683 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55916ee0b6f0 con 0x55916ee18ad0 2023-12-07T10:22:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.436+0000 7f8beeffd700 1 -- 172.21.15.119:0/1805526683 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 35) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8c000b0070 con 0x55916ee18ad0 2023-12-07T10:22:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.436+0000 7f8c29594b80 10 client.16449 fetch_fsmap finished waiting for FSMap version 35 2023-12-07T10:22:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.436+0000 7f8c29594b80 10 client.16449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:22:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.436+0000 7f8c29594b80 20 client.16449 populate_metadata read hostname 'smithi119' 2023-12-07T10:22:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.436+0000 7f8c29594b80 1 --2- 172.21.15.119:0/1805526683 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55916ee316d0 0x55916ee33af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.436+0000 7f8c29594b80 4 client.16449 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:22:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.436+0000 7f8c29594b80 1 -- 172.21.15.119:0/1805526683 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55916ede7740 con 0x55916ee316d0 2023-12-07T10:22:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.437+0000 7f8c0effd700 1 --2- 172.21.15.119:0/1805526683 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55916ee316d0 0x55916ee33af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.437+0000 7f8c0effd700 1 --2- 172.21.15.119:0/1805526683 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55916ee316d0 0x55916ee33af0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57f46d3a008da195 server_cookie=38c9b2f5a0e25aa9 in_seq=0 out_seq=0 2023-12-07T10:22:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.437+0000 7f8beeffd700 10 client.16449 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.438+0000 7f8beeffd700 1 -- 172.21.15.119:0/1805526683 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55916ede7740 con 0x55916ee316d0 2023-12-07T10:22:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.438+0000 7f8beeffd700 10 client.16449 handle_command_reply: tid=0 2023-12-07T10:22:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.438+0000 7f8c29594b80 1 client.16449 shutdown 2023-12-07T10:22:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.439+0000 7f8c29594b80 1 -- 172.21.15.119:0/1805526683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee18ad0 msgr2=0x55916ee21730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.439+0000 7f8c29594b80 1 --2- 172.21.15.119:0/1805526683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee18ad0 0x55916ee21730 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8c00000c00 tx=0x7f8c00022650 comp rx=0 tx=0).stop 2023-12-07T10:22:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.439+0000 7f8c29594b80 1 -- 172.21.15.119:0/1805526683 shutdown_connections 2023-12-07T10:22:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.439+0000 7f8c29594b80 1 --2- 172.21.15.119:0/1805526683 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55916ee316d0 0x55916ee33af0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.439+0000 7f8c29594b80 1 --2- 172.21.15.119:0/1805526683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916ee1e380 0x55916ee221b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.439+0000 7f8c29594b80 1 --2- 172.21.15.119:0/1805526683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee18ad0 0x55916ee21730 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.439+0000 7f8beeffd700 0 client.16449 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.439+0000 7f8c29594b80 1 --2- 172.21.15.119:0/1805526683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916ee1da60 0x55916ee21c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.439+0000 7f8c29594b80 1 -- 172.21.15.119:0/1805526683 >> 172.21.15.119:0/1805526683 conn(0x55916ed47300 msgr2=0x55916ee0d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.439+0000 7f8c29594b80 1 -- 172.21.15.119:0/1805526683 shutdown_connections 2023-12-07T10:22:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.439+0000 7f8c29594b80 1 -- 172.21.15.119:0/1805526683 wait complete. 2023-12-07T10:22:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.439+0000 7f8c29594b80 20 client.16449 trim_cache size 0 max 16384 2023-12-07T10:22:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.454+0000 7f8c29594b80 1 Processor -- start 2023-12-07T10:22:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.454+0000 7f8c29594b80 1 -- start start 2023-12-07T10:22:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.455+0000 7f8c29594b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916ee18ad0 0x55916ee33ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.455+0000 7f8c29594b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee1da60 0x55916ee21fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.455+0000 7f8c29594b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916ee1e380 0x55916ee28a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.455+0000 7f8c29594b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55916ebeb8c0 con 0x55916ee1da60 2023-12-07T10:22:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.455+0000 7f8c29594b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55916ee144b0 con 0x55916ee18ad0 2023-12-07T10:22:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.455+0000 7f8c29594b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55916ee13260 con 0x55916ee1e380 2023-12-07T10:22:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.455+0000 7f8c0effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee1da60 0x55916ee21fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.455+0000 7f8c0ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916ee1e380 0x55916ee28a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.455+0000 7f8c0ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916ee1e380 0x55916ee28a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:52578/0 (socket says 172.21.15.119:52578) 2023-12-07T10:22:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.455+0000 7f8c0effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee1da60 0x55916ee21fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57958/0 (socket says 172.21.15.119:57958) 2023-12-07T10:22:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.455+0000 7f8c0ffff700 1 -- 172.21.15.119:0/1039148870 learned_addr learned my addr 172.21.15.119:0/1039148870 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.455+0000 7f8c0f7fe700 1 --2- 172.21.15.119:0/1039148870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916ee18ad0 0x55916ee33ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.455+0000 7f8c0effd700 1 -- 172.21.15.119:0/1039148870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916ee1e380 msgr2=0x55916ee28a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.455+0000 7f8c0effd700 1 --2- 172.21.15.119:0/1039148870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916ee1e380 0x55916ee28a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.455+0000 7f8c0effd700 1 -- 172.21.15.119:0/1039148870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916ee18ad0 msgr2=0x55916ee33ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.455+0000 7f8c0effd700 1 --2- 172.21.15.119:0/1039148870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916ee18ad0 0x55916ee33ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.455+0000 7f8c0effd700 1 -- 172.21.15.119:0/1039148870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55916eb95d80 con 0x55916ee1da60 2023-12-07T10:22:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.456+0000 7f8c0effd700 1 --2- 172.21.15.119:0/1039148870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee1da60 0x55916ee21fd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x55916ee14890 tx=0x7f8c04014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea49c8977b300b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.456+0000 7f8bf67fc700 1 -- 172.21.15.119:0/1039148870 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c04022030 con 0x55916ee1da60 2023-12-07T10:22:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.456+0000 7f8bf67fc700 1 -- 172.21.15.119:0/1039148870 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c04023040 con 0x55916ee1da60 2023-12-07T10:22:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.456+0000 7f8bf67fc700 1 -- 172.21.15.119:0/1039148870 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c04005280 con 0x55916ee1da60 2023-12-07T10:22:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.456+0000 7f8c29594b80 1 -- 172.21.15.119:0/1039148870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee1da60 msgr2=0x55916ee21fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.456+0000 7f8c29594b80 1 --2- 172.21.15.119:0/1039148870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee1da60 0x55916ee21fd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x55916ee14890 tx=0x7f8c04014000 comp rx=0 tx=0).stop 2023-12-07T10:22:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.456+0000 7f8c29594b80 1 -- 172.21.15.119:0/1039148870 shutdown_connections 2023-12-07T10:22:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.456+0000 7f8c29594b80 1 --2- 172.21.15.119:0/1039148870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916ee1e380 0x55916ee28a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.456+0000 7f8c29594b80 1 --2- 172.21.15.119:0/1039148870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916ee18ad0 0x55916ee33ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.456+0000 7f8c29594b80 1 --2- 172.21.15.119:0/1039148870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee1da60 0x55916ee21fd0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.456+0000 7f8c29594b80 1 -- 172.21.15.119:0/1039148870 >> 172.21.15.119:0/1039148870 conn(0x55916ed47300 msgr2=0x55916ee1b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.457+0000 7f8c29594b80 1 -- 172.21.15.119:0/1039148870 shutdown_connections 2023-12-07T10:22:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.457+0000 7f8c29594b80 1 -- 172.21.15.119:0/1039148870 wait complete. 2023-12-07T10:22:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.457+0000 7f8c29594b80 1 -- start start 2023-12-07T10:22:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.457+0000 7f8c29594b80 1 Processor -- start 2023-12-07T10:22:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.458+0000 7f8c29594b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee18ad0 0x55916ee26a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.458+0000 7f8c29594b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916ee1da60 0x55916ee26f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.458+0000 7f8c29594b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916ee1e380 0x55916ee274c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.458+0000 7f8c29594b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55916ee86c70 con 0x55916ee18ad0 2023-12-07T10:22:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.458+0000 7f8c29594b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55916ee86de0 con 0x55916ee1da60 2023-12-07T10:22:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.458+0000 7f8c29594b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55916ee86fa0 con 0x55916ee1e380 2023-12-07T10:22:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.458+0000 7f8c29594b80 10 client.0 _ll_register_callbacks cb 0x55916ed0d8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:22:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.458+0000 7f8c0f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee18ad0 0x55916ee26a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.458+0000 7f8c0f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee18ad0 0x55916ee26a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57972/0 (socket says 172.21.15.119:57972) 2023-12-07T10:22:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.458+0000 7f8c0f7fe700 1 -- 172.21.15.119:0/459011862 learned_addr learned my addr 172.21.15.119:0/459011862 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:22:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.458+0000 7f8c0ffff700 1 --2- 172.21.15.119:0/459011862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916ee1e380 0x55916ee274c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.459+0000 7f8c0effd700 1 --2- 172.21.15.119:0/459011862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916ee1da60 0x55916ee26f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.459+0000 7f8c0f7fe700 1 -- 172.21.15.119:0/459011862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916ee1e380 msgr2=0x55916ee274c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.459+0000 7f8c0f7fe700 1 --2- 172.21.15.119:0/459011862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916ee1e380 0x55916ee274c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.459+0000 7f8c0f7fe700 1 -- 172.21.15.119:0/459011862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916ee1da60 msgr2=0x55916ee26f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:22:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.459+0000 7f8c0f7fe700 1 --2- 172.21.15.119:0/459011862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916ee1da60 0x55916ee26f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.459+0000 7f8c0f7fe700 1 -- 172.21.15.119:0/459011862 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c04017070 con 0x55916ee18ad0 2023-12-07T10:22:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.459+0000 7f8c0f7fe700 1 --2- 172.21.15.119:0/459011862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee18ad0 0x55916ee26a40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8c000998b0 tx=0x7f8c00005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9f7e834fd83979 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:22:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.459+0000 7f8beeffd700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:22:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.459+0000 7f8beeffd700 1 -- 172.21.15.119:0/459011862 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0009a4c0 con 0x55916ee18ad0 2023-12-07T10:22:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.459+0000 7f8beeffd700 1 -- 172.21.15.119:0/459011862 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f8c000a35e0 con 0x55916ee18ad0 2023-12-07T10:22:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.459+0000 7f8c29594b80 1 -- 172.21.15.119:0/459011862 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c0009f6b0 con 0x55916ee18ad0 2023-12-07T10:22:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.460+0000 7f8beeffd700 1 -- 172.21.15.119:0/459011862 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c00097350 con 0x55916ee18ad0 2023-12-07T10:22:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.460+0000 7f8c29594b80 10 client.16455 fetch_fsmap learned FSMap version 35 2023-12-07T10:22:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.460+0000 7f8c29594b80 1 -- 172.21.15.119:0/459011862 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55916eb7ec00 con 0x55916ee18ad0 2023-12-07T10:22:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.460+0000 7f8beeffd700 1 -- 172.21.15.119:0/459011862 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c0009f6b0 con 0x55916ee18ad0 2023-12-07T10:22:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.460+0000 7f8beeffd700 1 -- 172.21.15.119:0/459011862 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 35) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8c00097c70 con 0x55916ee18ad0 2023-12-07T10:22:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.460+0000 7f8c29594b80 10 client.16455 fetch_fsmap finished waiting for FSMap version 35 2023-12-07T10:22:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.460+0000 7f8c29594b80 10 client.16455 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:22:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.460+0000 7f8c29594b80 20 client.16455 populate_metadata read hostname 'smithi119' 2023-12-07T10:22:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.460+0000 7f8c29594b80 1 --2- 172.21.15.119:0/459011862 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55916ee88390 0x55916ee8a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:22:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.460+0000 7f8c29594b80 4 client.16455 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:22:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.460+0000 7f8c29594b80 1 -- 172.21.15.119:0/459011862 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55916ed32f70 con 0x55916ee88390 2023-12-07T10:22:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.460+0000 7f8c0effd700 1 --2- 172.21.15.119:0/459011862 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55916ee88390 0x55916ee8a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:22:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.461+0000 7f8c0effd700 1 --2- 172.21.15.119:0/459011862 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55916ee88390 0x55916ee8a7b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43e5244261cb8d51 server_cookie=80e3a81dc0d800c7 in_seq=0 out_seq=0 2023-12-07T10:22:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.461+0000 7f8beeffd700 10 client.16455 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.537+0000 7f8beeffd700 1 -- 172.21.15.119:0/459011862 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55916ed32f70 con 0x55916ee88390 2023-12-07T10:22:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.537+0000 7f8beeffd700 10 client.16455 handle_command_reply: tid=0 2023-12-07T10:22:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.537+0000 7f8c29594b80 1 client.16455 shutdown 2023-12-07T10:22:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.537+0000 7f8c29594b80 1 -- 172.21.15.119:0/459011862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee18ad0 msgr2=0x55916ee26a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.537+0000 7f8c29594b80 1 --2- 172.21.15.119:0/459011862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee18ad0 0x55916ee26a40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8c000998b0 tx=0x7f8c00005a90 comp rx=0 tx=0).stop 2023-12-07T10:22:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.537+0000 7f8c29594b80 1 -- 172.21.15.119:0/459011862 shutdown_connections 2023-12-07T10:22:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.537+0000 7f8c29594b80 1 --2- 172.21.15.119:0/459011862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916ee1e380 0x55916ee274c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.537+0000 7f8c29594b80 1 --2- 172.21.15.119:0/459011862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55916ee18ad0 0x55916ee26a40 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.537+0000 7f8c29594b80 1 --2- 172.21.15.119:0/459011862 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55916ee88390 0x55916ee8a7b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.537+0000 7f8c29594b80 1 --2- 172.21.15.119:0/459011862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916ee1da60 0x55916ee26f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.537+0000 7f8beeffd700 0 client.16455 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:22:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.537+0000 7f8c29594b80 1 -- 172.21.15.119:0/459011862 >> 172.21.15.119:0/459011862 conn(0x55916ed47300 msgr2=0x55916ee1c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.537+0000 7f8c29594b80 1 -- 172.21.15.119:0/459011862 shutdown_connections 2023-12-07T10:22:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.537+0000 7f8c29594b80 1 -- 172.21.15.119:0/459011862 wait complete. 2023-12-07T10:22:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.537+0000 7f8c29594b80 20 client.16455 trim_cache size 0 max 16384 2023-12-07T10:22:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:22:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:22:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:22:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:22:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.538+0000 7f8c15bff700 1 -- 172.21.15.119:0/2754743422 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8bf80a1c80 msgr2=0x7f8bf80a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.538+0000 7f8c15bff700 1 --2- 172.21.15.119:0/2754743422 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8bf80a1c80 0x7f8bf80a4140 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8c1019ab20 tx=0x7f8c04009260 comp rx=0 tx=0).stop 2023-12-07T10:22:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.538+0000 7f8c15bff700 1 -- 172.21.15.119:0/2754743422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c10101500 msgr2=0x7f8c1019f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:22:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.538+0000 7f8c15bff700 1 --2- 172.21.15.119:0/2754743422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c10101500 0x7f8c1019f830 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000b580 tx=0x7f8c0000f690 comp rx=0 tx=0).stop 2023-12-07T10:22:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.538+0000 7f8c15bff700 1 -- 172.21.15.119:0/2754743422 shutdown_connections 2023-12-07T10:22:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.538+0000 7f8c15bff700 1 --2- 172.21.15.119:0/2754743422 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8bf80a1c80 0x7f8bf80a4140 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.538+0000 7f8c15bff700 1 --2- 172.21.15.119:0/2754743422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c101075c0 0x7f8c10199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.538+0000 7f8c15bff700 1 --2- 172.21.15.119:0/2754743422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c10101500 0x7f8c1019f830 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.538+0000 7f8c15bff700 1 --2- 172.21.15.119:0/2754743422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c10101ec0 0x7f8c1019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:22:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.538+0000 7f8c15bff700 1 -- 172.21.15.119:0/2754743422 >> 172.21.15.119:0/2754743422 conn(0x7f8c100fcfb0 msgr2=0x7f8c100fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:22:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.538+0000 7f8c15bff700 1 -- 172.21.15.119:0/2754743422 shutdown_connections 2023-12-07T10:22:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:22:53.538+0000 7f8c15bff700 1 -- 172.21.15.119:0/2754743422 wait complete. 2023-12-07T10:22:53.546 INFO:tasks.workunit.client.0.smithi119.stdout: 1 109802 9.23 MB/sec warmup 66 sec latency 1876.302 ms 2023-12-07T10:22:54.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:53 smithi159 ceph-mon[134726]: pgmap v77: 129 pgs: 129 active+clean; 74 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 213 op/s 2023-12-07T10:22:54.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:53 smithi153 ceph-mon[133625]: pgmap v77: 129 pgs: 129 active+clean; 74 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 213 op/s 2023-12-07T10:22:54.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:53 smithi119 ceph-mon[97908]: pgmap v77: 129 pgs: 129 active+clean; 74 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 213 op/s 2023-12-07T10:22:54.547 INFO:tasks.workunit.client.0.smithi119.stdout: 1 109802 9.09 MB/sec warmup 67 sec latency 2876.418 ms 2023-12-07T10:22:55.547 INFO:tasks.workunit.client.0.smithi119.stdout: 1 109815 8.96 MB/sec warmup 68 sec latency 3626.325 ms 2023-12-07T10:22:56.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:55 smithi159 ceph-mon[134726]: pgmap v78: 129 pgs: 129 active+clean; 74 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 213 op/s 2023-12-07T10:22:56.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:55 smithi153 ceph-mon[133625]: pgmap v78: 129 pgs: 129 active+clean; 74 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 213 op/s 2023-12-07T10:22:56.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:55 smithi119 ceph-mon[97908]: pgmap v78: 129 pgs: 129 active+clean; 74 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 213 op/s 2023-12-07T10:22:56.547 INFO:tasks.workunit.client.0.smithi119.stdout: 1 109815 8.83 MB/sec warmup 69 sec latency 1231.108 ms 2023-12-07T10:22:57.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:57 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:22:57.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:57 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:22:57.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:57 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:22:57.547 INFO:tasks.workunit.client.0.smithi119.stdout: 1 109815 8.70 MB/sec warmup 70 sec latency 2231.264 ms 2023-12-07T10:22:58.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:22:58 smithi153 ceph-mon[133625]: pgmap v79: 129 pgs: 129 active+clean; 74 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 248 op/s 2023-12-07T10:22:58.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:22:58 smithi119 ceph-mon[97908]: pgmap v79: 129 pgs: 129 active+clean; 74 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 248 op/s 2023-12-07T10:22:58.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:22:58 smithi159 ceph-mon[134726]: pgmap v79: 129 pgs: 129 active+clean; 74 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 248 op/s 2023-12-07T10:22:58.547 INFO:tasks.workunit.client.0.smithi119.stdout: 1 110389 8.64 MB/sec warmup 71 sec latency 2748.837 ms 2023-12-07T10:22:59.547 INFO:tasks.workunit.client.0.smithi119.stdout: 1 112441 8.62 MB/sec warmup 72 sec latency 16.481 ms 2023-12-07T10:23:00.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:00 smithi153 ceph-mon[133625]: pgmap v80: 129 pgs: 129 active+clean; 74 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 786 KiB/s wr, 131 op/s 2023-12-07T10:23:00.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:00 smithi119 ceph-mon[97908]: pgmap v80: 129 pgs: 129 active+clean; 74 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 786 KiB/s wr, 131 op/s 2023-12-07T10:23:00.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:00 smithi159 ceph-mon[134726]: pgmap v80: 129 pgs: 129 active+clean; 74 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 786 KiB/s wr, 131 op/s 2023-12-07T10:23:00.547 INFO:tasks.workunit.client.0.smithi119.stdout: 1 113796 8.67 MB/sec warmup 73 sec latency 333.409 ms 2023-12-07T10:23:00.810 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-07T10:23:01.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.068+0000 7fdcce5ff700 1 -- 172.21.15.119:0/1379138854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcc8101550 msgr2=0x7fdcc81019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:01.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.068+0000 7fdcce5ff700 1 --2- 172.21.15.119:0/1379138854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcc8101550 0x7fdcc81019d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0098f0 tx=0x7fdcbc009c00 comp rx=0 tx=0).stop 2023-12-07T10:23:01.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.068+0000 7fdcce5ff700 1 -- 172.21.15.119:0/1379138854 shutdown_connections 2023-12-07T10:23:01.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.068+0000 7fdcce5ff700 1 --2- 172.21.15.119:0/1379138854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcc8101f10 0x7fdcc810e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.068+0000 7fdcce5ff700 1 --2- 172.21.15.119:0/1379138854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcc8107610 0x7fdcc81079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.068+0000 7fdcce5ff700 1 --2- 172.21.15.119:0/1379138854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcc8101550 0x7fdcc81019d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.068+0000 7fdcce5ff700 1 -- 172.21.15.119:0/1379138854 >> 172.21.15.119:0/1379138854 conn(0x7fdcc80fd020 msgr2=0x7fdcc80ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:01.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.068+0000 7fdcce5ff700 1 -- 172.21.15.119:0/1379138854 shutdown_connections 2023-12-07T10:23:01.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.068+0000 7fdcce5ff700 1 -- 172.21.15.119:0/1379138854 wait complete. 2023-12-07T10:23:01.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.069+0000 7fdcce5ff700 1 Processor -- start 2023-12-07T10:23:01.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.069+0000 7fdcce5ff700 1 -- start start 2023-12-07T10:23:01.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.070+0000 7fdcce5ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcc8101550 0x7fdcc8073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.070+0000 7fdcce5ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcc8101f10 0x7fdcc8073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.070+0000 7fdcce5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcc8107610 0x7fdcc8074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.070+0000 7fdcce5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc80758e0 con 0x7fdcc8107610 2023-12-07T10:23:01.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.070+0000 7fdcce5ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc81aa400 con 0x7fdcc8101550 2023-12-07T10:23:01.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.070+0000 7fdcce5ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc81aa5c0 con 0x7fdcc8101f10 2023-12-07T10:23:01.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.071+0000 7fdcc7fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcc8101550 0x7fdcc8073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.071+0000 7fdcc7fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcc8101550 0x7fdcc8073910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:41700/0 (socket says 172.21.15.119:41700) 2023-12-07T10:23:01.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.071+0000 7fdcc7fff700 1 -- 172.21.15.119:0/3328351365 learned_addr learned my addr 172.21.15.119:0/3328351365 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:01.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.071+0000 7fdcc77fe700 1 --2- 172.21.15.119:0/3328351365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcc8101f10 0x7fdcc8073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.071+0000 7fdcccb9c700 1 --2- 172.21.15.119:0/3328351365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcc8107610 0x7fdcc8074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.071+0000 7fdcc7fff700 1 -- 172.21.15.119:0/3328351365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcc8101f10 msgr2=0x7fdcc8073e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:01.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.071+0000 7fdcc7fff700 1 --2- 172.21.15.119:0/3328351365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcc8101f10 0x7fdcc8073e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.071+0000 7fdcc7fff700 1 -- 172.21.15.119:0/3328351365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcc8107610 msgr2=0x7fdcc8074390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:01.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.071+0000 7fdcc7fff700 1 --2- 172.21.15.119:0/3328351365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcc8107610 0x7fdcc8074390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.071+0000 7fdcc7fff700 1 -- 172.21.15.119:0/3328351365 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb80095f0 con 0x7fdcc8101550 2023-12-07T10:23:01.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.071+0000 7fdcccb9c700 1 --2- 172.21.15.119:0/3328351365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcc8107610 0x7fdcc8074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:23:01.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.072+0000 7fdcc7fff700 1 --2- 172.21.15.119:0/3328351365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcc8101550 0x7fdcc8073910 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800ec10 tx=0x7fdcb800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:01.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.072+0000 7fdcc57fa700 1 -- 172.21.15.119:0/3328351365 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb800cd40 con 0x7fdcc8101550 2023-12-07T10:23:01.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.072+0000 7fdcce5ff700 1 -- 172.21.15.119:0/3328351365 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcbc009520 con 0x7fdcc8101550 2023-12-07T10:23:01.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.072+0000 7fdcc57fa700 1 -- 172.21.15.119:0/3328351365 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcb80044e0 con 0x7fdcc8101550 2023-12-07T10:23:01.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.072+0000 7fdcc57fa700 1 -- 172.21.15.119:0/3328351365 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8010690 con 0x7fdcc8101550 2023-12-07T10:23:01.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.072+0000 7fdcce5ff700 1 -- 172.21.15.119:0/3328351365 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc81aabf0 con 0x7fdcc8101550 2023-12-07T10:23:01.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.075+0000 7fdcc57fa700 1 -- 172.21.15.119:0/3328351365 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fdcb8003710 con 0x7fdcc8101550 2023-12-07T10:23:01.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.076+0000 7fdcc57fa700 1 --2- 172.21.15.119:0/3328351365 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fdcb00a1c80 0x7fdcb00a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.077+0000 7fdcc57fa700 1 -- 172.21.15.119:0/3328351365 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fdcb8014070 con 0x7fdcc8101550 2023-12-07T10:23:01.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.077+0000 7fdcc77fe700 1 --2- 172.21.15.119:0/3328351365 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fdcb00a1c80 0x7fdcb00a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.077+0000 7fdce1eecb80 1 Processor -- start 2023-12-07T10:23:01.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.077+0000 7fdce1eecb80 1 -- start start 2023-12-07T10:23:01.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.077+0000 7fdcc77fe700 1 --2- 172.21.15.119:0/3328351365 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fdcb00a1c80 0x7fdcb00a4140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0098c0 tx=0x7fdcbc019040 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:01.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.077+0000 7fdce1eecb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8ade0bb40 0x55f8ade0bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.077+0000 7fdce1eecb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8ade0c460 0x55f8ade0fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.077+0000 7fdce1eecb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ade10150 0x55f8ade10570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.077+0000 7fdce1eecb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8add7f7c0 con 0x55f8ade0c460 2023-12-07T10:23:01.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.077+0000 7fdce1eecb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8add667a0 con 0x55f8ade10150 2023-12-07T10:23:01.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.077+0000 7fdce1eecb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8add640c0 con 0x55f8ade0bb40 2023-12-07T10:23:01.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.078+0000 7fdcccb9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ade10150 0x55f8ade10570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.078+0000 7fdcc77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8ade0c460 0x55f8ade0fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.078+0000 7fdcccb9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ade10150 0x55f8ade10570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:41702/0 (socket says 172.21.15.119:41702) 2023-12-07T10:23:01.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.078+0000 7fdcccb9c700 1 -- 172.21.15.119:0/562002693 learned_addr learned my addr 172.21.15.119:0/562002693 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:01.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.078+0000 7fdcc7fff700 1 --2- 172.21.15.119:0/562002693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8ade0bb40 0x55f8ade0bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.078+0000 7fdcccb9c700 1 -- 172.21.15.119:0/562002693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8ade0bb40 msgr2=0x55f8ade0bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:01.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.078+0000 7fdcccb9c700 1 --2- 172.21.15.119:0/562002693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8ade0bb40 0x55f8ade0bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.078+0000 7fdcccb9c700 1 -- 172.21.15.119:0/562002693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8ade0c460 msgr2=0x55f8ade0fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:01.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.078+0000 7fdcccb9c700 1 --2- 172.21.15.119:0/562002693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8ade0c460 0x55f8ade0fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.078+0000 7fdcccb9c700 1 -- 172.21.15.119:0/562002693 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8adbdc630 con 0x55f8ade10150 2023-12-07T10:23:01.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.078+0000 7fdcccb9c700 1 --2- 172.21.15.119:0/562002693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ade10150 0x55f8ade10570 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009440 tx=0x7fdcc0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62b3c11a03c7cd01 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:01.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.079+0000 7fdcae7fc700 1 -- 172.21.15.119:0/562002693 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc000e070 con 0x55f8ade10150 2023-12-07T10:23:01.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.079+0000 7fdcae7fc700 1 -- 172.21.15.119:0/562002693 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcc000f040 con 0x55f8ade10150 2023-12-07T10:23:01.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.079+0000 7fdcae7fc700 1 -- 172.21.15.119:0/562002693 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0015c90 con 0x55f8ade10150 2023-12-07T10:23:01.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.080+0000 7fdce1eecb80 1 -- 172.21.15.119:0/562002693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ade10150 msgr2=0x55f8ade10570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:01.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.080+0000 7fdce1eecb80 1 --2- 172.21.15.119:0/562002693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ade10150 0x55f8ade10570 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009440 tx=0x7fdcc0009920 comp rx=0 tx=0).stop 2023-12-07T10:23:01.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.080+0000 7fdce1eecb80 1 -- 172.21.15.119:0/562002693 shutdown_connections 2023-12-07T10:23:01.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.080+0000 7fdce1eecb80 1 --2- 172.21.15.119:0/562002693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ade10150 0x55f8ade10570 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.080+0000 7fdce1eecb80 1 --2- 172.21.15.119:0/562002693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8ade0bb40 0x55f8ade0bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.080+0000 7fdce1eecb80 1 --2- 172.21.15.119:0/562002693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8ade0c460 0x55f8ade0fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.080+0000 7fdce1eecb80 1 -- 172.21.15.119:0/562002693 >> 172.21.15.119:0/562002693 conn(0x55f8add3a360 msgr2=0x55f8ade06130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:01.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.080+0000 7fdce1eecb80 1 -- 172.21.15.119:0/562002693 shutdown_connections 2023-12-07T10:23:01.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.080+0000 7fdce1eecb80 1 -- 172.21.15.119:0/562002693 wait complete. 2023-12-07T10:23:01.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.080+0000 7fdce1eecb80 1 -- start start 2023-12-07T10:23:01.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.081+0000 7fdce1eecb80 1 Processor -- start 2023-12-07T10:23:01.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.081+0000 7fdce1eecb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8ade0bb40 0x55f8ade178f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.081+0000 7fdce1eecb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8ade0c460 0x55f8ade14c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.081+0000 7fdce1eecb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ade10150 0x55f8ade15390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.081+0000 7fdce1eecb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8add5f9b0 con 0x55f8ade0bb40 2023-12-07T10:23:01.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.081+0000 7fdce1eecb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8add19560 con 0x55f8ade10150 2023-12-07T10:23:01.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.081+0000 7fdce1eecb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8add19200 con 0x55f8ade0c460 2023-12-07T10:23:01.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.081+0000 7fdce1eecb80 10 client.0 _ll_register_callbacks cb 0x55f8add00940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:01.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.081+0000 7fdcccb9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ade10150 0x55f8ade15390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.081+0000 7fdcccb9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ade10150 0x55f8ade15390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:41716/0 (socket says 172.21.15.119:41716) 2023-12-07T10:23:01.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.081+0000 7fdcccb9c700 1 -- 172.21.15.119:0/2449666251 learned_addr learned my addr 172.21.15.119:0/2449666251 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:01.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.081+0000 7fdcc7fff700 1 --2- 172.21.15.119:0/2449666251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8ade0bb40 0x55f8ade178f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.082+0000 7fdcc77fe700 1 --2- 172.21.15.119:0/2449666251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8ade0c460 0x55f8ade14c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.082+0000 7fdcccb9c700 1 -- 172.21.15.119:0/2449666251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8ade0c460 msgr2=0x55f8ade14c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:01.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.082+0000 7fdcccb9c700 1 --2- 172.21.15.119:0/2449666251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8ade0c460 0x55f8ade14c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.082+0000 7fdcccb9c700 1 -- 172.21.15.119:0/2449666251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8ade0bb40 msgr2=0x55f8ade178f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:01.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.082+0000 7fdcccb9c700 1 --2- 172.21.15.119:0/2449666251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8ade0bb40 0x55f8ade178f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.082+0000 7fdcccb9c700 1 -- 172.21.15.119:0/2449666251 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc00090f0 con 0x55f8ade10150 2023-12-07T10:23:01.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.082+0000 7fdcc7fff700 1 --2- 172.21.15.119:0/2449666251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8ade0bb40 0x55f8ade178f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:23:01.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.082+0000 7fdcccb9c700 1 --2- 172.21.15.119:0/2449666251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ade10150 0x55f8ade15390 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0007ac0 tx=0x7fdcc0016a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a928be9ebf54f005 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:01.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.082+0000 7fdca77fe700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:23:01.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.082+0000 7fdca77fe700 1 -- 172.21.15.119:0/2449666251 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc000e040 con 0x55f8ade10150 2023-12-07T10:23:01.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.083+0000 7fdce1eecb80 1 -- 172.21.15.119:0/2449666251 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f8addfeb00 con 0x55f8ade10150 2023-12-07T10:23:01.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.083+0000 7fdca77fe700 1 -- 172.21.15.119:0/2449666251 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fdcc0003c00 con 0x55f8ade10150 2023-12-07T10:23:01.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.083+0000 7fdca77fe700 1 -- 172.21.15.119:0/2449666251 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdcc00095e0 con 0x55f8ade10150 2023-12-07T10:23:01.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.083+0000 7fdce1eecb80 10 client.26486 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:01.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.083+0000 7fdce1eecb80 1 -- 172.21.15.119:0/2449666251 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55f8adb88de0 con 0x55f8ade10150 2023-12-07T10:23:01.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.083+0000 7fdca77fe700 1 -- 172.21.15.119:0/2449666251 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f8addfeb00 con 0x55f8ade10150 2023-12-07T10:23:01.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.083+0000 7fdca77fe700 1 -- 172.21.15.119:0/2449666251 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fdcc001c050 con 0x55f8ade10150 2023-12-07T10:23:01.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.084+0000 7fdce1eecb80 10 client.26486 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:01.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.084+0000 7fdce1eecb80 10 client.26486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:23:01.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.084+0000 7fdce1eecb80 20 client.26486 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:01.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.084+0000 7fdce1eecb80 1 --2- 172.21.15.119:0/2449666251 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f8ade253c0 0x55f8ade277e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.084+0000 7fdce1eecb80 4 client.26486 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:23:01.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.084+0000 7fdce1eecb80 1 -- 172.21.15.119:0/2449666251 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f8addda7a0 con 0x55f8ade253c0 2023-12-07T10:23:01.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.084+0000 7fdcc7fff700 1 --2- 172.21.15.119:0/2449666251 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f8ade253c0 0x55f8ade277e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.084+0000 7fdcc7fff700 1 --2- 172.21.15.119:0/2449666251 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f8ade253c0 0x55f8ade277e0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=890c88a98790959e server_cookie=66ea61962fb1a92b in_seq=0 out_seq=0 2023-12-07T10:23:01.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.084+0000 7fdca77fe700 10 client.26486 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:01.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.085+0000 7fdca77fe700 1 -- 172.21.15.119:0/2449666251 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f8addda7a0 con 0x55f8ade253c0 2023-12-07T10:23:01.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.085+0000 7fdca77fe700 10 client.26486 handle_command_reply: tid=0 2023-12-07T10:23:01.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.085+0000 7fdce1eecb80 1 client.26486 shutdown 2023-12-07T10:23:01.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.085+0000 7fdce1eecb80 1 -- 172.21.15.119:0/2449666251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ade10150 msgr2=0x55f8ade15390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:01.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.085+0000 7fdce1eecb80 1 --2- 172.21.15.119:0/2449666251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ade10150 0x55f8ade15390 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0007ac0 tx=0x7fdcc0016a70 comp rx=0 tx=0).stop 2023-12-07T10:23:01.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.085+0000 7fdce1eecb80 1 -- 172.21.15.119:0/2449666251 shutdown_connections 2023-12-07T10:23:01.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.085+0000 7fdce1eecb80 1 --2- 172.21.15.119:0/2449666251 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f8ade253c0 0x55f8ade277e0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.085+0000 7fdce1eecb80 1 --2- 172.21.15.119:0/2449666251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ade10150 0x55f8ade15390 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.085+0000 7fdce1eecb80 1 --2- 172.21.15.119:0/2449666251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8ade0bb40 0x55f8ade178f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.085+0000 7fdca77fe700 0 client.26486 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:01.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.085+0000 7fdce1eecb80 1 --2- 172.21.15.119:0/2449666251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8ade0c460 0x55f8ade14c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.085+0000 7fdce1eecb80 1 -- 172.21.15.119:0/2449666251 >> 172.21.15.119:0/2449666251 conn(0x55f8add3a360 msgr2=0x55f8ade11620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:01.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.085+0000 7fdce1eecb80 1 -- 172.21.15.119:0/2449666251 shutdown_connections 2023-12-07T10:23:01.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.086+0000 7fdce1eecb80 1 -- 172.21.15.119:0/2449666251 wait complete. 2023-12-07T10:23:01.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.086+0000 7fdce1eecb80 20 client.26486 trim_cache size 0 max 16384 2023-12-07T10:23:01.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.105+0000 7fdce1eecb80 1 Processor -- start 2023-12-07T10:23:01.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.105+0000 7fdce1eecb80 1 -- start start 2023-12-07T10:23:01.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.106+0000 7fdce1eecb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8ade0bb40 0x55f8ade07940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.106+0000 7fdce1eecb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8ade0c460 0x55f8ade06000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.106+0000 7fdce1eecb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ade10150 0x55f8ade2c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.106+0000 7fdce1eecb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8adbde920 con 0x55f8ade0c460 2023-12-07T10:23:01.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.106+0000 7fdce1eecb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8ade06ff0 con 0x55f8ade10150 2023-12-07T10:23:01.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.106+0000 7fdce1eecb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8ade131a0 con 0x55f8ade0bb40 2023-12-07T10:23:01.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.106+0000 7fdcc7fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8ade0bb40 0x55f8ade07940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.106+0000 7fdcc7fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8ade0bb40 0x55f8ade07940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:52644/0 (socket says 172.21.15.119:52644) 2023-12-07T10:23:01.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.106+0000 7fdcccb9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ade10150 0x55f8ade2c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.106+0000 7fdcc7fff700 1 -- 172.21.15.119:0/598459949 learned_addr learned my addr 172.21.15.119:0/598459949 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:01.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.107+0000 7fdcccb9c700 1 -- 172.21.15.119:0/598459949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8ade0bb40 msgr2=0x55f8ade07940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:01.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.107+0000 7fdcccb9c700 1 --2- 172.21.15.119:0/598459949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8ade0bb40 0x55f8ade07940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.107+0000 7fdcccb9c700 1 -- 172.21.15.119:0/598459949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8ade0c460 msgr2=0x55f8ade06000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:01.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.107+0000 7fdcccb9c700 1 --2- 172.21.15.119:0/598459949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8ade0c460 0x55f8ade06000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.107+0000 7fdcccb9c700 1 -- 172.21.15.119:0/598459949 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8adb71c60 con 0x55f8ade10150 2023-12-07T10:23:01.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.107+0000 7fdcc7fff700 1 --2- 172.21.15.119:0/598459949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8ade0bb40 0x55f8ade07940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:01.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.107+0000 7fdcccb9c700 1 --2- 172.21.15.119:0/598459949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ade10150 0x55f8ade2c120 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00094f0 tx=0x7fdcc0016f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d0448b0d470580d server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:01.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.107+0000 7fdcaeffd700 1 -- 172.21.15.119:0/598459949 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc00095e0 con 0x55f8ade10150 2023-12-07T10:23:01.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.107+0000 7fdcaeffd700 1 -- 172.21.15.119:0/598459949 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcc0016b90 con 0x55f8ade10150 2023-12-07T10:23:01.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.107+0000 7fdcaeffd700 1 -- 172.21.15.119:0/598459949 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc00165a0 con 0x55f8ade10150 2023-12-07T10:23:01.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.108+0000 7fdce1eecb80 1 -- 172.21.15.119:0/598459949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ade10150 msgr2=0x55f8ade2c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:01.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.108+0000 7fdce1eecb80 1 --2- 172.21.15.119:0/598459949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ade10150 0x55f8ade2c120 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00094f0 tx=0x7fdcc0016f50 comp rx=0 tx=0).stop 2023-12-07T10:23:01.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.108+0000 7fdce1eecb80 1 -- 172.21.15.119:0/598459949 shutdown_connections 2023-12-07T10:23:01.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.108+0000 7fdce1eecb80 1 --2- 172.21.15.119:0/598459949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ade10150 0x55f8ade2c120 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.108+0000 7fdce1eecb80 1 --2- 172.21.15.119:0/598459949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8ade0bb40 0x55f8ade07940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.108+0000 7fdce1eecb80 1 --2- 172.21.15.119:0/598459949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8ade0c460 0x55f8ade06000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.108+0000 7fdce1eecb80 1 -- 172.21.15.119:0/598459949 >> 172.21.15.119:0/598459949 conn(0x55f8add3a360 msgr2=0x55f8ade17620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:01.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.108+0000 7fdce1eecb80 1 -- 172.21.15.119:0/598459949 shutdown_connections 2023-12-07T10:23:01.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.108+0000 7fdce1eecb80 1 -- 172.21.15.119:0/598459949 wait complete. 2023-12-07T10:23:01.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.109+0000 7fdce1eecb80 1 -- start start 2023-12-07T10:23:01.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.109+0000 7fdce1eecb80 1 Processor -- start 2023-12-07T10:23:01.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.109+0000 7fdce1eecb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8ade0bb40 0x55f8ade1a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.109+0000 7fdce1eecb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8ade0c460 0x55f8ade1a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.110+0000 7fdce1eecb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ade10150 0x55f8ade7a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.110+0000 7fdce1eecb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8ade1b180 con 0x55f8ade0c460 2023-12-07T10:23:01.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.110+0000 7fdce1eecb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8ade7ade0 con 0x55f8ade10150 2023-12-07T10:23:01.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.110+0000 7fdce1eecb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8ade7af20 con 0x55f8ade0bb40 2023-12-07T10:23:01.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.110+0000 7fdce1eecb80 10 client.0 _ll_register_callbacks cb 0x55f8add00940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:01.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.110+0000 7fdcc7fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8ade0bb40 0x55f8ade1a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.110+0000 7fdcc7fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8ade0bb40 0x55f8ade1a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:52646/0 (socket says 172.21.15.119:52646) 2023-12-07T10:23:01.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.110+0000 7fdcc7fff700 1 -- 172.21.15.119:0/2209577995 learned_addr learned my addr 172.21.15.119:0/2209577995 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:01.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.110+0000 7fdcccb9c700 1 --2- 172.21.15.119:0/2209577995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ade10150 0x55f8ade7a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.110+0000 7fdcc7fff700 1 -- 172.21.15.119:0/2209577995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ade10150 msgr2=0x55f8ade7a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:01.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.110+0000 7fdcc7fff700 1 --2- 172.21.15.119:0/2209577995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ade10150 0x55f8ade7a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.110+0000 7fdcc7fff700 1 -- 172.21.15.119:0/2209577995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8ade0c460 msgr2=0x55f8ade1a8a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-07T10:23:01.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.110+0000 7fdcc7fff700 1 --2- 172.21.15.119:0/2209577995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8ade0c460 0x55f8ade1a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.110+0000 7fdcc7fff700 1 -- 172.21.15.119:0/2209577995 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc81aabf0 con 0x55f8ade0bb40 2023-12-07T10:23:01.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.111+0000 7fdcc7fff700 1 --2- 172.21.15.119:0/2209577995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8ade0bb40 0x55f8ade1a360 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80976e0 tx=0x7fdcb8003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a104dc8e88475f0c server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:01.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.111+0000 7fdca77fe700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:23:01.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.111+0000 7fdca77fe700 1 -- 172.21.15.119:0/2209577995 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb80662c0 con 0x55f8ade0bb40 2023-12-07T10:23:01.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.111+0000 7fdca77fe700 1 -- 172.21.15.119:0/2209577995 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fdcb80b3040 con 0x55f8ade0bb40 2023-12-07T10:23:01.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.111+0000 7fdce1eecb80 1 -- 172.21.15.119:0/2209577995 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdcc000f040 con 0x55f8ade0bb40 2023-12-07T10:23:01.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.111+0000 7fdca77fe700 1 -- 172.21.15.119:0/2209577995 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdcb809b370 con 0x55f8ade0bb40 2023-12-07T10:23:01.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.111+0000 7fdce1eecb80 10 client.26485 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:01.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.111+0000 7fdce1eecb80 1 -- 172.21.15.119:0/2209577995 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55f8adb88de0 con 0x55f8ade0bb40 2023-12-07T10:23:01.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.112+0000 7fdca77fe700 1 -- 172.21.15.119:0/2209577995 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcb8081030 con 0x55f8ade0bb40 2023-12-07T10:23:01.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.112+0000 7fdca77fe700 1 -- 172.21.15.119:0/2209577995 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fdcb80cb8a0 con 0x55f8ade0bb40 2023-12-07T10:23:01.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.112+0000 7fdce1eecb80 10 client.26485 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:01.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.112+0000 7fdce1eecb80 10 client.26485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:23:01.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.112+0000 7fdce1eecb80 20 client.26485 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:01.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.112+0000 7fdce1eecb80 1 --2- 172.21.15.119:0/2209577995 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f8ade7c720 0x55f8ade7eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.112+0000 7fdce1eecb80 4 client.26485 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-12-07T10:23:01.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.112+0000 7fdce1eecb80 1 -- 172.21.15.119:0/2209577995 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f8add25fd0 con 0x55f8ade7c720 2023-12-07T10:23:01.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.112+0000 7fdcc77fe700 1 --2- 172.21.15.119:0/2209577995 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f8ade7c720 0x55f8ade7eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.113+0000 7fdca77fe700 10 client.26485 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:01.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.113+0000 7fdcc77fe700 1 --2- 172.21.15.119:0/2209577995 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f8ade7c720 0x55f8ade7eb40 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55477d693bd89b12 server_cookie=f8062780d8023ed6 in_seq=0 out_seq=0 2023-12-07T10:23:01.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.113+0000 7fdca77fe700 1 -- 172.21.15.119:0/2209577995 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f8add25fd0 con 0x55f8ade7c720 2023-12-07T10:23:01.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.113+0000 7fdca77fe700 10 client.26485 handle_command_reply: tid=0 2023-12-07T10:23:01.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.113+0000 7fdce1eecb80 1 client.26485 shutdown 2023-12-07T10:23:01.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.113+0000 7fdce1eecb80 1 -- 172.21.15.119:0/2209577995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8ade0bb40 msgr2=0x55f8ade1a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:01.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.113+0000 7fdce1eecb80 1 --2- 172.21.15.119:0/2209577995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8ade0bb40 0x55f8ade1a360 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80976e0 tx=0x7fdcb8003680 comp rx=0 tx=0).stop 2023-12-07T10:23:01.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.113+0000 7fdce1eecb80 1 -- 172.21.15.119:0/2209577995 shutdown_connections 2023-12-07T10:23:01.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.113+0000 7fdce1eecb80 1 --2- 172.21.15.119:0/2209577995 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f8ade7c720 0x55f8ade7eb40 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.113+0000 7fdce1eecb80 1 --2- 172.21.15.119:0/2209577995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8ade10150 0x55f8ade7a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.113+0000 7fdce1eecb80 1 --2- 172.21.15.119:0/2209577995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8ade0bb40 0x55f8ade1a360 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.113+0000 7fdca77fe700 0 client.26485 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:01.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.113+0000 7fdce1eecb80 1 --2- 172.21.15.119:0/2209577995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8ade0c460 0x55f8ade1a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.113+0000 7fdce1eecb80 1 -- 172.21.15.119:0/2209577995 >> 172.21.15.119:0/2209577995 conn(0x55f8add3a360 msgr2=0x55f8ade12540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:01.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.113+0000 7fdce1eecb80 1 -- 172.21.15.119:0/2209577995 shutdown_connections 2023-12-07T10:23:01.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.113+0000 7fdce1eecb80 1 -- 172.21.15.119:0/2209577995 wait complete. 2023-12-07T10:23:01.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.114+0000 7fdce1eecb80 20 client.26485 trim_cache size 0 max 16384 2023-12-07T10:23:01.118 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:23:01.118 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "no active scrubs running", 2023-12-07T10:23:01.119 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": {} 2023-12-07T10:23:01.119 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:23:01.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.114+0000 7fdcce5ff700 1 -- 172.21.15.119:0/3328351365 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fdcb00a1c80 msgr2=0x7fdcb00a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:01.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.114+0000 7fdcce5ff700 1 --2- 172.21.15.119:0/3328351365 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fdcb00a1c80 0x7fdcb00a4140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0098c0 tx=0x7fdcbc019040 comp rx=0 tx=0).stop 2023-12-07T10:23:01.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.114+0000 7fdcce5ff700 1 -- 172.21.15.119:0/3328351365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcc8101550 msgr2=0x7fdcc8073910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:01.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.114+0000 7fdcce5ff700 1 --2- 172.21.15.119:0/3328351365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcc8101550 0x7fdcc8073910 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800ec10 tx=0x7fdcb800ef20 comp rx=0 tx=0).stop 2023-12-07T10:23:01.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.114+0000 7fdcce5ff700 1 -- 172.21.15.119:0/3328351365 shutdown_connections 2023-12-07T10:23:01.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.114+0000 7fdcce5ff700 1 --2- 172.21.15.119:0/3328351365 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fdcb00a1c80 0x7fdcb00a4140 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.114+0000 7fdcce5ff700 1 --2- 172.21.15.119:0/3328351365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcc8107610 0x7fdcc8074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.114+0000 7fdcce5ff700 1 --2- 172.21.15.119:0/3328351365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcc8101550 0x7fdcc8073910 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.114+0000 7fdcce5ff700 1 --2- 172.21.15.119:0/3328351365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcc8101f10 0x7fdcc8073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.114+0000 7fdcce5ff700 1 -- 172.21.15.119:0/3328351365 >> 172.21.15.119:0/3328351365 conn(0x7fdcc80fd020 msgr2=0x7fdcc80fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:01.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.115+0000 7fdcce5ff700 1 -- 172.21.15.119:0/3328351365 shutdown_connections 2023-12-07T10:23:01.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.115+0000 7fdcce5ff700 1 -- 172.21.15.119:0/3328351365 wait complete. 2023-12-07T10:23:01.135 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-07T10:23:01.136 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:23:01.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.377+0000 7fd98e1ff700 1 -- 172.21.15.119:0/1336257238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9880686e0 msgr2=0x7fd988068b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:01.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.377+0000 7fd98e1ff700 1 --2- 172.21.15.119:0/1336257238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9880686e0 0x7fd988068b60 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0098f0 tx=0x7fd97c009c00 comp rx=0 tx=0).stop 2023-12-07T10:23:01.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.378+0000 7fd98e1ff700 1 -- 172.21.15.119:0/1336257238 shutdown_connections 2023-12-07T10:23:01.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.378+0000 7fd98e1ff700 1 --2- 172.21.15.119:0/1336257238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd988103f40 0x7fd9881043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.378+0000 7fd98e1ff700 1 --2- 172.21.15.119:0/1336257238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9881063f0 0x7fd9881067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.378+0000 7fd98e1ff700 1 --2- 172.21.15.119:0/1336257238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9880686e0 0x7fd988068b60 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.378+0000 7fd98e1ff700 1 -- 172.21.15.119:0/1336257238 >> 172.21.15.119:0/1336257238 conn(0x7fd988077220 msgr2=0x7fd988077630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:01.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.378+0000 7fd98e1ff700 1 -- 172.21.15.119:0/1336257238 shutdown_connections 2023-12-07T10:23:01.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.378+0000 7fd98e1ff700 1 -- 172.21.15.119:0/1336257238 wait complete. 2023-12-07T10:23:01.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.379+0000 7fd98e1ff700 1 Processor -- start 2023-12-07T10:23:01.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.379+0000 7fd98e1ff700 1 -- start start 2023-12-07T10:23:01.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.379+0000 7fd98e1ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9880686e0 0x7fd98819dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.379+0000 7fd98e1ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd988103f40 0x7fd98819e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.379+0000 7fd98e1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9881063f0 0x7fd9881a28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.379+0000 7fd98e1ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9881a2f00 con 0x7fd9880686e0 2023-12-07T10:23:01.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.379+0000 7fd98e1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9881a3070 con 0x7fd9881063f0 2023-12-07T10:23:01.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.379+0000 7fd98e1ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9881a3230 con 0x7fd988103f40 2023-12-07T10:23:01.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.380+0000 7fd9877fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9880686e0 0x7fd98819dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.380+0000 7fd9877fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9880686e0 0x7fd98819dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58042/0 (socket says 172.21.15.119:58042) 2023-12-07T10:23:01.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.380+0000 7fd9877fe700 1 -- 172.21.15.119:0/2836403422 learned_addr learned my addr 172.21.15.119:0/2836403422 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:01.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.380+0000 7fd987fff700 1 --2- 172.21.15.119:0/2836403422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9881063f0 0x7fd9881a28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.380+0000 7fd9877fe700 1 -- 172.21.15.119:0/2836403422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd988103f40 msgr2=0x7fd98819e520 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-07T10:23:01.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.380+0000 7fd9877fe700 1 --2- 172.21.15.119:0/2836403422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd988103f40 0x7fd98819e520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.380+0000 7fd9877fe700 1 -- 172.21.15.119:0/2836403422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9881063f0 msgr2=0x7fd9881a28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:01.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.380+0000 7fd9877fe700 1 --2- 172.21.15.119:0/2836403422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9881063f0 0x7fd9881a28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.380+0000 7fd9877fe700 1 -- 172.21.15.119:0/2836403422 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd980009d00 con 0x7fd9880686e0 2023-12-07T10:23:01.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.380+0000 7fd987fff700 1 --2- 172.21.15.119:0/2836403422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9881063f0 0x7fd9881a28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:01.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.381+0000 7fd9877fe700 1 --2- 172.21.15.119:0/2836403422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9880686e0 0x7fd98819dfe0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd98000e3c0 tx=0x7fd98000e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:01.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.381+0000 7fd984ff9700 1 -- 172.21.15.119:0/2836403422 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd980018070 con 0x7fd9880686e0 2023-12-07T10:23:01.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.381+0000 7fd98e1ff700 1 -- 172.21.15.119:0/2836403422 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97c009520 con 0x7fd9880686e0 2023-12-07T10:23:01.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.381+0000 7fd984ff9700 1 -- 172.21.15.119:0/2836403422 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd98000f040 con 0x7fd9880686e0 2023-12-07T10:23:01.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.381+0000 7fd984ff9700 1 -- 172.21.15.119:0/2836403422 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9800043a0 con 0x7fd9880686e0 2023-12-07T10:23:01.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.381+0000 7fd98e1ff700 1 -- 172.21.15.119:0/2836403422 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9881aab70 con 0x7fd9880686e0 2023-12-07T10:23:01.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.383+0000 7fd98e1ff700 1 -- 172.21.15.119:0/2836403422 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd988100a20 con 0x7fd9880686e0 2023-12-07T10:23:01.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.385+0000 7fd984ff9700 1 -- 172.21.15.119:0/2836403422 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fd980003a00 con 0x7fd9880686e0 2023-12-07T10:23:01.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.387+0000 7fd984ff9700 1 --2- 172.21.15.119:0/2836403422 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd9700a1c80 0x7fd9700a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.387+0000 7fd984ff9700 1 -- 172.21.15.119:0/2836403422 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fd9800cd9e0 con 0x7fd9880686e0 2023-12-07T10:23:01.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.388+0000 7fd986ffd700 1 --2- 172.21.15.119:0/2836403422 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd9700a1c80 0x7fd9700a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.388+0000 7fd986ffd700 1 --2- 172.21.15.119:0/2836403422 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd9700a1c80 0x7fd9700a4140 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009230 tx=0x7fd97c009f50 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:01.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.393+0000 7fd984ff9700 1 -- 172.21.15.119:0/2836403422 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd9800967e0 con 0x7fd9880686e0 2023-12-07T10:23:01.547 INFO:tasks.workunit.client.0.smithi119.stdout: 1 114559 8.57 MB/sec warmup 74 sec latency 56.493 ms 2023-12-07T10:23:01.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.692+0000 7fd98e1ff700 1 -- 172.21.15.119:0/2836403422 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd988117a60 con 0x7fd9880686e0 2023-12-07T10:23:01.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.693+0000 7fd984ff9700 1 -- 172.21.15.119:0/2836403422 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+11401 (secure 0 0 0) 0x7fd980099e00 con 0x7fd9880686e0 2023-12-07T10:23:01.694 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:23:01.694 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-07T10:17:41.278311+0000","modified":"2023-12-07T10:22:55.653365+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24517,"mds_3":24505,"mds_4":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14535":{"gid":14535,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6841/1791998028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1791998028},{"type":"v1","addr":"172.21.15.119:6841","nonce":1791998028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24505":{"gid":24505,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/2925503246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2925503246},{"type":"v1","addr":"172.21.15.153:6837","nonce":2925503246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6839/3984962900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3984962900},{"type":"v1","addr":"172.21.15.159:6839","nonce":3984962900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.119:6839/1920762602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1920762602},{"type":"v1","addr":"172.21.15.119:6839","nonce":1920762602}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/566825184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":566825184},{"type":"v1","addr":"172.21.15.153:6839","nonce":566825184}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-07T10:23:01.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.695+0000 7fd98e1ff700 1 -- 172.21.15.119:0/2836403422 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd9700a1c80 msgr2=0x7fd9700a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:01.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.695+0000 7fd98e1ff700 1 --2- 172.21.15.119:0/2836403422 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd9700a1c80 0x7fd9700a4140 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009230 tx=0x7fd97c009f50 comp rx=0 tx=0).stop 2023-12-07T10:23:01.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.695+0000 7fd98e1ff700 1 -- 172.21.15.119:0/2836403422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9880686e0 msgr2=0x7fd98819dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:01.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.695+0000 7fd98e1ff700 1 --2- 172.21.15.119:0/2836403422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9880686e0 0x7fd98819dfe0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd98000e3c0 tx=0x7fd98000e6d0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.696+0000 7fd98e1ff700 1 -- 172.21.15.119:0/2836403422 shutdown_connections 2023-12-07T10:23:01.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.696+0000 7fd98e1ff700 1 --2- 172.21.15.119:0/2836403422 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd9700a1c80 0x7fd9700a4140 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.696+0000 7fd98e1ff700 1 --2- 172.21.15.119:0/2836403422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9881063f0 0x7fd9881a28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.696+0000 7fd98e1ff700 1 --2- 172.21.15.119:0/2836403422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9880686e0 0x7fd98819dfe0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.696+0000 7fd98e1ff700 1 --2- 172.21.15.119:0/2836403422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd988103f40 0x7fd98819e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.696+0000 7fd98e1ff700 1 -- 172.21.15.119:0/2836403422 >> 172.21.15.119:0/2836403422 conn(0x7fd988077220 msgr2=0x7fd9880fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:01.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.696+0000 7fd98e1ff700 1 -- 172.21.15.119:0/2836403422 shutdown_connections 2023-12-07T10:23:01.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.696+0000 7fd98e1ff700 1 -- 172.21.15.119:0/2836403422 wait complete. 2023-12-07T10:23:01.698 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 36 2023-12-07T10:23:01.717 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-12-07T10:23:01.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.975+0000 7f46e37ff700 1 -- 172.21.15.119:0/3809186290 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46dc107610 msgr2=0x7f46dc1079f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:01.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.975+0000 7f46e37ff700 1 --2- 172.21.15.119:0/3809186290 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46dc107610 0x7f46dc1079f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009990 tx=0x7f46d8009ca0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.976+0000 7f46e37ff700 1 -- 172.21.15.119:0/3809186290 shutdown_connections 2023-12-07T10:23:01.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.976+0000 7f46e37ff700 1 --2- 172.21.15.119:0/3809186290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46dc101f10 0x7f46dc10e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.976+0000 7f46e37ff700 1 --2- 172.21.15.119:0/3809186290 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46dc107610 0x7f46dc1079f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.976+0000 7f46e37ff700 1 --2- 172.21.15.119:0/3809186290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46dc101550 0x7f46dc1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.976+0000 7f46e37ff700 1 -- 172.21.15.119:0/3809186290 >> 172.21.15.119:0/3809186290 conn(0x7f46dc0fd020 msgr2=0x7f46dc0ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:01.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.976+0000 7f46e37ff700 1 -- 172.21.15.119:0/3809186290 shutdown_connections 2023-12-07T10:23:01.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.976+0000 7f46e37ff700 1 -- 172.21.15.119:0/3809186290 wait complete. 2023-12-07T10:23:01.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.977+0000 7f46e37ff700 1 Processor -- start 2023-12-07T10:23:01.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.977+0000 7f46e37ff700 1 -- start start 2023-12-07T10:23:01.978 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:01 smithi119 ceph-mon[97908]: pgmap v81: 129 pgs: 129 active+clean; 78 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 1.3 MiB/s wr, 202 op/s 2023-12-07T10:23:01.978 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:01 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2836403422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:23:01.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.977+0000 7f46e37ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46dc101550 0x7f46dc19f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.977+0000 7f46e37ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46dc101f10 0x7f46dc19fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.978+0000 7f46e37ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46dc107610 0x7f46dc1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.978+0000 7f46e37ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46dc19a000 con 0x7f46dc101f10 2023-12-07T10:23:01.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.978+0000 7f46e37ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46dc19a170 con 0x7f46dc107610 2023-12-07T10:23:01.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.978+0000 7f46e37ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46dc19a330 con 0x7f46dc101550 2023-12-07T10:23:01.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.978+0000 7f46e159b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46dc101550 0x7f46dc19f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.978+0000 7f46e0d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46dc101f10 0x7f46dc19fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.978+0000 7f46e159b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46dc101550 0x7f46dc19f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:52692/0 (socket says 172.21.15.119:52692) 2023-12-07T10:23:01.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.978+0000 7f46e159b700 1 -- 172.21.15.119:0/1384471406 learned_addr learned my addr 172.21.15.119:0/1384471406 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:01.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.978+0000 7f46e1d9c700 1 --2- 172.21.15.119:0/1384471406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46dc107610 0x7f46dc1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.978+0000 7f46e159b700 1 -- 172.21.15.119:0/1384471406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46dc107610 msgr2=0x7f46dc1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:01.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.978+0000 7f46e159b700 1 --2- 172.21.15.119:0/1384471406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46dc107610 0x7f46dc1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.978+0000 7f46e159b700 1 -- 172.21.15.119:0/1384471406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46dc101f10 msgr2=0x7f46dc19fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:01.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.978+0000 7f46e159b700 1 --2- 172.21.15.119:0/1384471406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46dc101f10 0x7f46dc19fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.978+0000 7f46e159b700 1 -- 172.21.15.119:0/1384471406 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d80095f0 con 0x7f46dc101550 2023-12-07T10:23:01.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.978+0000 7f46e159b700 1 --2- 172.21.15.119:0/1384471406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46dc101550 0x7f46dc19f870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f46d8000c00 tx=0x7f46d8005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:01.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.979+0000 7f46d27fc700 1 -- 172.21.15.119:0/1384471406 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d801d070 con 0x7f46dc101550 2023-12-07T10:23:01.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.979+0000 7f46d27fc700 1 -- 172.21.15.119:0/1384471406 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46d8022470 con 0x7f46dc101550 2023-12-07T10:23:01.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.979+0000 7f46d27fc700 1 -- 172.21.15.119:0/1384471406 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d8017650 con 0x7f46dc101550 2023-12-07T10:23:01.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.979+0000 7f46e37ff700 1 -- 172.21.15.119:0/1384471406 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46dc19a5b0 con 0x7f46dc101550 2023-12-07T10:23:01.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.979+0000 7f46e37ff700 1 -- 172.21.15.119:0/1384471406 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46dc1a6860 con 0x7f46dc101550 2023-12-07T10:23:01.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.983+0000 7f46d27fc700 1 -- 172.21.15.119:0/1384471406 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f46d80225e0 con 0x7f46dc101550 2023-12-07T10:23:01.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.984+0000 7f46d27fc700 1 --2- 172.21.15.119:0/1384471406 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f46c80a1d80 0x7f46c80a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.985+0000 7f46f7081b80 1 Processor -- start 2023-12-07T10:23:01.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.985+0000 7f46d27fc700 1 -- 172.21.15.119:0/1384471406 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f46d80cf020 con 0x7f46dc101550 2023-12-07T10:23:01.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.985+0000 7f46e0d9a700 1 --2- 172.21.15.119:0/1384471406 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f46c80a1d80 0x7f46c80a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.985+0000 7f46f7081b80 1 -- start start 2023-12-07T10:23:01.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.985+0000 7f46f7081b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559fed6b8350 0x559fed6b8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.985+0000 7f46f7081b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fed6b8c70 0x559fed6bc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.985+0000 7f46f7081b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559fed6bc960 0x559fed6bcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.985+0000 7f46f7081b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fed6237c0 con 0x559fed6bc960 2023-12-07T10:23:01.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.985+0000 7f46f7081b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fed60a7a0 con 0x559fed6b8350 2023-12-07T10:23:01.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.985+0000 7f46f7081b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fed6080c0 con 0x559fed6b8c70 2023-12-07T10:23:01.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.986+0000 7f46e0d9a700 1 --2- 172.21.15.119:0/1384471406 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f46c80a1d80 0x7f46c80a4240 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f46dc19ab60 tx=0x7f46cc009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:01.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.986+0000 7f46e159b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559fed6b8350 0x559fed6b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.986+0000 7f46e159b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559fed6b8350 0x559fed6b8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:41790/0 (socket says 172.21.15.119:41790) 2023-12-07T10:23:01.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.986+0000 7f46e159b700 1 -- 172.21.15.119:0/2428757141 learned_addr learned my addr 172.21.15.119:0/2428757141 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:01.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.986+0000 7f46e0d9a700 1 --2- 172.21.15.119:0/2428757141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fed6b8c70 0x559fed6bc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.986+0000 7f46e1d9c700 1 --2- 172.21.15.119:0/2428757141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559fed6bc960 0x559fed6bcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.986+0000 7f46e159b700 1 -- 172.21.15.119:0/2428757141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fed6b8c70 msgr2=0x559fed6bc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:01.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.986+0000 7f46e159b700 1 --2- 172.21.15.119:0/2428757141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fed6b8c70 0x559fed6bc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.986+0000 7f46e159b700 1 -- 172.21.15.119:0/2428757141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559fed6bc960 msgr2=0x559fed6bcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:01.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.986+0000 7f46e159b700 1 --2- 172.21.15.119:0/2428757141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559fed6bc960 0x559fed6bcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.986+0000 7f46e159b700 1 -- 172.21.15.119:0/2428757141 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fed480630 con 0x559fed6b8350 2023-12-07T10:23:01.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.986+0000 7f46e1d9c700 1 --2- 172.21.15.119:0/2428757141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559fed6bc960 0x559fed6bcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:23:01.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.986+0000 7f46e159b700 1 --2- 172.21.15.119:0/2428757141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559fed6b8350 0x559fed6b8730 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f46d8098510 tx=0x7f46d809b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6845ee1e6d122385 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:01.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.987+0000 7f46c77fe700 1 -- 172.21.15.119:0/2428757141 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d809c1a0 con 0x559fed6b8350 2023-12-07T10:23:01.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.987+0000 7f46c77fe700 1 -- 172.21.15.119:0/2428757141 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46d809baa0 con 0x559fed6b8350 2023-12-07T10:23:01.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.987+0000 7f46c77fe700 1 -- 172.21.15.119:0/2428757141 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d809bd00 con 0x559fed6b8350 2023-12-07T10:23:01.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.988+0000 7f46f7081b80 1 -- 172.21.15.119:0/2428757141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559fed6b8350 msgr2=0x559fed6b8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:01.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.988+0000 7f46f7081b80 1 --2- 172.21.15.119:0/2428757141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559fed6b8350 0x559fed6b8730 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f46d8098510 tx=0x7f46d809b8f0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.988+0000 7f46f7081b80 1 -- 172.21.15.119:0/2428757141 shutdown_connections 2023-12-07T10:23:01.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.988+0000 7f46f7081b80 1 --2- 172.21.15.119:0/2428757141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559fed6bc960 0x559fed6bcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.988+0000 7f46f7081b80 1 --2- 172.21.15.119:0/2428757141 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559fed6b8350 0x559fed6b8730 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.988+0000 7f46f7081b80 1 --2- 172.21.15.119:0/2428757141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fed6b8c70 0x559fed6bc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.988+0000 7f46f7081b80 1 -- 172.21.15.119:0/2428757141 >> 172.21.15.119:0/2428757141 conn(0x559fed5de360 msgr2=0x559fed6aa130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:01.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.988+0000 7f46f7081b80 1 -- 172.21.15.119:0/2428757141 shutdown_connections 2023-12-07T10:23:01.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.988+0000 7f46f7081b80 1 -- 172.21.15.119:0/2428757141 wait complete. 2023-12-07T10:23:01.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.989+0000 7f46f7081b80 1 -- start start 2023-12-07T10:23:01.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.989+0000 7f46f7081b80 1 Processor -- start 2023-12-07T10:23:01.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.990+0000 7f46f7081b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559fed6b8350 0x559fed6c4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.990+0000 7f46f7081b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fed6b8c70 0x559fed6c1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.990+0000 7f46f7081b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559fed6bc960 0x559fed6c1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.990+0000 7f46f7081b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fed6039b0 con 0x559fed6b8350 2023-12-07T10:23:01.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.990+0000 7f46f7081b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fed5bd560 con 0x559fed6bc960 2023-12-07T10:23:01.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.990+0000 7f46f7081b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fed5bd200 con 0x559fed6b8c70 2023-12-07T10:23:01.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.990+0000 7f46f7081b80 10 client.0 _ll_register_callbacks cb 0x559fed5a4940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:01.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.990+0000 7f46e0d9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fed6b8c70 0x559fed6c1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.990+0000 7f46e0d9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fed6b8c70 0x559fed6c1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:52710/0 (socket says 172.21.15.119:52710) 2023-12-07T10:23:01.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.990+0000 7f46e0d9a700 1 -- 172.21.15.119:0/3178959204 learned_addr learned my addr 172.21.15.119:0/3178959204 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:01.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.990+0000 7f46e159b700 1 --2- 172.21.15.119:0/3178959204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559fed6b8350 0x559fed6c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.990+0000 7f46e1d9c700 1 --2- 172.21.15.119:0/3178959204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559fed6bc960 0x559fed6c1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.991+0000 7f46e0d9a700 1 -- 172.21.15.119:0/3178959204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559fed6bc960 msgr2=0x559fed6c1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:01.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.991+0000 7f46e0d9a700 1 --2- 172.21.15.119:0/3178959204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559fed6bc960 0x559fed6c1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.991+0000 7f46e0d9a700 1 -- 172.21.15.119:0/3178959204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559fed6b8350 msgr2=0x559fed6c4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:01.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.991+0000 7f46e0d9a700 1 --2- 172.21.15.119:0/3178959204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559fed6b8350 0x559fed6c4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.991+0000 7f46e0d9a700 1 -- 172.21.15.119:0/3178959204 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46dc1a6860 con 0x559fed6b8c70 2023-12-07T10:23:01.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.991+0000 7f46e159b700 1 --2- 172.21.15.119:0/3178959204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559fed6b8350 0x559fed6c4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:23:01.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.991+0000 7f46e0d9a700 1 --2- 172.21.15.119:0/3178959204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fed6b8c70 0x559fed6c1430 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f46cc004480 tx=0x7f46cc0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9e8974042a58ca0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:01.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.991+0000 7f46c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:23:01.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.992+0000 7f46c4ff9700 1 -- 172.21.15.119:0/3178959204 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc022070 con 0x559fed6b8c70 2023-12-07T10:23:01.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.992+0000 7f46c4ff9700 1 -- 172.21.15.119:0/3178959204 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f46cc003e20 con 0x559fed6b8c70 2023-12-07T10:23:01.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.992+0000 7f46f7081b80 1 -- 172.21.15.119:0/3178959204 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559fed6a2b00 con 0x559fed6b8c70 2023-12-07T10:23:01.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.992+0000 7f46c4ff9700 1 -- 172.21.15.119:0/3178959204 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46cc027420 con 0x559fed6b8c70 2023-12-07T10:23:01.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.992+0000 7f46f7081b80 10 client.26500 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:01.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.992+0000 7f46f7081b80 1 -- 172.21.15.119:0/3178959204 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x559fed42cde0 con 0x559fed6b8c70 2023-12-07T10:23:01.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.992+0000 7f46c4ff9700 1 -- 172.21.15.119:0/3178959204 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559fed6a2b00 con 0x559fed6b8c70 2023-12-07T10:23:01.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.992+0000 7f46c4ff9700 1 -- 172.21.15.119:0/3178959204 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f46cc01d7e0 con 0x559fed6b8c70 2023-12-07T10:23:01.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.992+0000 7f46f7081b80 10 client.26500 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:01.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.992+0000 7f46f7081b80 10 client.26500 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2023-12-07T10:23:01.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.992+0000 7f46f7081b80 20 client.26500 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:01.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.992+0000 7f46f7081b80 1 --2- 172.21.15.119:0/3178959204 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x559fed6d1970 0x559fed6d3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:01.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.992+0000 7f46f7081b80 4 client.26500 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:23:01.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.992+0000 7f46f7081b80 1 -- 172.21.15.119:0/3178959204 --> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559fed67e7a0 con 0x559fed6d1970 2023-12-07T10:23:01.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.993+0000 7f46e159b700 1 --2- 172.21.15.119:0/3178959204 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x559fed6d1970 0x559fed6d3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:01.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.993+0000 7f46e159b700 1 --2- 172.21.15.119:0/3178959204 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x559fed6d1970 0x559fed6d3d90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c7a6195534fd2ed9 server_cookie=30c6de075d1aba00 in_seq=0 out_seq=0 2023-12-07T10:23:01.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.993+0000 7f46c4ff9700 10 client.26500 ms_handle_connect on v2:172.21.15.153:6834/1295649464 2023-12-07T10:23:01.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.994+0000 7f46c4ff9700 1 -- 172.21.15.119:0/3178959204 <== mds.4 v2:172.21.15.153:6834/1295649464 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559fed67e7a0 con 0x559fed6d1970 2023-12-07T10:23:01.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.994+0000 7f46c4ff9700 10 client.26500 handle_command_reply: tid=0 2023-12-07T10:23:01.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.994+0000 7f46f7081b80 1 client.26500 shutdown 2023-12-07T10:23:01.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.994+0000 7f46f7081b80 1 -- 172.21.15.119:0/3178959204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fed6b8c70 msgr2=0x559fed6c1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:01.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.994+0000 7f46f7081b80 1 --2- 172.21.15.119:0/3178959204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fed6b8c70 0x559fed6c1430 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f46cc004480 tx=0x7f46cc0047c0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.994+0000 7f46f7081b80 1 -- 172.21.15.119:0/3178959204 shutdown_connections 2023-12-07T10:23:01.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.994+0000 7f46f7081b80 1 --2- 172.21.15.119:0/3178959204 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559fed6bc960 0x559fed6c1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.994+0000 7f46f7081b80 1 --2- 172.21.15.119:0/3178959204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559fed6b8350 0x559fed6c4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.994+0000 7f46f7081b80 1 --2- 172.21.15.119:0/3178959204 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x559fed6d1970 0x559fed6d3d90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.994+0000 7f46f7081b80 1 --2- 172.21.15.119:0/3178959204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fed6b8c70 0x559fed6c1430 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:01.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.994+0000 7f46c4ff9700 0 client.26500 ms_handle_reset on v2:172.21.15.153:6834/1295649464 2023-12-07T10:23:01.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.994+0000 7f46f7081b80 1 -- 172.21.15.119:0/3178959204 >> 172.21.15.119:0/3178959204 conn(0x559fed5de360 msgr2=0x559fed6a4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:01.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.994+0000 7f46f7081b80 1 -- 172.21.15.119:0/3178959204 shutdown_connections 2023-12-07T10:23:01.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.995+0000 7f46f7081b80 1 -- 172.21.15.119:0/3178959204 wait complete. 2023-12-07T10:23:01.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:01.995+0000 7f46f7081b80 20 client.26500 trim_cache size 0 max 16384 2023-12-07T10:23:02.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.012+0000 7f46f7081b80 1 Processor -- start 2023-12-07T10:23:02.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.012+0000 7f46f7081b80 1 -- start start 2023-12-07T10:23:02.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.012+0000 7f46f7081b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559fed6b8350 0x559fed6c1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.012+0000 7f46f7081b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fed6b8c70 0x559fed6c0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.012+0000 7f46f7081b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559fed6bc960 0x559fed6baf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.012+0000 7f46f7081b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fed482920 con 0x559fed6bc960 2023-12-07T10:23:02.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.012+0000 7f46f7081b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fed6c1200 con 0x559fed6b8350 2023-12-07T10:23:02.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.012+0000 7f46f7081b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fed6bf980 con 0x559fed6b8c70 2023-12-07T10:23:02.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.013+0000 7f46e159b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559fed6b8350 0x559fed6c1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.013+0000 7f46e159b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559fed6b8350 0x559fed6c1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:41806/0 (socket says 172.21.15.119:41806) 2023-12-07T10:23:02.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.013+0000 7f46e159b700 1 -- 172.21.15.119:0/2827098654 learned_addr learned my addr 172.21.15.119:0/2827098654 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:02.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.013+0000 7f46e0d9a700 1 --2- 172.21.15.119:0/2827098654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fed6b8c70 0x559fed6c0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.013+0000 7f46e1d9c700 1 --2- 172.21.15.119:0/2827098654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559fed6bc960 0x559fed6baf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.013+0000 7f46e0d9a700 1 -- 172.21.15.119:0/2827098654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559fed6b8350 msgr2=0x559fed6c1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.013+0000 7f46e0d9a700 1 --2- 172.21.15.119:0/2827098654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559fed6b8350 0x559fed6c1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.013+0000 7f46e0d9a700 1 -- 172.21.15.119:0/2827098654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559fed6bc960 msgr2=0x559fed6baf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.013+0000 7f46e0d9a700 1 --2- 172.21.15.119:0/2827098654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559fed6bc960 0x559fed6baf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.013+0000 7f46e0d9a700 1 -- 172.21.15.119:0/2827098654 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fed415c60 con 0x559fed6b8c70 2023-12-07T10:23:02.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.013+0000 7f46e159b700 1 --2- 172.21.15.119:0/2827098654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559fed6b8350 0x559fed6c1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:02.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.013+0000 7f46e0d9a700 1 --2- 172.21.15.119:0/2827098654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fed6b8c70 0x559fed6c0cc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f46cc027000 tx=0x7f46cc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=215d00b83c4102b5 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:02.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.014+0000 7f46c7fff700 1 -- 172.21.15.119:0/2827098654 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc022070 con 0x559fed6b8c70 2023-12-07T10:23:02.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.014+0000 7f46c7fff700 1 -- 172.21.15.119:0/2827098654 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46cc01d8d0 con 0x559fed6b8c70 2023-12-07T10:23:02.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.014+0000 7f46c7fff700 1 -- 172.21.15.119:0/2827098654 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc027bb0 con 0x559fed6b8c70 2023-12-07T10:23:02.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.015+0000 7f46f7081b80 1 -- 172.21.15.119:0/2827098654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fed6b8c70 msgr2=0x559fed6c0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.015+0000 7f46f7081b80 1 --2- 172.21.15.119:0/2827098654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fed6b8c70 0x559fed6c0cc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f46cc027000 tx=0x7f46cc000bc0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.015+0000 7f46f7081b80 1 -- 172.21.15.119:0/2827098654 shutdown_connections 2023-12-07T10:23:02.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.015+0000 7f46f7081b80 1 --2- 172.21.15.119:0/2827098654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559fed6bc960 0x559fed6baf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.015+0000 7f46f7081b80 1 --2- 172.21.15.119:0/2827098654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559fed6b8350 0x559fed6c1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.015+0000 7f46f7081b80 1 --2- 172.21.15.119:0/2827098654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fed6b8c70 0x559fed6c0cc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.015+0000 7f46f7081b80 1 -- 172.21.15.119:0/2827098654 >> 172.21.15.119:0/2827098654 conn(0x559fed5de360 msgr2=0x559fed6bcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:02.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.015+0000 7f46f7081b80 1 -- 172.21.15.119:0/2827098654 shutdown_connections 2023-12-07T10:23:02.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.015+0000 7f46f7081b80 1 -- 172.21.15.119:0/2827098654 wait complete. 2023-12-07T10:23:02.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.015+0000 7f46f7081b80 1 -- start start 2023-12-07T10:23:02.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.015+0000 7f46f7081b80 1 Processor -- start 2023-12-07T10:23:02.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.016+0000 7f46f7081b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559fed6b8350 0x559fed6a4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.016+0000 7f46f7081b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fed6b8c70 0x559fed6a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.016+0000 7f46f7081b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559fed6bc960 0x559fed6a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.016+0000 7f46e159b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559fed6b8350 0x559fed6a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.016+0000 7f46e159b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559fed6b8350 0x559fed6a4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58104/0 (socket says 172.21.15.119:58104) 2023-12-07T10:23:02.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.016+0000 7f46e159b700 1 -- 172.21.15.119:0/45432646 learned_addr learned my addr 172.21.15.119:0/45432646 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:02.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.016+0000 7f46f7081b80 1 -- 172.21.15.119:0/45432646 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fed726e20 con 0x559fed6b8350 2023-12-07T10:23:02.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.017+0000 7f46f7081b80 1 -- 172.21.15.119:0/45432646 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fed726f60 con 0x559fed6bc960 2023-12-07T10:23:02.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.017+0000 7f46f7081b80 1 -- 172.21.15.119:0/45432646 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fed727120 con 0x559fed6b8c70 2023-12-07T10:23:02.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.017+0000 7f46e0d9a700 1 --2- 172.21.15.119:0/45432646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fed6b8c70 0x559fed6a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.017+0000 7f46f7081b80 10 client.0 _ll_register_callbacks cb 0x559fed5a4940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:02.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.017+0000 7f46e1d9c700 1 --2- 172.21.15.119:0/45432646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559fed6bc960 0x559fed6a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.017+0000 7f46e0d9a700 1 -- 172.21.15.119:0/45432646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559fed6bc960 msgr2=0x559fed6a4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.017+0000 7f46e0d9a700 1 --2- 172.21.15.119:0/45432646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559fed6bc960 0x559fed6a4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.017+0000 7f46e0d9a700 1 -- 172.21.15.119:0/45432646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559fed6b8350 msgr2=0x559fed6a4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.017+0000 7f46e0d9a700 1 --2- 172.21.15.119:0/45432646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559fed6b8350 0x559fed6a4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.017+0000 7f46e0d9a700 1 -- 172.21.15.119:0/45432646 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559fed480630 con 0x559fed6b8c70 2023-12-07T10:23:02.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.017+0000 7f46e159b700 1 --2- 172.21.15.119:0/45432646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559fed6b8350 0x559fed6a4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:02.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.017+0000 7f46e0d9a700 1 --2- 172.21.15.119:0/45432646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fed6b8c70 0x559fed6a49d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f46cc01db90 tx=0x7f46cc01d780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aea3c12847321b2 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:02.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.017+0000 7f46c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:23:02.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.017+0000 7f46c4ff9700 1 -- 172.21.15.119:0/45432646 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc022040 con 0x559fed6b8c70 2023-12-07T10:23:02.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.017+0000 7f46f7081b80 1 -- 172.21.15.119:0/45432646 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46d809baa0 con 0x559fed6b8c70 2023-12-07T10:23:02.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.018+0000 7f46c4ff9700 1 -- 172.21.15.119:0/45432646 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f46cc005690 con 0x559fed6b8c70 2023-12-07T10:23:02.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.018+0000 7f46c4ff9700 1 -- 172.21.15.119:0/45432646 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46cc0047c0 con 0x559fed6b8c70 2023-12-07T10:23:02.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.018+0000 7f46f7081b80 10 client.26506 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:02.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.018+0000 7f46f7081b80 1 -- 172.21.15.119:0/45432646 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x559fed42cde0 con 0x559fed6b8c70 2023-12-07T10:23:02.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.018+0000 7f46c4ff9700 1 -- 172.21.15.119:0/45432646 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46d809baa0 con 0x559fed6b8c70 2023-12-07T10:23:02.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.018+0000 7f46c4ff9700 1 -- 172.21.15.119:0/45432646 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f46cc035bc0 con 0x559fed6b8c70 2023-12-07T10:23:02.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.018+0000 7f46f7081b80 10 client.26506 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:02.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.018+0000 7f46f7081b80 10 client.26506 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2023-12-07T10:23:02.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.018+0000 7f46f7081b80 20 client.26506 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:02.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.018+0000 7f46f7081b80 1 --2- 172.21.15.119:0/45432646 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x559fed728600 0x559fed72aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.018+0000 7f46f7081b80 4 client.26506 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:23:02.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.018+0000 7f46f7081b80 1 -- 172.21.15.119:0/45432646 --> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559fed5c9fd0 con 0x559fed728600 2023-12-07T10:23:02.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.019+0000 7f46e159b700 1 --2- 172.21.15.119:0/45432646 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x559fed728600 0x559fed72aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.019+0000 7f46e159b700 1 --2- 172.21.15.119:0/45432646 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x559fed728600 0x559fed72aa20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=932d109e9a8a4bd3 server_cookie=82ae8fe61a1a9c13 in_seq=0 out_seq=0 2023-12-07T10:23:02.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.019+0000 7f46c4ff9700 10 client.26506 ms_handle_connect on v2:172.21.15.153:6834/1295649464 2023-12-07T10:23:02.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.020+0000 7f46c4ff9700 1 -- 172.21.15.119:0/45432646 <== mds.4 v2:172.21.15.153:6834/1295649464 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559fed5c9fd0 con 0x559fed728600 2023-12-07T10:23:02.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.020+0000 7f46c4ff9700 10 client.26506 handle_command_reply: tid=0 2023-12-07T10:23:02.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.020+0000 7f46f7081b80 1 client.26506 shutdown 2023-12-07T10:23:02.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.020+0000 7f46f7081b80 1 -- 172.21.15.119:0/45432646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fed6b8c70 msgr2=0x559fed6a49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.020+0000 7f46f7081b80 1 --2- 172.21.15.119:0/45432646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fed6b8c70 0x559fed6a49d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f46cc01db90 tx=0x7f46cc01d780 comp rx=0 tx=0).stop 2023-12-07T10:23:02.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.020+0000 7f46f7081b80 1 -- 172.21.15.119:0/45432646 shutdown_connections 2023-12-07T10:23:02.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.020+0000 7f46f7081b80 1 --2- 172.21.15.119:0/45432646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559fed6bc960 0x559fed6a4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.020+0000 7f46f7081b80 1 --2- 172.21.15.119:0/45432646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559fed6b8350 0x559fed6a4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.020+0000 7f46f7081b80 1 --2- 172.21.15.119:0/45432646 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x559fed728600 0x559fed72aa20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.020+0000 7f46f7081b80 1 --2- 172.21.15.119:0/45432646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fed6b8c70 0x559fed6a49d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.020+0000 7f46f7081b80 1 -- 172.21.15.119:0/45432646 >> 172.21.15.119:0/45432646 conn(0x559fed5de360 msgr2=0x559fed6bbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:02.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.020+0000 7f46c4ff9700 0 client.26506 ms_handle_reset on v2:172.21.15.153:6834/1295649464 2023-12-07T10:23:02.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.020+0000 7f46f7081b80 1 -- 172.21.15.119:0/45432646 shutdown_connections 2023-12-07T10:23:02.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.020+0000 7f46f7081b80 1 -- 172.21.15.119:0/45432646 wait complete. 2023-12-07T10:23:02.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.021+0000 7f46f7081b80 20 client.26506 trim_cache size 0 max 16384 2023-12-07T10:23:02.025 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:23:02.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.022+0000 7f46e37ff700 1 -- 172.21.15.119:0/1384471406 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f46c80a1d80 msgr2=0x7f46c80a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.022+0000 7f46e37ff700 1 --2- 172.21.15.119:0/1384471406 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f46c80a1d80 0x7f46c80a4240 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f46dc19ab60 tx=0x7f46cc009260 comp rx=0 tx=0).stop 2023-12-07T10:23:02.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.022+0000 7f46e37ff700 1 -- 172.21.15.119:0/1384471406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46dc101550 msgr2=0x7f46dc19f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.022+0000 7f46e37ff700 1 --2- 172.21.15.119:0/1384471406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46dc101550 0x7f46dc19f870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f46d8000c00 tx=0x7f46d8005740 comp rx=0 tx=0).stop 2023-12-07T10:23:02.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.022+0000 7f46e37ff700 1 -- 172.21.15.119:0/1384471406 shutdown_connections 2023-12-07T10:23:02.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.022+0000 7f46e37ff700 1 --2- 172.21.15.119:0/1384471406 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f46c80a1d80 0x7f46c80a4240 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.022+0000 7f46e37ff700 1 --2- 172.21.15.119:0/1384471406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46dc107610 0x7f46dc1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.022+0000 7f46e37ff700 1 --2- 172.21.15.119:0/1384471406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46dc101550 0x7f46dc19f870 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.022+0000 7f46e37ff700 1 --2- 172.21.15.119:0/1384471406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46dc101f10 0x7f46dc19fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.022+0000 7f46e37ff700 1 -- 172.21.15.119:0/1384471406 >> 172.21.15.119:0/1384471406 conn(0x7f46dc0fd020 msgr2=0x7f46dc0fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:02.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.022+0000 7f46e37ff700 1 -- 172.21.15.119:0/1384471406 shutdown_connections 2023-12-07T10:23:02.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.022+0000 7f46e37ff700 1 -- 172.21.15.119:0/1384471406 wait complete. 2023-12-07T10:23:02.044 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-12-07T10:23:02.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:01 smithi159 ceph-mon[134726]: pgmap v81: 129 pgs: 129 active+clean; 78 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 1.3 MiB/s wr, 202 op/s 2023-12-07T10:23:02.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:01 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2836403422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:23:02.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.271+0000 7f212c9ff700 1 -- 172.21.15.119:0/721181874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128100460 msgr2=0x7f21281008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.271+0000 7f212c9ff700 1 --2- 172.21.15.119:0/721181874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128100460 0x7f21281008e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f211c009990 tx=0x7f211c009ca0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.271+0000 7f212c9ff700 1 -- 172.21.15.119:0/721181874 shutdown_connections 2023-12-07T10:23:02.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.271+0000 7f212c9ff700 1 --2- 172.21.15.119:0/721181874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2128100e20 0x7f212810d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.271+0000 7f212c9ff700 1 --2- 172.21.15.119:0/721181874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f21281053d0 0x7f21281057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.271+0000 7f212c9ff700 1 --2- 172.21.15.119:0/721181874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128100460 0x7f21281008e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.271+0000 7f212c9ff700 1 -- 172.21.15.119:0/721181874 >> 172.21.15.119:0/721181874 conn(0x7f2128077610 msgr2=0x7f2128077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:02.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.271+0000 7f212c9ff700 1 -- 172.21.15.119:0/721181874 shutdown_connections 2023-12-07T10:23:02.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.271+0000 7f212c9ff700 1 -- 172.21.15.119:0/721181874 wait complete. 2023-12-07T10:23:02.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.272+0000 7f212c9ff700 1 Processor -- start 2023-12-07T10:23:02.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.272+0000 7f212c9ff700 1 -- start start 2023-12-07T10:23:02.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.273+0000 7f212c9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2128100460 0x7f21280fde50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.273+0000 7f212c9ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128100e20 0x7f21280fe390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.273+0000 7f212c9ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f21281053d0 0x7f21280fe8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.273+0000 7f212c9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21281a5e30 con 0x7f2128100460 2023-12-07T10:23:02.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.273+0000 7f212c9ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21281a5fa0 con 0x7f2128100e20 2023-12-07T10:23:02.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.273+0000 7f212c9ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21281a6160 con 0x7f21281053d0 2023-12-07T10:23:02.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.273+0000 7f2125d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128100e20 0x7f21280fe390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.273+0000 7f212659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2128100460 0x7f21280fde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.273+0000 7f2125d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128100e20 0x7f21280fe390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:41826/0 (socket says 172.21.15.119:41826) 2023-12-07T10:23:02.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.273+0000 7f2125d9b700 1 -- 172.21.15.119:0/3193224965 learned_addr learned my addr 172.21.15.119:0/3193224965 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:02.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.273+0000 7f2126d9d700 1 --2- 172.21.15.119:0/3193224965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f21281053d0 0x7f21280fe8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.274+0000 7f2125d9b700 1 -- 172.21.15.119:0/3193224965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f21281053d0 msgr2=0x7f21280fe8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.274+0000 7f2125d9b700 1 --2- 172.21.15.119:0/3193224965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f21281053d0 0x7f21280fe8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.274+0000 7f2125d9b700 1 -- 172.21.15.119:0/3193224965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2128100460 msgr2=0x7f21280fde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.274+0000 7f2125d9b700 1 --2- 172.21.15.119:0/3193224965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2128100460 0x7f21280fde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.274+0000 7f2125d9b700 1 -- 172.21.15.119:0/3193224965 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211c0095f0 con 0x7f2128100e20 2023-12-07T10:23:02.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.274+0000 7f2125d9b700 1 --2- 172.21.15.119:0/3193224965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128100e20 0x7f21280fe390 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f211c005910 tx=0x7f211c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:02.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.274+0000 7f21177fe700 1 -- 172.21.15.119:0/3193224965 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c01d070 con 0x7f2128100e20 2023-12-07T10:23:02.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.274+0000 7f21177fe700 1 -- 172.21.15.119:0/3193224965 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f211c022470 con 0x7f2128100e20 2023-12-07T10:23:02.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.274+0000 7f212c9ff700 1 -- 172.21.15.119:0/3193224965 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21281a63e0 con 0x7f2128100e20 2023-12-07T10:23:02.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.274+0000 7f21177fe700 1 -- 172.21.15.119:0/3193224965 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c017650 con 0x7f2128100e20 2023-12-07T10:23:02.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.274+0000 7f212c9ff700 1 -- 172.21.15.119:0/3193224965 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21281a6890 con 0x7f2128100e20 2023-12-07T10:23:02.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.278+0000 7f21177fe700 1 -- 172.21.15.119:0/3193224965 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f211c0225e0 con 0x7f2128100e20 2023-12-07T10:23:02.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:01 smithi153 ceph-mon[133625]: pgmap v81: 129 pgs: 129 active+clean; 78 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 1.3 MiB/s wr, 202 op/s 2023-12-07T10:23:02.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:01 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2836403422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:23:02.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.279+0000 7f21177fe700 1 --2- 172.21.15.119:0/3193224965 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f21100a1c80 0x7f21100a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.279+0000 7f21177fe700 1 -- 172.21.15.119:0/3193224965 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f211c0cf080 con 0x7f2128100e20 2023-12-07T10:23:02.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.280+0000 7f212659c700 1 --2- 172.21.15.119:0/3193224965 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f21100a1c80 0x7f21100a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.280+0000 7f214030eb80 1 Processor -- start 2023-12-07T10:23:02.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.280+0000 7f214030eb80 1 -- start start 2023-12-07T10:23:02.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.281+0000 7f214030eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581daf2bb40 0x5581daf2bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.281+0000 7f214030eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf2c460 0x5581daf2fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.281+0000 7f212659c700 1 --2- 172.21.15.119:0/3193224965 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f21100a1c80 0x7f21100a4140 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2118009be0 tx=0x7f2118009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:02.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.281+0000 7f214030eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581daf30150 0x5581daf30570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.281+0000 7f214030eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581dae9f7c0 con 0x5581daf2bb40 2023-12-07T10:23:02.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.281+0000 7f214030eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581dae867a0 con 0x5581daf2c460 2023-12-07T10:23:02.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.281+0000 7f214030eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581dae840c0 con 0x5581daf30150 2023-12-07T10:23:02.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.281+0000 7f2125d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf2c460 0x5581daf2fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.281+0000 7f2126d9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581daf30150 0x5581daf30570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.281+0000 7f2126d9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581daf30150 0x5581daf30570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:52768/0 (socket says 172.21.15.119:52768) 2023-12-07T10:23:02.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.281+0000 7f2125d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf2c460 0x5581daf2fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:41832/0 (socket says 172.21.15.119:41832) 2023-12-07T10:23:02.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.281+0000 7f2125d9b700 1 -- 172.21.15.119:0/623605766 learned_addr learned my addr 172.21.15.119:0/623605766 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:02.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.281+0000 7f212659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581daf2bb40 0x5581daf2bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.281+0000 7f2126d9d700 1 -- 172.21.15.119:0/623605766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf2c460 msgr2=0x5581daf2fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.281+0000 7f2126d9d700 1 --2- 172.21.15.119:0/623605766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf2c460 0x5581daf2fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.281+0000 7f2126d9d700 1 -- 172.21.15.119:0/623605766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581daf2bb40 msgr2=0x5581daf2bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.281+0000 7f2126d9d700 1 --2- 172.21.15.119:0/623605766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581daf2bb40 0x5581daf2bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.281+0000 7f2126d9d700 1 -- 172.21.15.119:0/623605766 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581dacfc630 con 0x5581daf30150 2023-12-07T10:23:02.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.281+0000 7f2125d9b700 1 --2- 172.21.15.119:0/623605766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf2c460 0x5581daf2fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:02.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.282+0000 7f2126d9d700 1 --2- 172.21.15.119:0/623605766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581daf30150 0x5581daf30570 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f212000f440 tx=0x7f2120007f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96af9e646cd39ca9 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:02.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.282+0000 7f2114ff9700 1 -- 172.21.15.119:0/623605766 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212000feb0 con 0x5581daf30150 2023-12-07T10:23:02.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.282+0000 7f2114ff9700 1 -- 172.21.15.119:0/623605766 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2120004bb0 con 0x5581daf30150 2023-12-07T10:23:02.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.282+0000 7f2114ff9700 1 -- 172.21.15.119:0/623605766 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2120005020 con 0x5581daf30150 2023-12-07T10:23:02.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.283+0000 7f214030eb80 1 -- 172.21.15.119:0/623605766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581daf30150 msgr2=0x5581daf30570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.283+0000 7f214030eb80 1 --2- 172.21.15.119:0/623605766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581daf30150 0x5581daf30570 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f212000f440 tx=0x7f2120007f90 comp rx=0 tx=0).stop 2023-12-07T10:23:02.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.283+0000 7f214030eb80 1 -- 172.21.15.119:0/623605766 shutdown_connections 2023-12-07T10:23:02.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.283+0000 7f214030eb80 1 --2- 172.21.15.119:0/623605766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581daf30150 0x5581daf30570 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.283+0000 7f214030eb80 1 --2- 172.21.15.119:0/623605766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581daf2bb40 0x5581daf2bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.283+0000 7f214030eb80 1 --2- 172.21.15.119:0/623605766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf2c460 0x5581daf2fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.283+0000 7f214030eb80 1 -- 172.21.15.119:0/623605766 >> 172.21.15.119:0/623605766 conn(0x5581dae5a360 msgr2=0x5581daf26130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:02.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.283+0000 7f214030eb80 1 -- 172.21.15.119:0/623605766 shutdown_connections 2023-12-07T10:23:02.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.283+0000 7f214030eb80 1 -- 172.21.15.119:0/623605766 wait complete. 2023-12-07T10:23:02.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.284+0000 7f214030eb80 1 -- start start 2023-12-07T10:23:02.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.284+0000 7f214030eb80 1 Processor -- start 2023-12-07T10:23:02.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.285+0000 7f214030eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf2bb40 0x5581daf379a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.285+0000 7f214030eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581daf2c460 0x5581daf34ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.285+0000 7f214030eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581daf30150 0x5581daf353d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.285+0000 7f214030eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581dae7f9b0 con 0x5581daf2c460 2023-12-07T10:23:02.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.285+0000 7f214030eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581dae39560 con 0x5581daf2bb40 2023-12-07T10:23:02.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.285+0000 7f214030eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581dae39200 con 0x5581daf30150 2023-12-07T10:23:02.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.285+0000 7f214030eb80 10 client.0 _ll_register_callbacks cb 0x5581dae20940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:02.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.285+0000 7f212659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf2bb40 0x5581daf379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.285+0000 7f212659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf2bb40 0x5581daf379a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:41842/0 (socket says 172.21.15.119:41842) 2023-12-07T10:23:02.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.285+0000 7f212659c700 1 -- 172.21.15.119:0/397407482 learned_addr learned my addr 172.21.15.119:0/397407482 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:02.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.285+0000 7f2126d9d700 1 --2- 172.21.15.119:0/397407482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581daf30150 0x5581daf353d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.285+0000 7f2125d9b700 1 --2- 172.21.15.119:0/397407482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581daf2c460 0x5581daf34ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.285+0000 7f212659c700 1 -- 172.21.15.119:0/397407482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581daf30150 msgr2=0x5581daf353d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.285+0000 7f212659c700 1 --2- 172.21.15.119:0/397407482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581daf30150 0x5581daf353d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.285+0000 7f212659c700 1 -- 172.21.15.119:0/397407482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581daf2c460 msgr2=0x5581daf34ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.285+0000 7f212659c700 1 --2- 172.21.15.119:0/397407482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581daf2c460 0x5581daf34ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.285+0000 7f212659c700 1 -- 172.21.15.119:0/397407482 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21281a6890 con 0x5581daf2bb40 2023-12-07T10:23:02.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.285+0000 7f2126d9d700 1 --2- 172.21.15.119:0/397407482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581daf30150 0x5581daf353d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:02.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.286+0000 7f212659c700 1 --2- 172.21.15.119:0/397407482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf2bb40 0x5581daf379a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f211801fbf0 tx=0x7f21180047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a23211f181aa77df server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:02.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.286+0000 7f210dffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:23:02.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.286+0000 7f210dffb700 1 -- 172.21.15.119:0/397407482 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2118011e30 con 0x5581daf2bb40 2023-12-07T10:23:02.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.286+0000 7f210dffb700 1 -- 172.21.15.119:0/397407482 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f2118004020 con 0x5581daf2bb40 2023-12-07T10:23:02.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.286+0000 7f214030eb80 1 -- 172.21.15.119:0/397407482 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581daf1eb00 con 0x5581daf2bb40 2023-12-07T10:23:02.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.286+0000 7f210dffb700 1 -- 172.21.15.119:0/397407482 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2118004220 con 0x5581daf2bb40 2023-12-07T10:23:02.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.286+0000 7f214030eb80 10 client.26528 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:02.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.286+0000 7f214030eb80 1 -- 172.21.15.119:0/397407482 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f2120007e00 con 0x5581daf2bb40 2023-12-07T10:23:02.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.286+0000 7f210dffb700 1 -- 172.21.15.119:0/397407482 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5581daf1eb00 con 0x5581daf2bb40 2023-12-07T10:23:02.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.287+0000 7f210dffb700 1 -- 172.21.15.119:0/397407482 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f21180038e0 con 0x5581daf2bb40 2023-12-07T10:23:02.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.287+0000 7f214030eb80 10 client.26528 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:02.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.287+0000 7f214030eb80 10 client.26528 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-12-07T10:23:02.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.287+0000 7f214030eb80 20 client.26528 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:02.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.287+0000 7f214030eb80 1 --2- 172.21.15.119:0/397407482 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x5581daf453c0 0x5581daf477e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.287+0000 7f214030eb80 4 client.26528 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:23:02.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.287+0000 7f214030eb80 1 -- 172.21.15.119:0/397407482 --> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581daefa7a0 con 0x5581daf453c0 2023-12-07T10:23:02.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.287+0000 7f2125d9b700 1 --2- 172.21.15.119:0/397407482 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x5581daf453c0 0x5581daf477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.287+0000 7f2125d9b700 1 --2- 172.21.15.119:0/397407482 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x5581daf453c0 0x5581daf477e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c9e274a8bd2b51c7 server_cookie=8cf0ed4b071454cb in_seq=0 out_seq=0 2023-12-07T10:23:02.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.287+0000 7f210dffb700 10 client.26528 ms_handle_connect on v2:172.21.15.159:6834/2406380792 2023-12-07T10:23:02.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.288+0000 7f210dffb700 1 -- 172.21.15.119:0/397407482 <== mds.3 v2:172.21.15.159:6834/2406380792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5581daefa7a0 con 0x5581daf453c0 2023-12-07T10:23:02.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.288+0000 7f210dffb700 10 client.26528 handle_command_reply: tid=0 2023-12-07T10:23:02.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.288+0000 7f214030eb80 1 client.26528 shutdown 2023-12-07T10:23:02.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.288+0000 7f214030eb80 1 -- 172.21.15.119:0/397407482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf2bb40 msgr2=0x5581daf379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.288+0000 7f214030eb80 1 --2- 172.21.15.119:0/397407482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf2bb40 0x5581daf379a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f211801fbf0 tx=0x7f21180047c0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.288+0000 7f214030eb80 1 -- 172.21.15.119:0/397407482 shutdown_connections 2023-12-07T10:23:02.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.288+0000 7f214030eb80 1 --2- 172.21.15.119:0/397407482 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x5581daf453c0 0x5581daf477e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.288+0000 7f214030eb80 1 --2- 172.21.15.119:0/397407482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581daf30150 0x5581daf353d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.288+0000 7f214030eb80 1 --2- 172.21.15.119:0/397407482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf2bb40 0x5581daf379a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.288+0000 7f210dffb700 0 client.26528 ms_handle_reset on v2:172.21.15.159:6834/2406380792 2023-12-07T10:23:02.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.288+0000 7f214030eb80 1 --2- 172.21.15.119:0/397407482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581daf2c460 0x5581daf34ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.288+0000 7f214030eb80 1 -- 172.21.15.119:0/397407482 >> 172.21.15.119:0/397407482 conn(0x5581dae5a360 msgr2=0x5581daf31620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:02.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.289+0000 7f214030eb80 1 -- 172.21.15.119:0/397407482 shutdown_connections 2023-12-07T10:23:02.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.289+0000 7f214030eb80 1 -- 172.21.15.119:0/397407482 wait complete. 2023-12-07T10:23:02.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.289+0000 7f214030eb80 20 client.26528 trim_cache size 0 max 16384 2023-12-07T10:23:02.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.310+0000 7f214030eb80 1 Processor -- start 2023-12-07T10:23:02.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.310+0000 7f214030eb80 1 -- start start 2023-12-07T10:23:02.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.311+0000 7f214030eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581daf2bb40 0x5581daf4c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.311+0000 7f214030eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581daf2c460 0x5581daf3c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.311+0000 7f214030eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf30150 0x5581daf3c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.311+0000 7f214030eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581dacfe920 con 0x5581daf2bb40 2023-12-07T10:23:02.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.311+0000 7f214030eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581daf273e0 con 0x5581daf30150 2023-12-07T10:23:02.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.311+0000 7f214030eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581daf42a40 con 0x5581daf2c460 2023-12-07T10:23:02.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.311+0000 7f2126d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf30150 0x5581daf3c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.311+0000 7f2126d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf30150 0x5581daf3c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:41850/0 (socket says 172.21.15.119:41850) 2023-12-07T10:23:02.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.311+0000 7f2126d9d700 1 -- 172.21.15.119:0/1982715062 learned_addr learned my addr 172.21.15.119:0/1982715062 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:02.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.311+0000 7f212659c700 1 --2- 172.21.15.119:0/1982715062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581daf2bb40 0x5581daf4c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.311+0000 7f2125d9b700 1 --2- 172.21.15.119:0/1982715062 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581daf2c460 0x5581daf3c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.311+0000 7f2126d9d700 1 -- 172.21.15.119:0/1982715062 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581daf2c460 msgr2=0x5581daf3c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.311+0000 7f2126d9d700 1 --2- 172.21.15.119:0/1982715062 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581daf2c460 0x5581daf3c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.311+0000 7f2126d9d700 1 -- 172.21.15.119:0/1982715062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581daf2bb40 msgr2=0x5581daf4c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.311+0000 7f2126d9d700 1 --2- 172.21.15.119:0/1982715062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581daf2bb40 0x5581daf4c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.312+0000 7f2126d9d700 1 -- 172.21.15.119:0/1982715062 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581daf20040 con 0x5581daf30150 2023-12-07T10:23:02.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.312+0000 7f2125d9b700 1 --2- 172.21.15.119:0/1982715062 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581daf2c460 0x5581daf3c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:23:02.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.312+0000 7f2126d9d700 1 --2- 172.21.15.119:0/1982715062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf30150 0x5581daf3c890 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f212000f440 tx=0x7f2120014d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd0da22f1ff305f0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:02.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.312+0000 7f21157fa700 1 -- 172.21.15.119:0/1982715062 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2120014780 con 0x5581daf30150 2023-12-07T10:23:02.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.312+0000 7f21157fa700 1 -- 172.21.15.119:0/1982715062 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f212000b030 con 0x5581daf30150 2023-12-07T10:23:02.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.312+0000 7f21157fa700 1 -- 172.21.15.119:0/1982715062 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2120004600 con 0x5581daf30150 2023-12-07T10:23:02.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.313+0000 7f214030eb80 1 -- 172.21.15.119:0/1982715062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf30150 msgr2=0x5581daf3c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.313+0000 7f214030eb80 1 --2- 172.21.15.119:0/1982715062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf30150 0x5581daf3c890 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f212000f440 tx=0x7f2120014d30 comp rx=0 tx=0).stop 2023-12-07T10:23:02.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.313+0000 7f214030eb80 1 -- 172.21.15.119:0/1982715062 shutdown_connections 2023-12-07T10:23:02.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.313+0000 7f214030eb80 1 --2- 172.21.15.119:0/1982715062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf30150 0x5581daf3c890 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.313+0000 7f214030eb80 1 --2- 172.21.15.119:0/1982715062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581daf2bb40 0x5581daf4c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.313+0000 7f214030eb80 1 --2- 172.21.15.119:0/1982715062 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581daf2c460 0x5581daf3c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.313+0000 7f214030eb80 1 -- 172.21.15.119:0/1982715062 >> 172.21.15.119:0/1982715062 conn(0x5581dae5a360 msgr2=0x5581daf25fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:02.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.313+0000 7f214030eb80 1 -- 172.21.15.119:0/1982715062 shutdown_connections 2023-12-07T10:23:02.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.313+0000 7f214030eb80 1 -- 172.21.15.119:0/1982715062 wait complete. 2023-12-07T10:23:02.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.314+0000 7f214030eb80 1 -- start start 2023-12-07T10:23:02.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.314+0000 7f214030eb80 1 Processor -- start 2023-12-07T10:23:02.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.315+0000 7f214030eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581daf2bb40 0x5581daf2a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.315+0000 7f214030eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581daf2c460 0x5581daf2ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.315+0000 7f214030eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf30150 0x5581daf2b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.315+0000 7f214030eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581daf9a9b0 con 0x5581daf2bb40 2023-12-07T10:23:02.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.315+0000 7f214030eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581daf9ab20 con 0x5581daf30150 2023-12-07T10:23:02.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.315+0000 7f214030eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581daf9ace0 con 0x5581daf2c460 2023-12-07T10:23:02.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.315+0000 7f214030eb80 10 client.0 _ll_register_callbacks cb 0x5581dae20940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:02.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.315+0000 7f212659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581daf2bb40 0x5581daf2a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.315+0000 7f212659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581daf2bb40 0x5581daf2a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58122/0 (socket says 172.21.15.119:58122) 2023-12-07T10:23:02.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.315+0000 7f212659c700 1 -- 172.21.15.119:0/3671817689 learned_addr learned my addr 172.21.15.119:0/3671817689 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:02.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.315+0000 7f2125d9b700 1 --2- 172.21.15.119:0/3671817689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581daf2c460 0x5581daf2ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.315+0000 7f2126d9d700 1 --2- 172.21.15.119:0/3671817689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf30150 0x5581daf2b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.316+0000 7f2125d9b700 1 -- 172.21.15.119:0/3671817689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf30150 msgr2=0x5581daf2b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.316+0000 7f2125d9b700 1 --2- 172.21.15.119:0/3671817689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf30150 0x5581daf2b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.316+0000 7f2125d9b700 1 -- 172.21.15.119:0/3671817689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581daf2bb40 msgr2=0x5581daf2a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.316+0000 7f2125d9b700 1 --2- 172.21.15.119:0/3671817689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581daf2bb40 0x5581daf2a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.316+0000 7f2125d9b700 1 -- 172.21.15.119:0/3671817689 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581dacfc630 con 0x5581daf2c460 2023-12-07T10:23:02.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.316+0000 7f2126d9d700 1 --2- 172.21.15.119:0/3671817689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf30150 0x5581daf2b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:23:02.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.316+0000 7f212659c700 1 --2- 172.21.15.119:0/3671817689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581daf2bb40 0x5581daf2a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.316+0000 7f2125d9b700 1 --2- 172.21.15.119:0/3671817689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581daf2c460 0x5581daf2ae80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f211c09ba90 tx=0x7f211c022f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=760ac2eea0de25c4 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.316+0000 7f210dffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:23:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.316+0000 7f210dffb700 1 -- 172.21.15.119:0/3671817689 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c084ea0 con 0x5581daf2c460 2023-12-07T10:23:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.316+0000 7f214030eb80 1 -- 172.21.15.119:0/3671817689 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2120004bb0 con 0x5581daf2c460 2023-12-07T10:23:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.316+0000 7f210dffb700 1 -- 172.21.15.119:0/3671817689 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f211c09fc50 con 0x5581daf2c460 2023-12-07T10:23:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.316+0000 7f210dffb700 1 -- 172.21.15.119:0/3671817689 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f211c0a3e00 con 0x5581daf2c460 2023-12-07T10:23:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.316+0000 7f214030eb80 10 client.26521 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.316+0000 7f214030eb80 1 -- 172.21.15.119:0/3671817689 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5581daca8de0 con 0x5581daf2c460 2023-12-07T10:23:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.317+0000 7f210dffb700 1 -- 172.21.15.119:0/3671817689 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f211c065080 con 0x5581daf2c460 2023-12-07T10:23:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.317+0000 7f210dffb700 1 -- 172.21.15.119:0/3671817689 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f211c09aff0 con 0x5581daf2c460 2023-12-07T10:23:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.317+0000 7f214030eb80 10 client.26521 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.317+0000 7f214030eb80 10 client.26521 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-12-07T10:23:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.317+0000 7f214030eb80 20 client.26521 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:02.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.317+0000 7f214030eb80 1 --2- 172.21.15.119:0/3671817689 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x5581daf9c140 0x5581daf9e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.317+0000 7f214030eb80 4 client.26521 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:23:02.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.317+0000 7f214030eb80 1 -- 172.21.15.119:0/3671817689 --> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5581dae45fd0 con 0x5581daf9c140 2023-12-07T10:23:02.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.317+0000 7f212659c700 1 --2- 172.21.15.119:0/3671817689 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x5581daf9c140 0x5581daf9e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.318+0000 7f212659c700 1 --2- 172.21.15.119:0/3671817689 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x5581daf9c140 0x5581daf9e560 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f313a9b1538930a server_cookie=95465647a47debd4 in_seq=0 out_seq=0 2023-12-07T10:23:02.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.318+0000 7f210dffb700 10 client.26521 ms_handle_connect on v2:172.21.15.159:6834/2406380792 2023-12-07T10:23:02.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.318+0000 7f210dffb700 1 -- 172.21.15.119:0/3671817689 <== mds.3 v2:172.21.15.159:6834/2406380792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5581dae45fd0 con 0x5581daf9c140 2023-12-07T10:23:02.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.318+0000 7f210dffb700 10 client.26521 handle_command_reply: tid=0 2023-12-07T10:23:02.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.318+0000 7f214030eb80 1 client.26521 shutdown 2023-12-07T10:23:02.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.318+0000 7f214030eb80 1 -- 172.21.15.119:0/3671817689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581daf2c460 msgr2=0x5581daf2ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.318+0000 7f214030eb80 1 --2- 172.21.15.119:0/3671817689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581daf2c460 0x5581daf2ae80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f211c09ba90 tx=0x7f211c022f90 comp rx=0 tx=0).stop 2023-12-07T10:23:02.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.318+0000 7f214030eb80 1 -- 172.21.15.119:0/3671817689 shutdown_connections 2023-12-07T10:23:02.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.318+0000 7f214030eb80 1 --2- 172.21.15.119:0/3671817689 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x5581daf9c140 0x5581daf9e560 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.318+0000 7f214030eb80 1 --2- 172.21.15.119:0/3671817689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5581daf30150 0x5581daf2b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.318+0000 7f214030eb80 1 --2- 172.21.15.119:0/3671817689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5581daf2bb40 0x5581daf2a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.318+0000 7f210dffb700 0 client.26521 ms_handle_reset on v2:172.21.15.159:6834/2406380792 2023-12-07T10:23:02.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.318+0000 7f214030eb80 1 --2- 172.21.15.119:0/3671817689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581daf2c460 0x5581daf2ae80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.318+0000 7f214030eb80 1 -- 172.21.15.119:0/3671817689 >> 172.21.15.119:0/3671817689 conn(0x5581dae5a360 msgr2=0x5581daf4c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:02.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.318+0000 7f214030eb80 1 -- 172.21.15.119:0/3671817689 shutdown_connections 2023-12-07T10:23:02.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.318+0000 7f214030eb80 1 -- 172.21.15.119:0/3671817689 wait complete. 2023-12-07T10:23:02.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.318+0000 7f214030eb80 20 client.26521 trim_cache size 0 max 16384 2023-12-07T10:23:02.324 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:23:02.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.319+0000 7f212c9ff700 1 -- 172.21.15.119:0/3193224965 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f21100a1c80 msgr2=0x7f21100a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.319+0000 7f212c9ff700 1 --2- 172.21.15.119:0/3193224965 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f21100a1c80 0x7f21100a4140 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2118009be0 tx=0x7f2118009260 comp rx=0 tx=0).stop 2023-12-07T10:23:02.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.319+0000 7f212c9ff700 1 -- 172.21.15.119:0/3193224965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128100e20 msgr2=0x7f21280fe390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.319+0000 7f212c9ff700 1 --2- 172.21.15.119:0/3193224965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128100e20 0x7f21280fe390 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f211c005910 tx=0x7f211c004730 comp rx=0 tx=0).stop 2023-12-07T10:23:02.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.319+0000 7f212c9ff700 1 -- 172.21.15.119:0/3193224965 shutdown_connections 2023-12-07T10:23:02.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.319+0000 7f212c9ff700 1 --2- 172.21.15.119:0/3193224965 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f21100a1c80 0x7f21100a4140 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.319+0000 7f212c9ff700 1 --2- 172.21.15.119:0/3193224965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f21281053d0 0x7f21280fe8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.319+0000 7f212c9ff700 1 --2- 172.21.15.119:0/3193224965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2128100460 0x7f21280fde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.319+0000 7f212c9ff700 1 --2- 172.21.15.119:0/3193224965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128100e20 0x7f21280fe390 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.319+0000 7f212c9ff700 1 -- 172.21.15.119:0/3193224965 >> 172.21.15.119:0/3193224965 conn(0x7f2128077610 msgr2=0x7f2128068ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:02.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.319+0000 7f212c9ff700 1 -- 172.21.15.119:0/3193224965 shutdown_connections 2023-12-07T10:23:02.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.320+0000 7f212c9ff700 1 -- 172.21.15.119:0/3193224965 wait complete. 2023-12-07T10:23:02.346 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-07T10:23:02.547 INFO:tasks.workunit.client.0.smithi119.stdout: 1 115879 8.53 MB/sec warmup 75 sec latency 71.349 ms 2023-12-07T10:23:02.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.596+0000 7f9cfbbff700 1 -- 172.21.15.119:0/3031671660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cf4075b40 msgr2=0x7f9cf4075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.596+0000 7f9cfbbff700 1 --2- 172.21.15.119:0/3031671660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cf4075b40 0x7f9cf4075f20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009990 tx=0x7f9cf0009ca0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.596+0000 7f9cfbbff700 1 -- 172.21.15.119:0/3031671660 shutdown_connections 2023-12-07T10:23:02.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.596+0000 7f9cfbbff700 1 --2- 172.21.15.119:0/3031671660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cf4112410 0x7f9cf4114800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.596+0000 7f9cfbbff700 1 --2- 172.21.15.119:0/3031671660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cf4075b40 0x7f9cf4075f20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.596+0000 7f9cfbbff700 1 --2- 172.21.15.119:0/3031671660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cf4076460 0x7f9cf4111ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.596+0000 7f9cfbbff700 1 -- 172.21.15.119:0/3031671660 >> 172.21.15.119:0/3031671660 conn(0x7f9cf40ff220 msgr2=0x7f9cf4101640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:02.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.596+0000 7f9cfbbff700 1 -- 172.21.15.119:0/3031671660 shutdown_connections 2023-12-07T10:23:02.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.596+0000 7f9cfbbff700 1 -- 172.21.15.119:0/3031671660 wait complete. 2023-12-07T10:23:02.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.597+0000 7f9cfbbff700 1 Processor -- start 2023-12-07T10:23:02.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.597+0000 7f9cfbbff700 1 -- start start 2023-12-07T10:23:02.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.598+0000 7f9cfbbff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cf4075b40 0x7f9cf419e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.598+0000 7f9cfbbff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cf4076460 0x7f9cf419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.598+0000 7f9cfbbff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cf4112410 0x7f9cf41a2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.598+0000 7f9cfbbff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf41a2fd0 con 0x7f9cf4112410 2023-12-07T10:23:02.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.598+0000 7f9cfbbff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf41a3140 con 0x7f9cf4075b40 2023-12-07T10:23:02.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.598+0000 7f9cfbbff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf41a3300 con 0x7f9cf4076460 2023-12-07T10:23:02.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.598+0000 7f9cf919a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cf4076460 0x7f9cf419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.598+0000 7f9cf999b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cf4075b40 0x7f9cf419e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.598+0000 7f9cf919a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cf4076460 0x7f9cf419e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:52830/0 (socket says 172.21.15.119:52830) 2023-12-07T10:23:02.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.598+0000 7f9cf919a700 1 -- 172.21.15.119:0/1954681056 learned_addr learned my addr 172.21.15.119:0/1954681056 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:02.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.598+0000 7f9cfa19c700 1 --2- 172.21.15.119:0/1954681056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cf4112410 0x7f9cf41a2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.598+0000 7f9cf919a700 1 -- 172.21.15.119:0/1954681056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cf4075b40 msgr2=0x7f9cf419e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.598+0000 7f9cf919a700 1 --2- 172.21.15.119:0/1954681056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cf4075b40 0x7f9cf419e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.598+0000 7f9cf919a700 1 -- 172.21.15.119:0/1954681056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cf4112410 msgr2=0x7f9cf41a2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.598+0000 7f9cf919a700 1 --2- 172.21.15.119:0/1954681056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cf4112410 0x7f9cf41a2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.598+0000 7f9cf919a700 1 -- 172.21.15.119:0/1954681056 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf00095f0 con 0x7f9cf4076460 2023-12-07T10:23:02.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.598+0000 7f9cf999b700 1 --2- 172.21.15.119:0/1954681056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cf4075b40 0x7f9cf419e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:02.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.599+0000 7f9cf919a700 1 --2- 172.21.15.119:0/1954681056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cf4076460 0x7f9cf419e5f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400e9a0 tx=0x7f9ce400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:02.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.599+0000 7f9ceaffd700 1 -- 172.21.15.119:0/1954681056 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4009790 con 0x7f9cf4076460 2023-12-07T10:23:02.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.599+0000 7f9ceaffd700 1 -- 172.21.15.119:0/1954681056 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ce4010430 con 0x7f9cf4076460 2023-12-07T10:23:02.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.599+0000 7f9cfbbff700 1 -- 172.21.15.119:0/1954681056 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf41a35e0 con 0x7f9cf4076460 2023-12-07T10:23:02.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.599+0000 7f9ceaffd700 1 -- 172.21.15.119:0/1954681056 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4018680 con 0x7f9cf4076460 2023-12-07T10:23:02.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.599+0000 7f9cfbbff700 1 -- 172.21.15.119:0/1954681056 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf41aae20 con 0x7f9cf4076460 2023-12-07T10:23:02.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.602+0000 7f9ceaffd700 1 -- 172.21.15.119:0/1954681056 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f9ce40105a0 con 0x7f9cf4076460 2023-12-07T10:23:02.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.604+0000 7f9ceaffd700 1 --2- 172.21.15.119:0/1954681056 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9ce00a1c80 0x7f9ce00a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.604+0000 7f9ceaffd700 1 -- 172.21.15.119:0/1954681056 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9ce4014070 con 0x7f9cf4076460 2023-12-07T10:23:02.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.604+0000 7f9d0f465b80 1 Processor -- start 2023-12-07T10:23:02.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.604+0000 7f9cf999b700 1 --2- 172.21.15.119:0/1954681056 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9ce00a1c80 0x7f9ce00a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.604+0000 7f9d0f465b80 1 -- start start 2023-12-07T10:23:02.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.605+0000 7f9d0f465b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d35acceb40 0x55d35accef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.605+0000 7f9d0f465b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d35accf460 0x55d35acd2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.605+0000 7f9d0f465b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 0x55d35acd3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.605+0000 7f9d0f465b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35ac427c0 con 0x55d35acceb40 2023-12-07T10:23:02.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.605+0000 7f9d0f465b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35ac297a0 con 0x55d35accf460 2023-12-07T10:23:02.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.605+0000 7f9d0f465b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35ac270c0 con 0x55d35acd3150 2023-12-07T10:23:02.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.605+0000 7f9cfa19c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 0x55d35acd3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.605+0000 7f9cf999b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d35acceb40 0x55d35accef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.605+0000 7f9cfa19c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 0x55d35acd3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:52842/0 (socket says 172.21.15.119:52842) 2023-12-07T10:23:02.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.605+0000 7f9cfa19c700 1 -- 172.21.15.119:0/16671177 learned_addr learned my addr 172.21.15.119:0/16671177 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:02.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.605+0000 7f9cf919a700 1 --2- 172.21.15.119:0/16671177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d35accf460 0x55d35acd2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.605+0000 7f9cf999b700 1 --2- 172.21.15.119:0/1954681056 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9ce00a1c80 0x7f9ce00a4140 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0006010 tx=0x7f9cf000b260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:02.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.605+0000 7f9cfa19c700 1 -- 172.21.15.119:0/16671177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d35accf460 msgr2=0x55d35acd2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.605+0000 7f9cfa19c700 1 --2- 172.21.15.119:0/16671177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d35accf460 0x55d35acd2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.605+0000 7f9cfa19c700 1 -- 172.21.15.119:0/16671177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d35acceb40 msgr2=0x55d35accef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.605+0000 7f9cfa19c700 1 --2- 172.21.15.119:0/16671177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d35acceb40 0x55d35accef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.605+0000 7f9cfa19c700 1 -- 172.21.15.119:0/16671177 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d35aa9f630 con 0x55d35acd3150 2023-12-07T10:23:02.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.606+0000 7f9cfa19c700 1 --2- 172.21.15.119:0/16671177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 0x55d35acd3570 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9cec011630 tx=0x7f9cec009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dce786b354de3d78 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:02.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.606+0000 7f9cdffff700 1 -- 172.21.15.119:0/16671177 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec00e040 con 0x55d35acd3150 2023-12-07T10:23:02.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.606+0000 7f9cdffff700 1 -- 172.21.15.119:0/16671177 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cec00f040 con 0x55d35acd3150 2023-12-07T10:23:02.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.606+0000 7f9cdffff700 1 -- 172.21.15.119:0/16671177 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec004720 con 0x55d35acd3150 2023-12-07T10:23:02.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.607+0000 7f9d0f465b80 1 -- 172.21.15.119:0/16671177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 msgr2=0x55d35acd3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.607+0000 7f9d0f465b80 1 --2- 172.21.15.119:0/16671177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 0x55d35acd3570 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9cec011630 tx=0x7f9cec009920 comp rx=0 tx=0).stop 2023-12-07T10:23:02.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.607+0000 7f9d0f465b80 1 -- 172.21.15.119:0/16671177 shutdown_connections 2023-12-07T10:23:02.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.607+0000 7f9d0f465b80 1 --2- 172.21.15.119:0/16671177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 0x55d35acd3570 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.607+0000 7f9d0f465b80 1 --2- 172.21.15.119:0/16671177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d35acceb40 0x55d35accef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.607+0000 7f9d0f465b80 1 --2- 172.21.15.119:0/16671177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d35accf460 0x55d35acd2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.607+0000 7f9d0f465b80 1 -- 172.21.15.119:0/16671177 >> 172.21.15.119:0/16671177 conn(0x55d35abfd360 msgr2=0x55d35acc9130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:02.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.607+0000 7f9d0f465b80 1 -- 172.21.15.119:0/16671177 shutdown_connections 2023-12-07T10:23:02.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.607+0000 7f9d0f465b80 1 -- 172.21.15.119:0/16671177 wait complete. 2023-12-07T10:23:02.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.608+0000 7f9d0f465b80 1 -- start start 2023-12-07T10:23:02.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.608+0000 7f9d0f465b80 1 Processor -- start 2023-12-07T10:23:02.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.609+0000 7f9d0f465b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d35acceb40 0x55d35acda930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.609+0000 7f9d0f465b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d35accf460 0x55d35acd7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.609+0000 7f9d0f465b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 0x55d35acd8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.609+0000 7f9d0f465b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35ac229b0 con 0x55d35accf460 2023-12-07T10:23:02.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.609+0000 7f9d0f465b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35abdc560 con 0x55d35acceb40 2023-12-07T10:23:02.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.609+0000 7f9d0f465b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35abdc200 con 0x55d35acd3150 2023-12-07T10:23:02.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.609+0000 7f9d0f465b80 10 client.0 _ll_register_callbacks cb 0x55d35abc3940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:02.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.609+0000 7f9cfa19c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 0x55d35acd8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.609+0000 7f9cf999b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d35acceb40 0x55d35acda930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.609+0000 7f9cfa19c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 0x55d35acd8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:52846/0 (socket says 172.21.15.119:52846) 2023-12-07T10:23:02.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.609+0000 7f9cfa19c700 1 -- 172.21.15.119:0/623691148 learned_addr learned my addr 172.21.15.119:0/623691148 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:02.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.609+0000 7f9cf919a700 1 --2- 172.21.15.119:0/623691148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d35accf460 0x55d35acd7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.610+0000 7f9cf999b700 1 -- 172.21.15.119:0/623691148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 msgr2=0x55d35acd8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.610+0000 7f9cf999b700 1 --2- 172.21.15.119:0/623691148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 0x55d35acd8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.610+0000 7f9cf999b700 1 -- 172.21.15.119:0/623691148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d35accf460 msgr2=0x55d35acd7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.610+0000 7f9cf999b700 1 --2- 172.21.15.119:0/623691148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d35accf460 0x55d35acd7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.610+0000 7f9cf999b700 1 -- 172.21.15.119:0/623691148 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cec0112b0 con 0x55d35acceb40 2023-12-07T10:23:02.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.610+0000 7f9cfa19c700 1 --2- 172.21.15.119:0/623691148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 0x55d35acd8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:02.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.610+0000 7f9cf999b700 1 --2- 172.21.15.119:0/623691148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d35acceb40 0x55d35acda930 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9cf000b6f0 tx=0x7f9cf001a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fedd03f4c511009 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:02.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.610+0000 7f9cdd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:23:02.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.610+0000 7f9cdd7fa700 1 -- 172.21.15.119:0/623691148 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf000f870 con 0x55d35acceb40 2023-12-07T10:23:02.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.610+0000 7f9d0f465b80 1 -- 172.21.15.119:0/623691148 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d35acc1b00 con 0x55d35acceb40 2023-12-07T10:23:02.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.610+0000 7f9cdd7fa700 1 -- 172.21.15.119:0/623691148 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9cf000f9d0 con 0x55d35acceb40 2023-12-07T10:23:02.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.611+0000 7f9cdd7fa700 1 -- 172.21.15.119:0/623691148 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cf00056f0 con 0x55d35acceb40 2023-12-07T10:23:02.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.611+0000 7f9d0f465b80 10 client.26546 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:02.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.611+0000 7f9d0f465b80 1 -- 172.21.15.119:0/623691148 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55d35aa4bde0 con 0x55d35acceb40 2023-12-07T10:23:02.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.611+0000 7f9cdd7fa700 1 -- 172.21.15.119:0/623691148 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d35acc1b00 con 0x55d35acceb40 2023-12-07T10:23:02.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.611+0000 7f9cdd7fa700 1 -- 172.21.15.119:0/623691148 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f9cf002fc00 con 0x55d35acceb40 2023-12-07T10:23:02.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.611+0000 7f9d0f465b80 10 client.26546 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:02.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.611+0000 7f9d0f465b80 10 client.26546 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-07T10:23:02.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.611+0000 7f9d0f465b80 20 client.26546 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:02.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.611+0000 7f9d0f465b80 1 --2- 172.21.15.119:0/623691148 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x55d35ace83b0 0x55d35acea7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.611+0000 7f9d0f465b80 4 client.26546 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:23:02.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.611+0000 7f9d0f465b80 1 -- 172.21.15.119:0/623691148 --> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d35ac9d7a0 con 0x55d35ace83b0 2023-12-07T10:23:02.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.612+0000 7f9cf919a700 1 --2- 172.21.15.119:0/623691148 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x55d35ace83b0 0x55d35acea7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.612+0000 7f9cf919a700 1 --2- 172.21.15.119:0/623691148 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x55d35ace83b0 0x55d35acea7a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5481a7fc467221c5 server_cookie=be374fde49dec21f in_seq=0 out_seq=0 2023-12-07T10:23:02.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.612+0000 7f9cdd7fa700 10 client.26546 ms_handle_connect on v2:172.21.15.159:6836/3106097520 2023-12-07T10:23:02.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.613+0000 7f9cdd7fa700 1 -- 172.21.15.119:0/623691148 <== mds.2 v2:172.21.15.159:6836/3106097520 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d35ac9d7a0 con 0x55d35ace83b0 2023-12-07T10:23:02.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.613+0000 7f9cdd7fa700 10 client.26546 handle_command_reply: tid=0 2023-12-07T10:23:02.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.613+0000 7f9d0f465b80 1 client.26546 shutdown 2023-12-07T10:23:02.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.613+0000 7f9d0f465b80 1 -- 172.21.15.119:0/623691148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d35acceb40 msgr2=0x55d35acda930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.613+0000 7f9d0f465b80 1 --2- 172.21.15.119:0/623691148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d35acceb40 0x55d35acda930 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9cf000b6f0 tx=0x7f9cf001a050 comp rx=0 tx=0).stop 2023-12-07T10:23:02.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.613+0000 7f9d0f465b80 1 -- 172.21.15.119:0/623691148 shutdown_connections 2023-12-07T10:23:02.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.613+0000 7f9d0f465b80 1 --2- 172.21.15.119:0/623691148 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x55d35ace83b0 0x55d35acea7a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.613+0000 7f9d0f465b80 1 --2- 172.21.15.119:0/623691148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 0x55d35acd8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.613+0000 7f9d0f465b80 1 --2- 172.21.15.119:0/623691148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d35acceb40 0x55d35acda930 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.613+0000 7f9cdd7fa700 0 client.26546 ms_handle_reset on v2:172.21.15.159:6836/3106097520 2023-12-07T10:23:02.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.613+0000 7f9d0f465b80 1 --2- 172.21.15.119:0/623691148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d35accf460 0x55d35acd7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.613+0000 7f9d0f465b80 1 -- 172.21.15.119:0/623691148 >> 172.21.15.119:0/623691148 conn(0x55d35abfd360 msgr2=0x55d35acd4620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:02.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.613+0000 7f9d0f465b80 1 -- 172.21.15.119:0/623691148 shutdown_connections 2023-12-07T10:23:02.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.613+0000 7f9d0f465b80 1 -- 172.21.15.119:0/623691148 wait complete. 2023-12-07T10:23:02.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.613+0000 7f9d0f465b80 20 client.26546 trim_cache size 0 max 16384 2023-12-07T10:23:02.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.630+0000 7f9d0f465b80 1 Processor -- start 2023-12-07T10:23:02.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.631+0000 7f9d0f465b80 1 -- start start 2023-12-07T10:23:02.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.631+0000 7f9d0f465b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d35acceb40 0x55d35acef100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.631+0000 7f9d0f465b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d35accf460 0x55d35acdf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.631+0000 7f9d0f465b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 0x55d35acdf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.631+0000 7f9d0f465b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35aaa1920 con 0x55d35acceb40 2023-12-07T10:23:02.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.631+0000 7f9d0f465b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35acc5b50 con 0x55d35accf460 2023-12-07T10:23:02.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.631+0000 7f9d0f465b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35acd89e0 con 0x55d35acd3150 2023-12-07T10:23:02.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.631+0000 7f9cfa19c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 0x55d35acdf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.631+0000 7f9cfa19c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 0x55d35acdf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:52862/0 (socket says 172.21.15.119:52862) 2023-12-07T10:23:02.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.631+0000 7f9cfa19c700 1 -- 172.21.15.119:0/2697788451 learned_addr learned my addr 172.21.15.119:0/2697788451 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:02.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.631+0000 7f9cf999b700 1 --2- 172.21.15.119:0/2697788451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d35acceb40 0x55d35acef100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.631+0000 7f9cf919a700 1 --2- 172.21.15.119:0/2697788451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d35accf460 0x55d35acdf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.632+0000 7f9cf919a700 1 -- 172.21.15.119:0/2697788451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 msgr2=0x55d35acdf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.632+0000 7f9cf919a700 1 --2- 172.21.15.119:0/2697788451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 0x55d35acdf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.632+0000 7f9cf919a700 1 -- 172.21.15.119:0/2697788451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d35acceb40 msgr2=0x55d35acef100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.632+0000 7f9cf919a700 1 --2- 172.21.15.119:0/2697788451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d35acceb40 0x55d35acef100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.632+0000 7f9cf919a700 1 -- 172.21.15.119:0/2697788451 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d35aa34c60 con 0x55d35accf460 2023-12-07T10:23:02.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.632+0000 7f9cfa19c700 1 --2- 172.21.15.119:0/2697788451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 0x55d35acdf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:02.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.632+0000 7f9cf919a700 1 --2- 172.21.15.119:0/2697788451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d35accf460 0x55d35acdf350 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40cb2b0 tx=0x7f9ce40ba040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d1db185bc6e595 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:02.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.632+0000 7f9ce8bff700 1 -- 172.21.15.119:0/2697788451 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce40a2c90 con 0x55d35accf460 2023-12-07T10:23:02.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.632+0000 7f9ce8bff700 1 -- 172.21.15.119:0/2697788451 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ce4098a40 con 0x55d35accf460 2023-12-07T10:23:02.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.632+0000 7f9ce8bff700 1 -- 172.21.15.119:0/2697788451 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4010980 con 0x55d35accf460 2023-12-07T10:23:02.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.633+0000 7f9d0f465b80 1 -- 172.21.15.119:0/2697788451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d35accf460 msgr2=0x55d35acdf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.633+0000 7f9d0f465b80 1 --2- 172.21.15.119:0/2697788451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d35accf460 0x55d35acdf350 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40cb2b0 tx=0x7f9ce40ba040 comp rx=0 tx=0).stop 2023-12-07T10:23:02.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.633+0000 7f9d0f465b80 1 -- 172.21.15.119:0/2697788451 shutdown_connections 2023-12-07T10:23:02.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.633+0000 7f9d0f465b80 1 --2- 172.21.15.119:0/2697788451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 0x55d35acdf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.633+0000 7f9d0f465b80 1 --2- 172.21.15.119:0/2697788451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d35acceb40 0x55d35acef100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.633+0000 7f9d0f465b80 1 --2- 172.21.15.119:0/2697788451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d35accf460 0x55d35acdf350 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.633+0000 7f9d0f465b80 1 -- 172.21.15.119:0/2697788451 >> 172.21.15.119:0/2697788451 conn(0x55d35abfd360 msgr2=0x55d35acda5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:02.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.633+0000 7f9d0f465b80 1 -- 172.21.15.119:0/2697788451 shutdown_connections 2023-12-07T10:23:02.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.633+0000 7f9d0f465b80 1 -- 172.21.15.119:0/2697788451 wait complete. 2023-12-07T10:23:02.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.634+0000 7f9d0f465b80 1 -- start start 2023-12-07T10:23:02.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.634+0000 7f9d0f465b80 1 Processor -- start 2023-12-07T10:23:02.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.635+0000 7f9d0f465b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d35acceb40 0x55d35acdd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.635+0000 7f9d0f465b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d35accf460 0x55d35acdda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.635+0000 7f9d0f465b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 0x55d35ad3d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.635+0000 7f9d0f465b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35ad3ddc0 con 0x55d35acceb40 2023-12-07T10:23:02.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.635+0000 7f9d0f465b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35ad3df00 con 0x55d35accf460 2023-12-07T10:23:02.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.635+0000 7f9d0f465b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35ad3e040 con 0x55d35acd3150 2023-12-07T10:23:02.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.635+0000 7f9d0f465b80 10 client.0 _ll_register_callbacks cb 0x55d35abc3940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:02.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.635+0000 7f9cfa19c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 0x55d35ad3d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.635+0000 7f9cfa19c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 0x55d35ad3d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:52878/0 (socket says 172.21.15.119:52878) 2023-12-07T10:23:02.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.635+0000 7f9cfa19c700 1 -- 172.21.15.119:0/2621398865 learned_addr learned my addr 172.21.15.119:0/2621398865 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:02.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.635+0000 7f9cf999b700 1 --2- 172.21.15.119:0/2621398865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d35acceb40 0x55d35acdd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.635+0000 7f9cf919a700 1 --2- 172.21.15.119:0/2621398865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d35accf460 0x55d35acdda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.635+0000 7f9cfa19c700 1 -- 172.21.15.119:0/2621398865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d35accf460 msgr2=0x55d35acdda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.635+0000 7f9cfa19c700 1 --2- 172.21.15.119:0/2621398865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d35accf460 0x55d35acdda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.635+0000 7f9cfa19c700 1 -- 172.21.15.119:0/2621398865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d35acceb40 msgr2=0x55d35acdd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.635+0000 7f9cfa19c700 1 --2- 172.21.15.119:0/2621398865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d35acceb40 0x55d35acdd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.635+0000 7f9cfa19c700 1 -- 172.21.15.119:0/2621398865 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d35aa9f630 con 0x55d35acd3150 2023-12-07T10:23:02.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.636+0000 7f9cfa19c700 1 --2- 172.21.15.119:0/2621398865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 0x55d35ad3d880 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9cec000f80 tx=0x7f9cec005610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a18ec31f411ff6de server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:02.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.636+0000 7f9cdd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:23:02.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.636+0000 7f9cdd7fa700 1 -- 172.21.15.119:0/2621398865 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec00e040 con 0x55d35acd3150 2023-12-07T10:23:02.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.636+0000 7f9d0f465b80 1 -- 172.21.15.119:0/2621398865 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cec00f040 con 0x55d35acd3150 2023-12-07T10:23:02.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.636+0000 7f9cdd7fa700 1 -- 172.21.15.119:0/2621398865 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9cec0092e0 con 0x55d35acd3150 2023-12-07T10:23:02.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.636+0000 7f9cdd7fa700 1 -- 172.21.15.119:0/2621398865 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cec005990 con 0x55d35acd3150 2023-12-07T10:23:02.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.636+0000 7f9d0f465b80 10 client.26539 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:02.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.636+0000 7f9d0f465b80 1 -- 172.21.15.119:0/2621398865 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f9cf41aae20 con 0x55d35acd3150 2023-12-07T10:23:02.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.637+0000 7f9cdd7fa700 1 -- 172.21.15.119:0/2621398865 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cec00f040 con 0x55d35acd3150 2023-12-07T10:23:02.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.637+0000 7f9cdd7fa700 1 -- 172.21.15.119:0/2621398865 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f9cec015570 con 0x55d35acd3150 2023-12-07T10:23:02.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.637+0000 7f9d0f465b80 10 client.26539 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:02.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.637+0000 7f9d0f465b80 10 client.26539 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-07T10:23:02.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.637+0000 7f9d0f465b80 20 client.26539 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:02.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.637+0000 7f9d0f465b80 1 --2- 172.21.15.119:0/2621398865 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x55d35ad3f840 0x55d35ad41c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.637+0000 7f9d0f465b80 4 client.26539 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:23:02.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.637+0000 7f9d0f465b80 1 -- 172.21.15.119:0/2621398865 --> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d35abe8fd0 con 0x55d35ad3f840 2023-12-07T10:23:02.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.637+0000 7f9cf999b700 1 --2- 172.21.15.119:0/2621398865 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x55d35ad3f840 0x55d35ad41c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.638+0000 7f9cf999b700 1 --2- 172.21.15.119:0/2621398865 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x55d35ad3f840 0x55d35ad41c60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=58cfe1f48cb8cfdf server_cookie=8132d4d65b9d57ed in_seq=0 out_seq=0 2023-12-07T10:23:02.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.638+0000 7f9cdd7fa700 10 client.26539 ms_handle_connect on v2:172.21.15.159:6836/3106097520 2023-12-07T10:23:02.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.638+0000 7f9cdd7fa700 1 -- 172.21.15.119:0/2621398865 <== mds.2 v2:172.21.15.159:6836/3106097520 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d35abe8fd0 con 0x55d35ad3f840 2023-12-07T10:23:02.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.638+0000 7f9cdd7fa700 10 client.26539 handle_command_reply: tid=0 2023-12-07T10:23:02.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.638+0000 7f9d0f465b80 1 client.26539 shutdown 2023-12-07T10:23:02.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.638+0000 7f9d0f465b80 1 -- 172.21.15.119:0/2621398865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 msgr2=0x55d35ad3d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.638+0000 7f9d0f465b80 1 --2- 172.21.15.119:0/2621398865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 0x55d35ad3d880 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9cec000f80 tx=0x7f9cec005610 comp rx=0 tx=0).stop 2023-12-07T10:23:02.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.638+0000 7f9d0f465b80 1 -- 172.21.15.119:0/2621398865 shutdown_connections 2023-12-07T10:23:02.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.638+0000 7f9d0f465b80 1 --2- 172.21.15.119:0/2621398865 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x55d35ad3f840 0x55d35ad41c60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.638+0000 7f9d0f465b80 1 --2- 172.21.15.119:0/2621398865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d35acd3150 0x55d35ad3d880 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.638+0000 7f9d0f465b80 1 --2- 172.21.15.119:0/2621398865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d35acceb40 0x55d35acdd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.638+0000 7f9cdd7fa700 0 client.26539 ms_handle_reset on v2:172.21.15.159:6836/3106097520 2023-12-07T10:23:02.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.638+0000 7f9d0f465b80 1 --2- 172.21.15.119:0/2621398865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d35accf460 0x55d35acdda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.638+0000 7f9d0f465b80 1 -- 172.21.15.119:0/2621398865 >> 172.21.15.119:0/2621398865 conn(0x55d35abfd360 msgr2=0x55d35acef100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:02.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.638+0000 7f9d0f465b80 1 -- 172.21.15.119:0/2621398865 shutdown_connections 2023-12-07T10:23:02.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.638+0000 7f9d0f465b80 1 -- 172.21.15.119:0/2621398865 wait complete. 2023-12-07T10:23:02.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.638+0000 7f9d0f465b80 20 client.26539 trim_cache size 0 max 16384 2023-12-07T10:23:02.650 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:23:02.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.639+0000 7f9cfbbff700 1 -- 172.21.15.119:0/1954681056 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9ce00a1c80 msgr2=0x7f9ce00a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.639+0000 7f9cfbbff700 1 --2- 172.21.15.119:0/1954681056 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9ce00a1c80 0x7f9ce00a4140 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0006010 tx=0x7f9cf000b260 comp rx=0 tx=0).stop 2023-12-07T10:23:02.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.639+0000 7f9cfbbff700 1 -- 172.21.15.119:0/1954681056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cf4076460 msgr2=0x7f9cf419e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.639+0000 7f9cfbbff700 1 --2- 172.21.15.119:0/1954681056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cf4076460 0x7f9cf419e5f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400e9a0 tx=0x7f9ce400ecb0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.639+0000 7f9cfbbff700 1 -- 172.21.15.119:0/1954681056 shutdown_connections 2023-12-07T10:23:02.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.639+0000 7f9cfbbff700 1 --2- 172.21.15.119:0/1954681056 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9ce00a1c80 0x7f9ce00a4140 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.639+0000 7f9cfbbff700 1 --2- 172.21.15.119:0/1954681056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cf4112410 0x7f9cf41a2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.639+0000 7f9cfbbff700 1 --2- 172.21.15.119:0/1954681056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cf4075b40 0x7f9cf419e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.639+0000 7f9cfbbff700 1 --2- 172.21.15.119:0/1954681056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cf4076460 0x7f9cf419e5f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.639+0000 7f9cfbbff700 1 -- 172.21.15.119:0/1954681056 >> 172.21.15.119:0/1954681056 conn(0x7f9cf40ff220 msgr2=0x7f9cf4101610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:02.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.639+0000 7f9cfbbff700 1 -- 172.21.15.119:0/1954681056 shutdown_connections 2023-12-07T10:23:02.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.639+0000 7f9cfbbff700 1 -- 172.21.15.119:0/1954681056 wait complete. 2023-12-07T10:23:02.660 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-07T10:23:02.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.890+0000 7fd88fdff700 1 -- 172.21.15.119:0/2316530170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd888100f40 msgr2=0x7fd888101320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.890+0000 7fd88fdff700 1 --2- 172.21.15.119:0/2316530170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd888100f40 0x7fd888101320 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd8840098f0 tx=0x7fd884009c00 comp rx=0 tx=0).stop 2023-12-07T10:23:02.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.890+0000 7fd88fdff700 1 -- 172.21.15.119:0/2316530170 shutdown_connections 2023-12-07T10:23:02.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.890+0000 7fd88fdff700 1 --2- 172.21.15.119:0/2316530170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd88810dfb0 0x7fd8881103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.890+0000 7fd88fdff700 1 --2- 172.21.15.119:0/2316530170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd888100f40 0x7fd888101320 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.890+0000 7fd88fdff700 1 --2- 172.21.15.119:0/2316530170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd888101860 0x7fd88810da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.890+0000 7fd88fdff700 1 -- 172.21.15.119:0/2316530170 >> 172.21.15.119:0/2316530170 conn(0x7fd888077610 msgr2=0x7fd888077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:02.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.890+0000 7fd88fdff700 1 -- 172.21.15.119:0/2316530170 shutdown_connections 2023-12-07T10:23:02.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.890+0000 7fd88fdff700 1 -- 172.21.15.119:0/2316530170 wait complete. 2023-12-07T10:23:02.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.891+0000 7fd88fdff700 1 Processor -- start 2023-12-07T10:23:02.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.891+0000 7fd88fdff700 1 -- start start 2023-12-07T10:23:02.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.892+0000 7fd88fdff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd888100f40 0x7fd88819f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.892+0000 7fd88fdff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd888101860 0x7fd88819fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.892+0000 7fd88fdff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd88810dfb0 0x7fd888199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.892+0000 7fd88fdff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88819a110 con 0x7fd888101860 2023-12-07T10:23:02.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.892+0000 7fd88fdff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88819a280 con 0x7fd888100f40 2023-12-07T10:23:02.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.892+0000 7fd88fdff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88819a440 con 0x7fd88810dfb0 2023-12-07T10:23:02.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.892+0000 7fd88e39c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd88810dfb0 0x7fd888199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.892+0000 7fd88d39a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd888101860 0x7fd88819fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.892+0000 7fd88e39c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd88810dfb0 0x7fd888199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:34908/0 (socket says 172.21.15.119:34908) 2023-12-07T10:23:02.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.892+0000 7fd88e39c700 1 -- 172.21.15.119:0/3986243545 learned_addr learned my addr 172.21.15.119:0/3986243545 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:02.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.892+0000 7fd88db9b700 1 --2- 172.21.15.119:0/3986243545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd888100f40 0x7fd88819f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.892+0000 7fd88e39c700 1 -- 172.21.15.119:0/3986243545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd888100f40 msgr2=0x7fd88819f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.892+0000 7fd88e39c700 1 --2- 172.21.15.119:0/3986243545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd888100f40 0x7fd88819f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.892+0000 7fd88e39c700 1 -- 172.21.15.119:0/3986243545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd888101860 msgr2=0x7fd88819fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.892+0000 7fd88e39c700 1 --2- 172.21.15.119:0/3986243545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd888101860 0x7fd88819fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.892+0000 7fd88e39c700 1 -- 172.21.15.119:0/3986243545 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8780095f0 con 0x7fd88810dfb0 2023-12-07T10:23:02.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.892+0000 7fd88d39a700 1 --2- 172.21.15.119:0/3986243545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd888101860 0x7fd88819fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:02.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.893+0000 7fd88e39c700 1 --2- 172.21.15.119:0/3986243545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd88810dfb0 0x7fd888199ab0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd88000efc0 tx=0x7fd88000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:02.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.893+0000 7fd87effd700 1 -- 172.21.15.119:0/3986243545 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880018070 con 0x7fd88810dfb0 2023-12-07T10:23:02.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.893+0000 7fd87effd700 1 -- 172.21.15.119:0/3986243545 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd88000f040 con 0x7fd88810dfb0 2023-12-07T10:23:02.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.893+0000 7fd88fdff700 1 -- 172.21.15.119:0/3986243545 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd884009520 con 0x7fd88810dfb0 2023-12-07T10:23:02.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.893+0000 7fd87effd700 1 -- 172.21.15.119:0/3986243545 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88000cd00 con 0x7fd88810dfb0 2023-12-07T10:23:02.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.893+0000 7fd88fdff700 1 -- 172.21.15.119:0/3986243545 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8881a6790 con 0x7fd88810dfb0 2023-12-07T10:23:02.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.897+0000 7fd87effd700 1 -- 172.21.15.119:0/3986243545 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fd880003a00 con 0x7fd88810dfb0 2023-12-07T10:23:02.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.898+0000 7fd8a3820b80 1 Processor -- start 2023-12-07T10:23:02.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.898+0000 7fd8a3820b80 1 -- start start 2023-12-07T10:23:02.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.898+0000 7fd8a3820b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24a2e5b40 0x55e24a2e5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.898+0000 7fd8a3820b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e24a2e6460 0x55e24a2e9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.898+0000 7fd8a3820b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e24a2ea150 0x55e24a2ea570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.898+0000 7fd8a3820b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24a2597c0 con 0x55e24a2e5b40 2023-12-07T10:23:02.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.898+0000 7fd8a3820b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24a2407a0 con 0x55e24a2ea150 2023-12-07T10:23:02.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.898+0000 7fd8a3820b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24a23e0c0 con 0x55e24a2e6460 2023-12-07T10:23:02.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.899+0000 7fd87effd700 1 --2- 172.21.15.119:0/3986243545 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd8740a1c80 0x7fd8740a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.899+0000 7fd88db9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e24a2e6460 0x55e24a2e9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.899+0000 7fd88db9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e24a2e6460 0x55e24a2e9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:34922/0 (socket says 172.21.15.119:34922) 2023-12-07T10:23:02.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.899+0000 7fd88db9b700 1 -- 172.21.15.119:0/115567156 learned_addr learned my addr 172.21.15.119:0/115567156 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:02.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.899+0000 7fd88d39a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24a2e5b40 0x55e24a2e5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.899+0000 7fd87effd700 1 -- 172.21.15.119:0/3986243545 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fd8800cd800 con 0x7fd88810dfb0 2023-12-07T10:23:02.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.899+0000 7fd88d39a700 1 --2- 172.21.15.119:0/115567156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e24a2ea150 0x55e24a2ea570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.899+0000 7fd88e39c700 1 --2- 172.21.15.119:0/3986243545 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd8740a1c80 0x7fd8740a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.899+0000 7fd88db9b700 1 -- 172.21.15.119:0/115567156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e24a2ea150 msgr2=0x55e24a2ea570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.900+0000 7fd88db9b700 1 --2- 172.21.15.119:0/115567156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e24a2ea150 0x55e24a2ea570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.900+0000 7fd88db9b700 1 -- 172.21.15.119:0/115567156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24a2e5b40 msgr2=0x55e24a2e5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.900+0000 7fd88db9b700 1 --2- 172.21.15.119:0/115567156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24a2e5b40 0x55e24a2e5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.900+0000 7fd88e39c700 1 --2- 172.21.15.119:0/3986243545 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd8740a1c80 0x7fd8740a4140 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd8800cac00 tx=0x7fd880009040 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:02.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.900+0000 7fd88db9b700 1 -- 172.21.15.119:0/115567156 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e24a0b6630 con 0x55e24a2e6460 2023-12-07T10:23:02.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.900+0000 7fd88d39a700 1 --2- 172.21.15.119:0/115567156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e24a2ea150 0x55e24a2ea570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:02.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.900+0000 7fd88db9b700 1 --2- 172.21.15.119:0/115567156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e24a2e6460 0x55e24a2e9c10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd8840098f0 tx=0x7fd884005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1213ad234b8402d server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:02.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.900+0000 7fd873fff700 1 -- 172.21.15.119:0/115567156 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88400fdd0 con 0x55e24a2e6460 2023-12-07T10:23:02.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.900+0000 7fd873fff700 1 -- 172.21.15.119:0/115567156 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd88401d040 con 0x55e24a2e6460 2023-12-07T10:23:02.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.900+0000 7fd873fff700 1 -- 172.21.15.119:0/115567156 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd884005040 con 0x55e24a2e6460 2023-12-07T10:23:02.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.901+0000 7fd8a3820b80 1 -- 172.21.15.119:0/115567156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e24a2e6460 msgr2=0x55e24a2e9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.901+0000 7fd8a3820b80 1 --2- 172.21.15.119:0/115567156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e24a2e6460 0x55e24a2e9c10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd8840098f0 tx=0x7fd884005dc0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.901+0000 7fd8a3820b80 1 -- 172.21.15.119:0/115567156 shutdown_connections 2023-12-07T10:23:02.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.901+0000 7fd8a3820b80 1 --2- 172.21.15.119:0/115567156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e24a2ea150 0x55e24a2ea570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.901+0000 7fd8a3820b80 1 --2- 172.21.15.119:0/115567156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24a2e5b40 0x55e24a2e5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.901+0000 7fd8a3820b80 1 --2- 172.21.15.119:0/115567156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e24a2e6460 0x55e24a2e9c10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.901+0000 7fd8a3820b80 1 -- 172.21.15.119:0/115567156 >> 172.21.15.119:0/115567156 conn(0x55e24a214360 msgr2=0x55e24a2e0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:02.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.901+0000 7fd8a3820b80 1 -- 172.21.15.119:0/115567156 shutdown_connections 2023-12-07T10:23:02.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.901+0000 7fd8a3820b80 1 -- 172.21.15.119:0/115567156 wait complete. 2023-12-07T10:23:02.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.902+0000 7fd8a3820b80 1 -- start start 2023-12-07T10:23:02.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.902+0000 7fd8a3820b80 1 Processor -- start 2023-12-07T10:23:02.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.902+0000 7fd8a3820b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24a2e5b40 0x55e24a2f1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.902+0000 7fd8a3820b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e24a2e6460 0x55e24a2eec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.902+0000 7fd8a3820b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e24a2ea150 0x55e24a2ef3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.903+0000 7fd8a3820b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24a2399b0 con 0x55e24a2e5b40 2023-12-07T10:23:02.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.903+0000 7fd8a3820b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24a1f3560 con 0x55e24a2e6460 2023-12-07T10:23:02.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.903+0000 7fd8a3820b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24a1f3200 con 0x55e24a2ea150 2023-12-07T10:23:02.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.903+0000 7fd8a3820b80 10 client.0 _ll_register_callbacks cb 0x55e24a1da940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:02.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.903+0000 7fd88db9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e24a2e6460 0x55e24a2eec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.903+0000 7fd88db9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e24a2e6460 0x55e24a2eec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:58726/0 (socket says 172.21.15.119:58726) 2023-12-07T10:23:02.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.903+0000 7fd88db9b700 1 -- 172.21.15.119:0/517753525 learned_addr learned my addr 172.21.15.119:0/517753525 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:02.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.903+0000 7fd88d39a700 1 --2- 172.21.15.119:0/517753525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24a2e5b40 0x55e24a2f1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.903+0000 7fd88d39a700 1 --2- 172.21.15.119:0/517753525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e24a2ea150 0x55e24a2ef3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.903+0000 7fd88db9b700 1 -- 172.21.15.119:0/517753525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e24a2ea150 msgr2=0x55e24a2ef3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.903+0000 7fd88db9b700 1 --2- 172.21.15.119:0/517753525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e24a2ea150 0x55e24a2ef3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.903+0000 7fd88db9b700 1 -- 172.21.15.119:0/517753525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24a2e5b40 msgr2=0x55e24a2f1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.903+0000 7fd88db9b700 1 --2- 172.21.15.119:0/517753525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24a2e5b40 0x55e24a2f1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.903+0000 7fd88db9b700 1 -- 172.21.15.119:0/517753525 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd878013c20 con 0x55e24a2e6460 2023-12-07T10:23:02.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.903+0000 7fd88d39a700 1 --2- 172.21.15.119:0/517753525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24a2e5b40 0x55e24a2f1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:23:02.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.903+0000 7fd88db9b700 1 --2- 172.21.15.119:0/517753525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e24a2e6460 0x55e24a2eec90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd884009c40 tx=0x7fd88400b5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9581411c5bb0cca3 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:02.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.903+0000 7fd8717fa700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:23:02.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.904+0000 7fd8717fa700 1 -- 172.21.15.119:0/517753525 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88400f660 con 0x55e24a2e6460 2023-12-07T10:23:02.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.904+0000 7fd8a3820b80 1 -- 172.21.15.119:0/517753525 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e24a2d8b00 con 0x55e24a2e6460 2023-12-07T10:23:02.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.904+0000 7fd8717fa700 1 -- 172.21.15.119:0/517753525 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fd8840058e0 con 0x55e24a2e6460 2023-12-07T10:23:02.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.904+0000 7fd8717fa700 1 -- 172.21.15.119:0/517753525 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd88400f7c0 con 0x55e24a2e6460 2023-12-07T10:23:02.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.904+0000 7fd8a3820b80 10 client.26564 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:02.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.904+0000 7fd8a3820b80 1 -- 172.21.15.119:0/517753525 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7fd888110ea0 con 0x55e24a2e6460 2023-12-07T10:23:02.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.904+0000 7fd8717fa700 1 -- 172.21.15.119:0/517753525 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e24a2d8b00 con 0x55e24a2e6460 2023-12-07T10:23:02.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.904+0000 7fd8717fa700 1 -- 172.21.15.119:0/517753525 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fd884015020 con 0x55e24a2e6460 2023-12-07T10:23:02.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.905+0000 7fd8a3820b80 10 client.26564 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:02.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.905+0000 7fd8a3820b80 10 client.26564 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2023-12-07T10:23:02.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.905+0000 7fd8a3820b80 20 client.26564 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:02.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.905+0000 7fd8a3820b80 1 --2- 172.21.15.119:0/517753525 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55e24a2fef80 0x55e24a3013a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.905+0000 7fd8a3820b80 4 client.26564 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:23:02.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.905+0000 7fd8a3820b80 1 -- 172.21.15.119:0/517753525 --> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e24a2b47a0 con 0x55e24a2fef80 2023-12-07T10:23:02.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.905+0000 7fd88d39a700 1 --2- 172.21.15.119:0/517753525 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55e24a2fef80 0x55e24a3013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.905+0000 7fd88d39a700 1 --2- 172.21.15.119:0/517753525 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55e24a2fef80 0x55e24a3013a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b5672c803b9d318 server_cookie=86afa0bece084843 in_seq=0 out_seq=0 2023-12-07T10:23:02.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.905+0000 7fd8717fa700 10 client.26564 ms_handle_connect on v2:172.21.15.119:6838/1920762602 2023-12-07T10:23:02.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.906+0000 7fd8717fa700 1 -- 172.21.15.119:0/517753525 <== mds.1 v2:172.21.15.119:6838/1920762602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e24a2b47a0 con 0x55e24a2fef80 2023-12-07T10:23:02.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.906+0000 7fd8717fa700 10 client.26564 handle_command_reply: tid=0 2023-12-07T10:23:02.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.906+0000 7fd8a3820b80 1 client.26564 shutdown 2023-12-07T10:23:02.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.906+0000 7fd8a3820b80 1 -- 172.21.15.119:0/517753525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e24a2e6460 msgr2=0x55e24a2eec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.906+0000 7fd8a3820b80 1 --2- 172.21.15.119:0/517753525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e24a2e6460 0x55e24a2eec90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd884009c40 tx=0x7fd88400b5d0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.906+0000 7fd8a3820b80 1 -- 172.21.15.119:0/517753525 shutdown_connections 2023-12-07T10:23:02.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.906+0000 7fd8a3820b80 1 --2- 172.21.15.119:0/517753525 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55e24a2fef80 0x55e24a3013a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.906+0000 7fd8a3820b80 1 --2- 172.21.15.119:0/517753525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e24a2ea150 0x55e24a2ef3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.906+0000 7fd8a3820b80 1 --2- 172.21.15.119:0/517753525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24a2e5b40 0x55e24a2f1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.906+0000 7fd8717fa700 0 client.26564 ms_handle_reset on v2:172.21.15.119:6838/1920762602 2023-12-07T10:23:02.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.906+0000 7fd8a3820b80 1 --2- 172.21.15.119:0/517753525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e24a2e6460 0x55e24a2eec90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.906+0000 7fd8a3820b80 1 -- 172.21.15.119:0/517753525 >> 172.21.15.119:0/517753525 conn(0x55e24a214360 msgr2=0x55e24a2dad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:02.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.907+0000 7fd8a3820b80 1 -- 172.21.15.119:0/517753525 shutdown_connections 2023-12-07T10:23:02.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.907+0000 7fd8a3820b80 1 -- 172.21.15.119:0/517753525 wait complete. 2023-12-07T10:23:02.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.907+0000 7fd8a3820b80 20 client.26564 trim_cache size 0 max 16384 2023-12-07T10:23:02.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.923+0000 7fd8a3820b80 1 Processor -- start 2023-12-07T10:23:02.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.923+0000 7fd8a3820b80 1 -- start start 2023-12-07T10:23:02.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.923+0000 7fd8a3820b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e24a2e5b40 0x55e24a2ee590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.923+0000 7fd8a3820b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e24a2e6460 0x55e24a2e1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.923+0000 7fd8a3820b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24a2ea150 0x55e24a2f1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.923+0000 7fd8a3820b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24a0b8920 con 0x55e24a2ea150 2023-12-07T10:23:02.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.923+0000 7fd8a3820b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24a2f0230 con 0x55e24a2e6460 2023-12-07T10:23:02.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.923+0000 7fd8a3820b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24a306570 con 0x55e24a2e5b40 2023-12-07T10:23:02.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.923+0000 7fd88d39a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e24a2e5b40 0x55e24a2ee590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.923+0000 7fd88d39a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e24a2e5b40 0x55e24a2ee590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:34934/0 (socket says 172.21.15.119:34934) 2023-12-07T10:23:02.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.923+0000 7fd88d39a700 1 -- 172.21.15.119:0/3358725735 learned_addr learned my addr 172.21.15.119:0/3358725735 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:02.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.923+0000 7fd88db9b700 1 --2- 172.21.15.119:0/3358725735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e24a2e6460 0x55e24a2e1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.923+0000 7fd88d39a700 1 --2- 172.21.15.119:0/3358725735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24a2ea150 0x55e24a2f1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.924+0000 7fd88d39a700 1 -- 172.21.15.119:0/3358725735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e24a2e6460 msgr2=0x55e24a2e1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.924+0000 7fd88d39a700 1 --2- 172.21.15.119:0/3358725735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e24a2e6460 0x55e24a2e1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.924+0000 7fd88d39a700 1 -- 172.21.15.119:0/3358725735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24a2ea150 msgr2=0x55e24a2f1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.924+0000 7fd88d39a700 1 --2- 172.21.15.119:0/3358725735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24a2ea150 0x55e24a2f1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.924+0000 7fd88d39a700 1 -- 172.21.15.119:0/3358725735 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e24a062de0 con 0x55e24a2e5b40 2023-12-07T10:23:02.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.924+0000 7fd88d39a700 1 --2- 172.21.15.119:0/3358725735 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e24a2e5b40 0x55e24a2ee590 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd878004790 tx=0x7fd878024040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=489b3425688d946a server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:02.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.924+0000 7fd87cbff700 1 -- 172.21.15.119:0/3358725735 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878027050 con 0x55e24a2e5b40 2023-12-07T10:23:02.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.924+0000 7fd87cbff700 1 -- 172.21.15.119:0/3358725735 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd878029280 con 0x55e24a2e5b40 2023-12-07T10:23:02.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.924+0000 7fd87cbff700 1 -- 172.21.15.119:0/3358725735 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878005280 con 0x55e24a2e5b40 2023-12-07T10:23:02.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.925+0000 7fd8a3820b80 1 -- 172.21.15.119:0/3358725735 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e24a2e5b40 msgr2=0x55e24a2ee590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.925+0000 7fd8a3820b80 1 --2- 172.21.15.119:0/3358725735 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e24a2e5b40 0x55e24a2ee590 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd878004790 tx=0x7fd878024040 comp rx=0 tx=0).stop 2023-12-07T10:23:02.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.925+0000 7fd8a3820b80 1 -- 172.21.15.119:0/3358725735 shutdown_connections 2023-12-07T10:23:02.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.925+0000 7fd8a3820b80 1 --2- 172.21.15.119:0/3358725735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24a2ea150 0x55e24a2f1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.925+0000 7fd8a3820b80 1 --2- 172.21.15.119:0/3358725735 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e24a2e5b40 0x55e24a2ee590 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.925+0000 7fd8a3820b80 1 --2- 172.21.15.119:0/3358725735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e24a2e6460 0x55e24a2e1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.925+0000 7fd8a3820b80 1 -- 172.21.15.119:0/3358725735 >> 172.21.15.119:0/3358725735 conn(0x55e24a214360 msgr2=0x55e24a2ea530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:02.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.925+0000 7fd8a3820b80 1 -- 172.21.15.119:0/3358725735 shutdown_connections 2023-12-07T10:23:02.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.925+0000 7fd8a3820b80 1 -- 172.21.15.119:0/3358725735 wait complete. 2023-12-07T10:23:02.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.926+0000 7fd8a3820b80 1 -- start start 2023-12-07T10:23:02.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.926+0000 7fd8a3820b80 1 Processor -- start 2023-12-07T10:23:02.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.926+0000 7fd8a3820b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e24a2e5b40 0x55e24a2f42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.926+0000 7fd8a3820b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e24a2e6460 0x55e24a2f47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.927+0000 7fd8a3820b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24a2ea150 0x55e24a354440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.927+0000 7fd8a3820b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24a2f50d0 con 0x55e24a2ea150 2023-12-07T10:23:02.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.927+0000 7fd8a3820b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24a354980 con 0x55e24a2e6460 2023-12-07T10:23:02.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.927+0000 7fd8a3820b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24a354ac0 con 0x55e24a2e5b40 2023-12-07T10:23:02.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.927+0000 7fd8a3820b80 10 client.0 _ll_register_callbacks cb 0x55e24a1da940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:02.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.927+0000 7fd88db9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e24a2e6460 0x55e24a2f47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.927+0000 7fd88db9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e24a2e6460 0x55e24a2f47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:58748/0 (socket says 172.21.15.119:58748) 2023-12-07T10:23:02.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.927+0000 7fd88db9b700 1 -- 172.21.15.119:0/246063439 learned_addr learned my addr 172.21.15.119:0/246063439 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:02.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.927+0000 7fd88d39a700 1 --2- 172.21.15.119:0/246063439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e24a2e5b40 0x55e24a2f42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.927+0000 7fd88d39a700 1 --2- 172.21.15.119:0/246063439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24a2ea150 0x55e24a354440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.927+0000 7fd88db9b700 1 -- 172.21.15.119:0/246063439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e24a2e5b40 msgr2=0x55e24a2f42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.927+0000 7fd88db9b700 1 --2- 172.21.15.119:0/246063439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e24a2e5b40 0x55e24a2f42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.927+0000 7fd88db9b700 1 -- 172.21.15.119:0/246063439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24a2ea150 msgr2=0x55e24a354440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:02.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.927+0000 7fd88db9b700 1 --2- 172.21.15.119:0/246063439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24a2ea150 0x55e24a354440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.927+0000 7fd88db9b700 1 -- 172.21.15.119:0/246063439 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8780132d0 con 0x55e24a2e6460 2023-12-07T10:23:02.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.927+0000 7fd88d39a700 1 --2- 172.21.15.119:0/246063439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e24a2e5b40 0x55e24a2f42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:23:02.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.928+0000 7fd88db9b700 1 --2- 172.21.15.119:0/246063439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e24a2e6460 0x55e24a2f47e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55e24a2f1e70 tx=0x7fd884009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12b7cdb6f76670c0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:02.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.928+0000 7fd8717fa700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:23:02.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.928+0000 7fd8717fa700 1 -- 172.21.15.119:0/246063439 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88400f870 con 0x55e24a2e6460 2023-12-07T10:23:02.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.928+0000 7fd8717fa700 1 -- 172.21.15.119:0/246063439 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fd88400b640 con 0x55e24a2e6460 2023-12-07T10:23:02.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.928+0000 7fd8a3820b80 1 -- 172.21.15.119:0/246063439 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd88401d040 con 0x55e24a2e6460 2023-12-07T10:23:02.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.928+0000 7fd8717fa700 1 -- 172.21.15.119:0/246063439 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd88400b8c0 con 0x55e24a2e6460 2023-12-07T10:23:02.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.928+0000 7fd8a3820b80 10 client.26570 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:02.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.928+0000 7fd8a3820b80 1 -- 172.21.15.119:0/246063439 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55e24a04bc60 con 0x55e24a2e6460 2023-12-07T10:23:02.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.928+0000 7fd8717fa700 1 -- 172.21.15.119:0/246063439 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd88401d040 con 0x55e24a2e6460 2023-12-07T10:23:02.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.929+0000 7fd8717fa700 1 -- 172.21.15.119:0/246063439 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fd884015070 con 0x55e24a2e6460 2023-12-07T10:23:02.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.929+0000 7fd8a3820b80 10 client.26570 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:02.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.929+0000 7fd8a3820b80 10 client.26570 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2023-12-07T10:23:02.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.929+0000 7fd8a3820b80 20 client.26570 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:02.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.929+0000 7fd8a3820b80 1 --2- 172.21.15.119:0/246063439 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55e24a355e10 0x55e24a358230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:02.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.929+0000 7fd8a3820b80 4 client.26570 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:23:02.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.929+0000 7fd8a3820b80 1 -- 172.21.15.119:0/246063439 --> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e24a1fffd0 con 0x55e24a355e10 2023-12-07T10:23:02.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.929+0000 7fd88d39a700 1 --2- 172.21.15.119:0/246063439 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55e24a355e10 0x55e24a358230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:02.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.929+0000 7fd88d39a700 1 --2- 172.21.15.119:0/246063439 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55e24a355e10 0x55e24a358230 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a1b2d684c17a82da server_cookie=fb5499f4f140b3c2 in_seq=0 out_seq=0 2023-12-07T10:23:02.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.929+0000 7fd8717fa700 10 client.26570 ms_handle_connect on v2:172.21.15.119:6838/1920762602 2023-12-07T10:23:02.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.930+0000 7fd8717fa700 1 -- 172.21.15.119:0/246063439 <== mds.1 v2:172.21.15.119:6838/1920762602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e24a1fffd0 con 0x55e24a355e10 2023-12-07T10:23:02.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.930+0000 7fd8717fa700 10 client.26570 handle_command_reply: tid=0 2023-12-07T10:23:02.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.930+0000 7fd8a3820b80 1 client.26570 shutdown 2023-12-07T10:23:02.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.930+0000 7fd8a3820b80 1 -- 172.21.15.119:0/246063439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e24a2e6460 msgr2=0x55e24a2f47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.930+0000 7fd8a3820b80 1 --2- 172.21.15.119:0/246063439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e24a2e6460 0x55e24a2f47e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55e24a2f1e70 tx=0x7fd884009f50 comp rx=0 tx=0).stop 2023-12-07T10:23:02.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.930+0000 7fd8a3820b80 1 -- 172.21.15.119:0/246063439 shutdown_connections 2023-12-07T10:23:02.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.930+0000 7fd8a3820b80 1 --2- 172.21.15.119:0/246063439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e24a2ea150 0x55e24a354440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.930+0000 7fd8a3820b80 1 --2- 172.21.15.119:0/246063439 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55e24a355e10 0x55e24a358230 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.930+0000 7fd8a3820b80 1 --2- 172.21.15.119:0/246063439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e24a2e5b40 0x55e24a2f42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.930+0000 7fd8a3820b80 1 --2- 172.21.15.119:0/246063439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e24a2e6460 0x55e24a2f47e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.930+0000 7fd8717fa700 0 client.26570 ms_handle_reset on v2:172.21.15.119:6838/1920762602 2023-12-07T10:23:02.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.930+0000 7fd8a3820b80 1 -- 172.21.15.119:0/246063439 >> 172.21.15.119:0/246063439 conn(0x55e24a214360 msgr2=0x55e24a2e0000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:02.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.930+0000 7fd8a3820b80 1 -- 172.21.15.119:0/246063439 shutdown_connections 2023-12-07T10:23:02.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.930+0000 7fd8a3820b80 1 -- 172.21.15.119:0/246063439 wait complete. 2023-12-07T10:23:02.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.930+0000 7fd8a3820b80 20 client.26570 trim_cache size 0 max 16384 2023-12-07T10:23:02.936 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:23:02.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.931+0000 7fd88fdff700 1 -- 172.21.15.119:0/3986243545 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd8740a1c80 msgr2=0x7fd8740a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.931+0000 7fd88fdff700 1 --2- 172.21.15.119:0/3986243545 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd8740a1c80 0x7fd8740a4140 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd8800cac00 tx=0x7fd880009040 comp rx=0 tx=0).stop 2023-12-07T10:23:02.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.931+0000 7fd88fdff700 1 -- 172.21.15.119:0/3986243545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd88810dfb0 msgr2=0x7fd888199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:02.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.931+0000 7fd88fdff700 1 --2- 172.21.15.119:0/3986243545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd88810dfb0 0x7fd888199ab0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd88000efc0 tx=0x7fd88000c3a0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.931+0000 7fd88fdff700 1 -- 172.21.15.119:0/3986243545 shutdown_connections 2023-12-07T10:23:02.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.931+0000 7fd88fdff700 1 --2- 172.21.15.119:0/3986243545 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd8740a1c80 0x7fd8740a4140 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.931+0000 7fd88fdff700 1 --2- 172.21.15.119:0/3986243545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd88810dfb0 0x7fd888199ab0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.931+0000 7fd88fdff700 1 --2- 172.21.15.119:0/3986243545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd888100f40 0x7fd88819f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.931+0000 7fd88fdff700 1 --2- 172.21.15.119:0/3986243545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd888101860 0x7fd88819fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:02.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.931+0000 7fd88fdff700 1 -- 172.21.15.119:0/3986243545 >> 172.21.15.119:0/3986243545 conn(0x7fd888077610 msgr2=0x7fd88810f690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:02.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.931+0000 7fd88fdff700 1 -- 172.21.15.119:0/3986243545 shutdown_connections 2023-12-07T10:23:02.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:02.931+0000 7fd88fdff700 1 -- 172.21.15.119:0/3986243545 wait complete. 2023-12-07T10:23:02.951 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-07T10:23:03.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.185+0000 7f8ff41ff700 1 -- 172.21.15.119:0/41864298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fec1053d0 msgr2=0x7f8fec1057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:03.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.185+0000 7f8ff41ff700 1 --2- 172.21.15.119:0/41864298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fec1053d0 0x7f8fec1057b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009990 tx=0x7f8fe8009ca0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.186+0000 7f8ff41ff700 1 -- 172.21.15.119:0/41864298 shutdown_connections 2023-12-07T10:23:03.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.186+0000 7f8ff41ff700 1 --2- 172.21.15.119:0/41864298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8fec100e20 0x7f8fec10d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.186+0000 7f8ff41ff700 1 --2- 172.21.15.119:0/41864298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fec1053d0 0x7f8fec1057b0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.186+0000 7f8ff41ff700 1 --2- 172.21.15.119:0/41864298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fec100460 0x7f8fec1008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.186+0000 7f8ff41ff700 1 -- 172.21.15.119:0/41864298 >> 172.21.15.119:0/41864298 conn(0x7f8fec077610 msgr2=0x7f8fec077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:03.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.186+0000 7f8ff41ff700 1 -- 172.21.15.119:0/41864298 shutdown_connections 2023-12-07T10:23:03.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.186+0000 7f8ff41ff700 1 -- 172.21.15.119:0/41864298 wait complete. 2023-12-07T10:23:03.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.187+0000 7f8ff41ff700 1 Processor -- start 2023-12-07T10:23:03.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.187+0000 7f8ff41ff700 1 -- start start 2023-12-07T10:23:03.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.187+0000 7f8ff41ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fec100460 0x7f8fec10fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:03.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.187+0000 7f8ff41ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fec100e20 0x7f8fec110120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:03.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.187+0000 7f8ff41ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8fec1053d0 0x7f8fec110660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:03.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.187+0000 7f8ff41ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec1a5da0 con 0x7f8fec100460 2023-12-07T10:23:03.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.187+0000 7f8ff41ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec1a5f10 con 0x7f8fec1053d0 2023-12-07T10:23:03.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.187+0000 7f8ff41ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec1a60d0 con 0x7f8fec100e20 2023-12-07T10:23:03.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.188+0000 7f8ff179a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fec100e20 0x7f8fec110120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:03.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.188+0000 7f8ff1f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fec100460 0x7f8fec10fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:03.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.188+0000 7f8ff279c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8fec1053d0 0x7f8fec110660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:03.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.188+0000 7f8ff179a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fec100e20 0x7f8fec110120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:34970/0 (socket says 172.21.15.119:34970) 2023-12-07T10:23:03.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.188+0000 7f8ff179a700 1 -- 172.21.15.119:0/3962379964 learned_addr learned my addr 172.21.15.119:0/3962379964 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:03.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.188+0000 7f8ff279c700 1 -- 172.21.15.119:0/3962379964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fec100e20 msgr2=0x7f8fec110120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:03.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.188+0000 7f8ff279c700 1 --2- 172.21.15.119:0/3962379964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fec100e20 0x7f8fec110120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.188+0000 7f8ff279c700 1 -- 172.21.15.119:0/3962379964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fec100460 msgr2=0x7f8fec10fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:03.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.188+0000 7f8ff279c700 1 --2- 172.21.15.119:0/3962379964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fec100460 0x7f8fec10fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.188+0000 7f8ff279c700 1 -- 172.21.15.119:0/3962379964 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe80095f0 con 0x7f8fec1053d0 2023-12-07T10:23:03.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.188+0000 7f8ff279c700 1 --2- 172.21.15.119:0/3962379964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8fec1053d0 0x7f8fec110660 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400efc0 tx=0x7f8fe400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:03.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.188+0000 7f8fe2ffd700 1 -- 172.21.15.119:0/3962379964 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4018070 con 0x7f8fec1053d0 2023-12-07T10:23:03.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.188+0000 7f8ff41ff700 1 -- 172.21.15.119:0/3962379964 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fec1a6350 con 0x7f8fec1053d0 2023-12-07T10:23:03.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.188+0000 7f8fe2ffd700 1 -- 172.21.15.119:0/3962379964 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fe400f040 con 0x7f8fec1053d0 2023-12-07T10:23:03.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.188+0000 7f8fe2ffd700 1 -- 172.21.15.119:0/3962379964 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe400cc80 con 0x7f8fec1053d0 2023-12-07T10:23:03.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.189+0000 7f8ff41ff700 1 -- 172.21.15.119:0/3962379964 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fec1a6810 con 0x7f8fec1053d0 2023-12-07T10:23:03.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.193+0000 7f8fe2ffd700 1 -- 172.21.15.119:0/3962379964 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f8fe4003a00 con 0x7f8fec1053d0 2023-12-07T10:23:03.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.194+0000 7f9007bc8b80 1 Processor -- start 2023-12-07T10:23:03.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.194+0000 7f8fe2ffd700 1 --2- 172.21.15.119:0/3962379964 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8fd80a1c80 0x7f8fd80a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:03.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.195+0000 7f9007bc8b80 1 -- start start 2023-12-07T10:23:03.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.195+0000 7f8fe2ffd700 1 -- 172.21.15.119:0/3962379964 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f8fe40cd820 con 0x7f8fec1053d0 2023-12-07T10:23:03.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.195+0000 7f8ff1f9b700 1 --2- 172.21.15.119:0/3962379964 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8fd80a1c80 0x7f8fd80a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:03.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.195+0000 7f9007bc8b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf63861b40 0x55bf63861f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:03.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.195+0000 7f9007bc8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf63862460 0x55bf63865c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:03.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.195+0000 7f9007bc8b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf63866150 0x55bf63866570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:03.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.195+0000 7f9007bc8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf637d57c0 con 0x55bf63862460 2023-12-07T10:23:03.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.195+0000 7f9007bc8b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf637bc7a0 con 0x55bf63866150 2023-12-07T10:23:03.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.195+0000 7f9007bc8b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf637ba0c0 con 0x55bf63861b40 2023-12-07T10:23:03.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.195+0000 7f8ff1f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf63861b40 0x55bf63861f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:03.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.195+0000 7f8ff1f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf63861b40 0x55bf63861f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:34972/0 (socket says 172.21.15.119:34972) 2023-12-07T10:23:03.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.195+0000 7f8ff1f9b700 1 -- 172.21.15.119:0/1973296497 learned_addr learned my addr 172.21.15.119:0/1973296497 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:03.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.195+0000 7f8ff179a700 1 --2- 172.21.15.119:0/1973296497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf63862460 0x55bf63865c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:03.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.196+0000 7f8ff279c700 1 --2- 172.21.15.119:0/1973296497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf63866150 0x55bf63866570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:03.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.196+0000 7f8ff279c700 1 -- 172.21.15.119:0/1973296497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf63861b40 msgr2=0x55bf63861f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:03.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.196+0000 7f8ff279c700 1 --2- 172.21.15.119:0/1973296497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf63861b40 0x55bf63861f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.196+0000 7f8ff279c700 1 -- 172.21.15.119:0/1973296497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf63862460 msgr2=0x55bf63865c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:03.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.196+0000 7f8ff279c700 1 --2- 172.21.15.119:0/1973296497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf63862460 0x55bf63865c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.196+0000 7f8ff279c700 1 -- 172.21.15.119:0/1973296497 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf63632630 con 0x55bf63866150 2023-12-07T10:23:03.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.196+0000 7f8ff1f9b700 1 --2- 172.21.15.119:0/3962379964 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8fd80a1c80 0x7f8fd80a4140 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8015000 tx=0x7f8fe8005320 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:03.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.196+0000 7f8ff279c700 1 --2- 172.21.15.119:0/1973296497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf63866150 0x55bf63866570 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40cac20 tx=0x7f8fe40966d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=436d255457a8120a server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:03.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.196+0000 7f8fd7fff700 1 -- 172.21.15.119:0/1973296497 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4099bc0 con 0x55bf63866150 2023-12-07T10:23:03.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.196+0000 7f8fd7fff700 1 -- 172.21.15.119:0/1973296497 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fe409f0f0 con 0x55bf63866150 2023-12-07T10:23:03.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.196+0000 7f8fd7fff700 1 -- 172.21.15.119:0/1973296497 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe409e440 con 0x55bf63866150 2023-12-07T10:23:03.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.197+0000 7f9007bc8b80 1 -- 172.21.15.119:0/1973296497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf63866150 msgr2=0x55bf63866570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:03.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.197+0000 7f9007bc8b80 1 --2- 172.21.15.119:0/1973296497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf63866150 0x55bf63866570 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40cac20 tx=0x7f8fe40966d0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.197+0000 7f9007bc8b80 1 -- 172.21.15.119:0/1973296497 shutdown_connections 2023-12-07T10:23:03.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.197+0000 7f9007bc8b80 1 --2- 172.21.15.119:0/1973296497 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf63866150 0x55bf63866570 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.197+0000 7f9007bc8b80 1 --2- 172.21.15.119:0/1973296497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf63861b40 0x55bf63861f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.197+0000 7f9007bc8b80 1 --2- 172.21.15.119:0/1973296497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf63862460 0x55bf63865c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.197+0000 7f9007bc8b80 1 -- 172.21.15.119:0/1973296497 >> 172.21.15.119:0/1973296497 conn(0x55bf63790360 msgr2=0x55bf6385c130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:03.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.197+0000 7f9007bc8b80 1 -- 172.21.15.119:0/1973296497 shutdown_connections 2023-12-07T10:23:03.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.197+0000 7f9007bc8b80 1 -- 172.21.15.119:0/1973296497 wait complete. 2023-12-07T10:23:03.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.198+0000 7f9007bc8b80 1 -- start start 2023-12-07T10:23:03.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.198+0000 7f9007bc8b80 1 Processor -- start 2023-12-07T10:23:03.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.199+0000 7f9007bc8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf63861b40 0x55bf6386d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:03.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.199+0000 7f9007bc8b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf63862460 0x55bf6386ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:03.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.199+0000 7f9007bc8b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf63866150 0x55bf6386b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:03.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.199+0000 7f9007bc8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf637b59b0 con 0x55bf63861b40 2023-12-07T10:23:03.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.199+0000 7f9007bc8b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf6376f560 con 0x55bf63866150 2023-12-07T10:23:03.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.199+0000 7f9007bc8b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf6376f200 con 0x55bf63862460 2023-12-07T10:23:03.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.199+0000 7f9007bc8b80 10 client.0 _ll_register_callbacks cb 0x55bf63756940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:03.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.199+0000 7f8ff1f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf63861b40 0x55bf6386d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:03.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.199+0000 7f8ff1f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf63861b40 0x55bf6386d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58888/0 (socket says 172.21.15.119:58888) 2023-12-07T10:23:03.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.199+0000 7f8ff1f9b700 1 -- 172.21.15.119:0/2389672910 learned_addr learned my addr 172.21.15.119:0/2389672910 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:03.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.199+0000 7f8ff179a700 1 --2- 172.21.15.119:0/2389672910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf63862460 0x55bf6386ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:03.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.199+0000 7f8ff279c700 1 --2- 172.21.15.119:0/2389672910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf63866150 0x55bf6386b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:03.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.200+0000 7f8ff1f9b700 1 -- 172.21.15.119:0/2389672910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf63862460 msgr2=0x55bf6386ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:03.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.200+0000 7f8ff1f9b700 1 --2- 172.21.15.119:0/2389672910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf63862460 0x55bf6386ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.200+0000 7f8ff1f9b700 1 -- 172.21.15.119:0/2389672910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf63866150 msgr2=0x55bf6386b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:03.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.200+0000 7f8ff1f9b700 1 --2- 172.21.15.119:0/2389672910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf63866150 0x55bf6386b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.200+0000 7f8ff1f9b700 1 -- 172.21.15.119:0/2389672910 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fec1a6810 con 0x55bf63861b40 2023-12-07T10:23:03.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.200+0000 7f8ff279c700 1 --2- 172.21.15.119:0/2389672910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf63866150 0x55bf6386b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:23:03.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.200+0000 7f8ff1f9b700 1 --2- 172.21.15.119:0/2389672910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf63861b40 0x55bf6386d8b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8000c00 tx=0x7f8fe8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bc793e28881584 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:03.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.200+0000 7f8fd57fa700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:23:03.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.200+0000 7f8fd57fa700 1 -- 172.21.15.119:0/2389672910 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe801d040 con 0x55bf63861b40 2023-12-07T10:23:03.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.200+0000 7f9007bc8b80 1 -- 172.21.15.119:0/2389672910 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf63854b00 con 0x55bf63861b40 2023-12-07T10:23:03.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.200+0000 7f8fd57fa700 1 -- 172.21.15.119:0/2389672910 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f8fe8004850 con 0x55bf63861b40 2023-12-07T10:23:03.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.201+0000 7f8fd57fa700 1 -- 172.21.15.119:0/2389672910 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fe8004a10 con 0x55bf63861b40 2023-12-07T10:23:03.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.201+0000 7f9007bc8b80 10 client.16554 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:03.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.201+0000 7f9007bc8b80 1 -- 172.21.15.119:0/2389672910 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55bf635dede0 con 0x55bf63861b40 2023-12-07T10:23:03.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.201+0000 7f8fd57fa700 1 -- 172.21.15.119:0/2389672910 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bf63854b00 con 0x55bf63861b40 2023-12-07T10:23:03.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.201+0000 7f8fd57fa700 1 -- 172.21.15.119:0/2389672910 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f8fe800fd10 con 0x55bf63861b40 2023-12-07T10:23:03.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.201+0000 7f9007bc8b80 10 client.16554 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:03.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.201+0000 7f9007bc8b80 10 client.16554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:23:03.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.201+0000 7f9007bc8b80 20 client.16554 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:03.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.201+0000 7f9007bc8b80 1 --2- 172.21.15.119:0/2389672910 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55bf6387b180 0x55bf6387d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:03.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.201+0000 7f9007bc8b80 4 client.16554 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:23:03.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.201+0000 7f9007bc8b80 1 -- 172.21.15.119:0/2389672910 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf638307a0 con 0x55bf6387b180 2023-12-07T10:23:03.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.202+0000 7f8ff179a700 1 --2- 172.21.15.119:0/2389672910 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55bf6387b180 0x55bf6387d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:03.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.202+0000 7f8ff179a700 1 --2- 172.21.15.119:0/2389672910 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55bf6387b180 0x55bf6387d5a0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7586e0887de481c1 server_cookie=684352ba16cccaf5 in_seq=0 out_seq=0 2023-12-07T10:23:03.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.202+0000 7f8fd57fa700 10 client.16554 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:03.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.203+0000 7f8fd57fa700 1 -- 172.21.15.119:0/2389672910 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bf638307a0 con 0x55bf6387b180 2023-12-07T10:23:03.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.203+0000 7f8fd57fa700 10 client.16554 handle_command_reply: tid=0 2023-12-07T10:23:03.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.203+0000 7f9007bc8b80 1 client.16554 shutdown 2023-12-07T10:23:03.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.203+0000 7f9007bc8b80 1 -- 172.21.15.119:0/2389672910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf63861b40 msgr2=0x55bf6386d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:03.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.203+0000 7f9007bc8b80 1 --2- 172.21.15.119:0/2389672910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf63861b40 0x55bf6386d8b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8000c00 tx=0x7f8fe8009ca0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.203+0000 7f9007bc8b80 1 -- 172.21.15.119:0/2389672910 shutdown_connections 2023-12-07T10:23:03.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.203+0000 7f9007bc8b80 1 --2- 172.21.15.119:0/2389672910 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55bf6387b180 0x55bf6387d5a0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.203+0000 7f9007bc8b80 1 --2- 172.21.15.119:0/2389672910 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf63866150 0x55bf6386b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.203+0000 7f9007bc8b80 1 --2- 172.21.15.119:0/2389672910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf63861b40 0x55bf6386d8b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.203+0000 7f8fd57fa700 0 client.16554 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:03.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.203+0000 7f9007bc8b80 1 --2- 172.21.15.119:0/2389672910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf63862460 0x55bf6386ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.203+0000 7f9007bc8b80 1 -- 172.21.15.119:0/2389672910 >> 172.21.15.119:0/2389672910 conn(0x55bf63790360 msgr2=0x55bf63867620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:03.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.204+0000 7f9007bc8b80 1 -- 172.21.15.119:0/2389672910 shutdown_connections 2023-12-07T10:23:03.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.204+0000 7f9007bc8b80 1 -- 172.21.15.119:0/2389672910 wait complete. 2023-12-07T10:23:03.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.204+0000 7f9007bc8b80 20 client.16554 trim_cache size 0 max 16384 2023-12-07T10:23:03.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.219+0000 7f9007bc8b80 1 Processor -- start 2023-12-07T10:23:03.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.219+0000 7f9007bc8b80 1 -- start start 2023-12-07T10:23:03.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.220+0000 7f9007bc8b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf63861b40 0x55bf63872200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:03.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.220+0000 7f9007bc8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf63862460 0x55bf63872740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:03.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.220+0000 7f9007bc8b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf63866150 0x55bf63876cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:03.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.220+0000 7f9007bc8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf63634920 con 0x55bf63862460 2023-12-07T10:23:03.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.220+0000 7f9007bc8b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf63856b10 con 0x55bf63866150 2023-12-07T10:23:03.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.220+0000 7f9007bc8b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf638827a0 con 0x55bf63861b40 2023-12-07T10:23:03.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.220+0000 7f8ff279c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf63866150 0x55bf63876cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:03.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.220+0000 7f8ff1f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf63861b40 0x55bf63872200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:03.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.220+0000 7f8ff279c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf63866150 0x55bf63876cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:58782/0 (socket says 172.21.15.119:58782) 2023-12-07T10:23:03.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.220+0000 7f8ff1f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf63861b40 0x55bf63872200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:34978/0 (socket says 172.21.15.119:34978) 2023-12-07T10:23:03.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.220+0000 7f8ff279c700 1 -- 172.21.15.119:0/1792025022 learned_addr learned my addr 172.21.15.119:0/1792025022 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:03.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.220+0000 7f8ff179a700 1 --2- 172.21.15.119:0/1792025022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf63862460 0x55bf63872740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:03.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.220+0000 7f8ff279c700 1 -- 172.21.15.119:0/1792025022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf63861b40 msgr2=0x55bf63872200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:03.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.220+0000 7f8ff279c700 1 --2- 172.21.15.119:0/1792025022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf63861b40 0x55bf63872200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.220+0000 7f8ff279c700 1 -- 172.21.15.119:0/1792025022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf63862460 msgr2=0x55bf63872740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:03.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.220+0000 7f8ff279c700 1 --2- 172.21.15.119:0/1792025022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf63862460 0x55bf63872740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.220+0000 7f8ff279c700 1 -- 172.21.15.119:0/1792025022 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf63856040 con 0x55bf63866150 2023-12-07T10:23:03.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.220+0000 7f8ff1f9b700 1 --2- 172.21.15.119:0/1792025022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf63861b40 0x55bf63872200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:03.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.220+0000 7f8ff179a700 1 --2- 172.21.15.119:0/1792025022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf63862460 0x55bf63872740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:23:03.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.221+0000 7f8ff279c700 1 --2- 172.21.15.119:0/1792025022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf63866150 0x55bf63876cb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40cacd0 tx=0x7f8fe4004d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5d7e3b289eef0d0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:03.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.221+0000 7f8fe0ff9700 1 -- 172.21.15.119:0/1792025022 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4004ad0 con 0x55bf63866150 2023-12-07T10:23:03.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.221+0000 7f8fe0ff9700 1 -- 172.21.15.119:0/1792025022 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fe4013e90 con 0x55bf63866150 2023-12-07T10:23:03.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.221+0000 7f8fe0ff9700 1 -- 172.21.15.119:0/1792025022 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe409e3b0 con 0x55bf63866150 2023-12-07T10:23:03.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.222+0000 7f9007bc8b80 1 -- 172.21.15.119:0/1792025022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf63866150 msgr2=0x55bf63876cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:03.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.222+0000 7f9007bc8b80 1 --2- 172.21.15.119:0/1792025022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf63866150 0x55bf63876cb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40cacd0 tx=0x7f8fe4004d20 comp rx=0 tx=0).stop 2023-12-07T10:23:03.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.222+0000 7f9007bc8b80 1 -- 172.21.15.119:0/1792025022 shutdown_connections 2023-12-07T10:23:03.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.222+0000 7f9007bc8b80 1 --2- 172.21.15.119:0/1792025022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf63866150 0x55bf63876cb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.222+0000 7f9007bc8b80 1 --2- 172.21.15.119:0/1792025022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf63861b40 0x55bf63872200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.222+0000 7f9007bc8b80 1 --2- 172.21.15.119:0/1792025022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf63862460 0x55bf63872740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.222+0000 7f9007bc8b80 1 -- 172.21.15.119:0/1792025022 >> 172.21.15.119:0/1792025022 conn(0x55bf63790360 msgr2=0x55bf6386d570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:03.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.222+0000 7f9007bc8b80 1 -- 172.21.15.119:0/1792025022 shutdown_connections 2023-12-07T10:23:03.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.222+0000 7f9007bc8b80 1 -- 172.21.15.119:0/1792025022 wait complete. 2023-12-07T10:23:03.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.222+0000 7f9007bc8b80 1 -- start start 2023-12-07T10:23:03.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.222+0000 7f9007bc8b80 1 Processor -- start 2023-12-07T10:23:03.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.223+0000 7f9007bc8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf63861b40 0x55bf638701e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:03.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.223+0000 7f9007bc8b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf63862460 0x55bf63870720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:03.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.223+0000 7f9007bc8b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf63866150 0x55bf63870c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:03.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.223+0000 7f9007bc8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf63872ab0 con 0x55bf63861b40 2023-12-07T10:23:03.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.223+0000 7f9007bc8b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf638d0660 con 0x55bf63866150 2023-12-07T10:23:03.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.223+0000 7f9007bc8b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf638d07a0 con 0x55bf63862460 2023-12-07T10:23:03.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.223+0000 7f9007bc8b80 10 client.0 _ll_register_callbacks cb 0x55bf63756940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:03.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.223+0000 7f8ff179a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf63862460 0x55bf63870720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:03.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.223+0000 7f8ff1f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf63861b40 0x55bf638701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:03.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.223+0000 7f8ff179a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf63862460 0x55bf63870720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:34988/0 (socket says 172.21.15.119:34988) 2023-12-07T10:23:03.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.223+0000 7f8ff179a700 1 -- 172.21.15.119:0/659282473 learned_addr learned my addr 172.21.15.119:0/659282473 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:03.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.224+0000 7f8ff279c700 1 --2- 172.21.15.119:0/659282473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf63866150 0x55bf63870c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:03.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.224+0000 7f8ff179a700 1 -- 172.21.15.119:0/659282473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf63866150 msgr2=0x55bf63870c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:03.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.224+0000 7f8ff179a700 1 --2- 172.21.15.119:0/659282473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf63866150 0x55bf63870c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.224+0000 7f8ff179a700 1 -- 172.21.15.119:0/659282473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf63861b40 msgr2=0x55bf638701e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:03.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.224+0000 7f8ff179a700 1 --2- 172.21.15.119:0/659282473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf63861b40 0x55bf638701e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.224+0000 7f8ff179a700 1 -- 172.21.15.119:0/659282473 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf635dede0 con 0x55bf63862460 2023-12-07T10:23:03.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.224+0000 7f8ff279c700 1 --2- 172.21.15.119:0/659282473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf63866150 0x55bf63870c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:23:03.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.224+0000 7f8ff179a700 1 --2- 172.21.15.119:0/659282473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf63862460 0x55bf63870720 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc0096b0 tx=0x7f8fdc0121a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7d7983cf73c0266 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:03.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.224+0000 7f8fd57fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:23:03.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.224+0000 7f8fd57fa700 1 -- 172.21.15.119:0/659282473 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc017030 con 0x55bf63862460 2023-12-07T10:23:03.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.224+0000 7f8fd57fa700 1 -- 172.21.15.119:0/659282473 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f8fdc013c30 con 0x55bf63862460 2023-12-07T10:23:03.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.224+0000 7f9007bc8b80 1 -- 172.21.15.119:0/659282473 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8fe409f0f0 con 0x55bf63862460 2023-12-07T10:23:03.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.225+0000 7f8fd57fa700 1 -- 172.21.15.119:0/659282473 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fdc013df0 con 0x55bf63862460 2023-12-07T10:23:03.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.225+0000 7f9007bc8b80 10 client.26569 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:03.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.225+0000 7f9007bc8b80 1 -- 172.21.15.119:0/659282473 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55bf63632630 con 0x55bf63862460 2023-12-07T10:23:03.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.225+0000 7f8fd57fa700 1 -- 172.21.15.119:0/659282473 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fe409f0f0 con 0x55bf63862460 2023-12-07T10:23:03.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.225+0000 7f8fd57fa700 1 -- 172.21.15.119:0/659282473 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f8fdc010730 con 0x55bf63862460 2023-12-07T10:23:03.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.225+0000 7f9007bc8b80 10 client.26569 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:03.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.225+0000 7f9007bc8b80 10 client.26569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:23:03.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.225+0000 7f9007bc8b80 20 client.26569 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:03.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.225+0000 7f9007bc8b80 1 --2- 172.21.15.119:0/659282473 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55bf638d20d0 0x55bf638d44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:03.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.225+0000 7f9007bc8b80 4 client.26569 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:23:03.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.225+0000 7f9007bc8b80 1 -- 172.21.15.119:0/659282473 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bf6377bfd0 con 0x55bf638d20d0 2023-12-07T10:23:03.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.226+0000 7f8ff1f9b700 1 --2- 172.21.15.119:0/659282473 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55bf638d20d0 0x55bf638d44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:03.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.226+0000 7f8ff1f9b700 1 --2- 172.21.15.119:0/659282473 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55bf638d20d0 0x55bf638d44f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e66c32b1f947062 server_cookie=c02f62dbcd4c3b92 in_seq=0 out_seq=0 2023-12-07T10:23:03.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.226+0000 7f8fd57fa700 10 client.26569 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:03.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.227+0000 7f8fd57fa700 1 -- 172.21.15.119:0/659282473 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bf6377bfd0 con 0x55bf638d20d0 2023-12-07T10:23:03.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.227+0000 7f8fd57fa700 10 client.26569 handle_command_reply: tid=0 2023-12-07T10:23:03.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.227+0000 7f9007bc8b80 1 client.26569 shutdown 2023-12-07T10:23:03.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.227+0000 7f9007bc8b80 1 -- 172.21.15.119:0/659282473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf63862460 msgr2=0x55bf63870720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:03.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.227+0000 7f9007bc8b80 1 --2- 172.21.15.119:0/659282473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf63862460 0x55bf63870720 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc0096b0 tx=0x7f8fdc0121a0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.227+0000 7f9007bc8b80 1 -- 172.21.15.119:0/659282473 shutdown_connections 2023-12-07T10:23:03.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.227+0000 7f9007bc8b80 1 --2- 172.21.15.119:0/659282473 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55bf638d20d0 0x55bf638d44f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.227+0000 7f9007bc8b80 1 --2- 172.21.15.119:0/659282473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bf63866150 0x55bf63870c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.227+0000 7f9007bc8b80 1 --2- 172.21.15.119:0/659282473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf63861b40 0x55bf638701e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.227+0000 7f9007bc8b80 1 --2- 172.21.15.119:0/659282473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf63862460 0x55bf63870720 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.227+0000 7f9007bc8b80 1 -- 172.21.15.119:0/659282473 >> 172.21.15.119:0/659282473 conn(0x55bf63790360 msgr2=0x55bf63876cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:03.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.227+0000 7f8fd57fa700 0 client.26569 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:03.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.227+0000 7f9007bc8b80 1 -- 172.21.15.119:0/659282473 shutdown_connections 2023-12-07T10:23:03.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.227+0000 7f9007bc8b80 1 -- 172.21.15.119:0/659282473 wait complete. 2023-12-07T10:23:03.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.228+0000 7f9007bc8b80 20 client.26569 trim_cache size 0 max 16384 2023-12-07T10:23:03.235 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:23:03.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.228+0000 7f8ff41ff700 1 -- 172.21.15.119:0/3962379964 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8fd80a1c80 msgr2=0x7f8fd80a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:03.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.228+0000 7f8ff41ff700 1 --2- 172.21.15.119:0/3962379964 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8fd80a1c80 0x7f8fd80a4140 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8015000 tx=0x7f8fe8005320 comp rx=0 tx=0).stop 2023-12-07T10:23:03.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.228+0000 7f8ff41ff700 1 -- 172.21.15.119:0/3962379964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8fec1053d0 msgr2=0x7f8fec110660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:03.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.228+0000 7f8ff41ff700 1 --2- 172.21.15.119:0/3962379964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8fec1053d0 0x7f8fec110660 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400efc0 tx=0x7f8fe400c3a0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.228+0000 7f8ff41ff700 1 -- 172.21.15.119:0/3962379964 shutdown_connections 2023-12-07T10:23:03.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.228+0000 7f8ff41ff700 1 --2- 172.21.15.119:0/3962379964 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8fd80a1c80 0x7f8fd80a4140 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.228+0000 7f8ff41ff700 1 --2- 172.21.15.119:0/3962379964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8fec1053d0 0x7f8fec110660 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.228+0000 7f8ff41ff700 1 --2- 172.21.15.119:0/3962379964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fec100460 0x7f8fec10fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.228+0000 7f8ff41ff700 1 --2- 172.21.15.119:0/3962379964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fec100e20 0x7f8fec110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:03.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.228+0000 7f8ff41ff700 1 -- 172.21.15.119:0/3962379964 >> 172.21.15.119:0/3962379964 conn(0x7f8fec077610 msgr2=0x7f8fec0fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:03.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.228+0000 7f8ff41ff700 1 -- 172.21.15.119:0/3962379964 shutdown_connections 2023-12-07T10:23:03.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:03.228+0000 7f8ff41ff700 1 -- 172.21.15.119:0/3962379964 wait complete. 2023-12-07T10:23:03.547 INFO:tasks.workunit.client.0.smithi119.stdout: 1 117463 8.58 MB/sec warmup 76 sec latency 12.595 ms 2023-12-07T10:23:04.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:03 smithi119 ceph-mon[97908]: pgmap v82: 129 pgs: 129 active+clean; 78 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 605 KiB/s wr, 106 op/s 2023-12-07T10:23:04.249 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-07T10:23:04.250 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-07T10:23:04.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:03 smithi159 ceph-mon[134726]: pgmap v82: 129 pgs: 129 active+clean; 78 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 605 KiB/s wr, 106 op/s 2023-12-07T10:23:04.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:03 smithi153 ceph-mon[133625]: pgmap v82: 129 pgs: 129 active+clean; 78 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 605 KiB/s wr, 106 op/s 2023-12-07T10:23:04.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.491+0000 7fd0fbdff700 1 -- 172.21.15.119:0/4209709491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0f4107620 msgr2=0x7fd0f4107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:04.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.491+0000 7fd0fbdff700 1 --2- 172.21.15.119:0/4209709491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0f4107620 0x7fd0f4107a00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd0f00099c0 tx=0x7fd0f0009cd0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.491+0000 7fd0fbdff700 1 -- 172.21.15.119:0/4209709491 shutdown_connections 2023-12-07T10:23:04.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.491+0000 7fd0fbdff700 1 --2- 172.21.15.119:0/4209709491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0f4101f20 0x7fd0f410e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.491+0000 7fd0fbdff700 1 --2- 172.21.15.119:0/4209709491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0f4107620 0x7fd0f4107a00 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.491+0000 7fd0fbdff700 1 --2- 172.21.15.119:0/4209709491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0f4101560 0x7fd0f41019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.491+0000 7fd0fbdff700 1 -- 172.21.15.119:0/4209709491 >> 172.21.15.119:0/4209709491 conn(0x7fd0f40fcfd0 msgr2=0x7fd0f40ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:04.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.491+0000 7fd0fbdff700 1 -- 172.21.15.119:0/4209709491 shutdown_connections 2023-12-07T10:23:04.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.491+0000 7fd0fbdff700 1 -- 172.21.15.119:0/4209709491 wait complete. 2023-12-07T10:23:04.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.493+0000 7fd0fbdff700 1 Processor -- start 2023-12-07T10:23:04.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.493+0000 7fd0fbdff700 1 -- start start 2023-12-07T10:23:04.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.493+0000 7fd0fbdff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0f4101560 0x7fd0f419f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.493+0000 7fd0fbdff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0f4101f20 0x7fd0f419fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.493+0000 7fd0fbdff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0f4107620 0x7fd0f4199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.493+0000 7fd0fbdff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f419a090 con 0x7fd0f4101560 2023-12-07T10:23:04.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.493+0000 7fd0fbdff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f419a200 con 0x7fd0f4101f20 2023-12-07T10:23:04.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.493+0000 7fd0fbdff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f419a3c0 con 0x7fd0f4107620 2023-12-07T10:23:04.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.494+0000 7fd0f939a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0f4101f20 0x7fd0f419fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.494+0000 7fd0f939a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0f4101f20 0x7fd0f419fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:58806/0 (socket says 172.21.15.119:58806) 2023-12-07T10:23:04.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.494+0000 7fd0f939a700 1 -- 172.21.15.119:0/2062581814 learned_addr learned my addr 172.21.15.119:0/2062581814 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:04.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.494+0000 7fd0f9b9b700 1 --2- 172.21.15.119:0/2062581814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0f4101560 0x7fd0f419f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.494+0000 7fd0fa39c700 1 --2- 172.21.15.119:0/2062581814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0f4107620 0x7fd0f4199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.494+0000 7fd0f9b9b700 1 -- 172.21.15.119:0/2062581814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0f4107620 msgr2=0x7fd0f4199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:04.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.494+0000 7fd0f9b9b700 1 --2- 172.21.15.119:0/2062581814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0f4107620 0x7fd0f4199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.494+0000 7fd0f9b9b700 1 -- 172.21.15.119:0/2062581814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0f4101f20 msgr2=0x7fd0f419fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:04.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.494+0000 7fd0f9b9b700 1 --2- 172.21.15.119:0/2062581814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0f4101f20 0x7fd0f419fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.494+0000 7fd0f9b9b700 1 -- 172.21.15.119:0/2062581814 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f00095f0 con 0x7fd0f4101560 2023-12-07T10:23:04.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.494+0000 7fd0f939a700 1 --2- 172.21.15.119:0/2062581814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0f4101f20 0x7fd0f419fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:04.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.494+0000 7fd0fa39c700 1 --2- 172.21.15.119:0/2062581814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0f4107620 0x7fd0f4199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:04.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.495+0000 7fd0f9b9b700 1 --2- 172.21.15.119:0/2062581814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0f4101560 0x7fd0f419f900 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000b580 tx=0x7fd0f000b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:04.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.495+0000 7fd0eaffd700 1 -- 172.21.15.119:0/2062581814 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f001d070 con 0x7fd0f4101560 2023-12-07T10:23:04.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.495+0000 7fd0eaffd700 1 -- 172.21.15.119:0/2062581814 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0f000be00 con 0x7fd0f4101560 2023-12-07T10:23:04.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.495+0000 7fd0eaffd700 1 -- 172.21.15.119:0/2062581814 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0005720 con 0x7fd0f4101560 2023-12-07T10:23:04.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.496+0000 7fd0fbdff700 1 -- 172.21.15.119:0/2062581814 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f419a640 con 0x7fd0f4101560 2023-12-07T10:23:04.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.497+0000 7fd0fbdff700 1 -- 172.21.15.119:0/2062581814 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f41a68f0 con 0x7fd0f4101560 2023-12-07T10:23:04.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.502+0000 7fd0eaffd700 1 -- 172.21.15.119:0/2062581814 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fd0f00047d0 con 0x7fd0f4101560 2023-12-07T10:23:04.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.503+0000 7fd10f779b80 1 Processor -- start 2023-12-07T10:23:04.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.503+0000 7fd0eaffd700 1 --2- 172.21.15.119:0/2062581814 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd0e00a1d00 0x7fd0e00a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.504+0000 7fd0eaffd700 1 -- 172.21.15.119:0/2062581814 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fd0f00ce730 con 0x7fd0f4101560 2023-12-07T10:23:04.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.504+0000 7fd10f779b80 1 -- start start 2023-12-07T10:23:04.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.504+0000 7fd0fa39c700 1 --2- 172.21.15.119:0/2062581814 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd0e00a1d00 0x7fd0e00a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.505+0000 7fd10f779b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558882568bc0 0x558882568fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.505+0000 7fd10f779b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588825694e0 0x55888256cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.505+0000 7fd10f779b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55888256d1d0 0x55888256d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.505+0000 7fd10f779b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588824dc840 con 0x5588825694e0 2023-12-07T10:23:04.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.505+0000 7fd10f779b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588824c3820 con 0x558882568bc0 2023-12-07T10:23:04.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.505+0000 7fd10f779b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588824c1140 con 0x55888256d1d0 2023-12-07T10:23:04.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.505+0000 7fd0f939a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588825694e0 0x55888256cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.505+0000 7fd0f939a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588825694e0 0x55888256cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58918/0 (socket says 172.21.15.119:58918) 2023-12-07T10:23:04.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.505+0000 7fd0f939a700 1 -- 172.21.15.119:0/3100253545 learned_addr learned my addr 172.21.15.119:0/3100253545 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:04.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.505+0000 7fd0fa39c700 1 --2- 172.21.15.119:0/2062581814 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd0e00a1d00 0x7fd0e00a41c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00a730 tx=0x7fd0ec00d040 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:04.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.506+0000 7fd0f9b9b700 1 --2- 172.21.15.119:0/3100253545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558882568bc0 0x558882568fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.505+0000 7fd0fa39c700 1 --2- 172.21.15.119:0/3100253545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55888256d1d0 0x55888256d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.506+0000 7fd0f939a700 1 -- 172.21.15.119:0/3100253545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55888256d1d0 msgr2=0x55888256d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:04.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.506+0000 7fd0f939a700 1 --2- 172.21.15.119:0/3100253545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55888256d1d0 0x55888256d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.506+0000 7fd0f939a700 1 -- 172.21.15.119:0/3100253545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558882568bc0 msgr2=0x558882568fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:04.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.506+0000 7fd0f939a700 1 --2- 172.21.15.119:0/3100253545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558882568bc0 0x558882568fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.506+0000 7fd0f939a700 1 -- 172.21.15.119:0/3100253545 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588823396b0 con 0x5588825694e0 2023-12-07T10:23:04.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.507+0000 7fd0f939a700 1 --2- 172.21.15.119:0/3100253545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588825694e0 0x55888256cc90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40116b0 tx=0x7fd0e40119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad486943a4ceb6f2 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:04.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.507+0000 7fd0fa39c700 1 --2- 172.21.15.119:0/3100253545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55888256d1d0 0x55888256d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:23:04.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.507+0000 7fd0f9b9b700 1 --2- 172.21.15.119:0/3100253545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558882568bc0 0x558882568fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:23:04.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.507+0000 7fd0dffff700 1 -- 172.21.15.119:0/3100253545 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4004030 con 0x5588825694e0 2023-12-07T10:23:04.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.507+0000 7fd0dffff700 1 -- 172.21.15.119:0/3100253545 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0e4003680 con 0x5588825694e0 2023-12-07T10:23:04.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.507+0000 7fd0dffff700 1 -- 172.21.15.119:0/3100253545 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4019810 con 0x5588825694e0 2023-12-07T10:23:04.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.508+0000 7fd10f779b80 1 -- 172.21.15.119:0/3100253545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588825694e0 msgr2=0x55888256cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:04.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.508+0000 7fd10f779b80 1 --2- 172.21.15.119:0/3100253545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588825694e0 0x55888256cc90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40116b0 tx=0x7fd0e40119c0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.508+0000 7fd10f779b80 1 -- 172.21.15.119:0/3100253545 shutdown_connections 2023-12-07T10:23:04.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.508+0000 7fd10f779b80 1 --2- 172.21.15.119:0/3100253545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55888256d1d0 0x55888256d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.508+0000 7fd10f779b80 1 --2- 172.21.15.119:0/3100253545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558882568bc0 0x558882568fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.508+0000 7fd10f779b80 1 --2- 172.21.15.119:0/3100253545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588825694e0 0x55888256cc90 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.508+0000 7fd10f779b80 1 -- 172.21.15.119:0/3100253545 >> 172.21.15.119:0/3100253545 conn(0x5588824973e0 msgr2=0x5588825631b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:04.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.508+0000 7fd10f779b80 1 -- 172.21.15.119:0/3100253545 shutdown_connections 2023-12-07T10:23:04.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.508+0000 7fd10f779b80 1 -- 172.21.15.119:0/3100253545 wait complete. 2023-12-07T10:23:04.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.508+0000 7fd10f779b80 1 -- start start 2023-12-07T10:23:04.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.509+0000 7fd10f779b80 1 Processor -- start 2023-12-07T10:23:04.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.509+0000 7fd10f779b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558882568bc0 0x558882574920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.510+0000 7fd10f779b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588825694e0 0x558882571d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.510+0000 7fd10f779b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55888256d1d0 0x558882572360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.510+0000 7fd10f779b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588824bca30 con 0x55888256d1d0 2023-12-07T10:23:04.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.510+0000 7fd10f779b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588824765e0 con 0x5588825694e0 2023-12-07T10:23:04.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.510+0000 7fd10f779b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558882476280 con 0x558882568bc0 2023-12-07T10:23:04.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.510+0000 7fd10f779b80 10 client.0 _ll_register_callbacks cb 0x55888245d9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:04.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.510+0000 7fd0fa39c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55888256d1d0 0x558882572360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.510+0000 7fd0fa39c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55888256d1d0 0x558882572360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58920/0 (socket says 172.21.15.119:58920) 2023-12-07T10:23:04.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.510+0000 7fd0fa39c700 1 -- 172.21.15.119:0/3053591003 learned_addr learned my addr 172.21.15.119:0/3053591003 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:04.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.510+0000 7fd0f9b9b700 1 --2- 172.21.15.119:0/3053591003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558882568bc0 0x558882574920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.510+0000 7fd0f939a700 1 --2- 172.21.15.119:0/3053591003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588825694e0 0x558882571d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.511+0000 7fd0fa39c700 1 -- 172.21.15.119:0/3053591003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558882568bc0 msgr2=0x558882574920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:04.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.511+0000 7fd0fa39c700 1 --2- 172.21.15.119:0/3053591003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558882568bc0 0x558882574920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.511+0000 7fd0fa39c700 1 -- 172.21.15.119:0/3053591003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588825694e0 msgr2=0x558882571d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:04.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.511+0000 7fd0fa39c700 1 --2- 172.21.15.119:0/3053591003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588825694e0 0x558882571d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.511+0000 7fd0fa39c700 1 -- 172.21.15.119:0/3053591003 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e4009e10 con 0x55888256d1d0 2023-12-07T10:23:04.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.511+0000 7fd0f939a700 1 --2- 172.21.15.119:0/3053591003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588825694e0 0x558882571d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:23:04.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.511+0000 7fd0fa39c700 1 --2- 172.21.15.119:0/3053591003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55888256d1d0 0x558882572360 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec015410 tx=0x7fd0ec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b485ec8b36c48930 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:04.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.511+0000 7fd0dd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:23:04.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.511+0000 7fd0dd7fa700 1 -- 172.21.15.119:0/3053591003 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec01b050 con 0x55888256d1d0 2023-12-07T10:23:04.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.511+0000 7fd10f779b80 1 -- 172.21.15.119:0/3053591003 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55888255bb80 con 0x55888256d1d0 2023-12-07T10:23:04.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.511+0000 7fd0dd7fa700 1 -- 172.21.15.119:0/3053591003 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fd0ec006e40 con 0x55888256d1d0 2023-12-07T10:23:04.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.511+0000 7fd0dd7fa700 1 -- 172.21.15.119:0/3053591003 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0ec010040 con 0x55888256d1d0 2023-12-07T10:23:04.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.511+0000 7fd10f779b80 10 client.16572 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:04.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.511+0000 7fd10f779b80 1 -- 172.21.15.119:0/3053591003 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5588822e5e60 con 0x55888256d1d0 2023-12-07T10:23:04.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.512+0000 7fd0dd7fa700 1 -- 172.21.15.119:0/3053591003 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55888255bb80 con 0x55888256d1d0 2023-12-07T10:23:04.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.512+0000 7fd0dd7fa700 1 -- 172.21.15.119:0/3053591003 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fd0ec02a6e0 con 0x55888256d1d0 2023-12-07T10:23:04.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.512+0000 7fd10f779b80 10 client.16572 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:04.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.512+0000 7fd10f779b80 10 client.16572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:23:04.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.512+0000 7fd10f779b80 20 client.16572 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:04.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.512+0000 7fd10f779b80 1 --2- 172.21.15.119:0/3053591003 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55888258acd0 0x55888258d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.512+0000 7fd10f779b80 4 client.16572 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:23:04.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.512+0000 7fd10f779b80 1 -- 172.21.15.119:0/3053591003 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558882537820 con 0x55888258acd0 2023-12-07T10:23:04.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.512+0000 7fd0f9b9b700 1 --2- 172.21.15.119:0/3053591003 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55888258acd0 0x55888258d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.513+0000 7fd0f9b9b700 1 --2- 172.21.15.119:0/3053591003 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55888258acd0 0x55888258d0f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dff4fd561a77f4b8 server_cookie=ee4c77b3835f3af5 in_seq=0 out_seq=0 2023-12-07T10:23:04.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.513+0000 7fd0dd7fa700 10 client.16572 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:04.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.513+0000 7fd0dd7fa700 1 -- 172.21.15.119:0/3053591003 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7fd0f0079e90 con 0x55888258acd0 2023-12-07T10:23:04.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.513+0000 7fd0dd7fa700 10 client.16572 handle_command_reply: tid=0 2023-12-07T10:23:04.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.513+0000 7fd10f779b80 1 client.16572 shutdown 2023-12-07T10:23:04.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.514+0000 7fd10f779b80 1 -- 172.21.15.119:0/3053591003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55888256d1d0 msgr2=0x558882572360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:04.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.514+0000 7fd10f779b80 1 --2- 172.21.15.119:0/3053591003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55888256d1d0 0x558882572360 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec015410 tx=0x7fd0ec004ad0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.514+0000 7fd10f779b80 1 -- 172.21.15.119:0/3053591003 shutdown_connections 2023-12-07T10:23:04.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.514+0000 7fd10f779b80 1 --2- 172.21.15.119:0/3053591003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55888256d1d0 0x558882572360 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.514+0000 7fd10f779b80 1 --2- 172.21.15.119:0/3053591003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558882568bc0 0x558882574920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.514+0000 7fd10f779b80 1 --2- 172.21.15.119:0/3053591003 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55888258acd0 0x55888258d0f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.514+0000 7fd10f779b80 1 --2- 172.21.15.119:0/3053591003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588825694e0 0x558882571d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.514+0000 7fd10f779b80 1 -- 172.21.15.119:0/3053591003 >> 172.21.15.119:0/3053591003 conn(0x5588824973e0 msgr2=0x55888255dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:04.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.514+0000 7fd0dd7fa700 0 client.16572 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:04.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.514+0000 7fd10f779b80 1 -- 172.21.15.119:0/3053591003 shutdown_connections 2023-12-07T10:23:04.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.514+0000 7fd10f779b80 1 -- 172.21.15.119:0/3053591003 wait complete. 2023-12-07T10:23:04.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.514+0000 7fd10f779b80 20 client.16572 trim_cache size 0 max 16384 2023-12-07T10:23:04.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.534+0000 7fd10f779b80 1 Processor -- start 2023-12-07T10:23:04.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.534+0000 7fd10f779b80 1 -- start start 2023-12-07T10:23:04.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.535+0000 7fd10f779b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558882568bc0 0x558882574830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.535+0000 7fd10f779b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588825694e0 0x558882563110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.535+0000 7fd10f779b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55888256d1d0 0x558882563650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.535+0000 7fd10f779b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55888256dad0 con 0x558882568bc0 2023-12-07T10:23:04.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.535+0000 7fd10f779b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55888233b9a0 con 0x5588825694e0 2023-12-07T10:23:04.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.535+0000 7fd10f779b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558882563ce0 con 0x55888256d1d0 2023-12-07T10:23:04.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.535+0000 7fd0f9b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558882568bc0 0x558882574830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.535+0000 7fd0f939a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588825694e0 0x558882563110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.535+0000 7fd0f9b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558882568bc0 0x558882574830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58932/0 (socket says 172.21.15.119:58932) 2023-12-07T10:23:04.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.535+0000 7fd0f939a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588825694e0 0x558882563110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:58848/0 (socket says 172.21.15.119:58848) 2023-12-07T10:23:04.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.535+0000 7fd0f9b9b700 1 -- 172.21.15.119:0/1854258452 learned_addr learned my addr 172.21.15.119:0/1854258452 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:04.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.535+0000 7fd0fa39c700 1 --2- 172.21.15.119:0/1854258452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55888256d1d0 0x558882563650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.535+0000 7fd0f939a700 1 -- 172.21.15.119:0/1854258452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55888256d1d0 msgr2=0x558882563650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:04.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.535+0000 7fd0f939a700 1 --2- 172.21.15.119:0/1854258452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55888256d1d0 0x558882563650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.535+0000 7fd0f939a700 1 -- 172.21.15.119:0/1854258452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558882568bc0 msgr2=0x558882574830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:04.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.536+0000 7fd0f939a700 1 --2- 172.21.15.119:0/1854258452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558882568bc0 0x558882574830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.536+0000 7fd0f939a700 1 -- 172.21.15.119:0/1854258452 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55888255d0c0 con 0x5588825694e0 2023-12-07T10:23:04.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.536+0000 7fd0f9b9b700 1 --2- 172.21.15.119:0/1854258452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558882568bc0 0x558882574830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:04.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.536+0000 7fd0f939a700 1 --2- 172.21.15.119:0/1854258452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588825694e0 0x558882563110 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4003f70 tx=0x7fd0e4005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39d741755eb8cb18 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:04.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.536+0000 7fd0e8bff700 1 -- 172.21.15.119:0/1854258452 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e40038f0 con 0x5588825694e0 2023-12-07T10:23:04.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.536+0000 7fd0e8bff700 1 -- 172.21.15.119:0/1854258452 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0e40050c0 con 0x5588825694e0 2023-12-07T10:23:04.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.536+0000 7fd0e8bff700 1 -- 172.21.15.119:0/1854258452 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e40197b0 con 0x5588825694e0 2023-12-07T10:23:04.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.537+0000 7fd10f779b80 1 -- 172.21.15.119:0/1854258452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588825694e0 msgr2=0x558882563110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:04.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.537+0000 7fd10f779b80 1 --2- 172.21.15.119:0/1854258452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588825694e0 0x558882563110 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4003f70 tx=0x7fd0e4005dc0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.537+0000 7fd10f779b80 1 -- 172.21.15.119:0/1854258452 shutdown_connections 2023-12-07T10:23:04.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.537+0000 7fd10f779b80 1 --2- 172.21.15.119:0/1854258452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55888256d1d0 0x558882563650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.537+0000 7fd10f779b80 1 --2- 172.21.15.119:0/1854258452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558882568bc0 0x558882574830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.537+0000 7fd10f779b80 1 --2- 172.21.15.119:0/1854258452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5588825694e0 0x558882563110 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.537+0000 7fd10f779b80 1 -- 172.21.15.119:0/1854258452 >> 172.21.15.119:0/1854258452 conn(0x5588824973e0 msgr2=0x55888257de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:04.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.537+0000 7fd10f779b80 1 -- 172.21.15.119:0/1854258452 shutdown_connections 2023-12-07T10:23:04.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.537+0000 7fd10f779b80 1 -- 172.21.15.119:0/1854258452 wait complete. 2023-12-07T10:23:04.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.538+0000 7fd10f779b80 1 -- start start 2023-12-07T10:23:04.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.538+0000 7fd10f779b80 1 Processor -- start 2023-12-07T10:23:04.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.539+0000 7fd10f779b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558882568bc0 0x558882579590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.539+0000 7fd10f779b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588825694e0 0x558882573440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.539+0000 7fd10f779b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55888256d1d0 0x558882573980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.539+0000 7fd10f779b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558882573ec0 con 0x55888256d1d0 2023-12-07T10:23:04.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.539+0000 7fd10f779b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558882574030 con 0x558882568bc0 2023-12-07T10:23:04.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.539+0000 7fd10f779b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588825741f0 con 0x5588825694e0 2023-12-07T10:23:04.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.539+0000 7fd10f779b80 10 client.0 _ll_register_callbacks cb 0x55888245d9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:04.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.539+0000 7fd0f9b9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558882568bc0 0x558882579590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.539+0000 7fd0f9b9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558882568bc0 0x558882579590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:58858/0 (socket says 172.21.15.119:58858) 2023-12-07T10:23:04.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.539+0000 7fd0f9b9b700 1 -- 172.21.15.119:0/3221806019 learned_addr learned my addr 172.21.15.119:0/3221806019 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:04.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.539+0000 7fd0f939a700 1 --2- 172.21.15.119:0/3221806019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588825694e0 0x558882573440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.539+0000 7fd0fa39c700 1 --2- 172.21.15.119:0/3221806019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55888256d1d0 0x558882573980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.539+0000 7fd0f9b9b700 1 -- 172.21.15.119:0/3221806019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588825694e0 msgr2=0x558882573440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:04.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.539+0000 7fd0f9b9b700 1 --2- 172.21.15.119:0/3221806019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588825694e0 0x558882573440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.539+0000 7fd0f9b9b700 1 -- 172.21.15.119:0/3221806019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55888256d1d0 msgr2=0x558882573980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:04.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.539+0000 7fd0f9b9b700 1 --2- 172.21.15.119:0/3221806019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55888256d1d0 0x558882573980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.539+0000 7fd0f9b9b700 1 -- 172.21.15.119:0/3221806019 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f41a68f0 con 0x558882568bc0 2023-12-07T10:23:04.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.539+0000 7fd0fa39c700 1 --2- 172.21.15.119:0/3221806019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55888256d1d0 0x558882573980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:23:04.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.540+0000 7fd0f9b9b700 1 --2- 172.21.15.119:0/3221806019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558882568bc0 0x558882579590 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0005d70 tx=0x7fd0f009b100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2ee8a7dc9d54727 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:04.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.540+0000 7fd0dd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:23:04.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.540+0000 7fd0dd7fa700 1 -- 172.21.15.119:0/3221806019 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f00a3510 con 0x558882568bc0 2023-12-07T10:23:04.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.540+0000 7fd0dd7fa700 1 -- 172.21.15.119:0/3221806019 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fd0f00995b0 con 0x558882568bc0 2023-12-07T10:23:04.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.540+0000 7fd10f779b80 1 -- 172.21.15.119:0/3221806019 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0e4003680 con 0x558882568bc0 2023-12-07T10:23:04.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.540+0000 7fd0dd7fa700 1 -- 172.21.15.119:0/3221806019 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0f00cbdb0 con 0x558882568bc0 2023-12-07T10:23:04.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.540+0000 7fd10f779b80 10 client.26603 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:04.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.540+0000 7fd10f779b80 1 -- 172.21.15.119:0/3221806019 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5588823396b0 con 0x558882568bc0 2023-12-07T10:23:04.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.540+0000 7fd0dd7fa700 1 -- 172.21.15.119:0/3221806019 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0e4003680 con 0x558882568bc0 2023-12-07T10:23:04.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.540+0000 7fd0dd7fa700 1 -- 172.21.15.119:0/3221806019 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fd0f009b470 con 0x558882568bc0 2023-12-07T10:23:04.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.541+0000 7fd10f779b80 10 client.26603 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:04.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.541+0000 7fd10f779b80 10 client.26603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:23:04.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.541+0000 7fd10f779b80 20 client.26603 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:04.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.541+0000 7fd10f779b80 1 --2- 172.21.15.119:0/3221806019 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5588825e2500 0x5588825e4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.541+0000 7fd10f779b80 4 client.26603 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-07T10:23:04.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.541+0000 7fd10f779b80 1 -- 172.21.15.119:0/3221806019 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558882483050 con 0x5588825e2500 2023-12-07T10:23:04.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.541+0000 7fd0f939a700 1 --2- 172.21.15.119:0/3221806019 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5588825e2500 0x5588825e4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.541+0000 7fd0f939a700 1 --2- 172.21.15.119:0/3221806019 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5588825e2500 0x5588825e4920 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=895b3cd564832ea7 server_cookie=c89014b4b441b7e4 in_seq=0 out_seq=0 2023-12-07T10:23:04.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.541+0000 7fd0dd7fa700 10 client.26603 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:04.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.542+0000 7fd0dd7fa700 1 -- 172.21.15.119:0/3221806019 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558882483050 con 0x5588825e2500 2023-12-07T10:23:04.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.543+0000 7fd0dd7fa700 10 client.26603 handle_command_reply: tid=0 2023-12-07T10:23:04.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.543+0000 7fd10f779b80 1 client.26603 shutdown 2023-12-07T10:23:04.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.543+0000 7fd10f779b80 1 -- 172.21.15.119:0/3221806019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558882568bc0 msgr2=0x558882579590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:04.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.543+0000 7fd10f779b80 1 --2- 172.21.15.119:0/3221806019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558882568bc0 0x558882579590 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0005d70 tx=0x7fd0f009b100 comp rx=0 tx=0).stop 2023-12-07T10:23:04.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.543+0000 7fd10f779b80 1 -- 172.21.15.119:0/3221806019 shutdown_connections 2023-12-07T10:23:04.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.543+0000 7fd10f779b80 1 --2- 172.21.15.119:0/3221806019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55888256d1d0 0x558882573980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.543+0000 7fd10f779b80 1 --2- 172.21.15.119:0/3221806019 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5588825e2500 0x5588825e4920 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.543+0000 7fd10f779b80 1 --2- 172.21.15.119:0/3221806019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558882568bc0 0x558882579590 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.543+0000 7fd10f779b80 1 --2- 172.21.15.119:0/3221806019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588825694e0 0x558882573440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.543+0000 7fd10f779b80 1 -- 172.21.15.119:0/3221806019 >> 172.21.15.119:0/3221806019 conn(0x5588824973e0 msgr2=0x55888257de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:04.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.543+0000 7fd0dd7fa700 0 client.26603 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:04.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.543+0000 7fd10f779b80 1 -- 172.21.15.119:0/3221806019 shutdown_connections 2023-12-07T10:23:04.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.543+0000 7fd10f779b80 1 -- 172.21.15.119:0/3221806019 wait complete. 2023-12-07T10:23:04.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.544+0000 7fd10f779b80 20 client.26603 trim_cache size 0 max 16384 2023-12-07T10:23:04.549 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:23:04.549 INFO:teuthology.orchestra.run.smithi119.stdout: "return_code": 0, 2023-12-07T10:23:04.549 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_tag": "8087ef31-45b6-4e4d-b58b-cce61b2af86d", 2023-12-07T10:23:04.549 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": "asynchronous" 2023-12-07T10:23:04.550 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:23:04.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.544+0000 7fd0fbdff700 1 -- 172.21.15.119:0/2062581814 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd0e00a1d00 msgr2=0x7fd0e00a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:04.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.544+0000 7fd0fbdff700 1 --2- 172.21.15.119:0/2062581814 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd0e00a1d00 0x7fd0e00a41c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00a730 tx=0x7fd0ec00d040 comp rx=0 tx=0).stop 2023-12-07T10:23:04.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.544+0000 7fd0fbdff700 1 -- 172.21.15.119:0/2062581814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0f4101560 msgr2=0x7fd0f419f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:04.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.544+0000 7fd0fbdff700 1 --2- 172.21.15.119:0/2062581814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0f4101560 0x7fd0f419f900 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000b580 tx=0x7fd0f000b9c0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.545+0000 7fd0fbdff700 1 -- 172.21.15.119:0/2062581814 shutdown_connections 2023-12-07T10:23:04.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.545+0000 7fd0fbdff700 1 --2- 172.21.15.119:0/2062581814 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd0e00a1d00 0x7fd0e00a41c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.545+0000 7fd0fbdff700 1 --2- 172.21.15.119:0/2062581814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0f4107620 0x7fd0f4199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.545+0000 7fd0fbdff700 1 --2- 172.21.15.119:0/2062581814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0f4101560 0x7fd0f419f900 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.545+0000 7fd0fbdff700 1 --2- 172.21.15.119:0/2062581814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0f4101f20 0x7fd0f419fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.545+0000 7fd0fbdff700 1 -- 172.21.15.119:0/2062581814 >> 172.21.15.119:0/2062581814 conn(0x7fd0f40fcfd0 msgr2=0x7fd0f40fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:04.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.545+0000 7fd0fbdff700 1 -- 172.21.15.119:0/2062581814 shutdown_connections 2023-12-07T10:23:04.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.545+0000 7fd0fbdff700 1 -- 172.21.15.119:0/2062581814 wait complete. 2023-12-07T10:23:04.551 INFO:tasks.workunit.client.0.smithi119.stdout: 1 119271 8.54 MB/sec warmup 77 sec latency 12.320 ms 2023-12-07T10:23:04.574 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-07T10:23:04.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.804+0000 7f448c5ff700 1 -- 172.21.15.119:0/2153919475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44841023d0 msgr2=0x7f44841027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:04.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.804+0000 7f448c5ff700 1 --2- 172.21.15.119:0/2153919475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44841023d0 0x7f44841027b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f448000b0a0 tx=0x7f448000b3b0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.804+0000 7f448c5ff700 1 -- 172.21.15.119:0/2153919475 shutdown_connections 2023-12-07T10:23:04.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.804+0000 7f448c5ff700 1 --2- 172.21.15.119:0/2153919475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44841010d0 0x7f448410d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.804+0000 7f448c5ff700 1 --2- 172.21.15.119:0/2153919475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44841023d0 0x7f44841027b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.804+0000 7f448c5ff700 1 --2- 172.21.15.119:0/2153919475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4484100710 0x7f4484100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.804+0000 7f448c5ff700 1 -- 172.21.15.119:0/2153919475 >> 172.21.15.119:0/2153919475 conn(0x7f4484078390 msgr2=0x7f44840787a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:04.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.805+0000 7f448c5ff700 1 -- 172.21.15.119:0/2153919475 shutdown_connections 2023-12-07T10:23:04.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.805+0000 7f448c5ff700 1 -- 172.21.15.119:0/2153919475 wait complete. 2023-12-07T10:23:04.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.806+0000 7f448c5ff700 1 Processor -- start 2023-12-07T10:23:04.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.806+0000 7f448c5ff700 1 -- start start 2023-12-07T10:23:04.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.806+0000 7f448c5ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4484100710 0x7f4484199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.806+0000 7f448c5ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44841010d0 0x7f448419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.806+0000 7f448c5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44841023d0 0x7f448419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.806+0000 7f448c5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448419edc0 con 0x7f44841023d0 2023-12-07T10:23:04.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.806+0000 7f448c5ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448419ef30 con 0x7f44841010d0 2023-12-07T10:23:04.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.806+0000 7f448c5ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448419f0f0 con 0x7f4484100710 2023-12-07T10:23:04.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.806+0000 7f448a39b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4484100710 0x7f4484199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.806+0000 7f4489b9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44841010d0 0x7f448419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.807+0000 7f448a39b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4484100710 0x7f4484199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:35058/0 (socket says 172.21.15.119:35058) 2023-12-07T10:23:04.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.807+0000 7f448a39b700 1 -- 172.21.15.119:0/3100065044 learned_addr learned my addr 172.21.15.119:0/3100065044 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:04.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.807+0000 7f448ab9c700 1 --2- 172.21.15.119:0/3100065044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44841023d0 0x7f448419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.807+0000 7f448a39b700 1 -- 172.21.15.119:0/3100065044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44841010d0 msgr2=0x7f448419a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:04.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.807+0000 7f448a39b700 1 --2- 172.21.15.119:0/3100065044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44841010d0 0x7f448419a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.807+0000 7f448a39b700 1 -- 172.21.15.119:0/3100065044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44841023d0 msgr2=0x7f448419e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:04.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.807+0000 7f448a39b700 1 --2- 172.21.15.119:0/3100065044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44841023d0 0x7f448419e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.807+0000 7f448a39b700 1 -- 172.21.15.119:0/3100065044 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4480009d00 con 0x7f4484100710 2023-12-07T10:23:04.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.807+0000 7f448a39b700 1 --2- 172.21.15.119:0/3100065044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4484100710 0x7f4484199ea0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f448000b780 tx=0x7f4480007940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:04.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.808+0000 7f447b7fe700 1 -- 172.21.15.119:0/3100065044 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448000e040 con 0x7f4484100710 2023-12-07T10:23:04.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.808+0000 7f448c5ff700 1 -- 172.21.15.119:0/3100065044 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448419f370 con 0x7f4484100710 2023-12-07T10:23:04.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.808+0000 7f447b7fe700 1 -- 172.21.15.119:0/3100065044 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f448001d470 con 0x7f4484100710 2023-12-07T10:23:04.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.808+0000 7f447b7fe700 1 -- 172.21.15.119:0/3100065044 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44800052a0 con 0x7f4484100710 2023-12-07T10:23:04.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.808+0000 7f448c5ff700 1 -- 172.21.15.119:0/3100065044 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44841a6c10 con 0x7f4484100710 2023-12-07T10:23:04.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.811+0000 7f447b7fe700 1 -- 172.21.15.119:0/3100065044 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f448001d5e0 con 0x7f4484100710 2023-12-07T10:23:04.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.813+0000 7f447b7fe700 1 --2- 172.21.15.119:0/3100065044 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f44700a1c80 0x7f44700a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.813+0000 7f44a000eb80 1 Processor -- start 2023-12-07T10:23:04.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.813+0000 7f447b7fe700 1 -- 172.21.15.119:0/3100065044 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f44800ced40 con 0x7f4484100710 2023-12-07T10:23:04.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.813+0000 7f4489b9a700 1 --2- 172.21.15.119:0/3100065044 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f44700a1c80 0x7f44700a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.813+0000 7f44a000eb80 1 -- start start 2023-12-07T10:23:04.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.814+0000 7f44a000eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562940785b40 0x562940785f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.814+0000 7f44a000eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562940786460 0x562940789c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.814+0000 7f44a000eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56294078a150 0x56294078a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.814+0000 7f44a000eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629406f97c0 con 0x562940785b40 2023-12-07T10:23:04.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.814+0000 7f44a000eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629406e07a0 con 0x56294078a150 2023-12-07T10:23:04.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.814+0000 7f44a000eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629406de0c0 con 0x562940786460 2023-12-07T10:23:04.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.814+0000 7f4489b9a700 1 --2- 172.21.15.119:0/3100065044 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f44700a1c80 0x7f44700a4140 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f448419b3f0 tx=0x7f4474009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:04.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.814+0000 7f448a39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562940785b40 0x562940785f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.814+0000 7f4489b9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562940786460 0x562940789c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.814+0000 7f448a39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562940785b40 0x562940785f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58974/0 (socket says 172.21.15.119:58974) 2023-12-07T10:23:04.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.814+0000 7f448a39b700 1 -- 172.21.15.119:0/4152034655 learned_addr learned my addr 172.21.15.119:0/4152034655 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:04.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.814+0000 7f448ab9c700 1 --2- 172.21.15.119:0/4152034655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56294078a150 0x56294078a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.814+0000 7f448a39b700 1 -- 172.21.15.119:0/4152034655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562940786460 msgr2=0x562940789c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:04.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.814+0000 7f448a39b700 1 --2- 172.21.15.119:0/4152034655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562940786460 0x562940789c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.814+0000 7f448a39b700 1 -- 172.21.15.119:0/4152034655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56294078a150 msgr2=0x56294078a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:04.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.815+0000 7f448a39b700 1 --2- 172.21.15.119:0/4152034655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56294078a150 0x56294078a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.815+0000 7f448a39b700 1 -- 172.21.15.119:0/4152034655 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562940556630 con 0x562940785b40 2023-12-07T10:23:04.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.815+0000 7f448ab9c700 1 --2- 172.21.15.119:0/4152034655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56294078a150 0x56294078a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:04.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.815+0000 7f448a39b700 1 --2- 172.21.15.119:0/4152034655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562940785b40 0x562940785f20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f448009b750 tx=0x7f4480097b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2895af5c543a5f8 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:04.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.815+0000 7f4478bfe700 1 -- 172.21.15.119:0/4152034655 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448009b120 con 0x562940785b40 2023-12-07T10:23:04.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.815+0000 7f4478bfe700 1 -- 172.21.15.119:0/4152034655 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44800bb0e0 con 0x562940785b40 2023-12-07T10:23:04.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.815+0000 7f4478bfe700 1 -- 172.21.15.119:0/4152034655 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44800a3b30 con 0x562940785b40 2023-12-07T10:23:04.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.816+0000 7f44a000eb80 1 -- 172.21.15.119:0/4152034655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562940785b40 msgr2=0x562940785f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:04.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.816+0000 7f44a000eb80 1 --2- 172.21.15.119:0/4152034655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562940785b40 0x562940785f20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f448009b750 tx=0x7f4480097b00 comp rx=0 tx=0).stop 2023-12-07T10:23:04.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.816+0000 7f44a000eb80 1 -- 172.21.15.119:0/4152034655 shutdown_connections 2023-12-07T10:23:04.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.816+0000 7f44a000eb80 1 --2- 172.21.15.119:0/4152034655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56294078a150 0x56294078a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.816+0000 7f44a000eb80 1 --2- 172.21.15.119:0/4152034655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562940785b40 0x562940785f20 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.816+0000 7f44a000eb80 1 --2- 172.21.15.119:0/4152034655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562940786460 0x562940789c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.816+0000 7f44a000eb80 1 -- 172.21.15.119:0/4152034655 >> 172.21.15.119:0/4152034655 conn(0x5629406b4360 msgr2=0x562940780130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:04.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.816+0000 7f44a000eb80 1 -- 172.21.15.119:0/4152034655 shutdown_connections 2023-12-07T10:23:04.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.816+0000 7f44a000eb80 1 -- 172.21.15.119:0/4152034655 wait complete. 2023-12-07T10:23:04.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.817+0000 7f44a000eb80 1 -- start start 2023-12-07T10:23:04.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.817+0000 7f44a000eb80 1 Processor -- start 2023-12-07T10:23:04.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.818+0000 7f44a000eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562940785b40 0x56294078ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.818+0000 7f44a000eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562940786460 0x56294078f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.818+0000 7f44a000eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56294078a150 0x56294078f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.818+0000 7f44a000eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629406d99b0 con 0x562940786460 2023-12-07T10:23:04.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.818+0000 7f44a000eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562940693560 con 0x56294078a150 2023-12-07T10:23:04.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.818+0000 7f44a000eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562940693200 con 0x562940785b40 2023-12-07T10:23:04.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.818+0000 7f44a000eb80 10 client.0 _ll_register_callbacks cb 0x56294067a940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:04.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.819+0000 7f448ab9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56294078a150 0x56294078f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.819+0000 7f448ab9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56294078a150 0x56294078f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:58898/0 (socket says 172.21.15.119:58898) 2023-12-07T10:23:04.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.819+0000 7f448ab9c700 1 -- 172.21.15.119:0/2553260916 learned_addr learned my addr 172.21.15.119:0/2553260916 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:04.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.819+0000 7f4489b9a700 1 --2- 172.21.15.119:0/2553260916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562940786460 0x56294078f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.819+0000 7f448a39b700 1 --2- 172.21.15.119:0/2553260916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562940785b40 0x56294078ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.819+0000 7f448ab9c700 1 -- 172.21.15.119:0/2553260916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562940785b40 msgr2=0x56294078ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:04.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.819+0000 7f448ab9c700 1 --2- 172.21.15.119:0/2553260916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562940785b40 0x56294078ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.819+0000 7f448ab9c700 1 -- 172.21.15.119:0/2553260916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562940786460 msgr2=0x56294078f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:04.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.819+0000 7f448ab9c700 1 --2- 172.21.15.119:0/2553260916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562940786460 0x56294078f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.819+0000 7f448ab9c700 1 -- 172.21.15.119:0/2553260916 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447c0098c0 con 0x56294078a150 2023-12-07T10:23:04.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.820+0000 7f448ab9c700 1 --2- 172.21.15.119:0/2553260916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56294078a150 0x56294078f670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f447c012980 tx=0x7f447c012c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2909dd0fe8f9c405 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:04.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.820+0000 7f446dffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:23:04.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.820+0000 7f446dffb700 1 -- 172.21.15.119:0/2553260916 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c017070 con 0x56294078a150 2023-12-07T10:23:04.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.820+0000 7f446dffb700 1 -- 172.21.15.119:0/2553260916 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f447c004050 con 0x56294078a150 2023-12-07T10:23:04.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.820+0000 7f44a000eb80 1 -- 172.21.15.119:0/2553260916 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562940778b00 con 0x56294078a150 2023-12-07T10:23:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.820+0000 7f446dffb700 1 -- 172.21.15.119:0/2553260916 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f447c004210 con 0x56294078a150 2023-12-07T10:23:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.820+0000 7f44a000eb80 10 client.26615 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.820+0000 7f44a000eb80 1 -- 172.21.15.119:0/2553260916 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f44841a6c10 con 0x56294078a150 2023-12-07T10:23:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.821+0000 7f446dffb700 1 -- 172.21.15.119:0/2553260916 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562940778b00 con 0x56294078a150 2023-12-07T10:23:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.821+0000 7f446dffb700 1 -- 172.21.15.119:0/2553260916 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f447c00f680 con 0x56294078a150 2023-12-07T10:23:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.821+0000 7f44a000eb80 10 client.26615 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.821+0000 7f44a000eb80 10 client.26615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:23:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.821+0000 7f44a000eb80 20 client.26615 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.821+0000 7f44a000eb80 1 --2- 172.21.15.119:0/2553260916 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56294079f150 0x5629407a1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.821+0000 7f44a000eb80 4 client.26615 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:23:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.821+0000 7f44a000eb80 1 -- 172.21.15.119:0/2553260916 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629407547a0 con 0x56294079f150 2023-12-07T10:23:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.821+0000 7f448a39b700 1 --2- 172.21.15.119:0/2553260916 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56294079f150 0x5629407a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.822+0000 7f448a39b700 1 --2- 172.21.15.119:0/2553260916 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56294079f150 0x5629407a1570 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=262cbc16ed2ece1b server_cookie=c0571261a7968a77 in_seq=0 out_seq=0 2023-12-07T10:23:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.822+0000 7f446dffb700 10 client.26615 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.822+0000 7f446dffb700 1 -- 172.21.15.119:0/2553260916 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5629407547a0 con 0x56294079f150 2023-12-07T10:23:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.822+0000 7f446dffb700 10 client.26615 handle_command_reply: tid=0 2023-12-07T10:23:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.823+0000 7f44a000eb80 1 client.26615 shutdown 2023-12-07T10:23:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.823+0000 7f44a000eb80 1 -- 172.21.15.119:0/2553260916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56294078a150 msgr2=0x56294078f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.823+0000 7f44a000eb80 1 --2- 172.21.15.119:0/2553260916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56294078a150 0x56294078f670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f447c012980 tx=0x7f447c012c90 comp rx=0 tx=0).stop 2023-12-07T10:23:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.823+0000 7f44a000eb80 1 -- 172.21.15.119:0/2553260916 shutdown_connections 2023-12-07T10:23:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.823+0000 7f44a000eb80 1 --2- 172.21.15.119:0/2553260916 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56294079f150 0x5629407a1570 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.823+0000 7f44a000eb80 1 --2- 172.21.15.119:0/2553260916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56294078a150 0x56294078f670 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.823+0000 7f446dffb700 0 client.26615 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.823+0000 7f44a000eb80 1 --2- 172.21.15.119:0/2553260916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562940785b40 0x56294078ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.823+0000 7f44a000eb80 1 --2- 172.21.15.119:0/2553260916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562940786460 0x56294078f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.823+0000 7f44a000eb80 1 -- 172.21.15.119:0/2553260916 >> 172.21.15.119:0/2553260916 conn(0x5629406b4360 msgr2=0x56294077ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:04.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.823+0000 7f44a000eb80 1 -- 172.21.15.119:0/2553260916 shutdown_connections 2023-12-07T10:23:04.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.823+0000 7f44a000eb80 1 -- 172.21.15.119:0/2553260916 wait complete. 2023-12-07T10:23:04.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.823+0000 7f44a000eb80 20 client.26615 trim_cache size 0 max 16384 2023-12-07T10:23:04.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.843+0000 7f44a000eb80 1 Processor -- start 2023-12-07T10:23:04.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.843+0000 7f44a000eb80 1 -- start start 2023-12-07T10:23:04.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.843+0000 7f44a000eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562940785b40 0x562940785f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.843+0000 7f44a000eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562940786460 0x56294078f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.843+0000 7f44a000eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56294078a150 0x562940796000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.843+0000 7f44a000eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562940558920 con 0x562940785b40 2023-12-07T10:23:04.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.843+0000 7f44a000eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562940790290 con 0x56294078a150 2023-12-07T10:23:04.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.843+0000 7f44a000eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562940791d70 con 0x562940786460 2023-12-07T10:23:04.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.843+0000 7f448a39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562940785b40 0x562940785f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.843+0000 7f448ab9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56294078a150 0x562940796000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.843+0000 7f448a39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562940785b40 0x562940785f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58998/0 (socket says 172.21.15.119:58998) 2023-12-07T10:23:04.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.843+0000 7f448ab9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56294078a150 0x562940796000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:58902/0 (socket says 172.21.15.119:58902) 2023-12-07T10:23:04.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.843+0000 7f4489b9a700 1 --2- 172.21.15.119:0/2894049382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562940786460 0x56294078f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.843+0000 7f448a39b700 1 -- 172.21.15.119:0/2894049382 learned_addr learned my addr 172.21.15.119:0/2894049382 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:04.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.844+0000 7f448ab9c700 1 -- 172.21.15.119:0/2894049382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562940786460 msgr2=0x56294078f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:04.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.844+0000 7f448ab9c700 1 --2- 172.21.15.119:0/2894049382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562940786460 0x56294078f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.844+0000 7f448ab9c700 1 -- 172.21.15.119:0/2894049382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562940785b40 msgr2=0x562940785f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:04.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.844+0000 7f448ab9c700 1 --2- 172.21.15.119:0/2894049382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562940785b40 0x562940785f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.844+0000 7f448ab9c700 1 -- 172.21.15.119:0/2894049382 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562940502de0 con 0x56294078a150 2023-12-07T10:23:04.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.844+0000 7f448ab9c700 1 --2- 172.21.15.119:0/2894049382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56294078a150 0x562940796000 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f447c00fa30 tx=0x7f447c00ff80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=629fff3c61dd28c0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:04.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.844+0000 7f44793ff700 1 -- 172.21.15.119:0/2894049382 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c017040 con 0x56294078a150 2023-12-07T10:23:04.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.844+0000 7f44793ff700 1 -- 172.21.15.119:0/2894049382 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f447c00f5f0 con 0x56294078a150 2023-12-07T10:23:04.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.844+0000 7f44793ff700 1 -- 172.21.15.119:0/2894049382 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c003bc0 con 0x56294078a150 2023-12-07T10:23:04.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.845+0000 7f44a000eb80 1 -- 172.21.15.119:0/2894049382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56294078a150 msgr2=0x562940796000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:04.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.845+0000 7f44a000eb80 1 --2- 172.21.15.119:0/2894049382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56294078a150 0x562940796000 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f447c00fa30 tx=0x7f447c00ff80 comp rx=0 tx=0).stop 2023-12-07T10:23:04.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.845+0000 7f44a000eb80 1 -- 172.21.15.119:0/2894049382 shutdown_connections 2023-12-07T10:23:04.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.845+0000 7f44a000eb80 1 --2- 172.21.15.119:0/2894049382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56294078a150 0x562940796000 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.845+0000 7f44a000eb80 1 --2- 172.21.15.119:0/2894049382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562940785b40 0x562940785f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.845+0000 7f44a000eb80 1 --2- 172.21.15.119:0/2894049382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562940786460 0x56294078f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.845+0000 7f44a000eb80 1 -- 172.21.15.119:0/2894049382 >> 172.21.15.119:0/2894049382 conn(0x5629406b4360 msgr2=0x56294078a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:04.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.845+0000 7f44a000eb80 1 -- 172.21.15.119:0/2894049382 shutdown_connections 2023-12-07T10:23:04.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.845+0000 7f44a000eb80 1 -- 172.21.15.119:0/2894049382 wait complete. 2023-12-07T10:23:04.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.846+0000 7f44a000eb80 1 -- start start 2023-12-07T10:23:04.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.846+0000 7f44a000eb80 1 Processor -- start 2023-12-07T10:23:04.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.847+0000 7f44a000eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562940785b40 0x562940796500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.847+0000 7f44a000eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562940786460 0x56294078c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.847+0000 7f44a000eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56294078a150 0x56294078af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.847+0000 7f44a000eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294078cb60 con 0x562940785b40 2023-12-07T10:23:04.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.847+0000 7f44a000eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294078ccd0 con 0x56294078a150 2023-12-07T10:23:04.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.847+0000 7f44a000eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294078b4c0 con 0x562940786460 2023-12-07T10:23:04.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.847+0000 7f44a000eb80 10 client.0 _ll_register_callbacks cb 0x56294067a940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:04.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.847+0000 7f448a39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562940785b40 0x562940796500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.847+0000 7f4489b9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562940786460 0x56294078c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.847+0000 7f4489b9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562940786460 0x56294078c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:35076/0 (socket says 172.21.15.119:35076) 2023-12-07T10:23:04.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.847+0000 7f4489b9a700 1 -- 172.21.15.119:0/3007265156 learned_addr learned my addr 172.21.15.119:0/3007265156 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:04.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.847+0000 7f448a39b700 1 --2- 172.21.15.119:0/3007265156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562940785b40 0x562940796500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59008/0 (socket says 172.21.15.119:59008) 2023-12-07T10:23:04.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.847+0000 7f448ab9c700 1 --2- 172.21.15.119:0/3007265156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56294078a150 0x56294078af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.847+0000 7f4489b9a700 1 -- 172.21.15.119:0/3007265156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56294078a150 msgr2=0x56294078af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:04.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.847+0000 7f4489b9a700 1 --2- 172.21.15.119:0/3007265156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56294078a150 0x56294078af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.847+0000 7f4489b9a700 1 -- 172.21.15.119:0/3007265156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562940785b40 msgr2=0x562940796500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:04.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.847+0000 7f4489b9a700 1 --2- 172.21.15.119:0/3007265156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562940785b40 0x562940796500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.847+0000 7f4489b9a700 1 -- 172.21.15.119:0/3007265156 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44841a6c10 con 0x562940786460 2023-12-07T10:23:04.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.848+0000 7f4489b9a700 1 --2- 172.21.15.119:0/3007265156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562940786460 0x56294078c290 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f44740097c0 tx=0x7f44740047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77839a2ae3f19c9 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:04.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.848+0000 7f446dffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:23:04.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.848+0000 7f446dffb700 1 -- 172.21.15.119:0/3007265156 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4474028040 con 0x562940786460 2023-12-07T10:23:04.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.848+0000 7f44a000eb80 1 -- 172.21.15.119:0/3007265156 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44800bb0e0 con 0x562940786460 2023-12-07T10:23:04.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.848+0000 7f446dffb700 1 -- 172.21.15.119:0/3007265156 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f447400e8d0 con 0x562940786460 2023-12-07T10:23:04.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.849+0000 7f446dffb700 1 -- 172.21.15.119:0/3007265156 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f447401c070 con 0x562940786460 2023-12-07T10:23:04.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.849+0000 7f44a000eb80 10 client.26599 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:04.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.849+0000 7f44a000eb80 1 -- 172.21.15.119:0/3007265156 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5629404ebc60 con 0x562940786460 2023-12-07T10:23:04.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.849+0000 7f446dffb700 1 -- 172.21.15.119:0/3007265156 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44800bb0e0 con 0x562940786460 2023-12-07T10:23:04.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.849+0000 7f446dffb700 1 -- 172.21.15.119:0/3007265156 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f4474022410 con 0x562940786460 2023-12-07T10:23:04.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.849+0000 7f44a000eb80 10 client.26599 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:04.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.849+0000 7f44a000eb80 10 client.26599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:23:04.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.849+0000 7f44a000eb80 20 client.26599 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:04.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.849+0000 7f44a000eb80 1 --2- 172.21.15.119:0/3007265156 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5629407f6540 0x5629407f8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:04.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.849+0000 7f44a000eb80 4 client.26599 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-12-07T10:23:04.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.850+0000 7f44a000eb80 1 -- 172.21.15.119:0/3007265156 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56294069ffd0 con 0x5629407f6540 2023-12-07T10:23:04.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.850+0000 7f448a39b700 1 --2- 172.21.15.119:0/3007265156 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5629407f6540 0x5629407f8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:04.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.850+0000 7f448a39b700 1 --2- 172.21.15.119:0/3007265156 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5629407f6540 0x5629407f8960 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8f437a8cf160ab3 server_cookie=f2c644e0a292958e in_seq=0 out_seq=0 2023-12-07T10:23:04.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.850+0000 7f446dffb700 10 client.26599 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:04.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.850+0000 7f446dffb700 1 -- 172.21.15.119:0/3007265156 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56294069ffd0 con 0x5629407f6540 2023-12-07T10:23:04.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.850+0000 7f446dffb700 10 client.26599 handle_command_reply: tid=0 2023-12-07T10:23:04.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.850+0000 7f44a000eb80 1 client.26599 shutdown 2023-12-07T10:23:04.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.851+0000 7f44a000eb80 1 -- 172.21.15.119:0/3007265156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562940786460 msgr2=0x56294078c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:04.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.851+0000 7f44a000eb80 1 --2- 172.21.15.119:0/3007265156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562940786460 0x56294078c290 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f44740097c0 tx=0x7f44740047c0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.851+0000 7f44a000eb80 1 -- 172.21.15.119:0/3007265156 shutdown_connections 2023-12-07T10:23:04.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.851+0000 7f44a000eb80 1 --2- 172.21.15.119:0/3007265156 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5629407f6540 0x5629407f8960 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.851+0000 7f44a000eb80 1 --2- 172.21.15.119:0/3007265156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56294078a150 0x56294078af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.851+0000 7f446dffb700 0 client.26599 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:04.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.851+0000 7f44a000eb80 1 --2- 172.21.15.119:0/3007265156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562940785b40 0x562940796500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.851+0000 7f44a000eb80 1 --2- 172.21.15.119:0/3007265156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562940786460 0x56294078c290 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.851+0000 7f44a000eb80 1 -- 172.21.15.119:0/3007265156 >> 172.21.15.119:0/3007265156 conn(0x5629406b4360 msgr2=0x56294077ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:04.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.851+0000 7f44a000eb80 1 -- 172.21.15.119:0/3007265156 shutdown_connections 2023-12-07T10:23:04.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.851+0000 7f44a000eb80 1 -- 172.21.15.119:0/3007265156 wait complete. 2023-12-07T10:23:04.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.851+0000 7f44a000eb80 20 client.26599 trim_cache size 0 max 16384 2023-12-07T10:23:04.858 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:23:04.858 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-07T10:23:04.858 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": { 2023-12-07T10:23:04.858 INFO:teuthology.orchestra.run.smithi119.stdout: "8087ef31-45b6-4e4d-b58b-cce61b2af86d": { 2023-12-07T10:23:04.858 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/", 2023-12-07T10:23:04.858 INFO:teuthology.orchestra.run.smithi119.stdout: "tag": "8087ef31-45b6-4e4d-b58b-cce61b2af86d", 2023-12-07T10:23:04.858 INFO:teuthology.orchestra.run.smithi119.stdout: "options": "recursive,force" 2023-12-07T10:23:04.858 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:23:04.858 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:23:04.859 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:23:04.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.852+0000 7f448c5ff700 1 -- 172.21.15.119:0/3100065044 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f44700a1c80 msgr2=0x7f44700a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:04.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.852+0000 7f448c5ff700 1 --2- 172.21.15.119:0/3100065044 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f44700a1c80 0x7f44700a4140 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f448419b3f0 tx=0x7f4474009260 comp rx=0 tx=0).stop 2023-12-07T10:23:04.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.852+0000 7f448c5ff700 1 -- 172.21.15.119:0/3100065044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4484100710 msgr2=0x7f4484199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:04.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.852+0000 7f448c5ff700 1 --2- 172.21.15.119:0/3100065044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4484100710 0x7f4484199ea0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f448000b780 tx=0x7f4480007940 comp rx=0 tx=0).stop 2023-12-07T10:23:04.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.852+0000 7f448c5ff700 1 -- 172.21.15.119:0/3100065044 shutdown_connections 2023-12-07T10:23:04.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.852+0000 7f448c5ff700 1 --2- 172.21.15.119:0/3100065044 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f44700a1c80 0x7f44700a4140 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.852+0000 7f448c5ff700 1 --2- 172.21.15.119:0/3100065044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44841023d0 0x7f448419e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.852+0000 7f448c5ff700 1 --2- 172.21.15.119:0/3100065044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4484100710 0x7f4484199ea0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.852+0000 7f448c5ff700 1 --2- 172.21.15.119:0/3100065044 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44841010d0 0x7f448419a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:04.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.852+0000 7f448c5ff700 1 -- 172.21.15.119:0/3100065044 >> 172.21.15.119:0/3100065044 conn(0x7f4484078390 msgr2=0x7f44840fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:04.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.852+0000 7f448c5ff700 1 -- 172.21.15.119:0/3100065044 shutdown_connections 2023-12-07T10:23:04.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:04.852+0000 7f448c5ff700 1 -- 172.21.15.119:0/3100065044 wait complete. 2023-12-07T10:23:04.872 INFO:tasks.cephfs.filesystem:scrub status for tag:8087ef31-45b6-4e4d-b58b-cce61b2af86d - {'path': '/', 'tag': '8087ef31-45b6-4e4d-b58b-cce61b2af86d', 'options': 'recursive,force'} 2023-12-07T10:23:05.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:04 smithi119 ceph-mon[97908]: scrub queued for path: / 2023-12-07T10:23:05.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:04 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:05.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:04 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:23:05.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:04 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:23:05.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:04 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:05.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:04 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:05.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:04 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:23:05.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:04 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:05.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:04 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:23:05.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:04 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:05.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:04 smithi159 ceph-mon[134726]: scrub queued for path: / 2023-12-07T10:23:05.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:04 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:05.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:04 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:23:05.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:04 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:23:05.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:04 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:05.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:04 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:05.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:04 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:23:05.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:04 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:05.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:04 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:23:05.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:04 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:05.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:04 smithi153 ceph-mon[133625]: scrub queued for path: / 2023-12-07T10:23:05.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:04 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:05.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:04 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:23:05.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:04 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:23:05.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:04 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:05.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:04 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:05.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:04 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:23:05.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:04 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:05.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:04 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:23:05.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:04 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:05.548 INFO:tasks.workunit.client.0.smithi119.stdout: 1 120955 8.60 MB/sec warmup 78 sec latency 54.925 ms 2023-12-07T10:23:06.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:05 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:23:06.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:05 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:23:06.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:05 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:23:06.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:05 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:06.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:05 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:06.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:05 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:23:06.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:05 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:06.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:05 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:06.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:05 smithi159 ceph-mon[134726]: pgmap v83: 129 pgs: 129 active+clean; 78 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 605 KiB/s wr, 106 op/s 2023-12-07T10:23:06.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:05 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:23:06.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:05 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:23:06.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:05 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:23:06.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:05 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:06.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:05 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:06.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:05 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:23:06.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:05 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:06.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:05 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:06.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:05 smithi153 ceph-mon[133625]: pgmap v83: 129 pgs: 129 active+clean; 78 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 605 KiB/s wr, 106 op/s 2023-12-07T10:23:06.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:05 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:23:06.556 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:05 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:23:06.556 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:05 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:23:06.556 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:05 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:06.556 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:05 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:06.556 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:05 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:23:06.556 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:05 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:06.557 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:05 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:06.557 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:05 smithi119 ceph-mon[97908]: pgmap v83: 129 pgs: 129 active+clean; 78 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 605 KiB/s wr, 106 op/s 2023-12-07T10:23:06.557 INFO:tasks.workunit.client.0.smithi119.stdout: 1 122319 8.56 MB/sec warmup 79 sec latency 19.884 ms 2023-12-07T10:23:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.702+0000 7f5cda3ff700 1 -- 172.21.15.119:0/3386366600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cd4101810 msgr2=0x7f5cd410da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.702+0000 7f5cda3ff700 1 --2- 172.21.15.119:0/3386366600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cd4101810 0x7f5cd410da20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5cc80098f0 tx=0x7f5cc8009c00 comp rx=0 tx=0).stop 2023-12-07T10:23:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.702+0000 7f5cda3ff700 1 -- 172.21.15.119:0/3386366600 shutdown_connections 2023-12-07T10:23:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.702+0000 7f5cda3ff700 1 --2- 172.21.15.119:0/3386366600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5cd410df60 0x7f5cd4110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.702+0000 7f5cda3ff700 1 --2- 172.21.15.119:0/3386366600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cd4100ef0 0x7f5cd41012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.702+0000 7f5cda3ff700 1 --2- 172.21.15.119:0/3386366600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cd4101810 0x7f5cd410da20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.702+0000 7f5cda3ff700 1 -- 172.21.15.119:0/3386366600 >> 172.21.15.119:0/3386366600 conn(0x7f5cd40775b0 msgr2=0x7f5cd40779c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.702+0000 7f5cda3ff700 1 -- 172.21.15.119:0/3386366600 shutdown_connections 2023-12-07T10:23:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.702+0000 7f5cda3ff700 1 -- 172.21.15.119:0/3386366600 wait complete. 2023-12-07T10:23:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.703+0000 7f5cda3ff700 1 Processor -- start 2023-12-07T10:23:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.703+0000 7f5cda3ff700 1 -- start start 2023-12-07T10:23:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.703+0000 7f5cda3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cd4100ef0 0x7f5cd419f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.703+0000 7f5cda3ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cd4101810 0x7f5cd419fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.703+0000 7f5cda3ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5cd410df60 0x7f5cd4199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.703+0000 7f5cda3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd419a130 con 0x7f5cd4100ef0 2023-12-07T10:23:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.703+0000 7f5cda3ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd419a2a0 con 0x7f5cd4101810 2023-12-07T10:23:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.703+0000 7f5cda3ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd419a460 con 0x7f5cd410df60 2023-12-07T10:23:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.704+0000 7f5cd3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cd4100ef0 0x7f5cd419f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.704+0000 7f5cd3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cd4100ef0 0x7f5cd419f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59020/0 (socket says 172.21.15.119:59020) 2023-12-07T10:23:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.704+0000 7f5cd3fff700 1 -- 172.21.15.119:0/708384578 learned_addr learned my addr 172.21.15.119:0/708384578 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.704+0000 7f5cd899c700 1 --2- 172.21.15.119:0/708384578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5cd410df60 0x7f5cd4199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.704+0000 7f5cd37fe700 1 --2- 172.21.15.119:0/708384578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cd4101810 0x7f5cd419fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.704+0000 7f5cd899c700 1 -- 172.21.15.119:0/708384578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cd4101810 msgr2=0x7f5cd419fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.704+0000 7f5cd899c700 1 --2- 172.21.15.119:0/708384578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cd4101810 0x7f5cd419fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.704+0000 7f5cd899c700 1 -- 172.21.15.119:0/708384578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cd4100ef0 msgr2=0x7f5cd419f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.704+0000 7f5cd899c700 1 --2- 172.21.15.119:0/708384578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cd4100ef0 0x7f5cd419f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.704+0000 7f5cd899c700 1 -- 172.21.15.119:0/708384578 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc8009520 con 0x7f5cd410df60 2023-12-07T10:23:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.704+0000 7f5cd899c700 1 --2- 172.21.15.119:0/708384578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5cd410df60 0x7f5cd4199ad0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00ef90 tx=0x7f5ccc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.705+0000 7f5cd17fa700 1 -- 172.21.15.119:0/708384578 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc018070 con 0x7f5cd410df60 2023-12-07T10:23:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.705+0000 7f5cda3ff700 1 -- 172.21.15.119:0/708384578 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd419a6e0 con 0x7f5cd410df60 2023-12-07T10:23:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.705+0000 7f5cd17fa700 1 -- 172.21.15.119:0/708384578 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ccc00f040 con 0x7f5cd410df60 2023-12-07T10:23:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.705+0000 7f5cd17fa700 1 -- 172.21.15.119:0/708384578 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc00cc20 con 0x7f5cd410df60 2023-12-07T10:23:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.705+0000 7f5cda3ff700 1 -- 172.21.15.119:0/708384578 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd41a6990 con 0x7f5cd410df60 2023-12-07T10:23:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.708+0000 7f5cd17fa700 1 -- 172.21.15.119:0/708384578 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f5ccc003a00 con 0x7f5cd410df60 2023-12-07T10:23:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.709+0000 7f5cedd2ab80 1 Processor -- start 2023-12-07T10:23:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.709+0000 7f5cd17fa700 1 --2- 172.21.15.119:0/708384578 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5cbc0a1c80 0x7f5cbc0a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.709+0000 7f5cedd2ab80 1 -- start start 2023-12-07T10:23:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.710+0000 7f5cd17fa700 1 -- 172.21.15.119:0/708384578 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f5ccc0cd7f0 con 0x7f5cd410df60 2023-12-07T10:23:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.710+0000 7f5cedd2ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559d9141a60 0x5559d9141e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.710+0000 7f5cd37fe700 1 --2- 172.21.15.119:0/708384578 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5cbc0a1c80 0x7f5cbc0a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.710+0000 7f5cedd2ab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559d9142380 0x5559d913c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.710+0000 7f5cedd2ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559d913cad0 0x5559d913cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.710+0000 7f5cedd2ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559d90b0760 con 0x5559d913cad0 2023-12-07T10:23:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.710+0000 7f5cedd2ab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559d9097740 con 0x5559d9142380 2023-12-07T10:23:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.710+0000 7f5cedd2ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559d9095060 con 0x5559d9141a60 2023-12-07T10:23:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.710+0000 7f5cd3fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559d9141a60 0x5559d9141e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.710+0000 7f5cd37fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559d9142380 0x5559d913c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.710+0000 7f5cd3fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559d9141a60 0x5559d9141e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:35110/0 (socket says 172.21.15.119:35110) 2023-12-07T10:23:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.710+0000 7f5cd3fff700 1 -- 172.21.15.119:0/426464601 learned_addr learned my addr 172.21.15.119:0/426464601 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.710+0000 7f5cd899c700 1 --2- 172.21.15.119:0/426464601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559d913cad0 0x5559d913cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.710+0000 7f5cd3fff700 1 -- 172.21.15.119:0/426464601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559d9142380 msgr2=0x5559d913c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.710+0000 7f5cd3fff700 1 --2- 172.21.15.119:0/426464601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559d9142380 0x5559d913c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.710+0000 7f5cd3fff700 1 -- 172.21.15.119:0/426464601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559d913cad0 msgr2=0x5559d913cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.710+0000 7f5cd3fff700 1 --2- 172.21.15.119:0/426464601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559d913cad0 0x5559d913cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.710+0000 7f5cd3fff700 1 -- 172.21.15.119:0/426464601 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559d8f0d5d0 con 0x5559d9141a60 2023-12-07T10:23:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.711+0000 7f5cd37fe700 1 --2- 172.21.15.119:0/708384578 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5cbc0a1c80 0x7f5cbc0a4140 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8005e90 tx=0x7f5cc8005dc0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.711+0000 7f5cd3fff700 1 --2- 172.21.15.119:0/426464601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559d9141a60 0x5559d9141e40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009e50 tx=0x7f5cc4011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f722b99035088c8 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.711+0000 7f5cba7fc700 1 -- 172.21.15.119:0/426464601 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4004a30 con 0x5559d9141a60 2023-12-07T10:23:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.711+0000 7f5cba7fc700 1 -- 172.21.15.119:0/426464601 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cc40054e0 con 0x5559d9141a60 2023-12-07T10:23:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.711+0000 7f5cba7fc700 1 -- 172.21.15.119:0/426464601 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4019860 con 0x5559d9141a60 2023-12-07T10:23:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.712+0000 7f5cedd2ab80 1 -- 172.21.15.119:0/426464601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559d9141a60 msgr2=0x5559d9141e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.712+0000 7f5cedd2ab80 1 --2- 172.21.15.119:0/426464601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559d9141a60 0x5559d9141e40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009e50 tx=0x7f5cc4011720 comp rx=0 tx=0).stop 2023-12-07T10:23:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.712+0000 7f5cedd2ab80 1 -- 172.21.15.119:0/426464601 shutdown_connections 2023-12-07T10:23:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.712+0000 7f5cedd2ab80 1 --2- 172.21.15.119:0/426464601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559d913cad0 0x5559d913cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.712+0000 7f5cedd2ab80 1 --2- 172.21.15.119:0/426464601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559d9141a60 0x5559d9141e40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.712+0000 7f5cedd2ab80 1 --2- 172.21.15.119:0/426464601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559d9142380 0x5559d913c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.712+0000 7f5cedd2ab80 1 -- 172.21.15.119:0/426464601 >> 172.21.15.119:0/426464601 conn(0x5559d906b300 msgr2=0x5559d9136b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.712+0000 7f5cedd2ab80 1 -- 172.21.15.119:0/426464601 shutdown_connections 2023-12-07T10:23:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.712+0000 7f5cedd2ab80 1 -- 172.21.15.119:0/426464601 wait complete. 2023-12-07T10:23:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.713+0000 7f5cedd2ab80 1 -- start start 2023-12-07T10:23:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.713+0000 7f5cedd2ab80 1 Processor -- start 2023-12-07T10:23:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.714+0000 7f5cedd2ab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559d913cad0 0x5559d9145740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.714+0000 7f5cedd2ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559d9141a60 0x5559d9145c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.714+0000 7f5cedd2ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559d9142380 0x5559d91461c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.714+0000 7f5cedd2ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559d9090950 con 0x5559d9141a60 2023-12-07T10:23:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.714+0000 7f5cedd2ab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559d904a500 con 0x5559d913cad0 2023-12-07T10:23:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.714+0000 7f5cedd2ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559d904a1a0 con 0x5559d9142380 2023-12-07T10:23:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.714+0000 7f5cedd2ab80 10 client.0 _ll_register_callbacks cb 0x5559d90318e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.714+0000 7f5cd3fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559d913cad0 0x5559d9145740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.714+0000 7f5cd3fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559d913cad0 0x5559d9145740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:58946/0 (socket says 172.21.15.119:58946) 2023-12-07T10:23:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.714+0000 7f5cd3fff700 1 -- 172.21.15.119:0/3146825221 learned_addr learned my addr 172.21.15.119:0/3146825221 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.714+0000 7f5cd37fe700 1 --2- 172.21.15.119:0/3146825221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559d9141a60 0x5559d9145c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.714+0000 7f5cd899c700 1 --2- 172.21.15.119:0/3146825221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559d9142380 0x5559d91461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.715+0000 7f5cd37fe700 1 -- 172.21.15.119:0/3146825221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559d9142380 msgr2=0x5559d91461c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.715+0000 7f5cd37fe700 1 --2- 172.21.15.119:0/3146825221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559d9142380 0x5559d91461c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.715+0000 7f5cd37fe700 1 -- 172.21.15.119:0/3146825221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559d913cad0 msgr2=0x5559d9145740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.715+0000 7f5cd37fe700 1 --2- 172.21.15.119:0/3146825221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559d913cad0 0x5559d9145740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.715+0000 7f5cd37fe700 1 -- 172.21.15.119:0/3146825221 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc4009a80 con 0x5559d9141a60 2023-12-07T10:23:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.715+0000 7f5cd899c700 1 --2- 172.21.15.119:0/3146825221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559d9142380 0x5559d91461c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.715+0000 7f5cd37fe700 1 --2- 172.21.15.119:0/3146825221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559d9141a60 0x5559d9145c80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8000c00 tx=0x7f5cc8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c6c5959094045b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.715+0000 7f5cb37fe700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:23:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.715+0000 7f5cb37fe700 1 -- 172.21.15.119:0/3146825221 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8005720 con 0x5559d9141a60 2023-12-07T10:23:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.715+0000 7f5cb37fe700 1 -- 172.21.15.119:0/3146825221 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f5cc800f910 con 0x5559d9141a60 2023-12-07T10:23:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.715+0000 7f5cedd2ab80 1 -- 172.21.15.119:0/3146825221 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5559d912f6f0 con 0x5559d9141a60 2023-12-07T10:23:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.716+0000 7f5cb37fe700 1 -- 172.21.15.119:0/3146825221 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cc800fe30 con 0x5559d9141a60 2023-12-07T10:23:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.716+0000 7f5cedd2ab80 10 client.16608 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.716+0000 7f5cedd2ab80 1 -- 172.21.15.119:0/3146825221 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5559d8eb9d80 con 0x5559d9141a60 2023-12-07T10:23:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.716+0000 7f5cb37fe700 1 -- 172.21.15.119:0/3146825221 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5559d912f6f0 con 0x5559d9141a60 2023-12-07T10:23:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.716+0000 7f5cb37fe700 1 -- 172.21.15.119:0/3146825221 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f5cc802e070 con 0x5559d9141a60 2023-12-07T10:23:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.716+0000 7f5cedd2ab80 10 client.16608 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.716+0000 7f5cedd2ab80 10 client.16608 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:23:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.716+0000 7f5cedd2ab80 20 client.16608 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.716+0000 7f5cedd2ab80 1 --2- 172.21.15.119:0/3146825221 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5559d9155ed0 0x5559d91582f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.716+0000 7f5cedd2ab80 4 client.16608 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:23:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.716+0000 7f5cedd2ab80 1 -- 172.21.15.119:0/3146825221 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559d910b740 con 0x5559d9155ed0 2023-12-07T10:23:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.717+0000 7f5cd3fff700 1 --2- 172.21.15.119:0/3146825221 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5559d9155ed0 0x5559d91582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.717+0000 7f5cd3fff700 1 --2- 172.21.15.119:0/3146825221 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5559d9155ed0 0x5559d91582f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aed1dcedecba674f server_cookie=783601ece29a990f in_seq=0 out_seq=0 2023-12-07T10:23:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.717+0000 7f5cb37fe700 10 client.16608 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.718+0000 7f5cb37fe700 1 -- 172.21.15.119:0/3146825221 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5559d910b740 con 0x5559d9155ed0 2023-12-07T10:23:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.718+0000 7f5cb37fe700 10 client.16608 handle_command_reply: tid=0 2023-12-07T10:23:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.718+0000 7f5cedd2ab80 1 client.16608 shutdown 2023-12-07T10:23:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.718+0000 7f5cedd2ab80 1 -- 172.21.15.119:0/3146825221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559d9141a60 msgr2=0x5559d9145c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.718+0000 7f5cedd2ab80 1 --2- 172.21.15.119:0/3146825221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559d9141a60 0x5559d9145c80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8000c00 tx=0x7f5cc8009c00 comp rx=0 tx=0).stop 2023-12-07T10:23:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.718+0000 7f5cedd2ab80 1 -- 172.21.15.119:0/3146825221 shutdown_connections 2023-12-07T10:23:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.718+0000 7f5cedd2ab80 1 --2- 172.21.15.119:0/3146825221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559d9142380 0x5559d91461c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.718+0000 7f5cedd2ab80 1 --2- 172.21.15.119:0/3146825221 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5559d9155ed0 0x5559d91582f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.718+0000 7f5cedd2ab80 1 --2- 172.21.15.119:0/3146825221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559d913cad0 0x5559d9145740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.718+0000 7f5cedd2ab80 1 --2- 172.21.15.119:0/3146825221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559d9141a60 0x5559d9145c80 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.718+0000 7f5cb37fe700 0 client.16608 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.718+0000 7f5cedd2ab80 1 -- 172.21.15.119:0/3146825221 >> 172.21.15.119:0/3146825221 conn(0x5559d906b300 msgr2=0x5559d9131980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.718+0000 7f5cedd2ab80 1 -- 172.21.15.119:0/3146825221 shutdown_connections 2023-12-07T10:23:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.718+0000 7f5cedd2ab80 1 -- 172.21.15.119:0/3146825221 wait complete. 2023-12-07T10:23:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.719+0000 7f5cedd2ab80 20 client.16608 trim_cache size 0 max 16384 2023-12-07T10:23:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.736+0000 7f5cedd2ab80 1 Processor -- start 2023-12-07T10:23:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.736+0000 7f5cedd2ab80 1 -- start start 2023-12-07T10:23:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.736+0000 7f5cedd2ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559d913cad0 0x5559d9145fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.736+0000 7f5cedd2ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559d9141a60 0x5559d914cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.737+0000 7f5cedd2ab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559d9142380 0x5559d914d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.737+0000 7f5cedd2ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559d8f0f8c0 con 0x5559d9141a60 2023-12-07T10:23:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.737+0000 7f5cedd2ab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559d91416e0 con 0x5559d9142380 2023-12-07T10:23:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.737+0000 7f5cedd2ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559d9146d40 con 0x5559d913cad0 2023-12-07T10:23:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.737+0000 7f5cd3fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559d913cad0 0x5559d9145fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.737+0000 7f5cd3fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559d913cad0 0x5559d9145fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:35130/0 (socket says 172.21.15.119:35130) 2023-12-07T10:23:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.737+0000 7f5cd3fff700 1 -- 172.21.15.119:0/4161603786 learned_addr learned my addr 172.21.15.119:0/4161603786 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.737+0000 7f5cd37fe700 1 --2- 172.21.15.119:0/4161603786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559d9141a60 0x5559d914cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.737+0000 7f5cd899c700 1 --2- 172.21.15.119:0/4161603786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559d9142380 0x5559d914d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.737+0000 7f5cd3fff700 1 -- 172.21.15.119:0/4161603786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559d9142380 msgr2=0x5559d914d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.737+0000 7f5cd3fff700 1 --2- 172.21.15.119:0/4161603786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559d9142380 0x5559d914d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.737+0000 7f5cd3fff700 1 -- 172.21.15.119:0/4161603786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559d9141a60 msgr2=0x5559d914cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.737+0000 7f5cd3fff700 1 --2- 172.21.15.119:0/4161603786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559d9141a60 0x5559d914cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.737+0000 7f5cd3fff700 1 -- 172.21.15.119:0/4161603786 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559d8ea2c00 con 0x5559d913cad0 2023-12-07T10:23:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.737+0000 7f5cd37fe700 1 --2- 172.21.15.119:0/4161603786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559d9141a60 0x5559d914cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:23:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.737+0000 7f5cd3fff700 1 --2- 172.21.15.119:0/4161603786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559d913cad0 0x5559d9145fe0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4005fa0 tx=0x7f5cc4011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc258ba299529ae5 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.738+0000 7f5cbaffd700 1 -- 172.21.15.119:0/4161603786 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4005ce0 con 0x5559d913cad0 2023-12-07T10:23:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.738+0000 7f5cbaffd700 1 -- 172.21.15.119:0/4161603786 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cc402a2c0 con 0x5559d913cad0 2023-12-07T10:23:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.738+0000 7f5cbaffd700 1 -- 172.21.15.119:0/4161603786 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc40038f0 con 0x5559d913cad0 2023-12-07T10:23:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.738+0000 7f5cedd2ab80 1 -- 172.21.15.119:0/4161603786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559d913cad0 msgr2=0x5559d9145fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.738+0000 7f5cedd2ab80 1 --2- 172.21.15.119:0/4161603786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559d913cad0 0x5559d9145fe0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4005fa0 tx=0x7f5cc4011720 comp rx=0 tx=0).stop 2023-12-07T10:23:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.738+0000 7f5cedd2ab80 1 -- 172.21.15.119:0/4161603786 shutdown_connections 2023-12-07T10:23:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.738+0000 7f5cedd2ab80 1 --2- 172.21.15.119:0/4161603786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559d9142380 0x5559d914d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.738+0000 7f5cedd2ab80 1 --2- 172.21.15.119:0/4161603786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559d913cad0 0x5559d9145fe0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.739+0000 7f5cedd2ab80 1 --2- 172.21.15.119:0/4161603786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559d9141a60 0x5559d914cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.739+0000 7f5cedd2ab80 1 -- 172.21.15.119:0/4161603786 >> 172.21.15.119:0/4161603786 conn(0x5559d906b300 msgr2=0x5559d913f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.739+0000 7f5cedd2ab80 1 -- 172.21.15.119:0/4161603786 shutdown_connections 2023-12-07T10:23:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.739+0000 7f5cedd2ab80 1 -- 172.21.15.119:0/4161603786 wait complete. 2023-12-07T10:23:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.739+0000 7f5cedd2ab80 1 -- start start 2023-12-07T10:23:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.739+0000 7f5cedd2ab80 1 Processor -- start 2023-12-07T10:23:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.740+0000 7f5cedd2ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559d913cad0 0x5559d914a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.740+0000 7f5cedd2ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559d9141a60 0x5559d914ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.740+0000 7f5cedd2ab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559d9142380 0x5559d914b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.740+0000 7f5cedd2ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559d91ab3d0 con 0x5559d9141a60 2023-12-07T10:23:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.740+0000 7f5cedd2ab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559d91ab510 con 0x5559d9142380 2023-12-07T10:23:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.740+0000 7f5cedd2ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559d91ab6a0 con 0x5559d913cad0 2023-12-07T10:23:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.740+0000 7f5cedd2ab80 10 client.0 _ll_register_callbacks cb 0x5559d90318e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.740+0000 7f5cd37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559d9141a60 0x5559d914ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.740+0000 7f5cd37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559d9141a60 0x5559d914ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59058/0 (socket says 172.21.15.119:59058) 2023-12-07T10:23:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.740+0000 7f5cd37fe700 1 -- 172.21.15.119:0/2747410816 learned_addr learned my addr 172.21.15.119:0/2747410816 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.741+0000 7f5cd3fff700 1 --2- 172.21.15.119:0/2747410816 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559d913cad0 0x5559d914a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.741+0000 7f5cd899c700 1 --2- 172.21.15.119:0/2747410816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559d9142380 0x5559d914b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.741+0000 7f5cd37fe700 1 -- 172.21.15.119:0/2747410816 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559d913cad0 msgr2=0x5559d914a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.741+0000 7f5cd37fe700 1 --2- 172.21.15.119:0/2747410816 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559d913cad0 0x5559d914a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.741+0000 7f5cd37fe700 1 -- 172.21.15.119:0/2747410816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559d9142380 msgr2=0x5559d914b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.741+0000 7f5cd37fe700 1 --2- 172.21.15.119:0/2747410816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559d9142380 0x5559d914b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.741+0000 7f5cd37fe700 1 -- 172.21.15.119:0/2747410816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5559d8f0d5d0 con 0x5559d9141a60 2023-12-07T10:23:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.741+0000 7f5cd3fff700 1 --2- 172.21.15.119:0/2747410816 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559d913cad0 0x5559d914a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.741+0000 7f5cd899c700 1 --2- 172.21.15.119:0/2747410816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559d9142380 0x5559d914b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.741+0000 7f5cd37fe700 1 --2- 172.21.15.119:0/2747410816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559d9141a60 0x5559d914ae50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x5559d912f5e0 tx=0x7f5cc800b2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e59b1188bc326d server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.741+0000 7f5cb37fe700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:23:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.742+0000 7f5cb37fe700 1 -- 172.21.15.119:0/2747410816 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8005900 con 0x5559d9141a60 2023-12-07T10:23:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.742+0000 7f5cb37fe700 1 -- 172.21.15.119:0/2747410816 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f5cc800fe30 con 0x5559d9141a60 2023-12-07T10:23:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.742+0000 7f5cedd2ab80 1 -- 172.21.15.119:0/2747410816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5cc40054e0 con 0x5559d9141a60 2023-12-07T10:23:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.742+0000 7f5cb37fe700 1 -- 172.21.15.119:0/2747410816 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cc800f8f0 con 0x5559d9141a60 2023-12-07T10:23:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.742+0000 7f5cedd2ab80 10 client.16614 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.742+0000 7f5cedd2ab80 1 -- 172.21.15.119:0/2747410816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5559d8ee01f0 con 0x5559d9141a60 2023-12-07T10:23:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.742+0000 7f5cb37fe700 1 -- 172.21.15.119:0/2747410816 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cc40054e0 con 0x5559d9141a60 2023-12-07T10:23:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.743+0000 7f5cb37fe700 1 -- 172.21.15.119:0/2747410816 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f5cc8015020 con 0x5559d9141a60 2023-12-07T10:23:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.743+0000 7f5cedd2ab80 10 client.16614 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.743+0000 7f5cedd2ab80 10 client.16614 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:23:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.743+0000 7f5cedd2ab80 20 client.16614 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.743+0000 7f5cedd2ab80 1 --2- 172.21.15.119:0/2747410816 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5559d91acc40 0x5559d91af060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.743+0000 7f5cedd2ab80 4 client.16614 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:23:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.743+0000 7f5cedd2ab80 1 -- 172.21.15.119:0/2747410816 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5559d9056f70 con 0x5559d91acc40 2023-12-07T10:23:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.743+0000 7f5cd3fff700 1 --2- 172.21.15.119:0/2747410816 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5559d91acc40 0x5559d91af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.743+0000 7f5cd3fff700 1 --2- 172.21.15.119:0/2747410816 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5559d91acc40 0x5559d91af060 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a16b4fe892ebc579 server_cookie=7c1e0eff425edbea in_seq=0 out_seq=0 2023-12-07T10:23:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.743+0000 7f5cb37fe700 10 client.16614 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.802+0000 7f5cb37fe700 1 -- 172.21.15.119:0/2747410816 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5559d9056f70 con 0x5559d91acc40 2023-12-07T10:23:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.802+0000 7f5cb37fe700 10 client.16614 handle_command_reply: tid=0 2023-12-07T10:23:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.802+0000 7f5cedd2ab80 1 client.16614 shutdown 2023-12-07T10:23:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.803+0000 7f5cedd2ab80 1 -- 172.21.15.119:0/2747410816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559d9141a60 msgr2=0x5559d914ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.803+0000 7f5cedd2ab80 1 --2- 172.21.15.119:0/2747410816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559d9141a60 0x5559d914ae50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x5559d912f5e0 tx=0x7f5cc800b2c0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.803+0000 7f5cedd2ab80 1 -- 172.21.15.119:0/2747410816 shutdown_connections 2023-12-07T10:23:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.803+0000 7f5cedd2ab80 1 --2- 172.21.15.119:0/2747410816 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5559d91acc40 0x5559d91af060 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.803+0000 7f5cedd2ab80 1 --2- 172.21.15.119:0/2747410816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559d9142380 0x5559d914b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.803+0000 7f5cedd2ab80 1 --2- 172.21.15.119:0/2747410816 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559d913cad0 0x5559d914a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.803+0000 7f5cb37fe700 0 client.16614 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.803+0000 7f5cedd2ab80 1 --2- 172.21.15.119:0/2747410816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559d9141a60 0x5559d914ae50 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.803+0000 7f5cedd2ab80 1 -- 172.21.15.119:0/2747410816 >> 172.21.15.119:0/2747410816 conn(0x5559d906b300 msgr2=0x5559d9146020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.803+0000 7f5cedd2ab80 1 -- 172.21.15.119:0/2747410816 shutdown_connections 2023-12-07T10:23:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.803+0000 7f5cedd2ab80 1 -- 172.21.15.119:0/2747410816 wait complete. 2023-12-07T10:23:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.803+0000 7f5cedd2ab80 20 client.16614 trim_cache size 0 max 16384 2023-12-07T10:23:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:23:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:23:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:23:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:23:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.804+0000 7f5cda3ff700 1 -- 172.21.15.119:0/708384578 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5cbc0a1c80 msgr2=0x7f5cbc0a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.804+0000 7f5cda3ff700 1 --2- 172.21.15.119:0/708384578 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5cbc0a1c80 0x7f5cbc0a4140 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8005e90 tx=0x7f5cc8005dc0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.804+0000 7f5cda3ff700 1 -- 172.21.15.119:0/708384578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5cd410df60 msgr2=0x7f5cd4199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.804+0000 7f5cda3ff700 1 --2- 172.21.15.119:0/708384578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5cd410df60 0x7f5cd4199ad0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00ef90 tx=0x7f5ccc00c3a0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.804+0000 7f5cda3ff700 1 -- 172.21.15.119:0/708384578 shutdown_connections 2023-12-07T10:23:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.804+0000 7f5cda3ff700 1 --2- 172.21.15.119:0/708384578 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5cbc0a1c80 0x7f5cbc0a4140 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.804+0000 7f5cda3ff700 1 --2- 172.21.15.119:0/708384578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5cd410df60 0x7f5cd4199ad0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.804+0000 7f5cda3ff700 1 --2- 172.21.15.119:0/708384578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cd4100ef0 0x7f5cd419f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.804+0000 7f5cda3ff700 1 --2- 172.21.15.119:0/708384578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cd4101810 0x7f5cd419fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.804+0000 7f5cda3ff700 1 -- 172.21.15.119:0/708384578 >> 172.21.15.119:0/708384578 conn(0x7f5cd40775b0 msgr2=0x7f5cd410f820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.804+0000 7f5cda3ff700 1 -- 172.21.15.119:0/708384578 shutdown_connections 2023-12-07T10:23:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:06.804+0000 7f5cda3ff700 1 -- 172.21.15.119:0/708384578 wait complete. 2023-12-07T10:23:07.548 INFO:tasks.workunit.client.0.smithi119.stdout: 1 124457 8.62 MB/sec warmup 80 sec latency 15.934 ms 2023-12-07T10:23:08.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:07 smithi159 ceph-mon[134726]: pgmap v84: 129 pgs: 129 active+clean; 85 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 1.7 MiB/s wr, 219 op/s 2023-12-07T10:23:08.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:07 smithi153 ceph-mon[133625]: pgmap v84: 129 pgs: 129 active+clean; 85 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 1.7 MiB/s wr, 219 op/s 2023-12-07T10:23:08.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:07 smithi119 ceph-mon[97908]: pgmap v84: 129 pgs: 129 active+clean; 85 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 1.7 MiB/s wr, 219 op/s 2023-12-07T10:23:08.548 INFO:tasks.workunit.client.0.smithi119.stdout: 1 125962 8.59 MB/sec warmup 81 sec latency 13.018 ms 2023-12-07T10:23:09.548 INFO:tasks.workunit.client.0.smithi119.stdout: 1 128008 8.65 MB/sec warmup 82 sec latency 11.551 ms 2023-12-07T10:23:10.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:09 smithi159 ceph-mon[134726]: pgmap v85: 129 pgs: 129 active+clean; 85 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2023-12-07T10:23:10.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:09 smithi153 ceph-mon[133625]: pgmap v85: 129 pgs: 129 active+clean; 85 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2023-12-07T10:23:10.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:09 smithi119 ceph-mon[97908]: pgmap v85: 129 pgs: 129 active+clean; 85 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2023-12-07T10:23:10.548 INFO:tasks.workunit.client.0.smithi119.stdout: 1 129152 8.61 MB/sec warmup 83 sec latency 57.919 ms 2023-12-07T10:23:11.548 INFO:tasks.workunit.client.0.smithi119.stdout: 1 131413 8.66 MB/sec warmup 84 sec latency 17.146 ms 2023-12-07T10:23:12.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:11 smithi119 ceph-mon[97908]: pgmap v86: 129 pgs: 129 active+clean; 88 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 328 op/s 2023-12-07T10:23:12.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:11 smithi159 ceph-mon[134726]: pgmap v86: 129 pgs: 129 active+clean; 88 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 328 op/s 2023-12-07T10:23:12.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:11 smithi153 ceph-mon[133625]: pgmap v86: 129 pgs: 129 active+clean; 88 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 328 op/s 2023-12-07T10:23:12.548 INFO:tasks.workunit.client.0.smithi119.stdout: 1 132460 8.59 MB/sec warmup 85 sec latency 12.433 ms 2023-12-07T10:23:13.548 INFO:tasks.workunit.client.0.smithi119.stdout: 1 134784 8.67 MB/sec warmup 86 sec latency 53.512 ms 2023-12-07T10:23:14.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:13 smithi119 ceph-mon[97908]: pgmap v87: 129 pgs: 129 active+clean; 88 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 257 op/s 2023-12-07T10:23:14.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:13 smithi159 ceph-mon[134726]: pgmap v87: 129 pgs: 129 active+clean; 88 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 257 op/s 2023-12-07T10:23:14.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:13 smithi153 ceph-mon[133625]: pgmap v87: 129 pgs: 129 active+clean; 88 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 257 op/s 2023-12-07T10:23:14.548 INFO:tasks.workunit.client.0.smithi119.stdout: 1 135858 8.62 MB/sec warmup 87 sec latency 13.796 ms 2023-12-07T10:23:15.548 INFO:tasks.workunit.client.0.smithi119.stdout: 1 138099 8.66 MB/sec warmup 88 sec latency 47.521 ms 2023-12-07T10:23:16.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:15 smithi119 ceph-mon[97908]: pgmap v88: 129 pgs: 129 active+clean; 88 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 257 op/s 2023-12-07T10:23:16.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:15 smithi159 ceph-mon[134726]: pgmap v88: 129 pgs: 129 active+clean; 88 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 257 op/s 2023-12-07T10:23:16.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:15 smithi153 ceph-mon[133625]: pgmap v88: 129 pgs: 129 active+clean; 88 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 257 op/s 2023-12-07T10:23:16.548 INFO:tasks.workunit.client.0.smithi119.stdout: 1 139090 8.62 MB/sec warmup 89 sec latency 23.572 ms 2023-12-07T10:23:17.548 INFO:tasks.workunit.client.0.smithi119.stdout: 1 141555 8.67 MB/sec warmup 90 sec latency 12.738 ms 2023-12-07T10:23:18.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:17 smithi159 ceph-mon[134726]: pgmap v89: 129 pgs: 129 active+clean; 95 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 376 op/s 2023-12-07T10:23:18.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:17 smithi153 ceph-mon[133625]: pgmap v89: 129 pgs: 129 active+clean; 95 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 376 op/s 2023-12-07T10:23:18.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:17 smithi119 ceph-mon[97908]: pgmap v89: 129 pgs: 129 active+clean; 95 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 376 op/s 2023-12-07T10:23:18.548 INFO:tasks.workunit.client.0.smithi119.stdout: 1 142648 8.64 MB/sec warmup 91 sec latency 20.015 ms 2023-12-07T10:23:19.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:18 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:23:19.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:18 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:23:19.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:18 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:23:19.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:18 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:23:19.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:18 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:23:19.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:18 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:23:19.548 INFO:tasks.workunit.client.0.smithi119.stdout: 1 145083 8.68 MB/sec warmup 92 sec latency 13.503 ms 2023-12-07T10:23:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.972+0000 7f80d03ff700 1 -- 172.21.15.119:0/3581337366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80c810a200 msgr2=0x7f80c80ff600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.972+0000 7f80d03ff700 1 --2- 172.21.15.119:0/3581337366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80c810a200 0x7f80c80ff600 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f80b8009990 tx=0x7f80b8009ca0 comp rx=0 tx=0).stop 2023-12-07T10:23:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.972+0000 7f80d03ff700 1 -- 172.21.15.119:0/3581337366 shutdown_connections 2023-12-07T10:23:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.972+0000 7f80d03ff700 1 --2- 172.21.15.119:0/3581337366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80c80ffd30 0x7f80c8110690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.972+0000 7f80d03ff700 1 --2- 172.21.15.119:0/3581337366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80c8109850 0x7f80c8109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.972+0000 7f80d03ff700 1 --2- 172.21.15.119:0/3581337366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80c810a200 0x7f80c80ff600 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.972+0000 7f80d03ff700 1 -- 172.21.15.119:0/3581337366 >> 172.21.15.119:0/3581337366 conn(0x7f80c8076590 msgr2=0x7f80c80fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.973+0000 7f80d03ff700 1 -- 172.21.15.119:0/3581337366 shutdown_connections 2023-12-07T10:23:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.973+0000 7f80d03ff700 1 -- 172.21.15.119:0/3581337366 wait complete. 2023-12-07T10:23:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.975+0000 7f80d03ff700 1 Processor -- start 2023-12-07T10:23:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.975+0000 7f80d03ff700 1 -- start start 2023-12-07T10:23:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.975+0000 7f80d03ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80c80ffd30 0x7f80c81103c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.975+0000 7f80d03ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80c8109850 0x7f80c810d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.975+0000 7f80d03ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80c810d9d0 0x7f80c810de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.975+0000 7f80d03ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c81aa520 con 0x7f80c8109850 2023-12-07T10:23:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.975+0000 7f80d03ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c8198ce0 con 0x7f80c810d9d0 2023-12-07T10:23:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.975+0000 7f80d03ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c8198ea0 con 0x7f80c80ffd30 2023-12-07T10:23:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.976+0000 7f80ce19b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80c80ffd30 0x7f80c81103c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.976+0000 7f80cd99a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80c8109850 0x7f80c810d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.976+0000 7f80ce19b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80c80ffd30 0x7f80c81103c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41340/0 (socket says 172.21.15.119:41340) 2023-12-07T10:23:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.976+0000 7f80cd99a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80c8109850 0x7f80c810d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36106/0 (socket says 172.21.15.119:36106) 2023-12-07T10:23:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.976+0000 7f80ce19b700 1 -- 172.21.15.119:0/4276631663 learned_addr learned my addr 172.21.15.119:0/4276631663 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.976+0000 7f80ce99c700 1 --2- 172.21.15.119:0/4276631663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80c810d9d0 0x7f80c810de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.976+0000 7f80ce19b700 1 -- 172.21.15.119:0/4276631663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80c810d9d0 msgr2=0x7f80c810de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.976+0000 7f80ce19b700 1 --2- 172.21.15.119:0/4276631663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80c810d9d0 0x7f80c810de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.976+0000 7f80ce19b700 1 -- 172.21.15.119:0/4276631663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80c8109850 msgr2=0x7f80c810d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.976+0000 7f80ce19b700 1 --2- 172.21.15.119:0/4276631663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80c8109850 0x7f80c810d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.976+0000 7f80ce19b700 1 -- 172.21.15.119:0/4276631663 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b80095f0 con 0x7f80c80ffd30 2023-12-07T10:23:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.976+0000 7f80cd99a700 1 --2- 172.21.15.119:0/4276631663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80c8109850 0x7f80c810d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.976+0000 7f80ce99c700 1 --2- 172.21.15.119:0/4276631663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80c810d9d0 0x7f80c810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.977+0000 7f80ce19b700 1 --2- 172.21.15.119:0/4276631663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80c80ffd30 0x7f80c81103c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f80c400e9a0 tx=0x7f80c400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.977+0000 7f80bf7fe700 1 -- 172.21.15.119:0/4276631663 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c4009790 con 0x7f80c80ffd30 2023-12-07T10:23:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.977+0000 7f80bf7fe700 1 -- 172.21.15.119:0/4276631663 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80c4010430 con 0x7f80c80ffd30 2023-12-07T10:23:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.977+0000 7f80bf7fe700 1 -- 172.21.15.119:0/4276631663 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c4018680 con 0x7f80c80ffd30 2023-12-07T10:23:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.977+0000 7f80d03ff700 1 -- 172.21.15.119:0/4276631663 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c8199180 con 0x7f80c80ffd30 2023-12-07T10:23:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.977+0000 7f80d03ff700 1 -- 172.21.15.119:0/4276631663 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c8199690 con 0x7f80c80ffd30 2023-12-07T10:23:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.981+0000 7f80bf7fe700 1 -- 172.21.15.119:0/4276631663 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f80c40105a0 con 0x7f80c80ffd30 2023-12-07T10:23:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.983+0000 7f80bf7fe700 1 --2- 172.21.15.119:0/4276631663 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f80b40a1d00 0x7f80b40a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.983+0000 7f80bf7fe700 1 -- 172.21.15.119:0/4276631663 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f80c4014070 con 0x7f80c80ffd30 2023-12-07T10:23:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.984+0000 7f80e3e18b80 1 Processor -- start 2023-12-07T10:23:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.984+0000 7f80cd99a700 1 --2- 172.21.15.119:0/4276631663 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f80b40a1d00 0x7f80b40a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.984+0000 7f80e3e18b80 1 -- start start 2023-12-07T10:23:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.984+0000 7f80e3e18b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561df1e98a60 0x561df1e98e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.984+0000 7f80e3e18b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df1e99380 0x561df1e93540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.984+0000 7f80e3e18b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561df1e93ad0 0x561df1e93f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.984+0000 7f80e3e18b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df1e07760 con 0x561df1e99380 2023-12-07T10:23:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.984+0000 7f80e3e18b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df1dee740 con 0x561df1e98a60 2023-12-07T10:23:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.984+0000 7f80e3e18b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df1dec060 con 0x561df1e93ad0 2023-12-07T10:23:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.985+0000 7f80ce19b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561df1e98a60 0x561df1e98e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.985+0000 7f80ce19b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561df1e98a60 0x561df1e98e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:38096/0 (socket says 172.21.15.119:38096) 2023-12-07T10:23:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.985+0000 7f80ce19b700 1 -- 172.21.15.119:0/847440629 learned_addr learned my addr 172.21.15.119:0/847440629 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.985+0000 7f80ce99c700 1 --2- 172.21.15.119:0/847440629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561df1e93ad0 0x561df1e93f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.985+0000 7f80cd99a700 1 --2- 172.21.15.119:0/4276631663 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f80b40a1d00 0x7f80b40a41c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f80c810ebc0 tx=0x7f80b8005fd0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.985+0000 7f80ce19b700 1 -- 172.21.15.119:0/847440629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561df1e93ad0 msgr2=0x561df1e93f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.985+0000 7f80cd99a700 1 --2- 172.21.15.119:0/847440629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df1e99380 0x561df1e93540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.985+0000 7f80ce19b700 1 --2- 172.21.15.119:0/847440629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561df1e93ad0 0x561df1e93f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.985+0000 7f80ce19b700 1 -- 172.21.15.119:0/847440629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df1e99380 msgr2=0x561df1e93540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.985+0000 7f80ce19b700 1 --2- 172.21.15.119:0/847440629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df1e99380 0x561df1e93540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.985+0000 7f80ce19b700 1 -- 172.21.15.119:0/847440629 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561df1c645d0 con 0x561df1e98a60 2023-12-07T10:23:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.986+0000 7f80ce19b700 1 --2- 172.21.15.119:0/847440629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561df1e98a60 0x561df1e98e40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f80c40a26e0 tx=0x7f80c409a4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2040d89f8198e0fd server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.986+0000 7f80bcbfe700 1 -- 172.21.15.119:0/847440629 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c40850f0 con 0x561df1e98a60 2023-12-07T10:23:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.986+0000 7f80bcbfe700 1 -- 172.21.15.119:0/847440629 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80c409e9d0 con 0x561df1e98a60 2023-12-07T10:23:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.986+0000 7f80bcbfe700 1 -- 172.21.15.119:0/847440629 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c409ad40 con 0x561df1e98a60 2023-12-07T10:23:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.987+0000 7f80e3e18b80 1 -- 172.21.15.119:0/847440629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561df1e98a60 msgr2=0x561df1e98e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.987+0000 7f80e3e18b80 1 --2- 172.21.15.119:0/847440629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561df1e98a60 0x561df1e98e40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f80c40a26e0 tx=0x7f80c409a4d0 comp rx=0 tx=0).stop 2023-12-07T10:23:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.987+0000 7f80e3e18b80 1 -- 172.21.15.119:0/847440629 shutdown_connections 2023-12-07T10:23:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.987+0000 7f80e3e18b80 1 --2- 172.21.15.119:0/847440629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561df1e93ad0 0x561df1e93f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.987+0000 7f80e3e18b80 1 --2- 172.21.15.119:0/847440629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561df1e98a60 0x561df1e98e40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.987+0000 7f80e3e18b80 1 --2- 172.21.15.119:0/847440629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df1e99380 0x561df1e93540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.987+0000 7f80e3e18b80 1 -- 172.21.15.119:0/847440629 >> 172.21.15.119:0/847440629 conn(0x561df1dc2300 msgr2=0x561df1e8db60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.987+0000 7f80e3e18b80 1 -- 172.21.15.119:0/847440629 shutdown_connections 2023-12-07T10:23:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.988+0000 7f80e3e18b80 1 -- 172.21.15.119:0/847440629 wait complete. 2023-12-07T10:23:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.988+0000 7f80e3e18b80 1 -- start start 2023-12-07T10:23:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.988+0000 7f80e3e18b80 1 Processor -- start 2023-12-07T10:23:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.989+0000 7f80e3e18b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df1e93ad0 0x561df1e9c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.989+0000 7f80e3e18b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561df1e98a60 0x561df1e9cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.989+0000 7f80e3e18b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561df1e99380 0x561df1e9d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.989+0000 7f80e3e18b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df1de7950 con 0x561df1e93ad0 2023-12-07T10:23:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.989+0000 7f80e3e18b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df1da1500 con 0x561df1e99380 2023-12-07T10:23:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.989+0000 7f80e3e18b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df1da11a0 con 0x561df1e98a60 2023-12-07T10:23:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.989+0000 7f80e3e18b80 10 client.0 _ll_register_callbacks cb 0x561df1d888e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.989+0000 7f80cd99a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561df1e98a60 0x561df1e9cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.990+0000 7f80cd99a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561df1e98a60 0x561df1e9cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41358/0 (socket says 172.21.15.119:41358) 2023-12-07T10:23:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.990+0000 7f80cd99a700 1 -- 172.21.15.119:0/2111136826 learned_addr learned my addr 172.21.15.119:0/2111136826 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.990+0000 7f80ce19b700 1 --2- 172.21.15.119:0/2111136826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df1e93ad0 0x561df1e9c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.990+0000 7f80ce99c700 1 --2- 172.21.15.119:0/2111136826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561df1e99380 0x561df1e9d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.990+0000 7f80cd99a700 1 -- 172.21.15.119:0/2111136826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561df1e99380 msgr2=0x561df1e9d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.990+0000 7f80cd99a700 1 --2- 172.21.15.119:0/2111136826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561df1e99380 0x561df1e9d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.990+0000 7f80cd99a700 1 -- 172.21.15.119:0/2111136826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df1e93ad0 msgr2=0x561df1e9c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.990+0000 7f80cd99a700 1 --2- 172.21.15.119:0/2111136826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df1e93ad0 0x561df1e9c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.990+0000 7f80cd99a700 1 -- 172.21.15.119:0/2111136826 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c8199690 con 0x561df1e98a60 2023-12-07T10:23:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.990+0000 7f80cd99a700 1 --2- 172.21.15.119:0/2111136826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561df1e98a60 0x561df1e9cc70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f80b8009990 tx=0x7f80b8009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf63fa2d7bee6b29 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.990+0000 7f80b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:23:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.991+0000 7f80b1ffb700 1 -- 172.21.15.119:0/2111136826 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b8024c80 con 0x561df1e98a60 2023-12-07T10:23:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.991+0000 7f80e3e18b80 1 -- 172.21.15.119:0/2111136826 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561df1e866f0 con 0x561df1e98a60 2023-12-07T10:23:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.991+0000 7f80b1ffb700 1 -- 172.21.15.119:0/2111136826 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f80b8024de0 con 0x561df1e98a60 2023-12-07T10:23:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.991+0000 7f80b1ffb700 1 -- 172.21.15.119:0/2111136826 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80b8005050 con 0x561df1e98a60 2023-12-07T10:23:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.991+0000 7f80e3e18b80 10 client.26629 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.991+0000 7f80e3e18b80 1 -- 172.21.15.119:0/2111136826 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x561df1c10d80 con 0x561df1e98a60 2023-12-07T10:23:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.991+0000 7f80b1ffb700 1 -- 172.21.15.119:0/2111136826 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561df1e866f0 con 0x561df1e98a60 2023-12-07T10:23:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.991+0000 7f80b1ffb700 1 -- 172.21.15.119:0/2111136826 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f80b8004cc0 con 0x561df1e98a60 2023-12-07T10:23:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.991+0000 7f80e3e18b80 10 client.26629 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.991+0000 7f80e3e18b80 10 client.26629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:23:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.991+0000 7f80e3e18b80 20 client.26629 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.991+0000 7f80e3e18b80 1 --2- 172.21.15.119:0/2111136826 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x561df1ebdf40 0x561df1ec0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.991+0000 7f80e3e18b80 4 client.26629 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:23:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.991+0000 7f80e3e18b80 1 -- 172.21.15.119:0/2111136826 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561df1e62740 con 0x561df1ebdf40 2023-12-07T10:23:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.992+0000 7f80ce19b700 1 --2- 172.21.15.119:0/2111136826 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x561df1ebdf40 0x561df1ec0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.992+0000 7f80ce19b700 1 --2- 172.21.15.119:0/2111136826 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x561df1ebdf40 0x561df1ec0360 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb9de0c225a99f6d server_cookie=bfc6b6829ed89cc6 in_seq=0 out_seq=0 2023-12-07T10:23:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.992+0000 7f80b1ffb700 10 client.26629 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.993+0000 7f80b1ffb700 1 -- 172.21.15.119:0/2111136826 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561df1e62740 con 0x561df1ebdf40 2023-12-07T10:23:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.993+0000 7f80b1ffb700 10 client.26629 handle_command_reply: tid=0 2023-12-07T10:23:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.993+0000 7f80e3e18b80 1 client.26629 shutdown 2023-12-07T10:23:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.994+0000 7f80e3e18b80 1 -- 172.21.15.119:0/2111136826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561df1e98a60 msgr2=0x561df1e9cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.994+0000 7f80e3e18b80 1 --2- 172.21.15.119:0/2111136826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561df1e98a60 0x561df1e9cc70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f80b8009990 tx=0x7f80b8009ca0 comp rx=0 tx=0).stop 2023-12-07T10:23:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.994+0000 7f80e3e18b80 1 -- 172.21.15.119:0/2111136826 shutdown_connections 2023-12-07T10:23:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.994+0000 7f80e3e18b80 1 --2- 172.21.15.119:0/2111136826 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x561df1ebdf40 0x561df1ec0360 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.994+0000 7f80e3e18b80 1 --2- 172.21.15.119:0/2111136826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561df1e99380 0x561df1e9d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.994+0000 7f80b1ffb700 0 client.26629 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.994+0000 7f80e3e18b80 1 --2- 172.21.15.119:0/2111136826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df1e93ad0 0x561df1e9c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.994+0000 7f80e3e18b80 1 --2- 172.21.15.119:0/2111136826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561df1e98a60 0x561df1e9cc70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.994+0000 7f80e3e18b80 1 -- 172.21.15.119:0/2111136826 >> 172.21.15.119:0/2111136826 conn(0x561df1dc2300 msgr2=0x561df1e88980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.994+0000 7f80e3e18b80 1 -- 172.21.15.119:0/2111136826 shutdown_connections 2023-12-07T10:23:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.994+0000 7f80e3e18b80 1 -- 172.21.15.119:0/2111136826 wait complete. 2023-12-07T10:23:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:19.995+0000 7f80e3e18b80 20 client.26629 trim_cache size 0 max 16384 2023-12-07T10:23:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.014+0000 7f80e3e18b80 1 Processor -- start 2023-12-07T10:23:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.014+0000 7f80e3e18b80 1 -- start start 2023-12-07T10:23:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.014+0000 7f80e3e18b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561df1e93ad0 0x561df1e93540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.014+0000 7f80e3e18b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df1e98a60 0x561df1eb4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.014+0000 7f80e3e18b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561df1e99380 0x561df1eb5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.014+0000 7f80e3e18b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df1c668c0 con 0x561df1e98a60 2023-12-07T10:23:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.014+0000 7f80e3e18b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df1e866f0 con 0x561df1e99380 2023-12-07T10:23:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.014+0000 7f80e3e18b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df1e8a7f0 con 0x561df1e93ad0 2023-12-07T10:23:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.015+0000 7f80ce19b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561df1e93ad0 0x561df1e93540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.015+0000 7f80ce99c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561df1e99380 0x561df1eb5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.015+0000 7f80cd99a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df1e98a60 0x561df1eb4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.015+0000 7f80ce19b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561df1e93ad0 0x561df1e93540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41366/0 (socket says 172.21.15.119:41366) 2023-12-07T10:23:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.015+0000 7f80ce19b700 1 -- 172.21.15.119:0/587229347 learned_addr learned my addr 172.21.15.119:0/587229347 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.015+0000 7f80ce19b700 1 -- 172.21.15.119:0/587229347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561df1e99380 msgr2=0x561df1eb5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.015+0000 7f80ce19b700 1 --2- 172.21.15.119:0/587229347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561df1e99380 0x561df1eb5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.015+0000 7f80ce19b700 1 -- 172.21.15.119:0/587229347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df1e98a60 msgr2=0x561df1eb4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.015+0000 7f80ce19b700 1 --2- 172.21.15.119:0/587229347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df1e98a60 0x561df1eb4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.015+0000 7f80ce19b700 1 -- 172.21.15.119:0/587229347 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561df1bf9c00 con 0x561df1e93ad0 2023-12-07T10:23:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.015+0000 7f80ce19b700 1 --2- 172.21.15.119:0/587229347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561df1e93ad0 0x561df1e93540 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f80c4096d80 tx=0x7f80c4010c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a00820e06b4876f server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.015+0000 7f80bd3ff700 1 -- 172.21.15.119:0/587229347 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c40850f0 con 0x561df1e93ad0 2023-12-07T10:23:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.016+0000 7f80bd3ff700 1 -- 172.21.15.119:0/587229347 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80c4010880 con 0x561df1e93ad0 2023-12-07T10:23:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.016+0000 7f80bd3ff700 1 -- 172.21.15.119:0/587229347 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c409ad20 con 0x561df1e93ad0 2023-12-07T10:23:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.016+0000 7f80e3e18b80 1 -- 172.21.15.119:0/587229347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561df1e93ad0 msgr2=0x561df1e93540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.016+0000 7f80e3e18b80 1 --2- 172.21.15.119:0/587229347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561df1e93ad0 0x561df1e93540 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f80c4096d80 tx=0x7f80c4010c80 comp rx=0 tx=0).stop 2023-12-07T10:23:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.016+0000 7f80e3e18b80 1 -- 172.21.15.119:0/587229347 shutdown_connections 2023-12-07T10:23:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.016+0000 7f80e3e18b80 1 --2- 172.21.15.119:0/587229347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561df1e99380 0x561df1eb5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.016+0000 7f80e3e18b80 1 --2- 172.21.15.119:0/587229347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561df1e93ad0 0x561df1e93540 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.016+0000 7f80e3e18b80 1 --2- 172.21.15.119:0/587229347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df1e98a60 0x561df1eb4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.016+0000 7f80e3e18b80 1 -- 172.21.15.119:0/587229347 >> 172.21.15.119:0/587229347 conn(0x561df1dc2300 msgr2=0x561df1e96960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.016+0000 7f80e3e18b80 1 -- 172.21.15.119:0/587229347 shutdown_connections 2023-12-07T10:23:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.016+0000 7f80e3e18b80 1 -- 172.21.15.119:0/587229347 wait complete. 2023-12-07T10:23:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.017+0000 7f80e3e18b80 1 -- start start 2023-12-07T10:23:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.017+0000 7f80e3e18b80 1 Processor -- start 2023-12-07T10:23:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.018+0000 7f80e3e18b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df1e93ad0 0x561df1eb2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.018+0000 7f80e3e18b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561df1e98a60 0x561df1eb34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.018+0000 7f80e3e18b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561df1e99380 0x561df1f13460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.018+0000 7f80e3e18b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df1eb3be0 con 0x561df1e93ad0 2023-12-07T10:23:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.018+0000 7f80e3e18b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df1f139a0 con 0x561df1e99380 2023-12-07T10:23:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.018+0000 7f80e3e18b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df1f13ae0 con 0x561df1e98a60 2023-12-07T10:23:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.018+0000 7f80e3e18b80 10 client.0 _ll_register_callbacks cb 0x561df1d888e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.018+0000 7f80ce99c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561df1e99380 0x561df1f13460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.018+0000 7f80ce99c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561df1e99380 0x561df1f13460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:38120/0 (socket says 172.21.15.119:38120) 2023-12-07T10:23:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.018+0000 7f80ce19b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df1e93ad0 0x561df1eb2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.018+0000 7f80ce99c700 1 -- 172.21.15.119:0/1290366397 learned_addr learned my addr 172.21.15.119:0/1290366397 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.018+0000 7f80ce99c700 1 -- 172.21.15.119:0/1290366397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561df1e98a60 msgr2=0x561df1eb34f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-07T10:23:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.019+0000 7f80cd99a700 1 --2- 172.21.15.119:0/1290366397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561df1e98a60 0x561df1eb34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.019+0000 7f80ce99c700 1 --2- 172.21.15.119:0/1290366397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561df1e98a60 0x561df1eb34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.019+0000 7f80ce99c700 1 -- 172.21.15.119:0/1290366397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df1e93ad0 msgr2=0x561df1eb2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.019+0000 7f80ce99c700 1 --2- 172.21.15.119:0/1290366397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df1e93ad0 0x561df1eb2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.019+0000 7f80ce99c700 1 -- 172.21.15.119:0/1290366397 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561df1c645d0 con 0x561df1e99380 2023-12-07T10:23:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.019+0000 7f80ce19b700 1 --2- 172.21.15.119:0/1290366397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df1e93ad0 0x561df1eb2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:23:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.019+0000 7f80ce99c700 1 --2- 172.21.15.119:0/1290366397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561df1e99380 0x561df1f13460 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f80c00045b0 tx=0x7f80c0004690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cef90952c3c54e23 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.019+0000 7f80b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:23:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.019+0000 7f80b1ffb700 1 -- 172.21.15.119:0/1290366397 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c0007aa0 con 0x561df1e99380 2023-12-07T10:23:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.019+0000 7f80e3e18b80 1 -- 172.21.15.119:0/1290366397 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80c409e9d0 con 0x561df1e99380 2023-12-07T10:23:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.019+0000 7f80b1ffb700 1 -- 172.21.15.119:0/1290366397 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f80c0007c00 con 0x561df1e99380 2023-12-07T10:23:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.019+0000 7f80b1ffb700 1 -- 172.21.15.119:0/1290366397 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80c0007dc0 con 0x561df1e99380 2023-12-07T10:23:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.019+0000 7f80e3e18b80 10 client.26657 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.019+0000 7f80e3e18b80 1 -- 172.21.15.119:0/1290366397 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x561df1c371f0 con 0x561df1e99380 2023-12-07T10:23:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.020+0000 7f80b1ffb700 1 -- 172.21.15.119:0/1290366397 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80c409e9d0 con 0x561df1e99380 2023-12-07T10:23:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.020+0000 7f80b1ffb700 1 -- 172.21.15.119:0/1290366397 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f80c0018630 con 0x561df1e99380 2023-12-07T10:23:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.020+0000 7f80e3e18b80 10 client.26657 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.020+0000 7f80e3e18b80 10 client.26657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:23:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.020+0000 7f80e3e18b80 20 client.26657 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.020+0000 7f80e3e18b80 1 --2- 172.21.15.119:0/1290366397 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x561df1f15430 0x561df1f17850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.020+0000 7f80e3e18b80 4 client.26657 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:23:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.020+0000 7f80e3e18b80 1 -- 172.21.15.119:0/1290366397 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561df1dadf70 con 0x561df1f15430 2023-12-07T10:23:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.020+0000 7f80ce19b700 1 --2- 172.21.15.119:0/1290366397 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x561df1f15430 0x561df1f17850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.021+0000 7f80ce19b700 1 --2- 172.21.15.119:0/1290366397 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x561df1f15430 0x561df1f17850 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cc4902ca69abdf server_cookie=fd7ca65e8f2e8a9d in_seq=0 out_seq=0 2023-12-07T10:23:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.021+0000 7f80b1ffb700 10 client.26657 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.060+0000 7f80b1ffb700 1 -- 172.21.15.119:0/1290366397 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561df1dadf70 con 0x561df1f15430 2023-12-07T10:23:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.060+0000 7f80b1ffb700 10 client.26657 handle_command_reply: tid=0 2023-12-07T10:23:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.061+0000 7f80e3e18b80 1 client.26657 shutdown 2023-12-07T10:23:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.061+0000 7f80e3e18b80 1 -- 172.21.15.119:0/1290366397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561df1e99380 msgr2=0x561df1f13460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.061+0000 7f80e3e18b80 1 --2- 172.21.15.119:0/1290366397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561df1e99380 0x561df1f13460 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f80c00045b0 tx=0x7f80c0004690 comp rx=0 tx=0).stop 2023-12-07T10:23:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.061+0000 7f80e3e18b80 1 -- 172.21.15.119:0/1290366397 shutdown_connections 2023-12-07T10:23:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.061+0000 7f80e3e18b80 1 --2- 172.21.15.119:0/1290366397 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x561df1f15430 0x561df1f17850 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.061+0000 7f80e3e18b80 1 --2- 172.21.15.119:0/1290366397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561df1e99380 0x561df1f13460 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.061+0000 7f80e3e18b80 1 --2- 172.21.15.119:0/1290366397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df1e93ad0 0x561df1eb2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.061+0000 7f80e3e18b80 1 --2- 172.21.15.119:0/1290366397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561df1e98a60 0x561df1eb34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.061+0000 7f80e3e18b80 1 -- 172.21.15.119:0/1290366397 >> 172.21.15.119:0/1290366397 conn(0x561df1dc2300 msgr2=0x561df1e9b020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.061+0000 7f80b1ffb700 0 client.26657 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.061+0000 7f80e3e18b80 1 -- 172.21.15.119:0/1290366397 shutdown_connections 2023-12-07T10:23:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.061+0000 7f80e3e18b80 1 -- 172.21.15.119:0/1290366397 wait complete. 2023-12-07T10:23:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.062+0000 7f80e3e18b80 20 client.26657 trim_cache size 0 max 16384 2023-12-07T10:23:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:23:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:23:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:23:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:23:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.062+0000 7f80d03ff700 1 -- 172.21.15.119:0/4276631663 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f80b40a1d00 msgr2=0x7f80b40a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.062+0000 7f80d03ff700 1 --2- 172.21.15.119:0/4276631663 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f80b40a1d00 0x7f80b40a41c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f80c810ebc0 tx=0x7f80b8005fd0 comp rx=0 tx=0).stop 2023-12-07T10:23:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.062+0000 7f80d03ff700 1 -- 172.21.15.119:0/4276631663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80c80ffd30 msgr2=0x7f80c81103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.062+0000 7f80d03ff700 1 --2- 172.21.15.119:0/4276631663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80c80ffd30 0x7f80c81103c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f80c400e9a0 tx=0x7f80c400ecb0 comp rx=0 tx=0).stop 2023-12-07T10:23:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.062+0000 7f80d03ff700 1 -- 172.21.15.119:0/4276631663 shutdown_connections 2023-12-07T10:23:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.062+0000 7f80d03ff700 1 --2- 172.21.15.119:0/4276631663 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f80b40a1d00 0x7f80b40a41c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.062+0000 7f80d03ff700 1 --2- 172.21.15.119:0/4276631663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80c810d9d0 0x7f80c810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.062+0000 7f80d03ff700 1 --2- 172.21.15.119:0/4276631663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80c80ffd30 0x7f80c81103c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.062+0000 7f80d03ff700 1 --2- 172.21.15.119:0/4276631663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80c8109850 0x7f80c810d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.062+0000 7f80d03ff700 1 -- 172.21.15.119:0/4276631663 >> 172.21.15.119:0/4276631663 conn(0x7f80c8076590 msgr2=0x7f80c8107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.062+0000 7f80d03ff700 1 -- 172.21.15.119:0/4276631663 shutdown_connections 2023-12-07T10:23:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:20.063+0000 7f80d03ff700 1 -- 172.21.15.119:0/4276631663 wait complete. 2023-12-07T10:23:20.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:19 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:23:20.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:19 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:23:20.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:19 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:23:20.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:19 smithi159 ceph-mon[134726]: pgmap v90: 129 pgs: 129 active+clean; 95 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 263 op/s 2023-12-07T10:23:20.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:19 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:23:20.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:19 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:23:20.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:19 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:23:20.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:19 smithi153 ceph-mon[133625]: pgmap v90: 129 pgs: 129 active+clean; 95 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 263 op/s 2023-12-07T10:23:20.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:19 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:23:20.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:19 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:23:20.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:19 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:23:20.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:19 smithi119 ceph-mon[97908]: pgmap v90: 129 pgs: 129 active+clean; 95 MiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 263 op/s 2023-12-07T10:23:20.548 INFO:tasks.workunit.client.0.smithi119.stdout: 1 146136 8.66 MB/sec warmup 93 sec latency 38.242 ms 2023-12-07T10:23:21.548 INFO:tasks.workunit.client.0.smithi119.stdout: 1 148358 8.69 MB/sec warmup 94 sec latency 39.421 ms 2023-12-07T10:23:22.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:21 smithi119 ceph-mon[97908]: pgmap v91: 129 pgs: 129 active+clean; 101 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 395 op/s 2023-12-07T10:23:22.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:21 smithi159 ceph-mon[134726]: pgmap v91: 129 pgs: 129 active+clean; 101 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 395 op/s 2023-12-07T10:23:22.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:21 smithi153 ceph-mon[133625]: pgmap v91: 129 pgs: 129 active+clean; 101 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 395 op/s 2023-12-07T10:23:22.548 INFO:tasks.workunit.client.0.smithi119.stdout: 1 149620 8.67 MB/sec warmup 95 sec latency 16.800 ms 2023-12-07T10:23:23.548 INFO:tasks.workunit.client.0.smithi119.stdout: 1 151921 8.71 MB/sec warmup 96 sec latency 11.640 ms 2023-12-07T10:23:24.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:23 smithi119 ceph-mon[97908]: pgmap v92: 129 pgs: 129 active+clean; 101 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 251 op/s 2023-12-07T10:23:24.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:23 smithi159 ceph-mon[134726]: pgmap v92: 129 pgs: 129 active+clean; 101 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 251 op/s 2023-12-07T10:23:24.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:23 smithi153 ceph-mon[133625]: pgmap v92: 129 pgs: 129 active+clean; 101 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 251 op/s 2023-12-07T10:23:24.548 INFO:tasks.workunit.client.0.smithi119.stdout: 1 153229 8.70 MB/sec warmup 97 sec latency 25.988 ms 2023-12-07T10:23:25.549 INFO:tasks.workunit.client.0.smithi119.stdout: 1 155524 8.73 MB/sec warmup 98 sec latency 12.510 ms 2023-12-07T10:23:26.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:25 smithi159 ceph-mon[134726]: pgmap v93: 129 pgs: 129 active+clean; 101 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 251 op/s 2023-12-07T10:23:26.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:25 smithi153 ceph-mon[133625]: pgmap v93: 129 pgs: 129 active+clean; 101 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 251 op/s 2023-12-07T10:23:26.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:25 smithi119 ceph-mon[97908]: pgmap v93: 129 pgs: 129 active+clean; 101 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 251 op/s 2023-12-07T10:23:26.549 INFO:tasks.workunit.client.0.smithi119.stdout: 1 156693 8.71 MB/sec warmup 99 sec latency 33.461 ms 2023-12-07T10:23:27.549 INFO:tasks.workunit.client.0.smithi119.stdout: 1 158966 8.74 MB/sec warmup 100 sec latency 16.450 ms 2023-12-07T10:23:28.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:27 smithi159 ceph-mon[134726]: pgmap v94: 129 pgs: 129 active+clean; 100 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 363 op/s 2023-12-07T10:23:28.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:27 smithi153 ceph-mon[133625]: pgmap v94: 129 pgs: 129 active+clean; 100 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 363 op/s 2023-12-07T10:23:28.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:27 smithi119 ceph-mon[97908]: pgmap v94: 129 pgs: 129 active+clean; 100 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 363 op/s 2023-12-07T10:23:28.549 INFO:tasks.workunit.client.0.smithi119.stdout: 1 160208 8.73 MB/sec warmup 101 sec latency 12.042 ms 2023-12-07T10:23:29.549 INFO:tasks.workunit.client.0.smithi119.stdout: 1 162419 8.75 MB/sec warmup 102 sec latency 12.713 ms 2023-12-07T10:23:30.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:29 smithi159 ceph-mon[134726]: pgmap v95: 129 pgs: 129 active+clean; 100 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 244 op/s 2023-12-07T10:23:30.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:29 smithi153 ceph-mon[133625]: pgmap v95: 129 pgs: 129 active+clean; 100 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 244 op/s 2023-12-07T10:23:30.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:29 smithi119 ceph-mon[97908]: pgmap v95: 129 pgs: 129 active+clean; 100 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 244 op/s 2023-12-07T10:23:30.549 INFO:tasks.workunit.client.0.smithi119.stdout: 1 163772 8.74 MB/sec warmup 103 sec latency 12.612 ms 2023-12-07T10:23:31.549 INFO:tasks.workunit.client.0.smithi119.stdout: 1 165950 8.77 MB/sec warmup 104 sec latency 16.845 ms 2023-12-07T10:23:32.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:31 smithi119 ceph-mon[97908]: pgmap v96: 129 pgs: 129 active+clean; 103 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 344 op/s 2023-12-07T10:23:32.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:31 smithi159 ceph-mon[134726]: pgmap v96: 129 pgs: 129 active+clean; 103 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 344 op/s 2023-12-07T10:23:32.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:31 smithi153 ceph-mon[133625]: pgmap v96: 129 pgs: 129 active+clean; 103 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 344 op/s 2023-12-07T10:23:32.549 INFO:tasks.workunit.client.0.smithi119.stdout: 1 167284 8.76 MB/sec warmup 105 sec latency 17.696 ms 2023-12-07T10:23:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.225+0000 7fcb579ff700 1 -- 172.21.15.119:0/1741275180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb5010dd50 msgr2=0x7fcb50110180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.225+0000 7fcb579ff700 1 --2- 172.21.15.119:0/1741275180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb5010dd50 0x7fcb50110180 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800b0a0 tx=0x7fcb4800b3b0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.225+0000 7fcb579ff700 1 -- 172.21.15.119:0/1741275180 shutdown_connections 2023-12-07T10:23:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.225+0000 7fcb579ff700 1 --2- 172.21.15.119:0/1741275180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb5010dd50 0x7fcb50110180 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.225+0000 7fcb579ff700 1 --2- 172.21.15.119:0/1741275180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb50073a00 0x7fcb50073de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.225+0000 7fcb579ff700 1 --2- 172.21.15.119:0/1741275180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb50074320 0x7fcb5010d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.225+0000 7fcb579ff700 1 -- 172.21.15.119:0/1741275180 >> 172.21.15.119:0/1741275180 conn(0x7fcb500fcfe0 msgr2=0x7fcb500ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.225+0000 7fcb579ff700 1 -- 172.21.15.119:0/1741275180 shutdown_connections 2023-12-07T10:23:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.226+0000 7fcb579ff700 1 -- 172.21.15.119:0/1741275180 wait complete. 2023-12-07T10:23:33.228 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.227+0000 7fcb579ff700 1 Processor -- start 2023-12-07T10:23:33.228 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.227+0000 7fcb579ff700 1 -- start start 2023-12-07T10:23:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.228+0000 7fcb579ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb50073a00 0x7fcb5019fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.228+0000 7fcb579ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb50074320 0x7fcb5019ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.228+0000 7fcb579ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb5010dd50 0x7fcb50199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.228+0000 7fcb579ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5019a190 con 0x7fcb50073a00 2023-12-07T10:23:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.228+0000 7fcb579ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5019a300 con 0x7fcb50074320 2023-12-07T10:23:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.228+0000 7fcb579ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5019a4c0 con 0x7fcb5010dd50 2023-12-07T10:23:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.228+0000 7fcb55f9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb5010dd50 0x7fcb50199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.228+0000 7fcb5579b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb50073a00 0x7fcb5019fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.228+0000 7fcb55f9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb5010dd50 0x7fcb50199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41450/0 (socket says 172.21.15.119:41450) 2023-12-07T10:23:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.228+0000 7fcb55f9c700 1 -- 172.21.15.119:0/1842672517 learned_addr learned my addr 172.21.15.119:0/1842672517 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.228+0000 7fcb54f9a700 1 --2- 172.21.15.119:0/1842672517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb50074320 0x7fcb5019ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.228+0000 7fcb5579b700 1 -- 172.21.15.119:0/1842672517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb5010dd50 msgr2=0x7fcb50199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.228+0000 7fcb5579b700 1 --2- 172.21.15.119:0/1842672517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb5010dd50 0x7fcb50199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.228+0000 7fcb5579b700 1 -- 172.21.15.119:0/1842672517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb50074320 msgr2=0x7fcb5019ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.228+0000 7fcb5579b700 1 --2- 172.21.15.119:0/1842672517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb50074320 0x7fcb5019ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.228+0000 7fcb5579b700 1 -- 172.21.15.119:0/1842672517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb48009d00 con 0x7fcb50073a00 2023-12-07T10:23:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.229+0000 7fcb5579b700 1 --2- 172.21.15.119:0/1842672517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb50073a00 0x7fcb5019fa00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00e9f0 tx=0x7fcb4c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.229+0000 7fcb467fc700 1 -- 172.21.15.119:0/1842672517 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c00c9c0 con 0x7fcb50073a00 2023-12-07T10:23:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.229+0000 7fcb579ff700 1 -- 172.21.15.119:0/1842672517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb5019a7a0 con 0x7fcb50073a00 2023-12-07T10:23:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.229+0000 7fcb467fc700 1 -- 172.21.15.119:0/1842672517 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb4c010430 con 0x7fcb50073a00 2023-12-07T10:23:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.229+0000 7fcb467fc700 1 -- 172.21.15.119:0/1842672517 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c018680 con 0x7fcb50073a00 2023-12-07T10:23:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.229+0000 7fcb579ff700 1 -- 172.21.15.119:0/1842672517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb501a69f0 con 0x7fcb50073a00 2023-12-07T10:23:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.233+0000 7fcb467fc700 1 -- 172.21.15.119:0/1842672517 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fcb4c0187e0 con 0x7fcb50073a00 2023-12-07T10:23:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.234+0000 7fcb6b2b4b80 1 Processor -- start 2023-12-07T10:23:33.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.234+0000 7fcb6b2b4b80 1 -- start start 2023-12-07T10:23:33.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.234+0000 7fcb467fc700 1 --2- 172.21.15.119:0/1842672517 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fcb3c0a1d00 0x7fcb3c0a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:33.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.234+0000 7fcb6b2b4b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a918640a60 0x55a918640e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.234+0000 7fcb6b2b4b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a918641380 0x55a91863b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.234+0000 7fcb6b2b4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a91863bad0 0x55a91863bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.234+0000 7fcb6b2b4b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9185af760 con 0x55a918640a60 2023-12-07T10:23:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.234+0000 7fcb6b2b4b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a918596740 con 0x55a918641380 2023-12-07T10:23:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.234+0000 7fcb6b2b4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a918594060 con 0x55a91863bad0 2023-12-07T10:23:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.234+0000 7fcb467fc700 1 -- 172.21.15.119:0/1842672517 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fcb4c014070 con 0x7fcb50073a00 2023-12-07T10:23:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.235+0000 7fcb55f9c700 1 --2- 172.21.15.119:0/1842672517 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fcb3c0a1d00 0x7fcb3c0a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.235+0000 7fcb54f9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a918641380 0x55a91863b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.235+0000 7fcb54f9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a918641380 0x55a91863b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:55820/0 (socket says 172.21.15.119:55820) 2023-12-07T10:23:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.235+0000 7fcb54f9a700 1 -- 172.21.15.119:0/636226198 learned_addr learned my addr 172.21.15.119:0/636226198 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.235+0000 7fcb55f9c700 1 --2- 172.21.15.119:0/636226198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a91863bad0 0x55a91863bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.235+0000 7fcb5579b700 1 --2- 172.21.15.119:0/636226198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a918640a60 0x55a918640e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.235+0000 7fcb54f9a700 1 -- 172.21.15.119:0/636226198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a91863bad0 msgr2=0x55a91863bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.235+0000 7fcb54f9a700 1 --2- 172.21.15.119:0/636226198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a91863bad0 0x55a91863bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.235+0000 7fcb54f9a700 1 -- 172.21.15.119:0/636226198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a918640a60 msgr2=0x55a918640e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.235+0000 7fcb54f9a700 1 --2- 172.21.15.119:0/636226198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a918640a60 0x55a918640e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.235+0000 7fcb54f9a700 1 -- 172.21.15.119:0/636226198 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a91840c5d0 con 0x55a918641380 2023-12-07T10:23:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.235+0000 7fcb54f9a700 1 --2- 172.21.15.119:0/636226198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a918641380 0x55a91863b540 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009ca0 tx=0x7fcb40011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2e05159de7b04a1 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.236+0000 7fcb55f9c700 1 --2- 172.21.15.119:0/1842672517 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fcb3c0a1d00 0x7fcb3c0a41c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55a918637670 tx=0x7fcb480093a0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.236+0000 7fcb3b7fe700 1 -- 172.21.15.119:0/636226198 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb40011e30 con 0x55a918641380 2023-12-07T10:23:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.236+0000 7fcb3b7fe700 1 -- 172.21.15.119:0/636226198 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb40004cf0 con 0x55a918641380 2023-12-07T10:23:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.236+0000 7fcb3b7fe700 1 -- 172.21.15.119:0/636226198 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb40019630 con 0x55a918641380 2023-12-07T10:23:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.236+0000 7fcb6b2b4b80 1 -- 172.21.15.119:0/636226198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a918641380 msgr2=0x55a91863b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.236+0000 7fcb6b2b4b80 1 --2- 172.21.15.119:0/636226198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a918641380 0x55a91863b540 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009ca0 tx=0x7fcb40011440 comp rx=0 tx=0).stop 2023-12-07T10:23:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.236+0000 7fcb6b2b4b80 1 -- 172.21.15.119:0/636226198 shutdown_connections 2023-12-07T10:23:33.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.236+0000 7fcb6b2b4b80 1 --2- 172.21.15.119:0/636226198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a91863bad0 0x55a91863bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.236+0000 7fcb6b2b4b80 1 --2- 172.21.15.119:0/636226198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a918640a60 0x55a918640e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.236+0000 7fcb6b2b4b80 1 --2- 172.21.15.119:0/636226198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a918641380 0x55a91863b540 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.236+0000 7fcb6b2b4b80 1 -- 172.21.15.119:0/636226198 >> 172.21.15.119:0/636226198 conn(0x55a91856a300 msgr2=0x55a918635b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:33.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.237+0000 7fcb6b2b4b80 1 -- 172.21.15.119:0/636226198 shutdown_connections 2023-12-07T10:23:33.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.237+0000 7fcb6b2b4b80 1 -- 172.21.15.119:0/636226198 wait complete. 2023-12-07T10:23:33.243 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.237+0000 7fcb6b2b4b80 1 -- start start 2023-12-07T10:23:33.243 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.237+0000 7fcb6b2b4b80 1 Processor -- start 2023-12-07T10:23:33.243 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.238+0000 7fcb6b2b4b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a91863bad0 0x55a918644760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:33.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.238+0000 7fcb6b2b4b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a918640a60 0x55a918644ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:33.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.238+0000 7fcb6b2b4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a918641380 0x55a9186451e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:33.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.238+0000 7fcb6b2b4b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a91858f950 con 0x55a918640a60 2023-12-07T10:23:33.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.238+0000 7fcb6b2b4b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a918549500 con 0x55a91863bad0 2023-12-07T10:23:33.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.238+0000 7fcb6b2b4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9185491a0 con 0x55a918641380 2023-12-07T10:23:33.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.238+0000 7fcb6b2b4b80 10 client.0 _ll_register_callbacks cb 0x55a9185308e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:33.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.238+0000 7fcb54f9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a918640a60 0x55a918644ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:33.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.238+0000 7fcb55f9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a918641380 0x55a9186451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:33.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.238+0000 7fcb5579b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a91863bad0 0x55a918644760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:33.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.238+0000 7fcb5579b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a91863bad0 0x55a918644760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:55836/0 (socket says 172.21.15.119:55836) 2023-12-07T10:23:33.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.238+0000 7fcb55f9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a918641380 0x55a9186451e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41466/0 (socket says 172.21.15.119:41466) 2023-12-07T10:23:33.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.238+0000 7fcb5579b700 1 -- 172.21.15.119:0/1659269960 learned_addr learned my addr 172.21.15.119:0/1659269960 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:33.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.238+0000 7fcb54f9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a918640a60 0x55a918644ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34624/0 (socket says 172.21.15.119:34624) 2023-12-07T10:23:33.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.239+0000 7fcb55f9c700 1 -- 172.21.15.119:0/1659269960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a91863bad0 msgr2=0x55a918644760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:33.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.239+0000 7fcb55f9c700 1 --2- 172.21.15.119:0/1659269960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a91863bad0 0x55a918644760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.239+0000 7fcb55f9c700 1 -- 172.21.15.119:0/1659269960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a918640a60 msgr2=0x55a918644ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:33.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.239+0000 7fcb55f9c700 1 --2- 172.21.15.119:0/1659269960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a918640a60 0x55a918644ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.239+0000 7fcb55f9c700 1 -- 172.21.15.119:0/1659269960 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb40009900 con 0x55a918641380 2023-12-07T10:23:33.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.239+0000 7fcb5579b700 1 --2- 172.21.15.119:0/1659269960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a91863bad0 0x55a918644760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:33.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.239+0000 7fcb55f9c700 1 --2- 172.21.15.119:0/1659269960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a918641380 0x55a9186451e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009cd0 tx=0x7fcb48009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5c9f24e23616054 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:33.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.239+0000 7fcb38ff9700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:23:33.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.240+0000 7fcb38ff9700 1 -- 172.21.15.119:0/1659269960 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb48027070 con 0x55a918641380 2023-12-07T10:23:33.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.240+0000 7fcb38ff9700 1 -- 172.21.15.119:0/1659269960 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fcb480074e0 con 0x55a918641380 2023-12-07T10:23:33.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.240+0000 7fcb6b2b4b80 1 -- 172.21.15.119:0/1659269960 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a91862e6f0 con 0x55a918641380 2023-12-07T10:23:33.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.240+0000 7fcb38ff9700 1 -- 172.21.15.119:0/1659269960 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb4800e040 con 0x55a918641380 2023-12-07T10:23:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.240+0000 7fcb6b2b4b80 10 client.26644 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.240+0000 7fcb6b2b4b80 1 -- 172.21.15.119:0/1659269960 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55a9183b8d80 con 0x55a918641380 2023-12-07T10:23:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.240+0000 7fcb38ff9700 1 -- 172.21.15.119:0/1659269960 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb48035080 con 0x55a918641380 2023-12-07T10:23:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.241+0000 7fcb38ff9700 1 -- 172.21.15.119:0/1659269960 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fcb4801ec80 con 0x55a918641380 2023-12-07T10:23:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.241+0000 7fcb6b2b4b80 10 client.26644 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.241+0000 7fcb6b2b4b80 10 client.26644 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:23:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.241+0000 7fcb6b2b4b80 20 client.26644 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.241+0000 7fcb6b2b4b80 1 --2- 172.21.15.119:0/1659269960 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a918655030 0x55a918657450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.241+0000 7fcb6b2b4b80 4 client.26644 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:23:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.241+0000 7fcb6b2b4b80 1 -- 172.21.15.119:0/1659269960 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a91860a740 con 0x55a918655030 2023-12-07T10:23:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.241+0000 7fcb5579b700 1 --2- 172.21.15.119:0/1659269960 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a918655030 0x55a918657450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.241+0000 7fcb5579b700 1 --2- 172.21.15.119:0/1659269960 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a918655030 0x55a918657450 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44f4dcdb3bcd35a2 server_cookie=bfd54c7a99cdde85 in_seq=0 out_seq=0 2023-12-07T10:23:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.241+0000 7fcb38ff9700 10 client.26644 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.242+0000 7fcb38ff9700 1 -- 172.21.15.119:0/1659269960 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a91860a740 con 0x55a918655030 2023-12-07T10:23:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.242+0000 7fcb38ff9700 10 client.26644 handle_command_reply: tid=0 2023-12-07T10:23:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.242+0000 7fcb6b2b4b80 1 client.26644 shutdown 2023-12-07T10:23:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.242+0000 7fcb6b2b4b80 1 -- 172.21.15.119:0/1659269960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a918641380 msgr2=0x55a9186451e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.242+0000 7fcb6b2b4b80 1 --2- 172.21.15.119:0/1659269960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a918641380 0x55a9186451e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009cd0 tx=0x7fcb48009780 comp rx=0 tx=0).stop 2023-12-07T10:23:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.242+0000 7fcb6b2b4b80 1 -- 172.21.15.119:0/1659269960 shutdown_connections 2023-12-07T10:23:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.242+0000 7fcb6b2b4b80 1 --2- 172.21.15.119:0/1659269960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a918641380 0x55a9186451e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.242+0000 7fcb6b2b4b80 1 --2- 172.21.15.119:0/1659269960 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a918655030 0x55a918657450 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.243+0000 7fcb6b2b4b80 1 --2- 172.21.15.119:0/1659269960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a91863bad0 0x55a918644760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.243+0000 7fcb38ff9700 0 client.26644 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.243+0000 7fcb6b2b4b80 1 --2- 172.21.15.119:0/1659269960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a918640a60 0x55a918644ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.243+0000 7fcb6b2b4b80 1 -- 172.21.15.119:0/1659269960 >> 172.21.15.119:0/1659269960 conn(0x55a91856a300 msgr2=0x55a918630980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.243+0000 7fcb6b2b4b80 1 -- 172.21.15.119:0/1659269960 shutdown_connections 2023-12-07T10:23:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.243+0000 7fcb6b2b4b80 1 -- 172.21.15.119:0/1659269960 wait complete. 2023-12-07T10:23:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.243+0000 7fcb6b2b4b80 20 client.26644 trim_cache size 0 max 16384 2023-12-07T10:23:33.266 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.265+0000 7fcb6b2b4b80 1 Processor -- start 2023-12-07T10:23:33.266 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.266+0000 7fcb6b2b4b80 1 -- start start 2023-12-07T10:23:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.266+0000 7fcb6b2b4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a91863bad0 0x55a91863b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.266+0000 7fcb6b2b4b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a918640a60 0x55a91864bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.266+0000 7fcb6b2b4b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a918641380 0x55a91864c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.266+0000 7fcb6b2b4b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a91840e8c0 con 0x55a918641380 2023-12-07T10:23:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.266+0000 7fcb6b2b4b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a918652560 con 0x55a918640a60 2023-12-07T10:23:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.266+0000 7fcb6b2b4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9186477a0 con 0x55a91863bad0 2023-12-07T10:23:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.266+0000 7fcb5579b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a91863bad0 0x55a91863b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.266+0000 7fcb55f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a918641380 0x55a91864c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.267+0000 7fcb54f9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a918640a60 0x55a91864bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.267+0000 7fcb5579b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a91863bad0 0x55a91863b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41474/0 (socket says 172.21.15.119:41474) 2023-12-07T10:23:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.267+0000 7fcb5579b700 1 -- 172.21.15.119:0/3103334294 learned_addr learned my addr 172.21.15.119:0/3103334294 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.267+0000 7fcb54f9a700 1 -- 172.21.15.119:0/3103334294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a91863bad0 msgr2=0x55a91863b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.267+0000 7fcb54f9a700 1 --2- 172.21.15.119:0/3103334294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a91863bad0 0x55a91863b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.267+0000 7fcb54f9a700 1 -- 172.21.15.119:0/3103334294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a918641380 msgr2=0x55a91864c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.267+0000 7fcb54f9a700 1 --2- 172.21.15.119:0/3103334294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a918641380 0x55a91864c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.267+0000 7fcb54f9a700 1 -- 172.21.15.119:0/3103334294 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9183a1c00 con 0x55a918640a60 2023-12-07T10:23:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.267+0000 7fcb55f9c700 1 --2- 172.21.15.119:0/3103334294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a918641380 0x55a91864c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.267+0000 7fcb54f9a700 1 --2- 172.21.15.119:0/3103334294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a918640a60 0x55a91864bdd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcb4001b040 tx=0x7fcb40005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a30a1e9d4e99e4cf server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.267+0000 7fcb3bfff700 1 -- 172.21.15.119:0/3103334294 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb40011c30 con 0x55a918640a60 2023-12-07T10:23:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.268+0000 7fcb3bfff700 1 -- 172.21.15.119:0/3103334294 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb40005320 con 0x55a918640a60 2023-12-07T10:23:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.268+0000 7fcb3bfff700 1 -- 172.21.15.119:0/3103334294 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb40003e00 con 0x55a918640a60 2023-12-07T10:23:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.268+0000 7fcb6b2b4b80 1 -- 172.21.15.119:0/3103334294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a918640a60 msgr2=0x55a91864bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.268+0000 7fcb6b2b4b80 1 --2- 172.21.15.119:0/3103334294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a918640a60 0x55a91864bdd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcb4001b040 tx=0x7fcb40005db0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.268+0000 7fcb6b2b4b80 1 -- 172.21.15.119:0/3103334294 shutdown_connections 2023-12-07T10:23:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.268+0000 7fcb6b2b4b80 1 --2- 172.21.15.119:0/3103334294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a918641380 0x55a91864c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.268+0000 7fcb6b2b4b80 1 --2- 172.21.15.119:0/3103334294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a91863bad0 0x55a91863b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.268+0000 7fcb6b2b4b80 1 --2- 172.21.15.119:0/3103334294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a918640a60 0x55a91864bdd0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.273 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.268+0000 7fcb6b2b4b80 1 -- 172.21.15.119:0/3103334294 >> 172.21.15.119:0/3103334294 conn(0x55a91856a300 msgr2=0x55a91865bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:33.273 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.268+0000 7fcb6b2b4b80 1 -- 172.21.15.119:0/3103334294 shutdown_connections 2023-12-07T10:23:33.273 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.268+0000 7fcb6b2b4b80 1 -- 172.21.15.119:0/3103334294 wait complete. 2023-12-07T10:23:33.277 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.269+0000 7fcb6b2b4b80 1 -- start start 2023-12-07T10:23:33.277 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.269+0000 7fcb6b2b4b80 1 Processor -- start 2023-12-07T10:23:33.277 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.270+0000 7fcb6b2b4b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a91863bad0 0x55a918649db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:33.277 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.270+0000 7fcb6b2b4b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a918640a60 0x55a91864a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:33.277 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.270+0000 7fcb6b2b4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a918641380 0x55a91864a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:33.278 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.270+0000 7fcb6b2b4b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9186aa530 con 0x55a91863bad0 2023-12-07T10:23:33.278 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.270+0000 7fcb6b2b4b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9186aa670 con 0x55a918640a60 2023-12-07T10:23:33.278 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.270+0000 7fcb6b2b4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9186aa830 con 0x55a918641380 2023-12-07T10:23:33.278 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.270+0000 7fcb6b2b4b80 10 client.0 _ll_register_callbacks cb 0x55a9185308e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:33.278 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.270+0000 7fcb55f9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a918641380 0x55a91864a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:33.278 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.270+0000 7fcb5579b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a91863bad0 0x55a918649db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:33.278 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.270+0000 7fcb55f9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a918641380 0x55a91864a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41484/0 (socket says 172.21.15.119:41484) 2023-12-07T10:23:33.278 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.270+0000 7fcb5579b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a91863bad0 0x55a918649db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34636/0 (socket says 172.21.15.119:34636) 2023-12-07T10:23:33.278 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.270+0000 7fcb55f9c700 1 -- 172.21.15.119:0/1828483314 learned_addr learned my addr 172.21.15.119:0/1828483314 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:33.278 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.270+0000 7fcb54f9a700 1 --2- 172.21.15.119:0/1828483314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a918640a60 0x55a91864a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:33.278 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.270+0000 7fcb55f9c700 1 -- 172.21.15.119:0/1828483314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a918640a60 msgr2=0x55a91864a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:33.278 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.270+0000 7fcb55f9c700 1 --2- 172.21.15.119:0/1828483314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a918640a60 0x55a91864a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.278 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.270+0000 7fcb55f9c700 1 -- 172.21.15.119:0/1828483314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a91863bad0 msgr2=0x55a918649db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.270+0000 7fcb55f9c700 1 --2- 172.21.15.119:0/1828483314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a91863bad0 0x55a918649db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.270+0000 7fcb55f9c700 1 -- 172.21.15.119:0/1828483314 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9183b8d80 con 0x55a918641380 2023-12-07T10:23:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.271+0000 7fcb55f9c700 1 --2- 172.21.15.119:0/1828483314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a918641380 0x55a91864a830 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcb480094e0 tx=0x7fcb48005500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff5a83d421a3b0fe server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.271+0000 7fcb38ff9700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:23:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.271+0000 7fcb38ff9700 1 -- 172.21.15.119:0/1828483314 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb48027040 con 0x55a918641380 2023-12-07T10:23:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.271+0000 7fcb6b2b4b80 1 -- 172.21.15.119:0/1828483314 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb40004cf0 con 0x55a918641380 2023-12-07T10:23:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.271+0000 7fcb38ff9700 1 -- 172.21.15.119:0/1828483314 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fcb4801e460 con 0x55a918641380 2023-12-07T10:23:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.271+0000 7fcb38ff9700 1 -- 172.21.15.119:0/1828483314 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb48007e10 con 0x55a918641380 2023-12-07T10:23:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.271+0000 7fcb6b2b4b80 10 client.26650 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.271+0000 7fcb6b2b4b80 1 -- 172.21.15.119:0/1828483314 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55a91840c5d0 con 0x55a918641380 2023-12-07T10:23:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.271+0000 7fcb38ff9700 1 -- 172.21.15.119:0/1828483314 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb40004cf0 con 0x55a918641380 2023-12-07T10:23:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.272+0000 7fcb38ff9700 1 -- 172.21.15.119:0/1828483314 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fcb4802c5e0 con 0x55a918641380 2023-12-07T10:23:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.272+0000 7fcb6b2b4b80 10 client.26650 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:33.279 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.272+0000 7fcb6b2b4b80 10 client.26650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:23:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.272+0000 7fcb6b2b4b80 20 client.26650 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.272+0000 7fcb6b2b4b80 1 --2- 172.21.15.119:0/1828483314 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a918655030 0x55a9186add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.272+0000 7fcb6b2b4b80 4 client.26650 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:23:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.272+0000 7fcb6b2b4b80 1 -- 172.21.15.119:0/1828483314 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a918555f70 con 0x55a918655030 2023-12-07T10:23:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.272+0000 7fcb5579b700 1 --2- 172.21.15.119:0/1828483314 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a918655030 0x55a9186add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.272+0000 7fcb38ff9700 10 client.26650 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.272+0000 7fcb5579b700 1 --2- 172.21.15.119:0/1828483314 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a918655030 0x55a9186add30 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1340e78c47402984 server_cookie=caef6b002cf94ea6 in_seq=0 out_seq=0 2023-12-07T10:23:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.326+0000 7fcb38ff9700 1 -- 172.21.15.119:0/1828483314 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a918555f70 con 0x55a918655030 2023-12-07T10:23:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.326+0000 7fcb38ff9700 10 client.26650 handle_command_reply: tid=0 2023-12-07T10:23:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.326+0000 7fcb6b2b4b80 1 client.26650 shutdown 2023-12-07T10:23:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.327+0000 7fcb6b2b4b80 1 -- 172.21.15.119:0/1828483314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a918641380 msgr2=0x55a91864a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.327+0000 7fcb6b2b4b80 1 --2- 172.21.15.119:0/1828483314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a918641380 0x55a91864a830 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcb480094e0 tx=0x7fcb48005500 comp rx=0 tx=0).stop 2023-12-07T10:23:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.327+0000 7fcb6b2b4b80 1 -- 172.21.15.119:0/1828483314 shutdown_connections 2023-12-07T10:23:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.327+0000 7fcb6b2b4b80 1 --2- 172.21.15.119:0/1828483314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a918641380 0x55a91864a830 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.327+0000 7fcb6b2b4b80 1 --2- 172.21.15.119:0/1828483314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a91863bad0 0x55a918649db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.327+0000 7fcb6b2b4b80 1 --2- 172.21.15.119:0/1828483314 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a918655030 0x55a9186add30 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.327+0000 7fcb6b2b4b80 1 --2- 172.21.15.119:0/1828483314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a918640a60 0x55a91864a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.328 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.327+0000 7fcb6b2b4b80 1 -- 172.21.15.119:0/1828483314 >> 172.21.15.119:0/1828483314 conn(0x55a91856a300 msgr2=0x55a918642ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.327+0000 7fcb38ff9700 0 client.26650 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.327+0000 7fcb6b2b4b80 1 -- 172.21.15.119:0/1828483314 shutdown_connections 2023-12-07T10:23:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.328+0000 7fcb6b2b4b80 1 -- 172.21.15.119:0/1828483314 wait complete. 2023-12-07T10:23:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.328+0000 7fcb6b2b4b80 20 client.26650 trim_cache size 0 max 16384 2023-12-07T10:23:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:23:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:23:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:23:33.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:23:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.328+0000 7fcb579ff700 1 -- 172.21.15.119:0/1842672517 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fcb3c0a1d00 msgr2=0x7fcb3c0a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.328+0000 7fcb579ff700 1 --2- 172.21.15.119:0/1842672517 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fcb3c0a1d00 0x7fcb3c0a41c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55a918637670 tx=0x7fcb480093a0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.328+0000 7fcb579ff700 1 -- 172.21.15.119:0/1842672517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb50073a00 msgr2=0x7fcb5019fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.328+0000 7fcb579ff700 1 --2- 172.21.15.119:0/1842672517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb50073a00 0x7fcb5019fa00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00e9f0 tx=0x7fcb4c00ed00 comp rx=0 tx=0).stop 2023-12-07T10:23:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.328+0000 7fcb579ff700 1 -- 172.21.15.119:0/1842672517 shutdown_connections 2023-12-07T10:23:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.328+0000 7fcb579ff700 1 --2- 172.21.15.119:0/1842672517 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fcb3c0a1d00 0x7fcb3c0a41c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.328+0000 7fcb579ff700 1 --2- 172.21.15.119:0/1842672517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb5010dd50 0x7fcb50199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.328+0000 7fcb579ff700 1 --2- 172.21.15.119:0/1842672517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb50073a00 0x7fcb5019fa00 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.328+0000 7fcb579ff700 1 --2- 172.21.15.119:0/1842672517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb50074320 0x7fcb5019ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.328+0000 7fcb579ff700 1 -- 172.21.15.119:0/1842672517 >> 172.21.15.119:0/1842672517 conn(0x7fcb500fcfe0 msgr2=0x7fcb500fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.329+0000 7fcb579ff700 1 -- 172.21.15.119:0/1842672517 shutdown_connections 2023-12-07T10:23:33.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:33.329+0000 7fcb579ff700 1 -- 172.21.15.119:0/1842672517 wait complete. 2023-12-07T10:23:33.549 INFO:tasks.workunit.client.0.smithi119.stdout: 1 169340 8.75 MB/sec warmup 106 sec latency 32.455 ms 2023-12-07T10:23:34.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:33 smithi119 ceph-mon[97908]: pgmap v97: 129 pgs: 129 active+clean; 103 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.4 MiB/s wr, 211 op/s 2023-12-07T10:23:34.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:33 smithi159 ceph-mon[134726]: pgmap v97: 129 pgs: 129 active+clean; 103 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.4 MiB/s wr, 211 op/s 2023-12-07T10:23:34.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:33 smithi153 ceph-mon[133625]: pgmap v97: 129 pgs: 129 active+clean; 103 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.4 MiB/s wr, 211 op/s 2023-12-07T10:23:34.549 INFO:tasks.workunit.client.0.smithi119.stdout: 1 170826 8.77 MB/sec warmup 107 sec latency 10.388 ms 2023-12-07T10:23:34.873 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-07T10:23:35.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.146+0000 7f9aa3fff700 1 -- 172.21.15.119:0/916314524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a9c101f10 msgr2=0x7f9a9c10e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:35.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.146+0000 7f9aa3fff700 1 --2- 172.21.15.119:0/916314524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a9c101f10 0x7f9a9c10e3d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9a9400b0d0 tx=0x7f9a9400b3e0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.146+0000 7f9aa3fff700 1 -- 172.21.15.119:0/916314524 shutdown_connections 2023-12-07T10:23:35.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.146+0000 7f9aa3fff700 1 --2- 172.21.15.119:0/916314524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a9c101f10 0x7f9a9c10e3d0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.146+0000 7f9aa3fff700 1 --2- 172.21.15.119:0/916314524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a9c107610 0x7f9a9c1079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.146+0000 7f9aa3fff700 1 --2- 172.21.15.119:0/916314524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a9c101550 0x7f9a9c1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.146+0000 7f9aa3fff700 1 -- 172.21.15.119:0/916314524 >> 172.21.15.119:0/916314524 conn(0x7f9a9c0fd020 msgr2=0x7f9a9c0ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:35.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.146+0000 7f9aa3fff700 1 -- 172.21.15.119:0/916314524 shutdown_connections 2023-12-07T10:23:35.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.146+0000 7f9aa3fff700 1 -- 172.21.15.119:0/916314524 wait complete. 2023-12-07T10:23:35.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.148+0000 7f9aa3fff700 1 Processor -- start 2023-12-07T10:23:35.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.148+0000 7f9aa3fff700 1 -- start start 2023-12-07T10:23:35.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.148+0000 7f9aa3fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a9c101550 0x7f9a9c19d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:35.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.148+0000 7f9aa3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a9c101f10 0x7f9a9c19dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:35.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.148+0000 7f9aa3fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a9c107610 0x7f9a9c1978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:35.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.148+0000 7f9aa3fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c197f10 con 0x7f9a9c101f10 2023-12-07T10:23:35.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.148+0000 7f9aa3fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c198080 con 0x7f9a9c107610 2023-12-07T10:23:35.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.149+0000 7f9aa3fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c198240 con 0x7f9a9c101550 2023-12-07T10:23:35.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.149+0000 7f9aa1d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a9c101550 0x7f9a9c19d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:35.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.149+0000 7f9aa1d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a9c101550 0x7f9a9c19d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41504/0 (socket says 172.21.15.119:41504) 2023-12-07T10:23:35.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.149+0000 7f9aa1d9b700 1 -- 172.21.15.119:0/3192432786 learned_addr learned my addr 172.21.15.119:0/3192432786 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:35.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.149+0000 7f9aa259c700 1 --2- 172.21.15.119:0/3192432786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a9c107610 0x7f9a9c1978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:35.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.149+0000 7f9aa1d9b700 1 -- 172.21.15.119:0/3192432786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a9c107610 msgr2=0x7f9a9c1978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:35.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.149+0000 7f9aa1d9b700 1 --2- 172.21.15.119:0/3192432786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a9c107610 0x7f9a9c1978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.149+0000 7f9aa1d9b700 1 -- 172.21.15.119:0/3192432786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a9c101f10 msgr2=0x7f9a9c19dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:35.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.149+0000 7f9aa1d9b700 1 --2- 172.21.15.119:0/3192432786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a9c101f10 0x7f9a9c19dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.149+0000 7f9aa1d9b700 1 -- 172.21.15.119:0/3192432786 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a98009520 con 0x7f9a9c101550 2023-12-07T10:23:35.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.149+0000 7f9aa1d9b700 1 --2- 172.21.15.119:0/3192432786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a9c101550 0x7f9a9c19d720 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9a9800eb30 tx=0x7f9a9800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:35.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.149+0000 7f9a92ffd700 1 -- 172.21.15.119:0/3192432786 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a98009880 con 0x7f9a9c101550 2023-12-07T10:23:35.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.150+0000 7f9aa3fff700 1 -- 172.21.15.119:0/3192432786 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a94009d00 con 0x7f9a9c101550 2023-12-07T10:23:35.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.150+0000 7f9a92ffd700 1 -- 172.21.15.119:0/3192432786 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a980044e0 con 0x7f9a9c101550 2023-12-07T10:23:35.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.150+0000 7f9a92ffd700 1 -- 172.21.15.119:0/3192432786 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a98003e30 con 0x7f9a9c101550 2023-12-07T10:23:35.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.150+0000 7f9aa3fff700 1 -- 172.21.15.119:0/3192432786 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a9c198710 con 0x7f9a9c101550 2023-12-07T10:23:35.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.153+0000 7f9a92ffd700 1 -- 172.21.15.119:0/3192432786 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f9a98004000 con 0x7f9a9c101550 2023-12-07T10:23:35.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.155+0000 7f9ab7831b80 1 Processor -- start 2023-12-07T10:23:35.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.155+0000 7f9ab7831b80 1 -- start start 2023-12-07T10:23:35.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.155+0000 7f9a92ffd700 1 --2- 172.21.15.119:0/3192432786 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9a880a1c80 0x7f9a880a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:35.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.155+0000 7f9ab7831b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eeadeb9f50 0x55eeadeba330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:35.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.155+0000 7f9a92ffd700 1 -- 172.21.15.119:0/3192432786 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9a98014070 con 0x7f9a9c101550 2023-12-07T10:23:35.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.155+0000 7f9ab7831b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeadeba870 0x55eeadebe020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:35.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.155+0000 7f9ab7831b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeadebe560 0x55eeadebe980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:35.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.155+0000 7f9ab7831b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeade297c0 con 0x55eeadebe560 2023-12-07T10:23:35.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.155+0000 7f9ab7831b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeade107a0 con 0x55eeadeba870 2023-12-07T10:23:35.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.155+0000 7f9ab7831b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeade0e0c0 con 0x55eeadeb9f50 2023-12-07T10:23:35.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.156+0000 7f9aa259c700 1 --2- 172.21.15.119:0/3192432786 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9a880a1c80 0x7f9a880a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:35.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.156+0000 7f9aa1d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eeadeb9f50 0x55eeadeba330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:35.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.156+0000 7f9aa1d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eeadeb9f50 0x55eeadeba330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41512/0 (socket says 172.21.15.119:41512) 2023-12-07T10:23:35.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.156+0000 7f9aa1d9b700 1 -- 172.21.15.119:0/103029714 learned_addr learned my addr 172.21.15.119:0/103029714 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:35.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.156+0000 7f9aa259c700 1 --2- 172.21.15.119:0/103029714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeadebe560 0x55eeadebe980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:35.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.156+0000 7f9aa1d9b700 1 -- 172.21.15.119:0/103029714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeadeba870 msgr2=0x55eeadebe020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:35.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.156+0000 7f9aa159a700 1 --2- 172.21.15.119:0/103029714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeadeba870 0x55eeadebe020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:35.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.156+0000 7f9aa1d9b700 1 --2- 172.21.15.119:0/103029714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeadeba870 0x55eeadebe020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.156+0000 7f9aa1d9b700 1 -- 172.21.15.119:0/103029714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeadebe560 msgr2=0x55eeadebe980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:35.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.156+0000 7f9aa1d9b700 1 --2- 172.21.15.119:0/103029714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeadebe560 0x55eeadebe980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.156+0000 7f9aa1d9b700 1 -- 172.21.15.119:0/103029714 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eeadc86630 con 0x55eeadeb9f50 2023-12-07T10:23:35.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.156+0000 7f9aa159a700 1 --2- 172.21.15.119:0/103029714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeadeba870 0x55eeadebe020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:23:35.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.156+0000 7f9aa1d9b700 1 --2- 172.21.15.119:0/103029714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eeadeb9f50 0x55eeadeba330 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9a9809a780 tx=0x7f9a9809a690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da25e581f3694dd5 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:35.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.157+0000 7f9aa259c700 1 --2- 172.21.15.119:0/3192432786 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9a880a1c80 0x7f9a880a4140 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9a94009610 tx=0x7f9a94009360 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:35.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.157+0000 7f9a87fff700 1 -- 172.21.15.119:0/103029714 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9809aa80 con 0x55eeadeb9f50 2023-12-07T10:23:35.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.157+0000 7f9a87fff700 1 -- 172.21.15.119:0/103029714 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a9809ebd0 con 0x55eeadeb9f50 2023-12-07T10:23:35.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.157+0000 7f9a87fff700 1 -- 172.21.15.119:0/103029714 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9801fec0 con 0x55eeadeb9f50 2023-12-07T10:23:35.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.157+0000 7f9ab7831b80 1 -- 172.21.15.119:0/103029714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eeadeb9f50 msgr2=0x55eeadeba330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:35.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.158+0000 7f9ab7831b80 1 --2- 172.21.15.119:0/103029714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eeadeb9f50 0x55eeadeba330 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9a9809a780 tx=0x7f9a9809a690 comp rx=0 tx=0).stop 2023-12-07T10:23:35.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.158+0000 7f9ab7831b80 1 -- 172.21.15.119:0/103029714 shutdown_connections 2023-12-07T10:23:35.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.158+0000 7f9ab7831b80 1 --2- 172.21.15.119:0/103029714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeadebe560 0x55eeadebe980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.158+0000 7f9ab7831b80 1 --2- 172.21.15.119:0/103029714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eeadeb9f50 0x55eeadeba330 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.158+0000 7f9ab7831b80 1 --2- 172.21.15.119:0/103029714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeadeba870 0x55eeadebe020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.158+0000 7f9ab7831b80 1 -- 172.21.15.119:0/103029714 >> 172.21.15.119:0/103029714 conn(0x55eeadde4360 msgr2=0x55eeadeb0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:35.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.158+0000 7f9ab7831b80 1 -- 172.21.15.119:0/103029714 shutdown_connections 2023-12-07T10:23:35.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.158+0000 7f9ab7831b80 1 -- 172.21.15.119:0/103029714 wait complete. 2023-12-07T10:23:35.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.158+0000 7f9ab7831b80 1 -- start start 2023-12-07T10:23:35.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.159+0000 7f9ab7831b80 1 Processor -- start 2023-12-07T10:23:35.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.159+0000 7f9ab7831b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eeadeb9f50 0x55eeadec5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:35.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.159+0000 7f9ab7831b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeadeba870 0x55eeadec3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:35.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.159+0000 7f9ab7831b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeadebe560 0x55eeadec3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:35.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.159+0000 7f9ab7831b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeade099b0 con 0x55eeadebe560 2023-12-07T10:23:35.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.159+0000 7f9ab7831b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeaddc3560 con 0x55eeadeba870 2023-12-07T10:23:35.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.159+0000 7f9ab7831b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeaddc3200 con 0x55eeadeb9f50 2023-12-07T10:23:35.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.159+0000 7f9ab7831b80 10 client.0 _ll_register_callbacks cb 0x55eeaddaa940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:35.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.159+0000 7f9aa1d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eeadeb9f50 0x55eeadec5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:35.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.160+0000 7f9aa259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeadebe560 0x55eeadec3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:35.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.160+0000 7f9aa1d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eeadeb9f50 0x55eeadec5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41526/0 (socket says 172.21.15.119:41526) 2023-12-07T10:23:35.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.160+0000 7f9aa1d9b700 1 -- 172.21.15.119:0/949583409 learned_addr learned my addr 172.21.15.119:0/949583409 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:35.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.160+0000 7f9aa159a700 1 --2- 172.21.15.119:0/949583409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeadeba870 0x55eeadec3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:35.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.160+0000 7f9aa259c700 1 -- 172.21.15.119:0/949583409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eeadeb9f50 msgr2=0x55eeadec5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:35.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.160+0000 7f9aa259c700 1 --2- 172.21.15.119:0/949583409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eeadeb9f50 0x55eeadec5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.160+0000 7f9aa259c700 1 -- 172.21.15.119:0/949583409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeadeba870 msgr2=0x55eeadec3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:35.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.160+0000 7f9aa259c700 1 --2- 172.21.15.119:0/949583409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeadeba870 0x55eeadec3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.160+0000 7f9aa259c700 1 -- 172.21.15.119:0/949583409 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a94009d00 con 0x55eeadebe560 2023-12-07T10:23:35.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.160+0000 7f9aa1d9b700 1 --2- 172.21.15.119:0/949583409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eeadeb9f50 0x55eeadec5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:35.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.160+0000 7f9aa259c700 1 --2- 172.21.15.119:0/949583409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeadebe560 0x55eeadec3770 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c19e420 tx=0x7f9a9400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb31f96fef4366c server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:35.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.160+0000 7f9a857fa700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:23:35.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.161+0000 7f9a857fa700 1 -- 172.21.15.119:0/949583409 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9400e040 con 0x55eeadebe560 2023-12-07T10:23:35.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.161+0000 7f9ab7831b80 1 -- 172.21.15.119:0/949583409 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eeadea8b00 con 0x55eeadebe560 2023-12-07T10:23:35.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.161+0000 7f9a857fa700 1 -- 172.21.15.119:0/949583409 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9a94003c00 con 0x55eeadebe560 2023-12-07T10:23:35.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.161+0000 7f9a857fa700 1 -- 172.21.15.119:0/949583409 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a94003dc0 con 0x55eeadebe560 2023-12-07T10:23:35.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.161+0000 7f9ab7831b80 10 client.16656 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:35.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.161+0000 7f9ab7831b80 1 -- 172.21.15.119:0/949583409 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55eeadc32de0 con 0x55eeadebe560 2023-12-07T10:23:35.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.161+0000 7f9a857fa700 1 -- 172.21.15.119:0/949583409 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eeadea8b00 con 0x55eeadebe560 2023-12-07T10:23:35.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.162+0000 7f9a857fa700 1 -- 172.21.15.119:0/949583409 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f9a94021690 con 0x55eeadebe560 2023-12-07T10:23:35.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.162+0000 7f9ab7831b80 10 client.16656 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:35.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.162+0000 7f9ab7831b80 10 client.16656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:23:35.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.162+0000 7f9ab7831b80 20 client.16656 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:35.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.162+0000 7f9ab7831b80 1 --2- 172.21.15.119:0/949583409 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55eeaded3830 0x55eeaded5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:35.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.162+0000 7f9ab7831b80 4 client.16656 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:23:35.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.162+0000 7f9ab7831b80 1 -- 172.21.15.119:0/949583409 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eeade847a0 con 0x55eeaded3830 2023-12-07T10:23:35.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.162+0000 7f9aa1d9b700 1 --2- 172.21.15.119:0/949583409 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55eeaded3830 0x55eeaded5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:35.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.162+0000 7f9aa1d9b700 1 --2- 172.21.15.119:0/949583409 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55eeaded3830 0x55eeaded5c50 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87fe404d6a6895f1 server_cookie=dac2d7cff0c54302 in_seq=0 out_seq=0 2023-12-07T10:23:35.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.162+0000 7f9a857fa700 10 client.16656 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:35.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.163+0000 7f9a857fa700 1 -- 172.21.15.119:0/949583409 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eeade847a0 con 0x55eeaded3830 2023-12-07T10:23:35.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.163+0000 7f9a857fa700 10 client.16656 handle_command_reply: tid=0 2023-12-07T10:23:35.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.163+0000 7f9ab7831b80 1 client.16656 shutdown 2023-12-07T10:23:35.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.163+0000 7f9ab7831b80 1 -- 172.21.15.119:0/949583409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeadebe560 msgr2=0x55eeadec3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:35.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.163+0000 7f9ab7831b80 1 --2- 172.21.15.119:0/949583409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeadebe560 0x55eeadec3770 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c19e420 tx=0x7f9a9400b040 comp rx=0 tx=0).stop 2023-12-07T10:23:35.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.163+0000 7f9ab7831b80 1 -- 172.21.15.119:0/949583409 shutdown_connections 2023-12-07T10:23:35.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.163+0000 7f9ab7831b80 1 --2- 172.21.15.119:0/949583409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeadebe560 0x55eeadec3770 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.163+0000 7f9ab7831b80 1 --2- 172.21.15.119:0/949583409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eeadeb9f50 0x55eeadec5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.164+0000 7f9ab7831b80 1 --2- 172.21.15.119:0/949583409 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55eeaded3830 0x55eeaded5c50 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.164+0000 7f9ab7831b80 1 --2- 172.21.15.119:0/949583409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeadeba870 0x55eeadec3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.164+0000 7f9a857fa700 0 client.16656 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:35.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.164+0000 7f9ab7831b80 1 -- 172.21.15.119:0/949583409 >> 172.21.15.119:0/949583409 conn(0x55eeadde4360 msgr2=0x55eeadeaad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:35.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.164+0000 7f9ab7831b80 1 -- 172.21.15.119:0/949583409 shutdown_connections 2023-12-07T10:23:35.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.164+0000 7f9ab7831b80 1 -- 172.21.15.119:0/949583409 wait complete. 2023-12-07T10:23:35.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.164+0000 7f9ab7831b80 20 client.16656 trim_cache size 0 max 16384 2023-12-07T10:23:35.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.186+0000 7f9ab7831b80 1 Processor -- start 2023-12-07T10:23:35.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.186+0000 7f9ab7831b80 1 -- start start 2023-12-07T10:23:35.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.186+0000 7f9ab7831b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeadeb9f50 0x55eeadea9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:35.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.186+0000 7f9ab7831b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeadeba870 0x55eeadeda570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:35.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.186+0000 7f9ab7831b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eeadebe560 0x55eeadebd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:35.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.186+0000 7f9ab7831b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeadc88920 con 0x55eeadeb9f50 2023-12-07T10:23:35.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.186+0000 7f9ab7831b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeadec6540 con 0x55eeadeba870 2023-12-07T10:23:35.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.186+0000 7f9ab7831b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeadeb17d0 con 0x55eeadebe560 2023-12-07T10:23:35.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.186+0000 7f9aa1d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeadeb9f50 0x55eeadea9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:35.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.187+0000 7f9aa159a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeadeba870 0x55eeadeda570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:35.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.187+0000 7f9aa1d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeadeb9f50 0x55eeadea9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34678/0 (socket says 172.21.15.119:34678) 2023-12-07T10:23:35.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.187+0000 7f9aa1d9b700 1 -- 172.21.15.119:0/804067667 learned_addr learned my addr 172.21.15.119:0/804067667 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:35.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.187+0000 7f9aa259c700 1 --2- 172.21.15.119:0/804067667 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eeadebe560 0x55eeadebd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:35.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.187+0000 7f9aa1d9b700 1 -- 172.21.15.119:0/804067667 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eeadebe560 msgr2=0x55eeadebd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:35.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.187+0000 7f9aa1d9b700 1 --2- 172.21.15.119:0/804067667 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eeadebe560 0x55eeadebd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.187+0000 7f9aa1d9b700 1 -- 172.21.15.119:0/804067667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeadeba870 msgr2=0x55eeadeda570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:35.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.187+0000 7f9aa1d9b700 1 --2- 172.21.15.119:0/804067667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeadeba870 0x55eeadeda570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.187+0000 7f9aa1d9b700 1 -- 172.21.15.119:0/804067667 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eeadc1bc60 con 0x55eeadeb9f50 2023-12-07T10:23:35.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.187+0000 7f9aa1d9b700 1 --2- 172.21.15.119:0/804067667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeadeb9f50 0x55eeadea9d70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9a980974a0 tx=0x7f9a9809a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd141faef40b8b51 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:35.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.187+0000 7f9a90ff9700 1 -- 172.21.15.119:0/804067667 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9809aa80 con 0x55eeadeb9f50 2023-12-07T10:23:35.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.188+0000 7f9a90ff9700 1 -- 172.21.15.119:0/804067667 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a98098e20 con 0x55eeadeb9f50 2023-12-07T10:23:35.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.188+0000 7f9a90ff9700 1 -- 172.21.15.119:0/804067667 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9809ace0 con 0x55eeadeb9f50 2023-12-07T10:23:35.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.188+0000 7f9ab7831b80 1 -- 172.21.15.119:0/804067667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeadeb9f50 msgr2=0x55eeadea9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:35.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.188+0000 7f9ab7831b80 1 --2- 172.21.15.119:0/804067667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeadeb9f50 0x55eeadea9d70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9a980974a0 tx=0x7f9a9809a510 comp rx=0 tx=0).stop 2023-12-07T10:23:35.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.188+0000 7f9ab7831b80 1 -- 172.21.15.119:0/804067667 shutdown_connections 2023-12-07T10:23:35.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.188+0000 7f9ab7831b80 1 --2- 172.21.15.119:0/804067667 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eeadebe560 0x55eeadebd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.188+0000 7f9ab7831b80 1 --2- 172.21.15.119:0/804067667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeadeb9f50 0x55eeadea9d70 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.188+0000 7f9ab7831b80 1 --2- 172.21.15.119:0/804067667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeadeba870 0x55eeadeda570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.188+0000 7f9ab7831b80 1 -- 172.21.15.119:0/804067667 >> 172.21.15.119:0/804067667 conn(0x55eeadde4360 msgr2=0x55eeadebcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:35.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.188+0000 7f9ab7831b80 1 -- 172.21.15.119:0/804067667 shutdown_connections 2023-12-07T10:23:35.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.189+0000 7f9ab7831b80 1 -- 172.21.15.119:0/804067667 wait complete. 2023-12-07T10:23:35.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.189+0000 7f9ab7831b80 1 -- start start 2023-12-07T10:23:35.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.189+0000 7f9ab7831b80 1 Processor -- start 2023-12-07T10:23:35.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.190+0000 7f9ab7831b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeadeb9f50 0x55eeadec6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:35.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.190+0000 7f9ab7831b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eeadeba870 0x55eeadec6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:35.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.190+0000 7f9ab7831b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeadebe560 0x55eeadf28ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:35.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.190+0000 7f9ab7831b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeadf29220 con 0x55eeadeb9f50 2023-12-07T10:23:35.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.190+0000 7f9ab7831b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeadf29360 con 0x55eeadebe560 2023-12-07T10:23:35.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.190+0000 7f9ab7831b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeadf294a0 con 0x55eeadeba870 2023-12-07T10:23:35.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.190+0000 7f9ab7831b80 10 client.0 _ll_register_callbacks cb 0x55eeaddaa940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:35.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.190+0000 7f9aa1d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeadeb9f50 0x55eeadec6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:35.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.190+0000 7f9aa1d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeadeb9f50 0x55eeadec6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34684/0 (socket says 172.21.15.119:34684) 2023-12-07T10:23:35.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.190+0000 7f9aa159a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eeadeba870 0x55eeadec6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:35.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.190+0000 7f9aa1d9b700 1 -- 172.21.15.119:0/2793675289 learned_addr learned my addr 172.21.15.119:0/2793675289 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:35.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.190+0000 7f9aa259c700 1 --2- 172.21.15.119:0/2793675289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeadebe560 0x55eeadf28ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:35.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.190+0000 7f9aa1d9b700 1 -- 172.21.15.119:0/2793675289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eeadeba870 msgr2=0x55eeadec6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:35.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.190+0000 7f9aa1d9b700 1 --2- 172.21.15.119:0/2793675289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eeadeba870 0x55eeadec6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.190+0000 7f9aa1d9b700 1 -- 172.21.15.119:0/2793675289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeadebe560 msgr2=0x55eeadf28ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:35.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.190+0000 7f9aa1d9b700 1 --2- 172.21.15.119:0/2793675289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeadebe560 0x55eeadf28ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.190+0000 7f9aa1d9b700 1 -- 172.21.15.119:0/2793675289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eeadc86630 con 0x55eeadeb9f50 2023-12-07T10:23:35.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.191+0000 7f9aa1d9b700 1 --2- 172.21.15.119:0/2793675289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeadeb9f50 0x55eeadec6920 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9a9809a780 tx=0x7f9a9801ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533b5b0ceec154bb server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:35.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.191+0000 7f9a857fa700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:23:35.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.191+0000 7f9a857fa700 1 -- 172.21.15.119:0/2793675289 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9809aa80 con 0x55eeadeb9f50 2023-12-07T10:23:35.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.191+0000 7f9a857fa700 1 -- 172.21.15.119:0/2793675289 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9a980850b0 con 0x55eeadeb9f50 2023-12-07T10:23:35.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.191+0000 7f9ab7831b80 1 -- 172.21.15.119:0/2793675289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a9809ebd0 con 0x55eeadeb9f50 2023-12-07T10:23:35.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.191+0000 7f9a857fa700 1 -- 172.21.15.119:0/2793675289 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a98085270 con 0x55eeadeb9f50 2023-12-07T10:23:35.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.191+0000 7f9ab7831b80 10 client.16662 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:35.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.191+0000 7f9ab7831b80 1 -- 172.21.15.119:0/2793675289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55eeadc59250 con 0x55eeadeb9f50 2023-12-07T10:23:35.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.191+0000 7f9a857fa700 1 -- 172.21.15.119:0/2793675289 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a9809ebd0 con 0x55eeadeb9f50 2023-12-07T10:23:35.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.192+0000 7f9a857fa700 1 -- 172.21.15.119:0/2793675289 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f9a9809ad70 con 0x55eeadeb9f50 2023-12-07T10:23:35.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.192+0000 7f9ab7831b80 10 client.16662 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:35.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.192+0000 7f9ab7831b80 10 client.16662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:23:35.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.192+0000 7f9ab7831b80 20 client.16662 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:35.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.192+0000 7f9ab7831b80 1 --2- 172.21.15.119:0/2793675289 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55eeadf2a7c0 0x55eeadf2cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:35.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.192+0000 7f9ab7831b80 4 client.16662 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-12-07T10:23:35.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.192+0000 7f9ab7831b80 1 -- 172.21.15.119:0/2793675289 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55eeaddcffd0 con 0x55eeadf2a7c0 2023-12-07T10:23:35.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.192+0000 7f9aa159a700 1 --2- 172.21.15.119:0/2793675289 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55eeadf2a7c0 0x55eeadf2cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:35.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.193+0000 7f9aa159a700 1 --2- 172.21.15.119:0/2793675289 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55eeadf2a7c0 0x55eeadf2cbe0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45dbdba6b2f4ffe2 server_cookie=e6bb1b5ff11bc88f in_seq=0 out_seq=0 2023-12-07T10:23:35.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.193+0000 7f9a857fa700 10 client.16662 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:35.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.193+0000 7f9a857fa700 1 -- 172.21.15.119:0/2793675289 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55eeaddcffd0 con 0x55eeadf2a7c0 2023-12-07T10:23:35.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.193+0000 7f9a857fa700 10 client.16662 handle_command_reply: tid=0 2023-12-07T10:23:35.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.193+0000 7f9ab7831b80 1 client.16662 shutdown 2023-12-07T10:23:35.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.193+0000 7f9ab7831b80 1 -- 172.21.15.119:0/2793675289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeadeb9f50 msgr2=0x55eeadec6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:35.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.193+0000 7f9ab7831b80 1 --2- 172.21.15.119:0/2793675289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeadeb9f50 0x55eeadec6920 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9a9809a780 tx=0x7f9a9801ff90 comp rx=0 tx=0).stop 2023-12-07T10:23:35.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.193+0000 7f9ab7831b80 1 -- 172.21.15.119:0/2793675289 shutdown_connections 2023-12-07T10:23:35.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.193+0000 7f9ab7831b80 1 --2- 172.21.15.119:0/2793675289 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55eeadf2a7c0 0x55eeadf2cbe0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.193+0000 7f9ab7831b80 1 --2- 172.21.15.119:0/2793675289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55eeadebe560 0x55eeadf28ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.193+0000 7f9ab7831b80 1 --2- 172.21.15.119:0/2793675289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeadeb9f50 0x55eeadec6920 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.193+0000 7f9ab7831b80 1 --2- 172.21.15.119:0/2793675289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eeadeba870 0x55eeadec6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.193+0000 7f9a857fa700 0 client.16662 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:35.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.193+0000 7f9ab7831b80 1 -- 172.21.15.119:0/2793675289 >> 172.21.15.119:0/2793675289 conn(0x55eeadde4360 msgr2=0x55eeadebd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:35.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.193+0000 7f9ab7831b80 1 -- 172.21.15.119:0/2793675289 shutdown_connections 2023-12-07T10:23:35.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.193+0000 7f9ab7831b80 1 -- 172.21.15.119:0/2793675289 wait complete. 2023-12-07T10:23:35.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.193+0000 7f9ab7831b80 20 client.16662 trim_cache size 0 max 16384 2023-12-07T10:23:35.199 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:23:35.199 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "no active scrubs running", 2023-12-07T10:23:35.199 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": {} 2023-12-07T10:23:35.199 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:23:35.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.194+0000 7f9aa3fff700 1 -- 172.21.15.119:0/3192432786 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9a880a1c80 msgr2=0x7f9a880a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:35.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.194+0000 7f9aa3fff700 1 --2- 172.21.15.119:0/3192432786 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9a880a1c80 0x7f9a880a4140 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9a94009610 tx=0x7f9a94009360 comp rx=0 tx=0).stop 2023-12-07T10:23:35.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.194+0000 7f9aa3fff700 1 -- 172.21.15.119:0/3192432786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a9c101550 msgr2=0x7f9a9c19d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:35.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.194+0000 7f9aa3fff700 1 --2- 172.21.15.119:0/3192432786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a9c101550 0x7f9a9c19d720 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9a9800eb30 tx=0x7f9a9800ee40 comp rx=0 tx=0).stop 2023-12-07T10:23:35.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.194+0000 7f9aa3fff700 1 -- 172.21.15.119:0/3192432786 shutdown_connections 2023-12-07T10:23:35.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.194+0000 7f9aa3fff700 1 --2- 172.21.15.119:0/3192432786 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9a880a1c80 0x7f9a880a4140 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.194+0000 7f9aa3fff700 1 --2- 172.21.15.119:0/3192432786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9a9c107610 0x7f9a9c1978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.194+0000 7f9aa3fff700 1 --2- 172.21.15.119:0/3192432786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a9c101550 0x7f9a9c19d720 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.194+0000 7f9aa3fff700 1 --2- 172.21.15.119:0/3192432786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a9c101f10 0x7f9a9c19dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.194+0000 7f9aa3fff700 1 -- 172.21.15.119:0/3192432786 >> 172.21.15.119:0/3192432786 conn(0x7f9a9c0fd020 msgr2=0x7f9a9c0fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:35.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.194+0000 7f9aa3fff700 1 -- 172.21.15.119:0/3192432786 shutdown_connections 2023-12-07T10:23:35.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.194+0000 7f9aa3fff700 1 -- 172.21.15.119:0/3192432786 wait complete. 2023-12-07T10:23:35.216 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-07T10:23:35.216 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:23:35.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.449+0000 7fe8d8bff700 1 -- 172.21.15.119:0/1808497337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8d4107560 msgr2=0x7fe8d4107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:35.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.449+0000 7fe8d8bff700 1 --2- 172.21.15.119:0/1808497337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8d4107560 0x7fe8d4107940 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40099c0 tx=0x7fe8c4009cd0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.450+0000 7fe8d8bff700 1 -- 172.21.15.119:0/1808497337 shutdown_connections 2023-12-07T10:23:35.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.450+0000 7fe8d8bff700 1 --2- 172.21.15.119:0/1808497337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8d4069010 0x7fe8d410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.450+0000 7fe8d8bff700 1 --2- 172.21.15.119:0/1808497337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8d4107560 0x7fe8d4107940 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.450+0000 7fe8d8bff700 1 --2- 172.21.15.119:0/1808497337 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8d4068650 0x7fe8d4068ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.450+0000 7fe8d8bff700 1 -- 172.21.15.119:0/1808497337 >> 172.21.15.119:0/1808497337 conn(0x7fe8d4077220 msgr2=0x7fe8d4077630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:35.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.450+0000 7fe8d8bff700 1 -- 172.21.15.119:0/1808497337 shutdown_connections 2023-12-07T10:23:35.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.450+0000 7fe8d8bff700 1 -- 172.21.15.119:0/1808497337 wait complete. 2023-12-07T10:23:35.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.451+0000 7fe8d8bff700 1 Processor -- start 2023-12-07T10:23:35.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.451+0000 7fe8d8bff700 1 -- start start 2023-12-07T10:23:35.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.451+0000 7fe8d8bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8d4068650 0x7fe8d419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:35.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.451+0000 7fe8d8bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8d4069010 0x7fe8d419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:35.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.451+0000 7fe8d8bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8d4107560 0x7fe8d41977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:35.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.451+0000 7fe8d8bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d4197e20 con 0x7fe8d4068650 2023-12-07T10:23:35.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.451+0000 7fe8d8bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d4197f90 con 0x7fe8d4069010 2023-12-07T10:23:35.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.451+0000 7fe8d8bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d4198150 con 0x7fe8d4107560 2023-12-07T10:23:35.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.452+0000 7fe8d259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8d4068650 0x7fe8d419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:35.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.452+0000 7fe8d2d9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8d4107560 0x7fe8d41977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:35.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.452+0000 7fe8d1d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8d4069010 0x7fe8d419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:35.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.452+0000 7fe8d2d9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8d4107560 0x7fe8d41977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41570/0 (socket says 172.21.15.119:41570) 2023-12-07T10:23:35.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.452+0000 7fe8d2d9d700 1 -- 172.21.15.119:0/932587890 learned_addr learned my addr 172.21.15.119:0/932587890 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:35.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.452+0000 7fe8d259c700 1 -- 172.21.15.119:0/932587890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8d4107560 msgr2=0x7fe8d41977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:35.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.452+0000 7fe8d259c700 1 --2- 172.21.15.119:0/932587890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8d4107560 0x7fe8d41977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.452+0000 7fe8d259c700 1 -- 172.21.15.119:0/932587890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8d4069010 msgr2=0x7fe8d419dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:35.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.452+0000 7fe8d259c700 1 --2- 172.21.15.119:0/932587890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8d4069010 0x7fe8d419dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.452+0000 7fe8d259c700 1 -- 172.21.15.119:0/932587890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c40095f0 con 0x7fe8d4068650 2023-12-07T10:23:35.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.452+0000 7fe8d1d9b700 1 --2- 172.21.15.119:0/932587890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8d4069010 0x7fe8d419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:35.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.452+0000 7fe8d2d9d700 1 --2- 172.21.15.119:0/932587890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8d4107560 0x7fe8d41977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:35.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.452+0000 7fe8d259c700 1 --2- 172.21.15.119:0/932587890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8d4068650 0x7fe8d419d680 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40098e0 tx=0x7fe8c400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:35.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.453+0000 7fe8c37fe700 1 -- 172.21.15.119:0/932587890 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c401c070 con 0x7fe8d4068650 2023-12-07T10:23:35.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.453+0000 7fe8c37fe700 1 -- 172.21.15.119:0/932587890 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8c400fd00 con 0x7fe8d4068650 2023-12-07T10:23:35.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.453+0000 7fe8d8bff700 1 -- 172.21.15.119:0/932587890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d41983d0 con 0x7fe8d4068650 2023-12-07T10:23:35.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.453+0000 7fe8c37fe700 1 -- 172.21.15.119:0/932587890 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4022c60 con 0x7fe8d4068650 2023-12-07T10:23:35.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.453+0000 7fe8d8bff700 1 -- 172.21.15.119:0/932587890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d41a4670 con 0x7fe8d4068650 2023-12-07T10:23:35.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.454+0000 7fe8d8bff700 1 -- 172.21.15.119:0/932587890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d41099e0 con 0x7fe8d4068650 2023-12-07T10:23:35.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.456+0000 7fe8c37fe700 1 -- 172.21.15.119:0/932587890 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fe8c4020070 con 0x7fe8d4068650 2023-12-07T10:23:35.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.457+0000 7fe8c37fe700 1 --2- 172.21.15.119:0/932587890 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe8bc0a1d00 0x7fe8bc0a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:35.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.457+0000 7fe8c37fe700 1 -- 172.21.15.119:0/932587890 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fe8c40ce830 con 0x7fe8d4068650 2023-12-07T10:23:35.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.458+0000 7fe8d1d9b700 1 --2- 172.21.15.119:0/932587890 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe8bc0a1d00 0x7fe8bc0a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:35.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.458+0000 7fe8d1d9b700 1 --2- 172.21.15.119:0/932587890 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe8bc0a1d00 0x7fe8bc0a41c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4198bd0 tx=0x7fe8c8009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:35.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.462+0000 7fe8c37fe700 1 -- 172.21.15.119:0/932587890 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe8c4097650 con 0x7fe8d4068650 2023-12-07T10:23:35.549 INFO:tasks.workunit.client.0.smithi119.stdout: 1 172771 8.76 MB/sec warmup 108 sec latency 16.093 ms 2023-12-07T10:23:35.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.731+0000 7fe8d8bff700 1 -- 172.21.15.119:0/932587890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8d4102c00 con 0x7fe8d4068650 2023-12-07T10:23:35.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.731+0000 7fe8c37fe700 1 -- 172.21.15.119:0/932587890 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+11401 (secure 0 0 0) 0x7fe8c409ac70 con 0x7fe8d4068650 2023-12-07T10:23:35.732 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:23:35.732 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-07T10:17:41.278311+0000","modified":"2023-12-07T10:22:55.653365+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24517,"mds_3":24505,"mds_4":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14535":{"gid":14535,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6841/1791998028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1791998028},{"type":"v1","addr":"172.21.15.119:6841","nonce":1791998028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24505":{"gid":24505,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/2925503246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2925503246},{"type":"v1","addr":"172.21.15.153:6837","nonce":2925503246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6839/3984962900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3984962900},{"type":"v1","addr":"172.21.15.159:6839","nonce":3984962900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.119:6839/1920762602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1920762602},{"type":"v1","addr":"172.21.15.119:6839","nonce":1920762602}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/566825184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":566825184},{"type":"v1","addr":"172.21.15.153:6839","nonce":566825184}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-07T10:23:35.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.734+0000 7fe8d8bff700 1 -- 172.21.15.119:0/932587890 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe8bc0a1d00 msgr2=0x7fe8bc0a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:35.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.734+0000 7fe8d8bff700 1 --2- 172.21.15.119:0/932587890 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe8bc0a1d00 0x7fe8bc0a41c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4198bd0 tx=0x7fe8c8009260 comp rx=0 tx=0).stop 2023-12-07T10:23:35.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.734+0000 7fe8d8bff700 1 -- 172.21.15.119:0/932587890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8d4068650 msgr2=0x7fe8d419d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:35.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.734+0000 7fe8d8bff700 1 --2- 172.21.15.119:0/932587890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8d4068650 0x7fe8d419d680 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40098e0 tx=0x7fe8c400f690 comp rx=0 tx=0).stop 2023-12-07T10:23:35.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.734+0000 7fe8d8bff700 1 -- 172.21.15.119:0/932587890 shutdown_connections 2023-12-07T10:23:35.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.734+0000 7fe8d8bff700 1 --2- 172.21.15.119:0/932587890 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe8bc0a1d00 0x7fe8bc0a41c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.734+0000 7fe8d8bff700 1 --2- 172.21.15.119:0/932587890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8d4107560 0x7fe8d41977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.734+0000 7fe8d8bff700 1 --2- 172.21.15.119:0/932587890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8d4068650 0x7fe8d419d680 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.734+0000 7fe8d8bff700 1 --2- 172.21.15.119:0/932587890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8d4069010 0x7fe8d419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:35.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.734+0000 7fe8d8bff700 1 -- 172.21.15.119:0/932587890 >> 172.21.15.119:0/932587890 conn(0x7fe8d4077220 msgr2=0x7fe8d40ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:35.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.734+0000 7fe8d8bff700 1 -- 172.21.15.119:0/932587890 shutdown_connections 2023-12-07T10:23:35.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:35.734+0000 7fe8d8bff700 1 -- 172.21.15.119:0/932587890 wait complete. 2023-12-07T10:23:35.736 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 36 2023-12-07T10:23:35.756 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-12-07T10:23:36.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.030+0000 7f66b71ff700 1 -- 172.21.15.119:0/3947195974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66b0101f10 msgr2=0x7f66b010e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.030+0000 7f66b71ff700 1 --2- 172.21.15.119:0/3947195974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66b0101f10 0x7f66b010e3d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f66a800b0a0 tx=0x7f66a800b3b0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.030+0000 7f66b71ff700 1 -- 172.21.15.119:0/3947195974 shutdown_connections 2023-12-07T10:23:36.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.030+0000 7f66b71ff700 1 --2- 172.21.15.119:0/3947195974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66b0101f10 0x7f66b010e3d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.030+0000 7f66b71ff700 1 --2- 172.21.15.119:0/3947195974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66b0107610 0x7f66b01079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.030+0000 7f66b71ff700 1 --2- 172.21.15.119:0/3947195974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b0101550 0x7f66b01019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.030+0000 7f66b71ff700 1 -- 172.21.15.119:0/3947195974 >> 172.21.15.119:0/3947195974 conn(0x7f66b00fd020 msgr2=0x7f66b00ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:36.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.030+0000 7f66b71ff700 1 -- 172.21.15.119:0/3947195974 shutdown_connections 2023-12-07T10:23:36.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.030+0000 7f66b71ff700 1 -- 172.21.15.119:0/3947195974 wait complete. 2023-12-07T10:23:36.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.031+0000 7f66b71ff700 1 Processor -- start 2023-12-07T10:23:36.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.032+0000 7f66b71ff700 1 -- start start 2023-12-07T10:23:36.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.032+0000 7f66b71ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66b0101550 0x7f66b0073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.032+0000 7f66b71ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66b0101f10 0x7f66b0073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.032+0000 7f66b71ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b0107610 0x7f66b0074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.032+0000 7f66b71ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b00758e0 con 0x7f66b0107610 2023-12-07T10:23:36.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.032+0000 7f66b71ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b01aa4a0 con 0x7f66b0101f10 2023-12-07T10:23:36.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.032+0000 7f66b71ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b01aa660 con 0x7f66b0101550 2023-12-07T10:23:36.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.032+0000 7f66affff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66b0101f10 0x7f66b0073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.032+0000 7f66b4f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66b0101550 0x7f66b0073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.032+0000 7f66affff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66b0101f10 0x7f66b0073e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:55936/0 (socket says 172.21.15.119:55936) 2023-12-07T10:23:36.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.032+0000 7f66b4f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66b0101550 0x7f66b0073910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41582/0 (socket says 172.21.15.119:41582) 2023-12-07T10:23:36.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.032+0000 7f66affff700 1 -- 172.21.15.119:0/311717508 learned_addr learned my addr 172.21.15.119:0/311717508 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:36.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.032+0000 7f66b579c700 1 --2- 172.21.15.119:0/311717508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b0107610 0x7f66b0074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.033+0000 7f66affff700 1 -- 172.21.15.119:0/311717508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66b0101550 msgr2=0x7f66b0073910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.033+0000 7f66affff700 1 --2- 172.21.15.119:0/311717508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66b0101550 0x7f66b0073910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.033+0000 7f66affff700 1 -- 172.21.15.119:0/311717508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b0107610 msgr2=0x7f66b0074390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.033+0000 7f66affff700 1 --2- 172.21.15.119:0/311717508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b0107610 0x7f66b0074390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.033+0000 7f66affff700 1 -- 172.21.15.119:0/311717508 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a8009d00 con 0x7f66b0101f10 2023-12-07T10:23:36.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.033+0000 7f66affff700 1 --2- 172.21.15.119:0/311717508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66b0101f10 0x7f66b0073e50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f669c00e9a0 tx=0x7f669c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:36.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.033+0000 7f66adffb700 1 -- 172.21.15.119:0/311717508 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c009790 con 0x7f66b0101f10 2023-12-07T10:23:36.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.033+0000 7f66adffb700 1 -- 172.21.15.119:0/311717508 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f669c010430 con 0x7f66b0101f10 2023-12-07T10:23:36.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.033+0000 7f66b71ff700 1 -- 172.21.15.119:0/311717508 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b01aa940 con 0x7f66b0101f10 2023-12-07T10:23:36.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.033+0000 7f66b71ff700 1 -- 172.21.15.119:0/311717508 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b01aae00 con 0x7f66b0101f10 2023-12-07T10:23:36.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.033+0000 7f66adffb700 1 -- 172.21.15.119:0/311717508 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c018680 con 0x7f66b0101f10 2023-12-07T10:23:36.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.037+0000 7f66adffb700 1 -- 172.21.15.119:0/311717508 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f669c0105a0 con 0x7f66b0101f10 2023-12-07T10:23:36.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.038+0000 7f66adffb700 1 --2- 172.21.15.119:0/311717508 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f66a00a1c80 0x7f66a00a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.038+0000 7f66cab4fb80 1 Processor -- start 2023-12-07T10:23:36.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.038+0000 7f66adffb700 1 -- 172.21.15.119:0/311717508 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f669c014070 con 0x7f66b0101f10 2023-12-07T10:23:36.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.039+0000 7f66cab4fb80 1 -- start start 2023-12-07T10:23:36.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.039+0000 7f66b4f9b700 1 --2- 172.21.15.119:0/311717508 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f66a00a1c80 0x7f66a00a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.039+0000 7f66cab4fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561ec1a70b40 0x561ec1a70f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.039+0000 7f66cab4fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ec1a71460 0x561ec1a74c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.039+0000 7f66cab4fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ec1a75150 0x561ec1a75570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.039+0000 7f66cab4fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec19e47c0 con 0x561ec1a75150 2023-12-07T10:23:36.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.039+0000 7f66cab4fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec19cb7a0 con 0x561ec1a70b40 2023-12-07T10:23:36.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.039+0000 7f66cab4fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec19c90c0 con 0x561ec1a71460 2023-12-07T10:23:36.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.039+0000 7f66b4f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561ec1a70b40 0x561ec1a70f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.039+0000 7f66affff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ec1a71460 0x561ec1a74c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.039+0000 7f66b579c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ec1a75150 0x561ec1a75570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.039+0000 7f66affff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ec1a71460 0x561ec1a74c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41588/0 (socket says 172.21.15.119:41588) 2023-12-07T10:23:36.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.039+0000 7f66affff700 1 -- 172.21.15.119:0/2270796708 learned_addr learned my addr 172.21.15.119:0/2270796708 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:36.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.040+0000 7f66affff700 1 -- 172.21.15.119:0/2270796708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561ec1a70b40 msgr2=0x561ec1a70f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.040+0000 7f66affff700 1 --2- 172.21.15.119:0/2270796708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561ec1a70b40 0x561ec1a70f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.040+0000 7f66affff700 1 -- 172.21.15.119:0/2270796708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ec1a75150 msgr2=0x561ec1a75570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.040+0000 7f66affff700 1 --2- 172.21.15.119:0/2270796708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ec1a75150 0x561ec1a75570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.040+0000 7f66affff700 1 -- 172.21.15.119:0/2270796708 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ec1841630 con 0x561ec1a71460 2023-12-07T10:23:36.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.040+0000 7f66b4f9b700 1 --2- 172.21.15.119:0/311717508 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f66a00a1c80 0x7f66a00a4140 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f66a4011550 tx=0x7f66a4011910 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:36.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.040+0000 7f66affff700 1 --2- 172.21.15.119:0/2270796708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ec1a71460 0x561ec1a74c10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f669c0a2760 tx=0x7f669c09a550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9654bdf4d1b5ad91 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:36.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.040+0000 7f669affd700 1 -- 172.21.15.119:0/2270796708 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c09ea50 con 0x561ec1a71460 2023-12-07T10:23:36.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.040+0000 7f669affd700 1 -- 172.21.15.119:0/2270796708 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f669c0a2c90 con 0x561ec1a71460 2023-12-07T10:23:36.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.040+0000 7f669affd700 1 -- 172.21.15.119:0/2270796708 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c0ba2f0 con 0x561ec1a71460 2023-12-07T10:23:36.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.041+0000 7f66cab4fb80 1 -- 172.21.15.119:0/2270796708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ec1a71460 msgr2=0x561ec1a74c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.041+0000 7f66cab4fb80 1 --2- 172.21.15.119:0/2270796708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ec1a71460 0x561ec1a74c10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f669c0a2760 tx=0x7f669c09a550 comp rx=0 tx=0).stop 2023-12-07T10:23:36.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.041+0000 7f66cab4fb80 1 -- 172.21.15.119:0/2270796708 shutdown_connections 2023-12-07T10:23:36.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.041+0000 7f66cab4fb80 1 --2- 172.21.15.119:0/2270796708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ec1a75150 0x561ec1a75570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.041+0000 7f66cab4fb80 1 --2- 172.21.15.119:0/2270796708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561ec1a70b40 0x561ec1a70f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.041+0000 7f66cab4fb80 1 --2- 172.21.15.119:0/2270796708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ec1a71460 0x561ec1a74c10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.041+0000 7f66cab4fb80 1 -- 172.21.15.119:0/2270796708 >> 172.21.15.119:0/2270796708 conn(0x561ec199f360 msgr2=0x561ec1a6b130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:36.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.041+0000 7f66cab4fb80 1 -- 172.21.15.119:0/2270796708 shutdown_connections 2023-12-07T10:23:36.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.041+0000 7f66cab4fb80 1 -- 172.21.15.119:0/2270796708 wait complete. 2023-12-07T10:23:36.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.042+0000 7f66cab4fb80 1 -- start start 2023-12-07T10:23:36.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.042+0000 7f66cab4fb80 1 Processor -- start 2023-12-07T10:23:36.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.043+0000 7f66cab4fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561ec1a70b40 0x561ec1a7c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.043+0000 7f66cab4fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ec1a71460 0x561ec1a79c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.043+0000 7f66cab4fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ec1a75150 0x561ec1a7a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.043+0000 7f66cab4fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec19c49b0 con 0x561ec1a71460 2023-12-07T10:23:36.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.043+0000 7f66cab4fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec197e560 con 0x561ec1a70b40 2023-12-07T10:23:36.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.043+0000 7f66cab4fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec197e200 con 0x561ec1a75150 2023-12-07T10:23:36.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.043+0000 7f66cab4fb80 10 client.0 _ll_register_callbacks cb 0x561ec1965940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:36.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.043+0000 7f66b4f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561ec1a70b40 0x561ec1a7c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.043+0000 7f66b4f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561ec1a70b40 0x561ec1a7c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:55950/0 (socket says 172.21.15.119:55950) 2023-12-07T10:23:36.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.043+0000 7f66b4f9b700 1 -- 172.21.15.119:0/1394156876 learned_addr learned my addr 172.21.15.119:0/1394156876 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:36.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.043+0000 7f66affff700 1 --2- 172.21.15.119:0/1394156876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ec1a71460 0x561ec1a79c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.043+0000 7f66b579c700 1 --2- 172.21.15.119:0/1394156876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ec1a75150 0x561ec1a7a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.043+0000 7f66b4f9b700 1 -- 172.21.15.119:0/1394156876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ec1a75150 msgr2=0x561ec1a7a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.043+0000 7f66b4f9b700 1 --2- 172.21.15.119:0/1394156876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ec1a75150 0x561ec1a7a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.043+0000 7f66b4f9b700 1 -- 172.21.15.119:0/1394156876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ec1a71460 msgr2=0x561ec1a79c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.043+0000 7f66b4f9b700 1 --2- 172.21.15.119:0/1394156876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ec1a71460 0x561ec1a79c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.043+0000 7f66b4f9b700 1 -- 172.21.15.119:0/1394156876 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b01aae00 con 0x561ec1a70b40 2023-12-07T10:23:36.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.043+0000 7f66b579c700 1 --2- 172.21.15.119:0/1394156876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ec1a75150 0x561ec1a7a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:23:36.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.043+0000 7f66b4f9b700 1 --2- 172.21.15.119:0/1394156876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561ec1a70b40 0x561ec1a7c940 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f66a4004560 tx=0x7f66a40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46e294fa6fbe5ad server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:36.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.044+0000 7f668ffff700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:23:36.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.044+0000 7f668ffff700 1 -- 172.21.15.119:0/1394156876 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a4020830 con 0x561ec1a70b40 2023-12-07T10:23:36.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.044+0000 7f66cab4fb80 1 -- 172.21.15.119:0/1394156876 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ec1a63b00 con 0x561ec1a70b40 2023-12-07T10:23:36.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.044+0000 7f668ffff700 1 -- 172.21.15.119:0/1394156876 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f66a4003e20 con 0x561ec1a70b40 2023-12-07T10:23:36.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.044+0000 7f668ffff700 1 -- 172.21.15.119:0/1394156876 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66a4004590 con 0x561ec1a70b40 2023-12-07T10:23:36.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.044+0000 7f66cab4fb80 10 client.26708 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:36.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.044+0000 7f66cab4fb80 1 -- 172.21.15.119:0/1394156876 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x561ec17edde0 con 0x561ec1a70b40 2023-12-07T10:23:36.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.044+0000 7f668ffff700 1 -- 172.21.15.119:0/1394156876 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561ec1a63b00 con 0x561ec1a70b40 2023-12-07T10:23:36.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.044+0000 7f668ffff700 1 -- 172.21.15.119:0/1394156876 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f66a4029b80 con 0x561ec1a70b40 2023-12-07T10:23:36.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.045+0000 7f66cab4fb80 10 client.26708 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:36.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.045+0000 7f66cab4fb80 10 client.26708 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2023-12-07T10:23:36.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.045+0000 7f66cab4fb80 20 client.26708 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:36.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.045+0000 7f66cab4fb80 1 --2- 172.21.15.119:0/1394156876 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x561ec1a9b340 0x561ec1a9d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.045+0000 7f66cab4fb80 4 client.26708 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:23:36.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.045+0000 7f66cab4fb80 1 -- 172.21.15.119:0/1394156876 --> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ec1a3f7a0 con 0x561ec1a9b340 2023-12-07T10:23:36.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.045+0000 7f66affff700 1 --2- 172.21.15.119:0/1394156876 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x561ec1a9b340 0x561ec1a9d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.045+0000 7f668ffff700 10 client.26708 ms_handle_connect on v2:172.21.15.153:6834/1295649464 2023-12-07T10:23:36.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.045+0000 7f66affff700 1 --2- 172.21.15.119:0/1394156876 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x561ec1a9b340 0x561ec1a9d760 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5eb79d9bd6bbcad9 server_cookie=2151d0e5ff8cb9f6 in_seq=0 out_seq=0 2023-12-07T10:23:36.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.046+0000 7f668ffff700 1 -- 172.21.15.119:0/1394156876 <== mds.4 v2:172.21.15.153:6834/1295649464 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561ec1a3f7a0 con 0x561ec1a9b340 2023-12-07T10:23:36.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.046+0000 7f668ffff700 10 client.26708 handle_command_reply: tid=0 2023-12-07T10:23:36.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.046+0000 7f66cab4fb80 1 client.26708 shutdown 2023-12-07T10:23:36.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.046+0000 7f66cab4fb80 1 -- 172.21.15.119:0/1394156876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561ec1a70b40 msgr2=0x561ec1a7c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.046+0000 7f66cab4fb80 1 --2- 172.21.15.119:0/1394156876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561ec1a70b40 0x561ec1a7c940 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f66a4004560 tx=0x7f66a40047c0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.046+0000 7f66cab4fb80 1 -- 172.21.15.119:0/1394156876 shutdown_connections 2023-12-07T10:23:36.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.046+0000 7f66cab4fb80 1 --2- 172.21.15.119:0/1394156876 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x561ec1a9b340 0x561ec1a9d760 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.046+0000 7f66cab4fb80 1 --2- 172.21.15.119:0/1394156876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ec1a75150 0x561ec1a7a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.046+0000 7f66cab4fb80 1 --2- 172.21.15.119:0/1394156876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561ec1a70b40 0x561ec1a7c940 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.046+0000 7f668ffff700 0 client.26708 ms_handle_reset on v2:172.21.15.153:6834/1295649464 2023-12-07T10:23:36.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.046+0000 7f66cab4fb80 1 --2- 172.21.15.119:0/1394156876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ec1a71460 0x561ec1a79c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.046+0000 7f66cab4fb80 1 -- 172.21.15.119:0/1394156876 >> 172.21.15.119:0/1394156876 conn(0x561ec199f360 msgr2=0x561ec1a76620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:36.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.047+0000 7f66cab4fb80 1 -- 172.21.15.119:0/1394156876 shutdown_connections 2023-12-07T10:23:36.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.047+0000 7f66cab4fb80 1 -- 172.21.15.119:0/1394156876 wait complete. 2023-12-07T10:23:36.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.047+0000 7f66cab4fb80 20 client.26708 trim_cache size 0 max 16384 2023-12-07T10:23:36.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.062+0000 7f66cab4fb80 1 Processor -- start 2023-12-07T10:23:36.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.062+0000 7f66cab4fb80 1 -- start start 2023-12-07T10:23:36.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.062+0000 7f66cab4fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ec1a70b40 0x561ec1a96da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.062+0000 7f66cab4fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ec1a71460 0x561ec1a6ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.062+0000 7f66cab4fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561ec1a75150 0x561ec1aa2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.062+0000 7f66cab4fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec1843920 con 0x561ec1a71460 2023-12-07T10:23:36.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.062+0000 7f66cab4fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec1a6c7d0 con 0x561ec1a75150 2023-12-07T10:23:36.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.062+0000 7f66cab4fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec1a67af0 con 0x561ec1a70b40 2023-12-07T10:23:36.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.063+0000 7f66b579c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561ec1a75150 0x561ec1aa2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.063+0000 7f66affff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ec1a71460 0x561ec1a6ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.063+0000 7f66b4f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ec1a70b40 0x561ec1a96da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.063+0000 7f66b579c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561ec1a75150 0x561ec1aa2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:55954/0 (socket says 172.21.15.119:55954) 2023-12-07T10:23:36.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.063+0000 7f66b579c700 1 -- 172.21.15.119:0/362371643 learned_addr learned my addr 172.21.15.119:0/362371643 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:36.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.063+0000 7f66affff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ec1a71460 0x561ec1a6ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34746/0 (socket says 172.21.15.119:34746) 2023-12-07T10:23:36.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.063+0000 7f66b579c700 1 -- 172.21.15.119:0/362371643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ec1a70b40 msgr2=0x561ec1a96da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.063+0000 7f66b579c700 1 --2- 172.21.15.119:0/362371643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ec1a70b40 0x561ec1a96da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.063+0000 7f66b579c700 1 -- 172.21.15.119:0/362371643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ec1a71460 msgr2=0x561ec1a6ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.063+0000 7f66b579c700 1 --2- 172.21.15.119:0/362371643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ec1a71460 0x561ec1a6ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.063+0000 7f66b579c700 1 -- 172.21.15.119:0/362371643 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ec17d6c60 con 0x561ec1a75150 2023-12-07T10:23:36.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.063+0000 7f66affff700 1 --2- 172.21.15.119:0/362371643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ec1a71460 0x561ec1a6ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:36.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.063+0000 7f66b579c700 1 --2- 172.21.15.119:0/362371643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561ec1a75150 0x561ec1aa2080 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f66a80094d0 tx=0x7f66a8009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88123e042b045046 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:36.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.064+0000 7f669b7fe700 1 -- 172.21.15.119:0/362371643 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a800e040 con 0x561ec1a75150 2023-12-07T10:23:36.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.064+0000 7f669b7fe700 1 -- 172.21.15.119:0/362371643 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66a8007de0 con 0x561ec1a75150 2023-12-07T10:23:36.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.064+0000 7f669b7fe700 1 -- 172.21.15.119:0/362371643 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a8007980 con 0x561ec1a75150 2023-12-07T10:23:36.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.064+0000 7f66cab4fb80 1 -- 172.21.15.119:0/362371643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561ec1a75150 msgr2=0x561ec1aa2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.064+0000 7f66cab4fb80 1 --2- 172.21.15.119:0/362371643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561ec1a75150 0x561ec1aa2080 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f66a80094d0 tx=0x7f66a8009500 comp rx=0 tx=0).stop 2023-12-07T10:23:36.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.064+0000 7f66cab4fb80 1 -- 172.21.15.119:0/362371643 shutdown_connections 2023-12-07T10:23:36.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.064+0000 7f66cab4fb80 1 --2- 172.21.15.119:0/362371643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561ec1a75150 0x561ec1aa2080 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.064+0000 7f66cab4fb80 1 --2- 172.21.15.119:0/362371643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ec1a70b40 0x561ec1a96da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.064+0000 7f66cab4fb80 1 --2- 172.21.15.119:0/362371643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ec1a71460 0x561ec1a6ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.064+0000 7f66cab4fb80 1 -- 172.21.15.119:0/362371643 >> 172.21.15.119:0/362371643 conn(0x561ec199f360 msgr2=0x561ec1a65db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:36.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.064+0000 7f66cab4fb80 1 -- 172.21.15.119:0/362371643 shutdown_connections 2023-12-07T10:23:36.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.064+0000 7f66cab4fb80 1 -- 172.21.15.119:0/362371643 wait complete. 2023-12-07T10:23:36.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.065+0000 7f66cab4fb80 1 -- start start 2023-12-07T10:23:36.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.065+0000 7f66cab4fb80 1 Processor -- start 2023-12-07T10:23:36.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.065+0000 7f66cab4fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ec1a70b40 0x561ec1a902d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.065+0000 7f66cab4fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ec1a71460 0x561ec1a90810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.065+0000 7f66cab4fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561ec1a75150 0x561ec1a90d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.065+0000 7f66cab4fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec1af0800 con 0x561ec1a70b40 2023-12-07T10:23:36.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.065+0000 7f66cab4fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec1af0940 con 0x561ec1a75150 2023-12-07T10:23:36.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.065+0000 7f66cab4fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ec1af0b00 con 0x561ec1a71460 2023-12-07T10:23:36.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.066+0000 7f66cab4fb80 10 client.0 _ll_register_callbacks cb 0x561ec1965940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:36.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.066+0000 7f66b4f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ec1a70b40 0x561ec1a902d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.066+0000 7f66b579c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561ec1a75150 0x561ec1a90d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.066+0000 7f66b579c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561ec1a75150 0x561ec1a90d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:55962/0 (socket says 172.21.15.119:55962) 2023-12-07T10:23:36.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.066+0000 7f66b579c700 1 -- 172.21.15.119:0/1662412842 learned_addr learned my addr 172.21.15.119:0/1662412842 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:36.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.066+0000 7f66affff700 1 --2- 172.21.15.119:0/1662412842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ec1a71460 0x561ec1a90810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.066+0000 7f66b579c700 1 -- 172.21.15.119:0/1662412842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ec1a71460 msgr2=0x561ec1a90810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.066+0000 7f66b579c700 1 --2- 172.21.15.119:0/1662412842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ec1a71460 0x561ec1a90810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.066+0000 7f66b579c700 1 -- 172.21.15.119:0/1662412842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ec1a70b40 msgr2=0x561ec1a902d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.066+0000 7f66b579c700 1 --2- 172.21.15.119:0/1662412842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ec1a70b40 0x561ec1a902d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.066+0000 7f66b579c700 1 -- 172.21.15.119:0/1662412842 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ec1841630 con 0x561ec1a75150 2023-12-07T10:23:36.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.067+0000 7f66b579c700 1 --2- 172.21.15.119:0/1662412842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561ec1a75150 0x561ec1a90d50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f66a80062a0 tx=0x7f66a8006070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b0b1fdac285062b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:36.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.067+0000 7f668ffff700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:23:36.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.067+0000 7f668ffff700 1 -- 172.21.15.119:0/1662412842 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a800e070 con 0x561ec1a75150 2023-12-07T10:23:36.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.067+0000 7f668ffff700 1 -- 172.21.15.119:0/1662412842 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f66a8007be0 con 0x561ec1a75150 2023-12-07T10:23:36.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.067+0000 7f66cab4fb80 1 -- 172.21.15.119:0/1662412842 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f669c0a2c90 con 0x561ec1a75150 2023-12-07T10:23:36.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.067+0000 7f668ffff700 1 -- 172.21.15.119:0/1662412842 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66a8003d60 con 0x561ec1a75150 2023-12-07T10:23:36.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.067+0000 7f66cab4fb80 10 client.26714 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:36.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.067+0000 7f66cab4fb80 1 -- 172.21.15.119:0/1662412842 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f66b010ecd0 con 0x561ec1a75150 2023-12-07T10:23:36.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.067+0000 7f668ffff700 1 -- 172.21.15.119:0/1662412842 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f669c0a2c90 con 0x561ec1a75150 2023-12-07T10:23:36.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.067+0000 7f668ffff700 1 -- 172.21.15.119:0/1662412842 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f66a800b0d0 con 0x561ec1a75150 2023-12-07T10:23:36.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.067+0000 7f66cab4fb80 10 client.26714 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:36.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.067+0000 7f66cab4fb80 10 client.26714 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2023-12-07T10:23:36.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.067+0000 7f66cab4fb80 20 client.26714 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:36.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.068+0000 7f66cab4fb80 1 --2- 172.21.15.119:0/1662412842 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x561ec1af1ec0 0x561ec1af42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.068+0000 7f66cab4fb80 4 client.26714 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:23:36.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.068+0000 7f66cab4fb80 1 -- 172.21.15.119:0/1662412842 --> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561ec198afd0 con 0x561ec1af1ec0 2023-12-07T10:23:36.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.068+0000 7f66b4f9b700 1 --2- 172.21.15.119:0/1662412842 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x561ec1af1ec0 0x561ec1af42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.068+0000 7f66b4f9b700 1 --2- 172.21.15.119:0/1662412842 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x561ec1af1ec0 0x561ec1af42b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bc5cee7180e3cfb6 server_cookie=91d89e5b01a8b873 in_seq=0 out_seq=0 2023-12-07T10:23:36.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.068+0000 7f668ffff700 10 client.26714 ms_handle_connect on v2:172.21.15.153:6834/1295649464 2023-12-07T10:23:36.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.068+0000 7f668ffff700 1 -- 172.21.15.119:0/1662412842 <== mds.4 v2:172.21.15.153:6834/1295649464 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561ec198afd0 con 0x561ec1af1ec0 2023-12-07T10:23:36.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.068+0000 7f668ffff700 10 client.26714 handle_command_reply: tid=0 2023-12-07T10:23:36.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.068+0000 7f66cab4fb80 1 client.26714 shutdown 2023-12-07T10:23:36.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.069+0000 7f66cab4fb80 1 -- 172.21.15.119:0/1662412842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561ec1a75150 msgr2=0x561ec1a90d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.069+0000 7f66cab4fb80 1 --2- 172.21.15.119:0/1662412842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561ec1a75150 0x561ec1a90d50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f66a80062a0 tx=0x7f66a8006070 comp rx=0 tx=0).stop 2023-12-07T10:23:36.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.069+0000 7f66cab4fb80 1 -- 172.21.15.119:0/1662412842 shutdown_connections 2023-12-07T10:23:36.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.069+0000 7f66cab4fb80 1 --2- 172.21.15.119:0/1662412842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561ec1a75150 0x561ec1a90d50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.069+0000 7f66cab4fb80 1 --2- 172.21.15.119:0/1662412842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ec1a70b40 0x561ec1a902d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.069+0000 7f66cab4fb80 1 --2- 172.21.15.119:0/1662412842 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x561ec1af1ec0 0x561ec1af42b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.069+0000 7f66cab4fb80 1 --2- 172.21.15.119:0/1662412842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561ec1a71460 0x561ec1a90810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.069+0000 7f66cab4fb80 1 -- 172.21.15.119:0/1662412842 >> 172.21.15.119:0/1662412842 conn(0x561ec199f360 msgr2=0x561ec1aa2080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:36.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.069+0000 7f668ffff700 0 client.26714 ms_handle_reset on v2:172.21.15.153:6834/1295649464 2023-12-07T10:23:36.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.069+0000 7f66cab4fb80 1 -- 172.21.15.119:0/1662412842 shutdown_connections 2023-12-07T10:23:36.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.069+0000 7f66cab4fb80 1 -- 172.21.15.119:0/1662412842 wait complete. 2023-12-07T10:23:36.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.069+0000 7f66cab4fb80 20 client.26714 trim_cache size 0 max 16384 2023-12-07T10:23:36.075 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:23:36.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.070+0000 7f66b71ff700 1 -- 172.21.15.119:0/311717508 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f66a00a1c80 msgr2=0x7f66a00a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.070+0000 7f66b71ff700 1 --2- 172.21.15.119:0/311717508 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f66a00a1c80 0x7f66a00a4140 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f66a4011550 tx=0x7f66a4011910 comp rx=0 tx=0).stop 2023-12-07T10:23:36.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.070+0000 7f66b71ff700 1 -- 172.21.15.119:0/311717508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66b0101f10 msgr2=0x7f66b0073e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.070+0000 7f66b71ff700 1 --2- 172.21.15.119:0/311717508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66b0101f10 0x7f66b0073e50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f669c00e9a0 tx=0x7f669c00ecb0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.070+0000 7f66b71ff700 1 -- 172.21.15.119:0/311717508 shutdown_connections 2023-12-07T10:23:36.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.070+0000 7f66b71ff700 1 --2- 172.21.15.119:0/311717508 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f66a00a1c80 0x7f66a00a4140 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.070+0000 7f66b71ff700 1 --2- 172.21.15.119:0/311717508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b0107610 0x7f66b0074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.070+0000 7f66b71ff700 1 --2- 172.21.15.119:0/311717508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66b0101550 0x7f66b0073910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.070+0000 7f66b71ff700 1 --2- 172.21.15.119:0/311717508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f66b0101f10 0x7f66b0073e50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.070+0000 7f66b71ff700 1 -- 172.21.15.119:0/311717508 >> 172.21.15.119:0/311717508 conn(0x7f66b00fd020 msgr2=0x7f66b00fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:36.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.070+0000 7f66b71ff700 1 -- 172.21.15.119:0/311717508 shutdown_connections 2023-12-07T10:23:36.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.070+0000 7f66b71ff700 1 -- 172.21.15.119:0/311717508 wait complete. 2023-12-07T10:23:36.090 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:35 smithi119 ceph-mon[97908]: pgmap v98: 129 pgs: 129 active+clean; 103 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.4 MiB/s wr, 211 op/s 2023-12-07T10:23:36.090 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:35 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/932587890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:23:36.091 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-12-07T10:23:36.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:35 smithi159 ceph-mon[134726]: pgmap v98: 129 pgs: 129 active+clean; 103 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.4 MiB/s wr, 211 op/s 2023-12-07T10:23:36.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:35 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/932587890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:23:36.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:35 smithi153 ceph-mon[133625]: pgmap v98: 129 pgs: 129 active+clean; 103 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.4 MiB/s wr, 211 op/s 2023-12-07T10:23:36.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:35 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/932587890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:23:36.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.334+0000 7f8b6e1ff700 1 -- 172.21.15.119:0/984712651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b680742e0 msgr2=0x7f8b6810d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.334+0000 7f8b6e1ff700 1 --2- 172.21.15.119:0/984712651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b680742e0 0x7f8b6810d8c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c009990 tx=0x7f8b5c009ca0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.334+0000 7f8b6e1ff700 1 -- 172.21.15.119:0/984712651 shutdown_connections 2023-12-07T10:23:36.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.334+0000 7f8b6e1ff700 1 --2- 172.21.15.119:0/984712651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b6810de00 0x7f8b681101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.334+0000 7f8b6e1ff700 1 --2- 172.21.15.119:0/984712651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b680739c0 0x7f8b68073da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.334+0000 7f8b6e1ff700 1 --2- 172.21.15.119:0/984712651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b680742e0 0x7f8b6810d8c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.334+0000 7f8b6e1ff700 1 -- 172.21.15.119:0/984712651 >> 172.21.15.119:0/984712651 conn(0x7f8b680fcfd0 msgr2=0x7f8b680ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:36.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.334+0000 7f8b6e1ff700 1 -- 172.21.15.119:0/984712651 shutdown_connections 2023-12-07T10:23:36.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.334+0000 7f8b6e1ff700 1 -- 172.21.15.119:0/984712651 wait complete. 2023-12-07T10:23:36.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.335+0000 7f8b6e1ff700 1 Processor -- start 2023-12-07T10:23:36.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.335+0000 7f8b6e1ff700 1 -- start start 2023-12-07T10:23:36.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.335+0000 7f8b6e1ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b680739c0 0x7f8b6819fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.336+0000 7f8b6e1ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b680742e0 0x7f8b6819ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.336+0000 7f8b6e1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b6810de00 0x7f8b68199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.336+0000 7f8b6e1ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6819a2d0 con 0x7f8b680742e0 2023-12-07T10:23:36.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.336+0000 7f8b6e1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6819a440 con 0x7f8b6810de00 2023-12-07T10:23:36.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.336+0000 7f8b6e1ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6819a600 con 0x7f8b680739c0 2023-12-07T10:23:36.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.336+0000 7f8b677fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b680739c0 0x7f8b6819fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.336+0000 7f8b66ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b680742e0 0x7f8b6819ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.336+0000 7f8b677fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b680739c0 0x7f8b6819fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41626/0 (socket says 172.21.15.119:41626) 2023-12-07T10:23:36.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.336+0000 7f8b677fe700 1 -- 172.21.15.119:0/1276250561 learned_addr learned my addr 172.21.15.119:0/1276250561 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:36.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.336+0000 7f8b67fff700 1 --2- 172.21.15.119:0/1276250561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b6810de00 0x7f8b68199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.336+0000 7f8b67fff700 1 -- 172.21.15.119:0/1276250561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b680739c0 msgr2=0x7f8b6819fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.336+0000 7f8b67fff700 1 --2- 172.21.15.119:0/1276250561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b680739c0 0x7f8b6819fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.336+0000 7f8b67fff700 1 -- 172.21.15.119:0/1276250561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b680742e0 msgr2=0x7f8b6819ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.336+0000 7f8b67fff700 1 --2- 172.21.15.119:0/1276250561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b680742e0 0x7f8b6819ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.336+0000 7f8b67fff700 1 -- 172.21.15.119:0/1276250561 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b58009520 con 0x7f8b6810de00 2023-12-07T10:23:36.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.336+0000 7f8b677fe700 1 --2- 172.21.15.119:0/1276250561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b680739c0 0x7f8b6819fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:36.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.336+0000 7f8b67fff700 1 --2- 172.21.15.119:0/1276250561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b6810de00 0x7f8b68199b00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000efb0 tx=0x7f8b6000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:36.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.337+0000 7f8b64ff9700 1 -- 172.21.15.119:0/1276250561 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b60003a00 con 0x7f8b6810de00 2023-12-07T10:23:36.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.337+0000 7f8b64ff9700 1 -- 172.21.15.119:0/1276250561 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b6000f040 con 0x7f8b6810de00 2023-12-07T10:23:36.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.337+0000 7f8b6e1ff700 1 -- 172.21.15.119:0/1276250561 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b60009c40 con 0x7f8b6810de00 2023-12-07T10:23:36.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.337+0000 7f8b64ff9700 1 -- 172.21.15.119:0/1276250561 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b60004540 con 0x7f8b6810de00 2023-12-07T10:23:36.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.337+0000 7f8b6e1ff700 1 -- 172.21.15.119:0/1276250561 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b5c0095f0 con 0x7f8b6810de00 2023-12-07T10:23:36.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.340+0000 7f8b64ff9700 1 -- 172.21.15.119:0/1276250561 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f8b60003bf0 con 0x7f8b6810de00 2023-12-07T10:23:36.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.341+0000 7f8b81adcb80 1 Processor -- start 2023-12-07T10:23:36.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.341+0000 7f8b81adcb80 1 -- start start 2023-12-07T10:23:36.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.342+0000 7f8b64ff9700 1 --2- 172.21.15.119:0/1276250561 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8b500a1d00 0x7f8b500a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.342+0000 7f8b81adcb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cb3082b40 0x562cb3082f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.342+0000 7f8b64ff9700 1 -- 172.21.15.119:0/1276250561 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f8b600cd850 con 0x7f8b6810de00 2023-12-07T10:23:36.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.342+0000 7f8b81adcb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cb3083460 0x562cb3086c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.342+0000 7f8b81adcb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb3087150 0x562cb3087570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.342+0000 7f8b81adcb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb2ff67c0 con 0x562cb3082b40 2023-12-07T10:23:36.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.342+0000 7f8b81adcb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb2fdd7a0 con 0x562cb3087150 2023-12-07T10:23:36.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.342+0000 7f8b81adcb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb2fdb0c0 con 0x562cb3083460 2023-12-07T10:23:36.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.342+0000 7f8b66ffd700 1 --2- 172.21.15.119:0/1276250561 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8b500a1d00 0x7f8b500a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.342+0000 7f8b677fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cb3082b40 0x562cb3082f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.342+0000 7f8b67fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb3087150 0x562cb3087570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.342+0000 7f8b67fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb3087150 0x562cb3087570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:55984/0 (socket says 172.21.15.119:55984) 2023-12-07T10:23:36.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.342+0000 7f8b67fff700 1 -- 172.21.15.119:0/1247936772 learned_addr learned my addr 172.21.15.119:0/1247936772 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:36.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.342+0000 7f8b66ffd700 1 --2- 172.21.15.119:0/1247936772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cb3083460 0x562cb3086c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.343+0000 7f8b677fe700 1 -- 172.21.15.119:0/1247936772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cb3083460 msgr2=0x562cb3086c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.343+0000 7f8b677fe700 1 --2- 172.21.15.119:0/1247936772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cb3083460 0x562cb3086c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.343+0000 7f8b677fe700 1 -- 172.21.15.119:0/1247936772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb3087150 msgr2=0x562cb3087570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.343+0000 7f8b677fe700 1 --2- 172.21.15.119:0/1247936772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb3087150 0x562cb3087570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.343+0000 7f8b677fe700 1 -- 172.21.15.119:0/1247936772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cb2e53630 con 0x562cb3082b40 2023-12-07T10:23:36.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.343+0000 7f8b67fff700 1 --2- 172.21.15.119:0/1247936772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb3087150 0x562cb3087570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:36.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.343+0000 7f8b66ffd700 1 --2- 172.21.15.119:0/1276250561 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8b500a1d00 0x7f8b500a41c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8b6819b270 tx=0x7f8b5c005c20 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:36.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.343+0000 7f8b677fe700 1 --2- 172.21.15.119:0/1247936772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cb3082b40 0x562cb3082f20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8b58011840 tx=0x7f8b58011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc1704c0262beaf server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:36.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.343+0000 7f8b4dffb700 1 -- 172.21.15.119:0/1247936772 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b58004020 con 0x562cb3082b40 2023-12-07T10:23:36.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.343+0000 7f8b4dffb700 1 -- 172.21.15.119:0/1247936772 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b58004830 con 0x562cb3082b40 2023-12-07T10:23:36.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.343+0000 7f8b4dffb700 1 -- 172.21.15.119:0/1247936772 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b58005390 con 0x562cb3082b40 2023-12-07T10:23:36.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.344+0000 7f8b81adcb80 1 -- 172.21.15.119:0/1247936772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cb3082b40 msgr2=0x562cb3082f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.344+0000 7f8b81adcb80 1 --2- 172.21.15.119:0/1247936772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cb3082b40 0x562cb3082f20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8b58011840 tx=0x7f8b58011b50 comp rx=0 tx=0).stop 2023-12-07T10:23:36.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.344+0000 7f8b81adcb80 1 -- 172.21.15.119:0/1247936772 shutdown_connections 2023-12-07T10:23:36.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.344+0000 7f8b81adcb80 1 --2- 172.21.15.119:0/1247936772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb3087150 0x562cb3087570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.344+0000 7f8b81adcb80 1 --2- 172.21.15.119:0/1247936772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cb3082b40 0x562cb3082f20 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.344+0000 7f8b81adcb80 1 --2- 172.21.15.119:0/1247936772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cb3083460 0x562cb3086c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.344+0000 7f8b81adcb80 1 -- 172.21.15.119:0/1247936772 >> 172.21.15.119:0/1247936772 conn(0x562cb2fb1360 msgr2=0x562cb307d130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:36.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.344+0000 7f8b81adcb80 1 -- 172.21.15.119:0/1247936772 shutdown_connections 2023-12-07T10:23:36.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.344+0000 7f8b81adcb80 1 -- 172.21.15.119:0/1247936772 wait complete. 2023-12-07T10:23:36.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.345+0000 7f8b81adcb80 1 -- start start 2023-12-07T10:23:36.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.345+0000 7f8b81adcb80 1 Processor -- start 2023-12-07T10:23:36.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.346+0000 7f8b81adcb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cb3082b40 0x562cb308e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.346+0000 7f8b81adcb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb3083460 0x562cb308bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.346+0000 7f8b81adcb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cb3087150 0x562cb308c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.346+0000 7f8b81adcb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb2fd69b0 con 0x562cb3087150 2023-12-07T10:23:36.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.346+0000 7f8b81adcb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb2f90560 con 0x562cb3083460 2023-12-07T10:23:36.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.346+0000 7f8b81adcb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb2f90200 con 0x562cb3082b40 2023-12-07T10:23:36.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.346+0000 7f8b81adcb80 10 client.0 _ll_register_callbacks cb 0x562cb2f77940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:36.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.346+0000 7f8b677fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cb3082b40 0x562cb308e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.346+0000 7f8b677fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cb3082b40 0x562cb308e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41642/0 (socket says 172.21.15.119:41642) 2023-12-07T10:23:36.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.346+0000 7f8b677fe700 1 -- 172.21.15.119:0/2426734903 learned_addr learned my addr 172.21.15.119:0/2426734903 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:36.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.346+0000 7f8b66ffd700 1 --2- 172.21.15.119:0/2426734903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb3083460 0x562cb308bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.346+0000 7f8b67fff700 1 --2- 172.21.15.119:0/2426734903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cb3087150 0x562cb308c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.347+0000 7f8b677fe700 1 -- 172.21.15.119:0/2426734903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb3083460 msgr2=0x562cb308bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.347+0000 7f8b677fe700 1 --2- 172.21.15.119:0/2426734903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb3083460 0x562cb308bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.347+0000 7f8b677fe700 1 -- 172.21.15.119:0/2426734903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cb3087150 msgr2=0x562cb308c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.347+0000 7f8b677fe700 1 --2- 172.21.15.119:0/2426734903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cb3087150 0x562cb308c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.347+0000 7f8b677fe700 1 -- 172.21.15.119:0/2426734903 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b5c0095f0 con 0x562cb3082b40 2023-12-07T10:23:36.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.347+0000 7f8b66ffd700 1 --2- 172.21.15.119:0/2426734903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb3083460 0x562cb308bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:23:36.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.347+0000 7f8b677fe700 1 --2- 172.21.15.119:0/2426734903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cb3082b40 0x562cb308e7f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8b58011810 tx=0x7f8b58004320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26ad6365e7fc9875 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:36.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.347+0000 7f8b46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:23:36.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.347+0000 7f8b46ffd700 1 -- 172.21.15.119:0/2426734903 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b58004540 con 0x562cb3082b40 2023-12-07T10:23:36.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.347+0000 7f8b46ffd700 1 -- 172.21.15.119:0/2426734903 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f8b58005ce0 con 0x562cb3082b40 2023-12-07T10:23:36.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.347+0000 7f8b81adcb80 1 -- 172.21.15.119:0/2426734903 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562cb3075b00 con 0x562cb3082b40 2023-12-07T10:23:36.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.348+0000 7f8b46ffd700 1 -- 172.21.15.119:0/2426734903 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b58005000 con 0x562cb3082b40 2023-12-07T10:23:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.348+0000 7f8b81adcb80 10 client.26704 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.348+0000 7f8b81adcb80 1 -- 172.21.15.119:0/2426734903 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f8b58011440 con 0x562cb3082b40 2023-12-07T10:23:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.348+0000 7f8b46ffd700 1 -- 172.21.15.119:0/2426734903 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562cb3075b00 con 0x562cb3082b40 2023-12-07T10:23:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.348+0000 7f8b46ffd700 1 -- 172.21.15.119:0/2426734903 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f8b58003be0 con 0x562cb3082b40 2023-12-07T10:23:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.348+0000 7f8b81adcb80 10 client.26704 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.348+0000 7f8b81adcb80 10 client.26704 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-12-07T10:23:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.348+0000 7f8b81adcb80 20 client.26704 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.348+0000 7f8b81adcb80 1 --2- 172.21.15.119:0/2426734903 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x562cb309c220 0x562cb309e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.348+0000 7f8b81adcb80 4 client.26704 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:23:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.348+0000 7f8b81adcb80 1 -- 172.21.15.119:0/2426734903 --> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562cb30517a0 con 0x562cb309c220 2023-12-07T10:23:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.348+0000 7f8b66ffd700 1 --2- 172.21.15.119:0/2426734903 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x562cb309c220 0x562cb309e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.349+0000 7f8b66ffd700 1 --2- 172.21.15.119:0/2426734903 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x562cb309c220 0x562cb309e640 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=39e6dd4f5ffc662c server_cookie=8b017e6a7221646a in_seq=0 out_seq=0 2023-12-07T10:23:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.349+0000 7f8b46ffd700 10 client.26704 ms_handle_connect on v2:172.21.15.159:6834/2406380792 2023-12-07T10:23:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.350+0000 7f8b46ffd700 1 -- 172.21.15.119:0/2426734903 <== mds.3 v2:172.21.15.159:6834/2406380792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562cb30517a0 con 0x562cb309c220 2023-12-07T10:23:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.350+0000 7f8b46ffd700 10 client.26704 handle_command_reply: tid=0 2023-12-07T10:23:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.350+0000 7f8b81adcb80 1 client.26704 shutdown 2023-12-07T10:23:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.350+0000 7f8b81adcb80 1 -- 172.21.15.119:0/2426734903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cb3082b40 msgr2=0x562cb308e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.350+0000 7f8b81adcb80 1 --2- 172.21.15.119:0/2426734903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cb3082b40 0x562cb308e7f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8b58011810 tx=0x7f8b58004320 comp rx=0 tx=0).stop 2023-12-07T10:23:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.350+0000 7f8b81adcb80 1 -- 172.21.15.119:0/2426734903 shutdown_connections 2023-12-07T10:23:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.350+0000 7f8b81adcb80 1 --2- 172.21.15.119:0/2426734903 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x562cb309c220 0x562cb309e640 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.350+0000 7f8b81adcb80 1 --2- 172.21.15.119:0/2426734903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cb3087150 0x562cb308c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.350+0000 7f8b46ffd700 0 client.26704 ms_handle_reset on v2:172.21.15.159:6834/2406380792 2023-12-07T10:23:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.350+0000 7f8b81adcb80 1 --2- 172.21.15.119:0/2426734903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cb3082b40 0x562cb308e7f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.350+0000 7f8b81adcb80 1 --2- 172.21.15.119:0/2426734903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb3083460 0x562cb308bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.350+0000 7f8b81adcb80 1 -- 172.21.15.119:0/2426734903 >> 172.21.15.119:0/2426734903 conn(0x562cb2fb1360 msgr2=0x562cb3077d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.350+0000 7f8b81adcb80 1 -- 172.21.15.119:0/2426734903 shutdown_connections 2023-12-07T10:23:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.350+0000 7f8b81adcb80 1 -- 172.21.15.119:0/2426734903 wait complete. 2023-12-07T10:23:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.350+0000 7f8b81adcb80 20 client.26704 trim_cache size 0 max 16384 2023-12-07T10:23:36.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.365+0000 7f8b81adcb80 1 Processor -- start 2023-12-07T10:23:36.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.365+0000 7f8b81adcb80 1 -- start start 2023-12-07T10:23:36.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.365+0000 7f8b81adcb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cb3082b40 0x562cb309e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.365+0000 7f8b81adcb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb3083460 0x562cb3084b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.365+0000 7f8b81adcb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cb3087150 0x562cb3093350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.365+0000 7f8b81adcb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb2e55920 con 0x562cb3087150 2023-12-07T10:23:36.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.365+0000 7f8b81adcb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb3079af0 con 0x562cb3083460 2023-12-07T10:23:36.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.365+0000 7f8b677fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cb3082b40 0x562cb309e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.365+0000 7f8b81adcb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb309bfc0 con 0x562cb3082b40 2023-12-07T10:23:36.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.365+0000 7f8b677fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cb3082b40 0x562cb309e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41654/0 (socket says 172.21.15.119:41654) 2023-12-07T10:23:36.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.365+0000 7f8b677fe700 1 -- 172.21.15.119:0/4287304515 learned_addr learned my addr 172.21.15.119:0/4287304515 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:36.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.366+0000 7f8b66ffd700 1 --2- 172.21.15.119:0/4287304515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb3083460 0x562cb3084b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.366+0000 7f8b67fff700 1 --2- 172.21.15.119:0/4287304515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cb3087150 0x562cb3093350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.366+0000 7f8b677fe700 1 -- 172.21.15.119:0/4287304515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb3083460 msgr2=0x562cb3084b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.366+0000 7f8b677fe700 1 --2- 172.21.15.119:0/4287304515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb3083460 0x562cb3084b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.366+0000 7f8b677fe700 1 -- 172.21.15.119:0/4287304515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cb3087150 msgr2=0x562cb3093350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.366+0000 7f8b677fe700 1 --2- 172.21.15.119:0/4287304515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cb3087150 0x562cb3093350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.366+0000 7f8b677fe700 1 -- 172.21.15.119:0/4287304515 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cb2dffde0 con 0x562cb3082b40 2023-12-07T10:23:36.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.366+0000 7f8b67fff700 1 --2- 172.21.15.119:0/4287304515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cb3087150 0x562cb3093350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:23:36.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.366+0000 7f8b677fe700 1 --2- 172.21.15.119:0/4287304515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cb3082b40 0x562cb309e610 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8b58011b90 tx=0x7f8b58004e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7ebf32d27a801a0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:36.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.366+0000 7f8b4e7fc700 1 -- 172.21.15.119:0/4287304515 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b58011ea0 con 0x562cb3082b40 2023-12-07T10:23:36.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.366+0000 7f8b4e7fc700 1 -- 172.21.15.119:0/4287304515 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b58004440 con 0x562cb3082b40 2023-12-07T10:23:36.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.366+0000 7f8b4e7fc700 1 -- 172.21.15.119:0/4287304515 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b58004ed0 con 0x562cb3082b40 2023-12-07T10:23:36.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.367+0000 7f8b81adcb80 1 -- 172.21.15.119:0/4287304515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cb3082b40 msgr2=0x562cb309e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.367+0000 7f8b81adcb80 1 --2- 172.21.15.119:0/4287304515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cb3082b40 0x562cb309e610 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8b58011b90 tx=0x7f8b58004e60 comp rx=0 tx=0).stop 2023-12-07T10:23:36.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.367+0000 7f8b81adcb80 1 -- 172.21.15.119:0/4287304515 shutdown_connections 2023-12-07T10:23:36.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.367+0000 7f8b81adcb80 1 --2- 172.21.15.119:0/4287304515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cb3087150 0x562cb3093350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.367+0000 7f8b81adcb80 1 --2- 172.21.15.119:0/4287304515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cb3082b40 0x562cb309e610 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.367+0000 7f8b81adcb80 1 --2- 172.21.15.119:0/4287304515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb3083460 0x562cb3084b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.367+0000 7f8b81adcb80 1 -- 172.21.15.119:0/4287304515 >> 172.21.15.119:0/4287304515 conn(0x562cb2fb1360 msgr2=0x562cb3097e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:36.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.367+0000 7f8b81adcb80 1 -- 172.21.15.119:0/4287304515 shutdown_connections 2023-12-07T10:23:36.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.367+0000 7f8b81adcb80 1 -- 172.21.15.119:0/4287304515 wait complete. 2023-12-07T10:23:36.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.368+0000 7f8b81adcb80 1 -- start start 2023-12-07T10:23:36.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.368+0000 7f8b81adcb80 1 Processor -- start 2023-12-07T10:23:36.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.368+0000 7f8b81adcb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cb3082b40 0x562cb3091330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.368+0000 7f8b81adcb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cb3083460 0x562cb3091870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.368+0000 7f8b81adcb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb3087150 0x562cb3091db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.368+0000 7f8b81adcb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb30f1760 con 0x562cb3083460 2023-12-07T10:23:36.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.368+0000 7f8b81adcb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb30f18d0 con 0x562cb3087150 2023-12-07T10:23:36.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.368+0000 7f8b81adcb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb30f1a90 con 0x562cb3082b40 2023-12-07T10:23:36.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.368+0000 7f8b81adcb80 10 client.0 _ll_register_callbacks cb 0x562cb2f77940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:36.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.369+0000 7f8b677fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cb3082b40 0x562cb3091330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.369+0000 7f8b66ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cb3083460 0x562cb3091870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.369+0000 7f8b67fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb3087150 0x562cb3091db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.369+0000 7f8b677fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cb3082b40 0x562cb3091330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41660/0 (socket says 172.21.15.119:41660) 2023-12-07T10:23:36.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.369+0000 7f8b677fe700 1 -- 172.21.15.119:0/296309162 learned_addr learned my addr 172.21.15.119:0/296309162 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:36.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.369+0000 7f8b67fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb3087150 0x562cb3091db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:56008/0 (socket says 172.21.15.119:56008) 2023-12-07T10:23:36.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.369+0000 7f8b67fff700 1 -- 172.21.15.119:0/296309162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cb3082b40 msgr2=0x562cb3091330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.369+0000 7f8b67fff700 1 --2- 172.21.15.119:0/296309162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cb3082b40 0x562cb3091330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.369+0000 7f8b67fff700 1 -- 172.21.15.119:0/296309162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cb3083460 msgr2=0x562cb3091870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.369+0000 7f8b67fff700 1 --2- 172.21.15.119:0/296309162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cb3083460 0x562cb3091870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.369+0000 7f8b67fff700 1 -- 172.21.15.119:0/296309162 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b58011440 con 0x562cb3087150 2023-12-07T10:23:36.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.369+0000 7f8b677fe700 1 --2- 172.21.15.119:0/296309162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cb3082b40 0x562cb3091330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:36.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.369+0000 7f8b67fff700 1 --2- 172.21.15.119:0/296309162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb3087150 0x562cb3091db0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8b600a2100 tx=0x7f8b600966f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e21ef7bcb5bccc11 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:36.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.369+0000 7f8b46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:23:36.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.369+0000 7f8b46ffd700 1 -- 172.21.15.119:0/296309162 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6009e480 con 0x562cb3087150 2023-12-07T10:23:36.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.370+0000 7f8b46ffd700 1 -- 172.21.15.119:0/296309162 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f8b600a2630 con 0x562cb3087150 2023-12-07T10:23:36.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.370+0000 7f8b81adcb80 1 -- 172.21.15.119:0/296309162 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b58004830 con 0x562cb3087150 2023-12-07T10:23:36.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.370+0000 7f8b46ffd700 1 -- 172.21.15.119:0/296309162 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b600b8360 con 0x562cb3087150 2023-12-07T10:23:36.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.370+0000 7f8b81adcb80 10 client.26732 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:36.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.370+0000 7f8b81adcb80 1 -- 172.21.15.119:0/296309162 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x562cb2de8c60 con 0x562cb3087150 2023-12-07T10:23:36.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.370+0000 7f8b46ffd700 1 -- 172.21.15.119:0/296309162 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b58004830 con 0x562cb3087150 2023-12-07T10:23:36.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.370+0000 7f8b46ffd700 1 -- 172.21.15.119:0/296309162 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f8b60099820 con 0x562cb3087150 2023-12-07T10:23:36.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.370+0000 7f8b81adcb80 10 client.26732 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:36.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.370+0000 7f8b81adcb80 10 client.26732 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-12-07T10:23:36.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.370+0000 7f8b81adcb80 20 client.26732 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:36.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.370+0000 7f8b81adcb80 1 --2- 172.21.15.119:0/296309162 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x562cb30f3210 0x562cb30f5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.370+0000 7f8b81adcb80 4 client.26732 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:23:36.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.370+0000 7f8b81adcb80 1 -- 172.21.15.119:0/296309162 --> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562cb2f9cfd0 con 0x562cb30f3210 2023-12-07T10:23:36.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.370+0000 7f8b677fe700 1 --2- 172.21.15.119:0/296309162 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x562cb30f3210 0x562cb30f5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.371+0000 7f8b46ffd700 10 client.26732 ms_handle_connect on v2:172.21.15.159:6834/2406380792 2023-12-07T10:23:36.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.371+0000 7f8b677fe700 1 --2- 172.21.15.119:0/296309162 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x562cb30f3210 0x562cb30f5630 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1d6976fd78607d99 server_cookie=5ee0a9e6e3e3b9b1 in_seq=0 out_seq=0 2023-12-07T10:23:36.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.371+0000 7f8b46ffd700 1 -- 172.21.15.119:0/296309162 <== mds.3 v2:172.21.15.159:6834/2406380792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562cb2f9cfd0 con 0x562cb30f3210 2023-12-07T10:23:36.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.371+0000 7f8b46ffd700 10 client.26732 handle_command_reply: tid=0 2023-12-07T10:23:36.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.371+0000 7f8b81adcb80 1 client.26732 shutdown 2023-12-07T10:23:36.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.372+0000 7f8b81adcb80 1 -- 172.21.15.119:0/296309162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb3087150 msgr2=0x562cb3091db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.372+0000 7f8b81adcb80 1 --2- 172.21.15.119:0/296309162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb3087150 0x562cb3091db0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8b600a2100 tx=0x7f8b600966f0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.372+0000 7f8b81adcb80 1 -- 172.21.15.119:0/296309162 shutdown_connections 2023-12-07T10:23:36.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.372+0000 7f8b81adcb80 1 --2- 172.21.15.119:0/296309162 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x562cb30f3210 0x562cb30f5630 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.372+0000 7f8b81adcb80 1 --2- 172.21.15.119:0/296309162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cb3087150 0x562cb3091db0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.372+0000 7f8b81adcb80 1 --2- 172.21.15.119:0/296309162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cb3082b40 0x562cb3091330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.372+0000 7f8b46ffd700 0 client.26732 ms_handle_reset on v2:172.21.15.159:6834/2406380792 2023-12-07T10:23:36.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.372+0000 7f8b81adcb80 1 --2- 172.21.15.119:0/296309162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cb3083460 0x562cb3091870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.372+0000 7f8b81adcb80 1 -- 172.21.15.119:0/296309162 >> 172.21.15.119:0/296309162 conn(0x562cb2fb1360 msgr2=0x562cb3097e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:36.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.372+0000 7f8b81adcb80 1 -- 172.21.15.119:0/296309162 shutdown_connections 2023-12-07T10:23:36.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.372+0000 7f8b81adcb80 1 -- 172.21.15.119:0/296309162 wait complete. 2023-12-07T10:23:36.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.372+0000 7f8b81adcb80 20 client.26732 trim_cache size 0 max 16384 2023-12-07T10:23:36.379 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:23:36.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.372+0000 7f8b6e1ff700 1 -- 172.21.15.119:0/1276250561 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8b500a1d00 msgr2=0x7f8b500a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.372+0000 7f8b6e1ff700 1 --2- 172.21.15.119:0/1276250561 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8b500a1d00 0x7f8b500a41c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8b6819b270 tx=0x7f8b5c005c20 comp rx=0 tx=0).stop 2023-12-07T10:23:36.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.373+0000 7f8b6e1ff700 1 -- 172.21.15.119:0/1276250561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b6810de00 msgr2=0x7f8b68199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.373+0000 7f8b6e1ff700 1 --2- 172.21.15.119:0/1276250561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b6810de00 0x7f8b68199b00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000efb0 tx=0x7f8b6000c3a0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.373+0000 7f8b6e1ff700 1 -- 172.21.15.119:0/1276250561 shutdown_connections 2023-12-07T10:23:36.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.373+0000 7f8b6e1ff700 1 --2- 172.21.15.119:0/1276250561 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8b500a1d00 0x7f8b500a41c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.373+0000 7f8b6e1ff700 1 --2- 172.21.15.119:0/1276250561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b6810de00 0x7f8b68199b00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.373+0000 7f8b6e1ff700 1 --2- 172.21.15.119:0/1276250561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b680739c0 0x7f8b6819fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.373+0000 7f8b6e1ff700 1 --2- 172.21.15.119:0/1276250561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b680742e0 0x7f8b6819ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.373+0000 7f8b6e1ff700 1 -- 172.21.15.119:0/1276250561 >> 172.21.15.119:0/1276250561 conn(0x7f8b680fcfd0 msgr2=0x7f8b680fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:36.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.373+0000 7f8b6e1ff700 1 -- 172.21.15.119:0/1276250561 shutdown_connections 2023-12-07T10:23:36.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.373+0000 7f8b6e1ff700 1 -- 172.21.15.119:0/1276250561 wait complete. 2023-12-07T10:23:36.393 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-07T10:23:36.550 INFO:tasks.workunit.client.0.smithi119.stdout: 1 174238 8.79 MB/sec warmup 109 sec latency 25.778 ms 2023-12-07T10:23:36.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.615+0000 7fb62c7ff700 1 -- 172.21.15.119:0/1501204081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb624101f10 msgr2=0x7fb62410e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.615+0000 7fb62c7ff700 1 --2- 172.21.15.119:0/1501204081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb624101f10 0x7fb62410e3d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00b0d0 tx=0x7fb61c00b3e0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.615+0000 7fb62c7ff700 1 -- 172.21.15.119:0/1501204081 shutdown_connections 2023-12-07T10:23:36.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.615+0000 7fb62c7ff700 1 --2- 172.21.15.119:0/1501204081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb624101f10 0x7fb62410e3d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.615+0000 7fb62c7ff700 1 --2- 172.21.15.119:0/1501204081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb624107610 0x7fb6241079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.615+0000 7fb62c7ff700 1 --2- 172.21.15.119:0/1501204081 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb624101550 0x7fb6241019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.615+0000 7fb62c7ff700 1 -- 172.21.15.119:0/1501204081 >> 172.21.15.119:0/1501204081 conn(0x7fb6240fd020 msgr2=0x7fb6240ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:36.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.616+0000 7fb62c7ff700 1 -- 172.21.15.119:0/1501204081 shutdown_connections 2023-12-07T10:23:36.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.616+0000 7fb62c7ff700 1 -- 172.21.15.119:0/1501204081 wait complete. 2023-12-07T10:23:36.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.617+0000 7fb62c7ff700 1 Processor -- start 2023-12-07T10:23:36.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.617+0000 7fb62c7ff700 1 -- start start 2023-12-07T10:23:36.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.617+0000 7fb62c7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb624101550 0x7fb62419f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.617+0000 7fb62c7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb624101f10 0x7fb62419fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.617+0000 7fb62c7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb624107610 0x7fb624199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.617+0000 7fb62c7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62419a0d0 con 0x7fb624101f10 2023-12-07T10:23:36.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.617+0000 7fb62c7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62419a240 con 0x7fb624101550 2023-12-07T10:23:36.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.617+0000 7fb62c7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62419a400 con 0x7fb624107610 2023-12-07T10:23:36.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.617+0000 7fb62a59b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb624101550 0x7fb62419f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.617+0000 7fb62a59b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb624101550 0x7fb62419f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:56018/0 (socket says 172.21.15.119:56018) 2023-12-07T10:23:36.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.617+0000 7fb62a59b700 1 -- 172.21.15.119:0/1559135940 learned_addr learned my addr 172.21.15.119:0/1559135940 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:36.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.617+0000 7fb629d9a700 1 --2- 172.21.15.119:0/1559135940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb624101f10 0x7fb62419fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.617+0000 7fb62ad9c700 1 --2- 172.21.15.119:0/1559135940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb624107610 0x7fb624199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.618+0000 7fb62ad9c700 1 -- 172.21.15.119:0/1559135940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb624101550 msgr2=0x7fb62419f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.618+0000 7fb62ad9c700 1 --2- 172.21.15.119:0/1559135940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb624101550 0x7fb62419f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.618+0000 7fb62ad9c700 1 -- 172.21.15.119:0/1559135940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb624101f10 msgr2=0x7fb62419fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.618+0000 7fb62ad9c700 1 --2- 172.21.15.119:0/1559135940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb624101f10 0x7fb62419fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.618+0000 7fb62ad9c700 1 -- 172.21.15.119:0/1559135940 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb620009520 con 0x7fb624107610 2023-12-07T10:23:36.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.618+0000 7fb62a59b700 1 --2- 172.21.15.119:0/1559135940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb624101550 0x7fb62419f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:36.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.618+0000 7fb62ad9c700 1 --2- 172.21.15.119:0/1559135940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb624107610 0x7fb624199a70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00b0d0 tx=0x7fb61c00bd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:36.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.618+0000 7fb61b7fe700 1 -- 172.21.15.119:0/1559135940 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c00e040 con 0x7fb624107610 2023-12-07T10:23:36.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.618+0000 7fb61b7fe700 1 -- 172.21.15.119:0/1559135940 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb61c01b8e0 con 0x7fb624107610 2023-12-07T10:23:36.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.618+0000 7fb62c7ff700 1 -- 172.21.15.119:0/1559135940 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb61c009d00 con 0x7fb624107610 2023-12-07T10:23:36.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.618+0000 7fb61b7fe700 1 -- 172.21.15.119:0/1559135940 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c012760 con 0x7fb624107610 2023-12-07T10:23:36.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.619+0000 7fb62c7ff700 1 -- 172.21.15.119:0/1559135940 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6241a6750 con 0x7fb624107610 2023-12-07T10:23:36.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.622+0000 7fb61b7fe700 1 -- 172.21.15.119:0/1559135940 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fb61c01ba50 con 0x7fb624107610 2023-12-07T10:23:36.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.623+0000 7fb6401c3b80 1 Processor -- start 2023-12-07T10:23:36.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.623+0000 7fb6401c3b80 1 -- start start 2023-12-07T10:23:36.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.623+0000 7fb6401c3b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa8a148b40 0x55fa8a148f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.624+0000 7fb6401c3b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa8a149460 0x55fa8a14cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.624+0000 7fb6401c3b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa8a14d150 0x55fa8a14d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.624+0000 7fb6401c3b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa8a0bc7c0 con 0x55fa8a14d150 2023-12-07T10:23:36.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.624+0000 7fb6401c3b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa8a0a37a0 con 0x55fa8a149460 2023-12-07T10:23:36.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.624+0000 7fb6401c3b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa8a0a10c0 con 0x55fa8a148b40 2023-12-07T10:23:36.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.624+0000 7fb61b7fe700 1 --2- 172.21.15.119:0/1559135940 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb6100a1c80 0x7fb6100a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.624+0000 7fb629d9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa8a148b40 0x55fa8a148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.624+0000 7fb629d9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa8a148b40 0x55fa8a148f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41680/0 (socket says 172.21.15.119:41680) 2023-12-07T10:23:36.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.624+0000 7fb629d9a700 1 -- 172.21.15.119:0/1474520175 learned_addr learned my addr 172.21.15.119:0/1474520175 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:36.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.624+0000 7fb629d9a700 1 --2- 172.21.15.119:0/1474520175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa8a14d150 0x55fa8a14d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.624+0000 7fb62a59b700 1 --2- 172.21.15.119:0/1474520175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa8a149460 0x55fa8a14cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.624+0000 7fb61b7fe700 1 -- 172.21.15.119:0/1559135940 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fb61c0cef80 con 0x7fb624107610 2023-12-07T10:23:36.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.624+0000 7fb62ad9c700 1 --2- 172.21.15.119:0/1559135940 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb6100a1c80 0x7fb6100a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.624+0000 7fb629d9a700 1 -- 172.21.15.119:0/1474520175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa8a149460 msgr2=0x55fa8a14cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.624+0000 7fb629d9a700 1 --2- 172.21.15.119:0/1474520175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa8a149460 0x55fa8a14cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.624+0000 7fb629d9a700 1 -- 172.21.15.119:0/1474520175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa8a14d150 msgr2=0x55fa8a14d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.624+0000 7fb629d9a700 1 --2- 172.21.15.119:0/1474520175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa8a14d150 0x55fa8a14d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.624+0000 7fb629d9a700 1 -- 172.21.15.119:0/1474520175 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa89f19630 con 0x55fa8a148b40 2023-12-07T10:23:36.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.625+0000 7fb629d9a700 1 --2- 172.21.15.119:0/1474520175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa8a148b40 0x55fa8a148f20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb6140118b0 tx=0x7fb6140058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7254469dac80bf8b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:36.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.625+0000 7fb62ad9c700 1 --2- 172.21.15.119:0/1559135940 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb6100a1c80 0x7fb6100a4140 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb61c098470 tx=0x7fb61c006230 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:36.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.625+0000 7fb618ff9700 1 -- 172.21.15.119:0/1474520175 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614004030 con 0x55fa8a148b40 2023-12-07T10:23:36.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.625+0000 7fb618ff9700 1 -- 172.21.15.119:0/1474520175 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb614004830 con 0x55fa8a148b40 2023-12-07T10:23:36.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.625+0000 7fb618ff9700 1 -- 172.21.15.119:0/1474520175 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6140053e0 con 0x55fa8a148b40 2023-12-07T10:23:36.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.626+0000 7fb6401c3b80 1 -- 172.21.15.119:0/1474520175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa8a148b40 msgr2=0x55fa8a148f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.626+0000 7fb6401c3b80 1 --2- 172.21.15.119:0/1474520175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa8a148b40 0x55fa8a148f20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb6140118b0 tx=0x7fb6140058e0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.626+0000 7fb6401c3b80 1 -- 172.21.15.119:0/1474520175 shutdown_connections 2023-12-07T10:23:36.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.626+0000 7fb6401c3b80 1 --2- 172.21.15.119:0/1474520175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa8a14d150 0x55fa8a14d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.626+0000 7fb6401c3b80 1 --2- 172.21.15.119:0/1474520175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa8a148b40 0x55fa8a148f20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.626+0000 7fb6401c3b80 1 --2- 172.21.15.119:0/1474520175 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa8a149460 0x55fa8a14cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.626+0000 7fb6401c3b80 1 -- 172.21.15.119:0/1474520175 >> 172.21.15.119:0/1474520175 conn(0x55fa8a077360 msgr2=0x55fa8a143130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:36.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.626+0000 7fb6401c3b80 1 -- 172.21.15.119:0/1474520175 shutdown_connections 2023-12-07T10:23:36.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.626+0000 7fb6401c3b80 1 -- 172.21.15.119:0/1474520175 wait complete. 2023-12-07T10:23:36.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.627+0000 7fb6401c3b80 1 -- start start 2023-12-07T10:23:36.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.627+0000 7fb6401c3b80 1 Processor -- start 2023-12-07T10:23:36.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.627+0000 7fb6401c3b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa8a148b40 0x55fa8a151d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.627+0000 7fb6401c3b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa8a149460 0x55fa8a152250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.627+0000 7fb6401c3b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa8a14d150 0x55fa8a152790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.627+0000 7fb6401c3b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa8a09c9b0 con 0x55fa8a148b40 2023-12-07T10:23:36.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.627+0000 7fb6401c3b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa8a056560 con 0x55fa8a149460 2023-12-07T10:23:36.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.627+0000 7fb6401c3b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa8a056200 con 0x55fa8a14d150 2023-12-07T10:23:36.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.627+0000 7fb6401c3b80 10 client.0 _ll_register_callbacks cb 0x55fa8a03d940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:36.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.628+0000 7fb62a59b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa8a149460 0x55fa8a152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.628+0000 7fb62a59b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa8a149460 0x55fa8a152250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:56044/0 (socket says 172.21.15.119:56044) 2023-12-07T10:23:36.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.628+0000 7fb62a59b700 1 -- 172.21.15.119:0/3116265549 learned_addr learned my addr 172.21.15.119:0/3116265549 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:36.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.628+0000 7fb629d9a700 1 --2- 172.21.15.119:0/3116265549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa8a148b40 0x55fa8a151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.628+0000 7fb62a59b700 1 -- 172.21.15.119:0/3116265549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa8a14d150 msgr2=0x55fa8a152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.628+0000 7fb629d9a700 1 --2- 172.21.15.119:0/3116265549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa8a14d150 0x55fa8a152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.628+0000 7fb62a59b700 1 --2- 172.21.15.119:0/3116265549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa8a14d150 0x55fa8a152790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.628+0000 7fb62a59b700 1 -- 172.21.15.119:0/3116265549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa8a148b40 msgr2=0x55fa8a151d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.628+0000 7fb62a59b700 1 --2- 172.21.15.119:0/3116265549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa8a148b40 0x55fa8a151d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.628+0000 7fb62a59b700 1 -- 172.21.15.119:0/3116265549 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb614019040 con 0x55fa8a149460 2023-12-07T10:23:36.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.628+0000 7fb629d9a700 1 --2- 172.21.15.119:0/3116265549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa8a14d150 0x55fa8a152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:23:36.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.628+0000 7fb62a59b700 1 --2- 172.21.15.119:0/3116265549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa8a149460 0x55fa8a152250 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb620012fd0 tx=0x7fb62000f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1f4611ede5c33d server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:36.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.628+0000 7fb60dffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:23:36.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.629+0000 7fb60dffb700 1 -- 172.21.15.119:0/3116265549 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620017070 con 0x55fa8a149460 2023-12-07T10:23:36.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.629+0000 7fb60dffb700 1 -- 172.21.15.119:0/3116265549 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fb620012070 con 0x55fa8a149460 2023-12-07T10:23:36.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.629+0000 7fb6401c3b80 1 -- 172.21.15.119:0/3116265549 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fa8a13bb00 con 0x55fa8a149460 2023-12-07T10:23:36.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.629+0000 7fb60dffb700 1 -- 172.21.15.119:0/3116265549 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb620012270 con 0x55fa8a149460 2023-12-07T10:23:36.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.629+0000 7fb6401c3b80 10 client.26744 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:36.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.629+0000 7fb6401c3b80 1 -- 172.21.15.119:0/3116265549 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55fa89ec5de0 con 0x55fa8a149460 2023-12-07T10:23:36.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.629+0000 7fb60dffb700 1 -- 172.21.15.119:0/3116265549 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fa8a13bb00 con 0x55fa8a149460 2023-12-07T10:23:36.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.629+0000 7fb60dffb700 1 -- 172.21.15.119:0/3116265549 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fb620010550 con 0x55fa8a149460 2023-12-07T10:23:36.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.630+0000 7fb6401c3b80 10 client.26744 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:36.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.630+0000 7fb6401c3b80 10 client.26744 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-07T10:23:36.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.630+0000 7fb6401c3b80 20 client.26744 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:36.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.630+0000 7fb6401c3b80 1 --2- 172.21.15.119:0/3116265549 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x55fa8a1623f0 0x55fa8a164810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.630+0000 7fb6401c3b80 4 client.26744 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:23:36.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.630+0000 7fb6401c3b80 1 -- 172.21.15.119:0/3116265549 --> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fa8a1177a0 con 0x55fa8a1623f0 2023-12-07T10:23:36.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.630+0000 7fb629d9a700 1 --2- 172.21.15.119:0/3116265549 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x55fa8a1623f0 0x55fa8a164810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.630+0000 7fb629d9a700 1 --2- 172.21.15.119:0/3116265549 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x55fa8a1623f0 0x55fa8a164810 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=56c3fbcdde829325 server_cookie=dd8423faa69f4b5 in_seq=0 out_seq=0 2023-12-07T10:23:36.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.630+0000 7fb60dffb700 10 client.26744 ms_handle_connect on v2:172.21.15.159:6836/3106097520 2023-12-07T10:23:36.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.631+0000 7fb60dffb700 1 -- 172.21.15.119:0/3116265549 <== mds.2 v2:172.21.15.159:6836/3106097520 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fa8a1177a0 con 0x55fa8a1623f0 2023-12-07T10:23:36.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.631+0000 7fb60dffb700 10 client.26744 handle_command_reply: tid=0 2023-12-07T10:23:36.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.631+0000 7fb6401c3b80 1 client.26744 shutdown 2023-12-07T10:23:36.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.631+0000 7fb6401c3b80 1 -- 172.21.15.119:0/3116265549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa8a149460 msgr2=0x55fa8a152250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.631+0000 7fb6401c3b80 1 --2- 172.21.15.119:0/3116265549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa8a149460 0x55fa8a152250 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb620012fd0 tx=0x7fb62000f6a0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.631+0000 7fb6401c3b80 1 -- 172.21.15.119:0/3116265549 shutdown_connections 2023-12-07T10:23:36.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.631+0000 7fb6401c3b80 1 --2- 172.21.15.119:0/3116265549 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x55fa8a1623f0 0x55fa8a164810 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.631+0000 7fb6401c3b80 1 --2- 172.21.15.119:0/3116265549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa8a14d150 0x55fa8a152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.631+0000 7fb6401c3b80 1 --2- 172.21.15.119:0/3116265549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa8a148b40 0x55fa8a151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.631+0000 7fb60dffb700 0 client.26744 ms_handle_reset on v2:172.21.15.159:6836/3106097520 2023-12-07T10:23:36.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.631+0000 7fb6401c3b80 1 --2- 172.21.15.119:0/3116265549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa8a149460 0x55fa8a152250 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.631+0000 7fb6401c3b80 1 -- 172.21.15.119:0/3116265549 >> 172.21.15.119:0/3116265549 conn(0x55fa8a077360 msgr2=0x55fa8a13dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:36.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.632+0000 7fb6401c3b80 1 -- 172.21.15.119:0/3116265549 shutdown_connections 2023-12-07T10:23:36.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.632+0000 7fb6401c3b80 1 -- 172.21.15.119:0/3116265549 wait complete. 2023-12-07T10:23:36.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.632+0000 7fb6401c3b80 20 client.26744 trim_cache size 0 max 16384 2023-12-07T10:23:36.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.649+0000 7fb6401c3b80 1 Processor -- start 2023-12-07T10:23:36.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.649+0000 7fb6401c3b80 1 -- start start 2023-12-07T10:23:36.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.649+0000 7fb6401c3b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa8a148b40 0x55fa8a148f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.649+0000 7fb6401c3b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa8a149460 0x55fa8a14b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.649+0000 7fb6401c3b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa8a14d150 0x55fa8a1430a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.649+0000 7fb6401c3b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa89f1b920 con 0x55fa8a149460 2023-12-07T10:23:36.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.649+0000 7fb6401c3b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa8a15faf0 con 0x55fa8a148b40 2023-12-07T10:23:36.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.649+0000 7fb6401c3b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa8a150290 con 0x55fa8a14d150 2023-12-07T10:23:36.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.649+0000 7fb629d9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa8a148b40 0x55fa8a148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.649+0000 7fb62a59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa8a149460 0x55fa8a14b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.649+0000 7fb629d9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa8a148b40 0x55fa8a148f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:56054/0 (socket says 172.21.15.119:56054) 2023-12-07T10:23:36.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.649+0000 7fb629d9a700 1 -- 172.21.15.119:0/1902173501 learned_addr learned my addr 172.21.15.119:0/1902173501 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:36.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.649+0000 7fb629d9a700 1 --2- 172.21.15.119:0/1902173501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa8a14d150 0x55fa8a1430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.650+0000 7fb629d9a700 1 -- 172.21.15.119:0/1902173501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa8a14d150 msgr2=0x55fa8a1430a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.650+0000 7fb629d9a700 1 --2- 172.21.15.119:0/1902173501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa8a14d150 0x55fa8a1430a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.650+0000 7fb629d9a700 1 -- 172.21.15.119:0/1902173501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa8a149460 msgr2=0x55fa8a14b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.650+0000 7fb629d9a700 1 --2- 172.21.15.119:0/1902173501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa8a149460 0x55fa8a14b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.650+0000 7fb629d9a700 1 -- 172.21.15.119:0/1902173501 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa89eaec60 con 0x55fa8a148b40 2023-12-07T10:23:36.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.650+0000 7fb62a59b700 1 --2- 172.21.15.119:0/1902173501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa8a149460 0x55fa8a14b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:36.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.650+0000 7fb629d9a700 1 --2- 172.21.15.119:0/1902173501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa8a148b40 0x55fa8a148f20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb614009c50 tx=0x7fb614009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51e69cb72edc4c81 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:36.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.650+0000 7fb6197fa700 1 -- 172.21.15.119:0/1902173501 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6140049a0 con 0x55fa8a148b40 2023-12-07T10:23:36.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.650+0000 7fb6197fa700 1 -- 172.21.15.119:0/1902173501 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb614036030 con 0x55fa8a148b40 2023-12-07T10:23:36.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.650+0000 7fb6197fa700 1 -- 172.21.15.119:0/1902173501 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6140051f0 con 0x55fa8a148b40 2023-12-07T10:23:36.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.651+0000 7fb6401c3b80 1 -- 172.21.15.119:0/1902173501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa8a148b40 msgr2=0x55fa8a148f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.651+0000 7fb6401c3b80 1 --2- 172.21.15.119:0/1902173501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa8a148b40 0x55fa8a148f20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb614009c50 tx=0x7fb614009da0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.651+0000 7fb6401c3b80 1 -- 172.21.15.119:0/1902173501 shutdown_connections 2023-12-07T10:23:36.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.651+0000 7fb6401c3b80 1 --2- 172.21.15.119:0/1902173501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa8a14d150 0x55fa8a1430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.651+0000 7fb6401c3b80 1 --2- 172.21.15.119:0/1902173501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa8a148b40 0x55fa8a148f20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.651+0000 7fb6401c3b80 1 --2- 172.21.15.119:0/1902173501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa8a149460 0x55fa8a14b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.651+0000 7fb6401c3b80 1 -- 172.21.15.119:0/1902173501 >> 172.21.15.119:0/1902173501 conn(0x55fa8a077360 msgr2=0x55fa8a14d530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:36.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.651+0000 7fb6401c3b80 1 -- 172.21.15.119:0/1902173501 shutdown_connections 2023-12-07T10:23:36.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.651+0000 7fb6401c3b80 1 -- 172.21.15.119:0/1902173501 wait complete. 2023-12-07T10:23:36.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.652+0000 7fb6401c3b80 1 -- start start 2023-12-07T10:23:36.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.652+0000 7fb6401c3b80 1 Processor -- start 2023-12-07T10:23:36.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.652+0000 7fb6401c3b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa8a148b40 0x55fa8a155320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.652+0000 7fb6401c3b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa8a149460 0x55fa8a155860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.652+0000 7fb6401c3b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa8a14d150 0x55fa8a155da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.652+0000 7fb6401c3b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa8a1b78e0 con 0x55fa8a148b40 2023-12-07T10:23:36.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.652+0000 7fb6401c3b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa8a1b7a20 con 0x55fa8a14d150 2023-12-07T10:23:36.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.652+0000 7fb6401c3b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa8a1b7b60 con 0x55fa8a149460 2023-12-07T10:23:36.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.652+0000 7fb6401c3b80 10 client.0 _ll_register_callbacks cb 0x55fa8a03d940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:36.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.653+0000 7fb629d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa8a148b40 0x55fa8a155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.653+0000 7fb62a59b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa8a149460 0x55fa8a155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.653+0000 7fb62a59b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa8a149460 0x55fa8a155860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41714/0 (socket says 172.21.15.119:41714) 2023-12-07T10:23:36.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.653+0000 7fb62a59b700 1 -- 172.21.15.119:0/3398926997 learned_addr learned my addr 172.21.15.119:0/3398926997 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:36.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.653+0000 7fb629d9a700 1 --2- 172.21.15.119:0/3398926997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa8a14d150 0x55fa8a155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.653+0000 7fb62a59b700 1 -- 172.21.15.119:0/3398926997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa8a14d150 msgr2=0x55fa8a155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.653+0000 7fb62a59b700 1 --2- 172.21.15.119:0/3398926997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa8a14d150 0x55fa8a155da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.653+0000 7fb62a59b700 1 -- 172.21.15.119:0/3398926997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa8a148b40 msgr2=0x55fa8a155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.653+0000 7fb62a59b700 1 --2- 172.21.15.119:0/3398926997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa8a148b40 0x55fa8a155320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.653+0000 7fb62a59b700 1 -- 172.21.15.119:0/3398926997 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa89ec5de0 con 0x55fa8a149460 2023-12-07T10:23:36.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.653+0000 7fb629d9a700 1 --2- 172.21.15.119:0/3398926997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa8a148b40 0x55fa8a155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:23:36.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.653+0000 7fb62a59b700 1 --2- 172.21.15.119:0/3398926997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa8a149460 0x55fa8a155860 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb620012fa0 tx=0x7fb620012330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=657c7a8488aee825 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:36.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.653+0000 7fb60dffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:23:36.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.654+0000 7fb60dffb700 1 -- 172.21.15.119:0/3398926997 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620017040 con 0x55fa8a149460 2023-12-07T10:23:36.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.654+0000 7fb60dffb700 1 -- 172.21.15.119:0/3398926997 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fb6200050c0 con 0x55fa8a149460 2023-12-07T10:23:36.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.654+0000 7fb6401c3b80 1 -- 172.21.15.119:0/3398926997 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb614004830 con 0x55fa8a149460 2023-12-07T10:23:36.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.654+0000 7fb60dffb700 1 -- 172.21.15.119:0/3398926997 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb620005280 con 0x55fa8a149460 2023-12-07T10:23:36.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.654+0000 7fb6401c3b80 10 client.26725 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:36.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.654+0000 7fb6401c3b80 1 -- 172.21.15.119:0/3398926997 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7fb614027040 con 0x55fa8a149460 2023-12-07T10:23:36.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.654+0000 7fb60dffb700 1 -- 172.21.15.119:0/3398926997 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb614004830 con 0x55fa8a149460 2023-12-07T10:23:36.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.654+0000 7fb60dffb700 1 -- 172.21.15.119:0/3398926997 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fb620003cc0 con 0x55fa8a149460 2023-12-07T10:23:36.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.654+0000 7fb6401c3b80 10 client.26725 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:36.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.654+0000 7fb6401c3b80 10 client.26725 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-07T10:23:36.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.654+0000 7fb6401c3b80 20 client.26725 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:36.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.654+0000 7fb6401c3b80 1 --2- 172.21.15.119:0/3398926997 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x55fa8a1b9370 0x55fa8a1bb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.654+0000 7fb6401c3b80 4 client.26725 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:23:36.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.654+0000 7fb6401c3b80 1 -- 172.21.15.119:0/3398926997 --> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fa8a062fd0 con 0x55fa8a1b9370 2023-12-07T10:23:36.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.655+0000 7fb629d9a700 1 --2- 172.21.15.119:0/3398926997 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x55fa8a1b9370 0x55fa8a1bb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.655+0000 7fb629d9a700 1 --2- 172.21.15.119:0/3398926997 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x55fa8a1b9370 0x55fa8a1bb790 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=802f27e32f0cffa5 server_cookie=ef0937c9503a0413 in_seq=0 out_seq=0 2023-12-07T10:23:36.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.655+0000 7fb60dffb700 10 client.26725 ms_handle_connect on v2:172.21.15.159:6836/3106097520 2023-12-07T10:23:36.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.655+0000 7fb60dffb700 1 -- 172.21.15.119:0/3398926997 <== mds.2 v2:172.21.15.159:6836/3106097520 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fa8a062fd0 con 0x55fa8a1b9370 2023-12-07T10:23:36.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.655+0000 7fb60dffb700 10 client.26725 handle_command_reply: tid=0 2023-12-07T10:23:36.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.655+0000 7fb6401c3b80 1 client.26725 shutdown 2023-12-07T10:23:36.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.655+0000 7fb6401c3b80 1 -- 172.21.15.119:0/3398926997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa8a149460 msgr2=0x55fa8a155860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.655+0000 7fb6401c3b80 1 --2- 172.21.15.119:0/3398926997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa8a149460 0x55fa8a155860 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb620012fa0 tx=0x7fb620012330 comp rx=0 tx=0).stop 2023-12-07T10:23:36.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.655+0000 7fb6401c3b80 1 -- 172.21.15.119:0/3398926997 shutdown_connections 2023-12-07T10:23:36.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.655+0000 7fb6401c3b80 1 --2- 172.21.15.119:0/3398926997 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x55fa8a1b9370 0x55fa8a1bb790 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.655+0000 7fb6401c3b80 1 --2- 172.21.15.119:0/3398926997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55fa8a14d150 0x55fa8a155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.655+0000 7fb6401c3b80 1 --2- 172.21.15.119:0/3398926997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa8a148b40 0x55fa8a155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.655+0000 7fb60dffb700 0 client.26725 ms_handle_reset on v2:172.21.15.159:6836/3106097520 2023-12-07T10:23:36.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.655+0000 7fb6401c3b80 1 --2- 172.21.15.119:0/3398926997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa8a149460 0x55fa8a155860 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.656+0000 7fb6401c3b80 1 -- 172.21.15.119:0/3398926997 >> 172.21.15.119:0/3398926997 conn(0x55fa8a077360 msgr2=0x55fa8a142fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:36.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.656+0000 7fb6401c3b80 1 -- 172.21.15.119:0/3398926997 shutdown_connections 2023-12-07T10:23:36.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.656+0000 7fb6401c3b80 1 -- 172.21.15.119:0/3398926997 wait complete. 2023-12-07T10:23:36.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.656+0000 7fb6401c3b80 20 client.26725 trim_cache size 0 max 16384 2023-12-07T10:23:36.663 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:23:36.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.656+0000 7fb62c7ff700 1 -- 172.21.15.119:0/1559135940 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb6100a1c80 msgr2=0x7fb6100a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.656+0000 7fb62c7ff700 1 --2- 172.21.15.119:0/1559135940 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb6100a1c80 0x7fb6100a4140 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb61c098470 tx=0x7fb61c006230 comp rx=0 tx=0).stop 2023-12-07T10:23:36.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.656+0000 7fb62c7ff700 1 -- 172.21.15.119:0/1559135940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb624107610 msgr2=0x7fb624199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.656+0000 7fb62c7ff700 1 --2- 172.21.15.119:0/1559135940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb624107610 0x7fb624199a70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00b0d0 tx=0x7fb61c00bd50 comp rx=0 tx=0).stop 2023-12-07T10:23:36.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.656+0000 7fb62c7ff700 1 -- 172.21.15.119:0/1559135940 shutdown_connections 2023-12-07T10:23:36.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.656+0000 7fb62c7ff700 1 --2- 172.21.15.119:0/1559135940 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb6100a1c80 0x7fb6100a4140 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.656+0000 7fb62c7ff700 1 --2- 172.21.15.119:0/1559135940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb624107610 0x7fb624199a70 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.656+0000 7fb62c7ff700 1 --2- 172.21.15.119:0/1559135940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb624101550 0x7fb62419f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.657+0000 7fb62c7ff700 1 --2- 172.21.15.119:0/1559135940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb624101f10 0x7fb62419fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.657+0000 7fb62c7ff700 1 -- 172.21.15.119:0/1559135940 >> 172.21.15.119:0/1559135940 conn(0x7fb6240fd020 msgr2=0x7fb6240fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:36.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.657+0000 7fb62c7ff700 1 -- 172.21.15.119:0/1559135940 shutdown_connections 2023-12-07T10:23:36.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.657+0000 7fb62c7ff700 1 -- 172.21.15.119:0/1559135940 wait complete. 2023-12-07T10:23:36.678 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-07T10:23:36.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.906+0000 7fa90bfff700 1 -- 172.21.15.119:0/1075896808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa904068a50 msgr2=0x7fa904068e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.906+0000 7fa90bfff700 1 --2- 172.21.15.119:0/1075896808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa904068a50 0x7fa904068e30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa900009990 tx=0x7fa900009ca0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.906+0000 7fa90bfff700 1 -- 172.21.15.119:0/1075896808 shutdown_connections 2023-12-07T10:23:36.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.906+0000 7fa90bfff700 1 --2- 172.21.15.119:0/1075896808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa904106ad0 0x7fa90410e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.906+0000 7fa90bfff700 1 --2- 172.21.15.119:0/1075896808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa904068a50 0x7fa904068e30 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.906+0000 7fa90bfff700 1 --2- 172.21.15.119:0/1075896808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa904106110 0x7fa904106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.906+0000 7fa90bfff700 1 -- 172.21.15.119:0/1075896808 >> 172.21.15.119:0/1075896808 conn(0x7fa904077610 msgr2=0x7fa904077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:36.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.908+0000 7fa90bfff700 1 -- 172.21.15.119:0/1075896808 shutdown_connections 2023-12-07T10:23:36.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.908+0000 7fa90bfff700 1 -- 172.21.15.119:0/1075896808 wait complete. 2023-12-07T10:23:36.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.909+0000 7fa90bfff700 1 Processor -- start 2023-12-07T10:23:36.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.910+0000 7fa90bfff700 1 -- start start 2023-12-07T10:23:36.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.910+0000 7fa90bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa904068a50 0x7fa904072f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.911+0000 7fa90bfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa904106110 0x7fa90406b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.911+0000 7fa90bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa904106ad0 0x7fa90406b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.911+0000 7fa90bfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90406bab0 con 0x7fa904106ad0 2023-12-07T10:23:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.911+0000 7fa90bfff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90406bc20 con 0x7fa904068a50 2023-12-07T10:23:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.911+0000 7fa90bfff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90406bde0 con 0x7fa904106110 2023-12-07T10:23:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.911+0000 7fa90959a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa904106110 0x7fa90406b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.911+0000 7fa90a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa904106ad0 0x7fa90406b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.911+0000 7fa90959a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa904106110 0x7fa90406b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41738/0 (socket says 172.21.15.119:41738) 2023-12-07T10:23:36.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.911+0000 7fa90959a700 1 -- 172.21.15.119:0/3225182170 learned_addr learned my addr 172.21.15.119:0/3225182170 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:36.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.911+0000 7fa909d9b700 1 --2- 172.21.15.119:0/3225182170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa904068a50 0x7fa904072f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.912+0000 7fa90959a700 1 -- 172.21.15.119:0/3225182170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa904068a50 msgr2=0x7fa904072f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.912+0000 7fa90959a700 1 --2- 172.21.15.119:0/3225182170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa904068a50 0x7fa904072f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.912+0000 7fa90959a700 1 -- 172.21.15.119:0/3225182170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa904106ad0 msgr2=0x7fa90406b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.912+0000 7fa90959a700 1 --2- 172.21.15.119:0/3225182170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa904106ad0 0x7fa90406b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.912+0000 7fa90959a700 1 -- 172.21.15.119:0/3225182170 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9000095f0 con 0x7fa904106110 2023-12-07T10:23:36.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.912+0000 7fa90a59c700 1 --2- 172.21.15.119:0/3225182170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa904106ad0 0x7fa90406b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:36.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.912+0000 7fa909d9b700 1 --2- 172.21.15.119:0/3225182170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa904068a50 0x7fa904072f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:23:36.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.913+0000 7fa90959a700 1 --2- 172.21.15.119:0/3225182170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa904106110 0x7fa90406b030 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa8f400d950 tx=0x7fa8f400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:36.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.913+0000 7fa8faffd700 1 -- 172.21.15.119:0/3225182170 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4009940 con 0x7fa904106110 2023-12-07T10:23:36.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.913+0000 7fa8faffd700 1 -- 172.21.15.119:0/3225182170 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8f4010460 con 0x7fa904106110 2023-12-07T10:23:36.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.913+0000 7fa90bfff700 1 -- 172.21.15.119:0/3225182170 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa904110b00 con 0x7fa904106110 2023-12-07T10:23:36.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.913+0000 7fa8faffd700 1 -- 172.21.15.119:0/3225182170 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f400f620 con 0x7fa904106110 2023-12-07T10:23:36.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.914+0000 7fa90bfff700 1 -- 172.21.15.119:0/3225182170 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa904111010 con 0x7fa904106110 2023-12-07T10:23:36.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.920+0000 7fa8faffd700 1 -- 172.21.15.119:0/3225182170 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fa8f400f780 con 0x7fa904106110 2023-12-07T10:23:36.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.921+0000 7fa91f95eb80 1 Processor -- start 2023-12-07T10:23:36.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.921+0000 7fa8faffd700 1 --2- 172.21.15.119:0/3225182170 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa8f00a1e90 0x7fa8f00a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.921+0000 7fa8faffd700 1 -- 172.21.15.119:0/3225182170 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fa8f40cdae0 con 0x7fa904106110 2023-12-07T10:23:36.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.921+0000 7fa90a59c700 1 --2- 172.21.15.119:0/3225182170 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa8f00a1e90 0x7fa8f00a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.922+0000 7fa91f95eb80 1 -- start start 2023-12-07T10:23:36.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.922+0000 7fa91f95eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3a78eb40 0x55df3a78ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.922+0000 7fa91f95eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df3a78f460 0x55df3a792c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.922+0000 7fa91f95eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df3a793150 0x55df3a793570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.922+0000 7fa91f95eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3a7027c0 con 0x55df3a78eb40 2023-12-07T10:23:36.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.922+0000 7fa91f95eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3a6e97a0 con 0x55df3a78f460 2023-12-07T10:23:36.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.922+0000 7fa91f95eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3a6e70c0 con 0x55df3a793150 2023-12-07T10:23:36.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.922+0000 7fa90a59c700 1 --2- 172.21.15.119:0/3225182170 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa8f00a1e90 0x7fa8f00a4350 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00a600 tx=0x7fa8fc00c040 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:36.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.922+0000 7fa90a59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df3a793150 0x55df3a793570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.922+0000 7fa90a59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df3a793150 0x55df3a793570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41744/0 (socket says 172.21.15.119:41744) 2023-12-07T10:23:36.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.922+0000 7fa90a59c700 1 -- 172.21.15.119:0/1140208940 learned_addr learned my addr 172.21.15.119:0/1140208940 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:36.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.923+0000 7fa90959a700 1 --2- 172.21.15.119:0/1140208940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df3a78f460 0x55df3a792c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.923+0000 7fa909d9b700 1 --2- 172.21.15.119:0/1140208940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3a78eb40 0x55df3a78ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.923+0000 7fa90959a700 1 -- 172.21.15.119:0/1140208940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df3a793150 msgr2=0x55df3a793570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.923+0000 7fa90959a700 1 --2- 172.21.15.119:0/1140208940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df3a793150 0x55df3a793570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.923+0000 7fa90959a700 1 -- 172.21.15.119:0/1140208940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3a78eb40 msgr2=0x55df3a78ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.923+0000 7fa90959a700 1 --2- 172.21.15.119:0/1140208940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3a78eb40 0x55df3a78ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.923+0000 7fa90959a700 1 -- 172.21.15.119:0/1140208940 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df3a55f630 con 0x55df3a78f460 2023-12-07T10:23:36.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.923+0000 7fa909d9b700 1 --2- 172.21.15.119:0/1140208940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3a78eb40 0x55df3a78ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:36.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.924+0000 7fa90959a700 1 --2- 172.21.15.119:0/1140208940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df3a78f460 0x55df3a792c10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa8f40cae80 tx=0x7fa8f4099eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d13e0a5f3b4310d server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:36.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.924+0000 7fa8effff700 1 -- 172.21.15.119:0/1140208940 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f409e620 con 0x55df3a78f460 2023-12-07T10:23:36.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.924+0000 7fa8effff700 1 -- 172.21.15.119:0/1140208940 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8f40a2860 con 0x55df3a78f460 2023-12-07T10:23:36.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.924+0000 7fa8effff700 1 -- 172.21.15.119:0/1140208940 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f409a970 con 0x55df3a78f460 2023-12-07T10:23:36.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.925+0000 7fa91f95eb80 1 -- 172.21.15.119:0/1140208940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df3a78f460 msgr2=0x55df3a792c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.925+0000 7fa91f95eb80 1 --2- 172.21.15.119:0/1140208940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df3a78f460 0x55df3a792c10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa8f40cae80 tx=0x7fa8f4099eb0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.925+0000 7fa91f95eb80 1 -- 172.21.15.119:0/1140208940 shutdown_connections 2023-12-07T10:23:36.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.925+0000 7fa91f95eb80 1 --2- 172.21.15.119:0/1140208940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df3a793150 0x55df3a793570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.925+0000 7fa91f95eb80 1 --2- 172.21.15.119:0/1140208940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3a78eb40 0x55df3a78ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.925+0000 7fa91f95eb80 1 --2- 172.21.15.119:0/1140208940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df3a78f460 0x55df3a792c10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.925+0000 7fa91f95eb80 1 -- 172.21.15.119:0/1140208940 >> 172.21.15.119:0/1140208940 conn(0x55df3a6bd360 msgr2=0x55df3a789130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:36.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.925+0000 7fa91f95eb80 1 -- 172.21.15.119:0/1140208940 shutdown_connections 2023-12-07T10:23:36.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.925+0000 7fa91f95eb80 1 -- 172.21.15.119:0/1140208940 wait complete. 2023-12-07T10:23:36.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.926+0000 7fa91f95eb80 1 -- start start 2023-12-07T10:23:36.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.926+0000 7fa91f95eb80 1 Processor -- start 2023-12-07T10:23:36.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.927+0000 7fa91f95eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df3a78eb40 0x55df3a79a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.927+0000 7fa91f95eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df3a78f460 0x55df3a797b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.927+0000 7fa91f95eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3a793150 0x55df3a798210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.927+0000 7fa91f95eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3a6e29b0 con 0x55df3a793150 2023-12-07T10:23:36.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.927+0000 7fa91f95eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3a69c560 con 0x55df3a78f460 2023-12-07T10:23:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.927+0000 7fa91f95eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3a69c200 con 0x55df3a78eb40 2023-12-07T10:23:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.927+0000 7fa91f95eb80 10 client.0 _ll_register_callbacks cb 0x55df3a683940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.927+0000 7fa90959a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df3a78f460 0x55df3a797b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.927+0000 7fa90959a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df3a78f460 0x55df3a797b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:56096/0 (socket says 172.21.15.119:56096) 2023-12-07T10:23:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.927+0000 7fa90959a700 1 -- 172.21.15.119:0/1932483596 learned_addr learned my addr 172.21.15.119:0/1932483596 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.927+0000 7fa90a59c700 1 --2- 172.21.15.119:0/1932483596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3a793150 0x55df3a798210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.927+0000 7fa90959a700 1 -- 172.21.15.119:0/1932483596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df3a78eb40 msgr2=0x55df3a79a8a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-07T10:23:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.927+0000 7fa90959a700 1 --2- 172.21.15.119:0/1932483596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df3a78eb40 0x55df3a79a8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.927+0000 7fa90959a700 1 -- 172.21.15.119:0/1932483596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3a793150 msgr2=0x55df3a798210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.927+0000 7fa90959a700 1 --2- 172.21.15.119:0/1932483596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3a793150 0x55df3a798210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.927+0000 7fa90959a700 1 -- 172.21.15.119:0/1932483596 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa90410ecb0 con 0x55df3a78f460 2023-12-07T10:23:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.927+0000 7fa90959a700 1 --2- 172.21.15.119:0/1932483596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df3a78f460 0x55df3a797b90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa8f409a410 tx=0x7fa8f4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a199577c2e1a70cb server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.928+0000 7fa8ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:23:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.928+0000 7fa91f95eb80 1 -- 172.21.15.119:0/1932483596 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df3a781b00 con 0x55df3a78f460 2023-12-07T10:23:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.928+0000 7fa8ed7fa700 1 -- 172.21.15.119:0/1932483596 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4099bc0 con 0x55df3a78f460 2023-12-07T10:23:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.928+0000 7fa8ed7fa700 1 -- 172.21.15.119:0/1932483596 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fa8f409a730 con 0x55df3a78f460 2023-12-07T10:23:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.928+0000 7fa8ed7fa700 1 -- 172.21.15.119:0/1932483596 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8f409a8f0 con 0x55df3a78f460 2023-12-07T10:23:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.928+0000 7fa91f95eb80 10 client.26756 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.928+0000 7fa91f95eb80 1 -- 172.21.15.119:0/1932483596 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({fsmap=36}) v3 -- 0x7fa904111010 con 0x55df3a78f460 2023-12-07T10:23:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.928+0000 7fa8ed7fa700 1 -- 172.21.15.119:0/1932483596 <== mon.2 v2:172.21.15.153:3300/0 4 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fa8f40aa070 con 0x55df3a78f460 2023-12-07T10:23:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.928+0000 7fa91f95eb80 10 client.26756 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.928+0000 7fa91f95eb80 10 client.26756 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2023-12-07T10:23:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.928+0000 7fa91f95eb80 20 client.26756 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.928+0000 7fa91f95eb80 1 --2- 172.21.15.119:0/1932483596 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55df3a7a7e20 0x55df3a7aa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.928+0000 7fa91f95eb80 4 client.26756 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:23:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.928+0000 7fa91f95eb80 1 -- 172.21.15.119:0/1932483596 --> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df3a75d7a0 con 0x55df3a7a7e20 2023-12-07T10:23:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.929+0000 7fa909d9b700 1 --2- 172.21.15.119:0/1932483596 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55df3a7a7e20 0x55df3a7aa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.929+0000 7fa909d9b700 1 --2- 172.21.15.119:0/1932483596 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55df3a7a7e20 0x55df3a7aa210 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=568a460ac5d13f8c server_cookie=22246d2a8264f41b in_seq=0 out_seq=0 2023-12-07T10:23:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.929+0000 7fa8ed7fa700 10 client.26756 ms_handle_connect on v2:172.21.15.119:6838/1920762602 2023-12-07T10:23:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.930+0000 7fa8ed7fa700 1 -- 172.21.15.119:0/1932483596 <== mds.1 v2:172.21.15.119:6838/1920762602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55df3a75d7a0 con 0x55df3a7a7e20 2023-12-07T10:23:36.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.930+0000 7fa8ed7fa700 10 client.26756 handle_command_reply: tid=0 2023-12-07T10:23:36.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.930+0000 7fa91f95eb80 1 client.26756 shutdown 2023-12-07T10:23:36.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.930+0000 7fa91f95eb80 1 -- 172.21.15.119:0/1932483596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df3a78f460 msgr2=0x55df3a797b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.931+0000 7fa91f95eb80 1 --2- 172.21.15.119:0/1932483596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df3a78f460 0x55df3a797b90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa8f409a410 tx=0x7fa8f4000bc0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.931+0000 7fa91f95eb80 1 -- 172.21.15.119:0/1932483596 shutdown_connections 2023-12-07T10:23:36.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.931+0000 7fa91f95eb80 1 --2- 172.21.15.119:0/1932483596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3a793150 0x55df3a798210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.931+0000 7fa91f95eb80 1 --2- 172.21.15.119:0/1932483596 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55df3a7a7e20 0x55df3a7aa210 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.931+0000 7fa91f95eb80 1 --2- 172.21.15.119:0/1932483596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df3a78eb40 0x55df3a79a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.931+0000 7fa91f95eb80 1 --2- 172.21.15.119:0/1932483596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df3a78f460 0x55df3a797b90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.931+0000 7fa8ed7fa700 0 client.26756 ms_handle_reset on v2:172.21.15.119:6838/1920762602 2023-12-07T10:23:36.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.931+0000 7fa91f95eb80 1 -- 172.21.15.119:0/1932483596 >> 172.21.15.119:0/1932483596 conn(0x55df3a6bd360 msgr2=0x55df3a783d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:36.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.931+0000 7fa91f95eb80 1 -- 172.21.15.119:0/1932483596 shutdown_connections 2023-12-07T10:23:36.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.931+0000 7fa91f95eb80 1 -- 172.21.15.119:0/1932483596 wait complete. 2023-12-07T10:23:36.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.931+0000 7fa91f95eb80 20 client.26756 trim_cache size 0 max 16384 2023-12-07T10:23:36.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.955+0000 7fa91f95eb80 1 Processor -- start 2023-12-07T10:23:36.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.956+0000 7fa91f95eb80 1 -- start start 2023-12-07T10:23:36.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.956+0000 7fa91f95eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df3a78eb40 0x55df3a7937b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.956+0000 7fa91f95eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3a78f460 0x55df3a792230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.956+0000 7fa91f95eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df3a793150 0x55df3a792770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.956+0000 7fa91f95eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3a561920 con 0x55df3a78f460 2023-12-07T10:23:36.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.956+0000 7fa91f95eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3a789a20 con 0x55df3a793150 2023-12-07T10:23:36.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.956+0000 7fa91f95eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3a7af400 con 0x55df3a78eb40 2023-12-07T10:23:36.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.956+0000 7fa909d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df3a78eb40 0x55df3a7937b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.956+0000 7fa90959a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3a78f460 0x55df3a792230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.956+0000 7fa909d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df3a78eb40 0x55df3a7937b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41758/0 (socket says 172.21.15.119:41758) 2023-12-07T10:23:36.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.956+0000 7fa909d9b700 1 -- 172.21.15.119:0/3609889479 learned_addr learned my addr 172.21.15.119:0/3609889479 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:36.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.957+0000 7fa90a59c700 1 --2- 172.21.15.119:0/3609889479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df3a793150 0x55df3a792770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.957+0000 7fa909d9b700 1 -- 172.21.15.119:0/3609889479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df3a793150 msgr2=0x55df3a792770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.957+0000 7fa909d9b700 1 --2- 172.21.15.119:0/3609889479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df3a793150 0x55df3a792770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.957+0000 7fa909d9b700 1 -- 172.21.15.119:0/3609889479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3a78f460 msgr2=0x55df3a792230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.957+0000 7fa909d9b700 1 --2- 172.21.15.119:0/3609889479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3a78f460 0x55df3a792230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.957+0000 7fa909d9b700 1 -- 172.21.15.119:0/3609889479 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df3a50bde0 con 0x55df3a78eb40 2023-12-07T10:23:36.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.957+0000 7fa90959a700 1 --2- 172.21.15.119:0/3609889479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3a78f460 0x55df3a792230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:36.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.957+0000 7fa909d9b700 1 --2- 172.21.15.119:0/3609889479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df3a78eb40 0x55df3a7937b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa900009990 tx=0x7fa90001c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc08b71250373782 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:36.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.958+0000 7fa8f8ff9700 1 -- 172.21.15.119:0/3609889479 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90001f040 con 0x55df3a78eb40 2023-12-07T10:23:36.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.958+0000 7fa8f8ff9700 1 -- 172.21.15.119:0/3609889479 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa90000ba90 con 0x55df3a78eb40 2023-12-07T10:23:36.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.958+0000 7fa8f8ff9700 1 -- 172.21.15.119:0/3609889479 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9000039f0 con 0x55df3a78eb40 2023-12-07T10:23:36.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.959+0000 7fa91f95eb80 1 -- 172.21.15.119:0/3609889479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df3a78eb40 msgr2=0x55df3a7937b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.959+0000 7fa91f95eb80 1 --2- 172.21.15.119:0/3609889479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df3a78eb40 0x55df3a7937b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa900009990 tx=0x7fa90001c040 comp rx=0 tx=0).stop 2023-12-07T10:23:36.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.959+0000 7fa91f95eb80 1 -- 172.21.15.119:0/3609889479 shutdown_connections 2023-12-07T10:23:36.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.959+0000 7fa91f95eb80 1 --2- 172.21.15.119:0/3609889479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df3a793150 0x55df3a792770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.959+0000 7fa91f95eb80 1 --2- 172.21.15.119:0/3609889479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df3a78eb40 0x55df3a7937b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.959+0000 7fa91f95eb80 1 --2- 172.21.15.119:0/3609889479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3a78f460 0x55df3a792230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.959+0000 7fa91f95eb80 1 -- 172.21.15.119:0/3609889479 >> 172.21.15.119:0/3609889479 conn(0x55df3a6bd360 msgr2=0x55df3a79a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:36.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.959+0000 7fa91f95eb80 1 -- 172.21.15.119:0/3609889479 shutdown_connections 2023-12-07T10:23:36.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.959+0000 7fa91f95eb80 1 -- 172.21.15.119:0/3609889479 wait complete. 2023-12-07T10:23:36.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.960+0000 7fa91f95eb80 1 -- start start 2023-12-07T10:23:36.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.960+0000 7fa91f95eb80 1 Processor -- start 2023-12-07T10:23:36.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.961+0000 7fa91f95eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3a78eb40 0x55df3a79f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.961+0000 7fa91f95eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df3a78f460 0x55df3a79f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.961+0000 7fa91f95eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df3a793150 0x55df3a79d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.961+0000 7fa91f95eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3a79de20 con 0x55df3a78eb40 2023-12-07T10:23:36.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.961+0000 7fa91f95eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3a79df90 con 0x55df3a793150 2023-12-07T10:23:36.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.961+0000 7fa91f95eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3a79e150 con 0x55df3a78f460 2023-12-07T10:23:36.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.961+0000 7fa91f95eb80 10 client.0 _ll_register_callbacks cb 0x55df3a683940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:36.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.961+0000 7fa909d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3a78eb40 0x55df3a79f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.961+0000 7fa90a59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df3a793150 0x55df3a79d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.961+0000 7fa909d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3a78eb40 0x55df3a79f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34926/0 (socket says 172.21.15.119:34926) 2023-12-07T10:23:36.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.961+0000 7fa90a59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df3a793150 0x55df3a79d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:56112/0 (socket says 172.21.15.119:56112) 2023-12-07T10:23:36.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.961+0000 7fa909d9b700 1 -- 172.21.15.119:0/4066220413 learned_addr learned my addr 172.21.15.119:0/4066220413 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:36.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.961+0000 7fa90959a700 1 --2- 172.21.15.119:0/4066220413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df3a78f460 0x55df3a79f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.962+0000 7fa909d9b700 1 -- 172.21.15.119:0/4066220413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df3a78f460 msgr2=0x55df3a79f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.962+0000 7fa909d9b700 1 --2- 172.21.15.119:0/4066220413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df3a78f460 0x55df3a79f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.962+0000 7fa909d9b700 1 -- 172.21.15.119:0/4066220413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df3a793150 msgr2=0x55df3a79d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:36.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.962+0000 7fa909d9b700 1 --2- 172.21.15.119:0/4066220413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df3a793150 0x55df3a79d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.962+0000 7fa909d9b700 1 -- 172.21.15.119:0/4066220413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa904111010 con 0x55df3a78eb40 2023-12-07T10:23:36.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.962+0000 7fa909d9b700 1 --2- 172.21.15.119:0/4066220413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3a78eb40 0x55df3a79f0c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa900009520 tx=0x7fa900009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f478e1f78696c4f2 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:36.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.962+0000 7fa8ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:23:36.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.962+0000 7fa8ed7fa700 1 -- 172.21.15.119:0/4066220413 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90001f070 con 0x55df3a78eb40 2023-12-07T10:23:36.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.962+0000 7fa8ed7fa700 1 -- 172.21.15.119:0/4066220413 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fa900026480 con 0x55df3a78eb40 2023-12-07T10:23:36.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.962+0000 7fa91f95eb80 1 -- 172.21.15.119:0/4066220413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa8f40a2860 con 0x55df3a78eb40 2023-12-07T10:23:36.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.963+0000 7fa8ed7fa700 1 -- 172.21.15.119:0/4066220413 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa900026660 con 0x55df3a78eb40 2023-12-07T10:23:36.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.963+0000 7fa91f95eb80 10 client.16737 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:36.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.963+0000 7fa91f95eb80 1 -- 172.21.15.119:0/4066220413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55df3a4f4c60 con 0x55df3a78eb40 2023-12-07T10:23:36.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.963+0000 7fa8ed7fa700 1 -- 172.21.15.119:0/4066220413 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9000267c0 con 0x55df3a78eb40 2023-12-07T10:23:36.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.963+0000 7fa8ed7fa700 1 -- 172.21.15.119:0/4066220413 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fa90002e070 con 0x55df3a78eb40 2023-12-07T10:23:36.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.963+0000 7fa91f95eb80 10 client.16737 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:36.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.963+0000 7fa91f95eb80 10 client.16737 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2023-12-07T10:23:36.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.963+0000 7fa91f95eb80 20 client.16737 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:36.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.963+0000 7fa91f95eb80 1 --2- 172.21.15.119:0/4066220413 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55df3a7ff220 0x55df3a801640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:36.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.963+0000 7fa91f95eb80 4 client.16737 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:23:36.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.963+0000 7fa91f95eb80 1 -- 172.21.15.119:0/4066220413 --> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fa8f40a2860 con 0x55df3a7ff220 2023-12-07T10:23:36.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.964+0000 7fa90959a700 1 --2- 172.21.15.119:0/4066220413 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55df3a7ff220 0x55df3a801640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:36.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.964+0000 7fa8ed7fa700 10 client.16737 ms_handle_connect on v2:172.21.15.119:6838/1920762602 2023-12-07T10:23:36.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.964+0000 7fa90959a700 1 --2- 172.21.15.119:0/4066220413 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55df3a7ff220 0x55df3a801640 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b18e89b743d6b6 server_cookie=9e1e93ea5a8a2f05 in_seq=0 out_seq=0 2023-12-07T10:23:36.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.964+0000 7fa8ed7fa700 1 -- 172.21.15.119:0/4066220413 <== mds.1 v2:172.21.15.119:6838/1920762602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fa8f40a2860 con 0x55df3a7ff220 2023-12-07T10:23:36.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.964+0000 7fa8ed7fa700 10 client.16737 handle_command_reply: tid=0 2023-12-07T10:23:36.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.964+0000 7fa91f95eb80 1 client.16737 shutdown 2023-12-07T10:23:36.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.965+0000 7fa91f95eb80 1 -- 172.21.15.119:0/4066220413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3a78eb40 msgr2=0x55df3a79f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.965+0000 7fa91f95eb80 1 --2- 172.21.15.119:0/4066220413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3a78eb40 0x55df3a79f0c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa900009520 tx=0x7fa900009290 comp rx=0 tx=0).stop 2023-12-07T10:23:36.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.965+0000 7fa91f95eb80 1 -- 172.21.15.119:0/4066220413 shutdown_connections 2023-12-07T10:23:36.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.965+0000 7fa91f95eb80 1 --2- 172.21.15.119:0/4066220413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55df3a793150 0x55df3a79d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.965+0000 7fa91f95eb80 1 --2- 172.21.15.119:0/4066220413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3a78eb40 0x55df3a79f0c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.965+0000 7fa91f95eb80 1 --2- 172.21.15.119:0/4066220413 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55df3a7ff220 0x55df3a801640 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.965+0000 7fa91f95eb80 1 --2- 172.21.15.119:0/4066220413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df3a78f460 0x55df3a79f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.965+0000 7fa91f95eb80 1 -- 172.21.15.119:0/4066220413 >> 172.21.15.119:0/4066220413 conn(0x55df3a6bd360 msgr2=0x55df3a792230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:36.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.965+0000 7fa8ed7fa700 0 client.16737 ms_handle_reset on v2:172.21.15.119:6838/1920762602 2023-12-07T10:23:36.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.965+0000 7fa91f95eb80 1 -- 172.21.15.119:0/4066220413 shutdown_connections 2023-12-07T10:23:36.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.965+0000 7fa91f95eb80 1 -- 172.21.15.119:0/4066220413 wait complete. 2023-12-07T10:23:36.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.965+0000 7fa91f95eb80 20 client.16737 trim_cache size 0 max 16384 2023-12-07T10:23:36.971 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:23:36.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.966+0000 7fa90bfff700 1 -- 172.21.15.119:0/3225182170 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa8f00a1e90 msgr2=0x7fa8f00a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.966+0000 7fa90bfff700 1 --2- 172.21.15.119:0/3225182170 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa8f00a1e90 0x7fa8f00a4350 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00a600 tx=0x7fa8fc00c040 comp rx=0 tx=0).stop 2023-12-07T10:23:36.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.966+0000 7fa90bfff700 1 -- 172.21.15.119:0/3225182170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa904106110 msgr2=0x7fa90406b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:36.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.966+0000 7fa90bfff700 1 --2- 172.21.15.119:0/3225182170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa904106110 0x7fa90406b030 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa8f400d950 tx=0x7fa8f400dc60 comp rx=0 tx=0).stop 2023-12-07T10:23:36.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.966+0000 7fa90bfff700 1 -- 172.21.15.119:0/3225182170 shutdown_connections 2023-12-07T10:23:36.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.966+0000 7fa90bfff700 1 --2- 172.21.15.119:0/3225182170 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa8f00a1e90 0x7fa8f00a4350 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.966+0000 7fa90bfff700 1 --2- 172.21.15.119:0/3225182170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa904106ad0 0x7fa90406b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.966+0000 7fa90bfff700 1 --2- 172.21.15.119:0/3225182170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa904068a50 0x7fa904072f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.966+0000 7fa90bfff700 1 --2- 172.21.15.119:0/3225182170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa904106110 0x7fa90406b030 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:36.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.966+0000 7fa90bfff700 1 -- 172.21.15.119:0/3225182170 >> 172.21.15.119:0/3225182170 conn(0x7fa904077610 msgr2=0x7fa9040fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:36.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.966+0000 7fa90bfff700 1 -- 172.21.15.119:0/3225182170 shutdown_connections 2023-12-07T10:23:36.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:36.966+0000 7fa90bfff700 1 -- 172.21.15.119:0/3225182170 wait complete. 2023-12-07T10:23:36.987 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-07T10:23:37.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.205+0000 7fd7e5b9d700 1 -- 172.21.15.119:0/3311052577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7e0107470 msgr2=0x7fd7e0107850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:37.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.205+0000 7fd7e5b9d700 1 --2- 172.21.15.119:0/3311052577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7e0107470 0x7fd7e0107850 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4009990 tx=0x7fd7d4009ca0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.205+0000 7fd7e5b9d700 1 -- 172.21.15.119:0/3311052577 shutdown_connections 2023-12-07T10:23:37.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.205+0000 7fd7e5b9d700 1 --2- 172.21.15.119:0/3311052577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7e0101d90 0x7fd7e010e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.205+0000 7fd7e5b9d700 1 --2- 172.21.15.119:0/3311052577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7e0107470 0x7fd7e0107850 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.205+0000 7fd7e5b9d700 1 --2- 172.21.15.119:0/3311052577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7e01013d0 0x7fd7e0101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.205+0000 7fd7e5b9d700 1 -- 172.21.15.119:0/3311052577 >> 172.21.15.119:0/3311052577 conn(0x7fd7e00fcee0 msgr2=0x7fd7e00ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:37.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.205+0000 7fd7e5b9d700 1 -- 172.21.15.119:0/3311052577 shutdown_connections 2023-12-07T10:23:37.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.205+0000 7fd7e5b9d700 1 -- 172.21.15.119:0/3311052577 wait complete. 2023-12-07T10:23:37.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.206+0000 7fd7e5b9d700 1 Processor -- start 2023-12-07T10:23:37.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.206+0000 7fd7e5b9d700 1 -- start start 2023-12-07T10:23:37.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.207+0000 7fd7e5b9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7e01013d0 0x7fd7e019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.207+0000 7fd7e5b9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7e0101d90 0x7fd7e019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.207+0000 7fd7e5b9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7e0107470 0x7fd7e0199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.207+0000 7fd7e5b9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e0199ea0 con 0x7fd7e0101d90 2023-12-07T10:23:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.207+0000 7fd7e5b9d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e019a010 con 0x7fd7e01013d0 2023-12-07T10:23:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.207+0000 7fd7e5b9d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e019a1d0 con 0x7fd7e0107470 2023-12-07T10:23:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.207+0000 7fd7e4b9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7e01013d0 0x7fd7e019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:37.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.207+0000 7fd7dffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7e0101d90 0x7fd7e019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:37.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.207+0000 7fd7e4b9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7e01013d0 0x7fd7e019f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:56120/0 (socket says 172.21.15.119:56120) 2023-12-07T10:23:37.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.207+0000 7fd7e4b9b700 1 -- 172.21.15.119:0/4111787445 learned_addr learned my addr 172.21.15.119:0/4111787445 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:37.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.207+0000 7fd7e539c700 1 --2- 172.21.15.119:0/4111787445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7e0107470 0x7fd7e0199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:37.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.207+0000 7fd7e4b9b700 1 -- 172.21.15.119:0/4111787445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7e0107470 msgr2=0x7fd7e0199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:37.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.207+0000 7fd7e4b9b700 1 --2- 172.21.15.119:0/4111787445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7e0107470 0x7fd7e0199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.207+0000 7fd7e4b9b700 1 -- 172.21.15.119:0/4111787445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7e0101d90 msgr2=0x7fd7e019fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:37.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.207+0000 7fd7e4b9b700 1 --2- 172.21.15.119:0/4111787445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7e0101d90 0x7fd7e019fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.207+0000 7fd7e4b9b700 1 -- 172.21.15.119:0/4111787445 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d40095f0 con 0x7fd7e01013d0 2023-12-07T10:23:37.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.207+0000 7fd7dffff700 1 --2- 172.21.15.119:0/4111787445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7e0101d90 0x7fd7e019fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:37.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.207+0000 7fd7e539c700 1 --2- 172.21.15.119:0/4111787445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7e0107470 0x7fd7e0199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:23:37.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.208+0000 7fd7e4b9b700 1 --2- 172.21.15.119:0/4111787445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7e01013d0 0x7fd7e019f700 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4005910 tx=0x7fd7d4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:37.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.208+0000 7fd7ddffb700 1 -- 172.21.15.119:0/4111787445 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d401d070 con 0x7fd7e01013d0 2023-12-07T10:23:37.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.208+0000 7fd7e5b9d700 1 -- 172.21.15.119:0/4111787445 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e019a450 con 0x7fd7e01013d0 2023-12-07T10:23:37.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.208+0000 7fd7ddffb700 1 -- 172.21.15.119:0/4111787445 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7d4022470 con 0x7fd7e01013d0 2023-12-07T10:23:37.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.208+0000 7fd7ddffb700 1 -- 172.21.15.119:0/4111787445 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4017650 con 0x7fd7e01013d0 2023-12-07T10:23:37.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.208+0000 7fd7e5b9d700 1 -- 172.21.15.119:0/4111787445 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e01a66f0 con 0x7fd7e01013d0 2023-12-07T10:23:37.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.211+0000 7fd7ddffb700 1 -- 172.21.15.119:0/4111787445 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fd7d40225e0 con 0x7fd7e01013d0 2023-12-07T10:23:37.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.212+0000 7fd7ddffb700 1 --2- 172.21.15.119:0/4111787445 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd7d00a1c80 0x7fd7d00a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:37.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.213+0000 7fd7fa822b80 1 Processor -- start 2023-12-07T10:23:37.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.213+0000 7fd7ddffb700 1 -- 172.21.15.119:0/4111787445 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fd7d40cf080 con 0x7fd7e01013d0 2023-12-07T10:23:37.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.213+0000 7fd7dffff700 1 --2- 172.21.15.119:0/4111787445 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd7d00a1c80 0x7fd7d00a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:37.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.213+0000 7fd7fa822b80 1 -- start start 2023-12-07T10:23:37.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.213+0000 7fd7fa822b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c398be9e0 0x557c398bedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:37.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.213+0000 7fd7fa822b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c398bf300 0x557c398c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:37.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.213+0000 7fd7fa822b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c398c32e0 0x557c398c3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:37.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.213+0000 7fd7fa822b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c398327c0 con 0x557c398be9e0 2023-12-07T10:23:37.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.213+0000 7fd7fa822b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c398197a0 con 0x557c398bf300 2023-12-07T10:23:37.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.213+0000 7fd7fa822b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c398170c0 con 0x557c398c32e0 2023-12-07T10:23:37.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.213+0000 7fd7dffff700 1 --2- 172.21.15.119:0/4111787445 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd7d00a1c80 0x7fd7d00a4140 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd7e019ac50 tx=0x7fd7cc009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:37.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.214+0000 7fd7dffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c398bf300 0x557c398c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:37.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.214+0000 7fd7dffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c398bf300 0x557c398c2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:56134/0 (socket says 172.21.15.119:56134) 2023-12-07T10:23:37.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.214+0000 7fd7e4b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c398be9e0 0x557c398bedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:37.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.214+0000 7fd7dffff700 1 -- 172.21.15.119:0/4182700442 learned_addr learned my addr 172.21.15.119:0/4182700442 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:37.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.214+0000 7fd7e539c700 1 --2- 172.21.15.119:0/4182700442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c398c32e0 0x557c398c3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:37.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.214+0000 7fd7dffff700 1 -- 172.21.15.119:0/4182700442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c398c32e0 msgr2=0x557c398c3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:37.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.214+0000 7fd7dffff700 1 --2- 172.21.15.119:0/4182700442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c398c32e0 0x557c398c3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.214+0000 7fd7dffff700 1 -- 172.21.15.119:0/4182700442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c398be9e0 msgr2=0x557c398bedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:37.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.214+0000 7fd7dffff700 1 --2- 172.21.15.119:0/4182700442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c398be9e0 0x557c398bedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.214+0000 7fd7dffff700 1 -- 172.21.15.119:0/4182700442 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c3968f630 con 0x557c398bf300 2023-12-07T10:23:37.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.214+0000 7fd7e4b9b700 1 --2- 172.21.15.119:0/4182700442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c398be9e0 0x557c398bedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:37.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.214+0000 7fd7e539c700 1 --2- 172.21.15.119:0/4182700442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c398c32e0 0x557c398c3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:37.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.214+0000 7fd7dffff700 1 --2- 172.21.15.119:0/4182700442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c398bf300 0x557c398c2da0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0043a0 tx=0x7fd7cc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df1162b6608d618e server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:37.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.214+0000 7fd7caffd700 1 -- 172.21.15.119:0/4182700442 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc003d10 con 0x557c398bf300 2023-12-07T10:23:37.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.214+0000 7fd7caffd700 1 -- 172.21.15.119:0/4182700442 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7cc003e70 con 0x557c398bf300 2023-12-07T10:23:37.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.215+0000 7fd7caffd700 1 -- 172.21.15.119:0/4182700442 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc01e770 con 0x557c398bf300 2023-12-07T10:23:37.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.215+0000 7fd7fa822b80 1 -- 172.21.15.119:0/4182700442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c398bf300 msgr2=0x557c398c2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:37.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.215+0000 7fd7fa822b80 1 --2- 172.21.15.119:0/4182700442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c398bf300 0x557c398c2da0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0043a0 tx=0x7fd7cc014000 comp rx=0 tx=0).stop 2023-12-07T10:23:37.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.215+0000 7fd7fa822b80 1 -- 172.21.15.119:0/4182700442 shutdown_connections 2023-12-07T10:23:37.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.215+0000 7fd7fa822b80 1 --2- 172.21.15.119:0/4182700442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c398c32e0 0x557c398c3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.215+0000 7fd7fa822b80 1 --2- 172.21.15.119:0/4182700442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c398be9e0 0x557c398bedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.215+0000 7fd7fa822b80 1 --2- 172.21.15.119:0/4182700442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c398bf300 0x557c398c2da0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.215+0000 7fd7fa822b80 1 -- 172.21.15.119:0/4182700442 >> 172.21.15.119:0/4182700442 conn(0x557c397ed360 msgr2=0x557c398b9170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:37.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.215+0000 7fd7fa822b80 1 -- 172.21.15.119:0/4182700442 shutdown_connections 2023-12-07T10:23:37.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.215+0000 7fd7fa822b80 1 -- 172.21.15.119:0/4182700442 wait complete. 2023-12-07T10:23:37.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.216+0000 7fd7fa822b80 1 -- start start 2023-12-07T10:23:37.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.216+0000 7fd7fa822b80 1 Processor -- start 2023-12-07T10:23:37.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.216+0000 7fd7fa822b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c398be9e0 0x557c398caa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:37.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.216+0000 7fd7fa822b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c398bf300 0x557c398c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:37.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.216+0000 7fd7fa822b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c398c32e0 0x557c398c8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:37.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.217+0000 7fd7fa822b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c398129b0 con 0x557c398bf300 2023-12-07T10:23:37.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.217+0000 7fd7fa822b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c397cc560 con 0x557c398c32e0 2023-12-07T10:23:37.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.217+0000 7fd7fa822b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c397cc200 con 0x557c398be9e0 2023-12-07T10:23:37.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.217+0000 7fd7fa822b80 10 client.0 _ll_register_callbacks cb 0x557c397b3940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:37.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.217+0000 7fd7e4b9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c398be9e0 0x557c398caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:37.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.217+0000 7fd7dffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c398bf300 0x557c398c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:37.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.217+0000 7fd7e4b9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c398be9e0 0x557c398caa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41810/0 (socket says 172.21.15.119:41810) 2023-12-07T10:23:37.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.217+0000 7fd7e4b9b700 1 -- 172.21.15.119:0/70937266 learned_addr learned my addr 172.21.15.119:0/70937266 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:37.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.217+0000 7fd7e539c700 1 --2- 172.21.15.119:0/70937266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c398c32e0 0x557c398c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:37.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.217+0000 7fd7e4b9b700 1 -- 172.21.15.119:0/70937266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c398c32e0 msgr2=0x557c398c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:37.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.217+0000 7fd7e4b9b700 1 --2- 172.21.15.119:0/70937266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c398c32e0 0x557c398c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.217+0000 7fd7e4b9b700 1 -- 172.21.15.119:0/70937266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c398bf300 msgr2=0x557c398c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:37.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.217+0000 7fd7e4b9b700 1 --2- 172.21.15.119:0/70937266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c398bf300 0x557c398c7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.217+0000 7fd7e4b9b700 1 -- 172.21.15.119:0/70937266 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d8009050 con 0x557c398be9e0 2023-12-07T10:23:37.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.217+0000 7fd7e4b9b700 1 --2- 172.21.15.119:0/70937266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c398be9e0 0x557c398caa20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd7d409ba90 tx=0x7fd7d409b640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2616971291d6cd37 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:37.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.217+0000 7fd7bffff700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:23:37.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.218+0000 7fd7bffff700 1 -- 172.21.15.119:0/70937266 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4084ea0 con 0x557c398be9e0 2023-12-07T10:23:37.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.218+0000 7fd7bffff700 1 -- 172.21.15.119:0/70937266 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fd7d409c200 con 0x557c398be9e0 2023-12-07T10:23:37.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.218+0000 7fd7fa822b80 1 -- 172.21.15.119:0/70937266 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c398b19a0 con 0x557c398be9e0 2023-12-07T10:23:37.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.218+0000 7fd7bffff700 1 -- 172.21.15.119:0/70937266 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7d409bb00 con 0x557c398be9e0 2023-12-07T10:23:37.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.218+0000 7fd7fa822b80 10 client.26752 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:37.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.218+0000 7fd7fa822b80 1 -- 172.21.15.119:0/70937266 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7fd7cc017070 con 0x557c398be9e0 2023-12-07T10:23:37.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.218+0000 7fd7bffff700 1 -- 172.21.15.119:0/70937266 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557c398b19a0 con 0x557c398be9e0 2023-12-07T10:23:37.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.218+0000 7fd7bffff700 1 -- 172.21.15.119:0/70937266 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fd7d40a3970 con 0x557c398be9e0 2023-12-07T10:23:37.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.218+0000 7fd7fa822b80 10 client.26752 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:37.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.218+0000 7fd7fa822b80 10 client.26752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:23:37.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.218+0000 7fd7fa822b80 20 client.26752 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:37.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.218+0000 7fd7fa822b80 1 --2- 172.21.15.119:0/70937266 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x557c398d8290 0x557c398da6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:37.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.218+0000 7fd7fa822b80 4 client.26752 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:23:37.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.218+0000 7fd7fa822b80 1 -- 172.21.15.119:0/70937266 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c3988d7a0 con 0x557c398d8290 2023-12-07T10:23:37.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.219+0000 7fd7dffff700 1 --2- 172.21.15.119:0/70937266 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x557c398d8290 0x557c398da6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:37.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.219+0000 7fd7bffff700 10 client.26752 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:37.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.219+0000 7fd7dffff700 1 --2- 172.21.15.119:0/70937266 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x557c398d8290 0x557c398da6b0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96e58d2795986a3e server_cookie=daf3fcfd25004adc in_seq=0 out_seq=0 2023-12-07T10:23:37.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.220+0000 7fd7bffff700 1 -- 172.21.15.119:0/70937266 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557c3988d7a0 con 0x557c398d8290 2023-12-07T10:23:37.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.220+0000 7fd7bffff700 10 client.26752 handle_command_reply: tid=0 2023-12-07T10:23:37.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.220+0000 7fd7fa822b80 1 client.26752 shutdown 2023-12-07T10:23:37.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.220+0000 7fd7fa822b80 1 -- 172.21.15.119:0/70937266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c398be9e0 msgr2=0x557c398caa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:37.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.220+0000 7fd7fa822b80 1 --2- 172.21.15.119:0/70937266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c398be9e0 0x557c398caa20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd7d409ba90 tx=0x7fd7d409b640 comp rx=0 tx=0).stop 2023-12-07T10:23:37.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.220+0000 7fd7fa822b80 1 -- 172.21.15.119:0/70937266 shutdown_connections 2023-12-07T10:23:37.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.220+0000 7fd7fa822b80 1 --2- 172.21.15.119:0/70937266 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x557c398d8290 0x557c398da6b0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.220+0000 7fd7fa822b80 1 --2- 172.21.15.119:0/70937266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c398c32e0 0x557c398c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.220+0000 7fd7fa822b80 1 --2- 172.21.15.119:0/70937266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c398be9e0 0x557c398caa20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.220+0000 7fd7fa822b80 1 --2- 172.21.15.119:0/70937266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c398bf300 0x557c398c7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.220+0000 7fd7bffff700 0 client.26752 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:37.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.220+0000 7fd7fa822b80 1 -- 172.21.15.119:0/70937266 >> 172.21.15.119:0/70937266 conn(0x557c397ed360 msgr2=0x557c398bd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:37.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.221+0000 7fd7fa822b80 1 -- 172.21.15.119:0/70937266 shutdown_connections 2023-12-07T10:23:37.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.221+0000 7fd7fa822b80 1 -- 172.21.15.119:0/70937266 wait complete. 2023-12-07T10:23:37.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.221+0000 7fd7fa822b80 20 client.26752 trim_cache size 0 max 16384 2023-12-07T10:23:37.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.235+0000 7fd7fa822b80 1 Processor -- start 2023-12-07T10:23:37.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.236+0000 7fd7fa822b80 1 -- start start 2023-12-07T10:23:37.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.236+0000 7fd7fa822b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c398be9e0 0x557c398ca750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:37.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.236+0000 7fd7fa822b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c398bf300 0x557c398cf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:37.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.236+0000 7fd7fa822b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c398c32e0 0x557c398cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:37.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.236+0000 7fd7fa822b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c39691920 con 0x557c398bf300 2023-12-07T10:23:37.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.236+0000 7fd7fa822b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c398cac90 con 0x557c398be9e0 2023-12-07T10:23:37.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.236+0000 7fd7fa822b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c398bdba0 con 0x557c398c32e0 2023-12-07T10:23:37.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.236+0000 7fd7e4b9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c398be9e0 0x557c398ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:37.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.236+0000 7fd7e4b9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c398be9e0 0x557c398ca750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:56152/0 (socket says 172.21.15.119:56152) 2023-12-07T10:23:37.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.236+0000 7fd7e4b9b700 1 -- 172.21.15.119:0/1999558279 learned_addr learned my addr 172.21.15.119:0/1999558279 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:37.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.236+0000 7fd7dffff700 1 --2- 172.21.15.119:0/1999558279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c398bf300 0x557c398cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:37.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.236+0000 7fd7e539c700 1 --2- 172.21.15.119:0/1999558279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c398c32e0 0x557c398cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:37.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.236+0000 7fd7e4b9b700 1 -- 172.21.15.119:0/1999558279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c398c32e0 msgr2=0x557c398cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:37.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.237+0000 7fd7e4b9b700 1 --2- 172.21.15.119:0/1999558279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c398c32e0 0x557c398cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.237+0000 7fd7e4b9b700 1 -- 172.21.15.119:0/1999558279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c398bf300 msgr2=0x557c398cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:37.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.237+0000 7fd7e4b9b700 1 --2- 172.21.15.119:0/1999558279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c398bf300 0x557c398cf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.237+0000 7fd7e4b9b700 1 -- 172.21.15.119:0/1999558279 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c3963bde0 con 0x557c398be9e0 2023-12-07T10:23:37.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.237+0000 7fd7e4b9b700 1 --2- 172.21.15.119:0/1999558279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c398be9e0 0x557c398ca750 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4099e50 tx=0x7fd7d4022c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3189d69ddae8a22 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:37.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.237+0000 7fd7cb7fe700 1 -- 172.21.15.119:0/1999558279 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4084ea0 con 0x557c398be9e0 2023-12-07T10:23:37.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.237+0000 7fd7cb7fe700 1 -- 172.21.15.119:0/1999558279 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7d4097e40 con 0x557c398be9e0 2023-12-07T10:23:37.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.237+0000 7fd7cb7fe700 1 -- 172.21.15.119:0/1999558279 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d40a3e00 con 0x557c398be9e0 2023-12-07T10:23:37.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.238+0000 7fd7fa822b80 1 -- 172.21.15.119:0/1999558279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c398be9e0 msgr2=0x557c398ca750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:37.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.238+0000 7fd7fa822b80 1 --2- 172.21.15.119:0/1999558279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c398be9e0 0x557c398ca750 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4099e50 tx=0x7fd7d4022c90 comp rx=0 tx=0).stop 2023-12-07T10:23:37.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.238+0000 7fd7fa822b80 1 -- 172.21.15.119:0/1999558279 shutdown_connections 2023-12-07T10:23:37.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.238+0000 7fd7fa822b80 1 --2- 172.21.15.119:0/1999558279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c398c32e0 0x557c398cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.238+0000 7fd7fa822b80 1 --2- 172.21.15.119:0/1999558279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c398be9e0 0x557c398ca750 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.238+0000 7fd7fa822b80 1 --2- 172.21.15.119:0/1999558279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c398bf300 0x557c398cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.238+0000 7fd7fa822b80 1 -- 172.21.15.119:0/1999558279 >> 172.21.15.119:0/1999558279 conn(0x557c397ed360 msgr2=0x557c398b9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:37.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.238+0000 7fd7fa822b80 1 -- 172.21.15.119:0/1999558279 shutdown_connections 2023-12-07T10:23:37.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.238+0000 7fd7fa822b80 1 -- 172.21.15.119:0/1999558279 wait complete. 2023-12-07T10:23:37.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.239+0000 7fd7fa822b80 1 -- start start 2023-12-07T10:23:37.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.239+0000 7fd7fa822b80 1 Processor -- start 2023-12-07T10:23:37.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.239+0000 7fd7fa822b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c398be9e0 0x557c398cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:37.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.239+0000 7fd7fa822b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c398bf300 0x557c398cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:37.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.239+0000 7fd7fa822b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c398c32e0 0x557c398cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:37.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.239+0000 7fd7fa822b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c398ce360 con 0x557c398bf300 2023-12-07T10:23:37.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.239+0000 7fd7fa822b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c3992d750 con 0x557c398be9e0 2023-12-07T10:23:37.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.239+0000 7fd7fa822b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c3992d890 con 0x557c398c32e0 2023-12-07T10:23:37.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.240+0000 7fd7fa822b80 10 client.0 _ll_register_callbacks cb 0x557c397b3940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:37.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.240+0000 7fd7e539c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c398c32e0 0x557c398cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:37.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.240+0000 7fd7e539c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c398c32e0 0x557c398cdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41830/0 (socket says 172.21.15.119:41830) 2023-12-07T10:23:37.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.240+0000 7fd7e539c700 1 -- 172.21.15.119:0/4055995704 learned_addr learned my addr 172.21.15.119:0/4055995704 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:37.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.240+0000 7fd7dffff700 1 --2- 172.21.15.119:0/4055995704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c398bf300 0x557c398cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:37.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.240+0000 7fd7e4b9b700 1 --2- 172.21.15.119:0/4055995704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c398be9e0 0x557c398cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:37.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.240+0000 7fd7e539c700 1 -- 172.21.15.119:0/4055995704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c398be9e0 msgr2=0x557c398cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:37.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.240+0000 7fd7e539c700 1 --2- 172.21.15.119:0/4055995704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c398be9e0 0x557c398cfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.240+0000 7fd7e539c700 1 -- 172.21.15.119:0/4055995704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c398bf300 msgr2=0x557c398cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.240+0000 7fd7e539c700 1 --2- 172.21.15.119:0/4055995704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c398bf300 0x557c398cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.240+0000 7fd7e539c700 1 -- 172.21.15.119:0/4055995704 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7cc017070 con 0x557c398c32e0 2023-12-07T10:23:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.241+0000 7fd7e539c700 1 --2- 172.21.15.119:0/4055995704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c398c32e0 0x557c398cdc80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8007f70 tx=0x7fd7d8009370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a5e8a55801e19ea server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.241+0000 7fd7bffff700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:23:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.241+0000 7fd7bffff700 1 -- 172.21.15.119:0/4055995704 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d800e040 con 0x557c398c32e0 2023-12-07T10:23:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.241+0000 7fd7fa822b80 1 -- 172.21.15.119:0/4055995704 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7cc003e70 con 0x557c398c32e0 2023-12-07T10:23:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.241+0000 7fd7bffff700 1 -- 172.21.15.119:0/4055995704 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fd7d801e070 con 0x557c398c32e0 2023-12-07T10:23:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.241+0000 7fd7bffff700 1 -- 172.21.15.119:0/4055995704 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7d8017c90 con 0x557c398c32e0 2023-12-07T10:23:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.241+0000 7fd7fa822b80 10 client.26758 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.241+0000 7fd7fa822b80 1 -- 172.21.15.119:0/4055995704 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x557c39624c60 con 0x557c398c32e0 2023-12-07T10:23:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.241+0000 7fd7bffff700 1 -- 172.21.15.119:0/4055995704 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7cc003e70 con 0x557c398c32e0 2023-12-07T10:23:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.241+0000 7fd7bffff700 1 -- 172.21.15.119:0/4055995704 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fd7d80046e0 con 0x557c398c32e0 2023-12-07T10:23:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.241+0000 7fd7fa822b80 10 client.26758 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.241+0000 7fd7fa822b80 10 client.26758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:23:37.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.241+0000 7fd7fa822b80 20 client.26758 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:37.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.241+0000 7fd7fa822b80 1 --2- 172.21.15.119:0/4055995704 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x557c3992f210 0x557c39931630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:37.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.241+0000 7fd7fa822b80 4 client.26758 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:23:37.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.241+0000 7fd7fa822b80 1 -- 172.21.15.119:0/4055995704 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557c397d8fd0 con 0x557c3992f210 2023-12-07T10:23:37.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.242+0000 7fd7e4b9b700 1 --2- 172.21.15.119:0/4055995704 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x557c3992f210 0x557c39931630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:37.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.242+0000 7fd7bffff700 10 client.26758 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:37.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.242+0000 7fd7e4b9b700 1 --2- 172.21.15.119:0/4055995704 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x557c3992f210 0x557c39931630 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac082909675e76e6 server_cookie=f909d35904017450 in_seq=0 out_seq=0 2023-12-07T10:23:37.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.242+0000 7fd7bffff700 1 -- 172.21.15.119:0/4055995704 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557c397d8fd0 con 0x557c3992f210 2023-12-07T10:23:37.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.242+0000 7fd7bffff700 10 client.26758 handle_command_reply: tid=0 2023-12-07T10:23:37.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.242+0000 7fd7fa822b80 1 client.26758 shutdown 2023-12-07T10:23:37.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.243+0000 7fd7fa822b80 1 -- 172.21.15.119:0/4055995704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c398c32e0 msgr2=0x557c398cdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:37.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.243+0000 7fd7fa822b80 1 --2- 172.21.15.119:0/4055995704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c398c32e0 0x557c398cdc80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8007f70 tx=0x7fd7d8009370 comp rx=0 tx=0).stop 2023-12-07T10:23:37.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.243+0000 7fd7fa822b80 1 -- 172.21.15.119:0/4055995704 shutdown_connections 2023-12-07T10:23:37.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.243+0000 7fd7fa822b80 1 --2- 172.21.15.119:0/4055995704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c398c32e0 0x557c398cdc80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.243+0000 7fd7fa822b80 1 --2- 172.21.15.119:0/4055995704 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x557c3992f210 0x557c39931630 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.243+0000 7fd7fa822b80 1 --2- 172.21.15.119:0/4055995704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557c398be9e0 0x557c398cfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.243+0000 7fd7fa822b80 1 --2- 172.21.15.119:0/4055995704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c398bf300 0x557c398cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.243+0000 7fd7fa822b80 1 -- 172.21.15.119:0/4055995704 >> 172.21.15.119:0/4055995704 conn(0x557c397ed360 msgr2=0x557c398ba9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:37.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.243+0000 7fd7bffff700 0 client.26758 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:37.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.243+0000 7fd7fa822b80 1 -- 172.21.15.119:0/4055995704 shutdown_connections 2023-12-07T10:23:37.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.243+0000 7fd7fa822b80 1 -- 172.21.15.119:0/4055995704 wait complete. 2023-12-07T10:23:37.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.243+0000 7fd7fa822b80 20 client.26758 trim_cache size 0 max 16384 2023-12-07T10:23:37.268 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:23:37.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.244+0000 7fd7e5b9d700 1 -- 172.21.15.119:0/4111787445 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd7d00a1c80 msgr2=0x7fd7d00a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:37.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.244+0000 7fd7e5b9d700 1 --2- 172.21.15.119:0/4111787445 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd7d00a1c80 0x7fd7d00a4140 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd7e019ac50 tx=0x7fd7cc009260 comp rx=0 tx=0).stop 2023-12-07T10:23:37.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.244+0000 7fd7e5b9d700 1 -- 172.21.15.119:0/4111787445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7e01013d0 msgr2=0x7fd7e019f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:37.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.244+0000 7fd7e5b9d700 1 --2- 172.21.15.119:0/4111787445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7e01013d0 0x7fd7e019f700 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4005910 tx=0x7fd7d4004730 comp rx=0 tx=0).stop 2023-12-07T10:23:37.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.244+0000 7fd7e5b9d700 1 -- 172.21.15.119:0/4111787445 shutdown_connections 2023-12-07T10:23:37.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.244+0000 7fd7e5b9d700 1 --2- 172.21.15.119:0/4111787445 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd7d00a1c80 0x7fd7d00a4140 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.244+0000 7fd7e5b9d700 1 --2- 172.21.15.119:0/4111787445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7e0107470 0x7fd7e0199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.244+0000 7fd7e5b9d700 1 --2- 172.21.15.119:0/4111787445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7e01013d0 0x7fd7e019f700 secure :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4005910 tx=0x7fd7d4004730 comp rx=0 tx=0).stop 2023-12-07T10:23:37.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.244+0000 7fd7e5b9d700 1 --2- 172.21.15.119:0/4111787445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7e0101d90 0x7fd7e019fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:37.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.244+0000 7fd7e5b9d700 1 -- 172.21.15.119:0/4111787445 >> 172.21.15.119:0/4111787445 conn(0x7fd7e00fcee0 msgr2=0x7fd7e010ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:37.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.245+0000 7fd7e5b9d700 1 -- 172.21.15.119:0/4111787445 shutdown_connections 2023-12-07T10:23:37.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:37.245+0000 7fd7e5b9d700 1 -- 172.21.15.119:0/4111787445 wait complete. 2023-12-07T10:23:37.549 INFO:tasks.workunit.client.0.smithi119.stdout: 1 176240 8.77 MB/sec warmup 110 sec latency 16.763 ms 2023-12-07T10:23:38.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:37 smithi159 ceph-mon[134726]: pgmap v99: 129 pgs: 129 active+clean; 108 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 347 op/s 2023-12-07T10:23:38.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:37 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:23:38.272 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-07T10:23:38.272 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-07T10:23:38.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:37 smithi153 ceph-mon[133625]: pgmap v99: 129 pgs: 129 active+clean; 108 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 347 op/s 2023-12-07T10:23:38.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:37 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:23:38.343 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:37 smithi119 ceph-mon[97908]: pgmap v99: 129 pgs: 129 active+clean; 108 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 347 op/s 2023-12-07T10:23:38.344 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:37 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:23:38.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.509+0000 7f335359e700 1 -- 172.21.15.119:0/3486866760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3354107480 msgr2=0x7f3354107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:38.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.509+0000 7f335359e700 1 --2- 172.21.15.119:0/3486866760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3354107480 0x7f3354107860 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3344009990 tx=0x7f3344009ca0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.509+0000 7f335359e700 1 -- 172.21.15.119:0/3486866760 shutdown_connections 2023-12-07T10:23:38.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.509+0000 7f335359e700 1 --2- 172.21.15.119:0/3486866760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3354101da0 0x7f335410e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.509+0000 7f335359e700 1 --2- 172.21.15.119:0/3486866760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3354107480 0x7f3354107860 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.509+0000 7f335359e700 1 --2- 172.21.15.119:0/3486866760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33541013e0 0x7f3354101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.509+0000 7f335359e700 1 -- 172.21.15.119:0/3486866760 >> 172.21.15.119:0/3486866760 conn(0x7f33540fcf10 msgr2=0x7f33540ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:38.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.509+0000 7f335359e700 1 -- 172.21.15.119:0/3486866760 shutdown_connections 2023-12-07T10:23:38.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.509+0000 7f335359e700 1 -- 172.21.15.119:0/3486866760 wait complete. 2023-12-07T10:23:38.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.510+0000 7f335359e700 1 Processor -- start 2023-12-07T10:23:38.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.511+0000 7f335359e700 1 -- start start 2023-12-07T10:23:38.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.511+0000 7f335359e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33541013e0 0x7f3354108310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.511+0000 7f335359e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3354101da0 0x7f3354108850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.511+0000 7f335359e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3354107480 0x7f3354108d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.511+0000 7f335359e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33541aa070 con 0x7f33541013e0 2023-12-07T10:23:38.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.511+0000 7f335359e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33541aa1b0 con 0x7f3354107480 2023-12-07T10:23:38.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.511+0000 7f335359e700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33541aa320 con 0x7f3354101da0 2023-12-07T10:23:38.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.512+0000 7f3351d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3354101da0 0x7f3354108850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.512+0000 7f3352d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3354107480 0x7f3354108d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.512+0000 7f3351d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3354101da0 0x7f3354108850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41844/0 (socket says 172.21.15.119:41844) 2023-12-07T10:23:38.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.512+0000 7f3351d9b700 1 -- 172.21.15.119:0/1757612987 learned_addr learned my addr 172.21.15.119:0/1757612987 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:38.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.512+0000 7f335259c700 1 --2- 172.21.15.119:0/1757612987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33541013e0 0x7f3354108310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.512+0000 7f3351d9b700 1 -- 172.21.15.119:0/1757612987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3354107480 msgr2=0x7f3354108d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:38.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.512+0000 7f3351d9b700 1 --2- 172.21.15.119:0/1757612987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3354107480 0x7f3354108d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.512+0000 7f3351d9b700 1 -- 172.21.15.119:0/1757612987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33541013e0 msgr2=0x7f3354108310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:38.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.512+0000 7f3351d9b700 1 --2- 172.21.15.119:0/1757612987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33541013e0 0x7f3354108310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.512+0000 7f3351d9b700 1 -- 172.21.15.119:0/1757612987 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33440095f0 con 0x7f3354101da0 2023-12-07T10:23:38.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.512+0000 7f3352d9d700 1 --2- 172.21.15.119:0/1757612987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3354107480 0x7f3354108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:38.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.512+0000 7f335259c700 1 --2- 172.21.15.119:0/1757612987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33541013e0 0x7f3354108310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:23:38.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.512+0000 7f3351d9b700 1 --2- 172.21.15.119:0/1757612987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3354101da0 0x7f3354108850 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f334800e9a0 tx=0x7f334800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:38.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.513+0000 7f33437fe700 1 -- 172.21.15.119:0/1757612987 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3348009790 con 0x7f3354101da0 2023-12-07T10:23:38.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.513+0000 7f33437fe700 1 -- 172.21.15.119:0/1757612987 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f334800cbd0 con 0x7f3354101da0 2023-12-07T10:23:38.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.513+0000 7f33437fe700 1 -- 172.21.15.119:0/1757612987 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3348018680 con 0x7f3354101da0 2023-12-07T10:23:38.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.513+0000 7f335359e700 1 -- 172.21.15.119:0/1757612987 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33541aa600 con 0x7f3354101da0 2023-12-07T10:23:38.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.513+0000 7f335359e700 1 -- 172.21.15.119:0/1757612987 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33541aab10 con 0x7f3354101da0 2023-12-07T10:23:38.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.516+0000 7f33437fe700 1 -- 172.21.15.119:0/1757612987 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f334800cd40 con 0x7f3354101da0 2023-12-07T10:23:38.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.517+0000 7f336c265b80 1 Processor -- start 2023-12-07T10:23:38.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.517+0000 7f336c265b80 1 -- start start 2023-12-07T10:23:38.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.518+0000 7f33437fe700 1 --2- 172.21.15.119:0/1757612987 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f333c0a1d00 0x7f333c0a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.518+0000 7f336c265b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3244cda60 0x55d3244cde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.518+0000 7f336c265b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3244ce380 0x55d3244d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.518+0000 7f33437fe700 1 -- 172.21.15.119:0/1757612987 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f3348014070 con 0x7f3354101da0 2023-12-07T10:23:38.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.518+0000 7f336c265b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3244d2360 0x55d3244d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.518+0000 7f336c265b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d324441840 con 0x55d3244ce380 2023-12-07T10:23:38.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.518+0000 7f336c265b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d324428820 con 0x55d3244d2360 2023-12-07T10:23:38.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.518+0000 7f336c265b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d324426140 con 0x55d3244cda60 2023-12-07T10:23:38.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.518+0000 7f3352d9d700 1 --2- 172.21.15.119:0/1757612987 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f333c0a1d00 0x7f333c0a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.518+0000 7f335259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3244cda60 0x55d3244cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.518+0000 7f335259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3244cda60 0x55d3244cde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41848/0 (socket says 172.21.15.119:41848) 2023-12-07T10:23:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.518+0000 7f335259c700 1 -- 172.21.15.119:0/1012278181 learned_addr learned my addr 172.21.15.119:0/1012278181 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.518+0000 7f3352d9d700 1 --2- 172.21.15.119:0/1012278181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3244d2360 0x55d3244d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.519+0000 7f3351d9b700 1 --2- 172.21.15.119:0/1012278181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3244ce380 0x55d3244d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.519+0000 7f335259c700 1 -- 172.21.15.119:0/1012278181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3244d2360 msgr2=0x55d3244d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.519+0000 7f335259c700 1 --2- 172.21.15.119:0/1012278181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3244d2360 0x55d3244d27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.519+0000 7f335259c700 1 -- 172.21.15.119:0/1012278181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3244ce380 msgr2=0x55d3244d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.519+0000 7f335259c700 1 --2- 172.21.15.119:0/1012278181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3244ce380 0x55d3244d1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.519+0000 7f335259c700 1 -- 172.21.15.119:0/1012278181 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d32429e6b0 con 0x55d3244cda60 2023-12-07T10:23:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.519+0000 7f335259c700 1 --2- 172.21.15.119:0/1012278181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3244cda60 0x55d3244cde40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f334400b2e0 tx=0x7f3344005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb9d07c2b6602711 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.519+0000 7f3352d9d700 1 --2- 172.21.15.119:0/1757612987 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f333c0a1d00 0x7f333c0a41c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f334c009ec0 tx=0x7f334c003cf0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.519+0000 7f3340ff9700 1 -- 172.21.15.119:0/1012278181 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344004a10 con 0x55d3244cda60 2023-12-07T10:23:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.519+0000 7f3340ff9700 1 -- 172.21.15.119:0/1012278181 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f334401e040 con 0x55d3244cda60 2023-12-07T10:23:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.519+0000 7f3340ff9700 1 -- 172.21.15.119:0/1012278181 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334400b9b0 con 0x55d3244cda60 2023-12-07T10:23:38.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.520+0000 7f336c265b80 1 -- 172.21.15.119:0/1012278181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3244cda60 msgr2=0x55d3244cde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:38.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.520+0000 7f336c265b80 1 --2- 172.21.15.119:0/1012278181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3244cda60 0x55d3244cde40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f334400b2e0 tx=0x7f3344005c00 comp rx=0 tx=0).stop 2023-12-07T10:23:38.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.520+0000 7f336c265b80 1 -- 172.21.15.119:0/1012278181 shutdown_connections 2023-12-07T10:23:38.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.520+0000 7f336c265b80 1 --2- 172.21.15.119:0/1012278181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3244d2360 0x55d3244d27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.520+0000 7f336c265b80 1 --2- 172.21.15.119:0/1012278181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3244cda60 0x55d3244cde40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.520+0000 7f336c265b80 1 --2- 172.21.15.119:0/1012278181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3244ce380 0x55d3244d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.520+0000 7f336c265b80 1 -- 172.21.15.119:0/1012278181 >> 172.21.15.119:0/1012278181 conn(0x55d3243fc3e0 msgr2=0x55d3244c81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:38.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.520+0000 7f336c265b80 1 -- 172.21.15.119:0/1012278181 shutdown_connections 2023-12-07T10:23:38.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.520+0000 7f336c265b80 1 -- 172.21.15.119:0/1012278181 wait complete. 2023-12-07T10:23:38.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.521+0000 7f336c265b80 1 -- start start 2023-12-07T10:23:38.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.521+0000 7f336c265b80 1 Processor -- start 2023-12-07T10:23:38.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.522+0000 7f336c265b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3244cda60 0x55d3244d9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.522+0000 7f336c265b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3244ce380 0x55d3244d6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.522+0000 7f336c265b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3244d2360 0x55d3244d7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.522+0000 7f336c265b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d324421a30 con 0x55d3244ce380 2023-12-07T10:23:38.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.522+0000 7f336c265b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3243db5e0 con 0x55d3244cda60 2023-12-07T10:23:38.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.522+0000 7f336c265b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3243db280 con 0x55d3244d2360 2023-12-07T10:23:38.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.522+0000 7f336c265b80 10 client.0 _ll_register_callbacks cb 0x55d3243c29c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:38.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.522+0000 7f335259c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3244cda60 0x55d3244d9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.522+0000 7f335259c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3244cda60 0x55d3244d9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:56212/0 (socket says 172.21.15.119:56212) 2023-12-07T10:23:38.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.522+0000 7f335259c700 1 -- 172.21.15.119:0/1179943357 learned_addr learned my addr 172.21.15.119:0/1179943357 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:38.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.522+0000 7f3352d9d700 1 --2- 172.21.15.119:0/1179943357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3244d2360 0x55d3244d7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.522+0000 7f3351d9b700 1 --2- 172.21.15.119:0/1179943357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3244ce380 0x55d3244d6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.522+0000 7f335259c700 1 -- 172.21.15.119:0/1179943357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3244d2360 msgr2=0x55d3244d7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:38.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.522+0000 7f335259c700 1 --2- 172.21.15.119:0/1179943357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3244d2360 0x55d3244d7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.522+0000 7f335259c700 1 -- 172.21.15.119:0/1179943357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3244ce380 msgr2=0x55d3244d6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:38.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.522+0000 7f335259c700 1 --2- 172.21.15.119:0/1179943357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3244ce380 0x55d3244d6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.522+0000 7f335259c700 1 -- 172.21.15.119:0/1179943357 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f335410ebb0 con 0x55d3244cda60 2023-12-07T10:23:38.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.523+0000 7f3352d9d700 1 --2- 172.21.15.119:0/1179943357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3244d2360 0x55d3244d7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:38.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.523+0000 7f335259c700 1 --2- 172.21.15.119:0/1179943357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3244cda60 0x55d3244d9b70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3344000c00 tx=0x7f3344005290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=789037ee2650fa5b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:38.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.523+0000 7f3339ffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:23:38.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.523+0000 7f3339ffb700 1 -- 172.21.15.119:0/1179943357 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344031070 con 0x55d3244cda60 2023-12-07T10:23:38.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.523+0000 7f336c265b80 1 -- 172.21.15.119:0/1179943357 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3244c0a20 con 0x55d3244cda60 2023-12-07T10:23:38.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.523+0000 7f3339ffb700 1 -- 172.21.15.119:0/1179943357 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f334400b750 con 0x55d3244cda60 2023-12-07T10:23:38.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.523+0000 7f3339ffb700 1 -- 172.21.15.119:0/1179943357 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33440056d0 con 0x55d3244cda60 2023-12-07T10:23:38.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.523+0000 7f336c265b80 10 client.26792 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:38.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.523+0000 7f336c265b80 1 -- 172.21.15.119:0/1179943357 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55d32424ae60 con 0x55d3244cda60 2023-12-07T10:23:38.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.524+0000 7f3339ffb700 1 -- 172.21.15.119:0/1179943357 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d3244c0a20 con 0x55d3244cda60 2023-12-07T10:23:38.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.524+0000 7f3339ffb700 1 -- 172.21.15.119:0/1179943357 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f3344034030 con 0x55d3244cda60 2023-12-07T10:23:38.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.524+0000 7f336c265b80 10 client.26792 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:38.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.524+0000 7f336c265b80 10 client.26792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:23:38.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.524+0000 7f336c265b80 20 client.26792 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:38.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.524+0000 7f336c265b80 1 --2- 172.21.15.119:0/1179943357 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d3244e71a0 0x55d3244e95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.524+0000 7f336c265b80 4 client.26792 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:23:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.524+0000 7f336c265b80 1 -- 172.21.15.119:0/1179943357 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d32449c820 con 0x55d3244e71a0 2023-12-07T10:23:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.524+0000 7f3351d9b700 1 --2- 172.21.15.119:0/1179943357 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d3244e71a0 0x55d3244e95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.525+0000 7f3339ffb700 10 client.26792 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.525+0000 7f3351d9b700 1 --2- 172.21.15.119:0/1179943357 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d3244e71a0 0x55d3244e95c0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=134b9fef46b41f2d server_cookie=b811e3f4dcd50dd2 in_seq=0 out_seq=0 2023-12-07T10:23:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.526+0000 7f3339ffb700 1 -- 172.21.15.119:0/1179943357 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d32449c820 con 0x55d3244e71a0 2023-12-07T10:23:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.526+0000 7f3339ffb700 10 client.26792 handle_command_reply: tid=0 2023-12-07T10:23:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.526+0000 7f336c265b80 1 client.26792 shutdown 2023-12-07T10:23:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.526+0000 7f336c265b80 1 -- 172.21.15.119:0/1179943357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3244cda60 msgr2=0x55d3244d9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.526+0000 7f336c265b80 1 --2- 172.21.15.119:0/1179943357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3244cda60 0x55d3244d9b70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3344000c00 tx=0x7f3344005290 comp rx=0 tx=0).stop 2023-12-07T10:23:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.526+0000 7f336c265b80 1 -- 172.21.15.119:0/1179943357 shutdown_connections 2023-12-07T10:23:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.526+0000 7f336c265b80 1 --2- 172.21.15.119:0/1179943357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3244d2360 0x55d3244d7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.526+0000 7f336c265b80 1 --2- 172.21.15.119:0/1179943357 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d3244e71a0 0x55d3244e95c0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.526+0000 7f336c265b80 1 --2- 172.21.15.119:0/1179943357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3244cda60 0x55d3244d9b70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.526+0000 7f336c265b80 1 --2- 172.21.15.119:0/1179943357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3244ce380 0x55d3244d6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.526+0000 7f3339ffb700 0 client.26792 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:38.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.526+0000 7f336c265b80 1 -- 172.21.15.119:0/1179943357 >> 172.21.15.119:0/1179943357 conn(0x55d3243fc3e0 msgr2=0x55d3244cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:38.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.526+0000 7f336c265b80 1 -- 172.21.15.119:0/1179943357 shutdown_connections 2023-12-07T10:23:38.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.526+0000 7f336c265b80 1 -- 172.21.15.119:0/1179943357 wait complete. 2023-12-07T10:23:38.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.526+0000 7f336c265b80 20 client.26792 trim_cache size 0 max 16384 2023-12-07T10:23:38.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.546+0000 7f336c265b80 1 Processor -- start 2023-12-07T10:23:38.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.546+0000 7f336c265b80 1 -- start start 2023-12-07T10:23:38.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.546+0000 7f336c265b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3244cda60 0x55d3244d9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.546+0000 7f336c265b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3244ce380 0x55d3244edf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.546+0000 7f336c265b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3244d2360 0x55d3244de5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.546+0000 7f336c265b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3242a09a0 con 0x55d3244d2360 2023-12-07T10:23:38.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.546+0000 7f336c265b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3244d7ac0 con 0x55d3244ce380 2023-12-07T10:23:38.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.547+0000 7f336c265b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3244c9e40 con 0x55d3244cda60 2023-12-07T10:23:38.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.547+0000 7f335259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3244cda60 0x55d3244d9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.547+0000 7f3351d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3244ce380 0x55d3244edf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.547+0000 7f335259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3244cda60 0x55d3244d9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41878/0 (socket says 172.21.15.119:41878) 2023-12-07T10:23:38.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.547+0000 7f335259c700 1 -- 172.21.15.119:0/1417011725 learned_addr learned my addr 172.21.15.119:0/1417011725 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:38.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.547+0000 7f3352d9d700 1 --2- 172.21.15.119:0/1417011725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3244d2360 0x55d3244de5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.547+0000 7f3351d9b700 1 -- 172.21.15.119:0/1417011725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3244cda60 msgr2=0x55d3244d9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:38.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.547+0000 7f3351d9b700 1 --2- 172.21.15.119:0/1417011725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3244cda60 0x55d3244d9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.547+0000 7f3351d9b700 1 -- 172.21.15.119:0/1417011725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3244d2360 msgr2=0x55d3244de5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:38.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.547+0000 7f3351d9b700 1 --2- 172.21.15.119:0/1417011725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3244d2360 0x55d3244de5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.547+0000 7f3351d9b700 1 -- 172.21.15.119:0/1417011725 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3244ca0e0 con 0x55d3244ce380 2023-12-07T10:23:38.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.547+0000 7f335259c700 1 --2- 172.21.15.119:0/1417011725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3244cda60 0x55d3244d9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:38.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.547+0000 7f3352d9d700 1 --2- 172.21.15.119:0/1417011725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3244d2360 0x55d3244de5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:23:38.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.547+0000 7f3351d9b700 1 --2- 172.21.15.119:0/1417011725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3244ce380 0x55d3244edf00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f33480cb300 tx=0x7f334801a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d20e70181b94cef server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:38.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.548+0000 7f33417fa700 1 -- 172.21.15.119:0/1417011725 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334809a950 con 0x55d3244ce380 2023-12-07T10:23:38.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.548+0000 7f33417fa700 1 -- 172.21.15.119:0/1417011725 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f334809eaa0 con 0x55d3244ce380 2023-12-07T10:23:38.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.548+0000 7f33417fa700 1 -- 172.21.15.119:0/1417011725 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33480ba0b0 con 0x55d3244ce380 2023-12-07T10:23:38.554 INFO:tasks.workunit.client.0.smithi119.stdout: 1 177782 8.80 MB/sec warmup 111 sec latency 11.986 ms 2023-12-07T10:23:38.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.549+0000 7f336c265b80 1 -- 172.21.15.119:0/1417011725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3244ce380 msgr2=0x55d3244edf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:38.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.549+0000 7f336c265b80 1 --2- 172.21.15.119:0/1417011725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3244ce380 0x55d3244edf00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f33480cb300 tx=0x7f334801a040 comp rx=0 tx=0).stop 2023-12-07T10:23:38.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.549+0000 7f336c265b80 1 -- 172.21.15.119:0/1417011725 shutdown_connections 2023-12-07T10:23:38.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.549+0000 7f336c265b80 1 --2- 172.21.15.119:0/1417011725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3244d2360 0x55d3244de5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.549+0000 7f336c265b80 1 --2- 172.21.15.119:0/1417011725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3244cda60 0x55d3244d9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.549+0000 7f336c265b80 1 --2- 172.21.15.119:0/1417011725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3244ce380 0x55d3244edf00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.549+0000 7f336c265b80 1 -- 172.21.15.119:0/1417011725 >> 172.21.15.119:0/1417011725 conn(0x55d3243fc3e0 msgr2=0x55d3244d2740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:38.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.549+0000 7f336c265b80 1 -- 172.21.15.119:0/1417011725 shutdown_connections 2023-12-07T10:23:38.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.549+0000 7f336c265b80 1 -- 172.21.15.119:0/1417011725 wait complete. 2023-12-07T10:23:38.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.549+0000 7f336c265b80 1 -- start start 2023-12-07T10:23:38.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.550+0000 7f336c265b80 1 Processor -- start 2023-12-07T10:23:38.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.550+0000 7f336c265b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3244cda60 0x55d3244dd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.550+0000 7f336c265b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3244ce380 0x55d3244ddb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.550+0000 7f336c265b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3244d2360 0x55d3244de090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.550+0000 7f336c265b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32453d7a0 con 0x55d3244ce380 2023-12-07T10:23:38.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.550+0000 7f336c265b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32453d910 con 0x55d3244cda60 2023-12-07T10:23:38.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.550+0000 7f336c265b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32453dad0 con 0x55d3244d2360 2023-12-07T10:23:38.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.550+0000 7f336c265b80 10 client.0 _ll_register_callbacks cb 0x55d3243c29c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:38.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.551+0000 7f3351d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3244ce380 0x55d3244ddb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.551+0000 7f335259c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3244cda60 0x55d3244dd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.551+0000 7f3351d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3244ce380 0x55d3244ddb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35034/0 (socket says 172.21.15.119:35034) 2023-12-07T10:23:38.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.551+0000 7f3351d9b700 1 -- 172.21.15.119:0/573720661 learned_addr learned my addr 172.21.15.119:0/573720661 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:38.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.551+0000 7f3352d9d700 1 --2- 172.21.15.119:0/573720661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3244d2360 0x55d3244de090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.551+0000 7f3351d9b700 1 -- 172.21.15.119:0/573720661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3244d2360 msgr2=0x55d3244de090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:38.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.551+0000 7f3351d9b700 1 --2- 172.21.15.119:0/573720661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3244d2360 0x55d3244de090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.551+0000 7f3351d9b700 1 -- 172.21.15.119:0/573720661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3244cda60 msgr2=0x55d3244dd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:38.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.551+0000 7f3351d9b700 1 --2- 172.21.15.119:0/573720661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3244cda60 0x55d3244dd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.551+0000 7f3351d9b700 1 -- 172.21.15.119:0/573720661 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d32424ae60 con 0x55d3244ce380 2023-12-07T10:23:38.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.551+0000 7f3351d9b700 1 --2- 172.21.15.119:0/573720661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3244ce380 0x55d3244ddb50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f334800c5b0 tx=0x7f3348096c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43de1380c22db0db server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:38.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.551+0000 7f3339ffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:23:38.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.551+0000 7f3339ffb700 1 -- 172.21.15.119:0/573720661 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334809a950 con 0x55d3244ce380 2023-12-07T10:23:38.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.552+0000 7f3339ffb700 1 -- 172.21.15.119:0/573720661 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f33480ba0b0 con 0x55d3244ce380 2023-12-07T10:23:38.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.552+0000 7f336c265b80 1 -- 172.21.15.119:0/573720661 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f334401e040 con 0x55d3244ce380 2023-12-07T10:23:38.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.552+0000 7f3339ffb700 1 -- 172.21.15.119:0/573720661 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33480ba270 con 0x55d3244ce380 2023-12-07T10:23:38.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.552+0000 7f336c265b80 10 client.16773 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:38.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.552+0000 7f336c265b80 1 -- 172.21.15.119:0/573720661 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f33541aab10 con 0x55d3244ce380 2023-12-07T10:23:38.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.552+0000 7f3339ffb700 1 -- 172.21.15.119:0/573720661 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f334401e040 con 0x55d3244ce380 2023-12-07T10:23:38.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.552+0000 7f3339ffb700 1 -- 172.21.15.119:0/573720661 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f33480a2850 con 0x55d3244ce380 2023-12-07T10:23:38.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.552+0000 7f336c265b80 10 client.16773 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:38.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.552+0000 7f336c265b80 10 client.16773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:23:38.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.552+0000 7f336c265b80 20 client.16773 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:38.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.552+0000 7f336c265b80 1 --2- 172.21.15.119:0/573720661 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d32453f100 0x55d324541520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.552+0000 7f336c265b80 4 client.16773 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-07T10:23:38.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.552+0000 7f336c265b80 1 -- 172.21.15.119:0/573720661 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d3243e8050 con 0x55d32453f100 2023-12-07T10:23:38.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.553+0000 7f335259c700 1 --2- 172.21.15.119:0/573720661 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d32453f100 0x55d324541520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.553+0000 7f335259c700 1 --2- 172.21.15.119:0/573720661 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d32453f100 0x55d324541520 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29acd6f67df6f4cd server_cookie=6e6a172db488b527 in_seq=0 out_seq=0 2023-12-07T10:23:38.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.553+0000 7f3339ffb700 10 client.16773 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:38.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.554+0000 7f3339ffb700 1 -- 172.21.15.119:0/573720661 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d3243e8050 con 0x55d32453f100 2023-12-07T10:23:38.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.554+0000 7f3339ffb700 10 client.16773 handle_command_reply: tid=0 2023-12-07T10:23:38.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.554+0000 7f336c265b80 1 client.16773 shutdown 2023-12-07T10:23:38.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.554+0000 7f336c265b80 1 -- 172.21.15.119:0/573720661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3244ce380 msgr2=0x55d3244ddb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:38.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.554+0000 7f336c265b80 1 --2- 172.21.15.119:0/573720661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3244ce380 0x55d3244ddb50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f334800c5b0 tx=0x7f3348096c90 comp rx=0 tx=0).stop 2023-12-07T10:23:38.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.554+0000 7f336c265b80 1 -- 172.21.15.119:0/573720661 shutdown_connections 2023-12-07T10:23:38.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.554+0000 7f336c265b80 1 --2- 172.21.15.119:0/573720661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3244d2360 0x55d3244de090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.554+0000 7f336c265b80 1 --2- 172.21.15.119:0/573720661 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d32453f100 0x55d324541520 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.554+0000 7f336c265b80 1 --2- 172.21.15.119:0/573720661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3244cda60 0x55d3244dd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.554+0000 7f336c265b80 1 --2- 172.21.15.119:0/573720661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3244ce380 0x55d3244ddb50 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.554+0000 7f336c265b80 1 -- 172.21.15.119:0/573720661 >> 172.21.15.119:0/573720661 conn(0x55d3243fc3e0 msgr2=0x55d3244edf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:38.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.555+0000 7f3339ffb700 0 client.16773 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:38.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.555+0000 7f336c265b80 1 -- 172.21.15.119:0/573720661 shutdown_connections 2023-12-07T10:23:38.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.555+0000 7f336c265b80 1 -- 172.21.15.119:0/573720661 wait complete. 2023-12-07T10:23:38.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.555+0000 7f336c265b80 20 client.16773 trim_cache size 0 max 16384 2023-12-07T10:23:38.562 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:23:38.562 INFO:teuthology.orchestra.run.smithi119.stdout: "return_code": 0, 2023-12-07T10:23:38.562 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_tag": "28eb8e32-22f7-4266-9d91-c26fd74d1618", 2023-12-07T10:23:38.562 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": "asynchronous" 2023-12-07T10:23:38.562 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:23:38.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.556+0000 7f335359e700 1 -- 172.21.15.119:0/1757612987 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f333c0a1d00 msgr2=0x7f333c0a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:38.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.556+0000 7f335359e700 1 --2- 172.21.15.119:0/1757612987 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f333c0a1d00 0x7f333c0a41c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f334c009ec0 tx=0x7f334c003cf0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.556+0000 7f335359e700 1 -- 172.21.15.119:0/1757612987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3354101da0 msgr2=0x7f3354108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:38.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.556+0000 7f335359e700 1 --2- 172.21.15.119:0/1757612987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3354101da0 0x7f3354108850 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f334800e9a0 tx=0x7f334800ecb0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.556+0000 7f335359e700 1 -- 172.21.15.119:0/1757612987 shutdown_connections 2023-12-07T10:23:38.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.556+0000 7f335359e700 1 --2- 172.21.15.119:0/1757612987 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f333c0a1d00 0x7f333c0a41c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.556+0000 7f335359e700 1 --2- 172.21.15.119:0/1757612987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3354107480 0x7f3354108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.556+0000 7f335359e700 1 --2- 172.21.15.119:0/1757612987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33541013e0 0x7f3354108310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.556+0000 7f335359e700 1 --2- 172.21.15.119:0/1757612987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3354101da0 0x7f3354108850 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.556+0000 7f335359e700 1 -- 172.21.15.119:0/1757612987 >> 172.21.15.119:0/1757612987 conn(0x7f33540fcf10 msgr2=0x7f335410caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:38.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.556+0000 7f335359e700 1 -- 172.21.15.119:0/1757612987 shutdown_connections 2023-12-07T10:23:38.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.556+0000 7f335359e700 1 -- 172.21.15.119:0/1757612987 wait complete. 2023-12-07T10:23:38.584 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-07T10:23:38.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.803+0000 7fa8609fe700 1 -- 172.21.15.119:0/239696092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa85c100190 msgr2=0x7fa85c100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:38.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.803+0000 7fa8609fe700 1 --2- 172.21.15.119:0/239696092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa85c100190 0x7fa85c100570 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009990 tx=0x7fa84c009ca0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.803+0000 7fa8609fe700 1 -- 172.21.15.119:0/239696092 shutdown_connections 2023-12-07T10:23:38.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.803+0000 7fa8609fe700 1 --2- 172.21.15.119:0/239696092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa85c10dd40 0x7fa85c110130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.803+0000 7fa8609fe700 1 --2- 172.21.15.119:0/239696092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa85c100190 0x7fa85c100570 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.803+0000 7fa8609fe700 1 --2- 172.21.15.119:0/239696092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa85c100b40 0x7fa85c10d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.803+0000 7fa8609fe700 1 -- 172.21.15.119:0/239696092 >> 172.21.15.119:0/239696092 conn(0x7fa85c0fbdd0 msgr2=0x7fa85c0fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:38.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.803+0000 7fa8609fe700 1 -- 172.21.15.119:0/239696092 shutdown_connections 2023-12-07T10:23:38.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.804+0000 7fa8609fe700 1 -- 172.21.15.119:0/239696092 wait complete. 2023-12-07T10:23:38.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.804+0000 7fa8609fe700 1 Processor -- start 2023-12-07T10:23:38.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.805+0000 7fa8609fe700 1 -- start start 2023-12-07T10:23:38.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.805+0000 7fa8609fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa85c100190 0x7fa85c19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.805+0000 7fa8609fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa85c100b40 0x7fa85c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.805+0000 7fa8609fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa85c10dd40 0x7fa85c199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.805+0000 7fa8609fe700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c199f90 con 0x7fa85c100b40 2023-12-07T10:23:38.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.805+0000 7fa8609fe700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c19a100 con 0x7fa85c10dd40 2023-12-07T10:23:38.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.805+0000 7fa8609fe700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c19a2c0 con 0x7fa85c100190 2023-12-07T10:23:38.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.805+0000 7fa853fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa85c100b40 0x7fa85c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.805+0000 7fa85ad9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa85c100190 0x7fa85c19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.805+0000 7fa85ad9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa85c100190 0x7fa85c19f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41914/0 (socket says 172.21.15.119:41914) 2023-12-07T10:23:38.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.805+0000 7fa853fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa85c100b40 0x7fa85c19fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35046/0 (socket says 172.21.15.119:35046) 2023-12-07T10:23:38.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.805+0000 7fa85ad9d700 1 -- 172.21.15.119:0/624742506 learned_addr learned my addr 172.21.15.119:0/624742506 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:38.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.805+0000 7fa85b59e700 1 --2- 172.21.15.119:0/624742506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa85c10dd40 0x7fa85c199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.806+0000 7fa85ad9d700 1 -- 172.21.15.119:0/624742506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa85c10dd40 msgr2=0x7fa85c199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:38.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.806+0000 7fa85ad9d700 1 --2- 172.21.15.119:0/624742506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa85c10dd40 0x7fa85c199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.806+0000 7fa85ad9d700 1 -- 172.21.15.119:0/624742506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa85c100b40 msgr2=0x7fa85c19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:38.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.806+0000 7fa85ad9d700 1 --2- 172.21.15.119:0/624742506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa85c100b40 0x7fa85c19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.806+0000 7fa85ad9d700 1 -- 172.21.15.119:0/624742506 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa844009520 con 0x7fa85c100190 2023-12-07T10:23:38.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.806+0000 7fa85ad9d700 1 --2- 172.21.15.119:0/624742506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa85c100190 0x7fa85c19f800 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa84c000c00 tx=0x7fa84c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:38.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.806+0000 7fa858d99700 1 -- 172.21.15.119:0/624742506 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c01d070 con 0x7fa85c100190 2023-12-07T10:23:38.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.806+0000 7fa858d99700 1 -- 172.21.15.119:0/624742506 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa84c022470 con 0x7fa85c100190 2023-12-07T10:23:38.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.806+0000 7fa858d99700 1 -- 172.21.15.119:0/624742506 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c017650 con 0x7fa85c100190 2023-12-07T10:23:38.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.806+0000 7fa8609fe700 1 -- 172.21.15.119:0/624742506 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa84c0095f0 con 0x7fa85c100190 2023-12-07T10:23:38.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.807+0000 7fa8609fe700 1 -- 172.21.15.119:0/624742506 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa85c19a780 con 0x7fa85c100190 2023-12-07T10:23:38.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.810+0000 7fa858d99700 1 -- 172.21.15.119:0/624742506 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fa84c0225e0 con 0x7fa85c100190 2023-12-07T10:23:38.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.811+0000 7fa874aa6b80 1 Processor -- start 2023-12-07T10:23:38.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.811+0000 7fa874aa6b80 1 -- start start 2023-12-07T10:23:38.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.812+0000 7fa858d99700 1 --2- 172.21.15.119:0/624742506 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa8480a1c80 0x7fa8480a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.812+0000 7fa874aa6b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286f7489e0 0x56286f748dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.812+0000 7fa874aa6b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56286f749300 0x56286f74cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.812+0000 7fa874aa6b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56286f74d2e0 0x56286f74d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.812+0000 7fa874aa6b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286f6bc7c0 con 0x56286f7489e0 2023-12-07T10:23:38.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.812+0000 7fa874aa6b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286f6a37a0 con 0x56286f74d2e0 2023-12-07T10:23:38.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.812+0000 7fa874aa6b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286f6a10c0 con 0x56286f749300 2023-12-07T10:23:38.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.812+0000 7fa858d99700 1 -- 172.21.15.119:0/624742506 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fa84c0ceff0 con 0x7fa85c100190 2023-12-07T10:23:38.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.812+0000 7fa853fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56286f749300 0x56286f74cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.812+0000 7fa85b59e700 1 --2- 172.21.15.119:0/624742506 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa8480a1c80 0x7fa8480a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.812+0000 7fa853fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56286f749300 0x56286f74cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41926/0 (socket says 172.21.15.119:41926) 2023-12-07T10:23:38.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.812+0000 7fa853fff700 1 -- 172.21.15.119:0/1549530739 learned_addr learned my addr 172.21.15.119:0/1549530739 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:38.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.812+0000 7fa85b59e700 1 --2- 172.21.15.119:0/1549530739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56286f74d2e0 0x56286f74d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.812+0000 7fa85ad9d700 1 --2- 172.21.15.119:0/1549530739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286f7489e0 0x56286f748dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.813+0000 7fa85ad9d700 1 -- 172.21.15.119:0/1549530739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56286f749300 msgr2=0x56286f74cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:38.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.813+0000 7fa85ad9d700 1 --2- 172.21.15.119:0/1549530739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56286f749300 0x56286f74cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.813+0000 7fa85ad9d700 1 -- 172.21.15.119:0/1549530739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56286f74d2e0 msgr2=0x56286f74d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:38.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.813+0000 7fa85ad9d700 1 --2- 172.21.15.119:0/1549530739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56286f74d2e0 0x56286f74d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.813+0000 7fa85ad9d700 1 -- 172.21.15.119:0/1549530739 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56286f519630 con 0x56286f7489e0 2023-12-07T10:23:38.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.813+0000 7fa85b59e700 1 --2- 172.21.15.119:0/624742506 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa8480a1c80 0x7fa8480a4140 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa854009ec0 tx=0x7fa8540134a0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:38.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.813+0000 7fa85ad9d700 1 --2- 172.21.15.119:0/1549530739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286f7489e0 0x56286f748dc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa84c0cc3d0 tx=0x7fa84c097df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5242d84a7c020003 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:38.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.813+0000 7fa8517fa700 1 -- 172.21.15.119:0/1549530739 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c09b410 con 0x56286f7489e0 2023-12-07T10:23:38.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.814+0000 7fa8517fa700 1 -- 172.21.15.119:0/1549530739 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa84c00c2d0 con 0x56286f7489e0 2023-12-07T10:23:38.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.814+0000 7fa8517fa700 1 -- 172.21.15.119:0/1549530739 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c09bb40 con 0x56286f7489e0 2023-12-07T10:23:38.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.814+0000 7fa874aa6b80 1 -- 172.21.15.119:0/1549530739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286f7489e0 msgr2=0x56286f748dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:38.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.814+0000 7fa874aa6b80 1 --2- 172.21.15.119:0/1549530739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286f7489e0 0x56286f748dc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa84c0cc3d0 tx=0x7fa84c097df0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.814+0000 7fa874aa6b80 1 -- 172.21.15.119:0/1549530739 shutdown_connections 2023-12-07T10:23:38.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.814+0000 7fa874aa6b80 1 --2- 172.21.15.119:0/1549530739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56286f74d2e0 0x56286f74d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.814+0000 7fa874aa6b80 1 --2- 172.21.15.119:0/1549530739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286f7489e0 0x56286f748dc0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.814+0000 7fa874aa6b80 1 --2- 172.21.15.119:0/1549530739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56286f749300 0x56286f74cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.814+0000 7fa874aa6b80 1 -- 172.21.15.119:0/1549530739 >> 172.21.15.119:0/1549530739 conn(0x56286f677360 msgr2=0x56286f743170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:38.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.815+0000 7fa874aa6b80 1 -- 172.21.15.119:0/1549530739 shutdown_connections 2023-12-07T10:23:38.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.815+0000 7fa874aa6b80 1 -- 172.21.15.119:0/1549530739 wait complete. 2023-12-07T10:23:38.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.815+0000 7fa874aa6b80 1 -- start start 2023-12-07T10:23:38.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.815+0000 7fa874aa6b80 1 Processor -- start 2023-12-07T10:23:38.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.816+0000 7fa874aa6b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56286f7489e0 0x56286f7549a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.816+0000 7fa874aa6b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56286f749300 0x56286f751d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.816+0000 7fa874aa6b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286f74d2e0 0x56286f752270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.816+0000 7fa874aa6b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286f69c9b0 con 0x56286f74d2e0 2023-12-07T10:23:38.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.816+0000 7fa874aa6b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286f656560 con 0x56286f749300 2023-12-07T10:23:38.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.816+0000 7fa874aa6b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286f656200 con 0x56286f7489e0 2023-12-07T10:23:38.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.816+0000 7fa874aa6b80 10 client.0 _ll_register_callbacks cb 0x56286f63d940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:38.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.816+0000 7fa853fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56286f749300 0x56286f751d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.816+0000 7fa85ad9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56286f7489e0 0x56286f7549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.816+0000 7fa853fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56286f749300 0x56286f751d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:56262/0 (socket says 172.21.15.119:56262) 2023-12-07T10:23:38.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.816+0000 7fa853fff700 1 -- 172.21.15.119:0/3370475120 learned_addr learned my addr 172.21.15.119:0/3370475120 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:38.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.816+0000 7fa85b59e700 1 --2- 172.21.15.119:0/3370475120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286f74d2e0 0x56286f752270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.816+0000 7fa853fff700 1 -- 172.21.15.119:0/3370475120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56286f7489e0 msgr2=0x56286f7549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:38.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.816+0000 7fa853fff700 1 --2- 172.21.15.119:0/3370475120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56286f7489e0 0x56286f7549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.816+0000 7fa853fff700 1 -- 172.21.15.119:0/3370475120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286f74d2e0 msgr2=0x56286f752270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:38.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.816+0000 7fa853fff700 1 --2- 172.21.15.119:0/3370475120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286f74d2e0 0x56286f752270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.816+0000 7fa853fff700 1 -- 172.21.15.119:0/3370475120 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa85c19a780 con 0x56286f749300 2023-12-07T10:23:38.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.817+0000 7fa85b59e700 1 --2- 172.21.15.119:0/3370475120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286f74d2e0 0x56286f752270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:23:38.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.817+0000 7fa853fff700 1 --2- 172.21.15.119:0/3370475120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56286f749300 0x56286f751d30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa844005c50 tx=0x7fa844011bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de867c87249e960 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:38.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.817+0000 7fa8427fc700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:23:38.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.817+0000 7fa8427fc700 1 -- 172.21.15.119:0/3370475120 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa844016070 con 0x56286f749300 2023-12-07T10:23:38.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.817+0000 7fa874aa6b80 1 -- 172.21.15.119:0/3370475120 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56286f73b9a0 con 0x56286f749300 2023-12-07T10:23:38.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.817+0000 7fa8427fc700 1 -- 172.21.15.119:0/3370475120 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fa844004260 con 0x56286f749300 2023-12-07T10:23:38.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.817+0000 7fa8427fc700 1 -- 172.21.15.119:0/3370475120 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa844004420 con 0x56286f749300 2023-12-07T10:23:38.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.817+0000 7fa874aa6b80 10 client.26810 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:38.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.817+0000 7fa874aa6b80 1 -- 172.21.15.119:0/3370475120 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x56286f4c5de0 con 0x56286f749300 2023-12-07T10:23:38.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.818+0000 7fa8427fc700 1 -- 172.21.15.119:0/3370475120 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56286f73b9a0 con 0x56286f749300 2023-12-07T10:23:38.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.818+0000 7fa8427fc700 1 -- 172.21.15.119:0/3370475120 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fa84400f8c0 con 0x56286f749300 2023-12-07T10:23:38.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.818+0000 7fa874aa6b80 10 client.26810 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:38.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.818+0000 7fa874aa6b80 10 client.26810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:23:38.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.818+0000 7fa874aa6b80 20 client.26810 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:38.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.818+0000 7fa874aa6b80 1 --2- 172.21.15.119:0/3370475120 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56286f76aad0 0x56286f76cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.818+0000 7fa874aa6b80 4 client.26810 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:23:38.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.818+0000 7fa874aa6b80 1 -- 172.21.15.119:0/3370475120 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56286f7177a0 con 0x56286f76aad0 2023-12-07T10:23:38.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.818+0000 7fa85ad9d700 1 --2- 172.21.15.119:0/3370475120 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56286f76aad0 0x56286f76cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.819+0000 7fa85ad9d700 1 --2- 172.21.15.119:0/3370475120 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56286f76aad0 0x56286f76cef0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f33e7bc9497154b server_cookie=c63fe9d50324f26a in_seq=0 out_seq=0 2023-12-07T10:23:38.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.819+0000 7fa8427fc700 10 client.26810 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:38.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.820+0000 7fa8427fc700 1 -- 172.21.15.119:0/3370475120 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56286f7177a0 con 0x56286f76aad0 2023-12-07T10:23:38.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.820+0000 7fa8427fc700 10 client.26810 handle_command_reply: tid=0 2023-12-07T10:23:38.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.820+0000 7fa874aa6b80 1 client.26810 shutdown 2023-12-07T10:23:38.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.820+0000 7fa874aa6b80 1 -- 172.21.15.119:0/3370475120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56286f749300 msgr2=0x56286f751d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:38.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.820+0000 7fa874aa6b80 1 --2- 172.21.15.119:0/3370475120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56286f749300 0x56286f751d30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa844005c50 tx=0x7fa844011bb0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.820+0000 7fa874aa6b80 1 -- 172.21.15.119:0/3370475120 shutdown_connections 2023-12-07T10:23:38.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.820+0000 7fa874aa6b80 1 --2- 172.21.15.119:0/3370475120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286f74d2e0 0x56286f752270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.820+0000 7fa874aa6b80 1 --2- 172.21.15.119:0/3370475120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56286f7489e0 0x56286f7549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.820+0000 7fa874aa6b80 1 --2- 172.21.15.119:0/3370475120 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56286f76aad0 0x56286f76cef0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.820+0000 7fa874aa6b80 1 --2- 172.21.15.119:0/3370475120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56286f749300 0x56286f751d30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.820+0000 7fa8427fc700 0 client.26810 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:38.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.820+0000 7fa874aa6b80 1 -- 172.21.15.119:0/3370475120 >> 172.21.15.119:0/3370475120 conn(0x56286f677360 msgr2=0x56286f7477e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:38.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.820+0000 7fa874aa6b80 1 -- 172.21.15.119:0/3370475120 shutdown_connections 2023-12-07T10:23:38.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.820+0000 7fa874aa6b80 1 -- 172.21.15.119:0/3370475120 wait complete. 2023-12-07T10:23:38.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.820+0000 7fa874aa6b80 20 client.26810 trim_cache size 0 max 16384 2023-12-07T10:23:38.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.840+0000 7fa874aa6b80 1 Processor -- start 2023-12-07T10:23:38.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.840+0000 7fa874aa6b80 1 -- start start 2023-12-07T10:23:38.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.841+0000 7fa874aa6b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56286f7489e0 0x56286f751270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.841+0000 7fa874aa6b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56286f749300 0x56286f771810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.841+0000 7fa874aa6b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286f74d2e0 0x56286f761a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.841+0000 7fa874aa6b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286f51b920 con 0x56286f74d2e0 2023-12-07T10:23:38.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.841+0000 7fa874aa6b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286f752850 con 0x56286f7489e0 2023-12-07T10:23:38.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.841+0000 7fa874aa6b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286f748610 con 0x56286f749300 2023-12-07T10:23:38.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.841+0000 7fa85ad9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56286f7489e0 0x56286f751270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.841+0000 7fa85ad9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56286f7489e0 0x56286f751270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:56278/0 (socket says 172.21.15.119:56278) 2023-12-07T10:23:38.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.841+0000 7fa85ad9d700 1 -- 172.21.15.119:0/1826537572 learned_addr learned my addr 172.21.15.119:0/1826537572 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:38.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.841+0000 7fa853fff700 1 --2- 172.21.15.119:0/1826537572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56286f749300 0x56286f771810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.841+0000 7fa85b59e700 1 --2- 172.21.15.119:0/1826537572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286f74d2e0 0x56286f761a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.841+0000 7fa85ad9d700 1 -- 172.21.15.119:0/1826537572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56286f749300 msgr2=0x56286f771810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:38.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.841+0000 7fa85ad9d700 1 --2- 172.21.15.119:0/1826537572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56286f749300 0x56286f771810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.841+0000 7fa85ad9d700 1 -- 172.21.15.119:0/1826537572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286f74d2e0 msgr2=0x56286f761a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:38.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.841+0000 7fa85ad9d700 1 --2- 172.21.15.119:0/1826537572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286f74d2e0 0x56286f761a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.841+0000 7fa85ad9d700 1 -- 172.21.15.119:0/1826537572 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56286f4aec60 con 0x56286f7489e0 2023-12-07T10:23:38.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.842+0000 7fa853fff700 1 --2- 172.21.15.119:0/1826537572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56286f749300 0x56286f771810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:38.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.842+0000 7fa85ad9d700 1 --2- 172.21.15.119:0/1826537572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56286f7489e0 0x56286f751270 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa84c005cb0 tx=0x7fa84c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5357cc820508ffaa server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:38.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.842+0000 7fa851ffb700 1 -- 172.21.15.119:0/1826537572 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c09c1b0 con 0x56286f7489e0 2023-12-07T10:23:38.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.842+0000 7fa851ffb700 1 -- 172.21.15.119:0/1826537572 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa84c09fc00 con 0x56286f7489e0 2023-12-07T10:23:38.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.842+0000 7fa851ffb700 1 -- 172.21.15.119:0/1826537572 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c0cc650 con 0x56286f7489e0 2023-12-07T10:23:38.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.843+0000 7fa874aa6b80 1 -- 172.21.15.119:0/1826537572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56286f7489e0 msgr2=0x56286f751270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:38.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.843+0000 7fa874aa6b80 1 --2- 172.21.15.119:0/1826537572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56286f7489e0 0x56286f751270 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa84c005cb0 tx=0x7fa84c009ca0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.843+0000 7fa874aa6b80 1 -- 172.21.15.119:0/1826537572 shutdown_connections 2023-12-07T10:23:38.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.843+0000 7fa874aa6b80 1 --2- 172.21.15.119:0/1826537572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286f74d2e0 0x56286f761a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.843+0000 7fa874aa6b80 1 --2- 172.21.15.119:0/1826537572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56286f7489e0 0x56286f751270 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.843+0000 7fa874aa6b80 1 --2- 172.21.15.119:0/1826537572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56286f749300 0x56286f771810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.843+0000 7fa874aa6b80 1 -- 172.21.15.119:0/1826537572 >> 172.21.15.119:0/1826537572 conn(0x56286f677360 msgr2=0x56286f754660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:38.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.843+0000 7fa874aa6b80 1 -- 172.21.15.119:0/1826537572 shutdown_connections 2023-12-07T10:23:38.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.843+0000 7fa874aa6b80 1 -- 172.21.15.119:0/1826537572 wait complete. 2023-12-07T10:23:38.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.843+0000 7fa874aa6b80 1 -- start start 2023-12-07T10:23:38.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.843+0000 7fa874aa6b80 1 Processor -- start 2023-12-07T10:23:38.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.844+0000 7fa874aa6b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56286f7489e0 0x56286f75fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.844+0000 7fa874aa6b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286f749300 0x56286f7601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.844+0000 7fa874aa6b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56286f74d2e0 0x56286f7bff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.844+0000 7fa874aa6b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286f7c04d0 con 0x56286f749300 2023-12-07T10:23:38.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.844+0000 7fa874aa6b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286f7c0610 con 0x56286f74d2e0 2023-12-07T10:23:38.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.844+0000 7fa874aa6b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286f7c0750 con 0x56286f7489e0 2023-12-07T10:23:38.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.844+0000 7fa874aa6b80 10 client.0 _ll_register_callbacks cb 0x56286f63d940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:38.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.844+0000 7fa85b59e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56286f74d2e0 0x56286f7bff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.844+0000 7fa853fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286f749300 0x56286f7601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.844+0000 7fa85b59e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56286f74d2e0 0x56286f7bff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:56282/0 (socket says 172.21.15.119:56282) 2023-12-07T10:23:38.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.844+0000 7fa85b59e700 1 -- 172.21.15.119:0/1729217850 learned_addr learned my addr 172.21.15.119:0/1729217850 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:38.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.844+0000 7fa85ad9d700 1 --2- 172.21.15.119:0/1729217850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56286f7489e0 0x56286f75fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.845+0000 7fa85b59e700 1 -- 172.21.15.119:0/1729217850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56286f7489e0 msgr2=0x56286f75fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:38.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.845+0000 7fa85b59e700 1 --2- 172.21.15.119:0/1729217850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56286f7489e0 0x56286f75fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.845+0000 7fa85b59e700 1 -- 172.21.15.119:0/1729217850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286f749300 msgr2=0x56286f7601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:38.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.845+0000 7fa85b59e700 1 --2- 172.21.15.119:0/1729217850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286f749300 0x56286f7601a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.845+0000 7fa85b59e700 1 -- 172.21.15.119:0/1729217850 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56286f4c5de0 con 0x56286f74d2e0 2023-12-07T10:23:38.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.845+0000 7fa853fff700 1 --2- 172.21.15.119:0/1729217850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286f749300 0x56286f7601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:38.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.845+0000 7fa85b59e700 1 --2- 172.21.15.119:0/1729217850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56286f74d2e0 0x56286f7bff90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa8540092c0 tx=0x7fa854004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec1d4c373e7dbcd server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:38.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.845+0000 7fa8427fc700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:23:38.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.845+0000 7fa8427fc700 1 -- 172.21.15.119:0/1729217850 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85400e040 con 0x56286f74d2e0 2023-12-07T10:23:38.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.845+0000 7fa874aa6b80 1 -- 172.21.15.119:0/1729217850 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa84c00c2d0 con 0x56286f74d2e0 2023-12-07T10:23:38.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.845+0000 7fa8427fc700 1 -- 172.21.15.119:0/1729217850 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fa854008e00 con 0x56286f74d2e0 2023-12-07T10:23:38.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.846+0000 7fa8427fc700 1 -- 172.21.15.119:0/1729217850 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa85401aeb0 con 0x56286f74d2e0 2023-12-07T10:23:38.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.846+0000 7fa874aa6b80 10 client.26816 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:38.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.846+0000 7fa874aa6b80 1 -- 172.21.15.119:0/1729217850 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x56286f519630 con 0x56286f74d2e0 2023-12-07T10:23:38.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.846+0000 7fa8427fc700 1 -- 172.21.15.119:0/1729217850 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa84c00c2d0 con 0x56286f74d2e0 2023-12-07T10:23:38.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.846+0000 7fa8427fc700 1 -- 172.21.15.119:0/1729217850 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fa85402d030 con 0x56286f74d2e0 2023-12-07T10:23:38.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.846+0000 7fa874aa6b80 10 client.26816 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:38.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.846+0000 7fa874aa6b80 10 client.26816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:23:38.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.846+0000 7fa874aa6b80 20 client.26816 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:38.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.846+0000 7fa874aa6b80 1 --2- 172.21.15.119:0/1729217850 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56286f7c2050 0x56286f7c4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:38.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.846+0000 7fa874aa6b80 4 client.26816 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-12-07T10:23:38.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.846+0000 7fa874aa6b80 1 -- 172.21.15.119:0/1729217850 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56286f662fd0 con 0x56286f7c2050 2023-12-07T10:23:38.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.846+0000 7fa85ad9d700 1 --2- 172.21.15.119:0/1729217850 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56286f7c2050 0x56286f7c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:38.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.847+0000 7fa85ad9d700 1 --2- 172.21.15.119:0/1729217850 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56286f7c2050 0x56286f7c4470 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89f5adf219097702 server_cookie=7dee1c413f4b78da in_seq=0 out_seq=0 2023-12-07T10:23:38.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.847+0000 7fa8427fc700 10 client.26816 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:38.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.847+0000 7fa8427fc700 1 -- 172.21.15.119:0/1729217850 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56286f662fd0 con 0x56286f7c2050 2023-12-07T10:23:38.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.847+0000 7fa8427fc700 10 client.26816 handle_command_reply: tid=0 2023-12-07T10:23:38.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.847+0000 7fa874aa6b80 1 client.26816 shutdown 2023-12-07T10:23:38.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.847+0000 7fa874aa6b80 1 -- 172.21.15.119:0/1729217850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56286f74d2e0 msgr2=0x56286f7bff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:38.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.847+0000 7fa874aa6b80 1 --2- 172.21.15.119:0/1729217850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56286f74d2e0 0x56286f7bff90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa8540092c0 tx=0x7fa854004b40 comp rx=0 tx=0).stop 2023-12-07T10:23:38.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.847+0000 7fa874aa6b80 1 -- 172.21.15.119:0/1729217850 shutdown_connections 2023-12-07T10:23:38.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.847+0000 7fa874aa6b80 1 --2- 172.21.15.119:0/1729217850 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56286f7c2050 0x56286f7c4470 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.847+0000 7fa874aa6b80 1 --2- 172.21.15.119:0/1729217850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56286f74d2e0 0x56286f7bff90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.847+0000 7fa874aa6b80 1 --2- 172.21.15.119:0/1729217850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56286f7489e0 0x56286f75fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.847+0000 7fa8427fc700 0 client.26816 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:38.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.847+0000 7fa874aa6b80 1 --2- 172.21.15.119:0/1729217850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56286f749300 0x56286f7601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.847+0000 7fa874aa6b80 1 -- 172.21.15.119:0/1729217850 >> 172.21.15.119:0/1729217850 conn(0x56286f677360 msgr2=0x56286f74b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:38.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.848+0000 7fa874aa6b80 1 -- 172.21.15.119:0/1729217850 shutdown_connections 2023-12-07T10:23:38.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.848+0000 7fa874aa6b80 1 -- 172.21.15.119:0/1729217850 wait complete. 2023-12-07T10:23:38.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.848+0000 7fa874aa6b80 20 client.26816 trim_cache size 0 max 16384 2023-12-07T10:23:38.852 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:23:38.852 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-07T10:23:38.853 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": { 2023-12-07T10:23:38.853 INFO:teuthology.orchestra.run.smithi119.stdout: "28eb8e32-22f7-4266-9d91-c26fd74d1618": { 2023-12-07T10:23:38.853 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/", 2023-12-07T10:23:38.853 INFO:teuthology.orchestra.run.smithi119.stdout: "tag": "28eb8e32-22f7-4266-9d91-c26fd74d1618", 2023-12-07T10:23:38.853 INFO:teuthology.orchestra.run.smithi119.stdout: "options": "recursive,force" 2023-12-07T10:23:38.853 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:23:38.853 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:23:38.853 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:23:38.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.848+0000 7fa8609fe700 1 -- 172.21.15.119:0/624742506 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa8480a1c80 msgr2=0x7fa8480a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:38.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.848+0000 7fa8609fe700 1 --2- 172.21.15.119:0/624742506 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa8480a1c80 0x7fa8480a4140 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa854009ec0 tx=0x7fa8540134a0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.848+0000 7fa8609fe700 1 -- 172.21.15.119:0/624742506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa85c100190 msgr2=0x7fa85c19f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:38.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.848+0000 7fa8609fe700 1 --2- 172.21.15.119:0/624742506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa85c100190 0x7fa85c19f800 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa84c000c00 tx=0x7fa84c005740 comp rx=0 tx=0).stop 2023-12-07T10:23:38.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.848+0000 7fa8609fe700 1 -- 172.21.15.119:0/624742506 shutdown_connections 2023-12-07T10:23:38.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.848+0000 7fa8609fe700 1 --2- 172.21.15.119:0/624742506 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa8480a1c80 0x7fa8480a4140 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.848+0000 7fa8609fe700 1 --2- 172.21.15.119:0/624742506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa85c10dd40 0x7fa85c199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.848+0000 7fa8609fe700 1 --2- 172.21.15.119:0/624742506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa85c100190 0x7fa85c19f800 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.848+0000 7fa8609fe700 1 --2- 172.21.15.119:0/624742506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa85c100b40 0x7fa85c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:38.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.848+0000 7fa8609fe700 1 -- 172.21.15.119:0/624742506 >> 172.21.15.119:0/624742506 conn(0x7fa85c0fbdd0 msgr2=0x7fa85c0fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:38.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.849+0000 7fa8609fe700 1 -- 172.21.15.119:0/624742506 shutdown_connections 2023-12-07T10:23:38.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:23:38.849+0000 7fa8609fe700 1 -- 172.21.15.119:0/624742506 wait complete. 2023-12-07T10:23:38.869 INFO:tasks.cephfs.filesystem:scrub status for tag:28eb8e32-22f7-4266-9d91-c26fd74d1618 - {'path': '/', 'tag': '28eb8e32-22f7-4266-9d91-c26fd74d1618', 'options': 'recursive,force'} 2023-12-07T10:23:39.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:38 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:23:39.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:38 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:23:39.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:38 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:23:39.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:38 smithi159 ceph-mon[134726]: scrub queued for path: / 2023-12-07T10:23:39.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:38 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:39.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:38 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:23:39.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:38 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:23:39.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:38 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:39.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:38 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:39.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:38 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:23:39.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:38 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:39.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:38 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:23:39.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:38 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:39.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:38 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:23:39.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:38 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:23:39.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:38 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:23:39.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:38 smithi153 ceph-mon[133625]: scrub queued for path: / 2023-12-07T10:23:39.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:38 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:39.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:38 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:23:39.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:38 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:23:39.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:38 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:39.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:38 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:39.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:38 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:23:39.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:38 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:39.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:38 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:23:39.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:38 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:39.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:38 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:23:39.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:38 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:23:39.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:38 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:23:39.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:38 smithi119 ceph-mon[97908]: scrub queued for path: / 2023-12-07T10:23:39.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:38 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:39.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:38 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:23:39.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:38 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:23:39.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:38 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:39.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:38 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:39.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:38 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:23:39.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:38 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:39.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:38 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:23:39.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:38 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:39.549 INFO:tasks.workunit.client.0.smithi119.stdout: 1 179806 8.79 MB/sec warmup 112 sec latency 18.074 ms 2023-12-07T10:23:40.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:39 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:23:40.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:39 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:23:40.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:39 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:23:40.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:39 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:40.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:39 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:40.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:39 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:23:40.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:39 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:40.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:39 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:40.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:39 smithi159 ceph-mon[134726]: pgmap v100: 129 pgs: 129 active+clean; 108 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 235 op/s 2023-12-07T10:23:40.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:39 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:23:40.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:39 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:23:40.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:39 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:23:40.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:39 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:40.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:39 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:40.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:39 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:23:40.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:39 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:40.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:39 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:40.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:39 smithi153 ceph-mon[133625]: pgmap v100: 129 pgs: 129 active+clean; 108 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 235 op/s 2023-12-07T10:23:40.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:39 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:23:40.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:39 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:23:40.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:39 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:23:40.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:39 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:40.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:39 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:40.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:39 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:23:40.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:39 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:40.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:39 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:23:40.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:39 smithi119 ceph-mon[97908]: pgmap v100: 129 pgs: 129 active+clean; 108 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 235 op/s 2023-12-07T10:23:40.549 INFO:tasks.workunit.client.0.smithi119.stdout: 1 181371 8.82 MB/sec warmup 113 sec latency 13.970 ms 2023-12-07T10:23:41.549 INFO:tasks.workunit.client.0.smithi119.stdout: 1 183314 8.80 MB/sec warmup 114 sec latency 83.593 ms 2023-12-07T10:23:42.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:41 smithi119 ceph-mon[97908]: pgmap v101: 129 pgs: 129 active+clean; 108 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 339 op/s 2023-12-07T10:23:42.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:41 smithi159 ceph-mon[134726]: pgmap v101: 129 pgs: 129 active+clean; 108 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 339 op/s 2023-12-07T10:23:42.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:41 smithi153 ceph-mon[133625]: pgmap v101: 129 pgs: 129 active+clean; 108 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 339 op/s 2023-12-07T10:23:42.549 INFO:tasks.workunit.client.0.smithi119.stdout: 1 184857 8.83 MB/sec warmup 115 sec latency 18.407 ms 2023-12-07T10:23:43.549 INFO:tasks.workunit.client.0.smithi119.stdout: 1 186880 8.81 MB/sec warmup 116 sec latency 10.962 ms 2023-12-07T10:23:44.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:43 smithi159 ceph-mon[134726]: pgmap v102: 129 pgs: 129 active+clean; 108 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 241 op/s 2023-12-07T10:23:44.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:43 smithi153 ceph-mon[133625]: pgmap v102: 129 pgs: 129 active+clean; 108 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 241 op/s 2023-12-07T10:23:44.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:43 smithi119 ceph-mon[97908]: pgmap v102: 129 pgs: 129 active+clean; 108 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 241 op/s 2023-12-07T10:23:44.549 INFO:tasks.workunit.client.0.smithi119.stdout: 1 188009 8.83 MB/sec warmup 117 sec latency 434.343 ms 2023-12-07T10:23:45.549 INFO:tasks.workunit.client.0.smithi119.stdout: 1 188009 8.75 MB/sec warmup 118 sec latency 1434.434 ms 2023-12-07T10:23:46.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:45 smithi159 ceph-mon[134726]: pgmap v103: 129 pgs: 129 active+clean; 108 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 241 op/s 2023-12-07T10:23:46.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:45 smithi153 ceph-mon[133625]: pgmap v103: 129 pgs: 129 active+clean; 108 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 241 op/s 2023-12-07T10:23:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.475+0000 7f73933ff700 1 -- 172.21.15.119:0/2447662714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f738c101500 msgr2=0x7f738c101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.475+0000 7f73933ff700 1 --2- 172.21.15.119:0/2447662714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f738c101500 0x7f738c101980 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f737c0099c0 tx=0x7f737c009cd0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.475+0000 7f73933ff700 1 -- 172.21.15.119:0/2447662714 shutdown_connections 2023-12-07T10:23:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.475+0000 7f73933ff700 1 --2- 172.21.15.119:0/2447662714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f738c101ec0 0x7f738c10e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.475+0000 7f73933ff700 1 --2- 172.21.15.119:0/2447662714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f738c1075c0 0x7f738c1079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.475+0000 7f73933ff700 1 --2- 172.21.15.119:0/2447662714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f738c101500 0x7f738c101980 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.475+0000 7f73933ff700 1 -- 172.21.15.119:0/2447662714 >> 172.21.15.119:0/2447662714 conn(0x7f738c0fcfb0 msgr2=0x7f738c0ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.475+0000 7f73933ff700 1 -- 172.21.15.119:0/2447662714 shutdown_connections 2023-12-07T10:23:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.475+0000 7f73933ff700 1 -- 172.21.15.119:0/2447662714 wait complete. 2023-12-07T10:23:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.476+0000 7f73933ff700 1 Processor -- start 2023-12-07T10:23:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.476+0000 7f73933ff700 1 -- start start 2023-12-07T10:23:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.477+0000 7f73933ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f738c101500 0x7f738c19d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.477+0000 7f73933ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f738c101ec0 0x7f738c19db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.477+0000 7f73933ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f738c1075c0 0x7f738c197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.477+0000 7f73933ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c197df0 con 0x7f738c101500 2023-12-07T10:23:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.477+0000 7f73933ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c197f60 con 0x7f738c101ec0 2023-12-07T10:23:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.477+0000 7f73933ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c198120 con 0x7f738c1075c0 2023-12-07T10:23:46.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:45 smithi119 ceph-mon[97908]: pgmap v103: 129 pgs: 129 active+clean; 108 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 241 op/s 2023-12-07T10:23:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.477+0000 7f739119b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f738c101500 0x7f738c19d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.477+0000 7f739119b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f738c101500 0x7f738c19d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35626/0 (socket says 172.21.15.119:35626) 2023-12-07T10:23:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.477+0000 7f739119b700 1 -- 172.21.15.119:0/698105965 learned_addr learned my addr 172.21.15.119:0/698105965 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.477+0000 7f739199c700 1 --2- 172.21.15.119:0/698105965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f738c1075c0 0x7f738c197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.477+0000 7f739099a700 1 --2- 172.21.15.119:0/698105965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f738c101ec0 0x7f738c19db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.477+0000 7f739119b700 1 -- 172.21.15.119:0/698105965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f738c1075c0 msgr2=0x7f738c197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.477+0000 7f739119b700 1 --2- 172.21.15.119:0/698105965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f738c1075c0 0x7f738c197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.477+0000 7f739119b700 1 -- 172.21.15.119:0/698105965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f738c101ec0 msgr2=0x7f738c19db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.477+0000 7f739119b700 1 --2- 172.21.15.119:0/698105965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f738c101ec0 0x7f738c19db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.477+0000 7f739119b700 1 -- 172.21.15.119:0/698105965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737c0095f0 con 0x7f738c101500 2023-12-07T10:23:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.478+0000 7f739119b700 1 --2- 172.21.15.119:0/698105965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f738c101500 0x7f738c19d610 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f738800e9a0 tx=0x7f738800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.478+0000 7f73827fc700 1 -- 172.21.15.119:0/698105965 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7388009790 con 0x7f738c101500 2023-12-07T10:23:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.478+0000 7f73827fc700 1 -- 172.21.15.119:0/698105965 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7388010430 con 0x7f738c101500 2023-12-07T10:23:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.478+0000 7f73827fc700 1 -- 172.21.15.119:0/698105965 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7388018680 con 0x7f738c101500 2023-12-07T10:23:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.478+0000 7f73933ff700 1 -- 172.21.15.119:0/698105965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f738c198400 con 0x7f738c101500 2023-12-07T10:23:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.478+0000 7f73933ff700 1 -- 172.21.15.119:0/698105965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738c1a4600 con 0x7f738c101500 2023-12-07T10:23:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.481+0000 7f73827fc700 1 -- 172.21.15.119:0/698105965 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f73880105a0 con 0x7f738c101500 2023-12-07T10:23:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.482+0000 7f73827fc700 1 --2- 172.21.15.119:0/698105965 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f73780a1d00 0x7f73780a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.482+0000 7f73a6c65b80 1 Processor -- start 2023-12-07T10:23:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.483+0000 7f73827fc700 1 -- 172.21.15.119:0/698105965 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f7388014070 con 0x7f738c101500 2023-12-07T10:23:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.483+0000 7f73a6c65b80 1 -- start start 2023-12-07T10:23:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.483+0000 7f739099a700 1 --2- 172.21.15.119:0/698105965 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f73780a1d00 0x7f73780a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.483+0000 7f73a6c65b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdcef10da0 0x55bdcef11180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.483+0000 7f73a6c65b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdcef0c950 0x55bdcef0cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.483+0000 7f73a6c65b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdcef0d310 0x55bdcef18cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.483+0000 7f73a6c65b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcee7c760 con 0x55bdcef10da0 2023-12-07T10:23:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.483+0000 7f73a6c65b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcee63740 con 0x55bdcef0d310 2023-12-07T10:23:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.483+0000 7f73a6c65b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcee61060 con 0x55bdcef0c950 2023-12-07T10:23:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.483+0000 7f739099a700 1 --2- 172.21.15.119:0/698105965 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f73780a1d00 0x7f73780a41c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f737c009990 tx=0x7f737c005fd0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.483+0000 7f739119b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdcef10da0 0x55bdcef11180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.484+0000 7f739119b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdcef10da0 0x55bdcef11180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35632/0 (socket says 172.21.15.119:35632) 2023-12-07T10:23:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.484+0000 7f739119b700 1 -- 172.21.15.119:0/3189927516 learned_addr learned my addr 172.21.15.119:0/3189927516 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.484+0000 7f739199c700 1 --2- 172.21.15.119:0/3189927516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdcef0d310 0x55bdcef18cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.484+0000 7f739099a700 1 --2- 172.21.15.119:0/3189927516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdcef0c950 0x55bdcef0cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.484+0000 7f739119b700 1 -- 172.21.15.119:0/3189927516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdcef0c950 msgr2=0x55bdcef0cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.484+0000 7f739119b700 1 --2- 172.21.15.119:0/3189927516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdcef0c950 0x55bdcef0cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.484+0000 7f739119b700 1 -- 172.21.15.119:0/3189927516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdcef0d310 msgr2=0x55bdcef18cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.484+0000 7f739119b700 1 --2- 172.21.15.119:0/3189927516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdcef0d310 0x55bdcef18cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.484+0000 7f739119b700 1 -- 172.21.15.119:0/3189927516 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdcecd95d0 con 0x55bdcef10da0 2023-12-07T10:23:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.484+0000 7f739119b700 1 --2- 172.21.15.119:0/3189927516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdcef10da0 0x55bdcef11180 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7388097310 tx=0x7f738809a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b58d2fb5ba5137f server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.484+0000 7f73777fe700 1 -- 172.21.15.119:0/3189927516 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738809e9f0 con 0x55bdcef10da0 2023-12-07T10:23:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.484+0000 7f73777fe700 1 -- 172.21.15.119:0/3189927516 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73880a2c30 con 0x55bdcef10da0 2023-12-07T10:23:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.484+0000 7f73777fe700 1 -- 172.21.15.119:0/3189927516 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738809ad60 con 0x55bdcef10da0 2023-12-07T10:23:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.485+0000 7f73a6c65b80 1 -- 172.21.15.119:0/3189927516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdcef10da0 msgr2=0x55bdcef11180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.485+0000 7f73a6c65b80 1 --2- 172.21.15.119:0/3189927516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdcef10da0 0x55bdcef11180 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7388097310 tx=0x7f738809a510 comp rx=0 tx=0).stop 2023-12-07T10:23:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.485+0000 7f73a6c65b80 1 -- 172.21.15.119:0/3189927516 shutdown_connections 2023-12-07T10:23:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.485+0000 7f73a6c65b80 1 --2- 172.21.15.119:0/3189927516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdcef0d310 0x55bdcef18cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.485+0000 7f73a6c65b80 1 --2- 172.21.15.119:0/3189927516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdcef10da0 0x55bdcef11180 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.485+0000 7f73a6c65b80 1 --2- 172.21.15.119:0/3189927516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdcef0c950 0x55bdcef0cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.485+0000 7f73a6c65b80 1 -- 172.21.15.119:0/3189927516 >> 172.21.15.119:0/3189927516 conn(0x55bdcee37300 msgr2=0x55bdcef02b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.485+0000 7f73a6c65b80 1 -- 172.21.15.119:0/3189927516 shutdown_connections 2023-12-07T10:23:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.485+0000 7f73a6c65b80 1 -- 172.21.15.119:0/3189927516 wait complete. 2023-12-07T10:23:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.486+0000 7f73a6c65b80 1 -- start start 2023-12-07T10:23:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.486+0000 7f73a6c65b80 1 Processor -- start 2023-12-07T10:23:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.486+0000 7f73a6c65b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdcef0c950 0x55bdcef18ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.486+0000 7f73a6c65b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdcef0d310 0x55bdcef16020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.486+0000 7f73a6c65b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdcef10da0 0x55bdcef16760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.486+0000 7f73a6c65b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcee5c950 con 0x55bdcef0c950 2023-12-07T10:23:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.486+0000 7f73a6c65b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcee16500 con 0x55bdcef0d310 2023-12-07T10:23:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.486+0000 7f73a6c65b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcee161a0 con 0x55bdcef10da0 2023-12-07T10:23:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.486+0000 7f73a6c65b80 10 client.0 _ll_register_callbacks cb 0x55bdcedfd8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.487+0000 7f739119b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdcef0c950 0x55bdcef18ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.487+0000 7f739119b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdcef0c950 0x55bdcef18ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35636/0 (socket says 172.21.15.119:35636) 2023-12-07T10:23:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.487+0000 7f739119b700 1 -- 172.21.15.119:0/3034258550 learned_addr learned my addr 172.21.15.119:0/3034258550 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.487+0000 7f739199c700 1 --2- 172.21.15.119:0/3034258550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdcef10da0 0x55bdcef16760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.487+0000 7f739099a700 1 --2- 172.21.15.119:0/3034258550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdcef0d310 0x55bdcef16020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.487+0000 7f739119b700 1 -- 172.21.15.119:0/3034258550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdcef10da0 msgr2=0x55bdcef16760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.487+0000 7f739119b700 1 --2- 172.21.15.119:0/3034258550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdcef10da0 0x55bdcef16760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.487+0000 7f739119b700 1 -- 172.21.15.119:0/3034258550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdcef0d310 msgr2=0x55bdcef16020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.487+0000 7f739119b700 1 --2- 172.21.15.119:0/3034258550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdcef0d310 0x55bdcef16020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.487+0000 7f739119b700 1 -- 172.21.15.119:0/3034258550 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738c1a4600 con 0x55bdcef0c950 2023-12-07T10:23:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.487+0000 7f739099a700 1 --2- 172.21.15.119:0/3034258550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdcef0d310 0x55bdcef16020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:23:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.487+0000 7f739119b700 1 --2- 172.21.15.119:0/3034258550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdcef0c950 0x55bdcef18ca0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f73880a2700 tx=0x7f73880965e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102077f546c6a65 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.487+0000 7f7374ff9700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:23:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.487+0000 7f7374ff9700 1 -- 172.21.15.119:0/3034258550 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73880b3070 con 0x55bdcef0c950 2023-12-07T10:23:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.488+0000 7f73a6c65b80 1 -- 172.21.15.119:0/3034258550 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdceefb6f0 con 0x55bdcef0c950 2023-12-07T10:23:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.487+0000 7f7374ff9700 1 -- 172.21.15.119:0/3034258550 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f7388099d90 con 0x55bdcef0c950 2023-12-07T10:23:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.488+0000 7f7374ff9700 1 -- 172.21.15.119:0/3034258550 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7388099f50 con 0x55bdcef0c950 2023-12-07T10:23:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.488+0000 7f73a6c65b80 10 client.16800 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.488+0000 7f73a6c65b80 1 -- 172.21.15.119:0/3034258550 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55bdcec85d80 con 0x55bdcef0c950 2023-12-07T10:23:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.488+0000 7f7374ff9700 1 -- 172.21.15.119:0/3034258550 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bdceefb6f0 con 0x55bdcef0c950 2023-12-07T10:23:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.488+0000 7f7374ff9700 1 -- 172.21.15.119:0/3034258550 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f73880a27a0 con 0x55bdcef0c950 2023-12-07T10:23:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.488+0000 7f73a6c65b80 10 client.16800 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.488+0000 7f73a6c65b80 10 client.16800 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:23:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.488+0000 7f73a6c65b80 20 client.16800 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.488+0000 7f73a6c65b80 1 --2- 172.21.15.119:0/3034258550 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55bdcef25ee0 0x55bdcef28300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.488+0000 7f73a6c65b80 4 client.16800 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:23:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.488+0000 7f73a6c65b80 1 -- 172.21.15.119:0/3034258550 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bdceed7740 con 0x55bdcef25ee0 2023-12-07T10:23:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.489+0000 7f739099a700 1 --2- 172.21.15.119:0/3034258550 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55bdcef25ee0 0x55bdcef28300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.489+0000 7f7374ff9700 10 client.16800 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.489+0000 7f739099a700 1 --2- 172.21.15.119:0/3034258550 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55bdcef25ee0 0x55bdcef28300 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52443a553193b68d server_cookie=cd96ef4b2a17f00d in_seq=0 out_seq=0 2023-12-07T10:23:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.490+0000 7f7374ff9700 1 -- 172.21.15.119:0/3034258550 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bdceed7740 con 0x55bdcef25ee0 2023-12-07T10:23:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.490+0000 7f7374ff9700 10 client.16800 handle_command_reply: tid=0 2023-12-07T10:23:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.490+0000 7f73a6c65b80 1 client.16800 shutdown 2023-12-07T10:23:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.490+0000 7f73a6c65b80 1 -- 172.21.15.119:0/3034258550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdcef0c950 msgr2=0x55bdcef18ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.490+0000 7f73a6c65b80 1 --2- 172.21.15.119:0/3034258550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdcef0c950 0x55bdcef18ca0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f73880a2700 tx=0x7f73880965e0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.490+0000 7f73a6c65b80 1 -- 172.21.15.119:0/3034258550 shutdown_connections 2023-12-07T10:23:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.490+0000 7f73a6c65b80 1 --2- 172.21.15.119:0/3034258550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdcef10da0 0x55bdcef16760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.490+0000 7f73a6c65b80 1 --2- 172.21.15.119:0/3034258550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdcef0c950 0x55bdcef18ca0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.490+0000 7f73a6c65b80 1 --2- 172.21.15.119:0/3034258550 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55bdcef25ee0 0x55bdcef28300 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.490+0000 7f73a6c65b80 1 --2- 172.21.15.119:0/3034258550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdcef0d310 0x55bdcef16020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.490+0000 7f7374ff9700 0 client.16800 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.490+0000 7f73a6c65b80 1 -- 172.21.15.119:0/3034258550 >> 172.21.15.119:0/3034258550 conn(0x55bdcee37300 msgr2=0x55bdceefd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.490+0000 7f73a6c65b80 1 -- 172.21.15.119:0/3034258550 shutdown_connections 2023-12-07T10:23:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.490+0000 7f73a6c65b80 1 -- 172.21.15.119:0/3034258550 wait complete. 2023-12-07T10:23:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.490+0000 7f73a6c65b80 20 client.16800 trim_cache size 0 max 16384 2023-12-07T10:23:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.506+0000 7f73a6c65b80 1 Processor -- start 2023-12-07T10:23:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.506+0000 7f73a6c65b80 1 -- start start 2023-12-07T10:23:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.506+0000 7f73a6c65b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdcef0c950 0x55bdcef0e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.506+0000 7f73a6c65b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdcef0d310 0x55bdcef13a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.506+0000 7f73a6c65b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdcef10da0 0x55bdcef1c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.506+0000 7f73a6c65b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcecdb8c0 con 0x55bdcef0d310 2023-12-07T10:23:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.506+0000 7f73a6c65b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcef145c0 con 0x55bdcef0c950 2023-12-07T10:23:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.506+0000 7f73a6c65b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcef0fd10 con 0x55bdcef10da0 2023-12-07T10:23:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.506+0000 7f739099a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdcef0d310 0x55bdcef13a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.506+0000 7f739099a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdcef0d310 0x55bdcef13a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35646/0 (socket says 172.21.15.119:35646) 2023-12-07T10:23:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.506+0000 7f739099a700 1 -- 172.21.15.119:0/2638227635 learned_addr learned my addr 172.21.15.119:0/2638227635 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.506+0000 7f739199c700 1 --2- 172.21.15.119:0/2638227635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdcef10da0 0x55bdcef1c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.506+0000 7f739119b700 1 --2- 172.21.15.119:0/2638227635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdcef0c950 0x55bdcef0e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.507+0000 7f739099a700 1 -- 172.21.15.119:0/2638227635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdcef10da0 msgr2=0x55bdcef1c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.507+0000 7f739099a700 1 --2- 172.21.15.119:0/2638227635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdcef10da0 0x55bdcef1c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.507+0000 7f739099a700 1 -- 172.21.15.119:0/2638227635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdcef0c950 msgr2=0x55bdcef0e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.507+0000 7f739099a700 1 --2- 172.21.15.119:0/2638227635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdcef0c950 0x55bdcef0e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.507+0000 7f739099a700 1 -- 172.21.15.119:0/2638227635 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdcec6ec00 con 0x55bdcef0d310 2023-12-07T10:23:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.507+0000 7f739099a700 1 --2- 172.21.15.119:0/2638227635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdcef0d310 0x55bdcef13a10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f737c005850 tx=0x7f737c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0fc091961add44 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.507+0000 7f7377fff700 1 -- 172.21.15.119:0/2638227635 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c00bdf0 con 0x55bdcef0d310 2023-12-07T10:23:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.507+0000 7f7377fff700 1 -- 172.21.15.119:0/2638227635 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f737c02a040 con 0x55bdcef0d310 2023-12-07T10:23:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.507+0000 7f7377fff700 1 -- 172.21.15.119:0/2638227635 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c032460 con 0x55bdcef0d310 2023-12-07T10:23:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.508+0000 7f73a6c65b80 1 -- 172.21.15.119:0/2638227635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdcef0d310 msgr2=0x55bdcef13a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.508+0000 7f73a6c65b80 1 --2- 172.21.15.119:0/2638227635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdcef0d310 0x55bdcef13a10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f737c005850 tx=0x7f737c020000 comp rx=0 tx=0).stop 2023-12-07T10:23:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.508+0000 7f73a6c65b80 1 -- 172.21.15.119:0/2638227635 shutdown_connections 2023-12-07T10:23:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.508+0000 7f73a6c65b80 1 --2- 172.21.15.119:0/2638227635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdcef10da0 0x55bdcef1c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.508+0000 7f73a6c65b80 1 --2- 172.21.15.119:0/2638227635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdcef0c950 0x55bdcef0e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.508+0000 7f73a6c65b80 1 --2- 172.21.15.119:0/2638227635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdcef0d310 0x55bdcef13a10 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.508+0000 7f73a6c65b80 1 -- 172.21.15.119:0/2638227635 >> 172.21.15.119:0/2638227635 conn(0x55bdcee37300 msgr2=0x55bdcef100f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.508+0000 7f73a6c65b80 1 -- 172.21.15.119:0/2638227635 shutdown_connections 2023-12-07T10:23:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.508+0000 7f73a6c65b80 1 -- 172.21.15.119:0/2638227635 wait complete. 2023-12-07T10:23:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.508+0000 7f73a6c65b80 1 -- start start 2023-12-07T10:23:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.508+0000 7f73a6c65b80 1 Processor -- start 2023-12-07T10:23:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.509+0000 7f73a6c65b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdcef0c950 0x55bdcef1cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.509+0000 7f73a6c65b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdcef0d310 0x55bdcef18850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.509+0000 7f73a6c65b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdcef10da0 0x55bdcef18db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.509+0000 7f73a6c65b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcef19510 con 0x55bdcef0d310 2023-12-07T10:23:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.509+0000 7f73a6c65b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcef19680 con 0x55bdcef0c950 2023-12-07T10:23:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.509+0000 7f73a6c65b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcef7b390 con 0x55bdcef10da0 2023-12-07T10:23:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.509+0000 7f73a6c65b80 10 client.0 _ll_register_callbacks cb 0x55bdcedfd8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.509+0000 7f739119b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdcef0c950 0x55bdcef1cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.509+0000 7f739099a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdcef0d310 0x55bdcef18850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.509+0000 7f739119b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdcef0c950 0x55bdcef1cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:50540/0 (socket says 172.21.15.119:50540) 2023-12-07T10:23:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.509+0000 7f739119b700 1 -- 172.21.15.119:0/3413513330 learned_addr learned my addr 172.21.15.119:0/3413513330 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.509+0000 7f739199c700 1 --2- 172.21.15.119:0/3413513330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdcef10da0 0x55bdcef18db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.509+0000 7f739119b700 1 -- 172.21.15.119:0/3413513330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdcef10da0 msgr2=0x55bdcef18db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.509+0000 7f739119b700 1 --2- 172.21.15.119:0/3413513330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdcef10da0 0x55bdcef18db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.509+0000 7f739119b700 1 -- 172.21.15.119:0/3413513330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdcef0d310 msgr2=0x55bdcef18850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.509+0000 7f739119b700 1 --2- 172.21.15.119:0/3413513330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdcef0d310 0x55bdcef18850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.509+0000 7f739119b700 1 -- 172.21.15.119:0/3413513330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738c10ec80 con 0x55bdcef0c950 2023-12-07T10:23:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.510+0000 7f739199c700 1 --2- 172.21.15.119:0/3413513330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdcef10da0 0x55bdcef18db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:23:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.510+0000 7f739119b700 1 --2- 172.21.15.119:0/3413513330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdcef0c950 0x55bdcef1cc60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f73880cb390 tx=0x7f73880ba040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=969e261cf73a416c server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.510+0000 7f7374ff9700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:23:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.510+0000 7f7374ff9700 1 -- 172.21.15.119:0/3413513330 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738809bbd0 con 0x55bdcef0c950 2023-12-07T10:23:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.510+0000 7f73a6c65b80 1 -- 172.21.15.119:0/3413513330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f73880a2c30 con 0x55bdcef0c950 2023-12-07T10:23:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.510+0000 7f7374ff9700 1 -- 172.21.15.119:0/3413513330 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f738809ab00 con 0x55bdcef0c950 2023-12-07T10:23:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.510+0000 7f7374ff9700 1 -- 172.21.15.119:0/3413513330 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f738809acc0 con 0x55bdcef0c950 2023-12-07T10:23:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.510+0000 7f73a6c65b80 10 client.26834 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.510+0000 7f73a6c65b80 1 -- 172.21.15.119:0/3413513330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55bdcecac1f0 con 0x55bdcef0c950 2023-12-07T10:23:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.510+0000 7f7374ff9700 1 -- 172.21.15.119:0/3413513330 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f738807f070 con 0x55bdcef0c950 2023-12-07T10:23:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.510+0000 7f7374ff9700 1 -- 172.21.15.119:0/3413513330 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f7388096690 con 0x55bdcef0c950 2023-12-07T10:23:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.511+0000 7f73a6c65b80 10 client.26834 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.511+0000 7f73a6c65b80 10 client.26834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:23:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.511+0000 7f73a6c65b80 20 client.26834 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.511+0000 7f73a6c65b80 1 --2- 172.21.15.119:0/3413513330 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55bdcef7c730 0x55bdcef7eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.511+0000 7f73a6c65b80 4 client.26834 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:23:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.511+0000 7f73a6c65b80 1 -- 172.21.15.119:0/3413513330 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bdcee22f70 con 0x55bdcef7c730 2023-12-07T10:23:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.511+0000 7f739099a700 1 --2- 172.21.15.119:0/3413513330 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55bdcef7c730 0x55bdcef7eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.511+0000 7f739099a700 1 --2- 172.21.15.119:0/3413513330 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55bdcef7c730 0x55bdcef7eb50 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61b37a74158c6dfe server_cookie=9b42d1b212a11e41 in_seq=0 out_seq=0 2023-12-07T10:23:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.511+0000 7f7374ff9700 10 client.26834 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.545+0000 7f7374ff9700 1 -- 172.21.15.119:0/3413513330 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bdcee22f70 con 0x55bdcef7c730 2023-12-07T10:23:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.545+0000 7f7374ff9700 10 client.26834 handle_command_reply: tid=0 2023-12-07T10:23:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.545+0000 7f73a6c65b80 1 client.26834 shutdown 2023-12-07T10:23:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.545+0000 7f73a6c65b80 1 -- 172.21.15.119:0/3413513330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdcef0c950 msgr2=0x55bdcef1cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.545+0000 7f73a6c65b80 1 --2- 172.21.15.119:0/3413513330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdcef0c950 0x55bdcef1cc60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f73880cb390 tx=0x7f73880ba040 comp rx=0 tx=0).stop 2023-12-07T10:23:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.545+0000 7f73a6c65b80 1 -- 172.21.15.119:0/3413513330 shutdown_connections 2023-12-07T10:23:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.545+0000 7f73a6c65b80 1 --2- 172.21.15.119:0/3413513330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdcef10da0 0x55bdcef18db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.545+0000 7f73a6c65b80 1 --2- 172.21.15.119:0/3413513330 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55bdcef7c730 0x55bdcef7eb50 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.545+0000 7f73a6c65b80 1 --2- 172.21.15.119:0/3413513330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55bdcef0c950 0x55bdcef1cc60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.545+0000 7f73a6c65b80 1 --2- 172.21.15.119:0/3413513330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bdcef0d310 0x55bdcef18850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.545+0000 7f7374ff9700 0 client.26834 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.545+0000 7f73a6c65b80 1 -- 172.21.15.119:0/3413513330 >> 172.21.15.119:0/3413513330 conn(0x55bdcee37300 msgr2=0x55bdcef13a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.546+0000 7f73a6c65b80 1 -- 172.21.15.119:0/3413513330 shutdown_connections 2023-12-07T10:23:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.546+0000 7f73a6c65b80 1 -- 172.21.15.119:0/3413513330 wait complete. 2023-12-07T10:23:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.546+0000 7f73a6c65b80 20 client.26834 trim_cache size 0 max 16384 2023-12-07T10:23:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:23:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:23:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:23:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:23:46.551 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.546+0000 7f73933ff700 1 -- 172.21.15.119:0/698105965 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f73780a1d00 msgr2=0x7f73780a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:46.551 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.546+0000 7f73933ff700 1 --2- 172.21.15.119:0/698105965 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f73780a1d00 0x7f73780a41c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f737c009990 tx=0x7f737c005fd0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.551 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.546+0000 7f73933ff700 1 -- 172.21.15.119:0/698105965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f738c101500 msgr2=0x7f738c19d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:46.551 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.546+0000 7f73933ff700 1 --2- 172.21.15.119:0/698105965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f738c101500 0x7f738c19d610 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f738800e9a0 tx=0x7f738800ecb0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.551 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.546+0000 7f73933ff700 1 -- 172.21.15.119:0/698105965 shutdown_connections 2023-12-07T10:23:46.551 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.546+0000 7f73933ff700 1 --2- 172.21.15.119:0/698105965 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f73780a1d00 0x7f73780a41c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.551 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.546+0000 7f73933ff700 1 --2- 172.21.15.119:0/698105965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f738c1075c0 0x7f738c197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.551 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.546+0000 7f73933ff700 1 --2- 172.21.15.119:0/698105965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f738c101500 0x7f738c19d610 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.551 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.546+0000 7f73933ff700 1 --2- 172.21.15.119:0/698105965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f738c101ec0 0x7f738c19db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:46.551 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.546+0000 7f73933ff700 1 -- 172.21.15.119:0/698105965 >> 172.21.15.119:0/698105965 conn(0x7f738c0fcfb0 msgr2=0x7f738c0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:46.551 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.547+0000 7f73933ff700 1 -- 172.21.15.119:0/698105965 shutdown_connections 2023-12-07T10:23:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:46.547+0000 7f73933ff700 1 -- 172.21.15.119:0/698105965 wait complete. 2023-12-07T10:23:46.552 INFO:tasks.workunit.client.0.smithi119.stdout: 1 188009 8.68 MB/sec warmup 119 sec latency 2434.480 ms 2023-12-07T10:23:48.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:47 smithi159 ceph-mon[134726]: pgmap v104: 129 pgs: 129 active+clean; 109 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 328 op/s 2023-12-07T10:23:48.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:47 smithi153 ceph-mon[133625]: pgmap v104: 129 pgs: 129 active+clean; 109 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 328 op/s 2023-12-07T10:23:48.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:47 smithi119 ceph-mon[97908]: pgmap v104: 129 pgs: 129 active+clean; 109 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 328 op/s 2023-12-07T10:23:48.550 INFO:tasks.workunit.client.0.smithi119.stdout: 1 188141 0.33 MB/sec execute 1 sec latency 3868.867 ms 2023-12-07T10:23:49.550 INFO:tasks.workunit.client.0.smithi119.stdout: 1 188141 0.16 MB/sec execute 2 sec latency 1470.382 ms 2023-12-07T10:23:50.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:49 smithi159 ceph-mon[134726]: pgmap v105: 129 pgs: 129 active+clean; 109 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 1.6 MiB/s wr, 193 op/s 2023-12-07T10:23:50.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:49 smithi153 ceph-mon[133625]: pgmap v105: 129 pgs: 129 active+clean; 109 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 1.6 MiB/s wr, 193 op/s 2023-12-07T10:23:50.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:49 smithi119 ceph-mon[97908]: pgmap v105: 129 pgs: 129 active+clean; 109 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 1.6 MiB/s wr, 193 op/s 2023-12-07T10:23:50.550 INFO:tasks.workunit.client.0.smithi119.stdout: 1 188141 0.11 MB/sec execute 3 sec latency 2470.493 ms 2023-12-07T10:23:51.550 INFO:tasks.workunit.client.0.smithi119.stdout: 1 188141 0.08 MB/sec execute 4 sec latency 3470.625 ms 2023-12-07T10:23:52.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:51 smithi119 ceph-mon[97908]: pgmap v106: 129 pgs: 129 active+clean; 114 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 2.3 MiB/s wr, 232 op/s 2023-12-07T10:23:52.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:51 smithi159 ceph-mon[134726]: pgmap v106: 129 pgs: 129 active+clean; 114 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 2.3 MiB/s wr, 232 op/s 2023-12-07T10:23:52.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:51 smithi153 ceph-mon[133625]: pgmap v106: 129 pgs: 129 active+clean; 114 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 2.3 MiB/s wr, 232 op/s 2023-12-07T10:23:52.551 INFO:tasks.workunit.client.0.smithi119.stdout: 1 188141 0.07 MB/sec execute 5 sec latency 4470.734 ms 2023-12-07T10:23:53.550 INFO:tasks.workunit.client.0.smithi119.stdout: 1 188473 0.27 MB/sec execute 6 sec latency 4926.586 ms 2023-12-07T10:23:54.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:53 smithi159 ceph-mon[134726]: pgmap v107: 129 pgs: 129 active+clean; 114 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 1.7 MiB/s wr, 131 op/s 2023-12-07T10:23:54.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:53 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:23:54.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:53 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:23:54.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:53 smithi153 ceph-mon[133625]: pgmap v107: 129 pgs: 129 active+clean; 114 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 1.7 MiB/s wr, 131 op/s 2023-12-07T10:23:54.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:53 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:23:54.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:53 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:23:54.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:53 smithi119 ceph-mon[97908]: pgmap v107: 129 pgs: 129 active+clean; 114 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 1.7 MiB/s wr, 131 op/s 2023-12-07T10:23:54.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:53 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:23:54.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:53 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:23:54.550 INFO:tasks.workunit.client.0.smithi119.stdout: 1 190396 1.24 MB/sec execute 7 sec latency 40.650 ms 2023-12-07T10:23:55.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:54 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:23:55.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:54 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:23:55.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:54 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:23:55.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:54 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:23:55.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:54 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:23:55.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:54 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:23:55.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:54 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:23:55.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:54 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:23:55.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:54 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:23:55.551 INFO:tasks.workunit.client.0.smithi119.stdout: 1 191950 2.60 MB/sec execute 8 sec latency 15.068 ms 2023-12-07T10:23:56.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:55 smithi159 ceph-mon[134726]: pgmap v108: 129 pgs: 129 active+clean; 114 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.7 MiB/s wr, 128 op/s 2023-12-07T10:23:56.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:55 smithi153 ceph-mon[133625]: pgmap v108: 129 pgs: 129 active+clean; 114 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.7 MiB/s wr, 128 op/s 2023-12-07T10:23:56.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:55 smithi119 ceph-mon[97908]: pgmap v108: 129 pgs: 129 active+clean; 114 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.7 MiB/s wr, 128 op/s 2023-12-07T10:23:56.550 INFO:tasks.workunit.client.0.smithi119.stdout: 1 193807 2.96 MB/sec execute 9 sec latency 64.842 ms 2023-12-07T10:23:57.550 INFO:tasks.workunit.client.0.smithi119.stdout: 1 195405 3.96 MB/sec execute 10 sec latency 23.112 ms 2023-12-07T10:23:58.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:57 smithi159 ceph-mon[134726]: pgmap v109: 129 pgs: 129 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 4.6 MiB/s wr, 280 op/s 2023-12-07T10:23:58.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:57 smithi153 ceph-mon[133625]: pgmap v109: 129 pgs: 129 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 4.6 MiB/s wr, 280 op/s 2023-12-07T10:23:58.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:57 smithi119 ceph-mon[97908]: pgmap v109: 129 pgs: 129 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 4.6 MiB/s wr, 280 op/s 2023-12-07T10:23:58.550 INFO:tasks.workunit.client.0.smithi119.stdout: 1 197157 4.14 MB/sec execute 11 sec latency 12.719 ms 2023-12-07T10:23:59.550 INFO:tasks.workunit.client.0.smithi119.stdout: 1 198983 4.92 MB/sec execute 12 sec latency 11.003 ms 2023-12-07T10:23:59.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.706+0000 7f849f7ff700 1 -- 172.21.15.119:0/3319252233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8498068ce0 msgr2=0x7f84980690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.706+0000 7f849f7ff700 1 --2- 172.21.15.119:0/3319252233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8498068ce0 0x7f84980690c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8494009990 tx=0x7f8494009ca0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.707+0000 7f849f7ff700 1 -- 172.21.15.119:0/3319252233 shutdown_connections 2023-12-07T10:23:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.707+0000 7f849f7ff700 1 --2- 172.21.15.119:0/3319252233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8498106a80 0x7f849810e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.707+0000 7f849f7ff700 1 --2- 172.21.15.119:0/3319252233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8498068ce0 0x7f84980690c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.707+0000 7f849f7ff700 1 --2- 172.21.15.119:0/3319252233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84981060c0 0x7f8498106540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.707+0000 7f849f7ff700 1 -- 172.21.15.119:0/3319252233 >> 172.21.15.119:0/3319252233 conn(0x7f84980775b0 msgr2=0x7f84980779c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.707+0000 7f849f7ff700 1 -- 172.21.15.119:0/3319252233 shutdown_connections 2023-12-07T10:23:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.707+0000 7f849f7ff700 1 -- 172.21.15.119:0/3319252233 wait complete. 2023-12-07T10:23:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.708+0000 7f849f7ff700 1 Processor -- start 2023-12-07T10:23:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.708+0000 7f849f7ff700 1 -- start start 2023-12-07T10:23:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.709+0000 7f849f7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8498068ce0 0x7f849819f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.709+0000 7f849f7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84981060c0 0x7f849819ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.709+0000 7f849f7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8498106a80 0x7f8498199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.709+0000 7f849f7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849819a230 con 0x7f84981060c0 2023-12-07T10:23:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.709+0000 7f849f7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849819a3a0 con 0x7f8498106a80 2023-12-07T10:23:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.709+0000 7f849f7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849819a560 con 0x7f8498068ce0 2023-12-07T10:23:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.709+0000 7f849d59b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8498068ce0 0x7f849819f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.709+0000 7f849d59b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8498068ce0 0x7f849819f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:50482/0 (socket says 172.21.15.119:50482) 2023-12-07T10:23:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.709+0000 7f849d59b700 1 -- 172.21.15.119:0/1922609123 learned_addr learned my addr 172.21.15.119:0/1922609123 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.709+0000 7f849dd9c700 1 --2- 172.21.15.119:0/1922609123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8498106a80 0x7f8498199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.709+0000 7f849cd9a700 1 --2- 172.21.15.119:0/1922609123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84981060c0 0x7f849819ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.710+0000 7f849d59b700 1 -- 172.21.15.119:0/1922609123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8498106a80 msgr2=0x7f8498199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.710+0000 7f849d59b700 1 --2- 172.21.15.119:0/1922609123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8498106a80 0x7f8498199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.710+0000 7f849d59b700 1 -- 172.21.15.119:0/1922609123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84981060c0 msgr2=0x7f849819ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.710+0000 7f849d59b700 1 --2- 172.21.15.119:0/1922609123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84981060c0 0x7f849819ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.710+0000 7f849d59b700 1 -- 172.21.15.119:0/1922609123 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84940095f0 con 0x7f8498068ce0 2023-12-07T10:23:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.710+0000 7f849cd9a700 1 --2- 172.21.15.119:0/1922609123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84981060c0 0x7f849819ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:23:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.710+0000 7f849d59b700 1 --2- 172.21.15.119:0/1922609123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8498068ce0 0x7f849819f9c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8494000c00 tx=0x7f8494005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.710+0000 7f848e7fc700 1 -- 172.21.15.119:0/1922609123 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849401d070 con 0x7f8498068ce0 2023-12-07T10:23:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.710+0000 7f848e7fc700 1 -- 172.21.15.119:0/1922609123 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8494022470 con 0x7f8498068ce0 2023-12-07T10:23:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.710+0000 7f849f7ff700 1 -- 172.21.15.119:0/1922609123 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f849819a7e0 con 0x7f8498068ce0 2023-12-07T10:23:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.710+0000 7f848e7fc700 1 -- 172.21.15.119:0/1922609123 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8494017650 con 0x7f8498068ce0 2023-12-07T10:23:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.710+0000 7f849f7ff700 1 -- 172.21.15.119:0/1922609123 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84981a6940 con 0x7f8498068ce0 2023-12-07T10:23:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.714+0000 7f848e7fc700 1 -- 172.21.15.119:0/1922609123 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f84940225e0 con 0x7f8498068ce0 2023-12-07T10:23:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.716+0000 7f84b30a8b80 1 Processor -- start 2023-12-07T10:23:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.716+0000 7f848e7fc700 1 --2- 172.21.15.119:0/1922609123 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f84840a1c80 0x7f84840a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.716+0000 7f84b30a8b80 1 -- start start 2023-12-07T10:23:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.716+0000 7f848e7fc700 1 -- 172.21.15.119:0/1922609123 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f84940cf0b0 con 0x7f8498068ce0 2023-12-07T10:23:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.716+0000 7f84b30a8b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619c75d4a60 0x5619c75d4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.716+0000 7f84b30a8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c75d5380 0x5619c75cf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.716+0000 7f84b30a8b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619c75cfad0 0x5619c75cff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.716+0000 7f84b30a8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c7543760 con 0x5619c75d5380 2023-12-07T10:23:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.716+0000 7f84b30a8b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c752a740 con 0x5619c75cfad0 2023-12-07T10:23:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.716+0000 7f84b30a8b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c7528060 con 0x5619c75d4a60 2023-12-07T10:23:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.716+0000 7f849dd9c700 1 --2- 172.21.15.119:0/1922609123 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f84840a1c80 0x7f84840a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.716+0000 7f849d59b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619c75d4a60 0x5619c75d4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.717+0000 7f849d59b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619c75d4a60 0x5619c75d4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:50498/0 (socket says 172.21.15.119:50498) 2023-12-07T10:23:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.717+0000 7f849d59b700 1 -- 172.21.15.119:0/1867057376 learned_addr learned my addr 172.21.15.119:0/1867057376 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.717+0000 7f849cd9a700 1 --2- 172.21.15.119:0/1867057376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c75d5380 0x5619c75cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.717+0000 7f849dd9c700 1 --2- 172.21.15.119:0/1867057376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619c75cfad0 0x5619c75cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.717+0000 7f849d59b700 1 -- 172.21.15.119:0/1867057376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619c75cfad0 msgr2=0x5619c75cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.717+0000 7f849d59b700 1 --2- 172.21.15.119:0/1867057376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619c75cfad0 0x5619c75cff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.717+0000 7f849d59b700 1 -- 172.21.15.119:0/1867057376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c75d5380 msgr2=0x5619c75cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.717+0000 7f849d59b700 1 --2- 172.21.15.119:0/1867057376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c75d5380 0x5619c75cf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.717+0000 7f849d59b700 1 -- 172.21.15.119:0/1867057376 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619c73a05d0 con 0x5619c75d4a60 2023-12-07T10:23:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.717+0000 7f849dd9c700 1 --2- 172.21.15.119:0/1922609123 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f84840a1c80 0x7f84840a4140 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f84981a0650 tx=0x7f849000fc00 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.717+0000 7f849d59b700 1 --2- 172.21.15.119:0/1867057376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619c75d4a60 0x5619c75d4e40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f84940985a0 tx=0x7f849409b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24efdd716baa92b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.717+0000 7f84837fe700 1 -- 172.21.15.119:0/1867057376 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849409c230 con 0x5619c75d4a60 2023-12-07T10:23:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.717+0000 7f84837fe700 1 -- 172.21.15.119:0/1867057376 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f849409bb30 con 0x5619c75d4a60 2023-12-07T10:23:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.717+0000 7f84837fe700 1 -- 172.21.15.119:0/1867057376 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849409bff0 con 0x5619c75d4a60 2023-12-07T10:23:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.718+0000 7f84b30a8b80 1 -- 172.21.15.119:0/1867057376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619c75d4a60 msgr2=0x5619c75d4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.718+0000 7f84b30a8b80 1 --2- 172.21.15.119:0/1867057376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619c75d4a60 0x5619c75d4e40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f84940985a0 tx=0x7f849409b9d0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.718+0000 7f84b30a8b80 1 -- 172.21.15.119:0/1867057376 shutdown_connections 2023-12-07T10:23:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.718+0000 7f84b30a8b80 1 --2- 172.21.15.119:0/1867057376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619c75cfad0 0x5619c75cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.718+0000 7f84b30a8b80 1 --2- 172.21.15.119:0/1867057376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619c75d4a60 0x5619c75d4e40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.718+0000 7f84b30a8b80 1 --2- 172.21.15.119:0/1867057376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c75d5380 0x5619c75cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.718+0000 7f84b30a8b80 1 -- 172.21.15.119:0/1867057376 >> 172.21.15.119:0/1867057376 conn(0x5619c74fe300 msgr2=0x5619c75c9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.718+0000 7f84b30a8b80 1 -- 172.21.15.119:0/1867057376 shutdown_connections 2023-12-07T10:23:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.719+0000 7f84b30a8b80 1 -- 172.21.15.119:0/1867057376 wait complete. 2023-12-07T10:23:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.719+0000 7f84b30a8b80 1 -- start start 2023-12-07T10:23:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.719+0000 7f84b30a8b80 1 Processor -- start 2023-12-07T10:23:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.720+0000 7f84b30a8b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619c75cfad0 0x5619c75db300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.720+0000 7f84b30a8b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619c75d4a60 0x5619c75d8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.720+0000 7f84b30a8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c75d5380 0x5619c75d8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.720+0000 7f84b30a8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c7523950 con 0x5619c75d5380 2023-12-07T10:23:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.720+0000 7f84b30a8b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c74dd500 con 0x5619c75d4a60 2023-12-07T10:23:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.720+0000 7f84b30a8b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c74dd1a0 con 0x5619c75cfad0 2023-12-07T10:23:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.720+0000 7f84b30a8b80 10 client.0 _ll_register_callbacks cb 0x5619c74c48e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.720+0000 7f849cd9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619c75d4a60 0x5619c75d8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.721+0000 7f849d59b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619c75cfad0 0x5619c75db300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.721+0000 7f849cd9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619c75d4a60 0x5619c75d8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:41060/0 (socket says 172.21.15.119:41060) 2023-12-07T10:23:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.721+0000 7f849cd9a700 1 -- 172.21.15.119:0/3517326216 learned_addr learned my addr 172.21.15.119:0/3517326216 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.721+0000 7f849dd9c700 1 --2- 172.21.15.119:0/3517326216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c75d5380 0x5619c75d8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.721+0000 7f849cd9a700 1 -- 172.21.15.119:0/3517326216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619c75cfad0 msgr2=0x5619c75db300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.721+0000 7f849cd9a700 1 --2- 172.21.15.119:0/3517326216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619c75cfad0 0x5619c75db300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.721+0000 7f849cd9a700 1 -- 172.21.15.119:0/3517326216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c75d5380 msgr2=0x5619c75d8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.721+0000 7f849cd9a700 1 --2- 172.21.15.119:0/3517326216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c75d5380 0x5619c75d8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.721+0000 7f849cd9a700 1 -- 172.21.15.119:0/3517326216 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84981a6940 con 0x5619c75d4a60 2023-12-07T10:23:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.721+0000 7f849d59b700 1 --2- 172.21.15.119:0/3517326216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619c75cfad0 0x5619c75db300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:59.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.721+0000 7f849cd9a700 1 --2- 172.21.15.119:0/3517326216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619c75d4a60 0x5619c75d8640 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8488012700 tx=0x7f8488012a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89af78aa177a37ea server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.721+0000 7f8480ff9700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:23:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.721+0000 7f8480ff9700 1 -- 172.21.15.119:0/3517326216 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8488017070 con 0x5619c75d4a60 2023-12-07T10:23:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.722+0000 7f84b30a8b80 1 -- 172.21.15.119:0/3517326216 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5619c75c26f0 con 0x5619c75d4a60 2023-12-07T10:23:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.722+0000 7f8480ff9700 1 -- 172.21.15.119:0/3517326216 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f8488004320 con 0x5619c75d4a60 2023-12-07T10:23:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.722+0000 7f8480ff9700 1 -- 172.21.15.119:0/3517326216 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84880044e0 con 0x5619c75d4a60 2023-12-07T10:23:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.722+0000 7f84b30a8b80 10 client.26846 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.722+0000 7f84b30a8b80 1 -- 172.21.15.119:0/3517326216 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5619c734cd80 con 0x5619c75d4a60 2023-12-07T10:23:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.722+0000 7f8480ff9700 1 -- 172.21.15.119:0/3517326216 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5619c75c26f0 con 0x5619c75d4a60 2023-12-07T10:23:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.722+0000 7f8480ff9700 1 -- 172.21.15.119:0/3517326216 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f848800f7a0 con 0x5619c75d4a60 2023-12-07T10:23:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.722+0000 7f84b30a8b80 10 client.26846 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.722+0000 7f84b30a8b80 10 client.26846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:23:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.722+0000 7f84b30a8b80 20 client.26846 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.722+0000 7f84b30a8b80 1 --2- 172.21.15.119:0/3517326216 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5619c75e8b70 0x5619c75eaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.722+0000 7f84b30a8b80 4 client.26846 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:23:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.722+0000 7f84b30a8b80 1 -- 172.21.15.119:0/3517326216 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619c759e740 con 0x5619c75e8b70 2023-12-07T10:23:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.723+0000 7f849d59b700 1 --2- 172.21.15.119:0/3517326216 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5619c75e8b70 0x5619c75eaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.723+0000 7f8480ff9700 10 client.26846 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:59.728 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.723+0000 7f849d59b700 1 --2- 172.21.15.119:0/3517326216 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5619c75e8b70 0x5619c75eaf90 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a3235dc4415d10 server_cookie=2e991d51d24422cd in_seq=0 out_seq=0 2023-12-07T10:23:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.724+0000 7f8480ff9700 1 -- 172.21.15.119:0/3517326216 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5619c759e740 con 0x5619c75e8b70 2023-12-07T10:23:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.724+0000 7f8480ff9700 10 client.26846 handle_command_reply: tid=0 2023-12-07T10:23:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.724+0000 7f84b30a8b80 1 client.26846 shutdown 2023-12-07T10:23:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.725+0000 7f84b30a8b80 1 -- 172.21.15.119:0/3517326216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619c75d4a60 msgr2=0x5619c75d8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.725+0000 7f84b30a8b80 1 --2- 172.21.15.119:0/3517326216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619c75d4a60 0x5619c75d8640 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8488012700 tx=0x7f8488012a10 comp rx=0 tx=0).stop 2023-12-07T10:23:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.725+0000 7f84b30a8b80 1 -- 172.21.15.119:0/3517326216 shutdown_connections 2023-12-07T10:23:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.725+0000 7f84b30a8b80 1 --2- 172.21.15.119:0/3517326216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c75d5380 0x5619c75d8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.725+0000 7f84b30a8b80 1 --2- 172.21.15.119:0/3517326216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619c75cfad0 0x5619c75db300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.725+0000 7f84b30a8b80 1 --2- 172.21.15.119:0/3517326216 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5619c75e8b70 0x5619c75eaf90 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.725+0000 7f84b30a8b80 1 --2- 172.21.15.119:0/3517326216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619c75d4a60 0x5619c75d8640 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.725+0000 7f8480ff9700 0 client.26846 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.725+0000 7f84b30a8b80 1 -- 172.21.15.119:0/3517326216 >> 172.21.15.119:0/3517326216 conn(0x5619c74fe300 msgr2=0x5619c75c4980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.725+0000 7f84b30a8b80 1 -- 172.21.15.119:0/3517326216 shutdown_connections 2023-12-07T10:23:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.725+0000 7f84b30a8b80 1 -- 172.21.15.119:0/3517326216 wait complete. 2023-12-07T10:23:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.726+0000 7f84b30a8b80 20 client.26846 trim_cache size 0 max 16384 2023-12-07T10:23:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.743+0000 7f84b30a8b80 1 Processor -- start 2023-12-07T10:23:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.743+0000 7f84b30a8b80 1 -- start start 2023-12-07T10:23:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.743+0000 7f84b30a8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c75cfad0 0x5619c75d8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.743+0000 7f84b30a8b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619c75d4a60 0x5619c75cf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.743+0000 7f84b30a8b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619c75d5380 0x5619c75c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.743+0000 7f84b30a8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c73a28c0 con 0x5619c75cfad0 2023-12-07T10:23:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.743+0000 7f84b30a8b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c75c26c0 con 0x5619c75d4a60 2023-12-07T10:23:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.743+0000 7f84b30a8b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c75e60e0 con 0x5619c75d5380 2023-12-07T10:23:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.743+0000 7f849cd9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619c75d4a60 0x5619c75cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.743+0000 7f849cd9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619c75d4a60 0x5619c75cf540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:41062/0 (socket says 172.21.15.119:41062) 2023-12-07T10:23:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.743+0000 7f849cd9a700 1 -- 172.21.15.119:0/3090016524 learned_addr learned my addr 172.21.15.119:0/3090016524 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.743+0000 7f849d59b700 1 --2- 172.21.15.119:0/3090016524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c75cfad0 0x5619c75d8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.744+0000 7f849dd9c700 1 --2- 172.21.15.119:0/3090016524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619c75d5380 0x5619c75c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.744+0000 7f849cd9a700 1 -- 172.21.15.119:0/3090016524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619c75d5380 msgr2=0x5619c75c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.744+0000 7f849cd9a700 1 --2- 172.21.15.119:0/3090016524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619c75d5380 0x5619c75c9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.744+0000 7f849cd9a700 1 -- 172.21.15.119:0/3090016524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c75cfad0 msgr2=0x5619c75d8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.744+0000 7f849cd9a700 1 --2- 172.21.15.119:0/3090016524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c75cfad0 0x5619c75d8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.744+0000 7f849cd9a700 1 -- 172.21.15.119:0/3090016524 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619c7335c00 con 0x5619c75d4a60 2023-12-07T10:23:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.744+0000 7f849dd9c700 1 --2- 172.21.15.119:0/3090016524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619c75d5380 0x5619c75c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:23:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.744+0000 7f849cd9a700 1 --2- 172.21.15.119:0/3090016524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619c75d4a60 0x5619c75cf540 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8488009630 tx=0x7f848801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a10c87ebf74651b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.744+0000 7f8483fff700 1 -- 172.21.15.119:0/3090016524 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8488017040 con 0x5619c75d4a60 2023-12-07T10:23:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.744+0000 7f8483fff700 1 -- 172.21.15.119:0/3090016524 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f848800f900 con 0x5619c75d4a60 2023-12-07T10:23:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.744+0000 7f8483fff700 1 -- 172.21.15.119:0/3090016524 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8488003a60 con 0x5619c75d4a60 2023-12-07T10:23:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.745+0000 7f84b30a8b80 1 -- 172.21.15.119:0/3090016524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619c75d4a60 msgr2=0x5619c75cf540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.745+0000 7f84b30a8b80 1 --2- 172.21.15.119:0/3090016524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619c75d4a60 0x5619c75cf540 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8488009630 tx=0x7f848801c480 comp rx=0 tx=0).stop 2023-12-07T10:23:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.745+0000 7f84b30a8b80 1 -- 172.21.15.119:0/3090016524 shutdown_connections 2023-12-07T10:23:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.745+0000 7f84b30a8b80 1 --2- 172.21.15.119:0/3090016524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619c75d5380 0x5619c75c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.745+0000 7f84b30a8b80 1 --2- 172.21.15.119:0/3090016524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c75cfad0 0x5619c75d8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.745+0000 7f84b30a8b80 1 --2- 172.21.15.119:0/3090016524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619c75d4a60 0x5619c75cf540 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.745+0000 7f84b30a8b80 1 -- 172.21.15.119:0/3090016524 >> 172.21.15.119:0/3090016524 conn(0x5619c74fe300 msgr2=0x5619c75c3960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.745+0000 7f84b30a8b80 1 -- 172.21.15.119:0/3090016524 shutdown_connections 2023-12-07T10:23:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.745+0000 7f84b30a8b80 1 -- 172.21.15.119:0/3090016524 wait complete. 2023-12-07T10:23:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.746+0000 7f84b30a8b80 1 -- start start 2023-12-07T10:23:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.746+0000 7f84b30a8b80 1 Processor -- start 2023-12-07T10:23:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.746+0000 7f84b30a8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c75cfad0 0x5619c75dfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.746+0000 7f84b30a8b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619c75d4a60 0x5619c75dd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.746+0000 7f84b30a8b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619c75d5380 0x5619c75ddf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.746+0000 7f84b30a8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c75de460 con 0x5619c75cfad0 2023-12-07T10:23:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.746+0000 7f84b30a8b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c75de5d0 con 0x5619c75d4a60 2023-12-07T10:23:59.750 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.746+0000 7f84b30a8b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c75de790 con 0x5619c75d5380 2023-12-07T10:23:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.747+0000 7f84b30a8b80 10 client.0 _ll_register_callbacks cb 0x5619c74c48e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:23:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.747+0000 7f849cd9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619c75d4a60 0x5619c75dd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.747+0000 7f849d59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c75cfad0 0x5619c75dfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.747+0000 7f849cd9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619c75d4a60 0x5619c75dd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:41070/0 (socket says 172.21.15.119:41070) 2023-12-07T10:23:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.747+0000 7f849cd9a700 1 -- 172.21.15.119:0/1549971056 learned_addr learned my addr 172.21.15.119:0/1549971056 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:23:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.747+0000 7f849dd9c700 1 --2- 172.21.15.119:0/1549971056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619c75d5380 0x5619c75ddf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.747+0000 7f849cd9a700 1 -- 172.21.15.119:0/1549971056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619c75d5380 msgr2=0x5619c75ddf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.747+0000 7f849cd9a700 1 --2- 172.21.15.119:0/1549971056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619c75d5380 0x5619c75ddf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.747+0000 7f849cd9a700 1 -- 172.21.15.119:0/1549971056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c75cfad0 msgr2=0x5619c75dfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:23:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.747+0000 7f849cd9a700 1 --2- 172.21.15.119:0/1549971056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c75cfad0 0x5619c75dfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.747+0000 7f849cd9a700 1 -- 172.21.15.119:0/1549971056 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619c734cd80 con 0x5619c75d4a60 2023-12-07T10:23:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.747+0000 7f849d59b700 1 --2- 172.21.15.119:0/1549971056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c75cfad0 0x5619c75dfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:23:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.747+0000 7f849dd9c700 1 --2- 172.21.15.119:0/1549971056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619c75d5380 0x5619c75ddf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:23:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.747+0000 7f849cd9a700 1 --2- 172.21.15.119:0/1549971056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619c75d4a60 0x5619c75dd9e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f84880094c0 tx=0x7f8488009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d198a02db01bcf5 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:23:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.747+0000 7f8480ff9700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:23:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.748+0000 7f8480ff9700 1 -- 172.21.15.119:0/1549971056 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8488005230 con 0x5619c75d4a60 2023-12-07T10:23:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.748+0000 7f84b30a8b80 1 -- 172.21.15.119:0/1549971056 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f849409bb30 con 0x5619c75d4a60 2023-12-07T10:23:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.748+0000 7f8480ff9700 1 -- 172.21.15.119:0/1549971056 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f8488005390 con 0x5619c75d4a60 2023-12-07T10:23:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.748+0000 7f8480ff9700 1 -- 172.21.15.119:0/1549971056 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8488005550 con 0x5619c75d4a60 2023-12-07T10:23:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.748+0000 7f84b30a8b80 10 client.26852 fetch_fsmap learned FSMap version 36 2023-12-07T10:23:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.748+0000 7f84b30a8b80 1 -- 172.21.15.119:0/1549971056 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5619c73731f0 con 0x5619c75d4a60 2023-12-07T10:23:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.748+0000 7f8480ff9700 1 -- 172.21.15.119:0/1549971056 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f849409bb30 con 0x5619c75d4a60 2023-12-07T10:23:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.748+0000 7f8480ff9700 1 -- 172.21.15.119:0/1549971056 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 36) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f848802f020 con 0x5619c75d4a60 2023-12-07T10:23:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.748+0000 7f84b30a8b80 10 client.26852 fetch_fsmap finished waiting for FSMap version 36 2023-12-07T10:23:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.748+0000 7f84b30a8b80 10 client.26852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:23:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.748+0000 7f84b30a8b80 20 client.26852 populate_metadata read hostname 'smithi119' 2023-12-07T10:23:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.748+0000 7f84b30a8b80 1 --2- 172.21.15.119:0/1549971056 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5619c763f250 0x5619c7641670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:23:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.748+0000 7f84b30a8b80 4 client.26852 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:23:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.748+0000 7f84b30a8b80 1 -- 172.21.15.119:0/1549971056 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5619c74e9f70 con 0x5619c763f250 2023-12-07T10:23:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.749+0000 7f849d59b700 1 --2- 172.21.15.119:0/1549971056 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5619c763f250 0x5619c7641670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:23:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.749+0000 7f8480ff9700 10 client.26852 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.749+0000 7f849d59b700 1 --2- 172.21.15.119:0/1549971056 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5619c763f250 0x5619c7641670 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61a2f6de1d70a291 server_cookie=1908532d7ef8aefb in_seq=0 out_seq=0 2023-12-07T10:23:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.799+0000 7f8480ff9700 1 -- 172.21.15.119:0/1549971056 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5619c74e9f70 con 0x5619c763f250 2023-12-07T10:23:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.799+0000 7f8480ff9700 10 client.26852 handle_command_reply: tid=0 2023-12-07T10:23:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.799+0000 7f84b30a8b80 1 client.26852 shutdown 2023-12-07T10:23:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.799+0000 7f84b30a8b80 1 -- 172.21.15.119:0/1549971056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619c75d4a60 msgr2=0x5619c75dd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.799+0000 7f84b30a8b80 1 --2- 172.21.15.119:0/1549971056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619c75d4a60 0x5619c75dd9e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f84880094c0 tx=0x7f8488009ed0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.799+0000 7f84b30a8b80 1 -- 172.21.15.119:0/1549971056 shutdown_connections 2023-12-07T10:23:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.799+0000 7f84b30a8b80 1 --2- 172.21.15.119:0/1549971056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619c75d5380 0x5619c75ddf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.799+0000 7f84b30a8b80 1 --2- 172.21.15.119:0/1549971056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c75cfad0 0x5619c75dfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.799+0000 7f84b30a8b80 1 --2- 172.21.15.119:0/1549971056 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5619c763f250 0x5619c7641670 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.799+0000 7f84b30a8b80 1 --2- 172.21.15.119:0/1549971056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5619c75d4a60 0x5619c75dd9e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.799+0000 7f84b30a8b80 1 -- 172.21.15.119:0/1549971056 >> 172.21.15.119:0/1549971056 conn(0x5619c74fe300 msgr2=0x5619c75c9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.799+0000 7f8480ff9700 0 client.26852 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:23:59.802 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.799+0000 7f84b30a8b80 1 -- 172.21.15.119:0/1549971056 shutdown_connections 2023-12-07T10:23:59.802 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.799+0000 7f84b30a8b80 1 -- 172.21.15.119:0/1549971056 wait complete. 2023-12-07T10:23:59.802 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.800+0000 7f84b30a8b80 20 client.26852 trim_cache size 0 max 16384 2023-12-07T10:23:59.802 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:23:59.802 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:23:59.802 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:23:59.802 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:23:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.800+0000 7f849f7ff700 1 -- 172.21.15.119:0/1922609123 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f84840a1c80 msgr2=0x7f84840a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.800+0000 7f849f7ff700 1 --2- 172.21.15.119:0/1922609123 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f84840a1c80 0x7f84840a4140 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f84981a0650 tx=0x7f849000fc00 comp rx=0 tx=0).stop 2023-12-07T10:23:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.800+0000 7f849f7ff700 1 -- 172.21.15.119:0/1922609123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8498068ce0 msgr2=0x7f849819f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:23:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.800+0000 7f849f7ff700 1 --2- 172.21.15.119:0/1922609123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8498068ce0 0x7f849819f9c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8494000c00 tx=0x7f8494005740 comp rx=0 tx=0).stop 2023-12-07T10:23:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.801+0000 7f849f7ff700 1 -- 172.21.15.119:0/1922609123 shutdown_connections 2023-12-07T10:23:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.801+0000 7f849f7ff700 1 --2- 172.21.15.119:0/1922609123 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f84840a1c80 0x7f84840a4140 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.801+0000 7f849f7ff700 1 --2- 172.21.15.119:0/1922609123 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8498106a80 0x7f8498199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.801+0000 7f849f7ff700 1 --2- 172.21.15.119:0/1922609123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8498068ce0 0x7f849819f9c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.801+0000 7f849f7ff700 1 --2- 172.21.15.119:0/1922609123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84981060c0 0x7f849819ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:23:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.801+0000 7f849f7ff700 1 -- 172.21.15.119:0/1922609123 >> 172.21.15.119:0/1922609123 conn(0x7f84980775b0 msgr2=0x7f849810c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:23:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.801+0000 7f849f7ff700 1 -- 172.21.15.119:0/1922609123 shutdown_connections 2023-12-07T10:23:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:23:59.801+0000 7f849f7ff700 1 -- 172.21.15.119:0/1922609123 wait complete. 2023-12-07T10:24:00.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:23:59 smithi159 ceph-mon[134726]: pgmap v110: 129 pgs: 129 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 3.6 MiB/s wr, 193 op/s 2023-12-07T10:24:00.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:23:59 smithi153 ceph-mon[133625]: pgmap v110: 129 pgs: 129 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 3.6 MiB/s wr, 193 op/s 2023-12-07T10:24:00.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:23:59 smithi119 ceph-mon[97908]: pgmap v110: 129 pgs: 129 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 3.6 MiB/s wr, 193 op/s 2023-12-07T10:24:00.550 INFO:tasks.workunit.client.0.smithi119.stdout: 1 200619 4.96 MB/sec execute 13 sec latency 33.762 ms 2023-12-07T10:24:01.550 INFO:tasks.workunit.client.0.smithi119.stdout: 1 201893 5.21 MB/sec execute 14 sec latency 62.333 ms 2023-12-07T10:24:02.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:01 smithi119 ceph-mon[97908]: pgmap v111: 129 pgs: 129 active+clean; 109 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 4.7 MiB/s wr, 288 op/s 2023-12-07T10:24:02.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:01 smithi159 ceph-mon[134726]: pgmap v111: 129 pgs: 129 active+clean; 109 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 4.7 MiB/s wr, 288 op/s 2023-12-07T10:24:02.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:01 smithi153 ceph-mon[133625]: pgmap v111: 129 pgs: 129 active+clean; 109 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 4.7 MiB/s wr, 288 op/s 2023-12-07T10:24:02.551 INFO:tasks.workunit.client.0.smithi119.stdout: 1 202645 5.21 MB/sec execute 15 sec latency 64.555 ms 2023-12-07T10:24:03.551 INFO:tasks.workunit.client.0.smithi119.stdout: 1 204630 5.33 MB/sec execute 16 sec latency 17.382 ms 2023-12-07T10:24:04.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:03 smithi159 ceph-mon[134726]: pgmap v112: 129 pgs: 129 active+clean; 111 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.3 MiB/s wr, 318 op/s 2023-12-07T10:24:04.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:03 smithi153 ceph-mon[133625]: pgmap v112: 129 pgs: 129 active+clean; 111 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.3 MiB/s wr, 318 op/s 2023-12-07T10:24:04.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:03 smithi119 ceph-mon[97908]: pgmap v112: 129 pgs: 129 active+clean; 111 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.3 MiB/s wr, 318 op/s 2023-12-07T10:24:04.551 INFO:tasks.workunit.client.0.smithi119.stdout: 1 206152 5.72 MB/sec execute 17 sec latency 11.994 ms 2023-12-07T10:24:05.551 INFO:tasks.workunit.client.0.smithi119.stdout: 1 208118 5.79 MB/sec execute 18 sec latency 11.365 ms 2023-12-07T10:24:06.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:05 smithi159 ceph-mon[134726]: pgmap v113: 129 pgs: 129 active+clean; 111 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.3 MiB/s wr, 316 op/s 2023-12-07T10:24:06.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:05 smithi153 ceph-mon[133625]: pgmap v113: 129 pgs: 129 active+clean; 111 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.3 MiB/s wr, 316 op/s 2023-12-07T10:24:06.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:05 smithi119 ceph-mon[97908]: pgmap v113: 129 pgs: 129 active+clean; 111 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.3 MiB/s wr, 316 op/s 2023-12-07T10:24:06.551 INFO:tasks.workunit.client.0.smithi119.stdout: 1 209666 6.12 MB/sec execute 19 sec latency 10.059 ms 2023-12-07T10:24:07.551 INFO:tasks.workunit.client.0.smithi119.stdout: 1 211603 6.16 MB/sec execute 20 sec latency 15.984 ms 2023-12-07T10:24:08.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:07 smithi159 ceph-mon[134726]: pgmap v114: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.9 MiB/s wr, 419 op/s 2023-12-07T10:24:08.385 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:07 smithi153 ceph-mon[133625]: pgmap v114: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.9 MiB/s wr, 419 op/s 2023-12-07T10:24:08.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:07 smithi119 ceph-mon[97908]: pgmap v114: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.9 MiB/s wr, 419 op/s 2023-12-07T10:24:08.551 INFO:tasks.workunit.client.0.smithi119.stdout: 1 213183 6.45 MB/sec execute 21 sec latency 18.355 ms 2023-12-07T10:24:08.870 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-07T10:24:09.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.099+0000 7fe95dfff700 1 -- 172.21.15.119:0/219803706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe958101560 msgr2=0x7fe9581019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:09.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.099+0000 7fe95dfff700 1 --2- 172.21.15.119:0/219803706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe958101560 0x7fe9581019e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0098f0 tx=0x7fe94c009c00 comp rx=0 tx=0).stop 2023-12-07T10:24:09.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.099+0000 7fe95dfff700 1 -- 172.21.15.119:0/219803706 shutdown_connections 2023-12-07T10:24:09.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.099+0000 7fe95dfff700 1 --2- 172.21.15.119:0/219803706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe958101f20 0x7fe95810e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.099+0000 7fe95dfff700 1 --2- 172.21.15.119:0/219803706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe958107620 0x7fe958107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.099+0000 7fe95dfff700 1 --2- 172.21.15.119:0/219803706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe958101560 0x7fe9581019e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.099+0000 7fe95dfff700 1 -- 172.21.15.119:0/219803706 >> 172.21.15.119:0/219803706 conn(0x7fe9580fcfd0 msgr2=0x7fe9580ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:09.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.099+0000 7fe95dfff700 1 -- 172.21.15.119:0/219803706 shutdown_connections 2023-12-07T10:24:09.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.099+0000 7fe95dfff700 1 -- 172.21.15.119:0/219803706 wait complete. 2023-12-07T10:24:09.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.100+0000 7fe95dfff700 1 Processor -- start 2023-12-07T10:24:09.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.100+0000 7fe95dfff700 1 -- start start 2023-12-07T10:24:09.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.100+0000 7fe95dfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe958101560 0x7fe95819f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.100+0000 7fe95dfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe958101f20 0x7fe95819fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.100+0000 7fe95dfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe958107620 0x7fe958199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.100+0000 7fe95dfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95819a090 con 0x7fe958107620 2023-12-07T10:24:09.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.100+0000 7fe95dfff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95819a200 con 0x7fe958101f20 2023-12-07T10:24:09.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.100+0000 7fe95dfff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95819a3c0 con 0x7fe958101560 2023-12-07T10:24:09.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.101+0000 7fe956ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe958101f20 0x7fe95819fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.101+0000 7fe956ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe958101f20 0x7fe95819fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:55842/0 (socket says 172.21.15.119:55842) 2023-12-07T10:24:09.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.101+0000 7fe956ffd700 1 -- 172.21.15.119:0/673806824 learned_addr learned my addr 172.21.15.119:0/673806824 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:09.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.101+0000 7fe9577fe700 1 --2- 172.21.15.119:0/673806824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe958101560 0x7fe95819f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.101+0000 7fe957fff700 1 --2- 172.21.15.119:0/673806824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe958107620 0x7fe958199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.101+0000 7fe956ffd700 1 -- 172.21.15.119:0/673806824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe958101560 msgr2=0x7fe95819f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:09.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.101+0000 7fe956ffd700 1 --2- 172.21.15.119:0/673806824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe958101560 0x7fe95819f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.101+0000 7fe956ffd700 1 -- 172.21.15.119:0/673806824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe958107620 msgr2=0x7fe958199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:09.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.101+0000 7fe956ffd700 1 --2- 172.21.15.119:0/673806824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe958107620 0x7fe958199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.101+0000 7fe956ffd700 1 -- 172.21.15.119:0/673806824 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94c009520 con 0x7fe958101f20 2023-12-07T10:24:09.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.101+0000 7fe956ffd700 1 --2- 172.21.15.119:0/673806824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe958101f20 0x7fe95819fe40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe94c005950 tx=0x7fe94c00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:09.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.102+0000 7fe954ff9700 1 -- 172.21.15.119:0/673806824 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c01c070 con 0x7fe958101f20 2023-12-07T10:24:09.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.102+0000 7fe95dfff700 1 -- 172.21.15.119:0/673806824 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95819a640 con 0x7fe958101f20 2023-12-07T10:24:09.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.102+0000 7fe954ff9700 1 -- 172.21.15.119:0/673806824 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe94c00bb20 con 0x7fe958101f20 2023-12-07T10:24:09.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.102+0000 7fe954ff9700 1 -- 172.21.15.119:0/673806824 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c0038e0 con 0x7fe958101f20 2023-12-07T10:24:09.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.102+0000 7fe95dfff700 1 -- 172.21.15.119:0/673806824 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9581a68f0 con 0x7fe958101f20 2023-12-07T10:24:09.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.105+0000 7fe954ff9700 1 -- 172.21.15.119:0/673806824 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fe94c020030 con 0x7fe958101f20 2023-12-07T10:24:09.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.106+0000 7fe97197cb80 1 Processor -- start 2023-12-07T10:24:09.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.106+0000 7fe97197cb80 1 -- start start 2023-12-07T10:24:09.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.107+0000 7fe97197cb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b1682db40 0x558b1682df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.107+0000 7fe97197cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b1682e460 0x558b16831c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.107+0000 7fe97197cb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b16832150 0x558b16832570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.107+0000 7fe97197cb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b167a17c0 con 0x558b16832150 2023-12-07T10:24:09.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.107+0000 7fe97197cb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b167887a0 con 0x558b1682db40 2023-12-07T10:24:09.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.107+0000 7fe97197cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b167860c0 con 0x558b1682e460 2023-12-07T10:24:09.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.107+0000 7fe954ff9700 1 --2- 172.21.15.119:0/673806824 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe9400a1c80 0x7fe9400a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.107+0000 7fe9577fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b1682e460 0x558b16831c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.107+0000 7fe957fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b1682db40 0x558b1682df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.107+0000 7fe9577fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b1682e460 0x558b16831c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55554/0 (socket says 172.21.15.119:55554) 2023-12-07T10:24:09.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.107+0000 7fe9577fe700 1 -- 172.21.15.119:0/365922370 learned_addr learned my addr 172.21.15.119:0/365922370 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:09.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.107+0000 7fe956ffd700 1 --2- 172.21.15.119:0/365922370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b16832150 0x558b16832570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.107+0000 7fe957fff700 1 --2- 172.21.15.119:0/673806824 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe9400a1c80 0x7fe9400a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.107+0000 7fe954ff9700 1 -- 172.21.15.119:0/673806824 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fe94c0ceb20 con 0x7fe958101f20 2023-12-07T10:24:09.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.107+0000 7fe9577fe700 1 -- 172.21.15.119:0/365922370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b1682db40 msgr2=0x558b1682df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:09.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.107+0000 7fe9577fe700 1 --2- 172.21.15.119:0/365922370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b1682db40 0x558b1682df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.107+0000 7fe9577fe700 1 -- 172.21.15.119:0/365922370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b16832150 msgr2=0x558b16832570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:09.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.107+0000 7fe9577fe700 1 --2- 172.21.15.119:0/365922370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b16832150 0x558b16832570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.107+0000 7fe9577fe700 1 -- 172.21.15.119:0/365922370 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b165fe630 con 0x558b1682e460 2023-12-07T10:24:09.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.107+0000 7fe956ffd700 1 --2- 172.21.15.119:0/365922370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b16832150 0x558b16832570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:24:09.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.108+0000 7fe9577fe700 1 --2- 172.21.15.119:0/365922370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b1682e460 0x558b16831c10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe948009d90 tx=0x7fe948011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c77294b3495fe5d server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:09.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.108+0000 7fe93dffb700 1 -- 172.21.15.119:0/365922370 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948019630 con 0x558b1682e460 2023-12-07T10:24:09.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.108+0000 7fe93dffb700 1 -- 172.21.15.119:0/365922370 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe948019c50 con 0x558b1682e460 2023-12-07T10:24:09.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.108+0000 7fe93dffb700 1 -- 172.21.15.119:0/365922370 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94800ec80 con 0x558b1682e460 2023-12-07T10:24:09.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.108+0000 7fe957fff700 1 --2- 172.21.15.119:0/673806824 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe9400a1c80 0x7fe9400a4140 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe950009ec0 tx=0x7fe95000b040 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:09.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.109+0000 7fe97197cb80 1 -- 172.21.15.119:0/365922370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b1682e460 msgr2=0x558b16831c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:09.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.109+0000 7fe97197cb80 1 --2- 172.21.15.119:0/365922370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b1682e460 0x558b16831c10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe948009d90 tx=0x7fe948011720 comp rx=0 tx=0).stop 2023-12-07T10:24:09.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.109+0000 7fe97197cb80 1 -- 172.21.15.119:0/365922370 shutdown_connections 2023-12-07T10:24:09.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.109+0000 7fe97197cb80 1 --2- 172.21.15.119:0/365922370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b16832150 0x558b16832570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.109+0000 7fe97197cb80 1 --2- 172.21.15.119:0/365922370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b1682db40 0x558b1682df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.109+0000 7fe97197cb80 1 --2- 172.21.15.119:0/365922370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b1682e460 0x558b16831c10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.109+0000 7fe97197cb80 1 -- 172.21.15.119:0/365922370 >> 172.21.15.119:0/365922370 conn(0x558b1675c360 msgr2=0x558b16828130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:09.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.109+0000 7fe97197cb80 1 -- 172.21.15.119:0/365922370 shutdown_connections 2023-12-07T10:24:09.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.109+0000 7fe97197cb80 1 -- 172.21.15.119:0/365922370 wait complete. 2023-12-07T10:24:09.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.110+0000 7fe97197cb80 1 -- start start 2023-12-07T10:24:09.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.110+0000 7fe97197cb80 1 Processor -- start 2023-12-07T10:24:09.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.110+0000 7fe97197cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b1682db40 0x558b16839940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.110+0000 7fe97197cb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b1682e460 0x558b16836d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.110+0000 7fe97197cb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b16832150 0x558b16837420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.110+0000 7fe97197cb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b167819b0 con 0x558b1682e460 2023-12-07T10:24:09.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.110+0000 7fe97197cb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b1673b560 con 0x558b16832150 2023-12-07T10:24:09.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.110+0000 7fe97197cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b1673b200 con 0x558b1682db40 2023-12-07T10:24:09.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.110+0000 7fe97197cb80 10 client.0 _ll_register_callbacks cb 0x558b16722940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:09.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.111+0000 7fe956ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b1682e460 0x558b16836d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.111+0000 7fe957fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b16832150 0x558b16837420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.111+0000 7fe957fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b16832150 0x558b16837420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:55856/0 (socket says 172.21.15.119:55856) 2023-12-07T10:24:09.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.111+0000 7fe9577fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b1682db40 0x558b16839940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.111+0000 7fe957fff700 1 -- 172.21.15.119:0/3278380416 learned_addr learned my addr 172.21.15.119:0/3278380416 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:09.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.111+0000 7fe9577fe700 1 -- 172.21.15.119:0/3278380416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b16832150 msgr2=0x558b16837420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:09.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.111+0000 7fe9577fe700 1 --2- 172.21.15.119:0/3278380416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b16832150 0x558b16837420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.111+0000 7fe9577fe700 1 -- 172.21.15.119:0/3278380416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b1682e460 msgr2=0x558b16836d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:09.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.111+0000 7fe9577fe700 1 --2- 172.21.15.119:0/3278380416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b1682e460 0x558b16836d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.111+0000 7fe9577fe700 1 -- 172.21.15.119:0/3278380416 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9480099c0 con 0x558b1682db40 2023-12-07T10:24:09.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.111+0000 7fe957fff700 1 --2- 172.21.15.119:0/3278380416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b16832150 0x558b16837420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:09.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.111+0000 7fe956ffd700 1 --2- 172.21.15.119:0/3278380416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b1682e460 0x558b16836d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:09.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.111+0000 7fe9577fe700 1 --2- 172.21.15.119:0/3278380416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b1682db40 0x558b16839940 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe948011b20 tx=0x7fe948009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b461fdc1493024a7 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:09.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.112+0000 7fe936ffd700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:24:09.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.112+0000 7fe936ffd700 1 -- 172.21.15.119:0/3278380416 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948004220 con 0x558b1682db40 2023-12-07T10:24:09.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.112+0000 7fe97197cb80 1 -- 172.21.15.119:0/3278380416 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b16820b00 con 0x558b1682db40 2023-12-07T10:24:09.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.112+0000 7fe936ffd700 1 -- 172.21.15.119:0/3278380416 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fe948019dc0 con 0x558b1682db40 2023-12-07T10:24:09.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.112+0000 7fe936ffd700 1 -- 172.21.15.119:0/3278380416 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe948004380 con 0x558b1682db40 2023-12-07T10:24:09.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.112+0000 7fe97197cb80 10 client.26839 fetch_fsmap learned FSMap version 37 2023-12-07T10:24:09.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.112+0000 7fe97197cb80 1 -- 172.21.15.119:0/3278380416 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x558b165aade0 con 0x558b1682db40 2023-12-07T10:24:09.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.112+0000 7fe936ffd700 1 -- 172.21.15.119:0/3278380416 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558b16820b00 con 0x558b1682db40 2023-12-07T10:24:09.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.113+0000 7fe936ffd700 1 -- 172.21.15.119:0/3278380416 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 37) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fe948029020 con 0x558b1682db40 2023-12-07T10:24:09.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.113+0000 7fe97197cb80 10 client.26839 fetch_fsmap finished waiting for FSMap version 37 2023-12-07T10:24:09.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.113+0000 7fe97197cb80 10 client.26839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:24:09.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.113+0000 7fe97197cb80 20 client.26839 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:09.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.113+0000 7fe97197cb80 1 --2- 172.21.15.119:0/3278380416 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558b16847280 0x558b168496a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.113+0000 7fe97197cb80 4 client.26839 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:24:09.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.113+0000 7fe97197cb80 1 -- 172.21.15.119:0/3278380416 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b167fc7a0 con 0x558b16847280 2023-12-07T10:24:09.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.113+0000 7fe956ffd700 1 --2- 172.21.15.119:0/3278380416 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558b16847280 0x558b168496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.114+0000 7fe936ffd700 10 client.26839 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:09.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.114+0000 7fe956ffd700 1 --2- 172.21.15.119:0/3278380416 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558b16847280 0x558b168496a0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de41ed9bfba8745 server_cookie=98bd94102993a67d in_seq=0 out_seq=0 2023-12-07T10:24:09.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.114+0000 7fe936ffd700 1 -- 172.21.15.119:0/3278380416 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558b167fc7a0 con 0x558b16847280 2023-12-07T10:24:09.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.114+0000 7fe936ffd700 10 client.26839 handle_command_reply: tid=0 2023-12-07T10:24:09.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.114+0000 7fe97197cb80 1 client.26839 shutdown 2023-12-07T10:24:09.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.115+0000 7fe97197cb80 1 -- 172.21.15.119:0/3278380416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b1682db40 msgr2=0x558b16839940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:09.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.115+0000 7fe97197cb80 1 --2- 172.21.15.119:0/3278380416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b1682db40 0x558b16839940 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe948011b20 tx=0x7fe948009800 comp rx=0 tx=0).stop 2023-12-07T10:24:09.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.115+0000 7fe97197cb80 1 -- 172.21.15.119:0/3278380416 shutdown_connections 2023-12-07T10:24:09.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.115+0000 7fe97197cb80 1 --2- 172.21.15.119:0/3278380416 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558b16847280 0x558b168496a0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.115+0000 7fe97197cb80 1 --2- 172.21.15.119:0/3278380416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b16832150 0x558b16837420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.115+0000 7fe97197cb80 1 --2- 172.21.15.119:0/3278380416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b1682db40 0x558b16839940 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.115+0000 7fe936ffd700 0 client.26839 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:09.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.115+0000 7fe97197cb80 1 --2- 172.21.15.119:0/3278380416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b1682e460 0x558b16836d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.115+0000 7fe97197cb80 1 -- 172.21.15.119:0/3278380416 >> 172.21.15.119:0/3278380416 conn(0x558b1675c360 msgr2=0x558b16833620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:09.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.115+0000 7fe97197cb80 1 -- 172.21.15.119:0/3278380416 shutdown_connections 2023-12-07T10:24:09.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.115+0000 7fe97197cb80 1 -- 172.21.15.119:0/3278380416 wait complete. 2023-12-07T10:24:09.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.115+0000 7fe97197cb80 20 client.26839 trim_cache size 0 max 16384 2023-12-07T10:24:09.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.135+0000 7fe97197cb80 1 Processor -- start 2023-12-07T10:24:09.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.135+0000 7fe97197cb80 1 -- start start 2023-12-07T10:24:09.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.136+0000 7fe97197cb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b1682db40 0x558b1683e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.136+0000 7fe97197cb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b1682e460 0x558b1683e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.136+0000 7fe97197cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b16832150 0x558b16842e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.136+0000 7fe97197cb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b16600920 con 0x558b1682db40 2023-12-07T10:24:09.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.136+0000 7fe97197cb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b16835260 con 0x558b1682e460 2023-12-07T10:24:09.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.136+0000 7fe97197cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b16837d00 con 0x558b16832150 2023-12-07T10:24:09.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.136+0000 7fe957fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b16832150 0x558b16842e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.136+0000 7fe956ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b1682e460 0x558b1683e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.136+0000 7fe9577fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b1682db40 0x558b1683e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.136+0000 7fe957fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b16832150 0x558b16842e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55558/0 (socket says 172.21.15.119:55558) 2023-12-07T10:24:09.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.136+0000 7fe957fff700 1 -- 172.21.15.119:0/2524339613 learned_addr learned my addr 172.21.15.119:0/2524339613 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:09.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.136+0000 7fe957fff700 1 -- 172.21.15.119:0/2524339613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b1682e460 msgr2=0x558b1683e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:09.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.136+0000 7fe957fff700 1 --2- 172.21.15.119:0/2524339613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b1682e460 0x558b1683e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.136+0000 7fe957fff700 1 -- 172.21.15.119:0/2524339613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b1682db40 msgr2=0x558b1683e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:09.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.136+0000 7fe957fff700 1 --2- 172.21.15.119:0/2524339613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b1682db40 0x558b1683e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.136+0000 7fe957fff700 1 -- 172.21.15.119:0/2524339613 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b16593c60 con 0x558b16832150 2023-12-07T10:24:09.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.136+0000 7fe956ffd700 1 --2- 172.21.15.119:0/2524339613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b1682e460 0x558b1683e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:09.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.137+0000 7fe957fff700 1 --2- 172.21.15.119:0/2524339613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b16832150 0x558b16842e00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe950003c90 tx=0x7fe950011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db6437623352db81 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:09.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.137+0000 7fe93e7fc700 1 -- 172.21.15.119:0/2524339613 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe950021050 con 0x558b16832150 2023-12-07T10:24:09.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.137+0000 7fe93e7fc700 1 -- 172.21.15.119:0/2524339613 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe950022040 con 0x558b16832150 2023-12-07T10:24:09.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.137+0000 7fe93e7fc700 1 -- 172.21.15.119:0/2524339613 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95000e030 con 0x558b16832150 2023-12-07T10:24:09.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.138+0000 7fe97197cb80 1 -- 172.21.15.119:0/2524339613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b16832150 msgr2=0x558b16842e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:09.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.138+0000 7fe97197cb80 1 --2- 172.21.15.119:0/2524339613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b16832150 0x558b16842e00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe950003c90 tx=0x7fe950011000 comp rx=0 tx=0).stop 2023-12-07T10:24:09.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.138+0000 7fe97197cb80 1 -- 172.21.15.119:0/2524339613 shutdown_connections 2023-12-07T10:24:09.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.138+0000 7fe97197cb80 1 --2- 172.21.15.119:0/2524339613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b16832150 0x558b16842e00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.138+0000 7fe97197cb80 1 --2- 172.21.15.119:0/2524339613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b1682db40 0x558b1683e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.138+0000 7fe97197cb80 1 --2- 172.21.15.119:0/2524339613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b1682e460 0x558b1683e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.138+0000 7fe97197cb80 1 -- 172.21.15.119:0/2524339613 >> 172.21.15.119:0/2524339613 conn(0x558b1675c360 msgr2=0x558b16830370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:09.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.138+0000 7fe97197cb80 1 -- 172.21.15.119:0/2524339613 shutdown_connections 2023-12-07T10:24:09.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.138+0000 7fe97197cb80 1 -- 172.21.15.119:0/2524339613 wait complete. 2023-12-07T10:24:09.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.138+0000 7fe97197cb80 1 -- start start 2023-12-07T10:24:09.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.138+0000 7fe97197cb80 1 Processor -- start 2023-12-07T10:24:09.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.139+0000 7fe97197cb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b1682db40 0x558b1683c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.139+0000 7fe97197cb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b1682e460 0x558b1683ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.139+0000 7fe97197cb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b16832150 0x558b1689c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.139+0000 7fe97197cb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b1689ccc0 con 0x558b1682e460 2023-12-07T10:24:09.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.139+0000 7fe97197cb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b1689ce00 con 0x558b16832150 2023-12-07T10:24:09.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.139+0000 7fe97197cb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b1689cf40 con 0x558b1682db40 2023-12-07T10:24:09.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.139+0000 7fe97197cb80 10 client.0 _ll_register_callbacks cb 0x558b16722940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:09.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.139+0000 7fe9577fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b1682db40 0x558b1683c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.139+0000 7fe957fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b16832150 0x558b1689c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.139+0000 7fe956ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b1682e460 0x558b1683ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.139+0000 7fe957fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b16832150 0x558b1689c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:55878/0 (socket says 172.21.15.119:55878) 2023-12-07T10:24:09.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.139+0000 7fe957fff700 1 -- 172.21.15.119:0/3675441786 learned_addr learned my addr 172.21.15.119:0/3675441786 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:09.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.140+0000 7fe9577fe700 1 -- 172.21.15.119:0/3675441786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b16832150 msgr2=0x558b1689c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:09.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.140+0000 7fe9577fe700 1 --2- 172.21.15.119:0/3675441786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b16832150 0x558b1689c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.140+0000 7fe9577fe700 1 -- 172.21.15.119:0/3675441786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b1682e460 msgr2=0x558b1683ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:09.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.140+0000 7fe9577fe700 1 --2- 172.21.15.119:0/3675441786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b1682e460 0x558b1683ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.140+0000 7fe9577fe700 1 -- 172.21.15.119:0/3675441786 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9581a68f0 con 0x558b1682db40 2023-12-07T10:24:09.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.140+0000 7fe957fff700 1 --2- 172.21.15.119:0/3675441786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b16832150 0x558b1689c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:09.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.140+0000 7fe9577fe700 1 --2- 172.21.15.119:0/3675441786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b1682db40 0x558b1683c520 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe948006010 tx=0x7fe948009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=293db853012a6e98 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:09.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.140+0000 7fe936ffd700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:24:09.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.140+0000 7fe936ffd700 1 -- 172.21.15.119:0/3675441786 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948004450 con 0x558b1682db40 2023-12-07T10:24:09.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.140+0000 7fe936ffd700 1 -- 172.21.15.119:0/3675441786 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fe94800ee80 con 0x558b1682db40 2023-12-07T10:24:09.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.140+0000 7fe97197cb80 1 -- 172.21.15.119:0/3675441786 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe948019c50 con 0x558b1682db40 2023-12-07T10:24:09.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.140+0000 7fe936ffd700 1 -- 172.21.15.119:0/3675441786 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe94800eab0 con 0x558b1682db40 2023-12-07T10:24:09.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.140+0000 7fe97197cb80 10 client.26845 fetch_fsmap learned FSMap version 37 2023-12-07T10:24:09.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.140+0000 7fe97197cb80 1 -- 172.21.15.119:0/3675441786 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7fe950003a00 con 0x558b1682db40 2023-12-07T10:24:09.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.141+0000 7fe936ffd700 1 -- 172.21.15.119:0/3675441786 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe948019c50 con 0x558b1682db40 2023-12-07T10:24:09.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.141+0000 7fe936ffd700 1 -- 172.21.15.119:0/3675441786 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 37) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fe94803c080 con 0x558b1682db40 2023-12-07T10:24:09.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.141+0000 7fe97197cb80 10 client.26845 fetch_fsmap finished waiting for FSMap version 37 2023-12-07T10:24:09.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.141+0000 7fe97197cb80 10 client.26845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:24:09.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.141+0000 7fe97197cb80 20 client.26845 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:09.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.141+0000 7fe97197cb80 1 --2- 172.21.15.119:0/3675441786 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558b1689e340 0x558b168a0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.141+0000 7fe97197cb80 4 client.26845 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-12-07T10:24:09.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.141+0000 7fe97197cb80 1 -- 172.21.15.119:0/3675441786 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558b16747fd0 con 0x558b1689e340 2023-12-07T10:24:09.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.141+0000 7fe956ffd700 1 --2- 172.21.15.119:0/3675441786 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558b1689e340 0x558b168a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.142+0000 7fe956ffd700 1 --2- 172.21.15.119:0/3675441786 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558b1689e340 0x558b168a0760 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27a936850dd07d6d server_cookie=a5e5514a4ce514bf in_seq=0 out_seq=0 2023-12-07T10:24:09.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.142+0000 7fe936ffd700 10 client.26845 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:09.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.142+0000 7fe936ffd700 1 -- 172.21.15.119:0/3675441786 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558b16747fd0 con 0x558b1689e340 2023-12-07T10:24:09.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.142+0000 7fe936ffd700 10 client.26845 handle_command_reply: tid=0 2023-12-07T10:24:09.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.142+0000 7fe97197cb80 1 client.26845 shutdown 2023-12-07T10:24:09.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.142+0000 7fe97197cb80 1 -- 172.21.15.119:0/3675441786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b1682db40 msgr2=0x558b1683c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:09.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.142+0000 7fe97197cb80 1 --2- 172.21.15.119:0/3675441786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b1682db40 0x558b1683c520 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe948006010 tx=0x7fe948009310 comp rx=0 tx=0).stop 2023-12-07T10:24:09.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.142+0000 7fe97197cb80 1 -- 172.21.15.119:0/3675441786 shutdown_connections 2023-12-07T10:24:09.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.142+0000 7fe97197cb80 1 --2- 172.21.15.119:0/3675441786 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558b1689e340 0x558b168a0760 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.142+0000 7fe97197cb80 1 --2- 172.21.15.119:0/3675441786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558b16832150 0x558b1689c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.142+0000 7fe97197cb80 1 --2- 172.21.15.119:0/3675441786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b1682db40 0x558b1683c520 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.142+0000 7fe97197cb80 1 --2- 172.21.15.119:0/3675441786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558b1682e460 0x558b1683ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.142+0000 7fe936ffd700 0 client.26845 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:09.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.142+0000 7fe97197cb80 1 -- 172.21.15.119:0/3675441786 >> 172.21.15.119:0/3675441786 conn(0x558b1675c360 msgr2=0x558b16828030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:09.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.142+0000 7fe97197cb80 1 -- 172.21.15.119:0/3675441786 shutdown_connections 2023-12-07T10:24:09.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.142+0000 7fe97197cb80 1 -- 172.21.15.119:0/3675441786 wait complete. 2023-12-07T10:24:09.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.143+0000 7fe97197cb80 20 client.26845 trim_cache size 0 max 16384 2023-12-07T10:24:09.150 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:24:09.150 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "no active scrubs running", 2023-12-07T10:24:09.150 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": {} 2023-12-07T10:24:09.150 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:24:09.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.143+0000 7fe95dfff700 1 -- 172.21.15.119:0/673806824 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe9400a1c80 msgr2=0x7fe9400a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:09.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.143+0000 7fe95dfff700 1 --2- 172.21.15.119:0/673806824 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe9400a1c80 0x7fe9400a4140 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe950009ec0 tx=0x7fe95000b040 comp rx=0 tx=0).stop 2023-12-07T10:24:09.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.143+0000 7fe95dfff700 1 -- 172.21.15.119:0/673806824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe958101f20 msgr2=0x7fe95819fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:09.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.143+0000 7fe95dfff700 1 --2- 172.21.15.119:0/673806824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe958101f20 0x7fe95819fe40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe94c005950 tx=0x7fe94c00fa00 comp rx=0 tx=0).stop 2023-12-07T10:24:09.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.143+0000 7fe95dfff700 1 -- 172.21.15.119:0/673806824 shutdown_connections 2023-12-07T10:24:09.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.143+0000 7fe95dfff700 1 --2- 172.21.15.119:0/673806824 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe9400a1c80 0x7fe9400a4140 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.143+0000 7fe95dfff700 1 --2- 172.21.15.119:0/673806824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe958107620 0x7fe958199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.143+0000 7fe95dfff700 1 --2- 172.21.15.119:0/673806824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe958101560 0x7fe95819f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.143+0000 7fe95dfff700 1 --2- 172.21.15.119:0/673806824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe958101f20 0x7fe95819fe40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.143+0000 7fe95dfff700 1 -- 172.21.15.119:0/673806824 >> 172.21.15.119:0/673806824 conn(0x7fe9580fcfd0 msgr2=0x7fe9580fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:09.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.143+0000 7fe95dfff700 1 -- 172.21.15.119:0/673806824 shutdown_connections 2023-12-07T10:24:09.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.143+0000 7fe95dfff700 1 -- 172.21.15.119:0/673806824 wait complete. 2023-12-07T10:24:09.165 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-07T10:24:09.165 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:24:09.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.391+0000 7f455bbff700 1 -- 172.21.15.119:0/2802813181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4554107560 msgr2=0x7f4554107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:09.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.391+0000 7f455bbff700 1 --2- 172.21.15.119:0/2802813181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4554107560 0x7f4554107940 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f45500099c0 tx=0x7f4550009cd0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.391+0000 7f455bbff700 1 -- 172.21.15.119:0/2802813181 shutdown_connections 2023-12-07T10:24:09.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.391+0000 7f455bbff700 1 --2- 172.21.15.119:0/2802813181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4554069010 0x7f455410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.391+0000 7f455bbff700 1 --2- 172.21.15.119:0/2802813181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4554107560 0x7f4554107940 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.391+0000 7f455bbff700 1 --2- 172.21.15.119:0/2802813181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4554068650 0x7f4554068ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.391+0000 7f455bbff700 1 -- 172.21.15.119:0/2802813181 >> 172.21.15.119:0/2802813181 conn(0x7f4554077220 msgr2=0x7f4554077630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:09.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.391+0000 7f455bbff700 1 -- 172.21.15.119:0/2802813181 shutdown_connections 2023-12-07T10:24:09.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.391+0000 7f455bbff700 1 -- 172.21.15.119:0/2802813181 wait complete. 2023-12-07T10:24:09.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.393+0000 7f455bbff700 1 Processor -- start 2023-12-07T10:24:09.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.393+0000 7f455bbff700 1 -- start start 2023-12-07T10:24:09.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.393+0000 7f455bbff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4554068650 0x7f45541007c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.393+0000 7f455bbff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4554069010 0x7f4554100d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.393+0000 7f455bbff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4554107560 0x7f4554101240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.393+0000 7f455bbff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45541a5df0 con 0x7f4554068650 2023-12-07T10:24:09.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.394+0000 7f455bbff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45541a5f60 con 0x7f4554107560 2023-12-07T10:24:09.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.394+0000 7f455bbff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45541a6120 con 0x7f4554069010 2023-12-07T10:24:09.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.394+0000 7f455999b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4554068650 0x7f45541007c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.394+0000 7f455999b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4554068650 0x7f45541007c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36246/0 (socket says 172.21.15.119:36246) 2023-12-07T10:24:09.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.394+0000 7f455999b700 1 -- 172.21.15.119:0/2417932942 learned_addr learned my addr 172.21.15.119:0/2417932942 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:09.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.394+0000 7f455919a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4554069010 0x7f4554100d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.394+0000 7f455a19c700 1 --2- 172.21.15.119:0/2417932942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4554107560 0x7f4554101240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.394+0000 7f455919a700 1 -- 172.21.15.119:0/2417932942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4554107560 msgr2=0x7f4554101240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:09.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.394+0000 7f455919a700 1 --2- 172.21.15.119:0/2417932942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4554107560 0x7f4554101240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.394+0000 7f455919a700 1 -- 172.21.15.119:0/2417932942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4554068650 msgr2=0x7f45541007c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:09.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.394+0000 7f455919a700 1 --2- 172.21.15.119:0/2417932942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4554068650 0x7f45541007c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.394+0000 7f455919a700 1 -- 172.21.15.119:0/2417932942 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45500095f0 con 0x7f4554069010 2023-12-07T10:24:09.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.394+0000 7f455999b700 1 --2- 172.21.15.119:0/2417932942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4554068650 0x7f45541007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:09.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.395+0000 7f455919a700 1 --2- 172.21.15.119:0/2417932942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4554069010 0x7f4554100d00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f454400e9f0 tx=0x7f454400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:09.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.395+0000 7f454affd700 1 -- 172.21.15.119:0/2417932942 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454400c9c0 con 0x7f4554069010 2023-12-07T10:24:09.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.395+0000 7f454affd700 1 -- 172.21.15.119:0/2417932942 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4544010430 con 0x7f4554069010 2023-12-07T10:24:09.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.395+0000 7f455bbff700 1 -- 172.21.15.119:0/2417932942 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45541a6400 con 0x7f4554069010 2023-12-07T10:24:09.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.395+0000 7f454affd700 1 -- 172.21.15.119:0/2417932942 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4544018680 con 0x7f4554069010 2023-12-07T10:24:09.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.395+0000 7f455bbff700 1 -- 172.21.15.119:0/2417932942 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45541a6910 con 0x7f4554069010 2023-12-07T10:24:09.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.396+0000 7f455bbff700 1 -- 172.21.15.119:0/2417932942 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45541099e0 con 0x7f4554069010 2023-12-07T10:24:09.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.399+0000 7f454affd700 1 -- 172.21.15.119:0/2417932942 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f4544010860 con 0x7f4554069010 2023-12-07T10:24:09.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.400+0000 7f454affd700 1 --2- 172.21.15.119:0/2417932942 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f45400a1c80 0x7f45400a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.401+0000 7f454affd700 1 -- 172.21.15.119:0/2417932942 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4544014070 con 0x7f4554069010 2023-12-07T10:24:09.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.401+0000 7f455999b700 1 --2- 172.21.15.119:0/2417932942 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f45400a1c80 0x7f45400a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.401+0000 7f455999b700 1 --2- 172.21.15.119:0/2417932942 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f45400a1c80 0x7f45400a4140 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4550009990 tx=0x7f4550005ab0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:09.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.402+0000 7f454affd700 1 -- 172.21.15.119:0/2417932942 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4544096ba0 con 0x7f4554069010 2023-12-07T10:24:09.551 INFO:tasks.workunit.client.0.smithi119.stdout: 1 214735 6.41 MB/sec execute 22 sec latency 12.157 ms 2023-12-07T10:24:09.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.713+0000 7f455bbff700 1 -- 172.21.15.119:0/2417932942 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4554102c00 con 0x7f4554069010 2023-12-07T10:24:09.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.714+0000 7f454affd700 1 -- 172.21.15.119:0/2417932942 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+11399 (secure 0 0 0) 0x7f454409a1c0 con 0x7f4554069010 2023-12-07T10:24:09.715 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:24:09.716 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-07T10:17:41.278311+0000","modified":"2023-12-07T10:24:08.056171+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24517,"mds_3":24505,"mds_4":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14535":{"gid":14535,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6841/1791998028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1791998028},{"type":"v1","addr":"172.21.15.119:6841","nonce":1791998028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24505":{"gid":24505,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/2925503246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2925503246},{"type":"v1","addr":"172.21.15.153:6837","nonce":2925503246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6839/3984962900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3984962900},{"type":"v1","addr":"172.21.15.159:6839","nonce":3984962900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.119:6839/1920762602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1920762602},{"type":"v1","addr":"172.21.15.119:6839","nonce":1920762602}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/566825184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":566825184},{"type":"v1","addr":"172.21.15.153:6839","nonce":566825184}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-07T10:24:09.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.716+0000 7f455bbff700 1 -- 172.21.15.119:0/2417932942 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f45400a1c80 msgr2=0x7f45400a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:09.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.716+0000 7f455bbff700 1 --2- 172.21.15.119:0/2417932942 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f45400a1c80 0x7f45400a4140 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4550009990 tx=0x7f4550005ab0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.716+0000 7f455bbff700 1 -- 172.21.15.119:0/2417932942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4554069010 msgr2=0x7f4554100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:09.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.716+0000 7f455bbff700 1 --2- 172.21.15.119:0/2417932942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4554069010 0x7f4554100d00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f454400e9f0 tx=0x7f454400ed00 comp rx=0 tx=0).stop 2023-12-07T10:24:09.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.716+0000 7f455bbff700 1 -- 172.21.15.119:0/2417932942 shutdown_connections 2023-12-07T10:24:09.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.716+0000 7f455bbff700 1 --2- 172.21.15.119:0/2417932942 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f45400a1c80 0x7f45400a4140 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.716+0000 7f455bbff700 1 --2- 172.21.15.119:0/2417932942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4554107560 0x7f4554101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.716+0000 7f455bbff700 1 --2- 172.21.15.119:0/2417932942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4554068650 0x7f45541007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.716+0000 7f455bbff700 1 --2- 172.21.15.119:0/2417932942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4554069010 0x7f4554100d00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.716+0000 7f455bbff700 1 -- 172.21.15.119:0/2417932942 >> 172.21.15.119:0/2417932942 conn(0x7f4554077220 msgr2=0x7f45540ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:09.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.716+0000 7f455bbff700 1 -- 172.21.15.119:0/2417932942 shutdown_connections 2023-12-07T10:24:09.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.717+0000 7f455bbff700 1 -- 172.21.15.119:0/2417932942 wait complete. 2023-12-07T10:24:09.719 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 37 2023-12-07T10:24:09.739 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-12-07T10:24:09.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.958+0000 7fb6617ff700 1 -- 172.21.15.119:0/272132811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb65c100f40 msgr2=0x7fb65c101320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:09.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.958+0000 7fb6617ff700 1 --2- 172.21.15.119:0/272132811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb65c100f40 0x7fb65c101320 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0098f0 tx=0x7fb64c009c00 comp rx=0 tx=0).stop 2023-12-07T10:24:09.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.958+0000 7fb6617ff700 1 -- 172.21.15.119:0/272132811 shutdown_connections 2023-12-07T10:24:09.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.958+0000 7fb6617ff700 1 --2- 172.21.15.119:0/272132811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb65c10dfb0 0x7fb65c1103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.958+0000 7fb6617ff700 1 --2- 172.21.15.119:0/272132811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb65c100f40 0x7fb65c101320 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.958+0000 7fb6617ff700 1 --2- 172.21.15.119:0/272132811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb65c101860 0x7fb65c10da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.958+0000 7fb6617ff700 1 -- 172.21.15.119:0/272132811 >> 172.21.15.119:0/272132811 conn(0x7fb65c077610 msgr2=0x7fb65c077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:09.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.958+0000 7fb6617ff700 1 -- 172.21.15.119:0/272132811 shutdown_connections 2023-12-07T10:24:09.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.958+0000 7fb6617ff700 1 -- 172.21.15.119:0/272132811 wait complete. 2023-12-07T10:24:09.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.959+0000 7fb6617ff700 1 Processor -- start 2023-12-07T10:24:09.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.959+0000 7fb6617ff700 1 -- start start 2023-12-07T10:24:09.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.960+0000 7fb6617ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb65c100f40 0x7fb65c1026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.960+0000 7fb6617ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb65c101860 0x7fb65c102c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.960+0000 7fb6617ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb65c10dfb0 0x7fb65c103160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.960+0000 7fb6617ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c1aa2b0 con 0x7fb65c101860 2023-12-07T10:24:09.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.960+0000 7fb6617ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c1aa3f0 con 0x7fb65c100f40 2023-12-07T10:24:09.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.960+0000 7fb6617ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c1aa560 con 0x7fb65c10dfb0 2023-12-07T10:24:09.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.960+0000 7fb65affd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb65c100f40 0x7fb65c1026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.960+0000 7fb65affd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb65c100f40 0x7fb65c1026e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:55908/0 (socket says 172.21.15.119:55908) 2023-12-07T10:24:09.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.960+0000 7fb65affd700 1 -- 172.21.15.119:0/3917621002 learned_addr learned my addr 172.21.15.119:0/3917621002 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:09.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.960+0000 7fb65a7fc700 1 --2- 172.21.15.119:0/3917621002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb65c101860 0x7fb65c102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.960+0000 7fb65b7fe700 1 --2- 172.21.15.119:0/3917621002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb65c10dfb0 0x7fb65c103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.960+0000 7fb65affd700 1 -- 172.21.15.119:0/3917621002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb65c10dfb0 msgr2=0x7fb65c103160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:09.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.960+0000 7fb65affd700 1 --2- 172.21.15.119:0/3917621002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb65c10dfb0 0x7fb65c103160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.960+0000 7fb65affd700 1 -- 172.21.15.119:0/3917621002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb65c101860 msgr2=0x7fb65c102c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:09.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.960+0000 7fb65affd700 1 --2- 172.21.15.119:0/3917621002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb65c101860 0x7fb65c102c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.960+0000 7fb65affd700 1 -- 172.21.15.119:0/3917621002 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64c009520 con 0x7fb65c100f40 2023-12-07T10:24:09.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.961+0000 7fb65affd700 1 --2- 172.21.15.119:0/3917621002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb65c100f40 0x7fb65c1026e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb64c005950 tx=0x7fb64c00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:09.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.961+0000 7fb643fff700 1 -- 172.21.15.119:0/3917621002 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c01c070 con 0x7fb65c100f40 2023-12-07T10:24:09.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.961+0000 7fb643fff700 1 -- 172.21.15.119:0/3917621002 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb64c00bb20 con 0x7fb65c100f40 2023-12-07T10:24:09.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.961+0000 7fb643fff700 1 -- 172.21.15.119:0/3917621002 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c0038e0 con 0x7fb65c100f40 2023-12-07T10:24:09.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.961+0000 7fb6617ff700 1 -- 172.21.15.119:0/3917621002 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb65c1aa7e0 con 0x7fb65c100f40 2023-12-07T10:24:09.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.961+0000 7fb6617ff700 1 -- 172.21.15.119:0/3917621002 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65c1aabb0 con 0x7fb65c100f40 2023-12-07T10:24:09.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.964+0000 7fb643fff700 1 -- 172.21.15.119:0/3917621002 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fb64c020030 con 0x7fb65c100f40 2023-12-07T10:24:09.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.966+0000 7fb643fff700 1 --2- 172.21.15.119:0/3917621002 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb6440a1d00 0x7fb6440a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.966+0000 7fb67506fb80 1 Processor -- start 2023-12-07T10:24:09.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.966+0000 7fb643fff700 1 -- 172.21.15.119:0/3917621002 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fb64c0cea20 con 0x7fb65c100f40 2023-12-07T10:24:09.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.966+0000 7fb67506fb80 1 -- start start 2023-12-07T10:24:09.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.966+0000 7fb65a7fc700 1 --2- 172.21.15.119:0/3917621002 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb6440a1d00 0x7fb6440a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.966+0000 7fb67506fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28735fb40 0x55a28735ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.966+0000 7fb67506fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a287360460 0x55a287363c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.966+0000 7fb67506fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a287364150 0x55a287364570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.966+0000 7fb67506fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2872d37c0 con 0x55a287364150 2023-12-07T10:24:09.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.966+0000 7fb67506fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2872ba7a0 con 0x55a287360460 2023-12-07T10:24:09.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.966+0000 7fb67506fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2872b80c0 con 0x55a28735fb40 2023-12-07T10:24:09.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.967+0000 7fb65affd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28735fb40 0x55a28735ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.967+0000 7fb65a7fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a287360460 0x55a287363c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.967+0000 7fb65affd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28735fb40 0x55a28735ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55610/0 (socket says 172.21.15.119:55610) 2023-12-07T10:24:09.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.967+0000 7fb65affd700 1 -- 172.21.15.119:0/913391678 learned_addr learned my addr 172.21.15.119:0/913391678 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:09.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.967+0000 7fb65b7fe700 1 --2- 172.21.15.119:0/913391678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a287364150 0x55a287364570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.967+0000 7fb65a7fc700 1 --2- 172.21.15.119:0/3917621002 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb6440a1d00 0x7fb6440a41c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb65c103e10 tx=0x7fb650009310 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:09.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.967+0000 7fb65affd700 1 -- 172.21.15.119:0/913391678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a287360460 msgr2=0x55a287363c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:09.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.967+0000 7fb65affd700 1 --2- 172.21.15.119:0/913391678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a287360460 0x55a287363c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.967+0000 7fb65affd700 1 -- 172.21.15.119:0/913391678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a287364150 msgr2=0x55a287364570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:09.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.967+0000 7fb65affd700 1 --2- 172.21.15.119:0/913391678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a287364150 0x55a287364570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.967+0000 7fb65affd700 1 -- 172.21.15.119:0/913391678 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a287130630 con 0x55a28735fb40 2023-12-07T10:24:09.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.967+0000 7fb65b7fe700 1 --2- 172.21.15.119:0/913391678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a287364150 0x55a287364570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:24:09.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.967+0000 7fb65affd700 1 --2- 172.21.15.119:0/913391678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28735fb40 0x55a28735ff20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb64c09b490 tx=0x7fb64c0054e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fada093295bf9531 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:09.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.967+0000 7fb6417fa700 1 -- 172.21.15.119:0/913391678 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c09b500 con 0x55a28735fb40 2023-12-07T10:24:09.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.967+0000 7fb6417fa700 1 -- 172.21.15.119:0/913391678 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb64c09f650 con 0x55a28735fb40 2023-12-07T10:24:09.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.967+0000 7fb6417fa700 1 -- 172.21.15.119:0/913391678 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c097c10 con 0x55a28735fb40 2023-12-07T10:24:09.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.968+0000 7fb67506fb80 1 -- 172.21.15.119:0/913391678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28735fb40 msgr2=0x55a28735ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:09.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.968+0000 7fb67506fb80 1 --2- 172.21.15.119:0/913391678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28735fb40 0x55a28735ff20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb64c09b490 tx=0x7fb64c0054e0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.968+0000 7fb67506fb80 1 -- 172.21.15.119:0/913391678 shutdown_connections 2023-12-07T10:24:09.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.968+0000 7fb67506fb80 1 --2- 172.21.15.119:0/913391678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a287364150 0x55a287364570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.968+0000 7fb67506fb80 1 --2- 172.21.15.119:0/913391678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28735fb40 0x55a28735ff20 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.968+0000 7fb67506fb80 1 --2- 172.21.15.119:0/913391678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a287360460 0x55a287363c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.968+0000 7fb67506fb80 1 -- 172.21.15.119:0/913391678 >> 172.21.15.119:0/913391678 conn(0x55a28728e360 msgr2=0x55a28735a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:09.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.968+0000 7fb67506fb80 1 -- 172.21.15.119:0/913391678 shutdown_connections 2023-12-07T10:24:09.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.968+0000 7fb67506fb80 1 -- 172.21.15.119:0/913391678 wait complete. 2023-12-07T10:24:09.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.969+0000 7fb67506fb80 1 -- start start 2023-12-07T10:24:09.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.969+0000 7fb67506fb80 1 Processor -- start 2023-12-07T10:24:09.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.970+0000 7fb67506fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28735fb40 0x55a28736b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.970+0000 7fb67506fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a287360460 0x55a287368c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.970+0000 7fb67506fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a287364150 0x55a287369370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.970+0000 7fb67506fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2872b39b0 con 0x55a287360460 2023-12-07T10:24:09.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.970+0000 7fb67506fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a28726d560 con 0x55a287364150 2023-12-07T10:24:09.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.970+0000 7fb67506fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a28726d200 con 0x55a28735fb40 2023-12-07T10:24:09.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.970+0000 7fb67506fb80 10 client.0 _ll_register_callbacks cb 0x55a287254940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:09.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.970+0000 7fb65affd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28735fb40 0x55a28736b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.970+0000 7fb65affd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28735fb40 0x55a28736b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55614/0 (socket says 172.21.15.119:55614) 2023-12-07T10:24:09.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.970+0000 7fb65affd700 1 -- 172.21.15.119:0/1217574992 learned_addr learned my addr 172.21.15.119:0/1217574992 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:09.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.970+0000 7fb65a7fc700 1 --2- 172.21.15.119:0/1217574992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a287360460 0x55a287368c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.970+0000 7fb65b7fe700 1 --2- 172.21.15.119:0/1217574992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a287364150 0x55a287369370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.971+0000 7fb65affd700 1 -- 172.21.15.119:0/1217574992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a287364150 msgr2=0x55a287369370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:09.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.971+0000 7fb65affd700 1 --2- 172.21.15.119:0/1217574992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a287364150 0x55a287369370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.971+0000 7fb65affd700 1 -- 172.21.15.119:0/1217574992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a287360460 msgr2=0x55a287368c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:09.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.971+0000 7fb65affd700 1 --2- 172.21.15.119:0/1217574992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a287360460 0x55a287368c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.971+0000 7fb65affd700 1 -- 172.21.15.119:0/1217574992 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65c1aabb0 con 0x55a28735fb40 2023-12-07T10:24:09.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.971+0000 7fb65a7fc700 1 --2- 172.21.15.119:0/1217574992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a287360460 0x55a287368c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:09.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.971+0000 7fb65affd700 1 --2- 172.21.15.119:0/1217574992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28735fb40 0x55a28736b9b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb64c009810 tx=0x7fb64c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27172244233ae90 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:09.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.971+0000 7fb63eff5700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:24:09.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.971+0000 7fb63eff5700 1 -- 172.21.15.119:0/1217574992 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c0b7070 con 0x55a28735fb40 2023-12-07T10:24:09.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.971+0000 7fb67506fb80 1 -- 172.21.15.119:0/1217574992 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a287352b00 con 0x55a28735fb40 2023-12-07T10:24:09.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.971+0000 7fb63eff5700 1 -- 172.21.15.119:0/1217574992 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fb64c0a3800 con 0x55a28735fb40 2023-12-07T10:24:09.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.972+0000 7fb63eff5700 1 -- 172.21.15.119:0/1217574992 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb64c0cc0a0 con 0x55a28735fb40 2023-12-07T10:24:09.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.972+0000 7fb67506fb80 10 client.26863 fetch_fsmap learned FSMap version 37 2023-12-07T10:24:09.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.972+0000 7fb67506fb80 1 -- 172.21.15.119:0/1217574992 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55a2870dcde0 con 0x55a28735fb40 2023-12-07T10:24:09.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.972+0000 7fb63eff5700 1 -- 172.21.15.119:0/1217574992 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a287352b00 con 0x55a28735fb40 2023-12-07T10:24:09.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.972+0000 7fb63eff5700 1 -- 172.21.15.119:0/1217574992 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 37) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fb64c005140 con 0x55a28735fb40 2023-12-07T10:24:09.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.972+0000 7fb67506fb80 10 client.26863 fetch_fsmap finished waiting for FSMap version 37 2023-12-07T10:24:09.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.972+0000 7fb67506fb80 10 client.26863 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2023-12-07T10:24:09.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.972+0000 7fb67506fb80 20 client.26863 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:09.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.972+0000 7fb67506fb80 1 --2- 172.21.15.119:0/1217574992 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x55a287378b70 0x55a28737af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:09.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.972+0000 7fb67506fb80 4 client.26863 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:24:09.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.972+0000 7fb67506fb80 1 -- 172.21.15.119:0/1217574992 --> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a28732e7a0 con 0x55a287378b70 2023-12-07T10:24:09.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.972+0000 7fb65a7fc700 1 --2- 172.21.15.119:0/1217574992 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x55a287378b70 0x55a28737af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:09.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.973+0000 7fb65a7fc700 1 --2- 172.21.15.119:0/1217574992 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x55a287378b70 0x55a28737af60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f6f72ae05e6715db server_cookie=4c53075e8cc6d1b4 in_seq=0 out_seq=0 2023-12-07T10:24:09.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.973+0000 7fb63eff5700 10 client.26863 ms_handle_connect on v2:172.21.15.153:6834/1295649464 2023-12-07T10:24:09.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.974+0000 7fb63eff5700 1 -- 172.21.15.119:0/1217574992 <== mds.4 v2:172.21.15.153:6834/1295649464 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a28732e7a0 con 0x55a287378b70 2023-12-07T10:24:09.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.974+0000 7fb63eff5700 10 client.26863 handle_command_reply: tid=0 2023-12-07T10:24:09.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.974+0000 7fb67506fb80 1 client.26863 shutdown 2023-12-07T10:24:09.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.974+0000 7fb67506fb80 1 -- 172.21.15.119:0/1217574992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28735fb40 msgr2=0x55a28736b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:09.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.974+0000 7fb67506fb80 1 --2- 172.21.15.119:0/1217574992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28735fb40 0x55a28736b9b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb64c009810 tx=0x7fb64c009260 comp rx=0 tx=0).stop 2023-12-07T10:24:09.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.974+0000 7fb67506fb80 1 -- 172.21.15.119:0/1217574992 shutdown_connections 2023-12-07T10:24:09.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.974+0000 7fb67506fb80 1 --2- 172.21.15.119:0/1217574992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a287364150 0x55a287369370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.974+0000 7fb67506fb80 1 --2- 172.21.15.119:0/1217574992 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x55a287378b70 0x55a28737af60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.974+0000 7fb67506fb80 1 --2- 172.21.15.119:0/1217574992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28735fb40 0x55a28736b9b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.974+0000 7fb67506fb80 1 --2- 172.21.15.119:0/1217574992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a287360460 0x55a287368c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:09.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.974+0000 7fb67506fb80 1 -- 172.21.15.119:0/1217574992 >> 172.21.15.119:0/1217574992 conn(0x55a28728e360 msgr2=0x55a287354d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:09.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.974+0000 7fb63eff5700 0 client.26863 ms_handle_reset on v2:172.21.15.153:6834/1295649464 2023-12-07T10:24:09.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.975+0000 7fb67506fb80 1 -- 172.21.15.119:0/1217574992 shutdown_connections 2023-12-07T10:24:09.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.975+0000 7fb67506fb80 1 -- 172.21.15.119:0/1217574992 wait complete. 2023-12-07T10:24:09.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.975+0000 7fb67506fb80 20 client.26863 trim_cache size 0 max 16384 2023-12-07T10:24:10.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.992+0000 7fb67506fb80 1 Processor -- start 2023-12-07T10:24:10.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.992+0000 7fb67506fb80 1 -- start start 2023-12-07T10:24:10.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.992+0000 7fb67506fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28735fb40 0x55a287362730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.992+0000 7fb67506fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a287360460 0x55a287361b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.992+0000 7fb67506fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a287364150 0x55a2873620c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.992+0000 7fb67506fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a287132920 con 0x55a287360460 2023-12-07T10:24:10.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.992+0000 7fb67506fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a287369bb0 con 0x55a28735fb40 2023-12-07T10:24:10.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.992+0000 7fb67506fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a28735b7d0 con 0x55a287364150 2023-12-07T10:24:10.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.992+0000 7fb65affd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28735fb40 0x55a287362730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.992+0000 7fb65a7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a287360460 0x55a287361b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.992+0000 7fb65affd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28735fb40 0x55a287362730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:55944/0 (socket says 172.21.15.119:55944) 2023-12-07T10:24:10.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.992+0000 7fb65affd700 1 -- 172.21.15.119:0/1177466349 learned_addr learned my addr 172.21.15.119:0/1177466349 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:10.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.992+0000 7fb65b7fe700 1 --2- 172.21.15.119:0/1177466349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a287364150 0x55a2873620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.992+0000 7fb65affd700 1 -- 172.21.15.119:0/1177466349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a287364150 msgr2=0x55a2873620c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.992+0000 7fb65affd700 1 --2- 172.21.15.119:0/1177466349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a287364150 0x55a2873620c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.992+0000 7fb65affd700 1 -- 172.21.15.119:0/1177466349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a287360460 msgr2=0x55a287361b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.992+0000 7fb65affd700 1 --2- 172.21.15.119:0/1177466349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a287360460 0x55a287361b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.992+0000 7fb65affd700 1 -- 172.21.15.119:0/1177466349 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2870c5c60 con 0x55a28735fb40 2023-12-07T10:24:10.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.992+0000 7fb65affd700 1 --2- 172.21.15.119:0/1177466349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28735fb40 0x55a287362730 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb64c09b490 tx=0x7fb64c099ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48e1a08c06615dd3 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:10.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.993+0000 7fb641ffb700 1 -- 172.21.15.119:0/1177466349 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c0b7030 con 0x55a28735fb40 2023-12-07T10:24:10.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.993+0000 7fb641ffb700 1 -- 172.21.15.119:0/1177466349 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb64c0a3800 con 0x55a28735fb40 2023-12-07T10:24:10.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.993+0000 7fb641ffb700 1 -- 172.21.15.119:0/1177466349 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c005140 con 0x55a28735fb40 2023-12-07T10:24:10.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.994+0000 7fb67506fb80 1 -- 172.21.15.119:0/1177466349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28735fb40 msgr2=0x55a287362730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.994+0000 7fb67506fb80 1 --2- 172.21.15.119:0/1177466349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28735fb40 0x55a287362730 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb64c09b490 tx=0x7fb64c099ab0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.994+0000 7fb67506fb80 1 -- 172.21.15.119:0/1177466349 shutdown_connections 2023-12-07T10:24:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.994+0000 7fb67506fb80 1 --2- 172.21.15.119:0/1177466349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a287364150 0x55a2873620c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.994+0000 7fb67506fb80 1 --2- 172.21.15.119:0/1177466349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28735fb40 0x55a287362730 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.994+0000 7fb67506fb80 1 --2- 172.21.15.119:0/1177466349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a287360460 0x55a287361b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.994+0000 7fb67506fb80 1 -- 172.21.15.119:0/1177466349 >> 172.21.15.119:0/1177466349 conn(0x55a28728e360 msgr2=0x55a287364530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.994+0000 7fb67506fb80 1 -- 172.21.15.119:0/1177466349 shutdown_connections 2023-12-07T10:24:10.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.994+0000 7fb67506fb80 1 -- 172.21.15.119:0/1177466349 wait complete. 2023-12-07T10:24:10.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.994+0000 7fb67506fb80 1 -- start start 2023-12-07T10:24:10.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.995+0000 7fb67506fb80 1 Processor -- start 2023-12-07T10:24:10.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.995+0000 7fb67506fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28735fb40 0x55a287370500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.995+0000 7fb67506fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a287360460 0x55a28736bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.995+0000 7fb67506fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a287364150 0x55a28736c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.995+0000 7fb67506fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a28736ce10 con 0x55a287360460 2023-12-07T10:24:10.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.995+0000 7fb67506fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2873ce000 con 0x55a28735fb40 2023-12-07T10:24:10.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.995+0000 7fb67506fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2873ce140 con 0x55a287364150 2023-12-07T10:24:10.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.995+0000 7fb67506fb80 10 client.0 _ll_register_callbacks cb 0x55a287254940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:10.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.995+0000 7fb65affd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28735fb40 0x55a287370500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.995+0000 7fb65a7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a287360460 0x55a28736bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.996+0000 7fb65a7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a287360460 0x55a28736bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36300/0 (socket says 172.21.15.119:36300) 2023-12-07T10:24:10.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.996+0000 7fb65a7fc700 1 -- 172.21.15.119:0/3170378447 learned_addr learned my addr 172.21.15.119:0/3170378447 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:10.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.995+0000 7fb65affd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28735fb40 0x55a287370500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:55948/0 (socket says 172.21.15.119:55948) 2023-12-07T10:24:10.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.996+0000 7fb65b7fe700 1 --2- 172.21.15.119:0/3170378447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a287364150 0x55a28736c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.996+0000 7fb65a7fc700 1 -- 172.21.15.119:0/3170378447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a287364150 msgr2=0x55a28736c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.996+0000 7fb65a7fc700 1 --2- 172.21.15.119:0/3170378447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a287364150 0x55a28736c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.996+0000 7fb65a7fc700 1 -- 172.21.15.119:0/3170378447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28735fb40 msgr2=0x55a287370500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.996+0000 7fb65a7fc700 1 --2- 172.21.15.119:0/3170378447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28735fb40 0x55a287370500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.996+0000 7fb65a7fc700 1 -- 172.21.15.119:0/3170378447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2870dcde0 con 0x55a287360460 2023-12-07T10:24:10.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.996+0000 7fb65affd700 1 --2- 172.21.15.119:0/3170378447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28735fb40 0x55a287370500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:10.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.996+0000 7fb65b7fe700 1 --2- 172.21.15.119:0/3170378447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a287364150 0x55a28736c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:24:10.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.996+0000 7fb65a7fc700 1 --2- 172.21.15.119:0/3170378447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a287360460 0x55a28736bfe0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0696c0 tx=0x7fb6500047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a5b97043cc9657 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:10.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.996+0000 7fb63eff5700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:24:10.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.997+0000 7fb63eff5700 1 -- 172.21.15.119:0/3170378447 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb650021040 con 0x55a287360460 2023-12-07T10:24:10.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.997+0000 7fb67506fb80 1 -- 172.21.15.119:0/3170378447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb64c09f650 con 0x55a287360460 2023-12-07T10:24:10.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.997+0000 7fb63eff5700 1 -- 172.21.15.119:0/3170378447 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fb65002a6e0 con 0x55a287360460 2023-12-07T10:24:10.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.997+0000 7fb63eff5700 1 -- 172.21.15.119:0/3170378447 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb65002a8a0 con 0x55a287360460 2023-12-07T10:24:10.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.997+0000 7fb67506fb80 10 client.16866 fetch_fsmap learned FSMap version 37 2023-12-07T10:24:10.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:09.997+0000 7fb67506fb80 1 -- 172.21.15.119:0/3170378447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55a287103250 con 0x55a287360460 2023-12-07T10:24:10.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.002+0000 7fb63eff5700 1 -- 172.21.15.119:0/3170378447 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb64c09f650 con 0x55a287360460 2023-12-07T10:24:10.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.002+0000 7fb63eff5700 1 -- 172.21.15.119:0/3170378447 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 37) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fb650003c50 con 0x55a287360460 2023-12-07T10:24:10.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.002+0000 7fb67506fb80 10 client.16866 fetch_fsmap finished waiting for FSMap version 37 2023-12-07T10:24:10.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.002+0000 7fb67506fb80 10 client.16866 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2023-12-07T10:24:10.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.002+0000 7fb67506fb80 20 client.16866 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:10.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.002+0000 7fb67506fb80 1 --2- 172.21.15.119:0/3170378447 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x55a2873cf9e0 0x55a2873d1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.002+0000 7fb67506fb80 4 client.16866 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:24:10.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.002+0000 7fb67506fb80 1 -- 172.21.15.119:0/3170378447 --> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a287279fd0 con 0x55a2873cf9e0 2023-12-07T10:24:10.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.002+0000 7fb65affd700 1 --2- 172.21.15.119:0/3170378447 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x55a2873cf9e0 0x55a2873d1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.003+0000 7fb65affd700 1 --2- 172.21.15.119:0/3170378447 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x55a2873cf9e0 0x55a2873d1e00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ae479d2f86257e30 server_cookie=eafd621396721750 in_seq=0 out_seq=0 2023-12-07T10:24:10.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.003+0000 7fb63eff5700 10 client.16866 ms_handle_connect on v2:172.21.15.153:6834/1295649464 2023-12-07T10:24:10.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.003+0000 7fb63eff5700 1 -- 172.21.15.119:0/3170378447 <== mds.4 v2:172.21.15.153:6834/1295649464 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a287279fd0 con 0x55a2873cf9e0 2023-12-07T10:24:10.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.003+0000 7fb63eff5700 10 client.16866 handle_command_reply: tid=0 2023-12-07T10:24:10.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.003+0000 7fb67506fb80 1 client.16866 shutdown 2023-12-07T10:24:10.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.003+0000 7fb67506fb80 1 -- 172.21.15.119:0/3170378447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a287360460 msgr2=0x55a28736bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.003+0000 7fb67506fb80 1 --2- 172.21.15.119:0/3170378447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a287360460 0x55a28736bfe0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0696c0 tx=0x7fb6500047c0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.003+0000 7fb67506fb80 1 -- 172.21.15.119:0/3170378447 shutdown_connections 2023-12-07T10:24:10.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.003+0000 7fb67506fb80 1 --2- 172.21.15.119:0/3170378447 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x55a2873cf9e0 0x55a2873d1e00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.003+0000 7fb67506fb80 1 --2- 172.21.15.119:0/3170378447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a287364150 0x55a28736c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.003+0000 7fb67506fb80 1 --2- 172.21.15.119:0/3170378447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a28735fb40 0x55a287370500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.003+0000 7fb67506fb80 1 --2- 172.21.15.119:0/3170378447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a287360460 0x55a28736bfe0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.003+0000 7fb63eff5700 0 client.16866 ms_handle_reset on v2:172.21.15.153:6834/1295649464 2023-12-07T10:24:10.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.003+0000 7fb67506fb80 1 -- 172.21.15.119:0/3170378447 >> 172.21.15.119:0/3170378447 conn(0x55a28728e360 msgr2=0x55a2873632a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:10.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.003+0000 7fb67506fb80 1 -- 172.21.15.119:0/3170378447 shutdown_connections 2023-12-07T10:24:10.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.003+0000 7fb67506fb80 1 -- 172.21.15.119:0/3170378447 wait complete. 2023-12-07T10:24:10.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.004+0000 7fb67506fb80 20 client.16866 trim_cache size 0 max 16384 2023-12-07T10:24:10.017 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:24:10.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.004+0000 7fb6617ff700 1 -- 172.21.15.119:0/3917621002 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb6440a1d00 msgr2=0x7fb6440a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.004+0000 7fb6617ff700 1 --2- 172.21.15.119:0/3917621002 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb6440a1d00 0x7fb6440a41c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb65c103e10 tx=0x7fb650009310 comp rx=0 tx=0).stop 2023-12-07T10:24:10.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.004+0000 7fb6617ff700 1 -- 172.21.15.119:0/3917621002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb65c100f40 msgr2=0x7fb65c1026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.004+0000 7fb6617ff700 1 --2- 172.21.15.119:0/3917621002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb65c100f40 0x7fb65c1026e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb64c005950 tx=0x7fb64c00fa00 comp rx=0 tx=0).stop 2023-12-07T10:24:10.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.004+0000 7fb6617ff700 1 -- 172.21.15.119:0/3917621002 shutdown_connections 2023-12-07T10:24:10.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.004+0000 7fb6617ff700 1 --2- 172.21.15.119:0/3917621002 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb6440a1d00 0x7fb6440a41c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.004+0000 7fb6617ff700 1 --2- 172.21.15.119:0/3917621002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb65c10dfb0 0x7fb65c103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.004+0000 7fb6617ff700 1 --2- 172.21.15.119:0/3917621002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb65c100f40 0x7fb65c1026e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.004+0000 7fb6617ff700 1 --2- 172.21.15.119:0/3917621002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb65c101860 0x7fb65c102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.004+0000 7fb6617ff700 1 -- 172.21.15.119:0/3917621002 >> 172.21.15.119:0/3917621002 conn(0x7fb65c077610 msgr2=0x7fb65c10f690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:10.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.004+0000 7fb6617ff700 1 -- 172.21.15.119:0/3917621002 shutdown_connections 2023-12-07T10:24:10.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.004+0000 7fb6617ff700 1 -- 172.21.15.119:0/3917621002 wait complete. 2023-12-07T10:24:10.027 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-12-07T10:24:10.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.254+0000 7f70e71ff700 1 -- 172.21.15.119:0/2333078013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70e0101550 msgr2=0x7f70e01019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.254+0000 7f70e71ff700 1 --2- 172.21.15.119:0/2333078013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70e0101550 0x7f70e01019d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f70cc0099c0 tx=0x7f70cc009cd0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.254+0000 7f70e71ff700 1 -- 172.21.15.119:0/2333078013 shutdown_connections 2023-12-07T10:24:10.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.254+0000 7f70e71ff700 1 --2- 172.21.15.119:0/2333078013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70e0101f10 0x7f70e010e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.254+0000 7f70e71ff700 1 --2- 172.21.15.119:0/2333078013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70e0107610 0x7f70e01079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.254+0000 7f70e71ff700 1 --2- 172.21.15.119:0/2333078013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70e0101550 0x7f70e01019d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.254+0000 7f70e71ff700 1 -- 172.21.15.119:0/2333078013 >> 172.21.15.119:0/2333078013 conn(0x7f70e00fd020 msgr2=0x7f70e00ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:10.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.254+0000 7f70e71ff700 1 -- 172.21.15.119:0/2333078013 shutdown_connections 2023-12-07T10:24:10.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.254+0000 7f70e71ff700 1 -- 172.21.15.119:0/2333078013 wait complete. 2023-12-07T10:24:10.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.256+0000 7f70e71ff700 1 Processor -- start 2023-12-07T10:24:10.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.256+0000 7f70e71ff700 1 -- start start 2023-12-07T10:24:10.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.256+0000 7f70e71ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70e0101550 0x7f70e019f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.256+0000 7f70e71ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70e0101f10 0x7f70e019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.256+0000 7f70e71ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70e0107610 0x7f70e01999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.256+0000 7f70e71ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e019a000 con 0x7f70e0107610 2023-12-07T10:24:10.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.256+0000 7f70e71ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e019a170 con 0x7f70e0101f10 2023-12-07T10:24:10.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.256+0000 7f70e71ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e019a330 con 0x7f70e0101550 2023-12-07T10:24:10.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.257+0000 7f70dffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70e0101f10 0x7f70e019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.257+0000 7f70e4f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70e0101550 0x7f70e019f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.257+0000 7f70dffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70e0101f10 0x7f70e019fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:55968/0 (socket says 172.21.15.119:55968) 2023-12-07T10:24:10.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.257+0000 7f70e4f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70e0101550 0x7f70e019f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55654/0 (socket says 172.21.15.119:55654) 2023-12-07T10:24:10.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.257+0000 7f70dffff700 1 -- 172.21.15.119:0/3547104096 learned_addr learned my addr 172.21.15.119:0/3547104096 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:10.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.257+0000 7f70e579c700 1 --2- 172.21.15.119:0/3547104096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70e0107610 0x7f70e01999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.257+0000 7f70e4f9b700 1 -- 172.21.15.119:0/3547104096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70e0101f10 msgr2=0x7f70e019fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.257+0000 7f70e4f9b700 1 --2- 172.21.15.119:0/3547104096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70e0101f10 0x7f70e019fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.257+0000 7f70e4f9b700 1 -- 172.21.15.119:0/3547104096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70e0107610 msgr2=0x7f70e01999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.257+0000 7f70e4f9b700 1 --2- 172.21.15.119:0/3547104096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70e0107610 0x7f70e01999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.257+0000 7f70e4f9b700 1 -- 172.21.15.119:0/3547104096 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d4009520 con 0x7f70e0101550 2023-12-07T10:24:10.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.257+0000 7f70dffff700 1 --2- 172.21.15.119:0/3547104096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70e0101f10 0x7f70e019fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:10.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.257+0000 7f70e4f9b700 1 --2- 172.21.15.119:0/3547104096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70e0101550 0x7f70e019f870 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f70d400e9f0 tx=0x7f70d400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:10.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.258+0000 7f70ddffb700 1 -- 172.21.15.119:0/3547104096 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d400cb70 con 0x7f70e0101550 2023-12-07T10:24:10.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.258+0000 7f70e71ff700 1 -- 172.21.15.119:0/3547104096 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70cc0095f0 con 0x7f70e0101550 2023-12-07T10:24:10.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.258+0000 7f70ddffb700 1 -- 172.21.15.119:0/3547104096 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70d400ee90 con 0x7f70e0101550 2023-12-07T10:24:10.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.258+0000 7f70ddffb700 1 -- 172.21.15.119:0/3547104096 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4003cf0 con 0x7f70e0101550 2023-12-07T10:24:10.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.258+0000 7f70e71ff700 1 -- 172.21.15.119:0/3547104096 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e019a800 con 0x7f70e0101550 2023-12-07T10:24:10.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.261+0000 7f70ddffb700 1 -- 172.21.15.119:0/3547104096 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f70d400ccd0 con 0x7f70e0101550 2023-12-07T10:24:10.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:09 smithi159 ceph-mon[134726]: pgmap v115: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 267 op/s 2023-12-07T10:24:10.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:09 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:24:10.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:09 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2417932942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:24:10.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.262+0000 7f70faab0b80 1 Processor -- start 2023-12-07T10:24:10.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.262+0000 7f70faab0b80 1 -- start start 2023-12-07T10:24:10.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.263+0000 7f70ddffb700 1 --2- 172.21.15.119:0/3547104096 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f70d00a1d00 0x7f70d00a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.263+0000 7f70faab0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a72b40 0x55d3e9a72f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.263+0000 7f70faab0b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e9a73460 0x55d3e9a76c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.263+0000 7f70ddffb700 1 -- 172.21.15.119:0/3547104096 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f70d4014070 con 0x7f70e0101550 2023-12-07T10:24:10.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.263+0000 7f70faab0b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3e9a77150 0x55d3e9a77570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.263+0000 7f70e579c700 1 --2- 172.21.15.119:0/3547104096 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f70d00a1d00 0x7f70d00a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.263+0000 7f70faab0b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e99e67c0 con 0x55d3e9a73460 2023-12-07T10:24:10.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.263+0000 7f70faab0b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e99cd7a0 con 0x55d3e9a77150 2023-12-07T10:24:10.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.263+0000 7f70faab0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e99cb0c0 con 0x55d3e9a72b40 2023-12-07T10:24:10.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.263+0000 7f70dffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e9a73460 0x55d3e9a76c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.263+0000 7f70e4f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a72b40 0x55d3e9a72f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.263+0000 7f70e4f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a72b40 0x55d3e9a72f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55662/0 (socket says 172.21.15.119:55662) 2023-12-07T10:24:10.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.263+0000 7f70e4f9b700 1 -- 172.21.15.119:0/1791033240 learned_addr learned my addr 172.21.15.119:0/1791033240 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:10.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.263+0000 7f70e579c700 1 --2- 172.21.15.119:0/1791033240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3e9a77150 0x55d3e9a77570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.263+0000 7f70e4f9b700 1 -- 172.21.15.119:0/1791033240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3e9a77150 msgr2=0x55d3e9a77570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.263+0000 7f70e4f9b700 1 --2- 172.21.15.119:0/1791033240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3e9a77150 0x55d3e9a77570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.263+0000 7f70e4f9b700 1 -- 172.21.15.119:0/1791033240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e9a73460 msgr2=0x55d3e9a76c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.263+0000 7f70e4f9b700 1 --2- 172.21.15.119:0/1791033240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e9a73460 0x55d3e9a76c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.264+0000 7f70e4f9b700 1 -- 172.21.15.119:0/1791033240 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3e9843630 con 0x55d3e9a72b40 2023-12-07T10:24:10.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.264+0000 7f70e4f9b700 1 --2- 172.21.15.119:0/1791033240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a72b40 0x55d3e9a72f20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f70d4005cb0 tx=0x7f70d409a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c74a14f70427d14f server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:10.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.264+0000 7f70e579c700 1 --2- 172.21.15.119:0/3547104096 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f70d00a1d00 0x7f70d00a41c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009ec0 tx=0x7f70d8009920 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:10.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.264+0000 7f70caffd700 1 -- 172.21.15.119:0/1791033240 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d40b3210 con 0x55d3e9a72b40 2023-12-07T10:24:10.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.264+0000 7f70caffd700 1 -- 172.21.15.119:0/1791033240 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70d409eab0 con 0x55d3e9a72b40 2023-12-07T10:24:10.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.264+0000 7f70caffd700 1 -- 172.21.15.119:0/1791033240 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d409ae20 con 0x55d3e9a72b40 2023-12-07T10:24:10.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.265+0000 7f70faab0b80 1 -- 172.21.15.119:0/1791033240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a72b40 msgr2=0x55d3e9a72f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.265+0000 7f70faab0b80 1 --2- 172.21.15.119:0/1791033240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a72b40 0x55d3e9a72f20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f70d4005cb0 tx=0x7f70d409a340 comp rx=0 tx=0).stop 2023-12-07T10:24:10.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.265+0000 7f70faab0b80 1 -- 172.21.15.119:0/1791033240 shutdown_connections 2023-12-07T10:24:10.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.265+0000 7f70faab0b80 1 --2- 172.21.15.119:0/1791033240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3e9a77150 0x55d3e9a77570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.265+0000 7f70faab0b80 1 --2- 172.21.15.119:0/1791033240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a72b40 0x55d3e9a72f20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.265+0000 7f70faab0b80 1 --2- 172.21.15.119:0/1791033240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e9a73460 0x55d3e9a76c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.265+0000 7f70faab0b80 1 -- 172.21.15.119:0/1791033240 >> 172.21.15.119:0/1791033240 conn(0x55d3e99a1360 msgr2=0x55d3e9a6d130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:10.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.265+0000 7f70faab0b80 1 -- 172.21.15.119:0/1791033240 shutdown_connections 2023-12-07T10:24:10.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.265+0000 7f70faab0b80 1 -- 172.21.15.119:0/1791033240 wait complete. 2023-12-07T10:24:10.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.266+0000 7f70faab0b80 1 -- start start 2023-12-07T10:24:10.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.266+0000 7f70faab0b80 1 Processor -- start 2023-12-07T10:24:10.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.266+0000 7f70faab0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a72b40 0x55d3e9a7e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.266+0000 7f70faab0b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e9a73460 0x55d3e9a7bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.266+0000 7f70faab0b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3e9a77150 0x55d3e9a7c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.266+0000 7f70faab0b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e99c69b0 con 0x55d3e9a73460 2023-12-07T10:24:10.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.266+0000 7f70faab0b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e9980560 con 0x55d3e9a77150 2023-12-07T10:24:10.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.266+0000 7f70faab0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e9980200 con 0x55d3e9a72b40 2023-12-07T10:24:10.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.266+0000 7f70faab0b80 10 client.0 _ll_register_callbacks cb 0x55d3e9967940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:10.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.267+0000 7f70e4f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a72b40 0x55d3e9a7e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.267+0000 7f70e4f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a72b40 0x55d3e9a7e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55664/0 (socket says 172.21.15.119:55664) 2023-12-07T10:24:10.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.267+0000 7f70e4f9b700 1 -- 172.21.15.119:0/1179709587 learned_addr learned my addr 172.21.15.119:0/1179709587 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:10.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.267+0000 7f70dffff700 1 --2- 172.21.15.119:0/1179709587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e9a73460 0x55d3e9a7bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.267+0000 7f70e579c700 1 --2- 172.21.15.119:0/1179709587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3e9a77150 0x55d3e9a7c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.267+0000 7f70e4f9b700 1 -- 172.21.15.119:0/1179709587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3e9a77150 msgr2=0x55d3e9a7c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.267+0000 7f70e4f9b700 1 --2- 172.21.15.119:0/1179709587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3e9a77150 0x55d3e9a7c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.267+0000 7f70e4f9b700 1 -- 172.21.15.119:0/1179709587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e9a73460 msgr2=0x55d3e9a7bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.267+0000 7f70e4f9b700 1 --2- 172.21.15.119:0/1179709587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e9a73460 0x55d3e9a7bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.267+0000 7f70e4f9b700 1 -- 172.21.15.119:0/1179709587 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e019a800 con 0x55d3e9a72b40 2023-12-07T10:24:10.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.268+0000 7f70e4f9b700 1 --2- 172.21.15.119:0/1179709587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a72b40 0x55d3e9a7e920 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f70d409a660 tx=0x7f70d40a2cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=149dd91abbbc388 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:10.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.268+0000 7f70bffff700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:24:10.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.268+0000 7f70bffff700 1 -- 172.21.15.119:0/1179709587 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d40ba0c0 con 0x55d3e9a72b40 2023-12-07T10:24:10.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.268+0000 7f70faab0b80 1 -- 172.21.15.119:0/1179709587 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3e9a65b00 con 0x55d3e9a72b40 2023-12-07T10:24:10.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.268+0000 7f70bffff700 1 -- 172.21.15.119:0/1179709587 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f70d40ba220 con 0x55d3e9a72b40 2023-12-07T10:24:10.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.268+0000 7f70bffff700 1 -- 172.21.15.119:0/1179709587 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70d4099e50 con 0x55d3e9a72b40 2023-12-07T10:24:10.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.268+0000 7f70faab0b80 10 client.26881 fetch_fsmap learned FSMap version 37 2023-12-07T10:24:10.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.268+0000 7f70faab0b80 1 -- 172.21.15.119:0/1179709587 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55d3e97efde0 con 0x55d3e9a72b40 2023-12-07T10:24:10.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.268+0000 7f70bffff700 1 -- 172.21.15.119:0/1179709587 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d3e9a65b00 con 0x55d3e9a72b40 2023-12-07T10:24:10.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.268+0000 7f70bffff700 1 -- 172.21.15.119:0/1179709587 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 37) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f70d40a2860 con 0x55d3e9a72b40 2023-12-07T10:24:10.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.269+0000 7f70faab0b80 10 client.26881 fetch_fsmap finished waiting for FSMap version 37 2023-12-07T10:24:10.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.269+0000 7f70faab0b80 10 client.26881 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-12-07T10:24:10.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.269+0000 7f70faab0b80 20 client.26881 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:10.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.269+0000 7f70faab0b80 1 --2- 172.21.15.119:0/1179709587 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x55d3e9a8be50 0x55d3e9a8e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.269+0000 7f70faab0b80 4 client.26881 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:24:10.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.269+0000 7f70faab0b80 1 -- 172.21.15.119:0/1179709587 --> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3e9a417a0 con 0x55d3e9a8be50 2023-12-07T10:24:10.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.269+0000 7f70dffff700 1 --2- 172.21.15.119:0/1179709587 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x55d3e9a8be50 0x55d3e9a8e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.269+0000 7f70dffff700 1 --2- 172.21.15.119:0/1179709587 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x55d3e9a8be50 0x55d3e9a8e240 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=625a814dbe55b608 server_cookie=bbd8373c70c6badc in_seq=0 out_seq=0 2023-12-07T10:24:10.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.269+0000 7f70bffff700 10 client.26881 ms_handle_connect on v2:172.21.15.159:6834/2406380792 2023-12-07T10:24:10.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.270+0000 7f70bffff700 1 -- 172.21.15.119:0/1179709587 <== mds.3 v2:172.21.15.159:6834/2406380792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d3e9a417a0 con 0x55d3e9a8be50 2023-12-07T10:24:10.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.270+0000 7f70bffff700 10 client.26881 handle_command_reply: tid=0 2023-12-07T10:24:10.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.270+0000 7f70faab0b80 1 client.26881 shutdown 2023-12-07T10:24:10.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.270+0000 7f70faab0b80 1 -- 172.21.15.119:0/1179709587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a72b40 msgr2=0x55d3e9a7e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.270+0000 7f70faab0b80 1 --2- 172.21.15.119:0/1179709587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a72b40 0x55d3e9a7e920 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f70d409a660 tx=0x7f70d40a2cf0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.270+0000 7f70faab0b80 1 -- 172.21.15.119:0/1179709587 shutdown_connections 2023-12-07T10:24:10.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.270+0000 7f70faab0b80 1 --2- 172.21.15.119:0/1179709587 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x55d3e9a8be50 0x55d3e9a8e240 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.270+0000 7f70faab0b80 1 --2- 172.21.15.119:0/1179709587 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3e9a77150 0x55d3e9a7c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.270+0000 7f70faab0b80 1 --2- 172.21.15.119:0/1179709587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a72b40 0x55d3e9a7e920 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.270+0000 7f70bffff700 0 client.26881 ms_handle_reset on v2:172.21.15.159:6834/2406380792 2023-12-07T10:24:10.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.270+0000 7f70faab0b80 1 --2- 172.21.15.119:0/1179709587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e9a73460 0x55d3e9a7bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.270+0000 7f70faab0b80 1 -- 172.21.15.119:0/1179709587 >> 172.21.15.119:0/1179709587 conn(0x55d3e99a1360 msgr2=0x55d3e9a67d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:10.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.270+0000 7f70faab0b80 1 -- 172.21.15.119:0/1179709587 shutdown_connections 2023-12-07T10:24:10.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.270+0000 7f70faab0b80 1 -- 172.21.15.119:0/1179709587 wait complete. 2023-12-07T10:24:10.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.271+0000 7f70faab0b80 20 client.26881 trim_cache size 0 max 16384 2023-12-07T10:24:10.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:09 smithi153 ceph-mon[133625]: pgmap v115: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 267 op/s 2023-12-07T10:24:10.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:09 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:24:10.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:09 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2417932942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:24:10.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.286+0000 7f70faab0b80 1 Processor -- start 2023-12-07T10:24:10.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.286+0000 7f70faab0b80 1 -- start start 2023-12-07T10:24:10.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.286+0000 7f70faab0b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e9a72b40 0x55d3e9a74b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.286+0000 7f70faab0b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3e9a73460 0x55d3e9a750c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.286+0000 7f70faab0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a77150 0x55d3e9a87e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.286+0000 7f70faab0b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e9845920 con 0x55d3e9a72b40 2023-12-07T10:24:10.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.286+0000 7f70faab0b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e9a65ae0 con 0x55d3e9a73460 2023-12-07T10:24:10.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.286+0000 7f70faab0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e9a6e800 con 0x55d3e9a77150 2023-12-07T10:24:10.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.286+0000 7f70e579c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a77150 0x55d3e9a87e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.286+0000 7f70dffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3e9a73460 0x55d3e9a750c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.286+0000 7f70e579c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a77150 0x55d3e9a87e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55670/0 (socket says 172.21.15.119:55670) 2023-12-07T10:24:10.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.286+0000 7f70dffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3e9a73460 0x55d3e9a750c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:55990/0 (socket says 172.21.15.119:55990) 2023-12-07T10:24:10.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.286+0000 7f70e579c700 1 -- 172.21.15.119:0/1179197509 learned_addr learned my addr 172.21.15.119:0/1179197509 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:10.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.287+0000 7f70e4f9b700 1 --2- 172.21.15.119:0/1179197509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e9a72b40 0x55d3e9a74b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.287+0000 7f70dffff700 1 -- 172.21.15.119:0/1179197509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a77150 msgr2=0x55d3e9a87e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.287+0000 7f70dffff700 1 --2- 172.21.15.119:0/1179197509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a77150 0x55d3e9a87e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.287+0000 7f70dffff700 1 -- 172.21.15.119:0/1179197509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e9a72b40 msgr2=0x55d3e9a74b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.287+0000 7f70dffff700 1 --2- 172.21.15.119:0/1179197509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e9a72b40 0x55d3e9a74b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.287+0000 7f70dffff700 1 -- 172.21.15.119:0/1179197509 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3e9a67040 con 0x55d3e9a73460 2023-12-07T10:24:10.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.287+0000 7f70e579c700 1 --2- 172.21.15.119:0/1179197509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a77150 0x55d3e9a87e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:10.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.287+0000 7f70e4f9b700 1 --2- 172.21.15.119:0/1179197509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e9a72b40 0x55d3e9a74b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:24:10.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.287+0000 7f70dffff700 1 --2- 172.21.15.119:0/1179197509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3e9a73460 0x55d3e9a750c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f70cc006010 tx=0x7f70cc005ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93ace9d6aba6be86 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:10.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.288+0000 7f70cb7fe700 1 -- 172.21.15.119:0/1179197509 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc01d040 con 0x55d3e9a73460 2023-12-07T10:24:10.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.288+0000 7f70cb7fe700 1 -- 172.21.15.119:0/1179197509 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70cc01e040 con 0x55d3e9a73460 2023-12-07T10:24:10.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.288+0000 7f70cb7fe700 1 -- 172.21.15.119:0/1179197509 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc0280e0 con 0x55d3e9a73460 2023-12-07T10:24:10.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.289+0000 7f70faab0b80 1 -- 172.21.15.119:0/1179197509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3e9a73460 msgr2=0x55d3e9a750c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.289+0000 7f70faab0b80 1 --2- 172.21.15.119:0/1179197509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3e9a73460 0x55d3e9a750c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f70cc006010 tx=0x7f70cc005ab0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.289+0000 7f70faab0b80 1 -- 172.21.15.119:0/1179197509 shutdown_connections 2023-12-07T10:24:10.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.289+0000 7f70faab0b80 1 --2- 172.21.15.119:0/1179197509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a77150 0x55d3e9a87e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.289+0000 7f70faab0b80 1 --2- 172.21.15.119:0/1179197509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e9a72b40 0x55d3e9a74b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.289+0000 7f70faab0b80 1 --2- 172.21.15.119:0/1179197509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3e9a73460 0x55d3e9a750c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.289+0000 7f70faab0b80 1 -- 172.21.15.119:0/1179197509 >> 172.21.15.119:0/1179197509 conn(0x55d3e99a1360 msgr2=0x55d3e9a7e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:10.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.289+0000 7f70faab0b80 1 -- 172.21.15.119:0/1179197509 shutdown_connections 2023-12-07T10:24:10.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.289+0000 7f70faab0b80 1 -- 172.21.15.119:0/1179197509 wait complete. 2023-12-07T10:24:10.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.289+0000 7f70faab0b80 1 -- start start 2023-12-07T10:24:10.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.290+0000 7f70faab0b80 1 Processor -- start 2023-12-07T10:24:10.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.290+0000 7f70faab0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a72b40 0x55d3e9a83820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.290+0000 7f70faab0b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e9a73460 0x55d3e9a792d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.290+0000 7f70faab0b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3e9a77150 0x55d3e9a77f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.290+0000 7f70faab0b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e9a79be0 con 0x55d3e9a73460 2023-12-07T10:24:10.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.290+0000 7f70faab0b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e9a784c0 con 0x55d3e9a77150 2023-12-07T10:24:10.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.290+0000 7f70faab0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e9a78680 con 0x55d3e9a72b40 2023-12-07T10:24:10.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.290+0000 7f70faab0b80 10 client.0 _ll_register_callbacks cb 0x55d3e9967940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:10.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.290+0000 7f70e579c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3e9a77150 0x55d3e9a77f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.290+0000 7f70e4f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a72b40 0x55d3e9a83820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.290+0000 7f70dffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e9a73460 0x55d3e9a792d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.290+0000 7f70e4f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a72b40 0x55d3e9a83820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55682/0 (socket says 172.21.15.119:55682) 2023-12-07T10:24:10.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.290+0000 7f70e579c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3e9a77150 0x55d3e9a77f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:56006/0 (socket says 172.21.15.119:56006) 2023-12-07T10:24:10.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.290+0000 7f70e4f9b700 1 -- 172.21.15.119:0/3023371473 learned_addr learned my addr 172.21.15.119:0/3023371473 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:10.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.291+0000 7f70e4f9b700 1 -- 172.21.15.119:0/3023371473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3e9a77150 msgr2=0x55d3e9a77f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.291+0000 7f70e4f9b700 1 --2- 172.21.15.119:0/3023371473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3e9a77150 0x55d3e9a77f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.291+0000 7f70e4f9b700 1 -- 172.21.15.119:0/3023371473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e9a73460 msgr2=0x55d3e9a792d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.291+0000 7f70e4f9b700 1 --2- 172.21.15.119:0/3023371473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e9a73460 0x55d3e9a792d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.291+0000 7f70e4f9b700 1 -- 172.21.15.119:0/3023371473 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d8003a00 con 0x55d3e9a72b40 2023-12-07T10:24:10.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.291+0000 7f70e4f9b700 1 --2- 172.21.15.119:0/3023371473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a72b40 0x55d3e9a83820 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f70d409a8f0 tx=0x7f70d4099940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d217a82b55e84711 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:10.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.291+0000 7f70bffff700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:24:10.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.291+0000 7f70bffff700 1 -- 172.21.15.119:0/3023371473 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4099a30 con 0x55d3e9a72b40 2023-12-07T10:24:10.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.291+0000 7f70bffff700 1 -- 172.21.15.119:0/3023371473 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f70d40ba0b0 con 0x55d3e9a72b40 2023-12-07T10:24:10.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.291+0000 7f70faab0b80 1 -- 172.21.15.119:0/3023371473 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70d409eab0 con 0x55d3e9a72b40 2023-12-07T10:24:10.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.291+0000 7f70bffff700 1 -- 172.21.15.119:0/3023371473 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70d40ba270 con 0x55d3e9a72b40 2023-12-07T10:24:10.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.291+0000 7f70faab0b80 10 client.26887 fetch_fsmap learned FSMap version 37 2023-12-07T10:24:10.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.291+0000 7f70faab0b80 1 -- 172.21.15.119:0/3023371473 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7f70e010ecd0 con 0x55d3e9a72b40 2023-12-07T10:24:10.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.292+0000 7f70bffff700 1 -- 172.21.15.119:0/3023371473 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70d409eab0 con 0x55d3e9a72b40 2023-12-07T10:24:10.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.292+0000 7f70bffff700 1 -- 172.21.15.119:0/3023371473 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 37) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f70d4099ee0 con 0x55d3e9a72b40 2023-12-07T10:24:10.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.292+0000 7f70faab0b80 10 client.26887 fetch_fsmap finished waiting for FSMap version 37 2023-12-07T10:24:10.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.292+0000 7f70faab0b80 10 client.26887 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-12-07T10:24:10.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.292+0000 7f70faab0b80 20 client.26887 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:10.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.292+0000 7f70faab0b80 1 --2- 172.21.15.119:0/3023371473 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x55d3e9ae2b20 0x55d3e9ae4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.292+0000 7f70faab0b80 4 client.26887 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:24:10.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.292+0000 7f70faab0b80 1 -- 172.21.15.119:0/3023371473 --> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d3e998cfd0 con 0x55d3e9ae2b20 2023-12-07T10:24:10.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.292+0000 7f70dffff700 1 --2- 172.21.15.119:0/3023371473 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x55d3e9ae2b20 0x55d3e9ae4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.293+0000 7f70bffff700 10 client.26887 ms_handle_connect on v2:172.21.15.159:6834/2406380792 2023-12-07T10:24:10.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.293+0000 7f70dffff700 1 --2- 172.21.15.119:0/3023371473 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x55d3e9ae2b20 0x55d3e9ae4f10 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=51ac11f9a8f138fe server_cookie=bc7161e07ddefae1 in_seq=0 out_seq=0 2023-12-07T10:24:10.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.293+0000 7f70bffff700 1 -- 172.21.15.119:0/3023371473 <== mds.3 v2:172.21.15.159:6834/2406380792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d3e998cfd0 con 0x55d3e9ae2b20 2023-12-07T10:24:10.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.293+0000 7f70bffff700 10 client.26887 handle_command_reply: tid=0 2023-12-07T10:24:10.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.293+0000 7f70faab0b80 1 client.26887 shutdown 2023-12-07T10:24:10.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.293+0000 7f70faab0b80 1 -- 172.21.15.119:0/3023371473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a72b40 msgr2=0x55d3e9a83820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.293+0000 7f70faab0b80 1 --2- 172.21.15.119:0/3023371473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a72b40 0x55d3e9a83820 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f70d409a8f0 tx=0x7f70d4099940 comp rx=0 tx=0).stop 2023-12-07T10:24:10.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.293+0000 7f70faab0b80 1 -- 172.21.15.119:0/3023371473 shutdown_connections 2023-12-07T10:24:10.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.293+0000 7f70faab0b80 1 --2- 172.21.15.119:0/3023371473 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x55d3e9ae2b20 0x55d3e9ae4f10 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.293+0000 7f70faab0b80 1 --2- 172.21.15.119:0/3023371473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d3e9a77150 0x55d3e9a77f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.293+0000 7f70faab0b80 1 --2- 172.21.15.119:0/3023371473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3e9a72b40 0x55d3e9a83820 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.293+0000 7f70bffff700 0 client.26887 ms_handle_reset on v2:172.21.15.159:6834/2406380792 2023-12-07T10:24:10.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.293+0000 7f70faab0b80 1 --2- 172.21.15.119:0/3023371473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e9a73460 0x55d3e9a792d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.293+0000 7f70faab0b80 1 -- 172.21.15.119:0/3023371473 >> 172.21.15.119:0/3023371473 conn(0x55d3e99a1360 msgr2=0x55d3e9a87e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:10.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.294+0000 7f70faab0b80 1 -- 172.21.15.119:0/3023371473 shutdown_connections 2023-12-07T10:24:10.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.294+0000 7f70faab0b80 1 -- 172.21.15.119:0/3023371473 wait complete. 2023-12-07T10:24:10.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.294+0000 7f70faab0b80 20 client.26887 trim_cache size 0 max 16384 2023-12-07T10:24:10.300 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:24:10.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.294+0000 7f70e71ff700 1 -- 172.21.15.119:0/3547104096 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f70d00a1d00 msgr2=0x7f70d00a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.294+0000 7f70e71ff700 1 --2- 172.21.15.119:0/3547104096 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f70d00a1d00 0x7f70d00a41c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009ec0 tx=0x7f70d8009920 comp rx=0 tx=0).stop 2023-12-07T10:24:10.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.294+0000 7f70e71ff700 1 -- 172.21.15.119:0/3547104096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70e0101550 msgr2=0x7f70e019f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.294+0000 7f70e71ff700 1 --2- 172.21.15.119:0/3547104096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70e0101550 0x7f70e019f870 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f70d400e9f0 tx=0x7f70d400ed00 comp rx=0 tx=0).stop 2023-12-07T10:24:10.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.294+0000 7f70e71ff700 1 -- 172.21.15.119:0/3547104096 shutdown_connections 2023-12-07T10:24:10.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.294+0000 7f70e71ff700 1 --2- 172.21.15.119:0/3547104096 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f70d00a1d00 0x7f70d00a41c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.294+0000 7f70e71ff700 1 --2- 172.21.15.119:0/3547104096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70e0107610 0x7f70e01999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.294+0000 7f70e71ff700 1 --2- 172.21.15.119:0/3547104096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70e0101550 0x7f70e019f870 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.294+0000 7f70e71ff700 1 --2- 172.21.15.119:0/3547104096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70e0101f10 0x7f70e019fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.294+0000 7f70e71ff700 1 -- 172.21.15.119:0/3547104096 >> 172.21.15.119:0/3547104096 conn(0x7f70e00fd020 msgr2=0x7f70e00fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:10.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.295+0000 7f70e71ff700 1 -- 172.21.15.119:0/3547104096 shutdown_connections 2023-12-07T10:24:10.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.295+0000 7f70e71ff700 1 -- 172.21.15.119:0/3547104096 wait complete. 2023-12-07T10:24:10.313 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:09 smithi119 ceph-mon[97908]: pgmap v115: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 267 op/s 2023-12-07T10:24:10.313 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:09 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:24:10.313 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:09 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2417932942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:24:10.315 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-07T10:24:10.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.540+0000 7fa4a2f9d700 1 -- 172.21.15.119:0/3081315099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa49c102ed0 msgr2=0x7fa49c10b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.540+0000 7fa4a2f9d700 1 --2- 172.21.15.119:0/3081315099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa49c102ed0 0x7fa49c10b3b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009990 tx=0x7fa48c009ca0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.540+0000 7fa4a2f9d700 1 -- 172.21.15.119:0/3081315099 shutdown_connections 2023-12-07T10:24:10.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.540+0000 7fa4a2f9d700 1 --2- 172.21.15.119:0/3081315099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa49c10b9d0 0x7fa49c10de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.540+0000 7fa4a2f9d700 1 --2- 172.21.15.119:0/3081315099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa49c1025b0 0x7fa49c102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.540+0000 7fa4a2f9d700 1 --2- 172.21.15.119:0/3081315099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa49c102ed0 0x7fa49c10b3b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.540+0000 7fa4a2f9d700 1 -- 172.21.15.119:0/3081315099 >> 172.21.15.119:0/3081315099 conn(0x7fa49c0fbdd0 msgr2=0x7fa49c0fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:10.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.540+0000 7fa4a2f9d700 1 -- 172.21.15.119:0/3081315099 shutdown_connections 2023-12-07T10:24:10.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.540+0000 7fa4a2f9d700 1 -- 172.21.15.119:0/3081315099 wait complete. 2023-12-07T10:24:10.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.541+0000 7fa4a2f9d700 1 Processor -- start 2023-12-07T10:24:10.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.541+0000 7fa4a2f9d700 1 -- start start 2023-12-07T10:24:10.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.542+0000 7fa4a2f9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa49c1025b0 0x7fa49c19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.542+0000 7fa4a2f9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa49c102ed0 0x7fa49c19fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.542+0000 7fa4a2f9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa49c10b9d0 0x7fa49c199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.542+0000 7fa4a2f9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c199fa0 con 0x7fa49c1025b0 2023-12-07T10:24:10.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.542+0000 7fa4a2f9d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c19a110 con 0x7fa49c102ed0 2023-12-07T10:24:10.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.542+0000 7fa4a2f9d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c19a2d0 con 0x7fa49c10b9d0 2023-12-07T10:24:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.542+0000 7fa4a179a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa49c102ed0 0x7fa49c19fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.542+0000 7fa4a279c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa49c10b9d0 0x7fa49c199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.542+0000 7fa4a1f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa49c1025b0 0x7fa49c19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.542+0000 7fa4a179a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa49c102ed0 0x7fa49c19fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:56016/0 (socket says 172.21.15.119:56016) 2023-12-07T10:24:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.542+0000 7fa4a179a700 1 -- 172.21.15.119:0/64843481 learned_addr learned my addr 172.21.15.119:0/64843481 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.543+0000 7fa4a179a700 1 -- 172.21.15.119:0/64843481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa49c10b9d0 msgr2=0x7fa49c199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.543+0000 7fa4a179a700 1 --2- 172.21.15.119:0/64843481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa49c10b9d0 0x7fa49c199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.543+0000 7fa4a179a700 1 -- 172.21.15.119:0/64843481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa49c1025b0 msgr2=0x7fa49c19f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.543+0000 7fa4a179a700 1 --2- 172.21.15.119:0/64843481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa49c1025b0 0x7fa49c19f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.543+0000 7fa4a179a700 1 -- 172.21.15.119:0/64843481 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48c0095f0 con 0x7fa49c102ed0 2023-12-07T10:24:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.543+0000 7fa4a279c700 1 --2- 172.21.15.119:0/64843481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa49c10b9d0 0x7fa49c199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.543+0000 7fa4a179a700 1 --2- 172.21.15.119:0/64843481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa49c102ed0 0x7fa49c19fd50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa48c005910 tx=0x7fa48c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.543+0000 7fa492ffd700 1 -- 172.21.15.119:0/64843481 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c01d070 con 0x7fa49c102ed0 2023-12-07T10:24:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.543+0000 7fa4a2f9d700 1 -- 172.21.15.119:0/64843481 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49c19a550 con 0x7fa49c102ed0 2023-12-07T10:24:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.543+0000 7fa492ffd700 1 -- 172.21.15.119:0/64843481 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa48c022470 con 0x7fa49c102ed0 2023-12-07T10:24:10.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.543+0000 7fa492ffd700 1 -- 172.21.15.119:0/64843481 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c00bee0 con 0x7fa49c102ed0 2023-12-07T10:24:10.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.543+0000 7fa4a2f9d700 1 -- 172.21.15.119:0/64843481 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49c1a6800 con 0x7fa49c102ed0 2023-12-07T10:24:10.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.547+0000 7fa492ffd700 1 -- 172.21.15.119:0/64843481 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fa48c0225e0 con 0x7fa49c102ed0 2023-12-07T10:24:10.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.548+0000 7fa4b7ad9b80 1 Processor -- start 2023-12-07T10:24:10.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.548+0000 7fa4b7ad9b80 1 -- start start 2023-12-07T10:24:10.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.549+0000 7fa4b7ad9b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e81ef5b9e0 0x55e81ef5bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.549+0000 7fa4b7ad9b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e81ef5c300 0x55e81ef5fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.549+0000 7fa4b7ad9b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e81ef602e0 0x55e81ef60740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.549+0000 7fa4b7ad9b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81eecf7c0 con 0x55e81ef602e0 2023-12-07T10:24:10.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.549+0000 7fa4b7ad9b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81eeb67a0 con 0x55e81ef5b9e0 2023-12-07T10:24:10.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.549+0000 7fa4b7ad9b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81eeb40c0 con 0x55e81ef5c300 2023-12-07T10:24:10.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.549+0000 7fa4a279c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e81ef5b9e0 0x55e81ef5bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.549+0000 7fa492ffd700 1 --2- 172.21.15.119:0/64843481 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa4880a1c80 0x7fa4880a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.549+0000 7fa4a279c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e81ef5b9e0 0x55e81ef5bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:56028/0 (socket says 172.21.15.119:56028) 2023-12-07T10:24:10.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.549+0000 7fa4a279c700 1 -- 172.21.15.119:0/4248995447 learned_addr learned my addr 172.21.15.119:0/4248995447 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:10.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.549+0000 7fa4a1f9b700 1 --2- 172.21.15.119:0/4248995447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e81ef5c300 0x55e81ef5fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.549+0000 7fa4a179a700 1 --2- 172.21.15.119:0/4248995447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e81ef602e0 0x55e81ef60740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.549+0000 7fa492ffd700 1 -- 172.21.15.119:0/64843481 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fa48c0cef00 con 0x7fa49c102ed0 2023-12-07T10:24:10.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.550+0000 7fa4a279c700 1 --2- 172.21.15.119:0/64843481 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa4880a1c80 0x7fa4880a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.550+0000 7fa4a1f9b700 1 -- 172.21.15.119:0/4248995447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e81ef5b9e0 msgr2=0x55e81ef5bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.550+0000 7fa4a1f9b700 1 --2- 172.21.15.119:0/4248995447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e81ef5b9e0 0x55e81ef5bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.550+0000 7fa4a1f9b700 1 -- 172.21.15.119:0/4248995447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e81ef602e0 msgr2=0x55e81ef60740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.550+0000 7fa4a1f9b700 1 --2- 172.21.15.119:0/4248995447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e81ef602e0 0x55e81ef60740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.550+0000 7fa4a1f9b700 1 -- 172.21.15.119:0/4248995447 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e81ed2c630 con 0x55e81ef5c300 2023-12-07T10:24:10.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.550+0000 7fa4a1f9b700 1 --2- 172.21.15.119:0/4248995447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e81ef5c300 0x55e81ef5fda0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa498009cb0 tx=0x7fa498011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfbfe150ddc70a6a server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:10.552 INFO:tasks.workunit.client.0.smithi119.stdout: 1 216683 6.71 MB/sec execute 23 sec latency 11.155 ms 2023-12-07T10:24:10.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.550+0000 7fa4a279c700 1 --2- 172.21.15.119:0/64843481 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa4880a1c80 0x7fa4880a4140 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa494003fd0 tx=0x7fa494004b40 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:10.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.550+0000 7fa487fff700 1 -- 172.21.15.119:0/4248995447 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498011d20 con 0x55e81ef5c300 2023-12-07T10:24:10.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.550+0000 7fa487fff700 1 -- 172.21.15.119:0/4248995447 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa498011e80 con 0x55e81ef5c300 2023-12-07T10:24:10.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.550+0000 7fa487fff700 1 -- 172.21.15.119:0/4248995447 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498005280 con 0x55e81ef5c300 2023-12-07T10:24:10.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.551+0000 7fa4b7ad9b80 1 -- 172.21.15.119:0/4248995447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e81ef5c300 msgr2=0x55e81ef5fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.551+0000 7fa4b7ad9b80 1 --2- 172.21.15.119:0/4248995447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e81ef5c300 0x55e81ef5fda0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa498009cb0 tx=0x7fa498011440 comp rx=0 tx=0).stop 2023-12-07T10:24:10.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.551+0000 7fa4b7ad9b80 1 -- 172.21.15.119:0/4248995447 shutdown_connections 2023-12-07T10:24:10.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.551+0000 7fa4b7ad9b80 1 --2- 172.21.15.119:0/4248995447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e81ef602e0 0x55e81ef60740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.551+0000 7fa4b7ad9b80 1 --2- 172.21.15.119:0/4248995447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e81ef5b9e0 0x55e81ef5bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.551+0000 7fa4b7ad9b80 1 --2- 172.21.15.119:0/4248995447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e81ef5c300 0x55e81ef5fda0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.551+0000 7fa4b7ad9b80 1 -- 172.21.15.119:0/4248995447 >> 172.21.15.119:0/4248995447 conn(0x55e81ee8a360 msgr2=0x55e81ef56170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:10.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.551+0000 7fa4b7ad9b80 1 -- 172.21.15.119:0/4248995447 shutdown_connections 2023-12-07T10:24:10.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.551+0000 7fa4b7ad9b80 1 -- 172.21.15.119:0/4248995447 wait complete. 2023-12-07T10:24:10.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.552+0000 7fa4b7ad9b80 1 -- start start 2023-12-07T10:24:10.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.552+0000 7fa4b7ad9b80 1 Processor -- start 2023-12-07T10:24:10.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.553+0000 7fa4b7ad9b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e81ef5b9e0 0x55e81ef64df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.553+0000 7fa4b7ad9b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e81ef5c300 0x55e81ef65330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.553+0000 7fa4b7ad9b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e81ef602e0 0x55e81ef65870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.553+0000 7fa4b7ad9b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81eeaf9b0 con 0x55e81ef5b9e0 2023-12-07T10:24:10.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.553+0000 7fa4b7ad9b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81ee69560 con 0x55e81ef602e0 2023-12-07T10:24:10.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.553+0000 7fa4b7ad9b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81ee69200 con 0x55e81ef5c300 2023-12-07T10:24:10.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.553+0000 7fa4b7ad9b80 10 client.0 _ll_register_callbacks cb 0x55e81ee50940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:10.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.553+0000 7fa4a179a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e81ef5c300 0x55e81ef65330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.553+0000 7fa4a179a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e81ef5c300 0x55e81ef65330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55698/0 (socket says 172.21.15.119:55698) 2023-12-07T10:24:10.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.553+0000 7fa4a179a700 1 -- 172.21.15.119:0/3171346666 learned_addr learned my addr 172.21.15.119:0/3171346666 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:10.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.553+0000 7fa4a279c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e81ef602e0 0x55e81ef65870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.553+0000 7fa4a1f9b700 1 --2- 172.21.15.119:0/3171346666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e81ef5b9e0 0x55e81ef64df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.553+0000 7fa4a179a700 1 -- 172.21.15.119:0/3171346666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e81ef602e0 msgr2=0x55e81ef65870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.553+0000 7fa4a179a700 1 --2- 172.21.15.119:0/3171346666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e81ef602e0 0x55e81ef65870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.553+0000 7fa4a179a700 1 -- 172.21.15.119:0/3171346666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e81ef5b9e0 msgr2=0x55e81ef64df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.553+0000 7fa4a179a700 1 --2- 172.21.15.119:0/3171346666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e81ef5b9e0 0x55e81ef64df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.553+0000 7fa4a179a700 1 -- 172.21.15.119:0/3171346666 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4980098e0 con 0x55e81ef5c300 2023-12-07T10:24:10.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.553+0000 7fa4a279c700 1 --2- 172.21.15.119:0/3171346666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e81ef602e0 0x55e81ef65870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:10.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.553+0000 7fa4a1f9b700 1 --2- 172.21.15.119:0/3171346666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e81ef5b9e0 0x55e81ef64df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:10.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.554+0000 7fa4a179a700 1 --2- 172.21.15.119:0/3171346666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e81ef5c300 0x55e81ef65330 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00f740 tx=0x7fa48c06b7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83e6e42ec78413c8 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:10.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.554+0000 7fa4857fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:24:10.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.554+0000 7fa4857fa700 1 -- 172.21.15.119:0/3171346666 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c09b2e0 con 0x55e81ef5c300 2023-12-07T10:24:10.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.554+0000 7fa4857fa700 1 -- 172.21.15.119:0/3171346666 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fa48c0670d0 con 0x55e81ef5c300 2023-12-07T10:24:10.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.554+0000 7fa4b7ad9b80 1 -- 172.21.15.119:0/3171346666 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e81ef2a7a0 con 0x55e81ef5c300 2023-12-07T10:24:10.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.554+0000 7fa4857fa700 1 -- 172.21.15.119:0/3171346666 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa48c09c080 con 0x55e81ef5c300 2023-12-07T10:24:10.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.554+0000 7fa4b7ad9b80 10 client.26899 fetch_fsmap learned FSMap version 37 2023-12-07T10:24:10.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.554+0000 7fa4b7ad9b80 1 -- 172.21.15.119:0/3171346666 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55e81ecd8de0 con 0x55e81ef5c300 2023-12-07T10:24:10.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.554+0000 7fa4857fa700 1 -- 172.21.15.119:0/3171346666 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e81ef2a7a0 con 0x55e81ef5c300 2023-12-07T10:24:10.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.554+0000 7fa4857fa700 1 -- 172.21.15.119:0/3171346666 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 37) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa48c0d0070 con 0x55e81ef5c300 2023-12-07T10:24:10.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.555+0000 7fa4b7ad9b80 10 client.26899 fetch_fsmap finished waiting for FSMap version 37 2023-12-07T10:24:10.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.555+0000 7fa4b7ad9b80 10 client.26899 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-07T10:24:10.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.555+0000 7fa4b7ad9b80 20 client.26899 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:10.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.555+0000 7fa4b7ad9b80 1 --2- 172.21.15.119:0/3171346666 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x55e81ef751d0 0x55e81ef775f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.555+0000 7fa4b7ad9b80 4 client.26899 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:24:10.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.555+0000 7fa4b7ad9b80 1 -- 172.21.15.119:0/3171346666 --> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e81ee75fd0 con 0x55e81ef751d0 2023-12-07T10:24:10.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.555+0000 7fa4a1f9b700 1 --2- 172.21.15.119:0/3171346666 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x55e81ef751d0 0x55e81ef775f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.556+0000 7fa4857fa700 10 client.26899 ms_handle_connect on v2:172.21.15.159:6836/3106097520 2023-12-07T10:24:10.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.556+0000 7fa4a1f9b700 1 --2- 172.21.15.119:0/3171346666 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x55e81ef751d0 0x55e81ef775f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd20e3751fea01b8 server_cookie=49e64e48b6a30cdb in_seq=0 out_seq=0 2023-12-07T10:24:10.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.556+0000 7fa4857fa700 1 -- 172.21.15.119:0/3171346666 <== mds.2 v2:172.21.15.159:6836/3106097520 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e81ee75fd0 con 0x55e81ef751d0 2023-12-07T10:24:10.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.556+0000 7fa4857fa700 10 client.26899 handle_command_reply: tid=0 2023-12-07T10:24:10.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.556+0000 7fa4b7ad9b80 1 client.26899 shutdown 2023-12-07T10:24:10.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.557+0000 7fa4b7ad9b80 1 -- 172.21.15.119:0/3171346666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e81ef5c300 msgr2=0x55e81ef65330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.557+0000 7fa4b7ad9b80 1 --2- 172.21.15.119:0/3171346666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e81ef5c300 0x55e81ef65330 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00f740 tx=0x7fa48c06b7f0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.557+0000 7fa4b7ad9b80 1 -- 172.21.15.119:0/3171346666 shutdown_connections 2023-12-07T10:24:10.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.557+0000 7fa4b7ad9b80 1 --2- 172.21.15.119:0/3171346666 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x55e81ef751d0 0x55e81ef775f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.557+0000 7fa4b7ad9b80 1 --2- 172.21.15.119:0/3171346666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e81ef602e0 0x55e81ef65870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.557+0000 7fa4b7ad9b80 1 --2- 172.21.15.119:0/3171346666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e81ef5b9e0 0x55e81ef64df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.557+0000 7fa4b7ad9b80 1 --2- 172.21.15.119:0/3171346666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e81ef5c300 0x55e81ef65330 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.557+0000 7fa4857fa700 0 client.26899 ms_handle_reset on v2:172.21.15.159:6836/3106097520 2023-12-07T10:24:10.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.557+0000 7fa4b7ad9b80 1 -- 172.21.15.119:0/3171346666 >> 172.21.15.119:0/3171346666 conn(0x55e81ee8a360 msgr2=0x55e81ef5a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:10.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.557+0000 7fa4b7ad9b80 1 -- 172.21.15.119:0/3171346666 shutdown_connections 2023-12-07T10:24:10.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.557+0000 7fa4b7ad9b80 1 -- 172.21.15.119:0/3171346666 wait complete. 2023-12-07T10:24:10.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.557+0000 7fa4b7ad9b80 20 client.26899 trim_cache size 0 max 16384 2023-12-07T10:24:10.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.571+0000 7fa4b7ad9b80 1 Processor -- start 2023-12-07T10:24:10.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.571+0000 7fa4b7ad9b80 1 -- start start 2023-12-07T10:24:10.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.572+0000 7fa4b7ad9b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e81ef5b9e0 0x55e81ef5bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.572+0000 7fa4b7ad9b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e81ef5c300 0x55e81ef606c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.572+0000 7fa4b7ad9b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e81ef602e0 0x55e81ef5b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.572+0000 7fa4b7ad9b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81ed2e920 con 0x55e81ef602e0 2023-12-07T10:24:10.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.572+0000 7fa4b7ad9b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81ef5b670 con 0x55e81ef5c300 2023-12-07T10:24:10.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.572+0000 7fa4b7ad9b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81ef525f0 con 0x55e81ef5b9e0 2023-12-07T10:24:10.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.572+0000 7fa4a179a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e81ef5c300 0x55e81ef606c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.572+0000 7fa4a179a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e81ef5c300 0x55e81ef606c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:56052/0 (socket says 172.21.15.119:56052) 2023-12-07T10:24:10.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.572+0000 7fa4a179a700 1 -- 172.21.15.119:0/841260974 learned_addr learned my addr 172.21.15.119:0/841260974 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:10.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.572+0000 7fa4a1f9b700 1 --2- 172.21.15.119:0/841260974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e81ef5b9e0 0x55e81ef5bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.572+0000 7fa4a279c700 1 --2- 172.21.15.119:0/841260974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e81ef602e0 0x55e81ef5b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.572+0000 7fa4a179a700 1 -- 172.21.15.119:0/841260974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e81ef5b9e0 msgr2=0x55e81ef5bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.572+0000 7fa4a179a700 1 --2- 172.21.15.119:0/841260974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e81ef5b9e0 0x55e81ef5bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.572+0000 7fa4a179a700 1 -- 172.21.15.119:0/841260974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e81ef602e0 msgr2=0x55e81ef5b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.572+0000 7fa4a179a700 1 --2- 172.21.15.119:0/841260974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e81ef602e0 0x55e81ef5b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.572+0000 7fa4a179a700 1 -- 172.21.15.119:0/841260974 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e81ecc1c60 con 0x55e81ef5c300 2023-12-07T10:24:10.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.573+0000 7fa4a179a700 1 --2- 172.21.15.119:0/841260974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e81ef5c300 0x55e81ef606c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0bb1a0 tx=0x7fa48c022f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18c26d40261bf132 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:10.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.573+0000 7fa490ff9700 1 -- 172.21.15.119:0/841260974 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c0b41f0 con 0x55e81ef5c300 2023-12-07T10:24:10.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.573+0000 7fa490ff9700 1 -- 172.21.15.119:0/841260974 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa48c09b2e0 con 0x55e81ef5c300 2023-12-07T10:24:10.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.573+0000 7fa490ff9700 1 -- 172.21.15.119:0/841260974 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c09fb60 con 0x55e81ef5c300 2023-12-07T10:24:10.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.574+0000 7fa4b7ad9b80 1 -- 172.21.15.119:0/841260974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e81ef5c300 msgr2=0x55e81ef606c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.574+0000 7fa4b7ad9b80 1 --2- 172.21.15.119:0/841260974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e81ef5c300 0x55e81ef606c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0bb1a0 tx=0x7fa48c022f90 comp rx=0 tx=0).stop 2023-12-07T10:24:10.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.574+0000 7fa4b7ad9b80 1 -- 172.21.15.119:0/841260974 shutdown_connections 2023-12-07T10:24:10.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.574+0000 7fa4b7ad9b80 1 --2- 172.21.15.119:0/841260974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e81ef602e0 0x55e81ef5b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.574+0000 7fa4b7ad9b80 1 --2- 172.21.15.119:0/841260974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e81ef5b9e0 0x55e81ef5bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.574+0000 7fa4b7ad9b80 1 --2- 172.21.15.119:0/841260974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e81ef5c300 0x55e81ef606c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.574+0000 7fa4b7ad9b80 1 -- 172.21.15.119:0/841260974 >> 172.21.15.119:0/841260974 conn(0x55e81ee8a360 msgr2=0x55e81ef57d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:10.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.574+0000 7fa4b7ad9b80 1 -- 172.21.15.119:0/841260974 shutdown_connections 2023-12-07T10:24:10.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.574+0000 7fa4b7ad9b80 1 -- 172.21.15.119:0/841260974 wait complete. 2023-12-07T10:24:10.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.574+0000 7fa4b7ad9b80 1 -- start start 2023-12-07T10:24:10.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.575+0000 7fa4b7ad9b80 1 Processor -- start 2023-12-07T10:24:10.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.575+0000 7fa4b7ad9b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e81ef5b9e0 0x55e81ef6c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.575+0000 7fa4b7ad9b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e81ef5c300 0x55e81ef6a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.575+0000 7fa4b7ad9b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e81ef602e0 0x55e81ef6aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.575+0000 7fa4b7ad9b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81ef6afc0 con 0x55e81ef5b9e0 2023-12-07T10:24:10.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.575+0000 7fa4b7ad9b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81ef6b100 con 0x55e81ef602e0 2023-12-07T10:24:10.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.575+0000 7fa4b7ad9b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81ef6b2c0 con 0x55e81ef5c300 2023-12-07T10:24:10.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.575+0000 7fa4b7ad9b80 10 client.0 _ll_register_callbacks cb 0x55e81ee50940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:10.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.575+0000 7fa4a1f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e81ef5b9e0 0x55e81ef6c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.575+0000 7fa4a1f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e81ef5b9e0 0x55e81ef6c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36436/0 (socket says 172.21.15.119:36436) 2023-12-07T10:24:10.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.575+0000 7fa4a1f9b700 1 -- 172.21.15.119:0/3765235163 learned_addr learned my addr 172.21.15.119:0/3765235163 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:10.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.575+0000 7fa4a179a700 1 --2- 172.21.15.119:0/3765235163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e81ef5c300 0x55e81ef6a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.575+0000 7fa4a279c700 1 --2- 172.21.15.119:0/3765235163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e81ef602e0 0x55e81ef6aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.576+0000 7fa4a1f9b700 1 -- 172.21.15.119:0/3765235163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e81ef5c300 msgr2=0x55e81ef6a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.576+0000 7fa4a1f9b700 1 --2- 172.21.15.119:0/3765235163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e81ef5c300 0x55e81ef6a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.576+0000 7fa4a1f9b700 1 -- 172.21.15.119:0/3765235163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e81ef602e0 msgr2=0x55e81ef6aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.576+0000 7fa4a1f9b700 1 --2- 172.21.15.119:0/3765235163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e81ef602e0 0x55e81ef6aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.576+0000 7fa4a1f9b700 1 -- 172.21.15.119:0/3765235163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e81ecd8de0 con 0x55e81ef5b9e0 2023-12-07T10:24:10.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.576+0000 7fa4a1f9b700 1 --2- 172.21.15.119:0/3765235163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e81ef5b9e0 0x55e81ef6c750 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa498009cb0 tx=0x7fa4980096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed84bfeb169880b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:10.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.576+0000 7fa4857fa700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:24:10.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.576+0000 7fa4857fa700 1 -- 172.21.15.119:0/3765235163 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498011b20 con 0x55e81ef5b9e0 2023-12-07T10:24:10.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.576+0000 7fa4b7ad9b80 1 -- 172.21.15.119:0/3765235163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e81ef4e9a0 con 0x55e81ef5b9e0 2023-12-07T10:24:10.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.576+0000 7fa4857fa700 1 -- 172.21.15.119:0/3765235163 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fa498013e40 con 0x55e81ef5b9e0 2023-12-07T10:24:10.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.577+0000 7fa4857fa700 1 -- 172.21.15.119:0/3765235163 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa498011c80 con 0x55e81ef5b9e0 2023-12-07T10:24:10.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.577+0000 7fa4b7ad9b80 10 client.16902 fetch_fsmap learned FSMap version 37 2023-12-07T10:24:10.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.577+0000 7fa4b7ad9b80 1 -- 172.21.15.119:0/3765235163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55e81ecff250 con 0x55e81ef5b9e0 2023-12-07T10:24:10.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.577+0000 7fa4857fa700 1 -- 172.21.15.119:0/3765235163 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e81ef4e9a0 con 0x55e81ef5b9e0 2023-12-07T10:24:10.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.577+0000 7fa4857fa700 1 -- 172.21.15.119:0/3765235163 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 37) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa498003a70 con 0x55e81ef5b9e0 2023-12-07T10:24:10.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.577+0000 7fa4b7ad9b80 10 client.16902 fetch_fsmap finished waiting for FSMap version 37 2023-12-07T10:24:10.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.577+0000 7fa4b7ad9b80 10 client.16902 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-07T10:24:10.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.577+0000 7fa4b7ad9b80 20 client.16902 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:10.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.577+0000 7fa4b7ad9b80 1 --2- 172.21.15.119:0/3765235163 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x55e81efcbba0 0x55e81efcdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.577+0000 7fa4b7ad9b80 4 client.16902 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:24:10.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.577+0000 7fa4b7ad9b80 1 -- 172.21.15.119:0/3765235163 --> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e81ee68490 con 0x55e81efcbba0 2023-12-07T10:24:10.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.577+0000 7fa4a179a700 1 --2- 172.21.15.119:0/3765235163 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x55e81efcbba0 0x55e81efcdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.578+0000 7fa4a179a700 1 --2- 172.21.15.119:0/3765235163 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x55e81efcbba0 0x55e81efcdfc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=19e0901c64a899ef server_cookie=bfeab0e79260ad1d in_seq=0 out_seq=0 2023-12-07T10:24:10.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.578+0000 7fa4857fa700 10 client.16902 ms_handle_connect on v2:172.21.15.159:6836/3106097520 2023-12-07T10:24:10.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.578+0000 7fa4857fa700 1 -- 172.21.15.119:0/3765235163 <== mds.2 v2:172.21.15.159:6836/3106097520 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e81ee68490 con 0x55e81efcbba0 2023-12-07T10:24:10.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.578+0000 7fa4857fa700 10 client.16902 handle_command_reply: tid=0 2023-12-07T10:24:10.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.578+0000 7fa4b7ad9b80 1 client.16902 shutdown 2023-12-07T10:24:10.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.578+0000 7fa4b7ad9b80 1 -- 172.21.15.119:0/3765235163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e81ef5b9e0 msgr2=0x55e81ef6c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.578+0000 7fa4b7ad9b80 1 --2- 172.21.15.119:0/3765235163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e81ef5b9e0 0x55e81ef6c750 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa498009cb0 tx=0x7fa4980096d0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.578+0000 7fa4b7ad9b80 1 -- 172.21.15.119:0/3765235163 shutdown_connections 2023-12-07T10:24:10.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.578+0000 7fa4b7ad9b80 1 --2- 172.21.15.119:0/3765235163 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x55e81efcbba0 0x55e81efcdfc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.578+0000 7fa4b7ad9b80 1 --2- 172.21.15.119:0/3765235163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e81ef602e0 0x55e81ef6aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.578+0000 7fa4b7ad9b80 1 --2- 172.21.15.119:0/3765235163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e81ef5b9e0 0x55e81ef6c750 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.578+0000 7fa4b7ad9b80 1 --2- 172.21.15.119:0/3765235163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e81ef5c300 0x55e81ef6a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.578+0000 7fa4b7ad9b80 1 -- 172.21.15.119:0/3765235163 >> 172.21.15.119:0/3765235163 conn(0x55e81ee8a360 msgr2=0x55e81ef57d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:10.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.578+0000 7fa4857fa700 0 client.16902 ms_handle_reset on v2:172.21.15.159:6836/3106097520 2023-12-07T10:24:10.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.579+0000 7fa4b7ad9b80 1 -- 172.21.15.119:0/3765235163 shutdown_connections 2023-12-07T10:24:10.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.579+0000 7fa4b7ad9b80 1 -- 172.21.15.119:0/3765235163 wait complete. 2023-12-07T10:24:10.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.579+0000 7fa4b7ad9b80 20 client.16902 trim_cache size 0 max 16384 2023-12-07T10:24:10.583 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:24:10.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.579+0000 7fa4a2f9d700 1 -- 172.21.15.119:0/64843481 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa4880a1c80 msgr2=0x7fa4880a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.579+0000 7fa4a2f9d700 1 --2- 172.21.15.119:0/64843481 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa4880a1c80 0x7fa4880a4140 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa494003fd0 tx=0x7fa494004b40 comp rx=0 tx=0).stop 2023-12-07T10:24:10.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.579+0000 7fa4a2f9d700 1 -- 172.21.15.119:0/64843481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa49c102ed0 msgr2=0x7fa49c19fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.579+0000 7fa4a2f9d700 1 --2- 172.21.15.119:0/64843481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa49c102ed0 0x7fa49c19fd50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa48c005910 tx=0x7fa48c004730 comp rx=0 tx=0).stop 2023-12-07T10:24:10.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.579+0000 7fa4a2f9d700 1 -- 172.21.15.119:0/64843481 shutdown_connections 2023-12-07T10:24:10.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.579+0000 7fa4a2f9d700 1 --2- 172.21.15.119:0/64843481 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa4880a1c80 0x7fa4880a4140 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.579+0000 7fa4a2f9d700 1 --2- 172.21.15.119:0/64843481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa49c10b9d0 0x7fa49c199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.579+0000 7fa4a2f9d700 1 --2- 172.21.15.119:0/64843481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa49c1025b0 0x7fa49c19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.579+0000 7fa4a2f9d700 1 --2- 172.21.15.119:0/64843481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa49c102ed0 0x7fa49c19fd50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.579+0000 7fa4a2f9d700 1 -- 172.21.15.119:0/64843481 >> 172.21.15.119:0/64843481 conn(0x7fa49c0fbdd0 msgr2=0x7fa49c101410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:10.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.580+0000 7fa4a2f9d700 1 -- 172.21.15.119:0/64843481 shutdown_connections 2023-12-07T10:24:10.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.580+0000 7fa4a2f9d700 1 -- 172.21.15.119:0/64843481 wait complete. 2023-12-07T10:24:10.600 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-07T10:24:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.820+0000 7fb1561ff700 1 -- 172.21.15.119:0/3810238011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb150068a50 msgr2=0x7fb150068e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.820+0000 7fb1561ff700 1 --2- 172.21.15.119:0/3810238011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb150068a50 0x7fb150068e30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb140009990 tx=0x7fb140009ca0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.820+0000 7fb1561ff700 1 -- 172.21.15.119:0/3810238011 shutdown_connections 2023-12-07T10:24:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.820+0000 7fb1561ff700 1 --2- 172.21.15.119:0/3810238011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb150106ad0 0x7fb15010e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.820+0000 7fb1561ff700 1 --2- 172.21.15.119:0/3810238011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb150068a50 0x7fb150068e30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.820+0000 7fb1561ff700 1 --2- 172.21.15.119:0/3810238011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb150106110 0x7fb150106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.820+0000 7fb1561ff700 1 -- 172.21.15.119:0/3810238011 >> 172.21.15.119:0/3810238011 conn(0x7fb150077610 msgr2=0x7fb150077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.820+0000 7fb1561ff700 1 -- 172.21.15.119:0/3810238011 shutdown_connections 2023-12-07T10:24:10.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.820+0000 7fb1561ff700 1 -- 172.21.15.119:0/3810238011 wait complete. 2023-12-07T10:24:10.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.821+0000 7fb1561ff700 1 Processor -- start 2023-12-07T10:24:10.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.821+0000 7fb1561ff700 1 -- start start 2023-12-07T10:24:10.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.821+0000 7fb1561ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb150068a50 0x7fb15019f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.821+0000 7fb1561ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb150106110 0x7fb15019fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.821+0000 7fb1561ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb150106ad0 0x7fb150199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.821+0000 7fb1561ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15019a110 con 0x7fb150106110 2023-12-07T10:24:10.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.821+0000 7fb1561ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15019a280 con 0x7fb150106ad0 2023-12-07T10:24:10.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.821+0000 7fb1561ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15019a440 con 0x7fb150068a50 2023-12-07T10:24:10.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.822+0000 7fb14effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb150106110 0x7fb15019fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.822+0000 7fb14ffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb150106ad0 0x7fb150199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.822+0000 7fb14ffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb150106ad0 0x7fb150199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:56076/0 (socket says 172.21.15.119:56076) 2023-12-07T10:24:10.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.822+0000 7fb14effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb150106110 0x7fb15019fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36466/0 (socket says 172.21.15.119:36466) 2023-12-07T10:24:10.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.822+0000 7fb14ffff700 1 -- 172.21.15.119:0/1400546544 learned_addr learned my addr 172.21.15.119:0/1400546544 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:10.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.822+0000 7fb14f7fe700 1 --2- 172.21.15.119:0/1400546544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb150068a50 0x7fb15019f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.822+0000 7fb14ffff700 1 -- 172.21.15.119:0/1400546544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb150068a50 msgr2=0x7fb15019f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.822+0000 7fb14ffff700 1 --2- 172.21.15.119:0/1400546544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb150068a50 0x7fb15019f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.822+0000 7fb14ffff700 1 -- 172.21.15.119:0/1400546544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb150106110 msgr2=0x7fb15019fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.822+0000 7fb14ffff700 1 --2- 172.21.15.119:0/1400546544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb150106110 0x7fb15019fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.822+0000 7fb14ffff700 1 -- 172.21.15.119:0/1400546544 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1400095f0 con 0x7fb150106ad0 2023-12-07T10:24:10.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.822+0000 7fb14ffff700 1 --2- 172.21.15.119:0/1400546544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb150106ad0 0x7fb150199ab0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb14800efc0 tx=0x7fb14800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:10.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.823+0000 7fb14cff9700 1 -- 172.21.15.119:0/1400546544 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb148018070 con 0x7fb150106ad0 2023-12-07T10:24:10.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.823+0000 7fb14cff9700 1 -- 172.21.15.119:0/1400546544 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb14800f040 con 0x7fb150106ad0 2023-12-07T10:24:10.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.823+0000 7fb1561ff700 1 -- 172.21.15.119:0/1400546544 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb15019a6c0 con 0x7fb150106ad0 2023-12-07T10:24:10.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.823+0000 7fb14cff9700 1 -- 172.21.15.119:0/1400546544 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14800cc10 con 0x7fb150106ad0 2023-12-07T10:24:10.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.823+0000 7fb1561ff700 1 -- 172.21.15.119:0/1400546544 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1501a6970 con 0x7fb150106ad0 2023-12-07T10:24:10.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.826+0000 7fb14cff9700 1 -- 172.21.15.119:0/1400546544 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fb14800cd70 con 0x7fb150106ad0 2023-12-07T10:24:10.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.828+0000 7fb14cff9700 1 --2- 172.21.15.119:0/1400546544 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb1380a1c80 0x7fb1380a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.828+0000 7fb14cff9700 1 -- 172.21.15.119:0/1400546544 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fb1480cd6a0 con 0x7fb150106ad0 2023-12-07T10:24:10.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.828+0000 7fb14f7fe700 1 --2- 172.21.15.119:0/1400546544 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb1380a1c80 0x7fb1380a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.828+0000 7fb169b66b80 1 Processor -- start 2023-12-07T10:24:10.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.828+0000 7fb169b66b80 1 -- start start 2023-12-07T10:24:10.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.829+0000 7fb169b66b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcd3ff0b40 0x55dcd3ff0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.829+0000 7fb169b66b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcd3ff1460 0x55dcd3ff4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.829+0000 7fb169b66b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcd3ff5150 0x55dcd3ff5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.829+0000 7fb169b66b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd3f647c0 con 0x55dcd3ff5150 2023-12-07T10:24:10.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.829+0000 7fb169b66b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd3f4b7a0 con 0x55dcd3ff0b40 2023-12-07T10:24:10.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.829+0000 7fb169b66b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd3f490c0 con 0x55dcd3ff1460 2023-12-07T10:24:10.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.829+0000 7fb14f7fe700 1 --2- 172.21.15.119:0/1400546544 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb1380a1c80 0x7fb1380a4140 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb140006010 tx=0x7fb14000b260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:10.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.829+0000 7fb14f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcd3ff0b40 0x55dcd3ff0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.829+0000 7fb14effd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcd3ff1460 0x55dcd3ff4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.829+0000 7fb14f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcd3ff0b40 0x55dcd3ff0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:56084/0 (socket says 172.21.15.119:56084) 2023-12-07T10:24:10.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.829+0000 7fb14f7fe700 1 -- 172.21.15.119:0/2367879187 learned_addr learned my addr 172.21.15.119:0/2367879187 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:10.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.829+0000 7fb14ffff700 1 --2- 172.21.15.119:0/2367879187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcd3ff5150 0x55dcd3ff5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.830+0000 7fb14effd700 1 -- 172.21.15.119:0/2367879187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcd3ff0b40 msgr2=0x55dcd3ff0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.830+0000 7fb14effd700 1 --2- 172.21.15.119:0/2367879187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcd3ff0b40 0x55dcd3ff0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.830+0000 7fb14effd700 1 -- 172.21.15.119:0/2367879187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcd3ff5150 msgr2=0x55dcd3ff5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.830+0000 7fb14effd700 1 --2- 172.21.15.119:0/2367879187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcd3ff5150 0x55dcd3ff5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.830+0000 7fb14effd700 1 -- 172.21.15.119:0/2367879187 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcd3dc1630 con 0x55dcd3ff1460 2023-12-07T10:24:10.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.830+0000 7fb14effd700 1 --2- 172.21.15.119:0/2367879187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcd3ff1460 0x55dcd3ff4c10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb144009cb0 tx=0x7fb144011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1518247ef3a7e100 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:10.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.830+0000 7fb135ffb700 1 -- 172.21.15.119:0/2367879187 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb144011c60 con 0x55dcd3ff1460 2023-12-07T10:24:10.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.830+0000 7fb135ffb700 1 -- 172.21.15.119:0/2367879187 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb144004510 con 0x55dcd3ff1460 2023-12-07T10:24:10.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.830+0000 7fb135ffb700 1 -- 172.21.15.119:0/2367879187 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb144005280 con 0x55dcd3ff1460 2023-12-07T10:24:10.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.831+0000 7fb169b66b80 1 -- 172.21.15.119:0/2367879187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcd3ff1460 msgr2=0x55dcd3ff4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.831+0000 7fb169b66b80 1 --2- 172.21.15.119:0/2367879187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcd3ff1460 0x55dcd3ff4c10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb144009cb0 tx=0x7fb144011440 comp rx=0 tx=0).stop 2023-12-07T10:24:10.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.831+0000 7fb169b66b80 1 -- 172.21.15.119:0/2367879187 shutdown_connections 2023-12-07T10:24:10.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.831+0000 7fb169b66b80 1 --2- 172.21.15.119:0/2367879187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcd3ff5150 0x55dcd3ff5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.831+0000 7fb169b66b80 1 --2- 172.21.15.119:0/2367879187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcd3ff0b40 0x55dcd3ff0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.831+0000 7fb169b66b80 1 --2- 172.21.15.119:0/2367879187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcd3ff1460 0x55dcd3ff4c10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.831+0000 7fb169b66b80 1 -- 172.21.15.119:0/2367879187 >> 172.21.15.119:0/2367879187 conn(0x55dcd3f1f360 msgr2=0x55dcd3feb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:10.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.831+0000 7fb169b66b80 1 -- 172.21.15.119:0/2367879187 shutdown_connections 2023-12-07T10:24:10.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.831+0000 7fb169b66b80 1 -- 172.21.15.119:0/2367879187 wait complete. 2023-12-07T10:24:10.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.832+0000 7fb169b66b80 1 -- start start 2023-12-07T10:24:10.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.832+0000 7fb169b66b80 1 Processor -- start 2023-12-07T10:24:10.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.832+0000 7fb169b66b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcd3ff0b40 0x55dcd3ffc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.832+0000 7fb169b66b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcd3ff1460 0x55dcd3ff9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.833+0000 7fb169b66b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcd3ff5150 0x55dcd3ffa370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.833+0000 7fb169b66b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd3f449b0 con 0x55dcd3ff0b40 2023-12-07T10:24:10.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.833+0000 7fb169b66b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd3efe560 con 0x55dcd3ff5150 2023-12-07T10:24:10.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.833+0000 7fb169b66b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd3efe200 con 0x55dcd3ff1460 2023-12-07T10:24:10.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.833+0000 7fb169b66b80 10 client.0 _ll_register_callbacks cb 0x55dcd3ee5940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:10.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.833+0000 7fb14f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcd3ff0b40 0x55dcd3ffc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.833+0000 7fb14f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcd3ff0b40 0x55dcd3ffc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36494/0 (socket says 172.21.15.119:36494) 2023-12-07T10:24:10.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.833+0000 7fb14f7fe700 1 -- 172.21.15.119:0/1075540753 learned_addr learned my addr 172.21.15.119:0/1075540753 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:10.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.833+0000 7fb14effd700 1 --2- 172.21.15.119:0/1075540753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcd3ff1460 0x55dcd3ff9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.833+0000 7fb14ffff700 1 --2- 172.21.15.119:0/1075540753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcd3ff5150 0x55dcd3ffa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.833+0000 7fb14f7fe700 1 -- 172.21.15.119:0/1075540753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcd3ff1460 msgr2=0x55dcd3ff9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.833+0000 7fb14f7fe700 1 --2- 172.21.15.119:0/1075540753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcd3ff1460 0x55dcd3ff9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.833+0000 7fb14f7fe700 1 -- 172.21.15.119:0/1075540753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcd3ff5150 msgr2=0x55dcd3ffa370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.833+0000 7fb14f7fe700 1 --2- 172.21.15.119:0/1075540753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcd3ff5150 0x55dcd3ffa370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.833+0000 7fb14f7fe700 1 -- 172.21.15.119:0/1075540753 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1440098e0 con 0x55dcd3ff0b40 2023-12-07T10:24:10.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.834+0000 7fb14f7fe700 1 --2- 172.21.15.119:0/1075540753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcd3ff0b40 0x55dcd3ffc920 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb14000b6f0 tx=0x7fb14001a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732e7382108aa7f8 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:10.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.834+0000 7fb12effd700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:24:10.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.834+0000 7fb12effd700 1 -- 172.21.15.119:0/1075540753 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14000f660 con 0x55dcd3ff0b40 2023-12-07T10:24:10.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.834+0000 7fb169b66b80 1 -- 172.21.15.119:0/1075540753 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dcd3fe3b00 con 0x55dcd3ff0b40 2023-12-07T10:24:10.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.834+0000 7fb12effd700 1 -- 172.21.15.119:0/1075540753 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fb14000be80 con 0x55dcd3ff0b40 2023-12-07T10:24:10.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.834+0000 7fb12effd700 1 -- 172.21.15.119:0/1075540753 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb14000f7c0 con 0x55dcd3ff0b40 2023-12-07T10:24:10.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.834+0000 7fb169b66b80 10 client.16914 fetch_fsmap learned FSMap version 37 2023-12-07T10:24:10.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.834+0000 7fb169b66b80 1 -- 172.21.15.119:0/1075540753 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55dcd3d6dde0 con 0x55dcd3ff0b40 2023-12-07T10:24:10.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.834+0000 7fb12effd700 1 -- 172.21.15.119:0/1075540753 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dcd3fe3b00 con 0x55dcd3ff0b40 2023-12-07T10:24:10.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.835+0000 7fb12effd700 1 -- 172.21.15.119:0/1075540753 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 37) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fb14002fb20 con 0x55dcd3ff0b40 2023-12-07T10:24:10.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.835+0000 7fb169b66b80 10 client.16914 fetch_fsmap finished waiting for FSMap version 37 2023-12-07T10:24:10.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.835+0000 7fb169b66b80 10 client.16914 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2023-12-07T10:24:10.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.835+0000 7fb169b66b80 20 client.16914 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:10.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.835+0000 7fb169b66b80 1 --2- 172.21.15.119:0/1075540753 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55dcd400a4f0 0x55dcd400c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.835+0000 7fb169b66b80 4 client.16914 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:24:10.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.835+0000 7fb169b66b80 1 -- 172.21.15.119:0/1075540753 --> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dcd3fbf7a0 con 0x55dcd400a4f0 2023-12-07T10:24:10.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.835+0000 7fb14effd700 1 --2- 172.21.15.119:0/1075540753 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55dcd400a4f0 0x55dcd400c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.836+0000 7fb14effd700 1 --2- 172.21.15.119:0/1075540753 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55dcd400a4f0 0x55dcd400c910 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b8c7fc2dc002277 server_cookie=89eb20e9afd15902 in_seq=0 out_seq=0 2023-12-07T10:24:10.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.836+0000 7fb12effd700 10 client.16914 ms_handle_connect on v2:172.21.15.119:6838/1920762602 2023-12-07T10:24:10.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.836+0000 7fb12effd700 1 -- 172.21.15.119:0/1075540753 <== mds.1 v2:172.21.15.119:6838/1920762602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dcd3fbf7a0 con 0x55dcd400a4f0 2023-12-07T10:24:10.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.836+0000 7fb12effd700 10 client.16914 handle_command_reply: tid=0 2023-12-07T10:24:10.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.836+0000 7fb169b66b80 1 client.16914 shutdown 2023-12-07T10:24:10.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.837+0000 7fb169b66b80 1 -- 172.21.15.119:0/1075540753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcd3ff0b40 msgr2=0x55dcd3ffc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.837+0000 7fb169b66b80 1 --2- 172.21.15.119:0/1075540753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcd3ff0b40 0x55dcd3ffc920 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb14000b6f0 tx=0x7fb14001a050 comp rx=0 tx=0).stop 2023-12-07T10:24:10.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.837+0000 7fb169b66b80 1 -- 172.21.15.119:0/1075540753 shutdown_connections 2023-12-07T10:24:10.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.837+0000 7fb169b66b80 1 --2- 172.21.15.119:0/1075540753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcd3ff5150 0x55dcd3ffa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.837+0000 7fb169b66b80 1 --2- 172.21.15.119:0/1075540753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcd3ff0b40 0x55dcd3ffc920 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.837+0000 7fb169b66b80 1 --2- 172.21.15.119:0/1075540753 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55dcd400a4f0 0x55dcd400c910 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.837+0000 7fb169b66b80 1 --2- 172.21.15.119:0/1075540753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcd3ff1460 0x55dcd3ff9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.837+0000 7fb169b66b80 1 -- 172.21.15.119:0/1075540753 >> 172.21.15.119:0/1075540753 conn(0x55dcd3f1f360 msgr2=0x55dcd3ff6620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:10.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.837+0000 7fb12effd700 0 client.16914 ms_handle_reset on v2:172.21.15.119:6838/1920762602 2023-12-07T10:24:10.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.837+0000 7fb169b66b80 1 -- 172.21.15.119:0/1075540753 shutdown_connections 2023-12-07T10:24:10.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.837+0000 7fb169b66b80 1 -- 172.21.15.119:0/1075540753 wait complete. 2023-12-07T10:24:10.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.837+0000 7fb169b66b80 20 client.16914 trim_cache size 0 max 16384 2023-12-07T10:24:10.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.852+0000 7fb169b66b80 1 Processor -- start 2023-12-07T10:24:10.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.852+0000 7fb169b66b80 1 -- start start 2023-12-07T10:24:10.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.853+0000 7fb169b66b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcd3ff0b40 0x55dcd4011270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.853+0000 7fb169b66b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcd3ff1460 0x55dcd4001350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.853+0000 7fb169b66b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcd3ff5150 0x55dcd4001890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.853+0000 7fb169b66b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd3dc3920 con 0x55dcd3ff5150 2023-12-07T10:24:10.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.853+0000 7fb169b66b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd3ff81d0 con 0x55dcd3ff1460 2023-12-07T10:24:10.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.853+0000 7fb169b66b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd3ff3e50 con 0x55dcd3ff0b40 2023-12-07T10:24:10.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.853+0000 7fb14effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcd3ff1460 0x55dcd4001350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.853+0000 7fb14ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcd3ff5150 0x55dcd4001890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.853+0000 7fb14effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcd3ff1460 0x55dcd4001350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:56102/0 (socket says 172.21.15.119:56102) 2023-12-07T10:24:10.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.853+0000 7fb14effd700 1 -- 172.21.15.119:0/3576970575 learned_addr learned my addr 172.21.15.119:0/3576970575 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:10.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.853+0000 7fb14f7fe700 1 --2- 172.21.15.119:0/3576970575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcd3ff0b40 0x55dcd4011270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.853+0000 7fb14effd700 1 -- 172.21.15.119:0/3576970575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcd3ff0b40 msgr2=0x55dcd4011270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.853+0000 7fb14effd700 1 --2- 172.21.15.119:0/3576970575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcd3ff0b40 0x55dcd4011270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.853+0000 7fb14effd700 1 -- 172.21.15.119:0/3576970575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcd3ff5150 msgr2=0x55dcd4001890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.853+0000 7fb14effd700 1 --2- 172.21.15.119:0/3576970575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcd3ff5150 0x55dcd4001890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.853+0000 7fb14effd700 1 -- 172.21.15.119:0/3576970575 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcd3d56c60 con 0x55dcd3ff1460 2023-12-07T10:24:10.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.854+0000 7fb14effd700 1 --2- 172.21.15.119:0/3576970575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcd3ff1460 0x55dcd4001350 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb144000c00 tx=0x7fb144009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c148ec75b017d137 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:10.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.854+0000 7fb1367fc700 1 -- 172.21.15.119:0/3576970575 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb144004050 con 0x55dcd3ff1460 2023-12-07T10:24:10.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.854+0000 7fb1367fc700 1 -- 172.21.15.119:0/3576970575 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1440041b0 con 0x55dcd3ff1460 2023-12-07T10:24:10.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.854+0000 7fb1367fc700 1 -- 172.21.15.119:0/3576970575 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb144028900 con 0x55dcd3ff1460 2023-12-07T10:24:10.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.855+0000 7fb169b66b80 1 -- 172.21.15.119:0/3576970575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcd3ff1460 msgr2=0x55dcd4001350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.855+0000 7fb169b66b80 1 --2- 172.21.15.119:0/3576970575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcd3ff1460 0x55dcd4001350 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb144000c00 tx=0x7fb144009370 comp rx=0 tx=0).stop 2023-12-07T10:24:10.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.855+0000 7fb169b66b80 1 -- 172.21.15.119:0/3576970575 shutdown_connections 2023-12-07T10:24:10.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.855+0000 7fb169b66b80 1 --2- 172.21.15.119:0/3576970575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcd3ff5150 0x55dcd4001890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.855+0000 7fb169b66b80 1 --2- 172.21.15.119:0/3576970575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcd3ff0b40 0x55dcd4011270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.855+0000 7fb169b66b80 1 --2- 172.21.15.119:0/3576970575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcd3ff1460 0x55dcd4001350 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.855+0000 7fb169b66b80 1 -- 172.21.15.119:0/3576970575 >> 172.21.15.119:0/3576970575 conn(0x55dcd3f1f360 msgr2=0x55dcd3ffc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:10.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.855+0000 7fb169b66b80 1 -- 172.21.15.119:0/3576970575 shutdown_connections 2023-12-07T10:24:10.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.855+0000 7fb169b66b80 1 -- 172.21.15.119:0/3576970575 wait complete. 2023-12-07T10:24:10.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.855+0000 7fb169b66b80 1 -- start start 2023-12-07T10:24:10.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.856+0000 7fb169b66b80 1 Processor -- start 2023-12-07T10:24:10.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.856+0000 7fb169b66b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcd3ff0b40 0x55dcd3fefb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.856+0000 7fb169b66b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcd3ff1460 0x55dcd3ff0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.856+0000 7fb169b66b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcd3ff5150 0x55dcd405f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.856+0000 7fb169b66b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd3ff0730 con 0x55dcd3ff0b40 2023-12-07T10:24:10.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.856+0000 7fb169b66b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd405ff30 con 0x55dcd3ff1460 2023-12-07T10:24:10.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.856+0000 7fb169b66b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcd4060070 con 0x55dcd3ff5150 2023-12-07T10:24:10.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.856+0000 7fb169b66b80 10 client.0 _ll_register_callbacks cb 0x55dcd3fe6440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:10.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.856+0000 7fb14ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcd3ff5150 0x55dcd405f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.856+0000 7fb14effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcd3ff1460 0x55dcd3ff0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.856+0000 7fb14ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcd3ff5150 0x55dcd405f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55758/0 (socket says 172.21.15.119:55758) 2023-12-07T10:24:10.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.856+0000 7fb14ffff700 1 -- 172.21.15.119:0/1756636878 learned_addr learned my addr 172.21.15.119:0/1756636878 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:10.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.856+0000 7fb14f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcd3ff0b40 0x55dcd3fefb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.857+0000 7fb14ffff700 1 -- 172.21.15.119:0/1756636878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcd3ff1460 msgr2=0x55dcd3ff0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.857+0000 7fb14ffff700 1 --2- 172.21.15.119:0/1756636878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcd3ff1460 0x55dcd3ff0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.857+0000 7fb14ffff700 1 -- 172.21.15.119:0/1756636878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcd3ff0b40 msgr2=0x55dcd3fefb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:10.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.857+0000 7fb14ffff700 1 --2- 172.21.15.119:0/1756636878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcd3ff0b40 0x55dcd3fefb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.857+0000 7fb14ffff700 1 -- 172.21.15.119:0/1756636878 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb14401d040 con 0x55dcd3ff5150 2023-12-07T10:24:10.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.857+0000 7fb14effd700 1 --2- 172.21.15.119:0/1756636878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcd3ff1460 0x55dcd3ff0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:24:10.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.857+0000 7fb14ffff700 1 --2- 172.21.15.119:0/1756636878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcd3ff5150 0x55dcd405f9f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb1480965e0 tx=0x7fb1480098a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5121dfb049435457 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:10.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.857+0000 7fb12effd700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:24:10.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.857+0000 7fb12effd700 1 -- 172.21.15.119:0/1756636878 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1480997d0 con 0x55dcd3ff5150 2023-12-07T10:24:10.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.857+0000 7fb169b66b80 1 -- 172.21.15.119:0/1756636878 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb144004510 con 0x55dcd3ff5150 2023-12-07T10:24:10.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.857+0000 7fb12effd700 1 -- 172.21.15.119:0/1756636878 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fb14809a0e0 con 0x55dcd3ff5150 2023-12-07T10:24:10.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.857+0000 7fb12effd700 1 -- 172.21.15.119:0/1756636878 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb14809e230 con 0x55dcd3ff5150 2023-12-07T10:24:10.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.857+0000 7fb169b66b80 10 client.26923 fetch_fsmap learned FSMap version 37 2023-12-07T10:24:10.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.857+0000 7fb169b66b80 1 -- 172.21.15.119:0/1756636878 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55dcd3d94250 con 0x55dcd3ff5150 2023-12-07T10:24:10.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.857+0000 7fb12effd700 1 -- 172.21.15.119:0/1756636878 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb144004510 con 0x55dcd3ff5150 2023-12-07T10:24:10.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.858+0000 7fb12effd700 1 -- 172.21.15.119:0/1756636878 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 37) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fb1480a00b0 con 0x55dcd3ff5150 2023-12-07T10:24:10.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.858+0000 7fb169b66b80 10 client.26923 fetch_fsmap finished waiting for FSMap version 37 2023-12-07T10:24:10.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.858+0000 7fb169b66b80 10 client.26923 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2023-12-07T10:24:10.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.858+0000 7fb169b66b80 20 client.26923 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:10.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.858+0000 7fb169b66b80 1 --2- 172.21.15.119:0/1756636878 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55dcd4061850 0x55dcd4063c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:10.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.858+0000 7fb169b66b80 4 client.26923 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:24:10.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.858+0000 7fb169b66b80 1 -- 172.21.15.119:0/1756636878 --> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dcd3f0afd0 con 0x55dcd4061850 2023-12-07T10:24:10.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.858+0000 7fb14f7fe700 1 --2- 172.21.15.119:0/1756636878 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55dcd4061850 0x55dcd4063c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:10.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.858+0000 7fb14f7fe700 1 --2- 172.21.15.119:0/1756636878 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55dcd4061850 0x55dcd4063c70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae3a602bbc18cefa server_cookie=340f65fceda4aad5 in_seq=0 out_seq=0 2023-12-07T10:24:10.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.858+0000 7fb12effd700 10 client.26923 ms_handle_connect on v2:172.21.15.119:6838/1920762602 2023-12-07T10:24:10.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.859+0000 7fb12effd700 1 -- 172.21.15.119:0/1756636878 <== mds.1 v2:172.21.15.119:6838/1920762602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dcd3f0afd0 con 0x55dcd4061850 2023-12-07T10:24:10.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.859+0000 7fb12effd700 10 client.26923 handle_command_reply: tid=0 2023-12-07T10:24:10.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.859+0000 7fb169b66b80 1 client.26923 shutdown 2023-12-07T10:24:10.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.859+0000 7fb169b66b80 1 -- 172.21.15.119:0/1756636878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcd3ff5150 msgr2=0x55dcd405f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.859+0000 7fb169b66b80 1 --2- 172.21.15.119:0/1756636878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcd3ff5150 0x55dcd405f9f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb1480965e0 tx=0x7fb1480098a0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.859+0000 7fb169b66b80 1 -- 172.21.15.119:0/1756636878 shutdown_connections 2023-12-07T10:24:10.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.859+0000 7fb169b66b80 1 --2- 172.21.15.119:0/1756636878 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x55dcd4061850 0x55dcd4063c70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.859+0000 7fb169b66b80 1 --2- 172.21.15.119:0/1756636878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcd3ff5150 0x55dcd405f9f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.859+0000 7fb169b66b80 1 --2- 172.21.15.119:0/1756636878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dcd3ff0b40 0x55dcd3fefb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.859+0000 7fb169b66b80 1 --2- 172.21.15.119:0/1756636878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55dcd3ff1460 0x55dcd3ff0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.859+0000 7fb12effd700 0 client.26923 ms_handle_reset on v2:172.21.15.119:6838/1920762602 2023-12-07T10:24:10.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.859+0000 7fb169b66b80 1 -- 172.21.15.119:0/1756636878 >> 172.21.15.119:0/1756636878 conn(0x55dcd3f1f360 msgr2=0x55dcd4011270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:10.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.859+0000 7fb169b66b80 1 -- 172.21.15.119:0/1756636878 shutdown_connections 2023-12-07T10:24:10.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.859+0000 7fb169b66b80 1 -- 172.21.15.119:0/1756636878 wait complete. 2023-12-07T10:24:10.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.859+0000 7fb169b66b80 20 client.26923 trim_cache size 0 max 16384 2023-12-07T10:24:10.866 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:24:10.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.860+0000 7fb1561ff700 1 -- 172.21.15.119:0/1400546544 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb1380a1c80 msgr2=0x7fb1380a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.860+0000 7fb1561ff700 1 --2- 172.21.15.119:0/1400546544 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb1380a1c80 0x7fb1380a4140 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb140006010 tx=0x7fb14000b260 comp rx=0 tx=0).stop 2023-12-07T10:24:10.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.860+0000 7fb1561ff700 1 -- 172.21.15.119:0/1400546544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb150106ad0 msgr2=0x7fb150199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:10.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.860+0000 7fb1561ff700 1 --2- 172.21.15.119:0/1400546544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb150106ad0 0x7fb150199ab0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb14800efc0 tx=0x7fb14800c3a0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.860+0000 7fb1561ff700 1 -- 172.21.15.119:0/1400546544 shutdown_connections 2023-12-07T10:24:10.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.860+0000 7fb1561ff700 1 --2- 172.21.15.119:0/1400546544 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb1380a1c80 0x7fb1380a4140 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.860+0000 7fb1561ff700 1 --2- 172.21.15.119:0/1400546544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb150106ad0 0x7fb150199ab0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.860+0000 7fb1561ff700 1 --2- 172.21.15.119:0/1400546544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb150068a50 0x7fb15019f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.860+0000 7fb1561ff700 1 --2- 172.21.15.119:0/1400546544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb150106110 0x7fb15019fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:10.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.860+0000 7fb1561ff700 1 -- 172.21.15.119:0/1400546544 >> 172.21.15.119:0/1400546544 conn(0x7fb150077610 msgr2=0x7fb15010cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:10.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.860+0000 7fb1561ff700 1 -- 172.21.15.119:0/1400546544 shutdown_connections 2023-12-07T10:24:10.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:10.860+0000 7fb1561ff700 1 -- 172.21.15.119:0/1400546544 wait complete. 2023-12-07T10:24:10.881 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-07T10:24:11.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.121+0000 7f05841ff700 1 -- 172.21.15.119:0/1563578515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f057c109780 msgr2=0x7f057c109b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:11.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.121+0000 7f05841ff700 1 --2- 172.21.15.119:0/1563578515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f057c109780 0x7f057c109b60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0578009990 tx=0x7f0578009ca0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.121+0000 7f05841ff700 1 -- 172.21.15.119:0/1563578515 shutdown_connections 2023-12-07T10:24:11.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.121+0000 7f05841ff700 1 --2- 172.21.15.119:0/1563578515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f057c104130 0x7f057c110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.121+0000 7f05841ff700 1 --2- 172.21.15.119:0/1563578515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f057c109780 0x7f057c109b60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.121+0000 7f05841ff700 1 --2- 172.21.15.119:0/1563578515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f057c103770 0x7f057c103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.121+0000 7f05841ff700 1 -- 172.21.15.119:0/1563578515 >> 172.21.15.119:0/1563578515 conn(0x7f057c0ff220 msgr2=0x7f057c101640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:11.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.121+0000 7f05841ff700 1 -- 172.21.15.119:0/1563578515 shutdown_connections 2023-12-07T10:24:11.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.121+0000 7f05841ff700 1 -- 172.21.15.119:0/1563578515 wait complete. 2023-12-07T10:24:11.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.122+0000 7f05841ff700 1 Processor -- start 2023-12-07T10:24:11.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.123+0000 7f05841ff700 1 -- start start 2023-12-07T10:24:11.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.123+0000 7f05841ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f057c103770 0x7f057c19f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:11.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.123+0000 7f05841ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f057c104130 0x7f057c19fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:11.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.123+0000 7f05841ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f057c109780 0x7f057c1999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:11.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.123+0000 7f05841ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c19a020 con 0x7f057c103770 2023-12-07T10:24:11.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.123+0000 7f05841ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c19a190 con 0x7f057c109780 2023-12-07T10:24:11.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.123+0000 7f05841ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c19a350 con 0x7f057c104130 2023-12-07T10:24:11.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.123+0000 7f058179a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f057c104130 0x7f057c19fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:11.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.123+0000 7f058179a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f057c104130 0x7f057c19fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55774/0 (socket says 172.21.15.119:55774) 2023-12-07T10:24:11.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.123+0000 7f058179a700 1 -- 172.21.15.119:0/4127599804 learned_addr learned my addr 172.21.15.119:0/4127599804 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:11.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.123+0000 7f058279c700 1 --2- 172.21.15.119:0/4127599804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f057c109780 0x7f057c1999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:11.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.123+0000 7f0581f9b700 1 --2- 172.21.15.119:0/4127599804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f057c103770 0x7f057c19f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:11.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.124+0000 7f058279c700 1 -- 172.21.15.119:0/4127599804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f057c104130 msgr2=0x7f057c19fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:11.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.124+0000 7f058279c700 1 --2- 172.21.15.119:0/4127599804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f057c104130 0x7f057c19fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.124+0000 7f058279c700 1 -- 172.21.15.119:0/4127599804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f057c103770 msgr2=0x7f057c19f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:11.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.124+0000 7f058279c700 1 --2- 172.21.15.119:0/4127599804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f057c103770 0x7f057c19f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.124+0000 7f058279c700 1 -- 172.21.15.119:0/4127599804 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05780095f0 con 0x7f057c109780 2023-12-07T10:24:11.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.124+0000 7f058179a700 1 --2- 172.21.15.119:0/4127599804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f057c104130 0x7f057c19fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:11.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.124+0000 7f0581f9b700 1 --2- 172.21.15.119:0/4127599804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f057c103770 0x7f057c19f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:11.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.124+0000 7f058279c700 1 --2- 172.21.15.119:0/4127599804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f057c109780 0x7f057c1999c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f057400efc0 tx=0x7f057400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:11.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.124+0000 7f0572ffd700 1 -- 172.21.15.119:0/4127599804 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0574018070 con 0x7f057c109780 2023-12-07T10:24:11.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.125+0000 7f0572ffd700 1 -- 172.21.15.119:0/4127599804 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f057400f040 con 0x7f057c109780 2023-12-07T10:24:11.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.125+0000 7f0572ffd700 1 -- 172.21.15.119:0/4127599804 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057400cd40 con 0x7f057c109780 2023-12-07T10:24:11.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.125+0000 7f05841ff700 1 -- 172.21.15.119:0/4127599804 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f057c19a5d0 con 0x7f057c109780 2023-12-07T10:24:11.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.125+0000 7f05841ff700 1 -- 172.21.15.119:0/4127599804 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057c1a6880 con 0x7f057c109780 2023-12-07T10:24:11.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.129+0000 7f0572ffd700 1 -- 172.21.15.119:0/4127599804 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f0574003a00 con 0x7f057c109780 2023-12-07T10:24:11.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.130+0000 7f0597a56b80 1 Processor -- start 2023-12-07T10:24:11.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.131+0000 7f0597a56b80 1 -- start start 2023-12-07T10:24:11.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.131+0000 7f0572ffd700 1 --2- 172.21.15.119:0/4127599804 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f05680a1d00 0x7f05680a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:11.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.131+0000 7f0597a56b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613452de350 0x5613452de730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:11.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.131+0000 7f0597a56b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613452dec70 0x5613452e2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:11.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.131+0000 7f0597a56b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452e2960 0x5613452e2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:11.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.131+0000 7f0597a56b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613452497c0 con 0x5613452de350 2023-12-07T10:24:11.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.131+0000 7f0597a56b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613452307a0 con 0x5613452dec70 2023-12-07T10:24:11.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.131+0000 7f0597a56b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134522e0c0 con 0x5613452e2960 2023-12-07T10:24:11.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.131+0000 7f0572ffd700 1 -- 172.21.15.119:0/4127599804 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f05740cd780 con 0x7f057c109780 2023-12-07T10:24:11.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.131+0000 7f058279c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452e2960 0x5613452e2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:11.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.131+0000 7f058179a700 1 --2- 172.21.15.119:0/4127599804 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f05680a1d00 0x7f05680a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:11.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.132+0000 7f058279c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452e2960 0x5613452e2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55784/0 (socket says 172.21.15.119:55784) 2023-12-07T10:24:11.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.132+0000 7f058279c700 1 -- 172.21.15.119:0/5828000 learned_addr learned my addr 172.21.15.119:0/5828000 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:11.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.132+0000 7f058179a700 1 --2- 172.21.15.119:0/5828000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613452dec70 0x5613452e2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:11.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.132+0000 7f0581f9b700 1 --2- 172.21.15.119:0/5828000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613452de350 0x5613452de730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:11.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.132+0000 7f058279c700 1 -- 172.21.15.119:0/5828000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613452dec70 msgr2=0x5613452e2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:11.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.132+0000 7f058279c700 1 --2- 172.21.15.119:0/5828000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613452dec70 0x5613452e2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.132+0000 7f058279c700 1 -- 172.21.15.119:0/5828000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613452de350 msgr2=0x5613452de730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:11.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.132+0000 7f058279c700 1 --2- 172.21.15.119:0/5828000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613452de350 0x5613452de730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.132+0000 7f058279c700 1 -- 172.21.15.119:0/5828000 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613450a6630 con 0x5613452e2960 2023-12-07T10:24:11.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.132+0000 7f058279c700 1 --2- 172.21.15.119:0/5828000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452e2960 0x5613452e2d80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f057400c6f0 tx=0x7f057409a010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b2455c21063bfa1 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:11.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.132+0000 7f058179a700 1 --2- 172.21.15.119:0/4127599804 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f05680a1d00 0x7f05680a41c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f056c009fd0 tx=0x7f056c0134e0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:11.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.132+0000 7f0567fff700 1 -- 172.21.15.119:0/5828000 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057401b320 con 0x5613452e2960 2023-12-07T10:24:11.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.132+0000 7f0567fff700 1 -- 172.21.15.119:0/5828000 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0574020110 con 0x5613452e2960 2023-12-07T10:24:11.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.132+0000 7f0567fff700 1 -- 172.21.15.119:0/5828000 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057401b570 con 0x5613452e2960 2023-12-07T10:24:11.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.133+0000 7f0597a56b80 1 -- 172.21.15.119:0/5828000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452e2960 msgr2=0x5613452e2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:11.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.133+0000 7f0597a56b80 1 --2- 172.21.15.119:0/5828000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452e2960 0x5613452e2d80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f057400c6f0 tx=0x7f057409a010 comp rx=0 tx=0).stop 2023-12-07T10:24:11.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.133+0000 7f0597a56b80 1 -- 172.21.15.119:0/5828000 shutdown_connections 2023-12-07T10:24:11.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.133+0000 7f0597a56b80 1 --2- 172.21.15.119:0/5828000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452e2960 0x5613452e2d80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.133+0000 7f0597a56b80 1 --2- 172.21.15.119:0/5828000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613452de350 0x5613452de730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.133+0000 7f0597a56b80 1 --2- 172.21.15.119:0/5828000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613452dec70 0x5613452e2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.133+0000 7f0597a56b80 1 -- 172.21.15.119:0/5828000 >> 172.21.15.119:0/5828000 conn(0x561345204360 msgr2=0x5613452d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:11.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.134+0000 7f0597a56b80 1 -- 172.21.15.119:0/5828000 shutdown_connections 2023-12-07T10:24:11.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.134+0000 7f0597a56b80 1 -- 172.21.15.119:0/5828000 wait complete. 2023-12-07T10:24:11.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.134+0000 7f0597a56b80 1 -- start start 2023-12-07T10:24:11.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.134+0000 7f0597a56b80 1 Processor -- start 2023-12-07T10:24:11.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.135+0000 7f0597a56b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613452de350 0x5613452ea150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:11.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.135+0000 7f0597a56b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613452dec70 0x5613452e7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:11.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.135+0000 7f0597a56b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452e2960 0x5613452e7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:11.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.135+0000 7f0597a56b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613452299b0 con 0x5613452de350 2023-12-07T10:24:11.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.135+0000 7f0597a56b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613451e3560 con 0x5613452dec70 2023-12-07T10:24:11.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.135+0000 7f0597a56b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613451e3200 con 0x5613452e2960 2023-12-07T10:24:11.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.135+0000 7f0597a56b80 10 client.0 _ll_register_callbacks cb 0x5613451ca940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:11.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.135+0000 7f058279c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452e2960 0x5613452e7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:11.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.135+0000 7f058279c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452e2960 0x5613452e7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55794/0 (socket says 172.21.15.119:55794) 2023-12-07T10:24:11.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.135+0000 7f058279c700 1 -- 172.21.15.119:0/2184645325 learned_addr learned my addr 172.21.15.119:0/2184645325 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:11.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.135+0000 7f0581f9b700 1 --2- 172.21.15.119:0/2184645325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613452de350 0x5613452ea150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:11.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.136+0000 7f058179a700 1 --2- 172.21.15.119:0/2184645325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613452dec70 0x5613452e7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:11.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.136+0000 7f058279c700 1 -- 172.21.15.119:0/2184645325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613452dec70 msgr2=0x5613452e7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:11.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.136+0000 7f058279c700 1 --2- 172.21.15.119:0/2184645325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613452dec70 0x5613452e7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.136+0000 7f058279c700 1 -- 172.21.15.119:0/2184645325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613452de350 msgr2=0x5613452ea150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:11.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.136+0000 7f058279c700 1 --2- 172.21.15.119:0/2184645325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613452de350 0x5613452ea150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.136+0000 7f058279c700 1 -- 172.21.15.119:0/2184645325 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057c1a6880 con 0x5613452e2960 2023-12-07T10:24:11.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.136+0000 7f0581f9b700 1 --2- 172.21.15.119:0/2184645325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613452de350 0x5613452ea150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:11.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.136+0000 7f058179a700 1 --2- 172.21.15.119:0/2184645325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613452dec70 0x5613452e7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:24:11.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.136+0000 7f058279c700 1 --2- 172.21.15.119:0/2184645325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452e2960 0x5613452e7b80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f05740090a0 tx=0x7f05740cb010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bc98e75cc2b0545 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:11.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.136+0000 7f05657fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:24:11.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.136+0000 7f05657fa700 1 -- 172.21.15.119:0/2184645325 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057401b320 con 0x5613452e2960 2023-12-07T10:24:11.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.136+0000 7f0597a56b80 1 -- 172.21.15.119:0/2184645325 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5613452c8b00 con 0x5613452e2960 2023-12-07T10:24:11.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.137+0000 7f05657fa700 1 -- 172.21.15.119:0/2184645325 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f0574004ad0 con 0x5613452e2960 2023-12-07T10:24:11.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.137+0000 7f05657fa700 1 -- 172.21.15.119:0/2184645325 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f057409a940 con 0x5613452e2960 2023-12-07T10:24:11.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.137+0000 7f0597a56b80 10 client.26935 fetch_fsmap learned FSMap version 37 2023-12-07T10:24:11.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.137+0000 7f0597a56b80 1 -- 172.21.15.119:0/2184645325 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x561345052de0 con 0x5613452e2960 2023-12-07T10:24:11.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.137+0000 7f05657fa700 1 -- 172.21.15.119:0/2184645325 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5613452c8b00 con 0x5613452e2960 2023-12-07T10:24:11.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.138+0000 7f05657fa700 1 -- 172.21.15.119:0/2184645325 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 37) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f05740b8270 con 0x5613452e2960 2023-12-07T10:24:11.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.138+0000 7f0597a56b80 10 client.26935 fetch_fsmap finished waiting for FSMap version 37 2023-12-07T10:24:11.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.138+0000 7f0597a56b80 10 client.26935 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:24:11.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.138+0000 7f0597a56b80 20 client.26935 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:11.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.138+0000 7f0597a56b80 1 --2- 172.21.15.119:0/2184645325 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5613452f7820 0x5613452f9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:11.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.138+0000 7f0597a56b80 4 client.26935 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:24:11.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.138+0000 7f0597a56b80 1 -- 172.21.15.119:0/2184645325 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5613452a47a0 con 0x5613452f7820 2023-12-07T10:24:11.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.138+0000 7f0581f9b700 1 --2- 172.21.15.119:0/2184645325 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5613452f7820 0x5613452f9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:11.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.139+0000 7f05657fa700 10 client.26935 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:11.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.139+0000 7f0581f9b700 1 --2- 172.21.15.119:0/2184645325 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5613452f7820 0x5613452f9c40 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16fc75fd555b2d02 server_cookie=4e1e5df69edda37b in_seq=0 out_seq=0 2023-12-07T10:24:11.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.140+0000 7f05657fa700 1 -- 172.21.15.119:0/2184645325 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5613452a47a0 con 0x5613452f7820 2023-12-07T10:24:11.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.140+0000 7f05657fa700 10 client.26935 handle_command_reply: tid=0 2023-12-07T10:24:11.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.140+0000 7f0597a56b80 1 client.26935 shutdown 2023-12-07T10:24:11.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.140+0000 7f0597a56b80 1 -- 172.21.15.119:0/2184645325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452e2960 msgr2=0x5613452e7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:11.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.140+0000 7f0597a56b80 1 --2- 172.21.15.119:0/2184645325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452e2960 0x5613452e7b80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f05740090a0 tx=0x7f05740cb010 comp rx=0 tx=0).stop 2023-12-07T10:24:11.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.140+0000 7f0597a56b80 1 -- 172.21.15.119:0/2184645325 shutdown_connections 2023-12-07T10:24:11.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.140+0000 7f0597a56b80 1 --2- 172.21.15.119:0/2184645325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452e2960 0x5613452e7b80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.140+0000 7f0597a56b80 1 --2- 172.21.15.119:0/2184645325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613452de350 0x5613452ea150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.140+0000 7f0597a56b80 1 --2- 172.21.15.119:0/2184645325 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5613452f7820 0x5613452f9c40 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.140+0000 7f0597a56b80 1 --2- 172.21.15.119:0/2184645325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613452dec70 0x5613452e7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.140+0000 7f0597a56b80 1 -- 172.21.15.119:0/2184645325 >> 172.21.15.119:0/2184645325 conn(0x561345204360 msgr2=0x5613452e3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:11.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.140+0000 7f05657fa700 0 client.26935 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:11.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.140+0000 7f0597a56b80 1 -- 172.21.15.119:0/2184645325 shutdown_connections 2023-12-07T10:24:11.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.141+0000 7f0597a56b80 1 -- 172.21.15.119:0/2184645325 wait complete. 2023-12-07T10:24:11.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.141+0000 7f0597a56b80 20 client.26935 trim_cache size 0 max 16384 2023-12-07T10:24:11.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.155+0000 7f0597a56b80 1 Processor -- start 2023-12-07T10:24:11.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.155+0000 7f0597a56b80 1 -- start start 2023-12-07T10:24:11.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.155+0000 7f0597a56b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452de350 0x5613452e7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:11.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.155+0000 7f0597a56b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613452dec70 0x5613452cad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:11.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.155+0000 7f0597a56b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613452e2960 0x5613452d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:11.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.156+0000 7f0597a56b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613450a8920 con 0x5613452dec70 2023-12-07T10:24:11.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.156+0000 7f0597a56b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613452d17d0 con 0x5613452e2960 2023-12-07T10:24:11.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.156+0000 7f0597a56b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613452e59b0 con 0x5613452de350 2023-12-07T10:24:11.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.156+0000 7f058279c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613452e2960 0x5613452d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:11.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.156+0000 7f0581f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452de350 0x5613452e7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:11.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.156+0000 7f058279c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613452e2960 0x5613452d1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:56172/0 (socket says 172.21.15.119:56172) 2023-12-07T10:24:11.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.156+0000 7f0581f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452de350 0x5613452e7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55804/0 (socket says 172.21.15.119:55804) 2023-12-07T10:24:11.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.156+0000 7f058279c700 1 -- 172.21.15.119:0/359247033 learned_addr learned my addr 172.21.15.119:0/359247033 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:11.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.156+0000 7f058179a700 1 --2- 172.21.15.119:0/359247033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613452dec70 0x5613452cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:11.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.156+0000 7f0581f9b700 1 -- 172.21.15.119:0/359247033 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613452e2960 msgr2=0x5613452d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:11.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.156+0000 7f0581f9b700 1 --2- 172.21.15.119:0/359247033 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613452e2960 0x5613452d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.156+0000 7f0581f9b700 1 -- 172.21.15.119:0/359247033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613452dec70 msgr2=0x5613452cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:11.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.156+0000 7f0581f9b700 1 --2- 172.21.15.119:0/359247033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613452dec70 0x5613452cad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.156+0000 7f0581f9b700 1 -- 172.21.15.119:0/359247033 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56134503bc60 con 0x5613452de350 2023-12-07T10:24:11.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.156+0000 7f058279c700 1 --2- 172.21.15.119:0/359247033 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613452e2960 0x5613452d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:11.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.157+0000 7f0581f9b700 1 --2- 172.21.15.119:0/359247033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452de350 0x5613452e7400 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f057800b2e0 tx=0x7f0578005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de73c0ba09189a15 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:11.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.157+0000 7f0570ff9700 1 -- 172.21.15.119:0/359247033 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057801d040 con 0x5613452de350 2023-12-07T10:24:11.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.157+0000 7f0570ff9700 1 -- 172.21.15.119:0/359247033 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f057801e040 con 0x5613452de350 2023-12-07T10:24:11.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.157+0000 7f0570ff9700 1 -- 172.21.15.119:0/359247033 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05780288d0 con 0x5613452de350 2023-12-07T10:24:11.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.157+0000 7f0597a56b80 1 -- 172.21.15.119:0/359247033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452de350 msgr2=0x5613452e7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:11.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.157+0000 7f0597a56b80 1 --2- 172.21.15.119:0/359247033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452de350 0x5613452e7400 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f057800b2e0 tx=0x7f0578005fd0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.158+0000 7f0597a56b80 1 -- 172.21.15.119:0/359247033 shutdown_connections 2023-12-07T10:24:11.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.158+0000 7f0597a56b80 1 --2- 172.21.15.119:0/359247033 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613452e2960 0x5613452d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.158+0000 7f0597a56b80 1 --2- 172.21.15.119:0/359247033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452de350 0x5613452e7400 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.158+0000 7f0597a56b80 1 --2- 172.21.15.119:0/359247033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613452dec70 0x5613452cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.158+0000 7f0597a56b80 1 -- 172.21.15.119:0/359247033 >> 172.21.15.119:0/359247033 conn(0x561345204360 msgr2=0x5613452e2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:11.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.158+0000 7f0597a56b80 1 -- 172.21.15.119:0/359247033 shutdown_connections 2023-12-07T10:24:11.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.158+0000 7f0597a56b80 1 -- 172.21.15.119:0/359247033 wait complete. 2023-12-07T10:24:11.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.158+0000 7f0597a56b80 1 -- start start 2023-12-07T10:24:11.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.158+0000 7f0597a56b80 1 Processor -- start 2023-12-07T10:24:11.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.159+0000 7f0597a56b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613452de350 0x5613452eeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:11.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.159+0000 7f0597a56b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452dec70 0x5613452d4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:11.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.159+0000 7f0597a56b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613452e2960 0x5613452d4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:11.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.159+0000 7f0597a56b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613452ef280 con 0x5613452e2960 2023-12-07T10:24:11.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.159+0000 7f0597a56b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613452ef3f0 con 0x5613452de350 2023-12-07T10:24:11.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.159+0000 7f0597a56b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613452d53c0 con 0x5613452dec70 2023-12-07T10:24:11.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.159+0000 7f0597a56b80 10 client.0 _ll_register_callbacks cb 0x5613451ca940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:11.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.159+0000 7f0581f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613452de350 0x5613452eeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:11.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.159+0000 7f058179a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452dec70 0x5613452d4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:11.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.159+0000 7f058179a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452dec70 0x5613452d4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55812/0 (socket says 172.21.15.119:55812) 2023-12-07T10:24:11.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.159+0000 7f0581f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613452de350 0x5613452eeba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:56174/0 (socket says 172.21.15.119:56174) 2023-12-07T10:24:11.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.159+0000 7f058179a700 1 -- 172.21.15.119:0/729174052 learned_addr learned my addr 172.21.15.119:0/729174052 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:11.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.159+0000 7f058279c700 1 --2- 172.21.15.119:0/729174052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613452e2960 0x5613452d4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:11.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.160+0000 7f058179a700 1 -- 172.21.15.119:0/729174052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613452de350 msgr2=0x5613452eeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:11.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.160+0000 7f058179a700 1 --2- 172.21.15.119:0/729174052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613452de350 0x5613452eeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.160+0000 7f058179a700 1 -- 172.21.15.119:0/729174052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613452e2960 msgr2=0x5613452d4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:11.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.160+0000 7f058179a700 1 --2- 172.21.15.119:0/729174052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613452e2960 0x5613452d4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.160+0000 7f058179a700 1 -- 172.21.15.119:0/729174052 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561345052de0 con 0x5613452dec70 2023-12-07T10:24:11.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.160+0000 7f058179a700 1 --2- 172.21.15.119:0/729174052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452dec70 0x5613452d4940 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x5613452e8a10 tx=0x7f056c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2804314d3671539a server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:11.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.160+0000 7f05657fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:24:11.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.160+0000 7f05657fa700 1 -- 172.21.15.119:0/729174052 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c026070 con 0x5613452dec70 2023-12-07T10:24:11.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.160+0000 7f0597a56b80 1 -- 172.21.15.119:0/729174052 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0574020110 con 0x5613452dec70 2023-12-07T10:24:11.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.160+0000 7f05657fa700 1 -- 172.21.15.119:0/729174052 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f056c0109a0 con 0x5613452dec70 2023-12-07T10:24:11.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.160+0000 7f05657fa700 1 -- 172.21.15.119:0/729174052 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f056c010b60 con 0x5613452dec70 2023-12-07T10:24:11.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.160+0000 7f0597a56b80 10 client.26941 fetch_fsmap learned FSMap version 37 2023-12-07T10:24:11.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.160+0000 7f0597a56b80 1 -- 172.21.15.119:0/729174052 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7f057c110f40 con 0x5613452dec70 2023-12-07T10:24:11.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.161+0000 7f05657fa700 1 -- 172.21.15.119:0/729174052 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0574020110 con 0x5613452dec70 2023-12-07T10:24:11.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.161+0000 7f05657fa700 1 -- 172.21.15.119:0/729174052 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 37) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f056c01fce0 con 0x5613452dec70 2023-12-07T10:24:11.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.161+0000 7f0597a56b80 10 client.26941 fetch_fsmap finished waiting for FSMap version 37 2023-12-07T10:24:11.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.161+0000 7f0597a56b80 10 client.26941 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:24:11.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.161+0000 7f0597a56b80 20 client.26941 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:11.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.161+0000 7f0597a56b80 1 --2- 172.21.15.119:0/729174052 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56134534e010 0x561345350430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:11.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.161+0000 7f0597a56b80 4 client.26941 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:24:11.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.161+0000 7f0597a56b80 1 -- 172.21.15.119:0/729174052 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5613451effd0 con 0x56134534e010 2023-12-07T10:24:11.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.161+0000 7f0581f9b700 1 --2- 172.21.15.119:0/729174052 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56134534e010 0x561345350430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:11.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.162+0000 7f0581f9b700 1 --2- 172.21.15.119:0/729174052 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56134534e010 0x561345350430 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3c3e465d4834f22 server_cookie=368afd0563a7af56 in_seq=0 out_seq=0 2023-12-07T10:24:11.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.162+0000 7f05657fa700 10 client.26941 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:11.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.162+0000 7f05657fa700 1 -- 172.21.15.119:0/729174052 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5613451effd0 con 0x56134534e010 2023-12-07T10:24:11.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.162+0000 7f05657fa700 10 client.26941 handle_command_reply: tid=0 2023-12-07T10:24:11.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.162+0000 7f0597a56b80 1 client.26941 shutdown 2023-12-07T10:24:11.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.162+0000 7f0597a56b80 1 -- 172.21.15.119:0/729174052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452dec70 msgr2=0x5613452d4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:11.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.162+0000 7f0597a56b80 1 --2- 172.21.15.119:0/729174052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452dec70 0x5613452d4940 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x5613452e8a10 tx=0x7f056c0047c0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.162+0000 7f0597a56b80 1 -- 172.21.15.119:0/729174052 shutdown_connections 2023-12-07T10:24:11.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.162+0000 7f0597a56b80 1 --2- 172.21.15.119:0/729174052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613452e2960 0x5613452d4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.162+0000 7f0597a56b80 1 --2- 172.21.15.119:0/729174052 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56134534e010 0x561345350430 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.162+0000 7f0597a56b80 1 --2- 172.21.15.119:0/729174052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5613452de350 0x5613452eeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.162+0000 7f0597a56b80 1 --2- 172.21.15.119:0/729174052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613452dec70 0x5613452d4940 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.162+0000 7f05657fa700 0 client.26941 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:11.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.162+0000 7f0597a56b80 1 -- 172.21.15.119:0/729174052 >> 172.21.15.119:0/729174052 conn(0x561345204360 msgr2=0x5613452d1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:11.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.162+0000 7f0597a56b80 1 -- 172.21.15.119:0/729174052 shutdown_connections 2023-12-07T10:24:11.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.162+0000 7f0597a56b80 1 -- 172.21.15.119:0/729174052 wait complete. 2023-12-07T10:24:11.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.162+0000 7f0597a56b80 20 client.26941 trim_cache size 0 max 16384 2023-12-07T10:24:11.174 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:24:11.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.163+0000 7f05841ff700 1 -- 172.21.15.119:0/4127599804 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f05680a1d00 msgr2=0x7f05680a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:11.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.163+0000 7f05841ff700 1 --2- 172.21.15.119:0/4127599804 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f05680a1d00 0x7f05680a41c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f056c009fd0 tx=0x7f056c0134e0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.163+0000 7f05841ff700 1 -- 172.21.15.119:0/4127599804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f057c109780 msgr2=0x7f057c1999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:11.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.163+0000 7f05841ff700 1 --2- 172.21.15.119:0/4127599804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f057c109780 0x7f057c1999c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f057400efc0 tx=0x7f057400c3a0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.163+0000 7f05841ff700 1 -- 172.21.15.119:0/4127599804 shutdown_connections 2023-12-07T10:24:11.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.163+0000 7f05841ff700 1 --2- 172.21.15.119:0/4127599804 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f05680a1d00 0x7f05680a41c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.163+0000 7f05841ff700 1 --2- 172.21.15.119:0/4127599804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f057c109780 0x7f057c1999c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.163+0000 7f05841ff700 1 --2- 172.21.15.119:0/4127599804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f057c103770 0x7f057c19f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.163+0000 7f05841ff700 1 --2- 172.21.15.119:0/4127599804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f057c104130 0x7f057c19fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:11.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.163+0000 7f05841ff700 1 -- 172.21.15.119:0/4127599804 >> 172.21.15.119:0/4127599804 conn(0x7f057c0ff220 msgr2=0x7f057c0ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:11.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.163+0000 7f05841ff700 1 -- 172.21.15.119:0/4127599804 shutdown_connections 2023-12-07T10:24:11.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:11.163+0000 7f05841ff700 1 -- 172.21.15.119:0/4127599804 wait complete. 2023-12-07T10:24:11.551 INFO:tasks.workunit.client.0.smithi119.stdout: 1 217702 6.54 MB/sec execute 24 sec latency 12.951 ms 2023-12-07T10:24:12.183 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-07T10:24:12.184 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-07T10:24:12.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:11 smithi119 ceph-mon[97908]: pgmap v116: 129 pgs: 129 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 355 op/s 2023-12-07T10:24:12.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:11 smithi159 ceph-mon[134726]: pgmap v116: 129 pgs: 129 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 355 op/s 2023-12-07T10:24:12.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:11 smithi153 ceph-mon[133625]: pgmap v116: 129 pgs: 129 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 355 op/s 2023-12-07T10:24:12.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.416+0000 7fac149ff700 1 -- 172.21.15.119:0/1374524856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac10107620 msgr2=0x7fac10107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:12.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.416+0000 7fac149ff700 1 --2- 172.21.15.119:0/1374524856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac10107620 0x7fac10107a00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fac00009990 tx=0x7fac00009ca0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.416+0000 7fac149ff700 1 -- 172.21.15.119:0/1374524856 shutdown_connections 2023-12-07T10:24:12.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.416+0000 7fac149ff700 1 --2- 172.21.15.119:0/1374524856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac10101f20 0x7fac1010e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.416+0000 7fac149ff700 1 --2- 172.21.15.119:0/1374524856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac10107620 0x7fac10107a00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.416+0000 7fac149ff700 1 --2- 172.21.15.119:0/1374524856 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac10101560 0x7fac101019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.416+0000 7fac149ff700 1 -- 172.21.15.119:0/1374524856 >> 172.21.15.119:0/1374524856 conn(0x7fac100fcfd0 msgr2=0x7fac100ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:12.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.416+0000 7fac149ff700 1 -- 172.21.15.119:0/1374524856 shutdown_connections 2023-12-07T10:24:12.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.416+0000 7fac149ff700 1 -- 172.21.15.119:0/1374524856 wait complete. 2023-12-07T10:24:12.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.417+0000 7fac149ff700 1 Processor -- start 2023-12-07T10:24:12.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.418+0000 7fac149ff700 1 -- start start 2023-12-07T10:24:12.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.418+0000 7fac149ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac10101560 0x7fac1019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.418+0000 7fac149ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac10101f20 0x7fac1019dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.418+0000 7fac149ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac10107620 0x7fac101977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.418+0000 7fac149ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac10197e60 con 0x7fac10107620 2023-12-07T10:24:12.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.418+0000 7fac149ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac10197fd0 con 0x7fac10101f20 2023-12-07T10:24:12.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.418+0000 7fac149ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac10198190 con 0x7fac10101560 2023-12-07T10:24:12.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.418+0000 7fac0e59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac10101560 0x7fac1019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.418+0000 7fac0e59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac10101560 0x7fac1019d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55836/0 (socket says 172.21.15.119:55836) 2023-12-07T10:24:12.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.418+0000 7fac0e59c700 1 -- 172.21.15.119:0/368832478 learned_addr learned my addr 172.21.15.119:0/368832478 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:12.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.418+0000 7fac0dd9b700 1 --2- 172.21.15.119:0/368832478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac10101f20 0x7fac1019dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.418+0000 7fac0ed9d700 1 --2- 172.21.15.119:0/368832478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac10107620 0x7fac101977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.419+0000 7fac0e59c700 1 -- 172.21.15.119:0/368832478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac10101f20 msgr2=0x7fac1019dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:12.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.419+0000 7fac0e59c700 1 --2- 172.21.15.119:0/368832478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac10101f20 0x7fac1019dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.419+0000 7fac0e59c700 1 -- 172.21.15.119:0/368832478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac10107620 msgr2=0x7fac101977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:12.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.419+0000 7fac0e59c700 1 --2- 172.21.15.119:0/368832478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac10107620 0x7fac101977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.419+0000 7fac0e59c700 1 -- 172.21.15.119:0/368832478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac000095f0 con 0x7fac10101560 2023-12-07T10:24:12.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.419+0000 7fac0ed9d700 1 --2- 172.21.15.119:0/368832478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac10107620 0x7fac101977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:24:12.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.419+0000 7fac0e59c700 1 --2- 172.21.15.119:0/368832478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac10101560 0x7fac1019d6c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fac00000c00 tx=0x7fac00005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:12.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.419+0000 7fabff7fe700 1 -- 172.21.15.119:0/368832478 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0001d070 con 0x7fac10101560 2023-12-07T10:24:12.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.419+0000 7fac149ff700 1 -- 172.21.15.119:0/368832478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac10198410 con 0x7fac10101560 2023-12-07T10:24:12.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.419+0000 7fabff7fe700 1 -- 172.21.15.119:0/368832478 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac00022470 con 0x7fac10101560 2023-12-07T10:24:12.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.419+0000 7fabff7fe700 1 -- 172.21.15.119:0/368832478 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac00017650 con 0x7fac10101560 2023-12-07T10:24:12.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.419+0000 7fac149ff700 1 -- 172.21.15.119:0/368832478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac101a46b0 con 0x7fac10101560 2023-12-07T10:24:12.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.423+0000 7fabff7fe700 1 -- 172.21.15.119:0/368832478 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fac000225e0 con 0x7fac10101560 2023-12-07T10:24:12.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.424+0000 7fac28387b80 1 Processor -- start 2023-12-07T10:24:12.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.424+0000 7fabff7fe700 1 --2- 172.21.15.119:0/368832478 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fabf80a1d00 0x7fabf80a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.424+0000 7fac28387b80 1 -- start start 2023-12-07T10:24:12.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.425+0000 7fabff7fe700 1 -- 172.21.15.119:0/368832478 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fac000cef80 con 0x7fac10101560 2023-12-07T10:24:12.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.425+0000 7fac28387b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e10bc0 0x565017e10fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.425+0000 7fac28387b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565017e114e0 0x565017e14c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.425+0000 7fac0ed9d700 1 --2- 172.21.15.119:0/368832478 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fabf80a1d00 0x7fabf80a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.425+0000 7fac28387b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565017e151d0 0x565017e155f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.425+0000 7fac28387b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x565017d84840 con 0x565017e151d0 2023-12-07T10:24:12.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.425+0000 7fac28387b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x565017d6b820 con 0x565017e114e0 2023-12-07T10:24:12.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.425+0000 7fac28387b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x565017d69140 con 0x565017e10bc0 2023-12-07T10:24:12.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.425+0000 7fac0e59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e10bc0 0x565017e10fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.425+0000 7fac0ed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565017e151d0 0x565017e155f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.425+0000 7fac0e59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e10bc0 0x565017e10fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55850/0 (socket says 172.21.15.119:55850) 2023-12-07T10:24:12.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.425+0000 7fac0e59c700 1 -- 172.21.15.119:0/3664093770 learned_addr learned my addr 172.21.15.119:0/3664093770 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:12.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.425+0000 7fac0e59c700 1 -- 172.21.15.119:0/3664093770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565017e114e0 msgr2=0x565017e14c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:12.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.425+0000 7fac0e59c700 1 --2- 172.21.15.119:0/3664093770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565017e114e0 0x565017e14c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.425+0000 7fac0e59c700 1 -- 172.21.15.119:0/3664093770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565017e151d0 msgr2=0x565017e155f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:12.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.425+0000 7fac0e59c700 1 --2- 172.21.15.119:0/3664093770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565017e151d0 0x565017e155f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.426+0000 7fac0e59c700 1 -- 172.21.15.119:0/3664093770 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565017be16b0 con 0x565017e10bc0 2023-12-07T10:24:12.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.426+0000 7fac0e59c700 1 --2- 172.21.15.119:0/3664093770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e10bc0 0x565017e10fa0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fac00098470 tx=0x7fac00009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6320a378db179841 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:12.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.426+0000 7fac0ed9d700 1 --2- 172.21.15.119:0/368832478 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fabf80a1d00 0x7fabf80a41c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fac080094f0 tx=0x7fac08009240 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:12.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.426+0000 7fabfcff9700 1 -- 172.21.15.119:0/3664093770 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0009c100 con 0x565017e10bc0 2023-12-07T10:24:12.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.426+0000 7fabfcff9700 1 -- 172.21.15.119:0/3664093770 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac0009ba00 con 0x565017e10bc0 2023-12-07T10:24:12.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.426+0000 7fabfcff9700 1 -- 172.21.15.119:0/3664093770 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac000bb0b0 con 0x565017e10bc0 2023-12-07T10:24:12.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.427+0000 7fac28387b80 1 -- 172.21.15.119:0/3664093770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e10bc0 msgr2=0x565017e10fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:12.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.427+0000 7fac28387b80 1 --2- 172.21.15.119:0/3664093770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e10bc0 0x565017e10fa0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fac00098470 tx=0x7fac00009ca0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.427+0000 7fac28387b80 1 -- 172.21.15.119:0/3664093770 shutdown_connections 2023-12-07T10:24:12.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.427+0000 7fac28387b80 1 --2- 172.21.15.119:0/3664093770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565017e151d0 0x565017e155f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.427+0000 7fac28387b80 1 --2- 172.21.15.119:0/3664093770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e10bc0 0x565017e10fa0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.427+0000 7fac28387b80 1 --2- 172.21.15.119:0/3664093770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565017e114e0 0x565017e14c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.427+0000 7fac28387b80 1 -- 172.21.15.119:0/3664093770 >> 172.21.15.119:0/3664093770 conn(0x565017d3f3e0 msgr2=0x565017e0b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:12.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.427+0000 7fac28387b80 1 -- 172.21.15.119:0/3664093770 shutdown_connections 2023-12-07T10:24:12.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.427+0000 7fac28387b80 1 -- 172.21.15.119:0/3664093770 wait complete. 2023-12-07T10:24:12.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.428+0000 7fac28387b80 1 -- start start 2023-12-07T10:24:12.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.428+0000 7fac28387b80 1 Processor -- start 2023-12-07T10:24:12.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.428+0000 7fac28387b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565017e10bc0 0x565017e23e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.428+0000 7fac28387b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565017e114e0 0x565017e243d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.428+0000 7fac28387b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e151d0 0x565017e1e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.428+0000 7fac28387b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x565017d64a30 con 0x565017e10bc0 2023-12-07T10:24:12.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.428+0000 7fac28387b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x565017d1e5e0 con 0x565017e114e0 2023-12-07T10:24:12.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.428+0000 7fac28387b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x565017d1e280 con 0x565017e151d0 2023-12-07T10:24:12.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.428+0000 7fac28387b80 10 client.0 _ll_register_callbacks cb 0x565017d059c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:12.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.428+0000 7fac0ed9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e151d0 0x565017e1e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.429+0000 7fac0ed9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e151d0 0x565017e1e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55852/0 (socket says 172.21.15.119:55852) 2023-12-07T10:24:12.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.429+0000 7fac0ed9d700 1 -- 172.21.15.119:0/262966530 learned_addr learned my addr 172.21.15.119:0/262966530 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:12.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.429+0000 7fac0dd9b700 1 --2- 172.21.15.119:0/262966530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565017e114e0 0x565017e243d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.429+0000 7fac0e59c700 1 --2- 172.21.15.119:0/262966530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565017e10bc0 0x565017e23e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.429+0000 7fac0ed9d700 1 -- 172.21.15.119:0/262966530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565017e114e0 msgr2=0x565017e243d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:12.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.429+0000 7fac0ed9d700 1 --2- 172.21.15.119:0/262966530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565017e114e0 0x565017e243d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.429+0000 7fac0ed9d700 1 -- 172.21.15.119:0/262966530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565017e10bc0 msgr2=0x565017e23e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:12.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.429+0000 7fac0ed9d700 1 --2- 172.21.15.119:0/262966530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565017e10bc0 0x565017e23e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.429+0000 7fac0ed9d700 1 -- 172.21.15.119:0/262966530 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac000b9e70 con 0x565017e151d0 2023-12-07T10:24:12.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.429+0000 7fac0ed9d700 1 --2- 172.21.15.119:0/262966530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e151d0 0x565017e1e190 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fac08019fd0 tx=0x7fac08004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77c41f447ab2da65 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:12.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.429+0000 7fabf5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:24:12.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.430+0000 7fabf5ffb700 1 -- 172.21.15.119:0/262966530 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0801be80 con 0x565017e151d0 2023-12-07T10:24:12.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.430+0000 7fabf5ffb700 1 -- 172.21.15.119:0/262966530 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fac08021040 con 0x565017e151d0 2023-12-07T10:24:12.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.430+0000 7fac28387b80 1 -- 172.21.15.119:0/262966530 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565017e03b80 con 0x565017e151d0 2023-12-07T10:24:12.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.430+0000 7fabf5ffb700 1 -- 172.21.15.119:0/262966530 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac08009ac0 con 0x565017e151d0 2023-12-07T10:24:12.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.430+0000 7fac28387b80 10 client.26953 fetch_fsmap learned FSMap version 37 2023-12-07T10:24:12.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.430+0000 7fac28387b80 1 -- 172.21.15.119:0/262966530 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x565017b8de60 con 0x565017e151d0 2023-12-07T10:24:12.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.430+0000 7fabf5ffb700 1 -- 172.21.15.119:0/262966530 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x565017e03b80 con 0x565017e151d0 2023-12-07T10:24:12.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.430+0000 7fabf5ffb700 1 -- 172.21.15.119:0/262966530 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 37) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fac08025050 con 0x565017e151d0 2023-12-07T10:24:12.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.430+0000 7fac28387b80 10 client.26953 fetch_fsmap finished waiting for FSMap version 37 2023-12-07T10:24:12.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.430+0000 7fac28387b80 10 client.26953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:24:12.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.430+0000 7fac28387b80 20 client.26953 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:12.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.430+0000 7fac28387b80 1 --2- 172.21.15.119:0/262966530 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x565017e2e670 0x565017e30a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.431+0000 7fac28387b80 4 client.26953 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:24:12.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.431+0000 7fac28387b80 1 -- 172.21.15.119:0/262966530 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565017ddf820 con 0x565017e2e670 2023-12-07T10:24:12.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.431+0000 7fac0e59c700 1 --2- 172.21.15.119:0/262966530 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x565017e2e670 0x565017e30a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.431+0000 7fac0e59c700 1 --2- 172.21.15.119:0/262966530 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x565017e2e670 0x565017e30a90 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc54f05fe7916a1f server_cookie=16acdaa401d97851 in_seq=0 out_seq=0 2023-12-07T10:24:12.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.431+0000 7fabf5ffb700 10 client.26953 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:12.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.432+0000 7fabf5ffb700 1 -- 172.21.15.119:0/262966530 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565017ddf820 con 0x565017e2e670 2023-12-07T10:24:12.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.432+0000 7fabf5ffb700 10 client.26953 handle_command_reply: tid=0 2023-12-07T10:24:12.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.432+0000 7fac28387b80 1 client.26953 shutdown 2023-12-07T10:24:12.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.432+0000 7fac28387b80 1 -- 172.21.15.119:0/262966530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e151d0 msgr2=0x565017e1e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:12.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.432+0000 7fac28387b80 1 --2- 172.21.15.119:0/262966530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e151d0 0x565017e1e190 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fac08019fd0 tx=0x7fac08004b40 comp rx=0 tx=0).stop 2023-12-07T10:24:12.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.432+0000 7fac28387b80 1 -- 172.21.15.119:0/262966530 shutdown_connections 2023-12-07T10:24:12.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.432+0000 7fac28387b80 1 --2- 172.21.15.119:0/262966530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e151d0 0x565017e1e190 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.432+0000 7fac28387b80 1 --2- 172.21.15.119:0/262966530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565017e10bc0 0x565017e23e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.432+0000 7fac28387b80 1 --2- 172.21.15.119:0/262966530 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x565017e2e670 0x565017e30a90 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.432+0000 7fac28387b80 1 --2- 172.21.15.119:0/262966530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565017e114e0 0x565017e243d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.432+0000 7fabf5ffb700 0 client.26953 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:12.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.432+0000 7fac28387b80 1 -- 172.21.15.119:0/262966530 >> 172.21.15.119:0/262966530 conn(0x565017d3f3e0 msgr2=0x565017e05da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:12.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.432+0000 7fac28387b80 1 -- 172.21.15.119:0/262966530 shutdown_connections 2023-12-07T10:24:12.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.432+0000 7fac28387b80 1 -- 172.21.15.119:0/262966530 wait complete. 2023-12-07T10:24:12.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.433+0000 7fac28387b80 20 client.26953 trim_cache size 0 max 16384 2023-12-07T10:24:12.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.453+0000 7fac28387b80 1 Processor -- start 2023-12-07T10:24:12.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.453+0000 7fac28387b80 1 -- start start 2023-12-07T10:24:12.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.453+0000 7fac28387b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e10bc0 0x565017e10390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.453+0000 7fac28387b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565017e114e0 0x565017e1c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.453+0000 7fac28387b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565017e151d0 0x565017e23ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.453+0000 7fac28387b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x565017be39a0 con 0x565017e114e0 2023-12-07T10:24:12.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.453+0000 7fac28387b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x565017ddf820 con 0x565017e151d0 2023-12-07T10:24:12.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.453+0000 7fac28387b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x565017e2af30 con 0x565017e10bc0 2023-12-07T10:24:12.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.453+0000 7fac0e59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e10bc0 0x565017e10390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.454+0000 7fac0e59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e10bc0 0x565017e10390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55868/0 (socket says 172.21.15.119:55868) 2023-12-07T10:24:12.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.454+0000 7fac0e59c700 1 -- 172.21.15.119:0/2300184028 learned_addr learned my addr 172.21.15.119:0/2300184028 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:12.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.454+0000 7fac0ed9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565017e151d0 0x565017e23ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.454+0000 7fac0dd9b700 1 --2- 172.21.15.119:0/2300184028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565017e114e0 0x565017e1c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.454+0000 7fac0e59c700 1 -- 172.21.15.119:0/2300184028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565017e151d0 msgr2=0x565017e23ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:12.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.454+0000 7fac0e59c700 1 --2- 172.21.15.119:0/2300184028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565017e151d0 0x565017e23ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.454+0000 7fac0e59c700 1 -- 172.21.15.119:0/2300184028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565017e114e0 msgr2=0x565017e1c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:12.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.454+0000 7fac0e59c700 1 --2- 172.21.15.119:0/2300184028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565017e114e0 0x565017e1c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.454+0000 7fac0e59c700 1 -- 172.21.15.119:0/2300184028 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565017b76ce0 con 0x565017e10bc0 2023-12-07T10:24:12.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.454+0000 7fac0ed9d700 1 --2- 172.21.15.119:0/2300184028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565017e151d0 0x565017e23ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:12.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.454+0000 7fac0e59c700 1 --2- 172.21.15.119:0/2300184028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e10bc0 0x565017e10390 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fac0009b990 tx=0x7fac00005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72a5a094e50e67ea server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:12.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.454+0000 7fabfd7fa700 1 -- 172.21.15.119:0/2300184028 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0009c180 con 0x565017e10bc0 2023-12-07T10:24:12.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.454+0000 7fabfd7fa700 1 -- 172.21.15.119:0/2300184028 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac0009fb50 con 0x565017e10bc0 2023-12-07T10:24:12.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.454+0000 7fabfd7fa700 1 -- 172.21.15.119:0/2300184028 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac000bb310 con 0x565017e10bc0 2023-12-07T10:24:12.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.455+0000 7fac28387b80 1 -- 172.21.15.119:0/2300184028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e10bc0 msgr2=0x565017e10390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:12.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.455+0000 7fac28387b80 1 --2- 172.21.15.119:0/2300184028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e10bc0 0x565017e10390 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fac0009b990 tx=0x7fac00005fd0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.455+0000 7fac28387b80 1 -- 172.21.15.119:0/2300184028 shutdown_connections 2023-12-07T10:24:12.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.455+0000 7fac28387b80 1 --2- 172.21.15.119:0/2300184028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565017e151d0 0x565017e23ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.455+0000 7fac28387b80 1 --2- 172.21.15.119:0/2300184028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e10bc0 0x565017e10390 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.455+0000 7fac28387b80 1 --2- 172.21.15.119:0/2300184028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565017e114e0 0x565017e1c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.455+0000 7fac28387b80 1 -- 172.21.15.119:0/2300184028 >> 172.21.15.119:0/2300184028 conn(0x565017d3f3e0 msgr2=0x565017e155b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:12.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.455+0000 7fac28387b80 1 -- 172.21.15.119:0/2300184028 shutdown_connections 2023-12-07T10:24:12.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.455+0000 7fac28387b80 1 -- 172.21.15.119:0/2300184028 wait complete. 2023-12-07T10:24:12.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.456+0000 7fac28387b80 1 -- start start 2023-12-07T10:24:12.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.456+0000 7fac28387b80 1 Processor -- start 2023-12-07T10:24:12.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.456+0000 7fac28387b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565017e10bc0 0x565017e293e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.456+0000 7fac28387b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e114e0 0x565017e29920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.456+0000 7fac28387b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565017e151d0 0x565017e84b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.456+0000 7fac28387b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x565017e2a0b0 con 0x565017e151d0 2023-12-07T10:24:12.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.456+0000 7fac28387b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x565017e2a220 con 0x565017e10bc0 2023-12-07T10:24:12.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.456+0000 7fac28387b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x565017e85080 con 0x565017e114e0 2023-12-07T10:24:12.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.456+0000 7fac28387b80 10 client.0 _ll_register_callbacks cb 0x565017d059c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:12.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.457+0000 7fac0dd9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e114e0 0x565017e29920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.457+0000 7fac0dd9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e114e0 0x565017e29920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55870/0 (socket says 172.21.15.119:55870) 2023-12-07T10:24:12.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.457+0000 7fac0dd9b700 1 -- 172.21.15.119:0/245417807 learned_addr learned my addr 172.21.15.119:0/245417807 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:12.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.457+0000 7fac0ed9d700 1 --2- 172.21.15.119:0/245417807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565017e151d0 0x565017e84b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.457+0000 7fac0e59c700 1 --2- 172.21.15.119:0/245417807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565017e10bc0 0x565017e293e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.457+0000 7fac0dd9b700 1 -- 172.21.15.119:0/245417807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565017e10bc0 msgr2=0x565017e293e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:12.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.457+0000 7fac0dd9b700 1 --2- 172.21.15.119:0/245417807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565017e10bc0 0x565017e293e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.457+0000 7fac0dd9b700 1 -- 172.21.15.119:0/245417807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565017e151d0 msgr2=0x565017e84b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:12.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.457+0000 7fac0dd9b700 1 --2- 172.21.15.119:0/245417807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565017e151d0 0x565017e84b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.457+0000 7fac0dd9b700 1 -- 172.21.15.119:0/245417807 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565017b8de60 con 0x565017e114e0 2023-12-07T10:24:12.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.457+0000 7fac0dd9b700 1 --2- 172.21.15.119:0/245417807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e114e0 0x565017e29920 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fac04005950 tx=0x7fac0400bd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d665a3e5fc47160 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:12.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.457+0000 7fabf5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:24:12.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.458+0000 7fabf5ffb700 1 -- 172.21.15.119:0/245417807 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac04003e80 con 0x565017e114e0 2023-12-07T10:24:12.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.458+0000 7fac28387b80 1 -- 172.21.15.119:0/245417807 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac0009ba00 con 0x565017e114e0 2023-12-07T10:24:12.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.458+0000 7fabf5ffb700 1 -- 172.21.15.119:0/245417807 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fac04004830 con 0x565017e114e0 2023-12-07T10:24:12.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.458+0000 7fabf5ffb700 1 -- 172.21.15.119:0/245417807 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac040049f0 con 0x565017e114e0 2023-12-07T10:24:12.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.458+0000 7fac28387b80 10 client.26959 fetch_fsmap learned FSMap version 37 2023-12-07T10:24:12.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.458+0000 7fac28387b80 1 -- 172.21.15.119:0/245417807 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x565017be16b0 con 0x565017e114e0 2023-12-07T10:24:12.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.458+0000 7fabf5ffb700 1 -- 172.21.15.119:0/245417807 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac0009ba00 con 0x565017e114e0 2023-12-07T10:24:12.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.458+0000 7fabf5ffb700 1 -- 172.21.15.119:0/245417807 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 37) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fac0401b5b0 con 0x565017e114e0 2023-12-07T10:24:12.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.458+0000 7fac28387b80 10 client.26959 fetch_fsmap finished waiting for FSMap version 37 2023-12-07T10:24:12.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.458+0000 7fac28387b80 10 client.26959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:24:12.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.458+0000 7fac28387b80 20 client.26959 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:12.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.458+0000 7fac28387b80 1 --2- 172.21.15.119:0/245417807 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x565017e86b30 0x565017e88f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.458+0000 7fac28387b80 4 client.26959 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-07T10:24:12.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.458+0000 7fac28387b80 1 -- 172.21.15.119:0/245417807 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x565017d2b050 con 0x565017e86b30 2023-12-07T10:24:12.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.459+0000 7fac0e59c700 1 --2- 172.21.15.119:0/245417807 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x565017e86b30 0x565017e88f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.459+0000 7fabf5ffb700 10 client.26959 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:12.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.459+0000 7fac0e59c700 1 --2- 172.21.15.119:0/245417807 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x565017e86b30 0x565017e88f50 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d5496f863b685b9 server_cookie=68258928500ad0a2 in_seq=0 out_seq=0 2023-12-07T10:24:12.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.460+0000 7fabf5ffb700 1 -- 172.21.15.119:0/245417807 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x565017d2b050 con 0x565017e86b30 2023-12-07T10:24:12.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.460+0000 7fabf5ffb700 10 client.26959 handle_command_reply: tid=0 2023-12-07T10:24:12.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.460+0000 7fac28387b80 1 client.26959 shutdown 2023-12-07T10:24:12.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.460+0000 7fac28387b80 1 -- 172.21.15.119:0/245417807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e114e0 msgr2=0x565017e29920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:12.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.460+0000 7fac28387b80 1 --2- 172.21.15.119:0/245417807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e114e0 0x565017e29920 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fac04005950 tx=0x7fac0400bd40 comp rx=0 tx=0).stop 2023-12-07T10:24:12.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.460+0000 7fac28387b80 1 -- 172.21.15.119:0/245417807 shutdown_connections 2023-12-07T10:24:12.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.460+0000 7fac28387b80 1 --2- 172.21.15.119:0/245417807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565017e151d0 0x565017e84b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.460+0000 7fac28387b80 1 --2- 172.21.15.119:0/245417807 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x565017e86b30 0x565017e88f50 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.460+0000 7fac28387b80 1 --2- 172.21.15.119:0/245417807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565017e10bc0 0x565017e293e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.460+0000 7fac28387b80 1 --2- 172.21.15.119:0/245417807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565017e114e0 0x565017e29920 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.460+0000 7fabf5ffb700 0 client.26959 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:12.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.460+0000 7fac28387b80 1 -- 172.21.15.119:0/245417807 >> 172.21.15.119:0/245417807 conn(0x565017d3f3e0 msgr2=0x565017e1c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:12.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.461+0000 7fac28387b80 1 -- 172.21.15.119:0/245417807 shutdown_connections 2023-12-07T10:24:12.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.461+0000 7fac28387b80 1 -- 172.21.15.119:0/245417807 wait complete. 2023-12-07T10:24:12.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.461+0000 7fac28387b80 20 client.26959 trim_cache size 0 max 16384 2023-12-07T10:24:12.470 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:24:12.470 INFO:teuthology.orchestra.run.smithi119.stdout: "return_code": 0, 2023-12-07T10:24:12.470 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_tag": "89c6d464-946d-4c3d-ac53-08791ef988a5", 2023-12-07T10:24:12.470 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": "asynchronous" 2023-12-07T10:24:12.470 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:24:12.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.461+0000 7fac149ff700 1 -- 172.21.15.119:0/368832478 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fabf80a1d00 msgr2=0x7fabf80a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:12.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.461+0000 7fac149ff700 1 --2- 172.21.15.119:0/368832478 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fabf80a1d00 0x7fabf80a41c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fac080094f0 tx=0x7fac08009240 comp rx=0 tx=0).stop 2023-12-07T10:24:12.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.461+0000 7fac149ff700 1 -- 172.21.15.119:0/368832478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac10101560 msgr2=0x7fac1019d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:12.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.461+0000 7fac149ff700 1 --2- 172.21.15.119:0/368832478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac10101560 0x7fac1019d6c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fac00000c00 tx=0x7fac00005740 comp rx=0 tx=0).stop 2023-12-07T10:24:12.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.462+0000 7fac149ff700 1 -- 172.21.15.119:0/368832478 shutdown_connections 2023-12-07T10:24:12.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.462+0000 7fac149ff700 1 --2- 172.21.15.119:0/368832478 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fabf80a1d00 0x7fabf80a41c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.462+0000 7fac149ff700 1 --2- 172.21.15.119:0/368832478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac10107620 0x7fac101977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.462+0000 7fac149ff700 1 --2- 172.21.15.119:0/368832478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac10101560 0x7fac1019d6c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.462+0000 7fac149ff700 1 --2- 172.21.15.119:0/368832478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac10101f20 0x7fac1019dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.462+0000 7fac149ff700 1 -- 172.21.15.119:0/368832478 >> 172.21.15.119:0/368832478 conn(0x7fac100fcfd0 msgr2=0x7fac100fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:12.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.462+0000 7fac149ff700 1 -- 172.21.15.119:0/368832478 shutdown_connections 2023-12-07T10:24:12.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.462+0000 7fac149ff700 1 -- 172.21.15.119:0/368832478 wait complete. 2023-12-07T10:24:12.487 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-07T10:24:12.552 INFO:tasks.workunit.client.0.smithi119.stdout: 1 220125 6.90 MB/sec execute 25 sec latency 20.050 ms 2023-12-07T10:24:12.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.745+0000 7f4fdb7ff700 1 -- 172.21.15.119:0/3689378303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fd4068a50 msgr2=0x7f4fd4068e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:12.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.745+0000 7f4fdb7ff700 1 --2- 172.21.15.119:0/3689378303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fd4068a50 0x7f4fd4068e30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009990 tx=0x7f4fd0009ca0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.746+0000 7f4fdb7ff700 1 -- 172.21.15.119:0/3689378303 shutdown_connections 2023-12-07T10:24:12.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.746+0000 7f4fdb7ff700 1 --2- 172.21.15.119:0/3689378303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fd4106ad0 0x7f4fd410e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.746+0000 7f4fdb7ff700 1 --2- 172.21.15.119:0/3689378303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fd4068a50 0x7f4fd4068e30 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.746+0000 7f4fdb7ff700 1 --2- 172.21.15.119:0/3689378303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fd4106110 0x7f4fd4106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.746+0000 7f4fdb7ff700 1 -- 172.21.15.119:0/3689378303 >> 172.21.15.119:0/3689378303 conn(0x7f4fd4077610 msgr2=0x7f4fd4077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:12.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.746+0000 7f4fdb7ff700 1 -- 172.21.15.119:0/3689378303 shutdown_connections 2023-12-07T10:24:12.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.746+0000 7f4fdb7ff700 1 -- 172.21.15.119:0/3689378303 wait complete. 2023-12-07T10:24:12.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.747+0000 7f4fdb7ff700 1 Processor -- start 2023-12-07T10:24:12.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.747+0000 7f4fdb7ff700 1 -- start start 2023-12-07T10:24:12.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.748+0000 7f4fdb7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fd4068a50 0x7f4fd419f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.748+0000 7f4fdb7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fd4106110 0x7f4fd419fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.748+0000 7f4fdb7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fd4106ad0 0x7f4fd4199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.748+0000 7f4fdb7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd419a110 con 0x7f4fd4106ad0 2023-12-07T10:24:12.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.748+0000 7f4fdb7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd419a280 con 0x7f4fd4068a50 2023-12-07T10:24:12.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.748+0000 7f4fdb7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd419a440 con 0x7f4fd4106110 2023-12-07T10:24:12.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.748+0000 7f4fd959b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fd4068a50 0x7f4fd419f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.748+0000 7f4fd959b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fd4068a50 0x7f4fd419f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:56226/0 (socket says 172.21.15.119:56226) 2023-12-07T10:24:12.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.748+0000 7f4fd8d9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fd4106110 0x7f4fd419fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.748+0000 7f4fd959b700 1 -- 172.21.15.119:0/3777864861 learned_addr learned my addr 172.21.15.119:0/3777864861 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:12.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.748+0000 7f4fd9d9c700 1 --2- 172.21.15.119:0/3777864861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fd4106ad0 0x7f4fd4199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.748+0000 7f4fd959b700 1 -- 172.21.15.119:0/3777864861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fd4106110 msgr2=0x7f4fd419fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:12.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.749+0000 7f4fd959b700 1 --2- 172.21.15.119:0/3777864861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fd4106110 0x7f4fd419fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.749+0000 7f4fd959b700 1 -- 172.21.15.119:0/3777864861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fd4106ad0 msgr2=0x7f4fd4199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:12.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.749+0000 7f4fd959b700 1 --2- 172.21.15.119:0/3777864861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fd4106ad0 0x7f4fd4199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.749+0000 7f4fd959b700 1 -- 172.21.15.119:0/3777864861 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd00095f0 con 0x7f4fd4068a50 2023-12-07T10:24:12.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.749+0000 7f4fd8d9a700 1 --2- 172.21.15.119:0/3777864861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fd4106110 0x7f4fd419fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:12.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.749+0000 7f4fd959b700 1 --2- 172.21.15.119:0/3777864861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fd4068a50 0x7f4fd419f980 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0004730 tx=0x7f4fd0004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:12.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.749+0000 7f4fca7fc700 1 -- 172.21.15.119:0/3777864861 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd001d070 con 0x7f4fd4068a50 2023-12-07T10:24:12.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.749+0000 7f4fca7fc700 1 -- 172.21.15.119:0/3777864861 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fd0022470 con 0x7f4fd4068a50 2023-12-07T10:24:12.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.749+0000 7f4fdb7ff700 1 -- 172.21.15.119:0/3777864861 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd419a6c0 con 0x7f4fd4068a50 2023-12-07T10:24:12.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.749+0000 7f4fca7fc700 1 -- 172.21.15.119:0/3777864861 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0017650 con 0x7f4fd4068a50 2023-12-07T10:24:12.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.749+0000 7f4fdb7ff700 1 -- 172.21.15.119:0/3777864861 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd41a6970 con 0x7f4fd4068a50 2023-12-07T10:24:12.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.753+0000 7f4fca7fc700 1 -- 172.21.15.119:0/3777864861 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f4fd00225e0 con 0x7f4fd4068a50 2023-12-07T10:24:12.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.754+0000 7f4fef086b80 1 Processor -- start 2023-12-07T10:24:12.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.754+0000 7f4fca7fc700 1 --2- 172.21.15.119:0/3777864861 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4fc00a1d00 0x7f4fc00a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.754+0000 7f4fef086b80 1 -- start start 2023-12-07T10:24:12.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.754+0000 7f4fca7fc700 1 -- 172.21.15.119:0/3777864861 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4fd00cee10 con 0x7f4fd4068a50 2023-12-07T10:24:12.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.754+0000 7f4fd8d9a700 1 --2- 172.21.15.119:0/3777864861 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4fc00a1d00 0x7f4fc00a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.755+0000 7f4fef086b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6ee321b40 0x55d6ee321f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.755+0000 7f4fef086b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d6ee322460 0x55d6ee325c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.755+0000 7f4fef086b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee326150 0x55d6ee326570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.755+0000 7f4fef086b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ee2957c0 con 0x55d6ee321b40 2023-12-07T10:24:12.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.755+0000 7f4fef086b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ee27c7a0 con 0x55d6ee326150 2023-12-07T10:24:12.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.755+0000 7f4fef086b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ee27a0c0 con 0x55d6ee322460 2023-12-07T10:24:12.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.755+0000 7f4fd9d9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee326150 0x55d6ee326570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.755+0000 7f4fd9d9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee326150 0x55d6ee326570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:56230/0 (socket says 172.21.15.119:56230) 2023-12-07T10:24:12.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.755+0000 7f4fd9d9c700 1 -- 172.21.15.119:0/2336434528 learned_addr learned my addr 172.21.15.119:0/2336434528 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:12.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.755+0000 7f4fd959b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6ee321b40 0x55d6ee321f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.755+0000 7f4fd8d9a700 1 --2- 172.21.15.119:0/2336434528 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d6ee322460 0x55d6ee325c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.755+0000 7f4fd9d9c700 1 -- 172.21.15.119:0/2336434528 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d6ee322460 msgr2=0x55d6ee325c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:12.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.755+0000 7f4fd9d9c700 1 --2- 172.21.15.119:0/2336434528 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d6ee322460 0x55d6ee325c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.755+0000 7f4fd9d9c700 1 -- 172.21.15.119:0/2336434528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6ee321b40 msgr2=0x55d6ee321f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:12.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.755+0000 7f4fd9d9c700 1 --2- 172.21.15.119:0/2336434528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6ee321b40 0x55d6ee321f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.755+0000 7f4fd9d9c700 1 -- 172.21.15.119:0/2336434528 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6ee0f2630 con 0x55d6ee326150 2023-12-07T10:24:12.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.755+0000 7f4fd8d9a700 1 --2- 172.21.15.119:0/2336434528 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d6ee322460 0x55d6ee325c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:24:12.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.756+0000 7f4fd8d9a700 1 --2- 172.21.15.119:0/3777864861 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4fc00a1d00 0x7f4fc00a41c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4fd419ac70 tx=0x7f4fc4009f90 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:12.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.756+0000 7f4fd9d9c700 1 --2- 172.21.15.119:0/2336434528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee326150 0x55d6ee326570 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc011670 tx=0x7f4fcc009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5229528ce5683f7 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:12.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.756+0000 7f4fbf7fe700 1 -- 172.21.15.119:0/2336434528 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc00e040 con 0x55d6ee326150 2023-12-07T10:24:12.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.756+0000 7f4fbf7fe700 1 -- 172.21.15.119:0/2336434528 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fcc00f040 con 0x55d6ee326150 2023-12-07T10:24:12.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.756+0000 7f4fbf7fe700 1 -- 172.21.15.119:0/2336434528 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc004870 con 0x55d6ee326150 2023-12-07T10:24:12.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.757+0000 7f4fef086b80 1 -- 172.21.15.119:0/2336434528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee326150 msgr2=0x55d6ee326570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:12.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.757+0000 7f4fef086b80 1 --2- 172.21.15.119:0/2336434528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee326150 0x55d6ee326570 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc011670 tx=0x7f4fcc009920 comp rx=0 tx=0).stop 2023-12-07T10:24:12.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.757+0000 7f4fef086b80 1 -- 172.21.15.119:0/2336434528 shutdown_connections 2023-12-07T10:24:12.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.757+0000 7f4fef086b80 1 --2- 172.21.15.119:0/2336434528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee326150 0x55d6ee326570 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.757+0000 7f4fef086b80 1 --2- 172.21.15.119:0/2336434528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6ee321b40 0x55d6ee321f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.757+0000 7f4fef086b80 1 --2- 172.21.15.119:0/2336434528 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d6ee322460 0x55d6ee325c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.757+0000 7f4fef086b80 1 -- 172.21.15.119:0/2336434528 >> 172.21.15.119:0/2336434528 conn(0x55d6ee250360 msgr2=0x55d6ee31c130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:12.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.757+0000 7f4fef086b80 1 -- 172.21.15.119:0/2336434528 shutdown_connections 2023-12-07T10:24:12.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.757+0000 7f4fef086b80 1 -- 172.21.15.119:0/2336434528 wait complete. 2023-12-07T10:24:12.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.757+0000 7f4fef086b80 1 -- start start 2023-12-07T10:24:12.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.758+0000 7f4fef086b80 1 Processor -- start 2023-12-07T10:24:12.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.758+0000 7f4fef086b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee321b40 0x55d6ee32d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.758+0000 7f4fef086b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d6ee322460 0x55d6ee32aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.758+0000 7f4fef086b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6ee326150 0x55d6ee32b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.758+0000 7f4fef086b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ee2759b0 con 0x55d6ee326150 2023-12-07T10:24:12.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.758+0000 7f4fef086b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ee22f560 con 0x55d6ee321b40 2023-12-07T10:24:12.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.758+0000 7f4fef086b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ee22f200 con 0x55d6ee322460 2023-12-07T10:24:12.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.758+0000 7f4fef086b80 10 client.0 _ll_register_callbacks cb 0x55d6ee216940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:12.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.758+0000 7f4fd959b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee321b40 0x55d6ee32d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.759+0000 7f4fd959b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee321b40 0x55d6ee32d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:56234/0 (socket says 172.21.15.119:56234) 2023-12-07T10:24:12.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.759+0000 7f4fd959b700 1 -- 172.21.15.119:0/59706770 learned_addr learned my addr 172.21.15.119:0/59706770 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:12.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.759+0000 7f4fd8d9a700 1 --2- 172.21.15.119:0/59706770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d6ee322460 0x55d6ee32aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.759+0000 7f4fd9d9c700 1 --2- 172.21.15.119:0/59706770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6ee326150 0x55d6ee32b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.759+0000 7f4fd959b700 1 -- 172.21.15.119:0/59706770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d6ee322460 msgr2=0x55d6ee32aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:12.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.759+0000 7f4fd959b700 1 --2- 172.21.15.119:0/59706770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d6ee322460 0x55d6ee32aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.759+0000 7f4fd959b700 1 -- 172.21.15.119:0/59706770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6ee326150 msgr2=0x55d6ee32b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:12.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.759+0000 7f4fd959b700 1 --2- 172.21.15.119:0/59706770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6ee326150 0x55d6ee32b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.759+0000 7f4fd959b700 1 -- 172.21.15.119:0/59706770 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fcc011350 con 0x55d6ee321b40 2023-12-07T10:24:12.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.759+0000 7f4fd959b700 1 --2- 172.21.15.119:0/59706770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee321b40 0x55d6ee32d830 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4fd009b880 tx=0x7f4fd009b4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c00ff3efae7532d server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:12.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.759+0000 7f4fbcff9700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:24:12.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.759+0000 7f4fbcff9700 1 -- 172.21.15.119:0/59706770 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0022ac0 con 0x55d6ee321b40 2023-12-07T10:24:12.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.759+0000 7f4fbcff9700 1 -- 172.21.15.119:0/59706770 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4fd009b8f0 con 0x55d6ee321b40 2023-12-07T10:24:12.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.759+0000 7f4fef086b80 1 -- 172.21.15.119:0/59706770 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d6ee314b00 con 0x55d6ee321b40 2023-12-07T10:24:12.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.760+0000 7f4fbcff9700 1 -- 172.21.15.119:0/59706770 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fd009fa40 con 0x55d6ee321b40 2023-12-07T10:24:12.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.760+0000 7f4fef086b80 10 client.26990 fetch_fsmap learned FSMap version 37 2023-12-07T10:24:12.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.760+0000 7f4fef086b80 1 -- 172.21.15.119:0/59706770 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55d6ee09ede0 con 0x55d6ee321b40 2023-12-07T10:24:12.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.760+0000 7f4fbcff9700 1 -- 172.21.15.119:0/59706770 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d6ee314b00 con 0x55d6ee321b40 2023-12-07T10:24:12.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.760+0000 7f4fbcff9700 1 -- 172.21.15.119:0/59706770 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 37) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f4fd00a3760 con 0x55d6ee321b40 2023-12-07T10:24:12.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.760+0000 7f4fef086b80 10 client.26990 fetch_fsmap finished waiting for FSMap version 37 2023-12-07T10:24:12.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.760+0000 7f4fef086b80 10 client.26990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:24:12.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.760+0000 7f4fef086b80 20 client.26990 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:12.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.760+0000 7f4fef086b80 1 --2- 172.21.15.119:0/59706770 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d6ee33b000 0x55d6ee33d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.760+0000 7f4fef086b80 4 client.26990 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:24:12.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.760+0000 7f4fef086b80 1 -- 172.21.15.119:0/59706770 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d6ee2f07a0 con 0x55d6ee33b000 2023-12-07T10:24:12.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.761+0000 7f4fd8d9a700 1 --2- 172.21.15.119:0/59706770 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d6ee33b000 0x55d6ee33d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.761+0000 7f4fd8d9a700 1 --2- 172.21.15.119:0/59706770 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d6ee33b000 0x55d6ee33d420 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ba23e3c82bf50fc server_cookie=cc0cec2d989aef72 in_seq=0 out_seq=0 2023-12-07T10:24:12.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.761+0000 7f4fbcff9700 10 client.26990 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:12.767 INFO:journalctl@ceph.mds.f.smithi159.stdout:Dec 07 10:24:12 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-f[171910]: 2023-12-07T10:24:12.466+0000 7f876ea46700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000215 [4,head] /volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT auth{0=1} v936 ap=1 s=260096 n(v0 rc2023-12-07T10:21:47.681309+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25454=0-4194304@3} caps={25454=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@153},l=25454 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55be4e727080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000215:[<0x2000000020b/ROSE.CPT v936>,<0x20000000004/COREL v13356>,<0x20000000003/~dmtmp v158>,<0x20000000002/client0 v104>,<0x20000000001/clients v74>,<0x20000000000/tmp v74>,<0x100000001fb/client.0 v72>,<0x100000001fa/a0552b93-a7a4-4db9-8129-ee198f2aab69 v91>,<0x10000000001/sv_1 v53>,<0x10000000000/_nogroup v38>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x20000000215:[<0x2000000020b/ROSE.CPT v936>,<0x20000000004/COREL v29589>,<0x20000000003/~dmtmp v11716>,<0x20000000002/client0 v314>,<0x20000000001/clients v274>,<0x20000000000/tmp v274>,<0x100000001fb/client.0 v272>,<0x100000001fa/a0552b93-a7a4-4db9-8129-ee198f2aab69 v284>,<0x10000000001/sv_1 v77>,<0x10000000000/_nogroup v28>,<0x1/volumes v60>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-07T10:24:12.768 INFO:journalctl@ceph.mds.f.smithi159.stdout:Dec 07 10:24:12 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-f[171910]: 2023-12-07T10:24:12.466+0000 7f876ea46700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000211 [4,head] /volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT auth{0=1} v962 ap=1 s=260096 n(v0 rc2023-12-07T10:21:47.662310+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25454=0-4194304@3} caps={25454=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@153},l=25454 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55be4e22cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000211:[<0x2000000020b/TEAPOT2.CPT v962>,<0x20000000004/COREL v13356>,<0x20000000003/~dmtmp v158>,<0x20000000002/client0 v104>,<0x20000000001/clients v74>,<0x20000000000/tmp v74>,<0x100000001fb/client.0 v72>,<0x100000001fa/a0552b93-a7a4-4db9-8129-ee198f2aab69 v91>,<0x10000000001/sv_1 v53>,<0x10000000000/_nogroup v38>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x20000000211:[<0x2000000020b/TEAPOT2.CPT v962>,<0x20000000004/COREL v29589>,<0x20000000003/~dmtmp v11716>,<0x20000000002/client0 v314>,<0x20000000001/clients v274>,<0x20000000000/tmp v274>,<0x100000001fb/client.0 v272>,<0x100000001fa/a0552b93-a7a4-4db9-8129-ee198f2aab69 v284>,<0x10000000001/sv_1 v77>,<0x10000000000/_nogroup v28>,<0x1/volumes v60>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-07T10:24:12.768 INFO:journalctl@ceph.mds.f.smithi159.stdout:Dec 07 10:24:12 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-f[171910]: 2023-12-07T10:24:12.466+0000 7f876ea46700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000214 [4,head] /volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR auth{0=1} v1241 ap=1 s=260096 n(v0 rc2023-12-07T10:21:47.676310+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25454=0-4194304@3} caps={25454=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@153},l=25454 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55be4e6aa680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000214:[<0x2000000020b/GRAPHIC2.CDR v1241>,<0x20000000004/COREL v13356>,<0x20000000003/~dmtmp v158>,<0x20000000002/client0 v104>,<0x20000000001/clients v74>,<0x20000000000/tmp v74>,<0x100000001fb/client.0 v72>,<0x100000001fa/a0552b93-a7a4-4db9-8129-ee198f2aab69 v91>,<0x10000000001/sv_1 v53>,<0x10000000000/_nogroup v38>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x20000000214:[<0x2000000020b/GRAPHIC2.CDR v1241>,<0x20000000004/COREL v29589>,<0x20000000003/~dmtmp v11716>,<0x20000000002/client0 v314>,<0x20000000001/clients v274>,<0x20000000000/tmp v274>,<0x100000001fb/client.0 v272>,<0x100000001fa/a0552b93-a7a4-4db9-8129-ee198f2aab69 v284>,<0x10000000001/sv_1 v77>,<0x10000000000/_nogroup v28>,<0x1/volumes v60>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-07T10:24:12.768 INFO:journalctl@ceph.mds.f.smithi159.stdout:Dec 07 10:24:12 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-f[171910]: 2023-12-07T10:24:12.467+0000 7f876ea46700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000020e [4,head] /volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL auth{0=1} v1204 ap=1 s=260096 n(v0 rc2023-12-07T10:21:47.647310+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25454=0-4194304@3} caps={25454=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@153},l=25454 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55be4e787700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x2000000020e:[<0x2000000020b/ARTISTIC.ACL v1204>,<0x20000000004/COREL v13356>,<0x20000000003/~dmtmp v158>,<0x20000000002/client0 v104>,<0x20000000001/clients v74>,<0x20000000000/tmp v74>,<0x100000001fb/client.0 v72>,<0x100000001fa/a0552b93-a7a4-4db9-8129-ee198f2aab69 v91>,<0x10000000001/sv_1 v53>,<0x10000000000/_nogroup v38>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x2000000020e:[<0x2000000020b/ARTISTIC.ACL v1204>,<0x20000000004/COREL v29589>,<0x20000000003/~dmtmp v11716>,<0x20000000002/client0 v314>,<0x20000000001/clients v274>,<0x20000000000/tmp v274>,<0x100000001fb/client.0 v272>,<0x100000001fa/a0552b93-a7a4-4db9-8129-ee198f2aab69 v284>,<0x10000000001/sv_1 v77>,<0x10000000000/_nogroup v28>,<0x1/volumes v60>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-07T10:24:12.768 INFO:journalctl@ceph.mds.f.smithi159.stdout:Dec 07 10:24:12 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-f[171910]: 2023-12-07T10:24:12.467+0000 7f876ea46700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000212 [4,head] /volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT auth{0=1} v958 ap=1 s=260096 n(v0 rc2023-12-07T10:21:47.666310+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25454=0-4194304@3} caps={25454=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@153},l=25454 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55be4e675b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000212:[<0x2000000020b/SIGN2.CPT v958>,<0x20000000004/COREL v13356>,<0x20000000003/~dmtmp v158>,<0x20000000002/client0 v104>,<0x20000000001/clients v74>,<0x20000000000/tmp v74>,<0x100000001fb/client.0 v72>,<0x100000001fa/a0552b93-a7a4-4db9-8129-ee198f2aab69 v91>,<0x10000000001/sv_1 v53>,<0x10000000000/_nogroup v38>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x20000000212:[<0x2000000020b/SIGN2.CPT v958>,<0x20000000004/COREL v29589>,<0x20000000003/~dmtmp v11716>,<0x20000000002/client0 v314>,<0x20000000001/clients v274>,<0x20000000000/tmp v274>,<0x100000001fb/client.0 v272>,<0x100000001fa/a0552b93-a7a4-4db9-8129-ee198f2aab69 v284>,<0x10000000001/sv_1 v77>,<0x10000000000/_nogroup v28>,<0x1/volumes v60>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-07T10:24:12.768 INFO:journalctl@ceph.mds.f.smithi159.stdout:Dec 07 10:24:12 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-f[171910]: 2023-12-07T10:24:12.468+0000 7f876ea46700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000210 [4,head] /volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT auth{0=1} v932 ap=1 s=260096 n(v0 rc2023-12-07T10:21:47.657310+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25454=0-4194304@3} caps={25454=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@153},l=25454 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55be4e4df600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000210:[<0x2000000020b/MAILBOX2.CPT v932>,<0x20000000004/COREL v13356>,<0x20000000003/~dmtmp v158>,<0x20000000002/client0 v104>,<0x20000000001/clients v74>,<0x20000000000/tmp v74>,<0x100000001fb/client.0 v72>,<0x100000001fa/a0552b93-a7a4-4db9-8129-ee198f2aab69 v91>,<0x10000000001/sv_1 v53>,<0x10000000000/_nogroup v38>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x20000000210:[<0x2000000020b/MAILBOX2.CPT v932>,<0x20000000004/COREL v29589>,<0x20000000003/~dmtmp v11716>,<0x20000000002/client0 v314>,<0x20000000001/clients v274>,<0x20000000000/tmp v274>,<0x100000001fb/client.0 v272>,<0x100000001fa/a0552b93-a7a4-4db9-8129-ee198f2aab69 v284>,<0x10000000001/sv_1 v77>,<0x10000000000/_nogroup v28>,<0x1/volumes v60>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-07T10:24:12.768 INFO:journalctl@ceph.mds.f.smithi159.stdout:Dec 07 10:24:12 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-f[171910]: 2023-12-07T10:24:12.468+0000 7f876ea46700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000216 [4,head] /volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT auth{0=1} v1211 ap=1 s=260096 n(v0 rc2023-12-07T10:21:47.686309+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25454=0-4194304@3} caps={25454=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@153},l=25454 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55be4e764100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000216:[<0x2000000020b/CORELDRW.CDT v1211>,<0x20000000004/COREL v13356>,<0x20000000003/~dmtmp v158>,<0x20000000002/client0 v104>,<0x20000000001/clients v74>,<0x20000000000/tmp v74>,<0x100000001fb/client.0 v72>,<0x100000001fa/a0552b93-a7a4-4db9-8129-ee198f2aab69 v91>,<0x10000000001/sv_1 v53>,<0x10000000000/_nogroup v38>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x20000000216:[<0x2000000020b/CORELDRW.CDT v1211>,<0x20000000004/COREL v29589>,<0x20000000003/~dmtmp v11716>,<0x20000000002/client0 v314>,<0x20000000001/clients v274>,<0x20000000000/tmp v274>,<0x100000001fb/client.0 v272>,<0x100000001fa/a0552b93-a7a4-4db9-8129-ee198f2aab69 v284>,<0x10000000001/sv_1 v77>,<0x10000000000/_nogroup v28>,<0x1/volumes v60>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-07T10:24:12.768 INFO:journalctl@ceph.mds.f.smithi159.stdout:Dec 07 10:24:12 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-f[171910]: 2023-12-07T10:24:12.468+0000 7f876ea46700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x20000000213 [4,head] /volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR auth{0=1} v954 ap=1 s=260096 n(v0 rc2023-12-07T10:21:47.671310+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25454=0-4194304@3} caps={25454=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@153},l=25454 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55be4e760100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x20000000213:[<0x2000000020b/TRAIN.CDR v954>,<0x20000000004/COREL v13356>,<0x20000000003/~dmtmp v158>,<0x20000000002/client0 v104>,<0x20000000001/clients v74>,<0x20000000000/tmp v74>,<0x100000001fb/client.0 v72>,<0x100000001fa/a0552b93-a7a4-4db9-8129-ee198f2aab69 v91>,<0x10000000001/sv_1 v53>,<0x10000000000/_nogroup v38>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x20000000213:[<0x2000000020b/TRAIN.CDR v954>,<0x20000000004/COREL v29589>,<0x20000000003/~dmtmp v11716>,<0x20000000002/client0 v314>,<0x20000000001/clients v274>,<0x20000000000/tmp v274>,<0x100000001fb/client.0 v272>,<0x100000001fa/a0552b93-a7a4-4db9-8129-ee198f2aab69 v284>,<0x10000000001/sv_1 v77>,<0x10000000000/_nogroup v28>,<0x1/volumes v60>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-07T10:24:12.768 INFO:journalctl@ceph.mds.f.smithi159.stdout:Dec 07 10:24:12 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-f[171910]: 2023-12-07T10:24:12.468+0000 7f876ea46700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000020f [4,head] /volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR auth{0=1} v1213 ap=1 s=260096 n(v0 rc2023-12-07T10:21:47.653310+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25454=0-4194304@3} caps={25454=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@153},l=25454 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x55be4e760c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x2000000020f:[<0x2000000020b/GRAPHIC1.CDR v1213>,<0x20000000004/COREL v13356>,<0x20000000003/~dmtmp v158>,<0x20000000002/client0 v104>,<0x20000000001/clients v74>,<0x20000000000/tmp v74>,<0x100000001fb/client.0 v72>,<0x100000001fa/a0552b93-a7a4-4db9-8129-ee198f2aab69 v91>,<0x10000000001/sv_1 v53>,<0x10000000000/_nogroup v38>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x2000000020f:[<0x2000000020b/GRAPHIC1.CDR v1213>,<0x20000000004/COREL v29589>,<0x20000000003/~dmtmp v11716>,<0x20000000002/client0 v314>,<0x20000000001/clients v274>,<0x20000000000/tmp v274>,<0x100000001fb/client.0 v272>,<0x100000001fa/a0552b93-a7a4-4db9-8129-ee198f2aab69 v284>,<0x10000000001/sv_1 v77>,<0x10000000000/_nogroup v28>,<0x1/volumes v60>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-07T10:24:12.768 INFO:journalctl@ceph.mds.f.smithi159.stdout:Dec 07 10:24:12 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-f[171910]: 2023-12-07T10:24:12.469+0000 7f876ea46700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000020d [5,head] /volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG auth{0=1} v4299 ap=1 s=260096 n(v0 rc2023-12-07T10:23:59.295088+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25454=0-4194304@5} caps={25454=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@157},l=25454 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55be4e75eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x2000000020d:[<0x2000000020b/CDRBARS.CFG v1274>,<0x20000000004/COREL v13356>,<0x20000000003/~dmtmp v158>,<0x20000000002/client0 v104>,<0x20000000001/clients v74>,<0x20000000000/tmp v74>,<0x100000001fb/client.0 v72>,<0x100000001fa/a0552b93-a7a4-4db9-8129-ee198f2aab69 v91>,<0x10000000001/sv_1 v53>,<0x10000000000/_nogroup v38>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x2000000020d:[<0x2000000020b/CDRBARS.CFG v4299>,<0x20000000004/COREL v29589>,<0x20000000003/~dmtmp v11716>,<0x20000000002/client0 v314>,<0x20000000001/clients v274>,<0x20000000000/tmp v274>,<0x100000001fb/client.0 v272>,<0x100000001fa/a0552b93-a7a4-4db9-8129-ee198f2aab69 v284>,<0x10000000001/sv_1 v77>,<0x10000000000/_nogroup v28>,<0x1/volumes v62>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-07T10:24:12.768 INFO:journalctl@ceph.mds.f.smithi159.stdout:Dec 07 10:24:12 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-f[171910]: 2023-12-07T10:24:12.470+0000 7f876ea46700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000020c [5,head] /volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG auth{0=1} v4306 ap=1 s=260096 n(v0 rc2023-12-07T10:23:59.305088+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25454=0-4194304@5} caps={25454=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@157},l=25454 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55be4e998000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x2000000020c:[<0x2000000020b/CDRROLS.CFG v1276>,<0x20000000004/COREL v13356>,<0x20000000003/~dmtmp v158>,<0x20000000002/client0 v104>,<0x20000000001/clients v74>,<0x20000000000/tmp v74>,<0x100000001fb/client.0 v72>,<0x100000001fa/a0552b93-a7a4-4db9-8129-ee198f2aab69 v91>,<0x10000000001/sv_1 v53>,<0x10000000000/_nogroup v38>,<0x1/volumes v38>]//[3]","memoryvalue":"(4)0x2000000020c:[<0x2000000020b/CDRROLS.CFG v4306>,<0x20000000004/COREL v29589>,<0x20000000003/~dmtmp v11716>,<0x20000000002/client0 v314>,<0x20000000001/clients v274>,<0x20000000000/tmp v274>,<0x100000001fb/client.0 v272>,<0x100000001fa/a0552b93-a7a4-4db9-8129-ee198f2aab69 v284>,<0x10000000001/sv_1 v77>,<0x10000000000/_nogroup v28>,<0x1/volumes v62>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-07T10:24:12.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.762+0000 7f4fbcff9700 1 -- 172.21.15.119:0/59706770 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d6ee2f07a0 con 0x55d6ee33b000 2023-12-07T10:24:12.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.762+0000 7f4fbcff9700 10 client.26990 handle_command_reply: tid=0 2023-12-07T10:24:12.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.762+0000 7f4fef086b80 1 client.26990 shutdown 2023-12-07T10:24:12.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.762+0000 7f4fef086b80 1 -- 172.21.15.119:0/59706770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee321b40 msgr2=0x55d6ee32d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:12.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.762+0000 7f4fef086b80 1 --2- 172.21.15.119:0/59706770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee321b40 0x55d6ee32d830 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4fd009b880 tx=0x7f4fd009b4a0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.762+0000 7f4fef086b80 1 -- 172.21.15.119:0/59706770 shutdown_connections 2023-12-07T10:24:12.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.762+0000 7f4fef086b80 1 --2- 172.21.15.119:0/59706770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6ee326150 0x55d6ee32b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.762+0000 7f4fef086b80 1 --2- 172.21.15.119:0/59706770 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d6ee33b000 0x55d6ee33d420 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.762+0000 7f4fef086b80 1 --2- 172.21.15.119:0/59706770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee321b40 0x55d6ee32d830 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.762+0000 7f4fef086b80 1 --2- 172.21.15.119:0/59706770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d6ee322460 0x55d6ee32aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.762+0000 7f4fbcff9700 0 client.26990 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:12.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.762+0000 7f4fef086b80 1 -- 172.21.15.119:0/59706770 >> 172.21.15.119:0/59706770 conn(0x55d6ee250360 msgr2=0x55d6ee327620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:12.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.763+0000 7f4fef086b80 1 -- 172.21.15.119:0/59706770 shutdown_connections 2023-12-07T10:24:12.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.763+0000 7f4fef086b80 1 -- 172.21.15.119:0/59706770 wait complete. 2023-12-07T10:24:12.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.763+0000 7f4fef086b80 20 client.26990 trim_cache size 0 max 16384 2023-12-07T10:24:12.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.784+0000 7f4fef086b80 1 Processor -- start 2023-12-07T10:24:12.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.785+0000 7f4fef086b80 1 -- start start 2023-12-07T10:24:12.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.785+0000 7f4fef086b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6ee321b40 0x55d6ee341d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.785+0000 7f4fef086b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee322460 0x55d6ee332380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.785+0000 7f4fef086b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d6ee326150 0x55d6ee3328c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.785+0000 7f4fef086b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ee0f4920 con 0x55d6ee321b40 2023-12-07T10:24:12.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.785+0000 7f4fef086b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ee324b10 con 0x55d6ee322460 2023-12-07T10:24:12.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.785+0000 7f4fef086b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ee314940 con 0x55d6ee326150 2023-12-07T10:24:12.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.785+0000 7f4fd959b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6ee321b40 0x55d6ee341d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.785+0000 7f4fd8d9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee322460 0x55d6ee332380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.785+0000 7f4fd8d9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee322460 0x55d6ee332380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:56240/0 (socket says 172.21.15.119:56240) 2023-12-07T10:24:12.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.785+0000 7f4fd8d9a700 1 -- 172.21.15.119:0/1346518479 learned_addr learned my addr 172.21.15.119:0/1346518479 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:12.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.785+0000 7f4fd959b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6ee321b40 0x55d6ee341d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36676/0 (socket says 172.21.15.119:36676) 2023-12-07T10:24:12.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.785+0000 7f4fd9d9c700 1 --2- 172.21.15.119:0/1346518479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d6ee326150 0x55d6ee3328c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.786+0000 7f4fd959b700 1 -- 172.21.15.119:0/1346518479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d6ee326150 msgr2=0x55d6ee3328c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:12.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.786+0000 7f4fd959b700 1 --2- 172.21.15.119:0/1346518479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d6ee326150 0x55d6ee3328c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.786+0000 7f4fd959b700 1 -- 172.21.15.119:0/1346518479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee322460 msgr2=0x55d6ee332380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:12.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.786+0000 7f4fd959b700 1 --2- 172.21.15.119:0/1346518479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee322460 0x55d6ee332380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.786+0000 7f4fd959b700 1 -- 172.21.15.119:0/1346518479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6ee316040 con 0x55d6ee321b40 2023-12-07T10:24:12.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.786+0000 7f4fd8d9a700 1 --2- 172.21.15.119:0/1346518479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee322460 0x55d6ee332380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:12.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.786+0000 7f4fd959b700 1 --2- 172.21.15.119:0/1346518479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6ee321b40 0x55d6ee341d80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4fd009b5f0 tx=0x7f4fd00b4250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a4158ff276febf2 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:12.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.786+0000 7f4fbffff700 1 -- 172.21.15.119:0/1346518479 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0097c30 con 0x55d6ee321b40 2023-12-07T10:24:12.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.786+0000 7f4fbffff700 1 -- 172.21.15.119:0/1346518479 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fd0097a40 con 0x55d6ee321b40 2023-12-07T10:24:12.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.786+0000 7f4fbffff700 1 -- 172.21.15.119:0/1346518479 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd00a3bf0 con 0x55d6ee321b40 2023-12-07T10:24:12.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.787+0000 7f4fef086b80 1 -- 172.21.15.119:0/1346518479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6ee321b40 msgr2=0x55d6ee341d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:12.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.787+0000 7f4fef086b80 1 --2- 172.21.15.119:0/1346518479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6ee321b40 0x55d6ee341d80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4fd009b5f0 tx=0x7f4fd00b4250 comp rx=0 tx=0).stop 2023-12-07T10:24:12.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.787+0000 7f4fef086b80 1 -- 172.21.15.119:0/1346518479 shutdown_connections 2023-12-07T10:24:12.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.787+0000 7f4fef086b80 1 --2- 172.21.15.119:0/1346518479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d6ee326150 0x55d6ee3328c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.787+0000 7f4fef086b80 1 --2- 172.21.15.119:0/1346518479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6ee321b40 0x55d6ee341d80 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.787+0000 7f4fef086b80 1 --2- 172.21.15.119:0/1346518479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee322460 0x55d6ee332380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.787+0000 7f4fef086b80 1 -- 172.21.15.119:0/1346518479 >> 172.21.15.119:0/1346518479 conn(0x55d6ee250360 msgr2=0x55d6ee316c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:12.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.787+0000 7f4fef086b80 1 -- 172.21.15.119:0/1346518479 shutdown_connections 2023-12-07T10:24:12.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.787+0000 7f4fef086b80 1 -- 172.21.15.119:0/1346518479 wait complete. 2023-12-07T10:24:12.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.787+0000 7f4fef086b80 1 -- start start 2023-12-07T10:24:12.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.788+0000 7f4fef086b80 1 Processor -- start 2023-12-07T10:24:12.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.788+0000 7f4fef086b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d6ee321b40 0x55d6ee330360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.788+0000 7f4fef086b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee322460 0x55d6ee3308a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.788+0000 7f4fef086b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6ee326150 0x55d6ee330e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.788+0000 7f4fef086b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ee3906b0 con 0x55d6ee326150 2023-12-07T10:24:12.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.788+0000 7f4fef086b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ee390820 con 0x55d6ee322460 2023-12-07T10:24:12.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.788+0000 7f4fef086b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ee3909e0 con 0x55d6ee321b40 2023-12-07T10:24:12.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.788+0000 7f4fef086b80 10 client.0 _ll_register_callbacks cb 0x55d6ee216940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:12.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.788+0000 7f4fd959b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d6ee321b40 0x55d6ee330360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.788+0000 7f4fd8d9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee322460 0x55d6ee3308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.788+0000 7f4fd959b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d6ee321b40 0x55d6ee330360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55924/0 (socket says 172.21.15.119:55924) 2023-12-07T10:24:12.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.788+0000 7f4fd959b700 1 -- 172.21.15.119:0/1551609929 learned_addr learned my addr 172.21.15.119:0/1551609929 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:12.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.788+0000 7f4fd9d9c700 1 --2- 172.21.15.119:0/1551609929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6ee326150 0x55d6ee330e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.789+0000 7f4fd8d9a700 1 -- 172.21.15.119:0/1551609929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d6ee321b40 msgr2=0x55d6ee330360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:12.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.789+0000 7f4fd8d9a700 1 --2- 172.21.15.119:0/1551609929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d6ee321b40 0x55d6ee330360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.789+0000 7f4fd8d9a700 1 -- 172.21.15.119:0/1551609929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6ee326150 msgr2=0x55d6ee330e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:12.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.789+0000 7f4fd8d9a700 1 --2- 172.21.15.119:0/1551609929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6ee326150 0x55d6ee330e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.789+0000 7f4fd8d9a700 1 -- 172.21.15.119:0/1551609929 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc40040f0 con 0x55d6ee322460 2023-12-07T10:24:12.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.789+0000 7f4fd959b700 1 --2- 172.21.15.119:0/1551609929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d6ee321b40 0x55d6ee330360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:12.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.789+0000 7f4fd8d9a700 1 --2- 172.21.15.119:0/1551609929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee322460 0x55d6ee3308a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55d6ee32d430 tx=0x7f4fc4016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a98ffef9331bda18 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:12.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.789+0000 7f4fbcff9700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:24:12.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.789+0000 7f4fbcff9700 1 -- 172.21.15.119:0/1551609929 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4022040 con 0x55d6ee322460 2023-12-07T10:24:12.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.789+0000 7f4fbcff9700 1 -- 172.21.15.119:0/1551609929 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4fc402b2d0 con 0x55d6ee322460 2023-12-07T10:24:12.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.789+0000 7f4fef086b80 1 -- 172.21.15.119:0/1551609929 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4fcc00f040 con 0x55d6ee322460 2023-12-07T10:24:12.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.789+0000 7f4fbcff9700 1 -- 172.21.15.119:0/1551609929 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fc402b490 con 0x55d6ee322460 2023-12-07T10:24:12.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.789+0000 7f4fef086b80 10 client.26996 fetch_fsmap learned FSMap version 37 2023-12-07T10:24:12.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.789+0000 7f4fef086b80 1 -- 172.21.15.119:0/1551609929 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55d6ee09ede0 con 0x55d6ee322460 2023-12-07T10:24:12.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.790+0000 7f4fbcff9700 1 -- 172.21.15.119:0/1551609929 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fcc00f040 con 0x55d6ee322460 2023-12-07T10:24:12.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.790+0000 7f4fbcff9700 1 -- 172.21.15.119:0/1551609929 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 37) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f4fc4029b40 con 0x55d6ee322460 2023-12-07T10:24:12.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.790+0000 7f4fef086b80 10 client.26996 fetch_fsmap finished waiting for FSMap version 37 2023-12-07T10:24:12.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.790+0000 7f4fef086b80 10 client.26996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:24:12.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.790+0000 7f4fef086b80 20 client.26996 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:12.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.790+0000 7f4fef086b80 1 --2- 172.21.15.119:0/1551609929 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d6ee3922d0 0x55d6ee3946f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.790+0000 7f4fef086b80 4 client.26996 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-12-07T10:24:12.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.790+0000 7f4fef086b80 1 -- 172.21.15.119:0/1551609929 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d6ee23bfd0 con 0x55d6ee3922d0 2023-12-07T10:24:12.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.790+0000 7f4fd959b700 1 --2- 172.21.15.119:0/1551609929 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d6ee3922d0 0x55d6ee3946f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.791+0000 7f4fbcff9700 10 client.26996 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:12.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.791+0000 7f4fd959b700 1 --2- 172.21.15.119:0/1551609929 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d6ee3922d0 0x55d6ee3946f0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4025cc4fe27b05e0 server_cookie=a7c8aa17af0dea96 in_seq=0 out_seq=0 2023-12-07T10:24:12.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.792+0000 7f4fbcff9700 1 -- 172.21.15.119:0/1551609929 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d6ee23bfd0 con 0x55d6ee3922d0 2023-12-07T10:24:12.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.792+0000 7f4fbcff9700 10 client.26996 handle_command_reply: tid=0 2023-12-07T10:24:12.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.792+0000 7f4fef086b80 1 client.26996 shutdown 2023-12-07T10:24:12.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.792+0000 7f4fef086b80 1 -- 172.21.15.119:0/1551609929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee322460 msgr2=0x55d6ee3308a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:12.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.792+0000 7f4fef086b80 1 --2- 172.21.15.119:0/1551609929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee322460 0x55d6ee3308a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55d6ee32d430 tx=0x7f4fc4016000 comp rx=0 tx=0).stop 2023-12-07T10:24:12.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.792+0000 7f4fef086b80 1 -- 172.21.15.119:0/1551609929 shutdown_connections 2023-12-07T10:24:12.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.792+0000 7f4fef086b80 1 --2- 172.21.15.119:0/1551609929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6ee326150 0x55d6ee330e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.792+0000 7f4fef086b80 1 --2- 172.21.15.119:0/1551609929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d6ee321b40 0x55d6ee330360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.792+0000 7f4fef086b80 1 --2- 172.21.15.119:0/1551609929 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d6ee3922d0 0x55d6ee3946f0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.792+0000 7f4fef086b80 1 --2- 172.21.15.119:0/1551609929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d6ee322460 0x55d6ee3308a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.792+0000 7f4fbcff9700 0 client.26996 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:12.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.792+0000 7f4fef086b80 1 -- 172.21.15.119:0/1551609929 >> 172.21.15.119:0/1551609929 conn(0x55d6ee250360 msgr2=0x55d6ee31c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:12.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.792+0000 7f4fef086b80 1 -- 172.21.15.119:0/1551609929 shutdown_connections 2023-12-07T10:24:12.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.792+0000 7f4fef086b80 1 -- 172.21.15.119:0/1551609929 wait complete. 2023-12-07T10:24:12.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.792+0000 7f4fef086b80 20 client.26996 trim_cache size 0 max 16384 2023-12-07T10:24:12.797 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:24:12.797 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-07T10:24:12.798 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": { 2023-12-07T10:24:12.798 INFO:teuthology.orchestra.run.smithi119.stdout: "89c6d464-946d-4c3d-ac53-08791ef988a5": { 2023-12-07T10:24:12.798 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/", 2023-12-07T10:24:12.798 INFO:teuthology.orchestra.run.smithi119.stdout: "tag": "89c6d464-946d-4c3d-ac53-08791ef988a5", 2023-12-07T10:24:12.798 INFO:teuthology.orchestra.run.smithi119.stdout: "options": "recursive,force" 2023-12-07T10:24:12.798 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:24:12.798 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:24:12.798 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:24:12.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.793+0000 7f4fdb7ff700 1 -- 172.21.15.119:0/3777864861 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4fc00a1d00 msgr2=0x7f4fc00a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:12.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.793+0000 7f4fdb7ff700 1 --2- 172.21.15.119:0/3777864861 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4fc00a1d00 0x7f4fc00a41c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4fd419ac70 tx=0x7f4fc4009f90 comp rx=0 tx=0).stop 2023-12-07T10:24:12.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.793+0000 7f4fdb7ff700 1 -- 172.21.15.119:0/3777864861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fd4068a50 msgr2=0x7f4fd419f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:12.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.793+0000 7f4fdb7ff700 1 --2- 172.21.15.119:0/3777864861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fd4068a50 0x7f4fd419f980 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0004730 tx=0x7f4fd0004760 comp rx=0 tx=0).stop 2023-12-07T10:24:12.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.793+0000 7f4fdb7ff700 1 -- 172.21.15.119:0/3777864861 shutdown_connections 2023-12-07T10:24:12.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.793+0000 7f4fdb7ff700 1 --2- 172.21.15.119:0/3777864861 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4fc00a1d00 0x7f4fc00a41c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.793+0000 7f4fdb7ff700 1 --2- 172.21.15.119:0/3777864861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fd4106ad0 0x7f4fd4199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.793+0000 7f4fdb7ff700 1 --2- 172.21.15.119:0/3777864861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fd4068a50 0x7f4fd419f980 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.793+0000 7f4fdb7ff700 1 --2- 172.21.15.119:0/3777864861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fd4106110 0x7f4fd419fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.793+0000 7f4fdb7ff700 1 -- 172.21.15.119:0/3777864861 >> 172.21.15.119:0/3777864861 conn(0x7f4fd4077610 msgr2=0x7f4fd410cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:12.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.793+0000 7f4fdb7ff700 1 -- 172.21.15.119:0/3777864861 shutdown_connections 2023-12-07T10:24:12.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:12.793+0000 7f4fdb7ff700 1 -- 172.21.15.119:0/3777864861 wait complete. 2023-12-07T10:24:12.813 INFO:tasks.cephfs.filesystem:scrub status for tag:89c6d464-946d-4c3d-ac53-08791ef988a5 - {'path': '/', 'tag': '89c6d464-946d-4c3d-ac53-08791ef988a5', 'options': 'recursive,force'} 2023-12-07T10:24:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.961+0000 7f5f10bff700 1 -- 172.21.15.119:0/1708188002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f0c101500 msgr2=0x7f5f0c101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.961+0000 7f5f10bff700 1 --2- 172.21.15.119:0/1708188002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f0c101500 0x7f5f0c101980 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009920 tx=0x7f5f00009c30 comp rx=0 tx=0).stop 2023-12-07T10:24:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.961+0000 7f5f10bff700 1 -- 172.21.15.119:0/1708188002 shutdown_connections 2023-12-07T10:24:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.961+0000 7f5f10bff700 1 --2- 172.21.15.119:0/1708188002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f0c101ec0 0x7f5f0c10e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.961+0000 7f5f10bff700 1 --2- 172.21.15.119:0/1708188002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f0c1075c0 0x7f5f0c1079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.961+0000 7f5f10bff700 1 --2- 172.21.15.119:0/1708188002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f0c101500 0x7f5f0c101980 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.961+0000 7f5f10bff700 1 -- 172.21.15.119:0/1708188002 >> 172.21.15.119:0/1708188002 conn(0x7f5f0c0fcfb0 msgr2=0x7f5f0c0ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.961+0000 7f5f10bff700 1 -- 172.21.15.119:0/1708188002 shutdown_connections 2023-12-07T10:24:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.961+0000 7f5f10bff700 1 -- 172.21.15.119:0/1708188002 wait complete. 2023-12-07T10:24:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.962+0000 7f5f10bff700 1 Processor -- start 2023-12-07T10:24:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.962+0000 7f5f10bff700 1 -- start start 2023-12-07T10:24:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.963+0000 7f5f10bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f0c101500 0x7f5f0c073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.963+0000 7f5f10bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f0c101ec0 0x7f5f0c073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.963+0000 7f5f10bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f0c1075c0 0x7f5f0c074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.963+0000 7f5f10bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c1a5d70 con 0x7f5f0c101500 2023-12-07T10:24:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.963+0000 7f5f10bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c1a5ee0 con 0x7f5f0c101ec0 2023-12-07T10:24:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.963+0000 7f5f10bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c1a60a0 con 0x7f5f0c1075c0 2023-12-07T10:24:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.963+0000 7f5f09d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f0c101ec0 0x7f5f0c073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.963+0000 7f5f0a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f0c101500 0x7f5f0c073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.963+0000 7f5f09d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f0c101ec0 0x7f5f0c073e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:60860/0 (socket says 172.21.15.119:60860) 2023-12-07T10:24:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.963+0000 7f5f09d9b700 1 -- 172.21.15.119:0/3861241913 learned_addr learned my addr 172.21.15.119:0/3861241913 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.963+0000 7f5f0ad9d700 1 --2- 172.21.15.119:0/3861241913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f0c1075c0 0x7f5f0c074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.964+0000 7f5f0ad9d700 1 -- 172.21.15.119:0/3861241913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f0c101ec0 msgr2=0x7f5f0c073e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.964+0000 7f5f0ad9d700 1 --2- 172.21.15.119:0/3861241913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f0c101ec0 0x7f5f0c073e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.964+0000 7f5f0ad9d700 1 -- 172.21.15.119:0/3861241913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f0c101500 msgr2=0x7f5f0c073910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.964+0000 7f5f0ad9d700 1 --2- 172.21.15.119:0/3861241913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f0c101500 0x7f5f0c073910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.964+0000 7f5f0ad9d700 1 -- 172.21.15.119:0/3861241913 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f00009520 con 0x7f5f0c1075c0 2023-12-07T10:24:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.964+0000 7f5f09d9b700 1 --2- 172.21.15.119:0/3861241913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f0c101ec0 0x7f5f0c073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.964+0000 7f5f0ad9d700 1 --2- 172.21.15.119:0/3861241913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f0c1075c0 0x7f5f0c074390 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400efc0 tx=0x7f5f0400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.964+0000 7f5efb7fe700 1 -- 172.21.15.119:0/3861241913 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f04018070 con 0x7f5f0c1075c0 2023-12-07T10:24:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.964+0000 7f5f10bff700 1 -- 172.21.15.119:0/3861241913 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f0c1a6320 con 0x7f5f0c1075c0 2023-12-07T10:24:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.964+0000 7f5efb7fe700 1 -- 172.21.15.119:0/3861241913 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f0400f040 con 0x7f5f0c1075c0 2023-12-07T10:24:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.964+0000 7f5efb7fe700 1 -- 172.21.15.119:0/3861241913 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0400ccb0 con 0x7f5f0c1075c0 2023-12-07T10:24:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.964+0000 7f5f10bff700 1 -- 172.21.15.119:0/3861241913 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0c1a6830 con 0x7f5f0c1075c0 2023-12-07T10:24:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.968+0000 7f5efb7fe700 1 -- 172.21.15.119:0/3861241913 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f5f04003a00 con 0x7f5f0c1075c0 2023-12-07T10:24:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.969+0000 7f5f24521b80 1 Processor -- start 2023-12-07T10:24:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.969+0000 7f5efb7fe700 1 --2- 172.21.15.119:0/3861241913 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5ef40a1c80 0x7f5ef40a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.969+0000 7f5f24521b80 1 -- start start 2023-12-07T10:24:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.970+0000 7f5efb7fe700 1 -- 172.21.15.119:0/3861241913 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f5f040cd760 con 0x7f5f0c1075c0 2023-12-07T10:24:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.970+0000 7f5f09d9b700 1 --2- 172.21.15.119:0/3861241913 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5ef40a1c80 0x7f5ef40a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.970+0000 7f5f24521b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8b1efda60 0x55f8b1efde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.970+0000 7f5f24521b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b1efe380 0x55f8b1ef8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.970+0000 7f5f24521b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b1ef8ad0 0x55f8b1ef8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.970+0000 7f5f24521b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8b1e6c760 con 0x55f8b1ef8ad0 2023-12-07T10:24:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.970+0000 7f5f24521b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8b1e53740 con 0x55f8b1efe380 2023-12-07T10:24:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.970+0000 7f5f24521b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8b1e51060 con 0x55f8b1efda60 2023-12-07T10:24:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.970+0000 7f5f0a59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8b1efda60 0x55f8b1efde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.970+0000 7f5f0a59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8b1efda60 0x55f8b1efde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:52956/0 (socket says 172.21.15.119:52956) 2023-12-07T10:24:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.970+0000 7f5f0a59c700 1 -- 172.21.15.119:0/1807496862 learned_addr learned my addr 172.21.15.119:0/1807496862 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.970+0000 7f5f09d9b700 1 --2- 172.21.15.119:0/1807496862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b1efe380 0x55f8b1ef8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.970+0000 7f5f0ad9d700 1 --2- 172.21.15.119:0/1807496862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b1ef8ad0 0x55f8b1ef8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.971+0000 7f5f0a59c700 1 -- 172.21.15.119:0/1807496862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b1efe380 msgr2=0x55f8b1ef8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.971+0000 7f5f0a59c700 1 --2- 172.21.15.119:0/1807496862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b1efe380 0x55f8b1ef8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.971+0000 7f5f0a59c700 1 -- 172.21.15.119:0/1807496862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b1ef8ad0 msgr2=0x55f8b1ef8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.971+0000 7f5f0a59c700 1 --2- 172.21.15.119:0/1807496862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b1ef8ad0 0x55f8b1ef8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.971+0000 7f5f0a59c700 1 -- 172.21.15.119:0/1807496862 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8b1cc95d0 con 0x55f8b1efda60 2023-12-07T10:24:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.971+0000 7f5f0a59c700 1 --2- 172.21.15.119:0/1807496862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8b1efda60 0x55f8b1efde40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009d80 tx=0x7f5efc011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e338bed4b811a222 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.971+0000 7f5f09d9b700 1 --2- 172.21.15.119:0/3861241913 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5ef40a1c80 0x7f5ef40a4140 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c075040 tx=0x7f5f0000bf30 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.971+0000 7f5ef8ff9700 1 -- 172.21.15.119:0/1807496862 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc004230 con 0x55f8b1efda60 2023-12-07T10:24:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.971+0000 7f5ef8ff9700 1 -- 172.21.15.119:0/1807496862 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5efc004390 con 0x55f8b1efda60 2023-12-07T10:24:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.971+0000 7f5ef8ff9700 1 -- 172.21.15.119:0/1807496862 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc0197c0 con 0x55f8b1efda60 2023-12-07T10:24:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.973+0000 7f5f24521b80 1 -- 172.21.15.119:0/1807496862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8b1efda60 msgr2=0x55f8b1efde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.973+0000 7f5f24521b80 1 --2- 172.21.15.119:0/1807496862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8b1efda60 0x55f8b1efde40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009d80 tx=0x7f5efc011720 comp rx=0 tx=0).stop 2023-12-07T10:24:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.973+0000 7f5f24521b80 1 -- 172.21.15.119:0/1807496862 shutdown_connections 2023-12-07T10:24:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.973+0000 7f5f24521b80 1 --2- 172.21.15.119:0/1807496862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b1ef8ad0 0x55f8b1ef8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.973+0000 7f5f24521b80 1 --2- 172.21.15.119:0/1807496862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8b1efda60 0x55f8b1efde40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.973+0000 7f5f24521b80 1 --2- 172.21.15.119:0/1807496862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b1efe380 0x55f8b1ef8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.973+0000 7f5f24521b80 1 -- 172.21.15.119:0/1807496862 >> 172.21.15.119:0/1807496862 conn(0x55f8b1e27300 msgr2=0x55f8b1ef2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.973+0000 7f5f24521b80 1 -- 172.21.15.119:0/1807496862 shutdown_connections 2023-12-07T10:24:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.974+0000 7f5f24521b80 1 -- 172.21.15.119:0/1807496862 wait complete. 2023-12-07T10:24:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.974+0000 7f5f24521b80 1 -- start start 2023-12-07T10:24:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.974+0000 7f5f24521b80 1 Processor -- start 2023-12-07T10:24:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.975+0000 7f5f24521b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b1ef8ad0 0x55f8b1f043d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.975+0000 7f5f24521b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b1efda60 0x55f8b1f01710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.975+0000 7f5f24521b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8b1efe380 0x55f8b1f01e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.975+0000 7f5f24521b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8b1e4c950 con 0x55f8b1ef8ad0 2023-12-07T10:24:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.975+0000 7f5f24521b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8b1e06500 con 0x55f8b1efda60 2023-12-07T10:24:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.975+0000 7f5f24521b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8b1e061a0 con 0x55f8b1efe380 2023-12-07T10:24:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.975+0000 7f5f24521b80 10 client.0 _ll_register_callbacks cb 0x55f8b1ded8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.975+0000 7f5f0a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b1ef8ad0 0x55f8b1f043d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.975+0000 7f5f0a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b1ef8ad0 0x55f8b1f043d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34054/0 (socket says 172.21.15.119:34054) 2023-12-07T10:24:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.975+0000 7f5f0a59c700 1 -- 172.21.15.119:0/1097805551 learned_addr learned my addr 172.21.15.119:0/1097805551 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.976+0000 7f5f09d9b700 1 --2- 172.21.15.119:0/1097805551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b1efda60 0x55f8b1f01710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.976+0000 7f5f0ad9d700 1 --2- 172.21.15.119:0/1097805551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8b1efe380 0x55f8b1f01e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.976+0000 7f5f0a59c700 1 -- 172.21.15.119:0/1097805551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8b1efe380 msgr2=0x55f8b1f01e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:12.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.976+0000 7f5f0a59c700 1 --2- 172.21.15.119:0/1097805551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8b1efe380 0x55f8b1f01e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.976+0000 7f5f0a59c700 1 -- 172.21.15.119:0/1097805551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b1efda60 msgr2=0x55f8b1f01710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:12.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.976+0000 7f5f0a59c700 1 --2- 172.21.15.119:0/1097805551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b1efda60 0x55f8b1f01710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.976+0000 7f5f0a59c700 1 -- 172.21.15.119:0/1097805551 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5efc0099e0 con 0x55f8b1ef8ad0 2023-12-07T10:24:12.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.976+0000 7f5f09d9b700 1 --2- 172.21.15.119:0/1097805551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b1efda60 0x55f8b1f01710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:12.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.976+0000 7f5f0a59c700 1 --2- 172.21.15.119:0/1097805551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b1ef8ad0 0x55f8b1f043d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5efc01b040 tx=0x7f5efc003930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b75313be83a900 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.976+0000 7f5ef1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:24:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.977+0000 7f5ef1ffb700 1 -- 172.21.15.119:0/1097805551 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc005ce0 con 0x55f8b1ef8ad0 2023-12-07T10:24:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.977+0000 7f5ef1ffb700 1 -- 172.21.15.119:0/1097805551 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f5efc005040 con 0x55f8b1ef8ad0 2023-12-07T10:24:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.977+0000 7f5f24521b80 1 -- 172.21.15.119:0/1097805551 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f8b1eeb6f0 con 0x55f8b1ef8ad0 2023-12-07T10:24:12.979 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:12 smithi119 ceph-mon[97908]: scrub queued for path: / 2023-12-07T10:24:12.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:12 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:12.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:12 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:24:12.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:12 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:24:12.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:12 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:12.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:12 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:12.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:12 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:24:12.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:12 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:12.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:12 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:24:12.980 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:12 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.977+0000 7f5ef1ffb700 1 -- 172.21.15.119:0/1097805551 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5efc005200 con 0x55f8b1ef8ad0 2023-12-07T10:24:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.977+0000 7f5f24521b80 10 client.16986 fetch_fsmap learned FSMap version 37 2023-12-07T10:24:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.977+0000 7f5f24521b80 1 -- 172.21.15.119:0/1097805551 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55f8b1c75d80 con 0x55f8b1ef8ad0 2023-12-07T10:24:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.978+0000 7f5ef1ffb700 1 -- 172.21.15.119:0/1097805551 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f8b1eeb6f0 con 0x55f8b1ef8ad0 2023-12-07T10:24:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.978+0000 7f5ef1ffb700 1 -- 172.21.15.119:0/1097805551 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 37) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5efc021040 con 0x55f8b1ef8ad0 2023-12-07T10:24:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.978+0000 7f5f24521b80 10 client.16986 fetch_fsmap finished waiting for FSMap version 37 2023-12-07T10:24:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.978+0000 7f5f24521b80 10 client.16986 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:24:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.978+0000 7f5f24521b80 20 client.16986 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.978+0000 7f5f24521b80 1 --2- 172.21.15.119:0/1097805551 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f8b1f1a6f0 0x55f8b1f1cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.978+0000 7f5f24521b80 4 client.16986 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:24:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.979+0000 7f5f24521b80 1 -- 172.21.15.119:0/1097805551 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f8b1ec7740 con 0x55f8b1f1a6f0 2023-12-07T10:24:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.979+0000 7f5f09d9b700 1 --2- 172.21.15.119:0/1097805551 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f8b1f1a6f0 0x55f8b1f1cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.979+0000 7f5ef1ffb700 10 client.16986 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.979+0000 7f5f09d9b700 1 --2- 172.21.15.119:0/1097805551 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f8b1f1a6f0 0x55f8b1f1cb10 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a024dced27b7d4f7 server_cookie=f3d26e53294c293a in_seq=0 out_seq=0 2023-12-07T10:24:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.980+0000 7f5ef1ffb700 1 -- 172.21.15.119:0/1097805551 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f8b1ec7740 con 0x55f8b1f1a6f0 2023-12-07T10:24:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.980+0000 7f5ef1ffb700 10 client.16986 handle_command_reply: tid=0 2023-12-07T10:24:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.980+0000 7f5f24521b80 1 client.16986 shutdown 2023-12-07T10:24:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.980+0000 7f5f24521b80 1 -- 172.21.15.119:0/1097805551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b1ef8ad0 msgr2=0x55f8b1f043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.980+0000 7f5f24521b80 1 --2- 172.21.15.119:0/1097805551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b1ef8ad0 0x55f8b1f043d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5efc01b040 tx=0x7f5efc003930 comp rx=0 tx=0).stop 2023-12-07T10:24:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.980+0000 7f5f24521b80 1 -- 172.21.15.119:0/1097805551 shutdown_connections 2023-12-07T10:24:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.980+0000 7f5f24521b80 1 --2- 172.21.15.119:0/1097805551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8b1efe380 0x55f8b1f01e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.980+0000 7f5f24521b80 1 --2- 172.21.15.119:0/1097805551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b1ef8ad0 0x55f8b1f043d0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.981+0000 7f5f24521b80 1 --2- 172.21.15.119:0/1097805551 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f8b1f1a6f0 0x55f8b1f1cb10 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.981+0000 7f5f24521b80 1 --2- 172.21.15.119:0/1097805551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b1efda60 0x55f8b1f01710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.981+0000 7f5f24521b80 1 -- 172.21.15.119:0/1097805551 >> 172.21.15.119:0/1097805551 conn(0x55f8b1e27300 msgr2=0x55f8b1eed980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.981+0000 7f5ef1ffb700 0 client.16986 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.981+0000 7f5f24521b80 1 -- 172.21.15.119:0/1097805551 shutdown_connections 2023-12-07T10:24:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.981+0000 7f5f24521b80 1 -- 172.21.15.119:0/1097805551 wait complete. 2023-12-07T10:24:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.981+0000 7f5f24521b80 20 client.16986 trim_cache size 0 max 16384 2023-12-07T10:24:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.997+0000 7f5f24521b80 1 Processor -- start 2023-12-07T10:24:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.998+0000 7f5f24521b80 1 -- start start 2023-12-07T10:24:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.998+0000 7f5f24521b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8b1ef8ad0 0x55f8b1f16090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.998+0000 7f5f24521b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b1efda60 0x55f8b1f115e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.998+0000 7f5f24521b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b1efe380 0x55f8b1f11b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.998+0000 7f5f24521b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8b1ccb8c0 con 0x55f8b1efe380 2023-12-07T10:24:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.998+0000 7f5f24521b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8b1f17ca0 con 0x55f8b1efda60 2023-12-07T10:24:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.998+0000 7f5f24521b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8b1f02690 con 0x55f8b1ef8ad0 2023-12-07T10:24:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.998+0000 7f5f09d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b1efda60 0x55f8b1f115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.998+0000 7f5f09d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b1efda60 0x55f8b1f115e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:60900/0 (socket says 172.21.15.119:60900) 2023-12-07T10:24:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.998+0000 7f5f09d9b700 1 -- 172.21.15.119:0/3301152783 learned_addr learned my addr 172.21.15.119:0/3301152783 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.998+0000 7f5f0a59c700 1 --2- 172.21.15.119:0/3301152783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8b1ef8ad0 0x55f8b1f16090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.998+0000 7f5f0ad9d700 1 --2- 172.21.15.119:0/3301152783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b1efe380 0x55f8b1f11b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.999+0000 7f5f09d9b700 1 -- 172.21.15.119:0/3301152783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8b1ef8ad0 msgr2=0x55f8b1f16090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.999+0000 7f5f09d9b700 1 --2- 172.21.15.119:0/3301152783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8b1ef8ad0 0x55f8b1f16090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.999+0000 7f5f09d9b700 1 -- 172.21.15.119:0/3301152783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b1efe380 msgr2=0x55f8b1f11b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.999+0000 7f5f09d9b700 1 --2- 172.21.15.119:0/3301152783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b1efe380 0x55f8b1f11b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.999+0000 7f5f09d9b700 1 -- 172.21.15.119:0/3301152783 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8b1c5ec00 con 0x55f8b1efda60 2023-12-07T10:24:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.999+0000 7f5f0ad9d700 1 --2- 172.21.15.119:0/3301152783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b1efe380 0x55f8b1f11b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:24:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.999+0000 7f5f09d9b700 1 --2- 172.21.15.119:0/3301152783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b1efda60 0x55f8b1f115e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5f00000c00 tx=0x7f5f0002a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76a40300b8f4233b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.999+0000 7f5ef97fa700 1 -- 172.21.15.119:0/3301152783 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0001c040 con 0x55f8b1efda60 2023-12-07T10:24:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.999+0000 7f5ef97fa700 1 -- 172.21.15.119:0/3301152783 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f000327f0 con 0x55f8b1efda60 2023-12-07T10:24:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:12.999+0000 7f5ef97fa700 1 -- 172.21.15.119:0/3301152783 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f00005590 con 0x55f8b1efda60 2023-12-07T10:24:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.000+0000 7f5f24521b80 1 -- 172.21.15.119:0/3301152783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b1efda60 msgr2=0x55f8b1f115e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.000+0000 7f5f24521b80 1 --2- 172.21.15.119:0/3301152783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b1efda60 0x55f8b1f115e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5f00000c00 tx=0x7f5f0002a000 comp rx=0 tx=0).stop 2023-12-07T10:24:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.000+0000 7f5f24521b80 1 -- 172.21.15.119:0/3301152783 shutdown_connections 2023-12-07T10:24:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.000+0000 7f5f24521b80 1 --2- 172.21.15.119:0/3301152783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b1efe380 0x55f8b1f11b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.000+0000 7f5f24521b80 1 --2- 172.21.15.119:0/3301152783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8b1ef8ad0 0x55f8b1f16090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.000+0000 7f5f24521b80 1 --2- 172.21.15.119:0/3301152783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b1efda60 0x55f8b1f115e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.000+0000 7f5f24521b80 1 -- 172.21.15.119:0/3301152783 >> 172.21.15.119:0/3301152783 conn(0x55f8b1e27300 msgr2=0x55f8b1f21470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.000+0000 7f5f24521b80 1 -- 172.21.15.119:0/3301152783 shutdown_connections 2023-12-07T10:24:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.000+0000 7f5f24521b80 1 -- 172.21.15.119:0/3301152783 wait complete. 2023-12-07T10:24:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.001+0000 7f5f24521b80 1 -- start start 2023-12-07T10:24:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.001+0000 7f5f24521b80 1 Processor -- start 2023-12-07T10:24:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.002+0000 7f5f24521b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8b1ef8ad0 0x55f8b1f117d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.002+0000 7f5f24521b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b1efda60 0x55f8b1f0f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.002+0000 7f5f24521b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b1efe380 0x55f8b1f0fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.002+0000 7f5f24521b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8b1f10040 con 0x55f8b1efe380 2023-12-07T10:24:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.002+0000 7f5f24521b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8b1f10180 con 0x55f8b1efda60 2023-12-07T10:24:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.002+0000 7f5f24521b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8b1f10340 con 0x55f8b1ef8ad0 2023-12-07T10:24:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.002+0000 7f5f24521b80 10 client.0 _ll_register_callbacks cb 0x55f8b1ded8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.002+0000 7f5f0a59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8b1ef8ad0 0x55f8b1f117d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.002+0000 7f5f0a59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8b1ef8ad0 0x55f8b1f117d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:53000/0 (socket says 172.21.15.119:53000) 2023-12-07T10:24:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.002+0000 7f5f0a59c700 1 -- 172.21.15.119:0/1889067877 learned_addr learned my addr 172.21.15.119:0/1889067877 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.002+0000 7f5f09d9b700 1 --2- 172.21.15.119:0/1889067877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b1efda60 0x55f8b1f0f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.002+0000 7f5f0ad9d700 1 --2- 172.21.15.119:0/1889067877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b1efe380 0x55f8b1f0fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.002+0000 7f5f0a59c700 1 -- 172.21.15.119:0/1889067877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b1efda60 msgr2=0x55f8b1f0f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.002+0000 7f5f0a59c700 1 --2- 172.21.15.119:0/1889067877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b1efda60 0x55f8b1f0f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.002+0000 7f5f0a59c700 1 -- 172.21.15.119:0/1889067877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b1efe380 msgr2=0x55f8b1f0fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.002+0000 7f5f0a59c700 1 --2- 172.21.15.119:0/1889067877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b1efe380 0x55f8b1f0fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.002+0000 7f5f0a59c700 1 -- 172.21.15.119:0/1889067877 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0001f040 con 0x55f8b1ef8ad0 2023-12-07T10:24:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.002+0000 7f5f0ad9d700 1 --2- 172.21.15.119:0/1889067877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b1efe380 0x55f8b1f0fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:24:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.002+0000 7f5f0a59c700 1 --2- 172.21.15.119:0/1889067877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8b1ef8ad0 0x55f8b1f117d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5efc0234b0 tx=0x7f5efc005e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ff5c641e1806db2 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.002+0000 7f5ef1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:24:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.003+0000 7f5ef1ffb700 1 -- 172.21.15.119:0/1889067877 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc003d70 con 0x55f8b1ef8ad0 2023-12-07T10:24:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.003+0000 7f5ef1ffb700 1 -- 172.21.15.119:0/1889067877 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f5efc000e40 con 0x55f8b1ef8ad0 2023-12-07T10:24:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.003+0000 7f5f24521b80 1 -- 172.21.15.119:0/1889067877 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5efc004390 con 0x55f8b1ef8ad0 2023-12-07T10:24:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.003+0000 7f5ef1ffb700 1 -- 172.21.15.119:0/1889067877 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5efc023620 con 0x55f8b1ef8ad0 2023-12-07T10:24:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.003+0000 7f5f24521b80 10 client.26995 fetch_fsmap learned FSMap version 37 2023-12-07T10:24:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.003+0000 7f5f24521b80 1 -- 172.21.15.119:0/1889067877 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55f8b1c9c1f0 con 0x55f8b1ef8ad0 2023-12-07T10:24:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.003+0000 7f5ef1ffb700 1 -- 172.21.15.119:0/1889067877 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5efc004390 con 0x55f8b1ef8ad0 2023-12-07T10:24:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.003+0000 7f5ef1ffb700 1 -- 172.21.15.119:0/1889067877 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 37) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5efc021030 con 0x55f8b1ef8ad0 2023-12-07T10:24:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.003+0000 7f5f24521b80 10 client.26995 fetch_fsmap finished waiting for FSMap version 37 2023-12-07T10:24:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.003+0000 7f5f24521b80 10 client.26995 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:24:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.003+0000 7f5f24521b80 20 client.26995 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.003+0000 7f5f24521b80 1 --2- 172.21.15.119:0/1889067877 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f8b1f70d80 0x55f8b1f731a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.003+0000 7f5f24521b80 4 client.26995 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:24:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.003+0000 7f5f24521b80 1 -- 172.21.15.119:0/1889067877 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f8b1e12f70 con 0x55f8b1f70d80 2023-12-07T10:24:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.004+0000 7f5f09d9b700 1 --2- 172.21.15.119:0/1889067877 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f8b1f70d80 0x55f8b1f731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.004+0000 7f5ef1ffb700 10 client.26995 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.004+0000 7f5f09d9b700 1 --2- 172.21.15.119:0/1889067877 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f8b1f70d80 0x55f8b1f731a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d0376fa68e355cb server_cookie=990231dd5a292b05 in_seq=0 out_seq=0 2023-12-07T10:24:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.048+0000 7f5ef1ffb700 1 -- 172.21.15.119:0/1889067877 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f8b1e12f70 con 0x55f8b1f70d80 2023-12-07T10:24:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.048+0000 7f5ef1ffb700 10 client.26995 handle_command_reply: tid=0 2023-12-07T10:24:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.048+0000 7f5f24521b80 1 client.26995 shutdown 2023-12-07T10:24:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.049+0000 7f5f24521b80 1 -- 172.21.15.119:0/1889067877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8b1ef8ad0 msgr2=0x55f8b1f117d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.049+0000 7f5f24521b80 1 --2- 172.21.15.119:0/1889067877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8b1ef8ad0 0x55f8b1f117d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5efc0234b0 tx=0x7f5efc005e30 comp rx=0 tx=0).stop 2023-12-07T10:24:13.050 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.049+0000 7f5f24521b80 1 -- 172.21.15.119:0/1889067877 shutdown_connections 2023-12-07T10:24:13.051 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.049+0000 7f5f24521b80 1 --2- 172.21.15.119:0/1889067877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8b1efe380 0x55f8b1f0fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:13.051 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.049+0000 7f5f24521b80 1 --2- 172.21.15.119:0/1889067877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8b1ef8ad0 0x55f8b1f117d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:13.051 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.049+0000 7f5f24521b80 1 --2- 172.21.15.119:0/1889067877 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f8b1f70d80 0x55f8b1f731a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:13.051 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.049+0000 7f5f24521b80 1 --2- 172.21.15.119:0/1889067877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f8b1efda60 0x55f8b1f0f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:13.051 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.049+0000 7f5f24521b80 1 -- 172.21.15.119:0/1889067877 >> 172.21.15.119:0/1889067877 conn(0x55f8b1e27300 msgr2=0x55f8b1efb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:13.051 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.049+0000 7f5ef1ffb700 0 client.26995 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:13.051 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.050+0000 7f5f24521b80 1 -- 172.21.15.119:0/1889067877 shutdown_connections 2023-12-07T10:24:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.050+0000 7f5f24521b80 1 -- 172.21.15.119:0/1889067877 wait complete. 2023-12-07T10:24:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.050+0000 7f5f24521b80 20 client.26995 trim_cache size 0 max 16384 2023-12-07T10:24:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:24:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:24:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:24:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:24:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.051+0000 7f5f10bff700 1 -- 172.21.15.119:0/3861241913 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5ef40a1c80 msgr2=0x7f5ef40a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.051+0000 7f5f10bff700 1 --2- 172.21.15.119:0/3861241913 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5ef40a1c80 0x7f5ef40a4140 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c075040 tx=0x7f5f0000bf30 comp rx=0 tx=0).stop 2023-12-07T10:24:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.051+0000 7f5f10bff700 1 -- 172.21.15.119:0/3861241913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f0c1075c0 msgr2=0x7f5f0c074390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.051+0000 7f5f10bff700 1 --2- 172.21.15.119:0/3861241913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f0c1075c0 0x7f5f0c074390 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400efc0 tx=0x7f5f0400c3a0 comp rx=0 tx=0).stop 2023-12-07T10:24:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.051+0000 7f5f10bff700 1 -- 172.21.15.119:0/3861241913 shutdown_connections 2023-12-07T10:24:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.051+0000 7f5f10bff700 1 --2- 172.21.15.119:0/3861241913 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5ef40a1c80 0x7f5ef40a4140 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.051+0000 7f5f10bff700 1 --2- 172.21.15.119:0/3861241913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f0c1075c0 0x7f5f0c074390 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.051+0000 7f5f10bff700 1 --2- 172.21.15.119:0/3861241913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f0c101500 0x7f5f0c073910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.051+0000 7f5f10bff700 1 --2- 172.21.15.119:0/3861241913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f0c101ec0 0x7f5f0c073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.051+0000 7f5f10bff700 1 -- 172.21.15.119:0/3861241913 >> 172.21.15.119:0/3861241913 conn(0x7f5f0c0fcfb0 msgr2=0x7f5f0c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.051+0000 7f5f10bff700 1 -- 172.21.15.119:0/3861241913 shutdown_connections 2023-12-07T10:24:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:13.051+0000 7f5f10bff700 1 -- 172.21.15.119:0/3861241913 wait complete. 2023-12-07T10:24:13.264 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:12 smithi159 ceph-mon[134726]: scrub queued for path: / 2023-12-07T10:24:13.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:12 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:13.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:12 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:24:13.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:12 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:24:13.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:12 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:13.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:12 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:13.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:12 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:24:13.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:12 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:13.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:12 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:24:13.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:12 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:13.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:12 smithi153 ceph-mon[133625]: scrub queued for path: / 2023-12-07T10:24:13.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:12 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:13.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:12 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:24:13.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:12 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:24:13.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:12 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:13.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:12 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:13.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:12 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:24:13.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:12 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:13.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:12 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:24:13.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:12 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:13.551 INFO:tasks.workunit.client.0.smithi119.stdout: 1 221149 6.76 MB/sec execute 26 sec latency 25.135 ms 2023-12-07T10:24:14.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:13 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:24:14.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:13 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:24:14.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:13 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:24:14.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:13 smithi119 ceph-mon[97908]: Scrub error on inode 0x20000000215 (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:13 smithi119 ceph-mon[97908]: Scrub error on inode 0x20000000211 (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:13 smithi119 ceph-mon[97908]: Scrub error on inode 0x20000000214 (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:13 smithi119 ceph-mon[97908]: Scrub error on inode 0x2000000020e (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:13 smithi119 ceph-mon[97908]: Scrub error on inode 0x20000000212 (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:13 smithi119 ceph-mon[97908]: Scrub error on inode 0x20000000210 (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:13 smithi119 ceph-mon[97908]: Scrub error on inode 0x20000000216 (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:13 smithi119 ceph-mon[97908]: Scrub error on inode 0x20000000213 (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:13 smithi119 ceph-mon[97908]: Scrub error on inode 0x2000000020f (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:13 smithi119 ceph-mon[97908]: Scrub error on inode 0x2000000020d (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:13 smithi119 ceph-mon[97908]: Scrub error on inode 0x2000000020c (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:13 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:14.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:13 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:14.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:13 smithi119 ceph-mon[97908]: scrub summary: active paths [/] 2023-12-07T10:24:14.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:13 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:14.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:13 smithi119 ceph-mon[97908]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:14.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:13 smithi119 ceph-mon[97908]: pgmap v117: 129 pgs: 129 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2023-12-07T10:24:14.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:13 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:24:14.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:13 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:24:14.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:13 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:24:14.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:13 smithi159 ceph-mon[134726]: Scrub error on inode 0x20000000215 (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:13 smithi159 ceph-mon[134726]: Scrub error on inode 0x20000000211 (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:13 smithi159 ceph-mon[134726]: Scrub error on inode 0x20000000214 (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:13 smithi159 ceph-mon[134726]: Scrub error on inode 0x2000000020e (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:13 smithi159 ceph-mon[134726]: Scrub error on inode 0x20000000212 (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:13 smithi159 ceph-mon[134726]: Scrub error on inode 0x20000000210 (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:13 smithi159 ceph-mon[134726]: Scrub error on inode 0x20000000216 (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:13 smithi159 ceph-mon[134726]: Scrub error on inode 0x20000000213 (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:13 smithi159 ceph-mon[134726]: Scrub error on inode 0x2000000020f (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:13 smithi159 ceph-mon[134726]: Scrub error on inode 0x2000000020d (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:13 smithi159 ceph-mon[134726]: Scrub error on inode 0x2000000020c (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:13 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:14.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:13 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:14.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:13 smithi159 ceph-mon[134726]: scrub summary: active paths [/] 2023-12-07T10:24:14.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:13 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:14.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:13 smithi159 ceph-mon[134726]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:14.263 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:13 smithi159 ceph-mon[134726]: pgmap v117: 129 pgs: 129 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2023-12-07T10:24:14.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:13 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:24:14.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:13 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:24:14.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:13 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:24:14.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:13 smithi153 ceph-mon[133625]: Scrub error on inode 0x20000000215 (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:13 smithi153 ceph-mon[133625]: Scrub error on inode 0x20000000211 (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:13 smithi153 ceph-mon[133625]: Scrub error on inode 0x20000000214 (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:13 smithi153 ceph-mon[133625]: Scrub error on inode 0x2000000020e (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:13 smithi153 ceph-mon[133625]: Scrub error on inode 0x20000000212 (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:13 smithi153 ceph-mon[133625]: Scrub error on inode 0x20000000210 (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:13 smithi153 ceph-mon[133625]: Scrub error on inode 0x20000000216 (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:13 smithi153 ceph-mon[133625]: Scrub error on inode 0x20000000213 (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:13 smithi153 ceph-mon[133625]: Scrub error on inode 0x2000000020f (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:13 smithi153 ceph-mon[133625]: Scrub error on inode 0x2000000020d (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:13 smithi153 ceph-mon[133625]: Scrub error on inode 0x2000000020c (/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG) see mds.f log and `damage ls` output for details 2023-12-07T10:24:14.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:13 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:14.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:13 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:14.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:13 smithi153 ceph-mon[133625]: scrub summary: active paths [/] 2023-12-07T10:24:14.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:13 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:14.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:13 smithi153 ceph-mon[133625]: scrub summary: idle+waiting paths [/] 2023-12-07T10:24:14.280 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:13 smithi153 ceph-mon[133625]: pgmap v117: 129 pgs: 129 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2023-12-07T10:24:14.551 INFO:tasks.workunit.client.0.smithi119.stdout: 1 221149 6.51 MB/sec execute 27 sec latency 1001.766 ms 2023-12-07T10:24:15.552 INFO:tasks.workunit.client.0.smithi119.stdout: 1 221149 6.28 MB/sec execute 28 sec latency 2001.944 ms 2023-12-07T10:24:16.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:15 smithi159 ceph-mon[134726]: pgmap v118: 129 pgs: 129 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 191 op/s 2023-12-07T10:24:16.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:15 smithi153 ceph-mon[133625]: pgmap v118: 129 pgs: 129 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 191 op/s 2023-12-07T10:24:16.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:15 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mon-a[97884]: 2023-12-07T10:24:15.986+0000 7fc367b21700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-12-07T10:24:16.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:15 smithi119 ceph-mon[97908]: pgmap v118: 129 pgs: 129 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 191 op/s 2023-12-07T10:24:16.552 INFO:tasks.workunit.client.0.smithi119.stdout: 1 221149 6.06 MB/sec execute 29 sec latency 3002.088 ms 2023-12-07T10:24:17.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:17 smithi159 ceph-mon[134726]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-12-07T10:24:17.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:17 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] up:active 2023-12-07T10:24:17.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:17 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:24:17.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:17 smithi153 ceph-mon[133625]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-12-07T10:24:17.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:17 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] up:active 2023-12-07T10:24:17.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:17 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:24:17.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:17 smithi119 ceph-mon[97908]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-12-07T10:24:17.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:17 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] up:active 2023-12-07T10:24:17.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:17 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:24:17.552 INFO:tasks.workunit.client.0.smithi119.stdout: 1 221149 5.86 MB/sec execute 30 sec latency 4002.190 ms 2023-12-07T10:24:18.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:18 smithi159 ceph-mon[134726]: pgmap v119: 129 pgs: 129 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 261 op/s 2023-12-07T10:24:18.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:18 smithi153 ceph-mon[133625]: pgmap v119: 129 pgs: 129 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 261 op/s 2023-12-07T10:24:18.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:18 smithi119 ceph-mon[97908]: pgmap v119: 129 pgs: 129 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 261 op/s 2023-12-07T10:24:18.552 INFO:tasks.workunit.client.0.smithi119.stdout: 1 221149 5.67 MB/sec execute 31 sec latency 5002.267 ms 2023-12-07T10:24:19.552 INFO:tasks.workunit.client.0.smithi119.stdout: 1 221149 5.50 MB/sec execute 32 sec latency 6002.340 ms 2023-12-07T10:24:20.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:20 smithi159 ceph-mon[134726]: pgmap v120: 129 pgs: 129 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 931 KiB/s wr, 159 op/s 2023-12-07T10:24:20.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:20 smithi153 ceph-mon[133625]: pgmap v120: 129 pgs: 129 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 931 KiB/s wr, 159 op/s 2023-12-07T10:24:20.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:20 smithi119 ceph-mon[97908]: pgmap v120: 129 pgs: 129 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 931 KiB/s wr, 159 op/s 2023-12-07T10:24:20.553 INFO:tasks.workunit.client.0.smithi119.stdout: 1 221149 5.33 MB/sec execute 33 sec latency 7002.494 ms 2023-12-07T10:24:21.552 INFO:tasks.workunit.client.0.smithi119.stdout: 1 223454 5.62 MB/sec execute 34 sec latency 7115.719 ms 2023-12-07T10:24:22.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:21 smithi119 ceph-mon[97908]: pgmap v121: 129 pgs: 129 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 971 KiB/s wr, 171 op/s 2023-12-07T10:24:22.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:21 smithi159 ceph-mon[134726]: pgmap v121: 129 pgs: 129 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 971 KiB/s wr, 171 op/s 2023-12-07T10:24:22.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:21 smithi153 ceph-mon[133625]: pgmap v121: 129 pgs: 129 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 971 KiB/s wr, 171 op/s 2023-12-07T10:24:22.552 INFO:tasks.workunit.client.0.smithi119.stdout: 1 224639 5.57 MB/sec execute 35 sec latency 11.358 ms 2023-12-07T10:24:23.552 INFO:tasks.workunit.client.0.smithi119.stdout: 1 227146 5.85 MB/sec execute 36 sec latency 12.915 ms 2023-12-07T10:24:24.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:23 smithi159 ceph-mon[134726]: pgmap v122: 129 pgs: 129 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 407 KiB/s wr, 83 op/s 2023-12-07T10:24:24.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:23 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:24:24.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:23 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:24:24.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:23 smithi153 ceph-mon[133625]: pgmap v122: 129 pgs: 129 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 407 KiB/s wr, 83 op/s 2023-12-07T10:24:24.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:23 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:24:24.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:23 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:24:24.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:23 smithi119 ceph-mon[97908]: pgmap v122: 129 pgs: 129 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 407 KiB/s wr, 83 op/s 2023-12-07T10:24:24.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:23 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:24:24.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:23 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:24:24.552 INFO:tasks.workunit.client.0.smithi119.stdout: 1 228210 5.79 MB/sec execute 37 sec latency 16.210 ms 2023-12-07T10:24:25.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:24 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:24:25.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:24 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:24:25.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:24 smithi159 ceph-mon[134726]: scrub summary: idle 2023-12-07T10:24:25.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:24 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:24:25.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:24 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:24:25.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:24 smithi153 ceph-mon[133625]: scrub summary: idle 2023-12-07T10:24:25.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:24 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:24:25.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:24 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:24:25.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:24 smithi119 ceph-mon[97908]: scrub summary: idle 2023-12-07T10:24:25.553 INFO:tasks.workunit.client.0.smithi119.stdout: 1 230229 5.93 MB/sec execute 38 sec latency 225.819 ms 2023-12-07T10:24:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.223+0000 7ff2dadff700 1 -- 172.21.15.119:0/3912688757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2d4109720 msgr2=0x7ff2d4109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.223+0000 7ff2dadff700 1 --2- 172.21.15.119:0/3912688757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2d4109720 0x7ff2d4109b00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8009990 tx=0x7ff2c8009ca0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.224+0000 7ff2dadff700 1 -- 172.21.15.119:0/3912688757 shutdown_connections 2023-12-07T10:24:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.224+0000 7ff2dadff700 1 --2- 172.21.15.119:0/3912688757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2d41040d0 0x7ff2d41105e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.224+0000 7ff2dadff700 1 --2- 172.21.15.119:0/3912688757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2d4109720 0x7ff2d4109b00 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.224+0000 7ff2dadff700 1 --2- 172.21.15.119:0/3912688757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2d4103710 0x7ff2d4103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.224+0000 7ff2dadff700 1 -- 172.21.15.119:0/3912688757 >> 172.21.15.119:0/3912688757 conn(0x7ff2d40ff1c0 msgr2=0x7ff2d41015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.224+0000 7ff2dadff700 1 -- 172.21.15.119:0/3912688757 shutdown_connections 2023-12-07T10:24:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.224+0000 7ff2dadff700 1 -- 172.21.15.119:0/3912688757 wait complete. 2023-12-07T10:24:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.225+0000 7ff2dadff700 1 Processor -- start 2023-12-07T10:24:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.226+0000 7ff2dadff700 1 -- start start 2023-12-07T10:24:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.226+0000 7ff2dadff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2d4103710 0x7ff2d419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.226+0000 7ff2dadff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2d41040d0 0x7ff2d419fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.226+0000 7ff2dadff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2d4109720 0x7ff2d41999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.226+0000 7ff2dadff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d419a040 con 0x7ff2d4103710 2023-12-07T10:24:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.226+0000 7ff2dadff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d419a1b0 con 0x7ff2d4109720 2023-12-07T10:24:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.226+0000 7ff2dadff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d419a370 con 0x7ff2d41040d0 2023-12-07T10:24:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.226+0000 7ff2d3fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2d41040d0 0x7ff2d419fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.227+0000 7ff2d3fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2d41040d0 0x7ff2d419fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:46846/0 (socket says 172.21.15.119:46846) 2023-12-07T10:24:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.227+0000 7ff2d3fff700 1 -- 172.21.15.119:0/1797500717 learned_addr learned my addr 172.21.15.119:0/1797500717 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.227+0000 7ff2d8b9b700 1 --2- 172.21.15.119:0/1797500717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2d4103710 0x7ff2d419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.227+0000 7ff2d939c700 1 --2- 172.21.15.119:0/1797500717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2d4109720 0x7ff2d41999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.227+0000 7ff2d3fff700 1 -- 172.21.15.119:0/1797500717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2d4109720 msgr2=0x7ff2d41999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.227+0000 7ff2d3fff700 1 --2- 172.21.15.119:0/1797500717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2d4109720 0x7ff2d41999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.227+0000 7ff2d3fff700 1 -- 172.21.15.119:0/1797500717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2d4103710 msgr2=0x7ff2d419f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.227+0000 7ff2d3fff700 1 --2- 172.21.15.119:0/1797500717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2d4103710 0x7ff2d419f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.227+0000 7ff2d3fff700 1 -- 172.21.15.119:0/1797500717 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c80095f0 con 0x7ff2d41040d0 2023-12-07T10:24:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.227+0000 7ff2d3fff700 1 --2- 172.21.15.119:0/1797500717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2d41040d0 0x7ff2d419fdf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff2c000e9a0 tx=0x7ff2c000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.227+0000 7ff2d1ffb700 1 -- 172.21.15.119:0/1797500717 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0009790 con 0x7ff2d41040d0 2023-12-07T10:24:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.228+0000 7ff2d1ffb700 1 -- 172.21.15.119:0/1797500717 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2c0010430 con 0x7ff2d41040d0 2023-12-07T10:24:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.228+0000 7ff2dadff700 1 -- 172.21.15.119:0/1797500717 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d419a650 con 0x7ff2d41040d0 2023-12-07T10:24:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.228+0000 7ff2d1ffb700 1 -- 172.21.15.119:0/1797500717 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0018680 con 0x7ff2d41040d0 2023-12-07T10:24:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.228+0000 7ff2dadff700 1 -- 172.21.15.119:0/1797500717 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d41a68a0 con 0x7ff2d41040d0 2023-12-07T10:24:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.231+0000 7ff2d1ffb700 1 -- 172.21.15.119:0/1797500717 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7ff2c00187e0 con 0x7ff2d41040d0 2023-12-07T10:24:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.232+0000 7ff2d1ffb700 1 --2- 172.21.15.119:0/1797500717 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff2c40a1c80 0x7ff2c40a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.232+0000 7ff2ee770b80 1 Processor -- start 2023-12-07T10:24:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.232+0000 7ff2d1ffb700 1 -- 172.21.15.119:0/1797500717 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7ff2c0014070 con 0x7ff2d41040d0 2023-12-07T10:24:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.233+0000 7ff2ee770b80 1 -- start start 2023-12-07T10:24:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.233+0000 7ff2d8b9b700 1 --2- 172.21.15.119:0/1797500717 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff2c40a1c80 0x7ff2c40a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.233+0000 7ff2ee770b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560151162a60 0x560151162e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.233+0000 7ff2ee770b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560151163380 0x56015115d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.233+0000 7ff2ee770b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56015115dad0 0x56015115df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.233+0000 7ff2ee770b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601510d1760 con 0x560151162a60 2023-12-07T10:24:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.233+0000 7ff2ee770b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601510b8740 con 0x56015115dad0 2023-12-07T10:24:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.233+0000 7ff2ee770b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601510b6060 con 0x560151163380 2023-12-07T10:24:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.233+0000 7ff2d3fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560151163380 0x56015115d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.233+0000 7ff2d8b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560151162a60 0x560151162e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.233+0000 7ff2d3fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560151163380 0x56015115d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:46852/0 (socket says 172.21.15.119:46852) 2023-12-07T10:24:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.233+0000 7ff2d3fff700 1 -- 172.21.15.119:0/1548952381 learned_addr learned my addr 172.21.15.119:0/1548952381 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.233+0000 7ff2d939c700 1 --2- 172.21.15.119:0/1548952381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56015115dad0 0x56015115df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.233+0000 7ff2d8b9b700 1 --2- 172.21.15.119:0/1797500717 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff2c40a1c80 0x7ff2c40a4140 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8006010 tx=0x7ff2c8005060 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.233+0000 7ff2d3fff700 1 -- 172.21.15.119:0/1548952381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56015115dad0 msgr2=0x56015115df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.233+0000 7ff2d3fff700 1 --2- 172.21.15.119:0/1548952381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56015115dad0 0x56015115df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.233+0000 7ff2d3fff700 1 -- 172.21.15.119:0/1548952381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560151162a60 msgr2=0x560151162e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.234+0000 7ff2d3fff700 1 --2- 172.21.15.119:0/1548952381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560151162a60 0x560151162e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.234+0000 7ff2d3fff700 1 -- 172.21.15.119:0/1548952381 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560150f2e5d0 con 0x560151163380 2023-12-07T10:24:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.234+0000 7ff2d8b9b700 1 --2- 172.21.15.119:0/1548952381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560151162a60 0x560151162e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:24:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.234+0000 7ff2d3fff700 1 --2- 172.21.15.119:0/1548952381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560151163380 0x56015115d540 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0097370 tx=0x7ff2c009a570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff7bb2be263cfec4 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.234+0000 7ff2beffd700 1 -- 172.21.15.119:0/1548952381 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c009a900 con 0x560151163380 2023-12-07T10:24:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.234+0000 7ff2beffd700 1 -- 172.21.15.119:0/1548952381 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2c009ea50 con 0x560151163380 2023-12-07T10:24:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.234+0000 7ff2beffd700 1 -- 172.21.15.119:0/1548952381 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c00ba070 con 0x560151163380 2023-12-07T10:24:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.235+0000 7ff2ee770b80 1 -- 172.21.15.119:0/1548952381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560151163380 msgr2=0x56015115d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.235+0000 7ff2ee770b80 1 --2- 172.21.15.119:0/1548952381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560151163380 0x56015115d540 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0097370 tx=0x7ff2c009a570 comp rx=0 tx=0).stop 2023-12-07T10:24:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.235+0000 7ff2ee770b80 1 -- 172.21.15.119:0/1548952381 shutdown_connections 2023-12-07T10:24:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.235+0000 7ff2ee770b80 1 --2- 172.21.15.119:0/1548952381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56015115dad0 0x56015115df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.235+0000 7ff2ee770b80 1 --2- 172.21.15.119:0/1548952381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560151162a60 0x560151162e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.235+0000 7ff2ee770b80 1 --2- 172.21.15.119:0/1548952381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560151163380 0x56015115d540 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.235+0000 7ff2ee770b80 1 -- 172.21.15.119:0/1548952381 >> 172.21.15.119:0/1548952381 conn(0x56015108c300 msgr2=0x560151157b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.235+0000 7ff2ee770b80 1 -- 172.21.15.119:0/1548952381 shutdown_connections 2023-12-07T10:24:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.235+0000 7ff2ee770b80 1 -- 172.21.15.119:0/1548952381 wait complete. 2023-12-07T10:24:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.236+0000 7ff2ee770b80 1 -- start start 2023-12-07T10:24:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.236+0000 7ff2ee770b80 1 Processor -- start 2023-12-07T10:24:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.236+0000 7ff2ee770b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56015115dad0 0x560151166830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.236+0000 7ff2ee770b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560151162a60 0x560151166d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.236+0000 7ff2ee770b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560151163380 0x5601511672b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.236+0000 7ff2ee770b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601510b1950 con 0x560151163380 2023-12-07T10:24:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.236+0000 7ff2ee770b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56015106b500 con 0x560151162a60 2023-12-07T10:24:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.236+0000 7ff2ee770b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56015106b1a0 con 0x56015115dad0 2023-12-07T10:24:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.236+0000 7ff2ee770b80 10 client.0 _ll_register_callbacks cb 0x5601510528e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.236+0000 7ff2d3fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560151162a60 0x560151166d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.237+0000 7ff2d8b9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56015115dad0 0x560151166830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.237+0000 7ff2d3fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560151162a60 0x560151166d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:34910/0 (socket says 172.21.15.119:34910) 2023-12-07T10:24:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.237+0000 7ff2d3fff700 1 -- 172.21.15.119:0/2531885659 learned_addr learned my addr 172.21.15.119:0/2531885659 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.237+0000 7ff2d939c700 1 --2- 172.21.15.119:0/2531885659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560151163380 0x5601511672b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.237+0000 7ff2d3fff700 1 -- 172.21.15.119:0/2531885659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56015115dad0 msgr2=0x560151166830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.237+0000 7ff2d3fff700 1 --2- 172.21.15.119:0/2531885659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56015115dad0 0x560151166830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.237+0000 7ff2d3fff700 1 -- 172.21.15.119:0/2531885659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560151163380 msgr2=0x5601511672b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.237+0000 7ff2d3fff700 1 --2- 172.21.15.119:0/2531885659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560151163380 0x5601511672b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.237+0000 7ff2d3fff700 1 -- 172.21.15.119:0/2531885659 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d41a68a0 con 0x560151162a60 2023-12-07T10:24:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.237+0000 7ff2d3fff700 1 --2- 172.21.15.119:0/2531885659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560151162a60 0x560151166d70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0009230 tx=0x7ff2c0098eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44c885d1fe7271db server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.237+0000 7ff2b3fff700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:24:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.237+0000 7ff2b3fff700 1 -- 172.21.15.119:0/2531885659 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c009a900 con 0x560151162a60 2023-12-07T10:24:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.237+0000 7ff2ee770b80 1 -- 172.21.15.119:0/2531885659 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601511506f0 con 0x560151162a60 2023-12-07T10:24:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.237+0000 7ff2b3fff700 1 -- 172.21.15.119:0/2531885659 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7ff2c00b3120 con 0x560151162a60 2023-12-07T10:24:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.237+0000 7ff2b3fff700 1 -- 172.21.15.119:0/2531885659 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2c0010880 con 0x560151162a60 2023-12-07T10:24:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.237+0000 7ff2ee770b80 10 client.27026 fetch_fsmap learned FSMap version 38 2023-12-07T10:24:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.237+0000 7ff2ee770b80 1 -- 172.21.15.119:0/2531885659 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x560150edad80 con 0x560151162a60 2023-12-07T10:24:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.238+0000 7ff2b3fff700 1 -- 172.21.15.119:0/2531885659 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5601511506f0 con 0x560151162a60 2023-12-07T10:24:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.238+0000 7ff2b3fff700 1 -- 172.21.15.119:0/2531885659 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 38) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ff2c0099e60 con 0x560151162a60 2023-12-07T10:24:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.238+0000 7ff2ee770b80 10 client.27026 fetch_fsmap finished waiting for FSMap version 38 2023-12-07T10:24:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.238+0000 7ff2ee770b80 10 client.27026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:24:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.238+0000 7ff2ee770b80 20 client.27026 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.238+0000 7ff2ee770b80 1 --2- 172.21.15.119:0/2531885659 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560151176a80 0x560151178ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.238+0000 7ff2ee770b80 4 client.27026 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:24:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.238+0000 7ff2ee770b80 1 -- 172.21.15.119:0/2531885659 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56015112c740 con 0x560151176a80 2023-12-07T10:24:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.238+0000 7ff2d8b9b700 1 --2- 172.21.15.119:0/2531885659 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560151176a80 0x560151178ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.239+0000 7ff2d8b9b700 1 --2- 172.21.15.119:0/2531885659 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560151176a80 0x560151178ea0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=939f89774f784603 server_cookie=a7d3a8df529ba6cc in_seq=0 out_seq=0 2023-12-07T10:24:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.239+0000 7ff2b3fff700 10 client.27026 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.239+0000 7ff2b3fff700 1 -- 172.21.15.119:0/2531885659 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56015112c740 con 0x560151176a80 2023-12-07T10:24:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.239+0000 7ff2b3fff700 10 client.27026 handle_command_reply: tid=0 2023-12-07T10:24:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.239+0000 7ff2ee770b80 1 client.27026 shutdown 2023-12-07T10:24:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.240+0000 7ff2ee770b80 1 -- 172.21.15.119:0/2531885659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560151162a60 msgr2=0x560151166d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.240+0000 7ff2ee770b80 1 --2- 172.21.15.119:0/2531885659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560151162a60 0x560151166d70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0009230 tx=0x7ff2c0098eb0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.240+0000 7ff2ee770b80 1 -- 172.21.15.119:0/2531885659 shutdown_connections 2023-12-07T10:24:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.240+0000 7ff2ee770b80 1 --2- 172.21.15.119:0/2531885659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560151163380 0x5601511672b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.240+0000 7ff2ee770b80 1 --2- 172.21.15.119:0/2531885659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56015115dad0 0x560151166830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.240+0000 7ff2ee770b80 1 --2- 172.21.15.119:0/2531885659 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560151176a80 0x560151178ea0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.240+0000 7ff2ee770b80 1 --2- 172.21.15.119:0/2531885659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560151162a60 0x560151166d70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.240+0000 7ff2b3fff700 0 client.27026 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.240+0000 7ff2ee770b80 1 -- 172.21.15.119:0/2531885659 >> 172.21.15.119:0/2531885659 conn(0x56015108c300 msgr2=0x560151152980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.240+0000 7ff2ee770b80 1 -- 172.21.15.119:0/2531885659 shutdown_connections 2023-12-07T10:24:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.240+0000 7ff2ee770b80 1 -- 172.21.15.119:0/2531885659 wait complete. 2023-12-07T10:24:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.240+0000 7ff2ee770b80 20 client.27026 trim_cache size 0 max 16384 2023-12-07T10:24:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.260+0000 7ff2ee770b80 1 Processor -- start 2023-12-07T10:24:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.260+0000 7ff2ee770b80 1 -- start start 2023-12-07T10:24:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.261+0000 7ff2ee770b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56015115dad0 0x56015115d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.261+0000 7ff2ee770b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560151162a60 0x56015115f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.261+0000 7ff2ee770b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560151163380 0x56015117d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.261+0000 7ff2ee770b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560150f308c0 con 0x560151162a60 2023-12-07T10:24:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.261+0000 7ff2ee770b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601511547a0 con 0x56015115dad0 2023-12-07T10:24:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.261+0000 7ff2ee770b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601511691f0 con 0x560151163380 2023-12-07T10:24:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.261+0000 7ff2d8b9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56015115dad0 0x56015115d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.261+0000 7ff2d8b9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56015115dad0 0x56015115d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:34920/0 (socket says 172.21.15.119:34920) 2023-12-07T10:24:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.261+0000 7ff2d8b9b700 1 -- 172.21.15.119:0/3904169269 learned_addr learned my addr 172.21.15.119:0/3904169269 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.261+0000 7ff2d3fff700 1 --2- 172.21.15.119:0/3904169269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560151162a60 0x56015115f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.261+0000 7ff2d939c700 1 --2- 172.21.15.119:0/3904169269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560151163380 0x56015117d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.261+0000 7ff2d8b9b700 1 -- 172.21.15.119:0/3904169269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560151163380 msgr2=0x56015117d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.261+0000 7ff2d8b9b700 1 --2- 172.21.15.119:0/3904169269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560151163380 0x56015117d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.261+0000 7ff2d8b9b700 1 -- 172.21.15.119:0/3904169269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560151162a60 msgr2=0x56015115f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.262+0000 7ff2d8b9b700 1 --2- 172.21.15.119:0/3904169269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560151162a60 0x56015115f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.262+0000 7ff2d8b9b700 1 -- 172.21.15.119:0/3904169269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560150ec3c00 con 0x56015115dad0 2023-12-07T10:24:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.262+0000 7ff2d939c700 1 --2- 172.21.15.119:0/3904169269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560151163380 0x56015117d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:24:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.262+0000 7ff2d8b9b700 1 --2- 172.21.15.119:0/3904169269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56015115dad0 0x56015115d540 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8009990 tx=0x7ff2c8022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ceaf3f456c26a8e server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:26.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:26 smithi159 ceph-mon[134726]: pgmap v123: 129 pgs: 129 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 407 KiB/s wr, 83 op/s 2023-12-07T10:24:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.262+0000 7ff2bf7fe700 1 -- 172.21.15.119:0/3904169269 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8035220 con 0x56015115dad0 2023-12-07T10:24:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.262+0000 7ff2bf7fe700 1 -- 172.21.15.119:0/3904169269 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2c801e040 con 0x56015115dad0 2023-12-07T10:24:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.262+0000 7ff2bf7fe700 1 -- 172.21.15.119:0/3904169269 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8004cd0 con 0x56015115dad0 2023-12-07T10:24:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.263+0000 7ff2ee770b80 1 -- 172.21.15.119:0/3904169269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56015115dad0 msgr2=0x56015115d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.263+0000 7ff2ee770b80 1 --2- 172.21.15.119:0/3904169269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56015115dad0 0x56015115d540 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8009990 tx=0x7ff2c8022000 comp rx=0 tx=0).stop 2023-12-07T10:24:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.263+0000 7ff2ee770b80 1 -- 172.21.15.119:0/3904169269 shutdown_connections 2023-12-07T10:24:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.263+0000 7ff2ee770b80 1 --2- 172.21.15.119:0/3904169269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560151163380 0x56015117d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.263+0000 7ff2ee770b80 1 --2- 172.21.15.119:0/3904169269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56015115dad0 0x56015115d540 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.263+0000 7ff2ee770b80 1 --2- 172.21.15.119:0/3904169269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560151162a60 0x56015115f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.263+0000 7ff2ee770b80 1 -- 172.21.15.119:0/3904169269 >> 172.21.15.119:0/3904169269 conn(0x56015108c300 msgr2=0x56015115ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.263+0000 7ff2ee770b80 1 -- 172.21.15.119:0/3904169269 shutdown_connections 2023-12-07T10:24:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.263+0000 7ff2ee770b80 1 -- 172.21.15.119:0/3904169269 wait complete. 2023-12-07T10:24:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.264+0000 7ff2ee770b80 1 -- start start 2023-12-07T10:24:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.264+0000 7ff2ee770b80 1 Processor -- start 2023-12-07T10:24:26.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.265+0000 7ff2ee770b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56015115dad0 0x560151169f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:26.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.265+0000 7ff2ee770b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560151162a60 0x56015116a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:26.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.265+0000 7ff2ee770b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560151163380 0x5601511cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:26.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.265+0000 7ff2ee770b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56015116ab80 con 0x560151163380 2023-12-07T10:24:26.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.265+0000 7ff2ee770b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601511cc4b0 con 0x56015115dad0 2023-12-07T10:24:26.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.265+0000 7ff2ee770b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601511cc5f0 con 0x560151162a60 2023-12-07T10:24:26.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.265+0000 7ff2ee770b80 10 client.0 _ll_register_callbacks cb 0x5601510528e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:26.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.265+0000 7ff2d8b9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56015115dad0 0x560151169f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:26.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.265+0000 7ff2d939c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560151163380 0x5601511cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:26.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.265+0000 7ff2d8b9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56015115dad0 0x560151169f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:34934/0 (socket says 172.21.15.119:34934) 2023-12-07T10:24:26.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.265+0000 7ff2d8b9b700 1 -- 172.21.15.119:0/1116989827 learned_addr learned my addr 172.21.15.119:0/1116989827 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:26.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.265+0000 7ff2d3fff700 1 --2- 172.21.15.119:0/1116989827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560151162a60 0x56015116a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:26.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.265+0000 7ff2d8b9b700 1 -- 172.21.15.119:0/1116989827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560151162a60 msgr2=0x56015116a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.265+0000 7ff2d8b9b700 1 --2- 172.21.15.119:0/1116989827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560151162a60 0x56015116a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.265+0000 7ff2d8b9b700 1 -- 172.21.15.119:0/1116989827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560151163380 msgr2=0x5601511cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.265+0000 7ff2d8b9b700 1 --2- 172.21.15.119:0/1116989827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560151163380 0x5601511cbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.265+0000 7ff2d8b9b700 1 -- 172.21.15.119:0/1116989827 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c802b040 con 0x56015115dad0 2023-12-07T10:24:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.265+0000 7ff2d939c700 1 --2- 172.21.15.119:0/1116989827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560151163380 0x5601511cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.266+0000 7ff2b3fff700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:24:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.266+0000 7ff2d8b9b700 1 --2- 172.21.15.119:0/1116989827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56015115dad0 0x560151169f60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800b6f0 tx=0x7ff2c8004a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e722011f3684526 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.266+0000 7ff2b3fff700 1 -- 172.21.15.119:0/1116989827 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8004500 con 0x56015115dad0 2023-12-07T10:24:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.267+0000 7ff2ee770b80 1 -- 172.21.15.119:0/1116989827 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2c009ea50 con 0x56015115dad0 2023-12-07T10:24:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.267+0000 7ff2b3fff700 1 -- 172.21.15.119:0/1116989827 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7ff2c8004d20 con 0x56015115dad0 2023-12-07T10:24:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.267+0000 7ff2b3fff700 1 -- 172.21.15.119:0/1116989827 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2c8004ee0 con 0x56015115dad0 2023-12-07T10:24:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.267+0000 7ff2ee770b80 10 client.27032 fetch_fsmap learned FSMap version 38 2023-12-07T10:24:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.267+0000 7ff2ee770b80 1 -- 172.21.15.119:0/1116989827 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x560150f011f0 con 0x56015115dad0 2023-12-07T10:24:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.267+0000 7ff2b3fff700 1 -- 172.21.15.119:0/1116989827 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2c009ea50 con 0x56015115dad0 2023-12-07T10:24:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.268+0000 7ff2b3fff700 1 -- 172.21.15.119:0/1116989827 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 38) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ff2c800f7a0 con 0x56015115dad0 2023-12-07T10:24:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.268+0000 7ff2ee770b80 10 client.27032 fetch_fsmap finished waiting for FSMap version 38 2023-12-07T10:24:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.268+0000 7ff2ee770b80 10 client.27032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:24:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.268+0000 7ff2ee770b80 20 client.27032 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.268+0000 7ff2ee770b80 1 --2- 172.21.15.119:0/1116989827 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5601511d61b0 0x5601511d85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.268+0000 7ff2ee770b80 4 client.27032 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:24:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.268+0000 7ff2ee770b80 1 -- 172.21.15.119:0/1116989827 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560151077f70 con 0x5601511d61b0 2023-12-07T10:24:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.269+0000 7ff2d3fff700 1 --2- 172.21.15.119:0/1116989827 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5601511d61b0 0x5601511d85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.269+0000 7ff2d3fff700 1 --2- 172.21.15.119:0/1116989827 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5601511d61b0 0x5601511d85d0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0025a7434fad6f9 server_cookie=90f73e4dec5c4cca in_seq=0 out_seq=0 2023-12-07T10:24:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.269+0000 7ff2b3fff700 10 client.27032 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:26.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:26 smithi153 ceph-mon[133625]: pgmap v123: 129 pgs: 129 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 407 KiB/s wr, 83 op/s 2023-12-07T10:24:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.307+0000 7ff2b3fff700 1 -- 172.21.15.119:0/1116989827 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560151077f70 con 0x5601511d61b0 2023-12-07T10:24:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.307+0000 7ff2b3fff700 10 client.27032 handle_command_reply: tid=0 2023-12-07T10:24:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.307+0000 7ff2ee770b80 1 client.27032 shutdown 2023-12-07T10:24:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.307+0000 7ff2ee770b80 1 -- 172.21.15.119:0/1116989827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56015115dad0 msgr2=0x560151169f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.307+0000 7ff2ee770b80 1 --2- 172.21.15.119:0/1116989827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56015115dad0 0x560151169f60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800b6f0 tx=0x7ff2c8004a70 comp rx=0 tx=0).stop 2023-12-07T10:24:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.308+0000 7ff2ee770b80 1 -- 172.21.15.119:0/1116989827 shutdown_connections 2023-12-07T10:24:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.308+0000 7ff2ee770b80 1 --2- 172.21.15.119:0/1116989827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560151163380 0x5601511cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.308+0000 7ff2ee770b80 1 --2- 172.21.15.119:0/1116989827 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5601511d61b0 0x5601511d85d0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.308+0000 7ff2ee770b80 1 --2- 172.21.15.119:0/1116989827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56015115dad0 0x560151169f60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.308+0000 7ff2b3fff700 0 client.27032 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.308+0000 7ff2ee770b80 1 --2- 172.21.15.119:0/1116989827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560151162a60 0x56015116a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.308+0000 7ff2ee770b80 1 -- 172.21.15.119:0/1116989827 >> 172.21.15.119:0/1116989827 conn(0x56015108c300 msgr2=0x560151160a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.308+0000 7ff2ee770b80 1 -- 172.21.15.119:0/1116989827 shutdown_connections 2023-12-07T10:24:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.308+0000 7ff2ee770b80 1 -- 172.21.15.119:0/1116989827 wait complete. 2023-12-07T10:24:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.308+0000 7ff2ee770b80 20 client.27032 trim_cache size 0 max 16384 2023-12-07T10:24:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:24:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:24:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:24:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:24:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.309+0000 7ff2dadff700 1 -- 172.21.15.119:0/1797500717 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff2c40a1c80 msgr2=0x7ff2c40a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.309+0000 7ff2dadff700 1 --2- 172.21.15.119:0/1797500717 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff2c40a1c80 0x7ff2c40a4140 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8006010 tx=0x7ff2c8005060 comp rx=0 tx=0).stop 2023-12-07T10:24:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.309+0000 7ff2dadff700 1 -- 172.21.15.119:0/1797500717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2d41040d0 msgr2=0x7ff2d419fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.309+0000 7ff2dadff700 1 --2- 172.21.15.119:0/1797500717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2d41040d0 0x7ff2d419fdf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff2c000e9a0 tx=0x7ff2c000ecb0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.309+0000 7ff2dadff700 1 -- 172.21.15.119:0/1797500717 shutdown_connections 2023-12-07T10:24:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.309+0000 7ff2dadff700 1 --2- 172.21.15.119:0/1797500717 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff2c40a1c80 0x7ff2c40a4140 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.309+0000 7ff2dadff700 1 --2- 172.21.15.119:0/1797500717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2d4109720 0x7ff2d41999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.309+0000 7ff2dadff700 1 --2- 172.21.15.119:0/1797500717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2d4103710 0x7ff2d419f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.309+0000 7ff2dadff700 1 --2- 172.21.15.119:0/1797500717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2d41040d0 0x7ff2d419fdf0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.309+0000 7ff2dadff700 1 -- 172.21.15.119:0/1797500717 >> 172.21.15.119:0/1797500717 conn(0x7ff2d40ff1c0 msgr2=0x7ff2d40ffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.309+0000 7ff2dadff700 1 -- 172.21.15.119:0/1797500717 shutdown_connections 2023-12-07T10:24:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:26.309+0000 7ff2dadff700 1 -- 172.21.15.119:0/1797500717 wait complete. 2023-12-07T10:24:26.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:26 smithi119 ceph-mon[97908]: pgmap v123: 129 pgs: 129 active+clean; 125 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 407 KiB/s wr, 83 op/s 2023-12-07T10:24:26.552 INFO:tasks.workunit.client.0.smithi119.stdout: 1 231396 5.94 MB/sec execute 39 sec latency 16.408 ms 2023-12-07T10:24:27.552 INFO:tasks.workunit.client.0.smithi119.stdout: 1 233046 5.96 MB/sec execute 40 sec latency 85.474 ms 2023-12-07T10:24:28.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:28 smithi159 ceph-mon[134726]: pgmap v124: 129 pgs: 129 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 994 KiB/s wr, 172 op/s 2023-12-07T10:24:28.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:28 smithi153 ceph-mon[133625]: pgmap v124: 129 pgs: 129 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 994 KiB/s wr, 172 op/s 2023-12-07T10:24:28.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:28 smithi119 ceph-mon[97908]: pgmap v124: 129 pgs: 129 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 994 KiB/s wr, 172 op/s 2023-12-07T10:24:28.552 INFO:tasks.workunit.client.0.smithi119.stdout: 1 234411 6.10 MB/sec execute 41 sec latency 31.151 ms 2023-12-07T10:24:29.552 INFO:tasks.workunit.client.0.smithi119.stdout: 1 235842 6.08 MB/sec execute 42 sec latency 11.376 ms 2023-12-07T10:24:30.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:30 smithi153 ceph-mon[133625]: pgmap v125: 129 pgs: 129 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 627 KiB/s wr, 102 op/s 2023-12-07T10:24:30.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:30 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:24:30.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:30 smithi119 ceph-mon[97908]: pgmap v125: 129 pgs: 129 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 627 KiB/s wr, 102 op/s 2023-12-07T10:24:30.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:30 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:24:30.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:30 smithi159 ceph-mon[134726]: pgmap v125: 129 pgs: 129 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 627 KiB/s wr, 102 op/s 2023-12-07T10:24:30.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:30 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:24:30.552 INFO:tasks.workunit.client.0.smithi119.stdout: 1 237989 6.26 MB/sec execute 43 sec latency 11.593 ms 2023-12-07T10:24:31.552 INFO:tasks.workunit.client.0.smithi119.stdout: 1 239590 6.25 MB/sec execute 44 sec latency 11.078 ms 2023-12-07T10:24:32.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:31 smithi119 ceph-mon[97908]: pgmap v126: 129 pgs: 129 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 314 op/s 2023-12-07T10:24:32.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:31 smithi159 ceph-mon[134726]: pgmap v126: 129 pgs: 129 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 314 op/s 2023-12-07T10:24:32.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:31 smithi153 ceph-mon[133625]: pgmap v126: 129 pgs: 129 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 314 op/s 2023-12-07T10:24:32.552 INFO:tasks.workunit.client.0.smithi119.stdout: 1 241501 6.40 MB/sec execute 45 sec latency 65.262 ms 2023-12-07T10:24:33.552 INFO:tasks.workunit.client.0.smithi119.stdout: 1 242818 6.38 MB/sec execute 46 sec latency 15.263 ms 2023-12-07T10:24:34.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:33 smithi159 ceph-mon[134726]: pgmap v127: 129 pgs: 129 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 301 op/s 2023-12-07T10:24:34.284 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:33 smithi153 ceph-mon[133625]: pgmap v127: 129 pgs: 129 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 301 op/s 2023-12-07T10:24:34.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:33 smithi119 ceph-mon[97908]: pgmap v127: 129 pgs: 129 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 301 op/s 2023-12-07T10:24:34.553 INFO:tasks.workunit.client.0.smithi119.stdout: 1 245051 6.54 MB/sec execute 47 sec latency 11.085 ms 2023-12-07T10:24:35.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:35 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:24:35.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:35 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:24:35.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:35 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:24:35.553 INFO:tasks.workunit.client.0.smithi119.stdout: 1 246481 6.52 MB/sec execute 48 sec latency 17.133 ms 2023-12-07T10:24:36.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:36 smithi159 ceph-mon[134726]: pgmap v128: 129 pgs: 129 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 301 op/s 2023-12-07T10:24:36.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:36 smithi153 ceph-mon[133625]: pgmap v128: 129 pgs: 129 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 301 op/s 2023-12-07T10:24:36.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:36 smithi119 ceph-mon[97908]: pgmap v128: 129 pgs: 129 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 301 op/s 2023-12-07T10:24:36.553 INFO:tasks.workunit.client.0.smithi119.stdout: 1 248609 6.66 MB/sec execute 49 sec latency 11.650 ms 2023-12-07T10:24:37.553 INFO:tasks.workunit.client.0.smithi119.stdout: 1 250021 6.64 MB/sec execute 50 sec latency 13.092 ms 2023-12-07T10:24:38.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:38 smithi159 ceph-mon[134726]: pgmap v129: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 393 op/s 2023-12-07T10:24:38.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:38 smithi153 ceph-mon[133625]: pgmap v129: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 393 op/s 2023-12-07T10:24:38.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:38 smithi119 ceph-mon[97908]: pgmap v129: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 393 op/s 2023-12-07T10:24:38.553 INFO:tasks.workunit.client.0.smithi119.stdout: 1 252138 6.78 MB/sec execute 51 sec latency 11.260 ms 2023-12-07T10:24:39.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:39 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:24:39.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:39 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:24:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.477+0000 7f9ef5bff700 1 -- 172.21.15.119:0/2458310685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ef00742e0 msgr2=0x7f9ef010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.477+0000 7f9ef5bff700 1 --2- 172.21.15.119:0/2458310685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ef00742e0 0x7f9ef010d880 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40098f0 tx=0x7f9ee4009c00 comp rx=0 tx=0).stop 2023-12-07T10:24:39.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:39 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:24:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.477+0000 7f9ef5bff700 1 -- 172.21.15.119:0/2458310685 shutdown_connections 2023-12-07T10:24:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.477+0000 7f9ef5bff700 1 --2- 172.21.15.119:0/2458310685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ef010ddc0 0x7f9ef01101b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.477+0000 7f9ef5bff700 1 --2- 172.21.15.119:0/2458310685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ef00739c0 0x7f9ef0073da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.477+0000 7f9ef5bff700 1 --2- 172.21.15.119:0/2458310685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ef00742e0 0x7f9ef010d880 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.477+0000 7f9ef5bff700 1 -- 172.21.15.119:0/2458310685 >> 172.21.15.119:0/2458310685 conn(0x7f9ef00fd010 msgr2=0x7f9ef00ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.477+0000 7f9ef5bff700 1 -- 172.21.15.119:0/2458310685 shutdown_connections 2023-12-07T10:24:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.478+0000 7f9ef5bff700 1 -- 172.21.15.119:0/2458310685 wait complete. 2023-12-07T10:24:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.479+0000 7f9ef5bff700 1 Processor -- start 2023-12-07T10:24:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.479+0000 7f9ef5bff700 1 -- start start 2023-12-07T10:24:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.479+0000 7f9ef5bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ef00739c0 0x7f9ef019f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.479+0000 7f9ef5bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ef00742e0 0x7f9ef019feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.479+0000 7f9ef5bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ef010ddc0 0x7f9ef0199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.479+0000 7f9ef5bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef019a1e0 con 0x7f9ef00742e0 2023-12-07T10:24:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.479+0000 7f9ef5bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef019a350 con 0x7f9ef010ddc0 2023-12-07T10:24:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.479+0000 7f9ef5bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef019a510 con 0x7f9ef00739c0 2023-12-07T10:24:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.480+0000 7f9eef7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ef00739c0 0x7f9ef019f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.480+0000 7f9eef7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ef00739c0 0x7f9ef019f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55926/0 (socket says 172.21.15.119:55926) 2023-12-07T10:24:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.480+0000 7f9eef7fe700 1 -- 172.21.15.119:0/13156036 learned_addr learned my addr 172.21.15.119:0/13156036 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.480+0000 7f9eeeffd700 1 --2- 172.21.15.119:0/13156036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ef00742e0 0x7f9ef019feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.480+0000 7f9eeffff700 1 --2- 172.21.15.119:0/13156036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ef010ddc0 0x7f9ef0199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.480+0000 7f9eef7fe700 1 -- 172.21.15.119:0/13156036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ef010ddc0 msgr2=0x7f9ef0199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.480+0000 7f9eef7fe700 1 --2- 172.21.15.119:0/13156036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ef010ddc0 0x7f9ef0199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.480+0000 7f9eef7fe700 1 -- 172.21.15.119:0/13156036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ef00742e0 msgr2=0x7f9ef019feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.480+0000 7f9eef7fe700 1 --2- 172.21.15.119:0/13156036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ef00742e0 0x7f9ef019feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.480+0000 7f9eef7fe700 1 -- 172.21.15.119:0/13156036 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee4009520 con 0x7f9ef00739c0 2023-12-07T10:24:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.480+0000 7f9eeffff700 1 --2- 172.21.15.119:0/13156036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ef010ddc0 0x7f9ef0199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:24:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.480+0000 7f9eeeffd700 1 --2- 172.21.15.119:0/13156036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ef00742e0 0x7f9ef019feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:24:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.481+0000 7f9eef7fe700 1 --2- 172.21.15.119:0/13156036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ef00739c0 0x7f9ef019f970 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000e9c0 tx=0x7f9ee000ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.481+0000 7f9eecff9700 1 -- 172.21.15.119:0/13156036 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee000cbd0 con 0x7f9ef00739c0 2023-12-07T10:24:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.481+0000 7f9eecff9700 1 -- 172.21.15.119:0/13156036 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ee000cd30 con 0x7f9ef00739c0 2023-12-07T10:24:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.481+0000 7f9eecff9700 1 -- 172.21.15.119:0/13156036 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0005580 con 0x7f9ef00739c0 2023-12-07T10:24:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.481+0000 7f9ef5bff700 1 -- 172.21.15.119:0/13156036 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef019a7f0 con 0x7f9ef00739c0 2023-12-07T10:24:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.481+0000 7f9ef5bff700 1 -- 172.21.15.119:0/13156036 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef01a68f0 con 0x7f9ef00739c0 2023-12-07T10:24:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.485+0000 7f9eecff9700 1 -- 172.21.15.119:0/13156036 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f9ee0004750 con 0x7f9ef00739c0 2023-12-07T10:24:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.486+0000 7f9f095a4b80 1 Processor -- start 2023-12-07T10:24:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.486+0000 7f9eecff9700 1 --2- 172.21.15.119:0/13156036 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9ed80a1d00 0x7f9ed80a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.487+0000 7f9eecff9700 1 -- 172.21.15.119:0/13156036 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9ee0014070 con 0x7f9ef00739c0 2023-12-07T10:24:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.487+0000 7f9eeffff700 1 --2- 172.21.15.119:0/13156036 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9ed80a1d00 0x7f9ed80a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.487+0000 7f9f095a4b80 1 -- start start 2023-12-07T10:24:39.488 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.488+0000 7f9f095a4b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559aecb0da60 0x559aecb0de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.488+0000 7f9f095a4b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559aecb0e380 0x559aecb08540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.488+0000 7f9f095a4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aecb08ad0 0x559aecb08f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.488+0000 7f9f095a4b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aeca7c760 con 0x559aecb0da60 2023-12-07T10:24:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.488+0000 7f9f095a4b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aeca63740 con 0x559aecb0e380 2023-12-07T10:24:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.488+0000 7f9f095a4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aeca61060 con 0x559aecb08ad0 2023-12-07T10:24:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.488+0000 7f9eef7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559aecb0da60 0x559aecb0de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.488+0000 7f9eef7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559aecb0da60 0x559aecb0de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52862/0 (socket says 172.21.15.119:52862) 2023-12-07T10:24:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.488+0000 7f9eef7fe700 1 -- 172.21.15.119:0/782302973 learned_addr learned my addr 172.21.15.119:0/782302973 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.488+0000 7f9eeeffd700 1 --2- 172.21.15.119:0/782302973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559aecb0e380 0x559aecb08540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.488+0000 7f9eeffff700 1 --2- 172.21.15.119:0/13156036 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9ed80a1d00 0x7f9ed80a41c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8009fd0 tx=0x7f9ee800b040 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.488+0000 7f9eef7fe700 1 -- 172.21.15.119:0/782302973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aecb08ad0 msgr2=0x559aecb08f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.488+0000 7f9eef7fe700 1 --2- 172.21.15.119:0/782302973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aecb08ad0 0x559aecb08f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.489+0000 7f9eef7fe700 1 -- 172.21.15.119:0/782302973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559aecb0e380 msgr2=0x559aecb08540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.489+0000 7f9eef7fe700 1 --2- 172.21.15.119:0/782302973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559aecb0e380 0x559aecb08540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.489+0000 7f9eef7fe700 1 -- 172.21.15.119:0/782302973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559aec8d95d0 con 0x559aecb0da60 2023-12-07T10:24:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.489+0000 7f9eef7fe700 1 --2- 172.21.15.119:0/782302973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559aecb0da60 0x559aecb0de40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00a27d0 tx=0x7f9ee009a350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a20f9998d6c6fc server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.489+0000 7f9ed5ffb700 1 -- 172.21.15.119:0/782302973 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee00b3210 con 0x559aecb0da60 2023-12-07T10:24:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.489+0000 7f9ed5ffb700 1 -- 172.21.15.119:0/782302973 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ee009a970 con 0x559aecb0da60 2023-12-07T10:24:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.489+0000 7f9ed5ffb700 1 -- 172.21.15.119:0/782302973 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0016330 con 0x559aecb0da60 2023-12-07T10:24:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.490+0000 7f9f095a4b80 1 -- 172.21.15.119:0/782302973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559aecb0da60 msgr2=0x559aecb0de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.490+0000 7f9f095a4b80 1 --2- 172.21.15.119:0/782302973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559aecb0da60 0x559aecb0de40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00a27d0 tx=0x7f9ee009a350 comp rx=0 tx=0).stop 2023-12-07T10:24:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.490+0000 7f9f095a4b80 1 -- 172.21.15.119:0/782302973 shutdown_connections 2023-12-07T10:24:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.490+0000 7f9f095a4b80 1 --2- 172.21.15.119:0/782302973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aecb08ad0 0x559aecb08f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.490+0000 7f9f095a4b80 1 --2- 172.21.15.119:0/782302973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559aecb0da60 0x559aecb0de40 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.490+0000 7f9f095a4b80 1 --2- 172.21.15.119:0/782302973 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559aecb0e380 0x559aecb08540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.490+0000 7f9f095a4b80 1 -- 172.21.15.119:0/782302973 >> 172.21.15.119:0/782302973 conn(0x559aeca37300 msgr2=0x559aecb02b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.490+0000 7f9f095a4b80 1 -- 172.21.15.119:0/782302973 shutdown_connections 2023-12-07T10:24:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.491+0000 7f9f095a4b80 1 -- 172.21.15.119:0/782302973 wait complete. 2023-12-07T10:24:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.491+0000 7f9f095a4b80 1 -- start start 2023-12-07T10:24:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.492+0000 7f9f095a4b80 1 Processor -- start 2023-12-07T10:24:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.492+0000 7f9f095a4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aecb08ad0 0x559aecb117a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.492+0000 7f9f095a4b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559aecb0da60 0x559aecb11ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.492+0000 7f9f095a4b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559aecb0e380 0x559aecb12220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.492+0000 7f9f095a4b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aeca5c950 con 0x559aecb0da60 2023-12-07T10:24:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.492+0000 7f9f095a4b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aeca16500 con 0x559aecb0e380 2023-12-07T10:24:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.492+0000 7f9f095a4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aeca161a0 con 0x559aecb08ad0 2023-12-07T10:24:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.493+0000 7f9f095a4b80 10 client.0 _ll_register_callbacks cb 0x559aec9fd8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.493+0000 7f9eeeffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559aecb0da60 0x559aecb11ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.493+0000 7f9eef7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aecb08ad0 0x559aecb117a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.493+0000 7f9eeeffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559aecb0da60 0x559aecb11ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52878/0 (socket says 172.21.15.119:52878) 2023-12-07T10:24:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.493+0000 7f9eeeffd700 1 -- 172.21.15.119:0/895209555 learned_addr learned my addr 172.21.15.119:0/895209555 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.493+0000 7f9eeffff700 1 --2- 172.21.15.119:0/895209555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559aecb0e380 0x559aecb12220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.493+0000 7f9eef7fe700 1 -- 172.21.15.119:0/895209555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559aecb0e380 msgr2=0x559aecb12220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.493+0000 7f9eef7fe700 1 --2- 172.21.15.119:0/895209555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559aecb0e380 0x559aecb12220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.493+0000 7f9eef7fe700 1 -- 172.21.15.119:0/895209555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559aecb0da60 msgr2=0x559aecb11ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.493+0000 7f9eef7fe700 1 --2- 172.21.15.119:0/895209555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559aecb0da60 0x559aecb11ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.493+0000 7f9eef7fe700 1 -- 172.21.15.119:0/895209555 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef01a68f0 con 0x559aecb08ad0 2023-12-07T10:24:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.493+0000 7f9eeffff700 1 --2- 172.21.15.119:0/895209555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559aecb0e380 0x559aecb12220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:24:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.493+0000 7f9eeeffd700 1 --2- 172.21.15.119:0/895209555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559aecb0da60 0x559aecb11ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.494+0000 7f9eef7fe700 1 --2- 172.21.15.119:0/895209555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aecb08ad0 0x559aecb117a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00cb3d0 tx=0x7f9ee00972b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac15b1da584af65e server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.494+0000 7f9eceffd700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:24:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.494+0000 7f9eceffd700 1 -- 172.21.15.119:0/895209555 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee00b3210 con 0x559aecb08ad0 2023-12-07T10:24:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.494+0000 7f9eceffd700 1 -- 172.21.15.119:0/895209555 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9ee00a2d00 con 0x559aecb08ad0 2023-12-07T10:24:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.494+0000 7f9f095a4b80 1 -- 172.21.15.119:0/895209555 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559aecafb6f0 con 0x559aecb08ad0 2023-12-07T10:24:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.494+0000 7f9eceffd700 1 -- 172.21.15.119:0/895209555 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ee00ba0c0 con 0x559aecb08ad0 2023-12-07T10:24:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.494+0000 7f9f095a4b80 10 client.27022 fetch_fsmap learned FSMap version 41 2023-12-07T10:24:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.494+0000 7f9f095a4b80 1 -- 172.21.15.119:0/895209555 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x559aec885d80 con 0x559aecb08ad0 2023-12-07T10:24:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.495+0000 7f9eceffd700 1 -- 172.21.15.119:0/895209555 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559aecafb6f0 con 0x559aecb08ad0 2023-12-07T10:24:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.495+0000 7f9eceffd700 1 -- 172.21.15.119:0/895209555 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 41) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f9ee0099f60 con 0x559aecb08ad0 2023-12-07T10:24:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.495+0000 7f9f095a4b80 10 client.27022 fetch_fsmap finished waiting for FSMap version 41 2023-12-07T10:24:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.495+0000 7f9f095a4b80 10 client.27022 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:24:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.495+0000 7f9f095a4b80 20 client.27022 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.495+0000 7f9f095a4b80 1 --2- 172.21.15.119:0/895209555 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x559aecb21960 0x559aecb23d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.495+0000 7f9f095a4b80 4 client.27022 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:24:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.495+0000 7f9f095a4b80 1 -- 172.21.15.119:0/895209555 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559aecad7740 con 0x559aecb21960 2023-12-07T10:24:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.495+0000 7f9eeeffd700 1 --2- 172.21.15.119:0/895209555 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x559aecb21960 0x559aecb23d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.496+0000 7f9eceffd700 10 client.27022 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.496+0000 7f9eeeffd700 1 --2- 172.21.15.119:0/895209555 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x559aecb21960 0x559aecb23d80 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd3f327ccb1e0291 server_cookie=93e89102c3719c2 in_seq=0 out_seq=0 2023-12-07T10:24:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.497+0000 7f9eceffd700 1 -- 172.21.15.119:0/895209555 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559aecad7740 con 0x559aecb21960 2023-12-07T10:24:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.497+0000 7f9eceffd700 10 client.27022 handle_command_reply: tid=0 2023-12-07T10:24:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.497+0000 7f9f095a4b80 1 client.27022 shutdown 2023-12-07T10:24:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.497+0000 7f9f095a4b80 1 -- 172.21.15.119:0/895209555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aecb08ad0 msgr2=0x559aecb117a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.497+0000 7f9f095a4b80 1 --2- 172.21.15.119:0/895209555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aecb08ad0 0x559aecb117a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00cb3d0 tx=0x7f9ee00972b0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.497+0000 7f9f095a4b80 1 -- 172.21.15.119:0/895209555 shutdown_connections 2023-12-07T10:24:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.497+0000 7f9f095a4b80 1 --2- 172.21.15.119:0/895209555 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x559aecb21960 0x559aecb23d80 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.497+0000 7f9f095a4b80 1 --2- 172.21.15.119:0/895209555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559aecb0e380 0x559aecb12220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.498+0000 7f9f095a4b80 1 --2- 172.21.15.119:0/895209555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aecb08ad0 0x559aecb117a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.498+0000 7f9f095a4b80 1 --2- 172.21.15.119:0/895209555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559aecb0da60 0x559aecb11ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.498+0000 7f9f095a4b80 1 -- 172.21.15.119:0/895209555 >> 172.21.15.119:0/895209555 conn(0x559aeca37300 msgr2=0x559aecafd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.498+0000 7f9eceffd700 0 client.27022 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.498+0000 7f9f095a4b80 1 -- 172.21.15.119:0/895209555 shutdown_connections 2023-12-07T10:24:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.498+0000 7f9f095a4b80 1 -- 172.21.15.119:0/895209555 wait complete. 2023-12-07T10:24:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.498+0000 7f9f095a4b80 20 client.27022 trim_cache size 0 max 16384 2023-12-07T10:24:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.516+0000 7f9f095a4b80 1 Processor -- start 2023-12-07T10:24:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.516+0000 7f9f095a4b80 1 -- start start 2023-12-07T10:24:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.517+0000 7f9f095a4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aecb0da60 0x559aecb23d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.517+0000 7f9f095a4b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559aecb0e380 0x559aecb08540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.517+0000 7f9f095a4b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559aecb12070 0x559aecb0c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.517+0000 7f9f095a4b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aec8db8c0 con 0x559aecb12070 2023-12-07T10:24:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.517+0000 7f9f095a4b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aecb12c60 con 0x559aecb0e380 2023-12-07T10:24:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.517+0000 7f9f095a4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aecafb6c0 con 0x559aecb0da60 2023-12-07T10:24:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.517+0000 7f9eeeffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559aecb0e380 0x559aecb08540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.517+0000 7f9eeeffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559aecb0e380 0x559aecb08540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:59258/0 (socket says 172.21.15.119:59258) 2023-12-07T10:24:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.517+0000 7f9eeeffd700 1 -- 172.21.15.119:0/2125064182 learned_addr learned my addr 172.21.15.119:0/2125064182 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.517+0000 7f9eeffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559aecb12070 0x559aecb0c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.517+0000 7f9eef7fe700 1 --2- 172.21.15.119:0/2125064182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aecb0da60 0x559aecb23d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.517+0000 7f9eeeffd700 1 -- 172.21.15.119:0/2125064182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aecb0da60 msgr2=0x559aecb23d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.517+0000 7f9eeeffd700 1 --2- 172.21.15.119:0/2125064182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aecb0da60 0x559aecb23d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.517+0000 7f9eeeffd700 1 -- 172.21.15.119:0/2125064182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559aecb12070 msgr2=0x559aecb0c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.517+0000 7f9eeeffd700 1 --2- 172.21.15.119:0/2125064182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559aecb12070 0x559aecb0c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.517+0000 7f9eeeffd700 1 -- 172.21.15.119:0/2125064182 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559aec86ec00 con 0x559aecb0e380 2023-12-07T10:24:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.518+0000 7f9eeeffd700 1 --2- 172.21.15.119:0/2125064182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559aecb0e380 0x559aecb08540 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4000c00 tx=0x7f9ee400bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1dfd5b615133ff7 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.518+0000 7f9ed67fc700 1 -- 172.21.15.119:0/2125064182 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee401c040 con 0x559aecb0e380 2023-12-07T10:24:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.518+0000 7f9ed67fc700 1 -- 172.21.15.119:0/2125064182 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ee401d040 con 0x559aecb0e380 2023-12-07T10:24:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.518+0000 7f9ed67fc700 1 -- 172.21.15.119:0/2125064182 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4005600 con 0x559aecb0e380 2023-12-07T10:24:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.519+0000 7f9f095a4b80 1 -- 172.21.15.119:0/2125064182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559aecb0e380 msgr2=0x559aecb08540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.519+0000 7f9f095a4b80 1 --2- 172.21.15.119:0/2125064182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559aecb0e380 0x559aecb08540 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4000c00 tx=0x7f9ee400bef0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.519+0000 7f9f095a4b80 1 -- 172.21.15.119:0/2125064182 shutdown_connections 2023-12-07T10:24:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.519+0000 7f9f095a4b80 1 --2- 172.21.15.119:0/2125064182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559aecb12070 0x559aecb0c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.519+0000 7f9f095a4b80 1 --2- 172.21.15.119:0/2125064182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aecb0da60 0x559aecb23d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.519+0000 7f9f095a4b80 1 --2- 172.21.15.119:0/2125064182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559aecb0e380 0x559aecb08540 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.519+0000 7f9f095a4b80 1 -- 172.21.15.119:0/2125064182 >> 172.21.15.119:0/2125064182 conn(0x559aeca37300 msgr2=0x559aecb0ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.519+0000 7f9f095a4b80 1 -- 172.21.15.119:0/2125064182 shutdown_connections 2023-12-07T10:24:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.519+0000 7f9f095a4b80 1 -- 172.21.15.119:0/2125064182 wait complete. 2023-12-07T10:24:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.520+0000 7f9f095a4b80 1 -- start start 2023-12-07T10:24:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.520+0000 7f9f095a4b80 1 Processor -- start 2023-12-07T10:24:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.521+0000 7f9f095a4b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559aecb0da60 0x559aecb17f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.521+0000 7f9f095a4b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aecb0e380 0x559aecb184c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.521+0000 7f9f095a4b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559aecb12070 0x559aecb18a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.521+0000 7f9f095a4b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aecb19720 con 0x559aecb0da60 2023-12-07T10:24:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.521+0000 7f9f095a4b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aecb76eb0 con 0x559aecb12070 2023-12-07T10:24:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.521+0000 7f9f095a4b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aecb77070 con 0x559aecb0e380 2023-12-07T10:24:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.521+0000 7f9f095a4b80 10 client.0 _ll_register_callbacks cb 0x559aec9fd8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.521+0000 7f9eef7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559aecb0da60 0x559aecb17f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.521+0000 7f9eeffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559aecb12070 0x559aecb18a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.521+0000 7f9eeeffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aecb0e380 0x559aecb184c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.521+0000 7f9eef7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559aecb0da60 0x559aecb17f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52904/0 (socket says 172.21.15.119:52904) 2023-12-07T10:24:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.521+0000 7f9eeeffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aecb0e380 0x559aecb184c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55956/0 (socket says 172.21.15.119:55956) 2023-12-07T10:24:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.521+0000 7f9eef7fe700 1 -- 172.21.15.119:0/3036593601 learned_addr learned my addr 172.21.15.119:0/3036593601 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.521+0000 7f9eeffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559aecb12070 0x559aecb18a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:59260/0 (socket says 172.21.15.119:59260) 2023-12-07T10:24:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.521+0000 7f9eeeffd700 1 -- 172.21.15.119:0/3036593601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559aecb12070 msgr2=0x559aecb18a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.521+0000 7f9eeeffd700 1 --2- 172.21.15.119:0/3036593601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559aecb12070 0x559aecb18a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.521+0000 7f9eeeffd700 1 -- 172.21.15.119:0/3036593601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559aecb0da60 msgr2=0x559aecb17f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.521+0000 7f9eeeffd700 1 --2- 172.21.15.119:0/3036593601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559aecb0da60 0x559aecb17f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.521+0000 7f9eeeffd700 1 -- 172.21.15.119:0/3036593601 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef0110e70 con 0x559aecb0e380 2023-12-07T10:24:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.522+0000 7f9eeeffd700 1 --2- 172.21.15.119:0/3036593601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aecb0e380 0x559aecb184c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x559aecb24290 tx=0x7f9ee400f820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3b590afdb89aeff server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.522+0000 7f9eceffd700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:24:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.522+0000 7f9eceffd700 1 -- 172.21.15.119:0/3036593601 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee401c070 con 0x559aecb0e380 2023-12-07T10:24:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.522+0000 7f9f095a4b80 1 -- 172.21.15.119:0/3036593601 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ee009a970 con 0x559aecb0e380 2023-12-07T10:24:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.522+0000 7f9eceffd700 1 -- 172.21.15.119:0/3036593601 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9ee4005690 con 0x559aecb0e380 2023-12-07T10:24:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.522+0000 7f9eceffd700 1 -- 172.21.15.119:0/3036593601 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ee400b560 con 0x559aecb0e380 2023-12-07T10:24:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.522+0000 7f9f095a4b80 10 client.27028 fetch_fsmap learned FSMap version 41 2023-12-07T10:24:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.522+0000 7f9f095a4b80 1 -- 172.21.15.119:0/3036593601 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x559aec8ac1f0 con 0x559aecb0e380 2023-12-07T10:24:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.522+0000 7f9eceffd700 1 -- 172.21.15.119:0/3036593601 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ee009a970 con 0x559aecb0e380 2023-12-07T10:24:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.523+0000 7f9eceffd700 1 -- 172.21.15.119:0/3036593601 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 41) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f9ee4015020 con 0x559aecb0e380 2023-12-07T10:24:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.523+0000 7f9f095a4b80 10 client.27028 fetch_fsmap finished waiting for FSMap version 41 2023-12-07T10:24:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.523+0000 7f9f095a4b80 10 client.27028 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:24:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.523+0000 7f9f095a4b80 20 client.27028 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.523+0000 7f9f095a4b80 1 --2- 172.21.15.119:0/3036593601 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x559aecb78540 0x559aecb7a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.523+0000 7f9f095a4b80 4 client.27028 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:24:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.523+0000 7f9f095a4b80 1 -- 172.21.15.119:0/3036593601 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559aeca22f70 con 0x559aecb78540 2023-12-07T10:24:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.523+0000 7f9eef7fe700 1 --2- 172.21.15.119:0/3036593601 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x559aecb78540 0x559aecb7a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.523+0000 7f9eef7fe700 1 --2- 172.21.15.119:0/3036593601 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x559aecb78540 0x559aecb7a960 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f6e6af4f6a7a1e5 server_cookie=88d1b5dedcb1c5b3 in_seq=0 out_seq=0 2023-12-07T10:24:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.523+0000 7f9eceffd700 10 client.27028 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.552+0000 7f9eceffd700 1 -- 172.21.15.119:0/3036593601 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559aeca22f70 con 0x559aecb78540 2023-12-07T10:24:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.552+0000 7f9eceffd700 10 client.27028 handle_command_reply: tid=0 2023-12-07T10:24:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.552+0000 7f9f095a4b80 1 client.27028 shutdown 2023-12-07T10:24:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.552+0000 7f9f095a4b80 1 -- 172.21.15.119:0/3036593601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aecb0e380 msgr2=0x559aecb184c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.552+0000 7f9f095a4b80 1 --2- 172.21.15.119:0/3036593601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aecb0e380 0x559aecb184c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x559aecb24290 tx=0x7f9ee400f820 comp rx=0 tx=0).stop 2023-12-07T10:24:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.552+0000 7f9f095a4b80 1 -- 172.21.15.119:0/3036593601 shutdown_connections 2023-12-07T10:24:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.552+0000 7f9f095a4b80 1 --2- 172.21.15.119:0/3036593601 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x559aecb78540 0x559aecb7a960 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.552+0000 7f9f095a4b80 1 --2- 172.21.15.119:0/3036593601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x559aecb12070 0x559aecb18a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.552+0000 7f9f095a4b80 1 --2- 172.21.15.119:0/3036593601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559aecb0da60 0x559aecb17f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.552+0000 7f9f095a4b80 1 --2- 172.21.15.119:0/3036593601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aecb0e380 0x559aecb184c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.552+0000 7f9f095a4b80 1 -- 172.21.15.119:0/3036593601 >> 172.21.15.119:0/3036593601 conn(0x559aeca37300 msgr2=0x559aecb0c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:39.554 INFO:tasks.workunit.client.0.smithi119.stdout: 1 253066 6.69 MB/sec execute 52 sec latency 24.735 ms 2023-12-07T10:24:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.552+0000 7f9eceffd700 0 client.27028 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.552+0000 7f9f095a4b80 1 -- 172.21.15.119:0/3036593601 shutdown_connections 2023-12-07T10:24:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.552+0000 7f9f095a4b80 1 -- 172.21.15.119:0/3036593601 wait complete. 2023-12-07T10:24:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.553+0000 7f9f095a4b80 20 client.27028 trim_cache size 0 max 16384 2023-12-07T10:24:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:24:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:24:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:24:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:24:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.553+0000 7f9ef5bff700 1 -- 172.21.15.119:0/13156036 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9ed80a1d00 msgr2=0x7f9ed80a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.553+0000 7f9ef5bff700 1 --2- 172.21.15.119:0/13156036 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9ed80a1d00 0x7f9ed80a41c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8009fd0 tx=0x7f9ee800b040 comp rx=0 tx=0).stop 2023-12-07T10:24:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.553+0000 7f9ef5bff700 1 -- 172.21.15.119:0/13156036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ef00739c0 msgr2=0x7f9ef019f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.553+0000 7f9ef5bff700 1 --2- 172.21.15.119:0/13156036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ef00739c0 0x7f9ef019f970 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000e9c0 tx=0x7f9ee000ed80 comp rx=0 tx=0).stop 2023-12-07T10:24:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.553+0000 7f9ef5bff700 1 -- 172.21.15.119:0/13156036 shutdown_connections 2023-12-07T10:24:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.553+0000 7f9ef5bff700 1 --2- 172.21.15.119:0/13156036 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9ed80a1d00 0x7f9ed80a41c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.553+0000 7f9ef5bff700 1 --2- 172.21.15.119:0/13156036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ef010ddc0 0x7f9ef0199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.553+0000 7f9ef5bff700 1 --2- 172.21.15.119:0/13156036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ef00739c0 0x7f9ef019f970 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.553+0000 7f9ef5bff700 1 --2- 172.21.15.119:0/13156036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ef00742e0 0x7f9ef019feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.553+0000 7f9ef5bff700 1 -- 172.21.15.119:0/13156036 >> 172.21.15.119:0/13156036 conn(0x7f9ef00fd010 msgr2=0x7f9ef00fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.554+0000 7f9ef5bff700 1 -- 172.21.15.119:0/13156036 shutdown_connections 2023-12-07T10:24:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:39.554+0000 7f9ef5bff700 1 -- 172.21.15.119:0/13156036 wait complete. 2023-12-07T10:24:40.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:40 smithi153 ceph-mon[133625]: pgmap v130: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 304 op/s 2023-12-07T10:24:40.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:40 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:24:40.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:40 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:24:40.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:40 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:24:40.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:40 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:24:40.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:40 smithi119 ceph-mon[97908]: pgmap v130: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 304 op/s 2023-12-07T10:24:40.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:40 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:24:40.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:40 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:24:40.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:40 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:24:40.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:40 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:24:40.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:40 smithi159 ceph-mon[134726]: pgmap v130: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 304 op/s 2023-12-07T10:24:40.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:40 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:24:40.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:40 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:24:40.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:40 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:24:40.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:40 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:24:40.553 INFO:tasks.workunit.client.0.smithi119.stdout: 1 255554 6.86 MB/sec execute 53 sec latency 11.025 ms 2023-12-07T10:24:41.553 INFO:tasks.workunit.client.0.smithi119.stdout: 1 256546 6.80 MB/sec execute 54 sec latency 16.724 ms 2023-12-07T10:24:42.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:41 smithi119 ceph-mon[97908]: pgmap v131: 129 pgs: 129 active+clean; 139 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 401 op/s 2023-12-07T10:24:42.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:41 smithi159 ceph-mon[134726]: pgmap v131: 129 pgs: 129 active+clean; 139 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 401 op/s 2023-12-07T10:24:42.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:41 smithi153 ceph-mon[133625]: pgmap v131: 129 pgs: 129 active+clean; 139 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 401 op/s 2023-12-07T10:24:42.553 INFO:tasks.workunit.client.0.smithi119.stdout: 1 259074 6.96 MB/sec execute 55 sec latency 10.835 ms 2023-12-07T10:24:42.815 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-07T10:24:43.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.057+0000 7f4f231ff700 1 -- 172.21.15.119:0/3739336496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f1c068a50 msgr2=0x7f4f1c068e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:43.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.057+0000 7f4f231ff700 1 --2- 172.21.15.119:0/3739336496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f1c068a50 0x7f4f1c068e30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4f100098f0 tx=0x7f4f10009c00 comp rx=0 tx=0).stop 2023-12-07T10:24:43.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.058+0000 7f4f231ff700 1 -- 172.21.15.119:0/3739336496 shutdown_connections 2023-12-07T10:24:43.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.058+0000 7f4f231ff700 1 --2- 172.21.15.119:0/3739336496 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c106ad0 0x7f4f1c10e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.058+0000 7f4f231ff700 1 --2- 172.21.15.119:0/3739336496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f1c068a50 0x7f4f1c068e30 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.058+0000 7f4f231ff700 1 --2- 172.21.15.119:0/3739336496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f1c106110 0x7f4f1c106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.058+0000 7f4f231ff700 1 -- 172.21.15.119:0/3739336496 >> 172.21.15.119:0/3739336496 conn(0x7f4f1c077610 msgr2=0x7f4f1c077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:43.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.058+0000 7f4f231ff700 1 -- 172.21.15.119:0/3739336496 shutdown_connections 2023-12-07T10:24:43.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.058+0000 7f4f231ff700 1 -- 172.21.15.119:0/3739336496 wait complete. 2023-12-07T10:24:43.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.059+0000 7f4f231ff700 1 Processor -- start 2023-12-07T10:24:43.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.060+0000 7f4f231ff700 1 -- start start 2023-12-07T10:24:43.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.060+0000 7f4f231ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f1c068a50 0x7f4f1c1083c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.060+0000 7f4f231ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c106110 0x7f4f1c108900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.060+0000 7f4f231ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f1c106ad0 0x7f4f1c108e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.060+0000 7f4f231ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c1aa280 con 0x7f4f1c106ad0 2023-12-07T10:24:43.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.060+0000 7f4f231ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c1aa3c0 con 0x7f4f1c106110 2023-12-07T10:24:43.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.060+0000 7f4f231ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c1aa530 con 0x7f4f1c068a50 2023-12-07T10:24:43.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.061+0000 7f4f1bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c106110 0x7f4f1c108900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.061+0000 7f4f1bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c106110 0x7f4f1c108900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45902/0 (socket says 172.21.15.119:45902) 2023-12-07T10:24:43.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.061+0000 7f4f1bfff700 1 -- 172.21.15.119:0/3267711502 learned_addr learned my addr 172.21.15.119:0/3267711502 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:43.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.061+0000 7f4f20f9b700 1 --2- 172.21.15.119:0/3267711502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f1c068a50 0x7f4f1c1083c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.061+0000 7f4f2179c700 1 --2- 172.21.15.119:0/3267711502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f1c106ad0 0x7f4f1c108e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.061+0000 7f4f20f9b700 1 -- 172.21.15.119:0/3267711502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c106110 msgr2=0x7f4f1c108900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:43.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.061+0000 7f4f20f9b700 1 --2- 172.21.15.119:0/3267711502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c106110 0x7f4f1c108900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.061+0000 7f4f20f9b700 1 -- 172.21.15.119:0/3267711502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f1c106ad0 msgr2=0x7f4f1c108e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:43.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.061+0000 7f4f20f9b700 1 --2- 172.21.15.119:0/3267711502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f1c106ad0 0x7f4f1c108e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.061+0000 7f4f20f9b700 1 -- 172.21.15.119:0/3267711502 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f10009520 con 0x7f4f1c068a50 2023-12-07T10:24:43.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.061+0000 7f4f1bfff700 1 --2- 172.21.15.119:0/3267711502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c106110 0x7f4f1c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:43.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.061+0000 7f4f20f9b700 1 --2- 172.21.15.119:0/3267711502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f1c068a50 0x7f4f1c1083c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4f10009fd0 tx=0x7f4f1000f9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:43.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.062+0000 7f4f19ffb700 1 -- 172.21.15.119:0/3267711502 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1001c070 con 0x7f4f1c068a50 2023-12-07T10:24:43.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.062+0000 7f4f231ff700 1 -- 172.21.15.119:0/3267711502 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f1c1aa7b0 con 0x7f4f1c068a50 2023-12-07T10:24:43.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.062+0000 7f4f19ffb700 1 -- 172.21.15.119:0/3267711502 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f1000baa0 con 0x7f4f1c068a50 2023-12-07T10:24:43.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.062+0000 7f4f19ffb700 1 -- 172.21.15.119:0/3267711502 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f100038e0 con 0x7f4f1c068a50 2023-12-07T10:24:43.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.062+0000 7f4f231ff700 1 -- 172.21.15.119:0/3267711502 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1c1aabe0 con 0x7f4f1c068a50 2023-12-07T10:24:43.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.066+0000 7f4f19ffb700 1 -- 172.21.15.119:0/3267711502 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f4f10020030 con 0x7f4f1c068a50 2023-12-07T10:24:43.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.068+0000 7f4f36a61b80 1 Processor -- start 2023-12-07T10:24:43.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.068+0000 7f4f36a61b80 1 -- start start 2023-12-07T10:24:43.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.068+0000 7f4f19ffb700 1 --2- 172.21.15.119:0/3267711502 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4f0c0a1c80 0x7f4f0c0a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.068+0000 7f4f36a61b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abd3ecb40 0x562abd3ecf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.068+0000 7f4f36a61b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562abd3ed460 0x562abd3f0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.068+0000 7f4f36a61b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abd3f1150 0x562abd3f1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.068+0000 7f4f36a61b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562abd3607c0 con 0x562abd3f1150 2023-12-07T10:24:43.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.068+0000 7f4f36a61b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562abd3477a0 con 0x562abd3ed460 2023-12-07T10:24:43.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.068+0000 7f4f36a61b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562abd3450c0 con 0x562abd3ecb40 2023-12-07T10:24:43.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.068+0000 7f4f19ffb700 1 -- 172.21.15.119:0/3267711502 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4f100cea70 con 0x7f4f1c068a50 2023-12-07T10:24:43.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.068+0000 7f4f20f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abd3ecb40 0x562abd3ecf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.068+0000 7f4f20f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abd3ecb40 0x562abd3ecf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55708/0 (socket says 172.21.15.119:55708) 2023-12-07T10:24:43.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.068+0000 7f4f20f9b700 1 -- 172.21.15.119:0/1577609489 learned_addr learned my addr 172.21.15.119:0/1577609489 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:43.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.069+0000 7f4f2179c700 1 --2- 172.21.15.119:0/3267711502 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4f0c0a1c80 0x7f4f0c0a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.069+0000 7f4f1bfff700 1 --2- 172.21.15.119:0/1577609489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562abd3ed460 0x562abd3f0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.069+0000 7f4f20f9b700 1 -- 172.21.15.119:0/1577609489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562abd3ed460 msgr2=0x562abd3f0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:43.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.069+0000 7f4f20f9b700 1 --2- 172.21.15.119:0/1577609489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562abd3ed460 0x562abd3f0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.069+0000 7f4f20f9b700 1 -- 172.21.15.119:0/1577609489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abd3f1150 msgr2=0x562abd3f1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:43.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.069+0000 7f4f20f9b700 1 --2- 172.21.15.119:0/1577609489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abd3f1150 0x562abd3f1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.069+0000 7f4f20f9b700 1 -- 172.21.15.119:0/1577609489 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562abd1bd630 con 0x562abd3ecb40 2023-12-07T10:24:43.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.069+0000 7f4f20f9b700 1 --2- 172.21.15.119:0/1577609489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abd3ecb40 0x562abd3ecf20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4f100a3320 tx=0x7f4f10097910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f15b25c768d428d server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:43.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.069+0000 7f4f06ffd700 1 -- 172.21.15.119:0/1577609489 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1009aeb0 con 0x562abd3ecb40 2023-12-07T10:24:43.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.069+0000 7f4f06ffd700 1 -- 172.21.15.119:0/1577609489 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f10073190 con 0x562abd3ecb40 2023-12-07T10:24:43.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.069+0000 7f4f06ffd700 1 -- 172.21.15.119:0/1577609489 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1009f6a0 con 0x562abd3ecb40 2023-12-07T10:24:43.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.070+0000 7f4f36a61b80 1 -- 172.21.15.119:0/1577609489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abd3ecb40 msgr2=0x562abd3ecf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:43.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.070+0000 7f4f36a61b80 1 --2- 172.21.15.119:0/1577609489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abd3ecb40 0x562abd3ecf20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4f100a3320 tx=0x7f4f10097910 comp rx=0 tx=0).stop 2023-12-07T10:24:43.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.070+0000 7f4f2179c700 1 --2- 172.21.15.119:0/3267711502 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4f0c0a1c80 0x7f4f0c0a4140 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4f14009e10 tx=0x7f4f1400b040 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:43.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.070+0000 7f4f36a61b80 1 -- 172.21.15.119:0/1577609489 shutdown_connections 2023-12-07T10:24:43.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.070+0000 7f4f36a61b80 1 --2- 172.21.15.119:0/1577609489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abd3f1150 0x562abd3f1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.070+0000 7f4f36a61b80 1 --2- 172.21.15.119:0/1577609489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abd3ecb40 0x562abd3ecf20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.070+0000 7f4f36a61b80 1 --2- 172.21.15.119:0/1577609489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562abd3ed460 0x562abd3f0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.070+0000 7f4f36a61b80 1 -- 172.21.15.119:0/1577609489 >> 172.21.15.119:0/1577609489 conn(0x562abd31b360 msgr2=0x562abd3e7130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:43.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.070+0000 7f4f36a61b80 1 -- 172.21.15.119:0/1577609489 shutdown_connections 2023-12-07T10:24:43.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.070+0000 7f4f36a61b80 1 -- 172.21.15.119:0/1577609489 wait complete. 2023-12-07T10:24:43.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.071+0000 7f4f36a61b80 1 -- start start 2023-12-07T10:24:43.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.071+0000 7f4f36a61b80 1 Processor -- start 2023-12-07T10:24:43.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.072+0000 7f4f36a61b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abd3ecb40 0x562abd3f88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.072+0000 7f4f36a61b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abd3ed460 0x562abd3f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.072+0000 7f4f36a61b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562abd3f1150 0x562abd3f6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.072+0000 7f4f36a61b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562abd3409b0 con 0x562abd3ed460 2023-12-07T10:24:43.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.072+0000 7f4f36a61b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562abd2fa560 con 0x562abd3f1150 2023-12-07T10:24:43.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.072+0000 7f4f36a61b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562abd2fa200 con 0x562abd3ecb40 2023-12-07T10:24:43.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.072+0000 7f4f36a61b80 10 client.0 _ll_register_callbacks cb 0x562abd2e1940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:43.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.072+0000 7f4f20f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abd3ecb40 0x562abd3f88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.072+0000 7f4f2179c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562abd3f1150 0x562abd3f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.072+0000 7f4f20f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abd3ecb40 0x562abd3f88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55716/0 (socket says 172.21.15.119:55716) 2023-12-07T10:24:43.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.072+0000 7f4f20f9b700 1 -- 172.21.15.119:0/2076553912 learned_addr learned my addr 172.21.15.119:0/2076553912 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:43.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.072+0000 7f4f20f9b700 1 -- 172.21.15.119:0/2076553912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562abd3f1150 msgr2=0x562abd3f6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:43.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.072+0000 7f4f20f9b700 1 --2- 172.21.15.119:0/2076553912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562abd3f1150 0x562abd3f6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.072+0000 7f4f20f9b700 1 -- 172.21.15.119:0/2076553912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abd3ed460 msgr2=0x562abd3f5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:43.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.072+0000 7f4f1bfff700 1 --2- 172.21.15.119:0/2076553912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abd3ed460 0x562abd3f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.072+0000 7f4f20f9b700 1 --2- 172.21.15.119:0/2076553912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abd3ed460 0x562abd3f5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.072+0000 7f4f20f9b700 1 -- 172.21.15.119:0/2076553912 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1c1aabe0 con 0x562abd3ecb40 2023-12-07T10:24:43.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.072+0000 7f4f1bfff700 1 --2- 172.21.15.119:0/2076553912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abd3ed460 0x562abd3f5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:24:43.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.073+0000 7f4f20f9b700 1 --2- 172.21.15.119:0/2076553912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abd3ecb40 0x562abd3f88f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4f1009b250 tx=0x7f4f10005360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=906d839549ffe968 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:43.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.073+0000 7f4efbfff700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:24:43.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.073+0000 7f4efbfff700 1 -- 172.21.15.119:0/2076553912 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1009bc50 con 0x562abd3ecb40 2023-12-07T10:24:43.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.073+0000 7f4efbfff700 1 -- 172.21.15.119:0/2076553912 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4f100a3850 con 0x562abd3ecb40 2023-12-07T10:24:43.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.073+0000 7f4f36a61b80 1 -- 172.21.15.119:0/2076553912 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562abd3dfb00 con 0x562abd3ecb40 2023-12-07T10:24:43.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.073+0000 7f4efbfff700 1 -- 172.21.15.119:0/2076553912 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f1009f6a0 con 0x562abd3ecb40 2023-12-07T10:24:43.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.073+0000 7f4f36a61b80 10 client.27040 fetch_fsmap learned FSMap version 41 2023-12-07T10:24:43.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.073+0000 7f4f36a61b80 1 -- 172.21.15.119:0/2076553912 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x562abd169de0 con 0x562abd3ecb40 2023-12-07T10:24:43.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.073+0000 7f4efbfff700 1 -- 172.21.15.119:0/2076553912 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562abd3dfb00 con 0x562abd3ecb40 2023-12-07T10:24:43.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.073+0000 7f4efbfff700 1 -- 172.21.15.119:0/2076553912 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 41) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f4f1009b7b0 con 0x562abd3ecb40 2023-12-07T10:24:43.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.074+0000 7f4f36a61b80 10 client.27040 fetch_fsmap finished waiting for FSMap version 41 2023-12-07T10:24:43.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.074+0000 7f4f36a61b80 10 client.27040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:24:43.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.074+0000 7f4f36a61b80 20 client.27040 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:43.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.074+0000 7f4f36a61b80 1 --2- 172.21.15.119:0/2076553912 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562abd405c40 0x562abd408060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.074+0000 7f4f36a61b80 4 client.27040 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:24:43.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.074+0000 7f4f36a61b80 1 -- 172.21.15.119:0/2076553912 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562abd3bb7a0 con 0x562abd405c40 2023-12-07T10:24:43.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.074+0000 7f4f1bfff700 1 --2- 172.21.15.119:0/2076553912 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562abd405c40 0x562abd408060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.074+0000 7f4efbfff700 10 client.27040 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:43.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.074+0000 7f4f1bfff700 1 --2- 172.21.15.119:0/2076553912 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562abd405c40 0x562abd408060 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1ee68cf4e8eec20 server_cookie=4be6a18d75576fdb in_seq=0 out_seq=0 2023-12-07T10:24:43.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.075+0000 7f4efbfff700 1 -- 172.21.15.119:0/2076553912 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562abd3bb7a0 con 0x562abd405c40 2023-12-07T10:24:43.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.075+0000 7f4efbfff700 10 client.27040 handle_command_reply: tid=0 2023-12-07T10:24:43.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.075+0000 7f4f36a61b80 1 client.27040 shutdown 2023-12-07T10:24:43.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.075+0000 7f4f36a61b80 1 -- 172.21.15.119:0/2076553912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abd3ecb40 msgr2=0x562abd3f88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:43.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.075+0000 7f4f36a61b80 1 --2- 172.21.15.119:0/2076553912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abd3ecb40 0x562abd3f88f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4f1009b250 tx=0x7f4f10005360 comp rx=0 tx=0).stop 2023-12-07T10:24:43.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.075+0000 7f4f36a61b80 1 -- 172.21.15.119:0/2076553912 shutdown_connections 2023-12-07T10:24:43.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.075+0000 7f4f36a61b80 1 --2- 172.21.15.119:0/2076553912 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562abd405c40 0x562abd408060 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.075+0000 7f4f36a61b80 1 --2- 172.21.15.119:0/2076553912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562abd3f1150 0x562abd3f6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.075+0000 7f4f36a61b80 1 --2- 172.21.15.119:0/2076553912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abd3ecb40 0x562abd3f88f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.075+0000 7f4efbfff700 0 client.27040 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:43.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.075+0000 7f4f36a61b80 1 --2- 172.21.15.119:0/2076553912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abd3ed460 0x562abd3f5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.075+0000 7f4f36a61b80 1 -- 172.21.15.119:0/2076553912 >> 172.21.15.119:0/2076553912 conn(0x562abd31b360 msgr2=0x562abd3e1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:43.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.075+0000 7f4f36a61b80 1 -- 172.21.15.119:0/2076553912 shutdown_connections 2023-12-07T10:24:43.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.075+0000 7f4f36a61b80 1 -- 172.21.15.119:0/2076553912 wait complete. 2023-12-07T10:24:43.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.076+0000 7f4f36a61b80 20 client.27040 trim_cache size 0 max 16384 2023-12-07T10:24:43.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.094+0000 7f4f36a61b80 1 Processor -- start 2023-12-07T10:24:43.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.094+0000 7f4f36a61b80 1 -- start start 2023-12-07T10:24:43.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.094+0000 7f4f36a61b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abd3ecb40 0x562abd3ef790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.094+0000 7f4f36a61b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abd3ed460 0x562abd3f1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.094+0000 7f4f36a61b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562abd3f1150 0x562abd3fd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.094+0000 7f4f36a61b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562abd1bf920 con 0x562abd3ed460 2023-12-07T10:24:43.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.094+0000 7f4f36a61b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562abd3bb7a0 con 0x562abd3f1150 2023-12-07T10:24:43.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.094+0000 7f4f36a61b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562abd3dfb00 con 0x562abd3ecb40 2023-12-07T10:24:43.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.095+0000 7f4f2179c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562abd3f1150 0x562abd3fd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.095+0000 7f4f2179c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562abd3f1150 0x562abd3fd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45920/0 (socket says 172.21.15.119:45920) 2023-12-07T10:24:43.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.095+0000 7f4f20f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abd3ecb40 0x562abd3ef790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.095+0000 7f4f2179c700 1 -- 172.21.15.119:0/3002261259 learned_addr learned my addr 172.21.15.119:0/3002261259 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:43.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.095+0000 7f4f1bfff700 1 --2- 172.21.15.119:0/3002261259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abd3ed460 0x562abd3f1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.095+0000 7f4f2179c700 1 -- 172.21.15.119:0/3002261259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abd3ecb40 msgr2=0x562abd3ef790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:43.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.095+0000 7f4f2179c700 1 --2- 172.21.15.119:0/3002261259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abd3ecb40 0x562abd3ef790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.095+0000 7f4f2179c700 1 -- 172.21.15.119:0/3002261259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abd3ed460 msgr2=0x562abd3f1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:43.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.095+0000 7f4f2179c700 1 --2- 172.21.15.119:0/3002261259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abd3ed460 0x562abd3f1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.095+0000 7f4f2179c700 1 -- 172.21.15.119:0/3002261259 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562abd152c60 con 0x562abd3f1150 2023-12-07T10:24:43.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.095+0000 7f4f20f9b700 1 --2- 172.21.15.119:0/3002261259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abd3ecb40 0x562abd3ef790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:43.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.095+0000 7f4f1bfff700 1 --2- 172.21.15.119:0/3002261259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abd3ed460 0x562abd3f1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:43.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.095+0000 7f4f2179c700 1 --2- 172.21.15.119:0/3002261259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562abd3f1150 0x562abd3fd000 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4f14007e60 tx=0x7f4f14014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=819b9f109963bcc2 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:43.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.096+0000 7f4f077fe700 1 -- 172.21.15.119:0/3002261259 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f14004d50 con 0x562abd3f1150 2023-12-07T10:24:43.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.096+0000 7f4f077fe700 1 -- 172.21.15.119:0/3002261259 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f1401a070 con 0x562abd3f1150 2023-12-07T10:24:43.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.096+0000 7f4f077fe700 1 -- 172.21.15.119:0/3002261259 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f14005730 con 0x562abd3f1150 2023-12-07T10:24:43.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.096+0000 7f4f36a61b80 1 -- 172.21.15.119:0/3002261259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562abd3f1150 msgr2=0x562abd3fd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:43.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.096+0000 7f4f36a61b80 1 --2- 172.21.15.119:0/3002261259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562abd3f1150 0x562abd3fd000 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4f14007e60 tx=0x7f4f14014000 comp rx=0 tx=0).stop 2023-12-07T10:24:43.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.096+0000 7f4f36a61b80 1 -- 172.21.15.119:0/3002261259 shutdown_connections 2023-12-07T10:24:43.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.097+0000 7f4f36a61b80 1 --2- 172.21.15.119:0/3002261259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562abd3f1150 0x562abd3fd000 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.097+0000 7f4f36a61b80 1 --2- 172.21.15.119:0/3002261259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abd3ecb40 0x562abd3ef790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.097+0000 7f4f36a61b80 1 --2- 172.21.15.119:0/3002261259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abd3ed460 0x562abd3f1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.097+0000 7f4f36a61b80 1 -- 172.21.15.119:0/3002261259 >> 172.21.15.119:0/3002261259 conn(0x562abd31b360 msgr2=0x562abd3f8620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:43.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.097+0000 7f4f36a61b80 1 -- 172.21.15.119:0/3002261259 shutdown_connections 2023-12-07T10:24:43.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.097+0000 7f4f36a61b80 1 -- 172.21.15.119:0/3002261259 wait complete. 2023-12-07T10:24:43.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.097+0000 7f4f36a61b80 1 -- start start 2023-12-07T10:24:43.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.097+0000 7f4f36a61b80 1 Processor -- start 2023-12-07T10:24:43.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.098+0000 7f4f36a61b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562abd3ecb40 0x562abd3fd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.098+0000 7f4f36a61b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abd3ed460 0x562abd3fa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.098+0000 7f4f36a61b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abd3f1150 0x562abd3fa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.098+0000 7f4f36a61b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562abd3fd5a0 con 0x562abd3ed460 2023-12-07T10:24:43.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.098+0000 7f4f36a61b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562abd3fd710 con 0x562abd3ecb40 2023-12-07T10:24:43.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.098+0000 7f4f36a61b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562abd3fd8d0 con 0x562abd3f1150 2023-12-07T10:24:43.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.098+0000 7f4f36a61b80 10 client.0 _ll_register_callbacks cb 0x562abd2e1940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:43.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.098+0000 7f4f20f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562abd3ecb40 0x562abd3fd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.098+0000 7f4f20f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562abd3ecb40 0x562abd3fd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45922/0 (socket says 172.21.15.119:45922) 2023-12-07T10:24:43.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.098+0000 7f4f20f9b700 1 -- 172.21.15.119:0/3550961137 learned_addr learned my addr 172.21.15.119:0/3550961137 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:43.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.098+0000 7f4f2179c700 1 --2- 172.21.15.119:0/3550961137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abd3f1150 0x562abd3fa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.099+0000 7f4f1bfff700 1 --2- 172.21.15.119:0/3550961137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abd3ed460 0x562abd3fa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.099+0000 7f4f20f9b700 1 -- 172.21.15.119:0/3550961137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abd3f1150 msgr2=0x562abd3fa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:43.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.099+0000 7f4f20f9b700 1 --2- 172.21.15.119:0/3550961137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abd3f1150 0x562abd3fa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.099+0000 7f4f20f9b700 1 -- 172.21.15.119:0/3550961137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abd3ed460 msgr2=0x562abd3fa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:43.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.099+0000 7f4f20f9b700 1 --2- 172.21.15.119:0/3550961137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abd3ed460 0x562abd3fa030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.099+0000 7f4f20f9b700 1 -- 172.21.15.119:0/3550961137 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562abd169de0 con 0x562abd3ecb40 2023-12-07T10:24:43.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.099+0000 7f4f1bfff700 1 --2- 172.21.15.119:0/3550961137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abd3ed460 0x562abd3fa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:24:43.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.099+0000 7f4f20f9b700 1 --2- 172.21.15.119:0/3550961137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562abd3ecb40 0x562abd3fd060 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4f10000c00 tx=0x7f4f10009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a2981ad4d5dae54 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:43.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.099+0000 7f4efbfff700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:24:43.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.099+0000 7f4efbfff700 1 -- 172.21.15.119:0/3550961137 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1009b320 con 0x562abd3ecb40 2023-12-07T10:24:43.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.099+0000 7f4efbfff700 1 -- 172.21.15.119:0/3550961137 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f4f1009bc50 con 0x562abd3ecb40 2023-12-07T10:24:43.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.099+0000 7f4f36a61b80 1 -- 172.21.15.119:0/3550961137 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f10073190 con 0x562abd3ecb40 2023-12-07T10:24:43.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.099+0000 7f4efbfff700 1 -- 172.21.15.119:0/3550961137 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f1009aeb0 con 0x562abd3ecb40 2023-12-07T10:24:43.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.099+0000 7f4f36a61b80 10 client.27062 fetch_fsmap learned FSMap version 41 2023-12-07T10:24:43.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.099+0000 7f4f36a61b80 1 -- 172.21.15.119:0/3550961137 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f4f080126d0 con 0x562abd3ecb40 2023-12-07T10:24:43.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.100+0000 7f4efbfff700 1 -- 172.21.15.119:0/3550961137 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f10073190 con 0x562abd3ecb40 2023-12-07T10:24:43.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.100+0000 7f4efbfff700 1 -- 172.21.15.119:0/3550961137 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 41) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f4f10066030 con 0x562abd3ecb40 2023-12-07T10:24:43.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.100+0000 7f4f36a61b80 10 client.27062 fetch_fsmap finished waiting for FSMap version 41 2023-12-07T10:24:43.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.100+0000 7f4f36a61b80 10 client.27062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:24:43.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.100+0000 7f4f36a61b80 20 client.27062 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:43.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.100+0000 7f4f36a61b80 1 --2- 172.21.15.119:0/3550961137 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562abd45ce00 0x562abd45f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.100+0000 7f4f36a61b80 4 client.27062 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-12-07T10:24:43.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.100+0000 7f4f36a61b80 1 -- 172.21.15.119:0/3550961137 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562abd306fd0 con 0x562abd45ce00 2023-12-07T10:24:43.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.100+0000 7f4f1bfff700 1 --2- 172.21.15.119:0/3550961137 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562abd45ce00 0x562abd45f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.101+0000 7f4efbfff700 10 client.27062 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:43.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.101+0000 7f4f1bfff700 1 --2- 172.21.15.119:0/3550961137 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562abd45ce00 0x562abd45f220 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3ec65b4fa63cb11 server_cookie=11f9c0087f76300c in_seq=0 out_seq=0 2023-12-07T10:24:43.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.101+0000 7f4efbfff700 1 -- 172.21.15.119:0/3550961137 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562abd306fd0 con 0x562abd45ce00 2023-12-07T10:24:43.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.101+0000 7f4efbfff700 10 client.27062 handle_command_reply: tid=0 2023-12-07T10:24:43.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.101+0000 7f4f36a61b80 1 client.27062 shutdown 2023-12-07T10:24:43.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.101+0000 7f4f36a61b80 1 -- 172.21.15.119:0/3550961137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562abd3ecb40 msgr2=0x562abd3fd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:43.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.101+0000 7f4f36a61b80 1 --2- 172.21.15.119:0/3550961137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562abd3ecb40 0x562abd3fd060 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4f10000c00 tx=0x7f4f10009c00 comp rx=0 tx=0).stop 2023-12-07T10:24:43.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.101+0000 7f4f36a61b80 1 -- 172.21.15.119:0/3550961137 shutdown_connections 2023-12-07T10:24:43.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.101+0000 7f4f36a61b80 1 --2- 172.21.15.119:0/3550961137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abd3f1150 0x562abd3fa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.101+0000 7f4f36a61b80 1 --2- 172.21.15.119:0/3550961137 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562abd45ce00 0x562abd45f220 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.101+0000 7f4f36a61b80 1 --2- 172.21.15.119:0/3550961137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562abd3ecb40 0x562abd3fd060 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.101+0000 7f4f36a61b80 1 --2- 172.21.15.119:0/3550961137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562abd3ed460 0x562abd3fa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.101+0000 7f4f36a61b80 1 -- 172.21.15.119:0/3550961137 >> 172.21.15.119:0/3550961137 conn(0x562abd31b360 msgr2=0x562abd3f1530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:43.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.101+0000 7f4efbfff700 0 client.27062 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:43.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.101+0000 7f4f36a61b80 1 -- 172.21.15.119:0/3550961137 shutdown_connections 2023-12-07T10:24:43.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.101+0000 7f4f36a61b80 1 -- 172.21.15.119:0/3550961137 wait complete. 2023-12-07T10:24:43.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.102+0000 7f4f36a61b80 20 client.27062 trim_cache size 0 max 16384 2023-12-07T10:24:43.107 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:24:43.107 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "no active scrubs running", 2023-12-07T10:24:43.107 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": {} 2023-12-07T10:24:43.107 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:24:43.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.102+0000 7f4f231ff700 1 -- 172.21.15.119:0/3267711502 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4f0c0a1c80 msgr2=0x7f4f0c0a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:43.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.102+0000 7f4f231ff700 1 --2- 172.21.15.119:0/3267711502 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4f0c0a1c80 0x7f4f0c0a4140 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4f14009e10 tx=0x7f4f1400b040 comp rx=0 tx=0).stop 2023-12-07T10:24:43.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.102+0000 7f4f231ff700 1 -- 172.21.15.119:0/3267711502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f1c068a50 msgr2=0x7f4f1c1083c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:43.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.102+0000 7f4f231ff700 1 --2- 172.21.15.119:0/3267711502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f1c068a50 0x7f4f1c1083c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4f10009fd0 tx=0x7f4f1000f9e0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.102+0000 7f4f231ff700 1 -- 172.21.15.119:0/3267711502 shutdown_connections 2023-12-07T10:24:43.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.102+0000 7f4f231ff700 1 --2- 172.21.15.119:0/3267711502 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4f0c0a1c80 0x7f4f0c0a4140 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.102+0000 7f4f231ff700 1 --2- 172.21.15.119:0/3267711502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f1c106ad0 0x7f4f1c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.102+0000 7f4f231ff700 1 --2- 172.21.15.119:0/3267711502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f1c068a50 0x7f4f1c1083c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.102+0000 7f4f231ff700 1 --2- 172.21.15.119:0/3267711502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c106110 0x7f4f1c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.102+0000 7f4f231ff700 1 -- 172.21.15.119:0/3267711502 >> 172.21.15.119:0/3267711502 conn(0x7f4f1c077610 msgr2=0x7f4f1c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:43.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.102+0000 7f4f231ff700 1 -- 172.21.15.119:0/3267711502 shutdown_connections 2023-12-07T10:24:43.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.102+0000 7f4f231ff700 1 -- 172.21.15.119:0/3267711502 wait complete. 2023-12-07T10:24:43.122 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-07T10:24:43.122 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:24:43.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.336+0000 7f3d861ff700 1 -- 172.21.15.119:0/3186140182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d80107650 msgr2=0x7f3d80107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:43.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.336+0000 7f3d861ff700 1 --2- 172.21.15.119:0/3186140182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d80107650 0x7f3d80107a30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009990 tx=0x7f3d70009ca0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.336+0000 7f3d861ff700 1 -- 172.21.15.119:0/3186140182 shutdown_connections 2023-12-07T10:24:43.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.336+0000 7f3d861ff700 1 --2- 172.21.15.119:0/3186140182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d80101f50 0x7f3d8010e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.336+0000 7f3d861ff700 1 --2- 172.21.15.119:0/3186140182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d80107650 0x7f3d80107a30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.336+0000 7f3d861ff700 1 --2- 172.21.15.119:0/3186140182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d80101590 0x7f3d80101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.336+0000 7f3d861ff700 1 -- 172.21.15.119:0/3186140182 >> 172.21.15.119:0/3186140182 conn(0x7f3d800fd080 msgr2=0x7f3d800ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:43.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.336+0000 7f3d861ff700 1 -- 172.21.15.119:0/3186140182 shutdown_connections 2023-12-07T10:24:43.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.337+0000 7f3d861ff700 1 -- 172.21.15.119:0/3186140182 wait complete. 2023-12-07T10:24:43.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.338+0000 7f3d861ff700 1 Processor -- start 2023-12-07T10:24:43.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.338+0000 7f3d861ff700 1 -- start start 2023-12-07T10:24:43.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.338+0000 7f3d861ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d80101590 0x7f3d8019f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.338+0000 7f3d861ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d80101f50 0x7f3d8019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.338+0000 7f3d861ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d80107650 0x7f3d80199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.338+0000 7f3d861ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8019a3b0 con 0x7f3d80101590 2023-12-07T10:24:43.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.338+0000 7f3d861ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8019a520 con 0x7f3d80101f50 2023-12-07T10:24:43.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.338+0000 7f3d861ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8019a6e0 con 0x7f3d80107650 2023-12-07T10:24:43.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.339+0000 7f3d7ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d80107650 0x7f3d80199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.339+0000 7f3d7ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d80107650 0x7f3d80199c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55760/0 (socket says 172.21.15.119:55760) 2023-12-07T10:24:43.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.339+0000 7f3d7ffff700 1 -- 172.21.15.119:0/3198493399 learned_addr learned my addr 172.21.15.119:0/3198493399 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:43.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.339+0000 7f3d7f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d80101590 0x7f3d8019f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.339+0000 7f3d7effd700 1 --2- 172.21.15.119:0/3198493399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d80101f50 0x7f3d8019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.339+0000 7f3d7ffff700 1 -- 172.21.15.119:0/3198493399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d80101f50 msgr2=0x7f3d8019fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:43.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.339+0000 7f3d7ffff700 1 --2- 172.21.15.119:0/3198493399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d80101f50 0x7f3d8019fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.339+0000 7f3d7ffff700 1 -- 172.21.15.119:0/3198493399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d80101590 msgr2=0x7f3d8019f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:43.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.339+0000 7f3d7ffff700 1 --2- 172.21.15.119:0/3198493399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d80101590 0x7f3d8019f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.339+0000 7f3d7ffff700 1 -- 172.21.15.119:0/3198493399 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d700095f0 con 0x7f3d80107650 2023-12-07T10:24:43.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.339+0000 7f3d7effd700 1 --2- 172.21.15.119:0/3198493399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d80101f50 0x7f3d8019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:24:43.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.339+0000 7f3d7ffff700 1 --2- 172.21.15.119:0/3198493399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d80107650 0x7f3d80199c10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800ef30 tx=0x7f3d7800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:43.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.340+0000 7f3d7cff9700 1 -- 172.21.15.119:0/3198493399 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d78018070 con 0x7f3d80107650 2023-12-07T10:24:43.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.340+0000 7f3d7cff9700 1 -- 172.21.15.119:0/3198493399 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d7800f040 con 0x7f3d80107650 2023-12-07T10:24:43.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.340+0000 7f3d7cff9700 1 -- 172.21.15.119:0/3198493399 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7800cdb0 con 0x7f3d80107650 2023-12-07T10:24:43.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.340+0000 7f3d861ff700 1 -- 172.21.15.119:0/3198493399 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d8019a960 con 0x7f3d80107650 2023-12-07T10:24:43.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.340+0000 7f3d861ff700 1 -- 172.21.15.119:0/3198493399 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d801a6850 con 0x7f3d80107650 2023-12-07T10:24:43.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.341+0000 7f3d861ff700 1 -- 172.21.15.119:0/3198493399 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d80102cb0 con 0x7f3d80107650 2023-12-07T10:24:43.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.343+0000 7f3d7cff9700 1 -- 172.21.15.119:0/3198493399 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f3d78003a00 con 0x7f3d80107650 2023-12-07T10:24:43.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.345+0000 7f3d7cff9700 1 --2- 172.21.15.119:0/3198493399 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f3d680a1d00 0x7f3d680a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.345+0000 7f3d7cff9700 1 -- 172.21.15.119:0/3198493399 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f3d780cd9b0 con 0x7f3d80107650 2023-12-07T10:24:43.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.345+0000 7f3d7f7fe700 1 --2- 172.21.15.119:0/3198493399 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f3d680a1d00 0x7f3d680a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.346+0000 7f3d7f7fe700 1 --2- 172.21.15.119:0/3198493399 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f3d680a1d00 0x7f3d680a41c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000b2e0 tx=0x7f3d70005fd0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:43.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.348+0000 7f3d7cff9700 1 -- 172.21.15.119:0/3198493399 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3d78096770 con 0x7f3d80107650 2023-12-07T10:24:43.553 INFO:tasks.workunit.client.0.smithi119.stdout: 1 260133 6.90 MB/sec execute 56 sec latency 13.041 ms 2023-12-07T10:24:43.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.634+0000 7f3d861ff700 1 -- 172.21.15.119:0/3198493399 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d80068770 con 0x7f3d80107650 2023-12-07T10:24:43.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.635+0000 7f3d7cff9700 1 -- 172.21.15.119:0/3198493399 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+11394 (secure 0 0 0) 0x7f3d78099d90 con 0x7f3d80107650 2023-12-07T10:24:43.636 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:24:43.636 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-07T10:17:41.278311+0000","modified":"2023-12-07T10:24:38.056754+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24517,"mds_3":24505,"mds_4":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14535":{"gid":14535,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6841/1791998028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1791998028},{"type":"v1","addr":"172.21.15.119:6841","nonce":1791998028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24505":{"gid":24505,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":106,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/2925503246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2925503246},{"type":"v1","addr":"172.21.15.153:6837","nonce":2925503246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6839/3984962900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3984962900},{"type":"v1","addr":"172.21.15.159:6839","nonce":3984962900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.119:6839/1920762602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1920762602},{"type":"v1","addr":"172.21.15.119:6839","nonce":1920762602}]},"join_fscid":-1,"export_targets":[4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/566825184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":566825184},{"type":"v1","addr":"172.21.15.153:6839","nonce":566825184}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-07T10:24:43.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.638+0000 7f3d861ff700 1 -- 172.21.15.119:0/3198493399 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f3d680a1d00 msgr2=0x7f3d680a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:43.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.638+0000 7f3d861ff700 1 --2- 172.21.15.119:0/3198493399 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f3d680a1d00 0x7f3d680a41c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000b2e0 tx=0x7f3d70005fd0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.638+0000 7f3d861ff700 1 -- 172.21.15.119:0/3198493399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d80107650 msgr2=0x7f3d80199c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:43.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.638+0000 7f3d861ff700 1 --2- 172.21.15.119:0/3198493399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d80107650 0x7f3d80199c10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800ef30 tx=0x7f3d7800c3a0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.638+0000 7f3d861ff700 1 -- 172.21.15.119:0/3198493399 shutdown_connections 2023-12-07T10:24:43.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.638+0000 7f3d861ff700 1 --2- 172.21.15.119:0/3198493399 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f3d680a1d00 0x7f3d680a41c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.638+0000 7f3d861ff700 1 --2- 172.21.15.119:0/3198493399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d80107650 0x7f3d80199c10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.638+0000 7f3d861ff700 1 --2- 172.21.15.119:0/3198493399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d80101590 0x7f3d8019f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.638+0000 7f3d861ff700 1 --2- 172.21.15.119:0/3198493399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d80101f50 0x7f3d8019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.638+0000 7f3d861ff700 1 -- 172.21.15.119:0/3198493399 >> 172.21.15.119:0/3198493399 conn(0x7f3d800fd080 msgr2=0x7f3d800fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:43.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.638+0000 7f3d861ff700 1 -- 172.21.15.119:0/3198493399 shutdown_connections 2023-12-07T10:24:43.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.638+0000 7f3d861ff700 1 -- 172.21.15.119:0/3198493399 wait complete. 2023-12-07T10:24:43.640 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 41 2023-12-07T10:24:43.665 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-12-07T10:24:43.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.902+0000 7f8089dff700 1 -- 172.21.15.119:0/1101522076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8084107620 msgr2=0x7f8084107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:43.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.902+0000 7f8089dff700 1 --2- 172.21.15.119:0/1101522076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8084107620 0x7f8084107a00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8074009990 tx=0x7f8074009ca0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.902+0000 7f8089dff700 1 -- 172.21.15.119:0/1101522076 shutdown_connections 2023-12-07T10:24:43.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.902+0000 7f8089dff700 1 --2- 172.21.15.119:0/1101522076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8084101f20 0x7f808410e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.902+0000 7f8089dff700 1 --2- 172.21.15.119:0/1101522076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8084107620 0x7f8084107a00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.902+0000 7f8089dff700 1 --2- 172.21.15.119:0/1101522076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8084101560 0x7f80841019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.902+0000 7f8089dff700 1 -- 172.21.15.119:0/1101522076 >> 172.21.15.119:0/1101522076 conn(0x7f80840fcfd0 msgr2=0x7f80840ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:43.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.902+0000 7f8089dff700 1 -- 172.21.15.119:0/1101522076 shutdown_connections 2023-12-07T10:24:43.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.903+0000 7f8089dff700 1 -- 172.21.15.119:0/1101522076 wait complete. 2023-12-07T10:24:43.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.903+0000 7f8089dff700 1 Processor -- start 2023-12-07T10:24:43.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.904+0000 7f8089dff700 1 -- start start 2023-12-07T10:24:43.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.904+0000 7f8089dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8084101560 0x7f808419d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.904+0000 7f8089dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8084101f20 0x7f808419dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.904+0000 7f8089dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8084107620 0x7f80841978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.904+0000 7f8089dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8084197f30 con 0x7f8084107620 2023-12-07T10:24:43.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.904+0000 7f8089dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80841980a0 con 0x7f8084101f20 2023-12-07T10:24:43.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.904+0000 7f8089dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8084198260 con 0x7f8084101560 2023-12-07T10:24:43.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.904+0000 7f80837fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8084101560 0x7f808419d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.904+0000 7f8082ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8084101f20 0x7f808419dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.904+0000 7f80837fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8084101560 0x7f808419d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55780/0 (socket says 172.21.15.119:55780) 2023-12-07T10:24:43.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.904+0000 7f80837fe700 1 -- 172.21.15.119:0/3355330185 learned_addr learned my addr 172.21.15.119:0/3355330185 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:43.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.904+0000 7f8083fff700 1 --2- 172.21.15.119:0/3355330185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8084107620 0x7f80841978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.905+0000 7f80837fe700 1 -- 172.21.15.119:0/3355330185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8084101f20 msgr2=0x7f808419dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:43.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.905+0000 7f80837fe700 1 --2- 172.21.15.119:0/3355330185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8084101f20 0x7f808419dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.905+0000 7f80837fe700 1 -- 172.21.15.119:0/3355330185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8084107620 msgr2=0x7f80841978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:43.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.905+0000 7f80837fe700 1 --2- 172.21.15.119:0/3355330185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8084107620 0x7f80841978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.905+0000 7f80837fe700 1 -- 172.21.15.119:0/3355330185 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c009c40 con 0x7f8084101560 2023-12-07T10:24:43.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.905+0000 7f80837fe700 1 --2- 172.21.15.119:0/3355330185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8084101560 0x7f808419d7a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8074005910 tx=0x7f8074004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:43.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.905+0000 7f8080ff9700 1 -- 172.21.15.119:0/3355330185 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807401d070 con 0x7f8084101560 2023-12-07T10:24:43.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.905+0000 7f8080ff9700 1 -- 172.21.15.119:0/3355330185 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8074022470 con 0x7f8084101560 2023-12-07T10:24:43.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.905+0000 7f8089dff700 1 -- 172.21.15.119:0/3355330185 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80740095f0 con 0x7f8084101560 2023-12-07T10:24:43.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.905+0000 7f8080ff9700 1 -- 172.21.15.119:0/3355330185 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8074017650 con 0x7f8084101560 2023-12-07T10:24:43.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.906+0000 7f8089dff700 1 -- 172.21.15.119:0/3355330185 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80841a45b0 con 0x7f8084101560 2023-12-07T10:24:43.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.909+0000 7f8080ff9700 1 -- 172.21.15.119:0/3355330185 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f80740225e0 con 0x7f8084101560 2023-12-07T10:24:43.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.910+0000 7f809d79bb80 1 Processor -- start 2023-12-07T10:24:43.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.910+0000 7f8080ff9700 1 --2- 172.21.15.119:0/3355330185 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f806c0a1d00 0x7f806c0a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.910+0000 7f809d79bb80 1 -- start start 2023-12-07T10:24:43.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.910+0000 7f8080ff9700 1 -- 172.21.15.119:0/3355330185 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f80740ced70 con 0x7f8084101560 2023-12-07T10:24:43.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.910+0000 7f8083fff700 1 --2- 172.21.15.119:0/3355330185 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f806c0a1d00 0x7f806c0a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.911+0000 7f809d79bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5573ab9caf50 0x5573ab9cb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.911+0000 7f809d79bb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573ab9cb870 0x5573ab9cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.911+0000 7f809d79bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573ab9cf560 0x5573ab9cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.911+0000 7f809d79bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ab93a7c0 con 0x5573ab9caf50 2023-12-07T10:24:43.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.911+0000 7f809d79bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ab9217a0 con 0x5573ab9cf560 2023-12-07T10:24:43.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.911+0000 7f809d79bb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ab91f0c0 con 0x5573ab9cb870 2023-12-07T10:24:43.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.911+0000 7f80837fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5573ab9caf50 0x5573ab9cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.911+0000 7f8082ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573ab9cb870 0x5573ab9cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.911+0000 7f80837fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5573ab9caf50 0x5573ab9cb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45016/0 (socket says 172.21.15.119:45016) 2023-12-07T10:24:43.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.911+0000 7f80837fe700 1 -- 172.21.15.119:0/3403463602 learned_addr learned my addr 172.21.15.119:0/3403463602 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:43.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.911+0000 7f8083fff700 1 --2- 172.21.15.119:0/3355330185 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f806c0a1d00 0x7f806c0a41c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f807c009fd0 tx=0x7f807c00b040 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:43.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.911+0000 7f8083fff700 1 --2- 172.21.15.119:0/3403463602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573ab9cf560 0x5573ab9cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.911+0000 7f8082ffd700 1 -- 172.21.15.119:0/3403463602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573ab9cf560 msgr2=0x5573ab9cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:43.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.911+0000 7f8082ffd700 1 --2- 172.21.15.119:0/3403463602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573ab9cf560 0x5573ab9cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.911+0000 7f8082ffd700 1 -- 172.21.15.119:0/3403463602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5573ab9caf50 msgr2=0x5573ab9cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:43.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.911+0000 7f8082ffd700 1 --2- 172.21.15.119:0/3403463602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5573ab9caf50 0x5573ab9cb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.911+0000 7f8082ffd700 1 -- 172.21.15.119:0/3403463602 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573ab797630 con 0x5573ab9cb870 2023-12-07T10:24:43.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.911+0000 7f80837fe700 1 --2- 172.21.15.119:0/3403463602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5573ab9caf50 0x5573ab9cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:43.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.912+0000 7f8082ffd700 1 --2- 172.21.15.119:0/3403463602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573ab9cb870 0x5573ab9cf020 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8078009cb0 tx=0x7f8078011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bb24520dae99541 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:43.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.912+0000 7f8069ffb700 1 -- 172.21.15.119:0/3403463602 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8078011d20 con 0x5573ab9cb870 2023-12-07T10:24:43.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.912+0000 7f8069ffb700 1 -- 172.21.15.119:0/3403463602 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8078011e80 con 0x5573ab9cb870 2023-12-07T10:24:43.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.912+0000 7f8069ffb700 1 -- 172.21.15.119:0/3403463602 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8078005280 con 0x5573ab9cb870 2023-12-07T10:24:43.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.913+0000 7f809d79bb80 1 -- 172.21.15.119:0/3403463602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573ab9cb870 msgr2=0x5573ab9cf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:43.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.913+0000 7f809d79bb80 1 --2- 172.21.15.119:0/3403463602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573ab9cb870 0x5573ab9cf020 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8078009cb0 tx=0x7f8078011440 comp rx=0 tx=0).stop 2023-12-07T10:24:43.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.913+0000 7f809d79bb80 1 -- 172.21.15.119:0/3403463602 shutdown_connections 2023-12-07T10:24:43.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.913+0000 7f809d79bb80 1 --2- 172.21.15.119:0/3403463602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573ab9cf560 0x5573ab9cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.913+0000 7f809d79bb80 1 --2- 172.21.15.119:0/3403463602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5573ab9caf50 0x5573ab9cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.913+0000 7f809d79bb80 1 --2- 172.21.15.119:0/3403463602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573ab9cb870 0x5573ab9cf020 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.913+0000 7f809d79bb80 1 -- 172.21.15.119:0/3403463602 >> 172.21.15.119:0/3403463602 conn(0x5573ab8f5360 msgr2=0x5573ab9c1130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:43.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.913+0000 7f809d79bb80 1 -- 172.21.15.119:0/3403463602 shutdown_connections 2023-12-07T10:24:43.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.913+0000 7f809d79bb80 1 -- 172.21.15.119:0/3403463602 wait complete. 2023-12-07T10:24:43.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.914+0000 7f809d79bb80 1 -- start start 2023-12-07T10:24:43.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.914+0000 7f809d79bb80 1 Processor -- start 2023-12-07T10:24:43.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.915+0000 7f809d79bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573ab9caf50 0x5573ab9d6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.915+0000 7f809d79bb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573ab9cb870 0x5573ab9d4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.915+0000 7f809d79bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5573ab9cf560 0x5573ab9d47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.915+0000 7f809d79bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ab91a9b0 con 0x5573ab9cf560 2023-12-07T10:24:43.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.915+0000 7f809d79bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ab8d4560 con 0x5573ab9caf50 2023-12-07T10:24:43.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.915+0000 7f809d79bb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ab8d4200 con 0x5573ab9cb870 2023-12-07T10:24:43.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.915+0000 7f809d79bb80 10 client.0 _ll_register_callbacks cb 0x5573ab8bb940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:43.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.915+0000 7f8082ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573ab9cb870 0x5573ab9d4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.915+0000 7f8082ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573ab9cb870 0x5573ab9d4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55796/0 (socket says 172.21.15.119:55796) 2023-12-07T10:24:43.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.915+0000 7f8082ffd700 1 -- 172.21.15.119:0/4273071182 learned_addr learned my addr 172.21.15.119:0/4273071182 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:43.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.915+0000 7f8083fff700 1 --2- 172.21.15.119:0/4273071182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5573ab9cf560 0x5573ab9d47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.915+0000 7f80837fe700 1 --2- 172.21.15.119:0/4273071182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573ab9caf50 0x5573ab9d6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.915+0000 7f8082ffd700 1 -- 172.21.15.119:0/4273071182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573ab9caf50 msgr2=0x5573ab9d6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:43.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.915+0000 7f8082ffd700 1 --2- 172.21.15.119:0/4273071182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573ab9caf50 0x5573ab9d6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.915+0000 7f8082ffd700 1 -- 172.21.15.119:0/4273071182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5573ab9cf560 msgr2=0x5573ab9d47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:43.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.915+0000 7f8082ffd700 1 --2- 172.21.15.119:0/4273071182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5573ab9cf560 0x5573ab9d47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.915+0000 7f8082ffd700 1 -- 172.21.15.119:0/4273071182 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80841a45b0 con 0x5573ab9cb870 2023-12-07T10:24:43.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.916+0000 7f8082ffd700 1 --2- 172.21.15.119:0/4273071182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573ab9cb870 0x5573ab9d4070 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8078005c50 tx=0x7f8078005e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a90024abecd1585 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:43.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.916+0000 7f8062ffd700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:24:43.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.916+0000 7f8062ffd700 1 -- 172.21.15.119:0/4273071182 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8078004260 con 0x5573ab9cb870 2023-12-07T10:24:43.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.916+0000 7f809d79bb80 1 -- 172.21.15.119:0/4273071182 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573ab9957a0 con 0x5573ab9cb870 2023-12-07T10:24:43.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.916+0000 7f8062ffd700 1 -- 172.21.15.119:0/4273071182 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f80780043c0 con 0x5573ab9cb870 2023-12-07T10:24:43.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.916+0000 7f8062ffd700 1 -- 172.21.15.119:0/4273071182 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8078004580 con 0x5573ab9cb870 2023-12-07T10:24:43.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.916+0000 7f809d79bb80 10 client.27064 fetch_fsmap learned FSMap version 41 2023-12-07T10:24:43.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.916+0000 7f809d79bb80 1 -- 172.21.15.119:0/4273071182 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f80780098e0 con 0x5573ab9cb870 2023-12-07T10:24:43.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.916+0000 7f8062ffd700 1 -- 172.21.15.119:0/4273071182 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5573ab9957a0 con 0x5573ab9cb870 2023-12-07T10:24:43.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.917+0000 7f8062ffd700 1 -- 172.21.15.119:0/4273071182 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 41) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f80780052c0 con 0x5573ab9cb870 2023-12-07T10:24:43.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.917+0000 7f809d79bb80 10 client.27064 fetch_fsmap finished waiting for FSMap version 41 2023-12-07T10:24:43.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.917+0000 7f809d79bb80 10 client.27064 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2023-12-07T10:24:43.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.917+0000 7f809d79bb80 20 client.27064 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:43.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.917+0000 7f809d79bb80 1 --2- 172.21.15.119:0/4273071182 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x5573ab9e4780 0x5573ab9e6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.917+0000 7f809d79bb80 4 client.27064 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:24:43.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.917+0000 7f809d79bb80 1 -- 172.21.15.119:0/4273071182 --> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573ab8e0fd0 con 0x5573ab9e4780 2023-12-07T10:24:43.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.917+0000 7f80837fe700 1 --2- 172.21.15.119:0/4273071182 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x5573ab9e4780 0x5573ab9e6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.918+0000 7f80837fe700 1 --2- 172.21.15.119:0/4273071182 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x5573ab9e4780 0x5573ab9e6ba0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bd53b4090f55b6a4 server_cookie=601f30573ec228fd in_seq=0 out_seq=0 2023-12-07T10:24:43.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.918+0000 7f8062ffd700 10 client.27064 ms_handle_connect on v2:172.21.15.153:6834/1295649464 2023-12-07T10:24:43.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.918+0000 7f8062ffd700 1 -- 172.21.15.119:0/4273071182 <== mds.4 v2:172.21.15.153:6834/1295649464 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5573ab8e0fd0 con 0x5573ab9e4780 2023-12-07T10:24:43.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.918+0000 7f8062ffd700 10 client.27064 handle_command_reply: tid=0 2023-12-07T10:24:43.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.918+0000 7f809d79bb80 1 client.27064 shutdown 2023-12-07T10:24:43.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.919+0000 7f809d79bb80 1 -- 172.21.15.119:0/4273071182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573ab9cb870 msgr2=0x5573ab9d4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:43.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.919+0000 7f809d79bb80 1 --2- 172.21.15.119:0/4273071182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573ab9cb870 0x5573ab9d4070 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8078005c50 tx=0x7f8078005e30 comp rx=0 tx=0).stop 2023-12-07T10:24:43.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.919+0000 7f809d79bb80 1 -- 172.21.15.119:0/4273071182 shutdown_connections 2023-12-07T10:24:43.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.919+0000 7f809d79bb80 1 --2- 172.21.15.119:0/4273071182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5573ab9cf560 0x5573ab9d47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.919+0000 7f809d79bb80 1 --2- 172.21.15.119:0/4273071182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573ab9caf50 0x5573ab9d6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.919+0000 7f809d79bb80 1 --2- 172.21.15.119:0/4273071182 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x5573ab9e4780 0x5573ab9e6ba0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.919+0000 7f809d79bb80 1 --2- 172.21.15.119:0/4273071182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573ab9cb870 0x5573ab9d4070 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.919+0000 7f809d79bb80 1 -- 172.21.15.119:0/4273071182 >> 172.21.15.119:0/4273071182 conn(0x5573ab8f5360 msgr2=0x5573ab9d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:43.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.919+0000 7f8062ffd700 0 client.27064 ms_handle_reset on v2:172.21.15.153:6834/1295649464 2023-12-07T10:24:43.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.919+0000 7f809d79bb80 1 -- 172.21.15.119:0/4273071182 shutdown_connections 2023-12-07T10:24:43.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.919+0000 7f809d79bb80 1 -- 172.21.15.119:0/4273071182 wait complete. 2023-12-07T10:24:43.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.919+0000 7f809d79bb80 20 client.27064 trim_cache size 0 max 16384 2023-12-07T10:24:43.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.935+0000 7f809d79bb80 1 Processor -- start 2023-12-07T10:24:43.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.935+0000 7f809d79bb80 1 -- start start 2023-12-07T10:24:43.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.935+0000 7f809d79bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573ab9caf50 0x5573ab9e0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.935+0000 7f809d79bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5573ab9cb870 0x5573ab9cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.935+0000 7f809d79bb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573ab9cf560 0x5573ab9bbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.935+0000 7f809d79bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ab799920 con 0x5573ab9cb870 2023-12-07T10:24:43.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.935+0000 7f809d79bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ab9ebda0 con 0x5573ab9caf50 2023-12-07T10:24:43.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.935+0000 7f809d79bb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ab9d4b60 con 0x5573ab9cf560 2023-12-07T10:24:43.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.935+0000 7f80837fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573ab9caf50 0x5573ab9e0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.935+0000 7f8083fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573ab9cf560 0x5573ab9bbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.935+0000 7f8083fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573ab9cf560 0x5573ab9bbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55814/0 (socket says 172.21.15.119:55814) 2023-12-07T10:24:43.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.935+0000 7f8083fff700 1 -- 172.21.15.119:0/4135544400 learned_addr learned my addr 172.21.15.119:0/4135544400 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:43.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.935+0000 7f8082ffd700 1 --2- 172.21.15.119:0/4135544400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5573ab9cb870 0x5573ab9cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.936+0000 7f80837fe700 1 -- 172.21.15.119:0/4135544400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573ab9cf560 msgr2=0x5573ab9bbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:43.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.936+0000 7f80837fe700 1 --2- 172.21.15.119:0/4135544400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573ab9cf560 0x5573ab9bbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.936+0000 7f80837fe700 1 -- 172.21.15.119:0/4135544400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5573ab9cb870 msgr2=0x5573ab9cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:43.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.936+0000 7f80837fe700 1 --2- 172.21.15.119:0/4135544400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5573ab9cb870 0x5573ab9cf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.936+0000 7f80837fe700 1 -- 172.21.15.119:0/4135544400 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573ab743de0 con 0x5573ab9caf50 2023-12-07T10:24:43.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.936+0000 7f8082ffd700 1 --2- 172.21.15.119:0/4135544400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5573ab9cb870 0x5573ab9cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:24:43.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.936+0000 7f80837fe700 1 --2- 172.21.15.119:0/4135544400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573ab9caf50 0x5573ab9e0210 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f807409b7e0 tx=0x7f807401a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc7d9c6f635fa8fa server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:43.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.936+0000 7f806a7fc700 1 -- 172.21.15.119:0/4135544400 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807409bf50 con 0x5573ab9caf50 2023-12-07T10:24:43.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.936+0000 7f806a7fc700 1 -- 172.21.15.119:0/4135544400 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f807409b850 con 0x5573ab9caf50 2023-12-07T10:24:43.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.936+0000 7f806a7fc700 1 -- 172.21.15.119:0/4135544400 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80740228e0 con 0x5573ab9caf50 2023-12-07T10:24:43.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.937+0000 7f809d79bb80 1 -- 172.21.15.119:0/4135544400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573ab9caf50 msgr2=0x5573ab9e0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:43.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.937+0000 7f809d79bb80 1 --2- 172.21.15.119:0/4135544400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573ab9caf50 0x5573ab9e0210 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f807409b7e0 tx=0x7f807401a040 comp rx=0 tx=0).stop 2023-12-07T10:24:43.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.937+0000 7f809d79bb80 1 -- 172.21.15.119:0/4135544400 shutdown_connections 2023-12-07T10:24:43.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.937+0000 7f809d79bb80 1 --2- 172.21.15.119:0/4135544400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573ab9cf560 0x5573ab9bbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.937+0000 7f809d79bb80 1 --2- 172.21.15.119:0/4135544400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573ab9caf50 0x5573ab9e0210 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.937+0000 7f809d79bb80 1 --2- 172.21.15.119:0/4135544400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5573ab9cb870 0x5573ab9cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.937+0000 7f809d79bb80 1 -- 172.21.15.119:0/4135544400 >> 172.21.15.119:0/4135544400 conn(0x5573ab8f5360 msgr2=0x5573ab9d3900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:43.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.937+0000 7f809d79bb80 1 -- 172.21.15.119:0/4135544400 shutdown_connections 2023-12-07T10:24:43.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.937+0000 7f809d79bb80 1 -- 172.21.15.119:0/4135544400 wait complete. 2023-12-07T10:24:43.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.938+0000 7f809d79bb80 1 -- start start 2023-12-07T10:24:43.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.938+0000 7f809d79bb80 1 Processor -- start 2023-12-07T10:24:43.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.938+0000 7f809d79bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573ab9caf50 0x5573ab9d9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.938+0000 7f809d79bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5573ab9cb870 0x5573ab9d9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.938+0000 7f809d79bb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573ab9cf560 0x5573ab9da1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.938+0000 7f809d79bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573aba39c40 con 0x5573ab9cb870 2023-12-07T10:24:43.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.938+0000 7f809d79bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573aba39d80 con 0x5573ab9caf50 2023-12-07T10:24:43.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.938+0000 7f809d79bb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573aba39ec0 con 0x5573ab9cf560 2023-12-07T10:24:43.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.938+0000 7f809d79bb80 10 client.0 _ll_register_callbacks cb 0x5573ab8bb940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:43.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.939+0000 7f80837fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573ab9caf50 0x5573ab9d9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.939+0000 7f8083fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573ab9cf560 0x5573ab9da1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.939+0000 7f80837fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573ab9caf50 0x5573ab9d9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:46010/0 (socket says 172.21.15.119:46010) 2023-12-07T10:24:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.939+0000 7f80837fe700 1 -- 172.21.15.119:0/1555165184 learned_addr learned my addr 172.21.15.119:0/1555165184 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.939+0000 7f8082ffd700 1 --2- 172.21.15.119:0/1555165184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5573ab9cb870 0x5573ab9d9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.939+0000 7f80837fe700 1 -- 172.21.15.119:0/1555165184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573ab9cf560 msgr2=0x5573ab9da1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.939+0000 7f80837fe700 1 --2- 172.21.15.119:0/1555165184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573ab9cf560 0x5573ab9da1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.939+0000 7f80837fe700 1 -- 172.21.15.119:0/1555165184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5573ab9cb870 msgr2=0x5573ab9d9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.939+0000 7f80837fe700 1 --2- 172.21.15.119:0/1555165184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5573ab9cb870 0x5573ab9d9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.939+0000 7f80837fe700 1 -- 172.21.15.119:0/1555165184 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80740095f0 con 0x5573ab9caf50 2023-12-07T10:24:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.939+0000 7f80837fe700 1 --2- 172.21.15.119:0/1555165184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573ab9caf50 0x5573ab9d9740 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f807409b550 tx=0x7f807409bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=312a21e54586f292 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.939+0000 7f8062ffd700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:24:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.939+0000 7f8062ffd700 1 -- 172.21.15.119:0/1555165184 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80740228e0 con 0x5573ab9caf50 2023-12-07T10:24:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.939+0000 7f8062ffd700 1 -- 172.21.15.119:0/1555165184 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f807409bab0 con 0x5573ab9caf50 2023-12-07T10:24:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.939+0000 7f809d79bb80 1 -- 172.21.15.119:0/1555165184 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573ab9b9b00 con 0x5573ab9caf50 2023-12-07T10:24:43.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.940+0000 7f8062ffd700 1 -- 172.21.15.119:0/1555165184 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f807409bc70 con 0x5573ab9caf50 2023-12-07T10:24:43.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.940+0000 7f809d79bb80 10 client.27086 fetch_fsmap learned FSMap version 41 2023-12-07T10:24:43.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.940+0000 7f809d79bb80 1 -- 172.21.15.119:0/1555165184 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5573ab72cc60 con 0x5573ab9caf50 2023-12-07T10:24:43.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.940+0000 7f8062ffd700 1 -- 172.21.15.119:0/1555165184 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5573ab9b9b00 con 0x5573ab9caf50 2023-12-07T10:24:43.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.940+0000 7f8062ffd700 1 -- 172.21.15.119:0/1555165184 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 41) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f80740a3730 con 0x5573ab9caf50 2023-12-07T10:24:43.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.940+0000 7f809d79bb80 10 client.27086 fetch_fsmap finished waiting for FSMap version 41 2023-12-07T10:24:43.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.940+0000 7f809d79bb80 10 client.27086 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2023-12-07T10:24:43.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.940+0000 7f809d79bb80 20 client.27086 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:43.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.940+0000 7f809d79bb80 1 --2- 172.21.15.119:0/1555165184 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x5573aba3b300 0x5573aba3d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:43.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.940+0000 7f809d79bb80 4 client.27086 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:24:43.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.940+0000 7f809d79bb80 1 -- 172.21.15.119:0/1555165184 --> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5573ab8d3490 con 0x5573aba3b300 2023-12-07T10:24:43.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.941+0000 7f8082ffd700 1 --2- 172.21.15.119:0/1555165184 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x5573aba3b300 0x5573aba3d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:43.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.941+0000 7f8082ffd700 1 --2- 172.21.15.119:0/1555165184 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x5573aba3b300 0x5573aba3d720 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c1347f6bb6642e3e server_cookie=9787f0805d5fad9f in_seq=0 out_seq=0 2023-12-07T10:24:43.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.941+0000 7f8062ffd700 10 client.27086 ms_handle_connect on v2:172.21.15.153:6834/1295649464 2023-12-07T10:24:43.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.941+0000 7f8062ffd700 1 -- 172.21.15.119:0/1555165184 <== mds.4 v2:172.21.15.153:6834/1295649464 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5573ab8d3490 con 0x5573aba3b300 2023-12-07T10:24:43.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.941+0000 7f8062ffd700 10 client.27086 handle_command_reply: tid=0 2023-12-07T10:24:43.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.941+0000 7f809d79bb80 1 client.27086 shutdown 2023-12-07T10:24:43.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.941+0000 7f809d79bb80 1 -- 172.21.15.119:0/1555165184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573ab9caf50 msgr2=0x5573ab9d9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:43.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.941+0000 7f809d79bb80 1 --2- 172.21.15.119:0/1555165184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573ab9caf50 0x5573ab9d9740 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f807409b550 tx=0x7f807409bf50 comp rx=0 tx=0).stop 2023-12-07T10:24:43.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.941+0000 7f809d79bb80 1 -- 172.21.15.119:0/1555165184 shutdown_connections 2023-12-07T10:24:43.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.941+0000 7f809d79bb80 1 --2- 172.21.15.119:0/1555165184 >> [v2:172.21.15.153:6834/1295649464,v1:172.21.15.153:6835/1295649464] conn(0x5573aba3b300 0x5573aba3d720 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.941+0000 7f809d79bb80 1 --2- 172.21.15.119:0/1555165184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573ab9cf560 0x5573ab9da1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.941+0000 7f809d79bb80 1 --2- 172.21.15.119:0/1555165184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5573ab9caf50 0x5573ab9d9740 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.941+0000 7f8062ffd700 0 client.27086 ms_handle_reset on v2:172.21.15.153:6834/1295649464 2023-12-07T10:24:43.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.941+0000 7f809d79bb80 1 --2- 172.21.15.119:0/1555165184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5573ab9cb870 0x5573ab9d9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.941+0000 7f809d79bb80 1 -- 172.21.15.119:0/1555165184 >> 172.21.15.119:0/1555165184 conn(0x5573ab8f5360 msgr2=0x5573ab9cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:43.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.942+0000 7f809d79bb80 1 -- 172.21.15.119:0/1555165184 shutdown_connections 2023-12-07T10:24:43.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.942+0000 7f809d79bb80 1 -- 172.21.15.119:0/1555165184 wait complete. 2023-12-07T10:24:43.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.942+0000 7f809d79bb80 20 client.27086 trim_cache size 0 max 16384 2023-12-07T10:24:43.948 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:24:43.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.942+0000 7f8089dff700 1 -- 172.21.15.119:0/3355330185 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f806c0a1d00 msgr2=0x7f806c0a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:43.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.942+0000 7f8089dff700 1 --2- 172.21.15.119:0/3355330185 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f806c0a1d00 0x7f806c0a41c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f807c009fd0 tx=0x7f807c00b040 comp rx=0 tx=0).stop 2023-12-07T10:24:43.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.942+0000 7f8089dff700 1 -- 172.21.15.119:0/3355330185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8084101560 msgr2=0x7f808419d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:43.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.942+0000 7f8089dff700 1 --2- 172.21.15.119:0/3355330185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8084101560 0x7f808419d7a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8074005910 tx=0x7f8074004730 comp rx=0 tx=0).stop 2023-12-07T10:24:43.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.942+0000 7f8089dff700 1 -- 172.21.15.119:0/3355330185 shutdown_connections 2023-12-07T10:24:43.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.942+0000 7f8089dff700 1 --2- 172.21.15.119:0/3355330185 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f806c0a1d00 0x7f806c0a41c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.942+0000 7f8089dff700 1 --2- 172.21.15.119:0/3355330185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8084107620 0x7f80841978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.942+0000 7f8089dff700 1 --2- 172.21.15.119:0/3355330185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8084101560 0x7f808419d7a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.942+0000 7f8089dff700 1 --2- 172.21.15.119:0/3355330185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8084101f20 0x7f808419dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:43.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.942+0000 7f8089dff700 1 -- 172.21.15.119:0/3355330185 >> 172.21.15.119:0/3355330185 conn(0x7f80840fcfd0 msgr2=0x7f80840fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:43.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.943+0000 7f8089dff700 1 -- 172.21.15.119:0/3355330185 shutdown_connections 2023-12-07T10:24:43.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:43.943+0000 7f8089dff700 1 -- 172.21.15.119:0/3355330185 wait complete. 2023-12-07T10:24:43.964 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-12-07T10:24:44.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.191+0000 7f3f9539d700 1 -- 172.21.15.119:0/1518463751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f901013d0 msgr2=0x7f3f90101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.191+0000 7f3f9539d700 1 --2- 172.21.15.119:0/1518463751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f901013d0 0x7f3f90101850 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3f84009990 tx=0x7f3f84009ca0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.192+0000 7f3f9539d700 1 -- 172.21.15.119:0/1518463751 shutdown_connections 2023-12-07T10:24:44.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.192+0000 7f3f9539d700 1 --2- 172.21.15.119:0/1518463751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3f90101d90 0x7f3f9010e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.192+0000 7f3f9539d700 1 --2- 172.21.15.119:0/1518463751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f90107470 0x7f3f90107850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.192+0000 7f3f9539d700 1 --2- 172.21.15.119:0/1518463751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f901013d0 0x7f3f90101850 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.192+0000 7f3f9539d700 1 -- 172.21.15.119:0/1518463751 >> 172.21.15.119:0/1518463751 conn(0x7f3f900fcee0 msgr2=0x7f3f900ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:44.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.192+0000 7f3f9539d700 1 -- 172.21.15.119:0/1518463751 shutdown_connections 2023-12-07T10:24:44.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.192+0000 7f3f9539d700 1 -- 172.21.15.119:0/1518463751 wait complete. 2023-12-07T10:24:44.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.193+0000 7f3f9539d700 1 Processor -- start 2023-12-07T10:24:44.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.193+0000 7f3f9539d700 1 -- start start 2023-12-07T10:24:44.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.193+0000 7f3f9539d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f901013d0 0x7f3f9019d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.193+0000 7f3f9539d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f90101d90 0x7f3f9019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.193+0000 7f3f9539d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3f90107470 0x7f3f901976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.193+0000 7f3f9539d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f90197d00 con 0x7f3f90101d90 2023-12-07T10:24:44.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.193+0000 7f3f9539d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f90197e70 con 0x7f3f901013d0 2023-12-07T10:24:44.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.193+0000 7f3f9539d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f90198030 con 0x7f3f90107470 2023-12-07T10:24:44.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.193+0000 7f3f8ffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f901013d0 0x7f3f9019d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.193+0000 7f3f8f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f90101d90 0x7f3f9019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.193+0000 7f3f8ffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f901013d0 0x7f3f9019d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:46030/0 (socket says 172.21.15.119:46030) 2023-12-07T10:24:44.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.193+0000 7f3f94b9c700 1 --2- 172.21.15.119:0/2228673780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3f90107470 0x7f3f901976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.193+0000 7f3f8ffff700 1 -- 172.21.15.119:0/2228673780 learned_addr learned my addr 172.21.15.119:0/2228673780 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:44.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.194+0000 7f3f8ffff700 1 -- 172.21.15.119:0/2228673780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3f90107470 msgr2=0x7f3f901976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.194+0000 7f3f8ffff700 1 --2- 172.21.15.119:0/2228673780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3f90107470 0x7f3f901976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.194+0000 7f3f8ffff700 1 -- 172.21.15.119:0/2228673780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f90101d90 msgr2=0x7f3f9019dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.194+0000 7f3f8ffff700 1 --2- 172.21.15.119:0/2228673780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f90101d90 0x7f3f9019dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.194+0000 7f3f8ffff700 1 -- 172.21.15.119:0/2228673780 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f840095f0 con 0x7f3f901013d0 2023-12-07T10:24:44.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.194+0000 7f3f94b9c700 1 --2- 172.21.15.119:0/2228673780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3f90107470 0x7f3f901976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:44.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.194+0000 7f3f8ffff700 1 --2- 172.21.15.119:0/2228673780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f901013d0 0x7f3f9019d570 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000e9a0 tx=0x7f3f8000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:44.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.194+0000 7f3f8d7fa700 1 -- 172.21.15.119:0/2228673780 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f80009790 con 0x7f3f901013d0 2023-12-07T10:24:44.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.194+0000 7f3f8d7fa700 1 -- 172.21.15.119:0/2228673780 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f80010430 con 0x7f3f901013d0 2023-12-07T10:24:44.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.194+0000 7f3f8d7fa700 1 -- 172.21.15.119:0/2228673780 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f80018680 con 0x7f3f901013d0 2023-12-07T10:24:44.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.194+0000 7f3f9539d700 1 -- 172.21.15.119:0/2228673780 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f90198310 con 0x7f3f901013d0 2023-12-07T10:24:44.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.195+0000 7f3f9539d700 1 -- 172.21.15.119:0/2228673780 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f901a4560 con 0x7f3f901013d0 2023-12-07T10:24:44.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.198+0000 7f3f8d7fa700 1 -- 172.21.15.119:0/2228673780 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f3f800105a0 con 0x7f3f901013d0 2023-12-07T10:24:44.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.199+0000 7f3f8d7fa700 1 --2- 172.21.15.119:0/2228673780 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f3f780a1d00 0x7f3f780a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.199+0000 7f3f8d7fa700 1 -- 172.21.15.119:0/2228673780 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f3f80014070 con 0x7f3f901013d0 2023-12-07T10:24:44.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.199+0000 7f3f8f7fe700 1 --2- 172.21.15.119:0/2228673780 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f3f780a1d00 0x7f3f780a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.200+0000 7f3fa9e83b80 1 Processor -- start 2023-12-07T10:24:44.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.200+0000 7f3fa9e83b80 1 -- start start 2023-12-07T10:24:44.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.200+0000 7f3f8f7fe700 1 --2- 172.21.15.119:0/2228673780 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f3f780a1d00 0x7f3f780a41c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3f90198db0 tx=0x7f3f84005fd0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:44.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.200+0000 7f3fa9e83b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556d216b39e0 0x556d216b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.200+0000 7f3fa9e83b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d216b4300 0x556d216b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.200+0000 7f3fa9e83b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d216b82e0 0x556d216b8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.200+0000 7f3fa9e83b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d216277c0 con 0x556d216b4300 2023-12-07T10:24:44.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.200+0000 7f3fa9e83b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2160e7a0 con 0x556d216b82e0 2023-12-07T10:24:44.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.200+0000 7f3fa9e83b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2160c0c0 con 0x556d216b39e0 2023-12-07T10:24:44.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.201+0000 7f3f8ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556d216b39e0 0x556d216b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.201+0000 7f3f8ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556d216b39e0 0x556d216b3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55846/0 (socket says 172.21.15.119:55846) 2023-12-07T10:24:44.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.201+0000 7f3f8ffff700 1 -- 172.21.15.119:0/3459533963 learned_addr learned my addr 172.21.15.119:0/3459533963 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:44.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.201+0000 7f3f8f7fe700 1 --2- 172.21.15.119:0/3459533963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d216b4300 0x556d216b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.201+0000 7f3f94b9c700 1 --2- 172.21.15.119:0/3459533963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d216b82e0 0x556d216b8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.201+0000 7f3f8ffff700 1 -- 172.21.15.119:0/3459533963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d216b82e0 msgr2=0x556d216b8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:44.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.201+0000 7f3f8ffff700 1 --2- 172.21.15.119:0/3459533963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d216b82e0 0x556d216b8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.201+0000 7f3f8ffff700 1 -- 172.21.15.119:0/3459533963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d216b4300 msgr2=0x556d216b7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:44.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.201+0000 7f3f8ffff700 1 --2- 172.21.15.119:0/3459533963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d216b4300 0x556d216b7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.201+0000 7f3f8ffff700 1 -- 172.21.15.119:0/3459533963 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d21484630 con 0x556d216b39e0 2023-12-07T10:24:44.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.201+0000 7f3f8f7fe700 1 --2- 172.21.15.119:0/3459533963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d216b4300 0x556d216b7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:44.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.201+0000 7f3f8ffff700 1 --2- 172.21.15.119:0/3459533963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556d216b39e0 0x556d216b3dc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3f800a2670 tx=0x7f3f8009a1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=633db52443897672 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:44.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.202+0000 7f3f767fc700 1 -- 172.21.15.119:0/3459533963 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8009e960 con 0x556d216b39e0 2023-12-07T10:24:44.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.202+0000 7f3f767fc700 1 -- 172.21.15.119:0/3459533963 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f800a2ba0 con 0x556d216b39e0 2023-12-07T10:24:44.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.202+0000 7f3f767fc700 1 -- 172.21.15.119:0/3459533963 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8009ad40 con 0x556d216b39e0 2023-12-07T10:24:44.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.202+0000 7f3fa9e83b80 1 -- 172.21.15.119:0/3459533963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556d216b39e0 msgr2=0x556d216b3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.202+0000 7f3fa9e83b80 1 --2- 172.21.15.119:0/3459533963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556d216b39e0 0x556d216b3dc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3f800a2670 tx=0x7f3f8009a1f0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.202+0000 7f3fa9e83b80 1 -- 172.21.15.119:0/3459533963 shutdown_connections 2023-12-07T10:24:44.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.202+0000 7f3fa9e83b80 1 --2- 172.21.15.119:0/3459533963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d216b82e0 0x556d216b8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.202+0000 7f3fa9e83b80 1 --2- 172.21.15.119:0/3459533963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556d216b39e0 0x556d216b3dc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.202+0000 7f3fa9e83b80 1 --2- 172.21.15.119:0/3459533963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d216b4300 0x556d216b7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.202+0000 7f3fa9e83b80 1 -- 172.21.15.119:0/3459533963 >> 172.21.15.119:0/3459533963 conn(0x556d215e2360 msgr2=0x556d216ae170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:44.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.203+0000 7f3fa9e83b80 1 -- 172.21.15.119:0/3459533963 shutdown_connections 2023-12-07T10:24:44.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.203+0000 7f3fa9e83b80 1 -- 172.21.15.119:0/3459533963 wait complete. 2023-12-07T10:24:44.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.203+0000 7f3fa9e83b80 1 -- start start 2023-12-07T10:24:44.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.203+0000 7f3fa9e83b80 1 Processor -- start 2023-12-07T10:24:44.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.204+0000 7f3fa9e83b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d216b39e0 0x556d216bfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.204+0000 7f3fa9e83b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556d216b4300 0x556d216bce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.204+0000 7f3fa9e83b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d216b82e0 0x556d216bd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.204+0000 7f3fa9e83b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d216079b0 con 0x556d216b82e0 2023-12-07T10:24:44.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.204+0000 7f3fa9e83b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d215c1560 con 0x556d216b39e0 2023-12-07T10:24:44.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.204+0000 7f3fa9e83b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d215c1200 con 0x556d216b4300 2023-12-07T10:24:44.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.204+0000 7f3fa9e83b80 10 client.0 _ll_register_callbacks cb 0x556d215a8940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:44.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.204+0000 7f3f8f7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556d216b4300 0x556d216bce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.204+0000 7f3f8f7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556d216b4300 0x556d216bce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55862/0 (socket says 172.21.15.119:55862) 2023-12-07T10:24:44.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.204+0000 7f3f8f7fe700 1 -- 172.21.15.119:0/3581122275 learned_addr learned my addr 172.21.15.119:0/3581122275 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:44.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.204+0000 7f3f8ffff700 1 --2- 172.21.15.119:0/3581122275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d216b39e0 0x556d216bfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.204+0000 7f3f94b9c700 1 --2- 172.21.15.119:0/3581122275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d216b82e0 0x556d216bd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.204+0000 7f3f8f7fe700 1 -- 172.21.15.119:0/3581122275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d216b39e0 msgr2=0x556d216bfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:44.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.204+0000 7f3f8f7fe700 1 --2- 172.21.15.119:0/3581122275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d216b39e0 0x556d216bfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.204+0000 7f3f8f7fe700 1 -- 172.21.15.119:0/3581122275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d216b82e0 msgr2=0x556d216bd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:44.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.204+0000 7f3f8f7fe700 1 --2- 172.21.15.119:0/3581122275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d216b82e0 0x556d216bd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.204+0000 7f3f8f7fe700 1 -- 172.21.15.119:0/3581122275 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f9010eba0 con 0x556d216b4300 2023-12-07T10:24:44.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.205+0000 7f3f8f7fe700 1 --2- 172.21.15.119:0/3581122275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556d216b4300 0x556d216bce10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3f84009990 tx=0x7f3f84009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b23c5a00678309a9 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:44.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.205+0000 7f3f6f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:24:44.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.205+0000 7f3f6f7fe700 1 -- 172.21.15.119:0/3581122275 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f84005680 con 0x556d216b4300 2023-12-07T10:24:44.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.205+0000 7f3fa9e83b80 1 -- 172.21.15.119:0/3581122275 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d216a69a0 con 0x556d216b4300 2023-12-07T10:24:44.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.205+0000 7f3f6f7fe700 1 -- 172.21.15.119:0/3581122275 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f3f8400f460 con 0x556d216b4300 2023-12-07T10:24:44.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.205+0000 7f3f6f7fe700 1 -- 172.21.15.119:0/3581122275 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f84004f00 con 0x556d216b4300 2023-12-07T10:24:44.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.205+0000 7f3fa9e83b80 10 client.27082 fetch_fsmap learned FSMap version 42 2023-12-07T10:24:44.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.205+0000 7f3fa9e83b80 1 -- 172.21.15.119:0/3581122275 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f3f901a4560 con 0x556d216b4300 2023-12-07T10:24:44.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.205+0000 7f3f6f7fe700 1 -- 172.21.15.119:0/3581122275 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556d216a69a0 con 0x556d216b4300 2023-12-07T10:24:44.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.206+0000 7f3f6f7fe700 1 -- 172.21.15.119:0/3581122275 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f3f8402f070 con 0x556d216b4300 2023-12-07T10:24:44.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.206+0000 7f3fa9e83b80 10 client.27082 fetch_fsmap finished waiting for FSMap version 42 2023-12-07T10:24:44.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.206+0000 7f3fa9e83b80 10 client.27082 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-12-07T10:24:44.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.206+0000 7f3fa9e83b80 20 client.27082 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:44.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.206+0000 7f3fa9e83b80 1 --2- 172.21.15.119:0/3581122275 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x556d216cd6e0 0x556d216cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.206+0000 7f3fa9e83b80 4 client.27082 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:24:44.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.206+0000 7f3fa9e83b80 1 -- 172.21.15.119:0/3581122275 --> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d216827a0 con 0x556d216cd6e0 2023-12-07T10:24:44.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.206+0000 7f3f8ffff700 1 --2- 172.21.15.119:0/3581122275 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x556d216cd6e0 0x556d216cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.206+0000 7f3f8ffff700 1 --2- 172.21.15.119:0/3581122275 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x556d216cd6e0 0x556d216cfb00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9f1a1f345b87516b server_cookie=229c72ad8fe89574 in_seq=0 out_seq=0 2023-12-07T10:24:44.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.206+0000 7f3f6f7fe700 10 client.27082 ms_handle_connect on v2:172.21.15.159:6834/2406380792 2023-12-07T10:24:44.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.207+0000 7f3f6f7fe700 1 -- 172.21.15.119:0/3581122275 <== mds.3 v2:172.21.15.159:6834/2406380792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556d216827a0 con 0x556d216cd6e0 2023-12-07T10:24:44.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.207+0000 7f3f6f7fe700 10 client.27082 handle_command_reply: tid=0 2023-12-07T10:24:44.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.207+0000 7f3fa9e83b80 1 client.27082 shutdown 2023-12-07T10:24:44.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.207+0000 7f3fa9e83b80 1 -- 172.21.15.119:0/3581122275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556d216b4300 msgr2=0x556d216bce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.207+0000 7f3fa9e83b80 1 --2- 172.21.15.119:0/3581122275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556d216b4300 0x556d216bce10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3f84009990 tx=0x7f3f84009ca0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.208+0000 7f3fa9e83b80 1 -- 172.21.15.119:0/3581122275 shutdown_connections 2023-12-07T10:24:44.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.208+0000 7f3fa9e83b80 1 --2- 172.21.15.119:0/3581122275 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x556d216cd6e0 0x556d216cfb00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.208+0000 7f3fa9e83b80 1 --2- 172.21.15.119:0/3581122275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d216b82e0 0x556d216bd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.208+0000 7f3fa9e83b80 1 --2- 172.21.15.119:0/3581122275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d216b39e0 0x556d216bfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.208+0000 7f3fa9e83b80 1 --2- 172.21.15.119:0/3581122275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556d216b4300 0x556d216bce10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.208+0000 7f3f6f7fe700 0 client.27082 ms_handle_reset on v2:172.21.15.159:6834/2406380792 2023-12-07T10:24:44.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.208+0000 7f3fa9e83b80 1 -- 172.21.15.119:0/3581122275 >> 172.21.15.119:0/3581122275 conn(0x556d215e2360 msgr2=0x556d216b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:44.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.208+0000 7f3fa9e83b80 1 -- 172.21.15.119:0/3581122275 shutdown_connections 2023-12-07T10:24:44.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.208+0000 7f3fa9e83b80 1 -- 172.21.15.119:0/3581122275 wait complete. 2023-12-07T10:24:44.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.208+0000 7f3fa9e83b80 20 client.27082 trim_cache size 0 max 16384 2023-12-07T10:24:44.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.224+0000 7f3fa9e83b80 1 Processor -- start 2023-12-07T10:24:44.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.225+0000 7f3fa9e83b80 1 -- start start 2023-12-07T10:24:44.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.225+0000 7f3fa9e83b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d216b39e0 0x556d216d4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.225+0000 7f3fa9e83b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556d216b4300 0x556d216c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.225+0000 7f3fa9e83b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d216b82e0 0x556d216c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.225+0000 7f3fa9e83b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d21486920 con 0x556d216b39e0 2023-12-07T10:24:44.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.225+0000 7f3fa9e83b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d216ca030 con 0x556d216b82e0 2023-12-07T10:24:44.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.225+0000 7f3fa9e83b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d216aec60 con 0x556d216b4300 2023-12-07T10:24:44.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.225+0000 7f3f8ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d216b39e0 0x556d216d4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.225+0000 7f3f8ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d216b39e0 0x556d216d4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45074/0 (socket says 172.21.15.119:45074) 2023-12-07T10:24:44.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.225+0000 7f3f8ffff700 1 -- 172.21.15.119:0/2382690751 learned_addr learned my addr 172.21.15.119:0/2382690751 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:44.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.225+0000 7f3f8f7fe700 1 --2- 172.21.15.119:0/2382690751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556d216b4300 0x556d216c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.225+0000 7f3f94b9c700 1 --2- 172.21.15.119:0/2382690751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d216b82e0 0x556d216c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.226+0000 7f3f8ffff700 1 -- 172.21.15.119:0/2382690751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556d216b4300 msgr2=0x556d216c44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:44.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.226+0000 7f3f8ffff700 1 --2- 172.21.15.119:0/2382690751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556d216b4300 0x556d216c44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.226+0000 7f3f8ffff700 1 -- 172.21.15.119:0/2382690751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d216b82e0 msgr2=0x556d216c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:44.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.226+0000 7f3f8ffff700 1 --2- 172.21.15.119:0/2382690751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d216b82e0 0x556d216c4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.226+0000 7f3f8ffff700 1 -- 172.21.15.119:0/2382690751 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d21430de0 con 0x556d216b39e0 2023-12-07T10:24:44.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.226+0000 7f3f94b9c700 1 --2- 172.21.15.119:0/2382690751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d216b82e0 0x556d216c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:44.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:43 smithi119 ceph-mon[97908]: pgmap v132: 129 pgs: 129 active+clean; 139 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 189 op/s 2023-12-07T10:24:44.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:43 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3198493399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:24:44.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.226+0000 7f3f8ffff700 1 --2- 172.21.15.119:0/2382690751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d216b39e0 0x556d216d4420 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3f8009a2e0 tx=0x7f3f80010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89244fbab94dd3a0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:44.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.226+0000 7f3f76ffd700 1 -- 172.21.15.119:0/2382690751 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8009e960 con 0x556d216b39e0 2023-12-07T10:24:44.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.226+0000 7f3f76ffd700 1 -- 172.21.15.119:0/2382690751 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f80098bb0 con 0x556d216b39e0 2023-12-07T10:24:44.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.226+0000 7f3f76ffd700 1 -- 172.21.15.119:0/2382690751 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f800109f0 con 0x556d216b39e0 2023-12-07T10:24:44.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.227+0000 7f3fa9e83b80 1 -- 172.21.15.119:0/2382690751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d216b39e0 msgr2=0x556d216d4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.227+0000 7f3fa9e83b80 1 --2- 172.21.15.119:0/2382690751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d216b39e0 0x556d216d4420 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3f8009a2e0 tx=0x7f3f80010c80 comp rx=0 tx=0).stop 2023-12-07T10:24:44.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.227+0000 7f3fa9e83b80 1 -- 172.21.15.119:0/2382690751 shutdown_connections 2023-12-07T10:24:44.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.227+0000 7f3fa9e83b80 1 --2- 172.21.15.119:0/2382690751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d216b82e0 0x556d216c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.227+0000 7f3fa9e83b80 1 --2- 172.21.15.119:0/2382690751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d216b39e0 0x556d216d4420 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.227+0000 7f3fa9e83b80 1 --2- 172.21.15.119:0/2382690751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556d216b4300 0x556d216c44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.227+0000 7f3fa9e83b80 1 -- 172.21.15.119:0/2382690751 >> 172.21.15.119:0/2382690751 conn(0x556d215e2360 msgr2=0x556d216ade80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.227+0000 7f3fa9e83b80 1 -- 172.21.15.119:0/2382690751 shutdown_connections 2023-12-07T10:24:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.227+0000 7f3fa9e83b80 1 -- 172.21.15.119:0/2382690751 wait complete. 2023-12-07T10:24:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.228+0000 7f3fa9e83b80 1 -- start start 2023-12-07T10:24:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.228+0000 7f3fa9e83b80 1 Processor -- start 2023-12-07T10:24:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.228+0000 7f3fa9e83b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d216b39e0 0x556d216bc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.228+0000 7f3fa9e83b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d216b4300 0x556d216bc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.228+0000 7f3fa9e83b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556d216b82e0 0x556d216bcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.228+0000 7f3fa9e83b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d216c3580 con 0x556d216b4300 2023-12-07T10:24:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.228+0000 7f3fa9e83b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d216c36f0 con 0x556d216b39e0 2023-12-07T10:24:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.228+0000 7f3fa9e83b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d216c38b0 con 0x556d216b82e0 2023-12-07T10:24:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.229+0000 7f3fa9e83b80 10 client.0 _ll_register_callbacks cb 0x556d215a8940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.229+0000 7f3f8ffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d216b39e0 0x556d216bc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.229+0000 7f3f8ffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d216b39e0 0x556d216bc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:46066/0 (socket says 172.21.15.119:46066) 2023-12-07T10:24:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.229+0000 7f3f8ffff700 1 -- 172.21.15.119:0/143556610 learned_addr learned my addr 172.21.15.119:0/143556610 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.229+0000 7f3f8f7fe700 1 --2- 172.21.15.119:0/143556610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d216b4300 0x556d216bc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.229+0000 7f3f94b9c700 1 --2- 172.21.15.119:0/143556610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556d216b82e0 0x556d216bcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.229+0000 7f3f8ffff700 1 -- 172.21.15.119:0/143556610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556d216b82e0 msgr2=0x556d216bcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.229+0000 7f3f8ffff700 1 --2- 172.21.15.119:0/143556610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556d216b82e0 0x556d216bcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.229+0000 7f3f8ffff700 1 -- 172.21.15.119:0/143556610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d216b4300 msgr2=0x556d216bc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.229+0000 7f3f8ffff700 1 --2- 172.21.15.119:0/143556610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d216b4300 0x556d216bc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.229+0000 7f3f8ffff700 1 -- 172.21.15.119:0/143556610 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f880091c0 con 0x556d216b39e0 2023-12-07T10:24:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.229+0000 7f3f94b9c700 1 --2- 172.21.15.119:0/143556610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556d216b82e0 0x556d216bcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:24:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.229+0000 7f3f8ffff700 1 --2- 172.21.15.119:0/143556610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d216b39e0 0x556d216bc350 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3f800cb740 tx=0x7f3f8009a670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94c161f8f50b211b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.229+0000 7f3f6f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:24:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.230+0000 7f3f6f7fe700 1 -- 172.21.15.119:0/143556610 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8009aa70 con 0x556d216b39e0 2023-12-07T10:24:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.230+0000 7f3f6f7fe700 1 -- 172.21.15.119:0/143556610 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f3f8009abd0 con 0x556d216b39e0 2023-12-07T10:24:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.230+0000 7f3fa9e83b80 1 -- 172.21.15.119:0/143556610 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f800a2ba0 con 0x556d216b39e0 2023-12-07T10:24:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.230+0000 7f3f6f7fe700 1 -- 172.21.15.119:0/143556610 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f80099d00 con 0x556d216b39e0 2023-12-07T10:24:44.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.230+0000 7f3fa9e83b80 10 client.27104 fetch_fsmap learned FSMap version 42 2023-12-07T10:24:44.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.230+0000 7f3fa9e83b80 1 -- 172.21.15.119:0/143556610 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x556d21484630 con 0x556d216b39e0 2023-12-07T10:24:44.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.230+0000 7f3f6f7fe700 1 -- 172.21.15.119:0/143556610 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f800a2ba0 con 0x556d216b39e0 2023-12-07T10:24:44.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.230+0000 7f3f6f7fe700 1 -- 172.21.15.119:0/143556610 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 42) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f3f80083bb0 con 0x556d216b39e0 2023-12-07T10:24:44.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.230+0000 7f3fa9e83b80 10 client.27104 fetch_fsmap finished waiting for FSMap version 42 2023-12-07T10:24:44.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.230+0000 7f3fa9e83b80 10 client.27104 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-12-07T10:24:44.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.230+0000 7f3fa9e83b80 20 client.27104 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:44.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.230+0000 7f3fa9e83b80 1 --2- 172.21.15.119:0/143556610 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x556d21728700 0x556d2172aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.230+0000 7f3fa9e83b80 4 client.27104 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:24:44.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.230+0000 7f3fa9e83b80 1 -- 172.21.15.119:0/143556610 --> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556d215cdfd0 con 0x556d21728700 2023-12-07T10:24:44.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.231+0000 7f3f8f7fe700 1 --2- 172.21.15.119:0/143556610 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x556d21728700 0x556d2172aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.231+0000 7f3f8f7fe700 1 --2- 172.21.15.119:0/143556610 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x556d21728700 0x556d2172aaf0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dc139fa8a8631416 server_cookie=788ef453005adce6 in_seq=0 out_seq=0 2023-12-07T10:24:44.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.231+0000 7f3f6f7fe700 10 client.27104 ms_handle_connect on v2:172.21.15.159:6834/2406380792 2023-12-07T10:24:44.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.231+0000 7f3f6f7fe700 1 -- 172.21.15.119:0/143556610 <== mds.3 v2:172.21.15.159:6834/2406380792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+2604 (crc 0 0 0) 0x556d215cdfd0 con 0x556d21728700 2023-12-07T10:24:44.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.231+0000 7f3f6f7fe700 10 client.27104 handle_command_reply: tid=0 2023-12-07T10:24:44.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.231+0000 7f3fa9e83b80 1 client.27104 shutdown 2023-12-07T10:24:44.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.232+0000 7f3fa9e83b80 1 -- 172.21.15.119:0/143556610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d216b39e0 msgr2=0x556d216bc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.232+0000 7f3fa9e83b80 1 --2- 172.21.15.119:0/143556610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d216b39e0 0x556d216bc350 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3f800cb740 tx=0x7f3f8009a670 comp rx=0 tx=0).stop 2023-12-07T10:24:44.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.232+0000 7f3fa9e83b80 1 -- 172.21.15.119:0/143556610 shutdown_connections 2023-12-07T10:24:44.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.232+0000 7f3fa9e83b80 1 --2- 172.21.15.119:0/143556610 >> [v2:172.21.15.159:6834/2406380792,v1:172.21.15.159:6835/2406380792] conn(0x556d21728700 0x556d2172aaf0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.232+0000 7f3fa9e83b80 1 --2- 172.21.15.119:0/143556610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556d216b82e0 0x556d216bcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.232+0000 7f3fa9e83b80 1 --2- 172.21.15.119:0/143556610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556d216b39e0 0x556d216bc350 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.232+0000 7f3fa9e83b80 1 --2- 172.21.15.119:0/143556610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556d216b4300 0x556d216bc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.232+0000 7f3f6f7fe700 0 client.27104 ms_handle_reset on v2:172.21.15.159:6834/2406380792 2023-12-07T10:24:44.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.232+0000 7f3fa9e83b80 1 -- 172.21.15.119:0/143556610 >> 172.21.15.119:0/143556610 conn(0x556d215e2360 msgr2=0x556d216d4420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:44.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.232+0000 7f3fa9e83b80 1 -- 172.21.15.119:0/143556610 shutdown_connections 2023-12-07T10:24:44.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.232+0000 7f3fa9e83b80 1 -- 172.21.15.119:0/143556610 wait complete. 2023-12-07T10:24:44.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.232+0000 7f3fa9e83b80 20 client.27104 trim_cache size 0 max 16384 2023-12-07T10:24:44.239 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-12-07T10:24:44.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:24:44.239 INFO:teuthology.orchestra.run.smithi119.stdout: "damage_type": "backtrace", 2023-12-07T10:24:44.239 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 252908257, 2023-12-07T10:24:44.239 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 2199023256086, 2023-12-07T10:24:44.239 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT" 2023-12-07T10:24:44.239 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:24:44.239 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:24:44.239 INFO:teuthology.orchestra.run.smithi119.stdout: "damage_type": "backtrace", 2023-12-07T10:24:44.239 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 265023097, 2023-12-07T10:24:44.239 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 2199023256080, 2023-12-07T10:24:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT" 2023-12-07T10:24:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:24:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:24:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: "damage_type": "backtrace", 2023-12-07T10:24:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 534473751, 2023-12-07T10:24:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 2199023256076, 2023-12-07T10:24:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG" 2023-12-07T10:24:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:24:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:24:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: "damage_type": "backtrace", 2023-12-07T10:24:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 614771431, 2023-12-07T10:24:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 2199023256084, 2023-12-07T10:24:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR" 2023-12-07T10:24:44.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:24:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:24:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: "damage_type": "backtrace", 2023-12-07T10:24:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 2313634423, 2023-12-07T10:24:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 2199023256079, 2023-12-07T10:24:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR" 2023-12-07T10:24:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:24:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:24:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: "damage_type": "backtrace", 2023-12-07T10:24:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 2672878301, 2023-12-07T10:24:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 2199023256081, 2023-12-07T10:24:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT" 2023-12-07T10:24:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:24:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:24:44.241 INFO:teuthology.orchestra.run.smithi119.stdout: "damage_type": "backtrace", 2023-12-07T10:24:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 2857839008, 2023-12-07T10:24:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 2199023256085, 2023-12-07T10:24:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT" 2023-12-07T10:24:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:24:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:24:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: "damage_type": "backtrace", 2023-12-07T10:24:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 2961137135, 2023-12-07T10:24:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 2199023256078, 2023-12-07T10:24:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL" 2023-12-07T10:24:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:24:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:24:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: "damage_type": "backtrace", 2023-12-07T10:24:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 3282374905, 2023-12-07T10:24:44.242 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 2199023256082, 2023-12-07T10:24:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT" 2023-12-07T10:24:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:24:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:24:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: "damage_type": "backtrace", 2023-12-07T10:24:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 3943609432, 2023-12-07T10:24:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 2199023256077, 2023-12-07T10:24:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG" 2023-12-07T10:24:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:24:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:24:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: "damage_type": "backtrace", 2023-12-07T10:24:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 3995310063, 2023-12-07T10:24:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 2199023256083, 2023-12-07T10:24:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR" 2023-12-07T10:24:44.243 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:24:44.244 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-12-07T10:24:44.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.232+0000 7f3f9539d700 1 -- 172.21.15.119:0/2228673780 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f3f780a1d00 msgr2=0x7f3f780a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.232+0000 7f3f9539d700 1 --2- 172.21.15.119:0/2228673780 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f3f780a1d00 0x7f3f780a41c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3f90198db0 tx=0x7f3f84005fd0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.232+0000 7f3f9539d700 1 -- 172.21.15.119:0/2228673780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f901013d0 msgr2=0x7f3f9019d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.232+0000 7f3f9539d700 1 --2- 172.21.15.119:0/2228673780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f901013d0 0x7f3f9019d570 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000e9a0 tx=0x7f3f8000ecb0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.233+0000 7f3f9539d700 1 -- 172.21.15.119:0/2228673780 shutdown_connections 2023-12-07T10:24:44.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.233+0000 7f3f9539d700 1 --2- 172.21.15.119:0/2228673780 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f3f780a1d00 0x7f3f780a41c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.233+0000 7f3f9539d700 1 --2- 172.21.15.119:0/2228673780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3f90107470 0x7f3f901976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.233+0000 7f3f9539d700 1 --2- 172.21.15.119:0/2228673780 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f901013d0 0x7f3f9019d570 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.233+0000 7f3f9539d700 1 --2- 172.21.15.119:0/2228673780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f90101d90 0x7f3f9019dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.233+0000 7f3f9539d700 1 -- 172.21.15.119:0/2228673780 >> 172.21.15.119:0/2228673780 conn(0x7f3f900fcee0 msgr2=0x7f3f900fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:44.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.233+0000 7f3f9539d700 1 -- 172.21.15.119:0/2228673780 shutdown_connections 2023-12-07T10:24:44.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.233+0000 7f3f9539d700 1 -- 172.21.15.119:0/2228673780 wait complete. 2023-12-07T10:24:44.253 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-07T10:24:44.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:43 smithi159 ceph-mon[134726]: pgmap v132: 129 pgs: 129 active+clean; 139 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 189 op/s 2023-12-07T10:24:44.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:43 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3198493399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:24:44.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:43 smithi153 ceph-mon[133625]: pgmap v132: 129 pgs: 129 active+clean; 139 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 189 op/s 2023-12-07T10:24:44.282 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:43 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3198493399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:24:44.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.477+0000 7f2a001ff700 1 -- 172.21.15.119:0/1823856836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29f8107620 msgr2=0x7f29f8107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.477+0000 7f2a001ff700 1 --2- 172.21.15.119:0/1823856836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29f8107620 0x7f29f8107a00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009920 tx=0x7f29f4009c30 comp rx=0 tx=0).stop 2023-12-07T10:24:44.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.477+0000 7f2a001ff700 1 -- 172.21.15.119:0/1823856836 shutdown_connections 2023-12-07T10:24:44.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.477+0000 7f2a001ff700 1 --2- 172.21.15.119:0/1823856836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29f8101f20 0x7f29f810e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.477+0000 7f2a001ff700 1 --2- 172.21.15.119:0/1823856836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29f8107620 0x7f29f8107a00 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.477+0000 7f2a001ff700 1 --2- 172.21.15.119:0/1823856836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29f8101560 0x7f29f81019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.477+0000 7f2a001ff700 1 -- 172.21.15.119:0/1823856836 >> 172.21.15.119:0/1823856836 conn(0x7f29f80fcfd0 msgr2=0x7f29f80ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:44.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.477+0000 7f2a001ff700 1 -- 172.21.15.119:0/1823856836 shutdown_connections 2023-12-07T10:24:44.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.477+0000 7f2a001ff700 1 -- 172.21.15.119:0/1823856836 wait complete. 2023-12-07T10:24:44.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.478+0000 7f2a001ff700 1 Processor -- start 2023-12-07T10:24:44.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.479+0000 7f2a001ff700 1 -- start start 2023-12-07T10:24:44.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.479+0000 7f2a001ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29f8101560 0x7f29f819f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.479+0000 7f2a001ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29f8101f20 0x7f29f819fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.479+0000 7f2a001ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29f8107620 0x7f29f8199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.479+0000 7f2a001ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f819a0d0 con 0x7f29f8101f20 2023-12-07T10:24:44.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.479+0000 7f2a001ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f819a240 con 0x7f29f8101560 2023-12-07T10:24:44.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.479+0000 7f2a001ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f819a400 con 0x7f29f8107620 2023-12-07T10:24:44.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.480+0000 7f29fdf9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29f8101560 0x7f29f819f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.480+0000 7f29fd79a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29f8101f20 0x7f29f819fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.480+0000 7f29fdf9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29f8101560 0x7f29f819f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:46074/0 (socket says 172.21.15.119:46074) 2023-12-07T10:24:44.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.480+0000 7f29fdf9b700 1 -- 172.21.15.119:0/3537637124 learned_addr learned my addr 172.21.15.119:0/3537637124 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:44.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.480+0000 7f29fe79c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29f8107620 0x7f29f8199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.480+0000 7f29fdf9b700 1 -- 172.21.15.119:0/3537637124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29f8107620 msgr2=0x7f29f8199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.480+0000 7f29fdf9b700 1 --2- 172.21.15.119:0/3537637124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29f8107620 0x7f29f8199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.480+0000 7f29fdf9b700 1 -- 172.21.15.119:0/3537637124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29f8101f20 msgr2=0x7f29f819fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.480+0000 7f29fdf9b700 1 --2- 172.21.15.119:0/3537637124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29f8101f20 0x7f29f819fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.480+0000 7f29fdf9b700 1 -- 172.21.15.119:0/3537637124 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e80095f0 con 0x7f29f8101560 2023-12-07T10:24:44.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.480+0000 7f29fdf9b700 1 --2- 172.21.15.119:0/3537637124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29f8101560 0x7f29f819f940 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f29f4000c00 tx=0x7f29f400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:44.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.480+0000 7f29eeffd700 1 -- 172.21.15.119:0/3537637124 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f401c070 con 0x7f29f8101560 2023-12-07T10:24:44.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.480+0000 7f29eeffd700 1 -- 172.21.15.119:0/3537637124 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29f400ba20 con 0x7f29f8101560 2023-12-07T10:24:44.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.480+0000 7f29eeffd700 1 -- 172.21.15.119:0/3537637124 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f40038e0 con 0x7f29f8101560 2023-12-07T10:24:44.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.481+0000 7f2a001ff700 1 -- 172.21.15.119:0/3537637124 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29f4009520 con 0x7f29f8101560 2023-12-07T10:24:44.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.481+0000 7f2a001ff700 1 -- 172.21.15.119:0/3537637124 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29f819a860 con 0x7f29f8101560 2023-12-07T10:24:44.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.484+0000 7f29eeffd700 1 -- 172.21.15.119:0/3537637124 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f29f4020070 con 0x7f29f8101560 2023-12-07T10:24:44.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.485+0000 7f29eeffd700 1 --2- 172.21.15.119:0/3537637124 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f29e40a1c80 0x7f29e40a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.485+0000 7f29eeffd700 1 -- 172.21.15.119:0/3537637124 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f29f40ce670 con 0x7f29f8101560 2023-12-07T10:24:44.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.485+0000 7f29fd79a700 1 --2- 172.21.15.119:0/3537637124 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f29e40a1c80 0x7f29e40a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.485+0000 7f2a13adeb80 1 Processor -- start 2023-12-07T10:24:44.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.486+0000 7f2a13adeb80 1 -- start start 2023-12-07T10:24:44.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.486+0000 7f2a13adeb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e5b40 0x5592fd0e5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.486+0000 7f2a13adeb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592fd0e6460 0x5592fd0e9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.486+0000 7f2a13adeb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fd0ea150 0x5592fd0ea570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.486+0000 7f2a13adeb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592fd0597c0 con 0x5592fd0ea150 2023-12-07T10:24:44.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.486+0000 7f2a13adeb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592fd0407a0 con 0x5592fd0e6460 2023-12-07T10:24:44.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.486+0000 7f2a13adeb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592fd03e0c0 con 0x5592fd0e5b40 2023-12-07T10:24:44.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.486+0000 7f29fd79a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592fd0e6460 0x5592fd0e9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.486+0000 7f29fdf9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e5b40 0x5592fd0e5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.487+0000 7f29fdf9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e5b40 0x5592fd0e5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55902/0 (socket says 172.21.15.119:55902) 2023-12-07T10:24:44.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.487+0000 7f29fdf9b700 1 -- 172.21.15.119:0/2293204092 learned_addr learned my addr 172.21.15.119:0/2293204092 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:44.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.487+0000 7f29fd79a700 1 --2- 172.21.15.119:0/2293204092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592fd0e6460 0x5592fd0e9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:46076/0 (socket says 172.21.15.119:46076) 2023-12-07T10:24:44.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.487+0000 7f29fd79a700 1 --2- 172.21.15.119:0/3537637124 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f29e40a1c80 0x7f29e40a4140 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f29f819ac30 tx=0x7f29e8009310 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:44.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.487+0000 7f29fe79c700 1 --2- 172.21.15.119:0/2293204092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fd0ea150 0x5592fd0ea570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.487+0000 7f29fdf9b700 1 -- 172.21.15.119:0/2293204092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592fd0e6460 msgr2=0x5592fd0e9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:44.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.487+0000 7f29fdf9b700 1 --2- 172.21.15.119:0/2293204092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592fd0e6460 0x5592fd0e9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.487+0000 7f29fdf9b700 1 -- 172.21.15.119:0/2293204092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fd0ea150 msgr2=0x5592fd0ea570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:44.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.487+0000 7f29fdf9b700 1 --2- 172.21.15.119:0/2293204092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fd0ea150 0x5592fd0ea570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.487+0000 7f29fdf9b700 1 -- 172.21.15.119:0/2293204092 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592fceb6630 con 0x5592fd0e5b40 2023-12-07T10:24:44.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.487+0000 7f29fd79a700 1 --2- 172.21.15.119:0/2293204092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592fd0e6460 0x5592fd0e9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:44.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.487+0000 7f29fdf9b700 1 --2- 172.21.15.119:0/2293204092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e5b40 0x5592fd0e5f20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f29f409adf0 tx=0x7f29f4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abef738c3b5ac805 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:44.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.487+0000 7f29e3fff700 1 -- 172.21.15.119:0/2293204092 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f409b7f0 con 0x5592fd0e5b40 2023-12-07T10:24:44.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.487+0000 7f29e3fff700 1 -- 172.21.15.119:0/2293204092 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29f409b0f0 con 0x5592fd0e5b40 2023-12-07T10:24:44.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.487+0000 7f29e3fff700 1 -- 172.21.15.119:0/2293204092 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4097800 con 0x5592fd0e5b40 2023-12-07T10:24:44.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.488+0000 7f2a13adeb80 1 -- 172.21.15.119:0/2293204092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e5b40 msgr2=0x5592fd0e5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.488+0000 7f2a13adeb80 1 --2- 172.21.15.119:0/2293204092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e5b40 0x5592fd0e5f20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f29f409adf0 tx=0x7f29f4009c30 comp rx=0 tx=0).stop 2023-12-07T10:24:44.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.488+0000 7f2a13adeb80 1 -- 172.21.15.119:0/2293204092 shutdown_connections 2023-12-07T10:24:44.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.488+0000 7f2a13adeb80 1 --2- 172.21.15.119:0/2293204092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fd0ea150 0x5592fd0ea570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.488+0000 7f2a13adeb80 1 --2- 172.21.15.119:0/2293204092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e5b40 0x5592fd0e5f20 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.488+0000 7f2a13adeb80 1 --2- 172.21.15.119:0/2293204092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592fd0e6460 0x5592fd0e9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.488+0000 7f2a13adeb80 1 -- 172.21.15.119:0/2293204092 >> 172.21.15.119:0/2293204092 conn(0x5592fd014360 msgr2=0x5592fd0e0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:44.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.489+0000 7f2a13adeb80 1 -- 172.21.15.119:0/2293204092 shutdown_connections 2023-12-07T10:24:44.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.489+0000 7f2a13adeb80 1 -- 172.21.15.119:0/2293204092 wait complete. 2023-12-07T10:24:44.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.489+0000 7f2a13adeb80 1 -- start start 2023-12-07T10:24:44.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.489+0000 7f2a13adeb80 1 Processor -- start 2023-12-07T10:24:44.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.490+0000 7f2a13adeb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e5b40 0x5592fd0f1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.490+0000 7f2a13adeb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592fd0e6460 0x5592fd0eeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.490+0000 7f2a13adeb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fd0ea150 0x5592fd0ef280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.490+0000 7f2a13adeb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592fd0399b0 con 0x5592fd0ea150 2023-12-07T10:24:44.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.490+0000 7f2a13adeb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592fcff3560 con 0x5592fd0e6460 2023-12-07T10:24:44.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.490+0000 7f2a13adeb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592fcff3200 con 0x5592fd0e5b40 2023-12-07T10:24:44.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.490+0000 7f2a13adeb80 10 client.0 _ll_register_callbacks cb 0x5592fcfda940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:44.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.490+0000 7f29fdf9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e5b40 0x5592fd0f1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.490+0000 7f29fdf9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e5b40 0x5592fd0f1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55908/0 (socket says 172.21.15.119:55908) 2023-12-07T10:24:44.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.490+0000 7f29fdf9b700 1 -- 172.21.15.119:0/2707969768 learned_addr learned my addr 172.21.15.119:0/2707969768 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:44.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.490+0000 7f29fe79c700 1 --2- 172.21.15.119:0/2707969768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fd0ea150 0x5592fd0ef280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.490+0000 7f29fd79a700 1 --2- 172.21.15.119:0/2707969768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592fd0e6460 0x5592fd0eeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.491+0000 7f29fdf9b700 1 -- 172.21.15.119:0/2707969768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592fd0e6460 msgr2=0x5592fd0eeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:44.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.491+0000 7f29fdf9b700 1 --2- 172.21.15.119:0/2707969768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592fd0e6460 0x5592fd0eeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.491+0000 7f29fdf9b700 1 -- 172.21.15.119:0/2707969768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fd0ea150 msgr2=0x5592fd0ef280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:44.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.491+0000 7f29fdf9b700 1 --2- 172.21.15.119:0/2707969768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fd0ea150 0x5592fd0ef280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.491+0000 7f29fdf9b700 1 -- 172.21.15.119:0/2707969768 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e8017070 con 0x5592fd0e5b40 2023-12-07T10:24:44.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.491+0000 7f29fd79a700 1 --2- 172.21.15.119:0/2707969768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592fd0e6460 0x5592fd0eeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:24:44.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.491+0000 7f29fdf9b700 1 --2- 172.21.15.119:0/2707969768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e5b40 0x5592fd0f1910 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f29f40a2ec0 tx=0x7f29f4096d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=772d42531c07677b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:44.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.491+0000 7f29e17fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:24:44.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.491+0000 7f29e17fa700 1 -- 172.21.15.119:0/2707969768 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f40ce470 con 0x5592fd0e5b40 2023-12-07T10:24:44.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.491+0000 7f29e17fa700 1 -- 172.21.15.119:0/2707969768 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f29f40aee40 con 0x5592fd0e5b40 2023-12-07T10:24:44.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.491+0000 7f2a13adeb80 1 -- 172.21.15.119:0/2707969768 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592fd0d8b00 con 0x5592fd0e5b40 2023-12-07T10:24:44.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.491+0000 7f29e17fa700 1 -- 172.21.15.119:0/2707969768 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29f400bb90 con 0x5592fd0e5b40 2023-12-07T10:24:44.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.491+0000 7f2a13adeb80 10 client.27100 fetch_fsmap learned FSMap version 42 2023-12-07T10:24:44.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.491+0000 7f2a13adeb80 1 -- 172.21.15.119:0/2707969768 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f29f819a860 con 0x5592fd0e5b40 2023-12-07T10:24:44.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.492+0000 7f29e17fa700 1 -- 172.21.15.119:0/2707969768 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5592fd0d8b00 con 0x5592fd0e5b40 2023-12-07T10:24:44.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.492+0000 7f29e17fa700 1 -- 172.21.15.119:0/2707969768 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f29f409a700 con 0x5592fd0e5b40 2023-12-07T10:24:44.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.492+0000 7f2a13adeb80 10 client.27100 fetch_fsmap finished waiting for FSMap version 42 2023-12-07T10:24:44.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.492+0000 7f2a13adeb80 10 client.27100 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-07T10:24:44.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.492+0000 7f2a13adeb80 20 client.27100 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:44.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.492+0000 7f2a13adeb80 1 --2- 172.21.15.119:0/2707969768 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x5592fd0feb20 0x5592fd100f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.492+0000 7f2a13adeb80 4 client.27100 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:24:44.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.492+0000 7f2a13adeb80 1 -- 172.21.15.119:0/2707969768 --> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592fd0b47a0 con 0x5592fd0feb20 2023-12-07T10:24:44.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.492+0000 7f29fd79a700 1 --2- 172.21.15.119:0/2707969768 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x5592fd0feb20 0x5592fd100f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.493+0000 7f29fd79a700 1 --2- 172.21.15.119:0/2707969768 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x5592fd0feb20 0x5592fd100f10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb5410dc2af08295 server_cookie=6685f16cad440045 in_seq=0 out_seq=0 2023-12-07T10:24:44.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.493+0000 7f29e17fa700 10 client.27100 ms_handle_connect on v2:172.21.15.159:6836/3106097520 2023-12-07T10:24:44.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.493+0000 7f29e17fa700 1 -- 172.21.15.119:0/2707969768 <== mds.2 v2:172.21.15.159:6836/3106097520 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5592fd0b47a0 con 0x5592fd0feb20 2023-12-07T10:24:44.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.493+0000 7f29e17fa700 10 client.27100 handle_command_reply: tid=0 2023-12-07T10:24:44.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.493+0000 7f2a13adeb80 1 client.27100 shutdown 2023-12-07T10:24:44.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.494+0000 7f2a13adeb80 1 -- 172.21.15.119:0/2707969768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e5b40 msgr2=0x5592fd0f1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.494+0000 7f2a13adeb80 1 --2- 172.21.15.119:0/2707969768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e5b40 0x5592fd0f1910 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f29f40a2ec0 tx=0x7f29f4096d80 comp rx=0 tx=0).stop 2023-12-07T10:24:44.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.494+0000 7f2a13adeb80 1 -- 172.21.15.119:0/2707969768 shutdown_connections 2023-12-07T10:24:44.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.494+0000 7f2a13adeb80 1 --2- 172.21.15.119:0/2707969768 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x5592fd0feb20 0x5592fd100f10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.494+0000 7f2a13adeb80 1 --2- 172.21.15.119:0/2707969768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fd0ea150 0x5592fd0ef280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.494+0000 7f29e17fa700 0 client.27100 ms_handle_reset on v2:172.21.15.159:6836/3106097520 2023-12-07T10:24:44.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.494+0000 7f2a13adeb80 1 --2- 172.21.15.119:0/2707969768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e5b40 0x5592fd0f1910 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.494+0000 7f2a13adeb80 1 --2- 172.21.15.119:0/2707969768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592fd0e6460 0x5592fd0eeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.494+0000 7f2a13adeb80 1 -- 172.21.15.119:0/2707969768 >> 172.21.15.119:0/2707969768 conn(0x5592fd014360 msgr2=0x5592fd0dad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:44.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.494+0000 7f2a13adeb80 1 -- 172.21.15.119:0/2707969768 shutdown_connections 2023-12-07T10:24:44.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.494+0000 7f2a13adeb80 1 -- 172.21.15.119:0/2707969768 wait complete. 2023-12-07T10:24:44.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.494+0000 7f2a13adeb80 20 client.27100 trim_cache size 0 max 16384 2023-12-07T10:24:44.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.511+0000 7f2a13adeb80 1 Processor -- start 2023-12-07T10:24:44.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.511+0000 7f2a13adeb80 1 -- start start 2023-12-07T10:24:44.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.511+0000 7f2a13adeb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fd0e5b40 0x5592fd0f5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.511+0000 7f2a13adeb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e6460 0x5592fd0f64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.511+0000 7f2a13adeb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592fd0ea150 0x5592fd0faa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.511+0000 7f2a13adeb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592fceb8920 con 0x5592fd0e5b40 2023-12-07T10:24:44.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.511+0000 7f2a13adeb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592fd0ef860 con 0x5592fd0ea150 2023-12-07T10:24:44.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.511+0000 7f2a13adeb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592fd0eabe0 con 0x5592fd0e6460 2023-12-07T10:24:44.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.511+0000 7f29fe79c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592fd0ea150 0x5592fd0faa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.511+0000 7f29fdf9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fd0e5b40 0x5592fd0f5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.511+0000 7f29fd79a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e6460 0x5592fd0f64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.511+0000 7f29fe79c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592fd0ea150 0x5592fd0faa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:46086/0 (socket says 172.21.15.119:46086) 2023-12-07T10:24:44.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.511+0000 7f29fd79a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e6460 0x5592fd0f64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55916/0 (socket says 172.21.15.119:55916) 2023-12-07T10:24:44.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.511+0000 7f29fe79c700 1 -- 172.21.15.119:0/1620946698 learned_addr learned my addr 172.21.15.119:0/1620946698 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:44.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.512+0000 7f29fe79c700 1 -- 172.21.15.119:0/1620946698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e6460 msgr2=0x5592fd0f64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:44.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.512+0000 7f29fe79c700 1 --2- 172.21.15.119:0/1620946698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e6460 0x5592fd0f64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.512+0000 7f29fe79c700 1 -- 172.21.15.119:0/1620946698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fd0e5b40 msgr2=0x5592fd0f5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:44.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.512+0000 7f29fe79c700 1 --2- 172.21.15.119:0/1620946698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fd0e5b40 0x5592fd0f5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.512+0000 7f29fe79c700 1 -- 172.21.15.119:0/1620946698 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592fce62de0 con 0x5592fd0ea150 2023-12-07T10:24:44.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.512+0000 7f29fdf9b700 1 --2- 172.21.15.119:0/1620946698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fd0e5b40 0x5592fd0f5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:44.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.512+0000 7f29fe79c700 1 --2- 172.21.15.119:0/1620946698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592fd0ea150 0x5592fd0faa60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f29f0007630 tx=0x7f29f000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc0ea3e55a1da586 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:44.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.512+0000 7f29ecff9700 1 -- 172.21.15.119:0/1620946698 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f000e040 con 0x5592fd0ea150 2023-12-07T10:24:44.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.512+0000 7f29ecff9700 1 -- 172.21.15.119:0/1620946698 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29f000f040 con 0x5592fd0ea150 2023-12-07T10:24:44.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.512+0000 7f29ecff9700 1 -- 172.21.15.119:0/1620946698 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f0017630 con 0x5592fd0ea150 2023-12-07T10:24:44.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.513+0000 7f2a13adeb80 1 -- 172.21.15.119:0/1620946698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592fd0ea150 msgr2=0x5592fd0faa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.513+0000 7f2a13adeb80 1 --2- 172.21.15.119:0/1620946698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592fd0ea150 0x5592fd0faa60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f29f0007630 tx=0x7f29f000b040 comp rx=0 tx=0).stop 2023-12-07T10:24:44.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.513+0000 7f2a13adeb80 1 -- 172.21.15.119:0/1620946698 shutdown_connections 2023-12-07T10:24:44.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.513+0000 7f2a13adeb80 1 --2- 172.21.15.119:0/1620946698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592fd0ea150 0x5592fd0faa60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.513+0000 7f2a13adeb80 1 --2- 172.21.15.119:0/1620946698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fd0e5b40 0x5592fd0f5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.513+0000 7f2a13adeb80 1 --2- 172.21.15.119:0/1620946698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e6460 0x5592fd0f64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.513+0000 7f2a13adeb80 1 -- 172.21.15.119:0/1620946698 >> 172.21.15.119:0/1620946698 conn(0x5592fd014360 msgr2=0x5592fd0e92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:44.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.513+0000 7f2a13adeb80 1 -- 172.21.15.119:0/1620946698 shutdown_connections 2023-12-07T10:24:44.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.513+0000 7f2a13adeb80 1 -- 172.21.15.119:0/1620946698 wait complete. 2023-12-07T10:24:44.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.514+0000 7f2a13adeb80 1 -- start start 2023-12-07T10:24:44.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.514+0000 7f2a13adeb80 1 Processor -- start 2023-12-07T10:24:44.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.515+0000 7f2a13adeb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fd0e5b40 0x5592fd0f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.515+0000 7f2a13adeb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e6460 0x5592fd0eb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.515+0000 7f2a13adeb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592fd0ea150 0x5592fd0eb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.515+0000 7f2a13adeb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592fd0f6700 con 0x5592fd0e5b40 2023-12-07T10:24:44.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.515+0000 7f2a13adeb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592fd0f6870 con 0x5592fd0ea150 2023-12-07T10:24:44.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.515+0000 7f2a13adeb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592fd0ebbb0 con 0x5592fd0e6460 2023-12-07T10:24:44.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.515+0000 7f2a13adeb80 10 client.0 _ll_register_callbacks cb 0x5592fcfda940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:44.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.515+0000 7f29fd79a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e6460 0x5592fd0eb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.515+0000 7f29fd79a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e6460 0x5592fd0eb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55930/0 (socket says 172.21.15.119:55930) 2023-12-07T10:24:44.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.515+0000 7f29fe79c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592fd0ea150 0x5592fd0eb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.515+0000 7f29fd79a700 1 -- 172.21.15.119:0/3156013465 learned_addr learned my addr 172.21.15.119:0/3156013465 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:44.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.515+0000 7f29fdf9b700 1 --2- 172.21.15.119:0/3156013465 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fd0e5b40 0x5592fd0f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.515+0000 7f29fd79a700 1 -- 172.21.15.119:0/3156013465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592fd0ea150 msgr2=0x5592fd0eb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:44.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.515+0000 7f29fd79a700 1 --2- 172.21.15.119:0/3156013465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592fd0ea150 0x5592fd0eb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.515+0000 7f29fd79a700 1 -- 172.21.15.119:0/3156013465 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fd0e5b40 msgr2=0x5592fd0f5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:44.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.515+0000 7f29fd79a700 1 --2- 172.21.15.119:0/3156013465 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fd0e5b40 0x5592fd0f5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.515+0000 7f29fd79a700 1 -- 172.21.15.119:0/3156013465 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29f819a860 con 0x5592fd0e6460 2023-12-07T10:24:44.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.515+0000 7f29fe79c700 1 --2- 172.21.15.119:0/3156013465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592fd0ea150 0x5592fd0eb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:44.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.515+0000 7f29fdf9b700 1 --2- 172.21.15.119:0/3156013465 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fd0e5b40 0x5592fd0f5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:44.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.516+0000 7f29fd79a700 1 --2- 172.21.15.119:0/3156013465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e6460 0x5592fd0eb100 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f29e8013010 tx=0x7f29e80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17919186cd8c7fda server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:44.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.516+0000 7f29e17fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:24:44.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.516+0000 7f29e17fa700 1 -- 172.21.15.119:0/3156013465 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e800ee00 con 0x5592fd0e6460 2023-12-07T10:24:44.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.516+0000 7f2a13adeb80 1 -- 172.21.15.119:0/3156013465 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592fd0e2030 con 0x5592fd0e6460 2023-12-07T10:24:44.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.516+0000 7f29e17fa700 1 -- 172.21.15.119:0/3156013465 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f29e8004020 con 0x5592fd0e6460 2023-12-07T10:24:44.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.516+0000 7f29e17fa700 1 -- 172.21.15.119:0/3156013465 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29e80041e0 con 0x5592fd0e6460 2023-12-07T10:24:44.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.516+0000 7f2a13adeb80 10 client.27106 fetch_fsmap learned FSMap version 42 2023-12-07T10:24:44.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.516+0000 7f2a13adeb80 1 -- 172.21.15.119:0/3156013465 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f29f0009300 con 0x5592fd0e6460 2023-12-07T10:24:44.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.517+0000 7f29e17fa700 1 -- 172.21.15.119:0/3156013465 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5592fd0e2030 con 0x5592fd0e6460 2023-12-07T10:24:44.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.517+0000 7f29e17fa700 1 -- 172.21.15.119:0/3156013465 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f29e801d4a0 con 0x5592fd0e6460 2023-12-07T10:24:44.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.517+0000 7f2a13adeb80 10 client.27106 fetch_fsmap finished waiting for FSMap version 42 2023-12-07T10:24:44.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.517+0000 7f2a13adeb80 10 client.27106 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-07T10:24:44.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.517+0000 7f2a13adeb80 20 client.27106 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:44.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.517+0000 7f2a13adeb80 1 --2- 172.21.15.119:0/3156013465 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x5592fd156330 0x5592fd158750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.517+0000 7f2a13adeb80 4 client.27106 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:24:44.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.517+0000 7f2a13adeb80 1 -- 172.21.15.119:0/3156013465 --> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f29f409b0f0 con 0x5592fd156330 2023-12-07T10:24:44.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.517+0000 7f29fdf9b700 1 --2- 172.21.15.119:0/3156013465 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x5592fd156330 0x5592fd158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.517+0000 7f29fdf9b700 1 --2- 172.21.15.119:0/3156013465 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x5592fd156330 0x5592fd158750 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7e81ecd48dc13e95 server_cookie=e990ce149d53d124 in_seq=0 out_seq=0 2023-12-07T10:24:44.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.517+0000 7f29e17fa700 10 client.27106 ms_handle_connect on v2:172.21.15.159:6836/3106097520 2023-12-07T10:24:44.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.518+0000 7f29e17fa700 1 -- 172.21.15.119:0/3156013465 <== mds.2 v2:172.21.15.159:6836/3106097520 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f29f409b0f0 con 0x5592fd156330 2023-12-07T10:24:44.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.518+0000 7f29e17fa700 10 client.27106 handle_command_reply: tid=0 2023-12-07T10:24:44.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.518+0000 7f2a13adeb80 1 client.27106 shutdown 2023-12-07T10:24:44.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.518+0000 7f2a13adeb80 1 -- 172.21.15.119:0/3156013465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e6460 msgr2=0x5592fd0eb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.518+0000 7f2a13adeb80 1 --2- 172.21.15.119:0/3156013465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e6460 0x5592fd0eb100 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f29e8013010 tx=0x7f29e80047c0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.518+0000 7f2a13adeb80 1 -- 172.21.15.119:0/3156013465 shutdown_connections 2023-12-07T10:24:44.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.518+0000 7f2a13adeb80 1 --2- 172.21.15.119:0/3156013465 >> [v2:172.21.15.159:6836/3106097520,v1:172.21.15.159:6837/3106097520] conn(0x5592fd156330 0x5592fd158750 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.518+0000 7f2a13adeb80 1 --2- 172.21.15.119:0/3156013465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5592fd0ea150 0x5592fd0eb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.518+0000 7f2a13adeb80 1 --2- 172.21.15.119:0/3156013465 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592fd0e5b40 0x5592fd0f5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.518+0000 7f2a13adeb80 1 --2- 172.21.15.119:0/3156013465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592fd0e6460 0x5592fd0eb100 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.518+0000 7f2a13adeb80 1 -- 172.21.15.119:0/3156013465 >> 172.21.15.119:0/3156013465 conn(0x5592fd014360 msgr2=0x5592fd0faa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:44.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.518+0000 7f29e17fa700 0 client.27106 ms_handle_reset on v2:172.21.15.159:6836/3106097520 2023-12-07T10:24:44.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.518+0000 7f2a13adeb80 1 -- 172.21.15.119:0/3156013465 shutdown_connections 2023-12-07T10:24:44.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.518+0000 7f2a13adeb80 1 -- 172.21.15.119:0/3156013465 wait complete. 2023-12-07T10:24:44.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.518+0000 7f2a13adeb80 20 client.27106 trim_cache size 0 max 16384 2023-12-07T10:24:44.527 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:24:44.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.519+0000 7f2a001ff700 1 -- 172.21.15.119:0/3537637124 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f29e40a1c80 msgr2=0x7f29e40a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.519+0000 7f2a001ff700 1 --2- 172.21.15.119:0/3537637124 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f29e40a1c80 0x7f29e40a4140 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f29f819ac30 tx=0x7f29e8009310 comp rx=0 tx=0).stop 2023-12-07T10:24:44.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.519+0000 7f2a001ff700 1 -- 172.21.15.119:0/3537637124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29f8101560 msgr2=0x7f29f819f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.519+0000 7f2a001ff700 1 --2- 172.21.15.119:0/3537637124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29f8101560 0x7f29f819f940 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f29f4000c00 tx=0x7f29f400fa00 comp rx=0 tx=0).stop 2023-12-07T10:24:44.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.519+0000 7f2a001ff700 1 -- 172.21.15.119:0/3537637124 shutdown_connections 2023-12-07T10:24:44.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.519+0000 7f2a001ff700 1 --2- 172.21.15.119:0/3537637124 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f29e40a1c80 0x7f29e40a4140 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.519+0000 7f2a001ff700 1 --2- 172.21.15.119:0/3537637124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29f8107620 0x7f29f8199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.519+0000 7f2a001ff700 1 --2- 172.21.15.119:0/3537637124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29f8101560 0x7f29f819f940 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.519+0000 7f2a001ff700 1 --2- 172.21.15.119:0/3537637124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29f8101f20 0x7f29f819fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.519+0000 7f2a001ff700 1 -- 172.21.15.119:0/3537637124 >> 172.21.15.119:0/3537637124 conn(0x7f29f80fcfd0 msgr2=0x7f29f80fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:44.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.519+0000 7f2a001ff700 1 -- 172.21.15.119:0/3537637124 shutdown_connections 2023-12-07T10:24:44.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.519+0000 7f2a001ff700 1 -- 172.21.15.119:0/3537637124 wait complete. 2023-12-07T10:24:44.541 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-07T10:24:44.553 INFO:tasks.workunit.client.0.smithi119.stdout: 1 262585 7.05 MB/sec execute 57 sec latency 13.665 ms 2023-12-07T10:24:44.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.768+0000 7f07c63ff700 1 -- 172.21.15.119:0/3278227500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c0101560 msgr2=0x7f07c01019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.768+0000 7f07c63ff700 1 --2- 172.21.15.119:0/3278227500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c0101560 0x7f07c01019e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009920 tx=0x7f07b4009c30 comp rx=0 tx=0).stop 2023-12-07T10:24:44.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.768+0000 7f07c63ff700 1 -- 172.21.15.119:0/3278227500 shutdown_connections 2023-12-07T10:24:44.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.768+0000 7f07c63ff700 1 --2- 172.21.15.119:0/3278227500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07c0101f20 0x7f07c010e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.768+0000 7f07c63ff700 1 --2- 172.21.15.119:0/3278227500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07c0107620 0x7f07c0107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.768+0000 7f07c63ff700 1 --2- 172.21.15.119:0/3278227500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c0101560 0x7f07c01019e0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.768+0000 7f07c63ff700 1 -- 172.21.15.119:0/3278227500 >> 172.21.15.119:0/3278227500 conn(0x7f07c00fcfd0 msgr2=0x7f07c00ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:44.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.769+0000 7f07c63ff700 1 -- 172.21.15.119:0/3278227500 shutdown_connections 2023-12-07T10:24:44.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.769+0000 7f07c63ff700 1 -- 172.21.15.119:0/3278227500 wait complete. 2023-12-07T10:24:44.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.770+0000 7f07c63ff700 1 Processor -- start 2023-12-07T10:24:44.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.770+0000 7f07c63ff700 1 -- start start 2023-12-07T10:24:44.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.771+0000 7f07c63ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07c0101560 0x7f07c019f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.771+0000 7f07c63ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c0101f20 0x7f07c019ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.771+0000 7f07c63ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07c0107620 0x7f07c0199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.771+0000 7f07c63ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c019a150 con 0x7f07c0101f20 2023-12-07T10:24:44.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.771+0000 7f07c63ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c019a2c0 con 0x7f07c0107620 2023-12-07T10:24:44.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.771+0000 7f07c63ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c019a480 con 0x7f07c0101560 2023-12-07T10:24:44.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.771+0000 7f07bffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07c0101560 0x7f07c019f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.771+0000 7f07bffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07c0101560 0x7f07c019f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55962/0 (socket says 172.21.15.119:55962) 2023-12-07T10:24:44.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.771+0000 7f07bffff700 1 -- 172.21.15.119:0/2360733770 learned_addr learned my addr 172.21.15.119:0/2360733770 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:44.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.771+0000 7f07c499c700 1 --2- 172.21.15.119:0/2360733770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07c0107620 0x7f07c0199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.772+0000 7f07bf7fe700 1 --2- 172.21.15.119:0/2360733770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c0101f20 0x7f07c019ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.772+0000 7f07bffff700 1 -- 172.21.15.119:0/2360733770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07c0107620 msgr2=0x7f07c0199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.772+0000 7f07bffff700 1 --2- 172.21.15.119:0/2360733770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07c0107620 0x7f07c0199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.772+0000 7f07bffff700 1 -- 172.21.15.119:0/2360733770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c0101f20 msgr2=0x7f07c019ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.772+0000 7f07bffff700 1 --2- 172.21.15.119:0/2360733770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c0101f20 0x7f07c019ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.772+0000 7f07bffff700 1 -- 172.21.15.119:0/2360733770 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b00095f0 con 0x7f07c0101560 2023-12-07T10:24:44.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.772+0000 7f07bffff700 1 --2- 172.21.15.119:0/2360733770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07c0101560 0x7f07c019f9c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f07b000ec10 tx=0x7f07b000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:44.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.772+0000 7f07bd7fa700 1 -- 172.21.15.119:0/2360733770 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b000cc90 con 0x7f07c0101560 2023-12-07T10:24:44.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.772+0000 7f07c63ff700 1 -- 172.21.15.119:0/2360733770 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07b4009520 con 0x7f07c0101560 2023-12-07T10:24:44.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.773+0000 7f07bd7fa700 1 -- 172.21.15.119:0/2360733770 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07b00044e0 con 0x7f07c0101560 2023-12-07T10:24:44.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.773+0000 7f07bd7fa700 1 -- 172.21.15.119:0/2360733770 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b0010430 con 0x7f07c0101560 2023-12-07T10:24:44.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.773+0000 7f07c63ff700 1 -- 172.21.15.119:0/2360733770 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c019a950 con 0x7f07c0101560 2023-12-07T10:24:44.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.776+0000 7f07bd7fa700 1 -- 172.21.15.119:0/2360733770 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f07b0003fb0 con 0x7f07c0101560 2023-12-07T10:24:44.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.777+0000 7f07d9d9ab80 1 Processor -- start 2023-12-07T10:24:44.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.778+0000 7f07bd7fa700 1 --2- 172.21.15.119:0/2360733770 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f07a80a1c80 0x7f07a80a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.778+0000 7f07d9d9ab80 1 -- start start 2023-12-07T10:24:44.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.778+0000 7f07bd7fa700 1 -- 172.21.15.119:0/2360733770 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f07b0014070 con 0x7f07c0101560 2023-12-07T10:24:44.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.778+0000 7f07d9d9ab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937ab40 0x55555937af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.778+0000 7f07c499c700 1 --2- 172.21.15.119:0/2360733770 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f07a80a1c80 0x7f07a80a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.778+0000 7f07d9d9ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55555937b460 0x55555937ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.778+0000 7f07d9d9ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555937f150 0x55555937f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.778+0000 7f07d9d9ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555592ee7c0 con 0x55555937b460 2023-12-07T10:24:44.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.778+0000 7f07d9d9ab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555592d57a0 con 0x55555937ab40 2023-12-07T10:24:44.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.778+0000 7f07d9d9ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555592d30c0 con 0x55555937f150 2023-12-07T10:24:44.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.778+0000 7f07bffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937ab40 0x55555937af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.778+0000 7f07bffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937ab40 0x55555937af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:46120/0 (socket says 172.21.15.119:46120) 2023-12-07T10:24:44.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.778+0000 7f07bffff700 1 -- 172.21.15.119:0/2097682 learned_addr learned my addr 172.21.15.119:0/2097682 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:44.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.779+0000 7f07bf7fe700 1 --2- 172.21.15.119:0/2097682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55555937b460 0x55555937ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.779+0000 7f07c499c700 1 --2- 172.21.15.119:0/2097682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555937f150 0x55555937f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.779+0000 7f07bffff700 1 -- 172.21.15.119:0/2097682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555937f150 msgr2=0x55555937f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:44.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.779+0000 7f07bffff700 1 --2- 172.21.15.119:0/2097682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555937f150 0x55555937f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.779+0000 7f07bffff700 1 -- 172.21.15.119:0/2097682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55555937b460 msgr2=0x55555937ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:44.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.779+0000 7f07bffff700 1 --2- 172.21.15.119:0/2097682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55555937b460 0x55555937ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.779+0000 7f07bffff700 1 -- 172.21.15.119:0/2097682 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55555914b630 con 0x55555937ab40 2023-12-07T10:24:44.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.779+0000 7f07c499c700 1 --2- 172.21.15.119:0/2360733770 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f07a80a1c80 0x7f07a80a4140 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f07b800f540 tx=0x7f07b8009920 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:44.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.779+0000 7f07bffff700 1 --2- 172.21.15.119:0/2097682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937ab40 0x55555937af20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f07b0083fd0 tx=0x7f07b009aad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cd66382eae728e0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:44.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.779+0000 7f07a67fc700 1 -- 172.21.15.119:0/2097682 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b00ba070 con 0x55555937ab40 2023-12-07T10:24:44.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.779+0000 7f07a67fc700 1 -- 172.21.15.119:0/2097682 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07b009b330 con 0x55555937ab40 2023-12-07T10:24:44.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.779+0000 7f07a67fc700 1 -- 172.21.15.119:0/2097682 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b0099080 con 0x55555937ab40 2023-12-07T10:24:44.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.780+0000 7f07d9d9ab80 1 -- 172.21.15.119:0/2097682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937ab40 msgr2=0x55555937af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.780+0000 7f07d9d9ab80 1 --2- 172.21.15.119:0/2097682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937ab40 0x55555937af20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f07b0083fd0 tx=0x7f07b009aad0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.780+0000 7f07d9d9ab80 1 -- 172.21.15.119:0/2097682 shutdown_connections 2023-12-07T10:24:44.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.780+0000 7f07d9d9ab80 1 --2- 172.21.15.119:0/2097682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555937f150 0x55555937f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.780+0000 7f07d9d9ab80 1 --2- 172.21.15.119:0/2097682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937ab40 0x55555937af20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.780+0000 7f07d9d9ab80 1 --2- 172.21.15.119:0/2097682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55555937b460 0x55555937ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.780+0000 7f07d9d9ab80 1 -- 172.21.15.119:0/2097682 >> 172.21.15.119:0/2097682 conn(0x5555592a9360 msgr2=0x555559375130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:44.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.780+0000 7f07d9d9ab80 1 -- 172.21.15.119:0/2097682 shutdown_connections 2023-12-07T10:24:44.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.780+0000 7f07d9d9ab80 1 -- 172.21.15.119:0/2097682 wait complete. 2023-12-07T10:24:44.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.781+0000 7f07d9d9ab80 1 -- start start 2023-12-07T10:24:44.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.781+0000 7f07d9d9ab80 1 Processor -- start 2023-12-07T10:24:44.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.782+0000 7f07d9d9ab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937ab40 0x5555593868a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.782+0000 7f07d9d9ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555937b460 0x555559383b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.782+0000 7f07d9d9ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55555937f150 0x555559384210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.782+0000 7f07d9d9ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555592ce9b0 con 0x55555937f150 2023-12-07T10:24:44.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.782+0000 7f07d9d9ab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555559288560 con 0x55555937ab40 2023-12-07T10:24:44.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.782+0000 7f07d9d9ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555559288200 con 0x55555937b460 2023-12-07T10:24:44.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.782+0000 7f07d9d9ab80 10 client.0 _ll_register_callbacks cb 0x55555926f940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:44.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.782+0000 7f07bffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937ab40 0x5555593868a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.782+0000 7f07bffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937ab40 0x5555593868a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:46130/0 (socket says 172.21.15.119:46130) 2023-12-07T10:24:44.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.782+0000 7f07bffff700 1 -- 172.21.15.119:0/369930000 learned_addr learned my addr 172.21.15.119:0/369930000 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:44.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.782+0000 7f07bf7fe700 1 --2- 172.21.15.119:0/369930000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555937b460 0x555559383b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.782+0000 7f07c499c700 1 --2- 172.21.15.119:0/369930000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55555937f150 0x555559384210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.783+0000 7f07bffff700 1 -- 172.21.15.119:0/369930000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555937b460 msgr2=0x555559383b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:44.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.783+0000 7f07bffff700 1 --2- 172.21.15.119:0/369930000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555937b460 0x555559383b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.783+0000 7f07bffff700 1 -- 172.21.15.119:0/369930000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55555937f150 msgr2=0x555559384210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:44.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.783+0000 7f07bffff700 1 --2- 172.21.15.119:0/369930000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55555937f150 0x555559384210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.783+0000 7f07bffff700 1 -- 172.21.15.119:0/369930000 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c019a950 con 0x55555937ab40 2023-12-07T10:24:44.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.783+0000 7f07bffff700 1 --2- 172.21.15.119:0/369930000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937ab40 0x5555593868a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f07b00a2a90 tx=0x7f07b0003a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=423bba9d35c392eb server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:44.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.783+0000 7f079f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:24:44.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.783+0000 7f079f7fe700 1 -- 172.21.15.119:0/369930000 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b00ba040 con 0x55555937ab40 2023-12-07T10:24:44.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.783+0000 7f079f7fe700 1 -- 172.21.15.119:0/369930000 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f07b009ed80 con 0x55555937ab40 2023-12-07T10:24:44.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.783+0000 7f07d9d9ab80 1 -- 172.21.15.119:0/369930000 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55555936db00 con 0x55555937ab40 2023-12-07T10:24:44.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.783+0000 7f079f7fe700 1 -- 172.21.15.119:0/369930000 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07b009ac30 con 0x55555937ab40 2023-12-07T10:24:44.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.783+0000 7f07d9d9ab80 10 client.27134 fetch_fsmap learned FSMap version 42 2023-12-07T10:24:44.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.783+0000 7f07d9d9ab80 1 -- 172.21.15.119:0/369930000 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5555590f7de0 con 0x55555937ab40 2023-12-07T10:24:44.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.784+0000 7f079f7fe700 1 -- 172.21.15.119:0/369930000 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55555936db00 con 0x55555937ab40 2023-12-07T10:24:44.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.784+0000 7f079f7fe700 1 -- 172.21.15.119:0/369930000 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 42) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f07b009a1b0 con 0x55555937ab40 2023-12-07T10:24:44.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.784+0000 7f07d9d9ab80 10 client.27134 fetch_fsmap finished waiting for FSMap version 42 2023-12-07T10:24:44.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.784+0000 7f07d9d9ab80 10 client.27134 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2023-12-07T10:24:44.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.784+0000 7f07d9d9ab80 20 client.27134 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:44.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.784+0000 7f07d9d9ab80 1 --2- 172.21.15.119:0/369930000 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x555559393ec0 0x5555593962e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.784+0000 7f07d9d9ab80 4 client.27134 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:24:44.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.784+0000 7f07d9d9ab80 1 -- 172.21.15.119:0/369930000 --> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5555593497a0 con 0x555559393ec0 2023-12-07T10:24:44.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.784+0000 7f07bf7fe700 1 --2- 172.21.15.119:0/369930000 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x555559393ec0 0x5555593962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.785+0000 7f07bf7fe700 1 --2- 172.21.15.119:0/369930000 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x555559393ec0 0x5555593962e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28443a040d697dc2 server_cookie=c97c5d16e1990cd0 in_seq=0 out_seq=0 2023-12-07T10:24:44.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.785+0000 7f079f7fe700 10 client.27134 ms_handle_connect on v2:172.21.15.119:6838/1920762602 2023-12-07T10:24:44.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.785+0000 7f079f7fe700 1 -- 172.21.15.119:0/369930000 <== mds.1 v2:172.21.15.119:6838/1920762602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5555593497a0 con 0x555559393ec0 2023-12-07T10:24:44.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.785+0000 7f079f7fe700 10 client.27134 handle_command_reply: tid=0 2023-12-07T10:24:44.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.786+0000 7f07d9d9ab80 1 client.27134 shutdown 2023-12-07T10:24:44.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.786+0000 7f07d9d9ab80 1 -- 172.21.15.119:0/369930000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937ab40 msgr2=0x5555593868a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.786+0000 7f07d9d9ab80 1 --2- 172.21.15.119:0/369930000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937ab40 0x5555593868a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f07b00a2a90 tx=0x7f07b0003a10 comp rx=0 tx=0).stop 2023-12-07T10:24:44.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.786+0000 7f07d9d9ab80 1 -- 172.21.15.119:0/369930000 shutdown_connections 2023-12-07T10:24:44.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.786+0000 7f07d9d9ab80 1 --2- 172.21.15.119:0/369930000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55555937f150 0x555559384210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.786+0000 7f07d9d9ab80 1 --2- 172.21.15.119:0/369930000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937ab40 0x5555593868a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.786+0000 7f07d9d9ab80 1 --2- 172.21.15.119:0/369930000 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x555559393ec0 0x5555593962e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.786+0000 7f07d9d9ab80 1 --2- 172.21.15.119:0/369930000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555937b460 0x555559383b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.786+0000 7f07d9d9ab80 1 -- 172.21.15.119:0/369930000 >> 172.21.15.119:0/369930000 conn(0x5555592a9360 msgr2=0x55555936fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:44.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.786+0000 7f079f7fe700 0 client.27134 ms_handle_reset on v2:172.21.15.119:6838/1920762602 2023-12-07T10:24:44.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.786+0000 7f07d9d9ab80 1 -- 172.21.15.119:0/369930000 shutdown_connections 2023-12-07T10:24:44.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.786+0000 7f07d9d9ab80 1 -- 172.21.15.119:0/369930000 wait complete. 2023-12-07T10:24:44.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.786+0000 7f07d9d9ab80 20 client.27134 trim_cache size 0 max 16384 2023-12-07T10:24:44.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.801+0000 7f07d9d9ab80 1 Processor -- start 2023-12-07T10:24:44.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.801+0000 7f07d9d9ab80 1 -- start start 2023-12-07T10:24:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.802+0000 7f07d9d9ab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937ab40 0x55555937f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.802+0000 7f07d9d9ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555937b460 0x55555938b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.802+0000 7f07d9d9ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55555937f150 0x55555938b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.802+0000 7f07d9d9ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55555914d920 con 0x55555937f150 2023-12-07T10:24:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.802+0000 7f07d9d9ab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555559386c50 con 0x55555937ab40 2023-12-07T10:24:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.802+0000 7f07d9d9ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555559384b20 con 0x55555937b460 2023-12-07T10:24:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.802+0000 7f07bffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937ab40 0x55555937f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.802+0000 7f07bffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937ab40 0x55555937f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:46134/0 (socket says 172.21.15.119:46134) 2023-12-07T10:24:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.802+0000 7f07bffff700 1 -- 172.21.15.119:0/657079989 learned_addr learned my addr 172.21.15.119:0/657079989 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.802+0000 7f07bf7fe700 1 --2- 172.21.15.119:0/657079989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555937b460 0x55555938b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.802+0000 7f07c499c700 1 --2- 172.21.15.119:0/657079989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55555937f150 0x55555938b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.802+0000 7f07bffff700 1 -- 172.21.15.119:0/657079989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555937b460 msgr2=0x55555938b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.802+0000 7f07bffff700 1 --2- 172.21.15.119:0/657079989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555937b460 0x55555938b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.802+0000 7f07bffff700 1 -- 172.21.15.119:0/657079989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55555937f150 msgr2=0x55555938b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:44.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.802+0000 7f07bffff700 1 --2- 172.21.15.119:0/657079989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55555937f150 0x55555938b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.802+0000 7f07bffff700 1 -- 172.21.15.119:0/657079989 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555590e0c60 con 0x55555937ab40 2023-12-07T10:24:44.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.802+0000 7f07bf7fe700 1 --2- 172.21.15.119:0/657079989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555937b460 0x55555938b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:44.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.803+0000 7f07bffff700 1 --2- 172.21.15.119:0/657079989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937ab40 0x55555937f530 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f07b0097130 tx=0x7f07b0004470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edcfcf0595056e29 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:44.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.803+0000 7f07a6ffd700 1 -- 172.21.15.119:0/657079989 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b009a770 con 0x55555937ab40 2023-12-07T10:24:44.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.803+0000 7f07a6ffd700 1 -- 172.21.15.119:0/657079989 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07b009ac30 con 0x55555937ab40 2023-12-07T10:24:44.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.803+0000 7f07a6ffd700 1 -- 172.21.15.119:0/657079989 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b009ae90 con 0x55555937ab40 2023-12-07T10:24:44.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.804+0000 7f07d9d9ab80 1 -- 172.21.15.119:0/657079989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937ab40 msgr2=0x55555937f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.804+0000 7f07d9d9ab80 1 --2- 172.21.15.119:0/657079989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937ab40 0x55555937f530 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f07b0097130 tx=0x7f07b0004470 comp rx=0 tx=0).stop 2023-12-07T10:24:44.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.804+0000 7f07d9d9ab80 1 -- 172.21.15.119:0/657079989 shutdown_connections 2023-12-07T10:24:44.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.804+0000 7f07d9d9ab80 1 --2- 172.21.15.119:0/657079989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55555937f150 0x55555938b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.804+0000 7f07d9d9ab80 1 --2- 172.21.15.119:0/657079989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937ab40 0x55555937f530 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.804+0000 7f07d9d9ab80 1 --2- 172.21.15.119:0/657079989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555937b460 0x55555938b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.804+0000 7f07d9d9ab80 1 -- 172.21.15.119:0/657079989 >> 172.21.15.119:0/657079989 conn(0x5555592a9360 msgr2=0x55555937d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:44.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.804+0000 7f07d9d9ab80 1 -- 172.21.15.119:0/657079989 shutdown_connections 2023-12-07T10:24:44.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.804+0000 7f07d9d9ab80 1 -- 172.21.15.119:0/657079989 wait complete. 2023-12-07T10:24:44.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.804+0000 7f07d9d9ab80 1 -- start start 2023-12-07T10:24:44.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.805+0000 7f07d9d9ab80 1 Processor -- start 2023-12-07T10:24:44.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.805+0000 7f07d9d9ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55555937ab40 0x5555593872c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.806+0000 7f07d9d9ab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937b460 0x555559387800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.806+0000 7f07d9d9ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555937f150 0x555559387d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.806+0000 7f07d9d9ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555593e9370 con 0x55555937ab40 2023-12-07T10:24:44.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.806+0000 7f07d9d9ab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555593e94b0 con 0x55555937b460 2023-12-07T10:24:44.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.806+0000 7f07d9d9ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555593e95f0 con 0x55555937f150 2023-12-07T10:24:44.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.806+0000 7f07d9d9ab80 10 client.0 _ll_register_callbacks cb 0x55555926f940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:44.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.806+0000 7f07bf7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937b460 0x555559387800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.806+0000 7f07c499c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555937f150 0x555559387d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.806+0000 7f07bf7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937b460 0x555559387800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:46144/0 (socket says 172.21.15.119:46144) 2023-12-07T10:24:44.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.806+0000 7f07c499c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555937f150 0x555559387d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55992/0 (socket says 172.21.15.119:55992) 2023-12-07T10:24:44.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.806+0000 7f07bffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55555937ab40 0x5555593872c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.806+0000 7f07bf7fe700 1 -- 172.21.15.119:0/1040660579 learned_addr learned my addr 172.21.15.119:0/1040660579 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:44.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.806+0000 7f07bf7fe700 1 -- 172.21.15.119:0/1040660579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555937f150 msgr2=0x555559387d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:44.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.806+0000 7f07bf7fe700 1 --2- 172.21.15.119:0/1040660579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555937f150 0x555559387d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.806+0000 7f07bf7fe700 1 -- 172.21.15.119:0/1040660579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55555937ab40 msgr2=0x5555593872c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:44.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.806+0000 7f07bf7fe700 1 --2- 172.21.15.119:0/1040660579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55555937ab40 0x5555593872c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.806+0000 7f07bf7fe700 1 -- 172.21.15.119:0/1040660579 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b400fe70 con 0x55555937b460 2023-12-07T10:24:44.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.806+0000 7f07bf7fe700 1 --2- 172.21.15.119:0/1040660579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937b460 0x555559387800 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009920 tx=0x7f07b400f670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bd4a4e57bdad1b1 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:44.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.807+0000 7f079f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:24:44.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.807+0000 7f079f7fe700 1 -- 172.21.15.119:0/1040660579 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b4005310 con 0x55555937b460 2023-12-07T10:24:44.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.807+0000 7f07d9d9ab80 1 -- 172.21.15.119:0/1040660579 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07b009b330 con 0x55555937b460 2023-12-07T10:24:44.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.807+0000 7f079f7fe700 1 -- 172.21.15.119:0/1040660579 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f07b400f8f0 con 0x55555937b460 2023-12-07T10:24:44.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.807+0000 7f079f7fe700 1 -- 172.21.15.119:0/1040660579 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07b4005470 con 0x55555937b460 2023-12-07T10:24:44.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.807+0000 7f07d9d9ab80 10 client.27140 fetch_fsmap learned FSMap version 42 2023-12-07T10:24:44.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.807+0000 7f07d9d9ab80 1 -- 172.21.15.119:0/1040660579 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5555590f7de0 con 0x55555937b460 2023-12-07T10:24:44.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.807+0000 7f079f7fe700 1 -- 172.21.15.119:0/1040660579 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07b009b330 con 0x55555937b460 2023-12-07T10:24:44.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.807+0000 7f079f7fe700 1 -- 172.21.15.119:0/1040660579 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 42) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f07b4009950 con 0x55555937b460 2023-12-07T10:24:44.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.808+0000 7f07d9d9ab80 10 client.27140 fetch_fsmap finished waiting for FSMap version 42 2023-12-07T10:24:44.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.808+0000 7f07d9d9ab80 10 client.27140 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2023-12-07T10:24:44.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.808+0000 7f07d9d9ab80 20 client.27140 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:44.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.808+0000 7f07d9d9ab80 1 --2- 172.21.15.119:0/1040660579 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x5555593eae20 0x5555593ed240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:44.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.808+0000 7f07d9d9ab80 4 client.27140 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:24:44.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.808+0000 7f07d9d9ab80 1 -- 172.21.15.119:0/1040660579 --> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555559294fd0 con 0x5555593eae20 2023-12-07T10:24:44.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.808+0000 7f07bffff700 1 --2- 172.21.15.119:0/1040660579 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x5555593eae20 0x5555593ed240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:44.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.808+0000 7f07bffff700 1 --2- 172.21.15.119:0/1040660579 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x5555593eae20 0x5555593ed240 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=82fd50157d36789f server_cookie=3c2273d70351c7 in_seq=0 out_seq=0 2023-12-07T10:24:44.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.808+0000 7f079f7fe700 10 client.27140 ms_handle_connect on v2:172.21.15.119:6838/1920762602 2023-12-07T10:24:44.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.808+0000 7f079f7fe700 1 -- 172.21.15.119:0/1040660579 <== mds.1 v2:172.21.15.119:6838/1920762602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555559294fd0 con 0x5555593eae20 2023-12-07T10:24:44.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.808+0000 7f079f7fe700 10 client.27140 handle_command_reply: tid=0 2023-12-07T10:24:44.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.809+0000 7f07d9d9ab80 1 client.27140 shutdown 2023-12-07T10:24:44.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.809+0000 7f07d9d9ab80 1 -- 172.21.15.119:0/1040660579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937b460 msgr2=0x555559387800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.809+0000 7f07d9d9ab80 1 --2- 172.21.15.119:0/1040660579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937b460 0x555559387800 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009920 tx=0x7f07b400f670 comp rx=0 tx=0).stop 2023-12-07T10:24:44.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.809+0000 7f07d9d9ab80 1 -- 172.21.15.119:0/1040660579 shutdown_connections 2023-12-07T10:24:44.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.809+0000 7f07d9d9ab80 1 --2- 172.21.15.119:0/1040660579 >> [v2:172.21.15.119:6838/1920762602,v1:172.21.15.119:6839/1920762602] conn(0x5555593eae20 0x5555593ed240 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.809+0000 7f07d9d9ab80 1 --2- 172.21.15.119:0/1040660579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555937f150 0x555559387d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.809+0000 7f07d9d9ab80 1 --2- 172.21.15.119:0/1040660579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55555937ab40 0x5555593872c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.809+0000 7f07d9d9ab80 1 --2- 172.21.15.119:0/1040660579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55555937b460 0x555559387800 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.809+0000 7f079f7fe700 0 client.27140 ms_handle_reset on v2:172.21.15.119:6838/1920762602 2023-12-07T10:24:44.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.809+0000 7f07d9d9ab80 1 -- 172.21.15.119:0/1040660579 >> 172.21.15.119:0/1040660579 conn(0x5555592a9360 msgr2=0x55555938fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:44.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.809+0000 7f07d9d9ab80 1 -- 172.21.15.119:0/1040660579 shutdown_connections 2023-12-07T10:24:44.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.809+0000 7f07d9d9ab80 1 -- 172.21.15.119:0/1040660579 wait complete. 2023-12-07T10:24:44.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.809+0000 7f07d9d9ab80 20 client.27140 trim_cache size 0 max 16384 2023-12-07T10:24:44.816 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:24:44.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.810+0000 7f07c63ff700 1 -- 172.21.15.119:0/2360733770 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f07a80a1c80 msgr2=0x7f07a80a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.810+0000 7f07c63ff700 1 --2- 172.21.15.119:0/2360733770 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f07a80a1c80 0x7f07a80a4140 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f07b800f540 tx=0x7f07b8009920 comp rx=0 tx=0).stop 2023-12-07T10:24:44.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.810+0000 7f07c63ff700 1 -- 172.21.15.119:0/2360733770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07c0101560 msgr2=0x7f07c019f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:44.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.810+0000 7f07c63ff700 1 --2- 172.21.15.119:0/2360733770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07c0101560 0x7f07c019f9c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f07b000ec10 tx=0x7f07b000ef20 comp rx=0 tx=0).stop 2023-12-07T10:24:44.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.810+0000 7f07c63ff700 1 -- 172.21.15.119:0/2360733770 shutdown_connections 2023-12-07T10:24:44.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.810+0000 7f07c63ff700 1 --2- 172.21.15.119:0/2360733770 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f07a80a1c80 0x7f07a80a4140 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.810+0000 7f07c63ff700 1 --2- 172.21.15.119:0/2360733770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f07c0107620 0x7f07c0199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.810+0000 7f07c63ff700 1 --2- 172.21.15.119:0/2360733770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07c0101560 0x7f07c019f9c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.810+0000 7f07c63ff700 1 --2- 172.21.15.119:0/2360733770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07c0101f20 0x7f07c019ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:44.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.810+0000 7f07c63ff700 1 -- 172.21.15.119:0/2360733770 >> 172.21.15.119:0/2360733770 conn(0x7f07c00fcfd0 msgr2=0x7f07c00fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:44.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.810+0000 7f07c63ff700 1 -- 172.21.15.119:0/2360733770 shutdown_connections 2023-12-07T10:24:44.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:44.810+0000 7f07c63ff700 1 -- 172.21.15.119:0/2360733770 wait complete. 2023-12-07T10:24:44.832 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-07T10:24:45.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.112+0000 7ff75119d700 1 -- 172.21.15.119:0/3231062021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff74c100f00 msgr2=0x7ff74c104f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:45.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.112+0000 7ff75119d700 1 --2- 172.21.15.119:0/3231062021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff74c100f00 0x7ff74c104f40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff740009990 tx=0x7ff740009ca0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.112+0000 7ff75119d700 1 -- 172.21.15.119:0/3231062021 shutdown_connections 2023-12-07T10:24:45.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.112+0000 7ff75119d700 1 --2- 172.21.15.119:0/3231062021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff74c105670 0x7ff74c10d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.112+0000 7ff75119d700 1 --2- 172.21.15.119:0/3231062021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff74c100550 0x7ff74c100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.112+0000 7ff75119d700 1 --2- 172.21.15.119:0/3231062021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff74c100f00 0x7ff74c104f40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.112+0000 7ff75119d700 1 -- 172.21.15.119:0/3231062021 >> 172.21.15.119:0/3231062021 conn(0x7ff74c0fbdd0 msgr2=0x7ff74c0fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:45.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.112+0000 7ff75119d700 1 -- 172.21.15.119:0/3231062021 shutdown_connections 2023-12-07T10:24:45.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.112+0000 7ff75119d700 1 -- 172.21.15.119:0/3231062021 wait complete. 2023-12-07T10:24:45.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.113+0000 7ff75119d700 1 Processor -- start 2023-12-07T10:24:45.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.113+0000 7ff75119d700 1 -- start start 2023-12-07T10:24:45.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.114+0000 7ff75119d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff74c100550 0x7ff74c19d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:45.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.114+0000 7ff75119d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff74c105670 0x7ff74c19db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:45.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.114+0000 7ff75119d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff74c1976f0 0x7ff74c197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:45.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.114+0000 7ff75119d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c198170 con 0x7ff74c100550 2023-12-07T10:24:45.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.114+0000 7ff75119d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c1982b0 con 0x7ff74c1976f0 2023-12-07T10:24:45.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.114+0000 7ff75119d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c198470 con 0x7ff74c105670 2023-12-07T10:24:45.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.114+0000 7ff74bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff74c100550 0x7ff74c19d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:45.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.114+0000 7ff74b7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff74c105670 0x7ff74c19db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:45.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.114+0000 7ff74bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff74c100550 0x7ff74c19d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45194/0 (socket says 172.21.15.119:45194) 2023-12-07T10:24:45.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.114+0000 7ff74bfff700 1 -- 172.21.15.119:0/162694196 learned_addr learned my addr 172.21.15.119:0/162694196 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:45.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.114+0000 7ff75099c700 1 --2- 172.21.15.119:0/162694196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff74c1976f0 0x7ff74c197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:45.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.114+0000 7ff74b7fe700 1 -- 172.21.15.119:0/162694196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff74c1976f0 msgr2=0x7ff74c197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:45.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.114+0000 7ff74b7fe700 1 --2- 172.21.15.119:0/162694196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff74c1976f0 0x7ff74c197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.114+0000 7ff74b7fe700 1 -- 172.21.15.119:0/162694196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff74c100550 msgr2=0x7ff74c19d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:45.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.114+0000 7ff74b7fe700 1 --2- 172.21.15.119:0/162694196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff74c100550 0x7ff74c19d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.114+0000 7ff74b7fe700 1 -- 172.21.15.119:0/162694196 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7400095f0 con 0x7ff74c105670 2023-12-07T10:24:45.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.114+0000 7ff74bfff700 1 --2- 172.21.15.119:0/162694196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff74c100550 0x7ff74c19d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:45.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.115+0000 7ff74b7fe700 1 --2- 172.21.15.119:0/162694196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff74c105670 0x7ff74c19db00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff7400098e0 tx=0x7ff740005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:45.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.115+0000 7ff7497fa700 1 -- 172.21.15.119:0/162694196 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74001d070 con 0x7ff74c105670 2023-12-07T10:24:45.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.115+0000 7ff7497fa700 1 -- 172.21.15.119:0/162694196 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff740022470 con 0x7ff74c105670 2023-12-07T10:24:45.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.115+0000 7ff75119d700 1 -- 172.21.15.119:0/162694196 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff74c1a43d0 con 0x7ff74c105670 2023-12-07T10:24:45.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.115+0000 7ff7497fa700 1 -- 172.21.15.119:0/162694196 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff740017650 con 0x7ff74c105670 2023-12-07T10:24:45.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.115+0000 7ff75119d700 1 -- 172.21.15.119:0/162694196 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74c1a4900 con 0x7ff74c105670 2023-12-07T10:24:45.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.118+0000 7ff7497fa700 1 -- 172.21.15.119:0/162694196 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7ff7400225e0 con 0x7ff74c105670 2023-12-07T10:24:45.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.120+0000 7ff7497fa700 1 --2- 172.21.15.119:0/162694196 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff7340a1d00 0x7ff7340a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:45.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.120+0000 7ff765ce0b80 1 Processor -- start 2023-12-07T10:24:45.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.120+0000 7ff7497fa700 1 -- 172.21.15.119:0/162694196 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7ff7400cef10 con 0x7ff74c105670 2023-12-07T10:24:45.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.120+0000 7ff74bfff700 1 --2- 172.21.15.119:0/162694196 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff7340a1d00 0x7ff7340a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:45.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.120+0000 7ff765ce0b80 1 -- start start 2023-12-07T10:24:45.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.121+0000 7ff765ce0b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ca0614e570 0x55ca0614e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:45.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.121+0000 7ff765ce0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca0614ee90 0x55ca06148c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:45.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.121+0000 7ff765ce0b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca06149140 0x55ca06149580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:45.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.121+0000 7ff765ce0b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca060b87c0 con 0x55ca0614e570 2023-12-07T10:24:45.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.121+0000 7ff765ce0b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0609f7a0 con 0x55ca06149140 2023-12-07T10:24:45.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.121+0000 7ff765ce0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0609d0c0 con 0x55ca0614ee90 2023-12-07T10:24:45.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.121+0000 7ff74b7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca0614ee90 0x55ca06148c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:45.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.121+0000 7ff74b7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca0614ee90 0x55ca06148c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:56004/0 (socket says 172.21.15.119:56004) 2023-12-07T10:24:45.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.121+0000 7ff74b7fe700 1 -- 172.21.15.119:0/4091170474 learned_addr learned my addr 172.21.15.119:0/4091170474 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:45.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.121+0000 7ff74bfff700 1 --2- 172.21.15.119:0/162694196 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff7340a1d00 0x7ff7340a41c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff73c011550 tx=0x7ff73c009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:45.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.121+0000 7ff74bfff700 1 --2- 172.21.15.119:0/4091170474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ca0614e570 0x55ca0614e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:45.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.121+0000 7ff75099c700 1 --2- 172.21.15.119:0/4091170474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca06149140 0x55ca06149580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:45.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.121+0000 7ff74b7fe700 1 -- 172.21.15.119:0/4091170474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca06149140 msgr2=0x55ca06149580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:45.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.121+0000 7ff74b7fe700 1 --2- 172.21.15.119:0/4091170474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca06149140 0x55ca06149580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.121+0000 7ff74b7fe700 1 -- 172.21.15.119:0/4091170474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ca0614e570 msgr2=0x55ca0614e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:45.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.121+0000 7ff74b7fe700 1 --2- 172.21.15.119:0/4091170474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ca0614e570 0x55ca0614e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.121+0000 7ff74b7fe700 1 -- 172.21.15.119:0/4091170474 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca05f15630 con 0x55ca0614ee90 2023-12-07T10:24:45.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.121+0000 7ff75099c700 1 --2- 172.21.15.119:0/4091170474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca06149140 0x55ca06149580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:24:45.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.122+0000 7ff74b7fe700 1 --2- 172.21.15.119:0/4091170474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca0614ee90 0x55ca06148c00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff740098400 tx=0x7ff74009b5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9d2374dc913b258 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:45.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.122+0000 7ff7327fc700 1 -- 172.21.15.119:0/4091170474 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74009b990 con 0x55ca0614ee90 2023-12-07T10:24:45.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.122+0000 7ff7327fc700 1 -- 172.21.15.119:0/4091170474 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff74009fae0 con 0x55ca0614ee90 2023-12-07T10:24:45.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.122+0000 7ff7327fc700 1 -- 172.21.15.119:0/4091170474 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7400bb0b0 con 0x55ca0614ee90 2023-12-07T10:24:45.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.122+0000 7ff765ce0b80 1 -- 172.21.15.119:0/4091170474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca0614ee90 msgr2=0x55ca06148c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:45.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.122+0000 7ff765ce0b80 1 --2- 172.21.15.119:0/4091170474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca0614ee90 0x55ca06148c00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff740098400 tx=0x7ff74009b5a0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.123+0000 7ff765ce0b80 1 -- 172.21.15.119:0/4091170474 shutdown_connections 2023-12-07T10:24:45.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.123+0000 7ff765ce0b80 1 --2- 172.21.15.119:0/4091170474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca06149140 0x55ca06149580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.123+0000 7ff765ce0b80 1 --2- 172.21.15.119:0/4091170474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ca0614e570 0x55ca0614e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.123+0000 7ff765ce0b80 1 --2- 172.21.15.119:0/4091170474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca0614ee90 0x55ca06148c00 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.123+0000 7ff765ce0b80 1 -- 172.21.15.119:0/4091170474 >> 172.21.15.119:0/4091170474 conn(0x55ca06073360 msgr2=0x55ca0613f170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:45.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.123+0000 7ff765ce0b80 1 -- 172.21.15.119:0/4091170474 shutdown_connections 2023-12-07T10:24:45.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.123+0000 7ff765ce0b80 1 -- 172.21.15.119:0/4091170474 wait complete. 2023-12-07T10:24:45.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.123+0000 7ff765ce0b80 1 -- start start 2023-12-07T10:24:45.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.124+0000 7ff765ce0b80 1 Processor -- start 2023-12-07T10:24:45.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.124+0000 7ff765ce0b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ca06149140 0x55ca06152260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:45.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.124+0000 7ff765ce0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca0614e570 0x55ca061527a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:45.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.124+0000 7ff765ce0b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca0614ee90 0x55ca06152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:45.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.124+0000 7ff765ce0b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca060989b0 con 0x55ca06149140 2023-12-07T10:24:45.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.124+0000 7ff765ce0b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca06052560 con 0x55ca0614ee90 2023-12-07T10:24:45.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.124+0000 7ff765ce0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca06052200 con 0x55ca0614e570 2023-12-07T10:24:45.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.124+0000 7ff765ce0b80 10 client.0 _ll_register_callbacks cb 0x55ca06039940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:45.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.124+0000 7ff75099c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca0614ee90 0x55ca06152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:45.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.124+0000 7ff75099c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca0614ee90 0x55ca06152ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:46172/0 (socket says 172.21.15.119:46172) 2023-12-07T10:24:45.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.124+0000 7ff75099c700 1 -- 172.21.15.119:0/2446317861 learned_addr learned my addr 172.21.15.119:0/2446317861 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:45.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.124+0000 7ff74bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ca06149140 0x55ca06152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:45.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.124+0000 7ff74b7fe700 1 --2- 172.21.15.119:0/2446317861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca0614e570 0x55ca061527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:45.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.125+0000 7ff75099c700 1 -- 172.21.15.119:0/2446317861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca0614e570 msgr2=0x55ca061527a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:45.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.125+0000 7ff75099c700 1 --2- 172.21.15.119:0/2446317861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca0614e570 0x55ca061527a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.125+0000 7ff75099c700 1 -- 172.21.15.119:0/2446317861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ca06149140 msgr2=0x55ca06152260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:45.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.125+0000 7ff75099c700 1 --2- 172.21.15.119:0/2446317861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ca06149140 0x55ca06152260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.125+0000 7ff75099c700 1 -- 172.21.15.119:0/2446317861 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74c1a4900 con 0x55ca0614ee90 2023-12-07T10:24:45.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.125+0000 7ff74bfff700 1 --2- 172.21.15.119:0/2446317861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ca06149140 0x55ca06152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:24:45.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.125+0000 7ff74b7fe700 1 --2- 172.21.15.119:0/2446317861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca0614e570 0x55ca061527a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:45.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.125+0000 7ff75099c700 1 --2- 172.21.15.119:0/2446317861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca0614ee90 0x55ca06152ce0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff744007c00 tx=0x7ff744007c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a87eac6f06dd5f1c server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:45.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.125+0000 7ff72b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:24:45.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.125+0000 7ff72b7fe700 1 -- 172.21.15.119:0/2446317861 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74400e040 con 0x55ca0614ee90 2023-12-07T10:24:45.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.125+0000 7ff765ce0b80 1 -- 172.21.15.119:0/2446317861 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca061379a0 con 0x55ca0614ee90 2023-12-07T10:24:45.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.125+0000 7ff72b7fe700 1 -- 172.21.15.119:0/2446317861 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7ff744004db0 con 0x55ca0614ee90 2023-12-07T10:24:45.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.126+0000 7ff72b7fe700 1 -- 172.21.15.119:0/2446317861 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff744013930 con 0x55ca0614ee90 2023-12-07T10:24:45.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.126+0000 7ff765ce0b80 10 client.27152 fetch_fsmap learned FSMap version 42 2023-12-07T10:24:45.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.126+0000 7ff765ce0b80 1 -- 172.21.15.119:0/2446317861 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55ca05ec1de0 con 0x55ca0614ee90 2023-12-07T10:24:45.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.126+0000 7ff72b7fe700 1 -- 172.21.15.119:0/2446317861 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ca061379a0 con 0x55ca0614ee90 2023-12-07T10:24:45.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.126+0000 7ff72b7fe700 1 -- 172.21.15.119:0/2446317861 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 42) v1 ==== 7910+0+0 (secure 0 0 0) 0x7ff7440169d0 con 0x55ca0614ee90 2023-12-07T10:24:45.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.126+0000 7ff765ce0b80 10 client.27152 fetch_fsmap finished waiting for FSMap version 42 2023-12-07T10:24:45.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.126+0000 7ff765ce0b80 10 client.27152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:24:45.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.126+0000 7ff765ce0b80 20 client.27152 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:45.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.126+0000 7ff765ce0b80 1 --2- 172.21.15.119:0/2446317861 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55ca06147130 0x55ca06147540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:45.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.126+0000 7ff765ce0b80 4 client.27152 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:24:45.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.126+0000 7ff765ce0b80 1 -- 172.21.15.119:0/2446317861 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca061137a0 con 0x55ca06147130 2023-12-07T10:24:45.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.127+0000 7ff74bfff700 1 --2- 172.21.15.119:0/2446317861 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55ca06147130 0x55ca06147540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:45.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.127+0000 7ff74bfff700 1 --2- 172.21.15.119:0/2446317861 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55ca06147130 0x55ca06147540 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8bb35b6529d06ff server_cookie=bdbdeef88a334ecc in_seq=0 out_seq=0 2023-12-07T10:24:45.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.127+0000 7ff72b7fe700 10 client.27152 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:45.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.128+0000 7ff72b7fe700 1 -- 172.21.15.119:0/2446317861 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ca061137a0 con 0x55ca06147130 2023-12-07T10:24:45.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.128+0000 7ff72b7fe700 10 client.27152 handle_command_reply: tid=0 2023-12-07T10:24:45.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.128+0000 7ff765ce0b80 1 client.27152 shutdown 2023-12-07T10:24:45.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.128+0000 7ff765ce0b80 1 -- 172.21.15.119:0/2446317861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca0614ee90 msgr2=0x55ca06152ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:45.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.128+0000 7ff765ce0b80 1 --2- 172.21.15.119:0/2446317861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca0614ee90 0x55ca06152ce0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff744007c00 tx=0x7ff744007c30 comp rx=0 tx=0).stop 2023-12-07T10:24:45.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.128+0000 7ff765ce0b80 1 -- 172.21.15.119:0/2446317861 shutdown_connections 2023-12-07T10:24:45.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.128+0000 7ff765ce0b80 1 --2- 172.21.15.119:0/2446317861 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55ca06147130 0x55ca06147540 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.128+0000 7ff765ce0b80 1 --2- 172.21.15.119:0/2446317861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca0614ee90 0x55ca06152ce0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.128+0000 7ff765ce0b80 1 --2- 172.21.15.119:0/2446317861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ca06149140 0x55ca06152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.128+0000 7ff765ce0b80 1 --2- 172.21.15.119:0/2446317861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca0614e570 0x55ca061527a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.128+0000 7ff765ce0b80 1 -- 172.21.15.119:0/2446317861 >> 172.21.15.119:0/2446317861 conn(0x55ca06073360 msgr2=0x55ca061437e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:45.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.129+0000 7ff72b7fe700 0 client.27152 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:45.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.129+0000 7ff765ce0b80 1 -- 172.21.15.119:0/2446317861 shutdown_connections 2023-12-07T10:24:45.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.129+0000 7ff765ce0b80 1 -- 172.21.15.119:0/2446317861 wait complete. 2023-12-07T10:24:45.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.129+0000 7ff765ce0b80 20 client.27152 trim_cache size 0 max 16384 2023-12-07T10:24:45.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.144+0000 7ff765ce0b80 1 Processor -- start 2023-12-07T10:24:45.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.144+0000 7ff765ce0b80 1 -- start start 2023-12-07T10:24:45.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.145+0000 7ff765ce0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca06149140 0x55ca0614c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:45.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.145+0000 7ff765ce0b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca0614e570 0x55ca0614c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:45.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.145+0000 7ff765ce0b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ca0614ee90 0x55ca0614d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:45.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.145+0000 7ff765ce0b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca05f17920 con 0x55ca0614ee90 2023-12-07T10:24:45.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.145+0000 7ff765ce0b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0613fc60 con 0x55ca0614e570 2023-12-07T10:24:45.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.145+0000 7ff765ce0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca061137a0 con 0x55ca06149140 2023-12-07T10:24:45.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.145+0000 7ff74b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca0614e570 0x55ca0614c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:45.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.145+0000 7ff74bfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca06149140 0x55ca0614c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:45.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.145+0000 7ff74b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca0614e570 0x55ca0614c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:46180/0 (socket says 172.21.15.119:46180) 2023-12-07T10:24:45.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.145+0000 7ff74b7fe700 1 -- 172.21.15.119:0/516922862 learned_addr learned my addr 172.21.15.119:0/516922862 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:45.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.145+0000 7ff75099c700 1 --2- 172.21.15.119:0/516922862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ca0614ee90 0x55ca0614d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:45.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.145+0000 7ff74b7fe700 1 -- 172.21.15.119:0/516922862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca06149140 msgr2=0x55ca0614c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:45.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.145+0000 7ff74b7fe700 1 --2- 172.21.15.119:0/516922862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca06149140 0x55ca0614c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.145+0000 7ff74b7fe700 1 -- 172.21.15.119:0/516922862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ca0614ee90 msgr2=0x55ca0614d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:45.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.145+0000 7ff74b7fe700 1 --2- 172.21.15.119:0/516922862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ca0614ee90 0x55ca0614d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.145+0000 7ff74b7fe700 1 -- 172.21.15.119:0/516922862 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca05eaac60 con 0x55ca0614e570 2023-12-07T10:24:45.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.145+0000 7ff74bfff700 1 --2- 172.21.15.119:0/516922862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca06149140 0x55ca0614c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:45.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.145+0000 7ff75099c700 1 --2- 172.21.15.119:0/516922862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ca0614ee90 0x55ca0614d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:24:45.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.146+0000 7ff74b7fe700 1 --2- 172.21.15.119:0/516922862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca0614e570 0x55ca0614c5f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff74000c010 tx=0x7ff740009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73ccf5335abb011 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:45.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.146+0000 7ff732ffd700 1 -- 172.21.15.119:0/516922862 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74009b990 con 0x55ca0614e570 2023-12-07T10:24:45.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.146+0000 7ff732ffd700 1 -- 172.21.15.119:0/516922862 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7400a3c90 con 0x55ca0614e570 2023-12-07T10:24:45.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.146+0000 7ff732ffd700 1 -- 172.21.15.119:0/516922862 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7400bb0b0 con 0x55ca0614e570 2023-12-07T10:24:45.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.146+0000 7ff765ce0b80 1 -- 172.21.15.119:0/516922862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca0614e570 msgr2=0x55ca0614c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:45.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.146+0000 7ff765ce0b80 1 --2- 172.21.15.119:0/516922862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca0614e570 0x55ca0614c5f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff74000c010 tx=0x7ff740009ca0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.146+0000 7ff765ce0b80 1 -- 172.21.15.119:0/516922862 shutdown_connections 2023-12-07T10:24:45.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.146+0000 7ff765ce0b80 1 --2- 172.21.15.119:0/516922862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ca0614ee90 0x55ca0614d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.146+0000 7ff765ce0b80 1 --2- 172.21.15.119:0/516922862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca06149140 0x55ca0614c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.146+0000 7ff765ce0b80 1 --2- 172.21.15.119:0/516922862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca0614e570 0x55ca0614c5f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.147+0000 7ff765ce0b80 1 -- 172.21.15.119:0/516922862 >> 172.21.15.119:0/516922862 conn(0x55ca06073360 msgr2=0x55ca06151af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:45.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.147+0000 7ff765ce0b80 1 -- 172.21.15.119:0/516922862 shutdown_connections 2023-12-07T10:24:45.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.147+0000 7ff765ce0b80 1 -- 172.21.15.119:0/516922862 wait complete. 2023-12-07T10:24:45.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.147+0000 7ff765ce0b80 1 -- start start 2023-12-07T10:24:45.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.147+0000 7ff765ce0b80 1 Processor -- start 2023-12-07T10:24:45.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.148+0000 7ff765ce0b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ca06149140 0x55ca061641c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:45.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.148+0000 7ff765ce0b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca0614e570 0x55ca06160fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:45.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.148+0000 7ff765ce0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca0614ee90 0x55ca06161520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:45.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.148+0000 7ff765ce0b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca06161c30 con 0x55ca06149140 2023-12-07T10:24:45.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.148+0000 7ff765ce0b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca06161da0 con 0x55ca0614e570 2023-12-07T10:24:45.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.148+0000 7ff765ce0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca061b35b0 con 0x55ca0614ee90 2023-12-07T10:24:45.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.148+0000 7ff765ce0b80 10 client.0 _ll_register_callbacks cb 0x55ca06039940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:45.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.148+0000 7ff74bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ca06149140 0x55ca061641c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:45.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.148+0000 7ff74b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca0614e570 0x55ca06160fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:45.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.148+0000 7ff74bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ca06149140 0x55ca061641c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45236/0 (socket says 172.21.15.119:45236) 2023-12-07T10:24:45.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.148+0000 7ff74bfff700 1 -- 172.21.15.119:0/205088677 learned_addr learned my addr 172.21.15.119:0/205088677 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:45.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.148+0000 7ff75099c700 1 --2- 172.21.15.119:0/205088677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca0614ee90 0x55ca06161520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:45.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.149+0000 7ff74b7fe700 1 -- 172.21.15.119:0/205088677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca0614ee90 msgr2=0x55ca06161520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:45.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.149+0000 7ff74b7fe700 1 --2- 172.21.15.119:0/205088677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca0614ee90 0x55ca06161520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.149+0000 7ff74b7fe700 1 -- 172.21.15.119:0/205088677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ca06149140 msgr2=0x55ca061641c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:45.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.149+0000 7ff74b7fe700 1 --2- 172.21.15.119:0/205088677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ca06149140 0x55ca061641c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.149+0000 7ff74b7fe700 1 -- 172.21.15.119:0/205088677 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff73c017070 con 0x55ca0614e570 2023-12-07T10:24:45.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.149+0000 7ff74b7fe700 1 --2- 172.21.15.119:0/205088677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca0614e570 0x55ca06160fa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55ca06153670 tx=0x7ff740097d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f99c4cd7228c007 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:45.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.149+0000 7ff72b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:24:45.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.149+0000 7ff72b7fe700 1 -- 172.21.15.119:0/205088677 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74009b990 con 0x55ca0614e570 2023-12-07T10:24:45.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.149+0000 7ff765ce0b80 1 -- 172.21.15.119:0/205088677 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff74009fae0 con 0x55ca0614e570 2023-12-07T10:24:45.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.149+0000 7ff72b7fe700 1 -- 172.21.15.119:0/205088677 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7ff74009ae80 con 0x55ca0614e570 2023-12-07T10:24:45.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.149+0000 7ff72b7fe700 1 -- 172.21.15.119:0/205088677 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff74009b040 con 0x55ca0614e570 2023-12-07T10:24:45.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.149+0000 7ff765ce0b80 10 client.27158 fetch_fsmap learned FSMap version 42 2023-12-07T10:24:45.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.149+0000 7ff765ce0b80 1 -- 172.21.15.119:0/205088677 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55ca05f15630 con 0x55ca0614e570 2023-12-07T10:24:45.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.150+0000 7ff72b7fe700 1 -- 172.21.15.119:0/205088677 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff74009fae0 con 0x55ca0614e570 2023-12-07T10:24:45.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.150+0000 7ff72b7fe700 1 -- 172.21.15.119:0/205088677 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 42) v1 ==== 7910+0+0 (secure 0 0 0) 0x7ff740097780 con 0x55ca0614e570 2023-12-07T10:24:45.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.150+0000 7ff765ce0b80 10 client.27158 fetch_fsmap finished waiting for FSMap version 42 2023-12-07T10:24:45.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.150+0000 7ff765ce0b80 10 client.27158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-07T10:24:45.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.150+0000 7ff765ce0b80 20 client.27158 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:45.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.150+0000 7ff765ce0b80 1 --2- 172.21.15.119:0/205088677 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55ca061b4760 0x55ca061b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:45.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.150+0000 7ff765ce0b80 4 client.27158 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-12-07T10:24:45.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.150+0000 7ff765ce0b80 1 -- 172.21.15.119:0/205088677 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ca0605efd0 con 0x55ca061b4760 2023-12-07T10:24:45.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.150+0000 7ff74bfff700 1 --2- 172.21.15.119:0/205088677 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55ca061b4760 0x55ca061b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:45.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.150+0000 7ff74bfff700 1 --2- 172.21.15.119:0/205088677 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55ca061b4760 0x55ca061b6b50 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2b88a8b38408567 server_cookie=78cb4974f15bdef8 in_seq=0 out_seq=0 2023-12-07T10:24:45.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.151+0000 7ff72b7fe700 10 client.27158 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:45.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.151+0000 7ff72b7fe700 1 -- 172.21.15.119:0/205088677 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ca0605efd0 con 0x55ca061b4760 2023-12-07T10:24:45.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.151+0000 7ff72b7fe700 10 client.27158 handle_command_reply: tid=0 2023-12-07T10:24:45.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.151+0000 7ff765ce0b80 1 client.27158 shutdown 2023-12-07T10:24:45.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.151+0000 7ff765ce0b80 1 -- 172.21.15.119:0/205088677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca0614e570 msgr2=0x55ca06160fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:45.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.151+0000 7ff765ce0b80 1 --2- 172.21.15.119:0/205088677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca0614e570 0x55ca06160fa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55ca06153670 tx=0x7ff740097d80 comp rx=0 tx=0).stop 2023-12-07T10:24:45.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.151+0000 7ff765ce0b80 1 -- 172.21.15.119:0/205088677 shutdown_connections 2023-12-07T10:24:45.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.151+0000 7ff765ce0b80 1 --2- 172.21.15.119:0/205088677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca0614ee90 0x55ca06161520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.151+0000 7ff765ce0b80 1 --2- 172.21.15.119:0/205088677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ca06149140 0x55ca061641c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.151+0000 7ff765ce0b80 1 --2- 172.21.15.119:0/205088677 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55ca061b4760 0x55ca061b6b50 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.151+0000 7ff765ce0b80 1 --2- 172.21.15.119:0/205088677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ca0614e570 0x55ca06160fa0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.151+0000 7ff765ce0b80 1 -- 172.21.15.119:0/205088677 >> 172.21.15.119:0/205088677 conn(0x55ca06073360 msgr2=0x55ca06152a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:45.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.151+0000 7ff72b7fe700 0 client.27158 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:45.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.152+0000 7ff765ce0b80 1 -- 172.21.15.119:0/205088677 shutdown_connections 2023-12-07T10:24:45.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.152+0000 7ff765ce0b80 1 -- 172.21.15.119:0/205088677 wait complete. 2023-12-07T10:24:45.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.152+0000 7ff765ce0b80 20 client.27158 trim_cache size 0 max 16384 2023-12-07T10:24:45.160 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2023-12-07T10:24:45.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.153+0000 7ff75119d700 1 -- 172.21.15.119:0/162694196 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff7340a1d00 msgr2=0x7ff7340a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:45.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.153+0000 7ff75119d700 1 --2- 172.21.15.119:0/162694196 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff7340a1d00 0x7ff7340a41c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff73c011550 tx=0x7ff73c009260 comp rx=0 tx=0).stop 2023-12-07T10:24:45.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.153+0000 7ff75119d700 1 -- 172.21.15.119:0/162694196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff74c105670 msgr2=0x7ff74c19db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:45.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.153+0000 7ff75119d700 1 --2- 172.21.15.119:0/162694196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff74c105670 0x7ff74c19db00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff7400098e0 tx=0x7ff740005740 comp rx=0 tx=0).stop 2023-12-07T10:24:45.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.153+0000 7ff75119d700 1 -- 172.21.15.119:0/162694196 shutdown_connections 2023-12-07T10:24:45.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.153+0000 7ff75119d700 1 --2- 172.21.15.119:0/162694196 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff7340a1d00 0x7ff7340a41c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.153+0000 7ff75119d700 1 --2- 172.21.15.119:0/162694196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff74c1976f0 0x7ff74c197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.153+0000 7ff75119d700 1 --2- 172.21.15.119:0/162694196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff74c100550 0x7ff74c19d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.153+0000 7ff75119d700 1 --2- 172.21.15.119:0/162694196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff74c105670 0x7ff74c19db00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:45.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.153+0000 7ff75119d700 1 -- 172.21.15.119:0/162694196 >> 172.21.15.119:0/162694196 conn(0x7ff74c0fbdd0 msgr2=0x7ff74c0fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:45.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.153+0000 7ff75119d700 1 -- 172.21.15.119:0/162694196 shutdown_connections 2023-12-07T10:24:45.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:24:45.153+0000 7ff75119d700 1 -- 172.21.15.119:0/162694196 wait complete. 2023-12-07T10:24:45.172 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:45 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:24:45.175 ERROR:tasks.fwd_scrub:rank 3 damaged: [{'damage_type': 'backtrace', 'id': 252908257, 'ino': 2199023256086, 'path': '/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT'}, {'damage_type': 'backtrace', 'id': 265023097, 'ino': 2199023256080, 'path': '/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT'}, {'damage_type': 'backtrace', 'id': 534473751, 'ino': 2199023256076, 'path': '/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG'}, {'damage_type': 'backtrace', 'id': 614771431, 'ino': 2199023256084, 'path': '/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR'}, {'damage_type': 'backtrace', 'id': 2313634423, 'ino': 2199023256079, 'path': '/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR'}, {'damage_type': 'backtrace', 'id': 2672878301, 'ino': 2199023256081, 'path': '/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT'}, {'damage_type': 'backtrace', 'id': 2857839008, 'ino': 2199023256085, 'path': '/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT'}, {'damage_type': 'backtrace', 'id': 2961137135, 'ino': 2199023256078, 'path': '/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL'}, {'damage_type': 'backtrace', 'id': 3282374905, 'ino': 2199023256082, 'path': '/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT'}, {'damage_type': 'backtrace', 'id': 3943609432, 'ino': 2199023256077, 'path': '/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG'}, {'damage_type': 'backtrace', 'id': 3995310063, 'ino': 2199023256083, 'path': '/volumes/_nogroup/sv_1/a0552b93-a7a4-4db9-8129-ee198f2aab69/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR'}] 2023-12-07T10:24:45.175 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2023-12-07T10:24:45.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:45 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:24:45.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:45 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:24:45.553 INFO:tasks.workunit.client.0.smithi119.stdout: 1 263456 6.97 MB/sec execute 58 sec latency 19.080 ms 2023-12-07T10:24:46.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:46 smithi159 ceph-mon[134726]: pgmap v133: 129 pgs: 129 active+clean; 139 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 189 op/s 2023-12-07T10:24:46.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:46 smithi153 ceph-mon[133625]: pgmap v133: 129 pgs: 129 active+clean; 139 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 189 op/s 2023-12-07T10:24:46.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:46 smithi119 ceph-mon[97908]: pgmap v133: 129 pgs: 129 active+clean; 139 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 189 op/s 2023-12-07T10:24:46.553 INFO:tasks.workunit.client.0.smithi119.stdout: 1 265870 7.08 MB/sec execute 59 sec latency 17.953 ms 2023-12-07T10:24:47.553 INFO:tasks.workunit.client.0.smithi119.stdout: 1 266974 7.05 MB/sec execute 60 sec latency 12.201 ms 2023-12-07T10:24:48.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:48 smithi159 ceph-mon[134726]: pgmap v134: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 282 op/s 2023-12-07T10:24:48.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:48 smithi153 ceph-mon[133625]: pgmap v134: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 282 op/s 2023-12-07T10:24:48.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:48 smithi119 ceph-mon[97908]: pgmap v134: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 282 op/s 2023-12-07T10:24:48.553 INFO:tasks.workunit.client.0.smithi119.stdout: 1 269423 7.16 MB/sec execute 61 sec latency 10.828 ms 2023-12-07T10:24:49.554 INFO:tasks.workunit.client.0.smithi119.stdout: 1 270425 7.13 MB/sec execute 62 sec latency 19.040 ms 2023-12-07T10:24:50.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:50 smithi153 ceph-mon[133625]: pgmap v135: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 190 op/s 2023-12-07T10:24:50.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:50 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:24:50.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:50 smithi119 ceph-mon[97908]: pgmap v135: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 190 op/s 2023-12-07T10:24:50.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:50 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:24:50.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:50 smithi159 ceph-mon[134726]: pgmap v135: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 190 op/s 2023-12-07T10:24:50.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:50 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:24:50.553 INFO:tasks.workunit.client.0.smithi119.stdout: 1 272959 7.24 MB/sec execute 63 sec latency 11.543 ms 2023-12-07T10:24:51.554 INFO:tasks.workunit.client.0.smithi119.stdout: 1 274075 7.21 MB/sec execute 64 sec latency 21.793 ms 2023-12-07T10:24:52.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:51 smithi119 ceph-mon[97908]: pgmap v136: 129 pgs: 129 active+clean; 151 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 282 op/s 2023-12-07T10:24:52.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:51 smithi159 ceph-mon[134726]: pgmap v136: 129 pgs: 129 active+clean; 151 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 282 op/s 2023-12-07T10:24:52.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:51 smithi153 ceph-mon[133625]: pgmap v136: 129 pgs: 129 active+clean; 151 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 282 op/s 2023-12-07T10:24:52.553 INFO:tasks.workunit.client.0.smithi119.stdout: 1 276513 7.31 MB/sec execute 65 sec latency 19.042 ms 2023-12-07T10:24:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.724+0000 7fae67fff700 1 -- 172.21.15.119:0/2234406771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae60103700 msgr2=0x7fae60103b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.724+0000 7fae67fff700 1 --2- 172.21.15.119:0/2234406771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae60103700 0x7fae60103b80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fae50009990 tx=0x7fae50009ca0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.724+0000 7fae67fff700 1 -- 172.21.15.119:0/2234406771 shutdown_connections 2023-12-07T10:24:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.724+0000 7fae67fff700 1 --2- 172.21.15.119:0/2234406771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae601040c0 0x7fae601105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.724+0000 7fae67fff700 1 --2- 172.21.15.119:0/2234406771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae60109710 0x7fae60109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.724+0000 7fae67fff700 1 --2- 172.21.15.119:0/2234406771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae60103700 0x7fae60103b80 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.724+0000 7fae67fff700 1 -- 172.21.15.119:0/2234406771 >> 172.21.15.119:0/2234406771 conn(0x7fae600ff190 msgr2=0x7fae601015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.724+0000 7fae67fff700 1 -- 172.21.15.119:0/2234406771 shutdown_connections 2023-12-07T10:24:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.724+0000 7fae67fff700 1 -- 172.21.15.119:0/2234406771 wait complete. 2023-12-07T10:24:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.726+0000 7fae67fff700 1 Processor -- start 2023-12-07T10:24:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.726+0000 7fae67fff700 1 -- start start 2023-12-07T10:24:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.727+0000 7fae67fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae60103700 0x7fae6019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.727+0000 7fae67fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae601040c0 0x7fae6019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.727+0000 7fae67fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae60109710 0x7fae60199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.727+0000 7fae67fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae60199fc0 con 0x7fae601040c0 2023-12-07T10:24:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.727+0000 7fae67fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6019a130 con 0x7fae60109710 2023-12-07T10:24:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.727+0000 7fae67fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6019a2f0 con 0x7fae60103700 2023-12-07T10:24:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.727+0000 7fae65d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae60103700 0x7fae6019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.727+0000 7fae65d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae60103700 0x7fae6019f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:56050/0 (socket says 172.21.15.119:56050) 2023-12-07T10:24:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.727+0000 7fae65d9b700 1 -- 172.21.15.119:0/3971562407 learned_addr learned my addr 172.21.15.119:0/3971562407 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.727+0000 7fae6559a700 1 --2- 172.21.15.119:0/3971562407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae601040c0 0x7fae6019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.727+0000 7fae6659c700 1 --2- 172.21.15.119:0/3971562407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae60109710 0x7fae60199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.727+0000 7fae65d9b700 1 -- 172.21.15.119:0/3971562407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae60109710 msgr2=0x7fae60199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.727+0000 7fae65d9b700 1 --2- 172.21.15.119:0/3971562407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae60109710 0x7fae60199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.727+0000 7fae65d9b700 1 -- 172.21.15.119:0/3971562407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae601040c0 msgr2=0x7fae6019fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.727+0000 7fae65d9b700 1 --2- 172.21.15.119:0/3971562407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae601040c0 0x7fae6019fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.727+0000 7fae65d9b700 1 -- 172.21.15.119:0/3971562407 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae500095f0 con 0x7fae60103700 2023-12-07T10:24:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.728+0000 7fae65d9b700 1 --2- 172.21.15.119:0/3971562407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae60103700 0x7fae6019f830 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00e9a0 tx=0x7fae5c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.728+0000 7fae56ffd700 1 -- 172.21.15.119:0/3971562407 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c009790 con 0x7fae60103700 2023-12-07T10:24:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.728+0000 7fae56ffd700 1 -- 172.21.15.119:0/3971562407 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae5c010430 con 0x7fae60103700 2023-12-07T10:24:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.728+0000 7fae56ffd700 1 -- 172.21.15.119:0/3971562407 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c018680 con 0x7fae60103700 2023-12-07T10:24:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.728+0000 7fae67fff700 1 -- 172.21.15.119:0/3971562407 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae6019a5d0 con 0x7fae60103700 2023-12-07T10:24:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.728+0000 7fae67fff700 1 -- 172.21.15.119:0/3971562407 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae601a6820 con 0x7fae60103700 2023-12-07T10:24:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.732+0000 7fae56ffd700 1 -- 172.21.15.119:0/3971562407 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fae5c0105a0 con 0x7fae60103700 2023-12-07T10:24:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.733+0000 7fae7ba21b80 1 Processor -- start 2023-12-07T10:24:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.733+0000 7fae56ffd700 1 --2- 172.21.15.119:0/3971562407 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fae4c0a1d00 0x7fae4c0a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.733+0000 7fae7ba21b80 1 -- start start 2023-12-07T10:24:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.734+0000 7fae56ffd700 1 -- 172.21.15.119:0/3971562407 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fae5c014070 con 0x7fae60103700 2023-12-07T10:24:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.734+0000 7fae6659c700 1 --2- 172.21.15.119:0/3971562407 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fae4c0a1d00 0x7fae4c0a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.734+0000 7fae7ba21b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55798d0cea60 0x55798d0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.734+0000 7fae7ba21b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d0cf380 0x55798d0c9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.734+0000 7fae7ba21b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55798d0c9ad0 0x55798d0c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.734+0000 7fae7ba21b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798d03d760 con 0x55798d0cea60 2023-12-07T10:24:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.734+0000 7fae7ba21b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798d024740 con 0x55798d0cf380 2023-12-07T10:24:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.734+0000 7fae7ba21b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798d022060 con 0x55798d0c9ad0 2023-12-07T10:24:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.734+0000 7fae6659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55798d0c9ad0 0x55798d0c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.734+0000 7fae6659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55798d0c9ad0 0x55798d0c9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:56054/0 (socket says 172.21.15.119:56054) 2023-12-07T10:24:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.734+0000 7fae6659c700 1 -- 172.21.15.119:0/3135403519 learned_addr learned my addr 172.21.15.119:0/3135403519 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.734+0000 7fae65d9b700 1 --2- 172.21.15.119:0/3135403519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55798d0cea60 0x55798d0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.734+0000 7fae6559a700 1 --2- 172.21.15.119:0/3135403519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d0cf380 0x55798d0c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.735+0000 7fae6659c700 1 -- 172.21.15.119:0/3135403519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d0cf380 msgr2=0x55798d0c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.735+0000 7fae6659c700 1 --2- 172.21.15.119:0/3135403519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d0cf380 0x55798d0c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.735+0000 7fae6659c700 1 -- 172.21.15.119:0/3135403519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55798d0cea60 msgr2=0x55798d0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.735+0000 7fae6659c700 1 --2- 172.21.15.119:0/3135403519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55798d0cea60 0x55798d0cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.735+0000 7fae6659c700 1 -- 172.21.15.119:0/3135403519 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55798ce9a5d0 con 0x55798d0c9ad0 2023-12-07T10:24:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.735+0000 7fae65d9b700 1 --2- 172.21.15.119:0/3135403519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55798d0cea60 0x55798d0cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:24:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.735+0000 7fae6659c700 1 --2- 172.21.15.119:0/3971562407 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fae4c0a1d00 0x7fae4c0a41c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fae5800f930 tx=0x7fae5800fcf0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.735+0000 7fae6659c700 1 --2- 172.21.15.119:0/3135403519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55798d0c9ad0 0x55798d0c9f30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fae58018640 tx=0x7fae5801d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=831d6f88769768fe server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.735+0000 7fae4bfff700 1 -- 172.21.15.119:0/3135403519 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae58027070 con 0x55798d0c9ad0 2023-12-07T10:24:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.735+0000 7fae4bfff700 1 -- 172.21.15.119:0/3135403519 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae5800f040 con 0x55798d0c9ad0 2023-12-07T10:24:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.735+0000 7fae4bfff700 1 -- 172.21.15.119:0/3135403519 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5800e040 con 0x55798d0c9ad0 2023-12-07T10:24:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.736+0000 7fae7ba21b80 1 -- 172.21.15.119:0/3135403519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55798d0c9ad0 msgr2=0x55798d0c9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.736+0000 7fae7ba21b80 1 --2- 172.21.15.119:0/3135403519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55798d0c9ad0 0x55798d0c9f30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fae58018640 tx=0x7fae5801d000 comp rx=0 tx=0).stop 2023-12-07T10:24:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.736+0000 7fae7ba21b80 1 -- 172.21.15.119:0/3135403519 shutdown_connections 2023-12-07T10:24:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.736+0000 7fae7ba21b80 1 --2- 172.21.15.119:0/3135403519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55798d0c9ad0 0x55798d0c9f30 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.736+0000 7fae7ba21b80 1 --2- 172.21.15.119:0/3135403519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55798d0cea60 0x55798d0cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.736+0000 7fae7ba21b80 1 --2- 172.21.15.119:0/3135403519 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d0cf380 0x55798d0c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.736+0000 7fae7ba21b80 1 -- 172.21.15.119:0/3135403519 >> 172.21.15.119:0/3135403519 conn(0x55798cff8300 msgr2=0x55798d0c3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.736+0000 7fae7ba21b80 1 -- 172.21.15.119:0/3135403519 shutdown_connections 2023-12-07T10:24:52.739 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.736+0000 7fae7ba21b80 1 -- 172.21.15.119:0/3135403519 wait complete. 2023-12-07T10:24:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.737+0000 7fae7ba21b80 1 -- start start 2023-12-07T10:24:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.737+0000 7fae7ba21b80 1 Processor -- start 2023-12-07T10:24:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.738+0000 7fae7ba21b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d0c9ad0 0x55798d0d27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.738+0000 7fae7ba21b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55798d0cea60 0x55798d0d2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.738+0000 7fae7ba21b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55798d0cf380 0x55798d0d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.738+0000 7fae7ba21b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798d01d950 con 0x55798d0cf380 2023-12-07T10:24:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.738+0000 7fae7ba21b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798cfd7500 con 0x55798d0c9ad0 2023-12-07T10:24:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.738+0000 7fae7ba21b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798cfd71a0 con 0x55798d0cea60 2023-12-07T10:24:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.738+0000 7fae7ba21b80 10 client.0 _ll_register_callbacks cb 0x55798cfbe8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.738+0000 7fae6559a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55798d0cea60 0x55798d0d2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.738+0000 7fae65d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d0c9ad0 0x55798d0d27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.738+0000 7fae6559a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55798d0cea60 0x55798d0d2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:56070/0 (socket says 172.21.15.119:56070) 2023-12-07T10:24:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.738+0000 7fae6559a700 1 -- 172.21.15.119:0/2353106999 learned_addr learned my addr 172.21.15.119:0/2353106999 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.738+0000 7fae6559a700 1 -- 172.21.15.119:0/2353106999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d0c9ad0 msgr2=0x55798d0d27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.738+0000 7fae6659c700 1 --2- 172.21.15.119:0/2353106999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55798d0cf380 0x55798d0d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.738+0000 7fae6559a700 1 --2- 172.21.15.119:0/2353106999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d0c9ad0 0x55798d0d27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.739+0000 7fae6559a700 1 -- 172.21.15.119:0/2353106999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55798d0cf380 msgr2=0x55798d0d3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.739+0000 7fae6559a700 1 --2- 172.21.15.119:0/2353106999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55798d0cf380 0x55798d0d3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.739+0000 7fae6559a700 1 -- 172.21.15.119:0/2353106999 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae601a6820 con 0x55798d0cea60 2023-12-07T10:24:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.739+0000 7fae6659c700 1 --2- 172.21.15.119:0/2353106999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55798d0cf380 0x55798d0d3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:24:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.739+0000 7fae6559a700 1 --2- 172.21.15.119:0/2353106999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55798d0cea60 0x55798d0d2d20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fae50009990 tx=0x7fae50005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34bd9a1244a6ba3f server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.739+0000 7fae497fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:24:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.739+0000 7fae497fa700 1 -- 172.21.15.119:0/2353106999 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5001d070 con 0x55798d0cea60 2023-12-07T10:24:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.739+0000 7fae497fa700 1 -- 172.21.15.119:0/2353106999 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fae50004500 con 0x55798d0cea60 2023-12-07T10:24:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.739+0000 7fae7ba21b80 1 -- 172.21.15.119:0/2353106999 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55798d0bc6f0 con 0x55798d0cea60 2023-12-07T10:24:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.739+0000 7fae497fa700 1 -- 172.21.15.119:0/2353106999 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae500046c0 con 0x55798d0cea60 2023-12-07T10:24:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.739+0000 7fae7ba21b80 10 client.27154 fetch_fsmap learned FSMap version 43 2023-12-07T10:24:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.739+0000 7fae7ba21b80 1 -- 172.21.15.119:0/2353106999 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7fae5800fe70 con 0x55798d0cea60 2023-12-07T10:24:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.740+0000 7fae497fa700 1 -- 172.21.15.119:0/2353106999 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55798d0bc6f0 con 0x55798d0cea60 2023-12-07T10:24:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.740+0000 7fae497fa700 1 -- 172.21.15.119:0/2353106999 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 43) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fae5000f8a0 con 0x55798d0cea60 2023-12-07T10:24:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.740+0000 7fae7ba21b80 10 client.27154 fetch_fsmap finished waiting for FSMap version 43 2023-12-07T10:24:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.740+0000 7fae7ba21b80 10 client.27154 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:24:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.740+0000 7fae7ba21b80 20 client.27154 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.740+0000 7fae7ba21b80 1 --2- 172.21.15.119:0/2353106999 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55798d0e2ff0 0x55798d0e5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.740+0000 7fae7ba21b80 4 client.27154 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:24:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.740+0000 7fae7ba21b80 1 -- 172.21.15.119:0/2353106999 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55798d098740 con 0x55798d0e2ff0 2023-12-07T10:24:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.740+0000 7fae65d9b700 1 --2- 172.21.15.119:0/2353106999 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55798d0e2ff0 0x55798d0e5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.741+0000 7fae497fa700 10 client.27154 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.741+0000 7fae65d9b700 1 --2- 172.21.15.119:0/2353106999 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55798d0e2ff0 0x55798d0e5410 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62433c574f51a32f server_cookie=76fa2532e5599c07 in_seq=0 out_seq=0 2023-12-07T10:24:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.741+0000 7fae497fa700 1 -- 172.21.15.119:0/2353106999 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55798d098740 con 0x55798d0e2ff0 2023-12-07T10:24:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.741+0000 7fae497fa700 10 client.27154 handle_command_reply: tid=0 2023-12-07T10:24:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.741+0000 7fae7ba21b80 1 client.27154 shutdown 2023-12-07T10:24:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.742+0000 7fae7ba21b80 1 -- 172.21.15.119:0/2353106999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55798d0cea60 msgr2=0x55798d0d2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.742+0000 7fae7ba21b80 1 --2- 172.21.15.119:0/2353106999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55798d0cea60 0x55798d0d2d20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fae50009990 tx=0x7fae50005800 comp rx=0 tx=0).stop 2023-12-07T10:24:52.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.742+0000 7fae7ba21b80 1 -- 172.21.15.119:0/2353106999 shutdown_connections 2023-12-07T10:24:52.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.742+0000 7fae7ba21b80 1 --2- 172.21.15.119:0/2353106999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55798d0cf380 0x55798d0d3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.742+0000 7fae7ba21b80 1 --2- 172.21.15.119:0/2353106999 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55798d0e2ff0 0x55798d0e5410 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.742+0000 7fae7ba21b80 1 --2- 172.21.15.119:0/2353106999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d0c9ad0 0x55798d0d27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.742+0000 7fae7ba21b80 1 --2- 172.21.15.119:0/2353106999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55798d0cea60 0x55798d0d2d20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.742+0000 7fae497fa700 0 client.27154 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:52.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.742+0000 7fae7ba21b80 1 -- 172.21.15.119:0/2353106999 >> 172.21.15.119:0/2353106999 conn(0x55798cff8300 msgr2=0x55798d0be980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:52.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.742+0000 7fae7ba21b80 1 -- 172.21.15.119:0/2353106999 shutdown_connections 2023-12-07T10:24:52.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.742+0000 7fae7ba21b80 1 -- 172.21.15.119:0/2353106999 wait complete. 2023-12-07T10:24:52.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.742+0000 7fae7ba21b80 20 client.27154 trim_cache size 0 max 16384 2023-12-07T10:24:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.760+0000 7fae7ba21b80 1 Processor -- start 2023-12-07T10:24:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.760+0000 7fae7ba21b80 1 -- start start 2023-12-07T10:24:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.760+0000 7fae7ba21b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55798d0c9ad0 0x55798d0e53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.760+0000 7fae7ba21b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d0cea60 0x55798d0c9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.760+0000 7fae7ba21b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55798d0cf380 0x55798d0d9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.760+0000 7fae7ba21b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798ce9c8c0 con 0x55798d0cf380 2023-12-07T10:24:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.760+0000 7fae7ba21b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798d0c5410 con 0x55798d0cea60 2023-12-07T10:24:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.760+0000 7fae7ba21b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798d0d25b0 con 0x55798d0c9ad0 2023-12-07T10:24:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.761+0000 7fae6559a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d0cea60 0x55798d0c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.761+0000 7fae6559a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d0cea60 0x55798d0c9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:46228/0 (socket says 172.21.15.119:46228) 2023-12-07T10:24:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.761+0000 7fae6559a700 1 -- 172.21.15.119:0/2860513575 learned_addr learned my addr 172.21.15.119:0/2860513575 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.761+0000 7fae6659c700 1 --2- 172.21.15.119:0/2860513575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55798d0cf380 0x55798d0d9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.761+0000 7fae65d9b700 1 --2- 172.21.15.119:0/2860513575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55798d0c9ad0 0x55798d0e53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.761+0000 7fae6559a700 1 -- 172.21.15.119:0/2860513575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55798d0c9ad0 msgr2=0x55798d0e53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.761+0000 7fae6559a700 1 --2- 172.21.15.119:0/2860513575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55798d0c9ad0 0x55798d0e53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.761+0000 7fae6559a700 1 -- 172.21.15.119:0/2860513575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55798d0cf380 msgr2=0x55798d0d9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.761+0000 7fae6559a700 1 --2- 172.21.15.119:0/2860513575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55798d0cf380 0x55798d0d9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.761+0000 7fae6559a700 1 -- 172.21.15.119:0/2860513575 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55798ce46d80 con 0x55798d0cea60 2023-12-07T10:24:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.761+0000 7fae6559a700 1 --2- 172.21.15.119:0/2860513575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d0cea60 0x55798d0c9540 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fae500098e0 tx=0x7fae50009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=735646f57d41e204 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.762+0000 7fae54ff9700 1 -- 172.21.15.119:0/2860513575 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5001d070 con 0x55798d0cea60 2023-12-07T10:24:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.762+0000 7fae54ff9700 1 -- 172.21.15.119:0/2860513575 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae5001e040 con 0x55798d0cea60 2023-12-07T10:24:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.762+0000 7fae54ff9700 1 -- 172.21.15.119:0/2860513575 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae50024ad0 con 0x55798d0cea60 2023-12-07T10:24:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.763+0000 7fae7ba21b80 1 -- 172.21.15.119:0/2860513575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d0cea60 msgr2=0x55798d0c9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.763+0000 7fae7ba21b80 1 --2- 172.21.15.119:0/2860513575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d0cea60 0x55798d0c9540 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fae500098e0 tx=0x7fae50009ca0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.763+0000 7fae7ba21b80 1 -- 172.21.15.119:0/2860513575 shutdown_connections 2023-12-07T10:24:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.763+0000 7fae7ba21b80 1 --2- 172.21.15.119:0/2860513575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55798d0cf380 0x55798d0d9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.763+0000 7fae7ba21b80 1 --2- 172.21.15.119:0/2860513575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55798d0c9ad0 0x55798d0e53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.763+0000 7fae7ba21b80 1 --2- 172.21.15.119:0/2860513575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d0cea60 0x55798d0c9540 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.763+0000 7fae7ba21b80 1 -- 172.21.15.119:0/2860513575 >> 172.21.15.119:0/2860513575 conn(0x55798cff8300 msgr2=0x55798d0e9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.763+0000 7fae7ba21b80 1 -- 172.21.15.119:0/2860513575 shutdown_connections 2023-12-07T10:24:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.763+0000 7fae7ba21b80 1 -- 172.21.15.119:0/2860513575 wait complete. 2023-12-07T10:24:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.763+0000 7fae7ba21b80 1 -- start start 2023-12-07T10:24:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.764+0000 7fae7ba21b80 1 Processor -- start 2023-12-07T10:24:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.764+0000 7fae7ba21b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55798d0c9ad0 0x55798d0d9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.764+0000 7fae7ba21b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55798d0cea60 0x55798d0d7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.764+0000 7fae7ba21b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d0cf380 0x55798d0d8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.764+0000 7fae7ba21b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798d0d8860 con 0x55798d0c9ad0 2023-12-07T10:24:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.764+0000 7fae7ba21b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798d0d89d0 con 0x55798d0cf380 2023-12-07T10:24:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.764+0000 7fae7ba21b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798d0d8b90 con 0x55798d0cea60 2023-12-07T10:24:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.764+0000 7fae7ba21b80 10 client.0 _ll_register_callbacks cb 0x55798cfbe8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:24:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.765+0000 7fae6559a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55798d0cea60 0x55798d0d7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.765+0000 7fae6559a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55798d0cea60 0x55798d0d7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:56088/0 (socket says 172.21.15.119:56088) 2023-12-07T10:24:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.765+0000 7fae6559a700 1 -- 172.21.15.119:0/2511445191 learned_addr learned my addr 172.21.15.119:0/2511445191 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:24:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.765+0000 7fae6659c700 1 --2- 172.21.15.119:0/2511445191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d0cf380 0x55798d0d8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.765+0000 7fae65d9b700 1 --2- 172.21.15.119:0/2511445191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55798d0c9ad0 0x55798d0d9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.765+0000 7fae6559a700 1 -- 172.21.15.119:0/2511445191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d0cf380 msgr2=0x55798d0d8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.765+0000 7fae6559a700 1 --2- 172.21.15.119:0/2511445191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d0cf380 0x55798d0d8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.765+0000 7fae6559a700 1 -- 172.21.15.119:0/2511445191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55798d0c9ad0 msgr2=0x55798d0d9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:24:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.765+0000 7fae6559a700 1 --2- 172.21.15.119:0/2511445191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55798d0c9ad0 0x55798d0d9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.765+0000 7fae6559a700 1 -- 172.21.15.119:0/2511445191 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae5800fe70 con 0x55798d0cea60 2023-12-07T10:24:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.765+0000 7fae65d9b700 1 --2- 172.21.15.119:0/2511445191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55798d0c9ad0 0x55798d0d9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:24:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.765+0000 7fae6559a700 1 --2- 172.21.15.119:0/2511445191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55798d0cea60 0x55798d0d7d50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fae50009990 tx=0x7fae5000bf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6e7ae6d6d3e438d server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:24:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.765+0000 7fae497fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:24:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.766+0000 7fae497fa700 1 -- 172.21.15.119:0/2511445191 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5001d040 con 0x55798d0cea60 2023-12-07T10:24:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.766+0000 7fae497fa700 1 -- 172.21.15.119:0/2511445191 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fae50000e40 con 0x55798d0cea60 2023-12-07T10:24:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.766+0000 7fae7ba21b80 1 -- 172.21.15.119:0/2511445191 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae5800f040 con 0x55798d0cea60 2023-12-07T10:24:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.766+0000 7fae497fa700 1 -- 172.21.15.119:0/2511445191 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae5000f870 con 0x55798d0cea60 2023-12-07T10:24:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.766+0000 7fae7ba21b80 10 client.27160 fetch_fsmap learned FSMap version 43 2023-12-07T10:24:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.766+0000 7fae7ba21b80 1 -- 172.21.15.119:0/2511445191 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55798ce2fc00 con 0x55798d0cea60 2023-12-07T10:24:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.766+0000 7fae497fa700 1 -- 172.21.15.119:0/2511445191 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae5800f040 con 0x55798d0cea60 2023-12-07T10:24:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.766+0000 7fae497fa700 1 -- 172.21.15.119:0/2511445191 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 43) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fae50004590 con 0x55798d0cea60 2023-12-07T10:24:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.766+0000 7fae7ba21b80 10 client.27160 fetch_fsmap finished waiting for FSMap version 43 2023-12-07T10:24:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.766+0000 7fae7ba21b80 10 client.27160 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:24:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.766+0000 7fae7ba21b80 20 client.27160 populate_metadata read hostname 'smithi119' 2023-12-07T10:24:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.767+0000 7fae7ba21b80 1 --2- 172.21.15.119:0/2511445191 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55798d141fe0 0x55798d144400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:24:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.767+0000 7fae7ba21b80 4 client.27160 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:24:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.767+0000 7fae7ba21b80 1 -- 172.21.15.119:0/2511445191 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55798cfe3f70 con 0x55798d141fe0 2023-12-07T10:24:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.767+0000 7fae65d9b700 1 --2- 172.21.15.119:0/2511445191 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55798d141fe0 0x55798d144400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:24:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.767+0000 7fae65d9b700 1 --2- 172.21.15.119:0/2511445191 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55798d141fe0 0x55798d144400 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fad8f87289de3c0a server_cookie=ee2e52860da2332f in_seq=0 out_seq=0 2023-12-07T10:24:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.767+0000 7fae497fa700 10 client.27160 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.791+0000 7fae497fa700 1 -- 172.21.15.119:0/2511445191 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55798cfe3f70 con 0x55798d141fe0 2023-12-07T10:24:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.792+0000 7fae497fa700 10 client.27160 handle_command_reply: tid=0 2023-12-07T10:24:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.792+0000 7fae7ba21b80 1 client.27160 shutdown 2023-12-07T10:24:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.792+0000 7fae7ba21b80 1 -- 172.21.15.119:0/2511445191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55798d0cea60 msgr2=0x55798d0d7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.792+0000 7fae7ba21b80 1 --2- 172.21.15.119:0/2511445191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55798d0cea60 0x55798d0d7d50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fae50009990 tx=0x7fae5000bf70 comp rx=0 tx=0).stop 2023-12-07T10:24:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.792+0000 7fae7ba21b80 1 -- 172.21.15.119:0/2511445191 shutdown_connections 2023-12-07T10:24:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.792+0000 7fae7ba21b80 1 --2- 172.21.15.119:0/2511445191 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55798d141fe0 0x55798d144400 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.792+0000 7fae7ba21b80 1 --2- 172.21.15.119:0/2511445191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55798d0cf380 0x55798d0d8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.792+0000 7fae7ba21b80 1 --2- 172.21.15.119:0/2511445191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55798d0c9ad0 0x55798d0d9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.792+0000 7fae7ba21b80 1 --2- 172.21.15.119:0/2511445191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55798d0cea60 0x55798d0d7d50 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.792+0000 7fae7ba21b80 1 -- 172.21.15.119:0/2511445191 >> 172.21.15.119:0/2511445191 conn(0x55798cff8300 msgr2=0x55798d0cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.792+0000 7fae497fa700 0 client.27160 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:24:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.792+0000 7fae7ba21b80 1 -- 172.21.15.119:0/2511445191 shutdown_connections 2023-12-07T10:24:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.792+0000 7fae7ba21b80 1 -- 172.21.15.119:0/2511445191 wait complete. 2023-12-07T10:24:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.792+0000 7fae7ba21b80 20 client.27160 trim_cache size 0 max 16384 2023-12-07T10:24:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:24:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:24:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:24:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:24:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.793+0000 7fae67fff700 1 -- 172.21.15.119:0/3971562407 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fae4c0a1d00 msgr2=0x7fae4c0a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.793+0000 7fae67fff700 1 --2- 172.21.15.119:0/3971562407 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fae4c0a1d00 0x7fae4c0a41c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fae5800f930 tx=0x7fae5800fcf0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.793+0000 7fae67fff700 1 -- 172.21.15.119:0/3971562407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae60103700 msgr2=0x7fae6019f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:24:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.793+0000 7fae67fff700 1 --2- 172.21.15.119:0/3971562407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae60103700 0x7fae6019f830 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00e9a0 tx=0x7fae5c00ecb0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.793+0000 7fae67fff700 1 -- 172.21.15.119:0/3971562407 shutdown_connections 2023-12-07T10:24:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.793+0000 7fae67fff700 1 --2- 172.21.15.119:0/3971562407 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fae4c0a1d00 0x7fae4c0a41c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.793+0000 7fae67fff700 1 --2- 172.21.15.119:0/3971562407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae60109710 0x7fae60199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.793+0000 7fae67fff700 1 --2- 172.21.15.119:0/3971562407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae60103700 0x7fae6019f830 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.793+0000 7fae67fff700 1 --2- 172.21.15.119:0/3971562407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae601040c0 0x7fae6019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:24:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.793+0000 7fae67fff700 1 -- 172.21.15.119:0/3971562407 >> 172.21.15.119:0/3971562407 conn(0x7fae600ff190 msgr2=0x7fae600ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:24:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.793+0000 7fae67fff700 1 -- 172.21.15.119:0/3971562407 shutdown_connections 2023-12-07T10:24:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:24:52.793+0000 7fae67fff700 1 -- 172.21.15.119:0/3971562407 wait complete. 2023-12-07T10:24:53.553 INFO:tasks.workunit.client.0.smithi119.stdout: 1 277608 7.28 MB/sec execute 66 sec latency 19.603 ms 2023-12-07T10:24:54.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:53 smithi159 ceph-mon[134726]: pgmap v137: 129 pgs: 129 active+clean; 151 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 1.2 MiB/s wr, 184 op/s 2023-12-07T10:24:54.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:53 smithi153 ceph-mon[133625]: pgmap v137: 129 pgs: 129 active+clean; 151 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 1.2 MiB/s wr, 184 op/s 2023-12-07T10:24:54.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:53 smithi119 ceph-mon[97908]: pgmap v137: 129 pgs: 129 active+clean; 151 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 1.2 MiB/s wr, 184 op/s 2023-12-07T10:24:54.553 INFO:tasks.workunit.client.0.smithi119.stdout: 1 280044 7.38 MB/sec execute 67 sec latency 20.825 ms 2023-12-07T10:24:55.554 INFO:tasks.workunit.client.0.smithi119.stdout: 1 281116 7.34 MB/sec execute 68 sec latency 20.960 ms 2023-12-07T10:24:56.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:55 smithi159 ceph-mon[134726]: pgmap v138: 129 pgs: 129 active+clean; 151 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 1.2 MiB/s wr, 184 op/s 2023-12-07T10:24:56.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:55 smithi153 ceph-mon[133625]: pgmap v138: 129 pgs: 129 active+clean; 151 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 1.2 MiB/s wr, 184 op/s 2023-12-07T10:24:56.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:55 smithi119 ceph-mon[97908]: pgmap v138: 129 pgs: 129 active+clean; 151 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 1.2 MiB/s wr, 184 op/s 2023-12-07T10:24:56.554 INFO:tasks.workunit.client.0.smithi119.stdout: 1 283581 7.44 MB/sec execute 69 sec latency 11.940 ms 2023-12-07T10:24:57.554 INFO:tasks.workunit.client.0.smithi119.stdout: 1 284522 7.40 MB/sec execute 70 sec latency 23.357 ms 2023-12-07T10:24:58.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:24:58 smithi159 ceph-mon[134726]: pgmap v139: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 284 op/s 2023-12-07T10:24:58.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:24:58 smithi153 ceph-mon[133625]: pgmap v139: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 284 op/s 2023-12-07T10:24:58.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:24:58 smithi119 ceph-mon[97908]: pgmap v139: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 284 op/s 2023-12-07T10:24:58.554 INFO:tasks.workunit.client.0.smithi119.stdout: 1 285877 7.38 MB/sec execute 71 sec latency 162.530 ms 2023-12-07T10:24:59.554 INFO:tasks.workunit.client.0.smithi119.stdout: 1 286953 7.38 MB/sec execute 72 sec latency 200.462 ms 2023-12-07T10:25:00.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:00 smithi159 ceph-mon[134726]: pgmap v140: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 1.2 MiB/s wr, 190 op/s 2023-12-07T10:25:00.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:00 smithi153 ceph-mon[133625]: pgmap v140: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 1.2 MiB/s wr, 190 op/s 2023-12-07T10:25:00.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:00 smithi119 ceph-mon[97908]: pgmap v140: 129 pgs: 129 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 1.2 MiB/s wr, 190 op/s 2023-12-07T10:25:00.554 INFO:tasks.workunit.client.0.smithi119.stdout: 1 288043 7.36 MB/sec execute 73 sec latency 18.068 ms 2023-12-07T10:25:01.554 INFO:tasks.workunit.client.0.smithi119.stdout: 1 289617 7.34 MB/sec execute 74 sec latency 63.022 ms 2023-12-07T10:25:02.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:01 smithi119 ceph-mon[97908]: pgmap v141: 129 pgs: 129 active+clean; 157 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 372 op/s 2023-12-07T10:25:02.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:01 smithi159 ceph-mon[134726]: pgmap v141: 129 pgs: 129 active+clean; 157 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 372 op/s 2023-12-07T10:25:02.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:01 smithi153 ceph-mon[133625]: pgmap v141: 129 pgs: 129 active+clean; 157 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 372 op/s 2023-12-07T10:25:02.554 INFO:tasks.workunit.client.0.smithi119.stdout: 1 290709 7.35 MB/sec execute 75 sec latency 64.372 ms 2023-12-07T10:25:03.554 INFO:tasks.workunit.client.0.smithi119.stdout: 1 291598 7.32 MB/sec execute 76 sec latency 127.182 ms 2023-12-07T10:25:04.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:03 smithi159 ceph-mon[134726]: pgmap v142: 129 pgs: 129 active+clean; 157 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 281 op/s 2023-12-07T10:25:04.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:03 smithi153 ceph-mon[133625]: pgmap v142: 129 pgs: 129 active+clean; 157 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 281 op/s 2023-12-07T10:25:04.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:03 smithi119 ceph-mon[97908]: pgmap v142: 129 pgs: 129 active+clean; 157 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 281 op/s 2023-12-07T10:25:04.554 INFO:tasks.workunit.client.0.smithi119.stdout: 1 293641 7.37 MB/sec execute 77 sec latency 36.476 ms 2023-12-07T10:25:05.554 INFO:tasks.workunit.client.0.smithi119.stdout: 1 294891 7.38 MB/sec execute 78 sec latency 18.378 ms 2023-12-07T10:25:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.961+0000 7fc2eb5ff700 1 -- 172.21.15.119:0/1744636372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2e4101500 msgr2=0x7fc2e4101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.961+0000 7fc2eb5ff700 1 --2- 172.21.15.119:0/1744636372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2e4101500 0x7fc2e4101980 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40099c0 tx=0x7fc2d4009cd0 comp rx=0 tx=0).stop 2023-12-07T10:25:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.961+0000 7fc2eb5ff700 1 -- 172.21.15.119:0/1744636372 shutdown_connections 2023-12-07T10:25:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.961+0000 7fc2eb5ff700 1 --2- 172.21.15.119:0/1744636372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2e4101ec0 0x7fc2e410e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.961+0000 7fc2eb5ff700 1 --2- 172.21.15.119:0/1744636372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2e41075c0 0x7fc2e41079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.961+0000 7fc2eb5ff700 1 --2- 172.21.15.119:0/1744636372 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2e4101500 0x7fc2e4101980 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.961+0000 7fc2eb5ff700 1 -- 172.21.15.119:0/1744636372 >> 172.21.15.119:0/1744636372 conn(0x7fc2e40fcfb0 msgr2=0x7fc2e40ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.961+0000 7fc2eb5ff700 1 -- 172.21.15.119:0/1744636372 shutdown_connections 2023-12-07T10:25:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.961+0000 7fc2eb5ff700 1 -- 172.21.15.119:0/1744636372 wait complete. 2023-12-07T10:25:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.962+0000 7fc2eb5ff700 1 Processor -- start 2023-12-07T10:25:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.962+0000 7fc2eb5ff700 1 -- start start 2023-12-07T10:25:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.962+0000 7fc2eb5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2e4101500 0x7fc2e419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.962+0000 7fc2eb5ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2e4101ec0 0x7fc2e419fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.962+0000 7fc2eb5ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2e41075c0 0x7fc2e4199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.962+0000 7fc2eb5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e4199fd0 con 0x7fc2e4101500 2023-12-07T10:25:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.962+0000 7fc2eb5ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e419a140 con 0x7fc2e41075c0 2023-12-07T10:25:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.962+0000 7fc2eb5ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e419a300 con 0x7fc2e4101ec0 2023-12-07T10:25:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.963+0000 7fc2e8b9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2e4101ec0 0x7fc2e419fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.963+0000 7fc2e939b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2e4101500 0x7fc2e419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.963+0000 7fc2e8b9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2e4101ec0 0x7fc2e419fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:56218/0 (socket says 172.21.15.119:56218) 2023-12-07T10:25:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.963+0000 7fc2e8b9a700 1 -- 172.21.15.119:0/954775338 learned_addr learned my addr 172.21.15.119:0/954775338 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:25:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.963+0000 7fc2e9b9c700 1 --2- 172.21.15.119:0/954775338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2e41075c0 0x7fc2e4199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.963+0000 7fc2e939b700 1 -- 172.21.15.119:0/954775338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2e4101ec0 msgr2=0x7fc2e419fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.963+0000 7fc2e939b700 1 --2- 172.21.15.119:0/954775338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2e4101ec0 0x7fc2e419fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.963+0000 7fc2e939b700 1 -- 172.21.15.119:0/954775338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2e41075c0 msgr2=0x7fc2e4199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.963+0000 7fc2e939b700 1 --2- 172.21.15.119:0/954775338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2e41075c0 0x7fc2e4199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.963+0000 7fc2e939b700 1 -- 172.21.15.119:0/954775338 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d40095f0 con 0x7fc2e4101500 2023-12-07T10:25:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.963+0000 7fc2e8b9a700 1 --2- 172.21.15.119:0/954775338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2e4101ec0 0x7fc2e419fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:25:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.963+0000 7fc2e9b9c700 1 --2- 172.21.15.119:0/954775338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2e41075c0 0x7fc2e4199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:25:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.963+0000 7fc2e939b700 1 --2- 172.21.15.119:0/954775338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2e4101500 0x7fc2e419f840 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc2e000e9f0 tx=0x7fc2e000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.963+0000 7fc2da7fc700 1 -- 172.21.15.119:0/954775338 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e000c9c0 con 0x7fc2e4101500 2023-12-07T10:25:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.963+0000 7fc2da7fc700 1 -- 172.21.15.119:0/954775338 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2e0010430 con 0x7fc2e4101500 2023-12-07T10:25:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.963+0000 7fc2eb5ff700 1 -- 172.21.15.119:0/954775338 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e419a5e0 con 0x7fc2e4101500 2023-12-07T10:25:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.963+0000 7fc2da7fc700 1 -- 172.21.15.119:0/954775338 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0018680 con 0x7fc2e4101500 2023-12-07T10:25:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.964+0000 7fc2eb5ff700 1 -- 172.21.15.119:0/954775338 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e41a6830 con 0x7fc2e4101500 2023-12-07T10:25:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.966+0000 7fc2da7fc700 1 -- 172.21.15.119:0/954775338 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fc2e0010860 con 0x7fc2e4101500 2023-12-07T10:25:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.968+0000 7fc2feea3b80 1 Processor -- start 2023-12-07T10:25:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.968+0000 7fc2feea3b80 1 -- start start 2023-12-07T10:25:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.968+0000 7fc2da7fc700 1 --2- 172.21.15.119:0/954775338 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fc2d00a1c80 0x7fc2d00a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.968+0000 7fc2feea3b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561bc534a60 0x5561bc534e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.968+0000 7fc2feea3b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561bc535380 0x5561bc52f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.968+0000 7fc2feea3b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561bc52fad0 0x5561bc52ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.968+0000 7fc2feea3b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561bc4a3760 con 0x5561bc52fad0 2023-12-07T10:25:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.968+0000 7fc2feea3b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561bc48a740 con 0x5561bc535380 2023-12-07T10:25:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.968+0000 7fc2feea3b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561bc488060 con 0x5561bc534a60 2023-12-07T10:25:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.968+0000 7fc2da7fc700 1 -- 172.21.15.119:0/954775338 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fc2e0014070 con 0x7fc2e4101500 2023-12-07T10:25:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.969+0000 7fc2e8b9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561bc535380 0x5561bc52f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.969+0000 7fc2e939b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561bc534a60 0x5561bc534e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.969+0000 7fc2e9b9c700 1 --2- 172.21.15.119:0/954775338 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fc2d00a1c80 0x7fc2d00a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.969+0000 7fc2e8b9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561bc535380 0x5561bc52f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:40070/0 (socket says 172.21.15.119:40070) 2023-12-07T10:25:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.969+0000 7fc2e8b9a700 1 -- 172.21.15.119:0/3812997847 learned_addr learned my addr 172.21.15.119:0/3812997847 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:25:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.969+0000 7fc2e9b9c700 1 --2- 172.21.15.119:0/3812997847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561bc52fad0 0x5561bc52ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.969+0000 7fc2e8b9a700 1 -- 172.21.15.119:0/3812997847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561bc534a60 msgr2=0x5561bc534e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.969+0000 7fc2e8b9a700 1 --2- 172.21.15.119:0/3812997847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561bc534a60 0x5561bc534e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.969+0000 7fc2e8b9a700 1 -- 172.21.15.119:0/3812997847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561bc52fad0 msgr2=0x5561bc52ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.969+0000 7fc2e8b9a700 1 --2- 172.21.15.119:0/3812997847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561bc52fad0 0x5561bc52ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.969+0000 7fc2e8b9a700 1 -- 172.21.15.119:0/3812997847 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561bc3005d0 con 0x5561bc535380 2023-12-07T10:25:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.969+0000 7fc2e939b700 1 --2- 172.21.15.119:0/3812997847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561bc534a60 0x5561bc534e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:25:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.969+0000 7fc2e9b9c700 1 --2- 172.21.15.119:0/3812997847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561bc52fad0 0x5561bc52ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:25:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.969+0000 7fc2e8b9a700 1 --2- 172.21.15.119:0/3812997847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561bc535380 0x5561bc52f540 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4005bc0 tx=0x7fc2d4005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=324009ed4b98e30 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.969+0000 7fc2e9b9c700 1 --2- 172.21.15.119:0/954775338 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fc2d00a1c80 0x7fc2d00a4140 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x5561bc52b670 tx=0x7fc2dc009920 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.970+0000 7fc2cf7fe700 1 -- 172.21.15.119:0/3812997847 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d401c050 con 0x5561bc535380 2023-12-07T10:25:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.970+0000 7fc2cf7fe700 1 -- 172.21.15.119:0/3812997847 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2d401d040 con 0x5561bc535380 2023-12-07T10:25:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.970+0000 7fc2cf7fe700 1 -- 172.21.15.119:0/3812997847 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4003bf0 con 0x5561bc535380 2023-12-07T10:25:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.970+0000 7fc2feea3b80 1 -- 172.21.15.119:0/3812997847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561bc535380 msgr2=0x5561bc52f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.970+0000 7fc2feea3b80 1 --2- 172.21.15.119:0/3812997847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561bc535380 0x5561bc52f540 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4005bc0 tx=0x7fc2d4005bf0 comp rx=0 tx=0).stop 2023-12-07T10:25:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.971+0000 7fc2feea3b80 1 -- 172.21.15.119:0/3812997847 shutdown_connections 2023-12-07T10:25:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.971+0000 7fc2feea3b80 1 --2- 172.21.15.119:0/3812997847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561bc52fad0 0x5561bc52ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.971+0000 7fc2feea3b80 1 --2- 172.21.15.119:0/3812997847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561bc534a60 0x5561bc534e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.971+0000 7fc2feea3b80 1 --2- 172.21.15.119:0/3812997847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561bc535380 0x5561bc52f540 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.971+0000 7fc2feea3b80 1 -- 172.21.15.119:0/3812997847 >> 172.21.15.119:0/3812997847 conn(0x5561bc45e300 msgr2=0x5561bc529b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.971+0000 7fc2feea3b80 1 -- 172.21.15.119:0/3812997847 shutdown_connections 2023-12-07T10:25:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.971+0000 7fc2feea3b80 1 -- 172.21.15.119:0/3812997847 wait complete. 2023-12-07T10:25:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.971+0000 7fc2feea3b80 1 -- start start 2023-12-07T10:25:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.972+0000 7fc2feea3b80 1 Processor -- start 2023-12-07T10:25:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.972+0000 7fc2feea3b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561bc52fad0 0x5561bc53b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.972+0000 7fc2feea3b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561bc534a60 0x5561bc538680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.972+0000 7fc2feea3b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561bc535380 0x5561bc538dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.972+0000 7fc2feea3b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561bc483950 con 0x5561bc52fad0 2023-12-07T10:25:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.972+0000 7fc2feea3b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561bc43d500 con 0x5561bc534a60 2023-12-07T10:25:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.972+0000 7fc2feea3b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561bc43d1a0 con 0x5561bc535380 2023-12-07T10:25:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.973+0000 7fc2feea3b80 10 client.0 _ll_register_callbacks cb 0x5561bc4248e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:25:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.973+0000 7fc2e939b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561bc52fad0 0x5561bc53b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.973+0000 7fc2e9b9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561bc535380 0x5561bc538dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.973+0000 7fc2e8b9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561bc534a60 0x5561bc538680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.973+0000 7fc2e9b9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561bc535380 0x5561bc538dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:56236/0 (socket says 172.21.15.119:56236) 2023-12-07T10:25:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.973+0000 7fc2e9b9c700 1 -- 172.21.15.119:0/858444716 learned_addr learned my addr 172.21.15.119:0/858444716 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:25:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.973+0000 7fc2e8b9a700 1 -- 172.21.15.119:0/858444716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561bc535380 msgr2=0x5561bc538dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.973+0000 7fc2e8b9a700 1 --2- 172.21.15.119:0/858444716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561bc535380 0x5561bc538dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.973+0000 7fc2e8b9a700 1 -- 172.21.15.119:0/858444716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561bc52fad0 msgr2=0x5561bc53b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.973+0000 7fc2e8b9a700 1 --2- 172.21.15.119:0/858444716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561bc52fad0 0x5561bc53b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.973+0000 7fc2e8b9a700 1 -- 172.21.15.119:0/858444716 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e41a6830 con 0x5561bc534a60 2023-12-07T10:25:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.973+0000 7fc2e9b9c700 1 --2- 172.21.15.119:0/858444716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561bc535380 0x5561bc538dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:25:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.973+0000 7fc2e8b9a700 1 --2- 172.21.15.119:0/858444716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561bc534a60 0x5561bc538680 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4009990 tx=0x7fc2d40036f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c511ed62a3ab609b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.973+0000 7fc2ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:25:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.974+0000 7fc2ccff9700 1 -- 172.21.15.119:0/858444716 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4005520 con 0x5561bc534a60 2023-12-07T10:25:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.974+0000 7fc2ccff9700 1 -- 172.21.15.119:0/858444716 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fc2d40228f0 con 0x5561bc534a60 2023-12-07T10:25:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.974+0000 7fc2feea3b80 1 -- 172.21.15.119:0/858444716 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5561bc5226f0 con 0x5561bc534a60 2023-12-07T10:25:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.974+0000 7fc2ccff9700 1 -- 172.21.15.119:0/858444716 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2d4022ab0 con 0x5561bc534a60 2023-12-07T10:25:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.974+0000 7fc2feea3b80 10 client.27188 fetch_fsmap learned FSMap version 43 2023-12-07T10:25:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.974+0000 7fc2feea3b80 1 -- 172.21.15.119:0/858444716 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7fc2e410ec80 con 0x5561bc534a60 2023-12-07T10:25:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.974+0000 7fc2ccff9700 1 -- 172.21.15.119:0/858444716 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5561bc5226f0 con 0x5561bc534a60 2023-12-07T10:25:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.974+0000 7fc2ccff9700 1 -- 172.21.15.119:0/858444716 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 43) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fc2d400f460 con 0x5561bc534a60 2023-12-07T10:25:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.975+0000 7fc2feea3b80 10 client.27188 fetch_fsmap finished waiting for FSMap version 43 2023-12-07T10:25:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.975+0000 7fc2feea3b80 10 client.27188 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:25:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.975+0000 7fc2feea3b80 20 client.27188 populate_metadata read hostname 'smithi119' 2023-12-07T10:25:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.975+0000 7fc2feea3b80 1 --2- 172.21.15.119:0/858444716 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5561bc548a40 0x5561bc54ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.975+0000 7fc2feea3b80 4 client.27188 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:25:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.975+0000 7fc2feea3b80 1 -- 172.21.15.119:0/858444716 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5561bc4fe740 con 0x5561bc548a40 2023-12-07T10:25:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.975+0000 7fc2e939b700 1 --2- 172.21.15.119:0/858444716 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5561bc548a40 0x5561bc54ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.975+0000 7fc2e939b700 1 --2- 172.21.15.119:0/858444716 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5561bc548a40 0x5561bc54ae60 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8469e108f4c4faf8 server_cookie=4a6ab8031f040ccc in_seq=0 out_seq=0 2023-12-07T10:25:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.975+0000 7fc2ccff9700 10 client.27188 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:25:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.976+0000 7fc2ccff9700 1 -- 172.21.15.119:0/858444716 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5561bc4fe740 con 0x5561bc548a40 2023-12-07T10:25:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.976+0000 7fc2ccff9700 10 client.27188 handle_command_reply: tid=0 2023-12-07T10:25:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.976+0000 7fc2feea3b80 1 client.27188 shutdown 2023-12-07T10:25:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.976+0000 7fc2feea3b80 1 -- 172.21.15.119:0/858444716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561bc534a60 msgr2=0x5561bc538680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.976+0000 7fc2feea3b80 1 --2- 172.21.15.119:0/858444716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561bc534a60 0x5561bc538680 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4009990 tx=0x7fc2d40036f0 comp rx=0 tx=0).stop 2023-12-07T10:25:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.976+0000 7fc2feea3b80 1 -- 172.21.15.119:0/858444716 shutdown_connections 2023-12-07T10:25:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.976+0000 7fc2feea3b80 1 --2- 172.21.15.119:0/858444716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561bc535380 0x5561bc538dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.976+0000 7fc2feea3b80 1 --2- 172.21.15.119:0/858444716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561bc52fad0 0x5561bc53b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.976+0000 7fc2feea3b80 1 --2- 172.21.15.119:0/858444716 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5561bc548a40 0x5561bc54ae60 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.976+0000 7fc2feea3b80 1 --2- 172.21.15.119:0/858444716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561bc534a60 0x5561bc538680 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.976+0000 7fc2ccff9700 0 client.27188 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:25:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.976+0000 7fc2feea3b80 1 -- 172.21.15.119:0/858444716 >> 172.21.15.119:0/858444716 conn(0x5561bc45e300 msgr2=0x5561bc524980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.976+0000 7fc2feea3b80 1 -- 172.21.15.119:0/858444716 shutdown_connections 2023-12-07T10:25:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.977+0000 7fc2feea3b80 1 -- 172.21.15.119:0/858444716 wait complete. 2023-12-07T10:25:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.977+0000 7fc2feea3b80 20 client.27188 trim_cache size 0 max 16384 2023-12-07T10:25:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.994+0000 7fc2feea3b80 1 Processor -- start 2023-12-07T10:25:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.994+0000 7fc2feea3b80 1 -- start start 2023-12-07T10:25:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.995+0000 7fc2feea3b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561bc52fad0 0x5561bc53fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.995+0000 7fc2feea3b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561bc534a60 0x5561bc5402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.995+0000 7fc2feea3b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561bc535380 0x5561bc532900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.995+0000 7fc2feea3b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561bc3028c0 con 0x5561bc535380 2023-12-07T10:25:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.995+0000 7fc2feea3b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561bc539c50 con 0x5561bc52fad0 2023-12-07T10:25:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.995+0000 7fc2feea3b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561bc5226d0 con 0x5561bc534a60 2023-12-07T10:25:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.995+0000 7fc2e939b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561bc52fad0 0x5561bc53fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.995+0000 7fc2e939b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561bc52fad0 0x5561bc53fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:40078/0 (socket says 172.21.15.119:40078) 2023-12-07T10:25:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.995+0000 7fc2e939b700 1 -- 172.21.15.119:0/1377244161 learned_addr learned my addr 172.21.15.119:0/1377244161 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:25:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.995+0000 7fc2e8b9a700 1 --2- 172.21.15.119:0/1377244161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561bc534a60 0x5561bc5402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.995+0000 7fc2e9b9c700 1 --2- 172.21.15.119:0/1377244161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561bc535380 0x5561bc532900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.995+0000 7fc2e939b700 1 -- 172.21.15.119:0/1377244161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561bc534a60 msgr2=0x5561bc5402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.995+0000 7fc2e939b700 1 --2- 172.21.15.119:0/1377244161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561bc534a60 0x5561bc5402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.995+0000 7fc2e939b700 1 -- 172.21.15.119:0/1377244161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561bc535380 msgr2=0x5561bc532900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.995+0000 7fc2e939b700 1 --2- 172.21.15.119:0/1377244161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561bc535380 0x5561bc532900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.995+0000 7fc2e939b700 1 -- 172.21.15.119:0/1377244161 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561bc2acd80 con 0x5561bc52fad0 2023-12-07T10:25:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.995+0000 7fc2e8b9a700 1 --2- 172.21.15.119:0/1377244161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561bc534a60 0x5561bc5402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:25:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.996+0000 7fc2e939b700 1 --2- 172.21.15.119:0/1377244161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561bc52fad0 0x5561bc53fda0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc2e000c5b0 tx=0x7fc2e0096490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5da5b745deb132e4 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.996+0000 7fc2cffff700 1 -- 172.21.15.119:0/1377244161 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0067070 con 0x5561bc52fad0 2023-12-07T10:25:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.996+0000 7fc2cffff700 1 -- 172.21.15.119:0/1377244161 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2e006ad80 con 0x5561bc52fad0 2023-12-07T10:25:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.996+0000 7fc2cffff700 1 -- 172.21.15.119:0/1377244161 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0098ca0 con 0x5561bc52fad0 2023-12-07T10:25:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.997+0000 7fc2feea3b80 1 -- 172.21.15.119:0/1377244161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561bc52fad0 msgr2=0x5561bc53fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.997+0000 7fc2feea3b80 1 --2- 172.21.15.119:0/1377244161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561bc52fad0 0x5561bc53fda0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc2e000c5b0 tx=0x7fc2e0096490 comp rx=0 tx=0).stop 2023-12-07T10:25:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.997+0000 7fc2feea3b80 1 -- 172.21.15.119:0/1377244161 shutdown_connections 2023-12-07T10:25:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.997+0000 7fc2feea3b80 1 --2- 172.21.15.119:0/1377244161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561bc535380 0x5561bc532900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.997+0000 7fc2feea3b80 1 --2- 172.21.15.119:0/1377244161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561bc52fad0 0x5561bc53fda0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.997+0000 7fc2feea3b80 1 --2- 172.21.15.119:0/1377244161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561bc534a60 0x5561bc5402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.997+0000 7fc2feea3b80 1 -- 172.21.15.119:0/1377244161 >> 172.21.15.119:0/1377244161 conn(0x5561bc45e300 msgr2=0x5561bc53b020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.997+0000 7fc2feea3b80 1 -- 172.21.15.119:0/1377244161 shutdown_connections 2023-12-07T10:25:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.997+0000 7fc2feea3b80 1 -- 172.21.15.119:0/1377244161 wait complete. 2023-12-07T10:25:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.997+0000 7fc2feea3b80 1 -- start start 2023-12-07T10:25:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.997+0000 7fc2feea3b80 1 Processor -- start 2023-12-07T10:25:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.998+0000 7fc2feea3b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561bc52fad0 0x5561bc53df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.998+0000 7fc2feea3b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561bc534a60 0x5561bc53e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.998+0000 7fc2feea3b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561bc535380 0x5561bc59df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.998+0000 7fc2feea3b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561bc53eba0 con 0x5561bc535380 2023-12-07T10:25:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.998+0000 7fc2feea3b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561bc59e440 con 0x5561bc534a60 2023-12-07T10:25:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.998+0000 7fc2feea3b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561bc59e580 con 0x5561bc52fad0 2023-12-07T10:25:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.998+0000 7fc2feea3b80 10 client.0 _ll_register_callbacks cb 0x5561bc4248e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:25:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.998+0000 7fc2e939b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561bc52fad0 0x5561bc53df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.998+0000 7fc2e8b9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561bc534a60 0x5561bc53e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.998+0000 7fc2e939b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561bc52fad0 0x5561bc53df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:56266/0 (socket says 172.21.15.119:56266) 2023-12-07T10:25:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.998+0000 7fc2e939b700 1 -- 172.21.15.119:0/3130129208 learned_addr learned my addr 172.21.15.119:0/3130129208 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:25:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.998+0000 7fc2e9b9c700 1 --2- 172.21.15.119:0/3130129208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561bc535380 0x5561bc59df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.998+0000 7fc2e8b9a700 1 -- 172.21.15.119:0/3130129208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561bc52fad0 msgr2=0x5561bc53df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.998+0000 7fc2e8b9a700 1 --2- 172.21.15.119:0/3130129208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561bc52fad0 0x5561bc53df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.998+0000 7fc2e8b9a700 1 -- 172.21.15.119:0/3130129208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561bc535380 msgr2=0x5561bc59df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.998+0000 7fc2e8b9a700 1 --2- 172.21.15.119:0/3130129208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561bc535380 0x5561bc59df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.999+0000 7fc2e8b9a700 1 -- 172.21.15.119:0/3130129208 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e410ec80 con 0x5561bc534a60 2023-12-07T10:25:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.999+0000 7fc2e939b700 1 --2- 172.21.15.119:0/3130129208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561bc52fad0 0x5561bc53df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:25:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.999+0000 7fc2e8b9a700 1 --2- 172.21.15.119:0/3130129208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561bc534a60 0x5561bc53e4b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc2d400f7c0 tx=0x7fc2d40059a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd98580fc31c744b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.999+0000 7fc2ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:25:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.999+0000 7fc2ccff9700 1 -- 172.21.15.119:0/3130129208 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d401c040 con 0x5561bc534a60 2023-12-07T10:25:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.999+0000 7fc2feea3b80 1 -- 172.21.15.119:0/3130129208 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2d401d040 con 0x5561bc534a60 2023-12-07T10:25:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.999+0000 7fc2ccff9700 1 -- 172.21.15.119:0/3130129208 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fc2d4000e40 con 0x5561bc534a60 2023-12-07T10:25:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.999+0000 7fc2ccff9700 1 -- 172.21.15.119:0/3130129208 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2d4009290 con 0x5561bc534a60 2023-12-07T10:25:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.999+0000 7fc2feea3b80 10 client.27194 fetch_fsmap learned FSMap version 43 2023-12-07T10:25:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:05.999+0000 7fc2feea3b80 1 -- 172.21.15.119:0/3130129208 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7fc2e419a5e0 con 0x5561bc534a60 2023-12-07T10:25:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.000+0000 7fc2ccff9700 1 -- 172.21.15.119:0/3130129208 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2d401d040 con 0x5561bc534a60 2023-12-07T10:25:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.000+0000 7fc2ccff9700 1 -- 172.21.15.119:0/3130129208 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 43) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fc2d4022670 con 0x5561bc534a60 2023-12-07T10:25:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.000+0000 7fc2feea3b80 10 client.27194 fetch_fsmap finished waiting for FSMap version 43 2023-12-07T10:25:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.000+0000 7fc2feea3b80 10 client.27194 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:25:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.000+0000 7fc2feea3b80 20 client.27194 populate_metadata read hostname 'smithi119' 2023-12-07T10:25:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.000+0000 7fc2feea3b80 1 --2- 172.21.15.119:0/3130129208 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5561bc59f920 0x5561bc5a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.000+0000 7fc2feea3b80 4 client.27194 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:25:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.000+0000 7fc2feea3b80 1 -- 172.21.15.119:0/3130129208 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5561bc449f70 con 0x5561bc59f920 2023-12-07T10:25:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.000+0000 7fc2e939b700 1 --2- 172.21.15.119:0/3130129208 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5561bc59f920 0x5561bc5a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.001+0000 7fc2ccff9700 10 client.27194 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:25:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.001+0000 7fc2e939b700 1 --2- 172.21.15.119:0/3130129208 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5561bc59f920 0x5561bc5a1d40 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b5d5082692dd402 server_cookie=ebe72b4a7bfd6726 in_seq=0 out_seq=0 2023-12-07T10:25:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.034+0000 7fc2ccff9700 1 -- 172.21.15.119:0/3130129208 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5561bc449f70 con 0x5561bc59f920 2023-12-07T10:25:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.034+0000 7fc2ccff9700 10 client.27194 handle_command_reply: tid=0 2023-12-07T10:25:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.034+0000 7fc2feea3b80 1 client.27194 shutdown 2023-12-07T10:25:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.034+0000 7fc2feea3b80 1 -- 172.21.15.119:0/3130129208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561bc534a60 msgr2=0x5561bc53e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.035+0000 7fc2feea3b80 1 --2- 172.21.15.119:0/3130129208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561bc534a60 0x5561bc53e4b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc2d400f7c0 tx=0x7fc2d40059a0 comp rx=0 tx=0).stop 2023-12-07T10:25:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.035+0000 7fc2feea3b80 1 -- 172.21.15.119:0/3130129208 shutdown_connections 2023-12-07T10:25:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.035+0000 7fc2feea3b80 1 --2- 172.21.15.119:0/3130129208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561bc535380 0x5561bc59df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.035+0000 7fc2feea3b80 1 --2- 172.21.15.119:0/3130129208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561bc52fad0 0x5561bc53df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.035+0000 7fc2feea3b80 1 --2- 172.21.15.119:0/3130129208 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5561bc59f920 0x5561bc5a1d40 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.035+0000 7fc2feea3b80 1 --2- 172.21.15.119:0/3130129208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5561bc534a60 0x5561bc53e4b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.035+0000 7fc2ccff9700 0 client.27194 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:25:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.035+0000 7fc2feea3b80 1 -- 172.21.15.119:0/3130129208 >> 172.21.15.119:0/3130129208 conn(0x5561bc45e300 msgr2=0x5561bc532900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.035+0000 7fc2feea3b80 1 -- 172.21.15.119:0/3130129208 shutdown_connections 2023-12-07T10:25:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.035+0000 7fc2feea3b80 1 -- 172.21.15.119:0/3130129208 wait complete. 2023-12-07T10:25:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.035+0000 7fc2feea3b80 20 client.27194 trim_cache size 0 max 16384 2023-12-07T10:25:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:25:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:25:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:25:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:25:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.036+0000 7fc2eb5ff700 1 -- 172.21.15.119:0/954775338 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fc2d00a1c80 msgr2=0x7fc2d00a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.036+0000 7fc2eb5ff700 1 --2- 172.21.15.119:0/954775338 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fc2d00a1c80 0x7fc2d00a4140 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x5561bc52b670 tx=0x7fc2dc009920 comp rx=0 tx=0).stop 2023-12-07T10:25:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.036+0000 7fc2eb5ff700 1 -- 172.21.15.119:0/954775338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2e4101500 msgr2=0x7fc2e419f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.036+0000 7fc2eb5ff700 1 --2- 172.21.15.119:0/954775338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2e4101500 0x7fc2e419f840 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc2e000e9f0 tx=0x7fc2e000ed00 comp rx=0 tx=0).stop 2023-12-07T10:25:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.036+0000 7fc2eb5ff700 1 -- 172.21.15.119:0/954775338 shutdown_connections 2023-12-07T10:25:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.036+0000 7fc2eb5ff700 1 --2- 172.21.15.119:0/954775338 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fc2d00a1c80 0x7fc2d00a4140 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.036+0000 7fc2eb5ff700 1 --2- 172.21.15.119:0/954775338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2e41075c0 0x7fc2e4199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.036+0000 7fc2eb5ff700 1 --2- 172.21.15.119:0/954775338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2e4101500 0x7fc2e419f840 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.036+0000 7fc2eb5ff700 1 --2- 172.21.15.119:0/954775338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2e4101ec0 0x7fc2e419fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.036+0000 7fc2eb5ff700 1 -- 172.21.15.119:0/954775338 >> 172.21.15.119:0/954775338 conn(0x7fc2e40fcfb0 msgr2=0x7fc2e40fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.036+0000 7fc2eb5ff700 1 -- 172.21.15.119:0/954775338 shutdown_connections 2023-12-07T10:25:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:06.036+0000 7fc2eb5ff700 1 -- 172.21.15.119:0/954775338 wait complete. 2023-12-07T10:25:06.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:06 smithi159 ceph-mon[134726]: pgmap v143: 129 pgs: 129 active+clean; 157 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 281 op/s 2023-12-07T10:25:06.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:06 smithi153 ceph-mon[133625]: pgmap v143: 129 pgs: 129 active+clean; 157 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 281 op/s 2023-12-07T10:25:06.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:06 smithi119 ceph-mon[97908]: pgmap v143: 129 pgs: 129 active+clean; 157 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 281 op/s 2023-12-07T10:25:06.554 INFO:tasks.workunit.client.0.smithi119.stdout: 1 296924 7.38 MB/sec execute 79 sec latency 19.103 ms 2023-12-07T10:25:07.554 INFO:tasks.workunit.client.0.smithi119.stdout: 1 298376 7.43 MB/sec execute 80 sec latency 12.060 ms 2023-12-07T10:25:08.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:08 smithi159 ceph-mon[134726]: pgmap v144: 129 pgs: 129 active+clean; 160 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 414 op/s 2023-12-07T10:25:08.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:08 smithi153 ceph-mon[133625]: pgmap v144: 129 pgs: 129 active+clean; 160 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 414 op/s 2023-12-07T10:25:08.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:08 smithi119 ceph-mon[97908]: pgmap v144: 129 pgs: 129 active+clean; 160 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 414 op/s 2023-12-07T10:25:08.554 INFO:tasks.workunit.client.0.smithi119.stdout: 1 300223 7.41 MB/sec execute 81 sec latency 17.170 ms 2023-12-07T10:25:09.554 INFO:tasks.workunit.client.0.smithi119.stdout: 1 301843 7.48 MB/sec execute 82 sec latency 11.991 ms 2023-12-07T10:25:10.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:10 smithi159 ceph-mon[134726]: pgmap v145: 129 pgs: 129 active+clean; 160 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.8 MiB/s wr, 315 op/s 2023-12-07T10:25:10.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:10 smithi153 ceph-mon[133625]: pgmap v145: 129 pgs: 129 active+clean; 160 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.8 MiB/s wr, 315 op/s 2023-12-07T10:25:10.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:10 smithi119 ceph-mon[97908]: pgmap v145: 129 pgs: 129 active+clean; 160 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.8 MiB/s wr, 315 op/s 2023-12-07T10:25:10.554 INFO:tasks.workunit.client.0.smithi119.stdout: 1 303181 7.46 MB/sec execute 83 sec latency 28.166 ms 2023-12-07T10:25:11.554 INFO:tasks.workunit.client.0.smithi119.stdout: 1 305309 7.53 MB/sec execute 84 sec latency 11.495 ms 2023-12-07T10:25:12.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:12 smithi159 ceph-mon[134726]: pgmap v146: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 419 op/s 2023-12-07T10:25:12.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:12 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:25:12.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:12 smithi153 ceph-mon[133625]: pgmap v146: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 419 op/s 2023-12-07T10:25:12.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:12 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:25:12.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:12 smithi119 ceph-mon[97908]: pgmap v146: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 419 op/s 2023-12-07T10:25:12.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:12 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:25:12.554 INFO:tasks.workunit.client.0.smithi119.stdout: 1 306491 7.50 MB/sec execute 85 sec latency 13.882 ms 2023-12-07T10:25:13.554 INFO:tasks.workunit.client.0.smithi119.stdout: 1 308845 7.58 MB/sec execute 86 sec latency 12.701 ms 2023-12-07T10:25:14.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:14 smithi153 ceph-mon[133625]: pgmap v147: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 237 op/s 2023-12-07T10:25:14.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:14 smithi119 ceph-mon[97908]: pgmap v147: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 237 op/s 2023-12-07T10:25:14.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:14 smithi159 ceph-mon[134726]: pgmap v147: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 237 op/s 2023-12-07T10:25:14.555 INFO:tasks.workunit.client.0.smithi119.stdout: 1 310023 7.55 MB/sec execute 87 sec latency 19.028 ms 2023-12-07T10:25:15.554 INFO:tasks.workunit.client.0.smithi119.stdout: 1 312378 7.62 MB/sec execute 88 sec latency 11.355 ms 2023-12-07T10:25:16.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:16 smithi153 ceph-mon[133625]: pgmap v148: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 237 op/s 2023-12-07T10:25:16.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:16 smithi119 ceph-mon[97908]: pgmap v148: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 237 op/s 2023-12-07T10:25:16.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:16 smithi159 ceph-mon[134726]: pgmap v148: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 237 op/s 2023-12-07T10:25:16.554 INFO:tasks.workunit.client.0.smithi119.stdout: 1 313393 7.56 MB/sec execute 89 sec latency 15.074 ms 2023-12-07T10:25:17.555 INFO:tasks.workunit.client.0.smithi119.stdout: 1 315789 7.65 MB/sec execute 90 sec latency 56.088 ms 2023-12-07T10:25:18.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:18 smithi153 ceph-mon[133625]: pgmap v149: 129 pgs: 129 active+clean; 172 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 327 op/s 2023-12-07T10:25:18.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:18 smithi119 ceph-mon[97908]: pgmap v149: 129 pgs: 129 active+clean; 172 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 327 op/s 2023-12-07T10:25:18.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:18 smithi159 ceph-mon[134726]: pgmap v149: 129 pgs: 129 active+clean; 172 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 327 op/s 2023-12-07T10:25:18.555 INFO:tasks.workunit.client.0.smithi119.stdout: 1 316773 7.60 MB/sec execute 91 sec latency 11.377 ms 2023-12-07T10:25:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.204+0000 7f6f73fff700 1 -- 172.21.15.119:0/500981133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f6c1060c0 msgr2=0x7f6f6c106540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.204+0000 7f6f73fff700 1 --2- 172.21.15.119:0/500981133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f6c1060c0 0x7f6f6c106540 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0098f0 tx=0x7f6f5c009c00 comp rx=0 tx=0).stop 2023-12-07T10:25:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.204+0000 7f6f73fff700 1 -- 172.21.15.119:0/500981133 shutdown_connections 2023-12-07T10:25:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.204+0000 7f6f73fff700 1 --2- 172.21.15.119:0/500981133 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f6c106a80 0x7f6f6c10e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.204+0000 7f6f73fff700 1 --2- 172.21.15.119:0/500981133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f6c068ce0 0x7f6f6c0690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.204+0000 7f6f73fff700 1 --2- 172.21.15.119:0/500981133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f6c1060c0 0x7f6f6c106540 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.204+0000 7f6f73fff700 1 -- 172.21.15.119:0/500981133 >> 172.21.15.119:0/500981133 conn(0x7f6f6c0775b0 msgr2=0x7f6f6c0779c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.204+0000 7f6f73fff700 1 -- 172.21.15.119:0/500981133 shutdown_connections 2023-12-07T10:25:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.204+0000 7f6f73fff700 1 -- 172.21.15.119:0/500981133 wait complete. 2023-12-07T10:25:19.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.205+0000 7f6f73fff700 1 Processor -- start 2023-12-07T10:25:19.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.206+0000 7f6f73fff700 1 -- start start 2023-12-07T10:25:19.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.206+0000 7f6f73fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f6c068ce0 0x7f6f6c100480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:19.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.206+0000 7f6f73fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f6c1060c0 0x7f6f6c1009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:19.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.206+0000 7f6f73fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f6c106a80 0x7f6f6c100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:19.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.206+0000 7f6f73fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c1a5e20 con 0x7f6f6c1060c0 2023-12-07T10:25:19.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.206+0000 7f6f73fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c1a5f90 con 0x7f6f6c068ce0 2023-12-07T10:25:19.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.206+0000 7f6f73fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c1a6150 con 0x7f6f6c106a80 2023-12-07T10:25:19.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.206+0000 7f6f71d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f6c068ce0 0x7f6f6c100480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.206+0000 7f6f71d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f6c068ce0 0x7f6f6c100480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45280/0 (socket says 172.21.15.119:45280) 2023-12-07T10:25:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.206+0000 7f6f71d9b700 1 -- 172.21.15.119:0/1484472311 learned_addr learned my addr 172.21.15.119:0/1484472311 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:25:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.206+0000 7f6f7159a700 1 --2- 172.21.15.119:0/1484472311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f6c1060c0 0x7f6f6c1009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.206+0000 7f6f7259c700 1 --2- 172.21.15.119:0/1484472311 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f6c106a80 0x7f6f6c100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.207+0000 7f6f71d9b700 1 -- 172.21.15.119:0/1484472311 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f6c106a80 msgr2=0x7f6f6c100f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.207+0000 7f6f71d9b700 1 --2- 172.21.15.119:0/1484472311 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f6c106a80 0x7f6f6c100f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.207+0000 7f6f71d9b700 1 -- 172.21.15.119:0/1484472311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f6c1060c0 msgr2=0x7f6f6c1009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.207+0000 7f6f71d9b700 1 --2- 172.21.15.119:0/1484472311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f6c1060c0 0x7f6f6c1009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.207+0000 7f6f71d9b700 1 -- 172.21.15.119:0/1484472311 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f5c009520 con 0x7f6f6c068ce0 2023-12-07T10:25:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.207+0000 7f6f7259c700 1 --2- 172.21.15.119:0/1484472311 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f6c106a80 0x7f6f6c100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:25:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.207+0000 7f6f7159a700 1 --2- 172.21.15.119:0/1484472311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f6c1060c0 0x7f6f6c1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:25:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.207+0000 7f6f71d9b700 1 --2- 172.21.15.119:0/1484472311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f6c068ce0 0x7f6f6c100480 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6f6800e9c0 tx=0x7f6f6800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.207+0000 7f6f62ffd700 1 -- 172.21.15.119:0/1484472311 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f68004a30 con 0x7f6f6c068ce0 2023-12-07T10:25:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.207+0000 7f6f62ffd700 1 -- 172.21.15.119:0/1484472311 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f6800cc80 con 0x7f6f6c068ce0 2023-12-07T10:25:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.207+0000 7f6f62ffd700 1 -- 172.21.15.119:0/1484472311 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f680054e0 con 0x7f6f6c068ce0 2023-12-07T10:25:19.210 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.207+0000 7f6f73fff700 1 -- 172.21.15.119:0/1484472311 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f6c1a6430 con 0x7f6f6c068ce0 2023-12-07T10:25:19.210 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.207+0000 7f6f73fff700 1 -- 172.21.15.119:0/1484472311 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f6c1a6910 con 0x7f6f6c068ce0 2023-12-07T10:25:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.211+0000 7f6f62ffd700 1 -- 172.21.15.119:0/1484472311 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f6f68004b90 con 0x7f6f6c068ce0 2023-12-07T10:25:19.212 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.212+0000 7f6f87848b80 1 Processor -- start 2023-12-07T10:25:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.212+0000 7f6f87848b80 1 -- start start 2023-12-07T10:25:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.213+0000 7f6f87848b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2f36881a0 0x55c2f3688580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.213+0000 7f6f62ffd700 1 --2- 172.21.15.119:0/1484472311 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f6f580a1d00 0x7f6f580a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.213+0000 7f6f87848b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f3683d50 0x55c2f36841d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.213+0000 7f6f87848b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2f3684710 0x55c2f36900b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.213+0000 7f6f87848b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f35ef760 con 0x55c2f36881a0 2023-12-07T10:25:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.213+0000 7f6f87848b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f35d6740 con 0x55c2f3683d50 2023-12-07T10:25:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.213+0000 7f6f87848b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f35d4060 con 0x55c2f3684710 2023-12-07T10:25:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.213+0000 7f6f62ffd700 1 -- 172.21.15.119:0/1484472311 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f6f68014070 con 0x7f6f6c068ce0 2023-12-07T10:25:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.213+0000 7f6f7159a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f3683d50 0x55c2f36841d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.213+0000 7f6f7159a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f3683d50 0x55c2f36841d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45288/0 (socket says 172.21.15.119:45288) 2023-12-07T10:25:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.213+0000 7f6f7159a700 1 -- 172.21.15.119:0/1385996024 learned_addr learned my addr 172.21.15.119:0/1385996024 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:25:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.213+0000 7f6f7259c700 1 --2- 172.21.15.119:0/1484472311 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f6f580a1d00 0x7f6f580a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.213+0000 7f6f71d9b700 1 --2- 172.21.15.119:0/1385996024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2f36881a0 0x55c2f3688580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.213+0000 7f6f7259c700 1 --2- 172.21.15.119:0/1385996024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2f3684710 0x55c2f36900b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.214+0000 7f6f7159a700 1 -- 172.21.15.119:0/1385996024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2f3684710 msgr2=0x55c2f36900b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.214+0000 7f6f7159a700 1 --2- 172.21.15.119:0/1385996024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2f3684710 0x55c2f36900b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.214+0000 7f6f7159a700 1 -- 172.21.15.119:0/1385996024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2f36881a0 msgr2=0x55c2f3688580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.214+0000 7f6f7159a700 1 --2- 172.21.15.119:0/1385996024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2f36881a0 0x55c2f3688580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.214+0000 7f6f7159a700 1 -- 172.21.15.119:0/1385996024 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2f344c5d0 con 0x55c2f3683d50 2023-12-07T10:25:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.214+0000 7f6f7159a700 1 --2- 172.21.15.119:0/1385996024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f3683d50 0x55c2f36841d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c009230 tx=0x7f6f5c005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a97da472c97adba4 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.214+0000 7f6f7259c700 1 --2- 172.21.15.119:0/1484472311 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f6f580a1d00 0x7f6f580a41c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x55c2f36763b0 tx=0x7f6f64009920 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.214+0000 7f6f57fff700 1 -- 172.21.15.119:0/1385996024 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c01c050 con 0x55c2f3683d50 2023-12-07T10:25:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.214+0000 7f6f57fff700 1 -- 172.21.15.119:0/1385996024 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f5c01d040 con 0x55c2f3683d50 2023-12-07T10:25:19.217 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.214+0000 7f6f57fff700 1 -- 172.21.15.119:0/1385996024 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c0052a0 con 0x55c2f3683d50 2023-12-07T10:25:19.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.215+0000 7f6f87848b80 1 -- 172.21.15.119:0/1385996024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f3683d50 msgr2=0x55c2f36841d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:19.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.215+0000 7f6f87848b80 1 --2- 172.21.15.119:0/1385996024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f3683d50 0x55c2f36841d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c009230 tx=0x7f6f5c005dc0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.215+0000 7f6f87848b80 1 -- 172.21.15.119:0/1385996024 shutdown_connections 2023-12-07T10:25:19.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.215+0000 7f6f87848b80 1 --2- 172.21.15.119:0/1385996024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2f3684710 0x55c2f36900b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.215+0000 7f6f87848b80 1 --2- 172.21.15.119:0/1385996024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2f36881a0 0x55c2f3688580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.215+0000 7f6f87848b80 1 --2- 172.21.15.119:0/1385996024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f3683d50 0x55c2f36841d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.215+0000 7f6f87848b80 1 -- 172.21.15.119:0/1385996024 >> 172.21.15.119:0/1385996024 conn(0x55c2f35aa300 msgr2=0x55c2f3675b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:19.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.215+0000 7f6f87848b80 1 -- 172.21.15.119:0/1385996024 shutdown_connections 2023-12-07T10:25:19.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.215+0000 7f6f87848b80 1 -- 172.21.15.119:0/1385996024 wait complete. 2023-12-07T10:25:19.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.216+0000 7f6f87848b80 1 -- start start 2023-12-07T10:25:19.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.216+0000 7f6f87848b80 1 Processor -- start 2023-12-07T10:25:19.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.217+0000 7f6f87848b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2f3683d50 0x55c2f368ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.217+0000 7f6f87848b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2f3684710 0x55c2f368d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.217+0000 7f6f87848b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f36881a0 0x55c2f368d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.217+0000 7f6f87848b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f35cf950 con 0x55c2f3684710 2023-12-07T10:25:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.217+0000 7f6f87848b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f3589500 con 0x55c2f36881a0 2023-12-07T10:25:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.217+0000 7f6f87848b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f35891a0 con 0x55c2f3683d50 2023-12-07T10:25:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.217+0000 7f6f87848b80 10 client.0 _ll_register_callbacks cb 0x55c2f35708e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:25:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.217+0000 7f6f71d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2f3683d50 0x55c2f368ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.217+0000 7f6f7259c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f36881a0 0x55c2f368d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.217+0000 7f6f71d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2f3683d50 0x55c2f368ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55204/0 (socket says 172.21.15.119:55204) 2023-12-07T10:25:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.217+0000 7f6f71d9b700 1 -- 172.21.15.119:0/3288348165 learned_addr learned my addr 172.21.15.119:0/3288348165 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:25:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.217+0000 7f6f7159a700 1 --2- 172.21.15.119:0/3288348165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2f3684710 0x55c2f368d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.218+0000 7f6f71d9b700 1 -- 172.21.15.119:0/3288348165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f36881a0 msgr2=0x55c2f368d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.218+0000 7f6f71d9b700 1 --2- 172.21.15.119:0/3288348165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f36881a0 0x55c2f368d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.218+0000 7f6f71d9b700 1 -- 172.21.15.119:0/3288348165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2f3684710 msgr2=0x55c2f368d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.218+0000 7f6f71d9b700 1 --2- 172.21.15.119:0/3288348165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2f3684710 0x55c2f368d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.218+0000 7f6f71d9b700 1 -- 172.21.15.119:0/3288348165 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f6c10ec60 con 0x55c2f3683d50 2023-12-07T10:25:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.218+0000 7f6f7159a700 1 --2- 172.21.15.119:0/3288348165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2f3684710 0x55c2f368d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:25:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.218+0000 7f6f7259c700 1 --2- 172.21.15.119:0/3288348165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f36881a0 0x55c2f368d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:25:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.218+0000 7f6f71d9b700 1 --2- 172.21.15.119:0/3288348165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2f3683d50 0x55c2f368ffa0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6f68005950 tx=0x7f6f6809a190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66e2fe46e5bd874e server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.218+0000 7f6f557fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:25:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.218+0000 7f6f557fa700 1 -- 172.21.15.119:0/3288348165 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6800cdf0 con 0x55c2f3683d50 2023-12-07T10:25:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.218+0000 7f6f557fa700 1 -- 172.21.15.119:0/3288348165 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f6f6809aeb0 con 0x55c2f3683d50 2023-12-07T10:25:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.218+0000 7f6f87848b80 1 -- 172.21.15.119:0/3288348165 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2f366e6f0 con 0x55c2f3683d50 2023-12-07T10:25:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.218+0000 7f6f557fa700 1 -- 172.21.15.119:0/3288348165 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f680ac980 con 0x55c2f3683d50 2023-12-07T10:25:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.218+0000 7f6f87848b80 10 client.27190 fetch_fsmap learned FSMap version 44 2023-12-07T10:25:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.218+0000 7f6f87848b80 1 -- 172.21.15.119:0/3288348165 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55c2f33f8d80 con 0x55c2f3683d50 2023-12-07T10:25:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.218+0000 7f6f557fa700 1 -- 172.21.15.119:0/3288348165 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c2f366e6f0 con 0x55c2f3683d50 2023-12-07T10:25:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.219+0000 7f6f557fa700 1 -- 172.21.15.119:0/3288348165 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 44) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f6f680cb3e0 con 0x55c2f3683d50 2023-12-07T10:25:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.219+0000 7f6f87848b80 10 client.27190 fetch_fsmap finished waiting for FSMap version 44 2023-12-07T10:25:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.219+0000 7f6f87848b80 10 client.27190 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:25:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.219+0000 7f6f87848b80 20 client.27190 populate_metadata read hostname 'smithi119' 2023-12-07T10:25:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.219+0000 7f6f87848b80 1 --2- 172.21.15.119:0/3288348165 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c2f369d540 0x55c2f369f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.219+0000 7f6f87848b80 4 client.27190 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:25:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.219+0000 7f6f87848b80 1 -- 172.21.15.119:0/3288348165 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2f364a740 con 0x55c2f369d540 2023-12-07T10:25:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.219+0000 7f6f7159a700 1 --2- 172.21.15.119:0/3288348165 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c2f369d540 0x55c2f369f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:19.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.220+0000 7f6f7159a700 1 --2- 172.21.15.119:0/3288348165 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c2f369d540 0x55c2f369f960 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1594e1df6ccb1c7 server_cookie=307d5a5aa97922b2 in_seq=0 out_seq=0 2023-12-07T10:25:19.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.220+0000 7f6f557fa700 10 client.27190 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:25:19.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.220+0000 7f6f557fa700 1 -- 172.21.15.119:0/3288348165 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c2f364a740 con 0x55c2f369d540 2023-12-07T10:25:19.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.220+0000 7f6f557fa700 10 client.27190 handle_command_reply: tid=0 2023-12-07T10:25:19.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.220+0000 7f6f87848b80 1 client.27190 shutdown 2023-12-07T10:25:19.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.221+0000 7f6f87848b80 1 -- 172.21.15.119:0/3288348165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2f3683d50 msgr2=0x55c2f368ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:19.225 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.221+0000 7f6f87848b80 1 --2- 172.21.15.119:0/3288348165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2f3683d50 0x55c2f368ffa0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6f68005950 tx=0x7f6f6809a190 comp rx=0 tx=0).stop 2023-12-07T10:25:19.225 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.221+0000 7f6f87848b80 1 -- 172.21.15.119:0/3288348165 shutdown_connections 2023-12-07T10:25:19.225 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.221+0000 7f6f87848b80 1 --2- 172.21.15.119:0/3288348165 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c2f369d540 0x55c2f369f960 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.225 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.221+0000 7f6f87848b80 1 --2- 172.21.15.119:0/3288348165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f36881a0 0x55c2f368d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.225 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.221+0000 7f6f87848b80 1 --2- 172.21.15.119:0/3288348165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2f3683d50 0x55c2f368ffa0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.225 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.221+0000 7f6f87848b80 1 --2- 172.21.15.119:0/3288348165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2f3684710 0x55c2f368d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.225 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.221+0000 7f6f557fa700 0 client.27190 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:25:19.225 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.221+0000 7f6f87848b80 1 -- 172.21.15.119:0/3288348165 >> 172.21.15.119:0/3288348165 conn(0x55c2f35aa300 msgr2=0x55c2f3670980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:19.225 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.221+0000 7f6f87848b80 1 -- 172.21.15.119:0/3288348165 shutdown_connections 2023-12-07T10:25:19.225 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.221+0000 7f6f87848b80 1 -- 172.21.15.119:0/3288348165 wait complete. 2023-12-07T10:25:19.225 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.221+0000 7f6f87848b80 20 client.27190 trim_cache size 0 max 16384 2023-12-07T10:25:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.238+0000 7f6f87848b80 1 Processor -- start 2023-12-07T10:25:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.238+0000 7f6f87848b80 1 -- start start 2023-12-07T10:25:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.238+0000 7f6f87848b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2f3683d50 0x55c2f3685c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.238+0000 7f6f87848b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f3684710 0x55c2f3684130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.238+0000 7f6f87848b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2f36881a0 0x55c2f36945b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.238+0000 7f6f87848b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f344e8c0 con 0x55c2f36881a0 2023-12-07T10:25:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.238+0000 7f6f87848b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f3677410 con 0x55c2f3684710 2023-12-07T10:25:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.238+0000 7f6f87848b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f3677a60 con 0x55c2f3683d50 2023-12-07T10:25:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.238+0000 7f6f7159a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f3684710 0x55c2f3684130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.238+0000 7f6f7159a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f3684710 0x55c2f3684130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45306/0 (socket says 172.21.15.119:45306) 2023-12-07T10:25:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.238+0000 7f6f7159a700 1 -- 172.21.15.119:0/2942463751 learned_addr learned my addr 172.21.15.119:0/2942463751 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:25:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.238+0000 7f6f7259c700 1 --2- 172.21.15.119:0/2942463751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2f36881a0 0x55c2f36945b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.239+0000 7f6f71d9b700 1 --2- 172.21.15.119:0/2942463751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2f3683d50 0x55c2f3685c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.239+0000 7f6f7159a700 1 -- 172.21.15.119:0/2942463751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2f3683d50 msgr2=0x55c2f3685c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.239+0000 7f6f7159a700 1 --2- 172.21.15.119:0/2942463751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2f3683d50 0x55c2f3685c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.239+0000 7f6f7159a700 1 -- 172.21.15.119:0/2942463751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2f36881a0 msgr2=0x55c2f36945b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.239+0000 7f6f7159a700 1 --2- 172.21.15.119:0/2942463751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2f36881a0 0x55c2f36945b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.239+0000 7f6f7159a700 1 -- 172.21.15.119:0/2942463751 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2f33e1c00 con 0x55c2f3684710 2023-12-07T10:25:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.239+0000 7f6f7159a700 1 --2- 172.21.15.119:0/2942463751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f3684710 0x55c2f3684130 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00f950 tx=0x7f6f5c00f8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=232a0fd8b0450573 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.239+0000 7f6f60ff9700 1 -- 172.21.15.119:0/2942463751 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c01c040 con 0x55c2f3684710 2023-12-07T10:25:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.239+0000 7f6f60ff9700 1 -- 172.21.15.119:0/2942463751 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f5c0058e0 con 0x55c2f3684710 2023-12-07T10:25:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.239+0000 7f6f60ff9700 1 -- 172.21.15.119:0/2942463751 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c031220 con 0x55c2f3684710 2023-12-07T10:25:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.240+0000 7f6f87848b80 1 -- 172.21.15.119:0/2942463751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f3684710 msgr2=0x55c2f3684130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.240+0000 7f6f87848b80 1 --2- 172.21.15.119:0/2942463751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f3684710 0x55c2f3684130 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00f950 tx=0x7f6f5c00f8e0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.240+0000 7f6f87848b80 1 -- 172.21.15.119:0/2942463751 shutdown_connections 2023-12-07T10:25:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.240+0000 7f6f87848b80 1 --2- 172.21.15.119:0/2942463751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2f36881a0 0x55c2f36945b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.240+0000 7f6f87848b80 1 --2- 172.21.15.119:0/2942463751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2f3683d50 0x55c2f3685c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.240+0000 7f6f87848b80 1 --2- 172.21.15.119:0/2942463751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f3684710 0x55c2f3684130 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.240+0000 7f6f87848b80 1 -- 172.21.15.119:0/2942463751 >> 172.21.15.119:0/2942463751 conn(0x55c2f35aa300 msgr2=0x55c2f36a42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:19.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.240+0000 7f6f87848b80 1 -- 172.21.15.119:0/2942463751 shutdown_connections 2023-12-07T10:25:19.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.240+0000 7f6f87848b80 1 -- 172.21.15.119:0/2942463751 wait complete. 2023-12-07T10:25:19.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.241+0000 7f6f87848b80 1 -- start start 2023-12-07T10:25:19.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.241+0000 7f6f87848b80 1 Processor -- start 2023-12-07T10:25:19.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.242+0000 7f6f87848b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f3683d50 0x55c2f36945b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:19.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.242+0000 7f6f87848b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2f3684710 0x55c2f3694af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:19.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.242+0000 7f6f87848b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2f36881a0 0x55c2f36899a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:19.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.242+0000 7f6f87848b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f368a080 con 0x55c2f3684710 2023-12-07T10:25:19.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.242+0000 7f6f87848b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f368a1f0 con 0x55c2f3683d50 2023-12-07T10:25:19.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.242+0000 7f6f87848b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f368a3b0 con 0x55c2f36881a0 2023-12-07T10:25:19.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.242+0000 7f6f87848b80 10 client.0 _ll_register_callbacks cb 0x55c2f35708e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:25:19.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.242+0000 7f6f7259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2f36881a0 0x55c2f36899a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:19.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.242+0000 7f6f7259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2f36881a0 0x55c2f36899a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55228/0 (socket says 172.21.15.119:55228) 2023-12-07T10:25:19.244 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.242+0000 7f6f7259c700 1 -- 172.21.15.119:0/2764844808 learned_addr learned my addr 172.21.15.119:0/2764844808 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:25:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.242+0000 7f6f71d9b700 1 --2- 172.21.15.119:0/2764844808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f3683d50 0x55c2f36945b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.242+0000 7f6f7159a700 1 --2- 172.21.15.119:0/2764844808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2f3684710 0x55c2f3694af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.243+0000 7f6f7259c700 1 -- 172.21.15.119:0/2764844808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f3683d50 msgr2=0x55c2f36945b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.243+0000 7f6f7259c700 1 --2- 172.21.15.119:0/2764844808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f3683d50 0x55c2f36945b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.243+0000 7f6f7259c700 1 -- 172.21.15.119:0/2764844808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2f3684710 msgr2=0x55c2f3694af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.243+0000 7f6f7259c700 1 --2- 172.21.15.119:0/2764844808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2f3684710 0x55c2f3694af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.243+0000 7f6f7259c700 1 -- 172.21.15.119:0/2764844808 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2f344c5d0 con 0x55c2f36881a0 2023-12-07T10:25:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.243+0000 7f6f71d9b700 1 --2- 172.21.15.119:0/2764844808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f3683d50 0x55c2f36945b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:25:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.243+0000 7f6f7259c700 1 --2- 172.21.15.119:0/2764844808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2f36881a0 0x55c2f36899a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009ec0 tx=0x7f6f6400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3174b8578612c5d2 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.243+0000 7f6f557fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:25:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.243+0000 7f6f557fa700 1 -- 172.21.15.119:0/2764844808 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f64020040 con 0x55c2f36881a0 2023-12-07T10:25:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.243+0000 7f6f87848b80 1 -- 172.21.15.119:0/2764844808 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f5c01d040 con 0x55c2f36881a0 2023-12-07T10:25:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.243+0000 7f6f557fa700 1 -- 172.21.15.119:0/2764844808 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f6f6401ad60 con 0x55c2f36881a0 2023-12-07T10:25:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.244+0000 7f6f557fa700 1 -- 172.21.15.119:0/2764844808 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f64027410 con 0x55c2f36881a0 2023-12-07T10:25:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.244+0000 7f6f87848b80 10 client.27196 fetch_fsmap learned FSMap version 44 2023-12-07T10:25:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.244+0000 7f6f87848b80 1 -- 172.21.15.119:0/2764844808 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55c2f341f1f0 con 0x55c2f36881a0 2023-12-07T10:25:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.244+0000 7f6f557fa700 1 -- 172.21.15.119:0/2764844808 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f5c01d040 con 0x55c2f36881a0 2023-12-07T10:25:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.244+0000 7f6f557fa700 1 -- 172.21.15.119:0/2764844808 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 44) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f6f64019600 con 0x55c2f36881a0 2023-12-07T10:25:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.244+0000 7f6f87848b80 10 client.27196 fetch_fsmap finished waiting for FSMap version 44 2023-12-07T10:25:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.244+0000 7f6f87848b80 10 client.27196 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:25:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.244+0000 7f6f87848b80 20 client.27196 populate_metadata read hostname 'smithi119' 2023-12-07T10:25:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.244+0000 7f6f87848b80 1 --2- 172.21.15.119:0/2764844808 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c2f36f41d0 0x55c2f36f65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.244+0000 7f6f87848b80 4 client.27196 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:25:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.244+0000 7f6f87848b80 1 -- 172.21.15.119:0/2764844808 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c2f3595f70 con 0x55c2f36f41d0 2023-12-07T10:25:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.245+0000 7f6f71d9b700 1 --2- 172.21.15.119:0/2764844808 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c2f36f41d0 0x55c2f36f65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.245+0000 7f6f557fa700 10 client.27196 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:25:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.245+0000 7f6f71d9b700 1 --2- 172.21.15.119:0/2764844808 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c2f36f41d0 0x55c2f36f65f0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f108aea09de26545 server_cookie=5f032a03156b5792 in_seq=0 out_seq=0 2023-12-07T10:25:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.276+0000 7f6f557fa700 1 -- 172.21.15.119:0/2764844808 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c2f3595f70 con 0x55c2f36f41d0 2023-12-07T10:25:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.276+0000 7f6f557fa700 10 client.27196 handle_command_reply: tid=0 2023-12-07T10:25:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.276+0000 7f6f87848b80 1 client.27196 shutdown 2023-12-07T10:25:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.276+0000 7f6f87848b80 1 -- 172.21.15.119:0/2764844808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2f36881a0 msgr2=0x55c2f36899a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.276+0000 7f6f87848b80 1 --2- 172.21.15.119:0/2764844808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2f36881a0 0x55c2f36899a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009ec0 tx=0x7f6f6400b040 comp rx=0 tx=0).stop 2023-12-07T10:25:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.276+0000 7f6f87848b80 1 -- 172.21.15.119:0/2764844808 shutdown_connections 2023-12-07T10:25:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.276+0000 7f6f87848b80 1 --2- 172.21.15.119:0/2764844808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c2f36881a0 0x55c2f36899a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.276+0000 7f6f87848b80 1 --2- 172.21.15.119:0/2764844808 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c2f36f41d0 0x55c2f36f65f0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.276+0000 7f6f87848b80 1 --2- 172.21.15.119:0/2764844808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c2f3683d50 0x55c2f36945b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.276+0000 7f6f87848b80 1 --2- 172.21.15.119:0/2764844808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2f3684710 0x55c2f3694af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.276+0000 7f6f557fa700 0 client.27196 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:25:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.276+0000 7f6f87848b80 1 -- 172.21.15.119:0/2764844808 >> 172.21.15.119:0/2764844808 conn(0x55c2f35aa300 msgr2=0x55c2f36864f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.276+0000 7f6f87848b80 1 -- 172.21.15.119:0/2764844808 shutdown_connections 2023-12-07T10:25:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.276+0000 7f6f87848b80 1 -- 172.21.15.119:0/2764844808 wait complete. 2023-12-07T10:25:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.277+0000 7f6f87848b80 20 client.27196 trim_cache size 0 max 16384 2023-12-07T10:25:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:25:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:25:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:25:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:25:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.277+0000 7f6f73fff700 1 -- 172.21.15.119:0/1484472311 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f6f580a1d00 msgr2=0x7f6f580a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.277+0000 7f6f73fff700 1 --2- 172.21.15.119:0/1484472311 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f6f580a1d00 0x7f6f580a41c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x55c2f36763b0 tx=0x7f6f64009920 comp rx=0 tx=0).stop 2023-12-07T10:25:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.277+0000 7f6f73fff700 1 -- 172.21.15.119:0/1484472311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f6c068ce0 msgr2=0x7f6f6c100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.277+0000 7f6f73fff700 1 --2- 172.21.15.119:0/1484472311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f6c068ce0 0x7f6f6c100480 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6f6800e9c0 tx=0x7f6f6800ecd0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.277+0000 7f6f73fff700 1 -- 172.21.15.119:0/1484472311 shutdown_connections 2023-12-07T10:25:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.277+0000 7f6f73fff700 1 --2- 172.21.15.119:0/1484472311 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f6f580a1d00 0x7f6f580a41c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.277+0000 7f6f73fff700 1 --2- 172.21.15.119:0/1484472311 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f6c106a80 0x7f6f6c100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.277+0000 7f6f73fff700 1 --2- 172.21.15.119:0/1484472311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f6c068ce0 0x7f6f6c100480 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.277+0000 7f6f73fff700 1 --2- 172.21.15.119:0/1484472311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f6c1060c0 0x7f6f6c1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.277+0000 7f6f73fff700 1 -- 172.21.15.119:0/1484472311 >> 172.21.15.119:0/1484472311 conn(0x7f6f6c0775b0 msgr2=0x7f6f6c0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.278+0000 7f6f73fff700 1 -- 172.21.15.119:0/1484472311 shutdown_connections 2023-12-07T10:25:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:19.278+0000 7f6f73fff700 1 -- 172.21.15.119:0/1484472311 wait complete. 2023-12-07T10:25:19.555 INFO:tasks.workunit.client.0.smithi119.stdout: 1 319231 7.69 MB/sec execute 92 sec latency 26.556 ms 2023-12-07T10:25:20.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:20 smithi153 ceph-mon[133625]: pgmap v150: 129 pgs: 129 active+clean; 172 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 193 op/s 2023-12-07T10:25:20.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:20 smithi119 ceph-mon[97908]: pgmap v150: 129 pgs: 129 active+clean; 172 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 193 op/s 2023-12-07T10:25:20.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:20 smithi159 ceph-mon[134726]: pgmap v150: 129 pgs: 129 active+clean; 172 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 193 op/s 2023-12-07T10:25:20.555 INFO:tasks.workunit.client.0.smithi119.stdout: 1 320226 7.63 MB/sec execute 93 sec latency 19.379 ms 2023-12-07T10:25:21.555 INFO:tasks.workunit.client.0.smithi119.stdout: 1 322736 7.73 MB/sec execute 94 sec latency 10.735 ms 2023-12-07T10:25:22.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:21 smithi119 ceph-mon[97908]: pgmap v151: 129 pgs: 129 active+clean; 175 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 299 op/s 2023-12-07T10:25:22.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:21 smithi159 ceph-mon[134726]: pgmap v151: 129 pgs: 129 active+clean; 175 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 299 op/s 2023-12-07T10:25:22.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:21 smithi153 ceph-mon[133625]: pgmap v151: 129 pgs: 129 active+clean; 175 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 299 op/s 2023-12-07T10:25:22.555 INFO:tasks.workunit.client.0.smithi119.stdout: 1 323697 7.67 MB/sec execute 95 sec latency 15.348 ms 2023-12-07T10:25:23.555 INFO:tasks.workunit.client.0.smithi119.stdout: 1 326150 7.74 MB/sec execute 96 sec latency 10.701 ms 2023-12-07T10:25:24.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:24 smithi159 ceph-mon[134726]: pgmap v152: 129 pgs: 129 active+clean; 175 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 195 op/s 2023-12-07T10:25:24.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:24 smithi153 ceph-mon[133625]: pgmap v152: 129 pgs: 129 active+clean; 175 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 195 op/s 2023-12-07T10:25:24.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:24 smithi119 ceph-mon[97908]: pgmap v152: 129 pgs: 129 active+clean; 175 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 195 op/s 2023-12-07T10:25:24.555 INFO:tasks.workunit.client.0.smithi119.stdout: 1 327147 7.71 MB/sec execute 97 sec latency 24.836 ms 2023-12-07T10:25:25.555 INFO:tasks.workunit.client.0.smithi119.stdout: 1 329358 7.75 MB/sec execute 98 sec latency 18.937 ms 2023-12-07T10:25:26.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:26 smithi159 ceph-mon[134726]: pgmap v153: 129 pgs: 129 active+clean; 175 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 195 op/s 2023-12-07T10:25:26.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:26 smithi153 ceph-mon[133625]: pgmap v153: 129 pgs: 129 active+clean; 175 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 195 op/s 2023-12-07T10:25:26.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:26 smithi119 ceph-mon[97908]: pgmap v153: 129 pgs: 129 active+clean; 175 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 195 op/s 2023-12-07T10:25:26.555 INFO:tasks.workunit.client.0.smithi119.stdout: 1 330627 7.75 MB/sec execute 99 sec latency 12.510 ms 2023-12-07T10:25:27.555 INFO:tasks.workunit.client.0.smithi119.stdout: 1 332896 7.79 MB/sec execute 100 sec latency 15.184 ms 2023-12-07T10:25:28.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:28 smithi153 ceph-mon[133625]: pgmap v154: 129 pgs: 129 active+clean; 181 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 285 op/s 2023-12-07T10:25:28.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:28 smithi119 ceph-mon[97908]: pgmap v154: 129 pgs: 129 active+clean; 181 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 285 op/s 2023-12-07T10:25:28.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:28 smithi159 ceph-mon[134726]: pgmap v154: 129 pgs: 129 active+clean; 181 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 285 op/s 2023-12-07T10:25:28.555 INFO:tasks.workunit.client.0.smithi119.stdout: 1 334095 7.78 MB/sec execute 101 sec latency 15.830 ms 2023-12-07T10:25:29.555 INFO:tasks.workunit.client.0.smithi119.stdout: 1 336306 7.81 MB/sec execute 102 sec latency 20.869 ms 2023-12-07T10:25:30.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:30 smithi153 ceph-mon[133625]: pgmap v155: 129 pgs: 129 active+clean; 181 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 195 op/s 2023-12-07T10:25:30.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:30 smithi119 ceph-mon[97908]: pgmap v155: 129 pgs: 129 active+clean; 181 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 195 op/s 2023-12-07T10:25:30.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:30 smithi159 ceph-mon[134726]: pgmap v155: 129 pgs: 129 active+clean; 181 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 195 op/s 2023-12-07T10:25:30.556 INFO:tasks.workunit.client.0.smithi119.stdout: 1 337506 7.81 MB/sec execute 103 sec latency 19.040 ms 2023-12-07T10:25:31.555 INFO:tasks.workunit.client.0.smithi119.stdout: 1 339573 7.82 MB/sec execute 104 sec latency 14.033 ms 2023-12-07T10:25:32.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:31 smithi119 ceph-mon[97908]: pgmap v156: 129 pgs: 129 active+clean; 188 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 286 op/s 2023-12-07T10:25:32.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:31 smithi159 ceph-mon[134726]: pgmap v156: 129 pgs: 129 active+clean; 188 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 286 op/s 2023-12-07T10:25:32.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:31 smithi153 ceph-mon[133625]: pgmap v156: 129 pgs: 129 active+clean; 188 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 286 op/s 2023-12-07T10:25:32.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.440+0000 7f02b83ff700 1 -- 172.21.15.119:0/2466032492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02b0101ee0 msgr2=0x7f02b010e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:32.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.440+0000 7f02b83ff700 1 --2- 172.21.15.119:0/2466032492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02b0101ee0 0x7f02b010e3a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f02a800b0a0 tx=0x7f02a800b3b0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.440+0000 7f02b83ff700 1 -- 172.21.15.119:0/2466032492 shutdown_connections 2023-12-07T10:25:32.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.440+0000 7f02b83ff700 1 --2- 172.21.15.119:0/2466032492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02b0101ee0 0x7f02b010e3a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.440+0000 7f02b83ff700 1 --2- 172.21.15.119:0/2466032492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02b01075e0 0x7f02b01079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.440+0000 7f02b83ff700 1 --2- 172.21.15.119:0/2466032492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02b0101520 0x7f02b01019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.440+0000 7f02b83ff700 1 -- 172.21.15.119:0/2466032492 >> 172.21.15.119:0/2466032492 conn(0x7f02b00fd010 msgr2=0x7f02b00ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:32.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.440+0000 7f02b83ff700 1 -- 172.21.15.119:0/2466032492 shutdown_connections 2023-12-07T10:25:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.440+0000 7f02b83ff700 1 -- 172.21.15.119:0/2466032492 wait complete. 2023-12-07T10:25:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.441+0000 7f02b83ff700 1 Processor -- start 2023-12-07T10:25:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.441+0000 7f02b83ff700 1 -- start start 2023-12-07T10:25:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.442+0000 7f02b83ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02b0101520 0x7f02b019d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.442+0000 7f02b83ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02b0101ee0 0x7f02b019dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.442+0000 7f02b83ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02b01075e0 0x7f02b0197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.442+0000 7f02b83ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b0197ec0 con 0x7f02b0101520 2023-12-07T10:25:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.442+0000 7f02b83ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b0198030 con 0x7f02b0101ee0 2023-12-07T10:25:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.442+0000 7f02b83ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b01981f0 con 0x7f02b01075e0 2023-12-07T10:25:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.442+0000 7f02b599a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02b0101ee0 0x7f02b019dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.442+0000 7f02b699c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02b01075e0 0x7f02b0197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.442+0000 7f02b599a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02b0101ee0 0x7f02b019dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:46816/0 (socket says 172.21.15.119:46816) 2023-12-07T10:25:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.442+0000 7f02b599a700 1 -- 172.21.15.119:0/702229018 learned_addr learned my addr 172.21.15.119:0/702229018 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:25:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.442+0000 7f02b619b700 1 --2- 172.21.15.119:0/702229018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02b0101520 0x7f02b019d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.442+0000 7f02b599a700 1 -- 172.21.15.119:0/702229018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02b01075e0 msgr2=0x7f02b0197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.442+0000 7f02b599a700 1 --2- 172.21.15.119:0/702229018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02b01075e0 0x7f02b0197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.442+0000 7f02b599a700 1 -- 172.21.15.119:0/702229018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02b0101520 msgr2=0x7f02b019d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.442+0000 7f02b599a700 1 --2- 172.21.15.119:0/702229018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02b0101520 0x7f02b019d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.442+0000 7f02b599a700 1 -- 172.21.15.119:0/702229018 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a8009d00 con 0x7f02b0101ee0 2023-12-07T10:25:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.442+0000 7f02b699c700 1 --2- 172.21.15.119:0/702229018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02b01075e0 0x7f02b0197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:25:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.443+0000 7f02b599a700 1 --2- 172.21.15.119:0/702229018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02b0101ee0 0x7f02b019dc60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f02a000e9a0 tx=0x7f02a000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.443+0000 7f02a77fe700 1 -- 172.21.15.119:0/702229018 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a0009790 con 0x7f02b0101ee0 2023-12-07T10:25:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.443+0000 7f02b83ff700 1 -- 172.21.15.119:0/702229018 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b01984d0 con 0x7f02b0101ee0 2023-12-07T10:25:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.443+0000 7f02a77fe700 1 -- 172.21.15.119:0/702229018 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02a0010430 con 0x7f02b0101ee0 2023-12-07T10:25:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.443+0000 7f02a77fe700 1 -- 172.21.15.119:0/702229018 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a0018680 con 0x7f02b0101ee0 2023-12-07T10:25:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.443+0000 7f02b83ff700 1 -- 172.21.15.119:0/702229018 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b01a4710 con 0x7f02b0101ee0 2023-12-07T10:25:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.447+0000 7f02a77fe700 1 -- 172.21.15.119:0/702229018 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f02a00105a0 con 0x7f02b0101ee0 2023-12-07T10:25:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.447+0000 7f02cbcb7b80 1 Processor -- start 2023-12-07T10:25:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.448+0000 7f02cbcb7b80 1 -- start start 2023-12-07T10:25:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.448+0000 7f02cbcb7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b419eb1da0 0x55b419eb2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.448+0000 7f02cbcb7b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b419ead950 0x55b419eaddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.448+0000 7f02cbcb7b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419eae310 0x55b419eb9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.448+0000 7f02cbcb7b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b419e1d760 con 0x55b419ead950 2023-12-07T10:25:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.448+0000 7f02cbcb7b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b419e04740 con 0x55b419eae310 2023-12-07T10:25:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.448+0000 7f02cbcb7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b419e02060 con 0x55b419eb1da0 2023-12-07T10:25:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.449+0000 7f02b599a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419eae310 0x55b419eb9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.449+0000 7f02b699c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b419eb1da0 0x55b419eb2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.449+0000 7f02b699c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b419eb1da0 0x55b419eb2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:52072/0 (socket says 172.21.15.119:52072) 2023-12-07T10:25:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.449+0000 7f02b699c700 1 -- 172.21.15.119:0/1385164958 learned_addr learned my addr 172.21.15.119:0/1385164958 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:25:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.449+0000 7f02b599a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419eae310 0x55b419eb9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:46818/0 (socket says 172.21.15.119:46818) 2023-12-07T10:25:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.449+0000 7f02b619b700 1 --2- 172.21.15.119:0/1385164958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b419ead950 0x55b419eaddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.449+0000 7f02b699c700 1 -- 172.21.15.119:0/1385164958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419eae310 msgr2=0x55b419eb9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.449+0000 7f02b699c700 1 --2- 172.21.15.119:0/1385164958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419eae310 0x55b419eb9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.449+0000 7f02b699c700 1 -- 172.21.15.119:0/1385164958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b419ead950 msgr2=0x55b419eaddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.449+0000 7f02b699c700 1 --2- 172.21.15.119:0/1385164958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b419ead950 0x55b419eaddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.449+0000 7f02b699c700 1 -- 172.21.15.119:0/1385164958 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b419c7a5d0 con 0x55b419eb1da0 2023-12-07T10:25:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.449+0000 7f02a77fe700 1 --2- 172.21.15.119:0/702229018 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f029c0a1c80 0x7f029c0a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.449+0000 7f02a77fe700 1 -- 172.21.15.119:0/702229018 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f02a0014070 con 0x7f02b0101ee0 2023-12-07T10:25:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.449+0000 7f02b699c700 1 --2- 172.21.15.119:0/1385164958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b419eb1da0 0x55b419eb2180 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f02a800b070 tx=0x7f02a8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f84f780e716bf088 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.449+0000 7f02b619b700 1 --2- 172.21.15.119:0/702229018 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f029c0a1c80 0x7f029c0a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.450+0000 7f02a4bfe700 1 -- 172.21.15.119:0/1385164958 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a800e040 con 0x55b419eb1da0 2023-12-07T10:25:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.450+0000 7f02a4bfe700 1 -- 172.21.15.119:0/1385164958 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02a8003a00 con 0x55b419eb1da0 2023-12-07T10:25:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.450+0000 7f02a4bfe700 1 -- 172.21.15.119:0/1385164958 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a8005040 con 0x55b419eb1da0 2023-12-07T10:25:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.450+0000 7f02cbcb7b80 1 -- 172.21.15.119:0/1385164958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b419eb1da0 msgr2=0x55b419eb2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.450+0000 7f02cbcb7b80 1 --2- 172.21.15.119:0/1385164958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b419eb1da0 0x55b419eb2180 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f02a800b070 tx=0x7f02a8009b80 comp rx=0 tx=0).stop 2023-12-07T10:25:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.450+0000 7f02b619b700 1 --2- 172.21.15.119:0/702229018 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f029c0a1c80 0x7f029c0a4140 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x55b419ea57e0 tx=0x7f02ac009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.451+0000 7f02cbcb7b80 1 -- 172.21.15.119:0/1385164958 shutdown_connections 2023-12-07T10:25:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.451+0000 7f02cbcb7b80 1 --2- 172.21.15.119:0/1385164958 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419eae310 0x55b419eb9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.451+0000 7f02cbcb7b80 1 --2- 172.21.15.119:0/1385164958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b419eb1da0 0x55b419eb2180 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.451+0000 7f02cbcb7b80 1 --2- 172.21.15.119:0/1385164958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b419ead950 0x55b419eaddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.451+0000 7f02cbcb7b80 1 -- 172.21.15.119:0/1385164958 >> 172.21.15.119:0/1385164958 conn(0x55b419dd8300 msgr2=0x55b419ea3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.451+0000 7f02cbcb7b80 1 -- 172.21.15.119:0/1385164958 shutdown_connections 2023-12-07T10:25:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.451+0000 7f02cbcb7b80 1 -- 172.21.15.119:0/1385164958 wait complete. 2023-12-07T10:25:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.451+0000 7f02cbcb7b80 1 -- start start 2023-12-07T10:25:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.452+0000 7f02cbcb7b80 1 Processor -- start 2023-12-07T10:25:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.452+0000 7f02cbcb7b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419ead950 0x55b419eb9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.453+0000 7f02cbcb7b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b419eae310 0x55b419eb6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.453+0000 7f02cbcb7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b419eb1da0 0x55b419eb7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.453+0000 7f02cbcb7b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b419dfd950 con 0x55b419eae310 2023-12-07T10:25:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.453+0000 7f02cbcb7b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b419db7500 con 0x55b419ead950 2023-12-07T10:25:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.453+0000 7f02cbcb7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b419db71a0 con 0x55b419eb1da0 2023-12-07T10:25:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.453+0000 7f02cbcb7b80 10 client.0 _ll_register_callbacks cb 0x55b419d9e8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:25:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.453+0000 7f02b619b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419ead950 0x55b419eb9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.453+0000 7f02b619b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419ead950 0x55b419eb9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:46820/0 (socket says 172.21.15.119:46820) 2023-12-07T10:25:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.453+0000 7f02b619b700 1 -- 172.21.15.119:0/1775553325 learned_addr learned my addr 172.21.15.119:0/1775553325 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:25:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.453+0000 7f02b699c700 1 --2- 172.21.15.119:0/1775553325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b419eb1da0 0x55b419eb7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.453+0000 7f02b599a700 1 --2- 172.21.15.119:0/1775553325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b419eae310 0x55b419eb6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.453+0000 7f02b699c700 1 -- 172.21.15.119:0/1775553325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419ead950 msgr2=0x55b419eb9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.453+0000 7f02b699c700 1 --2- 172.21.15.119:0/1775553325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419ead950 0x55b419eb9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.453+0000 7f02b699c700 1 -- 172.21.15.119:0/1775553325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b419eae310 msgr2=0x55b419eb6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.453+0000 7f02b699c700 1 --2- 172.21.15.119:0/1775553325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b419eae310 0x55b419eb6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.453+0000 7f02b699c700 1 -- 172.21.15.119:0/1775553325 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b010eca0 con 0x55b419eb1da0 2023-12-07T10:25:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.453+0000 7f02b619b700 1 --2- 172.21.15.119:0/1775553325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419ead950 0x55b419eb9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:25:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.454+0000 7f02b699c700 1 --2- 172.21.15.119:0/1775553325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b419eb1da0 0x55b419eb7720 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f02a80062a0 tx=0x7f02a8003e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ceaf7dcb8c1ce3d2 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.454+0000 7f0299ffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:25:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.454+0000 7f0299ffb700 1 -- 172.21.15.119:0/1775553325 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a800e070 con 0x55b419eb1da0 2023-12-07T10:25:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.454+0000 7f02cbcb7b80 1 -- 172.21.15.119:0/1775553325 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b419e9c6f0 con 0x55b419eb1da0 2023-12-07T10:25:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.454+0000 7f0299ffb700 1 -- 172.21.15.119:0/1775553325 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f02a80050b0 con 0x55b419eb1da0 2023-12-07T10:25:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.454+0000 7f0299ffb700 1 -- 172.21.15.119:0/1775553325 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02a8009440 con 0x55b419eb1da0 2023-12-07T10:25:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.454+0000 7f02cbcb7b80 10 client.27208 fetch_fsmap learned FSMap version 44 2023-12-07T10:25:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.454+0000 7f02cbcb7b80 1 -- 172.21.15.119:0/1775553325 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55b419c26d80 con 0x55b419eb1da0 2023-12-07T10:25:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.454+0000 7f0299ffb700 1 -- 172.21.15.119:0/1775553325 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b419e9c6f0 con 0x55b419eb1da0 2023-12-07T10:25:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.455+0000 7f0299ffb700 1 -- 172.21.15.119:0/1775553325 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 44) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f02a800b0d0 con 0x55b419eb1da0 2023-12-07T10:25:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.455+0000 7f02cbcb7b80 10 client.27208 fetch_fsmap finished waiting for FSMap version 44 2023-12-07T10:25:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.455+0000 7f02cbcb7b80 10 client.27208 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:25:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.455+0000 7f02cbcb7b80 20 client.27208 populate_metadata read hostname 'smithi119' 2023-12-07T10:25:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.455+0000 7f02cbcb7b80 1 --2- 172.21.15.119:0/1775553325 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55b419ec6f40 0x55b419ec9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.455+0000 7f02cbcb7b80 4 client.27208 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:25:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.455+0000 7f02cbcb7b80 1 -- 172.21.15.119:0/1775553325 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b419e78740 con 0x55b419ec6f40 2023-12-07T10:25:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.455+0000 7f02b619b700 1 --2- 172.21.15.119:0/1775553325 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55b419ec6f40 0x55b419ec9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.456+0000 7f02b619b700 1 --2- 172.21.15.119:0/1775553325 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55b419ec6f40 0x55b419ec9360 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da1e7e062751319b server_cookie=cba386fb0b5ad6ed in_seq=0 out_seq=0 2023-12-07T10:25:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.456+0000 7f0299ffb700 10 client.27208 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:25:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.457+0000 7f0299ffb700 1 -- 172.21.15.119:0/1775553325 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b419e78740 con 0x55b419ec6f40 2023-12-07T10:25:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.457+0000 7f0299ffb700 10 client.27208 handle_command_reply: tid=0 2023-12-07T10:25:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.457+0000 7f02cbcb7b80 1 client.27208 shutdown 2023-12-07T10:25:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.457+0000 7f02cbcb7b80 1 -- 172.21.15.119:0/1775553325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b419eb1da0 msgr2=0x55b419eb7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.457+0000 7f02cbcb7b80 1 --2- 172.21.15.119:0/1775553325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b419eb1da0 0x55b419eb7720 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f02a80062a0 tx=0x7f02a8003e20 comp rx=0 tx=0).stop 2023-12-07T10:25:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.458+0000 7f02cbcb7b80 1 -- 172.21.15.119:0/1775553325 shutdown_connections 2023-12-07T10:25:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.458+0000 7f02cbcb7b80 1 --2- 172.21.15.119:0/1775553325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b419eb1da0 0x55b419eb7720 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.458+0000 7f02cbcb7b80 1 --2- 172.21.15.119:0/1775553325 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55b419ec6f40 0x55b419ec9360 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.458+0000 7f02cbcb7b80 1 --2- 172.21.15.119:0/1775553325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419ead950 0x55b419eb9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.458+0000 7f02cbcb7b80 1 --2- 172.21.15.119:0/1775553325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b419eae310 0x55b419eb6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.458+0000 7f0299ffb700 0 client.27208 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:25:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.458+0000 7f02cbcb7b80 1 -- 172.21.15.119:0/1775553325 >> 172.21.15.119:0/1775553325 conn(0x55b419dd8300 msgr2=0x55b419e9e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.458+0000 7f02cbcb7b80 1 -- 172.21.15.119:0/1775553325 shutdown_connections 2023-12-07T10:25:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.458+0000 7f02cbcb7b80 1 -- 172.21.15.119:0/1775553325 wait complete. 2023-12-07T10:25:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.458+0000 7f02cbcb7b80 20 client.27208 trim_cache size 0 max 16384 2023-12-07T10:25:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.476+0000 7f02cbcb7b80 1 Processor -- start 2023-12-07T10:25:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.477+0000 7f02cbcb7b80 1 -- start start 2023-12-07T10:25:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.477+0000 7f02cbcb7b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b419ead950 0x55b419e9d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.477+0000 7f02cbcb7b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419eae310 0x55b419ecdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.477+0000 7f02cbcb7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b419eb1da0 0x55b419ebd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.477+0000 7f02cbcb7b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b419c7c8c0 con 0x55b419ead950 2023-12-07T10:25:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.477+0000 7f02cbcb7b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b419ea5a60 con 0x55b419eae310 2023-12-07T10:25:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.477+0000 7f02cbcb7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b419ea4e60 con 0x55b419eb1da0 2023-12-07T10:25:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.477+0000 7f02b599a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419eae310 0x55b419ecdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.477+0000 7f02b619b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b419ead950 0x55b419e9d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.477+0000 7f02b599a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419eae310 0x55b419ecdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:46832/0 (socket says 172.21.15.119:46832) 2023-12-07T10:25:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.477+0000 7f02b619b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b419ead950 0x55b419e9d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46262/0 (socket says 172.21.15.119:46262) 2023-12-07T10:25:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.477+0000 7f02b599a700 1 -- 172.21.15.119:0/4175291427 learned_addr learned my addr 172.21.15.119:0/4175291427 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:25:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.477+0000 7f02b699c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b419eb1da0 0x55b419ebd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.478+0000 7f02b599a700 1 -- 172.21.15.119:0/4175291427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b419eb1da0 msgr2=0x55b419ebd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.478+0000 7f02b599a700 1 --2- 172.21.15.119:0/4175291427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b419eb1da0 0x55b419ebd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.478+0000 7f02b599a700 1 -- 172.21.15.119:0/4175291427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b419ead950 msgr2=0x55b419e9d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.478+0000 7f02b599a700 1 --2- 172.21.15.119:0/4175291427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b419ead950 0x55b419e9d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.478+0000 7f02b599a700 1 -- 172.21.15.119:0/4175291427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b419c0fc00 con 0x55b419eae310 2023-12-07T10:25:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.478+0000 7f02b619b700 1 --2- 172.21.15.119:0/4175291427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b419ead950 0x55b419e9d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:25:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.478+0000 7f02b599a700 1 --2- 172.21.15.119:0/4175291427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419eae310 0x55b419ecdc80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f02a0097370 tx=0x7f02a00d17c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b98aa30f9386dc7d server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.478+0000 7f02a53ff700 1 -- 172.21.15.119:0/4175291427 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a009a6d0 con 0x55b419eae310 2023-12-07T10:25:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.478+0000 7f02a53ff700 1 -- 172.21.15.119:0/4175291427 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02a009a440 con 0x55b419eae310 2023-12-07T10:25:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.478+0000 7f02a53ff700 1 -- 172.21.15.119:0/4175291427 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a00a2d20 con 0x55b419eae310 2023-12-07T10:25:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.479+0000 7f02cbcb7b80 1 -- 172.21.15.119:0/4175291427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419eae310 msgr2=0x55b419ecdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.479+0000 7f02cbcb7b80 1 --2- 172.21.15.119:0/4175291427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419eae310 0x55b419ecdc80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f02a0097370 tx=0x7f02a00d17c0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.479+0000 7f02cbcb7b80 1 -- 172.21.15.119:0/4175291427 shutdown_connections 2023-12-07T10:25:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.479+0000 7f02cbcb7b80 1 --2- 172.21.15.119:0/4175291427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b419eb1da0 0x55b419ebd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.479+0000 7f02cbcb7b80 1 --2- 172.21.15.119:0/4175291427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b419ead950 0x55b419e9d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.479+0000 7f02cbcb7b80 1 --2- 172.21.15.119:0/4175291427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419eae310 0x55b419ecdc80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.479+0000 7f02cbcb7b80 1 -- 172.21.15.119:0/4175291427 >> 172.21.15.119:0/4175291427 conn(0x55b419dd8300 msgr2=0x55b419eb6870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.479+0000 7f02cbcb7b80 1 -- 172.21.15.119:0/4175291427 shutdown_connections 2023-12-07T10:25:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.479+0000 7f02cbcb7b80 1 -- 172.21.15.119:0/4175291427 wait complete. 2023-12-07T10:25:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.480+0000 7f02cbcb7b80 1 -- start start 2023-12-07T10:25:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.480+0000 7f02cbcb7b80 1 Processor -- start 2023-12-07T10:25:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.480+0000 7f02cbcb7b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419ead950 0x55b419ebddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.481+0000 7f02cbcb7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b419eae310 0x55b419eb45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.481+0000 7f02cbcb7b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b419eb1da0 0x55b419eb4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.481+0000 7f02cbcb7b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b419eb5910 con 0x55b419eb1da0 2023-12-07T10:25:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.481+0000 7f02cbcb7b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b419eb5a80 con 0x55b419ead950 2023-12-07T10:25:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.481+0000 7f02cbcb7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b419eb5c40 con 0x55b419eae310 2023-12-07T10:25:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.481+0000 7f02cbcb7b80 10 client.0 _ll_register_callbacks cb 0x55b419d9e8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:25:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.481+0000 7f02b619b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419ead950 0x55b419ebddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.481+0000 7f02b619b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419ead950 0x55b419ebddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:46834/0 (socket says 172.21.15.119:46834) 2023-12-07T10:25:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.481+0000 7f02b619b700 1 -- 172.21.15.119:0/2628805936 learned_addr learned my addr 172.21.15.119:0/2628805936 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:25:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.481+0000 7f02b699c700 1 --2- 172.21.15.119:0/2628805936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b419eb1da0 0x55b419eb4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.481+0000 7f02b599a700 1 --2- 172.21.15.119:0/2628805936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b419eae310 0x55b419eb45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.481+0000 7f02b619b700 1 -- 172.21.15.119:0/2628805936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b419eae310 msgr2=0x55b419eb45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.481+0000 7f02b619b700 1 --2- 172.21.15.119:0/2628805936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b419eae310 0x55b419eb45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.481+0000 7f02b619b700 1 -- 172.21.15.119:0/2628805936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b419eb1da0 msgr2=0x55b419eb4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.481+0000 7f02b619b700 1 --2- 172.21.15.119:0/2628805936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b419eb1da0 0x55b419eb4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.481+0000 7f02b619b700 1 -- 172.21.15.119:0/2628805936 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ac012e30 con 0x55b419ead950 2023-12-07T10:25:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.481+0000 7f02b699c700 1 --2- 172.21.15.119:0/2628805936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b419eb1da0 0x55b419eb4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:25:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.482+0000 7f02b619b700 1 --2- 172.21.15.119:0/2628805936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419ead950 0x55b419ebddb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f02ac0095c0 tx=0x7f02ac004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5347c99dcf1eba4 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.482+0000 7f0299ffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:25:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.482+0000 7f0299ffb700 1 -- 172.21.15.119:0/2628805936 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac025040 con 0x55b419ead950 2023-12-07T10:25:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.482+0000 7f02cbcb7b80 1 -- 172.21.15.119:0/2628805936 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02a8003a00 con 0x55b419ead950 2023-12-07T10:25:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.482+0000 7f0299ffb700 1 -- 172.21.15.119:0/2628805936 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f02ac003740 con 0x55b419ead950 2023-12-07T10:25:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.482+0000 7f0299ffb700 1 -- 172.21.15.119:0/2628805936 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02ac017070 con 0x55b419ead950 2023-12-07T10:25:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.482+0000 7f02cbcb7b80 10 client.27230 fetch_fsmap learned FSMap version 44 2023-12-07T10:25:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.482+0000 7f02cbcb7b80 1 -- 172.21.15.119:0/2628805936 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f02b01a4710 con 0x55b419ead950 2023-12-07T10:25:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.482+0000 7f0299ffb700 1 -- 172.21.15.119:0/2628805936 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02a8003a00 con 0x55b419ead950 2023-12-07T10:25:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.483+0000 7f0299ffb700 1 -- 172.21.15.119:0/2628805936 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 44) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f02ac01fb00 con 0x55b419ead950 2023-12-07T10:25:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.483+0000 7f02cbcb7b80 10 client.27230 fetch_fsmap finished waiting for FSMap version 44 2023-12-07T10:25:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.483+0000 7f02cbcb7b80 10 client.27230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:25:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.483+0000 7f02cbcb7b80 20 client.27230 populate_metadata read hostname 'smithi119' 2023-12-07T10:25:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.483+0000 7f02cbcb7b80 1 --2- 172.21.15.119:0/2628805936 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55b419f1e070 0x55b419f20490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.483+0000 7f02cbcb7b80 4 client.27230 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:25:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.483+0000 7f02cbcb7b80 1 -- 172.21.15.119:0/2628805936 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b419dc3f70 con 0x55b419f1e070 2023-12-07T10:25:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.483+0000 7f02b599a700 1 --2- 172.21.15.119:0/2628805936 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55b419f1e070 0x55b419f20490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.483+0000 7f0299ffb700 10 client.27230 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:25:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.483+0000 7f02b599a700 1 --2- 172.21.15.119:0/2628805936 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55b419f1e070 0x55b419f20490 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afa9fe52444a7cba server_cookie=920986409676317c in_seq=0 out_seq=0 2023-12-07T10:25:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.538+0000 7f0299ffb700 1 -- 172.21.15.119:0/2628805936 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b419dc3f70 con 0x55b419f1e070 2023-12-07T10:25:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.538+0000 7f0299ffb700 10 client.27230 handle_command_reply: tid=0 2023-12-07T10:25:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.538+0000 7f02cbcb7b80 1 client.27230 shutdown 2023-12-07T10:25:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.538+0000 7f02cbcb7b80 1 -- 172.21.15.119:0/2628805936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419ead950 msgr2=0x55b419ebddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.538+0000 7f02cbcb7b80 1 --2- 172.21.15.119:0/2628805936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419ead950 0x55b419ebddb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f02ac0095c0 tx=0x7f02ac004750 comp rx=0 tx=0).stop 2023-12-07T10:25:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.538+0000 7f02cbcb7b80 1 -- 172.21.15.119:0/2628805936 shutdown_connections 2023-12-07T10:25:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.538+0000 7f02cbcb7b80 1 --2- 172.21.15.119:0/2628805936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b419eb1da0 0x55b419eb4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.538+0000 7f02cbcb7b80 1 --2- 172.21.15.119:0/2628805936 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55b419f1e070 0x55b419f20490 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.538+0000 7f02cbcb7b80 1 --2- 172.21.15.119:0/2628805936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b419ead950 0x55b419ebddb0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.538+0000 7f02cbcb7b80 1 --2- 172.21.15.119:0/2628805936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b419eae310 0x55b419eb45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.538+0000 7f0299ffb700 0 client.27230 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:25:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.538+0000 7f02cbcb7b80 1 -- 172.21.15.119:0/2628805936 >> 172.21.15.119:0/2628805936 conn(0x55b419dd8300 msgr2=0x55b419ecdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.538+0000 7f02cbcb7b80 1 -- 172.21.15.119:0/2628805936 shutdown_connections 2023-12-07T10:25:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.539+0000 7f02cbcb7b80 1 -- 172.21.15.119:0/2628805936 wait complete. 2023-12-07T10:25:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.539+0000 7f02cbcb7b80 20 client.27230 trim_cache size 0 max 16384 2023-12-07T10:25:32.541 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:25:32.541 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:25:32.541 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:25:32.541 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:25:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.539+0000 7f02b83ff700 1 -- 172.21.15.119:0/702229018 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f029c0a1c80 msgr2=0x7f029c0a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.539+0000 7f02b83ff700 1 --2- 172.21.15.119:0/702229018 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f029c0a1c80 0x7f029c0a4140 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x55b419ea57e0 tx=0x7f02ac009260 comp rx=0 tx=0).stop 2023-12-07T10:25:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.539+0000 7f02b83ff700 1 -- 172.21.15.119:0/702229018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02b0101ee0 msgr2=0x7f02b019dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.539+0000 7f02b83ff700 1 --2- 172.21.15.119:0/702229018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02b0101ee0 0x7f02b019dc60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f02a000e9a0 tx=0x7f02a000ecb0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.539+0000 7f02b83ff700 1 -- 172.21.15.119:0/702229018 shutdown_connections 2023-12-07T10:25:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.539+0000 7f02b83ff700 1 --2- 172.21.15.119:0/702229018 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f029c0a1c80 0x7f029c0a4140 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.539+0000 7f02b83ff700 1 --2- 172.21.15.119:0/702229018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02b01075e0 0x7f02b0197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.539+0000 7f02b83ff700 1 --2- 172.21.15.119:0/702229018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02b0101520 0x7f02b019d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.539+0000 7f02b83ff700 1 --2- 172.21.15.119:0/702229018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02b0101ee0 0x7f02b019dc60 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.539+0000 7f02b83ff700 1 -- 172.21.15.119:0/702229018 >> 172.21.15.119:0/702229018 conn(0x7f02b00fd010 msgr2=0x7f02b00fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.540+0000 7f02b83ff700 1 -- 172.21.15.119:0/702229018 shutdown_connections 2023-12-07T10:25:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:32.540+0000 7f02b83ff700 1 -- 172.21.15.119:0/702229018 wait complete. 2023-12-07T10:25:32.555 INFO:tasks.workunit.client.0.smithi119.stdout: 1 340945 7.85 MB/sec execute 105 sec latency 15.353 ms 2023-12-07T10:25:33.555 INFO:tasks.workunit.client.0.smithi119.stdout: 1 342625 7.83 MB/sec execute 106 sec latency 49.949 ms 2023-12-07T10:25:34.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:34 smithi159 ceph-mon[134726]: pgmap v157: 129 pgs: 129 active+clean; 188 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 180 op/s 2023-12-07T10:25:34.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:34 smithi153 ceph-mon[133625]: pgmap v157: 129 pgs: 129 active+clean; 188 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 180 op/s 2023-12-07T10:25:34.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:34 smithi119 ceph-mon[97908]: pgmap v157: 129 pgs: 129 active+clean; 188 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 180 op/s 2023-12-07T10:25:34.558 INFO:tasks.workunit.client.0.smithi119.stdout: 1 343889 7.84 MB/sec execute 107 sec latency 310.573 ms 2023-12-07T10:25:35.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:35 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:25:35.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:35 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:25:35.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:35 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:25:35.556 INFO:tasks.workunit.client.0.smithi119.stdout: 1 344687 7.81 MB/sec execute 108 sec latency 447.735 ms 2023-12-07T10:25:36.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:36 smithi153 ceph-mon[133625]: pgmap v158: 129 pgs: 129 active+clean; 188 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 180 op/s 2023-12-07T10:25:36.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:36 smithi119 ceph-mon[97908]: pgmap v158: 129 pgs: 129 active+clean; 188 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 180 op/s 2023-12-07T10:25:36.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:36 smithi159 ceph-mon[134726]: pgmap v158: 129 pgs: 129 active+clean; 188 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 180 op/s 2023-12-07T10:25:36.556 INFO:tasks.workunit.client.0.smithi119.stdout: 1 346625 7.81 MB/sec execute 109 sec latency 24.044 ms 2023-12-07T10:25:37.556 INFO:tasks.workunit.client.0.smithi119.stdout: 1 348055 7.84 MB/sec execute 110 sec latency 19.938 ms 2023-12-07T10:25:38.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:38 smithi153 ceph-mon[133625]: pgmap v159: 129 pgs: 129 active+clean; 207 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 362 op/s 2023-12-07T10:25:38.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:38 smithi119 ceph-mon[97908]: pgmap v159: 129 pgs: 129 active+clean; 207 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 362 op/s 2023-12-07T10:25:38.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:38 smithi159 ceph-mon[134726]: pgmap v159: 129 pgs: 129 active+clean; 207 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 362 op/s 2023-12-07T10:25:38.556 INFO:tasks.workunit.client.0.smithi119.stdout: 1 349482 7.82 MB/sec execute 111 sec latency 236.236 ms 2023-12-07T10:25:39.556 INFO:tasks.workunit.client.0.smithi119.stdout: 1 349482 7.75 MB/sec execute 112 sec latency 1236.313 ms 2023-12-07T10:25:40.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:40 smithi153 ceph-mon[133625]: pgmap v160: 129 pgs: 129 active+clean; 207 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 273 op/s 2023-12-07T10:25:40.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:40 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:25:40.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:40 smithi119 ceph-mon[97908]: pgmap v160: 129 pgs: 129 active+clean; 207 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 273 op/s 2023-12-07T10:25:40.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:40 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:25:40.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:40 smithi159 ceph-mon[134726]: pgmap v160: 129 pgs: 129 active+clean; 207 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 273 op/s 2023-12-07T10:25:40.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:40 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:25:40.556 INFO:tasks.workunit.client.0.smithi119.stdout: 1 349482 7.68 MB/sec execute 113 sec latency 2236.497 ms 2023-12-07T10:25:41.556 INFO:tasks.workunit.client.0.smithi119.stdout: 1 349482 7.61 MB/sec execute 114 sec latency 3236.622 ms 2023-12-07T10:25:42.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:41 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:25:42.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:41 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:25:42.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:41 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:25:42.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:41 smithi119 ceph-mon[97908]: pgmap v161: 129 pgs: 129 active+clean; 210 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.7 MiB/s wr, 370 op/s 2023-12-07T10:25:42.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:41 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:25:42.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:41 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:25:42.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:41 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:25:42.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:41 smithi159 ceph-mon[134726]: pgmap v161: 129 pgs: 129 active+clean; 210 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.7 MiB/s wr, 370 op/s 2023-12-07T10:25:42.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:41 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:25:42.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:41 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:25:42.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:41 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:25:42.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:41 smithi153 ceph-mon[133625]: pgmap v161: 129 pgs: 129 active+clean; 210 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.7 MiB/s wr, 370 op/s 2023-12-07T10:25:42.556 INFO:tasks.workunit.client.0.smithi119.stdout: 1 349482 7.54 MB/sec execute 115 sec latency 4236.795 ms 2023-12-07T10:25:43.556 INFO:tasks.workunit.client.0.smithi119.stdout: 1 349482 7.48 MB/sec execute 116 sec latency 5236.876 ms 2023-12-07T10:25:44.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:44 smithi153 ceph-mon[133625]: pgmap v162: 129 pgs: 129 active+clean; 210 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 279 op/s 2023-12-07T10:25:44.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:44 smithi119 ceph-mon[97908]: pgmap v162: 129 pgs: 129 active+clean; 210 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 279 op/s 2023-12-07T10:25:44.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:44 smithi159 ceph-mon[134726]: pgmap v162: 129 pgs: 129 active+clean; 210 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 279 op/s 2023-12-07T10:25:44.556 INFO:tasks.workunit.client.0.smithi119.stdout: 1 349482 7.41 MB/sec execute 117 sec latency 6237.005 ms 2023-12-07T10:25:45.557 INFO:tasks.workunit.client.0.smithi119.stdout: 1 349482 7.35 MB/sec execute 118 sec latency 7237.130 ms 2023-12-07T10:25:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.696+0000 7f09a97ff700 1 -- 172.21.15.119:0/26295845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09a4101500 msgr2=0x7f09a4101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.696+0000 7f09a97ff700 1 --2- 172.21.15.119:0/26295845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09a4101500 0x7f09a4101980 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f09980099c0 tx=0x7f0998009cd0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.696+0000 7f09a97ff700 1 -- 172.21.15.119:0/26295845 shutdown_connections 2023-12-07T10:25:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.696+0000 7f09a97ff700 1 --2- 172.21.15.119:0/26295845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09a4101ec0 0x7f09a410e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.696+0000 7f09a97ff700 1 --2- 172.21.15.119:0/26295845 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09a41075c0 0x7f09a41079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.696+0000 7f09a97ff700 1 --2- 172.21.15.119:0/26295845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09a4101500 0x7f09a4101980 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.696+0000 7f09a97ff700 1 -- 172.21.15.119:0/26295845 >> 172.21.15.119:0/26295845 conn(0x7f09a40fcfb0 msgr2=0x7f09a40ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.696+0000 7f09a97ff700 1 -- 172.21.15.119:0/26295845 shutdown_connections 2023-12-07T10:25:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.696+0000 7f09a97ff700 1 -- 172.21.15.119:0/26295845 wait complete. 2023-12-07T10:25:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.697+0000 7f09a97ff700 1 Processor -- start 2023-12-07T10:25:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.697+0000 7f09a97ff700 1 -- start start 2023-12-07T10:25:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.697+0000 7f09a97ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09a4101500 0x7f09a419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.697+0000 7f09a97ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09a4101ec0 0x7f09a419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.697+0000 7f09a97ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09a41075c0 0x7f09a4199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.697+0000 7f09a97ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a4199fb0 con 0x7f09a41075c0 2023-12-07T10:25:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.697+0000 7f09a97ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a419a120 con 0x7f09a4101ec0 2023-12-07T10:25:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.697+0000 7f09a97ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a419a2e0 con 0x7f09a4101500 2023-12-07T10:25:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.698+0000 7f09a2ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09a4101500 0x7f09a419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.698+0000 7f09a37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09a41075c0 0x7f09a4199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.698+0000 7f09a2ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09a4101500 0x7f09a419f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41762/0 (socket says 172.21.15.119:41762) 2023-12-07T10:25:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.698+0000 7f09a2ffd700 1 -- 172.21.15.119:0/1359717399 learned_addr learned my addr 172.21.15.119:0/1359717399 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:25:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.698+0000 7f09a27fc700 1 --2- 172.21.15.119:0/1359717399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09a4101ec0 0x7f09a419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.698+0000 7f09a37fe700 1 -- 172.21.15.119:0/1359717399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09a4101500 msgr2=0x7f09a419f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.698+0000 7f09a37fe700 1 --2- 172.21.15.119:0/1359717399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09a4101500 0x7f09a419f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.698+0000 7f09a37fe700 1 -- 172.21.15.119:0/1359717399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09a4101ec0 msgr2=0x7f09a419fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.698+0000 7f09a37fe700 1 --2- 172.21.15.119:0/1359717399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09a4101ec0 0x7f09a419fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.698+0000 7f09a37fe700 1 -- 172.21.15.119:0/1359717399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09980095f0 con 0x7f09a41075c0 2023-12-07T10:25:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.698+0000 7f09a2ffd700 1 --2- 172.21.15.119:0/1359717399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09a4101500 0x7f09a419f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:25:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.698+0000 7f09a37fe700 1 --2- 172.21.15.119:0/1359717399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09a41075c0 0x7f09a4199950 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f099c007f10 tx=0x7f099c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.699+0000 7f098bfff700 1 -- 172.21.15.119:0/1359717399 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c004db0 con 0x7f09a41075c0 2023-12-07T10:25:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.699+0000 7f098bfff700 1 -- 172.21.15.119:0/1359717399 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f099c00f040 con 0x7f09a41075c0 2023-12-07T10:25:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.699+0000 7f09a97ff700 1 -- 172.21.15.119:0/1359717399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a419a5c0 con 0x7f09a41075c0 2023-12-07T10:25:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.699+0000 7f098bfff700 1 -- 172.21.15.119:0/1359717399 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c005990 con 0x7f09a41075c0 2023-12-07T10:25:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.699+0000 7f09a97ff700 1 -- 172.21.15.119:0/1359717399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a41a6810 con 0x7f09a41075c0 2023-12-07T10:25:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.703+0000 7f098bfff700 1 -- 172.21.15.119:0/1359717399 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f099c00db20 con 0x7f09a41075c0 2023-12-07T10:25:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.704+0000 7f09bd09eb80 1 Processor -- start 2023-12-07T10:25:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.704+0000 7f09bd09eb80 1 -- start start 2023-12-07T10:25:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.705+0000 7f098bfff700 1 --2- 172.21.15.119:0/1359717399 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f098c0a1c80 0x7f098c0a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.705+0000 7f09bd09eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aeb7a2a60 0x561aeb7a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.705+0000 7f09bd09eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561aeb7a3380 0x561aeb79d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.705+0000 7f09bd09eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aeb79dad0 0x561aeb79df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.705+0000 7f098bfff700 1 -- 172.21.15.119:0/1359717399 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f099c0cd5e0 con 0x7f09a41075c0 2023-12-07T10:25:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.705+0000 7f09bd09eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aeb711760 con 0x561aeb79dad0 2023-12-07T10:25:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.705+0000 7f09bd09eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aeb6f8740 con 0x561aeb7a2a60 2023-12-07T10:25:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.705+0000 7f09bd09eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aeb6f6060 con 0x561aeb7a3380 2023-12-07T10:25:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.705+0000 7f09a27fc700 1 --2- 172.21.15.119:0/1359717399 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f098c0a1c80 0x7f098c0a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.705+0000 7f09a2ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aeb7a2a60 0x561aeb7a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.705+0000 7f09a2ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aeb7a2a60 0x561aeb7a2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:49758/0 (socket says 172.21.15.119:49758) 2023-12-07T10:25:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.705+0000 7f09a2ffd700 1 -- 172.21.15.119:0/2326814473 learned_addr learned my addr 172.21.15.119:0/2326814473 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:25:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.705+0000 7f09a37fe700 1 --2- 172.21.15.119:0/2326814473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aeb79dad0 0x561aeb79df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.705+0000 7f09a27fc700 1 --2- 172.21.15.119:0/2326814473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561aeb7a3380 0x561aeb79d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.706+0000 7f09a2ffd700 1 -- 172.21.15.119:0/2326814473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561aeb7a3380 msgr2=0x561aeb79d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.706+0000 7f09a2ffd700 1 --2- 172.21.15.119:0/2326814473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561aeb7a3380 0x561aeb79d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.706+0000 7f09a2ffd700 1 -- 172.21.15.119:0/2326814473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aeb79dad0 msgr2=0x561aeb79df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.706+0000 7f09a2ffd700 1 --2- 172.21.15.119:0/2326814473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aeb79dad0 0x561aeb79df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.706+0000 7f09a2ffd700 1 -- 172.21.15.119:0/2326814473 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561aeb56e5d0 con 0x561aeb7a2a60 2023-12-07T10:25:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.706+0000 7f09a27fc700 1 --2- 172.21.15.119:0/2326814473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561aeb7a3380 0x561aeb79d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:25:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.706+0000 7f09a2ffd700 1 --2- 172.21.15.119:0/2326814473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aeb7a2a60 0x561aeb7a2e40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f09940107a0 tx=0x7f0994010ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a326b812f965a309 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.706+0000 7f09a27fc700 1 --2- 172.21.15.119:0/1359717399 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f098c0a1c80 0x7f098c0a4140 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f099800b580 tx=0x7f0998005a90 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.706+0000 7f09897fa700 1 -- 172.21.15.119:0/2326814473 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0994011700 con 0x561aeb7a2a60 2023-12-07T10:25:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.706+0000 7f09897fa700 1 -- 172.21.15.119:0/2326814473 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0994015070 con 0x561aeb7a2a60 2023-12-07T10:25:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.706+0000 7f09897fa700 1 -- 172.21.15.119:0/2326814473 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0994019b40 con 0x561aeb7a2a60 2023-12-07T10:25:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.707+0000 7f09bd09eb80 1 -- 172.21.15.119:0/2326814473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aeb7a2a60 msgr2=0x561aeb7a2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.707+0000 7f09bd09eb80 1 --2- 172.21.15.119:0/2326814473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aeb7a2a60 0x561aeb7a2e40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f09940107a0 tx=0x7f0994010ab0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.707+0000 7f09bd09eb80 1 -- 172.21.15.119:0/2326814473 shutdown_connections 2023-12-07T10:25:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.707+0000 7f09bd09eb80 1 --2- 172.21.15.119:0/2326814473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aeb79dad0 0x561aeb79df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.707+0000 7f09bd09eb80 1 --2- 172.21.15.119:0/2326814473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aeb7a2a60 0x561aeb7a2e40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.707+0000 7f09bd09eb80 1 --2- 172.21.15.119:0/2326814473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561aeb7a3380 0x561aeb79d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.707+0000 7f09bd09eb80 1 -- 172.21.15.119:0/2326814473 >> 172.21.15.119:0/2326814473 conn(0x561aeb6cc300 msgr2=0x561aeb797b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.707+0000 7f09bd09eb80 1 -- 172.21.15.119:0/2326814473 shutdown_connections 2023-12-07T10:25:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.707+0000 7f09bd09eb80 1 -- 172.21.15.119:0/2326814473 wait complete. 2023-12-07T10:25:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.708+0000 7f09bd09eb80 1 -- start start 2023-12-07T10:25:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.708+0000 7f09bd09eb80 1 Processor -- start 2023-12-07T10:25:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.708+0000 7f09bd09eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aeb79dad0 0x561aeb7a9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.708+0000 7f09bd09eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aeb7a2a60 0x561aeb7a6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.708+0000 7f09bd09eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561aeb7a3380 0x561aeb7a6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.708+0000 7f09bd09eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aeb6f1950 con 0x561aeb79dad0 2023-12-07T10:25:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.708+0000 7f09bd09eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aeb6ab500 con 0x561aeb7a2a60 2023-12-07T10:25:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.708+0000 7f09bd09eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aeb6ab1a0 con 0x561aeb7a3380 2023-12-07T10:25:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.709+0000 7f09bd09eb80 10 client.0 _ll_register_callbacks cb 0x561aeb6928e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:25:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.709+0000 7f09a2ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aeb79dad0 0x561aeb7a9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.709+0000 7f09a2ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aeb79dad0 0x561aeb7a9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47710/0 (socket says 172.21.15.119:47710) 2023-12-07T10:25:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.709+0000 7f09a2ffd700 1 -- 172.21.15.119:0/2250607530 learned_addr learned my addr 172.21.15.119:0/2250607530 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:25:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.709+0000 7f09a27fc700 1 --2- 172.21.15.119:0/2250607530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aeb7a2a60 0x561aeb7a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.709+0000 7f09a37fe700 1 --2- 172.21.15.119:0/2250607530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561aeb7a3380 0x561aeb7a6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.709+0000 7f09a2ffd700 1 -- 172.21.15.119:0/2250607530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561aeb7a3380 msgr2=0x561aeb7a6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.709+0000 7f09a2ffd700 1 --2- 172.21.15.119:0/2250607530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561aeb7a3380 0x561aeb7a6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.709+0000 7f09a2ffd700 1 -- 172.21.15.119:0/2250607530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aeb7a2a60 msgr2=0x561aeb7a6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.709+0000 7f09a2ffd700 1 --2- 172.21.15.119:0/2250607530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aeb7a2a60 0x561aeb7a6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.709+0000 7f09a2ffd700 1 -- 172.21.15.119:0/2250607530 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09940103d0 con 0x561aeb79dad0 2023-12-07T10:25:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.709+0000 7f09a2ffd700 1 --2- 172.21.15.119:0/2250607530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aeb79dad0 0x561aeb7a9410 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f09940196c0 tx=0x7f09940193d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52386981a24d1ede server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.710+0000 7f0986ff5700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:25:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.710+0000 7f0986ff5700 1 -- 172.21.15.119:0/2250607530 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0994005300 con 0x561aeb79dad0 2023-12-07T10:25:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.710+0000 7f09bd09eb80 1 -- 172.21.15.119:0/2250607530 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561aeb7906f0 con 0x561aeb79dad0 2023-12-07T10:25:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.710+0000 7f0986ff5700 1 -- 172.21.15.119:0/2250607530 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f0994005460 con 0x561aeb79dad0 2023-12-07T10:25:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.710+0000 7f0986ff5700 1 -- 172.21.15.119:0/2250607530 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0994005620 con 0x561aeb79dad0 2023-12-07T10:25:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.710+0000 7f09bd09eb80 10 client.17211 fetch_fsmap learned FSMap version 45 2023-12-07T10:25:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.710+0000 7f09bd09eb80 1 -- 172.21.15.119:0/2250607530 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x561aeb51ad80 con 0x561aeb79dad0 2023-12-07T10:25:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.710+0000 7f0986ff5700 1 -- 172.21.15.119:0/2250607530 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561aeb7906f0 con 0x561aeb79dad0 2023-12-07T10:25:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.711+0000 7f0986ff5700 1 -- 172.21.15.119:0/2250607530 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 45) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f0994004b60 con 0x561aeb79dad0 2023-12-07T10:25:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.711+0000 7f09bd09eb80 10 client.17211 fetch_fsmap finished waiting for FSMap version 45 2023-12-07T10:25:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.711+0000 7f09bd09eb80 10 client.17211 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:25:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.711+0000 7f09bd09eb80 20 client.17211 populate_metadata read hostname 'smithi119' 2023-12-07T10:25:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.711+0000 7f09bd09eb80 1 --2- 172.21.15.119:0/2250607530 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x561aeb7b6e30 0x561aeb7b9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:45.717 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.711+0000 7f09bd09eb80 4 client.17211 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:25:45.717 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.711+0000 7f09bd09eb80 1 -- 172.21.15.119:0/2250607530 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561aeb76c740 con 0x561aeb7b6e30 2023-12-07T10:25:45.717 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.711+0000 7f09a27fc700 1 --2- 172.21.15.119:0/2250607530 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x561aeb7b6e30 0x561aeb7b9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:45.717 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.712+0000 7f0986ff5700 10 client.17211 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:25:45.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.712+0000 7f09a27fc700 1 --2- 172.21.15.119:0/2250607530 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x561aeb7b6e30 0x561aeb7b9250 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9f289b0bc94e0a0 server_cookie=8a292dfb0271d9d in_seq=0 out_seq=0 2023-12-07T10:25:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.712+0000 7f0986ff5700 1 -- 172.21.15.119:0/2250607530 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561aeb76c740 con 0x561aeb7b6e30 2023-12-07T10:25:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.712+0000 7f0986ff5700 10 client.17211 handle_command_reply: tid=0 2023-12-07T10:25:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.712+0000 7f09bd09eb80 1 client.17211 shutdown 2023-12-07T10:25:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.712+0000 7f09bd09eb80 1 -- 172.21.15.119:0/2250607530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aeb79dad0 msgr2=0x561aeb7a9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.712+0000 7f09bd09eb80 1 --2- 172.21.15.119:0/2250607530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aeb79dad0 0x561aeb7a9410 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f09940196c0 tx=0x7f09940193d0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.712+0000 7f09bd09eb80 1 -- 172.21.15.119:0/2250607530 shutdown_connections 2023-12-07T10:25:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.712+0000 7f09bd09eb80 1 --2- 172.21.15.119:0/2250607530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561aeb7a3380 0x561aeb7a6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.712+0000 7f09bd09eb80 1 --2- 172.21.15.119:0/2250607530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aeb79dad0 0x561aeb7a9410 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.712+0000 7f09bd09eb80 1 --2- 172.21.15.119:0/2250607530 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x561aeb7b6e30 0x561aeb7b9250 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.713+0000 7f09bd09eb80 1 --2- 172.21.15.119:0/2250607530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aeb7a2a60 0x561aeb7a6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.713+0000 7f09bd09eb80 1 -- 172.21.15.119:0/2250607530 >> 172.21.15.119:0/2250607530 conn(0x561aeb6cc300 msgr2=0x561aeb792980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.713+0000 7f0986ff5700 0 client.17211 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:25:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.713+0000 7f09bd09eb80 1 -- 172.21.15.119:0/2250607530 shutdown_connections 2023-12-07T10:25:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.713+0000 7f09bd09eb80 1 -- 172.21.15.119:0/2250607530 wait complete. 2023-12-07T10:25:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.713+0000 7f09bd09eb80 20 client.17211 trim_cache size 0 max 16384 2023-12-07T10:25:45.729 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.728+0000 7f09bd09eb80 1 Processor -- start 2023-12-07T10:25:45.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.728+0000 7f09bd09eb80 1 -- start start 2023-12-07T10:25:45.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.728+0000 7f09bd09eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aeb79dad0 0x561aeb7b2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:45.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.728+0000 7f09bd09eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561aeb7a2a60 0x561aeb7bdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:45.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.728+0000 7f09bd09eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aeb7a3380 0x561aeb7addd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:45.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.728+0000 7f09bd09eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aeb5708c0 con 0x561aeb7a3380 2023-12-07T10:25:45.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.728+0000 7f09bd09eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aeb7a4ca0 con 0x561aeb79dad0 2023-12-07T10:25:45.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.728+0000 7f09bd09eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aeb7a14d0 con 0x561aeb7a2a60 2023-12-07T10:25:45.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.729+0000 7f09a37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aeb7a3380 0x561aeb7addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:45.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.729+0000 7f09a2ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aeb79dad0 0x561aeb7b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:45.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.729+0000 7f09a2ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aeb79dad0 0x561aeb7b2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:49774/0 (socket says 172.21.15.119:49774) 2023-12-07T10:25:45.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.729+0000 7f09a2ffd700 1 -- 172.21.15.119:0/2277454379 learned_addr learned my addr 172.21.15.119:0/2277454379 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:25:45.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.729+0000 7f09a27fc700 1 --2- 172.21.15.119:0/2277454379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561aeb7a2a60 0x561aeb7bdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:45.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.729+0000 7f09a2ffd700 1 -- 172.21.15.119:0/2277454379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561aeb7a2a60 msgr2=0x561aeb7bdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:45.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.729+0000 7f09a2ffd700 1 --2- 172.21.15.119:0/2277454379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561aeb7a2a60 0x561aeb7bdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.729+0000 7f09a2ffd700 1 -- 172.21.15.119:0/2277454379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aeb7a3380 msgr2=0x561aeb7addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:45.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.729+0000 7f09a2ffd700 1 --2- 172.21.15.119:0/2277454379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aeb7a3380 0x561aeb7addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.729+0000 7f09a2ffd700 1 -- 172.21.15.119:0/2277454379 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561aeb503c00 con 0x561aeb79dad0 2023-12-07T10:25:45.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.729+0000 7f09a2ffd700 1 --2- 172.21.15.119:0/2277454379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aeb79dad0 0x561aeb7b2880 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f099400b6b0 tx=0x7f0994005990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b30227f8d916bdc9 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:45.734 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.730+0000 7f0989ffb700 1 -- 172.21.15.119:0/2277454379 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0994005b80 con 0x561aeb79dad0 2023-12-07T10:25:45.734 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.730+0000 7f0989ffb700 1 -- 172.21.15.119:0/2277454379 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0994019440 con 0x561aeb79dad0 2023-12-07T10:25:45.734 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.730+0000 7f0989ffb700 1 -- 172.21.15.119:0/2277454379 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0994019950 con 0x561aeb79dad0 2023-12-07T10:25:45.734 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.730+0000 7f09bd09eb80 1 -- 172.21.15.119:0/2277454379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aeb79dad0 msgr2=0x561aeb7b2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:45.734 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.730+0000 7f09bd09eb80 1 --2- 172.21.15.119:0/2277454379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aeb79dad0 0x561aeb7b2880 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f099400b6b0 tx=0x7f0994005990 comp rx=0 tx=0).stop 2023-12-07T10:25:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.730+0000 7f09bd09eb80 1 -- 172.21.15.119:0/2277454379 shutdown_connections 2023-12-07T10:25:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.730+0000 7f09bd09eb80 1 --2- 172.21.15.119:0/2277454379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aeb7a3380 0x561aeb7addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.730+0000 7f09bd09eb80 1 --2- 172.21.15.119:0/2277454379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aeb79dad0 0x561aeb7b2880 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.730+0000 7f09bd09eb80 1 --2- 172.21.15.119:0/2277454379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561aeb7a2a60 0x561aeb7bdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.730+0000 7f09bd09eb80 1 -- 172.21.15.119:0/2277454379 >> 172.21.15.119:0/2277454379 conn(0x561aeb6cc300 msgr2=0x561aeb797b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.730+0000 7f09bd09eb80 1 -- 172.21.15.119:0/2277454379 shutdown_connections 2023-12-07T10:25:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.730+0000 7f09bd09eb80 1 -- 172.21.15.119:0/2277454379 wait complete. 2023-12-07T10:25:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.731+0000 7f09bd09eb80 1 -- start start 2023-12-07T10:25:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.731+0000 7f09bd09eb80 1 Processor -- start 2023-12-07T10:25:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.731+0000 7f09bd09eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aeb79dad0 0x561aeb7ae030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.731+0000 7f09bd09eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561aeb7a2a60 0x561aeb7abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.731+0000 7f09bd09eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aeb7a3380 0x561aeb7ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.731+0000 7f09bd09eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aeb7ac8c0 con 0x561aeb79dad0 2023-12-07T10:25:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.731+0000 7f09bd09eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aeb7aca30 con 0x561aeb7a3380 2023-12-07T10:25:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.731+0000 7f09bd09eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aeb7acbf0 con 0x561aeb7a2a60 2023-12-07T10:25:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.731+0000 7f09bd09eb80 10 client.0 _ll_register_callbacks cb 0x561aeb6928e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:25:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.731+0000 7f09a2ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aeb79dad0 0x561aeb7ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.732+0000 7f09a27fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561aeb7a2a60 0x561aeb7abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.732+0000 7f09a37fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aeb7a3380 0x561aeb7ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.732+0000 7f09a27fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561aeb7a2a60 0x561aeb7abdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41792/0 (socket says 172.21.15.119:41792) 2023-12-07T10:25:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.732+0000 7f09a27fc700 1 -- 172.21.15.119:0/3354301502 learned_addr learned my addr 172.21.15.119:0/3354301502 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:25:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.732+0000 7f09a2ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aeb79dad0 0x561aeb7ae030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47730/0 (socket says 172.21.15.119:47730) 2023-12-07T10:25:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.732+0000 7f09a27fc700 1 -- 172.21.15.119:0/3354301502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aeb7a3380 msgr2=0x561aeb7ac2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.732+0000 7f09a27fc700 1 --2- 172.21.15.119:0/3354301502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aeb7a3380 0x561aeb7ac2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.732+0000 7f09a27fc700 1 -- 172.21.15.119:0/3354301502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aeb79dad0 msgr2=0x561aeb7ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.732+0000 7f09a27fc700 1 --2- 172.21.15.119:0/3354301502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aeb79dad0 0x561aeb7ae030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.732+0000 7f09a27fc700 1 -- 172.21.15.119:0/3354301502 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561aeb51ad80 con 0x561aeb7a2a60 2023-12-07T10:25:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.732+0000 7f09a2ffd700 1 --2- 172.21.15.119:0/3354301502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aeb79dad0 0x561aeb7ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:25:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.732+0000 7f09a37fe700 1 --2- 172.21.15.119:0/3354301502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aeb7a3380 0x561aeb7ac2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:25:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.732+0000 7f09a27fc700 1 --2- 172.21.15.119:0/3354301502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561aeb7a2a60 0x561aeb7abdb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f09980099c0 tx=0x7f0998009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c6fe1f8b8fb84dd server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.732+0000 7f0986ff5700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:25:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.733+0000 7f0986ff5700 1 -- 172.21.15.119:0/3354301502 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099801d040 con 0x561aeb7a2a60 2023-12-07T10:25:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.733+0000 7f09bd09eb80 1 -- 172.21.15.119:0/3354301502 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0994015070 con 0x561aeb7a2a60 2023-12-07T10:25:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.733+0000 7f0986ff5700 1 -- 172.21.15.119:0/3354301502 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f099800be70 con 0x561aeb7a2a60 2023-12-07T10:25:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.733+0000 7f0986ff5700 1 -- 172.21.15.119:0/3354301502 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0998033440 con 0x561aeb7a2a60 2023-12-07T10:25:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.733+0000 7f09bd09eb80 10 client.27229 fetch_fsmap learned FSMap version 45 2023-12-07T10:25:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.733+0000 7f09bd09eb80 1 -- 172.21.15.119:0/3354301502 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x561aeb5411f0 con 0x561aeb7a2a60 2023-12-07T10:25:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.733+0000 7f0986ff5700 1 -- 172.21.15.119:0/3354301502 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0994015070 con 0x561aeb7a2a60 2023-12-07T10:25:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.733+0000 7f0986ff5700 1 -- 172.21.15.119:0/3354301502 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 45) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f099800fcb0 con 0x561aeb7a2a60 2023-12-07T10:25:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.733+0000 7f09bd09eb80 10 client.27229 fetch_fsmap finished waiting for FSMap version 45 2023-12-07T10:25:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.733+0000 7f09bd09eb80 10 client.27229 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:25:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.733+0000 7f09bd09eb80 20 client.27229 populate_metadata read hostname 'smithi119' 2023-12-07T10:25:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.733+0000 7f09bd09eb80 1 --2- 172.21.15.119:0/3354301502 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x561aeb80da20 0x561aeb80fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.733+0000 7f09bd09eb80 4 client.27229 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:25:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.733+0000 7f09bd09eb80 1 -- 172.21.15.119:0/3354301502 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561aeb6b7f70 con 0x561aeb80da20 2023-12-07T10:25:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.734+0000 7f09a2ffd700 1 --2- 172.21.15.119:0/3354301502 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x561aeb80da20 0x561aeb80fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.734+0000 7f0986ff5700 10 client.27229 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:25:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.734+0000 7f09a2ffd700 1 --2- 172.21.15.119:0/3354301502 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x561aeb80da20 0x561aeb80fe40 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2dcbe71a946ec8f server_cookie=bb066e5eda59517e in_seq=0 out_seq=0 2023-12-07T10:25:45.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.750+0000 7f0986ff5700 1 -- 172.21.15.119:0/3354301502 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561aeb6b7f70 con 0x561aeb80da20 2023-12-07T10:25:45.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.750+0000 7f0986ff5700 10 client.27229 handle_command_reply: tid=0 2023-12-07T10:25:45.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.750+0000 7f09bd09eb80 1 client.27229 shutdown 2023-12-07T10:25:45.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.751+0000 7f09bd09eb80 1 -- 172.21.15.119:0/3354301502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561aeb7a2a60 msgr2=0x561aeb7abdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:45.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.751+0000 7f09bd09eb80 1 --2- 172.21.15.119:0/3354301502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561aeb7a2a60 0x561aeb7abdb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f09980099c0 tx=0x7f0998009310 comp rx=0 tx=0).stop 2023-12-07T10:25:45.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.751+0000 7f09bd09eb80 1 -- 172.21.15.119:0/3354301502 shutdown_connections 2023-12-07T10:25:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.751+0000 7f09bd09eb80 1 --2- 172.21.15.119:0/3354301502 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x561aeb80da20 0x561aeb80fe40 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.751+0000 7f09bd09eb80 1 --2- 172.21.15.119:0/3354301502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x561aeb7a3380 0x561aeb7ac2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.751+0000 7f0986ff5700 0 client.27229 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:25:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.751+0000 7f09bd09eb80 1 --2- 172.21.15.119:0/3354301502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561aeb79dad0 0x561aeb7ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.751+0000 7f09bd09eb80 1 --2- 172.21.15.119:0/3354301502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561aeb7a2a60 0x561aeb7abdb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.751+0000 7f09bd09eb80 1 -- 172.21.15.119:0/3354301502 >> 172.21.15.119:0/3354301502 conn(0x561aeb6cc300 msgr2=0x561aeb797b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.751+0000 7f09bd09eb80 1 -- 172.21.15.119:0/3354301502 shutdown_connections 2023-12-07T10:25:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.751+0000 7f09bd09eb80 1 -- 172.21.15.119:0/3354301502 wait complete. 2023-12-07T10:25:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.751+0000 7f09bd09eb80 20 client.27229 trim_cache size 0 max 16384 2023-12-07T10:25:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:25:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:25:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:25:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:25:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.752+0000 7f09a97ff700 1 -- 172.21.15.119:0/1359717399 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f098c0a1c80 msgr2=0x7f098c0a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.752+0000 7f09a97ff700 1 --2- 172.21.15.119:0/1359717399 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f098c0a1c80 0x7f098c0a4140 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f099800b580 tx=0x7f0998005a90 comp rx=0 tx=0).stop 2023-12-07T10:25:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.752+0000 7f09a97ff700 1 -- 172.21.15.119:0/1359717399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09a41075c0 msgr2=0x7f09a4199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.752+0000 7f09a97ff700 1 --2- 172.21.15.119:0/1359717399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09a41075c0 0x7f09a4199950 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f099c007f10 tx=0x7f099c00d3b0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.752+0000 7f09a97ff700 1 -- 172.21.15.119:0/1359717399 shutdown_connections 2023-12-07T10:25:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.752+0000 7f09a97ff700 1 --2- 172.21.15.119:0/1359717399 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f098c0a1c80 0x7f098c0a4140 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.752+0000 7f09a97ff700 1 --2- 172.21.15.119:0/1359717399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09a41075c0 0x7f09a4199950 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.752+0000 7f09a97ff700 1 --2- 172.21.15.119:0/1359717399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09a4101500 0x7f09a419f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.752+0000 7f09a97ff700 1 --2- 172.21.15.119:0/1359717399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09a4101ec0 0x7f09a419fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.752+0000 7f09a97ff700 1 -- 172.21.15.119:0/1359717399 >> 172.21.15.119:0/1359717399 conn(0x7f09a40fcfb0 msgr2=0x7f09a40fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.752+0000 7f09a97ff700 1 -- 172.21.15.119:0/1359717399 shutdown_connections 2023-12-07T10:25:45.756 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:45.752+0000 7f09a97ff700 1 -- 172.21.15.119:0/1359717399 wait complete. 2023-12-07T10:25:46.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:46 smithi153 ceph-mon[133625]: pgmap v163: 129 pgs: 129 active+clean; 210 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 279 op/s 2023-12-07T10:25:46.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:46 smithi119 ceph-mon[97908]: pgmap v163: 129 pgs: 129 active+clean; 210 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 279 op/s 2023-12-07T10:25:46.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:46 smithi159 ceph-mon[134726]: pgmap v163: 129 pgs: 129 active+clean; 210 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 279 op/s 2023-12-07T10:25:46.557 INFO:tasks.workunit.client.0.smithi119.stdout: 1 349482 7.29 MB/sec execute 119 sec latency 8237.245 ms 2023-12-07T10:25:47.557 INFO:tasks.workunit.client.0.smithi119.stdout: 1 349482 7.23 MB/sec execute 120 sec latency 9237.342 ms 2023-12-07T10:25:48.281 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:48 smithi153 ceph-mon[133625]: pgmap v164: 129 pgs: 129 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.0 MiB/s wr, 311 op/s 2023-12-07T10:25:48.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:48 smithi119 ceph-mon[97908]: pgmap v164: 129 pgs: 129 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.0 MiB/s wr, 311 op/s 2023-12-07T10:25:48.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:48 smithi159 ceph-mon[134726]: pgmap v164: 129 pgs: 129 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.0 MiB/s wr, 311 op/s 2023-12-07T10:25:48.557 INFO:tasks.workunit.client.0.smithi119.stdout: 1 349482 7.17 MB/sec execute 121 sec latency 10237.391 ms 2023-12-07T10:25:49.557 INFO:tasks.workunit.client.0.smithi119.stdout: 1 349482 7.11 MB/sec execute 122 sec latency 11237.505 ms 2023-12-07T10:25:50.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:50 smithi153 ceph-mon[133625]: pgmap v165: 129 pgs: 129 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 2.0 MiB/s wr, 129 op/s 2023-12-07T10:25:50.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:50 smithi119 ceph-mon[97908]: pgmap v165: 129 pgs: 129 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 2.0 MiB/s wr, 129 op/s 2023-12-07T10:25:50.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:50 smithi159 ceph-mon[134726]: pgmap v165: 129 pgs: 129 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 2.0 MiB/s wr, 129 op/s 2023-12-07T10:25:50.557 INFO:tasks.workunit.client.0.smithi119.stdout: 1 349482 7.05 MB/sec execute 123 sec latency 12237.636 ms 2023-12-07T10:25:51.557 INFO:tasks.workunit.client.0.smithi119.stdout: 1 349482 7.00 MB/sec execute 124 sec latency 13237.775 ms 2023-12-07T10:25:52.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:51 smithi119 ceph-mon[97908]: pgmap v166: 129 pgs: 129 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 2.0 MiB/s wr, 137 op/s 2023-12-07T10:25:52.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:51 smithi159 ceph-mon[134726]: pgmap v166: 129 pgs: 129 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 2.0 MiB/s wr, 137 op/s 2023-12-07T10:25:52.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:51 smithi153 ceph-mon[133625]: pgmap v166: 129 pgs: 129 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 2.0 MiB/s wr, 137 op/s 2023-12-07T10:25:52.557 INFO:tasks.workunit.client.0.smithi119.stdout: 1 349482 6.94 MB/sec execute 125 sec latency 14237.905 ms 2023-12-07T10:25:53.557 INFO:tasks.workunit.client.0.smithi119.stdout: 1 350589 6.94 MB/sec execute 126 sec latency 14764.090 ms 2023-12-07T10:25:54.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:54 smithi153 ceph-mon[133625]: pgmap v167: 129 pgs: 129 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 63 KiB/s wr, 39 op/s 2023-12-07T10:25:54.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:54 smithi119 ceph-mon[97908]: pgmap v167: 129 pgs: 129 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 63 KiB/s wr, 39 op/s 2023-12-07T10:25:54.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:54 smithi159 ceph-mon[134726]: pgmap v167: 129 pgs: 129 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 63 KiB/s wr, 39 op/s 2023-12-07T10:25:54.557 INFO:tasks.workunit.client.0.smithi119.stdout: 1 351880 6.94 MB/sec execute 127 sec latency 10.648 ms 2023-12-07T10:25:55.557 INFO:tasks.workunit.client.0.smithi119.stdout: 1 353767 6.95 MB/sec execute 128 sec latency 16.687 ms 2023-12-07T10:25:56.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:56 smithi153 ceph-mon[133625]: pgmap v168: 129 pgs: 129 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 63 KiB/s wr, 39 op/s 2023-12-07T10:25:56.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:56 smithi119 ceph-mon[97908]: pgmap v168: 129 pgs: 129 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 63 KiB/s wr, 39 op/s 2023-12-07T10:25:56.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:56 smithi159 ceph-mon[134726]: pgmap v168: 129 pgs: 129 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 63 KiB/s wr, 39 op/s 2023-12-07T10:25:56.558 INFO:tasks.workunit.client.0.smithi119.stdout: 1 355203 6.98 MB/sec execute 129 sec latency 11.958 ms 2023-12-07T10:25:57.558 INFO:tasks.workunit.client.0.smithi119.stdout: 1 356392 6.97 MB/sec execute 130 sec latency 20.378 ms 2023-12-07T10:25:58.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:25:58 smithi153 ceph-mon[133625]: pgmap v169: 129 pgs: 129 active+clean; 193 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 423 KiB/s wr, 91 op/s 2023-12-07T10:25:58.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:25:58 smithi119 ceph-mon[97908]: pgmap v169: 129 pgs: 129 active+clean; 193 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 423 KiB/s wr, 91 op/s 2023-12-07T10:25:58.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:25:58 smithi159 ceph-mon[134726]: pgmap v169: 129 pgs: 129 active+clean; 193 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 423 KiB/s wr, 91 op/s 2023-12-07T10:25:58.558 INFO:tasks.workunit.client.0.smithi119.stdout: 1 358308 7.01 MB/sec execute 131 sec latency 11.893 ms 2023-12-07T10:25:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.915+0000 7f9d4f5ff700 1 -- 172.21.15.119:0/3668456839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d48100ef0 msgr2=0x7f9d481012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.915+0000 7f9d4f5ff700 1 --2- 172.21.15.119:0/3668456839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d48100ef0 0x7f9d481012d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9d44009990 tx=0x7f9d44009ca0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.916+0000 7f9d4f5ff700 1 -- 172.21.15.119:0/3668456839 shutdown_connections 2023-12-07T10:25:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.916+0000 7f9d4f5ff700 1 --2- 172.21.15.119:0/3668456839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d4810df60 0x7f9d48110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.916+0000 7f9d4f5ff700 1 --2- 172.21.15.119:0/3668456839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d48100ef0 0x7f9d481012d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.916+0000 7f9d4f5ff700 1 --2- 172.21.15.119:0/3668456839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d48101810 0x7f9d4810da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.916+0000 7f9d4f5ff700 1 -- 172.21.15.119:0/3668456839 >> 172.21.15.119:0/3668456839 conn(0x7f9d480775b0 msgr2=0x7f9d480779c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.916+0000 7f9d4f5ff700 1 -- 172.21.15.119:0/3668456839 shutdown_connections 2023-12-07T10:25:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.916+0000 7f9d4f5ff700 1 -- 172.21.15.119:0/3668456839 wait complete. 2023-12-07T10:25:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.917+0000 7f9d4f5ff700 1 Processor -- start 2023-12-07T10:25:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.917+0000 7f9d4f5ff700 1 -- start start 2023-12-07T10:25:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.917+0000 7f9d4f5ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d48100ef0 0x7f9d4819f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.917+0000 7f9d4f5ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d48101810 0x7f9d4819fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.917+0000 7f9d4f5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d4810df60 0x7f9d48199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.917+0000 7f9d4f5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4819a0a0 con 0x7f9d4810df60 2023-12-07T10:25:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.917+0000 7f9d4f5ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4819a210 con 0x7f9d48101810 2023-12-07T10:25:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.917+0000 7f9d4f5ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4819a3d0 con 0x7f9d48100ef0 2023-12-07T10:25:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.918+0000 7f9d4cb9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d48101810 0x7f9d4819fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.918+0000 7f9d4d39b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d48100ef0 0x7f9d4819f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.918+0000 7f9d4d39b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d48100ef0 0x7f9d4819f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:52426/0 (socket says 172.21.15.119:52426) 2023-12-07T10:25:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.918+0000 7f9d4cb9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d48101810 0x7f9d4819fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:44862/0 (socket says 172.21.15.119:44862) 2023-12-07T10:25:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.918+0000 7f9d4d39b700 1 -- 172.21.15.119:0/3625597246 learned_addr learned my addr 172.21.15.119:0/3625597246 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:25:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.918+0000 7f9d4db9c700 1 --2- 172.21.15.119:0/3625597246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d4810df60 0x7f9d48199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.918+0000 7f9d4d39b700 1 -- 172.21.15.119:0/3625597246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d48101810 msgr2=0x7f9d4819fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.918+0000 7f9d4d39b700 1 --2- 172.21.15.119:0/3625597246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d48101810 0x7f9d4819fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.918+0000 7f9d4d39b700 1 -- 172.21.15.119:0/3625597246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d4810df60 msgr2=0x7f9d48199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.918+0000 7f9d4d39b700 1 --2- 172.21.15.119:0/3625597246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d4810df60 0x7f9d48199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.918+0000 7f9d4d39b700 1 -- 172.21.15.119:0/3625597246 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d440095f0 con 0x7f9d48100ef0 2023-12-07T10:25:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.918+0000 7f9d4cb9a700 1 --2- 172.21.15.119:0/3625597246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d48101810 0x7f9d4819fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:25:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.918+0000 7f9d4db9c700 1 --2- 172.21.15.119:0/3625597246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d4810df60 0x7f9d48199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:25:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.918+0000 7f9d4d39b700 1 --2- 172.21.15.119:0/3625597246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d48100ef0 0x7f9d4819f910 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9d44005910 tx=0x7f9d44004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.919+0000 7f9d3e7fc700 1 -- 172.21.15.119:0/3625597246 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4401d070 con 0x7f9d48100ef0 2023-12-07T10:25:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.919+0000 7f9d4f5ff700 1 -- 172.21.15.119:0/3625597246 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d4819a650 con 0x7f9d48100ef0 2023-12-07T10:25:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.919+0000 7f9d3e7fc700 1 -- 172.21.15.119:0/3625597246 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d44022470 con 0x7f9d48100ef0 2023-12-07T10:25:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.919+0000 7f9d3e7fc700 1 -- 172.21.15.119:0/3625597246 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d44017650 con 0x7f9d48100ef0 2023-12-07T10:25:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.919+0000 7f9d4f5ff700 1 -- 172.21.15.119:0/3625597246 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d481a6900 con 0x7f9d48100ef0 2023-12-07T10:25:58.923 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.923+0000 7f9d3e7fc700 1 -- 172.21.15.119:0/3625597246 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f9d440225e0 con 0x7f9d48100ef0 2023-12-07T10:25:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.924+0000 7f9d63004b80 1 Processor -- start 2023-12-07T10:25:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.925+0000 7f9d3e7fc700 1 --2- 172.21.15.119:0/3625597246 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9d340a1c80 0x7f9d340a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.925+0000 7f9d63004b80 1 -- start start 2023-12-07T10:25:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.925+0000 7f9d3e7fc700 1 -- 172.21.15.119:0/3625597246 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9d440cf090 con 0x7f9d48100ef0 2023-12-07T10:25:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.925+0000 7f9d63004b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560629c0da60 0x560629c0de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.925+0000 7f9d4db9c700 1 --2- 172.21.15.119:0/3625597246 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9d340a1c80 0x7f9d340a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.925+0000 7f9d63004b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560629c0e380 0x560629c08540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.925+0000 7f9d63004b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560629c08ad0 0x560629c08f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.925+0000 7f9d63004b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560629b7c760 con 0x560629c08ad0 2023-12-07T10:25:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.925+0000 7f9d63004b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x560629b63740 con 0x560629c0da60 2023-12-07T10:25:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.925+0000 7f9d63004b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560629b61060 con 0x560629c0e380 2023-12-07T10:25:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.925+0000 7f9d4d39b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560629c0da60 0x560629c0de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.925+0000 7f9d4d39b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560629c0da60 0x560629c0de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:44872/0 (socket says 172.21.15.119:44872) 2023-12-07T10:25:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.925+0000 7f9d4cb9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560629c0e380 0x560629c08540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.925+0000 7f9d4d39b700 1 -- 172.21.15.119:0/274814224 learned_addr learned my addr 172.21.15.119:0/274814224 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:25:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.926+0000 7f9d4d39b700 1 -- 172.21.15.119:0/274814224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560629c0e380 msgr2=0x560629c08540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:58.928 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.926+0000 7f9d4d39b700 1 --2- 172.21.15.119:0/274814224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560629c0e380 0x560629c08540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.928 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.926+0000 7f9d4db9c700 1 --2- 172.21.15.119:0/274814224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560629c08ad0 0x560629c08f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:58.928 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.926+0000 7f9d4d39b700 1 -- 172.21.15.119:0/274814224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560629c08ad0 msgr2=0x560629c08f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:58.928 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.926+0000 7f9d4d39b700 1 --2- 172.21.15.119:0/274814224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560629c08ad0 0x560629c08f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.928 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.926+0000 7f9d4d39b700 1 -- 172.21.15.119:0/274814224 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606299d95d0 con 0x560629c0da60 2023-12-07T10:25:58.928 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.926+0000 7f9d4db9c700 1 --2- 172.21.15.119:0/274814224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560629c08ad0 0x560629c08f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:25:58.928 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.926+0000 7f9d4db9c700 1 --2- 172.21.15.119:0/3625597246 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9d340a1c80 0x7f9d340a4140 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9d4806ad30 tx=0x7f9d40009920 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:58.928 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.926+0000 7f9d4d39b700 1 --2- 172.21.15.119:0/274814224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560629c0da60 0x560629c0de40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9d440985e0 tx=0x7f9d44000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df6fbc79c3c25ef2 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:58.928 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.926+0000 7f9d337fe700 1 -- 172.21.15.119:0/274814224 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4409bb70 con 0x560629c0da60 2023-12-07T10:25:58.928 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.926+0000 7f9d337fe700 1 -- 172.21.15.119:0/274814224 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d4409fcc0 con 0x560629c0da60 2023-12-07T10:25:58.928 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.926+0000 7f9d337fe700 1 -- 172.21.15.119:0/274814224 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4409b2c0 con 0x560629c0da60 2023-12-07T10:25:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.927+0000 7f9d63004b80 1 -- 172.21.15.119:0/274814224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560629c0da60 msgr2=0x560629c0de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.927+0000 7f9d63004b80 1 --2- 172.21.15.119:0/274814224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560629c0da60 0x560629c0de40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9d440985e0 tx=0x7f9d44000bc0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.927+0000 7f9d63004b80 1 -- 172.21.15.119:0/274814224 shutdown_connections 2023-12-07T10:25:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.927+0000 7f9d63004b80 1 --2- 172.21.15.119:0/274814224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560629c08ad0 0x560629c08f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.927+0000 7f9d63004b80 1 --2- 172.21.15.119:0/274814224 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560629c0da60 0x560629c0de40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.927+0000 7f9d63004b80 1 --2- 172.21.15.119:0/274814224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560629c0e380 0x560629c08540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.927+0000 7f9d63004b80 1 -- 172.21.15.119:0/274814224 >> 172.21.15.119:0/274814224 conn(0x560629b37300 msgr2=0x560629c02b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.927+0000 7f9d63004b80 1 -- 172.21.15.119:0/274814224 shutdown_connections 2023-12-07T10:25:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.927+0000 7f9d63004b80 1 -- 172.21.15.119:0/274814224 wait complete. 2023-12-07T10:25:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.928+0000 7f9d63004b80 1 -- start start 2023-12-07T10:25:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.928+0000 7f9d63004b80 1 Processor -- start 2023-12-07T10:25:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.929+0000 7f9d63004b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560629c08ad0 0x560629c14350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.929+0000 7f9d63004b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560629c0da60 0x560629c11750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.929+0000 7f9d63004b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560629c0e380 0x560629c11e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.929+0000 7f9d63004b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560629b5c950 con 0x560629c0e380 2023-12-07T10:25:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.929+0000 7f9d63004b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x560629b16500 con 0x560629c08ad0 2023-12-07T10:25:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.929+0000 7f9d63004b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560629b161a0 con 0x560629c0da60 2023-12-07T10:25:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.929+0000 7f9d63004b80 10 client.0 _ll_register_callbacks cb 0x560629afd8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:25:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.929+0000 7f9d4d39b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560629c08ad0 0x560629c14350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.929+0000 7f9d4d39b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560629c08ad0 0x560629c14350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:44882/0 (socket says 172.21.15.119:44882) 2023-12-07T10:25:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.929+0000 7f9d4d39b700 1 -- 172.21.15.119:0/1579830934 learned_addr learned my addr 172.21.15.119:0/1579830934 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:25:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.929+0000 7f9d4db9c700 1 --2- 172.21.15.119:0/1579830934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560629c0e380 0x560629c11e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.930+0000 7f9d4d39b700 1 -- 172.21.15.119:0/1579830934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560629c0da60 msgr2=0x560629c11750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.930+0000 7f9d4cb9a700 1 --2- 172.21.15.119:0/1579830934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560629c0da60 0x560629c11750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.930+0000 7f9d4d39b700 1 --2- 172.21.15.119:0/1579830934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560629c0da60 0x560629c11750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.930+0000 7f9d4d39b700 1 -- 172.21.15.119:0/1579830934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560629c0e380 msgr2=0x560629c11e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.930+0000 7f9d4d39b700 1 --2- 172.21.15.119:0/1579830934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560629c0e380 0x560629c11e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.930+0000 7f9d4d39b700 1 -- 172.21.15.119:0/1579830934 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d481a6900 con 0x560629c08ad0 2023-12-07T10:25:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.930+0000 7f9d4db9c700 1 --2- 172.21.15.119:0/1579830934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560629c0e380 0x560629c11e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:25:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.930+0000 7f9d4d39b700 1 --2- 172.21.15.119:0/1579830934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560629c08ad0 0x560629c14350 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9d440cc540 tx=0x7f9d44009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5b5d19310cf9512 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.930+0000 7f9d30ff9700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:25:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.930+0000 7f9d30ff9700 1 -- 172.21.15.119:0/1579830934 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4409cea0 con 0x560629c08ad0 2023-12-07T10:25:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.930+0000 7f9d30ff9700 1 -- 172.21.15.119:0/1579830934 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9d440a3e70 con 0x560629c08ad0 2023-12-07T10:25:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.930+0000 7f9d63004b80 1 -- 172.21.15.119:0/1579830934 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560629bfb6f0 con 0x560629c08ad0 2023-12-07T10:25:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.930+0000 7f9d30ff9700 1 -- 172.21.15.119:0/1579830934 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d4409b270 con 0x560629c08ad0 2023-12-07T10:25:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.930+0000 7f9d63004b80 10 client.27260 fetch_fsmap learned FSMap version 45 2023-12-07T10:25:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.930+0000 7f9d63004b80 1 -- 172.21.15.119:0/1579830934 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x560629985d80 con 0x560629c08ad0 2023-12-07T10:25:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.931+0000 7f9d30ff9700 1 -- 172.21.15.119:0/1579830934 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560629bfb6f0 con 0x560629c08ad0 2023-12-07T10:25:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.931+0000 7f9d30ff9700 1 -- 172.21.15.119:0/1579830934 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 45) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9d440a39e0 con 0x560629c08ad0 2023-12-07T10:25:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.931+0000 7f9d63004b80 10 client.27260 fetch_fsmap finished waiting for FSMap version 45 2023-12-07T10:25:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.931+0000 7f9d63004b80 10 client.27260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:25:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.931+0000 7f9d63004b80 20 client.27260 populate_metadata read hostname 'smithi119' 2023-12-07T10:25:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.931+0000 7f9d63004b80 1 --2- 172.21.15.119:0/1579830934 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560629c29d80 0x560629c2c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.931+0000 7f9d63004b80 4 client.27260 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:25:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.931+0000 7f9d63004b80 1 -- 172.21.15.119:0/1579830934 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560629bd7740 con 0x560629c29d80 2023-12-07T10:25:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.931+0000 7f9d4cb9a700 1 --2- 172.21.15.119:0/1579830934 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560629c29d80 0x560629c2c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.932+0000 7f9d4cb9a700 1 --2- 172.21.15.119:0/1579830934 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560629c29d80 0x560629c2c170 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1ec6d538a4cac0b server_cookie=70b4487255314473 in_seq=0 out_seq=0 2023-12-07T10:25:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.932+0000 7f9d30ff9700 10 client.27260 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:25:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.932+0000 7f9d30ff9700 1 -- 172.21.15.119:0/1579830934 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560629bd7740 con 0x560629c29d80 2023-12-07T10:25:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.932+0000 7f9d30ff9700 10 client.27260 handle_command_reply: tid=0 2023-12-07T10:25:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.932+0000 7f9d63004b80 1 client.27260 shutdown 2023-12-07T10:25:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.933+0000 7f9d63004b80 1 -- 172.21.15.119:0/1579830934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560629c08ad0 msgr2=0x560629c14350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.933+0000 7f9d63004b80 1 --2- 172.21.15.119:0/1579830934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560629c08ad0 0x560629c14350 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9d440cc540 tx=0x7f9d44009ca0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.933+0000 7f9d63004b80 1 -- 172.21.15.119:0/1579830934 shutdown_connections 2023-12-07T10:25:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.933+0000 7f9d63004b80 1 --2- 172.21.15.119:0/1579830934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560629c0e380 0x560629c11e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.933+0000 7f9d63004b80 1 --2- 172.21.15.119:0/1579830934 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560629c29d80 0x560629c2c170 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.933+0000 7f9d63004b80 1 --2- 172.21.15.119:0/1579830934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560629c08ad0 0x560629c14350 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.933+0000 7f9d63004b80 1 --2- 172.21.15.119:0/1579830934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560629c0da60 0x560629c11750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.933+0000 7f9d63004b80 1 -- 172.21.15.119:0/1579830934 >> 172.21.15.119:0/1579830934 conn(0x560629b37300 msgr2=0x560629bfd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.933+0000 7f9d30ff9700 0 client.27260 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:25:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.933+0000 7f9d63004b80 1 -- 172.21.15.119:0/1579830934 shutdown_connections 2023-12-07T10:25:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.933+0000 7f9d63004b80 1 -- 172.21.15.119:0/1579830934 wait complete. 2023-12-07T10:25:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.933+0000 7f9d63004b80 20 client.27260 trim_cache size 0 max 16384 2023-12-07T10:25:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.950+0000 7f9d63004b80 1 Processor -- start 2023-12-07T10:25:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.950+0000 7f9d63004b80 1 -- start start 2023-12-07T10:25:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.951+0000 7f9d63004b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560629c08ad0 0x560629c18ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.951+0000 7f9d63004b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560629c0da60 0x560629c19000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.951+0000 7f9d63004b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560629c0e380 0x560629c0b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.951+0000 7f9d63004b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606299db8c0 con 0x560629c0da60 2023-12-07T10:25:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.951+0000 7f9d63004b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x560629bff7a0 con 0x560629c0e380 2023-12-07T10:25:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.951+0000 7f9d63004b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560629c31380 con 0x560629c08ad0 2023-12-07T10:25:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.951+0000 7f9d4d39b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560629c08ad0 0x560629c18ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.951+0000 7f9d4d39b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560629c08ad0 0x560629c18ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:52456/0 (socket says 172.21.15.119:52456) 2023-12-07T10:25:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.951+0000 7f9d4d39b700 1 -- 172.21.15.119:0/631271042 learned_addr learned my addr 172.21.15.119:0/631271042 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:25:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.951+0000 7f9d4db9c700 1 --2- 172.21.15.119:0/631271042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560629c0e380 0x560629c0b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.951+0000 7f9d4cb9a700 1 --2- 172.21.15.119:0/631271042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560629c0da60 0x560629c19000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.952+0000 7f9d4d39b700 1 -- 172.21.15.119:0/631271042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560629c0e380 msgr2=0x560629c0b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.952+0000 7f9d4d39b700 1 --2- 172.21.15.119:0/631271042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560629c0e380 0x560629c0b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.952+0000 7f9d4d39b700 1 -- 172.21.15.119:0/631271042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560629c0da60 msgr2=0x560629c19000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.952+0000 7f9d4d39b700 1 --2- 172.21.15.119:0/631271042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560629c0da60 0x560629c19000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.952+0000 7f9d4d39b700 1 -- 172.21.15.119:0/631271042 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56062996ec00 con 0x560629c08ad0 2023-12-07T10:25:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.952+0000 7f9d4cb9a700 1 --2- 172.21.15.119:0/631271042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560629c0da60 0x560629c19000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:25:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.952+0000 7f9d4d39b700 1 --2- 172.21.15.119:0/631271042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560629c08ad0 0x560629c18ac0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9d44099ec0 tx=0x7f9d4409c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60329119aedfa8ca server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.952+0000 7f9d33fff700 1 -- 172.21.15.119:0/631271042 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4409cea0 con 0x560629c08ad0 2023-12-07T10:25:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.952+0000 7f9d33fff700 1 -- 172.21.15.119:0/631271042 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d4409b6b0 con 0x560629c08ad0 2023-12-07T10:25:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.952+0000 7f9d33fff700 1 -- 172.21.15.119:0/631271042 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4409bdd0 con 0x560629c08ad0 2023-12-07T10:25:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.953+0000 7f9d63004b80 1 -- 172.21.15.119:0/631271042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560629c08ad0 msgr2=0x560629c18ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.953+0000 7f9d63004b80 1 --2- 172.21.15.119:0/631271042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560629c08ad0 0x560629c18ac0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9d44099ec0 tx=0x7f9d4409c3e0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.953+0000 7f9d63004b80 1 -- 172.21.15.119:0/631271042 shutdown_connections 2023-12-07T10:25:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.953+0000 7f9d63004b80 1 --2- 172.21.15.119:0/631271042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560629c0e380 0x560629c0b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.953+0000 7f9d63004b80 1 --2- 172.21.15.119:0/631271042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560629c08ad0 0x560629c18ac0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.953+0000 7f9d63004b80 1 --2- 172.21.15.119:0/631271042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560629c0da60 0x560629c19000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.953+0000 7f9d63004b80 1 -- 172.21.15.119:0/631271042 >> 172.21.15.119:0/631271042 conn(0x560629b37300 msgr2=0x560629c14080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.953+0000 7f9d63004b80 1 -- 172.21.15.119:0/631271042 shutdown_connections 2023-12-07T10:25:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.953+0000 7f9d63004b80 1 -- 172.21.15.119:0/631271042 wait complete. 2023-12-07T10:25:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.953+0000 7f9d63004b80 1 -- start start 2023-12-07T10:25:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.954+0000 7f9d63004b80 1 Processor -- start 2023-12-07T10:25:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.954+0000 7f9d63004b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560629c08ad0 0x560629c18e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.954+0000 7f9d63004b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560629c0da60 0x560629c16aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.954+0000 7f9d63004b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560629c0e380 0x560629c16fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.954+0000 7f9d63004b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560629c176c0 con 0x560629c0e380 2023-12-07T10:25:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.954+0000 7f9d63004b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x560629c17830 con 0x560629c0da60 2023-12-07T10:25:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.954+0000 7f9d63004b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560629c7f270 con 0x560629c08ad0 2023-12-07T10:25:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.955+0000 7f9d63004b80 10 client.0 _ll_register_callbacks cb 0x560629afd8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:25:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.955+0000 7f9d4d39b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560629c08ad0 0x560629c18e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.955+0000 7f9d4d39b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560629c08ad0 0x560629c18e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:52472/0 (socket says 172.21.15.119:52472) 2023-12-07T10:25:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.955+0000 7f9d4d39b700 1 -- 172.21.15.119:0/1668528394 learned_addr learned my addr 172.21.15.119:0/1668528394 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:25:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.955+0000 7f9d4cb9a700 1 --2- 172.21.15.119:0/1668528394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560629c0da60 0x560629c16aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.955+0000 7f9d4db9c700 1 --2- 172.21.15.119:0/1668528394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560629c0e380 0x560629c16fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.955+0000 7f9d4d39b700 1 -- 172.21.15.119:0/1668528394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560629c0da60 msgr2=0x560629c16aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.955+0000 7f9d4d39b700 1 --2- 172.21.15.119:0/1668528394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560629c0da60 0x560629c16aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.955+0000 7f9d4d39b700 1 -- 172.21.15.119:0/1668528394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560629c0e380 msgr2=0x560629c16fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:25:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.955+0000 7f9d4d39b700 1 --2- 172.21.15.119:0/1668528394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560629c0e380 0x560629c16fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.955+0000 7f9d4d39b700 1 -- 172.21.15.119:0/1668528394 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560629985d80 con 0x560629c08ad0 2023-12-07T10:25:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.955+0000 7f9d4cb9a700 1 --2- 172.21.15.119:0/1668528394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560629c0da60 0x560629c16aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:25:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.955+0000 7f9d4db9c700 1 --2- 172.21.15.119:0/1668528394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560629c0e380 0x560629c16fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:25:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.955+0000 7f9d4d39b700 1 --2- 172.21.15.119:0/1668528394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560629c08ad0 0x560629c18e40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9d44022c00 tx=0x7f9d4400b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2cb6c63674755ea server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:25:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.955+0000 7f9d30ff9700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:25:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.956+0000 7f9d30ff9700 1 -- 172.21.15.119:0/1668528394 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4409cea0 con 0x560629c08ad0 2023-12-07T10:25:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.956+0000 7f9d30ff9700 1 -- 172.21.15.119:0/1668528394 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f9d440a3e70 con 0x560629c08ad0 2023-12-07T10:25:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.956+0000 7f9d63004b80 1 -- 172.21.15.119:0/1668528394 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560629b22f70 con 0x560629c08ad0 2023-12-07T10:25:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.956+0000 7f9d30ff9700 1 -- 172.21.15.119:0/1668528394 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d4409b6b0 con 0x560629c08ad0 2023-12-07T10:25:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.956+0000 7f9d63004b80 10 client.27244 fetch_fsmap learned FSMap version 45 2023-12-07T10:25:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.956+0000 7f9d63004b80 1 -- 172.21.15.119:0/1668528394 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5606299ac1f0 con 0x560629c08ad0 2023-12-07T10:25:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.956+0000 7f9d30ff9700 1 -- 172.21.15.119:0/1668528394 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560629b22f70 con 0x560629c08ad0 2023-12-07T10:25:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.956+0000 7f9d30ff9700 1 -- 172.21.15.119:0/1668528394 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 45) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f9d4409aad0 con 0x560629c08ad0 2023-12-07T10:25:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.956+0000 7f9d63004b80 10 client.27244 fetch_fsmap finished waiting for FSMap version 45 2023-12-07T10:25:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.956+0000 7f9d63004b80 10 client.27244 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:25:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.956+0000 7f9d63004b80 20 client.27244 populate_metadata read hostname 'smithi119' 2023-12-07T10:25:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.956+0000 7f9d63004b80 1 --2- 172.21.15.119:0/1668528394 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560629c804c0 0x560629c828b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:25:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.956+0000 7f9d63004b80 4 client.27244 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:25:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.956+0000 7f9d63004b80 1 -- 172.21.15.119:0/1668528394 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560629b15430 con 0x560629c804c0 2023-12-07T10:25:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.957+0000 7f9d4cb9a700 1 --2- 172.21.15.119:0/1668528394 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560629c804c0 0x560629c828b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:25:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.957+0000 7f9d4cb9a700 1 --2- 172.21.15.119:0/1668528394 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560629c804c0 0x560629c828b0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e5be4968c8b2282 server_cookie=aa4bf77119971589 in_seq=0 out_seq=0 2023-12-07T10:25:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.957+0000 7f9d30ff9700 10 client.27244 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:25:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.998+0000 7f9d30ff9700 1 -- 172.21.15.119:0/1668528394 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560629b15430 con 0x560629c804c0 2023-12-07T10:25:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.998+0000 7f9d30ff9700 10 client.27244 handle_command_reply: tid=0 2023-12-07T10:25:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.998+0000 7f9d63004b80 1 client.27244 shutdown 2023-12-07T10:25:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.998+0000 7f9d63004b80 1 -- 172.21.15.119:0/1668528394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560629c08ad0 msgr2=0x560629c18e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.998+0000 7f9d63004b80 1 --2- 172.21.15.119:0/1668528394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560629c08ad0 0x560629c18e40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9d44022c00 tx=0x7f9d4400b690 comp rx=0 tx=0).stop 2023-12-07T10:25:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.998+0000 7f9d63004b80 1 -- 172.21.15.119:0/1668528394 shutdown_connections 2023-12-07T10:25:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.998+0000 7f9d63004b80 1 --2- 172.21.15.119:0/1668528394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560629c0e380 0x560629c16fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.998+0000 7f9d63004b80 1 --2- 172.21.15.119:0/1668528394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560629c08ad0 0x560629c18e40 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.998+0000 7f9d63004b80 1 --2- 172.21.15.119:0/1668528394 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x560629c804c0 0x560629c828b0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.998+0000 7f9d63004b80 1 --2- 172.21.15.119:0/1668528394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560629c0da60 0x560629c16aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.998+0000 7f9d30ff9700 0 client.27244 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:25:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.998+0000 7f9d63004b80 1 -- 172.21.15.119:0/1668528394 >> 172.21.15.119:0/1668528394 conn(0x560629b37300 msgr2=0x560629c0b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.998+0000 7f9d63004b80 1 -- 172.21.15.119:0/1668528394 shutdown_connections 2023-12-07T10:25:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.998+0000 7f9d63004b80 1 -- 172.21.15.119:0/1668528394 wait complete. 2023-12-07T10:25:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.998+0000 7f9d63004b80 20 client.27244 trim_cache size 0 max 16384 2023-12-07T10:25:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:25:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:25:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:25:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:25:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.999+0000 7f9d4f5ff700 1 -- 172.21.15.119:0/3625597246 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9d340a1c80 msgr2=0x7f9d340a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.999+0000 7f9d4f5ff700 1 --2- 172.21.15.119:0/3625597246 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9d340a1c80 0x7f9d340a4140 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9d4806ad30 tx=0x7f9d40009920 comp rx=0 tx=0).stop 2023-12-07T10:25:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.999+0000 7f9d4f5ff700 1 -- 172.21.15.119:0/3625597246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d48100ef0 msgr2=0x7f9d4819f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:25:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.999+0000 7f9d4f5ff700 1 --2- 172.21.15.119:0/3625597246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d48100ef0 0x7f9d4819f910 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9d44005910 tx=0x7f9d44004730 comp rx=0 tx=0).stop 2023-12-07T10:25:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.999+0000 7f9d4f5ff700 1 -- 172.21.15.119:0/3625597246 shutdown_connections 2023-12-07T10:25:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.999+0000 7f9d4f5ff700 1 --2- 172.21.15.119:0/3625597246 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9d340a1c80 0x7f9d340a4140 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.999+0000 7f9d4f5ff700 1 --2- 172.21.15.119:0/3625597246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d4810df60 0x7f9d48199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.999+0000 7f9d4f5ff700 1 --2- 172.21.15.119:0/3625597246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d48100ef0 0x7f9d4819f910 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.999+0000 7f9d4f5ff700 1 --2- 172.21.15.119:0/3625597246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d48101810 0x7f9d4819fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:25:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.999+0000 7f9d4f5ff700 1 -- 172.21.15.119:0/3625597246 >> 172.21.15.119:0/3625597246 conn(0x7f9d480775b0 msgr2=0x7f9d4810f720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:25:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.999+0000 7f9d4f5ff700 1 -- 172.21.15.119:0/3625597246 shutdown_connections 2023-12-07T10:25:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:25:58.999+0000 7f9d4f5ff700 1 -- 172.21.15.119:0/3625597246 wait complete. 2023-12-07T10:25:59.558 INFO:tasks.workunit.client.0.smithi119.stdout: 1 359157 6.97 MB/sec execute 132 sec latency 39.493 ms 2023-12-07T10:26:00.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:00 smithi119 ceph-mon[97908]: pgmap v170: 129 pgs: 129 active+clean; 193 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 372 KiB/s wr, 59 op/s 2023-12-07T10:26:00.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:00 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:26:00.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:00 smithi159 ceph-mon[134726]: pgmap v170: 129 pgs: 129 active+clean; 193 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 372 KiB/s wr, 59 op/s 2023-12-07T10:26:00.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:00 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:26:00.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:00 smithi153 ceph-mon[133625]: pgmap v170: 129 pgs: 129 active+clean; 193 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 372 KiB/s wr, 59 op/s 2023-12-07T10:26:00.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:00 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:26:00.558 INFO:tasks.workunit.client.0.smithi119.stdout: 1 361023 7.00 MB/sec execute 133 sec latency 16.098 ms 2023-12-07T10:26:01.565 INFO:tasks.workunit.client.0.smithi119.stdout: 1 361744 6.99 MB/sec execute 134 sec latency 685.919 ms 2023-12-07T10:26:02.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:01 smithi119 ceph-mon[97908]: pgmap v171: 129 pgs: 129 active+clean; 199 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 945 KiB/s wr, 163 op/s 2023-12-07T10:26:02.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:01 smithi159 ceph-mon[134726]: pgmap v171: 129 pgs: 129 active+clean; 199 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 945 KiB/s wr, 163 op/s 2023-12-07T10:26:02.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:01 smithi153 ceph-mon[133625]: pgmap v171: 129 pgs: 129 active+clean; 199 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 945 KiB/s wr, 163 op/s 2023-12-07T10:26:02.558 INFO:tasks.workunit.client.0.smithi119.stdout: 1 361744 6.94 MB/sec execute 135 sec latency 1686.039 ms 2023-12-07T10:26:03.558 INFO:tasks.workunit.client.0.smithi119.stdout: 1 361984 6.90 MB/sec execute 136 sec latency 2114.087 ms 2023-12-07T10:26:04.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:04 smithi153 ceph-mon[133625]: pgmap v172: 129 pgs: 129 active+clean; 199 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 932 KiB/s wr, 155 op/s 2023-12-07T10:26:04.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:04 smithi119 ceph-mon[97908]: pgmap v172: 129 pgs: 129 active+clean; 199 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 932 KiB/s wr, 155 op/s 2023-12-07T10:26:04.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:04 smithi159 ceph-mon[134726]: pgmap v172: 129 pgs: 129 active+clean; 199 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 932 KiB/s wr, 155 op/s 2023-12-07T10:26:04.558 INFO:tasks.workunit.client.0.smithi119.stdout: 1 362396 6.85 MB/sec execute 137 sec latency 65.923 ms 2023-12-07T10:26:05.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:05 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:26:05.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:05 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:26:05.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:05 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:26:05.558 INFO:tasks.workunit.client.0.smithi119.stdout: 1 364169 6.85 MB/sec execute 138 sec latency 35.723 ms 2023-12-07T10:26:06.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:06 smithi119 ceph-mon[97908]: pgmap v173: 129 pgs: 129 active+clean; 199 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 932 KiB/s wr, 155 op/s 2023-12-07T10:26:06.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:06 smithi159 ceph-mon[134726]: pgmap v173: 129 pgs: 129 active+clean; 199 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 932 KiB/s wr, 155 op/s 2023-12-07T10:26:06.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:06 smithi153 ceph-mon[133625]: pgmap v173: 129 pgs: 129 active+clean; 199 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 932 KiB/s wr, 155 op/s 2023-12-07T10:26:06.558 INFO:tasks.workunit.client.0.smithi119.stdout: 1 365707 6.89 MB/sec execute 139 sec latency 11.458 ms 2023-12-07T10:26:07.558 INFO:tasks.workunit.client.0.smithi119.stdout: 1 366835 6.88 MB/sec execute 140 sec latency 17.117 ms 2023-12-07T10:26:08.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:08 smithi119 ceph-mon[97908]: pgmap v174: 129 pgs: 129 active+clean; 202 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 229 op/s 2023-12-07T10:26:08.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:08 smithi159 ceph-mon[134726]: pgmap v174: 129 pgs: 129 active+clean; 202 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 229 op/s 2023-12-07T10:26:08.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:08 smithi153 ceph-mon[133625]: pgmap v174: 129 pgs: 129 active+clean; 202 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 229 op/s 2023-12-07T10:26:08.558 INFO:tasks.workunit.client.0.smithi119.stdout: 1 368711 6.89 MB/sec execute 141 sec latency 19.399 ms 2023-12-07T10:26:09.558 INFO:tasks.workunit.client.0.smithi119.stdout: 1 369635 6.88 MB/sec execute 142 sec latency 60.767 ms 2023-12-07T10:26:10.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:10 smithi119 ceph-mon[97908]: pgmap v175: 129 pgs: 129 active+clean; 202 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 966 KiB/s wr, 177 op/s 2023-12-07T10:26:10.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:10 smithi159 ceph-mon[134726]: pgmap v175: 129 pgs: 129 active+clean; 202 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 966 KiB/s wr, 177 op/s 2023-12-07T10:26:10.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:10 smithi153 ceph-mon[133625]: pgmap v175: 129 pgs: 129 active+clean; 202 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 966 KiB/s wr, 177 op/s 2023-12-07T10:26:10.559 INFO:tasks.workunit.client.0.smithi119.stdout: 1 371292 6.88 MB/sec execute 143 sec latency 67.200 ms 2023-12-07T10:26:11.558 INFO:tasks.workunit.client.0.smithi119.stdout: 1 372804 6.92 MB/sec execute 144 sec latency 18.168 ms 2023-12-07T10:26:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.163+0000 7ff5191ff700 1 -- 172.21.15.119:0/4260385969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5141075e0 msgr2=0x7ff5141079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.163+0000 7ff5191ff700 1 --2- 172.21.15.119:0/4260385969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5141075e0 0x7ff5141079c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff504009990 tx=0x7ff504009ca0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.163+0000 7ff5191ff700 1 -- 172.21.15.119:0/4260385969 shutdown_connections 2023-12-07T10:26:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.163+0000 7ff5191ff700 1 --2- 172.21.15.119:0/4260385969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff514101ee0 0x7ff51410e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.163+0000 7ff5191ff700 1 --2- 172.21.15.119:0/4260385969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5141075e0 0x7ff5141079c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.163+0000 7ff5191ff700 1 --2- 172.21.15.119:0/4260385969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff514101520 0x7ff5141019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.163+0000 7ff5191ff700 1 -- 172.21.15.119:0/4260385969 >> 172.21.15.119:0/4260385969 conn(0x7ff5140fd010 msgr2=0x7ff5140ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:26:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.163+0000 7ff5191ff700 1 -- 172.21.15.119:0/4260385969 shutdown_connections 2023-12-07T10:26:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.163+0000 7ff5191ff700 1 -- 172.21.15.119:0/4260385969 wait complete. 2023-12-07T10:26:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.164+0000 7ff5191ff700 1 Processor -- start 2023-12-07T10:26:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.164+0000 7ff5191ff700 1 -- start start 2023-12-07T10:26:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.164+0000 7ff5191ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff514101520 0x7ff51419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.164+0000 7ff5191ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff514101ee0 0x7ff51419db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.164+0000 7ff5191ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5141075e0 0x7ff5141977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.164+0000 7ff5191ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff514197e10 con 0x7ff5141075e0 2023-12-07T10:26:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.165+0000 7ff5191ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff514197f80 con 0x7ff514101520 2023-12-07T10:26:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.165+0000 7ff5191ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff514198140 con 0x7ff514101ee0 2023-12-07T10:26:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.165+0000 7ff512d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff514101520 0x7ff51419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.165+0000 7ff512d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff514101520 0x7ff51419d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42120/0 (socket says 172.21.15.119:42120) 2023-12-07T10:26:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.165+0000 7ff512d9d700 1 -- 172.21.15.119:0/3108190549 learned_addr learned my addr 172.21.15.119:0/3108190549 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:26:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.165+0000 7ff51259c700 1 --2- 172.21.15.119:0/3108190549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff514101ee0 0x7ff51419db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.165+0000 7ff51359e700 1 --2- 172.21.15.119:0/3108190549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5141075e0 0x7ff5141977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.165+0000 7ff512d9d700 1 -- 172.21.15.119:0/3108190549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff514101ee0 msgr2=0x7ff51419db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.165+0000 7ff512d9d700 1 --2- 172.21.15.119:0/3108190549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff514101ee0 0x7ff51419db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.165+0000 7ff512d9d700 1 -- 172.21.15.119:0/3108190549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5141075e0 msgr2=0x7ff5141977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.165+0000 7ff512d9d700 1 --2- 172.21.15.119:0/3108190549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5141075e0 0x7ff5141977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.165+0000 7ff512d9d700 1 -- 172.21.15.119:0/3108190549 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5040095f0 con 0x7ff514101520 2023-12-07T10:26:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.165+0000 7ff512d9d700 1 --2- 172.21.15.119:0/3108190549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff514101520 0x7ff51419d630 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff504005910 tx=0x7ff504004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:26:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.166+0000 7ff4fbfff700 1 -- 172.21.15.119:0/3108190549 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50401d070 con 0x7ff514101520 2023-12-07T10:26:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.166+0000 7ff4fbfff700 1 -- 172.21.15.119:0/3108190549 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff504022470 con 0x7ff514101520 2023-12-07T10:26:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.166+0000 7ff4fbfff700 1 -- 172.21.15.119:0/3108190549 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504017650 con 0x7ff514101520 2023-12-07T10:26:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.166+0000 7ff5191ff700 1 -- 172.21.15.119:0/3108190549 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5141983c0 con 0x7ff514101520 2023-12-07T10:26:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.166+0000 7ff5191ff700 1 -- 172.21.15.119:0/3108190549 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5141a4620 con 0x7ff514101520 2023-12-07T10:26:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.169+0000 7ff4fbfff700 1 -- 172.21.15.119:0/3108190549 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7ff5040225e0 con 0x7ff514101520 2023-12-07T10:26:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.171+0000 7ff52cb3bb80 1 Processor -- start 2023-12-07T10:26:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.171+0000 7ff52cb3bb80 1 -- start start 2023-12-07T10:26:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.171+0000 7ff4fbfff700 1 --2- 172.21.15.119:0/3108190549 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff4fc0a1d00 0x7ff4fc0a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.171+0000 7ff52cb3bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d76854a60 0x562d76854e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.171+0000 7ff52cb3bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d76855380 0x562d7684f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.171+0000 7ff52cb3bb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d7684fad0 0x562d7684ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.171+0000 7ff4fbfff700 1 -- 172.21.15.119:0/3108190549 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7ff5040cefe0 con 0x7ff514101520 2023-12-07T10:26:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.171+0000 7ff52cb3bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d767c3760 con 0x562d76855380 2023-12-07T10:26:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.171+0000 7ff52cb3bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d767aa740 con 0x562d76854a60 2023-12-07T10:26:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.171+0000 7ff52cb3bb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d767a8060 con 0x562d7684fad0 2023-12-07T10:26:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.172+0000 7ff51359e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d7684fad0 0x562d7684ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.172+0000 7ff512d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d76854a60 0x562d76854e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.172+0000 7ff51359e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d7684fad0 0x562d7684ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:57634/0 (socket says 172.21.15.119:57634) 2023-12-07T10:26:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.172+0000 7ff51359e700 1 -- 172.21.15.119:0/3630521427 learned_addr learned my addr 172.21.15.119:0/3630521427 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:26:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.172+0000 7ff51359e700 1 --2- 172.21.15.119:0/3108190549 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff4fc0a1d00 0x7ff4fc0a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.172+0000 7ff51259c700 1 --2- 172.21.15.119:0/3630521427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d76855380 0x562d7684f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.172+0000 7ff512d9d700 1 -- 172.21.15.119:0/3630521427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d7684fad0 msgr2=0x562d7684ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.172+0000 7ff512d9d700 1 --2- 172.21.15.119:0/3630521427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d7684fad0 0x562d7684ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.172+0000 7ff512d9d700 1 -- 172.21.15.119:0/3630521427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d76855380 msgr2=0x562d7684f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.172+0000 7ff512d9d700 1 --2- 172.21.15.119:0/3630521427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d76855380 0x562d7684f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.172+0000 7ff512d9d700 1 -- 172.21.15.119:0/3630521427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d766205d0 con 0x562d76854a60 2023-12-07T10:26:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.172+0000 7ff51259c700 1 --2- 172.21.15.119:0/3630521427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d76855380 0x562d7684f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:26:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.172+0000 7ff512d9d700 1 --2- 172.21.15.119:0/3630521427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d76854a60 0x562d76854e40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff5040cc3a0 tx=0x7ff504009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6e513c677590343 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:26:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.173+0000 7ff51359e700 1 --2- 172.21.15.119:0/3108190549 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff4fc0a1d00 0x7ff4fc0a41c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x562d7684b670 tx=0x7ff50c00b040 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:26:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.173+0000 7ff4f97fa700 1 -- 172.21.15.119:0/3630521427 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50409c180 con 0x562d76854a60 2023-12-07T10:26:12.180 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.173+0000 7ff4f97fa700 1 -- 172.21.15.119:0/3630521427 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff50409ba80 con 0x562d76854a60 2023-12-07T10:26:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.173+0000 7ff4f97fa700 1 -- 172.21.15.119:0/3630521427 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5040862d0 con 0x562d76854a60 2023-12-07T10:26:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.174+0000 7ff52cb3bb80 1 -- 172.21.15.119:0/3630521427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d76854a60 msgr2=0x562d76854e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.174+0000 7ff52cb3bb80 1 --2- 172.21.15.119:0/3630521427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d76854a60 0x562d76854e40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff5040cc3a0 tx=0x7ff504009ca0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.174+0000 7ff52cb3bb80 1 -- 172.21.15.119:0/3630521427 shutdown_connections 2023-12-07T10:26:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.174+0000 7ff52cb3bb80 1 --2- 172.21.15.119:0/3630521427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d7684fad0 0x562d7684ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.174+0000 7ff52cb3bb80 1 --2- 172.21.15.119:0/3630521427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d76854a60 0x562d76854e40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.174+0000 7ff52cb3bb80 1 --2- 172.21.15.119:0/3630521427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d76855380 0x562d7684f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.174+0000 7ff52cb3bb80 1 -- 172.21.15.119:0/3630521427 >> 172.21.15.119:0/3630521427 conn(0x562d7677e300 msgr2=0x562d76849b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:26:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.174+0000 7ff52cb3bb80 1 -- 172.21.15.119:0/3630521427 shutdown_connections 2023-12-07T10:26:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.174+0000 7ff52cb3bb80 1 -- 172.21.15.119:0/3630521427 wait complete. 2023-12-07T10:26:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.174+0000 7ff52cb3bb80 1 -- start start 2023-12-07T10:26:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.175+0000 7ff52cb3bb80 1 Processor -- start 2023-12-07T10:26:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.175+0000 7ff52cb3bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d7684fad0 0x562d768587c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.175+0000 7ff52cb3bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d76854a60 0x562d76858d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.175+0000 7ff52cb3bb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d76855380 0x562d76859240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.175+0000 7ff52cb3bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d767a3950 con 0x562d7684fad0 2023-12-07T10:26:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.175+0000 7ff52cb3bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7675d500 con 0x562d76854a60 2023-12-07T10:26:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.175+0000 7ff52cb3bb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7675d1a0 con 0x562d76855380 2023-12-07T10:26:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.175+0000 7ff52cb3bb80 10 client.0 _ll_register_callbacks cb 0x562d767448e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:26:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.175+0000 7ff51259c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d76854a60 0x562d76858d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.175+0000 7ff51359e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d76855380 0x562d76859240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.175+0000 7ff51259c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d76854a60 0x562d76858d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42128/0 (socket says 172.21.15.119:42128) 2023-12-07T10:26:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.175+0000 7ff51259c700 1 -- 172.21.15.119:0/2256307115 learned_addr learned my addr 172.21.15.119:0/2256307115 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:26:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.175+0000 7ff512d9d700 1 --2- 172.21.15.119:0/2256307115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d7684fad0 0x562d768587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.176+0000 7ff51259c700 1 -- 172.21.15.119:0/2256307115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d76855380 msgr2=0x562d76859240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.176+0000 7ff51259c700 1 --2- 172.21.15.119:0/2256307115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d76855380 0x562d76859240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.176+0000 7ff51259c700 1 -- 172.21.15.119:0/2256307115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d7684fad0 msgr2=0x562d768587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.176+0000 7ff51259c700 1 --2- 172.21.15.119:0/2256307115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d7684fad0 0x562d768587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.176+0000 7ff51259c700 1 -- 172.21.15.119:0/2256307115 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5141a4620 con 0x562d76854a60 2023-12-07T10:26:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.176+0000 7ff51359e700 1 --2- 172.21.15.119:0/2256307115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d76855380 0x562d76859240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:26:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.176+0000 7ff51259c700 1 --2- 172.21.15.119:0/2256307115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d76854a60 0x562d76858d00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff508012700 tx=0x7ff508012a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d94e22f7ccc8d2 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:26:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.176+0000 7ff4f6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:26:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.176+0000 7ff4f6ff5700 1 -- 172.21.15.119:0/2256307115 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff508017070 con 0x562d76854a60 2023-12-07T10:26:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.176+0000 7ff52cb3bb80 1 -- 172.21.15.119:0/2256307115 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d768426f0 con 0x562d76854a60 2023-12-07T10:26:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.176+0000 7ff4f6ff5700 1 -- 172.21.15.119:0/2256307115 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7ff508004830 con 0x562d76854a60 2023-12-07T10:26:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.177+0000 7ff4f6ff5700 1 -- 172.21.15.119:0/2256307115 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5080049f0 con 0x562d76854a60 2023-12-07T10:26:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.177+0000 7ff52cb3bb80 10 client.27278 fetch_fsmap learned FSMap version 47 2023-12-07T10:26:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.177+0000 7ff52cb3bb80 1 -- 172.21.15.119:0/2256307115 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x562d765ccd80 con 0x562d76854a60 2023-12-07T10:26:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.177+0000 7ff4f6ff5700 1 -- 172.21.15.119:0/2256307115 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562d768426f0 con 0x562d76854a60 2023-12-07T10:26:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.177+0000 7ff4f6ff5700 1 -- 172.21.15.119:0/2256307115 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 47) v1 ==== 7910+0+0 (secure 0 0 0) 0x7ff508010550 con 0x562d76854a60 2023-12-07T10:26:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.177+0000 7ff52cb3bb80 10 client.27278 fetch_fsmap finished waiting for FSMap version 47 2023-12-07T10:26:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.177+0000 7ff52cb3bb80 10 client.27278 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:26:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.177+0000 7ff52cb3bb80 20 client.27278 populate_metadata read hostname 'smithi119' 2023-12-07T10:26:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.177+0000 7ff52cb3bb80 1 --2- 172.21.15.119:0/2256307115 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562d76868c70 0x562d7686b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.177+0000 7ff52cb3bb80 4 client.27278 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:26:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.177+0000 7ff52cb3bb80 1 -- 172.21.15.119:0/2256307115 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d7681e740 con 0x562d76868c70 2023-12-07T10:26:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.177+0000 7ff512d9d700 1 --2- 172.21.15.119:0/2256307115 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562d76868c70 0x562d7686b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.178+0000 7ff4f6ff5700 10 client.27278 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:26:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.178+0000 7ff512d9d700 1 --2- 172.21.15.119:0/2256307115 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562d76868c70 0x562d7686b090 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f71bf380bbb335e6 server_cookie=29f7b656e3e9e87f in_seq=0 out_seq=0 2023-12-07T10:26:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.178+0000 7ff4f6ff5700 1 -- 172.21.15.119:0/2256307115 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562d7681e740 con 0x562d76868c70 2023-12-07T10:26:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.179+0000 7ff4f6ff5700 10 client.27278 handle_command_reply: tid=0 2023-12-07T10:26:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.179+0000 7ff52cb3bb80 1 client.27278 shutdown 2023-12-07T10:26:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.179+0000 7ff52cb3bb80 1 -- 172.21.15.119:0/2256307115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d76854a60 msgr2=0x562d76858d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.179+0000 7ff52cb3bb80 1 --2- 172.21.15.119:0/2256307115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d76854a60 0x562d76858d00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff508012700 tx=0x7ff508012a10 comp rx=0 tx=0).stop 2023-12-07T10:26:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.179+0000 7ff52cb3bb80 1 -- 172.21.15.119:0/2256307115 shutdown_connections 2023-12-07T10:26:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.179+0000 7ff52cb3bb80 1 --2- 172.21.15.119:0/2256307115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d76855380 0x562d76859240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.179+0000 7ff52cb3bb80 1 --2- 172.21.15.119:0/2256307115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d7684fad0 0x562d768587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.179+0000 7ff52cb3bb80 1 --2- 172.21.15.119:0/2256307115 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562d76868c70 0x562d7686b090 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.179+0000 7ff52cb3bb80 1 --2- 172.21.15.119:0/2256307115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d76854a60 0x562d76858d00 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.179+0000 7ff52cb3bb80 1 -- 172.21.15.119:0/2256307115 >> 172.21.15.119:0/2256307115 conn(0x562d7677e300 msgr2=0x562d76844980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:26:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.179+0000 7ff4f6ff5700 0 client.27278 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:26:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.179+0000 7ff52cb3bb80 1 -- 172.21.15.119:0/2256307115 shutdown_connections 2023-12-07T10:26:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.179+0000 7ff52cb3bb80 1 -- 172.21.15.119:0/2256307115 wait complete. 2023-12-07T10:26:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.179+0000 7ff52cb3bb80 20 client.27278 trim_cache size 0 max 16384 2023-12-07T10:26:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.199+0000 7ff52cb3bb80 1 Processor -- start 2023-12-07T10:26:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.199+0000 7ff52cb3bb80 1 -- start start 2023-12-07T10:26:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.200+0000 7ff52cb3bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d7684fad0 0x562d7684b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.200+0000 7ff52cb3bb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d76854a60 0x562d7684f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.200+0000 7ff52cb3bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d76855380 0x562d76851ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.200+0000 7ff52cb3bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d766228c0 con 0x562d7684fad0 2023-12-07T10:26:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.200+0000 7ff52cb3bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d76859dc0 con 0x562d76855380 2023-12-07T10:26:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.200+0000 7ff52cb3bb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d76870260 con 0x562d76854a60 2023-12-07T10:26:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.200+0000 7ff51259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d76854a60 0x562d7684f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.200+0000 7ff512d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d7684fad0 0x562d7684b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.200+0000 7ff51259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d76854a60 0x562d7684f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:57650/0 (socket says 172.21.15.119:57650) 2023-12-07T10:26:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.200+0000 7ff51259c700 1 -- 172.21.15.119:0/2767810813 learned_addr learned my addr 172.21.15.119:0/2767810813 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:26:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.200+0000 7ff51359e700 1 --2- 172.21.15.119:0/2767810813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d76855380 0x562d76851ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.200+0000 7ff51259c700 1 -- 172.21.15.119:0/2767810813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d76855380 msgr2=0x562d76851ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.200+0000 7ff51259c700 1 --2- 172.21.15.119:0/2767810813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d76855380 0x562d76851ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.200+0000 7ff51259c700 1 -- 172.21.15.119:0/2767810813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d7684fad0 msgr2=0x562d7684b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.200+0000 7ff51259c700 1 --2- 172.21.15.119:0/2767810813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d7684fad0 0x562d7684b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.200+0000 7ff51259c700 1 -- 172.21.15.119:0/2767810813 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d765b5c00 con 0x562d76854a60 2023-12-07T10:26:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.200+0000 7ff51359e700 1 --2- 172.21.15.119:0/2767810813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d76855380 0x562d76851ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:26:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.200+0000 7ff512d9d700 1 --2- 172.21.15.119:0/2767810813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d7684fad0 0x562d7684b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:26:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.201+0000 7ff51259c700 1 --2- 172.21.15.119:0/2767810813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d76854a60 0x562d7684f540 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff508010930 tx=0x7ff5080096d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=570529b0cbd7909 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:26:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.201+0000 7ff4f9ffb700 1 -- 172.21.15.119:0/2767810813 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff508017070 con 0x562d76854a60 2023-12-07T10:26:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.201+0000 7ff4f9ffb700 1 -- 172.21.15.119:0/2767810813 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff508018040 con 0x562d76854a60 2023-12-07T10:26:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.201+0000 7ff4f9ffb700 1 -- 172.21.15.119:0/2767810813 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5080038e0 con 0x562d76854a60 2023-12-07T10:26:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.202+0000 7ff52cb3bb80 1 -- 172.21.15.119:0/2767810813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d76854a60 msgr2=0x562d7684f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.202+0000 7ff52cb3bb80 1 --2- 172.21.15.119:0/2767810813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d76854a60 0x562d7684f540 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff508010930 tx=0x7ff5080096d0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.202+0000 7ff52cb3bb80 1 -- 172.21.15.119:0/2767810813 shutdown_connections 2023-12-07T10:26:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.202+0000 7ff52cb3bb80 1 --2- 172.21.15.119:0/2767810813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d76855380 0x562d76851ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.202+0000 7ff52cb3bb80 1 --2- 172.21.15.119:0/2767810813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d7684fad0 0x562d7684b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.202+0000 7ff52cb3bb80 1 --2- 172.21.15.119:0/2767810813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d76854a60 0x562d7684f540 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.202+0000 7ff52cb3bb80 1 -- 172.21.15.119:0/2767810813 >> 172.21.15.119:0/2767810813 conn(0x562d7677e300 msgr2=0x562d76849b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:26:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.202+0000 7ff52cb3bb80 1 -- 172.21.15.119:0/2767810813 shutdown_connections 2023-12-07T10:26:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.202+0000 7ff52cb3bb80 1 -- 172.21.15.119:0/2767810813 wait complete. 2023-12-07T10:26:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.202+0000 7ff52cb3bb80 1 -- start start 2023-12-07T10:26:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.202+0000 7ff52cb3bb80 1 Processor -- start 2023-12-07T10:26:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.203+0000 7ff52cb3bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d7684fad0 0x562d7685fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.203+0000 7ff52cb3bb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d76854a60 0x562d7685d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.203+0000 7ff52cb3bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d76855380 0x562d7685def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.203+0000 7ff52cb3bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7685e430 con 0x562d7684fad0 2023-12-07T10:26:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.203+0000 7ff52cb3bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7685e5a0 con 0x562d76855380 2023-12-07T10:26:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.203+0000 7ff52cb3bb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7685e760 con 0x562d76854a60 2023-12-07T10:26:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.203+0000 7ff52cb3bb80 10 client.0 _ll_register_callbacks cb 0x562d767448e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:26:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.203+0000 7ff51259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d76854a60 0x562d7685d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.203+0000 7ff51259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d76854a60 0x562d7685d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:57658/0 (socket says 172.21.15.119:57658) 2023-12-07T10:26:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.203+0000 7ff51259c700 1 -- 172.21.15.119:0/3431957897 learned_addr learned my addr 172.21.15.119:0/3431957897 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:26:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.203+0000 7ff512d9d700 1 --2- 172.21.15.119:0/3431957897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d7684fad0 0x562d7685fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.204+0000 7ff51359e700 1 --2- 172.21.15.119:0/3431957897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d76855380 0x562d7685def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.204+0000 7ff51259c700 1 -- 172.21.15.119:0/3431957897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d76855380 msgr2=0x562d7685def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.204+0000 7ff51259c700 1 --2- 172.21.15.119:0/3431957897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d76855380 0x562d7685def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.204+0000 7ff51259c700 1 -- 172.21.15.119:0/3431957897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d7684fad0 msgr2=0x562d7685fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.204+0000 7ff51259c700 1 --2- 172.21.15.119:0/3431957897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d7684fad0 0x562d7685fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.204+0000 7ff51259c700 1 -- 172.21.15.119:0/3431957897 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d766205d0 con 0x562d76854a60 2023-12-07T10:26:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.204+0000 7ff512d9d700 1 --2- 172.21.15.119:0/3431957897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d7684fad0 0x562d7685fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:26:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.204+0000 7ff51259c700 1 --2- 172.21.15.119:0/3431957897 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d76854a60 0x562d7685d9b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff508010470 tx=0x7ff508012020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8eed116ea0aa4a8 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:26:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.204+0000 7ff4f6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:26:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.204+0000 7ff4f6ff5700 1 -- 172.21.15.119:0/3431957897 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff508035070 con 0x562d76854a60 2023-12-07T10:26:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.204+0000 7ff4f6ff5700 1 -- 172.21.15.119:0/3431957897 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7ff5080050b0 con 0x562d76854a60 2023-12-07T10:26:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.204+0000 7ff52cb3bb80 1 -- 172.21.15.119:0/3431957897 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff50409ba80 con 0x562d76854a60 2023-12-07T10:26:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.205+0000 7ff4f6ff5700 1 -- 172.21.15.119:0/3431957897 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff508017050 con 0x562d76854a60 2023-12-07T10:26:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.205+0000 7ff52cb3bb80 10 client.27262 fetch_fsmap learned FSMap version 47 2023-12-07T10:26:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.205+0000 7ff52cb3bb80 1 -- 172.21.15.119:0/3431957897 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x562d765ccd80 con 0x562d76854a60 2023-12-07T10:26:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.205+0000 7ff4f6ff5700 1 -- 172.21.15.119:0/3431957897 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff50409ba80 con 0x562d76854a60 2023-12-07T10:26:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.205+0000 7ff4f6ff5700 1 -- 172.21.15.119:0/3431957897 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 47) v1 ==== 7910+0+0 (secure 0 0 0) 0x7ff508038030 con 0x562d76854a60 2023-12-07T10:26:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.205+0000 7ff52cb3bb80 10 client.27262 fetch_fsmap finished waiting for FSMap version 47 2023-12-07T10:26:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.205+0000 7ff52cb3bb80 10 client.27262 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:26:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.205+0000 7ff52cb3bb80 20 client.27262 populate_metadata read hostname 'smithi119' 2023-12-07T10:26:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.205+0000 7ff52cb3bb80 1 --2- 172.21.15.119:0/3431957897 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562d768bf280 0x562d768c16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.205+0000 7ff52cb3bb80 4 client.27262 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:26:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.205+0000 7ff52cb3bb80 1 -- 172.21.15.119:0/3431957897 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562d76769f70 con 0x562d768bf280 2023-12-07T10:26:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.205+0000 7ff512d9d700 1 --2- 172.21.15.119:0/3431957897 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562d768bf280 0x562d768c16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.206+0000 7ff4f6ff5700 10 client.27262 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:26:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.206+0000 7ff512d9d700 1 --2- 172.21.15.119:0/3431957897 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562d768bf280 0x562d768c16a0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c79230377f0b16f server_cookie=7a1ab22974e481fd in_seq=0 out_seq=0 2023-12-07T10:26:12.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:11 smithi119 ceph-mon[97908]: pgmap v176: 129 pgs: 129 active+clean; 211 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 272 op/s 2023-12-07T10:26:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.245+0000 7ff4f6ff5700 1 -- 172.21.15.119:0/3431957897 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562d76769f70 con 0x562d768bf280 2023-12-07T10:26:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.245+0000 7ff4f6ff5700 10 client.27262 handle_command_reply: tid=0 2023-12-07T10:26:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.245+0000 7ff52cb3bb80 1 client.27262 shutdown 2023-12-07T10:26:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.245+0000 7ff52cb3bb80 1 -- 172.21.15.119:0/3431957897 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d76854a60 msgr2=0x562d7685d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.245+0000 7ff52cb3bb80 1 --2- 172.21.15.119:0/3431957897 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d76854a60 0x562d7685d9b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff508010470 tx=0x7ff508012020 comp rx=0 tx=0).stop 2023-12-07T10:26:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.245+0000 7ff52cb3bb80 1 -- 172.21.15.119:0/3431957897 shutdown_connections 2023-12-07T10:26:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.245+0000 7ff52cb3bb80 1 --2- 172.21.15.119:0/3431957897 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562d768bf280 0x562d768c16a0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.245+0000 7ff52cb3bb80 1 --2- 172.21.15.119:0/3431957897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562d76855380 0x562d7685def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.245+0000 7ff52cb3bb80 1 --2- 172.21.15.119:0/3431957897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d7684fad0 0x562d7685fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.245+0000 7ff52cb3bb80 1 --2- 172.21.15.119:0/3431957897 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d76854a60 0x562d7685d9b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.245+0000 7ff4f6ff5700 0 client.27262 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:26:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.245+0000 7ff52cb3bb80 1 -- 172.21.15.119:0/3431957897 >> 172.21.15.119:0/3431957897 conn(0x562d7677e300 msgr2=0x562d76851ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:26:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.245+0000 7ff52cb3bb80 1 -- 172.21.15.119:0/3431957897 shutdown_connections 2023-12-07T10:26:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.245+0000 7ff52cb3bb80 1 -- 172.21.15.119:0/3431957897 wait complete. 2023-12-07T10:26:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.245+0000 7ff52cb3bb80 20 client.27262 trim_cache size 0 max 16384 2023-12-07T10:26:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:26:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:26:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:26:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:26:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.246+0000 7ff5191ff700 1 -- 172.21.15.119:0/3108190549 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff4fc0a1d00 msgr2=0x7ff4fc0a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.246+0000 7ff5191ff700 1 --2- 172.21.15.119:0/3108190549 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff4fc0a1d00 0x7ff4fc0a41c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x562d7684b670 tx=0x7ff50c00b040 comp rx=0 tx=0).stop 2023-12-07T10:26:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.246+0000 7ff5191ff700 1 -- 172.21.15.119:0/3108190549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff514101520 msgr2=0x7ff51419d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.246+0000 7ff5191ff700 1 --2- 172.21.15.119:0/3108190549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff514101520 0x7ff51419d630 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff504005910 tx=0x7ff504004730 comp rx=0 tx=0).stop 2023-12-07T10:26:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.246+0000 7ff5191ff700 1 -- 172.21.15.119:0/3108190549 shutdown_connections 2023-12-07T10:26:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.246+0000 7ff5191ff700 1 --2- 172.21.15.119:0/3108190549 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff4fc0a1d00 0x7ff4fc0a41c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.246+0000 7ff5191ff700 1 --2- 172.21.15.119:0/3108190549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5141075e0 0x7ff5141977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.246+0000 7ff5191ff700 1 --2- 172.21.15.119:0/3108190549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff514101520 0x7ff51419d630 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.246+0000 7ff5191ff700 1 --2- 172.21.15.119:0/3108190549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff514101ee0 0x7ff51419db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.246+0000 7ff5191ff700 1 -- 172.21.15.119:0/3108190549 >> 172.21.15.119:0/3108190549 conn(0x7ff5140fd010 msgr2=0x7ff5140fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:26:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.246+0000 7ff5191ff700 1 -- 172.21.15.119:0/3108190549 shutdown_connections 2023-12-07T10:26:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:12.246+0000 7ff5191ff700 1 -- 172.21.15.119:0/3108190549 wait complete. 2023-12-07T10:26:12.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:11 smithi159 ceph-mon[134726]: pgmap v176: 129 pgs: 129 active+clean; 211 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 272 op/s 2023-12-07T10:26:12.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:11 smithi153 ceph-mon[133625]: pgmap v176: 129 pgs: 129 active+clean; 211 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 272 op/s 2023-12-07T10:26:12.558 INFO:tasks.workunit.client.0.smithi119.stdout: 1 373927 6.90 MB/sec execute 145 sec latency 14.399 ms 2023-12-07T10:26:13.558 INFO:tasks.workunit.client.0.smithi119.stdout: 1 375772 6.92 MB/sec execute 146 sec latency 19.934 ms 2023-12-07T10:26:14.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:14 smithi153 ceph-mon[133625]: pgmap v177: 129 pgs: 129 active+clean; 211 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2023-12-07T10:26:14.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:14 smithi119 ceph-mon[97908]: pgmap v177: 129 pgs: 129 active+clean; 211 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2023-12-07T10:26:14.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:14 smithi159 ceph-mon[134726]: pgmap v177: 129 pgs: 129 active+clean; 211 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2023-12-07T10:26:14.558 INFO:tasks.workunit.client.0.smithi119.stdout: 1 376697 6.91 MB/sec execute 147 sec latency 23.945 ms 2023-12-07T10:26:15.558 INFO:tasks.workunit.client.0.smithi119.stdout: 1 378516 6.91 MB/sec execute 148 sec latency 20.154 ms 2023-12-07T10:26:16.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:16 smithi153 ceph-mon[133625]: pgmap v178: 129 pgs: 129 active+clean; 211 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2023-12-07T10:26:16.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:16 smithi119 ceph-mon[97908]: pgmap v178: 129 pgs: 129 active+clean; 211 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2023-12-07T10:26:16.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:16 smithi159 ceph-mon[134726]: pgmap v178: 129 pgs: 129 active+clean; 211 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2023-12-07T10:26:16.559 INFO:tasks.workunit.client.0.smithi119.stdout: 1 379973 6.94 MB/sec execute 149 sec latency 14.997 ms 2023-12-07T10:26:17.559 INFO:tasks.workunit.client.0.smithi119.stdout: 1 381095 6.93 MB/sec execute 150 sec latency 19.567 ms 2023-12-07T10:26:18.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:18 smithi153 ceph-mon[133625]: pgmap v179: 129 pgs: 129 active+clean; 211 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 276 op/s 2023-12-07T10:26:18.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:18 smithi119 ceph-mon[97908]: pgmap v179: 129 pgs: 129 active+clean; 211 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 276 op/s 2023-12-07T10:26:18.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:18 smithi159 ceph-mon[134726]: pgmap v179: 129 pgs: 129 active+clean; 211 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 276 op/s 2023-12-07T10:26:18.559 INFO:tasks.workunit.client.0.smithi119.stdout: 1 383103 6.96 MB/sec execute 151 sec latency 15.182 ms 2023-12-07T10:26:19.559 INFO:tasks.workunit.client.0.smithi119.stdout: 1 384038 6.94 MB/sec execute 152 sec latency 15.627 ms 2023-12-07T10:26:20.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:20 smithi153 ceph-mon[133625]: pgmap v180: 129 pgs: 129 active+clean; 211 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 202 op/s 2023-12-07T10:26:20.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:20 smithi119 ceph-mon[97908]: pgmap v180: 129 pgs: 129 active+clean; 211 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 202 op/s 2023-12-07T10:26:20.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:20 smithi159 ceph-mon[134726]: pgmap v180: 129 pgs: 129 active+clean; 211 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 202 op/s 2023-12-07T10:26:20.559 INFO:tasks.workunit.client.0.smithi119.stdout: 1 385719 6.94 MB/sec execute 153 sec latency 16.809 ms 2023-12-07T10:26:21.559 INFO:tasks.workunit.client.0.smithi119.stdout: 1 387139 6.96 MB/sec execute 154 sec latency 27.334 ms 2023-12-07T10:26:22.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:22 smithi119 ceph-mon[97908]: pgmap v181: 129 pgs: 129 active+clean; 218 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 288 op/s 2023-12-07T10:26:22.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:22 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:26:22.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:22 smithi159 ceph-mon[134726]: pgmap v181: 129 pgs: 129 active+clean; 218 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 288 op/s 2023-12-07T10:26:22.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:22 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:26:22.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:22 smithi153 ceph-mon[133625]: pgmap v181: 129 pgs: 129 active+clean; 218 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 288 op/s 2023-12-07T10:26:22.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:22 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:26:22.559 INFO:tasks.workunit.client.0.smithi119.stdout: 1 388380 6.95 MB/sec execute 155 sec latency 14.427 ms 2023-12-07T10:26:23.561 INFO:tasks.workunit.client.0.smithi119.stdout: 1 390442 7.00 MB/sec execute 156 sec latency 17.323 ms 2023-12-07T10:26:24.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:24 smithi119 ceph-mon[97908]: pgmap v182: 129 pgs: 129 active+clean; 218 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 193 op/s 2023-12-07T10:26:24.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:24 smithi159 ceph-mon[134726]: pgmap v182: 129 pgs: 129 active+clean; 218 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 193 op/s 2023-12-07T10:26:24.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:24 smithi153 ceph-mon[133625]: pgmap v182: 129 pgs: 129 active+clean; 218 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 193 op/s 2023-12-07T10:26:24.559 INFO:tasks.workunit.client.0.smithi119.stdout: 1 391403 6.97 MB/sec execute 157 sec latency 25.741 ms 2023-12-07T10:26:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.440+0000 7ff8923ff700 1 -- 172.21.15.119:0/3733432844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff88c100410 msgr2=0x7ff88c100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.440+0000 7ff8923ff700 1 --2- 172.21.15.119:0/3733432844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff88c100410 0x7ff88c100890 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff88000ba80 tx=0x7ff88000bd90 comp rx=0 tx=0).stop 2023-12-07T10:26:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.441+0000 7ff8923ff700 1 -- 172.21.15.119:0/3733432844 shutdown_connections 2023-12-07T10:26:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.441+0000 7ff8923ff700 1 --2- 172.21.15.119:0/3733432844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff88c100dd0 0x7ff88c10d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.441+0000 7ff8923ff700 1 --2- 172.21.15.119:0/3733432844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff88c1064c0 0x7ff88c1068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.441+0000 7ff8923ff700 1 --2- 172.21.15.119:0/3733432844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff88c100410 0x7ff88c100890 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.441+0000 7ff8923ff700 1 -- 172.21.15.119:0/3733432844 >> 172.21.15.119:0/3733432844 conn(0x7ff88c0775b0 msgr2=0x7ff88c0779c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:26:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.441+0000 7ff8923ff700 1 -- 172.21.15.119:0/3733432844 shutdown_connections 2023-12-07T10:26:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.441+0000 7ff8923ff700 1 -- 172.21.15.119:0/3733432844 wait complete. 2023-12-07T10:26:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.442+0000 7ff8923ff700 1 Processor -- start 2023-12-07T10:26:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.442+0000 7ff8923ff700 1 -- start start 2023-12-07T10:26:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.442+0000 7ff8923ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff88c100410 0x7ff88c195770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.442+0000 7ff8923ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff88c100dd0 0x7ff88c195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.442+0000 7ff8923ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff88c1064c0 0x7ff88c198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.442+0000 7ff8923ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c1995a0 con 0x7ff88c1064c0 2023-12-07T10:26:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.442+0000 7ff8923ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c199710 con 0x7ff88c100dd0 2023-12-07T10:26:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.442+0000 7ff8923ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c1998d0 con 0x7ff88c100410 2023-12-07T10:26:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.443+0000 7ff88bfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff88c100410 0x7ff88c195770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.443+0000 7ff88bfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff88c100410 0x7ff88c195770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:49138/0 (socket says 172.21.15.119:49138) 2023-12-07T10:26:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.443+0000 7ff88bfff700 1 -- 172.21.15.119:0/4164607847 learned_addr learned my addr 172.21.15.119:0/4164607847 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:26:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.443+0000 7ff88b7fe700 1 --2- 172.21.15.119:0/4164607847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff88c100dd0 0x7ff88c195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.443+0000 7ff89099c700 1 --2- 172.21.15.119:0/4164607847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff88c1064c0 0x7ff88c198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.443+0000 7ff88bfff700 1 -- 172.21.15.119:0/4164607847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff88c100dd0 msgr2=0x7ff88c195cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.443+0000 7ff88bfff700 1 --2- 172.21.15.119:0/4164607847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff88c100dd0 0x7ff88c195cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.443+0000 7ff88bfff700 1 -- 172.21.15.119:0/4164607847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff88c1064c0 msgr2=0x7ff88c198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.443+0000 7ff88bfff700 1 --2- 172.21.15.119:0/4164607847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff88c1064c0 0x7ff88c198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.443+0000 7ff88bfff700 1 -- 172.21.15.119:0/4164607847 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff884009d00 con 0x7ff88c100410 2023-12-07T10:26:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.443+0000 7ff88bfff700 1 --2- 172.21.15.119:0/4164607847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff88c100410 0x7ff88c195770 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00e9a0 tx=0x7ff87c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:26:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.443+0000 7ff8897fa700 1 -- 172.21.15.119:0/4164607847 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c009790 con 0x7ff88c100410 2023-12-07T10:26:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.443+0000 7ff8923ff700 1 -- 172.21.15.119:0/4164607847 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff88000b730 con 0x7ff88c100410 2023-12-07T10:26:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.443+0000 7ff8897fa700 1 -- 172.21.15.119:0/4164607847 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff87c00cbd0 con 0x7ff88c100410 2023-12-07T10:26:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.443+0000 7ff8897fa700 1 -- 172.21.15.119:0/4164607847 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c018680 con 0x7ff88c100410 2023-12-07T10:26:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.444+0000 7ff8923ff700 1 -- 172.21.15.119:0/4164607847 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff88c1a6740 con 0x7ff88c100410 2023-12-07T10:26:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.447+0000 7ff8897fa700 1 -- 172.21.15.119:0/4164607847 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7ff87c00cd40 con 0x7ff88c100410 2023-12-07T10:26:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.448+0000 7ff8a5df1b80 1 Processor -- start 2023-12-07T10:26:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.448+0000 7ff8a5df1b80 1 -- start start 2023-12-07T10:26:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.448+0000 7ff8897fa700 1 --2- 172.21.15.119:0/4164607847 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff8740a1c80 0x7ff8740a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.449+0000 7ff8897fa700 1 -- 172.21.15.119:0/4164607847 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7ff87c014070 con 0x7ff88c100410 2023-12-07T10:26:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.449+0000 7ff8a5df1b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569a65f9a60 0x5569a65f9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.449+0000 7ff8a5df1b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569a65fa380 0x5569a65f4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.449+0000 7ff8a5df1b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5569a65f4ad0 0x5569a65f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.449+0000 7ff8a5df1b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569a6568760 con 0x5569a65f4ad0 2023-12-07T10:26:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.449+0000 7ff8a5df1b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569a654f740 con 0x5569a65fa380 2023-12-07T10:26:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.449+0000 7ff8a5df1b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569a654d060 con 0x5569a65f9a60 2023-12-07T10:26:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.449+0000 7ff89099c700 1 --2- 172.21.15.119:0/4164607847 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff8740a1c80 0x7ff8740a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.449+0000 7ff88bfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569a65f9a60 0x5569a65f9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.449+0000 7ff88bfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569a65f9a60 0x5569a65f9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:49144/0 (socket says 172.21.15.119:49144) 2023-12-07T10:26:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.449+0000 7ff88bfff700 1 -- 172.21.15.119:0/1609579660 learned_addr learned my addr 172.21.15.119:0/1609579660 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:26:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.449+0000 7ff88b7fe700 1 --2- 172.21.15.119:0/1609579660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569a65fa380 0x5569a65f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.449+0000 7ff89099c700 1 --2- 172.21.15.119:0/1609579660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5569a65f4ad0 0x5569a65f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.449+0000 7ff88bfff700 1 -- 172.21.15.119:0/1609579660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569a65fa380 msgr2=0x5569a65f4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.449+0000 7ff88bfff700 1 --2- 172.21.15.119:0/1609579660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569a65fa380 0x5569a65f4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.449+0000 7ff88bfff700 1 -- 172.21.15.119:0/1609579660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5569a65f4ad0 msgr2=0x5569a65f4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.449+0000 7ff88bfff700 1 --2- 172.21.15.119:0/1609579660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5569a65f4ad0 0x5569a65f4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.449+0000 7ff88bfff700 1 -- 172.21.15.119:0/1609579660 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569a63c55d0 con 0x5569a65f9a60 2023-12-07T10:26:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.449+0000 7ff89099c700 1 --2- 172.21.15.119:0/1609579660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5569a65f4ad0 0x5569a65f4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:26:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.449+0000 7ff88bfff700 1 --2- 172.21.15.119:0/1609579660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569a65f9a60 0x5569a65f9e40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff87c096ef0 tx=0x7ff87c09a350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f95c021429d1d6c server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:26:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.450+0000 7ff8727fc700 1 -- 172.21.15.119:0/1609579660 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c09eb40 con 0x5569a65f9a60 2023-12-07T10:26:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.450+0000 7ff89099c700 1 --2- 172.21.15.119:0/4164607847 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff8740a1c80 0x7ff8740a4140 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x5569a65f0670 tx=0x7ff8840099e0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:26:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.450+0000 7ff8727fc700 1 -- 172.21.15.119:0/1609579660 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff87c0a2d80 con 0x5569a65f9a60 2023-12-07T10:26:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.450+0000 7ff8727fc700 1 -- 172.21.15.119:0/1609579660 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c09a100 con 0x5569a65f9a60 2023-12-07T10:26:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.450+0000 7ff8a5df1b80 1 -- 172.21.15.119:0/1609579660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569a65f9a60 msgr2=0x5569a65f9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.450+0000 7ff8a5df1b80 1 --2- 172.21.15.119:0/1609579660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569a65f9a60 0x5569a65f9e40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff87c096ef0 tx=0x7ff87c09a350 comp rx=0 tx=0).stop 2023-12-07T10:26:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.450+0000 7ff8a5df1b80 1 -- 172.21.15.119:0/1609579660 shutdown_connections 2023-12-07T10:26:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.450+0000 7ff8a5df1b80 1 --2- 172.21.15.119:0/1609579660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5569a65f4ad0 0x5569a65f4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.450+0000 7ff8a5df1b80 1 --2- 172.21.15.119:0/1609579660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569a65f9a60 0x5569a65f9e40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.450+0000 7ff8a5df1b80 1 --2- 172.21.15.119:0/1609579660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569a65fa380 0x5569a65f4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.450+0000 7ff8a5df1b80 1 -- 172.21.15.119:0/1609579660 >> 172.21.15.119:0/1609579660 conn(0x5569a6523300 msgr2=0x5569a65eeb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:26:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.451+0000 7ff8a5df1b80 1 -- 172.21.15.119:0/1609579660 shutdown_connections 2023-12-07T10:26:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.451+0000 7ff8a5df1b80 1 -- 172.21.15.119:0/1609579660 wait complete. 2023-12-07T10:26:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.451+0000 7ff8a5df1b80 1 -- start start 2023-12-07T10:26:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.451+0000 7ff8a5df1b80 1 Processor -- start 2023-12-07T10:26:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.452+0000 7ff8a5df1b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5569a65f4ad0 0x5569a6600340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.452+0000 7ff8a5df1b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569a65f9a60 0x5569a65fd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.452+0000 7ff8a5df1b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569a65fa380 0x5569a65fdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.452+0000 7ff8a5df1b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569a6548950 con 0x5569a65f4ad0 2023-12-07T10:26:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.452+0000 7ff8a5df1b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569a6502500 con 0x5569a65fa380 2023-12-07T10:26:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.452+0000 7ff8a5df1b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569a65021a0 con 0x5569a65f9a60 2023-12-07T10:26:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.452+0000 7ff8a5df1b80 10 client.0 _ll_register_callbacks cb 0x5569a64e98e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:26:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.452+0000 7ff88b7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569a65f9a60 0x5569a65fd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.452+0000 7ff88bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5569a65f4ad0 0x5569a6600340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.452+0000 7ff88b7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569a65f9a60 0x5569a65fd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:49156/0 (socket says 172.21.15.119:49156) 2023-12-07T10:26:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.452+0000 7ff88b7fe700 1 -- 172.21.15.119:0/30918451 learned_addr learned my addr 172.21.15.119:0/30918451 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:26:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.452+0000 7ff89099c700 1 --2- 172.21.15.119:0/30918451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569a65fa380 0x5569a65fdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.453+0000 7ff88b7fe700 1 -- 172.21.15.119:0/30918451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569a65fa380 msgr2=0x5569a65fdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.453+0000 7ff88b7fe700 1 --2- 172.21.15.119:0/30918451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569a65fa380 0x5569a65fdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.453+0000 7ff88b7fe700 1 -- 172.21.15.119:0/30918451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5569a65f4ad0 msgr2=0x5569a6600340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.453+0000 7ff88b7fe700 1 --2- 172.21.15.119:0/30918451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5569a65f4ad0 0x5569a6600340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.453+0000 7ff88b7fe700 1 -- 172.21.15.119:0/30918451 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff88c1a6740 con 0x5569a65f9a60 2023-12-07T10:26:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.453+0000 7ff89099c700 1 --2- 172.21.15.119:0/30918451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569a65fa380 0x5569a65fdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:26:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.453+0000 7ff88b7fe700 1 --2- 172.21.15.119:0/30918451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569a65f9a60 0x5569a65fd6f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff88000ba80 tx=0x7ff88000da00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54ba8883f1c9665 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:26:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.453+0000 7ff86b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:26:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.453+0000 7ff86b7fe700 1 -- 172.21.15.119:0/30918451 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff880010040 con 0x5569a65f9a60 2023-12-07T10:26:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.453+0000 7ff86b7fe700 1 -- 172.21.15.119:0/30918451 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7ff880014910 con 0x5569a65f9a60 2023-12-07T10:26:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.453+0000 7ff8a5df1b80 1 -- 172.21.15.119:0/30918451 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5569a65e76f0 con 0x5569a65f9a60 2023-12-07T10:26:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.454+0000 7ff86b7fe700 1 -- 172.21.15.119:0/30918451 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff880014ad0 con 0x5569a65f9a60 2023-12-07T10:26:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.454+0000 7ff8a5df1b80 10 client.27274 fetch_fsmap learned FSMap version 48 2023-12-07T10:26:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.454+0000 7ff8a5df1b80 1 -- 172.21.15.119:0/30918451 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5569a6371d80 con 0x5569a65f9a60 2023-12-07T10:26:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.454+0000 7ff86b7fe700 1 -- 172.21.15.119:0/30918451 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5569a65e76f0 con 0x5569a65f9a60 2023-12-07T10:26:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.454+0000 7ff86b7fe700 1 -- 172.21.15.119:0/30918451 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 48) v1 ==== 7914+0+0 (secure 0 0 0) 0x7ff880003710 con 0x5569a65f9a60 2023-12-07T10:26:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.454+0000 7ff8a5df1b80 10 client.27274 fetch_fsmap finished waiting for FSMap version 48 2023-12-07T10:26:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.454+0000 7ff8a5df1b80 10 client.27274 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:26:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.454+0000 7ff8a5df1b80 20 client.27274 populate_metadata read hostname 'smithi119' 2023-12-07T10:26:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.454+0000 7ff8a5df1b80 1 --2- 172.21.15.119:0/30918451 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5569a660de40 0x5569a6610260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.454+0000 7ff8a5df1b80 4 client.27274 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:26:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.454+0000 7ff8a5df1b80 1 -- 172.21.15.119:0/30918451 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5569a65c3740 con 0x5569a660de40 2023-12-07T10:26:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.455+0000 7ff88bfff700 1 --2- 172.21.15.119:0/30918451 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5569a660de40 0x5569a6610260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.455+0000 7ff88bfff700 1 --2- 172.21.15.119:0/30918451 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5569a660de40 0x5569a6610260 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=226cecb8b09f7d71 server_cookie=2a0dd3fa3b78e2b3 in_seq=0 out_seq=0 2023-12-07T10:26:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.455+0000 7ff86b7fe700 10 client.27274 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:26:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.456+0000 7ff86b7fe700 1 -- 172.21.15.119:0/30918451 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5569a65c3740 con 0x5569a660de40 2023-12-07T10:26:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.456+0000 7ff86b7fe700 10 client.27274 handle_command_reply: tid=0 2023-12-07T10:26:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.456+0000 7ff8a5df1b80 1 client.27274 shutdown 2023-12-07T10:26:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.456+0000 7ff8a5df1b80 1 -- 172.21.15.119:0/30918451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569a65f9a60 msgr2=0x5569a65fd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.456+0000 7ff8a5df1b80 1 --2- 172.21.15.119:0/30918451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569a65f9a60 0x5569a65fd6f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff88000ba80 tx=0x7ff88000da00 comp rx=0 tx=0).stop 2023-12-07T10:26:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.457+0000 7ff8a5df1b80 1 -- 172.21.15.119:0/30918451 shutdown_connections 2023-12-07T10:26:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.457+0000 7ff8a5df1b80 1 --2- 172.21.15.119:0/30918451 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5569a660de40 0x5569a6610260 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.457+0000 7ff8a5df1b80 1 --2- 172.21.15.119:0/30918451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569a65fa380 0x5569a65fdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.457+0000 7ff8a5df1b80 1 --2- 172.21.15.119:0/30918451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5569a65f4ad0 0x5569a6600340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.457+0000 7ff8a5df1b80 1 --2- 172.21.15.119:0/30918451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569a65f9a60 0x5569a65fd6f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.457+0000 7ff86b7fe700 0 client.27274 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:26:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.457+0000 7ff8a5df1b80 1 -- 172.21.15.119:0/30918451 >> 172.21.15.119:0/30918451 conn(0x5569a6523300 msgr2=0x5569a65e9980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:26:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.457+0000 7ff8a5df1b80 1 -- 172.21.15.119:0/30918451 shutdown_connections 2023-12-07T10:26:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.457+0000 7ff8a5df1b80 1 -- 172.21.15.119:0/30918451 wait complete. 2023-12-07T10:26:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.457+0000 7ff8a5df1b80 20 client.27274 trim_cache size 0 max 16384 2023-12-07T10:26:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.475+0000 7ff8a5df1b80 1 Processor -- start 2023-12-07T10:26:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.475+0000 7ff8a5df1b80 1 -- start start 2023-12-07T10:26:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.476+0000 7ff8a5df1b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569a65f4ad0 0x5569a6604da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.476+0000 7ff8a5df1b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5569a65f9a60 0x5569a66052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.476+0000 7ff8a5df1b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569a65fa380 0x5569a65f7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.476+0000 7ff8a5df1b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569a63c78c0 con 0x5569a65f9a60 2023-12-07T10:26:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.476+0000 7ff8a5df1b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569a660a900 con 0x5569a65f4ad0 2023-12-07T10:26:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.476+0000 7ff8a5df1b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569a65eb7d0 con 0x5569a65fa380 2023-12-07T10:26:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.476+0000 7ff88b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5569a65f9a60 0x5569a66052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.476+0000 7ff89099c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569a65fa380 0x5569a65f7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.476+0000 7ff88bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569a65f4ad0 0x5569a6604da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.476+0000 7ff88b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5569a65f9a60 0x5569a66052e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47222/0 (socket says 172.21.15.119:47222) 2023-12-07T10:26:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.476+0000 7ff88b7fe700 1 -- 172.21.15.119:0/1328037612 learned_addr learned my addr 172.21.15.119:0/1328037612 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:26:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.476+0000 7ff88bfff700 1 -- 172.21.15.119:0/1328037612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569a65fa380 msgr2=0x5569a65f7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.476+0000 7ff88bfff700 1 --2- 172.21.15.119:0/1328037612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569a65fa380 0x5569a65f7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.476+0000 7ff88bfff700 1 -- 172.21.15.119:0/1328037612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5569a65f9a60 msgr2=0x5569a66052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.476+0000 7ff88bfff700 1 --2- 172.21.15.119:0/1328037612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5569a65f9a60 0x5569a66052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.476+0000 7ff88bfff700 1 -- 172.21.15.119:0/1328037612 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569a635ac00 con 0x5569a65f4ad0 2023-12-07T10:26:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.477+0000 7ff89099c700 1 --2- 172.21.15.119:0/1328037612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569a65fa380 0x5569a65f7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:26:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.477+0000 7ff88bfff700 1 --2- 172.21.15.119:0/1328037612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569a65f4ad0 0x5569a6604da0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff87c022000 tx=0x7ff87c085040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49193a6d0a327f7 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:26:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.477+0000 7ff872ffd700 1 -- 172.21.15.119:0/1328037612 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c0850b0 con 0x5569a65f4ad0 2023-12-07T10:26:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.477+0000 7ff872ffd700 1 -- 172.21.15.119:0/1328037612 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff87c085210 con 0x5569a65f4ad0 2023-12-07T10:26:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.477+0000 7ff872ffd700 1 -- 172.21.15.119:0/1328037612 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c09aeb0 con 0x5569a65f4ad0 2023-12-07T10:26:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.478+0000 7ff8a5df1b80 1 -- 172.21.15.119:0/1328037612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569a65f4ad0 msgr2=0x5569a6604da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.478+0000 7ff8a5df1b80 1 --2- 172.21.15.119:0/1328037612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569a65f4ad0 0x5569a6604da0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff87c022000 tx=0x7ff87c085040 comp rx=0 tx=0).stop 2023-12-07T10:26:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.478+0000 7ff8a5df1b80 1 -- 172.21.15.119:0/1328037612 shutdown_connections 2023-12-07T10:26:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.478+0000 7ff8a5df1b80 1 --2- 172.21.15.119:0/1328037612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569a65fa380 0x5569a65f7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.478+0000 7ff8a5df1b80 1 --2- 172.21.15.119:0/1328037612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569a65f4ad0 0x5569a6604da0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.478+0000 7ff8a5df1b80 1 --2- 172.21.15.119:0/1328037612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5569a65f9a60 0x5569a66052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.478+0000 7ff8a5df1b80 1 -- 172.21.15.119:0/1328037612 >> 172.21.15.119:0/1328037612 conn(0x5569a6523300 msgr2=0x5569a6600070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:26:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.478+0000 7ff8a5df1b80 1 -- 172.21.15.119:0/1328037612 shutdown_connections 2023-12-07T10:26:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.478+0000 7ff8a5df1b80 1 -- 172.21.15.119:0/1328037612 wait complete. 2023-12-07T10:26:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.478+0000 7ff8a5df1b80 1 -- start start 2023-12-07T10:26:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.478+0000 7ff8a5df1b80 1 Processor -- start 2023-12-07T10:26:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.479+0000 7ff8a5df1b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569a65f4ad0 0x5569a6605280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.479+0000 7ff8a5df1b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5569a65f9a60 0x5569a6602d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.479+0000 7ff8a5df1b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569a65fa380 0x5569a66034c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.479+0000 7ff8a5df1b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569a6603ba0 con 0x5569a65f9a60 2023-12-07T10:26:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.479+0000 7ff8a5df1b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569a666bb10 con 0x5569a65f4ad0 2023-12-07T10:26:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.479+0000 7ff8a5df1b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569a666bc50 con 0x5569a65fa380 2023-12-07T10:26:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.479+0000 7ff8a5df1b80 10 client.0 _ll_register_callbacks cb 0x5569a64e98e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:26:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.479+0000 7ff88bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569a65f4ad0 0x5569a6605280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.479+0000 7ff88b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5569a65f9a60 0x5569a6602d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.479+0000 7ff89099c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569a65fa380 0x5569a66034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.479+0000 7ff88bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569a65f4ad0 0x5569a6605280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:35058/0 (socket says 172.21.15.119:35058) 2023-12-07T10:26:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.479+0000 7ff88bfff700 1 -- 172.21.15.119:0/1361094005 learned_addr learned my addr 172.21.15.119:0/1361094005 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:26:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.480+0000 7ff88bfff700 1 -- 172.21.15.119:0/1361094005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569a65fa380 msgr2=0x5569a66034c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.480+0000 7ff88bfff700 1 --2- 172.21.15.119:0/1361094005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569a65fa380 0x5569a66034c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.480+0000 7ff88bfff700 1 -- 172.21.15.119:0/1361094005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5569a65f9a60 msgr2=0x5569a6602d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.480+0000 7ff88bfff700 1 --2- 172.21.15.119:0/1361094005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5569a65f9a60 0x5569a6602d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.480+0000 7ff88bfff700 1 -- 172.21.15.119:0/1361094005 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569a63c55d0 con 0x5569a65f4ad0 2023-12-07T10:26:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.480+0000 7ff88bfff700 1 --2- 172.21.15.119:0/1361094005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569a65f4ad0 0x5569a6605280 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0cb9a0 tx=0x7ff87c0cb480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=461a7663ce3dba1 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:26:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.480+0000 7ff86b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:26:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.480+0000 7ff86b7fe700 1 -- 172.21.15.119:0/1361094005 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c0850b0 con 0x5569a65f4ad0 2023-12-07T10:26:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.480+0000 7ff8a5df1b80 1 -- 172.21.15.119:0/1361094005 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff87c0a2d80 con 0x5569a65f4ad0 2023-12-07T10:26:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.480+0000 7ff86b7fe700 1 -- 172.21.15.119:0/1361094005 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7ff87c0b3260 con 0x5569a65f4ad0 2023-12-07T10:26:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.480+0000 7ff86b7fe700 1 -- 172.21.15.119:0/1361094005 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff87c09ae50 con 0x5569a65f4ad0 2023-12-07T10:26:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.480+0000 7ff8a5df1b80 10 client.27302 fetch_fsmap learned FSMap version 48 2023-12-07T10:26:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.480+0000 7ff8a5df1b80 1 -- 172.21.15.119:0/1361094005 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x5569a63981f0 con 0x5569a65f4ad0 2023-12-07T10:26:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.480+0000 7ff86b7fe700 1 -- 172.21.15.119:0/1361094005 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff87c0a2d80 con 0x5569a65f4ad0 2023-12-07T10:26:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.481+0000 7ff86b7fe700 1 -- 172.21.15.119:0/1361094005 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 48) v1 ==== 7914+0+0 (secure 0 0 0) 0x7ff87c09bd20 con 0x5569a65f4ad0 2023-12-07T10:26:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.481+0000 7ff8a5df1b80 10 client.27302 fetch_fsmap finished waiting for FSMap version 48 2023-12-07T10:26:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.481+0000 7ff8a5df1b80 10 client.27302 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:26:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.481+0000 7ff8a5df1b80 20 client.27302 populate_metadata read hostname 'smithi119' 2023-12-07T10:26:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.481+0000 7ff8a5df1b80 1 --2- 172.21.15.119:0/1361094005 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5569a666ce80 0x5569a666f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.481+0000 7ff8a5df1b80 4 client.27302 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:26:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.481+0000 7ff8a5df1b80 1 -- 172.21.15.119:0/1361094005 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5569a650ef70 con 0x5569a666ce80 2023-12-07T10:26:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.481+0000 7ff88b7fe700 1 --2- 172.21.15.119:0/1361094005 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5569a666ce80 0x5569a666f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.481+0000 7ff88b7fe700 1 --2- 172.21.15.119:0/1361094005 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5569a666ce80 0x5569a666f270 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ee8900af0efa589 server_cookie=b8590b48da866863 in_seq=0 out_seq=0 2023-12-07T10:26:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.481+0000 7ff86b7fe700 10 client.27302 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:26:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.519+0000 7ff86b7fe700 1 -- 172.21.15.119:0/1361094005 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5569a650ef70 con 0x5569a666ce80 2023-12-07T10:26:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.519+0000 7ff86b7fe700 10 client.27302 handle_command_reply: tid=0 2023-12-07T10:26:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.519+0000 7ff8a5df1b80 1 client.27302 shutdown 2023-12-07T10:26:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.519+0000 7ff8a5df1b80 1 -- 172.21.15.119:0/1361094005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569a65f4ad0 msgr2=0x5569a6605280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.519+0000 7ff8a5df1b80 1 --2- 172.21.15.119:0/1361094005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569a65f4ad0 0x5569a6605280 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0cb9a0 tx=0x7ff87c0cb480 comp rx=0 tx=0).stop 2023-12-07T10:26:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.519+0000 7ff8a5df1b80 1 -- 172.21.15.119:0/1361094005 shutdown_connections 2023-12-07T10:26:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.519+0000 7ff8a5df1b80 1 --2- 172.21.15.119:0/1361094005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569a65fa380 0x5569a66034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.519+0000 7ff8a5df1b80 1 --2- 172.21.15.119:0/1361094005 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5569a666ce80 0x5569a666f270 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.520+0000 7ff8a5df1b80 1 --2- 172.21.15.119:0/1361094005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5569a65f4ad0 0x5569a6605280 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.520+0000 7ff8a5df1b80 1 --2- 172.21.15.119:0/1361094005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5569a65f9a60 0x5569a6602d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.520+0000 7ff86b7fe700 0 client.27302 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:26:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.520+0000 7ff8a5df1b80 1 -- 172.21.15.119:0/1361094005 >> 172.21.15.119:0/1361094005 conn(0x5569a6523300 msgr2=0x5569a65f7960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:26:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.520+0000 7ff8a5df1b80 1 -- 172.21.15.119:0/1361094005 shutdown_connections 2023-12-07T10:26:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.520+0000 7ff8a5df1b80 1 -- 172.21.15.119:0/1361094005 wait complete. 2023-12-07T10:26:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.520+0000 7ff8a5df1b80 20 client.27302 trim_cache size 0 max 16384 2023-12-07T10:26:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:26:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:26:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:26:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:26:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.521+0000 7ff8923ff700 1 -- 172.21.15.119:0/4164607847 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff8740a1c80 msgr2=0x7ff8740a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.521+0000 7ff8923ff700 1 --2- 172.21.15.119:0/4164607847 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff8740a1c80 0x7ff8740a4140 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x5569a65f0670 tx=0x7ff8840099e0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.521+0000 7ff8923ff700 1 -- 172.21.15.119:0/4164607847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff88c100410 msgr2=0x7ff88c195770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.521+0000 7ff8923ff700 1 --2- 172.21.15.119:0/4164607847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff88c100410 0x7ff88c195770 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00e9a0 tx=0x7ff87c00ecb0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.521+0000 7ff8923ff700 1 -- 172.21.15.119:0/4164607847 shutdown_connections 2023-12-07T10:26:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.521+0000 7ff8923ff700 1 --2- 172.21.15.119:0/4164607847 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff8740a1c80 0x7ff8740a4140 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.521+0000 7ff8923ff700 1 --2- 172.21.15.119:0/4164607847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff88c1064c0 0x7ff88c198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.521+0000 7ff8923ff700 1 --2- 172.21.15.119:0/4164607847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff88c100410 0x7ff88c195770 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.521+0000 7ff8923ff700 1 --2- 172.21.15.119:0/4164607847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff88c100dd0 0x7ff88c195cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.521+0000 7ff8923ff700 1 -- 172.21.15.119:0/4164607847 >> 172.21.15.119:0/4164607847 conn(0x7ff88c0775b0 msgr2=0x7ff88c0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:26:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.521+0000 7ff8923ff700 1 -- 172.21.15.119:0/4164607847 shutdown_connections 2023-12-07T10:26:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:25.521+0000 7ff8923ff700 1 -- 172.21.15.119:0/4164607847 wait complete. 2023-12-07T10:26:25.559 INFO:tasks.workunit.client.0.smithi119.stdout: 1 392978 6.98 MB/sec execute 158 sec latency 59.611 ms 2023-12-07T10:26:26.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:26 smithi119 ceph-mon[97908]: pgmap v183: 129 pgs: 129 active+clean; 218 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 193 op/s 2023-12-07T10:26:26.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:26 smithi159 ceph-mon[134726]: pgmap v183: 129 pgs: 129 active+clean; 218 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 193 op/s 2023-12-07T10:26:26.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:26 smithi153 ceph-mon[133625]: pgmap v183: 129 pgs: 129 active+clean; 218 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 193 op/s 2023-12-07T10:26:26.559 INFO:tasks.workunit.client.0.smithi119.stdout: 1 394235 6.99 MB/sec execute 159 sec latency 33.285 ms 2023-12-07T10:26:27.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:27 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:26:27.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:27 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:26:27.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:27 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:26:27.559 INFO:tasks.workunit.client.0.smithi119.stdout: 1 395967 6.98 MB/sec execute 160 sec latency 15.828 ms 2023-12-07T10:26:28.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:28 smithi119 ceph-mon[97908]: pgmap v184: 129 pgs: 129 active+clean; 227 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 296 op/s 2023-12-07T10:26:28.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:28 smithi159 ceph-mon[134726]: pgmap v184: 129 pgs: 129 active+clean; 227 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 296 op/s 2023-12-07T10:26:28.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:28 smithi153 ceph-mon[133625]: pgmap v184: 129 pgs: 129 active+clean; 227 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 296 op/s 2023-12-07T10:26:28.559 INFO:tasks.workunit.client.0.smithi119.stdout: 1 397631 7.02 MB/sec execute 161 sec latency 13.811 ms 2023-12-07T10:26:29.559 INFO:tasks.workunit.client.0.smithi119.stdout: 1 398777 7.01 MB/sec execute 162 sec latency 14.116 ms 2023-12-07T10:26:30.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:30 smithi119 ceph-mon[97908]: pgmap v185: 129 pgs: 129 active+clean; 227 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 188 op/s 2023-12-07T10:26:30.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:30 smithi159 ceph-mon[134726]: pgmap v185: 129 pgs: 129 active+clean; 227 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 188 op/s 2023-12-07T10:26:30.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:30 smithi153 ceph-mon[133625]: pgmap v185: 129 pgs: 129 active+clean; 227 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 188 op/s 2023-12-07T10:26:30.559 INFO:tasks.workunit.client.0.smithi119.stdout: 1 400482 7.01 MB/sec execute 163 sec latency 52.254 ms 2023-12-07T10:26:31.559 INFO:tasks.workunit.client.0.smithi119.stdout: 1 401464 7.01 MB/sec execute 164 sec latency 22.561 ms 2023-12-07T10:26:32.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:31 smithi119 ceph-mon[97908]: pgmap v186: 129 pgs: 129 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 289 op/s 2023-12-07T10:26:32.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:31 smithi159 ceph-mon[134726]: pgmap v186: 129 pgs: 129 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 289 op/s 2023-12-07T10:26:32.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:31 smithi153 ceph-mon[133625]: pgmap v186: 129 pgs: 129 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 289 op/s 2023-12-07T10:26:32.559 INFO:tasks.workunit.client.0.smithi119.stdout: 1 403267 7.01 MB/sec execute 165 sec latency 18.070 ms 2023-12-07T10:26:33.559 INFO:tasks.workunit.client.0.smithi119.stdout: 1 404770 7.04 MB/sec execute 166 sec latency 13.419 ms 2023-12-07T10:26:34.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:34 smithi153 ceph-mon[133625]: pgmap v187: 129 pgs: 129 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 203 op/s 2023-12-07T10:26:34.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:34 smithi119 ceph-mon[97908]: pgmap v187: 129 pgs: 129 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 203 op/s 2023-12-07T10:26:34.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:34 smithi159 ceph-mon[134726]: pgmap v187: 129 pgs: 129 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 203 op/s 2023-12-07T10:26:34.559 INFO:tasks.workunit.client.0.smithi119.stdout: 1 405886 7.02 MB/sec execute 167 sec latency 16.540 ms 2023-12-07T10:26:35.561 INFO:tasks.workunit.client.0.smithi119.stdout: 1 407665 7.04 MB/sec execute 168 sec latency 17.392 ms 2023-12-07T10:26:36.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:36 smithi153 ceph-mon[133625]: pgmap v188: 129 pgs: 129 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 203 op/s 2023-12-07T10:26:36.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:36 smithi119 ceph-mon[97908]: pgmap v188: 129 pgs: 129 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 203 op/s 2023-12-07T10:26:36.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:36 smithi159 ceph-mon[134726]: pgmap v188: 129 pgs: 129 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 203 op/s 2023-12-07T10:26:36.560 INFO:tasks.workunit.client.0.smithi119.stdout: 1 408609 7.03 MB/sec execute 169 sec latency 17.923 ms 2023-12-07T10:26:37.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:37 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:26:37.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:37 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:26:37.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:37 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:26:37.560 INFO:tasks.workunit.client.0.smithi119.stdout: 1 410350 7.02 MB/sec execute 170 sec latency 22.502 ms 2023-12-07T10:26:38.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:38 smithi119 ceph-mon[97908]: pgmap v189: 129 pgs: 129 active+clean; 235 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 296 op/s 2023-12-07T10:26:38.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:38 smithi159 ceph-mon[134726]: pgmap v189: 129 pgs: 129 active+clean; 235 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 296 op/s 2023-12-07T10:26:38.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:38 smithi153 ceph-mon[133625]: pgmap v189: 129 pgs: 129 active+clean; 235 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 296 op/s 2023-12-07T10:26:38.560 INFO:tasks.workunit.client.0.smithi119.stdout: 1 411854 7.05 MB/sec execute 171 sec latency 12.169 ms 2023-12-07T10:26:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.688+0000 7fb297fff700 1 -- 172.21.15.119:0/3742729252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2981073f0 msgr2=0x7fb2981077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.688+0000 7fb297fff700 1 --2- 172.21.15.119:0/3742729252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2981073f0 0x7fb2981077d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb2880099c0 tx=0x7fb288009cd0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.688+0000 7fb297fff700 1 -- 172.21.15.119:0/3742729252 shutdown_connections 2023-12-07T10:26:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.688+0000 7fb297fff700 1 --2- 172.21.15.119:0/3742729252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2980692a0 0x7fb29810e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.688+0000 7fb297fff700 1 --2- 172.21.15.119:0/3742729252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2981073f0 0x7fb2981077d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.688+0000 7fb297fff700 1 --2- 172.21.15.119:0/3742729252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2980688e0 0x7fb298068d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.688+0000 7fb297fff700 1 -- 172.21.15.119:0/3742729252 >> 172.21.15.119:0/3742729252 conn(0x7fb298075e70 msgr2=0x7fb298076280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:26:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.688+0000 7fb297fff700 1 -- 172.21.15.119:0/3742729252 shutdown_connections 2023-12-07T10:26:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.688+0000 7fb297fff700 1 -- 172.21.15.119:0/3742729252 wait complete. 2023-12-07T10:26:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.689+0000 7fb297fff700 1 Processor -- start 2023-12-07T10:26:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.689+0000 7fb297fff700 1 -- start start 2023-12-07T10:26:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.689+0000 7fb297fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2980688e0 0x7fb29819d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.689+0000 7fb297fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2980692a0 0x7fb29819da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.689+0000 7fb297fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2981073f0 0x7fb298197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.689+0000 7fb297fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb298197cb0 con 0x7fb2981073f0 2023-12-07T10:26:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.689+0000 7fb297fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb298197e20 con 0x7fb2980692a0 2023-12-07T10:26:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.689+0000 7fb297fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb298197fe0 con 0x7fb2980688e0 2023-12-07T10:26:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.690+0000 7fb2977fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2981073f0 0x7fb298197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.690+0000 7fb2967fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2980692a0 0x7fb29819da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.690+0000 7fb296ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2980688e0 0x7fb29819d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.690+0000 7fb2977fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2981073f0 0x7fb298197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54220/0 (socket says 172.21.15.119:54220) 2023-12-07T10:26:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.690+0000 7fb2967fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2980692a0 0x7fb29819da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:55574/0 (socket says 172.21.15.119:55574) 2023-12-07T10:26:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.690+0000 7fb2977fe700 1 -- 172.21.15.119:0/2698421772 learned_addr learned my addr 172.21.15.119:0/2698421772 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:26:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.690+0000 7fb296ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2980688e0 0x7fb29819d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:59518/0 (socket says 172.21.15.119:59518) 2023-12-07T10:26:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.690+0000 7fb296ffd700 1 -- 172.21.15.119:0/2698421772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2980692a0 msgr2=0x7fb29819da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.690+0000 7fb296ffd700 1 --2- 172.21.15.119:0/2698421772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2980692a0 0x7fb29819da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.690+0000 7fb296ffd700 1 -- 172.21.15.119:0/2698421772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2981073f0 msgr2=0x7fb298197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.690+0000 7fb296ffd700 1 --2- 172.21.15.119:0/2698421772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2981073f0 0x7fb298197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.690+0000 7fb296ffd700 1 -- 172.21.15.119:0/2698421772 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2880095f0 con 0x7fb2980688e0 2023-12-07T10:26:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.690+0000 7fb296ffd700 1 --2- 172.21.15.119:0/2698421772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2980688e0 0x7fb29819d520 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb2880098e0 tx=0x7fb28800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:26:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.691+0000 7fb29c9fd700 1 -- 172.21.15.119:0/2698421772 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28801c070 con 0x7fb2980688e0 2023-12-07T10:26:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.691+0000 7fb29c9fd700 1 -- 172.21.15.119:0/2698421772 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb28800fd00 con 0x7fb2980688e0 2023-12-07T10:26:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.691+0000 7fb29c9fd700 1 -- 172.21.15.119:0/2698421772 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb288022c60 con 0x7fb2980688e0 2023-12-07T10:26:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.691+0000 7fb297fff700 1 -- 172.21.15.119:0/2698421772 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb298198260 con 0x7fb2980688e0 2023-12-07T10:26:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.691+0000 7fb297fff700 1 -- 172.21.15.119:0/2698421772 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2981a4510 con 0x7fb2980688e0 2023-12-07T10:26:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.694+0000 7fb29c9fd700 1 -- 172.21.15.119:0/2698421772 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fb288020070 con 0x7fb2980688e0 2023-12-07T10:26:38.696 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.696+0000 7fb2b1312b80 1 Processor -- start 2023-12-07T10:26:38.696 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.696+0000 7fb2b1312b80 1 -- start start 2023-12-07T10:26:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.696+0000 7fb29c9fd700 1 --2- 172.21.15.119:0/2698421772 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb2800a1d00 0x7fb2800a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.696+0000 7fb2b1312b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645a99efca0 0x5645a99f0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.696+0000 7fb2b1312b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645a99f05c0 0x5645a99ea540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.696+0000 7fb2b1312b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645a99eaa80 0x5645a99eaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.696+0000 7fb2b1312b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645a995e760 con 0x5645a99efca0 2023-12-07T10:26:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.696+0000 7fb2b1312b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645a9945740 con 0x5645a99f05c0 2023-12-07T10:26:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.696+0000 7fb2b1312b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645a9943060 con 0x5645a99eaa80 2023-12-07T10:26:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.696+0000 7fb29c9fd700 1 -- 172.21.15.119:0/2698421772 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fb2880ce850 con 0x7fb2980688e0 2023-12-07T10:26:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.697+0000 7fb2967fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645a99f05c0 0x5645a99ea540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.697+0000 7fb296ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645a99efca0 0x5645a99f0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.697+0000 7fb2967fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645a99f05c0 0x5645a99ea540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:55590/0 (socket says 172.21.15.119:55590) 2023-12-07T10:26:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.697+0000 7fb2967fc700 1 -- 172.21.15.119:0/1417943579 learned_addr learned my addr 172.21.15.119:0/1417943579 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:26:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.697+0000 7fb2977fe700 1 --2- 172.21.15.119:0/1417943579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645a99eaa80 0x5645a99eaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.697+0000 7fb2977fe700 1 --2- 172.21.15.119:0/2698421772 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb2800a1d00 0x7fb2800a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.697+0000 7fb296ffd700 1 -- 172.21.15.119:0/1417943579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645a99eaa80 msgr2=0x5645a99eaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.697+0000 7fb296ffd700 1 --2- 172.21.15.119:0/1417943579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645a99eaa80 0x5645a99eaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.697+0000 7fb296ffd700 1 -- 172.21.15.119:0/1417943579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645a99f05c0 msgr2=0x5645a99ea540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.697+0000 7fb296ffd700 1 --2- 172.21.15.119:0/1417943579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645a99f05c0 0x5645a99ea540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.697+0000 7fb296ffd700 1 -- 172.21.15.119:0/1417943579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645a97bb5d0 con 0x5645a99efca0 2023-12-07T10:26:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.697+0000 7fb2967fc700 1 --2- 172.21.15.119:0/1417943579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645a99f05c0 0x5645a99ea540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:26:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.697+0000 7fb296ffd700 1 --2- 172.21.15.119:0/1417943579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645a99efca0 0x5645a99f0080 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb2880cbbf0 tx=0x7fb28800ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42389a501b54d5a6 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:26:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.697+0000 7fb2977fe700 1 --2- 172.21.15.119:0/2698421772 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb2800a1d00 0x7fb2800a41c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb290009d30 tx=0x7fb29000b040 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:26:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.698+0000 7fb27dffb700 1 -- 172.21.15.119:0/1417943579 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28809ae10 con 0x5645a99efca0 2023-12-07T10:26:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.698+0000 7fb27dffb700 1 -- 172.21.15.119:0/1417943579 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb28809ac30 con 0x5645a99efca0 2023-12-07T10:26:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.698+0000 7fb27dffb700 1 -- 172.21.15.119:0/1417943579 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb288099700 con 0x5645a99efca0 2023-12-07T10:26:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.698+0000 7fb2b1312b80 1 -- 172.21.15.119:0/1417943579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645a99efca0 msgr2=0x5645a99f0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.698+0000 7fb2b1312b80 1 --2- 172.21.15.119:0/1417943579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645a99efca0 0x5645a99f0080 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb2880cbbf0 tx=0x7fb28800ff20 comp rx=0 tx=0).stop 2023-12-07T10:26:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.698+0000 7fb2b1312b80 1 -- 172.21.15.119:0/1417943579 shutdown_connections 2023-12-07T10:26:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.698+0000 7fb2b1312b80 1 --2- 172.21.15.119:0/1417943579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645a99eaa80 0x5645a99eaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.698+0000 7fb2b1312b80 1 --2- 172.21.15.119:0/1417943579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645a99efca0 0x5645a99f0080 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.698+0000 7fb2b1312b80 1 --2- 172.21.15.119:0/1417943579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645a99f05c0 0x5645a99ea540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.698+0000 7fb2b1312b80 1 -- 172.21.15.119:0/1417943579 >> 172.21.15.119:0/1417943579 conn(0x5645a9919300 msgr2=0x5645a99e4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:26:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.699+0000 7fb2b1312b80 1 -- 172.21.15.119:0/1417943579 shutdown_connections 2023-12-07T10:26:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.699+0000 7fb2b1312b80 1 -- 172.21.15.119:0/1417943579 wait complete. 2023-12-07T10:26:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.699+0000 7fb2b1312b80 1 -- start start 2023-12-07T10:26:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.700+0000 7fb2b1312b80 1 Processor -- start 2023-12-07T10:26:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.700+0000 7fb2b1312b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645a99eaa80 0x5645a99f65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.700+0000 7fb2b1312b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645a99efca0 0x5645a99f3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.700+0000 7fb2b1312b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645a99f05c0 0x5645a99f4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.700+0000 7fb2b1312b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645a993e950 con 0x5645a99f05c0 2023-12-07T10:26:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.700+0000 7fb2b1312b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645a98f8500 con 0x5645a99efca0 2023-12-07T10:26:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.700+0000 7fb2b1312b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645a98f81a0 con 0x5645a99eaa80 2023-12-07T10:26:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.700+0000 7fb2b1312b80 10 client.0 _ll_register_callbacks cb 0x5645a98df8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:26:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.701+0000 7fb296ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645a99eaa80 0x5645a99f65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.701+0000 7fb296ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645a99eaa80 0x5645a99f65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:59542/0 (socket says 172.21.15.119:59542) 2023-12-07T10:26:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.701+0000 7fb296ffd700 1 -- 172.21.15.119:0/1187590697 learned_addr learned my addr 172.21.15.119:0/1187590697 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:26:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.701+0000 7fb2977fe700 1 --2- 172.21.15.119:0/1187590697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645a99f05c0 0x5645a99f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.701+0000 7fb296ffd700 1 -- 172.21.15.119:0/1187590697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645a99efca0 msgr2=0x5645a99f3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.701+0000 7fb296ffd700 1 --2- 172.21.15.119:0/1187590697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645a99efca0 0x5645a99f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.701+0000 7fb296ffd700 1 -- 172.21.15.119:0/1187590697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645a99f05c0 msgr2=0x5645a99f4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.701+0000 7fb296ffd700 1 --2- 172.21.15.119:0/1187590697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645a99f05c0 0x5645a99f4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.704 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.701+0000 7fb296ffd700 1 -- 172.21.15.119:0/1187590697 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb28c009980 con 0x5645a99eaa80 2023-12-07T10:26:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.701+0000 7fb296ffd700 1 --2- 172.21.15.119:0/1187590697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645a99eaa80 0x5645a99f65c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb2880bc010 tx=0x7fb28809af40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=140e51a89fed2a6e server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:26:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.701+0000 7fb276ffd700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:26:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.701+0000 7fb276ffd700 1 -- 172.21.15.119:0/1187590697 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28809f220 con 0x5645a99eaa80 2023-12-07T10:26:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.702+0000 7fb276ffd700 1 -- 172.21.15.119:0/1187590697 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fb2880c3070 con 0x5645a99eaa80 2023-12-07T10:26:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.702+0000 7fb2b1312b80 1 -- 172.21.15.119:0/1187590697 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645a99dd6f0 con 0x5645a99eaa80 2023-12-07T10:26:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.702+0000 7fb276ffd700 1 -- 172.21.15.119:0/1187590697 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2880ce640 con 0x5645a99eaa80 2023-12-07T10:26:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.702+0000 7fb2b1312b80 10 client.27292 fetch_fsmap learned FSMap version 50 2023-12-07T10:26:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.702+0000 7fb2b1312b80 1 -- 172.21.15.119:0/1187590697 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7fb2981a4510 con 0x5645a99eaa80 2023-12-07T10:26:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.702+0000 7fb276ffd700 1 -- 172.21.15.119:0/1187590697 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5645a99dd6f0 con 0x5645a99eaa80 2023-12-07T10:26:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.702+0000 7fb276ffd700 1 -- 172.21.15.119:0/1187590697 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 50) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fb28809b530 con 0x5645a99eaa80 2023-12-07T10:26:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.702+0000 7fb2b1312b80 10 client.27292 fetch_fsmap finished waiting for FSMap version 50 2023-12-07T10:26:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.702+0000 7fb2b1312b80 10 client.27292 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:26:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.702+0000 7fb2b1312b80 20 client.27292 populate_metadata read hostname 'smithi119' 2023-12-07T10:26:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.702+0000 7fb2b1312b80 1 --2- 172.21.15.119:0/1187590697 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5645a9a07ca0 0x5645a9a0a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.703+0000 7fb2b1312b80 4 client.27292 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:26:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.703+0000 7fb2b1312b80 1 -- 172.21.15.119:0/1187590697 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645a99b9740 con 0x5645a9a07ca0 2023-12-07T10:26:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.703+0000 7fb2967fc700 1 --2- 172.21.15.119:0/1187590697 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5645a9a07ca0 0x5645a9a0a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.703+0000 7fb276ffd700 10 client.27292 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:26:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.703+0000 7fb2967fc700 1 --2- 172.21.15.119:0/1187590697 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5645a9a07ca0 0x5645a9a0a0c0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13d7b4829db94b3a server_cookie=5668a0fb84a09a6e in_seq=0 out_seq=0 2023-12-07T10:26:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.704+0000 7fb276ffd700 1 -- 172.21.15.119:0/1187590697 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5645a99b9740 con 0x5645a9a07ca0 2023-12-07T10:26:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.704+0000 7fb276ffd700 10 client.27292 handle_command_reply: tid=0 2023-12-07T10:26:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.704+0000 7fb2b1312b80 1 client.27292 shutdown 2023-12-07T10:26:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.704+0000 7fb2b1312b80 1 -- 172.21.15.119:0/1187590697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645a99eaa80 msgr2=0x5645a99f65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.704+0000 7fb2b1312b80 1 --2- 172.21.15.119:0/1187590697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645a99eaa80 0x5645a99f65c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb2880bc010 tx=0x7fb28809af40 comp rx=0 tx=0).stop 2023-12-07T10:26:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.704+0000 7fb2b1312b80 1 -- 172.21.15.119:0/1187590697 shutdown_connections 2023-12-07T10:26:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.704+0000 7fb2b1312b80 1 --2- 172.21.15.119:0/1187590697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645a99f05c0 0x5645a99f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.704+0000 7fb2b1312b80 1 --2- 172.21.15.119:0/1187590697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645a99eaa80 0x5645a99f65c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.704+0000 7fb2b1312b80 1 --2- 172.21.15.119:0/1187590697 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5645a9a07ca0 0x5645a9a0a0c0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.704+0000 7fb2b1312b80 1 --2- 172.21.15.119:0/1187590697 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645a99efca0 0x5645a99f3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.704+0000 7fb276ffd700 0 client.27292 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:26:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.704+0000 7fb2b1312b80 1 -- 172.21.15.119:0/1187590697 >> 172.21.15.119:0/1187590697 conn(0x5645a9919300 msgr2=0x5645a99df980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:26:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.705+0000 7fb2b1312b80 1 -- 172.21.15.119:0/1187590697 shutdown_connections 2023-12-07T10:26:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.705+0000 7fb2b1312b80 1 -- 172.21.15.119:0/1187590697 wait complete. 2023-12-07T10:26:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.705+0000 7fb2b1312b80 20 client.27292 trim_cache size 0 max 16384 2023-12-07T10:26:38.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.722+0000 7fb2b1312b80 1 Processor -- start 2023-12-07T10:26:38.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.722+0000 7fb2b1312b80 1 -- start start 2023-12-07T10:26:38.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.722+0000 7fb2b1312b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645a99eaa80 0x5645a99ea540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:38.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.722+0000 7fb2b1312b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645a99efca0 0x5645a99f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:38.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.722+0000 7fb2b1312b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645a99f05c0 0x5645a99ff020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:38.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.722+0000 7fb2b1312b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645a97bd8c0 con 0x5645a99f05c0 2023-12-07T10:26:38.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.722+0000 7fb2b1312b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645a99e66a0 con 0x5645a99eaa80 2023-12-07T10:26:38.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.722+0000 7fb2b1312b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645a99f0300 con 0x5645a99efca0 2023-12-07T10:26:38.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.722+0000 7fb2977fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645a99f05c0 0x5645a99ff020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.722+0000 7fb296ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645a99eaa80 0x5645a99ea540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.723+0000 7fb296ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645a99eaa80 0x5645a99ea540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:55606/0 (socket says 172.21.15.119:55606) 2023-12-07T10:26:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.723+0000 7fb296ffd700 1 -- 172.21.15.119:0/1950836959 learned_addr learned my addr 172.21.15.119:0/1950836959 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:26:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.723+0000 7fb2967fc700 1 --2- 172.21.15.119:0/1950836959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645a99efca0 0x5645a99f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.723+0000 7fb296ffd700 1 -- 172.21.15.119:0/1950836959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645a99efca0 msgr2=0x5645a99f30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.723+0000 7fb296ffd700 1 --2- 172.21.15.119:0/1950836959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645a99efca0 0x5645a99f30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.723+0000 7fb296ffd700 1 -- 172.21.15.119:0/1950836959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645a99f05c0 msgr2=0x5645a99ff020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.723+0000 7fb296ffd700 1 --2- 172.21.15.119:0/1950836959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645a99f05c0 0x5645a99ff020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.723+0000 7fb296ffd700 1 -- 172.21.15.119:0/1950836959 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645a9767d80 con 0x5645a99eaa80 2023-12-07T10:26:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.723+0000 7fb2977fe700 1 --2- 172.21.15.119:0/1950836959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645a99f05c0 0x5645a99ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:26:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.723+0000 7fb296ffd700 1 --2- 172.21.15.119:0/1950836959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645a99eaa80 0x5645a99ea540 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb28800b580 tx=0x7fb28800ff00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6de78b454165c82 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:26:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.723+0000 7fb27e7fc700 1 -- 172.21.15.119:0/1950836959 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2880ce640 con 0x5645a99eaa80 2023-12-07T10:26:38.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.724+0000 7fb27e7fc700 1 -- 172.21.15.119:0/1950836959 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb28809b9d0 con 0x5645a99eaa80 2023-12-07T10:26:38.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.724+0000 7fb27e7fc700 1 -- 172.21.15.119:0/1950836959 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28800f7c0 con 0x5645a99eaa80 2023-12-07T10:26:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.724+0000 7fb2b1312b80 1 -- 172.21.15.119:0/1950836959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645a99eaa80 msgr2=0x5645a99ea540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.724+0000 7fb2b1312b80 1 --2- 172.21.15.119:0/1950836959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645a99eaa80 0x5645a99ea540 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb28800b580 tx=0x7fb28800ff00 comp rx=0 tx=0).stop 2023-12-07T10:26:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.724+0000 7fb2b1312b80 1 -- 172.21.15.119:0/1950836959 shutdown_connections 2023-12-07T10:26:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.724+0000 7fb2b1312b80 1 --2- 172.21.15.119:0/1950836959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645a99f05c0 0x5645a99ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.724+0000 7fb2b1312b80 1 --2- 172.21.15.119:0/1950836959 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645a99eaa80 0x5645a99ea540 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.724+0000 7fb2b1312b80 1 --2- 172.21.15.119:0/1950836959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645a99efca0 0x5645a99f30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.724+0000 7fb2b1312b80 1 -- 172.21.15.119:0/1950836959 >> 172.21.15.119:0/1950836959 conn(0x5645a9919300 msgr2=0x5645a99edb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:26:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.724+0000 7fb2b1312b80 1 -- 172.21.15.119:0/1950836959 shutdown_connections 2023-12-07T10:26:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.724+0000 7fb2b1312b80 1 -- 172.21.15.119:0/1950836959 wait complete. 2023-12-07T10:26:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.725+0000 7fb2b1312b80 1 -- start start 2023-12-07T10:26:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.725+0000 7fb2b1312b80 1 Processor -- start 2023-12-07T10:26:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.726+0000 7fb2b1312b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645a99eaa80 0x5645a99ff210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.726+0000 7fb2b1312b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645a99efca0 0x5645a99fd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.726+0000 7fb2b1312b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645a99f05c0 0x5645a99fd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.726+0000 7fb2b1312b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645a99fdac0 con 0x5645a99f05c0 2023-12-07T10:26:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.726+0000 7fb2b1312b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645a99fdc30 con 0x5645a99efca0 2023-12-07T10:26:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.726+0000 7fb2b1312b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645a99fddf0 con 0x5645a99eaa80 2023-12-07T10:26:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.726+0000 7fb2b1312b80 10 client.0 _ll_register_callbacks cb 0x5645a98df8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:26:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.726+0000 7fb296ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645a99eaa80 0x5645a99ff210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.726+0000 7fb296ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645a99eaa80 0x5645a99ff210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:59560/0 (socket says 172.21.15.119:59560) 2023-12-07T10:26:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.726+0000 7fb2967fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645a99efca0 0x5645a99fd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.726+0000 7fb296ffd700 1 -- 172.21.15.119:0/905254332 learned_addr learned my addr 172.21.15.119:0/905254332 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:26:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.726+0000 7fb2977fe700 1 --2- 172.21.15.119:0/905254332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645a99f05c0 0x5645a99fd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.726+0000 7fb296ffd700 1 -- 172.21.15.119:0/905254332 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645a99efca0 msgr2=0x5645a99fd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.726+0000 7fb296ffd700 1 --2- 172.21.15.119:0/905254332 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645a99efca0 0x5645a99fd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.726+0000 7fb296ffd700 1 -- 172.21.15.119:0/905254332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645a99f05c0 msgr2=0x5645a99fd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.726+0000 7fb296ffd700 1 --2- 172.21.15.119:0/905254332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645a99f05c0 0x5645a99fd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.726+0000 7fb296ffd700 1 -- 172.21.15.119:0/905254332 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2900183f0 con 0x5645a99eaa80 2023-12-07T10:26:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.726+0000 7fb296ffd700 1 --2- 172.21.15.119:0/905254332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645a99eaa80 0x5645a99ff210 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb28809ada0 tx=0x7fb288099880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd789b2f1c494bd6 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:26:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.726+0000 7fb276ffd700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:26:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.727+0000 7fb276ffd700 1 -- 172.21.15.119:0/905254332 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28809f250 con 0x5645a99eaa80 2023-12-07T10:26:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.727+0000 7fb276ffd700 1 -- 172.21.15.119:0/905254332 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7fb28809f420 con 0x5645a99eaa80 2023-12-07T10:26:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.727+0000 7fb2b1312b80 1 -- 172.21.15.119:0/905254332 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb28809ac30 con 0x5645a99eaa80 2023-12-07T10:26:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.727+0000 7fb276ffd700 1 -- 172.21.15.119:0/905254332 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2880ce640 con 0x5645a99eaa80 2023-12-07T10:26:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.727+0000 7fb2b1312b80 10 client.27298 fetch_fsmap learned FSMap version 50 2023-12-07T10:26:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.727+0000 7fb2b1312b80 1 -- 172.21.15.119:0/905254332 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7fb2981a4510 con 0x5645a99eaa80 2023-12-07T10:26:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.727+0000 7fb276ffd700 1 -- 172.21.15.119:0/905254332 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb28809ac30 con 0x5645a99eaa80 2023-12-07T10:26:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.727+0000 7fb276ffd700 1 -- 172.21.15.119:0/905254332 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 50) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fb2880a3140 con 0x5645a99eaa80 2023-12-07T10:26:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.727+0000 7fb2b1312b80 10 client.27298 fetch_fsmap finished waiting for FSMap version 50 2023-12-07T10:26:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.727+0000 7fb2b1312b80 10 client.27298 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:26:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.727+0000 7fb2b1312b80 20 client.27298 populate_metadata read hostname 'smithi119' 2023-12-07T10:26:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.727+0000 7fb2b1312b80 1 --2- 172.21.15.119:0/905254332 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5645a9a5e2e0 0x5645a9a60700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.727+0000 7fb2b1312b80 4 client.27298 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:26:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.727+0000 7fb2b1312b80 1 -- 172.21.15.119:0/905254332 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5645a9904f70 con 0x5645a9a5e2e0 2023-12-07T10:26:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.728+0000 7fb2967fc700 1 --2- 172.21.15.119:0/905254332 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5645a9a5e2e0 0x5645a9a60700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.728+0000 7fb276ffd700 10 client.27298 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:26:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.728+0000 7fb2967fc700 1 --2- 172.21.15.119:0/905254332 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5645a9a5e2e0 0x5645a9a60700 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=736beb6216a6eef7 server_cookie=315bbfe437b7d9d5 in_seq=0 out_seq=0 2023-12-07T10:26:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.791+0000 7fb276ffd700 1 -- 172.21.15.119:0/905254332 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5645a9904f70 con 0x5645a9a5e2e0 2023-12-07T10:26:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.791+0000 7fb276ffd700 10 client.27298 handle_command_reply: tid=0 2023-12-07T10:26:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.791+0000 7fb2b1312b80 1 client.27298 shutdown 2023-12-07T10:26:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.791+0000 7fb2b1312b80 1 -- 172.21.15.119:0/905254332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645a99eaa80 msgr2=0x5645a99ff210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.791+0000 7fb2b1312b80 1 --2- 172.21.15.119:0/905254332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645a99eaa80 0x5645a99ff210 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb28809ada0 tx=0x7fb288099880 comp rx=0 tx=0).stop 2023-12-07T10:26:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.791+0000 7fb2b1312b80 1 -- 172.21.15.119:0/905254332 shutdown_connections 2023-12-07T10:26:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.791+0000 7fb2b1312b80 1 --2- 172.21.15.119:0/905254332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5645a99f05c0 0x5645a99fd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.791+0000 7fb2b1312b80 1 --2- 172.21.15.119:0/905254332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645a99eaa80 0x5645a99ff210 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.791+0000 7fb2b1312b80 1 --2- 172.21.15.119:0/905254332 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5645a9a5e2e0 0x5645a9a60700 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.791+0000 7fb2b1312b80 1 --2- 172.21.15.119:0/905254332 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5645a99efca0 0x5645a99fd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.791+0000 7fb276ffd700 0 client.27298 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:26:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.791+0000 7fb2b1312b80 1 -- 172.21.15.119:0/905254332 >> 172.21.15.119:0/905254332 conn(0x5645a9919300 msgr2=0x5645a99eeac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:26:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.791+0000 7fb2b1312b80 1 -- 172.21.15.119:0/905254332 shutdown_connections 2023-12-07T10:26:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.791+0000 7fb2b1312b80 1 -- 172.21.15.119:0/905254332 wait complete. 2023-12-07T10:26:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.792+0000 7fb2b1312b80 20 client.27298 trim_cache size 0 max 16384 2023-12-07T10:26:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:26:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:26:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:26:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:26:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.792+0000 7fb297fff700 1 -- 172.21.15.119:0/2698421772 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb2800a1d00 msgr2=0x7fb2800a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.792+0000 7fb297fff700 1 --2- 172.21.15.119:0/2698421772 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb2800a1d00 0x7fb2800a41c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb290009d30 tx=0x7fb29000b040 comp rx=0 tx=0).stop 2023-12-07T10:26:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.792+0000 7fb297fff700 1 -- 172.21.15.119:0/2698421772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2980688e0 msgr2=0x7fb29819d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.792+0000 7fb297fff700 1 --2- 172.21.15.119:0/2698421772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2980688e0 0x7fb29819d520 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb2880098e0 tx=0x7fb28800f690 comp rx=0 tx=0).stop 2023-12-07T10:26:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.792+0000 7fb297fff700 1 -- 172.21.15.119:0/2698421772 shutdown_connections 2023-12-07T10:26:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.792+0000 7fb297fff700 1 --2- 172.21.15.119:0/2698421772 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb2800a1d00 0x7fb2800a41c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.792+0000 7fb297fff700 1 --2- 172.21.15.119:0/2698421772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2981073f0 0x7fb298197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.792+0000 7fb297fff700 1 --2- 172.21.15.119:0/2698421772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2980688e0 0x7fb29819d520 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.792+0000 7fb297fff700 1 --2- 172.21.15.119:0/2698421772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2980692a0 0x7fb29819da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.792+0000 7fb297fff700 1 -- 172.21.15.119:0/2698421772 >> 172.21.15.119:0/2698421772 conn(0x7fb298075e70 msgr2=0x7fb2980ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:26:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.793+0000 7fb297fff700 1 -- 172.21.15.119:0/2698421772 shutdown_connections 2023-12-07T10:26:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:38.793+0000 7fb297fff700 1 -- 172.21.15.119:0/2698421772 wait complete. 2023-12-07T10:26:39.560 INFO:tasks.workunit.client.0.smithi119.stdout: 1 412917 7.04 MB/sec execute 172 sec latency 20.065 ms 2023-12-07T10:26:40.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:40 smithi119 ceph-mon[97908]: pgmap v190: 129 pgs: 129 active+clean; 235 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2023-12-07T10:26:40.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:40 smithi159 ceph-mon[134726]: pgmap v190: 129 pgs: 129 active+clean; 235 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2023-12-07T10:26:40.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:40 smithi153 ceph-mon[133625]: pgmap v190: 129 pgs: 129 active+clean; 235 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2023-12-07T10:26:40.561 INFO:tasks.workunit.client.0.smithi119.stdout: 1 414666 7.05 MB/sec execute 173 sec latency 21.304 ms 2023-12-07T10:26:41.560 INFO:tasks.workunit.client.0.smithi119.stdout: 1 415601 7.04 MB/sec execute 174 sec latency 46.669 ms 2023-12-07T10:26:42.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:41 smithi119 ceph-mon[97908]: pgmap v191: 129 pgs: 129 active+clean; 246 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 314 op/s 2023-12-07T10:26:42.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:41 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:26:42.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:41 smithi159 ceph-mon[134726]: pgmap v191: 129 pgs: 129 active+clean; 246 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 314 op/s 2023-12-07T10:26:42.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:41 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:26:42.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:41 smithi153 ceph-mon[133625]: pgmap v191: 129 pgs: 129 active+clean; 246 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 314 op/s 2023-12-07T10:26:42.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:41 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:26:42.560 INFO:tasks.workunit.client.0.smithi119.stdout: 1 416957 7.03 MB/sec execute 175 sec latency 26.750 ms 2023-12-07T10:26:43.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:42 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:26:43.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:42 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:26:43.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:42 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:26:43.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:42 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:26:43.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:42 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:26:43.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:42 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:26:43.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:42 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:26:43.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:42 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:26:43.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:42 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:26:43.564 INFO:tasks.workunit.client.0.smithi119.stdout: 1 418835 7.07 MB/sec execute 176 sec latency 16.333 ms 2023-12-07T10:26:44.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:43 smithi119 ceph-mon[97908]: pgmap v192: 129 pgs: 129 active+clean; 246 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 213 op/s 2023-12-07T10:26:44.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:43 smithi159 ceph-mon[134726]: pgmap v192: 129 pgs: 129 active+clean; 246 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 213 op/s 2023-12-07T10:26:44.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:43 smithi153 ceph-mon[133625]: pgmap v192: 129 pgs: 129 active+clean; 246 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 213 op/s 2023-12-07T10:26:44.560 INFO:tasks.workunit.client.0.smithi119.stdout: 1 419846 7.04 MB/sec execute 177 sec latency 15.322 ms 2023-12-07T10:26:45.560 INFO:tasks.workunit.client.0.smithi119.stdout: 1 421497 7.06 MB/sec execute 178 sec latency 17.126 ms 2023-12-07T10:26:46.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:46 smithi153 ceph-mon[133625]: pgmap v193: 129 pgs: 129 active+clean; 246 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 213 op/s 2023-12-07T10:26:46.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:46 smithi119 ceph-mon[97908]: pgmap v193: 129 pgs: 129 active+clean; 246 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 213 op/s 2023-12-07T10:26:46.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:46 smithi159 ceph-mon[134726]: pgmap v193: 129 pgs: 129 active+clean; 246 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 213 op/s 2023-12-07T10:26:46.560 INFO:tasks.workunit.client.0.smithi119.stdout: 1 422117 7.05 MB/sec execute 179 sec latency 342.849 ms 2023-12-07T10:26:47.560 INFO:tasks.workunit.client.0.smithi119.stdout: 1 422796 7.02 MB/sec execute 180 sec latency 29.549 ms 2023-12-07T10:26:48.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:48 smithi119 ceph-mon[97908]: pgmap v194: 129 pgs: 129 active+clean; 271 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.6 MiB/s wr, 342 op/s 2023-12-07T10:26:48.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:48 smithi159 ceph-mon[134726]: pgmap v194: 129 pgs: 129 active+clean; 271 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.6 MiB/s wr, 342 op/s 2023-12-07T10:26:48.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:48 smithi153 ceph-mon[133625]: pgmap v194: 129 pgs: 129 active+clean; 271 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.6 MiB/s wr, 342 op/s 2023-12-07T10:26:48.560 INFO:tasks.workunit.client.0.smithi119.stdout: 1 424501 7.02 MB/sec execute 181 sec latency 27.171 ms 2023-12-07T10:26:49.560 INFO:tasks.workunit.client.0.smithi119.stdout: 1 426049 7.05 MB/sec execute 182 sec latency 23.235 ms 2023-12-07T10:26:50.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:50 smithi119 ceph-mon[97908]: pgmap v195: 129 pgs: 129 active+clean; 271 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 249 op/s 2023-12-07T10:26:50.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:50 smithi159 ceph-mon[134726]: pgmap v195: 129 pgs: 129 active+clean; 271 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 249 op/s 2023-12-07T10:26:50.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:50 smithi153 ceph-mon[133625]: pgmap v195: 129 pgs: 129 active+clean; 271 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 249 op/s 2023-12-07T10:26:50.560 INFO:tasks.workunit.client.0.smithi119.stdout: 1 427179 7.04 MB/sec execute 183 sec latency 16.158 ms 2023-12-07T10:26:51.560 INFO:tasks.workunit.client.0.smithi119.stdout: 1 428935 7.05 MB/sec execute 184 sec latency 52.847 ms 2023-12-07T10:26:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.955+0000 7f8d5c3ff700 1 -- 172.21.15.119:0/3561777703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d541060c0 msgr2=0x7f8d54106540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.955+0000 7f8d5c3ff700 1 --2- 172.21.15.119:0/3561777703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d541060c0 0x7f8d54106540 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8d44009990 tx=0x7f8d44009ca0 comp rx=0 tx=0).stop 2023-12-07T10:26:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.955+0000 7f8d5c3ff700 1 -- 172.21.15.119:0/3561777703 shutdown_connections 2023-12-07T10:26:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.955+0000 7f8d5c3ff700 1 --2- 172.21.15.119:0/3561777703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d54106a80 0x7f8d5410e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.955+0000 7f8d5c3ff700 1 --2- 172.21.15.119:0/3561777703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d54068ce0 0x7f8d540690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.955+0000 7f8d5c3ff700 1 --2- 172.21.15.119:0/3561777703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d541060c0 0x7f8d54106540 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.955+0000 7f8d5c3ff700 1 -- 172.21.15.119:0/3561777703 >> 172.21.15.119:0/3561777703 conn(0x7f8d540775b0 msgr2=0x7f8d540779c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:26:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.955+0000 7f8d5c3ff700 1 -- 172.21.15.119:0/3561777703 shutdown_connections 2023-12-07T10:26:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.955+0000 7f8d5c3ff700 1 -- 172.21.15.119:0/3561777703 wait complete. 2023-12-07T10:26:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.957+0000 7f8d5c3ff700 1 Processor -- start 2023-12-07T10:26:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.957+0000 7f8d5c3ff700 1 -- start start 2023-12-07T10:26:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.957+0000 7f8d5c3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d54068ce0 0x7f8d5419f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:51.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.957+0000 7f8d5c3ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d541060c0 0x7f8d5419fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.957+0000 7f8d5c3ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d54106a80 0x7f8d54199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.957+0000 7f8d5c3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5419a0d0 con 0x7f8d54068ce0 2023-12-07T10:26:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.957+0000 7f8d5c3ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5419a240 con 0x7f8d54106a80 2023-12-07T10:26:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.957+0000 7f8d5c3ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5419a400 con 0x7f8d541060c0 2023-12-07T10:26:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.958+0000 7f8d5a19b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d54068ce0 0x7f8d5419f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.958+0000 7f8d5a99c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d54106a80 0x7f8d54199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.958+0000 7f8d5a19b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d54068ce0 0x7f8d5419f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36774/0 (socket says 172.21.15.119:36774) 2023-12-07T10:26:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.958+0000 7f8d5a19b700 1 -- 172.21.15.119:0/3951324995 learned_addr learned my addr 172.21.15.119:0/3951324995 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:26:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.958+0000 7f8d5a99c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d54106a80 0x7f8d54199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:50956/0 (socket says 172.21.15.119:50956) 2023-12-07T10:26:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.958+0000 7f8d5999a700 1 --2- 172.21.15.119:0/3951324995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d541060c0 0x7f8d5419fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.958+0000 7f8d5a99c700 1 -- 172.21.15.119:0/3951324995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d541060c0 msgr2=0x7f8d5419fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.958+0000 7f8d5a99c700 1 --2- 172.21.15.119:0/3951324995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d541060c0 0x7f8d5419fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.958+0000 7f8d5a99c700 1 -- 172.21.15.119:0/3951324995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d54068ce0 msgr2=0x7f8d5419f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:51.960 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.958+0000 7f8d5a99c700 1 --2- 172.21.15.119:0/3951324995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d54068ce0 0x7f8d5419f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.958+0000 7f8d5a99c700 1 -- 172.21.15.119:0/3951324995 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d440095f0 con 0x7f8d54106a80 2023-12-07T10:26:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.958+0000 7f8d5a99c700 1 --2- 172.21.15.119:0/3951324995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d54106a80 0x7f8d54199a70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00efc0 tx=0x7f8d4c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:26:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.959+0000 7f8d4b7fe700 1 -- 172.21.15.119:0/3951324995 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c018070 con 0x7f8d54106a80 2023-12-07T10:26:51.961 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.959+0000 7f8d4b7fe700 1 -- 172.21.15.119:0/3951324995 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d4c00f040 con 0x7f8d54106a80 2023-12-07T10:26:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.959+0000 7f8d4b7fe700 1 -- 172.21.15.119:0/3951324995 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c00cd40 con 0x7f8d54106a80 2023-12-07T10:26:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.959+0000 7f8d5c3ff700 1 -- 172.21.15.119:0/3951324995 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5419a680 con 0x7f8d54106a80 2023-12-07T10:26:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.959+0000 7f8d5c3ff700 1 -- 172.21.15.119:0/3951324995 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d541a6930 con 0x7f8d54106a80 2023-12-07T10:26:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.962+0000 7f8d4b7fe700 1 -- 172.21.15.119:0/3951324995 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f8d4c003a00 con 0x7f8d54106a80 2023-12-07T10:26:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.964+0000 7f8d4b7fe700 1 --2- 172.21.15.119:0/3951324995 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8d400a1d00 0x7f8d400a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.964+0000 7f8d4b7fe700 1 -- 172.21.15.119:0/3951324995 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f8d4c0cd810 con 0x7f8d54106a80 2023-12-07T10:26:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.964+0000 7f8d5a19b700 1 --2- 172.21.15.119:0/3951324995 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8d400a1d00 0x7f8d400a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.964+0000 7f8d6fdb0b80 1 Processor -- start 2023-12-07T10:26:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.965+0000 7f8d6fdb0b80 1 -- start start 2023-12-07T10:26:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.965+0000 7f8d6fdb0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15e0a60 0x5565a15e0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.965+0000 7f8d5a19b700 1 --2- 172.21.15.119:0/3951324995 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8d400a1d00 0x7f8d400a41c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8d50009d20 tx=0x7f8d50009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:26:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.965+0000 7f8d6fdb0b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565a15e1380 0x5565a15db540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.965+0000 7f8d6fdb0b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565a15dbad0 0x5565a15dbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.965+0000 7f8d6fdb0b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565a154f760 con 0x5565a15e1380 2023-12-07T10:26:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.965+0000 7f8d6fdb0b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565a1536740 con 0x5565a15dbad0 2023-12-07T10:26:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.965+0000 7f8d6fdb0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565a1534060 con 0x5565a15e0a60 2023-12-07T10:26:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.965+0000 7f8d5a19b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15e0a60 0x5565a15e0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.965+0000 7f8d5999a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565a15e1380 0x5565a15db540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.965+0000 7f8d5a19b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15e0a60 0x5565a15e0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:58560/0 (socket says 172.21.15.119:58560) 2023-12-07T10:26:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.965+0000 7f8d5a19b700 1 -- 172.21.15.119:0/354850625 learned_addr learned my addr 172.21.15.119:0/354850625 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:26:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.965+0000 7f8d5a99c700 1 --2- 172.21.15.119:0/354850625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565a15dbad0 0x5565a15dbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.966+0000 7f8d5a99c700 1 -- 172.21.15.119:0/354850625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15e0a60 msgr2=0x5565a15e0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.966+0000 7f8d5a99c700 1 --2- 172.21.15.119:0/354850625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15e0a60 0x5565a15e0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.966+0000 7f8d5a99c700 1 -- 172.21.15.119:0/354850625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565a15e1380 msgr2=0x5565a15db540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.966+0000 7f8d5a99c700 1 --2- 172.21.15.119:0/354850625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565a15e1380 0x5565a15db540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.966+0000 7f8d5a99c700 1 -- 172.21.15.119:0/354850625 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565a13ac5d0 con 0x5565a15dbad0 2023-12-07T10:26:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.966+0000 7f8d5a19b700 1 --2- 172.21.15.119:0/354850625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15e0a60 0x5565a15e0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:26:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.966+0000 7f8d5a99c700 1 --2- 172.21.15.119:0/354850625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565a15dbad0 0x5565a15dbf30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c0cabb0 tx=0x7f8d4c09a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac068c131394f423 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:26:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.966+0000 7f8d48bfe700 1 -- 172.21.15.119:0/354850625 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c094ea0 con 0x5565a15dbad0 2023-12-07T10:26:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.966+0000 7f8d48bfe700 1 -- 172.21.15.119:0/354850625 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d4c004ad0 con 0x5565a15dbad0 2023-12-07T10:26:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.966+0000 7f8d48bfe700 1 -- 172.21.15.119:0/354850625 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c0969a0 con 0x5565a15dbad0 2023-12-07T10:26:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.967+0000 7f8d6fdb0b80 1 -- 172.21.15.119:0/354850625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565a15dbad0 msgr2=0x5565a15dbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.967+0000 7f8d6fdb0b80 1 --2- 172.21.15.119:0/354850625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565a15dbad0 0x5565a15dbf30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c0cabb0 tx=0x7f8d4c09a060 comp rx=0 tx=0).stop 2023-12-07T10:26:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.967+0000 7f8d6fdb0b80 1 -- 172.21.15.119:0/354850625 shutdown_connections 2023-12-07T10:26:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.967+0000 7f8d6fdb0b80 1 --2- 172.21.15.119:0/354850625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565a15dbad0 0x5565a15dbf30 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.967+0000 7f8d6fdb0b80 1 --2- 172.21.15.119:0/354850625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15e0a60 0x5565a15e0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.967+0000 7f8d6fdb0b80 1 --2- 172.21.15.119:0/354850625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565a15e1380 0x5565a15db540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.967+0000 7f8d6fdb0b80 1 -- 172.21.15.119:0/354850625 >> 172.21.15.119:0/354850625 conn(0x5565a150a300 msgr2=0x5565a15d5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:26:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.967+0000 7f8d6fdb0b80 1 -- 172.21.15.119:0/354850625 shutdown_connections 2023-12-07T10:26:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.967+0000 7f8d6fdb0b80 1 -- 172.21.15.119:0/354850625 wait complete. 2023-12-07T10:26:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.968+0000 7f8d6fdb0b80 1 -- start start 2023-12-07T10:26:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.968+0000 7f8d6fdb0b80 1 Processor -- start 2023-12-07T10:26:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.969+0000 7f8d6fdb0b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565a15dbad0 0x5565a15e4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.969+0000 7f8d6fdb0b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565a15e0a60 0x5565a15e4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.969+0000 7f8d6fdb0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15e1380 0x5565a15e5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.969+0000 7f8d6fdb0b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565a152f950 con 0x5565a15dbad0 2023-12-07T10:26:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.969+0000 7f8d6fdb0b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565a14e9500 con 0x5565a15e0a60 2023-12-07T10:26:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.969+0000 7f8d6fdb0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565a14e91a0 con 0x5565a15e1380 2023-12-07T10:26:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.969+0000 7f8d6fdb0b80 10 client.0 _ll_register_callbacks cb 0x5565a14d08e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:26:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.969+0000 7f8d5a99c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15e1380 0x5565a15e5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.969+0000 7f8d5a19b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565a15dbad0 0x5565a15e4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.969+0000 7f8d5a99c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15e1380 0x5565a15e5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:58568/0 (socket says 172.21.15.119:58568) 2023-12-07T10:26:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.969+0000 7f8d5a99c700 1 -- 172.21.15.119:0/2841170840 learned_addr learned my addr 172.21.15.119:0/2841170840 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:26:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.969+0000 7f8d5a19b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565a15dbad0 0x5565a15e4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36804/0 (socket says 172.21.15.119:36804) 2023-12-07T10:26:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.969+0000 7f8d5999a700 1 --2- 172.21.15.119:0/2841170840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565a15e0a60 0x5565a15e4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.969+0000 7f8d5a99c700 1 -- 172.21.15.119:0/2841170840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565a15e0a60 msgr2=0x5565a15e4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.969+0000 7f8d5a99c700 1 --2- 172.21.15.119:0/2841170840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565a15e0a60 0x5565a15e4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.969+0000 7f8d5a99c700 1 -- 172.21.15.119:0/2841170840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565a15dbad0 msgr2=0x5565a15e4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.969+0000 7f8d5a99c700 1 --2- 172.21.15.119:0/2841170840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565a15dbad0 0x5565a15e4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.969+0000 7f8d5a99c700 1 -- 172.21.15.119:0/2841170840 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d50017070 con 0x5565a15e1380 2023-12-07T10:26:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.970+0000 7f8d5a99c700 1 --2- 172.21.15.119:0/2841170840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15e1380 0x5565a15e5210 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c0cac10 tx=0x7f8d4c09a290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a9dacf08114c4a0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:26:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.970+0000 7f8d3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:26:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.970+0000 7f8d3dffb700 1 -- 172.21.15.119:0/2841170840 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c09a990 con 0x5565a15e1380 2023-12-07T10:26:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.970+0000 7f8d3dffb700 1 -- 172.21.15.119:0/2841170840 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f8d4c09e3e0 con 0x5565a15e1380 2023-12-07T10:26:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.970+0000 7f8d6fdb0b80 1 -- 172.21.15.119:0/2841170840 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565a15ce6f0 con 0x5565a15e1380 2023-12-07T10:26:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.970+0000 7f8d3dffb700 1 -- 172.21.15.119:0/2841170840 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d4c0a2590 con 0x5565a15e1380 2023-12-07T10:26:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.970+0000 7f8d6fdb0b80 10 client.27310 fetch_fsmap learned FSMap version 50 2023-12-07T10:26:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.970+0000 7f8d6fdb0b80 1 -- 172.21.15.119:0/2841170840 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7f8d541a6930 con 0x5565a15e1380 2023-12-07T10:26:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.970+0000 7f8d3dffb700 1 -- 172.21.15.119:0/2841170840 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d4c080080 con 0x5565a15e1380 2023-12-07T10:26:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.971+0000 7f8d3dffb700 1 -- 172.21.15.119:0/2841170840 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 50) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f8d4c099780 con 0x5565a15e1380 2023-12-07T10:26:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.971+0000 7f8d6fdb0b80 10 client.27310 fetch_fsmap finished waiting for FSMap version 50 2023-12-07T10:26:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.971+0000 7f8d6fdb0b80 10 client.27310 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:26:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.971+0000 7f8d6fdb0b80 20 client.27310 populate_metadata read hostname 'smithi119' 2023-12-07T10:26:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.971+0000 7f8d6fdb0b80 1 --2- 172.21.15.119:0/2841170840 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5565a15f4a30 0x5565a15f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.971+0000 7f8d6fdb0b80 4 client.27310 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:26:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.971+0000 7f8d6fdb0b80 1 -- 172.21.15.119:0/2841170840 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565a15aa740 con 0x5565a15f4a30 2023-12-07T10:26:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.971+0000 7f8d5a19b700 1 --2- 172.21.15.119:0/2841170840 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5565a15f4a30 0x5565a15f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.972+0000 7f8d3dffb700 10 client.27310 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:26:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.972+0000 7f8d5a19b700 1 --2- 172.21.15.119:0/2841170840 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5565a15f4a30 0x5565a15f6e20 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4554e48e7b8008d server_cookie=2db228dae068927a in_seq=0 out_seq=0 2023-12-07T10:26:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.973+0000 7f8d3dffb700 1 -- 172.21.15.119:0/2841170840 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5565a15aa740 con 0x5565a15f4a30 2023-12-07T10:26:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.973+0000 7f8d3dffb700 10 client.27310 handle_command_reply: tid=0 2023-12-07T10:26:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.973+0000 7f8d6fdb0b80 1 client.27310 shutdown 2023-12-07T10:26:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.973+0000 7f8d6fdb0b80 1 -- 172.21.15.119:0/2841170840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15e1380 msgr2=0x5565a15e5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.973+0000 7f8d6fdb0b80 1 --2- 172.21.15.119:0/2841170840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15e1380 0x5565a15e5210 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c0cac10 tx=0x7f8d4c09a290 comp rx=0 tx=0).stop 2023-12-07T10:26:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.973+0000 7f8d6fdb0b80 1 -- 172.21.15.119:0/2841170840 shutdown_connections 2023-12-07T10:26:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.973+0000 7f8d6fdb0b80 1 --2- 172.21.15.119:0/2841170840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15e1380 0x5565a15e5210 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.973+0000 7f8d6fdb0b80 1 --2- 172.21.15.119:0/2841170840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565a15dbad0 0x5565a15e4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.973+0000 7f8d6fdb0b80 1 --2- 172.21.15.119:0/2841170840 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5565a15f4a30 0x5565a15f6e20 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.973+0000 7f8d6fdb0b80 1 --2- 172.21.15.119:0/2841170840 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565a15e0a60 0x5565a15e4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.973+0000 7f8d3dffb700 0 client.27310 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:26:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.973+0000 7f8d6fdb0b80 1 -- 172.21.15.119:0/2841170840 >> 172.21.15.119:0/2841170840 conn(0x5565a150a300 msgr2=0x5565a15d0980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:26:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.974+0000 7f8d6fdb0b80 1 -- 172.21.15.119:0/2841170840 shutdown_connections 2023-12-07T10:26:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.974+0000 7f8d6fdb0b80 1 -- 172.21.15.119:0/2841170840 wait complete. 2023-12-07T10:26:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:51.974+0000 7f8d6fdb0b80 20 client.27310 trim_cache size 0 max 16384 2023-12-07T10:26:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.000+0000 7f8d6fdb0b80 1 Processor -- start 2023-12-07T10:26:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.000+0000 7f8d6fdb0b80 1 -- start start 2023-12-07T10:26:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.001+0000 7f8d6fdb0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15dbad0 0x5565a15db540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.001+0000 7f8d6fdb0b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565a15e0a60 0x5565a15e5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.001+0000 7f8d6fdb0b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565a15e1380 0x5565a15dd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.001+0000 7f8d6fdb0b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565a13ae8c0 con 0x5565a15e1380 2023-12-07T10:26:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.001+0000 7f8d6fdb0b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565a15e5d90 con 0x5565a15e0a60 2023-12-07T10:26:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.001+0000 7f8d6fdb0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565a15d6260 con 0x5565a15dbad0 2023-12-07T10:26:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.002+0000 7f8d5a19b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15dbad0 0x5565a15db540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.002+0000 7f8d5a19b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15dbad0 0x5565a15db540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:58572/0 (socket says 172.21.15.119:58572) 2023-12-07T10:26:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.002+0000 7f8d5a19b700 1 -- 172.21.15.119:0/1532104148 learned_addr learned my addr 172.21.15.119:0/1532104148 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:26:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.002+0000 7f8d5999a700 1 --2- 172.21.15.119:0/1532104148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565a15e0a60 0x5565a15e5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.002+0000 7f8d5a19b700 1 -- 172.21.15.119:0/1532104148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565a15e0a60 msgr2=0x5565a15e5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.002+0000 7f8d5a19b700 1 --2- 172.21.15.119:0/1532104148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565a15e0a60 0x5565a15e5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.002+0000 7f8d5a19b700 1 -- 172.21.15.119:0/1532104148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565a15e1380 msgr2=0x5565a15dd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.003+0000 7f8d5a99c700 1 --2- 172.21.15.119:0/1532104148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565a15e1380 0x5565a15dd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.003+0000 7f8d5a19b700 1 --2- 172.21.15.119:0/1532104148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565a15e1380 0x5565a15dd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.003+0000 7f8d5a19b700 1 -- 172.21.15.119:0/1532104148 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565a1358d80 con 0x5565a15dbad0 2023-12-07T10:26:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.003+0000 7f8d5999a700 1 --2- 172.21.15.119:0/1532104148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565a15e0a60 0x5565a15e5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:26:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.003+0000 7f8d5a99c700 1 --2- 172.21.15.119:0/1532104148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565a15e1380 0x5565a15dd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:26:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.003+0000 7f8d5a19b700 1 --2- 172.21.15.119:0/1532104148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15dbad0 0x5565a15db540 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8d50013010 tx=0x7f8d50014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59f8378d062e98a4 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:26:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.003+0000 7f8d493ff700 1 -- 172.21.15.119:0/1532104148 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d50022040 con 0x5565a15dbad0 2023-12-07T10:26:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.003+0000 7f8d493ff700 1 -- 172.21.15.119:0/1532104148 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d50023040 con 0x5565a15dbad0 2023-12-07T10:26:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.003+0000 7f8d493ff700 1 -- 172.21.15.119:0/1532104148 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5001dce0 con 0x5565a15dbad0 2023-12-07T10:26:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.004+0000 7f8d6fdb0b80 1 -- 172.21.15.119:0/1532104148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15dbad0 msgr2=0x5565a15db540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.004+0000 7f8d6fdb0b80 1 --2- 172.21.15.119:0/1532104148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15dbad0 0x5565a15db540 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8d50013010 tx=0x7f8d50014000 comp rx=0 tx=0).stop 2023-12-07T10:26:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.004+0000 7f8d6fdb0b80 1 -- 172.21.15.119:0/1532104148 shutdown_connections 2023-12-07T10:26:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.004+0000 7f8d6fdb0b80 1 --2- 172.21.15.119:0/1532104148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565a15e1380 0x5565a15dd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.004+0000 7f8d6fdb0b80 1 --2- 172.21.15.119:0/1532104148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15dbad0 0x5565a15db540 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.004+0000 7f8d6fdb0b80 1 --2- 172.21.15.119:0/1532104148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565a15e0a60 0x5565a15e5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.004+0000 7f8d6fdb0b80 1 -- 172.21.15.119:0/1532104148 >> 172.21.15.119:0/1532104148 conn(0x5565a150a300 msgr2=0x5565a15fb6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:26:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.005+0000 7f8d6fdb0b80 1 -- 172.21.15.119:0/1532104148 shutdown_connections 2023-12-07T10:26:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.005+0000 7f8d6fdb0b80 1 -- 172.21.15.119:0/1532104148 wait complete. 2023-12-07T10:26:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.005+0000 7f8d6fdb0b80 1 -- start start 2023-12-07T10:26:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.006+0000 7f8d6fdb0b80 1 Processor -- start 2023-12-07T10:26:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.006+0000 7f8d6fdb0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15dbad0 0x5565a15e7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.006+0000 7f8d6fdb0b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565a15e0a60 0x5565a15e82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.006+0000 7f8d6fdb0b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565a15e1380 0x5565a15e8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.007+0000 7f8d6fdb0b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565a16527e0 con 0x5565a15e0a60 2023-12-07T10:26:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.007+0000 7f8d6fdb0b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565a1652950 con 0x5565a15e1380 2023-12-07T10:26:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.007+0000 7f8d6fdb0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565a1642e90 con 0x5565a15dbad0 2023-12-07T10:26:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.007+0000 7f8d6fdb0b80 10 client.0 _ll_register_callbacks cb 0x5565a14d08e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:26:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.007+0000 7f8d5a19b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15dbad0 0x5565a15e7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.007+0000 7f8d5999a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565a15e0a60 0x5565a15e82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.007+0000 7f8d5a19b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15dbad0 0x5565a15e7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:58578/0 (socket says 172.21.15.119:58578) 2023-12-07T10:26:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.007+0000 7f8d5a19b700 1 -- 172.21.15.119:0/2403186476 learned_addr learned my addr 172.21.15.119:0/2403186476 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:26:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.007+0000 7f8d5a99c700 1 --2- 172.21.15.119:0/2403186476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565a15e1380 0x5565a15e8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.007+0000 7f8d5a19b700 1 -- 172.21.15.119:0/2403186476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565a15e1380 msgr2=0x5565a15e8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.007+0000 7f8d5a19b700 1 --2- 172.21.15.119:0/2403186476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565a15e1380 0x5565a15e8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.007+0000 7f8d5a19b700 1 -- 172.21.15.119:0/2403186476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565a15e0a60 msgr2=0x5565a15e82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:26:52.010 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.007+0000 7f8d5a19b700 1 --2- 172.21.15.119:0/2403186476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565a15e0a60 0x5565a15e82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:52.010 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.007+0000 7f8d5a19b700 1 -- 172.21.15.119:0/2403186476 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565a1341c00 con 0x5565a15dbad0 2023-12-07T10:26:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.008+0000 7f8d5a19b700 1 --2- 172.21.15.119:0/2403186476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15dbad0 0x5565a15e7da0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8d5000e3f0 tx=0x7f8d5001d670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee484b41df6c4da2 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:26:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.008+0000 7f8d3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:26:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.008+0000 7f8d3dffb700 1 -- 172.21.15.119:0/2403186476 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5001dcd0 con 0x5565a15dbad0 2023-12-07T10:26:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.008+0000 7f8d6fdb0b80 1 -- 172.21.15.119:0/2403186476 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d4c004ad0 con 0x5565a15dbad0 2023-12-07T10:26:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.008+0000 7f8d3dffb700 1 -- 172.21.15.119:0/2403186476 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9373+0+0 (secure 0 0 0) 0x7f8d5001de30 con 0x5565a15dbad0 2023-12-07T10:26:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.008+0000 7f8d3dffb700 1 -- 172.21.15.119:0/2403186476 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d50005040 con 0x5565a15dbad0 2023-12-07T10:26:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.008+0000 7f8d6fdb0b80 10 client.27316 fetch_fsmap learned FSMap version 50 2023-12-07T10:26:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.008+0000 7f8d6fdb0b80 1 -- 172.21.15.119:0/2403186476 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5565a137f1f0 con 0x5565a15dbad0 2023-12-07T10:26:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.009+0000 7f8d3dffb700 1 -- 172.21.15.119:0/2403186476 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d4c004ad0 con 0x5565a15dbad0 2023-12-07T10:26:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.009+0000 7f8d3dffb700 1 -- 172.21.15.119:0/2403186476 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 50) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f8d5003b020 con 0x5565a15dbad0 2023-12-07T10:26:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.009+0000 7f8d6fdb0b80 10 client.27316 fetch_fsmap finished waiting for FSMap version 50 2023-12-07T10:26:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.009+0000 7f8d6fdb0b80 10 client.27316 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:26:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.009+0000 7f8d6fdb0b80 20 client.27316 populate_metadata read hostname 'smithi119' 2023-12-07T10:26:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.009+0000 7f8d6fdb0b80 1 --2- 172.21.15.119:0/2403186476 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5565a1644570 0x5565a1646960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:26:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.009+0000 7f8d6fdb0b80 4 client.27316 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:26:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.009+0000 7f8d6fdb0b80 1 -- 172.21.15.119:0/2403186476 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5565a14f5f70 con 0x5565a1644570 2023-12-07T10:26:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.009+0000 7f8d5999a700 1 --2- 172.21.15.119:0/2403186476 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5565a1644570 0x5565a1646960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:26:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.010+0000 7f8d5999a700 1 --2- 172.21.15.119:0/2403186476 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5565a1644570 0x5565a1646960 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b52da0d538808540 server_cookie=4743a38a363e3396 in_seq=0 out_seq=0 2023-12-07T10:26:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.010+0000 7f8d3dffb700 10 client.27316 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:26:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.049+0000 7f8d3dffb700 1 -- 172.21.15.119:0/2403186476 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5565a14f5f70 con 0x5565a1644570 2023-12-07T10:26:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.049+0000 7f8d3dffb700 10 client.27316 handle_command_reply: tid=0 2023-12-07T10:26:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.050+0000 7f8d6fdb0b80 1 client.27316 shutdown 2023-12-07T10:26:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.050+0000 7f8d6fdb0b80 1 -- 172.21.15.119:0/2403186476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15dbad0 msgr2=0x5565a15e7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.050+0000 7f8d6fdb0b80 1 --2- 172.21.15.119:0/2403186476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15dbad0 0x5565a15e7da0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8d5000e3f0 tx=0x7f8d5001d670 comp rx=0 tx=0).stop 2023-12-07T10:26:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.050+0000 7f8d6fdb0b80 1 -- 172.21.15.119:0/2403186476 shutdown_connections 2023-12-07T10:26:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.050+0000 7f8d6fdb0b80 1 --2- 172.21.15.119:0/2403186476 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5565a1644570 0x5565a1646960 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.050+0000 7f8d6fdb0b80 1 --2- 172.21.15.119:0/2403186476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5565a15e1380 0x5565a15e8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.050+0000 7f8d3dffb700 0 client.27316 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:26:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.050+0000 7f8d6fdb0b80 1 --2- 172.21.15.119:0/2403186476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565a15dbad0 0x5565a15e7da0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.050+0000 7f8d6fdb0b80 1 --2- 172.21.15.119:0/2403186476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565a15e0a60 0x5565a15e82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.050+0000 7f8d6fdb0b80 1 -- 172.21.15.119:0/2403186476 >> 172.21.15.119:0/2403186476 conn(0x5565a150a300 msgr2=0x5565a15dfa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:26:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.050+0000 7f8d6fdb0b80 1 -- 172.21.15.119:0/2403186476 shutdown_connections 2023-12-07T10:26:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.050+0000 7f8d6fdb0b80 1 -- 172.21.15.119:0/2403186476 wait complete. 2023-12-07T10:26:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.050+0000 7f8d6fdb0b80 20 client.27316 trim_cache size 0 max 16384 2023-12-07T10:26:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:26:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:26:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:26:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:26:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.051+0000 7f8d5c3ff700 1 -- 172.21.15.119:0/3951324995 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8d400a1d00 msgr2=0x7f8d400a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.051+0000 7f8d5c3ff700 1 --2- 172.21.15.119:0/3951324995 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8d400a1d00 0x7f8d400a41c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8d50009d20 tx=0x7f8d50009260 comp rx=0 tx=0).stop 2023-12-07T10:26:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.051+0000 7f8d5c3ff700 1 -- 172.21.15.119:0/3951324995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d54106a80 msgr2=0x7f8d54199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:26:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.051+0000 7f8d5c3ff700 1 --2- 172.21.15.119:0/3951324995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d54106a80 0x7f8d54199a70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00efc0 tx=0x7f8d4c00c3a0 comp rx=0 tx=0).stop 2023-12-07T10:26:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.051+0000 7f8d5c3ff700 1 -- 172.21.15.119:0/3951324995 shutdown_connections 2023-12-07T10:26:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.051+0000 7f8d5c3ff700 1 --2- 172.21.15.119:0/3951324995 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8d400a1d00 0x7f8d400a41c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.051+0000 7f8d5c3ff700 1 --2- 172.21.15.119:0/3951324995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d54106a80 0x7f8d54199a70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.051+0000 7f8d5c3ff700 1 --2- 172.21.15.119:0/3951324995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d54068ce0 0x7f8d5419f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.051+0000 7f8d5c3ff700 1 --2- 172.21.15.119:0/3951324995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d541060c0 0x7f8d5419fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:26:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.051+0000 7f8d5c3ff700 1 -- 172.21.15.119:0/3951324995 >> 172.21.15.119:0/3951324995 conn(0x7f8d540775b0 msgr2=0x7f8d5410c950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:26:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.051+0000 7f8d5c3ff700 1 -- 172.21.15.119:0/3951324995 shutdown_connections 2023-12-07T10:26:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:26:52.051+0000 7f8d5c3ff700 1 -- 172.21.15.119:0/3951324995 wait complete. 2023-12-07T10:26:52.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:51 smithi119 ceph-mon[97908]: pgmap v196: 129 pgs: 129 active+clean; 258 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.9 MiB/s wr, 343 op/s 2023-12-07T10:26:52.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:51 smithi159 ceph-mon[134726]: pgmap v196: 129 pgs: 129 active+clean; 258 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.9 MiB/s wr, 343 op/s 2023-12-07T10:26:52.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:51 smithi153 ceph-mon[133625]: pgmap v196: 129 pgs: 129 active+clean; 258 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.9 MiB/s wr, 343 op/s 2023-12-07T10:26:52.560 INFO:tasks.workunit.client.0.smithi119.stdout: 1 429944 7.04 MB/sec execute 185 sec latency 17.688 ms 2023-12-07T10:26:53.560 INFO:tasks.workunit.client.0.smithi119.stdout: 1 431029 7.03 MB/sec execute 186 sec latency 64.899 ms 2023-12-07T10:26:54.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:54 smithi119 ceph-mon[97908]: pgmap v197: 129 pgs: 129 active+clean; 247 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 284 op/s 2023-12-07T10:26:54.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:54 smithi159 ceph-mon[134726]: pgmap v197: 129 pgs: 129 active+clean; 247 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 284 op/s 2023-12-07T10:26:54.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:54 smithi153 ceph-mon[133625]: pgmap v197: 129 pgs: 129 active+clean; 247 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 284 op/s 2023-12-07T10:26:54.560 INFO:tasks.workunit.client.0.smithi119.stdout: 1 432571 7.04 MB/sec execute 187 sec latency 320.056 ms 2023-12-07T10:26:55.561 INFO:tasks.workunit.client.0.smithi119.stdout: 1 432571 7.00 MB/sec execute 188 sec latency 1320.125 ms 2023-12-07T10:26:56.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:56 smithi119 ceph-mon[97908]: pgmap v198: 129 pgs: 129 active+clean; 247 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 284 op/s 2023-12-07T10:26:56.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:56 smithi159 ceph-mon[134726]: pgmap v198: 129 pgs: 129 active+clean; 247 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 284 op/s 2023-12-07T10:26:56.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:56 smithi153 ceph-mon[133625]: pgmap v198: 129 pgs: 129 active+clean; 247 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 284 op/s 2023-12-07T10:26:56.561 INFO:tasks.workunit.client.0.smithi119.stdout: 1 432571 6.96 MB/sec execute 189 sec latency 2320.242 ms 2023-12-07T10:26:57.561 INFO:tasks.workunit.client.0.smithi119.stdout: 1 432571 6.93 MB/sec execute 190 sec latency 3320.330 ms 2023-12-07T10:26:58.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:26:58 smithi119 ceph-mon[97908]: pgmap v199: 129 pgs: 129 active+clean; 247 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 365 op/s 2023-12-07T10:26:58.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:26:58 smithi159 ceph-mon[134726]: pgmap v199: 129 pgs: 129 active+clean; 247 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 365 op/s 2023-12-07T10:26:58.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:26:58 smithi153 ceph-mon[133625]: pgmap v199: 129 pgs: 129 active+clean; 247 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 365 op/s 2023-12-07T10:26:58.561 INFO:tasks.workunit.client.0.smithi119.stdout: 1 433150 6.92 MB/sec execute 191 sec latency 3747.174 ms 2023-12-07T10:26:59.561 INFO:tasks.workunit.client.0.smithi119.stdout: 1 434355 6.91 MB/sec execute 192 sec latency 15.555 ms 2023-12-07T10:27:00.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:00 smithi119 ceph-mon[97908]: pgmap v200: 129 pgs: 129 active+clean; 247 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 237 op/s 2023-12-07T10:27:00.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:00 smithi159 ceph-mon[134726]: pgmap v200: 129 pgs: 129 active+clean; 247 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 237 op/s 2023-12-07T10:27:00.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:00 smithi153 ceph-mon[133625]: pgmap v200: 129 pgs: 129 active+clean; 247 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 237 op/s 2023-12-07T10:27:00.562 INFO:tasks.workunit.client.0.smithi119.stdout: 1 436005 6.92 MB/sec execute 193 sec latency 54.261 ms 2023-12-07T10:27:01.561 INFO:tasks.workunit.client.0.smithi119.stdout: 1 436561 6.91 MB/sec execute 194 sec latency 62.759 ms 2023-12-07T10:27:01.978 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:01 smithi119 ceph-mon[97908]: pgmap v201: 129 pgs: 129 active+clean; 250 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 271 op/s 2023-12-07T10:27:02.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:01 smithi159 ceph-mon[134726]: pgmap v201: 129 pgs: 129 active+clean; 250 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 271 op/s 2023-12-07T10:27:02.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:01 smithi153 ceph-mon[133625]: pgmap v201: 129 pgs: 129 active+clean; 250 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 271 op/s 2023-12-07T10:27:02.561 INFO:tasks.workunit.client.0.smithi119.stdout: 1 437571 6.88 MB/sec execute 195 sec latency 16.116 ms 2023-12-07T10:27:03.561 INFO:tasks.workunit.client.0.smithi119.stdout: 1 439214 6.90 MB/sec execute 196 sec latency 19.757 ms 2023-12-07T10:27:04.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:04 smithi119 ceph-mon[97908]: pgmap v202: 129 pgs: 129 active+clean; 250 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 215 op/s 2023-12-07T10:27:04.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:04 smithi159 ceph-mon[134726]: pgmap v202: 129 pgs: 129 active+clean; 250 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 215 op/s 2023-12-07T10:27:04.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:04 smithi153 ceph-mon[133625]: pgmap v202: 129 pgs: 129 active+clean; 250 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 215 op/s 2023-12-07T10:27:04.561 INFO:tasks.workunit.client.0.smithi119.stdout: 1 440327 6.90 MB/sec execute 197 sec latency 41.888 ms 2023-12-07T10:27:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.250+0000 7fda3a7ff700 1 -- 172.21.15.119:0/3518981468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda34101ec0 msgr2=0x7fda3410e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.250+0000 7fda3a7ff700 1 --2- 172.21.15.119:0/3518981468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda34101ec0 0x7fda3410e380 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00b0d0 tx=0x7fda2c00b3e0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.250+0000 7fda3a7ff700 1 -- 172.21.15.119:0/3518981468 shutdown_connections 2023-12-07T10:27:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.250+0000 7fda3a7ff700 1 --2- 172.21.15.119:0/3518981468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda34101ec0 0x7fda3410e380 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.250+0000 7fda3a7ff700 1 --2- 172.21.15.119:0/3518981468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda341075c0 0x7fda341079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.250+0000 7fda3a7ff700 1 --2- 172.21.15.119:0/3518981468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda34101500 0x7fda34101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.250+0000 7fda3a7ff700 1 -- 172.21.15.119:0/3518981468 >> 172.21.15.119:0/3518981468 conn(0x7fda340fcfb0 msgr2=0x7fda340ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.250+0000 7fda3a7ff700 1 -- 172.21.15.119:0/3518981468 shutdown_connections 2023-12-07T10:27:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.250+0000 7fda3a7ff700 1 -- 172.21.15.119:0/3518981468 wait complete. 2023-12-07T10:27:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.251+0000 7fda3a7ff700 1 Processor -- start 2023-12-07T10:27:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.252+0000 7fda3a7ff700 1 -- start start 2023-12-07T10:27:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.252+0000 7fda3a7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda34101500 0x7fda3419b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.252+0000 7fda3a7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda34101ec0 0x7fda3419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.252+0000 7fda3a7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda341075c0 0x7fda341956a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.252+0000 7fda3a7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda34195d00 con 0x7fda341075c0 2023-12-07T10:27:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.252+0000 7fda3a7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda34195e70 con 0x7fda34101500 2023-12-07T10:27:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.252+0000 7fda3a7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda34196030 con 0x7fda34101ec0 2023-12-07T10:27:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.252+0000 7fda33fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda34101500 0x7fda3419b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.252+0000 7fda33fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda34101500 0x7fda3419b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:41932/0 (socket says 172.21.15.119:41932) 2023-12-07T10:27:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.252+0000 7fda33fff700 1 -- 172.21.15.119:0/3762903409 learned_addr learned my addr 172.21.15.119:0/3762903409 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:27:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.252+0000 7fda337fe700 1 --2- 172.21.15.119:0/3762903409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda34101ec0 0x7fda3419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.252+0000 7fda38d9c700 1 --2- 172.21.15.119:0/3762903409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda341075c0 0x7fda341956a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.253+0000 7fda33fff700 1 -- 172.21.15.119:0/3762903409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda34101ec0 msgr2=0x7fda3419ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.253+0000 7fda33fff700 1 --2- 172.21.15.119:0/3762903409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda34101ec0 0x7fda3419ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.253+0000 7fda33fff700 1 -- 172.21.15.119:0/3762903409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda341075c0 msgr2=0x7fda341956a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.253+0000 7fda33fff700 1 --2- 172.21.15.119:0/3762903409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda341075c0 0x7fda341956a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.253+0000 7fda33fff700 1 -- 172.21.15.119:0/3762903409 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda28013b90 con 0x7fda34101500 2023-12-07T10:27:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.253+0000 7fda33fff700 1 --2- 172.21.15.119:0/3762903409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda34101500 0x7fda3419b520 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fda2400bc50 tx=0x7fda2400bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.253+0000 7fda317fa700 1 -- 172.21.15.119:0/3762903409 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda240089c0 con 0x7fda34101500 2023-12-07T10:27:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.253+0000 7fda317fa700 1 -- 172.21.15.119:0/3762903409 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda24015400 con 0x7fda34101500 2023-12-07T10:27:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.253+0000 7fda3a7ff700 1 -- 172.21.15.119:0/3762903409 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda2c009d00 con 0x7fda34101500 2023-12-07T10:27:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.253+0000 7fda317fa700 1 -- 172.21.15.119:0/3762903409 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda24014600 con 0x7fda34101500 2023-12-07T10:27:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.253+0000 7fda3a7ff700 1 -- 172.21.15.119:0/3762903409 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda34196500 con 0x7fda34101500 2023-12-07T10:27:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.257+0000 7fda317fa700 1 -- 172.21.15.119:0/3762903409 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fda24015570 con 0x7fda34101500 2023-12-07T10:27:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.258+0000 7fda317fa700 1 --2- 172.21.15.119:0/3762903409 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fda1c0a1d00 0x7fda1c0a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:05.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.258+0000 7fda317fa700 1 -- 172.21.15.119:0/3762903409 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9549+0+0 (secure 0 0 0) 0x7fda240cddd0 con 0x7fda34101500 2023-12-07T10:27:05.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.258+0000 7fda337fe700 1 --2- 172.21.15.119:0/3762903409 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fda1c0a1d00 0x7fda1c0a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:05.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.259+0000 7fda4e0b9b80 1 Processor -- start 2023-12-07T10:27:05.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.259+0000 7fda4e0b9b80 1 -- start start 2023-12-07T10:27:05.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.259+0000 7fda4e0b9b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c933101a60 0x55c933101e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:05.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.259+0000 7fda4e0b9b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c933102380 0x55c9330fc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:05.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.259+0000 7fda4e0b9b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9330fcad0 0x55c9330fcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:05.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.259+0000 7fda4e0b9b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c933070760 con 0x55c933102380 2023-12-07T10:27:05.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.259+0000 7fda4e0b9b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c933057740 con 0x55c9330fcad0 2023-12-07T10:27:05.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.259+0000 7fda4e0b9b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c933055060 con 0x55c933101a60 2023-12-07T10:27:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.259+0000 7fda337fe700 1 --2- 172.21.15.119:0/3762903409 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fda1c0a1d00 0x7fda1c0a41c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fda34196d60 tx=0x7fda28009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.259+0000 7fda33fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c933101a60 0x55c933101e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.259+0000 7fda33fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c933101a60 0x55c933101e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:36852/0 (socket says 172.21.15.119:36852) 2023-12-07T10:27:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.259+0000 7fda33fff700 1 -- 172.21.15.119:0/358717612 learned_addr learned my addr 172.21.15.119:0/358717612 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:27:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.259+0000 7fda38d9c700 1 --2- 172.21.15.119:0/358717612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9330fcad0 0x55c9330fcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.259+0000 7fda337fe700 1 --2- 172.21.15.119:0/358717612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c933102380 0x55c9330fc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.260+0000 7fda33fff700 1 -- 172.21.15.119:0/358717612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9330fcad0 msgr2=0x55c9330fcf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.260+0000 7fda33fff700 1 --2- 172.21.15.119:0/358717612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9330fcad0 0x55c9330fcf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.260+0000 7fda33fff700 1 -- 172.21.15.119:0/358717612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c933102380 msgr2=0x55c9330fc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.260+0000 7fda33fff700 1 --2- 172.21.15.119:0/358717612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c933102380 0x55c9330fc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.260+0000 7fda33fff700 1 -- 172.21.15.119:0/358717612 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c932ecd5d0 con 0x55c933101a60 2023-12-07T10:27:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.260+0000 7fda337fe700 1 --2- 172.21.15.119:0/358717612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c933102380 0x55c9330fc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:27:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.260+0000 7fda33fff700 1 --2- 172.21.15.119:0/358717612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c933101a60 0x55c933101e40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fda240a2570 tx=0x7fda2409a320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93ccb516f9d980cc server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.260+0000 7fda1a7fc700 1 -- 172.21.15.119:0/358717612 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda240ba0c0 con 0x55c933101a60 2023-12-07T10:27:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.260+0000 7fda1a7fc700 1 -- 172.21.15.119:0/358717612 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda2409ae10 con 0x55c933101a60 2023-12-07T10:27:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.260+0000 7fda1a7fc700 1 -- 172.21.15.119:0/358717612 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda24015940 con 0x55c933101a60 2023-12-07T10:27:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.261+0000 7fda4e0b9b80 1 -- 172.21.15.119:0/358717612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c933101a60 msgr2=0x55c933101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.261+0000 7fda4e0b9b80 1 --2- 172.21.15.119:0/358717612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c933101a60 0x55c933101e40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fda240a2570 tx=0x7fda2409a320 comp rx=0 tx=0).stop 2023-12-07T10:27:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.261+0000 7fda4e0b9b80 1 -- 172.21.15.119:0/358717612 shutdown_connections 2023-12-07T10:27:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.261+0000 7fda4e0b9b80 1 --2- 172.21.15.119:0/358717612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c9330fcad0 0x55c9330fcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.261+0000 7fda4e0b9b80 1 --2- 172.21.15.119:0/358717612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c933101a60 0x55c933101e40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.261+0000 7fda4e0b9b80 1 --2- 172.21.15.119:0/358717612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c933102380 0x55c9330fc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.261+0000 7fda4e0b9b80 1 -- 172.21.15.119:0/358717612 >> 172.21.15.119:0/358717612 conn(0x55c93302b300 msgr2=0x55c9330f6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.261+0000 7fda4e0b9b80 1 -- 172.21.15.119:0/358717612 shutdown_connections 2023-12-07T10:27:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.262+0000 7fda4e0b9b80 1 -- 172.21.15.119:0/358717612 wait complete. 2023-12-07T10:27:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.262+0000 7fda4e0b9b80 1 -- start start 2023-12-07T10:27:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.262+0000 7fda4e0b9b80 1 Processor -- start 2023-12-07T10:27:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.263+0000 7fda4e0b9b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9330fcad0 0x55c933105710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.263+0000 7fda4e0b9b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c933101a60 0x55c933105c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.263+0000 7fda4e0b9b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c933102380 0x55c933106190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.263+0000 7fda4e0b9b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c933050950 con 0x55c9330fcad0 2023-12-07T10:27:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.263+0000 7fda4e0b9b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93300a500 con 0x55c933101a60 2023-12-07T10:27:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.263+0000 7fda4e0b9b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93300a1a0 con 0x55c933102380 2023-12-07T10:27:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.264+0000 7fda4e0b9b80 10 client.0 _ll_register_callbacks cb 0x55c932ff18e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:27:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.264+0000 7fda337fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c933101a60 0x55c933105c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.264+0000 7fda38d9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c933102380 0x55c933106190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.264+0000 7fda337fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c933101a60 0x55c933105c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:41954/0 (socket says 172.21.15.119:41954) 2023-12-07T10:27:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.264+0000 7fda337fe700 1 -- 172.21.15.119:0/2702953896 learned_addr learned my addr 172.21.15.119:0/2702953896 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:27:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.264+0000 7fda33fff700 1 --2- 172.21.15.119:0/2702953896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9330fcad0 0x55c933105710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.264+0000 7fda337fe700 1 -- 172.21.15.119:0/2702953896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c933102380 msgr2=0x55c933106190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.264+0000 7fda337fe700 1 --2- 172.21.15.119:0/2702953896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c933102380 0x55c933106190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.264+0000 7fda337fe700 1 -- 172.21.15.119:0/2702953896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9330fcad0 msgr2=0x55c933105710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.264+0000 7fda337fe700 1 --2- 172.21.15.119:0/2702953896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9330fcad0 0x55c933105710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.264+0000 7fda337fe700 1 -- 172.21.15.119:0/2702953896 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda34196500 con 0x55c933101a60 2023-12-07T10:27:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.264+0000 7fda337fe700 1 --2- 172.21.15.119:0/2702953896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c933101a60 0x55c933105c50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fda2801fc10 tx=0x7fda2801d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bb193495fb59afe server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.264+0000 7fda137fe700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:27:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.265+0000 7fda137fe700 1 -- 172.21.15.119:0/2702953896 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda28020c90 con 0x55c933101a60 2023-12-07T10:27:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.265+0000 7fda4e0b9b80 1 -- 172.21.15.119:0/2702953896 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c9330ef6f0 con 0x55c933101a60 2023-12-07T10:27:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.265+0000 7fda137fe700 1 -- 172.21.15.119:0/2702953896 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9549+0+0 (secure 0 0 0) 0x7fda28020df0 con 0x55c933101a60 2023-12-07T10:27:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.265+0000 7fda137fe700 1 -- 172.21.15.119:0/2702953896 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda28035860 con 0x55c933101a60 2023-12-07T10:27:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.265+0000 7fda4e0b9b80 10 client.27344 fetch_fsmap learned FSMap version 50 2023-12-07T10:27:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.265+0000 7fda4e0b9b80 1 -- 172.21.15.119:0/2702953896 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55c932e79d80 con 0x55c933101a60 2023-12-07T10:27:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.265+0000 7fda137fe700 1 -- 172.21.15.119:0/2702953896 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c9330ef6f0 con 0x55c933101a60 2023-12-07T10:27:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.265+0000 7fda137fe700 1 -- 172.21.15.119:0/2702953896 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 50) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fda2802c600 con 0x55c933101a60 2023-12-07T10:27:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.266+0000 7fda4e0b9b80 10 client.27344 fetch_fsmap finished waiting for FSMap version 50 2023-12-07T10:27:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.266+0000 7fda4e0b9b80 10 client.27344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:27:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.266+0000 7fda4e0b9b80 20 client.27344 populate_metadata read hostname 'smithi119' 2023-12-07T10:27:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.266+0000 7fda4e0b9b80 1 --2- 172.21.15.119:0/2702953896 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c933115f10 0x55c933118330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.266+0000 7fda4e0b9b80 4 client.27344 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:27:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.266+0000 7fda4e0b9b80 1 -- 172.21.15.119:0/2702953896 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9330cb740 con 0x55c933115f10 2023-12-07T10:27:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.266+0000 7fda33fff700 1 --2- 172.21.15.119:0/2702953896 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c933115f10 0x55c933118330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.266+0000 7fda33fff700 1 --2- 172.21.15.119:0/2702953896 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c933115f10 0x55c933118330 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f21bf85a4c50b73 server_cookie=f924af3d4a4af1f9 in_seq=0 out_seq=0 2023-12-07T10:27:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.266+0000 7fda137fe700 10 client.27344 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:27:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.267+0000 7fda137fe700 1 -- 172.21.15.119:0/2702953896 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c9330cb740 con 0x55c933115f10 2023-12-07T10:27:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.267+0000 7fda137fe700 10 client.27344 handle_command_reply: tid=0 2023-12-07T10:27:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.267+0000 7fda4e0b9b80 1 client.27344 shutdown 2023-12-07T10:27:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.267+0000 7fda4e0b9b80 1 -- 172.21.15.119:0/2702953896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c933101a60 msgr2=0x55c933105c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.267+0000 7fda4e0b9b80 1 --2- 172.21.15.119:0/2702953896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c933101a60 0x55c933105c50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fda2801fc10 tx=0x7fda2801d000 comp rx=0 tx=0).stop 2023-12-07T10:27:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.267+0000 7fda4e0b9b80 1 -- 172.21.15.119:0/2702953896 shutdown_connections 2023-12-07T10:27:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.267+0000 7fda4e0b9b80 1 --2- 172.21.15.119:0/2702953896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c933102380 0x55c933106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.267+0000 7fda4e0b9b80 1 --2- 172.21.15.119:0/2702953896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9330fcad0 0x55c933105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.267+0000 7fda4e0b9b80 1 --2- 172.21.15.119:0/2702953896 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c933115f10 0x55c933118330 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.267+0000 7fda4e0b9b80 1 --2- 172.21.15.119:0/2702953896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c933101a60 0x55c933105c50 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.267+0000 7fda137fe700 0 client.27344 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:27:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.267+0000 7fda4e0b9b80 1 -- 172.21.15.119:0/2702953896 >> 172.21.15.119:0/2702953896 conn(0x55c93302b300 msgr2=0x55c9330f1980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.268+0000 7fda4e0b9b80 1 -- 172.21.15.119:0/2702953896 shutdown_connections 2023-12-07T10:27:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.268+0000 7fda4e0b9b80 1 -- 172.21.15.119:0/2702953896 wait complete. 2023-12-07T10:27:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.268+0000 7fda4e0b9b80 20 client.27344 trim_cache size 0 max 16384 2023-12-07T10:27:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.285+0000 7fda4e0b9b80 1 Processor -- start 2023-12-07T10:27:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.285+0000 7fda4e0b9b80 1 -- start start 2023-12-07T10:27:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.285+0000 7fda4e0b9b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9330fcad0 0x55c933106030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.285+0000 7fda4e0b9b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c933101a60 0x55c93310cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.285+0000 7fda4e0b9b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c933102380 0x55c93310d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.285+0000 7fda4e0b9b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c932ecf8c0 con 0x55c933102380 2023-12-07T10:27:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.285+0000 7fda4e0b9b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93311d5c0 con 0x55c933101a60 2023-12-07T10:27:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.285+0000 7fda4e0b9b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c933103d20 con 0x55c9330fcad0 2023-12-07T10:27:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.285+0000 7fda33fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9330fcad0 0x55c933106030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.286+0000 7fda33fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9330fcad0 0x55c933106030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:36868/0 (socket says 172.21.15.119:36868) 2023-12-07T10:27:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.286+0000 7fda337fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c933101a60 0x55c93310cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.286+0000 7fda33fff700 1 -- 172.21.15.119:0/282008636 learned_addr learned my addr 172.21.15.119:0/282008636 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:27:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.286+0000 7fda38d9c700 1 --2- 172.21.15.119:0/282008636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c933102380 0x55c93310d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.286+0000 7fda337fe700 1 -- 172.21.15.119:0/282008636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9330fcad0 msgr2=0x55c933106030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.286+0000 7fda337fe700 1 --2- 172.21.15.119:0/282008636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9330fcad0 0x55c933106030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.286+0000 7fda337fe700 1 -- 172.21.15.119:0/282008636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c933102380 msgr2=0x55c93310d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.286+0000 7fda337fe700 1 --2- 172.21.15.119:0/282008636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c933102380 0x55c93310d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.286+0000 7fda337fe700 1 -- 172.21.15.119:0/282008636 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c932e62c00 con 0x55c933101a60 2023-12-07T10:27:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.286+0000 7fda33fff700 1 --2- 172.21.15.119:0/282008636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9330fcad0 0x55c933106030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:27:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.286+0000 7fda337fe700 1 --2- 172.21.15.119:0/282008636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c933101a60 0x55c93310cd10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fda28013b60 tx=0x7fda28014fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43f0c23556fa8e9b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.286+0000 7fda1affd700 1 -- 172.21.15.119:0/282008636 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2802c670 con 0x55c933101a60 2023-12-07T10:27:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.287+0000 7fda1affd700 1 -- 172.21.15.119:0/282008636 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda28026050 con 0x55c933101a60 2023-12-07T10:27:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.287+0000 7fda1affd700 1 -- 172.21.15.119:0/282008636 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda280046b0 con 0x55c933101a60 2023-12-07T10:27:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.287+0000 7fda4e0b9b80 1 -- 172.21.15.119:0/282008636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c933101a60 msgr2=0x55c93310cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.287+0000 7fda4e0b9b80 1 --2- 172.21.15.119:0/282008636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c933101a60 0x55c93310cd10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fda28013b60 tx=0x7fda28014fe0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.287+0000 7fda4e0b9b80 1 -- 172.21.15.119:0/282008636 shutdown_connections 2023-12-07T10:27:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.287+0000 7fda4e0b9b80 1 --2- 172.21.15.119:0/282008636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c933102380 0x55c93310d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.287+0000 7fda4e0b9b80 1 --2- 172.21.15.119:0/282008636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9330fcad0 0x55c933106030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.287+0000 7fda4e0b9b80 1 --2- 172.21.15.119:0/282008636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c933101a60 0x55c93310cd10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.287+0000 7fda4e0b9b80 1 -- 172.21.15.119:0/282008636 >> 172.21.15.119:0/282008636 conn(0x55c93302b300 msgr2=0x55c9330f0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.287+0000 7fda4e0b9b80 1 -- 172.21.15.119:0/282008636 shutdown_connections 2023-12-07T10:27:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.287+0000 7fda4e0b9b80 1 -- 172.21.15.119:0/282008636 wait complete. 2023-12-07T10:27:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.288+0000 7fda4e0b9b80 1 -- start start 2023-12-07T10:27:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.288+0000 7fda4e0b9b80 1 Processor -- start 2023-12-07T10:27:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.289+0000 7fda4e0b9b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9330fcad0 0x55c933160590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.289+0000 7fda4e0b9b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c933101a60 0x55c933160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.289+0000 7fda4e0b9b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c933102380 0x55c93316b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.289+0000 7fda4e0b9b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93316b960 con 0x55c9330fcad0 2023-12-07T10:27:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.289+0000 7fda4e0b9b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93316baa0 con 0x55c933101a60 2023-12-07T10:27:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.289+0000 7fda4e0b9b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93316bbe0 con 0x55c933102380 2023-12-07T10:27:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.289+0000 7fda4e0b9b80 10 client.0 _ll_register_callbacks cb 0x55c932ff18e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:27:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.289+0000 7fda337fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c933101a60 0x55c933160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.289+0000 7fda38d9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c933102380 0x55c93316b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.289+0000 7fda38d9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c933102380 0x55c93316b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:36884/0 (socket says 172.21.15.119:36884) 2023-12-07T10:27:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.289+0000 7fda337fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c933101a60 0x55c933160ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:41978/0 (socket says 172.21.15.119:41978) 2023-12-07T10:27:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.289+0000 7fda38d9c700 1 -- 172.21.15.119:0/3148455740 learned_addr learned my addr 172.21.15.119:0/3148455740 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:27:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.289+0000 7fda33fff700 1 --2- 172.21.15.119:0/3148455740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9330fcad0 0x55c933160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.289+0000 7fda38d9c700 1 -- 172.21.15.119:0/3148455740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c933101a60 msgr2=0x55c933160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.289+0000 7fda38d9c700 1 --2- 172.21.15.119:0/3148455740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c933101a60 0x55c933160ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.289+0000 7fda38d9c700 1 -- 172.21.15.119:0/3148455740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9330fcad0 msgr2=0x55c933160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.289+0000 7fda38d9c700 1 --2- 172.21.15.119:0/3148455740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9330fcad0 0x55c933160590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.289+0000 7fda38d9c700 1 -- 172.21.15.119:0/3148455740 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c932ecd5d0 con 0x55c933102380 2023-12-07T10:27:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.289+0000 7fda337fe700 1 --2- 172.21.15.119:0/3148455740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c933101a60 0x55c933160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:27:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.290+0000 7fda38d9c700 1 --2- 172.21.15.119:0/3148455740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c933102380 0x55c93316b420 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fda2c015000 tx=0x7fda2c00bcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa266e69f9a03d1 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.290+0000 7fda137fe700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:27:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.290+0000 7fda137fe700 1 -- 172.21.15.119:0/3148455740 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c00e040 con 0x55c933102380 2023-12-07T10:27:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.290+0000 7fda4e0b9b80 1 -- 172.21.15.119:0/3148455740 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda2409ae10 con 0x55c933102380 2023-12-07T10:27:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.290+0000 7fda137fe700 1 -- 172.21.15.119:0/3148455740 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9549+0+0 (secure 0 0 0) 0x7fda2c0078f0 con 0x55c933102380 2023-12-07T10:27:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.290+0000 7fda137fe700 1 -- 172.21.15.119:0/3148455740 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda2c00bea0 con 0x55c933102380 2023-12-07T10:27:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.290+0000 7fda4e0b9b80 10 client.27334 fetch_fsmap learned FSMap version 50 2023-12-07T10:27:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.290+0000 7fda4e0b9b80 1 -- 172.21.15.119:0/3148455740 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55c932e79d80 con 0x55c933102380 2023-12-07T10:27:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.290+0000 7fda137fe700 1 -- 172.21.15.119:0/3148455740 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda2409ae10 con 0x55c933102380 2023-12-07T10:27:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.291+0000 7fda137fe700 1 -- 172.21.15.119:0/3148455740 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 50) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fda2c00b100 con 0x55c933102380 2023-12-07T10:27:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.291+0000 7fda4e0b9b80 10 client.27334 fetch_fsmap finished waiting for FSMap version 50 2023-12-07T10:27:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.291+0000 7fda4e0b9b80 10 client.27334 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:27:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.291+0000 7fda4e0b9b80 20 client.27334 populate_metadata read hostname 'smithi119' 2023-12-07T10:27:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.291+0000 7fda4e0b9b80 1 --2- 172.21.15.119:0/3148455740 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c93316d440 0x55c93316f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.291+0000 7fda4e0b9b80 4 client.27334 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:27:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.291+0000 7fda4e0b9b80 1 -- 172.21.15.119:0/3148455740 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c933016f70 con 0x55c93316d440 2023-12-07T10:27:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.291+0000 7fda33fff700 1 --2- 172.21.15.119:0/3148455740 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c93316d440 0x55c93316f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.292+0000 7fda137fe700 10 client.27334 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:27:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.292+0000 7fda33fff700 1 --2- 172.21.15.119:0/3148455740 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c93316d440 0x55c93316f860 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de44e370004f82db server_cookie=faff9a1959c16b78 in_seq=0 out_seq=0 2023-12-07T10:27:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.324+0000 7fda137fe700 1 -- 172.21.15.119:0/3148455740 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c933016f70 con 0x55c93316d440 2023-12-07T10:27:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.324+0000 7fda137fe700 10 client.27334 handle_command_reply: tid=0 2023-12-07T10:27:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.324+0000 7fda4e0b9b80 1 client.27334 shutdown 2023-12-07T10:27:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.325+0000 7fda4e0b9b80 1 -- 172.21.15.119:0/3148455740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c933102380 msgr2=0x55c93316b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.325+0000 7fda4e0b9b80 1 --2- 172.21.15.119:0/3148455740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c933102380 0x55c93316b420 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fda2c015000 tx=0x7fda2c00bcd0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.325+0000 7fda4e0b9b80 1 -- 172.21.15.119:0/3148455740 shutdown_connections 2023-12-07T10:27:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.326+0000 7fda4e0b9b80 1 --2- 172.21.15.119:0/3148455740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c933102380 0x55c93316b420 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.326+0000 7fda4e0b9b80 1 --2- 172.21.15.119:0/3148455740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9330fcad0 0x55c933160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.326+0000 7fda4e0b9b80 1 --2- 172.21.15.119:0/3148455740 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c93316d440 0x55c93316f860 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.326+0000 7fda4e0b9b80 1 --2- 172.21.15.119:0/3148455740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c933101a60 0x55c933160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.326+0000 7fda4e0b9b80 1 -- 172.21.15.119:0/3148455740 >> 172.21.15.119:0/3148455740 conn(0x55c93302b300 msgr2=0x55c933106030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.326+0000 7fda137fe700 0 client.27334 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:27:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.326+0000 7fda4e0b9b80 1 -- 172.21.15.119:0/3148455740 shutdown_connections 2023-12-07T10:27:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.326+0000 7fda4e0b9b80 1 -- 172.21.15.119:0/3148455740 wait complete. 2023-12-07T10:27:05.327 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.326+0000 7fda4e0b9b80 20 client.27334 trim_cache size 0 max 16384 2023-12-07T10:27:05.328 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:27:05.328 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:27:05.328 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:27:05.328 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:27:05.328 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.327+0000 7fda3a7ff700 1 -- 172.21.15.119:0/3762903409 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fda1c0a1d00 msgr2=0x7fda1c0a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:05.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.327+0000 7fda3a7ff700 1 --2- 172.21.15.119:0/3762903409 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fda1c0a1d00 0x7fda1c0a41c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fda34196d60 tx=0x7fda28009260 comp rx=0 tx=0).stop 2023-12-07T10:27:05.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.327+0000 7fda3a7ff700 1 -- 172.21.15.119:0/3762903409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda34101500 msgr2=0x7fda3419b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:05.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.327+0000 7fda3a7ff700 1 --2- 172.21.15.119:0/3762903409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda34101500 0x7fda3419b520 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fda2400bc50 tx=0x7fda2400bf60 comp rx=0 tx=0).stop 2023-12-07T10:27:05.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.327+0000 7fda3a7ff700 1 -- 172.21.15.119:0/3762903409 shutdown_connections 2023-12-07T10:27:05.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.327+0000 7fda3a7ff700 1 --2- 172.21.15.119:0/3762903409 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fda1c0a1d00 0x7fda1c0a41c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.327+0000 7fda3a7ff700 1 --2- 172.21.15.119:0/3762903409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda341075c0 0x7fda341956a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.327+0000 7fda3a7ff700 1 --2- 172.21.15.119:0/3762903409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda34101500 0x7fda3419b520 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.327+0000 7fda3a7ff700 1 --2- 172.21.15.119:0/3762903409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda34101ec0 0x7fda3419ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.327+0000 7fda3a7ff700 1 -- 172.21.15.119:0/3762903409 >> 172.21.15.119:0/3762903409 conn(0x7fda340fcfb0 msgr2=0x7fda340fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.328+0000 7fda3a7ff700 1 -- 172.21.15.119:0/3762903409 shutdown_connections 2023-12-07T10:27:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:05.328+0000 7fda3a7ff700 1 -- 172.21.15.119:0/3762903409 wait complete. 2023-12-07T10:27:05.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:05 smithi119 ceph-mon[97908]: osdmap e95: 12 total, 12 up, 12 in 2023-12-07T10:27:05.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:05 smithi159 ceph-mon[134726]: osdmap e95: 12 total, 12 up, 12 in 2023-12-07T10:27:05.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:05 smithi153 ceph-mon[133625]: osdmap e95: 12 total, 12 up, 12 in 2023-12-07T10:27:05.562 INFO:tasks.workunit.client.0.smithi119.stdout: 1 441428 6.89 MB/sec execute 198 sec latency 27.005 ms 2023-12-07T10:27:06.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:06 smithi119 ceph-mon[97908]: pgmap v204: 129 pgs: 129 active+clean; 250 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2023-12-07T10:27:06.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:06 smithi159 ceph-mon[134726]: pgmap v204: 129 pgs: 129 active+clean; 250 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2023-12-07T10:27:06.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:06 smithi153 ceph-mon[133625]: pgmap v204: 129 pgs: 129 active+clean; 250 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2023-12-07T10:27:06.561 INFO:tasks.workunit.client.0.smithi119.stdout: 1 443021 6.90 MB/sec execute 199 sec latency 26.968 ms 2023-12-07T10:27:07.561 INFO:tasks.workunit.client.0.smithi119.stdout: 1 444088 6.89 MB/sec execute 200 sec latency 32.445 ms 2023-12-07T10:27:08.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:08 smithi119 ceph-mon[97908]: pgmap v205: 129 pgs: 129 active+clean; 252 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 191 op/s 2023-12-07T10:27:08.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:08 smithi159 ceph-mon[134726]: pgmap v205: 129 pgs: 129 active+clean; 252 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 191 op/s 2023-12-07T10:27:08.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:08 smithi153 ceph-mon[133625]: pgmap v205: 129 pgs: 129 active+clean; 252 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 191 op/s 2023-12-07T10:27:08.561 INFO:tasks.workunit.client.0.smithi119.stdout: 1 445496 6.89 MB/sec execute 201 sec latency 16.580 ms 2023-12-07T10:27:09.561 INFO:tasks.workunit.client.0.smithi119.stdout: 1 447230 6.92 MB/sec execute 202 sec latency 11.245 ms 2023-12-07T10:27:10.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:10 smithi119 ceph-mon[97908]: pgmap v206: 129 pgs: 129 active+clean; 252 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 191 op/s 2023-12-07T10:27:10.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:10 smithi159 ceph-mon[134726]: pgmap v206: 129 pgs: 129 active+clean; 252 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 191 op/s 2023-12-07T10:27:10.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:10 smithi153 ceph-mon[133625]: pgmap v206: 129 pgs: 129 active+clean; 252 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 191 op/s 2023-12-07T10:27:10.561 INFO:tasks.workunit.client.0.smithi119.stdout: 1 448331 6.91 MB/sec execute 203 sec latency 18.316 ms 2023-12-07T10:27:11.561 INFO:tasks.workunit.client.0.smithi119.stdout: 1 450310 6.92 MB/sec execute 204 sec latency 20.643 ms 2023-12-07T10:27:12.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:11 smithi119 ceph-mon[97908]: osdmap e96: 12 total, 12 up, 12 in 2023-12-07T10:27:12.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:11 smithi119 ceph-mon[97908]: pgmap v208: 129 pgs: 129 active+clean; 257 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 221 op/s 2023-12-07T10:27:12.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:11 smithi159 ceph-mon[134726]: osdmap e96: 12 total, 12 up, 12 in 2023-12-07T10:27:12.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:11 smithi159 ceph-mon[134726]: pgmap v208: 129 pgs: 129 active+clean; 257 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 221 op/s 2023-12-07T10:27:12.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:11 smithi153 ceph-mon[133625]: osdmap e96: 12 total, 12 up, 12 in 2023-12-07T10:27:12.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:11 smithi153 ceph-mon[133625]: pgmap v208: 129 pgs: 129 active+clean; 257 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 221 op/s 2023-12-07T10:27:12.562 INFO:tasks.workunit.client.0.smithi119.stdout: 1 451403 6.91 MB/sec execute 205 sec latency 12.852 ms 2023-12-07T10:27:13.562 INFO:tasks.workunit.client.0.smithi119.stdout: 1 452985 6.91 MB/sec execute 206 sec latency 21.637 ms 2023-12-07T10:27:14.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:14 smithi119 ceph-mon[97908]: pgmap v209: 129 pgs: 129 active+clean; 259 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.6 MiB/s wr, 227 op/s 2023-12-07T10:27:14.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:14 smithi159 ceph-mon[134726]: pgmap v209: 129 pgs: 129 active+clean; 259 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.6 MiB/s wr, 227 op/s 2023-12-07T10:27:14.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:14 smithi153 ceph-mon[133625]: pgmap v209: 129 pgs: 129 active+clean; 259 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.6 MiB/s wr, 227 op/s 2023-12-07T10:27:14.562 INFO:tasks.workunit.client.0.smithi119.stdout: 1 454485 6.94 MB/sec execute 207 sec latency 13.158 ms 2023-12-07T10:27:15.562 INFO:tasks.workunit.client.0.smithi119.stdout: 1 455733 6.93 MB/sec execute 208 sec latency 43.784 ms 2023-12-07T10:27:16.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:16 smithi119 ceph-mon[97908]: pgmap v210: 129 pgs: 129 active+clean; 259 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 203 op/s 2023-12-07T10:27:16.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:16 smithi159 ceph-mon[134726]: pgmap v210: 129 pgs: 129 active+clean; 259 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 203 op/s 2023-12-07T10:27:16.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:16 smithi153 ceph-mon[133625]: pgmap v210: 129 pgs: 129 active+clean; 259 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 203 op/s 2023-12-07T10:27:16.562 INFO:tasks.workunit.client.0.smithi119.stdout: 1 457797 6.96 MB/sec execute 209 sec latency 18.807 ms 2023-12-07T10:27:17.562 INFO:tasks.workunit.client.0.smithi119.stdout: 1 458345 6.93 MB/sec execute 210 sec latency 42.182 ms 2023-12-07T10:27:18.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:18 smithi119 ceph-mon[97908]: pgmap v211: 129 pgs: 129 active+clean; 266 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 208 op/s 2023-12-07T10:27:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.491+0000 7f0efabff700 1 -- 172.21.15.119:0/1234003111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef4101ec0 msgr2=0x7f0ef410e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.491+0000 7f0efabff700 1 --2- 172.21.15.119:0/1234003111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef4101ec0 0x7f0ef410e380 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00b0a0 tx=0x7f0eec00b3b0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.491+0000 7f0efabff700 1 -- 172.21.15.119:0/1234003111 shutdown_connections 2023-12-07T10:27:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.491+0000 7f0efabff700 1 --2- 172.21.15.119:0/1234003111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef4101ec0 0x7f0ef410e380 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.491+0000 7f0efabff700 1 --2- 172.21.15.119:0/1234003111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ef41075c0 0x7f0ef41079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.491+0000 7f0efabff700 1 --2- 172.21.15.119:0/1234003111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ef4101500 0x7f0ef4101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.491+0000 7f0efabff700 1 -- 172.21.15.119:0/1234003111 >> 172.21.15.119:0/1234003111 conn(0x7f0ef40fcfb0 msgr2=0x7f0ef40ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.491+0000 7f0efabff700 1 -- 172.21.15.119:0/1234003111 shutdown_connections 2023-12-07T10:27:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.491+0000 7f0efabff700 1 -- 172.21.15.119:0/1234003111 wait complete. 2023-12-07T10:27:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.493+0000 7f0efabff700 1 Processor -- start 2023-12-07T10:27:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.493+0000 7f0efabff700 1 -- start start 2023-12-07T10:27:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.493+0000 7f0efabff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ef4101500 0x7f0ef4073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.493+0000 7f0efabff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ef4101ec0 0x7f0ef4073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.493+0000 7f0efabff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef41075c0 0x7f0ef4074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.494+0000 7f0efabff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef41a5db0 con 0x7f0ef4101ec0 2023-12-07T10:27:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.494+0000 7f0efabff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef41a5f20 con 0x7f0ef4101500 2023-12-07T10:27:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.494+0000 7f0efabff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef41a60e0 con 0x7f0ef41075c0 2023-12-07T10:27:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.494+0000 7f0ef899b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ef4101500 0x7f0ef4073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.494+0000 7f0ef899b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ef4101500 0x7f0ef4073910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:59450/0 (socket says 172.21.15.119:59450) 2023-12-07T10:27:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.494+0000 7f0ef899b700 1 -- 172.21.15.119:0/1596002160 learned_addr learned my addr 172.21.15.119:0/1596002160 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:27:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.494+0000 7f0ef3fff700 1 --2- 172.21.15.119:0/1596002160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ef4101ec0 0x7f0ef4073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.494+0000 7f0ef919c700 1 --2- 172.21.15.119:0/1596002160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef41075c0 0x7f0ef4074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.494+0000 7f0ef899b700 1 -- 172.21.15.119:0/1596002160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef41075c0 msgr2=0x7f0ef4074390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.494+0000 7f0ef899b700 1 --2- 172.21.15.119:0/1596002160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef41075c0 0x7f0ef4074390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.494+0000 7f0ef899b700 1 -- 172.21.15.119:0/1596002160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ef4101ec0 msgr2=0x7f0ef4073e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.494+0000 7f0ef899b700 1 --2- 172.21.15.119:0/1596002160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ef4101ec0 0x7f0ef4073e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.494+0000 7f0ef899b700 1 -- 172.21.15.119:0/1596002160 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eec009d00 con 0x7f0ef4101500 2023-12-07T10:27:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.494+0000 7f0ef899b700 1 --2- 172.21.15.119:0/1596002160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ef4101500 0x7f0ef4073910 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0ee800d980 tx=0x7f0ee800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.495+0000 7f0ef1ffb700 1 -- 172.21.15.119:0/1596002160 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8004220 con 0x7f0ef4101500 2023-12-07T10:27:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.495+0000 7f0ef1ffb700 1 -- 172.21.15.119:0/1596002160 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ee8007640 con 0x7f0ef4101500 2023-12-07T10:27:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.495+0000 7f0efabff700 1 -- 172.21.15.119:0/1596002160 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef41a63c0 con 0x7f0ef4101500 2023-12-07T10:27:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.495+0000 7f0ef1ffb700 1 -- 172.21.15.119:0/1596002160 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8005690 con 0x7f0ef4101500 2023-12-07T10:27:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.495+0000 7f0efabff700 1 -- 172.21.15.119:0/1596002160 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef41a68d0 con 0x7f0ef4101500 2023-12-07T10:27:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.498+0000 7f0ef1ffb700 1 -- 172.21.15.119:0/1596002160 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f0ee8004750 con 0x7f0ef4101500 2023-12-07T10:27:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.500+0000 7f0f0e502b80 1 Processor -- start 2023-12-07T10:27:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.500+0000 7f0f0e502b80 1 -- start start 2023-12-07T10:27:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.500+0000 7f0ef1ffb700 1 --2- 172.21.15.119:0/1596002160 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0ee40a1c80 0x7f0ee40a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.500+0000 7f0f0e502b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ad4323a60 0x556ad4323e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.500+0000 7f0f0e502b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ad4324380 0x556ad431e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.500+0000 7f0f0e502b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ad431ead0 0x556ad431ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.501+0000 7f0f0e502b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ad4292760 con 0x556ad4324380 2023-12-07T10:27:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.501+0000 7f0f0e502b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ad4279740 con 0x556ad431ead0 2023-12-07T10:27:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.501+0000 7f0f0e502b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ad4277060 con 0x556ad4323a60 2023-12-07T10:27:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.501+0000 7f0ef1ffb700 1 -- 172.21.15.119:0/1596002160 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0ee80cdb20 con 0x7f0ef4101500 2023-12-07T10:27:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.501+0000 7f0ef899b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ad4323a60 0x556ad4323e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.501+0000 7f0ef919c700 1 --2- 172.21.15.119:0/1596002160 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0ee40a1c80 0x7f0ee40a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.501+0000 7f0ef899b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ad4323a60 0x556ad4323e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:37666/0 (socket says 172.21.15.119:37666) 2023-12-07T10:27:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.501+0000 7f0ef899b700 1 -- 172.21.15.119:0/1096929000 learned_addr learned my addr 172.21.15.119:0/1096929000 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:27:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.501+0000 7f0ef3fff700 1 --2- 172.21.15.119:0/1096929000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ad4324380 0x556ad431e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.501+0000 7f0ef919c700 1 --2- 172.21.15.119:0/1096929000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ad431ead0 0x556ad431ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.501+0000 7f0ef899b700 1 -- 172.21.15.119:0/1096929000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ad431ead0 msgr2=0x556ad431ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.501+0000 7f0ef899b700 1 --2- 172.21.15.119:0/1096929000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ad431ead0 0x556ad431ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.501+0000 7f0ef899b700 1 -- 172.21.15.119:0/1096929000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ad4324380 msgr2=0x556ad431e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.501+0000 7f0ef899b700 1 --2- 172.21.15.119:0/1096929000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ad4324380 0x556ad431e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.501+0000 7f0ef899b700 1 -- 172.21.15.119:0/1096929000 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ad40ef5d0 con 0x556ad4323a60 2023-12-07T10:27:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.501+0000 7f0ef3fff700 1 --2- 172.21.15.119:0/1096929000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ad4324380 0x556ad431e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:27:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.501+0000 7f0ef899b700 1 --2- 172.21.15.119:0/1096929000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ad4323a60 0x556ad4323e40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0ee80969b0 tx=0x7f0ee8099e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc9249641ddb174b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:18.507 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.502+0000 7f0ef919c700 1 --2- 172.21.15.119:0/1596002160 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0ee40a1c80 0x7f0ee40a4140 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x556ad431a670 tx=0x7f0eec009360 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:18.507 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.502+0000 7f0edeffd700 1 -- 172.21.15.119:0/1096929000 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee80690e0 con 0x556ad4323a60 2023-12-07T10:27:18.507 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.502+0000 7f0edeffd700 1 -- 172.21.15.119:0/1096929000 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ee8066940 con 0x556ad4323a60 2023-12-07T10:27:18.507 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.502+0000 7f0edeffd700 1 -- 172.21.15.119:0/1096929000 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8004380 con 0x556ad4323a60 2023-12-07T10:27:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.503+0000 7f0f0e502b80 1 -- 172.21.15.119:0/1096929000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ad4323a60 msgr2=0x556ad4323e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.503+0000 7f0f0e502b80 1 --2- 172.21.15.119:0/1096929000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ad4323a60 0x556ad4323e40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0ee80969b0 tx=0x7f0ee8099e10 comp rx=0 tx=0).stop 2023-12-07T10:27:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.503+0000 7f0f0e502b80 1 -- 172.21.15.119:0/1096929000 shutdown_connections 2023-12-07T10:27:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.503+0000 7f0f0e502b80 1 --2- 172.21.15.119:0/1096929000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ad431ead0 0x556ad431ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.503+0000 7f0f0e502b80 1 --2- 172.21.15.119:0/1096929000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ad4323a60 0x556ad4323e40 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.503+0000 7f0f0e502b80 1 --2- 172.21.15.119:0/1096929000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ad4324380 0x556ad431e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.503+0000 7f0f0e502b80 1 -- 172.21.15.119:0/1096929000 >> 172.21.15.119:0/1096929000 conn(0x556ad424d300 msgr2=0x556ad4318b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.503+0000 7f0f0e502b80 1 -- 172.21.15.119:0/1096929000 shutdown_connections 2023-12-07T10:27:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.503+0000 7f0f0e502b80 1 -- 172.21.15.119:0/1096929000 wait complete. 2023-12-07T10:27:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.504+0000 7f0f0e502b80 1 -- start start 2023-12-07T10:27:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.504+0000 7f0f0e502b80 1 Processor -- start 2023-12-07T10:27:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.505+0000 7f0f0e502b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ad431ead0 0x556ad4327720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.505+0000 7f0f0e502b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ad4323a60 0x556ad4327c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.505+0000 7f0f0e502b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ad4324380 0x556ad43281a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.505+0000 7f0f0e502b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ad4272950 con 0x556ad4324380 2023-12-07T10:27:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.505+0000 7f0f0e502b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ad422c500 con 0x556ad431ead0 2023-12-07T10:27:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.505+0000 7f0f0e502b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ad422c1a0 con 0x556ad4323a60 2023-12-07T10:27:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.505+0000 7f0f0e502b80 10 client.0 _ll_register_callbacks cb 0x556ad42138e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:27:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.505+0000 7f0ef899b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ad431ead0 0x556ad4327720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.506+0000 7f0ef899b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ad431ead0 0x556ad4327720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:59470/0 (socket says 172.21.15.119:59470) 2023-12-07T10:27:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.506+0000 7f0ef899b700 1 -- 172.21.15.119:0/370200919 learned_addr learned my addr 172.21.15.119:0/370200919 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:27:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.506+0000 7f0ef919c700 1 --2- 172.21.15.119:0/370200919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ad4324380 0x556ad43281a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.506+0000 7f0ef3fff700 1 --2- 172.21.15.119:0/370200919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ad4323a60 0x556ad4327c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.506+0000 7f0ef899b700 1 -- 172.21.15.119:0/370200919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ad4323a60 msgr2=0x556ad4327c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.506+0000 7f0ef899b700 1 --2- 172.21.15.119:0/370200919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ad4323a60 0x556ad4327c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.506+0000 7f0ef899b700 1 -- 172.21.15.119:0/370200919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ad4324380 msgr2=0x556ad43281a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.506+0000 7f0ef899b700 1 --2- 172.21.15.119:0/370200919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ad4324380 0x556ad43281a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.506+0000 7f0ef899b700 1 -- 172.21.15.119:0/370200919 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef41a68d0 con 0x556ad431ead0 2023-12-07T10:27:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.506+0000 7f0ef899b700 1 --2- 172.21.15.119:0/370200919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ad431ead0 0x556ad4327720 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8000c00 tx=0x7f0ee809a4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9792015406e4c54c server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.506+0000 7f0ed3fff700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:27:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.506+0000 7f0ed3fff700 1 -- 172.21.15.119:0/370200919 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee80cb0e0 con 0x556ad431ead0 2023-12-07T10:27:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.507+0000 7f0f0e502b80 1 -- 172.21.15.119:0/370200919 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ad43116f0 con 0x556ad431ead0 2023-12-07T10:27:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.507+0000 7f0ed3fff700 1 -- 172.21.15.119:0/370200919 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0ee80cb240 con 0x556ad431ead0 2023-12-07T10:27:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.507+0000 7f0ed3fff700 1 -- 172.21.15.119:0/370200919 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ee809a710 con 0x556ad431ead0 2023-12-07T10:27:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.507+0000 7f0f0e502b80 10 client.27362 fetch_fsmap learned FSMap version 50 2023-12-07T10:27:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.507+0000 7f0f0e502b80 1 -- 172.21.15.119:0/370200919 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x556ad409bd80 con 0x556ad431ead0 2023-12-07T10:27:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.507+0000 7f0ed3fff700 1 -- 172.21.15.119:0/370200919 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556ad43116f0 con 0x556ad431ead0 2023-12-07T10:27:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.507+0000 7f0ed3fff700 1 -- 172.21.15.119:0/370200919 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 50) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f0ee80a23b0 con 0x556ad431ead0 2023-12-07T10:27:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.507+0000 7f0f0e502b80 10 client.27362 fetch_fsmap finished waiting for FSMap version 50 2023-12-07T10:27:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.507+0000 7f0f0e502b80 10 client.27362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:27:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.507+0000 7f0f0e502b80 20 client.27362 populate_metadata read hostname 'smithi119' 2023-12-07T10:27:18.514 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.507+0000 7f0f0e502b80 1 --2- 172.21.15.119:0/370200919 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556ad4337a90 0x556ad4339eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:18.514 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.507+0000 7f0f0e502b80 4 client.27362 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:27:18.514 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.507+0000 7f0f0e502b80 1 -- 172.21.15.119:0/370200919 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ad42ed740 con 0x556ad4337a90 2023-12-07T10:27:18.514 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.508+0000 7f0ef3fff700 1 --2- 172.21.15.119:0/370200919 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556ad4337a90 0x556ad4339eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:18.514 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.508+0000 7f0ed3fff700 10 client.27362 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:27:18.514 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.508+0000 7f0ef3fff700 1 --2- 172.21.15.119:0/370200919 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556ad4337a90 0x556ad4339eb0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=930cd13a90c7ccca server_cookie=8807583a9447c77c in_seq=0 out_seq=0 2023-12-07T10:27:18.514 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.509+0000 7f0ed3fff700 1 -- 172.21.15.119:0/370200919 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556ad42ed740 con 0x556ad4337a90 2023-12-07T10:27:18.514 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.509+0000 7f0ed3fff700 10 client.27362 handle_command_reply: tid=0 2023-12-07T10:27:18.514 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.509+0000 7f0f0e502b80 1 client.27362 shutdown 2023-12-07T10:27:18.514 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.509+0000 7f0f0e502b80 1 -- 172.21.15.119:0/370200919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ad431ead0 msgr2=0x556ad4327720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:18.514 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.509+0000 7f0f0e502b80 1 --2- 172.21.15.119:0/370200919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ad431ead0 0x556ad4327720 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8000c00 tx=0x7f0ee809a4b0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.514 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.509+0000 7f0f0e502b80 1 -- 172.21.15.119:0/370200919 shutdown_connections 2023-12-07T10:27:18.514 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.509+0000 7f0f0e502b80 1 --2- 172.21.15.119:0/370200919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ad4324380 0x556ad43281a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.509+0000 7f0f0e502b80 1 --2- 172.21.15.119:0/370200919 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556ad4337a90 0x556ad4339eb0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.509+0000 7f0f0e502b80 1 --2- 172.21.15.119:0/370200919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ad431ead0 0x556ad4327720 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.509+0000 7f0f0e502b80 1 --2- 172.21.15.119:0/370200919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ad4323a60 0x556ad4327c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.509+0000 7f0f0e502b80 1 -- 172.21.15.119:0/370200919 >> 172.21.15.119:0/370200919 conn(0x556ad424d300 msgr2=0x556ad4313980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.509+0000 7f0ed3fff700 0 client.27362 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:27:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.510+0000 7f0f0e502b80 1 -- 172.21.15.119:0/370200919 shutdown_connections 2023-12-07T10:27:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.510+0000 7f0f0e502b80 1 -- 172.21.15.119:0/370200919 wait complete. 2023-12-07T10:27:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.510+0000 7f0f0e502b80 20 client.27362 trim_cache size 0 max 16384 2023-12-07T10:27:18.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:18 smithi159 ceph-mon[134726]: pgmap v211: 129 pgs: 129 active+clean; 266 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 208 op/s 2023-12-07T10:27:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.526+0000 7f0f0e502b80 1 Processor -- start 2023-12-07T10:27:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.527+0000 7f0f0e502b80 1 -- start start 2023-12-07T10:27:18.528 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.527+0000 7f0f0e502b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ad431ead0 0x556ad431e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:18.528 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.527+0000 7f0f0e502b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ad4323a60 0x556ad4322910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:18.528 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.527+0000 7f0f0e502b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ad4324380 0x556ad4322e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:18.528 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.527+0000 7f0f0e502b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ad40f18c0 con 0x556ad4324380 2023-12-07T10:27:18.528 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.527+0000 7f0f0e502b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ad43116c0 con 0x556ad4323a60 2023-12-07T10:27:18.528 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.527+0000 7f0f0e502b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ad43157a0 con 0x556ad431ead0 2023-12-07T10:27:18.528 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.527+0000 7f0ef3fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ad4323a60 0x556ad4322910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:18.528 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.527+0000 7f0ef899b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ad431ead0 0x556ad431e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:18.528 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.527+0000 7f0ef3fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ad4323a60 0x556ad4322910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:59480/0 (socket says 172.21.15.119:59480) 2023-12-07T10:27:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.527+0000 7f0ef899b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ad431ead0 0x556ad431e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:37686/0 (socket says 172.21.15.119:37686) 2023-12-07T10:27:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.527+0000 7f0ef3fff700 1 -- 172.21.15.119:0/2661247736 learned_addr learned my addr 172.21.15.119:0/2661247736 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:27:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.527+0000 7f0ef919c700 1 --2- 172.21.15.119:0/2661247736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ad4324380 0x556ad4322e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.528+0000 7f0ef3fff700 1 -- 172.21.15.119:0/2661247736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ad431ead0 msgr2=0x556ad431e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.528+0000 7f0ef3fff700 1 --2- 172.21.15.119:0/2661247736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ad431ead0 0x556ad431e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.528+0000 7f0ef3fff700 1 -- 172.21.15.119:0/2661247736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ad4324380 msgr2=0x556ad4322e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.528+0000 7f0ef3fff700 1 --2- 172.21.15.119:0/2661247736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ad4324380 0x556ad4322e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.528+0000 7f0ef3fff700 1 -- 172.21.15.119:0/2661247736 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ad4084c00 con 0x556ad4323a60 2023-12-07T10:27:18.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:18 smithi153 ceph-mon[133625]: pgmap v211: 129 pgs: 129 active+clean; 266 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 208 op/s 2023-12-07T10:27:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.528+0000 7f0ef3fff700 1 --2- 172.21.15.119:0/2661247736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ad4323a60 0x556ad4322910 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x556ad431a890 tx=0x7f0ee0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d54a3f8525fad6b7 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.528+0000 7f0edf7fe700 1 -- 172.21.15.119:0/2661247736 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0017030 con 0x556ad4323a60 2023-12-07T10:27:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.528+0000 7f0edf7fe700 1 -- 172.21.15.119:0/2661247736 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ee0018040 con 0x556ad4323a60 2023-12-07T10:27:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.528+0000 7f0edf7fe700 1 -- 172.21.15.119:0/2661247736 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0003e80 con 0x556ad4323a60 2023-12-07T10:27:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.529+0000 7f0f0e502b80 1 -- 172.21.15.119:0/2661247736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ad4323a60 msgr2=0x556ad4322910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.529+0000 7f0f0e502b80 1 --2- 172.21.15.119:0/2661247736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ad4323a60 0x556ad4322910 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x556ad431a890 tx=0x7f0ee0009260 comp rx=0 tx=0).stop 2023-12-07T10:27:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.529+0000 7f0f0e502b80 1 -- 172.21.15.119:0/2661247736 shutdown_connections 2023-12-07T10:27:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.529+0000 7f0f0e502b80 1 --2- 172.21.15.119:0/2661247736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ad4324380 0x556ad4322e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.529+0000 7f0f0e502b80 1 --2- 172.21.15.119:0/2661247736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ad431ead0 0x556ad431e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.529+0000 7f0f0e502b80 1 --2- 172.21.15.119:0/2661247736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ad4323a60 0x556ad4322910 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.529+0000 7f0f0e502b80 1 -- 172.21.15.119:0/2661247736 >> 172.21.15.119:0/2661247736 conn(0x556ad424d300 msgr2=0x556ad4328040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.529+0000 7f0f0e502b80 1 -- 172.21.15.119:0/2661247736 shutdown_connections 2023-12-07T10:27:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.529+0000 7f0f0e502b80 1 -- 172.21.15.119:0/2661247736 wait complete. 2023-12-07T10:27:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.530+0000 7f0f0e502b80 1 -- start start 2023-12-07T10:27:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.530+0000 7f0f0e502b80 1 Processor -- start 2023-12-07T10:27:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.531+0000 7f0f0e502b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ad431ead0 0x556ad432cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.531+0000 7f0f0e502b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ad4323a60 0x556ad432d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.531+0000 7f0f0e502b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ad4324380 0x556ad43957a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.531+0000 7f0f0e502b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ad432da00 con 0x556ad431ead0 2023-12-07T10:27:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.531+0000 7f0f0e502b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ad4395ce0 con 0x556ad4324380 2023-12-07T10:27:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.531+0000 7f0f0e502b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ad4395e20 con 0x556ad4323a60 2023-12-07T10:27:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.531+0000 7f0f0e502b80 10 client.0 _ll_register_callbacks cb 0x556ad42138e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:27:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.531+0000 7f0ef919c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ad4324380 0x556ad43957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.531+0000 7f0ef899b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ad431ead0 0x556ad432cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.531+0000 7f0ef919c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ad4324380 0x556ad43957a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:59488/0 (socket says 172.21.15.119:59488) 2023-12-07T10:27:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.531+0000 7f0ef919c700 1 -- 172.21.15.119:0/830219923 learned_addr learned my addr 172.21.15.119:0/830219923 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:27:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.531+0000 7f0ef3fff700 1 --2- 172.21.15.119:0/830219923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ad4323a60 0x556ad432d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.532+0000 7f0ef919c700 1 -- 172.21.15.119:0/830219923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ad4323a60 msgr2=0x556ad432d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.532+0000 7f0ef919c700 1 --2- 172.21.15.119:0/830219923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ad4323a60 0x556ad432d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.532+0000 7f0ef919c700 1 -- 172.21.15.119:0/830219923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ad431ead0 msgr2=0x556ad432cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.532+0000 7f0ef919c700 1 --2- 172.21.15.119:0/830219923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ad431ead0 0x556ad432cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.532+0000 7f0ef919c700 1 -- 172.21.15.119:0/830219923 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee0009a80 con 0x556ad4324380 2023-12-07T10:27:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.532+0000 7f0ef919c700 1 --2- 172.21.15.119:0/830219923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ad4324380 0x556ad43957a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00b0a0 tx=0x7f0eec0099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d20657b5f4de49 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.532+0000 7f0ed3fff700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:27:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.532+0000 7f0ed3fff700 1 -- 172.21.15.119:0/830219923 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec00e040 con 0x556ad4324380 2023-12-07T10:27:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.532+0000 7f0f0e502b80 1 -- 172.21.15.119:0/830219923 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ee8066940 con 0x556ad4324380 2023-12-07T10:27:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.532+0000 7f0ed3fff700 1 -- 172.21.15.119:0/830219923 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0eec005040 con 0x556ad4324380 2023-12-07T10:27:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.533+0000 7f0ed3fff700 1 -- 172.21.15.119:0/830219923 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0eec0079c0 con 0x556ad4324380 2023-12-07T10:27:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.533+0000 7f0f0e502b80 10 client.27368 fetch_fsmap learned FSMap version 50 2023-12-07T10:27:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.533+0000 7f0f0e502b80 1 -- 172.21.15.119:0/830219923 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x556ad40c21f0 con 0x556ad4324380 2023-12-07T10:27:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.533+0000 7f0ed3fff700 1 -- 172.21.15.119:0/830219923 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0eec037020 con 0x556ad4324380 2023-12-07T10:27:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.533+0000 7f0ed3fff700 1 -- 172.21.15.119:0/830219923 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 50) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f0eec01e580 con 0x556ad4324380 2023-12-07T10:27:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.533+0000 7f0f0e502b80 10 client.27368 fetch_fsmap finished waiting for FSMap version 50 2023-12-07T10:27:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.533+0000 7f0f0e502b80 10 client.27368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:27:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.533+0000 7f0f0e502b80 20 client.27368 populate_metadata read hostname 'smithi119' 2023-12-07T10:27:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.533+0000 7f0f0e502b80 1 --2- 172.21.15.119:0/830219923 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556ad4397750 0x556ad4399b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.533+0000 7f0f0e502b80 4 client.27368 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:27:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.533+0000 7f0f0e502b80 1 -- 172.21.15.119:0/830219923 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556ad4238f70 con 0x556ad4397750 2023-12-07T10:27:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.533+0000 7f0ef899b700 1 --2- 172.21.15.119:0/830219923 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556ad4397750 0x556ad4399b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.534+0000 7f0ed3fff700 10 client.27368 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:27:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.534+0000 7f0ef899b700 1 --2- 172.21.15.119:0/830219923 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556ad4397750 0x556ad4399b70 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa729d898bc2b947 server_cookie=f89e28e64ca99749 in_seq=0 out_seq=0 2023-12-07T10:27:18.562 INFO:tasks.workunit.client.0.smithi119.stdout: 1 459558 7.01 MB/sec execute 211 sec latency 59.529 ms 2023-12-07T10:27:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.581+0000 7f0ed3fff700 1 -- 172.21.15.119:0/830219923 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556ad4238f70 con 0x556ad4397750 2023-12-07T10:27:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.581+0000 7f0ed3fff700 10 client.27368 handle_command_reply: tid=0 2023-12-07T10:27:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.581+0000 7f0f0e502b80 1 client.27368 shutdown 2023-12-07T10:27:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.581+0000 7f0f0e502b80 1 -- 172.21.15.119:0/830219923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ad4324380 msgr2=0x556ad43957a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.581+0000 7f0f0e502b80 1 --2- 172.21.15.119:0/830219923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ad4324380 0x556ad43957a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00b0a0 tx=0x7f0eec0099e0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.581+0000 7f0f0e502b80 1 -- 172.21.15.119:0/830219923 shutdown_connections 2023-12-07T10:27:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.581+0000 7f0f0e502b80 1 --2- 172.21.15.119:0/830219923 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556ad4397750 0x556ad4399b70 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.581+0000 7f0f0e502b80 1 --2- 172.21.15.119:0/830219923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556ad4324380 0x556ad43957a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.581+0000 7f0f0e502b80 1 --2- 172.21.15.119:0/830219923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556ad431ead0 0x556ad432cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.581+0000 7f0f0e502b80 1 --2- 172.21.15.119:0/830219923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ad4323a60 0x556ad432d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.581+0000 7f0ed3fff700 0 client.27368 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:27:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.581+0000 7f0f0e502b80 1 -- 172.21.15.119:0/830219923 >> 172.21.15.119:0/830219923 conn(0x556ad424d300 msgr2=0x556ad4328040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.582+0000 7f0f0e502b80 1 -- 172.21.15.119:0/830219923 shutdown_connections 2023-12-07T10:27:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.582+0000 7f0f0e502b80 1 -- 172.21.15.119:0/830219923 wait complete. 2023-12-07T10:27:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.582+0000 7f0f0e502b80 20 client.27368 trim_cache size 0 max 16384 2023-12-07T10:27:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:27:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:27:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:27:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:27:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.582+0000 7f0efabff700 1 -- 172.21.15.119:0/1596002160 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0ee40a1c80 msgr2=0x7f0ee40a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.582+0000 7f0efabff700 1 --2- 172.21.15.119:0/1596002160 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0ee40a1c80 0x7f0ee40a4140 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x556ad431a670 tx=0x7f0eec009360 comp rx=0 tx=0).stop 2023-12-07T10:27:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.582+0000 7f0efabff700 1 -- 172.21.15.119:0/1596002160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ef4101500 msgr2=0x7f0ef4073910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.582+0000 7f0efabff700 1 --2- 172.21.15.119:0/1596002160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ef4101500 0x7f0ef4073910 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0ee800d980 tx=0x7f0ee800dc90 comp rx=0 tx=0).stop 2023-12-07T10:27:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.583+0000 7f0efabff700 1 -- 172.21.15.119:0/1596002160 shutdown_connections 2023-12-07T10:27:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.583+0000 7f0efabff700 1 --2- 172.21.15.119:0/1596002160 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0ee40a1c80 0x7f0ee40a4140 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.583+0000 7f0efabff700 1 --2- 172.21.15.119:0/1596002160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef41075c0 0x7f0ef4074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.583+0000 7f0efabff700 1 --2- 172.21.15.119:0/1596002160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ef4101500 0x7f0ef4073910 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.583+0000 7f0efabff700 1 --2- 172.21.15.119:0/1596002160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ef4101ec0 0x7f0ef4073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.583+0000 7f0efabff700 1 -- 172.21.15.119:0/1596002160 >> 172.21.15.119:0/1596002160 conn(0x7f0ef40fcfb0 msgr2=0x7f0ef40fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.583+0000 7f0efabff700 1 -- 172.21.15.119:0/1596002160 shutdown_connections 2023-12-07T10:27:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:18.583+0000 7f0efabff700 1 -- 172.21.15.119:0/1596002160 wait complete. 2023-12-07T10:27:19.562 INFO:tasks.workunit.client.0.smithi119.stdout: 1 461695 7.03 MB/sec execute 212 sec latency 40.340 ms 2023-12-07T10:27:20.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:20 smithi119 ceph-mon[97908]: pgmap v212: 129 pgs: 129 active+clean; 266 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 208 op/s 2023-12-07T10:27:20.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:20 smithi159 ceph-mon[134726]: pgmap v212: 129 pgs: 129 active+clean; 266 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 208 op/s 2023-12-07T10:27:20.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:20 smithi153 ceph-mon[133625]: pgmap v212: 129 pgs: 129 active+clean; 266 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 208 op/s 2023-12-07T10:27:20.562 INFO:tasks.workunit.client.0.smithi119.stdout: 1 462777 7.02 MB/sec execute 213 sec latency 17.864 ms 2023-12-07T10:27:21.562 INFO:tasks.workunit.client.0.smithi119.stdout: 1 464668 7.04 MB/sec execute 214 sec latency 20.705 ms 2023-12-07T10:27:22.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:21 smithi119 ceph-mon[97908]: pgmap v213: 129 pgs: 129 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 309 op/s 2023-12-07T10:27:22.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:21 smithi159 ceph-mon[134726]: pgmap v213: 129 pgs: 129 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 309 op/s 2023-12-07T10:27:22.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:21 smithi153 ceph-mon[133625]: pgmap v213: 129 pgs: 129 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 309 op/s 2023-12-07T10:27:22.562 INFO:tasks.workunit.client.0.smithi119.stdout: 1 465578 7.04 MB/sec execute 215 sec latency 172.288 ms 2023-12-07T10:27:23.562 INFO:tasks.workunit.client.0.smithi119.stdout: 1 466051 7.01 MB/sec execute 216 sec latency 283.264 ms 2023-12-07T10:27:24.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:24 smithi119 ceph-mon[97908]: pgmap v214: 129 pgs: 129 active+clean; 273 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2023-12-07T10:27:24.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:24 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:27:24.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:24 smithi159 ceph-mon[134726]: pgmap v214: 129 pgs: 129 active+clean; 273 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2023-12-07T10:27:24.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:24 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:27:24.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:24 smithi153 ceph-mon[133625]: pgmap v214: 129 pgs: 129 active+clean; 273 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2023-12-07T10:27:24.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:24 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:27:24.562 INFO:tasks.workunit.client.0.smithi119.stdout: 1 466051 6.98 MB/sec execute 217 sec latency 1283.414 ms 2023-12-07T10:27:25.562 INFO:tasks.workunit.client.0.smithi119.stdout: 1 466051 6.95 MB/sec execute 218 sec latency 2283.543 ms 2023-12-07T10:27:26.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:26 smithi119 ceph-mon[97908]: pgmap v215: 129 pgs: 129 active+clean; 273 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 196 op/s 2023-12-07T10:27:26.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:26 smithi159 ceph-mon[134726]: pgmap v215: 129 pgs: 129 active+clean; 273 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 196 op/s 2023-12-07T10:27:26.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:26 smithi153 ceph-mon[133625]: pgmap v215: 129 pgs: 129 active+clean; 273 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 196 op/s 2023-12-07T10:27:26.562 INFO:tasks.workunit.client.0.smithi119.stdout: 1 466051 6.91 MB/sec execute 219 sec latency 3283.658 ms 2023-12-07T10:27:27.563 INFO:tasks.workunit.client.0.smithi119.stdout: 1 466051 6.88 MB/sec execute 220 sec latency 4283.722 ms 2023-12-07T10:27:28.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:28 smithi119 ceph-mon[97908]: pgmap v216: 129 pgs: 129 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 317 op/s 2023-12-07T10:27:28.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:28 smithi159 ceph-mon[134726]: pgmap v216: 129 pgs: 129 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 317 op/s 2023-12-07T10:27:28.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:28 smithi153 ceph-mon[133625]: pgmap v216: 129 pgs: 129 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 317 op/s 2023-12-07T10:27:28.563 INFO:tasks.workunit.client.0.smithi119.stdout: 1 466871 6.88 MB/sec execute 221 sec latency 4724.195 ms 2023-12-07T10:27:29.563 INFO:tasks.workunit.client.0.smithi119.stdout: 1 468821 6.89 MB/sec execute 222 sec latency 18.785 ms 2023-12-07T10:27:30.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:30 smithi119 ceph-mon[97908]: pgmap v217: 129 pgs: 129 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 3.2 MiB/s wr, 227 op/s 2023-12-07T10:27:30.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:30 smithi159 ceph-mon[134726]: pgmap v217: 129 pgs: 129 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 3.2 MiB/s wr, 227 op/s 2023-12-07T10:27:30.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:30 smithi153 ceph-mon[133625]: pgmap v217: 129 pgs: 129 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 3.2 MiB/s wr, 227 op/s 2023-12-07T10:27:30.563 INFO:tasks.workunit.client.0.smithi119.stdout: 1 469936 6.88 MB/sec execute 223 sec latency 64.300 ms 2023-12-07T10:27:31.563 INFO:tasks.workunit.client.0.smithi119.stdout: 1 471916 6.90 MB/sec execute 224 sec latency 17.293 ms 2023-12-07T10:27:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.749+0000 7f0e015ff700 1 -- 172.21.15.119:0/1316288207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dfc102f10 msgr2=0x7f0dfc10b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.749+0000 7f0e015ff700 1 --2- 172.21.15.119:0/1316288207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dfc102f10 0x7f0dfc10b3f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0df0009990 tx=0x7f0df0009ca0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.750+0000 7f0e015ff700 1 -- 172.21.15.119:0/1316288207 shutdown_connections 2023-12-07T10:27:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.750+0000 7f0e015ff700 1 --2- 172.21.15.119:0/1316288207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dfc10ba10 0x7f0dfc10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.750+0000 7f0e015ff700 1 --2- 172.21.15.119:0/1316288207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0dfc1025f0 0x7f0dfc1029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.750+0000 7f0e015ff700 1 --2- 172.21.15.119:0/1316288207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dfc102f10 0x7f0dfc10b3f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.750+0000 7f0e015ff700 1 -- 172.21.15.119:0/1316288207 >> 172.21.15.119:0/1316288207 conn(0x7f0dfc0fbdf0 msgr2=0x7f0dfc0fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.750+0000 7f0e015ff700 1 -- 172.21.15.119:0/1316288207 shutdown_connections 2023-12-07T10:27:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.750+0000 7f0e015ff700 1 -- 172.21.15.119:0/1316288207 wait complete. 2023-12-07T10:27:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.751+0000 7f0e015ff700 1 Processor -- start 2023-12-07T10:27:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.751+0000 7f0e015ff700 1 -- start start 2023-12-07T10:27:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.752+0000 7f0e015ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dfc1025f0 0x7f0dfc19f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.752+0000 7f0e015ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dfc102f10 0x7f0dfc19fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.752+0000 7f0dfaffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dfc1025f0 0x7f0dfc19f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.752+0000 7f0e015ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0dfc10ba10 0x7f0dfc1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.752+0000 7f0e015ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc19a000 con 0x7f0dfc102f10 2023-12-07T10:27:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.752+0000 7f0e015ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc19a170 con 0x7f0dfc1025f0 2023-12-07T10:27:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.752+0000 7f0e015ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc19a330 con 0x7f0dfc10ba10 2023-12-07T10:27:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.752+0000 7f0dfaffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dfc1025f0 0x7f0dfc19f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:40682/0 (socket says 172.21.15.119:40682) 2023-12-07T10:27:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.752+0000 7f0dfaffd700 1 -- 172.21.15.119:0/3044439805 learned_addr learned my addr 172.21.15.119:0/3044439805 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:27:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.752+0000 7f0dfa7fc700 1 --2- 172.21.15.119:0/3044439805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dfc102f10 0x7f0dfc19fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.752+0000 7f0dfb7fe700 1 --2- 172.21.15.119:0/3044439805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0dfc10ba10 0x7f0dfc1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.753+0000 7f0dfaffd700 1 -- 172.21.15.119:0/3044439805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0dfc10ba10 msgr2=0x7f0dfc1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.753+0000 7f0dfaffd700 1 --2- 172.21.15.119:0/3044439805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0dfc10ba10 0x7f0dfc1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.753+0000 7f0dfaffd700 1 -- 172.21.15.119:0/3044439805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dfc102f10 msgr2=0x7f0dfc19fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.753+0000 7f0dfaffd700 1 --2- 172.21.15.119:0/3044439805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dfc102f10 0x7f0dfc19fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.753+0000 7f0dfaffd700 1 -- 172.21.15.119:0/3044439805 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df00095f0 con 0x7f0dfc1025f0 2023-12-07T10:27:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.753+0000 7f0dfb7fe700 1 --2- 172.21.15.119:0/3044439805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0dfc10ba10 0x7f0dfc1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:27:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.753+0000 7f0dfaffd700 1 --2- 172.21.15.119:0/3044439805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dfc1025f0 0x7f0dfc19f870 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00e9a0 tx=0x7f0dec00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.753+0000 7f0de3fff700 1 -- 172.21.15.119:0/3044439805 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec009790 con 0x7f0dfc1025f0 2023-12-07T10:27:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.753+0000 7f0de3fff700 1 -- 172.21.15.119:0/3044439805 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0dec00cbd0 con 0x7f0dfc1025f0 2023-12-07T10:27:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.753+0000 7f0e015ff700 1 -- 172.21.15.119:0/3044439805 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dfc19a610 con 0x7f0dfc1025f0 2023-12-07T10:27:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.753+0000 7f0de3fff700 1 -- 172.21.15.119:0/3044439805 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec018680 con 0x7f0dfc1025f0 2023-12-07T10:27:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.753+0000 7f0e015ff700 1 -- 172.21.15.119:0/3044439805 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dfc1a6860 con 0x7f0dfc1025f0 2023-12-07T10:27:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.757+0000 7f0de3fff700 1 -- 172.21.15.119:0/3044439805 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f0dec00cd40 con 0x7f0dfc1025f0 2023-12-07T10:27:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.758+0000 7f0de3fff700 1 --2- 172.21.15.119:0/3044439805 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0de40a1cf0 0x7f0de40a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.758+0000 7f0e14ed2b80 1 Processor -- start 2023-12-07T10:27:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.758+0000 7f0de3fff700 1 -- 172.21.15.119:0/3044439805 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0dec014070 con 0x7f0dfc1025f0 2023-12-07T10:27:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.758+0000 7f0dfa7fc700 1 --2- 172.21.15.119:0/3044439805 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0de40a1cf0 0x7f0de40a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.758+0000 7f0e14ed2b80 1 -- start start 2023-12-07T10:27:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.758+0000 7f0e14ed2b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56017aa9ca60 0x56017aa9ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.759+0000 7f0e14ed2b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56017aa9d380 0x56017aa97540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.759+0000 7f0e14ed2b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56017aa97ad0 0x56017aa97f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.759+0000 7f0e14ed2b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017aa0b760 con 0x56017aa97ad0 2023-12-07T10:27:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.759+0000 7f0e14ed2b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017a9f2740 con 0x56017aa9ca60 2023-12-07T10:27:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.759+0000 7f0e14ed2b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017a9f0060 con 0x56017aa9d380 2023-12-07T10:27:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.759+0000 7f0dfaffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56017aa9ca60 0x56017aa9ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.759+0000 7f0dfa7fc700 1 --2- 172.21.15.119:0/3044439805 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0de40a1cf0 0x7f0de40a41b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc19ab60 tx=0x7f0df000b260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.759+0000 7f0dfa7fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56017aa9d380 0x56017aa97540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.759+0000 7f0dfb7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56017aa97ad0 0x56017aa97f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.759+0000 7f0dfaffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56017aa9ca60 0x56017aa9ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:40688/0 (socket says 172.21.15.119:40688) 2023-12-07T10:27:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.759+0000 7f0dfaffd700 1 -- 172.21.15.119:0/1599173815 learned_addr learned my addr 172.21.15.119:0/1599173815 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:27:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.759+0000 7f0dfa7fc700 1 -- 172.21.15.119:0/1599173815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56017aa9ca60 msgr2=0x56017aa9ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.759+0000 7f0dfa7fc700 1 --2- 172.21.15.119:0/1599173815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56017aa9ca60 0x56017aa9ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.759+0000 7f0dfa7fc700 1 -- 172.21.15.119:0/1599173815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56017aa97ad0 msgr2=0x56017aa97f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.759+0000 7f0dfa7fc700 1 --2- 172.21.15.119:0/1599173815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56017aa97ad0 0x56017aa97f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.759+0000 7f0dfa7fc700 1 -- 172.21.15.119:0/1599173815 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56017a8685d0 con 0x56017aa9d380 2023-12-07T10:27:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.759+0000 7f0dfaffd700 1 --2- 172.21.15.119:0/1599173815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56017aa9ca60 0x56017aa9ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:27:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.760+0000 7f0dfa7fc700 1 --2- 172.21.15.119:0/1599173815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56017aa9d380 0x56017aa97540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0df0006010 tx=0x7f0df0020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17df76b04054faa9 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.760+0000 7f0de17fa700 1 -- 172.21.15.119:0/1599173815 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df000be90 con 0x56017aa9d380 2023-12-07T10:27:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.760+0000 7f0de17fa700 1 -- 172.21.15.119:0/1599173815 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0df001e040 con 0x56017aa9d380 2023-12-07T10:27:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.760+0000 7f0de17fa700 1 -- 172.21.15.119:0/1599173815 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df000f8b0 con 0x56017aa9d380 2023-12-07T10:27:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.760+0000 7f0e14ed2b80 1 -- 172.21.15.119:0/1599173815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56017aa9d380 msgr2=0x56017aa97540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.761+0000 7f0e14ed2b80 1 --2- 172.21.15.119:0/1599173815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56017aa9d380 0x56017aa97540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0df0006010 tx=0x7f0df0020000 comp rx=0 tx=0).stop 2023-12-07T10:27:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.761+0000 7f0e14ed2b80 1 -- 172.21.15.119:0/1599173815 shutdown_connections 2023-12-07T10:27:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.761+0000 7f0e14ed2b80 1 --2- 172.21.15.119:0/1599173815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56017aa97ad0 0x56017aa97f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.761+0000 7f0e14ed2b80 1 --2- 172.21.15.119:0/1599173815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56017aa9ca60 0x56017aa9ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.761+0000 7f0e14ed2b80 1 --2- 172.21.15.119:0/1599173815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56017aa9d380 0x56017aa97540 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.761+0000 7f0e14ed2b80 1 -- 172.21.15.119:0/1599173815 >> 172.21.15.119:0/1599173815 conn(0x56017a9c6300 msgr2=0x56017aa91b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.761+0000 7f0e14ed2b80 1 -- 172.21.15.119:0/1599173815 shutdown_connections 2023-12-07T10:27:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.761+0000 7f0e14ed2b80 1 -- 172.21.15.119:0/1599173815 wait complete. 2023-12-07T10:27:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.761+0000 7f0e14ed2b80 1 -- start start 2023-12-07T10:27:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.761+0000 7f0e14ed2b80 1 Processor -- start 2023-12-07T10:27:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.762+0000 7f0e14ed2b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56017aa97ad0 0x56017aaa07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.762+0000 7f0e14ed2b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56017aa9ca60 0x56017aaa0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.762+0000 7f0e14ed2b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56017aa9d380 0x56017aaa1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.762+0000 7f0e14ed2b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017a9eb950 con 0x56017aa9d380 2023-12-07T10:27:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.762+0000 7f0e14ed2b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017a9a5500 con 0x56017aa9ca60 2023-12-07T10:27:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.762+0000 7f0e14ed2b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017a9a51a0 con 0x56017aa97ad0 2023-12-07T10:27:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.762+0000 7f0e14ed2b80 10 client.0 _ll_register_callbacks cb 0x56017a98c8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:27:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.762+0000 7f0dfaffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56017aa97ad0 0x56017aaa07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.762+0000 7f0dfa7fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56017aa9ca60 0x56017aaa0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.762+0000 7f0dfb7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56017aa9d380 0x56017aaa1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.762+0000 7f0dfaffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56017aa97ad0 0x56017aaa07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:39808/0 (socket says 172.21.15.119:39808) 2023-12-07T10:27:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.762+0000 7f0dfaffd700 1 -- 172.21.15.119:0/2645178038 learned_addr learned my addr 172.21.15.119:0/2645178038 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:27:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.763+0000 7f0dfa7fc700 1 -- 172.21.15.119:0/2645178038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56017aa97ad0 msgr2=0x56017aaa07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.763+0000 7f0dfa7fc700 1 --2- 172.21.15.119:0/2645178038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56017aa97ad0 0x56017aaa07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.763+0000 7f0dfa7fc700 1 -- 172.21.15.119:0/2645178038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56017aa9d380 msgr2=0x56017aaa1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.763+0000 7f0dfa7fc700 1 --2- 172.21.15.119:0/2645178038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56017aa9d380 0x56017aaa1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.763+0000 7f0dfa7fc700 1 -- 172.21.15.119:0/2645178038 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dfc1a6860 con 0x56017aa9ca60 2023-12-07T10:27:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.763+0000 7f0dfaffd700 1 --2- 172.21.15.119:0/2645178038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56017aa97ad0 0x56017aaa07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:27:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.763+0000 7f0dfa7fc700 1 --2- 172.21.15.119:0/2645178038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56017aa9ca60 0x56017aaa0d10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x56017aa93890 tx=0x7f0df000fcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b079d8135cf3680 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.763+0000 7f0ddeff5700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:27:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.763+0000 7f0ddeff5700 1 -- 172.21.15.119:0/2645178038 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df0004700 con 0x56017aa9ca60 2023-12-07T10:27:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.763+0000 7f0ddeff5700 1 -- 172.21.15.119:0/2645178038 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0df0004860 con 0x56017aa9ca60 2023-12-07T10:27:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.763+0000 7f0e14ed2b80 1 -- 172.21.15.119:0/2645178038 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56017aa8a6f0 con 0x56017aa9ca60 2023-12-07T10:27:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.764+0000 7f0ddeff5700 1 -- 172.21.15.119:0/2645178038 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0df000be90 con 0x56017aa9ca60 2023-12-07T10:27:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.764+0000 7f0e14ed2b80 10 client.27380 fetch_fsmap learned FSMap version 51 2023-12-07T10:27:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.764+0000 7f0e14ed2b80 1 -- 172.21.15.119:0/2645178038 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7f0dfc10eb40 con 0x56017aa9ca60 2023-12-07T10:27:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.764+0000 7f0ddeff5700 1 -- 172.21.15.119:0/2645178038 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56017aa8a6f0 con 0x56017aa9ca60 2023-12-07T10:27:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.764+0000 7f0ddeff5700 1 -- 172.21.15.119:0/2645178038 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 51) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f0df00308c0 con 0x56017aa9ca60 2023-12-07T10:27:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.764+0000 7f0e14ed2b80 10 client.27380 fetch_fsmap finished waiting for FSMap version 51 2023-12-07T10:27:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.764+0000 7f0e14ed2b80 10 client.27380 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:27:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.764+0000 7f0e14ed2b80 20 client.27380 populate_metadata read hostname 'smithi119' 2023-12-07T10:27:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.764+0000 7f0e14ed2b80 1 --2- 172.21.15.119:0/2645178038 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56017aab0a40 0x56017aab2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:31.770 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.764+0000 7f0e14ed2b80 4 client.27380 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:27:31.770 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.764+0000 7f0e14ed2b80 1 -- 172.21.15.119:0/2645178038 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56017aa66740 con 0x56017aab0a40 2023-12-07T10:27:31.770 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.765+0000 7f0dfaffd700 1 --2- 172.21.15.119:0/2645178038 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56017aab0a40 0x56017aab2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:31.770 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.765+0000 7f0dfaffd700 1 --2- 172.21.15.119:0/2645178038 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56017aab0a40 0x56017aab2e60 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11ff94007c9d5dc2 server_cookie=1932128fe7f9691f in_seq=0 out_seq=0 2023-12-07T10:27:31.770 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.765+0000 7f0ddeff5700 10 client.27380 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:27:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.766+0000 7f0ddeff5700 1 -- 172.21.15.119:0/2645178038 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56017aa66740 con 0x56017aab0a40 2023-12-07T10:27:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.766+0000 7f0ddeff5700 10 client.27380 handle_command_reply: tid=0 2023-12-07T10:27:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.766+0000 7f0e14ed2b80 1 client.27380 shutdown 2023-12-07T10:27:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.766+0000 7f0e14ed2b80 1 -- 172.21.15.119:0/2645178038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56017aa9ca60 msgr2=0x56017aaa0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.766+0000 7f0e14ed2b80 1 --2- 172.21.15.119:0/2645178038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56017aa9ca60 0x56017aaa0d10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x56017aa93890 tx=0x7f0df000fcd0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.766+0000 7f0e14ed2b80 1 -- 172.21.15.119:0/2645178038 shutdown_connections 2023-12-07T10:27:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.766+0000 7f0e14ed2b80 1 --2- 172.21.15.119:0/2645178038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56017aa9d380 0x56017aaa1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.766+0000 7f0e14ed2b80 1 --2- 172.21.15.119:0/2645178038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56017aa97ad0 0x56017aaa07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.766+0000 7f0e14ed2b80 1 --2- 172.21.15.119:0/2645178038 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56017aab0a40 0x56017aab2e60 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.766+0000 7f0e14ed2b80 1 --2- 172.21.15.119:0/2645178038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56017aa9ca60 0x56017aaa0d10 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.766+0000 7f0ddeff5700 0 client.27380 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:27:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.766+0000 7f0e14ed2b80 1 -- 172.21.15.119:0/2645178038 >> 172.21.15.119:0/2645178038 conn(0x56017a9c6300 msgr2=0x56017aa8c980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.766+0000 7f0e14ed2b80 1 -- 172.21.15.119:0/2645178038 shutdown_connections 2023-12-07T10:27:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.766+0000 7f0e14ed2b80 1 -- 172.21.15.119:0/2645178038 wait complete. 2023-12-07T10:27:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.767+0000 7f0e14ed2b80 20 client.27380 trim_cache size 0 max 16384 2023-12-07T10:27:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.783+0000 7f0e14ed2b80 1 Processor -- start 2023-12-07T10:27:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.783+0000 7f0e14ed2b80 1 -- start start 2023-12-07T10:27:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.784+0000 7f0e14ed2b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56017aa97ad0 0x56017aab2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.784+0000 7f0e14ed2b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56017aa9ca60 0x56017aa97540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.784+0000 7f0e14ed2b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56017aa9d380 0x56017aaa00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.784+0000 7f0e14ed2b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017a86a8c0 con 0x56017aa9ca60 2023-12-07T10:27:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.784+0000 7f0e14ed2b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017aab8010 con 0x56017aa9d380 2023-12-07T10:27:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.784+0000 7f0e14ed2b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017aa92480 con 0x56017aa97ad0 2023-12-07T10:27:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.784+0000 7f0dfaffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56017aa97ad0 0x56017aab2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.784+0000 7f0dfa7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56017aa9ca60 0x56017aa97540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.784+0000 7f0dfb7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56017aa9d380 0x56017aaa00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.784+0000 7f0dfaffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56017aa97ad0 0x56017aab2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:39810/0 (socket says 172.21.15.119:39810) 2023-12-07T10:27:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.784+0000 7f0dfaffd700 1 -- 172.21.15.119:0/4097747477 learned_addr learned my addr 172.21.15.119:0/4097747477 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:27:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.784+0000 7f0dfa7fc700 1 -- 172.21.15.119:0/4097747477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56017aa97ad0 msgr2=0x56017aab2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.784+0000 7f0dfa7fc700 1 --2- 172.21.15.119:0/4097747477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56017aa97ad0 0x56017aab2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.784+0000 7f0dfa7fc700 1 -- 172.21.15.119:0/4097747477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56017aa9d380 msgr2=0x56017aaa00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.784+0000 7f0dfa7fc700 1 --2- 172.21.15.119:0/4097747477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56017aa9d380 0x56017aaa00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.784+0000 7f0dfa7fc700 1 -- 172.21.15.119:0/4097747477 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56017a814d80 con 0x56017aa9ca60 2023-12-07T10:27:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.784+0000 7f0dfb7fe700 1 --2- 172.21.15.119:0/4097747477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56017aa9d380 0x56017aaa00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:27:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.785+0000 7f0dfa7fc700 1 --2- 172.21.15.119:0/4097747477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56017aa9ca60 0x56017aa97540 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0df0000c00 tx=0x7f0df000f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154f7c8555b2a69a server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.785+0000 7f0de1ffb700 1 -- 172.21.15.119:0/4097747477 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df000b390 con 0x56017aa9ca60 2023-12-07T10:27:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.785+0000 7f0de1ffb700 1 -- 172.21.15.119:0/4097747477 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0df0029050 con 0x56017aa9ca60 2023-12-07T10:27:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.785+0000 7f0de1ffb700 1 -- 172.21.15.119:0/4097747477 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df00306b0 con 0x56017aa9ca60 2023-12-07T10:27:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.786+0000 7f0e14ed2b80 1 -- 172.21.15.119:0/4097747477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56017aa9ca60 msgr2=0x56017aa97540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.786+0000 7f0e14ed2b80 1 --2- 172.21.15.119:0/4097747477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56017aa9ca60 0x56017aa97540 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0df0000c00 tx=0x7f0df000f460 comp rx=0 tx=0).stop 2023-12-07T10:27:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.786+0000 7f0e14ed2b80 1 -- 172.21.15.119:0/4097747477 shutdown_connections 2023-12-07T10:27:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.786+0000 7f0e14ed2b80 1 --2- 172.21.15.119:0/4097747477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56017aa9d380 0x56017aaa00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.786+0000 7f0e14ed2b80 1 --2- 172.21.15.119:0/4097747477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56017aa97ad0 0x56017aab2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.786+0000 7f0e14ed2b80 1 --2- 172.21.15.119:0/4097747477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56017aa9ca60 0x56017aa97540 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.786+0000 7f0e14ed2b80 1 -- 172.21.15.119:0/4097747477 >> 172.21.15.119:0/4097747477 conn(0x56017a9c6300 msgr2=0x56017aa99f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.786+0000 7f0e14ed2b80 1 -- 172.21.15.119:0/4097747477 shutdown_connections 2023-12-07T10:27:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.786+0000 7f0e14ed2b80 1 -- 172.21.15.119:0/4097747477 wait complete. 2023-12-07T10:27:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.786+0000 7f0e14ed2b80 1 -- start start 2023-12-07T10:27:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.787+0000 7f0e14ed2b80 1 Processor -- start 2023-12-07T10:27:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.787+0000 7f0e14ed2b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56017aa97ad0 0x56017aaa8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.787+0000 7f0e14ed2b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56017aa9ca60 0x56017aaa3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.787+0000 7f0e14ed2b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56017aa9d380 0x56017aaa4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.787+0000 7f0e14ed2b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017aaa4ba0 con 0x56017aa9ca60 2023-12-07T10:27:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.787+0000 7f0e14ed2b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017ab05ef0 con 0x56017aa9d380 2023-12-07T10:27:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.787+0000 7f0e14ed2b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017ab06030 con 0x56017aa97ad0 2023-12-07T10:27:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.787+0000 7f0e14ed2b80 10 client.0 _ll_register_callbacks cb 0x56017a98c8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:27:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.787+0000 7f0dfaffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56017aa97ad0 0x56017aaa8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.787+0000 7f0dfa7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56017aa9ca60 0x56017aaa3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.787+0000 7f0dfaffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56017aa97ad0 0x56017aaa8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:39820/0 (socket says 172.21.15.119:39820) 2023-12-07T10:27:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.787+0000 7f0dfaffd700 1 -- 172.21.15.119:0/2701985338 learned_addr learned my addr 172.21.15.119:0/2701985338 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:27:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.788+0000 7f0dfb7fe700 1 --2- 172.21.15.119:0/2701985338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56017aa9d380 0x56017aaa4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.788+0000 7f0dfaffd700 1 -- 172.21.15.119:0/2701985338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56017aa9d380 msgr2=0x56017aaa4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.788+0000 7f0dfaffd700 1 --2- 172.21.15.119:0/2701985338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56017aa9d380 0x56017aaa4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.788+0000 7f0dfaffd700 1 -- 172.21.15.119:0/2701985338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56017aa9ca60 msgr2=0x56017aaa3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.788+0000 7f0dfaffd700 1 --2- 172.21.15.119:0/2701985338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56017aa9ca60 0x56017aaa3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.788+0000 7f0dfaffd700 1 -- 172.21.15.119:0/2701985338 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df4009250 con 0x56017aa97ad0 2023-12-07T10:27:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.788+0000 7f0dfaffd700 1 --2- 172.21.15.119:0/2701985338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56017aa97ad0 0x56017aaa8290 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0dec005bf0 tx=0x7f0dec01f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f9d7ca1ba4d4ee9 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.788+0000 7f0ddeff5700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:27:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.788+0000 7f0ddeff5700 1 -- 172.21.15.119:0/2701985338 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec09eaa0 con 0x56017aa97ad0 2023-12-07T10:27:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.788+0000 7f0ddeff5700 1 -- 172.21.15.119:0/2701985338 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0dec0a2ce0 con 0x56017aa97ad0 2023-12-07T10:27:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.788+0000 7f0e14ed2b80 1 -- 172.21.15.119:0/2701985338 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0df001e040 con 0x56017aa97ad0 2023-12-07T10:27:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.789+0000 7f0ddeff5700 1 -- 172.21.15.119:0/2701985338 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0dec0850b0 con 0x56017aa97ad0 2023-12-07T10:27:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.789+0000 7f0e14ed2b80 10 client.27370 fetch_fsmap learned FSMap version 51 2023-12-07T10:27:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.789+0000 7f0e14ed2b80 1 -- 172.21.15.119:0/2701985338 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7f0dfc10eb40 con 0x56017aa97ad0 2023-12-07T10:27:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.789+0000 7f0ddeff5700 1 -- 172.21.15.119:0/2701985338 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0df001e040 con 0x56017aa97ad0 2023-12-07T10:27:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.789+0000 7f0ddeff5700 1 -- 172.21.15.119:0/2701985338 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 51) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f0dec099e40 con 0x56017aa97ad0 2023-12-07T10:27:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.789+0000 7f0e14ed2b80 10 client.27370 fetch_fsmap finished waiting for FSMap version 51 2023-12-07T10:27:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.789+0000 7f0e14ed2b80 10 client.27370 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:27:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.789+0000 7f0e14ed2b80 20 client.27370 populate_metadata read hostname 'smithi119' 2023-12-07T10:27:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.789+0000 7f0e14ed2b80 1 --2- 172.21.15.119:0/2701985338 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56017ab07750 0x56017ab09b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.789+0000 7f0e14ed2b80 4 client.27370 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:27:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.789+0000 7f0e14ed2b80 1 -- 172.21.15.119:0/2701985338 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56017a9b1f70 con 0x56017ab07750 2023-12-07T10:27:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.789+0000 7f0dfa7fc700 1 --2- 172.21.15.119:0/2701985338 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56017ab07750 0x56017ab09b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.790+0000 7f0dfa7fc700 1 --2- 172.21.15.119:0/2701985338 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56017ab07750 0x56017ab09b70 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adf1ec0e93bc674e server_cookie=88400801269d2a97 in_seq=0 out_seq=0 2023-12-07T10:27:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.790+0000 7f0ddeff5700 10 client.27370 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:27:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.835+0000 7f0ddeff5700 1 -- 172.21.15.119:0/2701985338 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56017a9b1f70 con 0x56017ab07750 2023-12-07T10:27:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.835+0000 7f0ddeff5700 10 client.27370 handle_command_reply: tid=0 2023-12-07T10:27:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.835+0000 7f0e14ed2b80 1 client.27370 shutdown 2023-12-07T10:27:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.835+0000 7f0e14ed2b80 1 -- 172.21.15.119:0/2701985338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56017aa97ad0 msgr2=0x56017aaa8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.835+0000 7f0e14ed2b80 1 --2- 172.21.15.119:0/2701985338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56017aa97ad0 0x56017aaa8290 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0dec005bf0 tx=0x7f0dec01f000 comp rx=0 tx=0).stop 2023-12-07T10:27:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.835+0000 7f0e14ed2b80 1 -- 172.21.15.119:0/2701985338 shutdown_connections 2023-12-07T10:27:31.837 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.835+0000 7f0e14ed2b80 1 --2- 172.21.15.119:0/2701985338 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x56017ab07750 0x56017ab09b70 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.835+0000 7f0e14ed2b80 1 --2- 172.21.15.119:0/2701985338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56017aa9d380 0x56017aaa4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.835+0000 7f0e14ed2b80 1 --2- 172.21.15.119:0/2701985338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56017aa97ad0 0x56017aaa8290 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.835+0000 7f0e14ed2b80 1 --2- 172.21.15.119:0/2701985338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56017aa9ca60 0x56017aaa3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.835+0000 7f0ddeff5700 0 client.27370 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:27:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.835+0000 7f0e14ed2b80 1 -- 172.21.15.119:0/2701985338 >> 172.21.15.119:0/2701985338 conn(0x56017a9c6300 msgr2=0x56017aa9b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.835+0000 7f0e14ed2b80 1 -- 172.21.15.119:0/2701985338 shutdown_connections 2023-12-07T10:27:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.835+0000 7f0e14ed2b80 1 -- 172.21.15.119:0/2701985338 wait complete. 2023-12-07T10:27:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.836+0000 7f0e14ed2b80 20 client.27370 trim_cache size 0 max 16384 2023-12-07T10:27:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:27:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:27:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:27:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:27:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.836+0000 7f0e015ff700 1 -- 172.21.15.119:0/3044439805 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0de40a1cf0 msgr2=0x7f0de40a41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.836+0000 7f0e015ff700 1 --2- 172.21.15.119:0/3044439805 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0de40a1cf0 0x7f0de40a41b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc19ab60 tx=0x7f0df000b260 comp rx=0 tx=0).stop 2023-12-07T10:27:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.836+0000 7f0e015ff700 1 -- 172.21.15.119:0/3044439805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dfc1025f0 msgr2=0x7f0dfc19f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.836+0000 7f0e015ff700 1 --2- 172.21.15.119:0/3044439805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dfc1025f0 0x7f0dfc19f870 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00e9a0 tx=0x7f0dec00ecb0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.836+0000 7f0e015ff700 1 -- 172.21.15.119:0/3044439805 shutdown_connections 2023-12-07T10:27:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.836+0000 7f0e015ff700 1 --2- 172.21.15.119:0/3044439805 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0de40a1cf0 0x7f0de40a41b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.836+0000 7f0e015ff700 1 --2- 172.21.15.119:0/3044439805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0dfc10ba10 0x7f0dfc1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.836+0000 7f0e015ff700 1 --2- 172.21.15.119:0/3044439805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dfc1025f0 0x7f0dfc19f870 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.836+0000 7f0e015ff700 1 --2- 172.21.15.119:0/3044439805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dfc102f10 0x7f0dfc19fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.836+0000 7f0e015ff700 1 -- 172.21.15.119:0/3044439805 >> 172.21.15.119:0/3044439805 conn(0x7f0dfc0fbdf0 msgr2=0x7f0dfc100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.836+0000 7f0e015ff700 1 -- 172.21.15.119:0/3044439805 shutdown_connections 2023-12-07T10:27:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:31.836+0000 7f0e015ff700 1 -- 172.21.15.119:0/3044439805 wait complete. 2023-12-07T10:27:32.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:31 smithi119 ceph-mon[97908]: pgmap v218: 129 pgs: 129 active+clean; 287 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.0 MiB/s wr, 278 op/s 2023-12-07T10:27:32.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:31 smithi159 ceph-mon[134726]: pgmap v218: 129 pgs: 129 active+clean; 287 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.0 MiB/s wr, 278 op/s 2023-12-07T10:27:32.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:31 smithi153 ceph-mon[133625]: pgmap v218: 129 pgs: 129 active+clean; 287 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.0 MiB/s wr, 278 op/s 2023-12-07T10:27:32.563 INFO:tasks.workunit.client.0.smithi119.stdout: 1 473133 6.90 MB/sec execute 225 sec latency 32.630 ms 2023-12-07T10:27:33.563 INFO:tasks.workunit.client.0.smithi119.stdout: 1 474968 6.90 MB/sec execute 226 sec latency 17.026 ms 2023-12-07T10:27:34.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:34 smithi119 ceph-mon[97908]: pgmap v219: 129 pgs: 129 active+clean; 296 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 5.4 MiB/s wr, 226 op/s 2023-12-07T10:27:34.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:34 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:27:34.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:34 smithi159 ceph-mon[134726]: pgmap v219: 129 pgs: 129 active+clean; 296 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 5.4 MiB/s wr, 226 op/s 2023-12-07T10:27:34.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:34 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:27:34.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:34 smithi153 ceph-mon[133625]: pgmap v219: 129 pgs: 129 active+clean; 296 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 5.4 MiB/s wr, 226 op/s 2023-12-07T10:27:34.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:34 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:27:34.563 INFO:tasks.workunit.client.0.smithi119.stdout: 1 476549 6.92 MB/sec execute 227 sec latency 10.497 ms 2023-12-07T10:27:35.563 INFO:tasks.workunit.client.0.smithi119.stdout: 1 478428 6.92 MB/sec execute 228 sec latency 16.702 ms 2023-12-07T10:27:36.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:36 smithi119 ceph-mon[97908]: pgmap v220: 129 pgs: 129 active+clean; 296 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 5.3 MiB/s wr, 207 op/s 2023-12-07T10:27:36.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:36 smithi159 ceph-mon[134726]: pgmap v220: 129 pgs: 129 active+clean; 296 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 5.3 MiB/s wr, 207 op/s 2023-12-07T10:27:36.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:36 smithi153 ceph-mon[133625]: pgmap v220: 129 pgs: 129 active+clean; 296 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 5.3 MiB/s wr, 207 op/s 2023-12-07T10:27:36.563 INFO:tasks.workunit.client.0.smithi119.stdout: 1 480061 6.95 MB/sec execute 229 sec latency 13.962 ms 2023-12-07T10:27:37.563 INFO:tasks.workunit.client.0.smithi119.stdout: 1 481361 6.94 MB/sec execute 230 sec latency 22.845 ms 2023-12-07T10:27:38.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:38 smithi119 ceph-mon[97908]: pgmap v221: 129 pgs: 129 active+clean; 303 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.0 MiB/s wr, 309 op/s 2023-12-07T10:27:38.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:38 smithi159 ceph-mon[134726]: pgmap v221: 129 pgs: 129 active+clean; 303 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.0 MiB/s wr, 309 op/s 2023-12-07T10:27:38.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:38 smithi153 ceph-mon[133625]: pgmap v221: 129 pgs: 129 active+clean; 303 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.0 MiB/s wr, 309 op/s 2023-12-07T10:27:38.563 INFO:tasks.workunit.client.0.smithi119.stdout: 1 483427 6.97 MB/sec execute 231 sec latency 20.630 ms 2023-12-07T10:27:39.563 INFO:tasks.workunit.client.0.smithi119.stdout: 1 484476 6.95 MB/sec execute 232 sec latency 14.617 ms 2023-12-07T10:27:40.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:40 smithi119 ceph-mon[97908]: pgmap v222: 129 pgs: 129 active+clean; 303 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 3.5 MiB/s wr, 188 op/s 2023-12-07T10:27:40.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:40 smithi159 ceph-mon[134726]: pgmap v222: 129 pgs: 129 active+clean; 303 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 3.5 MiB/s wr, 188 op/s 2023-12-07T10:27:40.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:40 smithi153 ceph-mon[133625]: pgmap v222: 129 pgs: 129 active+clean; 303 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 3.5 MiB/s wr, 188 op/s 2023-12-07T10:27:40.563 INFO:tasks.workunit.client.0.smithi119.stdout: 1 486566 6.97 MB/sec execute 233 sec latency 15.623 ms 2023-12-07T10:27:41.563 INFO:tasks.workunit.client.0.smithi119.stdout: 1 487673 6.97 MB/sec execute 234 sec latency 11.859 ms 2023-12-07T10:27:42.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:41 smithi119 ceph-mon[97908]: pgmap v223: 129 pgs: 129 active+clean; 307 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 246 op/s 2023-12-07T10:27:42.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:41 smithi159 ceph-mon[134726]: pgmap v223: 129 pgs: 129 active+clean; 307 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 246 op/s 2023-12-07T10:27:42.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:41 smithi153 ceph-mon[133625]: pgmap v223: 129 pgs: 129 active+clean; 307 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 246 op/s 2023-12-07T10:27:42.563 INFO:tasks.workunit.client.0.smithi119.stdout: 1 489520 6.98 MB/sec execute 235 sec latency 22.376 ms 2023-12-07T10:27:43.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:42 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:27:43.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:42 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:27:43.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:42 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:27:43.563 INFO:tasks.workunit.client.0.smithi119.stdout: 1 490764 6.98 MB/sec execute 236 sec latency 18.051 ms 2023-12-07T10:27:44.229 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:43 smithi119 ceph-mon[97908]: pgmap v224: 129 pgs: 129 active+clean; 310 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 224 op/s 2023-12-07T10:27:44.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:43 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:27:44.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:43 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:27:44.230 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:43 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:27:44.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:43 smithi159 ceph-mon[134726]: pgmap v224: 129 pgs: 129 active+clean; 310 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 224 op/s 2023-12-07T10:27:44.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:43 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:27:44.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:43 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:27:44.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:43 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:27:44.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:43 smithi153 ceph-mon[133625]: pgmap v224: 129 pgs: 129 active+clean; 310 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 224 op/s 2023-12-07T10:27:44.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:43 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:27:44.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:43 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:27:44.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:43 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:27:44.563 INFO:tasks.workunit.client.0.smithi119.stdout: 1 492448 6.98 MB/sec execute 237 sec latency 16.432 ms 2023-12-07T10:27:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.002+0000 7fe155dff700 1 -- 172.21.15.119:0/3446605460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe150109720 msgr2=0x7fe150109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.002+0000 7fe155dff700 1 --2- 172.21.15.119:0/3446605460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe150109720 0x7fe150109b00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe140009990 tx=0x7fe140009ca0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.002+0000 7fe155dff700 1 -- 172.21.15.119:0/3446605460 shutdown_connections 2023-12-07T10:27:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.002+0000 7fe155dff700 1 --2- 172.21.15.119:0/3446605460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1501040d0 0x7fe1501105e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.002+0000 7fe155dff700 1 --2- 172.21.15.119:0/3446605460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe150109720 0x7fe150109b00 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.002+0000 7fe155dff700 1 --2- 172.21.15.119:0/3446605460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe150103710 0x7fe150103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.002+0000 7fe155dff700 1 -- 172.21.15.119:0/3446605460 >> 172.21.15.119:0/3446605460 conn(0x7fe1500ff1c0 msgr2=0x7fe1501015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.002+0000 7fe155dff700 1 -- 172.21.15.119:0/3446605460 shutdown_connections 2023-12-07T10:27:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.002+0000 7fe155dff700 1 -- 172.21.15.119:0/3446605460 wait complete. 2023-12-07T10:27:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.004+0000 7fe155dff700 1 Processor -- start 2023-12-07T10:27:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.004+0000 7fe155dff700 1 -- start start 2023-12-07T10:27:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.004+0000 7fe155dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe150103710 0x7fe15019f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.004+0000 7fe155dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1501040d0 0x7fe15019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.004+0000 7fe155dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe150109720 0x7fe1501999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.004+0000 7fe155dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15019a000 con 0x7fe150109720 2023-12-07T10:27:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.004+0000 7fe155dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15019a170 con 0x7fe1501040d0 2023-12-07T10:27:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.004+0000 7fe155dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15019a330 con 0x7fe150103710 2023-12-07T10:27:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.005+0000 7fe14ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe150109720 0x7fe1501999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.005+0000 7fe14effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1501040d0 0x7fe15019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.005+0000 7fe14ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe150109720 0x7fe1501999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43222/0 (socket says 172.21.15.119:43222) 2023-12-07T10:27:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.005+0000 7fe14ffff700 1 -- 172.21.15.119:0/1751583749 learned_addr learned my addr 172.21.15.119:0/1751583749 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:27:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.005+0000 7fe14f7fe700 1 --2- 172.21.15.119:0/1751583749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe150103710 0x7fe15019f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.005+0000 7fe14effd700 1 -- 172.21.15.119:0/1751583749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe150103710 msgr2=0x7fe15019f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.005+0000 7fe14effd700 1 --2- 172.21.15.119:0/1751583749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe150103710 0x7fe15019f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.005+0000 7fe14effd700 1 -- 172.21.15.119:0/1751583749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe150109720 msgr2=0x7fe1501999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.005+0000 7fe14effd700 1 --2- 172.21.15.119:0/1751583749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe150109720 0x7fe1501999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.005+0000 7fe14effd700 1 -- 172.21.15.119:0/1751583749 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1400095f0 con 0x7fe1501040d0 2023-12-07T10:27:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.005+0000 7fe14ffff700 1 --2- 172.21.15.119:0/1751583749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe150109720 0x7fe1501999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:27:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.006+0000 7fe14effd700 1 --2- 172.21.15.119:0/1751583749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1501040d0 0x7fe15019fdb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe14400e9f0 tx=0x7fe14400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.006+0000 7fe14cff9700 1 -- 172.21.15.119:0/1751583749 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14400c9c0 con 0x7fe1501040d0 2023-12-07T10:27:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.006+0000 7fe155dff700 1 -- 172.21.15.119:0/1751583749 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe15019a610 con 0x7fe1501040d0 2023-12-07T10:27:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.006+0000 7fe14cff9700 1 -- 172.21.15.119:0/1751583749 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe144010430 con 0x7fe1501040d0 2023-12-07T10:27:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.006+0000 7fe14cff9700 1 -- 172.21.15.119:0/1751583749 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe144018680 con 0x7fe1501040d0 2023-12-07T10:27:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.006+0000 7fe155dff700 1 -- 172.21.15.119:0/1751583749 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1501a6860 con 0x7fe1501040d0 2023-12-07T10:27:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.009+0000 7fe14cff9700 1 -- 172.21.15.119:0/1751583749 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fe144010860 con 0x7fe1501040d0 2023-12-07T10:27:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.011+0000 7fe169822b80 1 Processor -- start 2023-12-07T10:27:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.011+0000 7fe169822b80 1 -- start start 2023-12-07T10:27:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.011+0000 7fe169822b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55611b39ca60 0x55611b39ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.011+0000 7fe169822b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55611b39d380 0x55611b397540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:45.025 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.011+0000 7fe14cff9700 1 --2- 172.21.15.119:0/1751583749 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe1380a1c80 0x7fe1380a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.011+0000 7fe169822b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55611b397ad0 0x55611b397f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.011+0000 7fe169822b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55611b30b760 con 0x55611b397ad0 2023-12-07T10:27:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.011+0000 7fe169822b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55611b2f2740 con 0x55611b39d380 2023-12-07T10:27:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.011+0000 7fe169822b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55611b2f0060 con 0x55611b39ca60 2023-12-07T10:27:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.012+0000 7fe14cff9700 1 -- 172.21.15.119:0/1751583749 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fe144014070 con 0x7fe1501040d0 2023-12-07T10:27:45.026 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.012+0000 7fe14ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55611b39ca60 0x55611b39ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.012+0000 7fe14effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55611b39d380 0x55611b397540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.012+0000 7fe14effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55611b39d380 0x55611b397540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:46868/0 (socket says 172.21.15.119:46868) 2023-12-07T10:27:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.012+0000 7fe14ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55611b39ca60 0x55611b39ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:54378/0 (socket says 172.21.15.119:54378) 2023-12-07T10:27:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.012+0000 7fe14f7fe700 1 --2- 172.21.15.119:0/1751583749 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe1380a1c80 0x7fe1380a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.012+0000 7fe14effd700 1 -- 172.21.15.119:0/146826007 learned_addr learned my addr 172.21.15.119:0/146826007 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:27:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.012+0000 7fe14ffff700 1 --2- 172.21.15.119:0/146826007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55611b397ad0 0x55611b397f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.012+0000 7fe14effd700 1 -- 172.21.15.119:0/146826007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55611b39ca60 msgr2=0x55611b39ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:45.028 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.012+0000 7fe14effd700 1 --2- 172.21.15.119:0/146826007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55611b39ca60 0x55611b39ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.028 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.012+0000 7fe14effd700 1 -- 172.21.15.119:0/146826007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55611b397ad0 msgr2=0x55611b397f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:45.028 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.012+0000 7fe14effd700 1 --2- 172.21.15.119:0/146826007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55611b397ad0 0x55611b397f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.028 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.012+0000 7fe14effd700 1 -- 172.21.15.119:0/146826007 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55611b1685d0 con 0x55611b39d380 2023-12-07T10:27:45.028 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.012+0000 7fe14ffff700 1 --2- 172.21.15.119:0/146826007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55611b397ad0 0x55611b397f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:27:45.029 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.012+0000 7fe14effd700 1 --2- 172.21.15.119:0/146826007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55611b39d380 0x55611b397540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe1440972e0 tx=0x7fe144096ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1df63f8eabb4ecb9 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:45.029 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.013+0000 7fe14f7fe700 1 --2- 172.21.15.119:0/1751583749 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe1380a1c80 0x7fe1380a4140 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe14000b2e0 tx=0x7fe140005960 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:45.029 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.013+0000 7fe135ffb700 1 -- 172.21.15.119:0/146826007 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe144098cb0 con 0x55611b39d380 2023-12-07T10:27:45.029 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.013+0000 7fe135ffb700 1 -- 172.21.15.119:0/146826007 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1440850b0 con 0x55611b39d380 2023-12-07T10:27:45.029 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.013+0000 7fe135ffb700 1 -- 172.21.15.119:0/146826007 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe144099440 con 0x55611b39d380 2023-12-07T10:27:45.029 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.013+0000 7fe169822b80 1 -- 172.21.15.119:0/146826007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55611b39d380 msgr2=0x55611b397540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:45.029 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.013+0000 7fe169822b80 1 --2- 172.21.15.119:0/146826007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55611b39d380 0x55611b397540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe1440972e0 tx=0x7fe144096ce0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.030 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.014+0000 7fe169822b80 1 -- 172.21.15.119:0/146826007 shutdown_connections 2023-12-07T10:27:45.030 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.014+0000 7fe169822b80 1 --2- 172.21.15.119:0/146826007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55611b397ad0 0x55611b397f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.030 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.014+0000 7fe169822b80 1 --2- 172.21.15.119:0/146826007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55611b39ca60 0x55611b39ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.030 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.014+0000 7fe169822b80 1 --2- 172.21.15.119:0/146826007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55611b39d380 0x55611b397540 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.030 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.014+0000 7fe169822b80 1 -- 172.21.15.119:0/146826007 >> 172.21.15.119:0/146826007 conn(0x55611b2c6300 msgr2=0x55611b391b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:45.030 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.014+0000 7fe169822b80 1 -- 172.21.15.119:0/146826007 shutdown_connections 2023-12-07T10:27:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.014+0000 7fe169822b80 1 -- 172.21.15.119:0/146826007 wait complete. 2023-12-07T10:27:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.014+0000 7fe169822b80 1 -- start start 2023-12-07T10:27:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.014+0000 7fe169822b80 1 Processor -- start 2023-12-07T10:27:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.015+0000 7fe169822b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55611b397ad0 0x55611b3a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.015+0000 7fe169822b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55611b39ca60 0x55611b3a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.015+0000 7fe169822b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55611b39d380 0x55611b3a1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.015+0000 7fe169822b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55611b2eb950 con 0x55611b397ad0 2023-12-07T10:27:45.032 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.015+0000 7fe169822b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55611b2a5500 con 0x55611b39ca60 2023-12-07T10:27:45.032 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.015+0000 7fe169822b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55611b2a51a0 con 0x55611b39d380 2023-12-07T10:27:45.032 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.015+0000 7fe169822b80 10 client.0 _ll_register_callbacks cb 0x55611b28c8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:27:45.032 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.015+0000 7fe14f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55611b397ad0 0x55611b3a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:45.032 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.015+0000 7fe14ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55611b39d380 0x55611b3a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:45.032 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.015+0000 7fe14effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55611b39ca60 0x55611b3a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:45.032 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.015+0000 7fe14f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55611b397ad0 0x55611b3a07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43250/0 (socket says 172.21.15.119:43250) 2023-12-07T10:27:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.015+0000 7fe14f7fe700 1 -- 172.21.15.119:0/137686657 learned_addr learned my addr 172.21.15.119:0/137686657 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:27:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.016+0000 7fe14f7fe700 1 -- 172.21.15.119:0/137686657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55611b39d380 msgr2=0x55611b3a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.016+0000 7fe14f7fe700 1 --2- 172.21.15.119:0/137686657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55611b39d380 0x55611b3a1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.016+0000 7fe14f7fe700 1 -- 172.21.15.119:0/137686657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55611b39ca60 msgr2=0x55611b3a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.016+0000 7fe14f7fe700 1 --2- 172.21.15.119:0/137686657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55611b39ca60 0x55611b3a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.033 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.016+0000 7fe14f7fe700 1 -- 172.21.15.119:0/137686657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1501a6860 con 0x55611b397ad0 2023-12-07T10:27:45.034 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.016+0000 7fe14ffff700 1 --2- 172.21.15.119:0/137686657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55611b39d380 0x55611b3a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:27:45.034 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.016+0000 7fe14effd700 1 --2- 172.21.15.119:0/137686657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55611b39ca60 0x55611b3a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:27:45.034 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.016+0000 7fe14f7fe700 1 --2- 172.21.15.119:0/137686657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55611b397ad0 0x55611b3a07a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe140015000 tx=0x7fe140009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e35c0357d22fd1 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:45.034 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.016+0000 7fe12effd700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:27:45.034 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.016+0000 7fe12effd700 1 -- 172.21.15.119:0/137686657 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe140004df0 con 0x55611b397ad0 2023-12-07T10:27:45.034 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.016+0000 7fe169822b80 1 -- 172.21.15.119:0/137686657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55611b38a6f0 con 0x55611b397ad0 2023-12-07T10:27:45.035 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.017+0000 7fe12effd700 1 -- 172.21.15.119:0/137686657 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fe14000be80 con 0x55611b397ad0 2023-12-07T10:27:45.035 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.017+0000 7fe12effd700 1 -- 172.21.15.119:0/137686657 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe140036710 con 0x55611b397ad0 2023-12-07T10:27:45.035 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.017+0000 7fe169822b80 10 client.17361 fetch_fsmap learned FSMap version 52 2023-12-07T10:27:45.035 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.017+0000 7fe169822b80 1 -- 172.21.15.119:0/137686657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55611b114d80 con 0x55611b397ad0 2023-12-07T10:27:45.035 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.017+0000 7fe12effd700 1 -- 172.21.15.119:0/137686657 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55611b38a6f0 con 0x55611b397ad0 2023-12-07T10:27:45.035 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.017+0000 7fe12effd700 1 -- 172.21.15.119:0/137686657 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 52) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fe14000f4f0 con 0x55611b397ad0 2023-12-07T10:27:45.035 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.017+0000 7fe169822b80 10 client.17361 fetch_fsmap finished waiting for FSMap version 52 2023-12-07T10:27:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.017+0000 7fe169822b80 10 client.17361 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:27:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.017+0000 7fe169822b80 20 client.17361 populate_metadata read hostname 'smithi119' 2023-12-07T10:27:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.017+0000 7fe169822b80 1 --2- 172.21.15.119:0/137686657 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55611b3b0e60 0x55611b3b3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.017+0000 7fe169822b80 4 client.17361 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:27:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.017+0000 7fe169822b80 1 -- 172.21.15.119:0/137686657 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55611b366740 con 0x55611b3b0e60 2023-12-07T10:27:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.018+0000 7fe14effd700 1 --2- 172.21.15.119:0/137686657 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55611b3b0e60 0x55611b3b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.018+0000 7fe12effd700 10 client.17361 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:27:45.037 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.018+0000 7fe14effd700 1 --2- 172.21.15.119:0/137686657 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55611b3b0e60 0x55611b3b3280 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=506e53c234c1d52d server_cookie=55d4db5a3bc8164d in_seq=0 out_seq=0 2023-12-07T10:27:45.037 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.019+0000 7fe12effd700 1 -- 172.21.15.119:0/137686657 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55611b366740 con 0x55611b3b0e60 2023-12-07T10:27:45.037 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.019+0000 7fe12effd700 10 client.17361 handle_command_reply: tid=0 2023-12-07T10:27:45.037 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.019+0000 7fe169822b80 1 client.17361 shutdown 2023-12-07T10:27:45.037 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.019+0000 7fe169822b80 1 -- 172.21.15.119:0/137686657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55611b397ad0 msgr2=0x55611b3a07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:45.037 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.019+0000 7fe169822b80 1 --2- 172.21.15.119:0/137686657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55611b397ad0 0x55611b3a07a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe140015000 tx=0x7fe140009ca0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.019+0000 7fe169822b80 1 -- 172.21.15.119:0/137686657 shutdown_connections 2023-12-07T10:27:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.019+0000 7fe169822b80 1 --2- 172.21.15.119:0/137686657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55611b39d380 0x55611b3a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.019+0000 7fe169822b80 1 --2- 172.21.15.119:0/137686657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55611b397ad0 0x55611b3a07a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.019+0000 7fe169822b80 1 --2- 172.21.15.119:0/137686657 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55611b3b0e60 0x55611b3b3280 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.019+0000 7fe169822b80 1 --2- 172.21.15.119:0/137686657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55611b39ca60 0x55611b3a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.019+0000 7fe12effd700 0 client.17361 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:27:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.019+0000 7fe169822b80 1 -- 172.21.15.119:0/137686657 >> 172.21.15.119:0/137686657 conn(0x55611b2c6300 msgr2=0x55611b38c980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.019+0000 7fe169822b80 1 -- 172.21.15.119:0/137686657 shutdown_connections 2023-12-07T10:27:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.019+0000 7fe169822b80 1 -- 172.21.15.119:0/137686657 wait complete. 2023-12-07T10:27:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.019+0000 7fe169822b80 20 client.17361 trim_cache size 0 max 16384 2023-12-07T10:27:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.037+0000 7fe169822b80 1 Processor -- start 2023-12-07T10:27:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.037+0000 7fe169822b80 1 -- start start 2023-12-07T10:27:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.038+0000 7fe169822b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55611b397ad0 0x55611b397540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.038+0000 7fe169822b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55611b39ca60 0x55611b3a1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.038+0000 7fe169822b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55611b39d380 0x55611b3a7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.038+0000 7fe169822b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55611b16a8c0 con 0x55611b397ad0 2023-12-07T10:27:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.038+0000 7fe169822b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55611b38a6c0 con 0x55611b39d380 2023-12-07T10:27:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.038+0000 7fe169822b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55611b3b8450 con 0x55611b39ca60 2023-12-07T10:27:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.038+0000 7fe14effd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55611b39ca60 0x55611b3a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.038+0000 7fe14f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55611b397ad0 0x55611b397540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.038+0000 7fe14f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55611b397ad0 0x55611b397540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43252/0 (socket says 172.21.15.119:43252) 2023-12-07T10:27:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.038+0000 7fe14f7fe700 1 -- 172.21.15.119:0/672659273 learned_addr learned my addr 172.21.15.119:0/672659273 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:27:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.038+0000 7fe14ffff700 1 --2- 172.21.15.119:0/672659273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55611b39d380 0x55611b3a7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.038+0000 7fe14effd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55611b39ca60 0x55611b3a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:54406/0 (socket says 172.21.15.119:54406) 2023-12-07T10:27:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.038+0000 7fe14f7fe700 1 -- 172.21.15.119:0/672659273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55611b39ca60 msgr2=0x55611b3a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.038+0000 7fe14f7fe700 1 --2- 172.21.15.119:0/672659273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55611b39ca60 0x55611b3a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.038+0000 7fe14f7fe700 1 -- 172.21.15.119:0/672659273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55611b39d380 msgr2=0x55611b3a7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.038+0000 7fe14f7fe700 1 --2- 172.21.15.119:0/672659273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55611b39d380 0x55611b3a7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.038+0000 7fe14f7fe700 1 -- 172.21.15.119:0/672659273 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55611b0fdc00 con 0x55611b397ad0 2023-12-07T10:27:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.039+0000 7fe14f7fe700 1 --2- 172.21.15.119:0/672659273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55611b397ad0 0x55611b397540 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe140000c00 tx=0x7fe14000fe80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ead789b02dd851b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.039+0000 7fe1367fc700 1 -- 172.21.15.119:0/672659273 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14000fc70 con 0x55611b397ad0 2023-12-07T10:27:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.039+0000 7fe1367fc700 1 -- 172.21.15.119:0/672659273 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe14001e040 con 0x55611b397ad0 2023-12-07T10:27:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.039+0000 7fe1367fc700 1 -- 172.21.15.119:0/672659273 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14003f7b0 con 0x55611b397ad0 2023-12-07T10:27:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.040+0000 7fe169822b80 1 -- 172.21.15.119:0/672659273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55611b397ad0 msgr2=0x55611b397540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.040+0000 7fe169822b80 1 --2- 172.21.15.119:0/672659273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55611b397ad0 0x55611b397540 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe140000c00 tx=0x7fe14000fe80 comp rx=0 tx=0).stop 2023-12-07T10:27:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.040+0000 7fe169822b80 1 -- 172.21.15.119:0/672659273 shutdown_connections 2023-12-07T10:27:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.040+0000 7fe169822b80 1 --2- 172.21.15.119:0/672659273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55611b39d380 0x55611b3a7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.040+0000 7fe169822b80 1 --2- 172.21.15.119:0/672659273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55611b397ad0 0x55611b397540 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.040+0000 7fe169822b80 1 --2- 172.21.15.119:0/672659273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55611b39ca60 0x55611b3a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.040+0000 7fe169822b80 1 -- 172.21.15.119:0/672659273 >> 172.21.15.119:0/672659273 conn(0x55611b2c6300 msgr2=0x55611b3919b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.040+0000 7fe169822b80 1 -- 172.21.15.119:0/672659273 shutdown_connections 2023-12-07T10:27:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.040+0000 7fe169822b80 1 -- 172.21.15.119:0/672659273 wait complete. 2023-12-07T10:27:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.040+0000 7fe169822b80 1 -- start start 2023-12-07T10:27:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.040+0000 7fe169822b80 1 Processor -- start 2023-12-07T10:27:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.041+0000 7fe169822b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55611b39ca60 0x55611b3a7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.041+0000 7fe169822b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55611b39d380 0x55611b3a82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.041+0000 7fe169822b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55611b3a5da0 0x55611b3a6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.041+0000 7fe169822b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55611b3a6900 con 0x55611b39d380 2023-12-07T10:27:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.041+0000 7fe169822b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55611b3a6a70 con 0x55611b39ca60 2023-12-07T10:27:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.041+0000 7fe169822b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55611b406320 con 0x55611b3a5da0 2023-12-07T10:27:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.041+0000 7fe169822b80 10 client.0 _ll_register_callbacks cb 0x55611b28c8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:27:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.041+0000 7fe14f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55611b39ca60 0x55611b3a7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.041+0000 7fe14ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55611b3a5da0 0x55611b3a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.041+0000 7fe14f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55611b39ca60 0x55611b3a7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:46892/0 (socket says 172.21.15.119:46892) 2023-12-07T10:27:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.041+0000 7fe14ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55611b3a5da0 0x55611b3a6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:54412/0 (socket says 172.21.15.119:54412) 2023-12-07T10:27:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.041+0000 7fe14f7fe700 1 -- 172.21.15.119:0/4181229035 learned_addr learned my addr 172.21.15.119:0/4181229035 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:27:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.041+0000 7fe14effd700 1 --2- 172.21.15.119:0/4181229035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55611b39d380 0x55611b3a82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.042+0000 7fe14f7fe700 1 -- 172.21.15.119:0/4181229035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55611b3a5da0 msgr2=0x55611b3a6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.042+0000 7fe14f7fe700 1 --2- 172.21.15.119:0/4181229035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55611b3a5da0 0x55611b3a6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.042+0000 7fe14f7fe700 1 -- 172.21.15.119:0/4181229035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55611b39d380 msgr2=0x55611b3a82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.042+0000 7fe14f7fe700 1 --2- 172.21.15.119:0/4181229035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55611b39d380 0x55611b3a82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.042+0000 7fe14f7fe700 1 -- 172.21.15.119:0/4181229035 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55611b114d80 con 0x55611b39ca60 2023-12-07T10:27:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.042+0000 7fe14f7fe700 1 --2- 172.21.15.119:0/4181229035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55611b39ca60 0x55611b3a7d70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe140009990 tx=0x7fe14000fda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c12415a04b8276da server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.042+0000 7fe12effd700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:27:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.043+0000 7fe12effd700 1 -- 172.21.15.119:0/4181229035 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14000f500 con 0x55611b39ca60 2023-12-07T10:27:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.043+0000 7fe12effd700 1 -- 172.21.15.119:0/4181229035 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fe14003f430 con 0x55611b39ca60 2023-12-07T10:27:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.043+0000 7fe169822b80 1 -- 172.21.15.119:0/4181229035 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe1440850b0 con 0x55611b39ca60 2023-12-07T10:27:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.044+0000 7fe12effd700 1 -- 172.21.15.119:0/4181229035 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe140004ed0 con 0x55611b39ca60 2023-12-07T10:27:45.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.044+0000 7fe169822b80 10 client.27404 fetch_fsmap learned FSMap version 52 2023-12-07T10:27:45.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.044+0000 7fe169822b80 1 -- 172.21.15.119:0/4181229035 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55611b13b1f0 con 0x55611b39ca60 2023-12-07T10:27:45.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.044+0000 7fe12effd700 1 -- 172.21.15.119:0/4181229035 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1440850b0 con 0x55611b39ca60 2023-12-07T10:27:45.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.044+0000 7fe12effd700 1 -- 172.21.15.119:0/4181229035 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 52) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fe140037070 con 0x55611b39ca60 2023-12-07T10:27:45.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.044+0000 7fe169822b80 10 client.27404 fetch_fsmap finished waiting for FSMap version 52 2023-12-07T10:27:45.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.044+0000 7fe169822b80 10 client.27404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:27:45.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.044+0000 7fe169822b80 20 client.27404 populate_metadata read hostname 'smithi119' 2023-12-07T10:27:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.044+0000 7fe169822b80 1 --2- 172.21.15.119:0/4181229035 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55611b407500 0x55611b4098f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.044+0000 7fe169822b80 4 client.27404 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:27:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.044+0000 7fe169822b80 1 -- 172.21.15.119:0/4181229035 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55611b2b1f70 con 0x55611b407500 2023-12-07T10:27:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.045+0000 7fe14effd700 1 --2- 172.21.15.119:0/4181229035 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55611b407500 0x55611b4098f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.045+0000 7fe14effd700 1 --2- 172.21.15.119:0/4181229035 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55611b407500 0x55611b4098f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5743705c3bff3059 server_cookie=3b556f8a5ec6ddb2 in_seq=0 out_seq=0 2023-12-07T10:27:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.045+0000 7fe12effd700 10 client.27404 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:27:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.094+0000 7fe12effd700 1 -- 172.21.15.119:0/4181229035 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55611b2b1f70 con 0x55611b407500 2023-12-07T10:27:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.094+0000 7fe12effd700 10 client.27404 handle_command_reply: tid=0 2023-12-07T10:27:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.094+0000 7fe169822b80 1 client.27404 shutdown 2023-12-07T10:27:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.094+0000 7fe169822b80 1 -- 172.21.15.119:0/4181229035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55611b39ca60 msgr2=0x55611b3a7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.094+0000 7fe169822b80 1 --2- 172.21.15.119:0/4181229035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55611b39ca60 0x55611b3a7d70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe140009990 tx=0x7fe14000fda0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.094+0000 7fe169822b80 1 -- 172.21.15.119:0/4181229035 shutdown_connections 2023-12-07T10:27:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.094+0000 7fe169822b80 1 --2- 172.21.15.119:0/4181229035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55611b3a5da0 0x55611b3a6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.094+0000 7fe169822b80 1 --2- 172.21.15.119:0/4181229035 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55611b407500 0x55611b4098f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.094+0000 7fe169822b80 1 --2- 172.21.15.119:0/4181229035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55611b39ca60 0x55611b3a7d70 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.094+0000 7fe169822b80 1 --2- 172.21.15.119:0/4181229035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55611b39d380 0x55611b3a82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.094+0000 7fe169822b80 1 -- 172.21.15.119:0/4181229035 >> 172.21.15.119:0/4181229035 conn(0x55611b2c6300 msgr2=0x55611b39f050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.094+0000 7fe12effd700 0 client.27404 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:27:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.094+0000 7fe169822b80 1 -- 172.21.15.119:0/4181229035 shutdown_connections 2023-12-07T10:27:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.094+0000 7fe169822b80 1 -- 172.21.15.119:0/4181229035 wait complete. 2023-12-07T10:27:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.095+0000 7fe169822b80 20 client.27404 trim_cache size 0 max 16384 2023-12-07T10:27:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:27:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:27:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:27:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:27:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.095+0000 7fe155dff700 1 -- 172.21.15.119:0/1751583749 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe1380a1c80 msgr2=0x7fe1380a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.095+0000 7fe155dff700 1 --2- 172.21.15.119:0/1751583749 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe1380a1c80 0x7fe1380a4140 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe14000b2e0 tx=0x7fe140005960 comp rx=0 tx=0).stop 2023-12-07T10:27:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.095+0000 7fe155dff700 1 -- 172.21.15.119:0/1751583749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1501040d0 msgr2=0x7fe15019fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.095+0000 7fe155dff700 1 --2- 172.21.15.119:0/1751583749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1501040d0 0x7fe15019fdb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe14400e9f0 tx=0x7fe14400ed00 comp rx=0 tx=0).stop 2023-12-07T10:27:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.095+0000 7fe155dff700 1 -- 172.21.15.119:0/1751583749 shutdown_connections 2023-12-07T10:27:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.095+0000 7fe155dff700 1 --2- 172.21.15.119:0/1751583749 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe1380a1c80 0x7fe1380a4140 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.095+0000 7fe155dff700 1 --2- 172.21.15.119:0/1751583749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe150109720 0x7fe1501999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.095+0000 7fe155dff700 1 --2- 172.21.15.119:0/1751583749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe150103710 0x7fe15019f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.095+0000 7fe155dff700 1 --2- 172.21.15.119:0/1751583749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1501040d0 0x7fe15019fdb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.096+0000 7fe155dff700 1 -- 172.21.15.119:0/1751583749 >> 172.21.15.119:0/1751583749 conn(0x7fe1500ff1c0 msgr2=0x7fe1500ffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.096+0000 7fe155dff700 1 -- 172.21.15.119:0/1751583749 shutdown_connections 2023-12-07T10:27:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:45.096+0000 7fe155dff700 1 -- 172.21.15.119:0/1751583749 wait complete. 2023-12-07T10:27:45.566 INFO:tasks.workunit.client.0.smithi119.stdout: 1 494146 7.00 MB/sec execute 238 sec latency 45.608 ms 2023-12-07T10:27:46.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:46 smithi119 ceph-mon[97908]: pgmap v225: 129 pgs: 129 active+clean; 310 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1.4 MiB/s wr, 189 op/s 2023-12-07T10:27:46.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:46 smithi159 ceph-mon[134726]: pgmap v225: 129 pgs: 129 active+clean; 310 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1.4 MiB/s wr, 189 op/s 2023-12-07T10:27:46.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:46 smithi153 ceph-mon[133625]: pgmap v225: 129 pgs: 129 active+clean; 310 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1.4 MiB/s wr, 189 op/s 2023-12-07T10:27:46.563 INFO:tasks.workunit.client.0.smithi119.stdout: 1 495501 7.00 MB/sec execute 239 sec latency 13.499 ms 2023-12-07T10:27:47.563 INFO:tasks.workunit.client.0.smithi119.stdout: 1 497610 7.03 MB/sec execute 240 sec latency 15.149 ms 2023-12-07T10:27:48.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:48 smithi119 ceph-mon[97908]: pgmap v226: 129 pgs: 129 active+clean; 317 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 283 op/s 2023-12-07T10:27:48.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:48 smithi159 ceph-mon[134726]: pgmap v226: 129 pgs: 129 active+clean; 317 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 283 op/s 2023-12-07T10:27:48.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:48 smithi153 ceph-mon[133625]: pgmap v226: 129 pgs: 129 active+clean; 317 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 283 op/s 2023-12-07T10:27:48.564 INFO:tasks.workunit.client.0.smithi119.stdout: 1 498826 7.02 MB/sec execute 241 sec latency 17.116 ms 2023-12-07T10:27:49.564 INFO:tasks.workunit.client.0.smithi119.stdout: 1 500610 7.02 MB/sec execute 242 sec latency 28.618 ms 2023-12-07T10:27:50.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:50 smithi119 ceph-mon[97908]: pgmap v227: 129 pgs: 129 active+clean; 317 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1.3 MiB/s wr, 182 op/s 2023-12-07T10:27:50.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:50 smithi159 ceph-mon[134726]: pgmap v227: 129 pgs: 129 active+clean; 317 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1.3 MiB/s wr, 182 op/s 2023-12-07T10:27:50.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:50 smithi153 ceph-mon[133625]: pgmap v227: 129 pgs: 129 active+clean; 317 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1.3 MiB/s wr, 182 op/s 2023-12-07T10:27:50.564 INFO:tasks.workunit.client.0.smithi119.stdout: 1 501816 7.02 MB/sec execute 243 sec latency 18.997 ms 2023-12-07T10:27:51.564 INFO:tasks.workunit.client.0.smithi119.stdout: 1 503708 7.04 MB/sec execute 244 sec latency 17.053 ms 2023-12-07T10:27:52.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:51 smithi119 ceph-mon[97908]: pgmap v228: 129 pgs: 129 active+clean; 321 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 246 op/s 2023-12-07T10:27:52.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:51 smithi159 ceph-mon[134726]: pgmap v228: 129 pgs: 129 active+clean; 321 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 246 op/s 2023-12-07T10:27:52.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:51 smithi153 ceph-mon[133625]: pgmap v228: 129 pgs: 129 active+clean; 321 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 246 op/s 2023-12-07T10:27:52.564 INFO:tasks.workunit.client.0.smithi119.stdout: 1 505056 7.04 MB/sec execute 245 sec latency 11.275 ms 2023-12-07T10:27:53.564 INFO:tasks.workunit.client.0.smithi119.stdout: 1 506792 7.04 MB/sec execute 246 sec latency 23.361 ms 2023-12-07T10:27:54.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:54 smithi119 ceph-mon[97908]: pgmap v229: 129 pgs: 129 active+clean; 324 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 218 op/s 2023-12-07T10:27:54.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:54 smithi159 ceph-mon[134726]: pgmap v229: 129 pgs: 129 active+clean; 324 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 218 op/s 2023-12-07T10:27:54.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:54 smithi153 ceph-mon[133625]: pgmap v229: 129 pgs: 129 active+clean; 324 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 218 op/s 2023-12-07T10:27:54.564 INFO:tasks.workunit.client.0.smithi119.stdout: 1 508307 7.06 MB/sec execute 247 sec latency 18.462 ms 2023-12-07T10:27:55.564 INFO:tasks.workunit.client.0.smithi119.stdout: 1 509533 7.05 MB/sec execute 248 sec latency 13.971 ms 2023-12-07T10:27:56.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:56 smithi119 ceph-mon[97908]: pgmap v230: 129 pgs: 129 active+clean; 324 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.4 MiB/s wr, 188 op/s 2023-12-07T10:27:56.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:56 smithi159 ceph-mon[134726]: pgmap v230: 129 pgs: 129 active+clean; 324 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.4 MiB/s wr, 188 op/s 2023-12-07T10:27:56.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:56 smithi153 ceph-mon[133625]: pgmap v230: 129 pgs: 129 active+clean; 324 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.4 MiB/s wr, 188 op/s 2023-12-07T10:27:56.564 INFO:tasks.workunit.client.0.smithi119.stdout: 1 511546 7.07 MB/sec execute 249 sec latency 15.856 ms 2023-12-07T10:27:57.564 INFO:tasks.workunit.client.0.smithi119.stdout: 1 512388 7.05 MB/sec execute 250 sec latency 87.536 ms 2023-12-07T10:27:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.327+0000 7fe863dff700 1 -- 172.21.15.119:0/1782051458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe85c076460 msgr2=0x7fe85c111e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.327+0000 7fe863dff700 1 --2- 172.21.15.119:0/1782051458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe85c076460 0x7fe85c111e70 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009990 tx=0x7fe84c009ca0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.327+0000 7fe863dff700 1 -- 172.21.15.119:0/1782051458 shutdown_connections 2023-12-07T10:27:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.327+0000 7fe863dff700 1 --2- 172.21.15.119:0/1782051458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe85c1123b0 0x7fe85c1147a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.327+0000 7fe863dff700 1 --2- 172.21.15.119:0/1782051458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe85c075b40 0x7fe85c075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.327+0000 7fe863dff700 1 --2- 172.21.15.119:0/1782051458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe85c076460 0x7fe85c111e70 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.327+0000 7fe863dff700 1 -- 172.21.15.119:0/1782051458 >> 172.21.15.119:0/1782051458 conn(0x7fe85c0ff1c0 msgr2=0x7fe85c1015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.327+0000 7fe863dff700 1 -- 172.21.15.119:0/1782051458 shutdown_connections 2023-12-07T10:27:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.328+0000 7fe863dff700 1 -- 172.21.15.119:0/1782051458 wait complete. 2023-12-07T10:27:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.330+0000 7fe863dff700 1 Processor -- start 2023-12-07T10:27:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.330+0000 7fe863dff700 1 -- start start 2023-12-07T10:27:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.330+0000 7fe863dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe85c075b40 0x7fe85c19e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.331+0000 7fe863dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe85c076460 0x7fe85c19e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.331+0000 7fe863dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe85c1123b0 0x7fe85c1a2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.331+0000 7fe863dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c1a2fd0 con 0x7fe85c1123b0 2023-12-07T10:27:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.331+0000 7fe863dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c1a3140 con 0x7fe85c076460 2023-12-07T10:27:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.331+0000 7fe863dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c1a3300 con 0x7fe85c075b40 2023-12-07T10:27:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.331+0000 7fe86239c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe85c1123b0 0x7fe85c1a2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.331+0000 7fe86139a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe85c076460 0x7fe85c19e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.331+0000 7fe86139a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe85c076460 0x7fe85c19e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:40764/0 (socket says 172.21.15.119:40764) 2023-12-07T10:27:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.331+0000 7fe86239c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe85c1123b0 0x7fe85c1a2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39966/0 (socket says 172.21.15.119:39966) 2023-12-07T10:27:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.331+0000 7fe86139a700 1 -- 172.21.15.119:0/2887396345 learned_addr learned my addr 172.21.15.119:0/2887396345 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:27:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.331+0000 7fe861b9b700 1 --2- 172.21.15.119:0/2887396345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe85c075b40 0x7fe85c19e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.332+0000 7fe861b9b700 1 -- 172.21.15.119:0/2887396345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe85c076460 msgr2=0x7fe85c19e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.332+0000 7fe861b9b700 1 --2- 172.21.15.119:0/2887396345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe85c076460 0x7fe85c19e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.332+0000 7fe861b9b700 1 -- 172.21.15.119:0/2887396345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe85c1123b0 msgr2=0x7fe85c1a2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:58.333 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.333+0000 7fe861b9b700 1 --2- 172.21.15.119:0/2887396345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe85c1123b0 0x7fe85c1a2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.333 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.333+0000 7fe861b9b700 1 -- 172.21.15.119:0/2887396345 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84c0095f0 con 0x7fe85c075b40 2023-12-07T10:27:58.333 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.333+0000 7fe86239c700 1 --2- 172.21.15.119:0/2887396345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe85c1123b0 0x7fe85c1a2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:27:58.334 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.334+0000 7fe861b9b700 1 --2- 172.21.15.119:0/2887396345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe85c075b40 0x7fe85c19e0b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe85800b450 tx=0x7fe85800b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.335+0000 7fe852ffd700 1 -- 172.21.15.119:0/2887396345 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe858011830 con 0x7fe85c075b40 2023-12-07T10:27:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.335+0000 7fe852ffd700 1 -- 172.21.15.119:0/2887396345 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe85800be60 con 0x7fe85c075b40 2023-12-07T10:27:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.335+0000 7fe863dff700 1 -- 172.21.15.119:0/2887396345 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe85c1a35e0 con 0x7fe85c075b40 2023-12-07T10:27:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.335+0000 7fe852ffd700 1 -- 172.21.15.119:0/2887396345 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe858010d20 con 0x7fe85c075b40 2023-12-07T10:27:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.335+0000 7fe863dff700 1 -- 172.21.15.119:0/2887396345 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe85c1aae20 con 0x7fe85c075b40 2023-12-07T10:27:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.342+0000 7fe852ffd700 1 -- 172.21.15.119:0/2887396345 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fe85801a400 con 0x7fe85c075b40 2023-12-07T10:27:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.345+0000 7fe8777f6b80 1 Processor -- start 2023-12-07T10:27:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.345+0000 7fe8777f6b80 1 -- start start 2023-12-07T10:27:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.347+0000 7fe8777f6b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f85a60 0x557e56f85e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.347+0000 7fe8777f6b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e56f86380 0x557e56f80540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.347+0000 7fe852ffd700 1 --2- 172.21.15.119:0/2887396345 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe8480a1d80 0x7fe8480a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.347+0000 7fe86239c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f85a60 0x557e56f85e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.347+0000 7fe86239c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f85a60 0x557e56f85e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:38816/0 (socket says 172.21.15.119:38816) 2023-12-07T10:27:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.347+0000 7fe86239c700 1 -- 172.21.15.119:0/95293863 learned_addr learned my addr 172.21.15.119:0/95293863 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:27:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.347+0000 7fe852ffd700 1 -- 172.21.15.119:0/2887396345 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fe8580cd580 con 0x7fe85c075b40 2023-12-07T10:27:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.348+0000 7fe861b9b700 1 --2- 172.21.15.119:0/95293863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e56f86380 0x557e56f80540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.348+0000 7fe8777f6b80 1 --2- 172.21.15.119:0/95293863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e56f80ad0 0x557e56f80f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.348+0000 7fe8777f6b80 1 -- 172.21.15.119:0/95293863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e56ef4760 con 0x557e56f86380 2023-12-07T10:27:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.348+0000 7fe8777f6b80 1 -- 172.21.15.119:0/95293863 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e56edb740 con 0x557e56f80ad0 2023-12-07T10:27:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.348+0000 7fe8777f6b80 1 -- 172.21.15.119:0/95293863 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e56ed9060 con 0x557e56f85a60 2023-12-07T10:27:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.348+0000 7fe86139a700 1 --2- 172.21.15.119:0/2887396345 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe8480a1d80 0x7fe8480a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.349+0000 7fe86239c700 1 --2- 172.21.15.119:0/95293863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e56f80ad0 0x557e56f80f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.349+0000 7fe861b9b700 1 -- 172.21.15.119:0/95293863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f85a60 msgr2=0x557e56f85e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.349+0000 7fe861b9b700 1 --2- 172.21.15.119:0/95293863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f85a60 0x557e56f85e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.349+0000 7fe861b9b700 1 -- 172.21.15.119:0/95293863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e56f80ad0 msgr2=0x557e56f80f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.349+0000 7fe861b9b700 1 --2- 172.21.15.119:0/95293863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e56f80ad0 0x557e56f80f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.349+0000 7fe861b9b700 1 -- 172.21.15.119:0/95293863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e56d515d0 con 0x557e56f86380 2023-12-07T10:27:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.350+0000 7fe861b9b700 1 --2- 172.21.15.119:0/95293863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e56f86380 0x557e56f80540 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe8580ca9d0 tx=0x7fe858099980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a12aa2356771e39 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.351+0000 7fe847fff700 1 -- 172.21.15.119:0/95293863 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85809e170 con 0x557e56f86380 2023-12-07T10:27:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.351+0000 7fe847fff700 1 -- 172.21.15.119:0/95293863 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8580a23b0 con 0x557e56f86380 2023-12-07T10:27:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.351+0000 7fe847fff700 1 -- 172.21.15.119:0/95293863 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe858059cb0 con 0x557e56f86380 2023-12-07T10:27:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.352+0000 7fe8777f6b80 1 -- 172.21.15.119:0/95293863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e56f86380 msgr2=0x557e56f80540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.352+0000 7fe8777f6b80 1 --2- 172.21.15.119:0/95293863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e56f86380 0x557e56f80540 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe8580ca9d0 tx=0x7fe858099980 comp rx=0 tx=0).stop 2023-12-07T10:27:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.352+0000 7fe86139a700 1 --2- 172.21.15.119:0/2887396345 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe8480a1d80 0x7fe8480a4240 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe85c19f600 tx=0x7fe84c00b260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.352+0000 7fe8777f6b80 1 -- 172.21.15.119:0/95293863 shutdown_connections 2023-12-07T10:27:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.352+0000 7fe8777f6b80 1 --2- 172.21.15.119:0/95293863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e56f80ad0 0x557e56f80f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.352+0000 7fe8777f6b80 1 --2- 172.21.15.119:0/95293863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f85a60 0x557e56f85e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.352+0000 7fe8777f6b80 1 --2- 172.21.15.119:0/95293863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e56f86380 0x557e56f80540 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.352+0000 7fe8777f6b80 1 -- 172.21.15.119:0/95293863 >> 172.21.15.119:0/95293863 conn(0x557e56eaf300 msgr2=0x557e56f7ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.353+0000 7fe8777f6b80 1 -- 172.21.15.119:0/95293863 shutdown_connections 2023-12-07T10:27:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.353+0000 7fe8777f6b80 1 -- 172.21.15.119:0/95293863 wait complete. 2023-12-07T10:27:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.354+0000 7fe8777f6b80 1 -- start start 2023-12-07T10:27:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.355+0000 7fe8777f6b80 1 Processor -- start 2023-12-07T10:27:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.356+0000 7fe8777f6b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f80ad0 0x557e56f8c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.356+0000 7fe8777f6b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e56f85a60 0x557e56f895d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.357+0000 7fe8777f6b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e56f86380 0x557e56f89c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.357+0000 7fe861b9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f80ad0 0x557e56f8c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.357+0000 7fe8777f6b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e56ed4950 con 0x557e56f86380 2023-12-07T10:27:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.357+0000 7fe861b9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f80ad0 0x557e56f8c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:38828/0 (socket says 172.21.15.119:38828) 2023-12-07T10:27:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.357+0000 7fe861b9b700 1 -- 172.21.15.119:0/499193679 learned_addr learned my addr 172.21.15.119:0/499193679 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:27:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.357+0000 7fe8777f6b80 1 -- 172.21.15.119:0/499193679 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e56e8e500 con 0x557e56f85a60 2023-12-07T10:27:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.357+0000 7fe8777f6b80 1 -- 172.21.15.119:0/499193679 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e56e8e1a0 con 0x557e56f80ad0 2023-12-07T10:27:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.357+0000 7fe8777f6b80 10 client.0 _ll_register_callbacks cb 0x557e56e758e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:27:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.357+0000 7fe86239c700 1 --2- 172.21.15.119:0/499193679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e56f86380 0x557e56f89c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.357+0000 7fe86139a700 1 --2- 172.21.15.119:0/499193679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e56f85a60 0x557e56f895d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.358+0000 7fe861b9b700 1 -- 172.21.15.119:0/499193679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e56f85a60 msgr2=0x557e56f895d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.358+0000 7fe861b9b700 1 --2- 172.21.15.119:0/499193679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e56f85a60 0x557e56f895d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.358+0000 7fe861b9b700 1 -- 172.21.15.119:0/499193679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e56f86380 msgr2=0x557e56f89c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.358+0000 7fe861b9b700 1 --2- 172.21.15.119:0/499193679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e56f86380 0x557e56f89c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.358+0000 7fe861b9b700 1 -- 172.21.15.119:0/499193679 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe85c1aae20 con 0x557e56f80ad0 2023-12-07T10:27:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.359+0000 7fe861b9b700 1 --2- 172.21.15.119:0/499193679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f80ad0 0x557e56f8c2e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe85800bfd0 tx=0x7fe85809a160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=515d20efacc76dc9 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.359+0000 7fe8457fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:27:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.360+0000 7fe8457fa700 1 -- 172.21.15.119:0/499193679 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8580cd130 con 0x557e56f80ad0 2023-12-07T10:27:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.360+0000 7fe8457fa700 1 -- 172.21.15.119:0/499193679 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fe8580cd290 con 0x557e56f80ad0 2023-12-07T10:27:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.360+0000 7fe8777f6b80 1 -- 172.21.15.119:0/499193679 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e56f736f0 con 0x557e56f80ad0 2023-12-07T10:27:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.360+0000 7fe8457fa700 1 -- 172.21.15.119:0/499193679 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe85809a280 con 0x557e56f80ad0 2023-12-07T10:27:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.361+0000 7fe8777f6b80 10 client.27397 fetch_fsmap learned FSMap version 52 2023-12-07T10:27:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.361+0000 7fe8777f6b80 1 -- 172.21.15.119:0/499193679 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x557e56cfdd80 con 0x557e56f80ad0 2023-12-07T10:27:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.361+0000 7fe8457fa700 1 -- 172.21.15.119:0/499193679 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557e56f736f0 con 0x557e56f80ad0 2023-12-07T10:27:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.361+0000 7fe8457fa700 1 -- 172.21.15.119:0/499193679 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 52) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fe8580a1f20 con 0x557e56f80ad0 2023-12-07T10:27:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.362+0000 7fe8777f6b80 10 client.27397 fetch_fsmap finished waiting for FSMap version 52 2023-12-07T10:27:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.362+0000 7fe8777f6b80 10 client.27397 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:27:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.362+0000 7fe8777f6b80 20 client.27397 populate_metadata read hostname 'smithi119' 2023-12-07T10:27:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.362+0000 7fe8777f6b80 1 --2- 172.21.15.119:0/499193679 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x557e56f99ae0 0x557e56f9bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.362+0000 7fe8777f6b80 4 client.27397 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:27:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.362+0000 7fe8777f6b80 1 -- 172.21.15.119:0/499193679 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e56f4f740 con 0x557e56f99ae0 2023-12-07T10:27:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.363+0000 7fe86139a700 1 --2- 172.21.15.119:0/499193679 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x557e56f99ae0 0x557e56f9bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.372+0000 7fe8457fa700 10 client.27397 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:27:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.372+0000 7fe86139a700 1 --2- 172.21.15.119:0/499193679 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x557e56f99ae0 0x557e56f9bed0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c4da6a28ab80337 server_cookie=7b6f4c5aa2406a2a in_seq=0 out_seq=0 2023-12-07T10:27:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.374+0000 7fe8457fa700 1 -- 172.21.15.119:0/499193679 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557e56f4f740 con 0x557e56f99ae0 2023-12-07T10:27:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.374+0000 7fe8457fa700 10 client.27397 handle_command_reply: tid=0 2023-12-07T10:27:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.374+0000 7fe8777f6b80 1 client.27397 shutdown 2023-12-07T10:27:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.374+0000 7fe8777f6b80 1 -- 172.21.15.119:0/499193679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f80ad0 msgr2=0x557e56f8c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.374+0000 7fe8777f6b80 1 --2- 172.21.15.119:0/499193679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f80ad0 0x557e56f8c2e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe85800bfd0 tx=0x7fe85809a160 comp rx=0 tx=0).stop 2023-12-07T10:27:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.375+0000 7fe8777f6b80 1 -- 172.21.15.119:0/499193679 shutdown_connections 2023-12-07T10:27:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.375+0000 7fe8777f6b80 1 --2- 172.21.15.119:0/499193679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e56f86380 0x557e56f89c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.375+0000 7fe8777f6b80 1 --2- 172.21.15.119:0/499193679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f80ad0 0x557e56f8c2e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.375+0000 7fe8777f6b80 1 --2- 172.21.15.119:0/499193679 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x557e56f99ae0 0x557e56f9bed0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.375+0000 7fe8777f6b80 1 --2- 172.21.15.119:0/499193679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e56f85a60 0x557e56f895d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.375+0000 7fe8777f6b80 1 -- 172.21.15.119:0/499193679 >> 172.21.15.119:0/499193679 conn(0x557e56eaf300 msgr2=0x557e56f75980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.375+0000 7fe8457fa700 0 client.27397 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:27:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.375+0000 7fe8777f6b80 1 -- 172.21.15.119:0/499193679 shutdown_connections 2023-12-07T10:27:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.375+0000 7fe8777f6b80 1 -- 172.21.15.119:0/499193679 wait complete. 2023-12-07T10:27:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.376+0000 7fe8777f6b80 20 client.27397 trim_cache size 0 max 16384 2023-12-07T10:27:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.409+0000 7fe8777f6b80 1 Processor -- start 2023-12-07T10:27:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.410+0000 7fe8777f6b80 1 -- start start 2023-12-07T10:27:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.410+0000 7fe8777f6b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f80ad0 0x557e56f80540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.410+0000 7fe8777f6b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e56f85a60 0x557e56f958b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.410+0000 7fe8777f6b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e56f86380 0x557e56f95df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.410+0000 7fe8777f6b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e56d538c0 con 0x557e56f86380 2023-12-07T10:27:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.410+0000 7fe8777f6b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e56f856b0 con 0x557e56f85a60 2023-12-07T10:27:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.410+0000 7fe8777f6b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e56f777a0 con 0x557e56f80ad0 2023-12-07T10:27:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.411+0000 7fe861b9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f80ad0 0x557e56f80540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.411+0000 7fe86239c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e56f86380 0x557e56f95df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.411+0000 7fe86139a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e56f85a60 0x557e56f958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.411+0000 7fe861b9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f80ad0 0x557e56f80540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:38842/0 (socket says 172.21.15.119:38842) 2023-12-07T10:27:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.411+0000 7fe861b9b700 1 -- 172.21.15.119:0/1842693649 learned_addr learned my addr 172.21.15.119:0/1842693649 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:27:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.412+0000 7fe861b9b700 1 -- 172.21.15.119:0/1842693649 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e56f85a60 msgr2=0x557e56f958b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.412+0000 7fe861b9b700 1 --2- 172.21.15.119:0/1842693649 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e56f85a60 0x557e56f958b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.412+0000 7fe861b9b700 1 -- 172.21.15.119:0/1842693649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e56f86380 msgr2=0x557e56f95df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.412+0000 7fe861b9b700 1 --2- 172.21.15.119:0/1842693649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e56f86380 0x557e56f95df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.412+0000 7fe861b9b700 1 -- 172.21.15.119:0/1842693649 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e56ce6c00 con 0x557e56f80ad0 2023-12-07T10:27:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.413+0000 7fe861b9b700 1 --2- 172.21.15.119:0/1842693649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f80ad0 0x557e56f80540 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe858009fd0 tx=0x7fe858099980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20491e4f3fa5d440 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.413+0000 7fe850bff700 1 -- 172.21.15.119:0/1842693649 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe858099510 con 0x557e56f80ad0 2023-12-07T10:27:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.413+0000 7fe850bff700 1 -- 172.21.15.119:0/1842693649 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe85809a720 con 0x557e56f80ad0 2023-12-07T10:27:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.413+0000 7fe850bff700 1 -- 172.21.15.119:0/1842693649 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8580a20a0 con 0x557e56f80ad0 2023-12-07T10:27:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.414+0000 7fe8777f6b80 1 -- 172.21.15.119:0/1842693649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f80ad0 msgr2=0x557e56f80540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.414+0000 7fe8777f6b80 1 --2- 172.21.15.119:0/1842693649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f80ad0 0x557e56f80540 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe858009fd0 tx=0x7fe858099980 comp rx=0 tx=0).stop 2023-12-07T10:27:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.415+0000 7fe8777f6b80 1 -- 172.21.15.119:0/1842693649 shutdown_connections 2023-12-07T10:27:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.415+0000 7fe8777f6b80 1 --2- 172.21.15.119:0/1842693649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e56f86380 0x557e56f95df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.415+0000 7fe8777f6b80 1 --2- 172.21.15.119:0/1842693649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f80ad0 0x557e56f80540 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.415+0000 7fe8777f6b80 1 --2- 172.21.15.119:0/1842693649 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e56f85a60 0x557e56f958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.415+0000 7fe8777f6b80 1 -- 172.21.15.119:0/1842693649 >> 172.21.15.119:0/1842693649 conn(0x557e56eaf300 msgr2=0x557e56f83820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.415+0000 7fe8777f6b80 1 -- 172.21.15.119:0/1842693649 shutdown_connections 2023-12-07T10:27:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.415+0000 7fe8777f6b80 1 -- 172.21.15.119:0/1842693649 wait complete. 2023-12-07T10:27:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.416+0000 7fe8777f6b80 1 -- start start 2023-12-07T10:27:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.417+0000 7fe8777f6b80 1 Processor -- start 2023-12-07T10:27:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.418+0000 7fe8777f6b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e56f80ad0 0x557e56f912d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.418+0000 7fe8777f6b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f85a60 0x557e56f829c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.418+0000 7fe8777f6b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e56f86380 0x557e56f815c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.418+0000 7fe8777f6b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e56f832d0 con 0x557e56f80ad0 2023-12-07T10:27:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.419+0000 7fe8777f6b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e56f81b00 con 0x557e56f86380 2023-12-07T10:27:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.419+0000 7fe8777f6b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e56f81cc0 con 0x557e56f85a60 2023-12-07T10:27:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.419+0000 7fe86139a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f85a60 0x557e56f829c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.419+0000 7fe86139a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f85a60 0x557e56f829c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:38856/0 (socket says 172.21.15.119:38856) 2023-12-07T10:27:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.419+0000 7fe86139a700 1 -- 172.21.15.119:0/1725145018 learned_addr learned my addr 172.21.15.119:0/1725145018 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:27:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.419+0000 7fe86239c700 1 --2- 172.21.15.119:0/1725145018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e56f86380 0x557e56f815c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.419+0000 7fe8777f6b80 10 client.0 _ll_register_callbacks cb 0x557e56e758e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:27:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.419+0000 7fe861b9b700 1 --2- 172.21.15.119:0/1725145018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e56f80ad0 0x557e56f912d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.420+0000 7fe86139a700 1 -- 172.21.15.119:0/1725145018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e56f86380 msgr2=0x557e56f815c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.420+0000 7fe86139a700 1 --2- 172.21.15.119:0/1725145018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e56f86380 0x557e56f815c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.420+0000 7fe86139a700 1 -- 172.21.15.119:0/1725145018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e56f80ad0 msgr2=0x557e56f912d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:27:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.420+0000 7fe86139a700 1 --2- 172.21.15.119:0/1725145018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e56f80ad0 0x557e56f912d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.420+0000 7fe86139a700 1 -- 172.21.15.119:0/1725145018 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e56cfdd80 con 0x557e56f85a60 2023-12-07T10:27:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.420+0000 7fe861b9b700 1 --2- 172.21.15.119:0/1725145018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e56f80ad0 0x557e56f912d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:27:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.421+0000 7fe86139a700 1 --2- 172.21.15.119:0/1725145018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f85a60 0x557e56f829c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe84c000c00 tx=0x7fe84c009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1235790620636210 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:27:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.421+0000 7fe8457fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:27:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.422+0000 7fe8457fa700 1 -- 172.21.15.119:0/1725145018 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c035390 con 0x557e56f85a60 2023-12-07T10:27:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.422+0000 7fe8457fa700 1 -- 172.21.15.119:0/1725145018 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fe84c0354f0 con 0x557e56f85a60 2023-12-07T10:27:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.422+0000 7fe8777f6b80 1 -- 172.21.15.119:0/1725145018 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8580a23b0 con 0x557e56f85a60 2023-12-07T10:27:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.423+0000 7fe8457fa700 1 -- 172.21.15.119:0/1725145018 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe84c0356b0 con 0x557e56f85a60 2023-12-07T10:27:58.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.423+0000 7fe8777f6b80 10 client.27403 fetch_fsmap learned FSMap version 52 2023-12-07T10:27:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.423+0000 7fe8777f6b80 1 -- 172.21.15.119:0/1725145018 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x557e56d241f0 con 0x557e56f85a60 2023-12-07T10:27:58.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.424+0000 7fe8457fa700 1 -- 172.21.15.119:0/1725145018 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8580a23b0 con 0x557e56f85a60 2023-12-07T10:27:58.425 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.425+0000 7fe8457fa700 1 -- 172.21.15.119:0/1725145018 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 52) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fe84c00f460 con 0x557e56f85a60 2023-12-07T10:27:58.425 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.425+0000 7fe8777f6b80 10 client.27403 fetch_fsmap finished waiting for FSMap version 52 2023-12-07T10:27:58.425 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.425+0000 7fe8777f6b80 10 client.27403 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:27:58.425 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.425+0000 7fe8777f6b80 20 client.27403 populate_metadata read hostname 'smithi119' 2023-12-07T10:27:58.425 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.425+0000 7fe8777f6b80 1 --2- 172.21.15.119:0/1725145018 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x557e56ff0e20 0x557e56ff3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:27:58.425 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.425+0000 7fe8777f6b80 4 client.27403 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:27:58.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.425+0000 7fe8777f6b80 1 -- 172.21.15.119:0/1725145018 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557e56e9af70 con 0x557e56ff0e20 2023-12-07T10:27:58.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.425+0000 7fe861b9b700 1 --2- 172.21.15.119:0/1725145018 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x557e56ff0e20 0x557e56ff3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:27:58.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.426+0000 7fe861b9b700 1 --2- 172.21.15.119:0/1725145018 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x557e56ff0e20 0x557e56ff3240 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39e187bc23a770a9 server_cookie=ecc1a64f081844d3 in_seq=0 out_seq=0 2023-12-07T10:27:58.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.427+0000 7fe8457fa700 10 client.27403 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:27:58.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:27:58 smithi119 ceph-mon[97908]: pgmap v231: 129 pgs: 129 active+clean; 331 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 272 op/s 2023-12-07T10:27:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.490+0000 7fe8457fa700 1 -- 172.21.15.119:0/1725145018 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557e56e9af70 con 0x557e56ff0e20 2023-12-07T10:27:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.490+0000 7fe8457fa700 10 client.27403 handle_command_reply: tid=0 2023-12-07T10:27:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.490+0000 7fe8777f6b80 1 client.27403 shutdown 2023-12-07T10:27:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.490+0000 7fe8777f6b80 1 -- 172.21.15.119:0/1725145018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f85a60 msgr2=0x557e56f829c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.490+0000 7fe8777f6b80 1 --2- 172.21.15.119:0/1725145018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f85a60 0x557e56f829c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe84c000c00 tx=0x7fe84c009ca0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.490+0000 7fe8777f6b80 1 -- 172.21.15.119:0/1725145018 shutdown_connections 2023-12-07T10:27:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.490+0000 7fe8777f6b80 1 --2- 172.21.15.119:0/1725145018 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x557e56ff0e20 0x557e56ff3240 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.490+0000 7fe8777f6b80 1 --2- 172.21.15.119:0/1725145018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557e56f86380 0x557e56f815c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.490+0000 7fe8457fa700 0 client.27403 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:27:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.490+0000 7fe8777f6b80 1 --2- 172.21.15.119:0/1725145018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e56f80ad0 0x557e56f912d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.490+0000 7fe8777f6b80 1 --2- 172.21.15.119:0/1725145018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e56f85a60 0x557e56f829c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.490+0000 7fe8777f6b80 1 -- 172.21.15.119:0/1725145018 >> 172.21.15.119:0/1725145018 conn(0x557e56eaf300 msgr2=0x557e56f958b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.491+0000 7fe8777f6b80 1 -- 172.21.15.119:0/1725145018 shutdown_connections 2023-12-07T10:27:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.491+0000 7fe8777f6b80 1 -- 172.21.15.119:0/1725145018 wait complete. 2023-12-07T10:27:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.492+0000 7fe8777f6b80 20 client.27403 trim_cache size 0 max 16384 2023-12-07T10:27:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:27:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:27:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:27:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:27:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.493+0000 7fe863dff700 1 -- 172.21.15.119:0/2887396345 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe8480a1d80 msgr2=0x7fe8480a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.493+0000 7fe863dff700 1 --2- 172.21.15.119:0/2887396345 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe8480a1d80 0x7fe8480a4240 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe85c19f600 tx=0x7fe84c00b260 comp rx=0 tx=0).stop 2023-12-07T10:27:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.493+0000 7fe863dff700 1 -- 172.21.15.119:0/2887396345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe85c075b40 msgr2=0x7fe85c19e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:27:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.493+0000 7fe863dff700 1 --2- 172.21.15.119:0/2887396345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe85c075b40 0x7fe85c19e0b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe85800b450 tx=0x7fe85800b760 comp rx=0 tx=0).stop 2023-12-07T10:27:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.494+0000 7fe863dff700 1 -- 172.21.15.119:0/2887396345 shutdown_connections 2023-12-07T10:27:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.494+0000 7fe863dff700 1 --2- 172.21.15.119:0/2887396345 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe8480a1d80 0x7fe8480a4240 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.494+0000 7fe863dff700 1 --2- 172.21.15.119:0/2887396345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe85c1123b0 0x7fe85c1a2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.494+0000 7fe863dff700 1 --2- 172.21.15.119:0/2887396345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe85c075b40 0x7fe85c19e0b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.494+0000 7fe863dff700 1 --2- 172.21.15.119:0/2887396345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe85c076460 0x7fe85c19e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:27:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.494+0000 7fe863dff700 1 -- 172.21.15.119:0/2887396345 >> 172.21.15.119:0/2887396345 conn(0x7fe85c0ff1c0 msgr2=0x7fe85c1015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:27:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.494+0000 7fe863dff700 1 -- 172.21.15.119:0/2887396345 shutdown_connections 2023-12-07T10:27:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:27:58.495+0000 7fe863dff700 1 -- 172.21.15.119:0/2887396345 wait complete. 2023-12-07T10:27:58.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:27:58 smithi159 ceph-mon[134726]: pgmap v231: 129 pgs: 129 active+clean; 331 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 272 op/s 2023-12-07T10:27:58.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:27:58 smithi153 ceph-mon[133625]: pgmap v231: 129 pgs: 129 active+clean; 331 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 272 op/s 2023-12-07T10:27:58.564 INFO:tasks.workunit.client.0.smithi119.stdout: 1 512832 7.03 MB/sec execute 251 sec latency 273.768 ms 2023-12-07T10:27:59.564 INFO:tasks.workunit.client.0.smithi119.stdout: 1 514569 7.05 MB/sec execute 252 sec latency 35.833 ms 2023-12-07T10:28:00.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:00 smithi119 ceph-mon[97908]: pgmap v232: 129 pgs: 129 active+clean; 331 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 178 op/s 2023-12-07T10:28:00.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:00 smithi159 ceph-mon[134726]: pgmap v232: 129 pgs: 129 active+clean; 331 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 178 op/s 2023-12-07T10:28:00.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:00 smithi153 ceph-mon[133625]: pgmap v232: 129 pgs: 129 active+clean; 331 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 178 op/s 2023-12-07T10:28:00.564 INFO:tasks.workunit.client.0.smithi119.stdout: 1 515801 7.04 MB/sec execute 253 sec latency 12.996 ms 2023-12-07T10:28:01.564 INFO:tasks.workunit.client.0.smithi119.stdout: 1 516158 7.02 MB/sec execute 254 sec latency 65.612 ms 2023-12-07T10:28:02.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:01 smithi119 ceph-mon[97908]: pgmap v233: 129 pgs: 129 active+clean; 325 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 291 op/s 2023-12-07T10:28:02.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:01 smithi159 ceph-mon[134726]: pgmap v233: 129 pgs: 129 active+clean; 325 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 291 op/s 2023-12-07T10:28:02.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:01 smithi153 ceph-mon[133625]: pgmap v233: 129 pgs: 129 active+clean; 325 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 291 op/s 2023-12-07T10:28:02.564 INFO:tasks.workunit.client.0.smithi119.stdout: 1 518022 7.03 MB/sec execute 255 sec latency 59.246 ms 2023-12-07T10:28:03.564 INFO:tasks.workunit.client.0.smithi119.stdout: 1 519356 7.04 MB/sec execute 256 sec latency 15.025 ms 2023-12-07T10:28:04.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:04 smithi119 ceph-mon[97908]: pgmap v234: 129 pgs: 129 active+clean; 327 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 323 op/s 2023-12-07T10:28:04.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:04 smithi159 ceph-mon[134726]: pgmap v234: 129 pgs: 129 active+clean; 327 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 323 op/s 2023-12-07T10:28:04.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:04 smithi153 ceph-mon[133625]: pgmap v234: 129 pgs: 129 active+clean; 327 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 323 op/s 2023-12-07T10:28:04.564 INFO:tasks.workunit.client.0.smithi119.stdout: 1 520923 7.03 MB/sec execute 257 sec latency 40.928 ms 2023-12-07T10:28:05.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:05 smithi119 ceph-mon[97908]: osdmap e97: 12 total, 12 up, 12 in 2023-12-07T10:28:05.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:05 smithi159 ceph-mon[134726]: osdmap e97: 12 total, 12 up, 12 in 2023-12-07T10:28:05.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:05 smithi153 ceph-mon[133625]: osdmap e97: 12 total, 12 up, 12 in 2023-12-07T10:28:05.564 INFO:tasks.workunit.client.0.smithi119.stdout: 1 522533 7.05 MB/sec execute 258 sec latency 19.205 ms 2023-12-07T10:28:06.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:06 smithi119 ceph-mon[97908]: pgmap v236: 129 pgs: 129 active+clean; 327 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.5 MiB/s wr, 351 op/s 2023-12-07T10:28:06.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:06 smithi159 ceph-mon[134726]: pgmap v236: 129 pgs: 129 active+clean; 327 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.5 MiB/s wr, 351 op/s 2023-12-07T10:28:06.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:06 smithi153 ceph-mon[133625]: pgmap v236: 129 pgs: 129 active+clean; 327 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.5 MiB/s wr, 351 op/s 2023-12-07T10:28:06.564 INFO:tasks.workunit.client.0.smithi119.stdout: 1 523873 7.05 MB/sec execute 259 sec latency 14.298 ms 2023-12-07T10:28:07.565 INFO:tasks.workunit.client.0.smithi119.stdout: 1 525986 7.07 MB/sec execute 260 sec latency 15.256 ms 2023-12-07T10:28:08.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:08 smithi119 ceph-mon[97908]: pgmap v237: 129 pgs: 129 active+clean; 330 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.5 MiB/s wr, 351 op/s 2023-12-07T10:28:08.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:08 smithi159 ceph-mon[134726]: pgmap v237: 129 pgs: 129 active+clean; 330 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.5 MiB/s wr, 351 op/s 2023-12-07T10:28:08.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:08 smithi153 ceph-mon[133625]: pgmap v237: 129 pgs: 129 active+clean; 330 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.5 MiB/s wr, 351 op/s 2023-12-07T10:28:08.564 INFO:tasks.workunit.client.0.smithi119.stdout: 1 527217 7.07 MB/sec execute 261 sec latency 12.104 ms 2023-12-07T10:28:09.564 INFO:tasks.workunit.client.0.smithi119.stdout: 1 529314 7.09 MB/sec execute 262 sec latency 17.346 ms 2023-12-07T10:28:10.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:10 smithi119 ceph-mon[97908]: pgmap v238: 129 pgs: 129 active+clean; 330 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.5 MiB/s wr, 351 op/s 2023-12-07T10:28:10.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:10 smithi159 ceph-mon[134726]: pgmap v238: 129 pgs: 129 active+clean; 330 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.5 MiB/s wr, 351 op/s 2023-12-07T10:28:10.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:10 smithi153 ceph-mon[133625]: pgmap v238: 129 pgs: 129 active+clean; 330 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.5 MiB/s wr, 351 op/s 2023-12-07T10:28:10.564 INFO:tasks.workunit.client.0.smithi119.stdout: 1 530461 7.07 MB/sec execute 263 sec latency 15.521 ms 2023-12-07T10:28:11.564 INFO:tasks.workunit.client.0.smithi119.stdout: 1 532330 7.09 MB/sec execute 264 sec latency 18.676 ms 2023-12-07T10:28:11.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.676+0000 7f4d439ff700 1 -- 172.21.15.119:0/2912175702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d3c100ef0 msgr2=0x7f4d3c1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:11.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.676+0000 7f4d439ff700 1 --2- 172.21.15.119:0/2912175702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d3c100ef0 0x7f4d3c1012d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009990 tx=0x7f4d38009ca0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.677+0000 7f4d439ff700 1 -- 172.21.15.119:0/2912175702 shutdown_connections 2023-12-07T10:28:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.677+0000 7f4d439ff700 1 --2- 172.21.15.119:0/2912175702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d3c10df60 0x7f4d3c110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.677+0000 7f4d439ff700 1 --2- 172.21.15.119:0/2912175702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d3c100ef0 0x7f4d3c1012d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.677+0000 7f4d439ff700 1 --2- 172.21.15.119:0/2912175702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d3c101810 0x7f4d3c10da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.677+0000 7f4d439ff700 1 -- 172.21.15.119:0/2912175702 >> 172.21.15.119:0/2912175702 conn(0x7f4d3c0775b0 msgr2=0x7f4d3c0779c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:28:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.677+0000 7f4d439ff700 1 -- 172.21.15.119:0/2912175702 shutdown_connections 2023-12-07T10:28:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.677+0000 7f4d439ff700 1 -- 172.21.15.119:0/2912175702 wait complete. 2023-12-07T10:28:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.678+0000 7f4d439ff700 1 Processor -- start 2023-12-07T10:28:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.678+0000 7f4d439ff700 1 -- start start 2023-12-07T10:28:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.679+0000 7f4d439ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d3c100ef0 0x7f4d3c19f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.679+0000 7f4d439ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d3c101810 0x7f4d3c19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.680+0000 7f4d439ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d3c10df60 0x7f4d3c199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.680+0000 7f4d439ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c19a0c0 con 0x7f4d3c101810 2023-12-07T10:28:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.680+0000 7f4d439ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c19a230 con 0x7f4d3c100ef0 2023-12-07T10:28:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.680+0000 7f4d439ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c19a3f0 con 0x7f4d3c10df60 2023-12-07T10:28:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.680+0000 7f4d4179b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d3c100ef0 0x7f4d3c19f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.680+0000 7f4d4179b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d3c100ef0 0x7f4d3c19f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45460/0 (socket says 172.21.15.119:45460) 2023-12-07T10:28:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.680+0000 7f4d41f9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d3c10df60 0x7f4d3c199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.680+0000 7f4d4179b700 1 -- 172.21.15.119:0/32390178 learned_addr learned my addr 172.21.15.119:0/32390178 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:28:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.680+0000 7f4d40f9a700 1 --2- 172.21.15.119:0/32390178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d3c101810 0x7f4d3c19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.680+0000 7f4d41f9c700 1 -- 172.21.15.119:0/32390178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d3c100ef0 msgr2=0x7f4d3c19f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.680+0000 7f4d41f9c700 1 --2- 172.21.15.119:0/32390178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d3c100ef0 0x7f4d3c19f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.680+0000 7f4d41f9c700 1 -- 172.21.15.119:0/32390178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d3c101810 msgr2=0x7f4d3c19fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.680+0000 7f4d41f9c700 1 --2- 172.21.15.119:0/32390178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d3c101810 0x7f4d3c19fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.680+0000 7f4d41f9c700 1 -- 172.21.15.119:0/32390178 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d380095f0 con 0x7f4d3c10df60 2023-12-07T10:28:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.680+0000 7f4d4179b700 1 --2- 172.21.15.119:0/32390178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d3c100ef0 0x7f4d3c19f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:28:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.680+0000 7f4d41f9c700 1 --2- 172.21.15.119:0/32390178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d3c10df60 0x7f4d3c199a60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400ef90 tx=0x7f4d3400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:28:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.681+0000 7f4d327fc700 1 -- 172.21.15.119:0/32390178 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d34018070 con 0x7f4d3c10df60 2023-12-07T10:28:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.681+0000 7f4d327fc700 1 -- 172.21.15.119:0/32390178 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d3400f040 con 0x7f4d3c10df60 2023-12-07T10:28:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.681+0000 7f4d439ff700 1 -- 172.21.15.119:0/32390178 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d3c19a670 con 0x7f4d3c10df60 2023-12-07T10:28:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.681+0000 7f4d327fc700 1 -- 172.21.15.119:0/32390178 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3400cc30 con 0x7f4d3c10df60 2023-12-07T10:28:11.684 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.681+0000 7f4d439ff700 1 -- 172.21.15.119:0/32390178 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d3c1a6900 con 0x7f4d3c10df60 2023-12-07T10:28:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.684+0000 7f4d327fc700 1 -- 172.21.15.119:0/32390178 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f4d34003a00 con 0x7f4d3c10df60 2023-12-07T10:28:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.686+0000 7f4d5737db80 1 Processor -- start 2023-12-07T10:28:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.686+0000 7f4d327fc700 1 --2- 172.21.15.119:0/32390178 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4d280a1c80 0x7f4d280a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.686+0000 7f4d5737db80 1 -- start start 2023-12-07T10:28:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.686+0000 7f4d327fc700 1 -- 172.21.15.119:0/32390178 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f4d340cd860 con 0x7f4d3c10df60 2023-12-07T10:28:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.687+0000 7f4d5737db80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55556e11ba60 0x55556e11be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.687+0000 7f4d40f9a700 1 --2- 172.21.15.119:0/32390178 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4d280a1c80 0x7f4d280a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.687+0000 7f4d5737db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55556e11c380 0x55556e116540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.687+0000 7f4d5737db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55556e116ad0 0x55556e116f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.687+0000 7f4d5737db80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556e08a760 con 0x55556e11ba60 2023-12-07T10:28:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.687+0000 7f4d5737db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556e071740 con 0x55556e11c380 2023-12-07T10:28:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.687+0000 7f4d5737db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556e06f060 con 0x55556e116ad0 2023-12-07T10:28:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.687+0000 7f4d4179b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55556e11ba60 0x55556e11be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.687+0000 7f4d41f9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55556e116ad0 0x55556e116f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.687+0000 7f4d41f9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55556e116ad0 0x55556e116f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:35458/0 (socket says 172.21.15.119:35458) 2023-12-07T10:28:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.687+0000 7f4d41f9c700 1 -- 172.21.15.119:0/3141598002 learned_addr learned my addr 172.21.15.119:0/3141598002 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:28:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.687+0000 7f4d40f9a700 1 --2- 172.21.15.119:0/3141598002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55556e11c380 0x55556e116540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.687+0000 7f4d41f9c700 1 -- 172.21.15.119:0/3141598002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55556e11c380 msgr2=0x55556e116540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.687+0000 7f4d41f9c700 1 --2- 172.21.15.119:0/3141598002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55556e11c380 0x55556e116540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.687+0000 7f4d41f9c700 1 -- 172.21.15.119:0/3141598002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55556e11ba60 msgr2=0x55556e11be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.687+0000 7f4d41f9c700 1 --2- 172.21.15.119:0/3141598002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55556e11ba60 0x55556e11be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.687+0000 7f4d41f9c700 1 -- 172.21.15.119:0/3141598002 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55556dee75d0 con 0x55556e116ad0 2023-12-07T10:28:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.687+0000 7f4d4179b700 1 --2- 172.21.15.119:0/3141598002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55556e11ba60 0x55556e11be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:28:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.688+0000 7f4d40f9a700 1 --2- 172.21.15.119:0/32390178 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4d280a1c80 0x7f4d280a4140 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0ff860 tx=0x7f4d2c009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:28:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.688+0000 7f4d41f9c700 1 --2- 172.21.15.119:0/3141598002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55556e116ad0 0x55556e116f30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4d340a20b0 tx=0x7f4d34099e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb33a40229ddcf80 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:28:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.688+0000 7f4d277fe700 1 -- 172.21.15.119:0/3141598002 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3409a2e0 con 0x55556e116ad0 2023-12-07T10:28:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.688+0000 7f4d277fe700 1 -- 172.21.15.119:0/3141598002 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d3409e430 con 0x55556e116ad0 2023-12-07T10:28:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.688+0000 7f4d277fe700 1 -- 172.21.15.119:0/3141598002 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3401b320 con 0x55556e116ad0 2023-12-07T10:28:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.689+0000 7f4d5737db80 1 -- 172.21.15.119:0/3141598002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55556e116ad0 msgr2=0x55556e116f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.689+0000 7f4d5737db80 1 --2- 172.21.15.119:0/3141598002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55556e116ad0 0x55556e116f30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4d340a20b0 tx=0x7f4d34099e20 comp rx=0 tx=0).stop 2023-12-07T10:28:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.689+0000 7f4d5737db80 1 -- 172.21.15.119:0/3141598002 shutdown_connections 2023-12-07T10:28:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.689+0000 7f4d5737db80 1 --2- 172.21.15.119:0/3141598002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55556e116ad0 0x55556e116f30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.689+0000 7f4d5737db80 1 --2- 172.21.15.119:0/3141598002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55556e11ba60 0x55556e11be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.689+0000 7f4d5737db80 1 --2- 172.21.15.119:0/3141598002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55556e11c380 0x55556e116540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.689+0000 7f4d5737db80 1 -- 172.21.15.119:0/3141598002 >> 172.21.15.119:0/3141598002 conn(0x55556e045300 msgr2=0x55556e110b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:28:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.689+0000 7f4d5737db80 1 -- 172.21.15.119:0/3141598002 shutdown_connections 2023-12-07T10:28:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.689+0000 7f4d5737db80 1 -- 172.21.15.119:0/3141598002 wait complete. 2023-12-07T10:28:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.690+0000 7f4d5737db80 1 -- start start 2023-12-07T10:28:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.690+0000 7f4d5737db80 1 Processor -- start 2023-12-07T10:28:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.691+0000 7f4d5737db80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55556e116ad0 0x55556e11f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.691+0000 7f4d5737db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55556e11ba60 0x55556e11fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.691+0000 7f4d5737db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55556e11c380 0x55556e120250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.691+0000 7f4d5737db80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556e06a950 con 0x55556e116ad0 2023-12-07T10:28:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.691+0000 7f4d5737db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556e024500 con 0x55556e11c380 2023-12-07T10:28:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.691+0000 7f4d5737db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556e0241a0 con 0x55556e11ba60 2023-12-07T10:28:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.691+0000 7f4d5737db80 10 client.0 _ll_register_callbacks cb 0x55556e00b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:28:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.691+0000 7f4d4179b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55556e116ad0 0x55556e11f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.691+0000 7f4d4179b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55556e116ad0 0x55556e11f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42154/0 (socket says 172.21.15.119:42154) 2023-12-07T10:28:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.691+0000 7f4d4179b700 1 -- 172.21.15.119:0/1556261777 learned_addr learned my addr 172.21.15.119:0/1556261777 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:28:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.691+0000 7f4d41f9c700 1 --2- 172.21.15.119:0/1556261777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55556e11c380 0x55556e120250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.691+0000 7f4d40f9a700 1 --2- 172.21.15.119:0/1556261777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55556e11ba60 0x55556e11fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.692+0000 7f4d4179b700 1 -- 172.21.15.119:0/1556261777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55556e11ba60 msgr2=0x55556e11fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.692+0000 7f4d4179b700 1 --2- 172.21.15.119:0/1556261777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55556e11ba60 0x55556e11fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.692+0000 7f4d4179b700 1 -- 172.21.15.119:0/1556261777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55556e11c380 msgr2=0x55556e120250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.692+0000 7f4d4179b700 1 --2- 172.21.15.119:0/1556261777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55556e11c380 0x55556e120250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.692+0000 7f4d4179b700 1 -- 172.21.15.119:0/1556261777 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d3c110e50 con 0x55556e116ad0 2023-12-07T10:28:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.692+0000 7f4d41f9c700 1 --2- 172.21.15.119:0/1556261777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55556e11c380 0x55556e120250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:28:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.692+0000 7f4d4179b700 1 --2- 172.21.15.119:0/1556261777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55556e116ad0 0x55556e11f7d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4d38005280 tx=0x7f4d380052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875770ff5e1b7d14 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:28:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.692+0000 7f4d24ff9700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:28:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.692+0000 7f4d24ff9700 1 -- 172.21.15.119:0/1556261777 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3801d070 con 0x55556e116ad0 2023-12-07T10:28:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.692+0000 7f4d24ff9700 1 -- 172.21.15.119:0/1556261777 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f4d380056b0 con 0x55556e116ad0 2023-12-07T10:28:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.692+0000 7f4d5737db80 1 -- 172.21.15.119:0/1556261777 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55556e1096f0 con 0x55556e116ad0 2023-12-07T10:28:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.693+0000 7f4d24ff9700 1 -- 172.21.15.119:0/1556261777 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d38004910 con 0x55556e116ad0 2023-12-07T10:28:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.693+0000 7f4d5737db80 10 client.17397 fetch_fsmap learned FSMap version 52 2023-12-07T10:28:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.693+0000 7f4d5737db80 1 -- 172.21.15.119:0/1556261777 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x7f4d3c1a6900 con 0x55556e116ad0 2023-12-07T10:28:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.693+0000 7f4d24ff9700 1 -- 172.21.15.119:0/1556261777 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55556e1096f0 con 0x55556e116ad0 2023-12-07T10:28:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.693+0000 7f4d24ff9700 1 -- 172.21.15.119:0/1556261777 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 52) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f4d3800fc50 con 0x55556e116ad0 2023-12-07T10:28:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.693+0000 7f4d5737db80 10 client.17397 fetch_fsmap finished waiting for FSMap version 52 2023-12-07T10:28:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.693+0000 7f4d5737db80 10 client.17397 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:28:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.693+0000 7f4d5737db80 20 client.17397 populate_metadata read hostname 'smithi119' 2023-12-07T10:28:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.693+0000 7f4d5737db80 1 --2- 172.21.15.119:0/1556261777 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55556e130030 0x55556e132450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.693+0000 7f4d5737db80 4 client.17397 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:28:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.693+0000 7f4d5737db80 1 -- 172.21.15.119:0/1556261777 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55556e0e5740 con 0x55556e130030 2023-12-07T10:28:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.694+0000 7f4d40f9a700 1 --2- 172.21.15.119:0/1556261777 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55556e130030 0x55556e132450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.695+0000 7f4d40f9a700 1 --2- 172.21.15.119:0/1556261777 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55556e130030 0x55556e132450 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8569d943bac92fc server_cookie=6e8c3452b34f10fc in_seq=0 out_seq=0 2023-12-07T10:28:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.695+0000 7f4d24ff9700 10 client.17397 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:28:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.695+0000 7f4d24ff9700 1 -- 172.21.15.119:0/1556261777 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55556e0e5740 con 0x55556e130030 2023-12-07T10:28:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.695+0000 7f4d24ff9700 10 client.17397 handle_command_reply: tid=0 2023-12-07T10:28:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.696+0000 7f4d5737db80 1 client.17397 shutdown 2023-12-07T10:28:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.696+0000 7f4d5737db80 1 -- 172.21.15.119:0/1556261777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55556e116ad0 msgr2=0x55556e11f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:11.699 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.696+0000 7f4d5737db80 1 --2- 172.21.15.119:0/1556261777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55556e116ad0 0x55556e11f7d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4d38005280 tx=0x7f4d380052b0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.696+0000 7f4d5737db80 1 -- 172.21.15.119:0/1556261777 shutdown_connections 2023-12-07T10:28:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.696+0000 7f4d5737db80 1 --2- 172.21.15.119:0/1556261777 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55556e130030 0x55556e132450 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.696+0000 7f4d5737db80 1 --2- 172.21.15.119:0/1556261777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55556e11c380 0x55556e120250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.696+0000 7f4d5737db80 1 --2- 172.21.15.119:0/1556261777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55556e116ad0 0x55556e11f7d0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.696+0000 7f4d5737db80 1 --2- 172.21.15.119:0/1556261777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55556e11ba60 0x55556e11fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.696+0000 7f4d5737db80 1 -- 172.21.15.119:0/1556261777 >> 172.21.15.119:0/1556261777 conn(0x55556e045300 msgr2=0x55556e10b980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:28:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.696+0000 7f4d24ff9700 0 client.17397 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:28:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.697+0000 7f4d5737db80 1 -- 172.21.15.119:0/1556261777 shutdown_connections 2023-12-07T10:28:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.697+0000 7f4d5737db80 1 -- 172.21.15.119:0/1556261777 wait complete. 2023-12-07T10:28:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.697+0000 7f4d5737db80 20 client.17397 trim_cache size 0 max 16384 2023-12-07T10:28:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.715+0000 7f4d5737db80 1 Processor -- start 2023-12-07T10:28:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.715+0000 7f4d5737db80 1 -- start start 2023-12-07T10:28:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.715+0000 7f4d5737db80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55556e116ad0 0x55556e116540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.715+0000 7f4d5737db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55556e11ba60 0x55556e1185d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.715+0000 7f4d5737db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55556e11c380 0x55556e136d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.715+0000 7f4d5737db80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556dee98c0 con 0x55556e116ad0 2023-12-07T10:28:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.715+0000 7f4d5737db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556e111480 con 0x55556e11ba60 2023-12-07T10:28:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.715+0000 7f4d5737db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556e113000 con 0x55556e11c380 2023-12-07T10:28:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.716+0000 7f4d40f9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55556e11ba60 0x55556e1185d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.716+0000 7f4d40f9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55556e11ba60 0x55556e1185d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45498/0 (socket says 172.21.15.119:45498) 2023-12-07T10:28:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.716+0000 7f4d40f9a700 1 -- 172.21.15.119:0/3595520347 learned_addr learned my addr 172.21.15.119:0/3595520347 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:28:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.716+0000 7f4d4179b700 1 --2- 172.21.15.119:0/3595520347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55556e116ad0 0x55556e116540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.716+0000 7f4d41f9c700 1 --2- 172.21.15.119:0/3595520347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55556e11c380 0x55556e136d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.716+0000 7f4d40f9a700 1 -- 172.21.15.119:0/3595520347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55556e11c380 msgr2=0x55556e136d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.716+0000 7f4d40f9a700 1 --2- 172.21.15.119:0/3595520347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55556e11c380 0x55556e136d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.716+0000 7f4d40f9a700 1 -- 172.21.15.119:0/3595520347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55556e116ad0 msgr2=0x55556e116540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.716+0000 7f4d40f9a700 1 --2- 172.21.15.119:0/3595520347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55556e116ad0 0x55556e116540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.716+0000 7f4d40f9a700 1 -- 172.21.15.119:0/3595520347 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55556de93d80 con 0x55556e11ba60 2023-12-07T10:28:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.716+0000 7f4d41f9c700 1 --2- 172.21.15.119:0/3595520347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55556e11c380 0x55556e136d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:28:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.716+0000 7f4d40f9a700 1 --2- 172.21.15.119:0/3595520347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55556e11ba60 0x55556e1185d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c069950 tx=0x7f4d2c01c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=affa2f4de489583f server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:28:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.717+0000 7f4d27fff700 1 -- 172.21.15.119:0/3595520347 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c01c400 con 0x55556e11ba60 2023-12-07T10:28:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.717+0000 7f4d27fff700 1 -- 172.21.15.119:0/3595520347 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d2c0044e0 con 0x55556e11ba60 2023-12-07T10:28:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.717+0000 7f4d27fff700 1 -- 172.21.15.119:0/3595520347 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c003ea0 con 0x55556e11ba60 2023-12-07T10:28:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.718+0000 7f4d5737db80 1 -- 172.21.15.119:0/3595520347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55556e11ba60 msgr2=0x55556e1185d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.718+0000 7f4d5737db80 1 --2- 172.21.15.119:0/3595520347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55556e11ba60 0x55556e1185d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c069950 tx=0x7f4d2c01c000 comp rx=0 tx=0).stop 2023-12-07T10:28:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.718+0000 7f4d5737db80 1 -- 172.21.15.119:0/3595520347 shutdown_connections 2023-12-07T10:28:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.718+0000 7f4d5737db80 1 --2- 172.21.15.119:0/3595520347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55556e11c380 0x55556e136d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.718+0000 7f4d5737db80 1 --2- 172.21.15.119:0/3595520347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55556e116ad0 0x55556e116540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.718+0000 7f4d5737db80 1 --2- 172.21.15.119:0/3595520347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55556e11ba60 0x55556e1185d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.718+0000 7f4d5737db80 1 -- 172.21.15.119:0/3595520347 >> 172.21.15.119:0/3595520347 conn(0x55556e045300 msgr2=0x55556e118f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:28:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.718+0000 7f4d5737db80 1 -- 172.21.15.119:0/3595520347 shutdown_connections 2023-12-07T10:28:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.718+0000 7f4d5737db80 1 -- 172.21.15.119:0/3595520347 wait complete. 2023-12-07T10:28:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.718+0000 7f4d5737db80 1 -- start start 2023-12-07T10:28:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.718+0000 7f4d5737db80 1 Processor -- start 2023-12-07T10:28:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.719+0000 7f4d5737db80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55556e116ad0 0x55556e126fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.719+0000 7f4d5737db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55556e11ba60 0x55556e122da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.719+0000 7f4d5737db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55556e11c380 0x55556e1232e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.719+0000 7f4d5737db80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556e123860 con 0x55556e116ad0 2023-12-07T10:28:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.719+0000 7f4d5737db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556e1239d0 con 0x55556e11c380 2023-12-07T10:28:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.719+0000 7f4d5737db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556e123b90 con 0x55556e11ba60 2023-12-07T10:28:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.719+0000 7f4d5737db80 10 client.0 _ll_register_callbacks cb 0x55556e00b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:28:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.719+0000 7f4d4179b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55556e116ad0 0x55556e126fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.719+0000 7f4d41f9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55556e11c380 0x55556e1232e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.719+0000 7f4d41f9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55556e11c380 0x55556e1232e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:45502/0 (socket says 172.21.15.119:45502) 2023-12-07T10:28:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.719+0000 7f4d41f9c700 1 -- 172.21.15.119:0/4160163754 learned_addr learned my addr 172.21.15.119:0/4160163754 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:28:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.719+0000 7f4d40f9a700 1 --2- 172.21.15.119:0/4160163754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55556e11ba60 0x55556e122da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.720+0000 7f4d41f9c700 1 -- 172.21.15.119:0/4160163754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55556e11ba60 msgr2=0x55556e122da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.720+0000 7f4d41f9c700 1 --2- 172.21.15.119:0/4160163754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55556e11ba60 0x55556e122da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.720+0000 7f4d41f9c700 1 -- 172.21.15.119:0/4160163754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55556e116ad0 msgr2=0x55556e126fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.720+0000 7f4d41f9c700 1 --2- 172.21.15.119:0/4160163754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55556e116ad0 0x55556e126fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.720+0000 7f4d41f9c700 1 -- 172.21.15.119:0/4160163754 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d2c021040 con 0x55556e11c380 2023-12-07T10:28:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.720+0000 7f4d41f9c700 1 --2- 172.21.15.119:0/4160163754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55556e11c380 0x55556e1232e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4d34004bf0 tx=0x7f4d340a2600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ef81affc9cac6 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:28:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.720+0000 7f4d24ff9700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:28:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.720+0000 7f4d24ff9700 1 -- 172.21.15.119:0/4160163754 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3409a2e0 con 0x55556e11c380 2023-12-07T10:28:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.720+0000 7f4d24ff9700 1 -- 172.21.15.119:0/4160163754 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f4d3409e070 con 0x55556e11c380 2023-12-07T10:28:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.720+0000 7f4d5737db80 1 -- 172.21.15.119:0/4160163754 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d3409e430 con 0x55556e11c380 2023-12-07T10:28:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.720+0000 7f4d24ff9700 1 -- 172.21.15.119:0/4160163754 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d3409e230 con 0x55556e11c380 2023-12-07T10:28:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.720+0000 7f4d5737db80 10 client.27440 fetch_fsmap learned FSMap version 52 2023-12-07T10:28:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.720+0000 7f4d5737db80 1 -- 172.21.15.119:0/4160163754 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55556de7cc00 con 0x55556e11c380 2023-12-07T10:28:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.721+0000 7f4d24ff9700 1 -- 172.21.15.119:0/4160163754 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d3409e430 con 0x55556e11c380 2023-12-07T10:28:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.721+0000 7f4d24ff9700 1 -- 172.21.15.119:0/4160163754 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 52) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f4d340a2150 con 0x55556e11c380 2023-12-07T10:28:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.721+0000 7f4d5737db80 10 client.27440 fetch_fsmap finished waiting for FSMap version 52 2023-12-07T10:28:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.721+0000 7f4d5737db80 10 client.27440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:28:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.721+0000 7f4d5737db80 20 client.27440 populate_metadata read hostname 'smithi119' 2023-12-07T10:28:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.721+0000 7f4d5737db80 1 --2- 172.21.15.119:0/4160163754 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55556e186720 0x55556e188b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.721+0000 7f4d5737db80 4 client.27440 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:28:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.721+0000 7f4d5737db80 1 -- 172.21.15.119:0/4160163754 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55556e030f70 con 0x55556e186720 2023-12-07T10:28:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.721+0000 7f4d4179b700 1 --2- 172.21.15.119:0/4160163754 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55556e186720 0x55556e188b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.722+0000 7f4d24ff9700 10 client.27440 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:28:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.722+0000 7f4d4179b700 1 --2- 172.21.15.119:0/4160163754 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55556e186720 0x55556e188b40 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1f16b09f991d29b server_cookie=efc5c2190180ae4f in_seq=0 out_seq=0 2023-12-07T10:28:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.766+0000 7f4d24ff9700 1 -- 172.21.15.119:0/4160163754 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55556e030f70 con 0x55556e186720 2023-12-07T10:28:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.766+0000 7f4d24ff9700 10 client.27440 handle_command_reply: tid=0 2023-12-07T10:28:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.766+0000 7f4d5737db80 1 client.27440 shutdown 2023-12-07T10:28:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.766+0000 7f4d5737db80 1 -- 172.21.15.119:0/4160163754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55556e11c380 msgr2=0x55556e1232e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.766+0000 7f4d5737db80 1 --2- 172.21.15.119:0/4160163754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55556e11c380 0x55556e1232e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4d34004bf0 tx=0x7f4d340a2600 comp rx=0 tx=0).stop 2023-12-07T10:28:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.766+0000 7f4d5737db80 1 -- 172.21.15.119:0/4160163754 shutdown_connections 2023-12-07T10:28:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.766+0000 7f4d5737db80 1 --2- 172.21.15.119:0/4160163754 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55556e186720 0x55556e188b40 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.766+0000 7f4d5737db80 1 --2- 172.21.15.119:0/4160163754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55556e11c380 0x55556e1232e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.766+0000 7f4d5737db80 1 --2- 172.21.15.119:0/4160163754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55556e116ad0 0x55556e126fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.766+0000 7f4d5737db80 1 --2- 172.21.15.119:0/4160163754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55556e11ba60 0x55556e122da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.766+0000 7f4d24ff9700 0 client.27440 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:28:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.766+0000 7f4d5737db80 1 -- 172.21.15.119:0/4160163754 >> 172.21.15.119:0/4160163754 conn(0x55556e045300 msgr2=0x55556e119a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:28:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.766+0000 7f4d5737db80 1 -- 172.21.15.119:0/4160163754 shutdown_connections 2023-12-07T10:28:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.766+0000 7f4d5737db80 1 -- 172.21.15.119:0/4160163754 wait complete. 2023-12-07T10:28:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.766+0000 7f4d5737db80 20 client.27440 trim_cache size 0 max 16384 2023-12-07T10:28:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:28:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:28:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:28:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:28:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.767+0000 7f4d439ff700 1 -- 172.21.15.119:0/32390178 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4d280a1c80 msgr2=0x7f4d280a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.767+0000 7f4d439ff700 1 --2- 172.21.15.119:0/32390178 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4d280a1c80 0x7f4d280a4140 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0ff860 tx=0x7f4d2c009260 comp rx=0 tx=0).stop 2023-12-07T10:28:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.767+0000 7f4d439ff700 1 -- 172.21.15.119:0/32390178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d3c10df60 msgr2=0x7f4d3c199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.767+0000 7f4d439ff700 1 --2- 172.21.15.119:0/32390178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d3c10df60 0x7f4d3c199a60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400ef90 tx=0x7f4d3400c3a0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.767+0000 7f4d439ff700 1 -- 172.21.15.119:0/32390178 shutdown_connections 2023-12-07T10:28:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.767+0000 7f4d439ff700 1 --2- 172.21.15.119:0/32390178 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4d280a1c80 0x7f4d280a4140 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.767+0000 7f4d439ff700 1 --2- 172.21.15.119:0/32390178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d3c10df60 0x7f4d3c199a60 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.767+0000 7f4d439ff700 1 --2- 172.21.15.119:0/32390178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d3c100ef0 0x7f4d3c19f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.767+0000 7f4d439ff700 1 --2- 172.21.15.119:0/32390178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d3c101810 0x7f4d3c19fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.767+0000 7f4d439ff700 1 -- 172.21.15.119:0/32390178 >> 172.21.15.119:0/32390178 conn(0x7f4d3c0775b0 msgr2=0x7f4d3c10f710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:28:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.767+0000 7f4d439ff700 1 -- 172.21.15.119:0/32390178 shutdown_connections 2023-12-07T10:28:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:11.767+0000 7f4d439ff700 1 -- 172.21.15.119:0/32390178 wait complete. 2023-12-07T10:28:12.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:11 smithi119 ceph-mon[97908]: osdmap e98: 12 total, 12 up, 12 in 2023-12-07T10:28:12.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:11 smithi119 ceph-mon[97908]: pgmap v240: 129 pgs: 129 active+clean; 339 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 242 op/s 2023-12-07T10:28:12.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:11 smithi159 ceph-mon[134726]: osdmap e98: 12 total, 12 up, 12 in 2023-12-07T10:28:12.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:11 smithi159 ceph-mon[134726]: pgmap v240: 129 pgs: 129 active+clean; 339 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 242 op/s 2023-12-07T10:28:12.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:11 smithi153 ceph-mon[133625]: osdmap e98: 12 total, 12 up, 12 in 2023-12-07T10:28:12.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:11 smithi153 ceph-mon[133625]: pgmap v240: 129 pgs: 129 active+clean; 339 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 242 op/s 2023-12-07T10:28:12.565 INFO:tasks.workunit.client.0.smithi119.stdout: 1 533622 7.09 MB/sec execute 265 sec latency 29.360 ms 2023-12-07T10:28:13.565 INFO:tasks.workunit.client.0.smithi119.stdout: 1 535445 7.09 MB/sec execute 266 sec latency 19.609 ms 2023-12-07T10:28:14.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:14 smithi119 ceph-mon[97908]: pgmap v241: 129 pgs: 129 active+clean; 340 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 230 op/s 2023-12-07T10:28:14.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:14 smithi159 ceph-mon[134726]: pgmap v241: 129 pgs: 129 active+clean; 340 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 230 op/s 2023-12-07T10:28:14.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:14 smithi153 ceph-mon[133625]: pgmap v241: 129 pgs: 129 active+clean; 340 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 230 op/s 2023-12-07T10:28:14.565 INFO:tasks.workunit.client.0.smithi119.stdout: 1 536957 7.10 MB/sec execute 267 sec latency 12.203 ms 2023-12-07T10:28:15.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:15 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:28:15.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:15 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:28:15.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:15 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:28:15.565 INFO:tasks.workunit.client.0.smithi119.stdout: 1 538815 7.10 MB/sec execute 268 sec latency 15.614 ms 2023-12-07T10:28:16.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:16 smithi119 ceph-mon[97908]: pgmap v242: 129 pgs: 129 active+clean; 340 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 206 op/s 2023-12-07T10:28:16.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:16 smithi159 ceph-mon[134726]: pgmap v242: 129 pgs: 129 active+clean; 340 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 206 op/s 2023-12-07T10:28:16.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:16 smithi153 ceph-mon[133625]: pgmap v242: 129 pgs: 129 active+clean; 340 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 206 op/s 2023-12-07T10:28:16.565 INFO:tasks.workunit.client.0.smithi119.stdout: 1 540341 7.12 MB/sec execute 269 sec latency 16.929 ms 2023-12-07T10:28:17.565 INFO:tasks.workunit.client.0.smithi119.stdout: 1 541683 7.12 MB/sec execute 270 sec latency 16.334 ms 2023-12-07T10:28:18.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:18 smithi119 ceph-mon[97908]: pgmap v243: 129 pgs: 129 active+clean; 349 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 221 op/s 2023-12-07T10:28:18.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:18 smithi159 ceph-mon[134726]: pgmap v243: 129 pgs: 129 active+clean; 349 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 221 op/s 2023-12-07T10:28:18.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:18 smithi153 ceph-mon[133625]: pgmap v243: 129 pgs: 129 active+clean; 349 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 221 op/s 2023-12-07T10:28:18.565 INFO:tasks.workunit.client.0.smithi119.stdout: 1 543762 7.14 MB/sec execute 271 sec latency 15.332 ms 2023-12-07T10:28:19.565 INFO:tasks.workunit.client.0.smithi119.stdout: 1 545026 7.13 MB/sec execute 272 sec latency 12.936 ms 2023-12-07T10:28:20.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:20 smithi119 ceph-mon[97908]: pgmap v244: 129 pgs: 129 active+clean; 349 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 221 op/s 2023-12-07T10:28:20.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:20 smithi159 ceph-mon[134726]: pgmap v244: 129 pgs: 129 active+clean; 349 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 221 op/s 2023-12-07T10:28:20.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:20 smithi153 ceph-mon[133625]: pgmap v244: 129 pgs: 129 active+clean; 349 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 221 op/s 2023-12-07T10:28:20.565 INFO:tasks.workunit.client.0.smithi119.stdout: 1 547145 7.15 MB/sec execute 273 sec latency 15.353 ms 2023-12-07T10:28:21.565 INFO:tasks.workunit.client.0.smithi119.stdout: 1 548139 7.14 MB/sec execute 274 sec latency 19.139 ms 2023-12-07T10:28:22.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:21 smithi119 ceph-mon[97908]: pgmap v245: 129 pgs: 129 active+clean; 355 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 309 op/s 2023-12-07T10:28:22.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:21 smithi159 ceph-mon[134726]: pgmap v245: 129 pgs: 129 active+clean; 355 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 309 op/s 2023-12-07T10:28:22.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:21 smithi153 ceph-mon[133625]: pgmap v245: 129 pgs: 129 active+clean; 355 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 309 op/s 2023-12-07T10:28:22.565 INFO:tasks.workunit.client.0.smithi119.stdout: 1 550049 7.15 MB/sec execute 275 sec latency 21.822 ms 2023-12-07T10:28:23.565 INFO:tasks.workunit.client.0.smithi119.stdout: 1 551346 7.15 MB/sec execute 276 sec latency 14.396 ms 2023-12-07T10:28:24.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:24 smithi119 ceph-mon[97908]: pgmap v246: 129 pgs: 129 active+clean; 356 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2023-12-07T10:28:24.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:24 smithi159 ceph-mon[134726]: pgmap v246: 129 pgs: 129 active+clean; 356 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2023-12-07T10:28:24.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:24 smithi153 ceph-mon[133625]: pgmap v246: 129 pgs: 129 active+clean; 356 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2023-12-07T10:28:24.565 INFO:tasks.workunit.client.0.smithi119.stdout: 1 553105 7.15 MB/sec execute 277 sec latency 19.074 ms 2023-12-07T10:28:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.931+0000 7f13743ff700 1 -- 172.21.15.119:0/1371358635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f136c1060c0 msgr2=0x7f136c106540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.931+0000 7f13743ff700 1 --2- 172.21.15.119:0/1371358635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f136c1060c0 0x7f136c106540 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f135c009990 tx=0x7f135c009ca0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.931+0000 7f13743ff700 1 -- 172.21.15.119:0/1371358635 shutdown_connections 2023-12-07T10:28:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.932+0000 7f13743ff700 1 --2- 172.21.15.119:0/1371358635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f136c106a80 0x7f136c10e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.932+0000 7f13743ff700 1 --2- 172.21.15.119:0/1371358635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f136c068ce0 0x7f136c0690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.932+0000 7f13743ff700 1 --2- 172.21.15.119:0/1371358635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f136c1060c0 0x7f136c106540 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.932+0000 7f13743ff700 1 -- 172.21.15.119:0/1371358635 >> 172.21.15.119:0/1371358635 conn(0x7f136c0775b0 msgr2=0x7f136c0779c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:28:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.932+0000 7f13743ff700 1 -- 172.21.15.119:0/1371358635 shutdown_connections 2023-12-07T10:28:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.932+0000 7f13743ff700 1 -- 172.21.15.119:0/1371358635 wait complete. 2023-12-07T10:28:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.933+0000 7f13743ff700 1 Processor -- start 2023-12-07T10:28:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.933+0000 7f13743ff700 1 -- start start 2023-12-07T10:28:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.933+0000 7f13743ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f136c068ce0 0x7f136c19f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.933+0000 7f13743ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f136c1060c0 0x7f136c19fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.934+0000 7f13743ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f136c106a80 0x7f136c199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.934+0000 7f13743ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c19a090 con 0x7f136c106a80 2023-12-07T10:28:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.934+0000 7f13743ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c19a200 con 0x7f136c1060c0 2023-12-07T10:28:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.934+0000 7f13743ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c19a3c0 con 0x7f136c068ce0 2023-12-07T10:28:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.934+0000 7f137219b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f136c068ce0 0x7f136c19f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.934+0000 7f137219b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f136c068ce0 0x7f136c19f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:49952/0 (socket says 172.21.15.119:49952) 2023-12-07T10:28:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.934+0000 7f137219b700 1 -- 172.21.15.119:0/1525752857 learned_addr learned my addr 172.21.15.119:0/1525752857 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:28:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.934+0000 7f137299c700 1 --2- 172.21.15.119:0/1525752857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f136c106a80 0x7f136c199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.934+0000 7f137199a700 1 --2- 172.21.15.119:0/1525752857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f136c1060c0 0x7f136c19fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.934+0000 7f137219b700 1 -- 172.21.15.119:0/1525752857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f136c1060c0 msgr2=0x7f136c19fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.934+0000 7f137219b700 1 --2- 172.21.15.119:0/1525752857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f136c1060c0 0x7f136c19fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.934+0000 7f137219b700 1 -- 172.21.15.119:0/1525752857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f136c106a80 msgr2=0x7f136c199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.934+0000 7f137219b700 1 --2- 172.21.15.119:0/1525752857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f136c106a80 0x7f136c199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.934+0000 7f137219b700 1 -- 172.21.15.119:0/1525752857 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135c0095f0 con 0x7f136c068ce0 2023-12-07T10:28:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.934+0000 7f137199a700 1 --2- 172.21.15.119:0/1525752857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f136c1060c0 0x7f136c19fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:28:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.935+0000 7f137219b700 1 --2- 172.21.15.119:0/1525752857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f136c068ce0 0x7f136c19f900 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f136800e9a0 tx=0x7f136800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:28:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.935+0000 7f13637fe700 1 -- 172.21.15.119:0/1525752857 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1368009790 con 0x7f136c068ce0 2023-12-07T10:28:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.935+0000 7f13637fe700 1 -- 172.21.15.119:0/1525752857 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f136800cbd0 con 0x7f136c068ce0 2023-12-07T10:28:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.935+0000 7f13743ff700 1 -- 172.21.15.119:0/1525752857 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f136c19a6a0 con 0x7f136c068ce0 2023-12-07T10:28:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.935+0000 7f13637fe700 1 -- 172.21.15.119:0/1525752857 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1368018680 con 0x7f136c068ce0 2023-12-07T10:28:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.935+0000 7f13743ff700 1 -- 172.21.15.119:0/1525752857 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f136c1a68f0 con 0x7f136c068ce0 2023-12-07T10:28:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.938+0000 7f13637fe700 1 -- 172.21.15.119:0/1525752857 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f136800cd40 con 0x7f136c068ce0 2023-12-07T10:28:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.939+0000 7f1387d93b80 1 Processor -- start 2023-12-07T10:28:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.940+0000 7f1387d93b80 1 -- start start 2023-12-07T10:28:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.940+0000 7f1387d93b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bd0f4ca60 0x555bd0f4ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.940+0000 7f1387d93b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bd0f4d380 0x555bd0f47540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.940+0000 7f1387d93b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bd0f47ad0 0x555bd0f47f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.940+0000 7f1387d93b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd0ebb760 con 0x555bd0f4d380 2023-12-07T10:28:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.940+0000 7f1387d93b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd0ea2740 con 0x555bd0f4ca60 2023-12-07T10:28:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.940+0000 7f1387d93b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd0ea0060 con 0x555bd0f47ad0 2023-12-07T10:28:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.940+0000 7f137299c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bd0f4ca60 0x555bd0f4ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.940+0000 7f137299c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bd0f4ca60 0x555bd0f4ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42580/0 (socket says 172.21.15.119:42580) 2023-12-07T10:28:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.940+0000 7f137299c700 1 -- 172.21.15.119:0/1089278298 learned_addr learned my addr 172.21.15.119:0/1089278298 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:28:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.940+0000 7f13637fe700 1 --2- 172.21.15.119:0/1525752857 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f13580a1d00 0x7f13580a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.940+0000 7f137199a700 1 --2- 172.21.15.119:0/1089278298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bd0f47ad0 0x555bd0f47f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.940+0000 7f137219b700 1 --2- 172.21.15.119:0/1089278298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bd0f4d380 0x555bd0f47540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.941+0000 7f13637fe700 1 -- 172.21.15.119:0/1525752857 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f1368014070 con 0x7f136c068ce0 2023-12-07T10:28:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.941+0000 7f137299c700 1 -- 172.21.15.119:0/1089278298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bd0f47ad0 msgr2=0x555bd0f47f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.941+0000 7f137299c700 1 --2- 172.21.15.119:0/1089278298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bd0f47ad0 0x555bd0f47f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.941+0000 7f137299c700 1 -- 172.21.15.119:0/1089278298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bd0f4d380 msgr2=0x555bd0f47540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.941+0000 7f137299c700 1 --2- 172.21.15.119:0/1089278298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bd0f4d380 0x555bd0f47540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.941+0000 7f137299c700 1 -- 172.21.15.119:0/1089278298 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bd0d185d0 con 0x555bd0f4ca60 2023-12-07T10:28:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.941+0000 7f137219b700 1 --2- 172.21.15.119:0/1089278298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bd0f4d380 0x555bd0f47540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:28:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.941+0000 7f137299c700 1 --2- 172.21.15.119:0/1525752857 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f13580a1d00 0x7f13580a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:24.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.941+0000 7f137299c700 1 --2- 172.21.15.119:0/1089278298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bd0f4ca60 0x555bd0f4ce40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1364011790 tx=0x7f1364011aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15182c107789858f server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:28:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.942+0000 7f1360bfe700 1 -- 172.21.15.119:0/1089278298 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136400e040 con 0x555bd0f4ca60 2023-12-07T10:28:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.942+0000 7f1360bfe700 1 -- 172.21.15.119:0/1089278298 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1364027070 con 0x555bd0f4ca60 2023-12-07T10:28:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.942+0000 7f1360bfe700 1 -- 172.21.15.119:0/1089278298 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1364023870 con 0x555bd0f4ca60 2023-12-07T10:28:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.943+0000 7f1387d93b80 1 -- 172.21.15.119:0/1089278298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bd0f4ca60 msgr2=0x555bd0f4ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.943+0000 7f137299c700 1 --2- 172.21.15.119:0/1525752857 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f13580a1d00 0x7f13580a41c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1364022000 tx=0x7f136401f000 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:28:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.943+0000 7f1387d93b80 1 --2- 172.21.15.119:0/1089278298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bd0f4ca60 0x555bd0f4ce40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1364011790 tx=0x7f1364011aa0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.943+0000 7f1387d93b80 1 -- 172.21.15.119:0/1089278298 shutdown_connections 2023-12-07T10:28:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.943+0000 7f1387d93b80 1 --2- 172.21.15.119:0/1089278298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bd0f47ad0 0x555bd0f47f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.943+0000 7f1387d93b80 1 --2- 172.21.15.119:0/1089278298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bd0f4ca60 0x555bd0f4ce40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.943+0000 7f1387d93b80 1 --2- 172.21.15.119:0/1089278298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bd0f4d380 0x555bd0f47540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.943+0000 7f1387d93b80 1 -- 172.21.15.119:0/1089278298 >> 172.21.15.119:0/1089278298 conn(0x555bd0e76300 msgr2=0x555bd0f41b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:28:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.943+0000 7f1387d93b80 1 -- 172.21.15.119:0/1089278298 shutdown_connections 2023-12-07T10:28:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.943+0000 7f1387d93b80 1 -- 172.21.15.119:0/1089278298 wait complete. 2023-12-07T10:28:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.943+0000 7f1387d93b80 1 -- start start 2023-12-07T10:28:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.944+0000 7f1387d93b80 1 Processor -- start 2023-12-07T10:28:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.944+0000 7f1387d93b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bd0f47ad0 0x555bd0f506f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.944+0000 7f1387d93b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bd0f4ca60 0x555bd0f50c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.944+0000 7f1387d93b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bd0f4d380 0x555bd0f51170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.944+0000 7f1387d93b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd0e9b950 con 0x555bd0f4d380 2023-12-07T10:28:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.944+0000 7f1387d93b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd0e55500 con 0x555bd0f4ca60 2023-12-07T10:28:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.944+0000 7f1387d93b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd0e551a0 con 0x555bd0f47ad0 2023-12-07T10:28:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.944+0000 7f1387d93b80 10 client.0 _ll_register_callbacks cb 0x555bd0e3c8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:28:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.944+0000 7f137219b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bd0f47ad0 0x555bd0f506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.944+0000 7f137219b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bd0f47ad0 0x555bd0f506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:49958/0 (socket says 172.21.15.119:49958) 2023-12-07T10:28:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.944+0000 7f137219b700 1 -- 172.21.15.119:0/3788792938 learned_addr learned my addr 172.21.15.119:0/3788792938 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:28:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.944+0000 7f137199a700 1 --2- 172.21.15.119:0/3788792938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bd0f4ca60 0x555bd0f50c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.945+0000 7f137299c700 1 --2- 172.21.15.119:0/3788792938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bd0f4d380 0x555bd0f51170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.945+0000 7f137219b700 1 -- 172.21.15.119:0/3788792938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bd0f4ca60 msgr2=0x555bd0f50c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.945+0000 7f137219b700 1 --2- 172.21.15.119:0/3788792938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bd0f4ca60 0x555bd0f50c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.945+0000 7f137219b700 1 -- 172.21.15.119:0/3788792938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bd0f4d380 msgr2=0x555bd0f51170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.945+0000 7f137219b700 1 --2- 172.21.15.119:0/3788792938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bd0f4d380 0x555bd0f51170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.945+0000 7f137219b700 1 -- 172.21.15.119:0/3788792938 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1364011380 con 0x555bd0f47ad0 2023-12-07T10:28:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.945+0000 7f137219b700 1 --2- 172.21.15.119:0/3788792938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bd0f47ad0 0x555bd0f506f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f136809a650 tx=0x7f136809a3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43326a8a3918b56a server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:28:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.945+0000 7f1355ffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:28:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.945+0000 7f1355ffb700 1 -- 172.21.15.119:0/3788792938 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136809eaa0 con 0x555bd0f47ad0 2023-12-07T10:28:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.945+0000 7f1355ffb700 1 -- 172.21.15.119:0/3788792938 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f13680a2ce0 con 0x555bd0f47ad0 2023-12-07T10:28:24.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.945+0000 7f1387d93b80 1 -- 172.21.15.119:0/3788792938 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555bd0f3a6f0 con 0x555bd0f47ad0 2023-12-07T10:28:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.945+0000 7f1355ffb700 1 -- 172.21.15.119:0/3788792938 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13680ba0f0 con 0x555bd0f47ad0 2023-12-07T10:28:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.946+0000 7f1387d93b80 10 client.27433 fetch_fsmap learned FSMap version 53 2023-12-07T10:28:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.946+0000 7f1387d93b80 1 -- 172.21.15.119:0/3788792938 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x555bd0cc4d80 con 0x555bd0f47ad0 2023-12-07T10:28:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.946+0000 7f1355ffb700 1 -- 172.21.15.119:0/3788792938 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555bd0f3a6f0 con 0x555bd0f47ad0 2023-12-07T10:28:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.946+0000 7f1355ffb700 1 -- 172.21.15.119:0/3788792938 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 53) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f1368099e40 con 0x555bd0f47ad0 2023-12-07T10:28:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.946+0000 7f1387d93b80 10 client.27433 fetch_fsmap finished waiting for FSMap version 53 2023-12-07T10:28:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.946+0000 7f1387d93b80 10 client.27433 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:28:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.946+0000 7f1387d93b80 20 client.27433 populate_metadata read hostname 'smithi119' 2023-12-07T10:28:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.946+0000 7f1387d93b80 1 --2- 172.21.15.119:0/3788792938 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555bd0f60d80 0x555bd0f631a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.946+0000 7f1387d93b80 4 client.27433 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:28:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.946+0000 7f1387d93b80 1 -- 172.21.15.119:0/3788792938 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555bd0f16740 con 0x555bd0f60d80 2023-12-07T10:28:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.946+0000 7f137199a700 1 --2- 172.21.15.119:0/3788792938 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555bd0f60d80 0x555bd0f631a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.947+0000 7f137199a700 1 --2- 172.21.15.119:0/3788792938 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555bd0f60d80 0x555bd0f631a0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fa25cc6ba6a0f38 server_cookie=4607ebf44c095ef6 in_seq=0 out_seq=0 2023-12-07T10:28:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.947+0000 7f1355ffb700 10 client.27433 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:28:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.947+0000 7f1355ffb700 1 -- 172.21.15.119:0/3788792938 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555bd0f16740 con 0x555bd0f60d80 2023-12-07T10:28:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.947+0000 7f1355ffb700 10 client.27433 handle_command_reply: tid=0 2023-12-07T10:28:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.948+0000 7f1387d93b80 1 client.27433 shutdown 2023-12-07T10:28:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.948+0000 7f1387d93b80 1 -- 172.21.15.119:0/3788792938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bd0f47ad0 msgr2=0x555bd0f506f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.948+0000 7f1387d93b80 1 --2- 172.21.15.119:0/3788792938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bd0f47ad0 0x555bd0f506f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f136809a650 tx=0x7f136809a3b0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.948+0000 7f1387d93b80 1 -- 172.21.15.119:0/3788792938 shutdown_connections 2023-12-07T10:28:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.948+0000 7f1387d93b80 1 --2- 172.21.15.119:0/3788792938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bd0f4d380 0x555bd0f51170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.948+0000 7f1387d93b80 1 --2- 172.21.15.119:0/3788792938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bd0f47ad0 0x555bd0f506f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.948+0000 7f1387d93b80 1 --2- 172.21.15.119:0/3788792938 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555bd0f60d80 0x555bd0f631a0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.948+0000 7f1387d93b80 1 --2- 172.21.15.119:0/3788792938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bd0f4ca60 0x555bd0f50c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.948+0000 7f1355ffb700 0 client.27433 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:28:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.948+0000 7f1387d93b80 1 -- 172.21.15.119:0/3788792938 >> 172.21.15.119:0/3788792938 conn(0x555bd0e76300 msgr2=0x555bd0f3c980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:28:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.948+0000 7f1387d93b80 1 -- 172.21.15.119:0/3788792938 shutdown_connections 2023-12-07T10:28:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.948+0000 7f1387d93b80 1 -- 172.21.15.119:0/3788792938 wait complete. 2023-12-07T10:28:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.948+0000 7f1387d93b80 20 client.27433 trim_cache size 0 max 16384 2023-12-07T10:28:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.969+0000 7f1387d93b80 1 Processor -- start 2023-12-07T10:28:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.969+0000 7f1387d93b80 1 -- start start 2023-12-07T10:28:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.969+0000 7f1387d93b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bd0f47ad0 0x555bd0f47540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.969+0000 7f1387d93b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bd0f4ca60 0x555bd0f5c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.969+0000 7f1387d93b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bd0f4d380 0x555bd0f57dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.969+0000 7f1387d93b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd0d1a8c0 con 0x555bd0f4ca60 2023-12-07T10:28:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.969+0000 7f1387d93b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd0f68390 con 0x555bd0f47ad0 2023-12-07T10:28:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.969+0000 7f1387d93b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd0f4d090 con 0x555bd0f4d380 2023-12-07T10:28:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.970+0000 7f137219b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bd0f47ad0 0x555bd0f47540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.970+0000 7f137219b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bd0f47ad0 0x555bd0f47540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42598/0 (socket says 172.21.15.119:42598) 2023-12-07T10:28:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.970+0000 7f137219b700 1 -- 172.21.15.119:0/3490380909 learned_addr learned my addr 172.21.15.119:0/3490380909 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:28:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.970+0000 7f137199a700 1 --2- 172.21.15.119:0/3490380909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bd0f4ca60 0x555bd0f5c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.970+0000 7f137299c700 1 --2- 172.21.15.119:0/3490380909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bd0f4d380 0x555bd0f57dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.970+0000 7f137219b700 1 -- 172.21.15.119:0/3490380909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bd0f4d380 msgr2=0x555bd0f57dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.970+0000 7f137219b700 1 --2- 172.21.15.119:0/3490380909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bd0f4d380 0x555bd0f57dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.970+0000 7f137219b700 1 -- 172.21.15.119:0/3490380909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bd0f4ca60 msgr2=0x555bd0f5c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.970+0000 7f137219b700 1 --2- 172.21.15.119:0/3490380909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bd0f4ca60 0x555bd0f5c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.970+0000 7f137219b700 1 -- 172.21.15.119:0/3490380909 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bd0cadc00 con 0x555bd0f47ad0 2023-12-07T10:28:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.970+0000 7f137219b700 1 --2- 172.21.15.119:0/3490380909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bd0f47ad0 0x555bd0f47540 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f136800e970 tx=0x7f136800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c1d15e1e29d5498 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:28:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.971+0000 7f13613ff700 1 -- 172.21.15.119:0/3490380909 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13680a2ce0 con 0x555bd0f47ad0 2023-12-07T10:28:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.971+0000 7f13613ff700 1 -- 172.21.15.119:0/3490380909 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f136809eaa0 con 0x555bd0f47ad0 2023-12-07T10:28:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.971+0000 7f13613ff700 1 -- 172.21.15.119:0/3490380909 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136809abb0 con 0x555bd0f47ad0 2023-12-07T10:28:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.971+0000 7f1387d93b80 1 -- 172.21.15.119:0/3490380909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bd0f47ad0 msgr2=0x555bd0f47540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.971+0000 7f1387d93b80 1 --2- 172.21.15.119:0/3490380909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bd0f47ad0 0x555bd0f47540 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f136800e970 tx=0x7f136800eef0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.971+0000 7f1387d93b80 1 -- 172.21.15.119:0/3490380909 shutdown_connections 2023-12-07T10:28:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.971+0000 7f1387d93b80 1 --2- 172.21.15.119:0/3490380909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bd0f4d380 0x555bd0f57dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.971+0000 7f1387d93b80 1 --2- 172.21.15.119:0/3490380909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bd0f47ad0 0x555bd0f47540 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.971+0000 7f1387d93b80 1 --2- 172.21.15.119:0/3490380909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bd0f4ca60 0x555bd0f5c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.971+0000 7f1387d93b80 1 -- 172.21.15.119:0/3490380909 >> 172.21.15.119:0/3490380909 conn(0x555bd0e76300 msgr2=0x555bd0f50ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:28:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.971+0000 7f1387d93b80 1 -- 172.21.15.119:0/3490380909 shutdown_connections 2023-12-07T10:28:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.972+0000 7f1387d93b80 1 -- 172.21.15.119:0/3490380909 wait complete. 2023-12-07T10:28:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.972+0000 7f1387d93b80 1 -- start start 2023-12-07T10:28:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.972+0000 7f1387d93b80 1 Processor -- start 2023-12-07T10:28:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.973+0000 7f1387d93b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bd0f47ad0 0x555bd0fab210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.973+0000 7f1387d93b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bd0f4ca60 0x555bd0fab750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.973+0000 7f1387d93b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bd0f4d380 0x555bd0fb6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.973+0000 7f1387d93b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd0fabf20 con 0x555bd0f4ca60 2023-12-07T10:28:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.973+0000 7f1387d93b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd0fac060 con 0x555bd0f4d380 2023-12-07T10:28:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.973+0000 7f1387d93b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bd0fb67d0 con 0x555bd0f47ad0 2023-12-07T10:28:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.973+0000 7f1387d93b80 10 client.0 _ll_register_callbacks cb 0x555bd0e3c8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:28:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.973+0000 7f137219b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bd0f47ad0 0x555bd0fab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.973+0000 7f137219b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bd0f47ad0 0x555bd0fab210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:49976/0 (socket says 172.21.15.119:49976) 2023-12-07T10:28:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.973+0000 7f137219b700 1 -- 172.21.15.119:0/2780951004 learned_addr learned my addr 172.21.15.119:0/2780951004 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:28:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.973+0000 7f137299c700 1 --2- 172.21.15.119:0/2780951004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bd0f4d380 0x555bd0fb6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.973+0000 7f137199a700 1 --2- 172.21.15.119:0/2780951004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bd0f4ca60 0x555bd0fab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.973+0000 7f137219b700 1 -- 172.21.15.119:0/2780951004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bd0f4d380 msgr2=0x555bd0fb6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.973+0000 7f137219b700 1 --2- 172.21.15.119:0/2780951004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bd0f4d380 0x555bd0fb6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.973+0000 7f137219b700 1 -- 172.21.15.119:0/2780951004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bd0f4ca60 msgr2=0x555bd0fab750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.973+0000 7f137219b700 1 --2- 172.21.15.119:0/2780951004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bd0f4ca60 0x555bd0fab750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.973+0000 7f137219b700 1 -- 172.21.15.119:0/2780951004 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555bd0cc4d80 con 0x555bd0f47ad0 2023-12-07T10:28:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.973+0000 7f137219b700 1 --2- 172.21.15.119:0/2780951004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bd0f47ad0 0x555bd0fab210 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f136809a1e0 tx=0x7f13680cb860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d90a8cde232602f0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:28:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.973+0000 7f1355ffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:28:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.974+0000 7f1355ffb700 1 -- 172.21.15.119:0/2780951004 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13680a2ce0 con 0x555bd0f47ad0 2023-12-07T10:28:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.974+0000 7f1355ffb700 1 -- 172.21.15.119:0/2780951004 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f13680ba110 con 0x555bd0f47ad0 2023-12-07T10:28:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.974+0000 7f1387d93b80 1 -- 172.21.15.119:0/2780951004 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1364027070 con 0x555bd0f47ad0 2023-12-07T10:28:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.974+0000 7f1355ffb700 1 -- 172.21.15.119:0/2780951004 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13680ba310 con 0x555bd0f47ad0 2023-12-07T10:28:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.974+0000 7f1387d93b80 10 client.27439 fetch_fsmap learned FSMap version 53 2023-12-07T10:28:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.974+0000 7f1387d93b80 1 -- 172.21.15.119:0/2780951004 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x555bd0ceb1f0 con 0x555bd0f47ad0 2023-12-07T10:28:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.974+0000 7f1355ffb700 1 -- 172.21.15.119:0/2780951004 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1364027070 con 0x555bd0f47ad0 2023-12-07T10:28:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.974+0000 7f1355ffb700 1 -- 172.21.15.119:0/2780951004 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 53) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f136809f7e0 con 0x555bd0f47ad0 2023-12-07T10:28:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.974+0000 7f1387d93b80 10 client.27439 fetch_fsmap finished waiting for FSMap version 53 2023-12-07T10:28:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.974+0000 7f1387d93b80 10 client.27439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:28:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.974+0000 7f1387d93b80 20 client.27439 populate_metadata read hostname 'smithi119' 2023-12-07T10:28:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.974+0000 7f1387d93b80 1 --2- 172.21.15.119:0/2780951004 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555bd0fb7b80 0x555bd0fb9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.974+0000 7f1387d93b80 4 client.27439 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:28:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.974+0000 7f1387d93b80 1 -- 172.21.15.119:0/2780951004 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555bd0e61f70 con 0x555bd0fb7b80 2023-12-07T10:28:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.975+0000 7f137199a700 1 --2- 172.21.15.119:0/2780951004 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555bd0fb7b80 0x555bd0fb9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.975+0000 7f1355ffb700 10 client.27439 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:28:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:24.975+0000 7f137199a700 1 --2- 172.21.15.119:0/2780951004 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555bd0fb7b80 0x555bd0fb9f70 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f1456b5e4409424 server_cookie=9104ba5ed6bedcc2 in_seq=0 out_seq=0 2023-12-07T10:28:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.027+0000 7f1355ffb700 1 -- 172.21.15.119:0/2780951004 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555bd0e61f70 con 0x555bd0fb7b80 2023-12-07T10:28:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.027+0000 7f1355ffb700 10 client.27439 handle_command_reply: tid=0 2023-12-07T10:28:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.027+0000 7f1387d93b80 1 client.27439 shutdown 2023-12-07T10:28:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.027+0000 7f1387d93b80 1 -- 172.21.15.119:0/2780951004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bd0f47ad0 msgr2=0x555bd0fab210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.027+0000 7f1387d93b80 1 --2- 172.21.15.119:0/2780951004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bd0f47ad0 0x555bd0fab210 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f136809a1e0 tx=0x7f13680cb860 comp rx=0 tx=0).stop 2023-12-07T10:28:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.027+0000 7f1387d93b80 1 -- 172.21.15.119:0/2780951004 shutdown_connections 2023-12-07T10:28:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.027+0000 7f1387d93b80 1 --2- 172.21.15.119:0/2780951004 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555bd0fb7b80 0x555bd0fb9f70 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.027+0000 7f1387d93b80 1 --2- 172.21.15.119:0/2780951004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555bd0f4d380 0x555bd0fb6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.027+0000 7f1387d93b80 1 --2- 172.21.15.119:0/2780951004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bd0f47ad0 0x555bd0fab210 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.027+0000 7f1355ffb700 0 client.27439 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:28:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.027+0000 7f1387d93b80 1 --2- 172.21.15.119:0/2780951004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555bd0f4ca60 0x555bd0fab750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.027+0000 7f1387d93b80 1 -- 172.21.15.119:0/2780951004 >> 172.21.15.119:0/2780951004 conn(0x555bd0e76300 msgr2=0x555bd0f4a960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:28:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.027+0000 7f1387d93b80 1 -- 172.21.15.119:0/2780951004 shutdown_connections 2023-12-07T10:28:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.027+0000 7f1387d93b80 1 -- 172.21.15.119:0/2780951004 wait complete. 2023-12-07T10:28:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.028+0000 7f1387d93b80 20 client.27439 trim_cache size 0 max 16384 2023-12-07T10:28:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:28:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:28:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:28:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:28:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.028+0000 7f13743ff700 1 -- 172.21.15.119:0/1525752857 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f13580a1d00 msgr2=0x7f13580a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.028+0000 7f13743ff700 1 --2- 172.21.15.119:0/1525752857 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f13580a1d00 0x7f13580a41c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1364022000 tx=0x7f136401f000 comp rx=0 tx=0).stop 2023-12-07T10:28:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.028+0000 7f13743ff700 1 -- 172.21.15.119:0/1525752857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f136c068ce0 msgr2=0x7f136c19f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.028+0000 7f13743ff700 1 --2- 172.21.15.119:0/1525752857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f136c068ce0 0x7f136c19f900 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f136800e9a0 tx=0x7f136800ecb0 comp rx=0 tx=0).stop 2023-12-07T10:28:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.028+0000 7f13743ff700 1 -- 172.21.15.119:0/1525752857 shutdown_connections 2023-12-07T10:28:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.028+0000 7f13743ff700 1 --2- 172.21.15.119:0/1525752857 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f13580a1d00 0x7f13580a41c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.028+0000 7f13743ff700 1 --2- 172.21.15.119:0/1525752857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f136c106a80 0x7f136c199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.028+0000 7f13743ff700 1 --2- 172.21.15.119:0/1525752857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f136c068ce0 0x7f136c19f900 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.028+0000 7f13743ff700 1 --2- 172.21.15.119:0/1525752857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f136c1060c0 0x7f136c19fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.028+0000 7f13743ff700 1 -- 172.21.15.119:0/1525752857 >> 172.21.15.119:0/1525752857 conn(0x7f136c0775b0 msgr2=0x7f136c0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:28:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.029+0000 7f13743ff700 1 -- 172.21.15.119:0/1525752857 shutdown_connections 2023-12-07T10:28:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:25.029+0000 7f13743ff700 1 -- 172.21.15.119:0/1525752857 wait complete. 2023-12-07T10:28:25.565 INFO:tasks.workunit.client.0.smithi119.stdout: 1 553976 7.15 MB/sec execute 278 sec latency 329.067 ms 2023-12-07T10:28:26.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:26 smithi119 ceph-mon[97908]: pgmap v247: 129 pgs: 129 active+clean; 356 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 182 op/s 2023-12-07T10:28:26.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:26 smithi159 ceph-mon[134726]: pgmap v247: 129 pgs: 129 active+clean; 356 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 182 op/s 2023-12-07T10:28:26.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:26 smithi153 ceph-mon[133625]: pgmap v247: 129 pgs: 129 active+clean; 356 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 182 op/s 2023-12-07T10:28:26.566 INFO:tasks.workunit.client.0.smithi119.stdout: 1 553976 7.12 MB/sec execute 279 sec latency 1245.952 ms 2023-12-07T10:28:27.566 INFO:tasks.workunit.client.0.smithi119.stdout: 1 553976 7.10 MB/sec execute 280 sec latency 2246.088 ms 2023-12-07T10:28:28.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:28 smithi119 ceph-mon[97908]: pgmap v248: 129 pgs: 129 active+clean; 363 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 268 op/s 2023-12-07T10:28:28.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:28 smithi159 ceph-mon[134726]: pgmap v248: 129 pgs: 129 active+clean; 363 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 268 op/s 2023-12-07T10:28:28.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:28 smithi153 ceph-mon[133625]: pgmap v248: 129 pgs: 129 active+clean; 363 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 268 op/s 2023-12-07T10:28:28.566 INFO:tasks.workunit.client.0.smithi119.stdout: 1 553976 7.07 MB/sec execute 281 sec latency 3246.200 ms 2023-12-07T10:28:29.566 INFO:tasks.workunit.client.0.smithi119.stdout: 1 553976 7.05 MB/sec execute 282 sec latency 4246.334 ms 2023-12-07T10:28:30.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:30 smithi119 ceph-mon[97908]: pgmap v249: 129 pgs: 129 active+clean; 363 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 172 op/s 2023-12-07T10:28:30.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:30 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:28:30.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:30 smithi159 ceph-mon[134726]: pgmap v249: 129 pgs: 129 active+clean; 363 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 172 op/s 2023-12-07T10:28:30.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:30 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:28:30.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:30 smithi153 ceph-mon[133625]: pgmap v249: 129 pgs: 129 active+clean; 363 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 172 op/s 2023-12-07T10:28:30.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:30 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:28:30.566 INFO:tasks.workunit.client.0.smithi119.stdout: 1 554382 7.04 MB/sec execute 283 sec latency 4985.851 ms 2023-12-07T10:28:31.566 INFO:tasks.workunit.client.0.smithi119.stdout: 1 555482 7.03 MB/sec execute 284 sec latency 13.800 ms 2023-12-07T10:28:32.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:31 smithi119 ceph-mon[97908]: pgmap v250: 129 pgs: 129 active+clean; 364 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 195 op/s 2023-12-07T10:28:32.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:31 smithi159 ceph-mon[134726]: pgmap v250: 129 pgs: 129 active+clean; 364 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 195 op/s 2023-12-07T10:28:32.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:31 smithi153 ceph-mon[133625]: pgmap v250: 129 pgs: 129 active+clean; 364 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 195 op/s 2023-12-07T10:28:32.566 INFO:tasks.workunit.client.0.smithi119.stdout: 1 557523 7.04 MB/sec execute 285 sec latency 16.780 ms 2023-12-07T10:28:33.566 INFO:tasks.workunit.client.0.smithi119.stdout: 1 558573 7.04 MB/sec execute 286 sec latency 16.473 ms 2023-12-07T10:28:34.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:34 smithi119 ceph-mon[97908]: pgmap v251: 129 pgs: 129 active+clean; 364 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 900 KiB/s wr, 125 op/s 2023-12-07T10:28:34.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:34 smithi159 ceph-mon[134726]: pgmap v251: 129 pgs: 129 active+clean; 364 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 900 KiB/s wr, 125 op/s 2023-12-07T10:28:34.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:34 smithi153 ceph-mon[133625]: pgmap v251: 129 pgs: 129 active+clean; 364 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 900 KiB/s wr, 125 op/s 2023-12-07T10:28:34.566 INFO:tasks.workunit.client.0.smithi119.stdout: 1 559684 7.03 MB/sec execute 287 sec latency 162.081 ms 2023-12-07T10:28:35.566 INFO:tasks.workunit.client.0.smithi119.stdout: 1 561066 7.03 MB/sec execute 288 sec latency 40.067 ms 2023-12-07T10:28:36.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:36 smithi119 ceph-mon[97908]: pgmap v252: 129 pgs: 129 active+clean; 364 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 788 KiB/s wr, 114 op/s 2023-12-07T10:28:36.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:36 smithi159 ceph-mon[134726]: pgmap v252: 129 pgs: 129 active+clean; 364 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 788 KiB/s wr, 114 op/s 2023-12-07T10:28:36.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:36 smithi153 ceph-mon[133625]: pgmap v252: 129 pgs: 129 active+clean; 364 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 788 KiB/s wr, 114 op/s 2023-12-07T10:28:36.566 INFO:tasks.workunit.client.0.smithi119.stdout: 1 562156 7.03 MB/sec execute 289 sec latency 20.187 ms 2023-12-07T10:28:37.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:37 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:28:37.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:37 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:28:37.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:37 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:28:37.566 INFO:tasks.workunit.client.0.smithi119.stdout: 1 564098 7.04 MB/sec execute 290 sec latency 16.238 ms 2023-12-07T10:28:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.201+0000 7f84f71ff700 1 -- 172.21.15.119:0/3680098331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84f01064c0 msgr2=0x7f84f01068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.201+0000 7f84f71ff700 1 --2- 172.21.15.119:0/3680098331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84f01064c0 0x7f84f01068a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f84e400b0a0 tx=0x7f84e400b3b0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.201+0000 7f84f71ff700 1 -- 172.21.15.119:0/3680098331 shutdown_connections 2023-12-07T10:28:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.201+0000 7f84f71ff700 1 --2- 172.21.15.119:0/3680098331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84f0100dd0 0x7f84f010d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.201+0000 7f84f71ff700 1 --2- 172.21.15.119:0/3680098331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84f01064c0 0x7f84f01068a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.201+0000 7f84f71ff700 1 --2- 172.21.15.119:0/3680098331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84f0100410 0x7f84f0100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.201+0000 7f84f71ff700 1 -- 172.21.15.119:0/3680098331 >> 172.21.15.119:0/3680098331 conn(0x7f84f00775b0 msgr2=0x7f84f00779c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:28:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.201+0000 7f84f71ff700 1 -- 172.21.15.119:0/3680098331 shutdown_connections 2023-12-07T10:28:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.201+0000 7f84f71ff700 1 -- 172.21.15.119:0/3680098331 wait complete. 2023-12-07T10:28:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.202+0000 7f84f71ff700 1 Processor -- start 2023-12-07T10:28:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.203+0000 7f84f71ff700 1 -- start start 2023-12-07T10:28:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.203+0000 7f84f71ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84f0100410 0x7f84f019fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.203+0000 7f84f71ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84f0100dd0 0x7f84f019ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.203+0000 7f84f71ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84f01064c0 0x7f84f0199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.203+0000 7f84f71ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f019a2d0 con 0x7f84f0100dd0 2023-12-07T10:28:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.203+0000 7f84f71ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f019a440 con 0x7f84f0100410 2023-12-07T10:28:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.203+0000 7f84f71ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f019a600 con 0x7f84f01064c0 2023-12-07T10:28:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.203+0000 7f84f4f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84f0100410 0x7f84f019fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.203+0000 7f84f4f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84f0100410 0x7f84f019fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:40738/0 (socket says 172.21.15.119:40738) 2023-12-07T10:28:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.203+0000 7f84f4f9b700 1 -- 172.21.15.119:0/2179639821 learned_addr learned my addr 172.21.15.119:0/2179639821 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:28:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.203+0000 7f84f579c700 1 --2- 172.21.15.119:0/2179639821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84f01064c0 0x7f84f0199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.203+0000 7f84effff700 1 --2- 172.21.15.119:0/2179639821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84f0100dd0 0x7f84f019ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.204+0000 7f84f4f9b700 1 -- 172.21.15.119:0/2179639821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84f01064c0 msgr2=0x7f84f0199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:38.210 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.204+0000 7f84f4f9b700 1 --2- 172.21.15.119:0/2179639821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84f01064c0 0x7f84f0199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.210 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.204+0000 7f84f4f9b700 1 -- 172.21.15.119:0/2179639821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84f0100dd0 msgr2=0x7f84f019ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:38.210 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.204+0000 7f84f4f9b700 1 --2- 172.21.15.119:0/2179639821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84f0100dd0 0x7f84f019ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.210 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.204+0000 7f84f4f9b700 1 -- 172.21.15.119:0/2179639821 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e4009d00 con 0x7f84f0100410 2023-12-07T10:28:38.210 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.204+0000 7f84f4f9b700 1 --2- 172.21.15.119:0/2179639821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84f0100410 0x7f84f019fa60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f84e400b070 tx=0x7f84e4007990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:28:38.210 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.204+0000 7f84edffb700 1 -- 172.21.15.119:0/2179639821 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e400e040 con 0x7f84f0100410 2023-12-07T10:28:38.211 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.204+0000 7f84edffb700 1 -- 172.21.15.119:0/2179639821 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84e401d470 con 0x7f84f0100410 2023-12-07T10:28:38.211 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.204+0000 7f84edffb700 1 -- 172.21.15.119:0/2179639821 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e401b6c0 con 0x7f84f0100410 2023-12-07T10:28:38.211 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.204+0000 7f84f71ff700 1 -- 172.21.15.119:0/2179639821 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f019a880 con 0x7f84f0100410 2023-12-07T10:28:38.211 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.204+0000 7f84f71ff700 1 -- 172.21.15.119:0/2179639821 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f01a69e0 con 0x7f84f0100410 2023-12-07T10:28:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.209+0000 7f84edffb700 1 -- 172.21.15.119:0/2179639821 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f84e401d5e0 con 0x7f84f0100410 2023-12-07T10:28:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.210+0000 7f850aaebb80 1 Processor -- start 2023-12-07T10:28:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.210+0000 7f850aaebb80 1 -- start start 2023-12-07T10:28:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.210+0000 7f850aaebb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f837366a60 0x55f837366e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.210+0000 7f850aaebb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837367380 0x55f837361540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.210+0000 7f84edffb700 1 --2- 172.21.15.119:0/2179639821 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f84e00a1c80 0x7f84e00a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.210+0000 7f850aaebb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f837361ad0 0x55f837361f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.210+0000 7f850aaebb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8372d5760 con 0x55f837361ad0 2023-12-07T10:28:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.210+0000 7f850aaebb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8372bc740 con 0x55f837366a60 2023-12-07T10:28:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.210+0000 7f850aaebb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8372ba060 con 0x55f837367380 2023-12-07T10:28:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.211+0000 7f84edffb700 1 -- 172.21.15.119:0/2179639821 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f84e40cec10 con 0x7f84f0100410 2023-12-07T10:28:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.211+0000 7f84effff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837367380 0x55f837361540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.211+0000 7f84effff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837367380 0x55f837361540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:39496/0 (socket says 172.21.15.119:39496) 2023-12-07T10:28:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.211+0000 7f84effff700 1 -- 172.21.15.119:0/4096972772 learned_addr learned my addr 172.21.15.119:0/4096972772 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:28:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.211+0000 7f84f579c700 1 --2- 172.21.15.119:0/4096972772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f837366a60 0x55f837366e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.211+0000 7f84f579c700 1 --2- 172.21.15.119:0/4096972772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f837361ad0 0x55f837361f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.211+0000 7f84f4f9b700 1 --2- 172.21.15.119:0/2179639821 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f84e00a1c80 0x7f84e00a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.211+0000 7f84effff700 1 -- 172.21.15.119:0/4096972772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f837366a60 msgr2=0x55f837366e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.211+0000 7f84effff700 1 --2- 172.21.15.119:0/4096972772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f837366a60 0x55f837366e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.211+0000 7f84effff700 1 -- 172.21.15.119:0/4096972772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f837361ad0 msgr2=0x55f837361f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.211+0000 7f84effff700 1 --2- 172.21.15.119:0/4096972772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f837361ad0 0x55f837361f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.211+0000 7f84effff700 1 -- 172.21.15.119:0/4096972772 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8371325d0 con 0x55f837367380 2023-12-07T10:28:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.211+0000 7f84f579c700 1 --2- 172.21.15.119:0/4096972772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f837361ad0 0x55f837361f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:28:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.211+0000 7f84effff700 1 --2- 172.21.15.119:0/4096972772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837367380 0x55f837361540 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f84dc007ce0 tx=0x7f84dc0103d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79f4c67b82555e3 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:28:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.212+0000 7f84f4f9b700 1 --2- 172.21.15.119:0/2179639821 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f84e00a1c80 0x7f84e00a4140 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f84e409b620 tx=0x7f84e40099e0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:28:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.212+0000 7f84daffd700 1 -- 172.21.15.119:0/4096972772 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc010dc0 con 0x55f837367380 2023-12-07T10:28:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.212+0000 7f84daffd700 1 -- 172.21.15.119:0/4096972772 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84dc015070 con 0x55f837367380 2023-12-07T10:28:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.212+0000 7f84daffd700 1 -- 172.21.15.119:0/4096972772 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc019660 con 0x55f837367380 2023-12-07T10:28:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.212+0000 7f850aaebb80 1 -- 172.21.15.119:0/4096972772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837367380 msgr2=0x55f837361540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.212+0000 7f850aaebb80 1 --2- 172.21.15.119:0/4096972772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837367380 0x55f837361540 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f84dc007ce0 tx=0x7f84dc0103d0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.213+0000 7f850aaebb80 1 -- 172.21.15.119:0/4096972772 shutdown_connections 2023-12-07T10:28:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.213+0000 7f850aaebb80 1 --2- 172.21.15.119:0/4096972772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f837361ad0 0x55f837361f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.213+0000 7f850aaebb80 1 --2- 172.21.15.119:0/4096972772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f837366a60 0x55f837366e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.213+0000 7f850aaebb80 1 --2- 172.21.15.119:0/4096972772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837367380 0x55f837361540 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.213+0000 7f850aaebb80 1 -- 172.21.15.119:0/4096972772 >> 172.21.15.119:0/4096972772 conn(0x55f837290300 msgr2=0x55f83735bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:28:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.213+0000 7f850aaebb80 1 -- 172.21.15.119:0/4096972772 shutdown_connections 2023-12-07T10:28:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.213+0000 7f850aaebb80 1 -- 172.21.15.119:0/4096972772 wait complete. 2023-12-07T10:28:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.213+0000 7f850aaebb80 1 -- start start 2023-12-07T10:28:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.213+0000 7f850aaebb80 1 Processor -- start 2023-12-07T10:28:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.214+0000 7f850aaebb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f837361ad0 0x55f83736d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.214+0000 7f850aaebb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837366a60 0x55f83736a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.214+0000 7f850aaebb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f837367380 0x55f83736adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.214+0000 7f850aaebb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8372b5950 con 0x55f837361ad0 2023-12-07T10:28:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.214+0000 7f850aaebb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f83726f500 con 0x55f837367380 2023-12-07T10:28:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.214+0000 7f850aaebb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f83726f1a0 con 0x55f837366a60 2023-12-07T10:28:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.214+0000 7f850aaebb80 10 client.0 _ll_register_callbacks cb 0x55f8372568e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:28:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.214+0000 7f84f579c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f837361ad0 0x55f83736d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.214+0000 7f84f579c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f837361ad0 0x55f83736d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38266/0 (socket says 172.21.15.119:38266) 2023-12-07T10:28:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.214+0000 7f84f579c700 1 -- 172.21.15.119:0/2934527447 learned_addr learned my addr 172.21.15.119:0/2934527447 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:28:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.215+0000 7f84f579c700 1 --2- 172.21.15.119:0/2934527447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f837367380 0x55f83736adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.215+0000 7f84effff700 1 --2- 172.21.15.119:0/2934527447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837366a60 0x55f83736a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.215+0000 7f84f579c700 1 -- 172.21.15.119:0/2934527447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837366a60 msgr2=0x55f83736a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.215+0000 7f84f579c700 1 --2- 172.21.15.119:0/2934527447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837366a60 0x55f83736a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.215+0000 7f84f579c700 1 -- 172.21.15.119:0/2934527447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f837367380 msgr2=0x55f83736adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.215+0000 7f84f579c700 1 --2- 172.21.15.119:0/2934527447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f837367380 0x55f83736adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.215+0000 7f84f579c700 1 -- 172.21.15.119:0/2934527447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84dc007940 con 0x55f837361ad0 2023-12-07T10:28:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.215+0000 7f84effff700 1 --2- 172.21.15.119:0/2934527447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837366a60 0x55f83736a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:28:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.215+0000 7f84f579c700 1 --2- 172.21.15.119:0/2934527447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f837361ad0 0x55f83736d370 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f84e8011e80 tx=0x7f84e80058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be6a600979fe428 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:28:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.215+0000 7f84cffff700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:28:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.215+0000 7f84cffff700 1 -- 172.21.15.119:0/2934527447 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e8015070 con 0x55f837361ad0 2023-12-07T10:28:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.215+0000 7f850aaebb80 1 -- 172.21.15.119:0/2934527447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f8373546f0 con 0x55f837361ad0 2023-12-07T10:28:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.215+0000 7f84cffff700 1 -- 172.21.15.119:0/2934527447 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f84e8010e10 con 0x55f837361ad0 2023-12-07T10:28:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.216+0000 7f84cffff700 1 -- 172.21.15.119:0/2934527447 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84e8004830 con 0x55f837361ad0 2023-12-07T10:28:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.216+0000 7f850aaebb80 10 client.17430 fetch_fsmap learned FSMap version 55 2023-12-07T10:28:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.216+0000 7f850aaebb80 1 -- 172.21.15.119:0/2934527447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55f8370ded80 con 0x55f837361ad0 2023-12-07T10:28:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.216+0000 7f84cffff700 1 -- 172.21.15.119:0/2934527447 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f8373546f0 con 0x55f837361ad0 2023-12-07T10:28:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.216+0000 7f84cffff700 1 -- 172.21.15.119:0/2934527447 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 55) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f84e8020060 con 0x55f837361ad0 2023-12-07T10:28:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.217+0000 7f850aaebb80 10 client.17430 fetch_fsmap finished waiting for FSMap version 55 2023-12-07T10:28:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.217+0000 7f850aaebb80 10 client.17430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:28:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.217+0000 7f850aaebb80 20 client.17430 populate_metadata read hostname 'smithi119' 2023-12-07T10:28:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.217+0000 7f850aaebb80 1 --2- 172.21.15.119:0/2934527447 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f83737ae30 0x55f83737d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.217+0000 7f850aaebb80 4 client.17430 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:28:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.217+0000 7f850aaebb80 1 -- 172.21.15.119:0/2934527447 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f837330740 con 0x55f83737ae30 2023-12-07T10:28:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.217+0000 7f84effff700 1 --2- 172.21.15.119:0/2934527447 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f83737ae30 0x55f83737d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.217+0000 7f84effff700 1 --2- 172.21.15.119:0/2934527447 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f83737ae30 0x55f83737d250 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1780d7b1b9c895f server_cookie=b4394fda6c70aab7 in_seq=0 out_seq=0 2023-12-07T10:28:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.217+0000 7f84cffff700 10 client.17430 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:28:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.218+0000 7f84cffff700 1 -- 172.21.15.119:0/2934527447 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f837330740 con 0x55f83737ae30 2023-12-07T10:28:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.218+0000 7f84cffff700 10 client.17430 handle_command_reply: tid=0 2023-12-07T10:28:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.218+0000 7f850aaebb80 1 client.17430 shutdown 2023-12-07T10:28:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.218+0000 7f850aaebb80 1 -- 172.21.15.119:0/2934527447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f837361ad0 msgr2=0x55f83736d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.218+0000 7f850aaebb80 1 --2- 172.21.15.119:0/2934527447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f837361ad0 0x55f83736d370 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f84e8011e80 tx=0x7f84e80058e0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.218+0000 7f850aaebb80 1 -- 172.21.15.119:0/2934527447 shutdown_connections 2023-12-07T10:28:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.218+0000 7f850aaebb80 1 --2- 172.21.15.119:0/2934527447 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f83737ae30 0x55f83737d250 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.218+0000 7f850aaebb80 1 --2- 172.21.15.119:0/2934527447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f837367380 0x55f83736adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.218+0000 7f850aaebb80 1 --2- 172.21.15.119:0/2934527447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f837361ad0 0x55f83736d370 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.218+0000 7f84cffff700 0 client.17430 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:28:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.218+0000 7f850aaebb80 1 --2- 172.21.15.119:0/2934527447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837366a60 0x55f83736a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.218+0000 7f850aaebb80 1 -- 172.21.15.119:0/2934527447 >> 172.21.15.119:0/2934527447 conn(0x55f837290300 msgr2=0x55f837356980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:28:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.219+0000 7f850aaebb80 1 -- 172.21.15.119:0/2934527447 shutdown_connections 2023-12-07T10:28:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.219+0000 7f850aaebb80 1 -- 172.21.15.119:0/2934527447 wait complete. 2023-12-07T10:28:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.219+0000 7f850aaebb80 20 client.17430 trim_cache size 0 max 16384 2023-12-07T10:28:38.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.238+0000 7f850aaebb80 1 Processor -- start 2023-12-07T10:28:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.238+0000 7f850aaebb80 1 -- start start 2023-12-07T10:28:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.238+0000 7f850aaebb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837361ad0 0x55f837371e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.238+0000 7f850aaebb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f837366a60 0x55f837372340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.238+0000 7f850aaebb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f837367380 0x55f837364960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.239+0000 7f850aaebb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8371348c0 con 0x55f837367380 2023-12-07T10:28:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.239+0000 7f850aaebb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8373546c0 con 0x55f837366a60 2023-12-07T10:28:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.239+0000 7f850aaebb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f83735c260 con 0x55f837361ad0 2023-12-07T10:28:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.239+0000 7f84effff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f837366a60 0x55f837372340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:38.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.239+0000 7f84f579c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837361ad0 0x55f837371e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:38.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.239+0000 7f84f579c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837361ad0 0x55f837371e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:39510/0 (socket says 172.21.15.119:39510) 2023-12-07T10:28:38.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.239+0000 7f84f579c700 1 -- 172.21.15.119:0/966526821 learned_addr learned my addr 172.21.15.119:0/966526821 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:28:38.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.239+0000 7f84f579c700 1 --2- 172.21.15.119:0/966526821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f837367380 0x55f837364960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:38.242 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.240+0000 7f84f579c700 1 -- 172.21.15.119:0/966526821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f837366a60 msgr2=0x55f837372340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:38.242 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.240+0000 7f84f579c700 1 --2- 172.21.15.119:0/966526821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f837366a60 0x55f837372340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.242 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.240+0000 7f84f579c700 1 -- 172.21.15.119:0/966526821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f837367380 msgr2=0x55f837364960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:38.243 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.240+0000 7f84f579c700 1 --2- 172.21.15.119:0/966526821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f837367380 0x55f837364960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.243 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.240+0000 7f84f579c700 1 -- 172.21.15.119:0/966526821 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8370c7c00 con 0x55f837361ad0 2023-12-07T10:28:38.243 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.240+0000 7f84f579c700 1 --2- 172.21.15.119:0/966526821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837361ad0 0x55f837371e00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f84e8005a80 tx=0x7f84e80058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41c1a08a97209195 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:28:38.243 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.241+0000 7f84db7fe700 1 -- 172.21.15.119:0/966526821 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e8015030 con 0x55f837361ad0 2023-12-07T10:28:38.243 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.241+0000 7f84db7fe700 1 -- 172.21.15.119:0/966526821 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84e80099a0 con 0x55f837361ad0 2023-12-07T10:28:38.243 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.241+0000 7f84db7fe700 1 -- 172.21.15.119:0/966526821 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e8005650 con 0x55f837361ad0 2023-12-07T10:28:38.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.243+0000 7f850aaebb80 1 -- 172.21.15.119:0/966526821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837361ad0 msgr2=0x55f837371e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:38.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.243+0000 7f850aaebb80 1 --2- 172.21.15.119:0/966526821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837361ad0 0x55f837371e00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f84e8005a80 tx=0x7f84e80058e0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.243+0000 7f850aaebb80 1 -- 172.21.15.119:0/966526821 shutdown_connections 2023-12-07T10:28:38.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.243+0000 7f850aaebb80 1 --2- 172.21.15.119:0/966526821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f837367380 0x55f837364960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.243+0000 7f850aaebb80 1 --2- 172.21.15.119:0/966526821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837361ad0 0x55f837371e00 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.243+0000 7f850aaebb80 1 --2- 172.21.15.119:0/966526821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f837366a60 0x55f837372340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.245 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.243+0000 7f850aaebb80 1 -- 172.21.15.119:0/966526821 >> 172.21.15.119:0/966526821 conn(0x55f837290300 msgr2=0x55f83736d030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:28:38.246 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.244+0000 7f850aaebb80 1 -- 172.21.15.119:0/966526821 shutdown_connections 2023-12-07T10:28:38.246 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.244+0000 7f850aaebb80 1 -- 172.21.15.119:0/966526821 wait complete. 2023-12-07T10:28:38.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.245+0000 7f850aaebb80 1 -- start start 2023-12-07T10:28:38.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.245+0000 7f850aaebb80 1 Processor -- start 2023-12-07T10:28:38.247 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.246+0000 7f850aaebb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f837361ad0 0x55f83736fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:38.248 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.246+0000 7f850aaebb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837366a60 0x55f8373702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:38.248 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.246+0000 7f850aaebb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f837367380 0x55f837370830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:38.248 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.246+0000 7f850aaebb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8373ce450 con 0x55f837367380 2023-12-07T10:28:38.248 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.246+0000 7f850aaebb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8373ce5c0 con 0x55f837361ad0 2023-12-07T10:28:38.248 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.246+0000 7f850aaebb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8373ce780 con 0x55f837366a60 2023-12-07T10:28:38.248 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.246+0000 7f850aaebb80 10 client.0 _ll_register_callbacks cb 0x55f8372568e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:28:38.248 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.247+0000 7f84effff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837366a60 0x55f8373702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:38.249 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.247+0000 7f84effff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837366a60 0x55f8373702f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:39524/0 (socket says 172.21.15.119:39524) 2023-12-07T10:28:38.249 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.247+0000 7f84effff700 1 -- 172.21.15.119:0/3035192621 learned_addr learned my addr 172.21.15.119:0/3035192621 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:28:38.250 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.247+0000 7f84f579c700 1 --2- 172.21.15.119:0/3035192621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f837367380 0x55f837370830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:38.250 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.248+0000 7f84effff700 1 -- 172.21.15.119:0/3035192621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f837361ad0 msgr2=0x55f83736fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:38.250 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.248+0000 7f84effff700 1 --2- 172.21.15.119:0/3035192621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f837361ad0 0x55f83736fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.250 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.248+0000 7f84effff700 1 -- 172.21.15.119:0/3035192621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f837367380 msgr2=0x55f837370830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:38.250 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.248+0000 7f84effff700 1 --2- 172.21.15.119:0/3035192621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f837367380 0x55f837370830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.250 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.248+0000 7f84effff700 1 -- 172.21.15.119:0/3035192621 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f8370ded80 con 0x55f837366a60 2023-12-07T10:28:38.250 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.249+0000 7f84effff700 1 --2- 172.21.15.119:0/3035192621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837366a60 0x55f8373702f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55f83735d450 tx=0x7f84dc010ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81ea233d5db8b76f server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:28:38.250 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.249+0000 7f84cffff700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:28:38.250 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.249+0000 7f84cffff700 1 -- 172.21.15.119:0/3035192621 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc005460 con 0x55f837366a60 2023-12-07T10:28:38.250 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.249+0000 7f850aaebb80 1 -- 172.21.15.119:0/3035192621 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84dc015070 con 0x55f837366a60 2023-12-07T10:28:38.251 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.250+0000 7f84cffff700 1 -- 172.21.15.119:0/3035192621 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f84dc0055c0 con 0x55f837366a60 2023-12-07T10:28:38.251 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.250+0000 7f84cffff700 1 -- 172.21.15.119:0/3035192621 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84dc02a130 con 0x55f837366a60 2023-12-07T10:28:38.251 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.250+0000 7f850aaebb80 10 client.27457 fetch_fsmap learned FSMap version 55 2023-12-07T10:28:38.251 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.251+0000 7f850aaebb80 1 -- 172.21.15.119:0/3035192621 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55f8371051f0 con 0x55f837366a60 2023-12-07T10:28:38.251 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.251+0000 7f84cffff700 1 -- 172.21.15.119:0/3035192621 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84dc015070 con 0x55f837366a60 2023-12-07T10:28:38.252 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.251+0000 7f84cffff700 1 -- 172.21.15.119:0/3035192621 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 55) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f84dc003a60 con 0x55f837366a60 2023-12-07T10:28:38.252 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.251+0000 7f850aaebb80 10 client.27457 fetch_fsmap finished waiting for FSMap version 55 2023-12-07T10:28:38.252 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.251+0000 7f850aaebb80 10 client.27457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:28:38.252 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.251+0000 7f850aaebb80 20 client.27457 populate_metadata read hostname 'smithi119' 2023-12-07T10:28:38.252 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.252+0000 7f850aaebb80 1 --2- 172.21.15.119:0/3035192621 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f8373e3060 0x55f8373e5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:38.252 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.252+0000 7f850aaebb80 4 client.27457 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:28:38.252 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.252+0000 7f850aaebb80 1 -- 172.21.15.119:0/3035192621 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f83727bf70 con 0x55f8373e3060 2023-12-07T10:28:38.253 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.252+0000 7f84f579c700 1 --2- 172.21.15.119:0/3035192621 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f8373e3060 0x55f8373e5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:38.254 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.254+0000 7f84f579c700 1 --2- 172.21.15.119:0/3035192621 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f8373e3060 0x55f8373e5480 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b1e911fbb1a1331 server_cookie=d6794aca9058aa85 in_seq=0 out_seq=0 2023-12-07T10:28:38.254 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.254+0000 7f84cffff700 10 client.27457 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:28:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.309+0000 7f84cffff700 1 -- 172.21.15.119:0/3035192621 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f83727bf70 con 0x55f8373e3060 2023-12-07T10:28:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.309+0000 7f84cffff700 10 client.27457 handle_command_reply: tid=0 2023-12-07T10:28:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.309+0000 7f850aaebb80 1 client.27457 shutdown 2023-12-07T10:28:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.309+0000 7f850aaebb80 1 -- 172.21.15.119:0/3035192621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837366a60 msgr2=0x55f8373702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.309+0000 7f850aaebb80 1 --2- 172.21.15.119:0/3035192621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837366a60 0x55f8373702f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55f83735d450 tx=0x7f84dc010ce0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.309+0000 7f850aaebb80 1 -- 172.21.15.119:0/3035192621 shutdown_connections 2023-12-07T10:28:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.309+0000 7f850aaebb80 1 --2- 172.21.15.119:0/3035192621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f837367380 0x55f837370830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.309+0000 7f850aaebb80 1 --2- 172.21.15.119:0/3035192621 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f8373e3060 0x55f8373e5480 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.309+0000 7f850aaebb80 1 --2- 172.21.15.119:0/3035192621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f837361ad0 0x55f83736fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.309+0000 7f850aaebb80 1 --2- 172.21.15.119:0/3035192621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f837366a60 0x55f8373702f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.309+0000 7f84cffff700 0 client.27457 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:28:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.309+0000 7f850aaebb80 1 -- 172.21.15.119:0/3035192621 >> 172.21.15.119:0/3035192621 conn(0x55f837290300 msgr2=0x55f837364960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:28:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.310+0000 7f850aaebb80 1 -- 172.21.15.119:0/3035192621 shutdown_connections 2023-12-07T10:28:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.310+0000 7f850aaebb80 1 -- 172.21.15.119:0/3035192621 wait complete. 2023-12-07T10:28:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.311+0000 7f850aaebb80 20 client.27457 trim_cache size 0 max 16384 2023-12-07T10:28:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:28:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:28:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:28:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:28:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.311+0000 7f84f71ff700 1 -- 172.21.15.119:0/2179639821 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f84e00a1c80 msgr2=0x7f84e00a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.311+0000 7f84f71ff700 1 --2- 172.21.15.119:0/2179639821 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f84e00a1c80 0x7f84e00a4140 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f84e409b620 tx=0x7f84e40099e0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.311+0000 7f84f71ff700 1 -- 172.21.15.119:0/2179639821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84f0100410 msgr2=0x7f84f019fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.311+0000 7f84f71ff700 1 --2- 172.21.15.119:0/2179639821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84f0100410 0x7f84f019fa60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f84e400b070 tx=0x7f84e4007990 comp rx=0 tx=0).stop 2023-12-07T10:28:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.312+0000 7f84f71ff700 1 -- 172.21.15.119:0/2179639821 shutdown_connections 2023-12-07T10:28:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.312+0000 7f84f71ff700 1 --2- 172.21.15.119:0/2179639821 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f84e00a1c80 0x7f84e00a4140 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.312+0000 7f84f71ff700 1 --2- 172.21.15.119:0/2179639821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84f01064c0 0x7f84f0199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.312+0000 7f84f71ff700 1 --2- 172.21.15.119:0/2179639821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84f0100410 0x7f84f019fa60 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.312+0000 7f84f71ff700 1 --2- 172.21.15.119:0/2179639821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84f0100dd0 0x7f84f019ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.312+0000 7f84f71ff700 1 -- 172.21.15.119:0/2179639821 >> 172.21.15.119:0/2179639821 conn(0x7f84f00775b0 msgr2=0x7f84f00ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:28:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.314+0000 7f84f71ff700 1 -- 172.21.15.119:0/2179639821 shutdown_connections 2023-12-07T10:28:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:38.314+0000 7f84f71ff700 1 -- 172.21.15.119:0/2179639821 wait complete. 2023-12-07T10:28:38.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:38 smithi119 ceph-mon[97908]: pgmap v253: 129 pgs: 129 active+clean; 367 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 300 op/s 2023-12-07T10:28:38.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:38 smithi159 ceph-mon[134726]: pgmap v253: 129 pgs: 129 active+clean; 367 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 300 op/s 2023-12-07T10:28:38.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:38 smithi153 ceph-mon[133625]: pgmap v253: 129 pgs: 129 active+clean; 367 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 300 op/s 2023-12-07T10:28:38.566 INFO:tasks.workunit.client.0.smithi119.stdout: 1 565320 7.04 MB/sec execute 291 sec latency 22.070 ms 2023-12-07T10:28:39.566 INFO:tasks.workunit.client.0.smithi119.stdout: 1 567129 7.04 MB/sec execute 292 sec latency 17.497 ms 2023-12-07T10:28:40.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:40 smithi119 ceph-mon[97908]: pgmap v254: 129 pgs: 129 active+clean; 367 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 3.9 MiB/s wr, 215 op/s 2023-12-07T10:28:40.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:40 smithi159 ceph-mon[134726]: pgmap v254: 129 pgs: 129 active+clean; 367 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 3.9 MiB/s wr, 215 op/s 2023-12-07T10:28:40.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:40 smithi153 ceph-mon[133625]: pgmap v254: 129 pgs: 129 active+clean; 367 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 3.9 MiB/s wr, 215 op/s 2023-12-07T10:28:40.566 INFO:tasks.workunit.client.0.smithi119.stdout: 1 568676 7.06 MB/sec execute 293 sec latency 12.859 ms 2023-12-07T10:28:41.566 INFO:tasks.workunit.client.0.smithi119.stdout: 1 570036 7.06 MB/sec execute 294 sec latency 17.463 ms 2023-12-07T10:28:42.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:41 smithi119 ceph-mon[97908]: pgmap v255: 129 pgs: 129 active+clean; 368 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 291 op/s 2023-12-07T10:28:42.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:41 smithi159 ceph-mon[134726]: pgmap v255: 129 pgs: 129 active+clean; 368 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 291 op/s 2023-12-07T10:28:42.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:41 smithi153 ceph-mon[133625]: pgmap v255: 129 pgs: 129 active+clean; 368 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 291 op/s 2023-12-07T10:28:42.566 INFO:tasks.workunit.client.0.smithi119.stdout: 1 572026 7.07 MB/sec execute 295 sec latency 63.502 ms 2023-12-07T10:28:43.567 INFO:tasks.workunit.client.0.smithi119.stdout: 1 573013 7.06 MB/sec execute 296 sec latency 19.540 ms 2023-12-07T10:28:44.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:44 smithi119 ceph-mon[97908]: pgmap v256: 129 pgs: 129 active+clean; 370 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 287 op/s 2023-12-07T10:28:44.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:44 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:28:44.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:44 smithi159 ceph-mon[134726]: pgmap v256: 129 pgs: 129 active+clean; 370 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 287 op/s 2023-12-07T10:28:44.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:44 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:28:44.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:44 smithi153 ceph-mon[133625]: pgmap v256: 129 pgs: 129 active+clean; 370 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 287 op/s 2023-12-07T10:28:44.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:44 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:28:44.567 INFO:tasks.workunit.client.0.smithi119.stdout: 1 574884 7.07 MB/sec execute 297 sec latency 18.893 ms 2023-12-07T10:28:45.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:45 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:28:45.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:45 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:28:45.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:45 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:28:45.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:45 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:28:45.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:45 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:28:45.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:45 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:28:45.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:45 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:28:45.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:45 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:28:45.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:45 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:28:45.567 INFO:tasks.workunit.client.0.smithi119.stdout: 1 576053 7.07 MB/sec execute 298 sec latency 12.634 ms 2023-12-07T10:28:46.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:46 smithi119 ceph-mon[97908]: pgmap v257: 129 pgs: 129 active+clean; 370 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 282 op/s 2023-12-07T10:28:46.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:46 smithi159 ceph-mon[134726]: pgmap v257: 129 pgs: 129 active+clean; 370 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 282 op/s 2023-12-07T10:28:46.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:46 smithi153 ceph-mon[133625]: pgmap v257: 129 pgs: 129 active+clean; 370 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 282 op/s 2023-12-07T10:28:46.567 INFO:tasks.workunit.client.0.smithi119.stdout: 1 577832 7.07 MB/sec execute 299 sec latency 16.320 ms 2023-12-07T10:28:47.567 INFO:tasks.workunit.client.0.smithi119.stdout: 1 579341 7.09 MB/sec execute 300 sec latency 12.952 ms 2023-12-07T10:28:48.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:48 smithi119 ceph-mon[97908]: pgmap v258: 129 pgs: 129 active+clean; 378 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 367 op/s 2023-12-07T10:28:48.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:48 smithi159 ceph-mon[134726]: pgmap v258: 129 pgs: 129 active+clean; 378 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 367 op/s 2023-12-07T10:28:48.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:48 smithi153 ceph-mon[133625]: pgmap v258: 129 pgs: 129 active+clean; 378 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.3 MiB/s wr, 367 op/s 2023-12-07T10:28:48.567 INFO:tasks.workunit.client.0.smithi119.stdout: 1 580568 7.08 MB/sec execute 301 sec latency 15.132 ms 2023-12-07T10:28:49.567 INFO:tasks.workunit.client.0.smithi119.stdout: 1 582464 7.10 MB/sec execute 302 sec latency 25.926 ms 2023-12-07T10:28:50.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:50 smithi119 ceph-mon[97908]: pgmap v259: 129 pgs: 129 active+clean; 378 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 180 op/s 2023-12-07T10:28:50.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:50 smithi159 ceph-mon[134726]: pgmap v259: 129 pgs: 129 active+clean; 378 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 180 op/s 2023-12-07T10:28:50.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:50 smithi153 ceph-mon[133625]: pgmap v259: 129 pgs: 129 active+clean; 378 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 180 op/s 2023-12-07T10:28:50.567 INFO:tasks.workunit.client.0.smithi119.stdout: 1 583505 7.08 MB/sec execute 303 sec latency 15.543 ms 2023-12-07T10:28:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.484+0000 7f91d81ff700 1 -- 172.21.15.119:0/308694384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91d010dda0 msgr2=0x7f91d01101d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.484+0000 7f91d81ff700 1 --2- 172.21.15.119:0/308694384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91d010dda0 0x7f91d01101d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f91c800b0a0 tx=0x7f91c800b3b0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.484+0000 7f91d81ff700 1 -- 172.21.15.119:0/308694384 shutdown_connections 2023-12-07T10:28:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.484+0000 7f91d81ff700 1 --2- 172.21.15.119:0/308694384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91d010dda0 0x7f91d01101d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.484+0000 7f91d81ff700 1 --2- 172.21.15.119:0/308694384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d00739e0 0x7f91d0073dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.484+0000 7f91d81ff700 1 --2- 172.21.15.119:0/308694384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91d0074300 0x7f91d010d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.484+0000 7f91d81ff700 1 -- 172.21.15.119:0/308694384 >> 172.21.15.119:0/308694384 conn(0x7f91d00fd010 msgr2=0x7f91d00ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:28:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.484+0000 7f91d81ff700 1 -- 172.21.15.119:0/308694384 shutdown_connections 2023-12-07T10:28:51.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.484+0000 7f91d81ff700 1 -- 172.21.15.119:0/308694384 wait complete. 2023-12-07T10:28:51.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.486+0000 7f91d81ff700 1 Processor -- start 2023-12-07T10:28:51.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.486+0000 7f91d81ff700 1 -- start start 2023-12-07T10:28:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.486+0000 7f91d81ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91d00739e0 0x7f91d0199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.486+0000 7f91d81ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91d0074300 0x7f91d019a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.486+0000 7f91d81ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d010dda0 0x7f91d019d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.487+0000 7f91d81ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d019db60 con 0x7f91d010dda0 2023-12-07T10:28:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.487+0000 7f91d81ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d019dcd0 con 0x7f91d00739e0 2023-12-07T10:28:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.487+0000 7f91d81ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d019de90 con 0x7f91d0074300 2023-12-07T10:28:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.487+0000 7f91d5f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91d00739e0 0x7f91d0199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.487+0000 7f91d5f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91d00739e0 0x7f91d0199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:36080/0 (socket says 172.21.15.119:36080) 2023-12-07T10:28:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.487+0000 7f91d5f9b700 1 -- 172.21.15.119:0/3659506916 learned_addr learned my addr 172.21.15.119:0/3659506916 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:28:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.487+0000 7f91d579a700 1 --2- 172.21.15.119:0/3659506916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91d0074300 0x7f91d019a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.487+0000 7f91d679c700 1 --2- 172.21.15.119:0/3659506916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d010dda0 0x7f91d019d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.487+0000 7f91d5f9b700 1 -- 172.21.15.119:0/3659506916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91d0074300 msgr2=0x7f91d019a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.487+0000 7f91d5f9b700 1 --2- 172.21.15.119:0/3659506916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91d0074300 0x7f91d019a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.487+0000 7f91d5f9b700 1 -- 172.21.15.119:0/3659506916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d010dda0 msgr2=0x7f91d019d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.487+0000 7f91d5f9b700 1 --2- 172.21.15.119:0/3659506916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d010dda0 0x7f91d019d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.487+0000 7f91d5f9b700 1 -- 172.21.15.119:0/3659506916 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c8009d00 con 0x7f91d00739e0 2023-12-07T10:28:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.487+0000 7f91d5f9b700 1 --2- 172.21.15.119:0/3659506916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91d00739e0 0x7f91d0199d30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00e9a0 tx=0x7f91cc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:28:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.488+0000 7f91c6ffd700 1 -- 172.21.15.119:0/3659506916 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc009790 con 0x7f91d00739e0 2023-12-07T10:28:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.488+0000 7f91d81ff700 1 -- 172.21.15.119:0/3659506916 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d019e170 con 0x7f91d00739e0 2023-12-07T10:28:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.488+0000 7f91c6ffd700 1 -- 172.21.15.119:0/3659506916 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91cc010430 con 0x7f91d00739e0 2023-12-07T10:28:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.488+0000 7f91c6ffd700 1 -- 172.21.15.119:0/3659506916 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc018680 con 0x7f91d00739e0 2023-12-07T10:28:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.488+0000 7f91d81ff700 1 -- 172.21.15.119:0/3659506916 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d01a6ad0 con 0x7f91d00739e0 2023-12-07T10:28:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.492+0000 7f91c6ffd700 1 -- 172.21.15.119:0/3659506916 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f91cc0105a0 con 0x7f91d00739e0 2023-12-07T10:28:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.494+0000 7f91eba5bb80 1 Processor -- start 2023-12-07T10:28:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.494+0000 7f91c6ffd700 1 --2- 172.21.15.119:0/3659506916 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f91bc0a1c80 0x7f91bc0a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.494+0000 7f91eba5bb80 1 -- start start 2023-12-07T10:28:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.494+0000 7f91c6ffd700 1 -- 172.21.15.119:0/3659506916 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f91cc014070 con 0x7f91d00739e0 2023-12-07T10:28:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.494+0000 7f91eba5bb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ea97f4a60 0x564ea97f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.494+0000 7f91d679c700 1 --2- 172.21.15.119:0/3659506916 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f91bc0a1c80 0x7f91bc0a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.494+0000 7f91eba5bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea97f5380 0x564ea97ef540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.494+0000 7f91eba5bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea97efad0 0x564ea97eff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.494+0000 7f91eba5bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea9763760 con 0x564ea97f5380 2023-12-07T10:28:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.494+0000 7f91eba5bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea974a740 con 0x564ea97efad0 2023-12-07T10:28:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.494+0000 7f91eba5bb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea9748060 con 0x564ea97f4a60 2023-12-07T10:28:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.495+0000 7f91d5f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ea97f4a60 0x564ea97f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.495+0000 7f91d5f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ea97f4a60 0x564ea97f4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:36392/0 (socket says 172.21.15.119:36392) 2023-12-07T10:28:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.495+0000 7f91d679c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea97efad0 0x564ea97eff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.495+0000 7f91d5f9b700 1 -- 172.21.15.119:0/3218527798 learned_addr learned my addr 172.21.15.119:0/3218527798 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:28:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.495+0000 7f91d579a700 1 --2- 172.21.15.119:0/3218527798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea97f5380 0x564ea97ef540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.495+0000 7f91d5f9b700 1 -- 172.21.15.119:0/3218527798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea97efad0 msgr2=0x564ea97eff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.495+0000 7f91d5f9b700 1 --2- 172.21.15.119:0/3218527798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea97efad0 0x564ea97eff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.495+0000 7f91d5f9b700 1 -- 172.21.15.119:0/3218527798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea97f5380 msgr2=0x564ea97ef540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.495+0000 7f91d5f9b700 1 --2- 172.21.15.119:0/3218527798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea97f5380 0x564ea97ef540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.495+0000 7f91d5f9b700 1 -- 172.21.15.119:0/3218527798 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ea95c05d0 con 0x564ea97f4a60 2023-12-07T10:28:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.495+0000 7f91d679c700 1 --2- 172.21.15.119:0/3659506916 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f91bc0a1c80 0x7f91bc0a4140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f91c800b780 tx=0x7f91c8007d90 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:28:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.495+0000 7f91d5f9b700 1 --2- 172.21.15.119:0/3218527798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ea97f4a60 0x564ea97f4e40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0a2760 tx=0x7f91cc09a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=444fa1e97ee096aa server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:28:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.496+0000 7f91bbfff700 1 -- 172.21.15.119:0/3218527798 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc09ea50 con 0x564ea97f4a60 2023-12-07T10:28:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.496+0000 7f91bbfff700 1 -- 172.21.15.119:0/3218527798 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91cc0a2c90 con 0x564ea97f4a60 2023-12-07T10:28:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.496+0000 7f91bbfff700 1 -- 172.21.15.119:0/3218527798 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0ba2d0 con 0x564ea97f4a60 2023-12-07T10:28:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.496+0000 7f91eba5bb80 1 -- 172.21.15.119:0/3218527798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ea97f4a60 msgr2=0x564ea97f4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.496+0000 7f91eba5bb80 1 --2- 172.21.15.119:0/3218527798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ea97f4a60 0x564ea97f4e40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0a2760 tx=0x7f91cc09a330 comp rx=0 tx=0).stop 2023-12-07T10:28:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.497+0000 7f91eba5bb80 1 -- 172.21.15.119:0/3218527798 shutdown_connections 2023-12-07T10:28:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.497+0000 7f91eba5bb80 1 --2- 172.21.15.119:0/3218527798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea97efad0 0x564ea97eff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.497+0000 7f91eba5bb80 1 --2- 172.21.15.119:0/3218527798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ea97f4a60 0x564ea97f4e40 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.497+0000 7f91eba5bb80 1 --2- 172.21.15.119:0/3218527798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea97f5380 0x564ea97ef540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.497+0000 7f91eba5bb80 1 -- 172.21.15.119:0/3218527798 >> 172.21.15.119:0/3218527798 conn(0x564ea971e300 msgr2=0x564ea97e9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:28:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.497+0000 7f91eba5bb80 1 -- 172.21.15.119:0/3218527798 shutdown_connections 2023-12-07T10:28:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.497+0000 7f91eba5bb80 1 -- 172.21.15.119:0/3218527798 wait complete. 2023-12-07T10:28:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.497+0000 7f91eba5bb80 1 -- start start 2023-12-07T10:28:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.497+0000 7f91eba5bb80 1 Processor -- start 2023-12-07T10:28:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.498+0000 7f91eba5bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea97efad0 0x564ea97f8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.498+0000 7f91eba5bb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ea97f4a60 0x564ea97f8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.498+0000 7f91eba5bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea97f5380 0x564ea97f91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.498+0000 7f91eba5bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea9743950 con 0x564ea97f5380 2023-12-07T10:28:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.498+0000 7f91eba5bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea96fd500 con 0x564ea97efad0 2023-12-07T10:28:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.498+0000 7f91eba5bb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea96fd1a0 con 0x564ea97f4a60 2023-12-07T10:28:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.498+0000 7f91eba5bb80 10 client.0 _ll_register_callbacks cb 0x564ea96e48e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:28:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.498+0000 7f91d5f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea97efad0 0x564ea97f8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.498+0000 7f91d5f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea97efad0 0x564ea97f8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:36104/0 (socket says 172.21.15.119:36104) 2023-12-07T10:28:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.498+0000 7f91d5f9b700 1 -- 172.21.15.119:0/3199326026 learned_addr learned my addr 172.21.15.119:0/3199326026 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:28:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.498+0000 7f91d679c700 1 --2- 172.21.15.119:0/3199326026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea97f5380 0x564ea97f91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.498+0000 7f91d579a700 1 --2- 172.21.15.119:0/3199326026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ea97f4a60 0x564ea97f8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.498+0000 7f91d5f9b700 1 -- 172.21.15.119:0/3199326026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ea97f4a60 msgr2=0x564ea97f8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.498+0000 7f91d5f9b700 1 --2- 172.21.15.119:0/3199326026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ea97f4a60 0x564ea97f8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.498+0000 7f91d5f9b700 1 -- 172.21.15.119:0/3199326026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea97f5380 msgr2=0x564ea97f91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.498+0000 7f91d5f9b700 1 --2- 172.21.15.119:0/3199326026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea97f5380 0x564ea97f91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.498+0000 7f91d5f9b700 1 -- 172.21.15.119:0/3199326026 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d01a6ad0 con 0x564ea97efad0 2023-12-07T10:28:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.499+0000 7f91d5f9b700 1 --2- 172.21.15.119:0/3199326026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea97efad0 0x564ea97f8740 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0ce030 tx=0x7f91cc096d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36bd2bbf20121da8 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:28:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.499+0000 7f91b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:28:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.499+0000 7f91b97fa700 1 -- 172.21.15.119:0/3199326026 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc010880 con 0x564ea97efad0 2023-12-07T10:28:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.499+0000 7f91b97fa700 1 -- 172.21.15.119:0/3199326026 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f91cc09ab60 con 0x564ea97efad0 2023-12-07T10:28:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.499+0000 7f91eba5bb80 1 -- 172.21.15.119:0/3199326026 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ea97e26f0 con 0x564ea97efad0 2023-12-07T10:28:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.499+0000 7f91b97fa700 1 -- 172.21.15.119:0/3199326026 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91cc0109e0 con 0x564ea97efad0 2023-12-07T10:28:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.499+0000 7f91eba5bb80 10 client.27485 fetch_fsmap learned FSMap version 55 2023-12-07T10:28:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.499+0000 7f91eba5bb80 1 -- 172.21.15.119:0/3199326026 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x564ea956cd80 con 0x564ea97efad0 2023-12-07T10:28:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.499+0000 7f91b97fa700 1 -- 172.21.15.119:0/3199326026 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564ea97e26f0 con 0x564ea97efad0 2023-12-07T10:28:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.499+0000 7f91b97fa700 1 -- 172.21.15.119:0/3199326026 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 55) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f91cc0a2800 con 0x564ea97efad0 2023-12-07T10:28:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.500+0000 7f91eba5bb80 10 client.27485 fetch_fsmap finished waiting for FSMap version 55 2023-12-07T10:28:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.500+0000 7f91eba5bb80 10 client.27485 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:28:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.500+0000 7f91eba5bb80 20 client.27485 populate_metadata read hostname 'smithi119' 2023-12-07T10:28:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.500+0000 7f91eba5bb80 1 --2- 172.21.15.119:0/3199326026 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x564ea9808ad0 0x564ea980aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.500+0000 7f91eba5bb80 4 client.27485 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:28:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.500+0000 7f91eba5bb80 1 -- 172.21.15.119:0/3199326026 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ea97be740 con 0x564ea9808ad0 2023-12-07T10:28:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.500+0000 7f91d579a700 1 --2- 172.21.15.119:0/3199326026 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x564ea9808ad0 0x564ea980aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.500+0000 7f91d579a700 1 --2- 172.21.15.119:0/3199326026 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x564ea9808ad0 0x564ea980aec0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e373a439ec9e05 server_cookie=37770f06751b6792 in_seq=0 out_seq=0 2023-12-07T10:28:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.500+0000 7f91b97fa700 10 client.27485 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:28:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.501+0000 7f91b97fa700 1 -- 172.21.15.119:0/3199326026 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564ea97be740 con 0x564ea9808ad0 2023-12-07T10:28:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.501+0000 7f91b97fa700 10 client.27485 handle_command_reply: tid=0 2023-12-07T10:28:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.501+0000 7f91eba5bb80 1 client.27485 shutdown 2023-12-07T10:28:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.501+0000 7f91eba5bb80 1 -- 172.21.15.119:0/3199326026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea97efad0 msgr2=0x564ea97f8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.501+0000 7f91eba5bb80 1 --2- 172.21.15.119:0/3199326026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea97efad0 0x564ea97f8740 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0ce030 tx=0x7f91cc096d50 comp rx=0 tx=0).stop 2023-12-07T10:28:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.501+0000 7f91eba5bb80 1 -- 172.21.15.119:0/3199326026 shutdown_connections 2023-12-07T10:28:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.501+0000 7f91eba5bb80 1 --2- 172.21.15.119:0/3199326026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea97f5380 0x564ea97f91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.501+0000 7f91eba5bb80 1 --2- 172.21.15.119:0/3199326026 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x564ea9808ad0 0x564ea980aec0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.501+0000 7f91eba5bb80 1 --2- 172.21.15.119:0/3199326026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea97efad0 0x564ea97f8740 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.501+0000 7f91b97fa700 0 client.27485 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:28:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.501+0000 7f91eba5bb80 1 --2- 172.21.15.119:0/3199326026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ea97f4a60 0x564ea97f8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.501+0000 7f91eba5bb80 1 -- 172.21.15.119:0/3199326026 >> 172.21.15.119:0/3199326026 conn(0x564ea971e300 msgr2=0x564ea97e4980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:28:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.502+0000 7f91eba5bb80 1 -- 172.21.15.119:0/3199326026 shutdown_connections 2023-12-07T10:28:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.502+0000 7f91eba5bb80 1 -- 172.21.15.119:0/3199326026 wait complete. 2023-12-07T10:28:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.502+0000 7f91eba5bb80 20 client.27485 trim_cache size 0 max 16384 2023-12-07T10:28:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.519+0000 7f91eba5bb80 1 Processor -- start 2023-12-07T10:28:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.519+0000 7f91eba5bb80 1 -- start start 2023-12-07T10:28:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.520+0000 7f91eba5bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea97efad0 0x564ea97ef540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.520+0000 7f91eba5bb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ea97f4a60 0x564ea97f8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.520+0000 7f91eba5bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea97f5380 0x564ea97ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.520+0000 7f91eba5bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea95c28c0 con 0x564ea97efad0 2023-12-07T10:28:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.520+0000 7f91eba5bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea97e67a0 con 0x564ea97f5380 2023-12-07T10:28:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.520+0000 7f91eba5bb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea98100b0 con 0x564ea97f4a60 2023-12-07T10:28:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.520+0000 7f91d579a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ea97f4a60 0x564ea97f8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.520+0000 7f91d679c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea97f5380 0x564ea97ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.520+0000 7f91d5f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea97efad0 0x564ea97ef540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.520+0000 7f91d579a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ea97f4a60 0x564ea97f8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:36396/0 (socket says 172.21.15.119:36396) 2023-12-07T10:28:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.520+0000 7f91d579a700 1 -- 172.21.15.119:0/893072499 learned_addr learned my addr 172.21.15.119:0/893072499 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:28:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.520+0000 7f91d579a700 1 -- 172.21.15.119:0/893072499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea97f5380 msgr2=0x564ea97ffdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.520+0000 7f91d579a700 1 --2- 172.21.15.119:0/893072499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea97f5380 0x564ea97ffdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.520+0000 7f91d579a700 1 -- 172.21.15.119:0/893072499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea97efad0 msgr2=0x564ea97ef540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.520+0000 7f91d579a700 1 --2- 172.21.15.119:0/893072499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea97efad0 0x564ea97ef540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.520+0000 7f91d579a700 1 -- 172.21.15.119:0/893072499 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ea9555c00 con 0x564ea97f4a60 2023-12-07T10:28:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.520+0000 7f91d579a700 1 --2- 172.21.15.119:0/893072499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ea97f4a60 0x564ea97f8fe0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f91d019b240 tx=0x7f91c0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bcd837ef3b3c6f2 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:28:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.521+0000 7f91c4ff9700 1 -- 172.21.15.119:0/893072499 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c00039a0 con 0x564ea97f4a60 2023-12-07T10:28:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.521+0000 7f91c4ff9700 1 -- 172.21.15.119:0/893072499 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91c0018040 con 0x564ea97f4a60 2023-12-07T10:28:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.521+0000 7f91c4ff9700 1 -- 172.21.15.119:0/893072499 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c0010690 con 0x564ea97f4a60 2023-12-07T10:28:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.522+0000 7f91eba5bb80 1 -- 172.21.15.119:0/893072499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ea97f4a60 msgr2=0x564ea97f8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.522+0000 7f91eba5bb80 1 --2- 172.21.15.119:0/893072499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ea97f4a60 0x564ea97f8fe0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f91d019b240 tx=0x7f91c0004750 comp rx=0 tx=0).stop 2023-12-07T10:28:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.522+0000 7f91eba5bb80 1 -- 172.21.15.119:0/893072499 shutdown_connections 2023-12-07T10:28:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.522+0000 7f91eba5bb80 1 --2- 172.21.15.119:0/893072499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea97f5380 0x564ea97ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.522+0000 7f91eba5bb80 1 --2- 172.21.15.119:0/893072499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea97efad0 0x564ea97ef540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.522+0000 7f91eba5bb80 1 --2- 172.21.15.119:0/893072499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ea97f4a60 0x564ea97f8fe0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.522+0000 7f91eba5bb80 1 -- 172.21.15.119:0/893072499 >> 172.21.15.119:0/893072499 conn(0x564ea971e300 msgr2=0x564ea97f2990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:28:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.522+0000 7f91eba5bb80 1 -- 172.21.15.119:0/893072499 shutdown_connections 2023-12-07T10:28:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.522+0000 7f91eba5bb80 1 -- 172.21.15.119:0/893072499 wait complete. 2023-12-07T10:28:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.522+0000 7f91eba5bb80 1 -- start start 2023-12-07T10:28:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.523+0000 7f91eba5bb80 1 Processor -- start 2023-12-07T10:28:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.523+0000 7f91eba5bb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ea97efad0 0x564ea97fdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.523+0000 7f91eba5bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea97f4a60 0x564ea97fe450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.523+0000 7f91eba5bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea97f5380 0x564ea98667b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.523+0000 7f91eba5bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea97feb10 con 0x564ea97f5380 2023-12-07T10:28:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.523+0000 7f91eba5bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea9866cf0 con 0x564ea97f4a60 2023-12-07T10:28:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.523+0000 7f91eba5bb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea9866e30 con 0x564ea97efad0 2023-12-07T10:28:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.523+0000 7f91eba5bb80 10 client.0 _ll_register_callbacks cb 0x564ea96e48e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:28:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.523+0000 7f91d579a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea97f4a60 0x564ea97fe450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.523+0000 7f91d579a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea97f4a60 0x564ea97fe450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:36112/0 (socket says 172.21.15.119:36112) 2023-12-07T10:28:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.524+0000 7f91d579a700 1 -- 172.21.15.119:0/1660734763 learned_addr learned my addr 172.21.15.119:0/1660734763 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:28:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.524+0000 7f91d5f9b700 1 --2- 172.21.15.119:0/1660734763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ea97efad0 0x564ea97fdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.524+0000 7f91d679c700 1 --2- 172.21.15.119:0/1660734763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea97f5380 0x564ea98667b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.524+0000 7f91d5f9b700 1 -- 172.21.15.119:0/1660734763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea97f4a60 msgr2=0x564ea97fe450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.524+0000 7f91d5f9b700 1 --2- 172.21.15.119:0/1660734763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea97f4a60 0x564ea97fe450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.524+0000 7f91d5f9b700 1 -- 172.21.15.119:0/1660734763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea97f5380 msgr2=0x564ea98667b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:28:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.524+0000 7f91d5f9b700 1 --2- 172.21.15.119:0/1660734763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea97f5380 0x564ea98667b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.524+0000 7f91d5f9b700 1 -- 172.21.15.119:0/1660734763 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c0009e20 con 0x564ea97efad0 2023-12-07T10:28:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.524+0000 7f91d579a700 1 --2- 172.21.15.119:0/1660734763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea97f4a60 0x564ea97fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:28:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.524+0000 7f91d5f9b700 1 --2- 172.21.15.119:0/1660734763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ea97efad0 0x564ea97fdf10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f91cc097370 tx=0x7f91cc010ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afadc1fd32814b86 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:28:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.524+0000 7f91b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:28:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.524+0000 7f91b97fa700 1 -- 172.21.15.119:0/1660734763 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc09a000 con 0x564ea97efad0 2023-12-07T10:28:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.524+0000 7f91b97fa700 1 -- 172.21.15.119:0/1660734763 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f91cc09ab60 con 0x564ea97efad0 2023-12-07T10:28:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.524+0000 7f91eba5bb80 1 -- 172.21.15.119:0/1660734763 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91cc0a2c90 con 0x564ea97efad0 2023-12-07T10:28:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.525+0000 7f91b97fa700 1 -- 172.21.15.119:0/1660734763 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91cc09ad20 con 0x564ea97efad0 2023-12-07T10:28:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.525+0000 7f91eba5bb80 10 client.27475 fetch_fsmap learned FSMap version 55 2023-12-07T10:28:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.525+0000 7f91eba5bb80 1 -- 172.21.15.119:0/1660734763 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x564ea95931f0 con 0x564ea97efad0 2023-12-07T10:28:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.525+0000 7f91b97fa700 1 -- 172.21.15.119:0/1660734763 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91cc0a2c90 con 0x564ea97efad0 2023-12-07T10:28:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.525+0000 7f91b97fa700 1 -- 172.21.15.119:0/1660734763 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 55) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f91cc0ae870 con 0x564ea97efad0 2023-12-07T10:28:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.525+0000 7f91eba5bb80 10 client.27475 fetch_fsmap finished waiting for FSMap version 55 2023-12-07T10:28:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.525+0000 7f91eba5bb80 10 client.27475 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:28:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.525+0000 7f91eba5bb80 20 client.27475 populate_metadata read hostname 'smithi119' 2023-12-07T10:28:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.525+0000 7f91eba5bb80 1 --2- 172.21.15.119:0/1660734763 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x564ea9868100 0x564ea986a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:28:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.525+0000 7f91eba5bb80 4 client.27475 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:28:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.525+0000 7f91eba5bb80 1 -- 172.21.15.119:0/1660734763 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564ea9709f70 con 0x564ea9868100 2023-12-07T10:28:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.526+0000 7f91d579a700 1 --2- 172.21.15.119:0/1660734763 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x564ea9868100 0x564ea986a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:28:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.526+0000 7f91d579a700 1 --2- 172.21.15.119:0/1660734763 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x564ea9868100 0x564ea986a4f0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94af9a2956a8308e server_cookie=3a4a29bd4cccc85e in_seq=0 out_seq=0 2023-12-07T10:28:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.526+0000 7f91b97fa700 10 client.27475 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:28:51.567 INFO:tasks.workunit.client.0.smithi119.stdout: 1 585429 7.10 MB/sec execute 304 sec latency 36.904 ms 2023-12-07T10:28:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.580+0000 7f91b97fa700 1 -- 172.21.15.119:0/1660734763 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564ea9709f70 con 0x564ea9868100 2023-12-07T10:28:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.580+0000 7f91b97fa700 10 client.27475 handle_command_reply: tid=0 2023-12-07T10:28:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.581+0000 7f91eba5bb80 1 client.27475 shutdown 2023-12-07T10:28:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.581+0000 7f91eba5bb80 1 -- 172.21.15.119:0/1660734763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ea97efad0 msgr2=0x564ea97fdf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.581+0000 7f91eba5bb80 1 --2- 172.21.15.119:0/1660734763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ea97efad0 0x564ea97fdf10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f91cc097370 tx=0x7f91cc010ab0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.581+0000 7f91eba5bb80 1 -- 172.21.15.119:0/1660734763 shutdown_connections 2023-12-07T10:28:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.581+0000 7f91eba5bb80 1 --2- 172.21.15.119:0/1660734763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ea97f5380 0x564ea98667b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.581+0000 7f91eba5bb80 1 --2- 172.21.15.119:0/1660734763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ea97efad0 0x564ea97fdf10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.581+0000 7f91eba5bb80 1 --2- 172.21.15.119:0/1660734763 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x564ea9868100 0x564ea986a4f0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.581+0000 7f91eba5bb80 1 --2- 172.21.15.119:0/1660734763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564ea97f4a60 0x564ea97fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.581+0000 7f91b97fa700 0 client.27475 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:28:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.581+0000 7f91eba5bb80 1 -- 172.21.15.119:0/1660734763 >> 172.21.15.119:0/1660734763 conn(0x564ea971e300 msgr2=0x564ea97f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:28:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.581+0000 7f91eba5bb80 1 -- 172.21.15.119:0/1660734763 shutdown_connections 2023-12-07T10:28:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.581+0000 7f91eba5bb80 1 -- 172.21.15.119:0/1660734763 wait complete. 2023-12-07T10:28:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.581+0000 7f91eba5bb80 20 client.27475 trim_cache size 0 max 16384 2023-12-07T10:28:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:28:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:28:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:28:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:28:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.582+0000 7f91d81ff700 1 -- 172.21.15.119:0/3659506916 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f91bc0a1c80 msgr2=0x7f91bc0a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.582+0000 7f91d81ff700 1 --2- 172.21.15.119:0/3659506916 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f91bc0a1c80 0x7f91bc0a4140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f91c800b780 tx=0x7f91c8007d90 comp rx=0 tx=0).stop 2023-12-07T10:28:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.582+0000 7f91d81ff700 1 -- 172.21.15.119:0/3659506916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91d00739e0 msgr2=0x7f91d0199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:28:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.582+0000 7f91d81ff700 1 --2- 172.21.15.119:0/3659506916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91d00739e0 0x7f91d0199d30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00e9a0 tx=0x7f91cc00ecb0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.582+0000 7f91d81ff700 1 -- 172.21.15.119:0/3659506916 shutdown_connections 2023-12-07T10:28:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.582+0000 7f91d81ff700 1 --2- 172.21.15.119:0/3659506916 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f91bc0a1c80 0x7f91bc0a4140 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.582+0000 7f91d81ff700 1 --2- 172.21.15.119:0/3659506916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d010dda0 0x7f91d019d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.582+0000 7f91d81ff700 1 --2- 172.21.15.119:0/3659506916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91d00739e0 0x7f91d0199d30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.582+0000 7f91d81ff700 1 --2- 172.21.15.119:0/3659506916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91d0074300 0x7f91d019a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:28:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.582+0000 7f91d81ff700 1 -- 172.21.15.119:0/3659506916 >> 172.21.15.119:0/3659506916 conn(0x7f91d00fd010 msgr2=0x7f91d00fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:28:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.582+0000 7f91d81ff700 1 -- 172.21.15.119:0/3659506916 shutdown_connections 2023-12-07T10:28:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:28:51.583+0000 7f91d81ff700 1 -- 172.21.15.119:0/3659506916 wait complete. 2023-12-07T10:28:52.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:51 smithi119 ceph-mon[97908]: pgmap v260: 129 pgs: 129 active+clean; 381 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 226 op/s 2023-12-07T10:28:52.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:51 smithi159 ceph-mon[134726]: pgmap v260: 129 pgs: 129 active+clean; 381 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 226 op/s 2023-12-07T10:28:52.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:51 smithi153 ceph-mon[133625]: pgmap v260: 129 pgs: 129 active+clean; 381 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 226 op/s 2023-12-07T10:28:52.567 INFO:tasks.workunit.client.0.smithi119.stdout: 1 586732 7.10 MB/sec execute 305 sec latency 38.783 ms 2023-12-07T10:28:53.567 INFO:tasks.workunit.client.0.smithi119.stdout: 1 588520 7.10 MB/sec execute 306 sec latency 17.706 ms 2023-12-07T10:28:54.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:54 smithi119 ceph-mon[97908]: pgmap v261: 129 pgs: 129 active+clean; 385 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 187 op/s 2023-12-07T10:28:54.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:54 smithi159 ceph-mon[134726]: pgmap v261: 129 pgs: 129 active+clean; 385 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 187 op/s 2023-12-07T10:28:54.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:54 smithi153 ceph-mon[133625]: pgmap v261: 129 pgs: 129 active+clean; 385 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 187 op/s 2023-12-07T10:28:54.567 INFO:tasks.workunit.client.0.smithi119.stdout: 1 590036 7.11 MB/sec execute 307 sec latency 11.725 ms 2023-12-07T10:28:55.567 INFO:tasks.workunit.client.0.smithi119.stdout: 1 591312 7.11 MB/sec execute 308 sec latency 22.009 ms 2023-12-07T10:28:56.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:56 smithi119 ceph-mon[97908]: pgmap v262: 129 pgs: 129 active+clean; 385 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 167 op/s 2023-12-07T10:28:56.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:56 smithi159 ceph-mon[134726]: pgmap v262: 129 pgs: 129 active+clean; 385 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 167 op/s 2023-12-07T10:28:56.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:56 smithi153 ceph-mon[133625]: pgmap v262: 129 pgs: 129 active+clean; 385 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 167 op/s 2023-12-07T10:28:56.567 INFO:tasks.workunit.client.0.smithi119.stdout: 1 593374 7.13 MB/sec execute 309 sec latency 21.051 ms 2023-12-07T10:28:57.568 INFO:tasks.workunit.client.0.smithi119.stdout: 1 594386 7.11 MB/sec execute 310 sec latency 13.825 ms 2023-12-07T10:28:58.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:28:58 smithi119 ceph-mon[97908]: pgmap v263: 129 pgs: 129 active+clean; 392 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 270 op/s 2023-12-07T10:28:58.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:28:58 smithi159 ceph-mon[134726]: pgmap v263: 129 pgs: 129 active+clean; 392 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 270 op/s 2023-12-07T10:28:58.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:28:58 smithi153 ceph-mon[133625]: pgmap v263: 129 pgs: 129 active+clean; 392 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 270 op/s 2023-12-07T10:28:58.567 INFO:tasks.workunit.client.0.smithi119.stdout: 1 596373 7.13 MB/sec execute 311 sec latency 19.496 ms 2023-12-07T10:28:59.568 INFO:tasks.workunit.client.0.smithi119.stdout: 1 597521 7.12 MB/sec execute 312 sec latency 14.481 ms 2023-12-07T10:29:00.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:00 smithi119 ceph-mon[97908]: pgmap v264: 129 pgs: 129 active+clean; 392 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 185 op/s 2023-12-07T10:29:00.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:00 smithi159 ceph-mon[134726]: pgmap v264: 129 pgs: 129 active+clean; 392 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 185 op/s 2023-12-07T10:29:00.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:00 smithi153 ceph-mon[133625]: pgmap v264: 129 pgs: 129 active+clean; 392 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 185 op/s 2023-12-07T10:29:00.568 INFO:tasks.workunit.client.0.smithi119.stdout: 1 598983 7.12 MB/sec execute 313 sec latency 88.890 ms 2023-12-07T10:29:01.568 INFO:tasks.workunit.client.0.smithi119.stdout: 1 599644 7.12 MB/sec execute 314 sec latency 67.341 ms 2023-12-07T10:29:02.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:02 smithi119 ceph-mon[97908]: pgmap v265: 129 pgs: 129 active+clean; 398 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 266 op/s 2023-12-07T10:29:02.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:02 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:29:02.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:02 smithi159 ceph-mon[134726]: pgmap v265: 129 pgs: 129 active+clean; 398 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 266 op/s 2023-12-07T10:29:02.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:02 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:29:02.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:02 smithi153 ceph-mon[133625]: pgmap v265: 129 pgs: 129 active+clean; 398 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 266 op/s 2023-12-07T10:29:02.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:02 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:29:02.568 INFO:tasks.workunit.client.0.smithi119.stdout: 1 600866 7.11 MB/sec execute 315 sec latency 58.124 ms 2023-12-07T10:29:03.568 INFO:tasks.workunit.client.0.smithi119.stdout: 1 602656 7.11 MB/sec execute 316 sec latency 16.050 ms 2023-12-07T10:29:04.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:04 smithi119 ceph-mon[97908]: pgmap v266: 129 pgs: 129 active+clean; 399 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 247 op/s 2023-12-07T10:29:04.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:04 smithi159 ceph-mon[134726]: pgmap v266: 129 pgs: 129 active+clean; 399 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 247 op/s 2023-12-07T10:29:04.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:04 smithi153 ceph-mon[133625]: pgmap v266: 129 pgs: 129 active+clean; 399 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 247 op/s 2023-12-07T10:29:04.568 INFO:tasks.workunit.client.0.smithi119.stdout: 1 604047 7.13 MB/sec execute 317 sec latency 64.798 ms 2023-12-07T10:29:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.748+0000 7f90dfbff700 1 -- 172.21.15.119:0/2436959036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90d8101500 msgr2=0x7f90d8101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.748+0000 7f90dfbff700 1 --2- 172.21.15.119:0/2436959036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90d8101500 0x7f90d8101980 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f90c8009990 tx=0x7f90c8009ca0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.748+0000 7f90dfbff700 1 -- 172.21.15.119:0/2436959036 shutdown_connections 2023-12-07T10:29:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.748+0000 7f90dfbff700 1 --2- 172.21.15.119:0/2436959036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90d8101ec0 0x7f90d810e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.748+0000 7f90dfbff700 1 --2- 172.21.15.119:0/2436959036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90d81075c0 0x7f90d81079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.748+0000 7f90dfbff700 1 --2- 172.21.15.119:0/2436959036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90d8101500 0x7f90d8101980 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.748+0000 7f90dfbff700 1 -- 172.21.15.119:0/2436959036 >> 172.21.15.119:0/2436959036 conn(0x7f90d80fcfb0 msgr2=0x7f90d80ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.748+0000 7f90dfbff700 1 -- 172.21.15.119:0/2436959036 shutdown_connections 2023-12-07T10:29:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.749+0000 7f90dfbff700 1 -- 172.21.15.119:0/2436959036 wait complete. 2023-12-07T10:29:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.750+0000 7f90dfbff700 1 Processor -- start 2023-12-07T10:29:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.750+0000 7f90dfbff700 1 -- start start 2023-12-07T10:29:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.750+0000 7f90dfbff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90d8101500 0x7f90d819d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.750+0000 7f90dfbff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90d8101ec0 0x7f90d819db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.751+0000 7f90dfbff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90d81075c0 0x7f90d8197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.751+0000 7f90dfbff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d8197df0 con 0x7f90d81075c0 2023-12-07T10:29:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.751+0000 7f90dfbff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d8197f60 con 0x7f90d8101500 2023-12-07T10:29:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.751+0000 7f90dfbff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d8198120 con 0x7f90d8101ec0 2023-12-07T10:29:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.751+0000 7f90dd99b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90d8101500 0x7f90d819d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.751+0000 7f90dd99b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90d8101500 0x7f90d819d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:60112/0 (socket says 172.21.15.119:60112) 2023-12-07T10:29:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.751+0000 7f90dd99b700 1 -- 172.21.15.119:0/84279388 learned_addr learned my addr 172.21.15.119:0/84279388 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:29:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.751+0000 7f90dd19a700 1 --2- 172.21.15.119:0/84279388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90d8101ec0 0x7f90d819db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.751+0000 7f90de19c700 1 --2- 172.21.15.119:0/84279388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90d81075c0 0x7f90d8197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.751+0000 7f90dd99b700 1 -- 172.21.15.119:0/84279388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90d8101ec0 msgr2=0x7f90d819db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.751+0000 7f90dd99b700 1 --2- 172.21.15.119:0/84279388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90d8101ec0 0x7f90d819db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.751+0000 7f90dd99b700 1 -- 172.21.15.119:0/84279388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90d81075c0 msgr2=0x7f90d8197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.751+0000 7f90dd99b700 1 --2- 172.21.15.119:0/84279388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90d81075c0 0x7f90d8197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.751+0000 7f90dd99b700 1 -- 172.21.15.119:0/84279388 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c80095f0 con 0x7f90d8101500 2023-12-07T10:29:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.751+0000 7f90de19c700 1 --2- 172.21.15.119:0/84279388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90d81075c0 0x7f90d8197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:29:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.751+0000 7f90dd19a700 1 --2- 172.21.15.119:0/84279388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90d8101ec0 0x7f90d819db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:29:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.751+0000 7f90dd99b700 1 --2- 172.21.15.119:0/84279388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90d8101500 0x7f90d819d610 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f90d400e9a0 tx=0x7f90d400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.752+0000 7f90ceffd700 1 -- 172.21.15.119:0/84279388 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4009790 con 0x7f90d8101500 2023-12-07T10:29:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.752+0000 7f90ceffd700 1 -- 172.21.15.119:0/84279388 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90d400cbd0 con 0x7f90d8101500 2023-12-07T10:29:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.752+0000 7f90dfbff700 1 -- 172.21.15.119:0/84279388 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d8198400 con 0x7f90d8101500 2023-12-07T10:29:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.752+0000 7f90ceffd700 1 -- 172.21.15.119:0/84279388 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4018680 con 0x7f90d8101500 2023-12-07T10:29:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.752+0000 7f90dfbff700 1 -- 172.21.15.119:0/84279388 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d81a4600 con 0x7f90d8101500 2023-12-07T10:29:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.755+0000 7f90ceffd700 1 -- 172.21.15.119:0/84279388 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f90d400cd40 con 0x7f90d8101500 2023-12-07T10:29:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.756+0000 7f90f34b8b80 1 Processor -- start 2023-12-07T10:29:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.757+0000 7f90f34b8b80 1 -- start start 2023-12-07T10:29:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.757+0000 7f90f34b8b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be19c68a60 0x55be19c68e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.757+0000 7f90f34b8b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c69380 0x55be19c63540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.757+0000 7f90f34b8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be19c63ad0 0x55be19c63f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.757+0000 7f90f34b8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be19bd7760 con 0x55be19c63ad0 2023-12-07T10:29:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.757+0000 7f90f34b8b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be19bbe740 con 0x55be19c68a60 2023-12-07T10:29:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.757+0000 7f90f34b8b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be19bbc060 con 0x55be19c69380 2023-12-07T10:29:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.757+0000 7f90ceffd700 1 --2- 172.21.15.119:0/84279388 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f90c40a1d00 0x7f90c40a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.757+0000 7f90dd99b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c69380 0x55be19c63540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.757+0000 7f90dd99b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c69380 0x55be19c63540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:54056/0 (socket says 172.21.15.119:54056) 2023-12-07T10:29:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.757+0000 7f90dd99b700 1 -- 172.21.15.119:0/2507047469 learned_addr learned my addr 172.21.15.119:0/2507047469 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:29:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.757+0000 7f90dd19a700 1 --2- 172.21.15.119:0/2507047469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be19c63ad0 0x55be19c63f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.757+0000 7f90ceffd700 1 -- 172.21.15.119:0/84279388 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f90d4014070 con 0x7f90d8101500 2023-12-07T10:29:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.757+0000 7f90de19c700 1 --2- 172.21.15.119:0/2507047469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be19c68a60 0x55be19c68e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.758+0000 7f90de19c700 1 --2- 172.21.15.119:0/84279388 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f90c40a1d00 0x7f90c40a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.758+0000 7f90dd99b700 1 -- 172.21.15.119:0/2507047469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be19c68a60 msgr2=0x55be19c68e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.758+0000 7f90dd99b700 1 --2- 172.21.15.119:0/2507047469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be19c68a60 0x55be19c68e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.758+0000 7f90dd99b700 1 -- 172.21.15.119:0/2507047469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be19c63ad0 msgr2=0x55be19c63f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.758+0000 7f90dd99b700 1 --2- 172.21.15.119:0/2507047469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be19c63ad0 0x55be19c63f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.758+0000 7f90dd99b700 1 -- 172.21.15.119:0/2507047469 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be19a345d0 con 0x55be19c69380 2023-12-07T10:29:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.758+0000 7f90dd99b700 1 --2- 172.21.15.119:0/2507047469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c69380 0x55be19c63540 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f90d40973c0 tx=0x7f90d406c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e487e4599941c04 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.758+0000 7f90c3fff700 1 -- 172.21.15.119:0/2507047469 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d409b050 con 0x55be19c69380 2023-12-07T10:29:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.758+0000 7f90de19c700 1 --2- 172.21.15.119:0/84279388 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f90c40a1d00 0x7f90c40a41c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f90d0012880 tx=0x7f90d000b040 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.758+0000 7f90c3fff700 1 -- 172.21.15.119:0/2507047469 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90d4065f80 con 0x55be19c69380 2023-12-07T10:29:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.758+0000 7f90c3fff700 1 -- 172.21.15.119:0/2507047469 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d40ba310 con 0x55be19c69380 2023-12-07T10:29:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.759+0000 7f90f34b8b80 1 -- 172.21.15.119:0/2507047469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c69380 msgr2=0x55be19c63540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.759+0000 7f90f34b8b80 1 --2- 172.21.15.119:0/2507047469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c69380 0x55be19c63540 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f90d40973c0 tx=0x7f90d406c5d0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.759+0000 7f90f34b8b80 1 -- 172.21.15.119:0/2507047469 shutdown_connections 2023-12-07T10:29:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.759+0000 7f90f34b8b80 1 --2- 172.21.15.119:0/2507047469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be19c63ad0 0x55be19c63f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.759+0000 7f90f34b8b80 1 --2- 172.21.15.119:0/2507047469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be19c68a60 0x55be19c68e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.759+0000 7f90f34b8b80 1 --2- 172.21.15.119:0/2507047469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c69380 0x55be19c63540 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.759+0000 7f90f34b8b80 1 -- 172.21.15.119:0/2507047469 >> 172.21.15.119:0/2507047469 conn(0x55be19b92300 msgr2=0x55be19c5db60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.759+0000 7f90f34b8b80 1 -- 172.21.15.119:0/2507047469 shutdown_connections 2023-12-07T10:29:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.759+0000 7f90f34b8b80 1 -- 172.21.15.119:0/2507047469 wait complete. 2023-12-07T10:29:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.760+0000 7f90f34b8b80 1 -- start start 2023-12-07T10:29:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.760+0000 7f90f34b8b80 1 Processor -- start 2023-12-07T10:29:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.761+0000 7f90f34b8b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c63ad0 0x55be19c6c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.761+0000 7f90f34b8b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be19c68a60 0x55be19c6cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.761+0000 7f90f34b8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be19c69380 0x55be19c6d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.761+0000 7f90f34b8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be19bb7950 con 0x55be19c69380 2023-12-07T10:29:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.761+0000 7f90f34b8b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be19b71500 con 0x55be19c68a60 2023-12-07T10:29:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.761+0000 7f90f34b8b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be19b711a0 con 0x55be19c63ad0 2023-12-07T10:29:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.761+0000 7f90f34b8b80 10 client.0 _ll_register_callbacks cb 0x55be19b588e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:29:04.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.761+0000 7f90dd99b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c63ad0 0x55be19c6c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:04.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.761+0000 7f90dd99b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c63ad0 0x55be19c6c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:54060/0 (socket says 172.21.15.119:54060) 2023-12-07T10:29:04.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.761+0000 7f90dd99b700 1 -- 172.21.15.119:0/130517478 learned_addr learned my addr 172.21.15.119:0/130517478 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:29:04.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.761+0000 7f90dd19a700 1 --2- 172.21.15.119:0/130517478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be19c68a60 0x55be19c6cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:04.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.762+0000 7f90de19c700 1 --2- 172.21.15.119:0/130517478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be19c69380 0x55be19c6d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:04.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.762+0000 7f90dd99b700 1 -- 172.21.15.119:0/130517478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be19c68a60 msgr2=0x55be19c6cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:04.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.762+0000 7f90dd99b700 1 --2- 172.21.15.119:0/130517478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be19c68a60 0x55be19c6cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.762+0000 7f90dd99b700 1 -- 172.21.15.119:0/130517478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be19c69380 msgr2=0x55be19c6d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:04.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.762+0000 7f90dd99b700 1 --2- 172.21.15.119:0/130517478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be19c69380 0x55be19c6d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.762+0000 7f90dd99b700 1 -- 172.21.15.119:0/130517478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d81a4600 con 0x55be19c63ad0 2023-12-07T10:29:04.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.762+0000 7f90dd99b700 1 --2- 172.21.15.119:0/130517478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c63ad0 0x55be19c6c680 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f90d409a650 tx=0x7f90d409b050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edc2544fab54d37f server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:04.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.762+0000 7f90c17fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:29:04.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.762+0000 7f90c17fa700 1 -- 172.21.15.119:0/130517478 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d40b3220 con 0x55be19c63ad0 2023-12-07T10:29:04.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.762+0000 7f90c17fa700 1 -- 172.21.15.119:0/130517478 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f90d409a950 con 0x55be19c63ad0 2023-12-07T10:29:04.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.762+0000 7f90f34b8b80 1 -- 172.21.15.119:0/130517478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be19c566f0 con 0x55be19c63ad0 2023-12-07T10:29:04.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.763+0000 7f90c17fa700 1 -- 172.21.15.119:0/130517478 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90d40ba2c0 con 0x55be19c63ad0 2023-12-07T10:29:04.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.763+0000 7f90f34b8b80 10 client.27487 fetch_fsmap learned FSMap version 56 2023-12-07T10:29:04.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.763+0000 7f90f34b8b80 1 -- 172.21.15.119:0/130517478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55be199e0d80 con 0x55be19c63ad0 2023-12-07T10:29:04.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.763+0000 7f90c17fa700 1 -- 172.21.15.119:0/130517478 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55be19c566f0 con 0x55be19c63ad0 2023-12-07T10:29:04.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.763+0000 7f90c17fa700 1 -- 172.21.15.119:0/130517478 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 56) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f90d4068020 con 0x55be19c63ad0 2023-12-07T10:29:04.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.763+0000 7f90f34b8b80 10 client.27487 fetch_fsmap finished waiting for FSMap version 56 2023-12-07T10:29:04.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.763+0000 7f90f34b8b80 10 client.27487 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:29:04.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.763+0000 7f90f34b8b80 20 client.27487 populate_metadata read hostname 'smithi119' 2023-12-07T10:29:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.763+0000 7f90f34b8b80 1 --2- 172.21.15.119:0/130517478 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55be19c80ed0 0x55be19c832c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.763+0000 7f90f34b8b80 4 client.27487 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:29:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.763+0000 7f90f34b8b80 1 -- 172.21.15.119:0/130517478 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be19c32740 con 0x55be19c80ed0 2023-12-07T10:29:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.763+0000 7f90dd19a700 1 --2- 172.21.15.119:0/130517478 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55be19c80ed0 0x55be19c832c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.764+0000 7f90c17fa700 10 client.27487 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:29:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.764+0000 7f90dd19a700 1 --2- 172.21.15.119:0/130517478 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55be19c80ed0 0x55be19c832c0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c18e2ea9b51a078 server_cookie=302224866290170d in_seq=0 out_seq=0 2023-12-07T10:29:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.765+0000 7f90c17fa700 1 -- 172.21.15.119:0/130517478 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55be19c32740 con 0x55be19c80ed0 2023-12-07T10:29:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.765+0000 7f90c17fa700 10 client.27487 handle_command_reply: tid=0 2023-12-07T10:29:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.765+0000 7f90f34b8b80 1 client.27487 shutdown 2023-12-07T10:29:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.765+0000 7f90f34b8b80 1 -- 172.21.15.119:0/130517478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c63ad0 msgr2=0x55be19c6c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.765+0000 7f90f34b8b80 1 --2- 172.21.15.119:0/130517478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c63ad0 0x55be19c6c680 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f90d409a650 tx=0x7f90d409b050 comp rx=0 tx=0).stop 2023-12-07T10:29:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.765+0000 7f90f34b8b80 1 -- 172.21.15.119:0/130517478 shutdown_connections 2023-12-07T10:29:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.765+0000 7f90f34b8b80 1 --2- 172.21.15.119:0/130517478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be19c69380 0x55be19c6d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.765+0000 7f90f34b8b80 1 --2- 172.21.15.119:0/130517478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c63ad0 0x55be19c6c680 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.765+0000 7f90f34b8b80 1 --2- 172.21.15.119:0/130517478 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55be19c80ed0 0x55be19c832c0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.765+0000 7f90f34b8b80 1 --2- 172.21.15.119:0/130517478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be19c68a60 0x55be19c6cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.765+0000 7f90f34b8b80 1 -- 172.21.15.119:0/130517478 >> 172.21.15.119:0/130517478 conn(0x55be19b92300 msgr2=0x55be19c58980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.765+0000 7f90c17fa700 0 client.27487 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:29:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.765+0000 7f90f34b8b80 1 -- 172.21.15.119:0/130517478 shutdown_connections 2023-12-07T10:29:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.765+0000 7f90f34b8b80 1 -- 172.21.15.119:0/130517478 wait complete. 2023-12-07T10:29:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.766+0000 7f90f34b8b80 20 client.27487 trim_cache size 0 max 16384 2023-12-07T10:29:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.783+0000 7f90f34b8b80 1 Processor -- start 2023-12-07T10:29:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.783+0000 7f90f34b8b80 1 -- start start 2023-12-07T10:29:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.784+0000 7f90f34b8b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be19c68a60 0x55be19c63540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.784+0000 7f90f34b8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be19c69380 0x55be19c6cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.784+0000 7f90f34b8b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c67010 0x55be19c78200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.784+0000 7f90f34b8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be19a368c0 con 0x55be19c69380 2023-12-07T10:29:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.784+0000 7f90f34b8b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be19c69090 con 0x55be19c68a60 2023-12-07T10:29:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.784+0000 7f90f34b8b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be19c566c0 con 0x55be19c67010 2023-12-07T10:29:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.784+0000 7f90dd99b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be19c68a60 0x55be19c63540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.784+0000 7f90dd99b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be19c68a60 0x55be19c63540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:60138/0 (socket says 172.21.15.119:60138) 2023-12-07T10:29:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.784+0000 7f90dd99b700 1 -- 172.21.15.119:0/988314807 learned_addr learned my addr 172.21.15.119:0/988314807 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:29:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.784+0000 7f90dd19a700 1 --2- 172.21.15.119:0/988314807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be19c69380 0x55be19c6cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.784+0000 7f90de19c700 1 --2- 172.21.15.119:0/988314807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c67010 0x55be19c78200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.784+0000 7f90dd99b700 1 -- 172.21.15.119:0/988314807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c67010 msgr2=0x55be19c78200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.784+0000 7f90dd99b700 1 --2- 172.21.15.119:0/988314807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c67010 0x55be19c78200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.784+0000 7f90dd99b700 1 -- 172.21.15.119:0/988314807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be19c69380 msgr2=0x55be19c6cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.784+0000 7f90dd99b700 1 --2- 172.21.15.119:0/988314807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be19c69380 0x55be19c6cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.784+0000 7f90dd99b700 1 -- 172.21.15.119:0/988314807 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be19c66190 con 0x55be19c68a60 2023-12-07T10:29:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.784+0000 7f90de19c700 1 --2- 172.21.15.119:0/988314807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c67010 0x55be19c78200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:29:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.784+0000 7f90dd19a700 1 --2- 172.21.15.119:0/988314807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be19c69380 0x55be19c6cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:29:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.785+0000 7f90dd99b700 1 --2- 172.21.15.119:0/988314807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be19c68a60 0x55be19c63540 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f90d40973c0 tx=0x7f90d40092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5af8253d46f6fd6 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.785+0000 7f90ccbff700 1 -- 172.21.15.119:0/988314807 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d40b3220 con 0x55be19c68a60 2023-12-07T10:29:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.785+0000 7f90ccbff700 1 -- 172.21.15.119:0/988314807 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90d40a2920 con 0x55be19c68a60 2023-12-07T10:29:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.785+0000 7f90ccbff700 1 -- 172.21.15.119:0/988314807 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d409abb0 con 0x55be19c68a60 2023-12-07T10:29:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.785+0000 7f90f34b8b80 1 -- 172.21.15.119:0/988314807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be19c68a60 msgr2=0x55be19c63540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.785+0000 7f90f34b8b80 1 --2- 172.21.15.119:0/988314807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be19c68a60 0x55be19c63540 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f90d40973c0 tx=0x7f90d40092d0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.786+0000 7f90f34b8b80 1 -- 172.21.15.119:0/988314807 shutdown_connections 2023-12-07T10:29:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.786+0000 7f90f34b8b80 1 --2- 172.21.15.119:0/988314807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c67010 0x55be19c78200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.786+0000 7f90f34b8b80 1 --2- 172.21.15.119:0/988314807 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be19c68a60 0x55be19c63540 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.786+0000 7f90f34b8b80 1 --2- 172.21.15.119:0/988314807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be19c69380 0x55be19c6cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.786+0000 7f90f34b8b80 1 -- 172.21.15.119:0/988314807 >> 172.21.15.119:0/988314807 conn(0x55be19b92300 msgr2=0x55be19c66900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.786+0000 7f90f34b8b80 1 -- 172.21.15.119:0/988314807 shutdown_connections 2023-12-07T10:29:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.786+0000 7f90f34b8b80 1 -- 172.21.15.119:0/988314807 wait complete. 2023-12-07T10:29:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.786+0000 7f90f34b8b80 1 -- start start 2023-12-07T10:29:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.787+0000 7f90f34b8b80 1 Processor -- start 2023-12-07T10:29:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.787+0000 7f90f34b8b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c68a60 0x55be19c78650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.787+0000 7f90f34b8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be19c69380 0x55be19c759e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.787+0000 7f90f34b8b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be19c75f70 0x55be19c763d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.787+0000 7f90f34b8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be19cd6480 con 0x55be19c69380 2023-12-07T10:29:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.787+0000 7f90f34b8b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be19cd65f0 con 0x55be19c75f70 2023-12-07T10:29:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.787+0000 7f90f34b8b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be19cd67b0 con 0x55be19c68a60 2023-12-07T10:29:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.787+0000 7f90f34b8b80 10 client.0 _ll_register_callbacks cb 0x55be19b588e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:29:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.788+0000 7f90dd99b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c68a60 0x55be19c78650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.788+0000 7f90dd99b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c68a60 0x55be19c78650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:54080/0 (socket says 172.21.15.119:54080) 2023-12-07T10:29:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.788+0000 7f90dd99b700 1 -- 172.21.15.119:0/3424984244 learned_addr learned my addr 172.21.15.119:0/3424984244 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:29:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.788+0000 7f90dd19a700 1 --2- 172.21.15.119:0/3424984244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be19c69380 0x55be19c759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.788+0000 7f90de19c700 1 --2- 172.21.15.119:0/3424984244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be19c75f70 0x55be19c763d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.788+0000 7f90dd99b700 1 -- 172.21.15.119:0/3424984244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be19c75f70 msgr2=0x55be19c763d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.788+0000 7f90dd99b700 1 --2- 172.21.15.119:0/3424984244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be19c75f70 0x55be19c763d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.788+0000 7f90dd99b700 1 -- 172.21.15.119:0/3424984244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be19c69380 msgr2=0x55be19c759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.788+0000 7f90dd99b700 1 --2- 172.21.15.119:0/3424984244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be19c69380 0x55be19c759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.788+0000 7f90dd99b700 1 -- 172.21.15.119:0/3424984244 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be199e0d80 con 0x55be19c68a60 2023-12-07T10:29:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.788+0000 7f90dd19a700 1 --2- 172.21.15.119:0/3424984244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be19c69380 0x55be19c759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:29:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.788+0000 7f90dd99b700 1 --2- 172.21.15.119:0/3424984244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c68a60 0x55be19c78650 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f90d40cb900 tx=0x7f90d400efa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbf4e400a57a3fdf server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.788+0000 7f90c17fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:29:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.789+0000 7f90c17fa700 1 -- 172.21.15.119:0/3424984244 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4096940 con 0x55be19c68a60 2023-12-07T10:29:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.789+0000 7f90c17fa700 1 -- 172.21.15.119:0/3424984244 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f90d409a950 con 0x55be19c68a60 2023-12-07T10:29:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.789+0000 7f90f34b8b80 1 -- 172.21.15.119:0/3424984244 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f90d4065f80 con 0x55be19c68a60 2023-12-07T10:29:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.789+0000 7f90c17fa700 1 -- 172.21.15.119:0/3424984244 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90d40b3220 con 0x55be19c68a60 2023-12-07T10:29:04.792 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.789+0000 7f90f34b8b80 10 client.27493 fetch_fsmap learned FSMap version 56 2023-12-07T10:29:04.792 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.789+0000 7f90f34b8b80 1 -- 172.21.15.119:0/3424984244 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55be199c9c00 con 0x55be19c68a60 2023-12-07T10:29:04.792 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.789+0000 7f90c17fa700 1 -- 172.21.15.119:0/3424984244 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90d4065f80 con 0x55be19c68a60 2023-12-07T10:29:04.792 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.789+0000 7f90c17fa700 1 -- 172.21.15.119:0/3424984244 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 56) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f90d4068020 con 0x55be19c68a60 2023-12-07T10:29:04.792 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.789+0000 7f90f34b8b80 10 client.27493 fetch_fsmap finished waiting for FSMap version 56 2023-12-07T10:29:04.792 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.789+0000 7f90f34b8b80 10 client.27493 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:29:04.792 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.789+0000 7f90f34b8b80 20 client.27493 populate_metadata read hostname 'smithi119' 2023-12-07T10:29:04.792 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.789+0000 7f90f34b8b80 1 --2- 172.21.15.119:0/3424984244 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55be19cd7ae0 0x55be19cd9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:04.792 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.789+0000 7f90f34b8b80 4 client.27493 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:29:04.792 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.789+0000 7f90f34b8b80 1 -- 172.21.15.119:0/3424984244 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55be19b7df70 con 0x55be19cd7ae0 2023-12-07T10:29:04.792 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.790+0000 7f90dd19a700 1 --2- 172.21.15.119:0/3424984244 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55be19cd7ae0 0x55be19cd9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:04.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.790+0000 7f90dd19a700 1 --2- 172.21.15.119:0/3424984244 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55be19cd7ae0 0x55be19cd9ed0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f1b2aa9c4b0aae2 server_cookie=4eb72e0e7d861308 in_seq=0 out_seq=0 2023-12-07T10:29:04.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.790+0000 7f90c17fa700 10 client.27493 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:29:04.863 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.862+0000 7f90c17fa700 1 -- 172.21.15.119:0/3424984244 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55be19b7df70 con 0x55be19cd7ae0 2023-12-07T10:29:04.863 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.862+0000 7f90c17fa700 10 client.27493 handle_command_reply: tid=0 2023-12-07T10:29:04.863 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.862+0000 7f90f34b8b80 1 client.27493 shutdown 2023-12-07T10:29:04.863 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.862+0000 7f90f34b8b80 1 -- 172.21.15.119:0/3424984244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c68a60 msgr2=0x55be19c78650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:04.863 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.862+0000 7f90f34b8b80 1 --2- 172.21.15.119:0/3424984244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c68a60 0x55be19c78650 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f90d40cb900 tx=0x7f90d400efa0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.864 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.862+0000 7f90f34b8b80 1 -- 172.21.15.119:0/3424984244 shutdown_connections 2023-12-07T10:29:04.864 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.862+0000 7f90f34b8b80 1 --2- 172.21.15.119:0/3424984244 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55be19cd7ae0 0x55be19cd9ed0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.864 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.863+0000 7f90f34b8b80 1 --2- 172.21.15.119:0/3424984244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be19c75f70 0x55be19c763d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.864 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.863+0000 7f90f34b8b80 1 --2- 172.21.15.119:0/3424984244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be19c68a60 0x55be19c78650 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.864 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.863+0000 7f90f34b8b80 1 --2- 172.21.15.119:0/3424984244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55be19c69380 0x55be19c759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.864 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.863+0000 7f90f34b8b80 1 -- 172.21.15.119:0/3424984244 >> 172.21.15.119:0/3424984244 conn(0x55be19b92300 msgr2=0x55be19c6cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:04.864 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.863+0000 7f90c17fa700 0 client.27493 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:29:04.864 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.863+0000 7f90f34b8b80 1 -- 172.21.15.119:0/3424984244 shutdown_connections 2023-12-07T10:29:04.864 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.863+0000 7f90f34b8b80 1 -- 172.21.15.119:0/3424984244 wait complete. 2023-12-07T10:29:04.864 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.863+0000 7f90f34b8b80 20 client.27493 trim_cache size 0 max 16384 2023-12-07T10:29:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:29:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:29:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:29:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:29:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.864+0000 7f90dfbff700 1 -- 172.21.15.119:0/84279388 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f90c40a1d00 msgr2=0x7f90c40a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.864+0000 7f90dfbff700 1 --2- 172.21.15.119:0/84279388 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f90c40a1d00 0x7f90c40a41c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f90d0012880 tx=0x7f90d000b040 comp rx=0 tx=0).stop 2023-12-07T10:29:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.864+0000 7f90dfbff700 1 -- 172.21.15.119:0/84279388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90d8101500 msgr2=0x7f90d819d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.864+0000 7f90dfbff700 1 --2- 172.21.15.119:0/84279388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90d8101500 0x7f90d819d610 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f90d400e9a0 tx=0x7f90d400ecb0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.864+0000 7f90dfbff700 1 -- 172.21.15.119:0/84279388 shutdown_connections 2023-12-07T10:29:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.864+0000 7f90dfbff700 1 --2- 172.21.15.119:0/84279388 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f90c40a1d00 0x7f90c40a41c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.864+0000 7f90dfbff700 1 --2- 172.21.15.119:0/84279388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90d81075c0 0x7f90d8197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.864+0000 7f90dfbff700 1 --2- 172.21.15.119:0/84279388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90d8101500 0x7f90d819d610 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.864+0000 7f90dfbff700 1 --2- 172.21.15.119:0/84279388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90d8101ec0 0x7f90d819db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.864+0000 7f90dfbff700 1 -- 172.21.15.119:0/84279388 >> 172.21.15.119:0/84279388 conn(0x7f90d80fcfb0 msgr2=0x7f90d80fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.864+0000 7f90dfbff700 1 -- 172.21.15.119:0/84279388 shutdown_connections 2023-12-07T10:29:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:04.864+0000 7f90dfbff700 1 -- 172.21.15.119:0/84279388 wait complete. 2023-12-07T10:29:05.480 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:05 smithi119 ceph-mon[97908]: osdmap e99: 12 total, 12 up, 12 in 2023-12-07T10:29:05.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:05 smithi159 ceph-mon[134726]: osdmap e99: 12 total, 12 up, 12 in 2023-12-07T10:29:05.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:05 smithi153 ceph-mon[133625]: osdmap e99: 12 total, 12 up, 12 in 2023-12-07T10:29:05.568 INFO:tasks.workunit.client.0.smithi119.stdout: 1 604480 7.11 MB/sec execute 318 sec latency 254.522 ms 2023-12-07T10:29:06.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:06 smithi119 ceph-mon[97908]: pgmap v268: 129 pgs: 129 active+clean; 399 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 252 op/s 2023-12-07T10:29:06.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:06 smithi159 ceph-mon[134726]: pgmap v268: 129 pgs: 129 active+clean; 399 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 252 op/s 2023-12-07T10:29:06.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:06 smithi153 ceph-mon[133625]: pgmap v268: 129 pgs: 129 active+clean; 399 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 252 op/s 2023-12-07T10:29:06.568 INFO:tasks.workunit.client.0.smithi119.stdout: 1 605507 7.10 MB/sec execute 319 sec latency 77.879 ms 2023-12-07T10:29:07.568 INFO:tasks.workunit.client.0.smithi119.stdout: 1 607464 7.12 MB/sec execute 320 sec latency 25.823 ms 2023-12-07T10:29:08.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:08 smithi119 ceph-mon[97908]: pgmap v269: 129 pgs: 129 active+clean; 425 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 321 op/s 2023-12-07T10:29:08.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:08 smithi159 ceph-mon[134726]: pgmap v269: 129 pgs: 129 active+clean; 425 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 321 op/s 2023-12-07T10:29:08.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:08 smithi153 ceph-mon[133625]: pgmap v269: 129 pgs: 129 active+clean; 425 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 321 op/s 2023-12-07T10:29:08.568 INFO:tasks.workunit.client.0.smithi119.stdout: 1 608528 7.11 MB/sec execute 321 sec latency 12.862 ms 2023-12-07T10:29:09.568 INFO:tasks.workunit.client.0.smithi119.stdout: 1 610555 7.12 MB/sec execute 322 sec latency 15.450 ms 2023-12-07T10:29:10.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:10 smithi119 ceph-mon[97908]: pgmap v270: 129 pgs: 129 active+clean; 425 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 321 op/s 2023-12-07T10:29:10.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:10 smithi159 ceph-mon[134726]: pgmap v270: 129 pgs: 129 active+clean; 425 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 321 op/s 2023-12-07T10:29:10.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:10 smithi153 ceph-mon[133625]: pgmap v270: 129 pgs: 129 active+clean; 425 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 321 op/s 2023-12-07T10:29:10.568 INFO:tasks.workunit.client.0.smithi119.stdout: 1 611561 7.12 MB/sec execute 323 sec latency 44.394 ms 2023-12-07T10:29:11.568 INFO:tasks.workunit.client.0.smithi119.stdout: 1 613195 7.11 MB/sec execute 324 sec latency 30.048 ms 2023-12-07T10:29:12.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:11 smithi119 ceph-mon[97908]: osdmap e100: 12 total, 12 up, 12 in 2023-12-07T10:29:12.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:11 smithi119 ceph-mon[97908]: pgmap v272: 129 pgs: 129 active+clean; 413 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.2 MiB/s wr, 369 op/s 2023-12-07T10:29:12.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:11 smithi159 ceph-mon[134726]: osdmap e100: 12 total, 12 up, 12 in 2023-12-07T10:29:12.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:11 smithi159 ceph-mon[134726]: pgmap v272: 129 pgs: 129 active+clean; 413 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.2 MiB/s wr, 369 op/s 2023-12-07T10:29:12.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:11 smithi153 ceph-mon[133625]: osdmap e100: 12 total, 12 up, 12 in 2023-12-07T10:29:12.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:11 smithi153 ceph-mon[133625]: pgmap v272: 129 pgs: 129 active+clean; 413 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.2 MiB/s wr, 369 op/s 2023-12-07T10:29:12.568 INFO:tasks.workunit.client.0.smithi119.stdout: 1 614738 7.13 MB/sec execute 325 sec latency 16.721 ms 2023-12-07T10:29:13.568 INFO:tasks.workunit.client.0.smithi119.stdout: 1 615967 7.12 MB/sec execute 326 sec latency 17.498 ms 2023-12-07T10:29:14.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:14 smithi119 ceph-mon[97908]: pgmap v273: 129 pgs: 129 active+clean; 416 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.4 MiB/s wr, 392 op/s 2023-12-07T10:29:14.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:14 smithi159 ceph-mon[134726]: pgmap v273: 129 pgs: 129 active+clean; 416 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.4 MiB/s wr, 392 op/s 2023-12-07T10:29:14.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:14 smithi153 ceph-mon[133625]: pgmap v273: 129 pgs: 129 active+clean; 416 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.4 MiB/s wr, 392 op/s 2023-12-07T10:29:14.568 INFO:tasks.workunit.client.0.smithi119.stdout: 1 618070 7.14 MB/sec execute 327 sec latency 16.014 ms 2023-12-07T10:29:15.568 INFO:tasks.workunit.client.0.smithi119.stdout: 1 619055 7.13 MB/sec execute 328 sec latency 12.760 ms 2023-12-07T10:29:16.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:16 smithi119 ceph-mon[97908]: pgmap v274: 129 pgs: 129 active+clean; 416 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.7 MiB/s wr, 350 op/s 2023-12-07T10:29:16.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:16 smithi159 ceph-mon[134726]: pgmap v274: 129 pgs: 129 active+clean; 416 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.7 MiB/s wr, 350 op/s 2023-12-07T10:29:16.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:16 smithi153 ceph-mon[133625]: pgmap v274: 129 pgs: 129 active+clean; 416 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.7 MiB/s wr, 350 op/s 2023-12-07T10:29:16.568 INFO:tasks.workunit.client.0.smithi119.stdout: 1 620976 7.14 MB/sec execute 329 sec latency 16.584 ms 2023-12-07T10:29:17.568 INFO:tasks.workunit.client.0.smithi119.stdout: 1 622121 7.14 MB/sec execute 330 sec latency 19.632 ms 2023-12-07T10:29:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.034+0000 7f5aec9ff700 1 -- 172.21.15.119:0/723562898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ae80739e0 msgr2=0x7f5ae8073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.034+0000 7f5aec9ff700 1 --2- 172.21.15.119:0/723562898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ae80739e0 0x7f5ae8073dc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5ad80098f0 tx=0x7f5ad8009c00 comp rx=0 tx=0).stop 2023-12-07T10:29:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.034+0000 7f5aec9ff700 1 -- 172.21.15.119:0/723562898 shutdown_connections 2023-12-07T10:29:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.034+0000 7f5aec9ff700 1 --2- 172.21.15.119:0/723562898 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ae810dda0 0x7f5ae81101d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.034+0000 7f5aec9ff700 1 --2- 172.21.15.119:0/723562898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ae80739e0 0x7f5ae8073dc0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.034+0000 7f5aec9ff700 1 --2- 172.21.15.119:0/723562898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ae8074300 0x7f5ae810d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.034+0000 7f5aec9ff700 1 -- 172.21.15.119:0/723562898 >> 172.21.15.119:0/723562898 conn(0x7f5ae80fcfd0 msgr2=0x7f5ae80ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.034+0000 7f5aec9ff700 1 -- 172.21.15.119:0/723562898 shutdown_connections 2023-12-07T10:29:18.035 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.034+0000 7f5aec9ff700 1 -- 172.21.15.119:0/723562898 wait complete. 2023-12-07T10:29:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.035+0000 7f5aec9ff700 1 Processor -- start 2023-12-07T10:29:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.035+0000 7f5aec9ff700 1 -- start start 2023-12-07T10:29:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.036+0000 7f5aec9ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ae80739e0 0x7f5ae819f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.036+0000 7f5aec9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ae8074300 0x7f5ae819fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.036+0000 7f5aec9ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ae810dda0 0x7f5ae8199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.036+0000 7f5aec9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae819a110 con 0x7f5ae8074300 2023-12-07T10:29:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.036+0000 7f5aec9ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae819a280 con 0x7f5ae810dda0 2023-12-07T10:29:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.036+0000 7f5aec9ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae819a440 con 0x7f5ae80739e0 2023-12-07T10:29:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.036+0000 7f5ae659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ae80739e0 0x7f5ae819f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.036+0000 7f5ae5d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ae8074300 0x7f5ae819fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.036+0000 7f5ae6d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ae810dda0 0x7f5ae8199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.036+0000 7f5ae659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ae80739e0 0x7f5ae819f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:45696/0 (socket says 172.21.15.119:45696) 2023-12-07T10:29:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.036+0000 7f5ae659c700 1 -- 172.21.15.119:0/200118544 learned_addr learned my addr 172.21.15.119:0/200118544 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:29:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.037+0000 7f5ae659c700 1 -- 172.21.15.119:0/200118544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ae810dda0 msgr2=0x7f5ae8199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.037+0000 7f5ae659c700 1 --2- 172.21.15.119:0/200118544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ae810dda0 0x7f5ae8199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.037+0000 7f5ae659c700 1 -- 172.21.15.119:0/200118544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ae8074300 msgr2=0x7f5ae819fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.037+0000 7f5ae659c700 1 --2- 172.21.15.119:0/200118544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ae8074300 0x7f5ae819fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.037+0000 7f5ae659c700 1 -- 172.21.15.119:0/200118544 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad8009520 con 0x7f5ae80739e0 2023-12-07T10:29:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.037+0000 7f5ae659c700 1 --2- 172.21.15.119:0/200118544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ae80739e0 0x7f5ae819f980 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8005950 tx=0x7f5ad800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.037+0000 7f5ad77fe700 1 -- 172.21.15.119:0/200118544 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad801c070 con 0x7f5ae80739e0 2023-12-07T10:29:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.038+0000 7f5aec9ff700 1 -- 172.21.15.119:0/200118544 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae819a6c0 con 0x7f5ae80739e0 2023-12-07T10:29:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.038+0000 7f5ad77fe700 1 -- 172.21.15.119:0/200118544 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ad800bb20 con 0x7f5ae80739e0 2023-12-07T10:29:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.038+0000 7f5ad77fe700 1 -- 172.21.15.119:0/200118544 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad80038e0 con 0x7f5ae80739e0 2023-12-07T10:29:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.038+0000 7f5aec9ff700 1 -- 172.21.15.119:0/200118544 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae81a6970 con 0x7f5ae80739e0 2023-12-07T10:29:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.041+0000 7f5ad77fe700 1 -- 172.21.15.119:0/200118544 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f5ad8020030 con 0x7f5ae80739e0 2023-12-07T10:29:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.042+0000 7f5ad77fe700 1 --2- 172.21.15.119:0/200118544 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5ad00a1c80 0x7f5ad00a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.043+0000 7f5b002d7b80 1 Processor -- start 2023-12-07T10:29:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.043+0000 7f5ad77fe700 1 -- 172.21.15.119:0/200118544 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f5ad80ceb40 con 0x7f5ae80739e0 2023-12-07T10:29:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.043+0000 7f5b002d7b80 1 -- start start 2023-12-07T10:29:18.046 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.043+0000 7f5ae5d9b700 1 --2- 172.21.15.119:0/200118544 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5ad00a1c80 0x7f5ad00a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.043+0000 7f5b002d7b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564caed3ea60 0x564caed3ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.043+0000 7f5b002d7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564caed3f380 0x564caed39540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.043+0000 7f5b002d7b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564caed39ad0 0x564caed39f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.043+0000 7f5b002d7b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564caecad760 con 0x564caed39ad0 2023-12-07T10:29:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.043+0000 7f5b002d7b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x564caec94740 con 0x564caed3ea60 2023-12-07T10:29:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.043+0000 7f5b002d7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564caec92060 con 0x564caed3f380 2023-12-07T10:29:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.044+0000 7f5ae5d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564caed3f380 0x564caed39540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.044+0000 7f5ae659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564caed3ea60 0x564caed3ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.044+0000 7f5ae6d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564caed39ad0 0x564caed39f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.044+0000 7f5ae659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564caed3ea60 0x564caed3ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:55600/0 (socket says 172.21.15.119:55600) 2023-12-07T10:29:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.044+0000 7f5ae659c700 1 -- 172.21.15.119:0/3919252408 learned_addr learned my addr 172.21.15.119:0/3919252408 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:29:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.044+0000 7f5ae5d9b700 1 --2- 172.21.15.119:0/200118544 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5ad00a1c80 0x7f5ad00a4140 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5ae819ac70 tx=0x7f5adc009310 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.044+0000 7f5ae659c700 1 -- 172.21.15.119:0/3919252408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564caed3f380 msgr2=0x564caed39540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.044+0000 7f5ae659c700 1 --2- 172.21.15.119:0/3919252408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564caed3f380 0x564caed39540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.044+0000 7f5ae659c700 1 -- 172.21.15.119:0/3919252408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564caed39ad0 msgr2=0x564caed39f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.044+0000 7f5ae659c700 1 --2- 172.21.15.119:0/3919252408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564caed39ad0 0x564caed39f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.044+0000 7f5ae659c700 1 -- 172.21.15.119:0/3919252408 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564caeb0a5d0 con 0x564caed3ea60 2023-12-07T10:29:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.044+0000 7f5ae659c700 1 --2- 172.21.15.119:0/3919252408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564caed3ea60 0x564caed3ee40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8098010 tx=0x7f5ad80054e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afc233802e802479 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.045+0000 7f5ad4ff9700 1 -- 172.21.15.119:0/3919252408 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad809b5a0 con 0x564caed3ea60 2023-12-07T10:29:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.045+0000 7f5ad4ff9700 1 -- 172.21.15.119:0/3919252408 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ad809f6f0 con 0x564caed3ea60 2023-12-07T10:29:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.045+0000 7f5ad4ff9700 1 -- 172.21.15.119:0/3919252408 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8005340 con 0x564caed3ea60 2023-12-07T10:29:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.045+0000 7f5b002d7b80 1 -- 172.21.15.119:0/3919252408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564caed3ea60 msgr2=0x564caed3ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.045+0000 7f5b002d7b80 1 --2- 172.21.15.119:0/3919252408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564caed3ea60 0x564caed3ee40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8098010 tx=0x7f5ad80054e0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.046+0000 7f5b002d7b80 1 -- 172.21.15.119:0/3919252408 shutdown_connections 2023-12-07T10:29:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.046+0000 7f5b002d7b80 1 --2- 172.21.15.119:0/3919252408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564caed39ad0 0x564caed39f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.046+0000 7f5b002d7b80 1 --2- 172.21.15.119:0/3919252408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564caed3ea60 0x564caed3ee40 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.046+0000 7f5b002d7b80 1 --2- 172.21.15.119:0/3919252408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564caed3f380 0x564caed39540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.046+0000 7f5b002d7b80 1 -- 172.21.15.119:0/3919252408 >> 172.21.15.119:0/3919252408 conn(0x564caec68300 msgr2=0x564caed33b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.046+0000 7f5b002d7b80 1 -- 172.21.15.119:0/3919252408 shutdown_connections 2023-12-07T10:29:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.046+0000 7f5b002d7b80 1 -- 172.21.15.119:0/3919252408 wait complete. 2023-12-07T10:29:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.046+0000 7f5b002d7b80 1 -- start start 2023-12-07T10:29:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.047+0000 7f5b002d7b80 1 Processor -- start 2023-12-07T10:29:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.047+0000 7f5b002d7b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564caed39ad0 0x564caed42690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.047+0000 7f5b002d7b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564caed3ea60 0x564caed42bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.047+0000 7f5b002d7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564caed3f380 0x564caed43110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.047+0000 7f5b002d7b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564caec8d950 con 0x564caed3ea60 2023-12-07T10:29:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.047+0000 7f5b002d7b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x564caec47500 con 0x564caed39ad0 2023-12-07T10:29:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.047+0000 7f5b002d7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564caec471a0 con 0x564caed3f380 2023-12-07T10:29:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.047+0000 7f5b002d7b80 10 client.0 _ll_register_callbacks cb 0x564caec2e8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:29:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.047+0000 7f5ae659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564caed39ad0 0x564caed42690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.048+0000 7f5ae5d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564caed3ea60 0x564caed42bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.048+0000 7f5ae6d9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564caed3f380 0x564caed43110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.048+0000 7f5ae6d9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564caed3f380 0x564caed43110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:45704/0 (socket says 172.21.15.119:45704) 2023-12-07T10:29:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.048+0000 7f5ae6d9d700 1 -- 172.21.15.119:0/1048140682 learned_addr learned my addr 172.21.15.119:0/1048140682 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:29:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.048+0000 7f5ae659c700 1 --2- 172.21.15.119:0/1048140682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564caed39ad0 0x564caed42690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:55612/0 (socket says 172.21.15.119:55612) 2023-12-07T10:29:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.048+0000 7f5ae659c700 1 -- 172.21.15.119:0/1048140682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564caed3f380 msgr2=0x564caed43110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.048+0000 7f5ae659c700 1 --2- 172.21.15.119:0/1048140682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564caed3f380 0x564caed43110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.048+0000 7f5ae659c700 1 -- 172.21.15.119:0/1048140682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564caed3ea60 msgr2=0x564caed42bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.048+0000 7f5ae659c700 1 --2- 172.21.15.119:0/1048140682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564caed3ea60 0x564caed42bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.048+0000 7f5ae659c700 1 -- 172.21.15.119:0/1048140682 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae81a6970 con 0x564caed39ad0 2023-12-07T10:29:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.048+0000 7f5ae659c700 1 --2- 172.21.15.119:0/1048140682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564caed39ad0 0x564caed42690 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5ad80a3a60 tx=0x7f5ad80cc370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb2a8be35fe4a770 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.048+0000 7f5acdffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:29:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.048+0000 7f5acdffb700 1 -- 172.21.15.119:0/1048140682 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad80a38a0 con 0x564caed39ad0 2023-12-07T10:29:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.048+0000 7f5acdffb700 1 -- 172.21.15.119:0/1048140682 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f5ad809b5a0 con 0x564caed39ad0 2023-12-07T10:29:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.048+0000 7f5b002d7b80 1 -- 172.21.15.119:0/1048140682 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564caed2c6f0 con 0x564caed39ad0 2023-12-07T10:29:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.049+0000 7f5acdffb700 1 -- 172.21.15.119:0/1048140682 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ad8099940 con 0x564caed39ad0 2023-12-07T10:29:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.049+0000 7f5b002d7b80 10 client.27521 fetch_fsmap learned FSMap version 56 2023-12-07T10:29:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.049+0000 7f5b002d7b80 1 -- 172.21.15.119:0/1048140682 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x564caeab6d80 con 0x564caed39ad0 2023-12-07T10:29:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.049+0000 7f5acdffb700 1 -- 172.21.15.119:0/1048140682 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564caed2c6f0 con 0x564caed39ad0 2023-12-07T10:29:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.049+0000 7f5acdffb700 1 -- 172.21.15.119:0/1048140682 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 56) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f5ad809b800 con 0x564caed39ad0 2023-12-07T10:29:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.049+0000 7f5b002d7b80 10 client.27521 fetch_fsmap finished waiting for FSMap version 56 2023-12-07T10:29:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.049+0000 7f5b002d7b80 10 client.27521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:29:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.049+0000 7f5b002d7b80 20 client.27521 populate_metadata read hostname 'smithi119' 2023-12-07T10:29:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.049+0000 7f5b002d7b80 1 --2- 172.21.15.119:0/1048140682 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x564caed526c0 0x564caed54ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.049+0000 7f5b002d7b80 4 client.27521 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:29:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.049+0000 7f5b002d7b80 1 -- 172.21.15.119:0/1048140682 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564caed08740 con 0x564caed526c0 2023-12-07T10:29:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.050+0000 7f5ae5d9b700 1 --2- 172.21.15.119:0/1048140682 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x564caed526c0 0x564caed54ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.050+0000 7f5ae5d9b700 1 --2- 172.21.15.119:0/1048140682 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x564caed526c0 0x564caed54ae0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbb73feb28b11c61 server_cookie=e57ef8b1cffa7559 in_seq=0 out_seq=0 2023-12-07T10:29:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.050+0000 7f5acdffb700 10 client.27521 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:29:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.051+0000 7f5acdffb700 1 -- 172.21.15.119:0/1048140682 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564caed08740 con 0x564caed526c0 2023-12-07T10:29:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.051+0000 7f5acdffb700 10 client.27521 handle_command_reply: tid=0 2023-12-07T10:29:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.051+0000 7f5b002d7b80 1 client.27521 shutdown 2023-12-07T10:29:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.051+0000 7f5b002d7b80 1 -- 172.21.15.119:0/1048140682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564caed39ad0 msgr2=0x564caed42690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.051+0000 7f5b002d7b80 1 --2- 172.21.15.119:0/1048140682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564caed39ad0 0x564caed42690 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5ad80a3a60 tx=0x7f5ad80cc370 comp rx=0 tx=0).stop 2023-12-07T10:29:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.051+0000 7f5b002d7b80 1 -- 172.21.15.119:0/1048140682 shutdown_connections 2023-12-07T10:29:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.051+0000 7f5b002d7b80 1 --2- 172.21.15.119:0/1048140682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564caed3f380 0x564caed43110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.051+0000 7f5b002d7b80 1 --2- 172.21.15.119:0/1048140682 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x564caed526c0 0x564caed54ae0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.051+0000 7f5b002d7b80 1 --2- 172.21.15.119:0/1048140682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564caed39ad0 0x564caed42690 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.051+0000 7f5b002d7b80 1 --2- 172.21.15.119:0/1048140682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564caed3ea60 0x564caed42bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.051+0000 7f5acdffb700 0 client.27521 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:29:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.051+0000 7f5b002d7b80 1 -- 172.21.15.119:0/1048140682 >> 172.21.15.119:0/1048140682 conn(0x564caec68300 msgr2=0x564caed2e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.051+0000 7f5b002d7b80 1 -- 172.21.15.119:0/1048140682 shutdown_connections 2023-12-07T10:29:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.051+0000 7f5b002d7b80 1 -- 172.21.15.119:0/1048140682 wait complete. 2023-12-07T10:29:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.052+0000 7f5b002d7b80 20 client.27521 trim_cache size 0 max 16384 2023-12-07T10:29:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.075+0000 7f5b002d7b80 1 Processor -- start 2023-12-07T10:29:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.075+0000 7f5b002d7b80 1 -- start start 2023-12-07T10:29:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.077+0000 7f5b002d7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564caed39ad0 0x564caed3ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.077+0000 7f5b002d7b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564caed3ea60 0x564caed42f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.077+0000 7f5b002d7b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564caed3f380 0x564caed49a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.077+0000 7f5b002d7b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564caeb0c8c0 con 0x564caed3f380 2023-12-07T10:29:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.077+0000 7f5b002d7b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x564caed43c90 con 0x564caed3ea60 2023-12-07T10:29:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.077+0000 7f5b002d7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564caed307d0 con 0x564caed39ad0 2023-12-07T10:29:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.077+0000 7f5ae6d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564caed3f380 0x564caed49a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.077+0000 7f5ae659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564caed39ad0 0x564caed3ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.078+0000 7f5ae659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564caed39ad0 0x564caed3ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:45714/0 (socket says 172.21.15.119:45714) 2023-12-07T10:29:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.078+0000 7f5ae659c700 1 -- 172.21.15.119:0/846649802 learned_addr learned my addr 172.21.15.119:0/846649802 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:29:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.078+0000 7f5ae5d9b700 1 --2- 172.21.15.119:0/846649802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564caed3ea60 0x564caed42f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.078+0000 7f5ae6d9d700 1 -- 172.21.15.119:0/846649802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564caed39ad0 msgr2=0x564caed3ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.078+0000 7f5ae6d9d700 1 --2- 172.21.15.119:0/846649802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564caed39ad0 0x564caed3ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.078+0000 7f5ae6d9d700 1 -- 172.21.15.119:0/846649802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564caed3ea60 msgr2=0x564caed42f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.078+0000 7f5ae6d9d700 1 --2- 172.21.15.119:0/846649802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564caed3ea60 0x564caed42f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.078+0000 7f5ae6d9d700 1 -- 172.21.15.119:0/846649802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564caea9fc00 con 0x564caed3f380 2023-12-07T10:29:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.079+0000 7f5ae5d9b700 1 --2- 172.21.15.119:0/846649802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564caed3ea60 0x564caed42f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:29:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.079+0000 7f5ae6d9d700 1 --2- 172.21.15.119:0/846649802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564caed3f380 0x564caed49a60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0011820 tx=0x7f5ae0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59229a59f8fe0e0b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.080+0000 7f5ad57fa700 1 -- 172.21.15.119:0/846649802 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae000e040 con 0x564caed3f380 2023-12-07T10:29:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.081+0000 7f5ad57fa700 1 -- 172.21.15.119:0/846649802 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ae000f040 con 0x564caed3f380 2023-12-07T10:29:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.081+0000 7f5ad57fa700 1 -- 172.21.15.119:0/846649802 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0007ea0 con 0x564caed3f380 2023-12-07T10:29:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.082+0000 7f5b002d7b80 1 -- 172.21.15.119:0/846649802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564caed3f380 msgr2=0x564caed49a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.082+0000 7f5b002d7b80 1 --2- 172.21.15.119:0/846649802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564caed3f380 0x564caed49a60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0011820 tx=0x7f5ae0009920 comp rx=0 tx=0).stop 2023-12-07T10:29:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.082+0000 7f5b002d7b80 1 -- 172.21.15.119:0/846649802 shutdown_connections 2023-12-07T10:29:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.082+0000 7f5b002d7b80 1 --2- 172.21.15.119:0/846649802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564caed3f380 0x564caed49a60 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.082+0000 7f5b002d7b80 1 --2- 172.21.15.119:0/846649802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564caed39ad0 0x564caed3ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.082+0000 7f5b002d7b80 1 --2- 172.21.15.119:0/846649802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564caed3ea60 0x564caed42f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.082+0000 7f5b002d7b80 1 -- 172.21.15.119:0/846649802 >> 172.21.15.119:0/846649802 conn(0x564caec68300 msgr2=0x564caed3c900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.082+0000 7f5b002d7b80 1 -- 172.21.15.119:0/846649802 shutdown_connections 2023-12-07T10:29:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.083+0000 7f5b002d7b80 1 -- 172.21.15.119:0/846649802 wait complete. 2023-12-07T10:29:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.083+0000 7f5b002d7b80 1 -- start start 2023-12-07T10:29:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.083+0000 7f5b002d7b80 1 Processor -- start 2023-12-07T10:29:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.084+0000 7f5b002d7b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564caed39ad0 0x564caed47c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.084+0000 7f5b002d7b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564caed3ea60 0x564caed48170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.084+0000 7f5b002d7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564caed3f380 0x564caedb0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.084+0000 7f5b002d7b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564caed48910 con 0x564caed39ad0 2023-12-07T10:29:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.084+0000 7f5b002d7b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x564caedb0900 con 0x564caed3ea60 2023-12-07T10:29:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.084+0000 7f5b002d7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564caedb0ac0 con 0x564caed3f380 2023-12-07T10:29:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.084+0000 7f5b002d7b80 10 client.0 _ll_register_callbacks cb 0x564caec2e8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:29:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.084+0000 7f5ae659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564caed39ad0 0x564caed47c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.085+0000 7f5ae659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564caed39ad0 0x564caed47c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59026/0 (socket says 172.21.15.119:59026) 2023-12-07T10:29:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.085+0000 7f5ae5d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564caed3ea60 0x564caed48170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.085+0000 7f5ae659c700 1 -- 172.21.15.119:0/2450549380 learned_addr learned my addr 172.21.15.119:0/2450549380 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:29:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.085+0000 7f5ae6d9d700 1 --2- 172.21.15.119:0/2450549380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564caed3f380 0x564caedb0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.085+0000 7f5ae659c700 1 -- 172.21.15.119:0/2450549380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564caed3f380 msgr2=0x564caedb0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.085+0000 7f5ae659c700 1 --2- 172.21.15.119:0/2450549380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564caed3f380 0x564caedb0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.085+0000 7f5ae659c700 1 -- 172.21.15.119:0/2450549380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564caed3ea60 msgr2=0x564caed48170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.085+0000 7f5ae659c700 1 --2- 172.21.15.119:0/2450549380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564caed3ea60 0x564caed48170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.085+0000 7f5ae659c700 1 -- 172.21.15.119:0/2450549380 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae00114d0 con 0x564caed39ad0 2023-12-07T10:29:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.085+0000 7f5ae5d9b700 1 --2- 172.21.15.119:0/2450549380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564caed3ea60 0x564caed48170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:29:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.085+0000 7f5ae659c700 1 --2- 172.21.15.119:0/2450549380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564caed39ad0 0x564caed47c30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5ad809b4e0 tx=0x7f5ad80054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb4cbf350a2fb2c server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.085+0000 7f5acdffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:29:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.085+0000 7f5acdffb700 1 -- 172.21.15.119:0/2450549380 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8021380 con 0x564caed39ad0 2023-12-07T10:29:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.085+0000 7f5b002d7b80 1 -- 172.21.15.119:0/2450549380 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ad809f6f0 con 0x564caed39ad0 2023-12-07T10:29:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.086+0000 7f5acdffb700 1 -- 172.21.15.119:0/2450549380 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f5ad809aae0 con 0x564caed39ad0 2023-12-07T10:29:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.086+0000 7f5acdffb700 1 -- 172.21.15.119:0/2450549380 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ad809b5a0 con 0x564caed39ad0 2023-12-07T10:29:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.086+0000 7f5b002d7b80 10 client.17487 fetch_fsmap learned FSMap version 56 2023-12-07T10:29:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.086+0000 7f5b002d7b80 1 -- 172.21.15.119:0/2450549380 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x564caeadd1f0 con 0x564caed39ad0 2023-12-07T10:29:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.086+0000 7f5acdffb700 1 -- 172.21.15.119:0/2450549380 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ad809f6f0 con 0x564caed39ad0 2023-12-07T10:29:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.087+0000 7f5acdffb700 1 -- 172.21.15.119:0/2450549380 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 56) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f5ad80a3410 con 0x564caed39ad0 2023-12-07T10:29:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.087+0000 7f5b002d7b80 10 client.17487 fetch_fsmap finished waiting for FSMap version 56 2023-12-07T10:29:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.087+0000 7f5b002d7b80 10 client.17487 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:29:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.087+0000 7f5b002d7b80 20 client.17487 populate_metadata read hostname 'smithi119' 2023-12-07T10:29:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.087+0000 7f5b002d7b80 1 --2- 172.21.15.119:0/2450549380 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x564caedb1da0 0x564caedb4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.087+0000 7f5b002d7b80 4 client.17487 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:29:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.087+0000 7f5b002d7b80 1 -- 172.21.15.119:0/2450549380 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564caec53f70 con 0x564caedb1da0 2023-12-07T10:29:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.087+0000 7f5ae5d9b700 1 --2- 172.21.15.119:0/2450549380 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x564caedb1da0 0x564caedb4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.088+0000 7f5ae5d9b700 1 --2- 172.21.15.119:0/2450549380 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x564caedb1da0 0x564caedb4190 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fd752416357baef server_cookie=2ff55317666fbfc9 in_seq=0 out_seq=0 2023-12-07T10:29:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.088+0000 7f5acdffb700 10 client.17487 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:29:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.142+0000 7f5acdffb700 1 -- 172.21.15.119:0/2450549380 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564caec53f70 con 0x564caedb1da0 2023-12-07T10:29:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.142+0000 7f5acdffb700 10 client.17487 handle_command_reply: tid=0 2023-12-07T10:29:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.142+0000 7f5b002d7b80 1 client.17487 shutdown 2023-12-07T10:29:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.142+0000 7f5b002d7b80 1 -- 172.21.15.119:0/2450549380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564caed39ad0 msgr2=0x564caed47c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.142+0000 7f5b002d7b80 1 --2- 172.21.15.119:0/2450549380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564caed39ad0 0x564caed47c30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5ad809b4e0 tx=0x7f5ad80054e0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.142+0000 7f5b002d7b80 1 -- 172.21.15.119:0/2450549380 shutdown_connections 2023-12-07T10:29:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.142+0000 7f5b002d7b80 1 --2- 172.21.15.119:0/2450549380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564caed3f380 0x564caedb0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.142+0000 7f5b002d7b80 1 --2- 172.21.15.119:0/2450549380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564caed39ad0 0x564caed47c30 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.142+0000 7f5b002d7b80 1 --2- 172.21.15.119:0/2450549380 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x564caedb1da0 0x564caedb4190 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.142+0000 7f5b002d7b80 1 --2- 172.21.15.119:0/2450549380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x564caed3ea60 0x564caed48170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.142+0000 7f5acdffb700 0 client.17487 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:29:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.142+0000 7f5b002d7b80 1 -- 172.21.15.119:0/2450549380 >> 172.21.15.119:0/2450549380 conn(0x564caec68300 msgr2=0x564caed40e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.143+0000 7f5b002d7b80 1 -- 172.21.15.119:0/2450549380 shutdown_connections 2023-12-07T10:29:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.143+0000 7f5b002d7b80 1 -- 172.21.15.119:0/2450549380 wait complete. 2023-12-07T10:29:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.143+0000 7f5b002d7b80 20 client.17487 trim_cache size 0 max 16384 2023-12-07T10:29:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:29:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:29:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:29:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:29:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.143+0000 7f5aec9ff700 1 -- 172.21.15.119:0/200118544 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5ad00a1c80 msgr2=0x7f5ad00a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.143+0000 7f5aec9ff700 1 --2- 172.21.15.119:0/200118544 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5ad00a1c80 0x7f5ad00a4140 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5ae819ac70 tx=0x7f5adc009310 comp rx=0 tx=0).stop 2023-12-07T10:29:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.143+0000 7f5aec9ff700 1 -- 172.21.15.119:0/200118544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ae80739e0 msgr2=0x7f5ae819f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.143+0000 7f5aec9ff700 1 --2- 172.21.15.119:0/200118544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ae80739e0 0x7f5ae819f980 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8005950 tx=0x7f5ad800fa00 comp rx=0 tx=0).stop 2023-12-07T10:29:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.144+0000 7f5aec9ff700 1 -- 172.21.15.119:0/200118544 shutdown_connections 2023-12-07T10:29:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.144+0000 7f5aec9ff700 1 --2- 172.21.15.119:0/200118544 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5ad00a1c80 0x7f5ad00a4140 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.144+0000 7f5aec9ff700 1 --2- 172.21.15.119:0/200118544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ae810dda0 0x7f5ae8199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.144+0000 7f5aec9ff700 1 --2- 172.21.15.119:0/200118544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ae80739e0 0x7f5ae819f980 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.144+0000 7f5aec9ff700 1 --2- 172.21.15.119:0/200118544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ae8074300 0x7f5ae819fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.144+0000 7f5aec9ff700 1 -- 172.21.15.119:0/200118544 >> 172.21.15.119:0/200118544 conn(0x7f5ae80fcfd0 msgr2=0x7f5ae80fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.144+0000 7f5aec9ff700 1 -- 172.21.15.119:0/200118544 shutdown_connections 2023-12-07T10:29:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:18.144+0000 7f5aec9ff700 1 -- 172.21.15.119:0/200118544 wait complete. 2023-12-07T10:29:18.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:18 smithi119 ceph-mon[97908]: pgmap v275: 129 pgs: 129 active+clean; 415 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 262 op/s 2023-12-07T10:29:18.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:18 smithi159 ceph-mon[134726]: pgmap v275: 129 pgs: 129 active+clean; 415 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 262 op/s 2023-12-07T10:29:18.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:18 smithi153 ceph-mon[133625]: pgmap v275: 129 pgs: 129 active+clean; 415 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 262 op/s 2023-12-07T10:29:18.568 INFO:tasks.workunit.client.0.smithi119.stdout: 1 623886 7.14 MB/sec execute 331 sec latency 35.622 ms 2023-12-07T10:29:19.569 INFO:tasks.workunit.client.0.smithi119.stdout: 1 625447 7.15 MB/sec execute 332 sec latency 13.017 ms 2023-12-07T10:29:20.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:20 smithi119 ceph-mon[97908]: pgmap v276: 129 pgs: 129 active+clean; 415 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 262 op/s 2023-12-07T10:29:20.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:20 smithi159 ceph-mon[134726]: pgmap v276: 129 pgs: 129 active+clean; 415 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 262 op/s 2023-12-07T10:29:20.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:20 smithi153 ceph-mon[133625]: pgmap v276: 129 pgs: 129 active+clean; 415 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 262 op/s 2023-12-07T10:29:20.569 INFO:tasks.workunit.client.0.smithi119.stdout: 1 626771 7.15 MB/sec execute 333 sec latency 16.390 ms 2023-12-07T10:29:21.569 INFO:tasks.workunit.client.0.smithi119.stdout: 1 628846 7.17 MB/sec execute 334 sec latency 15.093 ms 2023-12-07T10:29:22.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:22 smithi119 ceph-mon[97908]: pgmap v277: 129 pgs: 129 active+clean; 419 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 371 op/s 2023-12-07T10:29:22.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:22 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:29:22.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:22 smithi159 ceph-mon[134726]: pgmap v277: 129 pgs: 129 active+clean; 419 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 371 op/s 2023-12-07T10:29:22.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:22 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:29:22.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:22 smithi153 ceph-mon[133625]: pgmap v277: 129 pgs: 129 active+clean; 419 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 371 op/s 2023-12-07T10:29:22.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:22 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:29:22.569 INFO:tasks.workunit.client.0.smithi119.stdout: 1 629803 7.15 MB/sec execute 335 sec latency 23.622 ms 2023-12-07T10:29:23.569 INFO:tasks.workunit.client.0.smithi119.stdout: 1 631806 7.16 MB/sec execute 336 sec latency 21.029 ms 2023-12-07T10:29:24.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:24 smithi119 ceph-mon[97908]: pgmap v278: 129 pgs: 129 active+clean; 419 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 234 op/s 2023-12-07T10:29:24.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:24 smithi159 ceph-mon[134726]: pgmap v278: 129 pgs: 129 active+clean; 419 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 234 op/s 2023-12-07T10:29:24.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:24 smithi153 ceph-mon[133625]: pgmap v278: 129 pgs: 129 active+clean; 419 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 234 op/s 2023-12-07T10:29:24.569 INFO:tasks.workunit.client.0.smithi119.stdout: 1 632947 7.16 MB/sec execute 337 sec latency 50.487 ms 2023-12-07T10:29:25.569 INFO:tasks.workunit.client.0.smithi119.stdout: 1 634762 7.16 MB/sec execute 338 sec latency 15.999 ms 2023-12-07T10:29:26.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:26 smithi119 ceph-mon[97908]: pgmap v279: 129 pgs: 129 active+clean; 419 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2023-12-07T10:29:26.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:26 smithi159 ceph-mon[134726]: pgmap v279: 129 pgs: 129 active+clean; 419 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2023-12-07T10:29:26.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:26 smithi153 ceph-mon[133625]: pgmap v279: 129 pgs: 129 active+clean; 419 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2023-12-07T10:29:26.569 INFO:tasks.workunit.client.0.smithi119.stdout: 1 636213 7.17 MB/sec execute 339 sec latency 12.676 ms 2023-12-07T10:29:27.569 INFO:tasks.workunit.client.0.smithi119.stdout: 1 637689 7.17 MB/sec execute 340 sec latency 22.926 ms 2023-12-07T10:29:28.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:28 smithi119 ceph-mon[97908]: pgmap v280: 129 pgs: 129 active+clean; 428 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 278 op/s 2023-12-07T10:29:28.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:28 smithi159 ceph-mon[134726]: pgmap v280: 129 pgs: 129 active+clean; 428 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 278 op/s 2023-12-07T10:29:28.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:28 smithi153 ceph-mon[133625]: pgmap v280: 129 pgs: 129 active+clean; 428 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 278 op/s 2023-12-07T10:29:28.569 INFO:tasks.workunit.client.0.smithi119.stdout: 1 639506 7.19 MB/sec execute 341 sec latency 18.966 ms 2023-12-07T10:29:29.569 INFO:tasks.workunit.client.0.smithi119.stdout: 1 640762 7.18 MB/sec execute 342 sec latency 14.778 ms 2023-12-07T10:29:30.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:30 smithi119 ceph-mon[97908]: pgmap v281: 129 pgs: 129 active+clean; 428 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 190 op/s 2023-12-07T10:29:30.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:30 smithi159 ceph-mon[134726]: pgmap v281: 129 pgs: 129 active+clean; 428 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 190 op/s 2023-12-07T10:29:30.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:30 smithi153 ceph-mon[133625]: pgmap v281: 129 pgs: 129 active+clean; 428 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 190 op/s 2023-12-07T10:29:30.569 INFO:tasks.workunit.client.0.smithi119.stdout: 1 642863 7.20 MB/sec execute 343 sec latency 16.038 ms 2023-12-07T10:29:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.310+0000 7f9ddb1ff700 1 -- 172.21.15.119:0/2513994675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dd4101ee0 msgr2=0x7f9dd410e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.310+0000 7f9ddb1ff700 1 --2- 172.21.15.119:0/2513994675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dd4101ee0 0x7f9dd410e3a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0099c0 tx=0x7f9dcc009cd0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.310+0000 7f9ddb1ff700 1 -- 172.21.15.119:0/2513994675 shutdown_connections 2023-12-07T10:29:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.310+0000 7f9ddb1ff700 1 --2- 172.21.15.119:0/2513994675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dd4101ee0 0x7f9dd410e3a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.310+0000 7f9ddb1ff700 1 --2- 172.21.15.119:0/2513994675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9dd41075e0 0x7f9dd41079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.310+0000 7f9ddb1ff700 1 --2- 172.21.15.119:0/2513994675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9dd4101520 0x7f9dd41019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.310+0000 7f9ddb1ff700 1 -- 172.21.15.119:0/2513994675 >> 172.21.15.119:0/2513994675 conn(0x7f9dd40fd010 msgr2=0x7f9dd40ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.310+0000 7f9ddb1ff700 1 -- 172.21.15.119:0/2513994675 shutdown_connections 2023-12-07T10:29:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.310+0000 7f9ddb1ff700 1 -- 172.21.15.119:0/2513994675 wait complete. 2023-12-07T10:29:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.311+0000 7f9ddb1ff700 1 Processor -- start 2023-12-07T10:29:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.312+0000 7f9ddb1ff700 1 -- start start 2023-12-07T10:29:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.312+0000 7f9ddb1ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9dd4101520 0x7f9dd419d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.312+0000 7f9ddb1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dd4101ee0 0x7f9dd419dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.312+0000 7f9ddb1ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9dd41075e0 0x7f9dd41976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.312+0000 7f9ddb1ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd4197d30 con 0x7f9dd41075e0 2023-12-07T10:29:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.312+0000 7f9ddb1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd4197ea0 con 0x7f9dd4101ee0 2023-12-07T10:29:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.312+0000 7f9ddb1ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd4198060 con 0x7f9dd4101520 2023-12-07T10:29:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.312+0000 7f9dd8f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9dd4101520 0x7f9dd419d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.312+0000 7f9dd8f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9dd4101520 0x7f9dd419d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:57960/0 (socket says 172.21.15.119:57960) 2023-12-07T10:29:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.312+0000 7f9dd8f9b700 1 -- 172.21.15.119:0/3027764857 learned_addr learned my addr 172.21.15.119:0/3027764857 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:29:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.312+0000 7f9dd3fff700 1 --2- 172.21.15.119:0/3027764857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dd4101ee0 0x7f9dd419dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.312+0000 7f9dd979c700 1 --2- 172.21.15.119:0/3027764857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9dd41075e0 0x7f9dd41976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.313+0000 7f9dd8f9b700 1 -- 172.21.15.119:0/3027764857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dd4101ee0 msgr2=0x7f9dd419dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.313+0000 7f9dd8f9b700 1 --2- 172.21.15.119:0/3027764857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dd4101ee0 0x7f9dd419dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.313+0000 7f9dd8f9b700 1 -- 172.21.15.119:0/3027764857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9dd41075e0 msgr2=0x7f9dd41976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.313+0000 7f9dd8f9b700 1 --2- 172.21.15.119:0/3027764857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9dd41075e0 0x7f9dd41976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.313+0000 7f9dd8f9b700 1 -- 172.21.15.119:0/3027764857 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc8009c40 con 0x7f9dd4101520 2023-12-07T10:29:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.313+0000 7f9dd3fff700 1 --2- 172.21.15.119:0/3027764857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dd4101ee0 0x7f9dd419dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:29:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.313+0000 7f9dd8f9b700 1 --2- 172.21.15.119:0/3027764857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9dd4101520 0x7f9dd419d5a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9dc800c370 tx=0x7f9dc800c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.313+0000 7f9dd1ffb700 1 -- 172.21.15.119:0/3027764857 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8017070 con 0x7f9dd4101520 2023-12-07T10:29:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.313+0000 7f9ddb1ff700 1 -- 172.21.15.119:0/3027764857 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dcc0095f0 con 0x7f9dd4101520 2023-12-07T10:29:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.313+0000 7f9dd1ffb700 1 -- 172.21.15.119:0/3027764857 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dc800f040 con 0x7f9dd4101520 2023-12-07T10:29:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.313+0000 7f9dd1ffb700 1 -- 172.21.15.119:0/3027764857 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc800e050 con 0x7f9dd4101520 2023-12-07T10:29:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.313+0000 7f9ddb1ff700 1 -- 172.21.15.119:0/3027764857 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd4073840 con 0x7f9dd4101520 2023-12-07T10:29:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.318+0000 7f9dd1ffb700 1 -- 172.21.15.119:0/3027764857 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f9dc8004440 con 0x7f9dd4101520 2023-12-07T10:29:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.319+0000 7f9deeb6eb80 1 Processor -- start 2023-12-07T10:29:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.319+0000 7f9deeb6eb80 1 -- start start 2023-12-07T10:29:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.319+0000 7f9deeb6eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55684f782a60 0x55684f782e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.320+0000 7f9deeb6eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55684f783380 0x55684f77d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.320+0000 7f9deeb6eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f77dad0 0x55684f77df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.320+0000 7f9deeb6eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684f6f1760 con 0x55684f783380 2023-12-07T10:29:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.320+0000 7f9deeb6eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684f6d8740 con 0x55684f77dad0 2023-12-07T10:29:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.320+0000 7f9deeb6eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684f6d6060 con 0x55684f782a60 2023-12-07T10:29:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.320+0000 7f9dd1ffb700 1 --2- 172.21.15.119:0/3027764857 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9dc40a1c80 0x7f9dc40a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.320+0000 7f9dd8f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55684f783380 0x55684f77d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.320+0000 7f9dd1ffb700 1 -- 172.21.15.119:0/3027764857 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f9dc80cd6f0 con 0x7f9dd4101520 2023-12-07T10:29:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.320+0000 7f9dd979c700 1 --2- 172.21.15.119:0/3027764857 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9dc40a1c80 0x7f9dc40a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.320+0000 7f9dd8f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55684f783380 0x55684f77d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55898/0 (socket says 172.21.15.119:55898) 2023-12-07T10:29:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.320+0000 7f9dd8f9b700 1 -- 172.21.15.119:0/2128253421 learned_addr learned my addr 172.21.15.119:0/2128253421 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:29:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.320+0000 7f9dd8f9b700 1 -- 172.21.15.119:0/2128253421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55684f782a60 msgr2=0x55684f782e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.321+0000 7f9dd8f9b700 1 --2- 172.21.15.119:0/2128253421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55684f782a60 0x55684f782e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.321+0000 7f9dd3fff700 1 --2- 172.21.15.119:0/2128253421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f77dad0 0x55684f77df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.321+0000 7f9dd8f9b700 1 -- 172.21.15.119:0/2128253421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f77dad0 msgr2=0x55684f77df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.321+0000 7f9dd8f9b700 1 --2- 172.21.15.119:0/2128253421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f77dad0 0x55684f77df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.321+0000 7f9dd8f9b700 1 -- 172.21.15.119:0/2128253421 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55684f54e5d0 con 0x55684f783380 2023-12-07T10:29:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.321+0000 7f9dd3fff700 1 --2- 172.21.15.119:0/2128253421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f77dad0 0x55684f77df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:29:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.321+0000 7f9dd979c700 1 --2- 172.21.15.119:0/3027764857 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9dc40a1c80 0x7f9dc40a4140 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc005b40 tx=0x7f9dcc01a040 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.321+0000 7f9dd8f9b700 1 --2- 172.21.15.119:0/2128253421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55684f783380 0x55684f77d540 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80caaf0 tx=0x7f9dc80964a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7f9f46a9672a8d server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.321+0000 7f9dbeffd700 1 -- 172.21.15.119:0/2128253421 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8099a40 con 0x55684f783380 2023-12-07T10:29:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.321+0000 7f9dbeffd700 1 -- 172.21.15.119:0/2128253421 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dc809a7e0 con 0x55684f783380 2023-12-07T10:29:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.321+0000 7f9dbeffd700 1 -- 172.21.15.119:0/2128253421 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8098480 con 0x55684f783380 2023-12-07T10:29:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.322+0000 7f9deeb6eb80 1 -- 172.21.15.119:0/2128253421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55684f783380 msgr2=0x55684f77d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.322+0000 7f9deeb6eb80 1 --2- 172.21.15.119:0/2128253421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55684f783380 0x55684f77d540 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80caaf0 tx=0x7f9dc80964a0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.322+0000 7f9deeb6eb80 1 -- 172.21.15.119:0/2128253421 shutdown_connections 2023-12-07T10:29:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.322+0000 7f9deeb6eb80 1 --2- 172.21.15.119:0/2128253421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f77dad0 0x55684f77df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.322+0000 7f9deeb6eb80 1 --2- 172.21.15.119:0/2128253421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55684f782a60 0x55684f782e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.322+0000 7f9deeb6eb80 1 --2- 172.21.15.119:0/2128253421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55684f783380 0x55684f77d540 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.322+0000 7f9deeb6eb80 1 -- 172.21.15.119:0/2128253421 >> 172.21.15.119:0/2128253421 conn(0x55684f6ac300 msgr2=0x55684f777b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.322+0000 7f9deeb6eb80 1 -- 172.21.15.119:0/2128253421 shutdown_connections 2023-12-07T10:29:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.322+0000 7f9deeb6eb80 1 -- 172.21.15.119:0/2128253421 wait complete. 2023-12-07T10:29:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.323+0000 7f9deeb6eb80 1 -- start start 2023-12-07T10:29:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.324+0000 7f9deeb6eb80 1 Processor -- start 2023-12-07T10:29:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.324+0000 7f9deeb6eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f77dad0 0x55684f7867c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.324+0000 7f9deeb6eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55684f782a60 0x55684f786d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.324+0000 7f9deeb6eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55684f783380 0x55684f787240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.324+0000 7f9deeb6eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684f6d1950 con 0x55684f782a60 2023-12-07T10:29:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.324+0000 7f9deeb6eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684f68b500 con 0x55684f77dad0 2023-12-07T10:29:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.324+0000 7f9deeb6eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684f68b1a0 con 0x55684f783380 2023-12-07T10:29:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.324+0000 7f9deeb6eb80 10 client.0 _ll_register_callbacks cb 0x55684f6728e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:29:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.324+0000 7f9dd8f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f77dad0 0x55684f7867c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.324+0000 7f9dd8f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f77dad0 0x55684f7867c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:38362/0 (socket says 172.21.15.119:38362) 2023-12-07T10:29:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.324+0000 7f9dd8f9b700 1 -- 172.21.15.119:0/2192294380 learned_addr learned my addr 172.21.15.119:0/2192294380 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:29:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.324+0000 7f9dd979c700 1 --2- 172.21.15.119:0/2192294380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55684f783380 0x55684f787240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.324+0000 7f9dd3fff700 1 --2- 172.21.15.119:0/2192294380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55684f782a60 0x55684f786d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.325+0000 7f9dd8f9b700 1 -- 172.21.15.119:0/2192294380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55684f783380 msgr2=0x55684f787240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.325+0000 7f9dd8f9b700 1 --2- 172.21.15.119:0/2192294380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55684f783380 0x55684f787240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.325+0000 7f9dd8f9b700 1 -- 172.21.15.119:0/2192294380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55684f782a60 msgr2=0x55684f786d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.325+0000 7f9dd8f9b700 1 --2- 172.21.15.119:0/2192294380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55684f782a60 0x55684f786d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.325+0000 7f9dd8f9b700 1 -- 172.21.15.119:0/2192294380 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd4073840 con 0x55684f77dad0 2023-12-07T10:29:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.325+0000 7f9dd3fff700 1 --2- 172.21.15.119:0/2192294380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55684f782a60 0x55684f786d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:29:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.325+0000 7f9dd8f9b700 1 --2- 172.21.15.119:0/2192294380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f77dad0 0x55684f7867c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80cab40 tx=0x7f9dc8099a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2afcc72b46fd42d2 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.325+0000 7f9db3fff700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:29:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.325+0000 7f9db3fff700 1 -- 172.21.15.119:0/2192294380 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc80192c0 con 0x55684f77dad0 2023-12-07T10:29:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.325+0000 7f9db3fff700 1 -- 172.21.15.119:0/2192294380 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f9dc80a2470 con 0x55684f77dad0 2023-12-07T10:29:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.325+0000 7f9deeb6eb80 1 -- 172.21.15.119:0/2192294380 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55684f7706f0 con 0x55684f77dad0 2023-12-07T10:29:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.325+0000 7f9db3fff700 1 -- 172.21.15.119:0/2192294380 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9dc80cd2e0 con 0x55684f77dad0 2023-12-07T10:29:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.325+0000 7f9deeb6eb80 10 client.27536 fetch_fsmap learned FSMap version 57 2023-12-07T10:29:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.325+0000 7f9deeb6eb80 1 -- 172.21.15.119:0/2192294380 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55684f4fad80 con 0x55684f77dad0 2023-12-07T10:29:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.326+0000 7f9db3fff700 1 -- 172.21.15.119:0/2192294380 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55684f7706f0 con 0x55684f77dad0 2023-12-07T10:29:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.326+0000 7f9db3fff700 1 -- 172.21.15.119:0/2192294380 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9dc80995d0 con 0x55684f77dad0 2023-12-07T10:29:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.326+0000 7f9deeb6eb80 10 client.27536 fetch_fsmap finished waiting for FSMap version 57 2023-12-07T10:29:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.326+0000 7f9deeb6eb80 10 client.27536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:29:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.326+0000 7f9deeb6eb80 20 client.27536 populate_metadata read hostname 'smithi119' 2023-12-07T10:29:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.326+0000 7f9deeb6eb80 1 --2- 172.21.15.119:0/2192294380 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55684f796b00 0x55684f798f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.326+0000 7f9deeb6eb80 4 client.27536 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:29:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.326+0000 7f9deeb6eb80 1 -- 172.21.15.119:0/2192294380 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55684f74c740 con 0x55684f796b00 2023-12-07T10:29:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.326+0000 7f9dd3fff700 1 --2- 172.21.15.119:0/2192294380 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55684f796b00 0x55684f798f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.327+0000 7f9db3fff700 10 client.27536 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:29:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.327+0000 7f9dd3fff700 1 --2- 172.21.15.119:0/2192294380 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55684f796b00 0x55684f798f20 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=398c0c5a4cce9ffd server_cookie=3c382abfa61d5c92 in_seq=0 out_seq=0 2023-12-07T10:29:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.327+0000 7f9db3fff700 1 -- 172.21.15.119:0/2192294380 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55684f74c740 con 0x55684f796b00 2023-12-07T10:29:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.327+0000 7f9db3fff700 10 client.27536 handle_command_reply: tid=0 2023-12-07T10:29:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.327+0000 7f9deeb6eb80 1 client.27536 shutdown 2023-12-07T10:29:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.328+0000 7f9deeb6eb80 1 -- 172.21.15.119:0/2192294380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f77dad0 msgr2=0x55684f7867c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.328+0000 7f9deeb6eb80 1 --2- 172.21.15.119:0/2192294380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f77dad0 0x55684f7867c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80cab40 tx=0x7f9dc8099a40 comp rx=0 tx=0).stop 2023-12-07T10:29:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.328+0000 7f9deeb6eb80 1 -- 172.21.15.119:0/2192294380 shutdown_connections 2023-12-07T10:29:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.328+0000 7f9deeb6eb80 1 --2- 172.21.15.119:0/2192294380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55684f783380 0x55684f787240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.328+0000 7f9deeb6eb80 1 --2- 172.21.15.119:0/2192294380 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55684f796b00 0x55684f798f20 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.328+0000 7f9deeb6eb80 1 --2- 172.21.15.119:0/2192294380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f77dad0 0x55684f7867c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.328+0000 7f9deeb6eb80 1 --2- 172.21.15.119:0/2192294380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55684f782a60 0x55684f786d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.328+0000 7f9deeb6eb80 1 -- 172.21.15.119:0/2192294380 >> 172.21.15.119:0/2192294380 conn(0x55684f6ac300 msgr2=0x55684f772980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.328+0000 7f9db3fff700 0 client.27536 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:29:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.328+0000 7f9deeb6eb80 1 -- 172.21.15.119:0/2192294380 shutdown_connections 2023-12-07T10:29:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.328+0000 7f9deeb6eb80 1 -- 172.21.15.119:0/2192294380 wait complete. 2023-12-07T10:29:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.328+0000 7f9deeb6eb80 20 client.27536 trim_cache size 0 max 16384 2023-12-07T10:29:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.346+0000 7f9deeb6eb80 1 Processor -- start 2023-12-07T10:29:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.346+0000 7f9deeb6eb80 1 -- start start 2023-12-07T10:29:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.346+0000 7f9deeb6eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f77dad0 0x55684f77d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.346+0000 7f9deeb6eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55684f782a60 0x55684f78ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.347+0000 7f9deeb6eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55684f783380 0x55684f78e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.347+0000 7f9deeb6eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684f5508c0 con 0x55684f782a60 2023-12-07T10:29:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.347+0000 7f9deeb6eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684f7784c0 con 0x55684f77dad0 2023-12-07T10:29:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.347+0000 7f9deeb6eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684f784d60 con 0x55684f783380 2023-12-07T10:29:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.347+0000 7f9dd8f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f77dad0 0x55684f77d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.347+0000 7f9dd8f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f77dad0 0x55684f77d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:38366/0 (socket says 172.21.15.119:38366) 2023-12-07T10:29:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.347+0000 7f9dd8f9b700 1 -- 172.21.15.119:0/1216301918 learned_addr learned my addr 172.21.15.119:0/1216301918 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:29:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.347+0000 7f9dd979c700 1 --2- 172.21.15.119:0/1216301918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55684f783380 0x55684f78e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.347+0000 7f9dd3fff700 1 --2- 172.21.15.119:0/1216301918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55684f782a60 0x55684f78ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.347+0000 7f9dd8f9b700 1 -- 172.21.15.119:0/1216301918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55684f783380 msgr2=0x55684f78e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.347+0000 7f9dd8f9b700 1 --2- 172.21.15.119:0/1216301918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55684f783380 0x55684f78e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.347+0000 7f9dd8f9b700 1 -- 172.21.15.119:0/1216301918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55684f782a60 msgr2=0x55684f78ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.347+0000 7f9dd8f9b700 1 --2- 172.21.15.119:0/1216301918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55684f782a60 0x55684f78ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.347+0000 7f9dd8f9b700 1 -- 172.21.15.119:0/1216301918 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55684f4e3c00 con 0x55684f77dad0 2023-12-07T10:29:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.347+0000 7f9dd3fff700 1 --2- 172.21.15.119:0/1216301918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55684f782a60 0x55684f78ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:29:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.347+0000 7f9dd8f9b700 1 --2- 172.21.15.119:0/1216301918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f77dad0 0x55684f77d540 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80cb090 tx=0x7f9dc8098690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db88ab3de525ab99 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.348+0000 7f9dbf7fe700 1 -- 172.21.15.119:0/1216301918 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc809a0e0 con 0x55684f77dad0 2023-12-07T10:29:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.348+0000 7f9dbf7fe700 1 -- 172.21.15.119:0/1216301918 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dc80192c0 con 0x55684f77dad0 2023-12-07T10:29:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.348+0000 7f9dbf7fe700 1 -- 172.21.15.119:0/1216301918 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc80cd2e0 con 0x55684f77dad0 2023-12-07T10:29:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.348+0000 7f9deeb6eb80 1 -- 172.21.15.119:0/1216301918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f77dad0 msgr2=0x55684f77d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.348+0000 7f9deeb6eb80 1 --2- 172.21.15.119:0/1216301918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f77dad0 0x55684f77d540 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80cb090 tx=0x7f9dc8098690 comp rx=0 tx=0).stop 2023-12-07T10:29:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.349+0000 7f9deeb6eb80 1 -- 172.21.15.119:0/1216301918 shutdown_connections 2023-12-07T10:29:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.349+0000 7f9deeb6eb80 1 --2- 172.21.15.119:0/1216301918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55684f783380 0x55684f78e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.349+0000 7f9deeb6eb80 1 --2- 172.21.15.119:0/1216301918 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f77dad0 0x55684f77d540 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.349+0000 7f9deeb6eb80 1 --2- 172.21.15.119:0/1216301918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55684f782a60 0x55684f78ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.349+0000 7f9deeb6eb80 1 -- 172.21.15.119:0/1216301918 >> 172.21.15.119:0/1216301918 conn(0x55684f6ac300 msgr2=0x55684f79d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.349+0000 7f9deeb6eb80 1 -- 172.21.15.119:0/1216301918 shutdown_connections 2023-12-07T10:29:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.349+0000 7f9deeb6eb80 1 -- 172.21.15.119:0/1216301918 wait complete. 2023-12-07T10:29:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.349+0000 7f9deeb6eb80 1 -- start start 2023-12-07T10:29:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.349+0000 7f9deeb6eb80 1 Processor -- start 2023-12-07T10:29:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.350+0000 7f9deeb6eb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55684f77dad0 0x55684f78ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.350+0000 7f9deeb6eb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55684f782a60 0x55684f78e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.350+0000 7f9deeb6eb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f783380 0x55684f78be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.350+0000 7f9deeb6eb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684f78c520 con 0x55684f782a60 2023-12-07T10:29:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.350+0000 7f9deeb6eb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684f78c690 con 0x55684f783380 2023-12-07T10:29:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.350+0000 7f9deeb6eb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684f78c850 con 0x55684f77dad0 2023-12-07T10:29:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.350+0000 7f9deeb6eb80 10 client.0 _ll_register_callbacks cb 0x55684f6728e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:29:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.350+0000 7f9dd979c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f783380 0x55684f78be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.350+0000 7f9dd8f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55684f77dad0 0x55684f78ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.350+0000 7f9dd3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55684f782a60 0x55684f78e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.350+0000 7f9dd979c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f783380 0x55684f78be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:38368/0 (socket says 172.21.15.119:38368) 2023-12-07T10:29:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.350+0000 7f9dd979c700 1 -- 172.21.15.119:0/221609484 learned_addr learned my addr 172.21.15.119:0/221609484 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:29:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.351+0000 7f9dd979c700 1 -- 172.21.15.119:0/221609484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55684f77dad0 msgr2=0x55684f78ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.351+0000 7f9dd979c700 1 --2- 172.21.15.119:0/221609484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55684f77dad0 0x55684f78ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.351+0000 7f9dd979c700 1 -- 172.21.15.119:0/221609484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55684f782a60 msgr2=0x55684f78e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.351+0000 7f9dd979c700 1 --2- 172.21.15.119:0/221609484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55684f782a60 0x55684f78e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.351+0000 7f9dd979c700 1 -- 172.21.15.119:0/221609484 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55684f4fad80 con 0x55684f783380 2023-12-07T10:29:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.351+0000 7f9dd8f9b700 1 --2- 172.21.15.119:0/221609484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55684f77dad0 0x55684f78ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:29:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.351+0000 7f9dd979c700 1 --2- 172.21.15.119:0/221609484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f783380 0x55684f78be10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0099c0 tx=0x7f9dcc005ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51a0422b1a307bc1 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.351+0000 7f9db3fff700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:29:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.351+0000 7f9db3fff700 1 -- 172.21.15.119:0/221609484 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc02b040 con 0x55684f783380 2023-12-07T10:29:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.351+0000 7f9deeb6eb80 1 -- 172.21.15.119:0/221609484 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9dc809a7e0 con 0x55684f783380 2023-12-07T10:29:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.351+0000 7f9db3fff700 1 -- 172.21.15.119:0/221609484 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f9dcc003950 con 0x55684f783380 2023-12-07T10:29:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.352+0000 7f9db3fff700 1 -- 172.21.15.119:0/221609484 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9dcc003b10 con 0x55684f783380 2023-12-07T10:29:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.352+0000 7f9deeb6eb80 10 client.27542 fetch_fsmap learned FSMap version 57 2023-12-07T10:29:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.352+0000 7f9deeb6eb80 1 -- 172.21.15.119:0/221609484 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55684f5211f0 con 0x55684f783380 2023-12-07T10:29:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.352+0000 7f9db3fff700 1 -- 172.21.15.119:0/221609484 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dc809a7e0 con 0x55684f783380 2023-12-07T10:29:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.352+0000 7f9db3fff700 1 -- 172.21.15.119:0/221609484 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9dcc032450 con 0x55684f783380 2023-12-07T10:29:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.352+0000 7f9deeb6eb80 10 client.27542 fetch_fsmap finished waiting for FSMap version 57 2023-12-07T10:29:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.352+0000 7f9deeb6eb80 10 client.27542 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:29:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.352+0000 7f9deeb6eb80 20 client.27542 populate_metadata read hostname 'smithi119' 2023-12-07T10:29:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.352+0000 7f9deeb6eb80 1 --2- 172.21.15.119:0/221609484 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55684f7ed400 0x55684f7ef820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.352+0000 7f9deeb6eb80 4 client.27542 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:29:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.352+0000 7f9deeb6eb80 1 -- 172.21.15.119:0/221609484 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55684f697f70 con 0x55684f7ed400 2023-12-07T10:29:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.352+0000 7f9dd8f9b700 1 --2- 172.21.15.119:0/221609484 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55684f7ed400 0x55684f7ef820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.353+0000 7f9dd8f9b700 1 --2- 172.21.15.119:0/221609484 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55684f7ed400 0x55684f7ef820 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99001d694917e57b server_cookie=4e1be78bdb8ee930 in_seq=0 out_seq=0 2023-12-07T10:29:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.353+0000 7f9db3fff700 10 client.27542 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:29:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.428+0000 7f9db3fff700 1 -- 172.21.15.119:0/221609484 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55684f697f70 con 0x55684f7ed400 2023-12-07T10:29:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.428+0000 7f9db3fff700 10 client.27542 handle_command_reply: tid=0 2023-12-07T10:29:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.428+0000 7f9deeb6eb80 1 client.27542 shutdown 2023-12-07T10:29:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.428+0000 7f9deeb6eb80 1 -- 172.21.15.119:0/221609484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f783380 msgr2=0x55684f78be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.428+0000 7f9deeb6eb80 1 --2- 172.21.15.119:0/221609484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f783380 0x55684f78be10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0099c0 tx=0x7f9dcc005ad0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.428+0000 7f9deeb6eb80 1 -- 172.21.15.119:0/221609484 shutdown_connections 2023-12-07T10:29:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.428+0000 7f9deeb6eb80 1 --2- 172.21.15.119:0/221609484 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55684f7ed400 0x55684f7ef820 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.428+0000 7f9deeb6eb80 1 --2- 172.21.15.119:0/221609484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55684f783380 0x55684f78be10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.428+0000 7f9deeb6eb80 1 --2- 172.21.15.119:0/221609484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55684f77dad0 0x55684f78ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.428+0000 7f9deeb6eb80 1 --2- 172.21.15.119:0/221609484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55684f782a60 0x55684f78e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.428+0000 7f9deeb6eb80 1 -- 172.21.15.119:0/221609484 >> 172.21.15.119:0/221609484 conn(0x55684f6ac300 msgr2=0x55684f7809d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.428+0000 7f9db3fff700 0 client.27542 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:29:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.428+0000 7f9deeb6eb80 1 -- 172.21.15.119:0/221609484 shutdown_connections 2023-12-07T10:29:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.428+0000 7f9deeb6eb80 1 -- 172.21.15.119:0/221609484 wait complete. 2023-12-07T10:29:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.429+0000 7f9deeb6eb80 20 client.27542 trim_cache size 0 max 16384 2023-12-07T10:29:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:29:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:29:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:29:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:29:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.429+0000 7f9ddb1ff700 1 -- 172.21.15.119:0/3027764857 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9dc40a1c80 msgr2=0x7f9dc40a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.429+0000 7f9ddb1ff700 1 --2- 172.21.15.119:0/3027764857 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9dc40a1c80 0x7f9dc40a4140 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc005b40 tx=0x7f9dcc01a040 comp rx=0 tx=0).stop 2023-12-07T10:29:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.429+0000 7f9ddb1ff700 1 -- 172.21.15.119:0/3027764857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9dd4101520 msgr2=0x7f9dd419d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.429+0000 7f9ddb1ff700 1 --2- 172.21.15.119:0/3027764857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9dd4101520 0x7f9dd419d5a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9dc800c370 tx=0x7f9dc800c680 comp rx=0 tx=0).stop 2023-12-07T10:29:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.429+0000 7f9ddb1ff700 1 -- 172.21.15.119:0/3027764857 shutdown_connections 2023-12-07T10:29:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.429+0000 7f9ddb1ff700 1 --2- 172.21.15.119:0/3027764857 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9dc40a1c80 0x7f9dc40a4140 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.429+0000 7f9ddb1ff700 1 --2- 172.21.15.119:0/3027764857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9dd41075e0 0x7f9dd41976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.429+0000 7f9ddb1ff700 1 --2- 172.21.15.119:0/3027764857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9dd4101520 0x7f9dd419d5a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.429+0000 7f9ddb1ff700 1 --2- 172.21.15.119:0/3027764857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dd4101ee0 0x7f9dd419dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.429+0000 7f9ddb1ff700 1 -- 172.21.15.119:0/3027764857 >> 172.21.15.119:0/3027764857 conn(0x7f9dd40fd010 msgr2=0x7f9dd40fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.429+0000 7f9ddb1ff700 1 -- 172.21.15.119:0/3027764857 shutdown_connections 2023-12-07T10:29:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:31.430+0000 7f9ddb1ff700 1 -- 172.21.15.119:0/3027764857 wait complete. 2023-12-07T10:29:31.569 INFO:tasks.workunit.client.0.smithi119.stdout: 1 643859 7.19 MB/sec execute 344 sec latency 34.953 ms 2023-12-07T10:29:32.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:31 smithi119 ceph-mon[97908]: pgmap v282: 129 pgs: 129 active+clean; 435 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 259 op/s 2023-12-07T10:29:32.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:31 smithi159 ceph-mon[134726]: pgmap v282: 129 pgs: 129 active+clean; 435 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 259 op/s 2023-12-07T10:29:32.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:31 smithi153 ceph-mon[133625]: pgmap v282: 129 pgs: 129 active+clean; 435 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 259 op/s 2023-12-07T10:29:32.569 INFO:tasks.workunit.client.0.smithi119.stdout: 1 645821 7.20 MB/sec execute 345 sec latency 15.648 ms 2023-12-07T10:29:33.569 INFO:tasks.workunit.client.0.smithi119.stdout: 1 647071 7.20 MB/sec execute 346 sec latency 20.780 ms 2023-12-07T10:29:34.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:34 smithi119 ceph-mon[97908]: pgmap v283: 129 pgs: 129 active+clean; 436 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2023-12-07T10:29:34.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:34 smithi159 ceph-mon[134726]: pgmap v283: 129 pgs: 129 active+clean; 436 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2023-12-07T10:29:34.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:34 smithi153 ceph-mon[133625]: pgmap v283: 129 pgs: 129 active+clean; 436 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2023-12-07T10:29:34.569 INFO:tasks.workunit.client.0.smithi119.stdout: 1 648842 7.20 MB/sec execute 347 sec latency 17.391 ms 2023-12-07T10:29:35.569 INFO:tasks.workunit.client.0.smithi119.stdout: 1 650337 7.21 MB/sec execute 348 sec latency 13.098 ms 2023-12-07T10:29:36.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:36 smithi119 ceph-mon[97908]: pgmap v284: 129 pgs: 129 active+clean; 436 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 180 op/s 2023-12-07T10:29:36.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:36 smithi159 ceph-mon[134726]: pgmap v284: 129 pgs: 129 active+clean; 436 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 180 op/s 2023-12-07T10:29:36.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:36 smithi153 ceph-mon[133625]: pgmap v284: 129 pgs: 129 active+clean; 436 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 180 op/s 2023-12-07T10:29:36.569 INFO:tasks.workunit.client.0.smithi119.stdout: 1 651870 7.20 MB/sec execute 349 sec latency 20.016 ms 2023-12-07T10:29:37.569 INFO:tasks.workunit.client.0.smithi119.stdout: 1 653701 7.22 MB/sec execute 350 sec latency 13.831 ms 2023-12-07T10:29:38.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:38 smithi119 ceph-mon[97908]: pgmap v285: 129 pgs: 129 active+clean; 445 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 301 op/s 2023-12-07T10:29:38.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:38 smithi159 ceph-mon[134726]: pgmap v285: 129 pgs: 129 active+clean; 445 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 301 op/s 2023-12-07T10:29:38.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:38 smithi153 ceph-mon[133625]: pgmap v285: 129 pgs: 129 active+clean; 445 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 301 op/s 2023-12-07T10:29:38.569 INFO:tasks.workunit.client.0.smithi119.stdout: 1 654946 7.22 MB/sec execute 351 sec latency 11.944 ms 2023-12-07T10:29:39.570 INFO:tasks.workunit.client.0.smithi119.stdout: 1 656927 7.23 MB/sec execute 352 sec latency 16.945 ms 2023-12-07T10:29:40.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:40 smithi119 ceph-mon[97908]: pgmap v286: 129 pgs: 129 active+clean; 445 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 211 op/s 2023-12-07T10:29:40.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:40 smithi159 ceph-mon[134726]: pgmap v286: 129 pgs: 129 active+clean; 445 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 211 op/s 2023-12-07T10:29:40.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:40 smithi153 ceph-mon[133625]: pgmap v286: 129 pgs: 129 active+clean; 445 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 211 op/s 2023-12-07T10:29:40.570 INFO:tasks.workunit.client.0.smithi119.stdout: 1 657722 7.22 MB/sec execute 353 sec latency 156.258 ms 2023-12-07T10:29:41.570 INFO:tasks.workunit.client.0.smithi119.stdout: 1 658632 7.21 MB/sec execute 354 sec latency 267.732 ms 2023-12-07T10:29:42.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:41 smithi119 ceph-mon[97908]: pgmap v287: 129 pgs: 129 active+clean; 448 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 236 op/s 2023-12-07T10:29:42.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:41 smithi159 ceph-mon[134726]: pgmap v287: 129 pgs: 129 active+clean; 448 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 236 op/s 2023-12-07T10:29:42.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:41 smithi153 ceph-mon[133625]: pgmap v287: 129 pgs: 129 active+clean; 448 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 236 op/s 2023-12-07T10:29:42.570 INFO:tasks.workunit.client.0.smithi119.stdout: 1 660155 7.21 MB/sec execute 355 sec latency 42.093 ms 2023-12-07T10:29:43.570 INFO:tasks.workunit.client.0.smithi119.stdout: 1 661218 7.21 MB/sec execute 356 sec latency 16.953 ms 2023-12-07T10:29:44.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:44 smithi119 ceph-mon[97908]: pgmap v288: 129 pgs: 129 active+clean; 459 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 233 op/s 2023-12-07T10:29:44.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:44 smithi159 ceph-mon[134726]: pgmap v288: 129 pgs: 129 active+clean; 459 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 233 op/s 2023-12-07T10:29:44.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:44 smithi153 ceph-mon[133625]: pgmap v288: 129 pgs: 129 active+clean; 459 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 233 op/s 2023-12-07T10:29:44.570 INFO:tasks.workunit.client.0.smithi119.stdout: 1 662939 7.21 MB/sec execute 357 sec latency 21.845 ms 2023-12-07T10:29:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.597+0000 7f7e855ff700 1 -- 172.21.15.119:0/2648970302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e801040d0 msgr2=0x7f7e801105e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.597+0000 7f7e855ff700 1 --2- 172.21.15.119:0/2648970302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e801040d0 0x7f7e801105e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800b0a0 tx=0x7f7e7800b3b0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.597+0000 7f7e855ff700 1 -- 172.21.15.119:0/2648970302 shutdown_connections 2023-12-07T10:29:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.597+0000 7f7e855ff700 1 --2- 172.21.15.119:0/2648970302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e801040d0 0x7f7e801105e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.597+0000 7f7e855ff700 1 --2- 172.21.15.119:0/2648970302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e80109720 0x7f7e80109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.597+0000 7f7e855ff700 1 --2- 172.21.15.119:0/2648970302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e80103710 0x7f7e80103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.597+0000 7f7e855ff700 1 -- 172.21.15.119:0/2648970302 >> 172.21.15.119:0/2648970302 conn(0x7f7e800ff1c0 msgr2=0x7f7e801015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.597+0000 7f7e855ff700 1 -- 172.21.15.119:0/2648970302 shutdown_connections 2023-12-07T10:29:44.599 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.597+0000 7f7e855ff700 1 -- 172.21.15.119:0/2648970302 wait complete. 2023-12-07T10:29:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.598+0000 7f7e855ff700 1 Processor -- start 2023-12-07T10:29:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.599+0000 7f7e855ff700 1 -- start start 2023-12-07T10:29:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.599+0000 7f7e855ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e80103710 0x7f7e801103a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.599+0000 7f7e855ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e801040d0 0x7f7e8010e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.599+0000 7f7e855ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e80109720 0x7f7e8010ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.599+0000 7f7e855ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8010f1a0 con 0x7f7e801040d0 2023-12-07T10:29:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.599+0000 7f7e855ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8010f310 con 0x7f7e80103710 2023-12-07T10:29:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.599+0000 7f7e855ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e801aa3e0 con 0x7f7e80109720 2023-12-07T10:29:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.599+0000 7f7e7effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e80103710 0x7f7e801103a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.599+0000 7f7e7effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e80103710 0x7f7e801103a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:46742/0 (socket says 172.21.15.119:46742) 2023-12-07T10:29:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.599+0000 7f7e7effd700 1 -- 172.21.15.119:0/2183075670 learned_addr learned my addr 172.21.15.119:0/2183075670 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:29:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.599+0000 7f7e7e7fc700 1 --2- 172.21.15.119:0/2183075670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e801040d0 0x7f7e8010e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.599+0000 7f7e7f7fe700 1 --2- 172.21.15.119:0/2183075670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e80109720 0x7f7e8010ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.600+0000 7f7e7effd700 1 -- 172.21.15.119:0/2183075670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e80109720 msgr2=0x7f7e8010ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.600+0000 7f7e7effd700 1 --2- 172.21.15.119:0/2183075670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e80109720 0x7f7e8010ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.600+0000 7f7e7effd700 1 -- 172.21.15.119:0/2183075670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e801040d0 msgr2=0x7f7e8010e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.600+0000 7f7e7effd700 1 --2- 172.21.15.119:0/2183075670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e801040d0 0x7f7e8010e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.600+0000 7f7e7effd700 1 -- 172.21.15.119:0/2183075670 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e78009d00 con 0x7f7e80103710 2023-12-07T10:29:44.601 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.600+0000 7f7e7effd700 1 --2- 172.21.15.119:0/2183075670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e80103710 0x7f7e801103a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000e9a0 tx=0x7f7e7000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.600+0000 7f7e67fff700 1 -- 172.21.15.119:0/2183075670 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e70009790 con 0x7f7e80103710 2023-12-07T10:29:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.600+0000 7f7e67fff700 1 -- 172.21.15.119:0/2183075670 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e70010430 con 0x7f7e80103710 2023-12-07T10:29:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.600+0000 7f7e67fff700 1 -- 172.21.15.119:0/2183075670 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e70018680 con 0x7f7e80103710 2023-12-07T10:29:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.600+0000 7f7e855ff700 1 -- 172.21.15.119:0/2183075670 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e801aa6c0 con 0x7f7e80103710 2023-12-07T10:29:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.600+0000 7f7e855ff700 1 -- 172.21.15.119:0/2183075670 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e801aabd0 con 0x7f7e80103710 2023-12-07T10:29:44.604 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.604+0000 7f7e67fff700 1 -- 172.21.15.119:0/2183075670 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f7e700105a0 con 0x7f7e80103710 2023-12-07T10:29:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.606+0000 7f7e98f87b80 1 Processor -- start 2023-12-07T10:29:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.606+0000 7f7e67fff700 1 --2- 172.21.15.119:0/2183075670 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f7e680a1d00 0x7f7e680a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.606+0000 7f7e98f87b80 1 -- start start 2023-12-07T10:29:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.606+0000 7f7e67fff700 1 -- 172.21.15.119:0/2183075670 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f7e70014070 con 0x7f7e80103710 2023-12-07T10:29:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.606+0000 7f7e7f7fe700 1 --2- 172.21.15.119:0/2183075670 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f7e680a1d00 0x7f7e680a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.606+0000 7f7e98f87b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594d0160a60 0x5594d0160e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.606+0000 7f7e98f87b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d0161380 0x5594d015b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.606+0000 7f7e98f87b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594d015bad0 0x5594d015bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.606+0000 7f7e98f87b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594d00cf760 con 0x5594d015bad0 2023-12-07T10:29:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.606+0000 7f7e98f87b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594d00b6740 con 0x5594d0160a60 2023-12-07T10:29:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.606+0000 7f7e98f87b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594d00b4060 con 0x5594d0161380 2023-12-07T10:29:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.607+0000 7f7e7effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594d0160a60 0x5594d0160e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.607+0000 7f7e7e7fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d0161380 0x5594d015b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.607+0000 7f7e7effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594d0160a60 0x5594d0160e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:46756/0 (socket says 172.21.15.119:46756) 2023-12-07T10:29:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.607+0000 7f7e7e7fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d0161380 0x5594d015b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41842/0 (socket says 172.21.15.119:41842) 2023-12-07T10:29:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.607+0000 7f7e7effd700 1 -- 172.21.15.119:0/2048503213 learned_addr learned my addr 172.21.15.119:0/2048503213 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:29:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.607+0000 7f7e7effd700 1 -- 172.21.15.119:0/2048503213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d0161380 msgr2=0x5594d015b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.607+0000 7f7e7effd700 1 --2- 172.21.15.119:0/2048503213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d0161380 0x5594d015b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.607+0000 7f7e7effd700 1 -- 172.21.15.119:0/2048503213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594d015bad0 msgr2=0x5594d015bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.607+0000 7f7e7f7fe700 1 --2- 172.21.15.119:0/2048503213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594d015bad0 0x5594d015bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.607+0000 7f7e7effd700 1 --2- 172.21.15.119:0/2048503213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594d015bad0 0x5594d015bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.607+0000 7f7e7effd700 1 -- 172.21.15.119:0/2048503213 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594cff2c5d0 con 0x5594d0160a60 2023-12-07T10:29:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.607+0000 7f7e7e7fc700 1 --2- 172.21.15.119:0/2048503213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d0161380 0x5594d015b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:29:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.607+0000 7f7e7f7fe700 1 --2- 172.21.15.119:0/2048503213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594d015bad0 0x5594d015bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:29:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.607+0000 7f7e7f7fe700 1 --2- 172.21.15.119:0/2183075670 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f7e680a1d00 0x7f7e680a41c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009610 tx=0x7f7e78007ce0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.607+0000 7f7e7effd700 1 --2- 172.21.15.119:0/2048503213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594d0160a60 0x5594d0160e40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7e70096d40 tx=0x7f7e7009b010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8051e2a37ea73b48 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.607+0000 7f7e657fa700 1 -- 172.21.15.119:0/2048503213 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e700a2bd0 con 0x5594d0160a60 2023-12-07T10:29:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.607+0000 7f7e657fa700 1 -- 172.21.15.119:0/2048503213 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e700ba0b0 con 0x5594d0160a60 2023-12-07T10:29:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.607+0000 7f7e657fa700 1 -- 172.21.15.119:0/2048503213 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7009aaa0 con 0x5594d0160a60 2023-12-07T10:29:44.613 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.608+0000 7f7e98f87b80 1 -- 172.21.15.119:0/2048503213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594d0160a60 msgr2=0x5594d0160e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:44.613 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.608+0000 7f7e98f87b80 1 --2- 172.21.15.119:0/2048503213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594d0160a60 0x5594d0160e40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7e70096d40 tx=0x7f7e7009b010 comp rx=0 tx=0).stop 2023-12-07T10:29:44.613 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.608+0000 7f7e98f87b80 1 -- 172.21.15.119:0/2048503213 shutdown_connections 2023-12-07T10:29:44.613 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.608+0000 7f7e98f87b80 1 --2- 172.21.15.119:0/2048503213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594d015bad0 0x5594d015bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.613 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.608+0000 7f7e98f87b80 1 --2- 172.21.15.119:0/2048503213 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594d0160a60 0x5594d0160e40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.613 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.608+0000 7f7e98f87b80 1 --2- 172.21.15.119:0/2048503213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d0161380 0x5594d015b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.613 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.608+0000 7f7e98f87b80 1 -- 172.21.15.119:0/2048503213 >> 172.21.15.119:0/2048503213 conn(0x5594d008a300 msgr2=0x5594d0155b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:44.613 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.608+0000 7f7e98f87b80 1 -- 172.21.15.119:0/2048503213 shutdown_connections 2023-12-07T10:29:44.613 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.609+0000 7f7e98f87b80 1 -- 172.21.15.119:0/2048503213 wait complete. 2023-12-07T10:29:44.613 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.609+0000 7f7e98f87b80 1 -- start start 2023-12-07T10:29:44.613 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.609+0000 7f7e98f87b80 1 Processor -- start 2023-12-07T10:29:44.613 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.610+0000 7f7e98f87b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594d015bad0 0x5594d01647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:44.613 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.610+0000 7f7e98f87b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d0160a60 0x5594d0164d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:44.614 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.610+0000 7f7e98f87b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594d0161380 0x5594d0165270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:44.614 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.610+0000 7f7e98f87b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594d00af950 con 0x5594d0161380 2023-12-07T10:29:44.614 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.610+0000 7f7e98f87b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594d0069500 con 0x5594d015bad0 2023-12-07T10:29:44.614 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.610+0000 7f7e98f87b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594d00691a0 con 0x5594d0160a60 2023-12-07T10:29:44.614 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.610+0000 7f7e98f87b80 10 client.0 _ll_register_callbacks cb 0x5594d00508e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:29:44.614 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.610+0000 7f7e7e7fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d0160a60 0x5594d0164d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:44.614 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.610+0000 7f7e7e7fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d0160a60 0x5594d0164d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41848/0 (socket says 172.21.15.119:41848) 2023-12-07T10:29:44.614 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.610+0000 7f7e7e7fc700 1 -- 172.21.15.119:0/4219305514 learned_addr learned my addr 172.21.15.119:0/4219305514 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:29:44.614 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.610+0000 7f7e7effd700 1 --2- 172.21.15.119:0/4219305514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594d015bad0 0x5594d01647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:44.614 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.610+0000 7f7e7f7fe700 1 --2- 172.21.15.119:0/4219305514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594d0161380 0x5594d0165270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:44.614 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.611+0000 7f7e7e7fc700 1 -- 172.21.15.119:0/4219305514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594d015bad0 msgr2=0x5594d01647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:44.614 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.611+0000 7f7e7e7fc700 1 --2- 172.21.15.119:0/4219305514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594d015bad0 0x5594d01647f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.614 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.611+0000 7f7e7e7fc700 1 -- 172.21.15.119:0/4219305514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594d0161380 msgr2=0x5594d0165270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:44.614 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.611+0000 7f7e7e7fc700 1 --2- 172.21.15.119:0/4219305514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594d0161380 0x5594d0165270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.615 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.611+0000 7f7e7e7fc700 1 -- 172.21.15.119:0/4219305514 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e740098e0 con 0x5594d0160a60 2023-12-07T10:29:44.615 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.611+0000 7f7e7e7fc700 1 --2- 172.21.15.119:0/4219305514 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d0160a60 0x5594d0164d30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7e74012820 tx=0x7f7e74012b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c77656ed36c8488 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:44.615 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.611+0000 7f7e62ff5700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:29:44.615 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.611+0000 7f7e62ff5700 1 -- 172.21.15.119:0/4219305514 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e74017070 con 0x5594d0160a60 2023-12-07T10:29:44.615 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.611+0000 7f7e62ff5700 1 -- 172.21.15.119:0/4219305514 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f7e7400f540 con 0x5594d0160a60 2023-12-07T10:29:44.615 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.611+0000 7f7e98f87b80 1 -- 172.21.15.119:0/4219305514 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5594d014e6f0 con 0x5594d0160a60 2023-12-07T10:29:44.615 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.612+0000 7f7e62ff5700 1 -- 172.21.15.119:0/4219305514 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e7400f700 con 0x5594d0160a60 2023-12-07T10:29:44.615 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.612+0000 7f7e98f87b80 10 client.27538 fetch_fsmap learned FSMap version 57 2023-12-07T10:29:44.615 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.612+0000 7f7e98f87b80 1 -- 172.21.15.119:0/4219305514 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x7f7e801aabd0 con 0x5594d0160a60 2023-12-07T10:29:44.615 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.612+0000 7f7e62ff5700 1 -- 172.21.15.119:0/4219305514 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5594d014e6f0 con 0x5594d0160a60 2023-12-07T10:29:44.615 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.612+0000 7f7e62ff5700 1 -- 172.21.15.119:0/4219305514 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f7e74010a00 con 0x5594d0160a60 2023-12-07T10:29:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.612+0000 7f7e98f87b80 10 client.27538 fetch_fsmap finished waiting for FSMap version 57 2023-12-07T10:29:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.612+0000 7f7e98f87b80 10 client.27538 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:29:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.612+0000 7f7e98f87b80 20 client.27538 populate_metadata read hostname 'smithi119' 2023-12-07T10:29:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.612+0000 7f7e98f87b80 1 --2- 172.21.15.119:0/4219305514 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5594d0175000 0x5594d0177420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.612+0000 7f7e98f87b80 4 client.27538 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:29:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.612+0000 7f7e98f87b80 1 -- 172.21.15.119:0/4219305514 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5594d012a740 con 0x5594d0175000 2023-12-07T10:29:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.613+0000 7f7e7effd700 1 --2- 172.21.15.119:0/4219305514 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5594d0175000 0x5594d0177420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:44.617 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.613+0000 7f7e7effd700 1 --2- 172.21.15.119:0/4219305514 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5594d0175000 0x5594d0177420 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cfaa3d0224871be server_cookie=76c1b2647248c09c in_seq=0 out_seq=0 2023-12-07T10:29:44.617 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.613+0000 7f7e62ff5700 10 client.27538 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:29:44.617 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.614+0000 7f7e62ff5700 1 -- 172.21.15.119:0/4219305514 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5594d012a740 con 0x5594d0175000 2023-12-07T10:29:44.617 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.614+0000 7f7e62ff5700 10 client.27538 handle_command_reply: tid=0 2023-12-07T10:29:44.618 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.614+0000 7f7e98f87b80 1 client.27538 shutdown 2023-12-07T10:29:44.618 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.614+0000 7f7e98f87b80 1 -- 172.21.15.119:0/4219305514 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d0160a60 msgr2=0x5594d0164d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:44.618 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.614+0000 7f7e98f87b80 1 --2- 172.21.15.119:0/4219305514 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d0160a60 0x5594d0164d30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7e74012820 tx=0x7f7e74012b30 comp rx=0 tx=0).stop 2023-12-07T10:29:44.618 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.614+0000 7f7e98f87b80 1 -- 172.21.15.119:0/4219305514 shutdown_connections 2023-12-07T10:29:44.618 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.614+0000 7f7e98f87b80 1 --2- 172.21.15.119:0/4219305514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594d0161380 0x5594d0165270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.618 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.614+0000 7f7e98f87b80 1 --2- 172.21.15.119:0/4219305514 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5594d0175000 0x5594d0177420 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.618 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.614+0000 7f7e98f87b80 1 --2- 172.21.15.119:0/4219305514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594d015bad0 0x5594d01647f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.618 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.614+0000 7f7e98f87b80 1 --2- 172.21.15.119:0/4219305514 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d0160a60 0x5594d0164d30 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.618 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.614+0000 7f7e98f87b80 1 -- 172.21.15.119:0/4219305514 >> 172.21.15.119:0/4219305514 conn(0x5594d008a300 msgr2=0x5594d0150980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:44.618 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.614+0000 7f7e62ff5700 0 client.27538 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:29:44.618 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.614+0000 7f7e98f87b80 1 -- 172.21.15.119:0/4219305514 shutdown_connections 2023-12-07T10:29:44.618 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.614+0000 7f7e98f87b80 1 -- 172.21.15.119:0/4219305514 wait complete. 2023-12-07T10:29:44.618 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.615+0000 7f7e98f87b80 20 client.27538 trim_cache size 0 max 16384 2023-12-07T10:29:44.632 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.631+0000 7f7e98f87b80 1 Processor -- start 2023-12-07T10:29:44.632 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.631+0000 7f7e98f87b80 1 -- start start 2023-12-07T10:29:44.632 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.631+0000 7f7e98f87b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d015bad0 0x5594d015b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:44.633 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.631+0000 7f7e98f87b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594d0160a60 0x5594d015df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:44.633 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.631+0000 7f7e98f87b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594d0161380 0x5594d015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:44.633 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.631+0000 7f7e98f87b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594cff2e8c0 con 0x5594d0161380 2023-12-07T10:29:44.633 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.631+0000 7f7e98f87b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594d0167770 con 0x5594d0160a60 2023-12-07T10:29:44.633 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.631+0000 7f7e98f87b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594d0171a30 con 0x5594d015bad0 2023-12-07T10:29:44.633 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.632+0000 7f7e7effd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d015bad0 0x5594d015b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:44.633 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.632+0000 7f7e7effd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d015bad0 0x5594d015b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41856/0 (socket says 172.21.15.119:41856) 2023-12-07T10:29:44.633 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.632+0000 7f7e7effd700 1 -- 172.21.15.119:0/2792604275 learned_addr learned my addr 172.21.15.119:0/2792604275 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:29:44.633 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.632+0000 7f7e7f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594d0161380 0x5594d015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:44.633 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.632+0000 7f7e7e7fc700 1 --2- 172.21.15.119:0/2792604275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594d0160a60 0x5594d015df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:44.633 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.632+0000 7f7e7effd700 1 -- 172.21.15.119:0/2792604275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594d0160a60 msgr2=0x5594d015df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:44.633 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.632+0000 7f7e7effd700 1 --2- 172.21.15.119:0/2792604275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594d0160a60 0x5594d015df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.633 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.632+0000 7f7e7effd700 1 -- 172.21.15.119:0/2792604275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594d0161380 msgr2=0x5594d015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:44.633 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.632+0000 7f7e7effd700 1 --2- 172.21.15.119:0/2792604275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594d0161380 0x5594d015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.634 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.632+0000 7f7e7effd700 1 -- 172.21.15.119:0/2792604275 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594cfed8d80 con 0x5594d015bad0 2023-12-07T10:29:44.634 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.632+0000 7f7e7effd700 1 --2- 172.21.15.119:0/2792604275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d015bad0 0x5594d015b540 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7e7009a7d0 tx=0x7f7e7009eb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91dc371359d9474c server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:44.636 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.632+0000 7f7e65ffb700 1 -- 172.21.15.119:0/2792604275 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7009a610 con 0x5594d015bad0 2023-12-07T10:29:44.636 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.633+0000 7f7e65ffb700 1 -- 172.21.15.119:0/2792604275 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e700b3120 con 0x5594d015bad0 2023-12-07T10:29:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.633+0000 7f7e65ffb700 1 -- 172.21.15.119:0/2792604275 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e70099d30 con 0x5594d015bad0 2023-12-07T10:29:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.633+0000 7f7e98f87b80 1 -- 172.21.15.119:0/2792604275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d015bad0 msgr2=0x5594d015b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.633+0000 7f7e98f87b80 1 --2- 172.21.15.119:0/2792604275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d015bad0 0x5594d015b540 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7e7009a7d0 tx=0x7f7e7009eb10 comp rx=0 tx=0).stop 2023-12-07T10:29:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.633+0000 7f7e98f87b80 1 -- 172.21.15.119:0/2792604275 shutdown_connections 2023-12-07T10:29:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.633+0000 7f7e98f87b80 1 --2- 172.21.15.119:0/2792604275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594d0161380 0x5594d015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.633+0000 7f7e98f87b80 1 --2- 172.21.15.119:0/2792604275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d015bad0 0x5594d015b540 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.633+0000 7f7e98f87b80 1 --2- 172.21.15.119:0/2792604275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594d0160a60 0x5594d015df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.633+0000 7f7e98f87b80 1 -- 172.21.15.119:0/2792604275 >> 172.21.15.119:0/2792604275 conn(0x5594d008a300 msgr2=0x5594d0155b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.633+0000 7f7e98f87b80 1 -- 172.21.15.119:0/2792604275 shutdown_connections 2023-12-07T10:29:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.633+0000 7f7e98f87b80 1 -- 172.21.15.119:0/2792604275 wait complete. 2023-12-07T10:29:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.634+0000 7f7e98f87b80 1 -- start start 2023-12-07T10:29:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.634+0000 7f7e98f87b80 1 Processor -- start 2023-12-07T10:29:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.635+0000 7f7e98f87b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d015bad0 0x5594d0167e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.635+0000 7f7e98f87b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594d0160a60 0x5594d0168340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.635+0000 7f7e98f87b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594d0161380 0x5594d01ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.635+0000 7f7e98f87b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594d0168b20 con 0x5594d0160a60 2023-12-07T10:29:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.635+0000 7f7e98f87b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594d0168c90 con 0x5594d0161380 2023-12-07T10:29:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.635+0000 7f7e98f87b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594d01caa40 con 0x5594d015bad0 2023-12-07T10:29:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.635+0000 7f7e98f87b80 10 client.0 _ll_register_callbacks cb 0x5594d00508e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:29:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.635+0000 7f7e7e7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594d0160a60 0x5594d0168340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.635+0000 7f7e7effd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d015bad0 0x5594d0167e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.635+0000 7f7e7e7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594d0160a60 0x5594d0168340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46420/0 (socket says 172.21.15.119:46420) 2023-12-07T10:29:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.635+0000 7f7e7e7fc700 1 -- 172.21.15.119:0/3675695784 learned_addr learned my addr 172.21.15.119:0/3675695784 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:29:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.635+0000 7f7e7f7fe700 1 --2- 172.21.15.119:0/3675695784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594d0161380 0x5594d01ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.635+0000 7f7e7effd700 1 -- 172.21.15.119:0/3675695784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594d0161380 msgr2=0x5594d01ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.635+0000 7f7e7effd700 1 --2- 172.21.15.119:0/3675695784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594d0161380 0x5594d01ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.635+0000 7f7e7effd700 1 -- 172.21.15.119:0/3675695784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594d0160a60 msgr2=0x5594d0168340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.635+0000 7f7e7effd700 1 --2- 172.21.15.119:0/3675695784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594d0160a60 0x5594d0168340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.635+0000 7f7e7effd700 1 -- 172.21.15.119:0/3675695784 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594cff2c5d0 con 0x5594d015bad0 2023-12-07T10:29:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.636+0000 7f7e7effd700 1 --2- 172.21.15.119:0/3675695784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d015bad0 0x5594d0167e00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000e970 tx=0x7f7e70010bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e96e9b9f939f5c2f server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.636+0000 7f7e62ff5700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:29:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.636+0000 7f7e62ff5700 1 -- 172.21.15.119:0/3675695784 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7009a610 con 0x5594d015bad0 2023-12-07T10:29:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.636+0000 7f7e62ff5700 1 -- 172.21.15.119:0/3675695784 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f7e700a2bd0 con 0x5594d015bad0 2023-12-07T10:29:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.636+0000 7f7e98f87b80 1 -- 172.21.15.119:0/3675695784 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5594d0075f70 con 0x5594d015bad0 2023-12-07T10:29:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.636+0000 7f7e62ff5700 1 -- 172.21.15.119:0/3675695784 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e70083be0 con 0x5594d015bad0 2023-12-07T10:29:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.636+0000 7f7e98f87b80 10 client.27544 fetch_fsmap learned FSMap version 57 2023-12-07T10:29:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.636+0000 7f7e98f87b80 1 -- 172.21.15.119:0/3675695784 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x5594cfec1c00 con 0x5594d015bad0 2023-12-07T10:29:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.636+0000 7f7e62ff5700 1 -- 172.21.15.119:0/3675695784 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5594d0075f70 con 0x5594d015bad0 2023-12-07T10:29:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.636+0000 7f7e62ff5700 1 -- 172.21.15.119:0/3675695784 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f7e70099d30 con 0x5594d015bad0 2023-12-07T10:29:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.637+0000 7f7e98f87b80 10 client.27544 fetch_fsmap finished waiting for FSMap version 57 2023-12-07T10:29:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.637+0000 7f7e98f87b80 10 client.27544 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:29:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.637+0000 7f7e98f87b80 20 client.27544 populate_metadata read hostname 'smithi119' 2023-12-07T10:29:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.637+0000 7f7e98f87b80 1 --2- 172.21.15.119:0/3675695784 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5594d01cbdf0 0x5594d01ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.637+0000 7f7e98f87b80 4 client.27544 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:29:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.637+0000 7f7e98f87b80 1 -- 172.21.15.119:0/3675695784 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5594d0068430 con 0x5594d01cbdf0 2023-12-07T10:29:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.637+0000 7f7e7e7fc700 1 --2- 172.21.15.119:0/3675695784 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5594d01cbdf0 0x5594d01ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.638+0000 7f7e7e7fc700 1 --2- 172.21.15.119:0/3675695784 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5594d01cbdf0 0x5594d01ce1e0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9df2308516d8c4a6 server_cookie=4f6a1c6541353c40 in_seq=0 out_seq=0 2023-12-07T10:29:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.638+0000 7f7e62ff5700 10 client.27544 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:29:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.677+0000 7f7e62ff5700 1 -- 172.21.15.119:0/3675695784 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5594d0068430 con 0x5594d01cbdf0 2023-12-07T10:29:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.677+0000 7f7e62ff5700 10 client.27544 handle_command_reply: tid=0 2023-12-07T10:29:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.677+0000 7f7e98f87b80 1 client.27544 shutdown 2023-12-07T10:29:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.677+0000 7f7e98f87b80 1 -- 172.21.15.119:0/3675695784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d015bad0 msgr2=0x5594d0167e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.677+0000 7f7e98f87b80 1 --2- 172.21.15.119:0/3675695784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d015bad0 0x5594d0167e00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000e970 tx=0x7f7e70010bb0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.677+0000 7f7e98f87b80 1 -- 172.21.15.119:0/3675695784 shutdown_connections 2023-12-07T10:29:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.677+0000 7f7e98f87b80 1 --2- 172.21.15.119:0/3675695784 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5594d01cbdf0 0x5594d01ce1e0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.677+0000 7f7e98f87b80 1 --2- 172.21.15.119:0/3675695784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594d0161380 0x5594d01ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.677+0000 7f7e98f87b80 1 --2- 172.21.15.119:0/3675695784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d015bad0 0x5594d0167e00 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.677+0000 7f7e98f87b80 1 --2- 172.21.15.119:0/3675695784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5594d0160a60 0x5594d0168340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.677+0000 7f7e62ff5700 0 client.27544 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:29:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.677+0000 7f7e98f87b80 1 -- 172.21.15.119:0/3675695784 >> 172.21.15.119:0/3675695784 conn(0x5594d008a300 msgr2=0x5594d015ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.678+0000 7f7e98f87b80 1 -- 172.21.15.119:0/3675695784 shutdown_connections 2023-12-07T10:29:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.678+0000 7f7e98f87b80 1 -- 172.21.15.119:0/3675695784 wait complete. 2023-12-07T10:29:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.678+0000 7f7e98f87b80 20 client.27544 trim_cache size 0 max 16384 2023-12-07T10:29:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:29:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:29:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:29:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:29:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.678+0000 7f7e855ff700 1 -- 172.21.15.119:0/2183075670 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f7e680a1d00 msgr2=0x7f7e680a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.678+0000 7f7e855ff700 1 --2- 172.21.15.119:0/2183075670 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f7e680a1d00 0x7f7e680a41c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009610 tx=0x7f7e78007ce0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.678+0000 7f7e855ff700 1 -- 172.21.15.119:0/2183075670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e80103710 msgr2=0x7f7e801103a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.678+0000 7f7e855ff700 1 --2- 172.21.15.119:0/2183075670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e80103710 0x7f7e801103a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000e9a0 tx=0x7f7e7000ecb0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.679+0000 7f7e855ff700 1 -- 172.21.15.119:0/2183075670 shutdown_connections 2023-12-07T10:29:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.679+0000 7f7e855ff700 1 --2- 172.21.15.119:0/2183075670 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f7e680a1d00 0x7f7e680a41c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.679+0000 7f7e855ff700 1 --2- 172.21.15.119:0/2183075670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e80109720 0x7f7e8010ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.679+0000 7f7e855ff700 1 --2- 172.21.15.119:0/2183075670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7e80103710 0x7f7e801103a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.679+0000 7f7e855ff700 1 --2- 172.21.15.119:0/2183075670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e801040d0 0x7f7e8010e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.679+0000 7f7e855ff700 1 -- 172.21.15.119:0/2183075670 >> 172.21.15.119:0/2183075670 conn(0x7f7e800ff1c0 msgr2=0x7f7e800ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.679+0000 7f7e855ff700 1 -- 172.21.15.119:0/2183075670 shutdown_connections 2023-12-07T10:29:44.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:44.679+0000 7f7e855ff700 1 -- 172.21.15.119:0/2183075670 wait complete. 2023-12-07T10:29:45.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:45 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:29:45.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:45 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:29:45.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:45 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:29:45.570 INFO:tasks.workunit.client.0.smithi119.stdout: 1 664360 7.22 MB/sec execute 358 sec latency 22.437 ms 2023-12-07T10:29:46.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:46 smithi119 ceph-mon[97908]: pgmap v289: 129 pgs: 129 active+clean; 459 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 211 op/s 2023-12-07T10:29:46.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:46 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:29:46.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:46 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:29:46.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:46 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:29:46.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:46 smithi159 ceph-mon[134726]: pgmap v289: 129 pgs: 129 active+clean; 459 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 211 op/s 2023-12-07T10:29:46.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:46 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:29:46.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:46 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:29:46.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:46 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:29:46.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:46 smithi153 ceph-mon[133625]: pgmap v289: 129 pgs: 129 active+clean; 459 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 211 op/s 2023-12-07T10:29:46.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:46 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:29:46.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:46 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:29:46.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:46 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:29:46.570 INFO:tasks.workunit.client.0.smithi119.stdout: 1 665132 7.21 MB/sec execute 359 sec latency 105.990 ms 2023-12-07T10:29:47.570 INFO:tasks.workunit.client.0.smithi119.stdout: 1 666834 7.21 MB/sec execute 360 sec latency 33.046 ms 2023-12-07T10:29:48.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:48 smithi119 ceph-mon[97908]: pgmap v290: 129 pgs: 129 active+clean; 458 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 373 op/s 2023-12-07T10:29:48.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:48 smithi159 ceph-mon[134726]: pgmap v290: 129 pgs: 129 active+clean; 458 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 373 op/s 2023-12-07T10:29:48.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:48 smithi153 ceph-mon[133625]: pgmap v290: 129 pgs: 129 active+clean; 458 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 373 op/s 2023-12-07T10:29:48.570 INFO:tasks.workunit.client.0.smithi119.stdout: 1 668096 7.21 MB/sec execute 361 sec latency 15.043 ms 2023-12-07T10:29:49.570 INFO:tasks.workunit.client.0.smithi119.stdout: 1 669602 7.21 MB/sec execute 362 sec latency 24.156 ms 2023-12-07T10:29:50.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:50 smithi119 ceph-mon[97908]: pgmap v291: 129 pgs: 129 active+clean; 458 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 253 op/s 2023-12-07T10:29:50.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:50 smithi159 ceph-mon[134726]: pgmap v291: 129 pgs: 129 active+clean; 458 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 253 op/s 2023-12-07T10:29:50.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:50 smithi153 ceph-mon[133625]: pgmap v291: 129 pgs: 129 active+clean; 458 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 253 op/s 2023-12-07T10:29:50.570 INFO:tasks.workunit.client.0.smithi119.stdout: 1 671392 7.23 MB/sec execute 363 sec latency 14.471 ms 2023-12-07T10:29:51.570 INFO:tasks.workunit.client.0.smithi119.stdout: 1 672574 7.22 MB/sec execute 364 sec latency 12.988 ms 2023-12-07T10:29:52.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:51 smithi119 ceph-mon[97908]: pgmap v292: 129 pgs: 129 active+clean; 458 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.2 MiB/s wr, 313 op/s 2023-12-07T10:29:52.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:51 smithi159 ceph-mon[134726]: pgmap v292: 129 pgs: 129 active+clean; 458 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.2 MiB/s wr, 313 op/s 2023-12-07T10:29:52.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:51 smithi153 ceph-mon[133625]: pgmap v292: 129 pgs: 129 active+clean; 458 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.2 MiB/s wr, 313 op/s 2023-12-07T10:29:52.570 INFO:tasks.workunit.client.0.smithi119.stdout: 1 674513 7.23 MB/sec execute 365 sec latency 17.016 ms 2023-12-07T10:29:53.570 INFO:tasks.workunit.client.0.smithi119.stdout: 1 675547 7.22 MB/sec execute 366 sec latency 12.383 ms 2023-12-07T10:29:54.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:54 smithi119 ceph-mon[97908]: pgmap v293: 129 pgs: 129 active+clean; 461 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.6 MiB/s wr, 345 op/s 2023-12-07T10:29:54.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:54 smithi159 ceph-mon[134726]: pgmap v293: 129 pgs: 129 active+clean; 461 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.6 MiB/s wr, 345 op/s 2023-12-07T10:29:54.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:54 smithi153 ceph-mon[133625]: pgmap v293: 129 pgs: 129 active+clean; 461 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.6 MiB/s wr, 345 op/s 2023-12-07T10:29:54.570 INFO:tasks.workunit.client.0.smithi119.stdout: 1 677234 7.22 MB/sec execute 367 sec latency 18.506 ms 2023-12-07T10:29:55.570 INFO:tasks.workunit.client.0.smithi119.stdout: 1 678667 7.23 MB/sec execute 368 sec latency 27.211 ms 2023-12-07T10:29:56.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:56 smithi119 ceph-mon[97908]: pgmap v294: 129 pgs: 129 active+clean; 461 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 279 op/s 2023-12-07T10:29:56.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:56 smithi159 ceph-mon[134726]: pgmap v294: 129 pgs: 129 active+clean; 461 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 279 op/s 2023-12-07T10:29:56.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:56 smithi153 ceph-mon[133625]: pgmap v294: 129 pgs: 129 active+clean; 461 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 279 op/s 2023-12-07T10:29:56.570 INFO:tasks.workunit.client.0.smithi119.stdout: 1 679867 7.23 MB/sec execute 369 sec latency 15.418 ms 2023-12-07T10:29:57.570 INFO:tasks.workunit.client.0.smithi119.stdout: 1 681933 7.24 MB/sec execute 370 sec latency 16.907 ms 2023-12-07T10:29:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.844+0000 7f62e13ff700 1 -- 172.21.15.119:0/779741279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62dc1060c0 msgr2=0x7f62dc106540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.844+0000 7f62e13ff700 1 --2- 172.21.15.119:0/779741279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62dc1060c0 0x7f62dc106540 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009990 tx=0x7f62d0009ca0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.844+0000 7f62e13ff700 1 -- 172.21.15.119:0/779741279 shutdown_connections 2023-12-07T10:29:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.844+0000 7f62e13ff700 1 --2- 172.21.15.119:0/779741279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62dc106a80 0x7f62dc10e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.844+0000 7f62e13ff700 1 --2- 172.21.15.119:0/779741279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62dc068ce0 0x7f62dc0690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.844+0000 7f62e13ff700 1 --2- 172.21.15.119:0/779741279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62dc1060c0 0x7f62dc106540 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.844+0000 7f62e13ff700 1 -- 172.21.15.119:0/779741279 >> 172.21.15.119:0/779741279 conn(0x7f62dc0775b0 msgr2=0x7f62dc0779c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.844+0000 7f62e13ff700 1 -- 172.21.15.119:0/779741279 shutdown_connections 2023-12-07T10:29:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.844+0000 7f62e13ff700 1 -- 172.21.15.119:0/779741279 wait complete. 2023-12-07T10:29:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.845+0000 7f62e13ff700 1 Processor -- start 2023-12-07T10:29:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.846+0000 7f62e13ff700 1 -- start start 2023-12-07T10:29:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.846+0000 7f62e13ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62dc068ce0 0x7f62dc19d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.846+0000 7f62e13ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62dc1060c0 0x7f62dc19dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.846+0000 7f62e13ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62dc106a80 0x7f62dc197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.846+0000 7f62e13ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc197e80 con 0x7f62dc1060c0 2023-12-07T10:29:57.846 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.846+0000 7f62e13ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc197ff0 con 0x7f62dc106a80 2023-12-07T10:29:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.846+0000 7f62e13ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc1981b0 con 0x7f62dc068ce0 2023-12-07T10:29:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.846+0000 7f62daffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62dc068ce0 0x7f62dc19d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.846+0000 7f62daffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62dc068ce0 0x7f62dc19d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:50816/0 (socket says 172.21.15.119:50816) 2023-12-07T10:29:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.846+0000 7f62daffd700 1 -- 172.21.15.119:0/125252073 learned_addr learned my addr 172.21.15.119:0/125252073 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:29:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.847+0000 7f62da7fc700 1 --2- 172.21.15.119:0/125252073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62dc1060c0 0x7f62dc19dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.847+0000 7f62db7fe700 1 --2- 172.21.15.119:0/125252073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62dc106a80 0x7f62dc197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.847+0000 7f62daffd700 1 -- 172.21.15.119:0/125252073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62dc106a80 msgr2=0x7f62dc197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.847+0000 7f62daffd700 1 --2- 172.21.15.119:0/125252073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62dc106a80 0x7f62dc197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.847+0000 7f62daffd700 1 -- 172.21.15.119:0/125252073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62dc1060c0 msgr2=0x7f62dc19dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.847+0000 7f62daffd700 1 --2- 172.21.15.119:0/125252073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62dc1060c0 0x7f62dc19dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.847+0000 7f62daffd700 1 -- 172.21.15.119:0/125252073 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d00095f0 con 0x7f62dc068ce0 2023-12-07T10:29:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.847+0000 7f62daffd700 1 --2- 172.21.15.119:0/125252073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62dc068ce0 0x7f62dc19d6e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00e9a0 tx=0x7f62cc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.847+0000 7f62c3fff700 1 -- 172.21.15.119:0/125252073 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc009790 con 0x7f62dc068ce0 2023-12-07T10:29:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.847+0000 7f62c3fff700 1 -- 172.21.15.119:0/125252073 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62cc010430 con 0x7f62dc068ce0 2023-12-07T10:29:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.848+0000 7f62c3fff700 1 -- 172.21.15.119:0/125252073 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc018680 con 0x7f62dc068ce0 2023-12-07T10:29:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.848+0000 7f62e13ff700 1 -- 172.21.15.119:0/125252073 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62dc198490 con 0x7f62dc068ce0 2023-12-07T10:29:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.848+0000 7f62e13ff700 1 -- 172.21.15.119:0/125252073 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62dc1a46d0 con 0x7f62dc068ce0 2023-12-07T10:29:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.851+0000 7f62c3fff700 1 -- 172.21.15.119:0/125252073 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f62cc0105a0 con 0x7f62dc068ce0 2023-12-07T10:29:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.852+0000 7f62c3fff700 1 --2- 172.21.15.119:0/125252073 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f62c40a1d00 0x7f62c40a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.852+0000 7f62f4d00b80 1 Processor -- start 2023-12-07T10:29:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.852+0000 7f62f4d00b80 1 -- start start 2023-12-07T10:29:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.852+0000 7f62c3fff700 1 -- 172.21.15.119:0/125252073 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f62cc014070 con 0x7f62dc068ce0 2023-12-07T10:29:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.852+0000 7f62da7fc700 1 --2- 172.21.15.119:0/125252073 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f62c40a1d00 0x7f62c40a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.852+0000 7f62f4d00b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e98de6fda0 0x55e98de70180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.853+0000 7f62f4d00b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e98de6b950 0x55e98de6bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.853+0000 7f62f4d00b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e98de6c310 0x55e98de77cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.853+0000 7f62f4d00b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98dddb760 con 0x55e98de6b950 2023-12-07T10:29:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.853+0000 7f62f4d00b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98ddc2740 con 0x55e98de6c310 2023-12-07T10:29:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.853+0000 7f62f4d00b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98ddc0060 con 0x55e98de6fda0 2023-12-07T10:29:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.853+0000 7f62daffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e98de6fda0 0x55e98de70180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.853+0000 7f62daffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e98de6fda0 0x55e98de70180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:50830/0 (socket says 172.21.15.119:50830) 2023-12-07T10:29:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.853+0000 7f62daffd700 1 -- 172.21.15.119:0/159255633 learned_addr learned my addr 172.21.15.119:0/159255633 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:29:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.853+0000 7f62da7fc700 1 --2- 172.21.15.119:0/159255633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e98de6b950 0x55e98de6bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.853+0000 7f62db7fe700 1 --2- 172.21.15.119:0/159255633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e98de6c310 0x55e98de77cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.853+0000 7f62daffd700 1 -- 172.21.15.119:0/159255633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e98de6c310 msgr2=0x55e98de77cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.853+0000 7f62daffd700 1 --2- 172.21.15.119:0/159255633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e98de6c310 0x55e98de77cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.853+0000 7f62daffd700 1 -- 172.21.15.119:0/159255633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e98de6b950 msgr2=0x55e98de6bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.853+0000 7f62daffd700 1 --2- 172.21.15.119:0/159255633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e98de6b950 0x55e98de6bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.853+0000 7f62daffd700 1 -- 172.21.15.119:0/159255633 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e98dc385d0 con 0x55e98de6fda0 2023-12-07T10:29:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.853+0000 7f62db7fe700 1 --2- 172.21.15.119:0/159255633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e98de6c310 0x55e98de77cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:29:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.853+0000 7f62da7fc700 1 --2- 172.21.15.119:0/159255633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e98de6b950 0x55e98de6bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:29:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.853+0000 7f62da7fc700 1 --2- 172.21.15.119:0/125252073 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f62c40a1d00 0x7f62c40a41c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f62dc198c30 tx=0x7f62d0005060 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.853+0000 7f62daffd700 1 --2- 172.21.15.119:0/159255633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e98de6fda0 0x55e98de70180 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f62cc097310 tx=0x7f62cc09a510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6a3e449093516d1 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.854+0000 7f62c17fa700 1 -- 172.21.15.119:0/159255633 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc0a2c30 con 0x55e98de6fda0 2023-12-07T10:29:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.854+0000 7f62c17fa700 1 -- 172.21.15.119:0/159255633 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62cc0850b0 con 0x55e98de6fda0 2023-12-07T10:29:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.854+0000 7f62c17fa700 1 -- 172.21.15.119:0/159255633 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc09ad60 con 0x55e98de6fda0 2023-12-07T10:29:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.854+0000 7f62f4d00b80 1 -- 172.21.15.119:0/159255633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e98de6fda0 msgr2=0x55e98de70180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.854+0000 7f62f4d00b80 1 --2- 172.21.15.119:0/159255633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e98de6fda0 0x55e98de70180 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f62cc097310 tx=0x7f62cc09a510 comp rx=0 tx=0).stop 2023-12-07T10:29:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.854+0000 7f62f4d00b80 1 -- 172.21.15.119:0/159255633 shutdown_connections 2023-12-07T10:29:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.854+0000 7f62f4d00b80 1 --2- 172.21.15.119:0/159255633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e98de6c310 0x55e98de77cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.854+0000 7f62f4d00b80 1 --2- 172.21.15.119:0/159255633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e98de6fda0 0x55e98de70180 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.855+0000 7f62f4d00b80 1 --2- 172.21.15.119:0/159255633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e98de6b950 0x55e98de6bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.855+0000 7f62f4d00b80 1 -- 172.21.15.119:0/159255633 >> 172.21.15.119:0/159255633 conn(0x55e98dd96300 msgr2=0x55e98de61b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.855+0000 7f62f4d00b80 1 -- 172.21.15.119:0/159255633 shutdown_connections 2023-12-07T10:29:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.855+0000 7f62f4d00b80 1 -- 172.21.15.119:0/159255633 wait complete. 2023-12-07T10:29:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.855+0000 7f62f4d00b80 1 -- start start 2023-12-07T10:29:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.855+0000 7f62f4d00b80 1 Processor -- start 2023-12-07T10:29:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.856+0000 7f62f4d00b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e98de6b950 0x55e98de77cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.856+0000 7f62f4d00b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e98de6c310 0x55e98de74fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.856+0000 7f62f4d00b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e98de6fda0 0x55e98de756e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.856+0000 7f62f4d00b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98ddbb950 con 0x55e98de6c310 2023-12-07T10:29:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.856+0000 7f62f4d00b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98dd75500 con 0x55e98de6fda0 2023-12-07T10:29:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.856+0000 7f62f4d00b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98dd751a0 con 0x55e98de6b950 2023-12-07T10:29:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.856+0000 7f62f4d00b80 10 client.0 _ll_register_callbacks cb 0x55e98dd5c8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:29:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.856+0000 7f62daffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e98de6b950 0x55e98de77cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.856+0000 7f62daffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e98de6b950 0x55e98de77cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:50838/0 (socket says 172.21.15.119:50838) 2023-12-07T10:29:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.856+0000 7f62daffd700 1 -- 172.21.15.119:0/132005348 learned_addr learned my addr 172.21.15.119:0/132005348 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:29:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.856+0000 7f62da7fc700 1 --2- 172.21.15.119:0/132005348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e98de6c310 0x55e98de74fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.856+0000 7f62db7fe700 1 --2- 172.21.15.119:0/132005348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e98de6fda0 0x55e98de756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.857+0000 7f62daffd700 1 -- 172.21.15.119:0/132005348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e98de6fda0 msgr2=0x55e98de756e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.857+0000 7f62daffd700 1 --2- 172.21.15.119:0/132005348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e98de6fda0 0x55e98de756e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.857+0000 7f62daffd700 1 -- 172.21.15.119:0/132005348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e98de6c310 msgr2=0x55e98de74fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.857+0000 7f62daffd700 1 --2- 172.21.15.119:0/132005348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e98de6c310 0x55e98de74fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.857+0000 7f62daffd700 1 -- 172.21.15.119:0/132005348 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62dc1a46d0 con 0x55e98de6b950 2023-12-07T10:29:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.857+0000 7f62daffd700 1 --2- 172.21.15.119:0/132005348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e98de6b950 0x55e98de77cb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0a2700 tx=0x7f62cc010ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62cc6a28a96b16cb server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.857+0000 7f62beff5700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:29:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.857+0000 7f62beff5700 1 -- 172.21.15.119:0/132005348 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc099d90 con 0x55e98de6b950 2023-12-07T10:29:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.857+0000 7f62beff5700 1 -- 172.21.15.119:0/132005348 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f62cc099ef0 con 0x55e98de6b950 2023-12-07T10:29:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.857+0000 7f62f4d00b80 1 -- 172.21.15.119:0/132005348 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e98de5a6f0 con 0x55e98de6b950 2023-12-07T10:29:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.857+0000 7f62beff5700 1 -- 172.21.15.119:0/132005348 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62cc09ab00 con 0x55e98de6b950 2023-12-07T10:29:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.857+0000 7f62f4d00b80 10 client.27556 fetch_fsmap learned FSMap version 57 2023-12-07T10:29:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.857+0000 7f62f4d00b80 1 -- 172.21.15.119:0/132005348 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55e98dbe4d80 con 0x55e98de6b950 2023-12-07T10:29:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.858+0000 7f62beff5700 1 -- 172.21.15.119:0/132005348 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e98de5a6f0 con 0x55e98de6b950 2023-12-07T10:29:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.858+0000 7f62beff5700 1 -- 172.21.15.119:0/132005348 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f62cc0a27a0 con 0x55e98de6b950 2023-12-07T10:29:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.858+0000 7f62f4d00b80 10 client.27556 fetch_fsmap finished waiting for FSMap version 57 2023-12-07T10:29:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.858+0000 7f62f4d00b80 10 client.27556 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:29:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.858+0000 7f62f4d00b80 20 client.27556 populate_metadata read hostname 'smithi119' 2023-12-07T10:29:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.858+0000 7f62f4d00b80 1 --2- 172.21.15.119:0/132005348 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55e98de84ee0 0x55e98de872d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.858+0000 7f62f4d00b80 4 client.27556 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:29:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.858+0000 7f62f4d00b80 1 -- 172.21.15.119:0/132005348 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e98de36740 con 0x55e98de84ee0 2023-12-07T10:29:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.858+0000 7f62da7fc700 1 --2- 172.21.15.119:0/132005348 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55e98de84ee0 0x55e98de872d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.859+0000 7f62da7fc700 1 --2- 172.21.15.119:0/132005348 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55e98de84ee0 0x55e98de872d0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd34eb3fc5b08fc5 server_cookie=5a9d18e09528482 in_seq=0 out_seq=0 2023-12-07T10:29:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.859+0000 7f62beff5700 10 client.27556 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:29:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.859+0000 7f62beff5700 1 -- 172.21.15.119:0/132005348 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e98de36740 con 0x55e98de84ee0 2023-12-07T10:29:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.859+0000 7f62beff5700 10 client.27556 handle_command_reply: tid=0 2023-12-07T10:29:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.859+0000 7f62f4d00b80 1 client.27556 shutdown 2023-12-07T10:29:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.860+0000 7f62f4d00b80 1 -- 172.21.15.119:0/132005348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e98de6b950 msgr2=0x55e98de77cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.860+0000 7f62f4d00b80 1 --2- 172.21.15.119:0/132005348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e98de6b950 0x55e98de77cb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0a2700 tx=0x7f62cc010ab0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.860+0000 7f62f4d00b80 1 -- 172.21.15.119:0/132005348 shutdown_connections 2023-12-07T10:29:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.860+0000 7f62f4d00b80 1 --2- 172.21.15.119:0/132005348 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55e98de84ee0 0x55e98de872d0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.860+0000 7f62f4d00b80 1 --2- 172.21.15.119:0/132005348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e98de6fda0 0x55e98de756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.860+0000 7f62f4d00b80 1 --2- 172.21.15.119:0/132005348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e98de6b950 0x55e98de77cb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.860+0000 7f62beff5700 0 client.27556 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:29:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.860+0000 7f62f4d00b80 1 --2- 172.21.15.119:0/132005348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e98de6c310 0x55e98de74fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.860+0000 7f62f4d00b80 1 -- 172.21.15.119:0/132005348 >> 172.21.15.119:0/132005348 conn(0x55e98dd96300 msgr2=0x55e98de5c980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.860+0000 7f62f4d00b80 1 -- 172.21.15.119:0/132005348 shutdown_connections 2023-12-07T10:29:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.860+0000 7f62f4d00b80 1 -- 172.21.15.119:0/132005348 wait complete. 2023-12-07T10:29:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.860+0000 7f62f4d00b80 20 client.27556 trim_cache size 0 max 16384 2023-12-07T10:29:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.879+0000 7f62f4d00b80 1 Processor -- start 2023-12-07T10:29:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.879+0000 7f62f4d00b80 1 -- start start 2023-12-07T10:29:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.879+0000 7f62f4d00b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e98de6b950 0x55e98de72bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.879+0000 7f62f4d00b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e98de6c310 0x55e98de6bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.879+0000 7f62f4d00b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e98de6fda0 0x55e98de6d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.879+0000 7f62f4d00b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98dc3a8c0 con 0x55e98de6fda0 2023-12-07T10:29:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.879+0000 7f62f4d00b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98de6dc40 con 0x55e98de6c310 2023-12-07T10:29:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.879+0000 7f62f4d00b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98de734f0 con 0x55e98de6b950 2023-12-07T10:29:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.879+0000 7f62daffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e98de6b950 0x55e98de72bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.879+0000 7f62daffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e98de6b950 0x55e98de72bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:50852/0 (socket says 172.21.15.119:50852) 2023-12-07T10:29:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.879+0000 7f62daffd700 1 -- 172.21.15.119:0/4237791734 learned_addr learned my addr 172.21.15.119:0/4237791734 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:29:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.879+0000 7f62da7fc700 1 --2- 172.21.15.119:0/4237791734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e98de6c310 0x55e98de6bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.880+0000 7f62db7fe700 1 --2- 172.21.15.119:0/4237791734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e98de6fda0 0x55e98de6d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.880+0000 7f62daffd700 1 -- 172.21.15.119:0/4237791734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e98de6c310 msgr2=0x55e98de6bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.880+0000 7f62daffd700 1 --2- 172.21.15.119:0/4237791734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e98de6c310 0x55e98de6bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.880+0000 7f62daffd700 1 -- 172.21.15.119:0/4237791734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e98de6fda0 msgr2=0x55e98de6d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.880+0000 7f62daffd700 1 --2- 172.21.15.119:0/4237791734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e98de6fda0 0x55e98de6d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.880+0000 7f62daffd700 1 -- 172.21.15.119:0/4237791734 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e98dbcdc00 con 0x55e98de6b950 2023-12-07T10:29:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.880+0000 7f62daffd700 1 --2- 172.21.15.119:0/4237791734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e98de6b950 0x55e98de72bc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f62cc09a830 tx=0x7f62cc09a510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6415fa68ad7f8a9d server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.880+0000 7f62c1ffb700 1 -- 172.21.15.119:0/4237791734 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc0a27a0 con 0x55e98de6b950 2023-12-07T10:29:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.880+0000 7f62c1ffb700 1 -- 172.21.15.119:0/4237791734 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62cc0a2900 con 0x55e98de6b950 2023-12-07T10:29:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.880+0000 7f62c1ffb700 1 -- 172.21.15.119:0/4237791734 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc099ff0 con 0x55e98de6b950 2023-12-07T10:29:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.881+0000 7f62f4d00b80 1 -- 172.21.15.119:0/4237791734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e98de6b950 msgr2=0x55e98de72bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.881+0000 7f62f4d00b80 1 --2- 172.21.15.119:0/4237791734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e98de6b950 0x55e98de72bc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f62cc09a830 tx=0x7f62cc09a510 comp rx=0 tx=0).stop 2023-12-07T10:29:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.881+0000 7f62f4d00b80 1 -- 172.21.15.119:0/4237791734 shutdown_connections 2023-12-07T10:29:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.881+0000 7f62f4d00b80 1 --2- 172.21.15.119:0/4237791734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e98de6fda0 0x55e98de6d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.881+0000 7f62f4d00b80 1 --2- 172.21.15.119:0/4237791734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e98de6b950 0x55e98de72bc0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.881+0000 7f62f4d00b80 1 --2- 172.21.15.119:0/4237791734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e98de6c310 0x55e98de6bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.881+0000 7f62f4d00b80 1 -- 172.21.15.119:0/4237791734 >> 172.21.15.119:0/4237791734 conn(0x55e98dd96300 msgr2=0x55e98de8bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.881+0000 7f62f4d00b80 1 -- 172.21.15.119:0/4237791734 shutdown_connections 2023-12-07T10:29:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.881+0000 7f62f4d00b80 1 -- 172.21.15.119:0/4237791734 wait complete. 2023-12-07T10:29:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.882+0000 7f62f4d00b80 1 -- start start 2023-12-07T10:29:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.882+0000 7f62f4d00b80 1 Processor -- start 2023-12-07T10:29:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.882+0000 7f62f4d00b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e98de6b950 0x55e98de7bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.882+0000 7f62f4d00b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e98de6c310 0x55e98de6c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.882+0000 7f62f4d00b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e98de6fda0 0x55e98de6ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.882+0000 7f62f4d00b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98de6d510 con 0x55e98de6c310 2023-12-07T10:29:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.882+0000 7f62f4d00b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98deda340 con 0x55e98de6b950 2023-12-07T10:29:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.882+0000 7f62f4d00b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98deda480 con 0x55e98de6fda0 2023-12-07T10:29:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.883+0000 7f62f4d00b80 10 client.0 _ll_register_callbacks cb 0x55e98dd5c8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:29:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.883+0000 7f62daffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e98de6b950 0x55e98de7bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.883+0000 7f62daffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e98de6b950 0x55e98de7bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:55364/0 (socket says 172.21.15.119:55364) 2023-12-07T10:29:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.883+0000 7f62daffd700 1 -- 172.21.15.119:0/591162084 learned_addr learned my addr 172.21.15.119:0/591162084 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:29:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.883+0000 7f62da7fc700 1 --2- 172.21.15.119:0/591162084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e98de6c310 0x55e98de6c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.883+0000 7f62db7fe700 1 --2- 172.21.15.119:0/591162084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e98de6fda0 0x55e98de6ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:57.887 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.883+0000 7f62daffd700 1 -- 172.21.15.119:0/591162084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e98de6fda0 msgr2=0x55e98de6ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:57.887 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.883+0000 7f62daffd700 1 --2- 172.21.15.119:0/591162084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e98de6fda0 0x55e98de6ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.887 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.883+0000 7f62daffd700 1 -- 172.21.15.119:0/591162084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e98de6c310 msgr2=0x55e98de6c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:29:57.887 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.883+0000 7f62daffd700 1 --2- 172.21.15.119:0/591162084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e98de6c310 0x55e98de6c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.887 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.883+0000 7f62daffd700 1 -- 172.21.15.119:0/591162084 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e98dbe4d80 con 0x55e98de6b950 2023-12-07T10:29:57.887 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.883+0000 7f62daffd700 1 --2- 172.21.15.119:0/591162084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e98de6b950 0x55e98de7bed0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f62cc09aa60 tx=0x7f62cc005b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50d6ec7659225746 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:29:57.887 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.883+0000 7f62beff5700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:29:57.887 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.883+0000 7f62beff5700 1 -- 172.21.15.119:0/591162084 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc0a27a0 con 0x55e98de6b950 2023-12-07T10:29:57.887 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.884+0000 7f62beff5700 1 -- 172.21.15.119:0/591162084 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f62cc09bdd0 con 0x55e98de6b950 2023-12-07T10:29:57.887 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.884+0000 7f62f4d00b80 1 -- 172.21.15.119:0/591162084 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f62cc0850b0 con 0x55e98de6b950 2023-12-07T10:29:57.887 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.884+0000 7f62beff5700 1 -- 172.21.15.119:0/591162084 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62cc09ab00 con 0x55e98de6b950 2023-12-07T10:29:57.887 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.884+0000 7f62f4d00b80 10 client.27578 fetch_fsmap learned FSMap version 57 2023-12-07T10:29:57.887 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.884+0000 7f62f4d00b80 1 -- 172.21.15.119:0/591162084 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55e98dc0b1f0 con 0x55e98de6b950 2023-12-07T10:29:57.887 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.884+0000 7f62beff5700 1 -- 172.21.15.119:0/591162084 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62cc0850b0 con 0x55e98de6b950 2023-12-07T10:29:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.884+0000 7f62beff5700 1 -- 172.21.15.119:0/591162084 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f62cc00ca10 con 0x55e98de6b950 2023-12-07T10:29:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.884+0000 7f62f4d00b80 10 client.27578 fetch_fsmap finished waiting for FSMap version 57 2023-12-07T10:29:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.884+0000 7f62f4d00b80 10 client.27578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:29:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.884+0000 7f62f4d00b80 20 client.27578 populate_metadata read hostname 'smithi119' 2023-12-07T10:29:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.884+0000 7f62f4d00b80 1 --2- 172.21.15.119:0/591162084 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55e98dedb6d0 0x55e98deddac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:29:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.884+0000 7f62f4d00b80 4 client.27578 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:29:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.884+0000 7f62f4d00b80 1 -- 172.21.15.119:0/591162084 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e98dd81f70 con 0x55e98dedb6d0 2023-12-07T10:29:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.884+0000 7f62da7fc700 1 --2- 172.21.15.119:0/591162084 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55e98dedb6d0 0x55e98deddac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:29:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.885+0000 7f62da7fc700 1 --2- 172.21.15.119:0/591162084 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55e98dedb6d0 0x55e98deddac0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a61e152c698a9bf3 server_cookie=a6e35d5f9683947b in_seq=0 out_seq=0 2023-12-07T10:29:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.885+0000 7f62beff5700 10 client.27578 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:29:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.944+0000 7f62beff5700 1 -- 172.21.15.119:0/591162084 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e98dd81f70 con 0x55e98dedb6d0 2023-12-07T10:29:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.944+0000 7f62beff5700 10 client.27578 handle_command_reply: tid=0 2023-12-07T10:29:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.944+0000 7f62f4d00b80 1 client.27578 shutdown 2023-12-07T10:29:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.944+0000 7f62f4d00b80 1 -- 172.21.15.119:0/591162084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e98de6b950 msgr2=0x55e98de7bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.944+0000 7f62f4d00b80 1 --2- 172.21.15.119:0/591162084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e98de6b950 0x55e98de7bed0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f62cc09aa60 tx=0x7f62cc005b40 comp rx=0 tx=0).stop 2023-12-07T10:29:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.944+0000 7f62f4d00b80 1 -- 172.21.15.119:0/591162084 shutdown_connections 2023-12-07T10:29:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.944+0000 7f62f4d00b80 1 --2- 172.21.15.119:0/591162084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e98de6fda0 0x55e98de6ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.944+0000 7f62f4d00b80 1 --2- 172.21.15.119:0/591162084 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55e98dedb6d0 0x55e98deddac0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.944+0000 7f62f4d00b80 1 --2- 172.21.15.119:0/591162084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e98de6b950 0x55e98de7bed0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.944+0000 7f62f4d00b80 1 --2- 172.21.15.119:0/591162084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e98de6c310 0x55e98de6c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.944+0000 7f62f4d00b80 1 -- 172.21.15.119:0/591162084 >> 172.21.15.119:0/591162084 conn(0x55e98dd96300 msgr2=0x55e98de6e130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.944+0000 7f62beff5700 0 client.27578 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:29:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.944+0000 7f62f4d00b80 1 -- 172.21.15.119:0/591162084 shutdown_connections 2023-12-07T10:29:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.944+0000 7f62f4d00b80 1 -- 172.21.15.119:0/591162084 wait complete. 2023-12-07T10:29:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.945+0000 7f62f4d00b80 20 client.27578 trim_cache size 0 max 16384 2023-12-07T10:29:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:29:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:29:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:29:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:29:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.945+0000 7f62e13ff700 1 -- 172.21.15.119:0/125252073 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f62c40a1d00 msgr2=0x7f62c40a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.945+0000 7f62e13ff700 1 --2- 172.21.15.119:0/125252073 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f62c40a1d00 0x7f62c40a41c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f62dc198c30 tx=0x7f62d0005060 comp rx=0 tx=0).stop 2023-12-07T10:29:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.945+0000 7f62e13ff700 1 -- 172.21.15.119:0/125252073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62dc068ce0 msgr2=0x7f62dc19d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:29:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.945+0000 7f62e13ff700 1 --2- 172.21.15.119:0/125252073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62dc068ce0 0x7f62dc19d6e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00e9a0 tx=0x7f62cc00ecb0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.945+0000 7f62e13ff700 1 -- 172.21.15.119:0/125252073 shutdown_connections 2023-12-07T10:29:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.945+0000 7f62e13ff700 1 --2- 172.21.15.119:0/125252073 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f62c40a1d00 0x7f62c40a41c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.945+0000 7f62e13ff700 1 --2- 172.21.15.119:0/125252073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62dc106a80 0x7f62dc197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.945+0000 7f62e13ff700 1 --2- 172.21.15.119:0/125252073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62dc068ce0 0x7f62dc19d6e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.945+0000 7f62e13ff700 1 --2- 172.21.15.119:0/125252073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62dc1060c0 0x7f62dc19dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:29:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.945+0000 7f62e13ff700 1 -- 172.21.15.119:0/125252073 >> 172.21.15.119:0/125252073 conn(0x7f62dc0775b0 msgr2=0x7f62dc0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:29:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.945+0000 7f62e13ff700 1 -- 172.21.15.119:0/125252073 shutdown_connections 2023-12-07T10:29:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:29:57.945+0000 7f62e13ff700 1 -- 172.21.15.119:0/125252073 wait complete. 2023-12-07T10:29:58.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:29:58 smithi119 ceph-mon[97908]: pgmap v295: 129 pgs: 129 active+clean; 469 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 357 op/s 2023-12-07T10:29:58.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:29:58 smithi159 ceph-mon[134726]: pgmap v295: 129 pgs: 129 active+clean; 469 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 357 op/s 2023-12-07T10:29:58.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:29:58 smithi153 ceph-mon[133625]: pgmap v295: 129 pgs: 129 active+clean; 469 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 357 op/s 2023-12-07T10:29:58.570 INFO:tasks.workunit.client.0.smithi119.stdout: 1 682836 7.23 MB/sec execute 371 sec latency 39.821 ms 2023-12-07T10:29:59.571 INFO:tasks.workunit.client.0.smithi119.stdout: 1 684772 7.24 MB/sec execute 372 sec latency 15.629 ms 2023-12-07T10:30:00.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:00 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mon-a[97884]: 2023-12-07T10:29:59.999+0000 7fc367b21700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-12-07T10:30:00.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:00 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mon-a[97884]: 2023-12-07T10:29:59.999+0000 7fc367b21700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-12-07T10:30:00.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:00 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mon-a[97884]: 2023-12-07T10:29:59.999+0000 7fc367b21700 -1 log_channel(cluster) log [ERR] : mds.f(mds.3): Metadata damage detected 2023-12-07T10:30:00.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:00 smithi119 ceph-mon[97908]: pgmap v296: 129 pgs: 129 active+clean; 469 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 194 op/s 2023-12-07T10:30:00.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:00 smithi119 ceph-mon[97908]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-12-07T10:30:00.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:00 smithi119 ceph-mon[97908]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-12-07T10:30:00.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:00 smithi119 ceph-mon[97908]: mds.f(mds.3): Metadata damage detected 2023-12-07T10:30:00.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:00 smithi159 ceph-mon[134726]: pgmap v296: 129 pgs: 129 active+clean; 469 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 194 op/s 2023-12-07T10:30:00.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:00 smithi159 ceph-mon[134726]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-12-07T10:30:00.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:00 smithi159 ceph-mon[134726]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-12-07T10:30:00.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:00 smithi159 ceph-mon[134726]: mds.f(mds.3): Metadata damage detected 2023-12-07T10:30:00.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:00 smithi153 ceph-mon[133625]: pgmap v296: 129 pgs: 129 active+clean; 469 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 194 op/s 2023-12-07T10:30:00.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:00 smithi153 ceph-mon[133625]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-12-07T10:30:00.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:00 smithi153 ceph-mon[133625]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-12-07T10:30:00.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:00 smithi153 ceph-mon[133625]: mds.f(mds.3): Metadata damage detected 2023-12-07T10:30:00.571 INFO:tasks.workunit.client.0.smithi119.stdout: 1 685762 7.24 MB/sec execute 373 sec latency 223.144 ms 2023-12-07T10:30:01.571 INFO:tasks.workunit.client.0.smithi119.stdout: 1 685762 7.22 MB/sec execute 374 sec latency 1223.246 ms 2023-12-07T10:30:02.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:01 smithi119 ceph-mon[97908]: pgmap v297: 129 pgs: 129 active+clean; 471 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 236 op/s 2023-12-07T10:30:02.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:01 smithi159 ceph-mon[134726]: pgmap v297: 129 pgs: 129 active+clean; 471 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 236 op/s 2023-12-07T10:30:02.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:01 smithi153 ceph-mon[133625]: pgmap v297: 129 pgs: 129 active+clean; 471 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 236 op/s 2023-12-07T10:30:02.571 INFO:tasks.workunit.client.0.smithi119.stdout: 1 685762 7.20 MB/sec execute 375 sec latency 2223.376 ms 2023-12-07T10:30:03.571 INFO:tasks.workunit.client.0.smithi119.stdout: 1 686180 7.18 MB/sec execute 376 sec latency 2739.684 ms 2023-12-07T10:30:04.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:04 smithi119 ceph-mon[97908]: pgmap v298: 129 pgs: 129 active+clean; 476 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 227 op/s 2023-12-07T10:30:04.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:04 smithi159 ceph-mon[134726]: pgmap v298: 129 pgs: 129 active+clean; 476 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 227 op/s 2023-12-07T10:30:04.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:04 smithi153 ceph-mon[133625]: pgmap v298: 129 pgs: 129 active+clean; 476 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 227 op/s 2023-12-07T10:30:04.571 INFO:tasks.workunit.client.0.smithi119.stdout: 1 686432 7.17 MB/sec execute 377 sec latency 193.498 ms 2023-12-07T10:30:05.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:05 smithi119 ceph-mon[97908]: osdmap e101: 12 total, 12 up, 12 in 2023-12-07T10:30:05.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:05 smithi159 ceph-mon[134726]: osdmap e101: 12 total, 12 up, 12 in 2023-12-07T10:30:05.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:05 smithi153 ceph-mon[133625]: osdmap e101: 12 total, 12 up, 12 in 2023-12-07T10:30:05.571 INFO:tasks.workunit.client.0.smithi119.stdout: 1 687958 7.16 MB/sec execute 378 sec latency 62.314 ms 2023-12-07T10:30:06.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:06 smithi119 ceph-mon[97908]: pgmap v300: 129 pgs: 129 active+clean; 476 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 205 op/s 2023-12-07T10:30:06.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:06 smithi159 ceph-mon[134726]: pgmap v300: 129 pgs: 129 active+clean; 476 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 205 op/s 2023-12-07T10:30:06.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:06 smithi153 ceph-mon[133625]: pgmap v300: 129 pgs: 129 active+clean; 476 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 205 op/s 2023-12-07T10:30:06.571 INFO:tasks.workunit.client.0.smithi119.stdout: 1 689355 7.17 MB/sec execute 379 sec latency 14.601 ms 2023-12-07T10:30:07.571 INFO:tasks.workunit.client.0.smithi119.stdout: 1 690774 7.17 MB/sec execute 380 sec latency 17.746 ms 2023-12-07T10:30:08.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:08 smithi119 ceph-mon[97908]: pgmap v301: 129 pgs: 129 active+clean; 478 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.3 MiB/s wr, 206 op/s 2023-12-07T10:30:08.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:08 smithi159 ceph-mon[134726]: pgmap v301: 129 pgs: 129 active+clean; 478 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.3 MiB/s wr, 206 op/s 2023-12-07T10:30:08.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:08 smithi153 ceph-mon[133625]: pgmap v301: 129 pgs: 129 active+clean; 478 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.3 MiB/s wr, 206 op/s 2023-12-07T10:30:08.571 INFO:tasks.workunit.client.0.smithi119.stdout: 1 692703 7.19 MB/sec execute 381 sec latency 14.013 ms 2023-12-07T10:30:09.571 INFO:tasks.workunit.client.0.smithi119.stdout: 1 693712 7.17 MB/sec execute 382 sec latency 12.460 ms 2023-12-07T10:30:10.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:10 smithi119 ceph-mon[97908]: pgmap v302: 129 pgs: 129 active+clean; 478 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.3 MiB/s wr, 206 op/s 2023-12-07T10:30:10.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:10 smithi159 ceph-mon[134726]: pgmap v302: 129 pgs: 129 active+clean; 478 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.3 MiB/s wr, 206 op/s 2023-12-07T10:30:10.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:10 smithi153 ceph-mon[133625]: pgmap v302: 129 pgs: 129 active+clean; 478 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.3 MiB/s wr, 206 op/s 2023-12-07T10:30:10.571 INFO:tasks.workunit.client.0.smithi119.stdout: 1 695204 7.18 MB/sec execute 383 sec latency 238.602 ms 2023-12-07T10:30:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.106+0000 7fd1577ff700 1 -- 172.21.15.119:0/1791138594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd150101520 msgr2=0x7fd1501019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.106+0000 7fd1577ff700 1 --2- 172.21.15.119:0/1791138594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd150101520 0x7fd1501019a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd1400099c0 tx=0x7fd140009cd0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.106+0000 7fd1577ff700 1 -- 172.21.15.119:0/1791138594 shutdown_connections 2023-12-07T10:30:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.106+0000 7fd1577ff700 1 --2- 172.21.15.119:0/1791138594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd150101ee0 0x7fd15010e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.106+0000 7fd1577ff700 1 --2- 172.21.15.119:0/1791138594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1501075e0 0x7fd1501079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.106+0000 7fd1577ff700 1 --2- 172.21.15.119:0/1791138594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd150101520 0x7fd1501019a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.106+0000 7fd1577ff700 1 -- 172.21.15.119:0/1791138594 >> 172.21.15.119:0/1791138594 conn(0x7fd1500fd010 msgr2=0x7fd1500ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:30:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.106+0000 7fd1577ff700 1 -- 172.21.15.119:0/1791138594 shutdown_connections 2023-12-07T10:30:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.106+0000 7fd1577ff700 1 -- 172.21.15.119:0/1791138594 wait complete. 2023-12-07T10:30:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.107+0000 7fd1577ff700 1 Processor -- start 2023-12-07T10:30:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.108+0000 7fd1577ff700 1 -- start start 2023-12-07T10:30:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.108+0000 7fd1577ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd150101520 0x7fd15019f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.108+0000 7fd1577ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd150101ee0 0x7fd15019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.108+0000 7fd1577ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1501075e0 0x7fd1501999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.108+0000 7fd1577ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15019a050 con 0x7fd150101ee0 2023-12-07T10:30:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.108+0000 7fd1577ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15019a1c0 con 0x7fd150101520 2023-12-07T10:30:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.108+0000 7fd1577ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15019a380 con 0x7fd1501075e0 2023-12-07T10:30:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.108+0000 7fd15559b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd150101520 0x7fd15019f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.108+0000 7fd15559b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd150101520 0x7fd15019f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:50230/0 (socket says 172.21.15.119:50230) 2023-12-07T10:30:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.108+0000 7fd15559b700 1 -- 172.21.15.119:0/3467930347 learned_addr learned my addr 172.21.15.119:0/3467930347 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:30:11.109 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.108+0000 7fd154d9a700 1 --2- 172.21.15.119:0/3467930347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd150101ee0 0x7fd15019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:11.110 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.108+0000 7fd155d9c700 1 --2- 172.21.15.119:0/3467930347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1501075e0 0x7fd1501999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:11.110 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.109+0000 7fd15559b700 1 -- 172.21.15.119:0/3467930347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1501075e0 msgr2=0x7fd1501999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:11.110 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.109+0000 7fd15559b700 1 --2- 172.21.15.119:0/3467930347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1501075e0 0x7fd1501999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.110 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.109+0000 7fd15559b700 1 -- 172.21.15.119:0/3467930347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd150101ee0 msgr2=0x7fd15019fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:11.110 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.109+0000 7fd15559b700 1 --2- 172.21.15.119:0/3467930347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd150101ee0 0x7fd15019fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.110 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.109+0000 7fd15559b700 1 -- 172.21.15.119:0/3467930347 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1400095f0 con 0x7fd150101520 2023-12-07T10:30:11.111 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.109+0000 7fd15559b700 1 --2- 172.21.15.119:0/3467930347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd150101520 0x7fd15019f870 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd14c00e9a0 tx=0x7fd14c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:30:11.111 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.109+0000 7fd1467fc700 1 -- 172.21.15.119:0/3467930347 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14c009790 con 0x7fd150101520 2023-12-07T10:30:11.111 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.109+0000 7fd1467fc700 1 -- 172.21.15.119:0/3467930347 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd14c010430 con 0x7fd150101520 2023-12-07T10:30:11.111 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.109+0000 7fd1577ff700 1 -- 172.21.15.119:0/3467930347 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd15019a660 con 0x7fd150101520 2023-12-07T10:30:11.111 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.109+0000 7fd1467fc700 1 -- 172.21.15.119:0/3467930347 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14c018680 con 0x7fd150101520 2023-12-07T10:30:11.111 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.110+0000 7fd1577ff700 1 -- 172.21.15.119:0/3467930347 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1501a6860 con 0x7fd150101520 2023-12-07T10:30:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.113+0000 7fd1467fc700 1 -- 172.21.15.119:0/3467930347 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fd14c0105a0 con 0x7fd150101520 2023-12-07T10:30:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.114+0000 7fd1467fc700 1 --2- 172.21.15.119:0/3467930347 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd13c0a1d00 0x7fd13c0a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.114+0000 7fd1467fc700 1 -- 172.21.15.119:0/3467930347 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fd14c014070 con 0x7fd150101520 2023-12-07T10:30:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.114+0000 7fd16b13bb80 1 Processor -- start 2023-12-07T10:30:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.115+0000 7fd154d9a700 1 --2- 172.21.15.119:0/3467930347 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd13c0a1d00 0x7fd13c0a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.115+0000 7fd16b13bb80 1 -- start start 2023-12-07T10:30:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.115+0000 7fd16b13bb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d397897a60 0x55d397897e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.115+0000 7fd16b13bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d397898380 0x55d397892540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.115+0000 7fd16b13bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d397892ad0 0x55d397892f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.115+0000 7fd16b13bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d397806760 con 0x55d397898380 2023-12-07T10:30:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.115+0000 7fd16b13bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3977ed740 con 0x55d397892ad0 2023-12-07T10:30:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.115+0000 7fd16b13bb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3977eb060 con 0x55d397897a60 2023-12-07T10:30:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.115+0000 7fd15559b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d397897a60 0x55d397897e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.115+0000 7fd15559b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d397897a60 0x55d397897e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:53780/0 (socket says 172.21.15.119:53780) 2023-12-07T10:30:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.115+0000 7fd15559b700 1 -- 172.21.15.119:0/3340459704 learned_addr learned my addr 172.21.15.119:0/3340459704 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:30:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.116+0000 7fd155d9c700 1 --2- 172.21.15.119:0/3340459704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d397892ad0 0x55d397892f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.116+0000 7fd154d9a700 1 --2- 172.21.15.119:0/3340459704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d397898380 0x55d397892540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.116+0000 7fd154d9a700 1 --2- 172.21.15.119:0/3467930347 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd13c0a1d00 0x7fd13c0a41c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd140009990 tx=0x7fd140005fd0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:30:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.116+0000 7fd15559b700 1 -- 172.21.15.119:0/3340459704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d397892ad0 msgr2=0x55d397892f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.116+0000 7fd15559b700 1 --2- 172.21.15.119:0/3340459704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d397892ad0 0x55d397892f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.116+0000 7fd15559b700 1 -- 172.21.15.119:0/3340459704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d397898380 msgr2=0x55d397892540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.116+0000 7fd15559b700 1 --2- 172.21.15.119:0/3340459704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d397898380 0x55d397892540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.116+0000 7fd15559b700 1 -- 172.21.15.119:0/3340459704 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3976635d0 con 0x55d397897a60 2023-12-07T10:30:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.116+0000 7fd154d9a700 1 --2- 172.21.15.119:0/3340459704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d397898380 0x55d397892540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:30:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.116+0000 7fd15559b700 1 --2- 172.21.15.119:0/3340459704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d397897a60 0x55d397897e40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd14c097310 tx=0x7fd14c09a510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8496cd7b7ebfebbf server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:30:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.116+0000 7fd13b7fe700 1 -- 172.21.15.119:0/3340459704 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14c09e9f0 con 0x55d397897a60 2023-12-07T10:30:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.116+0000 7fd13b7fe700 1 -- 172.21.15.119:0/3340459704 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd14c0a2c30 con 0x55d397897a60 2023-12-07T10:30:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.116+0000 7fd13b7fe700 1 -- 172.21.15.119:0/3340459704 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14c09ad60 con 0x55d397897a60 2023-12-07T10:30:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.117+0000 7fd16b13bb80 1 -- 172.21.15.119:0/3340459704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d397897a60 msgr2=0x55d397897e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.117+0000 7fd16b13bb80 1 --2- 172.21.15.119:0/3340459704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d397897a60 0x55d397897e40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd14c097310 tx=0x7fd14c09a510 comp rx=0 tx=0).stop 2023-12-07T10:30:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.117+0000 7fd16b13bb80 1 -- 172.21.15.119:0/3340459704 shutdown_connections 2023-12-07T10:30:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.117+0000 7fd16b13bb80 1 --2- 172.21.15.119:0/3340459704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d397892ad0 0x55d397892f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.117+0000 7fd16b13bb80 1 --2- 172.21.15.119:0/3340459704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d397897a60 0x55d397897e40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.117+0000 7fd16b13bb80 1 --2- 172.21.15.119:0/3340459704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d397898380 0x55d397892540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.117+0000 7fd16b13bb80 1 -- 172.21.15.119:0/3340459704 >> 172.21.15.119:0/3340459704 conn(0x55d3977c1300 msgr2=0x55d39788cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:30:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.117+0000 7fd16b13bb80 1 -- 172.21.15.119:0/3340459704 shutdown_connections 2023-12-07T10:30:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.117+0000 7fd16b13bb80 1 -- 172.21.15.119:0/3340459704 wait complete. 2023-12-07T10:30:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.118+0000 7fd16b13bb80 1 -- start start 2023-12-07T10:30:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.118+0000 7fd16b13bb80 1 Processor -- start 2023-12-07T10:30:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.119+0000 7fd16b13bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d397892ad0 0x55d39789e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.119+0000 7fd16b13bb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d397897a60 0x55d39789b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.119+0000 7fd16b13bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d397898380 0x55d39789be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.119+0000 7fd16b13bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3977e6950 con 0x55d397892ad0 2023-12-07T10:30:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.119+0000 7fd16b13bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3977a0500 con 0x55d397898380 2023-12-07T10:30:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.119+0000 7fd16b13bb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3977a01a0 con 0x55d397897a60 2023-12-07T10:30:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.119+0000 7fd16b13bb80 10 client.0 _ll_register_callbacks cb 0x55d3977878e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:30:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.119+0000 7fd154d9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d397897a60 0x55d39789b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.119+0000 7fd15559b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d397892ad0 0x55d39789e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.119+0000 7fd154d9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d397897a60 0x55d39789b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:53782/0 (socket says 172.21.15.119:53782) 2023-12-07T10:30:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.119+0000 7fd154d9a700 1 -- 172.21.15.119:0/1307359670 learned_addr learned my addr 172.21.15.119:0/1307359670 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:30:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.119+0000 7fd155d9c700 1 --2- 172.21.15.119:0/1307359670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d397898380 0x55d39789be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.120+0000 7fd154d9a700 1 -- 172.21.15.119:0/1307359670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d397898380 msgr2=0x55d39789be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.120+0000 7fd154d9a700 1 --2- 172.21.15.119:0/1307359670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d397898380 0x55d39789be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.120+0000 7fd154d9a700 1 -- 172.21.15.119:0/1307359670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d397892ad0 msgr2=0x55d39789e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.120+0000 7fd154d9a700 1 --2- 172.21.15.119:0/1307359670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d397892ad0 0x55d39789e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.120+0000 7fd154d9a700 1 -- 172.21.15.119:0/1307359670 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1501a6860 con 0x55d397897a60 2023-12-07T10:30:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.120+0000 7fd154d9a700 1 --2- 172.21.15.119:0/1307359670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d397897a60 0x55d39789b6f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd1400058e0 tx=0x7fd14001a070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95dbbce3c32fab2b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:30:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.120+0000 7fd138ff9700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:30:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.120+0000 7fd138ff9700 1 -- 172.21.15.119:0/1307359670 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd140024470 con 0x55d397897a60 2023-12-07T10:30:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.120+0000 7fd16b13bb80 1 -- 172.21.15.119:0/1307359670 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3978856f0 con 0x55d397897a60 2023-12-07T10:30:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.120+0000 7fd138ff9700 1 -- 172.21.15.119:0/1307359670 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fd1400245d0 con 0x55d397897a60 2023-12-07T10:30:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.121+0000 7fd138ff9700 1 -- 172.21.15.119:0/1307359670 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1400247d0 con 0x55d397897a60 2023-12-07T10:30:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.121+0000 7fd16b13bb80 10 client.27574 fetch_fsmap learned FSMap version 57 2023-12-07T10:30:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.121+0000 7fd16b13bb80 1 -- 172.21.15.119:0/1307359670 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55d39760fd80 con 0x55d397897a60 2023-12-07T10:30:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.121+0000 7fd138ff9700 1 -- 172.21.15.119:0/1307359670 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d3978856f0 con 0x55d397897a60 2023-12-07T10:30:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.121+0000 7fd138ff9700 1 -- 172.21.15.119:0/1307359670 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fd14002f830 con 0x55d397897a60 2023-12-07T10:30:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.121+0000 7fd16b13bb80 10 client.27574 fetch_fsmap finished waiting for FSMap version 57 2023-12-07T10:30:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.121+0000 7fd16b13bb80 10 client.27574 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:30:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.121+0000 7fd16b13bb80 20 client.27574 populate_metadata read hostname 'smithi119' 2023-12-07T10:30:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.121+0000 7fd16b13bb80 1 --2- 172.21.15.119:0/1307359670 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d3978abec0 0x55d3978ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.121+0000 7fd16b13bb80 4 client.27574 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:30:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.121+0000 7fd16b13bb80 1 -- 172.21.15.119:0/1307359670 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d397861740 con 0x55d3978abec0 2023-12-07T10:30:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.122+0000 7fd15559b700 1 --2- 172.21.15.119:0/1307359670 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d3978abec0 0x55d3978ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.122+0000 7fd15559b700 1 --2- 172.21.15.119:0/1307359670 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d3978abec0 0x55d3978ae2e0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3176ed3827c4f039 server_cookie=ee2ca8e9b35c1670 in_seq=0 out_seq=0 2023-12-07T10:30:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.122+0000 7fd138ff9700 10 client.27574 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:30:11.125 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.125+0000 7fd138ff9700 1 -- 172.21.15.119:0/1307359670 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d397861740 con 0x55d3978abec0 2023-12-07T10:30:11.125 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.125+0000 7fd138ff9700 10 client.27574 handle_command_reply: tid=0 2023-12-07T10:30:11.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.125+0000 7fd16b13bb80 1 client.27574 shutdown 2023-12-07T10:30:11.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.125+0000 7fd16b13bb80 1 -- 172.21.15.119:0/1307359670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d397897a60 msgr2=0x55d39789b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:11.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.125+0000 7fd16b13bb80 1 --2- 172.21.15.119:0/1307359670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d397897a60 0x55d39789b6f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd1400058e0 tx=0x7fd14001a070 comp rx=0 tx=0).stop 2023-12-07T10:30:11.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.125+0000 7fd16b13bb80 1 -- 172.21.15.119:0/1307359670 shutdown_connections 2023-12-07T10:30:11.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.125+0000 7fd16b13bb80 1 --2- 172.21.15.119:0/1307359670 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d3978abec0 0x55d3978ae2e0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.125+0000 7fd16b13bb80 1 --2- 172.21.15.119:0/1307359670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d397898380 0x55d39789be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.125+0000 7fd138ff9700 0 client.27574 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:30:11.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.125+0000 7fd16b13bb80 1 --2- 172.21.15.119:0/1307359670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d397892ad0 0x55d39789e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.125+0000 7fd16b13bb80 1 --2- 172.21.15.119:0/1307359670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d397897a60 0x55d39789b6f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.125+0000 7fd16b13bb80 1 -- 172.21.15.119:0/1307359670 >> 172.21.15.119:0/1307359670 conn(0x55d3977c1300 msgr2=0x55d397887980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:30:11.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.125+0000 7fd16b13bb80 1 -- 172.21.15.119:0/1307359670 shutdown_connections 2023-12-07T10:30:11.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.125+0000 7fd16b13bb80 1 -- 172.21.15.119:0/1307359670 wait complete. 2023-12-07T10:30:11.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.125+0000 7fd16b13bb80 20 client.27574 trim_cache size 0 max 16384 2023-12-07T10:30:11.143 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.143+0000 7fd16b13bb80 1 Processor -- start 2023-12-07T10:30:11.144 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.144+0000 7fd16b13bb80 1 -- start start 2023-12-07T10:30:11.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.144+0000 7fd16b13bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d397892ad0 0x55d397892540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:11.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.144+0000 7fd16b13bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d397897a60 0x55d39789be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:11.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.144+0000 7fd16b13bb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d397898380 0x55d3978a2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:11.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.144+0000 7fd16b13bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3976658c0 con 0x55d397897a60 2023-12-07T10:30:11.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.144+0000 7fd16b13bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3978897a0 con 0x55d397892ad0 2023-12-07T10:30:11.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.144+0000 7fd16b13bb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39788e4a0 con 0x55d397898380 2023-12-07T10:30:11.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.144+0000 7fd15559b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d397892ad0 0x55d397892540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:11.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.144+0000 7fd15559b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d397892ad0 0x55d397892540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:50262/0 (socket says 172.21.15.119:50262) 2023-12-07T10:30:11.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.144+0000 7fd15559b700 1 -- 172.21.15.119:0/3647450835 learned_addr learned my addr 172.21.15.119:0/3647450835 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:30:11.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.144+0000 7fd155d9c700 1 --2- 172.21.15.119:0/3647450835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d397898380 0x55d3978a2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:11.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.144+0000 7fd154d9a700 1 --2- 172.21.15.119:0/3647450835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d397897a60 0x55d39789be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:11.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.144+0000 7fd15559b700 1 -- 172.21.15.119:0/3647450835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d397898380 msgr2=0x55d3978a2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:11.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.144+0000 7fd15559b700 1 --2- 172.21.15.119:0/3647450835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d397898380 0x55d3978a2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.144+0000 7fd15559b700 1 -- 172.21.15.119:0/3647450835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d397897a60 msgr2=0x55d39789be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:11.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.144+0000 7fd15559b700 1 --2- 172.21.15.119:0/3647450835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d397897a60 0x55d39789be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.145+0000 7fd15559b700 1 -- 172.21.15.119:0/3647450835 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3975f8c00 con 0x55d397892ad0 2023-12-07T10:30:11.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.145+0000 7fd15559b700 1 --2- 172.21.15.119:0/3647450835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d397892ad0 0x55d397892540 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd14c09a830 tx=0x7fd14c09a3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=116888b5862dbf2 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:30:11.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.145+0000 7fd13bfff700 1 -- 172.21.15.119:0/3647450835 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14c0b3120 con 0x55d397892ad0 2023-12-07T10:30:11.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.145+0000 7fd13bfff700 1 -- 172.21.15.119:0/3647450835 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd14c0850b0 con 0x55d397892ad0 2023-12-07T10:30:11.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.145+0000 7fd13bfff700 1 -- 172.21.15.119:0/3647450835 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14c099ff0 con 0x55d397892ad0 2023-12-07T10:30:11.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.146+0000 7fd16b13bb80 1 -- 172.21.15.119:0/3647450835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d397892ad0 msgr2=0x55d397892540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:11.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.146+0000 7fd16b13bb80 1 --2- 172.21.15.119:0/3647450835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d397892ad0 0x55d397892540 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd14c09a830 tx=0x7fd14c09a3e0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.146+0000 7fd16b13bb80 1 -- 172.21.15.119:0/3647450835 shutdown_connections 2023-12-07T10:30:11.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.146+0000 7fd16b13bb80 1 --2- 172.21.15.119:0/3647450835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d397898380 0x55d3978a2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.146+0000 7fd16b13bb80 1 --2- 172.21.15.119:0/3647450835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d397892ad0 0x55d397892540 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.146+0000 7fd16b13bb80 1 --2- 172.21.15.119:0/3647450835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d397897a60 0x55d39789be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.146+0000 7fd16b13bb80 1 -- 172.21.15.119:0/3647450835 >> 172.21.15.119:0/3647450835 conn(0x55d3977c1300 msgr2=0x55d3978b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:30:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.146+0000 7fd16b13bb80 1 -- 172.21.15.119:0/3647450835 shutdown_connections 2023-12-07T10:30:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.146+0000 7fd16b13bb80 1 -- 172.21.15.119:0/3647450835 wait complete. 2023-12-07T10:30:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.146+0000 7fd16b13bb80 1 -- start start 2023-12-07T10:30:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.147+0000 7fd16b13bb80 1 Processor -- start 2023-12-07T10:30:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.147+0000 7fd16b13bb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d397892ad0 0x55d3978a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.147+0000 7fd16b13bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d397897a60 0x55d3978a1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.147+0000 7fd16b13bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d397898380 0x55d3978a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.147+0000 7fd16b13bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d397901460 con 0x55d397897a60 2023-12-07T10:30:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.147+0000 7fd16b13bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3979015a0 con 0x55d397898380 2023-12-07T10:30:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.147+0000 7fd16b13bb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d397901760 con 0x55d397892ad0 2023-12-07T10:30:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.148+0000 7fd16b13bb80 10 client.0 _ll_register_callbacks cb 0x55d3977878e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:30:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.148+0000 7fd154d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d397897a60 0x55d3978a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.148+0000 7fd155d9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d397898380 0x55d3978a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.148+0000 7fd154d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d397897a60 0x55d3978a1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38758/0 (socket says 172.21.15.119:38758) 2023-12-07T10:30:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.148+0000 7fd155d9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d397898380 0x55d3978a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:50278/0 (socket says 172.21.15.119:50278) 2023-12-07T10:30:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.148+0000 7fd154d9a700 1 -- 172.21.15.119:0/2226163473 learned_addr learned my addr 172.21.15.119:0/2226163473 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:30:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.148+0000 7fd15559b700 1 --2- 172.21.15.119:0/2226163473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d397892ad0 0x55d3978a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.148+0000 7fd154d9a700 1 -- 172.21.15.119:0/2226163473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d397892ad0 msgr2=0x55d3978a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.148+0000 7fd154d9a700 1 --2- 172.21.15.119:0/2226163473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d397892ad0 0x55d3978a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.148+0000 7fd154d9a700 1 -- 172.21.15.119:0/2226163473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d397898380 msgr2=0x55d3978a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.148+0000 7fd154d9a700 1 --2- 172.21.15.119:0/2226163473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d397898380 0x55d3978a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.148+0000 7fd154d9a700 1 -- 172.21.15.119:0/2226163473 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3976635d0 con 0x55d397897a60 2023-12-07T10:30:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.148+0000 7fd154d9a700 1 --2- 172.21.15.119:0/2226163473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d397897a60 0x55d3978a1290 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd14002fbc0 tx=0x7fd140005360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4abbca7e32e855ad server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:30:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.148+0000 7fd138ff9700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:30:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.149+0000 7fd138ff9700 1 -- 172.21.15.119:0/2226163473 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd140003960 con 0x55d397897a60 2023-12-07T10:30:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.149+0000 7fd16b13bb80 1 -- 172.21.15.119:0/2226163473 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd14c0a2c30 con 0x55d397897a60 2023-12-07T10:30:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.149+0000 7fd138ff9700 1 -- 172.21.15.119:0/2226163473 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fd14002f620 con 0x55d397897a60 2023-12-07T10:30:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.149+0000 7fd138ff9700 1 -- 172.21.15.119:0/2226163473 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd14002f7e0 con 0x55d397897a60 2023-12-07T10:30:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.149+0000 7fd16b13bb80 10 client.17556 fetch_fsmap learned FSMap version 57 2023-12-07T10:30:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.149+0000 7fd16b13bb80 1 -- 172.21.15.119:0/2226163473 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55d3976361f0 con 0x55d397897a60 2023-12-07T10:30:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.149+0000 7fd138ff9700 1 -- 172.21.15.119:0/2226163473 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd14c0a2c30 con 0x55d397897a60 2023-12-07T10:30:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.150+0000 7fd138ff9700 1 -- 172.21.15.119:0/2226163473 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 57) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fd14000bbd0 con 0x55d397897a60 2023-12-07T10:30:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.150+0000 7fd16b13bb80 10 client.17556 fetch_fsmap finished waiting for FSMap version 57 2023-12-07T10:30:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.150+0000 7fd16b13bb80 10 client.17556 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:30:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.150+0000 7fd16b13bb80 20 client.17556 populate_metadata read hostname 'smithi119' 2023-12-07T10:30:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.150+0000 7fd16b13bb80 1 --2- 172.21.15.119:0/2226163473 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d397902b50 0x55d397904f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.150+0000 7fd16b13bb80 4 client.17556 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:30:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.150+0000 7fd16b13bb80 1 -- 172.21.15.119:0/2226163473 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d3977acf70 con 0x55d397902b50 2023-12-07T10:30:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.150+0000 7fd15559b700 1 --2- 172.21.15.119:0/2226163473 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d397902b50 0x55d397904f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.151+0000 7fd15559b700 1 --2- 172.21.15.119:0/2226163473 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d397902b50 0x55d397904f70 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1aef807c5823bb4 server_cookie=bc2c0abf6e3de810 in_seq=0 out_seq=0 2023-12-07T10:30:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.151+0000 7fd138ff9700 10 client.17556 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:30:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.234+0000 7fd138ff9700 1 -- 172.21.15.119:0/2226163473 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d3977acf70 con 0x55d397902b50 2023-12-07T10:30:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.234+0000 7fd138ff9700 10 client.17556 handle_command_reply: tid=0 2023-12-07T10:30:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.234+0000 7fd16b13bb80 1 client.17556 shutdown 2023-12-07T10:30:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.234+0000 7fd16b13bb80 1 -- 172.21.15.119:0/2226163473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d397897a60 msgr2=0x55d3978a1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.234+0000 7fd16b13bb80 1 --2- 172.21.15.119:0/2226163473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d397897a60 0x55d3978a1290 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd14002fbc0 tx=0x7fd140005360 comp rx=0 tx=0).stop 2023-12-07T10:30:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.235+0000 7fd16b13bb80 1 -- 172.21.15.119:0/2226163473 shutdown_connections 2023-12-07T10:30:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.235+0000 7fd16b13bb80 1 --2- 172.21.15.119:0/2226163473 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d397902b50 0x55d397904f70 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.235+0000 7fd16b13bb80 1 --2- 172.21.15.119:0/2226163473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d397898380 0x55d3978a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.235+0000 7fd16b13bb80 1 --2- 172.21.15.119:0/2226163473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d397892ad0 0x55d3978a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.235+0000 7fd16b13bb80 1 --2- 172.21.15.119:0/2226163473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d397897a60 0x55d3978a1290 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.235+0000 7fd16b13bb80 1 -- 172.21.15.119:0/2226163473 >> 172.21.15.119:0/2226163473 conn(0x55d3977c1300 msgr2=0x55d397895960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:30:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.235+0000 7fd138ff9700 0 client.17556 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:30:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.235+0000 7fd16b13bb80 1 -- 172.21.15.119:0/2226163473 shutdown_connections 2023-12-07T10:30:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.235+0000 7fd16b13bb80 1 -- 172.21.15.119:0/2226163473 wait complete. 2023-12-07T10:30:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.236+0000 7fd16b13bb80 20 client.17556 trim_cache size 0 max 16384 2023-12-07T10:30:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:30:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:30:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:30:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:30:11.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.237+0000 7fd1577ff700 1 -- 172.21.15.119:0/3467930347 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd13c0a1d00 msgr2=0x7fd13c0a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:11.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.237+0000 7fd1577ff700 1 --2- 172.21.15.119:0/3467930347 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd13c0a1d00 0x7fd13c0a41c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd140009990 tx=0x7fd140005fd0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.237+0000 7fd1577ff700 1 -- 172.21.15.119:0/3467930347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd150101520 msgr2=0x7fd15019f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:11.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.237+0000 7fd1577ff700 1 --2- 172.21.15.119:0/3467930347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd150101520 0x7fd15019f870 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd14c00e9a0 tx=0x7fd14c00ecb0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.237+0000 7fd1577ff700 1 -- 172.21.15.119:0/3467930347 shutdown_connections 2023-12-07T10:30:11.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.237+0000 7fd1577ff700 1 --2- 172.21.15.119:0/3467930347 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fd13c0a1d00 0x7fd13c0a41c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.237+0000 7fd1577ff700 1 --2- 172.21.15.119:0/3467930347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1501075e0 0x7fd1501999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.237+0000 7fd1577ff700 1 --2- 172.21.15.119:0/3467930347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd150101520 0x7fd15019f870 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.237+0000 7fd1577ff700 1 --2- 172.21.15.119:0/3467930347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd150101ee0 0x7fd15019fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:11.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.237+0000 7fd1577ff700 1 -- 172.21.15.119:0/3467930347 >> 172.21.15.119:0/3467930347 conn(0x7fd1500fd010 msgr2=0x7fd1500fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:30:11.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.237+0000 7fd1577ff700 1 -- 172.21.15.119:0/3467930347 shutdown_connections 2023-12-07T10:30:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:11.238+0000 7fd1577ff700 1 -- 172.21.15.119:0/3467930347 wait complete. 2023-12-07T10:30:11.571 INFO:tasks.workunit.client.0.smithi119.stdout: 1 696423 7.18 MB/sec execute 384 sec latency 53.970 ms 2023-12-07T10:30:12.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:11 smithi119 ceph-mon[97908]: osdmap e102: 12 total, 12 up, 12 in 2023-12-07T10:30:12.228 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:11 smithi119 ceph-mon[97908]: pgmap v304: 129 pgs: 129 active+clean; 483 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 1.3 MiB/s wr, 186 op/s 2023-12-07T10:30:12.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:11 smithi159 ceph-mon[134726]: osdmap e102: 12 total, 12 up, 12 in 2023-12-07T10:30:12.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:11 smithi159 ceph-mon[134726]: pgmap v304: 129 pgs: 129 active+clean; 483 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 1.3 MiB/s wr, 186 op/s 2023-12-07T10:30:12.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:11 smithi153 ceph-mon[133625]: osdmap e102: 12 total, 12 up, 12 in 2023-12-07T10:30:12.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:11 smithi153 ceph-mon[133625]: pgmap v304: 129 pgs: 129 active+clean; 483 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 1.3 MiB/s wr, 186 op/s 2023-12-07T10:30:12.571 INFO:tasks.workunit.client.0.smithi119.stdout: 1 697649 7.18 MB/sec execute 385 sec latency 18.126 ms 2023-12-07T10:30:13.571 INFO:tasks.workunit.client.0.smithi119.stdout: 1 699565 7.19 MB/sec execute 386 sec latency 55.986 ms 2023-12-07T10:30:14.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:14 smithi119 ceph-mon[97908]: pgmap v305: 129 pgs: 129 active+clean; 487 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 219 op/s 2023-12-07T10:30:14.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:14 smithi159 ceph-mon[134726]: pgmap v305: 129 pgs: 129 active+clean; 487 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 219 op/s 2023-12-07T10:30:14.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:14 smithi153 ceph-mon[133625]: pgmap v305: 129 pgs: 129 active+clean; 487 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 219 op/s 2023-12-07T10:30:14.572 INFO:tasks.workunit.client.0.smithi119.stdout: 1 700495 7.18 MB/sec execute 387 sec latency 14.387 ms 2023-12-07T10:30:15.572 INFO:tasks.workunit.client.0.smithi119.stdout: 1 702357 7.18 MB/sec execute 388 sec latency 15.769 ms 2023-12-07T10:30:16.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:16 smithi119 ceph-mon[97908]: pgmap v306: 129 pgs: 129 active+clean; 487 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.5 MiB/s wr, 196 op/s 2023-12-07T10:30:16.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:16 smithi159 ceph-mon[134726]: pgmap v306: 129 pgs: 129 active+clean; 487 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.5 MiB/s wr, 196 op/s 2023-12-07T10:30:16.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:16 smithi153 ceph-mon[133625]: pgmap v306: 129 pgs: 129 active+clean; 487 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.5 MiB/s wr, 196 op/s 2023-12-07T10:30:16.572 INFO:tasks.workunit.client.0.smithi119.stdout: 1 703664 7.19 MB/sec execute 389 sec latency 21.816 ms 2023-12-07T10:30:17.572 INFO:tasks.workunit.client.0.smithi119.stdout: 1 704956 7.18 MB/sec execute 390 sec latency 36.988 ms 2023-12-07T10:30:18.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:18 smithi119 ceph-mon[97908]: pgmap v307: 129 pgs: 129 active+clean; 493 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 233 op/s 2023-12-07T10:30:18.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:18 smithi159 ceph-mon[134726]: pgmap v307: 129 pgs: 129 active+clean; 493 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 233 op/s 2023-12-07T10:30:18.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:18 smithi153 ceph-mon[133625]: pgmap v307: 129 pgs: 129 active+clean; 493 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 233 op/s 2023-12-07T10:30:18.572 INFO:tasks.workunit.client.0.smithi119.stdout: 1 706893 7.20 MB/sec execute 391 sec latency 22.581 ms 2023-12-07T10:30:19.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:19 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:30:19.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:19 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:30:19.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:19 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:30:19.572 INFO:tasks.workunit.client.0.smithi119.stdout: 1 708083 7.19 MB/sec execute 392 sec latency 13.147 ms 2023-12-07T10:30:20.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:20 smithi119 ceph-mon[97908]: pgmap v308: 129 pgs: 129 active+clean; 493 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 233 op/s 2023-12-07T10:30:20.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:20 smithi159 ceph-mon[134726]: pgmap v308: 129 pgs: 129 active+clean; 493 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 233 op/s 2023-12-07T10:30:20.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:20 smithi153 ceph-mon[133625]: pgmap v308: 129 pgs: 129 active+clean; 493 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 233 op/s 2023-12-07T10:30:20.572 INFO:tasks.workunit.client.0.smithi119.stdout: 1 710029 7.20 MB/sec execute 393 sec latency 18.695 ms 2023-12-07T10:30:21.572 INFO:tasks.workunit.client.0.smithi119.stdout: 1 711050 7.19 MB/sec execute 394 sec latency 13.205 ms 2023-12-07T10:30:22.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:21 smithi119 ceph-mon[97908]: pgmap v309: 129 pgs: 129 active+clean; 501 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 337 op/s 2023-12-07T10:30:22.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:21 smithi159 ceph-mon[134726]: pgmap v309: 129 pgs: 129 active+clean; 501 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 337 op/s 2023-12-07T10:30:22.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:21 smithi153 ceph-mon[133625]: pgmap v309: 129 pgs: 129 active+clean; 501 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 337 op/s 2023-12-07T10:30:22.572 INFO:tasks.workunit.client.0.smithi119.stdout: 1 712653 7.19 MB/sec execute 395 sec latency 63.790 ms 2023-12-07T10:30:23.572 INFO:tasks.workunit.client.0.smithi119.stdout: 1 714076 7.20 MB/sec execute 396 sec latency 35.855 ms 2023-12-07T10:30:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.403+0000 7efee4bff700 1 -- 172.21.15.119:0/357329174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee0106a80 msgr2=0x7efee010e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.403+0000 7efee4bff700 1 --2- 172.21.15.119:0/357329174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee0106a80 0x7efee010e360 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efed800b0a0 tx=0x7efed800b3b0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.404 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.403+0000 7efee4bff700 1 -- 172.21.15.119:0/357329174 shutdown_connections 2023-12-07T10:30:24.404 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.403+0000 7efee4bff700 1 --2- 172.21.15.119:0/357329174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee0106a80 0x7efee010e360 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.404 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.403+0000 7efee4bff700 1 --2- 172.21.15.119:0/357329174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efee0068ce0 0x7efee00690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.404 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.403+0000 7efee4bff700 1 --2- 172.21.15.119:0/357329174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efee01060c0 0x7efee0106540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.404 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.403+0000 7efee4bff700 1 -- 172.21.15.119:0/357329174 >> 172.21.15.119:0/357329174 conn(0x7efee00775b0 msgr2=0x7efee00779c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:30:24.404 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.403+0000 7efee4bff700 1 -- 172.21.15.119:0/357329174 shutdown_connections 2023-12-07T10:30:24.404 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.403+0000 7efee4bff700 1 -- 172.21.15.119:0/357329174 wait complete. 2023-12-07T10:30:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.404+0000 7efee4bff700 1 Processor -- start 2023-12-07T10:30:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.404+0000 7efee4bff700 1 -- start start 2023-12-07T10:30:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.404+0000 7efee4bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efee0068ce0 0x7efee0108370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.404+0000 7efee4bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee01060c0 0x7efee01088b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.404+0000 7efee4bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efee0106a80 0x7efee0108df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.404+0000 7efee4bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee01aa2a0 con 0x7efee0106a80 2023-12-07T10:30:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.404+0000 7efee4bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee01aa3e0 con 0x7efee01060c0 2023-12-07T10:30:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.404+0000 7efee4bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee01aa5a0 con 0x7efee0068ce0 2023-12-07T10:30:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.405+0000 7efeded9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efee0106a80 0x7efee0108df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.405+0000 7efeddd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee01060c0 0x7efee01088b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.405+0000 7efede59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efee0068ce0 0x7efee0108370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.405+0000 7efeddd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee01060c0 0x7efee01088b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:48354/0 (socket says 172.21.15.119:48354) 2023-12-07T10:30:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.405+0000 7efeddd9b700 1 -- 172.21.15.119:0/3971786449 learned_addr learned my addr 172.21.15.119:0/3971786449 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:30:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.405+0000 7efeded9d700 1 -- 172.21.15.119:0/3971786449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efee0068ce0 msgr2=0x7efee0108370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.405+0000 7efeded9d700 1 --2- 172.21.15.119:0/3971786449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efee0068ce0 0x7efee0108370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.405+0000 7efeded9d700 1 -- 172.21.15.119:0/3971786449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee01060c0 msgr2=0x7efee01088b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.405+0000 7efeded9d700 1 --2- 172.21.15.119:0/3971786449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee01060c0 0x7efee01088b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.405+0000 7efeded9d700 1 -- 172.21.15.119:0/3971786449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed4009520 con 0x7efee0106a80 2023-12-07T10:30:24.406 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.405+0000 7efeded9d700 1 --2- 172.21.15.119:0/3971786449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efee0106a80 0x7efee0108df0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7efed800b070 tx=0x7efed8007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:30:24.407 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.405+0000 7efecf7fe700 1 -- 172.21.15.119:0/3971786449 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed800e040 con 0x7efee0106a80 2023-12-07T10:30:24.407 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.406+0000 7efee4bff700 1 -- 172.21.15.119:0/3971786449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed8009d00 con 0x7efee0106a80 2023-12-07T10:30:24.407 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.406+0000 7efecf7fe700 1 -- 172.21.15.119:0/3971786449 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efed801d470 con 0x7efee0106a80 2023-12-07T10:30:24.407 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.406+0000 7efecf7fe700 1 -- 172.21.15.119:0/3971786449 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed80052a0 con 0x7efee0106a80 2023-12-07T10:30:24.407 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.406+0000 7efee4bff700 1 -- 172.21.15.119:0/3971786449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee01aaba0 con 0x7efee0106a80 2023-12-07T10:30:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.409+0000 7efecf7fe700 1 -- 172.21.15.119:0/3971786449 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7efed801d5e0 con 0x7efee0106a80 2023-12-07T10:30:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.410+0000 7efef85e3b80 1 Processor -- start 2023-12-07T10:30:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.411+0000 7efef85e3b80 1 -- start start 2023-12-07T10:30:24.412 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.411+0000 7efecf7fe700 1 --2- 172.21.15.119:0/3971786449 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7efec80a1c80 0x7efec80a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:24.412 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.411+0000 7efef85e3b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf75ea60 0x55e5cf75ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:24.412 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.411+0000 7efef85e3b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf75f380 0x55e5cf759540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:24.412 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.411+0000 7efef85e3b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf759ad0 0x55e5cf759f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.411+0000 7efef85e3b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5cf6cd760 con 0x55e5cf75ea60 2023-12-07T10:30:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.411+0000 7efef85e3b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5cf6b4740 con 0x55e5cf75f380 2023-12-07T10:30:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.411+0000 7efef85e3b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5cf6b2060 con 0x55e5cf759ad0 2023-12-07T10:30:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.411+0000 7efecf7fe700 1 -- 172.21.15.119:0/3971786449 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7efed80ceef0 con 0x7efee0106a80 2023-12-07T10:30:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.411+0000 7efeddd9b700 1 --2- 172.21.15.119:0/3971786449 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7efec80a1c80 0x7efec80a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.411+0000 7efede59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf75ea60 0x55e5cf75ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.411+0000 7efeddd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf75f380 0x55e5cf759540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.411+0000 7efeded9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf759ad0 0x55e5cf759f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.411+0000 7efede59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf75ea60 0x55e5cf75ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44698/0 (socket says 172.21.15.119:44698) 2023-12-07T10:30:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.411+0000 7efede59c700 1 -- 172.21.15.119:0/3407813174 learned_addr learned my addr 172.21.15.119:0/3407813174 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:30:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.412+0000 7efeded9d700 1 -- 172.21.15.119:0/3407813174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf75f380 msgr2=0x55e5cf759540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.412+0000 7efeded9d700 1 --2- 172.21.15.119:0/3407813174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf75f380 0x55e5cf759540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.412+0000 7efeded9d700 1 -- 172.21.15.119:0/3407813174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf75ea60 msgr2=0x55e5cf75ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.412+0000 7efeded9d700 1 --2- 172.21.15.119:0/3407813174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf75ea60 0x55e5cf75ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.412+0000 7efeded9d700 1 -- 172.21.15.119:0/3407813174 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5cf52a5d0 con 0x55e5cf759ad0 2023-12-07T10:30:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.412+0000 7efede59c700 1 --2- 172.21.15.119:0/3407813174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf75ea60 0x55e5cf75ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:30:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.412+0000 7efeded9d700 1 --2- 172.21.15.119:0/3407813174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf759ad0 0x55e5cf759f30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efed80cc290 tx=0x7efed809b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff9ae5c3a65d8efb server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:30:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.412+0000 7efeddd9b700 1 --2- 172.21.15.119:0/3971786449 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7efec80a1c80 0x7efec80a4140 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55e5cf755890 tx=0x7efed4009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:30:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.412+0000 7efeccff9700 1 -- 172.21.15.119:0/3407813174 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed809c070 con 0x55e5cf759ad0 2023-12-07T10:30:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.412+0000 7efeccff9700 1 -- 172.21.15.119:0/3407813174 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efed809b970 con 0x55e5cf759ad0 2023-12-07T10:30:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.412+0000 7efeccff9700 1 -- 172.21.15.119:0/3407813174 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed80bb0e0 con 0x55e5cf759ad0 2023-12-07T10:30:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.413+0000 7efef85e3b80 1 -- 172.21.15.119:0/3407813174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf759ad0 msgr2=0x55e5cf759f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.413+0000 7efef85e3b80 1 --2- 172.21.15.119:0/3407813174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf759ad0 0x55e5cf759f30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efed80cc290 tx=0x7efed809b7c0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.413+0000 7efef85e3b80 1 -- 172.21.15.119:0/3407813174 shutdown_connections 2023-12-07T10:30:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.413+0000 7efef85e3b80 1 --2- 172.21.15.119:0/3407813174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf759ad0 0x55e5cf759f30 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.413+0000 7efef85e3b80 1 --2- 172.21.15.119:0/3407813174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf75ea60 0x55e5cf75ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.413+0000 7efef85e3b80 1 --2- 172.21.15.119:0/3407813174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf75f380 0x55e5cf759540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.413+0000 7efef85e3b80 1 -- 172.21.15.119:0/3407813174 >> 172.21.15.119:0/3407813174 conn(0x55e5cf688300 msgr2=0x55e5cf753b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:30:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.413+0000 7efef85e3b80 1 -- 172.21.15.119:0/3407813174 shutdown_connections 2023-12-07T10:30:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.413+0000 7efef85e3b80 1 -- 172.21.15.119:0/3407813174 wait complete. 2023-12-07T10:30:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.414+0000 7efef85e3b80 1 -- start start 2023-12-07T10:30:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.414+0000 7efef85e3b80 1 Processor -- start 2023-12-07T10:30:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.415+0000 7efef85e3b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf759ad0 0x55e5cf765370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.415+0000 7efef85e3b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf75ea60 0x55e5cf762740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.415+0000 7efef85e3b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf75f380 0x55e5cf762e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.415+0000 7efef85e3b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5cf6ad950 con 0x55e5cf759ad0 2023-12-07T10:30:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.415+0000 7efef85e3b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5cf667500 con 0x55e5cf75f380 2023-12-07T10:30:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.415+0000 7efef85e3b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5cf6671a0 con 0x55e5cf75ea60 2023-12-07T10:30:24.419 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.415+0000 7efef85e3b80 10 client.0 _ll_register_callbacks cb 0x55e5cf64e8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:30:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.415+0000 7efede59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf759ad0 0x55e5cf765370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.415+0000 7efeded9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf75f380 0x55e5cf762e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.415+0000 7efeddd9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf75ea60 0x55e5cf762740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.415+0000 7efede59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf759ad0 0x55e5cf765370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44710/0 (socket says 172.21.15.119:44710) 2023-12-07T10:30:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.415+0000 7efede59c700 1 -- 172.21.15.119:0/115819946 learned_addr learned my addr 172.21.15.119:0/115819946 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:30:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.415+0000 7efeded9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf75f380 0x55e5cf762e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:48374/0 (socket says 172.21.15.119:48374) 2023-12-07T10:30:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.416+0000 7efeded9d700 1 -- 172.21.15.119:0/115819946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf75ea60 msgr2=0x55e5cf762740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.416+0000 7efeded9d700 1 --2- 172.21.15.119:0/115819946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf75ea60 0x55e5cf762740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.416+0000 7efeded9d700 1 -- 172.21.15.119:0/115819946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf759ad0 msgr2=0x55e5cf765370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.416+0000 7efeded9d700 1 --2- 172.21.15.119:0/115819946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf759ad0 0x55e5cf765370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.416+0000 7efeded9d700 1 -- 172.21.15.119:0/115819946 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee01aaba0 con 0x55e5cf75f380 2023-12-07T10:30:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.416+0000 7efede59c700 1 --2- 172.21.15.119:0/115819946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf759ad0 0x55e5cf765370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:30:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.416+0000 7efeded9d700 1 --2- 172.21.15.119:0/115819946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf75f380 0x55e5cf762e30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7efed80062a0 tx=0x7efed809c070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d61aed2bab44ad9 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:30:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.416+0000 7efec5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:30:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.416+0000 7efec5ffb700 1 -- 172.21.15.119:0/115819946 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed801dac0 con 0x55e5cf75f380 2023-12-07T10:30:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.416+0000 7efec5ffb700 1 -- 172.21.15.119:0/115819946 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7efed80bb0b0 con 0x55e5cf75f380 2023-12-07T10:30:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.416+0000 7efef85e3b80 1 -- 172.21.15.119:0/115819946 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e5cf74c6f0 con 0x55e5cf75f380 2023-12-07T10:30:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.416+0000 7efec5ffb700 1 -- 172.21.15.119:0/115819946 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7efed80bb270 con 0x55e5cf75f380 2023-12-07T10:30:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.416+0000 7efef85e3b80 10 client.27608 fetch_fsmap learned FSMap version 58 2023-12-07T10:30:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.416+0000 7efef85e3b80 1 -- 172.21.15.119:0/115819946 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55e5cf4d6d80 con 0x55e5cf75f380 2023-12-07T10:30:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.417+0000 7efec5ffb700 1 -- 172.21.15.119:0/115819946 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e5cf74c6f0 con 0x55e5cf75f380 2023-12-07T10:30:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.417+0000 7efec5ffb700 1 -- 172.21.15.119:0/115819946 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 58) v1 ==== 7894+0+0 (secure 0 0 0) 0x7efed80a37e0 con 0x55e5cf75f380 2023-12-07T10:30:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.417+0000 7efef85e3b80 10 client.27608 fetch_fsmap finished waiting for FSMap version 58 2023-12-07T10:30:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.417+0000 7efef85e3b80 10 client.27608 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:30:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.417+0000 7efef85e3b80 20 client.27608 populate_metadata read hostname 'smithi119' 2023-12-07T10:30:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.417+0000 7efef85e3b80 1 --2- 172.21.15.119:0/115819946 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55e5cf7726e0 0x55e5cf774ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.417+0000 7efef85e3b80 4 client.27608 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:30:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.417+0000 7efef85e3b80 1 -- 172.21.15.119:0/115819946 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5cf728740 con 0x55e5cf7726e0 2023-12-07T10:30:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.417+0000 7efede59c700 1 --2- 172.21.15.119:0/115819946 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55e5cf7726e0 0x55e5cf774ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.418+0000 7efec5ffb700 10 client.27608 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:30:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.418+0000 7efede59c700 1 --2- 172.21.15.119:0/115819946 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55e5cf7726e0 0x55e5cf774ad0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93555d2e455e1d9d server_cookie=e88006d5b884fad6 in_seq=0 out_seq=0 2023-12-07T10:30:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.419+0000 7efec5ffb700 1 -- 172.21.15.119:0/115819946 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e5cf728740 con 0x55e5cf7726e0 2023-12-07T10:30:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.419+0000 7efec5ffb700 10 client.27608 handle_command_reply: tid=0 2023-12-07T10:30:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.419+0000 7efef85e3b80 1 client.27608 shutdown 2023-12-07T10:30:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.419+0000 7efef85e3b80 1 -- 172.21.15.119:0/115819946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf75f380 msgr2=0x55e5cf762e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.419+0000 7efef85e3b80 1 --2- 172.21.15.119:0/115819946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf75f380 0x55e5cf762e30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7efed80062a0 tx=0x7efed809c070 comp rx=0 tx=0).stop 2023-12-07T10:30:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.419+0000 7efef85e3b80 1 -- 172.21.15.119:0/115819946 shutdown_connections 2023-12-07T10:30:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.419+0000 7efef85e3b80 1 --2- 172.21.15.119:0/115819946 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55e5cf7726e0 0x55e5cf774ad0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.419+0000 7efef85e3b80 1 --2- 172.21.15.119:0/115819946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf75f380 0x55e5cf762e30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.419+0000 7efef85e3b80 1 --2- 172.21.15.119:0/115819946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf759ad0 0x55e5cf765370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.419+0000 7efec5ffb700 0 client.27608 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:30:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.419+0000 7efef85e3b80 1 --2- 172.21.15.119:0/115819946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf75ea60 0x55e5cf762740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.419+0000 7efef85e3b80 1 -- 172.21.15.119:0/115819946 >> 172.21.15.119:0/115819946 conn(0x55e5cf688300 msgr2=0x55e5cf74e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:30:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.419+0000 7efef85e3b80 1 -- 172.21.15.119:0/115819946 shutdown_connections 2023-12-07T10:30:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.419+0000 7efef85e3b80 1 -- 172.21.15.119:0/115819946 wait complete. 2023-12-07T10:30:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.419+0000 7efef85e3b80 20 client.27608 trim_cache size 0 max 16384 2023-12-07T10:30:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.438+0000 7efef85e3b80 1 Processor -- start 2023-12-07T10:30:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.438+0000 7efef85e3b80 1 -- start start 2023-12-07T10:30:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.438+0000 7efef85e3b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf759ad0 0x55e5cf762000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.438+0000 7efef85e3b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf75ea60 0x55e5cf759540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.438+0000 7efef85e3b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf75f380 0x55e5cf762d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.438+0000 7efef85e3b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5cf52c8c0 con 0x55e5cf75f380 2023-12-07T10:30:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.438+0000 7efef85e3b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5cf74c6d0 con 0x55e5cf759ad0 2023-12-07T10:30:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.438+0000 7efef85e3b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5cf754ea0 con 0x55e5cf75ea60 2023-12-07T10:30:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.438+0000 7efede59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf759ad0 0x55e5cf762000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.438+0000 7efeded9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf75f380 0x55e5cf762d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.438+0000 7efede59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf759ad0 0x55e5cf762000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:48378/0 (socket says 172.21.15.119:48378) 2023-12-07T10:30:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.438+0000 7efede59c700 1 -- 172.21.15.119:0/172463164 learned_addr learned my addr 172.21.15.119:0/172463164 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:30:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.439+0000 7efeddd9b700 1 --2- 172.21.15.119:0/172463164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf75ea60 0x55e5cf759540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.439+0000 7efede59c700 1 -- 172.21.15.119:0/172463164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf75ea60 msgr2=0x55e5cf759540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.439+0000 7efede59c700 1 --2- 172.21.15.119:0/172463164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf75ea60 0x55e5cf759540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.439+0000 7efede59c700 1 -- 172.21.15.119:0/172463164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf75f380 msgr2=0x55e5cf762d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.439+0000 7efede59c700 1 --2- 172.21.15.119:0/172463164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf75f380 0x55e5cf762d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.439+0000 7efede59c700 1 -- 172.21.15.119:0/172463164 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5cf4bfc00 con 0x55e5cf759ad0 2023-12-07T10:30:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.439+0000 7efeded9d700 1 --2- 172.21.15.119:0/172463164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf75f380 0x55e5cf762d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:30:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.439+0000 7efede59c700 1 --2- 172.21.15.119:0/172463164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf759ad0 0x55e5cf762000 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55e5cf7753b0 tx=0x7efed0010c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=137bf3468f51bb84 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:30:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.439+0000 7efecd7fa700 1 -- 172.21.15.119:0/172463164 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed0004220 con 0x55e5cf759ad0 2023-12-07T10:30:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.439+0000 7efecd7fa700 1 -- 172.21.15.119:0/172463164 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efed0023070 con 0x55e5cf759ad0 2023-12-07T10:30:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.439+0000 7efecd7fa700 1 -- 172.21.15.119:0/172463164 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed001f340 con 0x55e5cf759ad0 2023-12-07T10:30:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.440+0000 7efef85e3b80 1 -- 172.21.15.119:0/172463164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf759ad0 msgr2=0x55e5cf762000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.440+0000 7efef85e3b80 1 --2- 172.21.15.119:0/172463164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf759ad0 0x55e5cf762000 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55e5cf7753b0 tx=0x7efed0010c30 comp rx=0 tx=0).stop 2023-12-07T10:30:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.440+0000 7efef85e3b80 1 -- 172.21.15.119:0/172463164 shutdown_connections 2023-12-07T10:30:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.440+0000 7efef85e3b80 1 --2- 172.21.15.119:0/172463164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf75f380 0x55e5cf762d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.440+0000 7efef85e3b80 1 --2- 172.21.15.119:0/172463164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf759ad0 0x55e5cf762000 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.440+0000 7efef85e3b80 1 --2- 172.21.15.119:0/172463164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf75ea60 0x55e5cf759540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.440+0000 7efef85e3b80 1 -- 172.21.15.119:0/172463164 >> 172.21.15.119:0/172463164 conn(0x55e5cf688300 msgr2=0x55e5cf75d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:30:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.440+0000 7efef85e3b80 1 -- 172.21.15.119:0/172463164 shutdown_connections 2023-12-07T10:30:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.440+0000 7efef85e3b80 1 -- 172.21.15.119:0/172463164 wait complete. 2023-12-07T10:30:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.440+0000 7efef85e3b80 1 -- start start 2023-12-07T10:30:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.441+0000 7efef85e3b80 1 Processor -- start 2023-12-07T10:30:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.441+0000 7efef85e3b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf759ad0 0x55e5cf767b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.441+0000 7efef85e3b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf75ea60 0x55e5cf7680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.441+0000 7efef85e3b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf75f380 0x55e5cf768610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.441+0000 7efef85e3b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5cf7c7d90 con 0x55e5cf75f380 2023-12-07T10:30:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.441+0000 7efef85e3b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5cf7c7f00 con 0x55e5cf75ea60 2023-12-07T10:30:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.441+0000 7efef85e3b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5cf7c80c0 con 0x55e5cf759ad0 2023-12-07T10:30:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.441+0000 7efef85e3b80 10 client.0 _ll_register_callbacks cb 0x55e5cf64e8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:30:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.441+0000 7efeddd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf75ea60 0x55e5cf7680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.441+0000 7efeddd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf75ea60 0x55e5cf7680b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:48384/0 (socket says 172.21.15.119:48384) 2023-12-07T10:30:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.441+0000 7efeded9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf75f380 0x55e5cf768610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.441+0000 7efeddd9b700 1 -- 172.21.15.119:0/1767310854 learned_addr learned my addr 172.21.15.119:0/1767310854 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:30:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.442+0000 7efede59c700 1 --2- 172.21.15.119:0/1767310854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf759ad0 0x55e5cf767b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.442+0000 7efeddd9b700 1 -- 172.21.15.119:0/1767310854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf759ad0 msgr2=0x55e5cf767b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.442+0000 7efeddd9b700 1 --2- 172.21.15.119:0/1767310854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf759ad0 0x55e5cf767b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.442+0000 7efeddd9b700 1 -- 172.21.15.119:0/1767310854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf75f380 msgr2=0x55e5cf768610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.442+0000 7efeddd9b700 1 --2- 172.21.15.119:0/1767310854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf75f380 0x55e5cf768610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.442+0000 7efeddd9b700 1 -- 172.21.15.119:0/1767310854 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5cf4d6d80 con 0x55e5cf75ea60 2023-12-07T10:30:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.442+0000 7efeded9d700 1 --2- 172.21.15.119:0/1767310854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf75f380 0x55e5cf768610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:30:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.442+0000 7efede59c700 1 --2- 172.21.15.119:0/1767310854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf759ad0 0x55e5cf767b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:30:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.442+0000 7efeddd9b700 1 --2- 172.21.15.119:0/1767310854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf75ea60 0x55e5cf7680b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7efed40097a0 tx=0x7efed4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4ffc37632f6ea9b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:30:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.442+0000 7efec5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:30:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.442+0000 7efec5ffb700 1 -- 172.21.15.119:0/1767310854 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed4013270 con 0x55e5cf75ea60 2023-12-07T10:30:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.442+0000 7efef85e3b80 1 -- 172.21.15.119:0/1767310854 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efed809b970 con 0x55e5cf75ea60 2023-12-07T10:30:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.442+0000 7efec5ffb700 1 -- 172.21.15.119:0/1767310854 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7efed4004830 con 0x55e5cf75ea60 2023-12-07T10:30:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.443+0000 7efec5ffb700 1 -- 172.21.15.119:0/1767310854 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7efed40049f0 con 0x55e5cf75ea60 2023-12-07T10:30:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.443+0000 7efef85e3b80 10 client.27614 fetch_fsmap learned FSMap version 58 2023-12-07T10:30:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.443+0000 7efef85e3b80 1 -- 172.21.15.119:0/1767310854 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x7efed0010710 con 0x55e5cf75ea60 2023-12-07T10:30:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.443+0000 7efec5ffb700 1 -- 172.21.15.119:0/1767310854 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efed809b970 con 0x55e5cf75ea60 2023-12-07T10:30:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.443+0000 7efec5ffb700 1 -- 172.21.15.119:0/1767310854 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 58) v1 ==== 7894+0+0 (secure 0 0 0) 0x7efed4011650 con 0x55e5cf75ea60 2023-12-07T10:30:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.443+0000 7efef85e3b80 10 client.27614 fetch_fsmap finished waiting for FSMap version 58 2023-12-07T10:30:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.443+0000 7efef85e3b80 10 client.27614 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:30:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.443+0000 7efef85e3b80 20 client.27614 populate_metadata read hostname 'smithi119' 2023-12-07T10:30:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.443+0000 7efef85e3b80 1 --2- 172.21.15.119:0/1767310854 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55e5cf7c9940 0x55e5cf7cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.443+0000 7efef85e3b80 4 client.27614 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:30:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.443+0000 7efef85e3b80 1 -- 172.21.15.119:0/1767310854 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e5cf673f70 con 0x55e5cf7c9940 2023-12-07T10:30:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.443+0000 7efede59c700 1 --2- 172.21.15.119:0/1767310854 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55e5cf7c9940 0x55e5cf7cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.444+0000 7efede59c700 1 --2- 172.21.15.119:0/1767310854 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55e5cf7c9940 0x55e5cf7cbd60 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6d1b5a8289684ed server_cookie=cf5b99c1d02c1d44 in_seq=0 out_seq=0 2023-12-07T10:30:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.444+0000 7efec5ffb700 10 client.27614 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:30:24.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:24 smithi119 ceph-mon[97908]: pgmap v310: 129 pgs: 129 active+clean; 503 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 253 op/s 2023-12-07T10:30:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.501+0000 7efec5ffb700 1 -- 172.21.15.119:0/1767310854 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e5cf673f70 con 0x55e5cf7c9940 2023-12-07T10:30:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.501+0000 7efec5ffb700 10 client.27614 handle_command_reply: tid=0 2023-12-07T10:30:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.501+0000 7efef85e3b80 1 client.27614 shutdown 2023-12-07T10:30:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.502+0000 7efef85e3b80 1 -- 172.21.15.119:0/1767310854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf75ea60 msgr2=0x55e5cf7680b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.502+0000 7efef85e3b80 1 --2- 172.21.15.119:0/1767310854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf75ea60 0x55e5cf7680b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7efed40097a0 tx=0x7efed4004750 comp rx=0 tx=0).stop 2023-12-07T10:30:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.502+0000 7efef85e3b80 1 -- 172.21.15.119:0/1767310854 shutdown_connections 2023-12-07T10:30:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.502+0000 7efef85e3b80 1 --2- 172.21.15.119:0/1767310854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e5cf75f380 0x55e5cf768610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.502+0000 7efef85e3b80 1 --2- 172.21.15.119:0/1767310854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5cf759ad0 0x55e5cf767b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.502+0000 7efef85e3b80 1 --2- 172.21.15.119:0/1767310854 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55e5cf7c9940 0x55e5cf7cbd60 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.502+0000 7efef85e3b80 1 --2- 172.21.15.119:0/1767310854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e5cf75ea60 0x55e5cf7680b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.502+0000 7efec5ffb700 0 client.27614 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:30:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.502+0000 7efef85e3b80 1 -- 172.21.15.119:0/1767310854 >> 172.21.15.119:0/1767310854 conn(0x55e5cf688300 msgr2=0x55e5cf75e160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:30:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.502+0000 7efef85e3b80 1 -- 172.21.15.119:0/1767310854 shutdown_connections 2023-12-07T10:30:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.502+0000 7efef85e3b80 1 -- 172.21.15.119:0/1767310854 wait complete. 2023-12-07T10:30:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.502+0000 7efef85e3b80 20 client.27614 trim_cache size 0 max 16384 2023-12-07T10:30:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:30:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:30:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:30:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:30:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.503+0000 7efee4bff700 1 -- 172.21.15.119:0/3971786449 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7efec80a1c80 msgr2=0x7efec80a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.503+0000 7efee4bff700 1 --2- 172.21.15.119:0/3971786449 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7efec80a1c80 0x7efec80a4140 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55e5cf755890 tx=0x7efed4009260 comp rx=0 tx=0).stop 2023-12-07T10:30:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.503+0000 7efee4bff700 1 -- 172.21.15.119:0/3971786449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efee0106a80 msgr2=0x7efee0108df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.503+0000 7efee4bff700 1 --2- 172.21.15.119:0/3971786449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efee0106a80 0x7efee0108df0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7efed800b070 tx=0x7efed8007860 comp rx=0 tx=0).stop 2023-12-07T10:30:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.503+0000 7efee4bff700 1 -- 172.21.15.119:0/3971786449 shutdown_connections 2023-12-07T10:30:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.503+0000 7efee4bff700 1 --2- 172.21.15.119:0/3971786449 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7efec80a1c80 0x7efec80a4140 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.503+0000 7efee4bff700 1 --2- 172.21.15.119:0/3971786449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efee0106a80 0x7efee0108df0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.503+0000 7efee4bff700 1 --2- 172.21.15.119:0/3971786449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efee0068ce0 0x7efee0108370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.503+0000 7efee4bff700 1 --2- 172.21.15.119:0/3971786449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efee01060c0 0x7efee01088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.503+0000 7efee4bff700 1 -- 172.21.15.119:0/3971786449 >> 172.21.15.119:0/3971786449 conn(0x7efee00775b0 msgr2=0x7efee010c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:30:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.503+0000 7efee4bff700 1 -- 172.21.15.119:0/3971786449 shutdown_connections 2023-12-07T10:30:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:24.503+0000 7efee4bff700 1 -- 172.21.15.119:0/3971786449 wait complete. 2023-12-07T10:30:24.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:24 smithi159 ceph-mon[134726]: pgmap v310: 129 pgs: 129 active+clean; 503 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 253 op/s 2023-12-07T10:30:24.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:24 smithi153 ceph-mon[133625]: pgmap v310: 129 pgs: 129 active+clean; 503 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 253 op/s 2023-12-07T10:30:24.572 INFO:tasks.workunit.client.0.smithi119.stdout: 1 715271 7.20 MB/sec execute 397 sec latency 39.547 ms 2023-12-07T10:30:25.572 INFO:tasks.workunit.client.0.smithi119.stdout: 1 717293 7.21 MB/sec execute 398 sec latency 18.187 ms 2023-12-07T10:30:26.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:26 smithi119 ceph-mon[97908]: pgmap v311: 129 pgs: 129 active+clean; 503 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 214 op/s 2023-12-07T10:30:26.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:26 smithi159 ceph-mon[134726]: pgmap v311: 129 pgs: 129 active+clean; 503 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 214 op/s 2023-12-07T10:30:26.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:26 smithi153 ceph-mon[133625]: pgmap v311: 129 pgs: 129 active+clean; 503 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 214 op/s 2023-12-07T10:30:26.572 INFO:tasks.workunit.client.0.smithi119.stdout: 1 718294 7.20 MB/sec execute 399 sec latency 13.243 ms 2023-12-07T10:30:27.573 INFO:tasks.workunit.client.0.smithi119.stdout: 1 719916 7.20 MB/sec execute 400 sec latency 51.643 ms 2023-12-07T10:30:28.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:28 smithi119 ceph-mon[97908]: pgmap v312: 129 pgs: 129 active+clean; 511 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 330 op/s 2023-12-07T10:30:28.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:28 smithi159 ceph-mon[134726]: pgmap v312: 129 pgs: 129 active+clean; 511 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 330 op/s 2023-12-07T10:30:28.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:28 smithi153 ceph-mon[133625]: pgmap v312: 129 pgs: 129 active+clean; 511 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 330 op/s 2023-12-07T10:30:28.572 INFO:tasks.workunit.client.0.smithi119.stdout: 1 721305 7.21 MB/sec execute 401 sec latency 22.401 ms 2023-12-07T10:30:29.572 INFO:tasks.workunit.client.0.smithi119.stdout: 1 723057 7.21 MB/sec execute 402 sec latency 16.816 ms 2023-12-07T10:30:30.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:30 smithi119 ceph-mon[97908]: pgmap v313: 129 pgs: 129 active+clean; 511 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 220 op/s 2023-12-07T10:30:30.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:30 smithi159 ceph-mon[134726]: pgmap v313: 129 pgs: 129 active+clean; 511 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 220 op/s 2023-12-07T10:30:30.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:30 smithi153 ceph-mon[133625]: pgmap v313: 129 pgs: 129 active+clean; 511 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 220 op/s 2023-12-07T10:30:30.572 INFO:tasks.workunit.client.0.smithi119.stdout: 1 724687 7.22 MB/sec execute 403 sec latency 49.860 ms 2023-12-07T10:30:31.572 INFO:tasks.workunit.client.0.smithi119.stdout: 1 725927 7.22 MB/sec execute 404 sec latency 33.344 ms 2023-12-07T10:30:32.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:31 smithi119 ceph-mon[97908]: pgmap v314: 129 pgs: 129 active+clean; 515 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 293 op/s 2023-12-07T10:30:32.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:31 smithi159 ceph-mon[134726]: pgmap v314: 129 pgs: 129 active+clean; 515 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 293 op/s 2023-12-07T10:30:32.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:31 smithi153 ceph-mon[133625]: pgmap v314: 129 pgs: 129 active+clean; 515 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 293 op/s 2023-12-07T10:30:32.573 INFO:tasks.workunit.client.0.smithi119.stdout: 1 727779 7.22 MB/sec execute 405 sec latency 32.855 ms 2023-12-07T10:30:33.573 INFO:tasks.workunit.client.0.smithi119.stdout: 1 728859 7.22 MB/sec execute 406 sec latency 21.214 ms 2023-12-07T10:30:34.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:34 smithi119 ceph-mon[97908]: pgmap v315: 129 pgs: 129 active+clean; 519 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 239 op/s 2023-12-07T10:30:34.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:34 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:30:34.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:34 smithi159 ceph-mon[134726]: pgmap v315: 129 pgs: 129 active+clean; 519 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 239 op/s 2023-12-07T10:30:34.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:34 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:30:34.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:34 smithi153 ceph-mon[133625]: pgmap v315: 129 pgs: 129 active+clean; 519 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 239 op/s 2023-12-07T10:30:34.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:34 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:30:34.573 INFO:tasks.workunit.client.0.smithi119.stdout: 1 730846 7.23 MB/sec execute 407 sec latency 17.018 ms 2023-12-07T10:30:35.573 INFO:tasks.workunit.client.0.smithi119.stdout: 1 731963 7.23 MB/sec execute 408 sec latency 114.729 ms 2023-12-07T10:30:36.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:36 smithi119 ceph-mon[97908]: pgmap v316: 129 pgs: 129 active+clean; 519 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 225 op/s 2023-12-07T10:30:36.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:36 smithi159 ceph-mon[134726]: pgmap v316: 129 pgs: 129 active+clean; 519 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 225 op/s 2023-12-07T10:30:36.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:36 smithi153 ceph-mon[133625]: pgmap v316: 129 pgs: 129 active+clean; 519 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 225 op/s 2023-12-07T10:30:36.573 INFO:tasks.workunit.client.0.smithi119.stdout: 1 733475 7.22 MB/sec execute 409 sec latency 18.783 ms 2023-12-07T10:30:37.573 INFO:tasks.workunit.client.0.smithi119.stdout: 1 735254 7.24 MB/sec execute 410 sec latency 18.886 ms 2023-12-07T10:30:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.663+0000 7f7104bff700 1 -- 172.21.15.119:0/2049229813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7100068ce0 msgr2=0x7f71000690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.663+0000 7f7104bff700 1 --2- 172.21.15.119:0/2049229813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7100068ce0 0x7f71000690c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009990 tx=0x7f70f0009ca0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.663+0000 7f7104bff700 1 -- 172.21.15.119:0/2049229813 shutdown_connections 2023-12-07T10:30:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.663+0000 7f7104bff700 1 --2- 172.21.15.119:0/2049229813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7100106a80 0x7f710010e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.663+0000 7f7104bff700 1 --2- 172.21.15.119:0/2049229813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7100068ce0 0x7f71000690c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.663+0000 7f7104bff700 1 --2- 172.21.15.119:0/2049229813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71001060c0 0x7f7100106540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.663+0000 7f7104bff700 1 -- 172.21.15.119:0/2049229813 >> 172.21.15.119:0/2049229813 conn(0x7f71000775b0 msgr2=0x7f71000779c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:30:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.663+0000 7f7104bff700 1 -- 172.21.15.119:0/2049229813 shutdown_connections 2023-12-07T10:30:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.663+0000 7f7104bff700 1 -- 172.21.15.119:0/2049229813 wait complete. 2023-12-07T10:30:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.664+0000 7f7104bff700 1 Processor -- start 2023-12-07T10:30:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.664+0000 7f7104bff700 1 -- start start 2023-12-07T10:30:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.664+0000 7f7104bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7100068ce0 0x7f7100100480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.664+0000 7f7104bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71001060c0 0x7f71001009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.664+0000 7f7104bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7100106a80 0x7f7100100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.664+0000 7f7104bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71001a5db0 con 0x7f71001060c0 2023-12-07T10:30:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.664+0000 7f7104bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71001a5f20 con 0x7f7100106a80 2023-12-07T10:30:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.664+0000 7f7104bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71001a60e0 con 0x7f7100068ce0 2023-12-07T10:30:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.665+0000 7f70fe59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7100068ce0 0x7f7100100480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.665+0000 7f70fed9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7100106a80 0x7f7100100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.665+0000 7f70fe59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7100068ce0 0x7f7100100480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:45904/0 (socket says 172.21.15.119:45904) 2023-12-07T10:30:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.665+0000 7f70f7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71001060c0 0x7f71001009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.665+0000 7f70fe59c700 1 -- 172.21.15.119:0/651025935 learned_addr learned my addr 172.21.15.119:0/651025935 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:30:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.665+0000 7f70fed9d700 1 -- 172.21.15.119:0/651025935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7100068ce0 msgr2=0x7f7100100480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.665+0000 7f70fed9d700 1 --2- 172.21.15.119:0/651025935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7100068ce0 0x7f7100100480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.667 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.665+0000 7f70fed9d700 1 -- 172.21.15.119:0/651025935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71001060c0 msgr2=0x7f71001009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:37.667 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.665+0000 7f70fed9d700 1 --2- 172.21.15.119:0/651025935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71001060c0 0x7f71001009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.667 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.665+0000 7f70fed9d700 1 -- 172.21.15.119:0/651025935 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f00095f0 con 0x7f7100106a80 2023-12-07T10:30:37.667 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.665+0000 7f70fed9d700 1 --2- 172.21.15.119:0/651025935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7100106a80 0x7f7100100f00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f70f800efc0 tx=0x7f70f800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:30:37.667 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.666+0000 7f70f77fe700 1 -- 172.21.15.119:0/651025935 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8018070 con 0x7f7100106a80 2023-12-07T10:30:37.667 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.666+0000 7f7104bff700 1 -- 172.21.15.119:0/651025935 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71001a6360 con 0x7f7100106a80 2023-12-07T10:30:37.668 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.666+0000 7f7104bff700 1 -- 172.21.15.119:0/651025935 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71001a6870 con 0x7f7100106a80 2023-12-07T10:30:37.668 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.666+0000 7f70f77fe700 1 -- 172.21.15.119:0/651025935 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70f800f040 con 0x7f7100106a80 2023-12-07T10:30:37.668 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.666+0000 7f70f77fe700 1 -- 172.21.15.119:0/651025935 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f800cd40 con 0x7f7100106a80 2023-12-07T10:30:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.670+0000 7f70f77fe700 1 -- 172.21.15.119:0/651025935 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f70f8004ad0 con 0x7f7100106a80 2023-12-07T10:30:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.671+0000 7f71185acb80 1 Processor -- start 2023-12-07T10:30:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.671+0000 7f70f77fe700 1 --2- 172.21.15.119:0/651025935 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f70ec0a1d00 0x7f70ec0a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.671+0000 7f71185acb80 1 -- start start 2023-12-07T10:30:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.671+0000 7f70f77fe700 1 -- 172.21.15.119:0/651025935 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f70f80cd780 con 0x7f7100106a80 2023-12-07T10:30:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.672+0000 7f70f7fff700 1 --2- 172.21.15.119:0/651025935 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f70ec0a1d00 0x7f70ec0a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.672+0000 7f71185acb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c620f20a60 0x55c620f20e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.672+0000 7f71185acb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c620f21380 0x55c620f1b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.672+0000 7f71185acb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c620f1bad0 0x55c620f1bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.672+0000 7f71185acb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c620e8f760 con 0x55c620f1bad0 2023-12-07T10:30:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.672+0000 7f71185acb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c620e76740 con 0x55c620f21380 2023-12-07T10:30:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.672+0000 7f71185acb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c620e74060 con 0x55c620f20a60 2023-12-07T10:30:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.672+0000 7f70fe59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c620f20a60 0x55c620f20e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.672+0000 7f70fe59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c620f20a60 0x55c620f20e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:45908/0 (socket says 172.21.15.119:45908) 2023-12-07T10:30:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.672+0000 7f70fe59c700 1 -- 172.21.15.119:0/455838679 learned_addr learned my addr 172.21.15.119:0/455838679 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:30:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.672+0000 7f70f7fff700 1 --2- 172.21.15.119:0/455838679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c620f21380 0x55c620f1b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.672+0000 7f70fed9d700 1 --2- 172.21.15.119:0/455838679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c620f1bad0 0x55c620f1bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.673+0000 7f70f7fff700 1 --2- 172.21.15.119:0/651025935 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f70ec0a1d00 0x7f70ec0a41c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7100101bb0 tx=0x7f70e800b410 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:30:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.673+0000 7f70fe59c700 1 -- 172.21.15.119:0/455838679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c620f21380 msgr2=0x55c620f1b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.673+0000 7f70fe59c700 1 --2- 172.21.15.119:0/455838679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c620f21380 0x55c620f1b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.673+0000 7f70fe59c700 1 -- 172.21.15.119:0/455838679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c620f1bad0 msgr2=0x55c620f1bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.673+0000 7f70fe59c700 1 --2- 172.21.15.119:0/455838679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c620f1bad0 0x55c620f1bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.673+0000 7f70fe59c700 1 -- 172.21.15.119:0/455838679 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c620cec5d0 con 0x55c620f20a60 2023-12-07T10:30:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.673+0000 7f70fed9d700 1 --2- 172.21.15.119:0/455838679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c620f1bad0 0x55c620f1bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:30:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.673+0000 7f70fe59c700 1 --2- 172.21.15.119:0/455838679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c620f20a60 0x55c620f20e40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f70f000b6f0 tx=0x7f70f0005bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f80fbd3ec267c1cc server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:30:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.673+0000 7f70f4ff9700 1 -- 172.21.15.119:0/455838679 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f001d070 con 0x55c620f20a60 2023-12-07T10:30:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.673+0000 7f70f4ff9700 1 -- 172.21.15.119:0/455838679 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70f001e040 con 0x55c620f20a60 2023-12-07T10:30:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.673+0000 7f70f4ff9700 1 -- 172.21.15.119:0/455838679 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0004920 con 0x55c620f20a60 2023-12-07T10:30:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.674+0000 7f71185acb80 1 -- 172.21.15.119:0/455838679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c620f20a60 msgr2=0x55c620f20e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.674+0000 7f71185acb80 1 --2- 172.21.15.119:0/455838679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c620f20a60 0x55c620f20e40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f70f000b6f0 tx=0x7f70f0005bc0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.674+0000 7f71185acb80 1 -- 172.21.15.119:0/455838679 shutdown_connections 2023-12-07T10:30:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.674+0000 7f71185acb80 1 --2- 172.21.15.119:0/455838679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c620f1bad0 0x55c620f1bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.674+0000 7f71185acb80 1 --2- 172.21.15.119:0/455838679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c620f20a60 0x55c620f20e40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.674+0000 7f71185acb80 1 --2- 172.21.15.119:0/455838679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c620f21380 0x55c620f1b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.674+0000 7f71185acb80 1 -- 172.21.15.119:0/455838679 >> 172.21.15.119:0/455838679 conn(0x55c620e4a300 msgr2=0x55c620f15b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:30:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.675+0000 7f71185acb80 1 -- 172.21.15.119:0/455838679 shutdown_connections 2023-12-07T10:30:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.675+0000 7f71185acb80 1 -- 172.21.15.119:0/455838679 wait complete. 2023-12-07T10:30:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.675+0000 7f71185acb80 1 -- start start 2023-12-07T10:30:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.675+0000 7f71185acb80 1 Processor -- start 2023-12-07T10:30:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.676+0000 7f71185acb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c620f1bad0 0x55c620f272f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.676+0000 7f71185acb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c620f20a60 0x55c620f24730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.676+0000 7f71185acb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c620f21380 0x55c620f24d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.676+0000 7f71185acb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c620e6f950 con 0x55c620f1bad0 2023-12-07T10:30:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.676+0000 7f71185acb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c620e29500 con 0x55c620f21380 2023-12-07T10:30:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.676+0000 7f71185acb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c620e291a0 con 0x55c620f20a60 2023-12-07T10:30:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.676+0000 7f71185acb80 10 client.0 _ll_register_callbacks cb 0x55c620e108e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:30:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.676+0000 7f70fe59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c620f1bad0 0x55c620f272f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.676+0000 7f70f7fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c620f20a60 0x55c620f24730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.676+0000 7f70f7fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c620f20a60 0x55c620f24730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:45912/0 (socket says 172.21.15.119:45912) 2023-12-07T10:30:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.676+0000 7f70fe59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c620f1bad0 0x55c620f272f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44638/0 (socket says 172.21.15.119:44638) 2023-12-07T10:30:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.676+0000 7f70f7fff700 1 -- 172.21.15.119:0/293618397 learned_addr learned my addr 172.21.15.119:0/293618397 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:30:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.676+0000 7f70fed9d700 1 --2- 172.21.15.119:0/293618397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c620f21380 0x55c620f24d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.677+0000 7f70f7fff700 1 -- 172.21.15.119:0/293618397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c620f21380 msgr2=0x55c620f24d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.677+0000 7f70f7fff700 1 --2- 172.21.15.119:0/293618397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c620f21380 0x55c620f24d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.677+0000 7f70f7fff700 1 -- 172.21.15.119:0/293618397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c620f1bad0 msgr2=0x55c620f272f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.677+0000 7f70f7fff700 1 --2- 172.21.15.119:0/293618397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c620f1bad0 0x55c620f272f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.677+0000 7f70f7fff700 1 -- 172.21.15.119:0/293618397 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f710010ec60 con 0x55c620f20a60 2023-12-07T10:30:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.677+0000 7f70fe59c700 1 --2- 172.21.15.119:0/293618397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c620f1bad0 0x55c620f272f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:30:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.677+0000 7f70f7fff700 1 --2- 172.21.15.119:0/293618397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c620f20a60 0x55c620f24730 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f70e80116f0 tx=0x7f70e80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6e86d97cf473ebf server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:30:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.677+0000 7f70e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:30:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.679+0000 7f70e5ffb700 1 -- 172.21.15.119:0/293618397 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e8022070 con 0x55c620f20a60 2023-12-07T10:30:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.679+0000 7f70e5ffb700 1 -- 172.21.15.119:0/293618397 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f70e8017050 con 0x55c620f20a60 2023-12-07T10:30:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.679+0000 7f71185acb80 1 -- 172.21.15.119:0/293618397 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c620f0e6f0 con 0x55c620f20a60 2023-12-07T10:30:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.679+0000 7f70e5ffb700 1 -- 172.21.15.119:0/293618397 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70e8011e10 con 0x55c620f20a60 2023-12-07T10:30:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.679+0000 7f71185acb80 10 client.27610 fetch_fsmap learned FSMap version 59 2023-12-07T10:30:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.679+0000 7f71185acb80 1 -- 172.21.15.119:0/293618397 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55c620c98d80 con 0x55c620f20a60 2023-12-07T10:30:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.679+0000 7f70e5ffb700 1 -- 172.21.15.119:0/293618397 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c620f0e6f0 con 0x55c620f20a60 2023-12-07T10:30:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.679+0000 7f70e5ffb700 1 -- 172.21.15.119:0/293618397 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 59) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f70e8005540 con 0x55c620f20a60 2023-12-07T10:30:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.679+0000 7f71185acb80 10 client.27610 fetch_fsmap finished waiting for FSMap version 59 2023-12-07T10:30:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.679+0000 7f71185acb80 10 client.27610 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:30:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.679+0000 7f71185acb80 20 client.27610 populate_metadata read hostname 'smithi119' 2023-12-07T10:30:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.679+0000 7f71185acb80 1 --2- 172.21.15.119:0/293618397 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c620f34ef0 0x55c620f37310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.679+0000 7f71185acb80 4 client.27610 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:30:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.679+0000 7f71185acb80 1 -- 172.21.15.119:0/293618397 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c620eea740 con 0x55c620f34ef0 2023-12-07T10:30:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.680+0000 7f70fe59c700 1 --2- 172.21.15.119:0/293618397 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c620f34ef0 0x55c620f37310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.680+0000 7f70fe59c700 1 --2- 172.21.15.119:0/293618397 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c620f34ef0 0x55c620f37310 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9aef7e0d56bdc2df server_cookie=79e64dfa967b7f31 in_seq=0 out_seq=0 2023-12-07T10:30:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.680+0000 7f70e5ffb700 10 client.27610 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:30:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.681+0000 7f70e5ffb700 1 -- 172.21.15.119:0/293618397 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c620eea740 con 0x55c620f34ef0 2023-12-07T10:30:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.681+0000 7f70e5ffb700 10 client.27610 handle_command_reply: tid=0 2023-12-07T10:30:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.681+0000 7f71185acb80 1 client.27610 shutdown 2023-12-07T10:30:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.682+0000 7f71185acb80 1 -- 172.21.15.119:0/293618397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c620f20a60 msgr2=0x55c620f24730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.682+0000 7f71185acb80 1 --2- 172.21.15.119:0/293618397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c620f20a60 0x55c620f24730 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f70e80116f0 tx=0x7f70e80047c0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.682+0000 7f71185acb80 1 -- 172.21.15.119:0/293618397 shutdown_connections 2023-12-07T10:30:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.682+0000 7f71185acb80 1 --2- 172.21.15.119:0/293618397 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c620f34ef0 0x55c620f37310 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.682+0000 7f71185acb80 1 --2- 172.21.15.119:0/293618397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c620f21380 0x55c620f24d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.682+0000 7f71185acb80 1 --2- 172.21.15.119:0/293618397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c620f1bad0 0x55c620f272f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.682+0000 7f71185acb80 1 --2- 172.21.15.119:0/293618397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c620f20a60 0x55c620f24730 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.682+0000 7f70e5ffb700 0 client.27610 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:30:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.682+0000 7f71185acb80 1 -- 172.21.15.119:0/293618397 >> 172.21.15.119:0/293618397 conn(0x55c620e4a300 msgr2=0x55c620f10980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:30:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.682+0000 7f71185acb80 1 -- 172.21.15.119:0/293618397 shutdown_connections 2023-12-07T10:30:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.682+0000 7f71185acb80 1 -- 172.21.15.119:0/293618397 wait complete. 2023-12-07T10:30:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.682+0000 7f71185acb80 20 client.27610 trim_cache size 0 max 16384 2023-12-07T10:30:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.700+0000 7f71185acb80 1 Processor -- start 2023-12-07T10:30:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.700+0000 7f71185acb80 1 -- start start 2023-12-07T10:30:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.700+0000 7f71185acb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c620f1bad0 0x55c620f0f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.700+0000 7f71185acb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c620f20a60 0x55c620f2bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.700+0000 7f71185acb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c620f21380 0x55c620f2c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.700+0000 7f71185acb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c620cee8c0 con 0x55c620f20a60 2023-12-07T10:30:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.700+0000 7f71185acb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c620f25c10 con 0x55c620f21380 2023-12-07T10:30:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.700+0000 7f71185acb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c620f3c5a0 con 0x55c620f1bad0 2023-12-07T10:30:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.700+0000 7f70f7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c620f20a60 0x55c620f2bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.701+0000 7f70fe59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c620f1bad0 0x55c620f0f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.701+0000 7f70fed9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c620f21380 0x55c620f2c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.701+0000 7f70f7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c620f20a60 0x55c620f2bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44646/0 (socket says 172.21.15.119:44646) 2023-12-07T10:30:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.701+0000 7f70f7fff700 1 -- 172.21.15.119:0/778763351 learned_addr learned my addr 172.21.15.119:0/778763351 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:30:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.701+0000 7f70fed9d700 1 -- 172.21.15.119:0/778763351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c620f1bad0 msgr2=0x55c620f0f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.701+0000 7f70fed9d700 1 --2- 172.21.15.119:0/778763351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c620f1bad0 0x55c620f0f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.701+0000 7f70fed9d700 1 -- 172.21.15.119:0/778763351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c620f20a60 msgr2=0x55c620f2bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.701+0000 7f70fed9d700 1 --2- 172.21.15.119:0/778763351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c620f20a60 0x55c620f2bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.701+0000 7f70fed9d700 1 -- 172.21.15.119:0/778763351 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c620c81c00 con 0x55c620f21380 2023-12-07T10:30:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.701+0000 7f70f7fff700 1 --2- 172.21.15.119:0/778763351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c620f20a60 0x55c620f2bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:30:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.701+0000 7f70fed9d700 1 --2- 172.21.15.119:0/778763351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c620f21380 0x55c620f2c310 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f70f80caac0 tx=0x7f70f8099bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9505a221759a6696 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:30:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.701+0000 7f70f57fa700 1 -- 172.21.15.119:0/778763351 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f80b1070 con 0x55c620f21380 2023-12-07T10:30:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.701+0000 7f70f57fa700 1 -- 172.21.15.119:0/778763351 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70f80092e0 con 0x55c620f21380 2023-12-07T10:30:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.701+0000 7f70f57fa700 1 -- 172.21.15.119:0/778763351 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f809a400 con 0x55c620f21380 2023-12-07T10:30:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.702+0000 7f71185acb80 1 -- 172.21.15.119:0/778763351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c620f21380 msgr2=0x55c620f2c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.702+0000 7f71185acb80 1 --2- 172.21.15.119:0/778763351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c620f21380 0x55c620f2c310 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f70f80caac0 tx=0x7f70f8099bf0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.702+0000 7f71185acb80 1 -- 172.21.15.119:0/778763351 shutdown_connections 2023-12-07T10:30:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.702+0000 7f71185acb80 1 --2- 172.21.15.119:0/778763351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c620f21380 0x55c620f2c310 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.702+0000 7f71185acb80 1 --2- 172.21.15.119:0/778763351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c620f1bad0 0x55c620f0f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.702+0000 7f71185acb80 1 --2- 172.21.15.119:0/778763351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c620f20a60 0x55c620f2bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.702+0000 7f71185acb80 1 -- 172.21.15.119:0/778763351 >> 172.21.15.119:0/778763351 conn(0x55c620e4a300 msgr2=0x55c620f270e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:30:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.702+0000 7f71185acb80 1 -- 172.21.15.119:0/778763351 shutdown_connections 2023-12-07T10:30:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.702+0000 7f71185acb80 1 -- 172.21.15.119:0/778763351 wait complete. 2023-12-07T10:30:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.703+0000 7f71185acb80 1 -- start start 2023-12-07T10:30:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.703+0000 7f71185acb80 1 Processor -- start 2023-12-07T10:30:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.703+0000 7f71185acb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c620f1bad0 0x55c620f29db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.703+0000 7f71185acb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c620f20a60 0x55c620f2a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.703+0000 7f71185acb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c620f21380 0x55c620f8a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.703+0000 7f71185acb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c620f2aaf0 con 0x55c620f21380 2023-12-07T10:30:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.703+0000 7f71185acb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c620f2ac60 con 0x55c620f1bad0 2023-12-07T10:30:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.703+0000 7f71185acb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c620f8a9b0 con 0x55c620f20a60 2023-12-07T10:30:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.704+0000 7f71185acb80 10 client.0 _ll_register_callbacks cb 0x55c620e108e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:30:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.704+0000 7f70f7fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c620f20a60 0x55c620f2a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.704+0000 7f70fe59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c620f1bad0 0x55c620f29db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.704+0000 7f70fe59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c620f1bad0 0x55c620f29db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:59170/0 (socket says 172.21.15.119:59170) 2023-12-07T10:30:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.704+0000 7f70fe59c700 1 -- 172.21.15.119:0/3246591043 learned_addr learned my addr 172.21.15.119:0/3246591043 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:30:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.704+0000 7f70fed9d700 1 --2- 172.21.15.119:0/3246591043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c620f21380 0x55c620f8a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.704+0000 7f70fe59c700 1 -- 172.21.15.119:0/3246591043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c620f20a60 msgr2=0x55c620f2a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.704+0000 7f70fe59c700 1 --2- 172.21.15.119:0/3246591043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c620f20a60 0x55c620f2a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.704+0000 7f70fe59c700 1 -- 172.21.15.119:0/3246591043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c620f21380 msgr2=0x55c620f8a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.704+0000 7f70fe59c700 1 --2- 172.21.15.119:0/3246591043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c620f21380 0x55c620f8a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.704+0000 7f70fe59c700 1 -- 172.21.15.119:0/3246591043 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c620c98d80 con 0x55c620f1bad0 2023-12-07T10:30:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.704+0000 7f70fed9d700 1 --2- 172.21.15.119:0/3246591043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c620f21380 0x55c620f8a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:30:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.704+0000 7f70fe59c700 1 --2- 172.21.15.119:0/3246591043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c620f1bad0 0x55c620f29db0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f70f0005320 tx=0x7f70f00057e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e356204ec9734ac server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:30:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.704+0000 7f70e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:30:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.704+0000 7f70e5ffb700 1 -- 172.21.15.119:0/3246591043 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0033040 con 0x55c620f1bad0 2023-12-07T10:30:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.704+0000 7f71185acb80 1 -- 172.21.15.119:0/3246591043 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70f001e040 con 0x55c620f1bad0 2023-12-07T10:30:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.704+0000 7f70e5ffb700 1 -- 172.21.15.119:0/3246591043 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f70f000b9d0 con 0x55c620f1bad0 2023-12-07T10:30:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.705+0000 7f70e5ffb700 1 -- 172.21.15.119:0/3246591043 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70f001d040 con 0x55c620f1bad0 2023-12-07T10:30:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.705+0000 7f71185acb80 10 client.27632 fetch_fsmap learned FSMap version 59 2023-12-07T10:30:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.705+0000 7f71185acb80 1 -- 172.21.15.119:0/3246591043 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x7f71001a6870 con 0x55c620f1bad0 2023-12-07T10:30:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.705+0000 7f70e5ffb700 1 -- 172.21.15.119:0/3246591043 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70f001e040 con 0x55c620f1bad0 2023-12-07T10:30:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.705+0000 7f70e5ffb700 1 -- 172.21.15.119:0/3246591043 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 59) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f70f0004590 con 0x55c620f1bad0 2023-12-07T10:30:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.705+0000 7f71185acb80 10 client.27632 fetch_fsmap finished waiting for FSMap version 59 2023-12-07T10:30:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.705+0000 7f71185acb80 10 client.27632 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:30:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.705+0000 7f71185acb80 20 client.27632 populate_metadata read hostname 'smithi119' 2023-12-07T10:30:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.705+0000 7f71185acb80 1 --2- 172.21.15.119:0/3246591043 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c620f8bd70 0x55c620f8e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.705+0000 7f71185acb80 4 client.27632 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:30:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.705+0000 7f71185acb80 1 -- 172.21.15.119:0/3246591043 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c620e35f70 con 0x55c620f8bd70 2023-12-07T10:30:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.705+0000 7f70f7fff700 1 --2- 172.21.15.119:0/3246591043 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c620f8bd70 0x55c620f8e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.706+0000 7f70e5ffb700 10 client.27632 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:30:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.706+0000 7f70f7fff700 1 --2- 172.21.15.119:0/3246591043 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c620f8bd70 0x55c620f8e190 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a47fe3f2febc288 server_cookie=c94bdd2c021e4e7 in_seq=0 out_seq=0 2023-12-07T10:30:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.751+0000 7f70e5ffb700 1 -- 172.21.15.119:0/3246591043 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c620e35f70 con 0x55c620f8bd70 2023-12-07T10:30:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.751+0000 7f70e5ffb700 10 client.27632 handle_command_reply: tid=0 2023-12-07T10:30:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.751+0000 7f71185acb80 1 client.27632 shutdown 2023-12-07T10:30:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.751+0000 7f71185acb80 1 -- 172.21.15.119:0/3246591043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c620f1bad0 msgr2=0x55c620f29db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.751+0000 7f71185acb80 1 --2- 172.21.15.119:0/3246591043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c620f1bad0 0x55c620f29db0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f70f0005320 tx=0x7f70f00057e0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.751+0000 7f71185acb80 1 -- 172.21.15.119:0/3246591043 shutdown_connections 2023-12-07T10:30:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.751+0000 7f71185acb80 1 --2- 172.21.15.119:0/3246591043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c620f21380 0x55c620f8a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.751+0000 7f71185acb80 1 --2- 172.21.15.119:0/3246591043 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c620f8bd70 0x55c620f8e190 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.751+0000 7f71185acb80 1 --2- 172.21.15.119:0/3246591043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c620f1bad0 0x55c620f29db0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.751+0000 7f70e5ffb700 0 client.27632 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:30:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.751+0000 7f71185acb80 1 --2- 172.21.15.119:0/3246591043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c620f20a60 0x55c620f2a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.751+0000 7f71185acb80 1 -- 172.21.15.119:0/3246591043 >> 172.21.15.119:0/3246591043 conn(0x55c620e4a300 msgr2=0x55c620f1e960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:30:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.751+0000 7f71185acb80 1 -- 172.21.15.119:0/3246591043 shutdown_connections 2023-12-07T10:30:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.751+0000 7f71185acb80 1 -- 172.21.15.119:0/3246591043 wait complete. 2023-12-07T10:30:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.752+0000 7f71185acb80 20 client.27632 trim_cache size 0 max 16384 2023-12-07T10:30:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:30:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:30:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:30:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:30:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.752+0000 7f7104bff700 1 -- 172.21.15.119:0/651025935 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f70ec0a1d00 msgr2=0x7f70ec0a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.752+0000 7f7104bff700 1 --2- 172.21.15.119:0/651025935 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f70ec0a1d00 0x7f70ec0a41c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7100101bb0 tx=0x7f70e800b410 comp rx=0 tx=0).stop 2023-12-07T10:30:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.752+0000 7f7104bff700 1 -- 172.21.15.119:0/651025935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7100106a80 msgr2=0x7f7100100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.752+0000 7f7104bff700 1 --2- 172.21.15.119:0/651025935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7100106a80 0x7f7100100f00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f70f800efc0 tx=0x7f70f800c3a0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.752+0000 7f7104bff700 1 -- 172.21.15.119:0/651025935 shutdown_connections 2023-12-07T10:30:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.752+0000 7f7104bff700 1 --2- 172.21.15.119:0/651025935 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f70ec0a1d00 0x7f70ec0a41c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.752+0000 7f7104bff700 1 --2- 172.21.15.119:0/651025935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7100106a80 0x7f7100100f00 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.752+0000 7f7104bff700 1 --2- 172.21.15.119:0/651025935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7100068ce0 0x7f7100100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.752+0000 7f7104bff700 1 --2- 172.21.15.119:0/651025935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71001060c0 0x7f71001009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:37.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.752+0000 7f7104bff700 1 -- 172.21.15.119:0/651025935 >> 172.21.15.119:0/651025935 conn(0x7f71000775b0 msgr2=0x7f71000fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:30:37.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.752+0000 7f7104bff700 1 -- 172.21.15.119:0/651025935 shutdown_connections 2023-12-07T10:30:37.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:37.752+0000 7f7104bff700 1 -- 172.21.15.119:0/651025935 wait complete. 2023-12-07T10:30:38.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:38 smithi119 ceph-mon[97908]: pgmap v317: 129 pgs: 129 active+clean; 527 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 318 op/s 2023-12-07T10:30:38.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:38 smithi159 ceph-mon[134726]: pgmap v317: 129 pgs: 129 active+clean; 527 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 318 op/s 2023-12-07T10:30:38.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:38 smithi153 ceph-mon[133625]: pgmap v317: 129 pgs: 129 active+clean; 527 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 318 op/s 2023-12-07T10:30:38.573 INFO:tasks.workunit.client.0.smithi119.stdout: 1 736173 7.23 MB/sec execute 411 sec latency 26.505 ms 2023-12-07T10:30:39.573 INFO:tasks.workunit.client.0.smithi119.stdout: 1 738104 7.23 MB/sec execute 412 sec latency 16.688 ms 2023-12-07T10:30:40.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:40 smithi119 ceph-mon[97908]: pgmap v318: 129 pgs: 129 active+clean; 527 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 201 op/s 2023-12-07T10:30:40.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:40 smithi159 ceph-mon[134726]: pgmap v318: 129 pgs: 129 active+clean; 527 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 201 op/s 2023-12-07T10:30:40.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:40 smithi153 ceph-mon[133625]: pgmap v318: 129 pgs: 129 active+clean; 527 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 201 op/s 2023-12-07T10:30:40.573 INFO:tasks.workunit.client.0.smithi119.stdout: 1 739328 7.23 MB/sec execute 413 sec latency 13.974 ms 2023-12-07T10:30:41.573 INFO:tasks.workunit.client.0.smithi119.stdout: 1 741181 7.23 MB/sec execute 414 sec latency 16.448 ms 2023-12-07T10:30:42.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:41 smithi119 ceph-mon[97908]: pgmap v319: 129 pgs: 129 active+clean; 535 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 298 op/s 2023-12-07T10:30:42.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:41 smithi159 ceph-mon[134726]: pgmap v319: 129 pgs: 129 active+clean; 535 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 298 op/s 2023-12-07T10:30:42.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:41 smithi153 ceph-mon[133625]: pgmap v319: 129 pgs: 129 active+clean; 535 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 298 op/s 2023-12-07T10:30:42.573 INFO:tasks.workunit.client.0.smithi119.stdout: 1 742561 7.24 MB/sec execute 415 sec latency 65.274 ms 2023-12-07T10:30:43.573 INFO:tasks.workunit.client.0.smithi119.stdout: 1 743814 7.24 MB/sec execute 416 sec latency 23.036 ms 2023-12-07T10:30:44.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:44 smithi119 ceph-mon[97908]: pgmap v320: 129 pgs: 129 active+clean; 535 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 235 op/s 2023-12-07T10:30:44.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:44 smithi159 ceph-mon[134726]: pgmap v320: 129 pgs: 129 active+clean; 535 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 235 op/s 2023-12-07T10:30:44.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:44 smithi153 ceph-mon[133625]: pgmap v320: 129 pgs: 129 active+clean; 535 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 235 op/s 2023-12-07T10:30:44.573 INFO:tasks.workunit.client.0.smithi119.stdout: 1 745832 7.25 MB/sec execute 417 sec latency 16.386 ms 2023-12-07T10:30:45.573 INFO:tasks.workunit.client.0.smithi119.stdout: 1 746864 7.24 MB/sec execute 418 sec latency 15.553 ms 2023-12-07T10:30:46.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:46 smithi119 ceph-mon[97908]: pgmap v321: 129 pgs: 129 active+clean; 535 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 200 op/s 2023-12-07T10:30:46.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:46 smithi159 ceph-mon[134726]: pgmap v321: 129 pgs: 129 active+clean; 535 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 200 op/s 2023-12-07T10:30:46.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:46 smithi153 ceph-mon[133625]: pgmap v321: 129 pgs: 129 active+clean; 535 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 200 op/s 2023-12-07T10:30:46.573 INFO:tasks.workunit.client.0.smithi119.stdout: 1 748844 7.25 MB/sec execute 419 sec latency 19.879 ms 2023-12-07T10:30:47.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:47 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:30:47.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:47 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:30:47.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:47 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:30:47.573 INFO:tasks.workunit.client.0.smithi119.stdout: 1 749793 7.25 MB/sec execute 420 sec latency 66.412 ms 2023-12-07T10:30:48.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:48 smithi119 ceph-mon[97908]: pgmap v322: 129 pgs: 129 active+clean; 543 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 302 op/s 2023-12-07T10:30:48.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:48 smithi159 ceph-mon[134726]: pgmap v322: 129 pgs: 129 active+clean; 543 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 302 op/s 2023-12-07T10:30:48.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:48 smithi153 ceph-mon[133625]: pgmap v322: 129 pgs: 129 active+clean; 543 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 302 op/s 2023-12-07T10:30:48.573 INFO:tasks.workunit.client.0.smithi119.stdout: 1 750911 7.24 MB/sec execute 421 sec latency 52.096 ms 2023-12-07T10:30:49.573 INFO:tasks.workunit.client.0.smithi119.stdout: 1 752761 7.25 MB/sec execute 422 sec latency 18.806 ms 2023-12-07T10:30:50.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:50 smithi119 ceph-mon[97908]: pgmap v323: 129 pgs: 129 active+clean; 543 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 208 op/s 2023-12-07T10:30:50.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:50 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:30:50.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:50 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:30:50.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:50 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:50.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:50 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:50.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:50 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:50.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:50 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:50.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:50 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:30:50.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:50 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:30:50.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:50 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:30:50.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:50 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:30:50.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:50 smithi159 ceph-mon[134726]: pgmap v323: 129 pgs: 129 active+clean; 543 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 208 op/s 2023-12-07T10:30:50.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:50 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:30:50.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:50 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:30:50.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:50 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:50.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:50 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:50.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:50 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:50.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:50 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:50.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:50 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:30:50.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:50 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:30:50.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:50 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:30:50.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:50 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:30:50.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:50 smithi153 ceph-mon[133625]: pgmap v323: 129 pgs: 129 active+clean; 543 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 208 op/s 2023-12-07T10:30:50.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:50 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:30:50.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:50 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:30:50.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:50 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:50.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:50 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:50.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:50 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:50.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:50 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:50.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:50 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:30:50.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:50 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:30:50.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:50 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:30:50.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:50 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:30:50.573 INFO:tasks.workunit.client.0.smithi119.stdout: 1 753642 7.24 MB/sec execute 423 sec latency 17.367 ms 2023-12-07T10:30:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.917+0000 7f0a7cfff700 1 -- 172.21.15.119:0/3319860207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a78109720 msgr2=0x7f0a78109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.917+0000 7f0a7cfff700 1 --2- 172.21.15.119:0/3319860207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a78109720 0x7f0a78109b00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009990 tx=0x7f0a68009ca0 comp rx=0 tx=0).stop 2023-12-07T10:30:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.917+0000 7f0a7cfff700 1 -- 172.21.15.119:0/3319860207 shutdown_connections 2023-12-07T10:30:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.917+0000 7f0a7cfff700 1 --2- 172.21.15.119:0/3319860207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a781040d0 0x7f0a781105e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.917+0000 7f0a7cfff700 1 --2- 172.21.15.119:0/3319860207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a78109720 0x7f0a78109b00 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.917+0000 7f0a7cfff700 1 --2- 172.21.15.119:0/3319860207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a78103710 0x7f0a78103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.917+0000 7f0a7cfff700 1 -- 172.21.15.119:0/3319860207 >> 172.21.15.119:0/3319860207 conn(0x7f0a780ff1c0 msgr2=0x7f0a781015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:30:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.917+0000 7f0a7cfff700 1 -- 172.21.15.119:0/3319860207 shutdown_connections 2023-12-07T10:30:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.917+0000 7f0a7cfff700 1 -- 172.21.15.119:0/3319860207 wait complete. 2023-12-07T10:30:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.918+0000 7f0a7cfff700 1 Processor -- start 2023-12-07T10:30:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.918+0000 7f0a7cfff700 1 -- start start 2023-12-07T10:30:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.919+0000 7f0a7cfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a78103710 0x7f0a7819f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.919+0000 7f0a7cfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a781040d0 0x7f0a7819fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.919+0000 7f0a7cfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a78109720 0x7f0a781999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.919+0000 7f0a7cfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7819a190 con 0x7f0a78103710 2023-12-07T10:30:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.919+0000 7f0a7cfff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7819a300 con 0x7f0a78109720 2023-12-07T10:30:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.919+0000 7f0a7cfff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7819a4c0 con 0x7f0a781040d0 2023-12-07T10:30:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.919+0000 7f0a7659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a78103710 0x7f0a7819f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.919+0000 7f0a76d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a78109720 0x7f0a781999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.919+0000 7f0a76d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a78109720 0x7f0a781999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:34576/0 (socket says 172.21.15.119:34576) 2023-12-07T10:30:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.919+0000 7f0a76d9d700 1 -- 172.21.15.119:0/2331928321 learned_addr learned my addr 172.21.15.119:0/2331928321 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:30:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.919+0000 7f0a7659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a78103710 0x7f0a7819f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34030/0 (socket says 172.21.15.119:34030) 2023-12-07T10:30:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.919+0000 7f0a6ffff700 1 --2- 172.21.15.119:0/2331928321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a781040d0 0x7f0a7819fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.920+0000 7f0a76d9d700 1 -- 172.21.15.119:0/2331928321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a781040d0 msgr2=0x7f0a7819fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.920+0000 7f0a76d9d700 1 --2- 172.21.15.119:0/2331928321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a781040d0 0x7f0a7819fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.920+0000 7f0a76d9d700 1 -- 172.21.15.119:0/2331928321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a78103710 msgr2=0x7f0a7819f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.920+0000 7f0a76d9d700 1 --2- 172.21.15.119:0/2331928321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a78103710 0x7f0a7819f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.920+0000 7f0a76d9d700 1 -- 172.21.15.119:0/2331928321 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a680095f0 con 0x7f0a78109720 2023-12-07T10:30:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.920+0000 7f0a76d9d700 1 --2- 172.21.15.119:0/2331928321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a78109720 0x7f0a781999c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000ef30 tx=0x7f0a7000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:30:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.920+0000 7f0a6f7fe700 1 -- 172.21.15.119:0/2331928321 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a70018070 con 0x7f0a78109720 2023-12-07T10:30:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.920+0000 7f0a6f7fe700 1 -- 172.21.15.119:0/2331928321 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a7000f040 con 0x7f0a78109720 2023-12-07T10:30:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.920+0000 7f0a7cfff700 1 -- 172.21.15.119:0/2331928321 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a7819a740 con 0x7f0a78109720 2023-12-07T10:30:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.920+0000 7f0a6f7fe700 1 -- 172.21.15.119:0/2331928321 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7000cd00 con 0x7f0a78109720 2023-12-07T10:30:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.921+0000 7f0a7cfff700 1 -- 172.21.15.119:0/2331928321 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a781a6850 con 0x7f0a78109720 2023-12-07T10:30:50.924 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.924+0000 7f0a6f7fe700 1 -- 172.21.15.119:0/2331928321 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f0a70003a00 con 0x7f0a78109720 2023-12-07T10:30:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.925+0000 7f0a909a7b80 1 Processor -- start 2023-12-07T10:30:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.925+0000 7f0a909a7b80 1 -- start start 2023-12-07T10:30:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.925+0000 7f0a909a7b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559393bea60 0x5559393bee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.925+0000 7f0a909a7b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559393bf380 0x5559393b9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.925+0000 7f0a909a7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559393b9ad0 0x5559393b9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.926+0000 7f0a909a7b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55593932d760 con 0x5559393bea60 2023-12-07T10:30:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.926+0000 7f0a909a7b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555939314740 con 0x5559393bf380 2023-12-07T10:30:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.926+0000 7f0a909a7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555939312060 con 0x5559393b9ad0 2023-12-07T10:30:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.926+0000 7f0a7659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559393bf380 0x5559393b9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.926+0000 7f0a7659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559393bf380 0x5559393b9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:34586/0 (socket says 172.21.15.119:34586) 2023-12-07T10:30:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.926+0000 7f0a7659c700 1 -- 172.21.15.119:0/2479865575 learned_addr learned my addr 172.21.15.119:0/2479865575 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:30:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.926+0000 7f0a6f7fe700 1 --2- 172.21.15.119:0/2331928321 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0a580a1d00 0x7f0a580a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.926+0000 7f0a7659c700 1 -- 172.21.15.119:0/2479865575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559393b9ad0 msgr2=0x5559393b9f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-07T10:30:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.926+0000 7f0a7659c700 1 --2- 172.21.15.119:0/2479865575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559393b9ad0 0x5559393b9f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.926+0000 7f0a7659c700 1 -- 172.21.15.119:0/2479865575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559393bea60 msgr2=0x5559393bee40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-07T10:30:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.926+0000 7f0a7659c700 1 --2- 172.21.15.119:0/2479865575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559393bea60 0x5559393bee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.926+0000 7f0a7659c700 1 -- 172.21.15.119:0/2479865575 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55593918a5d0 con 0x5559393bf380 2023-12-07T10:30:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.927+0000 7f0a6f7fe700 1 -- 172.21.15.119:0/2331928321 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0a700cd840 con 0x7f0a78109720 2023-12-07T10:30:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.927+0000 7f0a76d9d700 1 --2- 172.21.15.119:0/2331928321 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0a580a1d00 0x7f0a580a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.927+0000 7f0a7659c700 1 --2- 172.21.15.119:0/2479865575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559393bf380 0x5559393b9540 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0a68005cb0 tx=0x7f0a6800b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=786f523cc744c06a server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:30:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.927+0000 7f0a6cff9700 1 -- 172.21.15.119:0/2479865575 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6801d070 con 0x5559393bf380 2023-12-07T10:30:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.927+0000 7f0a6cff9700 1 -- 172.21.15.119:0/2479865575 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a6801e040 con 0x5559393bf380 2023-12-07T10:30:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.927+0000 7f0a6cff9700 1 -- 172.21.15.119:0/2479865575 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68004a30 con 0x5559393bf380 2023-12-07T10:30:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.928+0000 7f0a909a7b80 1 -- 172.21.15.119:0/2479865575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559393bf380 msgr2=0x5559393b9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.928+0000 7f0a909a7b80 1 --2- 172.21.15.119:0/2479865575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559393bf380 0x5559393b9540 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0a68005cb0 tx=0x7f0a6800b610 comp rx=0 tx=0).stop 2023-12-07T10:30:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.928+0000 7f0a76d9d700 1 --2- 172.21.15.119:0/2331928321 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0a580a1d00 0x7f0a580a41c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0a700cabe0 tx=0x7f0a700092e0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:30:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.928+0000 7f0a909a7b80 1 -- 172.21.15.119:0/2479865575 shutdown_connections 2023-12-07T10:30:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.928+0000 7f0a909a7b80 1 --2- 172.21.15.119:0/2479865575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559393b9ad0 0x5559393b9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.928+0000 7f0a909a7b80 1 --2- 172.21.15.119:0/2479865575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559393bea60 0x5559393bee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.928+0000 7f0a909a7b80 1 --2- 172.21.15.119:0/2479865575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559393bf380 0x5559393b9540 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.928+0000 7f0a909a7b80 1 -- 172.21.15.119:0/2479865575 >> 172.21.15.119:0/2479865575 conn(0x5559392e8300 msgr2=0x5559393b3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:30:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.929+0000 7f0a909a7b80 1 -- 172.21.15.119:0/2479865575 shutdown_connections 2023-12-07T10:30:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.929+0000 7f0a909a7b80 1 -- 172.21.15.119:0/2479865575 wait complete. 2023-12-07T10:30:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.929+0000 7f0a909a7b80 1 -- start start 2023-12-07T10:30:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.930+0000 7f0a909a7b80 1 Processor -- start 2023-12-07T10:30:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.930+0000 7f0a909a7b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559393b9ad0 0x5559393c2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.931+0000 7f0a909a7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559393bea60 0x5559393c2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.931+0000 7f0a909a7b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559393bf380 0x5559393c3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.931+0000 7f0a909a7b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55593930d950 con 0x5559393b9ad0 2023-12-07T10:30:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.931+0000 7f0a909a7b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559392c7500 con 0x5559393bf380 2023-12-07T10:30:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.931+0000 7f0a909a7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559392c71a0 con 0x5559393bea60 2023-12-07T10:30:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.931+0000 7f0a909a7b80 10 client.0 _ll_register_callbacks cb 0x5559392ae8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:30:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.931+0000 7f0a7659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559393bea60 0x5559393c2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.931+0000 7f0a6ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559393b9ad0 0x5559393c2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.931+0000 7f0a6ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559393b9ad0 0x5559393c2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34048/0 (socket says 172.21.15.119:34048) 2023-12-07T10:30:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.931+0000 7f0a6ffff700 1 -- 172.21.15.119:0/2715738754 learned_addr learned my addr 172.21.15.119:0/2715738754 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:30:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.931+0000 7f0a6ffff700 1 --2- 172.21.15.119:0/2715738754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559393bf380 0x5559393c3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.932+0000 7f0a6ffff700 1 -- 172.21.15.119:0/2715738754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559393bea60 msgr2=0x5559393c2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.932+0000 7f0a6ffff700 1 --2- 172.21.15.119:0/2715738754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559393bea60 0x5559393c2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.932+0000 7f0a6ffff700 1 -- 172.21.15.119:0/2715738754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559393b9ad0 msgr2=0x5559393c2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.932+0000 7f0a6ffff700 1 --2- 172.21.15.119:0/2715738754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559393b9ad0 0x5559393c2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.932+0000 7f0a6ffff700 1 -- 172.21.15.119:0/2715738754 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a78110ee0 con 0x5559393bf380 2023-12-07T10:30:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.932+0000 7f0a6ffff700 1 --2- 172.21.15.119:0/2715738754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559393bf380 0x5559393c3200 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0a60011f30 tx=0x7f0a60004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8201ccd700598c0c server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:30:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.932+0000 7f0a65ffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:30:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.934+0000 7f0a65ffb700 1 -- 172.21.15.119:0/2715738754 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6000dc00 con 0x5559393bf380 2023-12-07T10:30:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.934+0000 7f0a65ffb700 1 -- 172.21.15.119:0/2715738754 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0a6000dd60 con 0x5559393bf380 2023-12-07T10:30:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.934+0000 7f0a909a7b80 1 -- 172.21.15.119:0/2715738754 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5559393ac6f0 con 0x5559393bf380 2023-12-07T10:30:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.934+0000 7f0a65ffb700 1 -- 172.21.15.119:0/2715738754 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a6000ae60 con 0x5559393bf380 2023-12-07T10:30:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.934+0000 7f0a909a7b80 10 client.27644 fetch_fsmap learned FSMap version 59 2023-12-07T10:30:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.934+0000 7f0a909a7b80 1 -- 172.21.15.119:0/2715738754 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x555939136d80 con 0x5559393bf380 2023-12-07T10:30:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.935+0000 7f0a65ffb700 1 -- 172.21.15.119:0/2715738754 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5559393ac6f0 con 0x5559393bf380 2023-12-07T10:30:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.935+0000 7f0a65ffb700 1 -- 172.21.15.119:0/2715738754 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 59) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0a60003be0 con 0x5559393bf380 2023-12-07T10:30:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.935+0000 7f0a909a7b80 10 client.27644 fetch_fsmap finished waiting for FSMap version 59 2023-12-07T10:30:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.935+0000 7f0a909a7b80 10 client.27644 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:30:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.935+0000 7f0a909a7b80 20 client.27644 populate_metadata read hostname 'smithi119' 2023-12-07T10:30:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.935+0000 7f0a909a7b80 1 --2- 172.21.15.119:0/2715738754 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5559393d2eb0 0x5559393d52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.935+0000 7f0a909a7b80 4 client.27644 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:30:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.935+0000 7f0a909a7b80 1 -- 172.21.15.119:0/2715738754 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555939388740 con 0x5559393d2eb0 2023-12-07T10:30:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.935+0000 7f0a7659c700 1 --2- 172.21.15.119:0/2715738754 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5559393d2eb0 0x5559393d52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.936+0000 7f0a65ffb700 10 client.27644 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:30:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.936+0000 7f0a7659c700 1 --2- 172.21.15.119:0/2715738754 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5559393d2eb0 0x5559393d52d0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cde21a903709bb5 server_cookie=f4f547d652ad54e8 in_seq=0 out_seq=0 2023-12-07T10:30:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.937+0000 7f0a65ffb700 1 -- 172.21.15.119:0/2715738754 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555939388740 con 0x5559393d2eb0 2023-12-07T10:30:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.937+0000 7f0a65ffb700 10 client.27644 handle_command_reply: tid=0 2023-12-07T10:30:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.937+0000 7f0a909a7b80 1 client.27644 shutdown 2023-12-07T10:30:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.937+0000 7f0a909a7b80 1 -- 172.21.15.119:0/2715738754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559393bf380 msgr2=0x5559393c3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.937+0000 7f0a909a7b80 1 --2- 172.21.15.119:0/2715738754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559393bf380 0x5559393c3200 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0a60011f30 tx=0x7f0a60004750 comp rx=0 tx=0).stop 2023-12-07T10:30:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.937+0000 7f0a909a7b80 1 -- 172.21.15.119:0/2715738754 shutdown_connections 2023-12-07T10:30:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.937+0000 7f0a909a7b80 1 --2- 172.21.15.119:0/2715738754 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5559393d2eb0 0x5559393d52d0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.937+0000 7f0a909a7b80 1 --2- 172.21.15.119:0/2715738754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559393bf380 0x5559393c3200 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.937+0000 7f0a909a7b80 1 --2- 172.21.15.119:0/2715738754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559393b9ad0 0x5559393c2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.937+0000 7f0a909a7b80 1 --2- 172.21.15.119:0/2715738754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559393bea60 0x5559393c2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.937+0000 7f0a65ffb700 0 client.27644 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:30:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.937+0000 7f0a909a7b80 1 -- 172.21.15.119:0/2715738754 >> 172.21.15.119:0/2715738754 conn(0x5559392e8300 msgr2=0x5559393ae980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:30:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.937+0000 7f0a909a7b80 1 -- 172.21.15.119:0/2715738754 shutdown_connections 2023-12-07T10:30:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.937+0000 7f0a909a7b80 1 -- 172.21.15.119:0/2715738754 wait complete. 2023-12-07T10:30:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.938+0000 7f0a909a7b80 20 client.27644 trim_cache size 0 max 16384 2023-12-07T10:30:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.955+0000 7f0a909a7b80 1 Processor -- start 2023-12-07T10:30:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.955+0000 7f0a909a7b80 1 -- start start 2023-12-07T10:30:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.956+0000 7f0a909a7b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559393b9ad0 0x5559393bee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.956+0000 7f0a909a7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559393bea60 0x5559393d9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.956+0000 7f0a909a7b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559393bf380 0x5559393c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.956+0000 7f0a909a7b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55593918c8c0 con 0x5559393b9ad0 2023-12-07T10:30:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.956+0000 7f0a909a7b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559393d0540 con 0x5559393bf380 2023-12-07T10:30:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.956+0000 7f0a909a7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559393c3d30 con 0x5559393bea60 2023-12-07T10:30:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.956+0000 7f0a7659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559393bea60 0x5559393d9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.956+0000 7f0a6ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559393b9ad0 0x5559393bee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.956+0000 7f0a7659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559393bea60 0x5559393d9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:53486/0 (socket says 172.21.15.119:53486) 2023-12-07T10:30:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.956+0000 7f0a7659c700 1 -- 172.21.15.119:0/2558268917 learned_addr learned my addr 172.21.15.119:0/2558268917 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:30:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.957+0000 7f0a7659c700 1 -- 172.21.15.119:0/2558268917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559393bf380 msgr2=0x5559393c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.957+0000 7f0a7659c700 1 --2- 172.21.15.119:0/2558268917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559393bf380 0x5559393c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.957+0000 7f0a7659c700 1 -- 172.21.15.119:0/2558268917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559393b9ad0 msgr2=0x5559393bee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.957+0000 7f0a7659c700 1 --2- 172.21.15.119:0/2558268917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559393b9ad0 0x5559393bee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.957+0000 7f0a7659c700 1 -- 172.21.15.119:0/2558268917 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55593911fc00 con 0x5559393bea60 2023-12-07T10:30:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.957+0000 7f0a6ffff700 1 --2- 172.21.15.119:0/2558268917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559393b9ad0 0x5559393bee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:30:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.957+0000 7f0a7659c700 1 --2- 172.21.15.119:0/2558268917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559393bea60 0x5559393d9c30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0a680098e0 tx=0x7f0a680058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88d7c19df6777f1 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:30:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.957+0000 7f0a6d7fa700 1 -- 172.21.15.119:0/2558268917 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68037070 con 0x5559393bea60 2023-12-07T10:30:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.957+0000 7f0a6d7fa700 1 -- 172.21.15.119:0/2558268917 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a68033780 con 0x5559393bea60 2023-12-07T10:30:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.957+0000 7f0a6d7fa700 1 -- 172.21.15.119:0/2558268917 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68031040 con 0x5559393bea60 2023-12-07T10:30:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.958+0000 7f0a909a7b80 1 -- 172.21.15.119:0/2558268917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559393bea60 msgr2=0x5559393d9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.958+0000 7f0a909a7b80 1 --2- 172.21.15.119:0/2558268917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559393bea60 0x5559393d9c30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0a680098e0 tx=0x7f0a680058e0 comp rx=0 tx=0).stop 2023-12-07T10:30:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.958+0000 7f0a909a7b80 1 -- 172.21.15.119:0/2558268917 shutdown_connections 2023-12-07T10:30:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.958+0000 7f0a909a7b80 1 --2- 172.21.15.119:0/2558268917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559393bf380 0x5559393c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.958+0000 7f0a909a7b80 1 --2- 172.21.15.119:0/2558268917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559393b9ad0 0x5559393bee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.958+0000 7f0a909a7b80 1 --2- 172.21.15.119:0/2558268917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559393bea60 0x5559393d9c30 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:50.960 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.958+0000 7f0a909a7b80 1 -- 172.21.15.119:0/2558268917 >> 172.21.15.119:0/2558268917 conn(0x5559392e8300 msgr2=0x5559393bbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:30:50.960 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.958+0000 7f0a909a7b80 1 -- 172.21.15.119:0/2558268917 shutdown_connections 2023-12-07T10:30:50.960 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.958+0000 7f0a909a7b80 1 -- 172.21.15.119:0/2558268917 wait complete. 2023-12-07T10:30:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.959+0000 7f0a909a7b80 1 -- start start 2023-12-07T10:30:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.959+0000 7f0a909a7b80 1 Processor -- start 2023-12-07T10:30:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.959+0000 7f0a909a7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559393b9ad0 0x5559393b8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.959+0000 7f0a909a7b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559393bea60 0x5559393b8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.959+0000 7f0a909a7b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559393bf380 0x5559394273a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.959+0000 7f0a909a7b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559393b9750 con 0x5559393bf380 2023-12-07T10:30:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.959+0000 7f0a909a7b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559394278e0 con 0x5559393bea60 2023-12-07T10:30:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.959+0000 7f0a909a7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555939427a20 con 0x5559393b9ad0 2023-12-07T10:30:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.960+0000 7f0a909a7b80 10 client.0 _ll_register_callbacks cb 0x5559392ae8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:30:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.960+0000 7f0a6ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559393b9ad0 0x5559393b8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.960+0000 7f0a6ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559393bf380 0x5559394273a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.960+0000 7f0a7659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559393bea60 0x5559393b8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.960+0000 7f0a7659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559393bea60 0x5559393b8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:34614/0 (socket says 172.21.15.119:34614) 2023-12-07T10:30:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.960+0000 7f0a6ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559393b9ad0 0x5559393b8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:53496/0 (socket says 172.21.15.119:53496) 2023-12-07T10:30:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.960+0000 7f0a7659c700 1 -- 172.21.15.119:0/2103814948 learned_addr learned my addr 172.21.15.119:0/2103814948 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:30:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.960+0000 7f0a7659c700 1 -- 172.21.15.119:0/2103814948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559393b9ad0 msgr2=0x5559393b8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.960+0000 7f0a7659c700 1 --2- 172.21.15.119:0/2103814948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559393b9ad0 0x5559393b8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.960+0000 7f0a7659c700 1 -- 172.21.15.119:0/2103814948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559393bf380 msgr2=0x5559394273a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:30:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.960+0000 7f0a7659c700 1 --2- 172.21.15.119:0/2103814948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559393bf380 0x5559394273a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.960+0000 7f0a7659c700 1 -- 172.21.15.119:0/2103814948 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55593918a5d0 con 0x5559393bea60 2023-12-07T10:30:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.961+0000 7f0a7659c700 1 --2- 172.21.15.119:0/2103814948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559393bea60 0x5559393b8e40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0a68005950 tx=0x7f0a68033f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ab5ee18916e1c1a server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:30:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.961+0000 7f0a65ffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:30:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.961+0000 7f0a65ffb700 1 -- 172.21.15.119:0/2103814948 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68031070 con 0x5559393bea60 2023-12-07T10:30:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.961+0000 7f0a909a7b80 1 -- 172.21.15.119:0/2103814948 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a6801e040 con 0x5559393bea60 2023-12-07T10:30:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.961+0000 7f0a65ffb700 1 -- 172.21.15.119:0/2103814948 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0a68033af0 con 0x5559393bea60 2023-12-07T10:30:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.961+0000 7f0a65ffb700 1 -- 172.21.15.119:0/2103814948 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a68037040 con 0x5559393bea60 2023-12-07T10:30:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.961+0000 7f0a909a7b80 10 client.27647 fetch_fsmap learned FSMap version 59 2023-12-07T10:30:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.961+0000 7f0a909a7b80 1 -- 172.21.15.119:0/2103814948 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55593915d1f0 con 0x5559393bea60 2023-12-07T10:30:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.961+0000 7f0a65ffb700 1 -- 172.21.15.119:0/2103814948 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a6801e040 con 0x5559393bea60 2023-12-07T10:30:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.961+0000 7f0a65ffb700 1 -- 172.21.15.119:0/2103814948 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 59) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0a6803a070 con 0x5559393bea60 2023-12-07T10:30:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.962+0000 7f0a909a7b80 10 client.27647 fetch_fsmap finished waiting for FSMap version 59 2023-12-07T10:30:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.962+0000 7f0a909a7b80 10 client.27647 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:30:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.962+0000 7f0a909a7b80 20 client.27647 populate_metadata read hostname 'smithi119' 2023-12-07T10:30:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.962+0000 7f0a909a7b80 1 --2- 172.21.15.119:0/2103814948 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555939428d10 0x55593942b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:30:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.962+0000 7f0a909a7b80 4 client.27647 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:30:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.962+0000 7f0a909a7b80 1 -- 172.21.15.119:0/2103814948 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5559392d3f70 con 0x555939428d10 2023-12-07T10:30:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.962+0000 7f0a6ffff700 1 --2- 172.21.15.119:0/2103814948 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555939428d10 0x55593942b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:30:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.962+0000 7f0a6ffff700 1 --2- 172.21.15.119:0/2103814948 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555939428d10 0x55593942b130 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcc6fbe1a46d6ba server_cookie=878d30d36eceae39 in_seq=0 out_seq=0 2023-12-07T10:30:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:50.962+0000 7f0a65ffb700 10 client.27647 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:30:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.011+0000 7f0a65ffb700 1 -- 172.21.15.119:0/2103814948 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5559392d3f70 con 0x555939428d10 2023-12-07T10:30:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.011+0000 7f0a65ffb700 10 client.27647 handle_command_reply: tid=0 2023-12-07T10:30:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.011+0000 7f0a909a7b80 1 client.27647 shutdown 2023-12-07T10:30:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.011+0000 7f0a909a7b80 1 -- 172.21.15.119:0/2103814948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559393bea60 msgr2=0x5559393b8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.011+0000 7f0a909a7b80 1 --2- 172.21.15.119:0/2103814948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559393bea60 0x5559393b8e40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0a68005950 tx=0x7f0a68033f90 comp rx=0 tx=0).stop 2023-12-07T10:30:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.011+0000 7f0a909a7b80 1 -- 172.21.15.119:0/2103814948 shutdown_connections 2023-12-07T10:30:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.011+0000 7f0a909a7b80 1 --2- 172.21.15.119:0/2103814948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5559393bf380 0x5559394273a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.011+0000 7f0a909a7b80 1 --2- 172.21.15.119:0/2103814948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559393b9ad0 0x5559393b8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.011+0000 7f0a909a7b80 1 --2- 172.21.15.119:0/2103814948 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555939428d10 0x55593942b130 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.011+0000 7f0a909a7b80 1 --2- 172.21.15.119:0/2103814948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5559393bea60 0x5559393b8e40 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.011+0000 7f0a65ffb700 0 client.27647 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:30:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.011+0000 7f0a909a7b80 1 -- 172.21.15.119:0/2103814948 >> 172.21.15.119:0/2103814948 conn(0x5559392e8300 msgr2=0x5559393d9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:30:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.011+0000 7f0a909a7b80 1 -- 172.21.15.119:0/2103814948 shutdown_connections 2023-12-07T10:30:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.011+0000 7f0a909a7b80 1 -- 172.21.15.119:0/2103814948 wait complete. 2023-12-07T10:30:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.011+0000 7f0a909a7b80 20 client.27647 trim_cache size 0 max 16384 2023-12-07T10:30:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:30:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:30:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:30:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:30:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.012+0000 7f0a7cfff700 1 -- 172.21.15.119:0/2331928321 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0a580a1d00 msgr2=0x7f0a580a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.012+0000 7f0a7cfff700 1 --2- 172.21.15.119:0/2331928321 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0a580a1d00 0x7f0a580a41c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0a700cabe0 tx=0x7f0a700092e0 comp rx=0 tx=0).stop 2023-12-07T10:30:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.012+0000 7f0a7cfff700 1 -- 172.21.15.119:0/2331928321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a78109720 msgr2=0x7f0a781999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:30:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.012+0000 7f0a7cfff700 1 --2- 172.21.15.119:0/2331928321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a78109720 0x7f0a781999c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000ef30 tx=0x7f0a7000c3a0 comp rx=0 tx=0).stop 2023-12-07T10:30:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.012+0000 7f0a7cfff700 1 -- 172.21.15.119:0/2331928321 shutdown_connections 2023-12-07T10:30:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.012+0000 7f0a7cfff700 1 --2- 172.21.15.119:0/2331928321 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0a580a1d00 0x7f0a580a41c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.012+0000 7f0a7cfff700 1 --2- 172.21.15.119:0/2331928321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a78109720 0x7f0a781999c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.012+0000 7f0a7cfff700 1 --2- 172.21.15.119:0/2331928321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a78103710 0x7f0a7819f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.012+0000 7f0a7cfff700 1 --2- 172.21.15.119:0/2331928321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a781040d0 0x7f0a7819fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:30:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.012+0000 7f0a7cfff700 1 -- 172.21.15.119:0/2331928321 >> 172.21.15.119:0/2331928321 conn(0x7f0a780ff1c0 msgr2=0x7f0a780ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:30:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.013+0000 7f0a7cfff700 1 -- 172.21.15.119:0/2331928321 shutdown_connections 2023-12-07T10:30:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:30:51.013+0000 7f0a7cfff700 1 -- 172.21.15.119:0/2331928321 wait complete. 2023-12-07T10:30:51.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:51 smithi119 ceph-mon[97908]: Adjusting osd_memory_target on smithi153 to 196.2M 2023-12-07T10:30:51.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:51 smithi119 ceph-mon[97908]: Unable to set osd_memory_target on smithi153 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-12-07T10:30:51.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:51 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:51.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:51 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:51.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:51 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:51.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:51 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:51.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:51 smithi159 ceph-mon[134726]: Adjusting osd_memory_target on smithi153 to 196.2M 2023-12-07T10:30:51.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:51 smithi159 ceph-mon[134726]: Unable to set osd_memory_target on smithi153 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-12-07T10:30:51.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:51 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:51.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:51 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:51.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:51 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:51.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:51 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:51.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:51 smithi153 ceph-mon[133625]: Adjusting osd_memory_target on smithi153 to 196.2M 2023-12-07T10:30:51.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:51 smithi153 ceph-mon[133625]: Unable to set osd_memory_target on smithi153 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-12-07T10:30:51.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:51 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:51.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:51 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:51.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:51 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:51.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:51 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:51.573 INFO:tasks.workunit.client.0.smithi119.stdout: 1 755232 7.24 MB/sec execute 424 sec latency 37.943 ms 2023-12-07T10:30:52.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:52 smithi119 ceph-mon[97908]: Adjusting osd_memory_target on smithi159 to 196.2M 2023-12-07T10:30:52.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:52 smithi119 ceph-mon[97908]: Unable to set osd_memory_target on smithi159 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-12-07T10:30:52.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:52 smithi119 ceph-mon[97908]: pgmap v324: 129 pgs: 129 active+clean; 546 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 279 op/s 2023-12-07T10:30:52.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:52 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:52.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:52 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:52.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:52 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:52.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:52 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:52.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:52 smithi119 ceph-mon[97908]: Adjusting osd_memory_target on smithi119 to 221.9M 2023-12-07T10:30:52.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:52 smithi119 ceph-mon[97908]: Unable to set osd_memory_target on smithi119 to 232752128: error parsing value: Value '232752128' is below minimum 939524096 2023-12-07T10:30:52.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:52 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:30:52.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:52 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:30:52.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:52 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:30:52.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:52 smithi159 ceph-mon[134726]: Adjusting osd_memory_target on smithi159 to 196.2M 2023-12-07T10:30:52.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:52 smithi159 ceph-mon[134726]: Unable to set osd_memory_target on smithi159 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-12-07T10:30:52.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:52 smithi159 ceph-mon[134726]: pgmap v324: 129 pgs: 129 active+clean; 546 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 279 op/s 2023-12-07T10:30:52.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:52 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:52.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:52 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:52.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:52 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:52.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:52 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:52.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:52 smithi159 ceph-mon[134726]: Adjusting osd_memory_target on smithi119 to 221.9M 2023-12-07T10:30:52.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:52 smithi159 ceph-mon[134726]: Unable to set osd_memory_target on smithi119 to 232752128: error parsing value: Value '232752128' is below minimum 939524096 2023-12-07T10:30:52.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:52 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:30:52.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:52 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:30:52.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:52 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:30:52.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:52 smithi153 ceph-mon[133625]: Adjusting osd_memory_target on smithi159 to 196.2M 2023-12-07T10:30:52.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:52 smithi153 ceph-mon[133625]: Unable to set osd_memory_target on smithi159 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-12-07T10:30:52.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:52 smithi153 ceph-mon[133625]: pgmap v324: 129 pgs: 129 active+clean; 546 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 279 op/s 2023-12-07T10:30:52.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:52 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:52.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:52 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:52.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:52 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:52.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:52 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-07T10:30:52.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:52 smithi153 ceph-mon[133625]: Adjusting osd_memory_target on smithi119 to 221.9M 2023-12-07T10:30:52.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:52 smithi153 ceph-mon[133625]: Unable to set osd_memory_target on smithi119 to 232752128: error parsing value: Value '232752128' is below minimum 939524096 2023-12-07T10:30:52.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:52 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:30:52.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:52 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:30:52.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:52 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:30:52.574 INFO:tasks.workunit.client.0.smithi119.stdout: 1 756709 7.25 MB/sec execute 425 sec latency 13.527 ms 2023-12-07T10:30:53.574 INFO:tasks.workunit.client.0.smithi119.stdout: 1 757261 7.24 MB/sec execute 426 sec latency 263.070 ms 2023-12-07T10:30:54.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:54 smithi119 ceph-mon[97908]: pgmap v325: 129 pgs: 129 active+clean; 550 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 216 op/s 2023-12-07T10:30:54.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:54 smithi159 ceph-mon[134726]: pgmap v325: 129 pgs: 129 active+clean; 550 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 216 op/s 2023-12-07T10:30:54.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:54 smithi153 ceph-mon[133625]: pgmap v325: 129 pgs: 129 active+clean; 550 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 216 op/s 2023-12-07T10:30:54.574 INFO:tasks.workunit.client.0.smithi119.stdout: 1 759085 7.24 MB/sec execute 427 sec latency 18.809 ms 2023-12-07T10:30:55.574 INFO:tasks.workunit.client.0.smithi119.stdout: 1 760403 7.25 MB/sec execute 428 sec latency 13.600 ms 2023-12-07T10:30:56.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:56 smithi119 ceph-mon[97908]: pgmap v326: 129 pgs: 129 active+clean; 550 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 205 op/s 2023-12-07T10:30:56.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:56 smithi159 ceph-mon[134726]: pgmap v326: 129 pgs: 129 active+clean; 550 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 205 op/s 2023-12-07T10:30:56.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:56 smithi153 ceph-mon[133625]: pgmap v326: 129 pgs: 129 active+clean; 550 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 205 op/s 2023-12-07T10:30:56.574 INFO:tasks.workunit.client.0.smithi119.stdout: 1 761964 7.24 MB/sec execute 429 sec latency 26.148 ms 2023-12-07T10:30:57.574 INFO:tasks.workunit.client.0.smithi119.stdout: 1 763657 7.26 MB/sec execute 430 sec latency 16.480 ms 2023-12-07T10:30:58.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:30:58 smithi119 ceph-mon[97908]: pgmap v327: 129 pgs: 129 active+clean; 553 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 308 op/s 2023-12-07T10:30:58.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:30:58 smithi159 ceph-mon[134726]: pgmap v327: 129 pgs: 129 active+clean; 553 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 308 op/s 2023-12-07T10:30:58.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:30:58 smithi153 ceph-mon[133625]: pgmap v327: 129 pgs: 129 active+clean; 553 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 308 op/s 2023-12-07T10:30:58.574 INFO:tasks.workunit.client.0.smithi119.stdout: 1 764593 7.25 MB/sec execute 431 sec latency 37.411 ms 2023-12-07T10:30:59.574 INFO:tasks.workunit.client.0.smithi119.stdout: 1 766418 7.25 MB/sec execute 432 sec latency 17.995 ms 2023-12-07T10:31:00.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:00 smithi119 ceph-mon[97908]: pgmap v328: 129 pgs: 129 active+clean; 553 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 206 op/s 2023-12-07T10:31:00.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:00 smithi159 ceph-mon[134726]: pgmap v328: 129 pgs: 129 active+clean; 553 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 206 op/s 2023-12-07T10:31:00.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:00 smithi153 ceph-mon[133625]: pgmap v328: 129 pgs: 129 active+clean; 553 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 206 op/s 2023-12-07T10:31:00.574 INFO:tasks.workunit.client.0.smithi119.stdout: 1 767344 7.25 MB/sec execute 433 sec latency 324.130 ms 2023-12-07T10:31:01.574 INFO:tasks.workunit.client.0.smithi119.stdout: 1 767344 7.23 MB/sec execute 434 sec latency 1324.266 ms 2023-12-07T10:31:02.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:01 smithi119 ceph-mon[97908]: pgmap v329: 129 pgs: 129 active+clean; 559 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 278 op/s 2023-12-07T10:31:02.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:01 smithi159 ceph-mon[134726]: pgmap v329: 129 pgs: 129 active+clean; 559 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 278 op/s 2023-12-07T10:31:02.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:01 smithi153 ceph-mon[133625]: pgmap v329: 129 pgs: 129 active+clean; 559 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 278 op/s 2023-12-07T10:31:02.574 INFO:tasks.workunit.client.0.smithi119.stdout: 1 767344 7.22 MB/sec execute 435 sec latency 2324.336 ms 2023-12-07T10:31:03.574 INFO:tasks.workunit.client.0.smithi119.stdout: 1 767769 7.20 MB/sec execute 436 sec latency 2850.167 ms 2023-12-07T10:31:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.233+0000 7fb81f7ff700 1 -- 172.21.15.119:0/3940777237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8180742d0 msgr2=0x7fb818111fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.233+0000 7fb81f7ff700 1 --2- 172.21.15.119:0/3940777237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8180742d0 0x7fb818111fa0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb808009990 tx=0x7fb808009ca0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.234+0000 7fb81f7ff700 1 -- 172.21.15.119:0/3940777237 shutdown_connections 2023-12-07T10:31:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.234+0000 7fb81f7ff700 1 --2- 172.21.15.119:0/3940777237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8181124e0 0x7fb8181148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.234+0000 7fb81f7ff700 1 --2- 172.21.15.119:0/3940777237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8180739b0 0x7fb818073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.234+0000 7fb81f7ff700 1 --2- 172.21.15.119:0/3940777237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8180742d0 0x7fb818111fa0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.234+0000 7fb81f7ff700 1 -- 172.21.15.119:0/3940777237 >> 172.21.15.119:0/3940777237 conn(0x7fb8180fcfe0 msgr2=0x7fb8180ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.234+0000 7fb81f7ff700 1 -- 172.21.15.119:0/3940777237 shutdown_connections 2023-12-07T10:31:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.234+0000 7fb81f7ff700 1 -- 172.21.15.119:0/3940777237 wait complete. 2023-12-07T10:31:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.235+0000 7fb81f7ff700 1 Processor -- start 2023-12-07T10:31:04.237 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.236+0000 7fb81f7ff700 1 -- start start 2023-12-07T10:31:04.237 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.236+0000 7fb81f7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8180739b0 0x7fb81819e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:04.237 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.236+0000 7fb81f7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8180742d0 0x7fb81819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:04.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.237+0000 7fb81cd9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8180742d0 0x7fb81819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:04.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.237+0000 7fb81d59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8180739b0 0x7fb81819e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:04.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.237+0000 7fb81d59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8180739b0 0x7fb81819e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54214/0 (socket says 172.21.15.119:54214) 2023-12-07T10:31:04.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.237+0000 7fb81cd9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8180742d0 0x7fb81819e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42880/0 (socket says 172.21.15.119:42880) 2023-12-07T10:31:04.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.237+0000 7fb81f7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8181124e0 0x7fb8181a2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:04.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.237+0000 7fb81f7ff700 1 -- 172.21.15.119:0/1230509336 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8181a2f90 con 0x7fb8180739b0 2023-12-07T10:31:04.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.237+0000 7fb81d59b700 1 -- 172.21.15.119:0/1230509336 learned_addr learned my addr 172.21.15.119:0/1230509336 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:31:04.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.237+0000 7fb81f7ff700 1 -- 172.21.15.119:0/1230509336 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8181a3100 con 0x7fb8180742d0 2023-12-07T10:31:04.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.237+0000 7fb81f7ff700 1 -- 172.21.15.119:0/1230509336 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8181a32c0 con 0x7fb8181124e0 2023-12-07T10:31:04.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.238+0000 7fb81cd9a700 1 -- 172.21.15.119:0/1230509336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8181124e0 msgr2=0x7fb8181a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:04.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.238+0000 7fb81dd9c700 1 --2- 172.21.15.119:0/1230509336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8181124e0 0x7fb8181a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:04.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.238+0000 7fb81cd9a700 1 --2- 172.21.15.119:0/1230509336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8181124e0 0x7fb8181a2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.238+0000 7fb81cd9a700 1 -- 172.21.15.119:0/1230509336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8180739b0 msgr2=0x7fb81819e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:04.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.238+0000 7fb81cd9a700 1 --2- 172.21.15.119:0/1230509336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8180739b0 0x7fb81819e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.238+0000 7fb81cd9a700 1 -- 172.21.15.119:0/1230509336 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8080095f0 con 0x7fb8180742d0 2023-12-07T10:31:04.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.238+0000 7fb81d59b700 1 --2- 172.21.15.119:0/1230509336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8180739b0 0x7fb81819e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:31:04.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.239+0000 7fb81cd9a700 1 --2- 172.21.15.119:0/1230509336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8180742d0 0x7fb81819e5b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb808009990 tx=0x7fb80800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.240+0000 7fb80e7fc700 1 -- 172.21.15.119:0/1230509336 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80801d070 con 0x7fb8180742d0 2023-12-07T10:31:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.240+0000 7fb80e7fc700 1 -- 172.21.15.119:0/1230509336 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb808005050 con 0x7fb8180742d0 2023-12-07T10:31:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.240+0000 7fb80e7fc700 1 -- 172.21.15.119:0/1230509336 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb808004c70 con 0x7fb8180742d0 2023-12-07T10:31:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.240+0000 7fb81f7ff700 1 -- 172.21.15.119:0/1230509336 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8181a3540 con 0x7fb8180742d0 2023-12-07T10:31:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.241+0000 7fb81dd9c700 1 --2- 172.21.15.119:0/1230509336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8181124e0 0x7fb8181a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:31:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.241+0000 7fb81f7ff700 1 -- 172.21.15.119:0/1230509336 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8181a37c0 con 0x7fb8180742d0 2023-12-07T10:31:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.248+0000 7fb80e7fc700 1 -- 172.21.15.119:0/1230509336 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fb8080051c0 con 0x7fb8180742d0 2023-12-07T10:31:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.249+0000 7fb833091b80 1 Processor -- start 2023-12-07T10:31:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.249+0000 7fb833091b80 1 -- start start 2023-12-07T10:31:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.251+0000 7fb833091b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba869aea60 0x55ba869aee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.251+0000 7fb833091b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba869af380 0x55ba869a9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.251+0000 7fb833091b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba869a9ad0 0x55ba869a9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.251+0000 7fb833091b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8691d760 con 0x55ba869aea60 2023-12-07T10:31:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.251+0000 7fb833091b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba86904740 con 0x55ba869af380 2023-12-07T10:31:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.251+0000 7fb833091b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba86902060 con 0x55ba869a9ad0 2023-12-07T10:31:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.251+0000 7fb81cd9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba869a9ad0 0x55ba869a9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.251+0000 7fb81cd9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba869a9ad0 0x55ba869a9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:35830/0 (socket says 172.21.15.119:35830) 2023-12-07T10:31:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.251+0000 7fb81cd9a700 1 -- 172.21.15.119:0/2294935505 learned_addr learned my addr 172.21.15.119:0/2294935505 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:31:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.252+0000 7fb81dd9c700 1 --2- 172.21.15.119:0/2294935505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba869aea60 0x55ba869aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.252+0000 7fb80e7fc700 1 --2- 172.21.15.119:0/1230509336 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb8040a1e50 0x7fb8040a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.253+0000 7fb81d59b700 1 --2- 172.21.15.119:0/2294935505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba869af380 0x55ba869a9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.253+0000 7fb81d59b700 1 -- 172.21.15.119:0/2294935505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba869a9ad0 msgr2=0x55ba869a9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.253+0000 7fb81d59b700 1 --2- 172.21.15.119:0/2294935505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba869a9ad0 0x55ba869a9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.253+0000 7fb81d59b700 1 -- 172.21.15.119:0/2294935505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba869aea60 msgr2=0x55ba869aee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.253+0000 7fb81d59b700 1 --2- 172.21.15.119:0/2294935505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba869aea60 0x55ba869aee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.253+0000 7fb81d59b700 1 -- 172.21.15.119:0/2294935505 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba8677a5d0 con 0x55ba869af380 2023-12-07T10:31:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.253+0000 7fb80e7fc700 1 -- 172.21.15.119:0/1230509336 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9549+0+0 (secure 0 0 0) 0x7fb8080ce580 con 0x7fb8180742d0 2023-12-07T10:31:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.254+0000 7fb81dd9c700 1 --2- 172.21.15.119:0/1230509336 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb8040a1e50 0x7fb8040a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.254+0000 7fb81d59b700 1 --2- 172.21.15.119:0/2294935505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba869af380 0x55ba869a9540 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb81400c7e0 tx=0x7fb814007aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf47339859ff58b0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.254+0000 7fb8037fe700 1 -- 172.21.15.119:0/2294935505 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb814004bb0 con 0x55ba869af380 2023-12-07T10:31:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.254+0000 7fb8037fe700 1 -- 172.21.15.119:0/2294935505 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb814004d10 con 0x55ba869af380 2023-12-07T10:31:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.254+0000 7fb8037fe700 1 -- 172.21.15.119:0/2294935505 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8140052f0 con 0x55ba869af380 2023-12-07T10:31:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.256+0000 7fb833091b80 1 -- 172.21.15.119:0/2294935505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba869af380 msgr2=0x55ba869a9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.256+0000 7fb833091b80 1 --2- 172.21.15.119:0/2294935505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba869af380 0x55ba869a9540 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb81400c7e0 tx=0x7fb814007aa0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.256+0000 7fb81dd9c700 1 --2- 172.21.15.119:0/1230509336 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb8040a1e50 0x7fb8040a4310 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb81806ad50 tx=0x7fb810004750 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.256+0000 7fb833091b80 1 -- 172.21.15.119:0/2294935505 shutdown_connections 2023-12-07T10:31:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.256+0000 7fb833091b80 1 --2- 172.21.15.119:0/2294935505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba869a9ad0 0x55ba869a9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.256+0000 7fb833091b80 1 --2- 172.21.15.119:0/2294935505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba869aea60 0x55ba869aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.256+0000 7fb833091b80 1 --2- 172.21.15.119:0/2294935505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba869af380 0x55ba869a9540 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.256+0000 7fb833091b80 1 -- 172.21.15.119:0/2294935505 >> 172.21.15.119:0/2294935505 conn(0x55ba868d8300 msgr2=0x55ba869a3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.256+0000 7fb833091b80 1 -- 172.21.15.119:0/2294935505 shutdown_connections 2023-12-07T10:31:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.257+0000 7fb833091b80 1 -- 172.21.15.119:0/2294935505 wait complete. 2023-12-07T10:31:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.257+0000 7fb833091b80 1 -- start start 2023-12-07T10:31:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.258+0000 7fb833091b80 1 Processor -- start 2023-12-07T10:31:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.259+0000 7fb833091b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba869a9ad0 0x55ba869b5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.259+0000 7fb833091b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba869aea60 0x55ba869b2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.259+0000 7fb833091b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba869af380 0x55ba869b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.259+0000 7fb81d59b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba869a9ad0 0x55ba869b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.259+0000 7fb81cd9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba869aea60 0x55ba869b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.259+0000 7fb81cd9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba869aea60 0x55ba869b2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42892/0 (socket says 172.21.15.119:42892) 2023-12-07T10:31:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.259+0000 7fb833091b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba868fd950 con 0x55ba869af380 2023-12-07T10:31:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.259+0000 7fb833091b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba868b7500 con 0x55ba869aea60 2023-12-07T10:31:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.259+0000 7fb833091b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba868b71a0 con 0x55ba869a9ad0 2023-12-07T10:31:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.259+0000 7fb833091b80 10 client.0 _ll_register_callbacks cb 0x55ba8689e8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:31:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.260+0000 7fb81d59b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba869a9ad0 0x55ba869b5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:35838/0 (socket says 172.21.15.119:35838) 2023-12-07T10:31:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.260+0000 7fb81d59b700 1 -- 172.21.15.119:0/2349071759 learned_addr learned my addr 172.21.15.119:0/2349071759 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:31:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.260+0000 7fb81dd9c700 1 --2- 172.21.15.119:0/2349071759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba869af380 0x55ba869b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.260+0000 7fb81d59b700 1 -- 172.21.15.119:0/2349071759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba869aea60 msgr2=0x55ba869b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.261+0000 7fb81d59b700 1 --2- 172.21.15.119:0/2349071759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba869aea60 0x55ba869b2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.261+0000 7fb81d59b700 1 -- 172.21.15.119:0/2349071759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba869af380 msgr2=0x55ba869b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.261+0000 7fb81d59b700 1 --2- 172.21.15.119:0/2349071759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba869af380 0x55ba869b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.261+0000 7fb81d59b700 1 -- 172.21.15.119:0/2349071759 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81400c490 con 0x55ba869a9ad0 2023-12-07T10:31:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.261+0000 7fb81d59b700 1 --2- 172.21.15.119:0/2349071759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba869a9ad0 0x55ba869b5320 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb81400cdd0 tx=0x7fb814014e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ea81ef3047104fe server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.261+0000 7fb81cd9a700 1 --2- 172.21.15.119:0/2349071759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba869aea60 0x55ba869b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:31:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.261+0000 7fb800ff9700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:31:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.262+0000 7fb800ff9700 1 -- 172.21.15.119:0/2349071759 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb814005130 con 0x55ba869a9ad0 2023-12-07T10:31:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.262+0000 7fb800ff9700 1 -- 172.21.15.119:0/2349071759 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9549+0+0 (secure 0 0 0) 0x7fb814024070 con 0x55ba869a9ad0 2023-12-07T10:31:04.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.262+0000 7fb833091b80 1 -- 172.21.15.119:0/2349071759 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba8699c6f0 con 0x55ba869a9ad0 2023-12-07T10:31:04.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.262+0000 7fb800ff9700 1 -- 172.21.15.119:0/2349071759 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb814005290 con 0x55ba869a9ad0 2023-12-07T10:31:04.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.262+0000 7fb833091b80 10 client.27634 fetch_fsmap learned FSMap version 59 2023-12-07T10:31:04.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.262+0000 7fb833091b80 1 -- 172.21.15.119:0/2349071759 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55ba86726d80 con 0x55ba869a9ad0 2023-12-07T10:31:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.265+0000 7fb800ff9700 1 -- 172.21.15.119:0/2349071759 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ba8699c6f0 con 0x55ba869a9ad0 2023-12-07T10:31:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.265+0000 7fb800ff9700 1 -- 172.21.15.119:0/2349071759 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 59) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb814012070 con 0x55ba869a9ad0 2023-12-07T10:31:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.265+0000 7fb833091b80 10 client.27634 fetch_fsmap finished waiting for FSMap version 59 2023-12-07T10:31:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.265+0000 7fb833091b80 10 client.27634 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:31:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.265+0000 7fb833091b80 20 client.27634 populate_metadata read hostname 'smithi119' 2023-12-07T10:31:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.265+0000 7fb833091b80 1 --2- 172.21.15.119:0/2349071759 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55ba869c2d30 0x55ba869c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.265+0000 7fb833091b80 4 client.27634 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:31:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.265+0000 7fb833091b80 1 -- 172.21.15.119:0/2349071759 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba86978740 con 0x55ba869c2d30 2023-12-07T10:31:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.265+0000 7fb81cd9a700 1 --2- 172.21.15.119:0/2349071759 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55ba869c2d30 0x55ba869c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.266+0000 7fb800ff9700 10 client.27634 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:31:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.266+0000 7fb81cd9a700 1 --2- 172.21.15.119:0/2349071759 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55ba869c2d30 0x55ba869c5120 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f71a057ebe596350 server_cookie=f39c41c0e0d490cf in_seq=0 out_seq=0 2023-12-07T10:31:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.267+0000 7fb800ff9700 1 -- 172.21.15.119:0/2349071759 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ba86978740 con 0x55ba869c2d30 2023-12-07T10:31:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.267+0000 7fb800ff9700 10 client.27634 handle_command_reply: tid=0 2023-12-07T10:31:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.267+0000 7fb833091b80 1 client.27634 shutdown 2023-12-07T10:31:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.268+0000 7fb833091b80 1 -- 172.21.15.119:0/2349071759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba869a9ad0 msgr2=0x55ba869b5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.268+0000 7fb833091b80 1 --2- 172.21.15.119:0/2349071759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba869a9ad0 0x55ba869b5320 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb81400cdd0 tx=0x7fb814014e40 comp rx=0 tx=0).stop 2023-12-07T10:31:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.268+0000 7fb833091b80 1 -- 172.21.15.119:0/2349071759 shutdown_connections 2023-12-07T10:31:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.268+0000 7fb833091b80 1 --2- 172.21.15.119:0/2349071759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba869af380 0x55ba869b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.268+0000 7fb833091b80 1 --2- 172.21.15.119:0/2349071759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba869a9ad0 0x55ba869b5320 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.268+0000 7fb833091b80 1 --2- 172.21.15.119:0/2349071759 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55ba869c2d30 0x55ba869c5120 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.268+0000 7fb833091b80 1 --2- 172.21.15.119:0/2349071759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba869aea60 0x55ba869b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.268+0000 7fb833091b80 1 -- 172.21.15.119:0/2349071759 >> 172.21.15.119:0/2349071759 conn(0x55ba868d8300 msgr2=0x55ba8699e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.269+0000 7fb800ff9700 0 client.27634 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:31:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.269+0000 7fb833091b80 1 -- 172.21.15.119:0/2349071759 shutdown_connections 2023-12-07T10:31:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.269+0000 7fb833091b80 1 -- 172.21.15.119:0/2349071759 wait complete. 2023-12-07T10:31:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.270+0000 7fb833091b80 20 client.27634 trim_cache size 0 max 16384 2023-12-07T10:31:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.305+0000 7fb833091b80 1 Processor -- start 2023-12-07T10:31:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.306+0000 7fb833091b80 1 -- start start 2023-12-07T10:31:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.306+0000 7fb833091b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba869a9ad0 0x55ba869a9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.306+0000 7fb833091b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba869aea60 0x55ba869c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.306+0000 7fb833091b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba869af380 0x55ba869b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.306+0000 7fb833091b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8677c8c0 con 0x55ba869a9ad0 2023-12-07T10:31:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.306+0000 7fb833091b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba869b5bb0 con 0x55ba869af380 2023-12-07T10:31:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.306+0000 7fb833091b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba869a4240 con 0x55ba869aea60 2023-12-07T10:31:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.307+0000 7fb81cd9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba869aea60 0x55ba869c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.307+0000 7fb81dd9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba869af380 0x55ba869b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.307+0000 7fb81cd9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba869aea60 0x55ba869c9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:35850/0 (socket says 172.21.15.119:35850) 2023-12-07T10:31:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.307+0000 7fb81cd9a700 1 -- 172.21.15.119:0/1685380626 learned_addr learned my addr 172.21.15.119:0/1685380626 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:31:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.307+0000 7fb81d59b700 1 --2- 172.21.15.119:0/1685380626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba869a9ad0 0x55ba869a9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.307+0000 7fb81dd9c700 1 -- 172.21.15.119:0/1685380626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba869aea60 msgr2=0x55ba869c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.307+0000 7fb81dd9c700 1 --2- 172.21.15.119:0/1685380626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba869aea60 0x55ba869c9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.307+0000 7fb81dd9c700 1 -- 172.21.15.119:0/1685380626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba869a9ad0 msgr2=0x55ba869a9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.307+0000 7fb81dd9c700 1 --2- 172.21.15.119:0/1685380626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba869a9ad0 0x55ba869a9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.307+0000 7fb81dd9c700 1 -- 172.21.15.119:0/1685380626 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba8670fc00 con 0x55ba869af380 2023-12-07T10:31:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.307+0000 7fb81d59b700 1 --2- 172.21.15.119:0/1685380626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba869a9ad0 0x55ba869a9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:31:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.307+0000 7fb81cd9a700 1 --2- 172.21.15.119:0/1685380626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba869aea60 0x55ba869c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:31:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.307+0000 7fb81dd9c700 1 --2- 172.21.15.119:0/1685380626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba869af380 0x55ba869b9e30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb81000ab20 tx=0x7fb810014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a632c0288302fce0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.308+0000 7fb803fff700 1 -- 172.21.15.119:0/1685380626 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8100156b0 con 0x55ba869af380 2023-12-07T10:31:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.308+0000 7fb803fff700 1 -- 172.21.15.119:0/1685380626 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb81000c030 con 0x55ba869af380 2023-12-07T10:31:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.308+0000 7fb803fff700 1 -- 172.21.15.119:0/1685380626 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810021ce0 con 0x55ba869af380 2023-12-07T10:31:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.309+0000 7fb833091b80 1 -- 172.21.15.119:0/1685380626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba869af380 msgr2=0x55ba869b9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.309+0000 7fb833091b80 1 --2- 172.21.15.119:0/1685380626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba869af380 0x55ba869b9e30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb81000ab20 tx=0x7fb810014000 comp rx=0 tx=0).stop 2023-12-07T10:31:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.309+0000 7fb833091b80 1 -- 172.21.15.119:0/1685380626 shutdown_connections 2023-12-07T10:31:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.309+0000 7fb833091b80 1 --2- 172.21.15.119:0/1685380626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba869af380 0x55ba869b9e30 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.309+0000 7fb833091b80 1 --2- 172.21.15.119:0/1685380626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba869a9ad0 0x55ba869a9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.309+0000 7fb833091b80 1 --2- 172.21.15.119:0/1685380626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba869aea60 0x55ba869c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.310+0000 7fb833091b80 1 -- 172.21.15.119:0/1685380626 >> 172.21.15.119:0/1685380626 conn(0x55ba868d8300 msgr2=0x55ba869b5050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.310+0000 7fb833091b80 1 -- 172.21.15.119:0/1685380626 shutdown_connections 2023-12-07T10:31:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.310+0000 7fb833091b80 1 -- 172.21.15.119:0/1685380626 wait complete. 2023-12-07T10:31:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.311+0000 7fb833091b80 1 -- start start 2023-12-07T10:31:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.311+0000 7fb833091b80 1 Processor -- start 2023-12-07T10:31:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.312+0000 7fb833091b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba869a9ad0 0x55ba869ba020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.313+0000 7fb833091b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba869aea60 0x55ba869b7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.313+0000 7fb833091b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba869af380 0x55ba869b8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.313+0000 7fb833091b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba869ba770 con 0x55ba869aea60 2023-12-07T10:31:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.313+0000 7fb833091b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba869b88c0 con 0x55ba869a9ad0 2023-12-07T10:31:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.313+0000 7fb833091b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba869b8a80 con 0x55ba869af380 2023-12-07T10:31:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.313+0000 7fb833091b80 10 client.0 _ll_register_callbacks cb 0x55ba8689e8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:31:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.313+0000 7fb81d59b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba869a9ad0 0x55ba869ba020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.314+0000 7fb81d59b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba869a9ad0 0x55ba869ba020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42910/0 (socket says 172.21.15.119:42910) 2023-12-07T10:31:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.314+0000 7fb81d59b700 1 -- 172.21.15.119:0/1424169985 learned_addr learned my addr 172.21.15.119:0/1424169985 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:31:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.314+0000 7fb81dd9c700 1 --2- 172.21.15.119:0/1424169985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba869af380 0x55ba869b8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.314+0000 7fb81cd9a700 1 --2- 172.21.15.119:0/1424169985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba869aea60 0x55ba869b7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.316+0000 7fb81d59b700 1 -- 172.21.15.119:0/1424169985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba869af380 msgr2=0x55ba869b8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.316+0000 7fb81d59b700 1 --2- 172.21.15.119:0/1424169985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba869af380 0x55ba869b8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.316+0000 7fb81d59b700 1 -- 172.21.15.119:0/1424169985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba869aea60 msgr2=0x55ba869b7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.316+0000 7fb81d59b700 1 --2- 172.21.15.119:0/1424169985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba869aea60 0x55ba869b7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.316+0000 7fb81d59b700 1 -- 172.21.15.119:0/1424169985 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8181a3540 con 0x55ba869a9ad0 2023-12-07T10:31:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.316+0000 7fb81d59b700 1 --2- 172.21.15.119:0/1424169985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba869a9ad0 0x55ba869ba020 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb814014b30 tx=0x7fb814027830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=299fcf4c4f467df6 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.317+0000 7fb800ff9700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:31:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.317+0000 7fb800ff9700 1 -- 172.21.15.119:0/1424169985 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8140298d0 con 0x55ba869a9ad0 2023-12-07T10:31:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.317+0000 7fb833091b80 1 -- 172.21.15.119:0/1424169985 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb814004d10 con 0x55ba869a9ad0 2023-12-07T10:31:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.318+0000 7fb800ff9700 1 -- 172.21.15.119:0/1424169985 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9549+0+0 (secure 0 0 0) 0x7fb814024040 con 0x55ba869a9ad0 2023-12-07T10:31:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.318+0000 7fb800ff9700 1 -- 172.21.15.119:0/1424169985 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb814029de0 con 0x55ba869a9ad0 2023-12-07T10:31:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.318+0000 7fb833091b80 10 client.27665 fetch_fsmap learned FSMap version 59 2023-12-07T10:31:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.318+0000 7fb833091b80 1 -- 172.21.15.119:0/1424169985 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x7fb81001d040 con 0x55ba869a9ad0 2023-12-07T10:31:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.318+0000 7fb800ff9700 1 -- 172.21.15.119:0/1424169985 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb814004d10 con 0x55ba869a9ad0 2023-12-07T10:31:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.318+0000 7fb800ff9700 1 -- 172.21.15.119:0/1424169985 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 59) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb8140055c0 con 0x55ba869a9ad0 2023-12-07T10:31:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.319+0000 7fb833091b80 10 client.27665 fetch_fsmap finished waiting for FSMap version 59 2023-12-07T10:31:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.319+0000 7fb833091b80 10 client.27665 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:31:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.319+0000 7fb833091b80 20 client.27665 populate_metadata read hostname 'smithi119' 2023-12-07T10:31:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.319+0000 7fb833091b80 1 --2- 172.21.15.119:0/1424169985 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55ba86a19350 0x55ba86a1b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.319+0000 7fb833091b80 4 client.27665 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:31:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.319+0000 7fb833091b80 1 -- 172.21.15.119:0/1424169985 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ba868c3f70 con 0x55ba86a19350 2023-12-07T10:31:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.320+0000 7fb81cd9a700 1 --2- 172.21.15.119:0/1424169985 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55ba86a19350 0x55ba86a1b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.321+0000 7fb81cd9a700 1 --2- 172.21.15.119:0/1424169985 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55ba86a19350 0x55ba86a1b770 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a3d1149b580ee7e server_cookie=289d62f4c7226522 in_seq=0 out_seq=0 2023-12-07T10:31:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.321+0000 7fb800ff9700 10 client.27665 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:31:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.380+0000 7fb800ff9700 1 -- 172.21.15.119:0/1424169985 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ba868c3f70 con 0x55ba86a19350 2023-12-07T10:31:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.381+0000 7fb800ff9700 10 client.27665 handle_command_reply: tid=0 2023-12-07T10:31:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.381+0000 7fb833091b80 1 client.27665 shutdown 2023-12-07T10:31:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.381+0000 7fb833091b80 1 -- 172.21.15.119:0/1424169985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba869a9ad0 msgr2=0x55ba869ba020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.381+0000 7fb833091b80 1 --2- 172.21.15.119:0/1424169985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba869a9ad0 0x55ba869ba020 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb814014b30 tx=0x7fb814027830 comp rx=0 tx=0).stop 2023-12-07T10:31:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.381+0000 7fb833091b80 1 -- 172.21.15.119:0/1424169985 shutdown_connections 2023-12-07T10:31:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.381+0000 7fb833091b80 1 --2- 172.21.15.119:0/1424169985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba869af380 0x55ba869b8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.381+0000 7fb833091b80 1 --2- 172.21.15.119:0/1424169985 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55ba86a19350 0x55ba86a1b770 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.381+0000 7fb833091b80 1 --2- 172.21.15.119:0/1424169985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ba869a9ad0 0x55ba869ba020 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.381+0000 7fb833091b80 1 --2- 172.21.15.119:0/1424169985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ba869aea60 0x55ba869b7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.381+0000 7fb833091b80 1 -- 172.21.15.119:0/1424169985 >> 172.21.15.119:0/1424169985 conn(0x55ba868d8300 msgr2=0x55ba869ac8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.381+0000 7fb800ff9700 0 client.27665 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:31:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.381+0000 7fb833091b80 1 -- 172.21.15.119:0/1424169985 shutdown_connections 2023-12-07T10:31:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.382+0000 7fb833091b80 1 -- 172.21.15.119:0/1424169985 wait complete. 2023-12-07T10:31:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.382+0000 7fb833091b80 20 client.27665 trim_cache size 0 max 16384 2023-12-07T10:31:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:31:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:31:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:31:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:31:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.383+0000 7fb81f7ff700 1 -- 172.21.15.119:0/1230509336 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb8040a1e50 msgr2=0x7fb8040a4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.383+0000 7fb81f7ff700 1 --2- 172.21.15.119:0/1230509336 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb8040a1e50 0x7fb8040a4310 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb81806ad50 tx=0x7fb810004750 comp rx=0 tx=0).stop 2023-12-07T10:31:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.383+0000 7fb81f7ff700 1 -- 172.21.15.119:0/1230509336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8180742d0 msgr2=0x7fb81819e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.383+0000 7fb81f7ff700 1 --2- 172.21.15.119:0/1230509336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8180742d0 0x7fb81819e5b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb808009990 tx=0x7fb80800f690 comp rx=0 tx=0).stop 2023-12-07T10:31:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.384+0000 7fb81f7ff700 1 -- 172.21.15.119:0/1230509336 shutdown_connections 2023-12-07T10:31:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.384+0000 7fb81f7ff700 1 --2- 172.21.15.119:0/1230509336 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb8040a1e50 0x7fb8040a4310 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.384+0000 7fb81f7ff700 1 --2- 172.21.15.119:0/1230509336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8181124e0 0x7fb8181a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.384+0000 7fb81f7ff700 1 --2- 172.21.15.119:0/1230509336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8180739b0 0x7fb81819e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.384+0000 7fb81f7ff700 1 --2- 172.21.15.119:0/1230509336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8180742d0 0x7fb81819e5b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.384+0000 7fb81f7ff700 1 -- 172.21.15.119:0/1230509336 >> 172.21.15.119:0/1230509336 conn(0x7fb8180fcfe0 msgr2=0x7fb8180fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.384+0000 7fb81f7ff700 1 -- 172.21.15.119:0/1230509336 shutdown_connections 2023-12-07T10:31:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:04.384+0000 7fb81f7ff700 1 -- 172.21.15.119:0/1230509336 wait complete. 2023-12-07T10:31:04.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:04 smithi119 ceph-mon[97908]: pgmap v330: 129 pgs: 129 active+clean; 559 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 234 op/s 2023-12-07T10:31:04.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:04 smithi159 ceph-mon[134726]: pgmap v330: 129 pgs: 129 active+clean; 559 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 234 op/s 2023-12-07T10:31:04.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:04 smithi153 ceph-mon[133625]: pgmap v330: 129 pgs: 129 active+clean; 559 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 234 op/s 2023-12-07T10:31:04.574 INFO:tasks.workunit.client.0.smithi119.stdout: 1 767987 7.19 MB/sec execute 437 sec latency 224.295 ms 2023-12-07T10:31:05.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:05 smithi119 ceph-mon[97908]: osdmap e103: 12 total, 12 up, 12 in 2023-12-07T10:31:05.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:05 smithi159 ceph-mon[134726]: osdmap e103: 12 total, 12 up, 12 in 2023-12-07T10:31:05.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:05 smithi153 ceph-mon[133625]: osdmap e103: 12 total, 12 up, 12 in 2023-12-07T10:31:05.574 INFO:tasks.workunit.client.0.smithi119.stdout: 1 769304 7.18 MB/sec execute 438 sec latency 71.542 ms 2023-12-07T10:31:06.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:06 smithi119 ceph-mon[97908]: pgmap v332: 129 pgs: 129 active+clean; 559 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 241 op/s 2023-12-07T10:31:06.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:06 smithi159 ceph-mon[134726]: pgmap v332: 129 pgs: 129 active+clean; 559 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 241 op/s 2023-12-07T10:31:06.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:06 smithi153 ceph-mon[133625]: pgmap v332: 129 pgs: 129 active+clean; 559 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 241 op/s 2023-12-07T10:31:06.574 INFO:tasks.workunit.client.0.smithi119.stdout: 1 770771 7.19 MB/sec execute 439 sec latency 14.788 ms 2023-12-07T10:31:07.574 INFO:tasks.workunit.client.0.smithi119.stdout: 1 771783 7.18 MB/sec execute 440 sec latency 16.347 ms 2023-12-07T10:31:08.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:08 smithi119 ceph-mon[97908]: pgmap v333: 129 pgs: 129 active+clean; 562 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 226 op/s 2023-12-07T10:31:08.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:08 smithi159 ceph-mon[134726]: pgmap v333: 129 pgs: 129 active+clean; 562 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 226 op/s 2023-12-07T10:31:08.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:08 smithi153 ceph-mon[133625]: pgmap v333: 129 pgs: 129 active+clean; 562 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 226 op/s 2023-12-07T10:31:08.574 INFO:tasks.workunit.client.0.smithi119.stdout: 1 773431 7.19 MB/sec execute 441 sec latency 45.505 ms 2023-12-07T10:31:09.574 INFO:tasks.workunit.client.0.smithi119.stdout: 1 774581 7.19 MB/sec execute 442 sec latency 23.544 ms 2023-12-07T10:31:10.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:10 smithi119 ceph-mon[97908]: pgmap v334: 129 pgs: 129 active+clean; 562 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 226 op/s 2023-12-07T10:31:10.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:10 smithi159 ceph-mon[134726]: pgmap v334: 129 pgs: 129 active+clean; 562 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 226 op/s 2023-12-07T10:31:10.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:10 smithi153 ceph-mon[133625]: pgmap v334: 129 pgs: 129 active+clean; 562 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 226 op/s 2023-12-07T10:31:10.574 INFO:tasks.workunit.client.0.smithi119.stdout: 1 776383 7.19 MB/sec execute 443 sec latency 17.582 ms 2023-12-07T10:31:11.575 INFO:tasks.workunit.client.0.smithi119.stdout: 1 777889 7.20 MB/sec execute 444 sec latency 18.265 ms 2023-12-07T10:31:12.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:11 smithi119 ceph-mon[97908]: osdmap e104: 12 total, 12 up, 12 in 2023-12-07T10:31:12.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:11 smithi119 ceph-mon[97908]: pgmap v336: 129 pgs: 129 active+clean; 565 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 2.7 MiB/s wr, 221 op/s 2023-12-07T10:31:12.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:11 smithi159 ceph-mon[134726]: osdmap e104: 12 total, 12 up, 12 in 2023-12-07T10:31:12.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:11 smithi159 ceph-mon[134726]: pgmap v336: 129 pgs: 129 active+clean; 565 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 2.7 MiB/s wr, 221 op/s 2023-12-07T10:31:12.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:11 smithi153 ceph-mon[133625]: osdmap e104: 12 total, 12 up, 12 in 2023-12-07T10:31:12.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:11 smithi153 ceph-mon[133625]: pgmap v336: 129 pgs: 129 active+clean; 565 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 2.7 MiB/s wr, 221 op/s 2023-12-07T10:31:12.574 INFO:tasks.workunit.client.0.smithi119.stdout: 1 779096 7.20 MB/sec execute 445 sec latency 17.089 ms 2023-12-07T10:31:13.575 INFO:tasks.workunit.client.0.smithi119.stdout: 1 780764 7.20 MB/sec execute 446 sec latency 61.979 ms 2023-12-07T10:31:14.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:14 smithi119 ceph-mon[97908]: pgmap v337: 129 pgs: 129 active+clean; 570 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 250 op/s 2023-12-07T10:31:14.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:14 smithi159 ceph-mon[134726]: pgmap v337: 129 pgs: 129 active+clean; 570 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 250 op/s 2023-12-07T10:31:14.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:14 smithi153 ceph-mon[133625]: pgmap v337: 129 pgs: 129 active+clean; 570 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 250 op/s 2023-12-07T10:31:14.575 INFO:tasks.workunit.client.0.smithi119.stdout: 1 781861 7.20 MB/sec execute 447 sec latency 15.735 ms 2023-12-07T10:31:15.575 INFO:tasks.workunit.client.0.smithi119.stdout: 1 783617 7.20 MB/sec execute 448 sec latency 17.826 ms 2023-12-07T10:31:16.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:16 smithi119 ceph-mon[97908]: pgmap v338: 129 pgs: 129 active+clean; 570 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 223 op/s 2023-12-07T10:31:16.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:16 smithi159 ceph-mon[134726]: pgmap v338: 129 pgs: 129 active+clean; 570 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 223 op/s 2023-12-07T10:31:16.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:16 smithi153 ceph-mon[133625]: pgmap v338: 129 pgs: 129 active+clean; 570 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 223 op/s 2023-12-07T10:31:16.575 INFO:tasks.workunit.client.0.smithi119.stdout: 1 785082 7.21 MB/sec execute 449 sec latency 17.351 ms 2023-12-07T10:31:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.562+0000 7fb32099d700 1 -- 172.21.15.119:0/3160029238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb31c101f60 msgr2=0x7fb31c105fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.562+0000 7fb32099d700 1 --2- 172.21.15.119:0/3160029238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb31c101f60 0x7fb31c105fa0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb310009990 tx=0x7fb310009ca0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.563+0000 7fb32099d700 1 -- 172.21.15.119:0/3160029238 shutdown_connections 2023-12-07T10:31:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.563+0000 7fb32099d700 1 --2- 172.21.15.119:0/3160029238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb31c1066d0 0x7fb31c10e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.563+0000 7fb32099d700 1 --2- 172.21.15.119:0/3160029238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb31c1015b0 0x7fb31c101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.563+0000 7fb32099d700 1 --2- 172.21.15.119:0/3160029238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb31c101f60 0x7fb31c105fa0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.563+0000 7fb32099d700 1 -- 172.21.15.119:0/3160029238 >> 172.21.15.119:0/3160029238 conn(0x7fb31c0fce70 msgr2=0x7fb31c0ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.563+0000 7fb32099d700 1 -- 172.21.15.119:0/3160029238 shutdown_connections 2023-12-07T10:31:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.563+0000 7fb32099d700 1 -- 172.21.15.119:0/3160029238 wait complete. 2023-12-07T10:31:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.564+0000 7fb32099d700 1 Processor -- start 2023-12-07T10:31:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.564+0000 7fb32099d700 1 -- start start 2023-12-07T10:31:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.565+0000 7fb32099d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb31c1015b0 0x7fb31c19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.565+0000 7fb32099d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb31c1066d0 0x7fb31c19dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.565+0000 7fb32099d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb31c197810 0x7fb31c197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.565+0000 7fb32099d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c198290 con 0x7fb31c1066d0 2023-12-07T10:31:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.565+0000 7fb32099d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c1983d0 con 0x7fb31c1015b0 2023-12-07T10:31:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.565+0000 7fb32099d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c198590 con 0x7fb31c197810 2023-12-07T10:31:17.566 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.565+0000 7fb31bfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb31c197810 0x7fb31c197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.565+0000 7fb31affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb31c1066d0 0x7fb31c19dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.565+0000 7fb31bfff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb31c197810 0x7fb31c197cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:32932/0 (socket says 172.21.15.119:32932) 2023-12-07T10:31:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.565+0000 7fb31bfff700 1 -- 172.21.15.119:0/3129853205 learned_addr learned my addr 172.21.15.119:0/3129853205 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:31:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.565+0000 7fb31b7fe700 1 --2- 172.21.15.119:0/3129853205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb31c1015b0 0x7fb31c19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.565+0000 7fb31bfff700 1 -- 172.21.15.119:0/3129853205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb31c1015b0 msgr2=0x7fb31c19d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.565+0000 7fb31bfff700 1 --2- 172.21.15.119:0/3129853205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb31c1015b0 0x7fb31c19d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.565+0000 7fb31bfff700 1 -- 172.21.15.119:0/3129853205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb31c1066d0 msgr2=0x7fb31c19dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.565+0000 7fb31bfff700 1 --2- 172.21.15.119:0/3129853205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb31c1066d0 0x7fb31c19dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.565+0000 7fb31bfff700 1 -- 172.21.15.119:0/3129853205 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb314009c40 con 0x7fb31c197810 2023-12-07T10:31:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.565+0000 7fb31affd700 1 --2- 172.21.15.119:0/3129853205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb31c1066d0 0x7fb31c19dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:31:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.566+0000 7fb31bfff700 1 --2- 172.21.15.119:0/3129853205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb31c197810 0x7fb31c197cc0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb31400c370 tx=0x7fb31400c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.566+0000 7fb318ff9700 1 -- 172.21.15.119:0/3129853205 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31400e050 con 0x7fb31c197810 2023-12-07T10:31:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.566+0000 7fb32099d700 1 -- 172.21.15.119:0/3129853205 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3100095f0 con 0x7fb31c197810 2023-12-07T10:31:17.568 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.566+0000 7fb318ff9700 1 -- 172.21.15.119:0/3129853205 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb31400f040 con 0x7fb31c197810 2023-12-07T10:31:17.568 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.566+0000 7fb318ff9700 1 -- 172.21.15.119:0/3129853205 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb314013400 con 0x7fb31c197810 2023-12-07T10:31:17.568 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.566+0000 7fb32099d700 1 -- 172.21.15.119:0/3129853205 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31c1a4830 con 0x7fb31c197810 2023-12-07T10:31:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.569+0000 7fb318ff9700 1 -- 172.21.15.119:0/3129853205 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fb31400cb60 con 0x7fb31c197810 2023-12-07T10:31:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.571+0000 7fb318ff9700 1 --2- 172.21.15.119:0/3129853205 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb3040a1c80 0x7fb3040a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:17.571 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.571+0000 7fb318ff9700 1 -- 172.21.15.119:0/3129853205 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fb3140cdcc0 con 0x7fb31c197810 2023-12-07T10:31:17.571 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.571+0000 7fb31b7fe700 1 --2- 172.21.15.119:0/3129853205 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb3040a1c80 0x7fb3040a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:17.571 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.571+0000 7fb335444b80 1 Processor -- start 2023-12-07T10:31:17.571 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.571+0000 7fb335444b80 1 -- start start 2023-12-07T10:31:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.572+0000 7fb335444b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f74fe98ca0 0x55f74fe99080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.572+0000 7fb335444b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f74fe995c0 0x55f74fe93540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.572+0000 7fb335444b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74fe93a80 0x55f74fe93f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.572+0000 7fb335444b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74fe07760 con 0x55f74fe93a80 2023-12-07T10:31:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.572+0000 7fb335444b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74fdee740 con 0x55f74fe995c0 2023-12-07T10:31:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.572+0000 7fb335444b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74fdec060 con 0x55f74fe98ca0 2023-12-07T10:31:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.572+0000 7fb31b7fe700 1 --2- 172.21.15.119:0/3129853205 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb3040a1c80 0x7fb3040a4140 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb30c009c60 tx=0x7fb30c009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.572+0000 7fb31b7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f74fe98ca0 0x55f74fe99080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.572+0000 7fb31b7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f74fe98ca0 0x55f74fe99080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:32936/0 (socket says 172.21.15.119:32936) 2023-12-07T10:31:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.572+0000 7fb31b7fe700 1 -- 172.21.15.119:0/778031655 learned_addr learned my addr 172.21.15.119:0/778031655 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:31:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.572+0000 7fb31affd700 1 --2- 172.21.15.119:0/778031655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f74fe995c0 0x55f74fe93540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.572+0000 7fb31bfff700 1 --2- 172.21.15.119:0/778031655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74fe93a80 0x55f74fe93f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.572+0000 7fb31b7fe700 1 -- 172.21.15.119:0/778031655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f74fe995c0 msgr2=0x55f74fe93540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.572+0000 7fb31b7fe700 1 --2- 172.21.15.119:0/778031655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f74fe995c0 0x55f74fe93540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.572+0000 7fb31b7fe700 1 -- 172.21.15.119:0/778031655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74fe93a80 msgr2=0x55f74fe93f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.572+0000 7fb31b7fe700 1 --2- 172.21.15.119:0/778031655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74fe93a80 0x55f74fe93f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.572+0000 7fb31b7fe700 1 -- 172.21.15.119:0/778031655 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f74fc645d0 con 0x55f74fe98ca0 2023-12-07T10:31:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.572+0000 7fb31bfff700 1 --2- 172.21.15.119:0/778031655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74fe93a80 0x55f74fe93f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:31:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.573+0000 7fb31b7fe700 1 --2- 172.21.15.119:0/778031655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f74fe98ca0 0x55f74fe99080 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00e430 tx=0x7fb30c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91e1635d8101f2b5 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.573+0000 7fb301ffb700 1 -- 172.21.15.119:0/778031655 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c00edd0 con 0x55f74fe98ca0 2023-12-07T10:31:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.573+0000 7fb301ffb700 1 -- 172.21.15.119:0/778031655 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb30c021070 con 0x55f74fe98ca0 2023-12-07T10:31:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.573+0000 7fb301ffb700 1 -- 172.21.15.119:0/778031655 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c025700 con 0x55f74fe98ca0 2023-12-07T10:31:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.574+0000 7fb335444b80 1 -- 172.21.15.119:0/778031655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f74fe98ca0 msgr2=0x55f74fe99080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:17.576 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.574+0000 7fb335444b80 1 --2- 172.21.15.119:0/778031655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f74fe98ca0 0x55f74fe99080 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00e430 tx=0x7fb30c014000 comp rx=0 tx=0).stop 2023-12-07T10:31:17.576 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.574+0000 7fb335444b80 1 -- 172.21.15.119:0/778031655 shutdown_connections 2023-12-07T10:31:17.576 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.574+0000 7fb335444b80 1 --2- 172.21.15.119:0/778031655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74fe93a80 0x55f74fe93f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.576 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.574+0000 7fb335444b80 1 --2- 172.21.15.119:0/778031655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f74fe98ca0 0x55f74fe99080 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.576 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.574+0000 7fb335444b80 1 --2- 172.21.15.119:0/778031655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f74fe995c0 0x55f74fe93540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.576 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.574+0000 7fb335444b80 1 -- 172.21.15.119:0/778031655 >> 172.21.15.119:0/778031655 conn(0x55f74fdc2300 msgr2=0x55f74fe8dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:17.576 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.574+0000 7fb335444b80 1 -- 172.21.15.119:0/778031655 shutdown_connections 2023-12-07T10:31:17.576 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.574+0000 7fb335444b80 1 -- 172.21.15.119:0/778031655 wait complete. 2023-12-07T10:31:17.576 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.574+0000 7fb335444b80 1 -- start start 2023-12-07T10:31:17.576 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.575+0000 7fb335444b80 1 Processor -- start 2023-12-07T10:31:17.576 INFO:tasks.workunit.client.0.smithi119.stdout: 1 786508 7.20 MB/sec execute 450 sec latency 19.518 ms 2023-12-07T10:31:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.575+0000 7fb335444b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74fe93a80 0x55f74fe9f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.575+0000 7fb335444b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f74fe98ca0 0x55f74fe9c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.575+0000 7fb335444b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f74fe995c0 0x55f74fe9d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.575+0000 7fb335444b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74fde7950 con 0x55f74fe93a80 2023-12-07T10:31:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.575+0000 7fb335444b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74fda1500 con 0x55f74fe995c0 2023-12-07T10:31:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.575+0000 7fb335444b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74fda11a0 con 0x55f74fe98ca0 2023-12-07T10:31:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.575+0000 7fb335444b80 10 client.0 _ll_register_callbacks cb 0x55f74fd888e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:31:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.576+0000 7fb31affd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f74fe98ca0 0x55f74fe9c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.576+0000 7fb31b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74fe93a80 0x55f74fe9f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.576+0000 7fb31affd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f74fe98ca0 0x55f74fe9c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:32944/0 (socket says 172.21.15.119:32944) 2023-12-07T10:31:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.576+0000 7fb31affd700 1 -- 172.21.15.119:0/43229078 learned_addr learned my addr 172.21.15.119:0/43229078 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:31:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.576+0000 7fb31bfff700 1 --2- 172.21.15.119:0/43229078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f74fe995c0 0x55f74fe9d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.576+0000 7fb31affd700 1 -- 172.21.15.119:0/43229078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f74fe995c0 msgr2=0x55f74fe9d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.576+0000 7fb31affd700 1 --2- 172.21.15.119:0/43229078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f74fe995c0 0x55f74fe9d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.576+0000 7fb31affd700 1 -- 172.21.15.119:0/43229078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74fe93a80 msgr2=0x55f74fe9f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.576+0000 7fb31affd700 1 --2- 172.21.15.119:0/43229078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74fe93a80 0x55f74fe9f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.576+0000 7fb31affd700 1 -- 172.21.15.119:0/43229078 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31c1a4830 con 0x55f74fe98ca0 2023-12-07T10:31:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.576+0000 7fb31b7fe700 1 --2- 172.21.15.119:0/43229078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74fe93a80 0x55f74fe9f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:31:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.576+0000 7fb31affd700 1 --2- 172.21.15.119:0/43229078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f74fe98ca0 0x55f74fe9c920 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb310009990 tx=0x7fb310005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57c984f47f4aecd2 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.576+0000 7fb2faffd700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:31:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.577+0000 7fb2faffd700 1 -- 172.21.15.119:0/43229078 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31001d070 con 0x55f74fe98ca0 2023-12-07T10:31:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.577+0000 7fb335444b80 1 -- 172.21.15.119:0/43229078 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f74fe866f0 con 0x55f74fe98ca0 2023-12-07T10:31:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.577+0000 7fb2faffd700 1 -- 172.21.15.119:0/43229078 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fb3100056c0 con 0x55f74fe98ca0 2023-12-07T10:31:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.577+0000 7fb2faffd700 1 -- 172.21.15.119:0/43229078 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb310004910 con 0x55f74fe98ca0 2023-12-07T10:31:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.577+0000 7fb335444b80 10 client.27652 fetch_fsmap learned FSMap version 59 2023-12-07T10:31:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.577+0000 7fb335444b80 1 -- 172.21.15.119:0/43229078 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x7fb30c017070 con 0x55f74fe98ca0 2023-12-07T10:31:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.577+0000 7fb2faffd700 1 -- 172.21.15.119:0/43229078 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f74fe866f0 con 0x55f74fe98ca0 2023-12-07T10:31:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.577+0000 7fb2faffd700 1 -- 172.21.15.119:0/43229078 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 59) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb31000fa40 con 0x55f74fe98ca0 2023-12-07T10:31:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.577+0000 7fb335444b80 10 client.27652 fetch_fsmap finished waiting for FSMap version 59 2023-12-07T10:31:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.577+0000 7fb335444b80 10 client.27652 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:31:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.577+0000 7fb335444b80 20 client.27652 populate_metadata read hostname 'smithi119' 2023-12-07T10:31:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.577+0000 7fb335444b80 1 --2- 172.21.15.119:0/43229078 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f74feb14a0 0x55f74feb38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.577+0000 7fb335444b80 4 client.27652 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:31:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.577+0000 7fb335444b80 1 -- 172.21.15.119:0/43229078 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f74fe62740 con 0x55f74feb14a0 2023-12-07T10:31:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.578+0000 7fb31b7fe700 1 --2- 172.21.15.119:0/43229078 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f74feb14a0 0x55f74feb38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.578+0000 7fb31b7fe700 1 --2- 172.21.15.119:0/43229078 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f74feb14a0 0x55f74feb38c0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36ed5b6c5665ae81 server_cookie=c9fb6100055069ac in_seq=0 out_seq=0 2023-12-07T10:31:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.578+0000 7fb2faffd700 10 client.27652 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:31:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.579+0000 7fb2faffd700 1 -- 172.21.15.119:0/43229078 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f74fe62740 con 0x55f74feb14a0 2023-12-07T10:31:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.579+0000 7fb2faffd700 10 client.27652 handle_command_reply: tid=0 2023-12-07T10:31:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.579+0000 7fb335444b80 1 client.27652 shutdown 2023-12-07T10:31:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.579+0000 7fb335444b80 1 -- 172.21.15.119:0/43229078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f74fe98ca0 msgr2=0x55f74fe9c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.579+0000 7fb335444b80 1 --2- 172.21.15.119:0/43229078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f74fe98ca0 0x55f74fe9c920 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb310009990 tx=0x7fb310005270 comp rx=0 tx=0).stop 2023-12-07T10:31:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.579+0000 7fb335444b80 1 -- 172.21.15.119:0/43229078 shutdown_connections 2023-12-07T10:31:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.579+0000 7fb335444b80 1 --2- 172.21.15.119:0/43229078 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f74feb14a0 0x55f74feb38c0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.579+0000 7fb335444b80 1 --2- 172.21.15.119:0/43229078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f74fe995c0 0x55f74fe9d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.579+0000 7fb335444b80 1 --2- 172.21.15.119:0/43229078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74fe93a80 0x55f74fe9f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.579+0000 7fb335444b80 1 --2- 172.21.15.119:0/43229078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f74fe98ca0 0x55f74fe9c920 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.579+0000 7fb2faffd700 0 client.27652 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:31:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.579+0000 7fb335444b80 1 -- 172.21.15.119:0/43229078 >> 172.21.15.119:0/43229078 conn(0x55f74fdc2300 msgr2=0x55f74fe88980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.580+0000 7fb335444b80 1 -- 172.21.15.119:0/43229078 shutdown_connections 2023-12-07T10:31:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.580+0000 7fb335444b80 1 -- 172.21.15.119:0/43229078 wait complete. 2023-12-07T10:31:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.580+0000 7fb335444b80 20 client.27652 trim_cache size 0 max 16384 2023-12-07T10:31:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.598+0000 7fb335444b80 1 Processor -- start 2023-12-07T10:31:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.598+0000 7fb335444b80 1 -- start start 2023-12-07T10:31:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.598+0000 7fb335444b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74fe93a80 0x55f74fead4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.598+0000 7fb335444b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f74fe98ca0 0x55f74fe9f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.598+0000 7fb335444b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f74fe995c0 0x55f74feb8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.598+0000 7fb335444b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74fc668c0 con 0x55f74fe93a80 2023-12-07T10:31:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.598+0000 7fb335444b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74fe866d0 con 0x55f74fe98ca0 2023-12-07T10:31:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.598+0000 7fb335444b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74fe8e4c0 con 0x55f74fe995c0 2023-12-07T10:31:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.599+0000 7fb31b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74fe93a80 0x55f74fead4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.599+0000 7fb31affd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f74fe98ca0 0x55f74fe9f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.599+0000 7fb31affd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f74fe98ca0 0x55f74fe9f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:41464/0 (socket says 172.21.15.119:41464) 2023-12-07T10:31:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.599+0000 7fb31b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74fe93a80 0x55f74fead4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47774/0 (socket says 172.21.15.119:47774) 2023-12-07T10:31:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.599+0000 7fb31affd700 1 -- 172.21.15.119:0/1786512515 learned_addr learned my addr 172.21.15.119:0/1786512515 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:31:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.599+0000 7fb31bfff700 1 --2- 172.21.15.119:0/1786512515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f74fe995c0 0x55f74feb8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.599+0000 7fb31affd700 1 -- 172.21.15.119:0/1786512515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f74fe995c0 msgr2=0x55f74feb8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.599+0000 7fb31affd700 1 --2- 172.21.15.119:0/1786512515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f74fe995c0 0x55f74feb8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.599+0000 7fb31affd700 1 -- 172.21.15.119:0/1786512515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74fe93a80 msgr2=0x55f74fead4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.599+0000 7fb31affd700 1 --2- 172.21.15.119:0/1786512515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74fe93a80 0x55f74fead4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.599+0000 7fb31affd700 1 -- 172.21.15.119:0/1786512515 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f74fc10d80 con 0x55f74fe98ca0 2023-12-07T10:31:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.599+0000 7fb31bfff700 1 --2- 172.21.15.119:0/1786512515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f74fe995c0 0x55f74feb8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:31:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.599+0000 7fb31affd700 1 --2- 172.21.15.119:0/1786512515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f74fe98ca0 0x55f74fe9f2f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb31000fdf0 tx=0x7fb310005c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31209011dd26217 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.600+0000 7fb3027fc700 1 -- 172.21.15.119:0/1786512515 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31003b070 con 0x55f74fe98ca0 2023-12-07T10:31:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.600+0000 7fb3027fc700 1 -- 172.21.15.119:0/1786512515 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb31001e040 con 0x55f74fe98ca0 2023-12-07T10:31:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.600+0000 7fb3027fc700 1 -- 172.21.15.119:0/1786512515 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31001d070 con 0x55f74fe98ca0 2023-12-07T10:31:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.600+0000 7fb335444b80 1 -- 172.21.15.119:0/1786512515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f74fe98ca0 msgr2=0x55f74fe9f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.600+0000 7fb335444b80 1 --2- 172.21.15.119:0/1786512515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f74fe98ca0 0x55f74fe9f2f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb31000fdf0 tx=0x7fb310005c40 comp rx=0 tx=0).stop 2023-12-07T10:31:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.600+0000 7fb335444b80 1 -- 172.21.15.119:0/1786512515 shutdown_connections 2023-12-07T10:31:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.600+0000 7fb335444b80 1 --2- 172.21.15.119:0/1786512515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f74fe995c0 0x55f74feb8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.600+0000 7fb335444b80 1 --2- 172.21.15.119:0/1786512515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74fe93a80 0x55f74fead4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.600+0000 7fb335444b80 1 --2- 172.21.15.119:0/1786512515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f74fe98ca0 0x55f74fe9f2f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.600+0000 7fb335444b80 1 -- 172.21.15.119:0/1786512515 >> 172.21.15.119:0/1786512515 conn(0x55f74fdc2300 msgr2=0x55f74fe87960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.601+0000 7fb335444b80 1 -- 172.21.15.119:0/1786512515 shutdown_connections 2023-12-07T10:31:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.601+0000 7fb335444b80 1 -- 172.21.15.119:0/1786512515 wait complete. 2023-12-07T10:31:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.601+0000 7fb335444b80 1 -- start start 2023-12-07T10:31:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.601+0000 7fb335444b80 1 Processor -- start 2023-12-07T10:31:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.602+0000 7fb335444b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f74fe93a80 0x55f74fea4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.602+0000 7fb335444b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74fe98ca0 0x55f74fea4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.602+0000 7fb335444b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f74fe995c0 0x55f74fea2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.602+0000 7fb335444b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74fea2880 con 0x55f74fe98ca0 2023-12-07T10:31:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.602+0000 7fb335444b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74fea29f0 con 0x55f74fe93a80 2023-12-07T10:31:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.602+0000 7fb335444b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f74fea2bb0 con 0x55f74fe995c0 2023-12-07T10:31:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.602+0000 7fb335444b80 10 client.0 _ll_register_callbacks cb 0x55f74fd888e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:31:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.602+0000 7fb31b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f74fe93a80 0x55f74fea4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.602+0000 7fb31affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74fe98ca0 0x55f74fea4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.602+0000 7fb31b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f74fe93a80 0x55f74fea4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:41476/0 (socket says 172.21.15.119:41476) 2023-12-07T10:31:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.602+0000 7fb31b7fe700 1 -- 172.21.15.119:0/1458495622 learned_addr learned my addr 172.21.15.119:0/1458495622 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:31:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.602+0000 7fb31bfff700 1 --2- 172.21.15.119:0/1458495622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f74fe995c0 0x55f74fea2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.602+0000 7fb31b7fe700 1 -- 172.21.15.119:0/1458495622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f74fe995c0 msgr2=0x55f74fea2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.602+0000 7fb31b7fe700 1 --2- 172.21.15.119:0/1458495622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f74fe995c0 0x55f74fea2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.602+0000 7fb31b7fe700 1 -- 172.21.15.119:0/1458495622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74fe98ca0 msgr2=0x55f74fea4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.602+0000 7fb31b7fe700 1 --2- 172.21.15.119:0/1458495622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74fe98ca0 0x55f74fea4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.602+0000 7fb31b7fe700 1 -- 172.21.15.119:0/1458495622 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb30c017070 con 0x55f74fe93a80 2023-12-07T10:31:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.603+0000 7fb31b7fe700 1 --2- 172.21.15.119:0/1458495622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f74fe93a80 0x55f74fea4000 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00e3d0 tx=0x7fb30c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b285c76a0fd13ea server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.603+0000 7fb2faffd700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:31:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.603+0000 7fb2faffd700 1 -- 172.21.15.119:0/1458495622 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c005110 con 0x55f74fe93a80 2023-12-07T10:31:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.603+0000 7fb335444b80 1 -- 172.21.15.119:0/1458495622 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb30c021070 con 0x55f74fe93a80 2023-12-07T10:31:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.603+0000 7fb2faffd700 1 -- 172.21.15.119:0/1458495622 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fb30c005270 con 0x55f74fe93a80 2023-12-07T10:31:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.603+0000 7fb2faffd700 1 -- 172.21.15.119:0/1458495622 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb30c005430 con 0x55f74fe93a80 2023-12-07T10:31:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.603+0000 7fb335444b80 10 client.27683 fetch_fsmap learned FSMap version 59 2023-12-07T10:31:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.603+0000 7fb335444b80 1 -- 172.21.15.119:0/1458495622 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55f74fbf9c00 con 0x55f74fe93a80 2023-12-07T10:31:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.603+0000 7fb2faffd700 1 -- 172.21.15.119:0/1458495622 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb30c021070 con 0x55f74fe93a80 2023-12-07T10:31:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.603+0000 7fb2faffd700 1 -- 172.21.15.119:0/1458495622 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 59) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb30c003680 con 0x55f74fe93a80 2023-12-07T10:31:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.603+0000 7fb335444b80 10 client.27683 fetch_fsmap finished waiting for FSMap version 59 2023-12-07T10:31:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.604+0000 7fb335444b80 10 client.27683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:31:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.604+0000 7fb335444b80 20 client.27683 populate_metadata read hostname 'smithi119' 2023-12-07T10:31:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.604+0000 7fb335444b80 1 --2- 172.21.15.119:0/1458495622 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f74ff07d90 0x55f74ff0a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.604+0000 7fb335444b80 4 client.27683 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:31:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.604+0000 7fb335444b80 1 -- 172.21.15.119:0/1458495622 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f74fdadf70 con 0x55f74ff07d90 2023-12-07T10:31:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.604+0000 7fb31affd700 1 --2- 172.21.15.119:0/1458495622 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f74ff07d90 0x55f74ff0a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.604+0000 7fb31affd700 1 --2- 172.21.15.119:0/1458495622 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f74ff07d90 0x55f74ff0a1b0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62776c3912e820b9 server_cookie=ae75f8214c159e54 in_seq=0 out_seq=0 2023-12-07T10:31:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.604+0000 7fb2faffd700 10 client.27683 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:31:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.677+0000 7fb2faffd700 1 -- 172.21.15.119:0/1458495622 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f74fdadf70 con 0x55f74ff07d90 2023-12-07T10:31:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.677+0000 7fb2faffd700 10 client.27683 handle_command_reply: tid=0 2023-12-07T10:31:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.677+0000 7fb335444b80 1 client.27683 shutdown 2023-12-07T10:31:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.677+0000 7fb335444b80 1 -- 172.21.15.119:0/1458495622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f74fe93a80 msgr2=0x55f74fea4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.677+0000 7fb335444b80 1 --2- 172.21.15.119:0/1458495622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f74fe93a80 0x55f74fea4000 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00e3d0 tx=0x7fb30c00eeb0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.677+0000 7fb335444b80 1 -- 172.21.15.119:0/1458495622 shutdown_connections 2023-12-07T10:31:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.677+0000 7fb335444b80 1 --2- 172.21.15.119:0/1458495622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f74fe995c0 0x55f74fea2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.677+0000 7fb335444b80 1 --2- 172.21.15.119:0/1458495622 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55f74ff07d90 0x55f74ff0a1b0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.677+0000 7fb335444b80 1 --2- 172.21.15.119:0/1458495622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f74fe93a80 0x55f74fea4000 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.677+0000 7fb2faffd700 0 client.27683 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:31:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.677+0000 7fb335444b80 1 --2- 172.21.15.119:0/1458495622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f74fe98ca0 0x55f74fea4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.677+0000 7fb335444b80 1 -- 172.21.15.119:0/1458495622 >> 172.21.15.119:0/1458495622 conn(0x55f74fdc2300 msgr2=0x55f74fe9f250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.678+0000 7fb335444b80 1 -- 172.21.15.119:0/1458495622 shutdown_connections 2023-12-07T10:31:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.678+0000 7fb335444b80 1 -- 172.21.15.119:0/1458495622 wait complete. 2023-12-07T10:31:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.678+0000 7fb335444b80 20 client.27683 trim_cache size 0 max 16384 2023-12-07T10:31:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:31:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:31:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:31:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:31:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.678+0000 7fb32099d700 1 -- 172.21.15.119:0/3129853205 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb3040a1c80 msgr2=0x7fb3040a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.678+0000 7fb32099d700 1 --2- 172.21.15.119:0/3129853205 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb3040a1c80 0x7fb3040a4140 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb30c009c60 tx=0x7fb30c009260 comp rx=0 tx=0).stop 2023-12-07T10:31:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.678+0000 7fb32099d700 1 -- 172.21.15.119:0/3129853205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb31c197810 msgr2=0x7fb31c197cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.678+0000 7fb32099d700 1 --2- 172.21.15.119:0/3129853205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb31c197810 0x7fb31c197cc0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb31400c370 tx=0x7fb31400c680 comp rx=0 tx=0).stop 2023-12-07T10:31:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.679+0000 7fb32099d700 1 -- 172.21.15.119:0/3129853205 shutdown_connections 2023-12-07T10:31:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.679+0000 7fb32099d700 1 --2- 172.21.15.119:0/3129853205 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb3040a1c80 0x7fb3040a4140 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.679+0000 7fb32099d700 1 --2- 172.21.15.119:0/3129853205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb31c197810 0x7fb31c197cc0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.679+0000 7fb32099d700 1 --2- 172.21.15.119:0/3129853205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb31c1015b0 0x7fb31c19d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.679+0000 7fb32099d700 1 --2- 172.21.15.119:0/3129853205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb31c1066d0 0x7fb31c19dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.679+0000 7fb32099d700 1 -- 172.21.15.119:0/3129853205 >> 172.21.15.119:0/3129853205 conn(0x7fb31c0fce70 msgr2=0x7fb31c0fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.679+0000 7fb32099d700 1 -- 172.21.15.119:0/3129853205 shutdown_connections 2023-12-07T10:31:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:17.679+0000 7fb32099d700 1 -- 172.21.15.119:0/3129853205 wait complete. 2023-12-07T10:31:18.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:18 smithi119 ceph-mon[97908]: pgmap v339: 129 pgs: 129 active+clean; 580 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 236 op/s 2023-12-07T10:31:18.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:18 smithi159 ceph-mon[134726]: pgmap v339: 129 pgs: 129 active+clean; 580 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 236 op/s 2023-12-07T10:31:18.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:18 smithi153 ceph-mon[133625]: pgmap v339: 129 pgs: 129 active+clean; 580 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 236 op/s 2023-12-07T10:31:18.575 INFO:tasks.workunit.client.0.smithi119.stdout: 1 788038 7.21 MB/sec execute 451 sec latency 65.570 ms 2023-12-07T10:31:19.575 INFO:tasks.workunit.client.0.smithi119.stdout: 1 789020 7.20 MB/sec execute 452 sec latency 25.893 ms 2023-12-07T10:31:20.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:20 smithi119 ceph-mon[97908]: pgmap v340: 129 pgs: 129 active+clean; 580 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 236 op/s 2023-12-07T10:31:20.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:20 smithi159 ceph-mon[134726]: pgmap v340: 129 pgs: 129 active+clean; 580 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 236 op/s 2023-12-07T10:31:20.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:20 smithi153 ceph-mon[133625]: pgmap v340: 129 pgs: 129 active+clean; 580 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 236 op/s 2023-12-07T10:31:20.575 INFO:tasks.workunit.client.0.smithi119.stdout: 1 790825 7.20 MB/sec execute 453 sec latency 19.150 ms 2023-12-07T10:31:21.575 INFO:tasks.workunit.client.0.smithi119.stdout: 1 792238 7.21 MB/sec execute 454 sec latency 12.167 ms 2023-12-07T10:31:22.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:21 smithi119 ceph-mon[97908]: pgmap v341: 129 pgs: 129 active+clean; 589 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 380 op/s 2023-12-07T10:31:22.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:21 smithi159 ceph-mon[134726]: pgmap v341: 129 pgs: 129 active+clean; 589 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 380 op/s 2023-12-07T10:31:22.279 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:21 smithi153 ceph-mon[133625]: pgmap v341: 129 pgs: 129 active+clean; 589 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 380 op/s 2023-12-07T10:31:22.575 INFO:tasks.workunit.client.0.smithi119.stdout: 1 793956 7.21 MB/sec execute 455 sec latency 17.312 ms 2023-12-07T10:31:23.575 INFO:tasks.workunit.client.0.smithi119.stdout: 1 795527 7.22 MB/sec execute 456 sec latency 63.560 ms 2023-12-07T10:31:24.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:24 smithi119 ceph-mon[97908]: pgmap v342: 129 pgs: 129 active+clean; 590 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 279 op/s 2023-12-07T10:31:24.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:24 smithi159 ceph-mon[134726]: pgmap v342: 129 pgs: 129 active+clean; 590 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 279 op/s 2023-12-07T10:31:24.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:24 smithi153 ceph-mon[133625]: pgmap v342: 129 pgs: 129 active+clean; 590 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 279 op/s 2023-12-07T10:31:24.575 INFO:tasks.workunit.client.0.smithi119.stdout: 1 796358 7.21 MB/sec execute 457 sec latency 19.298 ms 2023-12-07T10:31:25.575 INFO:tasks.workunit.client.0.smithi119.stdout: 1 798095 7.21 MB/sec execute 458 sec latency 24.271 ms 2023-12-07T10:31:26.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:26 smithi119 ceph-mon[97908]: pgmap v343: 129 pgs: 129 active+clean; 590 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 240 op/s 2023-12-07T10:31:26.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:26 smithi159 ceph-mon[134726]: pgmap v343: 129 pgs: 129 active+clean; 590 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 240 op/s 2023-12-07T10:31:26.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:26 smithi153 ceph-mon[133625]: pgmap v343: 129 pgs: 129 active+clean; 590 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 240 op/s 2023-12-07T10:31:26.575 INFO:tasks.workunit.client.0.smithi119.stdout: 1 799304 7.22 MB/sec execute 459 sec latency 15.142 ms 2023-12-07T10:31:27.575 INFO:tasks.workunit.client.0.smithi119.stdout: 1 800678 7.21 MB/sec execute 460 sec latency 16.459 ms 2023-12-07T10:31:28.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:28 smithi119 ceph-mon[97908]: pgmap v344: 129 pgs: 129 active+clean; 599 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 338 op/s 2023-12-07T10:31:28.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:28 smithi159 ceph-mon[134726]: pgmap v344: 129 pgs: 129 active+clean; 599 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 338 op/s 2023-12-07T10:31:28.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:28 smithi153 ceph-mon[133625]: pgmap v344: 129 pgs: 129 active+clean; 599 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 338 op/s 2023-12-07T10:31:28.575 INFO:tasks.workunit.client.0.smithi119.stdout: 1 802131 7.22 MB/sec execute 461 sec latency 172.758 ms 2023-12-07T10:31:29.575 INFO:tasks.workunit.client.0.smithi119.stdout: 1 802969 7.21 MB/sec execute 462 sec latency 38.829 ms 2023-12-07T10:31:30.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:30 smithi119 ceph-mon[97908]: pgmap v345: 129 pgs: 129 active+clean; 599 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 237 op/s 2023-12-07T10:31:30.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:30 smithi159 ceph-mon[134726]: pgmap v345: 129 pgs: 129 active+clean; 599 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 237 op/s 2023-12-07T10:31:30.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:30 smithi153 ceph-mon[133625]: pgmap v345: 129 pgs: 129 active+clean; 599 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 237 op/s 2023-12-07T10:31:30.575 INFO:tasks.workunit.client.0.smithi119.stdout: 1 804552 7.21 MB/sec execute 463 sec latency 18.290 ms 2023-12-07T10:31:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.842+0000 7f557cdff700 1 -- 172.21.15.119:0/3366533156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55780739e0 msgr2=0x7f5578073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.842+0000 7f557cdff700 1 --2- 172.21.15.119:0/3366533156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55780739e0 0x7f5578073dc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5568009990 tx=0x7f5568009ca0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.842+0000 7f557cdff700 1 -- 172.21.15.119:0/3366533156 shutdown_connections 2023-12-07T10:31:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.842+0000 7f557cdff700 1 --2- 172.21.15.119:0/3366533156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f557810dcc0 0x7f55781100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.842+0000 7f557cdff700 1 --2- 172.21.15.119:0/3366533156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55780739e0 0x7f5578073dc0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.842+0000 7f557cdff700 1 --2- 172.21.15.119:0/3366533156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5578074300 0x7f557810d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.842+0000 7f557cdff700 1 -- 172.21.15.119:0/3366533156 >> 172.21.15.119:0/3366533156 conn(0x7f55780fcf50 msgr2=0x7f55780ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.842+0000 7f557cdff700 1 -- 172.21.15.119:0/3366533156 shutdown_connections 2023-12-07T10:31:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.842+0000 7f557cdff700 1 -- 172.21.15.119:0/3366533156 wait complete. 2023-12-07T10:31:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.843+0000 7f557cdff700 1 Processor -- start 2023-12-07T10:31:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.844+0000 7f557cdff700 1 -- start start 2023-12-07T10:31:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.844+0000 7f557cdff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55780739e0 0x7f557819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.844+0000 7f557cdff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5578074300 0x7f557819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.844+0000 7f557cdff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f557810dcc0 0x7f5578199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.844+0000 7f557cdff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5578199fb0 con 0x7f557810dcc0 2023-12-07T10:31:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.844+0000 7f557cdff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557819a120 con 0x7f5578074300 2023-12-07T10:31:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.844+0000 7f557cdff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557819a2e0 con 0x7f55780739e0 2023-12-07T10:31:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.844+0000 7f557659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55780739e0 0x7f557819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.844+0000 7f557659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55780739e0 0x7f557819f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:37952/0 (socket says 172.21.15.119:37952) 2023-12-07T10:31:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.844+0000 7f557659c700 1 -- 172.21.15.119:0/1568532380 learned_addr learned my addr 172.21.15.119:0/1568532380 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:31:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.844+0000 7f5575d9b700 1 --2- 172.21.15.119:0/1568532380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5578074300 0x7f557819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.844+0000 7f5576d9d700 1 --2- 172.21.15.119:0/1568532380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f557810dcc0 0x7f5578199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.845+0000 7f557659c700 1 -- 172.21.15.119:0/1568532380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5578074300 msgr2=0x7f557819fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.845+0000 7f557659c700 1 --2- 172.21.15.119:0/1568532380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5578074300 0x7f557819fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.845+0000 7f557659c700 1 -- 172.21.15.119:0/1568532380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f557810dcc0 msgr2=0x7f5578199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.845+0000 7f557659c700 1 --2- 172.21.15.119:0/1568532380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f557810dcc0 0x7f5578199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.845+0000 7f557659c700 1 -- 172.21.15.119:0/1568532380 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55680095f0 con 0x7f55780739e0 2023-12-07T10:31:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.845+0000 7f557659c700 1 --2- 172.21.15.119:0/1568532380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55780739e0 0x7f557819f820 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5568005910 tx=0x7f5568004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.845+0000 7f55677fe700 1 -- 172.21.15.119:0/1568532380 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556801d070 con 0x7f55780739e0 2023-12-07T10:31:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.845+0000 7f55677fe700 1 -- 172.21.15.119:0/1568532380 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5568022470 con 0x7f55780739e0 2023-12-07T10:31:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.845+0000 7f55677fe700 1 -- 172.21.15.119:0/1568532380 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5568017650 con 0x7f55780739e0 2023-12-07T10:31:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.845+0000 7f557cdff700 1 -- 172.21.15.119:0/1568532380 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f557819a560 con 0x7f55780739e0 2023-12-07T10:31:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.845+0000 7f557cdff700 1 -- 172.21.15.119:0/1568532380 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55781a6810 con 0x7f55780739e0 2023-12-07T10:31:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.850+0000 7f55677fe700 1 -- 172.21.15.119:0/1568532380 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f55680225e0 con 0x7f55780739e0 2023-12-07T10:31:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.851+0000 7f55906cdb80 1 Processor -- start 2023-12-07T10:31:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.851+0000 7f55906cdb80 1 -- start start 2023-12-07T10:31:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.852+0000 7f55677fe700 1 --2- 172.21.15.119:0/1568532380 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f55600a1cf0 0x7f55600a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.852+0000 7f55906cdb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b817effa60 0x55b817effe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.852+0000 7f55677fe700 1 -- 172.21.15.119:0/1568532380 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f55680cf000 con 0x7f55780739e0 2023-12-07T10:31:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.852+0000 7f55906cdb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b817f00380 0x55b817efa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.852+0000 7f55906cdb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b817efaad0 0x55b817efaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.852+0000 7f55906cdb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b817e6e760 con 0x55b817f00380 2023-12-07T10:31:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.852+0000 7f55906cdb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b817e55740 con 0x55b817effa60 2023-12-07T10:31:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.852+0000 7f55906cdb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b817e53060 con 0x55b817efaad0 2023-12-07T10:31:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.852+0000 7f557659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b817effa60 0x55b817effe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.852+0000 7f557659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b817effa60 0x55b817effe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:38308/0 (socket says 172.21.15.119:38308) 2023-12-07T10:31:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.852+0000 7f557659c700 1 -- 172.21.15.119:0/1376320352 learned_addr learned my addr 172.21.15.119:0/1376320352 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:31:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.852+0000 7f5576d9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b817efaad0 0x55b817efaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.852+0000 7f5576d9d700 1 --2- 172.21.15.119:0/1568532380 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f55600a1cf0 0x7f55600a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.852+0000 7f5575d9b700 1 --2- 172.21.15.119:0/1376320352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b817f00380 0x55b817efa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.853+0000 7f557659c700 1 -- 172.21.15.119:0/1376320352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b817efaad0 msgr2=0x55b817efaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.853+0000 7f557659c700 1 --2- 172.21.15.119:0/1376320352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b817efaad0 0x55b817efaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.853+0000 7f557659c700 1 -- 172.21.15.119:0/1376320352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b817f00380 msgr2=0x55b817efa540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.853+0000 7f557659c700 1 --2- 172.21.15.119:0/1376320352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b817f00380 0x55b817efa540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.853+0000 7f557659c700 1 -- 172.21.15.119:0/1376320352 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b817ccb5d0 con 0x55b817effa60 2023-12-07T10:31:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.853+0000 7f557659c700 1 --2- 172.21.15.119:0/1376320352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b817effa60 0x55b817effe40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f556800f740 tx=0x7f5568009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92fb509496e3aabc server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.853+0000 7f5564ff9700 1 -- 172.21.15.119:0/1376320352 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556809c180 con 0x55b817effa60 2023-12-07T10:31:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.853+0000 7f5576d9d700 1 --2- 172.21.15.119:0/1568532380 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f55600a1cf0 0x7f55600a41b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x55b817ef6670 tx=0x7f557000b040 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.853+0000 7f5564ff9700 1 -- 172.21.15.119:0/1376320352 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f556809ba80 con 0x55b817effa60 2023-12-07T10:31:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.853+0000 7f5564ff9700 1 -- 172.21.15.119:0/1376320352 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55680862d0 con 0x55b817effa60 2023-12-07T10:31:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.854+0000 7f55906cdb80 1 -- 172.21.15.119:0/1376320352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b817effa60 msgr2=0x55b817effe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.854+0000 7f55906cdb80 1 --2- 172.21.15.119:0/1376320352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b817effa60 0x55b817effe40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f556800f740 tx=0x7f5568009ca0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.854+0000 7f55906cdb80 1 -- 172.21.15.119:0/1376320352 shutdown_connections 2023-12-07T10:31:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.854+0000 7f55906cdb80 1 --2- 172.21.15.119:0/1376320352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b817efaad0 0x55b817efaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.854+0000 7f55906cdb80 1 --2- 172.21.15.119:0/1376320352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b817effa60 0x55b817effe40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.854+0000 7f55906cdb80 1 --2- 172.21.15.119:0/1376320352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b817f00380 0x55b817efa540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.854+0000 7f55906cdb80 1 -- 172.21.15.119:0/1376320352 >> 172.21.15.119:0/1376320352 conn(0x55b817e29300 msgr2=0x55b817ef4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.854+0000 7f55906cdb80 1 -- 172.21.15.119:0/1376320352 shutdown_connections 2023-12-07T10:31:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.854+0000 7f55906cdb80 1 -- 172.21.15.119:0/1376320352 wait complete. 2023-12-07T10:31:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.855+0000 7f55906cdb80 1 -- start start 2023-12-07T10:31:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.855+0000 7f55906cdb80 1 Processor -- start 2023-12-07T10:31:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.856+0000 7f55906cdb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b817efaad0 0x55b817f06380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.856+0000 7f55906cdb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b817effa60 0x55b817f03670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.856+0000 7f55906cdb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b817f00380 0x55b817f03db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.856+0000 7f55906cdb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b817e4e950 con 0x55b817effa60 2023-12-07T10:31:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.856+0000 7f55906cdb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b817e08500 con 0x55b817f00380 2023-12-07T10:31:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.856+0000 7f55906cdb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b817e081a0 con 0x55b817efaad0 2023-12-07T10:31:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.856+0000 7f55906cdb80 10 client.0 _ll_register_callbacks cb 0x55b817def8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:31:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.856+0000 7f557659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b817efaad0 0x55b817f06380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.856+0000 7f557659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b817efaad0 0x55b817f06380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:37972/0 (socket says 172.21.15.119:37972) 2023-12-07T10:31:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.856+0000 7f557659c700 1 -- 172.21.15.119:0/3613059023 learned_addr learned my addr 172.21.15.119:0/3613059023 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:31:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.856+0000 7f5575d9b700 1 --2- 172.21.15.119:0/3613059023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b817effa60 0x55b817f03670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.856+0000 7f5576d9d700 1 --2- 172.21.15.119:0/3613059023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b817f00380 0x55b817f03db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.856+0000 7f557659c700 1 -- 172.21.15.119:0/3613059023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b817f00380 msgr2=0x55b817f03db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.856+0000 7f557659c700 1 --2- 172.21.15.119:0/3613059023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b817f00380 0x55b817f03db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.856+0000 7f557659c700 1 -- 172.21.15.119:0/3613059023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b817effa60 msgr2=0x55b817f03670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.856+0000 7f557659c700 1 --2- 172.21.15.119:0/3613059023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b817effa60 0x55b817f03670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.856+0000 7f557659c700 1 -- 172.21.15.119:0/3613059023 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55781a6810 con 0x55b817efaad0 2023-12-07T10:31:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.856+0000 7f5575d9b700 1 --2- 172.21.15.119:0/3613059023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b817effa60 0x55b817f03670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:31:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.856+0000 7f5576d9d700 1 --2- 172.21.15.119:0/3613059023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b817f00380 0x55b817f03db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:31:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.857+0000 7f557659c700 1 --2- 172.21.15.119:0/3613059023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b817efaad0 0x55b817f06380 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f55680cc450 tx=0x7f5568022af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bab73f5d3e67f63e server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.857+0000 7f555dffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:31:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.857+0000 7f555dffb700 1 -- 172.21.15.119:0/3613059023 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556809fbd0 con 0x55b817efaad0 2023-12-07T10:31:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.857+0000 7f55906cdb80 1 -- 172.21.15.119:0/3613059023 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b817eed6f0 con 0x55b817efaad0 2023-12-07T10:31:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.857+0000 7f555dffb700 1 -- 172.21.15.119:0/3613059023 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f55680a3d80 con 0x55b817efaad0 2023-12-07T10:31:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.857+0000 7f555dffb700 1 -- 172.21.15.119:0/3613059023 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f556809af70 con 0x55b817efaad0 2023-12-07T10:31:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.857+0000 7f55906cdb80 10 client.27670 fetch_fsmap learned FSMap version 59 2023-12-07T10:31:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.857+0000 7f55906cdb80 1 -- 172.21.15.119:0/3613059023 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55b817c77d80 con 0x55b817efaad0 2023-12-07T10:31:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.857+0000 7f555dffb700 1 -- 172.21.15.119:0/3613059023 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b817eed6f0 con 0x55b817efaad0 2023-12-07T10:31:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.857+0000 7f555dffb700 1 -- 172.21.15.119:0/3613059023 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 59) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f55680860d0 con 0x55b817efaad0 2023-12-07T10:31:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.857+0000 7f55906cdb80 10 client.27670 fetch_fsmap finished waiting for FSMap version 59 2023-12-07T10:31:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.857+0000 7f55906cdb80 10 client.27670 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:31:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.857+0000 7f55906cdb80 20 client.27670 populate_metadata read hostname 'smithi119' 2023-12-07T10:31:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.857+0000 7f55906cdb80 1 --2- 172.21.15.119:0/3613059023 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55b817f13630 0x55b817f15a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.858+0000 7f55906cdb80 4 client.27670 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:31:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.858+0000 7f55906cdb80 1 -- 172.21.15.119:0/3613059023 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b817ec9740 con 0x55b817f13630 2023-12-07T10:31:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.858+0000 7f5575d9b700 1 --2- 172.21.15.119:0/3613059023 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55b817f13630 0x55b817f15a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.858+0000 7f555dffb700 10 client.27670 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:31:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.858+0000 7f5575d9b700 1 --2- 172.21.15.119:0/3613059023 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55b817f13630 0x55b817f15a50 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0dffdb11f94a0f7 server_cookie=5b59dc22e3a0ef6b in_seq=0 out_seq=0 2023-12-07T10:31:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.859+0000 7f555dffb700 1 -- 172.21.15.119:0/3613059023 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b817ec9740 con 0x55b817f13630 2023-12-07T10:31:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.859+0000 7f555dffb700 10 client.27670 handle_command_reply: tid=0 2023-12-07T10:31:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.859+0000 7f55906cdb80 1 client.27670 shutdown 2023-12-07T10:31:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.859+0000 7f55906cdb80 1 -- 172.21.15.119:0/3613059023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b817efaad0 msgr2=0x55b817f06380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.859+0000 7f55906cdb80 1 --2- 172.21.15.119:0/3613059023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b817efaad0 0x55b817f06380 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f55680cc450 tx=0x7f5568022af0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.859+0000 7f55906cdb80 1 -- 172.21.15.119:0/3613059023 shutdown_connections 2023-12-07T10:31:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.859+0000 7f55906cdb80 1 --2- 172.21.15.119:0/3613059023 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55b817f13630 0x55b817f15a50 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.859+0000 7f55906cdb80 1 --2- 172.21.15.119:0/3613059023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b817f00380 0x55b817f03db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.859+0000 7f55906cdb80 1 --2- 172.21.15.119:0/3613059023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b817efaad0 0x55b817f06380 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.859+0000 7f55906cdb80 1 --2- 172.21.15.119:0/3613059023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b817effa60 0x55b817f03670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.859+0000 7f555dffb700 0 client.27670 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:31:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.859+0000 7f55906cdb80 1 -- 172.21.15.119:0/3613059023 >> 172.21.15.119:0/3613059023 conn(0x55b817e29300 msgr2=0x55b817eef980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.859+0000 7f55906cdb80 1 -- 172.21.15.119:0/3613059023 shutdown_connections 2023-12-07T10:31:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.859+0000 7f55906cdb80 1 -- 172.21.15.119:0/3613059023 wait complete. 2023-12-07T10:31:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.860+0000 7f55906cdb80 20 client.27670 trim_cache size 0 max 16384 2023-12-07T10:31:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.877+0000 7f55906cdb80 1 Processor -- start 2023-12-07T10:31:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.877+0000 7f55906cdb80 1 -- start start 2023-12-07T10:31:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.878+0000 7f55906cdb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b817efaad0 0x55b817efa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.878+0000 7f55906cdb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b817effa60 0x55b817efd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.878+0000 7f55906cdb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b817f00380 0x55b817efde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.878+0000 7f55906cdb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b817ccd8c0 con 0x55b817f00380 2023-12-07T10:31:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.878+0000 7f55906cdb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b817f10580 con 0x55b817effa60 2023-12-07T10:31:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.878+0000 7f55906cdb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b817eed6c0 con 0x55b817efaad0 2023-12-07T10:31:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.878+0000 7f557659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b817efaad0 0x55b817efa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.878+0000 7f5576d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b817f00380 0x55b817efde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.878+0000 7f5576d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b817f00380 0x55b817efde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33264/0 (socket says 172.21.15.119:33264) 2023-12-07T10:31:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.878+0000 7f5575d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b817effa60 0x55b817efd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.878+0000 7f5576d9d700 1 -- 172.21.15.119:0/249801967 learned_addr learned my addr 172.21.15.119:0/249801967 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:31:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.878+0000 7f557659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b817efaad0 0x55b817efa540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:37988/0 (socket says 172.21.15.119:37988) 2023-12-07T10:31:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.878+0000 7f5575d9b700 1 -- 172.21.15.119:0/249801967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b817efaad0 msgr2=0x55b817efa540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.879+0000 7f5575d9b700 1 --2- 172.21.15.119:0/249801967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b817efaad0 0x55b817efa540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.879+0000 7f5575d9b700 1 -- 172.21.15.119:0/249801967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b817f00380 msgr2=0x55b817efde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.879+0000 7f5575d9b700 1 --2- 172.21.15.119:0/249801967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b817f00380 0x55b817efde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.879+0000 7f5575d9b700 1 -- 172.21.15.119:0/249801967 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b817c60c00 con 0x55b817effa60 2023-12-07T10:31:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.879+0000 7f5575d9b700 1 --2- 172.21.15.119:0/249801967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b817effa60 0x55b817efd8c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x55b817f162f0 tx=0x7f556c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=468fa9ee697d796b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.879+0000 7f55657fa700 1 -- 172.21.15.119:0/249801967 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c011dc0 con 0x55b817effa60 2023-12-07T10:31:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.879+0000 7f55657fa700 1 -- 172.21.15.119:0/249801967 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f556c018040 con 0x55b817effa60 2023-12-07T10:31:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.879+0000 7f55657fa700 1 -- 172.21.15.119:0/249801967 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c005280 con 0x55b817effa60 2023-12-07T10:31:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.880+0000 7f55906cdb80 1 -- 172.21.15.119:0/249801967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b817effa60 msgr2=0x55b817efd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.880+0000 7f55906cdb80 1 --2- 172.21.15.119:0/249801967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b817effa60 0x55b817efd8c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x55b817f162f0 tx=0x7f556c009260 comp rx=0 tx=0).stop 2023-12-07T10:31:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.880+0000 7f55906cdb80 1 -- 172.21.15.119:0/249801967 shutdown_connections 2023-12-07T10:31:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.880+0000 7f55906cdb80 1 --2- 172.21.15.119:0/249801967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b817f00380 0x55b817efde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.880+0000 7f55906cdb80 1 --2- 172.21.15.119:0/249801967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b817efaad0 0x55b817efa540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.880+0000 7f55906cdb80 1 --2- 172.21.15.119:0/249801967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b817effa60 0x55b817efd8c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.880+0000 7f55906cdb80 1 -- 172.21.15.119:0/249801967 >> 172.21.15.119:0/249801967 conn(0x55b817e29300 msgr2=0x55b817f060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.880+0000 7f55906cdb80 1 -- 172.21.15.119:0/249801967 shutdown_connections 2023-12-07T10:31:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.880+0000 7f55906cdb80 1 -- 172.21.15.119:0/249801967 wait complete. 2023-12-07T10:31:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.880+0000 7f55906cdb80 1 -- start start 2023-12-07T10:31:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.881+0000 7f55906cdb80 1 Processor -- start 2023-12-07T10:31:30.884 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.881+0000 7f55906cdb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b817efaad0 0x55b817efc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:30.884 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.881+0000 7f55906cdb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b817effa60 0x55b817efcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.881+0000 7f55906cdb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b817f00380 0x55b817efb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.881+0000 7f55906cdb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b817f0b280 con 0x55b817effa60 2023-12-07T10:31:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.881+0000 7f55906cdb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b817efbb00 con 0x55b817f00380 2023-12-07T10:31:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.881+0000 7f55906cdb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b817efbcc0 con 0x55b817efaad0 2023-12-07T10:31:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.881+0000 7f55906cdb80 10 client.0 _ll_register_callbacks cb 0x55b817def8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:31:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.881+0000 7f557659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b817efaad0 0x55b817efc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.881+0000 7f557659c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b817efaad0 0x55b817efc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:38002/0 (socket says 172.21.15.119:38002) 2023-12-07T10:31:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.881+0000 7f557659c700 1 -- 172.21.15.119:0/2585430374 learned_addr learned my addr 172.21.15.119:0/2585430374 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:31:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.881+0000 7f5576d9d700 1 --2- 172.21.15.119:0/2585430374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b817f00380 0x55b817efb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.882+0000 7f5575d9b700 1 --2- 172.21.15.119:0/2585430374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b817effa60 0x55b817efcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.882+0000 7f557659c700 1 -- 172.21.15.119:0/2585430374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b817f00380 msgr2=0x55b817efb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.882+0000 7f557659c700 1 --2- 172.21.15.119:0/2585430374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b817f00380 0x55b817efb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.882+0000 7f557659c700 1 -- 172.21.15.119:0/2585430374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b817effa60 msgr2=0x55b817efcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.882+0000 7f557659c700 1 --2- 172.21.15.119:0/2585430374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b817effa60 0x55b817efcea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.882+0000 7f557659c700 1 -- 172.21.15.119:0/2585430374 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f556c009eb0 con 0x55b817efaad0 2023-12-07T10:31:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.882+0000 7f5575d9b700 1 --2- 172.21.15.119:0/2585430374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b817effa60 0x55b817efcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:31:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.882+0000 7f557659c700 1 --2- 172.21.15.119:0/2585430374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b817efaad0 0x55b817efc960 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f556809c0e0 tx=0x7f556800b6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c7bfc312de6a6de server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.882+0000 7f555dffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:31:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.882+0000 7f555dffb700 1 -- 172.21.15.119:0/2585430374 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556809b170 con 0x55b817efaad0 2023-12-07T10:31:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.882+0000 7f55906cdb80 1 -- 172.21.15.119:0/2585430374 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f556809ba80 con 0x55b817efaad0 2023-12-07T10:31:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.882+0000 7f555dffb700 1 -- 172.21.15.119:0/2585430374 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f55680862c0 con 0x55b817efaad0 2023-12-07T10:31:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.882+0000 7f555dffb700 1 -- 172.21.15.119:0/2585430374 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f556809b850 con 0x55b817efaad0 2023-12-07T10:31:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.882+0000 7f55906cdb80 10 client.27676 fetch_fsmap learned FSMap version 59 2023-12-07T10:31:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.882+0000 7f55906cdb80 1 -- 172.21.15.119:0/2585430374 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55b817c9e1f0 con 0x55b817efaad0 2023-12-07T10:31:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.883+0000 7f555dffb700 1 -- 172.21.15.119:0/2585430374 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f556809ba80 con 0x55b817efaad0 2023-12-07T10:31:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.883+0000 7f555dffb700 1 -- 172.21.15.119:0/2585430374 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 59) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f556800fac0 con 0x55b817efaad0 2023-12-07T10:31:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.883+0000 7f55906cdb80 10 client.27676 fetch_fsmap finished waiting for FSMap version 59 2023-12-07T10:31:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.883+0000 7f55906cdb80 10 client.27676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:31:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.883+0000 7f55906cdb80 20 client.27676 populate_metadata read hostname 'smithi119' 2023-12-07T10:31:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.883+0000 7f55906cdb80 1 --2- 172.21.15.119:0/2585430374 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55b817f6a270 0x55b817f6c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.883+0000 7f55906cdb80 4 client.27676 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:31:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.883+0000 7f55906cdb80 1 -- 172.21.15.119:0/2585430374 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b817e14f70 con 0x55b817f6a270 2023-12-07T10:31:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.883+0000 7f5575d9b700 1 --2- 172.21.15.119:0/2585430374 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55b817f6a270 0x55b817f6c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.884+0000 7f555dffb700 10 client.27676 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:31:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.884+0000 7f5575d9b700 1 --2- 172.21.15.119:0/2585430374 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55b817f6a270 0x55b817f6c660 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b63122f94f1c90b2 server_cookie=87c076173a123e70 in_seq=0 out_seq=0 2023-12-07T10:31:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.945+0000 7f555dffb700 1 -- 172.21.15.119:0/2585430374 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b817e14f70 con 0x55b817f6a270 2023-12-07T10:31:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.945+0000 7f555dffb700 10 client.27676 handle_command_reply: tid=0 2023-12-07T10:31:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.945+0000 7f55906cdb80 1 client.27676 shutdown 2023-12-07T10:31:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.945+0000 7f55906cdb80 1 -- 172.21.15.119:0/2585430374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b817efaad0 msgr2=0x55b817efc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.945+0000 7f55906cdb80 1 --2- 172.21.15.119:0/2585430374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b817efaad0 0x55b817efc960 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f556809c0e0 tx=0x7f556800b6a0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.945+0000 7f55906cdb80 1 -- 172.21.15.119:0/2585430374 shutdown_connections 2023-12-07T10:31:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.945+0000 7f55906cdb80 1 --2- 172.21.15.119:0/2585430374 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55b817f6a270 0x55b817f6c660 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.945+0000 7f55906cdb80 1 --2- 172.21.15.119:0/2585430374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b817f00380 0x55b817efb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.945+0000 7f55906cdb80 1 --2- 172.21.15.119:0/2585430374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b817efaad0 0x55b817efc960 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.945+0000 7f555dffb700 0 client.27676 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:31:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.945+0000 7f55906cdb80 1 --2- 172.21.15.119:0/2585430374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b817effa60 0x55b817efcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.945+0000 7f55906cdb80 1 -- 172.21.15.119:0/2585430374 >> 172.21.15.119:0/2585430374 conn(0x55b817e29300 msgr2=0x55b817f01f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.945+0000 7f55906cdb80 1 -- 172.21.15.119:0/2585430374 shutdown_connections 2023-12-07T10:31:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.945+0000 7f55906cdb80 1 -- 172.21.15.119:0/2585430374 wait complete. 2023-12-07T10:31:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.946+0000 7f55906cdb80 20 client.27676 trim_cache size 0 max 16384 2023-12-07T10:31:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:31:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:31:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:31:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:31:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.946+0000 7f557cdff700 1 -- 172.21.15.119:0/1568532380 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f55600a1cf0 msgr2=0x7f55600a41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.946+0000 7f557cdff700 1 --2- 172.21.15.119:0/1568532380 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f55600a1cf0 0x7f55600a41b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x55b817ef6670 tx=0x7f557000b040 comp rx=0 tx=0).stop 2023-12-07T10:31:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.946+0000 7f557cdff700 1 -- 172.21.15.119:0/1568532380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55780739e0 msgr2=0x7f557819f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.946+0000 7f557cdff700 1 --2- 172.21.15.119:0/1568532380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55780739e0 0x7f557819f820 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5568005910 tx=0x7f5568004730 comp rx=0 tx=0).stop 2023-12-07T10:31:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.946+0000 7f557cdff700 1 -- 172.21.15.119:0/1568532380 shutdown_connections 2023-12-07T10:31:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.946+0000 7f557cdff700 1 --2- 172.21.15.119:0/1568532380 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f55600a1cf0 0x7f55600a41b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.946+0000 7f557cdff700 1 --2- 172.21.15.119:0/1568532380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f557810dcc0 0x7f5578199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.946+0000 7f557cdff700 1 --2- 172.21.15.119:0/1568532380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55780739e0 0x7f557819f820 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.946+0000 7f557cdff700 1 --2- 172.21.15.119:0/1568532380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5578074300 0x7f557819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.946+0000 7f557cdff700 1 -- 172.21.15.119:0/1568532380 >> 172.21.15.119:0/1568532380 conn(0x7f55780fcf50 msgr2=0x7f55780fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.946+0000 7f557cdff700 1 -- 172.21.15.119:0/1568532380 shutdown_connections 2023-12-07T10:31:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:30.947+0000 7f557cdff700 1 -- 172.21.15.119:0/1568532380 wait complete. 2023-12-07T10:31:31.576 INFO:tasks.workunit.client.0.smithi119.stdout: 1 806181 7.22 MB/sec execute 464 sec latency 53.311 ms 2023-12-07T10:31:32.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:31 smithi119 ceph-mon[97908]: pgmap v346: 129 pgs: 129 active+clean; 602 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 305 op/s 2023-12-07T10:31:32.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:31 smithi159 ceph-mon[134726]: pgmap v346: 129 pgs: 129 active+clean; 602 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 305 op/s 2023-12-07T10:31:32.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:31 smithi153 ceph-mon[133625]: pgmap v346: 129 pgs: 129 active+clean; 602 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 305 op/s 2023-12-07T10:31:32.576 INFO:tasks.workunit.client.0.smithi119.stdout: 1 807212 7.21 MB/sec execute 465 sec latency 12.551 ms 2023-12-07T10:31:33.576 INFO:tasks.workunit.client.0.smithi119.stdout: 1 809140 7.22 MB/sec execute 466 sec latency 17.575 ms 2023-12-07T10:31:34.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:34 smithi119 ceph-mon[97908]: pgmap v347: 129 pgs: 129 active+clean; 607 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 220 op/s 2023-12-07T10:31:34.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:34 smithi159 ceph-mon[134726]: pgmap v347: 129 pgs: 129 active+clean; 607 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 220 op/s 2023-12-07T10:31:34.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:34 smithi153 ceph-mon[133625]: pgmap v347: 129 pgs: 129 active+clean; 607 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 220 op/s 2023-12-07T10:31:34.576 INFO:tasks.workunit.client.0.smithi119.stdout: 1 810161 7.21 MB/sec execute 467 sec latency 21.500 ms 2023-12-07T10:31:35.576 INFO:tasks.workunit.client.0.smithi119.stdout: 1 811932 7.21 MB/sec execute 468 sec latency 16.411 ms 2023-12-07T10:31:36.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:36 smithi119 ceph-mon[97908]: pgmap v348: 129 pgs: 129 active+clean; 607 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 204 op/s 2023-12-07T10:31:36.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:36 smithi159 ceph-mon[134726]: pgmap v348: 129 pgs: 129 active+clean; 607 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 204 op/s 2023-12-07T10:31:36.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:36 smithi153 ceph-mon[133625]: pgmap v348: 129 pgs: 129 active+clean; 607 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 204 op/s 2023-12-07T10:31:36.576 INFO:tasks.workunit.client.0.smithi119.stdout: 1 813427 7.22 MB/sec execute 469 sec latency 12.548 ms 2023-12-07T10:31:37.576 INFO:tasks.workunit.client.0.smithi119.stdout: 1 814739 7.22 MB/sec execute 470 sec latency 18.435 ms 2023-12-07T10:31:38.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:38 smithi119 ceph-mon[97908]: pgmap v349: 129 pgs: 129 active+clean; 615 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 293 op/s 2023-12-07T10:31:38.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:38 smithi159 ceph-mon[134726]: pgmap v349: 129 pgs: 129 active+clean; 615 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 293 op/s 2023-12-07T10:31:38.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:38 smithi153 ceph-mon[133625]: pgmap v349: 129 pgs: 129 active+clean; 615 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 293 op/s 2023-12-07T10:31:38.576 INFO:tasks.workunit.client.0.smithi119.stdout: 1 816735 7.23 MB/sec execute 471 sec latency 14.035 ms 2023-12-07T10:31:39.576 INFO:tasks.workunit.client.0.smithi119.stdout: 1 817496 7.22 MB/sec execute 472 sec latency 64.618 ms 2023-12-07T10:31:40.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:40 smithi119 ceph-mon[97908]: pgmap v350: 129 pgs: 129 active+clean; 615 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 196 op/s 2023-12-07T10:31:40.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:40 smithi159 ceph-mon[134726]: pgmap v350: 129 pgs: 129 active+clean; 615 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 196 op/s 2023-12-07T10:31:40.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:40 smithi153 ceph-mon[133625]: pgmap v350: 129 pgs: 129 active+clean; 615 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 196 op/s 2023-12-07T10:31:40.576 INFO:tasks.workunit.client.0.smithi119.stdout: 1 819175 7.22 MB/sec execute 473 sec latency 25.683 ms 2023-12-07T10:31:41.576 INFO:tasks.workunit.client.0.smithi119.stdout: 1 820562 7.23 MB/sec execute 474 sec latency 13.288 ms 2023-12-07T10:31:42.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:41 smithi119 ceph-mon[97908]: pgmap v351: 129 pgs: 129 active+clean; 619 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 263 op/s 2023-12-07T10:31:42.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:41 smithi159 ceph-mon[134726]: pgmap v351: 129 pgs: 129 active+clean; 619 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 263 op/s 2023-12-07T10:31:42.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:41 smithi153 ceph-mon[133625]: pgmap v351: 129 pgs: 129 active+clean; 619 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 263 op/s 2023-12-07T10:31:42.576 INFO:tasks.workunit.client.0.smithi119.stdout: 1 821839 7.22 MB/sec execute 475 sec latency 21.554 ms 2023-12-07T10:31:43.576 INFO:tasks.workunit.client.0.smithi119.stdout: 1 823770 7.24 MB/sec execute 476 sec latency 35.408 ms 2023-12-07T10:31:44.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.115+0000 7f35145ff700 1 -- 172.21.15.119:0/2971504855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f350c101520 msgr2=0x7f350c1019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:44.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.115+0000 7f35145ff700 1 --2- 172.21.15.119:0/2971504855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f350c101520 0x7f350c1019a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f34fc009920 tx=0x7f34fc009c30 comp rx=0 tx=0).stop 2023-12-07T10:31:44.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.115+0000 7f35145ff700 1 -- 172.21.15.119:0/2971504855 shutdown_connections 2023-12-07T10:31:44.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.115+0000 7f35145ff700 1 --2- 172.21.15.119:0/2971504855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f350c101ee0 0x7f350c10e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.115+0000 7f35145ff700 1 --2- 172.21.15.119:0/2971504855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f350c1075e0 0x7f350c1079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.115+0000 7f35145ff700 1 --2- 172.21.15.119:0/2971504855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f350c101520 0x7f350c1019a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.115+0000 7f35145ff700 1 -- 172.21.15.119:0/2971504855 >> 172.21.15.119:0/2971504855 conn(0x7f350c0fd010 msgr2=0x7f350c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:44.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.115+0000 7f35145ff700 1 -- 172.21.15.119:0/2971504855 shutdown_connections 2023-12-07T10:31:44.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.115+0000 7f35145ff700 1 -- 172.21.15.119:0/2971504855 wait complete. 2023-12-07T10:31:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.116+0000 7f35145ff700 1 Processor -- start 2023-12-07T10:31:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.117+0000 7f35145ff700 1 -- start start 2023-12-07T10:31:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.117+0000 7f35145ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f350c101520 0x7f350c1957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.117+0000 7f35145ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f350c101ee0 0x7f350c195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.117+0000 7f35145ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f350c1075e0 0x7f350c199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.117+0000 7f35145ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c199600 con 0x7f350c1075e0 2023-12-07T10:31:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.117+0000 7f35145ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c199770 con 0x7f350c101520 2023-12-07T10:31:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.117+0000 7f35145ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c199930 con 0x7f350c101ee0 2023-12-07T10:31:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.117+0000 7f351239b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f350c101520 0x7f350c1957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:44.119 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.117+0000 7f3511b9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f350c101ee0 0x7f350c195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.117+0000 7f351239b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f350c101520 0x7f350c1957d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:54180/0 (socket says 172.21.15.119:54180) 2023-12-07T10:31:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.117+0000 7f351239b700 1 -- 172.21.15.119:0/1692015216 learned_addr learned my addr 172.21.15.119:0/1692015216 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:31:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.117+0000 7f3512b9c700 1 --2- 172.21.15.119:0/1692015216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f350c1075e0 0x7f350c199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.118+0000 7f3511b9a700 1 -- 172.21.15.119:0/1692015216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f350c101520 msgr2=0x7f350c1957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.118+0000 7f3511b9a700 1 --2- 172.21.15.119:0/1692015216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f350c101520 0x7f350c1957d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.118+0000 7f3511b9a700 1 -- 172.21.15.119:0/1692015216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f350c1075e0 msgr2=0x7f350c199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.118+0000 7f3511b9a700 1 --2- 172.21.15.119:0/1692015216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f350c1075e0 0x7f350c199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.118+0000 7f3511b9a700 1 -- 172.21.15.119:0/1692015216 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35080095f0 con 0x7f350c101ee0 2023-12-07T10:31:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.118+0000 7f3511b9a700 1 --2- 172.21.15.119:0/1692015216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f350c101ee0 0x7f350c195d10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f34fc01b040 tx=0x7f34fc00bb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.118+0000 7f35037fe700 1 -- 172.21.15.119:0/1692015216 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc005240 con 0x7f350c101ee0 2023-12-07T10:31:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.118+0000 7f35145ff700 1 -- 172.21.15.119:0/1692015216 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34fc009520 con 0x7f350c101ee0 2023-12-07T10:31:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.118+0000 7f35037fe700 1 -- 172.21.15.119:0/1692015216 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34fc018440 con 0x7f350c101ee0 2023-12-07T10:31:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.118+0000 7f35037fe700 1 -- 172.21.15.119:0/1692015216 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc0038e0 con 0x7f350c101ee0 2023-12-07T10:31:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.118+0000 7f35145ff700 1 -- 172.21.15.119:0/1692015216 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350c199e40 con 0x7f350c101ee0 2023-12-07T10:31:44.123 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.122+0000 7f35037fe700 1 -- 172.21.15.119:0/1692015216 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f34fc01f070 con 0x7f350c101ee0 2023-12-07T10:31:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.123+0000 7f3527eaeb80 1 Processor -- start 2023-12-07T10:31:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.124+0000 7f35037fe700 1 --2- 172.21.15.119:0/1692015216 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f34f80a1c80 0x7f34f80a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:44.125 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.124+0000 7f3527eaeb80 1 -- start start 2023-12-07T10:31:44.125 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.124+0000 7f35037fe700 1 -- 172.21.15.119:0/1692015216 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f34fc0d3770 con 0x7f350c101ee0 2023-12-07T10:31:44.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.124+0000 7f3512b9c700 1 --2- 172.21.15.119:0/1692015216 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f34f80a1c80 0x7f34f80a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:44.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.124+0000 7f3527eaeb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d6203c1a0 0x555d6203c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:44.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.124+0000 7f3527eaeb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d62037d50 0x555d620381d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:44.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.124+0000 7f3527eaeb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d62038710 0x555d620440b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:44.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.124+0000 7f3527eaeb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d61fa3760 con 0x555d6203c1a0 2023-12-07T10:31:44.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.124+0000 7f3527eaeb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d61f8a740 con 0x555d62038710 2023-12-07T10:31:44.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.124+0000 7f3527eaeb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d61f88060 con 0x555d62037d50 2023-12-07T10:31:44.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.125+0000 7f3512b9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d62038710 0x555d620440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:44.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.125+0000 7f3512b9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d62038710 0x555d620440b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:54190/0 (socket says 172.21.15.119:54190) 2023-12-07T10:31:44.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.125+0000 7f351239b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d6203c1a0 0x555d6203c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:44.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.125+0000 7f3512b9c700 1 -- 172.21.15.119:0/3274820273 learned_addr learned my addr 172.21.15.119:0/3274820273 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:31:44.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.125+0000 7f3511b9a700 1 --2- 172.21.15.119:0/3274820273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d62037d50 0x555d620381d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.125+0000 7f3511b9a700 1 -- 172.21.15.119:0/3274820273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d62038710 msgr2=0x555d620440b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.125+0000 7f3511b9a700 1 --2- 172.21.15.119:0/3274820273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d62038710 0x555d620440b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.125+0000 7f3511b9a700 1 -- 172.21.15.119:0/3274820273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d6203c1a0 msgr2=0x555d6203c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.125+0000 7f3511b9a700 1 --2- 172.21.15.119:0/3274820273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d6203c1a0 0x555d6203c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.125+0000 7f3511b9a700 1 -- 172.21.15.119:0/3274820273 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d61e005d0 con 0x555d62037d50 2023-12-07T10:31:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.125+0000 7f3512b9c700 1 --2- 172.21.15.119:0/3274820273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d62038710 0x555d620440b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:31:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.125+0000 7f351239b700 1 --2- 172.21.15.119:0/3274820273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d6203c1a0 0x555d6203c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:31:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.125+0000 7f3512b9c700 1 --2- 172.21.15.119:0/1692015216 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f34f80a1c80 0x7f34f80a4140 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f350400f930 tx=0x7f350400fcf0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.125+0000 7f3511b9a700 1 --2- 172.21.15.119:0/3274820273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d62037d50 0x555d620381d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f34fc0d0bc0 tx=0x7f34fc022430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fda7974214f3282 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.125+0000 7f3500bfe700 1 -- 172.21.15.119:0/3274820273 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc0a6040 con 0x555d62037d50 2023-12-07T10:31:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.125+0000 7f3500bfe700 1 -- 172.21.15.119:0/3274820273 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34fc0a02a0 con 0x555d62037d50 2023-12-07T10:31:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.126+0000 7f3500bfe700 1 -- 172.21.15.119:0/3274820273 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc0d3300 con 0x555d62037d50 2023-12-07T10:31:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.126+0000 7f3527eaeb80 1 -- 172.21.15.119:0/3274820273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d62037d50 msgr2=0x555d620381d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.126+0000 7f3527eaeb80 1 --2- 172.21.15.119:0/3274820273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d62037d50 0x555d620381d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f34fc0d0bc0 tx=0x7f34fc022430 comp rx=0 tx=0).stop 2023-12-07T10:31:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.126+0000 7f3527eaeb80 1 -- 172.21.15.119:0/3274820273 shutdown_connections 2023-12-07T10:31:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.126+0000 7f3527eaeb80 1 --2- 172.21.15.119:0/3274820273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d62038710 0x555d620440b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.126+0000 7f3527eaeb80 1 --2- 172.21.15.119:0/3274820273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d6203c1a0 0x555d6203c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.126+0000 7f3527eaeb80 1 --2- 172.21.15.119:0/3274820273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d62037d50 0x555d620381d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.126+0000 7f3527eaeb80 1 -- 172.21.15.119:0/3274820273 >> 172.21.15.119:0/3274820273 conn(0x555d61f5e300 msgr2=0x555d62029b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.126+0000 7f3527eaeb80 1 -- 172.21.15.119:0/3274820273 shutdown_connections 2023-12-07T10:31:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.126+0000 7f3527eaeb80 1 -- 172.21.15.119:0/3274820273 wait complete. 2023-12-07T10:31:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.127+0000 7f3527eaeb80 1 -- start start 2023-12-07T10:31:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.127+0000 7f3527eaeb80 1 Processor -- start 2023-12-07T10:31:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.128+0000 7f3527eaeb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d62037d50 0x555d62046ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.128+0000 7f3527eaeb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d62038710 0x555d62047420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.128+0000 7f3527eaeb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d6203c1a0 0x555d620411a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.128+0000 7f3527eaeb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d61f83950 con 0x555d62038710 2023-12-07T10:31:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.128+0000 7f3527eaeb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d61f3d500 con 0x555d6203c1a0 2023-12-07T10:31:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.128+0000 7f3527eaeb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d61f3d1a0 con 0x555d62037d50 2023-12-07T10:31:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.128+0000 7f3527eaeb80 10 client.0 _ll_register_callbacks cb 0x555d61f248e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:31:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.128+0000 7f351239b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d62037d50 0x555d62046ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.128+0000 7f351239b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d62037d50 0x555d62046ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:38562/0 (socket says 172.21.15.119:38562) 2023-12-07T10:31:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.128+0000 7f351239b700 1 -- 172.21.15.119:0/2575548286 learned_addr learned my addr 172.21.15.119:0/2575548286 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:31:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.128+0000 7f3512b9c700 1 --2- 172.21.15.119:0/2575548286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d6203c1a0 0x555d620411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.128+0000 7f351239b700 1 -- 172.21.15.119:0/2575548286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d6203c1a0 msgr2=0x555d620411a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.128+0000 7f351239b700 1 --2- 172.21.15.119:0/2575548286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d6203c1a0 0x555d620411a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.128+0000 7f351239b700 1 -- 172.21.15.119:0/2575548286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d62038710 msgr2=0x555d62047420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.128+0000 7f3511b9a700 1 --2- 172.21.15.119:0/2575548286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d62038710 0x555d62047420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.128+0000 7f351239b700 1 --2- 172.21.15.119:0/2575548286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d62038710 0x555d62047420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.128+0000 7f351239b700 1 -- 172.21.15.119:0/2575548286 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3508009c10 con 0x555d62037d50 2023-12-07T10:31:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.128+0000 7f3511b9a700 1 --2- 172.21.15.119:0/2575548286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d62038710 0x555d62047420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:31:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.129+0000 7f351239b700 1 --2- 172.21.15.119:0/2575548286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d62037d50 0x555d62046ee0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3508012d20 tx=0x7f350800f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c11944bf3f7ef2c5 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.129+0000 7f34f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:31:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.129+0000 7f34f5ffb700 1 -- 172.21.15.119:0/2575548286 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508017070 con 0x555d62037d50 2023-12-07T10:31:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.129+0000 7f3527eaeb80 1 -- 172.21.15.119:0/2575548286 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d620226f0 con 0x555d62037d50 2023-12-07T10:31:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.129+0000 7f34f5ffb700 1 -- 172.21.15.119:0/2575548286 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f350800fc30 con 0x555d62037d50 2023-12-07T10:31:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.129+0000 7f34f5ffb700 1 -- 172.21.15.119:0/2575548286 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f350800fdf0 con 0x555d62037d50 2023-12-07T10:31:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.129+0000 7f3527eaeb80 10 client.27688 fetch_fsmap learned FSMap version 59 2023-12-07T10:31:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.129+0000 7f3527eaeb80 1 -- 172.21.15.119:0/2575548286 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x7f350c199e40 con 0x555d62037d50 2023-12-07T10:31:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.129+0000 7f34f5ffb700 1 -- 172.21.15.119:0/2575548286 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555d620226f0 con 0x555d62037d50 2023-12-07T10:31:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.129+0000 7f34f5ffb700 1 -- 172.21.15.119:0/2575548286 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 59) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3508003b70 con 0x555d62037d50 2023-12-07T10:31:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.130+0000 7f3527eaeb80 10 client.27688 fetch_fsmap finished waiting for FSMap version 59 2023-12-07T10:31:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.130+0000 7f3527eaeb80 10 client.27688 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:31:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.130+0000 7f3527eaeb80 20 client.27688 populate_metadata read hostname 'smithi119' 2023-12-07T10:31:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.130+0000 7f3527eaeb80 1 --2- 172.21.15.119:0/2575548286 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555d62051450 0x555d62053870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.130+0000 7f3527eaeb80 4 client.27688 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:31:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.130+0000 7f3527eaeb80 1 -- 172.21.15.119:0/2575548286 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d61ffe740 con 0x555d62051450 2023-12-07T10:31:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.130+0000 7f3511b9a700 1 --2- 172.21.15.119:0/2575548286 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555d62051450 0x555d62053870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.131+0000 7f34f5ffb700 10 client.27688 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:31:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.131+0000 7f3511b9a700 1 --2- 172.21.15.119:0/2575548286 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555d62051450 0x555d62053870 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32066fef4124a058 server_cookie=a5028f7f822e6fce in_seq=0 out_seq=0 2023-12-07T10:31:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.131+0000 7f34f5ffb700 1 -- 172.21.15.119:0/2575548286 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555d61ffe740 con 0x555d62051450 2023-12-07T10:31:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.131+0000 7f34f5ffb700 10 client.27688 handle_command_reply: tid=0 2023-12-07T10:31:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.131+0000 7f3527eaeb80 1 client.27688 shutdown 2023-12-07T10:31:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.132+0000 7f3527eaeb80 1 -- 172.21.15.119:0/2575548286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d62037d50 msgr2=0x555d62046ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.132+0000 7f3527eaeb80 1 --2- 172.21.15.119:0/2575548286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d62037d50 0x555d62046ee0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3508012d20 tx=0x7f350800f6a0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.132+0000 7f3527eaeb80 1 -- 172.21.15.119:0/2575548286 shutdown_connections 2023-12-07T10:31:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.132+0000 7f3527eaeb80 1 --2- 172.21.15.119:0/2575548286 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555d62051450 0x555d62053870 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.132+0000 7f3527eaeb80 1 --2- 172.21.15.119:0/2575548286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d6203c1a0 0x555d620411a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.132+0000 7f3527eaeb80 1 --2- 172.21.15.119:0/2575548286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d62037d50 0x555d62046ee0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.132+0000 7f34f5ffb700 0 client.27688 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:31:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.132+0000 7f3527eaeb80 1 --2- 172.21.15.119:0/2575548286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d62038710 0x555d62047420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.132+0000 7f3527eaeb80 1 -- 172.21.15.119:0/2575548286 >> 172.21.15.119:0/2575548286 conn(0x555d61f5e300 msgr2=0x555d62024980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.132+0000 7f3527eaeb80 1 -- 172.21.15.119:0/2575548286 shutdown_connections 2023-12-07T10:31:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.132+0000 7f3527eaeb80 1 -- 172.21.15.119:0/2575548286 wait complete. 2023-12-07T10:31:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.132+0000 7f3527eaeb80 20 client.27688 trim_cache size 0 max 16384 2023-12-07T10:31:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.153+0000 7f3527eaeb80 1 Processor -- start 2023-12-07T10:31:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.153+0000 7f3527eaeb80 1 -- start start 2023-12-07T10:31:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.153+0000 7f3527eaeb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d62037d50 0x555d62038130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.153+0000 7f3527eaeb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d62038710 0x555d6203ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.153+0000 7f3527eaeb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d6203c1a0 0x555d6203b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.153+0000 7f3527eaeb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d61e028c0 con 0x555d6203c1a0 2023-12-07T10:31:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.153+0000 7f3527eaeb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d62058ae0 con 0x555d62038710 2023-12-07T10:31:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.153+0000 7f3527eaeb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d6203f1d0 con 0x555d62037d50 2023-12-07T10:31:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.154+0000 7f351239b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d62037d50 0x555d62038130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.154+0000 7f3511b9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d62038710 0x555d6203ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.154+0000 7f3512b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d6203c1a0 0x555d6203b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.154+0000 7f351239b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d62037d50 0x555d62038130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:38566/0 (socket says 172.21.15.119:38566) 2023-12-07T10:31:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.154+0000 7f351239b700 1 -- 172.21.15.119:0/4162389513 learned_addr learned my addr 172.21.15.119:0/4162389513 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:31:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.154+0000 7f3511b9a700 1 -- 172.21.15.119:0/4162389513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d62037d50 msgr2=0x555d62038130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.154+0000 7f3511b9a700 1 --2- 172.21.15.119:0/4162389513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d62037d50 0x555d62038130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.154+0000 7f3511b9a700 1 -- 172.21.15.119:0/4162389513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d6203c1a0 msgr2=0x555d6203b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.154+0000 7f3511b9a700 1 --2- 172.21.15.119:0/4162389513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d6203c1a0 0x555d6203b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.154+0000 7f3511b9a700 1 -- 172.21.15.119:0/4162389513 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d61dacd80 con 0x555d62038710 2023-12-07T10:31:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.154+0000 7f351239b700 1 --2- 172.21.15.119:0/4162389513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d62037d50 0x555d62038130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:31:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.154+0000 7f3511b9a700 1 --2- 172.21.15.119:0/4162389513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d62038710 0x555d6203ec10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x555d62038d00 tx=0x7f34fc000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8f15728956049d0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.155+0000 7f35013ff700 1 -- 172.21.15.119:0/4162389513 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc0a6040 con 0x555d62038710 2023-12-07T10:31:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.155+0000 7f35013ff700 1 -- 172.21.15.119:0/4162389513 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34fc0a43f0 con 0x555d62038710 2023-12-07T10:31:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.155+0000 7f35013ff700 1 -- 172.21.15.119:0/4162389513 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc0d3560 con 0x555d62038710 2023-12-07T10:31:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.155+0000 7f3527eaeb80 1 -- 172.21.15.119:0/4162389513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d62038710 msgr2=0x555d6203ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.155+0000 7f3527eaeb80 1 --2- 172.21.15.119:0/4162389513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d62038710 0x555d6203ec10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x555d62038d00 tx=0x7f34fc000bc0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.155+0000 7f3527eaeb80 1 -- 172.21.15.119:0/4162389513 shutdown_connections 2023-12-07T10:31:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.155+0000 7f3527eaeb80 1 --2- 172.21.15.119:0/4162389513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d6203c1a0 0x555d6203b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.155+0000 7f3527eaeb80 1 --2- 172.21.15.119:0/4162389513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d62037d50 0x555d62038130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.155+0000 7f3527eaeb80 1 --2- 172.21.15.119:0/4162389513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d62038710 0x555d6203ec10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.155+0000 7f3527eaeb80 1 -- 172.21.15.119:0/4162389513 >> 172.21.15.119:0/4162389513 conn(0x555d61f5e300 msgr2=0x555d6203fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.156+0000 7f3527eaeb80 1 -- 172.21.15.119:0/4162389513 shutdown_connections 2023-12-07T10:31:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.156+0000 7f3527eaeb80 1 -- 172.21.15.119:0/4162389513 wait complete. 2023-12-07T10:31:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.156+0000 7f3527eaeb80 1 -- start start 2023-12-07T10:31:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.156+0000 7f3527eaeb80 1 Processor -- start 2023-12-07T10:31:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.157+0000 7f3527eaeb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d62037d50 0x555d62049060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.157+0000 7f3527eaeb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d62038710 0x555d620495a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.157+0000 7f3527eaeb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d6203c1a0 0x555d62049ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.157+0000 7f3527eaeb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d620a6960 con 0x555d62038710 2023-12-07T10:31:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.157+0000 7f3527eaeb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d620a6aa0 con 0x555d62037d50 2023-12-07T10:31:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.157+0000 7f3527eaeb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d620a6be0 con 0x555d6203c1a0 2023-12-07T10:31:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.157+0000 7f3527eaeb80 10 client.0 _ll_register_callbacks cb 0x555d62041e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:31:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.157+0000 7f351239b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d62037d50 0x555d62049060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.157+0000 7f3512b9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d6203c1a0 0x555d62049ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.157+0000 7f3511b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d62038710 0x555d620495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.157+0000 7f3512b9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d6203c1a0 0x555d62049ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:38574/0 (socket says 172.21.15.119:38574) 2023-12-07T10:31:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.157+0000 7f351239b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d62037d50 0x555d62049060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:54218/0 (socket says 172.21.15.119:54218) 2023-12-07T10:31:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.157+0000 7f3512b9c700 1 -- 172.21.15.119:0/4043466600 learned_addr learned my addr 172.21.15.119:0/4043466600 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:31:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.157+0000 7f351239b700 1 -- 172.21.15.119:0/4043466600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d6203c1a0 msgr2=0x555d62049ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.157+0000 7f351239b700 1 --2- 172.21.15.119:0/4043466600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d6203c1a0 0x555d62049ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.157+0000 7f351239b700 1 -- 172.21.15.119:0/4043466600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d62038710 msgr2=0x555d620495a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.157+0000 7f351239b700 1 --2- 172.21.15.119:0/4043466600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d62038710 0x555d620495a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.157+0000 7f351239b700 1 -- 172.21.15.119:0/4043466600 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350c199e40 con 0x555d62037d50 2023-12-07T10:31:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.158+0000 7f351239b700 1 --2- 172.21.15.119:0/4043466600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d62037d50 0x555d62049060 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3508005d10 tx=0x7f3508005e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ebaf73b8e90931d server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.158+0000 7f34f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:31:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.158+0000 7f34f5ffb700 1 -- 172.21.15.119:0/4043466600 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350802d030 con 0x555d62037d50 2023-12-07T10:31:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.158+0000 7f3527eaeb80 1 -- 172.21.15.119:0/4043466600 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f34fc0a02a0 con 0x555d62037d50 2023-12-07T10:31:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.158+0000 7f34f5ffb700 1 -- 172.21.15.119:0/4043466600 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f350802e070 con 0x555d62037d50 2023-12-07T10:31:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.158+0000 7f34f5ffb700 1 -- 172.21.15.119:0/4043466600 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3508017040 con 0x555d62037d50 2023-12-07T10:31:44.162 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.158+0000 7f3527eaeb80 10 client.27716 fetch_fsmap learned FSMap version 59 2023-12-07T10:31:44.162 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.158+0000 7f3527eaeb80 1 -- 172.21.15.119:0/4043466600 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x555d61e005d0 con 0x555d62037d50 2023-12-07T10:31:44.162 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.158+0000 7f34f5ffb700 1 -- 172.21.15.119:0/4043466600 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34fc0a02a0 con 0x555d62037d50 2023-12-07T10:31:44.162 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.159+0000 7f34f5ffb700 1 -- 172.21.15.119:0/4043466600 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 59) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3508005380 con 0x555d62037d50 2023-12-07T10:31:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.159+0000 7f3527eaeb80 10 client.27716 fetch_fsmap finished waiting for FSMap version 59 2023-12-07T10:31:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.159+0000 7f3527eaeb80 10 client.27716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:31:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.159+0000 7f3527eaeb80 20 client.27716 populate_metadata read hostname 'smithi119' 2023-12-07T10:31:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.159+0000 7f3527eaeb80 1 --2- 172.21.15.119:0/4043466600 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555d620a8370 0x555d620aa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.159+0000 7f3527eaeb80 4 client.27716 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:31:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.159+0000 7f3527eaeb80 1 -- 172.21.15.119:0/4043466600 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555d61f49f70 con 0x555d620a8370 2023-12-07T10:31:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.159+0000 7f3511b9a700 1 --2- 172.21.15.119:0/4043466600 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555d620a8370 0x555d620aa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.159+0000 7f3511b9a700 1 --2- 172.21.15.119:0/4043466600 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555d620a8370 0x555d620aa760 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31d96862294308c2 server_cookie=fb473490127eb582 in_seq=0 out_seq=0 2023-12-07T10:31:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.159+0000 7f34f5ffb700 10 client.27716 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:31:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.234+0000 7f34f5ffb700 1 -- 172.21.15.119:0/4043466600 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555d61f49f70 con 0x555d620a8370 2023-12-07T10:31:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.234+0000 7f34f5ffb700 10 client.27716 handle_command_reply: tid=0 2023-12-07T10:31:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.234+0000 7f3527eaeb80 1 client.27716 shutdown 2023-12-07T10:31:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.234+0000 7f3527eaeb80 1 -- 172.21.15.119:0/4043466600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d62037d50 msgr2=0x555d62049060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.234+0000 7f3527eaeb80 1 --2- 172.21.15.119:0/4043466600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d62037d50 0x555d62049060 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3508005d10 tx=0x7f3508005e10 comp rx=0 tx=0).stop 2023-12-07T10:31:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.234+0000 7f3527eaeb80 1 -- 172.21.15.119:0/4043466600 shutdown_connections 2023-12-07T10:31:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.234+0000 7f3527eaeb80 1 --2- 172.21.15.119:0/4043466600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d6203c1a0 0x555d62049ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.234+0000 7f3527eaeb80 1 --2- 172.21.15.119:0/4043466600 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555d620a8370 0x555d620aa760 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.234+0000 7f3527eaeb80 1 --2- 172.21.15.119:0/4043466600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555d62037d50 0x555d62049060 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.234+0000 7f3527eaeb80 1 --2- 172.21.15.119:0/4043466600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d62038710 0x555d620495a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.234+0000 7f34f5ffb700 0 client.27716 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:31:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.234+0000 7f3527eaeb80 1 -- 172.21.15.119:0/4043466600 >> 172.21.15.119:0/4043466600 conn(0x555d61f5e300 msgr2=0x555d6203b290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.234+0000 7f3527eaeb80 1 -- 172.21.15.119:0/4043466600 shutdown_connections 2023-12-07T10:31:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.235+0000 7f3527eaeb80 1 -- 172.21.15.119:0/4043466600 wait complete. 2023-12-07T10:31:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.235+0000 7f3527eaeb80 20 client.27716 trim_cache size 0 max 16384 2023-12-07T10:31:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:31:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:31:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:31:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:31:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.235+0000 7f35145ff700 1 -- 172.21.15.119:0/1692015216 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f34f80a1c80 msgr2=0x7f34f80a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.235+0000 7f35145ff700 1 --2- 172.21.15.119:0/1692015216 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f34f80a1c80 0x7f34f80a4140 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f350400f930 tx=0x7f350400fcf0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.235+0000 7f35145ff700 1 -- 172.21.15.119:0/1692015216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f350c101ee0 msgr2=0x7f350c195d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.235+0000 7f35145ff700 1 --2- 172.21.15.119:0/1692015216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f350c101ee0 0x7f350c195d10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f34fc01b040 tx=0x7f34fc00bb50 comp rx=0 tx=0).stop 2023-12-07T10:31:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.236+0000 7f35145ff700 1 -- 172.21.15.119:0/1692015216 shutdown_connections 2023-12-07T10:31:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.236+0000 7f35145ff700 1 --2- 172.21.15.119:0/1692015216 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f34f80a1c80 0x7f34f80a4140 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.236+0000 7f35145ff700 1 --2- 172.21.15.119:0/1692015216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f350c1075e0 0x7f350c199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.236+0000 7f35145ff700 1 --2- 172.21.15.119:0/1692015216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f350c101520 0x7f350c1957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.236+0000 7f35145ff700 1 --2- 172.21.15.119:0/1692015216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f350c101ee0 0x7f350c195d10 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.236+0000 7f35145ff700 1 -- 172.21.15.119:0/1692015216 >> 172.21.15.119:0/1692015216 conn(0x7f350c0fd010 msgr2=0x7f350c0fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.236+0000 7f35145ff700 1 -- 172.21.15.119:0/1692015216 shutdown_connections 2023-12-07T10:31:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:44.236+0000 7f35145ff700 1 -- 172.21.15.119:0/1692015216 wait complete. 2023-12-07T10:31:44.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:44 smithi119 ceph-mon[97908]: pgmap v352: 129 pgs: 129 active+clean; 624 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 228 op/s 2023-12-07T10:31:44.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:44 smithi159 ceph-mon[134726]: pgmap v352: 129 pgs: 129 active+clean; 624 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 228 op/s 2023-12-07T10:31:44.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:44 smithi153 ceph-mon[133625]: pgmap v352: 129 pgs: 129 active+clean; 624 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 228 op/s 2023-12-07T10:31:44.576 INFO:tasks.workunit.client.0.smithi119.stdout: 1 824406 7.22 MB/sec execute 477 sec latency 61.382 ms 2023-12-07T10:31:45.576 INFO:tasks.workunit.client.0.smithi119.stdout: 1 826131 7.22 MB/sec execute 478 sec latency 30.061 ms 2023-12-07T10:31:46.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:46 smithi119 ceph-mon[97908]: pgmap v353: 129 pgs: 129 active+clean; 624 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 189 op/s 2023-12-07T10:31:46.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:46 smithi159 ceph-mon[134726]: pgmap v353: 129 pgs: 129 active+clean; 624 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 189 op/s 2023-12-07T10:31:46.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:46 smithi153 ceph-mon[133625]: pgmap v353: 129 pgs: 129 active+clean; 624 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 189 op/s 2023-12-07T10:31:46.576 INFO:tasks.workunit.client.0.smithi119.stdout: 1 827622 7.23 MB/sec execute 479 sec latency 16.084 ms 2023-12-07T10:31:47.577 INFO:tasks.workunit.client.0.smithi119.stdout: 1 829089 7.23 MB/sec execute 480 sec latency 17.842 ms 2023-12-07T10:31:48.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:48 smithi119 ceph-mon[97908]: pgmap v354: 129 pgs: 129 active+clean; 629 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 314 op/s 2023-12-07T10:31:48.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:48 smithi159 ceph-mon[134726]: pgmap v354: 129 pgs: 129 active+clean; 629 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 314 op/s 2023-12-07T10:31:48.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:48 smithi153 ceph-mon[133625]: pgmap v354: 129 pgs: 129 active+clean; 629 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 314 op/s 2023-12-07T10:31:48.576 INFO:tasks.workunit.client.0.smithi119.stdout: 1 831001 7.24 MB/sec execute 481 sec latency 17.400 ms 2023-12-07T10:31:49.577 INFO:tasks.workunit.client.0.smithi119.stdout: 1 831768 7.23 MB/sec execute 482 sec latency 61.377 ms 2023-12-07T10:31:50.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:50 smithi119 ceph-mon[97908]: pgmap v355: 129 pgs: 129 active+clean; 629 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 225 op/s 2023-12-07T10:31:50.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:50 smithi159 ceph-mon[134726]: pgmap v355: 129 pgs: 129 active+clean; 629 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 225 op/s 2023-12-07T10:31:50.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:50 smithi153 ceph-mon[133625]: pgmap v355: 129 pgs: 129 active+clean; 629 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 225 op/s 2023-12-07T10:31:50.576 INFO:tasks.workunit.client.0.smithi119.stdout: 1 833669 7.24 MB/sec execute 483 sec latency 21.758 ms 2023-12-07T10:31:51.577 INFO:tasks.workunit.client.0.smithi119.stdout: 1 834845 7.24 MB/sec execute 484 sec latency 15.094 ms 2023-12-07T10:31:52.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:51 smithi119 ceph-mon[97908]: pgmap v356: 129 pgs: 129 active+clean; 637 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 319 op/s 2023-12-07T10:31:52.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:51 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:31:52.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:51 smithi159 ceph-mon[134726]: pgmap v356: 129 pgs: 129 active+clean; 637 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 319 op/s 2023-12-07T10:31:52.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:51 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:31:52.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:51 smithi153 ceph-mon[133625]: pgmap v356: 129 pgs: 129 active+clean; 637 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 319 op/s 2023-12-07T10:31:52.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:51 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:31:52.577 INFO:tasks.workunit.client.0.smithi119.stdout: 1 836144 7.23 MB/sec execute 485 sec latency 21.310 ms 2023-12-07T10:31:53.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:52 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:31:53.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:52 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:31:53.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:52 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:31:53.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:52 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:31:53.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:52 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:31:53.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:52 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:31:53.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:52 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:31:53.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:52 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:31:53.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:52 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:31:53.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:52 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:31:53.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:52 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:31:53.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:52 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:31:53.577 INFO:tasks.workunit.client.0.smithi119.stdout: 1 838035 7.24 MB/sec execute 486 sec latency 16.519 ms 2023-12-07T10:31:54.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:53 smithi119 ceph-mon[97908]: pgmap v357: 129 pgs: 129 active+clean; 637 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 263 op/s 2023-12-07T10:31:54.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:53 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:31:54.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:53 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:31:54.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:53 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:31:54.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:53 smithi159 ceph-mon[134726]: pgmap v357: 129 pgs: 129 active+clean; 637 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 263 op/s 2023-12-07T10:31:54.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:53 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:31:54.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:53 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:31:54.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:53 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:31:54.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:53 smithi153 ceph-mon[133625]: pgmap v357: 129 pgs: 129 active+clean; 637 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 263 op/s 2023-12-07T10:31:54.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:53 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:31:54.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:53 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:31:54.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:53 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:31:54.577 INFO:tasks.workunit.client.0.smithi119.stdout: 1 838774 7.23 MB/sec execute 487 sec latency 55.000 ms 2023-12-07T10:31:55.577 INFO:tasks.workunit.client.0.smithi119.stdout: 1 840515 7.23 MB/sec execute 488 sec latency 20.593 ms 2023-12-07T10:31:56.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:56 smithi119 ceph-mon[97908]: pgmap v358: 129 pgs: 129 active+clean; 637 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 230 op/s 2023-12-07T10:31:56.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:56 smithi159 ceph-mon[134726]: pgmap v358: 129 pgs: 129 active+clean; 637 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 230 op/s 2023-12-07T10:31:56.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:56 smithi153 ceph-mon[133625]: pgmap v358: 129 pgs: 129 active+clean; 637 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 230 op/s 2023-12-07T10:31:56.577 INFO:tasks.workunit.client.0.smithi119.stdout: 1 841920 7.24 MB/sec execute 489 sec latency 15.640 ms 2023-12-07T10:31:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.403+0000 7f80eefff700 1 -- 172.21.15.119:0/594752478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80e8074300 msgr2=0x7f80e810d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.403+0000 7f80eefff700 1 --2- 172.21.15.119:0/594752478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80e8074300 0x7f80e810d860 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f80d4009990 tx=0x7f80d4009ca0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.404+0000 7f80eefff700 1 -- 172.21.15.119:0/594752478 shutdown_connections 2023-12-07T10:31:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.404+0000 7f80eefff700 1 --2- 172.21.15.119:0/594752478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80e810dda0 0x7f80e81101d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.404+0000 7f80eefff700 1 --2- 172.21.15.119:0/594752478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80e80739e0 0x7f80e8073dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.404+0000 7f80eefff700 1 --2- 172.21.15.119:0/594752478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80e8074300 0x7f80e810d860 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.404+0000 7f80eefff700 1 -- 172.21.15.119:0/594752478 >> 172.21.15.119:0/594752478 conn(0x7f80e80fd010 msgr2=0x7f80e80ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.404+0000 7f80eefff700 1 -- 172.21.15.119:0/594752478 shutdown_connections 2023-12-07T10:31:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.404+0000 7f80eefff700 1 -- 172.21.15.119:0/594752478 wait complete. 2023-12-07T10:31:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.405+0000 7f80eefff700 1 Processor -- start 2023-12-07T10:31:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.405+0000 7f80eefff700 1 -- start start 2023-12-07T10:31:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.405+0000 7f80eefff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80e80739e0 0x7f80e819f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.405+0000 7f80eefff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80e8074300 0x7f80e819fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.405+0000 7f80eefff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80e810dda0 0x7f80e8199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.405+0000 7f80eefff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e819a1d0 con 0x7f80e810dda0 2023-12-07T10:31:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.405+0000 7f80eefff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e819a340 con 0x7f80e80739e0 2023-12-07T10:31:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.405+0000 7f80eefff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e819a500 con 0x7f80e8074300 2023-12-07T10:31:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.406+0000 7f80ecd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80e80739e0 0x7f80e819f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.406+0000 7f80ecd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80e80739e0 0x7f80e819f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:46924/0 (socket says 172.21.15.119:46924) 2023-12-07T10:31:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.406+0000 7f80ecd9b700 1 -- 172.21.15.119:0/2024250787 learned_addr learned my addr 172.21.15.119:0/2024250787 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:31:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.406+0000 7f80ed59c700 1 --2- 172.21.15.119:0/2024250787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80e810dda0 0x7f80e8199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.406+0000 7f80e7fff700 1 --2- 172.21.15.119:0/2024250787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80e8074300 0x7f80e819fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.406+0000 7f80ecd9b700 1 -- 172.21.15.119:0/2024250787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80e8074300 msgr2=0x7f80e819fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.406+0000 7f80ecd9b700 1 --2- 172.21.15.119:0/2024250787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80e8074300 0x7f80e819fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.406+0000 7f80ecd9b700 1 -- 172.21.15.119:0/2024250787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80e810dda0 msgr2=0x7f80e8199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.406+0000 7f80ecd9b700 1 --2- 172.21.15.119:0/2024250787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80e810dda0 0x7f80e8199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.406+0000 7f80ecd9b700 1 -- 172.21.15.119:0/2024250787 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d40095f0 con 0x7f80e80739e0 2023-12-07T10:31:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.406+0000 7f80ecd9b700 1 --2- 172.21.15.119:0/2024250787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80e80739e0 0x7f80e819f960 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f80dc00e9a0 tx=0x7f80dc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.406+0000 7f80e5ffb700 1 -- 172.21.15.119:0/2024250787 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc009790 con 0x7f80e80739e0 2023-12-07T10:31:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.407+0000 7f80e5ffb700 1 -- 172.21.15.119:0/2024250787 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80dc010430 con 0x7f80e80739e0 2023-12-07T10:31:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.407+0000 7f80eefff700 1 -- 172.21.15.119:0/2024250787 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e819a7e0 con 0x7f80e80739e0 2023-12-07T10:31:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.407+0000 7f80e5ffb700 1 -- 172.21.15.119:0/2024250787 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc018680 con 0x7f80e80739e0 2023-12-07T10:31:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.407+0000 7f80eefff700 1 -- 172.21.15.119:0/2024250787 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e81a68e0 con 0x7f80e80739e0 2023-12-07T10:31:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.410+0000 7f80e5ffb700 1 -- 172.21.15.119:0/2024250787 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f80dc0105a0 con 0x7f80e80739e0 2023-12-07T10:31:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.412+0000 7f810285fb80 1 Processor -- start 2023-12-07T10:31:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.412+0000 7f80e5ffb700 1 --2- 172.21.15.119:0/2024250787 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f80d80a1c80 0x7f80d80a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.412+0000 7f810285fb80 1 -- start start 2023-12-07T10:31:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.412+0000 7f80e5ffb700 1 -- 172.21.15.119:0/2024250787 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f80dc014070 con 0x7f80e80739e0 2023-12-07T10:31:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.412+0000 7f80ed59c700 1 --2- 172.21.15.119:0/2024250787 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f80d80a1c80 0x7f80d80a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.412+0000 7f810285fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d32a82d1a0 0x55d32a82d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.413+0000 7f810285fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d32a828d50 0x55d32a8291d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.413+0000 7f810285fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32a829710 0x55d32a8350b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.413+0000 7f810285fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32a794760 con 0x55d32a828d50 2023-12-07T10:31:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.413+0000 7f810285fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32a77b740 con 0x55d32a82d1a0 2023-12-07T10:31:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.413+0000 7f810285fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32a779060 con 0x55d32a829710 2023-12-07T10:31:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.413+0000 7f80e7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d32a828d50 0x55d32a8291d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.413+0000 7f80e7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d32a828d50 0x55d32a8291d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45078/0 (socket says 172.21.15.119:45078) 2023-12-07T10:31:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.413+0000 7f80e7fff700 1 -- 172.21.15.119:0/2720853547 learned_addr learned my addr 172.21.15.119:0/2720853547 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:31:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.413+0000 7f80ed59c700 1 --2- 172.21.15.119:0/2720853547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32a829710 0x55d32a8350b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.413+0000 7f80ecd9b700 1 --2- 172.21.15.119:0/2720853547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d32a82d1a0 0x55d32a82d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.413+0000 7f80ed59c700 1 --2- 172.21.15.119:0/2024250787 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f80d80a1c80 0x7f80d80a4140 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f80e81a05f0 tx=0x7f80e0009100 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.413+0000 7f80e7fff700 1 -- 172.21.15.119:0/2720853547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32a829710 msgr2=0x55d32a8350b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.413+0000 7f80e7fff700 1 --2- 172.21.15.119:0/2720853547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32a829710 0x55d32a8350b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.413+0000 7f80e7fff700 1 -- 172.21.15.119:0/2720853547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d32a82d1a0 msgr2=0x55d32a82d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.413+0000 7f80e7fff700 1 --2- 172.21.15.119:0/2720853547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d32a82d1a0 0x55d32a82d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.413+0000 7f80e7fff700 1 -- 172.21.15.119:0/2720853547 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d32a5f15d0 con 0x55d32a828d50 2023-12-07T10:31:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.413+0000 7f80ecd9b700 1 --2- 172.21.15.119:0/2720853547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d32a82d1a0 0x55d32a82d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:31:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.414+0000 7f80e7fff700 1 --2- 172.21.15.119:0/2720853547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d32a828d50 0x55d32a8291d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f80d4009990 tx=0x7f80d400b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f56a96c67d6f51 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.414+0000 7f80d2ffd700 1 -- 172.21.15.119:0/2720853547 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d401d070 con 0x55d32a828d50 2023-12-07T10:31:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.414+0000 7f80d2ffd700 1 -- 172.21.15.119:0/2720853547 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80d401e040 con 0x55d32a828d50 2023-12-07T10:31:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.414+0000 7f80d2ffd700 1 -- 172.21.15.119:0/2720853547 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d4004820 con 0x55d32a828d50 2023-12-07T10:31:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.415+0000 7f810285fb80 1 -- 172.21.15.119:0/2720853547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d32a828d50 msgr2=0x55d32a8291d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.415+0000 7f810285fb80 1 --2- 172.21.15.119:0/2720853547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d32a828d50 0x55d32a8291d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f80d4009990 tx=0x7f80d400b610 comp rx=0 tx=0).stop 2023-12-07T10:31:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.415+0000 7f810285fb80 1 -- 172.21.15.119:0/2720853547 shutdown_connections 2023-12-07T10:31:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.415+0000 7f810285fb80 1 --2- 172.21.15.119:0/2720853547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32a829710 0x55d32a8350b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.415+0000 7f810285fb80 1 --2- 172.21.15.119:0/2720853547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d32a82d1a0 0x55d32a82d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.415+0000 7f810285fb80 1 --2- 172.21.15.119:0/2720853547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d32a828d50 0x55d32a8291d0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.415+0000 7f810285fb80 1 -- 172.21.15.119:0/2720853547 >> 172.21.15.119:0/2720853547 conn(0x55d32a74f300 msgr2=0x55d32a81ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.415+0000 7f810285fb80 1 -- 172.21.15.119:0/2720853547 shutdown_connections 2023-12-07T10:31:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.415+0000 7f810285fb80 1 -- 172.21.15.119:0/2720853547 wait complete. 2023-12-07T10:31:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.416+0000 7f810285fb80 1 -- start start 2023-12-07T10:31:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.416+0000 7f810285fb80 1 Processor -- start 2023-12-07T10:31:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.416+0000 7f810285fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d32a828d50 0x55d32a834ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.416+0000 7f810285fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32a829710 0x55d32a8321e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.416+0000 7f810285fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d32a82d1a0 0x55d32a832760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.416+0000 7f810285fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32a774950 con 0x55d32a82d1a0 2023-12-07T10:31:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.416+0000 7f810285fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32a72e500 con 0x55d32a828d50 2023-12-07T10:31:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.416+0000 7f810285fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32a72e1a0 con 0x55d32a829710 2023-12-07T10:31:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.417+0000 7f810285fb80 10 client.0 _ll_register_callbacks cb 0x55d32a7158e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:31:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.417+0000 7f80ed59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d32a82d1a0 0x55d32a832760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.417+0000 7f80e7fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32a829710 0x55d32a8321e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.417+0000 7f80e7fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32a829710 0x55d32a8321e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:35952/0 (socket says 172.21.15.119:35952) 2023-12-07T10:31:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.417+0000 7f80e7fff700 1 -- 172.21.15.119:0/3157618640 learned_addr learned my addr 172.21.15.119:0/3157618640 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:31:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.417+0000 7f80ecd9b700 1 --2- 172.21.15.119:0/3157618640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d32a828d50 0x55d32a834ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.417+0000 7f80e7fff700 1 -- 172.21.15.119:0/3157618640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d32a828d50 msgr2=0x55d32a834ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.417+0000 7f80e7fff700 1 --2- 172.21.15.119:0/3157618640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d32a828d50 0x55d32a834ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.417+0000 7f80e7fff700 1 -- 172.21.15.119:0/3157618640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d32a82d1a0 msgr2=0x55d32a832760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.417+0000 7f80e7fff700 1 --2- 172.21.15.119:0/3157618640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d32a82d1a0 0x55d32a832760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.417+0000 7f80e7fff700 1 -- 172.21.15.119:0/3157618640 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e81a68e0 con 0x55d32a829710 2023-12-07T10:31:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.418+0000 7f80e7fff700 1 --2- 172.21.15.119:0/3157618640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32a829710 0x55d32a8321e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f80d4005920 tx=0x7f80d400b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=721aa24e1cc291d5 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.418+0000 7f80c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:31:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.418+0000 7f80c7fff700 1 -- 172.21.15.119:0/3157618640 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d401d030 con 0x55d32a829710 2023-12-07T10:31:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.418+0000 7f810285fb80 1 -- 172.21.15.119:0/3157618640 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d32a8136f0 con 0x55d32a829710 2023-12-07T10:31:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.418+0000 7f80c7fff700 1 -- 172.21.15.119:0/3157618640 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f80d4004700 con 0x55d32a829710 2023-12-07T10:31:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.418+0000 7f80c7fff700 1 -- 172.21.15.119:0/3157618640 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80d40048c0 con 0x55d32a829710 2023-12-07T10:31:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.418+0000 7f810285fb80 10 client.27706 fetch_fsmap learned FSMap version 59 2023-12-07T10:31:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.418+0000 7f810285fb80 1 -- 172.21.15.119:0/3157618640 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x7f80e8110ee0 con 0x55d32a829710 2023-12-07T10:31:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.418+0000 7f80c7fff700 1 -- 172.21.15.119:0/3157618640 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d32a8136f0 con 0x55d32a829710 2023-12-07T10:31:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.418+0000 7f80c7fff700 1 -- 172.21.15.119:0/3157618640 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 59) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f80d400f9b0 con 0x55d32a829710 2023-12-07T10:31:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.419+0000 7f810285fb80 10 client.27706 fetch_fsmap finished waiting for FSMap version 59 2023-12-07T10:31:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.419+0000 7f810285fb80 10 client.27706 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:31:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.419+0000 7f810285fb80 20 client.27706 populate_metadata read hostname 'smithi119' 2023-12-07T10:31:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.419+0000 7f810285fb80 1 --2- 172.21.15.119:0/3157618640 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d32a842190 0x55d32a8445b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.419+0000 7f810285fb80 4 client.27706 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:31:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.419+0000 7f810285fb80 1 -- 172.21.15.119:0/3157618640 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d32a7ef740 con 0x55d32a842190 2023-12-07T10:31:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.419+0000 7f80ecd9b700 1 --2- 172.21.15.119:0/3157618640 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d32a842190 0x55d32a8445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.419+0000 7f80ecd9b700 1 --2- 172.21.15.119:0/3157618640 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d32a842190 0x55d32a8445b0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=926b8393576ae4df server_cookie=bb6c7f44327fa4be in_seq=0 out_seq=0 2023-12-07T10:31:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.419+0000 7f80c7fff700 10 client.27706 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:31:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.420+0000 7f80c7fff700 1 -- 172.21.15.119:0/3157618640 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d32a7ef740 con 0x55d32a842190 2023-12-07T10:31:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.420+0000 7f80c7fff700 10 client.27706 handle_command_reply: tid=0 2023-12-07T10:31:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.420+0000 7f810285fb80 1 client.27706 shutdown 2023-12-07T10:31:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.420+0000 7f810285fb80 1 -- 172.21.15.119:0/3157618640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32a829710 msgr2=0x55d32a8321e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.420+0000 7f810285fb80 1 --2- 172.21.15.119:0/3157618640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32a829710 0x55d32a8321e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f80d4005920 tx=0x7f80d400b9e0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.420+0000 7f810285fb80 1 -- 172.21.15.119:0/3157618640 shutdown_connections 2023-12-07T10:31:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.420+0000 7f810285fb80 1 --2- 172.21.15.119:0/3157618640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d32a82d1a0 0x55d32a832760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.420+0000 7f810285fb80 1 --2- 172.21.15.119:0/3157618640 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d32a842190 0x55d32a8445b0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.420+0000 7f810285fb80 1 --2- 172.21.15.119:0/3157618640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d32a828d50 0x55d32a834ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.420+0000 7f80c7fff700 0 client.27706 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:31:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.420+0000 7f810285fb80 1 --2- 172.21.15.119:0/3157618640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32a829710 0x55d32a8321e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.420+0000 7f810285fb80 1 -- 172.21.15.119:0/3157618640 >> 172.21.15.119:0/3157618640 conn(0x55d32a74f300 msgr2=0x55d32a815980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.421+0000 7f810285fb80 1 -- 172.21.15.119:0/3157618640 shutdown_connections 2023-12-07T10:31:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.421+0000 7f810285fb80 1 -- 172.21.15.119:0/3157618640 wait complete. 2023-12-07T10:31:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.421+0000 7f810285fb80 20 client.27706 trim_cache size 0 max 16384 2023-12-07T10:31:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.437+0000 7f810285fb80 1 Processor -- start 2023-12-07T10:31:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.437+0000 7f810285fb80 1 -- start start 2023-12-07T10:31:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.437+0000 7f810285fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32a828d50 0x55d32a832750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.437+0000 7f810285fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d32a829710 0x55d32a83e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.437+0000 7f810285fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d32a82d1a0 0x55d32a82ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.437+0000 7f810285fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32a5f38c0 con 0x55d32a829710 2023-12-07T10:31:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.437+0000 7f810285fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32a8307c0 con 0x55d32a82d1a0 2023-12-07T10:31:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.437+0000 7f810285fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32a82e090 con 0x55d32a828d50 2023-12-07T10:31:57.438 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.438+0000 7f80ecd9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32a828d50 0x55d32a832750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.438+0000 7f80ecd9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32a828d50 0x55d32a832750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:35968/0 (socket says 172.21.15.119:35968) 2023-12-07T10:31:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.438+0000 7f80ecd9b700 1 -- 172.21.15.119:0/4143298658 learned_addr learned my addr 172.21.15.119:0/4143298658 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:31:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.438+0000 7f80e7fff700 1 --2- 172.21.15.119:0/4143298658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d32a829710 0x55d32a83e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.438+0000 7f80ed59c700 1 --2- 172.21.15.119:0/4143298658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d32a82d1a0 0x55d32a82ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.438+0000 7f80ecd9b700 1 -- 172.21.15.119:0/4143298658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d32a82d1a0 msgr2=0x55d32a82ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.438+0000 7f80ecd9b700 1 --2- 172.21.15.119:0/4143298658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d32a82d1a0 0x55d32a82ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.438+0000 7f80ecd9b700 1 -- 172.21.15.119:0/4143298658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d32a829710 msgr2=0x55d32a83e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.438+0000 7f80ecd9b700 1 --2- 172.21.15.119:0/4143298658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d32a829710 0x55d32a83e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.438+0000 7f80ecd9b700 1 -- 172.21.15.119:0/4143298658 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d32a59dd80 con 0x55d32a828d50 2023-12-07T10:31:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.438+0000 7f80ed59c700 1 --2- 172.21.15.119:0/4143298658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d32a82d1a0 0x55d32a82ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:31:57.440 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.438+0000 7f80ecd9b700 1 --2- 172.21.15.119:0/4143298658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32a828d50 0x55d32a832750 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f80dc0cb310 tx=0x7f80dc0ba040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61c61b00a316bc2e server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:57.440 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.438+0000 7f80d37fe700 1 -- 172.21.15.119:0/4143298658 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc0a2c90 con 0x55d32a828d50 2023-12-07T10:31:57.440 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.438+0000 7f80d37fe700 1 -- 172.21.15.119:0/4143298658 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80dc098a40 con 0x55d32a828d50 2023-12-07T10:31:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.438+0000 7f80d37fe700 1 -- 172.21.15.119:0/4143298658 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc09ab60 con 0x55d32a828d50 2023-12-07T10:31:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.439+0000 7f810285fb80 1 -- 172.21.15.119:0/4143298658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32a828d50 msgr2=0x55d32a832750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.439+0000 7f810285fb80 1 --2- 172.21.15.119:0/4143298658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32a828d50 0x55d32a832750 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f80dc0cb310 tx=0x7f80dc0ba040 comp rx=0 tx=0).stop 2023-12-07T10:31:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.439+0000 7f810285fb80 1 -- 172.21.15.119:0/4143298658 shutdown_connections 2023-12-07T10:31:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.439+0000 7f810285fb80 1 --2- 172.21.15.119:0/4143298658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d32a82d1a0 0x55d32a82ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.439+0000 7f810285fb80 1 --2- 172.21.15.119:0/4143298658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32a828d50 0x55d32a832750 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.439+0000 7f810285fb80 1 --2- 172.21.15.119:0/4143298658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d32a829710 0x55d32a83e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.439+0000 7f810285fb80 1 -- 172.21.15.119:0/4143298658 >> 172.21.15.119:0/4143298658 conn(0x55d32a74f300 msgr2=0x55d32a81c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.439+0000 7f810285fb80 1 -- 172.21.15.119:0/4143298658 shutdown_connections 2023-12-07T10:31:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.439+0000 7f810285fb80 1 -- 172.21.15.119:0/4143298658 wait complete. 2023-12-07T10:31:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.440+0000 7f810285fb80 1 -- start start 2023-12-07T10:31:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.440+0000 7f810285fb80 1 Processor -- start 2023-12-07T10:31:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.441+0000 7f810285fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d32a828d50 0x55d32a8354d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.441+0000 7f810285fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d32a829710 0x55d32a835a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.441+0000 7f810285fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32a82d1a0 0x55d32a835f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.441+0000 7f810285fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32a897680 con 0x55d32a828d50 2023-12-07T10:31:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.441+0000 7f810285fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32a8977c0 con 0x55d32a829710 2023-12-07T10:31:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.441+0000 7f810285fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d32a897950 con 0x55d32a82d1a0 2023-12-07T10:31:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.441+0000 7f810285fb80 10 client.0 _ll_register_callbacks cb 0x55d32a7158e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:31:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.441+0000 7f80e7fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d32a829710 0x55d32a835a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.441+0000 7f80ed59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32a82d1a0 0x55d32a835f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.441+0000 7f80e7fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d32a829710 0x55d32a835a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:46960/0 (socket says 172.21.15.119:46960) 2023-12-07T10:31:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.441+0000 7f80e7fff700 1 -- 172.21.15.119:0/2349903206 learned_addr learned my addr 172.21.15.119:0/2349903206 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:31:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.441+0000 7f80ecd9b700 1 --2- 172.21.15.119:0/2349903206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d32a828d50 0x55d32a8354d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.442+0000 7f80ed59c700 1 -- 172.21.15.119:0/2349903206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d32a829710 msgr2=0x55d32a835a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.442+0000 7f80ed59c700 1 --2- 172.21.15.119:0/2349903206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d32a829710 0x55d32a835a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.442+0000 7f80ed59c700 1 -- 172.21.15.119:0/2349903206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d32a828d50 msgr2=0x55d32a8354d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:31:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.442+0000 7f80ed59c700 1 --2- 172.21.15.119:0/2349903206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d32a828d50 0x55d32a8354d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.442+0000 7f80ed59c700 1 -- 172.21.15.119:0/2349903206 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e819a7e0 con 0x55d32a82d1a0 2023-12-07T10:31:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.442+0000 7f80e7fff700 1 --2- 172.21.15.119:0/2349903206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d32a829710 0x55d32a835a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:31:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.442+0000 7f80ed59c700 1 --2- 172.21.15.119:0/2349903206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32a82d1a0 0x55d32a835f50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f80e0003c70 tx=0x7f80e0004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b6632c2d7862bc7 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:31:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.442+0000 7f80c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:31:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.442+0000 7f80c7fff700 1 -- 172.21.15.119:0/2349903206 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e0004db0 con 0x55d32a82d1a0 2023-12-07T10:31:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.442+0000 7f810285fb80 1 -- 172.21.15.119:0/2349903206 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80d401e040 con 0x55d32a82d1a0 2023-12-07T10:31:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.442+0000 7f80c7fff700 1 -- 172.21.15.119:0/2349903206 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f80e0023650 con 0x55d32a82d1a0 2023-12-07T10:31:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.443+0000 7f80c7fff700 1 -- 172.21.15.119:0/2349903206 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80e0004910 con 0x55d32a82d1a0 2023-12-07T10:31:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.443+0000 7f810285fb80 10 client.27712 fetch_fsmap learned FSMap version 59 2023-12-07T10:31:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.443+0000 7f810285fb80 1 -- 172.21.15.119:0/2349903206 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55d32a586c00 con 0x55d32a82d1a0 2023-12-07T10:31:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.443+0000 7f80c7fff700 1 -- 172.21.15.119:0/2349903206 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80d401e040 con 0x55d32a82d1a0 2023-12-07T10:31:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.443+0000 7f80c7fff700 1 -- 172.21.15.119:0/2349903206 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 59) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f80e0005460 con 0x55d32a82d1a0 2023-12-07T10:31:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.443+0000 7f810285fb80 10 client.27712 fetch_fsmap finished waiting for FSMap version 59 2023-12-07T10:31:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.443+0000 7f810285fb80 10 client.27712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:31:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.443+0000 7f810285fb80 20 client.27712 populate_metadata read hostname 'smithi119' 2023-12-07T10:31:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.443+0000 7f810285fb80 1 --2- 172.21.15.119:0/2349903206 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d32a899080 0x55d32a89b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:31:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.443+0000 7f810285fb80 4 client.27712 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:31:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.443+0000 7f810285fb80 1 -- 172.21.15.119:0/2349903206 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d32a73af70 con 0x55d32a899080 2023-12-07T10:31:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.443+0000 7f80ecd9b700 1 --2- 172.21.15.119:0/2349903206 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d32a899080 0x55d32a89b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:31:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.444+0000 7f80c7fff700 10 client.27712 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:31:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.444+0000 7f80ecd9b700 1 --2- 172.21.15.119:0/2349903206 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d32a899080 0x55d32a89b4a0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f9314e1843421c8 server_cookie=7eee6c1db240651e in_seq=0 out_seq=0 2023-12-07T10:31:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.499+0000 7f80c7fff700 1 -- 172.21.15.119:0/2349903206 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d32a73af70 con 0x55d32a899080 2023-12-07T10:31:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.499+0000 7f80c7fff700 10 client.27712 handle_command_reply: tid=0 2023-12-07T10:31:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.499+0000 7f810285fb80 1 client.27712 shutdown 2023-12-07T10:31:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.499+0000 7f810285fb80 1 -- 172.21.15.119:0/2349903206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32a82d1a0 msgr2=0x55d32a835f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.499+0000 7f810285fb80 1 --2- 172.21.15.119:0/2349903206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32a82d1a0 0x55d32a835f50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f80e0003c70 tx=0x7f80e0004b40 comp rx=0 tx=0).stop 2023-12-07T10:31:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.500+0000 7f810285fb80 1 -- 172.21.15.119:0/2349903206 shutdown_connections 2023-12-07T10:31:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.500+0000 7f810285fb80 1 --2- 172.21.15.119:0/2349903206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32a82d1a0 0x55d32a835f50 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.500+0000 7f810285fb80 1 --2- 172.21.15.119:0/2349903206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d32a828d50 0x55d32a8354d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.500+0000 7f810285fb80 1 --2- 172.21.15.119:0/2349903206 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55d32a899080 0x55d32a89b4a0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.500+0000 7f810285fb80 1 --2- 172.21.15.119:0/2349903206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d32a829710 0x55d32a835a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.500+0000 7f810285fb80 1 -- 172.21.15.119:0/2349903206 >> 172.21.15.119:0/2349903206 conn(0x55d32a74f300 msgr2=0x55d32a82c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.500+0000 7f80c7fff700 0 client.27712 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:31:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.500+0000 7f810285fb80 1 -- 172.21.15.119:0/2349903206 shutdown_connections 2023-12-07T10:31:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.500+0000 7f810285fb80 1 -- 172.21.15.119:0/2349903206 wait complete. 2023-12-07T10:31:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.500+0000 7f810285fb80 20 client.27712 trim_cache size 0 max 16384 2023-12-07T10:31:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:31:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:31:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:31:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:31:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.501+0000 7f80eefff700 1 -- 172.21.15.119:0/2024250787 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f80d80a1c80 msgr2=0x7f80d80a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.501+0000 7f80eefff700 1 --2- 172.21.15.119:0/2024250787 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f80d80a1c80 0x7f80d80a4140 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f80e81a05f0 tx=0x7f80e0009100 comp rx=0 tx=0).stop 2023-12-07T10:31:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.501+0000 7f80eefff700 1 -- 172.21.15.119:0/2024250787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80e80739e0 msgr2=0x7f80e819f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:31:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.501+0000 7f80eefff700 1 --2- 172.21.15.119:0/2024250787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80e80739e0 0x7f80e819f960 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f80dc00e9a0 tx=0x7f80dc00ecb0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.501+0000 7f80eefff700 1 -- 172.21.15.119:0/2024250787 shutdown_connections 2023-12-07T10:31:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.501+0000 7f80eefff700 1 --2- 172.21.15.119:0/2024250787 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f80d80a1c80 0x7f80d80a4140 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.501+0000 7f80eefff700 1 --2- 172.21.15.119:0/2024250787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80e810dda0 0x7f80e8199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.501+0000 7f80eefff700 1 --2- 172.21.15.119:0/2024250787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80e80739e0 0x7f80e819f960 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.501+0000 7f80eefff700 1 --2- 172.21.15.119:0/2024250787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80e8074300 0x7f80e819fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:31:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.501+0000 7f80eefff700 1 -- 172.21.15.119:0/2024250787 >> 172.21.15.119:0/2024250787 conn(0x7f80e80fd010 msgr2=0x7f80e80fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:31:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.501+0000 7f80eefff700 1 -- 172.21.15.119:0/2024250787 shutdown_connections 2023-12-07T10:31:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:31:57.501+0000 7f80eefff700 1 -- 172.21.15.119:0/2024250787 wait complete. 2023-12-07T10:31:57.577 INFO:tasks.workunit.client.0.smithi119.stdout: 1 843427 7.24 MB/sec execute 490 sec latency 34.940 ms 2023-12-07T10:31:58.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:31:58 smithi119 ceph-mon[97908]: pgmap v359: 129 pgs: 129 active+clean; 645 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 325 op/s 2023-12-07T10:31:58.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:31:58 smithi159 ceph-mon[134726]: pgmap v359: 129 pgs: 129 active+clean; 645 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 325 op/s 2023-12-07T10:31:58.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:31:58 smithi153 ceph-mon[133625]: pgmap v359: 129 pgs: 129 active+clean; 645 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 325 op/s 2023-12-07T10:31:58.577 INFO:tasks.workunit.client.0.smithi119.stdout: 1 845281 7.25 MB/sec execute 491 sec latency 14.385 ms 2023-12-07T10:31:59.577 INFO:tasks.workunit.client.0.smithi119.stdout: 1 846207 7.24 MB/sec execute 492 sec latency 47.010 ms 2023-12-07T10:32:00.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:00 smithi119 ceph-mon[97908]: pgmap v360: 129 pgs: 129 active+clean; 645 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 200 op/s 2023-12-07T10:32:00.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:00 smithi159 ceph-mon[134726]: pgmap v360: 129 pgs: 129 active+clean; 645 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 200 op/s 2023-12-07T10:32:00.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:00 smithi153 ceph-mon[133625]: pgmap v360: 129 pgs: 129 active+clean; 645 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 200 op/s 2023-12-07T10:32:00.577 INFO:tasks.workunit.client.0.smithi119.stdout: 1 847939 7.25 MB/sec execute 493 sec latency 38.448 ms 2023-12-07T10:32:01.577 INFO:tasks.workunit.client.0.smithi119.stdout: 1 848279 7.24 MB/sec execute 494 sec latency 72.784 ms 2023-12-07T10:32:02.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:01 smithi119 ceph-mon[97908]: pgmap v361: 129 pgs: 129 active+clean; 650 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 298 op/s 2023-12-07T10:32:02.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:01 smithi159 ceph-mon[134726]: pgmap v361: 129 pgs: 129 active+clean; 650 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 298 op/s 2023-12-07T10:32:02.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:01 smithi153 ceph-mon[133625]: pgmap v361: 129 pgs: 129 active+clean; 650 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 298 op/s 2023-12-07T10:32:02.577 INFO:tasks.workunit.client.0.smithi119.stdout: 1 849240 7.23 MB/sec execute 495 sec latency 59.606 ms 2023-12-07T10:32:03.577 INFO:tasks.workunit.client.0.smithi119.stdout: 1 850258 7.23 MB/sec execute 496 sec latency 295.477 ms 2023-12-07T10:32:04.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:04 smithi119 ceph-mon[97908]: pgmap v362: 129 pgs: 129 active+clean; 653 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 252 op/s 2023-12-07T10:32:04.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:04 smithi159 ceph-mon[134726]: pgmap v362: 129 pgs: 129 active+clean; 653 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 252 op/s 2023-12-07T10:32:04.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:04 smithi153 ceph-mon[133625]: pgmap v362: 129 pgs: 129 active+clean; 653 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 252 op/s 2023-12-07T10:32:04.577 INFO:tasks.workunit.client.0.smithi119.stdout: 1 851477 7.23 MB/sec execute 497 sec latency 158.405 ms 2023-12-07T10:32:05.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:05 smithi119 ceph-mon[97908]: osdmap e105: 12 total, 12 up, 12 in 2023-12-07T10:32:05.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:05 smithi159 ceph-mon[134726]: osdmap e105: 12 total, 12 up, 12 in 2023-12-07T10:32:05.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:05 smithi153 ceph-mon[133625]: osdmap e105: 12 total, 12 up, 12 in 2023-12-07T10:32:05.578 INFO:tasks.workunit.client.0.smithi119.stdout: 1 852331 7.23 MB/sec execute 498 sec latency 248.452 ms 2023-12-07T10:32:06.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:06 smithi119 ceph-mon[97908]: pgmap v364: 129 pgs: 129 active+clean; 653 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 287 op/s 2023-12-07T10:32:06.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:06 smithi159 ceph-mon[134726]: pgmap v364: 129 pgs: 129 active+clean; 653 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 287 op/s 2023-12-07T10:32:06.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:06 smithi153 ceph-mon[133625]: pgmap v364: 129 pgs: 129 active+clean; 653 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 287 op/s 2023-12-07T10:32:06.577 INFO:tasks.workunit.client.0.smithi119.stdout: 1 853171 7.22 MB/sec execute 499 sec latency 52.900 ms 2023-12-07T10:32:07.577 INFO:tasks.workunit.client.0.smithi119.stdout: 1 855012 7.22 MB/sec execute 500 sec latency 17.412 ms 2023-12-07T10:32:08.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:08 smithi119 ceph-mon[97908]: pgmap v365: 129 pgs: 129 active+clean; 658 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.6 MiB/s wr, 351 op/s 2023-12-07T10:32:08.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:08 smithi159 ceph-mon[134726]: pgmap v365: 129 pgs: 129 active+clean; 658 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.6 MiB/s wr, 351 op/s 2023-12-07T10:32:08.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:08 smithi153 ceph-mon[133625]: pgmap v365: 129 pgs: 129 active+clean; 658 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.6 MiB/s wr, 351 op/s 2023-12-07T10:32:08.577 INFO:tasks.workunit.client.0.smithi119.stdout: 1 856111 7.22 MB/sec execute 501 sec latency 63.854 ms 2023-12-07T10:32:09.578 INFO:tasks.workunit.client.0.smithi119.stdout: 1 857312 7.22 MB/sec execute 502 sec latency 93.646 ms 2023-12-07T10:32:10.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:10 smithi119 ceph-mon[97908]: pgmap v366: 129 pgs: 129 active+clean; 658 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.6 MiB/s wr, 351 op/s 2023-12-07T10:32:10.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:10 smithi159 ceph-mon[134726]: pgmap v366: 129 pgs: 129 active+clean; 658 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.6 MiB/s wr, 351 op/s 2023-12-07T10:32:10.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:10 smithi153 ceph-mon[133625]: pgmap v366: 129 pgs: 129 active+clean; 658 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.6 MiB/s wr, 351 op/s 2023-12-07T10:32:10.578 INFO:tasks.workunit.client.0.smithi119.stdout: 1 859004 7.22 MB/sec execute 503 sec latency 31.742 ms 2023-12-07T10:32:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.669+0000 7f64f51ff700 1 -- 172.21.15.119:0/3025124317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64f0109720 msgr2=0x7f64f0109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.669+0000 7f64f51ff700 1 --2- 172.21.15.119:0/3025124317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64f0109720 0x7f64f0109b00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009990 tx=0x7f64e0009ca0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.669+0000 7f64f51ff700 1 -- 172.21.15.119:0/3025124317 shutdown_connections 2023-12-07T10:32:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.669+0000 7f64f51ff700 1 --2- 172.21.15.119:0/3025124317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64f01040d0 0x7f64f01105e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.669+0000 7f64f51ff700 1 --2- 172.21.15.119:0/3025124317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64f0109720 0x7f64f0109b00 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.670+0000 7f64f51ff700 1 --2- 172.21.15.119:0/3025124317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64f0103710 0x7f64f0103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.670+0000 7f64f51ff700 1 -- 172.21.15.119:0/3025124317 >> 172.21.15.119:0/3025124317 conn(0x7f64f00ff1c0 msgr2=0x7f64f01015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:32:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.670+0000 7f64f51ff700 1 -- 172.21.15.119:0/3025124317 shutdown_connections 2023-12-07T10:32:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.670+0000 7f64f51ff700 1 -- 172.21.15.119:0/3025124317 wait complete. 2023-12-07T10:32:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.671+0000 7f64f51ff700 1 Processor -- start 2023-12-07T10:32:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.671+0000 7f64f51ff700 1 -- start start 2023-12-07T10:32:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.672+0000 7f64f51ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64f0103710 0x7f64f019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.672+0000 7f64f51ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64f01040d0 0x7f64f019fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.672+0000 7f64f51ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64f0109720 0x7f64f0199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.672+0000 7f64f51ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f0199fe0 con 0x7f64f0109720 2023-12-07T10:32:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.672+0000 7f64f51ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f019a150 con 0x7f64f0103710 2023-12-07T10:32:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.672+0000 7f64f51ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f019a310 con 0x7f64f01040d0 2023-12-07T10:32:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.672+0000 7f64ee59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64f01040d0 0x7f64f019fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.672+0000 7f64ee59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64f01040d0 0x7f64f019fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:36832/0 (socket says 172.21.15.119:36832) 2023-12-07T10:32:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.672+0000 7f64ee59c700 1 -- 172.21.15.119:0/3587933239 learned_addr learned my addr 172.21.15.119:0/3587933239 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:32:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.672+0000 7f64eed9d700 1 --2- 172.21.15.119:0/3587933239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64f0103710 0x7f64f019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.672+0000 7f64ef59e700 1 --2- 172.21.15.119:0/3587933239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64f0109720 0x7f64f0199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.672+0000 7f64ee59c700 1 -- 172.21.15.119:0/3587933239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64f0103710 msgr2=0x7f64f019f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.672+0000 7f64ee59c700 1 --2- 172.21.15.119:0/3587933239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64f0103710 0x7f64f019f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.672+0000 7f64ee59c700 1 -- 172.21.15.119:0/3587933239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64f0109720 msgr2=0x7f64f0199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.672+0000 7f64ee59c700 1 --2- 172.21.15.119:0/3587933239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64f0109720 0x7f64f0199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.672+0000 7f64ee59c700 1 -- 172.21.15.119:0/3587933239 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e00095f0 con 0x7f64f01040d0 2023-12-07T10:32:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.672+0000 7f64ef59e700 1 --2- 172.21.15.119:0/3587933239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64f0109720 0x7f64f0199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:32:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.673+0000 7f64ee59c700 1 --2- 172.21.15.119:0/3587933239 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64f01040d0 0x7f64f019fd90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f64e400d950 tx=0x7f64e400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:32:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.673+0000 7f64d7fff700 1 -- 172.21.15.119:0/3587933239 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4009940 con 0x7f64f01040d0 2023-12-07T10:32:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.673+0000 7f64d7fff700 1 -- 172.21.15.119:0/3587933239 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64e4010460 con 0x7f64f01040d0 2023-12-07T10:32:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.673+0000 7f64f51ff700 1 -- 172.21.15.119:0/3587933239 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f019a5f0 con 0x7f64f01040d0 2023-12-07T10:32:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.673+0000 7f64d7fff700 1 -- 172.21.15.119:0/3587933239 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e400f620 con 0x7f64f01040d0 2023-12-07T10:32:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.673+0000 7f64f51ff700 1 -- 172.21.15.119:0/3587933239 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f01a6840 con 0x7f64f01040d0 2023-12-07T10:32:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.676+0000 7f64d7fff700 1 -- 172.21.15.119:0/3587933239 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f64e40108d0 con 0x7f64f01040d0 2023-12-07T10:32:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.678+0000 7f6508b9db80 1 Processor -- start 2023-12-07T10:32:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.678+0000 7f6508b9db80 1 -- start start 2023-12-07T10:32:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.678+0000 7f64d7fff700 1 --2- 172.21.15.119:0/3587933239 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f64d80a1c80 0x7f64d80a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.679+0000 7f64d7fff700 1 -- 172.21.15.119:0/3587933239 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f64e40cdc80 con 0x7f64f01040d0 2023-12-07T10:32:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.679+0000 7f6508b9db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fdc6b5a60 0x558fdc6b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.679+0000 7f6508b9db80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fdc6b6380 0x558fdc6b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.679+0000 7f6508b9db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fdc6b0ad0 0x558fdc6b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.679+0000 7f6508b9db80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fdc624760 con 0x558fdc6b6380 2023-12-07T10:32:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.679+0000 7f6508b9db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fdc60b740 con 0x558fdc6b5a60 2023-12-07T10:32:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.679+0000 7f6508b9db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fdc609060 con 0x558fdc6b0ad0 2023-12-07T10:32:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.679+0000 7f64ef59e700 1 --2- 172.21.15.119:0/3587933239 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f64d80a1c80 0x7f64d80a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.679+0000 7f64eed9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fdc6b5a60 0x558fdc6b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.679+0000 7f64eed9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fdc6b5a60 0x558fdc6b5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:53726/0 (socket says 172.21.15.119:53726) 2023-12-07T10:32:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.679+0000 7f64eed9d700 1 -- 172.21.15.119:0/2171607106 learned_addr learned my addr 172.21.15.119:0/2171607106 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:32:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.679+0000 7f64ee59c700 1 --2- 172.21.15.119:0/2171607106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fdc6b6380 0x558fdc6b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.679+0000 7f64ef59e700 1 --2- 172.21.15.119:0/2171607106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fdc6b0ad0 0x558fdc6b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.679+0000 7f64eed9d700 1 -- 172.21.15.119:0/2171607106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fdc6b0ad0 msgr2=0x558fdc6b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.679+0000 7f64eed9d700 1 --2- 172.21.15.119:0/2171607106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fdc6b0ad0 0x558fdc6b0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.679+0000 7f64eed9d700 1 -- 172.21.15.119:0/2171607106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fdc6b6380 msgr2=0x558fdc6b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.679+0000 7f64eed9d700 1 --2- 172.21.15.119:0/2171607106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fdc6b6380 0x558fdc6b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.679+0000 7f64eed9d700 1 -- 172.21.15.119:0/2171607106 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fdc4815d0 con 0x558fdc6b5a60 2023-12-07T10:32:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.680+0000 7f64eed9d700 1 --2- 172.21.15.119:0/2171607106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fdc6b5a60 0x558fdc6b5e40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f64e0005f30 tx=0x7f64e0009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92372db8005ac0b6 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:32:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.680+0000 7f64ef59e700 1 --2- 172.21.15.119:0/3587933239 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f64d80a1c80 0x7f64d80a4140 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f64e80076f0 tx=0x7f64e8007ab0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:32:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.680+0000 7f64d57fa700 1 -- 172.21.15.119:0/2171607106 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e000b7d0 con 0x558fdc6b5a60 2023-12-07T10:32:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.680+0000 7f64d57fa700 1 -- 172.21.15.119:0/2171607106 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64e001e040 con 0x558fdc6b5a60 2023-12-07T10:32:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.680+0000 7f64d57fa700 1 -- 172.21.15.119:0/2171607106 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0003a80 con 0x558fdc6b5a60 2023-12-07T10:32:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.681+0000 7f6508b9db80 1 -- 172.21.15.119:0/2171607106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fdc6b5a60 msgr2=0x558fdc6b5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.681+0000 7f6508b9db80 1 --2- 172.21.15.119:0/2171607106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fdc6b5a60 0x558fdc6b5e40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f64e0005f30 tx=0x7f64e0009550 comp rx=0 tx=0).stop 2023-12-07T10:32:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.681+0000 7f6508b9db80 1 -- 172.21.15.119:0/2171607106 shutdown_connections 2023-12-07T10:32:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.681+0000 7f6508b9db80 1 --2- 172.21.15.119:0/2171607106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fdc6b0ad0 0x558fdc6b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.681+0000 7f6508b9db80 1 --2- 172.21.15.119:0/2171607106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fdc6b5a60 0x558fdc6b5e40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.681+0000 7f6508b9db80 1 --2- 172.21.15.119:0/2171607106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fdc6b6380 0x558fdc6b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.681+0000 7f6508b9db80 1 -- 172.21.15.119:0/2171607106 >> 172.21.15.119:0/2171607106 conn(0x558fdc5df300 msgr2=0x558fdc6aab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:32:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.681+0000 7f6508b9db80 1 -- 172.21.15.119:0/2171607106 shutdown_connections 2023-12-07T10:32:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.681+0000 7f6508b9db80 1 -- 172.21.15.119:0/2171607106 wait complete. 2023-12-07T10:32:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.681+0000 7f6508b9db80 1 -- start start 2023-12-07T10:32:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.682+0000 7f6508b9db80 1 Processor -- start 2023-12-07T10:32:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.682+0000 7f6508b9db80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fdc6b0ad0 0x558fdc6bc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.682+0000 7f6508b9db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fdc6b5a60 0x558fdc6b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.682+0000 7f6508b9db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fdc6b6380 0x558fdc6b9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.682+0000 7f6508b9db80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fdc604950 con 0x558fdc6b0ad0 2023-12-07T10:32:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.682+0000 7f6508b9db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fdc5be500 con 0x558fdc6b5a60 2023-12-07T10:32:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.682+0000 7f6508b9db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fdc5be1a0 con 0x558fdc6b6380 2023-12-07T10:32:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.682+0000 7f6508b9db80 10 client.0 _ll_register_callbacks cb 0x558fdc5a58e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:32:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.682+0000 7f64ee59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fdc6b5a60 0x558fdc6b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.682+0000 7f64ee59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fdc6b5a60 0x558fdc6b9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:53734/0 (socket says 172.21.15.119:53734) 2023-12-07T10:32:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.683+0000 7f64ee59c700 1 -- 172.21.15.119:0/975173304 learned_addr learned my addr 172.21.15.119:0/975173304 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:32:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.683+0000 7f64eed9d700 1 --2- 172.21.15.119:0/975173304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fdc6b0ad0 0x558fdc6bc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.683+0000 7f64ef59e700 1 --2- 172.21.15.119:0/975173304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fdc6b6380 0x558fdc6b9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.683+0000 7f64ee59c700 1 -- 172.21.15.119:0/975173304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fdc6b6380 msgr2=0x558fdc6b9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.683+0000 7f64ee59c700 1 --2- 172.21.15.119:0/975173304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fdc6b6380 0x558fdc6b9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.683+0000 7f64ee59c700 1 -- 172.21.15.119:0/975173304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fdc6b0ad0 msgr2=0x558fdc6bc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.683+0000 7f64ee59c700 1 --2- 172.21.15.119:0/975173304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fdc6b0ad0 0x558fdc6bc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.683+0000 7f64ee59c700 1 -- 172.21.15.119:0/975173304 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f0110ee0 con 0x558fdc6b5a60 2023-12-07T10:32:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.683+0000 7f64eed9d700 1 --2- 172.21.15.119:0/975173304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fdc6b0ad0 0x558fdc6bc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:32:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.683+0000 7f64ee59c700 1 --2- 172.21.15.119:0/975173304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fdc6b5a60 0x558fdc6b9690 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f64e400b2f0 tx=0x7f64e409ad70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=370d3b842f48bd92 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:32:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.683+0000 7f64d2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:32:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.683+0000 7f64d2ff5700 1 -- 172.21.15.119:0/975173304 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e40a2950 con 0x558fdc6b5a60 2023-12-07T10:32:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.683+0000 7f6508b9db80 1 -- 172.21.15.119:0/975173304 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558fdc6a36f0 con 0x558fdc6b5a60 2023-12-07T10:32:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.683+0000 7f64d2ff5700 1 -- 172.21.15.119:0/975173304 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f64e409a820 con 0x558fdc6b5a60 2023-12-07T10:32:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.684+0000 7f64d2ff5700 1 -- 172.21.15.119:0/975173304 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64e409a9e0 con 0x558fdc6b5a60 2023-12-07T10:32:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.684+0000 7f6508b9db80 10 client.27743 fetch_fsmap learned FSMap version 59 2023-12-07T10:32:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.684+0000 7f6508b9db80 1 -- 172.21.15.119:0/975173304 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x558fdc42dd80 con 0x558fdc6b5a60 2023-12-07T10:32:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.684+0000 7f64d2ff5700 1 -- 172.21.15.119:0/975173304 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558fdc6a36f0 con 0x558fdc6b5a60 2023-12-07T10:32:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.684+0000 7f64d2ff5700 1 -- 172.21.15.119:0/975173304 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 59) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f64e40a24c0 con 0x558fdc6b5a60 2023-12-07T10:32:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.684+0000 7f6508b9db80 10 client.27743 fetch_fsmap finished waiting for FSMap version 59 2023-12-07T10:32:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.684+0000 7f6508b9db80 10 client.27743 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:32:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.684+0000 7f6508b9db80 20 client.27743 populate_metadata read hostname 'smithi119' 2023-12-07T10:32:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.684+0000 7f6508b9db80 1 --2- 172.21.15.119:0/975173304 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558fdc6c9e30 0x558fdc6cc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.684+0000 7f6508b9db80 4 client.27743 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:32:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.684+0000 7f6508b9db80 1 -- 172.21.15.119:0/975173304 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558fdc67f740 con 0x558fdc6c9e30 2023-12-07T10:32:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.685+0000 7f64eed9d700 1 --2- 172.21.15.119:0/975173304 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558fdc6c9e30 0x558fdc6cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.685+0000 7f64eed9d700 1 --2- 172.21.15.119:0/975173304 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558fdc6c9e30 0x558fdc6cc250 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f303708b01997496 server_cookie=3c369ab058f1c995 in_seq=0 out_seq=0 2023-12-07T10:32:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.685+0000 7f64d2ff5700 10 client.27743 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:32:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.686+0000 7f64d2ff5700 1 -- 172.21.15.119:0/975173304 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558fdc67f740 con 0x558fdc6c9e30 2023-12-07T10:32:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.686+0000 7f64d2ff5700 10 client.27743 handle_command_reply: tid=0 2023-12-07T10:32:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.686+0000 7f6508b9db80 1 client.27743 shutdown 2023-12-07T10:32:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.686+0000 7f6508b9db80 1 -- 172.21.15.119:0/975173304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fdc6b5a60 msgr2=0x558fdc6b9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.686+0000 7f6508b9db80 1 --2- 172.21.15.119:0/975173304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fdc6b5a60 0x558fdc6b9690 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f64e400b2f0 tx=0x7f64e409ad70 comp rx=0 tx=0).stop 2023-12-07T10:32:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.686+0000 7f6508b9db80 1 -- 172.21.15.119:0/975173304 shutdown_connections 2023-12-07T10:32:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.686+0000 7f6508b9db80 1 --2- 172.21.15.119:0/975173304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fdc6b6380 0x558fdc6b9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.686+0000 7f6508b9db80 1 --2- 172.21.15.119:0/975173304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fdc6b0ad0 0x558fdc6bc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.686+0000 7f6508b9db80 1 --2- 172.21.15.119:0/975173304 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558fdc6c9e30 0x558fdc6cc250 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.686+0000 7f64d2ff5700 0 client.27743 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:32:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.686+0000 7f6508b9db80 1 --2- 172.21.15.119:0/975173304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fdc6b5a60 0x558fdc6b9690 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.686+0000 7f6508b9db80 1 -- 172.21.15.119:0/975173304 >> 172.21.15.119:0/975173304 conn(0x558fdc5df300 msgr2=0x558fdc6a5980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:32:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.687+0000 7f6508b9db80 1 -- 172.21.15.119:0/975173304 shutdown_connections 2023-12-07T10:32:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.687+0000 7f6508b9db80 1 -- 172.21.15.119:0/975173304 wait complete. 2023-12-07T10:32:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.687+0000 7f6508b9db80 20 client.27743 trim_cache size 0 max 16384 2023-12-07T10:32:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.703+0000 7f6508b9db80 1 Processor -- start 2023-12-07T10:32:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.703+0000 7f6508b9db80 1 -- start start 2023-12-07T10:32:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.704+0000 7f6508b9db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fdc6b0ad0 0x558fdc6b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.704+0000 7f6508b9db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fdc6b5a60 0x558fdc6b9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.704+0000 7f6508b9db80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fdc6b6380 0x558fdc6c0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.704+0000 7f6508b9db80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fdc4838c0 con 0x558fdc6b6380 2023-12-07T10:32:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.704+0000 7f6508b9db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fdc6bc6b0 con 0x558fdc6b5a60 2023-12-07T10:32:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.704+0000 7f6508b9db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fdc6a3480 con 0x558fdc6b0ad0 2023-12-07T10:32:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.704+0000 7f64ef59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fdc6b6380 0x558fdc6c0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.704+0000 7f64eed9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fdc6b0ad0 0x558fdc6b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.704+0000 7f64ef59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fdc6b6380 0x558fdc6c0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58336/0 (socket says 172.21.15.119:58336) 2023-12-07T10:32:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.704+0000 7f64ef59e700 1 -- 172.21.15.119:0/1641680785 learned_addr learned my addr 172.21.15.119:0/1641680785 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:32:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.704+0000 7f64eed9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fdc6b0ad0 0x558fdc6b0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:36848/0 (socket says 172.21.15.119:36848) 2023-12-07T10:32:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.704+0000 7f64ee59c700 1 --2- 172.21.15.119:0/1641680785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fdc6b5a60 0x558fdc6b9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.704+0000 7f64eed9d700 1 -- 172.21.15.119:0/1641680785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fdc6b5a60 msgr2=0x558fdc6b9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.704+0000 7f64eed9d700 1 --2- 172.21.15.119:0/1641680785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fdc6b5a60 0x558fdc6b9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.704+0000 7f64eed9d700 1 -- 172.21.15.119:0/1641680785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fdc6b6380 msgr2=0x558fdc6c0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.704+0000 7f64eed9d700 1 --2- 172.21.15.119:0/1641680785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fdc6b6380 0x558fdc6c0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.704+0000 7f64eed9d700 1 -- 172.21.15.119:0/1641680785 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fdc416c00 con 0x558fdc6b0ad0 2023-12-07T10:32:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.705+0000 7f64eed9d700 1 --2- 172.21.15.119:0/1641680785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fdc6b0ad0 0x558fdc6b0540 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009990 tx=0x7f64e0005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f23c2096ca1a6e6 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:32:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.705+0000 7f64d5ffb700 1 -- 172.21.15.119:0/1641680785 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0031ea0 con 0x558fdc6b0ad0 2023-12-07T10:32:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.705+0000 7f64d5ffb700 1 -- 172.21.15.119:0/1641680785 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64e000b5d0 con 0x558fdc6b0ad0 2023-12-07T10:32:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.705+0000 7f64d5ffb700 1 -- 172.21.15.119:0/1641680785 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0003db0 con 0x558fdc6b0ad0 2023-12-07T10:32:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.706+0000 7f6508b9db80 1 -- 172.21.15.119:0/1641680785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fdc6b0ad0 msgr2=0x558fdc6b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.706+0000 7f6508b9db80 1 --2- 172.21.15.119:0/1641680785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fdc6b0ad0 0x558fdc6b0540 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009990 tx=0x7f64e0005b00 comp rx=0 tx=0).stop 2023-12-07T10:32:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.706+0000 7f6508b9db80 1 -- 172.21.15.119:0/1641680785 shutdown_connections 2023-12-07T10:32:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.706+0000 7f6508b9db80 1 --2- 172.21.15.119:0/1641680785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fdc6b6380 0x558fdc6c0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.706+0000 7f6508b9db80 1 --2- 172.21.15.119:0/1641680785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fdc6b0ad0 0x558fdc6b0540 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.706+0000 7f6508b9db80 1 --2- 172.21.15.119:0/1641680785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fdc6b5a60 0x558fdc6b9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.706+0000 7f6508b9db80 1 -- 172.21.15.119:0/1641680785 >> 172.21.15.119:0/1641680785 conn(0x558fdc5df300 msgr2=0x558fdc6aaa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:32:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.706+0000 7f6508b9db80 1 -- 172.21.15.119:0/1641680785 shutdown_connections 2023-12-07T10:32:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.706+0000 7f6508b9db80 1 -- 172.21.15.119:0/1641680785 wait complete. 2023-12-07T10:32:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.706+0000 7f6508b9db80 1 -- start start 2023-12-07T10:32:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.706+0000 7f6508b9db80 1 Processor -- start 2023-12-07T10:32:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.707+0000 7f6508b9db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fdc6b0ad0 0x558fdc6c1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.707+0000 7f6508b9db80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fdc6b5a60 0x558fdc6bcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.707+0000 7f6508b9db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fdc6b6380 0x558fdc6bd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.707+0000 7f6508b9db80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fdc6bd990 con 0x558fdc6b5a60 2023-12-07T10:32:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.707+0000 7f6508b9db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fdc6bdb00 con 0x558fdc6b6380 2023-12-07T10:32:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.707+0000 7f6508b9db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fdc71f320 con 0x558fdc6b0ad0 2023-12-07T10:32:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.707+0000 7f6508b9db80 10 client.0 _ll_register_callbacks cb 0x558fdc5a58e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:32:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.707+0000 7f64eed9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fdc6b0ad0 0x558fdc6c1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.707+0000 7f64eed9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fdc6b0ad0 0x558fdc6c1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:36856/0 (socket says 172.21.15.119:36856) 2023-12-07T10:32:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.707+0000 7f64eed9d700 1 -- 172.21.15.119:0/3882050599 learned_addr learned my addr 172.21.15.119:0/3882050599 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:32:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.707+0000 7f64ef59e700 1 --2- 172.21.15.119:0/3882050599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fdc6b6380 0x558fdc6bd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.707+0000 7f64ee59c700 1 --2- 172.21.15.119:0/3882050599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fdc6b5a60 0x558fdc6bcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.707+0000 7f64eed9d700 1 -- 172.21.15.119:0/3882050599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fdc6b6380 msgr2=0x558fdc6bd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.707+0000 7f64eed9d700 1 --2- 172.21.15.119:0/3882050599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fdc6b6380 0x558fdc6bd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.708+0000 7f64eed9d700 1 -- 172.21.15.119:0/3882050599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fdc6b5a60 msgr2=0x558fdc6bcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.708+0000 7f64eed9d700 1 --2- 172.21.15.119:0/3882050599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fdc6b5a60 0x558fdc6bcd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.708+0000 7f64eed9d700 1 -- 172.21.15.119:0/3882050599 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558fdc4815d0 con 0x558fdc6b0ad0 2023-12-07T10:32:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.708+0000 7f64eed9d700 1 --2- 172.21.15.119:0/3882050599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fdc6b0ad0 0x558fdc6c1130 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f64e000b580 tx=0x7f64e0005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bde1e5bf7c0254 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:32:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.708+0000 7f64d2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:32:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.708+0000 7f64d2ff5700 1 -- 172.21.15.119:0/3882050599 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0003bf0 con 0x558fdc6b0ad0 2023-12-07T10:32:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.708+0000 7f6508b9db80 1 -- 172.21.15.119:0/3882050599 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f64e001e040 con 0x558fdc6b0ad0 2023-12-07T10:32:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.708+0000 7f64d2ff5700 1 -- 172.21.15.119:0/3882050599 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f64e0031030 con 0x558fdc6b0ad0 2023-12-07T10:32:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.708+0000 7f64d2ff5700 1 -- 172.21.15.119:0/3882050599 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64e0003d50 con 0x558fdc6b0ad0 2023-12-07T10:32:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.708+0000 7f6508b9db80 10 client.27730 fetch_fsmap learned FSMap version 59 2023-12-07T10:32:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.708+0000 7f6508b9db80 1 -- 172.21.15.119:0/3882050599 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x558fdc4541f0 con 0x558fdc6b0ad0 2023-12-07T10:32:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.709+0000 7f64d2ff5700 1 -- 172.21.15.119:0/3882050599 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64e001e040 con 0x558fdc6b0ad0 2023-12-07T10:32:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.709+0000 7f64d2ff5700 1 -- 172.21.15.119:0/3882050599 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 59) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f64e00316f0 con 0x558fdc6b0ad0 2023-12-07T10:32:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.709+0000 7f6508b9db80 10 client.27730 fetch_fsmap finished waiting for FSMap version 59 2023-12-07T10:32:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.709+0000 7f6508b9db80 10 client.27730 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:32:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.709+0000 7f6508b9db80 20 client.27730 populate_metadata read hostname 'smithi119' 2023-12-07T10:32:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.709+0000 7f6508b9db80 1 --2- 172.21.15.119:0/3882050599 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558fdc6c9e30 0x558fdc722650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.709+0000 7f6508b9db80 4 client.27730 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:32:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.709+0000 7f6508b9db80 1 -- 172.21.15.119:0/3882050599 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558fdc5caf70 con 0x558fdc6c9e30 2023-12-07T10:32:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.709+0000 7f64ee59c700 1 --2- 172.21.15.119:0/3882050599 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558fdc6c9e30 0x558fdc722650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.710+0000 7f64ee59c700 1 --2- 172.21.15.119:0/3882050599 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558fdc6c9e30 0x558fdc722650 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5303a5e8dcf58c1a server_cookie=be5d56064a0301d6 in_seq=0 out_seq=0 2023-12-07T10:32:10.714 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.710+0000 7f64d2ff5700 10 client.27730 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:32:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.738+0000 7f64d2ff5700 1 -- 172.21.15.119:0/3882050599 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558fdc5caf70 con 0x558fdc6c9e30 2023-12-07T10:32:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.738+0000 7f64d2ff5700 10 client.27730 handle_command_reply: tid=0 2023-12-07T10:32:10.738 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.738+0000 7f6508b9db80 1 client.27730 shutdown 2023-12-07T10:32:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.738+0000 7f6508b9db80 1 -- 172.21.15.119:0/3882050599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fdc6b0ad0 msgr2=0x558fdc6c1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.738+0000 7f6508b9db80 1 --2- 172.21.15.119:0/3882050599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fdc6b0ad0 0x558fdc6c1130 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f64e000b580 tx=0x7f64e0005970 comp rx=0 tx=0).stop 2023-12-07T10:32:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.738+0000 7f6508b9db80 1 -- 172.21.15.119:0/3882050599 shutdown_connections 2023-12-07T10:32:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.738+0000 7f6508b9db80 1 --2- 172.21.15.119:0/3882050599 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558fdc6c9e30 0x558fdc722650 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.738+0000 7f6508b9db80 1 --2- 172.21.15.119:0/3882050599 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558fdc6b6380 0x558fdc6bd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.738+0000 7f6508b9db80 1 --2- 172.21.15.119:0/3882050599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558fdc6b0ad0 0x558fdc6c1130 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.738+0000 7f6508b9db80 1 --2- 172.21.15.119:0/3882050599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558fdc6b5a60 0x558fdc6bcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.738+0000 7f64d2ff5700 0 client.27730 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:32:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.738+0000 7f6508b9db80 1 -- 172.21.15.119:0/3882050599 >> 172.21.15.119:0/3882050599 conn(0x558fdc5df300 msgr2=0x558fdc6c5820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:32:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.738+0000 7f6508b9db80 1 -- 172.21.15.119:0/3882050599 shutdown_connections 2023-12-07T10:32:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.738+0000 7f6508b9db80 1 -- 172.21.15.119:0/3882050599 wait complete. 2023-12-07T10:32:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.738+0000 7f6508b9db80 20 client.27730 trim_cache size 0 max 16384 2023-12-07T10:32:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:32:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:32:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:32:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:32:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.739+0000 7f64f51ff700 1 -- 172.21.15.119:0/3587933239 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f64d80a1c80 msgr2=0x7f64d80a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.739+0000 7f64f51ff700 1 --2- 172.21.15.119:0/3587933239 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f64d80a1c80 0x7f64d80a4140 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f64e80076f0 tx=0x7f64e8007ab0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.739+0000 7f64f51ff700 1 -- 172.21.15.119:0/3587933239 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64f01040d0 msgr2=0x7f64f019fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.739+0000 7f64f51ff700 1 --2- 172.21.15.119:0/3587933239 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64f01040d0 0x7f64f019fd90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f64e400d950 tx=0x7f64e400dc60 comp rx=0 tx=0).stop 2023-12-07T10:32:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.739+0000 7f64f51ff700 1 -- 172.21.15.119:0/3587933239 shutdown_connections 2023-12-07T10:32:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.739+0000 7f64f51ff700 1 --2- 172.21.15.119:0/3587933239 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f64d80a1c80 0x7f64d80a4140 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.739+0000 7f64f51ff700 1 --2- 172.21.15.119:0/3587933239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64f0109720 0x7f64f0199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.739+0000 7f64f51ff700 1 --2- 172.21.15.119:0/3587933239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64f0103710 0x7f64f019f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.739+0000 7f64f51ff700 1 --2- 172.21.15.119:0/3587933239 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64f01040d0 0x7f64f019fd90 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.739+0000 7f64f51ff700 1 -- 172.21.15.119:0/3587933239 >> 172.21.15.119:0/3587933239 conn(0x7f64f00ff1c0 msgr2=0x7f64f00ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:32:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.739+0000 7f64f51ff700 1 -- 172.21.15.119:0/3587933239 shutdown_connections 2023-12-07T10:32:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:10.739+0000 7f64f51ff700 1 -- 172.21.15.119:0/3587933239 wait complete. 2023-12-07T10:32:11.578 INFO:tasks.workunit.client.0.smithi119.stdout: 1 860012 7.22 MB/sec execute 504 sec latency 15.372 ms 2023-12-07T10:32:12.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:11 smithi119 ceph-mon[97908]: osdmap e106: 12 total, 12 up, 12 in 2023-12-07T10:32:12.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:11 smithi119 ceph-mon[97908]: pgmap v368: 129 pgs: 129 active+clean; 665 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.4 MiB/s wr, 376 op/s 2023-12-07T10:32:12.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:11 smithi159 ceph-mon[134726]: osdmap e106: 12 total, 12 up, 12 in 2023-12-07T10:32:12.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:11 smithi159 ceph-mon[134726]: pgmap v368: 129 pgs: 129 active+clean; 665 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.4 MiB/s wr, 376 op/s 2023-12-07T10:32:12.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:11 smithi153 ceph-mon[133625]: osdmap e106: 12 total, 12 up, 12 in 2023-12-07T10:32:12.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:11 smithi153 ceph-mon[133625]: pgmap v368: 129 pgs: 129 active+clean; 665 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.4 MiB/s wr, 376 op/s 2023-12-07T10:32:12.578 INFO:tasks.workunit.client.0.smithi119.stdout: 1 861793 7.22 MB/sec execute 505 sec latency 17.965 ms 2023-12-07T10:32:13.578 INFO:tasks.workunit.client.0.smithi119.stdout: 1 863179 7.23 MB/sec execute 506 sec latency 22.142 ms 2023-12-07T10:32:14.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:14 smithi119 ceph-mon[97908]: pgmap v369: 129 pgs: 129 active+clean; 663 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.8 MiB/s wr, 355 op/s 2023-12-07T10:32:14.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:14 smithi159 ceph-mon[134726]: pgmap v369: 129 pgs: 129 active+clean; 663 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.8 MiB/s wr, 355 op/s 2023-12-07T10:32:14.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:14 smithi153 ceph-mon[133625]: pgmap v369: 129 pgs: 129 active+clean; 663 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.8 MiB/s wr, 355 op/s 2023-12-07T10:32:14.578 INFO:tasks.workunit.client.0.smithi119.stdout: 1 864638 7.22 MB/sec execute 507 sec latency 18.810 ms 2023-12-07T10:32:15.578 INFO:tasks.workunit.client.0.smithi119.stdout: 1 866525 7.23 MB/sec execute 508 sec latency 14.198 ms 2023-12-07T10:32:16.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:16 smithi119 ceph-mon[97908]: pgmap v370: 129 pgs: 129 active+clean; 663 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.1 MiB/s wr, 318 op/s 2023-12-07T10:32:16.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:16 smithi159 ceph-mon[134726]: pgmap v370: 129 pgs: 129 active+clean; 663 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.1 MiB/s wr, 318 op/s 2023-12-07T10:32:16.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:16 smithi153 ceph-mon[133625]: pgmap v370: 129 pgs: 129 active+clean; 663 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.1 MiB/s wr, 318 op/s 2023-12-07T10:32:16.578 INFO:tasks.workunit.client.0.smithi119.stdout: 1 867678 7.23 MB/sec execute 509 sec latency 14.385 ms 2023-12-07T10:32:17.578 INFO:tasks.workunit.client.0.smithi119.stdout: 1 869641 7.23 MB/sec execute 510 sec latency 16.614 ms 2023-12-07T10:32:18.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:18 smithi119 ceph-mon[97908]: pgmap v371: 129 pgs: 129 active+clean; 674 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 240 op/s 2023-12-07T10:32:18.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:18 smithi159 ceph-mon[134726]: pgmap v371: 129 pgs: 129 active+clean; 674 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 240 op/s 2023-12-07T10:32:18.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:18 smithi153 ceph-mon[133625]: pgmap v371: 129 pgs: 129 active+clean; 674 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 240 op/s 2023-12-07T10:32:18.578 INFO:tasks.workunit.client.0.smithi119.stdout: 1 870616 7.23 MB/sec execute 511 sec latency 21.790 ms 2023-12-07T10:32:19.578 INFO:tasks.workunit.client.0.smithi119.stdout: 1 872396 7.23 MB/sec execute 512 sec latency 19.360 ms 2023-12-07T10:32:20.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:20 smithi119 ceph-mon[97908]: pgmap v372: 129 pgs: 129 active+clean; 674 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 240 op/s 2023-12-07T10:32:20.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:20 smithi159 ceph-mon[134726]: pgmap v372: 129 pgs: 129 active+clean; 674 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 240 op/s 2023-12-07T10:32:20.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:20 smithi153 ceph-mon[133625]: pgmap v372: 129 pgs: 129 active+clean; 674 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 240 op/s 2023-12-07T10:32:20.578 INFO:tasks.workunit.client.0.smithi119.stdout: 1 873800 7.24 MB/sec execute 513 sec latency 37.123 ms 2023-12-07T10:32:21.578 INFO:tasks.workunit.client.0.smithi119.stdout: 1 875258 7.23 MB/sec execute 514 sec latency 18.119 ms 2023-12-07T10:32:22.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:21 smithi119 ceph-mon[97908]: pgmap v373: 129 pgs: 129 active+clean; 681 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 310 op/s 2023-12-07T10:32:22.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:21 smithi159 ceph-mon[134726]: pgmap v373: 129 pgs: 129 active+clean; 681 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 310 op/s 2023-12-07T10:32:22.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:21 smithi153 ceph-mon[133625]: pgmap v373: 129 pgs: 129 active+clean; 681 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 310 op/s 2023-12-07T10:32:22.578 INFO:tasks.workunit.client.0.smithi119.stdout: 1 877166 7.25 MB/sec execute 515 sec latency 15.255 ms 2023-12-07T10:32:23.578 INFO:tasks.workunit.client.0.smithi119.stdout: 1 878353 7.24 MB/sec execute 516 sec latency 13.247 ms 2023-12-07T10:32:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.903+0000 7f3375bff700 1 -- 172.21.15.119:0/2618735813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3370068ce0 msgr2=0x7f33700690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.903+0000 7f3375bff700 1 --2- 172.21.15.119:0/2618735813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3370068ce0 0x7f33700690c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3360009990 tx=0x7f3360009ca0 comp rx=0 tx=0).stop 2023-12-07T10:32:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.903+0000 7f3375bff700 1 -- 172.21.15.119:0/2618735813 shutdown_connections 2023-12-07T10:32:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.903+0000 7f3375bff700 1 --2- 172.21.15.119:0/2618735813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3370106a80 0x7f337010e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.903+0000 7f3375bff700 1 --2- 172.21.15.119:0/2618735813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3370068ce0 0x7f33700690c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.903+0000 7f3375bff700 1 --2- 172.21.15.119:0/2618735813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33701060c0 0x7f3370106540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.903+0000 7f3375bff700 1 -- 172.21.15.119:0/2618735813 >> 172.21.15.119:0/2618735813 conn(0x7f33700775b0 msgr2=0x7f33700779c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:32:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.904+0000 7f3375bff700 1 -- 172.21.15.119:0/2618735813 shutdown_connections 2023-12-07T10:32:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.904+0000 7f3375bff700 1 -- 172.21.15.119:0/2618735813 wait complete. 2023-12-07T10:32:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.905+0000 7f3375bff700 1 Processor -- start 2023-12-07T10:32:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.905+0000 7f3375bff700 1 -- start start 2023-12-07T10:32:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.905+0000 7f3375bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3370068ce0 0x7f337019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.905+0000 7f3375bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33701060c0 0x7f337019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.905+0000 7f3375bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3370106a80 0x7f33701999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.905+0000 7f3375bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337019a030 con 0x7f3370068ce0 2023-12-07T10:32:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.905+0000 7f3375bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337019a1a0 con 0x7f33701060c0 2023-12-07T10:32:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.905+0000 7f3375bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337019a360 con 0x7f3370106a80 2023-12-07T10:32:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.905+0000 7f336f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3370068ce0 0x7f337019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.905+0000 7f336ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3370106a80 0x7f33701999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.905+0000 7f336effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33701060c0 0x7f337019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.905+0000 7f336ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3370106a80 0x7f33701999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:48864/0 (socket says 172.21.15.119:48864) 2023-12-07T10:32:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.905+0000 7f336f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3370068ce0 0x7f337019f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42870/0 (socket says 172.21.15.119:42870) 2023-12-07T10:32:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.905+0000 7f336ffff700 1 -- 172.21.15.119:0/412113601 learned_addr learned my addr 172.21.15.119:0/412113601 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:32:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.906+0000 7f336ffff700 1 -- 172.21.15.119:0/412113601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33701060c0 msgr2=0x7f337019fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.906+0000 7f336ffff700 1 --2- 172.21.15.119:0/412113601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33701060c0 0x7f337019fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.906+0000 7f336ffff700 1 -- 172.21.15.119:0/412113601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3370068ce0 msgr2=0x7f337019f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.906+0000 7f336ffff700 1 --2- 172.21.15.119:0/412113601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3370068ce0 0x7f337019f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.906+0000 7f336ffff700 1 -- 172.21.15.119:0/412113601 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33600095f0 con 0x7f3370106a80 2023-12-07T10:32:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.906+0000 7f336effd700 1 --2- 172.21.15.119:0/412113601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33701060c0 0x7f337019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:32:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.906+0000 7f336ffff700 1 --2- 172.21.15.119:0/412113601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3370106a80 0x7f33701999d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f336800ef30 tx=0x7f336800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:32:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.906+0000 7f336cff9700 1 -- 172.21.15.119:0/412113601 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3368018070 con 0x7f3370106a80 2023-12-07T10:32:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.906+0000 7f336cff9700 1 -- 172.21.15.119:0/412113601 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f336800f040 con 0x7f3370106a80 2023-12-07T10:32:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.906+0000 7f3375bff700 1 -- 172.21.15.119:0/412113601 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f337019a5e0 con 0x7f3370106a80 2023-12-07T10:32:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.906+0000 7f336cff9700 1 -- 172.21.15.119:0/412113601 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336800cd00 con 0x7f3370106a80 2023-12-07T10:32:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.906+0000 7f3375bff700 1 -- 172.21.15.119:0/412113601 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33701a6890 con 0x7f3370106a80 2023-12-07T10:32:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.910+0000 7f336cff9700 1 -- 172.21.15.119:0/412113601 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f3368003a00 con 0x7f3370106a80 2023-12-07T10:32:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.911+0000 7f336cff9700 1 --2- 172.21.15.119:0/412113601 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f33580a1d00 0x7f33580a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.911+0000 7f338958db80 1 Processor -- start 2023-12-07T10:32:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.912+0000 7f336cff9700 1 -- 172.21.15.119:0/412113601 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f33680cd840 con 0x7f3370106a80 2023-12-07T10:32:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.912+0000 7f338958db80 1 -- start start 2023-12-07T10:32:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.912+0000 7f336f7fe700 1 --2- 172.21.15.119:0/412113601 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f33580a1d00 0x7f33580a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.912+0000 7f338958db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6e6cf91a0 0x55a6e6cf9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.912+0000 7f338958db80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e6cf4d50 0x55a6e6cf51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.912+0000 7f338958db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6e6cf5710 0x55a6e6d010b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.912+0000 7f338958db80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e6c60760 con 0x55a6e6cf4d50 2023-12-07T10:32:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.913+0000 7f338958db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e6c47740 con 0x55a6e6cf91a0 2023-12-07T10:32:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.913+0000 7f338958db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e6c45060 con 0x55a6e6cf5710 2023-12-07T10:32:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.913+0000 7f336ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6e6cf5710 0x55a6e6d010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.913+0000 7f336ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6e6cf5710 0x55a6e6d010b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:48878/0 (socket says 172.21.15.119:48878) 2023-12-07T10:32:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.913+0000 7f336ffff700 1 -- 172.21.15.119:0/1725251328 learned_addr learned my addr 172.21.15.119:0/1725251328 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:32:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.913+0000 7f336effd700 1 --2- 172.21.15.119:0/1725251328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e6cf4d50 0x55a6e6cf51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.913+0000 7f336ffff700 1 -- 172.21.15.119:0/1725251328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6e6cf91a0 msgr2=0x55a6e6cf9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.913+0000 7f336ffff700 1 --2- 172.21.15.119:0/1725251328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6e6cf91a0 0x55a6e6cf9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.913+0000 7f336ffff700 1 -- 172.21.15.119:0/1725251328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e6cf4d50 msgr2=0x55a6e6cf51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.913+0000 7f336ffff700 1 --2- 172.21.15.119:0/1725251328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e6cf4d50 0x55a6e6cf51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.913+0000 7f336ffff700 1 -- 172.21.15.119:0/1725251328 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6e6abd5d0 con 0x55a6e6cf5710 2023-12-07T10:32:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.913+0000 7f336effd700 1 --2- 172.21.15.119:0/1725251328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e6cf4d50 0x55a6e6cf51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:32:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.913+0000 7f336ffff700 1 --2- 172.21.15.119:0/1725251328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6e6cf5710 0x55a6e6d010b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3368009c10 tx=0x7f3368099e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e3e2a950dc53d94 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:32:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.914+0000 7f3355ffb700 1 -- 172.21.15.119:0/1725251328 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336801b320 con 0x55a6e6cf5710 2023-12-07T10:32:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.914+0000 7f3355ffb700 1 -- 172.21.15.119:0/1725251328 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f336809a960 con 0x55a6e6cf5710 2023-12-07T10:32:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.914+0000 7f336f7fe700 1 --2- 172.21.15.119:0/412113601 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f33580a1d00 0x7f33580a41c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3360006010 tx=0x7f3360005100 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:32:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.914+0000 7f3355ffb700 1 -- 172.21.15.119:0/1725251328 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3368096970 con 0x55a6e6cf5710 2023-12-07T10:32:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.914+0000 7f338958db80 1 -- 172.21.15.119:0/1725251328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6e6cf5710 msgr2=0x55a6e6d010b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.914+0000 7f338958db80 1 --2- 172.21.15.119:0/1725251328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6e6cf5710 0x55a6e6d010b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3368009c10 tx=0x7f3368099e20 comp rx=0 tx=0).stop 2023-12-07T10:32:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.914+0000 7f338958db80 1 -- 172.21.15.119:0/1725251328 shutdown_connections 2023-12-07T10:32:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.914+0000 7f338958db80 1 --2- 172.21.15.119:0/1725251328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6e6cf5710 0x55a6e6d010b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.914+0000 7f338958db80 1 --2- 172.21.15.119:0/1725251328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6e6cf91a0 0x55a6e6cf9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.914+0000 7f338958db80 1 --2- 172.21.15.119:0/1725251328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e6cf4d50 0x55a6e6cf51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.914+0000 7f338958db80 1 -- 172.21.15.119:0/1725251328 >> 172.21.15.119:0/1725251328 conn(0x55a6e6c1b300 msgr2=0x55a6e6ce6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:32:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.915+0000 7f338958db80 1 -- 172.21.15.119:0/1725251328 shutdown_connections 2023-12-07T10:32:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.915+0000 7f338958db80 1 -- 172.21.15.119:0/1725251328 wait complete. 2023-12-07T10:32:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.915+0000 7f338958db80 1 -- start start 2023-12-07T10:32:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.915+0000 7f338958db80 1 Processor -- start 2023-12-07T10:32:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.916+0000 7f338958db80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e6cf4d50 0x55a6e6cfe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.916+0000 7f338958db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6e6cf5710 0x55a6e6cfe900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.916+0000 7f338958db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6e6cf91a0 0x55a6e6cfee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.916+0000 7f338958db80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e6c40950 con 0x55a6e6cf4d50 2023-12-07T10:32:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.916+0000 7f338958db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e6bfa500 con 0x55a6e6cf5710 2023-12-07T10:32:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.916+0000 7f338958db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e6bfa1a0 con 0x55a6e6cf91a0 2023-12-07T10:32:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.916+0000 7f338958db80 10 client.0 _ll_register_callbacks cb 0x55a6e6be18e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:32:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.916+0000 7f336effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6e6cf5710 0x55a6e6cfe900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.916+0000 7f336effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6e6cf5710 0x55a6e6cfe900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:34348/0 (socket says 172.21.15.119:34348) 2023-12-07T10:32:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.916+0000 7f336effd700 1 -- 172.21.15.119:0/281031683 learned_addr learned my addr 172.21.15.119:0/281031683 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:32:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.916+0000 7f336f7fe700 1 --2- 172.21.15.119:0/281031683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e6cf4d50 0x55a6e6cfe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.916+0000 7f336ffff700 1 --2- 172.21.15.119:0/281031683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6e6cf91a0 0x55a6e6cfee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.917+0000 7f336effd700 1 -- 172.21.15.119:0/281031683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6e6cf91a0 msgr2=0x55a6e6cfee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.917+0000 7f336effd700 1 --2- 172.21.15.119:0/281031683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6e6cf91a0 0x55a6e6cfee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.917+0000 7f336effd700 1 -- 172.21.15.119:0/281031683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e6cf4d50 msgr2=0x55a6e6cfe3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.917+0000 7f336effd700 1 --2- 172.21.15.119:0/281031683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e6cf4d50 0x55a6e6cfe3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.917+0000 7f336effd700 1 -- 172.21.15.119:0/281031683 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33701a6890 con 0x55a6e6cf5710 2023-12-07T10:32:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.917+0000 7f336f7fe700 1 --2- 172.21.15.119:0/281031683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e6cf4d50 0x55a6e6cfe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:32:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.917+0000 7f336effd700 1 --2- 172.21.15.119:0/281031683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6e6cf5710 0x55a6e6cfe900 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3364013d30 tx=0x7f33640116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db94507b5e11e7ec server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:32:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.917+0000 7f334effd700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:32:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.917+0000 7f334effd700 1 -- 172.21.15.119:0/281031683 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3364011ee0 con 0x55a6e6cf5710 2023-12-07T10:32:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.917+0000 7f334effd700 1 -- 172.21.15.119:0/281031683 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f3364004a30 con 0x55a6e6cf5710 2023-12-07T10:32:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.917+0000 7f338958db80 1 -- 172.21.15.119:0/281031683 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a6e6cdf6f0 con 0x55a6e6cf5710 2023-12-07T10:32:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.918+0000 7f334effd700 1 -- 172.21.15.119:0/281031683 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3364004bf0 con 0x55a6e6cf5710 2023-12-07T10:32:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.918+0000 7f338958db80 10 client.27758 fetch_fsmap learned FSMap version 59 2023-12-07T10:32:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.918+0000 7f338958db80 1 -- 172.21.15.119:0/281031683 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55a6e6a69d80 con 0x55a6e6cf5710 2023-12-07T10:32:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.918+0000 7f334effd700 1 -- 172.21.15.119:0/281031683 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a6e6cdf6f0 con 0x55a6e6cf5710 2023-12-07T10:32:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.919+0000 7f334effd700 1 -- 172.21.15.119:0/281031683 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 59) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f336401b690 con 0x55a6e6cf5710 2023-12-07T10:32:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.919+0000 7f338958db80 10 client.27758 fetch_fsmap finished waiting for FSMap version 59 2023-12-07T10:32:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.919+0000 7f338958db80 10 client.27758 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:32:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.919+0000 7f338958db80 20 client.27758 populate_metadata read hostname 'smithi119' 2023-12-07T10:32:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.919+0000 7f338958db80 1 --2- 172.21.15.119:0/281031683 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a6e6d0e760 0x55a6e6d10b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.919+0000 7f338958db80 4 client.27758 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:32:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.919+0000 7f338958db80 1 -- 172.21.15.119:0/281031683 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6e6cbb740 con 0x55a6e6d0e760 2023-12-07T10:32:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.919+0000 7f336f7fe700 1 --2- 172.21.15.119:0/281031683 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a6e6d0e760 0x55a6e6d10b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.920+0000 7f334effd700 10 client.27758 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:32:23.922 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.920+0000 7f336f7fe700 1 --2- 172.21.15.119:0/281031683 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a6e6d0e760 0x55a6e6d10b80 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc364ed3b20d1a20 server_cookie=1445a7f00112b1d2 in_seq=0 out_seq=0 2023-12-07T10:32:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.921+0000 7f334effd700 1 -- 172.21.15.119:0/281031683 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a6e6cbb740 con 0x55a6e6d0e760 2023-12-07T10:32:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.921+0000 7f334effd700 10 client.27758 handle_command_reply: tid=0 2023-12-07T10:32:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.921+0000 7f338958db80 1 client.27758 shutdown 2023-12-07T10:32:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.921+0000 7f338958db80 1 -- 172.21.15.119:0/281031683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6e6cf5710 msgr2=0x55a6e6cfe900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.921+0000 7f338958db80 1 --2- 172.21.15.119:0/281031683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6e6cf5710 0x55a6e6cfe900 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3364013d30 tx=0x7f33640116c0 comp rx=0 tx=0).stop 2023-12-07T10:32:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.921+0000 7f338958db80 1 -- 172.21.15.119:0/281031683 shutdown_connections 2023-12-07T10:32:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.921+0000 7f338958db80 1 --2- 172.21.15.119:0/281031683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6e6cf91a0 0x55a6e6cfee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.921+0000 7f338958db80 1 --2- 172.21.15.119:0/281031683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e6cf4d50 0x55a6e6cfe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.921+0000 7f338958db80 1 --2- 172.21.15.119:0/281031683 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a6e6d0e760 0x55a6e6d10b80 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.921+0000 7f338958db80 1 --2- 172.21.15.119:0/281031683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6e6cf5710 0x55a6e6cfe900 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.921+0000 7f338958db80 1 -- 172.21.15.119:0/281031683 >> 172.21.15.119:0/281031683 conn(0x55a6e6c1b300 msgr2=0x55a6e6ce1980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:32:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.921+0000 7f334effd700 0 client.27758 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:32:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.921+0000 7f338958db80 1 -- 172.21.15.119:0/281031683 shutdown_connections 2023-12-07T10:32:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.921+0000 7f338958db80 1 -- 172.21.15.119:0/281031683 wait complete. 2023-12-07T10:32:23.925 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.921+0000 7f338958db80 20 client.27758 trim_cache size 0 max 16384 2023-12-07T10:32:23.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.940+0000 7f338958db80 1 Processor -- start 2023-12-07T10:32:23.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.940+0000 7f338958db80 1 -- start start 2023-12-07T10:32:23.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.941+0000 7f338958db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6e6cf4d50 0x55a6e6cf5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:23.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.941+0000 7f338958db80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e6cf5710 0x55a6e6cf6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:23.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.941+0000 7f338958db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6e6cf91a0 0x55a6e6cfbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:23.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.941+0000 7f338958db80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e6abf8c0 con 0x55a6e6cf5710 2023-12-07T10:32:23.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.941+0000 7f338958db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e6ce8a60 con 0x55a6e6cf91a0 2023-12-07T10:32:23.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.941+0000 7f338958db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e6ce84a0 con 0x55a6e6cf4d50 2023-12-07T10:32:23.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.941+0000 7f336f7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6e6cf4d50 0x55a6e6cf5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:23.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.941+0000 7f336ffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6e6cf91a0 0x55a6e6cfbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:23.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.941+0000 7f336effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e6cf5710 0x55a6e6cf6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:23.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.941+0000 7f336f7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6e6cf4d50 0x55a6e6cf5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:48890/0 (socket says 172.21.15.119:48890) 2023-12-07T10:32:23.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.941+0000 7f336f7fe700 1 -- 172.21.15.119:0/3344401390 learned_addr learned my addr 172.21.15.119:0/3344401390 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:32:23.944 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.941+0000 7f336f7fe700 1 -- 172.21.15.119:0/3344401390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6e6cf91a0 msgr2=0x55a6e6cfbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:23.944 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.941+0000 7f336f7fe700 1 --2- 172.21.15.119:0/3344401390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6e6cf91a0 0x55a6e6cfbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:23.944 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.941+0000 7f336f7fe700 1 -- 172.21.15.119:0/3344401390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e6cf5710 msgr2=0x55a6e6cf6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:23.944 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.941+0000 7f336f7fe700 1 --2- 172.21.15.119:0/3344401390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e6cf5710 0x55a6e6cf6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:23.944 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.941+0000 7f336f7fe700 1 -- 172.21.15.119:0/3344401390 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6e6a52c00 con 0x55a6e6cf4d50 2023-12-07T10:32:23.944 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.941+0000 7f336ffff700 1 --2- 172.21.15.119:0/3344401390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6e6cf91a0 0x55a6e6cfbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:32:23.944 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.942+0000 7f336f7fe700 1 --2- 172.21.15.119:0/3344401390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6e6cf4d50 0x55a6e6cf5130 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3360000c00 tx=0x7f3360023000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23e533cd29c38860 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:32:23.944 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.942+0000 7f33567fc700 1 -- 172.21.15.119:0/3344401390 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336000b8d0 con 0x55a6e6cf4d50 2023-12-07T10:32:23.944 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.942+0000 7f33567fc700 1 -- 172.21.15.119:0/3344401390 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f336001e040 con 0x55a6e6cf4d50 2023-12-07T10:32:23.944 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.942+0000 7f33567fc700 1 -- 172.21.15.119:0/3344401390 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336000fe60 con 0x55a6e6cf4d50 2023-12-07T10:32:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.943+0000 7f338958db80 1 -- 172.21.15.119:0/3344401390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6e6cf4d50 msgr2=0x55a6e6cf5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.943+0000 7f338958db80 1 --2- 172.21.15.119:0/3344401390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6e6cf4d50 0x55a6e6cf5130 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3360000c00 tx=0x7f3360023000 comp rx=0 tx=0).stop 2023-12-07T10:32:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.943+0000 7f338958db80 1 -- 172.21.15.119:0/3344401390 shutdown_connections 2023-12-07T10:32:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.943+0000 7f338958db80 1 --2- 172.21.15.119:0/3344401390 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6e6cf91a0 0x55a6e6cfbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.943+0000 7f338958db80 1 --2- 172.21.15.119:0/3344401390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6e6cf4d50 0x55a6e6cf5130 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.943+0000 7f338958db80 1 --2- 172.21.15.119:0/3344401390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e6cf5710 0x55a6e6cf6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.943+0000 7f338958db80 1 -- 172.21.15.119:0/3344401390 >> 172.21.15.119:0/3344401390 conn(0x55a6e6c1b300 msgr2=0x55a6e6ce0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:32:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.943+0000 7f338958db80 1 -- 172.21.15.119:0/3344401390 shutdown_connections 2023-12-07T10:32:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.943+0000 7f338958db80 1 -- 172.21.15.119:0/3344401390 wait complete. 2023-12-07T10:32:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.943+0000 7f338958db80 1 -- start start 2023-12-07T10:32:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.943+0000 7f338958db80 1 Processor -- start 2023-12-07T10:32:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.944+0000 7f338958db80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e6cf4d50 0x55a6e6d01680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.944+0000 7f338958db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6e6cf5710 0x55a6e6d01bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.944+0000 7f338958db80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6e6cf91a0 0x55a6e6d63c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.944+0000 7f338958db80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e6d02460 con 0x55a6e6cf4d50 2023-12-07T10:32:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.944+0000 7f338958db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e6d64190 con 0x55a6e6cf5710 2023-12-07T10:32:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.944+0000 7f338958db80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e6d642d0 con 0x55a6e6cf91a0 2023-12-07T10:32:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.944+0000 7f338958db80 10 client.0 _ll_register_callbacks cb 0x55a6e6be18e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:32:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.944+0000 7f336f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e6cf4d50 0x55a6e6d01680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.944+0000 7f336f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e6cf4d50 0x55a6e6d01680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42900/0 (socket says 172.21.15.119:42900) 2023-12-07T10:32:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.944+0000 7f336f7fe700 1 -- 172.21.15.119:0/3856074036 learned_addr learned my addr 172.21.15.119:0/3856074036 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:32:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.944+0000 7f336ffff700 1 --2- 172.21.15.119:0/3856074036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6e6cf91a0 0x55a6e6d63c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.944+0000 7f336effd700 1 --2- 172.21.15.119:0/3856074036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6e6cf5710 0x55a6e6d01bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.945+0000 7f336f7fe700 1 -- 172.21.15.119:0/3856074036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6e6cf91a0 msgr2=0x55a6e6d63c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.945+0000 7f336f7fe700 1 --2- 172.21.15.119:0/3856074036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6e6cf91a0 0x55a6e6d63c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.945+0000 7f336f7fe700 1 -- 172.21.15.119:0/3856074036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6e6cf5710 msgr2=0x55a6e6d01bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.945+0000 7f336f7fe700 1 --2- 172.21.15.119:0/3856074036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6e6cf5710 0x55a6e6d01bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.945+0000 7f336f7fe700 1 -- 172.21.15.119:0/3856074036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6e6abd5d0 con 0x55a6e6cf4d50 2023-12-07T10:32:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.945+0000 7f336effd700 1 --2- 172.21.15.119:0/3856074036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6e6cf5710 0x55a6e6d01bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:32:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.945+0000 7f336f7fe700 1 --2- 172.21.15.119:0/3856074036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e6cf4d50 0x55a6e6d01680 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3360009350 tx=0x7f336000f4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fddf45c0a34bd0d server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:32:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.945+0000 7f334effd700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:32:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.945+0000 7f334effd700 1 -- 172.21.15.119:0/3856074036 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336000fcb0 con 0x55a6e6cf4d50 2023-12-07T10:32:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.945+0000 7f334effd700 1 -- 172.21.15.119:0/3856074036 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f336000fe10 con 0x55a6e6cf4d50 2023-12-07T10:32:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.945+0000 7f338958db80 1 -- 172.21.15.119:0/3856074036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f336809a960 con 0x55a6e6cf4d50 2023-12-07T10:32:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.946+0000 7f334effd700 1 -- 172.21.15.119:0/3856074036 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f336000b8d0 con 0x55a6e6cf4d50 2023-12-07T10:32:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.946+0000 7f338958db80 10 client.17721 fetch_fsmap learned FSMap version 59 2023-12-07T10:32:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.946+0000 7f338958db80 1 -- 172.21.15.119:0/3856074036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x7f336002c040 con 0x55a6e6cf4d50 2023-12-07T10:32:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.946+0000 7f334effd700 1 -- 172.21.15.119:0/3856074036 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f336809a960 con 0x55a6e6cf4d50 2023-12-07T10:32:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.946+0000 7f334effd700 1 -- 172.21.15.119:0/3856074036 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 59) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3360031430 con 0x55a6e6cf4d50 2023-12-07T10:32:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.946+0000 7f338958db80 10 client.17721 fetch_fsmap finished waiting for FSMap version 59 2023-12-07T10:32:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.946+0000 7f338958db80 10 client.17721 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:32:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.946+0000 7f338958db80 20 client.17721 populate_metadata read hostname 'smithi119' 2023-12-07T10:32:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.946+0000 7f338958db80 1 --2- 172.21.15.119:0/3856074036 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a6e6d65520 0x55a6e6d67910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.946+0000 7f338958db80 4 client.17721 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:32:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.946+0000 7f338958db80 1 -- 172.21.15.119:0/3856074036 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a6e6c06f70 con 0x55a6e6d65520 2023-12-07T10:32:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.947+0000 7f336effd700 1 --2- 172.21.15.119:0/3856074036 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a6e6d65520 0x55a6e6d67910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.947+0000 7f336effd700 1 --2- 172.21.15.119:0/3856074036 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a6e6d65520 0x55a6e6d67910 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=398385426d364b04 server_cookie=81dc5c694d008db in_seq=0 out_seq=0 2023-12-07T10:32:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:23.947+0000 7f334effd700 10 client.17721 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:32:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.043+0000 7f334effd700 1 -- 172.21.15.119:0/3856074036 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a6e6c06f70 con 0x55a6e6d65520 2023-12-07T10:32:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.043+0000 7f334effd700 10 client.17721 handle_command_reply: tid=0 2023-12-07T10:32:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.043+0000 7f338958db80 1 client.17721 shutdown 2023-12-07T10:32:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.043+0000 7f338958db80 1 -- 172.21.15.119:0/3856074036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e6cf4d50 msgr2=0x55a6e6d01680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.043+0000 7f338958db80 1 --2- 172.21.15.119:0/3856074036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e6cf4d50 0x55a6e6d01680 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3360009350 tx=0x7f336000f4e0 comp rx=0 tx=0).stop 2023-12-07T10:32:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.043+0000 7f338958db80 1 -- 172.21.15.119:0/3856074036 shutdown_connections 2023-12-07T10:32:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.043+0000 7f338958db80 1 --2- 172.21.15.119:0/3856074036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6e6cf91a0 0x55a6e6d63c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.043+0000 7f338958db80 1 --2- 172.21.15.119:0/3856074036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a6e6cf4d50 0x55a6e6d01680 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.043+0000 7f338958db80 1 --2- 172.21.15.119:0/3856074036 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55a6e6d65520 0x55a6e6d67910 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.043+0000 7f338958db80 1 --2- 172.21.15.119:0/3856074036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55a6e6cf5710 0x55a6e6d01bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.044+0000 7f338958db80 1 -- 172.21.15.119:0/3856074036 >> 172.21.15.119:0/3856074036 conn(0x55a6e6c1b300 msgr2=0x55a6e6cfbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:32:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.044+0000 7f334effd700 0 client.17721 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:32:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.044+0000 7f338958db80 1 -- 172.21.15.119:0/3856074036 shutdown_connections 2023-12-07T10:32:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.044+0000 7f338958db80 1 -- 172.21.15.119:0/3856074036 wait complete. 2023-12-07T10:32:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.044+0000 7f338958db80 20 client.17721 trim_cache size 0 max 16384 2023-12-07T10:32:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:32:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:32:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:32:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:32:24.047 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.045+0000 7f3375bff700 1 -- 172.21.15.119:0/412113601 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f33580a1d00 msgr2=0x7f33580a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:24.047 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.045+0000 7f3375bff700 1 --2- 172.21.15.119:0/412113601 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f33580a1d00 0x7f33580a41c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3360006010 tx=0x7f3360005100 comp rx=0 tx=0).stop 2023-12-07T10:32:24.047 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.045+0000 7f3375bff700 1 -- 172.21.15.119:0/412113601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3370106a80 msgr2=0x7f33701999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:24.047 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.045+0000 7f3375bff700 1 --2- 172.21.15.119:0/412113601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3370106a80 0x7f33701999d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f336800ef30 tx=0x7f336800c3a0 comp rx=0 tx=0).stop 2023-12-07T10:32:24.047 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.045+0000 7f3375bff700 1 -- 172.21.15.119:0/412113601 shutdown_connections 2023-12-07T10:32:24.047 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.045+0000 7f3375bff700 1 --2- 172.21.15.119:0/412113601 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f33580a1d00 0x7f33580a41c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.045+0000 7f3375bff700 1 --2- 172.21.15.119:0/412113601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3370106a80 0x7f33701999d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.045+0000 7f3375bff700 1 --2- 172.21.15.119:0/412113601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3370068ce0 0x7f337019f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.045+0000 7f3375bff700 1 --2- 172.21.15.119:0/412113601 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33701060c0 0x7f337019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.045+0000 7f3375bff700 1 -- 172.21.15.119:0/412113601 >> 172.21.15.119:0/412113601 conn(0x7f33700775b0 msgr2=0x7f337010c990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:32:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.045+0000 7f3375bff700 1 -- 172.21.15.119:0/412113601 shutdown_connections 2023-12-07T10:32:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:24.045+0000 7f3375bff700 1 -- 172.21.15.119:0/412113601 wait complete. 2023-12-07T10:32:24.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:24 smithi119 ceph-mon[97908]: pgmap v374: 129 pgs: 129 active+clean; 685 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 183 op/s 2023-12-07T10:32:24.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:24 smithi159 ceph-mon[134726]: pgmap v374: 129 pgs: 129 active+clean; 685 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 183 op/s 2023-12-07T10:32:24.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:24 smithi153 ceph-mon[133625]: pgmap v374: 129 pgs: 129 active+clean; 685 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 183 op/s 2023-12-07T10:32:24.578 INFO:tasks.workunit.client.0.smithi119.stdout: 1 879914 7.24 MB/sec execute 517 sec latency 48.147 ms 2023-12-07T10:32:25.578 INFO:tasks.workunit.client.0.smithi119.stdout: 1 881071 7.24 MB/sec execute 518 sec latency 24.314 ms 2023-12-07T10:32:26.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:26 smithi119 ceph-mon[97908]: pgmap v375: 129 pgs: 129 active+clean; 685 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 169 op/s 2023-12-07T10:32:26.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:26 smithi159 ceph-mon[134726]: pgmap v375: 129 pgs: 129 active+clean; 685 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 169 op/s 2023-12-07T10:32:26.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:26 smithi153 ceph-mon[133625]: pgmap v375: 129 pgs: 129 active+clean; 685 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 169 op/s 2023-12-07T10:32:26.578 INFO:tasks.workunit.client.0.smithi119.stdout: 1 882812 7.24 MB/sec execute 519 sec latency 16.695 ms 2023-12-07T10:32:27.579 INFO:tasks.workunit.client.0.smithi119.stdout: 1 884317 7.25 MB/sec execute 520 sec latency 12.899 ms 2023-12-07T10:32:28.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:28 smithi119 ceph-mon[97908]: pgmap v376: 129 pgs: 129 active+clean; 690 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 292 op/s 2023-12-07T10:32:28.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:28 smithi159 ceph-mon[134726]: pgmap v376: 129 pgs: 129 active+clean; 690 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 292 op/s 2023-12-07T10:32:28.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:28 smithi153 ceph-mon[133625]: pgmap v376: 129 pgs: 129 active+clean; 690 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 292 op/s 2023-12-07T10:32:28.578 INFO:tasks.workunit.client.0.smithi119.stdout: 1 885505 7.25 MB/sec execute 521 sec latency 14.794 ms 2023-12-07T10:32:29.579 INFO:tasks.workunit.client.0.smithi119.stdout: 1 887368 7.25 MB/sec execute 522 sec latency 17.562 ms 2023-12-07T10:32:30.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:30 smithi119 ceph-mon[97908]: pgmap v377: 129 pgs: 129 active+clean; 690 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 210 op/s 2023-12-07T10:32:30.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:30 smithi159 ceph-mon[134726]: pgmap v377: 129 pgs: 129 active+clean; 690 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 210 op/s 2023-12-07T10:32:30.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:30 smithi153 ceph-mon[133625]: pgmap v377: 129 pgs: 129 active+clean; 690 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 210 op/s 2023-12-07T10:32:30.579 INFO:tasks.workunit.client.0.smithi119.stdout: 1 888260 7.25 MB/sec execute 523 sec latency 23.414 ms 2023-12-07T10:32:31.579 INFO:tasks.workunit.client.0.smithi119.stdout: 1 889904 7.25 MB/sec execute 524 sec latency 19.614 ms 2023-12-07T10:32:32.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:31 smithi119 ceph-mon[97908]: pgmap v378: 129 pgs: 129 active+clean; 698 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 266 op/s 2023-12-07T10:32:32.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:31 smithi159 ceph-mon[134726]: pgmap v378: 129 pgs: 129 active+clean; 698 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 266 op/s 2023-12-07T10:32:32.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:31 smithi153 ceph-mon[133625]: pgmap v378: 129 pgs: 129 active+clean; 698 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 266 op/s 2023-12-07T10:32:32.579 INFO:tasks.workunit.client.0.smithi119.stdout: 1 891396 7.25 MB/sec execute 525 sec latency 12.814 ms 2023-12-07T10:32:33.579 INFO:tasks.workunit.client.0.smithi119.stdout: 1 892534 7.25 MB/sec execute 526 sec latency 19.564 ms 2023-12-07T10:32:34.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:34 smithi119 ceph-mon[97908]: pgmap v379: 129 pgs: 129 active+clean; 700 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 226 op/s 2023-12-07T10:32:34.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:34 smithi159 ceph-mon[134726]: pgmap v379: 129 pgs: 129 active+clean; 700 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 226 op/s 2023-12-07T10:32:34.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:34 smithi153 ceph-mon[133625]: pgmap v379: 129 pgs: 129 active+clean; 700 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 226 op/s 2023-12-07T10:32:34.579 INFO:tasks.workunit.client.0.smithi119.stdout: 1 894447 7.26 MB/sec execute 527 sec latency 19.701 ms 2023-12-07T10:32:35.579 INFO:tasks.workunit.client.0.smithi119.stdout: 1 895367 7.25 MB/sec execute 528 sec latency 23.563 ms 2023-12-07T10:32:36.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:36 smithi119 ceph-mon[97908]: pgmap v380: 129 pgs: 129 active+clean; 700 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 201 op/s 2023-12-07T10:32:36.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:36 smithi159 ceph-mon[134726]: pgmap v380: 129 pgs: 129 active+clean; 700 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 201 op/s 2023-12-07T10:32:36.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:36 smithi153 ceph-mon[133625]: pgmap v380: 129 pgs: 129 active+clean; 700 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 201 op/s 2023-12-07T10:32:36.579 INFO:tasks.workunit.client.0.smithi119.stdout: 1 897066 7.25 MB/sec execute 529 sec latency 19.751 ms 2023-12-07T10:32:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.219+0000 7fe297fff700 1 -- 172.21.15.119:0/3801924433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe298101f50 msgr2=0x7fe298105f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.219+0000 7fe297fff700 1 --2- 172.21.15.119:0/3801924433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe298101f50 0x7fe298105f90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009990 tx=0x7fe28c009ca0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.220+0000 7fe297fff700 1 -- 172.21.15.119:0/3801924433 shutdown_connections 2023-12-07T10:32:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.220+0000 7fe297fff700 1 --2- 172.21.15.119:0/3801924433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2981066c0 0x7fe29810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.220+0000 7fe297fff700 1 --2- 172.21.15.119:0/3801924433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2981015a0 0x7fe298101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.220+0000 7fe297fff700 1 --2- 172.21.15.119:0/3801924433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe298101f50 0x7fe298105f90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.220+0000 7fe297fff700 1 -- 172.21.15.119:0/3801924433 >> 172.21.15.119:0/3801924433 conn(0x7fe2980fce40 msgr2=0x7fe2980ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:32:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.220+0000 7fe297fff700 1 -- 172.21.15.119:0/3801924433 shutdown_connections 2023-12-07T10:32:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.220+0000 7fe297fff700 1 -- 172.21.15.119:0/3801924433 wait complete. 2023-12-07T10:32:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.221+0000 7fe297fff700 1 Processor -- start 2023-12-07T10:32:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.221+0000 7fe297fff700 1 -- start start 2023-12-07T10:32:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.222+0000 7fe297fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2981015a0 0x7fe298199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.222+0000 7fe297fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2981066c0 0x7fe298199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.222+0000 7fe297fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe29819e340 0x7fe29819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.222+0000 7fe297fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29819ed30 con 0x7fe2981066c0 2023-12-07T10:32:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.222+0000 7fe297fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29819eea0 con 0x7fe2981015a0 2023-12-07T10:32:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.222+0000 7fe297fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29819f060 con 0x7fe29819e340 2023-12-07T10:32:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.222+0000 7fe296ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2981015a0 0x7fe298199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.222+0000 7fe296ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2981015a0 0x7fe298199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:35376/0 (socket says 172.21.15.119:35376) 2023-12-07T10:32:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.222+0000 7fe296ffd700 1 -- 172.21.15.119:0/4061574597 learned_addr learned my addr 172.21.15.119:0/4061574597 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:32:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.222+0000 7fe2977fe700 1 --2- 172.21.15.119:0/4061574597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe29819e340 0x7fe29819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.222+0000 7fe2967fc700 1 --2- 172.21.15.119:0/4061574597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2981066c0 0x7fe298199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.222+0000 7fe296ffd700 1 -- 172.21.15.119:0/4061574597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe29819e340 msgr2=0x7fe29819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.222+0000 7fe296ffd700 1 --2- 172.21.15.119:0/4061574597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe29819e340 0x7fe29819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.222+0000 7fe296ffd700 1 -- 172.21.15.119:0/4061574597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2981066c0 msgr2=0x7fe298199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.223+0000 7fe296ffd700 1 --2- 172.21.15.119:0/4061574597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2981066c0 0x7fe298199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.223+0000 7fe296ffd700 1 -- 172.21.15.119:0/4061574597 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28c0095f0 con 0x7fe2981015a0 2023-12-07T10:32:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.223+0000 7fe2967fc700 1 --2- 172.21.15.119:0/4061574597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2981066c0 0x7fe298199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:32:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.223+0000 7fe296ffd700 1 --2- 172.21.15.119:0/4061574597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2981015a0 0x7fe298199a30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe28800e9a0 tx=0x7fe28800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:32:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.223+0000 7fe27ffff700 1 -- 172.21.15.119:0/4061574597 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe288009790 con 0x7fe2981015a0 2023-12-07T10:32:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.223+0000 7fe27ffff700 1 -- 172.21.15.119:0/4061574597 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe28800cbd0 con 0x7fe2981015a0 2023-12-07T10:32:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.223+0000 7fe297fff700 1 -- 172.21.15.119:0/4061574597 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2981a65f0 con 0x7fe2981015a0 2023-12-07T10:32:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.223+0000 7fe27ffff700 1 -- 172.21.15.119:0/4061574597 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe288018680 con 0x7fe2981015a0 2023-12-07T10:32:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.223+0000 7fe297fff700 1 -- 172.21.15.119:0/4061574597 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2981a6b20 con 0x7fe2981015a0 2023-12-07T10:32:37.227 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.227+0000 7fe27ffff700 1 -- 172.21.15.119:0/4061574597 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fe28800cd40 con 0x7fe2981015a0 2023-12-07T10:32:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.228+0000 7fe2b1146b80 1 Processor -- start 2023-12-07T10:32:37.228 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.228+0000 7fe2b1146b80 1 -- start start 2023-12-07T10:32:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.228+0000 7fe2b1146b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55daf707eca0 0x55daf707f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.228+0000 7fe2b1146b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daf707f5c0 0x55daf7079540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.229+0000 7fe2b1146b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daf7079a80 0x55daf7079f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.229+0000 7fe2b1146b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf6fed760 con 0x55daf707f5c0 2023-12-07T10:32:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.229+0000 7fe2b1146b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf6fd4740 con 0x55daf7079a80 2023-12-07T10:32:37.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.229+0000 7fe2b1146b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf6fd2060 con 0x55daf707eca0 2023-12-07T10:32:37.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.229+0000 7fe2977fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55daf707eca0 0x55daf707f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:37.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.229+0000 7fe2977fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55daf707eca0 0x55daf707f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:56724/0 (socket says 172.21.15.119:56724) 2023-12-07T10:32:37.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.229+0000 7fe2977fe700 1 -- 172.21.15.119:0/2933644572 learned_addr learned my addr 172.21.15.119:0/2933644572 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:32:37.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.229+0000 7fe27ffff700 1 --2- 172.21.15.119:0/4061574597 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe2800a1d00 0x7fe2800a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:37.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.229+0000 7fe296ffd700 1 --2- 172.21.15.119:0/2933644572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daf707f5c0 0x55daf7079540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:37.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.229+0000 7fe2967fc700 1 --2- 172.21.15.119:0/2933644572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daf7079a80 0x55daf7079f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:37.232 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.229+0000 7fe2977fe700 1 --2- 172.21.15.119:0/4061574597 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe2800a1d00 0x7fe2800a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:37.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.229+0000 7fe2977fe700 1 -- 172.21.15.119:0/2933644572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daf7079a80 msgr2=0x55daf7079f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:37.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.229+0000 7fe2977fe700 1 --2- 172.21.15.119:0/2933644572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daf7079a80 0x55daf7079f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.229+0000 7fe2977fe700 1 -- 172.21.15.119:0/2933644572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daf707f5c0 msgr2=0x55daf7079540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:37.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.230+0000 7fe2977fe700 1 --2- 172.21.15.119:0/2933644572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daf707f5c0 0x55daf7079540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.230+0000 7fe2977fe700 1 -- 172.21.15.119:0/2933644572 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55daf6e4a5d0 con 0x55daf707eca0 2023-12-07T10:32:37.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.230+0000 7fe27ffff700 1 -- 172.21.15.119:0/4061574597 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fe288014070 con 0x7fe2981015a0 2023-12-07T10:32:37.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.230+0000 7fe2977fe700 1 --2- 172.21.15.119:0/2933644572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55daf707eca0 0x55daf707f080 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe290007a40 tx=0x7fe290007d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcd63ab775fc528e server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:32:37.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.230+0000 7fe27d7fa700 1 -- 172.21.15.119:0/2933644572 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe290027070 con 0x55daf707eca0 2023-12-07T10:32:37.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.230+0000 7fe27d7fa700 1 -- 172.21.15.119:0/2933644572 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe29001e070 con 0x55daf707eca0 2023-12-07T10:32:37.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.230+0000 7fe27d7fa700 1 -- 172.21.15.119:0/2933644572 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29000e070 con 0x55daf707eca0 2023-12-07T10:32:37.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.231+0000 7fe2b1146b80 1 -- 172.21.15.119:0/2933644572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55daf707eca0 msgr2=0x55daf707f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.231+0000 7fe2977fe700 1 --2- 172.21.15.119:0/4061574597 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe2800a1d00 0x7fe2800a41c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe290022000 tx=0x7fe29001e000 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:32:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.231+0000 7fe2b1146b80 1 --2- 172.21.15.119:0/2933644572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55daf707eca0 0x55daf707f080 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe290007a40 tx=0x7fe290007d50 comp rx=0 tx=0).stop 2023-12-07T10:32:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.231+0000 7fe2b1146b80 1 -- 172.21.15.119:0/2933644572 shutdown_connections 2023-12-07T10:32:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.231+0000 7fe2b1146b80 1 --2- 172.21.15.119:0/2933644572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daf7079a80 0x55daf7079f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.231+0000 7fe2b1146b80 1 --2- 172.21.15.119:0/2933644572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55daf707eca0 0x55daf707f080 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.231+0000 7fe2b1146b80 1 --2- 172.21.15.119:0/2933644572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daf707f5c0 0x55daf7079540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.231+0000 7fe2b1146b80 1 -- 172.21.15.119:0/2933644572 >> 172.21.15.119:0/2933644572 conn(0x55daf6fa8300 msgr2=0x55daf7073bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:32:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.231+0000 7fe2b1146b80 1 -- 172.21.15.119:0/2933644572 shutdown_connections 2023-12-07T10:32:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.231+0000 7fe2b1146b80 1 -- 172.21.15.119:0/2933644572 wait complete. 2023-12-07T10:32:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.232+0000 7fe2b1146b80 1 -- start start 2023-12-07T10:32:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.232+0000 7fe2b1146b80 1 Processor -- start 2023-12-07T10:32:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.233+0000 7fe2b1146b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daf7079a80 0x55daf7085590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.233+0000 7fe2b1146b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55daf707eca0 0x55daf7082880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.233+0000 7fe2b1146b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daf707f5c0 0x55daf7082fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.233+0000 7fe2b1146b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf6fcd950 con 0x55daf707f5c0 2023-12-07T10:32:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.233+0000 7fe2b1146b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf6f87500 con 0x55daf7079a80 2023-12-07T10:32:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.233+0000 7fe2b1146b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf6f871a0 con 0x55daf707eca0 2023-12-07T10:32:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.233+0000 7fe2b1146b80 10 client.0 _ll_register_callbacks cb 0x55daf6f6e8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:32:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.233+0000 7fe296ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daf7079a80 0x55daf7085590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.233+0000 7fe296ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daf7079a80 0x55daf7085590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:35408/0 (socket says 172.21.15.119:35408) 2023-12-07T10:32:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.233+0000 7fe296ffd700 1 -- 172.21.15.119:0/1498278995 learned_addr learned my addr 172.21.15.119:0/1498278995 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:32:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.233+0000 7fe2967fc700 1 --2- 172.21.15.119:0/1498278995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55daf707eca0 0x55daf7082880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.233+0000 7fe2977fe700 1 --2- 172.21.15.119:0/1498278995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daf707f5c0 0x55daf7082fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.233+0000 7fe296ffd700 1 -- 172.21.15.119:0/1498278995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55daf707eca0 msgr2=0x55daf7082880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.233+0000 7fe296ffd700 1 --2- 172.21.15.119:0/1498278995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55daf707eca0 0x55daf7082880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.233+0000 7fe296ffd700 1 -- 172.21.15.119:0/1498278995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daf707f5c0 msgr2=0x55daf7082fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.233+0000 7fe296ffd700 1 --2- 172.21.15.119:0/1498278995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daf707f5c0 0x55daf7082fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.233+0000 7fe296ffd700 1 -- 172.21.15.119:0/1498278995 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe290007620 con 0x55daf7079a80 2023-12-07T10:32:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.234+0000 7fe2977fe700 1 --2- 172.21.15.119:0/1498278995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daf707f5c0 0x55daf7082fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:32:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.234+0000 7fe296ffd700 1 --2- 172.21.15.119:0/1498278995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daf7079a80 0x55daf7085590 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe288000c00 tx=0x7fe2880cb900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b951e2a678a982e9 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:32:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.234+0000 7fe27aff5700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:32:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.234+0000 7fe27aff5700 1 -- 172.21.15.119:0/1498278995 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2880a2af0 con 0x55daf7079a80 2023-12-07T10:32:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.234+0000 7fe27aff5700 1 -- 172.21.15.119:0/1498278995 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fe2880a2d80 con 0x55daf7079a80 2023-12-07T10:32:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.234+0000 7fe2b1146b80 1 -- 172.21.15.119:0/1498278995 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55daf706c6f0 con 0x55daf7079a80 2023-12-07T10:32:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.234+0000 7fe27aff5700 1 -- 172.21.15.119:0/1498278995 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe28809a9f0 con 0x55daf7079a80 2023-12-07T10:32:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.234+0000 7fe2b1146b80 10 client.27776 fetch_fsmap learned FSMap version 59 2023-12-07T10:32:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.234+0000 7fe2b1146b80 1 -- 172.21.15.119:0/1498278995 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55daf6df6d80 con 0x55daf7079a80 2023-12-07T10:32:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.234+0000 7fe27aff5700 1 -- 172.21.15.119:0/1498278995 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2880b2070 con 0x55daf7079a80 2023-12-07T10:32:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.235+0000 7fe27aff5700 1 -- 172.21.15.119:0/1498278995 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 59) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe28809bd20 con 0x55daf7079a80 2023-12-07T10:32:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.235+0000 7fe2b1146b80 10 client.27776 fetch_fsmap finished waiting for FSMap version 59 2023-12-07T10:32:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.235+0000 7fe2b1146b80 10 client.27776 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:32:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.235+0000 7fe2b1146b80 20 client.27776 populate_metadata read hostname 'smithi119' 2023-12-07T10:32:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.235+0000 7fe2b1146b80 1 --2- 172.21.15.119:0/1498278995 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55daf7092fc0 0x55daf70953e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.235+0000 7fe2b1146b80 4 client.27776 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:32:37.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.235+0000 7fe2b1146b80 1 -- 172.21.15.119:0/1498278995 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55daf7048740 con 0x55daf7092fc0 2023-12-07T10:32:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.235+0000 7fe2967fc700 1 --2- 172.21.15.119:0/1498278995 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55daf7092fc0 0x55daf70953e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.235+0000 7fe2967fc700 1 --2- 172.21.15.119:0/1498278995 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55daf7092fc0 0x55daf70953e0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bcdbab1f001bdaf server_cookie=e9815383ad22302a in_seq=0 out_seq=0 2023-12-07T10:32:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.235+0000 7fe27aff5700 10 client.27776 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:32:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.236+0000 7fe27aff5700 1 -- 172.21.15.119:0/1498278995 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55daf7048740 con 0x55daf7092fc0 2023-12-07T10:32:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.236+0000 7fe27aff5700 10 client.27776 handle_command_reply: tid=0 2023-12-07T10:32:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.236+0000 7fe2b1146b80 1 client.27776 shutdown 2023-12-07T10:32:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.237+0000 7fe2b1146b80 1 -- 172.21.15.119:0/1498278995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daf7079a80 msgr2=0x55daf7085590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.237+0000 7fe2b1146b80 1 --2- 172.21.15.119:0/1498278995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daf7079a80 0x55daf7085590 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe288000c00 tx=0x7fe2880cb900 comp rx=0 tx=0).stop 2023-12-07T10:32:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.237+0000 7fe2b1146b80 1 -- 172.21.15.119:0/1498278995 shutdown_connections 2023-12-07T10:32:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.237+0000 7fe2b1146b80 1 --2- 172.21.15.119:0/1498278995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daf707f5c0 0x55daf7082fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.237+0000 7fe2b1146b80 1 --2- 172.21.15.119:0/1498278995 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55daf7092fc0 0x55daf70953e0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.237+0000 7fe2b1146b80 1 --2- 172.21.15.119:0/1498278995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daf7079a80 0x55daf7085590 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.242 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.237+0000 7fe2b1146b80 1 --2- 172.21.15.119:0/1498278995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55daf707eca0 0x55daf7082880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.242 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.237+0000 7fe2b1146b80 1 -- 172.21.15.119:0/1498278995 >> 172.21.15.119:0/1498278995 conn(0x55daf6fa8300 msgr2=0x55daf706e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:32:37.242 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.237+0000 7fe27aff5700 0 client.27776 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:32:37.242 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.237+0000 7fe2b1146b80 1 -- 172.21.15.119:0/1498278995 shutdown_connections 2023-12-07T10:32:37.242 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.237+0000 7fe2b1146b80 1 -- 172.21.15.119:0/1498278995 wait complete. 2023-12-07T10:32:37.242 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.237+0000 7fe2b1146b80 20 client.27776 trim_cache size 0 max 16384 2023-12-07T10:32:37.256 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.255+0000 7fe2b1146b80 1 Processor -- start 2023-12-07T10:32:37.256 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.255+0000 7fe2b1146b80 1 -- start start 2023-12-07T10:32:37.256 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.255+0000 7fe2b1146b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daf7079a80 0x55daf7079540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:37.256 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.255+0000 7fe2b1146b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55daf707eca0 0x55daf708a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.255+0000 7fe2b1146b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daf707f5c0 0x55daf708a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.255+0000 7fe2b1146b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf6e4c8c0 con 0x55daf707f5c0 2023-12-07T10:32:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.255+0000 7fe2b1146b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf7048740 con 0x55daf7079a80 2023-12-07T10:32:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.255+0000 7fe2b1146b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf7075470 con 0x55daf707eca0 2023-12-07T10:32:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.255+0000 7fe2967fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55daf707eca0 0x55daf708a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.256+0000 7fe2967fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55daf707eca0 0x55daf708a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:56738/0 (socket says 172.21.15.119:56738) 2023-12-07T10:32:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.256+0000 7fe2967fc700 1 -- 172.21.15.119:0/379140538 learned_addr learned my addr 172.21.15.119:0/379140538 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:32:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.256+0000 7fe296ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daf7079a80 0x55daf7079540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.256+0000 7fe2977fe700 1 --2- 172.21.15.119:0/379140538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daf707f5c0 0x55daf708a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.256+0000 7fe2967fc700 1 -- 172.21.15.119:0/379140538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daf7079a80 msgr2=0x55daf7079540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.256+0000 7fe2967fc700 1 --2- 172.21.15.119:0/379140538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daf7079a80 0x55daf7079540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.256+0000 7fe2967fc700 1 -- 172.21.15.119:0/379140538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daf707f5c0 msgr2=0x55daf708a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.256+0000 7fe2967fc700 1 --2- 172.21.15.119:0/379140538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daf707f5c0 0x55daf708a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.258 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.256+0000 7fe2967fc700 1 -- 172.21.15.119:0/379140538 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55daf6ddfc00 con 0x55daf707eca0 2023-12-07T10:32:37.258 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.256+0000 7fe2977fe700 1 --2- 172.21.15.119:0/379140538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daf707f5c0 0x55daf708a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:32:37.258 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.256+0000 7fe296ffd700 1 --2- 172.21.15.119:0/379140538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daf7079a80 0x55daf7079540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:32:37.258 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.256+0000 7fe2967fc700 1 --2- 172.21.15.119:0/379140538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55daf707eca0 0x55daf708a000 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe28c005850 tx=0x7fe28c005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e77fa32bbb682cab server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:32:37.258 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.256+0000 7fe27dffb700 1 -- 172.21.15.119:0/379140538 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c00b970 con 0x55daf707eca0 2023-12-07T10:32:37.258 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.257+0000 7fe27dffb700 1 -- 172.21.15.119:0/379140538 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe28c01e040 con 0x55daf707eca0 2023-12-07T10:32:37.258 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.257+0000 7fe27dffb700 1 -- 172.21.15.119:0/379140538 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c00fe60 con 0x55daf707eca0 2023-12-07T10:32:37.260 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.257+0000 7fe2b1146b80 1 -- 172.21.15.119:0/379140538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55daf707eca0 msgr2=0x55daf708a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:37.260 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.257+0000 7fe2b1146b80 1 --2- 172.21.15.119:0/379140538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55daf707eca0 0x55daf708a000 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe28c005850 tx=0x7fe28c005a90 comp rx=0 tx=0).stop 2023-12-07T10:32:37.260 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.257+0000 7fe2b1146b80 1 -- 172.21.15.119:0/379140538 shutdown_connections 2023-12-07T10:32:37.260 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.257+0000 7fe2b1146b80 1 --2- 172.21.15.119:0/379140538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daf707f5c0 0x55daf708a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.257+0000 7fe2b1146b80 1 --2- 172.21.15.119:0/379140538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daf7079a80 0x55daf7079540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.257+0000 7fe2b1146b80 1 --2- 172.21.15.119:0/379140538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55daf707eca0 0x55daf708a000 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.257+0000 7fe2b1146b80 1 -- 172.21.15.119:0/379140538 >> 172.21.15.119:0/379140538 conn(0x55daf6fa8300 msgr2=0x55daf7085250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:32:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.258+0000 7fe2b1146b80 1 -- 172.21.15.119:0/379140538 shutdown_connections 2023-12-07T10:32:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.258+0000 7fe2b1146b80 1 -- 172.21.15.119:0/379140538 wait complete. 2023-12-07T10:32:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.258+0000 7fe2b1146b80 1 -- start start 2023-12-07T10:32:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.258+0000 7fe2b1146b80 1 Processor -- start 2023-12-07T10:32:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.259+0000 7fe2b1146b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55daf7079a80 0x55daf7087fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.259+0000 7fe2b1146b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daf707eca0 0x55daf7088520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.259+0000 7fe2b1146b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daf707f5c0 0x55daf70e84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.259+0000 7fe2b1146b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf7088e00 con 0x55daf707f5c0 2023-12-07T10:32:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.259+0000 7fe2b1146b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf70e8a20 con 0x55daf707eca0 2023-12-07T10:32:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.259+0000 7fe2b1146b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daf70e8b60 con 0x55daf7079a80 2023-12-07T10:32:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.259+0000 7fe2b1146b80 10 client.0 _ll_register_callbacks cb 0x55daf6f6e8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:32:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.259+0000 7fe296ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55daf7079a80 0x55daf7087fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.259+0000 7fe296ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55daf7079a80 0x55daf7087fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:56746/0 (socket says 172.21.15.119:56746) 2023-12-07T10:32:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.259+0000 7fe296ffd700 1 -- 172.21.15.119:0/3376360433 learned_addr learned my addr 172.21.15.119:0/3376360433 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:32:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.259+0000 7fe2967fc700 1 --2- 172.21.15.119:0/3376360433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daf707eca0 0x55daf7088520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.259+0000 7fe2977fe700 1 --2- 172.21.15.119:0/3376360433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daf707f5c0 0x55daf70e84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.260+0000 7fe296ffd700 1 -- 172.21.15.119:0/3376360433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daf707eca0 msgr2=0x55daf7088520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.260+0000 7fe296ffd700 1 --2- 172.21.15.119:0/3376360433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daf707eca0 0x55daf7088520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.260+0000 7fe296ffd700 1 -- 172.21.15.119:0/3376360433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daf707f5c0 msgr2=0x55daf70e84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.260+0000 7fe296ffd700 1 --2- 172.21.15.119:0/3376360433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daf707f5c0 0x55daf70e84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.260+0000 7fe296ffd700 1 -- 172.21.15.119:0/3376360433 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55daf6df6d80 con 0x55daf7079a80 2023-12-07T10:32:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.260+0000 7fe296ffd700 1 --2- 172.21.15.119:0/3376360433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55daf7079a80 0x55daf7087fe0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe288097460 tx=0x7fe288099f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2537eb695fa170d3 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:32:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.260+0000 7fe27aff5700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:32:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.260+0000 7fe27aff5700 1 -- 172.21.15.119:0/3376360433 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2880a2af0 con 0x55daf7079a80 2023-12-07T10:32:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.260+0000 7fe2b1146b80 1 -- 172.21.15.119:0/3376360433 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe29001e070 con 0x55daf7079a80 2023-12-07T10:32:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.260+0000 7fe27aff5700 1 -- 172.21.15.119:0/3376360433 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fe2880a2d80 con 0x55daf7079a80 2023-12-07T10:32:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.261+0000 7fe27aff5700 1 -- 172.21.15.119:0/3376360433 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe28809a0d0 con 0x55daf7079a80 2023-12-07T10:32:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.261+0000 7fe2b1146b80 10 client.27766 fetch_fsmap learned FSMap version 59 2023-12-07T10:32:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.261+0000 7fe2b1146b80 1 -- 172.21.15.119:0/3376360433 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x7fe29810eab0 con 0x55daf7079a80 2023-12-07T10:32:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.261+0000 7fe27aff5700 1 -- 172.21.15.119:0/3376360433 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe29001e070 con 0x55daf7079a80 2023-12-07T10:32:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.261+0000 7fe27aff5700 1 -- 172.21.15.119:0/3376360433 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 59) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe2880d0070 con 0x55daf7079a80 2023-12-07T10:32:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.261+0000 7fe2b1146b80 10 client.27766 fetch_fsmap finished waiting for FSMap version 59 2023-12-07T10:32:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.261+0000 7fe2b1146b80 10 client.27766 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:32:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.261+0000 7fe2b1146b80 20 client.27766 populate_metadata read hostname 'smithi119' 2023-12-07T10:32:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.261+0000 7fe2b1146b80 1 --2- 172.21.15.119:0/3376360433 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55daf70e9f80 0x55daf70ec3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.261+0000 7fe2b1146b80 4 client.27766 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:32:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.261+0000 7fe2b1146b80 1 -- 172.21.15.119:0/3376360433 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55daf6f93f70 con 0x55daf70e9f80 2023-12-07T10:32:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.261+0000 7fe2967fc700 1 --2- 172.21.15.119:0/3376360433 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55daf70e9f80 0x55daf70ec3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.262+0000 7fe2967fc700 1 --2- 172.21.15.119:0/3376360433 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55daf70e9f80 0x55daf70ec3a0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b5da1ae92ce09c8 server_cookie=9a1eebfb5fb452d5 in_seq=0 out_seq=0 2023-12-07T10:32:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.262+0000 7fe27aff5700 10 client.27766 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:32:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.314+0000 7fe27aff5700 1 -- 172.21.15.119:0/3376360433 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55daf6f93f70 con 0x55daf70e9f80 2023-12-07T10:32:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.314+0000 7fe27aff5700 10 client.27766 handle_command_reply: tid=0 2023-12-07T10:32:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.314+0000 7fe2b1146b80 1 client.27766 shutdown 2023-12-07T10:32:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.314+0000 7fe2b1146b80 1 -- 172.21.15.119:0/3376360433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55daf7079a80 msgr2=0x55daf7087fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.314+0000 7fe2b1146b80 1 --2- 172.21.15.119:0/3376360433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55daf7079a80 0x55daf7087fe0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe288097460 tx=0x7fe288099f60 comp rx=0 tx=0).stop 2023-12-07T10:32:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.314+0000 7fe2b1146b80 1 -- 172.21.15.119:0/3376360433 shutdown_connections 2023-12-07T10:32:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.314+0000 7fe2b1146b80 1 --2- 172.21.15.119:0/3376360433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daf707f5c0 0x55daf70e84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.314+0000 7fe2b1146b80 1 --2- 172.21.15.119:0/3376360433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55daf7079a80 0x55daf7087fe0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.314+0000 7fe2b1146b80 1 --2- 172.21.15.119:0/3376360433 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55daf70e9f80 0x55daf70ec3a0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.314+0000 7fe2b1146b80 1 --2- 172.21.15.119:0/3376360433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55daf707eca0 0x55daf7088520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.314+0000 7fe27aff5700 0 client.27766 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:32:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.314+0000 7fe2b1146b80 1 -- 172.21.15.119:0/3376360433 >> 172.21.15.119:0/3376360433 conn(0x55daf6fa8300 msgr2=0x55daf708eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:32:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.315+0000 7fe2b1146b80 1 -- 172.21.15.119:0/3376360433 shutdown_connections 2023-12-07T10:32:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.315+0000 7fe2b1146b80 1 -- 172.21.15.119:0/3376360433 wait complete. 2023-12-07T10:32:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.315+0000 7fe2b1146b80 20 client.27766 trim_cache size 0 max 16384 2023-12-07T10:32:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:32:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:32:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:32:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:32:37.317 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.316+0000 7fe297fff700 1 -- 172.21.15.119:0/4061574597 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe2800a1d00 msgr2=0x7fe2800a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:37.317 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.316+0000 7fe297fff700 1 --2- 172.21.15.119:0/4061574597 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe2800a1d00 0x7fe2800a41c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe290022000 tx=0x7fe29001e000 comp rx=0 tx=0).stop 2023-12-07T10:32:37.317 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.316+0000 7fe297fff700 1 -- 172.21.15.119:0/4061574597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2981015a0 msgr2=0x7fe298199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:37.317 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.316+0000 7fe297fff700 1 --2- 172.21.15.119:0/4061574597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2981015a0 0x7fe298199a30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe28800e9a0 tx=0x7fe28800ed60 comp rx=0 tx=0).stop 2023-12-07T10:32:37.317 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.316+0000 7fe297fff700 1 -- 172.21.15.119:0/4061574597 shutdown_connections 2023-12-07T10:32:37.317 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.316+0000 7fe297fff700 1 --2- 172.21.15.119:0/4061574597 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe2800a1d00 0x7fe2800a41c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.317 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.316+0000 7fe297fff700 1 --2- 172.21.15.119:0/4061574597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe29819e340 0x7fe29819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.317 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.316+0000 7fe297fff700 1 --2- 172.21.15.119:0/4061574597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2981015a0 0x7fe298199a30 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.316+0000 7fe297fff700 1 --2- 172.21.15.119:0/4061574597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2981066c0 0x7fe298199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.316+0000 7fe297fff700 1 -- 172.21.15.119:0/4061574597 >> 172.21.15.119:0/4061574597 conn(0x7fe2980fce40 msgr2=0x7fe2980fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:32:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.316+0000 7fe297fff700 1 -- 172.21.15.119:0/4061574597 shutdown_connections 2023-12-07T10:32:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:37.316+0000 7fe297fff700 1 -- 172.21.15.119:0/4061574597 wait complete. 2023-12-07T10:32:37.579 INFO:tasks.workunit.client.0.smithi119.stdout: 1 898495 7.26 MB/sec execute 530 sec latency 48.042 ms 2023-12-07T10:32:38.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:38 smithi119 ceph-mon[97908]: pgmap v381: 129 pgs: 129 active+clean; 708 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 282 op/s 2023-12-07T10:32:38.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:38 smithi159 ceph-mon[134726]: pgmap v381: 129 pgs: 129 active+clean; 708 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 282 op/s 2023-12-07T10:32:38.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:38 smithi153 ceph-mon[133625]: pgmap v381: 129 pgs: 129 active+clean; 708 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 282 op/s 2023-12-07T10:32:38.580 INFO:tasks.workunit.client.0.smithi119.stdout: 1 899695 7.25 MB/sec execute 531 sec latency 12.956 ms 2023-12-07T10:32:39.579 INFO:tasks.workunit.client.0.smithi119.stdout: 1 901114 7.26 MB/sec execute 532 sec latency 254.174 ms 2023-12-07T10:32:40.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:40 smithi119 ceph-mon[97908]: pgmap v382: 129 pgs: 129 active+clean; 708 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 159 op/s 2023-12-07T10:32:40.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:40 smithi159 ceph-mon[134726]: pgmap v382: 129 pgs: 129 active+clean; 708 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 159 op/s 2023-12-07T10:32:40.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:40 smithi153 ceph-mon[133625]: pgmap v382: 129 pgs: 129 active+clean; 708 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 159 op/s 2023-12-07T10:32:40.579 INFO:tasks.workunit.client.0.smithi119.stdout: 1 902016 7.25 MB/sec execute 533 sec latency 127.289 ms 2023-12-07T10:32:41.579 INFO:tasks.workunit.client.0.smithi119.stdout: 1 902662 7.24 MB/sec execute 534 sec latency 33.271 ms 2023-12-07T10:32:42.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:41 smithi119 ceph-mon[97908]: pgmap v383: 129 pgs: 129 active+clean; 733 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 267 op/s 2023-12-07T10:32:42.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:41 smithi159 ceph-mon[134726]: pgmap v383: 129 pgs: 129 active+clean; 733 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 267 op/s 2023-12-07T10:32:42.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:41 smithi153 ceph-mon[133625]: pgmap v383: 129 pgs: 129 active+clean; 733 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 267 op/s 2023-12-07T10:32:42.579 INFO:tasks.workunit.client.0.smithi119.stdout: 1 904246 7.24 MB/sec execute 535 sec latency 28.294 ms 2023-12-07T10:32:43.580 INFO:tasks.workunit.client.0.smithi119.stdout: 1 905685 7.25 MB/sec execute 536 sec latency 18.126 ms 2023-12-07T10:32:44.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:44 smithi119 ceph-mon[97908]: pgmap v384: 129 pgs: 129 active+clean; 738 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 249 op/s 2023-12-07T10:32:44.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:44 smithi159 ceph-mon[134726]: pgmap v384: 129 pgs: 129 active+clean; 738 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 249 op/s 2023-12-07T10:32:44.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:44 smithi153 ceph-mon[133625]: pgmap v384: 129 pgs: 129 active+clean; 738 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 249 op/s 2023-12-07T10:32:44.580 INFO:tasks.workunit.client.0.smithi119.stdout: 1 906982 7.25 MB/sec execute 537 sec latency 17.051 ms 2023-12-07T10:32:45.580 INFO:tasks.workunit.client.0.smithi119.stdout: 1 908500 7.25 MB/sec execute 538 sec latency 138.554 ms 2023-12-07T10:32:46.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:46 smithi119 ceph-mon[97908]: pgmap v385: 129 pgs: 129 active+clean; 738 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 227 op/s 2023-12-07T10:32:46.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:46 smithi159 ceph-mon[134726]: pgmap v385: 129 pgs: 129 active+clean; 738 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 227 op/s 2023-12-07T10:32:46.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:46 smithi153 ceph-mon[133625]: pgmap v385: 129 pgs: 129 active+clean; 738 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 227 op/s 2023-12-07T10:32:46.580 INFO:tasks.workunit.client.0.smithi119.stdout: 1 909396 7.24 MB/sec execute 539 sec latency 20.238 ms 2023-12-07T10:32:47.580 INFO:tasks.workunit.client.0.smithi119.stdout: 1 910980 7.24 MB/sec execute 540 sec latency 18.696 ms 2023-12-07T10:32:48.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:48 smithi119 ceph-mon[97908]: pgmap v386: 129 pgs: 129 active+clean; 719 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.7 MiB/s wr, 357 op/s 2023-12-07T10:32:48.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:48 smithi159 ceph-mon[134726]: pgmap v386: 129 pgs: 129 active+clean; 719 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.7 MiB/s wr, 357 op/s 2023-12-07T10:32:48.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:48 smithi153 ceph-mon[133625]: pgmap v386: 129 pgs: 129 active+clean; 719 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.7 MiB/s wr, 357 op/s 2023-12-07T10:32:48.580 INFO:tasks.workunit.client.0.smithi119.stdout: 1 912616 7.25 MB/sec execute 541 sec latency 14.657 ms 2023-12-07T10:32:49.580 INFO:tasks.workunit.client.0.smithi119.stdout: 1 913740 7.25 MB/sec execute 542 sec latency 16.073 ms 2023-12-07T10:32:50.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:50 smithi119 ceph-mon[97908]: pgmap v387: 129 pgs: 129 active+clean; 719 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 275 op/s 2023-12-07T10:32:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.478+0000 7f247ef9d700 1 -- 172.21.15.119:0/1989100877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24781066c0 msgr2=0x7f247810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.478+0000 7f247ef9d700 1 --2- 172.21.15.119:0/1989100877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24781066c0 0x7f247810e240 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f247000b0a0 tx=0x7f247000b3b0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.478+0000 7f247ef9d700 1 -- 172.21.15.119:0/1989100877 shutdown_connections 2023-12-07T10:32:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.478+0000 7f247ef9d700 1 --2- 172.21.15.119:0/1989100877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24781066c0 0x7f247810e240 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.478+0000 7f247ef9d700 1 --2- 172.21.15.119:0/1989100877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24781015a0 0x7f2478101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.478+0000 7f247ef9d700 1 --2- 172.21.15.119:0/1989100877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2478101f50 0x7f2478105f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.478+0000 7f247ef9d700 1 -- 172.21.15.119:0/1989100877 >> 172.21.15.119:0/1989100877 conn(0x7f24780fce40 msgr2=0x7f24780ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:32:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.478+0000 7f247ef9d700 1 -- 172.21.15.119:0/1989100877 shutdown_connections 2023-12-07T10:32:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.479+0000 7f247ef9d700 1 -- 172.21.15.119:0/1989100877 wait complete. 2023-12-07T10:32:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.480+0000 7f247ef9d700 1 Processor -- start 2023-12-07T10:32:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.480+0000 7f247ef9d700 1 -- start start 2023-12-07T10:32:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.480+0000 7f247ef9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24781015a0 0x7f247819f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.480+0000 7f247ef9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24781066c0 0x7f247819fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.480+0000 7f247ef9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2478199a70 0x7f2478199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.480+0000 7f247ef9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247819a4f0 con 0x7f2478199a70 2023-12-07T10:32:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.480+0000 7f247ef9d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247819a660 con 0x7f24781015a0 2023-12-07T10:32:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.480+0000 7f247ef9d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247819a820 con 0x7f24781066c0 2023-12-07T10:32:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.481+0000 7f247d79a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24781066c0 0x7f247819fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.481+0000 7f247df9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24781015a0 0x7f247819f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.481+0000 7f247d79a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24781066c0 0x7f247819fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:52864/0 (socket says 172.21.15.119:52864) 2023-12-07T10:32:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.481+0000 7f247d79a700 1 -- 172.21.15.119:0/1816214193 learned_addr learned my addr 172.21.15.119:0/1816214193 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:32:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.481+0000 7f247e79c700 1 --2- 172.21.15.119:0/1816214193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2478199a70 0x7f2478199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.481+0000 7f247d79a700 1 -- 172.21.15.119:0/1816214193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24781015a0 msgr2=0x7f247819f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.481+0000 7f247d79a700 1 --2- 172.21.15.119:0/1816214193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24781015a0 0x7f247819f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.481+0000 7f247d79a700 1 -- 172.21.15.119:0/1816214193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2478199a70 msgr2=0x7f2478199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.481+0000 7f247d79a700 1 --2- 172.21.15.119:0/1816214193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2478199a70 0x7f2478199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.481+0000 7f247d79a700 1 -- 172.21.15.119:0/1816214193 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2470009d00 con 0x7f24781066c0 2023-12-07T10:32:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.481+0000 7f247d79a700 1 --2- 172.21.15.119:0/1816214193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24781066c0 0x7f247819fe30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f246800e970 tx=0x7f246800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:32:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.481+0000 7f246effd700 1 -- 172.21.15.119:0/1816214193 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2468009790 con 0x7f24781066c0 2023-12-07T10:32:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.482+0000 7f246effd700 1 -- 172.21.15.119:0/1816214193 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f246800eec0 con 0x7f24781066c0 2023-12-07T10:32:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.482+0000 7f247ef9d700 1 -- 172.21.15.119:0/1816214193 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24781a67c0 con 0x7f24781066c0 2023-12-07T10:32:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.482+0000 7f246effd700 1 -- 172.21.15.119:0/1816214193 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2468005590 con 0x7f24781066c0 2023-12-07T10:32:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.482+0000 7f247ef9d700 1 -- 172.21.15.119:0/1816214193 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24781a6ca0 con 0x7f24781066c0 2023-12-07T10:32:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.485+0000 7f246effd700 1 -- 172.21.15.119:0/1816214193 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f246800cbd0 con 0x7f24781066c0 2023-12-07T10:32:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.486+0000 7f246effd700 1 --2- 172.21.15.119:0/1816214193 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f24640a1c80 0x7f24640a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.486+0000 7f246effd700 1 -- 172.21.15.119:0/1816214193 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f2468014070 con 0x7f24781066c0 2023-12-07T10:32:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.486+0000 7f2493b9ab80 1 Processor -- start 2023-12-07T10:32:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.487+0000 7f247df9b700 1 --2- 172.21.15.119:0/1816214193 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f24640a1c80 0x7f24640a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.487+0000 7f2493b9ab80 1 -- start start 2023-12-07T10:32:50.488 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.487+0000 7f2493b9ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556997aa6ca0 0x556997aa7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:50.488 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.487+0000 7f2493b9ab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556997aa75c0 0x556997aa1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:50.488 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.487+0000 7f2493b9ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556997aa1a80 0x556997aa1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:50.488 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.487+0000 7f2493b9ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556997a15760 con 0x556997aa6ca0 2023-12-07T10:32:50.489 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.487+0000 7f2493b9ab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569979fc740 con 0x556997aa75c0 2023-12-07T10:32:50.489 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.487+0000 7f2493b9ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569979fa060 con 0x556997aa1a80 2023-12-07T10:32:50.489 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.487+0000 7f247d79a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556997aa75c0 0x556997aa1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:50.489 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.487+0000 7f247e79c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556997aa1a80 0x556997aa1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:50.489 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.487+0000 7f247df9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556997aa6ca0 0x556997aa7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:50.489 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.487+0000 7f247e79c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556997aa1a80 0x556997aa1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:52872/0 (socket says 172.21.15.119:52872) 2023-12-07T10:32:50.489 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.487+0000 7f247d79a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556997aa75c0 0x556997aa1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:51350/0 (socket says 172.21.15.119:51350) 2023-12-07T10:32:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.487+0000 7f247e79c700 1 -- 172.21.15.119:0/382395125 learned_addr learned my addr 172.21.15.119:0/382395125 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:32:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.488+0000 7f247df9b700 1 --2- 172.21.15.119:0/1816214193 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f24640a1c80 0x7f24640a4140 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2474009c60 tx=0x7f2474009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:32:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.488+0000 7f247e79c700 1 -- 172.21.15.119:0/382395125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556997aa75c0 msgr2=0x556997aa1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.488+0000 7f247e79c700 1 --2- 172.21.15.119:0/382395125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556997aa75c0 0x556997aa1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.488+0000 7f247e79c700 1 -- 172.21.15.119:0/382395125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556997aa6ca0 msgr2=0x556997aa7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.488+0000 7f247e79c700 1 --2- 172.21.15.119:0/382395125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556997aa6ca0 0x556997aa7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.488+0000 7f247e79c700 1 -- 172.21.15.119:0/382395125 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569978725d0 con 0x556997aa1a80 2023-12-07T10:32:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.488+0000 7f247e79c700 1 --2- 172.21.15.119:0/382395125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556997aa1a80 0x556997aa1f00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f247000b0a0 tx=0x7f2470009c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fe87f5cb263d774 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:32:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.488+0000 7f2463fff700 1 -- 172.21.15.119:0/382395125 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247000e050 con 0x556997aa1a80 2023-12-07T10:32:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.488+0000 7f2463fff700 1 -- 172.21.15.119:0/382395125 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2470009460 con 0x556997aa1a80 2023-12-07T10:32:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.488+0000 7f2463fff700 1 -- 172.21.15.119:0/382395125 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2470004c50 con 0x556997aa1a80 2023-12-07T10:32:50.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.489+0000 7f2493b9ab80 1 -- 172.21.15.119:0/382395125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556997aa1a80 msgr2=0x556997aa1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:50.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.489+0000 7f2493b9ab80 1 --2- 172.21.15.119:0/382395125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556997aa1a80 0x556997aa1f00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f247000b0a0 tx=0x7f2470009c60 comp rx=0 tx=0).stop 2023-12-07T10:32:50.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.489+0000 7f2493b9ab80 1 -- 172.21.15.119:0/382395125 shutdown_connections 2023-12-07T10:32:50.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.489+0000 7f2493b9ab80 1 --2- 172.21.15.119:0/382395125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556997aa1a80 0x556997aa1f00 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.489+0000 7f2493b9ab80 1 --2- 172.21.15.119:0/382395125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556997aa6ca0 0x556997aa7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.489+0000 7f2493b9ab80 1 --2- 172.21.15.119:0/382395125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556997aa75c0 0x556997aa1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.489+0000 7f2493b9ab80 1 -- 172.21.15.119:0/382395125 >> 172.21.15.119:0/382395125 conn(0x5569979d0300 msgr2=0x556997a9bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:32:50.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.489+0000 7f2493b9ab80 1 -- 172.21.15.119:0/382395125 shutdown_connections 2023-12-07T10:32:50.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.489+0000 7f2493b9ab80 1 -- 172.21.15.119:0/382395125 wait complete. 2023-12-07T10:32:50.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.490+0000 7f2493b9ab80 1 -- start start 2023-12-07T10:32:50.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.490+0000 7f2493b9ab80 1 Processor -- start 2023-12-07T10:32:50.494 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.491+0000 7f2493b9ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556997aa1a80 0x556997aad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.491+0000 7f2493b9ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556997aa6ca0 0x556997aaa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.491+0000 7f2493b9ab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556997aa75c0 0x556997aaad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.491+0000 7f2493b9ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569979f5950 con 0x556997aa1a80 2023-12-07T10:32:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.491+0000 7f2493b9ab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569979af500 con 0x556997aa75c0 2023-12-07T10:32:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.491+0000 7f2493b9ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569979af1a0 con 0x556997aa6ca0 2023-12-07T10:32:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.491+0000 7f2493b9ab80 10 client.0 _ll_register_callbacks cb 0x5569979968e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:32:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.491+0000 7f247d79a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556997aa6ca0 0x556997aaa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.491+0000 7f247d79a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556997aa6ca0 0x556997aaa7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:52884/0 (socket says 172.21.15.119:52884) 2023-12-07T10:32:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.491+0000 7f247d79a700 1 -- 172.21.15.119:0/3883865532 learned_addr learned my addr 172.21.15.119:0/3883865532 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:32:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.491+0000 7f247df9b700 1 --2- 172.21.15.119:0/3883865532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556997aa1a80 0x556997aad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.491+0000 7f247e79c700 1 --2- 172.21.15.119:0/3883865532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556997aa75c0 0x556997aaad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.492+0000 7f247d79a700 1 -- 172.21.15.119:0/3883865532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556997aa75c0 msgr2=0x556997aaad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.492+0000 7f247d79a700 1 --2- 172.21.15.119:0/3883865532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556997aa75c0 0x556997aaad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.492+0000 7f247d79a700 1 -- 172.21.15.119:0/3883865532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556997aa1a80 msgr2=0x556997aad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.492+0000 7f247d79a700 1 --2- 172.21.15.119:0/3883865532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556997aa1a80 0x556997aad4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.492+0000 7f247d79a700 1 -- 172.21.15.119:0/3883865532 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f247810eab0 con 0x556997aa6ca0 2023-12-07T10:32:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.492+0000 7f247d79a700 1 --2- 172.21.15.119:0/3883865532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556997aa6ca0 0x556997aaa7a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f246800c5b0 tx=0x7f246809b120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46f32febf3d2ed98 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:32:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.492+0000 7f24617fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:32:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.492+0000 7f24617fa700 1 -- 172.21.15.119:0/3883865532 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24680b2070 con 0x556997aa6ca0 2023-12-07T10:32:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.492+0000 7f2493b9ab80 1 -- 172.21.15.119:0/3883865532 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556997a946f0 con 0x556997aa6ca0 2023-12-07T10:32:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.492+0000 7f24617fa700 1 -- 172.21.15.119:0/3883865532 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f24680a2c60 con 0x556997aa6ca0 2023-12-07T10:32:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.493+0000 7f24617fa700 1 -- 172.21.15.119:0/3883865532 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24680ba0b0 con 0x556997aa6ca0 2023-12-07T10:32:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.493+0000 7f2493b9ab80 10 client.27778 fetch_fsmap learned FSMap version 59 2023-12-07T10:32:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.493+0000 7f2493b9ab80 1 -- 172.21.15.119:0/3883865532 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55699781ed80 con 0x556997aa6ca0 2023-12-07T10:32:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.493+0000 7f24617fa700 1 -- 172.21.15.119:0/3883865532 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556997a946f0 con 0x556997aa6ca0 2023-12-07T10:32:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.493+0000 7f24617fa700 1 -- 172.21.15.119:0/3883865532 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 59) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f246809ab30 con 0x556997aa6ca0 2023-12-07T10:32:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.493+0000 7f2493b9ab80 10 client.27778 fetch_fsmap finished waiting for FSMap version 59 2023-12-07T10:32:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.493+0000 7f2493b9ab80 10 client.27778 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:32:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.493+0000 7f2493b9ab80 20 client.27778 populate_metadata read hostname 'smithi119' 2023-12-07T10:32:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.493+0000 7f2493b9ab80 1 --2- 172.21.15.119:0/3883865532 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556997abaf50 0x556997abd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.493+0000 7f2493b9ab80 4 client.27778 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:32:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.493+0000 7f2493b9ab80 1 -- 172.21.15.119:0/3883865532 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556997a70740 con 0x556997abaf50 2023-12-07T10:32:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.493+0000 7f247df9b700 1 --2- 172.21.15.119:0/3883865532 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556997abaf50 0x556997abd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.494+0000 7f247df9b700 1 --2- 172.21.15.119:0/3883865532 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556997abaf50 0x556997abd370 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea270e6f2bb0bb67 server_cookie=b282b875fe307868 in_seq=0 out_seq=0 2023-12-07T10:32:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.494+0000 7f24617fa700 10 client.27778 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:32:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.495+0000 7f24617fa700 1 -- 172.21.15.119:0/3883865532 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556997a70740 con 0x556997abaf50 2023-12-07T10:32:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.495+0000 7f24617fa700 10 client.27778 handle_command_reply: tid=0 2023-12-07T10:32:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.495+0000 7f2493b9ab80 1 client.27778 shutdown 2023-12-07T10:32:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.495+0000 7f2493b9ab80 1 -- 172.21.15.119:0/3883865532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556997aa6ca0 msgr2=0x556997aaa7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.495+0000 7f2493b9ab80 1 --2- 172.21.15.119:0/3883865532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556997aa6ca0 0x556997aaa7a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f246800c5b0 tx=0x7f246809b120 comp rx=0 tx=0).stop 2023-12-07T10:32:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.495+0000 7f2493b9ab80 1 -- 172.21.15.119:0/3883865532 shutdown_connections 2023-12-07T10:32:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.495+0000 7f2493b9ab80 1 --2- 172.21.15.119:0/3883865532 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556997abaf50 0x556997abd370 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.495+0000 7f2493b9ab80 1 --2- 172.21.15.119:0/3883865532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556997aa75c0 0x556997aaad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.495+0000 7f2493b9ab80 1 --2- 172.21.15.119:0/3883865532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556997aa1a80 0x556997aad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.495+0000 7f2493b9ab80 1 --2- 172.21.15.119:0/3883865532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556997aa6ca0 0x556997aaa7a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.495+0000 7f2493b9ab80 1 -- 172.21.15.119:0/3883865532 >> 172.21.15.119:0/3883865532 conn(0x5569979d0300 msgr2=0x556997a96980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:32:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.495+0000 7f24617fa700 0 client.27778 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:32:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.496+0000 7f2493b9ab80 1 -- 172.21.15.119:0/3883865532 shutdown_connections 2023-12-07T10:32:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.496+0000 7f2493b9ab80 1 -- 172.21.15.119:0/3883865532 wait complete. 2023-12-07T10:32:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.496+0000 7f2493b9ab80 20 client.27778 trim_cache size 0 max 16384 2023-12-07T10:32:50.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:50 smithi159 ceph-mon[134726]: pgmap v387: 129 pgs: 129 active+clean; 719 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 275 op/s 2023-12-07T10:32:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.513+0000 7f2493b9ab80 1 Processor -- start 2023-12-07T10:32:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.513+0000 7f2493b9ab80 1 -- start start 2023-12-07T10:32:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.513+0000 7f2493b9ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556997aa1a80 0x556997aa1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.513+0000 7f2493b9ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556997aa6ca0 0x556997aaad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.513+0000 7f2493b9ab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556997aa75c0 0x556997ac1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.513+0000 7f2493b9ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569978748c0 con 0x556997aa6ca0 2023-12-07T10:32:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.513+0000 7f2493b9ab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x556997a9c4c0 con 0x556997aa75c0 2023-12-07T10:32:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.513+0000 7f2493b9ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556997aad860 con 0x556997aa1a80 2023-12-07T10:32:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.514+0000 7f247e79c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556997aa75c0 0x556997ac1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.514+0000 7f247e79c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556997aa75c0 0x556997ac1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:51358/0 (socket says 172.21.15.119:51358) 2023-12-07T10:32:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.514+0000 7f247e79c700 1 -- 172.21.15.119:0/944268140 learned_addr learned my addr 172.21.15.119:0/944268140 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:32:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.514+0000 7f247df9b700 1 --2- 172.21.15.119:0/944268140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556997aa1a80 0x556997aa1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.514+0000 7f247d79a700 1 --2- 172.21.15.119:0/944268140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556997aa6ca0 0x556997aaad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.514+0000 7f247e79c700 1 -- 172.21.15.119:0/944268140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556997aa1a80 msgr2=0x556997aa1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.514+0000 7f247e79c700 1 --2- 172.21.15.119:0/944268140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556997aa1a80 0x556997aa1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.514+0000 7f247e79c700 1 -- 172.21.15.119:0/944268140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556997aa6ca0 msgr2=0x556997aaad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.514+0000 7f247e79c700 1 --2- 172.21.15.119:0/944268140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556997aa6ca0 0x556997aaad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.514+0000 7f247e79c700 1 -- 172.21.15.119:0/944268140 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556997a95c30 con 0x556997aa75c0 2023-12-07T10:32:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.514+0000 7f247df9b700 1 --2- 172.21.15.119:0/944268140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556997aa1a80 0x556997aa1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:32:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.514+0000 7f247e79c700 1 --2- 172.21.15.119:0/944268140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556997aa75c0 0x556997ac1cd0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f247000b780 tx=0x7f247000bc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25cb28a662875c0c server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:32:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.514+0000 7f246cff9700 1 -- 172.21.15.119:0/944268140 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2470007cc0 con 0x556997aa75c0 2023-12-07T10:32:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.515+0000 7f246cff9700 1 -- 172.21.15.119:0/944268140 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2470029070 con 0x556997aa75c0 2023-12-07T10:32:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.515+0000 7f246cff9700 1 -- 172.21.15.119:0/944268140 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24700052a0 con 0x556997aa75c0 2023-12-07T10:32:50.519 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.515+0000 7f2493b9ab80 1 -- 172.21.15.119:0/944268140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556997aa75c0 msgr2=0x556997ac1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:50.519 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.515+0000 7f2493b9ab80 1 --2- 172.21.15.119:0/944268140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556997aa75c0 0x556997ac1cd0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f247000b780 tx=0x7f247000bc90 comp rx=0 tx=0).stop 2023-12-07T10:32:50.519 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.515+0000 7f2493b9ab80 1 -- 172.21.15.119:0/944268140 shutdown_connections 2023-12-07T10:32:50.519 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.515+0000 7f2493b9ab80 1 --2- 172.21.15.119:0/944268140 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556997aa75c0 0x556997ac1cd0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.519 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.515+0000 7f2493b9ab80 1 --2- 172.21.15.119:0/944268140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556997aa1a80 0x556997aa1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.519 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.515+0000 7f2493b9ab80 1 --2- 172.21.15.119:0/944268140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556997aa6ca0 0x556997aaad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.519 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.515+0000 7f2493b9ab80 1 -- 172.21.15.119:0/944268140 >> 172.21.15.119:0/944268140 conn(0x5569979d0300 msgr2=0x556997ab6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:32:50.519 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.515+0000 7f2493b9ab80 1 -- 172.21.15.119:0/944268140 shutdown_connections 2023-12-07T10:32:50.519 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.515+0000 7f2493b9ab80 1 -- 172.21.15.119:0/944268140 wait complete. 2023-12-07T10:32:50.519 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.516+0000 7f2493b9ab80 1 -- start start 2023-12-07T10:32:50.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.516+0000 7f2493b9ab80 1 Processor -- start 2023-12-07T10:32:50.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.516+0000 7f2493b9ab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556997aa1a80 0x556997ab1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:50.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.517+0000 7f2493b9ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556997aa6ca0 0x556997ab24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:50.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.517+0000 7f2493b9ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556997aa75c0 0x556997aa2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:50.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.517+0000 7f2493b9ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556997aa2e70 con 0x556997aa6ca0 2023-12-07T10:32:50.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.517+0000 7f2493b9ab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x556997aa2fe0 con 0x556997aa1a80 2023-12-07T10:32:50.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.517+0000 7f2493b9ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556997aa31a0 con 0x556997aa75c0 2023-12-07T10:32:50.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.517+0000 7f2493b9ab80 10 client.0 _ll_register_callbacks cb 0x5569979968e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:32:50.520 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.517+0000 7f247d79a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556997aa6ca0 0x556997ab24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:50.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.517+0000 7f247df9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556997aa1a80 0x556997ab1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:50.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.517+0000 7f247e79c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556997aa75c0 0x556997aa2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:50.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.517+0000 7f247df9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556997aa1a80 0x556997ab1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:51366/0 (socket says 172.21.15.119:51366) 2023-12-07T10:32:50.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.517+0000 7f247df9b700 1 -- 172.21.15.119:0/4024314545 learned_addr learned my addr 172.21.15.119:0/4024314545 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:32:50.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.517+0000 7f247e79c700 1 -- 172.21.15.119:0/4024314545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556997aa1a80 msgr2=0x556997ab1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:50.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.517+0000 7f247e79c700 1 --2- 172.21.15.119:0/4024314545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556997aa1a80 0x556997ab1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.517+0000 7f247e79c700 1 -- 172.21.15.119:0/4024314545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556997aa6ca0 msgr2=0x556997ab24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:32:50.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.517+0000 7f247e79c700 1 --2- 172.21.15.119:0/4024314545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556997aa6ca0 0x556997ab24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.517+0000 7f247e79c700 1 -- 172.21.15.119:0/4024314545 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2474017070 con 0x556997aa75c0 2023-12-07T10:32:50.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.517+0000 7f247df9b700 1 --2- 172.21.15.119:0/4024314545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556997aa1a80 0x556997ab1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:32:50.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.518+0000 7f247e79c700 1 --2- 172.21.15.119:0/4024314545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556997aa75c0 0x556997aa2760 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f247000b3f0 tx=0x7f2470007980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87866f76f7e873f5 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:32:50.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.518+0000 7f24617fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:32:50.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.518+0000 7f24617fa700 1 -- 172.21.15.119:0/4024314545 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247000e040 con 0x556997aa75c0 2023-12-07T10:32:50.521 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.518+0000 7f2493b9ab80 1 -- 172.21.15.119:0/4024314545 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2470009460 con 0x556997aa75c0 2023-12-07T10:32:50.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.518+0000 7f24617fa700 1 -- 172.21.15.119:0/4024314545 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f247002b070 con 0x556997aa75c0 2023-12-07T10:32:50.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.518+0000 7f24617fa700 1 -- 172.21.15.119:0/4024314545 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2470004c50 con 0x556997aa75c0 2023-12-07T10:32:50.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.518+0000 7f2493b9ab80 10 client.27784 fetch_fsmap learned FSMap version 59 2023-12-07T10:32:50.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.518+0000 7f2493b9ab80 1 -- 172.21.15.119:0/4024314545 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x5569978725d0 con 0x556997aa75c0 2023-12-07T10:32:50.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.519+0000 7f24617fa700 1 -- 172.21.15.119:0/4024314545 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2470009460 con 0x556997aa75c0 2023-12-07T10:32:50.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.519+0000 7f24617fa700 1 -- 172.21.15.119:0/4024314545 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 59) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f247000b0d0 con 0x556997aa75c0 2023-12-07T10:32:50.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.519+0000 7f2493b9ab80 10 client.27784 fetch_fsmap finished waiting for FSMap version 59 2023-12-07T10:32:50.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.519+0000 7f2493b9ab80 10 client.27784 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:32:50.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.519+0000 7f2493b9ab80 20 client.27784 populate_metadata read hostname 'smithi119' 2023-12-07T10:32:50.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.519+0000 7f2493b9ab80 1 --2- 172.21.15.119:0/4024314545 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556997b11450 0x556997b13840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:32:50.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.519+0000 7f2493b9ab80 4 client.27784 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:32:50.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.519+0000 7f2493b9ab80 1 -- 172.21.15.119:0/4024314545 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5569979bbf70 con 0x556997b11450 2023-12-07T10:32:50.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.519+0000 7f247df9b700 1 --2- 172.21.15.119:0/4024314545 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556997b11450 0x556997b13840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:32:50.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.520+0000 7f24617fa700 10 client.27784 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:32:50.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.520+0000 7f247df9b700 1 --2- 172.21.15.119:0/4024314545 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556997b11450 0x556997b13840 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae84c649e3dbef98 server_cookie=4ad889b5f11143fb in_seq=0 out_seq=0 2023-12-07T10:32:50.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:50 smithi153 ceph-mon[133625]: pgmap v387: 129 pgs: 129 active+clean; 719 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 275 op/s 2023-12-07T10:32:50.580 INFO:tasks.workunit.client.0.smithi119.stdout: 1 915555 7.25 MB/sec execute 543 sec latency 51.345 ms 2023-12-07T10:32:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.588+0000 7f24617fa700 1 -- 172.21.15.119:0/4024314545 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5569979bbf70 con 0x556997b11450 2023-12-07T10:32:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.588+0000 7f24617fa700 10 client.27784 handle_command_reply: tid=0 2023-12-07T10:32:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.588+0000 7f2493b9ab80 1 client.27784 shutdown 2023-12-07T10:32:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.588+0000 7f2493b9ab80 1 -- 172.21.15.119:0/4024314545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556997aa75c0 msgr2=0x556997aa2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.588+0000 7f2493b9ab80 1 --2- 172.21.15.119:0/4024314545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556997aa75c0 0x556997aa2760 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f247000b3f0 tx=0x7f2470007980 comp rx=0 tx=0).stop 2023-12-07T10:32:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.589+0000 7f2493b9ab80 1 -- 172.21.15.119:0/4024314545 shutdown_connections 2023-12-07T10:32:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.589+0000 7f2493b9ab80 1 --2- 172.21.15.119:0/4024314545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556997aa75c0 0x556997aa2760 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.589+0000 7f2493b9ab80 1 --2- 172.21.15.119:0/4024314545 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556997b11450 0x556997b13840 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.589 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.589+0000 7f2493b9ab80 1 --2- 172.21.15.119:0/4024314545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556997aa1a80 0x556997ab1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.589+0000 7f24617fa700 0 client.27784 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:32:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.589+0000 7f2493b9ab80 1 --2- 172.21.15.119:0/4024314545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556997aa6ca0 0x556997ab24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.589+0000 7f2493b9ab80 1 -- 172.21.15.119:0/4024314545 >> 172.21.15.119:0/4024314545 conn(0x5569979d0300 msgr2=0x556997aa4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:32:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.589+0000 7f2493b9ab80 1 -- 172.21.15.119:0/4024314545 shutdown_connections 2023-12-07T10:32:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.589+0000 7f2493b9ab80 1 -- 172.21.15.119:0/4024314545 wait complete. 2023-12-07T10:32:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.589+0000 7f2493b9ab80 20 client.27784 trim_cache size 0 max 16384 2023-12-07T10:32:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:32:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:32:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:32:50.590 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:32:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.589+0000 7f247ef9d700 1 -- 172.21.15.119:0/1816214193 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f24640a1c80 msgr2=0x7f24640a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.589+0000 7f247ef9d700 1 --2- 172.21.15.119:0/1816214193 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f24640a1c80 0x7f24640a4140 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2474009c60 tx=0x7f2474009260 comp rx=0 tx=0).stop 2023-12-07T10:32:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.589+0000 7f247ef9d700 1 -- 172.21.15.119:0/1816214193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24781066c0 msgr2=0x7f247819fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:32:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.589+0000 7f247ef9d700 1 --2- 172.21.15.119:0/1816214193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24781066c0 0x7f247819fe30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f246800e970 tx=0x7f246800ec80 comp rx=0 tx=0).stop 2023-12-07T10:32:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.590+0000 7f247ef9d700 1 -- 172.21.15.119:0/1816214193 shutdown_connections 2023-12-07T10:32:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.590+0000 7f247ef9d700 1 --2- 172.21.15.119:0/1816214193 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f24640a1c80 0x7f24640a4140 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.590+0000 7f247ef9d700 1 --2- 172.21.15.119:0/1816214193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2478199a70 0x7f2478199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.590+0000 7f247ef9d700 1 --2- 172.21.15.119:0/1816214193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24781015a0 0x7f247819f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.590+0000 7f247ef9d700 1 --2- 172.21.15.119:0/1816214193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24781066c0 0x7f247819fe30 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:32:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.590+0000 7f247ef9d700 1 -- 172.21.15.119:0/1816214193 >> 172.21.15.119:0/1816214193 conn(0x7f24780fce40 msgr2=0x7f24780fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:32:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.590+0000 7f247ef9d700 1 -- 172.21.15.119:0/1816214193 shutdown_connections 2023-12-07T10:32:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:32:50.590+0000 7f247ef9d700 1 -- 172.21.15.119:0/1816214193 wait complete. 2023-12-07T10:32:51.580 INFO:tasks.workunit.client.0.smithi119.stdout: 1 916663 7.25 MB/sec execute 544 sec latency 58.092 ms 2023-12-07T10:32:52.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:51 smithi119 ceph-mon[97908]: pgmap v388: 129 pgs: 129 active+clean; 722 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.4 MiB/s wr, 366 op/s 2023-12-07T10:32:52.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:51 smithi159 ceph-mon[134726]: pgmap v388: 129 pgs: 129 active+clean; 722 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.4 MiB/s wr, 366 op/s 2023-12-07T10:32:52.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:51 smithi153 ceph-mon[133625]: pgmap v388: 129 pgs: 129 active+clean; 722 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.4 MiB/s wr, 366 op/s 2023-12-07T10:32:52.581 INFO:tasks.workunit.client.0.smithi119.stdout: 1 917354 7.27 MB/sec execute 545 sec latency 65.815 ms 2023-12-07T10:32:53.581 INFO:tasks.workunit.client.0.smithi119.stdout: 1 919041 7.27 MB/sec execute 546 sec latency 22.030 ms 2023-12-07T10:32:54.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:54 smithi119 ceph-mon[97908]: pgmap v389: 129 pgs: 129 active+clean; 722 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 271 op/s 2023-12-07T10:32:54.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:54 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:32:54.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:54 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:32:54.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:54 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:32:54.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:54 smithi159 ceph-mon[134726]: pgmap v389: 129 pgs: 129 active+clean; 722 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 271 op/s 2023-12-07T10:32:54.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:54 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:32:54.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:54 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:32:54.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:54 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:32:54.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:54 smithi153 ceph-mon[133625]: pgmap v389: 129 pgs: 129 active+clean; 722 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 271 op/s 2023-12-07T10:32:54.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:54 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:32:54.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:54 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:32:54.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:54 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:32:54.580 INFO:tasks.workunit.client.0.smithi119.stdout: 1 920496 7.28 MB/sec execute 547 sec latency 15.282 ms 2023-12-07T10:32:55.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:55 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:32:55.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:55 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:32:55.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:55 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:32:55.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:55 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:32:55.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:55 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:32:55.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:55 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:32:55.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:55 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:32:55.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:55 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:32:55.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:55 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:32:55.580 INFO:tasks.workunit.client.0.smithi119.stdout: 1 921673 7.28 MB/sec execute 548 sec latency 16.364 ms 2023-12-07T10:32:56.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:56 smithi119 ceph-mon[97908]: pgmap v390: 129 pgs: 129 active+clean; 722 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2023-12-07T10:32:56.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:56 smithi159 ceph-mon[134726]: pgmap v390: 129 pgs: 129 active+clean; 722 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2023-12-07T10:32:56.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:56 smithi153 ceph-mon[133625]: pgmap v390: 129 pgs: 129 active+clean; 722 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2023-12-07T10:32:56.580 INFO:tasks.workunit.client.0.smithi119.stdout: 1 923621 7.28 MB/sec execute 549 sec latency 16.830 ms 2023-12-07T10:32:57.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:57 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:32:57.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:57 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:32:57.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:57 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:32:57.580 INFO:tasks.workunit.client.0.smithi119.stdout: 1 924595 7.28 MB/sec execute 550 sec latency 21.861 ms 2023-12-07T10:32:58.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:32:58 smithi119 ceph-mon[97908]: pgmap v391: 129 pgs: 129 active+clean; 730 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 321 op/s 2023-12-07T10:32:58.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:32:58 smithi159 ceph-mon[134726]: pgmap v391: 129 pgs: 129 active+clean; 730 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 321 op/s 2023-12-07T10:32:58.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:32:58 smithi153 ceph-mon[133625]: pgmap v391: 129 pgs: 129 active+clean; 730 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 321 op/s 2023-12-07T10:32:58.580 INFO:tasks.workunit.client.0.smithi119.stdout: 1 926386 7.28 MB/sec execute 551 sec latency 17.480 ms 2023-12-07T10:32:59.581 INFO:tasks.workunit.client.0.smithi119.stdout: 1 927806 7.29 MB/sec execute 552 sec latency 12.273 ms 2023-12-07T10:33:00.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:00 smithi119 ceph-mon[97908]: pgmap v392: 129 pgs: 129 active+clean; 730 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1.3 MiB/s wr, 191 op/s 2023-12-07T10:33:00.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:00 smithi159 ceph-mon[134726]: pgmap v392: 129 pgs: 129 active+clean; 730 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1.3 MiB/s wr, 191 op/s 2023-12-07T10:33:00.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:00 smithi153 ceph-mon[133625]: pgmap v392: 129 pgs: 129 active+clean; 730 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1.3 MiB/s wr, 191 op/s 2023-12-07T10:33:00.581 INFO:tasks.workunit.client.0.smithi119.stdout: 1 929431 7.28 MB/sec execute 553 sec latency 19.967 ms 2023-12-07T10:33:01.581 INFO:tasks.workunit.client.0.smithi119.stdout: 1 929748 7.27 MB/sec execute 554 sec latency 70.859 ms 2023-12-07T10:33:02.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:02 smithi119 ceph-mon[97908]: pgmap v393: 129 pgs: 129 active+clean; 736 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 264 op/s 2023-12-07T10:33:02.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:02 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:33:02.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:02 smithi159 ceph-mon[134726]: pgmap v393: 129 pgs: 129 active+clean; 736 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 264 op/s 2023-12-07T10:33:02.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:02 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:33:02.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:02 smithi153 ceph-mon[133625]: pgmap v393: 129 pgs: 129 active+clean; 736 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 264 op/s 2023-12-07T10:33:02.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:02 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:33:02.581 INFO:tasks.workunit.client.0.smithi119.stdout: 1 931205 7.28 MB/sec execute 555 sec latency 17.440 ms 2023-12-07T10:33:03.581 INFO:tasks.workunit.client.0.smithi119.stdout: 1 932415 7.28 MB/sec execute 556 sec latency 22.392 ms 2023-12-07T10:33:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.784+0000 7f8c4639d700 1 -- 172.21.15.119:0/58989821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c40102270 msgr2=0x7f8c400688e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.784+0000 7f8c4639d700 1 --2- 172.21.15.119:0/58989821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c40102270 0x7f8c400688e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009990 tx=0x7f8c3c009ca0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.784+0000 7f8c4639d700 1 -- 172.21.15.119:0/58989821 shutdown_connections 2023-12-07T10:33:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.784+0000 7f8c4639d700 1 --2- 172.21.15.119:0/58989821 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c4010dc10 0x7f8c40110020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.784+0000 7f8c4639d700 1 --2- 172.21.15.119:0/58989821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c40102270 0x7f8c400688e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.784+0000 7f8c4639d700 1 --2- 172.21.15.119:0/58989821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c40068e20 0x7f8c400692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.785 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.784+0000 7f8c4639d700 1 -- 172.21.15.119:0/58989821 >> 172.21.15.119:0/58989821 conn(0x7f8c40075e70 msgr2=0x7f8c40076280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:03.786 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.784+0000 7f8c4639d700 1 -- 172.21.15.119:0/58989821 shutdown_connections 2023-12-07T10:33:03.786 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.784+0000 7f8c4639d700 1 -- 172.21.15.119:0/58989821 wait complete. 2023-12-07T10:33:03.786 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.785+0000 7f8c4639d700 1 Processor -- start 2023-12-07T10:33:03.786 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.786+0000 7f8c4639d700 1 -- start start 2023-12-07T10:33:03.787 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.786+0000 7f8c4639d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c40068e20 0x7f8c40199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:03.787 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.786+0000 7f8c4639d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c40102270 0x7f8c40199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:03.787 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.786+0000 7f8c4639d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c4010dc10 0x7f8c4019e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:03.787 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.786+0000 7f8c4639d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4019e950 con 0x7f8c40102270 2023-12-07T10:33:03.787 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.786+0000 7f8c4639d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4019eac0 con 0x7f8c4010dc10 2023-12-07T10:33:03.787 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.786+0000 7f8c4639d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4019ec80 con 0x7f8c40068e20 2023-12-07T10:33:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.786+0000 7f8c44b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c40102270 0x7f8c40199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.786+0000 7f8c44b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c40102270 0x7f8c40199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53926/0 (socket says 172.21.15.119:53926) 2023-12-07T10:33:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.786+0000 7f8c44b9a700 1 -- 172.21.15.119:0/2511053510 learned_addr learned my addr 172.21.15.119:0/2511053510 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:33:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.787+0000 7f8c45b9c700 1 --2- 172.21.15.119:0/2511053510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c4010dc10 0x7f8c4019e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.787+0000 7f8c4539b700 1 --2- 172.21.15.119:0/2511053510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c40068e20 0x7f8c40199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.787+0000 7f8c44b9a700 1 -- 172.21.15.119:0/2511053510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c40068e20 msgr2=0x7f8c40199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.787+0000 7f8c44b9a700 1 --2- 172.21.15.119:0/2511053510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c40068e20 0x7f8c40199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.787+0000 7f8c44b9a700 1 -- 172.21.15.119:0/2511053510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c4010dc10 msgr2=0x7f8c4019e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.787+0000 7f8c44b9a700 1 --2- 172.21.15.119:0/2511053510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c4010dc10 0x7f8c4019e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.787+0000 7f8c44b9a700 1 -- 172.21.15.119:0/2511053510 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3c0095f0 con 0x7f8c40102270 2023-12-07T10:33:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.787+0000 7f8c4539b700 1 --2- 172.21.15.119:0/2511053510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c40068e20 0x7f8c40199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:33:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.787+0000 7f8c44b9a700 1 --2- 172.21.15.119:0/2511053510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c40102270 0x7f8c40199fb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000e9a0 tx=0x7f8c3000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:03.789 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.787+0000 7f8c367fc700 1 -- 172.21.15.119:0/2511053510 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c30009790 con 0x7f8c40102270 2023-12-07T10:33:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.788+0000 7f8c4639d700 1 -- 172.21.15.119:0/2511053510 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c4019ef60 con 0x7f8c40102270 2023-12-07T10:33:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.788+0000 7f8c367fc700 1 -- 172.21.15.119:0/2511053510 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c3000cbd0 con 0x7f8c40102270 2023-12-07T10:33:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.788+0000 7f8c367fc700 1 -- 172.21.15.119:0/2511053510 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c30018680 con 0x7f8c40102270 2023-12-07T10:33:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.788+0000 7f8c4639d700 1 -- 172.21.15.119:0/2511053510 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c401a6810 con 0x7f8c40102270 2023-12-07T10:33:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.792+0000 7f8c5af3fb80 1 Processor -- start 2023-12-07T10:33:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.792+0000 7f8c367fc700 1 -- 172.21.15.119:0/2511053510 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f8c3000cd40 con 0x7f8c40102270 2023-12-07T10:33:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.792+0000 7f8c5af3fb80 1 -- start start 2023-12-07T10:33:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.793+0000 7f8c5af3fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646bf1dfca0 0x5646bf1e0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.793+0000 7f8c5af3fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646bf1e05c0 0x5646bf1da540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.793+0000 7f8c5af3fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646bf1daa80 0x5646bf1daf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.793+0000 7f8c5af3fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646bf14e760 con 0x5646bf1e05c0 2023-12-07T10:33:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.793+0000 7f8c5af3fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646bf135740 con 0x5646bf1daa80 2023-12-07T10:33:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.793+0000 7f8c5af3fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646bf133060 con 0x5646bf1dfca0 2023-12-07T10:33:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.793+0000 7f8c4539b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646bf1e05c0 0x5646bf1da540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.793+0000 7f8c4539b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646bf1e05c0 0x5646bf1da540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53938/0 (socket says 172.21.15.119:53938) 2023-12-07T10:33:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.793+0000 7f8c45b9c700 1 --2- 172.21.15.119:0/2141065734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646bf1dfca0 0x5646bf1e0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.793+0000 7f8c4539b700 1 -- 172.21.15.119:0/2141065734 learned_addr learned my addr 172.21.15.119:0/2141065734 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:33:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.793+0000 7f8c4539b700 1 -- 172.21.15.119:0/2141065734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646bf1dfca0 msgr2=0x5646bf1e0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.793+0000 7f8c4539b700 1 --2- 172.21.15.119:0/2141065734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646bf1dfca0 0x5646bf1e0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.793+0000 7f8c4539b700 1 -- 172.21.15.119:0/2141065734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646bf1daa80 msgr2=0x5646bf1daf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.793+0000 7f8c4539b700 1 --2- 172.21.15.119:0/2141065734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646bf1daa80 0x5646bf1daf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.793+0000 7f8c4539b700 1 -- 172.21.15.119:0/2141065734 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646befab5d0 con 0x5646bf1e05c0 2023-12-07T10:33:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.794+0000 7f8c4539b700 1 --2- 172.21.15.119:0/2141065734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646bf1e05c0 0x5646bf1da540 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c015000 tx=0x7f8c3c005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2731eb9b5fff174e server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.794+0000 7f8c2b7fe700 1 -- 172.21.15.119:0/2141065734 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c0056d0 con 0x5646bf1e05c0 2023-12-07T10:33:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.794+0000 7f8c2b7fe700 1 -- 172.21.15.119:0/2141065734 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c3c01e040 con 0x5646bf1e05c0 2023-12-07T10:33:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.794+0000 7f8c367fc700 1 --2- 172.21.15.119:0/2511053510 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8c2c0a1c80 0x7f8c2c0a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.794+0000 7f8c2b7fe700 1 -- 172.21.15.119:0/2141065734 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c004ad0 con 0x5646bf1e05c0 2023-12-07T10:33:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.794+0000 7f8c367fc700 1 -- 172.21.15.119:0/2511053510 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f8c30014070 con 0x7f8c40102270 2023-12-07T10:33:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.794+0000 7f8c45b9c700 1 --2- 172.21.15.119:0/2511053510 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8c2c0a1c80 0x7f8c2c0a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.795+0000 7f8c5af3fb80 1 -- 172.21.15.119:0/2141065734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646bf1e05c0 msgr2=0x5646bf1da540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.795+0000 7f8c5af3fb80 1 --2- 172.21.15.119:0/2141065734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646bf1e05c0 0x5646bf1da540 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c015000 tx=0x7f8c3c005c80 comp rx=0 tx=0).stop 2023-12-07T10:33:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.795+0000 7f8c45b9c700 1 --2- 172.21.15.119:0/2511053510 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8c2c0a1c80 0x7f8c2c0a4140 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8c400740b0 tx=0x7f8c3800b040 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.795+0000 7f8c5af3fb80 1 -- 172.21.15.119:0/2141065734 shutdown_connections 2023-12-07T10:33:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.795+0000 7f8c5af3fb80 1 --2- 172.21.15.119:0/2141065734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646bf1daa80 0x5646bf1daf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.795+0000 7f8c5af3fb80 1 --2- 172.21.15.119:0/2141065734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646bf1dfca0 0x5646bf1e0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.795+0000 7f8c5af3fb80 1 --2- 172.21.15.119:0/2141065734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646bf1e05c0 0x5646bf1da540 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.795+0000 7f8c5af3fb80 1 -- 172.21.15.119:0/2141065734 >> 172.21.15.119:0/2141065734 conn(0x5646bf109300 msgr2=0x5646bf1d4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.795+0000 7f8c5af3fb80 1 -- 172.21.15.119:0/2141065734 shutdown_connections 2023-12-07T10:33:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.795+0000 7f8c5af3fb80 1 -- 172.21.15.119:0/2141065734 wait complete. 2023-12-07T10:33:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.796+0000 7f8c5af3fb80 1 -- start start 2023-12-07T10:33:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.796+0000 7f8c5af3fb80 1 Processor -- start 2023-12-07T10:33:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.797+0000 7f8c5af3fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646bf1daa80 0x5646bf1e38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.797+0000 7f8c5af3fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646bf1dfca0 0x5646bf1e3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.797+0000 7f8c5af3fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646bf1e05c0 0x5646bf1e4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.797+0000 7f8c5af3fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646bf12e950 con 0x5646bf1daa80 2023-12-07T10:33:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.797+0000 7f8c5af3fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646bf0e8500 con 0x5646bf1e05c0 2023-12-07T10:33:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.797+0000 7f8c5af3fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646bf0e81a0 con 0x5646bf1dfca0 2023-12-07T10:33:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.797+0000 7f8c5af3fb80 10 client.0 _ll_register_callbacks cb 0x5646bf0cf8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:33:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.797+0000 7f8c4539b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646bf1daa80 0x5646bf1e38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.797+0000 7f8c45b9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646bf1e05c0 0x5646bf1e4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.797+0000 7f8c45b9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646bf1e05c0 0x5646bf1e4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:36582/0 (socket says 172.21.15.119:36582) 2023-12-07T10:33:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.797+0000 7f8c45b9c700 1 -- 172.21.15.119:0/2676158433 learned_addr learned my addr 172.21.15.119:0/2676158433 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:33:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.798+0000 7f8c45b9c700 1 -- 172.21.15.119:0/2676158433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646bf1dfca0 msgr2=0x5646bf1e3e20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-07T10:33:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.798+0000 7f8c45b9c700 1 --2- 172.21.15.119:0/2676158433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646bf1dfca0 0x5646bf1e3e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.798+0000 7f8c45b9c700 1 -- 172.21.15.119:0/2676158433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646bf1daa80 msgr2=0x5646bf1e38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.798+0000 7f8c45b9c700 1 --2- 172.21.15.119:0/2676158433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646bf1daa80 0x5646bf1e38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.798+0000 7f8c45b9c700 1 -- 172.21.15.119:0/2676158433 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c40110ce0 con 0x5646bf1e05c0 2023-12-07T10:33:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.798+0000 7f8c45b9c700 1 --2- 172.21.15.119:0/2676158433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646bf1e05c0 0x5646bf1e4360 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8c38004600 tx=0x7f8c38004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cba0590ce9eec61e server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.798+0000 7f8c28ff9700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:33:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.799+0000 7f8c28ff9700 1 -- 172.21.15.119:0/2676158433 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3800e040 con 0x5646bf1e05c0 2023-12-07T10:33:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.799+0000 7f8c5af3fb80 1 -- 172.21.15.119:0/2676158433 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5646bf1cd6f0 con 0x5646bf1e05c0 2023-12-07T10:33:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.799+0000 7f8c28ff9700 1 -- 172.21.15.119:0/2676158433 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f8c38016ca0 con 0x5646bf1e05c0 2023-12-07T10:33:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.799+0000 7f8c28ff9700 1 -- 172.21.15.119:0/2676158433 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c38016e60 con 0x5646bf1e05c0 2023-12-07T10:33:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.799+0000 7f8c5af3fb80 10 client.27809 fetch_fsmap learned FSMap version 61 2023-12-07T10:33:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.799+0000 7f8c5af3fb80 1 -- 172.21.15.119:0/2676158433 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x5646bef57d80 con 0x5646bf1e05c0 2023-12-07T10:33:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.799+0000 7f8c28ff9700 1 -- 172.21.15.119:0/2676158433 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5646bf1cd6f0 con 0x5646bf1e05c0 2023-12-07T10:33:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.800+0000 7f8c28ff9700 1 -- 172.21.15.119:0/2676158433 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 61) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f8c38015480 con 0x5646bf1e05c0 2023-12-07T10:33:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.800+0000 7f8c5af3fb80 10 client.27809 fetch_fsmap finished waiting for FSMap version 61 2023-12-07T10:33:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.800+0000 7f8c5af3fb80 10 client.27809 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:33:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.800+0000 7f8c5af3fb80 20 client.27809 populate_metadata read hostname 'smithi119' 2023-12-07T10:33:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.800+0000 7f8c5af3fb80 1 --2- 172.21.15.119:0/2676158433 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5646bf1f4100 0x5646bf1f6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.800+0000 7f8c5af3fb80 4 client.27809 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:33:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.800+0000 7f8c5af3fb80 1 -- 172.21.15.119:0/2676158433 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5646bf1a9740 con 0x5646bf1f4100 2023-12-07T10:33:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.800+0000 7f8c4539b700 1 --2- 172.21.15.119:0/2676158433 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5646bf1f4100 0x5646bf1f6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.801+0000 7f8c4539b700 1 --2- 172.21.15.119:0/2676158433 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5646bf1f4100 0x5646bf1f6520 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29f7ed72802ce8f2 server_cookie=d92a2ae25a25bc38 in_seq=0 out_seq=0 2023-12-07T10:33:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.801+0000 7f8c28ff9700 10 client.27809 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:33:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.802+0000 7f8c28ff9700 1 -- 172.21.15.119:0/2676158433 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5646bf1a9740 con 0x5646bf1f4100 2023-12-07T10:33:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.802+0000 7f8c28ff9700 10 client.27809 handle_command_reply: tid=0 2023-12-07T10:33:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.802+0000 7f8c5af3fb80 1 client.27809 shutdown 2023-12-07T10:33:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.802+0000 7f8c5af3fb80 1 -- 172.21.15.119:0/2676158433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646bf1e05c0 msgr2=0x5646bf1e4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.802+0000 7f8c5af3fb80 1 --2- 172.21.15.119:0/2676158433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646bf1e05c0 0x5646bf1e4360 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8c38004600 tx=0x7f8c38004b40 comp rx=0 tx=0).stop 2023-12-07T10:33:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.802+0000 7f8c5af3fb80 1 -- 172.21.15.119:0/2676158433 shutdown_connections 2023-12-07T10:33:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.802+0000 7f8c5af3fb80 1 --2- 172.21.15.119:0/2676158433 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5646bf1f4100 0x5646bf1f6520 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.802+0000 7f8c5af3fb80 1 --2- 172.21.15.119:0/2676158433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646bf1e05c0 0x5646bf1e4360 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.802+0000 7f8c5af3fb80 1 --2- 172.21.15.119:0/2676158433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646bf1daa80 0x5646bf1e38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.802+0000 7f8c5af3fb80 1 --2- 172.21.15.119:0/2676158433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646bf1dfca0 0x5646bf1e3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.802+0000 7f8c5af3fb80 1 -- 172.21.15.119:0/2676158433 >> 172.21.15.119:0/2676158433 conn(0x5646bf109300 msgr2=0x5646bf1cf980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.802+0000 7f8c28ff9700 0 client.27809 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:33:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.802+0000 7f8c5af3fb80 1 -- 172.21.15.119:0/2676158433 shutdown_connections 2023-12-07T10:33:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.802+0000 7f8c5af3fb80 1 -- 172.21.15.119:0/2676158433 wait complete. 2023-12-07T10:33:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.803+0000 7f8c5af3fb80 20 client.27809 trim_cache size 0 max 16384 2023-12-07T10:33:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.820+0000 7f8c5af3fb80 1 Processor -- start 2023-12-07T10:33:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.820+0000 7f8c5af3fb80 1 -- start start 2023-12-07T10:33:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.820+0000 7f8c5af3fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646bf1daa80 0x5646bf1efa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.820+0000 7f8c5af3fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646bf1dfca0 0x5646bf1df1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.820+0000 7f8c5af3fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646bf1e05c0 0x5646bf1e4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.820+0000 7f8c5af3fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646befad8c0 con 0x5646bf1dfca0 2023-12-07T10:33:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.820+0000 7f8c5af3fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646bf1d7030 con 0x5646bf1daa80 2023-12-07T10:33:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.820+0000 7f8c5af3fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646bf1e4f00 con 0x5646bf1e05c0 2023-12-07T10:33:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.821+0000 7f8c4539b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646bf1daa80 0x5646bf1efa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.821+0000 7f8c4539b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646bf1daa80 0x5646bf1efa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:36586/0 (socket says 172.21.15.119:36586) 2023-12-07T10:33:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.821+0000 7f8c4539b700 1 -- 172.21.15.119:0/3603134850 learned_addr learned my addr 172.21.15.119:0/3603134850 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:33:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.821+0000 7f8c44b9a700 1 --2- 172.21.15.119:0/3603134850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646bf1dfca0 0x5646bf1df1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.821+0000 7f8c45b9c700 1 --2- 172.21.15.119:0/3603134850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646bf1e05c0 0x5646bf1e4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.821+0000 7f8c4539b700 1 -- 172.21.15.119:0/3603134850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646bf1e05c0 msgr2=0x5646bf1e4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.821+0000 7f8c4539b700 1 --2- 172.21.15.119:0/3603134850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646bf1e05c0 0x5646bf1e4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.821+0000 7f8c4539b700 1 -- 172.21.15.119:0/3603134850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646bf1dfca0 msgr2=0x5646bf1df1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.821+0000 7f8c4539b700 1 --2- 172.21.15.119:0/3603134850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646bf1dfca0 0x5646bf1df1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.821+0000 7f8c4539b700 1 -- 172.21.15.119:0/3603134850 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646bef40c00 con 0x5646bf1daa80 2023-12-07T10:33:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.821+0000 7f8c44b9a700 1 --2- 172.21.15.119:0/3603134850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646bf1dfca0 0x5646bf1df1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:33:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.821+0000 7f8c45b9c700 1 --2- 172.21.15.119:0/3603134850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646bf1e05c0 0x5646bf1e4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:33:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.821+0000 7f8c4539b700 1 --2- 172.21.15.119:0/3603134850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646bf1daa80 0x5646bf1efa50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c025040 tx=0x7f8c3c004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25cb5886fb155617 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.822+0000 7f8c2bfff700 1 -- 172.21.15.119:0/3603134850 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c031ea0 con 0x5646bf1daa80 2023-12-07T10:33:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.822+0000 7f8c2bfff700 1 -- 172.21.15.119:0/3603134850 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c3c004e90 con 0x5646bf1daa80 2023-12-07T10:33:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.822+0000 7f8c2bfff700 1 -- 172.21.15.119:0/3603134850 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c031150 con 0x5646bf1daa80 2023-12-07T10:33:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.822+0000 7f8c5af3fb80 1 -- 172.21.15.119:0/3603134850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646bf1daa80 msgr2=0x5646bf1efa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.822+0000 7f8c5af3fb80 1 --2- 172.21.15.119:0/3603134850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646bf1daa80 0x5646bf1efa50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c025040 tx=0x7f8c3c004610 comp rx=0 tx=0).stop 2023-12-07T10:33:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.822+0000 7f8c5af3fb80 1 -- 172.21.15.119:0/3603134850 shutdown_connections 2023-12-07T10:33:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.822+0000 7f8c5af3fb80 1 --2- 172.21.15.119:0/3603134850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646bf1e05c0 0x5646bf1e4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.822+0000 7f8c5af3fb80 1 --2- 172.21.15.119:0/3603134850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646bf1daa80 0x5646bf1efa50 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.822+0000 7f8c5af3fb80 1 --2- 172.21.15.119:0/3603134850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646bf1dfca0 0x5646bf1df1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.822+0000 7f8c5af3fb80 1 -- 172.21.15.119:0/3603134850 >> 172.21.15.119:0/3603134850 conn(0x5646bf109300 msgr2=0x5646bf1deaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.823+0000 7f8c5af3fb80 1 -- 172.21.15.119:0/3603134850 shutdown_connections 2023-12-07T10:33:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.823+0000 7f8c5af3fb80 1 -- 172.21.15.119:0/3603134850 wait complete. 2023-12-07T10:33:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.823+0000 7f8c5af3fb80 1 -- start start 2023-12-07T10:33:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.823+0000 7f8c5af3fb80 1 Processor -- start 2023-12-07T10:33:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.824+0000 7f8c5af3fb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646bf1daa80 0x5646bf1eb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.824+0000 7f8c5af3fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646bf1dfca0 0x5646bf1e85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.824+0000 7f8c5af3fb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646bf1e05c0 0x5646bf1e8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.824+0000 7f8c5af3fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646bf1e9200 con 0x5646bf1dfca0 2023-12-07T10:33:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.824+0000 7f8c5af3fb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646bf1e9370 con 0x5646bf1e05c0 2023-12-07T10:33:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.824+0000 7f8c5af3fb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646bf249620 con 0x5646bf1daa80 2023-12-07T10:33:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.824+0000 7f8c5af3fb80 10 client.0 _ll_register_callbacks cb 0x5646bf0cf8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:33:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.824+0000 7f8c45b9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646bf1e05c0 0x5646bf1e8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.824+0000 7f8c45b9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646bf1e05c0 0x5646bf1e8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:36598/0 (socket says 172.21.15.119:36598) 2023-12-07T10:33:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.824+0000 7f8c45b9c700 1 -- 172.21.15.119:0/1482514260 learned_addr learned my addr 172.21.15.119:0/1482514260 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:33:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.824+0000 7f8c4539b700 1 --2- 172.21.15.119:0/1482514260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646bf1daa80 0x5646bf1eb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.825+0000 7f8c44b9a700 1 --2- 172.21.15.119:0/1482514260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646bf1dfca0 0x5646bf1e85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.825+0000 7f8c45b9c700 1 -- 172.21.15.119:0/1482514260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646bf1daa80 msgr2=0x5646bf1eb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.825+0000 7f8c45b9c700 1 --2- 172.21.15.119:0/1482514260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646bf1daa80 0x5646bf1eb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.825+0000 7f8c45b9c700 1 -- 172.21.15.119:0/1482514260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646bf1dfca0 msgr2=0x5646bf1e85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.825+0000 7f8c45b9c700 1 --2- 172.21.15.119:0/1482514260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646bf1dfca0 0x5646bf1e85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.825+0000 7f8c45b9c700 1 -- 172.21.15.119:0/1482514260 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5646befab5d0 con 0x5646bf1e05c0 2023-12-07T10:33:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.825+0000 7f8c44b9a700 1 --2- 172.21.15.119:0/1482514260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646bf1dfca0 0x5646bf1e85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:33:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.825+0000 7f8c4539b700 1 --2- 172.21.15.119:0/1482514260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646bf1daa80 0x5646bf1eb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:33:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.826+0000 7f8c45b9c700 1 --2- 172.21.15.119:0/1482514260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646bf1e05c0 0x5646bf1e8af0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8c38000f80 tx=0x7f8c38012550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac7b117e5589882e server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.826+0000 7f8c28ff9700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:33:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.826+0000 7f8c28ff9700 1 -- 172.21.15.119:0/1482514260 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3800e040 con 0x5646bf1e05c0 2023-12-07T10:33:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.826+0000 7f8c28ff9700 1 -- 172.21.15.119:0/1482514260 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f8c38015bd0 con 0x5646bf1e05c0 2023-12-07T10:33:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.826+0000 7f8c5af3fb80 1 -- 172.21.15.119:0/1482514260 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c3c01e040 con 0x5646bf1e05c0 2023-12-07T10:33:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.826+0000 7f8c28ff9700 1 -- 172.21.15.119:0/1482514260 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c38016aa0 con 0x5646bf1e05c0 2023-12-07T10:33:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.826+0000 7f8c5af3fb80 10 client.27815 fetch_fsmap learned FSMap version 61 2023-12-07T10:33:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.826+0000 7f8c5af3fb80 1 -- 172.21.15.119:0/1482514260 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x5646bef7e1f0 con 0x5646bf1e05c0 2023-12-07T10:33:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.827+0000 7f8c28ff9700 1 -- 172.21.15.119:0/1482514260 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c3c01e040 con 0x5646bf1e05c0 2023-12-07T10:33:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.827+0000 7f8c28ff9700 1 -- 172.21.15.119:0/1482514260 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 61) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f8c38008b50 con 0x5646bf1e05c0 2023-12-07T10:33:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.827+0000 7f8c5af3fb80 10 client.27815 fetch_fsmap finished waiting for FSMap version 61 2023-12-07T10:33:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.827+0000 7f8c5af3fb80 10 client.27815 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:33:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.827+0000 7f8c5af3fb80 20 client.27815 populate_metadata read hostname 'smithi119' 2023-12-07T10:33:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.827+0000 7f8c5af3fb80 1 --2- 172.21.15.119:0/1482514260 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5646bf24ac60 0x5646bf24d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.827+0000 7f8c5af3fb80 4 client.27815 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:33:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.827+0000 7f8c5af3fb80 1 -- 172.21.15.119:0/1482514260 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5646bf0f4f70 con 0x5646bf24ac60 2023-12-07T10:33:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.827+0000 7f8c4539b700 1 --2- 172.21.15.119:0/1482514260 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5646bf24ac60 0x5646bf24d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.828+0000 7f8c28ff9700 10 client.27815 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:33:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.828+0000 7f8c4539b700 1 --2- 172.21.15.119:0/1482514260 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5646bf24ac60 0x5646bf24d080 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a12edf2613bfa582 server_cookie=c925a7ede9d4b39d in_seq=0 out_seq=0 2023-12-07T10:33:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.890+0000 7f8c28ff9700 1 -- 172.21.15.119:0/1482514260 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5646bf0f4f70 con 0x5646bf24ac60 2023-12-07T10:33:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.890+0000 7f8c28ff9700 10 client.27815 handle_command_reply: tid=0 2023-12-07T10:33:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.890+0000 7f8c5af3fb80 1 client.27815 shutdown 2023-12-07T10:33:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.890+0000 7f8c5af3fb80 1 -- 172.21.15.119:0/1482514260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646bf1e05c0 msgr2=0x5646bf1e8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.890+0000 7f8c5af3fb80 1 --2- 172.21.15.119:0/1482514260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646bf1e05c0 0x5646bf1e8af0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8c38000f80 tx=0x7f8c38012550 comp rx=0 tx=0).stop 2023-12-07T10:33:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.890+0000 7f8c5af3fb80 1 -- 172.21.15.119:0/1482514260 shutdown_connections 2023-12-07T10:33:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.890+0000 7f8c5af3fb80 1 --2- 172.21.15.119:0/1482514260 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5646bf24ac60 0x5646bf24d080 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.890+0000 7f8c5af3fb80 1 --2- 172.21.15.119:0/1482514260 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5646bf1e05c0 0x5646bf1e8af0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.890+0000 7f8c5af3fb80 1 --2- 172.21.15.119:0/1482514260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646bf1daa80 0x5646bf1eb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.890+0000 7f8c5af3fb80 1 --2- 172.21.15.119:0/1482514260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646bf1dfca0 0x5646bf1e85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.891+0000 7f8c5af3fb80 1 -- 172.21.15.119:0/1482514260 >> 172.21.15.119:0/1482514260 conn(0x5646bf109300 msgr2=0x5646bf1e41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.891+0000 7f8c28ff9700 0 client.27815 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:33:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.891+0000 7f8c5af3fb80 1 -- 172.21.15.119:0/1482514260 shutdown_connections 2023-12-07T10:33:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.891+0000 7f8c5af3fb80 1 -- 172.21.15.119:0/1482514260 wait complete. 2023-12-07T10:33:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.891+0000 7f8c5af3fb80 20 client.27815 trim_cache size 0 max 16384 2023-12-07T10:33:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:33:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:33:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:33:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:33:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.892+0000 7f8c4639d700 1 -- 172.21.15.119:0/2511053510 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8c2c0a1c80 msgr2=0x7f8c2c0a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.892+0000 7f8c4639d700 1 --2- 172.21.15.119:0/2511053510 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8c2c0a1c80 0x7f8c2c0a4140 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8c400740b0 tx=0x7f8c3800b040 comp rx=0 tx=0).stop 2023-12-07T10:33:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.892+0000 7f8c4639d700 1 -- 172.21.15.119:0/2511053510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c40102270 msgr2=0x7f8c40199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.892+0000 7f8c4639d700 1 --2- 172.21.15.119:0/2511053510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c40102270 0x7f8c40199fb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000e9a0 tx=0x7f8c3000ecb0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.892+0000 7f8c4639d700 1 -- 172.21.15.119:0/2511053510 shutdown_connections 2023-12-07T10:33:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.892+0000 7f8c4639d700 1 --2- 172.21.15.119:0/2511053510 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8c2c0a1c80 0x7f8c2c0a4140 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.892+0000 7f8c4639d700 1 --2- 172.21.15.119:0/2511053510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c4010dc10 0x7f8c4019e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.892+0000 7f8c4639d700 1 --2- 172.21.15.119:0/2511053510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c40068e20 0x7f8c40199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.892+0000 7f8c4639d700 1 --2- 172.21.15.119:0/2511053510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c40102270 0x7f8c40199fb0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.892+0000 7f8c4639d700 1 -- 172.21.15.119:0/2511053510 >> 172.21.15.119:0/2511053510 conn(0x7f8c40075e70 msgr2=0x7f8c4010ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.892+0000 7f8c4639d700 1 -- 172.21.15.119:0/2511053510 shutdown_connections 2023-12-07T10:33:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:03.892+0000 7f8c4639d700 1 -- 172.21.15.119:0/2511053510 wait complete. 2023-12-07T10:33:04.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:04 smithi119 ceph-mon[97908]: pgmap v394: 129 pgs: 129 active+clean; 737 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 199 op/s 2023-12-07T10:33:04.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:04 smithi159 ceph-mon[134726]: pgmap v394: 129 pgs: 129 active+clean; 737 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 199 op/s 2023-12-07T10:33:04.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:04 smithi153 ceph-mon[133625]: pgmap v394: 129 pgs: 129 active+clean; 737 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 199 op/s 2023-12-07T10:33:04.581 INFO:tasks.workunit.client.0.smithi119.stdout: 1 933873 7.28 MB/sec execute 557 sec latency 60.736 ms 2023-12-07T10:33:05.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:05 smithi119 ceph-mon[97908]: osdmap e107: 12 total, 12 up, 12 in 2023-12-07T10:33:05.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:05 smithi159 ceph-mon[134726]: osdmap e107: 12 total, 12 up, 12 in 2023-12-07T10:33:05.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:05 smithi153 ceph-mon[133625]: osdmap e107: 12 total, 12 up, 12 in 2023-12-07T10:33:05.581 INFO:tasks.workunit.client.0.smithi119.stdout: 1 935117 7.28 MB/sec execute 558 sec latency 15.938 ms 2023-12-07T10:33:06.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:06 smithi119 ceph-mon[97908]: pgmap v396: 129 pgs: 129 active+clean; 737 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 223 op/s 2023-12-07T10:33:06.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:06 smithi159 ceph-mon[134726]: pgmap v396: 129 pgs: 129 active+clean; 737 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 223 op/s 2023-12-07T10:33:06.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:06 smithi153 ceph-mon[133625]: pgmap v396: 129 pgs: 129 active+clean; 737 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 223 op/s 2023-12-07T10:33:06.581 INFO:tasks.workunit.client.0.smithi119.stdout: 1 936926 7.28 MB/sec execute 559 sec latency 14.064 ms 2023-12-07T10:33:07.581 INFO:tasks.workunit.client.0.smithi119.stdout: 1 938416 7.28 MB/sec execute 560 sec latency 14.417 ms 2023-12-07T10:33:08.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:08 smithi119 ceph-mon[97908]: pgmap v397: 129 pgs: 129 active+clean; 719 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 235 op/s 2023-12-07T10:33:08.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:08 smithi159 ceph-mon[134726]: pgmap v397: 129 pgs: 129 active+clean; 719 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 235 op/s 2023-12-07T10:33:08.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:08 smithi153 ceph-mon[133625]: pgmap v397: 129 pgs: 129 active+clean; 719 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 235 op/s 2023-12-07T10:33:08.581 INFO:tasks.workunit.client.0.smithi119.stdout: 1 939707 7.28 MB/sec execute 561 sec latency 29.285 ms 2023-12-07T10:33:09.581 INFO:tasks.workunit.client.0.smithi119.stdout: 1 941643 7.29 MB/sec execute 562 sec latency 25.828 ms 2023-12-07T10:33:10.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:10 smithi119 ceph-mon[97908]: pgmap v398: 129 pgs: 129 active+clean; 719 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 235 op/s 2023-12-07T10:33:10.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:10 smithi159 ceph-mon[134726]: pgmap v398: 129 pgs: 129 active+clean; 719 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 235 op/s 2023-12-07T10:33:10.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:10 smithi153 ceph-mon[133625]: pgmap v398: 129 pgs: 129 active+clean; 719 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 235 op/s 2023-12-07T10:33:10.581 INFO:tasks.workunit.client.0.smithi119.stdout: 1 942635 7.28 MB/sec execute 563 sec latency 13.572 ms 2023-12-07T10:33:11.581 INFO:tasks.workunit.client.0.smithi119.stdout: 1 944493 7.29 MB/sec execute 564 sec latency 19.277 ms 2023-12-07T10:33:12.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:11 smithi119 ceph-mon[97908]: osdmap e108: 12 total, 12 up, 12 in 2023-12-07T10:33:12.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:11 smithi119 ceph-mon[97908]: pgmap v400: 129 pgs: 129 active+clean; 726 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 257 op/s 2023-12-07T10:33:12.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:11 smithi159 ceph-mon[134726]: osdmap e108: 12 total, 12 up, 12 in 2023-12-07T10:33:12.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:11 smithi159 ceph-mon[134726]: pgmap v400: 129 pgs: 129 active+clean; 726 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 257 op/s 2023-12-07T10:33:12.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:11 smithi153 ceph-mon[133625]: osdmap e108: 12 total, 12 up, 12 in 2023-12-07T10:33:12.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:11 smithi153 ceph-mon[133625]: pgmap v400: 129 pgs: 129 active+clean; 726 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 257 op/s 2023-12-07T10:33:12.581 INFO:tasks.workunit.client.0.smithi119.stdout: 1 945724 7.29 MB/sec execute 565 sec latency 14.287 ms 2023-12-07T10:33:13.581 INFO:tasks.workunit.client.0.smithi119.stdout: 1 947380 7.28 MB/sec execute 566 sec latency 33.119 ms 2023-12-07T10:33:14.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:14 smithi119 ceph-mon[97908]: pgmap v401: 129 pgs: 129 active+clean; 727 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 245 op/s 2023-12-07T10:33:14.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:14 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:33:14.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:14 smithi159 ceph-mon[134726]: pgmap v401: 129 pgs: 129 active+clean; 727 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 245 op/s 2023-12-07T10:33:14.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:14 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:33:14.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:14 smithi153 ceph-mon[133625]: pgmap v401: 129 pgs: 129 active+clean; 727 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 245 op/s 2023-12-07T10:33:14.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:14 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:33:14.581 INFO:tasks.workunit.client.0.smithi119.stdout: 1 948861 7.29 MB/sec execute 567 sec latency 86.000 ms 2023-12-07T10:33:15.582 INFO:tasks.workunit.client.0.smithi119.stdout: 1 949891 7.29 MB/sec execute 568 sec latency 18.923 ms 2023-12-07T10:33:16.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:16 smithi119 ceph-mon[97908]: pgmap v402: 129 pgs: 129 active+clean; 727 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 219 op/s 2023-12-07T10:33:16.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:16 smithi159 ceph-mon[134726]: pgmap v402: 129 pgs: 129 active+clean; 727 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 219 op/s 2023-12-07T10:33:16.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:16 smithi153 ceph-mon[133625]: pgmap v402: 129 pgs: 129 active+clean; 727 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 219 op/s 2023-12-07T10:33:16.582 INFO:tasks.workunit.client.0.smithi119.stdout: 1 951048 7.28 MB/sec execute 569 sec latency 186.329 ms 2023-12-07T10:33:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.106+0000 7f011cb9d700 1 -- 172.21.15.119:0/3434627761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0118074300 msgr2=0x7f011810d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.106+0000 7f011cb9d700 1 --2- 172.21.15.119:0/3434627761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0118074300 0x7f011810d680 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f010c009990 tx=0x7f010c009ca0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.106+0000 7f011cb9d700 1 -- 172.21.15.119:0/3434627761 shutdown_connections 2023-12-07T10:33:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.106+0000 7f011cb9d700 1 --2- 172.21.15.119:0/3434627761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f011810dbc0 0x7f011810fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.106+0000 7f011cb9d700 1 --2- 172.21.15.119:0/3434627761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01180739e0 0x7f0118073dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.106+0000 7f011cb9d700 1 --2- 172.21.15.119:0/3434627761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0118074300 0x7f011810d680 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.106+0000 7f011cb9d700 1 -- 172.21.15.119:0/3434627761 >> 172.21.15.119:0/3434627761 conn(0x7f01180fce30 msgr2=0x7f01180ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.106+0000 7f011cb9d700 1 -- 172.21.15.119:0/3434627761 shutdown_connections 2023-12-07T10:33:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.106+0000 7f011cb9d700 1 -- 172.21.15.119:0/3434627761 wait complete. 2023-12-07T10:33:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.108+0000 7f011cb9d700 1 Processor -- start 2023-12-07T10:33:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.108+0000 7f011cb9d700 1 -- start start 2023-12-07T10:33:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.108+0000 7f011cb9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01180739e0 0x7f011819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.108+0000 7f011cb9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0118074300 0x7f011819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.108+0000 7f011cb9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f011810dbc0 0x7f01181998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.108+0000 7f011cb9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0118199f50 con 0x7f0118074300 2023-12-07T10:33:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.108+0000 7f011cb9d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011819a0c0 con 0x7f01180739e0 2023-12-07T10:33:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.108+0000 7f011cb9d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011819a280 con 0x7f011810dbc0 2023-12-07T10:33:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.108+0000 7f01177fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01180739e0 0x7f011819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.108+0000 7f0117fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f011810dbc0 0x7f01181998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.109+0000 7f01177fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01180739e0 0x7f011819f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:35200/0 (socket says 172.21.15.119:35200) 2023-12-07T10:33:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.109+0000 7f01177fe700 1 -- 172.21.15.119:0/3231912019 learned_addr learned my addr 172.21.15.119:0/3231912019 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:33:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.109+0000 7f0116ffd700 1 --2- 172.21.15.119:0/3231912019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0118074300 0x7f011819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:17.110 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.109+0000 7f0117fff700 1 -- 172.21.15.119:0/3231912019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01180739e0 msgr2=0x7f011819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.109+0000 7f0117fff700 1 --2- 172.21.15.119:0/3231912019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01180739e0 0x7f011819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.109+0000 7f0117fff700 1 -- 172.21.15.119:0/3231912019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0118074300 msgr2=0x7f011819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.109+0000 7f0117fff700 1 --2- 172.21.15.119:0/3231912019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0118074300 0x7f011819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.109+0000 7f0117fff700 1 -- 172.21.15.119:0/3231912019 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010c0095f0 con 0x7f011810dbc0 2023-12-07T10:33:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.109+0000 7f01177fe700 1 --2- 172.21.15.119:0/3231912019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01180739e0 0x7f011819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:33:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.109+0000 7f0117fff700 1 --2- 172.21.15.119:0/3231912019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f011810dbc0 0x7f01181998f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f011000ef90 tx=0x7f011000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.109+0000 7f0114ff9700 1 -- 172.21.15.119:0/3231912019 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0110018070 con 0x7f011810dbc0 2023-12-07T10:33:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.109+0000 7f0114ff9700 1 -- 172.21.15.119:0/3231912019 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f011000f040 con 0x7f011810dbc0 2023-12-07T10:33:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.109+0000 7f0114ff9700 1 -- 172.21.15.119:0/3231912019 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011000cc30 con 0x7f011810dbc0 2023-12-07T10:33:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.109+0000 7f011cb9d700 1 -- 172.21.15.119:0/3231912019 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f011819a500 con 0x7f011810dbc0 2023-12-07T10:33:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.110+0000 7f011cb9d700 1 -- 172.21.15.119:0/3231912019 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01181a67b0 con 0x7f011810dbc0 2023-12-07T10:33:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.114+0000 7f0114ff9700 1 -- 172.21.15.119:0/3231912019 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f0110003a00 con 0x7f011810dbc0 2023-12-07T10:33:17.115 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.115+0000 7f01317dab80 1 Processor -- start 2023-12-07T10:33:17.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.116+0000 7f01317dab80 1 -- start start 2023-12-07T10:33:17.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.116+0000 7f0114ff9700 1 --2- 172.21.15.119:0/3231912019 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f01000a1d00 0x7f01000a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:17.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.116+0000 7f01317dab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b2374fca0 0x556b23750080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:17.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.116+0000 7f01317dab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b237505c0 0x556b2374a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.116+0000 7f01317dab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b2374aa80 0x556b2374af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.116+0000 7f01317dab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b236be760 con 0x556b237505c0 2023-12-07T10:33:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.116+0000 7f01317dab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b236a5740 con 0x556b2374fca0 2023-12-07T10:33:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.116+0000 7f01317dab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b236a3060 con 0x556b2374aa80 2023-12-07T10:33:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.116+0000 7f0114ff9700 1 -- 172.21.15.119:0/3231912019 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f01100cd800 con 0x7f011810dbc0 2023-12-07T10:33:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.116+0000 7f01177fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b2374fca0 0x556b23750080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.116+0000 7f01177fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b2374fca0 0x556b23750080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:35210/0 (socket says 172.21.15.119:35210) 2023-12-07T10:33:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.116+0000 7f01177fe700 1 -- 172.21.15.119:0/2133491134 learned_addr learned my addr 172.21.15.119:0/2133491134 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:33:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.117+0000 7f0117fff700 1 --2- 172.21.15.119:0/2133491134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b2374aa80 0x556b2374af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.117+0000 7f0116ffd700 1 --2- 172.21.15.119:0/3231912019 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f01000a1d00 0x7f01000a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.117+0000 7f01177fe700 1 -- 172.21.15.119:0/2133491134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b2374aa80 msgr2=0x556b2374af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.117+0000 7f01177fe700 1 --2- 172.21.15.119:0/2133491134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b2374aa80 0x556b2374af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.117+0000 7f01177fe700 1 -- 172.21.15.119:0/2133491134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b237505c0 msgr2=0x556b2374a540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-07T10:33:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.117+0000 7f01177fe700 1 --2- 172.21.15.119:0/2133491134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b237505c0 0x556b2374a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.117+0000 7f01177fe700 1 -- 172.21.15.119:0/2133491134 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b2351b5d0 con 0x556b2374fca0 2023-12-07T10:33:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.117+0000 7f0117fff700 1 --2- 172.21.15.119:0/2133491134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b2374aa80 0x556b2374af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:33:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.117+0000 7f01177fe700 1 --2- 172.21.15.119:0/2133491134 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b2374fca0 0x556b23750080 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f01080117e0 tx=0x7f0108011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed7f23f8362c136f server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.117+0000 7f00fdffb700 1 -- 172.21.15.119:0/2133491134 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010800f480 con 0x556b2374fca0 2023-12-07T10:33:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.117+0000 7f00fdffb700 1 -- 172.21.15.119:0/2133491134 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f010800faa0 con 0x556b2374fca0 2023-12-07T10:33:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.117+0000 7f00fdffb700 1 -- 172.21.15.119:0/2133491134 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0108019b60 con 0x556b2374fca0 2023-12-07T10:33:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.118+0000 7f01317dab80 1 -- 172.21.15.119:0/2133491134 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b2374fca0 msgr2=0x556b23750080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.118+0000 7f0116ffd700 1 --2- 172.21.15.119:0/3231912019 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f01000a1d00 0x7f01000a41c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f010c005b40 tx=0x7f010c01a040 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.118+0000 7f01317dab80 1 --2- 172.21.15.119:0/2133491134 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b2374fca0 0x556b23750080 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f01080117e0 tx=0x7f0108011af0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.118+0000 7f01317dab80 1 -- 172.21.15.119:0/2133491134 shutdown_connections 2023-12-07T10:33:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.118+0000 7f01317dab80 1 --2- 172.21.15.119:0/2133491134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b2374aa80 0x556b2374af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.118+0000 7f01317dab80 1 --2- 172.21.15.119:0/2133491134 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b2374fca0 0x556b23750080 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.118+0000 7f01317dab80 1 --2- 172.21.15.119:0/2133491134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b237505c0 0x556b2374a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.118+0000 7f01317dab80 1 -- 172.21.15.119:0/2133491134 >> 172.21.15.119:0/2133491134 conn(0x556b23679300 msgr2=0x556b23744bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.118+0000 7f01317dab80 1 -- 172.21.15.119:0/2133491134 shutdown_connections 2023-12-07T10:33:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.118+0000 7f01317dab80 1 -- 172.21.15.119:0/2133491134 wait complete. 2023-12-07T10:33:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.119+0000 7f01317dab80 1 -- start start 2023-12-07T10:33:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.119+0000 7f01317dab80 1 Processor -- start 2023-12-07T10:33:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.120+0000 7f01317dab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b2374aa80 0x556b237564d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.120+0000 7f01317dab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b2374fca0 0x556b237537c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.120+0000 7f01317dab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b237505c0 0x556b23753e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.120+0000 7f01317dab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b2369e950 con 0x556b237505c0 2023-12-07T10:33:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.120+0000 7f01317dab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b23658500 con 0x556b2374fca0 2023-12-07T10:33:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.120+0000 7f01317dab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b236581a0 con 0x556b2374aa80 2023-12-07T10:33:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.120+0000 7f01317dab80 10 client.0 _ll_register_callbacks cb 0x556b2363f8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:33:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.120+0000 7f01177fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b2374aa80 0x556b237564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.120+0000 7f01177fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b2374aa80 0x556b237564d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:47464/0 (socket says 172.21.15.119:47464) 2023-12-07T10:33:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.120+0000 7f01177fe700 1 -- 172.21.15.119:0/799372832 learned_addr learned my addr 172.21.15.119:0/799372832 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:33:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.120+0000 7f0116ffd700 1 --2- 172.21.15.119:0/799372832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b2374fca0 0x556b237537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.120+0000 7f0117fff700 1 --2- 172.21.15.119:0/799372832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b237505c0 0x556b23753e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.120+0000 7f01177fe700 1 -- 172.21.15.119:0/799372832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b2374fca0 msgr2=0x556b237537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.120+0000 7f01177fe700 1 --2- 172.21.15.119:0/799372832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b2374fca0 0x556b237537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.120+0000 7f01177fe700 1 -- 172.21.15.119:0/799372832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b237505c0 msgr2=0x556b23753e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.120+0000 7f01177fe700 1 --2- 172.21.15.119:0/799372832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b237505c0 0x556b23753e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.120+0000 7f01177fe700 1 -- 172.21.15.119:0/799372832 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0108011440 con 0x556b2374aa80 2023-12-07T10:33:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.121+0000 7f01177fe700 1 --2- 172.21.15.119:0/799372832 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b2374aa80 0x556b237564d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f01080196e0 tx=0x7f0108005d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5867833286102560 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.121+0000 7f00f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:33:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.121+0000 7f00f6ffd700 1 -- 172.21.15.119:0/799372832 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010800f790 con 0x556b2374aa80 2023-12-07T10:33:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.121+0000 7f00f6ffd700 1 -- 172.21.15.119:0/799372832 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f01080197e0 con 0x556b2374aa80 2023-12-07T10:33:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.121+0000 7f01317dab80 1 -- 172.21.15.119:0/799372832 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556b2373d6f0 con 0x556b2374aa80 2023-12-07T10:33:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.121+0000 7f00f6ffd700 1 -- 172.21.15.119:0/799372832 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f010800f8f0 con 0x556b2374aa80 2023-12-07T10:33:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.121+0000 7f01317dab80 10 client.27808 fetch_fsmap learned FSMap version 62 2023-12-07T10:33:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.121+0000 7f01317dab80 1 -- 172.21.15.119:0/799372832 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x556b234c7d80 con 0x556b2374aa80 2023-12-07T10:33:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.122+0000 7f00f6ffd700 1 -- 172.21.15.119:0/799372832 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556b2373d6f0 con 0x556b2374aa80 2023-12-07T10:33:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.122+0000 7f00f6ffd700 1 -- 172.21.15.119:0/799372832 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 62) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f0108004750 con 0x556b2374aa80 2023-12-07T10:33:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.122+0000 7f01317dab80 10 client.27808 fetch_fsmap finished waiting for FSMap version 62 2023-12-07T10:33:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.122+0000 7f01317dab80 10 client.27808 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:33:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.122+0000 7f01317dab80 20 client.27808 populate_metadata read hostname 'smithi119' 2023-12-07T10:33:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.122+0000 7f01317dab80 1 --2- 172.21.15.119:0/799372832 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556b23763f70 0x556b23766390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.122+0000 7f01317dab80 4 client.27808 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:33:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.122+0000 7f01317dab80 1 -- 172.21.15.119:0/799372832 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b23719740 con 0x556b23763f70 2023-12-07T10:33:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.122+0000 7f0116ffd700 1 --2- 172.21.15.119:0/799372832 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556b23763f70 0x556b23766390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.123+0000 7f00f6ffd700 10 client.27808 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:33:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.123+0000 7f0116ffd700 1 --2- 172.21.15.119:0/799372832 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556b23763f70 0x556b23766390 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4f8734bd2d4b353 server_cookie=547482658868ff11 in_seq=0 out_seq=0 2023-12-07T10:33:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.124+0000 7f00f6ffd700 1 -- 172.21.15.119:0/799372832 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556b23719740 con 0x556b23763f70 2023-12-07T10:33:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.124+0000 7f00f6ffd700 10 client.27808 handle_command_reply: tid=0 2023-12-07T10:33:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.124+0000 7f01317dab80 1 client.27808 shutdown 2023-12-07T10:33:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.124+0000 7f01317dab80 1 -- 172.21.15.119:0/799372832 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b2374aa80 msgr2=0x556b237564d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.124+0000 7f01317dab80 1 --2- 172.21.15.119:0/799372832 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b2374aa80 0x556b237564d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f01080196e0 tx=0x7f0108005d70 comp rx=0 tx=0).stop 2023-12-07T10:33:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.124+0000 7f01317dab80 1 -- 172.21.15.119:0/799372832 shutdown_connections 2023-12-07T10:33:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.124+0000 7f01317dab80 1 --2- 172.21.15.119:0/799372832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b237505c0 0x556b23753e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.124+0000 7f01317dab80 1 --2- 172.21.15.119:0/799372832 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b2374aa80 0x556b237564d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.124+0000 7f01317dab80 1 --2- 172.21.15.119:0/799372832 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556b23763f70 0x556b23766390 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.124+0000 7f01317dab80 1 --2- 172.21.15.119:0/799372832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b2374fca0 0x556b237537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.124+0000 7f01317dab80 1 -- 172.21.15.119:0/799372832 >> 172.21.15.119:0/799372832 conn(0x556b23679300 msgr2=0x556b2373f980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.124+0000 7f00f6ffd700 0 client.27808 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:33:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.124+0000 7f01317dab80 1 -- 172.21.15.119:0/799372832 shutdown_connections 2023-12-07T10:33:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.124+0000 7f01317dab80 1 -- 172.21.15.119:0/799372832 wait complete. 2023-12-07T10:33:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.124+0000 7f01317dab80 20 client.27808 trim_cache size 0 max 16384 2023-12-07T10:33:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.141+0000 7f01317dab80 1 Processor -- start 2023-12-07T10:33:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.141+0000 7f01317dab80 1 -- start start 2023-12-07T10:33:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.142+0000 7f01317dab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b2374aa80 0x556b2375fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.142+0000 7f01317dab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b2374fca0 0x556b2375fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.142+0000 7f01317dab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b237505c0 0x556b2374da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.142+0000 7f01317dab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b2351d8c0 con 0x556b2374aa80 2023-12-07T10:33:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.142+0000 7f01317dab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b2373d6d0 con 0x556b2374fca0 2023-12-07T10:33:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.142+0000 7f01317dab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b23741790 con 0x556b237505c0 2023-12-07T10:33:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.142+0000 7f01177fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b2374aa80 0x556b2375fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.142+0000 7f0117fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b237505c0 0x556b2374da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.142+0000 7f0116ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b2374fca0 0x556b2375fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.142+0000 7f01177fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b2374aa80 0x556b2375fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53544/0 (socket says 172.21.15.119:53544) 2023-12-07T10:33:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.142+0000 7f0117fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b237505c0 0x556b2374da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:47466/0 (socket says 172.21.15.119:47466) 2023-12-07T10:33:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.142+0000 7f01177fe700 1 -- 172.21.15.119:0/3123401108 learned_addr learned my addr 172.21.15.119:0/3123401108 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:33:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.142+0000 7f01177fe700 1 -- 172.21.15.119:0/3123401108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b237505c0 msgr2=0x556b2374da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.142+0000 7f01177fe700 1 --2- 172.21.15.119:0/3123401108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b237505c0 0x556b2374da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.142+0000 7f01177fe700 1 -- 172.21.15.119:0/3123401108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b2374fca0 msgr2=0x556b2375fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.142+0000 7f01177fe700 1 --2- 172.21.15.119:0/3123401108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b2374fca0 0x556b2375fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.142+0000 7f01177fe700 1 -- 172.21.15.119:0/3123401108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b234b0c00 con 0x556b2374aa80 2023-12-07T10:33:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.142+0000 7f0117fff700 1 --2- 172.21.15.119:0/3123401108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b237505c0 0x556b2374da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:33:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.142+0000 7f0116ffd700 1 --2- 172.21.15.119:0/3123401108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b2374fca0 0x556b2375fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:33:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.143+0000 7f01177fe700 1 --2- 172.21.15.119:0/3123401108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b2374aa80 0x556b2375fab0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0108011730 tx=0x7f0108011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2f96a735dbada5 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.143+0000 7f00fe7fc700 1 -- 172.21.15.119:0/3123401108 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0108005040 con 0x556b2374aa80 2023-12-07T10:33:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.143+0000 7f00fe7fc700 1 -- 172.21.15.119:0/3123401108 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0108005660 con 0x556b2374aa80 2023-12-07T10:33:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.143+0000 7f00fe7fc700 1 -- 172.21.15.119:0/3123401108 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0108003680 con 0x556b2374aa80 2023-12-07T10:33:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.144+0000 7f01317dab80 1 -- 172.21.15.119:0/3123401108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b2374aa80 msgr2=0x556b2375fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.144+0000 7f01317dab80 1 --2- 172.21.15.119:0/3123401108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b2374aa80 0x556b2375fab0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0108011730 tx=0x7f0108011af0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.144+0000 7f01317dab80 1 -- 172.21.15.119:0/3123401108 shutdown_connections 2023-12-07T10:33:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.144+0000 7f01317dab80 1 --2- 172.21.15.119:0/3123401108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b237505c0 0x556b2374da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.144+0000 7f01317dab80 1 --2- 172.21.15.119:0/3123401108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b2374aa80 0x556b2375fab0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.144+0000 7f01317dab80 1 --2- 172.21.15.119:0/3123401108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b2374fca0 0x556b2375fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.144+0000 7f01317dab80 1 -- 172.21.15.119:0/3123401108 >> 172.21.15.119:0/3123401108 conn(0x556b23679300 msgr2=0x556b23756200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.144+0000 7f01317dab80 1 -- 172.21.15.119:0/3123401108 shutdown_connections 2023-12-07T10:33:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.144+0000 7f01317dab80 1 -- 172.21.15.119:0/3123401108 wait complete. 2023-12-07T10:33:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.144+0000 7f01317dab80 1 -- start start 2023-12-07T10:33:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.145+0000 7f01317dab80 1 Processor -- start 2023-12-07T10:33:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.145+0000 7f01317dab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b2374aa80 0x556b2375b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.145+0000 7f01317dab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b2374fca0 0x556b2374b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.145+0000 7f01317dab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b237505c0 0x556b2374bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.145+0000 7f01317dab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b2374c370 con 0x556b2374aa80 2023-12-07T10:33:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.145+0000 7f01317dab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b2374c4e0 con 0x556b237505c0 2023-12-07T10:33:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.145+0000 7f01317dab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b2374c6a0 con 0x556b2374fca0 2023-12-07T10:33:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.145+0000 7f01317dab80 10 client.0 _ll_register_callbacks cb 0x556b2363f8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:33:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.146+0000 7f0116ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b2374fca0 0x556b2374b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.146+0000 7f0116ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b2374fca0 0x556b2374b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:47472/0 (socket says 172.21.15.119:47472) 2023-12-07T10:33:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.146+0000 7f0116ffd700 1 -- 172.21.15.119:0/45165536 learned_addr learned my addr 172.21.15.119:0/45165536 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:33:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.146+0000 7f0117fff700 1 --2- 172.21.15.119:0/45165536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b237505c0 0x556b2374bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.146+0000 7f01177fe700 1 --2- 172.21.15.119:0/45165536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b2374aa80 0x556b2375b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.146+0000 7f0116ffd700 1 -- 172.21.15.119:0/45165536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b237505c0 msgr2=0x556b2374bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.146+0000 7f0116ffd700 1 --2- 172.21.15.119:0/45165536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b237505c0 0x556b2374bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.146+0000 7f0116ffd700 1 -- 172.21.15.119:0/45165536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b2374aa80 msgr2=0x556b2375b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.146+0000 7f0116ffd700 1 --2- 172.21.15.119:0/45165536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b2374aa80 0x556b2375b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.146+0000 7f0116ffd700 1 -- 172.21.15.119:0/45165536 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0118110d00 con 0x556b2374fca0 2023-12-07T10:33:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.146+0000 7f01177fe700 1 --2- 172.21.15.119:0/45165536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b2374aa80 0x556b2375b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:33:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.146+0000 7f0117fff700 1 --2- 172.21.15.119:0/45165536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b237505c0 0x556b2374bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:33:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.146+0000 7f0116ffd700 1 --2- 172.21.15.119:0/45165536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b2374fca0 0x556b2374b760 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f010c000c00 tx=0x7f010c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=919efedff464adaa server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.146+0000 7f00f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:33:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.148+0000 7f00f6ffd700 1 -- 172.21.15.119:0/45165536 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c02c070 con 0x556b2374fca0 2023-12-07T10:33:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.148+0000 7f00f6ffd700 1 -- 172.21.15.119:0/45165536 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f010c00f460 con 0x556b2374fca0 2023-12-07T10:33:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.148+0000 7f01317dab80 1 -- 172.21.15.119:0/45165536 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f010800faa0 con 0x556b2374fca0 2023-12-07T10:33:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.149+0000 7f00f6ffd700 1 -- 172.21.15.119:0/45165536 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f010c00f620 con 0x556b2374fca0 2023-12-07T10:33:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.149+0000 7f01317dab80 10 client.27814 fetch_fsmap learned FSMap version 62 2023-12-07T10:33:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.149+0000 7f01317dab80 1 -- 172.21.15.119:0/45165536 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x7f01181a67b0 con 0x556b2374fca0 2023-12-07T10:33:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.149+0000 7f00f6ffd700 1 -- 172.21.15.119:0/45165536 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f010800faa0 con 0x556b2374fca0 2023-12-07T10:33:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.149+0000 7f00f6ffd700 1 -- 172.21.15.119:0/45165536 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 62) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f010c025760 con 0x556b2374fca0 2023-12-07T10:33:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.149+0000 7f01317dab80 10 client.27814 fetch_fsmap finished waiting for FSMap version 62 2023-12-07T10:33:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.149+0000 7f01317dab80 10 client.27814 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:33:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.149+0000 7f01317dab80 20 client.27814 populate_metadata read hostname 'smithi119' 2023-12-07T10:33:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.149+0000 7f01317dab80 1 --2- 172.21.15.119:0/45165536 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556b237bac20 0x556b237bd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.149+0000 7f01317dab80 4 client.27814 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:33:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.149+0000 7f01317dab80 1 -- 172.21.15.119:0/45165536 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556b23664f70 con 0x556b237bac20 2023-12-07T10:33:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.149+0000 7f01177fe700 1 --2- 172.21.15.119:0/45165536 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556b237bac20 0x556b237bd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.150+0000 7f00f6ffd700 10 client.27814 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:33:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.150+0000 7f01177fe700 1 --2- 172.21.15.119:0/45165536 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556b237bac20 0x556b237bd060 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e10fb8a41a1aa90 server_cookie=e62d2620a3936522 in_seq=0 out_seq=0 2023-12-07T10:33:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.202+0000 7f00f6ffd700 1 -- 172.21.15.119:0/45165536 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556b23664f70 con 0x556b237bac20 2023-12-07T10:33:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.202+0000 7f00f6ffd700 10 client.27814 handle_command_reply: tid=0 2023-12-07T10:33:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.202+0000 7f01317dab80 1 client.27814 shutdown 2023-12-07T10:33:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.202+0000 7f01317dab80 1 -- 172.21.15.119:0/45165536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b2374fca0 msgr2=0x556b2374b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.202+0000 7f01317dab80 1 --2- 172.21.15.119:0/45165536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b2374fca0 0x556b2374b760 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f010c000c00 tx=0x7f010c009310 comp rx=0 tx=0).stop 2023-12-07T10:33:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.203+0000 7f01317dab80 1 -- 172.21.15.119:0/45165536 shutdown_connections 2023-12-07T10:33:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.203+0000 7f01317dab80 1 --2- 172.21.15.119:0/45165536 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556b237bac20 0x556b237bd060 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.203+0000 7f01317dab80 1 --2- 172.21.15.119:0/45165536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556b237505c0 0x556b2374bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.203+0000 7f01317dab80 1 --2- 172.21.15.119:0/45165536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b2374aa80 0x556b2375b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.203+0000 7f01317dab80 1 --2- 172.21.15.119:0/45165536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b2374fca0 0x556b2374b760 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.203+0000 7f00f6ffd700 0 client.27814 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:33:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.203+0000 7f01317dab80 1 -- 172.21.15.119:0/45165536 >> 172.21.15.119:0/45165536 conn(0x556b23679300 msgr2=0x556b2375fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.203+0000 7f01317dab80 1 -- 172.21.15.119:0/45165536 shutdown_connections 2023-12-07T10:33:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.203+0000 7f01317dab80 1 -- 172.21.15.119:0/45165536 wait complete. 2023-12-07T10:33:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.203+0000 7f01317dab80 20 client.27814 trim_cache size 0 max 16384 2023-12-07T10:33:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:33:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:33:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:33:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:33:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.204+0000 7f011cb9d700 1 -- 172.21.15.119:0/3231912019 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f01000a1d00 msgr2=0x7f01000a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.204+0000 7f011cb9d700 1 --2- 172.21.15.119:0/3231912019 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f01000a1d00 0x7f01000a41c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f010c005b40 tx=0x7f010c01a040 comp rx=0 tx=0).stop 2023-12-07T10:33:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.204+0000 7f011cb9d700 1 -- 172.21.15.119:0/3231912019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f011810dbc0 msgr2=0x7f01181998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.204+0000 7f011cb9d700 1 --2- 172.21.15.119:0/3231912019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f011810dbc0 0x7f01181998f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f011000ef90 tx=0x7f011000c3a0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.204+0000 7f011cb9d700 1 -- 172.21.15.119:0/3231912019 shutdown_connections 2023-12-07T10:33:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.204+0000 7f011cb9d700 1 --2- 172.21.15.119:0/3231912019 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f01000a1d00 0x7f01000a41c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.204+0000 7f011cb9d700 1 --2- 172.21.15.119:0/3231912019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f011810dbc0 0x7f01181998f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.204+0000 7f011cb9d700 1 --2- 172.21.15.119:0/3231912019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01180739e0 0x7f011819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.204+0000 7f011cb9d700 1 --2- 172.21.15.119:0/3231912019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0118074300 0x7f011819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.204+0000 7f011cb9d700 1 -- 172.21.15.119:0/3231912019 >> 172.21.15.119:0/3231912019 conn(0x7f01180fce30 msgr2=0x7f01180fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.204+0000 7f011cb9d700 1 -- 172.21.15.119:0/3231912019 shutdown_connections 2023-12-07T10:33:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:17.204+0000 7f011cb9d700 1 -- 172.21.15.119:0/3231912019 wait complete. 2023-12-07T10:33:17.582 INFO:tasks.workunit.client.0.smithi119.stdout: 1 952269 7.29 MB/sec execute 570 sec latency 41.088 ms 2023-12-07T10:33:18.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:18 smithi119 ceph-mon[97908]: pgmap v403: 129 pgs: 129 active+clean; 748 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 208 op/s 2023-12-07T10:33:18.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:18 smithi159 ceph-mon[134726]: pgmap v403: 129 pgs: 129 active+clean; 748 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 208 op/s 2023-12-07T10:33:18.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:18 smithi153 ceph-mon[133625]: pgmap v403: 129 pgs: 129 active+clean; 748 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 208 op/s 2023-12-07T10:33:18.582 INFO:tasks.workunit.client.0.smithi119.stdout: 1 953067 7.28 MB/sec execute 571 sec latency 30.723 ms 2023-12-07T10:33:19.582 INFO:tasks.workunit.client.0.smithi119.stdout: 1 954650 7.28 MB/sec execute 572 sec latency 19.495 ms 2023-12-07T10:33:20.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:20 smithi119 ceph-mon[97908]: pgmap v404: 129 pgs: 129 active+clean; 748 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 208 op/s 2023-12-07T10:33:20.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:20 smithi159 ceph-mon[134726]: pgmap v404: 129 pgs: 129 active+clean; 748 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 208 op/s 2023-12-07T10:33:20.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:20 smithi153 ceph-mon[133625]: pgmap v404: 129 pgs: 129 active+clean; 748 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 208 op/s 2023-12-07T10:33:20.582 INFO:tasks.workunit.client.0.smithi119.stdout: 1 956051 7.28 MB/sec execute 573 sec latency 35.923 ms 2023-12-07T10:33:21.582 INFO:tasks.workunit.client.0.smithi119.stdout: 1 956816 7.28 MB/sec execute 574 sec latency 135.548 ms 2023-12-07T10:33:22.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:21 smithi159 ceph-mon[134726]: pgmap v405: 129 pgs: 129 active+clean; 759 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 5.8 MiB/s wr, 252 op/s 2023-12-07T10:33:22.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:21 smithi153 ceph-mon[133625]: pgmap v405: 129 pgs: 129 active+clean; 759 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 5.8 MiB/s wr, 252 op/s 2023-12-07T10:33:22.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:21 smithi119 ceph-mon[97908]: pgmap v405: 129 pgs: 129 active+clean; 759 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 5.8 MiB/s wr, 252 op/s 2023-12-07T10:33:22.582 INFO:tasks.workunit.client.0.smithi119.stdout: 1 958236 7.27 MB/sec execute 575 sec latency 32.676 ms 2023-12-07T10:33:23.582 INFO:tasks.workunit.client.0.smithi119.stdout: 1 959678 7.28 MB/sec execute 576 sec latency 25.398 ms 2023-12-07T10:33:24.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:24 smithi119 ceph-mon[97908]: pgmap v406: 129 pgs: 129 active+clean; 758 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.2 MiB/s wr, 244 op/s 2023-12-07T10:33:24.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:24 smithi159 ceph-mon[134726]: pgmap v406: 129 pgs: 129 active+clean; 758 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.2 MiB/s wr, 244 op/s 2023-12-07T10:33:24.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:24 smithi153 ceph-mon[133625]: pgmap v406: 129 pgs: 129 active+clean; 758 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.2 MiB/s wr, 244 op/s 2023-12-07T10:33:24.582 INFO:tasks.workunit.client.0.smithi119.stdout: 1 960687 7.28 MB/sec execute 577 sec latency 205.013 ms 2023-12-07T10:33:25.583 INFO:tasks.workunit.client.0.smithi119.stdout: 1 960687 7.26 MB/sec execute 578 sec latency 1205.172 ms 2023-12-07T10:33:26.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:26 smithi119 ceph-mon[97908]: pgmap v407: 129 pgs: 129 active+clean; 758 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.1 MiB/s wr, 233 op/s 2023-12-07T10:33:26.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:26 smithi159 ceph-mon[134726]: pgmap v407: 129 pgs: 129 active+clean; 758 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.1 MiB/s wr, 233 op/s 2023-12-07T10:33:26.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:26 smithi153 ceph-mon[133625]: pgmap v407: 129 pgs: 129 active+clean; 758 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.1 MiB/s wr, 233 op/s 2023-12-07T10:33:26.583 INFO:tasks.workunit.client.0.smithi119.stdout: 1 960687 7.25 MB/sec execute 579 sec latency 2205.312 ms 2023-12-07T10:33:27.583 INFO:tasks.workunit.client.0.smithi119.stdout: 1 960687 7.24 MB/sec execute 580 sec latency 3205.446 ms 2023-12-07T10:33:28.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:28 smithi119 ceph-mon[97908]: pgmap v408: 129 pgs: 129 active+clean; 759 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.7 MiB/s wr, 333 op/s 2023-12-07T10:33:28.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:28 smithi159 ceph-mon[134726]: pgmap v408: 129 pgs: 129 active+clean; 759 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.7 MiB/s wr, 333 op/s 2023-12-07T10:33:28.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:28 smithi153 ceph-mon[133625]: pgmap v408: 129 pgs: 129 active+clean; 759 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.7 MiB/s wr, 333 op/s 2023-12-07T10:33:28.583 INFO:tasks.workunit.client.0.smithi119.stdout: 1 960687 7.23 MB/sec execute 581 sec latency 4205.521 ms 2023-12-07T10:33:29.583 INFO:tasks.workunit.client.0.smithi119.stdout: 1 960687 7.21 MB/sec execute 582 sec latency 5205.585 ms 2023-12-07T10:33:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.382+0000 7fdb353ff700 1 -- 172.21.15.119:0/2102967721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb301040d0 msgr2=0x7fdb301105e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:30.383 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.382+0000 7fdb353ff700 1 --2- 172.21.15.119:0/2102967721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb301040d0 0x7fdb301105e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800b0a0 tx=0x7fdb2800b3b0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.383 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.382+0000 7fdb353ff700 1 -- 172.21.15.119:0/2102967721 shutdown_connections 2023-12-07T10:33:30.383 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.382+0000 7fdb353ff700 1 --2- 172.21.15.119:0/2102967721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb301040d0 0x7fdb301105e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.383 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.382+0000 7fdb353ff700 1 --2- 172.21.15.119:0/2102967721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb30109720 0x7fdb30109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.383 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.382+0000 7fdb353ff700 1 --2- 172.21.15.119:0/2102967721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb30103710 0x7fdb30103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.383 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.382+0000 7fdb353ff700 1 -- 172.21.15.119:0/2102967721 >> 172.21.15.119:0/2102967721 conn(0x7fdb300ff1c0 msgr2=0x7fdb301015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:30.383 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.382+0000 7fdb353ff700 1 -- 172.21.15.119:0/2102967721 shutdown_connections 2023-12-07T10:33:30.383 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.382+0000 7fdb353ff700 1 -- 172.21.15.119:0/2102967721 wait complete. 2023-12-07T10:33:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.384+0000 7fdb353ff700 1 Processor -- start 2023-12-07T10:33:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.384+0000 7fdb353ff700 1 -- start start 2023-12-07T10:33:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.384+0000 7fdb353ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb30103710 0x7fdb3019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.384+0000 7fdb353ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb301040d0 0x7fdb3019fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.384+0000 7fdb353ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb30109720 0x7fdb301999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.384+0000 7fdb353ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3019a020 con 0x7fdb30109720 2023-12-07T10:33:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.384+0000 7fdb353ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3019a190 con 0x7fdb301040d0 2023-12-07T10:33:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.384+0000 7fdb353ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3019a350 con 0x7fdb30103710 2023-12-07T10:33:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.384+0000 7fdb2e7fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb301040d0 0x7fdb3019fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.384+0000 7fdb2f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb30109720 0x7fdb301999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.384+0000 7fdb2e7fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb301040d0 0x7fdb3019fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:57714/0 (socket says 172.21.15.119:57714) 2023-12-07T10:33:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.384+0000 7fdb2e7fc700 1 -- 172.21.15.119:0/3894429896 learned_addr learned my addr 172.21.15.119:0/3894429896 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:33:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.385+0000 7fdb2effd700 1 --2- 172.21.15.119:0/3894429896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb30103710 0x7fdb3019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.385+0000 7fdb2e7fc700 1 -- 172.21.15.119:0/3894429896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb30103710 msgr2=0x7fdb3019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.385+0000 7fdb2e7fc700 1 --2- 172.21.15.119:0/3894429896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb30103710 0x7fdb3019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.385+0000 7fdb2e7fc700 1 -- 172.21.15.119:0/3894429896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb30109720 msgr2=0x7fdb301999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.385+0000 7fdb2e7fc700 1 --2- 172.21.15.119:0/3894429896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb30109720 0x7fdb301999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.385+0000 7fdb2e7fc700 1 -- 172.21.15.119:0/3894429896 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb28009d00 con 0x7fdb301040d0 2023-12-07T10:33:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.385+0000 7fdb2f7fe700 1 --2- 172.21.15.119:0/3894429896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb30109720 0x7fdb301999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:33:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.385+0000 7fdb2e7fc700 1 --2- 172.21.15.119:0/3894429896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb301040d0 0x7fdb3019fd80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdb2400e9a0 tx=0x7fdb2400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.385+0000 7fdb17fff700 1 -- 172.21.15.119:0/3894429896 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb24009790 con 0x7fdb301040d0 2023-12-07T10:33:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.386+0000 7fdb17fff700 1 -- 172.21.15.119:0/3894429896 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb24010430 con 0x7fdb301040d0 2023-12-07T10:33:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.386+0000 7fdb353ff700 1 -- 172.21.15.119:0/3894429896 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3019a630 con 0x7fdb301040d0 2023-12-07T10:33:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.386+0000 7fdb17fff700 1 -- 172.21.15.119:0/3894429896 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb24018680 con 0x7fdb301040d0 2023-12-07T10:33:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.386+0000 7fdb353ff700 1 -- 172.21.15.119:0/3894429896 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb301a6830 con 0x7fdb301040d0 2023-12-07T10:33:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.389+0000 7fdb17fff700 1 -- 172.21.15.119:0/3894429896 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fdb240105a0 con 0x7fdb301040d0 2023-12-07T10:33:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.390+0000 7fdb48d21b80 1 Processor -- start 2023-12-07T10:33:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.390+0000 7fdb48d21b80 1 -- start start 2023-12-07T10:33:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.390+0000 7fdb48d21b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563315ef9a60 0x563315ef9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.390+0000 7fdb48d21b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563315efa380 0x563315ef4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.391+0000 7fdb48d21b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563315ef4ad0 0x563315ef4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.391+0000 7fdb48d21b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563315e68760 con 0x563315ef9a60 2023-12-07T10:33:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.391+0000 7fdb48d21b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x563315e4f740 con 0x563315efa380 2023-12-07T10:33:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.391+0000 7fdb48d21b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563315e4d060 con 0x563315ef4ad0 2023-12-07T10:33:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.391+0000 7fdb17fff700 1 --2- 172.21.15.119:0/3894429896 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fdb180a1d00 0x7fdb180a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.391+0000 7fdb2effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563315efa380 0x563315ef4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.391+0000 7fdb2effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563315efa380 0x563315ef4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:57720/0 (socket says 172.21.15.119:57720) 2023-12-07T10:33:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.391+0000 7fdb2effd700 1 -- 172.21.15.119:0/821353632 learned_addr learned my addr 172.21.15.119:0/821353632 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:33:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.391+0000 7fdb2f7fe700 1 --2- 172.21.15.119:0/821353632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563315ef9a60 0x563315ef9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.391+0000 7fdb2e7fc700 1 --2- 172.21.15.119:0/821353632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563315ef4ad0 0x563315ef4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.391+0000 7fdb2f7fe700 1 --2- 172.21.15.119:0/3894429896 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fdb180a1d00 0x7fdb180a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.391+0000 7fdb17fff700 1 -- 172.21.15.119:0/3894429896 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fdb24014070 con 0x7fdb301040d0 2023-12-07T10:33:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.391+0000 7fdb2effd700 1 -- 172.21.15.119:0/821353632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563315ef4ad0 msgr2=0x563315ef4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.391+0000 7fdb2effd700 1 --2- 172.21.15.119:0/821353632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563315ef4ad0 0x563315ef4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.391+0000 7fdb2effd700 1 -- 172.21.15.119:0/821353632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563315ef9a60 msgr2=0x563315ef9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.391+0000 7fdb2effd700 1 --2- 172.21.15.119:0/821353632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563315ef9a60 0x563315ef9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.391+0000 7fdb2effd700 1 -- 172.21.15.119:0/821353632 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563315cc55d0 con 0x563315efa380 2023-12-07T10:33:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.392+0000 7fdb2effd700 1 --2- 172.21.15.119:0/821353632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563315efa380 0x563315ef4540 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009d00 tx=0x7fdb20011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82ea648d49378996 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.392+0000 7fdb2f7fe700 1 --2- 172.21.15.119:0/3894429896 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fdb180a1d00 0x7fdb180a41c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009cd0 tx=0x7fdb28006230 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.392+0000 7fdb157fa700 1 -- 172.21.15.119:0/821353632 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb20004230 con 0x563315efa380 2023-12-07T10:33:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.392+0000 7fdb157fa700 1 -- 172.21.15.119:0/821353632 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb20004390 con 0x563315efa380 2023-12-07T10:33:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.392+0000 7fdb157fa700 1 -- 172.21.15.119:0/821353632 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb20019880 con 0x563315efa380 2023-12-07T10:33:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.393+0000 7fdb48d21b80 1 -- 172.21.15.119:0/821353632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563315efa380 msgr2=0x563315ef4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.393+0000 7fdb48d21b80 1 --2- 172.21.15.119:0/821353632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563315efa380 0x563315ef4540 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009d00 tx=0x7fdb20011720 comp rx=0 tx=0).stop 2023-12-07T10:33:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.393+0000 7fdb48d21b80 1 -- 172.21.15.119:0/821353632 shutdown_connections 2023-12-07T10:33:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.393+0000 7fdb48d21b80 1 --2- 172.21.15.119:0/821353632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563315ef4ad0 0x563315ef4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.393+0000 7fdb48d21b80 1 --2- 172.21.15.119:0/821353632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563315ef9a60 0x563315ef9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.393+0000 7fdb48d21b80 1 --2- 172.21.15.119:0/821353632 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563315efa380 0x563315ef4540 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.393+0000 7fdb48d21b80 1 -- 172.21.15.119:0/821353632 >> 172.21.15.119:0/821353632 conn(0x563315e23300 msgr2=0x563315eeeb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.393+0000 7fdb48d21b80 1 -- 172.21.15.119:0/821353632 shutdown_connections 2023-12-07T10:33:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.393+0000 7fdb48d21b80 1 -- 172.21.15.119:0/821353632 wait complete. 2023-12-07T10:33:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.393+0000 7fdb48d21b80 1 -- start start 2023-12-07T10:33:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.394+0000 7fdb48d21b80 1 Processor -- start 2023-12-07T10:33:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.394+0000 7fdb48d21b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563315ef4ad0 0x563315f00370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.394+0000 7fdb48d21b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563315ef9a60 0x563315efd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.394+0000 7fdb48d21b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563315efa380 0x563315efdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.394+0000 7fdb48d21b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563315e48950 con 0x563315efa380 2023-12-07T10:33:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.394+0000 7fdb48d21b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x563315e02500 con 0x563315ef4ad0 2023-12-07T10:33:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.394+0000 7fdb48d21b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563315e021a0 con 0x563315ef9a60 2023-12-07T10:33:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.394+0000 7fdb48d21b80 10 client.0 _ll_register_callbacks cb 0x563315de98e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:33:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.394+0000 7fdb2effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563315ef4ad0 0x563315f00370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.394+0000 7fdb2e7fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563315ef9a60 0x563315efd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.394+0000 7fdb2f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563315efa380 0x563315efdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.394+0000 7fdb2effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563315ef4ad0 0x563315f00370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:57726/0 (socket says 172.21.15.119:57726) 2023-12-07T10:33:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.394+0000 7fdb2effd700 1 -- 172.21.15.119:0/845157217 learned_addr learned my addr 172.21.15.119:0/845157217 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:33:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.394+0000 7fdb2e7fc700 1 --2- 172.21.15.119:0/845157217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563315ef9a60 0x563315efd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:60530/0 (socket says 172.21.15.119:60530) 2023-12-07T10:33:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.395+0000 7fdb2effd700 1 -- 172.21.15.119:0/845157217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563315ef9a60 msgr2=0x563315efd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.395+0000 7fdb2effd700 1 --2- 172.21.15.119:0/845157217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563315ef9a60 0x563315efd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.395+0000 7fdb2effd700 1 -- 172.21.15.119:0/845157217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563315efa380 msgr2=0x563315efdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.395+0000 7fdb2effd700 1 --2- 172.21.15.119:0/845157217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563315efa380 0x563315efdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.395+0000 7fdb2effd700 1 -- 172.21.15.119:0/845157217 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb20009960 con 0x563315ef4ad0 2023-12-07T10:33:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.395+0000 7fdb2effd700 1 --2- 172.21.15.119:0/845157217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563315ef4ad0 0x563315f00370 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009230 tx=0x7fdb200038b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6cee69d17d4bf45 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.395+0000 7fdb12ff5700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:33:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.395+0000 7fdb12ff5700 1 -- 172.21.15.119:0/845157217 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb20005ce0 con 0x563315ef4ad0 2023-12-07T10:33:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.395+0000 7fdb48d21b80 1 -- 172.21.15.119:0/845157217 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563315ee76f0 con 0x563315ef4ad0 2023-12-07T10:33:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.395+0000 7fdb12ff5700 1 -- 172.21.15.119:0/845157217 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fdb20005040 con 0x563315ef4ad0 2023-12-07T10:33:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.396+0000 7fdb12ff5700 1 -- 172.21.15.119:0/845157217 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb20005200 con 0x563315ef4ad0 2023-12-07T10:33:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.396+0000 7fdb48d21b80 10 client.27845 fetch_fsmap learned FSMap version 62 2023-12-07T10:33:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.396+0000 7fdb48d21b80 1 -- 172.21.15.119:0/845157217 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x563315c71d80 con 0x563315ef4ad0 2023-12-07T10:33:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.396+0000 7fdb12ff5700 1 -- 172.21.15.119:0/845157217 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563315ee76f0 con 0x563315ef4ad0 2023-12-07T10:33:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.396+0000 7fdb12ff5700 1 -- 172.21.15.119:0/845157217 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 62) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fdb20028020 con 0x563315ef4ad0 2023-12-07T10:33:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.396+0000 7fdb48d21b80 10 client.27845 fetch_fsmap finished waiting for FSMap version 62 2023-12-07T10:33:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.396+0000 7fdb48d21b80 10 client.27845 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:33:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.396+0000 7fdb48d21b80 20 client.27845 populate_metadata read hostname 'smithi119' 2023-12-07T10:33:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.397+0000 7fdb48d21b80 1 --2- 172.21.15.119:0/845157217 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x563315f0ddb0 0x563315f101a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.397+0000 7fdb48d21b80 4 client.27845 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:33:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.397+0000 7fdb48d21b80 1 -- 172.21.15.119:0/845157217 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563315ec3740 con 0x563315f0ddb0 2023-12-07T10:33:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.397+0000 7fdb2e7fc700 1 --2- 172.21.15.119:0/845157217 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x563315f0ddb0 0x563315f101a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.397+0000 7fdb12ff5700 10 client.27845 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:33:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.397+0000 7fdb2e7fc700 1 --2- 172.21.15.119:0/845157217 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x563315f0ddb0 0x563315f101a0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a71cb19dfd3604b server_cookie=64fd06302b5009b5 in_seq=0 out_seq=0 2023-12-07T10:33:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.398+0000 7fdb12ff5700 1 -- 172.21.15.119:0/845157217 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563315ec3740 con 0x563315f0ddb0 2023-12-07T10:33:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.398+0000 7fdb12ff5700 10 client.27845 handle_command_reply: tid=0 2023-12-07T10:33:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.398+0000 7fdb48d21b80 1 client.27845 shutdown 2023-12-07T10:33:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.398+0000 7fdb48d21b80 1 -- 172.21.15.119:0/845157217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563315ef4ad0 msgr2=0x563315f00370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.398+0000 7fdb48d21b80 1 --2- 172.21.15.119:0/845157217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563315ef4ad0 0x563315f00370 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009230 tx=0x7fdb200038b0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.398+0000 7fdb48d21b80 1 -- 172.21.15.119:0/845157217 shutdown_connections 2023-12-07T10:33:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.398+0000 7fdb48d21b80 1 --2- 172.21.15.119:0/845157217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563315efa380 0x563315efdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.398+0000 7fdb48d21b80 1 --2- 172.21.15.119:0/845157217 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x563315f0ddb0 0x563315f101a0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.398+0000 7fdb48d21b80 1 --2- 172.21.15.119:0/845157217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563315ef4ad0 0x563315f00370 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.398+0000 7fdb48d21b80 1 --2- 172.21.15.119:0/845157217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563315ef9a60 0x563315efd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.398+0000 7fdb12ff5700 0 client.27845 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:33:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.398+0000 7fdb48d21b80 1 -- 172.21.15.119:0/845157217 >> 172.21.15.119:0/845157217 conn(0x563315e23300 msgr2=0x563315ee9980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.399+0000 7fdb48d21b80 1 -- 172.21.15.119:0/845157217 shutdown_connections 2023-12-07T10:33:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.399+0000 7fdb48d21b80 1 -- 172.21.15.119:0/845157217 wait complete. 2023-12-07T10:33:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.399+0000 7fdb48d21b80 20 client.27845 trim_cache size 0 max 16384 2023-12-07T10:33:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.420+0000 7fdb48d21b80 1 Processor -- start 2023-12-07T10:33:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.420+0000 7fdb48d21b80 1 -- start start 2023-12-07T10:33:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.421+0000 7fdb48d21b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563315ef4ad0 0x563315ef4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.421+0000 7fdb48d21b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563315ef9a60 0x563315f14b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.421+0000 7fdb48d21b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563315efa380 0x563315f04dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.421+0000 7fdb48d21b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563315cc78c0 con 0x563315ef4ad0 2023-12-07T10:33:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.421+0000 7fdb48d21b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x563315f0dbc0 con 0x563315efa380 2023-12-07T10:33:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.421+0000 7fdb48d21b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563315ef1050 con 0x563315ef9a60 2023-12-07T10:33:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.421+0000 7fdb2effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563315ef4ad0 0x563315ef4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.421+0000 7fdb2effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563315ef4ad0 0x563315ef4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41002/0 (socket says 172.21.15.119:41002) 2023-12-07T10:33:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.421+0000 7fdb2effd700 1 -- 172.21.15.119:0/1028034792 learned_addr learned my addr 172.21.15.119:0/1028034792 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:33:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.421+0000 7fdb2f7fe700 1 --2- 172.21.15.119:0/1028034792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563315efa380 0x563315f04dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.421+0000 7fdb2e7fc700 1 --2- 172.21.15.119:0/1028034792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563315ef9a60 0x563315f14b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.421+0000 7fdb2effd700 1 -- 172.21.15.119:0/1028034792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563315ef9a60 msgr2=0x563315f14b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.421+0000 7fdb2effd700 1 --2- 172.21.15.119:0/1028034792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563315ef9a60 0x563315f14b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.421+0000 7fdb2effd700 1 -- 172.21.15.119:0/1028034792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563315efa380 msgr2=0x563315f04dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.421+0000 7fdb2effd700 1 --2- 172.21.15.119:0/1028034792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563315efa380 0x563315f04dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.421+0000 7fdb2effd700 1 -- 172.21.15.119:0/1028034792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563315c5ac00 con 0x563315ef4ad0 2023-12-07T10:33:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.421+0000 7fdb2e7fc700 1 --2- 172.21.15.119:0/1028034792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563315ef9a60 0x563315f14b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:33:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.422+0000 7fdb2effd700 1 --2- 172.21.15.119:0/1028034792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563315ef4ad0 0x563315ef4540 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fdb200092a0 tx=0x7fdb20005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a710ed9df4a3291 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.422+0000 7fdb15ffb700 1 -- 172.21.15.119:0/1028034792 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb20003920 con 0x563315ef4ad0 2023-12-07T10:33:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.422+0000 7fdb15ffb700 1 -- 172.21.15.119:0/1028034792 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb20019940 con 0x563315ef4ad0 2023-12-07T10:33:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.422+0000 7fdb15ffb700 1 -- 172.21.15.119:0/1028034792 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2000f620 con 0x563315ef4ad0 2023-12-07T10:33:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.422+0000 7fdb48d21b80 1 -- 172.21.15.119:0/1028034792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563315ef4ad0 msgr2=0x563315ef4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.422+0000 7fdb48d21b80 1 --2- 172.21.15.119:0/1028034792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563315ef4ad0 0x563315ef4540 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fdb200092a0 tx=0x7fdb20005a00 comp rx=0 tx=0).stop 2023-12-07T10:33:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.422+0000 7fdb48d21b80 1 -- 172.21.15.119:0/1028034792 shutdown_connections 2023-12-07T10:33:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.422+0000 7fdb48d21b80 1 --2- 172.21.15.119:0/1028034792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563315efa380 0x563315f04dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.423+0000 7fdb48d21b80 1 --2- 172.21.15.119:0/1028034792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563315ef4ad0 0x563315ef4540 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.423+0000 7fdb48d21b80 1 --2- 172.21.15.119:0/1028034792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563315ef9a60 0x563315f14b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.423+0000 7fdb48d21b80 1 -- 172.21.15.119:0/1028034792 >> 172.21.15.119:0/1028034792 conn(0x563315e23300 msgr2=0x563315eee9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.423+0000 7fdb48d21b80 1 -- 172.21.15.119:0/1028034792 shutdown_connections 2023-12-07T10:33:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.423+0000 7fdb48d21b80 1 -- 172.21.15.119:0/1028034792 wait complete. 2023-12-07T10:33:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.423+0000 7fdb48d21b80 1 -- start start 2023-12-07T10:33:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.423+0000 7fdb48d21b80 1 Processor -- start 2023-12-07T10:33:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.424+0000 7fdb48d21b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563315ef4ad0 0x563315f04fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.424+0000 7fdb48d21b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563315ef9a60 0x563315f02db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.424+0000 7fdb48d21b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563315efa380 0x563315f032f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.424+0000 7fdb48d21b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563315f038a0 con 0x563315ef9a60 2023-12-07T10:33:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.424+0000 7fdb48d21b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x563315f03a10 con 0x563315ef4ad0 2023-12-07T10:33:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.424+0000 7fdb48d21b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563315f03bd0 con 0x563315efa380 2023-12-07T10:33:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.424+0000 7fdb48d21b80 10 client.0 _ll_register_callbacks cb 0x563315de98e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:33:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.424+0000 7fdb2e7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563315ef9a60 0x563315f02db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.424+0000 7fdb2e7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563315ef9a60 0x563315f02db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41014/0 (socket says 172.21.15.119:41014) 2023-12-07T10:33:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.424+0000 7fdb2e7fc700 1 -- 172.21.15.119:0/2168004992 learned_addr learned my addr 172.21.15.119:0/2168004992 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:33:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.424+0000 7fdb2effd700 1 --2- 172.21.15.119:0/2168004992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563315ef4ad0 0x563315f04fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.424+0000 7fdb2f7fe700 1 --2- 172.21.15.119:0/2168004992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563315efa380 0x563315f032f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.424+0000 7fdb2e7fc700 1 -- 172.21.15.119:0/2168004992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563315efa380 msgr2=0x563315f032f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.424+0000 7fdb2e7fc700 1 --2- 172.21.15.119:0/2168004992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563315efa380 0x563315f032f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.424+0000 7fdb2e7fc700 1 -- 172.21.15.119:0/2168004992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563315ef4ad0 msgr2=0x563315f04fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.424+0000 7fdb2e7fc700 1 --2- 172.21.15.119:0/2168004992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563315ef4ad0 0x563315f04fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.424+0000 7fdb2e7fc700 1 -- 172.21.15.119:0/2168004992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563315c71d80 con 0x563315ef9a60 2023-12-07T10:33:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.425+0000 7fdb2e7fc700 1 --2- 172.21.15.119:0/2168004992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563315ef9a60 0x563315f02db0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fdb24096da0 tx=0x7fdb24007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b3b8e593918bc9 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.425+0000 7fdb12ff5700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:33:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.425+0000 7fdb12ff5700 1 -- 172.21.15.119:0/2168004992 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb240a2c30 con 0x563315ef9a60 2023-12-07T10:33:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.425+0000 7fdb12ff5700 1 -- 172.21.15.119:0/2168004992 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fdb240850b0 con 0x563315ef9a60 2023-12-07T10:33:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.425+0000 7fdb48d21b80 1 -- 172.21.15.119:0/2168004992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb20004390 con 0x563315ef9a60 2023-12-07T10:33:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.425+0000 7fdb12ff5700 1 -- 172.21.15.119:0/2168004992 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb240ba0b0 con 0x563315ef9a60 2023-12-07T10:33:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.425+0000 7fdb48d21b80 10 client.17805 fetch_fsmap learned FSMap version 62 2023-12-07T10:33:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.425+0000 7fdb48d21b80 1 -- 172.21.15.119:0/2168004992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x563315c981f0 con 0x563315ef9a60 2023-12-07T10:33:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.425+0000 7fdb12ff5700 1 -- 172.21.15.119:0/2168004992 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb20004390 con 0x563315ef9a60 2023-12-07T10:33:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.425+0000 7fdb12ff5700 1 -- 172.21.15.119:0/2168004992 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 62) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fdb240a27a0 con 0x563315ef9a60 2023-12-07T10:33:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.425+0000 7fdb48d21b80 10 client.17805 fetch_fsmap finished waiting for FSMap version 62 2023-12-07T10:33:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.425+0000 7fdb48d21b80 10 client.17805 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:33:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.425+0000 7fdb48d21b80 20 client.17805 populate_metadata read hostname 'smithi119' 2023-12-07T10:33:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.425+0000 7fdb48d21b80 1 --2- 172.21.15.119:0/2168004992 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x563315f64950 0x563315f66d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.426+0000 7fdb48d21b80 4 client.17805 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:33:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.426+0000 7fdb48d21b80 1 -- 172.21.15.119:0/2168004992 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563315e0ef70 con 0x563315f64950 2023-12-07T10:33:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.426+0000 7fdb2effd700 1 --2- 172.21.15.119:0/2168004992 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x563315f64950 0x563315f66d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.426+0000 7fdb2effd700 1 --2- 172.21.15.119:0/2168004992 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x563315f64950 0x563315f66d70 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=607d61370cfeea1e server_cookie=50c5d92c9754aad in_seq=0 out_seq=0 2023-12-07T10:33:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.426+0000 7fdb12ff5700 10 client.17805 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:33:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.469+0000 7fdb12ff5700 1 -- 172.21.15.119:0/2168004992 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563315e0ef70 con 0x563315f64950 2023-12-07T10:33:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.469+0000 7fdb12ff5700 10 client.17805 handle_command_reply: tid=0 2023-12-07T10:33:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.469+0000 7fdb48d21b80 1 client.17805 shutdown 2023-12-07T10:33:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.469+0000 7fdb48d21b80 1 -- 172.21.15.119:0/2168004992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563315ef9a60 msgr2=0x563315f02db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.469+0000 7fdb48d21b80 1 --2- 172.21.15.119:0/2168004992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563315ef9a60 0x563315f02db0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fdb24096da0 tx=0x7fdb24007f90 comp rx=0 tx=0).stop 2023-12-07T10:33:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.469+0000 7fdb48d21b80 1 -- 172.21.15.119:0/2168004992 shutdown_connections 2023-12-07T10:33:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.469+0000 7fdb48d21b80 1 --2- 172.21.15.119:0/2168004992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563315efa380 0x563315f032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.469+0000 7fdb48d21b80 1 --2- 172.21.15.119:0/2168004992 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x563315f64950 0x563315f66d70 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.469+0000 7fdb48d21b80 1 --2- 172.21.15.119:0/2168004992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563315ef4ad0 0x563315f04fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.469+0000 7fdb48d21b80 1 --2- 172.21.15.119:0/2168004992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563315ef9a60 0x563315f02db0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.469+0000 7fdb48d21b80 1 -- 172.21.15.119:0/2168004992 >> 172.21.15.119:0/2168004992 conn(0x563315e23300 msgr2=0x563315eee9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.469+0000 7fdb12ff5700 0 client.17805 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:33:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.469+0000 7fdb48d21b80 1 -- 172.21.15.119:0/2168004992 shutdown_connections 2023-12-07T10:33:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.469+0000 7fdb48d21b80 1 -- 172.21.15.119:0/2168004992 wait complete. 2023-12-07T10:33:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.469+0000 7fdb48d21b80 20 client.17805 trim_cache size 0 max 16384 2023-12-07T10:33:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:33:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:33:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:33:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:33:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.470+0000 7fdb353ff700 1 -- 172.21.15.119:0/3894429896 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fdb180a1d00 msgr2=0x7fdb180a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.470+0000 7fdb353ff700 1 --2- 172.21.15.119:0/3894429896 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fdb180a1d00 0x7fdb180a41c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009cd0 tx=0x7fdb28006230 comp rx=0 tx=0).stop 2023-12-07T10:33:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.470+0000 7fdb353ff700 1 -- 172.21.15.119:0/3894429896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb301040d0 msgr2=0x7fdb3019fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.470+0000 7fdb353ff700 1 --2- 172.21.15.119:0/3894429896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb301040d0 0x7fdb3019fd80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdb2400e9a0 tx=0x7fdb2400ecb0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.470+0000 7fdb353ff700 1 -- 172.21.15.119:0/3894429896 shutdown_connections 2023-12-07T10:33:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.470+0000 7fdb353ff700 1 --2- 172.21.15.119:0/3894429896 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fdb180a1d00 0x7fdb180a41c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.470+0000 7fdb353ff700 1 --2- 172.21.15.119:0/3894429896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb30109720 0x7fdb301999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.470+0000 7fdb353ff700 1 --2- 172.21.15.119:0/3894429896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb30103710 0x7fdb3019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.470+0000 7fdb353ff700 1 --2- 172.21.15.119:0/3894429896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb301040d0 0x7fdb3019fd80 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.470+0000 7fdb353ff700 1 -- 172.21.15.119:0/3894429896 >> 172.21.15.119:0/3894429896 conn(0x7fdb300ff1c0 msgr2=0x7fdb300ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.470+0000 7fdb353ff700 1 -- 172.21.15.119:0/3894429896 shutdown_connections 2023-12-07T10:33:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:30.470+0000 7fdb353ff700 1 -- 172.21.15.119:0/3894429896 wait complete. 2023-12-07T10:33:30.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:30 smithi119 ceph-mon[97908]: pgmap v409: 129 pgs: 129 active+clean; 759 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 244 op/s 2023-12-07T10:33:30.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:30 smithi159 ceph-mon[134726]: pgmap v409: 129 pgs: 129 active+clean; 759 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 244 op/s 2023-12-07T10:33:30.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:30 smithi153 ceph-mon[133625]: pgmap v409: 129 pgs: 129 active+clean; 759 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 244 op/s 2023-12-07T10:33:30.583 INFO:tasks.workunit.client.0.smithi119.stdout: 1 960815 7.20 MB/sec execute 583 sec latency 6089.873 ms 2023-12-07T10:33:31.583 INFO:tasks.workunit.client.0.smithi119.stdout: 1 962663 7.21 MB/sec execute 584 sec latency 23.791 ms 2023-12-07T10:33:32.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:31 smithi159 ceph-mon[134726]: pgmap v410: 129 pgs: 129 active+clean; 761 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.0 MiB/s wr, 263 op/s 2023-12-07T10:33:32.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:31 smithi153 ceph-mon[133625]: pgmap v410: 129 pgs: 129 active+clean; 761 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.0 MiB/s wr, 263 op/s 2023-12-07T10:33:32.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:31 smithi119 ceph-mon[97908]: pgmap v410: 129 pgs: 129 active+clean; 761 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.0 MiB/s wr, 263 op/s 2023-12-07T10:33:32.583 INFO:tasks.workunit.client.0.smithi119.stdout: 1 963722 7.20 MB/sec execute 585 sec latency 11.831 ms 2023-12-07T10:33:33.585 INFO:tasks.workunit.client.0.smithi119.stdout: 1 965430 7.20 MB/sec execute 586 sec latency 23.273 ms 2023-12-07T10:33:34.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:34 smithi119 ceph-mon[97908]: pgmap v411: 129 pgs: 129 active+clean; 761 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 2.3 MiB/s wr, 166 op/s 2023-12-07T10:33:34.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:34 smithi159 ceph-mon[134726]: pgmap v411: 129 pgs: 129 active+clean; 761 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 2.3 MiB/s wr, 166 op/s 2023-12-07T10:33:34.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:34 smithi153 ceph-mon[133625]: pgmap v411: 129 pgs: 129 active+clean; 761 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 2.3 MiB/s wr, 166 op/s 2023-12-07T10:33:34.583 INFO:tasks.workunit.client.0.smithi119.stdout: 1 966833 7.21 MB/sec execute 587 sec latency 13.484 ms 2023-12-07T10:33:35.583 INFO:tasks.workunit.client.0.smithi119.stdout: 1 968294 7.21 MB/sec execute 588 sec latency 16.494 ms 2023-12-07T10:33:36.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:36 smithi119 ceph-mon[97908]: pgmap v412: 129 pgs: 129 active+clean; 761 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 1.8 MiB/s wr, 122 op/s 2023-12-07T10:33:36.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:36 smithi159 ceph-mon[134726]: pgmap v412: 129 pgs: 129 active+clean; 761 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 1.8 MiB/s wr, 122 op/s 2023-12-07T10:33:36.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:36 smithi153 ceph-mon[133625]: pgmap v412: 129 pgs: 129 active+clean; 761 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 1.8 MiB/s wr, 122 op/s 2023-12-07T10:33:36.583 INFO:tasks.workunit.client.0.smithi119.stdout: 1 970139 7.22 MB/sec execute 589 sec latency 20.528 ms 2023-12-07T10:33:37.583 INFO:tasks.workunit.client.0.smithi119.stdout: 1 971108 7.21 MB/sec execute 590 sec latency 17.742 ms 2023-12-07T10:33:38.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:38 smithi119 ceph-mon[97908]: pgmap v413: 129 pgs: 129 active+clean; 768 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 200 op/s 2023-12-07T10:33:38.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:38 smithi159 ceph-mon[134726]: pgmap v413: 129 pgs: 129 active+clean; 768 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 200 op/s 2023-12-07T10:33:38.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:38 smithi153 ceph-mon[133625]: pgmap v413: 129 pgs: 129 active+clean; 768 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 200 op/s 2023-12-07T10:33:38.583 INFO:tasks.workunit.client.0.smithi119.stdout: 1 973100 7.21 MB/sec execute 591 sec latency 15.594 ms 2023-12-07T10:33:39.584 INFO:tasks.workunit.client.0.smithi119.stdout: 1 974259 7.21 MB/sec execute 592 sec latency 13.378 ms 2023-12-07T10:33:40.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:40 smithi119 ceph-mon[97908]: pgmap v414: 129 pgs: 129 active+clean; 768 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 884 KiB/s wr, 100 op/s 2023-12-07T10:33:40.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:40 smithi159 ceph-mon[134726]: pgmap v414: 129 pgs: 129 active+clean; 768 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 884 KiB/s wr, 100 op/s 2023-12-07T10:33:40.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:40 smithi153 ceph-mon[133625]: pgmap v414: 129 pgs: 129 active+clean; 768 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 884 KiB/s wr, 100 op/s 2023-12-07T10:33:40.583 INFO:tasks.workunit.client.0.smithi119.stdout: 1 976026 7.21 MB/sec execute 593 sec latency 18.378 ms 2023-12-07T10:33:41.584 INFO:tasks.workunit.client.0.smithi119.stdout: 1 977427 7.22 MB/sec execute 594 sec latency 22.619 ms 2023-12-07T10:33:42.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:41 smithi159 ceph-mon[134726]: pgmap v415: 129 pgs: 129 active+clean; 776 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 177 op/s 2023-12-07T10:33:42.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:41 smithi153 ceph-mon[133625]: pgmap v415: 129 pgs: 129 active+clean; 776 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 177 op/s 2023-12-07T10:33:42.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:41 smithi119 ceph-mon[97908]: pgmap v415: 129 pgs: 129 active+clean; 776 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 177 op/s 2023-12-07T10:33:42.584 INFO:tasks.workunit.client.0.smithi119.stdout: 1 978703 7.22 MB/sec execute 595 sec latency 20.246 ms 2023-12-07T10:33:43.584 INFO:tasks.workunit.client.0.smithi119.stdout: 1 980677 7.23 MB/sec execute 596 sec latency 16.687 ms 2023-12-07T10:33:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.631+0000 7f76391ff700 1 -- 172.21.15.119:0/1111642378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7634074300 msgr2=0x7f763410d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.631+0000 7f76391ff700 1 --2- 172.21.15.119:0/1111642378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7634074300 0x7f763410d860 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7628009990 tx=0x7f7628009ca0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.632+0000 7f76391ff700 1 -- 172.21.15.119:0/1111642378 shutdown_connections 2023-12-07T10:33:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.632+0000 7f76391ff700 1 --2- 172.21.15.119:0/1111642378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f763410dda0 0x7f76341101d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.632+0000 7f76391ff700 1 --2- 172.21.15.119:0/1111642378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76340739e0 0x7f7634073dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.632+0000 7f76391ff700 1 --2- 172.21.15.119:0/1111642378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7634074300 0x7f763410d860 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.632+0000 7f76391ff700 1 -- 172.21.15.119:0/1111642378 >> 172.21.15.119:0/1111642378 conn(0x7f76340fd010 msgr2=0x7f76340ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.632+0000 7f76391ff700 1 -- 172.21.15.119:0/1111642378 shutdown_connections 2023-12-07T10:33:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.632+0000 7f76391ff700 1 -- 172.21.15.119:0/1111642378 wait complete. 2023-12-07T10:33:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.633+0000 7f76391ff700 1 Processor -- start 2023-12-07T10:33:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.633+0000 7f76391ff700 1 -- start start 2023-12-07T10:33:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.633+0000 7f76391ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76340739e0 0x7f763419f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.633+0000 7f76391ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7634074300 0x7f763419fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.633+0000 7f76391ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f763410dda0 0x7f7634199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.633+0000 7f76391ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763419a220 con 0x7f763410dda0 2023-12-07T10:33:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.633+0000 7f76391ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763419a390 con 0x7f7634074300 2023-12-07T10:33:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.633+0000 7f76391ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763419a550 con 0x7f76340739e0 2023-12-07T10:33:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.634+0000 7f7632d9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76340739e0 0x7f763419f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.634+0000 7f763259c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7634074300 0x7f763419fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.634+0000 7f7632d9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76340739e0 0x7f763419f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:56638/0 (socket says 172.21.15.119:56638) 2023-12-07T10:33:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.634+0000 7f7632d9d700 1 -- 172.21.15.119:0/942181814 learned_addr learned my addr 172.21.15.119:0/942181814 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:33:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.634+0000 7f763359e700 1 --2- 172.21.15.119:0/942181814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f763410dda0 0x7f7634199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.634+0000 7f7632d9d700 1 -- 172.21.15.119:0/942181814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7634074300 msgr2=0x7f763419fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.634+0000 7f7632d9d700 1 --2- 172.21.15.119:0/942181814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7634074300 0x7f763419fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.634+0000 7f7632d9d700 1 -- 172.21.15.119:0/942181814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f763410dda0 msgr2=0x7f7634199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.634+0000 7f7632d9d700 1 --2- 172.21.15.119:0/942181814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f763410dda0 0x7f7634199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.634+0000 7f7632d9d700 1 -- 172.21.15.119:0/942181814 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76280095f0 con 0x7f76340739e0 2023-12-07T10:33:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.634+0000 7f763359e700 1 --2- 172.21.15.119:0/942181814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f763410dda0 0x7f7634199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:33:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.634+0000 7f7632d9d700 1 --2- 172.21.15.119:0/942181814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76340739e0 0x7f763419f960 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f762400e9a0 tx=0x7f762400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.634+0000 7f761bfff700 1 -- 172.21.15.119:0/942181814 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7624009790 con 0x7f76340739e0 2023-12-07T10:33:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.634+0000 7f761bfff700 1 -- 172.21.15.119:0/942181814 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f762400cbd0 con 0x7f76340739e0 2023-12-07T10:33:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.634+0000 7f76391ff700 1 -- 172.21.15.119:0/942181814 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f763419a830 con 0x7f76340739e0 2023-12-07T10:33:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.634+0000 7f761bfff700 1 -- 172.21.15.119:0/942181814 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7624018680 con 0x7f76340739e0 2023-12-07T10:33:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.635+0000 7f76391ff700 1 -- 172.21.15.119:0/942181814 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76341a68e0 con 0x7f76340739e0 2023-12-07T10:33:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.639+0000 7f761bfff700 1 -- 172.21.15.119:0/942181814 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f762400cd40 con 0x7f76340739e0 2023-12-07T10:33:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.640+0000 7f764cbd0b80 1 Processor -- start 2023-12-07T10:33:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.641+0000 7f764cbd0b80 1 -- start start 2023-12-07T10:33:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.641+0000 7f761bfff700 1 --2- 172.21.15.119:0/942181814 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f761c0a1c80 0x7f761c0a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.641+0000 7f764cbd0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d8c1d4a60 0x558d8c1d4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.641+0000 7f764cbd0b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d8c1d5380 0x558d8c1cf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.641+0000 7f764cbd0b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d8c1cfad0 0x558d8c1cff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.641+0000 7f764cbd0b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8c143760 con 0x558d8c1d5380 2023-12-07T10:33:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.641+0000 7f764cbd0b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8c12a740 con 0x558d8c1cfad0 2023-12-07T10:33:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.641+0000 7f761bfff700 1 -- 172.21.15.119:0/942181814 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f7624014070 con 0x7f76340739e0 2023-12-07T10:33:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.641+0000 7f764cbd0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8c128060 con 0x558d8c1d4a60 2023-12-07T10:33:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.641+0000 7f763359e700 1 --2- 172.21.15.119:0/942181814 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f761c0a1c80 0x7f761c0a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.641+0000 7f763259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d8c1d5380 0x558d8c1cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.641+0000 7f763259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d8c1d5380 0x558d8c1cf540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45434/0 (socket says 172.21.15.119:45434) 2023-12-07T10:33:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.642+0000 7f763259c700 1 -- 172.21.15.119:0/487084328 learned_addr learned my addr 172.21.15.119:0/487084328 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:33:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.641+0000 7f7632d9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d8c1d4a60 0x558d8c1d4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.642+0000 7f763359e700 1 --2- 172.21.15.119:0/487084328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d8c1cfad0 0x558d8c1cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.642+0000 7f7632d9d700 1 -- 172.21.15.119:0/487084328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d8c1cfad0 msgr2=0x558d8c1cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.642+0000 7f7632d9d700 1 --2- 172.21.15.119:0/487084328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d8c1cfad0 0x558d8c1cff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.642+0000 7f7632d9d700 1 -- 172.21.15.119:0/487084328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d8c1d5380 msgr2=0x558d8c1cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.642+0000 7f7632d9d700 1 --2- 172.21.15.119:0/487084328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d8c1d5380 0x558d8c1cf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.642+0000 7f7632d9d700 1 -- 172.21.15.119:0/487084328 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d8bfa05d0 con 0x558d8c1d4a60 2023-12-07T10:33:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.642+0000 7f763359e700 1 --2- 172.21.15.119:0/487084328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d8c1cfad0 0x558d8c1cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:33:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.642+0000 7f763359e700 1 --2- 172.21.15.119:0/942181814 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f761c0a1c80 0x7f761c0a4140 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f76341a05f0 tx=0x7f762c007b00 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.642+0000 7f7632d9d700 1 --2- 172.21.15.119:0/487084328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d8c1d4a60 0x558d8c1d4e40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f76240cb3a0 tx=0x7f762409a840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd66fc8b730438e3 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.642+0000 7f76197fa700 1 -- 172.21.15.119:0/487084328 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762409b0f0 con 0x558d8c1d4a60 2023-12-07T10:33:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.642+0000 7f76197fa700 1 -- 172.21.15.119:0/487084328 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f762409a9f0 con 0x558d8c1d4a60 2023-12-07T10:33:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.642+0000 7f76197fa700 1 -- 172.21.15.119:0/487084328 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7624010430 con 0x558d8c1d4a60 2023-12-07T10:33:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.643+0000 7f764cbd0b80 1 -- 172.21.15.119:0/487084328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d8c1d4a60 msgr2=0x558d8c1d4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.643+0000 7f764cbd0b80 1 --2- 172.21.15.119:0/487084328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d8c1d4a60 0x558d8c1d4e40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f76240cb3a0 tx=0x7f762409a840 comp rx=0 tx=0).stop 2023-12-07T10:33:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.643+0000 7f764cbd0b80 1 -- 172.21.15.119:0/487084328 shutdown_connections 2023-12-07T10:33:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.643+0000 7f764cbd0b80 1 --2- 172.21.15.119:0/487084328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d8c1cfad0 0x558d8c1cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.643+0000 7f764cbd0b80 1 --2- 172.21.15.119:0/487084328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d8c1d4a60 0x558d8c1d4e40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.643+0000 7f764cbd0b80 1 --2- 172.21.15.119:0/487084328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d8c1d5380 0x558d8c1cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.643+0000 7f764cbd0b80 1 -- 172.21.15.119:0/487084328 >> 172.21.15.119:0/487084328 conn(0x558d8c0fe300 msgr2=0x558d8c1c9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.643+0000 7f764cbd0b80 1 -- 172.21.15.119:0/487084328 shutdown_connections 2023-12-07T10:33:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.643+0000 7f764cbd0b80 1 -- 172.21.15.119:0/487084328 wait complete. 2023-12-07T10:33:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.644+0000 7f764cbd0b80 1 -- start start 2023-12-07T10:33:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.644+0000 7f764cbd0b80 1 Processor -- start 2023-12-07T10:33:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.645+0000 7f764cbd0b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d8c1cfad0 0x558d8c1d8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.645+0000 7f764cbd0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d8c1d4a60 0x558d8c1d8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.645+0000 7f764cbd0b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d8c1d5380 0x558d8c1d91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.645+0000 7f764cbd0b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8c123950 con 0x558d8c1d5380 2023-12-07T10:33:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.645+0000 7f764cbd0b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8c0dd500 con 0x558d8c1cfad0 2023-12-07T10:33:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.645+0000 7f764cbd0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8c0dd1a0 con 0x558d8c1d4a60 2023-12-07T10:33:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.645+0000 7f764cbd0b80 10 client.0 _ll_register_callbacks cb 0x558d8c0c48e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:33:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.645+0000 7f7632d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d8c1cfad0 0x558d8c1d8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.645+0000 7f7632d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d8c1cfad0 0x558d8c1d8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:38622/0 (socket says 172.21.15.119:38622) 2023-12-07T10:33:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.645+0000 7f763259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d8c1d4a60 0x558d8c1d8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.645+0000 7f7632d9d700 1 -- 172.21.15.119:0/559444005 learned_addr learned my addr 172.21.15.119:0/559444005 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:33:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.645+0000 7f763359e700 1 --2- 172.21.15.119:0/559444005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d8c1d5380 0x558d8c1d91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.645+0000 7f7632d9d700 1 -- 172.21.15.119:0/559444005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d8c1d4a60 msgr2=0x558d8c1d8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.645+0000 7f7632d9d700 1 --2- 172.21.15.119:0/559444005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d8c1d4a60 0x558d8c1d8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.645+0000 7f7632d9d700 1 -- 172.21.15.119:0/559444005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d8c1d5380 msgr2=0x558d8c1d91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.645+0000 7f7632d9d700 1 --2- 172.21.15.119:0/559444005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d8c1d5380 0x558d8c1d91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.646+0000 7f7632d9d700 1 -- 172.21.15.119:0/559444005 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76341a68e0 con 0x558d8c1cfad0 2023-12-07T10:33:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.646+0000 7f7632d9d700 1 --2- 172.21.15.119:0/559444005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d8c1cfad0 0x558d8c1d8720 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f76240cb450 tx=0x7f7624085340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5acb5095c9037117 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.646+0000 7f7616ff5700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:33:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.646+0000 7f7616ff5700 1 -- 172.21.15.119:0/559444005 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76240a2d80 con 0x558d8c1cfad0 2023-12-07T10:33:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.646+0000 7f764cbd0b80 1 -- 172.21.15.119:0/559444005 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d8c1c26f0 con 0x558d8c1cfad0 2023-12-07T10:33:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.646+0000 7f7616ff5700 1 -- 172.21.15.119:0/559444005 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f762409ae60 con 0x558d8c1cfad0 2023-12-07T10:33:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.646+0000 7f7616ff5700 1 -- 172.21.15.119:0/559444005 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76240b3150 con 0x558d8c1cfad0 2023-12-07T10:33:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.646+0000 7f764cbd0b80 10 client.27863 fetch_fsmap learned FSMap version 63 2023-12-07T10:33:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.646+0000 7f764cbd0b80 1 -- 172.21.15.119:0/559444005 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x558d8bf4cd80 con 0x558d8c1cfad0 2023-12-07T10:33:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.646+0000 7f7616ff5700 1 -- 172.21.15.119:0/559444005 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558d8c1c26f0 con 0x558d8c1cfad0 2023-12-07T10:33:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.647+0000 7f7616ff5700 1 -- 172.21.15.119:0/559444005 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 63) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f762409bd20 con 0x558d8c1cfad0 2023-12-07T10:33:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.647+0000 7f764cbd0b80 10 client.27863 fetch_fsmap finished waiting for FSMap version 63 2023-12-07T10:33:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.647+0000 7f764cbd0b80 10 client.27863 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:33:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.647+0000 7f764cbd0b80 20 client.27863 populate_metadata read hostname 'smithi119' 2023-12-07T10:33:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.647+0000 7f764cbd0b80 1 --2- 172.21.15.119:0/559444005 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558d8c1f1200 0x558d8c1f3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.647+0000 7f764cbd0b80 4 client.27863 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:33:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.647+0000 7f764cbd0b80 1 -- 172.21.15.119:0/559444005 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d8c19e740 con 0x558d8c1f1200 2023-12-07T10:33:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.647+0000 7f763259c700 1 --2- 172.21.15.119:0/559444005 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558d8c1f1200 0x558d8c1f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.647+0000 7f7616ff5700 10 client.27863 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:33:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.647+0000 7f763259c700 1 --2- 172.21.15.119:0/559444005 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558d8c1f1200 0x558d8c1f3620 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbcdbf911284a8fc server_cookie=9eff215003cdffc2 in_seq=0 out_seq=0 2023-12-07T10:33:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.648+0000 7f7616ff5700 1 -- 172.21.15.119:0/559444005 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558d8c19e740 con 0x558d8c1f1200 2023-12-07T10:33:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.648+0000 7f7616ff5700 10 client.27863 handle_command_reply: tid=0 2023-12-07T10:33:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.648+0000 7f764cbd0b80 1 client.27863 shutdown 2023-12-07T10:33:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.648+0000 7f764cbd0b80 1 -- 172.21.15.119:0/559444005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d8c1cfad0 msgr2=0x558d8c1d8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.648+0000 7f764cbd0b80 1 --2- 172.21.15.119:0/559444005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d8c1cfad0 0x558d8c1d8720 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f76240cb450 tx=0x7f7624085340 comp rx=0 tx=0).stop 2023-12-07T10:33:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.648+0000 7f764cbd0b80 1 -- 172.21.15.119:0/559444005 shutdown_connections 2023-12-07T10:33:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.648+0000 7f764cbd0b80 1 --2- 172.21.15.119:0/559444005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d8c1d5380 0x558d8c1d91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.648+0000 7f764cbd0b80 1 --2- 172.21.15.119:0/559444005 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558d8c1f1200 0x558d8c1f3620 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.648+0000 7f764cbd0b80 1 --2- 172.21.15.119:0/559444005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d8c1cfad0 0x558d8c1d8720 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.648+0000 7f7616ff5700 0 client.27863 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:33:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.648+0000 7f764cbd0b80 1 --2- 172.21.15.119:0/559444005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d8c1d4a60 0x558d8c1d8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.648+0000 7f764cbd0b80 1 -- 172.21.15.119:0/559444005 >> 172.21.15.119:0/559444005 conn(0x558d8c0fe300 msgr2=0x558d8c1c4980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.649+0000 7f764cbd0b80 1 -- 172.21.15.119:0/559444005 shutdown_connections 2023-12-07T10:33:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.649+0000 7f764cbd0b80 1 -- 172.21.15.119:0/559444005 wait complete. 2023-12-07T10:33:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.649+0000 7f764cbd0b80 20 client.27863 trim_cache size 0 max 16384 2023-12-07T10:33:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.665+0000 7f764cbd0b80 1 Processor -- start 2023-12-07T10:33:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.665+0000 7f764cbd0b80 1 -- start start 2023-12-07T10:33:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.666+0000 7f764cbd0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d8c1cfad0 0x558d8c1cf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.666+0000 7f764cbd0b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d8c1d4a60 0x558d8c1e85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.666+0000 7f764cbd0b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d8c1d5380 0x558d8c1e8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.666+0000 7f764cbd0b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8bfa28c0 con 0x558d8c1d4a60 2023-12-07T10:33:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.666+0000 7f764cbd0b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8c1db8d0 con 0x558d8c1d5380 2023-12-07T10:33:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.666+0000 7f764cbd0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8c1c67a0 con 0x558d8c1cfad0 2023-12-07T10:33:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.666+0000 7f763259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d8c1d4a60 0x558d8c1e85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.666+0000 7f763359e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d8c1d5380 0x558d8c1e8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.666+0000 7f7632d9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d8c1cfad0 0x558d8c1cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.666+0000 7f7632d9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d8c1cfad0 0x558d8c1cf540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:56668/0 (socket says 172.21.15.119:56668) 2023-12-07T10:33:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.666+0000 7f763359e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d8c1d5380 0x558d8c1e8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:38632/0 (socket says 172.21.15.119:38632) 2023-12-07T10:33:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.666+0000 7f7632d9d700 1 -- 172.21.15.119:0/1820329294 learned_addr learned my addr 172.21.15.119:0/1820329294 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:33:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.666+0000 7f763259c700 1 -- 172.21.15.119:0/1820329294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d8c1cfad0 msgr2=0x558d8c1cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.666+0000 7f763259c700 1 --2- 172.21.15.119:0/1820329294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d8c1cfad0 0x558d8c1cf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.666+0000 7f763259c700 1 -- 172.21.15.119:0/1820329294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d8c1d5380 msgr2=0x558d8c1e8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.666+0000 7f763259c700 1 --2- 172.21.15.119:0/1820329294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d8c1d5380 0x558d8c1e8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.666+0000 7f763259c700 1 -- 172.21.15.119:0/1820329294 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d8c1c3c30 con 0x558d8c1d4a60 2023-12-07T10:33:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.666+0000 7f7632d9d700 1 --2- 172.21.15.119:0/1820329294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d8c1cfad0 0x558d8c1cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:33:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.666+0000 7f763359e700 1 --2- 172.21.15.119:0/1820329294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d8c1d5380 0x558d8c1e8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:33:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.667+0000 7f763259c700 1 --2- 172.21.15.119:0/1820329294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d8c1d4a60 0x558d8c1e85a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x558d8c1cb890 tx=0x7f7628005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2690c1e93912f335 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.667+0000 7f7619ffb700 1 -- 172.21.15.119:0/1820329294 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762802a070 con 0x558d8c1d4a60 2023-12-07T10:33:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.667+0000 7f7619ffb700 1 -- 172.21.15.119:0/1820329294 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f762801e040 con 0x558d8c1d4a60 2023-12-07T10:33:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.667+0000 7f7619ffb700 1 -- 172.21.15.119:0/1820329294 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762801d040 con 0x558d8c1d4a60 2023-12-07T10:33:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.668+0000 7f764cbd0b80 1 -- 172.21.15.119:0/1820329294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d8c1d4a60 msgr2=0x558d8c1e85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.668+0000 7f764cbd0b80 1 --2- 172.21.15.119:0/1820329294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d8c1d4a60 0x558d8c1e85a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x558d8c1cb890 tx=0x7f7628005960 comp rx=0 tx=0).stop 2023-12-07T10:33:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.668+0000 7f764cbd0b80 1 -- 172.21.15.119:0/1820329294 shutdown_connections 2023-12-07T10:33:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.668+0000 7f764cbd0b80 1 --2- 172.21.15.119:0/1820329294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d8c1d5380 0x558d8c1e8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.668+0000 7f764cbd0b80 1 --2- 172.21.15.119:0/1820329294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d8c1cfad0 0x558d8c1cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.668+0000 7f764cbd0b80 1 --2- 172.21.15.119:0/1820329294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d8c1d4a60 0x558d8c1e85a0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.668+0000 7f764cbd0b80 1 -- 172.21.15.119:0/1820329294 >> 172.21.15.119:0/1820329294 conn(0x558d8c0fe300 msgr2=0x558d8c1ed050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.668+0000 7f764cbd0b80 1 -- 172.21.15.119:0/1820329294 shutdown_connections 2023-12-07T10:33:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.668+0000 7f764cbd0b80 1 -- 172.21.15.119:0/1820329294 wait complete. 2023-12-07T10:33:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.668+0000 7f764cbd0b80 1 -- start start 2023-12-07T10:33:43.671 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.669+0000 7f764cbd0b80 1 Processor -- start 2023-12-07T10:33:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.669+0000 7f764cbd0b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d8c1cfad0 0x558d8c23b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.669+0000 7f764cbd0b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d8c1d4a60 0x558d8c23bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.669+0000 7f764cbd0b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d8c1d5380 0x558d8c23c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.669+0000 7f764cbd0b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8c246710 con 0x558d8c1d4a60 2023-12-07T10:33:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.669+0000 7f764cbd0b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8c246850 con 0x558d8c1d5380 2023-12-07T10:33:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.669+0000 7f764cbd0b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8c246a10 con 0x558d8c1cfad0 2023-12-07T10:33:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.669+0000 7f764cbd0b80 10 client.0 _ll_register_callbacks cb 0x558d8c0c48e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:33:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.669+0000 7f7632d9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d8c1cfad0 0x558d8c23b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.669+0000 7f7632d9d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d8c1cfad0 0x558d8c23b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:56676/0 (socket says 172.21.15.119:56676) 2023-12-07T10:33:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.669+0000 7f7632d9d700 1 -- 172.21.15.119:0/344231827 learned_addr learned my addr 172.21.15.119:0/344231827 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:33:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.669+0000 7f763259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d8c1d4a60 0x558d8c23bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.669+0000 7f763359e700 1 --2- 172.21.15.119:0/344231827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d8c1d5380 0x558d8c23c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.670+0000 7f7632d9d700 1 -- 172.21.15.119:0/344231827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d8c1d5380 msgr2=0x558d8c23c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.670+0000 7f7632d9d700 1 --2- 172.21.15.119:0/344231827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d8c1d5380 0x558d8c23c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.670+0000 7f7632d9d700 1 -- 172.21.15.119:0/344231827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d8c1d4a60 msgr2=0x558d8c23bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.670+0000 7f7632d9d700 1 --2- 172.21.15.119:0/344231827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d8c1d4a60 0x558d8c23bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.670+0000 7f7632d9d700 1 -- 172.21.15.119:0/344231827 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d8bf4cd80 con 0x558d8c1cfad0 2023-12-07T10:33:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.670+0000 7f7632d9d700 1 --2- 172.21.15.119:0/344231827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d8c1cfad0 0x558d8c23b690 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7624097460 tx=0x7f762409a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8873f645488cd674 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.670+0000 7f7616ff5700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:33:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.670+0000 7f7616ff5700 1 -- 172.21.15.119:0/344231827 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762409a530 con 0x558d8c1cfad0 2023-12-07T10:33:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.670+0000 7f7616ff5700 1 -- 172.21.15.119:0/344231827 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f762409b0f0 con 0x558d8c1cfad0 2023-12-07T10:33:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.670+0000 7f764cbd0b80 1 -- 172.21.15.119:0/344231827 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f762409a9f0 con 0x558d8c1cfad0 2023-12-07T10:33:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.670+0000 7f7616ff5700 1 -- 172.21.15.119:0/344231827 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7624010740 con 0x558d8c1cfad0 2023-12-07T10:33:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.670+0000 7f764cbd0b80 10 client.27850 fetch_fsmap learned FSMap version 63 2023-12-07T10:33:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.670+0000 7f764cbd0b80 1 -- 172.21.15.119:0/344231827 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x7f762c021040 con 0x558d8c1cfad0 2023-12-07T10:33:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.671+0000 7f7616ff5700 1 -- 172.21.15.119:0/344231827 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f762407d080 con 0x558d8c1cfad0 2023-12-07T10:33:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.671+0000 7f7616ff5700 1 -- 172.21.15.119:0/344231827 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 63) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7624097100 con 0x558d8c1cfad0 2023-12-07T10:33:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.671+0000 7f764cbd0b80 10 client.27850 fetch_fsmap finished waiting for FSMap version 63 2023-12-07T10:33:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.671+0000 7f764cbd0b80 10 client.27850 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:33:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.671+0000 7f764cbd0b80 20 client.27850 populate_metadata read hostname 'smithi119' 2023-12-07T10:33:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.671+0000 7f764cbd0b80 1 --2- 172.21.15.119:0/344231827 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558d8c247ec0 0x558d8c24a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.671+0000 7f764cbd0b80 4 client.27850 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:33:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.671+0000 7f764cbd0b80 1 -- 172.21.15.119:0/344231827 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558d8c0e9f70 con 0x558d8c247ec0 2023-12-07T10:33:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.671+0000 7f763259c700 1 --2- 172.21.15.119:0/344231827 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558d8c247ec0 0x558d8c24a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.672+0000 7f763259c700 1 --2- 172.21.15.119:0/344231827 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558d8c247ec0 0x558d8c24a2e0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2267de87191c6c2 server_cookie=8b7b5a3870d55873 in_seq=0 out_seq=0 2023-12-07T10:33:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.672+0000 7f7616ff5700 10 client.27850 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:33:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.751+0000 7f7616ff5700 1 -- 172.21.15.119:0/344231827 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558d8c0e9f70 con 0x558d8c247ec0 2023-12-07T10:33:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.751+0000 7f7616ff5700 10 client.27850 handle_command_reply: tid=0 2023-12-07T10:33:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.751+0000 7f764cbd0b80 1 client.27850 shutdown 2023-12-07T10:33:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.751+0000 7f764cbd0b80 1 -- 172.21.15.119:0/344231827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d8c1cfad0 msgr2=0x558d8c23b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.751+0000 7f764cbd0b80 1 --2- 172.21.15.119:0/344231827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d8c1cfad0 0x558d8c23b690 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7624097460 tx=0x7f762409a270 comp rx=0 tx=0).stop 2023-12-07T10:33:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.751+0000 7f764cbd0b80 1 -- 172.21.15.119:0/344231827 shutdown_connections 2023-12-07T10:33:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.751+0000 7f764cbd0b80 1 --2- 172.21.15.119:0/344231827 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x558d8c247ec0 0x558d8c24a2e0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.751+0000 7f764cbd0b80 1 --2- 172.21.15.119:0/344231827 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558d8c1d5380 0x558d8c23c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.751+0000 7f764cbd0b80 1 --2- 172.21.15.119:0/344231827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d8c1cfad0 0x558d8c23b690 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.751+0000 7f764cbd0b80 1 --2- 172.21.15.119:0/344231827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558d8c1d4a60 0x558d8c23bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.751+0000 7f7616ff5700 0 client.27850 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:33:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.751+0000 7f764cbd0b80 1 -- 172.21.15.119:0/344231827 >> 172.21.15.119:0/344231827 conn(0x558d8c0fe300 msgr2=0x558d8c1d3910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.751+0000 7f764cbd0b80 1 -- 172.21.15.119:0/344231827 shutdown_connections 2023-12-07T10:33:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.751+0000 7f764cbd0b80 1 -- 172.21.15.119:0/344231827 wait complete. 2023-12-07T10:33:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.751+0000 7f764cbd0b80 20 client.27850 trim_cache size 0 max 16384 2023-12-07T10:33:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:33:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:33:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:33:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:33:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.752+0000 7f76391ff700 1 -- 172.21.15.119:0/942181814 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f761c0a1c80 msgr2=0x7f761c0a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.752+0000 7f76391ff700 1 --2- 172.21.15.119:0/942181814 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f761c0a1c80 0x7f761c0a4140 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f76341a05f0 tx=0x7f762c007b00 comp rx=0 tx=0).stop 2023-12-07T10:33:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.752+0000 7f76391ff700 1 -- 172.21.15.119:0/942181814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76340739e0 msgr2=0x7f763419f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.752+0000 7f76391ff700 1 --2- 172.21.15.119:0/942181814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76340739e0 0x7f763419f960 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f762400e9a0 tx=0x7f762400ecb0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.752+0000 7f76391ff700 1 -- 172.21.15.119:0/942181814 shutdown_connections 2023-12-07T10:33:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.752+0000 7f76391ff700 1 --2- 172.21.15.119:0/942181814 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f761c0a1c80 0x7f761c0a4140 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.752+0000 7f76391ff700 1 --2- 172.21.15.119:0/942181814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f763410dda0 0x7f7634199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.752+0000 7f76391ff700 1 --2- 172.21.15.119:0/942181814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76340739e0 0x7f763419f960 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.752+0000 7f76391ff700 1 --2- 172.21.15.119:0/942181814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7634074300 0x7f763419fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.752+0000 7f76391ff700 1 -- 172.21.15.119:0/942181814 >> 172.21.15.119:0/942181814 conn(0x7f76340fd010 msgr2=0x7f76340fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.752+0000 7f76391ff700 1 -- 172.21.15.119:0/942181814 shutdown_connections 2023-12-07T10:33:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:43.752+0000 7f76391ff700 1 -- 172.21.15.119:0/942181814 wait complete. 2023-12-07T10:33:44.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:44 smithi119 ceph-mon[97908]: pgmap v416: 129 pgs: 129 active+clean; 776 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 163 op/s 2023-12-07T10:33:44.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:44 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:33:44.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:44 smithi159 ceph-mon[134726]: pgmap v416: 129 pgs: 129 active+clean; 776 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 163 op/s 2023-12-07T10:33:44.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:44 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:33:44.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:44 smithi153 ceph-mon[133625]: pgmap v416: 129 pgs: 129 active+clean; 776 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 163 op/s 2023-12-07T10:33:44.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:44 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:33:44.584 INFO:tasks.workunit.client.0.smithi119.stdout: 1 981537 7.22 MB/sec execute 597 sec latency 34.700 ms 2023-12-07T10:33:45.584 INFO:tasks.workunit.client.0.smithi119.stdout: 1 983443 7.22 MB/sec execute 598 sec latency 19.877 ms 2023-12-07T10:33:46.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:46 smithi119 ceph-mon[97908]: pgmap v417: 129 pgs: 129 active+clean; 776 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 159 op/s 2023-12-07T10:33:46.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:46 smithi159 ceph-mon[134726]: pgmap v417: 129 pgs: 129 active+clean; 776 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 159 op/s 2023-12-07T10:33:46.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:46 smithi153 ceph-mon[133625]: pgmap v417: 129 pgs: 129 active+clean; 776 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 159 op/s 2023-12-07T10:33:46.584 INFO:tasks.workunit.client.0.smithi119.stdout: 1 984702 7.22 MB/sec execute 599 sec latency 12.471 ms 2023-12-07T10:33:47.584 INFO:tasks.workunit.client.0.smithi119.stdout: 1 cleanup 600 sec 2023-12-07T10:33:47.855 INFO:tasks.workunit.client.0.smithi119.stdout: 0 cleanup 600 sec 2023-12-07T10:33:47.855 INFO:tasks.workunit.client.0.smithi119.stdout: 2023-12-07T10:33:47.855 INFO:tasks.workunit.client.0.smithi119.stdout: Operation Count AvgLat MaxLat 2023-12-07T10:33:47.855 INFO:tasks.workunit.client.0.smithi119.stdout: ---------------------------------------- 2023-12-07T10:33:47.855 INFO:tasks.workunit.client.0.smithi119.stdout: NTCreateX 138094 1.827 14764.077 2023-12-07T10:33:47.855 INFO:tasks.workunit.client.0.smithi119.stdout: Close 101441 0.003 0.879 2023-12-07T10:33:47.856 INFO:tasks.workunit.client.0.smithi119.stdout: Rename 5841 6.960 295.460 2023-12-07T10:33:47.856 INFO:tasks.workunit.client.0.smithi119.stdout: Unlink 27881 2.138 4724.182 2023-12-07T10:33:47.856 INFO:tasks.workunit.client.0.smithi119.stdout: Deltree 4 105.854 173.198 2023-12-07T10:33:47.856 INFO:tasks.workunit.client.0.smithi119.stdout: Mkdir 2 0.015 0.015 2023-12-07T10:33:47.856 INFO:tasks.workunit.client.0.smithi119.stdout: Qpathinfo 125136 1.357 7115.706 2023-12-07T10:33:47.856 INFO:tasks.workunit.client.0.smithi119.stdout: Qfileinfo 21929 0.002 0.319 2023-12-07T10:33:47.856 INFO:tasks.workunit.client.0.smithi119.stdout: Qfsinfo 22920 0.265 15.819 2023-12-07T10:33:47.856 INFO:tasks.workunit.client.0.smithi119.stdout: Sfileinfo 11242 0.034 11.067 2023-12-07T10:33:47.856 INFO:tasks.workunit.client.0.smithi119.stdout: Find 48340 0.273 263.058 2023-12-07T10:33:47.856 INFO:tasks.workunit.client.0.smithi119.stdout: WriteX 68885 0.162 224.283 2023-12-07T10:33:47.856 INFO:tasks.workunit.client.0.smithi119.stdout: ReadX 216086 0.012 10.190 2023-12-07T10:33:47.856 INFO:tasks.workunit.client.0.smithi119.stdout: LockX 448 0.630 3.176 2023-12-07T10:33:47.856 INFO:tasks.workunit.client.0.smithi119.stdout: UnlockX 448 0.611 3.548 2023-12-07T10:33:47.857 INFO:tasks.workunit.client.0.smithi119.stdout: Flush 9674 4.522 273.756 2023-12-07T10:33:47.857 INFO:tasks.workunit.client.0.smithi119.stdout: 2023-12-07T10:33:47.857 INFO:tasks.workunit.client.0.smithi119.stdout:Throughput 7.22428 MB/sec 1 clients 1 procs max_latency=14764.090 ms 2023-12-07T10:33:47.857 INFO:tasks.workunit.client.0.smithi119.stderr:+ dbench 10 2023-12-07T10:33:47.857 INFO:tasks.workunit.client.0.smithi119.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2023-12-07T10:33:47.857 INFO:tasks.workunit.client.0.smithi119.stdout: 2023-12-07T10:33:47.857 INFO:tasks.workunit.client.0.smithi119.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2023-12-07T10:33:47.859 INFO:tasks.workunit.client.0.smithi119.stdout:9 of 10 processes prepared for launch 0 sec 2023-12-07T10:33:47.960 INFO:tasks.workunit.client.0.smithi119.stdout:10 of 10 processes prepared for launch 0 sec 2023-12-07T10:33:47.960 INFO:tasks.workunit.client.0.smithi119.stdout:releasing clients 2023-12-07T10:33:48.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:48 smithi119 ceph-mon[97908]: pgmap v418: 129 pgs: 129 active+clean; 784 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 261 op/s 2023-12-07T10:33:48.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:48 smithi159 ceph-mon[134726]: pgmap v418: 129 pgs: 129 active+clean; 784 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 261 op/s 2023-12-07T10:33:48.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:48 smithi153 ceph-mon[133625]: pgmap v418: 129 pgs: 129 active+clean; 784 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 261 op/s 2023-12-07T10:33:48.960 INFO:tasks.workunit.client.0.smithi119.stdout: 10 344 123.94 MB/sec warmup 1 sec latency 93.997 ms 2023-12-07T10:33:49.960 INFO:tasks.workunit.client.0.smithi119.stdout: 10 699 102.78 MB/sec warmup 2 sec latency 129.343 ms 2023-12-07T10:33:50.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:50 smithi119 ceph-mon[97908]: pgmap v419: 129 pgs: 129 active+clean; 784 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 184 op/s 2023-12-07T10:33:50.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:50 smithi159 ceph-mon[134726]: pgmap v419: 129 pgs: 129 active+clean; 784 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 184 op/s 2023-12-07T10:33:50.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:50 smithi153 ceph-mon[133625]: pgmap v419: 129 pgs: 129 active+clean; 784 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 184 op/s 2023-12-07T10:33:50.960 INFO:tasks.workunit.client.0.smithi119.stdout: 10 990 73.05 MB/sec warmup 3 sec latency 351.388 ms 2023-12-07T10:33:51.960 INFO:tasks.workunit.client.0.smithi119.stdout: 10 1527 64.17 MB/sec warmup 4 sec latency 142.240 ms 2023-12-07T10:33:52.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:52 smithi119 ceph-mon[97908]: pgmap v420: 129 pgs: 129 active+clean; 789 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 239 op/s 2023-12-07T10:33:52.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:52 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:33:52.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:52 smithi159 ceph-mon[134726]: pgmap v420: 129 pgs: 129 active+clean; 789 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 239 op/s 2023-12-07T10:33:52.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:52 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:33:52.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:52 smithi153 ceph-mon[133625]: pgmap v420: 129 pgs: 129 active+clean; 789 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 239 op/s 2023-12-07T10:33:52.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:52 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:33:52.960 INFO:tasks.workunit.client.0.smithi119.stdout: 10 1841 52.06 MB/sec warmup 5 sec latency 473.105 ms 2023-12-07T10:33:53.960 INFO:tasks.workunit.client.0.smithi119.stdout: 10 2321 46.21 MB/sec warmup 6 sec latency 248.804 ms 2023-12-07T10:33:54.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:54 smithi119 ceph-mon[97908]: pgmap v421: 129 pgs: 129 active+clean; 794 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 212 op/s 2023-12-07T10:33:54.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:54 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:33:54.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:54 smithi159 ceph-mon[134726]: pgmap v421: 129 pgs: 129 active+clean; 794 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 212 op/s 2023-12-07T10:33:54.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:54 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:33:54.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:54 smithi153 ceph-mon[133625]: pgmap v421: 129 pgs: 129 active+clean; 794 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 212 op/s 2023-12-07T10:33:54.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:54 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:33:54.961 INFO:tasks.workunit.client.0.smithi119.stdout: 10 3481 53.07 MB/sec warmup 7 sec latency 61.378 ms 2023-12-07T10:33:55.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:55 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:33:55.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:55 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:33:55.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:55 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:33:55.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:55 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:33:55.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:55 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:33:55.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:55 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:33:55.960 INFO:tasks.workunit.client.0.smithi119.stdout: 10 4098 50.06 MB/sec warmup 8 sec latency 820.007 ms 2023-12-07T10:33:56.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:56 smithi119 ceph-mon[97908]: pgmap v422: 129 pgs: 129 active+clean; 794 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 207 op/s 2023-12-07T10:33:56.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:56 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:33:56.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:56 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:33:56.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:56 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:33:56.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:56 smithi159 ceph-mon[134726]: pgmap v422: 129 pgs: 129 active+clean; 794 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 207 op/s 2023-12-07T10:33:56.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:56 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:33:56.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:56 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:33:56.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:56 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:33:56.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:56 smithi153 ceph-mon[133625]: pgmap v422: 129 pgs: 129 active+clean; 794 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 207 op/s 2023-12-07T10:33:56.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:56 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:33:56.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:56 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:33:56.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:56 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:33:56.962 INFO:tasks.workunit.client.0.smithi119.stdout: 10 4815 48.05 MB/sec warmup 9 sec latency 1820.080 ms 2023-12-07T10:33:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.961+0000 7faaef3ff700 1 -- 172.21.15.119:0/2945439949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faae8109710 msgr2=0x7faae8109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.961+0000 7faaef3ff700 1 --2- 172.21.15.119:0/2945439949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faae8109710 0x7faae8109af0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7faae4009990 tx=0x7faae4009ca0 comp rx=0 tx=0).stop 2023-12-07T10:33:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.962+0000 7faaef3ff700 1 -- 172.21.15.119:0/2945439949 shutdown_connections 2023-12-07T10:33:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.962+0000 7faaef3ff700 1 --2- 172.21.15.119:0/2945439949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faae81040c0 0x7faae81105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.962+0000 7faaef3ff700 1 --2- 172.21.15.119:0/2945439949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faae8109710 0x7faae8109af0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.962+0000 7faaef3ff700 1 --2- 172.21.15.119:0/2945439949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faae8103700 0x7faae8103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.962+0000 7faaef3ff700 1 -- 172.21.15.119:0/2945439949 >> 172.21.15.119:0/2945439949 conn(0x7faae80ff190 msgr2=0x7faae81015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.962+0000 7faaef3ff700 1 -- 172.21.15.119:0/2945439949 shutdown_connections 2023-12-07T10:33:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.962+0000 7faaef3ff700 1 -- 172.21.15.119:0/2945439949 wait complete. 2023-12-07T10:33:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.963+0000 7faaef3ff700 1 Processor -- start 2023-12-07T10:33:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.963+0000 7faaef3ff700 1 -- start start 2023-12-07T10:33:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.964+0000 7faaef3ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faae8103700 0x7faae8110540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.964+0000 7faaef3ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faae81040c0 0x7faae810d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.964+0000 7faaef3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faae810dcd0 0x7faae81aa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.964+0000 7faaef3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae810e380 con 0x7faae810dcd0 2023-12-07T10:33:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.964+0000 7faaef3ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae810e4f0 con 0x7faae81040c0 2023-12-07T10:33:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.964+0000 7faaef3ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae81aab20 con 0x7faae8103700 2023-12-07T10:33:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.964+0000 7faaed19b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faae8103700 0x7faae8110540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.964+0000 7faaec99a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faae81040c0 0x7faae810d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.964+0000 7faaed19b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faae8103700 0x7faae8110540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55606/0 (socket says 172.21.15.119:55606) 2023-12-07T10:33:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.964+0000 7faaec99a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faae81040c0 0x7faae810d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:41896/0 (socket says 172.21.15.119:41896) 2023-12-07T10:33:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.964+0000 7faaed19b700 1 -- 172.21.15.119:0/2344607039 learned_addr learned my addr 172.21.15.119:0/2344607039 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:33:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.965+0000 7faaed99c700 1 --2- 172.21.15.119:0/2344607039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faae810dcd0 0x7faae81aa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.965+0000 7faaec99a700 1 -- 172.21.15.119:0/2344607039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faae8103700 msgr2=0x7faae8110540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.965+0000 7faaec99a700 1 --2- 172.21.15.119:0/2344607039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faae8103700 0x7faae8110540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.965+0000 7faaec99a700 1 -- 172.21.15.119:0/2344607039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faae810dcd0 msgr2=0x7faae81aa5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.965+0000 7faaec99a700 1 --2- 172.21.15.119:0/2344607039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faae810dcd0 0x7faae81aa5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.965+0000 7faaec99a700 1 -- 172.21.15.119:0/2344607039 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae40095f0 con 0x7faae81040c0 2023-12-07T10:33:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.965+0000 7faaed99c700 1 --2- 172.21.15.119:0/2344607039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faae810dcd0 0x7faae81aa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:33:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.965+0000 7faaec99a700 1 --2- 172.21.15.119:0/2344607039 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faae81040c0 0x7faae810d680 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7faad800ea20 tx=0x7faad800ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.965+0000 7faade7fc700 1 -- 172.21.15.119:0/2344607039 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad800cbd0 con 0x7faae81040c0 2023-12-07T10:33:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.966+0000 7faade7fc700 1 -- 172.21.15.119:0/2344607039 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faad800cd30 con 0x7faae81040c0 2023-12-07T10:33:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.966+0000 7faaef3ff700 1 -- 172.21.15.119:0/2344607039 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae81aad70 con 0x7faae81040c0 2023-12-07T10:33:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.966+0000 7faaef3ff700 1 -- 172.21.15.119:0/2344607039 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae81ab210 con 0x7faae81040c0 2023-12-07T10:33:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.967+0000 7faade7fc700 1 -- 172.21.15.119:0/2344607039 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad8018680 con 0x7faae81040c0 2023-12-07T10:33:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.971+0000 7faade7fc700 1 -- 172.21.15.119:0/2344607039 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7faad80187e0 con 0x7faae81040c0 2023-12-07T10:33:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.974+0000 7faade7fc700 1 --2- 172.21.15.119:0/2344607039 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7faad40a1d40 0x7faad40a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.974+0000 7faade7fc700 1 -- 172.21.15.119:0/2344607039 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7faad8014070 con 0x7faae81040c0 2023-12-07T10:33:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.974+0000 7fab02d9ab80 1 Processor -- start 2023-12-07T10:33:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.974+0000 7faaed19b700 1 --2- 172.21.15.119:0/2344607039 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7faad40a1d40 0x7faad40a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.975+0000 7fab02d9ab80 1 -- start start 2023-12-07T10:33:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.975+0000 7fab02d9ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f1973fa60 0x556f1973fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.975+0000 7fab02d9ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f19740380 0x556f1973a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.975+0000 7fab02d9ab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f1973aad0 0x556f1973af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.975+0000 7fab02d9ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f196ae760 con 0x556f19740380 2023-12-07T10:33:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.975+0000 7fab02d9ab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f19695740 con 0x556f1973aad0 2023-12-07T10:33:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.975+0000 7fab02d9ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f19693060 con 0x556f1973fa60 2023-12-07T10:33:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.976+0000 7faaed99c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f1973aad0 0x556f1973af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.976+0000 7faaed99c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f1973aad0 0x556f1973af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:41910/0 (socket says 172.21.15.119:41910) 2023-12-07T10:33:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.976+0000 7faaed99c700 1 -- 172.21.15.119:0/1113195768 learned_addr learned my addr 172.21.15.119:0/1113195768 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:33:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.976+0000 7faaed19b700 1 --2- 172.21.15.119:0/1113195768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f1973fa60 0x556f1973fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.976+0000 7faaed99c700 1 -- 172.21.15.119:0/1113195768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f1973fa60 msgr2=0x556f1973fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.976+0000 7faaed99c700 1 --2- 172.21.15.119:0/1113195768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f1973fa60 0x556f1973fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.976+0000 7faaed99c700 1 -- 172.21.15.119:0/1113195768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f19740380 msgr2=0x556f1973a540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-07T10:33:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.977+0000 7faaec99a700 1 --2- 172.21.15.119:0/1113195768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f19740380 0x556f1973a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.977+0000 7faaed99c700 1 --2- 172.21.15.119:0/1113195768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f19740380 0x556f1973a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.977+0000 7faaed99c700 1 -- 172.21.15.119:0/1113195768 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f1950b5d0 con 0x556f1973aad0 2023-12-07T10:33:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.977+0000 7faaed19b700 1 --2- 172.21.15.119:0/1113195768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f1973fa60 0x556f1973fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:33:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.977+0000 7faaed99c700 1 --2- 172.21.15.119:0/1113195768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f1973aad0 0x556f1973af30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7faae0007f30 tx=0x7faae000c5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8feaaa68af0b1adf server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.977+0000 7faad37fe700 1 -- 172.21.15.119:0/1113195768 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae000f440 con 0x556f1973aad0 2023-12-07T10:33:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.978+0000 7faad37fe700 1 -- 172.21.15.119:0/1113195768 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faae000fa60 con 0x556f1973aad0 2023-12-07T10:33:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.978+0000 7faad37fe700 1 -- 172.21.15.119:0/1113195768 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae000e9e0 con 0x556f1973aad0 2023-12-07T10:33:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.978+0000 7faaec99a700 1 --2- 172.21.15.119:0/1113195768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f19740380 0x556f1973a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:33:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.979+0000 7fab02d9ab80 1 -- 172.21.15.119:0/1113195768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f1973aad0 msgr2=0x556f1973af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.979+0000 7fab02d9ab80 1 --2- 172.21.15.119:0/1113195768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f1973aad0 0x556f1973af30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7faae0007f30 tx=0x7faae000c5a0 comp rx=0 tx=0).stop 2023-12-07T10:33:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.979+0000 7faaed19b700 1 --2- 172.21.15.119:0/2344607039 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7faad40a1d40 0x7faad40a4200 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faae400b6f0 tx=0x7faae4005c00 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.980+0000 7fab02d9ab80 1 -- 172.21.15.119:0/1113195768 shutdown_connections 2023-12-07T10:33:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.980+0000 7fab02d9ab80 1 --2- 172.21.15.119:0/1113195768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f1973aad0 0x556f1973af30 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.980+0000 7fab02d9ab80 1 --2- 172.21.15.119:0/1113195768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f1973fa60 0x556f1973fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.980+0000 7fab02d9ab80 1 --2- 172.21.15.119:0/1113195768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f19740380 0x556f1973a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.980+0000 7fab02d9ab80 1 -- 172.21.15.119:0/1113195768 >> 172.21.15.119:0/1113195768 conn(0x556f19669300 msgr2=0x556f19734b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.981+0000 7fab02d9ab80 1 -- 172.21.15.119:0/1113195768 shutdown_connections 2023-12-07T10:33:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.981+0000 7fab02d9ab80 1 -- 172.21.15.119:0/1113195768 wait complete. 2023-12-07T10:33:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.982+0000 7fab02d9ab80 1 -- start start 2023-12-07T10:33:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.982+0000 7fab02d9ab80 1 Processor -- start 2023-12-07T10:33:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.985+0000 7fab02d9ab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f1973aad0 0x556f197462d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.985+0000 7fab02d9ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f1973fa60 0x556f197435a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.985+0000 7fab02d9ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f19740380 0x556f19743bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.985+0000 7fab02d9ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f1968e950 con 0x556f19740380 2023-12-07T10:33:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.985+0000 7fab02d9ab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f19648500 con 0x556f1973aad0 2023-12-07T10:33:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.985+0000 7fab02d9ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f196481a0 con 0x556f1973fa60 2023-12-07T10:33:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.985+0000 7fab02d9ab80 10 client.0 _ll_register_callbacks cb 0x556f1962f8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:33:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.985+0000 7faaec99a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f1973fa60 0x556f197435a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.985+0000 7faaec99a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f1973fa60 0x556f197435a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55630/0 (socket says 172.21.15.119:55630) 2023-12-07T10:33:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.985+0000 7faaed99c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f19740380 0x556f19743bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.985+0000 7faaec99a700 1 -- 172.21.15.119:0/3605563836 learned_addr learned my addr 172.21.15.119:0/3605563836 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:33:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.985+0000 7faaed19b700 1 --2- 172.21.15.119:0/3605563836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f1973aad0 0x556f197462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.985+0000 7faaed99c700 1 -- 172.21.15.119:0/3605563836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f1973fa60 msgr2=0x556f197435a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.986+0000 7faaed99c700 1 --2- 172.21.15.119:0/3605563836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f1973fa60 0x556f197435a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.986+0000 7faaed99c700 1 -- 172.21.15.119:0/3605563836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f1973aad0 msgr2=0x556f197462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.986+0000 7faaed99c700 1 --2- 172.21.15.119:0/3605563836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f1973aad0 0x556f197462d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.986+0000 7faaed99c700 1 -- 172.21.15.119:0/3605563836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae0007b60 con 0x556f19740380 2023-12-07T10:33:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.986+0000 7faaec99a700 1 --2- 172.21.15.119:0/3605563836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f1973fa60 0x556f197435a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:33:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.986+0000 7faaed99c700 1 --2- 172.21.15.119:0/3605563836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f19740380 0x556f19743bb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7faae000c9e0 tx=0x7faae0007450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f280269d2cd14b43 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.986+0000 7faad0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:33:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.986+0000 7faad0ff9700 1 -- 172.21.15.119:0/3605563836 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae001e070 con 0x556f19740380 2023-12-07T10:33:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.986+0000 7faad0ff9700 1 -- 172.21.15.119:0/3605563836 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7faae0003cb0 con 0x556f19740380 2023-12-07T10:33:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.987+0000 7fab02d9ab80 1 -- 172.21.15.119:0/3605563836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f1972d6f0 con 0x556f19740380 2023-12-07T10:33:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.987+0000 7faad0ff9700 1 -- 172.21.15.119:0/3605563836 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7faae0003e70 con 0x556f19740380 2023-12-07T10:33:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.987+0000 7fab02d9ab80 10 client.17829 fetch_fsmap learned FSMap version 66 2023-12-07T10:33:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.987+0000 7fab02d9ab80 1 -- 172.21.15.119:0/3605563836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x556f194b7d80 con 0x556f19740380 2023-12-07T10:33:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.987+0000 7faad0ff9700 1 -- 172.21.15.119:0/3605563836 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556f1972d6f0 con 0x556f19740380 2023-12-07T10:33:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.988+0000 7faad0ff9700 1 -- 172.21.15.119:0/3605563836 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 66) v1 ==== 7930+0+0 (secure 0 0 0) 0x7faae0004040 con 0x556f19740380 2023-12-07T10:33:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.988+0000 7fab02d9ab80 10 client.17829 fetch_fsmap finished waiting for FSMap version 66 2023-12-07T10:33:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.988+0000 7fab02d9ab80 10 client.17829 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:33:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.988+0000 7fab02d9ab80 20 client.17829 populate_metadata read hostname 'smithi119' 2023-12-07T10:33:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.988+0000 7fab02d9ab80 1 --2- 172.21.15.119:0/3605563836 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556f19753c20 0x556f19756040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.988+0000 7fab02d9ab80 4 client.17829 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:33:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.988+0000 7fab02d9ab80 1 -- 172.21.15.119:0/3605563836 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f19709740 con 0x556f19753c20 2023-12-07T10:33:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.989+0000 7faaed19b700 1 --2- 172.21.15.119:0/3605563836 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556f19753c20 0x556f19756040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.990+0000 7faaed19b700 1 --2- 172.21.15.119:0/3605563836 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556f19753c20 0x556f19756040 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d457f99a663209b server_cookie=678773e83d7bbdb in_seq=0 out_seq=0 2023-12-07T10:33:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.990+0000 7faad0ff9700 10 client.17829 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:33:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.990+0000 7faad0ff9700 1 -- 172.21.15.119:0/3605563836 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556f19709740 con 0x556f19753c20 2023-12-07T10:33:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.990+0000 7faad0ff9700 10 client.17829 handle_command_reply: tid=0 2023-12-07T10:33:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.990+0000 7fab02d9ab80 1 client.17829 shutdown 2023-12-07T10:33:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.991+0000 7fab02d9ab80 1 -- 172.21.15.119:0/3605563836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f19740380 msgr2=0x556f19743bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.991+0000 7fab02d9ab80 1 --2- 172.21.15.119:0/3605563836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f19740380 0x556f19743bb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7faae000c9e0 tx=0x7faae0007450 comp rx=0 tx=0).stop 2023-12-07T10:33:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.991+0000 7fab02d9ab80 1 -- 172.21.15.119:0/3605563836 shutdown_connections 2023-12-07T10:33:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.991+0000 7fab02d9ab80 1 --2- 172.21.15.119:0/3605563836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f19740380 0x556f19743bb0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.991+0000 7fab02d9ab80 1 --2- 172.21.15.119:0/3605563836 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556f19753c20 0x556f19756040 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.991+0000 7fab02d9ab80 1 --2- 172.21.15.119:0/3605563836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f1973aad0 0x556f197462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.991+0000 7fab02d9ab80 1 --2- 172.21.15.119:0/3605563836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f1973fa60 0x556f197435a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.991+0000 7faad0ff9700 0 client.17829 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:33:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.991+0000 7fab02d9ab80 1 -- 172.21.15.119:0/3605563836 >> 172.21.15.119:0/3605563836 conn(0x556f19669300 msgr2=0x556f1972f980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.991+0000 7fab02d9ab80 1 -- 172.21.15.119:0/3605563836 shutdown_connections 2023-12-07T10:33:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.991+0000 7fab02d9ab80 1 -- 172.21.15.119:0/3605563836 wait complete. 2023-12-07T10:33:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:56.992+0000 7fab02d9ab80 20 client.17829 trim_cache size 0 max 16384 2023-12-07T10:33:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.015+0000 7fab02d9ab80 1 Processor -- start 2023-12-07T10:33:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.015+0000 7fab02d9ab80 1 -- start start 2023-12-07T10:33:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.016+0000 7fab02d9ab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f1973aad0 0x556f1974ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.016+0000 7fab02d9ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f1973fa60 0x556f1974b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.016+0000 7fab02d9ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f19740380 0x556f1973d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.016+0000 7fab02d9ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f1950d8c0 con 0x556f19740380 2023-12-07T10:33:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.016+0000 7fab02d9ab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f19736a80 con 0x556f1973aad0 2023-12-07T10:33:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.016+0000 7fab02d9ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f1972d6c0 con 0x556f1973fa60 2023-12-07T10:33:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.016+0000 7faaec99a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f1973fa60 0x556f1974b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.016+0000 7faaec99a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f1973fa60 0x556f1974b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:55638/0 (socket says 172.21.15.119:55638) 2023-12-07T10:33:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.016+0000 7faaec99a700 1 -- 172.21.15.119:0/4140941352 learned_addr learned my addr 172.21.15.119:0/4140941352 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:33:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.016+0000 7faaed99c700 1 --2- 172.21.15.119:0/4140941352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f19740380 0x556f1973d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.017+0000 7faaed19b700 1 --2- 172.21.15.119:0/4140941352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f1973aad0 0x556f1974ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.017+0000 7faaec99a700 1 -- 172.21.15.119:0/4140941352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f1973aad0 msgr2=0x556f1974ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.017+0000 7faaec99a700 1 --2- 172.21.15.119:0/4140941352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f1973aad0 0x556f1974ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.017+0000 7faaec99a700 1 -- 172.21.15.119:0/4140941352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f19740380 msgr2=0x556f1973d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.017+0000 7faaec99a700 1 --2- 172.21.15.119:0/4140941352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f19740380 0x556f1973d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.017+0000 7faaec99a700 1 -- 172.21.15.119:0/4140941352 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f194a0c00 con 0x556f1973fa60 2023-12-07T10:33:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.017+0000 7faaed99c700 1 --2- 172.21.15.119:0/4140941352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f19740380 0x556f1973d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:33:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.018+0000 7faaec99a700 1 --2- 172.21.15.119:0/4140941352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f1973fa60 0x556f1974b340 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7faad80cb7e0 tx=0x7faad8085040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef3ce3c0d0296053 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.018+0000 7faad3fff700 1 -- 172.21.15.119:0/4140941352 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad809a830 con 0x556f1973fa60 2023-12-07T10:33:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.018+0000 7faad3fff700 1 -- 172.21.15.119:0/4140941352 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faad809e980 con 0x556f1973fa60 2023-12-07T10:33:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.018+0000 7faad3fff700 1 -- 172.21.15.119:0/4140941352 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad8099f80 con 0x556f1973fa60 2023-12-07T10:33:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.019+0000 7fab02d9ab80 1 -- 172.21.15.119:0/4140941352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f1973fa60 msgr2=0x556f1974b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.019+0000 7fab02d9ab80 1 --2- 172.21.15.119:0/4140941352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f1973fa60 0x556f1974b340 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7faad80cb7e0 tx=0x7faad8085040 comp rx=0 tx=0).stop 2023-12-07T10:33:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.019+0000 7fab02d9ab80 1 -- 172.21.15.119:0/4140941352 shutdown_connections 2023-12-07T10:33:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.019+0000 7fab02d9ab80 1 --2- 172.21.15.119:0/4140941352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f19740380 0x556f1973d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.019+0000 7fab02d9ab80 1 --2- 172.21.15.119:0/4140941352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f1973aad0 0x556f1974ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.019+0000 7fab02d9ab80 1 --2- 172.21.15.119:0/4140941352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f1973fa60 0x556f1974b340 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.019+0000 7fab02d9ab80 1 -- 172.21.15.119:0/4140941352 >> 172.21.15.119:0/4140941352 conn(0x556f19669300 msgr2=0x556f1974f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.019+0000 7fab02d9ab80 1 -- 172.21.15.119:0/4140941352 shutdown_connections 2023-12-07T10:33:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.019+0000 7fab02d9ab80 1 -- 172.21.15.119:0/4140941352 wait complete. 2023-12-07T10:33:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.020+0000 7fab02d9ab80 1 -- start start 2023-12-07T10:33:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.021+0000 7fab02d9ab80 1 Processor -- start 2023-12-07T10:33:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.022+0000 7fab02d9ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f1973aad0 0x556f1974b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.022+0000 7fab02d9ab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f1973fa60 0x556f19741e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.022+0000 7fab02d9ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f19740380 0x556f19740760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.022+0000 7fab02d9ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f19742740 con 0x556f1973aad0 2023-12-07T10:33:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.022+0000 7fab02d9ab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f197428b0 con 0x556f1973fa60 2023-12-07T10:33:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.022+0000 7fab02d9ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f19740ca0 con 0x556f19740380 2023-12-07T10:33:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.022+0000 7fab02d9ab80 10 client.0 _ll_register_callbacks cb 0x556f1962f8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:33:57.023 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.022+0000 7faaec99a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f1973fa60 0x556f19741e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.022+0000 7faaec99a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f1973fa60 0x556f19741e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:41944/0 (socket says 172.21.15.119:41944) 2023-12-07T10:33:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.022+0000 7faaec99a700 1 -- 172.21.15.119:0/337236447 learned_addr learned my addr 172.21.15.119:0/337236447 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:33:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.023+0000 7faaed99c700 1 --2- 172.21.15.119:0/337236447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f19740380 0x556f19740760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.023+0000 7faaec99a700 1 -- 172.21.15.119:0/337236447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f19740380 msgr2=0x556f19740760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:33:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.023+0000 7faaec99a700 1 --2- 172.21.15.119:0/337236447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f19740380 0x556f19740760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.023+0000 7faaec99a700 1 -- 172.21.15.119:0/337236447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f1973aad0 msgr2=0x556f1974b2d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-07T10:33:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.023+0000 7faaec99a700 1 --2- 172.21.15.119:0/337236447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f1973aad0 0x556f1974b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.023+0000 7faaec99a700 1 -- 172.21.15.119:0/337236447 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae81ab210 con 0x556f1973fa60 2023-12-07T10:33:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.023+0000 7faaec99a700 1 --2- 172.21.15.119:0/337236447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f1973fa60 0x556f19741e30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7faad809a7c0 tx=0x7faad8005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1465781951ca1a72 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:33:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.023+0000 7faad0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:33:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.024+0000 7faad0ff9700 1 -- 172.21.15.119:0/337236447 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad809a830 con 0x556f1973fa60 2023-12-07T10:33:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.024+0000 7faad0ff9700 1 -- 172.21.15.119:0/337236447 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9461+0+0 (secure 0 0 0) 0x7faad809aa90 con 0x556f1973fa60 2023-12-07T10:33:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.024+0000 7fab02d9ab80 1 -- 172.21.15.119:0/337236447 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faae000fa60 con 0x556f1973fa60 2023-12-07T10:33:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.024+0000 7faad0ff9700 1 -- 172.21.15.119:0/337236447 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7faad809ac50 con 0x556f1973fa60 2023-12-07T10:33:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.024+0000 7fab02d9ab80 10 client.27887 fetch_fsmap learned FSMap version 66 2023-12-07T10:33:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.024+0000 7fab02d9ab80 1 -- 172.21.15.119:0/337236447 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x556f194de1f0 con 0x556f1973fa60 2023-12-07T10:33:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.024+0000 7faad0ff9700 1 -- 172.21.15.119:0/337236447 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faad8080030 con 0x556f1973fa60 2023-12-07T10:33:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.025+0000 7faad0ff9700 1 -- 172.21.15.119:0/337236447 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 66) v1 ==== 7930+0+0 (secure 0 0 0) 0x7faad8099930 con 0x556f1973fa60 2023-12-07T10:33:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.025+0000 7fab02d9ab80 10 client.27887 fetch_fsmap finished waiting for FSMap version 66 2023-12-07T10:33:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.025+0000 7fab02d9ab80 10 client.27887 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:33:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.025+0000 7fab02d9ab80 20 client.27887 populate_metadata read hostname 'smithi119' 2023-12-07T10:33:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.025+0000 7fab02d9ab80 1 --2- 172.21.15.119:0/337236447 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556f197a6b20 0x556f197a6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:33:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.025+0000 7fab02d9ab80 4 client.27887 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:33:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.025+0000 7fab02d9ab80 1 -- 172.21.15.119:0/337236447 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556f19654f70 con 0x556f197a6b20 2023-12-07T10:33:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.025+0000 7faaed19b700 1 --2- 172.21.15.119:0/337236447 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556f197a6b20 0x556f197a6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:33:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.026+0000 7faad0ff9700 10 client.27887 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:33:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.026+0000 7faaed19b700 1 --2- 172.21.15.119:0/337236447 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556f197a6b20 0x556f197a6f30 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=105f752d948f80c server_cookie=f280accdf1a5775 in_seq=0 out_seq=0 2023-12-07T10:33:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.124+0000 7faad0ff9700 1 -- 172.21.15.119:0/337236447 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556f19654f70 con 0x556f197a6b20 2023-12-07T10:33:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.124+0000 7faad0ff9700 10 client.27887 handle_command_reply: tid=0 2023-12-07T10:33:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.124+0000 7fab02d9ab80 1 client.27887 shutdown 2023-12-07T10:33:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.124+0000 7fab02d9ab80 1 -- 172.21.15.119:0/337236447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f1973fa60 msgr2=0x556f19741e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.125+0000 7fab02d9ab80 1 --2- 172.21.15.119:0/337236447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f1973fa60 0x556f19741e30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7faad809a7c0 tx=0x7faad8005ce0 comp rx=0 tx=0).stop 2023-12-07T10:33:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.125+0000 7fab02d9ab80 1 -- 172.21.15.119:0/337236447 shutdown_connections 2023-12-07T10:33:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.125+0000 7fab02d9ab80 1 --2- 172.21.15.119:0/337236447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f19740380 0x556f19740760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.125+0000 7fab02d9ab80 1 --2- 172.21.15.119:0/337236447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f1973aad0 0x556f1974b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.125+0000 7fab02d9ab80 1 --2- 172.21.15.119:0/337236447 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556f197a6b20 0x556f197a6f30 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.125+0000 7fab02d9ab80 1 --2- 172.21.15.119:0/337236447 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556f1973fa60 0x556f19741e30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.125+0000 7faad0ff9700 0 client.27887 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:33:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.125+0000 7fab02d9ab80 1 -- 172.21.15.119:0/337236447 >> 172.21.15.119:0/337236447 conn(0x556f19669300 msgr2=0x556f1974f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.125+0000 7fab02d9ab80 1 -- 172.21.15.119:0/337236447 shutdown_connections 2023-12-07T10:33:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.125+0000 7fab02d9ab80 1 -- 172.21.15.119:0/337236447 wait complete. 2023-12-07T10:33:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.125+0000 7fab02d9ab80 20 client.27887 trim_cache size 0 max 16384 2023-12-07T10:33:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:33:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:33:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:33:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:33:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.126+0000 7faaef3ff700 1 -- 172.21.15.119:0/2344607039 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7faad40a1d40 msgr2=0x7faad40a4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.126+0000 7faaef3ff700 1 --2- 172.21.15.119:0/2344607039 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7faad40a1d40 0x7faad40a4200 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faae400b6f0 tx=0x7faae4005c00 comp rx=0 tx=0).stop 2023-12-07T10:33:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.126+0000 7faaef3ff700 1 -- 172.21.15.119:0/2344607039 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faae81040c0 msgr2=0x7faae810d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:33:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.126+0000 7faaef3ff700 1 --2- 172.21.15.119:0/2344607039 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faae81040c0 0x7faae810d680 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7faad800ea20 tx=0x7faad800ed30 comp rx=0 tx=0).stop 2023-12-07T10:33:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.126+0000 7faaef3ff700 1 -- 172.21.15.119:0/2344607039 shutdown_connections 2023-12-07T10:33:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.126+0000 7faaef3ff700 1 --2- 172.21.15.119:0/2344607039 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7faad40a1d40 0x7faad40a4200 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.126+0000 7faaef3ff700 1 --2- 172.21.15.119:0/2344607039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faae810dcd0 0x7faae81aa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.126+0000 7faaef3ff700 1 --2- 172.21.15.119:0/2344607039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faae8103700 0x7faae8110540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.126+0000 7faaef3ff700 1 --2- 172.21.15.119:0/2344607039 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faae81040c0 0x7faae810d680 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:33:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.126+0000 7faaef3ff700 1 -- 172.21.15.119:0/2344607039 >> 172.21.15.119:0/2344607039 conn(0x7faae80ff190 msgr2=0x7faae80ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:33:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.126+0000 7faaef3ff700 1 -- 172.21.15.119:0/2344607039 shutdown_connections 2023-12-07T10:33:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:33:57.126+0000 7faaef3ff700 1 -- 172.21.15.119:0/2344607039 wait complete. 2023-12-07T10:33:57.961 INFO:tasks.workunit.client.0.smithi119.stdout: 10 5112 45.11 MB/sec warmup 10 sec latency 1905.581 ms 2023-12-07T10:33:58.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:58 smithi119 ceph-mon[97908]: pgmap v423: 129 pgs: 129 active+clean; 843 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 453 op/s 2023-12-07T10:33:58.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:58 smithi159 ceph-mon[134726]: pgmap v423: 129 pgs: 129 active+clean; 843 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 453 op/s 2023-12-07T10:33:58.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:58 smithi153 ceph-mon[133625]: pgmap v423: 129 pgs: 129 active+clean; 843 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 453 op/s 2023-12-07T10:33:58.961 INFO:tasks.workunit.client.0.smithi119.stdout: 10 5517 42.18 MB/sec warmup 11 sec latency 1424.706 ms 2023-12-07T10:33:59.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:33:59 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:33:59.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:33:59 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:33:59.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:33:59 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:33:59.961 INFO:tasks.workunit.client.0.smithi119.stdout: 10 6525 42.73 MB/sec warmup 12 sec latency 1310.844 ms 2023-12-07T10:34:00.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:00 smithi119 ceph-mon[97908]: pgmap v424: 129 pgs: 129 active+clean; 843 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 11 MiB/s wr, 351 op/s 2023-12-07T10:34:00.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:00 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:34:00.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:00 smithi159 ceph-mon[134726]: pgmap v424: 129 pgs: 129 active+clean; 843 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 11 MiB/s wr, 351 op/s 2023-12-07T10:34:00.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:00 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:34:00.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:00 smithi153 ceph-mon[133625]: pgmap v424: 129 pgs: 129 active+clean; 843 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 11 MiB/s wr, 351 op/s 2023-12-07T10:34:00.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:00 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:34:00.961 INFO:tasks.workunit.client.0.smithi119.stdout: 10 7216 42.55 MB/sec warmup 13 sec latency 1673.523 ms 2023-12-07T10:34:01.961 INFO:tasks.workunit.client.0.smithi119.stdout: 10 7224 39.67 MB/sec warmup 14 sec latency 1263.519 ms 2023-12-07T10:34:02.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:01 smithi159 ceph-mon[134726]: pgmap v425: 129 pgs: 129 active+clean; 961 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 21 MiB/s wr, 585 op/s 2023-12-07T10:34:02.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:01 smithi153 ceph-mon[133625]: pgmap v425: 129 pgs: 129 active+clean; 961 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 21 MiB/s wr, 585 op/s 2023-12-07T10:34:02.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:01 smithi119 ceph-mon[97908]: pgmap v425: 129 pgs: 129 active+clean; 961 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 21 MiB/s wr, 585 op/s 2023-12-07T10:34:02.961 INFO:tasks.workunit.client.0.smithi119.stdout: 10 7224 37.02 MB/sec warmup 15 sec latency 2263.650 ms 2023-12-07T10:34:03.961 INFO:tasks.workunit.client.0.smithi119.stdout: 10 7749 37.00 MB/sec warmup 16 sec latency 2311.616 ms 2023-12-07T10:34:04.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:04 smithi119 ceph-mon[97908]: pgmap v426: 129 pgs: 129 active+clean; 996 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 24 MiB/s wr, 587 op/s 2023-12-07T10:34:04.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:04 smithi159 ceph-mon[134726]: pgmap v426: 129 pgs: 129 active+clean; 996 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 24 MiB/s wr, 587 op/s 2023-12-07T10:34:04.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:04 smithi153 ceph-mon[133625]: pgmap v426: 129 pgs: 129 active+clean; 996 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 24 MiB/s wr, 587 op/s 2023-12-07T10:34:04.961 INFO:tasks.workunit.client.0.smithi119.stdout: 10 7802 35.01 MB/sec warmup 17 sec latency 1290.977 ms 2023-12-07T10:34:05.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:05 smithi119 ceph-mon[97908]: osdmap e109: 12 total, 12 up, 12 in 2023-12-07T10:34:05.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:05 smithi159 ceph-mon[134726]: osdmap e109: 12 total, 12 up, 12 in 2023-12-07T10:34:05.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:05 smithi153 ceph-mon[133625]: osdmap e109: 12 total, 12 up, 12 in 2023-12-07T10:34:05.961 INFO:tasks.workunit.client.0.smithi119.stdout: 10 7826 33.30 MB/sec warmup 18 sec latency 2291.034 ms 2023-12-07T10:34:06.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:06 smithi119 ceph-mon[97908]: pgmap v428: 129 pgs: 129 active+clean; 996 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 28 MiB/s wr, 644 op/s 2023-12-07T10:34:06.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:06 smithi159 ceph-mon[134726]: pgmap v428: 129 pgs: 129 active+clean; 996 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 28 MiB/s wr, 644 op/s 2023-12-07T10:34:06.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:06 smithi153 ceph-mon[133625]: pgmap v428: 129 pgs: 129 active+clean; 996 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 28 MiB/s wr, 644 op/s 2023-12-07T10:34:06.961 INFO:tasks.workunit.client.0.smithi119.stdout: 10 8194 32.93 MB/sec warmup 19 sec latency 3291.115 ms 2023-12-07T10:34:07.961 INFO:tasks.workunit.client.0.smithi119.stdout: 10 8940 32.79 MB/sec warmup 20 sec latency 4291.178 ms 2023-12-07T10:34:08.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:08 smithi119 ceph-mon[97908]: pgmap v429: 129 pgs: 129 active+clean; 1000 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 18 MiB/s wr, 598 op/s 2023-12-07T10:34:08.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:08 smithi159 ceph-mon[134726]: pgmap v429: 129 pgs: 129 active+clean; 1000 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 18 MiB/s wr, 598 op/s 2023-12-07T10:34:08.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:08 smithi153 ceph-mon[133625]: pgmap v429: 129 pgs: 129 active+clean; 1000 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 18 MiB/s wr, 598 op/s 2023-12-07T10:34:08.961 INFO:tasks.workunit.client.0.smithi119.stdout: 10 9661 33.21 MB/sec warmup 21 sec latency 5291.222 ms 2023-12-07T10:34:09.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:09 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:34:09.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:09 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:34:09.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:09 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:34:09.962 INFO:tasks.workunit.client.0.smithi119.stdout: 10 10347 33.29 MB/sec warmup 22 sec latency 6291.458 ms 2023-12-07T10:34:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.285+0000 7fa41559d700 1 -- 172.21.15.119:0/4129830532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa410101580 msgr2=0x7fa410101960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.285+0000 7fa41559d700 1 --2- 172.21.15.119:0/4129830532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa410101580 0x7fa410101960 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa400009990 tx=0x7fa400009ca0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.285+0000 7fa41559d700 1 -- 172.21.15.119:0/4129830532 shutdown_connections 2023-12-07T10:34:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.285+0000 7fa41559d700 1 --2- 172.21.15.119:0/4129830532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4101066a0 0x7fa41010e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.285+0000 7fa41559d700 1 --2- 172.21.15.119:0/4129830532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa410101580 0x7fa410101960 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.285+0000 7fa41559d700 1 --2- 172.21.15.119:0/4129830532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa410101f30 0x7fa410105f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.285+0000 7fa41559d700 1 -- 172.21.15.119:0/4129830532 >> 172.21.15.119:0/4129830532 conn(0x7fa4100fce20 msgr2=0x7fa4100ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:34:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.285+0000 7fa41559d700 1 -- 172.21.15.119:0/4129830532 shutdown_connections 2023-12-07T10:34:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.285+0000 7fa41559d700 1 -- 172.21.15.119:0/4129830532 wait complete. 2023-12-07T10:34:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.286+0000 7fa41559d700 1 Processor -- start 2023-12-07T10:34:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.286+0000 7fa41559d700 1 -- start start 2023-12-07T10:34:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.287+0000 7fa41559d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa410101580 0x7fa410073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.287+0000 7fa41559d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4101066a0 0x7fa410073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.287+0000 7fa41559d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa410074390 0x7fa4101a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.287+0000 7fa41559d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4101a60f0 con 0x7fa410074390 2023-12-07T10:34:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.287+0000 7fa41559d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4101a6230 con 0x7fa4101066a0 2023-12-07T10:34:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.287+0000 7fa41559d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4101a63f0 con 0x7fa410101580 2023-12-07T10:34:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.287+0000 7fa40f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4101066a0 0x7fa410073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.287+0000 7fa414d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa410074390 0x7fa4101a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.287+0000 7fa40ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa410101580 0x7fa410073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.287+0000 7fa40f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4101066a0 0x7fa410073e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:50346/0 (socket says 172.21.15.119:50346) 2023-12-07T10:34:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.287+0000 7fa40f7fe700 1 -- 172.21.15.119:0/877602800 learned_addr learned my addr 172.21.15.119:0/877602800 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:34:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.287+0000 7fa40f7fe700 1 -- 172.21.15.119:0/877602800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa410101580 msgr2=0x7fa410073910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.287+0000 7fa40f7fe700 1 --2- 172.21.15.119:0/877602800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa410101580 0x7fa410073910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.287+0000 7fa40f7fe700 1 -- 172.21.15.119:0/877602800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa410074390 msgr2=0x7fa4101a5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:10.292 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.287+0000 7fa40f7fe700 1 --2- 172.21.15.119:0/877602800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa410074390 0x7fa4101a5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.292 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.287+0000 7fa40f7fe700 1 -- 172.21.15.119:0/877602800 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4000095f0 con 0x7fa4101066a0 2023-12-07T10:34:10.292 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.288+0000 7fa40f7fe700 1 --2- 172.21.15.119:0/877602800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4101066a0 0x7fa410073e50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa40400e970 tx=0x7fa40400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:34:10.292 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.288+0000 7fa40d7fa700 1 -- 172.21.15.119:0/877602800 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa404009790 con 0x7fa4101066a0 2023-12-07T10:34:10.292 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.288+0000 7fa40d7fa700 1 -- 172.21.15.119:0/877602800 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa40400c7e0 con 0x7fa4101066a0 2023-12-07T10:34:10.292 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.288+0000 7fa41559d700 1 -- 172.21.15.119:0/877602800 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4101a66d0 con 0x7fa4101066a0 2023-12-07T10:34:10.292 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.288+0000 7fa40d7fa700 1 -- 172.21.15.119:0/877602800 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4040054e0 con 0x7fa4101066a0 2023-12-07T10:34:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.288+0000 7fa41559d700 1 -- 172.21.15.119:0/877602800 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4101a6be0 con 0x7fa4101066a0 2023-12-07T10:34:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.291+0000 7fa40d7fa700 1 -- 172.21.15.119:0/877602800 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fa40400c950 con 0x7fa4101066a0 2023-12-07T10:34:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.292+0000 7fa42a1ddb80 1 Processor -- start 2023-12-07T10:34:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.292+0000 7fa42a1ddb80 1 -- start start 2023-12-07T10:34:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.292+0000 7fa40d7fa700 1 --2- 172.21.15.119:0/877602800 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa3f80a1c80 0x7fa3f80a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.293+0000 7fa40d7fa700 1 -- 172.21.15.119:0/877602800 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9549+0+0 (secure 0 0 0) 0x7fa404014070 con 0x7fa4101066a0 2023-12-07T10:34:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.293+0000 7fa42a1ddb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f37315ca0 0x555f37316080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.293+0000 7fa414d9c700 1 --2- 172.21.15.119:0/877602800 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa3f80a1c80 0x7fa3f80a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.293+0000 7fa42a1ddb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f373165c0 0x555f37310540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.293+0000 7fa42a1ddb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f37310a80 0x555f37310f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.293+0000 7fa42a1ddb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f37284760 con 0x555f37310a80 2023-12-07T10:34:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.293+0000 7fa42a1ddb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f3726b740 con 0x555f37315ca0 2023-12-07T10:34:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.293+0000 7fa42a1ddb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f37269060 con 0x555f373165c0 2023-12-07T10:34:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.293+0000 7fa414d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f37310a80 0x555f37310f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.293+0000 7fa414d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f37310a80 0x555f37310f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58652/0 (socket says 172.21.15.119:58652) 2023-12-07T10:34:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.293+0000 7fa40f7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f373165c0 0x555f37310540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.293+0000 7fa414d9c700 1 -- 172.21.15.119:0/84132502 learned_addr learned my addr 172.21.15.119:0/84132502 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:34:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.293+0000 7fa40ffff700 1 --2- 172.21.15.119:0/84132502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f37315ca0 0x555f37316080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.293+0000 7fa40f7fe700 1 -- 172.21.15.119:0/84132502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f37315ca0 msgr2=0x555f37316080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.293+0000 7fa40f7fe700 1 --2- 172.21.15.119:0/84132502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f37315ca0 0x555f37316080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.293+0000 7fa40f7fe700 1 -- 172.21.15.119:0/84132502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f37310a80 msgr2=0x555f37310f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.293+0000 7fa40f7fe700 1 --2- 172.21.15.119:0/84132502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f37310a80 0x555f37310f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.293+0000 7fa414d9c700 1 --2- 172.21.15.119:0/877602800 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa3f80a1c80 0x7fa3f80a4140 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa40800f930 tx=0x7fa40800fcf0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:34:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.293+0000 7fa40f7fe700 1 -- 172.21.15.119:0/84132502 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f370e15d0 con 0x555f373165c0 2023-12-07T10:34:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.293+0000 7fa40ffff700 1 --2- 172.21.15.119:0/84132502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f37315ca0 0x555f37316080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:34:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.294+0000 7fa40f7fe700 1 --2- 172.21.15.119:0/84132502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f373165c0 0x555f37310540 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa4040cb320 tx=0x7fa40409a870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dc8582001032fa2 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:34:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.294+0000 7fa3f67fc700 1 -- 172.21.15.119:0/84132502 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4040b3210 con 0x555f373165c0 2023-12-07T10:34:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.294+0000 7fa3f67fc700 1 -- 172.21.15.119:0/84132502 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa40409a970 con 0x555f373165c0 2023-12-07T10:34:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.294+0000 7fa3f67fc700 1 -- 172.21.15.119:0/84132502 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4040cb760 con 0x555f373165c0 2023-12-07T10:34:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.295+0000 7fa42a1ddb80 1 -- 172.21.15.119:0/84132502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f373165c0 msgr2=0x555f37310540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.295+0000 7fa42a1ddb80 1 --2- 172.21.15.119:0/84132502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f373165c0 0x555f37310540 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa4040cb320 tx=0x7fa40409a870 comp rx=0 tx=0).stop 2023-12-07T10:34:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.295+0000 7fa42a1ddb80 1 -- 172.21.15.119:0/84132502 shutdown_connections 2023-12-07T10:34:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.295+0000 7fa42a1ddb80 1 --2- 172.21.15.119:0/84132502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f37310a80 0x555f37310f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.295+0000 7fa42a1ddb80 1 --2- 172.21.15.119:0/84132502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f37315ca0 0x555f37316080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.295+0000 7fa42a1ddb80 1 --2- 172.21.15.119:0/84132502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f373165c0 0x555f37310540 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.295+0000 7fa42a1ddb80 1 -- 172.21.15.119:0/84132502 >> 172.21.15.119:0/84132502 conn(0x555f3723f300 msgr2=0x555f3730abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:34:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.295+0000 7fa42a1ddb80 1 -- 172.21.15.119:0/84132502 shutdown_connections 2023-12-07T10:34:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.295+0000 7fa42a1ddb80 1 -- 172.21.15.119:0/84132502 wait complete. 2023-12-07T10:34:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.295+0000 7fa42a1ddb80 1 -- start start 2023-12-07T10:34:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.295+0000 7fa42a1ddb80 1 Processor -- start 2023-12-07T10:34:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.296+0000 7fa42a1ddb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f37310a80 0x555f37319950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.296+0000 7fa42a1ddb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f37315ca0 0x555f37319e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.296+0000 7fa42a1ddb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f373165c0 0x555f3731a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.296+0000 7fa42a1ddb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f37264950 con 0x555f373165c0 2023-12-07T10:34:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.296+0000 7fa42a1ddb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f3721e500 con 0x555f37315ca0 2023-12-07T10:34:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.296+0000 7fa42a1ddb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f3721e1a0 con 0x555f37310a80 2023-12-07T10:34:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.296+0000 7fa42a1ddb80 10 client.0 _ll_register_callbacks cb 0x555f372058e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:34:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.296+0000 7fa40ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f37310a80 0x555f37319950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.296+0000 7fa40ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f37310a80 0x555f37319950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41714/0 (socket says 172.21.15.119:41714) 2023-12-07T10:34:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.296+0000 7fa40ffff700 1 -- 172.21.15.119:0/3655682389 learned_addr learned my addr 172.21.15.119:0/3655682389 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:34:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.296+0000 7fa40f7fe700 1 --2- 172.21.15.119:0/3655682389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f37315ca0 0x555f37319e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.296+0000 7fa414d9c700 1 --2- 172.21.15.119:0/3655682389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f373165c0 0x555f3731a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.296+0000 7fa40ffff700 1 -- 172.21.15.119:0/3655682389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f37315ca0 msgr2=0x555f37319e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.296+0000 7fa40ffff700 1 --2- 172.21.15.119:0/3655682389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f37315ca0 0x555f37319e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.296+0000 7fa40ffff700 1 -- 172.21.15.119:0/3655682389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f373165c0 msgr2=0x555f3731a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.296+0000 7fa40ffff700 1 --2- 172.21.15.119:0/3655682389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f373165c0 0x555f3731a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.296+0000 7fa40ffff700 1 -- 172.21.15.119:0/3655682389 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa41010ea90 con 0x555f37310a80 2023-12-07T10:34:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.297+0000 7fa40ffff700 1 --2- 172.21.15.119:0/3655682389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f37310a80 0x555f37319950 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa400005800 tx=0x7fa400009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4b8a66c9a8c5704 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:34:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.297+0000 7fa3ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:34:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.297+0000 7fa3ef7fe700 1 -- 172.21.15.119:0/3655682389 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40001d070 con 0x555f37310a80 2023-12-07T10:34:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.297+0000 7fa42a1ddb80 1 -- 172.21.15.119:0/3655682389 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f373036f0 con 0x555f37310a80 2023-12-07T10:34:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.297+0000 7fa3ef7fe700 1 -- 172.21.15.119:0/3655682389 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9549+0+0 (secure 0 0 0) 0x7fa400004500 con 0x555f37310a80 2023-12-07T10:34:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.297+0000 7fa3ef7fe700 1 -- 172.21.15.119:0/3655682389 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa40000bee0 con 0x555f37310a80 2023-12-07T10:34:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.297+0000 7fa42a1ddb80 10 client.27874 fetch_fsmap learned FSMap version 69 2023-12-07T10:34:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.297+0000 7fa42a1ddb80 1 -- 172.21.15.119:0/3655682389 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x7fa4101a6be0 con 0x555f37310a80 2023-12-07T10:34:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.298+0000 7fa3ef7fe700 1 -- 172.21.15.119:0/3655682389 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555f373036f0 con 0x555f37310a80 2023-12-07T10:34:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.298+0000 7fa3ef7fe700 1 -- 172.21.15.119:0/3655682389 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 69) v1 ==== 7950+0+0 (secure 0 0 0) 0x7fa40000f760 con 0x555f37310a80 2023-12-07T10:34:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.298+0000 7fa42a1ddb80 10 client.27874 fetch_fsmap finished waiting for FSMap version 69 2023-12-07T10:34:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.298+0000 7fa42a1ddb80 10 client.27874 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:34:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.298+0000 7fa42a1ddb80 20 client.27874 populate_metadata read hostname 'smithi119' 2023-12-07T10:34:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.298+0000 7fa42a1ddb80 1 --2- 172.21.15.119:0/3655682389 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555f3732a210 0x555f3732c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.298+0000 7fa42a1ddb80 4 client.27874 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:34:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.298+0000 7fa42a1ddb80 1 -- 172.21.15.119:0/3655682389 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f372df740 con 0x555f3732a210 2023-12-07T10:34:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.298+0000 7fa40f7fe700 1 --2- 172.21.15.119:0/3655682389 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555f3732a210 0x555f3732c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.299+0000 7fa3ef7fe700 10 client.27874 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:34:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.299+0000 7fa40f7fe700 1 --2- 172.21.15.119:0/3655682389 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555f3732a210 0x555f3732c630 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f186d721d11ac8f server_cookie=41770ba4d75fd4cc in_seq=0 out_seq=0 2023-12-07T10:34:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.299+0000 7fa3ef7fe700 1 -- 172.21.15.119:0/3655682389 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555f372df740 con 0x555f3732a210 2023-12-07T10:34:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.299+0000 7fa3ef7fe700 10 client.27874 handle_command_reply: tid=0 2023-12-07T10:34:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.299+0000 7fa42a1ddb80 1 client.27874 shutdown 2023-12-07T10:34:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.300+0000 7fa42a1ddb80 1 -- 172.21.15.119:0/3655682389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f37310a80 msgr2=0x555f37319950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.300+0000 7fa42a1ddb80 1 --2- 172.21.15.119:0/3655682389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f37310a80 0x555f37319950 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa400005800 tx=0x7fa400009550 comp rx=0 tx=0).stop 2023-12-07T10:34:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.300+0000 7fa42a1ddb80 1 -- 172.21.15.119:0/3655682389 shutdown_connections 2023-12-07T10:34:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.300+0000 7fa42a1ddb80 1 --2- 172.21.15.119:0/3655682389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f373165c0 0x555f3731a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.300+0000 7fa42a1ddb80 1 --2- 172.21.15.119:0/3655682389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f37310a80 0x555f37319950 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.300+0000 7fa42a1ddb80 1 --2- 172.21.15.119:0/3655682389 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555f3732a210 0x555f3732c630 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.300+0000 7fa42a1ddb80 1 --2- 172.21.15.119:0/3655682389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f37315ca0 0x555f37319e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.300+0000 7fa3ef7fe700 0 client.27874 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:34:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.300+0000 7fa42a1ddb80 1 -- 172.21.15.119:0/3655682389 >> 172.21.15.119:0/3655682389 conn(0x555f3723f300 msgr2=0x555f37305980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:34:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.300+0000 7fa42a1ddb80 1 -- 172.21.15.119:0/3655682389 shutdown_connections 2023-12-07T10:34:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.300+0000 7fa42a1ddb80 1 -- 172.21.15.119:0/3655682389 wait complete. 2023-12-07T10:34:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.300+0000 7fa42a1ddb80 20 client.27874 trim_cache size 0 max 16384 2023-12-07T10:34:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.315+0000 7fa42a1ddb80 1 Processor -- start 2023-12-07T10:34:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.315+0000 7fa42a1ddb80 1 -- start start 2023-12-07T10:34:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.316+0000 7fa42a1ddb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f37310a80 0x555f3730c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.316+0000 7fa42a1ddb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f37315ca0 0x555f37321000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.316+0000 7fa42a1ddb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f373165c0 0x555f37321540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.316+0000 7fa42a1ddb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f370e38c0 con 0x555f373165c0 2023-12-07T10:34:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.316+0000 7fa42a1ddb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f3731afc0 con 0x555f37315ca0 2023-12-07T10:34:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.316+0000 7fa42a1ddb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f37303530 con 0x555f37310a80 2023-12-07T10:34:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.316+0000 7fa40ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f37310a80 0x555f3730c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.316+0000 7fa40ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f37310a80 0x555f3730c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:41730/0 (socket says 172.21.15.119:41730) 2023-12-07T10:34:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.316+0000 7fa40ffff700 1 -- 172.21.15.119:0/249618843 learned_addr learned my addr 172.21.15.119:0/249618843 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:34:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.316+0000 7fa40f7fe700 1 --2- 172.21.15.119:0/249618843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f37315ca0 0x555f37321000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.316+0000 7fa414d9c700 1 --2- 172.21.15.119:0/249618843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f373165c0 0x555f37321540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.316+0000 7fa40ffff700 1 -- 172.21.15.119:0/249618843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f37315ca0 msgr2=0x555f37321000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.316+0000 7fa40ffff700 1 --2- 172.21.15.119:0/249618843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f37315ca0 0x555f37321000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.316+0000 7fa40ffff700 1 -- 172.21.15.119:0/249618843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f373165c0 msgr2=0x555f37321540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.316+0000 7fa40ffff700 1 --2- 172.21.15.119:0/249618843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f373165c0 0x555f37321540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.316+0000 7fa40ffff700 1 -- 172.21.15.119:0/249618843 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f3708dd80 con 0x555f37310a80 2023-12-07T10:34:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.316+0000 7fa40f7fe700 1 --2- 172.21.15.119:0/249618843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f37315ca0 0x555f37321000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:34:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.317+0000 7fa40ffff700 1 --2- 172.21.15.119:0/249618843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f37310a80 0x555f3730c8c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa400005850 tx=0x7fa400009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a846726cb0fd7a9 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:34:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.317+0000 7fa3f6ffd700 1 -- 172.21.15.119:0/249618843 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40001d030 con 0x555f37310a80 2023-12-07T10:34:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.317+0000 7fa3f6ffd700 1 -- 172.21.15.119:0/249618843 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa40001e040 con 0x555f37310a80 2023-12-07T10:34:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.317+0000 7fa3f6ffd700 1 -- 172.21.15.119:0/249618843 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4000248b0 con 0x555f37310a80 2023-12-07T10:34:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.317+0000 7fa42a1ddb80 1 -- 172.21.15.119:0/249618843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f37310a80 msgr2=0x555f3730c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.317+0000 7fa42a1ddb80 1 --2- 172.21.15.119:0/249618843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f37310a80 0x555f3730c8c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa400005850 tx=0x7fa400009310 comp rx=0 tx=0).stop 2023-12-07T10:34:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.317+0000 7fa42a1ddb80 1 -- 172.21.15.119:0/249618843 shutdown_connections 2023-12-07T10:34:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.317+0000 7fa42a1ddb80 1 --2- 172.21.15.119:0/249618843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f373165c0 0x555f37321540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.318+0000 7fa42a1ddb80 1 --2- 172.21.15.119:0/249618843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f37310a80 0x555f3730c8c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.318+0000 7fa42a1ddb80 1 --2- 172.21.15.119:0/249618843 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f37315ca0 0x555f37321000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.318+0000 7fa42a1ddb80 1 -- 172.21.15.119:0/249618843 >> 172.21.15.119:0/249618843 conn(0x555f3723f300 msgr2=0x555f37330f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:34:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.318+0000 7fa42a1ddb80 1 -- 172.21.15.119:0/249618843 shutdown_connections 2023-12-07T10:34:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.318+0000 7fa42a1ddb80 1 -- 172.21.15.119:0/249618843 wait complete. 2023-12-07T10:34:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.318+0000 7fa42a1ddb80 1 -- start start 2023-12-07T10:34:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.318+0000 7fa42a1ddb80 1 Processor -- start 2023-12-07T10:34:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.319+0000 7fa42a1ddb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f37310a80 0x555f37378e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.319+0000 7fa42a1ddb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f37315ca0 0x555f37379350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.319+0000 7fa42a1ddb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f373165c0 0x555f373205d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.319+0000 7fa42a1ddb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f37320bb0 con 0x555f37315ca0 2023-12-07T10:34:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.319+0000 7fa42a1ddb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f37320d20 con 0x555f37310a80 2023-12-07T10:34:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.319+0000 7fa42a1ddb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f37320ee0 con 0x555f373165c0 2023-12-07T10:34:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.319+0000 7fa42a1ddb80 10 client.0 _ll_register_callbacks cb 0x555f372058e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:34:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.319+0000 7fa40f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f37315ca0 0x555f37379350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.319+0000 7fa40f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f37315ca0 0x555f37379350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58670/0 (socket says 172.21.15.119:58670) 2023-12-07T10:34:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.319+0000 7fa40f7fe700 1 -- 172.21.15.119:0/4149777373 learned_addr learned my addr 172.21.15.119:0/4149777373 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:34:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.319+0000 7fa414d9c700 1 --2- 172.21.15.119:0/4149777373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f373165c0 0x555f373205d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.319+0000 7fa40ffff700 1 --2- 172.21.15.119:0/4149777373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f37310a80 0x555f37378e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.319+0000 7fa40f7fe700 1 -- 172.21.15.119:0/4149777373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f373165c0 msgr2=0x555f373205d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.319+0000 7fa40f7fe700 1 --2- 172.21.15.119:0/4149777373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f373165c0 0x555f373205d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.319+0000 7fa40f7fe700 1 -- 172.21.15.119:0/4149777373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f37310a80 msgr2=0x555f37378e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.319+0000 7fa40f7fe700 1 --2- 172.21.15.119:0/4149777373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f37310a80 0x555f37378e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.319+0000 7fa40f7fe700 1 -- 172.21.15.119:0/4149777373 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f370e15d0 con 0x555f37315ca0 2023-12-07T10:34:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.319+0000 7fa40f7fe700 1 --2- 172.21.15.119:0/4149777373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f37315ca0 0x555f37379350 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa40409a670 tx=0x7fa404096cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346eee846c1f78e2 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:34:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.319+0000 7fa3ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:34:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.320+0000 7fa3ef7fe700 1 -- 172.21.15.119:0/4149777373 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40409eac0 con 0x555f37315ca0 2023-12-07T10:34:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.320+0000 7fa3ef7fe700 1 -- 172.21.15.119:0/4149777373 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9549+0+0 (secure 0 0 0) 0x7fa4040a2d00 con 0x555f37315ca0 2023-12-07T10:34:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.320+0000 7fa42a1ddb80 1 -- 172.21.15.119:0/4149777373 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa40409a970 con 0x555f37315ca0 2023-12-07T10:34:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.320+0000 7fa3ef7fe700 1 -- 172.21.15.119:0/4149777373 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa404097080 con 0x555f37315ca0 2023-12-07T10:34:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.320+0000 7fa42a1ddb80 10 client.17850 fetch_fsmap learned FSMap version 69 2023-12-07T10:34:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.320+0000 7fa42a1ddb80 1 -- 172.21.15.119:0/4149777373 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x7fa4101a6be0 con 0x555f37315ca0 2023-12-07T10:34:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.320+0000 7fa3ef7fe700 1 -- 172.21.15.119:0/4149777373 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa40409a970 con 0x555f37315ca0 2023-12-07T10:34:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.320+0000 7fa3ef7fe700 1 -- 172.21.15.119:0/4149777373 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 69) v1 ==== 7950+0+0 (secure 0 0 0) 0x7fa40409abd0 con 0x555f37315ca0 2023-12-07T10:34:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.320+0000 7fa42a1ddb80 10 client.17850 fetch_fsmap finished waiting for FSMap version 69 2023-12-07T10:34:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.320+0000 7fa42a1ddb80 10 client.17850 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:34:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.320+0000 7fa42a1ddb80 20 client.17850 populate_metadata read hostname 'smithi119' 2023-12-07T10:34:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.320+0000 7fa42a1ddb80 1 --2- 172.21.15.119:0/4149777373 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555f373809f0 0x555f37382e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.320+0000 7fa42a1ddb80 4 client.17850 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:34:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.320+0000 7fa42a1ddb80 1 -- 172.21.15.119:0/4149777373 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555f3722af70 con 0x555f373809f0 2023-12-07T10:34:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.321+0000 7fa40ffff700 1 --2- 172.21.15.119:0/4149777373 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555f373809f0 0x555f37382e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.321+0000 7fa3ef7fe700 10 client.17850 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:34:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.321+0000 7fa40ffff700 1 --2- 172.21.15.119:0/4149777373 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555f373809f0 0x555f37382e10 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bcf92aee210a3d3 server_cookie=2f3977780ebfa46d in_seq=0 out_seq=0 2023-12-07T10:34:10.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:10 smithi119 ceph-mon[97908]: pgmap v430: 129 pgs: 129 active+clean; 1000 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 18 MiB/s wr, 598 op/s 2023-12-07T10:34:10.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:10 smithi159 ceph-mon[134726]: pgmap v430: 129 pgs: 129 active+clean; 1000 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 18 MiB/s wr, 598 op/s 2023-12-07T10:34:10.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:10 smithi153 ceph-mon[133625]: pgmap v430: 129 pgs: 129 active+clean; 1000 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 18 MiB/s wr, 598 op/s 2023-12-07T10:34:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.720+0000 7fa3ef7fe700 1 -- 172.21.15.119:0/4149777373 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555f3722af70 con 0x555f373809f0 2023-12-07T10:34:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.720+0000 7fa3ef7fe700 10 client.17850 handle_command_reply: tid=0 2023-12-07T10:34:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.720+0000 7fa42a1ddb80 1 client.17850 shutdown 2023-12-07T10:34:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.721+0000 7fa42a1ddb80 1 -- 172.21.15.119:0/4149777373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f37315ca0 msgr2=0x555f37379350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.721+0000 7fa42a1ddb80 1 --2- 172.21.15.119:0/4149777373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f37315ca0 0x555f37379350 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa40409a670 tx=0x7fa404096cd0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.721+0000 7fa42a1ddb80 1 -- 172.21.15.119:0/4149777373 shutdown_connections 2023-12-07T10:34:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.721+0000 7fa42a1ddb80 1 --2- 172.21.15.119:0/4149777373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f373165c0 0x555f373205d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.721+0000 7fa42a1ddb80 1 --2- 172.21.15.119:0/4149777373 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x555f373809f0 0x555f37382e10 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.721+0000 7fa42a1ddb80 1 --2- 172.21.15.119:0/4149777373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555f37310a80 0x555f37378e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.721+0000 7fa42a1ddb80 1 --2- 172.21.15.119:0/4149777373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f37315ca0 0x555f37379350 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.721+0000 7fa42a1ddb80 1 -- 172.21.15.119:0/4149777373 >> 172.21.15.119:0/4149777373 conn(0x555f3723f300 msgr2=0x555f37313bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:34:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.721+0000 7fa3ef7fe700 0 client.17850 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:34:10.722 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.723+0000 7fa42a1ddb80 1 -- 172.21.15.119:0/4149777373 shutdown_connections 2023-12-07T10:34:10.723 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.723+0000 7fa42a1ddb80 1 -- 172.21.15.119:0/4149777373 wait complete. 2023-12-07T10:34:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:34:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:34:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:34:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:34:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.723+0000 7fa42a1ddb80 20 client.17850 trim_cache size 0 max 16384 2023-12-07T10:34:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.725+0000 7fa41559d700 1 -- 172.21.15.119:0/877602800 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa3f80a1c80 msgr2=0x7fa3f80a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.725+0000 7fa41559d700 1 --2- 172.21.15.119:0/877602800 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa3f80a1c80 0x7fa3f80a4140 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa40800f930 tx=0x7fa40800fcf0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.725+0000 7fa41559d700 1 -- 172.21.15.119:0/877602800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4101066a0 msgr2=0x7fa410073e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.725+0000 7fa41559d700 1 --2- 172.21.15.119:0/877602800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4101066a0 0x7fa410073e50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa40400e970 tx=0x7fa40400ec80 comp rx=0 tx=0).stop 2023-12-07T10:34:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.726+0000 7fa41559d700 1 -- 172.21.15.119:0/877602800 shutdown_connections 2023-12-07T10:34:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.726+0000 7fa41559d700 1 --2- 172.21.15.119:0/877602800 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa3f80a1c80 0x7fa3f80a4140 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.726+0000 7fa41559d700 1 --2- 172.21.15.119:0/877602800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa410074390 0x7fa4101a5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.726+0000 7fa41559d700 1 --2- 172.21.15.119:0/877602800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa410101580 0x7fa410073910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.726+0000 7fa41559d700 1 --2- 172.21.15.119:0/877602800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4101066a0 0x7fa410073e50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.726+0000 7fa41559d700 1 -- 172.21.15.119:0/877602800 >> 172.21.15.119:0/877602800 conn(0x7fa4100fce20 msgr2=0x7fa4100fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:34:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.726+0000 7fa41559d700 1 -- 172.21.15.119:0/877602800 shutdown_connections 2023-12-07T10:34:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:10.726+0000 7fa41559d700 1 -- 172.21.15.119:0/877602800 wait complete. 2023-12-07T10:34:10.962 INFO:tasks.workunit.client.0.smithi119.stdout: 10 10684 32.52 MB/sec warmup 23 sec latency 7291.785 ms 2023-12-07T10:34:11.962 INFO:tasks.workunit.client.0.smithi119.stdout: 10 11629 33.32 MB/sec warmup 24 sec latency 8291.843 ms 2023-12-07T10:34:12.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:11 smithi159 ceph-mon[134726]: osdmap e110: 12 total, 12 up, 12 in 2023-12-07T10:34:12.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:11 smithi159 ceph-mon[134726]: pgmap v432: 129 pgs: 129 active+clean; 1013 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 600 op/s 2023-12-07T10:34:12.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:11 smithi153 ceph-mon[133625]: osdmap e110: 12 total, 12 up, 12 in 2023-12-07T10:34:12.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:11 smithi153 ceph-mon[133625]: pgmap v432: 129 pgs: 129 active+clean; 1013 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 600 op/s 2023-12-07T10:34:12.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:11 smithi119 ceph-mon[97908]: osdmap e110: 12 total, 12 up, 12 in 2023-12-07T10:34:12.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:11 smithi119 ceph-mon[97908]: pgmap v432: 129 pgs: 129 active+clean; 1013 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 600 op/s 2023-12-07T10:34:12.962 INFO:tasks.workunit.client.0.smithi119.stdout: 10 12361 33.89 MB/sec warmup 25 sec latency 9291.894 ms 2023-12-07T10:34:13.965 INFO:tasks.workunit.client.0.smithi119.stdout: 10 12616 33.02 MB/sec warmup 26 sec latency 9536.475 ms 2023-12-07T10:34:14.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:14 smithi119 ceph-mon[97908]: pgmap v433: 129 pgs: 129 active+clean; 1020 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.6 MiB/s wr, 639 op/s 2023-12-07T10:34:14.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:14 smithi159 ceph-mon[134726]: pgmap v433: 129 pgs: 129 active+clean; 1020 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.6 MiB/s wr, 639 op/s 2023-12-07T10:34:14.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:14 smithi153 ceph-mon[133625]: pgmap v433: 129 pgs: 129 active+clean; 1020 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.6 MiB/s wr, 639 op/s 2023-12-07T10:34:14.962 INFO:tasks.workunit.client.0.smithi119.stdout: 10 12932 32.30 MB/sec warmup 27 sec latency 4251.178 ms 2023-12-07T10:34:15.962 INFO:tasks.workunit.client.0.smithi119.stdout: 10 13295 31.50 MB/sec warmup 28 sec latency 2504.685 ms 2023-12-07T10:34:16.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:16 smithi119 ceph-mon[97908]: pgmap v434: 129 pgs: 129 active+clean; 1020 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 571 op/s 2023-12-07T10:34:16.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:16 smithi159 ceph-mon[134726]: pgmap v434: 129 pgs: 129 active+clean; 1020 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 571 op/s 2023-12-07T10:34:16.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:16 smithi153 ceph-mon[133625]: pgmap v434: 129 pgs: 129 active+clean; 1020 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 571 op/s 2023-12-07T10:34:16.962 INFO:tasks.workunit.client.0.smithi119.stdout: 10 14333 32.37 MB/sec warmup 29 sec latency 3504.741 ms 2023-12-07T10:34:17.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:17 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:34:17.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:17 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:34:17.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:17 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:34:17.962 INFO:tasks.workunit.client.0.smithi119.stdout: 10 15401 33.70 MB/sec warmup 30 sec latency 4504.799 ms 2023-12-07T10:34:18.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:18 smithi119 ceph-mon[97908]: pgmap v435: 129 pgs: 129 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 12 MiB/s wr, 724 op/s 2023-12-07T10:34:18.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:18 smithi159 ceph-mon[134726]: pgmap v435: 129 pgs: 129 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 12 MiB/s wr, 724 op/s 2023-12-07T10:34:18.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:18 smithi153 ceph-mon[133625]: pgmap v435: 129 pgs: 129 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 12 MiB/s wr, 724 op/s 2023-12-07T10:34:18.963 INFO:tasks.workunit.client.0.smithi119.stdout: 10 16090 33.66 MB/sec warmup 31 sec latency 4719.367 ms 2023-12-07T10:34:19.963 INFO:tasks.workunit.client.0.smithi119.stdout: 10 16318 32.91 MB/sec warmup 32 sec latency 333.734 ms 2023-12-07T10:34:20.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:20 smithi119 ceph-mon[97908]: pgmap v436: 129 pgs: 129 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 12 MiB/s wr, 724 op/s 2023-12-07T10:34:20.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:20 smithi159 ceph-mon[134726]: pgmap v436: 129 pgs: 129 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 12 MiB/s wr, 724 op/s 2023-12-07T10:34:20.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:20 smithi153 ceph-mon[133625]: pgmap v436: 129 pgs: 129 active+clean; 1.0 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 12 MiB/s wr, 724 op/s 2023-12-07T10:34:20.962 INFO:tasks.workunit.client.0.smithi119.stdout: 10 16800 32.63 MB/sec warmup 33 sec latency 100.174 ms 2023-12-07T10:34:21.963 INFO:tasks.workunit.client.0.smithi119.stdout: 10 17902 33.46 MB/sec warmup 34 sec latency 54.744 ms 2023-12-07T10:34:22.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:22 smithi159 ceph-mon[134726]: pgmap v437: 129 pgs: 129 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 16 MiB/s wr, 934 op/s 2023-12-07T10:34:22.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:22 smithi153 ceph-mon[133625]: pgmap v437: 129 pgs: 129 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 16 MiB/s wr, 934 op/s 2023-12-07T10:34:22.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:22 smithi119 ceph-mon[97908]: pgmap v437: 129 pgs: 129 active+clean; 1.0 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 16 MiB/s wr, 934 op/s 2023-12-07T10:34:22.963 INFO:tasks.workunit.client.0.smithi119.stdout: 10 18809 34.10 MB/sec warmup 35 sec latency 53.082 ms 2023-12-07T10:34:23.964 INFO:tasks.workunit.client.0.smithi119.stdout: 10 19633 34.12 MB/sec warmup 36 sec latency 63.211 ms 2023-12-07T10:34:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:23.998+0000 7f2d707ff700 1 -- 172.21.15.119:0/3925321165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d680739b0 msgr2=0x7f2d68073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:23.998+0000 7f2d707ff700 1 --2- 172.21.15.119:0/3925321165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d680739b0 0x7f2d68073d90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c00b0a0 tx=0x7f2d5c00b3b0 comp rx=0 tx=0).stop 2023-12-07T10:34:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:23.998+0000 7f2d707ff700 1 -- 172.21.15.119:0/3925321165 shutdown_connections 2023-12-07T10:34:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:23.998+0000 7f2d707ff700 1 --2- 172.21.15.119:0/3925321165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d681124f0 0x7f2d681148e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:23.998+0000 7f2d707ff700 1 --2- 172.21.15.119:0/3925321165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d680739b0 0x7f2d68073d90 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:23.998+0000 7f2d707ff700 1 --2- 172.21.15.119:0/3925321165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d680742d0 0x7f2d68111fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:23.998+0000 7f2d707ff700 1 -- 172.21.15.119:0/3925321165 >> 172.21.15.119:0/3925321165 conn(0x7f2d680fcfb0 msgr2=0x7f2d680ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:34:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:23.999+0000 7f2d707ff700 1 -- 172.21.15.119:0/3925321165 shutdown_connections 2023-12-07T10:34:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:23.999+0000 7f2d707ff700 1 -- 172.21.15.119:0/3925321165 wait complete. 2023-12-07T10:34:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.001+0000 7f2d707ff700 1 Processor -- start 2023-12-07T10:34:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.001+0000 7f2d707ff700 1 -- start start 2023-12-07T10:34:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.002+0000 7f2d707ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d680739b0 0x7f2d6819e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.002+0000 7f2d707ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d680742d0 0x7f2d6819e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.003+0000 7f2d707ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d681124f0 0x7f2d681a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.003+0000 7f2d6ed9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d681124f0 0x7f2d681a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.004+0000 7f2d6e59b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d680739b0 0x7f2d6819e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.004+0000 7f2d6dd9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d680742d0 0x7f2d6819e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.004+0000 7f2d6dd9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d680742d0 0x7f2d6819e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:47716/0 (socket says 172.21.15.119:47716) 2023-12-07T10:34:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.004+0000 7f2d6dd9a700 1 -- 172.21.15.119:0/3288966381 learned_addr learned my addr 172.21.15.119:0/3288966381 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:34:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.004+0000 7f2d707ff700 1 -- 172.21.15.119:0/3288966381 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d681a3020 con 0x7f2d681124f0 2023-12-07T10:34:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.004+0000 7f2d707ff700 1 -- 172.21.15.119:0/3288966381 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d681a3190 con 0x7f2d680742d0 2023-12-07T10:34:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.004+0000 7f2d707ff700 1 -- 172.21.15.119:0/3288966381 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d681a3350 con 0x7f2d680739b0 2023-12-07T10:34:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.005+0000 7f2d6ed9c700 1 -- 172.21.15.119:0/3288966381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d680739b0 msgr2=0x7f2d6819e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.005+0000 7f2d6ed9c700 1 --2- 172.21.15.119:0/3288966381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d680739b0 0x7f2d6819e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.005+0000 7f2d6ed9c700 1 -- 172.21.15.119:0/3288966381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d680742d0 msgr2=0x7f2d6819e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.005+0000 7f2d6ed9c700 1 --2- 172.21.15.119:0/3288966381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d680742d0 0x7f2d6819e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.005+0000 7f2d6ed9c700 1 -- 172.21.15.119:0/3288966381 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d5c009d00 con 0x7f2d681124f0 2023-12-07T10:34:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.006+0000 7f2d6ed9c700 1 --2- 172.21.15.119:0/3288966381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d681124f0 0x7f2d681a29e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2d6806abd0 tx=0x7f2d6000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:34:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.006+0000 7f2d677fe700 1 -- 172.21.15.119:0/3288966381 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60004220 con 0x7f2d681124f0 2023-12-07T10:34:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.006+0000 7f2d677fe700 1 -- 172.21.15.119:0/3288966381 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d60004cf0 con 0x7f2d681124f0 2023-12-07T10:34:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.006+0000 7f2d677fe700 1 -- 172.21.15.119:0/3288966381 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d600056a0 con 0x7f2d681124f0 2023-12-07T10:34:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.006+0000 7f2d707ff700 1 -- 172.21.15.119:0/3288966381 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d681a3630 con 0x7f2d681124f0 2023-12-07T10:34:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.006+0000 7f2d707ff700 1 -- 172.21.15.119:0/3288966381 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d681aae70 con 0x7f2d681124f0 2023-12-07T10:34:24.015 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.015+0000 7f2d677fe700 1 -- 172.21.15.119:0/3288966381 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f2d60004380 con 0x7f2d681124f0 2023-12-07T10:34:24.015 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.015+0000 7f2d840d7b80 1 Processor -- start 2023-12-07T10:34:24.015 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.016+0000 7f2d840d7b80 1 -- start start 2023-12-07T10:34:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.018+0000 7f2d840d7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdf4d8a60 0x562cdf4d8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.018+0000 7f2d840d7b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cdf4d9380 0x562cdf4d3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.018+0000 7f2d840d7b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cdf4d3ad0 0x562cdf4d3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.018+0000 7f2d840d7b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdf447760 con 0x562cdf4d9380 2023-12-07T10:34:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.018+0000 7f2d840d7b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdf42e740 con 0x562cdf4d3ad0 2023-12-07T10:34:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.018+0000 7f2d840d7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdf42c060 con 0x562cdf4d8a60 2023-12-07T10:34:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.018+0000 7f2d6dd9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdf4d8a60 0x562cdf4d8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.018+0000 7f2d6dd9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdf4d8a60 0x562cdf4d8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:40014/0 (socket says 172.21.15.119:40014) 2023-12-07T10:34:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.018+0000 7f2d6dd9a700 1 -- 172.21.15.119:0/3294333143 learned_addr learned my addr 172.21.15.119:0/3294333143 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:34:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.018+0000 7f2d6dd9a700 1 --2- 172.21.15.119:0/3294333143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cdf4d3ad0 0x562cdf4d3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.020+0000 7f2d677fe700 1 --2- 172.21.15.119:0/3288966381 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f2d580a1d80 0x7f2d580a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.020+0000 7f2d677fe700 1 -- 172.21.15.119:0/3288966381 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f2d600cd850 con 0x7f2d681124f0 2023-12-07T10:34:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.020+0000 7f2d6e59b700 1 --2- 172.21.15.119:0/3294333143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cdf4d9380 0x562cdf4d3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.021+0000 7f2d6ed9c700 1 --2- 172.21.15.119:0/3288966381 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f2d580a1d80 0x7f2d580a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.021+0000 7f2d6e59b700 1 -- 172.21.15.119:0/3294333143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdf4d8a60 msgr2=0x562cdf4d8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.021+0000 7f2d6e59b700 1 --2- 172.21.15.119:0/3294333143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdf4d8a60 0x562cdf4d8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.021+0000 7f2d6e59b700 1 -- 172.21.15.119:0/3294333143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cdf4d3ad0 msgr2=0x562cdf4d3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.021+0000 7f2d6e59b700 1 --2- 172.21.15.119:0/3294333143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cdf4d3ad0 0x562cdf4d3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.021+0000 7f2d6e59b700 1 -- 172.21.15.119:0/3294333143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cdf2a45d0 con 0x562cdf4d9380 2023-12-07T10:34:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.022+0000 7f2d6e59b700 1 --2- 172.21.15.119:0/3294333143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cdf4d9380 0x562cdf4d3540 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c009fd0 tx=0x7f2d5c0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc455745e7dac16 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:34:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.024+0000 7f2d64ff9700 1 -- 172.21.15.119:0/3294333143 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c00e050 con 0x562cdf4d9380 2023-12-07T10:34:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.024+0000 7f2d64ff9700 1 -- 172.21.15.119:0/3294333143 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d5c0094d0 con 0x562cdf4d9380 2023-12-07T10:34:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.024+0000 7f2d64ff9700 1 -- 172.21.15.119:0/3294333143 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c007740 con 0x562cdf4d9380 2023-12-07T10:34:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.026+0000 7f2d840d7b80 1 -- 172.21.15.119:0/3294333143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cdf4d9380 msgr2=0x562cdf4d3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.026+0000 7f2d6ed9c700 1 --2- 172.21.15.119:0/3288966381 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f2d580a1d80 0x7f2d580a4240 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2d600a2040 tx=0x7f2d60005020 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:34:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.026+0000 7f2d840d7b80 1 --2- 172.21.15.119:0/3294333143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cdf4d9380 0x562cdf4d3540 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c009fd0 tx=0x7f2d5c0092e0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.027+0000 7f2d840d7b80 1 -- 172.21.15.119:0/3294333143 shutdown_connections 2023-12-07T10:34:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.027+0000 7f2d840d7b80 1 --2- 172.21.15.119:0/3294333143 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cdf4d3ad0 0x562cdf4d3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.027+0000 7f2d840d7b80 1 --2- 172.21.15.119:0/3294333143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdf4d8a60 0x562cdf4d8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.027+0000 7f2d840d7b80 1 --2- 172.21.15.119:0/3294333143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cdf4d9380 0x562cdf4d3540 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.029 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.027+0000 7f2d840d7b80 1 -- 172.21.15.119:0/3294333143 >> 172.21.15.119:0/3294333143 conn(0x562cdf402300 msgr2=0x562cdf4cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:34:24.029 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.028+0000 7f2d840d7b80 1 -- 172.21.15.119:0/3294333143 shutdown_connections 2023-12-07T10:34:24.030 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.028+0000 7f2d840d7b80 1 -- 172.21.15.119:0/3294333143 wait complete. 2023-12-07T10:34:24.030 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.029+0000 7f2d840d7b80 1 -- start start 2023-12-07T10:34:24.030 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.029+0000 7f2d840d7b80 1 Processor -- start 2023-12-07T10:34:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.031+0000 7f2d840d7b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cdf4d3ad0 0x562cdf4dc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.031+0000 7f2d840d7b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cdf4d8a60 0x562cdf4dcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.031+0000 7f2d840d7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdf4d9380 0x562cdf4dd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.031+0000 7f2d840d7b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdf427950 con 0x562cdf4d8a60 2023-12-07T10:34:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.031+0000 7f2d6dd9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cdf4d3ad0 0x562cdf4dc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.031+0000 7f2d6dd9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cdf4d3ad0 0x562cdf4dc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:47730/0 (socket says 172.21.15.119:47730) 2023-12-07T10:34:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.031+0000 7f2d6dd9a700 1 -- 172.21.15.119:0/1433392734 learned_addr learned my addr 172.21.15.119:0/1433392734 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:34:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.032+0000 7f2d840d7b80 1 -- 172.21.15.119:0/1433392734 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdf3e1500 con 0x562cdf4d3ad0 2023-12-07T10:34:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.032+0000 7f2d840d7b80 1 -- 172.21.15.119:0/1433392734 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdf3e11a0 con 0x562cdf4d9380 2023-12-07T10:34:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.032+0000 7f2d840d7b80 10 client.0 _ll_register_callbacks cb 0x562cdf3c88e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:34:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.032+0000 7f2d6dd9a700 1 --2- 172.21.15.119:0/1433392734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdf4d9380 0x562cdf4dd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.032+0000 7f2d6dd9a700 1 -- 172.21.15.119:0/1433392734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdf4d9380 msgr2=0x562cdf4dd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.032+0000 7f2d6dd9a700 1 --2- 172.21.15.119:0/1433392734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdf4d9380 0x562cdf4dd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.032+0000 7f2d6dd9a700 1 -- 172.21.15.119:0/1433392734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cdf4d8a60 msgr2=0x562cdf4dcd30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-07T10:34:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.033+0000 7f2d6e59b700 1 --2- 172.21.15.119:0/1433392734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cdf4d8a60 0x562cdf4dcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.033+0000 7f2d6dd9a700 1 --2- 172.21.15.119:0/1433392734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cdf4d8a60 0x562cdf4dcd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.033+0000 7f2d6dd9a700 1 -- 172.21.15.119:0/1433392734 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d68115290 con 0x562cdf4d3ad0 2023-12-07T10:34:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.033+0000 7f2d6dd9a700 1 --2- 172.21.15.119:0/1433392734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cdf4d3ad0 0x562cdf4dc7f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2d54009680 tx=0x7f2d54004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2b0749952aeb34b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:34:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.033+0000 7f2d51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:34:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.034+0000 7f2d51ffb700 1 -- 172.21.15.119:0/1433392734 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5400dcf0 con 0x562cdf4d3ad0 2023-12-07T10:34:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.034+0000 7f2d51ffb700 1 -- 172.21.15.119:0/1433392734 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f2d5400de50 con 0x562cdf4d3ad0 2023-12-07T10:34:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.034+0000 7f2d840d7b80 1 -- 172.21.15.119:0/1433392734 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562cdf4c66f0 con 0x562cdf4d3ad0 2023-12-07T10:34:24.036 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.036+0000 7f2d51ffb700 1 -- 172.21.15.119:0/1433392734 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d54004050 con 0x562cdf4d3ad0 2023-12-07T10:34:24.036 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.036+0000 7f2d840d7b80 10 client.27917 fetch_fsmap learned FSMap version 70 2023-12-07T10:34:24.036 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.037+0000 7f2d840d7b80 1 -- 172.21.15.119:0/1433392734 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x562cdf250d80 con 0x562cdf4d3ad0 2023-12-07T10:34:24.037 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.037+0000 7f2d51ffb700 1 -- 172.21.15.119:0/1433392734 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562cdf4c66f0 con 0x562cdf4d3ad0 2023-12-07T10:34:24.037 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.037+0000 7f2d51ffb700 1 -- 172.21.15.119:0/1433392734 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 70) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f2d54021420 con 0x562cdf4d3ad0 2023-12-07T10:34:24.038 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.038+0000 7f2d840d7b80 10 client.27917 fetch_fsmap finished waiting for FSMap version 70 2023-12-07T10:34:24.038 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.038+0000 7f2d840d7b80 10 client.27917 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:34:24.038 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.038+0000 7f2d840d7b80 20 client.27917 populate_metadata read hostname 'smithi119' 2023-12-07T10:34:24.038 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.038+0000 7f2d840d7b80 1 --2- 172.21.15.119:0/1433392734 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562cdf4ecef0 0x562cdf4ef310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:24.038 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.038+0000 7f2d840d7b80 4 client.27917 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:34:24.038 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.038+0000 7f2d840d7b80 1 -- 172.21.15.119:0/1433392734 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562cdf4a2740 con 0x562cdf4ecef0 2023-12-07T10:34:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.039+0000 7f2d6e59b700 1 --2- 172.21.15.119:0/1433392734 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562cdf4ecef0 0x562cdf4ef310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.039+0000 7f2d6e59b700 1 --2- 172.21.15.119:0/1433392734 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562cdf4ecef0 0x562cdf4ef310 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=653ce23e6b6111a8 server_cookie=cf296f1395d47b1f in_seq=0 out_seq=0 2023-12-07T10:34:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.039+0000 7f2d51ffb700 10 client.27917 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:34:24.040 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.040+0000 7f2d51ffb700 1 -- 172.21.15.119:0/1433392734 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562cdf4a2740 con 0x562cdf4ecef0 2023-12-07T10:34:24.040 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.040+0000 7f2d51ffb700 10 client.27917 handle_command_reply: tid=0 2023-12-07T10:34:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.041+0000 7f2d840d7b80 1 client.27917 shutdown 2023-12-07T10:34:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.041+0000 7f2d840d7b80 1 -- 172.21.15.119:0/1433392734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cdf4d3ad0 msgr2=0x562cdf4dc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.041+0000 7f2d840d7b80 1 --2- 172.21.15.119:0/1433392734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cdf4d3ad0 0x562cdf4dc7f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2d54009680 tx=0x7f2d54004750 comp rx=0 tx=0).stop 2023-12-07T10:34:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.042+0000 7f2d840d7b80 1 -- 172.21.15.119:0/1433392734 shutdown_connections 2023-12-07T10:34:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.042+0000 7f2d840d7b80 1 --2- 172.21.15.119:0/1433392734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdf4d9380 0x562cdf4dd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.042+0000 7f2d840d7b80 1 --2- 172.21.15.119:0/1433392734 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562cdf4ecef0 0x562cdf4ef310 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.042+0000 7f2d840d7b80 1 --2- 172.21.15.119:0/1433392734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cdf4d3ad0 0x562cdf4dc7f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.042+0000 7f2d840d7b80 1 --2- 172.21.15.119:0/1433392734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cdf4d8a60 0x562cdf4dcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.042+0000 7f2d840d7b80 1 -- 172.21.15.119:0/1433392734 >> 172.21.15.119:0/1433392734 conn(0x562cdf402300 msgr2=0x562cdf4c8980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:34:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.042+0000 7f2d51ffb700 0 client.27917 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:34:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.042+0000 7f2d840d7b80 1 -- 172.21.15.119:0/1433392734 shutdown_connections 2023-12-07T10:34:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.042+0000 7f2d840d7b80 1 -- 172.21.15.119:0/1433392734 wait complete. 2023-12-07T10:34:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.043+0000 7f2d840d7b80 20 client.27917 trim_cache size 0 max 16384 2023-12-07T10:34:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.067+0000 7f2d840d7b80 1 Processor -- start 2023-12-07T10:34:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.067+0000 7f2d840d7b80 1 -- start start 2023-12-07T10:34:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.068+0000 7f2d840d7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdf4d3ad0 0x562cdf4d3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.068+0000 7f2d840d7b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cdf4d8a60 0x562cdf4db0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.068+0000 7f2d840d7b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cdf4d9380 0x562cdf4db5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.068+0000 7f2d840d7b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdf2a68c0 con 0x562cdf4d8a60 2023-12-07T10:34:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.068+0000 7f2d840d7b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdf4cf480 con 0x562cdf4d9380 2023-12-07T10:34:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.068+0000 7f2d840d7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdf4dbb20 con 0x562cdf4d3ad0 2023-12-07T10:34:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.068+0000 7f2d6dd9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdf4d3ad0 0x562cdf4d3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.068+0000 7f2d6dd9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cdf4d9380 0x562cdf4db5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.069+0000 7f2d6dd9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdf4d3ad0 0x562cdf4d3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:40028/0 (socket says 172.21.15.119:40028) 2023-12-07T10:34:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.069+0000 7f2d6dd9a700 1 -- 172.21.15.119:0/1778251641 learned_addr learned my addr 172.21.15.119:0/1778251641 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:34:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.069+0000 7f2d6e59b700 1 --2- 172.21.15.119:0/1778251641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cdf4d8a60 0x562cdf4db0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.069+0000 7f2d6dd9a700 1 -- 172.21.15.119:0/1778251641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cdf4d9380 msgr2=0x562cdf4db5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.069+0000 7f2d6dd9a700 1 --2- 172.21.15.119:0/1778251641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cdf4d9380 0x562cdf4db5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.069+0000 7f2d6dd9a700 1 -- 172.21.15.119:0/1778251641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cdf4d8a60 msgr2=0x562cdf4db0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.069+0000 7f2d6dd9a700 1 --2- 172.21.15.119:0/1778251641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cdf4d8a60 0x562cdf4db0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.069+0000 7f2d6dd9a700 1 -- 172.21.15.119:0/1778251641 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cdf239c00 con 0x562cdf4d3ad0 2023-12-07T10:34:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.069+0000 7f2d6dd9a700 1 --2- 172.21.15.119:0/1778251641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdf4d3ad0 0x562cdf4d3540 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2d54021800 tx=0x7f2d54000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9721fd7c148c2069 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:34:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.070+0000 7f2d657fa700 1 -- 172.21.15.119:0/1778251641 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d54014110 con 0x562cdf4d3ad0 2023-12-07T10:34:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.070+0000 7f2d657fa700 1 -- 172.21.15.119:0/1778251641 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d54014270 con 0x562cdf4d3ad0 2023-12-07T10:34:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.070+0000 7f2d657fa700 1 -- 172.21.15.119:0/1778251641 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d540047c0 con 0x562cdf4d3ad0 2023-12-07T10:34:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.071+0000 7f2d840d7b80 1 -- 172.21.15.119:0/1778251641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdf4d3ad0 msgr2=0x562cdf4d3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.071+0000 7f2d840d7b80 1 --2- 172.21.15.119:0/1778251641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdf4d3ad0 0x562cdf4d3540 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2d54021800 tx=0x7f2d54000bc0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.071+0000 7f2d840d7b80 1 -- 172.21.15.119:0/1778251641 shutdown_connections 2023-12-07T10:34:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.071+0000 7f2d840d7b80 1 --2- 172.21.15.119:0/1778251641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cdf4d9380 0x562cdf4db5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.071+0000 7f2d840d7b80 1 --2- 172.21.15.119:0/1778251641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdf4d3ad0 0x562cdf4d3540 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.072+0000 7f2d840d7b80 1 --2- 172.21.15.119:0/1778251641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cdf4d8a60 0x562cdf4db0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.072+0000 7f2d840d7b80 1 -- 172.21.15.119:0/1778251641 >> 172.21.15.119:0/1778251641 conn(0x562cdf402300 msgr2=0x562cdf4d6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:34:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.072+0000 7f2d840d7b80 1 -- 172.21.15.119:0/1778251641 shutdown_connections 2023-12-07T10:34:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.072+0000 7f2d840d7b80 1 -- 172.21.15.119:0/1778251641 wait complete. 2023-12-07T10:34:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.072+0000 7f2d840d7b80 1 -- start start 2023-12-07T10:34:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.073+0000 7f2d840d7b80 1 Processor -- start 2023-12-07T10:34:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.074+0000 7f2d840d7b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdf4d3ad0 0x562cdf4dfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.074+0000 7f2d840d7b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cdf4d8a60 0x562cdf4e02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.074+0000 7f2d840d7b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cdf4d9380 0x562cdf4e0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.074+0000 7f2d840d7b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdf542400 con 0x562cdf4d9380 2023-12-07T10:34:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.074+0000 7f2d840d7b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdf542540 con 0x562cdf4d8a60 2023-12-07T10:34:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.074+0000 7f2d840d7b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdf542680 con 0x562cdf4d3ad0 2023-12-07T10:34:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.074+0000 7f2d840d7b80 10 client.0 _ll_register_callbacks cb 0x562cdf3c88e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:34:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.075+0000 7f2d6dd9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdf4d3ad0 0x562cdf4dfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.076+0000 7f2d6dd9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdf4d3ad0 0x562cdf4dfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:40036/0 (socket says 172.21.15.119:40036) 2023-12-07T10:34:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.076+0000 7f2d6dd9a700 1 -- 172.21.15.119:0/1353861555 learned_addr learned my addr 172.21.15.119:0/1353861555 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:34:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.076+0000 7f2d6dd9a700 1 --2- 172.21.15.119:0/1353861555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cdf4d9380 0x562cdf4e0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.077+0000 7f2d6e59b700 1 --2- 172.21.15.119:0/1353861555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cdf4d8a60 0x562cdf4e02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.077+0000 7f2d6dd9a700 1 -- 172.21.15.119:0/1353861555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cdf4d8a60 msgr2=0x562cdf4e02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.077+0000 7f2d6dd9a700 1 --2- 172.21.15.119:0/1353861555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cdf4d8a60 0x562cdf4e02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.077+0000 7f2d6dd9a700 1 -- 172.21.15.119:0/1353861555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cdf4d9380 msgr2=0x562cdf4e0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.077+0000 7f2d6dd9a700 1 --2- 172.21.15.119:0/1353861555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cdf4d9380 0x562cdf4e0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.077+0000 7f2d6dd9a700 1 -- 172.21.15.119:0/1353861555 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d54033040 con 0x562cdf4d3ad0 2023-12-07T10:34:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.078+0000 7f2d6dd9a700 1 --2- 172.21.15.119:0/1353861555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdf4d3ad0 0x562cdf4dfda0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2d54005a80 tx=0x7f2d540056b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa51233bc07e5d97 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:34:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.078+0000 7f2d51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:34:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.079+0000 7f2d51ffb700 1 -- 172.21.15.119:0/1353861555 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5400dcf0 con 0x562cdf4d3ad0 2023-12-07T10:34:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.079+0000 7f2d840d7b80 1 -- 172.21.15.119:0/1353861555 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d5c0094d0 con 0x562cdf4d3ad0 2023-12-07T10:34:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.079+0000 7f2d51ffb700 1 -- 172.21.15.119:0/1353861555 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f2d54000e40 con 0x562cdf4d3ad0 2023-12-07T10:34:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.080+0000 7f2d51ffb700 1 -- 172.21.15.119:0/1353861555 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d540050d0 con 0x562cdf4d3ad0 2023-12-07T10:34:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.080+0000 7f2d840d7b80 10 client.27898 fetch_fsmap learned FSMap version 70 2023-12-07T10:34:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.080+0000 7f2d840d7b80 1 -- 172.21.15.119:0/1353861555 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x562cdf2771f0 con 0x562cdf4d3ad0 2023-12-07T10:34:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.081+0000 7f2d51ffb700 1 -- 172.21.15.119:0/1353861555 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d5c0094d0 con 0x562cdf4d3ad0 2023-12-07T10:34:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.081+0000 7f2d51ffb700 1 -- 172.21.15.119:0/1353861555 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 70) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f2d5400f410 con 0x562cdf4d3ad0 2023-12-07T10:34:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.081+0000 7f2d840d7b80 10 client.27898 fetch_fsmap finished waiting for FSMap version 70 2023-12-07T10:34:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.081+0000 7f2d840d7b80 10 client.27898 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:34:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.081+0000 7f2d840d7b80 20 client.27898 populate_metadata read hostname 'smithi119' 2023-12-07T10:34:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.081+0000 7f2d840d7b80 1 --2- 172.21.15.119:0/1353861555 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562cdf554bc0 0x562cdf556fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.081+0000 7f2d840d7b80 4 client.27898 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:34:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.081+0000 7f2d840d7b80 1 -- 172.21.15.119:0/1353861555 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562cdf3edf70 con 0x562cdf554bc0 2023-12-07T10:34:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.082+0000 7f2d6e59b700 1 --2- 172.21.15.119:0/1353861555 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562cdf554bc0 0x562cdf556fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.083+0000 7f2d6e59b700 1 --2- 172.21.15.119:0/1353861555 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562cdf554bc0 0x562cdf556fe0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d53bd44f40f378b2 server_cookie=e7cf7a0cad716d07 in_seq=0 out_seq=0 2023-12-07T10:34:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.083+0000 7f2d51ffb700 10 client.27898 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:34:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.476+0000 7f2d51ffb700 1 -- 172.21.15.119:0/1353861555 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562cdf3edf70 con 0x562cdf554bc0 2023-12-07T10:34:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.476+0000 7f2d51ffb700 10 client.27898 handle_command_reply: tid=0 2023-12-07T10:34:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.476+0000 7f2d840d7b80 1 client.27898 shutdown 2023-12-07T10:34:24.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:24 smithi119 ceph-mon[97908]: pgmap v438: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 14 MiB/s wr, 756 op/s 2023-12-07T10:34:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.477+0000 7f2d840d7b80 1 -- 172.21.15.119:0/1353861555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdf4d3ad0 msgr2=0x562cdf4dfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.477+0000 7f2d840d7b80 1 --2- 172.21.15.119:0/1353861555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdf4d3ad0 0x562cdf4dfda0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2d54005a80 tx=0x7f2d540056b0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.477+0000 7f2d840d7b80 1 -- 172.21.15.119:0/1353861555 shutdown_connections 2023-12-07T10:34:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.477+0000 7f2d840d7b80 1 --2- 172.21.15.119:0/1353861555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562cdf4d9380 0x562cdf4e0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.477+0000 7f2d840d7b80 1 --2- 172.21.15.119:0/1353861555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdf4d3ad0 0x562cdf4dfda0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.477+0000 7f2d840d7b80 1 --2- 172.21.15.119:0/1353861555 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562cdf554bc0 0x562cdf556fe0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.477+0000 7f2d840d7b80 1 --2- 172.21.15.119:0/1353861555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562cdf4d8a60 0x562cdf4e02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.477+0000 7f2d51ffb700 0 client.27898 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:34:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.477+0000 7f2d840d7b80 1 -- 172.21.15.119:0/1353861555 >> 172.21.15.119:0/1353861555 conn(0x562cdf402300 msgr2=0x562cdf4daca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:34:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.477+0000 7f2d840d7b80 1 -- 172.21.15.119:0/1353861555 shutdown_connections 2023-12-07T10:34:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.478+0000 7f2d840d7b80 1 -- 172.21.15.119:0/1353861555 wait complete. 2023-12-07T10:34:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.478+0000 7f2d840d7b80 20 client.27898 trim_cache size 0 max 16384 2023-12-07T10:34:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:34:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:34:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:34:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:34:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.479+0000 7f2d707ff700 1 -- 172.21.15.119:0/3288966381 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f2d580a1d80 msgr2=0x7f2d580a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.479+0000 7f2d707ff700 1 --2- 172.21.15.119:0/3288966381 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f2d580a1d80 0x7f2d580a4240 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2d600a2040 tx=0x7f2d60005020 comp rx=0 tx=0).stop 2023-12-07T10:34:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.479+0000 7f2d707ff700 1 -- 172.21.15.119:0/3288966381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d681124f0 msgr2=0x7f2d681a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.479+0000 7f2d707ff700 1 --2- 172.21.15.119:0/3288966381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d681124f0 0x7f2d681a29e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2d6806abd0 tx=0x7f2d6000dab0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.479+0000 7f2d707ff700 1 -- 172.21.15.119:0/3288966381 shutdown_connections 2023-12-07T10:34:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.479+0000 7f2d707ff700 1 --2- 172.21.15.119:0/3288966381 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f2d580a1d80 0x7f2d580a4240 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.479+0000 7f2d707ff700 1 --2- 172.21.15.119:0/3288966381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d681124f0 0x7f2d681a29e0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.479+0000 7f2d707ff700 1 --2- 172.21.15.119:0/3288966381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d680739b0 0x7f2d6819e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.479+0000 7f2d707ff700 1 --2- 172.21.15.119:0/3288966381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d680742d0 0x7f2d6819e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.479+0000 7f2d707ff700 1 -- 172.21.15.119:0/3288966381 >> 172.21.15.119:0/3288966381 conn(0x7f2d680fcfb0 msgr2=0x7f2d680fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:34:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.480+0000 7f2d707ff700 1 -- 172.21.15.119:0/3288966381 shutdown_connections 2023-12-07T10:34:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:24.481+0000 7f2d707ff700 1 -- 172.21.15.119:0/3288966381 wait complete. 2023-12-07T10:34:24.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:24 smithi159 ceph-mon[134726]: pgmap v438: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 14 MiB/s wr, 756 op/s 2023-12-07T10:34:24.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:24 smithi153 ceph-mon[133625]: pgmap v438: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 14 MiB/s wr, 756 op/s 2023-12-07T10:34:24.964 INFO:tasks.workunit.client.0.smithi119.stdout: 10 20299 34.23 MB/sec warmup 37 sec latency 240.921 ms 2023-12-07T10:34:25.964 INFO:tasks.workunit.client.0.smithi119.stdout: 10 21211 34.62 MB/sec warmup 38 sec latency 60.570 ms 2023-12-07T10:34:26.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:26 smithi119 ceph-mon[97908]: pgmap v439: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 14 MiB/s wr, 680 op/s 2023-12-07T10:34:26.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:26 smithi159 ceph-mon[134726]: pgmap v439: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 14 MiB/s wr, 680 op/s 2023-12-07T10:34:26.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:26 smithi153 ceph-mon[133625]: pgmap v439: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 14 MiB/s wr, 680 op/s 2023-12-07T10:34:26.964 INFO:tasks.workunit.client.0.smithi119.stdout: 10 22134 35.12 MB/sec warmup 39 sec latency 62.508 ms 2023-12-07T10:34:27.964 INFO:tasks.workunit.client.0.smithi119.stdout: 10 23050 35.20 MB/sec warmup 40 sec latency 49.482 ms 2023-12-07T10:34:28.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:28 smithi119 ceph-mon[97908]: pgmap v440: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 17 MiB/s wr, 1.06k op/s 2023-12-07T10:34:28.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:28 smithi159 ceph-mon[134726]: pgmap v440: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 17 MiB/s wr, 1.06k op/s 2023-12-07T10:34:28.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:28 smithi153 ceph-mon[133625]: pgmap v440: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 17 MiB/s wr, 1.06k op/s 2023-12-07T10:34:28.964 INFO:tasks.workunit.client.0.smithi119.stdout: 10 23867 35.60 MB/sec warmup 41 sec latency 67.417 ms 2023-12-07T10:34:29.964 INFO:tasks.workunit.client.0.smithi119.stdout: 10 24730 35.75 MB/sec warmup 42 sec latency 67.362 ms 2023-12-07T10:34:30.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:30 smithi119 ceph-mon[97908]: pgmap v441: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.8 MiB/s wr, 721 op/s 2023-12-07T10:34:30.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:30 smithi159 ceph-mon[134726]: pgmap v441: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.8 MiB/s wr, 721 op/s 2023-12-07T10:34:30.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:30 smithi153 ceph-mon[133625]: pgmap v441: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.8 MiB/s wr, 721 op/s 2023-12-07T10:34:30.964 INFO:tasks.workunit.client.0.smithi119.stdout: 10 25713 36.23 MB/sec warmup 43 sec latency 44.909 ms 2023-12-07T10:34:31.964 INFO:tasks.workunit.client.0.smithi119.stdout: 10 26516 36.41 MB/sec warmup 44 sec latency 56.527 ms 2023-12-07T10:34:32.262 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:32 smithi159 ceph-mon[134726]: pgmap v442: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 963 op/s 2023-12-07T10:34:32.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:32 smithi153 ceph-mon[133625]: pgmap v442: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 963 op/s 2023-12-07T10:34:32.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:32 smithi119 ceph-mon[97908]: pgmap v442: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 963 op/s 2023-12-07T10:34:32.966 INFO:tasks.workunit.client.0.smithi119.stdout: 10 27409 36.71 MB/sec warmup 45 sec latency 47.168 ms 2023-12-07T10:34:33.966 INFO:tasks.workunit.client.0.smithi119.stdout: 10 28135 36.70 MB/sec warmup 46 sec latency 99.151 ms 2023-12-07T10:34:34.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:34 smithi119 ceph-mon[97908]: pgmap v443: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 7.8 MiB/s wr, 736 op/s 2023-12-07T10:34:34.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:34 smithi159 ceph-mon[134726]: pgmap v443: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 7.8 MiB/s wr, 736 op/s 2023-12-07T10:34:34.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:34 smithi153 ceph-mon[133625]: pgmap v443: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 7.8 MiB/s wr, 736 op/s 2023-12-07T10:34:34.967 INFO:tasks.workunit.client.0.smithi119.stdout: 10 29154 37.06 MB/sec warmup 47 sec latency 72.211 ms 2023-12-07T10:34:35.967 INFO:tasks.workunit.client.0.smithi119.stdout: 10 30005 37.31 MB/sec warmup 48 sec latency 103.665 ms 2023-12-07T10:34:36.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:36 smithi119 ceph-mon[97908]: pgmap v444: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.5 MiB/s wr, 660 op/s 2023-12-07T10:34:36.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:36 smithi159 ceph-mon[134726]: pgmap v444: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.5 MiB/s wr, 660 op/s 2023-12-07T10:34:36.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:36 smithi153 ceph-mon[133625]: pgmap v444: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.5 MiB/s wr, 660 op/s 2023-12-07T10:34:36.967 INFO:tasks.workunit.client.0.smithi119.stdout: 10 30812 37.47 MB/sec warmup 49 sec latency 74.994 ms 2023-12-07T10:34:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.770+0000 7fb39a7ff700 1 -- 172.21.15.119:0/1039187163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb394105710 msgr2=0x7fb394105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.770+0000 7fb39a7ff700 1 --2- 172.21.15.119:0/1039187163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb394105710 0x7fb394105af0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb3840074e0 tx=0x7fb38400af90 comp rx=0 tx=0).stop 2023-12-07T10:34:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.770+0000 7fb39a7ff700 1 -- 172.21.15.119:0/1039187163 shutdown_connections 2023-12-07T10:34:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.770+0000 7fb39a7ff700 1 --2- 172.21.15.119:0/1039187163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb394100110 0x7fb39410c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.770+0000 7fb39a7ff700 1 --2- 172.21.15.119:0/1039187163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb394105710 0x7fb394105af0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.770+0000 7fb39a7ff700 1 --2- 172.21.15.119:0/1039187163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3940ff750 0x7fb3940ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.770+0000 7fb39a7ff700 1 -- 172.21.15.119:0/1039187163 >> 172.21.15.119:0/1039187163 conn(0x7fb394076590 msgr2=0x7fb3940fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:34:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.770+0000 7fb39a7ff700 1 -- 172.21.15.119:0/1039187163 shutdown_connections 2023-12-07T10:34:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.771+0000 7fb39a7ff700 1 -- 172.21.15.119:0/1039187163 wait complete. 2023-12-07T10:34:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.774+0000 7fb39a7ff700 1 Processor -- start 2023-12-07T10:34:37.775 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.774+0000 7fb39a7ff700 1 -- start start 2023-12-07T10:34:37.775 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.774+0000 7fb39a7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3940ff750 0x7fb39406bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:37.775 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.775+0000 7fb39a7ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb394100110 0x7fb39406c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.775+0000 7fb39a7ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb394105710 0x7fb39406c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.775+0000 7fb39a7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3940721f0 con 0x7fb3940ff750 2023-12-07T10:34:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.775+0000 7fb39a7ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb394072330 con 0x7fb394100110 2023-12-07T10:34:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.775+0000 7fb39a7ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3940724a0 con 0x7fb394105710 2023-12-07T10:34:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.775+0000 7fb3937fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb394100110 0x7fb39406c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.775+0000 7fb3937fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb394100110 0x7fb39406c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:60046/0 (socket says 172.21.15.119:60046) 2023-12-07T10:34:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.775+0000 7fb3937fe700 1 -- 172.21.15.119:0/637133337 learned_addr learned my addr 172.21.15.119:0/637133337 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:34:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.775+0000 7fb398d9c700 1 --2- 172.21.15.119:0/637133337 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb394105710 0x7fb39406c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.775+0000 7fb3937fe700 1 -- 172.21.15.119:0/637133337 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb394105710 msgr2=0x7fb39406c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.775+0000 7fb3937fe700 1 --2- 172.21.15.119:0/637133337 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb394105710 0x7fb39406c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.775+0000 7fb3937fe700 1 -- 172.21.15.119:0/637133337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3940ff750 msgr2=0x7fb39406bc70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-07T10:34:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.775+0000 7fb3937fe700 1 --2- 172.21.15.119:0/637133337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3940ff750 0x7fb39406bc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.776+0000 7fb3937fe700 1 -- 172.21.15.119:0/637133337 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3840071c0 con 0x7fb394100110 2023-12-07T10:34:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.776+0000 7fb3937fe700 1 --2- 172.21.15.119:0/637133337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb394100110 0x7fb39406c1b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb38800b550 tx=0x7fb38800b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:34:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.777+0000 7fb3917fa700 1 -- 172.21.15.119:0/637133337 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb388011830 con 0x7fb394100110 2023-12-07T10:34:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.777+0000 7fb3917fa700 1 -- 172.21.15.119:0/637133337 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb388011e50 con 0x7fb394100110 2023-12-07T10:34:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.777+0000 7fb3917fa700 1 -- 172.21.15.119:0/637133337 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38800f2e0 con 0x7fb394100110 2023-12-07T10:34:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.777+0000 7fb39a7ff700 1 -- 172.21.15.119:0/637133337 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb394072780 con 0x7fb394100110 2023-12-07T10:34:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.777+0000 7fb39a7ff700 1 -- 172.21.15.119:0/637133337 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb394072c40 con 0x7fb394100110 2023-12-07T10:34:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.790+0000 7fb3917fa700 1 -- 172.21.15.119:0/637133337 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fb388011990 con 0x7fb394100110 2023-12-07T10:34:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.794+0000 7fb3ae15ab80 1 Processor -- start 2023-12-07T10:34:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.794+0000 7fb3ae15ab80 1 -- start start 2023-12-07T10:34:37.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.794+0000 7fb3917fa700 1 --2- 172.21.15.119:0/637133337 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb37c0a1d00 0x7fb37c0a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:37.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.795+0000 7fb3917fa700 1 -- 172.21.15.119:0/637133337 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fb3880cd6f0 con 0x7fb394100110 2023-12-07T10:34:37.795 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.796+0000 7fb398d9c700 1 --2- 172.21.15.119:0/637133337 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb37c0a1d00 0x7fb37c0a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.796+0000 7fb3ae15ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f74145a60 0x562f74145e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.796+0000 7fb3ae15ab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74146380 0x562f74140540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.796+0000 7fb3ae15ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f74140ad0 0x562f74140f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.796+0000 7fb3ae15ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f740b4760 con 0x562f74145a60 2023-12-07T10:34:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.796+0000 7fb3ae15ab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f7409b740 con 0x562f74146380 2023-12-07T10:34:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.796+0000 7fb3ae15ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f74099060 con 0x562f74140ad0 2023-12-07T10:34:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.797+0000 7fb393fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f74145a60 0x562f74145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.797+0000 7fb393fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f74145a60 0x562f74145e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44412/0 (socket says 172.21.15.119:44412) 2023-12-07T10:34:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.797+0000 7fb393fff700 1 -- 172.21.15.119:0/1975424834 learned_addr learned my addr 172.21.15.119:0/1975424834 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:34:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.797+0000 7fb398d9c700 1 --2- 172.21.15.119:0/637133337 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb37c0a1d00 0x7fb37c0a41c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb38c003ee0 tx=0x7fb38c009130 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:34:37.798 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.797+0000 7fb398d9c700 1 --2- 172.21.15.119:0/1975424834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f74140ad0 0x562f74140f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:37.801 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.798+0000 7fb3937fe700 1 --2- 172.21.15.119:0/1975424834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74146380 0x562f74140540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:37.801 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.801+0000 7fb393fff700 1 -- 172.21.15.119:0/1975424834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f74140ad0 msgr2=0x562f74140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:37.801 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.801+0000 7fb393fff700 1 --2- 172.21.15.119:0/1975424834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f74140ad0 0x562f74140f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:37.801 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.801+0000 7fb393fff700 1 -- 172.21.15.119:0/1975424834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74146380 msgr2=0x562f74140540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:37.801 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.801+0000 7fb393fff700 1 --2- 172.21.15.119:0/1975424834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74146380 0x562f74140540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:37.801 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.801+0000 7fb393fff700 1 -- 172.21.15.119:0/1975424834 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f73f115d0 con 0x562f74145a60 2023-12-07T10:34:37.801 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.801+0000 7fb3937fe700 1 --2- 172.21.15.119:0/1975424834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74146380 0x562f74140540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:34:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.801+0000 7fb393fff700 1 --2- 172.21.15.119:0/1975424834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f74145a60 0x562f74145e40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb384006010 tx=0x7fb384005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9931f2b457a4ee5c server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:34:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.802+0000 7fb37a7fc700 1 -- 172.21.15.119:0/1975424834 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38400f070 con 0x562f74145a60 2023-12-07T10:34:37.803 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.802+0000 7fb37a7fc700 1 -- 172.21.15.119:0/1975424834 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb384007e80 con 0x562f74145a60 2023-12-07T10:34:37.803 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.802+0000 7fb37a7fc700 1 -- 172.21.15.119:0/1975424834 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38400cdf0 con 0x562f74145a60 2023-12-07T10:34:37.803 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.804+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/1975424834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f74145a60 msgr2=0x562f74145e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:37.803 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.804+0000 7fb3ae15ab80 1 --2- 172.21.15.119:0/1975424834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f74145a60 0x562f74145e40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb384006010 tx=0x7fb384005da0 comp rx=0 tx=0).stop 2023-12-07T10:34:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.804+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/1975424834 shutdown_connections 2023-12-07T10:34:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.804+0000 7fb3ae15ab80 1 --2- 172.21.15.119:0/1975424834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f74140ad0 0x562f74140f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.804+0000 7fb3ae15ab80 1 --2- 172.21.15.119:0/1975424834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f74145a60 0x562f74145e40 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.804+0000 7fb3ae15ab80 1 --2- 172.21.15.119:0/1975424834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74146380 0x562f74140540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.804+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/1975424834 >> 172.21.15.119:0/1975424834 conn(0x562f7406f300 msgr2=0x562f7413ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:34:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.805+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/1975424834 shutdown_connections 2023-12-07T10:34:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.806+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/1975424834 wait complete. 2023-12-07T10:34:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.807+0000 7fb3ae15ab80 1 -- start start 2023-12-07T10:34:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.807+0000 7fb3ae15ab80 1 Processor -- start 2023-12-07T10:34:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.808+0000 7fb3ae15ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f74140ad0 0x562f74149740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.809+0000 7fb3ae15ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f74145a60 0x562f74149c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.809+0000 7fb3ae15ab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74146380 0x562f7414a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.810+0000 7fb398d9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74146380 0x562f7414a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.810+0000 7fb398d9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74146380 0x562f7414a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:60056/0 (socket says 172.21.15.119:60056) 2023-12-07T10:34:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.810+0000 7fb398d9c700 1 -- 172.21.15.119:0/3935393063 learned_addr learned my addr 172.21.15.119:0/3935393063 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:34:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.810+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/3935393063 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f74094950 con 0x562f74145a60 2023-12-07T10:34:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.810+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/3935393063 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f7404e500 con 0x562f74146380 2023-12-07T10:34:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.810+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/3935393063 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f7404e1a0 con 0x562f74140ad0 2023-12-07T10:34:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.810+0000 7fb3ae15ab80 10 client.0 _ll_register_callbacks cb 0x562f740358e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:34:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.810+0000 7fb398d9c700 1 -- 172.21.15.119:0/3935393063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f74140ad0 msgr2=0x562f74149740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.810+0000 7fb393fff700 1 --2- 172.21.15.119:0/3935393063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f74140ad0 0x562f74149740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.811+0000 7fb398d9c700 1 --2- 172.21.15.119:0/3935393063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f74140ad0 0x562f74149740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.811+0000 7fb398d9c700 1 -- 172.21.15.119:0/3935393063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f74145a60 msgr2=0x562f74149c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.811+0000 7fb398d9c700 1 --2- 172.21.15.119:0/3935393063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f74145a60 0x562f74149c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.811+0000 7fb398d9c700 1 -- 172.21.15.119:0/3935393063 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb39410ced0 con 0x562f74146380 2023-12-07T10:34:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.811+0000 7fb393fff700 1 --2- 172.21.15.119:0/3935393063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f74140ad0 0x562f74149740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:34:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.812+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/3935393063 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f7410f740 con 0x562f74146380 2023-12-07T10:34:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.812+0000 7fb398d9c700 1 --2- 172.21.15.119:0/3935393063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74146380 0x562f7414a1c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0159f0 tx=0x7fb38c004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db2f011ce92b9ab3 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:34:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.812+0000 7fb3737fe700 10 client.27929 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:34:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.812+0000 7fb3737fe700 1 -- 172.21.15.119:0/3935393063 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c016c90 con 0x562f74146380 2023-12-07T10:34:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.813+0000 7fb3737fe700 1 -- 172.21.15.119:0/3935393063 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fb38c0048a0 con 0x562f74146380 2023-12-07T10:34:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.813+0000 7fb3737fe700 1 -- 172.21.15.119:0/3935393063 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb38c016c90 con 0x562f74146380 2023-12-07T10:34:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.813+0000 7fb3ae15ab80 10 client.27929 fetch_fsmap learned FSMap version 70 2023-12-07T10:34:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.813+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/3935393063 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({fsmap=70}) v3 -- 0x562f73ebdd80 con 0x562f74146380 2023-12-07T10:34:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.815+0000 7fb3737fe700 1 -- 172.21.15.119:0/3935393063 <== mon.2 v2:172.21.15.153:3300/0 4 ==== fsmap(e 70) v1 ==== 7954+0+0 (secure 0 0 0) 0x7fb38c02b440 con 0x562f74146380 2023-12-07T10:34:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.815+0000 7fb3ae15ab80 10 client.27929 fetch_fsmap finished waiting for FSMap version 70 2023-12-07T10:34:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.815+0000 7fb3ae15ab80 10 client.27929 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:34:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.815+0000 7fb3ae15ab80 20 client.27929 populate_metadata read hostname 'smithi119' 2023-12-07T10:34:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.815+0000 7fb3ae15ab80 1 --2- 172.21.15.119:0/3935393063 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562f74159e60 0x562f7415c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.815+0000 7fb3ae15ab80 4 client.27929 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:34:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.815+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/3935393063 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f7405af70 con 0x562f74159e60 2023-12-07T10:34:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.815+0000 7fb393fff700 1 --2- 172.21.15.119:0/3935393063 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562f74159e60 0x562f7415c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.816+0000 7fb393fff700 1 --2- 172.21.15.119:0/3935393063 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562f74159e60 0x562f7415c2a0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fb3a849990ce590 server_cookie=16e8d07841fa457c in_seq=0 out_seq=0 2023-12-07T10:34:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.816+0000 7fb3737fe700 10 client.27929 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:34:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.817+0000 7fb3737fe700 1 -- 172.21.15.119:0/3935393063 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562f7405af70 con 0x562f74159e60 2023-12-07T10:34:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.817+0000 7fb3737fe700 10 client.27929 handle_command_reply: tid=0 2023-12-07T10:34:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.818+0000 7fb3ae15ab80 1 client.27929 shutdown 2023-12-07T10:34:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.818+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/3935393063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74146380 msgr2=0x562f7414a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.818+0000 7fb3ae15ab80 1 --2- 172.21.15.119:0/3935393063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74146380 0x562f7414a1c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0159f0 tx=0x7fb38c004b40 comp rx=0 tx=0).stop 2023-12-07T10:34:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.818+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/3935393063 shutdown_connections 2023-12-07T10:34:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.818+0000 7fb3ae15ab80 1 --2- 172.21.15.119:0/3935393063 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562f74159e60 0x562f7415c2a0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.818+0000 7fb3ae15ab80 1 --2- 172.21.15.119:0/3935393063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74146380 0x562f7414a1c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.818+0000 7fb3ae15ab80 1 --2- 172.21.15.119:0/3935393063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f74140ad0 0x562f74149740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.818+0000 7fb3ae15ab80 1 --2- 172.21.15.119:0/3935393063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f74145a60 0x562f74149c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.818+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/3935393063 >> 172.21.15.119:0/3935393063 conn(0x562f7406f300 msgr2=0x562f74135980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:34:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.818+0000 7fb3737fe700 0 client.27929 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:34:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.819+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/3935393063 shutdown_connections 2023-12-07T10:34:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.819+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/3935393063 wait complete. 2023-12-07T10:34:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.819+0000 7fb3ae15ab80 20 client.27929 trim_cache size 0 max 16384 2023-12-07T10:34:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.856+0000 7fb3ae15ab80 1 Processor -- start 2023-12-07T10:34:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.856+0000 7fb3ae15ab80 1 -- start start 2023-12-07T10:34:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.856+0000 7fb3ae15ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f74140ad0 0x562f74150d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.856+0000 7fb3ae15ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f74145a60 0x562f741512b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.856+0000 7fb3ae15ab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74146380 0x562f741439a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.856+0000 7fb3ae15ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f73f138c0 con 0x562f74140ad0 2023-12-07T10:34:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.856+0000 7fb3ae15ab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f741456e0 con 0x562f74146380 2023-12-07T10:34:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.856+0000 7fb3ae15ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f7413b270 con 0x562f74145a60 2023-12-07T10:34:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.857+0000 7fb398d9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74146380 0x562f741439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.858+0000 7fb398d9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74146380 0x562f741439a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:60068/0 (socket says 172.21.15.119:60068) 2023-12-07T10:34:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.858+0000 7fb398d9c700 1 -- 172.21.15.119:0/3103296870 learned_addr learned my addr 172.21.15.119:0/3103296870 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:34:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.857+0000 7fb3937fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f74145a60 0x562f741512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.858+0000 7fb393fff700 1 --2- 172.21.15.119:0/3103296870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f74140ad0 0x562f74150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.859+0000 7fb398d9c700 1 -- 172.21.15.119:0/3103296870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f74145a60 msgr2=0x562f741512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.859+0000 7fb398d9c700 1 --2- 172.21.15.119:0/3103296870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f74145a60 0x562f741512b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.859+0000 7fb398d9c700 1 -- 172.21.15.119:0/3103296870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f74140ad0 msgr2=0x562f74150d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.859+0000 7fb398d9c700 1 --2- 172.21.15.119:0/3103296870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f74140ad0 0x562f74150d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.859+0000 7fb398d9c700 1 -- 172.21.15.119:0/3103296870 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f73ea6c00 con 0x562f74146380 2023-12-07T10:34:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.859+0000 7fb393fff700 1 --2- 172.21.15.119:0/3103296870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f74140ad0 0x562f74150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:34:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.859+0000 7fb398d9c700 1 --2- 172.21.15.119:0/3103296870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74146380 0x562f741439a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb38c016820 tx=0x7fb38c022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b279dba0f29fa3cd server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:34:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.860+0000 7fb37affd700 1 -- 172.21.15.119:0/3103296870 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c02b440 con 0x562f74146380 2023-12-07T10:34:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.860+0000 7fb37affd700 1 -- 172.21.15.119:0/3103296870 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562f7410f740 con 0x562f74146380 2023-12-07T10:34:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.860+0000 7fb37affd700 1 -- 172.21.15.119:0/3103296870 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c02b440 con 0x562f74146380 2023-12-07T10:34:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.861+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/3103296870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74146380 msgr2=0x562f741439a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.861+0000 7fb3ae15ab80 1 --2- 172.21.15.119:0/3103296870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74146380 0x562f741439a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb38c016820 tx=0x7fb38c022000 comp rx=0 tx=0).stop 2023-12-07T10:34:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.862+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/3103296870 shutdown_connections 2023-12-07T10:34:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.862+0000 7fb3ae15ab80 1 --2- 172.21.15.119:0/3103296870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74146380 0x562f741439a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.862+0000 7fb3ae15ab80 1 --2- 172.21.15.119:0/3103296870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f74140ad0 0x562f74150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.862+0000 7fb3ae15ab80 1 --2- 172.21.15.119:0/3103296870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f74145a60 0x562f741512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.862+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/3103296870 >> 172.21.15.119:0/3103296870 conn(0x562f7406f300 msgr2=0x562f7413a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:34:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.862+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/3103296870 shutdown_connections 2023-12-07T10:34:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.862+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/3103296870 wait complete. 2023-12-07T10:34:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.863+0000 7fb3ae15ab80 1 -- start start 2023-12-07T10:34:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.863+0000 7fb3ae15ab80 1 Processor -- start 2023-12-07T10:34:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.866+0000 7fb3ae15ab80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74140ad0 0x562f741a4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.866+0000 7fb3ae15ab80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f74145a60 0x562f741a4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.866+0000 7fb3ae15ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f74146380 0x562f741a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.866+0000 7fb3ae15ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f741af550 con 0x562f74146380 2023-12-07T10:34:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.866+0000 7fb3ae15ab80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f741af6c0 con 0x562f74140ad0 2023-12-07T10:34:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.866+0000 7fb3ae15ab80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f741af880 con 0x562f74145a60 2023-12-07T10:34:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.866+0000 7fb3ae15ab80 10 client.0 _ll_register_callbacks cb 0x562f740358e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:34:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.867+0000 7fb393fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74140ad0 0x562f741a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.867+0000 7fb393fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74140ad0 0x562f741a4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:60080/0 (socket says 172.21.15.119:60080) 2023-12-07T10:34:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.867+0000 7fb393fff700 1 -- 172.21.15.119:0/24759636 learned_addr learned my addr 172.21.15.119:0/24759636 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:34:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.867+0000 7fb3937fe700 1 --2- 172.21.15.119:0/24759636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f74145a60 0x562f741a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.867+0000 7fb393fff700 1 -- 172.21.15.119:0/24759636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f74145a60 msgr2=0x562f741a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.867+0000 7fb393fff700 1 --2- 172.21.15.119:0/24759636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f74145a60 0x562f741a4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.868+0000 7fb393fff700 1 -- 172.21.15.119:0/24759636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f74146380 msgr2=0x562f741a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.868+0000 7fb393fff700 1 --2- 172.21.15.119:0/24759636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f74146380 0x562f741a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.868+0000 7fb393fff700 1 -- 172.21.15.119:0/24759636 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f73ebdd80 con 0x562f74140ad0 2023-12-07T10:34:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.868+0000 7fb393fff700 1 --2- 172.21.15.119:0/24759636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74140ad0 0x562f741a4350 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb384006010 tx=0x7fb38400c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=605aea9ce05791d1 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:34:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.868+0000 7fb3737fe700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:34:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.868+0000 7fb3737fe700 1 -- 172.21.15.119:0/24759636 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38400f030 con 0x562f74140ad0 2023-12-07T10:34:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.868+0000 7fb3737fe700 1 -- 172.21.15.119:0/24759636 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fb384003890 con 0x562f74140ad0 2023-12-07T10:34:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.868+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/24759636 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f741336f0 con 0x562f74140ad0 2023-12-07T10:34:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.869+0000 7fb3737fe700 1 -- 172.21.15.119:0/24759636 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb38400cc40 con 0x562f74140ad0 2023-12-07T10:34:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.869+0000 7fb3ae15ab80 10 client.27935 fetch_fsmap learned FSMap version 70 2023-12-07T10:34:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.869+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/24759636 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x562f73ee41f0 con 0x562f74140ad0 2023-12-07T10:34:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.869+0000 7fb3737fe700 1 -- 172.21.15.119:0/24759636 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562f741336f0 con 0x562f74140ad0 2023-12-07T10:34:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.869+0000 7fb3737fe700 1 -- 172.21.15.119:0/24759636 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 70) v1 ==== 7954+0+0 (secure 0 0 0) 0x7fb384025040 con 0x562f74140ad0 2023-12-07T10:34:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.870+0000 7fb3ae15ab80 10 client.27935 fetch_fsmap finished waiting for FSMap version 70 2023-12-07T10:34:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.870+0000 7fb3ae15ab80 10 client.27935 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:34:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.870+0000 7fb3ae15ab80 20 client.27935 populate_metadata read hostname 'smithi119' 2023-12-07T10:34:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.870+0000 7fb3ae15ab80 1 --2- 172.21.15.119:0/24759636 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562f741b0f60 0x562f741b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.870+0000 7fb3ae15ab80 4 client.27935 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:34:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.870+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/24759636 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562f7404d430 con 0x562f741b0f60 2023-12-07T10:34:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.870+0000 7fb3937fe700 1 --2- 172.21.15.119:0/24759636 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562f741b0f60 0x562f741b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.871+0000 7fb3937fe700 1 --2- 172.21.15.119:0/24759636 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562f741b0f60 0x562f741b3380 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f5167de35b04a30 server_cookie=cf387eaecf2b90a2 in_seq=0 out_seq=0 2023-12-07T10:34:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:37.871+0000 7fb3737fe700 10 client.27935 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:34:37.967 INFO:tasks.workunit.client.0.smithi119.stdout: 10 31443 37.35 MB/sec warmup 50 sec latency 96.467 ms 2023-12-07T10:34:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.366+0000 7fb3737fe700 1 -- 172.21.15.119:0/24759636 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562f7404d430 con 0x562f741b0f60 2023-12-07T10:34:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.366+0000 7fb3737fe700 10 client.27935 handle_command_reply: tid=0 2023-12-07T10:34:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.366+0000 7fb3ae15ab80 1 client.27935 shutdown 2023-12-07T10:34:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.366+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/24759636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74140ad0 msgr2=0x562f741a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.366+0000 7fb3ae15ab80 1 --2- 172.21.15.119:0/24759636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74140ad0 0x562f741a4350 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb384006010 tx=0x7fb38400c6e0 comp rx=0 tx=0).stop 2023-12-07T10:34:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.367+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/24759636 shutdown_connections 2023-12-07T10:34:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.367+0000 7fb3ae15ab80 1 --2- 172.21.15.119:0/24759636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f74146380 0x562f741a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.367+0000 7fb3ae15ab80 1 --2- 172.21.15.119:0/24759636 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x562f741b0f60 0x562f741b3380 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.367+0000 7fb3ae15ab80 1 --2- 172.21.15.119:0/24759636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x562f74140ad0 0x562f741a4350 secure :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb384006010 tx=0x7fb38400c6e0 comp rx=0 tx=0).stop 2023-12-07T10:34:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.367+0000 7fb3ae15ab80 1 --2- 172.21.15.119:0/24759636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f74145a60 0x562f741a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.367+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/24759636 >> 172.21.15.119:0/24759636 conn(0x562f7406f300 msgr2=0x562f741439a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:34:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.367+0000 7fb3737fe700 0 client.27935 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:34:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.368+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/24759636 shutdown_connections 2023-12-07T10:34:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.369+0000 7fb3ae15ab80 1 -- 172.21.15.119:0/24759636 wait complete. 2023-12-07T10:34:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.370+0000 7fb3ae15ab80 20 client.27935 trim_cache size 0 max 16384 2023-12-07T10:34:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:34:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:34:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:34:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:34:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.371+0000 7fb39a7ff700 1 -- 172.21.15.119:0/637133337 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb37c0a1d00 msgr2=0x7fb37c0a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.371+0000 7fb39a7ff700 1 --2- 172.21.15.119:0/637133337 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb37c0a1d00 0x7fb37c0a41c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb38c003ee0 tx=0x7fb38c009130 comp rx=0 tx=0).stop 2023-12-07T10:34:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.371+0000 7fb39a7ff700 1 -- 172.21.15.119:0/637133337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb394100110 msgr2=0x7fb39406c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.371+0000 7fb39a7ff700 1 --2- 172.21.15.119:0/637133337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb394100110 0x7fb39406c1b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb38800b550 tx=0x7fb38800b860 comp rx=0 tx=0).stop 2023-12-07T10:34:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.371+0000 7fb39a7ff700 1 -- 172.21.15.119:0/637133337 shutdown_connections 2023-12-07T10:34:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.371+0000 7fb39a7ff700 1 --2- 172.21.15.119:0/637133337 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fb37c0a1d00 0x7fb37c0a41c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.371+0000 7fb39a7ff700 1 --2- 172.21.15.119:0/637133337 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb394105710 0x7fb39406c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.371+0000 7fb39a7ff700 1 --2- 172.21.15.119:0/637133337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3940ff750 0x7fb39406bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.371+0000 7fb39a7ff700 1 --2- 172.21.15.119:0/637133337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb394100110 0x7fb39406c1b0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.371+0000 7fb39a7ff700 1 -- 172.21.15.119:0/637133337 >> 172.21.15.119:0/637133337 conn(0x7fb394076590 msgr2=0x7fb3941026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:34:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.371+0000 7fb39a7ff700 1 -- 172.21.15.119:0/637133337 shutdown_connections 2023-12-07T10:34:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:38.371+0000 7fb39a7ff700 1 -- 172.21.15.119:0/637133337 wait complete. 2023-12-07T10:34:38.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:38 smithi119 ceph-mon[97908]: pgmap v445: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.5 MiB/s wr, 925 op/s 2023-12-07T10:34:38.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:38 smithi159 ceph-mon[134726]: pgmap v445: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.5 MiB/s wr, 925 op/s 2023-12-07T10:34:38.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:38 smithi153 ceph-mon[133625]: pgmap v445: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.5 MiB/s wr, 925 op/s 2023-12-07T10:34:38.967 INFO:tasks.workunit.client.0.smithi119.stdout: 10 31934 37.13 MB/sec warmup 51 sec latency 319.633 ms 2023-12-07T10:34:39.967 INFO:tasks.workunit.client.0.smithi119.stdout: 10 32813 37.23 MB/sec warmup 52 sec latency 58.509 ms 2023-12-07T10:34:40.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:40 smithi119 ceph-mon[97908]: pgmap v446: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 6.5 MiB/s wr, 549 op/s 2023-12-07T10:34:40.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:40 smithi159 ceph-mon[134726]: pgmap v446: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 6.5 MiB/s wr, 549 op/s 2023-12-07T10:34:40.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:40 smithi153 ceph-mon[133625]: pgmap v446: 129 pgs: 129 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 6.5 MiB/s wr, 549 op/s 2023-12-07T10:34:40.969 INFO:tasks.workunit.client.0.smithi119.stdout: 10 32875 36.61 MB/sec warmup 53 sec latency 737.068 ms 2023-12-07T10:34:41.967 INFO:tasks.workunit.client.0.smithi119.stdout: 10 33087 36.08 MB/sec warmup 54 sec latency 1451.200 ms 2023-12-07T10:34:42.278 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:42 smithi153 ceph-mon[133625]: pgmap v447: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 843 op/s 2023-12-07T10:34:42.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:42 smithi119 ceph-mon[97908]: pgmap v447: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 843 op/s 2023-12-07T10:34:42.516 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:42 smithi159 ceph-mon[134726]: pgmap v447: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 843 op/s 2023-12-07T10:34:42.967 INFO:tasks.workunit.client.0.smithi119.stdout: 10 33440 35.91 MB/sec warmup 55 sec latency 150.452 ms 2023-12-07T10:34:43.967 INFO:tasks.workunit.client.0.smithi119.stdout: 10 33885 35.83 MB/sec warmup 56 sec latency 156.814 ms 2023-12-07T10:34:44.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:44 smithi119 ceph-mon[97908]: pgmap v448: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 12 MiB/s wr, 732 op/s 2023-12-07T10:34:44.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:44 smithi159 ceph-mon[134726]: pgmap v448: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 12 MiB/s wr, 732 op/s 2023-12-07T10:34:44.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:44 smithi153 ceph-mon[133625]: pgmap v448: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 12 MiB/s wr, 732 op/s 2023-12-07T10:34:44.967 INFO:tasks.workunit.client.0.smithi119.stdout: 10 34626 35.80 MB/sec warmup 57 sec latency 58.694 ms 2023-12-07T10:34:45.967 INFO:tasks.workunit.client.0.smithi119.stdout: 10 34995 35.47 MB/sec warmup 58 sec latency 407.632 ms 2023-12-07T10:34:46.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:46 smithi119 ceph-mon[97908]: pgmap v449: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 11 MiB/s wr, 690 op/s 2023-12-07T10:34:46.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:46 smithi159 ceph-mon[134726]: pgmap v449: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 11 MiB/s wr, 690 op/s 2023-12-07T10:34:46.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:46 smithi153 ceph-mon[133625]: pgmap v449: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 11 MiB/s wr, 690 op/s 2023-12-07T10:34:46.968 INFO:tasks.workunit.client.0.smithi119.stdout: 10 35105 35.00 MB/sec warmup 59 sec latency 881.232 ms 2023-12-07T10:34:47.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:47 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:34:47.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:47 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:34:47.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:47 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:34:47.968 INFO:tasks.workunit.client.0.smithi119.stdout: 10 35436 34.77 MB/sec warmup 60 sec latency 980.625 ms 2023-12-07T10:34:48.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:48 smithi119 ceph-mon[97908]: pgmap v450: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-12-07T10:34:48.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:48 smithi159 ceph-mon[134726]: pgmap v450: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-12-07T10:34:48.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:48 smithi153 ceph-mon[133625]: pgmap v450: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-12-07T10:34:48.968 INFO:tasks.workunit.client.0.smithi119.stdout: 10 35957 34.52 MB/sec warmup 61 sec latency 93.440 ms 2023-12-07T10:34:49.968 INFO:tasks.workunit.client.0.smithi119.stdout: 10 36500 34.54 MB/sec warmup 62 sec latency 81.843 ms 2023-12-07T10:34:50.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:50 smithi119 ceph-mon[97908]: pgmap v451: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 13 MiB/s wr, 784 op/s 2023-12-07T10:34:50.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:50 smithi159 ceph-mon[134726]: pgmap v451: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 13 MiB/s wr, 784 op/s 2023-12-07T10:34:50.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:50 smithi153 ceph-mon[133625]: pgmap v451: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 13 MiB/s wr, 784 op/s 2023-12-07T10:34:50.968 INFO:tasks.workunit.client.0.smithi119.stdout: 10 37281 34.69 MB/sec warmup 63 sec latency 75.583 ms 2023-12-07T10:34:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.597+0000 7fc3b81ff700 1 -- 172.21.15.119:0/1782983351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3b010dd80 msgr2=0x7fc3b01101b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.597+0000 7fc3b81ff700 1 --2- 172.21.15.119:0/1782983351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3b010dd80 0x7fc3b01101b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80098f0 tx=0x7fc3a8009c00 comp rx=0 tx=0).stop 2023-12-07T10:34:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.597+0000 7fc3b81ff700 1 -- 172.21.15.119:0/1782983351 shutdown_connections 2023-12-07T10:34:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.597+0000 7fc3b81ff700 1 --2- 172.21.15.119:0/1782983351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3b010dd80 0x7fc3b01101b0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.597+0000 7fc3b81ff700 1 --2- 172.21.15.119:0/1782983351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3b0073a00 0x7fc3b0073de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.597+0000 7fc3b81ff700 1 --2- 172.21.15.119:0/1782983351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3b0074320 0x7fc3b010d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.597+0000 7fc3b81ff700 1 -- 172.21.15.119:0/1782983351 >> 172.21.15.119:0/1782983351 conn(0x7fc3b00fcfd0 msgr2=0x7fc3b00ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:34:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.598+0000 7fc3b81ff700 1 -- 172.21.15.119:0/1782983351 shutdown_connections 2023-12-07T10:34:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.598+0000 7fc3b81ff700 1 -- 172.21.15.119:0/1782983351 wait complete. 2023-12-07T10:34:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.599+0000 7fc3b81ff700 1 Processor -- start 2023-12-07T10:34:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.600+0000 7fc3b81ff700 1 -- start start 2023-12-07T10:34:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.600+0000 7fc3b81ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3b0073a00 0x7fc3b010d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.600+0000 7fc3b81ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3b0074320 0x7fc3b010d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.600+0000 7fc3b81ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3b010dd80 0x7fc3b010b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.600+0000 7fc3b81ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b010ba00 con 0x7fc3b0074320 2023-12-07T10:34:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.600+0000 7fc3b81ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b010bb70 con 0x7fc3b010dd80 2023-12-07T10:34:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.600+0000 7fc3b81ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b010bd30 con 0x7fc3b0073a00 2023-12-07T10:34:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.601+0000 7fc3b679c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3b010dd80 0x7fc3b010b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.601+0000 7fc3b5f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3b0073a00 0x7fc3b010d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.601+0000 7fc3b5f9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3b0073a00 0x7fc3b010d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:58666/0 (socket says 172.21.15.119:58666) 2023-12-07T10:34:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.601+0000 7fc3b5f9b700 1 -- 172.21.15.119:0/585518900 learned_addr learned my addr 172.21.15.119:0/585518900 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:34:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.601+0000 7fc3b679c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3b010dd80 0x7fc3b010b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:49586/0 (socket says 172.21.15.119:49586) 2023-12-07T10:34:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.601+0000 7fc3b5f9b700 1 -- 172.21.15.119:0/585518900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3b010dd80 msgr2=0x7fc3b010b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.601+0000 7fc3b5f9b700 1 --2- 172.21.15.119:0/585518900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3b010dd80 0x7fc3b010b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.601+0000 7fc3b5f9b700 1 -- 172.21.15.119:0/585518900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3b0074320 msgr2=0x7fc3b010d5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-07T10:34:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.601+0000 7fc3b5f9b700 1 --2- 172.21.15.119:0/585518900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3b0074320 0x7fc3b010d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.601+0000 7fc3b5f9b700 1 -- 172.21.15.119:0/585518900 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a8009520 con 0x7fc3b0073a00 2023-12-07T10:34:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.602+0000 7fc3b5f9b700 1 --2- 172.21.15.119:0/585518900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3b0073a00 0x7fc3b010d080 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac007fa0 tx=0x7fc3ac00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:34:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.603+0000 7fc3a6ffd700 1 -- 172.21.15.119:0/585518900 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac00dda0 con 0x7fc3b0073a00 2023-12-07T10:34:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.603+0000 7fc3a6ffd700 1 -- 172.21.15.119:0/585518900 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3ac00f040 con 0x7fc3b0073a00 2023-12-07T10:34:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.603+0000 7fc3a6ffd700 1 -- 172.21.15.119:0/585518900 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac014690 con 0x7fc3b0073a00 2023-12-07T10:34:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.603+0000 7fc3b81ff700 1 -- 172.21.15.119:0/585518900 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b010c010 con 0x7fc3b0073a00 2023-12-07T10:34:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.603+0000 7fc3b81ff700 1 -- 172.21.15.119:0/585518900 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b01aacd0 con 0x7fc3b0073a00 2023-12-07T10:34:51.611 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.610+0000 7fc3a6ffd700 1 -- 172.21.15.119:0/585518900 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fc3ac0159c0 con 0x7fc3b0073a00 2023-12-07T10:34:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.610+0000 7fc3cbaa8b80 1 Processor -- start 2023-12-07T10:34:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.610+0000 7fc3cbaa8b80 1 -- start start 2023-12-07T10:34:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.611+0000 7fc3cbaa8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564544c9a60 0x5564544c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.612+0000 7fc3cbaa8b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544ca380 0x5564544c4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.612+0000 7fc3cbaa8b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564544c4ad0 0x5564544c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.612+0000 7fc3cbaa8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556454438760 con 0x5564544c9a60 2023-12-07T10:34:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.612+0000 7fc3cbaa8b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645441f740 con 0x5564544ca380 2023-12-07T10:34:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.612+0000 7fc3cbaa8b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645441d060 con 0x5564544c4ad0 2023-12-07T10:34:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.612+0000 7fc3b5f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544ca380 0x5564544c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.612+0000 7fc3b5f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544ca380 0x5564544c4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:49596/0 (socket says 172.21.15.119:49596) 2023-12-07T10:34:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.612+0000 7fc3b5f9b700 1 -- 172.21.15.119:0/659024371 learned_addr learned my addr 172.21.15.119:0/659024371 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:34:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.612+0000 7fc3b679c700 1 --2- 172.21.15.119:0/659024371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564544c9a60 0x5564544c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.612+0000 7fc3b579a700 1 --2- 172.21.15.119:0/659024371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564544c4ad0 0x5564544c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.612+0000 7fc3b5f9b700 1 -- 172.21.15.119:0/659024371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564544c4ad0 msgr2=0x5564544c4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.612+0000 7fc3b5f9b700 1 --2- 172.21.15.119:0/659024371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564544c4ad0 0x5564544c4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.613+0000 7fc3b5f9b700 1 -- 172.21.15.119:0/659024371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564544c9a60 msgr2=0x5564544c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.613+0000 7fc3b5f9b700 1 --2- 172.21.15.119:0/659024371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564544c9a60 0x5564544c9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.613+0000 7fc3b5f9b700 1 -- 172.21.15.119:0/659024371 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564542955d0 con 0x5564544ca380 2023-12-07T10:34:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.614+0000 7fc3a6ffd700 1 --2- 172.21.15.119:0/585518900 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fc39c0a1d40 0x7fc39c0a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.614+0000 7fc3b5f9b700 1 --2- 172.21.15.119:0/659024371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544ca380 0x5564544c4540 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac0ca640 tx=0x7fc3ac0d53d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73f33c21fc168702 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:34:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.614+0000 7fc3a6ffd700 1 -- 172.21.15.119:0/585518900 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc3ac0cd330 con 0x7fc3b0073a00 2023-12-07T10:34:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.614+0000 7fc39bfff700 1 -- 172.21.15.119:0/659024371 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac096060 con 0x5564544ca380 2023-12-07T10:34:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.615+0000 7fc39bfff700 1 -- 172.21.15.119:0/659024371 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3ac099af0 con 0x5564544ca380 2023-12-07T10:34:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.615+0000 7fc39bfff700 1 -- 172.21.15.119:0/659024371 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac099d20 con 0x5564544ca380 2023-12-07T10:34:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.615+0000 7fc3b679c700 1 --2- 172.21.15.119:0/585518900 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fc39c0a1d40 0x7fc39c0a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.616+0000 7fc3cbaa8b80 1 -- 172.21.15.119:0/659024371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544ca380 msgr2=0x5564544c4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.616+0000 7fc3cbaa8b80 1 --2- 172.21.15.119:0/659024371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544ca380 0x5564544c4540 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac0ca640 tx=0x7fc3ac0d53d0 comp rx=0 tx=0).stop 2023-12-07T10:34:51.617 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.616+0000 7fc3cbaa8b80 1 -- 172.21.15.119:0/659024371 shutdown_connections 2023-12-07T10:34:51.617 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.617+0000 7fc3cbaa8b80 1 --2- 172.21.15.119:0/659024371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564544c4ad0 0x5564544c4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:51.617 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.617+0000 7fc3cbaa8b80 1 --2- 172.21.15.119:0/659024371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564544c9a60 0x5564544c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:51.617 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.617+0000 7fc3cbaa8b80 1 --2- 172.21.15.119:0/659024371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544ca380 0x5564544c4540 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:51.617 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.617+0000 7fc3cbaa8b80 1 -- 172.21.15.119:0/659024371 >> 172.21.15.119:0/659024371 conn(0x5564543f3300 msgr2=0x5564544beb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:34:51.617 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.617+0000 7fc3b679c700 1 --2- 172.21.15.119:0/585518900 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fc39c0a1d40 0x7fc39c0a4200 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc3b010dd10 tx=0x7fc3a8019040 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:34:51.618 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.617+0000 7fc3cbaa8b80 1 -- 172.21.15.119:0/659024371 shutdown_connections 2023-12-07T10:34:51.618 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.618+0000 7fc3cbaa8b80 1 -- 172.21.15.119:0/659024371 wait complete. 2023-12-07T10:34:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.619+0000 7fc3cbaa8b80 1 -- start start 2023-12-07T10:34:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.619+0000 7fc3cbaa8b80 1 Processor -- start 2023-12-07T10:34:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.626+0000 7fc3cbaa8b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544c4ad0 0x5564544d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.626+0000 7fc3cbaa8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564544c9a60 0x5564544cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.626+0000 7fc3cbaa8b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564544ca380 0x5564544cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.626+0000 7fc3cbaa8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556454418950 con 0x5564544c9a60 2023-12-07T10:34:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.626+0000 7fc3cbaa8b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564543d2500 con 0x5564544c4ad0 2023-12-07T10:34:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.626+0000 7fc3cbaa8b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564543d21a0 con 0x5564544ca380 2023-12-07T10:34:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.627+0000 7fc3cbaa8b80 10 client.0 _ll_register_callbacks cb 0x5564543b98e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:34:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.627+0000 7fc3b5f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544c4ad0 0x5564544d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.627+0000 7fc3b5f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544c4ad0 0x5564544d0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:49606/0 (socket says 172.21.15.119:49606) 2023-12-07T10:34:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.627+0000 7fc3b5f9b700 1 -- 172.21.15.119:0/775166272 learned_addr learned my addr 172.21.15.119:0/775166272 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:34:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.627+0000 7fc3b579a700 1 --2- 172.21.15.119:0/775166272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564544c9a60 0x5564544cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.628+0000 7fc3b5f9b700 1 -- 172.21.15.119:0/775166272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564544ca380 msgr2=0x5564544cdcb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-07T10:34:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.628+0000 7fc3b5f9b700 1 --2- 172.21.15.119:0/775166272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564544ca380 0x5564544cdcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.628+0000 7fc3b5f9b700 1 -- 172.21.15.119:0/775166272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564544c9a60 msgr2=0x5564544cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.628+0000 7fc3b5f9b700 1 --2- 172.21.15.119:0/775166272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564544c9a60 0x5564544cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.628+0000 7fc3b5f9b700 1 -- 172.21.15.119:0/775166272 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b01aacd0 con 0x5564544c4ad0 2023-12-07T10:34:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.629+0000 7fc3b5f9b700 1 --2- 172.21.15.119:0/775166272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544c4ad0 0x5564544d0340 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac0a1b80 tx=0x7fc3ac015c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76d05b5376f7e2c1 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:34:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.629+0000 7fc3997fa700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:34:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.630+0000 7fc3997fa700 1 -- 172.21.15.119:0/775166272 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac096060 con 0x5564544c4ad0 2023-12-07T10:34:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.630+0000 7fc3997fa700 1 -- 172.21.15.119:0/775166272 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc3ac09a420 con 0x5564544c4ad0 2023-12-07T10:34:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.630+0000 7fc3cbaa8b80 1 -- 172.21.15.119:0/775166272 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5564544b76f0 con 0x5564544c4ad0 2023-12-07T10:34:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.631+0000 7fc3997fa700 1 -- 172.21.15.119:0/775166272 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3ac099860 con 0x5564544c4ad0 2023-12-07T10:34:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.631+0000 7fc3cbaa8b80 10 client.27947 fetch_fsmap learned FSMap version 71 2023-12-07T10:34:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.631+0000 7fc3cbaa8b80 1 -- 172.21.15.119:0/775166272 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) v3 -- 0x556454241d80 con 0x5564544c4ad0 2023-12-07T10:34:51.632 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.632+0000 7fc3997fa700 1 -- 172.21.15.119:0/775166272 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5564544b76f0 con 0x5564544c4ad0 2023-12-07T10:34:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.632+0000 7fc3997fa700 1 -- 172.21.15.119:0/775166272 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 71) v1 ==== 7958+0+0 (secure 0 0 0) 0x7fc3ac099f80 con 0x5564544c4ad0 2023-12-07T10:34:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.632+0000 7fc3cbaa8b80 10 client.27947 fetch_fsmap finished waiting for FSMap version 71 2023-12-07T10:34:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.632+0000 7fc3cbaa8b80 10 client.27947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:34:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.632+0000 7fc3cbaa8b80 20 client.27947 populate_metadata read hostname 'smithi119' 2023-12-07T10:34:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.632+0000 7fc3cbaa8b80 1 --2- 172.21.15.119:0/775166272 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5564544dda80 0x5564544dfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.632+0000 7fc3cbaa8b80 4 client.27947 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:34:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.632+0000 7fc3cbaa8b80 1 -- 172.21.15.119:0/775166272 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556454493740 con 0x5564544dda80 2023-12-07T10:34:51.633 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.633+0000 7fc3b579a700 1 --2- 172.21.15.119:0/775166272 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5564544dda80 0x5564544dfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:51.634 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.634+0000 7fc3b579a700 1 --2- 172.21.15.119:0/775166272 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5564544dda80 0x5564544dfea0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ecebbfb11f51efd server_cookie=cb44102e15f79849 in_seq=0 out_seq=0 2023-12-07T10:34:51.634 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.634+0000 7fc3997fa700 10 client.27947 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:34:51.635 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.635+0000 7fc3997fa700 1 -- 172.21.15.119:0/775166272 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556454493740 con 0x5564544dda80 2023-12-07T10:34:51.635 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.636+0000 7fc3997fa700 10 client.27947 handle_command_reply: tid=0 2023-12-07T10:34:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.636+0000 7fc3cbaa8b80 1 client.27947 shutdown 2023-12-07T10:34:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.636+0000 7fc3cbaa8b80 1 -- 172.21.15.119:0/775166272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544c4ad0 msgr2=0x5564544d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.636+0000 7fc3cbaa8b80 1 --2- 172.21.15.119:0/775166272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544c4ad0 0x5564544d0340 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac0a1b80 tx=0x7fc3ac015c70 comp rx=0 tx=0).stop 2023-12-07T10:34:51.636 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.637+0000 7fc3cbaa8b80 1 -- 172.21.15.119:0/775166272 shutdown_connections 2023-12-07T10:34:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.637+0000 7fc3cbaa8b80 1 --2- 172.21.15.119:0/775166272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564544ca380 0x5564544cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.637+0000 7fc3cbaa8b80 1 --2- 172.21.15.119:0/775166272 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5564544dda80 0x5564544dfea0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.637+0000 7fc3cbaa8b80 1 --2- 172.21.15.119:0/775166272 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544c4ad0 0x5564544d0340 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.637+0000 7fc3cbaa8b80 1 --2- 172.21.15.119:0/775166272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564544c9a60 0x5564544cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.637+0000 7fc3cbaa8b80 1 -- 172.21.15.119:0/775166272 >> 172.21.15.119:0/775166272 conn(0x5564543f3300 msgr2=0x5564544b9980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:34:51.637 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.637+0000 7fc3997fa700 0 client.27947 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:34:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.638+0000 7fc3cbaa8b80 1 -- 172.21.15.119:0/775166272 shutdown_connections 2023-12-07T10:34:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.640+0000 7fc3cbaa8b80 1 -- 172.21.15.119:0/775166272 wait complete. 2023-12-07T10:34:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.640+0000 7fc3cbaa8b80 20 client.27947 trim_cache size 0 max 16384 2023-12-07T10:34:51.665 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.665+0000 7fc3cbaa8b80 1 Processor -- start 2023-12-07T10:34:51.665 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.665+0000 7fc3cbaa8b80 1 -- start start 2023-12-07T10:34:51.666 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.666+0000 7fc3cbaa8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564544c4ad0 0x5564544c4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:51.666 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.666+0000 7fc3cbaa8b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564544c9a60 0x5564544d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:51.666 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.666+0000 7fc3cbaa8b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544ca380 0x5564544d5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.666+0000 7fc3cbaa8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564542978c0 con 0x5564544c4ad0 2023-12-07T10:34:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.666+0000 7fc3cbaa8b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564544bb7a0 con 0x5564544ca380 2023-12-07T10:34:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.666+0000 7fc3cbaa8b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564544c04a0 con 0x5564544c9a60 2023-12-07T10:34:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.667+0000 7fc3b5f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564544c4ad0 0x5564544c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.667+0000 7fc3b679c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544ca380 0x5564544d5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.667+0000 7fc3b579a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564544c9a60 0x5564544d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.667+0000 7fc3b5f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564544c4ad0 0x5564544c4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48608/0 (socket says 172.21.15.119:48608) 2023-12-07T10:34:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.667+0000 7fc3b5f9b700 1 -- 172.21.15.119:0/2572840298 learned_addr learned my addr 172.21.15.119:0/2572840298 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:34:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.668+0000 7fc3b5f9b700 1 -- 172.21.15.119:0/2572840298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564544c9a60 msgr2=0x5564544d4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.668+0000 7fc3b5f9b700 1 --2- 172.21.15.119:0/2572840298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564544c9a60 0x5564544d4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:51.668 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.668+0000 7fc3b5f9b700 1 -- 172.21.15.119:0/2572840298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544ca380 msgr2=0x5564544d5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.668+0000 7fc3b5f9b700 1 --2- 172.21.15.119:0/2572840298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544ca380 0x5564544d5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.668+0000 7fc3b5f9b700 1 -- 172.21.15.119:0/2572840298 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55645422ac00 con 0x5564544c4ad0 2023-12-07T10:34:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.668+0000 7fc3b679c700 1 --2- 172.21.15.119:0/2572840298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544ca380 0x5564544d5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:34:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.668+0000 7fc3b579a700 1 --2- 172.21.15.119:0/2572840298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564544c9a60 0x5564544d4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:34:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.668+0000 7fc3b5f9b700 1 --2- 172.21.15.119:0/2572840298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564544c4ad0 0x5564544c4540 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac0095b0 tx=0x7fc3ac0982d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69687feaf1d4322 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:34:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.669+0000 7fc3a4ff9700 1 -- 172.21.15.119:0/2572840298 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac0d0950 con 0x5564544c4ad0 2023-12-07T10:34:51.669 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.669+0000 7fc3a4ff9700 1 -- 172.21.15.119:0/2572840298 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3ac099860 con 0x5564544c4ad0 2023-12-07T10:34:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.669+0000 7fc3a4ff9700 1 -- 172.21.15.119:0/2572840298 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac09de70 con 0x5564544c4ad0 2023-12-07T10:34:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.670+0000 7fc3cbaa8b80 1 -- 172.21.15.119:0/2572840298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564544c4ad0 msgr2=0x5564544c4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:51.670 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.670+0000 7fc3cbaa8b80 1 --2- 172.21.15.119:0/2572840298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564544c4ad0 0x5564544c4540 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac0095b0 tx=0x7fc3ac0982d0 comp rx=0 tx=0).stop 2023-12-07T10:34:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.671+0000 7fc3cbaa8b80 1 -- 172.21.15.119:0/2572840298 shutdown_connections 2023-12-07T10:34:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.671+0000 7fc3cbaa8b80 1 --2- 172.21.15.119:0/2572840298 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544ca380 0x5564544d5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.671+0000 7fc3cbaa8b80 1 --2- 172.21.15.119:0/2572840298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564544c4ad0 0x5564544c4540 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.671+0000 7fc3cbaa8b80 1 --2- 172.21.15.119:0/2572840298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564544c9a60 0x5564544d4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.671+0000 7fc3cbaa8b80 1 -- 172.21.15.119:0/2572840298 >> 172.21.15.119:0/2572840298 conn(0x5564543f3300 msgr2=0x5564544c7820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:34:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.671+0000 7fc3cbaa8b80 1 -- 172.21.15.119:0/2572840298 shutdown_connections 2023-12-07T10:34:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.671+0000 7fc3cbaa8b80 1 -- 172.21.15.119:0/2572840298 wait complete. 2023-12-07T10:34:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.672+0000 7fc3cbaa8b80 1 -- start start 2023-12-07T10:34:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.672+0000 7fc3cbaa8b80 1 Processor -- start 2023-12-07T10:34:51.675 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.675+0000 7fc3cbaa8b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544c4ad0 0x5564544d50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:51.675 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.675+0000 7fc3cbaa8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564544c9a60 0x5564544c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.676+0000 7fc3cbaa8b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564544ca380 0x5564544c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.676+0000 7fc3b5f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544c4ad0 0x5564544d50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.676+0000 7fc3b5f9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544c4ad0 0x5564544d50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:49622/0 (socket says 172.21.15.119:49622) 2023-12-07T10:34:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.676+0000 7fc3cbaa8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564544c6120 con 0x5564544c9a60 2023-12-07T10:34:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.676+0000 7fc3cbaa8b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564544c6290 con 0x5564544c4ad0 2023-12-07T10:34:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.676+0000 7fc3cbaa8b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564544c6450 con 0x5564544ca380 2023-12-07T10:34:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.676+0000 7fc3b679c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564544ca380 0x5564544c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.676+0000 7fc3b5f9b700 1 -- 172.21.15.119:0/1606706024 learned_addr learned my addr 172.21.15.119:0/1606706024 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:34:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.677+0000 7fc3cbaa8b80 10 client.0 _ll_register_callbacks cb 0x5564543b98e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:34:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.677+0000 7fc3b5f9b700 1 -- 172.21.15.119:0/1606706024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564544ca380 msgr2=0x5564544c5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.677+0000 7fc3b5f9b700 1 --2- 172.21.15.119:0/1606706024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564544ca380 0x5564544c5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.677+0000 7fc3b5f9b700 1 -- 172.21.15.119:0/1606706024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564544c9a60 msgr2=0x5564544c55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:34:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.677+0000 7fc3b5f9b700 1 --2- 172.21.15.119:0/1606706024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564544c9a60 0x5564544c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.677+0000 7fc3b5f9b700 1 -- 172.21.15.119:0/1606706024 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556454241d80 con 0x5564544c4ad0 2023-12-07T10:34:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.677+0000 7fc3b679c700 1 --2- 172.21.15.119:0/1606706024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564544ca380 0x5564544c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:34:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.677+0000 7fc3b5f9b700 1 --2- 172.21.15.119:0/1606706024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544c4ad0 0x5564544d50d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac0ca730 tx=0x7fc3ac0d0f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4676c2aba3f356f3 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:34:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.677+0000 7fc3997fa700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:34:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.678+0000 7fc3997fa700 1 -- 172.21.15.119:0/1606706024 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac0d0950 con 0x5564544c4ad0 2023-12-07T10:34:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.678+0000 7fc3997fa700 1 -- 172.21.15.119:0/1606706024 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc3ac09a420 con 0x5564544c4ad0 2023-12-07T10:34:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.678+0000 7fc3cbaa8b80 1 -- 172.21.15.119:0/1606706024 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5564543def70 con 0x5564544c4ad0 2023-12-07T10:34:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.680+0000 7fc3997fa700 1 -- 172.21.15.119:0/1606706024 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3ac09a190 con 0x5564544c4ad0 2023-12-07T10:34:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.680+0000 7fc3cbaa8b80 10 client.27953 fetch_fsmap learned FSMap version 71 2023-12-07T10:34:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.680+0000 7fc3cbaa8b80 1 -- 172.21.15.119:0/1606706024 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) v3 -- 0x5564542681f0 con 0x5564544c4ad0 2023-12-07T10:34:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.681+0000 7fc3997fa700 1 -- 172.21.15.119:0/1606706024 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5564543def70 con 0x5564544c4ad0 2023-12-07T10:34:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.681+0000 7fc3997fa700 1 -- 172.21.15.119:0/1606706024 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 71) v1 ==== 7958+0+0 (secure 0 0 0) 0x7fc3ac0adb00 con 0x5564544c4ad0 2023-12-07T10:34:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.681+0000 7fc3cbaa8b80 10 client.27953 fetch_fsmap finished waiting for FSMap version 71 2023-12-07T10:34:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.681+0000 7fc3cbaa8b80 10 client.27953 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:34:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.681+0000 7fc3cbaa8b80 20 client.27953 populate_metadata read hostname 'smithi119' 2023-12-07T10:34:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.681+0000 7fc3cbaa8b80 1 --2- 172.21.15.119:0/1606706024 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556454534320 0x556454536740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:34:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.681+0000 7fc3cbaa8b80 4 client.27953 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:34:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.681+0000 7fc3cbaa8b80 1 -- 172.21.15.119:0/1606706024 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5564543d1430 con 0x556454534320 2023-12-07T10:34:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.682+0000 7fc3b579a700 1 --2- 172.21.15.119:0/1606706024 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556454534320 0x556454536740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:34:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.682+0000 7fc3b579a700 1 --2- 172.21.15.119:0/1606706024 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556454534320 0x556454536740 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30f2786e39b7cf4d server_cookie=959fb928fbede5df in_seq=0 out_seq=0 2023-12-07T10:34:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:51.682+0000 7fc3997fa700 10 client.27953 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:34:51.968 INFO:tasks.workunit.client.0.smithi119.stdout: 10 37926 34.57 MB/sec warmup 64 sec latency 187.764 ms 2023-12-07T10:34:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.006+0000 7fc3997fa700 1 -- 172.21.15.119:0/1606706024 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5564543d1430 con 0x556454534320 2023-12-07T10:34:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.006+0000 7fc3997fa700 10 client.27953 handle_command_reply: tid=0 2023-12-07T10:34:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.007+0000 7fc3cbaa8b80 1 client.27953 shutdown 2023-12-07T10:34:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.007+0000 7fc3cbaa8b80 1 -- 172.21.15.119:0/1606706024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544c4ad0 msgr2=0x5564544d50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.007+0000 7fc3cbaa8b80 1 --2- 172.21.15.119:0/1606706024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544c4ad0 0x5564544d50d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac0ca730 tx=0x7fc3ac0d0f70 comp rx=0 tx=0).stop 2023-12-07T10:34:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.007+0000 7fc3cbaa8b80 1 -- 172.21.15.119:0/1606706024 shutdown_connections 2023-12-07T10:34:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.007+0000 7fc3cbaa8b80 1 --2- 172.21.15.119:0/1606706024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564544ca380 0x5564544c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.007+0000 7fc3cbaa8b80 1 --2- 172.21.15.119:0/1606706024 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556454534320 0x556454536740 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.007+0000 7fc3997fa700 0 client.27953 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:34:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.008+0000 7fc3cbaa8b80 1 --2- 172.21.15.119:0/1606706024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5564544c4ad0 0x5564544d50d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.008+0000 7fc3cbaa8b80 1 --2- 172.21.15.119:0/1606706024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564544c9a60 0x5564544c55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.008+0000 7fc3cbaa8b80 1 -- 172.21.15.119:0/1606706024 >> 172.21.15.119:0/1606706024 conn(0x5564543f3300 msgr2=0x5564544cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:34:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.008+0000 7fc3cbaa8b80 1 -- 172.21.15.119:0/1606706024 shutdown_connections 2023-12-07T10:34:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.008+0000 7fc3cbaa8b80 1 -- 172.21.15.119:0/1606706024 wait complete. 2023-12-07T10:34:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.008+0000 7fc3cbaa8b80 20 client.27953 trim_cache size 0 max 16384 2023-12-07T10:34:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:34:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:34:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:34:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:34:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.010+0000 7fc3b81ff700 1 -- 172.21.15.119:0/585518900 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fc39c0a1d40 msgr2=0x7fc39c0a4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.010+0000 7fc3b81ff700 1 --2- 172.21.15.119:0/585518900 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fc39c0a1d40 0x7fc39c0a4200 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc3b010dd10 tx=0x7fc3a8019040 comp rx=0 tx=0).stop 2023-12-07T10:34:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.010+0000 7fc3b81ff700 1 -- 172.21.15.119:0/585518900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3b0073a00 msgr2=0x7fc3b010d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:34:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.010+0000 7fc3b81ff700 1 --2- 172.21.15.119:0/585518900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3b0073a00 0x7fc3b010d080 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac007fa0 tx=0x7fc3ac00d3b0 comp rx=0 tx=0).stop 2023-12-07T10:34:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.010+0000 7fc3b81ff700 1 -- 172.21.15.119:0/585518900 shutdown_connections 2023-12-07T10:34:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.010+0000 7fc3b81ff700 1 --2- 172.21.15.119:0/585518900 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fc39c0a1d40 0x7fc39c0a4200 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.010+0000 7fc3b81ff700 1 --2- 172.21.15.119:0/585518900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3b010dd80 0x7fc3b010b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.010+0000 7fc3b81ff700 1 --2- 172.21.15.119:0/585518900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3b0073a00 0x7fc3b010d080 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.010+0000 7fc3b81ff700 1 --2- 172.21.15.119:0/585518900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3b0074320 0x7fc3b010d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:34:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.010+0000 7fc3b81ff700 1 -- 172.21.15.119:0/585518900 >> 172.21.15.119:0/585518900 conn(0x7fc3b00fcfd0 msgr2=0x7fc3b00fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:34:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.011+0000 7fc3b81ff700 1 -- 172.21.15.119:0/585518900 shutdown_connections 2023-12-07T10:34:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:34:52.011+0000 7fc3b81ff700 1 -- 172.21.15.119:0/585518900 wait complete. 2023-12-07T10:34:52.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:52 smithi119 ceph-mon[97908]: pgmap v452: 129 pgs: 129 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 18 MiB/s wr, 1.03k op/s 2023-12-07T10:34:52.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:52 smithi159 ceph-mon[134726]: pgmap v452: 129 pgs: 129 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 18 MiB/s wr, 1.03k op/s 2023-12-07T10:34:52.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:52 smithi153 ceph-mon[133625]: pgmap v452: 129 pgs: 129 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 18 MiB/s wr, 1.03k op/s 2023-12-07T10:34:52.968 INFO:tasks.workunit.client.0.smithi119.stdout: 10 38848 34.77 MB/sec warmup 65 sec latency 173.276 ms 2023-12-07T10:34:53.970 INFO:tasks.workunit.client.0.smithi119.stdout: 10 39718 35.09 MB/sec warmup 66 sec latency 80.310 ms 2023-12-07T10:34:54.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:54 smithi119 ceph-mon[97908]: pgmap v453: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 15 MiB/s wr, 909 op/s 2023-12-07T10:34:54.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:54 smithi159 ceph-mon[134726]: pgmap v453: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 15 MiB/s wr, 909 op/s 2023-12-07T10:34:54.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:54 smithi153 ceph-mon[133625]: pgmap v453: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 15 MiB/s wr, 909 op/s 2023-12-07T10:34:54.969 INFO:tasks.workunit.client.0.smithi119.stdout: 10 40483 35.25 MB/sec warmup 67 sec latency 52.428 ms 2023-12-07T10:34:55.969 INFO:tasks.workunit.client.0.smithi119.stdout: 10 41150 35.14 MB/sec warmup 68 sec latency 52.031 ms 2023-12-07T10:34:56.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:56 smithi119 ceph-mon[97908]: pgmap v454: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 12 MiB/s wr, 778 op/s 2023-12-07T10:34:56.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:56 smithi159 ceph-mon[134726]: pgmap v454: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 12 MiB/s wr, 778 op/s 2023-12-07T10:34:56.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:56 smithi153 ceph-mon[133625]: pgmap v454: 129 pgs: 129 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 12 MiB/s wr, 778 op/s 2023-12-07T10:34:56.969 INFO:tasks.workunit.client.0.smithi119.stdout: 10 42026 35.28 MB/sec warmup 69 sec latency 51.692 ms 2023-12-07T10:34:57.484 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:57 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:34:57.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:57 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:34:57.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:57 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:34:57.969 INFO:tasks.workunit.client.0.smithi119.stdout: 10 43021 35.60 MB/sec warmup 70 sec latency 60.233 ms 2023-12-07T10:34:58.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:58 smithi159 ceph-mon[134726]: pgmap v455: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 1.11k op/s 2023-12-07T10:34:58.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:58 smithi153 ceph-mon[133625]: pgmap v455: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 1.11k op/s 2023-12-07T10:34:58.969 INFO:tasks.workunit.client.0.smithi119.stdout: 10 43175 35.30 MB/sec warmup 71 sec latency 630.455 ms 2023-12-07T10:34:59.077 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:58 smithi119 ceph-mon[97908]: pgmap v455: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 1.11k op/s 2023-12-07T10:34:59.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:59 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:34:59.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:59 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:34:59.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:34:59 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:34:59.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:59 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:34:59.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:59 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:34:59.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:34:59 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:34:59.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:59 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:34:59.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:59 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:34:59.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:34:59 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:34:59.969 INFO:tasks.workunit.client.0.smithi119.stdout: 10 43361 34.90 MB/sec warmup 72 sec latency 1056.597 ms 2023-12-07T10:35:00.727 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:00 smithi119 ceph-mon[97908]: pgmap v456: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.9 MiB/s wr, 746 op/s 2023-12-07T10:35:00.761 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:00 smithi159 ceph-mon[134726]: pgmap v456: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.9 MiB/s wr, 746 op/s 2023-12-07T10:35:00.778 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:00 smithi153 ceph-mon[133625]: pgmap v456: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.9 MiB/s wr, 746 op/s 2023-12-07T10:35:00.969 INFO:tasks.workunit.client.0.smithi119.stdout: 10 43640 34.68 MB/sec warmup 73 sec latency 153.805 ms 2023-12-07T10:35:01.969 INFO:tasks.workunit.client.0.smithi119.stdout: 10 43829 34.33 MB/sec warmup 74 sec latency 595.394 ms 2023-12-07T10:35:02.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:02 smithi119 ceph-mon[97908]: pgmap v457: 129 pgs: 129 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 955 op/s 2023-12-07T10:35:02.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:02 smithi159 ceph-mon[134726]: pgmap v457: 129 pgs: 129 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 955 op/s 2023-12-07T10:35:02.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:02 smithi153 ceph-mon[133625]: pgmap v457: 129 pgs: 129 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 955 op/s 2023-12-07T10:35:02.969 INFO:tasks.workunit.client.0.smithi119.stdout: 10 44010 34.01 MB/sec warmup 75 sec latency 1144.239 ms 2023-12-07T10:35:03.969 INFO:tasks.workunit.client.0.smithi119.stdout: 10 44552 33.87 MB/sec warmup 76 sec latency 91.880 ms 2023-12-07T10:35:04.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:04 smithi119 ceph-mon[97908]: pgmap v458: 129 pgs: 129 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 864 op/s 2023-12-07T10:35:04.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:04 smithi159 ceph-mon[134726]: pgmap v458: 129 pgs: 129 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 864 op/s 2023-12-07T10:35:04.527 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:04 smithi153 ceph-mon[133625]: pgmap v458: 129 pgs: 129 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 864 op/s 2023-12-07T10:35:04.969 INFO:tasks.workunit.client.0.smithi119.stdout: 10 44863 33.57 MB/sec warmup 77 sec latency 167.173 ms 2023-12-07T10:35:05.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:05 smithi119 ceph-mon[97908]: osdmap e111: 12 total, 12 up, 12 in 2023-12-07T10:35:05.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:05 smithi159 ceph-mon[134726]: osdmap e111: 12 total, 12 up, 12 in 2023-12-07T10:35:05.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:05 smithi153 ceph-mon[133625]: osdmap e111: 12 total, 12 up, 12 in 2023-12-07T10:35:05.969 INFO:tasks.workunit.client.0.smithi119.stdout: 10 45248 33.42 MB/sec warmup 78 sec latency 382.445 ms 2023-12-07T10:35:06.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:06 smithi159 ceph-mon[134726]: pgmap v460: 129 pgs: 129 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 835 op/s 2023-12-07T10:35:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.522+0000 7f1e2159a700 1 -- 172.21.15.119:0/1467851886 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e14003a90 con 0x7f1e24101520 2023-12-07T10:35:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.523+0000 7f1e28dff700 1 -- 172.21.15.119:0/1467851886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e24101520 msgr2=0x7f1e241019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.523+0000 7f1e28dff700 1 --2- 172.21.15.119:0/1467851886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e24101520 0x7f1e241019a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1e140099c0 tx=0x7f1e14009cd0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.523+0000 7f1e28dff700 1 -- 172.21.15.119:0/1467851886 shutdown_connections 2023-12-07T10:35:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.523+0000 7f1e28dff700 1 --2- 172.21.15.119:0/1467851886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e24101ee0 0x7f1e2410e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.523+0000 7f1e28dff700 1 --2- 172.21.15.119:0/1467851886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e241075e0 0x7f1e241079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.523+0000 7f1e28dff700 1 --2- 172.21.15.119:0/1467851886 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e24101520 0x7f1e241019a0 secure :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1e140099c0 tx=0x7f1e14009cd0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.523+0000 7f1e28dff700 1 -- 172.21.15.119:0/1467851886 >> 172.21.15.119:0/1467851886 conn(0x7f1e240fd010 msgr2=0x7f1e240ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:35:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.524+0000 7f1e28dff700 1 -- 172.21.15.119:0/1467851886 shutdown_connections 2023-12-07T10:35:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.524+0000 7f1e28dff700 1 -- 172.21.15.119:0/1467851886 wait complete. 2023-12-07T10:35:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.525+0000 7f1e28dff700 1 Processor -- start 2023-12-07T10:35:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.525+0000 7f1e28dff700 1 -- start start 2023-12-07T10:35:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.525+0000 7f1e28dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e24101ee0 0x7f1e2410b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.525+0000 7f1e28dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e241075e0 0x7f1e2410bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.525+0000 7f1e28dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e2410c240 0x7f1e241aa160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.525+0000 7f1e28dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e241aa760 con 0x7f1e241075e0 2023-12-07T10:35:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.525+0000 7f1e28dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e241aa8d0 con 0x7f1e24101ee0 2023-12-07T10:35:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.525+0000 7f1e28dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e241aaa90 con 0x7f1e2410c240 2023-12-07T10:35:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.525+0000 7f1e2259c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e24101ee0 0x7f1e2410b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.525+0000 7f1e2259c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e24101ee0 0x7f1e2410b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42918/0 (socket says 172.21.15.119:42918) 2023-12-07T10:35:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.526+0000 7f1e2259c700 1 -- 172.21.15.119:0/4270918091 learned_addr learned my addr 172.21.15.119:0/4270918091 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:35:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.526+0000 7f1e22d9d700 1 --2- 172.21.15.119:0/4270918091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e2410c240 0x7f1e241aa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.526+0000 7f1e21d9b700 1 --2- 172.21.15.119:0/4270918091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e241075e0 0x7f1e2410bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.526+0000 7f1e2259c700 1 -- 172.21.15.119:0/4270918091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e2410c240 msgr2=0x7f1e241aa160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.526+0000 7f1e2259c700 1 --2- 172.21.15.119:0/4270918091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e2410c240 0x7f1e241aa160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.526+0000 7f1e2259c700 1 -- 172.21.15.119:0/4270918091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e241075e0 msgr2=0x7f1e2410bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.526+0000 7f1e2259c700 1 --2- 172.21.15.119:0/4270918091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e241075e0 0x7f1e2410bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.526+0000 7f1e2259c700 1 -- 172.21.15.119:0/4270918091 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c009520 con 0x7f1e24101ee0 2023-12-07T10:35:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.526+0000 7f1e2259c700 1 --2- 172.21.15.119:0/4270918091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e24101ee0 0x7f1e2410b7c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00eb30 tx=0x7f1e0c00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:35:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.527+0000 7f1e1b7fe700 1 -- 172.21.15.119:0/4270918091 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c009880 con 0x7f1e24101ee0 2023-12-07T10:35:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.527+0000 7f1e1b7fe700 1 -- 172.21.15.119:0/4270918091 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e0c0044e0 con 0x7f1e24101ee0 2023-12-07T10:35:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.527+0000 7f1e1b7fe700 1 -- 172.21.15.119:0/4270918091 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c003e90 con 0x7f1e24101ee0 2023-12-07T10:35:06.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:06 smithi153 ceph-mon[133625]: pgmap v460: 129 pgs: 129 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 835 op/s 2023-12-07T10:35:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.528+0000 7f1e28dff700 1 -- 172.21.15.119:0/4270918091 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e140095f0 con 0x7f1e24101ee0 2023-12-07T10:35:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.528+0000 7f1e28dff700 1 -- 172.21.15.119:0/4270918091 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e241ab070 con 0x7f1e24101ee0 2023-12-07T10:35:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.533+0000 7f1e1b7fe700 1 -- 172.21.15.119:0/4270918091 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f1e0c004000 con 0x7f1e24101ee0 2023-12-07T10:35:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.535+0000 7f1e3c6acb80 1 Processor -- start 2023-12-07T10:35:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.535+0000 7f1e3c6acb80 1 -- start start 2023-12-07T10:35:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.536+0000 7f1e3c6acb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a854a60 0x55974a854e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.536+0000 7f1e3c6acb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974a855380 0x55974a84f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.536+0000 7f1e3c6acb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55974a84fad0 0x55974a84ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.536+0000 7f1e3c6acb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974a7c3760 con 0x55974a855380 2023-12-07T10:35:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.536+0000 7f1e3c6acb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974a7aa740 con 0x55974a854a60 2023-12-07T10:35:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.536+0000 7f1e3c6acb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974a7a8060 con 0x55974a84fad0 2023-12-07T10:35:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.536+0000 7f1e22d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a854a60 0x55974a854e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.536+0000 7f1e22d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a854a60 0x55974a854e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42928/0 (socket says 172.21.15.119:42928) 2023-12-07T10:35:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.536+0000 7f1e22d9d700 1 -- 172.21.15.119:0/3694942598 learned_addr learned my addr 172.21.15.119:0/3694942598 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:35:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.536+0000 7f1e2259c700 1 --2- 172.21.15.119:0/3694942598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974a855380 0x55974a84f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.537+0000 7f1e22d9d700 1 -- 172.21.15.119:0/3694942598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55974a84fad0 msgr2=0x55974a84ff30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-07T10:35:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.537+0000 7f1e22d9d700 1 --2- 172.21.15.119:0/3694942598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55974a84fad0 0x55974a84ff30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.537+0000 7f1e22d9d700 1 -- 172.21.15.119:0/3694942598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974a855380 msgr2=0x55974a84f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.537+0000 7f1e22d9d700 1 --2- 172.21.15.119:0/3694942598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974a855380 0x55974a84f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.537+0000 7f1e22d9d700 1 -- 172.21.15.119:0/3694942598 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55974a6205d0 con 0x55974a854a60 2023-12-07T10:35:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.537+0000 7f1e1b7fe700 1 --2- 172.21.15.119:0/4270918091 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f1e100a1d00 0x7f1e100a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.537+0000 7f1e1b7fe700 1 -- 172.21.15.119:0/4270918091 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f1e0c014070 con 0x7f1e24101ee0 2023-12-07T10:35:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.538+0000 7f1e2259c700 1 --2- 172.21.15.119:0/4270918091 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f1e100a1d00 0x7f1e100a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.538+0000 7f1e22d9d700 1 --2- 172.21.15.119:0/3694942598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a854a60 0x55974a854e40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c011680 tx=0x7f1e1c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=701a4cb75cddcec7 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:35:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.538+0000 7f1e18ff9700 1 -- 172.21.15.119:0/3694942598 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c00e040 con 0x55974a854a60 2023-12-07T10:35:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.538+0000 7f1e18ff9700 1 -- 172.21.15.119:0/3694942598 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e1c00f040 con 0x55974a854a60 2023-12-07T10:35:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.538+0000 7f1e18ff9700 1 -- 172.21.15.119:0/3694942598 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c007500 con 0x55974a854a60 2023-12-07T10:35:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.539+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/3694942598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a854a60 msgr2=0x55974a854e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.539+0000 7f1e3c6acb80 1 --2- 172.21.15.119:0/3694942598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a854a60 0x55974a854e40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c011680 tx=0x7f1e1c009920 comp rx=0 tx=0).stop 2023-12-07T10:35:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.539+0000 7f1e2259c700 1 --2- 172.21.15.119:0/4270918091 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f1e100a1d00 0x7f1e100a41c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x55974a84b450 tx=0x7f1e0c003930 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:35:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.539+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/3694942598 shutdown_connections 2023-12-07T10:35:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.539+0000 7f1e3c6acb80 1 --2- 172.21.15.119:0/3694942598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55974a84fad0 0x55974a84ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.539+0000 7f1e3c6acb80 1 --2- 172.21.15.119:0/3694942598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a854a60 0x55974a854e40 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.541 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.539+0000 7f1e3c6acb80 1 --2- 172.21.15.119:0/3694942598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974a855380 0x55974a84f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.541 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.539+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/3694942598 >> 172.21.15.119:0/3694942598 conn(0x55974a77e300 msgr2=0x55974a849b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:35:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.540+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/3694942598 shutdown_connections 2023-12-07T10:35:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.540+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/3694942598 wait complete. 2023-12-07T10:35:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.541+0000 7f1e3c6acb80 1 -- start start 2023-12-07T10:35:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.541+0000 7f1e3c6acb80 1 Processor -- start 2023-12-07T10:35:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.542+0000 7f1e3c6acb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a84fad0 0x55974a85b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.542+0000 7f1e3c6acb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974a854a60 0x55974a858660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.542+0000 7f1e22d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a84fad0 0x55974a85b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.542+0000 7f1e22d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a84fad0 0x55974a85b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42938/0 (socket says 172.21.15.119:42938) 2023-12-07T10:35:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.542+0000 7f1e22d9d700 1 -- 172.21.15.119:0/2644673028 learned_addr learned my addr 172.21.15.119:0/2644673028 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:35:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.542+0000 7f1e3c6acb80 1 --2- 172.21.15.119:0/2644673028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55974a855380 0x55974a858da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.542+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/2644673028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974a7a3950 con 0x55974a854a60 2023-12-07T10:35:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.542+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/2644673028 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974a75d500 con 0x55974a84fad0 2023-12-07T10:35:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.542+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/2644673028 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974a75d1a0 con 0x55974a855380 2023-12-07T10:35:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.542+0000 7f1e3c6acb80 10 client.0 _ll_register_callbacks cb 0x55974a7448e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:35:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.543+0000 7f1e22d9d700 1 --2- 172.21.15.119:0/2644673028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55974a855380 0x55974a858da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.544+0000 7f1e21d9b700 1 --2- 172.21.15.119:0/2644673028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974a854a60 0x55974a858660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:06.544 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.544+0000 7f1e22d9d700 1 -- 172.21.15.119:0/2644673028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55974a855380 msgr2=0x55974a858da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:06.544 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.544+0000 7f1e22d9d700 1 --2- 172.21.15.119:0/2644673028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55974a855380 0x55974a858da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.544 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.544+0000 7f1e22d9d700 1 -- 172.21.15.119:0/2644673028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974a854a60 msgr2=0x55974a858660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:06.545 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.546+0000 7f1e22d9d700 1 --2- 172.21.15.119:0/2644673028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974a854a60 0x55974a858660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.545 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.546+0000 7f1e22d9d700 1 -- 172.21.15.119:0/2644673028 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1c011330 con 0x55974a84fad0 2023-12-07T10:35:06.546 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.546+0000 7f1e21d9b700 1 --2- 172.21.15.119:0/2644673028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974a854a60 0x55974a858660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:35:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.546+0000 7f1e22d9d700 1 --2- 172.21.15.119:0/2644673028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a84fad0 0x55974a85b390 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c000f80 tx=0x7f1e1c0043e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76ca16ecc027a99a server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:35:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.546+0000 7f1e09ffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:35:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.547+0000 7f1e09ffb700 1 -- 172.21.15.119:0/2644673028 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c026070 con 0x55974a84fad0 2023-12-07T10:35:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.547+0000 7f1e09ffb700 1 -- 172.21.15.119:0/2644673028 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f1e1c0093a0 con 0x55974a84fad0 2023-12-07T10:35:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.547+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/2644673028 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55974a8426f0 con 0x55974a84fad0 2023-12-07T10:35:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.548+0000 7f1e09ffb700 1 -- 172.21.15.119:0/2644673028 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e1c00e070 con 0x55974a84fad0 2023-12-07T10:35:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.548+0000 7f1e3c6acb80 10 client.27965 fetch_fsmap learned FSMap version 71 2023-12-07T10:35:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.548+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/2644673028 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) v3 -- 0x55974a5ccd80 con 0x55974a84fad0 2023-12-07T10:35:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.548+0000 7f1e09ffb700 1 -- 172.21.15.119:0/2644673028 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55974a8426f0 con 0x55974a84fad0 2023-12-07T10:35:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.549+0000 7f1e09ffb700 1 -- 172.21.15.119:0/2644673028 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 71) v1 ==== 7958+0+0 (secure 0 0 0) 0x7f1e1c029030 con 0x55974a84fad0 2023-12-07T10:35:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.549+0000 7f1e3c6acb80 10 client.27965 fetch_fsmap finished waiting for FSMap version 71 2023-12-07T10:35:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.549+0000 7f1e3c6acb80 10 client.27965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:35:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.549+0000 7f1e3c6acb80 20 client.27965 populate_metadata read hostname 'smithi119' 2023-12-07T10:35:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.549+0000 7f1e3c6acb80 1 --2- 172.21.15.119:0/2644673028 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55974a870050 0x55974a872470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.549+0000 7f1e3c6acb80 4 client.27965 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:35:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.549+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/2644673028 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55974a81e740 con 0x55974a870050 2023-12-07T10:35:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.549+0000 7f1e21d9b700 1 --2- 172.21.15.119:0/2644673028 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55974a870050 0x55974a872470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.551+0000 7f1e21d9b700 1 --2- 172.21.15.119:0/2644673028 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55974a870050 0x55974a872470 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70139722ec025453 server_cookie=e4abdbfa8b42829b in_seq=0 out_seq=0 2023-12-07T10:35:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.551+0000 7f1e09ffb700 10 client.27965 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:35:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.552+0000 7f1e09ffb700 1 -- 172.21.15.119:0/2644673028 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55974a81e740 con 0x55974a870050 2023-12-07T10:35:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.552+0000 7f1e09ffb700 10 client.27965 handle_command_reply: tid=0 2023-12-07T10:35:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.552+0000 7f1e3c6acb80 1 client.27965 shutdown 2023-12-07T10:35:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.554+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/2644673028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a84fad0 msgr2=0x55974a85b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.554+0000 7f1e3c6acb80 1 --2- 172.21.15.119:0/2644673028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a84fad0 0x55974a85b390 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c000f80 tx=0x7f1e1c0043e0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.554+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/2644673028 shutdown_connections 2023-12-07T10:35:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.554+0000 7f1e3c6acb80 1 --2- 172.21.15.119:0/2644673028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55974a855380 0x55974a858da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.554+0000 7f1e3c6acb80 1 --2- 172.21.15.119:0/2644673028 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55974a870050 0x55974a872470 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.554+0000 7f1e3c6acb80 1 --2- 172.21.15.119:0/2644673028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a84fad0 0x55974a85b390 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.554+0000 7f1e3c6acb80 1 --2- 172.21.15.119:0/2644673028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974a854a60 0x55974a858660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.554+0000 7f1e09ffb700 0 client.27965 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:35:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.554+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/2644673028 >> 172.21.15.119:0/2644673028 conn(0x55974a77e300 msgr2=0x55974a844980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:35:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.555+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/2644673028 shutdown_connections 2023-12-07T10:35:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.555+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/2644673028 wait complete. 2023-12-07T10:35:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.555+0000 7f1e3c6acb80 20 client.27965 trim_cache size 0 max 16384 2023-12-07T10:35:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.588+0000 7f1e3c6acb80 1 Processor -- start 2023-12-07T10:35:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.588+0000 7f1e3c6acb80 1 -- start start 2023-12-07T10:35:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.589+0000 7f1e3c6acb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974a84fad0 0x55974a858620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.589+0000 7f1e3c6acb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a854a60 0x55974a84f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.589+0000 7f1e3c6acb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55974a855380 0x55974a857f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.589+0000 7f1e3c6acb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974a6228c0 con 0x55974a84fad0 2023-12-07T10:35:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.589+0000 7f1e3c6acb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974a8584b0 con 0x55974a854a60 2023-12-07T10:35:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.589+0000 7f1e3c6acb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974a859c80 con 0x55974a855380 2023-12-07T10:35:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.590+0000 7f1e21d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a854a60 0x55974a84f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.590+0000 7f1e21d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a854a60 0x55974a84f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42940/0 (socket says 172.21.15.119:42940) 2023-12-07T10:35:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.590+0000 7f1e21d9b700 1 -- 172.21.15.119:0/4104364145 learned_addr learned my addr 172.21.15.119:0/4104364145 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:35:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.590+0000 7f1e22d9d700 1 --2- 172.21.15.119:0/4104364145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974a84fad0 0x55974a858620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.591+0000 7f1e22d9d700 1 --2- 172.21.15.119:0/4104364145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55974a855380 0x55974a857f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.591+0000 7f1e21d9b700 1 -- 172.21.15.119:0/4104364145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55974a855380 msgr2=0x55974a857f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.591+0000 7f1e21d9b700 1 --2- 172.21.15.119:0/4104364145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55974a855380 0x55974a857f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.591+0000 7f1e21d9b700 1 -- 172.21.15.119:0/4104364145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974a84fad0 msgr2=0x55974a858620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.591+0000 7f1e21d9b700 1 --2- 172.21.15.119:0/4104364145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974a84fad0 0x55974a858620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.591+0000 7f1e21d9b700 1 -- 172.21.15.119:0/4104364145 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55974a5b5c00 con 0x55974a854a60 2023-12-07T10:35:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.591+0000 7f1e22d9d700 1 --2- 172.21.15.119:0/4104364145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55974a855380 0x55974a857f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:35:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.591+0000 7f1e21d9b700 1 --2- 172.21.15.119:0/4104364145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a854a60 0x55974a84f540 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009990 tx=0x7f1e14009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b6f4dd6230203c5 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:35:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.592+0000 7f1e197fa700 1 -- 172.21.15.119:0/4104364145 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e14005680 con 0x55974a854a60 2023-12-07T10:35:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.592+0000 7f1e197fa700 1 -- 172.21.15.119:0/4104364145 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e1402b070 con 0x55974a854a60 2023-12-07T10:35:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.592+0000 7f1e197fa700 1 -- 172.21.15.119:0/4104364145 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1400bda0 con 0x55974a854a60 2023-12-07T10:35:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.593+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/4104364145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a854a60 msgr2=0x55974a84f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.593+0000 7f1e3c6acb80 1 --2- 172.21.15.119:0/4104364145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a854a60 0x55974a84f540 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009990 tx=0x7f1e14009cd0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.593+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/4104364145 shutdown_connections 2023-12-07T10:35:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.593+0000 7f1e3c6acb80 1 --2- 172.21.15.119:0/4104364145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55974a855380 0x55974a857f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.593+0000 7f1e3c6acb80 1 --2- 172.21.15.119:0/4104364145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974a84fad0 0x55974a858620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.593+0000 7f1e3c6acb80 1 --2- 172.21.15.119:0/4104364145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a854a60 0x55974a84f540 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.593+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/4104364145 >> 172.21.15.119:0/4104364145 conn(0x55974a77e300 msgr2=0x55974a85b050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:35:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.593+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/4104364145 shutdown_connections 2023-12-07T10:35:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.594+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/4104364145 wait complete. 2023-12-07T10:35:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.594+0000 7f1e3c6acb80 1 -- start start 2023-12-07T10:35:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.595+0000 7f1e3c6acb80 1 Processor -- start 2023-12-07T10:35:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.595+0000 7f1e3c6acb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a84fad0 0x55974a880840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.595+0000 7f1e3c6acb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974a854a60 0x55974a851ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.595+0000 7f1e3c6acb80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55974a855380 0x55974a8505c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.595+0000 7f1e3c6acb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974a8523e0 con 0x55974a854a60 2023-12-07T10:35:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.595+0000 7f1e3c6acb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974a850b00 con 0x55974a84fad0 2023-12-07T10:35:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.595+0000 7f1e3c6acb80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55974a850cc0 con 0x55974a855380 2023-12-07T10:35:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.595+0000 7f1e3c6acb80 10 client.0 _ll_register_callbacks cb 0x55974a7448e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:35:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.595+0000 7f1e22d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a84fad0 0x55974a880840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.595+0000 7f1e22d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a84fad0 0x55974a880840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:42942/0 (socket says 172.21.15.119:42942) 2023-12-07T10:35:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.595+0000 7f1e22d9d700 1 -- 172.21.15.119:0/3158732157 learned_addr learned my addr 172.21.15.119:0/3158732157 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:35:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.596+0000 7f1e22d9d700 1 --2- 172.21.15.119:0/3158732157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55974a855380 0x55974a8505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.596+0000 7f1e21d9b700 1 --2- 172.21.15.119:0/3158732157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974a854a60 0x55974a851ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.596+0000 7f1e22d9d700 1 -- 172.21.15.119:0/3158732157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55974a855380 msgr2=0x55974a8505c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.596+0000 7f1e22d9d700 1 --2- 172.21.15.119:0/3158732157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55974a855380 0x55974a8505c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.596+0000 7f1e22d9d700 1 -- 172.21.15.119:0/3158732157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974a854a60 msgr2=0x55974a851ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.596+0000 7f1e22d9d700 1 --2- 172.21.15.119:0/3158732157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974a854a60 0x55974a851ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.596+0000 7f1e22d9d700 1 -- 172.21.15.119:0/3158732157 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e2410eca0 con 0x55974a84fad0 2023-12-07T10:35:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.596+0000 7f1e21d9b700 1 --2- 172.21.15.119:0/3158732157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974a854a60 0x55974a851ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:35:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.596+0000 7f1e22d9d700 1 --2- 172.21.15.119:0/3158732157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a84fad0 0x55974a880840 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c023040 tx=0x7f1e1c02d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=272f3f1c00f43d96 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:35:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.596+0000 7f1e09ffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:35:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.597+0000 7f1e09ffb700 1 -- 172.21.15.119:0/3158732157 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c00e030 con 0x55974a84fad0 2023-12-07T10:35:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.597+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/3158732157 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e1c00f040 con 0x55974a84fad0 2023-12-07T10:35:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.597+0000 7f1e09ffb700 1 -- 172.21.15.119:0/3158732157 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f1e1c0093a0 con 0x55974a84fad0 2023-12-07T10:35:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.598+0000 7f1e09ffb700 1 -- 172.21.15.119:0/3158732157 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e1c026040 con 0x55974a84fad0 2023-12-07T10:35:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.598+0000 7f1e3c6acb80 10 client.27971 fetch_fsmap learned FSMap version 71 2023-12-07T10:35:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.598+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/3158732157 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) v3 -- 0x55974a5f31f0 con 0x55974a84fad0 2023-12-07T10:35:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.598+0000 7f1e09ffb700 1 -- 172.21.15.119:0/3158732157 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e1c00f040 con 0x55974a84fad0 2023-12-07T10:35:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.599+0000 7f1e09ffb700 1 -- 172.21.15.119:0/3158732157 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 71) v1 ==== 7958+0+0 (secure 0 0 0) 0x7f1e1c029050 con 0x55974a84fad0 2023-12-07T10:35:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.599+0000 7f1e3c6acb80 10 client.27971 fetch_fsmap finished waiting for FSMap version 71 2023-12-07T10:35:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.599+0000 7f1e3c6acb80 10 client.27971 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:35:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.599+0000 7f1e3c6acb80 20 client.27971 populate_metadata read hostname 'smithi119' 2023-12-07T10:35:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.599+0000 7f1e3c6acb80 1 --2- 172.21.15.119:0/3158732157 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55974a8f27a0 0x55974a8f4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.599+0000 7f1e3c6acb80 4 client.27971 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:35:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.599+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/3158732157 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55974a769f70 con 0x55974a8f27a0 2023-12-07T10:35:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.599+0000 7f1e21d9b700 1 --2- 172.21.15.119:0/3158732157 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55974a8f27a0 0x55974a8f4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.600+0000 7f1e09ffb700 10 client.27971 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:35:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.600+0000 7f1e21d9b700 1 --2- 172.21.15.119:0/3158732157 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55974a8f27a0 0x55974a8f4bc0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3144b14c64c8a04 server_cookie=f5dd68075e4140f in_seq=0 out_seq=0 2023-12-07T10:35:06.727 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:06 smithi119 ceph-mon[97908]: pgmap v460: 129 pgs: 129 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 835 op/s 2023-12-07T10:35:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.743+0000 7f1e09ffb700 1 -- 172.21.15.119:0/3158732157 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55974a769f70 con 0x55974a8f27a0 2023-12-07T10:35:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.743+0000 7f1e09ffb700 10 client.27971 handle_command_reply: tid=0 2023-12-07T10:35:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.743+0000 7f1e3c6acb80 1 client.27971 shutdown 2023-12-07T10:35:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.744+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/3158732157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a84fad0 msgr2=0x55974a880840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.744+0000 7f1e3c6acb80 1 --2- 172.21.15.119:0/3158732157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a84fad0 0x55974a880840 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c023040 tx=0x7f1e1c02d040 comp rx=0 tx=0).stop 2023-12-07T10:35:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.744+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/3158732157 shutdown_connections 2023-12-07T10:35:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.744+0000 7f1e3c6acb80 1 --2- 172.21.15.119:0/3158732157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55974a855380 0x55974a8505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.744+0000 7f1e3c6acb80 1 --2- 172.21.15.119:0/3158732157 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55974a8f27a0 0x55974a8f4bc0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.744+0000 7f1e3c6acb80 1 --2- 172.21.15.119:0/3158732157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55974a84fad0 0x55974a880840 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.744+0000 7f1e3c6acb80 1 --2- 172.21.15.119:0/3158732157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55974a854a60 0x55974a851ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.744+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/3158732157 >> 172.21.15.119:0/3158732157 conn(0x55974a77e300 msgr2=0x55974a852860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:35:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.744+0000 7f1e09ffb700 0 client.27971 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:35:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.744+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/3158732157 shutdown_connections 2023-12-07T10:35:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.744+0000 7f1e3c6acb80 1 -- 172.21.15.119:0/3158732157 wait complete. 2023-12-07T10:35:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.745+0000 7f1e3c6acb80 20 client.27971 trim_cache size 0 max 16384 2023-12-07T10:35:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:35:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:35:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:35:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:35:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.745+0000 7f1e28dff700 1 -- 172.21.15.119:0/4270918091 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f1e100a1d00 msgr2=0x7f1e100a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:06.746 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.745+0000 7f1e28dff700 1 --2- 172.21.15.119:0/4270918091 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f1e100a1d00 0x7f1e100a41c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x55974a84b450 tx=0x7f1e0c003930 comp rx=0 tx=0).stop 2023-12-07T10:35:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.745+0000 7f1e28dff700 1 -- 172.21.15.119:0/4270918091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e24101ee0 msgr2=0x7f1e2410b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.745+0000 7f1e28dff700 1 --2- 172.21.15.119:0/4270918091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e24101ee0 0x7f1e2410b7c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00eb30 tx=0x7f1e0c00ee40 comp rx=0 tx=0).stop 2023-12-07T10:35:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.746+0000 7f1e28dff700 1 -- 172.21.15.119:0/4270918091 shutdown_connections 2023-12-07T10:35:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.746+0000 7f1e28dff700 1 --2- 172.21.15.119:0/4270918091 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f1e100a1d00 0x7f1e100a41c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.746+0000 7f1e28dff700 1 --2- 172.21.15.119:0/4270918091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e2410c240 0x7f1e241aa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.746+0000 7f1e28dff700 1 --2- 172.21.15.119:0/4270918091 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e24101ee0 0x7f1e2410b7c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.746+0000 7f1e28dff700 1 --2- 172.21.15.119:0/4270918091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e241075e0 0x7f1e2410bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.746+0000 7f1e28dff700 1 -- 172.21.15.119:0/4270918091 >> 172.21.15.119:0/4270918091 conn(0x7f1e240fd010 msgr2=0x7f1e240fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:35:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.746+0000 7f1e28dff700 1 -- 172.21.15.119:0/4270918091 shutdown_connections 2023-12-07T10:35:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:06.746+0000 7f1e28dff700 1 -- 172.21.15.119:0/4270918091 wait complete. 2023-12-07T10:35:06.969 INFO:tasks.workunit.client.0.smithi119.stdout: 10 46004 33.54 MB/sec warmup 79 sec latency 106.020 ms 2023-12-07T10:35:07.970 INFO:tasks.workunit.client.0.smithi119.stdout: 10 46843 33.76 MB/sec warmup 80 sec latency 57.167 ms 2023-12-07T10:35:08.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:08 smithi159 ceph-mon[134726]: pgmap v461: 129 pgs: 5 active+clean+snaptrim, 111 active+clean, 13 active+clean+snaptrim_wait; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 17 MiB/s wr, 744 op/s 2023-12-07T10:35:08.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:08 smithi153 ceph-mon[133625]: pgmap v461: 129 pgs: 5 active+clean+snaptrim, 111 active+clean, 13 active+clean+snaptrim_wait; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 17 MiB/s wr, 744 op/s 2023-12-07T10:35:08.727 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:08 smithi119 ceph-mon[97908]: pgmap v461: 129 pgs: 5 active+clean+snaptrim, 111 active+clean, 13 active+clean+snaptrim_wait; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 17 MiB/s wr, 744 op/s 2023-12-07T10:35:08.970 INFO:tasks.workunit.client.0.smithi119.stdout: 10 47363 33.69 MB/sec warmup 81 sec latency 69.343 ms 2023-12-07T10:35:09.971 INFO:tasks.workunit.client.0.smithi119.stdout: 10 47955 33.63 MB/sec warmup 82 sec latency 58.882 ms 2023-12-07T10:35:10.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:10 smithi159 ceph-mon[134726]: pgmap v462: 129 pgs: 5 active+clean+snaptrim, 111 active+clean, 13 active+clean+snaptrim_wait; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 17 MiB/s wr, 744 op/s 2023-12-07T10:35:10.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:10 smithi153 ceph-mon[133625]: pgmap v462: 129 pgs: 5 active+clean+snaptrim, 111 active+clean, 13 active+clean+snaptrim_wait; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 17 MiB/s wr, 744 op/s 2023-12-07T10:35:10.727 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:10 smithi119 ceph-mon[97908]: pgmap v462: 129 pgs: 5 active+clean+snaptrim, 111 active+clean, 13 active+clean+snaptrim_wait; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 17 MiB/s wr, 744 op/s 2023-12-07T10:35:10.970 INFO:tasks.workunit.client.0.smithi119.stdout: 10 49103 33.89 MB/sec warmup 83 sec latency 35.815 ms 2023-12-07T10:35:11.970 INFO:tasks.workunit.client.0.smithi119.stdout: 10 50078 34.32 MB/sec warmup 84 sec latency 75.382 ms 2023-12-07T10:35:12.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:12 smithi119 ceph-mon[97908]: pgmap v463: 129 pgs: 2 active+clean+snaptrim, 120 active+clean, 7 active+clean+snaptrim_wait; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 12 MiB/s wr, 698 op/s 2023-12-07T10:35:12.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:12 smithi159 ceph-mon[134726]: pgmap v463: 129 pgs: 2 active+clean+snaptrim, 120 active+clean, 7 active+clean+snaptrim_wait; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 12 MiB/s wr, 698 op/s 2023-12-07T10:35:12.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:12 smithi153 ceph-mon[133625]: pgmap v463: 129 pgs: 2 active+clean+snaptrim, 120 active+clean, 7 active+clean+snaptrim_wait; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 12 MiB/s wr, 698 op/s 2023-12-07T10:35:12.970 INFO:tasks.workunit.client.0.smithi119.stdout: 10 50612 34.18 MB/sec warmup 85 sec latency 81.499 ms 2023-12-07T10:35:13.970 INFO:tasks.workunit.client.0.smithi119.stdout: 10 51231 34.10 MB/sec warmup 86 sec latency 140.729 ms 2023-12-07T10:35:14.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:14 smithi119 ceph-mon[97908]: pgmap v464: 129 pgs: 129 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 8.6 MiB/s wr, 644 op/s 2023-12-07T10:35:14.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:14 smithi159 ceph-mon[134726]: pgmap v464: 129 pgs: 129 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 8.6 MiB/s wr, 644 op/s 2023-12-07T10:35:14.527 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:14 smithi153 ceph-mon[133625]: pgmap v464: 129 pgs: 129 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 8.6 MiB/s wr, 644 op/s 2023-12-07T10:35:14.970 INFO:tasks.workunit.client.0.smithi119.stdout: 10 52206 34.17 MB/sec warmup 87 sec latency 65.033 ms 2023-12-07T10:35:15.970 INFO:tasks.workunit.client.0.smithi119.stdout: 10 52901 34.40 MB/sec warmup 88 sec latency 145.286 ms 2023-12-07T10:35:16.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:16 smithi119 ceph-mon[97908]: pgmap v465: 129 pgs: 129 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.9 MiB/s wr, 589 op/s 2023-12-07T10:35:16.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:16 smithi119 ceph-mon[97908]: osdmap e112: 12 total, 12 up, 12 in 2023-12-07T10:35:16.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:16 smithi159 ceph-mon[134726]: pgmap v465: 129 pgs: 129 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.9 MiB/s wr, 589 op/s 2023-12-07T10:35:16.516 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:16 smithi159 ceph-mon[134726]: osdmap e112: 12 total, 12 up, 12 in 2023-12-07T10:35:16.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:16 smithi153 ceph-mon[133625]: pgmap v465: 129 pgs: 129 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.9 MiB/s wr, 589 op/s 2023-12-07T10:35:16.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:16 smithi153 ceph-mon[133625]: osdmap e112: 12 total, 12 up, 12 in 2023-12-07T10:35:16.970 INFO:tasks.workunit.client.0.smithi119.stdout: 10 53591 34.49 MB/sec warmup 89 sec latency 119.284 ms 2023-12-07T10:35:17.971 INFO:tasks.workunit.client.0.smithi119.stdout: 10 54280 34.37 MB/sec warmup 90 sec latency 80.844 ms 2023-12-07T10:35:18.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:18 smithi119 ceph-mon[97908]: pgmap v467: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.7 MiB/s wr, 640 op/s 2023-12-07T10:35:18.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:18 smithi159 ceph-mon[134726]: pgmap v467: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.7 MiB/s wr, 640 op/s 2023-12-07T10:35:18.527 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:18 smithi153 ceph-mon[133625]: pgmap v467: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.7 MiB/s wr, 640 op/s 2023-12-07T10:35:18.971 INFO:tasks.workunit.client.0.smithi119.stdout: 10 54925 34.40 MB/sec warmup 91 sec latency 100.344 ms 2023-12-07T10:35:19.971 INFO:tasks.workunit.client.0.smithi119.stdout: 10 56003 34.66 MB/sec warmup 92 sec latency 44.436 ms 2023-12-07T10:35:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.119+0000 7f93c35ff700 1 -- 172.21.15.119:0/1445745884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93bc11adf0 msgr2=0x7f93bc11d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.119+0000 7f93c35ff700 1 --2- 172.21.15.119:0/1445745884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93bc11adf0 0x7f93bc11d1e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f93b400b0a0 tx=0x7f93b400b3b0 comp rx=0 tx=0).stop 2023-12-07T10:35:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.120+0000 7f93c35ff700 1 -- 172.21.15.119:0/1445745884 shutdown_connections 2023-12-07T10:35:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.120+0000 7f93c35ff700 1 --2- 172.21.15.119:0/1445745884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93bc11adf0 0x7f93bc11d1e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.120+0000 7f93c35ff700 1 --2- 172.21.15.119:0/1445745884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93bc10fd30 0x7f93bc108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.120+0000 7f93c35ff700 1 --2- 172.21.15.119:0/1445745884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93bc1101a0 0x7f93bc108b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.120+0000 7f93c35ff700 1 -- 172.21.15.119:0/1445745884 >> 172.21.15.119:0/1445745884 conn(0x7f93bc06c5e0 msgr2=0x7f93bc06c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:35:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.125+0000 7f93c35ff700 1 -- 172.21.15.119:0/1445745884 shutdown_connections 2023-12-07T10:35:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.125+0000 7f93c35ff700 1 -- 172.21.15.119:0/1445745884 wait complete. 2023-12-07T10:35:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.126+0000 7f93c35ff700 1 Processor -- start 2023-12-07T10:35:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.127+0000 7f93c35ff700 1 -- start start 2023-12-07T10:35:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.133+0000 7f93c35ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93bc10fd30 0x7f93bc1ac470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.134+0000 7f93c35ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93bc1101a0 0x7f93bc1ac9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.134+0000 7f93c35ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93bc11adf0 0x7f93bc1a6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.134+0000 7f93c0b9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93bc1101a0 0x7f93bc1ac9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.134+0000 7f93c0b9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93bc1101a0 0x7f93bc1ac9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:52480/0 (socket says 172.21.15.119:52480) 2023-12-07T10:35:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.135+0000 7f93c0b9a700 1 -- 172.21.15.119:0/339051808 learned_addr learned my addr 172.21.15.119:0/339051808 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:35:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.135+0000 7f93c35ff700 1 -- 172.21.15.119:0/339051808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc1a6c60 con 0x7f93bc11adf0 2023-12-07T10:35:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.135+0000 7f93c35ff700 1 -- 172.21.15.119:0/339051808 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc1a6dd0 con 0x7f93bc1101a0 2023-12-07T10:35:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.136+0000 7f93c1b9c700 1 --2- 172.21.15.119:0/339051808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93bc11adf0 0x7f93bc1a6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.136+0000 7f93c35ff700 1 -- 172.21.15.119:0/339051808 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc1a6f90 con 0x7f93bc10fd30 2023-12-07T10:35:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.136+0000 7f93c139b700 1 --2- 172.21.15.119:0/339051808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93bc10fd30 0x7f93bc1ac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:20.137 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.137+0000 7f93c0b9a700 1 -- 172.21.15.119:0/339051808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93bc10fd30 msgr2=0x7f93bc1ac470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:20.137 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.137+0000 7f93c0b9a700 1 --2- 172.21.15.119:0/339051808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93bc10fd30 0x7f93bc1ac470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:20.137 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.137+0000 7f93c0b9a700 1 -- 172.21.15.119:0/339051808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93bc11adf0 msgr2=0x7f93bc1a6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:20.137 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.137+0000 7f93c0b9a700 1 --2- 172.21.15.119:0/339051808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93bc11adf0 0x7f93bc1a6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:20.137 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.137+0000 7f93c0b9a700 1 -- 172.21.15.119:0/339051808 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b4009d00 con 0x7f93bc1101a0 2023-12-07T10:35:20.138 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.138+0000 7f93c139b700 1 --2- 172.21.15.119:0/339051808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93bc10fd30 0x7f93bc1ac470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:35:20.138 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.138+0000 7f93c0b9a700 1 --2- 172.21.15.119:0/339051808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93bc1101a0 0x7f93bc1ac9b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00c890 tx=0x7f93ac00cba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:35:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.138+0000 7f93ba7fc700 1 -- 172.21.15.119:0/339051808 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac004220 con 0x7f93bc1101a0 2023-12-07T10:35:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.139+0000 7f93ba7fc700 1 -- 172.21.15.119:0/339051808 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93ac004cf0 con 0x7f93bc1101a0 2023-12-07T10:35:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.139+0000 7f93ba7fc700 1 -- 172.21.15.119:0/339051808 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac005710 con 0x7f93bc1101a0 2023-12-07T10:35:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.140+0000 7f93c35ff700 1 -- 172.21.15.119:0/339051808 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93bc1a7270 con 0x7f93bc1101a0 2023-12-07T10:35:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.140+0000 7f93c35ff700 1 -- 172.21.15.119:0/339051808 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93bc1b3510 con 0x7f93bc1101a0 2023-12-07T10:35:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.153+0000 7f93ba7fc700 1 -- 172.21.15.119:0/339051808 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f93ac004e60 con 0x7f93bc1101a0 2023-12-07T10:35:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.155+0000 7f93ba7fc700 1 --2- 172.21.15.119:0/339051808 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f93a40a1d80 0x7f93a40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.156+0000 7f93ba7fc700 1 -- 172.21.15.119:0/339051808 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f93ac0cdb40 con 0x7f93bc1101a0 2023-12-07T10:35:20.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.156+0000 7f93c139b700 1 --2- 172.21.15.119:0/339051808 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f93a40a1d80 0x7f93a40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.157+0000 7f93c139b700 1 --2- 172.21.15.119:0/339051808 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f93a40a1d80 0x7f93a40a4240 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f93b0011490 tx=0x7f93b0009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:35:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.207+0000 7f93d6fd9b80 1 Processor -- start 2023-12-07T10:35:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.208+0000 7f93d6fd9b80 1 -- start start 2023-12-07T10:35:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.208+0000 7f93d6fd9b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c28354ba60 0x55c28354be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.208+0000 7f93d6fd9b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c28354c380 0x55c283546540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.208+0000 7f93d6fd9b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c283546ad0 0x55c283546f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.208+0000 7f93d6fd9b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2834ba760 con 0x55c283546ad0 2023-12-07T10:35:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.208+0000 7f93d6fd9b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2834a1740 con 0x55c28354c380 2023-12-07T10:35:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.208+0000 7f93d6fd9b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c28349f060 con 0x55c28354ba60 2023-12-07T10:35:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.209+0000 7f93c0b9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c28354c380 0x55c283546540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.209+0000 7f93c0b9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c28354c380 0x55c283546540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:52488/0 (socket says 172.21.15.119:52488) 2023-12-07T10:35:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.209+0000 7f93c0b9a700 1 -- 172.21.15.119:0/431692773 learned_addr learned my addr 172.21.15.119:0/431692773 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:35:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.210+0000 7f93c139b700 1 --2- 172.21.15.119:0/431692773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c28354ba60 0x55c28354be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.212+0000 7f93c139b700 1 -- 172.21.15.119:0/431692773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c28354c380 msgr2=0x55c283546540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.212+0000 7f93c139b700 1 --2- 172.21.15.119:0/431692773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c28354c380 0x55c283546540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.212+0000 7f93c139b700 1 -- 172.21.15.119:0/431692773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c283546ad0 msgr2=0x55c283546f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.212+0000 7f93c139b700 1 --2- 172.21.15.119:0/431692773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c283546ad0 0x55c283546f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.212+0000 7f93c139b700 1 -- 172.21.15.119:0/431692773 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2833175d0 con 0x55c28354ba60 2023-12-07T10:35:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.213+0000 7f93c139b700 1 --2- 172.21.15.119:0/431692773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c28354ba60 0x55c28354be40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f93b00042f0 tx=0x7f93b0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9193000bcd868856 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:35:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.213+0000 7f93a37fe700 1 -- 172.21.15.119:0/431692773 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0003dd0 con 0x55c28354ba60 2023-12-07T10:35:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.213+0000 7f93a37fe700 1 -- 172.21.15.119:0/431692773 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93b00268c0 con 0x55c28354ba60 2023-12-07T10:35:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.213+0000 7f93a37fe700 1 -- 172.21.15.119:0/431692773 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b001e890 con 0x55c28354ba60 2023-12-07T10:35:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.215+0000 7f93d6fd9b80 1 -- 172.21.15.119:0/431692773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c28354ba60 msgr2=0x55c28354be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.215+0000 7f93d6fd9b80 1 --2- 172.21.15.119:0/431692773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c28354ba60 0x55c28354be40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f93b00042f0 tx=0x7f93b0014000 comp rx=0 tx=0).stop 2023-12-07T10:35:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.221+0000 7f93d6fd9b80 1 -- 172.21.15.119:0/431692773 shutdown_connections 2023-12-07T10:35:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.221+0000 7f93d6fd9b80 1 --2- 172.21.15.119:0/431692773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c283546ad0 0x55c283546f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.221+0000 7f93d6fd9b80 1 --2- 172.21.15.119:0/431692773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c28354ba60 0x55c28354be40 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.221+0000 7f93d6fd9b80 1 --2- 172.21.15.119:0/431692773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c28354c380 0x55c283546540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.221+0000 7f93d6fd9b80 1 -- 172.21.15.119:0/431692773 >> 172.21.15.119:0/431692773 conn(0x55c283475300 msgr2=0x55c283540b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:35:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.221+0000 7f93d6fd9b80 1 -- 172.21.15.119:0/431692773 shutdown_connections 2023-12-07T10:35:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.221+0000 7f93d6fd9b80 1 -- 172.21.15.119:0/431692773 wait complete. 2023-12-07T10:35:20.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.223+0000 7f93d6fd9b80 1 -- start start 2023-12-07T10:35:20.223 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.223+0000 7f93d6fd9b80 1 Processor -- start 2023-12-07T10:35:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.225+0000 7f93d6fd9b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c283546ad0 0x55c28354f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.225+0000 7f93d6fd9b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c28354ba60 0x55c28354fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.225+0000 7f93d6fd9b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c28354c380 0x55c283550220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.225+0000 7f93d6fd9b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c28349a950 con 0x55c283546ad0 2023-12-07T10:35:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.225+0000 7f93d6fd9b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c283454500 con 0x55c28354ba60 2023-12-07T10:35:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.225+0000 7f93d6fd9b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2834541a0 con 0x55c28354c380 2023-12-07T10:35:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.226+0000 7f93d6fd9b80 10 client.0 _ll_register_callbacks cb 0x55c28343b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:35:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.226+0000 7f93c1b9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c28354c380 0x55c283550220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.226+0000 7f93c1b9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c28354c380 0x55c283550220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:48822/0 (socket says 172.21.15.119:48822) 2023-12-07T10:35:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.226+0000 7f93c1b9c700 1 -- 172.21.15.119:0/2739929551 learned_addr learned my addr 172.21.15.119:0/2739929551 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:35:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.227+0000 7f93c1b9c700 1 -- 172.21.15.119:0/2739929551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c28354ba60 msgr2=0x55c28354fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.227+0000 7f93c1b9c700 1 --2- 172.21.15.119:0/2739929551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c28354ba60 0x55c28354fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.227+0000 7f93c1b9c700 1 -- 172.21.15.119:0/2739929551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c283546ad0 msgr2=0x55c28354f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.227+0000 7f93c1b9c700 1 --2- 172.21.15.119:0/2739929551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c283546ad0 0x55c28354f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.227+0000 7f93c1b9c700 1 -- 172.21.15.119:0/2739929551 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b0017070 con 0x55c28354c380 2023-12-07T10:35:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.228+0000 7f93a0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:35:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.228+0000 7f93c1b9c700 1 --2- 172.21.15.119:0/2739929551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c28354c380 0x55c283550220 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f93b400b780 tx=0x7f93b4004d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bd2bacc4082bdef server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:35:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.229+0000 7f93a0ff9700 1 -- 172.21.15.119:0/2739929551 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b400e050 con 0x55c28354c380 2023-12-07T10:35:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.229+0000 7f93a0ff9700 1 -- 172.21.15.119:0/2739929551 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f93b4009450 con 0x55c28354c380 2023-12-07T10:35:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.230+0000 7f93d6fd9b80 1 -- 172.21.15.119:0/2739929551 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2835396f0 con 0x55c28354c380 2023-12-07T10:35:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.231+0000 7f93a0ff9700 1 -- 172.21.15.119:0/2739929551 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93b40078f0 con 0x55c28354c380 2023-12-07T10:35:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.231+0000 7f93d6fd9b80 10 client.27949 fetch_fsmap learned FSMap version 71 2023-12-07T10:35:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.231+0000 7f93d6fd9b80 1 -- 172.21.15.119:0/2739929551 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) v3 -- 0x55c2832c3d80 con 0x55c28354c380 2023-12-07T10:35:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.233+0000 7f93a0ff9700 1 -- 172.21.15.119:0/2739929551 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c2835396f0 con 0x55c28354c380 2023-12-07T10:35:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.234+0000 7f93a0ff9700 1 -- 172.21.15.119:0/2739929551 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 71) v1 ==== 7958+0+0 (secure 0 0 0) 0x7f93b400b0d0 con 0x55c28354c380 2023-12-07T10:35:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.235+0000 7f93d6fd9b80 10 client.27949 fetch_fsmap finished waiting for FSMap version 71 2023-12-07T10:35:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.235+0000 7f93d6fd9b80 10 client.27949 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:35:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.235+0000 7f93d6fd9b80 20 client.27949 populate_metadata read hostname 'smithi119' 2023-12-07T10:35:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.235+0000 7f93d6fd9b80 1 --2- 172.21.15.119:0/2739929551 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c28355ffd0 0x55c2835623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.235+0000 7f93d6fd9b80 4 client.27949 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:35:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.235+0000 7f93d6fd9b80 1 -- 172.21.15.119:0/2739929551 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c283515740 con 0x55c28355ffd0 2023-12-07T10:35:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.237+0000 7f93c139b700 1 --2- 172.21.15.119:0/2739929551 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c28355ffd0 0x55c2835623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.237+0000 7f93c139b700 1 --2- 172.21.15.119:0/2739929551 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c28355ffd0 0x55c2835623f0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2b91be6e2cd10e2 server_cookie=947ced1dd9754887 in_seq=0 out_seq=0 2023-12-07T10:35:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.238+0000 7f93a0ff9700 10 client.27949 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:35:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.248+0000 7f93a0ff9700 1 -- 172.21.15.119:0/2739929551 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c283515740 con 0x55c28355ffd0 2023-12-07T10:35:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.248+0000 7f93a0ff9700 10 client.27949 handle_command_reply: tid=0 2023-12-07T10:35:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.249+0000 7f93d6fd9b80 1 client.27949 shutdown 2023-12-07T10:35:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.250+0000 7f93d6fd9b80 1 -- 172.21.15.119:0/2739929551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c28354c380 msgr2=0x55c283550220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.250+0000 7f93d6fd9b80 1 --2- 172.21.15.119:0/2739929551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c28354c380 0x55c283550220 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f93b400b780 tx=0x7f93b4004d30 comp rx=0 tx=0).stop 2023-12-07T10:35:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.250+0000 7f93d6fd9b80 1 -- 172.21.15.119:0/2739929551 shutdown_connections 2023-12-07T10:35:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.250+0000 7f93d6fd9b80 1 --2- 172.21.15.119:0/2739929551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c28354c380 0x55c283550220 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.250+0000 7f93d6fd9b80 1 --2- 172.21.15.119:0/2739929551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c283546ad0 0x55c28354f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.250+0000 7f93d6fd9b80 1 --2- 172.21.15.119:0/2739929551 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c28355ffd0 0x55c2835623f0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.250+0000 7f93d6fd9b80 1 --2- 172.21.15.119:0/2739929551 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c28354ba60 0x55c28354fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.250+0000 7f93a0ff9700 0 client.27949 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:35:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.250+0000 7f93d6fd9b80 1 -- 172.21.15.119:0/2739929551 >> 172.21.15.119:0/2739929551 conn(0x55c283475300 msgr2=0x55c28353b980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:35:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.251+0000 7f93d6fd9b80 1 -- 172.21.15.119:0/2739929551 shutdown_connections 2023-12-07T10:35:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.251+0000 7f93d6fd9b80 1 -- 172.21.15.119:0/2739929551 wait complete. 2023-12-07T10:35:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.252+0000 7f93d6fd9b80 20 client.27949 trim_cache size 0 max 16384 2023-12-07T10:35:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.285+0000 7f93d6fd9b80 1 Processor -- start 2023-12-07T10:35:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.285+0000 7f93d6fd9b80 1 -- start start 2023-12-07T10:35:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.290+0000 7f93d6fd9b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c283546ad0 0x55c283546540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.290+0000 7f93d6fd9b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c28354ba60 0x55c283566d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.290+0000 7f93d6fd9b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c28354c380 0x55c283549a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.290+0000 7f93d6fd9b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2833198c0 con 0x55c28354ba60 2023-12-07T10:35:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.290+0000 7f93d6fd9b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c28354b6e0 con 0x55c28354c380 2023-12-07T10:35:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.290+0000 7f93d6fd9b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c283550da0 con 0x55c283546ad0 2023-12-07T10:35:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.290+0000 7f93c139b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c283546ad0 0x55c283546540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.291+0000 7f93c139b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c283546ad0 0x55c283546540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:48826/0 (socket says 172.21.15.119:48826) 2023-12-07T10:35:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.291+0000 7f93c1b9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c28354c380 0x55c283549a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.291+0000 7f93c1b9c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c28354c380 0x55c283549a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:52514/0 (socket says 172.21.15.119:52514) 2023-12-07T10:35:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.291+0000 7f93c1b9c700 1 -- 172.21.15.119:0/704583621 learned_addr learned my addr 172.21.15.119:0/704583621 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:35:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.291+0000 7f93c0b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c28354ba60 0x55c283566d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.292+0000 7f93c0b9a700 1 -- 172.21.15.119:0/704583621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c283546ad0 msgr2=0x55c283546540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.293+0000 7f93c0b9a700 1 --2- 172.21.15.119:0/704583621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c283546ad0 0x55c283546540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.293+0000 7f93c0b9a700 1 -- 172.21.15.119:0/704583621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c28354c380 msgr2=0x55c283549a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.293+0000 7f93c0b9a700 1 --2- 172.21.15.119:0/704583621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c28354c380 0x55c283549a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.293+0000 7f93c0b9a700 1 -- 172.21.15.119:0/704583621 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2832acc00 con 0x55c28354ba60 2023-12-07T10:35:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.293+0000 7f93c1b9c700 1 --2- 172.21.15.119:0/704583621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c28354c380 0x55c283549a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:35:20.294 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.294+0000 7f93c0b9a700 1 --2- 172.21.15.119:0/704583621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c28354ba60 0x55c283566d90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f93ac096f40 tx=0x7f93ac09a010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4312f47e17fd69 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:35:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.296+0000 7f93a3fff700 1 -- 172.21.15.119:0/704583621 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac09abd0 con 0x55c28354ba60 2023-12-07T10:35:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.296+0000 7f93a3fff700 1 -- 172.21.15.119:0/704583621 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93ac09a4d0 con 0x55c28354ba60 2023-12-07T10:35:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.296+0000 7f93a3fff700 1 -- 172.21.15.119:0/704583621 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac00e3b0 con 0x55c28354ba60 2023-12-07T10:35:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.298+0000 7f93d6fd9b80 1 -- 172.21.15.119:0/704583621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c28354ba60 msgr2=0x55c283566d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.298+0000 7f93d6fd9b80 1 --2- 172.21.15.119:0/704583621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c28354ba60 0x55c283566d90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f93ac096f40 tx=0x7f93ac09a010 comp rx=0 tx=0).stop 2023-12-07T10:35:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.299+0000 7f93d6fd9b80 1 -- 172.21.15.119:0/704583621 shutdown_connections 2023-12-07T10:35:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.299+0000 7f93d6fd9b80 1 --2- 172.21.15.119:0/704583621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c28354c380 0x55c283549a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.299+0000 7f93d6fd9b80 1 --2- 172.21.15.119:0/704583621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c283546ad0 0x55c283546540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.299+0000 7f93d6fd9b80 1 --2- 172.21.15.119:0/704583621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c28354ba60 0x55c283566d90 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.299+0000 7f93d6fd9b80 1 -- 172.21.15.119:0/704583621 >> 172.21.15.119:0/704583621 conn(0x55c283475300 msgr2=0x55c28354ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:35:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.299+0000 7f93d6fd9b80 1 -- 172.21.15.119:0/704583621 shutdown_connections 2023-12-07T10:35:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.300+0000 7f93d6fd9b80 1 -- 172.21.15.119:0/704583621 wait complete. 2023-12-07T10:35:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.301+0000 7f93d6fd9b80 1 -- start start 2023-12-07T10:35:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.302+0000 7f93d6fd9b80 1 Processor -- start 2023-12-07T10:35:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.305+0000 7f93d6fd9b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c283546ad0 0x55c283552da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.305+0000 7f93d6fd9b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c28354ba60 0x55c2835532e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.305+0000 7f93d6fd9b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c28354c380 0x55c283553820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.305+0000 7f93d6fd9b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2835b5680 con 0x55c28354c380 2023-12-07T10:35:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.305+0000 7f93d6fd9b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2835b57f0 con 0x55c283546ad0 2023-12-07T10:35:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.305+0000 7f93d6fd9b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2835b59b0 con 0x55c28354ba60 2023-12-07T10:35:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.305+0000 7f93d6fd9b80 10 client.0 _ll_register_callbacks cb 0x55c28343b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:35:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.308+0000 7f93c139b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c283546ad0 0x55c283552da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.308+0000 7f93c1b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c28354c380 0x55c283553820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.308+0000 7f93c1b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c28354c380 0x55c283553820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60908/0 (socket says 172.21.15.119:60908) 2023-12-07T10:35:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.308+0000 7f93c1b9c700 1 -- 172.21.15.119:0/961723796 learned_addr learned my addr 172.21.15.119:0/961723796 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:35:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.309+0000 7f93c0b9a700 1 --2- 172.21.15.119:0/961723796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c28354ba60 0x55c2835532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.310+0000 7f93c139b700 1 -- 172.21.15.119:0/961723796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c28354ba60 msgr2=0x55c2835532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.310+0000 7f93c139b700 1 --2- 172.21.15.119:0/961723796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c28354ba60 0x55c2835532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.310+0000 7f93c139b700 1 -- 172.21.15.119:0/961723796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c28354c380 msgr2=0x55c283553820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.311+0000 7f93c139b700 1 --2- 172.21.15.119:0/961723796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c28354c380 0x55c283553820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.311+0000 7f93c139b700 1 -- 172.21.15.119:0/961723796 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2832c3d80 con 0x55c283546ad0 2023-12-07T10:35:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.311+0000 7f93c1b9c700 1 --2- 172.21.15.119:0/961723796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c28354c380 0x55c283553820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:35:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.314+0000 7f93c139b700 1 --2- 172.21.15.119:0/961723796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c283546ad0 0x55c283552da0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x55c283562d10 tx=0x7f93b0003bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbf1be2cc835721b server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:35:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.314+0000 7f93a0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:35:20.321 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.322+0000 7f93a0ff9700 1 -- 172.21.15.119:0/961723796 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b002a040 con 0x55c283546ad0 2023-12-07T10:35:20.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.322+0000 7f93a0ff9700 1 -- 172.21.15.119:0/961723796 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f93b0026a30 con 0x55c283546ad0 2023-12-07T10:35:20.322 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.323+0000 7f93d6fd9b80 1 -- 172.21.15.119:0/961723796 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93b00268c0 con 0x55c283546ad0 2023-12-07T10:35:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.326+0000 7f93a0ff9700 1 -- 172.21.15.119:0/961723796 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93b0026c10 con 0x55c283546ad0 2023-12-07T10:35:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.327+0000 7f93d6fd9b80 10 client.27986 fetch_fsmap learned FSMap version 71 2023-12-07T10:35:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.327+0000 7f93d6fd9b80 1 -- 172.21.15.119:0/961723796 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) v3 -- 0x7f93bc1b3510 con 0x55c283546ad0 2023-12-07T10:35:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.329+0000 7f93a0ff9700 1 -- 172.21.15.119:0/961723796 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93b00268c0 con 0x55c283546ad0 2023-12-07T10:35:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.330+0000 7f93a0ff9700 1 -- 172.21.15.119:0/961723796 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 71) v1 ==== 7958+0+0 (secure 0 0 0) 0x7f93b0024020 con 0x55c283546ad0 2023-12-07T10:35:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.330+0000 7f93d6fd9b80 10 client.27986 fetch_fsmap finished waiting for FSMap version 71 2023-12-07T10:35:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.330+0000 7f93d6fd9b80 10 client.27986 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:35:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.330+0000 7f93d6fd9b80 20 client.27986 populate_metadata read hostname 'smithi119' 2023-12-07T10:35:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.330+0000 7f93d6fd9b80 1 --2- 172.21.15.119:0/961723796 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c2835b71c0 0x55c2835b95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.330+0000 7f93d6fd9b80 4 client.27986 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:35:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.330+0000 7f93d6fd9b80 1 -- 172.21.15.119:0/961723796 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c283460f70 con 0x55c2835b71c0 2023-12-07T10:35:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.330+0000 7f93c0b9a700 1 --2- 172.21.15.119:0/961723796 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c2835b71c0 0x55c2835b95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.333+0000 7f93c0b9a700 1 --2- 172.21.15.119:0/961723796 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c2835b71c0 0x55c2835b95e0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8984e3a340e88eb8 server_cookie=9a4bb923d9a35a38 in_seq=0 out_seq=0 2023-12-07T10:35:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:20.334+0000 7f93a0ff9700 10 client.27986 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:35:20.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:20 smithi119 ceph-mon[97908]: pgmap v468: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.7 MiB/s wr, 640 op/s 2023-12-07T10:35:20.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:20 smithi159 ceph-mon[134726]: pgmap v468: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.7 MiB/s wr, 640 op/s 2023-12-07T10:35:20.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:20 smithi153 ceph-mon[133625]: pgmap v468: 129 pgs: 129 active+clean; 1.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.7 MiB/s wr, 640 op/s 2023-12-07T10:35:20.975 INFO:tasks.workunit.client.0.smithi119.stdout: 10 56203 34.46 MB/sec warmup 93 sec latency 574.725 ms 2023-12-07T10:35:21.973 INFO:tasks.workunit.client.0.smithi119.stdout: 10 56328 34.16 MB/sec warmup 94 sec latency 1371.392 ms 2023-12-07T10:35:22.227 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.227+0000 7f93a0ff9700 1 -- 172.21.15.119:0/961723796 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c283460f70 con 0x55c2835b71c0 2023-12-07T10:35:22.227 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.227+0000 7f93a0ff9700 10 client.27986 handle_command_reply: tid=0 2023-12-07T10:35:22.227 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.227+0000 7f93d6fd9b80 1 client.27986 shutdown 2023-12-07T10:35:22.227 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.228+0000 7f93d6fd9b80 1 -- 172.21.15.119:0/961723796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c283546ad0 msgr2=0x55c283552da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.228+0000 7f93d6fd9b80 1 --2- 172.21.15.119:0/961723796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c283546ad0 0x55c283552da0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x55c283562d10 tx=0x7f93b0003bd0 comp rx=0 tx=0).stop 2023-12-07T10:35:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.229+0000 7f93d6fd9b80 1 -- 172.21.15.119:0/961723796 shutdown_connections 2023-12-07T10:35:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.229+0000 7f93d6fd9b80 1 --2- 172.21.15.119:0/961723796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c28354c380 0x55c283553820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:22.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.229+0000 7f93d6fd9b80 1 --2- 172.21.15.119:0/961723796 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x55c2835b71c0 0x55c2835b95e0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:22.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.229+0000 7f93a0ff9700 0 client.27986 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:35:22.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.229+0000 7f93d6fd9b80 1 --2- 172.21.15.119:0/961723796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c283546ad0 0x55c283552da0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:22.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.229+0000 7f93d6fd9b80 1 --2- 172.21.15.119:0/961723796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c28354ba60 0x55c2835532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:22.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.229+0000 7f93d6fd9b80 1 -- 172.21.15.119:0/961723796 >> 172.21.15.119:0/961723796 conn(0x55c283475300 msgr2=0x55c283549a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:35:22.229 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.229+0000 7f93d6fd9b80 1 -- 172.21.15.119:0/961723796 shutdown_connections 2023-12-07T10:35:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.230+0000 7f93d6fd9b80 1 -- 172.21.15.119:0/961723796 wait complete. 2023-12-07T10:35:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.231+0000 7f93d6fd9b80 20 client.27986 trim_cache size 0 max 16384 2023-12-07T10:35:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:35:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:35:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:35:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:35:22.232 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.232+0000 7f93c35ff700 1 -- 172.21.15.119:0/339051808 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f93a40a1d80 msgr2=0x7f93a40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:22.232 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.232+0000 7f93c35ff700 1 --2- 172.21.15.119:0/339051808 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f93a40a1d80 0x7f93a40a4240 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f93b0011490 tx=0x7f93b0009260 comp rx=0 tx=0).stop 2023-12-07T10:35:22.232 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.232+0000 7f93c35ff700 1 -- 172.21.15.119:0/339051808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93bc1101a0 msgr2=0x7f93bc1ac9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:22.232 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.232+0000 7f93c35ff700 1 --2- 172.21.15.119:0/339051808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93bc1101a0 0x7f93bc1ac9b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00c890 tx=0x7f93ac00cba0 comp rx=0 tx=0).stop 2023-12-07T10:35:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.233+0000 7f93c35ff700 1 -- 172.21.15.119:0/339051808 shutdown_connections 2023-12-07T10:35:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.233+0000 7f93c35ff700 1 --2- 172.21.15.119:0/339051808 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f93a40a1d80 0x7f93a40a4240 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.233+0000 7f93c35ff700 1 --2- 172.21.15.119:0/339051808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93bc11adf0 0x7f93bc1a6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.233+0000 7f93c35ff700 1 --2- 172.21.15.119:0/339051808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93bc10fd30 0x7f93bc1ac470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.233+0000 7f93c35ff700 1 --2- 172.21.15.119:0/339051808 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93bc1101a0 0x7f93bc1ac9b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.233+0000 7f93c35ff700 1 -- 172.21.15.119:0/339051808 >> 172.21.15.119:0/339051808 conn(0x7f93bc06c5e0 msgr2=0x7f93bc118de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:35:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.233+0000 7f93c35ff700 1 -- 172.21.15.119:0/339051808 shutdown_connections 2023-12-07T10:35:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:22.233+0000 7f93c35ff700 1 -- 172.21.15.119:0/339051808 wait complete. 2023-12-07T10:35:22.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:22 smithi119 ceph-mon[97908]: pgmap v469: 129 pgs: 129 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.9 MiB/s wr, 639 op/s 2023-12-07T10:35:22.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:22 smithi159 ceph-mon[134726]: pgmap v469: 129 pgs: 129 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.9 MiB/s wr, 639 op/s 2023-12-07T10:35:22.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:22 smithi153 ceph-mon[133625]: pgmap v469: 129 pgs: 129 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.9 MiB/s wr, 639 op/s 2023-12-07T10:35:22.972 INFO:tasks.workunit.client.0.smithi119.stdout: 10 56660 34.00 MB/sec warmup 95 sec latency 964.990 ms 2023-12-07T10:35:23.973 INFO:tasks.workunit.client.0.smithi119.stdout: 10 57021 33.87 MB/sec warmup 96 sec latency 128.685 ms 2023-12-07T10:35:24.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:24 smithi119 ceph-mon[97908]: pgmap v470: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 14 MiB/s wr, 743 op/s 2023-12-07T10:35:24.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:24 smithi159 ceph-mon[134726]: pgmap v470: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 14 MiB/s wr, 743 op/s 2023-12-07T10:35:24.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:24 smithi153 ceph-mon[133625]: pgmap v470: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 14 MiB/s wr, 743 op/s 2023-12-07T10:35:24.973 INFO:tasks.workunit.client.0.smithi119.stdout: 10 57472 33.71 MB/sec warmup 97 sec latency 105.221 ms 2023-12-07T10:35:25.973 INFO:tasks.workunit.client.0.smithi119.stdout: 10 58082 33.72 MB/sec warmup 98 sec latency 95.709 ms 2023-12-07T10:35:26.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:26 smithi119 ceph-mon[97908]: pgmap v471: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 14 MiB/s wr, 743 op/s 2023-12-07T10:35:26.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:26 smithi159 ceph-mon[134726]: pgmap v471: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 14 MiB/s wr, 743 op/s 2023-12-07T10:35:26.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:26 smithi153 ceph-mon[133625]: pgmap v471: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 14 MiB/s wr, 743 op/s 2023-12-07T10:35:26.973 INFO:tasks.workunit.client.0.smithi119.stdout: 10 58612 33.67 MB/sec warmup 99 sec latency 68.228 ms 2023-12-07T10:35:27.973 INFO:tasks.workunit.client.0.smithi119.stdout: 10 59491 33.76 MB/sec warmup 100 sec latency 84.324 ms 2023-12-07T10:35:28.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:28 smithi119 ceph-mon[97908]: pgmap v472: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 16 MiB/s wr, 963 op/s 2023-12-07T10:35:28.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:28 smithi159 ceph-mon[134726]: pgmap v472: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 16 MiB/s wr, 963 op/s 2023-12-07T10:35:28.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:28 smithi153 ceph-mon[133625]: pgmap v472: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 16 MiB/s wr, 963 op/s 2023-12-07T10:35:28.974 INFO:tasks.workunit.client.0.smithi119.stdout: 10 60337 33.91 MB/sec warmup 101 sec latency 52.764 ms 2023-12-07T10:35:29.975 INFO:tasks.workunit.client.0.smithi119.stdout: 10 61140 34.00 MB/sec warmup 102 sec latency 53.173 ms 2023-12-07T10:35:30.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:30 smithi119 ceph-mon[97908]: pgmap v473: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 13 MiB/s wr, 756 op/s 2023-12-07T10:35:30.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:30 smithi159 ceph-mon[134726]: pgmap v473: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 13 MiB/s wr, 756 op/s 2023-12-07T10:35:30.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:30 smithi153 ceph-mon[133625]: pgmap v473: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 13 MiB/s wr, 756 op/s 2023-12-07T10:35:30.974 INFO:tasks.workunit.client.0.smithi119.stdout: 10 61801 34.06 MB/sec warmup 103 sec latency 80.003 ms 2023-12-07T10:35:31.974 INFO:tasks.workunit.client.0.smithi119.stdout: 10 62402 34.04 MB/sec warmup 104 sec latency 88.258 ms 2023-12-07T10:35:32.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:32 smithi119 ceph-mon[97908]: pgmap v474: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 14 MiB/s wr, 834 op/s 2023-12-07T10:35:32.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:32 smithi159 ceph-mon[134726]: pgmap v474: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 14 MiB/s wr, 834 op/s 2023-12-07T10:35:32.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:32 smithi153 ceph-mon[133625]: pgmap v474: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 14 MiB/s wr, 834 op/s 2023-12-07T10:35:32.974 INFO:tasks.workunit.client.0.smithi119.stdout: 10 63163 34.06 MB/sec warmup 105 sec latency 80.712 ms 2023-12-07T10:35:33.974 INFO:tasks.workunit.client.0.smithi119.stdout: 10 64105 34.19 MB/sec warmup 106 sec latency 68.724 ms 2023-12-07T10:35:34.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:34 smithi119 ceph-mon[97908]: pgmap v475: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 12 MiB/s wr, 816 op/s 2023-12-07T10:35:34.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:34 smithi159 ceph-mon[134726]: pgmap v475: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 12 MiB/s wr, 816 op/s 2023-12-07T10:35:34.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:34 smithi153 ceph-mon[133625]: pgmap v475: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 12 MiB/s wr, 816 op/s 2023-12-07T10:35:34.974 INFO:tasks.workunit.client.0.smithi119.stdout: 10 65103 34.44 MB/sec warmup 107 sec latency 71.613 ms 2023-12-07T10:35:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.919+0000 7ff34bbff700 1 -- 172.21.15.119:0/3690900800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3441075c0 msgr2=0x7ff3441079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.919+0000 7ff34bbff700 1 --2- 172.21.15.119:0/3690900800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3441075c0 0x7ff3441079a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff340009990 tx=0x7ff340009ca0 comp rx=0 tx=0).stop 2023-12-07T10:35:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.920+0000 7ff34bbff700 1 -- 172.21.15.119:0/3690900800 shutdown_connections 2023-12-07T10:35:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.920+0000 7ff34bbff700 1 --2- 172.21.15.119:0/3690900800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff344101ec0 0x7ff34410e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.920+0000 7ff34bbff700 1 --2- 172.21.15.119:0/3690900800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3441075c0 0x7ff3441079a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.920+0000 7ff34bbff700 1 --2- 172.21.15.119:0/3690900800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff344101500 0x7ff344101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.920+0000 7ff34bbff700 1 -- 172.21.15.119:0/3690900800 >> 172.21.15.119:0/3690900800 conn(0x7ff3440fcfb0 msgr2=0x7ff3440ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:35:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.921+0000 7ff34bbff700 1 -- 172.21.15.119:0/3690900800 shutdown_connections 2023-12-07T10:35:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.921+0000 7ff34bbff700 1 -- 172.21.15.119:0/3690900800 wait complete. 2023-12-07T10:35:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.922+0000 7ff34bbff700 1 Processor -- start 2023-12-07T10:35:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.922+0000 7ff34bbff700 1 -- start start 2023-12-07T10:35:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.923+0000 7ff34bbff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff344101500 0x7ff344072b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.923+0000 7ff34bbff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff344101ec0 0x7ff3440730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.924+0000 7ff34bbff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3441075c0 0x7ff34406bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.924+0000 7ff34bbff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34406c3b0 con 0x7ff344101500 2023-12-07T10:35:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.924+0000 7ff34bbff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34406c520 con 0x7ff3441075c0 2023-12-07T10:35:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.924+0000 7ff34bbff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34406c6e0 con 0x7ff344101ec0 2023-12-07T10:35:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.924+0000 7ff34919a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff344101ec0 0x7ff3440730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.924+0000 7ff34919a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff344101ec0 0x7ff3440730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:36554/0 (socket says 172.21.15.119:36554) 2023-12-07T10:35:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.924+0000 7ff34919a700 1 -- 172.21.15.119:0/1278679165 learned_addr learned my addr 172.21.15.119:0/1278679165 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:35:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.925+0000 7ff34919a700 1 -- 172.21.15.119:0/1278679165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3441075c0 msgr2=0x7ff34406bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.925+0000 7ff34919a700 1 --2- 172.21.15.119:0/1278679165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3441075c0 0x7ff34406bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.925+0000 7ff34919a700 1 -- 172.21.15.119:0/1278679165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff344101500 msgr2=0x7ff344072b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.925+0000 7ff34919a700 1 --2- 172.21.15.119:0/1278679165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff344101500 0x7ff344072b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.925+0000 7ff34919a700 1 -- 172.21.15.119:0/1278679165 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3400095f0 con 0x7ff344101ec0 2023-12-07T10:35:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.926+0000 7ff34919a700 1 --2- 172.21.15.119:0/1278679165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff344101ec0 0x7ff3440730b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff33400b550 tx=0x7ff33400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:35:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.927+0000 7ff33affd700 1 -- 172.21.15.119:0/1278679165 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff334011830 con 0x7ff344101ec0 2023-12-07T10:35:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.927+0000 7ff34bbff700 1 -- 172.21.15.119:0/1278679165 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff34406c9c0 con 0x7ff344101ec0 2023-12-07T10:35:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.928+0000 7ff34bbff700 1 -- 172.21.15.119:0/1278679165 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3440709b0 con 0x7ff344101ec0 2023-12-07T10:35:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.928+0000 7ff33affd700 1 -- 172.21.15.119:0/1278679165 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff334011e50 con 0x7ff344101ec0 2023-12-07T10:35:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.928+0000 7ff33affd700 1 -- 172.21.15.119:0/1278679165 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff334011830 con 0x7ff344101ec0 2023-12-07T10:35:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.939+0000 7ff33affd700 1 -- 172.21.15.119:0/1278679165 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7ff334010cc0 con 0x7ff344101ec0 2023-12-07T10:35:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.942+0000 7ff33affd700 1 --2- 172.21.15.119:0/1278679165 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff3300a6190 0x7ff3300a8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.942+0000 7ff33affd700 1 -- 172.21.15.119:0/1278679165 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7ff3340cd3d0 con 0x7ff344101ec0 2023-12-07T10:35:35.945 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.945+0000 7ff34999b700 1 --2- 172.21.15.119:0/1278679165 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff3300a6190 0x7ff3300a8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:35.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.946+0000 7ff35f486b80 1 Processor -- start 2023-12-07T10:35:35.946 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.946+0000 7ff35f486b80 1 -- start start 2023-12-07T10:35:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.947+0000 7ff35f486b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572be36d1a0 0x5572be36d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.947+0000 7ff35f486b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572be368d50 0x5572be3691d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.947+0000 7ff35f486b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572be369710 0x5572be3750b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.948+0000 7ff35f486b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572be2d4760 con 0x5572be369710 2023-12-07T10:35:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.948+0000 7ff35f486b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572be2bb740 con 0x5572be368d50 2023-12-07T10:35:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.948+0000 7ff35f486b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572be2b9060 con 0x5572be36d1a0 2023-12-07T10:35:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.948+0000 7ff34999b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572be36d1a0 0x5572be36d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.948+0000 7ff34999b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572be36d1a0 0x5572be36d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:36560/0 (socket says 172.21.15.119:36560) 2023-12-07T10:35:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.948+0000 7ff34999b700 1 -- 172.21.15.119:0/3004946034 learned_addr learned my addr 172.21.15.119:0/3004946034 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:35:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.948+0000 7ff34999b700 1 --2- 172.21.15.119:0/1278679165 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff3300a6190 0x7ff3300a8650 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff34000b5c0 tx=0x7ff340005fb0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:35:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.952+0000 7ff34a19c700 1 --2- 172.21.15.119:0/3004946034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572be369710 0x5572be3750b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.952+0000 7ff34919a700 1 --2- 172.21.15.119:0/3004946034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572be368d50 0x5572be3691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.952+0000 7ff34999b700 1 -- 172.21.15.119:0/3004946034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572be368d50 msgr2=0x5572be3691d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.952+0000 7ff34999b700 1 --2- 172.21.15.119:0/3004946034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572be368d50 0x5572be3691d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.952+0000 7ff34999b700 1 -- 172.21.15.119:0/3004946034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572be369710 msgr2=0x5572be3750b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.952+0000 7ff34999b700 1 --2- 172.21.15.119:0/3004946034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572be369710 0x5572be3750b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.952+0000 7ff34999b700 1 -- 172.21.15.119:0/3004946034 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572be1315d0 con 0x5572be36d1a0 2023-12-07T10:35:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.952+0000 7ff34919a700 1 --2- 172.21.15.119:0/3004946034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572be368d50 0x5572be3691d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:35:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.957+0000 7ff34999b700 1 --2- 172.21.15.119:0/3004946034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572be36d1a0 0x5572be36d580 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff34000bfb0 tx=0x7ff34001d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3caf10f32048426f server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:35:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.958+0000 7ff32ffff700 1 -- 172.21.15.119:0/3004946034 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff340029070 con 0x5572be36d1a0 2023-12-07T10:35:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.958+0000 7ff32ffff700 1 -- 172.21.15.119:0/3004946034 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff340005460 con 0x5572be36d1a0 2023-12-07T10:35:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.961+0000 7ff32ffff700 1 -- 172.21.15.119:0/3004946034 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff340029070 con 0x5572be36d1a0 2023-12-07T10:35:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.961+0000 7ff35f486b80 1 -- 172.21.15.119:0/3004946034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572be36d1a0 msgr2=0x5572be36d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.961+0000 7ff35f486b80 1 --2- 172.21.15.119:0/3004946034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572be36d1a0 0x5572be36d580 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff34000bfb0 tx=0x7ff34001d000 comp rx=0 tx=0).stop 2023-12-07T10:35:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.961+0000 7ff35f486b80 1 -- 172.21.15.119:0/3004946034 shutdown_connections 2023-12-07T10:35:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.961+0000 7ff35f486b80 1 --2- 172.21.15.119:0/3004946034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572be369710 0x5572be3750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.962+0000 7ff35f486b80 1 --2- 172.21.15.119:0/3004946034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572be36d1a0 0x5572be36d580 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.962+0000 7ff35f486b80 1 --2- 172.21.15.119:0/3004946034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572be368d50 0x5572be3691d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.962+0000 7ff35f486b80 1 -- 172.21.15.119:0/3004946034 >> 172.21.15.119:0/3004946034 conn(0x5572be28f300 msgr2=0x5572be35ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:35:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.962+0000 7ff35f486b80 1 -- 172.21.15.119:0/3004946034 shutdown_connections 2023-12-07T10:35:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.962+0000 7ff35f486b80 1 -- 172.21.15.119:0/3004946034 wait complete. 2023-12-07T10:35:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.963+0000 7ff35f486b80 1 -- start start 2023-12-07T10:35:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.964+0000 7ff35f486b80 1 Processor -- start 2023-12-07T10:35:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.965+0000 7ff35f486b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572be368d50 0x5572be374ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.966+0000 7ff35f486b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572be369710 0x5572be3722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.966+0000 7ff35f486b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572be36d1a0 0x5572be372820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.967+0000 7ff34a19c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572be36d1a0 0x5572be372820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.967+0000 7ff34a19c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572be36d1a0 0x5572be372820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:36572/0 (socket says 172.21.15.119:36572) 2023-12-07T10:35:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.967+0000 7ff34a19c700 1 -- 172.21.15.119:0/1954253115 learned_addr learned my addr 172.21.15.119:0/1954253115 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:35:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.967+0000 7ff35f486b80 1 -- 172.21.15.119:0/1954253115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572be2b4950 con 0x5572be369710 2023-12-07T10:35:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.967+0000 7ff35f486b80 1 -- 172.21.15.119:0/1954253115 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572be26e500 con 0x5572be368d50 2023-12-07T10:35:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.967+0000 7ff35f486b80 1 -- 172.21.15.119:0/1954253115 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572be26e1a0 con 0x5572be36d1a0 2023-12-07T10:35:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.968+0000 7ff34a19c700 1 -- 172.21.15.119:0/1954253115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572be368d50 msgr2=0x5572be374ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.967+0000 7ff34999b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572be368d50 0x5572be374ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.968+0000 7ff35f486b80 10 client.0 _ll_register_callbacks cb 0x5572be2558e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:35:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.969+0000 7ff34919a700 1 --2- 172.21.15.119:0/1954253115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572be369710 0x5572be3722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.969+0000 7ff34a19c700 1 --2- 172.21.15.119:0/1954253115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572be368d50 0x5572be374ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.969+0000 7ff34a19c700 1 -- 172.21.15.119:0/1954253115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572be369710 msgr2=0x5572be3722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.969+0000 7ff34a19c700 1 --2- 172.21.15.119:0/1954253115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572be369710 0x5572be3722e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.969+0000 7ff34a19c700 1 -- 172.21.15.119:0/1954253115 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3440709b0 con 0x5572be36d1a0 2023-12-07T10:35:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.969+0000 7ff34999b700 1 --2- 172.21.15.119:0/1954253115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572be368d50 0x5572be374ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:35:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.969+0000 7ff35f486b80 1 -- 172.21.15.119:0/1954253115 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572be3536f0 con 0x5572be36d1a0 2023-12-07T10:35:35.974 INFO:tasks.workunit.client.0.smithi119.stdout: 10 65805 34.50 MB/sec warmup 108 sec latency 77.106 ms 2023-12-07T10:35:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.980+0000 7ff34a19c700 1 --2- 172.21.15.119:0/1954253115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572be36d1a0 0x5572be372820 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00c540 tx=0x7ff33c00c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1773d58b0dfbe7d server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:35:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.982+0000 7ff32d7fa700 10 client.27967 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:35:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.982+0000 7ff32d7fa700 1 -- 172.21.15.119:0/1954253115 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c011800 con 0x5572be36d1a0 2023-12-07T10:35:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.982+0000 7ff32d7fa700 1 -- 172.21.15.119:0/1954253115 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7ff33c011960 con 0x5572be36d1a0 2023-12-07T10:35:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.982+0000 7ff32d7fa700 1 -- 172.21.15.119:0/1954253115 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff33c011b80 con 0x5572be36d1a0 2023-12-07T10:35:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.982+0000 7ff35f486b80 10 client.27967 fetch_fsmap learned FSMap version 71 2023-12-07T10:35:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.982+0000 7ff35f486b80 1 -- 172.21.15.119:0/1954253115 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({fsmap=71}) v3 -- 0x7ff34410ec80 con 0x5572be36d1a0 2023-12-07T10:35:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.983+0000 7ff32d7fa700 1 -- 172.21.15.119:0/1954253115 <== mon.1 v2:172.21.15.159:3300/0 4 ==== fsmap(e 71) v1 ==== 7958+0+0 (secure 0 0 0) 0x7ff33c011ce0 con 0x5572be36d1a0 2023-12-07T10:35:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.983+0000 7ff35f486b80 10 client.27967 fetch_fsmap finished waiting for FSMap version 71 2023-12-07T10:35:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.983+0000 7ff35f486b80 10 client.27967 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:35:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.983+0000 7ff35f486b80 20 client.27967 populate_metadata read hostname 'smithi119' 2023-12-07T10:35:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.983+0000 7ff35f486b80 1 --2- 172.21.15.119:0/1954253115 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5572be382490 0x5572be3848b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.983+0000 7ff35f486b80 4 client.27967 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:35:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.983+0000 7ff35f486b80 1 -- 172.21.15.119:0/1954253115 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5572be32f740 con 0x5572be382490 2023-12-07T10:35:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:35.985+0000 7ff34999b700 1 --2- 172.21.15.119:0/1954253115 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5572be382490 0x5572be3848b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.001+0000 7ff34999b700 1 --2- 172.21.15.119:0/1954253115 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5572be382490 0x5572be3848b0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6b86341cc1b56c server_cookie=429fe22512f198a2 in_seq=0 out_seq=0 2023-12-07T10:35:36.002 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.001+0000 7ff32d7fa700 10 client.27967 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:35:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.010+0000 7ff32d7fa700 1 -- 172.21.15.119:0/1954253115 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5572be32f740 con 0x5572be382490 2023-12-07T10:35:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.013+0000 7ff32d7fa700 10 client.27967 handle_command_reply: tid=0 2023-12-07T10:35:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.015+0000 7ff35f486b80 1 client.27967 shutdown 2023-12-07T10:35:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.025+0000 7ff35f486b80 1 -- 172.21.15.119:0/1954253115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572be36d1a0 msgr2=0x5572be372820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.025+0000 7ff35f486b80 1 --2- 172.21.15.119:0/1954253115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572be36d1a0 0x5572be372820 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00c540 tx=0x7ff33c00c900 comp rx=0 tx=0).stop 2023-12-07T10:35:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.025+0000 7ff35f486b80 1 -- 172.21.15.119:0/1954253115 shutdown_connections 2023-12-07T10:35:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.026+0000 7ff35f486b80 1 --2- 172.21.15.119:0/1954253115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572be36d1a0 0x5572be372820 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.026+0000 7ff35f486b80 1 --2- 172.21.15.119:0/1954253115 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5572be382490 0x5572be3848b0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.026+0000 7ff35f486b80 1 --2- 172.21.15.119:0/1954253115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572be368d50 0x5572be374ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.026+0000 7ff35f486b80 1 --2- 172.21.15.119:0/1954253115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572be369710 0x5572be3722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.026+0000 7ff35f486b80 1 -- 172.21.15.119:0/1954253115 >> 172.21.15.119:0/1954253115 conn(0x5572be28f300 msgr2=0x5572be355980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:35:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.026+0000 7ff32d7fa700 0 client.27967 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:35:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.026+0000 7ff35f486b80 1 -- 172.21.15.119:0/1954253115 shutdown_connections 2023-12-07T10:35:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.027+0000 7ff35f486b80 1 -- 172.21.15.119:0/1954253115 wait complete. 2023-12-07T10:35:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.027+0000 7ff35f486b80 20 client.27967 trim_cache size 0 max 16384 2023-12-07T10:35:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.089+0000 7ff35f486b80 1 Processor -- start 2023-12-07T10:35:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.089+0000 7ff35f486b80 1 -- start start 2023-12-07T10:35:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.091+0000 7ff35f486b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572be368d50 0x5572be369130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.092+0000 7ff35f486b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572be369710 0x5572be36ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.092+0000 7ff35f486b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572be36d1a0 0x5572be3796b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.092+0000 7ff35f486b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572be1338c0 con 0x5572be369710 2023-12-07T10:35:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.092+0000 7ff35f486b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572be36dca0 con 0x5572be368d50 2023-12-07T10:35:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.092+0000 7ff35f486b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572be36bf90 con 0x5572be36d1a0 2023-12-07T10:35:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.093+0000 7ff34919a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572be369710 0x5572be36ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.096+0000 7ff34919a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572be369710 0x5572be36ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47134/0 (socket says 172.21.15.119:47134) 2023-12-07T10:35:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.096+0000 7ff34919a700 1 -- 172.21.15.119:0/2215661562 learned_addr learned my addr 172.21.15.119:0/2215661562 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:35:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.098+0000 7ff34a19c700 1 --2- 172.21.15.119:0/2215661562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572be36d1a0 0x5572be3796b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.101+0000 7ff34999b700 1 --2- 172.21.15.119:0/2215661562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572be368d50 0x5572be369130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.102+0000 7ff34999b700 1 -- 172.21.15.119:0/2215661562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572be36d1a0 msgr2=0x5572be3796b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.102+0000 7ff34999b700 1 --2- 172.21.15.119:0/2215661562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572be36d1a0 0x5572be3796b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.102+0000 7ff34999b700 1 -- 172.21.15.119:0/2215661562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572be369710 msgr2=0x5572be36ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.102+0000 7ff34999b700 1 --2- 172.21.15.119:0/2215661562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572be369710 0x5572be36ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.102+0000 7ff34999b700 1 -- 172.21.15.119:0/2215661562 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572be0ddd80 con 0x5572be368d50 2023-12-07T10:35:36.104 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.104+0000 7ff34999b700 1 --2- 172.21.15.119:0/2215661562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572be368d50 0x5572be369130 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff34000bf80 tx=0x7ff340000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a3cbda14a99e8c1 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:35:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.107+0000 7ff338bff700 1 -- 172.21.15.119:0/2215661562 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff340029040 con 0x5572be368d50 2023-12-07T10:35:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.107+0000 7ff338bff700 1 -- 172.21.15.119:0/2215661562 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff340004e60 con 0x5572be368d50 2023-12-07T10:35:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.107+0000 7ff338bff700 1 -- 172.21.15.119:0/2215661562 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34001d070 con 0x5572be368d50 2023-12-07T10:35:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.108+0000 7ff35f486b80 1 -- 172.21.15.119:0/2215661562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572be368d50 msgr2=0x5572be369130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.108+0000 7ff35f486b80 1 --2- 172.21.15.119:0/2215661562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572be368d50 0x5572be369130 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff34000bf80 tx=0x7ff340000bc0 comp rx=0 tx=0).stop 2023-12-07T10:35:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.109+0000 7ff35f486b80 1 -- 172.21.15.119:0/2215661562 shutdown_connections 2023-12-07T10:35:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.109+0000 7ff35f486b80 1 --2- 172.21.15.119:0/2215661562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572be36d1a0 0x5572be3796b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.109+0000 7ff35f486b80 1 --2- 172.21.15.119:0/2215661562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572be368d50 0x5572be369130 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.109+0000 7ff35f486b80 1 --2- 172.21.15.119:0/2215661562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572be369710 0x5572be36ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.109+0000 7ff35f486b80 1 -- 172.21.15.119:0/2215661562 >> 172.21.15.119:0/2215661562 conn(0x5572be28f300 msgr2=0x5572be389230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:35:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.110+0000 7ff35f486b80 1 -- 172.21.15.119:0/2215661562 shutdown_connections 2023-12-07T10:35:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.110+0000 7ff35f486b80 1 -- 172.21.15.119:0/2215661562 wait complete. 2023-12-07T10:35:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.110+0000 7ff35f486b80 1 -- start start 2023-12-07T10:35:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.111+0000 7ff35f486b80 1 Processor -- start 2023-12-07T10:35:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.112+0000 7ff35f486b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572be368d50 0x5572be379b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.112+0000 7ff35f486b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572be369710 0x5572be355260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.112+0000 7ff35f486b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572be36d1a0 0x5572be3558e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.113+0000 7ff35f486b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572be369af0 con 0x5572be368d50 2023-12-07T10:35:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.113+0000 7ff35f486b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572be369c60 con 0x5572be36d1a0 2023-12-07T10:35:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.113+0000 7ff35f486b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572be369e20 con 0x5572be369710 2023-12-07T10:35:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.113+0000 7ff35f486b80 10 client.0 _ll_register_callbacks cb 0x5572be2558e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:35:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.114+0000 7ff34999b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572be368d50 0x5572be379b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.114+0000 7ff34a19c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572be36d1a0 0x5572be3558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.114+0000 7ff34a19c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572be36d1a0 0x5572be3558e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:41150/0 (socket says 172.21.15.119:41150) 2023-12-07T10:35:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.114+0000 7ff34a19c700 1 -- 172.21.15.119:0/4024614935 learned_addr learned my addr 172.21.15.119:0/4024614935 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:35:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.114+0000 7ff34919a700 1 --2- 172.21.15.119:0/4024614935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572be369710 0x5572be355260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.116+0000 7ff34919a700 1 -- 172.21.15.119:0/4024614935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572be36d1a0 msgr2=0x5572be3558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.116+0000 7ff34919a700 1 --2- 172.21.15.119:0/4024614935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572be36d1a0 0x5572be3558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.116+0000 7ff34919a700 1 -- 172.21.15.119:0/4024614935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572be368d50 msgr2=0x5572be379b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.116+0000 7ff34919a700 1 --2- 172.21.15.119:0/4024614935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572be368d50 0x5572be379b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.117+0000 7ff34919a700 1 -- 172.21.15.119:0/4024614935 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5572be1315d0 con 0x5572be369710 2023-12-07T10:35:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.117+0000 7ff34a19c700 1 --2- 172.21.15.119:0/4024614935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572be36d1a0 0x5572be3558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:35:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.119+0000 7ff34919a700 1 --2- 172.21.15.119:0/4024614935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572be369710 0x5572be355260 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff3340cad70 tx=0x7ff334099f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc8e8ce2734f6f93 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:35:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.119+0000 7ff32d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2023-12-07T10:35:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.121+0000 7ff32d7fa700 1 -- 172.21.15.119:0/4024614935 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff334011830 con 0x5572be369710 2023-12-07T10:35:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.122+0000 7ff32d7fa700 1 -- 172.21.15.119:0/4024614935 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7ff3340172c0 con 0x5572be369710 2023-12-07T10:35:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.122+0000 7ff35f486b80 1 -- 172.21.15.119:0/4024614935 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff340005460 con 0x5572be369710 2023-12-07T10:35:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.123+0000 7ff32d7fa700 1 -- 172.21.15.119:0/4024614935 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff334017480 con 0x5572be369710 2023-12-07T10:35:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.123+0000 7ff35f486b80 10 client.27973 fetch_fsmap learned FSMap version 71 2023-12-07T10:35:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.123+0000 7ff35f486b80 1 -- 172.21.15.119:0/4024614935 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) v3 -- 0x5572be0c6c00 con 0x5572be369710 2023-12-07T10:35:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.125+0000 7ff32d7fa700 1 -- 172.21.15.119:0/4024614935 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff340005460 con 0x5572be369710 2023-12-07T10:35:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.125+0000 7ff32d7fa700 1 -- 172.21.15.119:0/4024614935 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 71) v1 ==== 7958+0+0 (secure 0 0 0) 0x7ff3340a1dd0 con 0x5572be369710 2023-12-07T10:35:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.125+0000 7ff35f486b80 10 client.27973 fetch_fsmap finished waiting for FSMap version 71 2023-12-07T10:35:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.127+0000 7ff35f486b80 10 client.27973 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:35:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.127+0000 7ff35f486b80 20 client.27973 populate_metadata read hostname 'smithi119' 2023-12-07T10:35:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.127+0000 7ff35f486b80 1 --2- 172.21.15.119:0/4024614935 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5572be3d9520 0x5572be3db940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.127+0000 7ff35f486b80 4 client.27973 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:35:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.127+0000 7ff35f486b80 1 -- 172.21.15.119:0/4024614935 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5572be27af70 con 0x5572be3d9520 2023-12-07T10:35:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.128+0000 7ff34999b700 1 --2- 172.21.15.119:0/4024614935 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5572be3d9520 0x5572be3db940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.134+0000 7ff34999b700 1 --2- 172.21.15.119:0/4024614935 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5572be3d9520 0x5572be3db940 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d0d1a1cfa27b904 server_cookie=28d089aa82bab7f3 in_seq=0 out_seq=0 2023-12-07T10:35:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:36.134+0000 7ff32d7fa700 10 client.27973 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:35:36.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:36 smithi159 ceph-mon[134726]: pgmap v476: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.2 MiB/s wr, 615 op/s 2023-12-07T10:35:36.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:36 smithi153 ceph-mon[133625]: pgmap v476: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.2 MiB/s wr, 615 op/s 2023-12-07T10:35:36.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:36 smithi119 ceph-mon[97908]: pgmap v476: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.2 MiB/s wr, 615 op/s 2023-12-07T10:35:36.975 INFO:tasks.workunit.client.0.smithi119.stdout: 10 65869 34.19 MB/sec warmup 109 sec latency 1030.828 ms 2023-12-07T10:35:37.975 INFO:tasks.workunit.client.0.smithi119.stdout: 10 66005 33.97 MB/sec warmup 110 sec latency 1558.109 ms 2023-12-07T10:35:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.154+0000 7ff32d7fa700 1 -- 172.21.15.119:0/4024614935 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5572be27af70 con 0x5572be3d9520 2023-12-07T10:35:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.154+0000 7ff32d7fa700 10 client.27973 handle_command_reply: tid=0 2023-12-07T10:35:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.154+0000 7ff35f486b80 1 client.27973 shutdown 2023-12-07T10:35:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.155+0000 7ff35f486b80 1 -- 172.21.15.119:0/4024614935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572be369710 msgr2=0x5572be355260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.155+0000 7ff35f486b80 1 --2- 172.21.15.119:0/4024614935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572be369710 0x5572be355260 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff3340cad70 tx=0x7ff334099f80 comp rx=0 tx=0).stop 2023-12-07T10:35:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.155+0000 7ff35f486b80 1 -- 172.21.15.119:0/4024614935 shutdown_connections 2023-12-07T10:35:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.155+0000 7ff35f486b80 1 --2- 172.21.15.119:0/4024614935 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5572be3d9520 0x5572be3db940 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.155+0000 7ff35f486b80 1 --2- 172.21.15.119:0/4024614935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5572be36d1a0 0x5572be3558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.155+0000 7ff35f486b80 1 --2- 172.21.15.119:0/4024614935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5572be368d50 0x5572be379b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.155+0000 7ff35f486b80 1 --2- 172.21.15.119:0/4024614935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572be369710 0x5572be355260 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.155+0000 7ff35f486b80 1 -- 172.21.15.119:0/4024614935 >> 172.21.15.119:0/4024614935 conn(0x5572be28f300 msgr2=0x5572be36b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:35:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.155+0000 7ff32d7fa700 0 client.27973 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:35:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.156+0000 7ff35f486b80 1 -- 172.21.15.119:0/4024614935 shutdown_connections 2023-12-07T10:35:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.156+0000 7ff35f486b80 1 -- 172.21.15.119:0/4024614935 wait complete. 2023-12-07T10:35:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.157+0000 7ff35f486b80 20 client.27973 trim_cache size 0 max 16384 2023-12-07T10:35:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:35:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:35:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:35:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:35:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.158+0000 7ff34bbff700 1 -- 172.21.15.119:0/1278679165 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff3300a6190 msgr2=0x7ff3300a8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.158+0000 7ff34bbff700 1 --2- 172.21.15.119:0/1278679165 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff3300a6190 0x7ff3300a8650 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff34000b5c0 tx=0x7ff340005fb0 comp rx=0 tx=0).stop 2023-12-07T10:35:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.158+0000 7ff34bbff700 1 -- 172.21.15.119:0/1278679165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff344101ec0 msgr2=0x7ff3440730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.158+0000 7ff34bbff700 1 --2- 172.21.15.119:0/1278679165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff344101ec0 0x7ff3440730b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff33400b550 tx=0x7ff33400b910 comp rx=0 tx=0).stop 2023-12-07T10:35:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.158+0000 7ff34bbff700 1 -- 172.21.15.119:0/1278679165 shutdown_connections 2023-12-07T10:35:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.158+0000 7ff34bbff700 1 --2- 172.21.15.119:0/1278679165 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7ff3300a6190 0x7ff3300a8650 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.158+0000 7ff34bbff700 1 --2- 172.21.15.119:0/1278679165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3441075c0 0x7ff34406bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.158+0000 7ff34bbff700 1 --2- 172.21.15.119:0/1278679165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff344101500 0x7ff344072b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.158+0000 7ff34bbff700 1 --2- 172.21.15.119:0/1278679165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff344101ec0 0x7ff3440730b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.158+0000 7ff34bbff700 1 -- 172.21.15.119:0/1278679165 >> 172.21.15.119:0/1278679165 conn(0x7ff3440fcfb0 msgr2=0x7ff3440fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:35:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.158+0000 7ff34bbff700 1 -- 172.21.15.119:0/1278679165 shutdown_connections 2023-12-07T10:35:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:38.158+0000 7ff34bbff700 1 -- 172.21.15.119:0/1278679165 wait complete. 2023-12-07T10:35:38.513 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:38 smithi159 ceph-mon[134726]: pgmap v477: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 882 op/s 2023-12-07T10:35:38.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:38 smithi153 ceph-mon[133625]: pgmap v477: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 882 op/s 2023-12-07T10:35:38.727 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:38 smithi119 ceph-mon[97908]: pgmap v477: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 882 op/s 2023-12-07T10:35:38.975 INFO:tasks.workunit.client.0.smithi119.stdout: 10 66241 33.73 MB/sec warmup 111 sec latency 1519.157 ms 2023-12-07T10:35:39.975 INFO:tasks.workunit.client.0.smithi119.stdout: 10 66561 33.57 MB/sec warmup 112 sec latency 140.687 ms 2023-12-07T10:35:40.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:40 smithi159 ceph-mon[134726]: pgmap v478: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 8.9 MiB/s wr, 577 op/s 2023-12-07T10:35:40.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:40 smithi153 ceph-mon[133625]: pgmap v478: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 8.9 MiB/s wr, 577 op/s 2023-12-07T10:35:40.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:40 smithi119 ceph-mon[97908]: pgmap v478: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 8.9 MiB/s wr, 577 op/s 2023-12-07T10:35:40.975 INFO:tasks.workunit.client.0.smithi119.stdout: 10 66933 33.47 MB/sec warmup 113 sec latency 131.215 ms 2023-12-07T10:35:41.975 INFO:tasks.workunit.client.0.smithi119.stdout: 10 67859 33.56 MB/sec warmup 114 sec latency 72.815 ms 2023-12-07T10:35:42.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:42 smithi119 ceph-mon[97908]: pgmap v479: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 719 op/s 2023-12-07T10:35:42.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:42 smithi159 ceph-mon[134726]: pgmap v479: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 719 op/s 2023-12-07T10:35:42.527 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:42 smithi153 ceph-mon[133625]: pgmap v479: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 719 op/s 2023-12-07T10:35:42.975 INFO:tasks.workunit.client.0.smithi119.stdout: 10 68723 33.77 MB/sec warmup 115 sec latency 83.487 ms 2023-12-07T10:35:43.975 INFO:tasks.workunit.client.0.smithi119.stdout: 10 69259 33.66 MB/sec warmup 116 sec latency 92.165 ms 2023-12-07T10:35:44.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:44 smithi119 ceph-mon[97908]: pgmap v480: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 916 op/s 2023-12-07T10:35:44.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:44 smithi159 ceph-mon[134726]: pgmap v480: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 916 op/s 2023-12-07T10:35:44.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:44 smithi153 ceph-mon[133625]: pgmap v480: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 916 op/s 2023-12-07T10:35:44.975 INFO:tasks.workunit.client.0.smithi119.stdout: 10 70052 33.73 MB/sec warmup 117 sec latency 58.747 ms 2023-12-07T10:35:45.975 INFO:tasks.workunit.client.0.smithi119.stdout: 10 70861 33.91 MB/sec warmup 118 sec latency 70.292 ms 2023-12-07T10:35:46.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:46 smithi119 ceph-mon[97908]: pgmap v481: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 12 MiB/s wr, 763 op/s 2023-12-07T10:35:46.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:46 smithi159 ceph-mon[134726]: pgmap v481: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 12 MiB/s wr, 763 op/s 2023-12-07T10:35:46.527 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:46 smithi153 ceph-mon[133625]: pgmap v481: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 12 MiB/s wr, 763 op/s 2023-12-07T10:35:46.975 INFO:tasks.workunit.client.0.smithi119.stdout: 10 71546 33.85 MB/sec warmup 119 sec latency 83.654 ms 2023-12-07T10:35:48.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:48 smithi119 ceph-mon[97908]: pgmap v482: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 936 op/s 2023-12-07T10:35:48.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:48 smithi159 ceph-mon[134726]: pgmap v482: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 936 op/s 2023-12-07T10:35:48.527 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:48 smithi153 ceph-mon[133625]: pgmap v482: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 936 op/s 2023-12-07T10:35:48.975 INFO:tasks.workunit.client.0.smithi119.stdout: 10 73074 37.23 MB/sec execute 1 sec latency 72.619 ms 2023-12-07T10:35:49.975 INFO:tasks.workunit.client.0.smithi119.stdout: 10 73933 42.64 MB/sec execute 2 sec latency 64.766 ms 2023-12-07T10:35:50.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:50 smithi119 ceph-mon[97908]: pgmap v483: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 759 op/s 2023-12-07T10:35:50.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:50 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:35:50.515 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:50 smithi159 ceph-mon[134726]: pgmap v483: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 759 op/s 2023-12-07T10:35:50.516 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:50 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:35:50.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:50 smithi153 ceph-mon[133625]: pgmap v483: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 759 op/s 2023-12-07T10:35:50.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:50 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:35:50.975 INFO:tasks.workunit.client.0.smithi119.stdout: 10 74607 41.38 MB/sec execute 3 sec latency 76.991 ms 2023-12-07T10:35:51.975 INFO:tasks.workunit.client.0.smithi119.stdout: 10 74901 33.75 MB/sec execute 4 sec latency 449.696 ms 2023-12-07T10:35:52.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:52 smithi119 ceph-mon[97908]: pgmap v484: 129 pgs: 129 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 830 op/s 2023-12-07T10:35:52.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:52 smithi159 ceph-mon[134726]: pgmap v484: 129 pgs: 129 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 830 op/s 2023-12-07T10:35:52.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:52 smithi153 ceph-mon[133625]: pgmap v484: 129 pgs: 129 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 830 op/s 2023-12-07T10:35:52.976 INFO:tasks.workunit.client.0.smithi119.stdout: 10 75045 29.03 MB/sec execute 5 sec latency 1015.163 ms 2023-12-07T10:35:53.976 INFO:tasks.workunit.client.0.smithi119.stdout: 10 75329 26.27 MB/sec execute 6 sec latency 171.604 ms 2023-12-07T10:35:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.328+0000 7f0c083ff700 1 -- 172.21.15.119:0/2894630235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c001040d0 msgr2=0x7f0c001105e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.328+0000 7f0c083ff700 1 --2- 172.21.15.119:0/2894630235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c001040d0 0x7f0c001105e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000b0a0 tx=0x7f0bf000b3b0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.328+0000 7f0c083ff700 1 -- 172.21.15.119:0/2894630235 shutdown_connections 2023-12-07T10:35:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.328+0000 7f0c083ff700 1 --2- 172.21.15.119:0/2894630235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c001040d0 0x7f0c001105e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.328+0000 7f0c083ff700 1 --2- 172.21.15.119:0/2894630235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c00109720 0x7f0c00109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.328+0000 7f0c083ff700 1 --2- 172.21.15.119:0/2894630235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c00103710 0x7f0c00103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.328+0000 7f0c083ff700 1 -- 172.21.15.119:0/2894630235 >> 172.21.15.119:0/2894630235 conn(0x7f0c000ff1c0 msgr2=0x7f0c001015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:35:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.328+0000 7f0c083ff700 1 -- 172.21.15.119:0/2894630235 shutdown_connections 2023-12-07T10:35:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.329+0000 7f0c083ff700 1 -- 172.21.15.119:0/2894630235 wait complete. 2023-12-07T10:35:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.330+0000 7f0c083ff700 1 Processor -- start 2023-12-07T10:35:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.330+0000 7f0c083ff700 1 -- start start 2023-12-07T10:35:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.331+0000 7f0c083ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c00103710 0x7f0c00073110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.331+0000 7f0c083ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c001040d0 0x7f0c0006b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.331+0000 7f0c083ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c00109720 0x7f0c0006b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.331+0000 7f0c083ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0006bdf0 con 0x7f0c00109720 2023-12-07T10:35:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.331+0000 7f0c083ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0006bf60 con 0x7f0c001040d0 2023-12-07T10:35:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.331+0000 7f0c083ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c001ae860 con 0x7f0c00103710 2023-12-07T10:35:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.331+0000 7f0c0619b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c00103710 0x7f0c00073110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.332+0000 7f0c0599a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c001040d0 0x7f0c0006b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.332+0000 7f0c0599a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c001040d0 0x7f0c0006b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:47914/0 (socket says 172.21.15.119:47914) 2023-12-07T10:35:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.332+0000 7f0c0599a700 1 -- 172.21.15.119:0/2235533065 learned_addr learned my addr 172.21.15.119:0/2235533065 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:35:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.332+0000 7f0c0699c700 1 --2- 172.21.15.119:0/2235533065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c00109720 0x7f0c0006b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.332+0000 7f0c0699c700 1 -- 172.21.15.119:0/2235533065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c00103710 msgr2=0x7f0c00073110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.332+0000 7f0c0699c700 1 --2- 172.21.15.119:0/2235533065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c00103710 0x7f0c00073110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.332+0000 7f0c0699c700 1 -- 172.21.15.119:0/2235533065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c001040d0 msgr2=0x7f0c0006b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.332+0000 7f0c0699c700 1 --2- 172.21.15.119:0/2235533065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c001040d0 0x7f0c0006b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.332+0000 7f0c0699c700 1 -- 172.21.15.119:0/2235533065 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf0009d00 con 0x7f0c00109720 2023-12-07T10:35:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.332+0000 7f0c0599a700 1 --2- 172.21.15.119:0/2235533065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c001040d0 0x7f0c0006b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:35:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.333+0000 7f0c0699c700 1 --2- 172.21.15.119:0/2235533065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c00109720 0x7f0c0006b660 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000b0a0 tx=0x7f0bf0007b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:35:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.333+0000 7f0bff7fe700 1 -- 172.21.15.119:0/2235533065 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf000e040 con 0x7f0c00109720 2023-12-07T10:35:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.333+0000 7f0bff7fe700 1 -- 172.21.15.119:0/2235533065 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bf0012e40 con 0x7f0c00109720 2023-12-07T10:35:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.333+0000 7f0c083ff700 1 -- 172.21.15.119:0/2235533065 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c001aeae0 con 0x7f0c00109720 2023-12-07T10:35:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.333+0000 7f0bff7fe700 1 -- 172.21.15.119:0/2235533065 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf001b450 con 0x7f0c00109720 2023-12-07T10:35:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.333+0000 7f0c083ff700 1 -- 172.21.15.119:0/2235533065 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c001aef60 con 0x7f0c00109720 2023-12-07T10:35:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.342+0000 7f0bff7fe700 1 -- 172.21.15.119:0/2235533065 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f0bf001d710 con 0x7f0c00109720 2023-12-07T10:35:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.346+0000 7f0bff7fe700 1 --2- 172.21.15.119:0/2235533065 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0be80a1d00 0x7f0be80a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.347+0000 7f0bff7fe700 1 -- 172.21.15.119:0/2235533065 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0bf00cfc60 con 0x7f0c00109720 2023-12-07T10:35:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.348+0000 7f0c0619b700 1 --2- 172.21.15.119:0/2235533065 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0be80a1d00 0x7f0be80a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.349+0000 7f0c0619b700 1 --2- 172.21.15.119:0/2235533065 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0be80a1d00 0x7f0be80a41c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8005950 tx=0x7f0bf8005c70 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:35:54.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:54 smithi119 ceph-mon[97908]: pgmap v485: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 804 op/s 2023-12-07T10:35:54.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:54 smithi159 ceph-mon[134726]: pgmap v485: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 804 op/s 2023-12-07T10:35:54.527 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:54 smithi153 ceph-mon[133625]: pgmap v485: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 804 op/s 2023-12-07T10:35:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.749+0000 7f0c1bde9b80 1 Processor -- start 2023-12-07T10:35:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.750+0000 7f0c1bde9b80 1 -- start start 2023-12-07T10:35:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.750+0000 7f0c1bde9b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556dc023d270 0x556dc023d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.750+0000 7f0c1bde9b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dc023db90 0x556dc0237d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.750+0000 7f0c1bde9b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556dc02382e0 0x556dc0238740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.750+0000 7f0c1bde9b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dc01a3760 con 0x556dc023db90 2023-12-07T10:35:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.750+0000 7f0c1bde9b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dc018a740 con 0x556dc023d270 2023-12-07T10:35:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.750+0000 7f0c1bde9b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dc0188060 con 0x556dc02382e0 2023-12-07T10:35:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.751+0000 7f0c0699c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556dc02382e0 0x556dc0238740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.752+0000 7f0c0699c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556dc02382e0 0x556dc0238740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:45704/0 (socket says 172.21.15.119:45704) 2023-12-07T10:35:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.752+0000 7f0c0699c700 1 -- 172.21.15.119:0/4120805126 learned_addr learned my addr 172.21.15.119:0/4120805126 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:35:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.751+0000 7f0c0619b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556dc023d270 0x556dc023d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.752+0000 7f0c0699c700 1 -- 172.21.15.119:0/4120805126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556dc023d270 msgr2=0x556dc023d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.752+0000 7f0c0699c700 1 --2- 172.21.15.119:0/4120805126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556dc023d270 0x556dc023d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.752+0000 7f0c0699c700 1 -- 172.21.15.119:0/4120805126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dc023db90 msgr2=0x556dc0237d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.752+0000 7f0c0699c700 1 --2- 172.21.15.119:0/4120805126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dc023db90 0x556dc0237d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.752+0000 7f0c0699c700 1 -- 172.21.15.119:0/4120805126 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556dc00005d0 con 0x556dc02382e0 2023-12-07T10:35:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.753+0000 7f0c0699c700 1 --2- 172.21.15.119:0/4120805126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556dc02382e0 0x556dc0238740 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0bf009c670 tx=0x7f0bf001dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c32add97d9cbecf9 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:35:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.753+0000 7f0bfcbfe700 1 -- 172.21.15.119:0/4120805126 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf00a0830 con 0x556dc02382e0 2023-12-07T10:35:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.753+0000 7f0bfcbfe700 1 -- 172.21.15.119:0/4120805126 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bf00a49e0 con 0x556dc02382e0 2023-12-07T10:35:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.753+0000 7f0bfcbfe700 1 -- 172.21.15.119:0/4120805126 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf009bbd0 con 0x556dc02382e0 2023-12-07T10:35:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.754+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/4120805126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556dc02382e0 msgr2=0x556dc0238740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.754+0000 7f0c1bde9b80 1 --2- 172.21.15.119:0/4120805126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556dc02382e0 0x556dc0238740 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0bf009c670 tx=0x7f0bf001dab0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.755+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/4120805126 shutdown_connections 2023-12-07T10:35:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.755+0000 7f0c1bde9b80 1 --2- 172.21.15.119:0/4120805126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556dc02382e0 0x556dc0238740 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.755+0000 7f0c1bde9b80 1 --2- 172.21.15.119:0/4120805126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556dc023d270 0x556dc023d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.755+0000 7f0c1bde9b80 1 --2- 172.21.15.119:0/4120805126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dc023db90 0x556dc0237d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.755+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/4120805126 >> 172.21.15.119:0/4120805126 conn(0x556dc015e300 msgr2=0x556dc0229b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:35:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.757+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/4120805126 shutdown_connections 2023-12-07T10:35:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.757+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/4120805126 wait complete. 2023-12-07T10:35:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.758+0000 7f0c1bde9b80 1 -- start start 2023-12-07T10:35:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.758+0000 7f0c1bde9b80 1 Processor -- start 2023-12-07T10:35:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.760+0000 7f0c1bde9b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556dc02382e0 0x556dc0240f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.760+0000 7f0c0619b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556dc02382e0 0x556dc0240f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.760+0000 7f0c0619b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556dc02382e0 0x556dc0240f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:47932/0 (socket says 172.21.15.119:47932) 2023-12-07T10:35:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.762+0000 7f0c0619b700 1 -- 172.21.15.119:0/3436177126 learned_addr learned my addr 172.21.15.119:0/3436177126 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:35:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.762+0000 7f0c1bde9b80 1 --2- 172.21.15.119:0/3436177126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dc023d270 0x556dc02414d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.762+0000 7f0c1bde9b80 1 --2- 172.21.15.119:0/3436177126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556dc023db90 0x556dc0241a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.762+0000 7f0c0699c700 1 --2- 172.21.15.119:0/3436177126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556dc023db90 0x556dc0241a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.763+0000 7f0c0599a700 1 --2- 172.21.15.119:0/3436177126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dc023d270 0x556dc02414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.763+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/3436177126 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dc0183950 con 0x556dc023d270 2023-12-07T10:35:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.763+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/3436177126 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dc013d500 con 0x556dc02382e0 2023-12-07T10:35:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.763+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/3436177126 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dc013d1a0 con 0x556dc023db90 2023-12-07T10:35:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.763+0000 7f0c1bde9b80 10 client.0 _ll_register_callbacks cb 0x556dc01248e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:35:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.764+0000 7f0c0619b700 1 -- 172.21.15.119:0/3436177126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556dc023db90 msgr2=0x556dc0241a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.764+0000 7f0c0619b700 1 --2- 172.21.15.119:0/3436177126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556dc023db90 0x556dc0241a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.764+0000 7f0c0619b700 1 -- 172.21.15.119:0/3436177126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dc023d270 msgr2=0x556dc02414d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.764+0000 7f0c0619b700 1 --2- 172.21.15.119:0/3436177126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dc023d270 0x556dc02414d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.764+0000 7f0c0619b700 1 -- 172.21.15.119:0/3436177126 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c001aef60 con 0x556dc02382e0 2023-12-07T10:35:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.764+0000 7f0c0619b700 1 --2- 172.21.15.119:0/3436177126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556dc02382e0 0x556dc0240f90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800f7a0 tx=0x7f0bf8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3008c720e070271c server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:35:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.764+0000 7f0be5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-12-07T10:35:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.765+0000 7f0be5ffb700 1 -- 172.21.15.119:0/3436177126 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf801d840 con 0x556dc02382e0 2023-12-07T10:35:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.765+0000 7f0be5ffb700 1 -- 172.21.15.119:0/3436177126 <== mon.2 v2:172.21.15.153:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0bf801d9a0 con 0x556dc02382e0 2023-12-07T10:35:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.765+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/3436177126 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556dc02226f0 con 0x556dc02382e0 2023-12-07T10:35:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.766+0000 7f0be5ffb700 1 -- 172.21.15.119:0/3436177126 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bf801db60 con 0x556dc02382e0 2023-12-07T10:35:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.766+0000 7f0c1bde9b80 10 client.28007 fetch_fsmap learned FSMap version 72 2023-12-07T10:35:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.766+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/3436177126 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x556dbffacd80 con 0x556dc02382e0 2023-12-07T10:35:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.766+0000 7f0be5ffb700 1 -- 172.21.15.119:0/3436177126 <== mon.2 v2:172.21.15.153:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556dc02226f0 con 0x556dc02382e0 2023-12-07T10:35:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.766+0000 7f0be5ffb700 1 -- 172.21.15.119:0/3436177126 <== mon.2 v2:172.21.15.153:3300/0 5 ==== fsmap(e 72) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f0bf8023020 con 0x556dc02382e0 2023-12-07T10:35:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.767+0000 7f0c1bde9b80 10 client.28007 fetch_fsmap finished waiting for FSMap version 72 2023-12-07T10:35:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.767+0000 7f0c1bde9b80 10 client.28007 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:35:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.767+0000 7f0c1bde9b80 20 client.28007 populate_metadata read hostname 'smithi119' 2023-12-07T10:35:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.767+0000 7f0c1bde9b80 1 --2- 172.21.15.119:0/3436177126 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556dc0251610 0x556dc0253a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.767+0000 7f0c1bde9b80 4 client.28007 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:35:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.767+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/3436177126 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556dc01fe740 con 0x556dc0251610 2023-12-07T10:35:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.767+0000 7f0c0599a700 1 --2- 172.21.15.119:0/3436177126 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556dc0251610 0x556dc0253a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.768+0000 7f0c0599a700 1 --2- 172.21.15.119:0/3436177126 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556dc0251610 0x556dc0253a30 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcae035e160a2a24 server_cookie=de5eabd2394dd6ef in_seq=0 out_seq=0 2023-12-07T10:35:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.768+0000 7f0be5ffb700 10 client.28007 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:35:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.769+0000 7f0be5ffb700 1 -- 172.21.15.119:0/3436177126 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556dc01fe740 con 0x556dc0251610 2023-12-07T10:35:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.769+0000 7f0be5ffb700 10 client.28007 handle_command_reply: tid=0 2023-12-07T10:35:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.769+0000 7f0c1bde9b80 1 client.28007 shutdown 2023-12-07T10:35:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.769+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/3436177126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556dc02382e0 msgr2=0x556dc0240f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.769+0000 7f0c1bde9b80 1 --2- 172.21.15.119:0/3436177126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556dc02382e0 0x556dc0240f90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800f7a0 tx=0x7f0bf8004750 comp rx=0 tx=0).stop 2023-12-07T10:35:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.770+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/3436177126 shutdown_connections 2023-12-07T10:35:54.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.770+0000 7f0c1bde9b80 1 --2- 172.21.15.119:0/3436177126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556dc023db90 0x556dc0241a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.770+0000 7f0c1bde9b80 1 --2- 172.21.15.119:0/3436177126 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556dc0251610 0x556dc0253a30 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.770+0000 7f0c1bde9b80 1 --2- 172.21.15.119:0/3436177126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556dc02382e0 0x556dc0240f90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.770+0000 7f0c1bde9b80 1 --2- 172.21.15.119:0/3436177126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dc023d270 0x556dc02414d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.770+0000 7f0be5ffb700 0 client.28007 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:35:54.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.770+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/3436177126 >> 172.21.15.119:0/3436177126 conn(0x556dc015e300 msgr2=0x556dc0224980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:35:54.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.770+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/3436177126 shutdown_connections 2023-12-07T10:35:54.771 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.770+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/3436177126 wait complete. 2023-12-07T10:35:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.771+0000 7f0c1bde9b80 20 client.28007 trim_cache size 0 max 16384 2023-12-07T10:35:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.802+0000 7f0c1bde9b80 1 Processor -- start 2023-12-07T10:35:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.802+0000 7f0c1bde9b80 1 -- start start 2023-12-07T10:35:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.803+0000 7f0c1bde9b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dc02382e0 0x556dc0237d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.803+0000 7f0c1bde9b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556dc023d270 0x556dc0241830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.803+0000 7f0c1bde9b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556dc023db90 0x556dc02483a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.803+0000 7f0c1bde9b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dc00028c0 con 0x556dc02382e0 2023-12-07T10:35:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.803+0000 7f0c1bde9b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dc0243f30 con 0x556dc023d270 2023-12-07T10:35:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.803+0000 7f0c1bde9b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dc0242530 con 0x556dc023db90 2023-12-07T10:35:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.803+0000 7f0c0599a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556dc023d270 0x556dc0241830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.803+0000 7f0c0599a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556dc023d270 0x556dc0241830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:47942/0 (socket says 172.21.15.119:47942) 2023-12-07T10:35:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.804+0000 7f0c0599a700 1 -- 172.21.15.119:0/260266495 learned_addr learned my addr 172.21.15.119:0/260266495 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:35:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.804+0000 7f0c0699c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556dc023db90 0x556dc02483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.804+0000 7f0c0599a700 1 -- 172.21.15.119:0/260266495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556dc023db90 msgr2=0x556dc02483a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.804+0000 7f0c0599a700 1 --2- 172.21.15.119:0/260266495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556dc023db90 0x556dc02483a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.804+0000 7f0c0599a700 1 -- 172.21.15.119:0/260266495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dc02382e0 msgr2=0x556dc0237d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.804+0000 7f0c0599a700 1 --2- 172.21.15.119:0/260266495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dc02382e0 0x556dc0237d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.804+0000 7f0c0599a700 1 -- 172.21.15.119:0/260266495 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556dbff95c00 con 0x556dc023d270 2023-12-07T10:35:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.805+0000 7f0c0599a700 1 --2- 172.21.15.119:0/260266495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556dc023d270 0x556dc0241830 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0c0006c850 tx=0x7f0bf400abb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d28166e311114f08 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:35:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.807+0000 7f0bfd3ff700 1 -- 172.21.15.119:0/260266495 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4017070 con 0x556dc023d270 2023-12-07T10:35:54.808 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.807+0000 7f0bfd3ff700 1 -- 172.21.15.119:0/260266495 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bf4018040 con 0x556dc023d270 2023-12-07T10:35:54.808 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.807+0000 7f0bfd3ff700 1 -- 172.21.15.119:0/260266495 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4019780 con 0x556dc023d270 2023-12-07T10:35:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.808+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/260266495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556dc023d270 msgr2=0x556dc0241830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.808+0000 7f0c1bde9b80 1 --2- 172.21.15.119:0/260266495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556dc023d270 0x556dc0241830 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0c0006c850 tx=0x7f0bf400abb0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.808+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/260266495 shutdown_connections 2023-12-07T10:35:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.808+0000 7f0c1bde9b80 1 --2- 172.21.15.119:0/260266495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556dc023db90 0x556dc02483a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.808+0000 7f0c1bde9b80 1 --2- 172.21.15.119:0/260266495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dc02382e0 0x556dc0237d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.808+0000 7f0c1bde9b80 1 --2- 172.21.15.119:0/260266495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556dc023d270 0x556dc0241830 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.809+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/260266495 >> 172.21.15.119:0/260266495 conn(0x556dc015e300 msgr2=0x556dc023b220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:35:54.810 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.809+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/260266495 shutdown_connections 2023-12-07T10:35:54.810 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.809+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/260266495 wait complete. 2023-12-07T10:35:54.810 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.810+0000 7f0c1bde9b80 1 -- start start 2023-12-07T10:35:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.810+0000 7f0c1bde9b80 1 Processor -- start 2023-12-07T10:35:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.812+0000 7f0c1bde9b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dc02382e0 0x556dc0246570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.812+0000 7f0c1bde9b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556dc023d270 0x556dc0246ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.812+0000 7f0c1bde9b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556dc023db90 0x556dc02a6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.813+0000 7f0c1bde9b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dc02a7010 con 0x556dc02382e0 2023-12-07T10:35:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.813+0000 7f0c0599a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556dc023d270 0x556dc0246ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.813+0000 7f0c0599a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556dc023d270 0x556dc0246ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:47952/0 (socket says 172.21.15.119:47952) 2023-12-07T10:35:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.813+0000 7f0c0599a700 1 -- 172.21.15.119:0/119060107 learned_addr learned my addr 172.21.15.119:0/119060107 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:35:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.813+0000 7f0c1bde9b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dc02a7150 con 0x556dc023d270 2023-12-07T10:35:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.813+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/119060107 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dc02a7290 con 0x556dc023db90 2023-12-07T10:35:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.813+0000 7f0c1bde9b80 10 client.0 _ll_register_callbacks cb 0x556dc01248e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:35:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.815+0000 7f0c0699c700 1 --2- 172.21.15.119:0/119060107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556dc023db90 0x556dc02a6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.815+0000 7f0c0619b700 1 --2- 172.21.15.119:0/119060107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dc02382e0 0x556dc0246570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.816+0000 7f0c0619b700 1 -- 172.21.15.119:0/119060107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556dc023db90 msgr2=0x556dc02a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.816+0000 7f0c0619b700 1 --2- 172.21.15.119:0/119060107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556dc023db90 0x556dc02a6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.816+0000 7f0c0619b700 1 -- 172.21.15.119:0/119060107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556dc023d270 msgr2=0x556dc0246ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:35:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.816+0000 7f0c0619b700 1 --2- 172.21.15.119:0/119060107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556dc023d270 0x556dc0246ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.816+0000 7f0c0619b700 1 -- 172.21.15.119:0/119060107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf4011440 con 0x556dc02382e0 2023-12-07T10:35:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.817+0000 7f0c0619b700 1 --2- 172.21.15.119:0/119060107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dc02382e0 0x556dc0246570 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0bf802fef0 tx=0x7f0bf802eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edbf049b7896ad74 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:35:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.817+0000 7f0be5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:35:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.817+0000 7f0be5ffb700 1 -- 172.21.15.119:0/119060107 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8005440 con 0x556dc02382e0 2023-12-07T10:35:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.818+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/119060107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0bf00a49e0 con 0x556dc02382e0 2023-12-07T10:35:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.818+0000 7f0be5ffb700 1 -- 172.21.15.119:0/119060107 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f0bf800fe40 con 0x556dc02382e0 2023-12-07T10:35:54.820 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.819+0000 7f0be5ffb700 1 -- 172.21.15.119:0/119060107 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bf802ee40 con 0x556dc02382e0 2023-12-07T10:35:54.820 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.819+0000 7f0c1bde9b80 10 client.17922 fetch_fsmap learned FSMap version 72 2023-12-07T10:35:54.820 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.820+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/119060107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x556dbffd31f0 con 0x556dc02382e0 2023-12-07T10:35:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.822+0000 7f0be5ffb700 1 -- 172.21.15.119:0/119060107 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bf00a49e0 con 0x556dc02382e0 2023-12-07T10:35:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.822+0000 7f0be5ffb700 1 -- 172.21.15.119:0/119060107 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 72) v1 ==== 7954+0+0 (secure 0 0 0) 0x7f0bf8023040 con 0x556dc02382e0 2023-12-07T10:35:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.823+0000 7f0c1bde9b80 10 client.17922 fetch_fsmap finished waiting for FSMap version 72 2023-12-07T10:35:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.823+0000 7f0c1bde9b80 10 client.17922 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:35:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.823+0000 7f0c1bde9b80 20 client.17922 populate_metadata read hostname 'smithi119' 2023-12-07T10:35:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.823+0000 7f0c1bde9b80 1 --2- 172.21.15.119:0/119060107 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556dc02a8ba0 0x556dc02aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:35:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.823+0000 7f0c1bde9b80 4 client.17922 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:35:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.823+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/119060107 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556dc0149f70 con 0x556dc02a8ba0 2023-12-07T10:35:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.825+0000 7f0c0599a700 1 --2- 172.21.15.119:0/119060107 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556dc02a8ba0 0x556dc02aafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:35:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.825+0000 7f0c0599a700 1 --2- 172.21.15.119:0/119060107 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556dc02a8ba0 0x556dc02aafc0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de41899a485f4aa1 server_cookie=28c70c2f24c63fb6 in_seq=0 out_seq=0 2023-12-07T10:35:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.825+0000 7f0be5ffb700 10 client.17922 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:35:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.964+0000 7f0be5ffb700 1 -- 172.21.15.119:0/119060107 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556dc0149f70 con 0x556dc02a8ba0 2023-12-07T10:35:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.964+0000 7f0be5ffb700 10 client.17922 handle_command_reply: tid=0 2023-12-07T10:35:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.964+0000 7f0c1bde9b80 1 client.17922 shutdown 2023-12-07T10:35:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.965+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/119060107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dc02382e0 msgr2=0x556dc0246570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.965+0000 7f0c1bde9b80 1 --2- 172.21.15.119:0/119060107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dc02382e0 0x556dc0246570 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0bf802fef0 tx=0x7f0bf802eb20 comp rx=0 tx=0).stop 2023-12-07T10:35:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.965+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/119060107 shutdown_connections 2023-12-07T10:35:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.965+0000 7f0c1bde9b80 1 --2- 172.21.15.119:0/119060107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556dc023db90 0x556dc02a6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.965+0000 7f0c1bde9b80 1 --2- 172.21.15.119:0/119060107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dc02382e0 0x556dc0246570 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.965+0000 7f0c1bde9b80 1 --2- 172.21.15.119:0/119060107 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x556dc02a8ba0 0x556dc02aafc0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.965+0000 7f0c1bde9b80 1 --2- 172.21.15.119:0/119060107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x556dc023d270 0x556dc0246ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.965+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/119060107 >> 172.21.15.119:0/119060107 conn(0x556dc015e300 msgr2=0x556dc023f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:35:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.965+0000 7f0be5ffb700 0 client.17922 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:35:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.965+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/119060107 shutdown_connections 2023-12-07T10:35:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.965+0000 7f0c1bde9b80 1 -- 172.21.15.119:0/119060107 wait complete. 2023-12-07T10:35:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.966+0000 7f0c1bde9b80 20 client.17922 trim_cache size 0 max 16384 2023-12-07T10:35:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:35:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:35:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:35:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:35:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.967+0000 7f0c083ff700 1 -- 172.21.15.119:0/2235533065 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0be80a1d00 msgr2=0x7f0be80a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.967+0000 7f0c083ff700 1 --2- 172.21.15.119:0/2235533065 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0be80a1d00 0x7f0be80a41c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8005950 tx=0x7f0bf8005c70 comp rx=0 tx=0).stop 2023-12-07T10:35:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.967+0000 7f0c083ff700 1 -- 172.21.15.119:0/2235533065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c00109720 msgr2=0x7f0c0006b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:35:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.967+0000 7f0c083ff700 1 --2- 172.21.15.119:0/2235533065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c00109720 0x7f0c0006b660 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000b0a0 tx=0x7f0bf0007b60 comp rx=0 tx=0).stop 2023-12-07T10:35:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.968+0000 7f0c083ff700 1 -- 172.21.15.119:0/2235533065 shutdown_connections 2023-12-07T10:35:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.968+0000 7f0c083ff700 1 --2- 172.21.15.119:0/2235533065 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f0be80a1d00 0x7f0be80a41c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.968+0000 7f0c083ff700 1 --2- 172.21.15.119:0/2235533065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c00109720 0x7f0c0006b660 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.968+0000 7f0c083ff700 1 --2- 172.21.15.119:0/2235533065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c00103710 0x7f0c00073110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.968+0000 7f0c083ff700 1 --2- 172.21.15.119:0/2235533065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c001040d0 0x7f0c0006b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:35:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.968+0000 7f0c083ff700 1 -- 172.21.15.119:0/2235533065 >> 172.21.15.119:0/2235533065 conn(0x7f0c000ff1c0 msgr2=0x7f0c000ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:35:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.968+0000 7f0c083ff700 1 -- 172.21.15.119:0/2235533065 shutdown_connections 2023-12-07T10:35:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:35:54.968+0000 7f0c083ff700 1 -- 172.21.15.119:0/2235533065 wait complete. 2023-12-07T10:35:54.975 INFO:tasks.workunit.client.0.smithi119.stdout: 10 75715 25.50 MB/sec execute 7 sec latency 140.430 ms 2023-12-07T10:35:55.976 INFO:tasks.workunit.client.0.smithi119.stdout: 10 76419 27.50 MB/sec execute 8 sec latency 151.849 ms 2023-12-07T10:35:56.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:56 smithi119 ceph-mon[97908]: pgmap v486: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 7.6 MiB/s wr, 529 op/s 2023-12-07T10:35:56.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:56 smithi159 ceph-mon[134726]: pgmap v486: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 7.6 MiB/s wr, 529 op/s 2023-12-07T10:35:56.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:56 smithi153 ceph-mon[133625]: pgmap v486: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 7.6 MiB/s wr, 529 op/s 2023-12-07T10:35:56.976 INFO:tasks.workunit.client.0.smithi119.stdout: 10 77098 28.46 MB/sec execute 9 sec latency 76.623 ms 2023-12-07T10:35:57.976 INFO:tasks.workunit.client.0.smithi119.stdout: 10 77701 29.23 MB/sec execute 10 sec latency 88.788 ms 2023-12-07T10:35:58.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:35:58 smithi119 ceph-mon[97908]: pgmap v487: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 828 op/s 2023-12-07T10:35:58.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:35:58 smithi159 ceph-mon[134726]: pgmap v487: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 828 op/s 2023-12-07T10:35:58.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:35:58 smithi153 ceph-mon[133625]: pgmap v487: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 828 op/s 2023-12-07T10:35:58.976 INFO:tasks.workunit.client.0.smithi119.stdout: 10 78406 29.39 MB/sec execute 11 sec latency 58.570 ms 2023-12-07T10:35:59.976 INFO:tasks.workunit.client.0.smithi119.stdout: 10 79156 30.35 MB/sec execute 12 sec latency 74.409 ms 2023-12-07T10:36:00.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:00 smithi119 ceph-mon[97908]: pgmap v488: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 13 MiB/s wr, 722 op/s 2023-12-07T10:36:00.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:00 smithi159 ceph-mon[134726]: pgmap v488: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 13 MiB/s wr, 722 op/s 2023-12-07T10:36:00.527 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:00 smithi153 ceph-mon[133625]: pgmap v488: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 13 MiB/s wr, 722 op/s 2023-12-07T10:36:00.976 INFO:tasks.workunit.client.0.smithi119.stdout: 10 79765 30.41 MB/sec execute 13 sec latency 231.028 ms 2023-12-07T10:36:01.977 INFO:tasks.workunit.client.0.smithi119.stdout: 10 79767 28.29 MB/sec execute 14 sec latency 1171.043 ms 2023-12-07T10:36:02.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:02 smithi119 ceph-mon[97908]: pgmap v489: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 13 MiB/s wr, 694 op/s 2023-12-07T10:36:02.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:02 smithi159 ceph-mon[134726]: pgmap v489: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 13 MiB/s wr, 694 op/s 2023-12-07T10:36:02.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:02 smithi153 ceph-mon[133625]: pgmap v489: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 13 MiB/s wr, 694 op/s 2023-12-07T10:36:02.977 INFO:tasks.workunit.client.0.smithi119.stdout: 10 79767 26.41 MB/sec execute 15 sec latency 2171.199 ms 2023-12-07T10:36:03.977 INFO:tasks.workunit.client.0.smithi119.stdout: 10 79767 24.76 MB/sec execute 16 sec latency 3171.365 ms 2023-12-07T10:36:04.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:04 smithi119 ceph-mon[97908]: pgmap v490: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 13 MiB/s wr, 740 op/s 2023-12-07T10:36:04.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:04 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:36:04.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:04 smithi159 ceph-mon[134726]: pgmap v490: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 13 MiB/s wr, 740 op/s 2023-12-07T10:36:04.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:04 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:36:04.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:04 smithi153 ceph-mon[133625]: pgmap v490: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 13 MiB/s wr, 740 op/s 2023-12-07T10:36:04.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:04 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-07T10:36:04.977 INFO:tasks.workunit.client.0.smithi119.stdout: 10 79767 23.30 MB/sec execute 17 sec latency 4171.500 ms 2023-12-07T10:36:05.764 INFO:tasks.workunit.client.0.smithi119.stdout:[80137] write failed on handle 10022 (Resource temporarily unavailable) 2023-12-07T10:36:05.764 INFO:tasks.workunit.client.0.smithi119.stdout:Child failed with status 1 2023-12-07T10:36:05.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-07T10:36:06.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:06 smithi119 ceph-mon[97908]: pgmap v491: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 9.4 MiB/s wr, 588 op/s 2023-12-07T10:36:06.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:06 smithi159 ceph-mon[134726]: pgmap v491: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 9.4 MiB/s wr, 588 op/s 2023-12-07T10:36:06.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:06 smithi153 ceph-mon[133625]: pgmap v491: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 9.4 MiB/s wr, 588 op/s 2023-12-07T10:36:07.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:07 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:36:07.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:07 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:36:07.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:07 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:36:07.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:07 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:36:07.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:07 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:36:07.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:07 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:36:07.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:07 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:36:07.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:07 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:36:07.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:07 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:36:07.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:07 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-07T10:36:07.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:07 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-07T10:36:07.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:07 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' 2023-12-07T10:36:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.202+0000 7fe2135ff700 1 -- 172.21.15.119:0/1209300995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe20c0739b0 msgr2=0x7fe20c073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.202+0000 7fe2135ff700 1 --2- 172.21.15.119:0/1209300995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe20c0739b0 0x7fe20c073d90 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe208009990 tx=0x7fe208009ca0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.202+0000 7fe2135ff700 1 -- 172.21.15.119:0/1209300995 shutdown_connections 2023-12-07T10:36:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.202+0000 7fe2135ff700 1 --2- 172.21.15.119:0/1209300995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe20c1124f0 0x7fe20c1148e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.202+0000 7fe2135ff700 1 --2- 172.21.15.119:0/1209300995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe20c0739b0 0x7fe20c073d90 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.202+0000 7fe2135ff700 1 --2- 172.21.15.119:0/1209300995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe20c0742d0 0x7fe20c111fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.202+0000 7fe2135ff700 1 -- 172.21.15.119:0/1209300995 >> 172.21.15.119:0/1209300995 conn(0x7fe20c0fcfb0 msgr2=0x7fe20c0ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.202+0000 7fe2135ff700 1 -- 172.21.15.119:0/1209300995 shutdown_connections 2023-12-07T10:36:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.202+0000 7fe2135ff700 1 -- 172.21.15.119:0/1209300995 wait complete. 2023-12-07T10:36:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.203+0000 7fe2135ff700 1 Processor -- start 2023-12-07T10:36:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.203+0000 7fe2135ff700 1 -- start start 2023-12-07T10:36:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.204+0000 7fe2135ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe20c0739b0 0x7fe20c19e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.204+0000 7fe2135ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe20c0742d0 0x7fe20c19e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.204+0000 7fe2135ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe20c1124f0 0x7fe20c1a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.204+0000 7fe2135ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c1a2ee0 con 0x7fe20c0742d0 2023-12-07T10:36:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.204+0000 7fe2135ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c1a3050 con 0x7fe20c0739b0 2023-12-07T10:36:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.204+0000 7fe2135ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c1a3210 con 0x7fe20c1124f0 2023-12-07T10:36:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.204+0000 7fe210b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe20c0742d0 0x7fe20c19e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.204+0000 7fe211b9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe20c1124f0 0x7fe20c1a28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.204+0000 7fe211b9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe20c1124f0 0x7fe20c1a28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:53390/0 (socket says 172.21.15.119:53390) 2023-12-07T10:36:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.204+0000 7fe211b9c700 1 -- 172.21.15.119:0/4248291933 learned_addr learned my addr 172.21.15.119:0/4248291933 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:36:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.204+0000 7fe21139b700 1 --2- 172.21.15.119:0/4248291933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe20c0739b0 0x7fe20c19e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.204+0000 7fe211b9c700 1 -- 172.21.15.119:0/4248291933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe20c0739b0 msgr2=0x7fe20c19e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.204+0000 7fe211b9c700 1 --2- 172.21.15.119:0/4248291933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe20c0739b0 0x7fe20c19e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.204+0000 7fe211b9c700 1 -- 172.21.15.119:0/4248291933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe20c0742d0 msgr2=0x7fe20c19e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.204+0000 7fe211b9c700 1 --2- 172.21.15.119:0/4248291933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe20c0742d0 0x7fe20c19e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.204+0000 7fe211b9c700 1 -- 172.21.15.119:0/4248291933 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2080095f0 con 0x7fe20c1124f0 2023-12-07T10:36:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.205+0000 7fe211b9c700 1 --2- 172.21.15.119:0/4248291933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe20c1124f0 0x7fe20c1a28a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe2040094f0 tx=0x7fe20400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.205+0000 7fe2027fc700 1 -- 172.21.15.119:0/4248291933 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204004db0 con 0x7fe20c1124f0 2023-12-07T10:36:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.205+0000 7fe2135ff700 1 -- 172.21.15.119:0/4248291933 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20c1a34f0 con 0x7fe20c1124f0 2023-12-07T10:36:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.205+0000 7fe2027fc700 1 -- 172.21.15.119:0/4248291933 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe20400f040 con 0x7fe20c1124f0 2023-12-07T10:36:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.205+0000 7fe2027fc700 1 -- 172.21.15.119:0/4248291933 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204005980 con 0x7fe20c1124f0 2023-12-07T10:36:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.205+0000 7fe2135ff700 1 -- 172.21.15.119:0/4248291933 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe20c1aad30 con 0x7fe20c1124f0 2023-12-07T10:36:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.208+0000 7fe2027fc700 1 -- 172.21.15.119:0/4248291933 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fe204005020 con 0x7fe20c1124f0 2023-12-07T10:36:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.209+0000 7fe226f22b80 1 Processor -- start 2023-12-07T10:36:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.209+0000 7fe226f22b80 1 -- start start 2023-12-07T10:36:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.210+0000 7fe2027fc700 1 --2- 172.21.15.119:0/4248291933 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe1f80a1c80 0x7fe1f80a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.210+0000 7fe226f22b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625a3c7da60 0x5625a3c7de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.210+0000 7fe226f22b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625a3c7e380 0x5625a3c78540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.210+0000 7fe226f22b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625a3c78ad0 0x5625a3c78f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.210+0000 7fe226f22b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625a3bec760 con 0x5625a3c7e380 2023-12-07T10:36:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.210+0000 7fe226f22b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625a3bd3740 con 0x5625a3c7da60 2023-12-07T10:36:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.210+0000 7fe226f22b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625a3bd1060 con 0x5625a3c78ad0 2023-12-07T10:36:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.210+0000 7fe2027fc700 1 -- 172.21.15.119:0/4248291933 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fe2040cd640 con 0x7fe20c1124f0 2023-12-07T10:36:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.210+0000 7fe211b9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625a3c78ad0 0x5625a3c78f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.210+0000 7fe21139b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625a3c7da60 0x5625a3c7de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.210+0000 7fe210b9a700 1 --2- 172.21.15.119:0/4248291933 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe1f80a1c80 0x7fe1f80a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.210+0000 7fe211b9c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625a3c78ad0 0x5625a3c78f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:53406/0 (socket says 172.21.15.119:53406) 2023-12-07T10:36:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.210+0000 7fe211b9c700 1 -- 172.21.15.119:0/3207351882 learned_addr learned my addr 172.21.15.119:0/3207351882 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:36:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.210+0000 7fe210b9a700 1 --2- 172.21.15.119:0/3207351882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625a3c7e380 0x5625a3c78540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.210+0000 7fe211b9c700 1 -- 172.21.15.119:0/3207351882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625a3c7da60 msgr2=0x5625a3c7de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:36:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.210+0000 7fe211b9c700 1 --2- 172.21.15.119:0/3207351882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625a3c7da60 0x5625a3c7de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.210+0000 7fe211b9c700 1 -- 172.21.15.119:0/3207351882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625a3c7e380 msgr2=0x5625a3c78540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:36:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.211+0000 7fe211b9c700 1 --2- 172.21.15.119:0/3207351882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625a3c7e380 0x5625a3c78540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.211+0000 7fe211b9c700 1 -- 172.21.15.119:0/3207351882 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625a3a495d0 con 0x5625a3c78ad0 2023-12-07T10:36:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.211+0000 7fe211b9c700 1 --2- 172.21.15.119:0/3207351882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625a3c78ad0 0x5625a3c78f30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe204096a40 tx=0x7fe204096440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1744bf09e7a1b069 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.211+0000 7fe210b9a700 1 --2- 172.21.15.119:0/4248291933 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe1f80a1c80 0x7fe1f80a4140 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x5625a3c74890 tx=0x7fe1fc0099b0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.211+0000 7fe1f77fe700 1 -- 172.21.15.119:0/3207351882 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20409a6d0 con 0x5625a3c78ad0 2023-12-07T10:36:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.211+0000 7fe1f77fe700 1 -- 172.21.15.119:0/3207351882 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe204099fd0 con 0x5625a3c78ad0 2023-12-07T10:36:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.211+0000 7fe1f77fe700 1 -- 172.21.15.119:0/3207351882 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2040cd410 con 0x5625a3c78ad0 2023-12-07T10:36:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.212+0000 7fe226f22b80 1 -- 172.21.15.119:0/3207351882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625a3c78ad0 msgr2=0x5625a3c78f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.212+0000 7fe226f22b80 1 --2- 172.21.15.119:0/3207351882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625a3c78ad0 0x5625a3c78f30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe204096a40 tx=0x7fe204096440 comp rx=0 tx=0).stop 2023-12-07T10:36:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.212+0000 7fe226f22b80 1 -- 172.21.15.119:0/3207351882 shutdown_connections 2023-12-07T10:36:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.212+0000 7fe226f22b80 1 --2- 172.21.15.119:0/3207351882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625a3c78ad0 0x5625a3c78f30 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.212+0000 7fe226f22b80 1 --2- 172.21.15.119:0/3207351882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625a3c7da60 0x5625a3c7de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.212+0000 7fe226f22b80 1 --2- 172.21.15.119:0/3207351882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625a3c7e380 0x5625a3c78540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.212+0000 7fe226f22b80 1 -- 172.21.15.119:0/3207351882 >> 172.21.15.119:0/3207351882 conn(0x5625a3ba7300 msgr2=0x5625a3c72b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.212+0000 7fe226f22b80 1 -- 172.21.15.119:0/3207351882 shutdown_connections 2023-12-07T10:36:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.212+0000 7fe226f22b80 1 -- 172.21.15.119:0/3207351882 wait complete. 2023-12-07T10:36:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.212+0000 7fe226f22b80 1 -- start start 2023-12-07T10:36:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.213+0000 7fe226f22b80 1 Processor -- start 2023-12-07T10:36:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.213+0000 7fe226f22b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625a3c78ad0 0x5625a3c81780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.213+0000 7fe226f22b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625a3c7da60 0x5625a3c81cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.213+0000 7fe226f22b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625a3c7e380 0x5625a3c82200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.213+0000 7fe226f22b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625a3bcc950 con 0x5625a3c7e380 2023-12-07T10:36:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.213+0000 7fe226f22b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625a3b86500 con 0x5625a3c78ad0 2023-12-07T10:36:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.213+0000 7fe226f22b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625a3b861a0 con 0x5625a3c7da60 2023-12-07T10:36:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.213+0000 7fe226f22b80 10 client.0 _ll_register_callbacks cb 0x5625a3b6d8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:36:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.213+0000 7fe211b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625a3c7e380 0x5625a3c82200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.213+0000 7fe21139b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625a3c78ad0 0x5625a3c81780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.213+0000 7fe211b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625a3c7e380 0x5625a3c82200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55054/0 (socket says 172.21.15.119:55054) 2023-12-07T10:36:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.213+0000 7fe210b9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625a3c7da60 0x5625a3c81cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.213+0000 7fe211b9c700 1 -- 172.21.15.119:0/4084984071 learned_addr learned my addr 172.21.15.119:0/4084984071 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:36:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.214+0000 7fe211b9c700 1 -- 172.21.15.119:0/4084984071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625a3c7da60 msgr2=0x5625a3c81cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:36:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.214+0000 7fe211b9c700 1 --2- 172.21.15.119:0/4084984071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625a3c7da60 0x5625a3c81cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.214+0000 7fe211b9c700 1 -- 172.21.15.119:0/4084984071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625a3c78ad0 msgr2=0x5625a3c81780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:36:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.214+0000 7fe211b9c700 1 --2- 172.21.15.119:0/4084984071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625a3c78ad0 0x5625a3c81780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.214+0000 7fe211b9c700 1 -- 172.21.15.119:0/4084984071 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe20c1aad30 con 0x5625a3c7e380 2023-12-07T10:36:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.214+0000 7fe211b9c700 1 --2- 172.21.15.119:0/4084984071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625a3c7e380 0x5625a3c82200 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe2040a1e30 tx=0x7fe2040cd420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dbb79f8079d9706 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.214+0000 7fe1f4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:36:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.214+0000 7fe1f4ff9700 1 -- 172.21.15.119:0/4084984071 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2040a2360 con 0x5625a3c7e380 2023-12-07T10:36:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.214+0000 7fe1f4ff9700 1 -- 172.21.15.119:0/4084984071 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fe20409e120 con 0x5625a3c7e380 2023-12-07T10:36:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.214+0000 7fe226f22b80 1 -- 172.21.15.119:0/4084984071 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5625a3c6b6f0 con 0x5625a3c7e380 2023-12-07T10:36:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.214+0000 7fe1f4ff9700 1 -- 172.21.15.119:0/4084984071 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe204094df0 con 0x5625a3c7e380 2023-12-07T10:36:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.214+0000 7fe226f22b80 10 client.17934 fetch_fsmap learned FSMap version 73 2023-12-07T10:36:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.214+0000 7fe226f22b80 1 -- 172.21.15.119:0/4084984071 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) v3 -- 0x5625a39f5d80 con 0x5625a3c7e380 2023-12-07T10:36:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.215+0000 7fe1f4ff9700 1 -- 172.21.15.119:0/4084984071 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe204063020 con 0x5625a3c7e380 2023-12-07T10:36:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.215+0000 7fe1f4ff9700 1 -- 172.21.15.119:0/4084984071 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 73) v1 ==== 7946+0+0 (secure 0 0 0) 0x7fe2040994c0 con 0x5625a3c7e380 2023-12-07T10:36:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.215+0000 7fe226f22b80 10 client.17934 fetch_fsmap finished waiting for FSMap version 73 2023-12-07T10:36:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.215+0000 7fe226f22b80 10 client.17934 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:36:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.215+0000 7fe226f22b80 20 client.17934 populate_metadata read hostname 'smithi119' 2023-12-07T10:36:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.215+0000 7fe226f22b80 1 --2- 172.21.15.119:0/4084984071 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5625a3c91c90 0x5625a3c940b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.215+0000 7fe226f22b80 4 client.17934 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-07T10:36:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.215+0000 7fe226f22b80 1 -- 172.21.15.119:0/4084984071 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625a3c47740 con 0x5625a3c91c90 2023-12-07T10:36:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.215+0000 7fe21139b700 1 --2- 172.21.15.119:0/4084984071 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5625a3c91c90 0x5625a3c940b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.216+0000 7fe21139b700 1 --2- 172.21.15.119:0/4084984071 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5625a3c91c90 0x5625a3c940b0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84bfa5937122aad9 server_cookie=e20961712396b7a8 in_seq=0 out_seq=0 2023-12-07T10:36:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.216+0000 7fe1f4ff9700 10 client.17934 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:36:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.217+0000 7fe1f4ff9700 1 -- 172.21.15.119:0/4084984071 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5625a3c47740 con 0x5625a3c91c90 2023-12-07T10:36:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.217+0000 7fe1f4ff9700 10 client.17934 handle_command_reply: tid=0 2023-12-07T10:36:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.217+0000 7fe226f22b80 1 client.17934 shutdown 2023-12-07T10:36:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.217+0000 7fe226f22b80 1 -- 172.21.15.119:0/4084984071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625a3c7e380 msgr2=0x5625a3c82200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.217+0000 7fe226f22b80 1 --2- 172.21.15.119:0/4084984071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625a3c7e380 0x5625a3c82200 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe2040a1e30 tx=0x7fe2040cd420 comp rx=0 tx=0).stop 2023-12-07T10:36:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.217+0000 7fe226f22b80 1 -- 172.21.15.119:0/4084984071 shutdown_connections 2023-12-07T10:36:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.217+0000 7fe226f22b80 1 --2- 172.21.15.119:0/4084984071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625a3c7e380 0x5625a3c82200 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.217+0000 7fe226f22b80 1 --2- 172.21.15.119:0/4084984071 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5625a3c91c90 0x5625a3c940b0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.217+0000 7fe226f22b80 1 --2- 172.21.15.119:0/4084984071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625a3c78ad0 0x5625a3c81780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.217+0000 7fe226f22b80 1 --2- 172.21.15.119:0/4084984071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625a3c7da60 0x5625a3c81cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.217+0000 7fe1f4ff9700 0 client.17934 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:36:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.217+0000 7fe226f22b80 1 -- 172.21.15.119:0/4084984071 >> 172.21.15.119:0/4084984071 conn(0x5625a3ba7300 msgr2=0x5625a3c6d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.217+0000 7fe226f22b80 1 -- 172.21.15.119:0/4084984071 shutdown_connections 2023-12-07T10:36:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.217+0000 7fe226f22b80 1 -- 172.21.15.119:0/4084984071 wait complete. 2023-12-07T10:36:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.217+0000 7fe226f22b80 20 client.17934 trim_cache size 0 max 16384 2023-12-07T10:36:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.232+0000 7fe226f22b80 1 Processor -- start 2023-12-07T10:36:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.232+0000 7fe226f22b80 1 -- start start 2023-12-07T10:36:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.233+0000 7fe226f22b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625a3c78ad0 0x5625a3c78540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.233+0000 7fe226f22b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625a3c7da60 0x5625a3c80f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.233+0000 7fe226f22b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625a3c7e380 0x5625a3c7c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.233+0000 7fe226f22b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625a3a4b8c0 con 0x5625a3c7da60 2023-12-07T10:36:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.233+0000 7fe226f22b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625a3c73a90 con 0x5625a3c7e380 2023-12-07T10:36:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.233+0000 7fe226f22b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625a3c82d30 con 0x5625a3c78ad0 2023-12-07T10:36:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.233+0000 7fe210b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625a3c7da60 0x5625a3c80f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.233+0000 7fe210b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625a3c7da60 0x5625a3c80f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55064/0 (socket says 172.21.15.119:55064) 2023-12-07T10:36:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.233+0000 7fe210b9a700 1 -- 172.21.15.119:0/3261913665 learned_addr learned my addr 172.21.15.119:0/3261913665 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:36:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.233+0000 7fe21139b700 1 --2- 172.21.15.119:0/3261913665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625a3c78ad0 0x5625a3c78540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.233+0000 7fe211b9c700 1 --2- 172.21.15.119:0/3261913665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625a3c7e380 0x5625a3c7c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.233+0000 7fe21139b700 1 -- 172.21.15.119:0/3261913665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625a3c7e380 msgr2=0x5625a3c7c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:36:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.233+0000 7fe21139b700 1 --2- 172.21.15.119:0/3261913665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625a3c7e380 0x5625a3c7c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.233+0000 7fe21139b700 1 -- 172.21.15.119:0/3261913665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625a3c7da60 msgr2=0x5625a3c80f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:36:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.233+0000 7fe21139b700 1 --2- 172.21.15.119:0/3261913665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625a3c7da60 0x5625a3c80f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.233+0000 7fe21139b700 1 -- 172.21.15.119:0/3261913665 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625a39dec00 con 0x5625a3c78ad0 2023-12-07T10:36:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.233+0000 7fe210b9a700 1 --2- 172.21.15.119:0/3261913665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625a3c7da60 0x5625a3c80f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:36:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.234+0000 7fe21139b700 1 --2- 172.21.15.119:0/3261913665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625a3c78ad0 0x5625a3c78540 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x5625a3c949a0 tx=0x7fe208005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a729e74a437a720c server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.234+0000 7fe1f7fff700 1 -- 172.21.15.119:0/3261913665 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20802b070 con 0x5625a3c78ad0 2023-12-07T10:36:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.234+0000 7fe1f7fff700 1 -- 172.21.15.119:0/3261913665 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe20801e040 con 0x5625a3c78ad0 2023-12-07T10:36:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.234+0000 7fe1f7fff700 1 -- 172.21.15.119:0/3261913665 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20801d040 con 0x5625a3c78ad0 2023-12-07T10:36:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.235+0000 7fe226f22b80 1 -- 172.21.15.119:0/3261913665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625a3c78ad0 msgr2=0x5625a3c78540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.235+0000 7fe226f22b80 1 --2- 172.21.15.119:0/3261913665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625a3c78ad0 0x5625a3c78540 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x5625a3c949a0 tx=0x7fe208005fd0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.235+0000 7fe226f22b80 1 -- 172.21.15.119:0/3261913665 shutdown_connections 2023-12-07T10:36:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.235+0000 7fe226f22b80 1 --2- 172.21.15.119:0/3261913665 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625a3c7e380 0x5625a3c7c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.235+0000 7fe226f22b80 1 --2- 172.21.15.119:0/3261913665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625a3c78ad0 0x5625a3c78540 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.235+0000 7fe226f22b80 1 --2- 172.21.15.119:0/3261913665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625a3c7da60 0x5625a3c80f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.235+0000 7fe226f22b80 1 -- 172.21.15.119:0/3261913665 >> 172.21.15.119:0/3261913665 conn(0x5625a3ba7300 msgr2=0x5625a3c7b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.235+0000 7fe226f22b80 1 -- 172.21.15.119:0/3261913665 shutdown_connections 2023-12-07T10:36:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.235+0000 7fe226f22b80 1 -- 172.21.15.119:0/3261913665 wait complete. 2023-12-07T10:36:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.235+0000 7fe226f22b80 1 -- start start 2023-12-07T10:36:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.235+0000 7fe226f22b80 1 Processor -- start 2023-12-07T10:36:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.236+0000 7fe226f22b80 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625a3c78ad0 0x5625a3c86f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.236+0000 7fe226f22b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625a3c7da60 0x5625a3c87480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.236+0000 7fe226f22b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625a3c7e380 0x5625a3ce7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.236+0000 7fe226f22b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625a3c87b90 con 0x5625a3c7e380 2023-12-07T10:36:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.236+0000 7fe226f22b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625a3ce76b0 con 0x5625a3c7da60 2023-12-07T10:36:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.236+0000 7fe226f22b80 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625a3ce7820 con 0x5625a3c78ad0 2023-12-07T10:36:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.236+0000 7fe226f22b80 10 client.0 _ll_register_callbacks cb 0x5625a3b6d8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-07T10:36:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.236+0000 7fe211b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625a3c7e380 0x5625a3ce7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.236+0000 7fe21139b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625a3c78ad0 0x5625a3c86f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.236+0000 7fe210b9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625a3c7da60 0x5625a3c87480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.236+0000 7fe21139b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625a3c78ad0 0x5625a3c86f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:53418/0 (socket says 172.21.15.119:53418) 2023-12-07T10:36:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.236+0000 7fe21139b700 1 -- 172.21.15.119:0/651752785 learned_addr learned my addr 172.21.15.119:0/651752785 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:36:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.236+0000 7fe211b9c700 1 -- 172.21.15.119:0/651752785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625a3c78ad0 msgr2=0x5625a3c86f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:36:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.236+0000 7fe211b9c700 1 --2- 172.21.15.119:0/651752785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625a3c78ad0 0x5625a3c86f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.236+0000 7fe211b9c700 1 -- 172.21.15.119:0/651752785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625a3c7da60 msgr2=0x5625a3c87480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-07T10:36:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.236+0000 7fe211b9c700 1 --2- 172.21.15.119:0/651752785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625a3c7da60 0x5625a3c87480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.236+0000 7fe211b9c700 1 -- 172.21.15.119:0/651752785 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1fc019040 con 0x5625a3c7e380 2023-12-07T10:36:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.236+0000 7fe21139b700 1 --2- 172.21.15.119:0/651752785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625a3c78ad0 0x5625a3c86f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:36:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.236+0000 7fe210b9a700 1 --2- 172.21.15.119:0/651752785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625a3c7da60 0x5625a3c87480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:36:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.237+0000 7fe211b9c700 1 --2- 172.21.15.119:0/651752785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625a3c7e380 0x5625a3ce7170 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe204096a40 tx=0x7fe20409a810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13585e48cfcaf199 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.237+0000 7fe1f4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-12-07T10:36:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.237+0000 7fe1f4ff9700 1 -- 172.21.15.119:0/651752785 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20409a430 con 0x5625a3c7e380 2023-12-07T10:36:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.237+0000 7fe1f4ff9700 1 -- 172.21.15.119:0/651752785 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fe204064080 con 0x5625a3c7e380 2023-12-07T10:36:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.237+0000 7fe226f22b80 1 -- 172.21.15.119:0/651752785 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe204099fd0 con 0x5625a3c7e380 2023-12-07T10:36:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.237+0000 7fe1f4ff9700 1 -- 172.21.15.119:0/651752785 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe204009e20 con 0x5625a3c7e380 2023-12-07T10:36:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.237+0000 7fe226f22b80 10 client.17940 fetch_fsmap learned FSMap version 73 2023-12-07T10:36:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.237+0000 7fe226f22b80 1 -- 172.21.15.119:0/651752785 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) v3 -- 0x7fe20c115290 con 0x5625a3c7e380 2023-12-07T10:36:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.237+0000 7fe1f4ff9700 1 -- 172.21.15.119:0/651752785 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe204099fd0 con 0x5625a3c7e380 2023-12-07T10:36:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.237+0000 7fe1f4ff9700 1 -- 172.21.15.119:0/651752785 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 73) v1 ==== 7946+0+0 (secure 0 0 0) 0x7fe20405fd10 con 0x5625a3c7e380 2023-12-07T10:36:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.237+0000 7fe226f22b80 10 client.17940 fetch_fsmap finished waiting for FSMap version 73 2023-12-07T10:36:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.237+0000 7fe226f22b80 10 client.17940 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-07T10:36:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.237+0000 7fe226f22b80 20 client.17940 populate_metadata read hostname 'smithi119' 2023-12-07T10:36:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.238+0000 7fe226f22b80 1 --2- 172.21.15.119:0/651752785 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5625a3ce8bf0 0x5625a3ceb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.238+0000 7fe226f22b80 4 client.17940 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-07T10:36:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.238+0000 7fe226f22b80 1 -- 172.21.15.119:0/651752785 --> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5625a3b92f70 con 0x5625a3ce8bf0 2023-12-07T10:36:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.238+0000 7fe21139b700 1 --2- 172.21.15.119:0/651752785 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5625a3ce8bf0 0x5625a3ceb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.238+0000 7fe1f4ff9700 10 client.17940 ms_handle_connect on v2:172.21.15.153:6838/566825184 2023-12-07T10:36:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.238+0000 7fe21139b700 1 --2- 172.21.15.119:0/651752785 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5625a3ce8bf0 0x5625a3ceb010 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0233f31d38a50da server_cookie=edf01f2aa75f87fc in_seq=0 out_seq=0 2023-12-07T10:36:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.389+0000 7fe1f4ff9700 1 -- 172.21.15.119:0/651752785 <== mds.0 v2:172.21.15.153:6838/566825184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5625a3b92f70 con 0x5625a3ce8bf0 2023-12-07T10:36:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.389+0000 7fe1f4ff9700 10 client.17940 handle_command_reply: tid=0 2023-12-07T10:36:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.389+0000 7fe226f22b80 1 client.17940 shutdown 2023-12-07T10:36:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.389+0000 7fe226f22b80 1 -- 172.21.15.119:0/651752785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625a3c7e380 msgr2=0x5625a3ce7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.389+0000 7fe226f22b80 1 --2- 172.21.15.119:0/651752785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625a3c7e380 0x5625a3ce7170 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe204096a40 tx=0x7fe20409a810 comp rx=0 tx=0).stop 2023-12-07T10:36:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.389+0000 7fe226f22b80 1 -- 172.21.15.119:0/651752785 shutdown_connections 2023-12-07T10:36:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.389+0000 7fe226f22b80 1 --2- 172.21.15.119:0/651752785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5625a3c7e380 0x5625a3ce7170 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.389+0000 7fe226f22b80 1 --2- 172.21.15.119:0/651752785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625a3c78ad0 0x5625a3c86f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.389+0000 7fe226f22b80 1 --2- 172.21.15.119:0/651752785 >> [v2:172.21.15.153:6838/566825184,v1:172.21.15.153:6839/566825184] conn(0x5625a3ce8bf0 0x5625a3ceb010 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.389+0000 7fe226f22b80 1 --2- 172.21.15.119:0/651752785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5625a3c7da60 0x5625a3c87480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.389+0000 7fe1f4ff9700 0 client.17940 ms_handle_reset on v2:172.21.15.153:6838/566825184 2023-12-07T10:36:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.389+0000 7fe226f22b80 1 -- 172.21.15.119:0/651752785 >> 172.21.15.119:0/651752785 conn(0x5625a3ba7300 msgr2=0x5625a3c81f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.389+0000 7fe226f22b80 1 -- 172.21.15.119:0/651752785 shutdown_connections 2023-12-07T10:36:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.389+0000 7fe226f22b80 1 -- 172.21.15.119:0/651752785 wait complete. 2023-12-07T10:36:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.390+0000 7fe226f22b80 20 client.17940 trim_cache size 0 max 16384 2023-12-07T10:36:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:{ 2023-12-07T10:36:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "message": "", 2023-12-07T10:36:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout: "return_code": 0 2023-12-07T10:36:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stdout:} 2023-12-07T10:36:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.390+0000 7fe2135ff700 1 -- 172.21.15.119:0/4248291933 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe1f80a1c80 msgr2=0x7fe1f80a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.390+0000 7fe2135ff700 1 --2- 172.21.15.119:0/4248291933 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe1f80a1c80 0x7fe1f80a4140 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x5625a3c74890 tx=0x7fe1fc0099b0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.390+0000 7fe2135ff700 1 -- 172.21.15.119:0/4248291933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe20c1124f0 msgr2=0x7fe20c1a28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.390+0000 7fe2135ff700 1 --2- 172.21.15.119:0/4248291933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe20c1124f0 0x7fe20c1a28a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe2040094f0 tx=0x7fe20400d3b0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.390+0000 7fe2135ff700 1 -- 172.21.15.119:0/4248291933 shutdown_connections 2023-12-07T10:36:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.390+0000 7fe2135ff700 1 --2- 172.21.15.119:0/4248291933 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe1f80a1c80 0x7fe1f80a4140 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.390+0000 7fe2135ff700 1 --2- 172.21.15.119:0/4248291933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe20c1124f0 0x7fe20c1a28a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.390+0000 7fe2135ff700 1 --2- 172.21.15.119:0/4248291933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe20c0739b0 0x7fe20c19e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.390+0000 7fe2135ff700 1 --2- 172.21.15.119:0/4248291933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe20c0742d0 0x7fe20c19e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.390+0000 7fe2135ff700 1 -- 172.21.15.119:0/4248291933 >> 172.21.15.119:0/4248291933 conn(0x7fe20c0fcfb0 msgr2=0x7fe20c0fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.390+0000 7fe2135ff700 1 -- 172.21.15.119:0/4248291933 shutdown_connections 2023-12-07T10:36:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi119.front.sepia.ceph.com.smithi119.stderr:2023-12-07T10:36:08.391+0000 7fe2135ff700 1 -- 172.21.15.119:0/4248291933 wait complete. 2023-12-07T10:36:08.727 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:08 smithi119 ceph-mon[97908]: pgmap v492: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.5 MiB/s wr, 615 op/s 2023-12-07T10:36:08.761 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:08 smithi159 ceph-mon[134726]: pgmap v492: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.5 MiB/s wr, 615 op/s 2023-12-07T10:36:08.778 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:08 smithi153 ceph-mon[133625]: pgmap v492: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 9.5 MiB/s wr, 615 op/s 2023-12-07T10:36:09.727 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:09 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:36:09.762 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:09 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:36:09.780 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:09 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:36:10.727 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:10 smithi119 ceph-mon[97908]: pgmap v493: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.2 MiB/s wr, 325 op/s 2023-12-07T10:36:10.727 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:10 smithi119 ceph-mon[97908]: osdmap e113: 12 total, 12 up, 12 in 2023-12-07T10:36:10.762 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:10 smithi159 ceph-mon[134726]: pgmap v493: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.2 MiB/s wr, 325 op/s 2023-12-07T10:36:10.762 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:10 smithi159 ceph-mon[134726]: osdmap e113: 12 total, 12 up, 12 in 2023-12-07T10:36:10.777 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:10 smithi153 ceph-mon[133625]: pgmap v493: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.2 MiB/s wr, 325 op/s 2023-12-07T10:36:10.778 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:10 smithi153 ceph-mon[133625]: osdmap e113: 12 total, 12 up, 12 in 2023-12-07T10:36:12.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:12 smithi119 ceph-mon[97908]: pgmap v495: 129 pgs: 6 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 109 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 290 op/s 2023-12-07T10:36:12.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:12 smithi159 ceph-mon[134726]: pgmap v495: 129 pgs: 6 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 109 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 290 op/s 2023-12-07T10:36:12.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:12 smithi153 ceph-mon[133625]: pgmap v495: 129 pgs: 6 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 109 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 290 op/s 2023-12-07T10:36:14.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:14 smithi119 ceph-mon[97908]: pgmap v496: 129 pgs: 8 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 101 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 5.1 MiB/s wr, 209 op/s 2023-12-07T10:36:14.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:14 smithi159 ceph-mon[134726]: pgmap v496: 129 pgs: 8 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 101 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 5.1 MiB/s wr, 209 op/s 2023-12-07T10:36:14.527 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:14 smithi153 ceph-mon[133625]: pgmap v496: 129 pgs: 8 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 101 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 5.1 MiB/s wr, 209 op/s 2023-12-07T10:36:15.687 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2023-12-07T10:36:15.688 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-12-07T10:36:16.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:16 smithi119 ceph-mon[97908]: pgmap v497: 129 pgs: 8 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 101 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 5.1 MiB/s wr, 209 op/s 2023-12-07T10:36:16.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:16 smithi159 ceph-mon[134726]: pgmap v497: 129 pgs: 8 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 101 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 5.1 MiB/s wr, 209 op/s 2023-12-07T10:36:16.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:16 smithi153 ceph-mon[133625]: pgmap v497: 129 pgs: 8 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 101 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 5.1 MiB/s wr, 209 op/s 2023-12-07T10:36:16.806 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi119 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2023-12-07T10:36:16.883 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=6c49b911ab864ff8ad53779d28ab3fed Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi119 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2023-12-07T10:36:16.886 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-12-07T10:36:16.898 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi119.front.sepia.ceph.com 2023-12-07T10:36:16.940 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-12-07T10:36:16.952 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-12-07T10:36:16.952 INFO:teuthology.task.exec:Executing custom commands... 2023-12-07T10:36:16.953 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi119.front.sepia.ceph.com 2023-12-07T10:36:16.953 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-12-07T10:36:17.024 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-12-07T10:36:17.099 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-12-07T10:36:17.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.362+0000 7f223abff700 1 -- 172.21.15.119:0/900986296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2234101ec0 msgr2=0x7f223410e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:17.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.362+0000 7f223abff700 1 --2- 172.21.15.119:0/900986296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2234101ec0 0x7f223410e380 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f222c00b0d0 tx=0x7f222c00b3e0 comp rx=0 tx=0).stop 2023-12-07T10:36:17.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.363+0000 7f223abff700 1 -- 172.21.15.119:0/900986296 shutdown_connections 2023-12-07T10:36:17.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.363+0000 7f223abff700 1 --2- 172.21.15.119:0/900986296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2234101ec0 0x7f223410e380 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:17.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.363+0000 7f223abff700 1 --2- 172.21.15.119:0/900986296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22341075c0 0x7f22341079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:17.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.363+0000 7f223abff700 1 --2- 172.21.15.119:0/900986296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2234101500 0x7f2234101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:17.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.363+0000 7f223abff700 1 -- 172.21.15.119:0/900986296 >> 172.21.15.119:0/900986296 conn(0x7f22340fcfb0 msgr2=0x7f22340ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:17.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.363+0000 7f223abff700 1 -- 172.21.15.119:0/900986296 shutdown_connections 2023-12-07T10:36:17.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.363+0000 7f223abff700 1 -- 172.21.15.119:0/900986296 wait complete. 2023-12-07T10:36:17.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.364+0000 7f223abff700 1 Processor -- start 2023-12-07T10:36:17.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.364+0000 7f223abff700 1 -- start start 2023-12-07T10:36:17.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.364+0000 7f223abff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2234101500 0x7f2234073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:17.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.364+0000 7f223abff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2234101ec0 0x7f2234073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:17.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.364+0000 7f223abff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22341075c0 0x7f2234074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:17.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.364+0000 7f223abff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22341a5dd0 con 0x7f22341075c0 2023-12-07T10:36:17.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.364+0000 7f223abff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22341a5f40 con 0x7f2234101500 2023-12-07T10:36:17.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.364+0000 7f223abff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22341a6100 con 0x7f2234101ec0 2023-12-07T10:36:17.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.364+0000 7f223899b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2234101500 0x7f2234073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:17.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.364+0000 7f223919c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22341075c0 0x7f2234074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:17.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.364+0000 7f223899b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2234101500 0x7f2234073910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:44630/0 (socket says 172.21.15.119:44630) 2023-12-07T10:36:17.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.364+0000 7f223899b700 1 -- 172.21.15.119:0/2610424460 learned_addr learned my addr 172.21.15.119:0/2610424460 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:36:17.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.364+0000 7f2233fff700 1 --2- 172.21.15.119:0/2610424460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2234101ec0 0x7f2234073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:17.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.365+0000 7f223899b700 1 -- 172.21.15.119:0/2610424460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2234101ec0 msgr2=0x7f2234073e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:17.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.365+0000 7f223899b700 1 --2- 172.21.15.119:0/2610424460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2234101ec0 0x7f2234073e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:17.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.365+0000 7f223899b700 1 -- 172.21.15.119:0/2610424460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22341075c0 msgr2=0x7f2234074390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:17.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.365+0000 7f223899b700 1 --2- 172.21.15.119:0/2610424460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22341075c0 0x7f2234074390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:17.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.365+0000 7f223899b700 1 -- 172.21.15.119:0/2610424460 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222c009d00 con 0x7f2234101500 2023-12-07T10:36:17.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.365+0000 7f223919c700 1 --2- 172.21.15.119:0/2610424460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22341075c0 0x7f2234074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:36:17.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.365+0000 7f223899b700 1 --2- 172.21.15.119:0/2610424460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2234101500 0x7f2234073910 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f222800e9a0 tx=0x7f222800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:17.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.365+0000 7f2231ffb700 1 -- 172.21.15.119:0/2610424460 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2228009790 con 0x7f2234101500 2023-12-07T10:36:17.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.365+0000 7f2231ffb700 1 -- 172.21.15.119:0/2610424460 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2228010430 con 0x7f2234101500 2023-12-07T10:36:17.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.365+0000 7f223abff700 1 -- 172.21.15.119:0/2610424460 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22341a63e0 con 0x7f2234101500 2023-12-07T10:36:17.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.365+0000 7f2231ffb700 1 -- 172.21.15.119:0/2610424460 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2228018680 con 0x7f2234101500 2023-12-07T10:36:17.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.365+0000 7f223abff700 1 -- 172.21.15.119:0/2610424460 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22341a6940 con 0x7f2234101500 2023-12-07T10:36:17.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.366+0000 7f223abff700 1 -- 172.21.15.119:0/2610424460 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2234109a60 con 0x7f2234101500 2023-12-07T10:36:17.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.368+0000 7f2231ffb700 1 -- 172.21.15.119:0/2610424460 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f22280105a0 con 0x7f2234101500 2023-12-07T10:36:17.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.370+0000 7f2231ffb700 1 --2- 172.21.15.119:0/2610424460 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f22240a1d00 0x7f22240a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:17.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.370+0000 7f2231ffb700 1 -- 172.21.15.119:0/2610424460 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f2228014070 con 0x7f2234101500 2023-12-07T10:36:17.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.370+0000 7f2233fff700 1 --2- 172.21.15.119:0/2610424460 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f22240a1d00 0x7f22240a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:17.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.370+0000 7f2233fff700 1 --2- 172.21.15.119:0/2610424460 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f22240a1d00 0x7f22240a41c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2234075040 tx=0x7f2220009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:17.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.372+0000 7f2231ffb700 1 -- 172.21.15.119:0/2610424460 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2228096c10 con 0x7f2234101500 2023-12-07T10:36:17.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.604+0000 7f223abff700 1 -- 172.21.15.119:0/2610424460 --> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f2234074df0 con 0x7f22240a1d00 2023-12-07T10:36:17.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.614+0000 7f2231ffb700 1 -- 172.21.15.119:0/2610424460 <== mgr.15612 v2:172.21.15.119:6800/3763427845 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f2234074df0 con 0x7f22240a1d00 2023-12-07T10:36:17.614 INFO:teuthology.orchestra.run.smithi119.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2023-12-07T00:00:00", "created": "2023-12-07T10:20:25", "first": "2023-12-07T10:21:00", "last": "2023-12-07T10:36:00", "last_pruned": "2023-12-07T10:36:00", "created_count": 16, "pruned_count": 10, "active": true} 2023-12-07T10:36:17.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.617+0000 7f223abff700 1 -- 172.21.15.119:0/2610424460 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f22240a1d00 msgr2=0x7f22240a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:17.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.617+0000 7f223abff700 1 --2- 172.21.15.119:0/2610424460 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f22240a1d00 0x7f22240a41c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2234075040 tx=0x7f2220009260 comp rx=0 tx=0).stop 2023-12-07T10:36:17.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.617+0000 7f223abff700 1 -- 172.21.15.119:0/2610424460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2234101500 msgr2=0x7f2234073910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:17.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.617+0000 7f223abff700 1 --2- 172.21.15.119:0/2610424460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2234101500 0x7f2234073910 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f222800e9a0 tx=0x7f222800ecb0 comp rx=0 tx=0).stop 2023-12-07T10:36:17.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.617+0000 7f223abff700 1 -- 172.21.15.119:0/2610424460 shutdown_connections 2023-12-07T10:36:17.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.617+0000 7f223abff700 1 --2- 172.21.15.119:0/2610424460 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f22240a1d00 0x7f22240a41c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:17.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.617+0000 7f223abff700 1 --2- 172.21.15.119:0/2610424460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22341075c0 0x7f2234074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:17.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.617+0000 7f223abff700 1 --2- 172.21.15.119:0/2610424460 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2234101500 0x7f2234073910 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:17.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.617+0000 7f223abff700 1 --2- 172.21.15.119:0/2610424460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2234101ec0 0x7f2234073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:17.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.617+0000 7f223abff700 1 -- 172.21.15.119:0/2610424460 >> 172.21.15.119:0/2610424460 conn(0x7f22340fcfb0 msgr2=0x7f22340fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:17.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.617+0000 7f223abff700 1 -- 172.21.15.119:0/2610424460 shutdown_connections 2023-12-07T10:36:17.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.617+0000 7f223abff700 1 -- 172.21.15.119:0/2610424460 wait complete. 2023-12-07T10:36:17.635 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-12-07T10:36:17.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.830+0000 7f79ab5ff700 1 -- 172.21.15.119:0/3286136643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79a4101ee0 msgr2=0x7f79a410e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:17.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.830+0000 7f79ab5ff700 1 --2- 172.21.15.119:0/3286136643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79a4101ee0 0x7f79a410e3a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f799c00b0d0 tx=0x7f799c00b3e0 comp rx=0 tx=0).stop 2023-12-07T10:36:17.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.830+0000 7f79ab5ff700 1 -- 172.21.15.119:0/3286136643 shutdown_connections 2023-12-07T10:36:17.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.830+0000 7f79ab5ff700 1 --2- 172.21.15.119:0/3286136643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79a4101ee0 0x7f79a410e3a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:17.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.830+0000 7f79ab5ff700 1 --2- 172.21.15.119:0/3286136643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79a41075e0 0x7f79a41079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:17.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.830+0000 7f79ab5ff700 1 --2- 172.21.15.119:0/3286136643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79a4101520 0x7f79a41019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:17.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.830+0000 7f79ab5ff700 1 -- 172.21.15.119:0/3286136643 >> 172.21.15.119:0/3286136643 conn(0x7f79a40fd010 msgr2=0x7f79a40ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:17.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.830+0000 7f79ab5ff700 1 -- 172.21.15.119:0/3286136643 shutdown_connections 2023-12-07T10:36:17.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.830+0000 7f79ab5ff700 1 -- 172.21.15.119:0/3286136643 wait complete. 2023-12-07T10:36:17.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.831+0000 7f79ab5ff700 1 Processor -- start 2023-12-07T10:36:17.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.831+0000 7f79ab5ff700 1 -- start start 2023-12-07T10:36:17.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.832+0000 7f79ab5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79a4101520 0x7f79a419f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:17.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.832+0000 7f79ab5ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79a4101ee0 0x7f79a419feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:17.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.832+0000 7f79ab5ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79a41075e0 0x7f79a4199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:17.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.832+0000 7f79ab5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a419a100 con 0x7f79a4101520 2023-12-07T10:36:17.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.832+0000 7f79ab5ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a419a270 con 0x7f79a4101ee0 2023-12-07T10:36:17.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.832+0000 7f79ab5ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a419a430 con 0x7f79a41075e0 2023-12-07T10:36:17.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.832+0000 7f79a939b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79a4101520 0x7f79a419f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:17.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.832+0000 7f79a939b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79a4101520 0x7f79a419f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41760/0 (socket says 172.21.15.119:41760) 2023-12-07T10:36:17.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.832+0000 7f79a939b700 1 -- 172.21.15.119:0/1238754579 learned_addr learned my addr 172.21.15.119:0/1238754579 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:36:17.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.832+0000 7f79a8b9a700 1 --2- 172.21.15.119:0/1238754579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79a4101ee0 0x7f79a419feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:17.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.832+0000 7f79a9b9c700 1 --2- 172.21.15.119:0/1238754579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79a41075e0 0x7f79a4199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:17.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.832+0000 7f79a939b700 1 -- 172.21.15.119:0/1238754579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79a41075e0 msgr2=0x7f79a4199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:17.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.832+0000 7f79a939b700 1 --2- 172.21.15.119:0/1238754579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79a41075e0 0x7f79a4199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:17.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.832+0000 7f79a939b700 1 -- 172.21.15.119:0/1238754579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79a4101ee0 msgr2=0x7f79a419feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:17.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.832+0000 7f79a939b700 1 --2- 172.21.15.119:0/1238754579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79a4101ee0 0x7f79a419feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:17.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.832+0000 7f79a939b700 1 -- 172.21.15.119:0/1238754579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c009d00 con 0x7f79a4101520 2023-12-07T10:36:17.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.833+0000 7f79a939b700 1 --2- 172.21.15.119:0/1238754579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79a4101520 0x7f79a419f970 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f79a000e9a0 tx=0x7f79a000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:17.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.833+0000 7f799a7fc700 1 -- 172.21.15.119:0/1238754579 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0009790 con 0x7f79a4101520 2023-12-07T10:36:17.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.833+0000 7f799a7fc700 1 -- 172.21.15.119:0/1238754579 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79a0010430 con 0x7f79a4101520 2023-12-07T10:36:17.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.833+0000 7f79ab5ff700 1 -- 172.21.15.119:0/1238754579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a419a710 con 0x7f79a4101520 2023-12-07T10:36:17.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.833+0000 7f799a7fc700 1 -- 172.21.15.119:0/1238754579 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0018680 con 0x7f79a4101520 2023-12-07T10:36:17.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.833+0000 7f79ab5ff700 1 -- 172.21.15.119:0/1238754579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a41a6960 con 0x7f79a4101520 2023-12-07T10:36:17.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.834+0000 7f79ab5ff700 1 -- 172.21.15.119:0/1238754579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a4102c40 con 0x7f79a4101520 2023-12-07T10:36:17.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.836+0000 7f799a7fc700 1 -- 172.21.15.119:0/1238754579 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f79a00105a0 con 0x7f79a4101520 2023-12-07T10:36:17.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.837+0000 7f799a7fc700 1 --2- 172.21.15.119:0/1238754579 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f79900a1d00 0x7f79900a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:17.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.838+0000 7f799a7fc700 1 -- 172.21.15.119:0/1238754579 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f79a0014070 con 0x7f79a4101520 2023-12-07T10:36:17.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.838+0000 7f79a8b9a700 1 --2- 172.21.15.119:0/1238754579 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f79900a1d00 0x7f79900a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:17.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.838+0000 7f79a8b9a700 1 --2- 172.21.15.119:0/1238754579 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f79900a1d00 0x7f79900a41c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f79a419ac60 tx=0x7f7994009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:17.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:17.842+0000 7f799a7fc700 1 -- 172.21.15.119:0/1238754579 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f79a0096b50 con 0x7f79a4101520 2023-12-07T10:36:18.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.079+0000 7f79ab5ff700 1 -- 172.21.15.119:0/1238754579 --> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f79a4111440 con 0x7f79900a1d00 2023-12-07T10:36:18.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.089+0000 7f799a7fc700 1 -- 172.21.15.119:0/1238754579 <== mgr.15612 v2:172.21.15.119:6800/3763427845 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f79a4111440 con 0x7f79900a1d00 2023-12-07T10:36:18.089 INFO:teuthology.orchestra.run.smithi119.stdout:/ 1m 6m3h 2023-12-07T10:36:18.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.092+0000 7f79ab5ff700 1 -- 172.21.15.119:0/1238754579 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f79900a1d00 msgr2=0x7f79900a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:18.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.092+0000 7f79ab5ff700 1 --2- 172.21.15.119:0/1238754579 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f79900a1d00 0x7f79900a41c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f79a419ac60 tx=0x7f7994009260 comp rx=0 tx=0).stop 2023-12-07T10:36:18.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.092+0000 7f79ab5ff700 1 -- 172.21.15.119:0/1238754579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79a4101520 msgr2=0x7f79a419f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:18.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.092+0000 7f79ab5ff700 1 --2- 172.21.15.119:0/1238754579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79a4101520 0x7f79a419f970 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f79a000e9a0 tx=0x7f79a000ecb0 comp rx=0 tx=0).stop 2023-12-07T10:36:18.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.092+0000 7f79ab5ff700 1 -- 172.21.15.119:0/1238754579 shutdown_connections 2023-12-07T10:36:18.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.092+0000 7f79ab5ff700 1 --2- 172.21.15.119:0/1238754579 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f79900a1d00 0x7f79900a41c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:18.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.092+0000 7f79ab5ff700 1 --2- 172.21.15.119:0/1238754579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79a41075e0 0x7f79a4199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:18.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.092+0000 7f79ab5ff700 1 --2- 172.21.15.119:0/1238754579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79a4101520 0x7f79a419f970 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:18.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.092+0000 7f79ab5ff700 1 --2- 172.21.15.119:0/1238754579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79a4101ee0 0x7f79a419feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:18.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.092+0000 7f79ab5ff700 1 -- 172.21.15.119:0/1238754579 >> 172.21.15.119:0/1238754579 conn(0x7f79a40fd010 msgr2=0x7f79a410c760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:18.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.092+0000 7f79ab5ff700 1 -- 172.21.15.119:0/1238754579 shutdown_connections 2023-12-07T10:36:18.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.092+0000 7f79ab5ff700 1 -- 172.21.15.119:0/1238754579 wait complete. 2023-12-07T10:36:18.111 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-12-07T10:36:18.136 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-12-07T10:36:18.137 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_3bd0bfa458b4399e7266c56d8215a2cf3a6091a0/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-12-07T10:36:18.142 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-12-07T10:36:18.157 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-07T10:36:18.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.360+0000 7fdf079ff700 1 -- 172.21.15.119:0/2819809779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf001014e0 msgr2=0x7fdf00101960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:18.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.360+0000 7fdf079ff700 1 --2- 172.21.15.119:0/2819809779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf001014e0 0x7fdf00101960 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fdef00099c0 tx=0x7fdef0009cd0 comp rx=0 tx=0).stop 2023-12-07T10:36:18.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.360+0000 7fdf079ff700 1 -- 172.21.15.119:0/2819809779 shutdown_connections 2023-12-07T10:36:18.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.360+0000 7fdf079ff700 1 --2- 172.21.15.119:0/2819809779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf00101ea0 0x7fdf0010e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:18.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.360+0000 7fdf079ff700 1 --2- 172.21.15.119:0/2819809779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf001075a0 0x7fdf00107980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:18.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.360+0000 7fdf079ff700 1 --2- 172.21.15.119:0/2819809779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf001014e0 0x7fdf00101960 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:18.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.360+0000 7fdf079ff700 1 -- 172.21.15.119:0/2819809779 >> 172.21.15.119:0/2819809779 conn(0x7fdf000fcff0 msgr2=0x7fdf000ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:18.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.360+0000 7fdf079ff700 1 -- 172.21.15.119:0/2819809779 shutdown_connections 2023-12-07T10:36:18.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.360+0000 7fdf079ff700 1 -- 172.21.15.119:0/2819809779 wait complete. 2023-12-07T10:36:18.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.361+0000 7fdf079ff700 1 Processor -- start 2023-12-07T10:36:18.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.361+0000 7fdf079ff700 1 -- start start 2023-12-07T10:36:18.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.361+0000 7fdf079ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf001014e0 0x7fdf0019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:18.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.361+0000 7fdf079ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf00101ea0 0x7fdf0019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:18.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.361+0000 7fdf079ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf001075a0 0x7fdf00197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:18.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.361+0000 7fdf079ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf00197dc0 con 0x7fdf00101ea0 2023-12-07T10:36:18.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.362+0000 7fdf079ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf00197f30 con 0x7fdf001075a0 2023-12-07T10:36:18.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.362+0000 7fdf079ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf001980f0 con 0x7fdf001014e0 2023-12-07T10:36:18.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.362+0000 7fdf04f9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf00101ea0 0x7fdf0019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:18.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.362+0000 7fdf04f9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf00101ea0 0x7fdf0019db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41778/0 (socket says 172.21.15.119:41778) 2023-12-07T10:36:18.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.362+0000 7fdf04f9a700 1 -- 172.21.15.119:0/831822882 learned_addr learned my addr 172.21.15.119:0/831822882 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:36:18.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.362+0000 7fdf0579b700 1 --2- 172.21.15.119:0/831822882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf001014e0 0x7fdf0019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:18.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.362+0000 7fdf05f9c700 1 --2- 172.21.15.119:0/831822882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf001075a0 0x7fdf00197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:18.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.362+0000 7fdf04f9a700 1 -- 172.21.15.119:0/831822882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf001014e0 msgr2=0x7fdf0019d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:18.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.362+0000 7fdf04f9a700 1 --2- 172.21.15.119:0/831822882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf001014e0 0x7fdf0019d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:18.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.362+0000 7fdf04f9a700 1 -- 172.21.15.119:0/831822882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf001075a0 msgr2=0x7fdf00197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:18.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.362+0000 7fdf04f9a700 1 --2- 172.21.15.119:0/831822882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf001075a0 0x7fdf00197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:18.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.362+0000 7fdf04f9a700 1 -- 172.21.15.119:0/831822882 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef00095f0 con 0x7fdf00101ea0 2023-12-07T10:36:18.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.362+0000 7fdf05f9c700 1 --2- 172.21.15.119:0/831822882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf001075a0 0x7fdf00197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-07T10:36:18.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.362+0000 7fdf04f9a700 1 --2- 172.21.15.119:0/831822882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf00101ea0 0x7fdf0019db70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdef0006010 tx=0x7fdef000bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:18.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.362+0000 7fdef67fc700 1 -- 172.21.15.119:0/831822882 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef001d070 con 0x7fdf00101ea0 2023-12-07T10:36:18.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.363+0000 7fdf079ff700 1 -- 172.21.15.119:0/831822882 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf00198370 con 0x7fdf00101ea0 2023-12-07T10:36:18.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.363+0000 7fdef67fc700 1 -- 172.21.15.119:0/831822882 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdef0004bd0 con 0x7fdf00101ea0 2023-12-07T10:36:18.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.363+0000 7fdef67fc700 1 -- 172.21.15.119:0/831822882 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef000fbd0 con 0x7fdf00101ea0 2023-12-07T10:36:18.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.363+0000 7fdf079ff700 1 -- 172.21.15.119:0/831822882 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf001a4620 con 0x7fdf00101ea0 2023-12-07T10:36:18.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.364+0000 7fdf079ff700 1 -- 172.21.15.119:0/831822882 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf00109a40 con 0x7fdf00101ea0 2023-12-07T10:36:18.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.366+0000 7fdef67fc700 1 -- 172.21.15.119:0/831822882 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fdef0003680 con 0x7fdf00101ea0 2023-12-07T10:36:18.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.367+0000 7fdef67fc700 1 --2- 172.21.15.119:0/831822882 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fdeec0a1c80 0x7fdeec0a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:18.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.367+0000 7fdef67fc700 1 -- 172.21.15.119:0/831822882 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9549+0+0 (secure 0 0 0) 0x7fdef00ce970 con 0x7fdf00101ea0 2023-12-07T10:36:18.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.367+0000 7fdf0579b700 1 --2- 172.21.15.119:0/831822882 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fdeec0a1c80 0x7fdeec0a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:18.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.368+0000 7fdf0579b700 1 --2- 172.21.15.119:0/831822882 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fdeec0a1c80 0x7fdeec0a4140 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009be0 tx=0x7fdefc009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:18.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.371+0000 7fdef67fc700 1 -- 172.21.15.119:0/831822882 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fdef00976e0 con 0x7fdf00101ea0 2023-12-07T10:36:18.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:18 smithi119 ceph-mon[97908]: pgmap v498: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 5.3 MiB/s wr, 218 op/s 2023-12-07T10:36:18.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:18 smithi159 ceph-mon[134726]: pgmap v498: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 5.3 MiB/s wr, 218 op/s 2023-12-07T10:36:18.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:18 smithi153 ceph-mon[133625]: pgmap v498: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 5.3 MiB/s wr, 218 op/s 2023-12-07T10:36:18.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.575+0000 7fdf079ff700 1 -- 172.21.15.119:0/831822882 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf00198c20 con 0x7fdf00101ea0 2023-12-07T10:36:18.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.577+0000 7fdef67fc700 1 -- 172.21.15.119:0/831822882 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+25423 (secure 0 0 0) 0x7fdef009ad00 con 0x7fdf00101ea0 2023-12-07T10:36:18.577 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:36:18.577 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":113,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","created":"2023-12-07T10:09:59.945863+0000","modified":"2023-12-07T10:36:09.420507+0000","last_up_change":"2023-12-07T10:16:50.929427+0000","last_in_change":"2023-12-07T10:16:33.558158+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T10:13:25.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-07T10:17:39.347813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-07T10:17:40.714501+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-07T10:17:44.257760+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"56b5818e-e625-40f1-8838-3ddcf2d34ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6803","nonce":2141950133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6805","nonce":2141950133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6809","nonce":2141950133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6807","nonce":2141950133}]},"public_addr":"172.21.15.119:6803/2141950133","cluster_addr":"172.21.15.119:6805/2141950133","heartbeat_back_addr":"172.21.15.119:6809/2141950133","heartbeat_front_addr":"172.21.15.119:6807/2141950133","state":["exists","up"]},{"osd":1,"uuid":"36489a6a-5278-4802-9525-9871521722e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6801","nonce":3390875370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6803","nonce":3390875370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6807","nonce":3390875370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6805","nonce":3390875370}]},"public_addr":"172.21.15.153:6801/3390875370","cluster_addr":"172.21.15.153:6803/3390875370","heartbeat_back_addr":"172.21.15.153:6807/3390875370","heartbeat_front_addr":"172.21.15.153:6805/3390875370","state":["exists","up"]},{"osd":2,"uuid":"a937ad26-5e9b-4c8a-b65a-626a8feda6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6801","nonce":708101637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6803","nonce":708101637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6807","nonce":708101637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6805","nonce":708101637}]},"public_addr":"172.21.15.159:6801/708101637","cluster_addr":"172.21.15.159:6803/708101637","heartbeat_back_addr":"172.21.15.159:6807/708101637","heartbeat_front_addr":"172.21.15.159:6805/708101637","state":["exists","up"]},{"osd":3,"uuid":"26ef8300-aa07-48f6-b31d-8b0f2217ffcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6811","nonce":3228047257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6813","nonce":3228047257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6817","nonce":3228047257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6815","nonce":3228047257}]},"public_addr":"172.21.15.119:6811/3228047257","cluster_addr":"172.21.15.119:6813/3228047257","heartbeat_back_addr":"172.21.15.119:6817/3228047257","heartbeat_front_addr":"172.21.15.119:6815/3228047257","state":["exists","up"]},{"osd":4,"uuid":"000ee4b9-9896-4901-8cc3-c85fdfc8cbaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6809","nonce":4021998299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6811","nonce":4021998299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6815","nonce":4021998299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6813","nonce":4021998299}]},"public_addr":"172.21.15.153:6809/4021998299","cluster_addr":"172.21.15.153:6811/4021998299","heartbeat_back_addr":"172.21.15.153:6815/4021998299","heartbeat_front_addr":"172.21.15.153:6813/4021998299","state":["exists","up"]},{"osd":5,"uuid":"8b2fc167-262d-474f-89f3-47bd29424421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6809","nonce":1832540768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6811","nonce":1832540768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6815","nonce":1832540768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6813","nonce":1832540768}]},"public_addr":"172.21.15.159:6809/1832540768","cluster_addr":"172.21.15.159:6811/1832540768","heartbeat_back_addr":"172.21.15.159:6815/1832540768","heartbeat_front_addr":"172.21.15.159:6813/1832540768","state":["exists","up"]},{"osd":6,"uuid":"f9aa6749-46cf-4b2a-8114-cbd034219d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162436840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162436840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6825","nonce":1162436840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162436840}]},"public_addr":"172.21.15.119:6819/1162436840","cluster_addr":"172.21.15.119:6821/1162436840","heartbeat_back_addr":"172.21.15.119:6825/1162436840","heartbeat_front_addr":"172.21.15.119:6823/1162436840","state":["exists","up"]},{"osd":7,"uuid":"39216361-e1c4-4fef-86cb-a4707656b547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6817","nonce":3254653457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6819","nonce":3254653457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6823","nonce":3254653457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6821","nonce":3254653457}]},"public_addr":"172.21.15.153:6817/3254653457","cluster_addr":"172.21.15.153:6819/3254653457","heartbeat_back_addr":"172.21.15.153:6823/3254653457","heartbeat_front_addr":"172.21.15.153:6821/3254653457","state":["exists","up"]},{"osd":8,"uuid":"cceb6ebc-7c02-48ea-9d16-b7a016dd449b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6817","nonce":2600946380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6819","nonce":2600946380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6823","nonce":2600946380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6821","nonce":2600946380}]},"public_addr":"172.21.15.159:6817/2600946380","cluster_addr":"172.21.15.159:6819/2600946380","heartbeat_back_addr":"172.21.15.159:6823/2600946380","heartbeat_front_addr":"172.21.15.159:6821/2600946380","state":["exists","up"]},{"osd":9,"uuid":"f337a2e4-e065-4a9f-91d9-e3615de2d6f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6827","nonce":1940737285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6829","nonce":1940737285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6833","nonce":1940737285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6831","nonce":1940737285}]},"public_addr":"172.21.15.119:6827/1940737285","cluster_addr":"172.21.15.119:6829/1940737285","heartbeat_back_addr":"172.21.15.119:6833/1940737285","heartbeat_front_addr":"172.21.15.119:6831/1940737285","state":["exists","up"]},{"osd":10,"uuid":"3a3bc347-85f4-4878-9797-d9da96137e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6825","nonce":663834864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6827","nonce":663834864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6831","nonce":663834864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6829","nonce":663834864}]},"public_addr":"172.21.15.153:6825/663834864","cluster_addr":"172.21.15.153:6827/663834864","heartbeat_back_addr":"172.21.15.153:6831/663834864","heartbeat_front_addr":"172.21.15.153:6829/663834864","state":["exists","up"]},{"osd":11,"uuid":"a0fc5909-279d-4172-977c-2dc2da0a7e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6825","nonce":1262216348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6827","nonce":1262216348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6831","nonce":1262216348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6829","nonce":1262216348}]},"public_addr":"172.21.15.159:6825/1262216348","cluster_addr":"172.21.15.159:6827/1262216348","heartbeat_back_addr":"172.21.15.159:6831/1262216348","heartbeat_front_addr":"172.21.15.159:6829/1262216348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:37.488109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:59.543060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:21.381585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:47.112729+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:09.732150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:30.893729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:56.678710+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:19.004256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:40.388818+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:06.391734+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:28.394053+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:49.634348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/2490678270":"2023-12-08T10:20:24.856450+0000","172.21.15.119:0/2580180729":"2023-12-08T10:20:24.856450+0000","172.21.15.119:0/149289687":"2023-12-08T10:20:24.856450+0000","172.21.15.119:0/2050367825":"2023-12-08T10:20:24.856450+0000","172.21.15.119:6800/1344846367":"2023-12-08T10:20:24.856450+0000","172.21.15.119:0/1628639392":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6801/1355617847":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6800/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/362307836":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/1344846367":"2023-12-08T10:20:24.856450+0000","172.21.15.119:0/128964558":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/2278813554":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/1411933614":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/1685698277":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6800/1355617847":"2023-12-08T10:11:00.585565+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T10:36:18.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.580+0000 7fdf079ff700 1 -- 172.21.15.119:0/831822882 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fdeec0a1c80 msgr2=0x7fdeec0a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:18.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.580+0000 7fdf079ff700 1 --2- 172.21.15.119:0/831822882 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fdeec0a1c80 0x7fdeec0a4140 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009be0 tx=0x7fdefc009260 comp rx=0 tx=0).stop 2023-12-07T10:36:18.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.580+0000 7fdf079ff700 1 -- 172.21.15.119:0/831822882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf00101ea0 msgr2=0x7fdf0019db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:18.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.580+0000 7fdf079ff700 1 --2- 172.21.15.119:0/831822882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf00101ea0 0x7fdf0019db70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdef0006010 tx=0x7fdef000bb00 comp rx=0 tx=0).stop 2023-12-07T10:36:18.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.580+0000 7fdf079ff700 1 -- 172.21.15.119:0/831822882 shutdown_connections 2023-12-07T10:36:18.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.580+0000 7fdf079ff700 1 --2- 172.21.15.119:0/831822882 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fdeec0a1c80 0x7fdeec0a4140 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:18.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.580+0000 7fdf079ff700 1 --2- 172.21.15.119:0/831822882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf001075a0 0x7fdf00197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:18.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.580+0000 7fdf079ff700 1 --2- 172.21.15.119:0/831822882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf001014e0 0x7fdf0019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:18.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.580+0000 7fdf079ff700 1 --2- 172.21.15.119:0/831822882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf00101ea0 0x7fdf0019db70 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:18.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.580+0000 7fdf079ff700 1 -- 172.21.15.119:0/831822882 >> 172.21.15.119:0/831822882 conn(0x7fdf000fcff0 msgr2=0x7fdf000fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:18.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.580+0000 7fdf079ff700 1 -- 172.21.15.119:0/831822882 shutdown_connections 2023-12-07T10:36:18.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.580+0000 7fdf079ff700 1 -- 172.21.15.119:0/831822882 wait complete. 2023-12-07T10:36:18.603 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-07T10:13:25.194366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-07T10:17:39.347813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-07T10:17:40.714501+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 113, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-07T10:17:44.257760+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 113, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-07T10:36:18.603 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-07T10:36:18.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.804+0000 7fcb621ff700 1 -- 172.21.15.119:0/2728556483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb5c106110 msgr2=0x7fcb5c106590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:18.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.804+0000 7fcb621ff700 1 --2- 172.21.15.119:0/2728556483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb5c106110 0x7fcb5c106590 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fcb44009990 tx=0x7fcb44009ca0 comp rx=0 tx=0).stop 2023-12-07T10:36:18.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.804+0000 7fcb621ff700 1 -- 172.21.15.119:0/2728556483 shutdown_connections 2023-12-07T10:36:18.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.804+0000 7fcb621ff700 1 --2- 172.21.15.119:0/2728556483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb5c106ad0 0x7fcb5c10e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:18.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.804+0000 7fcb621ff700 1 --2- 172.21.15.119:0/2728556483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb5c068a50 0x7fcb5c068e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:18.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.804+0000 7fcb621ff700 1 --2- 172.21.15.119:0/2728556483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb5c106110 0x7fcb5c106590 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:18.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.804+0000 7fcb621ff700 1 -- 172.21.15.119:0/2728556483 >> 172.21.15.119:0/2728556483 conn(0x7fcb5c077610 msgr2=0x7fcb5c077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:18.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.804+0000 7fcb621ff700 1 -- 172.21.15.119:0/2728556483 shutdown_connections 2023-12-07T10:36:18.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.804+0000 7fcb621ff700 1 -- 172.21.15.119:0/2728556483 wait complete. 2023-12-07T10:36:18.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.805+0000 7fcb621ff700 1 Processor -- start 2023-12-07T10:36:18.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.805+0000 7fcb621ff700 1 -- start start 2023-12-07T10:36:18.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.805+0000 7fcb621ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb5c068a50 0x7fcb5c19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:18.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.805+0000 7fcb621ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb5c106110 0x7fcb5c19dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:18.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.806+0000 7fcb621ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb5c106ad0 0x7fcb5c1977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:18.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.806+0000 7fcb621ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c197e60 con 0x7fcb5c068a50 2023-12-07T10:36:18.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.806+0000 7fcb621ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c197fd0 con 0x7fcb5c106ad0 2023-12-07T10:36:18.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.806+0000 7fcb621ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c198190 con 0x7fcb5c106110 2023-12-07T10:36:18.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.806+0000 7fcb5b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb5c068a50 0x7fcb5c19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:18.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.806+0000 7fcb53fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb5c106110 0x7fcb5c19dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:18.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.806+0000 7fcb5b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb5c068a50 0x7fcb5c19d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41790/0 (socket says 172.21.15.119:41790) 2023-12-07T10:36:18.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.806+0000 7fcb5b7fe700 1 -- 172.21.15.119:0/4098490763 learned_addr learned my addr 172.21.15.119:0/4098490763 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:36:18.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.806+0000 7fcb5bfff700 1 --2- 172.21.15.119:0/4098490763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb5c106ad0 0x7fcb5c1977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:18.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.806+0000 7fcb5b7fe700 1 -- 172.21.15.119:0/4098490763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb5c106110 msgr2=0x7fcb5c19dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:18.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.806+0000 7fcb5b7fe700 1 --2- 172.21.15.119:0/4098490763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb5c106110 0x7fcb5c19dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:18.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.806+0000 7fcb5b7fe700 1 -- 172.21.15.119:0/4098490763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb5c106ad0 msgr2=0x7fcb5c1977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:18.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.806+0000 7fcb5b7fe700 1 --2- 172.21.15.119:0/4098490763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb5c106ad0 0x7fcb5c1977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:18.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.806+0000 7fcb5b7fe700 1 -- 172.21.15.119:0/4098490763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb440095f0 con 0x7fcb5c068a50 2023-12-07T10:36:18.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.806+0000 7fcb5b7fe700 1 --2- 172.21.15.119:0/4098490763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb5c068a50 0x7fcb5c19d6c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00d980 tx=0x7fcb4c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:18.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.807+0000 7fcb597fa700 1 -- 172.21.15.119:0/4098490763 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c004220 con 0x7fcb5c068a50 2023-12-07T10:36:18.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.807+0000 7fcb597fa700 1 -- 172.21.15.119:0/4098490763 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb4c007640 con 0x7fcb5c068a50 2023-12-07T10:36:18.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.807+0000 7fcb621ff700 1 -- 172.21.15.119:0/4098490763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb5c198470 con 0x7fcb5c068a50 2023-12-07T10:36:18.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.807+0000 7fcb597fa700 1 -- 172.21.15.119:0/4098490763 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c005610 con 0x7fcb5c068a50 2023-12-07T10:36:18.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.807+0000 7fcb621ff700 1 -- 172.21.15.119:0/4098490763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb5c1a46b0 con 0x7fcb5c068a50 2023-12-07T10:36:18.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.808+0000 7fcb621ff700 1 -- 172.21.15.119:0/4098490763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb5c109a90 con 0x7fcb5c068a50 2023-12-07T10:36:18.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.811+0000 7fcb597fa700 1 -- 172.21.15.119:0/4098490763 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fcb4c004750 con 0x7fcb5c068a50 2023-12-07T10:36:18.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.812+0000 7fcb597fa700 1 --2- 172.21.15.119:0/4098490763 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fcb480a1d00 0x7fcb480a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:18.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.812+0000 7fcb597fa700 1 -- 172.21.15.119:0/4098490763 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9549+0+0 (secure 0 0 0) 0x7fcb4c0ce0b0 con 0x7fcb5c068a50 2023-12-07T10:36:18.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.813+0000 7fcb53fff700 1 --2- 172.21.15.119:0/4098490763 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fcb480a1d00 0x7fcb480a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:18.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.813+0000 7fcb53fff700 1 --2- 172.21.15.119:0/4098490763 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fcb480a1d00 0x7fcb480a41c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcb440098e0 tx=0x7fcb44005960 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:18.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:18.814+0000 7fcb597fa700 1 -- 172.21.15.119:0/4098490763 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fcb4c096d30 con 0x7fcb5c068a50 2023-12-07T10:36:19.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.016+0000 7fcb621ff700 1 -- 172.21.15.119:0/4098490763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcb5c198cb0 con 0x7fcb5c068a50 2023-12-07T10:36:19.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.017+0000 7fcb597fa700 1 -- 172.21.15.119:0/4098490763 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcb4c0160a0 con 0x7fcb5c068a50 2023-12-07T10:36:19.017 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-12-07T10:36:19.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.019+0000 7fcb621ff700 1 -- 172.21.15.119:0/4098490763 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fcb480a1d00 msgr2=0x7fcb480a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:19.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.019+0000 7fcb621ff700 1 --2- 172.21.15.119:0/4098490763 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fcb480a1d00 0x7fcb480a41c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcb440098e0 tx=0x7fcb44005960 comp rx=0 tx=0).stop 2023-12-07T10:36:19.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.019+0000 7fcb621ff700 1 -- 172.21.15.119:0/4098490763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb5c068a50 msgr2=0x7fcb5c19d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:19.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.019+0000 7fcb621ff700 1 --2- 172.21.15.119:0/4098490763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb5c068a50 0x7fcb5c19d6c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00d980 tx=0x7fcb4c00dc90 comp rx=0 tx=0).stop 2023-12-07T10:36:19.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.020+0000 7fcb621ff700 1 -- 172.21.15.119:0/4098490763 shutdown_connections 2023-12-07T10:36:19.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.020+0000 7fcb621ff700 1 --2- 172.21.15.119:0/4098490763 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fcb480a1d00 0x7fcb480a41c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:19.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.020+0000 7fcb621ff700 1 --2- 172.21.15.119:0/4098490763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb5c106ad0 0x7fcb5c1977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:19.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.020+0000 7fcb621ff700 1 --2- 172.21.15.119:0/4098490763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb5c068a50 0x7fcb5c19d6c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:19.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.020+0000 7fcb621ff700 1 --2- 172.21.15.119:0/4098490763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb5c106110 0x7fcb5c19dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:19.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.020+0000 7fcb621ff700 1 -- 172.21.15.119:0/4098490763 >> 172.21.15.119:0/4098490763 conn(0x7fcb5c077610 msgr2=0x7fcb5c0fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:19.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.020+0000 7fcb621ff700 1 -- 172.21.15.119:0/4098490763 shutdown_connections 2023-12-07T10:36:19.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.020+0000 7fcb621ff700 1 -- 172.21.15.119:0/4098490763 wait complete. 2023-12-07T10:36:19.038 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-07T10:36:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.241+0000 7f6122fff700 1 -- 172.21.15.119:0/3193151111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f611c101f20 msgr2=0x7f611c10e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.241+0000 7f6122fff700 1 --2- 172.21.15.119:0/3193151111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f611c101f20 0x7f611c10e3e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f611400b0d0 tx=0x7f611400b3e0 comp rx=0 tx=0).stop 2023-12-07T10:36:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.241+0000 7f6122fff700 1 -- 172.21.15.119:0/3193151111 shutdown_connections 2023-12-07T10:36:19.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.241+0000 7f6122fff700 1 --2- 172.21.15.119:0/3193151111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f611c101f20 0x7f611c10e3e0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:19.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.241+0000 7f6122fff700 1 --2- 172.21.15.119:0/3193151111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f611c107620 0x7f611c107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:19.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.241+0000 7f6122fff700 1 --2- 172.21.15.119:0/3193151111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f611c101560 0x7f611c1019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:19.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.241+0000 7f6122fff700 1 -- 172.21.15.119:0/3193151111 >> 172.21.15.119:0/3193151111 conn(0x7f611c0fcfd0 msgr2=0x7f611c0ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:19.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.241+0000 7f6122fff700 1 -- 172.21.15.119:0/3193151111 shutdown_connections 2023-12-07T10:36:19.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.241+0000 7f6122fff700 1 -- 172.21.15.119:0/3193151111 wait complete. 2023-12-07T10:36:19.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.242+0000 7f6122fff700 1 Processor -- start 2023-12-07T10:36:19.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.242+0000 7f6122fff700 1 -- start start 2023-12-07T10:36:19.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.243+0000 7f6122fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f611c101560 0x7f611c199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:19.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.243+0000 7f6122fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f611c101f20 0x7f611c19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:19.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.243+0000 7f6122fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f611c107620 0x7f611c19d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:19.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.243+0000 7f6122fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611c19db60 con 0x7f611c101560 2023-12-07T10:36:19.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.243+0000 7f6122fff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611c19dcd0 con 0x7f611c107620 2023-12-07T10:36:19.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.243+0000 7f6122fff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611c19de90 con 0x7f611c101f20 2023-12-07T10:36:19.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.243+0000 7f6120d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f611c101560 0x7f611c199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:19.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.243+0000 7f6120d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f611c101560 0x7f611c199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41810/0 (socket says 172.21.15.119:41810) 2023-12-07T10:36:19.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.243+0000 7f6120d9b700 1 -- 172.21.15.119:0/1000961030 learned_addr learned my addr 172.21.15.119:0/1000961030 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:36:19.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.243+0000 7f611bfff700 1 --2- 172.21.15.119:0/1000961030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f611c101f20 0x7f611c19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:19.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.243+0000 7f612159c700 1 --2- 172.21.15.119:0/1000961030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f611c107620 0x7f611c19d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:19.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.243+0000 7f6120d9b700 1 -- 172.21.15.119:0/1000961030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f611c101f20 msgr2=0x7f611c19a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:19.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.243+0000 7f6120d9b700 1 --2- 172.21.15.119:0/1000961030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f611c101f20 0x7f611c19a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:19.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.243+0000 7f6120d9b700 1 -- 172.21.15.119:0/1000961030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f611c107620 msgr2=0x7f611c19d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:19.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.243+0000 7f6120d9b700 1 --2- 172.21.15.119:0/1000961030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f611c107620 0x7f611c19d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:19.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.243+0000 7f6120d9b700 1 -- 172.21.15.119:0/1000961030 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6114009d00 con 0x7f611c101560 2023-12-07T10:36:19.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.243+0000 7f612159c700 1 --2- 172.21.15.119:0/1000961030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f611c107620 0x7f611c19d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:36:19.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.243+0000 7f6120d9b700 1 --2- 172.21.15.119:0/1000961030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f611c101560 0x7f611c199d30 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f611000e9a0 tx=0x7f611000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:19.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.244+0000 7f6119ffb700 1 -- 172.21.15.119:0/1000961030 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6110009790 con 0x7f611c101560 2023-12-07T10:36:19.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.244+0000 7f6119ffb700 1 -- 172.21.15.119:0/1000961030 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f611000cbd0 con 0x7f611c101560 2023-12-07T10:36:19.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.244+0000 7f6122fff700 1 -- 172.21.15.119:0/1000961030 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f611c19e170 con 0x7f611c101560 2023-12-07T10:36:19.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.244+0000 7f6119ffb700 1 -- 172.21.15.119:0/1000961030 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6110018680 con 0x7f611c101560 2023-12-07T10:36:19.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.244+0000 7f6122fff700 1 -- 172.21.15.119:0/1000961030 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f611c1a6ad0 con 0x7f611c101560 2023-12-07T10:36:19.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.245+0000 7f6122fff700 1 -- 172.21.15.119:0/1000961030 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f611c102c80 con 0x7f611c101560 2023-12-07T10:36:19.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.247+0000 7f6119ffb700 1 -- 172.21.15.119:0/1000961030 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f611000cd40 con 0x7f611c101560 2023-12-07T10:36:19.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.248+0000 7f6119ffb700 1 --2- 172.21.15.119:0/1000961030 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f610c0a1d00 0x7f610c0a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:19.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.248+0000 7f6119ffb700 1 -- 172.21.15.119:0/1000961030 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f6110014070 con 0x7f611c101560 2023-12-07T10:36:19.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.248+0000 7f611bfff700 1 --2- 172.21.15.119:0/1000961030 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f610c0a1d00 0x7f610c0a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:19.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.249+0000 7f611bfff700 1 --2- 172.21.15.119:0/1000961030 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f610c0a1d00 0x7f610c0a41c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f611c19b240 tx=0x7f6108009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:19.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.252+0000 7f6119ffb700 1 -- 172.21.15.119:0/1000961030 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f61100522f0 con 0x7f611c101560 2023-12-07T10:36:19.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.454+0000 7f6122fff700 1 -- 172.21.15.119:0/1000961030 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f611c1114d0 con 0x7f611c101560 2023-12-07T10:36:19.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.454+0000 7f6119ffb700 1 -- 172.21.15.119:0/1000961030 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6110096c00 con 0x7f611c101560 2023-12-07T10:36:19.455 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-12-07T10:36:19.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.457+0000 7f6122fff700 1 -- 172.21.15.119:0/1000961030 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f610c0a1d00 msgr2=0x7f610c0a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:19.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.457+0000 7f6122fff700 1 --2- 172.21.15.119:0/1000961030 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f610c0a1d00 0x7f610c0a41c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f611c19b240 tx=0x7f6108009260 comp rx=0 tx=0).stop 2023-12-07T10:36:19.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.457+0000 7f6122fff700 1 -- 172.21.15.119:0/1000961030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f611c101560 msgr2=0x7f611c199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:19.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.457+0000 7f6122fff700 1 --2- 172.21.15.119:0/1000961030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f611c101560 0x7f611c199d30 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f611000e9a0 tx=0x7f611000ecb0 comp rx=0 tx=0).stop 2023-12-07T10:36:19.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.457+0000 7f6122fff700 1 -- 172.21.15.119:0/1000961030 shutdown_connections 2023-12-07T10:36:19.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.457+0000 7f6122fff700 1 --2- 172.21.15.119:0/1000961030 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f610c0a1d00 0x7f610c0a41c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:19.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.457+0000 7f6122fff700 1 --2- 172.21.15.119:0/1000961030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f611c107620 0x7f611c19d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:19.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.457+0000 7f6122fff700 1 --2- 172.21.15.119:0/1000961030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f611c101560 0x7f611c199d30 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:19.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.457+0000 7f6122fff700 1 --2- 172.21.15.119:0/1000961030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f611c101f20 0x7f611c19a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:19.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.457+0000 7f6122fff700 1 -- 172.21.15.119:0/1000961030 >> 172.21.15.119:0/1000961030 conn(0x7f611c0fcfd0 msgr2=0x7f611c10d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:19.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.457+0000 7f6122fff700 1 -- 172.21.15.119:0/1000961030 shutdown_connections 2023-12-07T10:36:19.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.458+0000 7f6122fff700 1 -- 172.21.15.119:0/1000961030 wait complete. 2023-12-07T10:36:19.474 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:19 smithi119 ceph-mon[97908]: from='client.28037 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:36:19.474 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:19 smithi119 ceph-mon[97908]: from='client.17952 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:36:19.474 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:19 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/831822882' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:36:19.475 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:19 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/4098490763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:36:19.477 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-07T10:36:19.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:19 smithi159 ceph-mon[134726]: from='client.28037 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:36:19.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:19 smithi159 ceph-mon[134726]: from='client.17952 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:36:19.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:19 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/831822882' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:36:19.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:19 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/4098490763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:36:19.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:19 smithi153 ceph-mon[133625]: from='client.28037 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:36:19.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:19 smithi153 ceph-mon[133625]: from='client.17952 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-07T10:36:19.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:19 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/831822882' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:36:19.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:19 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/4098490763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:36:19.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.675+0000 7fe6aa1ff700 1 -- 172.21.15.119:0/1565693142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6a4101560 msgr2=0x7fe6a41019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:19.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.675+0000 7fe6aa1ff700 1 --2- 172.21.15.119:0/1565693142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6a4101560 0x7fe6a41019e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe6980099c0 tx=0x7fe698009cd0 comp rx=0 tx=0).stop 2023-12-07T10:36:19.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.675+0000 7fe6aa1ff700 1 -- 172.21.15.119:0/1565693142 shutdown_connections 2023-12-07T10:36:19.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.675+0000 7fe6aa1ff700 1 --2- 172.21.15.119:0/1565693142 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6a4101f20 0x7fe6a410e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:19.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.675+0000 7fe6aa1ff700 1 --2- 172.21.15.119:0/1565693142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6a4107620 0x7fe6a4107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:19.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.675+0000 7fe6aa1ff700 1 --2- 172.21.15.119:0/1565693142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6a4101560 0x7fe6a41019e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:19.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.675+0000 7fe6aa1ff700 1 -- 172.21.15.119:0/1565693142 >> 172.21.15.119:0/1565693142 conn(0x7fe6a40fcfd0 msgr2=0x7fe6a40ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:19.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.676+0000 7fe6aa1ff700 1 -- 172.21.15.119:0/1565693142 shutdown_connections 2023-12-07T10:36:19.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.676+0000 7fe6aa1ff700 1 -- 172.21.15.119:0/1565693142 wait complete. 2023-12-07T10:36:19.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.676+0000 7fe6aa1ff700 1 Processor -- start 2023-12-07T10:36:19.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.677+0000 7fe6aa1ff700 1 -- start start 2023-12-07T10:36:19.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.677+0000 7fe6aa1ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6a4101560 0x7fe6a4073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:19.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.677+0000 7fe6aa1ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6a4101f20 0x7fe6a4073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:19.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.677+0000 7fe6aa1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6a4107620 0x7fe6a4074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:19.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.677+0000 7fe6aa1ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a41a5d70 con 0x7fe6a4101f20 2023-12-07T10:36:19.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.677+0000 7fe6aa1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a41a5ee0 con 0x7fe6a4107620 2023-12-07T10:36:19.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.677+0000 7fe6aa1ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a41a60a0 con 0x7fe6a4101560 2023-12-07T10:36:19.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.677+0000 7fe6a2ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6a4101f20 0x7fe6a4073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:19.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.677+0000 7fe6a2ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6a4101f20 0x7fe6a4073e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41830/0 (socket says 172.21.15.119:41830) 2023-12-07T10:36:19.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.677+0000 7fe6a2ffd700 1 -- 172.21.15.119:0/3142650525 learned_addr learned my addr 172.21.15.119:0/3142650525 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:36:19.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.677+0000 7fe6a37fe700 1 --2- 172.21.15.119:0/3142650525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6a4101560 0x7fe6a4073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:19.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.677+0000 7fe6a3fff700 1 --2- 172.21.15.119:0/3142650525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6a4107620 0x7fe6a4074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:19.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.678+0000 7fe6a2ffd700 1 -- 172.21.15.119:0/3142650525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6a4101560 msgr2=0x7fe6a4073910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:19.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.678+0000 7fe6a2ffd700 1 --2- 172.21.15.119:0/3142650525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6a4101560 0x7fe6a4073910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:19.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.678+0000 7fe6a2ffd700 1 -- 172.21.15.119:0/3142650525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6a4107620 msgr2=0x7fe6a4074390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:19.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.678+0000 7fe6a2ffd700 1 --2- 172.21.15.119:0/3142650525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6a4107620 0x7fe6a4074390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:19.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.678+0000 7fe6a2ffd700 1 -- 172.21.15.119:0/3142650525 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6980095f0 con 0x7fe6a4101f20 2023-12-07T10:36:19.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.678+0000 7fe6a2ffd700 1 --2- 172.21.15.119:0/3142650525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6a4101f20 0x7fe6a4073e50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe698009990 tx=0x7fe698004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:19.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.678+0000 7fe6a0ff9700 1 -- 172.21.15.119:0/3142650525 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69801d070 con 0x7fe6a4101f20 2023-12-07T10:36:19.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.678+0000 7fe6aa1ff700 1 -- 172.21.15.119:0/3142650525 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a41a6320 con 0x7fe6a4101f20 2023-12-07T10:36:19.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.678+0000 7fe6a0ff9700 1 -- 172.21.15.119:0/3142650525 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe698004ca0 con 0x7fe6a4101f20 2023-12-07T10:36:19.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.678+0000 7fe6a0ff9700 1 -- 172.21.15.119:0/3142650525 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69800f8d0 con 0x7fe6a4101f20 2023-12-07T10:36:19.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.678+0000 7fe6aa1ff700 1 -- 172.21.15.119:0/3142650525 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a41a6750 con 0x7fe6a4101f20 2023-12-07T10:36:19.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.679+0000 7fe6aa1ff700 1 -- 172.21.15.119:0/3142650525 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a4109ac0 con 0x7fe6a4101f20 2023-12-07T10:36:19.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.681+0000 7fe6a0ff9700 1 -- 172.21.15.119:0/3142650525 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fe698005000 con 0x7fe6a4101f20 2023-12-07T10:36:19.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.683+0000 7fe6a0ff9700 1 --2- 172.21.15.119:0/3142650525 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe68c0a1c80 0x7fe68c0a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:19.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.683+0000 7fe6a0ff9700 1 -- 172.21.15.119:0/3142650525 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9549+0+0 (secure 0 0 0) 0x7fe6980ceb70 con 0x7fe6a4101f20 2023-12-07T10:36:19.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.683+0000 7fe6a37fe700 1 --2- 172.21.15.119:0/3142650525 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe68c0a1c80 0x7fe68c0a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:19.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.684+0000 7fe6a37fe700 1 --2- 172.21.15.119:0/3142650525 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe68c0a1c80 0x7fe68c0a4140 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe694005950 tx=0x7fe69400b410 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:19.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.687+0000 7fe6a0ff9700 1 -- 172.21.15.119:0/3142650525 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe698097880 con 0x7fe6a4101f20 2023-12-07T10:36:19.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.891+0000 7fe6aa1ff700 1 -- 172.21.15.119:0/3142650525 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe6a4074df0 con 0x7fe6a4101f20 2023-12-07T10:36:19.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.891+0000 7fe6a0ff9700 1 -- 172.21.15.119:0/3142650525 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe698026020 con 0x7fe6a4101f20 2023-12-07T10:36:19.891 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-12-07T10:36:19.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.894+0000 7fe6aa1ff700 1 -- 172.21.15.119:0/3142650525 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe68c0a1c80 msgr2=0x7fe68c0a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:19.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.894+0000 7fe6aa1ff700 1 --2- 172.21.15.119:0/3142650525 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe68c0a1c80 0x7fe68c0a4140 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe694005950 tx=0x7fe69400b410 comp rx=0 tx=0).stop 2023-12-07T10:36:19.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.894+0000 7fe6aa1ff700 1 -- 172.21.15.119:0/3142650525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6a4101f20 msgr2=0x7fe6a4073e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:19.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.894+0000 7fe6aa1ff700 1 --2- 172.21.15.119:0/3142650525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6a4101f20 0x7fe6a4073e50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe698009990 tx=0x7fe698004730 comp rx=0 tx=0).stop 2023-12-07T10:36:19.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.894+0000 7fe6aa1ff700 1 -- 172.21.15.119:0/3142650525 shutdown_connections 2023-12-07T10:36:19.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.894+0000 7fe6aa1ff700 1 --2- 172.21.15.119:0/3142650525 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fe68c0a1c80 0x7fe68c0a4140 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:19.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.894+0000 7fe6aa1ff700 1 --2- 172.21.15.119:0/3142650525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6a4107620 0x7fe6a4074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:19.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.894+0000 7fe6aa1ff700 1 --2- 172.21.15.119:0/3142650525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6a4101560 0x7fe6a4073910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:19.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.894+0000 7fe6aa1ff700 1 --2- 172.21.15.119:0/3142650525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6a4101f20 0x7fe6a4073e50 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:19.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.894+0000 7fe6aa1ff700 1 -- 172.21.15.119:0/3142650525 >> 172.21.15.119:0/3142650525 conn(0x7fe6a40fcfd0 msgr2=0x7fe6a40fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:19.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.894+0000 7fe6aa1ff700 1 -- 172.21.15.119:0/3142650525 shutdown_connections 2023-12-07T10:36:19.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:19.894+0000 7fe6aa1ff700 1 -- 172.21.15.119:0/3142650525 wait complete. 2023-12-07T10:36:19.913 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-07T10:36:20.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.114+0000 7fa2385ff700 1 -- 172.21.15.119:0/2699033365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa230101560 msgr2=0x7fa2301019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:20.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.114+0000 7fa2385ff700 1 --2- 172.21.15.119:0/2699033365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa230101560 0x7fa2301019e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fa220009920 tx=0x7fa220009c30 comp rx=0 tx=0).stop 2023-12-07T10:36:20.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.114+0000 7fa2385ff700 1 -- 172.21.15.119:0/2699033365 shutdown_connections 2023-12-07T10:36:20.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.114+0000 7fa2385ff700 1 --2- 172.21.15.119:0/2699033365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa230101f20 0x7fa23010e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:20.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.114+0000 7fa2385ff700 1 --2- 172.21.15.119:0/2699033365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa230107620 0x7fa230107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:20.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.114+0000 7fa2385ff700 1 --2- 172.21.15.119:0/2699033365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa230101560 0x7fa2301019e0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:20.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.114+0000 7fa2385ff700 1 -- 172.21.15.119:0/2699033365 >> 172.21.15.119:0/2699033365 conn(0x7fa2300fcfd0 msgr2=0x7fa2300ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:20.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.114+0000 7fa2385ff700 1 -- 172.21.15.119:0/2699033365 shutdown_connections 2023-12-07T10:36:20.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.114+0000 7fa2385ff700 1 -- 172.21.15.119:0/2699033365 wait complete. 2023-12-07T10:36:20.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.115+0000 7fa2385ff700 1 Processor -- start 2023-12-07T10:36:20.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.115+0000 7fa2385ff700 1 -- start start 2023-12-07T10:36:20.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.116+0000 7fa2385ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa230101560 0x7fa23019f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:20.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.116+0000 7fa2385ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa230101f20 0x7fa23019ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:20.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.116+0000 7fa2385ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa230107620 0x7fa230199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:20.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.116+0000 7fa2385ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23019a150 con 0x7fa230101f20 2023-12-07T10:36:20.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.116+0000 7fa2385ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23019a2c0 con 0x7fa230107620 2023-12-07T10:36:20.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.116+0000 7fa2385ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23019a480 con 0x7fa230101560 2023-12-07T10:36:20.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.116+0000 7fa235b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa230101f20 0x7fa23019ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:20.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.116+0000 7fa23639b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa230101560 0x7fa23019f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:20.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.116+0000 7fa235b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa230101f20 0x7fa23019ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41850/0 (socket says 172.21.15.119:41850) 2023-12-07T10:36:20.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.116+0000 7fa235b9a700 1 -- 172.21.15.119:0/3919050984 learned_addr learned my addr 172.21.15.119:0/3919050984 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:36:20.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.116+0000 7fa236b9c700 1 --2- 172.21.15.119:0/3919050984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa230107620 0x7fa230199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:20.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.116+0000 7fa235b9a700 1 -- 172.21.15.119:0/3919050984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa230101560 msgr2=0x7fa23019f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:20.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.116+0000 7fa235b9a700 1 --2- 172.21.15.119:0/3919050984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa230101560 0x7fa23019f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:20.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.116+0000 7fa235b9a700 1 -- 172.21.15.119:0/3919050984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa230107620 msgr2=0x7fa230199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:20.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.116+0000 7fa235b9a700 1 --2- 172.21.15.119:0/3919050984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa230107620 0x7fa230199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:20.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.116+0000 7fa235b9a700 1 -- 172.21.15.119:0/3919050984 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22c0095f0 con 0x7fa230101f20 2023-12-07T10:36:20.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.116+0000 7fa23639b700 1 --2- 172.21.15.119:0/3919050984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa230101560 0x7fa23019f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:36:20.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.117+0000 7fa235b9a700 1 --2- 172.21.15.119:0/3919050984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa230101f20 0x7fa23019ff00 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa220000c00 tx=0x7fa22000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:20.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.117+0000 7fa2277fe700 1 -- 172.21.15.119:0/3919050984 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22001c070 con 0x7fa230101f20 2023-12-07T10:36:20.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.117+0000 7fa2277fe700 1 -- 172.21.15.119:0/3919050984 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa22000ba20 con 0x7fa230101f20 2023-12-07T10:36:20.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.117+0000 7fa2385ff700 1 -- 172.21.15.119:0/3919050984 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa220009520 con 0x7fa230101f20 2023-12-07T10:36:20.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.117+0000 7fa2277fe700 1 -- 172.21.15.119:0/3919050984 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2200038e0 con 0x7fa230101f20 2023-12-07T10:36:20.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.117+0000 7fa2385ff700 1 -- 172.21.15.119:0/3919050984 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa23019a930 con 0x7fa230101f20 2023-12-07T10:36:20.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.118+0000 7fa2385ff700 1 -- 172.21.15.119:0/3919050984 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa230102c80 con 0x7fa230101f20 2023-12-07T10:36:20.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.120+0000 7fa2277fe700 1 -- 172.21.15.119:0/3919050984 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fa220020070 con 0x7fa230101f20 2023-12-07T10:36:20.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.121+0000 7fa2277fe700 1 --2- 172.21.15.119:0/3919050984 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa21c0a1d00 0x7fa21c0a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:20.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.121+0000 7fa2277fe700 1 -- 172.21.15.119:0/3919050984 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9549+0+0 (secure 0 0 0) 0x7fa2200ce8a0 con 0x7fa230101f20 2023-12-07T10:36:20.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.122+0000 7fa23639b700 1 --2- 172.21.15.119:0/3919050984 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa21c0a1d00 0x7fa21c0a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:20.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.122+0000 7fa23639b700 1 --2- 172.21.15.119:0/3919050984 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa21c0a1d00 0x7fa21c0a41c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa22c005fd0 tx=0x7fa22c009310 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:20.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.123+0000 7fa2277fe700 1 -- 172.21.15.119:0/3919050984 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa22005c2f0 con 0x7fa230101f20 2023-12-07T10:36:20.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.334+0000 7fa2385ff700 1 -- 172.21.15.119:0/3919050984 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa23019afe0 con 0x7fa230101f20 2023-12-07T10:36:20.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.334+0000 7fa2277fe700 1 -- 172.21.15.119:0/3919050984 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v113) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa2200975b0 con 0x7fa230101f20 2023-12-07T10:36:20.334 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-12-07T10:36:20.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.336+0000 7fa2385ff700 1 -- 172.21.15.119:0/3919050984 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa21c0a1d00 msgr2=0x7fa21c0a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:20.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.336+0000 7fa2385ff700 1 --2- 172.21.15.119:0/3919050984 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa21c0a1d00 0x7fa21c0a41c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa22c005fd0 tx=0x7fa22c009310 comp rx=0 tx=0).stop 2023-12-07T10:36:20.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.336+0000 7fa2385ff700 1 -- 172.21.15.119:0/3919050984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa230101f20 msgr2=0x7fa23019ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:20.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.336+0000 7fa2385ff700 1 --2- 172.21.15.119:0/3919050984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa230101f20 0x7fa23019ff00 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa220000c00 tx=0x7fa22000fa00 comp rx=0 tx=0).stop 2023-12-07T10:36:20.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.337+0000 7fa2385ff700 1 -- 172.21.15.119:0/3919050984 shutdown_connections 2023-12-07T10:36:20.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.337+0000 7fa2385ff700 1 --2- 172.21.15.119:0/3919050984 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa21c0a1d00 0x7fa21c0a41c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:20.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.337+0000 7fa2385ff700 1 --2- 172.21.15.119:0/3919050984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa230107620 0x7fa230199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:20.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.337+0000 7fa2385ff700 1 --2- 172.21.15.119:0/3919050984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa230101560 0x7fa23019f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:20.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.337+0000 7fa2385ff700 1 --2- 172.21.15.119:0/3919050984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa230101f20 0x7fa23019ff00 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:20.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.337+0000 7fa2385ff700 1 -- 172.21.15.119:0/3919050984 >> 172.21.15.119:0/3919050984 conn(0x7fa2300fcfd0 msgr2=0x7fa2300fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:20.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.337+0000 7fa2385ff700 1 -- 172.21.15.119:0/3919050984 shutdown_connections 2023-12-07T10:36:20.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.337+0000 7fa2385ff700 1 -- 172.21.15.119:0/3919050984 wait complete. 2023-12-07T10:36:20.355 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-12-07T10:36:20.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:20 smithi119 ceph-mon[97908]: pgmap v499: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 5.3 MiB/s wr, 223 op/s 2023-12-07T10:36:20.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:20 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1000961030' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:36:20.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:20 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3142650525' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:36:20.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:20 smithi159 ceph-mon[134726]: pgmap v499: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 5.3 MiB/s wr, 223 op/s 2023-12-07T10:36:20.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:20 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1000961030' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:36:20.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:20 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3142650525' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:36:20.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:20 smithi153 ceph-mon[133625]: pgmap v499: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 5.3 MiB/s wr, 223 op/s 2023-12-07T10:36:20.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:20 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1000961030' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:36:20.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:20 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3142650525' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:36:20.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.554+0000 7f2a527ff700 1 -- 172.21.15.119:0/3883491115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a4c101ea0 msgr2=0x7f2a4c10e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:20.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.554+0000 7f2a527ff700 1 --2- 172.21.15.119:0/3883491115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a4c101ea0 0x7f2a4c10e360 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400b0d0 tx=0x7f2a4400b3e0 comp rx=0 tx=0).stop 2023-12-07T10:36:20.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.554+0000 7f2a527ff700 1 -- 172.21.15.119:0/3883491115 shutdown_connections 2023-12-07T10:36:20.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.554+0000 7f2a527ff700 1 --2- 172.21.15.119:0/3883491115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a4c101ea0 0x7f2a4c10e360 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:20.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.554+0000 7f2a527ff700 1 --2- 172.21.15.119:0/3883491115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a4c1075a0 0x7f2a4c107980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:20.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.554+0000 7f2a527ff700 1 --2- 172.21.15.119:0/3883491115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a4c1014e0 0x7f2a4c101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:20.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.554+0000 7f2a527ff700 1 -- 172.21.15.119:0/3883491115 >> 172.21.15.119:0/3883491115 conn(0x7f2a4c0fcff0 msgr2=0x7f2a4c0ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:20.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.554+0000 7f2a527ff700 1 -- 172.21.15.119:0/3883491115 shutdown_connections 2023-12-07T10:36:20.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.554+0000 7f2a527ff700 1 -- 172.21.15.119:0/3883491115 wait complete. 2023-12-07T10:36:20.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.555+0000 7f2a527ff700 1 Processor -- start 2023-12-07T10:36:20.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.555+0000 7f2a527ff700 1 -- start start 2023-12-07T10:36:20.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.556+0000 7f2a527ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a4c1014e0 0x7f2a4c19f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:20.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.556+0000 7f2a527ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a4c101ea0 0x7f2a4c19fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:20.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.556+0000 7f2a527ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a4c1075a0 0x7f2a4c199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:20.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.556+0000 7f2a527ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c19a130 con 0x7f2a4c101ea0 2023-12-07T10:36:20.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.556+0000 7f2a527ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c19a2a0 con 0x7f2a4c1075a0 2023-12-07T10:36:20.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.556+0000 7f2a527ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c19a460 con 0x7f2a4c1014e0 2023-12-07T10:36:20.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.556+0000 7f2a4b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a4c101ea0 0x7f2a4c19fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:20.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.556+0000 7f2a4b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a4c101ea0 0x7f2a4c19fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41864/0 (socket says 172.21.15.119:41864) 2023-12-07T10:36:20.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.556+0000 7f2a4b7fe700 1 -- 172.21.15.119:0/3607586120 learned_addr learned my addr 172.21.15.119:0/3607586120 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:36:20.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.556+0000 7f2a4bfff700 1 --2- 172.21.15.119:0/3607586120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a4c1014e0 0x7f2a4c19f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:20.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.556+0000 7f2a50d9c700 1 --2- 172.21.15.119:0/3607586120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a4c1075a0 0x7f2a4c199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:20.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.556+0000 7f2a4b7fe700 1 -- 172.21.15.119:0/3607586120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a4c1014e0 msgr2=0x7f2a4c19f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:20.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.556+0000 7f2a4b7fe700 1 --2- 172.21.15.119:0/3607586120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a4c1014e0 0x7f2a4c19f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:20.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.556+0000 7f2a4b7fe700 1 -- 172.21.15.119:0/3607586120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a4c1075a0 msgr2=0x7f2a4c199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:20.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.556+0000 7f2a4b7fe700 1 --2- 172.21.15.119:0/3607586120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a4c1075a0 0x7f2a4c199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:20.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.556+0000 7f2a4b7fe700 1 -- 172.21.15.119:0/3607586120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a44009d00 con 0x7f2a4c101ea0 2023-12-07T10:36:20.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.556+0000 7f2a50d9c700 1 --2- 172.21.15.119:0/3607586120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a4c1075a0 0x7f2a4c199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:36:20.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.556+0000 7f2a4bfff700 1 --2- 172.21.15.119:0/3607586120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a4c1014e0 0x7f2a4c19f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:36:20.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.557+0000 7f2a4b7fe700 1 --2- 172.21.15.119:0/3607586120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a4c101ea0 0x7f2a4c19fee0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2a4000e9a0 tx=0x7f2a4000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:20.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.557+0000 7f2a497fa700 1 -- 172.21.15.119:0/3607586120 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a40009790 con 0x7f2a4c101ea0 2023-12-07T10:36:20.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.557+0000 7f2a497fa700 1 -- 172.21.15.119:0/3607586120 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a4000cbd0 con 0x7f2a4c101ea0 2023-12-07T10:36:20.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.557+0000 7f2a497fa700 1 -- 172.21.15.119:0/3607586120 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a40018680 con 0x7f2a4c101ea0 2023-12-07T10:36:20.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.557+0000 7f2a527ff700 1 -- 172.21.15.119:0/3607586120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a4c19a740 con 0x7f2a4c101ea0 2023-12-07T10:36:20.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.557+0000 7f2a527ff700 1 -- 172.21.15.119:0/3607586120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a4c1a6990 con 0x7f2a4c101ea0 2023-12-07T10:36:20.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.558+0000 7f2a527ff700 1 -- 172.21.15.119:0/3607586120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a4c102c00 con 0x7f2a4c101ea0 2023-12-07T10:36:20.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.560+0000 7f2a497fa700 1 -- 172.21.15.119:0/3607586120 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f2a4000cd40 con 0x7f2a4c101ea0 2023-12-07T10:36:20.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.561+0000 7f2a497fa700 1 --2- 172.21.15.119:0/3607586120 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f2a340a1d00 0x7f2a340a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:20.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.561+0000 7f2a497fa700 1 -- 172.21.15.119:0/3607586120 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9549+0+0 (secure 0 0 0) 0x7f2a40014070 con 0x7f2a4c101ea0 2023-12-07T10:36:20.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.562+0000 7f2a4bfff700 1 --2- 172.21.15.119:0/3607586120 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f2a340a1d00 0x7f2a340a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:20.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.562+0000 7f2a4bfff700 1 --2- 172.21.15.119:0/3607586120 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f2a340a1d00 0x7f2a340a41c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c009cb0 tx=0x7f2a3c009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:20.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.565+0000 7f2a497fa700 1 -- 172.21.15.119:0/3607586120 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2a400522f0 con 0x7f2a4c101ea0 2023-12-07T10:36:20.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.825+0000 7f2a527ff700 1 -- 172.21.15.119:0/3607586120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f2a4c068350 con 0x7f2a4c101ea0 2023-12-07T10:36:20.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.836+0000 7f2a497fa700 1 -- 172.21.15.119:0/3607586120 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+736350 (secure 0 0 0) 0x7f2a4009a800 con 0x7f2a4c101ea0 2023-12-07T10:36:20.846 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:36:20.846 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:36:20.846 INFO:teuthology.orchestra.run.smithi119.stdout: "epoch": 26, 2023-12-07T10:36:20.846 INFO:teuthology.orchestra.run.smithi119.stdout: "active_gid": 15612, 2023-12-07T10:36:20.846 INFO:teuthology.orchestra.run.smithi119.stdout: "active_name": "x", 2023-12-07T10:36:20.846 INFO:teuthology.orchestra.run.smithi119.stdout: "active_addrs": { 2023-12-07T10:36:20.847 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2023-12-07T10:36:20.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:20.847 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2023-12-07T10:36:20.847 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6800", 2023-12-07T10:36:20.847 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 3763427845 2023-12-07T10:36:20.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.847 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:20.847 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2023-12-07T10:36:20.848 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:6801", 2023-12-07T10:36:20.848 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 3763427845 2023-12-07T10:36:20.848 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:20.848 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-12-07T10:36:20.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.848 INFO:teuthology.orchestra.run.smithi119.stdout: "active_addr": "172.21.15.119:6801/3763427845", 2023-12-07T10:36:20.848 INFO:teuthology.orchestra.run.smithi119.stdout: "active_change": "2023-12-07T10:20:24.856676+0000", 2023-12-07T10:36:20.848 INFO:teuthology.orchestra.run.smithi119.stdout: "active_mgr_features": 4540138322906710015, 2023-12-07T10:36:20.848 INFO:teuthology.orchestra.run.smithi119.stdout: "available": true, 2023-12-07T10:36:20.849 INFO:teuthology.orchestra.run.smithi119.stdout: "standbys": [ 2023-12-07T10:36:20.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:20.849 INFO:teuthology.orchestra.run.smithi119.stdout: "gid": 25606, 2023-12-07T10:36:20.849 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "z", 2023-12-07T10:36:20.849 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_features": 4540138322906710015, 2023-12-07T10:36:20.849 INFO:teuthology.orchestra.run.smithi119.stdout: "available_modules": [ 2023-12-07T10:36:20.849 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:20.849 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "alerts", 2023-12-07T10:36:20.849 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:20.850 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:20.850 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:20.850 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2023-12-07T10:36:20.850 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2023-12-07T10:36:20.850 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:20.850 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.850 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.850 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2023-12-07T10:36:20.851 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.851 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.851 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.851 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "How frequently to reexamine health status", 2023-12-07T10:36:20.851 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.851 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.851 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.851 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:20.852 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:20.852 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.852 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.852 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.852 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:20.852 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.852 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.852 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:20.853 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:20.853 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:20.853 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:20.853 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:20.853 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:20.853 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:20.853 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:20.853 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:20.854 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.854 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.854 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.854 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:20.854 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:20.854 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:20.854 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.854 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.855 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:20.855 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.855 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.855 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.855 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:20.855 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.855 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.855 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.856 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:20.856 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:20.856 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.856 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.856 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.856 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:20.856 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.856 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.857 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:20.857 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:20.857 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:20.858 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:20.858 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:20.858 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:20.858 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:20.858 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:20.858 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:20.858 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.859 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.859 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.859 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:20.859 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:20.859 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:20.859 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.859 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.859 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:20.860 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.860 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.860 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.860 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:20.860 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.860 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.860 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.860 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_destination": { 2023-12-07T10:36:20.861 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_destination", 2023-12-07T10:36:20.861 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.861 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.861 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.861 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:20.861 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.861 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.861 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.861 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Email address to send alerts to", 2023-12-07T10:36:20.862 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.862 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.862 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.862 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_from_name": { 2023-12-07T10:36:20.862 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_from_name", 2023-12-07T10:36:20.862 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.862 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.863 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.863 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "Ceph", 2023-12-07T10:36:20.863 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.863 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.863 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.863 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Email From: name", 2023-12-07T10:36:20.863 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.863 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.863 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.864 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_host": { 2023-12-07T10:36:20.864 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_host", 2023-12-07T10:36:20.864 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.864 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.864 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.864 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:20.864 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.864 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.865 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.865 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "SMTP server", 2023-12-07T10:36:20.865 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.865 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.865 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.865 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_password": { 2023-12-07T10:36:20.865 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_password", 2023-12-07T10:36:20.865 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.866 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.866 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.866 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:20.866 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.866 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.866 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.866 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Password to authenticate with", 2023-12-07T10:36:20.866 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.866 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.867 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.867 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_port": { 2023-12-07T10:36:20.867 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_port", 2023-12-07T10:36:20.867 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:20.867 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.867 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.867 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "465", 2023-12-07T10:36:20.868 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.868 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.868 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.868 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "SMTP port", 2023-12-07T10:36:20.868 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.868 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.868 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.868 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_sender": { 2023-12-07T10:36:20.869 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_sender", 2023-12-07T10:36:20.869 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.869 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.869 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.869 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:20.869 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.869 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.869 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.869 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "SMTP envelope sender", 2023-12-07T10:36:20.870 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.870 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.870 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.870 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.870 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_ssl": { 2023-12-07T10:36:20.870 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_ssl", 2023-12-07T10:36:20.870 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:20.871 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.871 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.871 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:20.871 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.871 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.871 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.871 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Use SSL to connect to SMTP server", 2023-12-07T10:36:20.871 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.871 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.871 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.872 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_user": { 2023-12-07T10:36:20.872 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_user", 2023-12-07T10:36:20.872 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.872 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.872 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.872 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:20.872 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.872 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.873 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.873 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "User to authenticate as", 2023-12-07T10:36:20.873 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.873 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.873 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.873 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:20.873 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:20.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.874 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:20.874 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "balancer", 2023-12-07T10:36:20.874 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:20.874 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:20.874 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:20.874 INFO:teuthology.orchestra.run.smithi119.stdout: "active": { 2023-12-07T10:36:20.874 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "active", 2023-12-07T10:36:20.874 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:20.875 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.875 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.875 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:20.875 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.875 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.875 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.875 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "automatically balance PGs across cluster", 2023-12-07T10:36:20.875 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.875 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.876 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.876 INFO:teuthology.orchestra.run.smithi119.stdout: "begin_time": { 2023-12-07T10:36:20.876 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "begin_time", 2023-12-07T10:36:20.876 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.876 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.876 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.876 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0000", 2023-12-07T10:36:20.876 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.877 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.877 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.877 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "beginning time of day to automatically balance", 2023-12-07T10:36:20.877 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-07T10:36:20.877 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.877 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.877 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.877 INFO:teuthology.orchestra.run.smithi119.stdout: "begin_weekday": { 2023-12-07T10:36:20.878 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "begin_weekday", 2023-12-07T10:36:20.878 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2023-12-07T10:36:20.878 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.878 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.878 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2023-12-07T10:36:20.878 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "0", 2023-12-07T10:36:20.878 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "6", 2023-12-07T10:36:20.878 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.878 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-12-07T10:36:20.879 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-07T10:36:20.879 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.879 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.879 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_compat_max_iterations": { 2023-12-07T10:36:20.879 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crush_compat_max_iterations", 2023-12-07T10:36:20.879 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2023-12-07T10:36:20.879 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.879 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.880 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "25", 2023-12-07T10:36:20.880 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1", 2023-12-07T10:36:20.880 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "250", 2023-12-07T10:36:20.880 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.880 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-12-07T10:36:20.880 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.880 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.880 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.881 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_compat_metrics": { 2023-12-07T10:36:20.881 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crush_compat_metrics", 2023-12-07T10:36:20.881 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.881 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.881 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.881 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "pgs,objects,bytes", 2023-12-07T10:36:20.881 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.881 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.881 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.882 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-12-07T10:36:20.882 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-12-07T10:36:20.882 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.882 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.882 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_compat_step": { 2023-12-07T10:36:20.882 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crush_compat_step", 2023-12-07T10:36:20.882 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2023-12-07T10:36:20.882 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.883 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.883 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0.5", 2023-12-07T10:36:20.883 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "0.001", 2023-12-07T10:36:20.883 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "0.999", 2023-12-07T10:36:20.883 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.883 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "aggressiveness of optimization", 2023-12-07T10:36:20.883 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-12-07T10:36:20.883 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.884 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.884 INFO:teuthology.orchestra.run.smithi119.stdout: "end_time": { 2023-12-07T10:36:20.884 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "end_time", 2023-12-07T10:36:20.884 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.884 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.884 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.884 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "2359", 2023-12-07T10:36:20.884 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.885 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.885 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.885 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "ending time of day to automatically balance", 2023-12-07T10:36:20.885 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-07T10:36:20.885 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.885 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.885 INFO:teuthology.orchestra.run.smithi119.stdout: "end_weekday": { 2023-12-07T10:36:20.885 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "end_weekday", 2023-12-07T10:36:20.886 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2023-12-07T10:36:20.886 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.886 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.886 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2023-12-07T10:36:20.886 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "0", 2023-12-07T10:36:20.886 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "6", 2023-12-07T10:36:20.886 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.886 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-12-07T10:36:20.886 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-07T10:36:20.887 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.887 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.887 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:20.887 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:20.887 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.887 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.887 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.887 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:20.887 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.888 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.888 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:20.888 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:20.888 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:20.888 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:20.888 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:20.888 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:20.888 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:20.888 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:20.889 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:20.889 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.889 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.889 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.889 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:20.889 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:20.889 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:20.889 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.890 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.890 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:20.890 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.890 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.890 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.890 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:20.890 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.891 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.891 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.891 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.891 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:20.891 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:20.891 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.891 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.891 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.891 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:20.892 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.892 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.892 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:20.892 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:20.892 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:20.892 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:20.892 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:20.892 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:20.893 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:20.893 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:20.893 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:20.893 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.893 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.893 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.893 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.893 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:20.893 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:20.894 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:20.894 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.894 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.894 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:20.894 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.894 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.894 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.894 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:20.894 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.895 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.895 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.895 INFO:teuthology.orchestra.run.smithi119.stdout: "min_score": { 2023-12-07T10:36:20.895 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "min_score", 2023-12-07T10:36:20.895 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2023-12-07T10:36:20.895 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.895 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.895 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2023-12-07T10:36:20.895 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-12-07T10:36:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": { 2023-12-07T10:36:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mode", 2023-12-07T10:36:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "upmap", 2023-12-07T10:36:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:20.897 INFO:teuthology.orchestra.run.smithi119.stdout: "crush-compat", 2023-12-07T10:36:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: "none", 2023-12-07T10:36:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: "upmap" 2023-12-07T10:36:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Balancer mode", 2023-12-07T10:36:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.898 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_ids": { 2023-12-07T10:36:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "pool_ids", 2023-12-07T10:36:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.899 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-12-07T10:36:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2023-12-07T10:36:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2023-12-07T10:36:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.900 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2023-12-07T10:36:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-12-07T10:36:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.901 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: "upmap_max_deviation": { 2023-12-07T10:36:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "upmap_max_deviation", 2023-12-07T10:36:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5", 2023-12-07T10:36:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1", 2023-12-07T10:36:20.902 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "deviation below which no optimization is attempted", 2023-12-07T10:36:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-12-07T10:36:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: "upmap_max_optimizations": { 2023-12-07T10:36:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "upmap_max_optimizations", 2023-12-07T10:36:20.903 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2023-12-07T10:36:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2023-12-07T10:36:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-12-07T10:36:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.904 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cephadm", 2023-12-07T10:36:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:20.905 INFO:teuthology.orchestra.run.smithi119.stdout: "agent_down_multiplier": { 2023-12-07T10:36:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "agent_down_multiplier", 2023-12-07T10:36:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2023-12-07T10:36:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "3.0", 2023-12-07T10:36:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.906 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-12-07T10:36:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: "agent_refresh_rate": { 2023-12-07T10:36:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "agent_refresh_rate", 2023-12-07T10:36:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.907 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "20", 2023-12-07T10:36:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-12-07T10:36:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: "agent_starting_port": { 2023-12-07T10:36:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "agent_starting_port", 2023-12-07T10:36:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "4721", 2023-12-07T10:36:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.909 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-12-07T10:36:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "allow_ptrace": { 2023-12-07T10:36:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "allow_ptrace", 2023-12-07T10:36:20.910 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-12-07T10:36:20.911 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-12-07T10:36:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: "autotune_interval": { 2023-12-07T10:36:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "autotune_interval", 2023-12-07T10:36:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2023-12-07T10:36:20.912 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.913 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.913 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.913 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to autotune daemon memory", 2023-12-07T10:36:20.913 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.913 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.913 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.913 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.913 INFO:teuthology.orchestra.run.smithi119.stdout: "autotune_memory_target_ratio": { 2023-12-07T10:36:20.914 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "autotune_memory_target_ratio", 2023-12-07T10:36:20.914 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2023-12-07T10:36:20.914 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.914 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.914 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0.7", 2023-12-07T10:36:20.914 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.914 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.914 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.915 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-12-07T10:36:20.915 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.915 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.915 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.915 INFO:teuthology.orchestra.run.smithi119.stdout: "cephadm_log_destination": { 2023-12-07T10:36:20.915 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cephadm_log_destination", 2023-12-07T10:36:20.915 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.915 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.915 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.916 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:20.916 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.916 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.916 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:20.916 INFO:teuthology.orchestra.run.smithi119.stdout: "file", 2023-12-07T10:36:20.916 INFO:teuthology.orchestra.run.smithi119.stdout: "file,syslog", 2023-12-07T10:36:20.916 INFO:teuthology.orchestra.run.smithi119.stdout: "syslog" 2023-12-07T10:36:20.916 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:20.917 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-12-07T10:36:20.917 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.917 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.917 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.917 INFO:teuthology.orchestra.run.smithi119.stdout: "cgroups_split": { 2023-12-07T10:36:20.917 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cgroups_split", 2023-12-07T10:36:20.917 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:20.917 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.917 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.918 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:20.918 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.918 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.918 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.918 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-12-07T10:36:20.918 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.918 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.918 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.918 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.919 INFO:teuthology.orchestra.run.smithi119.stdout: "config_checks_enabled": { 2023-12-07T10:36:20.919 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "config_checks_enabled", 2023-12-07T10:36:20.919 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:20.919 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.919 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.919 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:20.919 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.919 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.919 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.920 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-12-07T10:36:20.920 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.920 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.920 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.920 INFO:teuthology.orchestra.run.smithi119.stdout: "config_dashboard": { 2023-12-07T10:36:20.920 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "config_dashboard", 2023-12-07T10:36:20.920 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:20.920 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.921 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.921 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:20.921 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.921 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.921 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.921 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-12-07T10:36:20.921 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.921 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.922 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.922 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_alertmanager": { 2023-12-07T10:36:20.922 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_alertmanager", 2023-12-07T10:36:20.922 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.922 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.922 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.922 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-12-07T10:36:20.922 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.923 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.923 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.923 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Prometheus container image", 2023-12-07T10:36:20.923 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.923 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.923 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.923 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.923 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_base": { 2023-12-07T10:36:20.923 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_base", 2023-12-07T10:36:20.924 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.924 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.924 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.924 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/ceph", 2023-12-07T10:36:20.924 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.924 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.924 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.924 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Container image name, without the tag", 2023-12-07T10:36:20.924 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.925 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.925 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.925 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_elasticsearch": { 2023-12-07T10:36:20.925 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_elasticsearch", 2023-12-07T10:36:20.925 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.925 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.925 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.925 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-12-07T10:36:20.926 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.926 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.926 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.926 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "elasticsearch container image", 2023-12-07T10:36:20.926 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.926 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.926 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.926 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.927 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_grafana": { 2023-12-07T10:36:20.927 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_grafana", 2023-12-07T10:36:20.927 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.927 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.927 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.927 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-12-07T10:36:20.927 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.927 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.927 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.928 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Prometheus container image", 2023-12-07T10:36:20.928 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.928 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.928 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.928 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_haproxy": { 2023-12-07T10:36:20.928 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_haproxy", 2023-12-07T10:36:20.928 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.928 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.928 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.929 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-12-07T10:36:20.929 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.929 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.929 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.929 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "HAproxy container image", 2023-12-07T10:36:20.929 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.929 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.929 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.929 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.930 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_jaeger_agent": { 2023-12-07T10:36:20.930 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_jaeger_agent", 2023-12-07T10:36:20.930 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.930 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.930 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.930 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-12-07T10:36:20.930 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.930 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.930 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.931 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Jaeger agent container image", 2023-12-07T10:36:20.931 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.931 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.931 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.931 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_jaeger_collector": { 2023-12-07T10:36:20.931 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_jaeger_collector", 2023-12-07T10:36:20.931 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.932 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.932 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.932 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-12-07T10:36:20.932 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.932 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.932 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.932 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Jaeger collector container image", 2023-12-07T10:36:20.932 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.932 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.933 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.933 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.933 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_jaeger_query": { 2023-12-07T10:36:20.933 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_jaeger_query", 2023-12-07T10:36:20.933 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.933 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.933 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.933 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-12-07T10:36:20.933 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.934 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.934 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.934 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Jaeger query container image", 2023-12-07T10:36:20.934 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.934 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.934 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.934 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_keepalived": { 2023-12-07T10:36:20.934 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_keepalived", 2023-12-07T10:36:20.935 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.935 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.935 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.935 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-12-07T10:36:20.935 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.935 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.935 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.935 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Keepalived container image", 2023-12-07T10:36:20.935 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.936 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.936 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.936 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.936 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_loki": { 2023-12-07T10:36:20.936 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_loki", 2023-12-07T10:36:20.936 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.936 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.936 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.936 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-12-07T10:36:20.937 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.937 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.937 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.937 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Loki container image", 2023-12-07T10:36:20.937 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.937 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.937 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.937 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.937 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_node_exporter": { 2023-12-07T10:36:20.938 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_node_exporter", 2023-12-07T10:36:20.938 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.938 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.938 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.938 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-12-07T10:36:20.938 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.938 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.938 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.938 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Prometheus container image", 2023-12-07T10:36:20.939 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.939 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.939 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.939 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_nvmeof": { 2023-12-07T10:36:20.939 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_nvmeof", 2023-12-07T10:36:20.939 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.939 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.939 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.940 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2023-12-07T10:36:20.940 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.940 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.940 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.940 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Nvme-of container image", 2023-12-07T10:36:20.940 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.940 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.940 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.941 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_prometheus": { 2023-12-07T10:36:20.941 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_prometheus", 2023-12-07T10:36:20.941 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.941 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.941 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.941 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-12-07T10:36:20.941 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.941 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.941 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.942 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Prometheus container image", 2023-12-07T10:36:20.942 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.942 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.942 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.942 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_promtail": { 2023-12-07T10:36:20.942 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_promtail", 2023-12-07T10:36:20.942 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.943 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.943 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.943 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-12-07T10:36:20.943 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.943 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.943 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.943 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Promtail container image", 2023-12-07T10:36:20.943 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.943 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.944 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.944 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_snmp_gateway": { 2023-12-07T10:36:20.944 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_snmp_gateway", 2023-12-07T10:36:20.944 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.944 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.944 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.944 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-12-07T10:36:20.944 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.945 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.945 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.945 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "SNMP Gateway container image", 2023-12-07T10:36:20.945 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.945 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.945 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.945 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.945 INFO:teuthology.orchestra.run.smithi119.stdout: "container_init": { 2023-12-07T10:36:20.945 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_init", 2023-12-07T10:36:20.946 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:20.946 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.946 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.946 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:20.946 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.946 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.946 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.946 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Run podman/docker with `--init`", 2023-12-07T10:36:20.946 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.947 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.947 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.947 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.947 INFO:teuthology.orchestra.run.smithi119.stdout: "daemon_cache_timeout": { 2023-12-07T10:36:20.947 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "daemon_cache_timeout", 2023-12-07T10:36:20.947 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:20.947 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.947 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.947 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2023-12-07T10:36:20.948 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.948 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.948 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.948 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-12-07T10:36:20.948 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.948 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.948 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.948 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.948 INFO:teuthology.orchestra.run.smithi119.stdout: "default_cephadm_command_timeout": { 2023-12-07T10:36:20.949 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "default_cephadm_command_timeout", 2023-12-07T10:36:20.949 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:20.949 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.949 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.949 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "900", 2023-12-07T10:36:20.949 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.949 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.949 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.949 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-12-07T10:36:20.950 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.950 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.950 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.950 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.950 INFO:teuthology.orchestra.run.smithi119.stdout: "default_registry": { 2023-12-07T10:36:20.950 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "default_registry", 2023-12-07T10:36:20.950 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.950 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.950 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.951 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "docker.io", 2023-12-07T10:36:20.951 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.951 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.951 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.951 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-12-07T10:36:20.951 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.951 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.951 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.952 INFO:teuthology.orchestra.run.smithi119.stdout: "device_cache_timeout": { 2023-12-07T10:36:20.952 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "device_cache_timeout", 2023-12-07T10:36:20.952 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:20.952 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.952 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.952 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "1800", 2023-12-07T10:36:20.952 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.952 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.953 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.953 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "seconds to cache device inventory", 2023-12-07T10:36:20.953 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.953 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.953 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.953 INFO:teuthology.orchestra.run.smithi119.stdout: "device_enhanced_scan": { 2023-12-07T10:36:20.953 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "device_enhanced_scan", 2023-12-07T10:36:20.953 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:20.953 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.954 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.954 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:20.954 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.954 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.954 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.954 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Use libstoragemgmt during device scans", 2023-12-07T10:36:20.954 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.954 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.955 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.955 INFO:teuthology.orchestra.run.smithi119.stdout: "facts_cache_timeout": { 2023-12-07T10:36:20.955 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "facts_cache_timeout", 2023-12-07T10:36:20.955 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:20.955 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.955 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.955 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2023-12-07T10:36:20.955 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.955 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.956 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.956 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "seconds to cache host facts data", 2023-12-07T10:36:20.956 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.956 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.956 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.956 INFO:teuthology.orchestra.run.smithi119.stdout: "host_check_interval": { 2023-12-07T10:36:20.956 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "host_check_interval", 2023-12-07T10:36:20.956 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:20.957 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.957 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.957 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2023-12-07T10:36:20.957 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.957 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.957 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.957 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to perform a host check", 2023-12-07T10:36:20.957 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.957 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.958 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.958 INFO:teuthology.orchestra.run.smithi119.stdout: "inventory_list_all": { 2023-12-07T10:36:20.958 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "inventory_list_all", 2023-12-07T10:36:20.958 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:20.958 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.958 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.958 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:20.958 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.959 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.959 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.959 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-12-07T10:36:20.959 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.959 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.959 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.959 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:20.959 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:20.959 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.960 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.960 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.960 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:20.960 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.960 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.960 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:20.960 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:20.960 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:20.960 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:20.961 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:20.961 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:20.961 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:20.961 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:20.961 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:20.961 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.961 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.961 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.961 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.962 INFO:teuthology.orchestra.run.smithi119.stdout: "log_refresh_metadata": { 2023-12-07T10:36:20.962 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_refresh_metadata", 2023-12-07T10:36:20.962 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:20.962 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.962 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.962 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:20.962 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.962 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.963 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.963 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-12-07T10:36:20.963 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.963 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.963 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.963 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:20.963 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:20.963 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:20.964 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.964 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.964 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:20.964 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.964 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.964 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.964 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-12-07T10:36:20.964 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.964 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.965 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.965 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:20.965 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:20.965 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.965 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.965 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.965 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:20.965 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.966 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.966 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:20.966 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:20.966 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:20.966 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:20.966 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:20.966 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:20.966 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:20.966 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:20.967 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:20.967 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.967 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.967 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.967 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:20.967 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:20.967 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:20.967 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.968 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:20.968 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:20.968 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.968 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.968 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.968 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:20.968 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.968 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.968 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.969 INFO:teuthology.orchestra.run.smithi119.stdout: "manage_etc_ceph_ceph_conf": { 2023-12-07T10:36:20.969 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-12-07T10:36:20.969 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:20.969 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.969 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.969 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:20.969 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.969 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.970 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.970 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-12-07T10:36:20.970 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.970 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.970 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.970 INFO:teuthology.orchestra.run.smithi119.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-12-07T10:36:20.970 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-12-07T10:36:20.970 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.971 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.971 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.971 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "*", 2023-12-07T10:36:20.971 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.971 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.971 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.971 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-12-07T10:36:20.971 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.971 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.972 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.972 INFO:teuthology.orchestra.run.smithi119.stdout: "max_count_per_host": { 2023-12-07T10:36:20.972 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_count_per_host", 2023-12-07T10:36:20.972 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:20.972 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.972 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.972 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2023-12-07T10:36:20.972 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.973 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.973 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.973 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "max number of daemons per service per host", 2023-12-07T10:36:20.973 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.973 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.973 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.973 INFO:teuthology.orchestra.run.smithi119.stdout: "max_osd_draining_count": { 2023-12-07T10:36:20.973 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_osd_draining_count", 2023-12-07T10:36:20.974 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:20.974 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.974 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.974 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2023-12-07T10:36:20.974 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.974 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.974 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.974 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-12-07T10:36:20.975 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.975 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.975 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.975 INFO:teuthology.orchestra.run.smithi119.stdout: "migration_current": { 2023-12-07T10:36:20.975 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "migration_current", 2023-12-07T10:36:20.975 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:20.975 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.975 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.976 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:20.976 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.976 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.976 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.976 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "internal - do not modify", 2023-12-07T10:36:20.976 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.976 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.976 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.977 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": { 2023-12-07T10:36:20.977 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mode", 2023-12-07T10:36:20.977 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.977 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.977 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.977 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "root", 2023-12-07T10:36:20.977 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.977 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.977 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:20.978 INFO:teuthology.orchestra.run.smithi119.stdout: "cephadm-package", 2023-12-07T10:36:20.978 INFO:teuthology.orchestra.run.smithi119.stdout: "root" 2023-12-07T10:36:20.978 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:20.978 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "mode for remote execution of cephadm", 2023-12-07T10:36:20.978 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.978 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.978 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.978 INFO:teuthology.orchestra.run.smithi119.stdout: "prometheus_alerts_path": { 2023-12-07T10:36:20.979 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "prometheus_alerts_path", 2023-12-07T10:36:20.979 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.979 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.979 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.979 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-12-07T10:36:20.979 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.979 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.979 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.979 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-12-07T10:36:20.980 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.980 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.980 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.980 INFO:teuthology.orchestra.run.smithi119.stdout: "registry_insecure": { 2023-12-07T10:36:20.980 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "registry_insecure", 2023-12-07T10:36:20.980 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:20.980 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.980 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.980 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:20.981 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.981 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.981 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.981 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-12-07T10:36:20.981 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.981 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.981 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.981 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.981 INFO:teuthology.orchestra.run.smithi119.stdout: "registry_password": { 2023-12-07T10:36:20.982 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "registry_password", 2023-12-07T10:36:20.982 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.982 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.982 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.982 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:20.982 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.982 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.982 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.982 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-12-07T10:36:20.983 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.983 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.983 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.983 INFO:teuthology.orchestra.run.smithi119.stdout: "registry_url": { 2023-12-07T10:36:20.983 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "registry_url", 2023-12-07T10:36:20.983 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.983 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.984 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.984 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:20.984 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.984 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.984 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.984 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-12-07T10:36:20.984 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.984 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.984 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.985 INFO:teuthology.orchestra.run.smithi119.stdout: "registry_username": { 2023-12-07T10:36:20.985 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "registry_username", 2023-12-07T10:36:20.985 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.985 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.985 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.985 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:20.985 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.985 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.985 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.986 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-12-07T10:36:20.986 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.986 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.986 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.986 INFO:teuthology.orchestra.run.smithi119.stdout: "secure_monitoring_stack": { 2023-12-07T10:36:20.986 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "secure_monitoring_stack", 2023-12-07T10:36:20.986 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:20.987 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.987 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.987 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:20.987 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.987 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.987 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.987 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-12-07T10:36:20.987 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.988 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.988 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.988 INFO:teuthology.orchestra.run.smithi119.stdout: "service_discovery_port": { 2023-12-07T10:36:20.988 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "service_discovery_port", 2023-12-07T10:36:20.988 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:20.988 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.988 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.988 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8765", 2023-12-07T10:36:20.988 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.989 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.989 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.989 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "cephadm service discovery port", 2023-12-07T10:36:20.989 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.989 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.989 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.989 INFO:teuthology.orchestra.run.smithi119.stdout: "ssh_config_file": { 2023-12-07T10:36:20.989 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ssh_config_file", 2023-12-07T10:36:20.990 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:20.990 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.990 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.990 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:20.990 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.990 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.990 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.990 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-12-07T10:36:20.990 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.991 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.991 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.991 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.991 INFO:teuthology.orchestra.run.smithi119.stdout: "use_agent": { 2023-12-07T10:36:20.991 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "use_agent", 2023-12-07T10:36:20.991 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:20.991 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.991 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.992 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:20.992 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.992 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.992 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.992 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-12-07T10:36:20.992 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.992 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.992 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.993 INFO:teuthology.orchestra.run.smithi119.stdout: "use_repo_digest": { 2023-12-07T10:36:20.993 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "use_repo_digest", 2023-12-07T10:36:20.993 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:20.993 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.993 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.993 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:20.993 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.993 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.993 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.994 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-12-07T10:36:20.994 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.994 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.994 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.994 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_on_failed_host_check": { 2023-12-07T10:36:20.994 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_on_failed_host_check", 2023-12-07T10:36:20.994 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:20.995 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.995 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.995 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:20.995 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.995 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.995 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.995 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "raise a health warning if the host check fails", 2023-12-07T10:36:20.995 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.995 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.995 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.996 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.996 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_on_stray_daemons": { 2023-12-07T10:36:20.996 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_on_stray_daemons", 2023-12-07T10:36:20.996 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:20.996 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.996 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.996 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:20.996 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.996 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.997 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.997 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-12-07T10:36:20.997 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.997 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.997 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.997 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_on_stray_hosts": { 2023-12-07T10:36:20.997 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_on_stray_hosts", 2023-12-07T10:36:20.998 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:20.998 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:20.998 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:20.998 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:20.998 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:20.998 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:20.998 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:20.998 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-12-07T10:36:20.999 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:20.999 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:20.999 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:20.999 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:20.999 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:20.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:20.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:20.999 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crash", 2023-12-07T10:36:20.999 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.000 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.000 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.000 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.000 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.000 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.000 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.000 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.000 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.000 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.001 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.001 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.001 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.001 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.001 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.001 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.001 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.001 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.001 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.001 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.002 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.002 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.002 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.002 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.002 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.002 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.002 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.003 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.003 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.003 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.003 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.003 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.003 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.003 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.003 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.003 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.004 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.004 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.004 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.004 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.004 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.004 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.004 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.004 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.004 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.005 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.005 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.005 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.005 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.005 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.005 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.005 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.005 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.006 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.006 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.006 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.006 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.006 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.006 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.006 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.006 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.007 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.007 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.007 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.007 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.007 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.007 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.007 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.007 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.008 INFO:teuthology.orchestra.run.smithi119.stdout: "retain_interval": { 2023-12-07T10:36:21.008 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "retain_interval", 2023-12-07T10:36:21.008 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.008 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.008 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.008 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "31536000", 2023-12-07T10:36:21.008 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.008 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.008 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.009 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how long to retain crashes before pruning them", 2023-12-07T10:36:21.009 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.009 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.009 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.009 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_recent_interval": { 2023-12-07T10:36:21.009 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_recent_interval", 2023-12-07T10:36:21.009 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.009 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.009 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.010 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "1209600", 2023-12-07T10:36:21.010 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.010 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.010 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.010 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "time interval in which to warn about recent crashes", 2023-12-07T10:36:21.010 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.010 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.010 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.011 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.011 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.011 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "dashboard", 2023-12-07T10:36:21.011 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.011 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.011 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.011 INFO:teuthology.orchestra.run.smithi119.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-12-07T10:36:21.012 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-12-07T10:36:21.012 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.012 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.012 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.012 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2023-12-07T10:36:21.012 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.012 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.012 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.012 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.013 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.013 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.013 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.013 INFO:teuthology.orchestra.run.smithi119.stdout: "ALERTMANAGER_API_HOST": { 2023-12-07T10:36:21.013 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ALERTMANAGER_API_HOST", 2023-12-07T10:36:21.013 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.013 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.013 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.014 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.014 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.014 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.014 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.014 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.014 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.014 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.014 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.015 INFO:teuthology.orchestra.run.smithi119.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-12-07T10:36:21.015 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-12-07T10:36:21.015 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.015 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.015 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.015 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.015 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.015 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.016 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.016 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.016 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.016 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.016 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.016 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.016 INFO:teuthology.orchestra.run.smithi119.stdout: "AUDIT_API_ENABLED": { 2023-12-07T10:36:21.016 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "AUDIT_API_ENABLED", 2023-12-07T10:36:21.016 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.017 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.017 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.017 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.017 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.017 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.017 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.017 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.017 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.017 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.018 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.018 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.018 INFO:teuthology.orchestra.run.smithi119.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-12-07T10:36:21.018 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-12-07T10:36:21.018 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.018 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.018 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.018 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.018 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.019 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.019 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.019 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.019 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.019 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.019 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.019 INFO:teuthology.orchestra.run.smithi119.stdout: "ENABLE_BROWSABLE_API": { 2023-12-07T10:36:21.019 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ENABLE_BROWSABLE_API", 2023-12-07T10:36:21.020 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.020 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.020 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.020 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.020 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.020 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.020 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.020 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.020 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.021 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.021 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.021 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.021 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-12-07T10:36:21.021 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-12-07T10:36:21.021 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.021 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.021 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.021 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.022 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.022 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.022 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.022 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.022 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.022 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.022 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.022 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.022 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-12-07T10:36:21.023 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-12-07T10:36:21.023 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.023 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.023 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.023 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.023 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.023 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.023 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.023 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.024 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.024 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.024 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.024 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-12-07T10:36:21.024 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-12-07T10:36:21.024 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.024 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.024 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.025 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.025 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.025 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.025 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.025 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.025 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.025 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.025 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.025 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.026 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-12-07T10:36:21.026 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-12-07T10:36:21.026 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.026 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.026 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.026 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.026 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.026 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.027 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.027 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.027 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.027 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.027 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.027 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_NFS": { 2023-12-07T10:36:21.027 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-12-07T10:36:21.027 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.028 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.028 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.028 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.028 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.028 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.028 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.028 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.028 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.029 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.029 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.029 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.029 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_RBD": { 2023-12-07T10:36:21.029 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-12-07T10:36:21.029 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.029 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.029 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.029 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.030 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.030 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.030 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.030 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.030 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.030 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.030 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.030 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.031 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_RGW": { 2023-12-07T10:36:21.031 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-12-07T10:36:21.031 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.031 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.031 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.031 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.031 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.031 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.031 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.032 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.032 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.032 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.032 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.032 INFO:teuthology.orchestra.run.smithi119.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-12-07T10:36:21.032 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-12-07T10:36:21.032 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.032 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.032 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.033 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.033 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.033 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.033 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.033 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.033 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.033 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.033 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.033 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.034 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_API_PASSWORD": { 2023-12-07T10:36:21.034 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_API_PASSWORD", 2023-12-07T10:36:21.034 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.034 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.034 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.034 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "admin", 2023-12-07T10:36:21.034 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.034 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.035 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.035 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.035 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.035 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.035 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.035 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.035 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-12-07T10:36:21.035 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-12-07T10:36:21.035 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.036 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.036 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.036 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.036 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.036 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.036 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.036 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.036 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.036 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.037 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.037 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.037 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_API_URL": { 2023-12-07T10:36:21.037 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_API_URL", 2023-12-07T10:36:21.037 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.037 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.037 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.037 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.037 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.038 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.038 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.038 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.038 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.038 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.038 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.038 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.038 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_API_USERNAME": { 2023-12-07T10:36:21.039 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_API_USERNAME", 2023-12-07T10:36:21.039 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.039 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.039 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.039 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "admin", 2023-12-07T10:36:21.039 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.039 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.039 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.039 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.040 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.040 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.040 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.040 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-12-07T10:36:21.040 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-12-07T10:36:21.040 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.040 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.040 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.041 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.041 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.041 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.041 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.041 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.041 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.041 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.041 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.041 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.042 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-12-07T10:36:21.042 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-12-07T10:36:21.042 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.042 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.042 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.042 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.042 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.042 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.043 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.043 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.043 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.043 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.043 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.043 INFO:teuthology.orchestra.run.smithi119.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-12-07T10:36:21.043 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-12-07T10:36:21.043 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.044 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.044 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.044 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.044 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.044 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.044 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.044 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.044 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.044 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.045 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.045 INFO:teuthology.orchestra.run.smithi119.stdout: "ISSUE_TRACKER_API_KEY": { 2023-12-07T10:36:21.045 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-12-07T10:36:21.045 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.045 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.045 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.045 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.045 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.046 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.046 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.046 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.046 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.046 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.046 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.046 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.046 INFO:teuthology.orchestra.run.smithi119.stdout: "PROMETHEUS_API_HOST": { 2023-12-07T10:36:21.046 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PROMETHEUS_API_HOST", 2023-12-07T10:36:21.047 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.047 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.047 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.047 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.047 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.047 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.047 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.047 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.048 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.048 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.048 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.048 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.048 INFO:teuthology.orchestra.run.smithi119.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-12-07T10:36:21.048 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-12-07T10:36:21.048 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.048 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.048 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.049 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.049 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.049 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.049 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.049 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.049 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.049 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.050 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.050 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.050 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-12-07T10:36:21.050 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-12-07T10:36:21.050 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.050 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.050 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.050 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.050 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.051 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.051 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.051 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.051 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.051 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.051 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.051 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-12-07T10:36:21.052 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-12-07T10:36:21.052 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.052 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.052 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.052 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.052 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.052 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.052 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.052 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.053 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.053 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.053 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.053 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-12-07T10:36:21.053 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-12-07T10:36:21.053 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.053 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.053 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.054 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.054 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.054 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.054 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.054 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.054 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.054 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.054 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.055 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-12-07T10:36:21.055 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-12-07T10:36:21.055 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.055 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.055 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.055 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.055 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.055 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.056 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.056 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.056 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.056 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.056 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.056 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-12-07T10:36:21.056 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-12-07T10:36:21.056 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.057 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.057 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.057 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.057 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.057 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.057 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.057 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.057 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.057 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.058 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.058 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.058 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-12-07T10:36:21.058 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-12-07T10:36:21.058 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.058 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.058 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.058 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.058 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.059 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.059 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.059 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.059 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.059 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.059 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.060 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-12-07T10:36:21.060 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-12-07T10:36:21.060 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.060 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.060 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.060 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.060 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.060 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.060 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.061 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.061 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.061 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.061 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.061 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_ENABLED": { 2023-12-07T10:36:21.061 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_ENABLED", 2023-12-07T10:36:21.061 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.061 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.062 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.062 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.062 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.062 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.062 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.062 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.062 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.062 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.062 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.063 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-12-07T10:36:21.063 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-12-07T10:36:21.063 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.063 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.063 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.063 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-12-07T10:36:21.063 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.063 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.064 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.064 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.064 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.064 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.064 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.064 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.064 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-12-07T10:36:21.064 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-12-07T10:36:21.064 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.065 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.065 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.065 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2023-12-07T10:36:21.065 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.065 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.065 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.065 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.065 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.065 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.066 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.066 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.066 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-12-07T10:36:21.066 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-12-07T10:36:21.066 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.066 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.066 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.066 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8", 2023-12-07T10:36:21.066 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.067 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.067 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.067 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.067 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.067 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.067 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.067 INFO:teuthology.orchestra.run.smithi119.stdout: "REST_REQUESTS_TIMEOUT": { 2023-12-07T10:36:21.067 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-12-07T10:36:21.068 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.068 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.068 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.068 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "45", 2023-12-07T10:36:21.068 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.068 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.068 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.068 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.068 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.069 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.069 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.069 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.069 INFO:teuthology.orchestra.run.smithi119.stdout: "RGW_API_ACCESS_KEY": { 2023-12-07T10:36:21.069 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "RGW_API_ACCESS_KEY", 2023-12-07T10:36:21.069 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.069 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.069 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.069 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.070 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.070 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.070 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.070 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.070 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.070 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.070 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.071 INFO:teuthology.orchestra.run.smithi119.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-12-07T10:36:21.071 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-12-07T10:36:21.071 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.071 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.071 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.071 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "admin", 2023-12-07T10:36:21.071 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.071 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.071 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.072 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.072 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.072 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.072 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.072 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.072 INFO:teuthology.orchestra.run.smithi119.stdout: "RGW_API_SECRET_KEY": { 2023-12-07T10:36:21.072 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "RGW_API_SECRET_KEY", 2023-12-07T10:36:21.072 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.072 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.073 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.073 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.073 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.073 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.073 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.073 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.073 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.073 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.073 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.074 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.074 INFO:teuthology.orchestra.run.smithi119.stdout: "RGW_API_SSL_VERIFY": { 2023-12-07T10:36:21.074 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "RGW_API_SSL_VERIFY", 2023-12-07T10:36:21.074 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.074 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.074 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.074 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.074 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.074 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.075 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.075 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.075 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.075 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.075 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.075 INFO:teuthology.orchestra.run.smithi119.stdout: "UNSAFE_TLS_v1_2": { 2023-12-07T10:36:21.075 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "UNSAFE_TLS_v1_2", 2023-12-07T10:36:21.075 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.076 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.076 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.076 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.076 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.076 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.076 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.076 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.076 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.076 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.077 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.077 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.077 INFO:teuthology.orchestra.run.smithi119.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-12-07T10:36:21.077 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-12-07T10:36:21.077 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.077 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.077 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.077 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2023-12-07T10:36:21.077 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.078 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.078 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.078 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.078 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.078 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.078 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.078 INFO:teuthology.orchestra.run.smithi119.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-12-07T10:36:21.079 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-12-07T10:36:21.079 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.079 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.079 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.079 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2023-12-07T10:36:21.079 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.079 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.079 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.079 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.080 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.080 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.080 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.080 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.080 INFO:teuthology.orchestra.run.smithi119.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-12-07T10:36:21.080 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-12-07T10:36:21.080 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.080 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.081 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.081 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5", 2023-12-07T10:36:21.081 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.081 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.081 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.081 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.081 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.081 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.081 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.082 INFO:teuthology.orchestra.run.smithi119.stdout: "cross_origin_url": { 2023-12-07T10:36:21.082 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cross_origin_url", 2023-12-07T10:36:21.082 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.082 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.082 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.082 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.082 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.082 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.083 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.083 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.083 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.083 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.083 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.083 INFO:teuthology.orchestra.run.smithi119.stdout: "crt_file": { 2023-12-07T10:36:21.083 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crt_file", 2023-12-07T10:36:21.083 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.084 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.084 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.084 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.084 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.084 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.084 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.084 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.084 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.084 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.085 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.085 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.085 INFO:teuthology.orchestra.run.smithi119.stdout: "debug": { 2023-12-07T10:36:21.085 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "debug", 2023-12-07T10:36:21.085 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.085 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.085 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.085 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.085 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.086 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.086 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.086 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Enable/disable debug options", 2023-12-07T10:36:21.086 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.086 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.086 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.086 INFO:teuthology.orchestra.run.smithi119.stdout: "jwt_token_ttl": { 2023-12-07T10:36:21.086 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "jwt_token_ttl", 2023-12-07T10:36:21.087 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.087 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.087 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.087 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "28800", 2023-12-07T10:36:21.087 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.087 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.087 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.087 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.088 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.088 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.088 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.088 INFO:teuthology.orchestra.run.smithi119.stdout: "key_file": { 2023-12-07T10:36:21.088 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "key_file", 2023-12-07T10:36:21.088 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.088 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.088 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.089 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.089 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.089 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.089 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.089 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.089 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.089 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.089 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.089 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.090 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.090 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.090 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.090 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.090 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.090 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.090 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.090 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.091 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.091 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.091 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.091 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.091 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.091 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.091 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.091 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.092 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.092 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.092 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.092 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.092 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.092 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.092 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.092 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.093 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.093 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.093 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.093 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.093 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.093 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.093 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.093 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.093 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.094 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.094 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.094 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.094 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.094 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.094 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.094 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.094 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.095 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.095 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.095 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.095 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.095 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.095 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.095 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.095 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.095 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.096 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.096 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.096 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.096 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.096 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.096 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.096 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.096 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.097 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.097 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.097 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.097 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.097 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.097 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.097 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.097 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.097 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.098 INFO:teuthology.orchestra.run.smithi119.stdout: "motd": { 2023-12-07T10:36:21.098 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "motd", 2023-12-07T10:36:21.098 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.098 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.098 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.098 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.098 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.098 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.098 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.099 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "The message of the day", 2023-12-07T10:36:21.099 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.099 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.099 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.099 INFO:teuthology.orchestra.run.smithi119.stdout: "redirect_resolve_ip_addr": { 2023-12-07T10:36:21.099 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "redirect_resolve_ip_addr", 2023-12-07T10:36:21.099 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.099 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.100 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.100 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.100 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.100 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.100 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.100 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.100 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.100 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.100 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.101 INFO:teuthology.orchestra.run.smithi119.stdout: "server_addr": { 2023-12-07T10:36:21.101 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_addr", 2023-12-07T10:36:21.101 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.101 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.101 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.101 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "::", 2023-12-07T10:36:21.101 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.102 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.102 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.102 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.102 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.102 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.102 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.102 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.102 INFO:teuthology.orchestra.run.smithi119.stdout: "server_port": { 2023-12-07T10:36:21.102 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_port", 2023-12-07T10:36:21.103 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.103 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.103 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.103 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8080", 2023-12-07T10:36:21.103 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.103 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.103 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.103 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.103 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.104 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.104 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.104 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.104 INFO:teuthology.orchestra.run.smithi119.stdout: "ssl": { 2023-12-07T10:36:21.104 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ssl", 2023-12-07T10:36:21.104 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.104 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.104 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.104 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.104 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.105 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.105 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.105 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.105 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.105 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.105 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.105 INFO:teuthology.orchestra.run.smithi119.stdout: "ssl_server_port": { 2023-12-07T10:36:21.105 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ssl_server_port", 2023-12-07T10:36:21.106 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.106 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.106 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.106 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8443", 2023-12-07T10:36:21.106 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.106 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.106 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.106 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.106 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.107 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.107 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.107 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.107 INFO:teuthology.orchestra.run.smithi119.stdout: "standby_behaviour": { 2023-12-07T10:36:21.107 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "standby_behaviour", 2023-12-07T10:36:21.107 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.107 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.107 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.108 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "redirect", 2023-12-07T10:36:21.108 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.108 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.108 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.108 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.108 INFO:teuthology.orchestra.run.smithi119.stdout: "redirect" 2023-12-07T10:36:21.108 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.108 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.108 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.109 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.109 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.109 INFO:teuthology.orchestra.run.smithi119.stdout: "standby_error_status_code": { 2023-12-07T10:36:21.109 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "standby_error_status_code", 2023-12-07T10:36:21.109 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.109 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.109 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.109 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "500", 2023-12-07T10:36:21.110 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "400", 2023-12-07T10:36:21.110 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "599", 2023-12-07T10:36:21.110 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.110 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.110 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.110 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.110 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.110 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.110 INFO:teuthology.orchestra.run.smithi119.stdout: "url_prefix": { 2023-12-07T10:36:21.111 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "url_prefix", 2023-12-07T10:36:21.111 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.111 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.111 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.111 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.111 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.111 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.111 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.111 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.112 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.112 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.112 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.112 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.112 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.112 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "devicehealth", 2023-12-07T10:36:21.112 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.113 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.113 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.113 INFO:teuthology.orchestra.run.smithi119.stdout: "enable_monitoring": { 2023-12-07T10:36:21.113 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "enable_monitoring", 2023-12-07T10:36:21.113 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.113 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.113 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.113 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.113 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.114 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.114 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.114 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "monitor device health metrics", 2023-12-07T10:36:21.114 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.114 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.114 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.114 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.115 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.115 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.115 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.115 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.115 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.115 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.115 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.115 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.115 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.116 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.116 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.116 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.116 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.116 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.116 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.116 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.116 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.116 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.117 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.117 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.117 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.117 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.117 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.117 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.117 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.117 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.117 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.118 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.118 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.118 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.118 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.118 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.118 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.118 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.118 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.119 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.119 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.119 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.119 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.119 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.119 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.119 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.119 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.119 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.120 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.120 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.120 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.120 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.120 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.120 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.120 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.120 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.121 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.121 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.121 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.121 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.121 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.121 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.121 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.121 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.121 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.122 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.122 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.122 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.122 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.122 INFO:teuthology.orchestra.run.smithi119.stdout: "mark_out_threshold": { 2023-12-07T10:36:21.122 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mark_out_threshold", 2023-12-07T10:36:21.122 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.122 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.123 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.123 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "2419200", 2023-12-07T10:36:21.123 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.123 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.123 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.123 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-12-07T10:36:21.123 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.123 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.123 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.124 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_name": { 2023-12-07T10:36:21.124 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "pool_name", 2023-12-07T10:36:21.124 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.124 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.124 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.124 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "device_health_metrics", 2023-12-07T10:36:21.124 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.125 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.125 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.125 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "name of pool in which to store device health metrics", 2023-12-07T10:36:21.125 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.125 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.125 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.125 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.125 INFO:teuthology.orchestra.run.smithi119.stdout: "retention_period": { 2023-12-07T10:36:21.125 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "retention_period", 2023-12-07T10:36:21.126 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.126 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.126 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.126 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "15552000", 2023-12-07T10:36:21.126 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.126 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.126 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.126 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how long to retain device health metrics", 2023-12-07T10:36:21.126 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.127 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.127 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.127 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.127 INFO:teuthology.orchestra.run.smithi119.stdout: "scrape_frequency": { 2023-12-07T10:36:21.127 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "scrape_frequency", 2023-12-07T10:36:21.127 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.127 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.127 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.128 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "86400", 2023-12-07T10:36:21.128 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.128 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.128 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.128 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to scrape device health metrics", 2023-12-07T10:36:21.128 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.128 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.128 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.128 INFO:teuthology.orchestra.run.smithi119.stdout: "self_heal": { 2023-12-07T10:36:21.129 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "self_heal", 2023-12-07T10:36:21.129 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.129 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.129 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.129 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.129 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.129 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.129 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.129 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-12-07T10:36:21.130 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.130 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.130 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.130 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.130 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2023-12-07T10:36:21.130 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2023-12-07T10:36:21.130 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.130 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.130 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.131 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2023-12-07T10:36:21.131 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.131 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.131 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.131 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to wake up and check device health", 2023-12-07T10:36:21.131 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.131 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.131 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.132 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_threshold": { 2023-12-07T10:36:21.132 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_threshold", 2023-12-07T10:36:21.132 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.132 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.132 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.132 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "7257600", 2023-12-07T10:36:21.132 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.132 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.133 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.133 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-12-07T10:36:21.133 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.133 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.133 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.133 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.133 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.133 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.134 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "diskprediction_local", 2023-12-07T10:36:21.134 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.134 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.134 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.134 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.134 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.134 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.134 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.135 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.135 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.135 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.135 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.135 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.135 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.135 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.135 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.135 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.136 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.136 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.136 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.136 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.136 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.136 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.136 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.136 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.136 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.137 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.137 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.137 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.137 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.137 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.137 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.137 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.137 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.137 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.138 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.138 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.138 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.138 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.138 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.138 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.138 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.138 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.138 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.139 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.139 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.139 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.139 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.139 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.139 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.139 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.139 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.139 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.140 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.140 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.140 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.140 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.140 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.140 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.140 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.140 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.141 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.141 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.141 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.141 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.141 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.141 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.141 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.141 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.141 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.142 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.142 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.142 INFO:teuthology.orchestra.run.smithi119.stdout: "predict_interval": { 2023-12-07T10:36:21.142 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "predict_interval", 2023-12-07T10:36:21.142 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.142 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.142 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.142 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "86400", 2023-12-07T10:36:21.143 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.143 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.143 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.143 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.143 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.143 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.143 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.143 INFO:teuthology.orchestra.run.smithi119.stdout: "predictor_model": { 2023-12-07T10:36:21.144 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "predictor_model", 2023-12-07T10:36:21.144 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.144 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.144 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.144 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "prophetstor", 2023-12-07T10:36:21.144 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.144 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.144 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.144 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.145 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.145 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.145 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.145 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2023-12-07T10:36:21.145 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2023-12-07T10:36:21.145 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.145 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.146 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.146 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2023-12-07T10:36:21.146 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.146 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.146 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.146 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.146 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.146 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.146 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.147 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.147 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.147 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.147 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "influx", 2023-12-07T10:36:21.147 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": false, 2023-12-07T10:36:21.147 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "influxdb python module not found", 2023-12-07T10:36:21.147 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.147 INFO:teuthology.orchestra.run.smithi119.stdout: "batch_size": { 2023-12-07T10:36:21.148 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "batch_size", 2023-12-07T10:36:21.148 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.148 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.148 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.148 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5000", 2023-12-07T10:36:21.148 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.148 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.148 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.148 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-12-07T10:36:21.149 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.149 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.149 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.149 INFO:teuthology.orchestra.run.smithi119.stdout: "database": { 2023-12-07T10:36:21.149 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "database", 2023-12-07T10:36:21.149 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.149 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.149 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.150 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "ceph", 2023-12-07T10:36:21.150 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.150 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.150 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.150 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-12-07T10:36:21.150 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.150 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.150 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.151 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.151 INFO:teuthology.orchestra.run.smithi119.stdout: "hostname": { 2023-12-07T10:36:21.151 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "hostname", 2023-12-07T10:36:21.151 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.151 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.151 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.151 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.151 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.151 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.152 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.152 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "InfluxDB server hostname", 2023-12-07T10:36:21.152 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.152 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.152 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.152 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.152 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2023-12-07T10:36:21.152 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2023-12-07T10:36:21.153 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.153 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.153 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.153 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "30", 2023-12-07T10:36:21.153 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "5", 2023-12-07T10:36:21.153 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.153 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.153 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-12-07T10:36:21.153 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.154 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.154 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.154 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.154 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.154 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.154 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.154 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.155 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.155 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.155 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.155 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.155 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.155 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.155 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.155 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.155 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.156 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.156 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.156 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.156 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.156 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.156 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.156 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.156 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.157 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.157 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.157 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.157 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.157 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.157 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.157 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.157 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.157 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.158 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.158 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.158 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.158 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.158 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.158 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.158 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.158 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.158 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.159 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.159 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.159 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.159 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.159 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.159 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.159 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.159 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.159 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.160 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.160 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.160 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.160 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.160 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.160 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.160 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.160 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.161 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.161 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.161 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.161 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.161 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.161 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.161 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.161 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.161 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.161 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.162 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.162 INFO:teuthology.orchestra.run.smithi119.stdout: "password": { 2023-12-07T10:36:21.162 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "password", 2023-12-07T10:36:21.162 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.162 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.162 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.162 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.163 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.163 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.163 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.163 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "password of InfluxDB server user", 2023-12-07T10:36:21.163 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.163 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.163 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.163 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.163 INFO:teuthology.orchestra.run.smithi119.stdout: "port": { 2023-12-07T10:36:21.163 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "port", 2023-12-07T10:36:21.164 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.164 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.164 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.164 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8086", 2023-12-07T10:36:21.164 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.164 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.164 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.164 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "InfluxDB server port", 2023-12-07T10:36:21.164 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.165 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.165 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.165 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.165 INFO:teuthology.orchestra.run.smithi119.stdout: "ssl": { 2023-12-07T10:36:21.165 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ssl", 2023-12-07T10:36:21.165 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.165 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.165 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.165 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "false", 2023-12-07T10:36:21.165 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.166 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.166 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.166 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-12-07T10:36:21.166 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.166 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.166 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.166 INFO:teuthology.orchestra.run.smithi119.stdout: "threads": { 2023-12-07T10:36:21.166 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "threads", 2023-12-07T10:36:21.167 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.167 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.167 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.167 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5", 2023-12-07T10:36:21.167 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1", 2023-12-07T10:36:21.167 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "32", 2023-12-07T10:36:21.167 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.167 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-12-07T10:36:21.167 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.168 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.168 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.168 INFO:teuthology.orchestra.run.smithi119.stdout: "username": { 2023-12-07T10:36:21.168 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "username", 2023-12-07T10:36:21.168 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.168 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.168 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.168 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.169 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.169 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.169 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.169 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "username of InfluxDB server user", 2023-12-07T10:36:21.169 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.169 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.169 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.169 INFO:teuthology.orchestra.run.smithi119.stdout: "verify_ssl": { 2023-12-07T10:36:21.170 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "verify_ssl", 2023-12-07T10:36:21.170 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.170 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.170 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.170 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "true", 2023-12-07T10:36:21.170 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.170 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.170 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.170 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-12-07T10:36:21.171 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.171 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.171 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.171 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.171 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.171 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "insights", 2023-12-07T10:36:21.172 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.172 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.172 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.172 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.172 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.172 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.172 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.172 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.172 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.173 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.173 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.173 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.173 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.173 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.173 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.173 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.173 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.174 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.174 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.174 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.174 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.174 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.174 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.174 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.174 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.174 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.175 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.175 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.175 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.175 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.175 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.175 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.175 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.175 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.175 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.176 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.176 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.176 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.176 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.176 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.176 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.176 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.177 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.177 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.177 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.177 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.177 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.177 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.177 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.177 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.177 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.178 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.178 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.178 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.178 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.178 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.178 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.178 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.178 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.179 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.179 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.179 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.179 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.179 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.179 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.179 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.179 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.179 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.180 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.180 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.180 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.180 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.180 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.180 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.180 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "iostat", 2023-12-07T10:36:21.180 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.181 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.181 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.181 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.181 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.181 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.181 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.181 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.181 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.181 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.182 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.182 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.182 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.182 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.182 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.182 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.182 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.182 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.183 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.183 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.183 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.183 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.183 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.183 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.183 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.183 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.184 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.184 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.184 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.184 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.184 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.184 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.184 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.184 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.184 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.185 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.185 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.185 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.185 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.185 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.185 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.185 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.185 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.186 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.186 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.186 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.186 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.186 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.186 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.186 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.186 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.186 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.187 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.187 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.187 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.187 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.187 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.187 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.187 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.187 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.188 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.188 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.188 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.188 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.188 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.188 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.188 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.188 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.188 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.189 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.189 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.189 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "k8sevents", 2023-12-07T10:36:21.189 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.189 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.189 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.189 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_event_retention_days": { 2023-12-07T10:36:21.190 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ceph_event_retention_days", 2023-12-07T10:36:21.190 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.190 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.190 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.190 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "7", 2023-12-07T10:36:21.190 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.190 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.190 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.191 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Days to hold ceph event information within local cache", 2023-12-07T10:36:21.191 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.191 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.191 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.191 INFO:teuthology.orchestra.run.smithi119.stdout: "config_check_secs": { 2023-12-07T10:36:21.191 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "config_check_secs", 2023-12-07T10:36:21.191 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.191 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.192 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.192 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2023-12-07T10:36:21.192 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "10", 2023-12-07T10:36:21.192 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.192 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.192 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-12-07T10:36:21.192 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.192 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.192 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.193 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.193 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.193 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.193 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.193 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.193 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.193 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.193 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.193 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.194 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.194 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.194 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.194 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.194 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.194 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.194 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.194 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.194 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.195 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.195 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.195 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.195 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.195 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.195 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.195 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.195 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.195 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.196 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.196 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.196 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.196 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.196 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.196 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.196 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.196 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.197 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.197 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.197 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.197 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.197 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.197 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.197 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.197 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.197 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.198 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.198 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.198 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.198 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.198 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.198 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.198 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.199 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.199 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.199 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.199 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.199 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.199 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.199 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.199 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.199 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.200 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.200 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.200 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.200 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.200 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.200 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.200 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.200 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.201 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.201 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.201 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.201 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.201 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "localpool", 2023-12-07T10:36:21.201 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.201 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.201 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.201 INFO:teuthology.orchestra.run.smithi119.stdout: "failure_domain": { 2023-12-07T10:36:21.202 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "failure_domain", 2023-12-07T10:36:21.202 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.202 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.202 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.202 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "host", 2023-12-07T10:36:21.202 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.202 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.202 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.203 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "failure domain for any created local pool", 2023-12-07T10:36:21.203 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-12-07T10:36:21.203 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.203 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.203 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.203 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.203 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.203 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.204 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.204 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.204 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.204 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.204 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.204 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.204 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.204 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.204 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.205 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.205 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.205 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.205 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.205 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.205 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.205 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.205 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.205 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.206 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.206 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.206 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.206 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.206 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.206 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.206 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.206 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.207 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.207 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.207 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.207 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.207 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.207 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.207 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.207 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.208 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.208 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.208 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.208 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.208 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.208 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.208 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.208 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.208 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.209 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.209 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.209 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.209 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.209 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.209 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.209 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.210 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.210 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.210 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.210 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.210 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.210 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.210 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.210 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.210 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.211 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.211 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.211 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.211 INFO:teuthology.orchestra.run.smithi119.stdout: "min_size": { 2023-12-07T10:36:21.211 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "min_size", 2023-12-07T10:36:21.211 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.211 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.211 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.212 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.212 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.212 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.212 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.212 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "default min_size for any created local pool", 2023-12-07T10:36:21.212 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-12-07T10:36:21.212 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.212 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.213 INFO:teuthology.orchestra.run.smithi119.stdout: "num_rep": { 2023-12-07T10:36:21.213 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "num_rep", 2023-12-07T10:36:21.213 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.213 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.213 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.213 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "3", 2023-12-07T10:36:21.213 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.213 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.213 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.214 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "default replica count for any created local pool", 2023-12-07T10:36:21.214 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.214 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.214 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.214 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num": { 2023-12-07T10:36:21.214 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "pg_num", 2023-12-07T10:36:21.214 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.215 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.215 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.215 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "128", 2023-12-07T10:36:21.215 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.215 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.215 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.215 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "default pg_num for any created local pool", 2023-12-07T10:36:21.215 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.215 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.216 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.216 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.216 INFO:teuthology.orchestra.run.smithi119.stdout: "prefix": { 2023-12-07T10:36:21.216 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "prefix", 2023-12-07T10:36:21.216 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.216 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.216 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.216 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.217 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.217 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.217 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.217 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "name prefix for any created local pool", 2023-12-07T10:36:21.217 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.217 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.217 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.217 INFO:teuthology.orchestra.run.smithi119.stdout: "subtree": { 2023-12-07T10:36:21.217 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "subtree", 2023-12-07T10:36:21.218 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.218 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.218 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.218 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "rack", 2023-12-07T10:36:21.218 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.218 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.218 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.218 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "CRUSH level for which to create a local pool", 2023-12-07T10:36:21.218 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-12-07T10:36:21.219 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.219 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.219 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.219 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.219 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.219 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mds_autoscaler", 2023-12-07T10:36:21.219 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.220 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.220 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.220 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.220 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.220 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.220 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.220 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.220 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.220 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.221 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.221 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.221 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.221 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.221 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.221 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.221 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.221 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.222 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.222 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.222 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.222 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.222 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.222 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.222 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.222 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.223 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.223 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.223 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.223 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.223 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.223 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.223 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.223 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.223 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.224 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.224 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.224 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.224 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.224 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.224 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.224 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.224 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.224 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.225 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.225 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.225 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.225 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.225 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.225 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.225 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.225 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.226 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.226 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.226 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.226 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.226 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.226 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.226 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.226 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.226 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.227 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.227 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.227 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.227 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.227 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.227 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.227 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.227 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.228 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.228 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.228 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.228 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.228 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.228 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.228 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mirroring", 2023-12-07T10:36:21.228 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.228 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.229 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.229 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.229 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.229 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.229 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.229 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.229 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.229 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.229 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.229 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.230 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.230 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.230 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.230 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.230 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.230 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.230 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.230 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.231 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.231 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.231 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.231 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.231 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.231 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.231 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.231 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.231 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.232 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.232 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.232 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.232 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.232 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.232 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.232 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.232 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.233 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.233 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.233 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.233 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.233 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.233 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.233 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.233 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.233 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.234 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.234 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.234 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.234 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.234 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.234 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.234 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.234 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.235 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.235 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.235 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.235 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.235 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.235 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.235 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.235 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.236 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.236 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.236 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.236 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.236 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.236 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.236 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.236 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.236 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.237 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.237 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.237 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.237 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "nfs", 2023-12-07T10:36:21.237 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.237 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.237 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.237 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.237 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.238 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.238 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.238 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.238 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.238 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.238 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.238 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.238 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.239 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.239 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.239 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.239 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.239 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.239 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.239 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.239 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.239 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.240 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.240 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.240 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.240 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.240 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.240 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.240 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.240 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.241 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.241 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.241 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.241 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.241 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.241 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.241 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.242 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.242 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.242 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.242 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.242 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.242 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.243 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.243 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.243 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.243 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.243 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.243 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.244 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.244 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.244 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.244 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.244 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.245 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.245 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.245 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.245 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.245 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.245 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.246 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.246 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.246 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.246 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.246 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.247 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.247 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.247 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.247 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.247 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.248 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.248 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.248 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.248 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.249 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "orchestrator", 2023-12-07T10:36:21.249 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.249 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.249 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.249 INFO:teuthology.orchestra.run.smithi119.stdout: "fail_fs": { 2023-12-07T10:36:21.250 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "fail_fs", 2023-12-07T10:36:21.250 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.250 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.250 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.250 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.250 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.251 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.251 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.251 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-12-07T10:36:21.251 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.251 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.252 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.252 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.252 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.252 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.252 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.252 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.252 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.252 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.253 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.253 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.253 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.253 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.253 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.253 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.253 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.253 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.253 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.253 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.254 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.254 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.254 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.254 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.254 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.254 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.254 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.254 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.255 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.255 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.255 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.255 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.255 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.255 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.255 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.255 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.255 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.256 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.256 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.256 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.256 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.256 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.256 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.256 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.256 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.256 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.257 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.257 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.257 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.257 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.257 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.257 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.257 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.257 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.257 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.257 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.258 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.258 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.258 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.258 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.258 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.258 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.258 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.258 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.259 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.259 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.259 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.259 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.259 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.259 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.259 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.259 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.259 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.260 INFO:teuthology.orchestra.run.smithi119.stdout: "orchestrator": { 2023-12-07T10:36:21.260 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "orchestrator", 2023-12-07T10:36:21.260 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.260 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.260 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.260 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.260 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.260 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.260 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.261 INFO:teuthology.orchestra.run.smithi119.stdout: "cephadm", 2023-12-07T10:36:21.261 INFO:teuthology.orchestra.run.smithi119.stdout: "rook", 2023-12-07T10:36:21.261 INFO:teuthology.orchestra.run.smithi119.stdout: "test_orchestrator" 2023-12-07T10:36:21.261 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.261 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Orchestrator backend", 2023-12-07T10:36:21.261 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.261 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.261 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.261 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.262 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.262 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.262 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "osd_perf_query", 2023-12-07T10:36:21.262 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.262 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.262 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.262 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.262 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.263 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.263 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.263 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.263 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.263 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.263 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.263 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.263 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.263 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.264 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.264 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.264 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.264 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.264 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.264 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.264 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.264 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.264 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.265 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.265 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.265 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.265 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.265 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.265 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.265 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.265 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.266 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.266 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.266 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.266 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.266 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.266 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.266 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.266 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.266 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.267 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.267 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.267 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.267 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.267 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.267 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.267 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.267 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.268 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.268 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.268 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.268 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.268 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.268 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.268 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.268 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.268 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.268 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.269 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.269 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.269 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.269 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.269 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.269 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.269 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.269 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.269 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.270 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.270 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.270 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.270 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.270 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.270 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.270 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.270 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "osd_support", 2023-12-07T10:36:21.270 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.271 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.271 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.271 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.271 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.271 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.271 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.271 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.271 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.272 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.272 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.272 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.272 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.272 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.272 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.272 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.272 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.272 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.273 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.273 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.273 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.273 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.273 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.273 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.273 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.273 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.274 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.274 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.274 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.274 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.274 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.274 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.274 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.274 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.274 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.275 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.275 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.275 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.275 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.275 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.275 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.275 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.275 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.276 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.276 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.276 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.276 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.276 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.276 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.276 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.276 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.277 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.277 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.277 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.277 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.277 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.277 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.277 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.277 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.278 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.278 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.278 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.278 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.278 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.278 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.278 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.278 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.279 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.279 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.279 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.279 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.279 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "pg_autoscaler", 2023-12-07T10:36:21.279 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.279 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.280 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.280 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.280 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.280 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.280 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.280 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.280 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.280 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.280 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.280 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.281 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.281 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.281 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.281 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.281 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.281 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.281 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.281 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.281 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.282 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.282 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.282 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.282 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.282 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.282 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.282 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.283 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.283 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.283 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.283 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.283 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.283 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.283 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.283 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.284 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.284 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.284 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.284 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.284 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.284 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.284 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.284 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.284 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.285 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.285 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.285 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.285 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.285 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.285 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.285 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.285 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.285 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.286 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.286 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.286 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.286 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.286 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.286 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.286 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.287 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.287 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.287 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.287 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.287 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.287 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.287 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.287 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.287 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.288 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2023-12-07T10:36:21.288 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2023-12-07T10:36:21.288 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.288 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.288 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.288 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2023-12-07T10:36:21.288 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.288 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.288 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.288 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.289 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.289 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.289 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.289 INFO:teuthology.orchestra.run.smithi119.stdout: "threshold": { 2023-12-07T10:36:21.289 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "threshold", 2023-12-07T10:36:21.289 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2023-12-07T10:36:21.289 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.289 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.290 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "3.0", 2023-12-07T10:36:21.290 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1.0", 2023-12-07T10:36:21.290 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.290 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.290 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "scaling threshold", 2023-12-07T10:36:21.290 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-12-07T10:36:21.290 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.290 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.291 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.291 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.291 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.291 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "progress", 2023-12-07T10:36:21.291 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.291 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.291 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.291 INFO:teuthology.orchestra.run.smithi119.stdout: "allow_pg_recovery_event": { 2023-12-07T10:36:21.292 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "allow_pg_recovery_event", 2023-12-07T10:36:21.292 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.292 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.292 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.292 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.292 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.292 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.292 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.292 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "allow the module to show pg recovery progress", 2023-12-07T10:36:21.293 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.293 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.293 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.293 INFO:teuthology.orchestra.run.smithi119.stdout: "enabled": { 2023-12-07T10:36:21.293 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "enabled", 2023-12-07T10:36:21.293 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.293 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.293 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.294 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.294 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.294 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.294 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.294 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.294 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.294 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.294 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.295 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.295 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.295 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.295 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.295 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.295 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.295 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.295 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.295 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.296 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.296 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.296 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.296 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.296 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.296 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.296 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.296 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.297 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.297 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.297 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.297 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.297 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.297 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.297 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.297 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.298 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.298 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.298 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.298 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.298 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.298 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.298 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.298 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.298 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.299 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.299 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.299 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.299 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.299 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.299 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.299 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.299 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.299 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.300 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.300 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.300 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.300 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.300 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.300 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.300 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.300 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.300 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.301 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.301 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.301 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.301 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.301 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.301 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.301 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.301 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.301 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.302 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.302 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.302 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.302 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.302 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.302 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.302 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.303 INFO:teuthology.orchestra.run.smithi119.stdout: "max_completed_events": { 2023-12-07T10:36:21.303 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_completed_events", 2023-12-07T10:36:21.303 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.303 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.303 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.303 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "50", 2023-12-07T10:36:21.303 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.303 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.303 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.304 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "number of past completed events to remember", 2023-12-07T10:36:21.304 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.304 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.304 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.304 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2023-12-07T10:36:21.304 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2023-12-07T10:36:21.304 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.304 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.305 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.305 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5", 2023-12-07T10:36:21.305 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.305 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.305 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.305 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how long the module is going to sleep", 2023-12-07T10:36:21.305 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.305 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.305 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.306 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.306 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.306 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.306 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.306 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "prometheus", 2023-12-07T10:36:21.306 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.306 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.306 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.306 INFO:teuthology.orchestra.run.smithi119.stdout: "cache": { 2023-12-07T10:36:21.307 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cache", 2023-12-07T10:36:21.307 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.307 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.307 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.307 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.307 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.307 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.307 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.307 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.308 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.308 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.308 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.308 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.308 INFO:teuthology.orchestra.run.smithi119.stdout: "exclude_perf_counters": { 2023-12-07T10:36:21.308 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "exclude_perf_counters", 2023-12-07T10:36:21.308 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.308 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.308 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.309 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.309 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.309 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.309 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.309 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-12-07T10:36:21.309 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-12-07T10:36:21.309 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.309 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.310 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.310 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.310 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.310 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.310 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.310 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.310 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.310 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.311 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.311 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.311 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.311 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.311 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.311 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.311 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.311 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.311 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.311 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.312 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.312 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.312 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.312 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.312 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.312 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.312 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.312 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.313 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.313 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.313 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.313 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.313 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.313 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.313 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.313 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.313 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.314 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.314 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.314 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.314 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.314 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.314 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.314 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.314 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.314 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.315 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.315 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.315 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.315 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.315 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.315 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.315 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.315 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.316 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.316 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.316 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.316 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.316 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.316 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.316 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.316 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.317 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.317 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.317 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.317 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.317 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.317 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.317 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.317 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_stats_pools": { 2023-12-07T10:36:21.318 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rbd_stats_pools", 2023-12-07T10:36:21.318 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.318 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.318 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.318 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.318 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.318 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.318 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.318 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.319 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.319 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.319 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.319 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.319 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_stats_pools_refresh_interval": { 2023-12-07T10:36:21.319 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-12-07T10:36:21.319 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.319 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.319 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.320 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "300", 2023-12-07T10:36:21.320 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.320 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.320 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.320 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.320 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.320 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.320 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.321 INFO:teuthology.orchestra.run.smithi119.stdout: "scrape_interval": { 2023-12-07T10:36:21.321 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "scrape_interval", 2023-12-07T10:36:21.321 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2023-12-07T10:36:21.321 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.321 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.321 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "15.0", 2023-12-07T10:36:21.321 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.321 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.321 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.322 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.322 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.322 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.322 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.322 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.322 INFO:teuthology.orchestra.run.smithi119.stdout: "server_addr": { 2023-12-07T10:36:21.322 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_addr", 2023-12-07T10:36:21.322 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.325 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.325 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.325 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "::", 2023-12-07T10:36:21.325 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.325 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.325 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.326 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-12-07T10:36:21.326 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.326 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.326 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.326 INFO:teuthology.orchestra.run.smithi119.stdout: "server_port": { 2023-12-07T10:36:21.326 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_port", 2023-12-07T10:36:21.326 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.327 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.327 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.327 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "9283", 2023-12-07T10:36:21.327 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.327 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.327 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.327 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-12-07T10:36:21.327 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.327 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.327 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.328 INFO:teuthology.orchestra.run.smithi119.stdout: "stale_cache_strategy": { 2023-12-07T10:36:21.328 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "stale_cache_strategy", 2023-12-07T10:36:21.328 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.328 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.328 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.328 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "log", 2023-12-07T10:36:21.328 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.328 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.329 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.329 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.329 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.329 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.329 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.329 INFO:teuthology.orchestra.run.smithi119.stdout: "standby_behaviour": { 2023-12-07T10:36:21.329 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "standby_behaviour", 2023-12-07T10:36:21.329 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.329 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.330 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.330 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "default", 2023-12-07T10:36:21.330 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.330 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.330 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.330 INFO:teuthology.orchestra.run.smithi119.stdout: "default", 2023-12-07T10:36:21.330 INFO:teuthology.orchestra.run.smithi119.stdout: "error" 2023-12-07T10:36:21.330 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.330 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.331 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.331 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.331 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.331 INFO:teuthology.orchestra.run.smithi119.stdout: "standby_error_status_code": { 2023-12-07T10:36:21.331 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "standby_error_status_code", 2023-12-07T10:36:21.331 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.331 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.331 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.331 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "500", 2023-12-07T10:36:21.332 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "400", 2023-12-07T10:36:21.332 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "599", 2023-12-07T10:36:21.332 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.332 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.332 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.332 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.332 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.332 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.332 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.332 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rbd_support", 2023-12-07T10:36:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.333 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.334 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.335 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.336 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.337 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.338 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.339 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.340 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "max_concurrent_snap_create": { 2023-12-07T10:36:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_concurrent_snap_create", 2023-12-07T10:36:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2023-12-07T10:36:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.341 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "mirror_snapshot_schedule": { 2023-12-07T10:36:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mirror_snapshot_schedule", 2023-12-07T10:36:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.342 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "trash_purge_schedule": { 2023-12-07T10:36:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "trash_purge_schedule", 2023-12-07T10:36:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.344 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "restful", 2023-12-07T10:36:21.345 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "enable_auth": { 2023-12-07T10:36:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "enable_auth", 2023-12-07T10:36:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.346 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "key_file": { 2023-12-07T10:36:21.347 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "key_file", 2023-12-07T10:36:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.348 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.350 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.351 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.352 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.353 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.354 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.356 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.357 INFO:teuthology.orchestra.run.smithi119.stdout: "server_addr": { 2023-12-07T10:36:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_addr", 2023-12-07T10:36:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.358 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: "server_port": { 2023-12-07T10:36:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_port", 2023-12-07T10:36:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.359 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.360 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rgw", 2023-12-07T10:36:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.361 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.362 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.363 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.364 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.364 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.364 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.364 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.364 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.364 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.364 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.364 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.364 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.364 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.364 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.365 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.366 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.367 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.368 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rook", 2023-12-07T10:36:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: "drive_group_interval": { 2023-12-07T10:36:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "drive_group_interval", 2023-12-07T10:36:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2023-12-07T10:36:21.369 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "300.0", 2023-12-07T10:36:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-12-07T10:36:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.370 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.371 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.372 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.373 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.374 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.375 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.376 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.377 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: "storage_class": { 2023-12-07T10:36:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "storage_class", 2023-12-07T10:36:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.378 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "local", 2023-12-07T10:36:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-12-07T10:36:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.379 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "selftest", 2023-12-07T10:36:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.380 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.381 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.383 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.384 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.385 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.386 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.387 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.388 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.388 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.388 INFO:teuthology.orchestra.run.smithi119.stdout: "roption1": { 2023-12-07T10:36:21.388 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "roption1", 2023-12-07T10:36:21.388 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.388 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.388 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.388 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.388 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.389 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.389 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.389 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.389 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.389 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.389 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.389 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.389 INFO:teuthology.orchestra.run.smithi119.stdout: "roption2": { 2023-12-07T10:36:21.389 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "roption2", 2023-12-07T10:36:21.389 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.390 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.390 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.390 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "xyz", 2023-12-07T10:36:21.390 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.390 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.390 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.390 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.390 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.391 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.391 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.391 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption1": { 2023-12-07T10:36:21.391 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption1", 2023-12-07T10:36:21.391 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.391 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.391 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.391 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.391 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.392 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.392 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.392 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.392 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.392 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.392 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.392 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.392 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption2": { 2023-12-07T10:36:21.392 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption2", 2023-12-07T10:36:21.393 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.393 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.393 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.393 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.393 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.393 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.393 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.393 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.393 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.393 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.394 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.394 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption3": { 2023-12-07T10:36:21.394 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption3", 2023-12-07T10:36:21.394 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2023-12-07T10:36:21.394 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.394 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.394 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.395 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.395 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.395 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.395 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.395 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.395 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.395 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.395 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption4": { 2023-12-07T10:36:21.395 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption4", 2023-12-07T10:36:21.396 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.396 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.396 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.396 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.396 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.396 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.396 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.396 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.397 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.397 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.397 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.397 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption5": { 2023-12-07T10:36:21.397 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption5", 2023-12-07T10:36:21.397 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.397 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.397 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.397 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.398 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.398 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.398 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.398 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.398 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.398 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.398 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.398 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.399 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption6": { 2023-12-07T10:36:21.399 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption6", 2023-12-07T10:36:21.399 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.399 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.399 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.399 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.399 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.399 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.399 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.400 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.400 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.400 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.400 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.400 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.400 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption7": { 2023-12-07T10:36:21.400 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption7", 2023-12-07T10:36:21.400 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.400 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.401 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.401 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.401 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1", 2023-12-07T10:36:21.401 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "42", 2023-12-07T10:36:21.401 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.401 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.401 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.401 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.401 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.402 INFO:teuthology.orchestra.run.smithi119.stdout: "testkey": { 2023-12-07T10:36:21.402 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "testkey", 2023-12-07T10:36:21.402 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.402 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.402 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.402 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.402 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.402 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.403 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.403 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.403 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.403 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.403 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.403 INFO:teuthology.orchestra.run.smithi119.stdout: "testlkey": { 2023-12-07T10:36:21.403 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "testlkey", 2023-12-07T10:36:21.403 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.403 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.404 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.404 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.404 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.404 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.404 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.404 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.404 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.404 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.404 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.405 INFO:teuthology.orchestra.run.smithi119.stdout: "testnewline": { 2023-12-07T10:36:21.405 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "testnewline", 2023-12-07T10:36:21.405 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.405 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.405 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.405 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.405 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.405 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.406 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.406 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.406 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.406 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.406 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.406 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.406 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.406 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.406 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.407 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "snap_schedule", 2023-12-07T10:36:21.407 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.407 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.407 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.407 INFO:teuthology.orchestra.run.smithi119.stdout: "allow_m_granularity": { 2023-12-07T10:36:21.407 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "allow_m_granularity", 2023-12-07T10:36:21.407 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.407 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.407 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.408 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.408 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.408 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.408 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.408 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "allow minute scheduled snapshots", 2023-12-07T10:36:21.408 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.408 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.408 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.408 INFO:teuthology.orchestra.run.smithi119.stdout: "dump_on_update": { 2023-12-07T10:36:21.409 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "dump_on_update", 2023-12-07T10:36:21.409 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.409 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.409 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.409 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.409 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.409 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.409 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.409 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "dump database to debug log on update", 2023-12-07T10:36:21.410 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.410 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.410 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.410 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.410 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.410 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.410 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.411 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.411 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.411 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.411 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.411 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.411 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.411 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.411 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.411 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.412 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.412 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.412 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.412 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.412 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.412 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.412 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.412 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.412 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.413 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.413 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.413 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.413 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.413 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.413 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.413 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.413 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.413 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.414 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.414 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.414 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.414 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.414 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.414 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.414 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.414 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.414 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.415 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.415 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.415 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.415 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.415 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.415 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.415 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.415 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.415 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.416 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.416 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.416 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.416 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.416 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.416 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.416 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.417 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.417 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.417 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.417 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.417 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.417 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.417 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.417 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.417 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.418 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.418 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.418 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.418 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.418 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.418 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "stats", 2023-12-07T10:36:21.418 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.418 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.419 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.419 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.419 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.419 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.419 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.419 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.419 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.419 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.419 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.420 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.420 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.420 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.420 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.420 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.420 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.420 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.420 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.420 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.420 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.421 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.421 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.421 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.421 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.421 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.421 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.421 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.421 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.422 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.422 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.422 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.422 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.422 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.422 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.422 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.422 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.422 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.423 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.423 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.423 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.423 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.423 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.423 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.423 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.423 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.423 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.423 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.424 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.424 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.424 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.424 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.424 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.424 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.424 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.424 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.424 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.425 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.425 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.425 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.425 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.425 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.425 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.425 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.425 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.425 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.425 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.426 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.426 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.426 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.426 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.426 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.426 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.426 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.426 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "status", 2023-12-07T10:36:21.427 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.427 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.427 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.427 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.427 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.427 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.427 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.427 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.427 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.428 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.428 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.428 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.428 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.428 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.428 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.428 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.428 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.428 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.428 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.429 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.429 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.429 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.429 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.429 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.429 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.429 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.429 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.430 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.430 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.430 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.430 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.430 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.430 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.430 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.430 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.430 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.431 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.431 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.431 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.431 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.431 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.431 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.431 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.431 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.431 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.432 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.432 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.432 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.432 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.432 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.432 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.432 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.432 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.432 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.433 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.433 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.433 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.433 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.433 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.433 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.433 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.433 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.433 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.434 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.434 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.434 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.434 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.434 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.434 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.434 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.434 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.435 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "telegraf", 2023-12-07T10:36:21.435 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.435 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.435 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.435 INFO:teuthology.orchestra.run.smithi119.stdout: "address": { 2023-12-07T10:36:21.435 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "address", 2023-12-07T10:36:21.435 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.435 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.436 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.436 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-12-07T10:36:21.436 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.436 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.436 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.436 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.436 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.436 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.436 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.437 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2023-12-07T10:36:21.437 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2023-12-07T10:36:21.437 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.437 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.437 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.437 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "15", 2023-12-07T10:36:21.437 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.437 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.438 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.438 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.438 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.438 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.438 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.438 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.438 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.438 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.439 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.439 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.439 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.439 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.439 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.439 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.439 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.439 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.439 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.440 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.440 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.440 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.440 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.440 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.440 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.440 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.440 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.441 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.441 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.441 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.441 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.441 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.441 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.441 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.441 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.442 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.442 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.442 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.442 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.442 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.442 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.442 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.442 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.443 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.443 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.443 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.443 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.443 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.443 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.443 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.443 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.443 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.444 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.444 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.444 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.444 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.444 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.444 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.444 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.444 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.444 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.444 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.445 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.445 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.445 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.445 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.445 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.445 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.445 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.445 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.445 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.446 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.446 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.446 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.446 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.446 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.446 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.446 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.446 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.447 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "telemetry", 2023-12-07T10:36:21.447 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.447 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.447 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.447 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_basic": { 2023-12-07T10:36:21.447 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_basic", 2023-12-07T10:36:21.447 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.447 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.447 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.448 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.448 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.448 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.448 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.448 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share basic cluster information (size, version)", 2023-12-07T10:36:21.448 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.448 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.448 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.448 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.449 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_crash": { 2023-12-07T10:36:21.449 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_crash", 2023-12-07T10:36:21.449 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.449 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.449 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.449 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.449 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.449 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.449 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.450 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-12-07T10:36:21.450 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.450 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.450 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.450 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.450 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_device": { 2023-12-07T10:36:21.450 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_device", 2023-12-07T10:36:21.450 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.450 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.451 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.451 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.451 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.451 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.451 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.451 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-12-07T10:36:21.451 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.451 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.452 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.452 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_ident": { 2023-12-07T10:36:21.452 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_ident", 2023-12-07T10:36:21.452 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.452 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.452 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.452 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.452 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.453 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.453 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.453 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-12-07T10:36:21.453 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.453 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.453 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.453 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_perf": { 2023-12-07T10:36:21.453 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_perf", 2023-12-07T10:36:21.453 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.454 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.454 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.454 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.454 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.454 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.454 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.454 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share various performance metrics of a cluster", 2023-12-07T10:36:21.454 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.454 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.455 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.455 INFO:teuthology.orchestra.run.smithi119.stdout: "contact": { 2023-12-07T10:36:21.455 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "contact", 2023-12-07T10:36:21.455 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.455 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.455 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.455 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.455 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.456 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.456 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.456 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.456 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.456 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.456 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.456 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.456 INFO:teuthology.orchestra.run.smithi119.stdout: "description": { 2023-12-07T10:36:21.456 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "description", 2023-12-07T10:36:21.457 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.457 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.457 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.457 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.457 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.457 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.457 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.457 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.457 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.458 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.458 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.458 INFO:teuthology.orchestra.run.smithi119.stdout: "device_url": { 2023-12-07T10:36:21.458 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "device_url", 2023-12-07T10:36:21.458 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.458 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.458 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.458 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-12-07T10:36:21.459 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.459 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.459 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.459 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.459 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.459 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.459 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.459 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.459 INFO:teuthology.orchestra.run.smithi119.stdout: "enabled": { 2023-12-07T10:36:21.460 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "enabled", 2023-12-07T10:36:21.460 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.460 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.460 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.460 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.460 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.460 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.460 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.460 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.461 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.461 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.461 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.461 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.461 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2023-12-07T10:36:21.461 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2023-12-07T10:36:21.461 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.461 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.462 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.462 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "24", 2023-12-07T10:36:21.462 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "8", 2023-12-07T10:36:21.462 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.462 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.462 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.462 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.462 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.462 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.463 INFO:teuthology.orchestra.run.smithi119.stdout: "last_opt_revision": { 2023-12-07T10:36:21.463 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "last_opt_revision", 2023-12-07T10:36:21.463 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.463 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.463 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.463 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "1", 2023-12-07T10:36:21.463 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.463 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.464 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.464 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.464 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.464 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.464 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.464 INFO:teuthology.orchestra.run.smithi119.stdout: "leaderboard": { 2023-12-07T10:36:21.464 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "leaderboard", 2023-12-07T10:36:21.464 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.465 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.465 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.465 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.465 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.465 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.465 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.465 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.465 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.465 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.466 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.466 INFO:teuthology.orchestra.run.smithi119.stdout: "leaderboard_description": { 2023-12-07T10:36:21.466 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "leaderboard_description", 2023-12-07T10:36:21.466 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.466 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.466 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.466 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.466 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.467 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.467 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.467 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.467 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.467 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.467 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.467 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.467 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.468 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.468 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.468 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.468 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.468 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.468 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.468 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.468 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.468 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.469 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.469 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.469 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.469 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.469 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.469 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.469 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.469 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.469 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.470 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.470 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.470 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.470 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.470 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.470 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.470 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.470 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.471 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.471 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.471 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.471 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.471 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.471 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.471 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.471 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.472 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.472 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.472 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.472 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.472 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.472 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.472 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.472 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.473 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.473 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.473 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.473 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.473 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.473 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.473 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.473 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.473 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.474 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.474 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.474 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.474 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.474 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.474 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.474 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.474 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.474 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.475 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.475 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.475 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.475 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.475 INFO:teuthology.orchestra.run.smithi119.stdout: "organization": { 2023-12-07T10:36:21.475 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "organization", 2023-12-07T10:36:21.475 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.475 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.476 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.476 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.476 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.476 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.476 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.476 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.476 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.476 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.477 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.477 INFO:teuthology.orchestra.run.smithi119.stdout: "proxy": { 2023-12-07T10:36:21.477 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "proxy", 2023-12-07T10:36:21.477 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.477 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.477 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.477 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.477 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.478 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.478 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.478 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.478 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.478 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.478 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.478 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.478 INFO:teuthology.orchestra.run.smithi119.stdout: "url": { 2023-12-07T10:36:21.478 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "url", 2023-12-07T10:36:21.479 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.479 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.479 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.479 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-12-07T10:36:21.479 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.479 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.479 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.479 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.479 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.479 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.480 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.480 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.480 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.480 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.480 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "test_orchestrator", 2023-12-07T10:36:21.480 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.480 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.480 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.481 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.481 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.481 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.481 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.481 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.481 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.481 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.481 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.481 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.481 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.482 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.482 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.482 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.482 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.482 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.482 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.482 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.482 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.482 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.483 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.483 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.483 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.483 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.483 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.483 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.483 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.483 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.484 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.484 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.484 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.484 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.484 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.484 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.484 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.484 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.484 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.485 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.485 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.485 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.485 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.485 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.485 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.485 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.485 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.485 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.486 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.486 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.486 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.486 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.486 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.486 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.486 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.486 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.486 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.487 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.487 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.487 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.487 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.487 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.487 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.487 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.487 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.487 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.488 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.488 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.488 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.488 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.488 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.488 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.488 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.489 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "volumes", 2023-12-07T10:36:21.489 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.489 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.489 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.489 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.489 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.489 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.489 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.490 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.490 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.490 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.490 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.490 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.490 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.490 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.490 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.490 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.491 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.491 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.491 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.491 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.491 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.491 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.491 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.491 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.492 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.492 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.492 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.492 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.492 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.492 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.492 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.492 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.492 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.493 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.493 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.493 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.493 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.493 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.493 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.493 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.493 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.494 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.494 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.494 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.494 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.494 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.494 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.494 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.494 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.495 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.495 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.495 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.495 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.495 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.495 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.495 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.495 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.495 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.496 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.496 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.496 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.496 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.496 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.496 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.496 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.496 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.496 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.497 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.497 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.497 INFO:teuthology.orchestra.run.smithi119.stdout: "max_concurrent_clones": { 2023-12-07T10:36:21.497 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_concurrent_clones", 2023-12-07T10:36:21.497 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.497 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.497 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.497 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "4", 2023-12-07T10:36:21.497 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.498 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.498 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.498 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Number of asynchronous cloner threads", 2023-12-07T10:36:21.498 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.498 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.498 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.498 INFO:teuthology.orchestra.run.smithi119.stdout: "periodic_async_work": { 2023-12-07T10:36:21.498 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "periodic_async_work", 2023-12-07T10:36:21.499 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.499 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.499 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.499 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.499 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.499 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.499 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.499 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Periodically check for async work", 2023-12-07T10:36:21.499 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.500 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.500 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.500 INFO:teuthology.orchestra.run.smithi119.stdout: "snapshot_clone_delay": { 2023-12-07T10:36:21.500 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "snapshot_clone_delay", 2023-12-07T10:36:21.500 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.500 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.500 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.500 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2023-12-07T10:36:21.501 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.501 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.501 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.501 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-12-07T10:36:21.501 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.501 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.501 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.501 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.502 INFO:teuthology.orchestra.run.smithi119.stdout: "snapshot_clone_no_wait": { 2023-12-07T10:36:21.502 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "snapshot_clone_no_wait", 2023-12-07T10:36:21.502 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.502 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.502 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.502 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.502 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.502 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.502 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.503 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2023-12-07T10:36:21.503 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.503 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.503 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.503 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.503 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.503 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.503 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "zabbix", 2023-12-07T10:36:21.504 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.504 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.504 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.504 INFO:teuthology.orchestra.run.smithi119.stdout: "discovery_interval": { 2023-12-07T10:36:21.504 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "discovery_interval", 2023-12-07T10:36:21.504 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2023-12-07T10:36:21.504 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.504 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.505 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "100", 2023-12-07T10:36:21.505 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.505 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.505 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.505 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.505 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.505 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.505 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.506 INFO:teuthology.orchestra.run.smithi119.stdout: "identifier": { 2023-12-07T10:36:21.506 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "identifier", 2023-12-07T10:36:21.506 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.506 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.506 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.506 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.506 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.506 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.507 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.507 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.507 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.507 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.507 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.507 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2023-12-07T10:36:21.507 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2023-12-07T10:36:21.507 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.508 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.508 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.508 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2023-12-07T10:36:21.508 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.508 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.508 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.508 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.508 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.508 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.509 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.509 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.509 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.509 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.509 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.509 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.509 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.509 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.510 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.510 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.510 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.510 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.510 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.510 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.510 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.510 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.510 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.511 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.511 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.511 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.511 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.511 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.511 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.511 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.511 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.512 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.512 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.512 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.512 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.512 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.512 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.512 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.512 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.512 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.513 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.513 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.513 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.513 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.513 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.513 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.513 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.513 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.513 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.514 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.514 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.514 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.514 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.514 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.514 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.514 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.514 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.515 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.515 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.515 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.515 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.515 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.515 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.515 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.515 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.516 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.516 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.516 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.516 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.516 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.516 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.516 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.516 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.517 INFO:teuthology.orchestra.run.smithi119.stdout: "zabbix_host": { 2023-12-07T10:36:21.517 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "zabbix_host", 2023-12-07T10:36:21.517 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.517 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.517 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.517 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.517 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.517 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.517 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.518 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.518 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.518 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.518 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.518 INFO:teuthology.orchestra.run.smithi119.stdout: "zabbix_port": { 2023-12-07T10:36:21.518 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "zabbix_port", 2023-12-07T10:36:21.518 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.518 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.519 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.519 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10051", 2023-12-07T10:36:21.519 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.519 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.519 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.519 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.519 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.519 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.520 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.520 INFO:teuthology.orchestra.run.smithi119.stdout: "zabbix_sender": { 2023-12-07T10:36:21.520 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "zabbix_sender", 2023-12-07T10:36:21.520 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.520 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.520 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.520 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-12-07T10:36:21.520 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.521 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.521 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.521 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.521 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.521 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.521 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.521 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.521 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.521 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.522 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-12-07T10:36:21.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.522 INFO:teuthology.orchestra.run.smithi119.stdout: "gid": 25609, 2023-12-07T10:36:21.522 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "y", 2023-12-07T10:36:21.522 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr_features": 4540138322906710015, 2023-12-07T10:36:21.522 INFO:teuthology.orchestra.run.smithi119.stdout: "available_modules": [ 2023-12-07T10:36:21.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.522 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "alerts", 2023-12-07T10:36:21.523 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.523 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.523 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.523 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2023-12-07T10:36:21.523 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2023-12-07T10:36:21.523 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.523 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.523 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.523 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2023-12-07T10:36:21.524 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.524 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.524 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.524 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "How frequently to reexamine health status", 2023-12-07T10:36:21.524 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.524 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.524 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.524 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.525 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.525 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.525 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.525 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.525 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.525 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.525 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.525 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.525 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.525 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.526 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.526 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.526 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.526 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.526 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.526 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.526 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.529 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.530 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.530 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.530 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.530 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.530 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.530 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.530 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.531 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.531 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.531 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.531 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.531 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.531 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.531 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.531 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.532 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.532 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.532 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.532 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.532 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.532 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.532 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.532 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.532 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.533 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.533 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.533 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.533 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.533 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.533 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.533 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.533 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.533 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.534 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.534 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.534 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.534 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.534 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.534 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.534 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.534 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.535 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.535 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.535 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.535 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.535 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.535 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.535 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_destination": { 2023-12-07T10:36:21.535 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_destination", 2023-12-07T10:36:21.536 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.536 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.536 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.536 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.536 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.536 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.536 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.536 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Email address to send alerts to", 2023-12-07T10:36:21.536 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.537 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.537 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.537 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_from_name": { 2023-12-07T10:36:21.537 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_from_name", 2023-12-07T10:36:21.537 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.537 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.537 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.537 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "Ceph", 2023-12-07T10:36:21.538 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.538 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.538 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.538 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Email From: name", 2023-12-07T10:36:21.538 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.538 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.538 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.539 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_host": { 2023-12-07T10:36:21.539 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_host", 2023-12-07T10:36:21.539 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.539 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.539 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.539 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.539 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.539 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.539 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.540 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "SMTP server", 2023-12-07T10:36:21.540 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.540 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.540 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.540 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_password": { 2023-12-07T10:36:21.540 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_password", 2023-12-07T10:36:21.540 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.540 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.541 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.541 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.541 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.541 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.541 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.541 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Password to authenticate with", 2023-12-07T10:36:21.541 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.541 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.541 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.542 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_port": { 2023-12-07T10:36:21.542 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_port", 2023-12-07T10:36:21.542 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.542 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.542 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.542 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "465", 2023-12-07T10:36:21.542 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.543 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.543 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.543 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "SMTP port", 2023-12-07T10:36:21.543 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.543 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.543 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.543 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_sender": { 2023-12-07T10:36:21.543 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_sender", 2023-12-07T10:36:21.544 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.544 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.544 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.544 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.544 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.544 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.544 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.544 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "SMTP envelope sender", 2023-12-07T10:36:21.544 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.545 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.545 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.545 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_ssl": { 2023-12-07T10:36:21.545 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_ssl", 2023-12-07T10:36:21.545 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.545 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.545 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.545 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.546 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.546 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.546 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.546 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Use SSL to connect to SMTP server", 2023-12-07T10:36:21.546 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.546 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.546 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.546 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_user": { 2023-12-07T10:36:21.547 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_user", 2023-12-07T10:36:21.547 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.547 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.547 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.547 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.547 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.547 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.547 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.547 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "User to authenticate as", 2023-12-07T10:36:21.548 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.548 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.548 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.548 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.548 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.548 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "balancer", 2023-12-07T10:36:21.549 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.549 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.549 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.549 INFO:teuthology.orchestra.run.smithi119.stdout: "active": { 2023-12-07T10:36:21.549 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "active", 2023-12-07T10:36:21.549 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.549 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.549 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.549 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.550 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.550 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.550 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.550 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "automatically balance PGs across cluster", 2023-12-07T10:36:21.550 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.550 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.550 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.551 INFO:teuthology.orchestra.run.smithi119.stdout: "begin_time": { 2023-12-07T10:36:21.551 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "begin_time", 2023-12-07T10:36:21.551 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.551 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.551 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.551 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0000", 2023-12-07T10:36:21.551 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.552 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.552 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.552 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "beginning time of day to automatically balance", 2023-12-07T10:36:21.552 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-07T10:36:21.552 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.552 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.552 INFO:teuthology.orchestra.run.smithi119.stdout: "begin_weekday": { 2023-12-07T10:36:21.552 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "begin_weekday", 2023-12-07T10:36:21.553 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2023-12-07T10:36:21.553 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.553 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.553 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2023-12-07T10:36:21.553 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "0", 2023-12-07T10:36:21.553 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "6", 2023-12-07T10:36:21.553 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.553 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-12-07T10:36:21.553 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-07T10:36:21.554 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.554 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.554 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_compat_max_iterations": { 2023-12-07T10:36:21.554 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crush_compat_max_iterations", 2023-12-07T10:36:21.554 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2023-12-07T10:36:21.554 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.554 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.554 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "25", 2023-12-07T10:36:21.555 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1", 2023-12-07T10:36:21.555 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "250", 2023-12-07T10:36:21.555 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.555 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-12-07T10:36:21.555 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.555 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.555 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.555 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_compat_metrics": { 2023-12-07T10:36:21.556 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crush_compat_metrics", 2023-12-07T10:36:21.556 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.556 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.556 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.556 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "pgs,objects,bytes", 2023-12-07T10:36:21.556 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.556 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.556 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.557 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-12-07T10:36:21.557 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-12-07T10:36:21.557 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.557 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.557 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_compat_step": { 2023-12-07T10:36:21.557 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crush_compat_step", 2023-12-07T10:36:21.557 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2023-12-07T10:36:21.557 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.558 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.558 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0.5", 2023-12-07T10:36:21.558 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "0.001", 2023-12-07T10:36:21.558 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "0.999", 2023-12-07T10:36:21.558 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.558 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "aggressiveness of optimization", 2023-12-07T10:36:21.558 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-12-07T10:36:21.558 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.558 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.559 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.559 INFO:teuthology.orchestra.run.smithi119.stdout: "end_time": { 2023-12-07T10:36:21.559 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "end_time", 2023-12-07T10:36:21.559 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.559 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.559 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.559 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "2359", 2023-12-07T10:36:21.559 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.559 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.560 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.560 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "ending time of day to automatically balance", 2023-12-07T10:36:21.560 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-07T10:36:21.560 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.560 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.560 INFO:teuthology.orchestra.run.smithi119.stdout: "end_weekday": { 2023-12-07T10:36:21.560 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "end_weekday", 2023-12-07T10:36:21.561 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2023-12-07T10:36:21.561 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.561 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.561 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2023-12-07T10:36:21.561 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "0", 2023-12-07T10:36:21.561 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "6", 2023-12-07T10:36:21.561 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.561 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-12-07T10:36:21.561 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-07T10:36:21.562 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.562 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.562 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.562 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.562 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.562 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.562 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.562 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.563 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.563 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.563 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.563 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.563 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.563 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.563 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.563 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.563 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.564 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.564 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.564 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.564 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.564 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.564 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.564 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.564 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.565 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.565 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.565 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.565 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.565 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.565 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.565 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.565 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.565 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.566 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.566 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.566 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.566 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.566 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.566 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.566 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.566 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.567 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.567 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.567 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.567 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.567 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.567 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.567 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.567 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.567 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.568 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.568 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.568 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.568 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.568 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.568 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.568 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.568 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.569 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.569 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.569 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.569 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.569 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.569 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.569 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.569 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.569 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.570 INFO:teuthology.orchestra.run.smithi119.stdout: "min_score": { 2023-12-07T10:36:21.570 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "min_score", 2023-12-07T10:36:21.570 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2023-12-07T10:36:21.570 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.570 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.570 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2023-12-07T10:36:21.570 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.570 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.571 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.571 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-12-07T10:36:21.571 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.571 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.571 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.571 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": { 2023-12-07T10:36:21.571 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mode", 2023-12-07T10:36:21.571 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.572 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.572 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.572 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "upmap", 2023-12-07T10:36:21.572 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.572 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.572 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.572 INFO:teuthology.orchestra.run.smithi119.stdout: "crush-compat", 2023-12-07T10:36:21.572 INFO:teuthology.orchestra.run.smithi119.stdout: "none", 2023-12-07T10:36:21.572 INFO:teuthology.orchestra.run.smithi119.stdout: "upmap" 2023-12-07T10:36:21.572 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.573 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Balancer mode", 2023-12-07T10:36:21.573 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.573 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.573 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.573 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.573 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_ids": { 2023-12-07T10:36:21.573 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "pool_ids", 2023-12-07T10:36:21.573 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.574 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.574 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.574 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.574 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.574 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.574 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.574 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-12-07T10:36:21.574 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.574 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.575 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.575 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.575 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2023-12-07T10:36:21.575 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2023-12-07T10:36:21.575 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.575 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.575 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.575 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2023-12-07T10:36:21.575 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.576 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.576 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.576 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-12-07T10:36:21.576 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.576 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.576 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.576 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.576 INFO:teuthology.orchestra.run.smithi119.stdout: "upmap_max_deviation": { 2023-12-07T10:36:21.577 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "upmap_max_deviation", 2023-12-07T10:36:21.577 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.577 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.577 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.577 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5", 2023-12-07T10:36:21.577 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1", 2023-12-07T10:36:21.577 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.577 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.577 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "deviation below which no optimization is attempted", 2023-12-07T10:36:21.578 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-12-07T10:36:21.578 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.578 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.578 INFO:teuthology.orchestra.run.smithi119.stdout: "upmap_max_optimizations": { 2023-12-07T10:36:21.578 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "upmap_max_optimizations", 2023-12-07T10:36:21.578 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2023-12-07T10:36:21.578 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.578 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.579 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2023-12-07T10:36:21.579 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.579 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.579 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.579 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-12-07T10:36:21.579 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.579 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.579 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.579 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.580 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.580 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cephadm", 2023-12-07T10:36:21.580 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.580 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.580 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.580 INFO:teuthology.orchestra.run.smithi119.stdout: "agent_down_multiplier": { 2023-12-07T10:36:21.580 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "agent_down_multiplier", 2023-12-07T10:36:21.581 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2023-12-07T10:36:21.581 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.581 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.581 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "3.0", 2023-12-07T10:36:21.581 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.581 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.581 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.581 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-12-07T10:36:21.582 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.582 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.582 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.582 INFO:teuthology.orchestra.run.smithi119.stdout: "agent_refresh_rate": { 2023-12-07T10:36:21.582 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "agent_refresh_rate", 2023-12-07T10:36:21.582 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.582 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.582 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.583 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "20", 2023-12-07T10:36:21.583 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.583 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.583 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.583 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-12-07T10:36:21.583 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.583 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.583 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.584 INFO:teuthology.orchestra.run.smithi119.stdout: "agent_starting_port": { 2023-12-07T10:36:21.584 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "agent_starting_port", 2023-12-07T10:36:21.584 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.584 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.584 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.584 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "4721", 2023-12-07T10:36:21.584 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.584 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.585 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.585 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-12-07T10:36:21.585 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.585 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.585 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.585 INFO:teuthology.orchestra.run.smithi119.stdout: "allow_ptrace": { 2023-12-07T10:36:21.585 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "allow_ptrace", 2023-12-07T10:36:21.585 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.585 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.586 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.586 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.586 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.586 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.586 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.586 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-12-07T10:36:21.586 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-12-07T10:36:21.586 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.587 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.587 INFO:teuthology.orchestra.run.smithi119.stdout: "autotune_interval": { 2023-12-07T10:36:21.587 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "autotune_interval", 2023-12-07T10:36:21.587 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.587 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.587 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.587 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2023-12-07T10:36:21.587 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.588 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.588 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.588 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to autotune daemon memory", 2023-12-07T10:36:21.588 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.588 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.588 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.588 INFO:teuthology.orchestra.run.smithi119.stdout: "autotune_memory_target_ratio": { 2023-12-07T10:36:21.589 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "autotune_memory_target_ratio", 2023-12-07T10:36:21.589 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2023-12-07T10:36:21.589 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.589 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.589 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0.7", 2023-12-07T10:36:21.589 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.589 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.589 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.589 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-12-07T10:36:21.590 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.590 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.590 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.590 INFO:teuthology.orchestra.run.smithi119.stdout: "cephadm_log_destination": { 2023-12-07T10:36:21.590 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cephadm_log_destination", 2023-12-07T10:36:21.590 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.590 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.591 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.591 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.591 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.591 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.591 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.591 INFO:teuthology.orchestra.run.smithi119.stdout: "file", 2023-12-07T10:36:21.591 INFO:teuthology.orchestra.run.smithi119.stdout: "file,syslog", 2023-12-07T10:36:21.591 INFO:teuthology.orchestra.run.smithi119.stdout: "syslog" 2023-12-07T10:36:21.591 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.592 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-12-07T10:36:21.592 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.592 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.592 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.592 INFO:teuthology.orchestra.run.smithi119.stdout: "cgroups_split": { 2023-12-07T10:36:21.592 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cgroups_split", 2023-12-07T10:36:21.592 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.592 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.593 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.593 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.593 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.593 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.593 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.593 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-12-07T10:36:21.593 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.593 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.593 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.594 INFO:teuthology.orchestra.run.smithi119.stdout: "config_checks_enabled": { 2023-12-07T10:36:21.594 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "config_checks_enabled", 2023-12-07T10:36:21.594 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.594 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.594 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.594 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.594 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.595 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.595 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.595 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-12-07T10:36:21.595 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.595 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.595 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.595 INFO:teuthology.orchestra.run.smithi119.stdout: "config_dashboard": { 2023-12-07T10:36:21.595 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "config_dashboard", 2023-12-07T10:36:21.596 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.596 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.596 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.596 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.596 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.596 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.596 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.596 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-12-07T10:36:21.596 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.597 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.597 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.597 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_alertmanager": { 2023-12-07T10:36:21.597 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_alertmanager", 2023-12-07T10:36:21.597 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.597 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.597 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.597 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-12-07T10:36:21.598 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.598 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.598 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.598 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Prometheus container image", 2023-12-07T10:36:21.598 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.598 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.598 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.598 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_base": { 2023-12-07T10:36:21.599 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_base", 2023-12-07T10:36:21.599 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.599 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.599 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.599 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/ceph", 2023-12-07T10:36:21.599 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.599 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.599 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.599 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Container image name, without the tag", 2023-12-07T10:36:21.600 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.600 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.600 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.600 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_elasticsearch": { 2023-12-07T10:36:21.600 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_elasticsearch", 2023-12-07T10:36:21.600 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.600 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.601 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.601 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-12-07T10:36:21.601 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.601 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.601 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.601 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "elasticsearch container image", 2023-12-07T10:36:21.601 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.601 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.601 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.602 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_grafana": { 2023-12-07T10:36:21.602 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_grafana", 2023-12-07T10:36:21.602 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.602 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.602 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.602 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-12-07T10:36:21.602 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.603 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.603 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.603 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Prometheus container image", 2023-12-07T10:36:21.603 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.603 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.603 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.603 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.603 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_haproxy": { 2023-12-07T10:36:21.603 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_haproxy", 2023-12-07T10:36:21.603 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.604 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.604 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.604 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-12-07T10:36:21.604 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.604 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.604 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.604 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "HAproxy container image", 2023-12-07T10:36:21.604 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.604 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.605 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.605 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.605 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_jaeger_agent": { 2023-12-07T10:36:21.605 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_jaeger_agent", 2023-12-07T10:36:21.605 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.605 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.605 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.605 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-12-07T10:36:21.606 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.606 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.606 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.606 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Jaeger agent container image", 2023-12-07T10:36:21.606 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.606 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.606 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.606 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_jaeger_collector": { 2023-12-07T10:36:21.607 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_jaeger_collector", 2023-12-07T10:36:21.607 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.607 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.607 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.607 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-12-07T10:36:21.607 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.607 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.607 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.607 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Jaeger collector container image", 2023-12-07T10:36:21.608 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.608 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.608 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.608 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_jaeger_query": { 2023-12-07T10:36:21.608 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_jaeger_query", 2023-12-07T10:36:21.608 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.608 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.608 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.609 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-12-07T10:36:21.609 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.609 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.609 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.609 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Jaeger query container image", 2023-12-07T10:36:21.609 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.609 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.609 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.610 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_keepalived": { 2023-12-07T10:36:21.610 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_keepalived", 2023-12-07T10:36:21.610 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.610 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.610 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.610 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-12-07T10:36:21.610 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.610 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.611 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.611 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Keepalived container image", 2023-12-07T10:36:21.611 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.611 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.611 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.611 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.611 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_loki": { 2023-12-07T10:36:21.611 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_loki", 2023-12-07T10:36:21.612 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.612 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.612 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.612 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-12-07T10:36:21.612 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.612 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.612 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.612 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Loki container image", 2023-12-07T10:36:21.612 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.613 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.613 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.613 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.613 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_node_exporter": { 2023-12-07T10:36:21.613 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_node_exporter", 2023-12-07T10:36:21.613 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.613 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.613 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.613 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-12-07T10:36:21.614 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.614 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.614 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.614 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Prometheus container image", 2023-12-07T10:36:21.614 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.614 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.614 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.614 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.614 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_nvmeof": { 2023-12-07T10:36:21.615 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_nvmeof", 2023-12-07T10:36:21.615 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.615 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.615 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.615 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2023-12-07T10:36:21.615 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.615 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.615 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.615 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Nvme-of container image", 2023-12-07T10:36:21.616 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.616 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.616 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.616 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_prometheus": { 2023-12-07T10:36:21.616 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_prometheus", 2023-12-07T10:36:21.616 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.616 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.616 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.617 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-12-07T10:36:21.617 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.617 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.617 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.617 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Prometheus container image", 2023-12-07T10:36:21.617 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.617 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.617 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.618 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.618 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_promtail": { 2023-12-07T10:36:21.618 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_promtail", 2023-12-07T10:36:21.618 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.618 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.618 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.618 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-12-07T10:36:21.618 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.619 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.619 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.619 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Promtail container image", 2023-12-07T10:36:21.619 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.619 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.619 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.619 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_snmp_gateway": { 2023-12-07T10:36:21.619 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_snmp_gateway", 2023-12-07T10:36:21.619 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.620 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.620 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.620 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-12-07T10:36:21.620 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.620 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.620 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.620 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "SNMP Gateway container image", 2023-12-07T10:36:21.620 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.620 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.620 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.621 INFO:teuthology.orchestra.run.smithi119.stdout: "container_init": { 2023-12-07T10:36:21.621 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_init", 2023-12-07T10:36:21.621 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.621 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.621 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.621 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.621 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.622 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.622 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.622 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Run podman/docker with `--init`", 2023-12-07T10:36:21.622 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.622 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.622 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.622 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.622 INFO:teuthology.orchestra.run.smithi119.stdout: "daemon_cache_timeout": { 2023-12-07T10:36:21.622 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "daemon_cache_timeout", 2023-12-07T10:36:21.623 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.623 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.623 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.623 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2023-12-07T10:36:21.623 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.623 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.623 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.623 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-12-07T10:36:21.623 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.624 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.624 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.624 INFO:teuthology.orchestra.run.smithi119.stdout: "default_cephadm_command_timeout": { 2023-12-07T10:36:21.624 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "default_cephadm_command_timeout", 2023-12-07T10:36:21.624 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.624 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.624 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.624 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "900", 2023-12-07T10:36:21.625 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.625 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.625 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.625 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-12-07T10:36:21.625 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.625 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.625 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.625 INFO:teuthology.orchestra.run.smithi119.stdout: "default_registry": { 2023-12-07T10:36:21.625 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "default_registry", 2023-12-07T10:36:21.626 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.626 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.626 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.626 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "docker.io", 2023-12-07T10:36:21.626 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.626 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.626 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.626 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-12-07T10:36:21.627 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.627 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.627 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.627 INFO:teuthology.orchestra.run.smithi119.stdout: "device_cache_timeout": { 2023-12-07T10:36:21.627 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "device_cache_timeout", 2023-12-07T10:36:21.627 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.627 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.627 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.628 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "1800", 2023-12-07T10:36:21.628 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.628 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.628 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.628 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "seconds to cache device inventory", 2023-12-07T10:36:21.628 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.628 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.628 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.629 INFO:teuthology.orchestra.run.smithi119.stdout: "device_enhanced_scan": { 2023-12-07T10:36:21.629 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "device_enhanced_scan", 2023-12-07T10:36:21.629 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.629 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.629 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.629 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.629 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.629 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.629 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.630 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Use libstoragemgmt during device scans", 2023-12-07T10:36:21.630 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.630 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.630 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.630 INFO:teuthology.orchestra.run.smithi119.stdout: "facts_cache_timeout": { 2023-12-07T10:36:21.630 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "facts_cache_timeout", 2023-12-07T10:36:21.630 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.631 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.631 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.631 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2023-12-07T10:36:21.631 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.631 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.631 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.631 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "seconds to cache host facts data", 2023-12-07T10:36:21.631 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.631 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.632 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.632 INFO:teuthology.orchestra.run.smithi119.stdout: "host_check_interval": { 2023-12-07T10:36:21.632 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "host_check_interval", 2023-12-07T10:36:21.632 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.632 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.632 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.632 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2023-12-07T10:36:21.632 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.633 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.633 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.633 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to perform a host check", 2023-12-07T10:36:21.633 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.633 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.633 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.633 INFO:teuthology.orchestra.run.smithi119.stdout: "inventory_list_all": { 2023-12-07T10:36:21.633 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "inventory_list_all", 2023-12-07T10:36:21.634 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.634 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.634 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.634 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.634 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.634 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.634 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.634 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-12-07T10:36:21.634 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.635 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.635 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.635 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.635 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.635 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.635 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.635 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.635 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.636 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.636 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.636 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.636 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.636 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.636 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.636 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.636 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.636 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.637 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.637 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.637 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.637 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.637 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.637 INFO:teuthology.orchestra.run.smithi119.stdout: "log_refresh_metadata": { 2023-12-07T10:36:21.637 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_refresh_metadata", 2023-12-07T10:36:21.637 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.638 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.638 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.638 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.638 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.638 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.638 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.638 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-12-07T10:36:21.638 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.639 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.639 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.639 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.639 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.639 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.639 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.639 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.639 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.640 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.640 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.640 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.640 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-12-07T10:36:21.640 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.640 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.640 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.640 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.641 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.641 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.641 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.641 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.641 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.641 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.641 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.641 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.641 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.642 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.642 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.642 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.642 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.642 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.642 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.642 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.642 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.642 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.643 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.643 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.643 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.643 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.643 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.643 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.643 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.644 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.644 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.644 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.644 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.644 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.644 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.644 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.644 INFO:teuthology.orchestra.run.smithi119.stdout: "manage_etc_ceph_ceph_conf": { 2023-12-07T10:36:21.644 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-12-07T10:36:21.645 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.645 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.645 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.645 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.645 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.645 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.645 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.645 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-12-07T10:36:21.645 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.646 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.646 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.646 INFO:teuthology.orchestra.run.smithi119.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-12-07T10:36:21.646 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-12-07T10:36:21.646 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.646 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.646 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.646 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "*", 2023-12-07T10:36:21.647 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.647 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.647 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.647 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-12-07T10:36:21.647 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.647 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.647 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.648 INFO:teuthology.orchestra.run.smithi119.stdout: "max_count_per_host": { 2023-12-07T10:36:21.648 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_count_per_host", 2023-12-07T10:36:21.648 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.648 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.648 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.648 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2023-12-07T10:36:21.648 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.648 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.648 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.649 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "max number of daemons per service per host", 2023-12-07T10:36:21.649 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.649 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.649 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.649 INFO:teuthology.orchestra.run.smithi119.stdout: "max_osd_draining_count": { 2023-12-07T10:36:21.649 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_osd_draining_count", 2023-12-07T10:36:21.649 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.649 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.650 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.650 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2023-12-07T10:36:21.650 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.650 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.650 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.650 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-12-07T10:36:21.650 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.650 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.650 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.651 INFO:teuthology.orchestra.run.smithi119.stdout: "migration_current": { 2023-12-07T10:36:21.651 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "migration_current", 2023-12-07T10:36:21.651 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.651 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.651 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.651 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.651 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.651 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.652 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.652 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "internal - do not modify", 2023-12-07T10:36:21.652 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.652 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.652 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.652 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": { 2023-12-07T10:36:21.652 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mode", 2023-12-07T10:36:21.652 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.652 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.653 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.653 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "root", 2023-12-07T10:36:21.653 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.653 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.653 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.653 INFO:teuthology.orchestra.run.smithi119.stdout: "cephadm-package", 2023-12-07T10:36:21.653 INFO:teuthology.orchestra.run.smithi119.stdout: "root" 2023-12-07T10:36:21.653 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.653 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "mode for remote execution of cephadm", 2023-12-07T10:36:21.654 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.654 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.654 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.654 INFO:teuthology.orchestra.run.smithi119.stdout: "prometheus_alerts_path": { 2023-12-07T10:36:21.654 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "prometheus_alerts_path", 2023-12-07T10:36:21.654 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.654 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.655 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.655 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-12-07T10:36:21.655 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.655 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.655 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.655 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-12-07T10:36:21.655 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.655 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.655 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.656 INFO:teuthology.orchestra.run.smithi119.stdout: "registry_insecure": { 2023-12-07T10:36:21.656 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "registry_insecure", 2023-12-07T10:36:21.656 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.656 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.656 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.656 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.656 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.657 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.657 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.657 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-12-07T10:36:21.657 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.657 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.657 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.657 INFO:teuthology.orchestra.run.smithi119.stdout: "registry_password": { 2023-12-07T10:36:21.657 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "registry_password", 2023-12-07T10:36:21.657 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.658 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.658 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.658 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.658 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.658 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.658 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.658 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-12-07T10:36:21.658 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.659 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.659 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.659 INFO:teuthology.orchestra.run.smithi119.stdout: "registry_url": { 2023-12-07T10:36:21.659 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "registry_url", 2023-12-07T10:36:21.659 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.659 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.659 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.659 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.660 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.660 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.660 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.660 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-12-07T10:36:21.660 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.660 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.660 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.660 INFO:teuthology.orchestra.run.smithi119.stdout: "registry_username": { 2023-12-07T10:36:21.661 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "registry_username", 2023-12-07T10:36:21.661 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.661 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.661 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.661 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.661 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.661 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.661 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.661 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-12-07T10:36:21.662 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.662 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.662 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.662 INFO:teuthology.orchestra.run.smithi119.stdout: "secure_monitoring_stack": { 2023-12-07T10:36:21.662 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "secure_monitoring_stack", 2023-12-07T10:36:21.662 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.662 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.662 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.663 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.663 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.663 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.663 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.663 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-12-07T10:36:21.663 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.663 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.663 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.664 INFO:teuthology.orchestra.run.smithi119.stdout: "service_discovery_port": { 2023-12-07T10:36:21.664 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "service_discovery_port", 2023-12-07T10:36:21.664 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.664 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.664 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.664 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8765", 2023-12-07T10:36:21.664 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.664 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.664 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.665 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "cephadm service discovery port", 2023-12-07T10:36:21.665 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.665 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.665 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.665 INFO:teuthology.orchestra.run.smithi119.stdout: "ssh_config_file": { 2023-12-07T10:36:21.665 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ssh_config_file", 2023-12-07T10:36:21.665 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.666 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.666 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.666 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.666 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.666 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.666 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.666 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-12-07T10:36:21.666 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.666 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.667 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.667 INFO:teuthology.orchestra.run.smithi119.stdout: "use_agent": { 2023-12-07T10:36:21.667 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "use_agent", 2023-12-07T10:36:21.667 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.667 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.667 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.667 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.668 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.668 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.668 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.668 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-12-07T10:36:21.668 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.668 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.668 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.668 INFO:teuthology.orchestra.run.smithi119.stdout: "use_repo_digest": { 2023-12-07T10:36:21.669 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "use_repo_digest", 2023-12-07T10:36:21.669 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.669 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.669 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.669 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.669 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.669 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.669 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.669 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-12-07T10:36:21.670 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.670 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.670 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.670 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_on_failed_host_check": { 2023-12-07T10:36:21.670 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_on_failed_host_check", 2023-12-07T10:36:21.670 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.670 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.671 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.671 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.671 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.671 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.671 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.671 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "raise a health warning if the host check fails", 2023-12-07T10:36:21.671 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.671 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.671 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.672 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_on_stray_daemons": { 2023-12-07T10:36:21.672 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_on_stray_daemons", 2023-12-07T10:36:21.672 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.672 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.672 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.672 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.672 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.673 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.673 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.673 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-12-07T10:36:21.673 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.673 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.673 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.673 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_on_stray_hosts": { 2023-12-07T10:36:21.673 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_on_stray_hosts", 2023-12-07T10:36:21.674 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.674 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.674 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.674 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.674 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.674 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.674 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.674 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-12-07T10:36:21.674 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.675 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.675 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.675 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.675 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.675 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crash", 2023-12-07T10:36:21.675 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.675 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.676 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.676 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.676 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.676 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.676 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.676 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.676 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.676 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.676 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.677 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.677 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.677 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.677 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.677 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.677 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.677 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.677 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.678 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.678 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.678 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.678 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.678 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.678 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.678 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.678 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.679 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.679 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.679 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.679 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.679 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.679 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.679 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.679 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.679 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.680 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.680 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.680 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.680 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.680 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.680 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.680 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.681 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.681 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.681 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.681 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.681 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.681 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.681 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.681 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.681 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.682 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.682 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.682 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.682 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.682 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.682 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.682 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.683 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.683 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.683 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.683 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.683 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.683 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.683 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.683 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.683 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.684 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.684 INFO:teuthology.orchestra.run.smithi119.stdout: "retain_interval": { 2023-12-07T10:36:21.684 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "retain_interval", 2023-12-07T10:36:21.684 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.684 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.684 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.684 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "31536000", 2023-12-07T10:36:21.684 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.684 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.685 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.685 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how long to retain crashes before pruning them", 2023-12-07T10:36:21.685 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.685 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.685 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.685 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_recent_interval": { 2023-12-07T10:36:21.685 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_recent_interval", 2023-12-07T10:36:21.686 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.686 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.686 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.686 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "1209600", 2023-12-07T10:36:21.686 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.686 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.686 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.686 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "time interval in which to warn about recent crashes", 2023-12-07T10:36:21.687 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.687 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.687 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.687 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.687 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.687 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "dashboard", 2023-12-07T10:36:21.687 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.688 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.688 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.688 INFO:teuthology.orchestra.run.smithi119.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-12-07T10:36:21.688 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-12-07T10:36:21.688 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.688 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.688 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.688 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2023-12-07T10:36:21.689 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.689 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.689 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.689 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.689 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.689 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.689 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.689 INFO:teuthology.orchestra.run.smithi119.stdout: "ALERTMANAGER_API_HOST": { 2023-12-07T10:36:21.690 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ALERTMANAGER_API_HOST", 2023-12-07T10:36:21.690 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.690 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.690 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.690 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.690 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.690 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.690 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.690 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.691 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.691 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.691 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.691 INFO:teuthology.orchestra.run.smithi119.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-12-07T10:36:21.691 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-12-07T10:36:21.691 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.691 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.692 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.692 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.692 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.692 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.692 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.692 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.692 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.692 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.692 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.693 INFO:teuthology.orchestra.run.smithi119.stdout: "AUDIT_API_ENABLED": { 2023-12-07T10:36:21.693 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "AUDIT_API_ENABLED", 2023-12-07T10:36:21.693 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.693 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.693 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.693 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.693 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.693 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.694 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.694 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.694 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.694 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.694 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.694 INFO:teuthology.orchestra.run.smithi119.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-12-07T10:36:21.694 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-12-07T10:36:21.694 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.695 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.695 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.695 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.695 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.695 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.695 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.695 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.695 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.695 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.696 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.696 INFO:teuthology.orchestra.run.smithi119.stdout: "ENABLE_BROWSABLE_API": { 2023-12-07T10:36:21.696 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ENABLE_BROWSABLE_API", 2023-12-07T10:36:21.696 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.696 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.696 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.696 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.697 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.697 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.697 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.697 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.697 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.697 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.697 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.697 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.697 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-12-07T10:36:21.698 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-12-07T10:36:21.698 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.698 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.698 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.698 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.698 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.698 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.698 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.698 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.699 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.699 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.699 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.699 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.699 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-12-07T10:36:21.699 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-12-07T10:36:21.699 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.699 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.700 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.700 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.700 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.700 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.700 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.700 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.700 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.700 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.700 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.701 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-12-07T10:36:21.701 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-12-07T10:36:21.701 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.701 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.701 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.701 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.701 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.701 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.702 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.702 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.702 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.702 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.702 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.702 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-12-07T10:36:21.702 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-12-07T10:36:21.703 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.703 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.703 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.703 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.703 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.703 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.703 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.703 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.703 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.704 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.704 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.704 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_NFS": { 2023-12-07T10:36:21.704 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-12-07T10:36:21.704 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.704 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.704 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.704 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.705 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.705 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.705 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.705 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.705 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.705 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.705 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.705 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_RBD": { 2023-12-07T10:36:21.706 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-12-07T10:36:21.706 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.706 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.706 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.706 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.706 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.706 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.706 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.707 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.707 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.707 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.707 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.707 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_RGW": { 2023-12-07T10:36:21.707 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-12-07T10:36:21.707 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.707 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.708 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.708 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.708 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.708 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.708 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.708 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.708 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.708 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.708 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.709 INFO:teuthology.orchestra.run.smithi119.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-12-07T10:36:21.709 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-12-07T10:36:21.709 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.709 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.709 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.709 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.709 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.709 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.710 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.710 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.710 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.710 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.710 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.710 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_API_PASSWORD": { 2023-12-07T10:36:21.710 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_API_PASSWORD", 2023-12-07T10:36:21.711 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.711 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.711 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.711 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "admin", 2023-12-07T10:36:21.711 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.711 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.711 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.711 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.711 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.712 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.712 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.712 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-12-07T10:36:21.712 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-12-07T10:36:21.712 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.712 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.712 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.713 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.713 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.713 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.713 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.713 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.713 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.713 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.713 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.714 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_API_URL": { 2023-12-07T10:36:21.714 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_API_URL", 2023-12-07T10:36:21.714 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.714 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.714 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.714 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.714 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.714 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.714 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.715 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.715 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.715 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.715 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.715 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_API_USERNAME": { 2023-12-07T10:36:21.715 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_API_USERNAME", 2023-12-07T10:36:21.715 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.715 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.716 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.716 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "admin", 2023-12-07T10:36:21.716 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.716 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.716 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.716 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.716 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.716 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.716 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.717 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-12-07T10:36:21.717 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-12-07T10:36:21.717 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.717 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.717 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.717 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.717 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.717 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.718 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.718 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.718 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.718 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.718 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.718 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-12-07T10:36:21.718 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-12-07T10:36:21.718 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.719 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.719 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.719 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.719 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.719 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.719 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.719 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.719 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.719 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.720 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.720 INFO:teuthology.orchestra.run.smithi119.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-12-07T10:36:21.720 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-12-07T10:36:21.720 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.720 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.720 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.720 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.720 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.721 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.721 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.721 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.721 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.721 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.721 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.721 INFO:teuthology.orchestra.run.smithi119.stdout: "ISSUE_TRACKER_API_KEY": { 2023-12-07T10:36:21.721 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-12-07T10:36:21.722 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.722 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.722 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.722 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.722 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.722 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.722 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.722 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.723 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.723 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.723 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.723 INFO:teuthology.orchestra.run.smithi119.stdout: "PROMETHEUS_API_HOST": { 2023-12-07T10:36:21.723 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PROMETHEUS_API_HOST", 2023-12-07T10:36:21.723 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.723 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.724 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.724 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.724 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.724 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.724 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.724 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.724 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.724 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.724 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.725 INFO:teuthology.orchestra.run.smithi119.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-12-07T10:36:21.725 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-12-07T10:36:21.725 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.725 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.725 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.725 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.725 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.725 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.726 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.726 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.726 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.726 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.726 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.726 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-12-07T10:36:21.726 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-12-07T10:36:21.727 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.727 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.727 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.727 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.727 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.727 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.727 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.727 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.727 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.728 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.728 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.728 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-12-07T10:36:21.728 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-12-07T10:36:21.728 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.728 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.728 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.728 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.729 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.729 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.729 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.729 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.729 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.729 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.729 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.730 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-12-07T10:36:21.730 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-12-07T10:36:21.730 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.730 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.730 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.730 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.730 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.733 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.733 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.733 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.733 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.733 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.733 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.734 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-12-07T10:36:21.734 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-12-07T10:36:21.734 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.734 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.734 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.734 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.734 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.735 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.735 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.735 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.735 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.735 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.735 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.735 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-12-07T10:36:21.736 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-12-07T10:36:21.736 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.736 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.736 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.736 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.736 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.736 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.736 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.736 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.737 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.737 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.737 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.737 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-12-07T10:36:21.737 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-12-07T10:36:21.737 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.737 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.738 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.738 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.738 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.738 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.738 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.738 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.738 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.738 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.738 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.739 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.739 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-12-07T10:36:21.739 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-12-07T10:36:21.739 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.739 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.739 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.739 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.739 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.739 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.740 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.740 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.740 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.740 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.740 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.740 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_ENABLED": { 2023-12-07T10:36:21.740 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_ENABLED", 2023-12-07T10:36:21.740 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.741 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.741 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.741 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.741 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.741 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.741 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.741 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.741 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.742 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.742 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.742 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-12-07T10:36:21.742 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-12-07T10:36:21.742 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.742 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.742 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.743 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-12-07T10:36:21.743 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.743 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.743 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.743 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.743 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.743 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.743 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.744 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-12-07T10:36:21.744 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-12-07T10:36:21.744 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.744 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.744 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.744 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2023-12-07T10:36:21.744 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.744 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.744 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.745 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.745 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.745 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.745 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.745 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-12-07T10:36:21.745 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-12-07T10:36:21.745 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.746 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.746 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.746 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8", 2023-12-07T10:36:21.746 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.746 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.746 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.746 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.746 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.746 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.747 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.747 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.747 INFO:teuthology.orchestra.run.smithi119.stdout: "REST_REQUESTS_TIMEOUT": { 2023-12-07T10:36:21.747 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-12-07T10:36:21.747 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.747 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.747 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.747 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "45", 2023-12-07T10:36:21.747 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.748 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.748 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.748 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.748 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.748 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.748 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.748 INFO:teuthology.orchestra.run.smithi119.stdout: "RGW_API_ACCESS_KEY": { 2023-12-07T10:36:21.748 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "RGW_API_ACCESS_KEY", 2023-12-07T10:36:21.749 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.749 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.749 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.749 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.749 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.749 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.749 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.749 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.749 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.750 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.750 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.750 INFO:teuthology.orchestra.run.smithi119.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-12-07T10:36:21.750 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-12-07T10:36:21.750 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.750 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.750 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.751 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "admin", 2023-12-07T10:36:21.751 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.751 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.751 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.751 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.751 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.751 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.751 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.752 INFO:teuthology.orchestra.run.smithi119.stdout: "RGW_API_SECRET_KEY": { 2023-12-07T10:36:21.752 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "RGW_API_SECRET_KEY", 2023-12-07T10:36:21.752 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.752 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.752 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.752 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.752 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.752 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.752 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.753 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.753 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.753 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.753 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.753 INFO:teuthology.orchestra.run.smithi119.stdout: "RGW_API_SSL_VERIFY": { 2023-12-07T10:36:21.753 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "RGW_API_SSL_VERIFY", 2023-12-07T10:36:21.753 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.753 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.754 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.754 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.754 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.754 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.754 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.754 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.754 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.755 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.755 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.755 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.755 INFO:teuthology.orchestra.run.smithi119.stdout: "UNSAFE_TLS_v1_2": { 2023-12-07T10:36:21.755 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "UNSAFE_TLS_v1_2", 2023-12-07T10:36:21.755 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.755 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.755 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.755 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.756 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.756 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.756 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.756 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.756 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.756 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.756 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.756 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.756 INFO:teuthology.orchestra.run.smithi119.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-12-07T10:36:21.757 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-12-07T10:36:21.757 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.757 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.757 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.757 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2023-12-07T10:36:21.757 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.757 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.757 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.757 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.758 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.758 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.758 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.758 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.758 INFO:teuthology.orchestra.run.smithi119.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-12-07T10:36:21.758 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-12-07T10:36:21.758 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.758 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.758 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.759 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2023-12-07T10:36:21.759 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.759 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.759 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.759 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.759 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.759 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.759 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.760 INFO:teuthology.orchestra.run.smithi119.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-12-07T10:36:21.760 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-12-07T10:36:21.760 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.760 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.760 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.760 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5", 2023-12-07T10:36:21.760 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.760 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.760 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.761 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.761 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.761 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.761 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.761 INFO:teuthology.orchestra.run.smithi119.stdout: "cross_origin_url": { 2023-12-07T10:36:21.761 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cross_origin_url", 2023-12-07T10:36:21.761 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.761 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.762 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.762 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.762 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.762 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.762 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.762 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.762 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.762 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.762 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.763 INFO:teuthology.orchestra.run.smithi119.stdout: "crt_file": { 2023-12-07T10:36:21.763 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crt_file", 2023-12-07T10:36:21.763 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.763 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.763 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.763 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.763 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.763 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.764 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.764 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.764 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.764 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.764 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.764 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.764 INFO:teuthology.orchestra.run.smithi119.stdout: "debug": { 2023-12-07T10:36:21.764 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "debug", 2023-12-07T10:36:21.764 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.765 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.765 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.765 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.765 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.765 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.765 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.765 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Enable/disable debug options", 2023-12-07T10:36:21.765 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.765 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.766 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.766 INFO:teuthology.orchestra.run.smithi119.stdout: "jwt_token_ttl": { 2023-12-07T10:36:21.766 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "jwt_token_ttl", 2023-12-07T10:36:21.766 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.766 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.766 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.766 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "28800", 2023-12-07T10:36:21.766 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.767 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.767 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.767 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.767 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.767 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.767 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.767 INFO:teuthology.orchestra.run.smithi119.stdout: "key_file": { 2023-12-07T10:36:21.768 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "key_file", 2023-12-07T10:36:21.768 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.768 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.768 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.768 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.768 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.768 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.768 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.768 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.769 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.769 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.769 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.769 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.769 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.769 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.769 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.769 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.769 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.770 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.770 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.770 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.770 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.770 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.770 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.770 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.770 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.770 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.771 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.771 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.771 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.771 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.771 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.771 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.771 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.771 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.771 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.772 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.772 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.772 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.772 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.772 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.772 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.772 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.772 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.772 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.773 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.773 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.773 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.773 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.773 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.773 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.773 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.773 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.773 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.774 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.774 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.774 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.774 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.774 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.774 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.774 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.774 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.774 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.775 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.775 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.775 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.775 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.775 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.775 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.775 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.775 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.776 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.776 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.776 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.776 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.776 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.776 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.776 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.776 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.777 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.777 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.777 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.777 INFO:teuthology.orchestra.run.smithi119.stdout: "motd": { 2023-12-07T10:36:21.777 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "motd", 2023-12-07T10:36:21.777 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.777 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.777 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.777 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.778 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.778 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.778 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.778 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "The message of the day", 2023-12-07T10:36:21.778 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.778 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.778 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.778 INFO:teuthology.orchestra.run.smithi119.stdout: "redirect_resolve_ip_addr": { 2023-12-07T10:36:21.779 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "redirect_resolve_ip_addr", 2023-12-07T10:36:21.779 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.779 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.779 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.779 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.779 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.779 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.779 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.779 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.780 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.780 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.780 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.780 INFO:teuthology.orchestra.run.smithi119.stdout: "server_addr": { 2023-12-07T10:36:21.780 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_addr", 2023-12-07T10:36:21.780 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.780 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.780 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.780 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "::", 2023-12-07T10:36:21.781 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.781 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.781 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.781 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.781 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.781 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.781 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.781 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.781 INFO:teuthology.orchestra.run.smithi119.stdout: "server_port": { 2023-12-07T10:36:21.782 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_port", 2023-12-07T10:36:21.782 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.782 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.782 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.782 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8080", 2023-12-07T10:36:21.782 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.782 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.782 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.782 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.783 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.783 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.783 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.783 INFO:teuthology.orchestra.run.smithi119.stdout: "ssl": { 2023-12-07T10:36:21.783 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ssl", 2023-12-07T10:36:21.783 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.783 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.784 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.784 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.784 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.784 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.784 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.784 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.784 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.784 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.784 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.784 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.785 INFO:teuthology.orchestra.run.smithi119.stdout: "ssl_server_port": { 2023-12-07T10:36:21.785 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ssl_server_port", 2023-12-07T10:36:21.785 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.785 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.785 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.785 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8443", 2023-12-07T10:36:21.785 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.785 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.786 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.786 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.786 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.786 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.786 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.786 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.786 INFO:teuthology.orchestra.run.smithi119.stdout: "standby_behaviour": { 2023-12-07T10:36:21.786 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "standby_behaviour", 2023-12-07T10:36:21.786 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.787 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.787 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.787 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "redirect", 2023-12-07T10:36:21.787 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.787 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.787 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.787 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.787 INFO:teuthology.orchestra.run.smithi119.stdout: "redirect" 2023-12-07T10:36:21.788 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.788 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.788 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.788 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.788 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.788 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.788 INFO:teuthology.orchestra.run.smithi119.stdout: "standby_error_status_code": { 2023-12-07T10:36:21.788 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "standby_error_status_code", 2023-12-07T10:36:21.788 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.789 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.789 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.789 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "500", 2023-12-07T10:36:21.789 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "400", 2023-12-07T10:36:21.789 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "599", 2023-12-07T10:36:21.789 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.789 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.789 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.789 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.790 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.790 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.790 INFO:teuthology.orchestra.run.smithi119.stdout: "url_prefix": { 2023-12-07T10:36:21.790 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "url_prefix", 2023-12-07T10:36:21.790 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.790 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.790 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.790 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.790 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.791 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.791 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.791 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.791 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.791 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.791 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.791 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.791 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.792 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.792 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "devicehealth", 2023-12-07T10:36:21.792 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.792 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.792 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.792 INFO:teuthology.orchestra.run.smithi119.stdout: "enable_monitoring": { 2023-12-07T10:36:21.792 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "enable_monitoring", 2023-12-07T10:36:21.792 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.793 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.793 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.793 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.793 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.793 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.793 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.793 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "monitor device health metrics", 2023-12-07T10:36:21.793 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.793 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.794 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.794 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.794 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.794 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.794 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.794 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.794 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.794 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.795 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.795 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.795 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.795 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.795 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.795 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.795 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.795 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.795 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.795 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.796 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.796 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.796 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.796 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.796 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.796 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.796 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.796 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.797 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.797 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.797 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.797 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.797 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.797 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.797 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.797 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.798 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.798 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.798 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.798 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.798 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.798 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.798 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.798 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.799 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.799 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.799 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.799 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.799 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.799 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.799 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.799 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.799 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.800 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.800 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.800 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.800 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.800 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.800 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.800 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.800 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.801 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.801 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.801 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.801 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.801 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.801 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.801 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.801 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.801 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.802 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.802 INFO:teuthology.orchestra.run.smithi119.stdout: "mark_out_threshold": { 2023-12-07T10:36:21.802 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mark_out_threshold", 2023-12-07T10:36:21.802 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.802 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.802 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.802 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "2419200", 2023-12-07T10:36:21.802 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.803 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.803 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.803 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-12-07T10:36:21.803 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.803 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.803 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.803 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_name": { 2023-12-07T10:36:21.803 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "pool_name", 2023-12-07T10:36:21.804 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.804 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.804 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.804 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "device_health_metrics", 2023-12-07T10:36:21.804 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.804 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.804 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.804 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "name of pool in which to store device health metrics", 2023-12-07T10:36:21.804 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.805 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.805 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.805 INFO:teuthology.orchestra.run.smithi119.stdout: "retention_period": { 2023-12-07T10:36:21.805 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "retention_period", 2023-12-07T10:36:21.805 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.805 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.805 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.805 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "15552000", 2023-12-07T10:36:21.806 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.806 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.806 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.806 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how long to retain device health metrics", 2023-12-07T10:36:21.806 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.806 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.806 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.806 INFO:teuthology.orchestra.run.smithi119.stdout: "scrape_frequency": { 2023-12-07T10:36:21.807 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "scrape_frequency", 2023-12-07T10:36:21.807 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.807 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.807 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.807 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "86400", 2023-12-07T10:36:21.807 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.807 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.807 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.807 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to scrape device health metrics", 2023-12-07T10:36:21.808 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.808 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.808 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.808 INFO:teuthology.orchestra.run.smithi119.stdout: "self_heal": { 2023-12-07T10:36:21.808 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "self_heal", 2023-12-07T10:36:21.808 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.808 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.808 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.809 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.809 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.809 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.809 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.809 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-12-07T10:36:21.809 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.809 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.809 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.810 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2023-12-07T10:36:21.810 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2023-12-07T10:36:21.810 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.810 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.810 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.810 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2023-12-07T10:36:21.810 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.810 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.811 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.811 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to wake up and check device health", 2023-12-07T10:36:21.811 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.811 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.811 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.811 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_threshold": { 2023-12-07T10:36:21.811 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_threshold", 2023-12-07T10:36:21.811 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.812 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.812 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.812 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "7257600", 2023-12-07T10:36:21.812 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.812 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.812 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.812 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-12-07T10:36:21.812 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.812 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.813 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.813 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.813 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.813 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "diskprediction_local", 2023-12-07T10:36:21.813 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.813 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.814 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.814 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.814 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.814 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.814 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.814 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.814 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.814 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.814 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.815 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.815 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.815 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.815 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.815 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.815 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.815 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.815 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.816 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.816 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.816 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.816 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.816 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.816 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.816 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.816 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.817 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.817 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.817 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.817 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.817 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.817 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.817 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.817 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.817 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.818 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.818 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.818 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.818 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.818 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.818 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.819 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.819 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.819 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.819 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.819 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.819 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.819 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.819 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.819 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.820 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.820 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.820 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.820 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.820 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.820 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.820 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.820 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.821 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.821 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.821 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.821 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.821 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.821 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.821 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.821 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.821 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.822 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.822 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.822 INFO:teuthology.orchestra.run.smithi119.stdout: "predict_interval": { 2023-12-07T10:36:21.822 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "predict_interval", 2023-12-07T10:36:21.822 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.822 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.822 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.822 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "86400", 2023-12-07T10:36:21.822 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.823 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.823 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.823 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.823 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.823 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.823 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.823 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.823 INFO:teuthology.orchestra.run.smithi119.stdout: "predictor_model": { 2023-12-07T10:36:21.823 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "predictor_model", 2023-12-07T10:36:21.824 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.824 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.824 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.824 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "prophetstor", 2023-12-07T10:36:21.824 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.824 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.824 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.824 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.825 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.825 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.825 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.825 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2023-12-07T10:36:21.825 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2023-12-07T10:36:21.825 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.825 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.825 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.825 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2023-12-07T10:36:21.826 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.826 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.826 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.826 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.826 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.826 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.826 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.826 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.827 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.827 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.827 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.827 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "influx", 2023-12-07T10:36:21.827 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": false, 2023-12-07T10:36:21.827 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "influxdb python module not found", 2023-12-07T10:36:21.827 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.827 INFO:teuthology.orchestra.run.smithi119.stdout: "batch_size": { 2023-12-07T10:36:21.827 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "batch_size", 2023-12-07T10:36:21.828 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.828 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.828 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.828 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5000", 2023-12-07T10:36:21.828 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.828 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.828 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.828 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-12-07T10:36:21.829 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.829 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.829 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.829 INFO:teuthology.orchestra.run.smithi119.stdout: "database": { 2023-12-07T10:36:21.829 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "database", 2023-12-07T10:36:21.829 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.829 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.829 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.830 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "ceph", 2023-12-07T10:36:21.830 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.830 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.830 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.830 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-12-07T10:36:21.830 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.830 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.830 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.830 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.831 INFO:teuthology.orchestra.run.smithi119.stdout: "hostname": { 2023-12-07T10:36:21.831 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "hostname", 2023-12-07T10:36:21.831 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.831 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.831 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.831 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.831 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.831 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.831 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.832 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "InfluxDB server hostname", 2023-12-07T10:36:21.832 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.832 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.832 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.832 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2023-12-07T10:36:21.832 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2023-12-07T10:36:21.832 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.832 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.833 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.833 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "30", 2023-12-07T10:36:21.833 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "5", 2023-12-07T10:36:21.833 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.833 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.833 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-12-07T10:36:21.833 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.833 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.833 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.834 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.834 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.834 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.834 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.834 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.834 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.834 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.834 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.835 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.835 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.835 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.835 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.835 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.835 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.835 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.835 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.835 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.836 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.836 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.836 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.836 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.836 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.836 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.836 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.836 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.836 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.837 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.837 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.837 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.837 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.837 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.837 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.837 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.837 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.837 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.838 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.838 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.838 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.838 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.838 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.838 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.838 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.838 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.839 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.839 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.839 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.839 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.839 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.839 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.839 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.839 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.840 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.840 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.840 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.840 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.840 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.840 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.840 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.840 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.841 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.841 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.841 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.841 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.841 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.841 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.841 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.841 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.842 INFO:teuthology.orchestra.run.smithi119.stdout: "password": { 2023-12-07T10:36:21.842 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "password", 2023-12-07T10:36:21.842 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.842 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.842 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.842 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.842 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.842 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.842 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.843 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "password of InfluxDB server user", 2023-12-07T10:36:21.843 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.843 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.843 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.843 INFO:teuthology.orchestra.run.smithi119.stdout: "port": { 2023-12-07T10:36:21.843 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "port", 2023-12-07T10:36:21.843 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.843 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.844 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.844 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8086", 2023-12-07T10:36:21.844 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.844 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.844 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.844 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "InfluxDB server port", 2023-12-07T10:36:21.844 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.844 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.844 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.845 INFO:teuthology.orchestra.run.smithi119.stdout: "ssl": { 2023-12-07T10:36:21.845 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ssl", 2023-12-07T10:36:21.845 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.845 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.845 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.845 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "false", 2023-12-07T10:36:21.845 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.846 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.846 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.846 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-12-07T10:36:21.846 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.846 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.846 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.846 INFO:teuthology.orchestra.run.smithi119.stdout: "threads": { 2023-12-07T10:36:21.846 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "threads", 2023-12-07T10:36:21.847 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.847 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.847 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.847 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5", 2023-12-07T10:36:21.847 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1", 2023-12-07T10:36:21.847 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "32", 2023-12-07T10:36:21.847 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.847 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-12-07T10:36:21.847 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.848 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.848 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.848 INFO:teuthology.orchestra.run.smithi119.stdout: "username": { 2023-12-07T10:36:21.848 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "username", 2023-12-07T10:36:21.848 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.848 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.848 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.848 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.849 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.849 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.849 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.849 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "username of InfluxDB server user", 2023-12-07T10:36:21.849 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.849 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.849 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.849 INFO:teuthology.orchestra.run.smithi119.stdout: "verify_ssl": { 2023-12-07T10:36:21.849 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "verify_ssl", 2023-12-07T10:36:21.850 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.850 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.850 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.850 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "true", 2023-12-07T10:36:21.850 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.850 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.850 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.850 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-12-07T10:36:21.851 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.851 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.851 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.851 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.851 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.851 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.851 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "insights", 2023-12-07T10:36:21.851 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.852 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.852 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.852 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.852 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.852 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.852 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.852 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.852 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.852 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.853 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.853 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.853 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.853 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.853 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.853 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.853 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.853 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.853 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.853 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.854 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.854 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.854 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.854 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.854 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.854 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.854 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.855 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.855 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.855 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.855 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.855 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.855 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.855 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.855 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.856 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.856 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.856 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.856 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.856 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.856 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.856 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.856 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.857 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.857 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.857 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.857 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.857 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.857 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.857 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.857 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.858 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.858 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.858 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.858 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.858 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.858 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.858 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.858 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.859 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.859 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.859 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.859 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.859 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.859 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.859 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.859 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.859 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.860 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.860 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.860 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.860 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "iostat", 2023-12-07T10:36:21.860 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.860 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.860 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.861 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.861 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.861 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.861 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.861 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.861 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.861 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.861 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.862 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.862 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.862 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.862 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.862 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.862 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.862 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.862 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.862 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.863 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.863 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.863 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.863 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.863 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.863 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.863 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.863 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.864 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.864 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.864 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.864 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.864 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.864 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.864 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.864 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.865 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.865 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.865 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.865 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.865 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.865 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.865 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.865 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.865 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.865 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.866 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.866 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.866 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.866 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.866 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.866 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.866 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.866 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.866 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.867 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.867 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.867 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.867 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.867 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.867 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.867 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.868 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.868 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.868 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.868 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.868 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.868 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.868 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.868 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.868 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.869 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.869 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "k8sevents", 2023-12-07T10:36:21.869 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.869 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.869 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.869 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_event_retention_days": { 2023-12-07T10:36:21.869 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ceph_event_retention_days", 2023-12-07T10:36:21.869 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.869 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.870 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.870 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "7", 2023-12-07T10:36:21.870 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.870 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.870 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.870 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Days to hold ceph event information within local cache", 2023-12-07T10:36:21.870 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.870 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.871 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.871 INFO:teuthology.orchestra.run.smithi119.stdout: "config_check_secs": { 2023-12-07T10:36:21.871 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "config_check_secs", 2023-12-07T10:36:21.871 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.871 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.871 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.871 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2023-12-07T10:36:21.872 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "10", 2023-12-07T10:36:21.872 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.872 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.872 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-12-07T10:36:21.872 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.872 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.872 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.872 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.873 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.873 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.873 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.873 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.873 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.873 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.873 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.873 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.873 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.874 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.874 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.874 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.874 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.874 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.874 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.874 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.874 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.874 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.875 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.875 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.875 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.875 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.875 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.875 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.875 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.875 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.875 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.876 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.876 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.876 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.876 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.876 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.876 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.876 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.876 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.877 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.877 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.877 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.877 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.877 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.877 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.877 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.877 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.878 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.878 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.878 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.878 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.878 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.878 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.878 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.878 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.878 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.879 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.879 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.879 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.879 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.879 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.879 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.879 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.879 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.879 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.880 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.880 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.880 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.880 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.880 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.880 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.880 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.880 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.880 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.881 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "localpool", 2023-12-07T10:36:21.881 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.881 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.881 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.881 INFO:teuthology.orchestra.run.smithi119.stdout: "failure_domain": { 2023-12-07T10:36:21.881 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "failure_domain", 2023-12-07T10:36:21.881 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.881 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.881 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.882 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "host", 2023-12-07T10:36:21.882 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.882 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.882 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.882 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "failure domain for any created local pool", 2023-12-07T10:36:21.882 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-12-07T10:36:21.882 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.882 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.883 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.883 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.883 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.883 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.883 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.883 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.883 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.883 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.884 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.884 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.884 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.884 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.884 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.884 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.884 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.884 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.885 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.885 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.885 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.885 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.885 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.885 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.885 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.885 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.886 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.886 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.886 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.886 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.886 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.886 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.886 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.886 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.886 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.887 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.887 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.887 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.887 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.887 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.887 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.887 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.887 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.888 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.888 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.888 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.888 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.888 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.888 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.888 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.888 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.888 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.889 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.889 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.889 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.889 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.889 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.889 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.889 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.889 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.889 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.890 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.890 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.890 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.890 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.890 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.890 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.890 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.890 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.890 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.891 INFO:teuthology.orchestra.run.smithi119.stdout: "min_size": { 2023-12-07T10:36:21.891 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "min_size", 2023-12-07T10:36:21.891 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.891 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.891 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.891 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.891 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.891 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.891 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.892 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "default min_size for any created local pool", 2023-12-07T10:36:21.892 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-12-07T10:36:21.892 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.892 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.892 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.892 INFO:teuthology.orchestra.run.smithi119.stdout: "num_rep": { 2023-12-07T10:36:21.892 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "num_rep", 2023-12-07T10:36:21.892 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.892 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.893 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.893 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "3", 2023-12-07T10:36:21.893 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.893 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.893 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.893 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "default replica count for any created local pool", 2023-12-07T10:36:21.893 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.893 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.893 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.894 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num": { 2023-12-07T10:36:21.894 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "pg_num", 2023-12-07T10:36:21.894 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.894 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.894 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.894 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "128", 2023-12-07T10:36:21.894 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.894 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.895 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.895 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "default pg_num for any created local pool", 2023-12-07T10:36:21.895 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.895 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.895 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.895 INFO:teuthology.orchestra.run.smithi119.stdout: "prefix": { 2023-12-07T10:36:21.895 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "prefix", 2023-12-07T10:36:21.895 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.896 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.896 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.896 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.896 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.896 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.896 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.896 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "name prefix for any created local pool", 2023-12-07T10:36:21.896 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.897 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.897 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.897 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.897 INFO:teuthology.orchestra.run.smithi119.stdout: "subtree": { 2023-12-07T10:36:21.897 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "subtree", 2023-12-07T10:36:21.897 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.897 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.897 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.897 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "rack", 2023-12-07T10:36:21.898 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.898 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.898 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.898 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "CRUSH level for which to create a local pool", 2023-12-07T10:36:21.898 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-12-07T10:36:21.898 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.898 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.898 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.899 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.899 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.899 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mds_autoscaler", 2023-12-07T10:36:21.899 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.899 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.899 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.899 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.899 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.900 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.900 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.900 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.900 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.900 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.900 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.900 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.900 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.900 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.901 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.901 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.901 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.901 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.901 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.901 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.901 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.901 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.902 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.902 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.902 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.902 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.902 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.902 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.902 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.902 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.903 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.903 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.903 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.903 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.903 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.903 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.903 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.903 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.903 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.904 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.904 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.904 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.904 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.904 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.904 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.904 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.905 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.905 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.905 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.905 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.905 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.905 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.905 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.905 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.906 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.906 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.906 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.906 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.906 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.906 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.906 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.906 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.906 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.907 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.907 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.907 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.907 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.907 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.907 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.907 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.907 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.907 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.908 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.908 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.908 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.908 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.908 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mirroring", 2023-12-07T10:36:21.908 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.908 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.908 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.909 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.909 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.909 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.909 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.909 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.909 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.909 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.909 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.909 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.910 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.910 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.910 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.910 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.910 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.910 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.910 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.910 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.910 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.911 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.911 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.911 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.911 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.911 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.911 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.911 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.911 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.911 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.912 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.912 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.912 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.912 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.912 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.912 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.912 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.912 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.913 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.913 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.913 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.913 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.913 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.913 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.913 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.913 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.913 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.914 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.914 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.914 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.914 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.914 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.914 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.914 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.914 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.915 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.915 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.915 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.915 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.915 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.915 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.915 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.915 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.915 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.915 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.916 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.916 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.916 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.916 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.916 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.916 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.916 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.916 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.916 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.917 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.917 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.917 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "nfs", 2023-12-07T10:36:21.917 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.917 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.917 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.917 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.917 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.917 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.918 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.918 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.918 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.918 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.918 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.918 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.918 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.918 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.919 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.919 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.919 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.919 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.919 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.919 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.919 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.919 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.919 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.920 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.920 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.920 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.920 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.920 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.920 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.920 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.920 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.921 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.921 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.921 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.921 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.921 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.921 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.921 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.921 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.922 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.922 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.922 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.922 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.922 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.922 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.922 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.922 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.923 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.923 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.923 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.923 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.923 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.923 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.923 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.923 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.923 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.924 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.924 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.924 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.924 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.924 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.924 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.924 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.924 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.924 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.925 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.925 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.925 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.925 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.925 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.925 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.925 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.925 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.925 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.926 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "orchestrator", 2023-12-07T10:36:21.926 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.926 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.926 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.926 INFO:teuthology.orchestra.run.smithi119.stdout: "fail_fs": { 2023-12-07T10:36:21.926 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "fail_fs", 2023-12-07T10:36:21.926 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.926 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.927 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.927 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.927 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.927 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.927 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.927 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-12-07T10:36:21.927 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.927 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.927 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.928 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.928 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.928 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.928 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.928 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.928 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.928 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.928 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.928 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.929 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.929 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.929 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.929 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.929 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.929 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.929 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.929 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.929 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.930 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.930 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.930 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.930 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.930 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.930 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.930 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.930 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.930 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.931 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.931 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.931 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.931 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.931 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.931 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.931 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.931 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.931 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.932 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.932 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.932 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.932 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.932 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.932 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.932 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.932 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.932 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.933 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.933 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.933 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.933 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.933 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.933 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.933 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.933 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.933 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.934 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.934 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.934 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.934 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.934 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.934 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.934 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.934 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.934 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.935 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.935 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.935 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.935 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.935 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.935 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.935 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.935 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.935 INFO:teuthology.orchestra.run.smithi119.stdout: "orchestrator": { 2023-12-07T10:36:21.936 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "orchestrator", 2023-12-07T10:36:21.936 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.936 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.936 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.936 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.936 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.936 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.936 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.936 INFO:teuthology.orchestra.run.smithi119.stdout: "cephadm", 2023-12-07T10:36:21.937 INFO:teuthology.orchestra.run.smithi119.stdout: "rook", 2023-12-07T10:36:21.937 INFO:teuthology.orchestra.run.smithi119.stdout: "test_orchestrator" 2023-12-07T10:36:21.937 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.937 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Orchestrator backend", 2023-12-07T10:36:21.937 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.937 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.937 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.937 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.937 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.938 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.938 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.938 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "osd_perf_query", 2023-12-07T10:36:21.938 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.938 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.938 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.938 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.938 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.938 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.939 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.939 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.939 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.939 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.939 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.939 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.939 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.939 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.939 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.940 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.940 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.940 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.940 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.940 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.940 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.940 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.940 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.940 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.941 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.941 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.941 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.941 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.941 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.941 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.941 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.941 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.941 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.942 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.942 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.942 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.942 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.942 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.942 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.942 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.942 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.942 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.943 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.943 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.943 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.943 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.943 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.943 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.943 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.943 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.943 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.944 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.944 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.944 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.944 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.944 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.944 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.944 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.944 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.944 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.945 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.945 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.945 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.945 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.945 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.945 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.945 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.945 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.945 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.946 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.946 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.946 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.946 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.946 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.946 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.946 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.946 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "osd_support", 2023-12-07T10:36:21.946 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.947 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.947 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.947 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.947 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.947 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.947 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.947 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.947 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.947 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.948 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.948 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.948 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.948 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.948 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.948 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.948 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.948 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.948 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.949 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.949 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.949 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.949 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.949 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.949 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.949 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.949 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.950 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.950 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.950 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.950 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.950 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.950 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.950 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.950 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.951 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.951 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.951 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.951 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.951 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.951 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.951 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.951 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.951 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.952 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.952 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.952 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.952 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.952 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.952 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.952 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.952 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.952 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.953 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.953 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.953 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.953 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.953 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.953 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.953 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.953 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.954 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.954 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.954 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.954 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.954 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.954 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.954 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.954 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.954 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.955 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.955 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.955 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.955 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "pg_autoscaler", 2023-12-07T10:36:21.955 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.955 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.955 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.956 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.956 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.956 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.956 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.956 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.956 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.956 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.956 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.956 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.957 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.957 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.957 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.957 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.957 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.957 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.957 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.957 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.957 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.958 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.958 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.958 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.958 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.958 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.958 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.958 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.959 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.959 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.959 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.959 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.959 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.959 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.959 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.959 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.959 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.960 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.960 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.960 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.960 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.960 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.960 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.960 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.960 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.960 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.961 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.961 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.961 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.961 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.961 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.961 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.961 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.961 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.961 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.962 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.962 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.962 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.962 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.962 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.962 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.962 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.962 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.963 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.963 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.963 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.963 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.963 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.963 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.963 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.963 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.963 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2023-12-07T10:36:21.964 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2023-12-07T10:36:21.964 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.964 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.964 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.964 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2023-12-07T10:36:21.964 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.964 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.964 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.965 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.965 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.965 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.965 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.965 INFO:teuthology.orchestra.run.smithi119.stdout: "threshold": { 2023-12-07T10:36:21.965 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "threshold", 2023-12-07T10:36:21.965 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2023-12-07T10:36:21.965 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.966 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.966 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "3.0", 2023-12-07T10:36:21.966 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1.0", 2023-12-07T10:36:21.966 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.966 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.966 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "scaling threshold", 2023-12-07T10:36:21.966 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-12-07T10:36:21.966 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.966 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.967 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.967 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.967 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.967 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "progress", 2023-12-07T10:36:21.967 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.967 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.967 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.967 INFO:teuthology.orchestra.run.smithi119.stdout: "allow_pg_recovery_event": { 2023-12-07T10:36:21.968 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "allow_pg_recovery_event", 2023-12-07T10:36:21.968 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.968 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.968 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.968 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.968 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.968 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.968 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.968 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "allow the module to show pg recovery progress", 2023-12-07T10:36:21.969 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.969 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.969 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.969 INFO:teuthology.orchestra.run.smithi119.stdout: "enabled": { 2023-12-07T10:36:21.969 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "enabled", 2023-12-07T10:36:21.969 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.969 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.970 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.970 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.970 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.970 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.970 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.970 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.970 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.970 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.970 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.971 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.971 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.971 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.971 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.971 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.971 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.971 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.972 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.972 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.972 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.972 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.972 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.972 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.972 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.972 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.972 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.973 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.973 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.973 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.973 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.973 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.973 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.973 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.973 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.974 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.974 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.974 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.974 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.974 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.974 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.974 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.974 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.975 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.975 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.975 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.975 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.975 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.975 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.975 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.975 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.976 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.976 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.976 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.976 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.976 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.976 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.976 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.976 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.976 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.977 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.977 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.977 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.977 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.977 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.977 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.977 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.978 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.978 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.978 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.978 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.978 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.978 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.978 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.978 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.978 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.979 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.979 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.979 INFO:teuthology.orchestra.run.smithi119.stdout: "max_completed_events": { 2023-12-07T10:36:21.979 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_completed_events", 2023-12-07T10:36:21.979 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.979 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.979 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.979 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "50", 2023-12-07T10:36:21.979 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.980 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.980 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.980 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "number of past completed events to remember", 2023-12-07T10:36:21.980 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.980 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.980 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.980 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.980 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2023-12-07T10:36:21.980 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2023-12-07T10:36:21.981 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:21.981 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.981 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.981 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5", 2023-12-07T10:36:21.981 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.981 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.981 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.981 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how long the module is going to sleep", 2023-12-07T10:36:21.981 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.982 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.982 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.982 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.982 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:21.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.982 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:21.982 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "prometheus", 2023-12-07T10:36:21.982 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:21.982 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:21.983 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:21.983 INFO:teuthology.orchestra.run.smithi119.stdout: "cache": { 2023-12-07T10:36:21.983 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cache", 2023-12-07T10:36:21.983 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.983 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.983 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.983 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.983 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.984 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.984 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.984 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.984 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.984 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.984 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.984 INFO:teuthology.orchestra.run.smithi119.stdout: "exclude_perf_counters": { 2023-12-07T10:36:21.984 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "exclude_perf_counters", 2023-12-07T10:36:21.985 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.985 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.985 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.985 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:21.985 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.985 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.985 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.985 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-12-07T10:36:21.985 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-12-07T10:36:21.986 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.986 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.986 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.986 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:21.986 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:21.986 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.986 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.986 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.986 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.987 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.987 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.987 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.987 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.987 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.987 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.987 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.987 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.987 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.988 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.988 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.988 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.988 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.988 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.988 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:21.988 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:21.988 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.989 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.989 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.989 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.989 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.989 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.989 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.989 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.989 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.989 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.990 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.990 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:21.990 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:21.990 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.990 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.990 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.990 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:21.990 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.991 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.991 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:21.991 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:21.991 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:21.991 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:21.991 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:21.991 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:21.991 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:21.991 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:21.992 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.992 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.992 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.992 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.992 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:21.992 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:21.992 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:21.993 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.993 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:21.993 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:21.993 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.993 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.993 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.993 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.993 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.993 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.993 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.994 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_stats_pools": { 2023-12-07T10:36:21.994 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rbd_stats_pools", 2023-12-07T10:36:21.994 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.994 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.994 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.994 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:21.994 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.995 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.995 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.995 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.995 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.995 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.995 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.995 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_stats_pools_refresh_interval": { 2023-12-07T10:36:21.995 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-12-07T10:36:21.996 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:21.996 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.996 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.996 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "300", 2023-12-07T10:36:21.996 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.996 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.996 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.996 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.996 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.996 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.997 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.997 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.997 INFO:teuthology.orchestra.run.smithi119.stdout: "scrape_interval": { 2023-12-07T10:36:21.997 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "scrape_interval", 2023-12-07T10:36:21.997 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2023-12-07T10:36:21.997 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.997 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.997 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "15.0", 2023-12-07T10:36:21.998 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.998 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.998 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.998 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:21.998 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:21.998 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:21.998 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:21.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:21.998 INFO:teuthology.orchestra.run.smithi119.stdout: "server_addr": { 2023-12-07T10:36:21.999 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_addr", 2023-12-07T10:36:21.999 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:21.999 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:21.999 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:21.999 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "::", 2023-12-07T10:36:21.999 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:21.999 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:21.999 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:21.999 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-12-07T10:36:22.000 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.000 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.000 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.000 INFO:teuthology.orchestra.run.smithi119.stdout: "server_port": { 2023-12-07T10:36:22.000 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_port", 2023-12-07T10:36:22.000 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.000 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.000 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.001 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "9283", 2023-12-07T10:36:22.001 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.001 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.001 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.001 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-12-07T10:36:22.001 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.001 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.001 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.002 INFO:teuthology.orchestra.run.smithi119.stdout: "stale_cache_strategy": { 2023-12-07T10:36:22.002 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "stale_cache_strategy", 2023-12-07T10:36:22.002 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.002 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.002 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.002 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "log", 2023-12-07T10:36:22.002 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.002 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.002 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.003 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.003 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.003 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.003 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.003 INFO:teuthology.orchestra.run.smithi119.stdout: "standby_behaviour": { 2023-12-07T10:36:22.003 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "standby_behaviour", 2023-12-07T10:36:22.003 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.003 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.004 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.004 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "default", 2023-12-07T10:36:22.004 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.004 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.004 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.004 INFO:teuthology.orchestra.run.smithi119.stdout: "default", 2023-12-07T10:36:22.004 INFO:teuthology.orchestra.run.smithi119.stdout: "error" 2023-12-07T10:36:22.004 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.004 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.005 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.005 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.005 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.005 INFO:teuthology.orchestra.run.smithi119.stdout: "standby_error_status_code": { 2023-12-07T10:36:22.005 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "standby_error_status_code", 2023-12-07T10:36:22.005 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.005 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.006 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.006 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "500", 2023-12-07T10:36:22.006 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "400", 2023-12-07T10:36:22.006 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "599", 2023-12-07T10:36:22.006 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.006 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.006 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.006 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.006 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.007 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.007 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.007 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rbd_support", 2023-12-07T10:36:22.007 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.007 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.007 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.008 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.008 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.008 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.008 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.008 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.008 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.008 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.008 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.008 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.009 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.009 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.009 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.009 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.009 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.009 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.009 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.009 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.009 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.010 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.010 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.010 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.010 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.010 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.010 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.010 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.011 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.011 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.011 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.011 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.011 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.011 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.011 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.011 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.012 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.012 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.012 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.012 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.012 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.012 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.012 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.012 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.013 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.013 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.013 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.013 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.013 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.013 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.013 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.013 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.013 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.014 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.014 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.014 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.014 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.014 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.014 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.014 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.015 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.015 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.015 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.015 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.015 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.015 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.015 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.015 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.015 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.015 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.016 INFO:teuthology.orchestra.run.smithi119.stdout: "max_concurrent_snap_create": { 2023-12-07T10:36:22.016 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_concurrent_snap_create", 2023-12-07T10:36:22.016 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.016 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.016 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.016 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2023-12-07T10:36:22.016 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.016 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.016 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.017 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.017 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.017 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.017 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.017 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.017 INFO:teuthology.orchestra.run.smithi119.stdout: "mirror_snapshot_schedule": { 2023-12-07T10:36:22.017 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mirror_snapshot_schedule", 2023-12-07T10:36:22.017 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.017 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.018 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.018 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.018 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.018 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.018 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.018 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.018 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.018 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.019 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.019 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.019 INFO:teuthology.orchestra.run.smithi119.stdout: "trash_purge_schedule": { 2023-12-07T10:36:22.019 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "trash_purge_schedule", 2023-12-07T10:36:22.019 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.019 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.019 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.019 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.019 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.020 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.020 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.020 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.020 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.020 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.020 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.020 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.020 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.020 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.021 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.021 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "restful", 2023-12-07T10:36:22.021 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.021 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.021 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.021 INFO:teuthology.orchestra.run.smithi119.stdout: "enable_auth": { 2023-12-07T10:36:22.021 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "enable_auth", 2023-12-07T10:36:22.021 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.021 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.022 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.022 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.022 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.022 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.022 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.022 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.022 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.022 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.023 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.023 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.023 INFO:teuthology.orchestra.run.smithi119.stdout: "key_file": { 2023-12-07T10:36:22.023 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "key_file", 2023-12-07T10:36:22.023 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.023 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.023 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.023 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.023 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.024 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.024 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.024 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.024 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.024 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.024 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.024 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.024 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.024 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.025 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.025 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.025 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.025 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.025 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.025 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.025 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.025 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.025 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.026 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.026 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.026 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.026 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.026 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.026 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.026 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.026 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.026 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.027 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.027 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.027 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.027 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.027 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.027 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.027 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.027 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.027 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.028 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.028 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.028 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.028 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.028 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.028 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.028 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.028 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.028 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.029 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.029 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.029 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.029 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.029 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.029 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.029 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.029 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.029 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.030 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.030 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.030 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.030 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.030 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.030 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.030 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.030 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.031 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.031 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.031 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.031 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.031 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.031 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.031 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.031 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.032 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.032 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.032 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.032 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.032 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.032 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.032 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.032 INFO:teuthology.orchestra.run.smithi119.stdout: "server_addr": { 2023-12-07T10:36:22.032 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_addr", 2023-12-07T10:36:22.033 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.033 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.033 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.033 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.033 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.033 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.033 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.033 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.033 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.033 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.034 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.034 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.034 INFO:teuthology.orchestra.run.smithi119.stdout: "server_port": { 2023-12-07T10:36:22.034 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_port", 2023-12-07T10:36:22.034 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.034 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.034 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.034 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.035 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.035 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.035 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.035 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.035 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.035 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.035 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.035 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.036 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.036 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.036 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.036 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rgw", 2023-12-07T10:36:22.036 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.036 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.036 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.036 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.036 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.037 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.037 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.037 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.037 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.037 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.037 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.037 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.037 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.038 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.038 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.038 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.038 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.038 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.038 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.038 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.038 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.038 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.039 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.039 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.039 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.039 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.039 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.039 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.039 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.039 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.039 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.040 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.040 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.040 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.040 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.040 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.040 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.040 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.040 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.040 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.041 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.041 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.041 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.041 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.041 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.041 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.041 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.041 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.042 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.042 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.042 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.042 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.042 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.042 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.042 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.042 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.043 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.043 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.043 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.043 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.043 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.043 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.043 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.043 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.043 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.044 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.044 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.044 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.044 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.044 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.044 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.044 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.044 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.044 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.045 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.045 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.045 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rook", 2023-12-07T10:36:22.045 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.045 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.045 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.045 INFO:teuthology.orchestra.run.smithi119.stdout: "drive_group_interval": { 2023-12-07T10:36:22.045 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "drive_group_interval", 2023-12-07T10:36:22.045 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2023-12-07T10:36:22.046 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.046 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.046 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "300.0", 2023-12-07T10:36:22.046 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.046 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.046 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.046 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-12-07T10:36:22.046 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.046 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.047 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.047 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.047 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.047 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.047 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.047 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.047 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.047 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.048 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.048 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.048 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.048 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.048 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.048 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.048 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.048 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.048 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.049 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.049 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.049 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.049 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.049 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.049 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.049 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.049 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.049 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.050 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.050 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.050 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.050 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.050 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.050 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.050 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.050 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.050 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.051 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.051 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.051 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.051 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.051 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.051 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.051 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.051 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.051 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.052 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.052 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.052 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.052 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.052 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.052 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.052 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.052 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.052 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.053 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.053 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.053 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.053 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.053 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.053 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.053 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.053 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.053 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.054 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.054 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.054 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.054 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.054 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.054 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.054 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.054 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.054 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.055 INFO:teuthology.orchestra.run.smithi119.stdout: "storage_class": { 2023-12-07T10:36:22.055 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "storage_class", 2023-12-07T10:36:22.055 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.055 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.055 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.055 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "local", 2023-12-07T10:36:22.055 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.055 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.056 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.056 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-12-07T10:36:22.056 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.056 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.056 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.056 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.056 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.056 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.057 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.057 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "selftest", 2023-12-07T10:36:22.057 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.057 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.057 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.057 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.057 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.057 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.057 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.058 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.058 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.058 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.058 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.058 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.058 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.058 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.058 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.058 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.059 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.059 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.059 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.059 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.059 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.059 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.059 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.059 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.060 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.060 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.060 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.060 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.060 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.060 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.060 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.060 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.060 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.061 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.061 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.061 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.061 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.061 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.061 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.061 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.061 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.061 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.062 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.062 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.062 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.062 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.062 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.062 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.062 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.062 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.063 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.063 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.063 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.063 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.063 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.063 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.063 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.063 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.063 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.064 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.064 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.064 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.064 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.064 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.064 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.064 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.064 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.065 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.065 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.065 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.065 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.065 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.065 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.065 INFO:teuthology.orchestra.run.smithi119.stdout: "roption1": { 2023-12-07T10:36:22.065 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "roption1", 2023-12-07T10:36:22.065 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.065 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.066 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.066 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.066 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.066 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.066 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.066 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.066 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.066 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.066 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.067 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.067 INFO:teuthology.orchestra.run.smithi119.stdout: "roption2": { 2023-12-07T10:36:22.067 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "roption2", 2023-12-07T10:36:22.067 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.067 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.067 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.067 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "xyz", 2023-12-07T10:36:22.067 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.068 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.068 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.068 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.068 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.068 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.068 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.068 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.068 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption1": { 2023-12-07T10:36:22.068 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption1", 2023-12-07T10:36:22.069 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.069 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.069 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.069 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.069 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.069 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.069 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.069 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.069 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.070 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.070 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.070 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.070 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption2": { 2023-12-07T10:36:22.070 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption2", 2023-12-07T10:36:22.070 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.070 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.070 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.070 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.071 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.071 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.071 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.071 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.071 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.071 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.071 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.071 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.071 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption3": { 2023-12-07T10:36:22.072 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption3", 2023-12-07T10:36:22.072 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2023-12-07T10:36:22.072 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.072 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.072 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.072 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.072 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.072 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.072 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.073 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.073 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.073 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.073 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.073 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption4": { 2023-12-07T10:36:22.073 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption4", 2023-12-07T10:36:22.073 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.073 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.073 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.074 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.074 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.074 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.074 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.074 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.074 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.074 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.074 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.075 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.075 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption5": { 2023-12-07T10:36:22.075 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption5", 2023-12-07T10:36:22.075 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.075 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.075 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.075 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.075 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.075 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.076 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.076 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.076 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.076 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.076 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.076 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.076 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption6": { 2023-12-07T10:36:22.076 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption6", 2023-12-07T10:36:22.076 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.077 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.077 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.077 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.077 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.077 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.077 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.077 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.077 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.078 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.078 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.078 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.078 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption7": { 2023-12-07T10:36:22.078 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption7", 2023-12-07T10:36:22.078 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.078 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.078 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.078 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.079 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1", 2023-12-07T10:36:22.079 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "42", 2023-12-07T10:36:22.079 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.079 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.079 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.079 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.079 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.079 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.079 INFO:teuthology.orchestra.run.smithi119.stdout: "testkey": { 2023-12-07T10:36:22.080 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "testkey", 2023-12-07T10:36:22.080 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.080 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.080 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.080 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.080 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.080 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.080 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.080 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.081 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.081 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.081 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.081 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.081 INFO:teuthology.orchestra.run.smithi119.stdout: "testlkey": { 2023-12-07T10:36:22.081 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "testlkey", 2023-12-07T10:36:22.081 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.081 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.082 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.082 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.082 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.082 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.082 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.082 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.082 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.082 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.082 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.083 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.083 INFO:teuthology.orchestra.run.smithi119.stdout: "testnewline": { 2023-12-07T10:36:22.083 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "testnewline", 2023-12-07T10:36:22.083 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.083 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.083 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.083 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.083 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.083 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.084 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.084 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.084 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.084 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.084 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.084 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.084 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.084 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.084 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.085 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "snap_schedule", 2023-12-07T10:36:22.085 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.085 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.085 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.085 INFO:teuthology.orchestra.run.smithi119.stdout: "allow_m_granularity": { 2023-12-07T10:36:22.085 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "allow_m_granularity", 2023-12-07T10:36:22.085 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.085 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.085 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.086 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.086 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.086 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.086 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.086 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "allow minute scheduled snapshots", 2023-12-07T10:36:22.086 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.086 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.086 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.086 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.087 INFO:teuthology.orchestra.run.smithi119.stdout: "dump_on_update": { 2023-12-07T10:36:22.087 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "dump_on_update", 2023-12-07T10:36:22.087 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.087 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.087 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.087 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.087 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.087 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.087 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.088 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "dump database to debug log on update", 2023-12-07T10:36:22.088 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.088 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.088 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.088 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.088 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.088 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.088 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.088 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.089 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.089 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.089 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.089 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.089 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.089 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.089 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.089 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.089 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.090 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.090 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.090 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.090 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.090 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.090 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.090 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.090 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.090 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.091 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.091 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.091 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.091 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.091 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.091 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.091 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.091 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.091 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.092 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.092 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.092 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.092 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.092 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.092 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.092 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.092 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.092 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.093 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.093 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.093 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.093 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.093 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.093 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.093 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.093 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.094 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.094 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.094 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.094 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.094 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.094 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.094 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.094 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.094 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.095 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.095 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.095 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.095 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.095 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.095 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.095 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.095 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.095 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.096 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.096 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.096 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.096 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.096 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.096 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.096 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.096 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "stats", 2023-12-07T10:36:22.097 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.097 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.097 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.097 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.097 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.097 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.097 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.097 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.097 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.098 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.098 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.098 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.098 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.098 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.098 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.098 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.098 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.098 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.099 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.099 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.099 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.099 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.099 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.099 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.099 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.099 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.100 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.100 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.100 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.100 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.100 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.100 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.100 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.100 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.100 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.101 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.101 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.101 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.101 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.101 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.101 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.101 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.101 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.101 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.102 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.102 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.102 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.102 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.102 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.102 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.102 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.102 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.102 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.103 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.103 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.103 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.103 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.103 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.103 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.103 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.103 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.103 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.104 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.104 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.104 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.104 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.104 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.104 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.104 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.104 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.104 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.105 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.105 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.105 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.105 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.105 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.105 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "status", 2023-12-07T10:36:22.105 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.105 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.105 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.106 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.106 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.106 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.106 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.106 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.106 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.106 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.106 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.107 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.107 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.107 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.107 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.107 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.107 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.107 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.107 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.107 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.108 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.108 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.108 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.108 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.108 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.108 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.108 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.108 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.108 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.108 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.109 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.109 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.109 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.109 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.109 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.109 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.109 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.109 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.110 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.110 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.110 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.110 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.110 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.110 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.110 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.110 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.110 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.111 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.111 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.111 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.111 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.111 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.111 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.111 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.111 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.111 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.112 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.112 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.112 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.112 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.112 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.112 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.112 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.112 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.113 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.113 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.113 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.113 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.113 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.113 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.113 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.113 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.113 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.114 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "telegraf", 2023-12-07T10:36:22.114 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.114 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.114 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.114 INFO:teuthology.orchestra.run.smithi119.stdout: "address": { 2023-12-07T10:36:22.114 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "address", 2023-12-07T10:36:22.115 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.115 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.115 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.115 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-12-07T10:36:22.115 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.115 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.115 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.115 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.115 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.116 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.116 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.116 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2023-12-07T10:36:22.116 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2023-12-07T10:36:22.116 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:22.116 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.116 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.117 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "15", 2023-12-07T10:36:22.117 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.117 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.117 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.117 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.117 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.117 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.117 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.117 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.118 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.118 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.118 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.118 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.118 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.118 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.118 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.118 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.119 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.119 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.119 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.119 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.119 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.119 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.119 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.119 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.119 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.120 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.120 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.120 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.120 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.120 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.120 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.120 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.120 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.120 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.121 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.121 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.121 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.121 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.121 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.121 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.121 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.121 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.122 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.122 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.122 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.122 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.122 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.122 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.122 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.122 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.123 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.123 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.123 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.123 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.123 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.123 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.123 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.123 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.123 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.124 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.124 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.124 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.124 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.124 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.124 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.124 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.125 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.125 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.125 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.125 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.125 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.125 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.125 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.125 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.125 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.126 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.126 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.126 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "telemetry", 2023-12-07T10:36:22.126 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.126 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.126 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.126 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_basic": { 2023-12-07T10:36:22.126 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_basic", 2023-12-07T10:36:22.126 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.127 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.127 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.127 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.127 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.127 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.127 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.127 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share basic cluster information (size, version)", 2023-12-07T10:36:22.127 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.127 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.128 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.128 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.128 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_crash": { 2023-12-07T10:36:22.128 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_crash", 2023-12-07T10:36:22.128 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.128 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.128 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.128 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.128 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.129 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.129 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.129 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-12-07T10:36:22.129 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.129 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.129 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.129 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.129 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_device": { 2023-12-07T10:36:22.129 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_device", 2023-12-07T10:36:22.130 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.130 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.130 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.130 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.130 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.130 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.130 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.130 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-12-07T10:36:22.130 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.131 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.131 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.131 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.131 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_ident": { 2023-12-07T10:36:22.131 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_ident", 2023-12-07T10:36:22.131 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.131 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.131 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.131 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.132 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.132 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.132 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.132 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-12-07T10:36:22.132 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.132 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.132 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.132 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_perf": { 2023-12-07T10:36:22.133 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_perf", 2023-12-07T10:36:22.133 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.133 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.133 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.133 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.133 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.133 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.133 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.133 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share various performance metrics of a cluster", 2023-12-07T10:36:22.134 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.134 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.134 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.134 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.134 INFO:teuthology.orchestra.run.smithi119.stdout: "contact": { 2023-12-07T10:36:22.134 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "contact", 2023-12-07T10:36:22.134 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.134 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.134 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.135 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.135 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.135 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.135 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.135 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.135 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.135 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.135 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.135 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.136 INFO:teuthology.orchestra.run.smithi119.stdout: "description": { 2023-12-07T10:36:22.136 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "description", 2023-12-07T10:36:22.136 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.136 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.136 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.136 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.136 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.136 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.136 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.137 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.137 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.137 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.137 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.137 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.137 INFO:teuthology.orchestra.run.smithi119.stdout: "device_url": { 2023-12-07T10:36:22.137 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "device_url", 2023-12-07T10:36:22.137 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.138 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.138 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.138 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-12-07T10:36:22.138 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.138 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.138 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.138 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.138 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.138 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.139 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.139 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.139 INFO:teuthology.orchestra.run.smithi119.stdout: "enabled": { 2023-12-07T10:36:22.139 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "enabled", 2023-12-07T10:36:22.139 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.139 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.139 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.139 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.139 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.140 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.140 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.140 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.140 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.140 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.140 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.140 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.140 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2023-12-07T10:36:22.140 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2023-12-07T10:36:22.141 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.141 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.141 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.141 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "24", 2023-12-07T10:36:22.141 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "8", 2023-12-07T10:36:22.141 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.141 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.141 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.141 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.142 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.142 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.142 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.142 INFO:teuthology.orchestra.run.smithi119.stdout: "last_opt_revision": { 2023-12-07T10:36:22.142 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "last_opt_revision", 2023-12-07T10:36:22.142 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.142 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.142 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.142 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "1", 2023-12-07T10:36:22.143 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.143 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.143 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.143 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.143 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.143 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.143 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.143 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.143 INFO:teuthology.orchestra.run.smithi119.stdout: "leaderboard": { 2023-12-07T10:36:22.144 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "leaderboard", 2023-12-07T10:36:22.144 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.144 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.144 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.144 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.144 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.144 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.144 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.144 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.145 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.145 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.145 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.145 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.145 INFO:teuthology.orchestra.run.smithi119.stdout: "leaderboard_description": { 2023-12-07T10:36:22.145 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "leaderboard_description", 2023-12-07T10:36:22.145 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.145 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.145 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.146 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.146 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.146 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.146 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.146 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.146 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.146 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.146 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.147 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.147 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.147 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.147 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.147 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.147 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.147 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.147 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.147 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.148 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.148 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.148 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.148 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.148 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.148 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.148 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.148 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.148 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.149 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.149 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.149 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.149 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.149 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.149 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.149 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.149 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.149 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.150 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.150 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.150 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.150 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.150 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.150 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.150 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.150 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.150 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.151 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.151 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.151 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.151 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.151 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.151 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.151 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.151 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.151 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.152 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.152 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.152 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.152 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.152 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.152 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.152 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.152 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.152 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.153 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.153 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.153 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.153 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.153 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.153 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.153 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.153 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.153 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.154 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.154 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.154 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.154 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.154 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.154 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.154 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.154 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.154 INFO:teuthology.orchestra.run.smithi119.stdout: "organization": { 2023-12-07T10:36:22.155 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "organization", 2023-12-07T10:36:22.155 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.155 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.155 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.155 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.155 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.155 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.155 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.155 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.155 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.156 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.156 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.156 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.156 INFO:teuthology.orchestra.run.smithi119.stdout: "proxy": { 2023-12-07T10:36:22.156 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "proxy", 2023-12-07T10:36:22.156 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.156 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.156 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.156 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.157 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.157 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.157 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.157 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.157 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.157 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.157 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.157 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.157 INFO:teuthology.orchestra.run.smithi119.stdout: "url": { 2023-12-07T10:36:22.158 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "url", 2023-12-07T10:36:22.158 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.158 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.158 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.158 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-12-07T10:36:22.158 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.158 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.158 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.158 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.159 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.159 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.159 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.159 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.159 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.159 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.159 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.159 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "test_orchestrator", 2023-12-07T10:36:22.159 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.160 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.160 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.160 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.160 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.160 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.160 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.160 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.160 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.160 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.161 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.161 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.161 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.161 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.161 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.161 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.161 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.161 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.161 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.162 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.162 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.162 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.162 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.162 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.162 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.162 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.162 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.162 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.163 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.163 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.163 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.163 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.163 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.163 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.163 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.163 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.163 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.164 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.164 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.164 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.164 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.164 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.164 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.164 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.164 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.164 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.165 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.165 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.165 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.165 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.165 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.165 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.165 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.165 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.165 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.166 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.166 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.166 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.166 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.166 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.166 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.166 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.166 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.166 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.167 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.167 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.167 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.167 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.167 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.167 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.167 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.167 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.168 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.168 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.168 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.168 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.168 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "volumes", 2023-12-07T10:36:22.168 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.168 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.168 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.168 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.168 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.169 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.169 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.169 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.169 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.169 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.169 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.169 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.169 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.170 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.170 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.170 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.170 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.170 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.170 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.170 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.170 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.170 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.171 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.171 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.171 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.171 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.171 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.171 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.171 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.171 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.172 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.172 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.172 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.172 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.172 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.172 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.172 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.172 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.172 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.173 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.173 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.173 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.173 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.173 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.173 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.173 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.173 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.174 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.174 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.174 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.174 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.174 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.174 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.174 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.174 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.174 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.175 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.175 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.175 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.175 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.175 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.175 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.175 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.176 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.176 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.176 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.176 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.176 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.176 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.176 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.176 INFO:teuthology.orchestra.run.smithi119.stdout: "max_concurrent_clones": { 2023-12-07T10:36:22.176 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_concurrent_clones", 2023-12-07T10:36:22.177 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.177 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.177 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.177 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "4", 2023-12-07T10:36:22.177 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.177 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.177 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.177 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Number of asynchronous cloner threads", 2023-12-07T10:36:22.177 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.178 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.178 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.178 INFO:teuthology.orchestra.run.smithi119.stdout: "periodic_async_work": { 2023-12-07T10:36:22.178 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "periodic_async_work", 2023-12-07T10:36:22.178 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.178 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.178 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.179 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.179 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.179 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.179 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.179 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Periodically check for async work", 2023-12-07T10:36:22.179 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.179 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.179 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.180 INFO:teuthology.orchestra.run.smithi119.stdout: "snapshot_clone_delay": { 2023-12-07T10:36:22.180 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "snapshot_clone_delay", 2023-12-07T10:36:22.180 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.180 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.180 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.180 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2023-12-07T10:36:22.180 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.180 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.180 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.181 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-12-07T10:36:22.181 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.181 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.181 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.181 INFO:teuthology.orchestra.run.smithi119.stdout: "snapshot_clone_no_wait": { 2023-12-07T10:36:22.181 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "snapshot_clone_no_wait", 2023-12-07T10:36:22.181 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.181 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.182 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.182 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.182 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.182 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.182 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.182 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2023-12-07T10:36:22.182 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.182 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.182 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.183 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.183 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.183 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "zabbix", 2023-12-07T10:36:22.183 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.183 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.183 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.184 INFO:teuthology.orchestra.run.smithi119.stdout: "discovery_interval": { 2023-12-07T10:36:22.184 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "discovery_interval", 2023-12-07T10:36:22.184 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2023-12-07T10:36:22.184 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.184 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.184 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "100", 2023-12-07T10:36:22.184 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.184 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.184 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.185 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.185 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.185 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.185 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.185 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.185 INFO:teuthology.orchestra.run.smithi119.stdout: "identifier": { 2023-12-07T10:36:22.185 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "identifier", 2023-12-07T10:36:22.185 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.185 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.186 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.186 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.186 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.186 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.186 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.186 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.186 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.186 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.186 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.187 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.187 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2023-12-07T10:36:22.187 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2023-12-07T10:36:22.187 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:22.187 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.187 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.187 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2023-12-07T10:36:22.187 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.187 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.188 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.188 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.188 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.188 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.188 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.188 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.188 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.189 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.189 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.189 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.189 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.189 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.189 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.189 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.189 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.189 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.190 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.190 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.190 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.190 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.190 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.190 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.190 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.190 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.190 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.191 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.191 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.191 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.191 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.191 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.191 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.191 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.191 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.192 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.192 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.192 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.192 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.192 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.192 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.192 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.192 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.192 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.193 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.193 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.193 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.193 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.193 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.193 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.193 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.193 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.193 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.195 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: "zabbix_host": { 2023-12-07T10:36:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "zabbix_host", 2023-12-07T10:36:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.197 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.197 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.197 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.197 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.197 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.197 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.197 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.197 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.197 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.198 INFO:teuthology.orchestra.run.smithi119.stdout: "zabbix_port": { 2023-12-07T10:36:22.198 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "zabbix_port", 2023-12-07T10:36:22.198 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.198 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.198 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.198 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10051", 2023-12-07T10:36:22.198 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.198 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.198 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.199 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.199 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.199 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.199 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.199 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.199 INFO:teuthology.orchestra.run.smithi119.stdout: "zabbix_sender": { 2023-12-07T10:36:22.199 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "zabbix_sender", 2023-12-07T10:36:22.199 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.200 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.200 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.200 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-12-07T10:36:22.200 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.200 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.200 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.200 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.200 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.200 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.201 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.201 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.201 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.201 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.201 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-12-07T10:36:22.201 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.201 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.201 INFO:teuthology.orchestra.run.smithi119.stdout: "modules": [ 2023-12-07T10:36:22.201 INFO:teuthology.orchestra.run.smithi119.stdout: "cephadm", 2023-12-07T10:36:22.202 INFO:teuthology.orchestra.run.smithi119.stdout: "dashboard", 2023-12-07T10:36:22.202 INFO:teuthology.orchestra.run.smithi119.stdout: "iostat", 2023-12-07T10:36:22.202 INFO:teuthology.orchestra.run.smithi119.stdout: "nfs", 2023-12-07T10:36:22.202 INFO:teuthology.orchestra.run.smithi119.stdout: "restful", 2023-12-07T10:36:22.202 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_schedule" 2023-12-07T10:36:22.202 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.202 INFO:teuthology.orchestra.run.smithi119.stdout: "available_modules": [ 2023-12-07T10:36:22.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.202 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "alerts", 2023-12-07T10:36:22.203 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.203 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.203 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.203 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2023-12-07T10:36:22.203 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2023-12-07T10:36:22.203 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:22.203 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.203 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.203 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2023-12-07T10:36:22.204 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.204 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.204 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.204 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "How frequently to reexamine health status", 2023-12-07T10:36:22.204 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.204 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.204 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.204 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.204 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.205 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.205 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.205 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.205 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.205 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.205 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.205 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.205 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.205 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.206 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.206 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.206 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.206 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.206 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.206 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.206 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.206 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.206 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.212 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.212 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.212 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.212 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.212 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.212 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.212 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.212 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_destination": { 2023-12-07T10:36:22.212 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_destination", 2023-12-07T10:36:22.213 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.213 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.213 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.213 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.213 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.213 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.213 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.213 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Email address to send alerts to", 2023-12-07T10:36:22.213 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.214 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.214 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.214 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.214 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_from_name": { 2023-12-07T10:36:22.214 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_from_name", 2023-12-07T10:36:22.214 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.214 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.214 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.214 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "Ceph", 2023-12-07T10:36:22.215 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.215 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.215 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.215 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Email From: name", 2023-12-07T10:36:22.215 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.215 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.215 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.215 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.215 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_host": { 2023-12-07T10:36:22.216 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_host", 2023-12-07T10:36:22.216 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.216 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.216 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.216 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.216 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.216 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.216 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.216 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "SMTP server", 2023-12-07T10:36:22.217 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.217 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.217 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.217 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.217 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_password": { 2023-12-07T10:36:22.217 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_password", 2023-12-07T10:36:22.217 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.217 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.217 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.218 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.218 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.218 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.218 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.218 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Password to authenticate with", 2023-12-07T10:36:22.218 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.218 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.218 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.219 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.219 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_port": { 2023-12-07T10:36:22.219 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_port", 2023-12-07T10:36:22.219 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.219 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.219 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.219 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "465", 2023-12-07T10:36:22.219 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.219 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.219 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.220 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "SMTP port", 2023-12-07T10:36:22.220 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.220 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.220 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.220 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.220 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_sender": { 2023-12-07T10:36:22.220 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_sender", 2023-12-07T10:36:22.220 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.220 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.221 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.221 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.221 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.221 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.221 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.221 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "SMTP envelope sender", 2023-12-07T10:36:22.221 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.221 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.221 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.222 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.222 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_ssl": { 2023-12-07T10:36:22.222 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_ssl", 2023-12-07T10:36:22.222 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.222 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.222 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.222 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.222 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.222 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.223 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.223 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Use SSL to connect to SMTP server", 2023-12-07T10:36:22.223 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.223 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.223 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.223 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.223 INFO:teuthology.orchestra.run.smithi119.stdout: "smtp_user": { 2023-12-07T10:36:22.223 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "smtp_user", 2023-12-07T10:36:22.223 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.224 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.224 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.224 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.224 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.224 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.224 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.224 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "User to authenticate as", 2023-12-07T10:36:22.224 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.224 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.225 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.225 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.225 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.225 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.225 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.225 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "balancer", 2023-12-07T10:36:22.225 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.225 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.225 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.226 INFO:teuthology.orchestra.run.smithi119.stdout: "active": { 2023-12-07T10:36:22.226 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "active", 2023-12-07T10:36:22.226 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.226 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.226 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.226 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.226 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.226 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.226 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.227 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "automatically balance PGs across cluster", 2023-12-07T10:36:22.227 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.227 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.227 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.227 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.227 INFO:teuthology.orchestra.run.smithi119.stdout: "begin_time": { 2023-12-07T10:36:22.227 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "begin_time", 2023-12-07T10:36:22.227 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.227 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.228 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.228 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0000", 2023-12-07T10:36:22.228 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.228 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.228 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.228 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "beginning time of day to automatically balance", 2023-12-07T10:36:22.228 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-07T10:36:22.228 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.228 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: "begin_weekday": { 2023-12-07T10:36:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "begin_weekday", 2023-12-07T10:36:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2023-12-07T10:36:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2023-12-07T10:36:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "0", 2023-12-07T10:36:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "6", 2023-12-07T10:36:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.230 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-12-07T10:36:22.230 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-07T10:36:22.230 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.230 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.230 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.230 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_compat_max_iterations": { 2023-12-07T10:36:22.230 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crush_compat_max_iterations", 2023-12-07T10:36:22.230 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2023-12-07T10:36:22.231 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.231 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.231 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "25", 2023-12-07T10:36:22.231 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1", 2023-12-07T10:36:22.231 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "250", 2023-12-07T10:36:22.231 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.231 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-12-07T10:36:22.231 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.231 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.232 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.232 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.232 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_compat_metrics": { 2023-12-07T10:36:22.232 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crush_compat_metrics", 2023-12-07T10:36:22.232 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.232 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.232 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.232 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "pgs,objects,bytes", 2023-12-07T10:36:22.232 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.233 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.233 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.233 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-12-07T10:36:22.233 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-12-07T10:36:22.233 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.233 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.233 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.233 INFO:teuthology.orchestra.run.smithi119.stdout: "crush_compat_step": { 2023-12-07T10:36:22.233 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crush_compat_step", 2023-12-07T10:36:22.233 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2023-12-07T10:36:22.234 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.234 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.234 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0.5", 2023-12-07T10:36:22.234 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "0.001", 2023-12-07T10:36:22.234 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "0.999", 2023-12-07T10:36:22.234 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.234 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "aggressiveness of optimization", 2023-12-07T10:36:22.234 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-12-07T10:36:22.235 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.235 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.235 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.235 INFO:teuthology.orchestra.run.smithi119.stdout: "end_time": { 2023-12-07T10:36:22.235 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "end_time", 2023-12-07T10:36:22.235 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.235 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.235 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.235 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "2359", 2023-12-07T10:36:22.235 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.236 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.236 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.236 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "ending time of day to automatically balance", 2023-12-07T10:36:22.236 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-07T10:36:22.236 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.236 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.236 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.236 INFO:teuthology.orchestra.run.smithi119.stdout: "end_weekday": { 2023-12-07T10:36:22.237 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "end_weekday", 2023-12-07T10:36:22.237 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2023-12-07T10:36:22.237 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.237 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.237 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2023-12-07T10:36:22.237 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "0", 2023-12-07T10:36:22.237 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "6", 2023-12-07T10:36:22.237 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.237 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-12-07T10:36:22.238 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-07T10:36:22.238 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.238 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.238 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.238 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.238 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.238 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.238 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.239 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.239 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.239 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.239 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.239 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.239 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.239 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.239 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.239 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.240 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.240 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.240 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.240 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.240 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.240 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.240 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.240 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.241 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.241 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.241 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.241 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.241 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.241 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.241 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.241 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.241 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.242 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.242 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.242 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.242 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.242 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.242 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.242 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.242 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.242 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.243 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.243 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.243 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.243 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.243 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.243 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.243 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.243 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.244 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.244 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.244 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.244 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.244 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.244 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.244 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.244 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.244 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.245 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.245 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.245 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.245 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.245 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.245 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.245 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.245 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.245 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.246 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.246 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.246 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.246 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.246 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.246 INFO:teuthology.orchestra.run.smithi119.stdout: "min_score": { 2023-12-07T10:36:22.246 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "min_score", 2023-12-07T10:36:22.246 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2023-12-07T10:36:22.246 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.247 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.247 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2023-12-07T10:36:22.247 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.247 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.247 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.247 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-12-07T10:36:22.247 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.247 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.247 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.248 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.248 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": { 2023-12-07T10:36:22.248 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mode", 2023-12-07T10:36:22.248 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.248 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.248 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.248 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "upmap", 2023-12-07T10:36:22.248 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.248 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.249 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.249 INFO:teuthology.orchestra.run.smithi119.stdout: "crush-compat", 2023-12-07T10:36:22.249 INFO:teuthology.orchestra.run.smithi119.stdout: "none", 2023-12-07T10:36:22.249 INFO:teuthology.orchestra.run.smithi119.stdout: "upmap" 2023-12-07T10:36:22.249 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.249 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Balancer mode", 2023-12-07T10:36:22.249 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.249 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.250 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.250 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.250 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_ids": { 2023-12-07T10:36:22.250 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "pool_ids", 2023-12-07T10:36:22.250 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.250 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.250 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.250 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.251 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.251 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.251 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.251 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-12-07T10:36:22.251 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.251 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.251 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.251 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.251 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2023-12-07T10:36:22.252 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2023-12-07T10:36:22.252 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:22.252 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.252 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.252 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2023-12-07T10:36:22.252 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.252 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.252 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-12-07T10:36:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: "upmap_max_deviation": { 2023-12-07T10:36:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "upmap_max_deviation", 2023-12-07T10:36:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.253 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.254 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5", 2023-12-07T10:36:22.254 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1", 2023-12-07T10:36:22.254 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.254 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.254 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "deviation below which no optimization is attempted", 2023-12-07T10:36:22.254 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-12-07T10:36:22.254 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.255 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.255 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.255 INFO:teuthology.orchestra.run.smithi119.stdout: "upmap_max_optimizations": { 2023-12-07T10:36:22.255 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "upmap_max_optimizations", 2023-12-07T10:36:22.255 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2023-12-07T10:36:22.255 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.255 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.255 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2023-12-07T10:36:22.255 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.256 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.256 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.256 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-12-07T10:36:22.256 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.256 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.256 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.256 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.256 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.257 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.257 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.257 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cephadm", 2023-12-07T10:36:22.257 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.257 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.257 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.257 INFO:teuthology.orchestra.run.smithi119.stdout: "agent_down_multiplier": { 2023-12-07T10:36:22.257 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "agent_down_multiplier", 2023-12-07T10:36:22.257 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2023-12-07T10:36:22.258 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.258 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.258 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "3.0", 2023-12-07T10:36:22.258 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.258 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.258 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.258 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-12-07T10:36:22.258 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.259 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.259 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.259 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.259 INFO:teuthology.orchestra.run.smithi119.stdout: "agent_refresh_rate": { 2023-12-07T10:36:22.259 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "agent_refresh_rate", 2023-12-07T10:36:22.259 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:22.259 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.259 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.259 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "20", 2023-12-07T10:36:22.260 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.260 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.260 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.260 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-12-07T10:36:22.260 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.260 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.260 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.260 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.260 INFO:teuthology.orchestra.run.smithi119.stdout: "agent_starting_port": { 2023-12-07T10:36:22.261 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "agent_starting_port", 2023-12-07T10:36:22.261 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.261 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.261 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.261 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "4721", 2023-12-07T10:36:22.261 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.261 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.261 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.261 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-12-07T10:36:22.262 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.262 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.262 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.262 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.262 INFO:teuthology.orchestra.run.smithi119.stdout: "allow_ptrace": { 2023-12-07T10:36:22.262 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "allow_ptrace", 2023-12-07T10:36:22.262 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.262 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.262 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.263 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.263 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.263 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.263 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.263 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-12-07T10:36:22.263 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-12-07T10:36:22.263 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.263 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.263 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.264 INFO:teuthology.orchestra.run.smithi119.stdout: "autotune_interval": { 2023-12-07T10:36:22.264 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "autotune_interval", 2023-12-07T10:36:22.264 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:22.264 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.264 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.264 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2023-12-07T10:36:22.264 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.264 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.264 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.265 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to autotune daemon memory", 2023-12-07T10:36:22.265 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.265 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.265 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.265 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.265 INFO:teuthology.orchestra.run.smithi119.stdout: "autotune_memory_target_ratio": { 2023-12-07T10:36:22.265 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "autotune_memory_target_ratio", 2023-12-07T10:36:22.265 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2023-12-07T10:36:22.266 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.266 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.266 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0.7", 2023-12-07T10:36:22.266 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.266 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.266 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.266 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-12-07T10:36:22.266 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.266 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.267 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.267 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.267 INFO:teuthology.orchestra.run.smithi119.stdout: "cephadm_log_destination": { 2023-12-07T10:36:22.267 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cephadm_log_destination", 2023-12-07T10:36:22.267 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.267 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.267 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.267 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.267 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.268 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.268 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.268 INFO:teuthology.orchestra.run.smithi119.stdout: "file", 2023-12-07T10:36:22.268 INFO:teuthology.orchestra.run.smithi119.stdout: "file,syslog", 2023-12-07T10:36:22.268 INFO:teuthology.orchestra.run.smithi119.stdout: "syslog" 2023-12-07T10:36:22.268 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.268 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-12-07T10:36:22.268 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.268 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.269 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.269 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.269 INFO:teuthology.orchestra.run.smithi119.stdout: "cgroups_split": { 2023-12-07T10:36:22.269 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cgroups_split", 2023-12-07T10:36:22.269 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.269 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.269 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.269 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.269 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.270 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.270 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.270 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-12-07T10:36:22.270 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.270 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.270 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.270 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.270 INFO:teuthology.orchestra.run.smithi119.stdout: "config_checks_enabled": { 2023-12-07T10:36:22.271 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "config_checks_enabled", 2023-12-07T10:36:22.271 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.271 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.271 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.271 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.271 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.271 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.271 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.271 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-12-07T10:36:22.272 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.272 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.272 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.272 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.272 INFO:teuthology.orchestra.run.smithi119.stdout: "config_dashboard": { 2023-12-07T10:36:22.272 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "config_dashboard", 2023-12-07T10:36:22.272 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.272 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.272 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.273 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.273 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.273 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.273 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.273 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-12-07T10:36:22.273 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.273 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.273 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.273 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.274 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_alertmanager": { 2023-12-07T10:36:22.274 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_alertmanager", 2023-12-07T10:36:22.274 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.274 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.274 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.274 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-12-07T10:36:22.274 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.274 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.275 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.275 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Prometheus container image", 2023-12-07T10:36:22.275 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.275 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.275 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.275 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.275 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_base": { 2023-12-07T10:36:22.275 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_base", 2023-12-07T10:36:22.275 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.276 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.276 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.276 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/ceph", 2023-12-07T10:36:22.276 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.276 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.276 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.276 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Container image name, without the tag", 2023-12-07T10:36:22.276 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.277 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.277 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.277 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.277 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_elasticsearch": { 2023-12-07T10:36:22.277 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_elasticsearch", 2023-12-07T10:36:22.277 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.277 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.277 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.277 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-12-07T10:36:22.278 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.278 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.278 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.278 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "elasticsearch container image", 2023-12-07T10:36:22.278 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.278 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.278 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.278 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.279 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_grafana": { 2023-12-07T10:36:22.279 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_grafana", 2023-12-07T10:36:22.279 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.279 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.279 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.279 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-12-07T10:36:22.279 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.279 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.279 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.280 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Prometheus container image", 2023-12-07T10:36:22.280 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.280 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.280 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.280 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.280 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_haproxy": { 2023-12-07T10:36:22.280 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_haproxy", 2023-12-07T10:36:22.280 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.280 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.281 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.281 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-12-07T10:36:22.281 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.281 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.281 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.281 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "HAproxy container image", 2023-12-07T10:36:22.281 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.281 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.281 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.282 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.282 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_jaeger_agent": { 2023-12-07T10:36:22.282 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_jaeger_agent", 2023-12-07T10:36:22.282 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.282 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.282 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.282 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-12-07T10:36:22.282 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.283 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.283 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.283 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Jaeger agent container image", 2023-12-07T10:36:22.283 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.283 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.283 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.283 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.283 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_jaeger_collector": { 2023-12-07T10:36:22.283 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_jaeger_collector", 2023-12-07T10:36:22.284 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.284 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.284 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.284 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-12-07T10:36:22.284 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.284 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.284 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.284 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Jaeger collector container image", 2023-12-07T10:36:22.284 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.285 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.285 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.285 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.285 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_jaeger_query": { 2023-12-07T10:36:22.285 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_jaeger_query", 2023-12-07T10:36:22.285 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.285 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.285 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.285 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-12-07T10:36:22.286 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.286 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.286 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.286 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Jaeger query container image", 2023-12-07T10:36:22.286 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.286 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.286 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.286 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.286 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_keepalived": { 2023-12-07T10:36:22.287 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_keepalived", 2023-12-07T10:36:22.287 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.287 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.287 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.287 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-12-07T10:36:22.287 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.287 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.287 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.287 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Keepalived container image", 2023-12-07T10:36:22.288 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.288 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.288 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.288 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.288 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_loki": { 2023-12-07T10:36:22.288 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_loki", 2023-12-07T10:36:22.288 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.288 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.288 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.289 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-12-07T10:36:22.289 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.289 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.289 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.289 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Loki container image", 2023-12-07T10:36:22.289 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.289 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.289 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.289 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.290 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_node_exporter": { 2023-12-07T10:36:22.290 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_node_exporter", 2023-12-07T10:36:22.290 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.290 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.290 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.290 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-12-07T10:36:22.290 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.290 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.290 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.291 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Prometheus container image", 2023-12-07T10:36:22.291 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.291 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.291 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.291 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.291 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_nvmeof": { 2023-12-07T10:36:22.291 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_nvmeof", 2023-12-07T10:36:22.291 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.291 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.292 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.292 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2023-12-07T10:36:22.292 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.292 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.292 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.292 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Nvme-of container image", 2023-12-07T10:36:22.292 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.292 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.292 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.293 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.293 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_prometheus": { 2023-12-07T10:36:22.293 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_prometheus", 2023-12-07T10:36:22.293 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.293 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.293 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.293 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-12-07T10:36:22.293 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.293 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.294 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.294 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Prometheus container image", 2023-12-07T10:36:22.294 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.294 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.294 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.294 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.294 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_promtail": { 2023-12-07T10:36:22.294 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_promtail", 2023-12-07T10:36:22.294 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.295 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.295 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.295 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-12-07T10:36:22.295 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.295 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.295 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.295 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Promtail container image", 2023-12-07T10:36:22.295 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.295 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.296 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.296 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.296 INFO:teuthology.orchestra.run.smithi119.stdout: "container_image_snmp_gateway": { 2023-12-07T10:36:22.296 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_image_snmp_gateway", 2023-12-07T10:36:22.296 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.296 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.296 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.296 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-12-07T10:36:22.296 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.297 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.297 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.297 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "SNMP Gateway container image", 2023-12-07T10:36:22.297 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.297 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.297 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.297 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.297 INFO:teuthology.orchestra.run.smithi119.stdout: "container_init": { 2023-12-07T10:36:22.297 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "container_init", 2023-12-07T10:36:22.298 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.298 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.298 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.298 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.298 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.298 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.298 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.298 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Run podman/docker with `--init`", 2023-12-07T10:36:22.298 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.299 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.299 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.299 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.299 INFO:teuthology.orchestra.run.smithi119.stdout: "daemon_cache_timeout": { 2023-12-07T10:36:22.299 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "daemon_cache_timeout", 2023-12-07T10:36:22.299 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:22.299 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.299 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.299 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2023-12-07T10:36:22.300 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.300 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.300 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.300 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-12-07T10:36:22.300 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.300 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.300 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.300 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.300 INFO:teuthology.orchestra.run.smithi119.stdout: "default_cephadm_command_timeout": { 2023-12-07T10:36:22.301 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "default_cephadm_command_timeout", 2023-12-07T10:36:22.301 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:22.301 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.301 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.301 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "900", 2023-12-07T10:36:22.301 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.301 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.301 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.301 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-12-07T10:36:22.302 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.302 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.302 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.302 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.302 INFO:teuthology.orchestra.run.smithi119.stdout: "default_registry": { 2023-12-07T10:36:22.302 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "default_registry", 2023-12-07T10:36:22.302 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.302 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.303 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.303 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "docker.io", 2023-12-07T10:36:22.303 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.303 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.303 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.303 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-12-07T10:36:22.303 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.303 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.303 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.304 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.304 INFO:teuthology.orchestra.run.smithi119.stdout: "device_cache_timeout": { 2023-12-07T10:36:22.304 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "device_cache_timeout", 2023-12-07T10:36:22.304 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:22.304 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.304 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.304 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "1800", 2023-12-07T10:36:22.304 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.305 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.305 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.305 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "seconds to cache device inventory", 2023-12-07T10:36:22.305 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.305 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.305 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.305 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.305 INFO:teuthology.orchestra.run.smithi119.stdout: "device_enhanced_scan": { 2023-12-07T10:36:22.305 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "device_enhanced_scan", 2023-12-07T10:36:22.306 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.306 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.306 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.306 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.306 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.306 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.306 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.306 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Use libstoragemgmt during device scans", 2023-12-07T10:36:22.307 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.307 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.307 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.307 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.307 INFO:teuthology.orchestra.run.smithi119.stdout: "facts_cache_timeout": { 2023-12-07T10:36:22.307 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "facts_cache_timeout", 2023-12-07T10:36:22.307 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:22.307 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.308 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.308 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2023-12-07T10:36:22.308 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.308 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.308 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.308 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "seconds to cache host facts data", 2023-12-07T10:36:22.308 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.308 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.309 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.309 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.309 INFO:teuthology.orchestra.run.smithi119.stdout: "host_check_interval": { 2023-12-07T10:36:22.309 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "host_check_interval", 2023-12-07T10:36:22.309 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:22.309 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.309 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.309 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2023-12-07T10:36:22.309 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.310 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.310 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.310 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to perform a host check", 2023-12-07T10:36:22.310 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.310 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.310 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.310 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.310 INFO:teuthology.orchestra.run.smithi119.stdout: "inventory_list_all": { 2023-12-07T10:36:22.310 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "inventory_list_all", 2023-12-07T10:36:22.311 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.311 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.311 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.311 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.311 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.311 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.311 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.311 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-12-07T10:36:22.311 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.312 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.312 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.312 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.312 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.312 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.312 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.312 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.312 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.313 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.313 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.313 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.313 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.313 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.313 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.313 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.313 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.313 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.314 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.314 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.314 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.314 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.314 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.314 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.314 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.314 INFO:teuthology.orchestra.run.smithi119.stdout: "log_refresh_metadata": { 2023-12-07T10:36:22.315 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_refresh_metadata", 2023-12-07T10:36:22.315 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.315 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.315 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.315 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.315 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.315 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.315 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.315 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-12-07T10:36:22.316 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.316 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.316 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.316 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.316 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.316 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.316 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.316 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.316 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.316 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.317 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.317 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.317 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.317 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-12-07T10:36:22.317 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.317 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.317 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.317 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.317 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.318 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.318 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.318 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.318 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.318 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.318 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.318 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.318 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.318 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.319 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.319 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.319 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.319 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.319 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.319 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.319 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.319 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.319 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.320 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.320 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.320 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.320 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.320 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.320 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.320 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.320 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.320 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.321 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.321 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.321 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.321 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.321 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.321 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.321 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.321 INFO:teuthology.orchestra.run.smithi119.stdout: "manage_etc_ceph_ceph_conf": { 2023-12-07T10:36:22.321 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-12-07T10:36:22.322 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.322 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.322 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.322 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.322 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.322 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.322 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.322 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-12-07T10:36:22.322 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.323 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.323 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.323 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.323 INFO:teuthology.orchestra.run.smithi119.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-12-07T10:36:22.323 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-12-07T10:36:22.323 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.323 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.323 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.323 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "*", 2023-12-07T10:36:22.324 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.324 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.324 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.324 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-12-07T10:36:22.324 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.324 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.324 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.324 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.324 INFO:teuthology.orchestra.run.smithi119.stdout: "max_count_per_host": { 2023-12-07T10:36:22.325 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_count_per_host", 2023-12-07T10:36:22.325 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.325 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.325 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.325 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2023-12-07T10:36:22.325 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.325 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.325 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.325 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "max number of daemons per service per host", 2023-12-07T10:36:22.326 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.326 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.326 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.326 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.326 INFO:teuthology.orchestra.run.smithi119.stdout: "max_osd_draining_count": { 2023-12-07T10:36:22.326 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_osd_draining_count", 2023-12-07T10:36:22.326 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.326 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.326 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.327 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2023-12-07T10:36:22.327 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.327 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.327 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.327 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-12-07T10:36:22.327 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.327 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.327 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.328 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.328 INFO:teuthology.orchestra.run.smithi119.stdout: "migration_current": { 2023-12-07T10:36:22.328 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "migration_current", 2023-12-07T10:36:22.328 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.328 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.328 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.328 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.328 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.328 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.329 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.329 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "internal - do not modify", 2023-12-07T10:36:22.329 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.329 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.329 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.329 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.329 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": { 2023-12-07T10:36:22.329 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mode", 2023-12-07T10:36:22.329 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.330 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.330 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.330 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "root", 2023-12-07T10:36:22.330 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.330 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.330 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.330 INFO:teuthology.orchestra.run.smithi119.stdout: "cephadm-package", 2023-12-07T10:36:22.330 INFO:teuthology.orchestra.run.smithi119.stdout: "root" 2023-12-07T10:36:22.330 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.331 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "mode for remote execution of cephadm", 2023-12-07T10:36:22.331 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.331 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.331 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.331 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.331 INFO:teuthology.orchestra.run.smithi119.stdout: "prometheus_alerts_path": { 2023-12-07T10:36:22.331 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "prometheus_alerts_path", 2023-12-07T10:36:22.331 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.332 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.332 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.332 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-12-07T10:36:22.332 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.332 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.332 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.332 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-12-07T10:36:22.332 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.332 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.333 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.333 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.333 INFO:teuthology.orchestra.run.smithi119.stdout: "registry_insecure": { 2023-12-07T10:36:22.333 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "registry_insecure", 2023-12-07T10:36:22.333 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.333 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.333 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.333 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.333 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.333 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.334 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.334 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-12-07T10:36:22.334 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.334 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.334 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.334 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.334 INFO:teuthology.orchestra.run.smithi119.stdout: "registry_password": { 2023-12-07T10:36:22.334 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "registry_password", 2023-12-07T10:36:22.334 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.335 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.335 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.335 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.335 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.335 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.335 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.335 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-12-07T10:36:22.335 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.335 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.336 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.336 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.336 INFO:teuthology.orchestra.run.smithi119.stdout: "registry_url": { 2023-12-07T10:36:22.336 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "registry_url", 2023-12-07T10:36:22.336 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.336 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.336 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.336 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.336 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.337 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.337 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.337 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-12-07T10:36:22.337 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.337 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.337 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.337 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.337 INFO:teuthology.orchestra.run.smithi119.stdout: "registry_username": { 2023-12-07T10:36:22.337 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "registry_username", 2023-12-07T10:36:22.338 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.338 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.338 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.338 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.338 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.338 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.338 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.338 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-12-07T10:36:22.338 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.339 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.339 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.339 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.339 INFO:teuthology.orchestra.run.smithi119.stdout: "secure_monitoring_stack": { 2023-12-07T10:36:22.339 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "secure_monitoring_stack", 2023-12-07T10:36:22.339 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.339 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.341 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.341 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.341 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.341 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.341 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.341 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-12-07T10:36:22.341 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.342 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.342 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.342 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.342 INFO:teuthology.orchestra.run.smithi119.stdout: "service_discovery_port": { 2023-12-07T10:36:22.342 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "service_discovery_port", 2023-12-07T10:36:22.342 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.342 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.342 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.342 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8765", 2023-12-07T10:36:22.343 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.343 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.343 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.343 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "cephadm service discovery port", 2023-12-07T10:36:22.343 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.343 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.343 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.343 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.344 INFO:teuthology.orchestra.run.smithi119.stdout: "ssh_config_file": { 2023-12-07T10:36:22.344 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ssh_config_file", 2023-12-07T10:36:22.344 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.344 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.344 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.344 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.344 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.344 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.344 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.345 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-12-07T10:36:22.345 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.345 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.345 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.345 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.345 INFO:teuthology.orchestra.run.smithi119.stdout: "use_agent": { 2023-12-07T10:36:22.345 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "use_agent", 2023-12-07T10:36:22.345 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.346 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.346 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.346 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.346 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.346 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.346 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.346 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-12-07T10:36:22.346 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.346 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.347 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.347 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.347 INFO:teuthology.orchestra.run.smithi119.stdout: "use_repo_digest": { 2023-12-07T10:36:22.347 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "use_repo_digest", 2023-12-07T10:36:22.347 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.347 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.347 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.347 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.348 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.348 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.348 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.348 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-12-07T10:36:22.348 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.348 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.348 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.348 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.348 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_on_failed_host_check": { 2023-12-07T10:36:22.349 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_on_failed_host_check", 2023-12-07T10:36:22.349 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.349 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.349 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.349 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.349 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.349 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.349 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.349 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "raise a health warning if the host check fails", 2023-12-07T10:36:22.350 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.350 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.350 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.350 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.350 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_on_stray_daemons": { 2023-12-07T10:36:22.350 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_on_stray_daemons", 2023-12-07T10:36:22.350 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.350 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.350 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.351 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.351 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.351 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.351 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.351 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-12-07T10:36:22.351 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.351 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.351 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.351 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.352 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_on_stray_hosts": { 2023-12-07T10:36:22.352 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_on_stray_hosts", 2023-12-07T10:36:22.352 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.352 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.352 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.352 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.352 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.352 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.352 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.353 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-12-07T10:36:22.353 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.353 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.353 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.353 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.353 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.353 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.353 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.353 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crash", 2023-12-07T10:36:22.354 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.354 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.354 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.354 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.354 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.354 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.354 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.354 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.354 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.355 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.355 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.355 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.355 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.355 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.355 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.355 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.355 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.356 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.356 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.356 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.356 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.356 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.356 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.356 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.356 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.356 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.357 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.357 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.357 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.357 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.357 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.357 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.357 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.357 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.357 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.358 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.358 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.358 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.358 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.358 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.358 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.358 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.358 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.359 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.359 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.359 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.359 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.359 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.359 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.359 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.359 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.359 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.360 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.360 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.360 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.360 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.360 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.360 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.360 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.360 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.361 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.361 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.361 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.361 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.361 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.361 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.361 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.361 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.361 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.362 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.362 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.362 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.362 INFO:teuthology.orchestra.run.smithi119.stdout: "retain_interval": { 2023-12-07T10:36:22.362 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "retain_interval", 2023-12-07T10:36:22.362 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:22.362 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.362 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.363 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "31536000", 2023-12-07T10:36:22.363 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.363 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.363 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.363 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how long to retain crashes before pruning them", 2023-12-07T10:36:22.363 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.363 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.363 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.363 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.363 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_recent_interval": { 2023-12-07T10:36:22.364 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_recent_interval", 2023-12-07T10:36:22.364 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:22.364 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.364 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.364 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "1209600", 2023-12-07T10:36:22.364 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.364 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.364 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.364 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "time interval in which to warn about recent crashes", 2023-12-07T10:36:22.365 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.365 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.365 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.365 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.365 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.365 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.365 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.365 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "dashboard", 2023-12-07T10:36:22.365 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.366 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.366 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.366 INFO:teuthology.orchestra.run.smithi119.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-12-07T10:36:22.366 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-12-07T10:36:22.366 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.366 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.366 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.366 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2023-12-07T10:36:22.367 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.367 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.367 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.367 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.367 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.367 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.367 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.367 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.367 INFO:teuthology.orchestra.run.smithi119.stdout: "ALERTMANAGER_API_HOST": { 2023-12-07T10:36:22.368 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ALERTMANAGER_API_HOST", 2023-12-07T10:36:22.368 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.368 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.368 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.368 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.368 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.368 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.368 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.368 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.369 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.369 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.369 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.369 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.369 INFO:teuthology.orchestra.run.smithi119.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-12-07T10:36:22.369 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-12-07T10:36:22.369 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.369 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.369 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.370 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.370 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.370 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.370 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.370 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.370 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.370 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.370 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.370 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.371 INFO:teuthology.orchestra.run.smithi119.stdout: "AUDIT_API_ENABLED": { 2023-12-07T10:36:22.371 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "AUDIT_API_ENABLED", 2023-12-07T10:36:22.371 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.371 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.371 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.371 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.371 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.371 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.371 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.372 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.372 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.372 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.372 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.372 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.372 INFO:teuthology.orchestra.run.smithi119.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-12-07T10:36:22.372 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-12-07T10:36:22.372 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.372 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.373 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.373 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.373 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.373 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.373 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.373 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.373 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.373 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.374 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.374 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.374 INFO:teuthology.orchestra.run.smithi119.stdout: "ENABLE_BROWSABLE_API": { 2023-12-07T10:36:22.374 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ENABLE_BROWSABLE_API", 2023-12-07T10:36:22.374 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.374 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.374 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.374 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.374 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.374 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.375 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.375 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.375 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.375 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.375 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.375 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.375 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-12-07T10:36:22.375 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-12-07T10:36:22.375 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.376 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.376 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.376 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.376 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.376 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.376 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.376 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.376 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.377 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.377 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.377 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.377 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-12-07T10:36:22.377 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-12-07T10:36:22.377 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.377 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.377 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.377 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.378 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.378 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.378 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.378 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.378 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.378 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.378 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.378 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.379 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-12-07T10:36:22.379 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-12-07T10:36:22.379 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.379 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.379 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.379 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.379 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.379 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.379 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.380 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.380 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.380 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.380 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.380 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.380 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-12-07T10:36:22.380 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-12-07T10:36:22.380 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.381 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.381 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.381 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.381 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.381 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.381 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.381 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.381 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.381 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.382 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.382 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.382 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_NFS": { 2023-12-07T10:36:22.382 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-12-07T10:36:22.382 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.382 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.382 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.382 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.382 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.383 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.383 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.383 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.383 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.383 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.383 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.383 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.383 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_RBD": { 2023-12-07T10:36:22.384 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-12-07T10:36:22.384 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.384 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.384 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.384 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.384 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.384 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.384 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.384 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.385 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.385 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.385 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.385 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.385 INFO:teuthology.orchestra.run.smithi119.stdout: "FEATURE_TOGGLE_RGW": { 2023-12-07T10:36:22.385 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-12-07T10:36:22.385 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.385 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.385 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.386 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.386 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.386 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.386 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.386 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.386 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.386 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.386 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.386 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.387 INFO:teuthology.orchestra.run.smithi119.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-12-07T10:36:22.387 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-12-07T10:36:22.387 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.387 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.387 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.387 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.387 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.387 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.387 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.388 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.388 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.388 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.388 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.388 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.388 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_API_PASSWORD": { 2023-12-07T10:36:22.388 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_API_PASSWORD", 2023-12-07T10:36:22.388 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.388 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.389 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.389 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "admin", 2023-12-07T10:36:22.389 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.389 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.389 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.389 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.389 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.389 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.390 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.390 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.390 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-12-07T10:36:22.390 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-12-07T10:36:22.390 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.390 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.390 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.390 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.390 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.391 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.391 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.391 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.391 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.391 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.391 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.391 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.391 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_API_URL": { 2023-12-07T10:36:22.391 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_API_URL", 2023-12-07T10:36:22.392 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.392 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.392 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.392 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.392 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.392 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.392 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.392 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.392 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.393 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.393 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.393 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.393 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_API_USERNAME": { 2023-12-07T10:36:22.393 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_API_USERNAME", 2023-12-07T10:36:22.393 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.393 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.393 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.393 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "admin", 2023-12-07T10:36:22.393 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.394 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.394 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.394 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.394 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.394 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.394 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.394 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-12-07T10:36:22.394 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-12-07T10:36:22.395 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.395 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.395 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.395 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.395 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.395 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.395 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.395 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.395 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.396 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.396 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.396 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.396 INFO:teuthology.orchestra.run.smithi119.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-12-07T10:36:22.396 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-12-07T10:36:22.396 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.396 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.396 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.397 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.397 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.397 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.397 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.397 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.397 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.397 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.397 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.397 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.397 INFO:teuthology.orchestra.run.smithi119.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-12-07T10:36:22.398 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-12-07T10:36:22.398 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.398 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.398 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.398 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.398 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.398 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.398 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.398 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.399 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.399 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.399 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.399 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.399 INFO:teuthology.orchestra.run.smithi119.stdout: "ISSUE_TRACKER_API_KEY": { 2023-12-07T10:36:22.399 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-12-07T10:36:22.399 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.399 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.400 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.400 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.400 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.400 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.400 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.400 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.400 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.400 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.400 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.401 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.401 INFO:teuthology.orchestra.run.smithi119.stdout: "PROMETHEUS_API_HOST": { 2023-12-07T10:36:22.401 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PROMETHEUS_API_HOST", 2023-12-07T10:36:22.401 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.401 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.401 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.401 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.401 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.401 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.401 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.402 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.402 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.402 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.402 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.402 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.402 INFO:teuthology.orchestra.run.smithi119.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-12-07T10:36:22.402 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-12-07T10:36:22.402 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.403 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.403 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.403 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.403 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.403 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.403 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.403 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.403 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.403 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.404 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.404 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-12-07T10:36:22.404 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-12-07T10:36:22.404 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.404 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.404 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.404 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.404 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.405 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.405 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.405 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.405 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.405 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.405 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.405 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.405 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-12-07T10:36:22.405 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-12-07T10:36:22.406 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.406 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.406 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.406 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.406 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.406 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.406 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.406 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.406 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.407 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.407 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.407 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.407 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-12-07T10:36:22.407 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-12-07T10:36:22.407 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.407 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.407 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.407 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.408 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.408 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.408 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.408 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.408 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.408 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.408 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.408 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.409 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-12-07T10:36:22.409 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-12-07T10:36:22.409 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.409 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.409 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.409 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.409 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.409 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.409 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.410 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.410 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.410 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.410 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.410 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.410 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-12-07T10:36:22.410 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-12-07T10:36:22.410 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.410 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.411 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.411 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.411 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.411 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.411 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.411 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.411 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.411 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.411 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.412 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.412 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-12-07T10:36:22.412 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-12-07T10:36:22.412 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.412 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.412 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.412 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.412 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.412 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.413 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.413 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.413 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.413 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.413 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.413 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.413 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-12-07T10:36:22.413 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-12-07T10:36:22.413 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.414 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.414 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.414 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.414 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.414 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.414 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.414 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.414 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.414 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.415 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.415 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.415 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_ENABLED": { 2023-12-07T10:36:22.415 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_ENABLED", 2023-12-07T10:36:22.415 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.415 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.415 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.415 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.415 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.416 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.416 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.416 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.416 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.416 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.416 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.416 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.416 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-12-07T10:36:22.416 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-12-07T10:36:22.417 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.417 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.417 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.417 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-12-07T10:36:22.417 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.417 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.417 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.417 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.417 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.418 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.418 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.418 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.418 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-12-07T10:36:22.418 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-12-07T10:36:22.418 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.418 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.418 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.419 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2023-12-07T10:36:22.419 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.419 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.419 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.419 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.419 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.419 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.419 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.419 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.419 INFO:teuthology.orchestra.run.smithi119.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-12-07T10:36:22.420 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-12-07T10:36:22.420 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.420 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.420 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.420 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8", 2023-12-07T10:36:22.420 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.420 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.420 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.421 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.421 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.421 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.421 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.421 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.421 INFO:teuthology.orchestra.run.smithi119.stdout: "REST_REQUESTS_TIMEOUT": { 2023-12-07T10:36:22.421 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-12-07T10:36:22.421 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.421 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.422 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.422 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "45", 2023-12-07T10:36:22.422 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.422 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.422 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.422 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.422 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.422 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.422 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.423 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.423 INFO:teuthology.orchestra.run.smithi119.stdout: "RGW_API_ACCESS_KEY": { 2023-12-07T10:36:22.423 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "RGW_API_ACCESS_KEY", 2023-12-07T10:36:22.423 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.423 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.423 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.423 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.423 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.423 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.424 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.424 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.424 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.424 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.424 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.424 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.424 INFO:teuthology.orchestra.run.smithi119.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-12-07T10:36:22.424 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-12-07T10:36:22.424 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.424 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.425 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.425 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "admin", 2023-12-07T10:36:22.425 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.425 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.425 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.425 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.425 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.425 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.425 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.426 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.426 INFO:teuthology.orchestra.run.smithi119.stdout: "RGW_API_SECRET_KEY": { 2023-12-07T10:36:22.426 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "RGW_API_SECRET_KEY", 2023-12-07T10:36:22.426 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.426 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.426 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.426 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.426 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.426 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.427 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.427 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.427 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.427 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.427 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.427 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.427 INFO:teuthology.orchestra.run.smithi119.stdout: "RGW_API_SSL_VERIFY": { 2023-12-07T10:36:22.427 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "RGW_API_SSL_VERIFY", 2023-12-07T10:36:22.427 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.428 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.428 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.428 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.428 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.428 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.428 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.428 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.428 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.428 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.429 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.429 INFO:teuthology.orchestra.run.smithi119.stdout: "UNSAFE_TLS_v1_2": { 2023-12-07T10:36:22.429 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "UNSAFE_TLS_v1_2", 2023-12-07T10:36:22.429 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.429 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.429 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.429 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.429 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.430 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.430 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.430 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.430 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.430 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.430 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.430 INFO:teuthology.orchestra.run.smithi119.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-12-07T10:36:22.431 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-12-07T10:36:22.431 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.431 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.431 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.431 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2023-12-07T10:36:22.431 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.431 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.431 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.431 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.432 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.432 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.432 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.432 INFO:teuthology.orchestra.run.smithi119.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-12-07T10:36:22.432 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-12-07T10:36:22.432 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.432 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.432 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.433 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2023-12-07T10:36:22.433 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.433 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.433 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.433 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.433 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.433 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.433 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.434 INFO:teuthology.orchestra.run.smithi119.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-12-07T10:36:22.434 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-12-07T10:36:22.434 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.434 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.434 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.434 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5", 2023-12-07T10:36:22.434 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.434 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.434 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.435 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.435 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.435 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.435 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.435 INFO:teuthology.orchestra.run.smithi119.stdout: "cross_origin_url": { 2023-12-07T10:36:22.435 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cross_origin_url", 2023-12-07T10:36:22.435 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.435 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.436 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.436 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.436 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.436 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.436 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.436 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.436 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.436 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.436 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.437 INFO:teuthology.orchestra.run.smithi119.stdout: "crt_file": { 2023-12-07T10:36:22.437 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "crt_file", 2023-12-07T10:36:22.437 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.437 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.437 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.437 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.437 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.437 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.438 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.438 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.438 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.438 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.438 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.438 INFO:teuthology.orchestra.run.smithi119.stdout: "debug": { 2023-12-07T10:36:22.438 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "debug", 2023-12-07T10:36:22.438 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.439 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.439 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.439 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.439 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.439 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.439 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.439 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Enable/disable debug options", 2023-12-07T10:36:22.439 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.440 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.440 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.440 INFO:teuthology.orchestra.run.smithi119.stdout: "jwt_token_ttl": { 2023-12-07T10:36:22.440 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "jwt_token_ttl", 2023-12-07T10:36:22.440 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.440 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.440 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.440 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "28800", 2023-12-07T10:36:22.440 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.441 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.441 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.441 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.441 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.441 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.441 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.441 INFO:teuthology.orchestra.run.smithi119.stdout: "key_file": { 2023-12-07T10:36:22.441 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "key_file", 2023-12-07T10:36:22.442 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.442 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.442 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.442 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.442 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.442 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.442 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.442 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.442 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.443 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.443 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.443 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.443 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.443 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.443 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.443 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.443 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.443 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.444 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.444 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.444 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.444 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.444 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.444 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.444 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.444 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.444 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.445 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.445 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.445 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.445 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.445 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.445 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.445 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.445 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.445 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.446 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.446 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.446 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.446 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.446 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.446 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.446 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.446 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.447 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.447 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.447 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.447 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.447 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.447 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.447 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.447 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.447 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.447 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.448 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.448 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.448 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.448 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.448 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.448 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.448 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.448 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.448 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.449 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.449 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.449 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.449 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.449 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.449 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.449 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.449 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.450 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.450 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.450 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.450 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.450 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.450 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.450 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.450 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.450 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.451 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.451 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.451 INFO:teuthology.orchestra.run.smithi119.stdout: "motd": { 2023-12-07T10:36:22.451 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "motd", 2023-12-07T10:36:22.451 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.451 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.451 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.451 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.451 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.452 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.452 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.452 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "The message of the day", 2023-12-07T10:36:22.452 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.452 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.452 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.452 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.452 INFO:teuthology.orchestra.run.smithi119.stdout: "redirect_resolve_ip_addr": { 2023-12-07T10:36:22.452 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "redirect_resolve_ip_addr", 2023-12-07T10:36:22.453 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.453 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.453 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.453 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.453 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.453 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.453 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.453 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.453 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.454 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.454 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.454 INFO:teuthology.orchestra.run.smithi119.stdout: "server_addr": { 2023-12-07T10:36:22.454 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_addr", 2023-12-07T10:36:22.454 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.454 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.454 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.454 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "::", 2023-12-07T10:36:22.455 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.455 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.455 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.455 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.455 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.455 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.455 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.455 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.455 INFO:teuthology.orchestra.run.smithi119.stdout: "server_port": { 2023-12-07T10:36:22.456 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_port", 2023-12-07T10:36:22.456 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.456 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.456 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.456 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8080", 2023-12-07T10:36:22.456 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.456 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.456 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.456 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.457 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.457 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.457 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.457 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.457 INFO:teuthology.orchestra.run.smithi119.stdout: "ssl": { 2023-12-07T10:36:22.457 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ssl", 2023-12-07T10:36:22.457 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.457 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.457 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.458 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.458 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.458 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.458 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.458 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.458 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.458 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.458 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.458 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.459 INFO:teuthology.orchestra.run.smithi119.stdout: "ssl_server_port": { 2023-12-07T10:36:22.459 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ssl_server_port", 2023-12-07T10:36:22.459 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.459 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.459 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.459 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8443", 2023-12-07T10:36:22.459 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.459 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.460 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.460 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.460 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.460 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.460 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.460 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.460 INFO:teuthology.orchestra.run.smithi119.stdout: "standby_behaviour": { 2023-12-07T10:36:22.460 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "standby_behaviour", 2023-12-07T10:36:22.460 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.461 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.461 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.461 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "redirect", 2023-12-07T10:36:22.461 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.461 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.461 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.461 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.461 INFO:teuthology.orchestra.run.smithi119.stdout: "redirect" 2023-12-07T10:36:22.461 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.461 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.462 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.462 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.462 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.462 INFO:teuthology.orchestra.run.smithi119.stdout: "standby_error_status_code": { 2023-12-07T10:36:22.462 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "standby_error_status_code", 2023-12-07T10:36:22.462 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.462 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.463 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.463 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "500", 2023-12-07T10:36:22.463 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "400", 2023-12-07T10:36:22.463 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "599", 2023-12-07T10:36:22.463 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.463 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.463 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.463 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.463 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.464 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.464 INFO:teuthology.orchestra.run.smithi119.stdout: "url_prefix": { 2023-12-07T10:36:22.464 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "url_prefix", 2023-12-07T10:36:22.464 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.464 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.464 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.464 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.464 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.464 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.465 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.465 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.465 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.465 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.465 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.465 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.465 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.466 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "devicehealth", 2023-12-07T10:36:22.466 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.466 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.466 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.466 INFO:teuthology.orchestra.run.smithi119.stdout: "enable_monitoring": { 2023-12-07T10:36:22.466 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "enable_monitoring", 2023-12-07T10:36:22.466 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.466 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.466 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.467 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.467 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.467 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.467 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.467 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "monitor device health metrics", 2023-12-07T10:36:22.467 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.467 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.467 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.468 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.468 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.468 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.468 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.468 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.468 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.468 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.468 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.468 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.469 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.469 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.469 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.469 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.469 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.469 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.469 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.469 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.469 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.470 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.470 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.470 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.470 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.470 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.470 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.470 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.471 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.471 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.471 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.471 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.471 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.471 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.471 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.471 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.472 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.472 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.472 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.472 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.472 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.472 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.472 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.472 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.472 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.472 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.473 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.473 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.473 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.473 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.473 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.473 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.473 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.473 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.473 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.474 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.474 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.474 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.474 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.474 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.474 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.474 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.475 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.475 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.475 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.475 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.475 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.475 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.475 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.475 INFO:teuthology.orchestra.run.smithi119.stdout: "mark_out_threshold": { 2023-12-07T10:36:22.476 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mark_out_threshold", 2023-12-07T10:36:22.476 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:22.476 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.476 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.476 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "2419200", 2023-12-07T10:36:22.476 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.476 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.476 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.476 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-12-07T10:36:22.477 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.477 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.477 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.477 INFO:teuthology.orchestra.run.smithi119.stdout: "pool_name": { 2023-12-07T10:36:22.477 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "pool_name", 2023-12-07T10:36:22.477 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.477 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.477 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.478 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "device_health_metrics", 2023-12-07T10:36:22.478 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.478 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.478 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.478 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "name of pool in which to store device health metrics", 2023-12-07T10:36:22.478 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.478 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.478 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.479 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.479 INFO:teuthology.orchestra.run.smithi119.stdout: "retention_period": { 2023-12-07T10:36:22.479 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "retention_period", 2023-12-07T10:36:22.479 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:22.479 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.479 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.479 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "15552000", 2023-12-07T10:36:22.479 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.479 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.480 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.480 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how long to retain device health metrics", 2023-12-07T10:36:22.480 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.480 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.480 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.480 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.480 INFO:teuthology.orchestra.run.smithi119.stdout: "scrape_frequency": { 2023-12-07T10:36:22.480 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "scrape_frequency", 2023-12-07T10:36:22.480 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:22.480 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.481 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.481 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "86400", 2023-12-07T10:36:22.481 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.481 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.481 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.481 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to scrape device health metrics", 2023-12-07T10:36:22.481 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.481 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.481 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.482 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.482 INFO:teuthology.orchestra.run.smithi119.stdout: "self_heal": { 2023-12-07T10:36:22.482 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "self_heal", 2023-12-07T10:36:22.482 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.482 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.482 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.482 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.482 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.482 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.483 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.483 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-12-07T10:36:22.483 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.483 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.483 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.483 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.483 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2023-12-07T10:36:22.483 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2023-12-07T10:36:22.483 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:22.484 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.484 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.484 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2023-12-07T10:36:22.484 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.484 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.484 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.484 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how frequently to wake up and check device health", 2023-12-07T10:36:22.484 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.484 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.485 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.485 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.485 INFO:teuthology.orchestra.run.smithi119.stdout: "warn_threshold": { 2023-12-07T10:36:22.485 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "warn_threshold", 2023-12-07T10:36:22.485 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:22.485 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.485 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.485 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "7257600", 2023-12-07T10:36:22.485 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.486 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.486 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.486 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-12-07T10:36:22.486 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.486 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.486 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.486 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.486 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.486 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.487 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.487 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "diskprediction_local", 2023-12-07T10:36:22.487 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.487 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.487 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.487 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.487 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.487 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.487 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.488 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.488 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.488 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.488 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.488 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.488 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.488 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.488 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.488 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.489 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.489 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.489 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.489 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.489 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.489 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.489 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.489 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.489 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.490 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.490 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.490 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.490 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.490 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.490 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.490 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.490 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.491 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.491 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.491 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.491 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.491 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.491 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.491 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.491 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.492 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.492 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.492 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.492 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.492 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.492 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.492 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.492 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.493 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.493 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.493 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.493 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.493 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.493 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.493 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.493 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.494 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.494 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.494 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.494 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.494 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.494 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.494 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.494 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.495 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.495 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.495 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.495 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.495 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.495 INFO:teuthology.orchestra.run.smithi119.stdout: "predict_interval": { 2023-12-07T10:36:22.495 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "predict_interval", 2023-12-07T10:36:22.495 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.496 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.496 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.496 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "86400", 2023-12-07T10:36:22.496 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.496 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.496 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.496 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.496 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.496 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.497 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.497 INFO:teuthology.orchestra.run.smithi119.stdout: "predictor_model": { 2023-12-07T10:36:22.497 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "predictor_model", 2023-12-07T10:36:22.497 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.497 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.497 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.497 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "prophetstor", 2023-12-07T10:36:22.498 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.498 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.498 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.498 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.498 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.498 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.498 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.498 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2023-12-07T10:36:22.499 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2023-12-07T10:36:22.499 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.499 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.499 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.499 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "600", 2023-12-07T10:36:22.499 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.499 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.499 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.499 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.500 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.500 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.500 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.500 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.500 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.500 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "influx", 2023-12-07T10:36:22.500 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": false, 2023-12-07T10:36:22.501 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "influxdb python module not found", 2023-12-07T10:36:22.501 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.501 INFO:teuthology.orchestra.run.smithi119.stdout: "batch_size": { 2023-12-07T10:36:22.501 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "batch_size", 2023-12-07T10:36:22.501 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.501 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.501 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.501 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5000", 2023-12-07T10:36:22.502 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.502 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.502 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.502 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-12-07T10:36:22.502 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.502 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.502 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.502 INFO:teuthology.orchestra.run.smithi119.stdout: "database": { 2023-12-07T10:36:22.503 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "database", 2023-12-07T10:36:22.503 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.503 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.503 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.503 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "ceph", 2023-12-07T10:36:22.503 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.503 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.503 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.503 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-12-07T10:36:22.504 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.504 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.504 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.504 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.504 INFO:teuthology.orchestra.run.smithi119.stdout: "hostname": { 2023-12-07T10:36:22.504 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "hostname", 2023-12-07T10:36:22.504 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.504 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.504 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.505 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.505 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.505 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.505 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.505 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "InfluxDB server hostname", 2023-12-07T10:36:22.505 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.505 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.505 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.506 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2023-12-07T10:36:22.506 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2023-12-07T10:36:22.506 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:22.506 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.506 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.506 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "30", 2023-12-07T10:36:22.506 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "5", 2023-12-07T10:36:22.506 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.507 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.507 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-12-07T10:36:22.507 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.507 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.507 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.507 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.507 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.507 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.508 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.508 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.508 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.508 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.508 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.508 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.508 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.508 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.508 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.509 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.509 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.509 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.509 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.509 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.509 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.509 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.509 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.510 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.510 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.510 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.510 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.510 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.510 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.510 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.510 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.510 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.511 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.511 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.511 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.511 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.511 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.512 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.512 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.512 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.512 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.512 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.512 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.512 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.512 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.513 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.513 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.513 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.513 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.513 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.513 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.513 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.513 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.513 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.514 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.514 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.514 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.514 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.514 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.514 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.514 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.514 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.515 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.515 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.515 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.515 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.515 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.515 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.515 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.515 INFO:teuthology.orchestra.run.smithi119.stdout: "password": { 2023-12-07T10:36:22.516 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "password", 2023-12-07T10:36:22.516 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.516 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.516 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.516 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.516 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.516 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.516 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.517 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "password of InfluxDB server user", 2023-12-07T10:36:22.517 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.517 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.517 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.517 INFO:teuthology.orchestra.run.smithi119.stdout: "port": { 2023-12-07T10:36:22.517 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "port", 2023-12-07T10:36:22.517 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.517 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.517 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.518 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "8086", 2023-12-07T10:36:22.518 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.518 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.518 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.518 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "InfluxDB server port", 2023-12-07T10:36:22.518 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.518 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.518 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.519 INFO:teuthology.orchestra.run.smithi119.stdout: "ssl": { 2023-12-07T10:36:22.519 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ssl", 2023-12-07T10:36:22.519 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.519 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.519 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.519 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "false", 2023-12-07T10:36:22.519 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.519 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.519 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.520 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-12-07T10:36:22.520 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.520 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.520 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.520 INFO:teuthology.orchestra.run.smithi119.stdout: "threads": { 2023-12-07T10:36:22.520 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "threads", 2023-12-07T10:36:22.520 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.520 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.521 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.521 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5", 2023-12-07T10:36:22.521 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1", 2023-12-07T10:36:22.521 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "32", 2023-12-07T10:36:22.521 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.521 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-12-07T10:36:22.521 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.521 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.521 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.522 INFO:teuthology.orchestra.run.smithi119.stdout: "username": { 2023-12-07T10:36:22.522 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "username", 2023-12-07T10:36:22.522 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.522 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.522 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.522 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.522 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.522 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.523 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.523 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "username of InfluxDB server user", 2023-12-07T10:36:22.523 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.523 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.523 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.523 INFO:teuthology.orchestra.run.smithi119.stdout: "verify_ssl": { 2023-12-07T10:36:22.523 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "verify_ssl", 2023-12-07T10:36:22.523 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.524 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.524 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.524 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "true", 2023-12-07T10:36:22.524 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.524 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.524 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.524 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-12-07T10:36:22.524 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.524 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.524 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.525 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.525 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.525 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.525 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "insights", 2023-12-07T10:36:22.525 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.525 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.525 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.525 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.526 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.526 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.526 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.526 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.526 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.526 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.526 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.526 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.527 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.527 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.527 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.527 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.527 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.527 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.527 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.527 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.527 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.528 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.528 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.528 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.528 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.528 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.528 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.528 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.528 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.529 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.529 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.529 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.529 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.529 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.529 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.529 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.529 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.530 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.530 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.530 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.530 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.530 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.530 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.530 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.530 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.531 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.531 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.531 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.531 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.531 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.531 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.531 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.531 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.531 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.531 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.532 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.532 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.532 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.532 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.532 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.532 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.532 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.532 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.533 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.533 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.533 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.533 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.533 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.533 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.533 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.533 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.534 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.534 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "iostat", 2023-12-07T10:36:22.534 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.534 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.534 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.534 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.534 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.534 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.534 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.534 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.535 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.535 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.535 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.535 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.535 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.535 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.535 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.535 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.535 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.536 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.536 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.536 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.536 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.536 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.536 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.536 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.536 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.537 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.537 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.537 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.537 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.537 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.537 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.537 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.537 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.537 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.538 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.538 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.538 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.538 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.538 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.538 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.538 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.539 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.539 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.539 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.539 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.539 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.539 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.539 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.539 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.539 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.539 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.540 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.540 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.540 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.540 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.540 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.540 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.540 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.540 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.541 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.541 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.541 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.541 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.541 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.541 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.541 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.541 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.541 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.542 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.542 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.542 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.542 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "k8sevents", 2023-12-07T10:36:22.542 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.542 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.542 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.543 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_event_retention_days": { 2023-12-07T10:36:22.543 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "ceph_event_retention_days", 2023-12-07T10:36:22.543 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.543 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.543 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.543 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "7", 2023-12-07T10:36:22.543 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.543 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.543 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.544 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Days to hold ceph event information within local cache", 2023-12-07T10:36:22.544 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.544 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.544 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.544 INFO:teuthology.orchestra.run.smithi119.stdout: "config_check_secs": { 2023-12-07T10:36:22.544 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "config_check_secs", 2023-12-07T10:36:22.544 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.545 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.545 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.545 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2023-12-07T10:36:22.545 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "10", 2023-12-07T10:36:22.545 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.545 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.545 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-12-07T10:36:22.545 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.546 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.546 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.546 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.546 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.546 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.546 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.546 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.546 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.547 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.547 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.547 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.547 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.547 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.547 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.547 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.547 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.547 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.548 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.548 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.548 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.548 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.548 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.548 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.548 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.548 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.549 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.549 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.549 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.549 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.549 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.549 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.549 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.549 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.549 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.550 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.550 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.550 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.550 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.550 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.550 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.550 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.550 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.551 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.551 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.551 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.551 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.551 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.551 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.551 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.553 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.553 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.553 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.553 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.553 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.553 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.554 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.554 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.554 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.554 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.554 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.554 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.554 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.554 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.554 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.555 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.555 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.555 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.555 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.555 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.555 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.555 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.555 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.556 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "localpool", 2023-12-07T10:36:22.556 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.556 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.556 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.556 INFO:teuthology.orchestra.run.smithi119.stdout: "failure_domain": { 2023-12-07T10:36:22.556 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "failure_domain", 2023-12-07T10:36:22.556 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.556 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.556 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.557 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "host", 2023-12-07T10:36:22.557 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.557 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.557 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.557 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "failure domain for any created local pool", 2023-12-07T10:36:22.557 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-12-07T10:36:22.557 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.557 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.558 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.558 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.558 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.558 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.558 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.558 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.558 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.558 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.559 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.559 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.559 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.559 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.559 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.559 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.559 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.559 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.559 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.559 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.560 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.560 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.560 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.560 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.560 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.560 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.560 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.560 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.561 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.561 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.561 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.561 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.561 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.561 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.561 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.561 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.562 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.562 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.562 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.562 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.562 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.562 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.562 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.562 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.562 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.563 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.563 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.563 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.563 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.563 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.563 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.563 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.563 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.563 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.564 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.564 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.564 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.564 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.564 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.564 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.564 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.564 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.565 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.565 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.565 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.565 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.565 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.565 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.565 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.565 INFO:teuthology.orchestra.run.smithi119.stdout: "min_size": { 2023-12-07T10:36:22.566 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "min_size", 2023-12-07T10:36:22.566 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.566 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.566 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.566 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.566 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.566 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.566 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.566 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "default min_size for any created local pool", 2023-12-07T10:36:22.567 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-12-07T10:36:22.567 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.567 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.567 INFO:teuthology.orchestra.run.smithi119.stdout: "num_rep": { 2023-12-07T10:36:22.567 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "num_rep", 2023-12-07T10:36:22.567 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.567 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.568 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.568 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "3", 2023-12-07T10:36:22.568 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.568 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.568 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.568 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "default replica count for any created local pool", 2023-12-07T10:36:22.568 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.568 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.569 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.569 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_num": { 2023-12-07T10:36:22.569 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "pg_num", 2023-12-07T10:36:22.569 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.569 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.569 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.569 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "128", 2023-12-07T10:36:22.569 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.570 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.570 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.570 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "default pg_num for any created local pool", 2023-12-07T10:36:22.570 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.570 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.570 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.570 INFO:teuthology.orchestra.run.smithi119.stdout: "prefix": { 2023-12-07T10:36:22.571 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "prefix", 2023-12-07T10:36:22.571 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.571 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.571 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.571 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.571 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.571 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.571 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.571 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "name prefix for any created local pool", 2023-12-07T10:36:22.571 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.572 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.572 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.572 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.572 INFO:teuthology.orchestra.run.smithi119.stdout: "subtree": { 2023-12-07T10:36:22.572 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "subtree", 2023-12-07T10:36:22.572 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.572 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.572 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.572 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "rack", 2023-12-07T10:36:22.573 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.573 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.573 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.573 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "CRUSH level for which to create a local pool", 2023-12-07T10:36:22.573 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-12-07T10:36:22.573 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.573 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.573 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.573 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.574 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.574 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.574 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mds_autoscaler", 2023-12-07T10:36:22.574 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.574 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.574 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.574 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.574 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.575 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.575 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.575 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.575 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.575 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.575 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.575 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.575 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.575 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.576 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.576 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.576 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.576 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.576 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.576 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.576 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.576 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.577 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.577 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.577 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.577 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.577 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.577 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.577 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.577 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.578 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.578 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.578 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.578 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.578 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.578 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.578 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.578 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.578 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.579 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.579 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.579 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.579 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.579 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.579 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.579 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.579 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.579 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.580 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.580 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.580 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.580 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.580 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.580 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.580 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.580 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.580 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.581 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.581 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.581 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.581 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.581 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.581 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.581 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.581 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.582 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.582 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.582 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.582 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.582 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.582 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.582 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.582 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.582 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.583 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mirroring", 2023-12-07T10:36:22.583 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.583 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.583 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.583 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.583 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.583 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.583 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.583 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.584 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.584 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.584 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.584 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.584 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.584 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.584 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.584 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.584 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.585 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.585 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.585 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.585 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.585 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.585 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.585 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.585 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.586 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.586 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.586 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.586 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.586 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.586 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.586 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.586 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.586 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.587 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.587 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.587 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.587 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.587 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.587 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.587 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.587 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.588 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.588 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.588 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.588 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.588 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.588 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.588 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.588 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.588 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.589 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.589 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.589 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.589 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.589 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.589 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.589 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.589 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.590 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.590 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.590 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.590 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.590 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.590 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.590 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.590 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.590 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.591 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.591 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.591 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.591 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "nfs", 2023-12-07T10:36:22.591 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.591 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.592 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.592 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.592 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.592 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.592 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.592 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.592 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.592 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.592 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.593 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.593 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.593 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.593 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.593 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.593 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.593 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.593 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.593 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.594 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.594 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.594 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.594 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.594 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.594 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.594 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.595 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.595 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.595 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.595 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.595 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.595 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.595 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.595 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.595 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.596 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.596 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.596 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.596 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.596 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.596 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.596 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.596 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.597 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.597 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.597 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.597 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.597 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.597 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.597 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.597 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.598 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.598 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.598 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.598 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.598 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.598 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.598 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.598 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.599 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.599 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.599 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.599 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.599 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.599 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.599 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.599 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.599 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.599 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.600 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.600 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "orchestrator", 2023-12-07T10:36:22.600 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.600 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.600 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.600 INFO:teuthology.orchestra.run.smithi119.stdout: "fail_fs": { 2023-12-07T10:36:22.600 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "fail_fs", 2023-12-07T10:36:22.601 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.601 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.601 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.601 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.601 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.601 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.601 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.601 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-12-07T10:36:22.601 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.601 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.602 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.602 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.602 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.602 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.602 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.602 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.602 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.602 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.603 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.603 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.603 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.603 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.603 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.603 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.603 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.603 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.603 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.603 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.604 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.604 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.604 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.604 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.604 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.604 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.604 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.604 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.604 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.605 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.605 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.605 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.605 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.605 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.605 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.605 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.605 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.605 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.606 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.606 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.606 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.606 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.606 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.606 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.606 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.606 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.606 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.607 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.607 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.607 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.607 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.607 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.607 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.607 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.607 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.607 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.608 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.608 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.608 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.608 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.608 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.608 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.608 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.608 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.608 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.609 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.609 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.609 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.609 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.609 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.609 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.609 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.609 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.610 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.610 INFO:teuthology.orchestra.run.smithi119.stdout: "orchestrator": { 2023-12-07T10:36:22.610 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "orchestrator", 2023-12-07T10:36:22.610 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.610 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.610 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.610 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.610 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.610 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.611 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.611 INFO:teuthology.orchestra.run.smithi119.stdout: "cephadm", 2023-12-07T10:36:22.611 INFO:teuthology.orchestra.run.smithi119.stdout: "rook", 2023-12-07T10:36:22.611 INFO:teuthology.orchestra.run.smithi119.stdout: "test_orchestrator" 2023-12-07T10:36:22.611 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.611 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Orchestrator backend", 2023-12-07T10:36:22.611 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.611 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.611 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.612 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.612 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.612 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.612 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.612 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "osd_perf_query", 2023-12-07T10:36:22.612 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.612 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.612 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.613 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.613 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.613 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.613 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.613 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.613 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.613 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.613 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.613 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.614 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.614 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.614 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.614 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.614 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.614 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.614 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.614 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.615 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.615 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.615 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.615 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.615 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.615 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.615 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.615 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.615 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.616 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.616 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.616 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.616 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.616 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.616 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.616 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.616 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.616 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.617 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.617 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.617 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.617 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.617 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.617 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.617 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.617 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.617 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.618 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.618 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.618 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.618 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.618 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.618 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.618 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.618 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.618 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.619 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.619 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.619 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.619 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.619 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.619 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.619 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.619 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.620 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.620 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.620 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.620 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.620 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.620 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.620 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.620 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.621 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.621 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.621 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.621 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.621 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "osd_support", 2023-12-07T10:36:22.621 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.621 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.621 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.621 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.622 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.622 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.622 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.622 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.622 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.622 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.622 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.622 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.623 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.623 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.623 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.623 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.623 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.623 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.623 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.623 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.624 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.624 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.624 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.624 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.624 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.624 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.624 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.624 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.624 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.625 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.625 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.625 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.625 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.625 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.625 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.625 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.625 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.625 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.626 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.626 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.626 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.626 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.626 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.626 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.626 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.626 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.627 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.627 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.627 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.627 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.627 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.627 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.627 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.627 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.627 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.628 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.628 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.628 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.628 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.628 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.628 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.628 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.628 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.628 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.629 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.629 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.629 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.629 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.629 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.629 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.629 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.629 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.629 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.630 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.630 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "pg_autoscaler", 2023-12-07T10:36:22.630 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.630 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.630 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.630 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.630 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.631 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.631 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.631 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.631 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.631 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.631 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.631 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.631 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.631 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.632 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.632 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.632 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.632 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.632 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.632 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.632 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.632 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.633 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.633 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.633 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.633 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.633 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.633 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.633 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.633 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.634 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.634 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.634 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.634 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.634 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.634 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.634 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.635 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.635 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.635 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.635 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.635 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.635 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.635 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.635 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.635 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.636 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.636 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.636 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.636 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.636 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.636 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.636 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.636 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.636 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.637 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.637 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.637 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.637 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.637 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.637 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.637 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.637 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.638 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.638 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.638 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.638 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.638 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.638 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.638 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2023-12-07T10:36:22.638 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2023-12-07T10:36:22.639 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:22.639 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.639 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.639 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2023-12-07T10:36:22.639 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.639 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.639 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.639 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.639 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.640 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.640 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.640 INFO:teuthology.orchestra.run.smithi119.stdout: "threshold": { 2023-12-07T10:36:22.640 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "threshold", 2023-12-07T10:36:22.640 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2023-12-07T10:36:22.640 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.640 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.640 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "3.0", 2023-12-07T10:36:22.640 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1.0", 2023-12-07T10:36:22.641 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.641 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.641 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "scaling threshold", 2023-12-07T10:36:22.641 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-12-07T10:36:22.641 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.641 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.641 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.641 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.642 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "progress", 2023-12-07T10:36:22.642 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.642 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.642 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.642 INFO:teuthology.orchestra.run.smithi119.stdout: "allow_pg_recovery_event": { 2023-12-07T10:36:22.642 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "allow_pg_recovery_event", 2023-12-07T10:36:22.642 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.642 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.643 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.643 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.643 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.643 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.643 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.643 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "allow the module to show pg recovery progress", 2023-12-07T10:36:22.643 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.643 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.643 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.644 INFO:teuthology.orchestra.run.smithi119.stdout: "enabled": { 2023-12-07T10:36:22.644 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "enabled", 2023-12-07T10:36:22.644 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.644 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.644 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.644 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.644 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.644 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.645 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.645 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.645 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.645 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.645 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.645 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.645 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.645 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.646 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.646 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.646 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.646 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.646 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.646 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.646 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.646 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.647 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.647 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.647 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.647 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.647 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.647 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.647 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.647 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.647 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.648 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.648 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.648 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.648 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.648 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.648 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.648 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.648 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.649 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.649 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.649 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.649 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.649 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.649 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.649 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.649 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.650 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.650 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.650 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.650 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.650 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.650 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.650 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.650 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.650 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.651 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.651 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.651 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.651 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.651 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.651 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.651 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.651 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.652 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.652 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.652 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.652 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.652 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.652 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.652 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.652 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.652 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.653 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.653 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.653 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.653 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.653 INFO:teuthology.orchestra.run.smithi119.stdout: "max_completed_events": { 2023-12-07T10:36:22.653 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_completed_events", 2023-12-07T10:36:22.653 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.654 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.654 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.654 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "50", 2023-12-07T10:36:22.654 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.654 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.654 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.654 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "number of past completed events to remember", 2023-12-07T10:36:22.654 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.654 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.655 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.655 INFO:teuthology.orchestra.run.smithi119.stdout: "sleep_interval": { 2023-12-07T10:36:22.655 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "sleep_interval", 2023-12-07T10:36:22.655 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:22.655 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.655 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.655 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "5", 2023-12-07T10:36:22.655 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.656 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.656 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.656 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "how long the module is going to sleep", 2023-12-07T10:36:22.656 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.656 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.656 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.656 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.656 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.657 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "prometheus", 2023-12-07T10:36:22.657 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.657 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.657 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.657 INFO:teuthology.orchestra.run.smithi119.stdout: "cache": { 2023-12-07T10:36:22.657 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cache", 2023-12-07T10:36:22.657 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.657 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.658 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.658 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.658 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.658 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.658 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.658 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.658 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.658 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.658 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.659 INFO:teuthology.orchestra.run.smithi119.stdout: "exclude_perf_counters": { 2023-12-07T10:36:22.659 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "exclude_perf_counters", 2023-12-07T10:36:22.659 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.659 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.659 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.659 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.659 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.659 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.660 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.660 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-12-07T10:36:22.660 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-12-07T10:36:22.660 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.660 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.660 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.660 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.660 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.661 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.661 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.661 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.661 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.661 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.661 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.661 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.661 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.661 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.662 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.662 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.662 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.662 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.662 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.662 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.662 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.662 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.663 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.663 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.663 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.663 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.663 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.663 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.663 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.663 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.664 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.664 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.664 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.664 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.664 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.664 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.664 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.664 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.665 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.665 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.665 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.665 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.665 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.665 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.665 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.665 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.665 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.666 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.666 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.666 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.666 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.666 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.666 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.666 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.666 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.667 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.667 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.667 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.667 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.667 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.667 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.667 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.667 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.667 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.667 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.668 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.668 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.668 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.668 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_stats_pools": { 2023-12-07T10:36:22.668 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rbd_stats_pools", 2023-12-07T10:36:22.668 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.668 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.668 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.669 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.669 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.669 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.669 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.669 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.669 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.669 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.669 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.670 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_stats_pools_refresh_interval": { 2023-12-07T10:36:22.670 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-12-07T10:36:22.670 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.670 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.670 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.670 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "300", 2023-12-07T10:36:22.670 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.670 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.671 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.671 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.671 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.671 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.671 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.671 INFO:teuthology.orchestra.run.smithi119.stdout: "scrape_interval": { 2023-12-07T10:36:22.671 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "scrape_interval", 2023-12-07T10:36:22.671 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2023-12-07T10:36:22.671 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.672 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.672 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "15.0", 2023-12-07T10:36:22.672 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.672 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.672 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.672 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.672 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.672 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.672 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.673 INFO:teuthology.orchestra.run.smithi119.stdout: "server_addr": { 2023-12-07T10:36:22.673 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_addr", 2023-12-07T10:36:22.673 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.673 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.673 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.673 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "::", 2023-12-07T10:36:22.673 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.673 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.673 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.674 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-12-07T10:36:22.674 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.674 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.674 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.674 INFO:teuthology.orchestra.run.smithi119.stdout: "server_port": { 2023-12-07T10:36:22.674 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_port", 2023-12-07T10:36:22.674 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.674 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.675 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.675 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "9283", 2023-12-07T10:36:22.675 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.675 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.675 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.675 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-12-07T10:36:22.675 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.675 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.676 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.676 INFO:teuthology.orchestra.run.smithi119.stdout: "stale_cache_strategy": { 2023-12-07T10:36:22.676 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "stale_cache_strategy", 2023-12-07T10:36:22.676 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.676 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.676 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.676 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "log", 2023-12-07T10:36:22.676 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.677 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.677 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.677 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.677 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.677 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.677 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.677 INFO:teuthology.orchestra.run.smithi119.stdout: "standby_behaviour": { 2023-12-07T10:36:22.677 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "standby_behaviour", 2023-12-07T10:36:22.677 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.678 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.678 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.678 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "default", 2023-12-07T10:36:22.678 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.678 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.678 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.678 INFO:teuthology.orchestra.run.smithi119.stdout: "default", 2023-12-07T10:36:22.678 INFO:teuthology.orchestra.run.smithi119.stdout: "error" 2023-12-07T10:36:22.678 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.679 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.679 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.679 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.679 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.679 INFO:teuthology.orchestra.run.smithi119.stdout: "standby_error_status_code": { 2023-12-07T10:36:22.679 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "standby_error_status_code", 2023-12-07T10:36:22.679 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.680 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.680 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.680 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "500", 2023-12-07T10:36:22.680 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "400", 2023-12-07T10:36:22.680 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "599", 2023-12-07T10:36:22.680 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.680 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.680 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.680 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.680 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.681 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.681 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.681 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rbd_support", 2023-12-07T10:36:22.681 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.681 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.681 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.681 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.682 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.682 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.682 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.682 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.682 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.682 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.682 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.682 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.682 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.683 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.683 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.683 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.683 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.683 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.683 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.683 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.683 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.683 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.684 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.684 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.684 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.684 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.684 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.684 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.684 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.684 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.684 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.685 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.685 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.685 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.685 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.685 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.685 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.685 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.685 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.686 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.686 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.686 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.686 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.686 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.686 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.686 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.686 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.687 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.687 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.687 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.687 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.687 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.687 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.687 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.687 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.687 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.688 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.688 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.688 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.688 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.688 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.688 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.688 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.688 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.688 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.689 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.689 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.689 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.689 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.689 INFO:teuthology.orchestra.run.smithi119.stdout: "max_concurrent_snap_create": { 2023-12-07T10:36:22.689 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_concurrent_snap_create", 2023-12-07T10:36:22.689 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.690 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.690 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.690 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10", 2023-12-07T10:36:22.690 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.690 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.690 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.690 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.690 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.690 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.691 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.691 INFO:teuthology.orchestra.run.smithi119.stdout: "mirror_snapshot_schedule": { 2023-12-07T10:36:22.691 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "mirror_snapshot_schedule", 2023-12-07T10:36:22.691 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.691 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.691 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.691 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.691 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.692 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.692 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.692 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.692 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.692 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.692 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.692 INFO:teuthology.orchestra.run.smithi119.stdout: "trash_purge_schedule": { 2023-12-07T10:36:22.692 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "trash_purge_schedule", 2023-12-07T10:36:22.693 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.693 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.693 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.693 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.693 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.693 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.693 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.693 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.693 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.694 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.694 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.694 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.694 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.694 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.694 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "restful", 2023-12-07T10:36:22.694 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.694 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.695 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.695 INFO:teuthology.orchestra.run.smithi119.stdout: "enable_auth": { 2023-12-07T10:36:22.695 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "enable_auth", 2023-12-07T10:36:22.695 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.695 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.695 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.695 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.695 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.695 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.696 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.696 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.696 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.696 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.696 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.696 INFO:teuthology.orchestra.run.smithi119.stdout: "key_file": { 2023-12-07T10:36:22.696 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "key_file", 2023-12-07T10:36:22.697 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.697 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.697 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.697 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.697 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.697 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.697 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.697 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.697 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.698 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.698 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.698 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.698 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.698 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.698 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.698 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.698 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.699 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.699 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.699 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.699 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.699 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.699 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.699 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.699 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.699 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.700 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.700 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.700 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.700 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.700 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.700 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.700 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.700 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.701 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.701 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.701 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.701 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.701 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.701 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.701 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.701 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.701 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.702 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.702 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.702 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.702 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.702 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.702 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.702 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.702 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.703 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.703 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.703 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.703 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.703 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.703 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.703 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.703 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.703 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.704 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.704 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.704 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.704 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.704 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.704 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.704 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.704 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.705 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.705 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.705 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.705 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.705 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.705 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.705 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.705 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.705 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.706 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.706 INFO:teuthology.orchestra.run.smithi119.stdout: "server_addr": { 2023-12-07T10:36:22.706 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_addr", 2023-12-07T10:36:22.706 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.706 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.706 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.706 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.707 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.707 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.707 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.707 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.707 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.707 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.707 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.708 INFO:teuthology.orchestra.run.smithi119.stdout: "server_port": { 2023-12-07T10:36:22.708 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "server_port", 2023-12-07T10:36:22.708 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.708 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.708 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.708 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.708 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.708 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.708 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.709 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.709 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.709 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.709 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.709 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.709 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.709 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rgw", 2023-12-07T10:36:22.710 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.710 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.710 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.710 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.710 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.710 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.710 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.710 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.710 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.711 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.711 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.711 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.711 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.711 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.711 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.711 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.711 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.711 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.712 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.712 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.712 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.712 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.712 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.712 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.712 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.712 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.713 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.713 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.713 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.713 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.713 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.713 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.713 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.713 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.714 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.714 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.714 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.714 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.714 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.714 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.714 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.714 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.715 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.715 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.715 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.715 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.715 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.715 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.715 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.715 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.715 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.716 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.716 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.716 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.716 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.716 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.716 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.716 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.716 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.717 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.717 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.717 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.717 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.717 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.717 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.717 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.717 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.717 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.718 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.718 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.718 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.718 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rook", 2023-12-07T10:36:22.718 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.718 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.718 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.719 INFO:teuthology.orchestra.run.smithi119.stdout: "drive_group_interval": { 2023-12-07T10:36:22.719 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "drive_group_interval", 2023-12-07T10:36:22.719 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2023-12-07T10:36:22.719 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.719 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.719 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "300.0", 2023-12-07T10:36:22.719 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.719 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.719 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.720 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-12-07T10:36:22.720 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.720 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.720 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.720 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.720 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.720 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.720 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.721 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.721 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.721 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.721 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.721 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.721 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.721 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.721 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.722 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.722 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.722 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.722 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.722 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.722 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.722 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.722 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.723 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.723 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.723 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.723 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.723 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.723 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.723 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.723 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.723 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.724 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.724 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.724 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.724 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.724 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.724 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.724 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.724 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.725 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.725 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.725 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.725 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.725 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.725 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.725 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.725 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.726 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.726 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.726 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.726 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.726 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.726 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.726 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.726 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.727 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.727 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.727 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.727 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.727 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.727 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.727 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.727 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.728 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.728 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.728 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.728 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.728 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.728 INFO:teuthology.orchestra.run.smithi119.stdout: "storage_class": { 2023-12-07T10:36:22.728 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "storage_class", 2023-12-07T10:36:22.728 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.729 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.729 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.729 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "local", 2023-12-07T10:36:22.729 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.729 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.729 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.729 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-12-07T10:36:22.729 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.729 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.730 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.730 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.730 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.730 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "selftest", 2023-12-07T10:36:22.730 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.730 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.730 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.730 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.731 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.731 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.731 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.731 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.731 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.731 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.731 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.731 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.732 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.732 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.732 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.732 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.732 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.732 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.732 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.732 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.732 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.733 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.733 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.733 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.733 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.733 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.733 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.733 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.733 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.734 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.734 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.734 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.734 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.734 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.734 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.734 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.734 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.734 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.735 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.735 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.735 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.735 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.735 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.735 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.735 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.735 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.735 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.736 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.736 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.736 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.736 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.736 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.736 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.736 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.736 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.737 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.737 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.737 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.737 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.737 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.737 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.737 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.737 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.738 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.738 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.738 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.738 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.738 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.738 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.738 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.738 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.738 INFO:teuthology.orchestra.run.smithi119.stdout: "roption1": { 2023-12-07T10:36:22.738 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "roption1", 2023-12-07T10:36:22.739 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.739 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.739 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.739 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.739 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.739 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.739 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.739 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.739 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.740 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.740 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.740 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.740 INFO:teuthology.orchestra.run.smithi119.stdout: "roption2": { 2023-12-07T10:36:22.740 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "roption2", 2023-12-07T10:36:22.740 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.740 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.740 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.741 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "xyz", 2023-12-07T10:36:22.741 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.741 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.741 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.741 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.741 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.741 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.741 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption1": { 2023-12-07T10:36:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption1", 2023-12-07T10:36:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.743 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.743 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.743 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.743 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.743 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.743 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.743 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption2": { 2023-12-07T10:36:22.743 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption2", 2023-12-07T10:36:22.743 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.744 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.744 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.744 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.744 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.744 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.744 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.744 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.744 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.744 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.745 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.745 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption3": { 2023-12-07T10:36:22.745 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption3", 2023-12-07T10:36:22.745 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "float", 2023-12-07T10:36:22.745 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.745 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.745 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.745 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.746 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.746 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.746 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.746 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.746 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.746 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.746 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption4": { 2023-12-07T10:36:22.746 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption4", 2023-12-07T10:36:22.747 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.747 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.747 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.747 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.747 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.747 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.747 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.747 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.747 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.748 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.748 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.748 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption5": { 2023-12-07T10:36:22.748 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption5", 2023-12-07T10:36:22.748 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.748 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.748 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.748 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.749 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.749 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.749 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.749 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.749 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.749 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.749 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.749 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.749 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption6": { 2023-12-07T10:36:22.750 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption6", 2023-12-07T10:36:22.750 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.750 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.750 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.750 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.750 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.750 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.750 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.751 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.751 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.751 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.751 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.751 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.751 INFO:teuthology.orchestra.run.smithi119.stdout: "rwoption7": { 2023-12-07T10:36:22.751 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rwoption7", 2023-12-07T10:36:22.751 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.751 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.752 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.752 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.752 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "1", 2023-12-07T10:36:22.752 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "42", 2023-12-07T10:36:22.752 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.752 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.752 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.752 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.752 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.753 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.753 INFO:teuthology.orchestra.run.smithi119.stdout: "testkey": { 2023-12-07T10:36:22.753 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "testkey", 2023-12-07T10:36:22.753 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.753 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.753 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.753 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.753 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.753 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.754 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.754 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.754 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.754 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.754 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.754 INFO:teuthology.orchestra.run.smithi119.stdout: "testlkey": { 2023-12-07T10:36:22.754 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "testlkey", 2023-12-07T10:36:22.754 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.755 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.755 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.755 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.755 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.755 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.755 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.755 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.755 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.757 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.757 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.757 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.757 INFO:teuthology.orchestra.run.smithi119.stdout: "testnewline": { 2023-12-07T10:36:22.757 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "testnewline", 2023-12-07T10:36:22.757 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.758 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.758 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.758 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.758 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.758 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.758 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.758 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.758 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.758 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.759 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.759 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.759 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.759 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.759 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.759 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "snap_schedule", 2023-12-07T10:36:22.759 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.759 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.759 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.760 INFO:teuthology.orchestra.run.smithi119.stdout: "allow_m_granularity": { 2023-12-07T10:36:22.760 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "allow_m_granularity", 2023-12-07T10:36:22.760 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.760 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.760 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.760 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.760 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.760 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.761 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.761 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "allow minute scheduled snapshots", 2023-12-07T10:36:22.761 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.761 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.761 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.761 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.761 INFO:teuthology.orchestra.run.smithi119.stdout: "dump_on_update": { 2023-12-07T10:36:22.761 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "dump_on_update", 2023-12-07T10:36:22.761 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.762 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.762 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.762 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.762 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.762 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.762 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.762 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "dump database to debug log on update", 2023-12-07T10:36:22.763 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.763 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.763 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.763 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.763 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.763 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.763 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.764 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.764 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.764 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.764 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.764 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.764 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.764 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.765 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.765 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.765 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.765 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.765 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.765 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.765 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.765 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.766 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.766 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.766 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.766 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.766 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.766 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.766 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.767 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.767 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.767 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.767 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.767 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.767 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.767 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.767 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.767 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.767 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.768 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.768 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.768 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.768 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.768 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.768 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.768 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.768 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.769 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.769 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.769 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.769 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.769 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.769 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.769 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.769 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.769 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.770 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.770 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.770 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.770 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.770 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.770 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.770 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.770 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.771 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.771 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.771 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.771 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.771 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.771 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.771 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.771 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.771 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.771 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.772 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.772 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.772 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.772 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "stats", 2023-12-07T10:36:22.772 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.772 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.772 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.772 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.773 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.773 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.773 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.773 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.773 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.773 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.773 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.773 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.773 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.774 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.774 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.774 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.774 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.774 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.774 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.774 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.774 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.774 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.775 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.775 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.775 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.775 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.775 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.775 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.775 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.775 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.775 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.775 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.776 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.776 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.776 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.776 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.776 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.776 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.776 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.776 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.776 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.777 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.777 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.777 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.777 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.777 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.777 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.777 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.777 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.777 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.778 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.778 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.778 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.778 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.778 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.778 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.778 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.778 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.778 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.779 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.779 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.779 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.779 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.779 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.779 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.779 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.779 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.779 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.779 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.780 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.780 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.780 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.780 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.780 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.780 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.780 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.780 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "status", 2023-12-07T10:36:22.781 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.781 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.781 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.781 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.781 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.781 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.781 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.781 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.781 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.781 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.782 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.783 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.783 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.783 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.783 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.783 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.783 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.783 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.783 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.783 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.783 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.784 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.784 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.784 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.784 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.784 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.784 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.784 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.784 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.784 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.785 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.785 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.785 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.785 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.785 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.785 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.785 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.785 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.785 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.785 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.786 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.786 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.786 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.786 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.786 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.786 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.786 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.786 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.787 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.787 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.787 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.787 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.787 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.787 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.787 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.787 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.787 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.788 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.788 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.788 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.788 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.788 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.788 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.788 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.788 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.788 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.789 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.789 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.789 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.789 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "telegraf", 2023-12-07T10:36:22.789 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.789 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.789 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.789 INFO:teuthology.orchestra.run.smithi119.stdout: "address": { 2023-12-07T10:36:22.790 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "address", 2023-12-07T10:36:22.790 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.790 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.790 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.790 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-12-07T10:36:22.790 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.790 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.790 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.790 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.791 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.791 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.791 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.791 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.791 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2023-12-07T10:36:22.791 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2023-12-07T10:36:22.791 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:22.791 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.791 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.792 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "15", 2023-12-07T10:36:22.792 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.792 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.792 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.792 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.792 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.792 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.792 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.792 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.793 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.793 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.793 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.793 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.793 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.793 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.793 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.793 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.793 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.794 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.794 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.794 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.794 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.794 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.794 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.794 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.794 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.795 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.795 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.795 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.795 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.795 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.795 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.795 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.795 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.795 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.796 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.796 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.796 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.796 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.796 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.796 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.796 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.796 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.796 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.797 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.797 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.797 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.797 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.797 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.797 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.797 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.797 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.797 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.798 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.798 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.798 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.798 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.798 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.798 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.798 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.798 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.798 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.799 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.799 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.799 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.799 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.799 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.799 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.799 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.799 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.800 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.800 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.800 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.800 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.800 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.800 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.800 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.800 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.800 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "telemetry", 2023-12-07T10:36:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_basic": { 2023-12-07T10:36:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_basic", 2023-12-07T10:36:22.801 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.802 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.802 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.802 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.802 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.802 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.802 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.802 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share basic cluster information (size, version)", 2023-12-07T10:36:22.802 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.802 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.803 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.803 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_crash": { 2023-12-07T10:36:22.803 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_crash", 2023-12-07T10:36:22.803 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.803 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.803 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.803 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.804 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.804 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.804 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.804 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-12-07T10:36:22.804 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.804 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.804 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.804 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_device": { 2023-12-07T10:36:22.805 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_device", 2023-12-07T10:36:22.805 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.805 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.805 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.805 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.805 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.805 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.805 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.805 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-12-07T10:36:22.806 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.806 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.806 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.806 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_ident": { 2023-12-07T10:36:22.806 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_ident", 2023-12-07T10:36:22.806 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.806 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.806 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.807 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.807 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.807 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.807 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.807 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-12-07T10:36:22.807 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.807 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.807 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.808 INFO:teuthology.orchestra.run.smithi119.stdout: "channel_perf": { 2023-12-07T10:36:22.808 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "channel_perf", 2023-12-07T10:36:22.808 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.808 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.808 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.808 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.808 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.808 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.808 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.809 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Share various performance metrics of a cluster", 2023-12-07T10:36:22.809 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.809 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.809 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.809 INFO:teuthology.orchestra.run.smithi119.stdout: "contact": { 2023-12-07T10:36:22.809 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "contact", 2023-12-07T10:36:22.809 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.809 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.810 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.810 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.810 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.810 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.810 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.810 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.810 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.810 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.810 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.811 INFO:teuthology.orchestra.run.smithi119.stdout: "description": { 2023-12-07T10:36:22.811 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "description", 2023-12-07T10:36:22.811 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.811 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.811 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.811 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.811 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.811 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.812 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.812 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.812 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.812 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.812 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.812 INFO:teuthology.orchestra.run.smithi119.stdout: "device_url": { 2023-12-07T10:36:22.812 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "device_url", 2023-12-07T10:36:22.812 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.812 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.813 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.813 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-12-07T10:36:22.813 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.813 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.813 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.813 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.813 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.813 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.813 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.814 INFO:teuthology.orchestra.run.smithi119.stdout: "enabled": { 2023-12-07T10:36:22.814 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "enabled", 2023-12-07T10:36:22.814 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.814 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.814 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.814 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.814 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.814 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.815 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.815 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.815 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.815 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.815 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.815 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2023-12-07T10:36:22.815 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2023-12-07T10:36:22.815 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.816 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.816 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.816 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "24", 2023-12-07T10:36:22.816 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "8", 2023-12-07T10:36:22.816 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.816 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.816 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.816 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.816 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.817 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.817 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.817 INFO:teuthology.orchestra.run.smithi119.stdout: "last_opt_revision": { 2023-12-07T10:36:22.817 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "last_opt_revision", 2023-12-07T10:36:22.817 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.817 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.817 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.817 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "1", 2023-12-07T10:36:22.817 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.818 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.818 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.818 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.818 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.818 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.818 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.818 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.818 INFO:teuthology.orchestra.run.smithi119.stdout: "leaderboard": { 2023-12-07T10:36:22.818 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "leaderboard", 2023-12-07T10:36:22.819 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.819 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.819 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.819 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.819 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.819 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.819 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.819 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.819 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.820 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.820 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.820 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.820 INFO:teuthology.orchestra.run.smithi119.stdout: "leaderboard_description": { 2023-12-07T10:36:22.820 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "leaderboard_description", 2023-12-07T10:36:22.820 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.820 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.820 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.820 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.820 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.821 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.821 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.821 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.821 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.821 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.821 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.821 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.821 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.821 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.822 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.822 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.822 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.822 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.822 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.822 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.822 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.822 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.823 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.823 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.823 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.823 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.823 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.823 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.823 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.823 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.823 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.824 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.824 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.824 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.824 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.824 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.824 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.824 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.824 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.824 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.825 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.825 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.825 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.825 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.825 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.825 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.825 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.825 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.825 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.826 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.826 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.826 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.826 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.826 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.826 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.826 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.826 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.826 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.827 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.827 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.827 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.827 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.827 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.827 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.827 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.827 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.827 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.828 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.828 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.828 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.828 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.828 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.828 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.828 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.828 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.828 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.829 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.829 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.829 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.829 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.829 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.829 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.829 INFO:teuthology.orchestra.run.smithi119.stdout: "organization": { 2023-12-07T10:36:22.829 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "organization", 2023-12-07T10:36:22.829 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.830 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.830 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.830 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.830 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.830 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.830 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.830 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.830 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.830 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.831 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.831 INFO:teuthology.orchestra.run.smithi119.stdout: "proxy": { 2023-12-07T10:36:22.831 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "proxy", 2023-12-07T10:36:22.831 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.831 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.831 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.831 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.831 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.831 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.832 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.832 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.832 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.832 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.832 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.832 INFO:teuthology.orchestra.run.smithi119.stdout: "url": { 2023-12-07T10:36:22.832 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "url", 2023-12-07T10:36:22.833 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.833 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.833 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.833 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-12-07T10:36:22.833 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.833 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.833 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.833 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.833 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.834 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.834 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.834 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.834 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.834 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.834 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.834 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "test_orchestrator", 2023-12-07T10:36:22.834 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.835 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.835 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.835 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.835 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.835 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.835 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.835 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.835 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.836 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.836 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.836 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.836 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.836 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.836 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.836 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.836 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.836 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.837 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.837 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.837 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.837 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.837 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.837 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.837 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.837 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.837 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.838 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.838 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.838 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.838 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.838 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.838 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.838 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.838 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.838 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.839 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.839 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.839 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.839 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.839 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.839 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.839 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.839 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.840 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.840 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.840 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.840 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.840 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.840 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.840 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.840 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.840 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.841 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.841 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.841 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.841 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.841 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.841 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.841 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.841 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.842 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.842 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.842 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.842 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.842 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.842 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.842 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.842 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.842 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.843 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.843 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.843 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.843 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "volumes", 2023-12-07T10:36:22.843 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.843 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.843 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.843 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.844 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.844 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.844 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.844 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.844 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.844 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.844 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.844 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.844 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.845 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.845 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.845 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.845 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.845 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.845 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.845 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.845 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.845 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.846 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.846 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.846 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.846 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.846 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.846 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.846 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.846 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.847 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.847 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.847 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.847 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.847 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.847 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.847 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.847 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.848 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.848 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.848 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.848 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.848 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.848 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.848 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.848 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.848 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.849 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.849 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.849 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.849 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.849 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.849 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.849 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.849 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.849 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.849 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.850 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.850 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.850 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.850 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.850 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.850 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.850 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.850 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.850 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.851 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.851 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.851 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.851 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.851 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.851 INFO:teuthology.orchestra.run.smithi119.stdout: "max_concurrent_clones": { 2023-12-07T10:36:22.851 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "max_concurrent_clones", 2023-12-07T10:36:22.851 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.852 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.852 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.852 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "4", 2023-12-07T10:36:22.852 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.852 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.852 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.852 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Number of asynchronous cloner threads", 2023-12-07T10:36:22.852 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.852 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.852 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.853 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.853 INFO:teuthology.orchestra.run.smithi119.stdout: "periodic_async_work": { 2023-12-07T10:36:22.853 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "periodic_async_work", 2023-12-07T10:36:22.853 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.853 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.853 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.853 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.853 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.853 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.854 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.854 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Periodically check for async work", 2023-12-07T10:36:22.854 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.854 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.854 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.854 INFO:teuthology.orchestra.run.smithi119.stdout: "snapshot_clone_delay": { 2023-12-07T10:36:22.854 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "snapshot_clone_delay", 2023-12-07T10:36:22.855 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.855 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.855 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.855 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "0", 2023-12-07T10:36:22.855 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.855 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.855 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.855 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-12-07T10:36:22.855 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.856 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.856 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.856 INFO:teuthology.orchestra.run.smithi119.stdout: "snapshot_clone_no_wait": { 2023-12-07T10:36:22.856 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "snapshot_clone_no_wait", 2023-12-07T10:36:22.856 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.856 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.856 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.856 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "True", 2023-12-07T10:36:22.857 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.857 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.857 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.857 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2023-12-07T10:36:22.857 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.857 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.857 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.857 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.857 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.858 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "zabbix", 2023-12-07T10:36:22.858 INFO:teuthology.orchestra.run.smithi119.stdout: "can_run": true, 2023-12-07T10:36:22.858 INFO:teuthology.orchestra.run.smithi119.stdout: "error_string": "", 2023-12-07T10:36:22.858 INFO:teuthology.orchestra.run.smithi119.stdout: "module_options": { 2023-12-07T10:36:22.858 INFO:teuthology.orchestra.run.smithi119.stdout: "discovery_interval": { 2023-12-07T10:36:22.858 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "discovery_interval", 2023-12-07T10:36:22.858 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "uint", 2023-12-07T10:36:22.859 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.859 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.859 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "100", 2023-12-07T10:36:22.859 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.859 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.859 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.859 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.859 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.859 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.860 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.860 INFO:teuthology.orchestra.run.smithi119.stdout: "identifier": { 2023-12-07T10:36:22.860 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "identifier", 2023-12-07T10:36:22.860 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.860 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.860 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.860 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.860 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.861 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.861 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.861 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.861 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.861 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.861 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.861 INFO:teuthology.orchestra.run.smithi119.stdout: "interval": { 2023-12-07T10:36:22.861 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "interval", 2023-12-07T10:36:22.862 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "secs", 2023-12-07T10:36:22.862 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.862 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.862 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "60", 2023-12-07T10:36:22.862 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.862 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.862 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.862 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.862 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.863 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.863 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.863 INFO:teuthology.orchestra.run.smithi119.stdout: "log_level": { 2023-12-07T10:36:22.863 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_level", 2023-12-07T10:36:22.863 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.863 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.863 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.863 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.864 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.864 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.864 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.864 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.864 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.864 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.864 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.864 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.864 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.865 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.865 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.865 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.865 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.865 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.865 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster": { 2023-12-07T10:36:22.865 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster", 2023-12-07T10:36:22.865 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.866 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.866 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.866 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.866 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.866 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.866 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.866 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.866 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.866 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.867 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.867 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_cluster_level": { 2023-12-07T10:36:22.867 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_cluster_level", 2023-12-07T10:36:22.867 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.867 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.867 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.867 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "info", 2023-12-07T10:36:22.868 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.868 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.868 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [ 2023-12-07T10:36:22.868 INFO:teuthology.orchestra.run.smithi119.stdout: "", 2023-12-07T10:36:22.868 INFO:teuthology.orchestra.run.smithi119.stdout: "critical", 2023-12-07T10:36:22.868 INFO:teuthology.orchestra.run.smithi119.stdout: "debug", 2023-12-07T10:36:22.868 INFO:teuthology.orchestra.run.smithi119.stdout: "error", 2023-12-07T10:36:22.868 INFO:teuthology.orchestra.run.smithi119.stdout: "info", 2023-12-07T10:36:22.868 INFO:teuthology.orchestra.run.smithi119.stdout: "warning" 2023-12-07T10:36:22.869 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.869 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.869 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.869 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.869 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.869 INFO:teuthology.orchestra.run.smithi119.stdout: "log_to_file": { 2023-12-07T10:36:22.869 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "log_to_file", 2023-12-07T10:36:22.869 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "bool", 2023-12-07T10:36:22.869 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.870 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 1, 2023-12-07T10:36:22.870 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "False", 2023-12-07T10:36:22.870 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.870 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.870 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.870 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.870 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.870 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.870 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.871 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.871 INFO:teuthology.orchestra.run.smithi119.stdout: "zabbix_host": { 2023-12-07T10:36:22.871 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "zabbix_host", 2023-12-07T10:36:22.871 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.871 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.871 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.871 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "", 2023-12-07T10:36:22.871 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.871 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.872 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.872 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.872 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.872 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.872 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.872 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.872 INFO:teuthology.orchestra.run.smithi119.stdout: "zabbix_port": { 2023-12-07T10:36:22.872 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "zabbix_port", 2023-12-07T10:36:22.873 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "int", 2023-12-07T10:36:22.873 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.873 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.873 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "10051", 2023-12-07T10:36:22.873 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.873 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.873 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.873 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.873 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.874 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.874 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.874 INFO:teuthology.orchestra.run.smithi119.stdout: "zabbix_sender": { 2023-12-07T10:36:22.874 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "zabbix_sender", 2023-12-07T10:36:22.874 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "str", 2023-12-07T10:36:22.874 INFO:teuthology.orchestra.run.smithi119.stdout: "level": "advanced", 2023-12-07T10:36:22.874 INFO:teuthology.orchestra.run.smithi119.stdout: "flags": 0, 2023-12-07T10:36:22.874 INFO:teuthology.orchestra.run.smithi119.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-12-07T10:36:22.875 INFO:teuthology.orchestra.run.smithi119.stdout: "min": "", 2023-12-07T10:36:22.875 INFO:teuthology.orchestra.run.smithi119.stdout: "max": "", 2023-12-07T10:36:22.875 INFO:teuthology.orchestra.run.smithi119.stdout: "enum_allowed": [], 2023-12-07T10:36:22.875 INFO:teuthology.orchestra.run.smithi119.stdout: "desc": "", 2023-12-07T10:36:22.875 INFO:teuthology.orchestra.run.smithi119.stdout: "long_desc": "", 2023-12-07T10:36:22.875 INFO:teuthology.orchestra.run.smithi119.stdout: "tags": [], 2023-12-07T10:36:22.875 INFO:teuthology.orchestra.run.smithi119.stdout: "see_also": [] 2023-12-07T10:36:22.875 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.875 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.876 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.876 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.876 INFO:teuthology.orchestra.run.smithi119.stdout: "services": { 2023-12-07T10:36:22.876 INFO:teuthology.orchestra.run.smithi119.stdout: "dashboard": "https://172.21.15.119:8443/" 2023-12-07T10:36:22.876 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.876 INFO:teuthology.orchestra.run.smithi119.stdout: "always_on_modules": { 2023-12-07T10:36:22.876 INFO:teuthology.orchestra.run.smithi119.stdout: "octopus": [ 2023-12-07T10:36:22.876 INFO:teuthology.orchestra.run.smithi119.stdout: "balancer", 2023-12-07T10:36:22.876 INFO:teuthology.orchestra.run.smithi119.stdout: "crash", 2023-12-07T10:36:22.877 INFO:teuthology.orchestra.run.smithi119.stdout: "devicehealth", 2023-12-07T10:36:22.877 INFO:teuthology.orchestra.run.smithi119.stdout: "orchestrator", 2023-12-07T10:36:22.877 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_autoscaler", 2023-12-07T10:36:22.877 INFO:teuthology.orchestra.run.smithi119.stdout: "progress", 2023-12-07T10:36:22.877 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_support", 2023-12-07T10:36:22.877 INFO:teuthology.orchestra.run.smithi119.stdout: "status", 2023-12-07T10:36:22.877 INFO:teuthology.orchestra.run.smithi119.stdout: "telemetry", 2023-12-07T10:36:22.877 INFO:teuthology.orchestra.run.smithi119.stdout: "volumes" 2023-12-07T10:36:22.877 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.878 INFO:teuthology.orchestra.run.smithi119.stdout: "pacific": [ 2023-12-07T10:36:22.878 INFO:teuthology.orchestra.run.smithi119.stdout: "balancer", 2023-12-07T10:36:22.878 INFO:teuthology.orchestra.run.smithi119.stdout: "crash", 2023-12-07T10:36:22.878 INFO:teuthology.orchestra.run.smithi119.stdout: "devicehealth", 2023-12-07T10:36:22.878 INFO:teuthology.orchestra.run.smithi119.stdout: "orchestrator", 2023-12-07T10:36:22.878 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_autoscaler", 2023-12-07T10:36:22.878 INFO:teuthology.orchestra.run.smithi119.stdout: "progress", 2023-12-07T10:36:22.878 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_support", 2023-12-07T10:36:22.878 INFO:teuthology.orchestra.run.smithi119.stdout: "status", 2023-12-07T10:36:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: "telemetry", 2023-12-07T10:36:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: "volumes" 2023-12-07T10:36:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: "quincy": [ 2023-12-07T10:36:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: "balancer", 2023-12-07T10:36:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: "crash", 2023-12-07T10:36:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: "devicehealth", 2023-12-07T10:36:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: "orchestrator", 2023-12-07T10:36:22.879 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_autoscaler", 2023-12-07T10:36:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: "progress", 2023-12-07T10:36:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_support", 2023-12-07T10:36:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: "status", 2023-12-07T10:36:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: "telemetry", 2023-12-07T10:36:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: "volumes" 2023-12-07T10:36:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2023-12-07T10:36:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: "reef": [ 2023-12-07T10:36:22.880 INFO:teuthology.orchestra.run.smithi119.stdout: "balancer", 2023-12-07T10:36:22.881 INFO:teuthology.orchestra.run.smithi119.stdout: "crash", 2023-12-07T10:36:22.881 INFO:teuthology.orchestra.run.smithi119.stdout: "devicehealth", 2023-12-07T10:36:22.881 INFO:teuthology.orchestra.run.smithi119.stdout: "orchestrator", 2023-12-07T10:36:22.881 INFO:teuthology.orchestra.run.smithi119.stdout: "pg_autoscaler", 2023-12-07T10:36:22.881 INFO:teuthology.orchestra.run.smithi119.stdout: "progress", 2023-12-07T10:36:22.881 INFO:teuthology.orchestra.run.smithi119.stdout: "rbd_support", 2023-12-07T10:36:22.881 INFO:teuthology.orchestra.run.smithi119.stdout: "status", 2023-12-07T10:36:22.881 INFO:teuthology.orchestra.run.smithi119.stdout: "telemetry", 2023-12-07T10:36:22.881 INFO:teuthology.orchestra.run.smithi119.stdout: "volumes" 2023-12-07T10:36:22.882 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-12-07T10:36:22.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.882 INFO:teuthology.orchestra.run.smithi119.stdout: "last_failure_osd_epoch": 94, 2023-12-07T10:36:22.882 INFO:teuthology.orchestra.run.smithi119.stdout: "active_clients": [ 2023-12-07T10:36:22.882 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.882 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "devicehealth", 2023-12-07T10:36:22.882 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2023-12-07T10:36:22.882 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.882 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2023-12-07T10:36:22.883 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:0", 2023-12-07T10:36:22.883 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 731660177 2023-12-07T10:36:22.883 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.883 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-12-07T10:36:22.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.883 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.883 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "libcephsqlite", 2023-12-07T10:36:22.883 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2023-12-07T10:36:22.883 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.884 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2023-12-07T10:36:22.884 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:0", 2023-12-07T10:36:22.884 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 2134609554 2023-12-07T10:36:22.884 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.884 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-12-07T10:36:22.884 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.884 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "rbd_support", 2023-12-07T10:36:22.884 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2023-12-07T10:36:22.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.885 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2023-12-07T10:36:22.885 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:0", 2023-12-07T10:36:22.885 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 2454335787 2023-12-07T10:36:22.885 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.885 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-12-07T10:36:22.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.885 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.885 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "snap_schedule", 2023-12-07T10:36:22.886 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2023-12-07T10:36:22.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.886 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2023-12-07T10:36:22.886 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:0", 2023-12-07T10:36:22.886 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 2930711659 2023-12-07T10:36:22.886 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.886 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-12-07T10:36:22.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.887 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "snap_schedule", 2023-12-07T10:36:22.887 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2023-12-07T10:36:22.887 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.887 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2023-12-07T10:36:22.887 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:0", 2023-12-07T10:36:22.887 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 3262597674 2023-12-07T10:36:22.887 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.887 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-12-07T10:36:22.887 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:22.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.888 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "volumes", 2023-12-07T10:36:22.888 INFO:teuthology.orchestra.run.smithi119.stdout: "addrvec": [ 2023-12-07T10:36:22.888 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-12-07T10:36:22.888 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v2", 2023-12-07T10:36:22.888 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "172.21.15.119:0", 2023-12-07T10:36:22.888 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 1469251363 2023-12-07T10:36:22.888 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.888 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-12-07T10:36:22.889 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:22.889 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-12-07T10:36:22.889 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:36:22.890 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:21 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3919050984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:36:22.891 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:21 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3607586120' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-12-07T10:36:22.891 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:21 smithi153 ceph-mon[133625]: osdmap e114: 12 total, 12 up, 12 in 2023-12-07T10:36:22.891 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:22 smithi153 ceph-mon[133625]: pgmap v501: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 3.8 MiB/s wr, 174 op/s 2023-12-07T10:36:22.891 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:21 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3919050984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:36:22.891 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:21 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3607586120' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-12-07T10:36:22.891 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:21 smithi159 ceph-mon[134726]: osdmap e114: 12 total, 12 up, 12 in 2023-12-07T10:36:22.891 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:22 smithi159 ceph-mon[134726]: pgmap v501: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 3.8 MiB/s wr, 174 op/s 2023-12-07T10:36:22.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.843+0000 7f2a527ff700 1 -- 172.21.15.119:0/3607586120 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f2a340a1d00 msgr2=0x7f2a340a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:22.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.843+0000 7f2a527ff700 1 --2- 172.21.15.119:0/3607586120 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f2a340a1d00 0x7f2a340a41c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c009cb0 tx=0x7f2a3c009260 comp rx=0 tx=0).stop 2023-12-07T10:36:22.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.843+0000 7f2a527ff700 1 -- 172.21.15.119:0/3607586120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a4c101ea0 msgr2=0x7f2a4c19fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:22.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.843+0000 7f2a527ff700 1 --2- 172.21.15.119:0/3607586120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a4c101ea0 0x7f2a4c19fee0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2a4000e9a0 tx=0x7f2a4000ed60 comp rx=0 tx=0).stop 2023-12-07T10:36:22.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.843+0000 7f2a527ff700 1 -- 172.21.15.119:0/3607586120 shutdown_connections 2023-12-07T10:36:22.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.843+0000 7f2a527ff700 1 --2- 172.21.15.119:0/3607586120 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f2a340a1d00 0x7f2a340a41c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:22.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.843+0000 7f2a527ff700 1 --2- 172.21.15.119:0/3607586120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a4c1075a0 0x7f2a4c199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:22.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.843+0000 7f2a527ff700 1 --2- 172.21.15.119:0/3607586120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a4c1014e0 0x7f2a4c19f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:22.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.843+0000 7f2a527ff700 1 --2- 172.21.15.119:0/3607586120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a4c101ea0 0x7f2a4c19fee0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:22.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.843+0000 7f2a527ff700 1 -- 172.21.15.119:0/3607586120 >> 172.21.15.119:0/3607586120 conn(0x7f2a4c0fcff0 msgr2=0x7f2a4c0fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:22.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.843+0000 7f2a527ff700 1 -- 172.21.15.119:0/3607586120 shutdown_connections 2023-12-07T10:36:22.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:20.844+0000 7f2a527ff700 1 -- 172.21.15.119:0/3607586120 wait complete. 2023-12-07T10:36:22.899 DEBUG:tasks.check_counter:Getting stats from a 2023-12-07T10:36:22.900 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph daemon mds.a perf dump 2023-12-07T10:36:22.900 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:21 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3919050984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:36:22.900 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:21 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3607586120' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-12-07T10:36:22.901 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:21 smithi119 ceph-mon[97908]: osdmap e114: 12 total, 12 up, 12 in 2023-12-07T10:36:22.901 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:22 smithi119 ceph-mon[97908]: pgmap v501: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 3.8 MiB/s wr, 174 op/s 2023-12-07T10:36:23.852 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:36:24.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:24 smithi119 ceph-mon[97908]: pgmap v502: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 396 KiB/s wr, 66 op/s 2023-12-07T10:36:24.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:24 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:36:24.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:24 smithi159 ceph-mon[134726]: pgmap v502: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 396 KiB/s wr, 66 op/s 2023-12-07T10:36:24.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:24 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:36:24.527 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:24 smithi153 ceph-mon[133625]: pgmap v502: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 396 KiB/s wr, 66 op/s 2023-12-07T10:36:24.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:24 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:36:25.685 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:36:25.685 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-0": { 2023-12-07T10:36:25.686 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 1808, 2023-12-07T10:36:25.686 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 1808, 2023-12-07T10:36:25.686 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 71855683, 2023-12-07T10:36:25.686 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 561493, 2023-12-07T10:36:25.686 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 8, 2023-12-07T10:36:25.686 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 4, 2023-12-07T10:36:25.686 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.327317128, 2023-12-07T10:36:25.686 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.082807646, 2023-12-07T10:36:25.687 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.161796618, 2023-12-07T10:36:25.687 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.061014500, 2023-12-07T10:36:25.687 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:25.687 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1807, 2023-12-07T10:36:25.687 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.115768787, 2023-12-07T10:36:25.687 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000064066 2023-12-07T10:36:25.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.687 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:25.687 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.687 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.688 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.688 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-12-07T10:36:25.688 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 448 2023-12-07T10:36:25.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.688 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-1": { 2023-12-07T10:36:25.688 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 1762, 2023-12-07T10:36:25.688 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 1704, 2023-12-07T10:36:25.688 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 77198927, 2023-12-07T10:36:25.689 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 614164, 2023-12-07T10:36:25.689 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 6, 2023-12-07T10:36:25.689 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 5, 2023-12-07T10:36:25.689 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.336147328, 2023-12-07T10:36:25.689 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.064730808, 2023-12-07T10:36:25.689 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.225299044, 2023-12-07T10:36:25.689 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.047074671, 2023-12-07T10:36:25.689 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:25.689 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1703, 2023-12-07T10:36:25.690 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.091781442, 2023-12-07T10:36:25.690 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000053893 2023-12-07T10:36:25.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.690 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:25.690 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.690 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.690 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.691 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 1928432, 2023-12-07T10:36:25.691 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 187104 2023-12-07T10:36:25.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.691 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-2": { 2023-12-07T10:36:25.691 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 752, 2023-12-07T10:36:25.691 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 980, 2023-12-07T10:36:25.691 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 63111806, 2023-12-07T10:36:25.691 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 873702, 2023-12-07T10:36:25.691 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 9, 2023-12-07T10:36:25.692 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 6, 2023-12-07T10:36:25.692 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.205186042, 2023-12-07T10:36:25.692 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.036238114, 2023-12-07T10:36:25.692 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.127383761, 2023-12-07T10:36:25.692 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.029149600, 2023-12-07T10:36:25.692 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:25.692 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 977, 2023-12-07T10:36:25.692 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.074100333, 2023-12-07T10:36:25.692 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000075844 2023-12-07T10:36:25.692 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.693 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:25.693 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.693 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.693 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.693 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 480, 2023-12-07T10:36:25.693 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 640752 2023-12-07T10:36:25.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.693 INFO:teuthology.orchestra.run.smithi119.stdout: "cct": { 2023-12-07T10:36:25.694 INFO:teuthology.orchestra.run.smithi119.stdout: "total_workers": 1, 2023-12-07T10:36:25.694 INFO:teuthology.orchestra.run.smithi119.stdout: "unhealthy_workers": 0 2023-12-07T10:36:25.694 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.694 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-MDSRank": { 2023-12-07T10:36:25.694 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2023-12-07T10:36:25.694 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2023-12-07T10:36:25.694 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 6719, 2023-12-07T10:36:25.694 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.203853207, 2023-12-07T10:36:25.694 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000030339 2023-12-07T10:36:25.695 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:25.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.695 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-PurgeQueue": { 2023-12-07T10:36:25.695 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2023-12-07T10:36:25.695 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2023-12-07T10:36:25.695 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.695 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.695 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.695 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:25.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.696 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2023-12-07T10:36:25.696 INFO:teuthology.orchestra.run.smithi119.stdout: "request": 0, 2023-12-07T10:36:25.696 INFO:teuthology.orchestra.run.smithi119.stdout: "reply": 0, 2023-12-07T10:36:25.696 INFO:teuthology.orchestra.run.smithi119.stdout: "reply_latency": { 2023-12-07T10:36:25.696 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.696 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.696 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.697 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_reply": 0, 2023-12-07T10:36:25.697 INFO:teuthology.orchestra.run.smithi119.stdout: "forward": 0, 2023-12-07T10:36:25.697 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_complete": 0, 2023-12-07T10:36:25.697 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_keys": 0, 2023-12-07T10:36:25.697 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_commit": 0, 2023-12-07T10:36:25.697 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_split": 0, 2023-12-07T10:36:25.697 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_merge": 0, 2023-12-07T10:36:25.697 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes": 13998, 2023-12-07T10:36:25.697 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_top": 35, 2023-12-07T10:36:25.698 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_bottom": 13963, 2023-12-07T10:36:25.698 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pin_tail": 0, 2023-12-07T10:36:25.698 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pinned": 13948, 2023-12-07T10:36:25.698 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_expired": 0, 2023-12-07T10:36:25.698 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_with_caps": 0, 2023-12-07T10:36:25.698 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": 0, 2023-12-07T10:36:25.698 INFO:teuthology.orchestra.run.smithi119.stdout: "subtrees": 10, 2023-12-07T10:36:25.698 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse": 0, 2023-12-07T10:36:25.699 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_hit": 0, 2023-12-07T10:36:25.699 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_forward": 0, 2023-12-07T10:36:25.699 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_discover": 0, 2023-12-07T10:36:25.699 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_dir_fetch": 0, 2023-12-07T10:36:25.699 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_remote_ino": 0, 2023-12-07T10:36:25.699 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_lock": 0, 2023-12-07T10:36:25.699 INFO:teuthology.orchestra.run.smithi119.stdout: "load_cent": 0, 2023-12-07T10:36:25.699 INFO:teuthology.orchestra.run.smithi119.stdout: "q": 0, 2023-12-07T10:36:25.699 INFO:teuthology.orchestra.run.smithi119.stdout: "exported": 0, 2023-12-07T10:36:25.700 INFO:teuthology.orchestra.run.smithi119.stdout: "exported_inodes": 0, 2023-12-07T10:36:25.700 INFO:teuthology.orchestra.run.smithi119.stdout: "imported": 0, 2023-12-07T10:36:25.700 INFO:teuthology.orchestra.run.smithi119.stdout: "imported_inodes": 0, 2023-12-07T10:36:25.700 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_dir_fetch": 0, 2023-12-07T10:36:25.700 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_backtrace_fetch": 0, 2023-12-07T10:36:25.700 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_peer_discover": 0, 2023-12-07T10:36:25.700 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rfiles": 0, 2023-12-07T10:36:25.700 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rbytes": 0, 2023-12-07T10:36:25.700 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rsnaps": 0, 2023-12-07T10:36:25.701 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_fetch": 0, 2023-12-07T10:36:25.701 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_set_tag": 0, 2023-12-07T10:36:25.701 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_repaired": 0, 2023-12-07T10:36:25.701 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_inotable_repaired": 0, 2023-12-07T10:36:25.701 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_inodes": 0, 2023-12-07T10:36:25.701 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_base_inodes": 0, 2023-12-07T10:36:25.701 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dirfrag_rstats": 0, 2023-12-07T10:36:25.701 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_file_inodes": 0, 2023-12-07T10:36:25.701 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_inode_file_caps": 0, 2023-12-07T10:36:25.702 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_revoke": 0, 2023-12-07T10:36:25.702 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_grant": 0, 2023-12-07T10:36:25.702 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_trunc": 0, 2023-12-07T10:36:25.702 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-07T10:36:25.702 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-07T10:36:25.702 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps": 0, 2023-12-07T10:36:25.702 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps_dirty": 0, 2023-12-07T10:36:25.702 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_cap_release": 0, 2023-12-07T10:36:25.702 INFO:teuthology.orchestra.run.smithi119.stdout: "process_request_cap_release": 0 2023-12-07T10:36:25.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.703 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cache": { 2023-12-07T10:36:25.703 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update": 0, 2023-12-07T10:36:25.703 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update_receipt": 0, 2023-12-07T10:36:25.703 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_try_discover": 0, 2023-12-07T10:36:25.703 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_send_discover": 0, 2023-12-07T10:36:25.703 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_handle_discover": 0, 2023-12-07T10:36:25.703 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays": 0, 2023-12-07T10:36:25.703 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_delayed": 0, 2023-12-07T10:36:25.704 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_enqueuing": 0, 2023-12-07T10:36:25.704 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_created": 0, 2023-12-07T10:36:25.704 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_enqueued": 0, 2023-12-07T10:36:25.704 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_reintegrated": 0, 2023-12-07T10:36:25.704 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_migrated": 0, 2023-12-07T10:36:25.704 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_processing": 0, 2023-12-07T10:36:25.704 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_enqueued": 0, 2023-12-07T10:36:25.704 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_prioritized": 0, 2023-12-07T10:36:25.704 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_started": 0, 2023-12-07T10:36:25.705 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_completed": 0, 2023-12-07T10:36:25.705 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_enqueue_scrub": 0, 2023-12-07T10:36:25.705 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_exportdir": 0, 2023-12-07T10:36:25.705 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_flush": 0, 2023-12-07T10:36:25.705 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragmentdir": 0, 2023-12-07T10:36:25.705 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragstats": 0, 2023-12-07T10:36:25.705 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_inodestats": 0 2023-12-07T10:36:25.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.705 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_log": { 2023-12-07T10:36:25.706 INFO:teuthology.orchestra.run.smithi119.stdout: "evlrg": 0, 2023-12-07T10:36:25.706 INFO:teuthology.orchestra.run.smithi119.stdout: "evadd": 0, 2023-12-07T10:36:25.706 INFO:teuthology.orchestra.run.smithi119.stdout: "evex": 0, 2023-12-07T10:36:25.706 INFO:teuthology.orchestra.run.smithi119.stdout: "evtrm": 0, 2023-12-07T10:36:25.706 INFO:teuthology.orchestra.run.smithi119.stdout: "ev": 84658, 2023-12-07T10:36:25.706 INFO:teuthology.orchestra.run.smithi119.stdout: "evexg": 0, 2023-12-07T10:36:25.706 INFO:teuthology.orchestra.run.smithi119.stdout: "evexd": 0, 2023-12-07T10:36:25.706 INFO:teuthology.orchestra.run.smithi119.stdout: "segadd": 0, 2023-12-07T10:36:25.707 INFO:teuthology.orchestra.run.smithi119.stdout: "segex": 0, 2023-12-07T10:36:25.707 INFO:teuthology.orchestra.run.smithi119.stdout: "segtrm": 0, 2023-12-07T10:36:25.707 INFO:teuthology.orchestra.run.smithi119.stdout: "seg": 114, 2023-12-07T10:36:25.707 INFO:teuthology.orchestra.run.smithi119.stdout: "segmjr": 8, 2023-12-07T10:36:25.707 INFO:teuthology.orchestra.run.smithi119.stdout: "segexg": 0, 2023-12-07T10:36:25.707 INFO:teuthology.orchestra.run.smithi119.stdout: "segexd": 0, 2023-12-07T10:36:25.707 INFO:teuthology.orchestra.run.smithi119.stdout: "expos": 4194304, 2023-12-07T10:36:25.707 INFO:teuthology.orchestra.run.smithi119.stdout: "wrpos": 213332721, 2023-12-07T10:36:25.707 INFO:teuthology.orchestra.run.smithi119.stdout: "rdpos": 213332122, 2023-12-07T10:36:25.708 INFO:teuthology.orchestra.run.smithi119.stdout: "jlat": { 2023-12-07T10:36:25.708 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.708 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.708 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.708 INFO:teuthology.orchestra.run.smithi119.stdout: "replayed": 84658 2023-12-07T10:36:25.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.708 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_mem": { 2023-12-07T10:36:25.708 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 680, 2023-12-07T10:36:25.708 INFO:teuthology.orchestra.run.smithi119.stdout: "ino+": 13996, 2023-12-07T10:36:25.709 INFO:teuthology.orchestra.run.smithi119.stdout: "ino-": 13316, 2023-12-07T10:36:25.709 INFO:teuthology.orchestra.run.smithi119.stdout: "dir": 45, 2023-12-07T10:36:25.709 INFO:teuthology.orchestra.run.smithi119.stdout: "dir+": 76, 2023-12-07T10:36:25.709 INFO:teuthology.orchestra.run.smithi119.stdout: "dir-": 31, 2023-12-07T10:36:25.709 INFO:teuthology.orchestra.run.smithi119.stdout: "dn": 13998, 2023-12-07T10:36:25.709 INFO:teuthology.orchestra.run.smithi119.stdout: "dn+": 14146, 2023-12-07T10:36:25.709 INFO:teuthology.orchestra.run.smithi119.stdout: "dn-": 148, 2023-12-07T10:36:25.709 INFO:teuthology.orchestra.run.smithi119.stdout: "cap": 0, 2023-12-07T10:36:25.709 INFO:teuthology.orchestra.run.smithi119.stdout: "cap+": 0, 2023-12-07T10:36:25.710 INFO:teuthology.orchestra.run.smithi119.stdout: "cap-": 0, 2023-12-07T10:36:25.710 INFO:teuthology.orchestra.run.smithi119.stdout: "rss": 65896, 2023-12-07T10:36:25.710 INFO:teuthology.orchestra.run.smithi119.stdout: "heap": 207228 2023-12-07T10:36:25.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.710 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_server": { 2023-12-07T10:36:25.710 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_client_request": 0, 2023-12-07T10:36:25.710 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_server_request": 0, 2023-12-07T10:36:25.710 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_request": 0, 2023-12-07T10:36:25.710 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_session": 0, 2023-12-07T10:36:25.711 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_peer_request": 0, 2023-12-07T10:36:25.711 INFO:teuthology.orchestra.run.smithi119.stdout: "req_create_latency": { 2023-12-07T10:36:25.711 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.711 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.711 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.711 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getattr_latency": { 2023-12-07T10:36:25.711 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.711 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.712 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.712 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getfilelock_latency": { 2023-12-07T10:36:25.712 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.712 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.712 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.712 INFO:teuthology.orchestra.run.smithi119.stdout: "req_link_latency": { 2023-12-07T10:36:25.712 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.713 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.713 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.713 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookup_latency": { 2023-12-07T10:36:25.713 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.713 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.713 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.713 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookuphash_latency": { 2023-12-07T10:36:25.714 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.714 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.714 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.714 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupino_latency": { 2023-12-07T10:36:25.714 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.714 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.714 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.714 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupname_latency": { 2023-12-07T10:36:25.715 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.715 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.715 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.715 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupparent_latency": { 2023-12-07T10:36:25.715 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.715 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.715 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.716 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupsnap_latency": { 2023-12-07T10:36:25.716 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.716 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.716 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.716 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lssnap_latency": { 2023-12-07T10:36:25.716 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.716 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.717 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.717 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mkdir_latency": { 2023-12-07T10:36:25.717 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.717 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.717 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.717 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mknod_latency": { 2023-12-07T10:36:25.717 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.718 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.718 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.718 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mksnap_latency": { 2023-12-07T10:36:25.718 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.718 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.718 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.719 INFO:teuthology.orchestra.run.smithi119.stdout: "req_open_latency": { 2023-12-07T10:36:25.719 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.719 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.719 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.719 INFO:teuthology.orchestra.run.smithi119.stdout: "req_readdir_latency": { 2023-12-07T10:36:25.719 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.719 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.719 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.720 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rename_latency": { 2023-12-07T10:36:25.720 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.720 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.720 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.720 INFO:teuthology.orchestra.run.smithi119.stdout: "req_renamesnap_latency": { 2023-12-07T10:36:25.720 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.720 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.720 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.721 INFO:teuthology.orchestra.run.smithi119.stdout: "req_snapdiff_latency": { 2023-12-07T10:36:25.721 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.721 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.721 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.721 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmdir_latency": { 2023-12-07T10:36:25.721 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.721 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.721 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.722 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmsnap_latency": { 2023-12-07T10:36:25.722 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.722 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.722 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.722 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmxattr_latency": { 2023-12-07T10:36:25.722 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.722 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.723 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.723 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setattr_latency": { 2023-12-07T10:36:25.723 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.723 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.723 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.723 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setdirlayout_latency": { 2023-12-07T10:36:25.723 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.723 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.724 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.724 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setfilelock_latency": { 2023-12-07T10:36:25.724 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.724 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.724 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.724 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setlayout_latency": { 2023-12-07T10:36:25.724 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.725 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.725 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.725 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setxattr_latency": { 2023-12-07T10:36:25.725 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.725 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.725 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.725 INFO:teuthology.orchestra.run.smithi119.stdout: "req_symlink_latency": { 2023-12-07T10:36:25.726 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.726 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.726 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.726 INFO:teuthology.orchestra.run.smithi119.stdout: "req_unlink_latency": { 2023-12-07T10:36:25.726 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.726 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.726 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.727 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_revoke_eviction": 0, 2023-12-07T10:36:25.727 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_acquisition_throttle": 0, 2023-12-07T10:36:25.727 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getvxattr_latency": { 2023-12-07T10:36:25.727 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.727 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.727 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.727 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:25.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.727 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_sessions": { 2023-12-07T10:36:25.728 INFO:teuthology.orchestra.run.smithi119.stdout: "session_count": 2, 2023-12-07T10:36:25.728 INFO:teuthology.orchestra.run.smithi119.stdout: "session_add": 2, 2023-12-07T10:36:25.728 INFO:teuthology.orchestra.run.smithi119.stdout: "session_remove": 0, 2023-12-07T10:36:25.728 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_open": 2, 2023-12-07T10:36:25.728 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_stale": 0, 2023-12-07T10:36:25.728 INFO:teuthology.orchestra.run.smithi119.stdout: "total_load": 0, 2023-12-07T10:36:25.728 INFO:teuthology.orchestra.run.smithi119.stdout: "average_load": 0, 2023-12-07T10:36:25.728 INFO:teuthology.orchestra.run.smithi119.stdout: "avg_session_uptime": 2591, 2023-12-07T10:36:25.728 INFO:teuthology.orchestra.run.smithi119.stdout: "mdthresh_evicted": 0 2023-12-07T10:36:25.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.729 INFO:teuthology.orchestra.run.smithi119.stdout: "mempool": { 2023-12-07T10:36:25.729 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_bytes": 0, 2023-12-07T10:36:25.729 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_items": 0, 2023-12-07T10:36:25.729 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_bytes": 0, 2023-12-07T10:36:25.729 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_items": 0, 2023-12-07T10:36:25.729 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_bytes": 0, 2023-12-07T10:36:25.729 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_items": 0, 2023-12-07T10:36:25.729 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-07T10:36:25.729 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_items": 0, 2023-12-07T10:36:25.730 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-07T10:36:25.730 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_items": 0, 2023-12-07T10:36:25.730 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_bytes": 0, 2023-12-07T10:36:25.730 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_items": 0, 2023-12-07T10:36:25.730 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-07T10:36:25.730 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_items": 0, 2023-12-07T10:36:25.730 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_bytes": 0, 2023-12-07T10:36:25.730 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_items": 0, 2023-12-07T10:36:25.730 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_bytes": 0, 2023-12-07T10:36:25.731 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_items": 0, 2023-12-07T10:36:25.731 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-07T10:36:25.731 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_items": 0, 2023-12-07T10:36:25.731 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-07T10:36:25.731 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_items": 0, 2023-12-07T10:36:25.731 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_bytes": 0, 2023-12-07T10:36:25.731 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_items": 0, 2023-12-07T10:36:25.731 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_bytes": 0, 2023-12-07T10:36:25.731 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_items": 0, 2023-12-07T10:36:25.732 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-07T10:36:25.732 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_items": 0, 2023-12-07T10:36:25.732 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_bytes": 0, 2023-12-07T10:36:25.732 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_items": 0, 2023-12-07T10:36:25.732 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_bytes": 0, 2023-12-07T10:36:25.732 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_items": 0, 2023-12-07T10:36:25.732 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_bytes": 0, 2023-12-07T10:36:25.732 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_items": 0, 2023-12-07T10:36:25.732 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_bytes": 0, 2023-12-07T10:36:25.733 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_items": 0, 2023-12-07T10:36:25.733 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_bytes": 264909, 2023-12-07T10:36:25.733 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_items": 808, 2023-12-07T10:36:25.733 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_bytes": 0, 2023-12-07T10:36:25.733 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_items": 0, 2023-12-07T10:36:25.733 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_bytes": 0, 2023-12-07T10:36:25.733 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_items": 0, 2023-12-07T10:36:25.733 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_bytes": 0, 2023-12-07T10:36:25.733 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_items": 0, 2023-12-07T10:36:25.734 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_bytes": 0, 2023-12-07T10:36:25.734 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_items": 0, 2023-12-07T10:36:25.734 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_bytes": 9616, 2023-12-07T10:36:25.734 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_items": 185, 2023-12-07T10:36:25.734 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_bytes": 0, 2023-12-07T10:36:25.734 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_items": 0, 2023-12-07T10:36:25.734 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_bytes": 0, 2023-12-07T10:36:25.734 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_items": 0, 2023-12-07T10:36:25.734 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_bytes": 11643138, 2023-12-07T10:36:25.735 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_items": 73424, 2023-12-07T10:36:25.735 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_bytes": 0, 2023-12-07T10:36:25.735 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_items": 0, 2023-12-07T10:36:25.735 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_bytes": 0, 2023-12-07T10:36:25.735 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_items": 0 2023-12-07T10:36:25.735 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.735 INFO:teuthology.orchestra.run.smithi119.stdout: "objecter": { 2023-12-07T10:36:25.735 INFO:teuthology.orchestra.run.smithi119.stdout: "op_active": 0, 2023-12-07T10:36:25.735 INFO:teuthology.orchestra.run.smithi119.stdout: "op_laggy": 0, 2023-12-07T10:36:25.736 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send": 3926, 2023-12-07T10:36:25.736 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send_bytes": 21, 2023-12-07T10:36:25.736 INFO:teuthology.orchestra.run.smithi119.stdout: "op_resend": 3, 2023-12-07T10:36:25.736 INFO:teuthology.orchestra.run.smithi119.stdout: "op_reply": 3926, 2023-12-07T10:36:25.736 INFO:teuthology.orchestra.run.smithi119.stdout: "op_latency": { 2023-12-07T10:36:25.736 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 3926, 2023-12-07T10:36:25.736 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 9.797766023, 2023-12-07T10:36:25.736 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.002495610 2023-12-07T10:36:25.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.737 INFO:teuthology.orchestra.run.smithi119.stdout: "op_inflight": 0, 2023-12-07T10:36:25.737 INFO:teuthology.orchestra.run.smithi119.stdout: "oplen_avg": { 2023-12-07T10:36:25.737 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 3926, 2023-12-07T10:36:25.737 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 3927 2023-12-07T10:36:25.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.737 INFO:teuthology.orchestra.run.smithi119.stdout: "op": 3926, 2023-12-07T10:36:25.737 INFO:teuthology.orchestra.run.smithi119.stdout: "op_r": 3926, 2023-12-07T10:36:25.737 INFO:teuthology.orchestra.run.smithi119.stdout: "op_w": 0, 2023-12-07T10:36:25.738 INFO:teuthology.orchestra.run.smithi119.stdout: "op_rmw": 0, 2023-12-07T10:36:25.738 INFO:teuthology.orchestra.run.smithi119.stdout: "op_pg": 0, 2023-12-07T10:36:25.738 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_stat": 2061, 2023-12-07T10:36:25.738 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_create": 0, 2023-12-07T10:36:25.738 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_read": 1863, 2023-12-07T10:36:25.738 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_write": 0, 2023-12-07T10:36:25.738 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writefull": 0, 2023-12-07T10:36:25.738 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writesame": 0, 2023-12-07T10:36:25.739 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_append": 0, 2023-12-07T10:36:25.739 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_zero": 0, 2023-12-07T10:36:25.739 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_truncate": 0, 2023-12-07T10:36:25.739 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_delete": 0, 2023-12-07T10:36:25.739 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_mapext": 0, 2023-12-07T10:36:25.739 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_sparse_read": 0, 2023-12-07T10:36:25.739 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_clonerange": 0, 2023-12-07T10:36:25.739 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_getxattr": 1, 2023-12-07T10:36:25.739 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_setxattr": 0, 2023-12-07T10:36:25.740 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_cmpxattr": 0, 2023-12-07T10:36:25.740 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_rmxattr": 0, 2023-12-07T10:36:25.740 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_resetxattrs": 0, 2023-12-07T10:36:25.740 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_call": 0, 2023-12-07T10:36:25.740 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_watch": 0, 2023-12-07T10:36:25.740 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_notify": 0, 2023-12-07T10:36:25.740 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_src_cmpxattr": 0, 2023-12-07T10:36:25.740 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls": 0, 2023-12-07T10:36:25.741 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls_filter": 0, 2023-12-07T10:36:25.741 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_other": 0, 2023-12-07T10:36:25.741 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_active": 0, 2023-12-07T10:36:25.741 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_send": 0, 2023-12-07T10:36:25.741 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_resend": 0, 2023-12-07T10:36:25.741 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_ping": 0, 2023-12-07T10:36:25.741 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_active": 0, 2023-12-07T10:36:25.741 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_send": 0, 2023-12-07T10:36:25.741 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_resend": 0, 2023-12-07T10:36:25.742 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_active": 0, 2023-12-07T10:36:25.742 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_send": 0, 2023-12-07T10:36:25.742 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_resend": 0, 2023-12-07T10:36:25.742 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_active": 0, 2023-12-07T10:36:25.742 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_send": 0, 2023-12-07T10:36:25.742 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_resend": 0, 2023-12-07T10:36:25.742 INFO:teuthology.orchestra.run.smithi119.stdout: "command_active": 0, 2023-12-07T10:36:25.742 INFO:teuthology.orchestra.run.smithi119.stdout: "command_send": 0, 2023-12-07T10:36:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: "command_resend": 0, 2023-12-07T10:36:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: "map_epoch": 114, 2023-12-07T10:36:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: "map_full": 0, 2023-12-07T10:36:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: "map_inc": 23, 2023-12-07T10:36:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_sessions": 11, 2023-12-07T10:36:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_open": 11, 2023-12-07T10:36:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_close": 0, 2023-12-07T10:36:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_laggy": 0, 2023-12-07T10:36:25.743 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_wr": 0, 2023-12-07T10:36:25.744 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_rd": 2, 2023-12-07T10:36:25.744 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_del": 0 2023-12-07T10:36:25.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.744 INFO:teuthology.orchestra.run.smithi119.stdout: "oft": { 2023-12-07T10:36:25.744 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_objs": 0, 2023-12-07T10:36:25.744 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_kv_pairs": 0, 2023-12-07T10:36:25.744 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_updates": 0, 2023-12-07T10:36:25.744 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_removes": 0 2023-12-07T10:36:25.744 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.745 INFO:teuthology.orchestra.run.smithi119.stdout: "osd-slow-ops": { 2023-12-07T10:36:25.745 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_ops_count": 0 2023-12-07T10:36:25.745 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.745 INFO:teuthology.orchestra.run.smithi119.stdout: "purge_queue": { 2023-12-07T10:36:25.745 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops": 0, 2023-12-07T10:36:25.745 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops_high_water": 0, 2023-12-07T10:36:25.745 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing": 0, 2023-12-07T10:36:25.745 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_high_water": 0, 2023-12-07T10:36:25.746 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed": 0, 2023-12-07T10:36:25.746 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_item_in_journal": 0 2023-12-07T10:36:25.746 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.746 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-07T10:36:25.746 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2023-12-07T10:36:25.746 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2023-12-07T10:36:25.746 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2023-12-07T10:36:25.746 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 4318, 2023-12-07T10:36:25.747 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 211755119, 2023-12-07T10:36:25.747 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:25.747 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 4318, 2023-12-07T10:36:25.747 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2023-12-07T10:36:25.747 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2023-12-07T10:36:25.747 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 4318, 2023-12-07T10:36:25.747 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 211755119, 2023-12-07T10:36:25.747 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2023-12-07T10:36:25.747 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.748 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.748 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.748 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:25.748 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.748 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_bytes": { 2023-12-07T10:36:25.748 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2023-12-07T10:36:25.748 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2023-12-07T10:36:25.748 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2023-12-07T10:36:25.748 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2023-12-07T10:36:25.748 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2023-12-07T10:36:25.749 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:25.749 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:25.749 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 3926, 2023-12-07T10:36:25.749 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 209147986, 2023-12-07T10:36:25.749 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 3926, 2023-12-07T10:36:25.749 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 209147986, 2023-12-07T10:36:25.749 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2023-12-07T10:36:25.749 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.749 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.750 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.750 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:25.750 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.750 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_ops": { 2023-12-07T10:36:25.750 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2023-12-07T10:36:25.750 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 1024, 2023-12-07T10:36:25.750 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2023-12-07T10:36:25.750 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2023-12-07T10:36:25.750 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2023-12-07T10:36:25.751 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:25.751 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:25.751 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 3926, 2023-12-07T10:36:25.751 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 3926, 2023-12-07T10:36:25.751 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 3926, 2023-12-07T10:36:25.751 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 3926, 2023-12-07T10:36:25.751 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2023-12-07T10:36:25.751 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.751 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.751 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.752 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:25.752 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.752 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle": { 2023-12-07T10:36:25.752 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2023-12-07T10:36:25.752 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2023-12-07T10:36:25.752 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2023-12-07T10:36:25.752 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2023-12-07T10:36:25.752 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2023-12-07T10:36:25.753 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:25.753 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:25.753 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2023-12-07T10:36:25.753 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2023-12-07T10:36:25.753 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 0, 2023-12-07T10:36:25.753 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 0, 2023-12-07T10:36:25.753 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2023-12-07T10:36:25.753 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.753 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.754 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.754 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:25.754 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:25.754 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle-0x5625e3936820": { 2023-12-07T10:36:25.754 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2023-12-07T10:36:25.754 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2023-12-07T10:36:25.754 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2023-12-07T10:36:25.754 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2023-12-07T10:36:25.755 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2023-12-07T10:36:25.755 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:25.755 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:25.755 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2023-12-07T10:36:25.755 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2023-12-07T10:36:25.755 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 0, 2023-12-07T10:36:25.755 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 0, 2023-12-07T10:36:25.755 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2023-12-07T10:36:25.755 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:25.756 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:25.756 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:25.756 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:25.756 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:25.756 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:36:26.179 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2023-12-07T10:36:26.180 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2023-12-07T10:36:26.180 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2023-12-07T10:36:26.180 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2023-12-07T10:36:26.180 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2023-12-07T10:36:26.180 DEBUG:tasks.check_counter:Getting stats from d 2023-12-07T10:36:26.180 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph daemon mds.d perf dump 2023-12-07T10:36:26.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:26 smithi119 ceph-mon[97908]: pgmap v503: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 396 KiB/s wr, 66 op/s 2023-12-07T10:36:26.549 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:26 smithi153 ceph-mon[133625]: pgmap v503: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 396 KiB/s wr, 66 op/s 2023-12-07T10:36:26.549 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:26 smithi159 ceph-mon[134726]: pgmap v503: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 396 KiB/s wr, 66 op/s 2023-12-07T10:36:26.995 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:36:28.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:28 smithi119 ceph-mon[97908]: pgmap v504: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 89 KiB/s wr, 30 op/s 2023-12-07T10:36:28.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:28 smithi159 ceph-mon[134726]: pgmap v504: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 89 KiB/s wr, 30 op/s 2023-12-07T10:36:28.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:28 smithi153 ceph-mon[133625]: pgmap v504: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 89 KiB/s wr, 30 op/s 2023-12-07T10:36:28.562 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:36:28.562 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-0": { 2023-12-07T10:36:28.562 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 323, 2023-12-07T10:36:28.562 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 310, 2023-12-07T10:36:28.562 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 1596640, 2023-12-07T10:36:28.563 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 185792, 2023-12-07T10:36:28.563 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 4, 2023-12-07T10:36:28.563 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 0, 2023-12-07T10:36:28.563 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.074659620, 2023-12-07T10:36:28.563 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.032768090, 2023-12-07T10:36:28.563 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.043476639, 2023-12-07T10:36:28.563 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.006391298, 2023-12-07T10:36:28.563 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:28.563 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 310, 2023-12-07T10:36:28.564 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.018262674, 2023-12-07T10:36:28.564 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000058911 2023-12-07T10:36:28.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:28.564 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:28.564 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:28.564 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:28.564 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:28.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:28.564 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 1596640, 2023-12-07T10:36:28.564 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 185792 2023-12-07T10:36:28.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:28.565 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-1": { 2023-12-07T10:36:28.565 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 3, 2023-12-07T10:36:28.565 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 236, 2023-12-07T10:36:28.565 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 480, 2023-12-07T10:36:28.565 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 272512, 2023-12-07T10:36:28.565 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 6, 2023-12-07T10:36:28.565 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 3, 2023-12-07T10:36:28.565 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.025091364, 2023-12-07T10:36:28.565 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.016737612, 2023-12-07T10:36:28.566 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.003455152, 2023-12-07T10:36:28.566 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-07T10:36:28.566 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:28.566 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 233, 2023-12-07T10:36:28.566 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.036615901, 2023-12-07T10:36:28.566 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000157149 2023-12-07T10:36:28.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:28.566 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:28.566 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:28.567 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:28.567 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:28.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:28.567 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 480, 2023-12-07T10:36:28.567 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 272512 2023-12-07T10:36:28.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:28.567 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-2": { 2023-12-07T10:36:28.567 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 0, 2023-12-07T10:36:28.568 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 2, 2023-12-07T10:36:28.568 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 0, 2023-12-07T10:36:28.568 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 0, 2023-12-07T10:36:28.568 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 2, 2023-12-07T10:36:28.568 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 1, 2023-12-07T10:36:28.568 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.000917027, 2023-12-07T10:36:28.568 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.000000000, 2023-12-07T10:36:28.568 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.000695256, 2023-12-07T10:36:28.568 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-07T10:36:28.569 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:28.569 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:28.569 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:28.569 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:28.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:28.569 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:28.569 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:28.569 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:28.569 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:28.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:28.570 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-07T10:36:28.570 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 0 2023-12-07T10:36:28.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:28.570 INFO:teuthology.orchestra.run.smithi119.stdout: "cct": { 2023-12-07T10:36:28.570 INFO:teuthology.orchestra.run.smithi119.stdout: "total_workers": 0, 2023-12-07T10:36:28.570 INFO:teuthology.orchestra.run.smithi119.stdout: "unhealthy_workers": 0 2023-12-07T10:36:28.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:28.570 INFO:teuthology.orchestra.run.smithi119.stdout: "mempool": { 2023-12-07T10:36:28.571 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_bytes": 0, 2023-12-07T10:36:28.571 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_items": 0, 2023-12-07T10:36:28.571 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_bytes": 0, 2023-12-07T10:36:28.571 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_items": 0, 2023-12-07T10:36:28.571 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_bytes": 0, 2023-12-07T10:36:28.571 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_items": 0, 2023-12-07T10:36:28.571 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-07T10:36:28.571 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_items": 0, 2023-12-07T10:36:28.571 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-07T10:36:28.572 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_items": 0, 2023-12-07T10:36:28.572 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_bytes": 0, 2023-12-07T10:36:28.572 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_items": 0, 2023-12-07T10:36:28.572 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-07T10:36:28.572 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_items": 0, 2023-12-07T10:36:28.572 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_bytes": 0, 2023-12-07T10:36:28.572 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_items": 0, 2023-12-07T10:36:28.572 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_bytes": 0, 2023-12-07T10:36:28.573 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_items": 0, 2023-12-07T10:36:28.573 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-07T10:36:28.573 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_items": 0, 2023-12-07T10:36:28.573 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-07T10:36:28.573 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_items": 0, 2023-12-07T10:36:28.573 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_bytes": 0, 2023-12-07T10:36:28.573 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_items": 0, 2023-12-07T10:36:28.573 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_bytes": 0, 2023-12-07T10:36:28.573 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_items": 0, 2023-12-07T10:36:28.574 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-07T10:36:28.574 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_items": 0, 2023-12-07T10:36:28.574 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_bytes": 0, 2023-12-07T10:36:28.574 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_items": 0, 2023-12-07T10:36:28.574 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_bytes": 0, 2023-12-07T10:36:28.574 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_items": 0, 2023-12-07T10:36:28.574 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_bytes": 0, 2023-12-07T10:36:28.574 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_items": 0, 2023-12-07T10:36:28.575 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_bytes": 0, 2023-12-07T10:36:28.575 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_items": 0, 2023-12-07T10:36:28.575 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_bytes": 147930, 2023-12-07T10:36:28.575 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_items": 28, 2023-12-07T10:36:28.575 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_bytes": 0, 2023-12-07T10:36:28.575 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_items": 0, 2023-12-07T10:36:28.575 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_bytes": 0, 2023-12-07T10:36:28.575 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_items": 0, 2023-12-07T10:36:28.575 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_bytes": 0, 2023-12-07T10:36:28.576 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_items": 0, 2023-12-07T10:36:28.576 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_bytes": 0, 2023-12-07T10:36:28.576 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_items": 0, 2023-12-07T10:36:28.576 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_bytes": 0, 2023-12-07T10:36:28.576 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_items": 0, 2023-12-07T10:36:28.576 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_bytes": 0, 2023-12-07T10:36:28.576 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_items": 0, 2023-12-07T10:36:28.576 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_bytes": 0, 2023-12-07T10:36:28.576 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_items": 0, 2023-12-07T10:36:28.577 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_bytes": 928, 2023-12-07T10:36:28.577 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_items": 2, 2023-12-07T10:36:28.577 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_bytes": 0, 2023-12-07T10:36:28.577 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_items": 0, 2023-12-07T10:36:28.577 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_bytes": 0, 2023-12-07T10:36:28.577 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_items": 0 2023-12-07T10:36:28.577 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:28.578 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-07T10:36:28.578 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2023-12-07T10:36:28.578 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2023-12-07T10:36:28.578 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2023-12-07T10:36:28.578 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 322, 2023-12-07T10:36:28.578 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 1548668, 2023-12-07T10:36:28.578 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:28.578 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 322, 2023-12-07T10:36:28.579 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2023-12-07T10:36:28.579 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2023-12-07T10:36:28.579 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 322, 2023-12-07T10:36:28.579 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 1548668, 2023-12-07T10:36:28.579 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2023-12-07T10:36:28.579 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:28.580 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:28.580 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:28.580 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:28.580 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:28.580 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:36:29.117 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-12-07T10:36:29.117 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-12-07T10:36:29.117 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2023-12-07T10:36:29.117 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2023-12-07T10:36:29.117 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2023-12-07T10:36:29.118 DEBUG:tasks.check_counter:Getting stats from g 2023-12-07T10:36:29.118 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph daemon mds.g perf dump 2023-12-07T10:36:29.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:29 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:36:29.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:29 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:36:29.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:29 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:36:29.896 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:36:30.473 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:30 smithi119 ceph-mon[97908]: pgmap v505: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 38 KiB/s wr, 18 op/s 2023-12-07T10:36:30.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:30 smithi159 ceph-mon[134726]: pgmap v505: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 38 KiB/s wr, 18 op/s 2023-12-07T10:36:30.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:30 smithi153 ceph-mon[133625]: pgmap v505: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 38 KiB/s wr, 18 op/s 2023-12-07T10:36:31.488 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:36:31.488 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-0": { 2023-12-07T10:36:31.488 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 224937, 2023-12-07T10:36:31.488 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 299446, 2023-12-07T10:36:31.488 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 53951239, 2023-12-07T10:36:31.489 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 484806106, 2023-12-07T10:36:31.489 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 12, 2023-12-07T10:36:31.489 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 5, 2023-12-07T10:36:31.489 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 21.496162392, 2023-12-07T10:36:31.489 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 9.486824112, 2023-12-07T10:36:31.489 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 11.732707503, 2023-12-07T10:36:31.489 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 1.650037381, 2023-12-07T10:36:31.489 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:31.489 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 299446, 2023-12-07T10:36:31.490 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 13.953204631, 2023-12-07T10:36:31.490 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000046596 2023-12-07T10:36:31.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.490 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:31.490 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 436182, 2023-12-07T10:36:31.490 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.071550570, 2023-12-07T10:36:31.490 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000164 2023-12-07T10:36:31.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.491 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-07T10:36:31.491 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 0 2023-12-07T10:36:31.491 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.491 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-1": { 2023-12-07T10:36:31.491 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 318490, 2023-12-07T10:36:31.491 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 404739, 2023-12-07T10:36:31.491 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 105398682, 2023-12-07T10:36:31.491 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 381761615, 2023-12-07T10:36:31.491 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 12, 2023-12-07T10:36:31.492 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 8, 2023-12-07T10:36:31.492 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 52.074513014, 2023-12-07T10:36:31.492 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 33.994547077, 2023-12-07T10:36:31.492 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 21.822578578, 2023-12-07T10:36:31.492 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.743086928, 2023-12-07T10:36:31.492 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:31.492 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 404737, 2023-12-07T10:36:31.492 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 31.754713562, 2023-12-07T10:36:31.492 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000078457 2023-12-07T10:36:31.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.493 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:31.493 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 617728, 2023-12-07T10:36:31.493 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.161455925, 2023-12-07T10:36:31.493 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000261 2023-12-07T10:36:31.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.493 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 81377856, 2023-12-07T10:36:31.493 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 215208688 2023-12-07T10:36:31.493 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.494 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-2": { 2023-12-07T10:36:31.494 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 110883, 2023-12-07T10:36:31.494 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 127039, 2023-12-07T10:36:31.494 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 27525865, 2023-12-07T10:36:31.494 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 190475369, 2023-12-07T10:36:31.494 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 31, 2023-12-07T10:36:31.495 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 4, 2023-12-07T10:36:31.495 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 10.721749557, 2023-12-07T10:36:31.495 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 4.579236518, 2023-12-07T10:36:31.495 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 6.086392409, 2023-12-07T10:36:31.495 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.789451669, 2023-12-07T10:36:31.495 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:31.495 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 127036, 2023-12-07T10:36:31.496 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 7.092934995, 2023-12-07T10:36:31.496 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000055834 2023-12-07T10:36:31.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.496 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:31.496 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 194887, 2023-12-07T10:36:31.496 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.036832840, 2023-12-07T10:36:31.496 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000188 2023-12-07T10:36:31.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.497 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 2928, 2023-12-07T10:36:31.497 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 135760 2023-12-07T10:36:31.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.497 INFO:teuthology.orchestra.run.smithi119.stdout: "cct": { 2023-12-07T10:36:31.497 INFO:teuthology.orchestra.run.smithi119.stdout: "total_workers": 1, 2023-12-07T10:36:31.497 INFO:teuthology.orchestra.run.smithi119.stdout: "unhealthy_workers": 0 2023-12-07T10:36:31.497 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.497 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-MDSRank": { 2023-12-07T10:36:31.497 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2023-12-07T10:36:31.498 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2023-12-07T10:36:31.498 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 112680, 2023-12-07T10:36:31.498 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 80.675532489, 2023-12-07T10:36:31.498 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000715970 2023-12-07T10:36:31.498 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:31.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.498 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-PurgeQueue": { 2023-12-07T10:36:31.498 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2023-12-07T10:36:31.499 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2023-12-07T10:36:31.499 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 21701, 2023-12-07T10:36:31.499 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 5.436062300, 2023-12-07T10:36:31.499 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000250498 2023-12-07T10:36:31.499 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:31.499 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.499 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2023-12-07T10:36:31.499 INFO:teuthology.orchestra.run.smithi119.stdout: "request": 145173, 2023-12-07T10:36:31.499 INFO:teuthology.orchestra.run.smithi119.stdout: "reply": 124640, 2023-12-07T10:36:31.500 INFO:teuthology.orchestra.run.smithi119.stdout: "reply_latency": { 2023-12-07T10:36:31.500 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 124640, 2023-12-07T10:36:31.500 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 596.422007255, 2023-12-07T10:36:31.500 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.004785157 2023-12-07T10:36:31.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.500 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_reply": 0, 2023-12-07T10:36:31.500 INFO:teuthology.orchestra.run.smithi119.stdout: "forward": 20533, 2023-12-07T10:36:31.500 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_complete": 5, 2023-12-07T10:36:31.501 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_keys": 0, 2023-12-07T10:36:31.501 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_commit": 45, 2023-12-07T10:36:31.501 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_split": 3, 2023-12-07T10:36:31.501 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_merge": 2, 2023-12-07T10:36:31.501 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes": 707, 2023-12-07T10:36:31.501 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_top": 54, 2023-12-07T10:36:31.501 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_bottom": 511, 2023-12-07T10:36:31.501 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pin_tail": 142, 2023-12-07T10:36:31.501 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pinned": 602, 2023-12-07T10:36:31.502 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_expired": 2295, 2023-12-07T10:36:31.502 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_with_caps": 173, 2023-12-07T10:36:31.502 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": 173, 2023-12-07T10:36:31.502 INFO:teuthology.orchestra.run.smithi119.stdout: "subtrees": 18, 2023-12-07T10:36:31.502 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse": 216004, 2023-12-07T10:36:31.502 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_hit": 155140, 2023-12-07T10:36:31.502 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_forward": 20519, 2023-12-07T10:36:31.502 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_discover": 32, 2023-12-07T10:36:31.503 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_dir_fetch": 0, 2023-12-07T10:36:31.503 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_remote_ino": 0, 2023-12-07T10:36:31.503 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_lock": 607, 2023-12-07T10:36:31.503 INFO:teuthology.orchestra.run.smithi119.stdout: "load_cent": 62667, 2023-12-07T10:36:31.503 INFO:teuthology.orchestra.run.smithi119.stdout: "q": 2, 2023-12-07T10:36:31.503 INFO:teuthology.orchestra.run.smithi119.stdout: "exported": 30, 2023-12-07T10:36:31.503 INFO:teuthology.orchestra.run.smithi119.stdout: "exported_inodes": 1893, 2023-12-07T10:36:31.503 INFO:teuthology.orchestra.run.smithi119.stdout: "imported": 16, 2023-12-07T10:36:31.503 INFO:teuthology.orchestra.run.smithi119.stdout: "imported_inodes": 647, 2023-12-07T10:36:31.504 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_dir_fetch": 0, 2023-12-07T10:36:31.504 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_backtrace_fetch": 271, 2023-12-07T10:36:31.504 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_peer_discover": 0, 2023-12-07T10:36:31.504 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rfiles": 65, 2023-12-07T10:36:31.504 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rbytes": 21133868, 2023-12-07T10:36:31.504 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rsnaps": 4, 2023-12-07T10:36:31.504 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_fetch": 271, 2023-12-07T10:36:31.504 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_set_tag": 0, 2023-12-07T10:36:31.505 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_repaired": 0, 2023-12-07T10:36:31.505 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_inotable_repaired": 0, 2023-12-07T10:36:31.505 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_inodes": 84, 2023-12-07T10:36:31.505 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_base_inodes": 0, 2023-12-07T10:36:31.505 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dirfrag_rstats": 84, 2023-12-07T10:36:31.505 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_file_inodes": 187, 2023-12-07T10:36:31.505 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_inode_file_caps": 18, 2023-12-07T10:36:31.505 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_revoke": 39805, 2023-12-07T10:36:31.505 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_grant": 45955, 2023-12-07T10:36:31.506 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_trunc": 0, 2023-12-07T10:36:31.506 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-07T10:36:31.506 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-07T10:36:31.506 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps": 59323, 2023-12-07T10:36:31.506 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps_dirty": 11811, 2023-12-07T10:36:31.506 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_cap_release": 5396, 2023-12-07T10:36:31.506 INFO:teuthology.orchestra.run.smithi119.stdout: "process_request_cap_release": 61785 2023-12-07T10:36:31.506 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.507 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cache": { 2023-12-07T10:36:31.507 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update": 15, 2023-12-07T10:36:31.507 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update_receipt": 49, 2023-12-07T10:36:31.507 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_try_discover": 30, 2023-12-07T10:36:31.507 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_send_discover": 61, 2023-12-07T10:36:31.507 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_handle_discover": 306, 2023-12-07T10:36:31.507 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays": 0, 2023-12-07T10:36:31.507 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_delayed": 0, 2023-12-07T10:36:31.507 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_enqueuing": 0, 2023-12-07T10:36:31.508 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_created": 11864, 2023-12-07T10:36:31.508 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_enqueued": 11864, 2023-12-07T10:36:31.508 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_reintegrated": 0, 2023-12-07T10:36:31.508 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_migrated": 0, 2023-12-07T10:36:31.508 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_processing": 0, 2023-12-07T10:36:31.508 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_enqueued": 0, 2023-12-07T10:36:31.508 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_prioritized": 0, 2023-12-07T10:36:31.508 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_started": 0, 2023-12-07T10:36:31.509 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_completed": 0, 2023-12-07T10:36:31.509 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_enqueue_scrub": 0, 2023-12-07T10:36:31.509 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_exportdir": 33, 2023-12-07T10:36:31.509 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_flush": 0, 2023-12-07T10:36:31.509 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragmentdir": 5, 2023-12-07T10:36:31.509 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragstats": 0, 2023-12-07T10:36:31.509 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_inodestats": 0 2023-12-07T10:36:31.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.509 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_log": { 2023-12-07T10:36:31.510 INFO:teuthology.orchestra.run.smithi119.stdout: "evlrg": 0, 2023-12-07T10:36:31.510 INFO:teuthology.orchestra.run.smithi119.stdout: "evadd": 98298, 2023-12-07T10:36:31.510 INFO:teuthology.orchestra.run.smithi119.stdout: "evex": 36255, 2023-12-07T10:36:31.510 INFO:teuthology.orchestra.run.smithi119.stdout: "evtrm": 24581, 2023-12-07T10:36:31.510 INFO:teuthology.orchestra.run.smithi119.stdout: "ev": 73717, 2023-12-07T10:36:31.510 INFO:teuthology.orchestra.run.smithi119.stdout: "evexg": 0, 2023-12-07T10:36:31.510 INFO:teuthology.orchestra.run.smithi119.stdout: "evexd": 11674, 2023-12-07T10:36:31.510 INFO:teuthology.orchestra.run.smithi119.stdout: "segadd": 184, 2023-12-07T10:36:31.511 INFO:teuthology.orchestra.run.smithi119.stdout: "segex": 56, 2023-12-07T10:36:31.511 INFO:teuthology.orchestra.run.smithi119.stdout: "segtrm": 37, 2023-12-07T10:36:31.511 INFO:teuthology.orchestra.run.smithi119.stdout: "seg": 147, 2023-12-07T10:36:31.511 INFO:teuthology.orchestra.run.smithi119.stdout: "segmjr": 6, 2023-12-07T10:36:31.511 INFO:teuthology.orchestra.run.smithi119.stdout: "segexg": 0, 2023-12-07T10:36:31.511 INFO:teuthology.orchestra.run.smithi119.stdout: "segexd": 19, 2023-12-07T10:36:31.511 INFO:teuthology.orchestra.run.smithi119.stdout: "expos": 88520617, 2023-12-07T10:36:31.511 INFO:teuthology.orchestra.run.smithi119.stdout: "wrpos": 669942112, 2023-12-07T10:36:31.511 INFO:teuthology.orchestra.run.smithi119.stdout: "rdpos": 0, 2023-12-07T10:36:31.512 INFO:teuthology.orchestra.run.smithi119.stdout: "jlat": { 2023-12-07T10:36:31.512 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 58058, 2023-12-07T10:36:31.512 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 215.411737205, 2023-12-07T10:36:31.512 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.003710285 2023-12-07T10:36:31.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.512 INFO:teuthology.orchestra.run.smithi119.stdout: "replayed": 0 2023-12-07T10:36:31.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.512 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_mem": { 2023-12-07T10:36:31.512 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 480, 2023-12-07T10:36:31.513 INFO:teuthology.orchestra.run.smithi119.stdout: "ino+": 13796, 2023-12-07T10:36:31.513 INFO:teuthology.orchestra.run.smithi119.stdout: "ino-": 13316, 2023-12-07T10:36:31.513 INFO:teuthology.orchestra.run.smithi119.stdout: "dir": 86, 2023-12-07T10:36:31.513 INFO:teuthology.orchestra.run.smithi119.stdout: "dir+": 205, 2023-12-07T10:36:31.513 INFO:teuthology.orchestra.run.smithi119.stdout: "dir-": 119, 2023-12-07T10:36:31.513 INFO:teuthology.orchestra.run.smithi119.stdout: "dn": 707, 2023-12-07T10:36:31.513 INFO:teuthology.orchestra.run.smithi119.stdout: "dn+": 26939, 2023-12-07T10:36:31.513 INFO:teuthology.orchestra.run.smithi119.stdout: "dn-": 26232, 2023-12-07T10:36:31.513 INFO:teuthology.orchestra.run.smithi119.stdout: "cap": 173, 2023-12-07T10:36:31.514 INFO:teuthology.orchestra.run.smithi119.stdout: "cap+": 13186, 2023-12-07T10:36:31.514 INFO:teuthology.orchestra.run.smithi119.stdout: "cap-": 13013, 2023-12-07T10:36:31.514 INFO:teuthology.orchestra.run.smithi119.stdout: "rss": 72216, 2023-12-07T10:36:31.514 INFO:teuthology.orchestra.run.smithi119.stdout: "heap": 199036 2023-12-07T10:36:31.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.514 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_server": { 2023-12-07T10:36:31.514 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_client_request": 207853, 2023-12-07T10:36:31.514 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_server_request": 38508, 2023-12-07T10:36:31.514 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_request": 145173, 2023-12-07T10:36:31.515 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_session": 1689, 2023-12-07T10:36:31.515 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_peer_request": 77625, 2023-12-07T10:36:31.515 INFO:teuthology.orchestra.run.smithi119.stdout: "req_create_latency": { 2023-12-07T10:36:31.515 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 12550, 2023-12-07T10:36:31.515 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 53.425156813, 2023-12-07T10:36:31.515 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.004256984 2023-12-07T10:36:31.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.515 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getattr_latency": { 2023-12-07T10:36:31.515 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 41382, 2023-12-07T10:36:31.516 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 252.616714949, 2023-12-07T10:36:31.516 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.006104507 2023-12-07T10:36:31.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.516 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getfilelock_latency": { 2023-12-07T10:36:31.516 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:31.516 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:31.516 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:31.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.516 INFO:teuthology.orchestra.run.smithi119.stdout: "req_link_latency": { 2023-12-07T10:36:31.517 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:31.517 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:31.517 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:31.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.517 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookup_latency": { 2023-12-07T10:36:31.517 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 19086, 2023-12-07T10:36:31.517 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 83.003965180, 2023-12-07T10:36:31.517 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.004348945 2023-12-07T10:36:31.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.518 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookuphash_latency": { 2023-12-07T10:36:31.518 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:31.518 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:31.518 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:31.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.518 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupino_latency": { 2023-12-07T10:36:31.519 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:31.519 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:31.519 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:31.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.519 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupname_latency": { 2023-12-07T10:36:31.519 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:31.519 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:31.520 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:31.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.520 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupparent_latency": { 2023-12-07T10:36:31.520 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:31.520 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:31.520 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:31.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.520 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupsnap_latency": { 2023-12-07T10:36:31.521 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:31.521 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:31.521 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:31.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.521 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lssnap_latency": { 2023-12-07T10:36:31.521 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:31.521 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:31.521 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:31.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.522 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mkdir_latency": { 2023-12-07T10:36:31.522 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 113, 2023-12-07T10:36:31.522 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.805408594, 2023-12-07T10:36:31.522 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.007127509 2023-12-07T10:36:31.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.523 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mknod_latency": { 2023-12-07T10:36:31.523 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:31.523 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:31.523 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:31.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.523 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mksnap_latency": { 2023-12-07T10:36:31.523 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:31.523 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:31.524 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:31.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.524 INFO:teuthology.orchestra.run.smithi119.stdout: "req_open_latency": { 2023-12-07T10:36:31.524 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 9, 2023-12-07T10:36:31.524 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.013757037, 2023-12-07T10:36:31.524 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.001528559 2023-12-07T10:36:31.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.524 INFO:teuthology.orchestra.run.smithi119.stdout: "req_readdir_latency": { 2023-12-07T10:36:31.525 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 37219, 2023-12-07T10:36:31.525 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 158.117410216, 2023-12-07T10:36:31.525 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.004248298 2023-12-07T10:36:31.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.525 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rename_latency": { 2023-12-07T10:36:31.525 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2130, 2023-12-07T10:36:31.525 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 8.209796973, 2023-12-07T10:36:31.526 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.003854364 2023-12-07T10:36:31.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.526 INFO:teuthology.orchestra.run.smithi119.stdout: "req_renamesnap_latency": { 2023-12-07T10:36:31.526 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:31.526 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:31.526 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:31.526 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.526 INFO:teuthology.orchestra.run.smithi119.stdout: "req_snapdiff_latency": { 2023-12-07T10:36:31.527 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:31.527 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:31.527 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:31.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.527 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmdir_latency": { 2023-12-07T10:36:31.527 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1, 2023-12-07T10:36:31.527 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000806490, 2023-12-07T10:36:31.527 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000806490 2023-12-07T10:36:31.527 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.528 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmsnap_latency": { 2023-12-07T10:36:31.528 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:31.528 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:31.528 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:31.528 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.528 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmxattr_latency": { 2023-12-07T10:36:31.528 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:31.528 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:31.528 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:31.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.529 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setattr_latency": { 2023-12-07T10:36:31.529 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 6, 2023-12-07T10:36:31.529 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.126827103, 2023-12-07T10:36:31.529 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.021137850 2023-12-07T10:36:31.529 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.529 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setdirlayout_latency": { 2023-12-07T10:36:31.529 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:31.530 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:31.530 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:31.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.530 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setfilelock_latency": { 2023-12-07T10:36:31.530 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 280, 2023-12-07T10:36:31.530 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.547633693, 2023-12-07T10:36:31.530 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.001955834 2023-12-07T10:36:31.530 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.530 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setlayout_latency": { 2023-12-07T10:36:31.531 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:31.531 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:31.531 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:31.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.531 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setxattr_latency": { 2023-12-07T10:36:31.531 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:31.531 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:31.531 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:31.531 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.532 INFO:teuthology.orchestra.run.smithi119.stdout: "req_symlink_latency": { 2023-12-07T10:36:31.532 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:31.532 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:31.532 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:31.532 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.532 INFO:teuthology.orchestra.run.smithi119.stdout: "req_unlink_latency": { 2023-12-07T10:36:31.532 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 11864, 2023-12-07T10:36:31.532 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 39.554530207, 2023-12-07T10:36:31.533 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.003333996 2023-12-07T10:36:31.533 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.533 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_revoke_eviction": 0, 2023-12-07T10:36:31.533 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_acquisition_throttle": 0, 2023-12-07T10:36:31.533 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getvxattr_latency": { 2023-12-07T10:36:31.533 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:31.533 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:31.533 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:31.534 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:31.534 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.534 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_sessions": { 2023-12-07T10:36:31.534 INFO:teuthology.orchestra.run.smithi119.stdout: "session_count": 1, 2023-12-07T10:36:31.534 INFO:teuthology.orchestra.run.smithi119.stdout: "session_add": 4, 2023-12-07T10:36:31.534 INFO:teuthology.orchestra.run.smithi119.stdout: "session_remove": 3, 2023-12-07T10:36:31.534 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_open": 1, 2023-12-07T10:36:31.534 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_stale": 0, 2023-12-07T10:36:31.534 INFO:teuthology.orchestra.run.smithi119.stdout: "total_load": 18034, 2023-12-07T10:36:31.535 INFO:teuthology.orchestra.run.smithi119.stdout: "average_load": 18034, 2023-12-07T10:36:31.535 INFO:teuthology.orchestra.run.smithi119.stdout: "avg_session_uptime": 983, 2023-12-07T10:36:31.535 INFO:teuthology.orchestra.run.smithi119.stdout: "mdthresh_evicted": 0 2023-12-07T10:36:31.535 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.535 INFO:teuthology.orchestra.run.smithi119.stdout: "mempool": { 2023-12-07T10:36:31.535 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_bytes": 120, 2023-12-07T10:36:31.535 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_items": 120, 2023-12-07T10:36:31.535 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_bytes": 0, 2023-12-07T10:36:31.535 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_items": 0, 2023-12-07T10:36:31.536 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_bytes": 0, 2023-12-07T10:36:31.536 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_items": 0, 2023-12-07T10:36:31.536 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-07T10:36:31.536 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_items": 0, 2023-12-07T10:36:31.536 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-07T10:36:31.536 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_items": 0, 2023-12-07T10:36:31.536 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_bytes": 0, 2023-12-07T10:36:31.536 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_items": 0, 2023-12-07T10:36:31.536 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-07T10:36:31.537 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_items": 0, 2023-12-07T10:36:31.537 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_bytes": 0, 2023-12-07T10:36:31.537 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_items": 0, 2023-12-07T10:36:31.537 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_bytes": 0, 2023-12-07T10:36:31.537 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_items": 0, 2023-12-07T10:36:31.537 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-07T10:36:31.537 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_items": 0, 2023-12-07T10:36:31.537 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-07T10:36:31.538 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_items": 0, 2023-12-07T10:36:31.538 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_bytes": 0, 2023-12-07T10:36:31.538 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_items": 0, 2023-12-07T10:36:31.538 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_bytes": 0, 2023-12-07T10:36:31.538 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_items": 0, 2023-12-07T10:36:31.538 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-07T10:36:31.538 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_items": 0, 2023-12-07T10:36:31.538 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_bytes": 0, 2023-12-07T10:36:31.539 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_items": 0, 2023-12-07T10:36:31.539 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_bytes": 0, 2023-12-07T10:36:31.539 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_items": 0, 2023-12-07T10:36:31.539 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_bytes": 0, 2023-12-07T10:36:31.539 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_items": 0, 2023-12-07T10:36:31.539 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_bytes": 0, 2023-12-07T10:36:31.539 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_items": 0, 2023-12-07T10:36:31.539 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_bytes": 300745, 2023-12-07T10:36:31.540 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_items": 426, 2023-12-07T10:36:31.540 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_bytes": 264, 2023-12-07T10:36:31.540 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_items": 3, 2023-12-07T10:36:31.540 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_bytes": 0, 2023-12-07T10:36:31.540 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_items": 0, 2023-12-07T10:36:31.540 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_bytes": 0, 2023-12-07T10:36:31.540 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_items": 0, 2023-12-07T10:36:31.540 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_bytes": 0, 2023-12-07T10:36:31.540 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_items": 0, 2023-12-07T10:36:31.541 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_bytes": 9616, 2023-12-07T10:36:31.541 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_items": 185, 2023-12-07T10:36:31.541 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_bytes": 0, 2023-12-07T10:36:31.541 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_items": 0, 2023-12-07T10:36:31.541 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_bytes": 0, 2023-12-07T10:36:31.541 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_items": 0, 2023-12-07T10:36:31.541 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_bytes": 2136943, 2023-12-07T10:36:31.541 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_items": 20401, 2023-12-07T10:36:31.542 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_bytes": 0, 2023-12-07T10:36:31.542 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_items": 0, 2023-12-07T10:36:31.542 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_bytes": 0, 2023-12-07T10:36:31.542 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_items": 0 2023-12-07T10:36:31.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.542 INFO:teuthology.orchestra.run.smithi119.stdout: "objecter": { 2023-12-07T10:36:31.542 INFO:teuthology.orchestra.run.smithi119.stdout: "op_active": 0, 2023-12-07T10:36:31.542 INFO:teuthology.orchestra.run.smithi119.stdout: "op_laggy": 0, 2023-12-07T10:36:31.543 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send": 83651, 2023-12-07T10:36:31.543 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send_bytes": 668305192, 2023-12-07T10:36:31.543 INFO:teuthology.orchestra.run.smithi119.stdout: "op_resend": 27, 2023-12-07T10:36:31.543 INFO:teuthology.orchestra.run.smithi119.stdout: "op_reply": 83651, 2023-12-07T10:36:31.543 INFO:teuthology.orchestra.run.smithi119.stdout: "op_latency": { 2023-12-07T10:36:31.543 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 83651, 2023-12-07T10:36:31.543 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 503.908955408, 2023-12-07T10:36:31.543 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.006023944 2023-12-07T10:36:31.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.544 INFO:teuthology.orchestra.run.smithi119.stdout: "op_inflight": 0, 2023-12-07T10:36:31.544 INFO:teuthology.orchestra.run.smithi119.stdout: "oplen_avg": { 2023-12-07T10:36:31.544 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 83651, 2023-12-07T10:36:31.544 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 84062 2023-12-07T10:36:31.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.544 INFO:teuthology.orchestra.run.smithi119.stdout: "op": 83651, 2023-12-07T10:36:31.544 INFO:teuthology.orchestra.run.smithi119.stdout: "op_r": 703, 2023-12-07T10:36:31.544 INFO:teuthology.orchestra.run.smithi119.stdout: "op_w": 82948, 2023-12-07T10:36:31.544 INFO:teuthology.orchestra.run.smithi119.stdout: "op_rmw": 0, 2023-12-07T10:36:31.545 INFO:teuthology.orchestra.run.smithi119.stdout: "op_pg": 0, 2023-12-07T10:36:31.545 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_stat": 0, 2023-12-07T10:36:31.545 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_create": 38, 2023-12-07T10:36:31.545 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_read": 425, 2023-12-07T10:36:31.545 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_write": 58483, 2023-12-07T10:36:31.545 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writefull": 264, 2023-12-07T10:36:31.545 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writesame": 0, 2023-12-07T10:36:31.545 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_append": 0, 2023-12-07T10:36:31.546 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_zero": 0, 2023-12-07T10:36:31.546 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_truncate": 3, 2023-12-07T10:36:31.546 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_delete": 23934, 2023-12-07T10:36:31.546 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_mapext": 0, 2023-12-07T10:36:31.546 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_sparse_read": 0, 2023-12-07T10:36:31.546 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_clonerange": 0, 2023-12-07T10:36:31.546 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_getxattr": 272, 2023-12-07T10:36:31.546 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_setxattr": 75, 2023-12-07T10:36:31.546 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_cmpxattr": 0, 2023-12-07T10:36:31.547 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_rmxattr": 0, 2023-12-07T10:36:31.547 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_resetxattrs": 0, 2023-12-07T10:36:31.547 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_call": 0, 2023-12-07T10:36:31.547 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_watch": 0, 2023-12-07T10:36:31.547 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_notify": 0, 2023-12-07T10:36:31.547 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_src_cmpxattr": 0, 2023-12-07T10:36:31.547 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls": 0, 2023-12-07T10:36:31.547 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls_filter": 0, 2023-12-07T10:36:31.547 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_other": 0, 2023-12-07T10:36:31.548 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_active": 0, 2023-12-07T10:36:31.548 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_send": 0, 2023-12-07T10:36:31.548 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_resend": 0, 2023-12-07T10:36:31.548 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_ping": 0, 2023-12-07T10:36:31.548 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_active": 0, 2023-12-07T10:36:31.548 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_send": 0, 2023-12-07T10:36:31.548 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_resend": 0, 2023-12-07T10:36:31.548 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_active": 0, 2023-12-07T10:36:31.548 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_send": 0, 2023-12-07T10:36:31.549 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_resend": 0, 2023-12-07T10:36:31.549 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_active": 0, 2023-12-07T10:36:31.549 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_send": 0, 2023-12-07T10:36:31.549 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_resend": 0, 2023-12-07T10:36:31.549 INFO:teuthology.orchestra.run.smithi119.stdout: "command_active": 0, 2023-12-07T10:36:31.549 INFO:teuthology.orchestra.run.smithi119.stdout: "command_send": 0, 2023-12-07T10:36:31.549 INFO:teuthology.orchestra.run.smithi119.stdout: "command_resend": 0, 2023-12-07T10:36:31.549 INFO:teuthology.orchestra.run.smithi119.stdout: "map_epoch": 114, 2023-12-07T10:36:31.550 INFO:teuthology.orchestra.run.smithi119.stdout: "map_full": 0, 2023-12-07T10:36:31.550 INFO:teuthology.orchestra.run.smithi119.stdout: "map_inc": 23, 2023-12-07T10:36:31.550 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_sessions": 12, 2023-12-07T10:36:31.550 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_open": 12, 2023-12-07T10:36:31.550 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_close": 0, 2023-12-07T10:36:31.550 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_laggy": 0, 2023-12-07T10:36:31.550 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_wr": 432, 2023-12-07T10:36:31.550 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_rd": 12, 2023-12-07T10:36:31.551 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_del": 124 2023-12-07T10:36:31.551 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.551 INFO:teuthology.orchestra.run.smithi119.stdout: "oft": { 2023-12-07T10:36:31.551 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_objs": 1, 2023-12-07T10:36:31.551 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_kv_pairs": 0, 2023-12-07T10:36:31.551 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_updates": 0, 2023-12-07T10:36:31.551 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_removes": 0 2023-12-07T10:36:31.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.552 INFO:teuthology.orchestra.run.smithi119.stdout: "osd-slow-ops": { 2023-12-07T10:36:31.552 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_ops_count": 2 2023-12-07T10:36:31.552 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.552 INFO:teuthology.orchestra.run.smithi119.stdout: "purge_queue": { 2023-12-07T10:36:31.552 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops": 0, 2023-12-07T10:36:31.552 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops_high_water": 10, 2023-12-07T10:36:31.553 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing": 0, 2023-12-07T10:36:31.553 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_high_water": 5, 2023-12-07T10:36:31.553 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed": 11864, 2023-12-07T10:36:31.553 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_item_in_journal": 0 2023-12-07T10:36:31.553 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.553 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-07T10:36:31.553 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2023-12-07T10:36:31.553 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2023-12-07T10:36:31.554 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2023-12-07T10:36:31.554 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 654306, 2023-12-07T10:36:31.554 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 118395294, 2023-12-07T10:36:31.554 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:31.554 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 654306, 2023-12-07T10:36:31.554 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2023-12-07T10:36:31.554 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2023-12-07T10:36:31.554 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 654306, 2023-12-07T10:36:31.555 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 118395294, 2023-12-07T10:36:31.555 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2023-12-07T10:36:31.555 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:31.555 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:31.555 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:31.555 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:31.555 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.555 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_bytes": { 2023-12-07T10:36:31.555 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2023-12-07T10:36:31.556 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2023-12-07T10:36:31.556 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2023-12-07T10:36:31.556 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2023-12-07T10:36:31.556 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2023-12-07T10:36:31.556 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:31.556 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:31.556 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 83651, 2023-12-07T10:36:31.556 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 669600404, 2023-12-07T10:36:31.556 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 83651, 2023-12-07T10:36:31.557 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 669600404, 2023-12-07T10:36:31.557 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2023-12-07T10:36:31.557 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:31.557 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:31.557 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:31.557 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:31.557 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.557 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_ops": { 2023-12-07T10:36:31.558 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2023-12-07T10:36:31.558 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 1024, 2023-12-07T10:36:31.558 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2023-12-07T10:36:31.558 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2023-12-07T10:36:31.558 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2023-12-07T10:36:31.558 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:31.558 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:31.558 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 83651, 2023-12-07T10:36:31.559 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 83651, 2023-12-07T10:36:31.559 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 83651, 2023-12-07T10:36:31.559 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 83651, 2023-12-07T10:36:31.559 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2023-12-07T10:36:31.559 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:31.559 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:31.559 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:31.559 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:31.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.560 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle": { 2023-12-07T10:36:31.560 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2023-12-07T10:36:31.560 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2023-12-07T10:36:31.560 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2023-12-07T10:36:31.560 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 11864, 2023-12-07T10:36:31.560 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 1295352, 2023-12-07T10:36:31.560 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:31.561 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 11864, 2023-12-07T10:36:31.561 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2023-12-07T10:36:31.561 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2023-12-07T10:36:31.561 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 425, 2023-12-07T10:36:31.561 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 1295352, 2023-12-07T10:36:31.561 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2023-12-07T10:36:31.561 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:31.561 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:31.561 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:31.562 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:31.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:31.562 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle-0x55fcaf3506e0": { 2023-12-07T10:36:31.562 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2023-12-07T10:36:31.562 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2023-12-07T10:36:31.562 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2023-12-07T10:36:31.562 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 98298, 2023-12-07T10:36:31.562 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 665747808, 2023-12-07T10:36:31.562 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:31.563 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 98298, 2023-12-07T10:36:31.563 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2023-12-07T10:36:31.563 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2023-12-07T10:36:31.563 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 58058, 2023-12-07T10:36:31.563 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 665747808, 2023-12-07T10:36:31.563 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2023-12-07T10:36:31.563 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:31.563 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:31.563 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:31.564 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:31.564 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:31.564 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:36:31.997 INFO:tasks.check_counter:Daemon mds.g mds.exported=30 2023-12-07T10:36:31.997 INFO:tasks.check_counter:Daemon mds.g mds.imported=16 2023-12-07T10:36:31.997 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=4 2023-12-07T10:36:31.997 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2023-12-07T10:36:31.998 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2023-12-07T10:36:31.998 DEBUG:tasks.check_counter:Getting stats from j 2023-12-07T10:36:31.998 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph daemon mds.j perf dump 2023-12-07T10:36:32.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:32 smithi119 ceph-mon[97908]: pgmap v506: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 40 KiB/s wr, 17 op/s 2023-12-07T10:36:32.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:32 smithi159 ceph-mon[134726]: pgmap v506: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 40 KiB/s wr, 17 op/s 2023-12-07T10:36:32.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:32 smithi153 ceph-mon[133625]: pgmap v506: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 40 KiB/s wr, 17 op/s 2023-12-07T10:36:32.775 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.a/config 2023-12-07T10:36:34.460 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-12-07T10:36:34.460 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-0": { 2023-12-07T10:36:34.460 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 0, 2023-12-07T10:36:34.460 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 2, 2023-12-07T10:36:34.461 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 0, 2023-12-07T10:36:34.461 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 0, 2023-12-07T10:36:34.461 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 4, 2023-12-07T10:36:34.461 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 0, 2023-12-07T10:36:34.461 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.003610073, 2023-12-07T10:36:34.461 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.000000000, 2023-12-07T10:36:34.461 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.001077068, 2023-12-07T10:36:34.461 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-07T10:36:34.461 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:34.462 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:34.462 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:34.462 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:34.462 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:34.462 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:34.462 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:34.462 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:34.462 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:34.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:34.463 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-07T10:36:34.463 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 0 2023-12-07T10:36:34.463 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:34.463 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-1": { 2023-12-07T10:36:34.463 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 322, 2023-12-07T10:36:34.463 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 311, 2023-12-07T10:36:34.463 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 1594512, 2023-12-07T10:36:34.463 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 187216, 2023-12-07T10:36:34.464 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 2, 2023-12-07T10:36:34.464 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 1, 2023-12-07T10:36:34.464 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.069020568, 2023-12-07T10:36:34.464 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.026700206, 2023-12-07T10:36:34.464 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.049245195, 2023-12-07T10:36:34.464 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.006531751, 2023-12-07T10:36:34.464 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:34.464 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 310, 2023-12-07T10:36:34.464 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.025667452, 2023-12-07T10:36:34.465 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000082798 2023-12-07T10:36:34.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:34.465 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:34.465 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:34.465 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:34.465 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:34.465 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:34.465 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 1594512, 2023-12-07T10:36:34.465 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 187216 2023-12-07T10:36:34.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:34.466 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-2": { 2023-12-07T10:36:34.466 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 7, 2023-12-07T10:36:34.466 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 240, 2023-12-07T10:36:34.466 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 3424, 2023-12-07T10:36:34.466 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 273312, 2023-12-07T10:36:34.466 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 6, 2023-12-07T10:36:34.466 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 3, 2023-12-07T10:36:34.467 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.025505815, 2023-12-07T10:36:34.467 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.017565213, 2023-12-07T10:36:34.467 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.003545026, 2023-12-07T10:36:34.467 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-07T10:36:34.467 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:34.467 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 238, 2023-12-07T10:36:34.467 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.038238732, 2023-12-07T10:36:34.467 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000160666 2023-12-07T10:36:34.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:34.468 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:34.468 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:34.468 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:34.468 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:34.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:34.468 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 3424, 2023-12-07T10:36:34.468 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 273312 2023-12-07T10:36:34.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:34.469 INFO:teuthology.orchestra.run.smithi119.stdout: "cct": { 2023-12-07T10:36:34.469 INFO:teuthology.orchestra.run.smithi119.stdout: "total_workers": 0, 2023-12-07T10:36:34.469 INFO:teuthology.orchestra.run.smithi119.stdout: "unhealthy_workers": 0 2023-12-07T10:36:34.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:34.469 INFO:teuthology.orchestra.run.smithi119.stdout: "mempool": { 2023-12-07T10:36:34.469 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_bytes": 0, 2023-12-07T10:36:34.469 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_items": 0, 2023-12-07T10:36:34.469 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_bytes": 0, 2023-12-07T10:36:34.469 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_items": 0, 2023-12-07T10:36:34.470 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_bytes": 0, 2023-12-07T10:36:34.470 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_items": 0, 2023-12-07T10:36:34.470 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-07T10:36:34.470 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_items": 0, 2023-12-07T10:36:34.470 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-07T10:36:34.470 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_items": 0, 2023-12-07T10:36:34.470 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_bytes": 0, 2023-12-07T10:36:34.471 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_items": 0, 2023-12-07T10:36:34.471 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-07T10:36:34.471 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_items": 0, 2023-12-07T10:36:34.471 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_bytes": 0, 2023-12-07T10:36:34.471 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_items": 0, 2023-12-07T10:36:34.471 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_bytes": 0, 2023-12-07T10:36:34.472 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_items": 0, 2023-12-07T10:36:34.472 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-07T10:36:34.472 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_items": 0, 2023-12-07T10:36:34.472 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-07T10:36:34.472 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_items": 0, 2023-12-07T10:36:34.472 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_bytes": 0, 2023-12-07T10:36:34.472 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_items": 0, 2023-12-07T10:36:34.473 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_bytes": 0, 2023-12-07T10:36:34.473 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_items": 0, 2023-12-07T10:36:34.473 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-07T10:36:34.473 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_items": 0, 2023-12-07T10:36:34.473 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_bytes": 0, 2023-12-07T10:36:34.473 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_items": 0, 2023-12-07T10:36:34.473 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_bytes": 0, 2023-12-07T10:36:34.473 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_items": 0, 2023-12-07T10:36:34.473 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_bytes": 0, 2023-12-07T10:36:34.474 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_items": 0, 2023-12-07T10:36:34.474 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_bytes": 0, 2023-12-07T10:36:34.474 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_items": 0, 2023-12-07T10:36:34.474 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_bytes": 147930, 2023-12-07T10:36:34.474 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_items": 28, 2023-12-07T10:36:34.474 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_bytes": 0, 2023-12-07T10:36:34.475 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_items": 0, 2023-12-07T10:36:34.475 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_bytes": 0, 2023-12-07T10:36:34.475 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_items": 0, 2023-12-07T10:36:34.475 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_bytes": 0, 2023-12-07T10:36:34.475 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_items": 0, 2023-12-07T10:36:34.475 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_bytes": 0, 2023-12-07T10:36:34.475 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_items": 0, 2023-12-07T10:36:34.475 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_bytes": 0, 2023-12-07T10:36:34.475 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_items": 0, 2023-12-07T10:36:34.476 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_bytes": 0, 2023-12-07T10:36:34.476 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_items": 0, 2023-12-07T10:36:34.476 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_bytes": 0, 2023-12-07T10:36:34.476 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_items": 0, 2023-12-07T10:36:34.476 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_bytes": 928, 2023-12-07T10:36:34.476 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_items": 2, 2023-12-07T10:36:34.476 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_bytes": 0, 2023-12-07T10:36:34.476 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_items": 0, 2023-12-07T10:36:34.476 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_bytes": 0, 2023-12-07T10:36:34.477 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_items": 0 2023-12-07T10:36:34.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-12-07T10:36:34.477 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-07T10:36:34.477 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2023-12-07T10:36:34.477 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2023-12-07T10:36:34.477 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2023-12-07T10:36:34.477 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 325, 2023-12-07T10:36:34.477 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 1549058, 2023-12-07T10:36:34.477 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:34.478 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 325, 2023-12-07T10:36:34.478 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2023-12-07T10:36:34.478 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2023-12-07T10:36:34.478 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 325, 2023-12-07T10:36:34.478 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 1549058, 2023-12-07T10:36:34.478 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2023-12-07T10:36:34.478 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2023-12-07T10:36:34.478 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2023-12-07T10:36:34.479 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2023-12-07T10:36:34.479 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:34.479 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-12-07T10:36:34.479 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-12-07T10:36:34.479 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:34 smithi119 ceph-mon[97908]: pgmap v507: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 47 KiB/s wr, 18 op/s 2023-12-07T10:36:34.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:34 smithi159 ceph-mon[134726]: pgmap v507: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 47 KiB/s wr, 18 op/s 2023-12-07T10:36:34.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:34 smithi153 ceph-mon[133625]: pgmap v507: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 47 KiB/s wr, 18 op/s 2023-12-07T10:36:34.983 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-12-07T10:36:34.983 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-12-07T10:36:34.983 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-12-07T10:36:34.984 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-12-07T10:36:34.984 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-12-07T10:36:34.984 DEBUG:tasks.check_counter:Getting stats from b 2023-12-07T10:36:34.984 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph daemon mds.b perf dump 2023-12-07T10:36:35.651 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T10:36:36.392 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:36 smithi153 ceph-mon[133625]: pgmap v508: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-12-07T10:36:36.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:36 smithi119 ceph-mon[97908]: pgmap v508: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-12-07T10:36:36.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:36 smithi159 ceph-mon[134726]: pgmap v508: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-12-07T10:36:36.887 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-12-07T10:36:36.888 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-0": { 2023-12-07T10:36:36.888 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 152685, 2023-12-07T10:36:36.888 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 237383, 2023-12-07T10:36:36.888 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 55128855, 2023-12-07T10:36:36.888 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 233558271, 2023-12-07T10:36:36.889 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 20, 2023-12-07T10:36:36.889 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 6, 2023-12-07T10:36:36.889 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 10.723792929, 2023-12-07T10:36:36.889 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 4.689672216, 2023-12-07T10:36:36.889 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 5.894264644, 2023-12-07T10:36:36.889 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.897394167, 2023-12-07T10:36:36.889 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:36.889 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 237376, 2023-12-07T10:36:36.890 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 29.793141955, 2023-12-07T10:36:36.890 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000125510 2023-12-07T10:36:36.890 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.890 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:36.890 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 278036, 2023-12-07T10:36:36.890 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.043114241, 2023-12-07T10:36:36.890 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000155 2023-12-07T10:36:36.890 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.890 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 48131216, 2023-12-07T10:36:36.891 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 159887600 2023-12-07T10:36:36.891 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.891 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-1": { 2023-12-07T10:36:36.891 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 41000, 2023-12-07T10:36:36.891 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 43712, 2023-12-07T10:36:36.891 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 13073091, 2023-12-07T10:36:36.891 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 79494091, 2023-12-07T10:36:36.891 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 24, 2023-12-07T10:36:36.891 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 5, 2023-12-07T10:36:36.891 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 2.831954876, 2023-12-07T10:36:36.892 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.968709885, 2023-12-07T10:36:36.892 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 1.098941117, 2023-12-07T10:36:36.892 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.786267138, 2023-12-07T10:36:36.892 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:36.892 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 43715, 2023-12-07T10:36:36.892 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 12.239526271, 2023-12-07T10:36:36.892 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000279984 2023-12-07T10:36:36.892 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.892 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:36.892 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 38285, 2023-12-07T10:36:36.893 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.006872877, 2023-12-07T10:36:36.893 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000179 2023-12-07T10:36:36.893 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.893 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 543696, 2023-12-07T10:36:36.893 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 300560 2023-12-07T10:36:36.893 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.893 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-2": { 2023-12-07T10:36:36.893 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 133965, 2023-12-07T10:36:36.894 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 116228, 2023-12-07T10:36:36.894 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 46249974, 2023-12-07T10:36:36.894 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 69905938, 2023-12-07T10:36:36.894 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 203, 2023-12-07T10:36:36.894 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 7, 2023-12-07T10:36:36.894 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 7.065868901, 2023-12-07T10:36:36.894 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 2.576839376, 2023-12-07T10:36:36.895 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 4.754282707, 2023-12-07T10:36:36.895 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.596332648, 2023-12-07T10:36:36.895 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:36.895 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 116227, 2023-12-07T10:36:36.895 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 11.785169679, 2023-12-07T10:36:36.897 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000101397 2023-12-07T10:36:36.897 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.897 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:36.897 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 214379, 2023-12-07T10:36:36.898 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.030135188, 2023-12-07T10:36:36.898 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000140 2023-12-07T10:36:36.898 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.898 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 2028800, 2023-12-07T10:36:36.898 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 212112 2023-12-07T10:36:36.898 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.898 INFO:teuthology.orchestra.run.smithi153.stdout: "cct": { 2023-12-07T10:36:36.898 INFO:teuthology.orchestra.run.smithi153.stdout: "total_workers": 1, 2023-12-07T10:36:36.899 INFO:teuthology.orchestra.run.smithi153.stdout: "unhealthy_workers": 0 2023-12-07T10:36:36.899 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.899 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-MDSRank": { 2023-12-07T10:36:36.899 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2023-12-07T10:36:36.899 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2023-12-07T10:36:36.899 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 40737, 2023-12-07T10:36:36.899 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 38.448987493, 2023-12-07T10:36:36.899 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000943834 2023-12-07T10:36:36.899 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:36.900 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.900 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-PurgeQueue": { 2023-12-07T10:36:36.900 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2023-12-07T10:36:36.900 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2023-12-07T10:36:36.900 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 30528, 2023-12-07T10:36:36.900 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 2.634132840, 2023-12-07T10:36:36.900 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000086285 2023-12-07T10:36:36.900 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:36.900 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.901 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2023-12-07T10:36:36.901 INFO:teuthology.orchestra.run.smithi153.stdout: "request": 94043, 2023-12-07T10:36:36.901 INFO:teuthology.orchestra.run.smithi153.stdout: "reply": 89651, 2023-12-07T10:36:36.901 INFO:teuthology.orchestra.run.smithi153.stdout: "reply_latency": { 2023-12-07T10:36:36.901 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 89651, 2023-12-07T10:36:36.901 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 291.159173909, 2023-12-07T10:36:36.901 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.003247695 2023-12-07T10:36:36.901 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.901 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_reply": 0, 2023-12-07T10:36:36.901 INFO:teuthology.orchestra.run.smithi153.stdout: "forward": 4392, 2023-12-07T10:36:36.902 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_complete": 8, 2023-12-07T10:36:36.902 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_keys": 0, 2023-12-07T10:36:36.902 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_commit": 890, 2023-12-07T10:36:36.902 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_split": 0, 2023-12-07T10:36:36.902 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_merge": 0, 2023-12-07T10:36:36.902 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes": 1134, 2023-12-07T10:36:36.902 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_top": 557, 2023-12-07T10:36:36.902 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_bottom": 455, 2023-12-07T10:36:36.902 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pin_tail": 122, 2023-12-07T10:36:36.903 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pinned": 337, 2023-12-07T10:36:36.903 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_expired": 545, 2023-12-07T10:36:36.903 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_with_caps": 89, 2023-12-07T10:36:36.903 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": 89, 2023-12-07T10:36:36.903 INFO:teuthology.orchestra.run.smithi153.stdout: "subtrees": 22, 2023-12-07T10:36:36.903 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse": 162487, 2023-12-07T10:36:36.903 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_hit": 78855, 2023-12-07T10:36:36.903 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_forward": 4392, 2023-12-07T10:36:36.903 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_discover": 28, 2023-12-07T10:36:36.904 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_dir_fetch": 0, 2023-12-07T10:36:36.904 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_remote_ino": 0, 2023-12-07T10:36:36.904 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_lock": 566, 2023-12-07T10:36:36.904 INFO:teuthology.orchestra.run.smithi153.stdout: "load_cent": 10376, 2023-12-07T10:36:36.904 INFO:teuthology.orchestra.run.smithi153.stdout: "q": 0, 2023-12-07T10:36:36.904 INFO:teuthology.orchestra.run.smithi153.stdout: "exported": 15, 2023-12-07T10:36:36.904 INFO:teuthology.orchestra.run.smithi153.stdout: "exported_inodes": 369, 2023-12-07T10:36:36.904 INFO:teuthology.orchestra.run.smithi153.stdout: "imported": 12, 2023-12-07T10:36:36.904 INFO:teuthology.orchestra.run.smithi153.stdout: "imported_inodes": 1036, 2023-12-07T10:36:36.904 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_dir_fetch": 0, 2023-12-07T10:36:36.905 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_backtrace_fetch": 102, 2023-12-07T10:36:36.905 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_peer_discover": 0, 2023-12-07T10:36:36.905 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rfiles": 702, 2023-12-07T10:36:36.905 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rbytes": 223115917, 2023-12-07T10:36:36.905 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rsnaps": 6, 2023-12-07T10:36:36.905 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_fetch": 102, 2023-12-07T10:36:36.905 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_set_tag": 0, 2023-12-07T10:36:36.905 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_repaired": 0, 2023-12-07T10:36:36.905 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_inotable_repaired": 0, 2023-12-07T10:36:36.906 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_inodes": 32, 2023-12-07T10:36:36.906 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_base_inodes": 8, 2023-12-07T10:36:36.906 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dirfrag_rstats": 24, 2023-12-07T10:36:36.906 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_file_inodes": 70, 2023-12-07T10:36:36.906 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_inode_file_caps": 12, 2023-12-07T10:36:36.906 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_revoke": 10144, 2023-12-07T10:36:36.906 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_grant": 59654, 2023-12-07T10:36:36.906 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_trunc": 0, 2023-12-07T10:36:36.907 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-07T10:36:36.907 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-07T10:36:36.907 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps": 20171, 2023-12-07T10:36:36.907 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps_dirty": 5651, 2023-12-07T10:36:36.907 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_cap_release": 2416, 2023-12-07T10:36:36.907 INFO:teuthology.orchestra.run.smithi153.stdout: "process_request_cap_release": 76021 2023-12-07T10:36:36.907 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.907 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_cache": { 2023-12-07T10:36:36.908 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update": 135, 2023-12-07T10:36:36.908 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update_receipt": 85, 2023-12-07T10:36:36.908 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_try_discover": 54, 2023-12-07T10:36:36.908 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_send_discover": 90, 2023-12-07T10:36:36.908 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_handle_discover": 224, 2023-12-07T10:36:36.908 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays": 0, 2023-12-07T10:36:36.908 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_delayed": 0, 2023-12-07T10:36:36.908 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_enqueuing": 0, 2023-12-07T10:36:36.908 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_created": 15842, 2023-12-07T10:36:36.909 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_enqueued": 15842, 2023-12-07T10:36:36.909 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_reintegrated": 0, 2023-12-07T10:36:36.909 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_migrated": 0, 2023-12-07T10:36:36.909 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_processing": 0, 2023-12-07T10:36:36.909 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_enqueued": 0, 2023-12-07T10:36:36.909 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_prioritized": 0, 2023-12-07T10:36:36.909 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_started": 0, 2023-12-07T10:36:36.909 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_completed": 0, 2023-12-07T10:36:36.909 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_enqueue_scrub": 8, 2023-12-07T10:36:36.909 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_exportdir": 18, 2023-12-07T10:36:36.910 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_flush": 0, 2023-12-07T10:36:36.910 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragmentdir": 0, 2023-12-07T10:36:36.910 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragstats": 0, 2023-12-07T10:36:36.910 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_inodestats": 0 2023-12-07T10:36:36.910 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.910 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_log": { 2023-12-07T10:36:36.910 INFO:teuthology.orchestra.run.smithi153.stdout: "evlrg": 0, 2023-12-07T10:36:36.910 INFO:teuthology.orchestra.run.smithi153.stdout: "evadd": 73654, 2023-12-07T10:36:36.911 INFO:teuthology.orchestra.run.smithi153.stdout: "evex": 73242, 2023-12-07T10:36:36.911 INFO:teuthology.orchestra.run.smithi153.stdout: "evtrm": 73242, 2023-12-07T10:36:36.911 INFO:teuthology.orchestra.run.smithi153.stdout: "ev": 412, 2023-12-07T10:36:36.911 INFO:teuthology.orchestra.run.smithi153.stdout: "evexg": 0, 2023-12-07T10:36:36.911 INFO:teuthology.orchestra.run.smithi153.stdout: "evexd": 1063, 2023-12-07T10:36:36.911 INFO:teuthology.orchestra.run.smithi153.stdout: "segadd": 128, 2023-12-07T10:36:36.911 INFO:teuthology.orchestra.run.smithi153.stdout: "segex": 127, 2023-12-07T10:36:36.911 INFO:teuthology.orchestra.run.smithi153.stdout: "segtrm": 127, 2023-12-07T10:36:36.911 INFO:teuthology.orchestra.run.smithi153.stdout: "seg": 1, 2023-12-07T10:36:36.912 INFO:teuthology.orchestra.run.smithi153.stdout: "segmjr": 1, 2023-12-07T10:36:36.912 INFO:teuthology.orchestra.run.smithi153.stdout: "segexg": 0, 2023-12-07T10:36:36.912 INFO:teuthology.orchestra.run.smithi153.stdout: "segexd": 2, 2023-12-07T10:36:36.912 INFO:teuthology.orchestra.run.smithi153.stdout: "expos": 159945888, 2023-12-07T10:36:36.912 INFO:teuthology.orchestra.run.smithi153.stdout: "wrpos": 161369461, 2023-12-07T10:36:36.912 INFO:teuthology.orchestra.run.smithi153.stdout: "rdpos": 0, 2023-12-07T10:36:36.912 INFO:teuthology.orchestra.run.smithi153.stdout: "jlat": { 2023-12-07T10:36:36.912 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 20856, 2023-12-07T10:36:36.913 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 140.071406604, 2023-12-07T10:36:36.913 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.006716120 2023-12-07T10:36:36.913 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.913 INFO:teuthology.orchestra.run.smithi153.stdout: "replayed": 0 2023-12-07T10:36:36.913 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.913 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_mem": { 2023-12-07T10:36:36.913 INFO:teuthology.orchestra.run.smithi153.stdout: "ino": 841, 2023-12-07T10:36:36.913 INFO:teuthology.orchestra.run.smithi153.stdout: "ino+": 17084, 2023-12-07T10:36:36.913 INFO:teuthology.orchestra.run.smithi153.stdout: "ino-": 16243, 2023-12-07T10:36:36.914 INFO:teuthology.orchestra.run.smithi153.stdout: "dir": 84, 2023-12-07T10:36:36.914 INFO:teuthology.orchestra.run.smithi153.stdout: "dir+": 118, 2023-12-07T10:36:36.914 INFO:teuthology.orchestra.run.smithi153.stdout: "dir-": 34, 2023-12-07T10:36:36.914 INFO:teuthology.orchestra.run.smithi153.stdout: "dn": 1110, 2023-12-07T10:36:36.914 INFO:teuthology.orchestra.run.smithi153.stdout: "dn+": 29224, 2023-12-07T10:36:36.914 INFO:teuthology.orchestra.run.smithi153.stdout: "dn-": 28114, 2023-12-07T10:36:36.914 INFO:teuthology.orchestra.run.smithi153.stdout: "cap": 88, 2023-12-07T10:36:36.914 INFO:teuthology.orchestra.run.smithi153.stdout: "cap+": 16498, 2023-12-07T10:36:36.915 INFO:teuthology.orchestra.run.smithi153.stdout: "cap-": 16410, 2023-12-07T10:36:36.915 INFO:teuthology.orchestra.run.smithi153.stdout: "rss": 74280, 2023-12-07T10:36:36.915 INFO:teuthology.orchestra.run.smithi153.stdout: "heap": 207228 2023-12-07T10:36:36.915 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.915 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_server": { 2023-12-07T10:36:36.915 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_client_request": 148994, 2023-12-07T10:36:36.915 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_server_request": 1920, 2023-12-07T10:36:36.915 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_request": 94043, 2023-12-07T10:36:36.915 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_session": 643, 2023-12-07T10:36:36.916 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_peer_request": 14722, 2023-12-07T10:36:36.916 INFO:teuthology.orchestra.run.smithi153.stdout: "req_create_latency": { 2023-12-07T10:36:36.916 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 15831, 2023-12-07T10:36:36.916 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 43.140454352, 2023-12-07T10:36:36.916 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.002725061 2023-12-07T10:36:36.916 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.916 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getattr_latency": { 2023-12-07T10:36:36.916 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 3458, 2023-12-07T10:36:36.917 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 68.536699220, 2023-12-07T10:36:36.917 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.019819751 2023-12-07T10:36:36.917 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.917 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getfilelock_latency": { 2023-12-07T10:36:36.917 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:36.917 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:36.917 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:36.917 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.918 INFO:teuthology.orchestra.run.smithi153.stdout: "req_link_latency": { 2023-12-07T10:36:36.918 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:36.918 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:36.918 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:36.918 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.918 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookup_latency": { 2023-12-07T10:36:36.918 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 20002, 2023-12-07T10:36:36.918 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 33.284209116, 2023-12-07T10:36:36.918 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.001664044 2023-12-07T10:36:36.919 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.919 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookuphash_latency": { 2023-12-07T10:36:36.919 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:36.919 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:36.919 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:36.919 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.919 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupino_latency": { 2023-12-07T10:36:36.919 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1, 2023-12-07T10:36:36.919 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000130322, 2023-12-07T10:36:36.919 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000130322 2023-12-07T10:36:36.920 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.920 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupname_latency": { 2023-12-07T10:36:36.920 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:36.920 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:36.920 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:36.920 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.920 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupparent_latency": { 2023-12-07T10:36:36.920 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:36.921 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:36.921 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:36.921 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.921 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupsnap_latency": { 2023-12-07T10:36:36.921 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:36.921 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:36.921 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:36.922 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.922 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lssnap_latency": { 2023-12-07T10:36:36.922 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 16, 2023-12-07T10:36:36.922 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.003705603, 2023-12-07T10:36:36.922 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000231600 2023-12-07T10:36:36.922 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.922 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mkdir_latency": { 2023-12-07T10:36:36.922 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 6, 2023-12-07T10:36:36.923 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.008246664, 2023-12-07T10:36:36.923 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.001374444 2023-12-07T10:36:36.923 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.923 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mknod_latency": { 2023-12-07T10:36:36.923 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:36.923 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:36.923 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:36.923 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.923 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mksnap_latency": { 2023-12-07T10:36:36.924 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 16, 2023-12-07T10:36:36.924 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 5.107669426, 2023-12-07T10:36:36.924 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.319229339 2023-12-07T10:36:36.924 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.924 INFO:teuthology.orchestra.run.smithi153.stdout: "req_open_latency": { 2023-12-07T10:36:36.924 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:36.924 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:36.925 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:36.925 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.925 INFO:teuthology.orchestra.run.smithi153.stdout: "req_readdir_latency": { 2023-12-07T10:36:36.925 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 33033, 2023-12-07T10:36:36.925 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 91.369625062, 2023-12-07T10:36:36.925 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.002766010 2023-12-07T10:36:36.925 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.925 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rename_latency": { 2023-12-07T10:36:36.925 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1244, 2023-12-07T10:36:36.926 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 4.279768082, 2023-12-07T10:36:36.926 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.003440328 2023-12-07T10:36:36.926 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.926 INFO:teuthology.orchestra.run.smithi153.stdout: "req_renamesnap_latency": { 2023-12-07T10:36:36.926 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:36.926 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:36.926 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:36.926 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.927 INFO:teuthology.orchestra.run.smithi153.stdout: "req_snapdiff_latency": { 2023-12-07T10:36:36.927 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:36.927 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:36.927 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:36.927 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.927 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmdir_latency": { 2023-12-07T10:36:36.927 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:36.927 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:36.927 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:36.927 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.928 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmsnap_latency": { 2023-12-07T10:36:36.928 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 10, 2023-12-07T10:36:36.928 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 10.460698961, 2023-12-07T10:36:36.928 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 1.046069896 2023-12-07T10:36:36.928 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.928 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmxattr_latency": { 2023-12-07T10:36:36.928 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:36.928 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:36.929 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:36.929 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.929 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setattr_latency": { 2023-12-07T10:36:36.929 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 13, 2023-12-07T10:36:36.929 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.289464535, 2023-12-07T10:36:36.929 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.022266502 2023-12-07T10:36:36.929 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.929 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setdirlayout_latency": { 2023-12-07T10:36:36.929 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:36.930 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:36.930 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:36.930 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.930 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setfilelock_latency": { 2023-12-07T10:36:36.930 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 172, 2023-12-07T10:36:36.930 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.266893070, 2023-12-07T10:36:36.930 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.001551703 2023-12-07T10:36:36.930 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.931 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setlayout_latency": { 2023-12-07T10:36:36.931 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:36.931 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:36.931 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:36.931 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.931 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setxattr_latency": { 2023-12-07T10:36:36.931 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 7, 2023-12-07T10:36:36.931 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.019717628, 2023-12-07T10:36:36.931 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.002816804 2023-12-07T10:36:36.932 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.932 INFO:teuthology.orchestra.run.smithi153.stdout: "req_symlink_latency": { 2023-12-07T10:36:36.932 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:36.932 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:36.932 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:36.932 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.932 INFO:teuthology.orchestra.run.smithi153.stdout: "req_unlink_latency": { 2023-12-07T10:36:36.932 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 15842, 2023-12-07T10:36:36.933 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 34.391891868, 2023-12-07T10:36:36.933 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.002170931 2023-12-07T10:36:36.933 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.933 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_revoke_eviction": 0, 2023-12-07T10:36:36.933 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_acquisition_throttle": 0, 2023-12-07T10:36:36.933 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getvxattr_latency": { 2023-12-07T10:36:36.933 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:36.933 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:36.934 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:36.934 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:36.934 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.934 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_sessions": { 2023-12-07T10:36:36.934 INFO:teuthology.orchestra.run.smithi153.stdout: "session_count": 2, 2023-12-07T10:36:36.934 INFO:teuthology.orchestra.run.smithi153.stdout: "session_add": 6, 2023-12-07T10:36:36.934 INFO:teuthology.orchestra.run.smithi153.stdout: "session_remove": 4, 2023-12-07T10:36:36.934 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_open": 2, 2023-12-07T10:36:36.934 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_stale": 0, 2023-12-07T10:36:36.935 INFO:teuthology.orchestra.run.smithi153.stdout: "total_load": 18239, 2023-12-07T10:36:36.935 INFO:teuthology.orchestra.run.smithi153.stdout: "average_load": 9119, 2023-12-07T10:36:36.935 INFO:teuthology.orchestra.run.smithi153.stdout: "avg_session_uptime": 960, 2023-12-07T10:36:36.935 INFO:teuthology.orchestra.run.smithi153.stdout: "mdthresh_evicted": 0 2023-12-07T10:36:36.935 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.935 INFO:teuthology.orchestra.run.smithi153.stdout: "mempool": { 2023-12-07T10:36:36.935 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_bytes": 120, 2023-12-07T10:36:36.935 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_items": 120, 2023-12-07T10:36:36.935 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_bytes": 0, 2023-12-07T10:36:36.936 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_items": 0, 2023-12-07T10:36:36.936 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_bytes": 0, 2023-12-07T10:36:36.936 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_items": 0, 2023-12-07T10:36:36.936 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-07T10:36:36.936 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_items": 0, 2023-12-07T10:36:36.936 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-07T10:36:36.936 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_items": 0, 2023-12-07T10:36:36.936 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_bytes": 0, 2023-12-07T10:36:36.936 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_items": 0, 2023-12-07T10:36:36.937 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-07T10:36:36.937 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_items": 0, 2023-12-07T10:36:36.937 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_bytes": 0, 2023-12-07T10:36:36.937 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_items": 0, 2023-12-07T10:36:36.937 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_bytes": 0, 2023-12-07T10:36:36.937 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_items": 0, 2023-12-07T10:36:36.937 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-07T10:36:36.937 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_items": 0, 2023-12-07T10:36:36.937 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-07T10:36:36.938 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_items": 0, 2023-12-07T10:36:36.938 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_bytes": 0, 2023-12-07T10:36:36.938 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_items": 0, 2023-12-07T10:36:36.938 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_bytes": 0, 2023-12-07T10:36:36.938 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_items": 0, 2023-12-07T10:36:36.938 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-07T10:36:36.938 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_items": 0, 2023-12-07T10:36:36.938 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_bytes": 0, 2023-12-07T10:36:36.938 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_items": 0, 2023-12-07T10:36:36.939 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_bytes": 0, 2023-12-07T10:36:36.939 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_items": 0, 2023-12-07T10:36:36.939 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_bytes": 0, 2023-12-07T10:36:36.939 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_items": 0, 2023-12-07T10:36:36.939 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_bytes": 0, 2023-12-07T10:36:36.939 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_items": 0, 2023-12-07T10:36:36.939 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_bytes": 333997, 2023-12-07T10:36:36.939 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_items": 822, 2023-12-07T10:36:36.939 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_bytes": 88, 2023-12-07T10:36:36.940 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_items": 1, 2023-12-07T10:36:36.940 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_bytes": 0, 2023-12-07T10:36:36.940 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_items": 0, 2023-12-07T10:36:36.940 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_bytes": 0, 2023-12-07T10:36:36.940 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_items": 0, 2023-12-07T10:36:36.940 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_bytes": 0, 2023-12-07T10:36:36.940 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_items": 0, 2023-12-07T10:36:36.940 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_bytes": 9616, 2023-12-07T10:36:36.940 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_items": 185, 2023-12-07T10:36:36.941 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_bytes": 0, 2023-12-07T10:36:36.941 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_items": 0, 2023-12-07T10:36:36.941 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_bytes": 0, 2023-12-07T10:36:36.941 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_items": 0, 2023-12-07T10:36:36.941 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_bytes": 3247115, 2023-12-07T10:36:36.941 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_items": 30043, 2023-12-07T10:36:36.941 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_bytes": 0, 2023-12-07T10:36:36.941 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_items": 0, 2023-12-07T10:36:36.941 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_bytes": 0, 2023-12-07T10:36:36.942 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_items": 0 2023-12-07T10:36:36.942 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.942 INFO:teuthology.orchestra.run.smithi153.stdout: "objecter": { 2023-12-07T10:36:36.942 INFO:teuthology.orchestra.run.smithi153.stdout: "op_active": 0, 2023-12-07T10:36:36.942 INFO:teuthology.orchestra.run.smithi153.stdout: "op_laggy": 0, 2023-12-07T10:36:36.942 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send": 65634, 2023-12-07T10:36:36.942 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send_bytes": 167720145, 2023-12-07T10:36:36.942 INFO:teuthology.orchestra.run.smithi153.stdout: "op_resend": 31, 2023-12-07T10:36:36.943 INFO:teuthology.orchestra.run.smithi153.stdout: "op_reply": 65634, 2023-12-07T10:36:36.943 INFO:teuthology.orchestra.run.smithi153.stdout: "op_latency": { 2023-12-07T10:36:36.943 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 65634, 2023-12-07T10:36:36.943 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 2068.758087129, 2023-12-07T10:36:36.943 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.031519610 2023-12-07T10:36:36.943 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.943 INFO:teuthology.orchestra.run.smithi153.stdout: "op_inflight": 0, 2023-12-07T10:36:36.943 INFO:teuthology.orchestra.run.smithi153.stdout: "oplen_avg": { 2023-12-07T10:36:36.943 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 65634, 2023-12-07T10:36:36.944 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 82084 2023-12-07T10:36:36.944 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.944 INFO:teuthology.orchestra.run.smithi153.stdout: "op": 65634, 2023-12-07T10:36:36.944 INFO:teuthology.orchestra.run.smithi153.stdout: "op_r": 770, 2023-12-07T10:36:36.944 INFO:teuthology.orchestra.run.smithi153.stdout: "op_w": 64864, 2023-12-07T10:36:36.944 INFO:teuthology.orchestra.run.smithi153.stdout: "op_rmw": 0, 2023-12-07T10:36:36.944 INFO:teuthology.orchestra.run.smithi153.stdout: "op_pg": 0, 2023-12-07T10:36:36.944 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_stat": 846, 2023-12-07T10:36:36.944 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_create": 9291, 2023-12-07T10:36:36.945 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_read": 620, 2023-12-07T10:36:36.945 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_write": 21468, 2023-12-07T10:36:36.945 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writefull": 628, 2023-12-07T10:36:36.945 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writesame": 0, 2023-12-07T10:36:36.945 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_append": 0, 2023-12-07T10:36:36.945 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_zero": 0, 2023-12-07T10:36:36.945 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_truncate": 0, 2023-12-07T10:36:36.945 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_delete": 32270, 2023-12-07T10:36:36.945 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_mapext": 0, 2023-12-07T10:36:36.946 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_sparse_read": 0, 2023-12-07T10:36:36.946 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_clonerange": 0, 2023-12-07T10:36:36.946 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_getxattr": 116, 2023-12-07T10:36:36.946 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_setxattr": 13992, 2023-12-07T10:36:36.946 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_cmpxattr": 0, 2023-12-07T10:36:36.946 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_rmxattr": 0, 2023-12-07T10:36:36.946 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_resetxattrs": 0, 2023-12-07T10:36:36.946 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_call": 0, 2023-12-07T10:36:36.947 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_watch": 0, 2023-12-07T10:36:36.947 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_notify": 0, 2023-12-07T10:36:36.947 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_src_cmpxattr": 0, 2023-12-07T10:36:36.947 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls": 0, 2023-12-07T10:36:36.947 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls_filter": 0, 2023-12-07T10:36:36.947 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_other": 0, 2023-12-07T10:36:36.947 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_active": 0, 2023-12-07T10:36:36.947 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_send": 0, 2023-12-07T10:36:36.947 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_resend": 0, 2023-12-07T10:36:36.948 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_ping": 0, 2023-12-07T10:36:36.948 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_active": 0, 2023-12-07T10:36:36.948 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_send": 0, 2023-12-07T10:36:36.948 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_resend": 0, 2023-12-07T10:36:36.948 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_active": 0, 2023-12-07T10:36:36.948 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_send": 0, 2023-12-07T10:36:36.948 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_resend": 0, 2023-12-07T10:36:36.948 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_active": 0, 2023-12-07T10:36:36.948 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_send": 0, 2023-12-07T10:36:36.948 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_resend": 0, 2023-12-07T10:36:36.949 INFO:teuthology.orchestra.run.smithi153.stdout: "command_active": 0, 2023-12-07T10:36:36.949 INFO:teuthology.orchestra.run.smithi153.stdout: "command_send": 0, 2023-12-07T10:36:36.949 INFO:teuthology.orchestra.run.smithi153.stdout: "command_resend": 0, 2023-12-07T10:36:36.949 INFO:teuthology.orchestra.run.smithi153.stdout: "map_epoch": 114, 2023-12-07T10:36:36.949 INFO:teuthology.orchestra.run.smithi153.stdout: "map_full": 0, 2023-12-07T10:36:36.949 INFO:teuthology.orchestra.run.smithi153.stdout: "map_inc": 30, 2023-12-07T10:36:36.949 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_sessions": 12, 2023-12-07T10:36:36.949 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_open": 12, 2023-12-07T10:36:36.949 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_close": 0, 2023-12-07T10:36:36.950 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_laggy": 0, 2023-12-07T10:36:36.950 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_wr": 2157, 2023-12-07T10:36:36.950 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_rd": 48, 2023-12-07T10:36:36.950 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_del": 648 2023-12-07T10:36:36.950 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.950 INFO:teuthology.orchestra.run.smithi153.stdout: "oft": { 2023-12-07T10:36:36.950 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_objs": 1, 2023-12-07T10:36:36.950 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_kv_pairs": 0, 2023-12-07T10:36:36.951 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_updates": 0, 2023-12-07T10:36:36.951 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_removes": 0 2023-12-07T10:36:36.951 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.951 INFO:teuthology.orchestra.run.smithi153.stdout: "osd-slow-ops": { 2023-12-07T10:36:36.951 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_ops_count": 0 2023-12-07T10:36:36.951 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.951 INFO:teuthology.orchestra.run.smithi153.stdout: "purge_queue": { 2023-12-07T10:36:36.951 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops": 0, 2023-12-07T10:36:36.951 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops_high_water": 10, 2023-12-07T10:36:36.952 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing": 0, 2023-12-07T10:36:36.952 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_high_water": 5, 2023-12-07T10:36:36.952 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed": 15842, 2023-12-07T10:36:36.952 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_item_in_journal": 0 2023-12-07T10:36:36.952 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.952 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-07T10:36:36.952 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2023-12-07T10:36:36.952 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2023-12-07T10:36:36.952 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2023-12-07T10:36:36.953 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 327646, 2023-12-07T10:36:36.953 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 78966525, 2023-12-07T10:36:36.953 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:36.953 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 327646, 2023-12-07T10:36:36.953 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2023-12-07T10:36:36.953 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2023-12-07T10:36:36.953 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 327646, 2023-12-07T10:36:36.953 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 78966525, 2023-12-07T10:36:36.953 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2023-12-07T10:36:36.953 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:36.954 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:36.954 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:36.954 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:36.954 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.954 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_bytes": { 2023-12-07T10:36:36.954 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2023-12-07T10:36:36.954 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2023-12-07T10:36:36.954 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2023-12-07T10:36:36.954 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2023-12-07T10:36:36.955 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2023-12-07T10:36:36.955 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:36.955 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:36.955 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 65634, 2023-12-07T10:36:36.955 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 169446899, 2023-12-07T10:36:36.955 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 65634, 2023-12-07T10:36:36.955 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 169446899, 2023-12-07T10:36:36.955 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2023-12-07T10:36:36.955 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:36.956 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:36.956 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:36.956 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:36.956 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.956 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_ops": { 2023-12-07T10:36:36.956 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2023-12-07T10:36:36.956 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 1024, 2023-12-07T10:36:36.956 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2023-12-07T10:36:36.956 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2023-12-07T10:36:36.957 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2023-12-07T10:36:36.957 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:36.957 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:36.957 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 65634, 2023-12-07T10:36:36.957 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 65634, 2023-12-07T10:36:36.957 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 65634, 2023-12-07T10:36:36.957 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 65634, 2023-12-07T10:36:36.957 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2023-12-07T10:36:36.957 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:36.958 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:36.958 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:36.958 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:36.958 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.958 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle": { 2023-12-07T10:36:36.958 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2023-12-07T10:36:36.958 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2023-12-07T10:36:36.958 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2023-12-07T10:36:36.958 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 15842, 2023-12-07T10:36:36.959 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 1727818, 2023-12-07T10:36:36.959 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:36.959 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 15842, 2023-12-07T10:36:36.959 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2023-12-07T10:36:36.959 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2023-12-07T10:36:36.959 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 612, 2023-12-07T10:36:36.959 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 1727818, 2023-12-07T10:36:36.959 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2023-12-07T10:36:36.959 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:36.959 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:36.960 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:36.960 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:36.960 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:36.960 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle-0x55f4a599c6e0": { 2023-12-07T10:36:36.960 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2023-12-07T10:36:36.960 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2023-12-07T10:36:36.960 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2023-12-07T10:36:36.960 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 73654, 2023-12-07T10:36:36.960 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 157175157, 2023-12-07T10:36:36.960 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:36.961 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 73654, 2023-12-07T10:36:36.961 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2023-12-07T10:36:36.961 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2023-12-07T10:36:36.961 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 20856, 2023-12-07T10:36:36.961 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 157175157, 2023-12-07T10:36:36.961 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2023-12-07T10:36:36.961 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:36.961 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:36.961 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:36.961 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:36.962 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:36.962 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-12-07T10:36:37.312 INFO:tasks.check_counter:Daemon mds.b mds.exported=15 2023-12-07T10:36:37.312 INFO:tasks.check_counter:Daemon mds.b mds.imported=12 2023-12-07T10:36:37.313 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2023-12-07T10:36:37.313 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=16 2023-12-07T10:36:37.313 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=10 2023-12-07T10:36:37.313 DEBUG:tasks.check_counter:Getting stats from e 2023-12-07T10:36:37.313 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph daemon mds.e perf dump 2023-12-07T10:36:37.905 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T10:36:38.439 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:38 smithi153 ceph-mon[133625]: pgmap v509: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 15 op/s 2023-12-07T10:36:38.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:38 smithi119 ceph-mon[97908]: pgmap v509: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 15 op/s 2023-12-07T10:36:38.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:38 smithi159 ceph-mon[134726]: pgmap v509: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 15 op/s 2023-12-07T10:36:39.120 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-12-07T10:36:39.121 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-0": { 2023-12-07T10:36:39.121 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 21645, 2023-12-07T10:36:39.121 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 20574, 2023-12-07T10:36:39.121 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 9365882, 2023-12-07T10:36:39.121 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 57592475, 2023-12-07T10:36:39.121 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 19, 2023-12-07T10:36:39.121 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 5, 2023-12-07T10:36:39.122 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 1.493239841, 2023-12-07T10:36:39.122 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.506447516, 2023-12-07T10:36:39.122 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.550900602, 2023-12-07T10:36:39.122 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.440847598, 2023-12-07T10:36:39.122 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:39.122 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 20573, 2023-12-07T10:36:39.122 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 1.812530704, 2023-12-07T10:36:39.122 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000088102 2023-12-07T10:36:39.122 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.123 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:39.123 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 13361, 2023-12-07T10:36:39.123 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.002468907, 2023-12-07T10:36:39.123 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000184 2023-12-07T10:36:39.123 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.123 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 1954704, 2023-12-07T10:36:39.123 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 211872 2023-12-07T10:36:39.123 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.123 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-1": { 2023-12-07T10:36:39.124 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 31235, 2023-12-07T10:36:39.124 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 29377, 2023-12-07T10:36:39.124 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 9166400, 2023-12-07T10:36:39.124 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 59149673, 2023-12-07T10:36:39.124 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 17, 2023-12-07T10:36:39.124 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 7, 2023-12-07T10:36:39.124 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 2.137991023, 2023-12-07T10:36:39.124 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.759775602, 2023-12-07T10:36:39.124 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.930266988, 2023-12-07T10:36:39.125 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.530279523, 2023-12-07T10:36:39.125 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:39.125 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 29374, 2023-12-07T10:36:39.125 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 2.817044666, 2023-12-07T10:36:39.125 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000095902 2023-12-07T10:36:39.125 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.125 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:39.125 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 29957, 2023-12-07T10:36:39.125 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.005274936, 2023-12-07T10:36:39.125 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000176 2023-12-07T10:36:39.126 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.126 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 320, 2023-12-07T10:36:39.126 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 560576 2023-12-07T10:36:39.126 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.126 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-2": { 2023-12-07T10:36:39.126 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 314284, 2023-12-07T10:36:39.126 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 344204, 2023-12-07T10:36:39.126 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 104593035, 2023-12-07T10:36:39.127 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 303751090, 2023-12-07T10:36:39.127 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 15, 2023-12-07T10:36:39.127 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 5, 2023-12-07T10:36:39.127 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 16.724064677, 2023-12-07T10:36:39.127 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 6.830855154, 2023-12-07T10:36:39.127 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 11.377322213, 2023-12-07T10:36:39.127 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.781082782, 2023-12-07T10:36:39.127 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:39.127 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 344203, 2023-12-07T10:36:39.128 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 17.388280316, 2023-12-07T10:36:39.128 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000050517 2023-12-07T10:36:39.128 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.128 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:39.128 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 561363, 2023-12-07T10:36:39.128 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.071067643, 2023-12-07T10:36:39.128 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000126 2023-12-07T10:36:39.129 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.129 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 45627936, 2023-12-07T10:36:39.129 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 159050368 2023-12-07T10:36:39.129 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.129 INFO:teuthology.orchestra.run.smithi153.stdout: "cct": { 2023-12-07T10:36:39.130 INFO:teuthology.orchestra.run.smithi153.stdout: "total_workers": 1, 2023-12-07T10:36:39.130 INFO:teuthology.orchestra.run.smithi153.stdout: "unhealthy_workers": 0 2023-12-07T10:36:39.130 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.130 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-MDSRank": { 2023-12-07T10:36:39.130 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2023-12-07T10:36:39.131 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2023-12-07T10:36:39.131 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 52798, 2023-12-07T10:36:39.131 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 33.901771330, 2023-12-07T10:36:39.131 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000642103 2023-12-07T10:36:39.132 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:39.132 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.132 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-PurgeQueue": { 2023-12-07T10:36:39.132 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2023-12-07T10:36:39.132 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2023-12-07T10:36:39.133 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 25406, 2023-12-07T10:36:39.133 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 2.111496085, 2023-12-07T10:36:39.133 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000083110 2023-12-07T10:36:39.133 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:39.133 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.134 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2023-12-07T10:36:39.134 INFO:teuthology.orchestra.run.smithi153.stdout: "request": 77762, 2023-12-07T10:36:39.134 INFO:teuthology.orchestra.run.smithi153.stdout: "reply": 76589, 2023-12-07T10:36:39.134 INFO:teuthology.orchestra.run.smithi153.stdout: "reply_latency": { 2023-12-07T10:36:39.134 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 76589, 2023-12-07T10:36:39.135 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 178.460817870, 2023-12-07T10:36:39.135 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.002330110 2023-12-07T10:36:39.135 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.135 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_reply": 0, 2023-12-07T10:36:39.135 INFO:teuthology.orchestra.run.smithi153.stdout: "forward": 1173, 2023-12-07T10:36:39.136 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_complete": 2, 2023-12-07T10:36:39.136 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_keys": 0, 2023-12-07T10:36:39.136 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_commit": 38, 2023-12-07T10:36:39.136 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_split": 3, 2023-12-07T10:36:39.136 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_merge": 3, 2023-12-07T10:36:39.137 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes": 1208, 2023-12-07T10:36:39.137 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_top": 296, 2023-12-07T10:36:39.137 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_bottom": 738, 2023-12-07T10:36:39.137 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pin_tail": 174, 2023-12-07T10:36:39.137 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pinned": 781, 2023-12-07T10:36:39.137 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_expired": 248, 2023-12-07T10:36:39.137 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_with_caps": 110, 2023-12-07T10:36:39.137 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": 110, 2023-12-07T10:36:39.138 INFO:teuthology.orchestra.run.smithi153.stdout: "subtrees": 19, 2023-12-07T10:36:39.138 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse": 176442, 2023-12-07T10:36:39.138 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_hit": 96431, 2023-12-07T10:36:39.138 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_forward": 1170, 2023-12-07T10:36:39.138 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_discover": 45, 2023-12-07T10:36:39.138 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_dir_fetch": 0, 2023-12-07T10:36:39.138 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_remote_ino": 0, 2023-12-07T10:36:39.138 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_lock": 4367, 2023-12-07T10:36:39.138 INFO:teuthology.orchestra.run.smithi153.stdout: "load_cent": 11506, 2023-12-07T10:36:39.139 INFO:teuthology.orchestra.run.smithi153.stdout: "q": 0, 2023-12-07T10:36:39.139 INFO:teuthology.orchestra.run.smithi153.stdout: "exported": 4, 2023-12-07T10:36:39.139 INFO:teuthology.orchestra.run.smithi153.stdout: "exported_inodes": 160, 2023-12-07T10:36:39.139 INFO:teuthology.orchestra.run.smithi153.stdout: "imported": 14, 2023-12-07T10:36:39.139 INFO:teuthology.orchestra.run.smithi153.stdout: "imported_inodes": 475, 2023-12-07T10:36:39.139 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_dir_fetch": 0, 2023-12-07T10:36:39.139 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_backtrace_fetch": 90, 2023-12-07T10:36:39.140 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_peer_discover": 1, 2023-12-07T10:36:39.140 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rfiles": 65, 2023-12-07T10:36:39.140 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rbytes": 21133868, 2023-12-07T10:36:39.140 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rsnaps": 4, 2023-12-07T10:36:39.140 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_fetch": 90, 2023-12-07T10:36:39.140 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_set_tag": 0, 2023-12-07T10:36:39.140 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_repaired": 0, 2023-12-07T10:36:39.141 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_inotable_repaired": 0, 2023-12-07T10:36:39.141 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_inodes": 0, 2023-12-07T10:36:39.141 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_base_inodes": 0, 2023-12-07T10:36:39.141 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dirfrag_rstats": 0, 2023-12-07T10:36:39.141 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_file_inodes": 90, 2023-12-07T10:36:39.142 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_inode_file_caps": 1, 2023-12-07T10:36:39.142 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_revoke": 4416, 2023-12-07T10:36:39.142 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_grant": 55073, 2023-12-07T10:36:39.142 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_trunc": 0, 2023-12-07T10:36:39.143 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-07T10:36:39.143 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-07T10:36:39.143 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps": 28736, 2023-12-07T10:36:39.143 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps_dirty": 16055, 2023-12-07T10:36:39.143 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_cap_release": 4360, 2023-12-07T10:36:39.144 INFO:teuthology.orchestra.run.smithi153.stdout: "process_request_cap_release": 72553 2023-12-07T10:36:39.144 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.144 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_cache": { 2023-12-07T10:36:39.144 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update": 61, 2023-12-07T10:36:39.144 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update_receipt": 144, 2023-12-07T10:36:39.145 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_try_discover": 86, 2023-12-07T10:36:39.145 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_send_discover": 161, 2023-12-07T10:36:39.145 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_handle_discover": 73, 2023-12-07T10:36:39.145 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays": 0, 2023-12-07T10:36:39.145 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_delayed": 0, 2023-12-07T10:36:39.146 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_enqueuing": 0, 2023-12-07T10:36:39.146 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_created": 13197, 2023-12-07T10:36:39.146 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_enqueued": 13197, 2023-12-07T10:36:39.146 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_reintegrated": 0, 2023-12-07T10:36:39.147 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_migrated": 0, 2023-12-07T10:36:39.147 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_processing": 0, 2023-12-07T10:36:39.147 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_enqueued": 0, 2023-12-07T10:36:39.147 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_prioritized": 0, 2023-12-07T10:36:39.147 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_started": 0, 2023-12-07T10:36:39.148 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_completed": 0, 2023-12-07T10:36:39.148 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_enqueue_scrub": 0, 2023-12-07T10:36:39.148 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_exportdir": 6, 2023-12-07T10:36:39.148 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_flush": 0, 2023-12-07T10:36:39.148 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragmentdir": 6, 2023-12-07T10:36:39.148 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragstats": 0, 2023-12-07T10:36:39.148 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_inodestats": 0 2023-12-07T10:36:39.149 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.149 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_log": { 2023-12-07T10:36:39.149 INFO:teuthology.orchestra.run.smithi153.stdout: "evlrg": 0, 2023-12-07T10:36:39.149 INFO:teuthology.orchestra.run.smithi153.stdout: "evadd": 84679, 2023-12-07T10:36:39.149 INFO:teuthology.orchestra.run.smithi153.stdout: "evex": 0, 2023-12-07T10:36:39.149 INFO:teuthology.orchestra.run.smithi153.stdout: "evtrm": 0, 2023-12-07T10:36:39.149 INFO:teuthology.orchestra.run.smithi153.stdout: "ev": 84679, 2023-12-07T10:36:39.149 INFO:teuthology.orchestra.run.smithi153.stdout: "evexg": 0, 2023-12-07T10:36:39.150 INFO:teuthology.orchestra.run.smithi153.stdout: "evexd": 0, 2023-12-07T10:36:39.150 INFO:teuthology.orchestra.run.smithi153.stdout: "segadd": 114, 2023-12-07T10:36:39.150 INFO:teuthology.orchestra.run.smithi153.stdout: "segex": 0, 2023-12-07T10:36:39.150 INFO:teuthology.orchestra.run.smithi153.stdout: "segtrm": 0, 2023-12-07T10:36:39.150 INFO:teuthology.orchestra.run.smithi153.stdout: "seg": 114, 2023-12-07T10:36:39.150 INFO:teuthology.orchestra.run.smithi153.stdout: "segmjr": 8, 2023-12-07T10:36:39.150 INFO:teuthology.orchestra.run.smithi153.stdout: "segexg": 0, 2023-12-07T10:36:39.150 INFO:teuthology.orchestra.run.smithi153.stdout: "segexd": 0, 2023-12-07T10:36:39.150 INFO:teuthology.orchestra.run.smithi153.stdout: "expos": 4194304, 2023-12-07T10:36:39.151 INFO:teuthology.orchestra.run.smithi153.stdout: "wrpos": 213373269, 2023-12-07T10:36:39.151 INFO:teuthology.orchestra.run.smithi153.stdout: "rdpos": 0, 2023-12-07T10:36:39.151 INFO:teuthology.orchestra.run.smithi153.stdout: "jlat": { 2023-12-07T10:36:39.151 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 26911, 2023-12-07T10:36:39.151 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 125.784908353, 2023-12-07T10:36:39.151 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.004674107 2023-12-07T10:36:39.151 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.152 INFO:teuthology.orchestra.run.smithi153.stdout: "replayed": 0 2023-12-07T10:36:39.152 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.153 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_mem": { 2023-12-07T10:36:39.153 INFO:teuthology.orchestra.run.smithi153.stdout: "ino": 696, 2023-12-07T10:36:39.153 INFO:teuthology.orchestra.run.smithi153.stdout: "ino+": 14089, 2023-12-07T10:36:39.153 INFO:teuthology.orchestra.run.smithi153.stdout: "ino-": 13393, 2023-12-07T10:36:39.153 INFO:teuthology.orchestra.run.smithi153.stdout: "dir": 69, 2023-12-07T10:36:39.154 INFO:teuthology.orchestra.run.smithi153.stdout: "dir+": 163, 2023-12-07T10:36:39.154 INFO:teuthology.orchestra.run.smithi153.stdout: "dir-": 94, 2023-12-07T10:36:39.154 INFO:teuthology.orchestra.run.smithi153.stdout: "dn": 1208, 2023-12-07T10:36:39.154 INFO:teuthology.orchestra.run.smithi153.stdout: "dn+": 19174, 2023-12-07T10:36:39.154 INFO:teuthology.orchestra.run.smithi153.stdout: "dn-": 17966, 2023-12-07T10:36:39.154 INFO:teuthology.orchestra.run.smithi153.stdout: "cap": 110, 2023-12-07T10:36:39.154 INFO:teuthology.orchestra.run.smithi153.stdout: "cap+": 14287, 2023-12-07T10:36:39.155 INFO:teuthology.orchestra.run.smithi153.stdout: "cap-": 14177, 2023-12-07T10:36:39.155 INFO:teuthology.orchestra.run.smithi153.stdout: "rss": 70216, 2023-12-07T10:36:39.155 INFO:teuthology.orchestra.run.smithi153.stdout: "heap": 199036 2023-12-07T10:36:39.155 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.155 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_server": { 2023-12-07T10:36:39.155 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_client_request": 141569, 2023-12-07T10:36:39.155 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_server_request": 8, 2023-12-07T10:36:39.155 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_request": 77762, 2023-12-07T10:36:39.155 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_session": 3369, 2023-12-07T10:36:39.156 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_peer_request": 16373, 2023-12-07T10:36:39.156 INFO:teuthology.orchestra.run.smithi153.stdout: "req_create_latency": { 2023-12-07T10:36:39.156 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 13194, 2023-12-07T10:36:39.156 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 42.960247352, 2023-12-07T10:36:39.156 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.003256044 2023-12-07T10:36:39.156 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.156 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getattr_latency": { 2023-12-07T10:36:39.156 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 87, 2023-12-07T10:36:39.156 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.242567744, 2023-12-07T10:36:39.157 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.002788134 2023-12-07T10:36:39.157 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.157 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getfilelock_latency": { 2023-12-07T10:36:39.157 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:39.157 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:39.157 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:39.157 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.157 INFO:teuthology.orchestra.run.smithi153.stdout: "req_link_latency": { 2023-12-07T10:36:39.157 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:39.158 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:39.158 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:39.158 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.158 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookup_latency": { 2023-12-07T10:36:39.158 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 22051, 2023-12-07T10:36:39.158 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 46.680724812, 2023-12-07T10:36:39.158 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.002116943 2023-12-07T10:36:39.158 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.158 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookuphash_latency": { 2023-12-07T10:36:39.159 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:39.159 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:39.159 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:39.159 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.159 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupino_latency": { 2023-12-07T10:36:39.159 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1, 2023-12-07T10:36:39.159 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000078620, 2023-12-07T10:36:39.159 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000078620 2023-12-07T10:36:39.159 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.160 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupname_latency": { 2023-12-07T10:36:39.160 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:39.160 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:39.160 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:39.160 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.160 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupparent_latency": { 2023-12-07T10:36:39.160 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:39.160 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:39.160 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:39.161 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.161 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupsnap_latency": { 2023-12-07T10:36:39.161 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:39.161 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:39.161 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:39.161 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.161 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lssnap_latency": { 2023-12-07T10:36:39.161 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:39.161 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:39.162 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:39.162 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.162 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mkdir_latency": { 2023-12-07T10:36:39.162 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:39.162 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:39.162 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:39.162 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.162 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mknod_latency": { 2023-12-07T10:36:39.163 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:39.163 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:39.163 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:39.163 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.163 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mksnap_latency": { 2023-12-07T10:36:39.163 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:39.163 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:39.163 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:39.164 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.164 INFO:teuthology.orchestra.run.smithi153.stdout: "req_open_latency": { 2023-12-07T10:36:39.164 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:39.164 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:39.164 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:39.164 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.164 INFO:teuthology.orchestra.run.smithi153.stdout: "req_readdir_latency": { 2023-12-07T10:36:39.164 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 22243, 2023-12-07T10:36:39.164 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 50.296213754, 2023-12-07T10:36:39.165 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.002261215 2023-12-07T10:36:39.165 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.165 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rename_latency": { 2023-12-07T10:36:39.165 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 4688, 2023-12-07T10:36:39.165 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 12.243313287, 2023-12-07T10:36:39.165 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.002611628 2023-12-07T10:36:39.166 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.166 INFO:teuthology.orchestra.run.smithi153.stdout: "req_renamesnap_latency": { 2023-12-07T10:36:39.166 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:39.166 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:39.166 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:39.166 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.166 INFO:teuthology.orchestra.run.smithi153.stdout: "req_snapdiff_latency": { 2023-12-07T10:36:39.167 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:39.167 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:39.167 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:39.167 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.167 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmdir_latency": { 2023-12-07T10:36:39.167 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:39.167 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:39.167 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:39.168 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.168 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmsnap_latency": { 2023-12-07T10:36:39.168 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:39.168 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:39.168 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:39.168 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.168 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmxattr_latency": { 2023-12-07T10:36:39.168 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:39.168 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:39.169 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:39.169 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.169 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setattr_latency": { 2023-12-07T10:36:39.169 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 16, 2023-12-07T10:36:39.169 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.152602455, 2023-12-07T10:36:39.169 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.009537653 2023-12-07T10:36:39.169 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.169 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setdirlayout_latency": { 2023-12-07T10:36:39.170 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:39.170 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:39.170 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:39.170 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.170 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setfilelock_latency": { 2023-12-07T10:36:39.170 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1112, 2023-12-07T10:36:39.170 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.400655493, 2023-12-07T10:36:39.170 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000360301 2023-12-07T10:36:39.171 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.171 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setlayout_latency": { 2023-12-07T10:36:39.171 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:39.171 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:39.171 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:39.171 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.171 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setxattr_latency": { 2023-12-07T10:36:39.171 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:39.171 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:39.172 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:39.172 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.172 INFO:teuthology.orchestra.run.smithi153.stdout: "req_symlink_latency": { 2023-12-07T10:36:39.172 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:39.172 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:39.172 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:39.172 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.172 INFO:teuthology.orchestra.run.smithi153.stdout: "req_unlink_latency": { 2023-12-07T10:36:39.172 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 13197, 2023-12-07T10:36:39.173 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 25.484414353, 2023-12-07T10:36:39.173 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.001931076 2023-12-07T10:36:39.173 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.173 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_revoke_eviction": 0, 2023-12-07T10:36:39.173 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_acquisition_throttle": 0, 2023-12-07T10:36:39.173 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getvxattr_latency": { 2023-12-07T10:36:39.173 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:39.174 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:39.174 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:39.174 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:39.174 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.174 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_sessions": { 2023-12-07T10:36:39.175 INFO:teuthology.orchestra.run.smithi153.stdout: "session_count": 2, 2023-12-07T10:36:39.175 INFO:teuthology.orchestra.run.smithi153.stdout: "session_add": 2, 2023-12-07T10:36:39.175 INFO:teuthology.orchestra.run.smithi153.stdout: "session_remove": 0, 2023-12-07T10:36:39.175 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_open": 2, 2023-12-07T10:36:39.175 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_stale": 0, 2023-12-07T10:36:39.175 INFO:teuthology.orchestra.run.smithi153.stdout: "total_load": 13509, 2023-12-07T10:36:39.175 INFO:teuthology.orchestra.run.smithi153.stdout: "average_load": 6754, 2023-12-07T10:36:39.175 INFO:teuthology.orchestra.run.smithi153.stdout: "avg_session_uptime": 951, 2023-12-07T10:36:39.175 INFO:teuthology.orchestra.run.smithi153.stdout: "mdthresh_evicted": 0 2023-12-07T10:36:39.176 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.176 INFO:teuthology.orchestra.run.smithi153.stdout: "mempool": { 2023-12-07T10:36:39.176 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_bytes": 0, 2023-12-07T10:36:39.176 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_items": 0, 2023-12-07T10:36:39.176 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_bytes": 0, 2023-12-07T10:36:39.176 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_items": 0, 2023-12-07T10:36:39.176 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_bytes": 0, 2023-12-07T10:36:39.176 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_items": 0, 2023-12-07T10:36:39.176 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-07T10:36:39.177 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_items": 0, 2023-12-07T10:36:39.177 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-07T10:36:39.177 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_items": 0, 2023-12-07T10:36:39.177 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_bytes": 0, 2023-12-07T10:36:39.177 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_items": 0, 2023-12-07T10:36:39.177 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-07T10:36:39.177 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_items": 0, 2023-12-07T10:36:39.177 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_bytes": 0, 2023-12-07T10:36:39.177 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_items": 0, 2023-12-07T10:36:39.178 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_bytes": 0, 2023-12-07T10:36:39.178 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_items": 0, 2023-12-07T10:36:39.178 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-07T10:36:39.178 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_items": 0, 2023-12-07T10:36:39.178 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-07T10:36:39.178 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_items": 0, 2023-12-07T10:36:39.178 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_bytes": 0, 2023-12-07T10:36:39.178 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_items": 0, 2023-12-07T10:36:39.179 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_bytes": 0, 2023-12-07T10:36:39.179 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_items": 0, 2023-12-07T10:36:39.179 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-07T10:36:39.179 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_items": 0, 2023-12-07T10:36:39.179 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_bytes": 0, 2023-12-07T10:36:39.179 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_items": 0, 2023-12-07T10:36:39.179 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_bytes": 0, 2023-12-07T10:36:39.179 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_items": 0, 2023-12-07T10:36:39.179 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_bytes": 0, 2023-12-07T10:36:39.180 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_items": 0, 2023-12-07T10:36:39.180 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_bytes": 0, 2023-12-07T10:36:39.180 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_items": 0, 2023-12-07T10:36:39.180 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_bytes": 316791, 2023-12-07T10:36:39.180 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_items": 586, 2023-12-07T10:36:39.180 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_bytes": 440, 2023-12-07T10:36:39.180 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_items": 5, 2023-12-07T10:36:39.180 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_bytes": 0, 2023-12-07T10:36:39.180 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_items": 0, 2023-12-07T10:36:39.181 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_bytes": 0, 2023-12-07T10:36:39.181 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_items": 0, 2023-12-07T10:36:39.181 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_bytes": 0, 2023-12-07T10:36:39.181 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_items": 0, 2023-12-07T10:36:39.181 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_bytes": 9616, 2023-12-07T10:36:39.181 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_items": 185, 2023-12-07T10:36:39.181 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_bytes": 0, 2023-12-07T10:36:39.181 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_items": 0, 2023-12-07T10:36:39.181 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_bytes": 0, 2023-12-07T10:36:39.182 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_items": 0, 2023-12-07T10:36:39.182 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_bytes": 2801332, 2023-12-07T10:36:39.182 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_items": 25425, 2023-12-07T10:36:39.182 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_bytes": 0, 2023-12-07T10:36:39.182 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_items": 0, 2023-12-07T10:36:39.182 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_bytes": 0, 2023-12-07T10:36:39.182 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_items": 0 2023-12-07T10:36:39.182 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.183 INFO:teuthology.orchestra.run.smithi153.stdout: "objecter": { 2023-12-07T10:36:39.183 INFO:teuthology.orchestra.run.smithi153.stdout: "op_active": 0, 2023-12-07T10:36:39.183 INFO:teuthology.orchestra.run.smithi153.stdout: "op_laggy": 0, 2023-12-07T10:36:39.183 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send": 55222, 2023-12-07T10:36:39.183 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send_bytes": 211832221, 2023-12-07T10:36:39.183 INFO:teuthology.orchestra.run.smithi153.stdout: "op_resend": 33, 2023-12-07T10:36:39.183 INFO:teuthology.orchestra.run.smithi153.stdout: "op_reply": 55222, 2023-12-07T10:36:39.183 INFO:teuthology.orchestra.run.smithi153.stdout: "op_latency": { 2023-12-07T10:36:39.183 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 55222, 2023-12-07T10:36:39.183 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 431.216659817, 2023-12-07T10:36:39.184 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.007808783 2023-12-07T10:36:39.184 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.184 INFO:teuthology.orchestra.run.smithi153.stdout: "op_inflight": 0, 2023-12-07T10:36:39.184 INFO:teuthology.orchestra.run.smithi153.stdout: "oplen_avg": { 2023-12-07T10:36:39.184 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 55222, 2023-12-07T10:36:39.184 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 55601 2023-12-07T10:36:39.185 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.185 INFO:teuthology.orchestra.run.smithi153.stdout: "op": 55222, 2023-12-07T10:36:39.185 INFO:teuthology.orchestra.run.smithi153.stdout: "op_r": 688, 2023-12-07T10:36:39.185 INFO:teuthology.orchestra.run.smithi153.stdout: "op_w": 54534, 2023-12-07T10:36:39.185 INFO:teuthology.orchestra.run.smithi153.stdout: "op_rmw": 0, 2023-12-07T10:36:39.185 INFO:teuthology.orchestra.run.smithi153.stdout: "op_pg": 0, 2023-12-07T10:36:39.185 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_stat": 0, 2023-12-07T10:36:39.185 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_create": 10, 2023-12-07T10:36:39.186 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_read": 592, 2023-12-07T10:36:39.186 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_write": 27503, 2023-12-07T10:36:39.186 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writefull": 331, 2023-12-07T10:36:39.186 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writesame": 0, 2023-12-07T10:36:39.186 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_append": 0, 2023-12-07T10:36:39.186 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_zero": 0, 2023-12-07T10:36:39.186 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_truncate": 3, 2023-12-07T10:36:39.186 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_delete": 26479, 2023-12-07T10:36:39.186 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_mapext": 0, 2023-12-07T10:36:39.187 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_sparse_read": 0, 2023-12-07T10:36:39.187 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_clonerange": 0, 2023-12-07T10:36:39.187 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_getxattr": 90, 2023-12-07T10:36:39.187 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_setxattr": 20, 2023-12-07T10:36:39.187 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_cmpxattr": 0, 2023-12-07T10:36:39.187 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_rmxattr": 0, 2023-12-07T10:36:39.187 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_resetxattrs": 0, 2023-12-07T10:36:39.187 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_call": 0, 2023-12-07T10:36:39.187 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_watch": 0, 2023-12-07T10:36:39.188 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_notify": 0, 2023-12-07T10:36:39.188 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_src_cmpxattr": 0, 2023-12-07T10:36:39.188 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls": 0, 2023-12-07T10:36:39.188 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls_filter": 0, 2023-12-07T10:36:39.188 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_other": 0, 2023-12-07T10:36:39.188 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_active": 0, 2023-12-07T10:36:39.188 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_send": 0, 2023-12-07T10:36:39.188 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_resend": 0, 2023-12-07T10:36:39.188 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_ping": 0, 2023-12-07T10:36:39.189 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_active": 0, 2023-12-07T10:36:39.189 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_send": 0, 2023-12-07T10:36:39.189 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_resend": 0, 2023-12-07T10:36:39.189 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_active": 0, 2023-12-07T10:36:39.189 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_send": 0, 2023-12-07T10:36:39.189 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_resend": 0, 2023-12-07T10:36:39.189 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_active": 0, 2023-12-07T10:36:39.189 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_send": 0, 2023-12-07T10:36:39.189 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_resend": 0, 2023-12-07T10:36:39.189 INFO:teuthology.orchestra.run.smithi153.stdout: "command_active": 0, 2023-12-07T10:36:39.189 INFO:teuthology.orchestra.run.smithi153.stdout: "command_send": 0, 2023-12-07T10:36:39.190 INFO:teuthology.orchestra.run.smithi153.stdout: "command_resend": 0, 2023-12-07T10:36:39.190 INFO:teuthology.orchestra.run.smithi153.stdout: "map_epoch": 114, 2023-12-07T10:36:39.190 INFO:teuthology.orchestra.run.smithi153.stdout: "map_full": 0, 2023-12-07T10:36:39.190 INFO:teuthology.orchestra.run.smithi153.stdout: "map_inc": 23, 2023-12-07T10:36:39.190 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_sessions": 12, 2023-12-07T10:36:39.190 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_open": 12, 2023-12-07T10:36:39.190 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_close": 0, 2023-12-07T10:36:39.190 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_laggy": 0, 2023-12-07T10:36:39.191 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_wr": 397, 2023-12-07T10:36:39.191 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_rd": 8, 2023-12-07T10:36:39.191 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_del": 168 2023-12-07T10:36:39.191 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.191 INFO:teuthology.orchestra.run.smithi153.stdout: "oft": { 2023-12-07T10:36:39.191 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_objs": 1, 2023-12-07T10:36:39.191 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_kv_pairs": 0, 2023-12-07T10:36:39.191 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_updates": 0, 2023-12-07T10:36:39.191 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_removes": 0 2023-12-07T10:36:39.192 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.192 INFO:teuthology.orchestra.run.smithi153.stdout: "osd-slow-ops": { 2023-12-07T10:36:39.192 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_ops_count": 0 2023-12-07T10:36:39.192 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.192 INFO:teuthology.orchestra.run.smithi153.stdout: "purge_queue": { 2023-12-07T10:36:39.192 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops": 0, 2023-12-07T10:36:39.192 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops_high_water": 10, 2023-12-07T10:36:39.192 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing": 0, 2023-12-07T10:36:39.192 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_high_water": 5, 2023-12-07T10:36:39.193 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed": 13197, 2023-12-07T10:36:39.193 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_item_in_journal": 0 2023-12-07T10:36:39.193 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.193 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-07T10:36:39.193 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2023-12-07T10:36:39.193 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2023-12-07T10:36:39.193 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2023-12-07T10:36:39.193 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 367160, 2023-12-07T10:36:39.193 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 84583315, 2023-12-07T10:36:39.194 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:39.194 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 367160, 2023-12-07T10:36:39.194 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2023-12-07T10:36:39.194 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2023-12-07T10:36:39.194 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 367160, 2023-12-07T10:36:39.194 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 84583315, 2023-12-07T10:36:39.194 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2023-12-07T10:36:39.194 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:39.195 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:39.195 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:39.195 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:39.195 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.195 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_bytes": { 2023-12-07T10:36:39.195 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2023-12-07T10:36:39.196 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2023-12-07T10:36:39.196 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2023-12-07T10:36:39.196 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2023-12-07T10:36:39.196 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2023-12-07T10:36:39.196 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:39.196 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:39.196 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 55222, 2023-12-07T10:36:39.196 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 213272571, 2023-12-07T10:36:39.197 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 55222, 2023-12-07T10:36:39.197 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 213272571, 2023-12-07T10:36:39.197 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2023-12-07T10:36:39.197 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:39.197 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:39.197 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:39.197 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:39.198 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.198 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_ops": { 2023-12-07T10:36:39.198 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2023-12-07T10:36:39.198 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 1024, 2023-12-07T10:36:39.198 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2023-12-07T10:36:39.198 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2023-12-07T10:36:39.198 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2023-12-07T10:36:39.198 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:39.199 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:39.199 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 55222, 2023-12-07T10:36:39.199 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 55222, 2023-12-07T10:36:39.199 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 55222, 2023-12-07T10:36:39.199 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 55222, 2023-12-07T10:36:39.199 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2023-12-07T10:36:39.199 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:39.200 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:39.200 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:39.200 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:39.200 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.200 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle": { 2023-12-07T10:36:39.200 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2023-12-07T10:36:39.200 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2023-12-07T10:36:39.200 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2023-12-07T10:36:39.200 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 13197, 2023-12-07T10:36:39.201 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 1440505, 2023-12-07T10:36:39.201 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:39.201 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 13197, 2023-12-07T10:36:39.201 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2023-12-07T10:36:39.201 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2023-12-07T10:36:39.201 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 592, 2023-12-07T10:36:39.201 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 1440505, 2023-12-07T10:36:39.202 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2023-12-07T10:36:39.202 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:39.202 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:39.202 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:39.202 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:39.202 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:39.202 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle-0x559aa5de86e0": { 2023-12-07T10:36:39.202 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 2296, 2023-12-07T10:36:39.202 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2023-12-07T10:36:39.203 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2023-12-07T10:36:39.203 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 84679, 2023-12-07T10:36:39.203 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 209178965, 2023-12-07T10:36:39.203 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:39.203 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 84679, 2023-12-07T10:36:39.203 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2023-12-07T10:36:39.203 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2023-12-07T10:36:39.203 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 26911, 2023-12-07T10:36:39.204 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 209176669, 2023-12-07T10:36:39.204 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2023-12-07T10:36:39.204 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:39.204 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:39.204 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:39.204 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:39.205 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:39.205 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-12-07T10:36:39.555 INFO:tasks.check_counter:Daemon mds.e mds.exported=4 2023-12-07T10:36:39.555 INFO:tasks.check_counter:Daemon mds.e mds.imported=14 2023-12-07T10:36:39.556 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=4 2023-12-07T10:36:39.556 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2023-12-07T10:36:39.556 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2023-12-07T10:36:39.556 DEBUG:tasks.check_counter:Getting stats from h 2023-12-07T10:36:39.556 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph daemon mds.h perf dump 2023-12-07T10:36:40.148 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T10:36:40.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:40 smithi119 ceph-mon[97908]: pgmap v510: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 48 KiB/s wr, 15 op/s 2023-12-07T10:36:40.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:40 smithi159 ceph-mon[134726]: pgmap v510: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 48 KiB/s wr, 15 op/s 2023-12-07T10:36:40.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:40 smithi153 ceph-mon[133625]: pgmap v510: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 48 KiB/s wr, 15 op/s 2023-12-07T10:36:42.405 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-12-07T10:36:42.405 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-0": { 2023-12-07T10:36:42.405 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 2638, 2023-12-07T10:36:42.405 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 2640, 2023-12-07T10:36:42.405 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 269205327, 2023-12-07T10:36:42.405 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 820425, 2023-12-07T10:36:42.405 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 8, 2023-12-07T10:36:42.406 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 4, 2023-12-07T10:36:42.406 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.322769856, 2023-12-07T10:36:42.406 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.058544865, 2023-12-07T10:36:42.406 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.177986146, 2023-12-07T10:36:42.406 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.067936226, 2023-12-07T10:36:42.406 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:42.406 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 2638, 2023-12-07T10:36:42.406 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.098800660, 2023-12-07T10:36:42.407 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000037452 2023-12-07T10:36:42.407 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.407 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:42.407 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.407 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.407 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.407 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.407 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-07T10:36:42.407 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 0 2023-12-07T10:36:42.408 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.408 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-1": { 2023-12-07T10:36:42.408 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 1653, 2023-12-07T10:36:42.408 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 1585, 2023-12-07T10:36:42.408 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 244511206, 2023-12-07T10:36:42.408 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 578864, 2023-12-07T10:36:42.408 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 6, 2023-12-07T10:36:42.408 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 5, 2023-12-07T10:36:42.408 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.247422534, 2023-12-07T10:36:42.408 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.037906132, 2023-12-07T10:36:42.409 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.175410547, 2023-12-07T10:36:42.409 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.036449202, 2023-12-07T10:36:42.409 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:42.409 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1585, 2023-12-07T10:36:42.409 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.052271048, 2023-12-07T10:36:42.409 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000032978 2023-12-07T10:36:42.409 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.409 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:42.410 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.410 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.410 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.410 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.410 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 1967152, 2023-12-07T10:36:42.410 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 191296 2023-12-07T10:36:42.410 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.410 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-2": { 2023-12-07T10:36:42.410 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 797, 2023-12-07T10:36:42.411 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 1031, 2023-12-07T10:36:42.411 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 155392888, 2023-12-07T10:36:42.411 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 810637, 2023-12-07T10:36:42.411 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 8, 2023-12-07T10:36:42.411 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 6, 2023-12-07T10:36:42.411 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.130661654, 2023-12-07T10:36:42.411 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.020665015, 2023-12-07T10:36:42.411 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.085521360, 2023-12-07T10:36:42.411 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.017782343, 2023-12-07T10:36:42.412 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:42.412 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1028, 2023-12-07T10:36:42.412 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.049723505, 2023-12-07T10:36:42.412 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000048369 2023-12-07T10:36:42.412 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.412 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:42.412 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.412 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.412 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.413 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.413 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 320, 2023-12-07T10:36:42.413 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 563392 2023-12-07T10:36:42.413 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.413 INFO:teuthology.orchestra.run.smithi153.stdout: "cct": { 2023-12-07T10:36:42.413 INFO:teuthology.orchestra.run.smithi153.stdout: "total_workers": 1, 2023-12-07T10:36:42.413 INFO:teuthology.orchestra.run.smithi153.stdout: "unhealthy_workers": 0 2023-12-07T10:36:42.413 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.413 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-MDSRank": { 2023-12-07T10:36:42.414 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2023-12-07T10:36:42.414 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2023-12-07T10:36:42.414 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 7659, 2023-12-07T10:36:42.414 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.172040819, 2023-12-07T10:36:42.414 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000022462 2023-12-07T10:36:42.414 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:42.414 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.414 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-PurgeQueue": { 2023-12-07T10:36:42.414 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2023-12-07T10:36:42.415 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2023-12-07T10:36:42.415 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.415 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.415 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.415 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:42.415 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.415 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2023-12-07T10:36:42.415 INFO:teuthology.orchestra.run.smithi153.stdout: "request": 0, 2023-12-07T10:36:42.415 INFO:teuthology.orchestra.run.smithi153.stdout: "reply": 0, 2023-12-07T10:36:42.415 INFO:teuthology.orchestra.run.smithi153.stdout: "reply_latency": { 2023-12-07T10:36:42.416 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.416 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.416 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.416 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.416 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_reply": 0, 2023-12-07T10:36:42.416 INFO:teuthology.orchestra.run.smithi153.stdout: "forward": 0, 2023-12-07T10:36:42.416 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_complete": 0, 2023-12-07T10:36:42.416 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_keys": 0, 2023-12-07T10:36:42.416 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_commit": 0, 2023-12-07T10:36:42.417 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_split": 0, 2023-12-07T10:36:42.417 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_merge": 0, 2023-12-07T10:36:42.417 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes": 12552, 2023-12-07T10:36:42.417 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_top": 2222, 2023-12-07T10:36:42.417 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_bottom": 10330, 2023-12-07T10:36:42.417 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pin_tail": 0, 2023-12-07T10:36:42.417 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pinned": 9377, 2023-12-07T10:36:42.417 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_expired": 0, 2023-12-07T10:36:42.417 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_with_caps": 0, 2023-12-07T10:36:42.418 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": 0, 2023-12-07T10:36:42.418 INFO:teuthology.orchestra.run.smithi153.stdout: "subtrees": 19, 2023-12-07T10:36:42.418 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse": 0, 2023-12-07T10:36:42.418 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_hit": 0, 2023-12-07T10:36:42.418 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_forward": 0, 2023-12-07T10:36:42.418 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_discover": 0, 2023-12-07T10:36:42.418 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_dir_fetch": 0, 2023-12-07T10:36:42.418 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_remote_ino": 0, 2023-12-07T10:36:42.418 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_lock": 0, 2023-12-07T10:36:42.419 INFO:teuthology.orchestra.run.smithi153.stdout: "load_cent": 0, 2023-12-07T10:36:42.419 INFO:teuthology.orchestra.run.smithi153.stdout: "q": 0, 2023-12-07T10:36:42.419 INFO:teuthology.orchestra.run.smithi153.stdout: "exported": 0, 2023-12-07T10:36:42.419 INFO:teuthology.orchestra.run.smithi153.stdout: "exported_inodes": 0, 2023-12-07T10:36:42.419 INFO:teuthology.orchestra.run.smithi153.stdout: "imported": 0, 2023-12-07T10:36:42.419 INFO:teuthology.orchestra.run.smithi153.stdout: "imported_inodes": 0, 2023-12-07T10:36:42.419 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_dir_fetch": 0, 2023-12-07T10:36:42.419 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_backtrace_fetch": 0, 2023-12-07T10:36:42.419 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_peer_discover": 0, 2023-12-07T10:36:42.420 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rfiles": 0, 2023-12-07T10:36:42.420 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rbytes": 0, 2023-12-07T10:36:42.420 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rsnaps": 0, 2023-12-07T10:36:42.420 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_fetch": 0, 2023-12-07T10:36:42.420 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_set_tag": 0, 2023-12-07T10:36:42.420 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_repaired": 0, 2023-12-07T10:36:42.420 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_inotable_repaired": 0, 2023-12-07T10:36:42.420 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_inodes": 0, 2023-12-07T10:36:42.420 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_base_inodes": 0, 2023-12-07T10:36:42.421 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dirfrag_rstats": 0, 2023-12-07T10:36:42.421 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_file_inodes": 0, 2023-12-07T10:36:42.421 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_inode_file_caps": 0, 2023-12-07T10:36:42.421 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_revoke": 0, 2023-12-07T10:36:42.421 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_grant": 0, 2023-12-07T10:36:42.421 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_trunc": 0, 2023-12-07T10:36:42.421 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-07T10:36:42.421 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-07T10:36:42.421 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps": 0, 2023-12-07T10:36:42.422 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps_dirty": 0, 2023-12-07T10:36:42.422 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_cap_release": 0, 2023-12-07T10:36:42.422 INFO:teuthology.orchestra.run.smithi153.stdout: "process_request_cap_release": 0 2023-12-07T10:36:42.422 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.422 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_cache": { 2023-12-07T10:36:42.422 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update": 0, 2023-12-07T10:36:42.422 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update_receipt": 0, 2023-12-07T10:36:42.422 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_try_discover": 0, 2023-12-07T10:36:42.422 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_send_discover": 0, 2023-12-07T10:36:42.423 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_handle_discover": 0, 2023-12-07T10:36:42.423 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays": 0, 2023-12-07T10:36:42.423 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_delayed": 0, 2023-12-07T10:36:42.423 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_enqueuing": 0, 2023-12-07T10:36:42.423 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_created": 0, 2023-12-07T10:36:42.423 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_enqueued": 0, 2023-12-07T10:36:42.423 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_reintegrated": 0, 2023-12-07T10:36:42.423 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_migrated": 0, 2023-12-07T10:36:42.425 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_processing": 0, 2023-12-07T10:36:42.425 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_enqueued": 0, 2023-12-07T10:36:42.425 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_prioritized": 0, 2023-12-07T10:36:42.425 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_started": 0, 2023-12-07T10:36:42.425 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_completed": 0, 2023-12-07T10:36:42.425 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_enqueue_scrub": 0, 2023-12-07T10:36:42.426 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_exportdir": 0, 2023-12-07T10:36:42.426 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_flush": 0, 2023-12-07T10:36:42.426 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragmentdir": 0, 2023-12-07T10:36:42.426 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragstats": 0, 2023-12-07T10:36:42.426 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_inodestats": 0 2023-12-07T10:36:42.426 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.426 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_log": { 2023-12-07T10:36:42.426 INFO:teuthology.orchestra.run.smithi153.stdout: "evlrg": 0, 2023-12-07T10:36:42.426 INFO:teuthology.orchestra.run.smithi153.stdout: "evadd": 0, 2023-12-07T10:36:42.427 INFO:teuthology.orchestra.run.smithi153.stdout: "evex": 0, 2023-12-07T10:36:42.427 INFO:teuthology.orchestra.run.smithi153.stdout: "evtrm": 0, 2023-12-07T10:36:42.427 INFO:teuthology.orchestra.run.smithi153.stdout: "ev": 98308, 2023-12-07T10:36:42.427 INFO:teuthology.orchestra.run.smithi153.stdout: "evexg": 0, 2023-12-07T10:36:42.427 INFO:teuthology.orchestra.run.smithi153.stdout: "evexd": 0, 2023-12-07T10:36:42.427 INFO:teuthology.orchestra.run.smithi153.stdout: "segadd": 0, 2023-12-07T10:36:42.427 INFO:teuthology.orchestra.run.smithi153.stdout: "segex": 0, 2023-12-07T10:36:42.427 INFO:teuthology.orchestra.run.smithi153.stdout: "segtrm": 0, 2023-12-07T10:36:42.427 INFO:teuthology.orchestra.run.smithi153.stdout: "seg": 147, 2023-12-07T10:36:42.428 INFO:teuthology.orchestra.run.smithi153.stdout: "segmjr": 9, 2023-12-07T10:36:42.428 INFO:teuthology.orchestra.run.smithi153.stdout: "segexg": 0, 2023-12-07T10:36:42.428 INFO:teuthology.orchestra.run.smithi153.stdout: "segexd": 0, 2023-12-07T10:36:42.428 INFO:teuthology.orchestra.run.smithi153.stdout: "expos": 88520617, 2023-12-07T10:36:42.428 INFO:teuthology.orchestra.run.smithi153.stdout: "wrpos": 670040793, 2023-12-07T10:36:42.428 INFO:teuthology.orchestra.run.smithi153.stdout: "rdpos": 670009663, 2023-12-07T10:36:42.428 INFO:teuthology.orchestra.run.smithi153.stdout: "jlat": { 2023-12-07T10:36:42.428 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.429 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.429 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.429 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.429 INFO:teuthology.orchestra.run.smithi153.stdout: "replayed": 98308 2023-12-07T10:36:42.429 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.429 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_mem": { 2023-12-07T10:36:42.429 INFO:teuthology.orchestra.run.smithi153.stdout: "ino": 490, 2023-12-07T10:36:42.429 INFO:teuthology.orchestra.run.smithi153.stdout: "ino+": 13730, 2023-12-07T10:36:42.429 INFO:teuthology.orchestra.run.smithi153.stdout: "ino-": 13240, 2023-12-07T10:36:42.430 INFO:teuthology.orchestra.run.smithi153.stdout: "dir": 59, 2023-12-07T10:36:42.430 INFO:teuthology.orchestra.run.smithi153.stdout: "dir+": 170, 2023-12-07T10:36:42.430 INFO:teuthology.orchestra.run.smithi153.stdout: "dir-": 111, 2023-12-07T10:36:42.430 INFO:teuthology.orchestra.run.smithi153.stdout: "dn": 12552, 2023-12-07T10:36:42.430 INFO:teuthology.orchestra.run.smithi153.stdout: "dn+": 14404, 2023-12-07T10:36:42.430 INFO:teuthology.orchestra.run.smithi153.stdout: "dn-": 1852, 2023-12-07T10:36:42.430 INFO:teuthology.orchestra.run.smithi153.stdout: "cap": 0, 2023-12-07T10:36:42.430 INFO:teuthology.orchestra.run.smithi153.stdout: "cap+": 0, 2023-12-07T10:36:42.430 INFO:teuthology.orchestra.run.smithi153.stdout: "cap-": 0, 2023-12-07T10:36:42.431 INFO:teuthology.orchestra.run.smithi153.stdout: "rss": 65064, 2023-12-07T10:36:42.431 INFO:teuthology.orchestra.run.smithi153.stdout: "heap": 207228 2023-12-07T10:36:42.431 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.431 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_server": { 2023-12-07T10:36:42.431 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_client_request": 0, 2023-12-07T10:36:42.431 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_server_request": 0, 2023-12-07T10:36:42.431 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_request": 0, 2023-12-07T10:36:42.431 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_session": 0, 2023-12-07T10:36:42.431 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_peer_request": 0, 2023-12-07T10:36:42.432 INFO:teuthology.orchestra.run.smithi153.stdout: "req_create_latency": { 2023-12-07T10:36:42.432 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.432 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.432 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.432 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.432 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getattr_latency": { 2023-12-07T10:36:42.432 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.432 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.432 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.433 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.433 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getfilelock_latency": { 2023-12-07T10:36:42.433 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.433 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.433 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.433 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.433 INFO:teuthology.orchestra.run.smithi153.stdout: "req_link_latency": { 2023-12-07T10:36:42.433 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.433 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.434 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.434 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.434 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookup_latency": { 2023-12-07T10:36:42.434 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.434 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.434 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.434 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.434 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookuphash_latency": { 2023-12-07T10:36:42.435 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.435 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.435 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.435 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.435 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupino_latency": { 2023-12-07T10:36:42.435 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.435 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.435 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.435 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.436 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupname_latency": { 2023-12-07T10:36:42.436 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.436 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.436 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.436 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.436 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupparent_latency": { 2023-12-07T10:36:42.436 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.436 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.436 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.437 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.437 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupsnap_latency": { 2023-12-07T10:36:42.437 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.437 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.437 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.437 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.437 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lssnap_latency": { 2023-12-07T10:36:42.437 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.437 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.438 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.438 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.438 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mkdir_latency": { 2023-12-07T10:36:42.438 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.438 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.438 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.438 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.438 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mknod_latency": { 2023-12-07T10:36:42.439 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.439 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.439 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.439 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.439 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mksnap_latency": { 2023-12-07T10:36:42.439 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.439 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.439 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.439 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.440 INFO:teuthology.orchestra.run.smithi153.stdout: "req_open_latency": { 2023-12-07T10:36:42.440 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.440 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.440 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.440 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.440 INFO:teuthology.orchestra.run.smithi153.stdout: "req_readdir_latency": { 2023-12-07T10:36:42.440 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.440 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.440 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.441 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.441 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rename_latency": { 2023-12-07T10:36:42.441 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.441 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.441 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.441 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.441 INFO:teuthology.orchestra.run.smithi153.stdout: "req_renamesnap_latency": { 2023-12-07T10:36:42.441 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.441 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.442 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.442 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.442 INFO:teuthology.orchestra.run.smithi153.stdout: "req_snapdiff_latency": { 2023-12-07T10:36:42.442 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.442 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.442 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.442 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.442 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmdir_latency": { 2023-12-07T10:36:42.442 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.443 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.443 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.443 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.443 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmsnap_latency": { 2023-12-07T10:36:42.443 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.443 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.443 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.443 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.443 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmxattr_latency": { 2023-12-07T10:36:42.444 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.444 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.444 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.444 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.444 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setattr_latency": { 2023-12-07T10:36:42.444 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.444 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.444 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.444 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.445 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setdirlayout_latency": { 2023-12-07T10:36:42.445 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.445 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.445 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.445 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.445 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setfilelock_latency": { 2023-12-07T10:36:42.445 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.445 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.445 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.446 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.446 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setlayout_latency": { 2023-12-07T10:36:42.446 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.446 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.446 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.446 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.446 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setxattr_latency": { 2023-12-07T10:36:42.446 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.446 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.447 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.447 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.447 INFO:teuthology.orchestra.run.smithi153.stdout: "req_symlink_latency": { 2023-12-07T10:36:42.447 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.447 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.447 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.447 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.447 INFO:teuthology.orchestra.run.smithi153.stdout: "req_unlink_latency": { 2023-12-07T10:36:42.447 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.448 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.448 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.448 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.448 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_revoke_eviction": 0, 2023-12-07T10:36:42.448 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_acquisition_throttle": 0, 2023-12-07T10:36:42.448 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getvxattr_latency": { 2023-12-07T10:36:42.448 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.448 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.448 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.449 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:42.449 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.449 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_sessions": { 2023-12-07T10:36:42.449 INFO:teuthology.orchestra.run.smithi153.stdout: "session_count": 1, 2023-12-07T10:36:42.449 INFO:teuthology.orchestra.run.smithi153.stdout: "session_add": 4, 2023-12-07T10:36:42.449 INFO:teuthology.orchestra.run.smithi153.stdout: "session_remove": 3, 2023-12-07T10:36:42.449 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_open": 3, 2023-12-07T10:36:42.449 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_stale": 0, 2023-12-07T10:36:42.449 INFO:teuthology.orchestra.run.smithi153.stdout: "total_load": 0, 2023-12-07T10:36:42.450 INFO:teuthology.orchestra.run.smithi153.stdout: "average_load": 0, 2023-12-07T10:36:42.450 INFO:teuthology.orchestra.run.smithi153.stdout: "avg_session_uptime": 6597, 2023-12-07T10:36:42.450 INFO:teuthology.orchestra.run.smithi153.stdout: "mdthresh_evicted": 0 2023-12-07T10:36:42.450 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.450 INFO:teuthology.orchestra.run.smithi153.stdout: "mempool": { 2023-12-07T10:36:42.450 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_bytes": 0, 2023-12-07T10:36:42.450 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_items": 0, 2023-12-07T10:36:42.450 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_bytes": 0, 2023-12-07T10:36:42.451 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_items": 0, 2023-12-07T10:36:42.451 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_bytes": 0, 2023-12-07T10:36:42.451 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_items": 0, 2023-12-07T10:36:42.451 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-07T10:36:42.451 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_items": 0, 2023-12-07T10:36:42.451 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-07T10:36:42.451 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_items": 0, 2023-12-07T10:36:42.451 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_bytes": 0, 2023-12-07T10:36:42.451 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_items": 0, 2023-12-07T10:36:42.452 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-07T10:36:42.452 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_items": 0, 2023-12-07T10:36:42.452 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_bytes": 0, 2023-12-07T10:36:42.452 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_items": 0, 2023-12-07T10:36:42.452 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_bytes": 0, 2023-12-07T10:36:42.452 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_items": 0, 2023-12-07T10:36:42.452 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-07T10:36:42.452 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_items": 0, 2023-12-07T10:36:42.452 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-07T10:36:42.453 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_items": 0, 2023-12-07T10:36:42.453 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_bytes": 0, 2023-12-07T10:36:42.453 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_items": 0, 2023-12-07T10:36:42.453 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_bytes": 0, 2023-12-07T10:36:42.453 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_items": 0, 2023-12-07T10:36:42.453 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-07T10:36:42.453 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_items": 0, 2023-12-07T10:36:42.453 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_bytes": 0, 2023-12-07T10:36:42.453 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_items": 0, 2023-12-07T10:36:42.454 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_bytes": 0, 2023-12-07T10:36:42.454 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_items": 0, 2023-12-07T10:36:42.454 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_bytes": 0, 2023-12-07T10:36:42.454 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_items": 0, 2023-12-07T10:36:42.454 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_bytes": 0, 2023-12-07T10:36:42.454 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_items": 0, 2023-12-07T10:36:42.454 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_bytes": 261479, 2023-12-07T10:36:42.454 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_items": 727, 2023-12-07T10:36:42.454 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_bytes": 0, 2023-12-07T10:36:42.455 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_items": 0, 2023-12-07T10:36:42.455 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_bytes": 0, 2023-12-07T10:36:42.455 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_items": 0, 2023-12-07T10:36:42.455 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_bytes": 0, 2023-12-07T10:36:42.455 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_items": 0, 2023-12-07T10:36:42.455 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_bytes": 0, 2023-12-07T10:36:42.455 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_items": 0, 2023-12-07T10:36:42.455 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_bytes": 9616, 2023-12-07T10:36:42.455 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_items": 185, 2023-12-07T10:36:42.456 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_bytes": 0, 2023-12-07T10:36:42.456 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_items": 0, 2023-12-07T10:36:42.456 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_bytes": 0, 2023-12-07T10:36:42.456 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_items": 0, 2023-12-07T10:36:42.456 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_bytes": 10327396, 2023-12-07T10:36:42.456 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_items": 65327, 2023-12-07T10:36:42.456 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_bytes": 0, 2023-12-07T10:36:42.456 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_items": 0, 2023-12-07T10:36:42.456 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_bytes": 0, 2023-12-07T10:36:42.457 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_items": 0 2023-12-07T10:36:42.457 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.457 INFO:teuthology.orchestra.run.smithi153.stdout: "objecter": { 2023-12-07T10:36:42.457 INFO:teuthology.orchestra.run.smithi153.stdout: "op_active": 0, 2023-12-07T10:36:42.457 INFO:teuthology.orchestra.run.smithi153.stdout: "op_laggy": 0, 2023-12-07T10:36:42.457 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send": 4679, 2023-12-07T10:36:42.457 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send_bytes": 21, 2023-12-07T10:36:42.457 INFO:teuthology.orchestra.run.smithi153.stdout: "op_resend": 3, 2023-12-07T10:36:42.458 INFO:teuthology.orchestra.run.smithi153.stdout: "op_reply": 4679, 2023-12-07T10:36:42.458 INFO:teuthology.orchestra.run.smithi153.stdout: "op_latency": { 2023-12-07T10:36:42.458 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 4679, 2023-12-07T10:36:42.458 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 7.856864931, 2023-12-07T10:36:42.458 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.001679176 2023-12-07T10:36:42.458 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.458 INFO:teuthology.orchestra.run.smithi153.stdout: "op_inflight": 0, 2023-12-07T10:36:42.458 INFO:teuthology.orchestra.run.smithi153.stdout: "oplen_avg": { 2023-12-07T10:36:42.458 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 4679, 2023-12-07T10:36:42.459 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 4680 2023-12-07T10:36:42.459 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.459 INFO:teuthology.orchestra.run.smithi153.stdout: "op": 4679, 2023-12-07T10:36:42.459 INFO:teuthology.orchestra.run.smithi153.stdout: "op_r": 4679, 2023-12-07T10:36:42.459 INFO:teuthology.orchestra.run.smithi153.stdout: "op_w": 0, 2023-12-07T10:36:42.459 INFO:teuthology.orchestra.run.smithi153.stdout: "op_rmw": 0, 2023-12-07T10:36:42.459 INFO:teuthology.orchestra.run.smithi153.stdout: "op_pg": 0, 2023-12-07T10:36:42.459 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_stat": 2600, 2023-12-07T10:36:42.459 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_create": 0, 2023-12-07T10:36:42.460 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_read": 2077, 2023-12-07T10:36:42.460 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_write": 0, 2023-12-07T10:36:42.460 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writefull": 0, 2023-12-07T10:36:42.460 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writesame": 0, 2023-12-07T10:36:42.460 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_append": 0, 2023-12-07T10:36:42.460 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_zero": 0, 2023-12-07T10:36:42.460 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_truncate": 0, 2023-12-07T10:36:42.460 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_delete": 0, 2023-12-07T10:36:42.460 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_mapext": 0, 2023-12-07T10:36:42.461 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_sparse_read": 0, 2023-12-07T10:36:42.461 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_clonerange": 0, 2023-12-07T10:36:42.461 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_getxattr": 1, 2023-12-07T10:36:42.461 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_setxattr": 0, 2023-12-07T10:36:42.461 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_cmpxattr": 0, 2023-12-07T10:36:42.461 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_rmxattr": 0, 2023-12-07T10:36:42.461 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_resetxattrs": 0, 2023-12-07T10:36:42.461 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_call": 0, 2023-12-07T10:36:42.461 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_watch": 0, 2023-12-07T10:36:42.462 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_notify": 0, 2023-12-07T10:36:42.462 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_src_cmpxattr": 0, 2023-12-07T10:36:42.462 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls": 0, 2023-12-07T10:36:42.462 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls_filter": 0, 2023-12-07T10:36:42.462 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_other": 0, 2023-12-07T10:36:42.462 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_active": 0, 2023-12-07T10:36:42.462 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_send": 0, 2023-12-07T10:36:42.462 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_resend": 0, 2023-12-07T10:36:42.462 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_ping": 0, 2023-12-07T10:36:42.463 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_active": 0, 2023-12-07T10:36:42.463 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_send": 0, 2023-12-07T10:36:42.463 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_resend": 0, 2023-12-07T10:36:42.463 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_active": 0, 2023-12-07T10:36:42.463 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_send": 0, 2023-12-07T10:36:42.463 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_resend": 0, 2023-12-07T10:36:42.463 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_active": 0, 2023-12-07T10:36:42.463 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_send": 0, 2023-12-07T10:36:42.463 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_resend": 0, 2023-12-07T10:36:42.464 INFO:teuthology.orchestra.run.smithi153.stdout: "command_active": 0, 2023-12-07T10:36:42.464 INFO:teuthology.orchestra.run.smithi153.stdout: "command_send": 0, 2023-12-07T10:36:42.464 INFO:teuthology.orchestra.run.smithi153.stdout: "command_resend": 0, 2023-12-07T10:36:42.464 INFO:teuthology.orchestra.run.smithi153.stdout: "map_epoch": 114, 2023-12-07T10:36:42.464 INFO:teuthology.orchestra.run.smithi153.stdout: "map_full": 0, 2023-12-07T10:36:42.464 INFO:teuthology.orchestra.run.smithi153.stdout: "map_inc": 23, 2023-12-07T10:36:42.464 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_sessions": 11, 2023-12-07T10:36:42.464 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_open": 11, 2023-12-07T10:36:42.464 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_close": 0, 2023-12-07T10:36:42.465 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_laggy": 0, 2023-12-07T10:36:42.465 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_wr": 0, 2023-12-07T10:36:42.465 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_rd": 2, 2023-12-07T10:36:42.465 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_del": 0 2023-12-07T10:36:42.465 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.465 INFO:teuthology.orchestra.run.smithi153.stdout: "oft": { 2023-12-07T10:36:42.465 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_objs": 0, 2023-12-07T10:36:42.465 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_kv_pairs": 0, 2023-12-07T10:36:42.465 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_updates": 0, 2023-12-07T10:36:42.466 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_removes": 0 2023-12-07T10:36:42.466 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.466 INFO:teuthology.orchestra.run.smithi153.stdout: "osd-slow-ops": { 2023-12-07T10:36:42.466 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_ops_count": 0 2023-12-07T10:36:42.466 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.466 INFO:teuthology.orchestra.run.smithi153.stdout: "purge_queue": { 2023-12-07T10:36:42.466 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops": 0, 2023-12-07T10:36:42.466 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops_high_water": 0, 2023-12-07T10:36:42.466 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing": 0, 2023-12-07T10:36:42.467 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_high_water": 0, 2023-12-07T10:36:42.467 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed": 0, 2023-12-07T10:36:42.467 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_item_in_journal": 0 2023-12-07T10:36:42.467 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.467 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-07T10:36:42.467 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2023-12-07T10:36:42.467 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2023-12-07T10:36:42.467 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2023-12-07T10:36:42.467 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 5084, 2023-12-07T10:36:42.468 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 668628565, 2023-12-07T10:36:42.468 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:42.468 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 5084, 2023-12-07T10:36:42.468 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2023-12-07T10:36:42.468 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2023-12-07T10:36:42.468 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 5084, 2023-12-07T10:36:42.468 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 668628565, 2023-12-07T10:36:42.468 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2023-12-07T10:36:42.468 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.469 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.469 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.469 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:42.469 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.469 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_bytes": { 2023-12-07T10:36:42.469 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2023-12-07T10:36:42.469 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2023-12-07T10:36:42.469 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2023-12-07T10:36:42.470 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2023-12-07T10:36:42.470 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2023-12-07T10:36:42.470 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:42.470 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:42.470 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 4679, 2023-12-07T10:36:42.470 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 665856065, 2023-12-07T10:36:42.470 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 4679, 2023-12-07T10:36:42.470 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 665856065, 2023-12-07T10:36:42.470 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2023-12-07T10:36:42.471 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.471 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.471 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.471 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:42.471 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.471 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_ops": { 2023-12-07T10:36:42.471 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2023-12-07T10:36:42.471 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 1024, 2023-12-07T10:36:42.471 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2023-12-07T10:36:42.472 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2023-12-07T10:36:42.472 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2023-12-07T10:36:42.472 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:42.472 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:42.472 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 4679, 2023-12-07T10:36:42.472 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 4679, 2023-12-07T10:36:42.472 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 4679, 2023-12-07T10:36:42.472 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 4679, 2023-12-07T10:36:42.472 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2023-12-07T10:36:42.473 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.473 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.473 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.473 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:42.473 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.473 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle": { 2023-12-07T10:36:42.473 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2023-12-07T10:36:42.473 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2023-12-07T10:36:42.473 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2023-12-07T10:36:42.473 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2023-12-07T10:36:42.474 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2023-12-07T10:36:42.474 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:42.474 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:42.474 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2023-12-07T10:36:42.474 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2023-12-07T10:36:42.474 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 0, 2023-12-07T10:36:42.474 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 0, 2023-12-07T10:36:42.474 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2023-12-07T10:36:42.475 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.475 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.475 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.475 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:42.475 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:42.475 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle-0x561bbde4a960": { 2023-12-07T10:36:42.475 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2023-12-07T10:36:42.475 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2023-12-07T10:36:42.475 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2023-12-07T10:36:42.476 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2023-12-07T10:36:42.476 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2023-12-07T10:36:42.476 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:42.476 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:42.476 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2023-12-07T10:36:42.476 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2023-12-07T10:36:42.476 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 0, 2023-12-07T10:36:42.476 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 0, 2023-12-07T10:36:42.476 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2023-12-07T10:36:42.476 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:42.477 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:42.477 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:42.477 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:42.477 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:42.477 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-12-07T10:36:42.478 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:42 smithi119 ceph-mon[97908]: pgmap v511: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 45 KiB/s wr, 14 op/s 2023-12-07T10:36:42.478 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:42 smithi153 ceph-mon[133625]: pgmap v511: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 45 KiB/s wr, 14 op/s 2023-12-07T10:36:42.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:42 smithi159 ceph-mon[134726]: pgmap v511: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 45 KiB/s wr, 14 op/s 2023-12-07T10:36:42.917 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2023-12-07T10:36:42.918 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2023-12-07T10:36:42.918 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2023-12-07T10:36:42.918 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2023-12-07T10:36:42.918 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2023-12-07T10:36:42.918 DEBUG:tasks.check_counter:Getting stats from k 2023-12-07T10:36:42.918 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph daemon mds.k perf dump 2023-12-07T10:36:43.512 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.b/config 2023-12-07T10:36:44.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:44 smithi119 ceph-mon[97908]: pgmap v512: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 KiB/s wr, 14 op/s 2023-12-07T10:36:44.488 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:44 smithi153 ceph-mon[133625]: pgmap v512: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 KiB/s wr, 14 op/s 2023-12-07T10:36:44.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:44 smithi159 ceph-mon[134726]: pgmap v512: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 KiB/s wr, 14 op/s 2023-12-07T10:36:44.680 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-12-07T10:36:44.680 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-0": { 2023-12-07T10:36:44.680 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 1047, 2023-12-07T10:36:44.680 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 1049, 2023-12-07T10:36:44.680 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 46421078, 2023-12-07T10:36:44.680 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 325681, 2023-12-07T10:36:44.681 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 8, 2023-12-07T10:36:44.681 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 3, 2023-12-07T10:36:44.681 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.107487718, 2023-12-07T10:36:44.681 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.024736086, 2023-12-07T10:36:44.681 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.047959539, 2023-12-07T10:36:44.681 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.026531112, 2023-12-07T10:36:44.681 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:44.681 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1047, 2023-12-07T10:36:44.681 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.034856988, 2023-12-07T10:36:44.682 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000033292 2023-12-07T10:36:44.682 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.682 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:44.682 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.682 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.682 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.682 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.682 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-07T10:36:44.682 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 0 2023-12-07T10:36:44.683 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.683 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-1": { 2023-12-07T10:36:44.683 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 1289, 2023-12-07T10:36:44.683 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 1524, 2023-12-07T10:36:44.683 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 63240168, 2023-12-07T10:36:44.683 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 961802, 2023-12-07T10:36:44.683 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 8, 2023-12-07T10:36:44.683 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 6, 2023-12-07T10:36:44.683 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.137822037, 2023-12-07T10:36:44.683 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.029777565, 2023-12-07T10:36:44.684 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.062848027, 2023-12-07T10:36:44.684 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.034873467, 2023-12-07T10:36:44.684 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:44.684 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 1522, 2023-12-07T10:36:44.684 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.069180977, 2023-12-07T10:36:44.684 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000045453 2023-12-07T10:36:44.684 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.684 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:44.684 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.685 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.685 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.685 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.685 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-12-07T10:36:44.685 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 562784 2023-12-07T10:36:44.685 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.685 INFO:teuthology.orchestra.run.smithi153.stdout: "AsyncMessenger::Worker-2": { 2023-12-07T10:36:44.685 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_messages": 2113, 2023-12-07T10:36:44.685 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages": 2054, 2023-12-07T10:36:44.686 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_bytes": 36243978, 2023-12-07T10:36:44.686 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_bytes": 726400, 2023-12-07T10:36:44.686 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_created_connections": 5, 2023-12-07T10:36:44.686 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_active_connections": 4, 2023-12-07T10:36:44.686 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_total_time": 0.211357963, 2023-12-07T10:36:44.686 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_send_time": 0.054956764, 2023-12-07T10:36:44.686 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_recv_time": 0.095784120, 2023-12-07T10:36:44.686 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_running_fast_dispatch_time": 0.058855470, 2023-12-07T10:36:44.687 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:44.687 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 2053, 2023-12-07T10:36:44.687 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.085394285, 2023-12-07T10:36:44.687 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000041594 2023-12-07T10:36:44.687 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.687 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:44.687 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.687 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.688 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.688 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.688 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_recv_encrypted_bytes": 1937824, 2023-12-07T10:36:44.688 INFO:teuthology.orchestra.run.smithi153.stdout: "msgr_send_encrypted_bytes": 193968 2023-12-07T10:36:44.688 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.688 INFO:teuthology.orchestra.run.smithi153.stdout: "cct": { 2023-12-07T10:36:44.688 INFO:teuthology.orchestra.run.smithi153.stdout: "total_workers": 1, 2023-12-07T10:36:44.688 INFO:teuthology.orchestra.run.smithi153.stdout: "unhealthy_workers": 0 2023-12-07T10:36:44.688 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.689 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-MDSRank": { 2023-12-07T10:36:44.689 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2023-12-07T10:36:44.689 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2023-12-07T10:36:44.689 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 6857, 2023-12-07T10:36:44.689 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.160432987, 2023-12-07T10:36:44.689 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000023396 2023-12-07T10:36:44.689 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:44.689 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.689 INFO:teuthology.orchestra.run.smithi153.stdout: "finisher-PurgeQueue": { 2023-12-07T10:36:44.690 INFO:teuthology.orchestra.run.smithi153.stdout: "queue_len": 0, 2023-12-07T10:36:44.690 INFO:teuthology.orchestra.run.smithi153.stdout: "complete_latency": { 2023-12-07T10:36:44.690 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.690 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.690 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.690 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:44.690 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.690 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2023-12-07T10:36:44.691 INFO:teuthology.orchestra.run.smithi153.stdout: "request": 0, 2023-12-07T10:36:44.691 INFO:teuthology.orchestra.run.smithi153.stdout: "reply": 0, 2023-12-07T10:36:44.691 INFO:teuthology.orchestra.run.smithi153.stdout: "reply_latency": { 2023-12-07T10:36:44.691 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.691 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.691 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.691 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.691 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_reply": 0, 2023-12-07T10:36:44.691 INFO:teuthology.orchestra.run.smithi153.stdout: "forward": 0, 2023-12-07T10:36:44.692 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_complete": 0, 2023-12-07T10:36:44.692 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_fetch_keys": 0, 2023-12-07T10:36:44.692 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_commit": 0, 2023-12-07T10:36:44.692 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_split": 0, 2023-12-07T10:36:44.692 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_merge": 0, 2023-12-07T10:36:44.692 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes": 9133, 2023-12-07T10:36:44.692 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_top": 11, 2023-12-07T10:36:44.692 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_bottom": 9122, 2023-12-07T10:36:44.693 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pin_tail": 0, 2023-12-07T10:36:44.693 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_pinned": 9117, 2023-12-07T10:36:44.693 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_expired": 0, 2023-12-07T10:36:44.693 INFO:teuthology.orchestra.run.smithi153.stdout: "inodes_with_caps": 0, 2023-12-07T10:36:44.693 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": 0, 2023-12-07T10:36:44.693 INFO:teuthology.orchestra.run.smithi153.stdout: "subtrees": 7, 2023-12-07T10:36:44.693 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse": 0, 2023-12-07T10:36:44.693 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_hit": 0, 2023-12-07T10:36:44.693 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_forward": 0, 2023-12-07T10:36:44.694 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_discover": 0, 2023-12-07T10:36:44.694 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_dir_fetch": 0, 2023-12-07T10:36:44.694 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_remote_ino": 0, 2023-12-07T10:36:44.694 INFO:teuthology.orchestra.run.smithi153.stdout: "traverse_lock": 0, 2023-12-07T10:36:44.694 INFO:teuthology.orchestra.run.smithi153.stdout: "load_cent": 0, 2023-12-07T10:36:44.694 INFO:teuthology.orchestra.run.smithi153.stdout: "q": 0, 2023-12-07T10:36:44.694 INFO:teuthology.orchestra.run.smithi153.stdout: "exported": 0, 2023-12-07T10:36:44.694 INFO:teuthology.orchestra.run.smithi153.stdout: "exported_inodes": 0, 2023-12-07T10:36:44.694 INFO:teuthology.orchestra.run.smithi153.stdout: "imported": 0, 2023-12-07T10:36:44.695 INFO:teuthology.orchestra.run.smithi153.stdout: "imported_inodes": 0, 2023-12-07T10:36:44.695 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_dir_fetch": 0, 2023-12-07T10:36:44.695 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_backtrace_fetch": 0, 2023-12-07T10:36:44.695 INFO:teuthology.orchestra.run.smithi153.stdout: "openino_peer_discover": 0, 2023-12-07T10:36:44.695 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rfiles": 0, 2023-12-07T10:36:44.695 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rbytes": 0, 2023-12-07T10:36:44.695 INFO:teuthology.orchestra.run.smithi153.stdout: "root_rsnaps": 0, 2023-12-07T10:36:44.695 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_fetch": 0, 2023-12-07T10:36:44.696 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_set_tag": 0, 2023-12-07T10:36:44.696 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_backtrace_repaired": 0, 2023-12-07T10:36:44.696 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_inotable_repaired": 0, 2023-12-07T10:36:44.696 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_inodes": 0, 2023-12-07T10:36:44.696 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dir_base_inodes": 0, 2023-12-07T10:36:44.696 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_dirfrag_rstats": 0, 2023-12-07T10:36:44.696 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_file_inodes": 0, 2023-12-07T10:36:44.697 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_inode_file_caps": 0, 2023-12-07T10:36:44.697 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_revoke": 0, 2023-12-07T10:36:44.697 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_grant": 0, 2023-12-07T10:36:44.697 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_trunc": 0, 2023-12-07T10:36:44.697 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-07T10:36:44.697 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-07T10:36:44.698 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps": 0, 2023-12-07T10:36:44.698 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_caps_dirty": 0, 2023-12-07T10:36:44.698 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_cap_release": 0, 2023-12-07T10:36:44.698 INFO:teuthology.orchestra.run.smithi153.stdout: "process_request_cap_release": 0 2023-12-07T10:36:44.698 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.698 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_cache": { 2023-12-07T10:36:44.698 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update": 0, 2023-12-07T10:36:44.698 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_update_receipt": 0, 2023-12-07T10:36:44.699 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_try_discover": 0, 2023-12-07T10:36:44.699 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_send_discover": 0, 2023-12-07T10:36:44.699 INFO:teuthology.orchestra.run.smithi153.stdout: "dir_handle_discover": 0, 2023-12-07T10:36:44.699 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays": 0, 2023-12-07T10:36:44.699 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_delayed": 0, 2023-12-07T10:36:44.699 INFO:teuthology.orchestra.run.smithi153.stdout: "num_strays_enqueuing": 0, 2023-12-07T10:36:44.699 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_created": 0, 2023-12-07T10:36:44.699 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_enqueued": 0, 2023-12-07T10:36:44.700 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_reintegrated": 0, 2023-12-07T10:36:44.700 INFO:teuthology.orchestra.run.smithi153.stdout: "strays_migrated": 0, 2023-12-07T10:36:44.700 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_processing": 0, 2023-12-07T10:36:44.700 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_enqueued": 0, 2023-12-07T10:36:44.700 INFO:teuthology.orchestra.run.smithi153.stdout: "num_recovering_prioritized": 0, 2023-12-07T10:36:44.700 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_started": 0, 2023-12-07T10:36:44.700 INFO:teuthology.orchestra.run.smithi153.stdout: "recovery_completed": 0, 2023-12-07T10:36:44.700 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_enqueue_scrub": 0, 2023-12-07T10:36:44.700 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_exportdir": 0, 2023-12-07T10:36:44.701 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_flush": 0, 2023-12-07T10:36:44.701 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragmentdir": 0, 2023-12-07T10:36:44.701 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_fragstats": 0, 2023-12-07T10:36:44.701 INFO:teuthology.orchestra.run.smithi153.stdout: "ireq_inodestats": 0 2023-12-07T10:36:44.701 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.701 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_log": { 2023-12-07T10:36:44.701 INFO:teuthology.orchestra.run.smithi153.stdout: "evlrg": 0, 2023-12-07T10:36:44.701 INFO:teuthology.orchestra.run.smithi153.stdout: "evadd": 0, 2023-12-07T10:36:44.701 INFO:teuthology.orchestra.run.smithi153.stdout: "evex": 0, 2023-12-07T10:36:44.702 INFO:teuthology.orchestra.run.smithi153.stdout: "evtrm": 0, 2023-12-07T10:36:44.702 INFO:teuthology.orchestra.run.smithi153.stdout: "ev": 49956, 2023-12-07T10:36:44.702 INFO:teuthology.orchestra.run.smithi153.stdout: "evexg": 0, 2023-12-07T10:36:44.702 INFO:teuthology.orchestra.run.smithi153.stdout: "evexd": 0, 2023-12-07T10:36:44.702 INFO:teuthology.orchestra.run.smithi153.stdout: "segadd": 0, 2023-12-07T10:36:44.702 INFO:teuthology.orchestra.run.smithi153.stdout: "segex": 0, 2023-12-07T10:36:44.702 INFO:teuthology.orchestra.run.smithi153.stdout: "segtrm": 0, 2023-12-07T10:36:44.702 INFO:teuthology.orchestra.run.smithi153.stdout: "seg": 74, 2023-12-07T10:36:44.702 INFO:teuthology.orchestra.run.smithi153.stdout: "segmjr": 6, 2023-12-07T10:36:44.703 INFO:teuthology.orchestra.run.smithi153.stdout: "segexg": 0, 2023-12-07T10:36:44.703 INFO:teuthology.orchestra.run.smithi153.stdout: "segexd": 0, 2023-12-07T10:36:44.703 INFO:teuthology.orchestra.run.smithi153.stdout: "expos": 4194304, 2023-12-07T10:36:44.703 INFO:teuthology.orchestra.run.smithi153.stdout: "wrpos": 147019662, 2023-12-07T10:36:44.703 INFO:teuthology.orchestra.run.smithi153.stdout: "rdpos": 147019088, 2023-12-07T10:36:44.703 INFO:teuthology.orchestra.run.smithi153.stdout: "jlat": { 2023-12-07T10:36:44.703 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.703 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.703 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.703 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.704 INFO:teuthology.orchestra.run.smithi153.stdout: "replayed": 49956 2023-12-07T10:36:44.704 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.704 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_mem": { 2023-12-07T10:36:44.704 INFO:teuthology.orchestra.run.smithi153.stdout: "ino": 503, 2023-12-07T10:36:44.704 INFO:teuthology.orchestra.run.smithi153.stdout: "ino+": 9374, 2023-12-07T10:36:44.704 INFO:teuthology.orchestra.run.smithi153.stdout: "ino-": 8871, 2023-12-07T10:36:44.705 INFO:teuthology.orchestra.run.smithi153.stdout: "dir": 44, 2023-12-07T10:36:44.705 INFO:teuthology.orchestra.run.smithi153.stdout: "dir+": 68, 2023-12-07T10:36:44.705 INFO:teuthology.orchestra.run.smithi153.stdout: "dir-": 24, 2023-12-07T10:36:44.705 INFO:teuthology.orchestra.run.smithi153.stdout: "dn": 9133, 2023-12-07T10:36:44.705 INFO:teuthology.orchestra.run.smithi153.stdout: "dn+": 9558, 2023-12-07T10:36:44.705 INFO:teuthology.orchestra.run.smithi153.stdout: "dn-": 425, 2023-12-07T10:36:44.705 INFO:teuthology.orchestra.run.smithi153.stdout: "cap": 0, 2023-12-07T10:36:44.705 INFO:teuthology.orchestra.run.smithi153.stdout: "cap+": 0, 2023-12-07T10:36:44.705 INFO:teuthology.orchestra.run.smithi153.stdout: "cap-": 0, 2023-12-07T10:36:44.706 INFO:teuthology.orchestra.run.smithi153.stdout: "rss": 61596, 2023-12-07T10:36:44.706 INFO:teuthology.orchestra.run.smithi153.stdout: "heap": 207228 2023-12-07T10:36:44.706 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.706 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_server": { 2023-12-07T10:36:44.706 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_client_request": 0, 2023-12-07T10:36:44.706 INFO:teuthology.orchestra.run.smithi153.stdout: "dispatch_server_request": 0, 2023-12-07T10:36:44.706 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_request": 0, 2023-12-07T10:36:44.706 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_client_session": 0, 2023-12-07T10:36:44.707 INFO:teuthology.orchestra.run.smithi153.stdout: "handle_peer_request": 0, 2023-12-07T10:36:44.707 INFO:teuthology.orchestra.run.smithi153.stdout: "req_create_latency": { 2023-12-07T10:36:44.707 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.707 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.707 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.707 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.707 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getattr_latency": { 2023-12-07T10:36:44.707 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.708 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.708 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.708 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.708 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getfilelock_latency": { 2023-12-07T10:36:44.708 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.708 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.708 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.708 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.708 INFO:teuthology.orchestra.run.smithi153.stdout: "req_link_latency": { 2023-12-07T10:36:44.709 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.709 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.709 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.709 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.709 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookup_latency": { 2023-12-07T10:36:44.709 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.709 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.709 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.709 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.709 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookuphash_latency": { 2023-12-07T10:36:44.710 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.710 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.710 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.710 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.710 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupino_latency": { 2023-12-07T10:36:44.710 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.710 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.710 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.710 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.711 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupname_latency": { 2023-12-07T10:36:44.711 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.711 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.711 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.711 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.711 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupparent_latency": { 2023-12-07T10:36:44.711 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.711 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.712 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.712 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.712 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lookupsnap_latency": { 2023-12-07T10:36:44.712 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.712 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.712 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.712 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.712 INFO:teuthology.orchestra.run.smithi153.stdout: "req_lssnap_latency": { 2023-12-07T10:36:44.712 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.713 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.713 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.713 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.713 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mkdir_latency": { 2023-12-07T10:36:44.713 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.713 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.713 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.713 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.714 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mknod_latency": { 2023-12-07T10:36:44.714 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.714 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.714 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.714 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.714 INFO:teuthology.orchestra.run.smithi153.stdout: "req_mksnap_latency": { 2023-12-07T10:36:44.714 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.714 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.714 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.715 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.715 INFO:teuthology.orchestra.run.smithi153.stdout: "req_open_latency": { 2023-12-07T10:36:44.715 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.715 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.715 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.715 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.715 INFO:teuthology.orchestra.run.smithi153.stdout: "req_readdir_latency": { 2023-12-07T10:36:44.715 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.715 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.716 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.716 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.716 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rename_latency": { 2023-12-07T10:36:44.716 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.716 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.716 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.716 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.717 INFO:teuthology.orchestra.run.smithi153.stdout: "req_renamesnap_latency": { 2023-12-07T10:36:44.717 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.717 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.717 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.717 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.717 INFO:teuthology.orchestra.run.smithi153.stdout: "req_snapdiff_latency": { 2023-12-07T10:36:44.717 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.717 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.717 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.718 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.718 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmdir_latency": { 2023-12-07T10:36:44.718 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.718 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.718 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.718 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.718 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmsnap_latency": { 2023-12-07T10:36:44.719 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.719 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.719 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.719 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.719 INFO:teuthology.orchestra.run.smithi153.stdout: "req_rmxattr_latency": { 2023-12-07T10:36:44.719 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.719 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.719 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.720 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.720 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setattr_latency": { 2023-12-07T10:36:44.720 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.720 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.720 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.720 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.720 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setdirlayout_latency": { 2023-12-07T10:36:44.720 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.721 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.721 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.721 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.721 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setfilelock_latency": { 2023-12-07T10:36:44.721 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.721 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.721 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.721 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.722 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setlayout_latency": { 2023-12-07T10:36:44.722 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.722 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.722 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.722 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.722 INFO:teuthology.orchestra.run.smithi153.stdout: "req_setxattr_latency": { 2023-12-07T10:36:44.723 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.723 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.723 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.723 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.723 INFO:teuthology.orchestra.run.smithi153.stdout: "req_symlink_latency": { 2023-12-07T10:36:44.723 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.723 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.723 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.724 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.724 INFO:teuthology.orchestra.run.smithi153.stdout: "req_unlink_latency": { 2023-12-07T10:36:44.724 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.724 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.724 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.724 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.724 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_revoke_eviction": 0, 2023-12-07T10:36:44.724 INFO:teuthology.orchestra.run.smithi153.stdout: "cap_acquisition_throttle": 0, 2023-12-07T10:36:44.725 INFO:teuthology.orchestra.run.smithi153.stdout: "req_getvxattr_latency": { 2023-12-07T10:36:44.725 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.725 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.725 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.725 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:44.725 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.725 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_sessions": { 2023-12-07T10:36:44.725 INFO:teuthology.orchestra.run.smithi153.stdout: "session_count": 1, 2023-12-07T10:36:44.726 INFO:teuthology.orchestra.run.smithi153.stdout: "session_add": 3, 2023-12-07T10:36:44.726 INFO:teuthology.orchestra.run.smithi153.stdout: "session_remove": 2, 2023-12-07T10:36:44.726 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_open": 2, 2023-12-07T10:36:44.726 INFO:teuthology.orchestra.run.smithi153.stdout: "sessions_stale": 0, 2023-12-07T10:36:44.726 INFO:teuthology.orchestra.run.smithi153.stdout: "total_load": 0, 2023-12-07T10:36:44.726 INFO:teuthology.orchestra.run.smithi153.stdout: "average_load": 0, 2023-12-07T10:36:44.726 INFO:teuthology.orchestra.run.smithi153.stdout: "avg_session_uptime": 4726, 2023-12-07T10:36:44.726 INFO:teuthology.orchestra.run.smithi153.stdout: "mdthresh_evicted": 0 2023-12-07T10:36:44.726 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.727 INFO:teuthology.orchestra.run.smithi153.stdout: "mempool": { 2023-12-07T10:36:44.727 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_bytes": 0, 2023-12-07T10:36:44.727 INFO:teuthology.orchestra.run.smithi153.stdout: "bloom_filter_items": 0, 2023-12-07T10:36:44.727 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_bytes": 0, 2023-12-07T10:36:44.727 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_alloc_items": 0, 2023-12-07T10:36:44.727 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_bytes": 0, 2023-12-07T10:36:44.727 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_data_items": 0, 2023-12-07T10:36:44.727 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-07T10:36:44.728 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_onode_items": 0, 2023-12-07T10:36:44.728 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-07T10:36:44.728 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_meta_items": 0, 2023-12-07T10:36:44.728 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_bytes": 0, 2023-12-07T10:36:44.728 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_other_items": 0, 2023-12-07T10:36:44.728 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-07T10:36:44.728 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_cache_buffer_items": 0, 2023-12-07T10:36:44.728 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_bytes": 0, 2023-12-07T10:36:44.728 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_extent_items": 0, 2023-12-07T10:36:44.729 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_bytes": 0, 2023-12-07T10:36:44.729 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_blob_items": 0, 2023-12-07T10:36:44.729 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-07T10:36:44.729 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_shared_blob_items": 0, 2023-12-07T10:36:44.729 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-07T10:36:44.729 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_inline_bl_items": 0, 2023-12-07T10:36:44.729 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_bytes": 0, 2023-12-07T10:36:44.729 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_fsck_items": 0, 2023-12-07T10:36:44.729 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_bytes": 0, 2023-12-07T10:36:44.730 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_txc_items": 0, 2023-12-07T10:36:44.730 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-07T10:36:44.730 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_deferred_items": 0, 2023-12-07T10:36:44.730 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_bytes": 0, 2023-12-07T10:36:44.730 INFO:teuthology.orchestra.run.smithi153.stdout: "bluestore_writing_items": 0, 2023-12-07T10:36:44.730 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_bytes": 0, 2023-12-07T10:36:44.730 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_items": 0, 2023-12-07T10:36:44.730 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_bytes": 0, 2023-12-07T10:36:44.730 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_reader_items": 0, 2023-12-07T10:36:44.731 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_bytes": 0, 2023-12-07T10:36:44.731 INFO:teuthology.orchestra.run.smithi153.stdout: "bluefs_file_writer_items": 0, 2023-12-07T10:36:44.731 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_bytes": 244145, 2023-12-07T10:36:44.731 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_anon_items": 643, 2023-12-07T10:36:44.731 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_bytes": 0, 2023-12-07T10:36:44.731 INFO:teuthology.orchestra.run.smithi153.stdout: "buffer_meta_items": 0, 2023-12-07T10:36:44.731 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_bytes": 0, 2023-12-07T10:36:44.731 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_items": 0, 2023-12-07T10:36:44.731 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_bytes": 0, 2023-12-07T10:36:44.732 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_mapbl_items": 0, 2023-12-07T10:36:44.732 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_bytes": 0, 2023-12-07T10:36:44.732 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_pglog_items": 0, 2023-12-07T10:36:44.732 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_bytes": 9616, 2023-12-07T10:36:44.732 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_items": 185, 2023-12-07T10:36:44.732 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_bytes": 0, 2023-12-07T10:36:44.732 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_mapping_items": 0, 2023-12-07T10:36:44.732 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_bytes": 0, 2023-12-07T10:36:44.732 INFO:teuthology.orchestra.run.smithi153.stdout: "pgmap_items": 0, 2023-12-07T10:36:44.733 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_bytes": 7793876, 2023-12-07T10:36:44.733 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_co_items": 50223, 2023-12-07T10:36:44.733 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_bytes": 0, 2023-12-07T10:36:44.733 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_1_items": 0, 2023-12-07T10:36:44.733 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_bytes": 0, 2023-12-07T10:36:44.733 INFO:teuthology.orchestra.run.smithi153.stdout: "unittest_2_items": 0 2023-12-07T10:36:44.733 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.733 INFO:teuthology.orchestra.run.smithi153.stdout: "objecter": { 2023-12-07T10:36:44.734 INFO:teuthology.orchestra.run.smithi153.stdout: "op_active": 0, 2023-12-07T10:36:44.734 INFO:teuthology.orchestra.run.smithi153.stdout: "op_laggy": 0, 2023-12-07T10:36:44.734 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send": 4042, 2023-12-07T10:36:44.734 INFO:teuthology.orchestra.run.smithi153.stdout: "op_send_bytes": 21, 2023-12-07T10:36:44.734 INFO:teuthology.orchestra.run.smithi153.stdout: "op_resend": 3, 2023-12-07T10:36:44.734 INFO:teuthology.orchestra.run.smithi153.stdout: "op_reply": 4042, 2023-12-07T10:36:44.734 INFO:teuthology.orchestra.run.smithi153.stdout: "op_latency": { 2023-12-07T10:36:44.734 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 4042, 2023-12-07T10:36:44.734 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 7.065879558, 2023-12-07T10:36:44.735 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.001748114 2023-12-07T10:36:44.735 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.735 INFO:teuthology.orchestra.run.smithi153.stdout: "op_inflight": 0, 2023-12-07T10:36:44.735 INFO:teuthology.orchestra.run.smithi153.stdout: "oplen_avg": { 2023-12-07T10:36:44.735 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 4042, 2023-12-07T10:36:44.735 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 4043 2023-12-07T10:36:44.735 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.735 INFO:teuthology.orchestra.run.smithi153.stdout: "op": 4042, 2023-12-07T10:36:44.736 INFO:teuthology.orchestra.run.smithi153.stdout: "op_r": 4042, 2023-12-07T10:36:44.736 INFO:teuthology.orchestra.run.smithi153.stdout: "op_w": 0, 2023-12-07T10:36:44.736 INFO:teuthology.orchestra.run.smithi153.stdout: "op_rmw": 0, 2023-12-07T10:36:44.736 INFO:teuthology.orchestra.run.smithi153.stdout: "op_pg": 0, 2023-12-07T10:36:44.736 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_stat": 2213, 2023-12-07T10:36:44.736 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_create": 0, 2023-12-07T10:36:44.736 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_read": 1827, 2023-12-07T10:36:44.736 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_write": 0, 2023-12-07T10:36:44.737 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writefull": 0, 2023-12-07T10:36:44.737 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_writesame": 0, 2023-12-07T10:36:44.737 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_append": 0, 2023-12-07T10:36:44.737 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_zero": 0, 2023-12-07T10:36:44.737 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_truncate": 0, 2023-12-07T10:36:44.737 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_delete": 0, 2023-12-07T10:36:44.737 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_mapext": 0, 2023-12-07T10:36:44.737 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_sparse_read": 0, 2023-12-07T10:36:44.737 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_clonerange": 0, 2023-12-07T10:36:44.738 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_getxattr": 1, 2023-12-07T10:36:44.738 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_setxattr": 0, 2023-12-07T10:36:44.738 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_cmpxattr": 0, 2023-12-07T10:36:44.738 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_rmxattr": 0, 2023-12-07T10:36:44.738 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_resetxattrs": 0, 2023-12-07T10:36:44.738 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_call": 0, 2023-12-07T10:36:44.738 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_watch": 0, 2023-12-07T10:36:44.738 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_notify": 0, 2023-12-07T10:36:44.739 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_src_cmpxattr": 0, 2023-12-07T10:36:44.739 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls": 0, 2023-12-07T10:36:44.739 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_pgls_filter": 0, 2023-12-07T10:36:44.739 INFO:teuthology.orchestra.run.smithi153.stdout: "osdop_other": 0, 2023-12-07T10:36:44.739 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_active": 0, 2023-12-07T10:36:44.739 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_send": 0, 2023-12-07T10:36:44.739 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_resend": 0, 2023-12-07T10:36:44.739 INFO:teuthology.orchestra.run.smithi153.stdout: "linger_ping": 0, 2023-12-07T10:36:44.739 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_active": 0, 2023-12-07T10:36:44.740 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_send": 0, 2023-12-07T10:36:44.740 INFO:teuthology.orchestra.run.smithi153.stdout: "poolop_resend": 0, 2023-12-07T10:36:44.740 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_active": 0, 2023-12-07T10:36:44.740 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_send": 0, 2023-12-07T10:36:44.740 INFO:teuthology.orchestra.run.smithi153.stdout: "poolstat_resend": 0, 2023-12-07T10:36:44.740 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_active": 0, 2023-12-07T10:36:44.740 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_send": 0, 2023-12-07T10:36:44.740 INFO:teuthology.orchestra.run.smithi153.stdout: "statfs_resend": 0, 2023-12-07T10:36:44.740 INFO:teuthology.orchestra.run.smithi153.stdout: "command_active": 0, 2023-12-07T10:36:44.741 INFO:teuthology.orchestra.run.smithi153.stdout: "command_send": 0, 2023-12-07T10:36:44.741 INFO:teuthology.orchestra.run.smithi153.stdout: "command_resend": 0, 2023-12-07T10:36:44.741 INFO:teuthology.orchestra.run.smithi153.stdout: "map_epoch": 114, 2023-12-07T10:36:44.741 INFO:teuthology.orchestra.run.smithi153.stdout: "map_full": 0, 2023-12-07T10:36:44.741 INFO:teuthology.orchestra.run.smithi153.stdout: "map_inc": 23, 2023-12-07T10:36:44.741 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_sessions": 9, 2023-12-07T10:36:44.741 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_open": 10, 2023-12-07T10:36:44.741 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_session_close": 1, 2023-12-07T10:36:44.742 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_laggy": 0, 2023-12-07T10:36:44.742 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_wr": 0, 2023-12-07T10:36:44.742 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_rd": 2, 2023-12-07T10:36:44.742 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_del": 0 2023-12-07T10:36:44.742 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.742 INFO:teuthology.orchestra.run.smithi153.stdout: "oft": { 2023-12-07T10:36:44.742 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_objs": 0, 2023-12-07T10:36:44.742 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_kv_pairs": 0, 2023-12-07T10:36:44.742 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_updates": 0, 2023-12-07T10:36:44.743 INFO:teuthology.orchestra.run.smithi153.stdout: "omap_total_removes": 0 2023-12-07T10:36:44.743 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.743 INFO:teuthology.orchestra.run.smithi153.stdout: "osd-slow-ops": { 2023-12-07T10:36:44.743 INFO:teuthology.orchestra.run.smithi153.stdout: "slow_ops_count": 0 2023-12-07T10:36:44.743 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.743 INFO:teuthology.orchestra.run.smithi153.stdout: "purge_queue": { 2023-12-07T10:36:44.743 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops": 0, 2023-12-07T10:36:44.743 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_ops_high_water": 0, 2023-12-07T10:36:44.744 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing": 0, 2023-12-07T10:36:44.744 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executing_high_water": 0, 2023-12-07T10:36:44.744 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_executed": 0, 2023-12-07T10:36:44.744 INFO:teuthology.orchestra.run.smithi153.stdout: "pq_item_in_journal": 0 2023-12-07T10:36:44.744 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.744 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-07T10:36:44.744 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2023-12-07T10:36:44.744 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2023-12-07T10:36:44.744 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2023-12-07T10:36:44.745 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 4445, 2023-12-07T10:36:44.745 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 145481917, 2023-12-07T10:36:44.745 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:44.745 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 4445, 2023-12-07T10:36:44.745 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2023-12-07T10:36:44.745 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2023-12-07T10:36:44.745 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 4445, 2023-12-07T10:36:44.745 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 145481917, 2023-12-07T10:36:44.746 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2023-12-07T10:36:44.746 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.746 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.746 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.746 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:44.746 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.746 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_bytes": { 2023-12-07T10:36:44.747 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2023-12-07T10:36:44.747 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 104857600, 2023-12-07T10:36:44.747 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2023-12-07T10:36:44.747 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2023-12-07T10:36:44.747 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2023-12-07T10:36:44.747 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:44.747 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:44.747 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 4042, 2023-12-07T10:36:44.748 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 142848068, 2023-12-07T10:36:44.748 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 4042, 2023-12-07T10:36:44.748 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 142848068, 2023-12-07T10:36:44.748 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2023-12-07T10:36:44.748 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.748 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.748 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.749 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:44.749 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.749 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-objecter_ops": { 2023-12-07T10:36:44.749 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2023-12-07T10:36:44.749 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 1024, 2023-12-07T10:36:44.749 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2023-12-07T10:36:44.749 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2023-12-07T10:36:44.749 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2023-12-07T10:36:44.750 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:44.750 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:44.750 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 4042, 2023-12-07T10:36:44.750 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 4042, 2023-12-07T10:36:44.750 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 4042, 2023-12-07T10:36:44.750 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 4042, 2023-12-07T10:36:44.750 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2023-12-07T10:36:44.750 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.750 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.751 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.751 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:44.751 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.751 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle": { 2023-12-07T10:36:44.751 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2023-12-07T10:36:44.751 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2023-12-07T10:36:44.751 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2023-12-07T10:36:44.751 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2023-12-07T10:36:44.751 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2023-12-07T10:36:44.752 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:44.752 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:44.752 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2023-12-07T10:36:44.752 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2023-12-07T10:36:44.752 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 0, 2023-12-07T10:36:44.752 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 0, 2023-12-07T10:36:44.752 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2023-12-07T10:36:44.752 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.753 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.753 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.753 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:44.753 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-12-07T10:36:44.753 INFO:teuthology.orchestra.run.smithi153.stdout: "throttle-write_buf_throttle-0x5598158ee820": { 2023-12-07T10:36:44.753 INFO:teuthology.orchestra.run.smithi153.stdout: "val": 0, 2023-12-07T10:36:44.753 INFO:teuthology.orchestra.run.smithi153.stdout: "max": 3758096384, 2023-12-07T10:36:44.753 INFO:teuthology.orchestra.run.smithi153.stdout: "get_started": 0, 2023-12-07T10:36:44.753 INFO:teuthology.orchestra.run.smithi153.stdout: "get": 0, 2023-12-07T10:36:44.754 INFO:teuthology.orchestra.run.smithi153.stdout: "get_sum": 0, 2023-12-07T10:36:44.754 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:44.754 INFO:teuthology.orchestra.run.smithi153.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:44.754 INFO:teuthology.orchestra.run.smithi153.stdout: "take": 0, 2023-12-07T10:36:44.754 INFO:teuthology.orchestra.run.smithi153.stdout: "take_sum": 0, 2023-12-07T10:36:44.754 INFO:teuthology.orchestra.run.smithi153.stdout: "put": 0, 2023-12-07T10:36:44.754 INFO:teuthology.orchestra.run.smithi153.stdout: "put_sum": 0, 2023-12-07T10:36:44.754 INFO:teuthology.orchestra.run.smithi153.stdout: "wait": { 2023-12-07T10:36:44.755 INFO:teuthology.orchestra.run.smithi153.stdout: "avgcount": 0, 2023-12-07T10:36:44.755 INFO:teuthology.orchestra.run.smithi153.stdout: "sum": 0.000000000, 2023-12-07T10:36:44.755 INFO:teuthology.orchestra.run.smithi153.stdout: "avgtime": 0.000000000 2023-12-07T10:36:44.755 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:44.755 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-12-07T10:36:44.755 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-12-07T10:36:45.076 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2023-12-07T10:36:45.076 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2023-12-07T10:36:45.076 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2023-12-07T10:36:45.077 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2023-12-07T10:36:45.077 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2023-12-07T10:36:45.077 DEBUG:tasks.check_counter:Getting stats from c 2023-12-07T10:36:45.077 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph daemon mds.c perf dump 2023-12-07T10:36:45.713 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.c/config 2023-12-07T10:36:46.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:46 smithi119 ceph-mon[97908]: pgmap v513: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-12-07T10:36:46.478 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:46 smithi159 ceph-mon[134726]: pgmap v513: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-12-07T10:36:46.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:46 smithi153 ceph-mon[133625]: pgmap v513: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-12-07T10:36:47.043 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2023-12-07T10:36:47.043 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2023-12-07T10:36:47.043 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 97724, 2023-12-07T10:36:47.043 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 63975, 2023-12-07T10:36:47.043 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 33827803, 2023-12-07T10:36:47.043 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 61974630, 2023-12-07T10:36:47.043 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 14, 2023-12-07T10:36:47.044 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 4, 2023-12-07T10:36:47.044 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 5.162473944, 2023-12-07T10:36:47.044 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 1.884123590, 2023-12-07T10:36:47.044 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 3.799977315, 2023-12-07T10:36:47.044 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.314574761, 2023-12-07T10:36:47.044 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:47.044 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 63973, 2023-12-07T10:36:47.044 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 3.195191057, 2023-12-07T10:36:47.044 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000049945 2023-12-07T10:36:47.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.045 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:47.045 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 140089, 2023-12-07T10:36:47.045 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.021318061, 2023-12-07T10:36:47.045 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000152 2023-12-07T10:36:47.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.045 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 1088, 2023-12-07T10:36:47.045 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 434048 2023-12-07T10:36:47.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.046 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2023-12-07T10:36:47.046 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 107157, 2023-12-07T10:36:47.046 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 155160, 2023-12-07T10:36:47.046 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 38152364, 2023-12-07T10:36:47.046 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 203210799, 2023-12-07T10:36:47.046 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 24, 2023-12-07T10:36:47.046 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 5, 2023-12-07T10:36:47.046 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 8.140399826, 2023-12-07T10:36:47.046 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 3.473486981, 2023-12-07T10:36:47.047 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 5.111029023, 2023-12-07T10:36:47.047 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.599923732, 2023-12-07T10:36:47.047 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:47.047 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 155157, 2023-12-07T10:36:47.047 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 19.977272326, 2023-12-07T10:36:47.047 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000128755 2023-12-07T10:36:47.047 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.047 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:47.047 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 195282, 2023-12-07T10:36:47.048 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.042837957, 2023-12-07T10:36:47.048 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000219 2023-12-07T10:36:47.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.048 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 32435488, 2023-12-07T10:36:47.048 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 119848448 2023-12-07T10:36:47.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.048 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2023-12-07T10:36:47.048 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 25394, 2023-12-07T10:36:47.048 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 22351, 2023-12-07T10:36:47.049 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 10689319, 2023-12-07T10:36:47.049 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 28370925, 2023-12-07T10:36:47.049 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 15, 2023-12-07T10:36:47.049 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 9, 2023-12-07T10:36:47.049 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 1.636892895, 2023-12-07T10:36:47.049 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.613102899, 2023-12-07T10:36:47.049 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.936191385, 2023-12-07T10:36:47.049 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.243654420, 2023-12-07T10:36:47.049 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:47.050 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 22351, 2023-12-07T10:36:47.050 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 2.763609624, 2023-12-07T10:36:47.050 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000123645 2023-12-07T10:36:47.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.050 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:47.050 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 33204, 2023-12-07T10:36:47.050 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.006773122, 2023-12-07T10:36:47.050 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000203 2023-12-07T10:36:47.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.051 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 1977136, 2023-12-07T10:36:47.051 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 203168 2023-12-07T10:36:47.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.051 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2023-12-07T10:36:47.051 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2023-12-07T10:36:47.051 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2023-12-07T10:36:47.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.051 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2023-12-07T10:36:47.052 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2023-12-07T10:36:47.052 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2023-12-07T10:36:47.052 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 24650, 2023-12-07T10:36:47.052 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 27.178812576, 2023-12-07T10:36:47.052 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001102588 2023-12-07T10:36:47.052 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:47.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.053 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2023-12-07T10:36:47.053 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2023-12-07T10:36:47.053 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2023-12-07T10:36:47.053 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 16810, 2023-12-07T10:36:47.053 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 1.856679791, 2023-12-07T10:36:47.053 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000110450 2023-12-07T10:36:47.053 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:47.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.054 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2023-12-07T10:36:47.054 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 59666, 2023-12-07T10:36:47.054 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 55837, 2023-12-07T10:36:47.054 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2023-12-07T10:36:47.054 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 55837, 2023-12-07T10:36:47.054 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 206.698923402, 2023-12-07T10:36:47.054 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.003701827 2023-12-07T10:36:47.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.055 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2023-12-07T10:36:47.055 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 3829, 2023-12-07T10:36:47.055 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 5, 2023-12-07T10:36:47.055 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2023-12-07T10:36:47.055 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 11, 2023-12-07T10:36:47.055 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2023-12-07T10:36:47.055 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2023-12-07T10:36:47.056 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 674, 2023-12-07T10:36:47.056 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 37, 2023-12-07T10:36:47.056 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 300, 2023-12-07T10:36:47.056 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 337, 2023-12-07T10:36:47.056 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 622, 2023-12-07T10:36:47.056 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 509, 2023-12-07T10:36:47.056 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 109, 2023-12-07T10:36:47.056 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 109, 2023-12-07T10:36:47.056 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 14, 2023-12-07T10:36:47.057 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 86510, 2023-12-07T10:36:47.057 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 49133, 2023-12-07T10:36:47.057 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 3821, 2023-12-07T10:36:47.057 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 62, 2023-12-07T10:36:47.057 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2023-12-07T10:36:47.057 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2023-12-07T10:36:47.057 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 274, 2023-12-07T10:36:47.057 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 8906, 2023-12-07T10:36:47.058 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2023-12-07T10:36:47.058 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 14, 2023-12-07T10:36:47.058 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 415, 2023-12-07T10:36:47.058 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 15, 2023-12-07T10:36:47.058 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 532, 2023-12-07T10:36:47.058 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2023-12-07T10:36:47.058 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 36, 2023-12-07T10:36:47.058 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2023-12-07T10:36:47.058 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 65, 2023-12-07T10:36:47.059 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 21133868, 2023-12-07T10:36:47.059 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 4, 2023-12-07T10:36:47.059 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 36, 2023-12-07T10:36:47.059 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2023-12-07T10:36:47.059 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2023-12-07T10:36:47.059 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2023-12-07T10:36:47.059 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 12, 2023-12-07T10:36:47.059 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2023-12-07T10:36:47.060 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 12, 2023-12-07T10:36:47.060 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 24, 2023-12-07T10:36:47.060 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 4, 2023-12-07T10:36:47.060 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 5869, 2023-12-07T10:36:47.060 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 38114, 2023-12-07T10:36:47.060 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2023-12-07T10:36:47.060 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-07T10:36:47.060 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-07T10:36:47.060 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 20270, 2023-12-07T10:36:47.060 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 8500, 2023-12-07T10:36:47.061 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 3141, 2023-12-07T10:36:47.061 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 42622 2023-12-07T10:36:47.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.061 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2023-12-07T10:36:47.061 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 11, 2023-12-07T10:36:47.061 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 175, 2023-12-07T10:36:47.061 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 102, 2023-12-07T10:36:47.061 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 223, 2023-12-07T10:36:47.061 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 36, 2023-12-07T10:36:47.062 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2023-12-07T10:36:47.062 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2023-12-07T10:36:47.062 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2023-12-07T10:36:47.062 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 8507, 2023-12-07T10:36:47.062 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 8507, 2023-12-07T10:36:47.062 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2023-12-07T10:36:47.062 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2023-12-07T10:36:47.062 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2023-12-07T10:36:47.063 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2023-12-07T10:36:47.063 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2023-12-07T10:36:47.063 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2023-12-07T10:36:47.063 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2023-12-07T10:36:47.063 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2023-12-07T10:36:47.063 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 18, 2023-12-07T10:36:47.063 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2023-12-07T10:36:47.063 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2023-12-07T10:36:47.063 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2023-12-07T10:36:47.064 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2023-12-07T10:36:47.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.064 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2023-12-07T10:36:47.064 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2023-12-07T10:36:47.064 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 49960, 2023-12-07T10:36:47.064 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2023-12-07T10:36:47.064 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2023-12-07T10:36:47.064 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 49960, 2023-12-07T10:36:47.064 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2023-12-07T10:36:47.065 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2023-12-07T10:36:47.065 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 74, 2023-12-07T10:36:47.065 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2023-12-07T10:36:47.065 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2023-12-07T10:36:47.065 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 74, 2023-12-07T10:36:47.065 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 6, 2023-12-07T10:36:47.065 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2023-12-07T10:36:47.065 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2023-12-07T10:36:47.065 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4194304, 2023-12-07T10:36:47.066 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 147027819, 2023-12-07T10:36:47.066 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 0, 2023-12-07T10:36:47.066 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2023-12-07T10:36:47.066 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 13557, 2023-12-07T10:36:47.066 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 128.887401098, 2023-12-07T10:36:47.066 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.009507073 2023-12-07T10:36:47.066 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.066 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 0 2023-12-07T10:36:47.066 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.067 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2023-12-07T10:36:47.067 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 533, 2023-12-07T10:36:47.067 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 9448, 2023-12-07T10:36:47.067 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 8915, 2023-12-07T10:36:47.067 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 60, 2023-12-07T10:36:47.067 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 192, 2023-12-07T10:36:47.067 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 132, 2023-12-07T10:36:47.067 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 674, 2023-12-07T10:36:47.067 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 17039, 2023-12-07T10:36:47.068 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 16365, 2023-12-07T10:36:47.068 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 109, 2023-12-07T10:36:47.068 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 9442, 2023-12-07T10:36:47.068 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 9333, 2023-12-07T10:36:47.068 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 67316, 2023-12-07T10:36:47.068 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 199036 2023-12-07T10:36:47.068 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.068 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2023-12-07T10:36:47.068 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 81608, 2023-12-07T10:36:47.069 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 474, 2023-12-07T10:36:47.069 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 59666, 2023-12-07T10:36:47.069 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 854, 2023-12-07T10:36:47.069 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 4491, 2023-12-07T10:36:47.069 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2023-12-07T10:36:47.069 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 8533, 2023-12-07T10:36:47.069 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 31.195413412, 2023-12-07T10:36:47.069 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.003655855 2023-12-07T10:36:47.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.070 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2023-12-07T10:36:47.070 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 792, 2023-12-07T10:36:47.070 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 15.669545007, 2023-12-07T10:36:47.070 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.019784779 2023-12-07T10:36:47.070 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.070 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2023-12-07T10:36:47.070 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.070 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.070 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.071 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.071 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2023-12-07T10:36:47.071 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.071 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.071 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.071 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.071 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2023-12-07T10:36:47.071 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 11639, 2023-12-07T10:36:47.072 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 22.045414418, 2023-12-07T10:36:47.072 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001894098 2023-12-07T10:36:47.072 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.072 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2023-12-07T10:36:47.072 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.072 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.072 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.072 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.072 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2023-12-07T10:36:47.072 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.073 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.073 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.073 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.073 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2023-12-07T10:36:47.073 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.073 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.073 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.073 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.074 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2023-12-07T10:36:47.074 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.074 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.074 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.074 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.074 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2023-12-07T10:36:47.074 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.074 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.074 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.075 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.075 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2023-12-07T10:36:47.075 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.075 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.075 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.075 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.075 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2023-12-07T10:36:47.075 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.076 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.076 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.076 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.076 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2023-12-07T10:36:47.076 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.076 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.076 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.076 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.076 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2023-12-07T10:36:47.077 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.077 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.077 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.077 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.077 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2023-12-07T10:36:47.077 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.077 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.077 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.077 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.078 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2023-12-07T10:36:47.078 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 24813, 2023-12-07T10:36:47.078 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 112.850038379, 2023-12-07T10:36:47.078 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.004548020 2023-12-07T10:36:47.078 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.078 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2023-12-07T10:36:47.078 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1304, 2023-12-07T10:36:47.078 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 3.451565506, 2023-12-07T10:36:47.079 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002646906 2023-12-07T10:36:47.079 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.079 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2023-12-07T10:36:47.079 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.079 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.079 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.079 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.079 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2023-12-07T10:36:47.079 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.080 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.080 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.080 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.080 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2023-12-07T10:36:47.080 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.080 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.080 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.080 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.081 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2023-12-07T10:36:47.081 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.081 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.081 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.081 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.081 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2023-12-07T10:36:47.081 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.081 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.081 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.082 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.082 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2023-12-07T10:36:47.082 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 5, 2023-12-07T10:36:47.082 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.088401594, 2023-12-07T10:36:47.082 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.017680318 2023-12-07T10:36:47.082 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.082 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2023-12-07T10:36:47.082 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.083 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.083 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.083 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.083 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2023-12-07T10:36:47.083 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 244, 2023-12-07T10:36:47.083 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.284810239, 2023-12-07T10:36:47.083 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001167255 2023-12-07T10:36:47.083 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.084 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2023-12-07T10:36:47.084 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.084 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.084 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.084 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.084 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2023-12-07T10:36:47.084 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.084 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.085 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.085 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.085 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2023-12-07T10:36:47.085 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.085 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.085 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.085 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.086 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2023-12-07T10:36:47.086 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 8507, 2023-12-07T10:36:47.086 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 21.113734847, 2023-12-07T10:36:47.086 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002481924 2023-12-07T10:36:47.086 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.086 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2023-12-07T10:36:47.086 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2023-12-07T10:36:47.086 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2023-12-07T10:36:47.087 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.087 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.087 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.087 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:47.087 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.087 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2023-12-07T10:36:47.087 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 1, 2023-12-07T10:36:47.087 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 3, 2023-12-07T10:36:47.087 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 2, 2023-12-07T10:36:47.088 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 1, 2023-12-07T10:36:47.088 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2023-12-07T10:36:47.088 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 14404, 2023-12-07T10:36:47.088 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 14404, 2023-12-07T10:36:47.088 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 892, 2023-12-07T10:36:47.088 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2023-12-07T10:36:47.088 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.088 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2023-12-07T10:36:47.089 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2023-12-07T10:36:47.089 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2023-12-07T10:36:47.089 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2023-12-07T10:36:47.089 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2023-12-07T10:36:47.089 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2023-12-07T10:36:47.089 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2023-12-07T10:36:47.089 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-07T10:36:47.089 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2023-12-07T10:36:47.089 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-07T10:36:47.090 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2023-12-07T10:36:47.090 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2023-12-07T10:36:47.090 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2023-12-07T10:36:47.090 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-07T10:36:47.090 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2023-12-07T10:36:47.090 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2023-12-07T10:36:47.090 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2023-12-07T10:36:47.090 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2023-12-07T10:36:47.091 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2023-12-07T10:36:47.091 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-07T10:36:47.091 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2023-12-07T10:36:47.091 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-07T10:36:47.091 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2023-12-07T10:36:47.091 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2023-12-07T10:36:47.091 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2023-12-07T10:36:47.091 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2023-12-07T10:36:47.091 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2023-12-07T10:36:47.092 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-07T10:36:47.092 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2023-12-07T10:36:47.092 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2023-12-07T10:36:47.092 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2023-12-07T10:36:47.092 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2023-12-07T10:36:47.092 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2023-12-07T10:36:47.092 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2023-12-07T10:36:47.092 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2023-12-07T10:36:47.093 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2023-12-07T10:36:47.093 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2023-12-07T10:36:47.093 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 361921, 2023-12-07T10:36:47.093 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 450, 2023-12-07T10:36:47.093 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 352, 2023-12-07T10:36:47.093 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 4, 2023-12-07T10:36:47.093 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2023-12-07T10:36:47.093 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2023-12-07T10:36:47.093 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2023-12-07T10:36:47.094 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2023-12-07T10:36:47.094 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2023-12-07T10:36:47.094 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2023-12-07T10:36:47.094 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 9616, 2023-12-07T10:36:47.094 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 185, 2023-12-07T10:36:47.094 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2023-12-07T10:36:47.094 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2023-12-07T10:36:47.094 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2023-12-07T10:36:47.095 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2023-12-07T10:36:47.095 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 2008026, 2023-12-07T10:36:47.095 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 17714, 2023-12-07T10:36:47.095 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2023-12-07T10:36:47.095 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2023-12-07T10:36:47.095 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2023-12-07T10:36:47.095 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2023-12-07T10:36:47.095 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.095 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2023-12-07T10:36:47.096 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2023-12-07T10:36:47.096 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2023-12-07T10:36:47.096 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 32128, 2023-12-07T10:36:47.096 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 144408911, 2023-12-07T10:36:47.096 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 33, 2023-12-07T10:36:47.096 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 32128, 2023-12-07T10:36:47.096 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2023-12-07T10:36:47.096 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 32128, 2023-12-07T10:36:47.096 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 422.780044652, 2023-12-07T10:36:47.097 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.013159239 2023-12-07T10:36:47.097 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.097 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2023-12-07T10:36:47.097 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2023-12-07T10:36:47.097 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 32128, 2023-12-07T10:36:47.097 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 32450 2023-12-07T10:36:47.097 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.097 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 32128, 2023-12-07T10:36:47.097 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 534, 2023-12-07T10:36:47.098 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 31594, 2023-12-07T10:36:47.098 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2023-12-07T10:36:47.098 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2023-12-07T10:36:47.098 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 0, 2023-12-07T10:36:47.098 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 10, 2023-12-07T10:36:47.098 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 486, 2023-12-07T10:36:47.098 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 14043, 2023-12-07T10:36:47.098 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 303, 2023-12-07T10:36:47.098 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2023-12-07T10:36:47.099 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2023-12-07T10:36:47.099 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2023-12-07T10:36:47.099 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2023-12-07T10:36:47.099 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 17060, 2023-12-07T10:36:47.099 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2023-12-07T10:36:47.099 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2023-12-07T10:36:47.099 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2023-12-07T10:36:47.099 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 36, 2023-12-07T10:36:47.099 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 20, 2023-12-07T10:36:47.100 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2023-12-07T10:36:47.100 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2023-12-07T10:36:47.100 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2023-12-07T10:36:47.100 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2023-12-07T10:36:47.100 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2023-12-07T10:36:47.100 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2023-12-07T10:36:47.100 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2023-12-07T10:36:47.100 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2023-12-07T10:36:47.100 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2023-12-07T10:36:47.101 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2023-12-07T10:36:47.101 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2023-12-07T10:36:47.101 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2023-12-07T10:36:47.101 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2023-12-07T10:36:47.101 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2023-12-07T10:36:47.101 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2023-12-07T10:36:47.101 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2023-12-07T10:36:47.101 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2023-12-07T10:36:47.102 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2023-12-07T10:36:47.102 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2023-12-07T10:36:47.102 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2023-12-07T10:36:47.102 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2023-12-07T10:36:47.102 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2023-12-07T10:36:47.102 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2023-12-07T10:36:47.102 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2023-12-07T10:36:47.102 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2023-12-07T10:36:47.102 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2023-12-07T10:36:47.103 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 114, 2023-12-07T10:36:47.103 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2023-12-07T10:36:47.103 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 23, 2023-12-07T10:36:47.103 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 12, 2023-12-07T10:36:47.103 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 12, 2023-12-07T10:36:47.103 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 0, 2023-12-07T10:36:47.103 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2023-12-07T10:36:47.103 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 340, 2023-12-07T10:36:47.103 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 17, 2023-12-07T10:36:47.104 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 135 2023-12-07T10:36:47.104 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.104 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2023-12-07T10:36:47.104 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 1, 2023-12-07T10:36:47.104 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2023-12-07T10:36:47.104 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2023-12-07T10:36:47.104 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2023-12-07T10:36:47.104 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.104 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2023-12-07T10:36:47.105 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 1 2023-12-07T10:36:47.105 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.105 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2023-12-07T10:36:47.105 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2023-12-07T10:36:47.105 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 10, 2023-12-07T10:36:47.105 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2023-12-07T10:36:47.105 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 5, 2023-12-07T10:36:47.105 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 8507, 2023-12-07T10:36:47.105 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2023-12-07T10:36:47.106 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.106 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-07T10:36:47.106 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-12-07T10:36:47.106 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2023-12-07T10:36:47.106 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-12-07T10:36:47.106 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 230271, 2023-12-07T10:36:47.106 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 58115803, 2023-12-07T10:36:47.106 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:47.106 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 230271, 2023-12-07T10:36:47.107 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-12-07T10:36:47.107 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-12-07T10:36:47.107 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 230271, 2023-12-07T10:36:47.107 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 58115803, 2023-12-07T10:36:47.107 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-12-07T10:36:47.107 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.107 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.107 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.107 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:47.107 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.108 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2023-12-07T10:36:47.108 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-12-07T10:36:47.108 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2023-12-07T10:36:47.108 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-12-07T10:36:47.108 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-12-07T10:36:47.108 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-12-07T10:36:47.108 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:47.108 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:47.109 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 32128, 2023-12-07T10:36:47.109 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 145338036, 2023-12-07T10:36:47.109 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 32128, 2023-12-07T10:36:47.109 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 145338036, 2023-12-07T10:36:47.109 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-12-07T10:36:47.109 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.109 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.109 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.109 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:47.110 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.110 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2023-12-07T10:36:47.110 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-12-07T10:36:47.110 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2023-12-07T10:36:47.110 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-12-07T10:36:47.110 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-12-07T10:36:47.110 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-12-07T10:36:47.110 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:47.110 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:47.111 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 32128, 2023-12-07T10:36:47.111 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 32128, 2023-12-07T10:36:47.111 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 32128, 2023-12-07T10:36:47.111 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 32128, 2023-12-07T10:36:47.111 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-12-07T10:36:47.111 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.111 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.111 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.111 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:47.112 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.112 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2023-12-07T10:36:47.112 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-12-07T10:36:47.112 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2023-12-07T10:36:47.112 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-12-07T10:36:47.112 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 8507, 2023-12-07T10:36:47.112 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 929423, 2023-12-07T10:36:47.112 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:47.113 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 8507, 2023-12-07T10:36:47.113 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-12-07T10:36:47.113 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-12-07T10:36:47.113 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 486, 2023-12-07T10:36:47.113 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 929423, 2023-12-07T10:36:47.113 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-12-07T10:36:47.113 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.113 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.113 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.114 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:47.114 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:47.114 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x559abe9586e0": { 2023-12-07T10:36:47.114 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-12-07T10:36:47.114 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2023-12-07T10:36:47.114 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-12-07T10:36:47.114 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 49960, 2023-12-07T10:36:47.114 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 142833515, 2023-12-07T10:36:47.115 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:47.115 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 49960, 2023-12-07T10:36:47.115 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-12-07T10:36:47.115 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-12-07T10:36:47.115 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 13557, 2023-12-07T10:36:47.115 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 142833515, 2023-12-07T10:36:47.115 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-12-07T10:36:47.115 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:47.115 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:47.116 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:47.116 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:47.116 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:47.116 INFO:teuthology.orchestra.run.smithi159.stdout:} 2023-12-07T10:36:47.416 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:47 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:36:47.459 INFO:tasks.check_counter:Daemon mds.c mds.exported=14 2023-12-07T10:36:47.459 INFO:tasks.check_counter:Daemon mds.c mds.imported=15 2023-12-07T10:36:47.459 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=4 2023-12-07T10:36:47.459 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2023-12-07T10:36:47.459 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2023-12-07T10:36:47.459 DEBUG:tasks.check_counter:Getting stats from f 2023-12-07T10:36:47.460 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph daemon mds.f perf dump 2023-12-07T10:36:47.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:47 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:36:47.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:47 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:36:48.059 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.c/config 2023-12-07T10:36:48.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:48 smithi119 ceph-mon[97908]: pgmap v514: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 15 op/s 2023-12-07T10:36:48.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:48 smithi159 ceph-mon[134726]: pgmap v514: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 15 op/s 2023-12-07T10:36:48.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:48 smithi153 ceph-mon[133625]: pgmap v514: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 15 op/s 2023-12-07T10:36:49.301 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2023-12-07T10:36:49.302 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2023-12-07T10:36:49.302 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 221736, 2023-12-07T10:36:49.302 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 261061, 2023-12-07T10:36:49.302 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 72158686, 2023-12-07T10:36:49.302 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 217596459, 2023-12-07T10:36:49.303 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 27, 2023-12-07T10:36:49.303 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 9, 2023-12-07T10:36:49.303 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 13.036354034, 2023-12-07T10:36:49.303 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 5.491686907, 2023-12-07T10:36:49.303 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 8.890915353, 2023-12-07T10:36:49.303 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.442873601, 2023-12-07T10:36:49.303 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:49.303 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 261057, 2023-12-07T10:36:49.303 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 20.467389522, 2023-12-07T10:36:49.304 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000078401 2023-12-07T10:36:49.304 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.304 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:49.304 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 419300, 2023-12-07T10:36:49.304 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.068238047, 2023-12-07T10:36:49.304 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000162 2023-12-07T10:36:49.304 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.304 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 37862928, 2023-12-07T10:36:49.304 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 136925952 2023-12-07T10:36:49.305 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.305 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2023-12-07T10:36:49.305 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 33464, 2023-12-07T10:36:49.305 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 36201, 2023-12-07T10:36:49.305 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 12159981, 2023-12-07T10:36:49.305 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 77423751, 2023-12-07T10:36:49.305 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 10, 2023-12-07T10:36:49.305 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 5, 2023-12-07T10:36:49.306 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 2.398131085, 2023-12-07T10:36:49.306 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.881628633, 2023-12-07T10:36:49.306 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.999499865, 2023-12-07T10:36:49.306 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.576160478, 2023-12-07T10:36:49.306 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:49.306 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 36204, 2023-12-07T10:36:49.306 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 4.209629645, 2023-12-07T10:36:49.306 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000116275 2023-12-07T10:36:49.306 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.307 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:49.307 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 34175, 2023-12-07T10:36:49.307 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.006499998, 2023-12-07T10:36:49.307 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000190 2023-12-07T10:36:49.307 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.307 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 1974064, 2023-12-07T10:36:49.307 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 212880 2023-12-07T10:36:49.307 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.307 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2023-12-07T10:36:49.308 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 24105, 2023-12-07T10:36:49.308 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 24277, 2023-12-07T10:36:49.308 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 6525657, 2023-12-07T10:36:49.308 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 82448791, 2023-12-07T10:36:49.308 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 17, 2023-12-07T10:36:49.308 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 8, 2023-12-07T10:36:49.308 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 1.699143787, 2023-12-07T10:36:49.308 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.545832320, 2023-12-07T10:36:49.309 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.552193625, 2023-12-07T10:36:49.309 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.571582598, 2023-12-07T10:36:49.309 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:49.309 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 24273, 2023-12-07T10:36:49.309 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 2.969695138, 2023-12-07T10:36:49.309 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000122345 2023-12-07T10:36:49.309 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.309 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:49.309 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 12311, 2023-12-07T10:36:49.310 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.002468153, 2023-12-07T10:36:49.310 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000200 2023-12-07T10:36:49.310 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.310 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 6240, 2023-12-07T10:36:49.310 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 142848 2023-12-07T10:36:49.310 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.310 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2023-12-07T10:36:49.310 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2023-12-07T10:36:49.310 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2023-12-07T10:36:49.311 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.311 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2023-12-07T10:36:49.311 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2023-12-07T10:36:49.311 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2023-12-07T10:36:49.311 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 35648, 2023-12-07T10:36:49.311 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 33.032769073, 2023-12-07T10:36:49.311 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000926637 2023-12-07T10:36:49.311 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:49.311 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.312 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2023-12-07T10:36:49.312 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2023-12-07T10:36:49.312 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2023-12-07T10:36:49.312 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 23966, 2023-12-07T10:36:49.312 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 2.167053189, 2023-12-07T10:36:49.312 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000090421 2023-12-07T10:36:49.312 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:49.312 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.312 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2023-12-07T10:36:49.313 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 71969, 2023-12-07T10:36:49.313 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 68342, 2023-12-07T10:36:49.313 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2023-12-07T10:36:49.313 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 68342, 2023-12-07T10:36:49.313 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 226.986750614, 2023-12-07T10:36:49.313 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.003321336 2023-12-07T10:36:49.313 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.313 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2023-12-07T10:36:49.314 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 3627, 2023-12-07T10:36:49.314 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 2, 2023-12-07T10:36:49.314 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2023-12-07T10:36:49.314 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 11, 2023-12-07T10:36:49.314 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2023-12-07T10:36:49.314 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2023-12-07T10:36:49.314 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 721, 2023-12-07T10:36:49.314 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 69, 2023-12-07T10:36:49.314 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 490, 2023-12-07T10:36:49.315 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 162, 2023-12-07T10:36:49.315 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 624, 2023-12-07T10:36:49.315 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 382, 2023-12-07T10:36:49.315 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 113, 2023-12-07T10:36:49.315 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 113, 2023-12-07T10:36:49.315 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 15, 2023-12-07T10:36:49.315 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 140000, 2023-12-07T10:36:49.315 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 73467, 2023-12-07T10:36:49.315 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 3624, 2023-12-07T10:36:49.316 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 50, 2023-12-07T10:36:49.316 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2023-12-07T10:36:49.316 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2023-12-07T10:36:49.316 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 2714, 2023-12-07T10:36:49.316 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 4282, 2023-12-07T10:36:49.316 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2023-12-07T10:36:49.316 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 5, 2023-12-07T10:36:49.316 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 287, 2023-12-07T10:36:49.317 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 8, 2023-12-07T10:36:49.317 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 366, 2023-12-07T10:36:49.317 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2023-12-07T10:36:49.317 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 44, 2023-12-07T10:36:49.317 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2023-12-07T10:36:49.317 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 65, 2023-12-07T10:36:49.317 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 21133868, 2023-12-07T10:36:49.317 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 4, 2023-12-07T10:36:49.317 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 44, 2023-12-07T10:36:49.318 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2023-12-07T10:36:49.318 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2023-12-07T10:36:49.318 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2023-12-07T10:36:49.318 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 0, 2023-12-07T10:36:49.318 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2023-12-07T10:36:49.318 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 0, 2023-12-07T10:36:49.318 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 44, 2023-12-07T10:36:49.318 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 5, 2023-12-07T10:36:49.319 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 2919, 2023-12-07T10:36:49.319 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 49989, 2023-12-07T10:36:49.319 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2023-12-07T10:36:49.319 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-07T10:36:49.319 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-07T10:36:49.319 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 19050, 2023-12-07T10:36:49.319 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 9604, 2023-12-07T10:36:49.319 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 4331, 2023-12-07T10:36:49.319 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 55230 2023-12-07T10:36:49.320 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.320 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2023-12-07T10:36:49.320 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 21, 2023-12-07T10:36:49.320 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 147, 2023-12-07T10:36:49.320 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 85, 2023-12-07T10:36:49.320 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 184, 2023-12-07T10:36:49.320 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 82, 2023-12-07T10:36:49.320 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2023-12-07T10:36:49.320 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2023-12-07T10:36:49.321 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2023-12-07T10:36:49.321 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 12541, 2023-12-07T10:36:49.321 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 12541, 2023-12-07T10:36:49.321 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2023-12-07T10:36:49.321 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2023-12-07T10:36:49.321 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2023-12-07T10:36:49.321 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2023-12-07T10:36:49.321 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2023-12-07T10:36:49.322 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2023-12-07T10:36:49.322 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2023-12-07T10:36:49.322 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2023-12-07T10:36:49.322 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 6, 2023-12-07T10:36:49.322 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2023-12-07T10:36:49.322 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2023-12-07T10:36:49.322 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2023-12-07T10:36:49.322 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2023-12-07T10:36:49.322 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.323 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2023-12-07T10:36:49.323 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2023-12-07T10:36:49.323 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 68178, 2023-12-07T10:36:49.323 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2023-12-07T10:36:49.323 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2023-12-07T10:36:49.323 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 68178, 2023-12-07T10:36:49.323 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2023-12-07T10:36:49.323 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2023-12-07T10:36:49.324 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 89, 2023-12-07T10:36:49.324 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2023-12-07T10:36:49.324 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2023-12-07T10:36:49.324 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 89, 2023-12-07T10:36:49.324 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 7, 2023-12-07T10:36:49.324 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2023-12-07T10:36:49.324 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2023-12-07T10:36:49.324 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4194304, 2023-12-07T10:36:49.324 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 203199245, 2023-12-07T10:36:49.325 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 0, 2023-12-07T10:36:49.325 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2023-12-07T10:36:49.325 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 18692, 2023-12-07T10:36:49.325 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 125.037337252, 2023-12-07T10:36:49.325 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.006689350 2023-12-07T10:36:49.325 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.325 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 0 2023-12-07T10:36:49.325 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.325 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2023-12-07T10:36:49.326 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 527, 2023-12-07T10:36:49.326 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 13346, 2023-12-07T10:36:49.326 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 12819, 2023-12-07T10:36:49.326 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 72, 2023-12-07T10:36:49.326 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 168, 2023-12-07T10:36:49.326 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 96, 2023-12-07T10:36:49.326 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 722, 2023-12-07T10:36:49.326 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 21776, 2023-12-07T10:36:49.327 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 21054, 2023-12-07T10:36:49.327 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 111, 2023-12-07T10:36:49.327 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 13076, 2023-12-07T10:36:49.327 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 12965, 2023-12-07T10:36:49.327 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 68668, 2023-12-07T10:36:49.327 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 199036 2023-12-07T10:36:49.327 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.327 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2023-12-07T10:36:49.327 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 116753, 2023-12-07T10:36:49.328 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 559, 2023-12-07T10:36:49.328 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 71969, 2023-12-07T10:36:49.328 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 1710, 2023-12-07T10:36:49.328 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 11167, 2023-12-07T10:36:49.328 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2023-12-07T10:36:49.328 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 12545, 2023-12-07T10:36:49.328 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 27.889244978, 2023-12-07T10:36:49.328 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002223136 2023-12-07T10:36:49.329 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.329 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2023-12-07T10:36:49.329 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 644, 2023-12-07T10:36:49.329 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 15.733152227, 2023-12-07T10:36:49.329 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.024430360 2023-12-07T10:36:49.329 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.329 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2023-12-07T10:36:49.329 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.329 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.330 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.330 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.330 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2023-12-07T10:36:49.330 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.330 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.330 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.330 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.330 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2023-12-07T10:36:49.330 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 20761, 2023-12-07T10:36:49.331 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 43.432792858, 2023-12-07T10:36:49.331 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002092037 2023-12-07T10:36:49.331 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.331 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2023-12-07T10:36:49.331 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.331 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.331 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.331 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.332 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2023-12-07T10:36:49.332 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.332 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.332 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.332 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.332 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2023-12-07T10:36:49.332 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.332 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.332 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.333 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.333 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2023-12-07T10:36:49.333 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.333 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.333 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.333 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.333 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2023-12-07T10:36:49.333 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.333 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.334 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.334 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.334 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2023-12-07T10:36:49.334 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.334 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.334 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.334 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.334 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2023-12-07T10:36:49.335 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.335 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.335 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.335 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.335 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2023-12-07T10:36:49.335 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.335 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.335 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.335 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.336 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2023-12-07T10:36:49.336 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.336 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.336 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.336 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.336 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2023-12-07T10:36:49.336 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.336 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.336 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.337 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.337 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2023-12-07T10:36:49.337 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 17982, 2023-12-07T10:36:49.337 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 97.545284102, 2023-12-07T10:36:49.337 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.005424607 2023-12-07T10:36:49.337 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.337 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2023-12-07T10:36:49.337 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 3669, 2023-12-07T10:36:49.337 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 8.588144100, 2023-12-07T10:36:49.338 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002340731 2023-12-07T10:36:49.338 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.338 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2023-12-07T10:36:49.338 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.338 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.338 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.338 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.338 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2023-12-07T10:36:49.338 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.339 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.339 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.339 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.339 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2023-12-07T10:36:49.339 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.339 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.339 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.339 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.339 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2023-12-07T10:36:49.340 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.340 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.340 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.340 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.340 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2023-12-07T10:36:49.340 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.340 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.340 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.340 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.341 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2023-12-07T10:36:49.341 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 8, 2023-12-07T10:36:49.341 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.081809017, 2023-12-07T10:36:49.341 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.010226127 2023-12-07T10:36:49.341 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.341 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2023-12-07T10:36:49.341 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.341 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.342 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.342 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.342 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2023-12-07T10:36:49.342 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 192, 2023-12-07T10:36:49.342 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.166021834, 2023-12-07T10:36:49.342 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000864697 2023-12-07T10:36:49.342 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.342 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2023-12-07T10:36:49.342 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.343 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.343 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.343 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.343 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2023-12-07T10:36:49.343 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.343 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.343 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.343 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.343 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2023-12-07T10:36:49.344 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.344 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.344 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.344 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.344 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2023-12-07T10:36:49.344 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 12541, 2023-12-07T10:36:49.344 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 33.550301498, 2023-12-07T10:36:49.344 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002675249 2023-12-07T10:36:49.344 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.345 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2023-12-07T10:36:49.345 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2023-12-07T10:36:49.345 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2023-12-07T10:36:49.345 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.345 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.345 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.345 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:49.345 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.346 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2023-12-07T10:36:49.346 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 2, 2023-12-07T10:36:49.346 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 3, 2023-12-07T10:36:49.346 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 1, 2023-12-07T10:36:49.346 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 2, 2023-12-07T10:36:49.346 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2023-12-07T10:36:49.346 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 15511, 2023-12-07T10:36:49.346 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 7755, 2023-12-07T10:36:49.346 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 921, 2023-12-07T10:36:49.347 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2023-12-07T10:36:49.347 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.347 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2023-12-07T10:36:49.347 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2023-12-07T10:36:49.347 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2023-12-07T10:36:49.347 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2023-12-07T10:36:49.347 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2023-12-07T10:36:49.347 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2023-12-07T10:36:49.348 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2023-12-07T10:36:49.348 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-07T10:36:49.348 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2023-12-07T10:36:49.348 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-07T10:36:49.348 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2023-12-07T10:36:49.348 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2023-12-07T10:36:49.348 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2023-12-07T10:36:49.348 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-07T10:36:49.348 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2023-12-07T10:36:49.349 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2023-12-07T10:36:49.349 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2023-12-07T10:36:49.349 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2023-12-07T10:36:49.349 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2023-12-07T10:36:49.349 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-07T10:36:49.349 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2023-12-07T10:36:49.349 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-07T10:36:49.349 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2023-12-07T10:36:49.350 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2023-12-07T10:36:49.350 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2023-12-07T10:36:49.350 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2023-12-07T10:36:49.350 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2023-12-07T10:36:49.350 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-07T10:36:49.350 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2023-12-07T10:36:49.350 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2023-12-07T10:36:49.350 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2023-12-07T10:36:49.350 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2023-12-07T10:36:49.351 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2023-12-07T10:36:49.351 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2023-12-07T10:36:49.351 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2023-12-07T10:36:49.351 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2023-12-07T10:36:49.351 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2023-12-07T10:36:49.351 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 500697, 2023-12-07T10:36:49.351 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 523, 2023-12-07T10:36:49.351 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 176, 2023-12-07T10:36:49.352 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 2, 2023-12-07T10:36:49.352 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2023-12-07T10:36:49.352 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2023-12-07T10:36:49.352 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2023-12-07T10:36:49.352 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2023-12-07T10:36:49.352 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2023-12-07T10:36:49.352 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2023-12-07T10:36:49.352 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 9616, 2023-12-07T10:36:49.352 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 185, 2023-12-07T10:36:49.353 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2023-12-07T10:36:49.353 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2023-12-07T10:36:49.353 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2023-12-07T10:36:49.353 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2023-12-07T10:36:49.353 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 2127111, 2023-12-07T10:36:49.353 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 18778, 2023-12-07T10:36:49.353 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2023-12-07T10:36:49.353 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2023-12-07T10:36:49.353 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2023-12-07T10:36:49.354 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2023-12-07T10:36:49.354 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.354 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2023-12-07T10:36:49.354 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2023-12-07T10:36:49.354 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2023-12-07T10:36:49.354 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 45530, 2023-12-07T10:36:49.354 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 201241072, 2023-12-07T10:36:49.354 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 33, 2023-12-07T10:36:49.354 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 45530, 2023-12-07T10:36:49.355 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2023-12-07T10:36:49.355 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 45530, 2023-12-07T10:36:49.355 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 440.931700062, 2023-12-07T10:36:49.355 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.009684421 2023-12-07T10:36:49.355 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.355 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2023-12-07T10:36:49.355 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2023-12-07T10:36:49.355 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 45530, 2023-12-07T10:36:49.356 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 45876 2023-12-07T10:36:49.356 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.356 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 45530, 2023-12-07T10:36:49.356 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 615, 2023-12-07T10:36:49.356 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 44915, 2023-12-07T10:36:49.356 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2023-12-07T10:36:49.356 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2023-12-07T10:36:49.356 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 0, 2023-12-07T10:36:49.356 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 10, 2023-12-07T10:36:49.357 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 563, 2023-12-07T10:36:49.357 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 19255, 2023-12-07T10:36:49.357 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 327, 2023-12-07T10:36:49.357 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2023-12-07T10:36:49.357 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2023-12-07T10:36:49.357 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2023-12-07T10:36:49.357 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2023-12-07T10:36:49.357 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 25141, 2023-12-07T10:36:49.357 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2023-12-07T10:36:49.358 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2023-12-07T10:36:49.358 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2023-12-07T10:36:49.358 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 46, 2023-12-07T10:36:49.358 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 20, 2023-12-07T10:36:49.358 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2023-12-07T10:36:49.358 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2023-12-07T10:36:49.358 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2023-12-07T10:36:49.358 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2023-12-07T10:36:49.359 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2023-12-07T10:36:49.359 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2023-12-07T10:36:49.359 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2023-12-07T10:36:49.359 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2023-12-07T10:36:49.359 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2023-12-07T10:36:49.359 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2023-12-07T10:36:49.359 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2023-12-07T10:36:49.359 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2023-12-07T10:36:49.359 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2023-12-07T10:36:49.360 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2023-12-07T10:36:49.360 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2023-12-07T10:36:49.360 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2023-12-07T10:36:49.360 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2023-12-07T10:36:49.360 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2023-12-07T10:36:49.360 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2023-12-07T10:36:49.360 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2023-12-07T10:36:49.360 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2023-12-07T10:36:49.360 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2023-12-07T10:36:49.360 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2023-12-07T10:36:49.361 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2023-12-07T10:36:49.361 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2023-12-07T10:36:49.361 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2023-12-07T10:36:49.361 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 114, 2023-12-07T10:36:49.361 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2023-12-07T10:36:49.361 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 23, 2023-12-07T10:36:49.361 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 12, 2023-12-07T10:36:49.361 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 12, 2023-12-07T10:36:49.361 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 0, 2023-12-07T10:36:49.362 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2023-12-07T10:36:49.362 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 347, 2023-12-07T10:36:49.362 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 10, 2023-12-07T10:36:49.362 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 157 2023-12-07T10:36:49.362 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.362 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2023-12-07T10:36:49.362 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 1, 2023-12-07T10:36:49.362 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2023-12-07T10:36:49.363 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2023-12-07T10:36:49.363 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2023-12-07T10:36:49.363 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.363 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2023-12-07T10:36:49.363 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 0 2023-12-07T10:36:49.363 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.363 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2023-12-07T10:36:49.363 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2023-12-07T10:36:49.364 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 10, 2023-12-07T10:36:49.364 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2023-12-07T10:36:49.364 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 5, 2023-12-07T10:36:49.364 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 12541, 2023-12-07T10:36:49.364 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2023-12-07T10:36:49.364 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.364 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-07T10:36:49.364 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-12-07T10:36:49.364 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2023-12-07T10:36:49.365 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-12-07T10:36:49.365 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 279301, 2023-12-07T10:36:49.365 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 61072012, 2023-12-07T10:36:49.365 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:49.365 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 279301, 2023-12-07T10:36:49.365 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-12-07T10:36:49.365 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-12-07T10:36:49.365 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 279301, 2023-12-07T10:36:49.365 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 61072012, 2023-12-07T10:36:49.366 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-12-07T10:36:49.366 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.366 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.366 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.366 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:49.366 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.366 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2023-12-07T10:36:49.366 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-12-07T10:36:49.367 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2023-12-07T10:36:49.367 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-12-07T10:36:49.367 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-12-07T10:36:49.367 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-12-07T10:36:49.367 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:49.367 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:49.367 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 45530, 2023-12-07T10:36:49.367 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 202610485, 2023-12-07T10:36:49.368 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 45530, 2023-12-07T10:36:49.368 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 202610485, 2023-12-07T10:36:49.368 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-12-07T10:36:49.368 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.368 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.368 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.368 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:49.368 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.368 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2023-12-07T10:36:49.369 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-12-07T10:36:49.369 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2023-12-07T10:36:49.369 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-12-07T10:36:49.369 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-12-07T10:36:49.369 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-12-07T10:36:49.369 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:49.369 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:49.369 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 45530, 2023-12-07T10:36:49.369 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 45530, 2023-12-07T10:36:49.370 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 45530, 2023-12-07T10:36:49.370 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 45530, 2023-12-07T10:36:49.370 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-12-07T10:36:49.370 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.370 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.370 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.370 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:49.370 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.370 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2023-12-07T10:36:49.371 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-12-07T10:36:49.371 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2023-12-07T10:36:49.371 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-12-07T10:36:49.371 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 12541, 2023-12-07T10:36:49.371 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 1369641, 2023-12-07T10:36:49.371 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:49.371 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 12541, 2023-12-07T10:36:49.371 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-12-07T10:36:49.371 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-12-07T10:36:49.372 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 563, 2023-12-07T10:36:49.372 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 1369641, 2023-12-07T10:36:49.372 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-12-07T10:36:49.372 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.372 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.372 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.372 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:49.372 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:49.373 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x55be4e1b26e0": { 2023-12-07T10:36:49.373 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 9490, 2023-12-07T10:36:49.373 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2023-12-07T10:36:49.373 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-12-07T10:36:49.373 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 68178, 2023-12-07T10:36:49.373 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 199004941, 2023-12-07T10:36:49.373 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:49.373 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 68178, 2023-12-07T10:36:49.373 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-12-07T10:36:49.374 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-12-07T10:36:49.374 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 18692, 2023-12-07T10:36:49.374 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 198995451, 2023-12-07T10:36:49.374 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-12-07T10:36:49.374 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:49.374 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:49.374 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:49.374 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:49.375 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:49.375 INFO:teuthology.orchestra.run.smithi159.stdout:} 2023-12-07T10:36:49.445 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:49 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:36:49.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:49 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:36:49.719 INFO:tasks.check_counter:Daemon mds.f mds.exported=5 2023-12-07T10:36:49.720 INFO:tasks.check_counter:Daemon mds.f mds.imported=8 2023-12-07T10:36:49.720 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=4 2023-12-07T10:36:49.720 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2023-12-07T10:36:49.720 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2023-12-07T10:36:49.720 DEBUG:tasks.check_counter:Getting stats from i 2023-12-07T10:36:49.720 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph daemon mds.i perf dump 2023-12-07T10:36:49.726 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:49 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:36:50.317 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.c/config 2023-12-07T10:36:50.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:50 smithi159 ceph-mon[134726]: pgmap v515: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 13 op/s 2023-12-07T10:36:50.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:50 smithi153 ceph-mon[133625]: pgmap v515: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 13 op/s 2023-12-07T10:36:50.727 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:50 smithi119 ceph-mon[97908]: pgmap v515: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 13 op/s 2023-12-07T10:36:51.504 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2023-12-07T10:36:51.504 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2023-12-07T10:36:51.504 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 2592, 2023-12-07T10:36:51.504 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 2522, 2023-12-07T10:36:51.504 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 57153076, 2023-12-07T10:36:51.505 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 870688, 2023-12-07T10:36:51.505 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 7, 2023-12-07T10:36:51.505 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 3, 2023-12-07T10:36:51.505 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.261751102, 2023-12-07T10:36:51.505 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.063332147, 2023-12-07T10:36:51.505 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.117640099, 2023-12-07T10:36:51.505 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.073849160, 2023-12-07T10:36:51.505 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:51.505 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2522, 2023-12-07T10:36:51.506 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.136353118, 2023-12-07T10:36:51.506 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000054065 2023-12-07T10:36:51.506 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.506 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:51.506 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.506 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.506 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.506 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.507 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 1980624, 2023-12-07T10:36:51.507 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 192704 2023-12-07T10:36:51.507 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.507 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2023-12-07T10:36:51.507 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 922, 2023-12-07T10:36:51.507 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 1158, 2023-12-07T10:36:51.507 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 52584954, 2023-12-07T10:36:51.507 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 853098, 2023-12-07T10:36:51.507 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 9, 2023-12-07T10:36:51.508 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 7, 2023-12-07T10:36:51.508 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.100904509, 2023-12-07T10:36:51.508 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.025877351, 2023-12-07T10:36:51.508 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.043663692, 2023-12-07T10:36:51.508 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.023723107, 2023-12-07T10:36:51.508 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:51.508 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1155, 2023-12-07T10:36:51.508 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.086679091, 2023-12-07T10:36:51.508 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000075046 2023-12-07T10:36:51.509 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.509 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:51.509 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.509 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.509 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.509 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.509 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 320, 2023-12-07T10:36:51.509 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 566912 2023-12-07T10:36:51.509 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.510 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2023-12-07T10:36:51.510 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 1075, 2023-12-07T10:36:51.510 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 1077, 2023-12-07T10:36:51.510 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 50601590, 2023-12-07T10:36:51.510 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 334332, 2023-12-07T10:36:51.510 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 6, 2023-12-07T10:36:51.510 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 5, 2023-12-07T10:36:51.510 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.108308012, 2023-12-07T10:36:51.511 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.026381413, 2023-12-07T10:36:51.511 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.044911840, 2023-12-07T10:36:51.511 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.029601684, 2023-12-07T10:36:51.511 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:51.511 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1075, 2023-12-07T10:36:51.511 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.096456906, 2023-12-07T10:36:51.511 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000089727 2023-12-07T10:36:51.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.511 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:51.512 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.512 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.512 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.512 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.512 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-07T10:36:51.512 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 0 2023-12-07T10:36:51.512 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.512 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2023-12-07T10:36:51.513 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2023-12-07T10:36:51.513 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2023-12-07T10:36:51.513 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.513 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2023-12-07T10:36:51.513 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2023-12-07T10:36:51.513 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2023-12-07T10:36:51.513 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 7082, 2023-12-07T10:36:51.513 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.272550736, 2023-12-07T10:36:51.513 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000038484 2023-12-07T10:36:51.514 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:51.514 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.514 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2023-12-07T10:36:51.514 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2023-12-07T10:36:51.514 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2023-12-07T10:36:51.514 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.514 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.514 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.515 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:51.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.515 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2023-12-07T10:36:51.515 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 0, 2023-12-07T10:36:51.515 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 0, 2023-12-07T10:36:51.515 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2023-12-07T10:36:51.515 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.515 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.515 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.516 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.516 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2023-12-07T10:36:51.516 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 0, 2023-12-07T10:36:51.516 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2023-12-07T10:36:51.516 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2023-12-07T10:36:51.516 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 0, 2023-12-07T10:36:51.516 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2023-12-07T10:36:51.516 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2023-12-07T10:36:51.516 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 16931, 2023-12-07T10:36:51.517 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 11640, 2023-12-07T10:36:51.517 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 5291, 2023-12-07T10:36:51.517 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 0, 2023-12-07T10:36:51.517 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 302, 2023-12-07T10:36:51.517 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 0, 2023-12-07T10:36:51.517 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 0, 2023-12-07T10:36:51.517 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 0, 2023-12-07T10:36:51.517 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 19, 2023-12-07T10:36:51.517 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 0, 2023-12-07T10:36:51.518 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 0, 2023-12-07T10:36:51.518 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 0, 2023-12-07T10:36:51.518 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 0, 2023-12-07T10:36:51.518 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2023-12-07T10:36:51.518 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2023-12-07T10:36:51.518 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 0, 2023-12-07T10:36:51.518 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2023-12-07T10:36:51.518 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2023-12-07T10:36:51.519 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 0, 2023-12-07T10:36:51.519 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 0, 2023-12-07T10:36:51.519 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 0, 2023-12-07T10:36:51.519 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 0, 2023-12-07T10:36:51.519 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2023-12-07T10:36:51.519 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 0, 2023-12-07T10:36:51.519 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2023-12-07T10:36:51.519 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 702, 2023-12-07T10:36:51.519 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 223115917, 2023-12-07T10:36:51.520 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 6, 2023-12-07T10:36:51.520 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 0, 2023-12-07T10:36:51.520 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2023-12-07T10:36:51.520 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2023-12-07T10:36:51.520 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2023-12-07T10:36:51.520 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 0, 2023-12-07T10:36:51.520 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2023-12-07T10:36:51.520 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 0, 2023-12-07T10:36:51.520 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 0, 2023-12-07T10:36:51.521 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 0, 2023-12-07T10:36:51.521 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 0, 2023-12-07T10:36:51.521 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 0, 2023-12-07T10:36:51.521 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2023-12-07T10:36:51.521 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-07T10:36:51.521 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-07T10:36:51.521 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 0, 2023-12-07T10:36:51.521 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 0, 2023-12-07T10:36:51.521 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 0, 2023-12-07T10:36:51.522 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 0 2023-12-07T10:36:51.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.522 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2023-12-07T10:36:51.522 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 0, 2023-12-07T10:36:51.522 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 0, 2023-12-07T10:36:51.522 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 0, 2023-12-07T10:36:51.522 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 0, 2023-12-07T10:36:51.522 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 0, 2023-12-07T10:36:51.523 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2023-12-07T10:36:51.523 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2023-12-07T10:36:51.523 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2023-12-07T10:36:51.523 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 0, 2023-12-07T10:36:51.523 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 0, 2023-12-07T10:36:51.523 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2023-12-07T10:36:51.523 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2023-12-07T10:36:51.523 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2023-12-07T10:36:51.523 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2023-12-07T10:36:51.524 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2023-12-07T10:36:51.524 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2023-12-07T10:36:51.524 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2023-12-07T10:36:51.524 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2023-12-07T10:36:51.524 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 0, 2023-12-07T10:36:51.524 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2023-12-07T10:36:51.524 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2023-12-07T10:36:51.524 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2023-12-07T10:36:51.524 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2023-12-07T10:36:51.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.525 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2023-12-07T10:36:51.525 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2023-12-07T10:36:51.525 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 0, 2023-12-07T10:36:51.525 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2023-12-07T10:36:51.525 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2023-12-07T10:36:51.526 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 73660, 2023-12-07T10:36:51.526 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2023-12-07T10:36:51.526 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2023-12-07T10:36:51.526 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 0, 2023-12-07T10:36:51.526 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2023-12-07T10:36:51.526 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2023-12-07T10:36:51.526 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 2, 2023-12-07T10:36:51.526 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 72, 2023-12-07T10:36:51.527 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2023-12-07T10:36:51.527 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2023-12-07T10:36:51.527 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 159945888, 2023-12-07T10:36:51.527 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 161403066, 2023-12-07T10:36:51.527 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 161394657, 2023-12-07T10:36:51.527 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2023-12-07T10:36:51.527 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.527 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.527 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.528 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 73660 2023-12-07T10:36:51.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.528 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2023-12-07T10:36:51.528 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 920, 2023-12-07T10:36:51.528 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 16975, 2023-12-07T10:36:51.528 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 16055, 2023-12-07T10:36:51.528 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 61, 2023-12-07T10:36:51.528 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 66, 2023-12-07T10:36:51.529 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 5, 2023-12-07T10:36:51.529 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 16931, 2023-12-07T10:36:51.529 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 17280, 2023-12-07T10:36:51.529 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 349, 2023-12-07T10:36:51.529 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 0, 2023-12-07T10:36:51.529 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 0, 2023-12-07T10:36:51.529 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 0, 2023-12-07T10:36:51.529 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 71728, 2023-12-07T10:36:51.530 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 207228 2023-12-07T10:36:51.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.530 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2023-12-07T10:36:51.530 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 0, 2023-12-07T10:36:51.530 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 0, 2023-12-07T10:36:51.530 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 0, 2023-12-07T10:36:51.530 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 0, 2023-12-07T10:36:51.530 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 0, 2023-12-07T10:36:51.530 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2023-12-07T10:36:51.531 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.531 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.531 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.531 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2023-12-07T10:36:51.531 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.531 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.531 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.532 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2023-12-07T10:36:51.532 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.532 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.532 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.532 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2023-12-07T10:36:51.532 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.532 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.532 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.533 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2023-12-07T10:36:51.533 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.533 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.533 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.533 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2023-12-07T10:36:51.533 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.534 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.534 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.534 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2023-12-07T10:36:51.534 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.534 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.534 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.535 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2023-12-07T10:36:51.535 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.535 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.535 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.535 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2023-12-07T10:36:51.535 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.535 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.535 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.536 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2023-12-07T10:36:51.536 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.536 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.536 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.536 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2023-12-07T10:36:51.536 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.536 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.537 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.537 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2023-12-07T10:36:51.537 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.537 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.537 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.537 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2023-12-07T10:36:51.537 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.538 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.538 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.538 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2023-12-07T10:36:51.538 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.538 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.538 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.539 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2023-12-07T10:36:51.539 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.539 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.539 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.539 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2023-12-07T10:36:51.539 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.539 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.539 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.540 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2023-12-07T10:36:51.540 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.540 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.540 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.540 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2023-12-07T10:36:51.540 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.540 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.540 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.541 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2023-12-07T10:36:51.541 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.541 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.541 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.541 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2023-12-07T10:36:51.541 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.542 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.542 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.542 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2023-12-07T10:36:51.542 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.542 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.542 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.542 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2023-12-07T10:36:51.543 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.543 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.543 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.543 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2023-12-07T10:36:51.543 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.543 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.543 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.544 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2023-12-07T10:36:51.544 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.544 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.544 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.544 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2023-12-07T10:36:51.544 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.544 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.544 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.545 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2023-12-07T10:36:51.545 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.545 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.545 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.545 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2023-12-07T10:36:51.545 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.546 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.546 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.546 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2023-12-07T10:36:51.546 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.546 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.546 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.546 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2023-12-07T10:36:51.547 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.547 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.547 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.547 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2023-12-07T10:36:51.547 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2023-12-07T10:36:51.547 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2023-12-07T10:36:51.547 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.548 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.548 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.548 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:51.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.548 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2023-12-07T10:36:51.548 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 2, 2023-12-07T10:36:51.548 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 6, 2023-12-07T10:36:51.548 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 4, 2023-12-07T10:36:51.548 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 4, 2023-12-07T10:36:51.549 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2023-12-07T10:36:51.549 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 0, 2023-12-07T10:36:51.549 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 0, 2023-12-07T10:36:51.549 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 4720, 2023-12-07T10:36:51.549 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2023-12-07T10:36:51.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.549 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2023-12-07T10:36:51.549 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2023-12-07T10:36:51.549 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2023-12-07T10:36:51.550 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2023-12-07T10:36:51.550 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2023-12-07T10:36:51.550 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2023-12-07T10:36:51.550 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2023-12-07T10:36:51.550 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-07T10:36:51.550 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2023-12-07T10:36:51.550 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-07T10:36:51.550 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2023-12-07T10:36:51.550 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2023-12-07T10:36:51.551 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2023-12-07T10:36:51.551 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-07T10:36:51.551 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2023-12-07T10:36:51.551 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2023-12-07T10:36:51.551 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2023-12-07T10:36:51.551 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2023-12-07T10:36:51.551 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2023-12-07T10:36:51.551 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-07T10:36:51.552 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2023-12-07T10:36:51.552 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-07T10:36:51.552 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2023-12-07T10:36:51.552 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2023-12-07T10:36:51.552 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2023-12-07T10:36:51.552 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2023-12-07T10:36:51.552 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2023-12-07T10:36:51.552 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-07T10:36:51.552 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2023-12-07T10:36:51.553 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2023-12-07T10:36:51.553 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2023-12-07T10:36:51.553 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2023-12-07T10:36:51.553 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2023-12-07T10:36:51.553 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2023-12-07T10:36:51.553 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2023-12-07T10:36:51.553 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2023-12-07T10:36:51.553 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2023-12-07T10:36:51.554 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 276899, 2023-12-07T10:36:51.554 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 1117, 2023-12-07T10:36:51.554 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2023-12-07T10:36:51.554 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2023-12-07T10:36:51.554 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2023-12-07T10:36:51.554 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2023-12-07T10:36:51.554 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2023-12-07T10:36:51.554 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2023-12-07T10:36:51.555 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2023-12-07T10:36:51.555 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2023-12-07T10:36:51.555 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 9616, 2023-12-07T10:36:51.555 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 185, 2023-12-07T10:36:51.555 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2023-12-07T10:36:51.555 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2023-12-07T10:36:51.555 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2023-12-07T10:36:51.555 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2023-12-07T10:36:51.555 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 14248459, 2023-12-07T10:36:51.556 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 91728, 2023-12-07T10:36:51.556 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2023-12-07T10:36:51.556 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2023-12-07T10:36:51.556 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2023-12-07T10:36:51.556 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2023-12-07T10:36:51.556 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.556 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2023-12-07T10:36:51.556 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2023-12-07T10:36:51.557 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2023-12-07T10:36:51.557 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 4175, 2023-12-07T10:36:51.557 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 26, 2023-12-07T10:36:51.557 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 4, 2023-12-07T10:36:51.557 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 4175, 2023-12-07T10:36:51.557 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2023-12-07T10:36:51.557 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 4175, 2023-12-07T10:36:51.557 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 5.102903831, 2023-12-07T10:36:51.557 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001222252 2023-12-07T10:36:51.558 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.558 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2023-12-07T10:36:51.558 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2023-12-07T10:36:51.558 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 4175, 2023-12-07T10:36:51.558 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 4176 2023-12-07T10:36:51.558 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.558 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 4175, 2023-12-07T10:36:51.558 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 4175, 2023-12-07T10:36:51.558 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 0, 2023-12-07T10:36:51.559 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2023-12-07T10:36:51.559 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2023-12-07T10:36:51.559 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 2268, 2023-12-07T10:36:51.559 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 0, 2023-12-07T10:36:51.559 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 1904, 2023-12-07T10:36:51.559 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 0, 2023-12-07T10:36:51.559 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 0, 2023-12-07T10:36:51.559 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2023-12-07T10:36:51.559 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2023-12-07T10:36:51.560 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2023-12-07T10:36:51.560 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2023-12-07T10:36:51.560 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 0, 2023-12-07T10:36:51.560 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2023-12-07T10:36:51.560 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2023-12-07T10:36:51.560 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2023-12-07T10:36:51.560 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 2, 2023-12-07T10:36:51.560 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 0, 2023-12-07T10:36:51.560 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2023-12-07T10:36:51.561 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2023-12-07T10:36:51.561 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2023-12-07T10:36:51.561 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2023-12-07T10:36:51.561 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2023-12-07T10:36:51.561 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2023-12-07T10:36:51.561 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2023-12-07T10:36:51.561 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2023-12-07T10:36:51.561 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2023-12-07T10:36:51.562 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2023-12-07T10:36:51.562 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2023-12-07T10:36:51.562 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2023-12-07T10:36:51.562 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2023-12-07T10:36:51.562 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2023-12-07T10:36:51.562 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2023-12-07T10:36:51.562 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2023-12-07T10:36:51.562 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2023-12-07T10:36:51.562 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2023-12-07T10:36:51.563 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2023-12-07T10:36:51.563 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2023-12-07T10:36:51.563 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2023-12-07T10:36:51.563 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2023-12-07T10:36:51.563 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2023-12-07T10:36:51.563 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2023-12-07T10:36:51.563 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2023-12-07T10:36:51.563 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2023-12-07T10:36:51.564 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 114, 2023-12-07T10:36:51.564 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2023-12-07T10:36:51.564 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 23, 2023-12-07T10:36:51.564 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 11, 2023-12-07T10:36:51.564 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 11, 2023-12-07T10:36:51.564 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 0, 2023-12-07T10:36:51.564 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2023-12-07T10:36:51.564 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 0, 2023-12-07T10:36:51.565 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 2, 2023-12-07T10:36:51.565 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 0 2023-12-07T10:36:51.565 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.565 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2023-12-07T10:36:51.565 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 0, 2023-12-07T10:36:51.565 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2023-12-07T10:36:51.565 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2023-12-07T10:36:51.565 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2023-12-07T10:36:51.566 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.566 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2023-12-07T10:36:51.566 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 0 2023-12-07T10:36:51.566 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.566 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2023-12-07T10:36:51.566 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2023-12-07T10:36:51.566 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 0, 2023-12-07T10:36:51.566 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2023-12-07T10:36:51.566 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 0, 2023-12-07T10:36:51.567 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 0, 2023-12-07T10:36:51.567 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2023-12-07T10:36:51.567 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.567 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-07T10:36:51.567 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-12-07T10:36:51.567 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2023-12-07T10:36:51.567 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-12-07T10:36:51.567 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 4585, 2023-12-07T10:36:51.568 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 159903442, 2023-12-07T10:36:51.568 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:51.568 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 4585, 2023-12-07T10:36:51.568 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-12-07T10:36:51.568 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-12-07T10:36:51.568 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 4585, 2023-12-07T10:36:51.568 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 159903442, 2023-12-07T10:36:51.568 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-12-07T10:36:51.569 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.569 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.569 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.569 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:51.569 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.569 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2023-12-07T10:36:51.569 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-12-07T10:36:51.569 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2023-12-07T10:36:51.570 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-12-07T10:36:51.570 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-12-07T10:36:51.570 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-12-07T10:36:51.570 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:51.570 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:51.570 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 4175, 2023-12-07T10:36:51.570 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 157209150, 2023-12-07T10:36:51.570 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 4175, 2023-12-07T10:36:51.570 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 157209150, 2023-12-07T10:36:51.571 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-12-07T10:36:51.571 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.571 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.571 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.571 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:51.571 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.571 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2023-12-07T10:36:51.571 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-12-07T10:36:51.572 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2023-12-07T10:36:51.572 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-12-07T10:36:51.572 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-12-07T10:36:51.572 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-12-07T10:36:51.572 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:51.572 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:51.572 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 4175, 2023-12-07T10:36:51.572 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 4175, 2023-12-07T10:36:51.572 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 4175, 2023-12-07T10:36:51.573 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 4175, 2023-12-07T10:36:51.573 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-12-07T10:36:51.573 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.573 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.573 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.573 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:51.573 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.573 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2023-12-07T10:36:51.573 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-12-07T10:36:51.574 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2023-12-07T10:36:51.574 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-12-07T10:36:51.574 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-12-07T10:36:51.574 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-12-07T10:36:51.574 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:51.574 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:51.574 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-12-07T10:36:51.574 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-12-07T10:36:51.575 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2023-12-07T10:36:51.575 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2023-12-07T10:36:51.575 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-12-07T10:36:51.575 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.575 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.575 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.575 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:51.575 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:51.575 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x562df3b2a960": { 2023-12-07T10:36:51.576 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-12-07T10:36:51.576 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2023-12-07T10:36:51.576 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-12-07T10:36:51.576 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-12-07T10:36:51.576 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-12-07T10:36:51.576 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:51.576 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:51.576 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-12-07T10:36:51.577 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-12-07T10:36:51.577 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2023-12-07T10:36:51.577 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2023-12-07T10:36:51.577 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-12-07T10:36:51.577 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:51.577 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:51.577 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:51.577 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:51.577 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:51.578 INFO:teuthology.orchestra.run.smithi159.stdout:} 2023-12-07T10:36:51.873 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2023-12-07T10:36:51.873 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2023-12-07T10:36:51.873 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2023-12-07T10:36:51.873 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2023-12-07T10:36:51.873 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2023-12-07T10:36:51.873 DEBUG:tasks.check_counter:Getting stats from l 2023-12-07T10:36:51.874 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 shell --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 -- ceph daemon mds.l perf dump 2023-12-07T10:36:52.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:52 smithi159 ceph-mon[134726]: pgmap v516: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-12-07T10:36:52.469 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/mon.c/config 2023-12-07T10:36:52.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:52 smithi119 ceph-mon[97908]: pgmap v516: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-12-07T10:36:52.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:52 smithi153 ceph-mon[133625]: pgmap v516: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-12-07T10:36:53.645 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2023-12-07T10:36:53.645 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2023-12-07T10:36:53.645 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 2132, 2023-12-07T10:36:53.645 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 2134, 2023-12-07T10:36:53.645 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 59359561, 2023-12-07T10:36:53.646 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 663053, 2023-12-07T10:36:53.646 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 8, 2023-12-07T10:36:53.646 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 4, 2023-12-07T10:36:53.646 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.202308000, 2023-12-07T10:36:53.646 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.051258474, 2023-12-07T10:36:53.646 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.072216644, 2023-12-07T10:36:53.646 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.061861902, 2023-12-07T10:36:53.646 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:53.646 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2132, 2023-12-07T10:36:53.647 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.087416932, 2023-12-07T10:36:53.647 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000041002 2023-12-07T10:36:53.647 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.647 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:53.647 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.647 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.647 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.647 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.648 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-07T10:36:53.648 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 0 2023-12-07T10:36:53.648 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.648 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2023-12-07T10:36:53.648 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 1768, 2023-12-07T10:36:53.648 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 1706, 2023-12-07T10:36:53.648 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 81998769, 2023-12-07T10:36:53.648 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 619179, 2023-12-07T10:36:53.648 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 6, 2023-12-07T10:36:53.648 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 5, 2023-12-07T10:36:53.649 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.181408580, 2023-12-07T10:36:53.649 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.047842592, 2023-12-07T10:36:53.649 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.094993578, 2023-12-07T10:36:53.649 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.041362748, 2023-12-07T10:36:53.649 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:53.649 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1705, 2023-12-07T10:36:53.649 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.075315002, 2023-12-07T10:36:53.649 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000044173 2023-12-07T10:36:53.650 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.650 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:53.650 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.650 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.650 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.650 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.650 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 1957792, 2023-12-07T10:36:53.650 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 195840 2023-12-07T10:36:53.650 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.651 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2023-12-07T10:36:53.651 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 575, 2023-12-07T10:36:53.651 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 813, 2023-12-07T10:36:53.651 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 60741151, 2023-12-07T10:36:53.651 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 745023, 2023-12-07T10:36:53.651 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 8, 2023-12-07T10:36:53.651 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 6, 2023-12-07T10:36:53.651 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.093517572, 2023-12-07T10:36:53.651 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.017687944, 2023-12-07T10:36:53.652 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.054057764, 2023-12-07T10:36:53.652 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.015363715, 2023-12-07T10:36:53.652 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2023-12-07T10:36:53.652 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 811, 2023-12-07T10:36:53.652 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.048521834, 2023-12-07T10:36:53.652 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000059829 2023-12-07T10:36:53.652 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.652 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2023-12-07T10:36:53.652 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.653 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.653 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.653 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.653 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-12-07T10:36:53.653 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 568064 2023-12-07T10:36:53.653 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.653 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2023-12-07T10:36:53.653 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2023-12-07T10:36:53.653 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2023-12-07T10:36:53.654 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.654 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2023-12-07T10:36:53.654 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2023-12-07T10:36:53.654 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2023-12-07T10:36:53.654 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 6938, 2023-12-07T10:36:53.654 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.161179118, 2023-12-07T10:36:53.654 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000023231 2023-12-07T10:36:53.654 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:53.654 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.655 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2023-12-07T10:36:53.655 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2023-12-07T10:36:53.655 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2023-12-07T10:36:53.655 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.655 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.655 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.655 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:53.655 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.656 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2023-12-07T10:36:53.656 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 0, 2023-12-07T10:36:53.656 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 0, 2023-12-07T10:36:53.656 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2023-12-07T10:36:53.656 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.656 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.656 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.656 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.656 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2023-12-07T10:36:53.657 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 0, 2023-12-07T10:36:53.657 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2023-12-07T10:36:53.657 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2023-12-07T10:36:53.657 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 0, 2023-12-07T10:36:53.657 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2023-12-07T10:36:53.657 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2023-12-07T10:36:53.657 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 13172, 2023-12-07T10:36:53.657 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 18, 2023-12-07T10:36:53.657 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 13154, 2023-12-07T10:36:53.657 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 0, 2023-12-07T10:36:53.658 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 13147, 2023-12-07T10:36:53.658 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 0, 2023-12-07T10:36:53.658 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 0, 2023-12-07T10:36:53.658 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 0, 2023-12-07T10:36:53.658 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 7, 2023-12-07T10:36:53.658 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 0, 2023-12-07T10:36:53.658 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 0, 2023-12-07T10:36:53.658 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 0, 2023-12-07T10:36:53.658 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 0, 2023-12-07T10:36:53.659 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2023-12-07T10:36:53.659 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2023-12-07T10:36:53.659 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 0, 2023-12-07T10:36:53.659 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2023-12-07T10:36:53.659 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2023-12-07T10:36:53.659 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 0, 2023-12-07T10:36:53.659 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 0, 2023-12-07T10:36:53.659 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 0, 2023-12-07T10:36:53.659 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 0, 2023-12-07T10:36:53.659 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2023-12-07T10:36:53.659 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 0, 2023-12-07T10:36:53.660 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2023-12-07T10:36:53.660 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 0, 2023-12-07T10:36:53.660 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 0, 2023-12-07T10:36:53.660 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 0, 2023-12-07T10:36:53.660 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 0, 2023-12-07T10:36:53.660 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2023-12-07T10:36:53.660 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2023-12-07T10:36:53.660 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2023-12-07T10:36:53.660 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 0, 2023-12-07T10:36:53.660 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2023-12-07T10:36:53.661 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 0, 2023-12-07T10:36:53.661 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 0, 2023-12-07T10:36:53.661 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 0, 2023-12-07T10:36:53.661 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 0, 2023-12-07T10:36:53.661 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 0, 2023-12-07T10:36:53.661 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2023-12-07T10:36:53.661 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-07T10:36:53.661 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-07T10:36:53.661 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 0, 2023-12-07T10:36:53.662 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 0, 2023-12-07T10:36:53.662 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 0, 2023-12-07T10:36:53.662 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 0 2023-12-07T10:36:53.662 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.662 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2023-12-07T10:36:53.662 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 0, 2023-12-07T10:36:53.662 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 0, 2023-12-07T10:36:53.662 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 0, 2023-12-07T10:36:53.663 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 0, 2023-12-07T10:36:53.663 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 0, 2023-12-07T10:36:53.663 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2023-12-07T10:36:53.663 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2023-12-07T10:36:53.663 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2023-12-07T10:36:53.663 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 0, 2023-12-07T10:36:53.663 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 0, 2023-12-07T10:36:53.663 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2023-12-07T10:36:53.663 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2023-12-07T10:36:53.664 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2023-12-07T10:36:53.664 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2023-12-07T10:36:53.664 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2023-12-07T10:36:53.664 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2023-12-07T10:36:53.664 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2023-12-07T10:36:53.664 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2023-12-07T10:36:53.664 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 0, 2023-12-07T10:36:53.664 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2023-12-07T10:36:53.664 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2023-12-07T10:36:53.665 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2023-12-07T10:36:53.665 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2023-12-07T10:36:53.665 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.665 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2023-12-07T10:36:53.665 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2023-12-07T10:36:53.665 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 0, 2023-12-07T10:36:53.665 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2023-12-07T10:36:53.665 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2023-12-07T10:36:53.665 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 68177, 2023-12-07T10:36:53.666 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2023-12-07T10:36:53.666 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2023-12-07T10:36:53.666 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 0, 2023-12-07T10:36:53.666 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2023-12-07T10:36:53.666 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2023-12-07T10:36:53.666 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 89, 2023-12-07T10:36:53.666 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 7, 2023-12-07T10:36:53.666 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2023-12-07T10:36:53.666 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2023-12-07T10:36:53.667 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4194304, 2023-12-07T10:36:53.667 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 203189755, 2023-12-07T10:36:53.667 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 203187408, 2023-12-07T10:36:53.667 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2023-12-07T10:36:53.667 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.667 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.667 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.667 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.667 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 68177 2023-12-07T10:36:53.667 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.668 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2023-12-07T10:36:53.668 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 478, 2023-12-07T10:36:53.668 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 13266, 2023-12-07T10:36:53.668 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 12788, 2023-12-07T10:36:53.668 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 45, 2023-12-07T10:36:53.668 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 58, 2023-12-07T10:36:53.668 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 13, 2023-12-07T10:36:53.668 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 13172, 2023-12-07T10:36:53.668 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 13461, 2023-12-07T10:36:53.668 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 289, 2023-12-07T10:36:53.669 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 0, 2023-12-07T10:36:53.669 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 0, 2023-12-07T10:36:53.669 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 0, 2023-12-07T10:36:53.669 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 63868, 2023-12-07T10:36:53.669 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 207228 2023-12-07T10:36:53.669 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.669 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2023-12-07T10:36:53.669 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 0, 2023-12-07T10:36:53.669 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 0, 2023-12-07T10:36:53.670 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 0, 2023-12-07T10:36:53.670 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 0, 2023-12-07T10:36:53.670 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 0, 2023-12-07T10:36:53.670 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2023-12-07T10:36:53.670 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.670 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.670 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.670 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.671 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2023-12-07T10:36:53.671 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.671 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.671 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.671 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.671 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2023-12-07T10:36:53.671 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.671 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.671 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.671 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.672 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2023-12-07T10:36:53.672 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.672 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.672 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.672 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.672 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2023-12-07T10:36:53.672 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.672 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.672 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.673 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.673 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2023-12-07T10:36:53.673 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.673 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.673 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.673 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.673 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2023-12-07T10:36:53.673 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.673 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.674 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.674 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.674 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2023-12-07T10:36:53.674 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.674 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.674 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.674 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.674 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2023-12-07T10:36:53.675 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.675 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.675 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.675 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.675 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2023-12-07T10:36:53.675 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.675 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.675 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.675 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.675 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2023-12-07T10:36:53.676 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.676 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.676 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.676 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.676 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2023-12-07T10:36:53.676 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.676 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.676 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.676 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.677 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2023-12-07T10:36:53.677 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.677 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.677 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.677 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.677 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2023-12-07T10:36:53.677 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.677 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.677 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.678 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.678 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2023-12-07T10:36:53.678 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.678 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.678 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.678 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.678 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2023-12-07T10:36:53.678 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.679 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.679 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.679 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.679 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2023-12-07T10:36:53.679 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.679 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.679 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.679 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.679 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2023-12-07T10:36:53.680 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.680 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.680 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.680 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.680 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2023-12-07T10:36:53.680 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.680 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.680 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.680 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.680 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2023-12-07T10:36:53.681 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.681 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.681 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.681 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.681 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2023-12-07T10:36:53.681 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.681 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.681 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.681 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.682 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2023-12-07T10:36:53.682 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.682 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.682 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.682 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.682 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2023-12-07T10:36:53.682 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.682 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.682 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.683 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.683 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2023-12-07T10:36:53.683 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.683 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.683 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.683 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.683 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2023-12-07T10:36:53.683 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.683 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.683 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.684 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.684 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2023-12-07T10:36:53.684 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.684 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.684 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.684 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.684 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2023-12-07T10:36:53.684 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.684 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.684 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.685 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.685 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2023-12-07T10:36:53.685 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.685 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.685 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.685 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.685 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2023-12-07T10:36:53.685 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.685 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.685 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.686 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.686 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2023-12-07T10:36:53.686 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2023-12-07T10:36:53.686 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2023-12-07T10:36:53.686 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.686 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.686 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.686 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:53.686 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.687 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2023-12-07T10:36:53.687 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 2, 2023-12-07T10:36:53.687 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 3, 2023-12-07T10:36:53.687 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 1, 2023-12-07T10:36:53.687 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 3, 2023-12-07T10:36:53.687 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2023-12-07T10:36:53.687 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 0, 2023-12-07T10:36:53.687 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 0, 2023-12-07T10:36:53.687 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 3786, 2023-12-07T10:36:53.687 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2023-12-07T10:36:53.688 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.688 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2023-12-07T10:36:53.688 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2023-12-07T10:36:53.688 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2023-12-07T10:36:53.688 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2023-12-07T10:36:53.688 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2023-12-07T10:36:53.688 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2023-12-07T10:36:53.688 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2023-12-07T10:36:53.688 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-07T10:36:53.688 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2023-12-07T10:36:53.689 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-07T10:36:53.689 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2023-12-07T10:36:53.689 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2023-12-07T10:36:53.689 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2023-12-07T10:36:53.689 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-07T10:36:53.689 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2023-12-07T10:36:53.689 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2023-12-07T10:36:53.689 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2023-12-07T10:36:53.689 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2023-12-07T10:36:53.690 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2023-12-07T10:36:53.690 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-07T10:36:53.690 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2023-12-07T10:36:53.690 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-07T10:36:53.690 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2023-12-07T10:36:53.690 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2023-12-07T10:36:53.690 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2023-12-07T10:36:53.690 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2023-12-07T10:36:53.690 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2023-12-07T10:36:53.691 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-07T10:36:53.691 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2023-12-07T10:36:53.691 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2023-12-07T10:36:53.691 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2023-12-07T10:36:53.691 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2023-12-07T10:36:53.691 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2023-12-07T10:36:53.691 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2023-12-07T10:36:53.691 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2023-12-07T10:36:53.691 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2023-12-07T10:36:53.692 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2023-12-07T10:36:53.692 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 260647, 2023-12-07T10:36:53.692 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 640, 2023-12-07T10:36:53.692 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2023-12-07T10:36:53.692 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2023-12-07T10:36:53.692 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2023-12-07T10:36:53.692 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2023-12-07T10:36:53.692 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2023-12-07T10:36:53.692 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2023-12-07T10:36:53.693 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2023-12-07T10:36:53.693 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2023-12-07T10:36:53.693 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 9616, 2023-12-07T10:36:53.693 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 185, 2023-12-07T10:36:53.693 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2023-12-07T10:36:53.693 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2023-12-07T10:36:53.693 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2023-12-07T10:36:53.693 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2023-12-07T10:36:53.693 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 10607359, 2023-12-07T10:36:53.694 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 65974, 2023-12-07T10:36:53.694 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2023-12-07T10:36:53.694 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2023-12-07T10:36:53.694 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2023-12-07T10:36:53.694 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2023-12-07T10:36:53.694 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.694 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2023-12-07T10:36:53.694 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2023-12-07T10:36:53.694 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2023-12-07T10:36:53.695 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 4062, 2023-12-07T10:36:53.695 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 21, 2023-12-07T10:36:53.695 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 3, 2023-12-07T10:36:53.695 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 4062, 2023-12-07T10:36:53.695 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2023-12-07T10:36:53.695 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 4062, 2023-12-07T10:36:53.695 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 10.359805124, 2023-12-07T10:36:53.695 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002550419 2023-12-07T10:36:53.695 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.696 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2023-12-07T10:36:53.696 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2023-12-07T10:36:53.696 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 4062, 2023-12-07T10:36:53.696 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 4063 2023-12-07T10:36:53.696 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.696 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 4062, 2023-12-07T10:36:53.696 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 4062, 2023-12-07T10:36:53.696 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 0, 2023-12-07T10:36:53.696 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2023-12-07T10:36:53.697 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2023-12-07T10:36:53.697 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 2165, 2023-12-07T10:36:53.697 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 0, 2023-12-07T10:36:53.697 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 1895, 2023-12-07T10:36:53.697 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 0, 2023-12-07T10:36:53.697 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 0, 2023-12-07T10:36:53.697 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2023-12-07T10:36:53.697 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2023-12-07T10:36:53.697 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2023-12-07T10:36:53.698 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2023-12-07T10:36:53.698 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 0, 2023-12-07T10:36:53.698 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2023-12-07T10:36:53.698 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2023-12-07T10:36:53.698 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2023-12-07T10:36:53.698 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 1, 2023-12-07T10:36:53.698 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 0, 2023-12-07T10:36:53.698 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2023-12-07T10:36:53.699 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2023-12-07T10:36:53.699 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2023-12-07T10:36:53.699 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2023-12-07T10:36:53.699 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2023-12-07T10:36:53.699 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2023-12-07T10:36:53.699 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2023-12-07T10:36:53.699 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2023-12-07T10:36:53.699 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2023-12-07T10:36:53.699 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2023-12-07T10:36:53.700 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2023-12-07T10:36:53.700 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2023-12-07T10:36:53.700 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2023-12-07T10:36:53.700 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2023-12-07T10:36:53.700 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2023-12-07T10:36:53.700 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2023-12-07T10:36:53.700 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2023-12-07T10:36:53.700 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2023-12-07T10:36:53.700 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2023-12-07T10:36:53.700 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2023-12-07T10:36:53.701 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2023-12-07T10:36:53.701 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2023-12-07T10:36:53.701 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2023-12-07T10:36:53.701 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2023-12-07T10:36:53.701 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2023-12-07T10:36:53.701 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2023-12-07T10:36:53.701 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 114, 2023-12-07T10:36:53.701 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2023-12-07T10:36:53.701 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 23, 2023-12-07T10:36:53.702 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 11, 2023-12-07T10:36:53.702 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 11, 2023-12-07T10:36:53.702 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 0, 2023-12-07T10:36:53.702 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2023-12-07T10:36:53.702 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 0, 2023-12-07T10:36:53.702 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 2, 2023-12-07T10:36:53.702 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 0 2023-12-07T10:36:53.702 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.702 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2023-12-07T10:36:53.703 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 0, 2023-12-07T10:36:53.703 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2023-12-07T10:36:53.703 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2023-12-07T10:36:53.703 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2023-12-07T10:36:53.703 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.703 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2023-12-07T10:36:53.703 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 0 2023-12-07T10:36:53.703 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.703 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2023-12-07T10:36:53.704 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2023-12-07T10:36:53.704 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 0, 2023-12-07T10:36:53.704 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2023-12-07T10:36:53.704 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 0, 2023-12-07T10:36:53.704 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 0, 2023-12-07T10:36:53.704 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2023-12-07T10:36:53.704 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.704 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-07T10:36:53.704 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-12-07T10:36:53.705 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2023-12-07T10:36:53.705 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-12-07T10:36:53.705 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 4471, 2023-12-07T10:36:53.705 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 201673549, 2023-12-07T10:36:53.705 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:53.705 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 4471, 2023-12-07T10:36:53.705 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-12-07T10:36:53.705 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-12-07T10:36:53.706 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 4471, 2023-12-07T10:36:53.706 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 201673549, 2023-12-07T10:36:53.706 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-12-07T10:36:53.706 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.706 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.706 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.706 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:53.706 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.706 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2023-12-07T10:36:53.707 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-12-07T10:36:53.707 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2023-12-07T10:36:53.707 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-12-07T10:36:53.707 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-12-07T10:36:53.707 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-12-07T10:36:53.707 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:53.707 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:53.707 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 4062, 2023-12-07T10:36:53.707 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 199016447, 2023-12-07T10:36:53.708 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 4062, 2023-12-07T10:36:53.708 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 199016447, 2023-12-07T10:36:53.708 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-12-07T10:36:53.708 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.708 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.708 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.708 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:53.708 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.708 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2023-12-07T10:36:53.708 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-12-07T10:36:53.709 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2023-12-07T10:36:53.709 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-12-07T10:36:53.709 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-12-07T10:36:53.709 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-12-07T10:36:53.709 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:53.709 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:53.709 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 4062, 2023-12-07T10:36:53.709 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 4062, 2023-12-07T10:36:53.710 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 4062, 2023-12-07T10:36:53.710 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 4062, 2023-12-07T10:36:53.710 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-12-07T10:36:53.710 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.710 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.710 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.710 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:53.710 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.710 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2023-12-07T10:36:53.711 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-12-07T10:36:53.711 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2023-12-07T10:36:53.711 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-12-07T10:36:53.711 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-12-07T10:36:53.711 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-12-07T10:36:53.711 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:53.711 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:53.711 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-12-07T10:36:53.711 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-12-07T10:36:53.712 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2023-12-07T10:36:53.712 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2023-12-07T10:36:53.712 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-12-07T10:36:53.712 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.712 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.712 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.712 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:53.712 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2023-12-07T10:36:53.712 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x55c729e7c820": { 2023-12-07T10:36:53.712 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2023-12-07T10:36:53.713 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2023-12-07T10:36:53.713 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2023-12-07T10:36:53.713 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2023-12-07T10:36:53.713 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2023-12-07T10:36:53.713 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2023-12-07T10:36:53.713 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2023-12-07T10:36:53.713 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2023-12-07T10:36:53.713 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2023-12-07T10:36:53.713 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2023-12-07T10:36:53.714 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2023-12-07T10:36:53.714 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2023-12-07T10:36:53.714 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2023-12-07T10:36:53.714 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2023-12-07T10:36:53.714 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2023-12-07T10:36:53.714 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:53.714 INFO:teuthology.orchestra.run.smithi159.stdout: } 2023-12-07T10:36:53.714 INFO:teuthology.orchestra.run.smithi159.stdout:} 2023-12-07T10:36:54.033 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2023-12-07T10:36:54.033 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2023-12-07T10:36:54.033 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2023-12-07T10:36:54.033 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2023-12-07T10:36:54.033 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2023-12-07T10:36:54.034 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-12-07T10:36:54.044 INFO:tasks.kclient:Unmounting kernel clients... 2023-12-07T10:36:54.044 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-12-07T10:36:54.044 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-07T10:36:54.113 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-12-07T10:36:54.113 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-07T10:36:54.159 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-07T10:36:54.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.404+0000 7f4c837ff700 1 -- 172.21.15.119:0/3517148462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c7c1014e0 msgr2=0x7f4c7c101960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:54.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.404+0000 7f4c837ff700 1 --2- 172.21.15.119:0/3517148462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c7c1014e0 0x7f4c7c101960 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0099c0 tx=0x7f4c6c009cd0 comp rx=0 tx=0).stop 2023-12-07T10:36:54.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.404+0000 7f4c837ff700 1 -- 172.21.15.119:0/3517148462 shutdown_connections 2023-12-07T10:36:54.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.404+0000 7f4c837ff700 1 --2- 172.21.15.119:0/3517148462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c7c101ea0 0x7f4c7c10e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:54.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.404+0000 7f4c837ff700 1 --2- 172.21.15.119:0/3517148462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c7c1075a0 0x7f4c7c107980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:54.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.404+0000 7f4c837ff700 1 --2- 172.21.15.119:0/3517148462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c7c1014e0 0x7f4c7c101960 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:54.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.404+0000 7f4c837ff700 1 -- 172.21.15.119:0/3517148462 >> 172.21.15.119:0/3517148462 conn(0x7f4c7c0fcf90 msgr2=0x7f4c7c0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:54.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.404+0000 7f4c837ff700 1 -- 172.21.15.119:0/3517148462 shutdown_connections 2023-12-07T10:36:54.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.404+0000 7f4c837ff700 1 -- 172.21.15.119:0/3517148462 wait complete. 2023-12-07T10:36:54.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.405+0000 7f4c837ff700 1 Processor -- start 2023-12-07T10:36:54.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.405+0000 7f4c837ff700 1 -- start start 2023-12-07T10:36:54.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.406+0000 7f4c837ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c7c1014e0 0x7f4c7c19d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:54.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.406+0000 7f4c837ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c7c101ea0 0x7f4c7c19db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:54.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.406+0000 7f4c837ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c7c1075a0 0x7f4c7c197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:54.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.406+0000 7f4c837ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c197d90 con 0x7f4c7c1014e0 2023-12-07T10:36:54.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.406+0000 7f4c837ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c197f00 con 0x7f4c7c1075a0 2023-12-07T10:36:54.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.406+0000 7f4c837ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c1980c0 con 0x7f4c7c101ea0 2023-12-07T10:36:54.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.406+0000 7f4c8159b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c7c1014e0 0x7f4c7c19d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:54.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.406+0000 7f4c80d9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c7c101ea0 0x7f4c7c19db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:54.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.406+0000 7f4c8159b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c7c1014e0 0x7f4c7c19d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58052/0 (socket says 172.21.15.119:58052) 2023-12-07T10:36:54.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.406+0000 7f4c8159b700 1 -- 172.21.15.119:0/2348732885 learned_addr learned my addr 172.21.15.119:0/2348732885 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:36:54.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.406+0000 7f4c81d9c700 1 --2- 172.21.15.119:0/2348732885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c7c1075a0 0x7f4c7c197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:54.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.406+0000 7f4c80d9a700 1 -- 172.21.15.119:0/2348732885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c7c1075a0 msgr2=0x7f4c7c197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:54.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.406+0000 7f4c80d9a700 1 --2- 172.21.15.119:0/2348732885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c7c1075a0 0x7f4c7c197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:54.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.407+0000 7f4c80d9a700 1 -- 172.21.15.119:0/2348732885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c7c1014e0 msgr2=0x7f4c7c19d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:54.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.407+0000 7f4c80d9a700 1 --2- 172.21.15.119:0/2348732885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c7c1014e0 0x7f4c7c19d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:54.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.407+0000 7f4c80d9a700 1 -- 172.21.15.119:0/2348732885 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c78009520 con 0x7f4c7c101ea0 2023-12-07T10:36:54.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.407+0000 7f4c81d9c700 1 --2- 172.21.15.119:0/2348732885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c7c1075a0 0x7f4c7c197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:36:54.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.407+0000 7f4c80d9a700 1 --2- 172.21.15.119:0/2348732885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c7c101ea0 0x7f4c7c19db40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0098e0 tx=0x7f4c6c00b9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:54.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.407+0000 7f4c727fc700 1 -- 172.21.15.119:0/2348732885 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c01d070 con 0x7f4c7c101ea0 2023-12-07T10:36:54.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.407+0000 7f4c837ff700 1 -- 172.21.15.119:0/2348732885 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c6c0095f0 con 0x7f4c7c101ea0 2023-12-07T10:36:54.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.407+0000 7f4c727fc700 1 -- 172.21.15.119:0/2348732885 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c6c00beb0 con 0x7f4c7c101ea0 2023-12-07T10:36:54.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.407+0000 7f4c727fc700 1 -- 172.21.15.119:0/2348732885 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c00f9f0 con 0x7f4c7c101ea0 2023-12-07T10:36:54.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.407+0000 7f4c837ff700 1 -- 172.21.15.119:0/2348732885 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c7c1a4410 con 0x7f4c7c101ea0 2023-12-07T10:36:54.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.408+0000 7f4c837ff700 1 -- 172.21.15.119:0/2348732885 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c7c109a40 con 0x7f4c7c101ea0 2023-12-07T10:36:54.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.410+0000 7f4c727fc700 1 -- 172.21.15.119:0/2348732885 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f4c6c00fbd0 con 0x7f4c7c101ea0 2023-12-07T10:36:54.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.411+0000 7f4c727fc700 1 --2- 172.21.15.119:0/2348732885 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4c680a1c80 0x7f4c680a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:54.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.411+0000 7f4c727fc700 1 -- 172.21.15.119:0/2348732885 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f4c6c0ce920 con 0x7f4c7c101ea0 2023-12-07T10:36:54.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.411+0000 7f4c8159b700 1 --2- 172.21.15.119:0/2348732885 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4c680a1c80 0x7f4c680a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:54.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.412+0000 7f4c8159b700 1 --2- 172.21.15.119:0/2348732885 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4c680a1c80 0x7f4c680a4140 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4c78009ef0 tx=0x7f4c78009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:54.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.413+0000 7f4c727fc700 1 -- 172.21.15.119:0/2348732885 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4c6c097680 con 0x7f4c7c101ea0 2023-12-07T10:36:54.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:54 smithi119 ceph-mon[97908]: pgmap v517: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 15 op/s 2023-12-07T10:36:54.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:54 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:36:54.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:54 smithi159 ceph-mon[134726]: pgmap v517: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 15 op/s 2023-12-07T10:36:54.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:54 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:36:54.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:54 smithi153 ceph-mon[133625]: pgmap v517: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 15 op/s 2023-12-07T10:36:54.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:54 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:36:54.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.617+0000 7f4c837ff700 1 -- 172.21.15.119:0/2348732885 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c7c102c00 con 0x7f4c7c101ea0 2023-12-07T10:36:54.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.619+0000 7f4c727fc700 1 -- 172.21.15.119:0/2348732885 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+25286 (secure 0 0 0) 0x7f4c6c09aca0 con 0x7f4c7c101ea0 2023-12-07T10:36:54.619 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:36:54.620 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":114,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","created":"2023-12-07T10:09:59.945863+0000","modified":"2023-12-07T10:36:21.052096+0000","last_up_change":"2023-12-07T10:16:50.929427+0000","last_in_change":"2023-12-07T10:16:33.558158+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T10:13:25.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-07T10:17:39.347813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-07T10:17:40.714501+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-07T10:17:44.257760+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"56b5818e-e625-40f1-8838-3ddcf2d34ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6803","nonce":2141950133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6805","nonce":2141950133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6809","nonce":2141950133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6807","nonce":2141950133}]},"public_addr":"172.21.15.119:6803/2141950133","cluster_addr":"172.21.15.119:6805/2141950133","heartbeat_back_addr":"172.21.15.119:6809/2141950133","heartbeat_front_addr":"172.21.15.119:6807/2141950133","state":["exists","up"]},{"osd":1,"uuid":"36489a6a-5278-4802-9525-9871521722e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6801","nonce":3390875370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6803","nonce":3390875370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6807","nonce":3390875370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6805","nonce":3390875370}]},"public_addr":"172.21.15.153:6801/3390875370","cluster_addr":"172.21.15.153:6803/3390875370","heartbeat_back_addr":"172.21.15.153:6807/3390875370","heartbeat_front_addr":"172.21.15.153:6805/3390875370","state":["exists","up"]},{"osd":2,"uuid":"a937ad26-5e9b-4c8a-b65a-626a8feda6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6801","nonce":708101637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6803","nonce":708101637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6807","nonce":708101637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6805","nonce":708101637}]},"public_addr":"172.21.15.159:6801/708101637","cluster_addr":"172.21.15.159:6803/708101637","heartbeat_back_addr":"172.21.15.159:6807/708101637","heartbeat_front_addr":"172.21.15.159:6805/708101637","state":["exists","up"]},{"osd":3,"uuid":"26ef8300-aa07-48f6-b31d-8b0f2217ffcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6811","nonce":3228047257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6813","nonce":3228047257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6817","nonce":3228047257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6815","nonce":3228047257}]},"public_addr":"172.21.15.119:6811/3228047257","cluster_addr":"172.21.15.119:6813/3228047257","heartbeat_back_addr":"172.21.15.119:6817/3228047257","heartbeat_front_addr":"172.21.15.119:6815/3228047257","state":["exists","up"]},{"osd":4,"uuid":"000ee4b9-9896-4901-8cc3-c85fdfc8cbaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6809","nonce":4021998299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6811","nonce":4021998299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6815","nonce":4021998299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6813","nonce":4021998299}]},"public_addr":"172.21.15.153:6809/4021998299","cluster_addr":"172.21.15.153:6811/4021998299","heartbeat_back_addr":"172.21.15.153:6815/4021998299","heartbeat_front_addr":"172.21.15.153:6813/4021998299","state":["exists","up"]},{"osd":5,"uuid":"8b2fc167-262d-474f-89f3-47bd29424421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6809","nonce":1832540768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6811","nonce":1832540768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6815","nonce":1832540768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6813","nonce":1832540768}]},"public_addr":"172.21.15.159:6809/1832540768","cluster_addr":"172.21.15.159:6811/1832540768","heartbeat_back_addr":"172.21.15.159:6815/1832540768","heartbeat_front_addr":"172.21.15.159:6813/1832540768","state":["exists","up"]},{"osd":6,"uuid":"f9aa6749-46cf-4b2a-8114-cbd034219d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162436840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162436840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6825","nonce":1162436840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162436840}]},"public_addr":"172.21.15.119:6819/1162436840","cluster_addr":"172.21.15.119:6821/1162436840","heartbeat_back_addr":"172.21.15.119:6825/1162436840","heartbeat_front_addr":"172.21.15.119:6823/1162436840","state":["exists","up"]},{"osd":7,"uuid":"39216361-e1c4-4fef-86cb-a4707656b547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6817","nonce":3254653457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6819","nonce":3254653457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6823","nonce":3254653457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6821","nonce":3254653457}]},"public_addr":"172.21.15.153:6817/3254653457","cluster_addr":"172.21.15.153:6819/3254653457","heartbeat_back_addr":"172.21.15.153:6823/3254653457","heartbeat_front_addr":"172.21.15.153:6821/3254653457","state":["exists","up"]},{"osd":8,"uuid":"cceb6ebc-7c02-48ea-9d16-b7a016dd449b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6817","nonce":2600946380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6819","nonce":2600946380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6823","nonce":2600946380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6821","nonce":2600946380}]},"public_addr":"172.21.15.159:6817/2600946380","cluster_addr":"172.21.15.159:6819/2600946380","heartbeat_back_addr":"172.21.15.159:6823/2600946380","heartbeat_front_addr":"172.21.15.159:6821/2600946380","state":["exists","up"]},{"osd":9,"uuid":"f337a2e4-e065-4a9f-91d9-e3615de2d6f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6827","nonce":1940737285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6829","nonce":1940737285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6833","nonce":1940737285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6831","nonce":1940737285}]},"public_addr":"172.21.15.119:6827/1940737285","cluster_addr":"172.21.15.119:6829/1940737285","heartbeat_back_addr":"172.21.15.119:6833/1940737285","heartbeat_front_addr":"172.21.15.119:6831/1940737285","state":["exists","up"]},{"osd":10,"uuid":"3a3bc347-85f4-4878-9797-d9da96137e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6825","nonce":663834864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6827","nonce":663834864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6831","nonce":663834864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6829","nonce":663834864}]},"public_addr":"172.21.15.153:6825/663834864","cluster_addr":"172.21.15.153:6827/663834864","heartbeat_back_addr":"172.21.15.153:6831/663834864","heartbeat_front_addr":"172.21.15.153:6829/663834864","state":["exists","up"]},{"osd":11,"uuid":"a0fc5909-279d-4172-977c-2dc2da0a7e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6825","nonce":1262216348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6827","nonce":1262216348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6831","nonce":1262216348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6829","nonce":1262216348}]},"public_addr":"172.21.15.159:6825/1262216348","cluster_addr":"172.21.15.159:6827/1262216348","heartbeat_back_addr":"172.21.15.159:6831/1262216348","heartbeat_front_addr":"172.21.15.159:6829/1262216348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:37.488109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:59.543060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:21.381585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:47.112729+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:09.732150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:30.893729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:56.678710+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:19.004256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:40.388818+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:06.391734+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:28.394053+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:49.634348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/2490678270":"2023-12-08T10:20:24.856450+0000","172.21.15.119:0/2580180729":"2023-12-08T10:20:24.856450+0000","172.21.15.119:0/149289687":"2023-12-08T10:20:24.856450+0000","172.21.15.119:0/2050367825":"2023-12-08T10:20:24.856450+0000","172.21.15.119:6800/1344846367":"2023-12-08T10:20:24.856450+0000","172.21.15.119:0/1628639392":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6801/1355617847":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6800/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/362307836":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/1344846367":"2023-12-08T10:20:24.856450+0000","172.21.15.119:0/128964558":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/2278813554":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/1411933614":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/1685698277":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6800/1355617847":"2023-12-08T10:11:00.585565+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T10:36:54.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.622+0000 7f4c837ff700 1 -- 172.21.15.119:0/2348732885 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4c680a1c80 msgr2=0x7f4c680a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:54.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.622+0000 7f4c837ff700 1 --2- 172.21.15.119:0/2348732885 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4c680a1c80 0x7f4c680a4140 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4c78009ef0 tx=0x7f4c78009260 comp rx=0 tx=0).stop 2023-12-07T10:36:54.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.622+0000 7f4c837ff700 1 -- 172.21.15.119:0/2348732885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c7c101ea0 msgr2=0x7f4c7c19db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:54.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.622+0000 7f4c837ff700 1 --2- 172.21.15.119:0/2348732885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c7c101ea0 0x7f4c7c19db40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0098e0 tx=0x7f4c6c00b9c0 comp rx=0 tx=0).stop 2023-12-07T10:36:54.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.622+0000 7f4c837ff700 1 -- 172.21.15.119:0/2348732885 shutdown_connections 2023-12-07T10:36:54.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.622+0000 7f4c837ff700 1 --2- 172.21.15.119:0/2348732885 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4c680a1c80 0x7f4c680a4140 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:54.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.622+0000 7f4c837ff700 1 --2- 172.21.15.119:0/2348732885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c7c1075a0 0x7f4c7c197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:54.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.622+0000 7f4c837ff700 1 --2- 172.21.15.119:0/2348732885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c7c1014e0 0x7f4c7c19d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:54.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.622+0000 7f4c837ff700 1 --2- 172.21.15.119:0/2348732885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c7c101ea0 0x7f4c7c19db40 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:54.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.622+0000 7f4c837ff700 1 -- 172.21.15.119:0/2348732885 >> 172.21.15.119:0/2348732885 conn(0x7f4c7c0fcf90 msgr2=0x7f4c7c0fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:54.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.622+0000 7f4c837ff700 1 -- 172.21.15.119:0/2348732885 shutdown_connections 2023-12-07T10:36:54.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.622+0000 7f4c837ff700 1 -- 172.21.15.119:0/2348732885 wait complete. 2023-12-07T10:36:54.641 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-07T10:13:25.194366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-07T10:17:39.347813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-07T10:17:40.714501+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 113, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-07T10:17:44.257760+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 113, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-07T10:36:54.642 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-07T10:36:54.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.843+0000 7f6d4f5ff700 1 -- 172.21.15.119:0/1940620970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d480739b0 msgr2=0x7f6d48073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:54.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.843+0000 7f6d4f5ff700 1 --2- 172.21.15.119:0/1940620970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d480739b0 0x7f6d48073d90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6d44009990 tx=0x7f6d44009ca0 comp rx=0 tx=0).stop 2023-12-07T10:36:54.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.843+0000 7f6d4f5ff700 1 -- 172.21.15.119:0/1940620970 shutdown_connections 2023-12-07T10:36:54.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.843+0000 7f6d4f5ff700 1 --2- 172.21.15.119:0/1940620970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d48112550 0x7f6d48114940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:54.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.843+0000 7f6d4f5ff700 1 --2- 172.21.15.119:0/1940620970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d480739b0 0x7f6d48073d90 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:54.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.843+0000 7f6d4f5ff700 1 --2- 172.21.15.119:0/1940620970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d480742d0 0x7f6d48112010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:54.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.843+0000 7f6d4f5ff700 1 -- 172.21.15.119:0/1940620970 >> 172.21.15.119:0/1940620970 conn(0x7f6d480fcfd0 msgr2=0x7f6d480ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:54.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.843+0000 7f6d4f5ff700 1 -- 172.21.15.119:0/1940620970 shutdown_connections 2023-12-07T10:36:54.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.843+0000 7f6d4f5ff700 1 -- 172.21.15.119:0/1940620970 wait complete. 2023-12-07T10:36:54.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.844+0000 7f6d4f5ff700 1 Processor -- start 2023-12-07T10:36:54.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.845+0000 7f6d4f5ff700 1 -- start start 2023-12-07T10:36:54.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.845+0000 7f6d4f5ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d480739b0 0x7f6d4819e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:54.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.845+0000 7f6d4f5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d480742d0 0x7f6d4819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:54.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.845+0000 7f6d4f5ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d48112550 0x7f6d481a2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:54.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.845+0000 7f6d4f5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d481a2fd0 con 0x7f6d480742d0 2023-12-07T10:36:54.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.845+0000 7f6d4f5ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d481a3140 con 0x7f6d480739b0 2023-12-07T10:36:54.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.845+0000 7f6d4f5ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d481a3300 con 0x7f6d48112550 2023-12-07T10:36:54.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.845+0000 7f6d4cb9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d480742d0 0x7f6d4819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:54.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.845+0000 7f6d4cb9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d480742d0 0x7f6d4819e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58072/0 (socket says 172.21.15.119:58072) 2023-12-07T10:36:54.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.845+0000 7f6d4cb9a700 1 -- 172.21.15.119:0/61143036 learned_addr learned my addr 172.21.15.119:0/61143036 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:36:54.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.845+0000 7f6d4d39b700 1 --2- 172.21.15.119:0/61143036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d480739b0 0x7f6d4819e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:54.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.845+0000 7f6d4db9c700 1 --2- 172.21.15.119:0/61143036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d48112550 0x7f6d481a2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:54.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.846+0000 7f6d4cb9a700 1 -- 172.21.15.119:0/61143036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d48112550 msgr2=0x7f6d481a2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:54.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.846+0000 7f6d4cb9a700 1 --2- 172.21.15.119:0/61143036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d48112550 0x7f6d481a2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:54.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.846+0000 7f6d4cb9a700 1 -- 172.21.15.119:0/61143036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d480739b0 msgr2=0x7f6d4819e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:54.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.846+0000 7f6d4cb9a700 1 --2- 172.21.15.119:0/61143036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d480739b0 0x7f6d4819e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:54.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.846+0000 7f6d4cb9a700 1 -- 172.21.15.119:0/61143036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d440095f0 con 0x7f6d480742d0 2023-12-07T10:36:54.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.846+0000 7f6d4cb9a700 1 --2- 172.21.15.119:0/61143036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d480742d0 0x7f6d4819e5f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800e9a0 tx=0x7f6d3800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:54.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.846+0000 7f6d3e7fc700 1 -- 172.21.15.119:0/61143036 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38009790 con 0x7f6d480742d0 2023-12-07T10:36:54.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.846+0000 7f6d4f5ff700 1 -- 172.21.15.119:0/61143036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d481a35e0 con 0x7f6d480742d0 2023-12-07T10:36:54.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.846+0000 7f6d3e7fc700 1 -- 172.21.15.119:0/61143036 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d38010430 con 0x7f6d480742d0 2023-12-07T10:36:54.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.846+0000 7f6d3e7fc700 1 -- 172.21.15.119:0/61143036 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38018680 con 0x7f6d480742d0 2023-12-07T10:36:54.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.846+0000 7f6d4f5ff700 1 -- 172.21.15.119:0/61143036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d481aae20 con 0x7f6d480742d0 2023-12-07T10:36:54.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.847+0000 7f6d4f5ff700 1 -- 172.21.15.119:0/61143036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d48104a30 con 0x7f6d480742d0 2023-12-07T10:36:54.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.849+0000 7f6d3e7fc700 1 -- 172.21.15.119:0/61143036 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f6d380105a0 con 0x7f6d480742d0 2023-12-07T10:36:54.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.850+0000 7f6d3e7fc700 1 --2- 172.21.15.119:0/61143036 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f6d340a1c80 0x7f6d340a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:54.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.851+0000 7f6d3e7fc700 1 -- 172.21.15.119:0/61143036 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f6d38014070 con 0x7f6d480742d0 2023-12-07T10:36:54.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.851+0000 7f6d4d39b700 1 --2- 172.21.15.119:0/61143036 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f6d340a1c80 0x7f6d340a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:54.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.851+0000 7f6d4d39b700 1 --2- 172.21.15.119:0/61143036 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f6d340a1c80 0x7f6d340a4140 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400b2e0 tx=0x7f6d44005fd0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:54.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:54.854+0000 7f6d3e7fc700 1 -- 172.21.15.119:0/61143036 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f6d380522f0 con 0x7f6d480742d0 2023-12-07T10:36:55.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.055+0000 7f6d4f5ff700 1 -- 172.21.15.119:0/61143036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6d4819f3b0 con 0x7f6d480742d0 2023-12-07T10:36:55.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.056+0000 7f6d3e7fc700 1 -- 172.21.15.119:0/61143036 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6d380d2070 con 0x7f6d480742d0 2023-12-07T10:36:55.056 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-12-07T10:36:55.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.059+0000 7f6d4f5ff700 1 -- 172.21.15.119:0/61143036 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f6d340a1c80 msgr2=0x7f6d340a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:55.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.059+0000 7f6d4f5ff700 1 --2- 172.21.15.119:0/61143036 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f6d340a1c80 0x7f6d340a4140 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400b2e0 tx=0x7f6d44005fd0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.059+0000 7f6d4f5ff700 1 -- 172.21.15.119:0/61143036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d480742d0 msgr2=0x7f6d4819e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:55.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.059+0000 7f6d4f5ff700 1 --2- 172.21.15.119:0/61143036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d480742d0 0x7f6d4819e5f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800e9a0 tx=0x7f6d3800ecb0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.059+0000 7f6d4f5ff700 1 -- 172.21.15.119:0/61143036 shutdown_connections 2023-12-07T10:36:55.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.059+0000 7f6d4f5ff700 1 --2- 172.21.15.119:0/61143036 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f6d340a1c80 0x7f6d340a4140 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.059+0000 7f6d4f5ff700 1 --2- 172.21.15.119:0/61143036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d48112550 0x7f6d481a2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.059+0000 7f6d4f5ff700 1 --2- 172.21.15.119:0/61143036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d480739b0 0x7f6d4819e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.059+0000 7f6d4f5ff700 1 --2- 172.21.15.119:0/61143036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d480742d0 0x7f6d4819e5f0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.059+0000 7f6d4f5ff700 1 -- 172.21.15.119:0/61143036 >> 172.21.15.119:0/61143036 conn(0x7f6d480fcfd0 msgr2=0x7f6d480fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:55.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.059+0000 7f6d4f5ff700 1 -- 172.21.15.119:0/61143036 shutdown_connections 2023-12-07T10:36:55.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.059+0000 7f6d4f5ff700 1 -- 172.21.15.119:0/61143036 wait complete. 2023-12-07T10:36:55.078 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-07T10:36:55.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.283+0000 7f8c205ff700 1 -- 172.21.15.119:0/2623667296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c18068a50 msgr2=0x7f8c18068e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:55.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.283+0000 7f8c205ff700 1 --2- 172.21.15.119:0/2623667296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c18068a50 0x7f8c18068e30 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8c14009990 tx=0x7f8c14009ca0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.283+0000 7f8c205ff700 1 -- 172.21.15.119:0/2623667296 shutdown_connections 2023-12-07T10:36:55.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.283+0000 7f8c205ff700 1 --2- 172.21.15.119:0/2623667296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c18106ad0 0x7f8c1810e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.283+0000 7f8c205ff700 1 --2- 172.21.15.119:0/2623667296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c18068a50 0x7f8c18068e30 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.283+0000 7f8c205ff700 1 --2- 172.21.15.119:0/2623667296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c18106110 0x7f8c18106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.283+0000 7f8c205ff700 1 -- 172.21.15.119:0/2623667296 >> 172.21.15.119:0/2623667296 conn(0x7f8c18077610 msgr2=0x7f8c18077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:55.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.283+0000 7f8c205ff700 1 -- 172.21.15.119:0/2623667296 shutdown_connections 2023-12-07T10:36:55.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.283+0000 7f8c205ff700 1 -- 172.21.15.119:0/2623667296 wait complete. 2023-12-07T10:36:55.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.284+0000 7f8c205ff700 1 Processor -- start 2023-12-07T10:36:55.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.284+0000 7f8c205ff700 1 -- start start 2023-12-07T10:36:55.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.285+0000 7f8c205ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c18068a50 0x7f8c18197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:55.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.285+0000 7f8c205ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c18106110 0x7f8c18197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:55.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.285+0000 7f8c205ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c18106ad0 0x7f8c1819c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:55.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.285+0000 7f8c205ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1819c900 con 0x7f8c18106ad0 2023-12-07T10:36:55.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.285+0000 7f8c205ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1819ca70 con 0x7f8c18106110 2023-12-07T10:36:55.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.285+0000 7f8c205ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1819cc30 con 0x7f8c18068a50 2023-12-07T10:36:55.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.285+0000 7f8c1eb9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c18106ad0 0x7f8c1819c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:55.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.285+0000 7f8c1eb9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c18106ad0 0x7f8c1819c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58088/0 (socket says 172.21.15.119:58088) 2023-12-07T10:36:55.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.285+0000 7f8c1e39b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c18068a50 0x7f8c18197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:55.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.285+0000 7f8c1eb9c700 1 -- 172.21.15.119:0/2084344180 learned_addr learned my addr 172.21.15.119:0/2084344180 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:36:55.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.285+0000 7f8c1db9a700 1 --2- 172.21.15.119:0/2084344180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c18106110 0x7f8c18197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:55.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.285+0000 7f8c1eb9c700 1 -- 172.21.15.119:0/2084344180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c18068a50 msgr2=0x7f8c18197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:55.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.285+0000 7f8c1eb9c700 1 --2- 172.21.15.119:0/2084344180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c18068a50 0x7f8c18197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.285+0000 7f8c1eb9c700 1 -- 172.21.15.119:0/2084344180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c18106110 msgr2=0x7f8c18197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:55.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.285+0000 7f8c1eb9c700 1 --2- 172.21.15.119:0/2084344180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c18106110 0x7f8c18197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.285+0000 7f8c1eb9c700 1 -- 172.21.15.119:0/2084344180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c140095f0 con 0x7f8c18106ad0 2023-12-07T10:36:55.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.286+0000 7f8c1eb9c700 1 --2- 172.21.15.119:0/2084344180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c18106ad0 0x7f8c1819c330 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8c10007f10 tx=0x7f8c1000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:55.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.286+0000 7f8c0f7fe700 1 -- 172.21.15.119:0/2084344180 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c10017070 con 0x7f8c18106ad0 2023-12-07T10:36:55.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.286+0000 7f8c0f7fe700 1 -- 172.21.15.119:0/2084344180 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c1000f040 con 0x7f8c18106ad0 2023-12-07T10:36:55.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.286+0000 7f8c0f7fe700 1 -- 172.21.15.119:0/2084344180 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c100054b0 con 0x7f8c18106ad0 2023-12-07T10:36:55.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.286+0000 7f8c205ff700 1 -- 172.21.15.119:0/2084344180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1819cf10 con 0x7f8c18106ad0 2023-12-07T10:36:55.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.286+0000 7f8c205ff700 1 -- 172.21.15.119:0/2084344180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c181a47c0 con 0x7f8c18106ad0 2023-12-07T10:36:55.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.287+0000 7f8c205ff700 1 -- 172.21.15.119:0/2084344180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c18109a70 con 0x7f8c18106ad0 2023-12-07T10:36:55.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.289+0000 7f8c0f7fe700 1 -- 172.21.15.119:0/2084344180 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f8c10004ad0 con 0x7f8c18106ad0 2023-12-07T10:36:55.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.290+0000 7f8c0f7fe700 1 --2- 172.21.15.119:0/2084344180 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8c040a1d00 0x7f8c040a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:55.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.290+0000 7f8c0f7fe700 1 -- 172.21.15.119:0/2084344180 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f8c100cdac0 con 0x7f8c18106ad0 2023-12-07T10:36:55.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.290+0000 7f8c1e39b700 1 --2- 172.21.15.119:0/2084344180 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8c040a1d00 0x7f8c040a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:55.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.291+0000 7f8c1e39b700 1 --2- 172.21.15.119:0/2084344180 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8c040a1d00 0x7f8c040a41c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8c14006010 tx=0x7f8c1400b260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:55.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.292+0000 7f8c0f7fe700 1 -- 172.21.15.119:0/2084344180 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8c100966f0 con 0x7f8c18106ad0 2023-12-07T10:36:55.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:55 smithi119 ceph-mon[97908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:36:55.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:55 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2348732885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:36:55.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:55 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/61143036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:36:55.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.498+0000 7f8c205ff700 1 -- 172.21.15.119:0/2084344180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8c18198d20 con 0x7f8c18106ad0 2023-12-07T10:36:55.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.498+0000 7f8c0f7fe700 1 -- 172.21.15.119:0/2084344180 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v114) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8c10099d10 con 0x7f8c18106ad0 2023-12-07T10:36:55.498 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-12-07T10:36:55.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.500+0000 7f8c205ff700 1 -- 172.21.15.119:0/2084344180 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8c040a1d00 msgr2=0x7f8c040a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:55.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.500+0000 7f8c205ff700 1 --2- 172.21.15.119:0/2084344180 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8c040a1d00 0x7f8c040a41c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8c14006010 tx=0x7f8c1400b260 comp rx=0 tx=0).stop 2023-12-07T10:36:55.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.500+0000 7f8c205ff700 1 -- 172.21.15.119:0/2084344180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c18106ad0 msgr2=0x7f8c1819c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:55.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.500+0000 7f8c205ff700 1 --2- 172.21.15.119:0/2084344180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c18106ad0 0x7f8c1819c330 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8c10007f10 tx=0x7f8c1000d3b0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.501+0000 7f8c205ff700 1 -- 172.21.15.119:0/2084344180 shutdown_connections 2023-12-07T10:36:55.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.501+0000 7f8c205ff700 1 --2- 172.21.15.119:0/2084344180 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f8c040a1d00 0x7f8c040a41c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.501+0000 7f8c205ff700 1 --2- 172.21.15.119:0/2084344180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c18106ad0 0x7f8c1819c330 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.501+0000 7f8c205ff700 1 --2- 172.21.15.119:0/2084344180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c18068a50 0x7f8c18197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.501+0000 7f8c205ff700 1 --2- 172.21.15.119:0/2084344180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c18106110 0x7f8c18197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.501+0000 7f8c205ff700 1 -- 172.21.15.119:0/2084344180 >> 172.21.15.119:0/2084344180 conn(0x7f8c18077610 msgr2=0x7f8c180fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:55.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.501+0000 7f8c205ff700 1 -- 172.21.15.119:0/2084344180 shutdown_connections 2023-12-07T10:36:55.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.501+0000 7f8c205ff700 1 -- 172.21.15.119:0/2084344180 wait complete. 2023-12-07T10:36:55.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:55 smithi159 ceph-mon[134726]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:36:55.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:55 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2348732885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:36:55.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:55 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/61143036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:36:55.519 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-07T10:36:55.527 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:55 smithi153 ceph-mon[133625]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-07T10:36:55.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:55 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2348732885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:36:55.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:55 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/61143036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-07T10:36:55.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.721+0000 7f0035bff700 1 -- 172.21.15.119:0/4033874830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0030101560 msgr2=0x7f00301019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:55.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.721+0000 7f0035bff700 1 --2- 172.21.15.119:0/4033874830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0030101560 0x7f00301019e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f00180099c0 tx=0x7f0018009cd0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.721+0000 7f0035bff700 1 -- 172.21.15.119:0/4033874830 shutdown_connections 2023-12-07T10:36:55.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.721+0000 7f0035bff700 1 --2- 172.21.15.119:0/4033874830 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0030101f20 0x7f003010e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.721+0000 7f0035bff700 1 --2- 172.21.15.119:0/4033874830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0030107620 0x7f0030107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.721+0000 7f0035bff700 1 --2- 172.21.15.119:0/4033874830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0030101560 0x7f00301019e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.721+0000 7f0035bff700 1 -- 172.21.15.119:0/4033874830 >> 172.21.15.119:0/4033874830 conn(0x7f00300fcfd0 msgr2=0x7f00300ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:55.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.721+0000 7f0035bff700 1 -- 172.21.15.119:0/4033874830 shutdown_connections 2023-12-07T10:36:55.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.721+0000 7f0035bff700 1 -- 172.21.15.119:0/4033874830 wait complete. 2023-12-07T10:36:55.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.722+0000 7f0035bff700 1 Processor -- start 2023-12-07T10:36:55.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.722+0000 7f0035bff700 1 -- start start 2023-12-07T10:36:55.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.723+0000 7f0035bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0030101560 0x7f003019f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:55.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.723+0000 7f0035bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0030101f20 0x7f003019fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:55.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.723+0000 7f0035bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0030107620 0x7f00301999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:55.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.723+0000 7f0035bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003019a050 con 0x7f0030101f20 2023-12-07T10:36:55.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.723+0000 7f0035bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003019a1c0 con 0x7f0030107620 2023-12-07T10:36:55.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.723+0000 7f0035bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003019a380 con 0x7f0030101560 2023-12-07T10:36:55.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.723+0000 7f0027fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0030101f20 0x7f003019fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:55.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.723+0000 7f0027fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0030101f20 0x7f003019fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58112/0 (socket says 172.21.15.119:58112) 2023-12-07T10:36:55.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.723+0000 7f0027fff700 1 -- 172.21.15.119:0/3991528095 learned_addr learned my addr 172.21.15.119:0/3991528095 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:36:55.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.723+0000 7f002f7fe700 1 --2- 172.21.15.119:0/3991528095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0030101560 0x7f003019f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:55.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.723+0000 7f002ffff700 1 --2- 172.21.15.119:0/3991528095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0030107620 0x7f00301999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:55.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.723+0000 7f0027fff700 1 -- 172.21.15.119:0/3991528095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0030101560 msgr2=0x7f003019f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:55.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.723+0000 7f0027fff700 1 --2- 172.21.15.119:0/3991528095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0030101560 0x7f003019f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.723+0000 7f0027fff700 1 -- 172.21.15.119:0/3991528095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0030107620 msgr2=0x7f00301999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:55.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.723+0000 7f0027fff700 1 --2- 172.21.15.119:0/3991528095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0030107620 0x7f00301999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.723+0000 7f0027fff700 1 -- 172.21.15.119:0/3991528095 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00180095f0 con 0x7f0030101f20 2023-12-07T10:36:55.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.723+0000 7f002ffff700 1 --2- 172.21.15.119:0/3991528095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0030107620 0x7f00301999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:36:55.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.724+0000 7f0027fff700 1 --2- 172.21.15.119:0/3991528095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0030101f20 0x7f003019fe00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f001800b580 tx=0x7f001800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:55.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.724+0000 7f002d7fa700 1 -- 172.21.15.119:0/3991528095 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001801c070 con 0x7f0030101f20 2023-12-07T10:36:55.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.724+0000 7f002d7fa700 1 -- 172.21.15.119:0/3991528095 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f001800fdf0 con 0x7f0030101f20 2023-12-07T10:36:55.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.724+0000 7f0035bff700 1 -- 172.21.15.119:0/3991528095 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f003019a600 con 0x7f0030101f20 2023-12-07T10:36:55.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.724+0000 7f002d7fa700 1 -- 172.21.15.119:0/3991528095 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0018022c40 con 0x7f0030101f20 2023-12-07T10:36:55.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.724+0000 7f0035bff700 1 -- 172.21.15.119:0/3991528095 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00301a68b0 con 0x7f0030101f20 2023-12-07T10:36:55.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.725+0000 7f0035bff700 1 -- 172.21.15.119:0/3991528095 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0030102c80 con 0x7f0030101f20 2023-12-07T10:36:55.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.727+0000 7f002d7fa700 1 -- 172.21.15.119:0/3991528095 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f0018020070 con 0x7f0030101f20 2023-12-07T10:36:55.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.728+0000 7f002d7fa700 1 --2- 172.21.15.119:0/3991528095 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f001c0a1d00 0x7f001c0a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:55.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.728+0000 7f002d7fa700 1 -- 172.21.15.119:0/3991528095 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f00180ceae0 con 0x7f0030101f20 2023-12-07T10:36:55.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.728+0000 7f002f7fe700 1 --2- 172.21.15.119:0/3991528095 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f001c0a1d00 0x7f001c0a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:55.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.729+0000 7f002f7fe700 1 --2- 172.21.15.119:0/3991528095 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f001c0a1d00 0x7f001c0a41c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0020009c60 tx=0x7f0020009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:55.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.732+0000 7f002d7fa700 1 -- 172.21.15.119:0/3991528095 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0018097840 con 0x7f0030101f20 2023-12-07T10:36:55.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.932+0000 7f0035bff700 1 -- 172.21.15.119:0/3991528095 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f00301114d0 con 0x7f0030101f20 2023-12-07T10:36:55.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.933+0000 7f002d7fa700 1 -- 172.21.15.119:0/3991528095 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v114) v1 ==== 100+0+11 (secure 0 0 0) 0x7f001809ae60 con 0x7f0030101f20 2023-12-07T10:36:55.933 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-12-07T10:36:55.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.935+0000 7f0035bff700 1 -- 172.21.15.119:0/3991528095 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f001c0a1d00 msgr2=0x7f001c0a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:55.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.935+0000 7f0035bff700 1 --2- 172.21.15.119:0/3991528095 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f001c0a1d00 0x7f001c0a41c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0020009c60 tx=0x7f0020009260 comp rx=0 tx=0).stop 2023-12-07T10:36:55.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.935+0000 7f0035bff700 1 -- 172.21.15.119:0/3991528095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0030101f20 msgr2=0x7f003019fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:55.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.935+0000 7f0035bff700 1 --2- 172.21.15.119:0/3991528095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0030101f20 0x7f003019fe00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f001800b580 tx=0x7f001800bfd0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.935+0000 7f0035bff700 1 -- 172.21.15.119:0/3991528095 shutdown_connections 2023-12-07T10:36:55.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.935+0000 7f0035bff700 1 --2- 172.21.15.119:0/3991528095 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f001c0a1d00 0x7f001c0a41c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.935+0000 7f0035bff700 1 --2- 172.21.15.119:0/3991528095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0030107620 0x7f00301999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.935+0000 7f0035bff700 1 --2- 172.21.15.119:0/3991528095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0030101560 0x7f003019f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.935+0000 7f0035bff700 1 --2- 172.21.15.119:0/3991528095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0030101f20 0x7f003019fe00 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:55.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.935+0000 7f0035bff700 1 -- 172.21.15.119:0/3991528095 >> 172.21.15.119:0/3991528095 conn(0x7f00300fcfd0 msgr2=0x7f00300fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:55.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.935+0000 7f0035bff700 1 -- 172.21.15.119:0/3991528095 shutdown_connections 2023-12-07T10:36:55.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:55.936+0000 7f0035bff700 1 -- 172.21.15.119:0/3991528095 wait complete. 2023-12-07T10:36:55.955 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-07T10:36:56.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.156+0000 7f5c445ff700 1 -- 172.21.15.119:0/719346355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c3c101f20 msgr2=0x7f5c3c10e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:56.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.156+0000 7f5c445ff700 1 --2- 172.21.15.119:0/719346355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c3c101f20 0x7f5c3c10e3e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400b0d0 tx=0x7f5c3400b3e0 comp rx=0 tx=0).stop 2023-12-07T10:36:56.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.156+0000 7f5c445ff700 1 -- 172.21.15.119:0/719346355 shutdown_connections 2023-12-07T10:36:56.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.156+0000 7f5c445ff700 1 --2- 172.21.15.119:0/719346355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c3c101f20 0x7f5c3c10e3e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:56.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.156+0000 7f5c445ff700 1 --2- 172.21.15.119:0/719346355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c3c107620 0x7f5c3c107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:56.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.156+0000 7f5c445ff700 1 --2- 172.21.15.119:0/719346355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c3c101560 0x7f5c3c1019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:56.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.156+0000 7f5c445ff700 1 -- 172.21.15.119:0/719346355 >> 172.21.15.119:0/719346355 conn(0x7f5c3c0fcfd0 msgr2=0x7f5c3c0ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:56.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.157+0000 7f5c445ff700 1 -- 172.21.15.119:0/719346355 shutdown_connections 2023-12-07T10:36:56.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.157+0000 7f5c445ff700 1 -- 172.21.15.119:0/719346355 wait complete. 2023-12-07T10:36:56.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.157+0000 7f5c445ff700 1 Processor -- start 2023-12-07T10:36:56.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.158+0000 7f5c445ff700 1 -- start start 2023-12-07T10:36:56.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.158+0000 7f5c445ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c3c101560 0x7f5c3c19d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:56.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.158+0000 7f5c445ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c3c101f20 0x7f5c3c19dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:56.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.158+0000 7f5c445ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c3c107620 0x7f5c3c1978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:56.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.158+0000 7f5c445ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3c197f30 con 0x7f5c3c107620 2023-12-07T10:36:56.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.158+0000 7f5c445ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3c1980a0 con 0x7f5c3c101560 2023-12-07T10:36:56.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.158+0000 7f5c445ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3c198260 con 0x7f5c3c101f20 2023-12-07T10:36:56.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.158+0000 7f5c4239b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c3c101560 0x7f5c3c19d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:56.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.158+0000 7f5c41b9a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c3c101f20 0x7f5c3c19dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:56.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.158+0000 7f5c4239b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c3c101560 0x7f5c3c19d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:58722/0 (socket says 172.21.15.119:58722) 2023-12-07T10:36:56.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.158+0000 7f5c4239b700 1 -- 172.21.15.119:0/3878282068 learned_addr learned my addr 172.21.15.119:0/3878282068 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:36:56.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.158+0000 7f5c42b9c700 1 --2- 172.21.15.119:0/3878282068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c3c107620 0x7f5c3c1978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:56.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.159+0000 7f5c4239b700 1 -- 172.21.15.119:0/3878282068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c3c101f20 msgr2=0x7f5c3c19dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:56.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.159+0000 7f5c4239b700 1 --2- 172.21.15.119:0/3878282068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c3c101f20 0x7f5c3c19dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:56.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.159+0000 7f5c4239b700 1 -- 172.21.15.119:0/3878282068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c3c107620 msgr2=0x7f5c3c1978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:56.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.159+0000 7f5c4239b700 1 --2- 172.21.15.119:0/3878282068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c3c107620 0x7f5c3c1978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:56.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.159+0000 7f5c4239b700 1 -- 172.21.15.119:0/3878282068 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c34009d00 con 0x7f5c3c101560 2023-12-07T10:36:56.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.159+0000 7f5c41b9a700 1 --2- 172.21.15.119:0/3878282068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c3c101f20 0x7f5c3c19dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:36:56.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.159+0000 7f5c42b9c700 1 --2- 172.21.15.119:0/3878282068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c3c107620 0x7f5c3c1978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:36:56.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.159+0000 7f5c4239b700 1 --2- 172.21.15.119:0/3878282068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c3c101560 0x7f5c3c19d7a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800e9a0 tx=0x7f5c3800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:56.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.159+0000 7f5c337fe700 1 -- 172.21.15.119:0/3878282068 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c38009790 con 0x7f5c3c101560 2023-12-07T10:36:56.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.159+0000 7f5c445ff700 1 -- 172.21.15.119:0/3878282068 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c3c198540 con 0x7f5c3c101560 2023-12-07T10:36:56.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.159+0000 7f5c337fe700 1 -- 172.21.15.119:0/3878282068 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c38010430 con 0x7f5c3c101560 2023-12-07T10:36:56.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.159+0000 7f5c337fe700 1 -- 172.21.15.119:0/3878282068 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c38018680 con 0x7f5c3c101560 2023-12-07T10:36:56.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.159+0000 7f5c445ff700 1 -- 172.21.15.119:0/3878282068 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c3c1a4790 con 0x7f5c3c101560 2023-12-07T10:36:56.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.160+0000 7f5c445ff700 1 -- 172.21.15.119:0/3878282068 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c3c109ac0 con 0x7f5c3c101560 2023-12-07T10:36:56.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.162+0000 7f5c337fe700 1 -- 172.21.15.119:0/3878282068 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f5c380105a0 con 0x7f5c3c101560 2023-12-07T10:36:56.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.164+0000 7f5c337fe700 1 --2- 172.21.15.119:0/3878282068 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5c280a1c80 0x7f5c280a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:56.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.164+0000 7f5c337fe700 1 -- 172.21.15.119:0/3878282068 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f5c38014070 con 0x7f5c3c101560 2023-12-07T10:36:56.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.164+0000 7f5c41b9a700 1 --2- 172.21.15.119:0/3878282068 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5c280a1c80 0x7f5c280a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:56.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.164+0000 7f5c41b9a700 1 --2- 172.21.15.119:0/3878282068 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5c280a1c80 0x7f5c280a4140 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c198a90 tx=0x7f5c2c009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:56.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.166+0000 7f5c337fe700 1 -- 172.21.15.119:0/3878282068 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5c38096e30 con 0x7f5c3c101560 2023-12-07T10:36:56.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.371+0000 7f5c445ff700 1 -- 172.21.15.119:0/3878282068 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5c3c198d90 con 0x7f5c3c101560 2023-12-07T10:36:56.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.372+0000 7f5c337fe700 1 -- 172.21.15.119:0/3878282068 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v114) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5c380260a0 con 0x7f5c3c101560 2023-12-07T10:36:56.372 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-12-07T10:36:56.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.374+0000 7f5c445ff700 1 -- 172.21.15.119:0/3878282068 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5c280a1c80 msgr2=0x7f5c280a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:56.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.374+0000 7f5c445ff700 1 --2- 172.21.15.119:0/3878282068 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5c280a1c80 0x7f5c280a4140 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c198a90 tx=0x7f5c2c009260 comp rx=0 tx=0).stop 2023-12-07T10:36:56.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.374+0000 7f5c445ff700 1 -- 172.21.15.119:0/3878282068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c3c101560 msgr2=0x7f5c3c19d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.374+0000 7f5c445ff700 1 --2- 172.21.15.119:0/3878282068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c3c101560 0x7f5c3c19d7a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800e9a0 tx=0x7f5c3800ecb0 comp rx=0 tx=0).stop 2023-12-07T10:36:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.374+0000 7f5c445ff700 1 -- 172.21.15.119:0/3878282068 shutdown_connections 2023-12-07T10:36:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.374+0000 7f5c445ff700 1 --2- 172.21.15.119:0/3878282068 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5c280a1c80 0x7f5c280a4140 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.374+0000 7f5c445ff700 1 --2- 172.21.15.119:0/3878282068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c3c107620 0x7f5c3c1978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.374+0000 7f5c445ff700 1 --2- 172.21.15.119:0/3878282068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c3c101560 0x7f5c3c19d7a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.374+0000 7f5c445ff700 1 --2- 172.21.15.119:0/3878282068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c3c101f20 0x7f5c3c19dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.374+0000 7f5c445ff700 1 -- 172.21.15.119:0/3878282068 >> 172.21.15.119:0/3878282068 conn(0x7f5c3c0fcfd0 msgr2=0x7f5c3c0fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.374+0000 7f5c445ff700 1 -- 172.21.15.119:0/3878282068 shutdown_connections 2023-12-07T10:36:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.374+0000 7f5c445ff700 1 -- 172.21.15.119:0/3878282068 wait complete. 2023-12-07T10:36:56.393 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-07T10:36:56.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:56 smithi119 ceph-mon[97908]: pgmap v518: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 12 op/s 2023-12-07T10:36:56.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:56 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2084344180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:36:56.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:56 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3991528095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:36:56.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:56 smithi159 ceph-mon[134726]: pgmap v518: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 12 op/s 2023-12-07T10:36:56.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:56 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2084344180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:36:56.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:56 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3991528095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:36:56.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:56 smithi153 ceph-mon[133625]: pgmap v518: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 12 op/s 2023-12-07T10:36:56.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:56 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2084344180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-07T10:36:56.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:56 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3991528095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-07T10:36:56.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.591+0000 7f70513ff700 1 -- 172.21.15.119:0/1557413476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f704c101ea0 msgr2=0x7f704c10e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:56.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.591+0000 7f70513ff700 1 --2- 172.21.15.119:0/1557413476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f704c101ea0 0x7f704c10e360 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7044009fc0 tx=0x7f704400b330 comp rx=0 tx=0).stop 2023-12-07T10:36:56.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.592+0000 7f70513ff700 1 -- 172.21.15.119:0/1557413476 shutdown_connections 2023-12-07T10:36:56.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.592+0000 7f70513ff700 1 --2- 172.21.15.119:0/1557413476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f704c101ea0 0x7f704c10e360 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:56.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.592+0000 7f70513ff700 1 --2- 172.21.15.119:0/1557413476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f704c1075a0 0x7f704c107980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:56.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.592+0000 7f70513ff700 1 --2- 172.21.15.119:0/1557413476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f704c1014e0 0x7f704c101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:56.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.592+0000 7f70513ff700 1 -- 172.21.15.119:0/1557413476 >> 172.21.15.119:0/1557413476 conn(0x7f704c0fcfb0 msgr2=0x7f704c0ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:56.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.592+0000 7f70513ff700 1 -- 172.21.15.119:0/1557413476 shutdown_connections 2023-12-07T10:36:56.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.592+0000 7f70513ff700 1 -- 172.21.15.119:0/1557413476 wait complete. 2023-12-07T10:36:56.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.593+0000 7f70513ff700 1 Processor -- start 2023-12-07T10:36:56.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.593+0000 7f70513ff700 1 -- start start 2023-12-07T10:36:56.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.593+0000 7f70513ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f704c1014e0 0x7f704c19f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:56.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.593+0000 7f70513ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f704c101ea0 0x7f704c19fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:56.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.593+0000 7f70513ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f704c1075a0 0x7f704c199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:56.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.593+0000 7f70513ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c19a090 con 0x7f704c1014e0 2023-12-07T10:36:56.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.593+0000 7f70513ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c19a200 con 0x7f704c101ea0 2023-12-07T10:36:56.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.593+0000 7f70513ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c19a3c0 con 0x7f704c1075a0 2023-12-07T10:36:56.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.593+0000 7f704affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f704c1014e0 0x7f704c19f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:56.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.593+0000 7f704affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f704c1014e0 0x7f704c19f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58146/0 (socket says 172.21.15.119:58146) 2023-12-07T10:36:56.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.593+0000 7f704affd700 1 -- 172.21.15.119:0/1745165482 learned_addr learned my addr 172.21.15.119:0/1745165482 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:36:56.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.594+0000 7f704b7fe700 1 --2- 172.21.15.119:0/1745165482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f704c1075a0 0x7f704c199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:56.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.594+0000 7f704a7fc700 1 --2- 172.21.15.119:0/1745165482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f704c101ea0 0x7f704c19fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:56.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.594+0000 7f704affd700 1 -- 172.21.15.119:0/1745165482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f704c1075a0 msgr2=0x7f704c199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:56.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.594+0000 7f704affd700 1 --2- 172.21.15.119:0/1745165482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f704c1075a0 0x7f704c199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:56.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.594+0000 7f704affd700 1 -- 172.21.15.119:0/1745165482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f704c101ea0 msgr2=0x7f704c19fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:56.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.594+0000 7f704affd700 1 --2- 172.21.15.119:0/1745165482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f704c101ea0 0x7f704c19fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:56.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.594+0000 7f704affd700 1 -- 172.21.15.119:0/1745165482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7044009c40 con 0x7f704c1014e0 2023-12-07T10:36:56.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.594+0000 7f704affd700 1 --2- 172.21.15.119:0/1745165482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f704c1014e0 0x7f704c19f900 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f703c00e9c0 tx=0x7f703c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:56.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.594+0000 7f7033fff700 1 -- 172.21.15.119:0/1745165482 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c004a30 con 0x7f704c1014e0 2023-12-07T10:36:56.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.594+0000 7f7033fff700 1 -- 172.21.15.119:0/1745165482 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f703c00cc80 con 0x7f704c1014e0 2023-12-07T10:36:56.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.594+0000 7f70513ff700 1 -- 172.21.15.119:0/1745165482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f704c19a6a0 con 0x7f704c1014e0 2023-12-07T10:36:56.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.594+0000 7f7033fff700 1 -- 172.21.15.119:0/1745165482 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c0054e0 con 0x7f704c1014e0 2023-12-07T10:36:56.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.594+0000 7f70513ff700 1 -- 172.21.15.119:0/1745165482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f704c1a68f0 con 0x7f704c1014e0 2023-12-07T10:36:56.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.595+0000 7f70513ff700 1 -- 172.21.15.119:0/1745165482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f704c102c00 con 0x7f704c1014e0 2023-12-07T10:36:56.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.597+0000 7f7033fff700 1 -- 172.21.15.119:0/1745165482 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f703c004b90 con 0x7f704c1014e0 2023-12-07T10:36:56.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.599+0000 7f7033fff700 1 --2- 172.21.15.119:0/1745165482 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f70340a1c80 0x7f70340a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:56.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.600+0000 7f7033fff700 1 -- 172.21.15.119:0/1745165482 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f703c014070 con 0x7f704c1014e0 2023-12-07T10:36:56.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.600+0000 7f704a7fc700 1 --2- 172.21.15.119:0/1745165482 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f70340a1c80 0x7f70340a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:56.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.600+0000 7f704a7fc700 1 --2- 172.21.15.119:0/1745165482 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f70340a1c80 0x7f70340a4140 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f704c19abf0 tx=0x7f7040009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:56.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.604+0000 7f7033fff700 1 -- 172.21.15.119:0/1745165482 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f703c096b30 con 0x7f704c1014e0 2023-12-07T10:36:56.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.821+0000 7f70513ff700 1 -- 172.21.15.119:0/1745165482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f704c068350 con 0x7f704c1014e0 2023-12-07T10:36:56.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.821+0000 7f7033fff700 1 -- 172.21.15.119:0/1745165482 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v114) v1 ==== 81+0+959 (secure 0 0 0) 0x7f703c09a150 con 0x7f704c1014e0 2023-12-07T10:36:56.821 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/2490678270 2023-12-08T10:20:24.856450+0000 2023-12-07T10:36:56.822 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/2580180729 2023-12-08T10:20:24.856450+0000 2023-12-07T10:36:56.822 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/149289687 2023-12-08T10:20:24.856450+0000 2023-12-07T10:36:56.822 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/2050367825 2023-12-08T10:20:24.856450+0000 2023-12-07T10:36:56.822 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6800/1344846367 2023-12-08T10:20:24.856450+0000 2023-12-07T10:36:56.823 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/1628639392 2023-12-08T10:11:00.585565+0000 2023-12-07T10:36:56.823 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6801/1355617847 2023-12-08T10:11:00.585565+0000 2023-12-07T10:36:56.823 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6800/3515858087 2023-12-08T10:10:32.177930+0000 2023-12-07T10:36:56.823 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/362307836 2023-12-08T10:10:32.177930+0000 2023-12-07T10:36:56.823 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6801/1344846367 2023-12-08T10:20:24.856450+0000 2023-12-07T10:36:56.824 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/128964558 2023-12-08T10:11:00.585565+0000 2023-12-07T10:36:56.824 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/2278813554 2023-12-08T10:10:32.177930+0000 2023-12-07T10:36:56.824 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6801/3515858087 2023-12-08T10:10:32.177930+0000 2023-12-07T10:36:56.824 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/1411933614 2023-12-08T10:11:00.585565+0000 2023-12-07T10:36:56.824 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/1685698277 2023-12-08T10:10:32.177930+0000 2023-12-07T10:36:56.825 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6800/1355617847 2023-12-08T10:11:00.585565+0000 2023-12-07T10:36:56.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.824+0000 7f70513ff700 1 -- 172.21.15.119:0/1745165482 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f70340a1c80 msgr2=0x7f70340a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:56.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.824+0000 7f70513ff700 1 --2- 172.21.15.119:0/1745165482 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f70340a1c80 0x7f70340a4140 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f704c19abf0 tx=0x7f7040009260 comp rx=0 tx=0).stop 2023-12-07T10:36:56.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.824+0000 7f70513ff700 1 -- 172.21.15.119:0/1745165482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f704c1014e0 msgr2=0x7f704c19f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:56.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.824+0000 7f70513ff700 1 --2- 172.21.15.119:0/1745165482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f704c1014e0 0x7f704c19f900 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f703c00e9c0 tx=0x7f703c00ecd0 comp rx=0 tx=0).stop 2023-12-07T10:36:56.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.824+0000 7f70513ff700 1 -- 172.21.15.119:0/1745165482 shutdown_connections 2023-12-07T10:36:56.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.824+0000 7f70513ff700 1 --2- 172.21.15.119:0/1745165482 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f70340a1c80 0x7f70340a4140 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:56.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.824+0000 7f70513ff700 1 --2- 172.21.15.119:0/1745165482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f704c1075a0 0x7f704c199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:56.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.824+0000 7f70513ff700 1 --2- 172.21.15.119:0/1745165482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f704c1014e0 0x7f704c19f900 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:56.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.824+0000 7f70513ff700 1 --2- 172.21.15.119:0/1745165482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f704c101ea0 0x7f704c19fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:56.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.824+0000 7f70513ff700 1 -- 172.21.15.119:0/1745165482 >> 172.21.15.119:0/1745165482 conn(0x7f704c0fcfb0 msgr2=0x7f704c0fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:56.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.824+0000 7f70513ff700 1 -- 172.21.15.119:0/1745165482 shutdown_connections 2023-12-07T10:36:56.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:56.824+0000 7f70513ff700 1 -- 172.21.15.119:0/1745165482 wait complete. 2023-12-07T10:36:56.829 INFO:teuthology.orchestra.run.smithi119.stderr:listed 16 entries 2023-12-07T10:36:56.843 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-07T10:36:56.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:36:56.844 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-07T10:36:57.141 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-12-07T10:36:57.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:36:57.142 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-07T10:36:57.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:36:57.247 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-12-07T10:36:57.362 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-07T10:36:57.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:36:57.362 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-07T10:36:57.427 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:57 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3878282068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:36:57.427 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:57 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1745165482' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-07T10:36:57.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:36:57.437 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link delete ceph-brx 2023-12-07T10:36:57.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:57 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3878282068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:36:57.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:57 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1745165482' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-07T10:36:57.527 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:57 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3878282068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-07T10:36:57.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:57 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1745165482' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-07T10:36:57.552 INFO:tasks.kclient:Unmounting kernel clients... 2023-12-07T10:36:57.553 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-12-07T10:36:57.553 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-07T10:36:57.596 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348292k,nr_inodes=4087073,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275612k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/94bb8aa2c375f56012694782590080dec9ba5b314d15ac84afa4b081b2ebf79b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/AVUXTC4J7AELOWTQN75GDDVZ62:/var/lib/containers/storage/overlay/l/U3SWTXFJSRT4IG5EA2RCPB6S6Z,upperdir=/var/lib/containers/storage/overlay/94bb8aa2c375f56012694782590080dec9ba5b314d15ac84afa4b081b2ebf79b/diff,workdir=/var/lib/containers/storage/overlay/94bb8aa2c375f56012694782590080dec9ba5b314d15ac84afa4b081b2ebf79b/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/bbeb8f0c70f78e52f69dcffc3a76b5e5b7b2fa89e5aecc3364074fd24e7f7370/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/AVUXTC4J7AELOWTQN75GDDVZ62:/var/lib/containers/storage/overlay/l/U3SWTXFJSRT4IG5EA2RCPB6S6Z,upperdir=/var/lib/containers/storage/overlay/bbeb8f0c70f78e52f69dcffc3a76b5e5b7b2fa89e5aecc3364074fd24e7f7370/diff,workdir=/var/lib/containers/storage/overlay/bbeb8f0c70f78e52f69dcffc3a76b5e5b7b2fa89e5aecc3364074fd24e7f7370/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/fb938085bf938dca2ac12f760375de694a5aad90d0c97f7f3ed1a53ee9df19d1/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/AVUXTC4J7AELOWTQN75GDDVZ62:/var/lib/containers/storage/overlay/l/U3SWTXFJSRT4IG5EA2RCPB6S6Z,upperdir=/var/lib/containers/storage/overlay/fb938085bf938dca2ac12f760375de694a5aad90d0c97f7f3ed1a53ee9df19d1/diff,workdir=/var/lib/containers/storage/overlay/fb938085bf938dca2ac12f760375de694a5aad90d0c97f7f3ed1a53ee9df19d1/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e3b6c058f929cffaacc05317ef3ade43152453a2853efdf257f132518c8b30f2/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/AVUXTC4J7AELOWTQN75GDDVZ62:/var/lib/containers/storage/overlay/l/U3SWTXFJSRT4IG5EA2RCPB6S6Z,upperdir=/var/lib/containers/storage/overlay/e3b6c058f929cffaacc05317ef3ade43152453a2853efdf257f132518c8b30f2/diff,workdir=/var/lib/containers/storage/overlay/e3b6c058f929cffaacc05317ef3ade43152453a2853efdf257f132518c8b30f2/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/db1482fa00767390e0caa938e2d2329eebcf39230121e249f9e9682e887d19b7/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/AVUXTC4J7AELOWTQN75GDDVZ62:/var/lib/containers/storage/overlay/l/U3SWTXFJSRT4IG5EA2RCPB6S6Z,upperdir=/var/lib/containers/storage/overlay/db1482fa00767390e0caa938e2d2329eebcf39230121e249f9e9682e887d19b7/diff,workdir=/var/lib/containers/storage/overlay/db1482fa00767390e0caa938e2d2329eebcf39230121e249f9e9682e887d19b7/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/2230e27a3e49e9b75348d545fa1f2a58c1f1193297d29f7b48c8ca08cd0e59e6/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/AVUXTC4J7AELOWTQN75GDDVZ62:/var/lib/containers/storage/overlay/l/U3SWTXFJSRT4IG5EA2RCPB6S6Z,upperdir=/var/lib/containers/storage/overlay/2230e27a3e49e9b75348d545fa1f2a58c1f1193297d29f7b48c8ca08cd0e59e6/diff,workdir=/var/lib/containers/storage/overlay/2230e27a3e49e9b75348d545fa1f2a58c1f1193297d29f7b48c8ca08cd0e59e6/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/760f8af43848714c12d17dd65ca1a5e8f15925bd6ba1cb1c3182f3c918d80f15/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/AVUXTC4J7AELOWTQN75GDDVZ62:/var/lib/containers/storage/overlay/l/U3SWTXFJSRT4IG5EA2RCPB6S6Z,upperdir=/var/lib/containers/storage/overlay/760f8af43848714c12d17dd65ca1a5e8f15925bd6ba1cb1c3182f3c918d80f15/diff,workdir=/var/lib/containers/storage/overlay/760f8af43848714c12d17dd65ca1a5e8f15925bd6ba1cb1c3182f3c918d80f15/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/8fae3477a54b4cd3db1cd04e10b4f8d31b8d36c6c7fe73c5cb64304ed9298f8c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/AVUXTC4J7AELOWTQN75GDDVZ62:/var/lib/containers/storage/overlay/l/U3SWTXFJSRT4IG5EA2RCPB6S6Z,upperdir=/var/lib/containers/storage/overlay/8fae3477a54b4cd3db1cd04e10b4f8d31b8d36c6c7fe73c5cb64304ed9298f8c/diff,workdir=/var/lib/containers/storage/overlay/8fae3477a54b4cd3db1cd04e10b4f8d31b8d36c6c7fe73c5cb64304ed9298f8c/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9841295f6850ed447d649104b3a3e5338a38b821da290e4a12f4c84938c11b9c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/AVUXTC4J7AELOWTQN75GDDVZ62:/var/lib/containers/storage/overlay/l/U3SWTXFJSRT4IG5EA2RCPB6S6Z,upperdir=/var/lib/containers/storage/overlay/9841295f6850ed447d649104b3a3e5338a38b821da290e4a12f4c84938c11b9c/diff,workdir=/var/lib/containers/storage/overlay/9841295f6850ed447d649104b3a3e5338a38b821da290e4a12f4c84938c11b9c/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e1a9a41ea1213b79e9159322ec5a6ae50c4f0e54d055cabbe1c3e8064731eb91/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/AVUXTC4J7AELOWTQN75GDDVZ62:/var/lib/containers/storage/overlay/l/U3SWTXFJSRT4IG5EA2RCPB6S6Z,upperdir=/var/lib/containers/storage/overlay/e1a9a41ea1213b79e9159322ec5a6ae50c4f0e54d055cabbe1c3e8064731eb91/diff,workdir=/var/lib/containers/storage/overlay/e1a9a41ea1213b79e9159322ec5a6ae50c4f0e54d055cabbe1c3e8064731eb91/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-12-07T10:36:57.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:36:57.596 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-12-07T10:36:57.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-07T10:36:57.682 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link delete ceph-brx 2023-12-07T10:36:57.856 INFO:teuthology.orchestra.run.smithi119.stderr:Cannot find device "ceph-brx" 2023-12-07T10:36:57.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-07T10:36:57.859 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-12-07T10:36:57.871 INFO:tasks.cephadm:Teardown begin 2023-12-07T10:36:57.871 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-07T10:36:57.872 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:36:58.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.074+0000 7f5d1b1ff700 1 -- 172.21.15.119:0/3189412032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d14101ea0 msgr2=0x7f5d1410e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:58.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.074+0000 7f5d1b1ff700 1 --2- 172.21.15.119:0/3189412032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d14101ea0 0x7f5d1410e360 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c00b0d0 tx=0x7f5d0c00b3e0 comp rx=0 tx=0).stop 2023-12-07T10:36:58.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.074+0000 7f5d1b1ff700 1 -- 172.21.15.119:0/3189412032 shutdown_connections 2023-12-07T10:36:58.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.074+0000 7f5d1b1ff700 1 --2- 172.21.15.119:0/3189412032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d14101ea0 0x7f5d1410e360 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:58.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.074+0000 7f5d1b1ff700 1 --2- 172.21.15.119:0/3189412032 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d141075a0 0x7f5d14107980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:58.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.074+0000 7f5d1b1ff700 1 --2- 172.21.15.119:0/3189412032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d141014e0 0x7f5d14101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:58.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.074+0000 7f5d1b1ff700 1 -- 172.21.15.119:0/3189412032 >> 172.21.15.119:0/3189412032 conn(0x7f5d140fcf90 msgr2=0x7f5d140ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:58.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.074+0000 7f5d1b1ff700 1 -- 172.21.15.119:0/3189412032 shutdown_connections 2023-12-07T10:36:58.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.074+0000 7f5d1b1ff700 1 -- 172.21.15.119:0/3189412032 wait complete. 2023-12-07T10:36:58.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.075+0000 7f5d1b1ff700 1 Processor -- start 2023-12-07T10:36:58.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.075+0000 7f5d1b1ff700 1 -- start start 2023-12-07T10:36:58.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.075+0000 7f5d1b1ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d141014e0 0x7f5d1419f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:58.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.076+0000 7f5d1b1ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d14101ea0 0x7f5d1419fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:58.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.076+0000 7f5d1b1ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d141075a0 0x7f5d14199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:58.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.076+0000 7f5d1b1ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1419a0a0 con 0x7f5d14101ea0 2023-12-07T10:36:58.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.076+0000 7f5d1b1ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1419a210 con 0x7f5d141075a0 2023-12-07T10:36:58.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.076+0000 7f5d1b1ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1419a3d0 con 0x7f5d141014e0 2023-12-07T10:36:58.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.076+0000 7f5d13fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d14101ea0 0x7f5d1419fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:58.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.076+0000 7f5d13fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d14101ea0 0x7f5d1419fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58164/0 (socket says 172.21.15.119:58164) 2023-12-07T10:36:58.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.076+0000 7f5d13fff700 1 -- 172.21.15.119:0/3833815927 learned_addr learned my addr 172.21.15.119:0/3833815927 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:36:58.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.076+0000 7f5d18f9b700 1 --2- 172.21.15.119:0/3833815927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d141014e0 0x7f5d1419f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:58.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.076+0000 7f5d1979c700 1 --2- 172.21.15.119:0/3833815927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d141075a0 0x7f5d14199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:58.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.076+0000 7f5d13fff700 1 -- 172.21.15.119:0/3833815927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d141014e0 msgr2=0x7f5d1419f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:58.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.076+0000 7f5d13fff700 1 --2- 172.21.15.119:0/3833815927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d141014e0 0x7f5d1419f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:58.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.076+0000 7f5d13fff700 1 -- 172.21.15.119:0/3833815927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d141075a0 msgr2=0x7f5d14199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:58.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.076+0000 7f5d13fff700 1 --2- 172.21.15.119:0/3833815927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d141075a0 0x7f5d14199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:58.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.076+0000 7f5d13fff700 1 -- 172.21.15.119:0/3833815927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0c009d00 con 0x7f5d14101ea0 2023-12-07T10:36:58.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.076+0000 7f5d13fff700 1 --2- 172.21.15.119:0/3833815927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d14101ea0 0x7f5d1419fe50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000e9a0 tx=0x7f5d0000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:58.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.077+0000 7f5d11ffb700 1 -- 172.21.15.119:0/3833815927 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d00009790 con 0x7f5d14101ea0 2023-12-07T10:36:58.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.077+0000 7f5d11ffb700 1 -- 172.21.15.119:0/3833815927 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d0000cbd0 con 0x7f5d14101ea0 2023-12-07T10:36:58.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.077+0000 7f5d1b1ff700 1 -- 172.21.15.119:0/3833815927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d1419a6b0 con 0x7f5d14101ea0 2023-12-07T10:36:58.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.077+0000 7f5d11ffb700 1 -- 172.21.15.119:0/3833815927 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d00018680 con 0x7f5d14101ea0 2023-12-07T10:36:58.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.077+0000 7f5d1b1ff700 1 -- 172.21.15.119:0/3833815927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d141a6900 con 0x7f5d14101ea0 2023-12-07T10:36:58.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.078+0000 7f5d1b1ff700 1 -- 172.21.15.119:0/3833815927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d14102c00 con 0x7f5d14101ea0 2023-12-07T10:36:58.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.080+0000 7f5d11ffb700 1 -- 172.21.15.119:0/3833815927 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f5d0000cd40 con 0x7f5d14101ea0 2023-12-07T10:36:58.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.082+0000 7f5d11ffb700 1 --2- 172.21.15.119:0/3833815927 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5d040a1c80 0x7f5d040a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:58.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.082+0000 7f5d11ffb700 1 -- 172.21.15.119:0/3833815927 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f5d00014070 con 0x7f5d14101ea0 2023-12-07T10:36:58.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.082+0000 7f5d18f9b700 1 --2- 172.21.15.119:0/3833815927 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5d040a1c80 0x7f5d040a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:58.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.082+0000 7f5d18f9b700 1 --2- 172.21.15.119:0/3833815927 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5d040a1c80 0x7f5d040a4140 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009c60 tx=0x7f5d08009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:58.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.085+0000 7f5d11ffb700 1 -- 172.21.15.119:0/3833815927 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5d000522f0 con 0x7f5d14101ea0 2023-12-07T10:36:58.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.339+0000 7f5d1b1ff700 1 -- 172.21.15.119:0/3833815927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d14068350 con 0x7f5d14101ea0 2023-12-07T10:36:58.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.340+0000 7f5d11ffb700 1 -- 172.21.15.119:0/3833815927 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+11392 (secure 0 0 0) 0x7f5d00096d30 con 0x7f5d14101ea0 2023-12-07T10:36:58.340 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:36:58.340 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":80,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-07T10:17:41.278311+0000","modified":"2023-12-07T10:36:53.580123+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24517,"mds_3":24505,"mds_4":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14535":{"gid":14535,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6841/1791998028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1791998028},{"type":"v1","addr":"172.21.15.119:6841","nonce":1791998028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24505":{"gid":24505,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":106,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/2925503246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2925503246},{"type":"v1","addr":"172.21.15.153:6837","nonce":2925503246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6839/3984962900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3984962900},{"type":"v1","addr":"172.21.15.159:6839","nonce":3984962900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.119:6839/1920762602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1920762602},{"type":"v1","addr":"172.21.15.119:6839","nonce":1920762602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/566825184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":566825184},{"type":"v1","addr":"172.21.15.153:6839","nonce":566825184}]},"join_fscid":-1,"export_targets":[1,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-07T10:36:58.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.343+0000 7f5d1b1ff700 1 -- 172.21.15.119:0/3833815927 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5d040a1c80 msgr2=0x7f5d040a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:58.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.343+0000 7f5d1b1ff700 1 --2- 172.21.15.119:0/3833815927 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5d040a1c80 0x7f5d040a4140 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009c60 tx=0x7f5d08009260 comp rx=0 tx=0).stop 2023-12-07T10:36:58.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.343+0000 7f5d1b1ff700 1 -- 172.21.15.119:0/3833815927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d14101ea0 msgr2=0x7f5d1419fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:58.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.343+0000 7f5d1b1ff700 1 --2- 172.21.15.119:0/3833815927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d14101ea0 0x7f5d1419fe50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000e9a0 tx=0x7f5d0000ecb0 comp rx=0 tx=0).stop 2023-12-07T10:36:58.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.343+0000 7f5d1b1ff700 1 -- 172.21.15.119:0/3833815927 shutdown_connections 2023-12-07T10:36:58.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.343+0000 7f5d1b1ff700 1 --2- 172.21.15.119:0/3833815927 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f5d040a1c80 0x7f5d040a4140 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:58.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.343+0000 7f5d1b1ff700 1 --2- 172.21.15.119:0/3833815927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d141075a0 0x7f5d14199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:58.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.343+0000 7f5d1b1ff700 1 --2- 172.21.15.119:0/3833815927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d141014e0 0x7f5d1419f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:58.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.343+0000 7f5d1b1ff700 1 --2- 172.21.15.119:0/3833815927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d14101ea0 0x7f5d1419fe50 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:58.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.343+0000 7f5d1b1ff700 1 -- 172.21.15.119:0/3833815927 >> 172.21.15.119:0/3833815927 conn(0x7f5d140fcf90 msgr2=0x7f5d140fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:58.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.343+0000 7f5d1b1ff700 1 -- 172.21.15.119:0/3833815927 shutdown_connections 2023-12-07T10:36:58.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.343+0000 7f5d1b1ff700 1 -- 172.21.15.119:0/3833815927 wait complete. 2023-12-07T10:36:58.345 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 80 2023-12-07T10:36:58.362 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-07T10:36:58.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:58 smithi159 ceph-mon[134726]: pgmap v519: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 46 KiB/s wr, 16 op/s 2023-12-07T10:36:58.527 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:58 smithi153 ceph-mon[133625]: pgmap v519: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 46 KiB/s wr, 16 op/s 2023-12-07T10:36:58.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.560+0000 7f9c18bff700 1 -- 172.21.15.119:0/3036024236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c141075a0 msgr2=0x7f9c14107980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:58.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.560+0000 7f9c18bff700 1 --2- 172.21.15.119:0/3036024236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c141075a0 0x7f9c14107980 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9c040099c0 tx=0x7f9c04009cd0 comp rx=0 tx=0).stop 2023-12-07T10:36:58.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.560+0000 7f9c18bff700 1 -- 172.21.15.119:0/3036024236 shutdown_connections 2023-12-07T10:36:58.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.560+0000 7f9c18bff700 1 --2- 172.21.15.119:0/3036024236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c14101ea0 0x7f9c1410e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:58.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.560+0000 7f9c18bff700 1 --2- 172.21.15.119:0/3036024236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c141075a0 0x7f9c14107980 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:58.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.560+0000 7f9c18bff700 1 --2- 172.21.15.119:0/3036024236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c141014e0 0x7f9c14101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:58.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.560+0000 7f9c18bff700 1 -- 172.21.15.119:0/3036024236 >> 172.21.15.119:0/3036024236 conn(0x7f9c140fcf90 msgr2=0x7f9c140ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:58.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.560+0000 7f9c18bff700 1 -- 172.21.15.119:0/3036024236 shutdown_connections 2023-12-07T10:36:58.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.560+0000 7f9c18bff700 1 -- 172.21.15.119:0/3036024236 wait complete. 2023-12-07T10:36:58.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.561+0000 7f9c18bff700 1 Processor -- start 2023-12-07T10:36:58.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.561+0000 7f9c18bff700 1 -- start start 2023-12-07T10:36:58.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.562+0000 7f9c18bff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c141014e0 0x7f9c1419d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:58.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.562+0000 7f9c18bff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c14101ea0 0x7f9c1419dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:58.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.562+0000 7f9c18bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c141075a0 0x7f9c141976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:58.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.562+0000 7f9c18bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c14197d10 con 0x7f9c141075a0 2023-12-07T10:36:58.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.562+0000 7f9c18bff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c14197e80 con 0x7f9c141014e0 2023-12-07T10:36:58.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.562+0000 7f9c18bff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c14198040 con 0x7f9c14101ea0 2023-12-07T10:36:58.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.562+0000 7f9c11d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c14101ea0 0x7f9c1419dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:58.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.562+0000 7f9c12d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c141075a0 0x7f9c141976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:58.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.562+0000 7f9c11d9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c14101ea0 0x7f9c1419dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.119:36058/0 (socket says 172.21.15.119:36058) 2023-12-07T10:36:58.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.562+0000 7f9c11d9b700 1 -- 172.21.15.119:0/2029424549 learned_addr learned my addr 172.21.15.119:0/2029424549 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:36:58.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.562+0000 7f9c1259c700 1 --2- 172.21.15.119:0/2029424549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c141014e0 0x7f9c1419d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:58.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.562+0000 7f9c12d9d700 1 -- 172.21.15.119:0/2029424549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c14101ea0 msgr2=0x7f9c1419dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:58.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.563+0000 7f9c12d9d700 1 --2- 172.21.15.119:0/2029424549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c14101ea0 0x7f9c1419dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:58.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.563+0000 7f9c12d9d700 1 -- 172.21.15.119:0/2029424549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c141014e0 msgr2=0x7f9c1419d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:58.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.563+0000 7f9c12d9d700 1 --2- 172.21.15.119:0/2029424549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c141014e0 0x7f9c1419d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:58.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.563+0000 7f9c12d9d700 1 -- 172.21.15.119:0/2029424549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c040095f0 con 0x7f9c141075a0 2023-12-07T10:36:58.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.563+0000 7f9c11d9b700 1 --2- 172.21.15.119:0/2029424549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c14101ea0 0x7f9c1419dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:36:58.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.563+0000 7f9c12d9d700 1 --2- 172.21.15.119:0/2029424549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c141075a0 0x7f9c141976b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c00cc50 tx=0x7f9c0c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:58.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.563+0000 7f9c037fe700 1 -- 172.21.15.119:0/2029424549 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c00aa90 con 0x7f9c141075a0 2023-12-07T10:36:58.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.563+0000 7f9c037fe700 1 -- 172.21.15.119:0/2029424549 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c0c00abf0 con 0x7f9c141075a0 2023-12-07T10:36:58.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.563+0000 7f9c037fe700 1 -- 172.21.15.119:0/2029424549 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c0115e0 con 0x7f9c141075a0 2023-12-07T10:36:58.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.563+0000 7f9c18bff700 1 -- 172.21.15.119:0/2029424549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c14198320 con 0x7f9c141075a0 2023-12-07T10:36:58.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.563+0000 7f9c18bff700 1 -- 172.21.15.119:0/2029424549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c141a4570 con 0x7f9c141075a0 2023-12-07T10:36:58.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.564+0000 7f9c18bff700 1 -- 172.21.15.119:0/2029424549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c14109a20 con 0x7f9c141075a0 2023-12-07T10:36:58.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.566+0000 7f9c037fe700 1 -- 172.21.15.119:0/2029424549 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f9c0c012420 con 0x7f9c141075a0 2023-12-07T10:36:58.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.567+0000 7f9c037fe700 1 --2- 172.21.15.119:0/2029424549 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9bfc0a1c80 0x7f9bfc0a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:58.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.568+0000 7f9c037fe700 1 -- 172.21.15.119:0/2029424549 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f9c0c0cdd00 con 0x7f9c141075a0 2023-12-07T10:36:58.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.568+0000 7f9c1259c700 1 --2- 172.21.15.119:0/2029424549 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9bfc0a1c80 0x7f9bfc0a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:58.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.568+0000 7f9c1259c700 1 --2- 172.21.15.119:0/2029424549 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9bfc0a1c80 0x7f9bfc0a4140 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009990 tx=0x7f9c04003680 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:58.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.571+0000 7f9c037fe700 1 -- 172.21.15.119:0/2029424549 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9c0c096b40 con 0x7f9c141075a0 2023-12-07T10:36:58.727 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:58 smithi119 ceph-mon[97908]: pgmap v519: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 46 KiB/s wr, 16 op/s 2023-12-07T10:36:58.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.825+0000 7f9c18bff700 1 -- 172.21.15.119:0/2029424549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c14102c00 con 0x7f9c141075a0 2023-12-07T10:36:58.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.826+0000 7f9c037fe700 1 -- 172.21.15.119:0/2029424549 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+11392 (secure 0 0 0) 0x7f9c0c09a160 con 0x7f9c141075a0 2023-12-07T10:36:58.826 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:36:58.826 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1556173472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1556173472},{"type":"v1","addr":"172.21.15.119:6835","nonce":1556173472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/184007634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":184007634},{"type":"v1","addr":"172.21.15.119:6837","nonce":184007634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":80,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-07T10:17:41.278311+0000","modified":"2023-12-07T10:36:53.580123+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24517,"mds_3":24505,"mds_4":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14535":{"gid":14535,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6841/1791998028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6840","nonce":1791998028},{"type":"v1","addr":"172.21.15.119:6841","nonce":1791998028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24445":{"gid":24445,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/935461464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":935461464},{"type":"v1","addr":"172.21.15.159:6833","nonce":935461464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6833/209059922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":209059922},{"type":"v1","addr":"172.21.15.153:6833","nonce":209059922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.153:6835/1295649464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1295649464},{"type":"v1","addr":"172.21.15.153:6835","nonce":1295649464}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24505":{"gid":24505,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":106,"addr":"172.21.15.159:6835/2406380792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2406380792},{"type":"v1","addr":"172.21.15.159:6835","nonce":2406380792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.159:6837/3106097520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3106097520},{"type":"v1","addr":"172.21.15.159:6837","nonce":3106097520}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6837/2925503246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6836","nonce":2925503246},{"type":"v1","addr":"172.21.15.153:6837","nonce":2925503246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6839/3984962900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":3984962900},{"type":"v1","addr":"172.21.15.159:6839","nonce":3984962900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.119:6839/1920762602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1920762602},{"type":"v1","addr":"172.21.15.119:6839","nonce":1920762602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.153:6839/566825184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6838","nonce":566825184},{"type":"v1","addr":"172.21.15.153:6839","nonce":566825184}]},"join_fscid":-1,"export_targets":[1,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-07T10:36:58.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.828+0000 7f9c18bff700 1 -- 172.21.15.119:0/2029424549 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9bfc0a1c80 msgr2=0x7f9bfc0a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:58.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.828+0000 7f9c18bff700 1 --2- 172.21.15.119:0/2029424549 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9bfc0a1c80 0x7f9bfc0a4140 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009990 tx=0x7f9c04003680 comp rx=0 tx=0).stop 2023-12-07T10:36:58.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.828+0000 7f9c18bff700 1 -- 172.21.15.119:0/2029424549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c141075a0 msgr2=0x7f9c141976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:58.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.828+0000 7f9c18bff700 1 --2- 172.21.15.119:0/2029424549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c141075a0 0x7f9c141976b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c00cc50 tx=0x7f9c0c00cf60 comp rx=0 tx=0).stop 2023-12-07T10:36:58.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.828+0000 7f9c18bff700 1 -- 172.21.15.119:0/2029424549 shutdown_connections 2023-12-07T10:36:58.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.828+0000 7f9c18bff700 1 --2- 172.21.15.119:0/2029424549 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f9bfc0a1c80 0x7f9bfc0a4140 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:58.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.828+0000 7f9c18bff700 1 --2- 172.21.15.119:0/2029424549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c141075a0 0x7f9c141976b0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:58.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.828+0000 7f9c18bff700 1 --2- 172.21.15.119:0/2029424549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c141014e0 0x7f9c1419d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:58.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.828+0000 7f9c18bff700 1 --2- 172.21.15.119:0/2029424549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c14101ea0 0x7f9c1419dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:58.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.828+0000 7f9c18bff700 1 -- 172.21.15.119:0/2029424549 >> 172.21.15.119:0/2029424549 conn(0x7f9c140fcf90 msgr2=0x7f9c140fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:58.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.828+0000 7f9c18bff700 1 -- 172.21.15.119:0/2029424549 shutdown_connections 2023-12-07T10:36:58.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:58.828+0000 7f9c18bff700 1 -- 172.21.15.119:0/2029424549 wait complete. 2023-12-07T10:36:58.829 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 80 2023-12-07T10:36:58.848 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-07T10:36:59.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.046+0000 7fc879dff700 1 -- 172.21.15.119:0/3191653470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc874101ea0 msgr2=0x7fc87410e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:59.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.046+0000 7fc879dff700 1 --2- 172.21.15.119:0/3191653470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc874101ea0 0x7fc87410e360 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009fc0 tx=0x7fc86c00b330 comp rx=0 tx=0).stop 2023-12-07T10:36:59.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.046+0000 7fc879dff700 1 -- 172.21.15.119:0/3191653470 shutdown_connections 2023-12-07T10:36:59.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.046+0000 7fc879dff700 1 --2- 172.21.15.119:0/3191653470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc874101ea0 0x7fc87410e360 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:59.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.046+0000 7fc879dff700 1 --2- 172.21.15.119:0/3191653470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8741075a0 0x7fc874107980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:59.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.046+0000 7fc879dff700 1 --2- 172.21.15.119:0/3191653470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8741014e0 0x7fc874101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:59.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.046+0000 7fc879dff700 1 -- 172.21.15.119:0/3191653470 >> 172.21.15.119:0/3191653470 conn(0x7fc8740fcf90 msgr2=0x7fc8740ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:59.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.046+0000 7fc879dff700 1 -- 172.21.15.119:0/3191653470 shutdown_connections 2023-12-07T10:36:59.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.046+0000 7fc879dff700 1 -- 172.21.15.119:0/3191653470 wait complete. 2023-12-07T10:36:59.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.047+0000 7fc879dff700 1 Processor -- start 2023-12-07T10:36:59.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.047+0000 7fc879dff700 1 -- start start 2023-12-07T10:36:59.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.048+0000 7fc879dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8741014e0 0x7fc87419f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:59.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.048+0000 7fc879dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc874101ea0 0x7fc87419fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:59.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.048+0000 7fc879dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8741075a0 0x7fc874199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:59.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.048+0000 7fc879dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87419a0a0 con 0x7fc8741075a0 2023-12-07T10:36:59.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.048+0000 7fc879dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87419a210 con 0x7fc8741014e0 2023-12-07T10:36:59.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.048+0000 7fc879dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87419a3d0 con 0x7fc874101ea0 2023-12-07T10:36:59.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.048+0000 7fc873fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8741075a0 0x7fc874199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:59.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.048+0000 7fc873fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8741075a0 0x7fc874199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58214/0 (socket says 172.21.15.119:58214) 2023-12-07T10:36:59.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.048+0000 7fc873fff700 1 -- 172.21.15.119:0/2673635540 learned_addr learned my addr 172.21.15.119:0/2673635540 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:36:59.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.048+0000 7fc8737fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8741014e0 0x7fc87419f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:59.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.048+0000 7fc86bfff700 1 --2- 172.21.15.119:0/2673635540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc874101ea0 0x7fc87419fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:59.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.048+0000 7fc873fff700 1 -- 172.21.15.119:0/2673635540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc874101ea0 msgr2=0x7fc87419fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:59.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.048+0000 7fc873fff700 1 --2- 172.21.15.119:0/2673635540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc874101ea0 0x7fc87419fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:59.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.048+0000 7fc873fff700 1 -- 172.21.15.119:0/2673635540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8741014e0 msgr2=0x7fc87419f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:59.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.048+0000 7fc873fff700 1 --2- 172.21.15.119:0/2673635540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8741014e0 0x7fc87419f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:59.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.048+0000 7fc873fff700 1 -- 172.21.15.119:0/2673635540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86c009c40 con 0x7fc8741075a0 2023-12-07T10:36:59.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.049+0000 7fc873fff700 1 --2- 172.21.15.119:0/2673635540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8741075a0 0x7fc874199a40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc86c006070 tx=0x7fc86c009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:59.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.049+0000 7fc8717fa700 1 -- 172.21.15.119:0/2673635540 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c00e040 con 0x7fc8741075a0 2023-12-07T10:36:59.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.049+0000 7fc8717fa700 1 -- 172.21.15.119:0/2673635540 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc86c01b450 con 0x7fc8741075a0 2023-12-07T10:36:59.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.049+0000 7fc879dff700 1 -- 172.21.15.119:0/2673635540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc87419a650 con 0x7fc8741075a0 2023-12-07T10:36:59.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.049+0000 7fc8717fa700 1 -- 172.21.15.119:0/2673635540 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c0052a0 con 0x7fc8741075a0 2023-12-07T10:36:59.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.049+0000 7fc879dff700 1 -- 172.21.15.119:0/2673635540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8741a6900 con 0x7fc8741075a0 2023-12-07T10:36:59.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.050+0000 7fc879dff700 1 -- 172.21.15.119:0/2673635540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc874102c00 con 0x7fc8741075a0 2023-12-07T10:36:59.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.052+0000 7fc8717fa700 1 -- 172.21.15.119:0/2673635540 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fc86c01b5c0 con 0x7fc8741075a0 2023-12-07T10:36:59.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.054+0000 7fc8717fa700 1 --2- 172.21.15.119:0/2673635540 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fc8600a1c80 0x7fc8600a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:59.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.054+0000 7fc8717fa700 1 -- 172.21.15.119:0/2673635540 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7fc86c0cefb0 con 0x7fc8741075a0 2023-12-07T10:36:59.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.054+0000 7fc8737fe700 1 --2- 172.21.15.119:0/2673635540 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fc8600a1c80 0x7fc8600a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:59.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.055+0000 7fc8737fe700 1 --2- 172.21.15.119:0/2673635540 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fc8600a1c80 0x7fc8600a4140 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc864009c60 tx=0x7fc864009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:59.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.058+0000 7fc8717fa700 1 -- 172.21.15.119:0/2673635540 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc86c097d70 con 0x7fc8741075a0 2023-12-07T10:36:59.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.267+0000 7fc879dff700 1 -- 172.21.15.119:0/2673635540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc87419af00 con 0x7fc8741075a0 2023-12-07T10:36:59.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.269+0000 7fc8717fa700 1 -- 172.21.15.119:0/2673635540 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+25286 (secure 0 0 0) 0x7fc86c09b390 con 0x7fc8741075a0 2023-12-07T10:36:59.269 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-12-07T10:36:59.269 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":114,"fsid":"a4c642c6-94e8-11ee-95a2-87774f69a715","created":"2023-12-07T10:09:59.945863+0000","modified":"2023-12-07T10:36:21.052096+0000","last_up_change":"2023-12-07T10:16:50.929427+0000","last_in_change":"2023-12-07T10:16:33.558158+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-07T10:13:25.194366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-07T10:17:39.347813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-07T10:17:40.714501+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-07T10:17:44.257760+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"56b5818e-e625-40f1-8838-3ddcf2d34ea7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6803","nonce":2141950133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6805","nonce":2141950133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6809","nonce":2141950133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2141950133},{"type":"v1","addr":"172.21.15.119:6807","nonce":2141950133}]},"public_addr":"172.21.15.119:6803/2141950133","cluster_addr":"172.21.15.119:6805/2141950133","heartbeat_back_addr":"172.21.15.119:6809/2141950133","heartbeat_front_addr":"172.21.15.119:6807/2141950133","state":["exists","up"]},{"osd":1,"uuid":"36489a6a-5278-4802-9525-9871521722e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6801","nonce":3390875370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6803","nonce":3390875370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6807","nonce":3390875370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":3390875370},{"type":"v1","addr":"172.21.15.153:6805","nonce":3390875370}]},"public_addr":"172.21.15.153:6801/3390875370","cluster_addr":"172.21.15.153:6803/3390875370","heartbeat_back_addr":"172.21.15.153:6807/3390875370","heartbeat_front_addr":"172.21.15.153:6805/3390875370","state":["exists","up"]},{"osd":2,"uuid":"a937ad26-5e9b-4c8a-b65a-626a8feda6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6801","nonce":708101637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6803","nonce":708101637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6807","nonce":708101637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":708101637},{"type":"v1","addr":"172.21.15.159:6805","nonce":708101637}]},"public_addr":"172.21.15.159:6801/708101637","cluster_addr":"172.21.15.159:6803/708101637","heartbeat_back_addr":"172.21.15.159:6807/708101637","heartbeat_front_addr":"172.21.15.159:6805/708101637","state":["exists","up"]},{"osd":3,"uuid":"26ef8300-aa07-48f6-b31d-8b0f2217ffcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6811","nonce":3228047257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6813","nonce":3228047257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6817","nonce":3228047257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":3228047257},{"type":"v1","addr":"172.21.15.119:6815","nonce":3228047257}]},"public_addr":"172.21.15.119:6811/3228047257","cluster_addr":"172.21.15.119:6813/3228047257","heartbeat_back_addr":"172.21.15.119:6817/3228047257","heartbeat_front_addr":"172.21.15.119:6815/3228047257","state":["exists","up"]},{"osd":4,"uuid":"000ee4b9-9896-4901-8cc3-c85fdfc8cbaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6809","nonce":4021998299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6811","nonce":4021998299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6815","nonce":4021998299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":4021998299},{"type":"v1","addr":"172.21.15.153:6813","nonce":4021998299}]},"public_addr":"172.21.15.153:6809/4021998299","cluster_addr":"172.21.15.153:6811/4021998299","heartbeat_back_addr":"172.21.15.153:6815/4021998299","heartbeat_front_addr":"172.21.15.153:6813/4021998299","state":["exists","up"]},{"osd":5,"uuid":"8b2fc167-262d-474f-89f3-47bd29424421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6809","nonce":1832540768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6811","nonce":1832540768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6815","nonce":1832540768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":1832540768},{"type":"v1","addr":"172.21.15.159:6813","nonce":1832540768}]},"public_addr":"172.21.15.159:6809/1832540768","cluster_addr":"172.21.15.159:6811/1832540768","heartbeat_back_addr":"172.21.15.159:6815/1832540768","heartbeat_front_addr":"172.21.15.159:6813/1832540768","state":["exists","up"]},{"osd":6,"uuid":"f9aa6749-46cf-4b2a-8114-cbd034219d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6819","nonce":1162436840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6821","nonce":1162436840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6825","nonce":1162436840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1162436840},{"type":"v1","addr":"172.21.15.119:6823","nonce":1162436840}]},"public_addr":"172.21.15.119:6819/1162436840","cluster_addr":"172.21.15.119:6821/1162436840","heartbeat_back_addr":"172.21.15.119:6825/1162436840","heartbeat_front_addr":"172.21.15.119:6823/1162436840","state":["exists","up"]},{"osd":7,"uuid":"39216361-e1c4-4fef-86cb-a4707656b547","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6817","nonce":3254653457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6819","nonce":3254653457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6823","nonce":3254653457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":3254653457},{"type":"v1","addr":"172.21.15.153:6821","nonce":3254653457}]},"public_addr":"172.21.15.153:6817/3254653457","cluster_addr":"172.21.15.153:6819/3254653457","heartbeat_back_addr":"172.21.15.153:6823/3254653457","heartbeat_front_addr":"172.21.15.153:6821/3254653457","state":["exists","up"]},{"osd":8,"uuid":"cceb6ebc-7c02-48ea-9d16-b7a016dd449b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6817","nonce":2600946380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6819","nonce":2600946380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6823","nonce":2600946380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2600946380},{"type":"v1","addr":"172.21.15.159:6821","nonce":2600946380}]},"public_addr":"172.21.15.159:6817/2600946380","cluster_addr":"172.21.15.159:6819/2600946380","heartbeat_back_addr":"172.21.15.159:6823/2600946380","heartbeat_front_addr":"172.21.15.159:6821/2600946380","state":["exists","up"]},{"osd":9,"uuid":"f337a2e4-e065-4a9f-91d9-e3615de2d6f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6827","nonce":1940737285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6829","nonce":1940737285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6833","nonce":1940737285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1940737285},{"type":"v1","addr":"172.21.15.119:6831","nonce":1940737285}]},"public_addr":"172.21.15.119:6827/1940737285","cluster_addr":"172.21.15.119:6829/1940737285","heartbeat_back_addr":"172.21.15.119:6833/1940737285","heartbeat_front_addr":"172.21.15.119:6831/1940737285","state":["exists","up"]},{"osd":10,"uuid":"3a3bc347-85f4-4878-9797-d9da96137e56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6825","nonce":663834864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6827","nonce":663834864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6831","nonce":663834864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":663834864},{"type":"v1","addr":"172.21.15.153:6829","nonce":663834864}]},"public_addr":"172.21.15.153:6825/663834864","cluster_addr":"172.21.15.153:6827/663834864","heartbeat_back_addr":"172.21.15.153:6831/663834864","heartbeat_front_addr":"172.21.15.153:6829/663834864","state":["exists","up"]},{"osd":11,"uuid":"a0fc5909-279d-4172-977c-2dc2da0a7e34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6825","nonce":1262216348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6827","nonce":1262216348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6831","nonce":1262216348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1262216348},{"type":"v1","addr":"172.21.15.159:6829","nonce":1262216348}]},"public_addr":"172.21.15.159:6825/1262216348","cluster_addr":"172.21.15.159:6827/1262216348","heartbeat_back_addr":"172.21.15.159:6831/1262216348","heartbeat_front_addr":"172.21.15.159:6829/1262216348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:37.488109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:12:59.543060+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:21.381585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:13:47.112729+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:09.732150+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:30.893729+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:14:56.678710+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:19.004256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:15:40.388818+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:06.391734+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:28.394053+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-07T10:16:49.634348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6},{"from":5,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/2490678270":"2023-12-08T10:20:24.856450+0000","172.21.15.119:0/2580180729":"2023-12-08T10:20:24.856450+0000","172.21.15.119:0/149289687":"2023-12-08T10:20:24.856450+0000","172.21.15.119:0/2050367825":"2023-12-08T10:20:24.856450+0000","172.21.15.119:6800/1344846367":"2023-12-08T10:20:24.856450+0000","172.21.15.119:0/1628639392":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6801/1355617847":"2023-12-08T10:11:00.585565+0000","172.21.15.119:6800/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/362307836":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/1344846367":"2023-12-08T10:20:24.856450+0000","172.21.15.119:0/128964558":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/2278813554":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6801/3515858087":"2023-12-08T10:10:32.177930+0000","172.21.15.119:0/1411933614":"2023-12-08T10:11:00.585565+0000","172.21.15.119:0/1685698277":"2023-12-08T10:10:32.177930+0000","172.21.15.119:6800/1355617847":"2023-12-08T10:11:00.585565+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-07T10:36:59.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.271+0000 7fc879dff700 1 -- 172.21.15.119:0/2673635540 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fc8600a1c80 msgr2=0x7fc8600a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:59.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.271+0000 7fc879dff700 1 --2- 172.21.15.119:0/2673635540 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fc8600a1c80 0x7fc8600a4140 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc864009c60 tx=0x7fc864009260 comp rx=0 tx=0).stop 2023-12-07T10:36:59.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.271+0000 7fc879dff700 1 -- 172.21.15.119:0/2673635540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8741075a0 msgr2=0x7fc874199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:59.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.271+0000 7fc879dff700 1 --2- 172.21.15.119:0/2673635540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8741075a0 0x7fc874199a40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc86c006070 tx=0x7fc86c009490 comp rx=0 tx=0).stop 2023-12-07T10:36:59.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.272+0000 7fc879dff700 1 -- 172.21.15.119:0/2673635540 shutdown_connections 2023-12-07T10:36:59.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.272+0000 7fc879dff700 1 --2- 172.21.15.119:0/2673635540 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fc8600a1c80 0x7fc8600a4140 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:59.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.272+0000 7fc879dff700 1 --2- 172.21.15.119:0/2673635540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8741075a0 0x7fc874199a40 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:59.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.272+0000 7fc879dff700 1 --2- 172.21.15.119:0/2673635540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8741014e0 0x7fc87419f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:59.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.272+0000 7fc879dff700 1 --2- 172.21.15.119:0/2673635540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc874101ea0 0x7fc87419fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:59.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.272+0000 7fc879dff700 1 -- 172.21.15.119:0/2673635540 >> 172.21.15.119:0/2673635540 conn(0x7fc8740fcf90 msgr2=0x7fc8740fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:59.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.272+0000 7fc879dff700 1 -- 172.21.15.119:0/2673635540 shutdown_connections 2023-12-07T10:36:59.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.272+0000 7fc879dff700 1 -- 172.21.15.119:0/2673635540 wait complete. 2023-12-07T10:36:59.291 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-07T10:36:59.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.491+0000 7f03a7dff700 1 -- 172.21.15.119:0/874358113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03a0101ea0 msgr2=0x7f03a010e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:59.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.491+0000 7f03a7dff700 1 --2- 172.21.15.119:0/874358113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03a0101ea0 0x7f03a010e360 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f039800b0d0 tx=0x7f039800b3e0 comp rx=0 tx=0).stop 2023-12-07T10:36:59.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.491+0000 7f03a7dff700 1 -- 172.21.15.119:0/874358113 shutdown_connections 2023-12-07T10:36:59.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.491+0000 7f03a7dff700 1 --2- 172.21.15.119:0/874358113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03a0101ea0 0x7f03a010e360 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:59.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.491+0000 7f03a7dff700 1 --2- 172.21.15.119:0/874358113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03a01075a0 0x7f03a0107980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:59.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.491+0000 7f03a7dff700 1 --2- 172.21.15.119:0/874358113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a01014e0 0x7f03a0101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:59.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.491+0000 7f03a7dff700 1 -- 172.21.15.119:0/874358113 >> 172.21.15.119:0/874358113 conn(0x7f03a00fcfb0 msgr2=0x7f03a00ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:59.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.491+0000 7f03a7dff700 1 -- 172.21.15.119:0/874358113 shutdown_connections 2023-12-07T10:36:59.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.491+0000 7f03a7dff700 1 -- 172.21.15.119:0/874358113 wait complete. 2023-12-07T10:36:59.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.492+0000 7f03a7dff700 1 Processor -- start 2023-12-07T10:36:59.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.492+0000 7f03a7dff700 1 -- start start 2023-12-07T10:36:59.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.493+0000 7f03a7dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a01014e0 0x7f03a019f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:59.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.493+0000 7f03a7dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03a0101ea0 0x7f03a019fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:59.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.493+0000 7f03a7dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03a01075a0 0x7f03a0199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:59.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.493+0000 7f03a7dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a019a0b0 con 0x7f03a01075a0 2023-12-07T10:36:59.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.493+0000 7f03a7dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a019a220 con 0x7f03a01014e0 2023-12-07T10:36:59.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.493+0000 7f03a7dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a019a3e0 con 0x7f03a0101ea0 2023-12-07T10:36:59.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.493+0000 7f03a639c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03a01075a0 0x7f03a0199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:59.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.493+0000 7f03a639c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03a01075a0 0x7f03a0199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58238/0 (socket says 172.21.15.119:58238) 2023-12-07T10:36:59.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.493+0000 7f03a639c700 1 -- 172.21.15.119:0/377947734 learned_addr learned my addr 172.21.15.119:0/377947734 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:36:59.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.493+0000 7f03a539a700 1 --2- 172.21.15.119:0/377947734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03a0101ea0 0x7f03a019fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:59.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.493+0000 7f03a5b9b700 1 --2- 172.21.15.119:0/377947734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a01014e0 0x7f03a019f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:59.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.493+0000 7f03a639c700 1 -- 172.21.15.119:0/377947734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03a0101ea0 msgr2=0x7f03a019fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:59.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.493+0000 7f03a639c700 1 --2- 172.21.15.119:0/377947734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03a0101ea0 0x7f03a019fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:59.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.493+0000 7f03a639c700 1 -- 172.21.15.119:0/377947734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a01014e0 msgr2=0x7f03a019f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:59.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.493+0000 7f03a639c700 1 --2- 172.21.15.119:0/377947734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a01014e0 0x7f03a019f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:59.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.493+0000 7f03a639c700 1 -- 172.21.15.119:0/377947734 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0398009d00 con 0x7f03a01075a0 2023-12-07T10:36:59.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.494+0000 7f03a639c700 1 --2- 172.21.15.119:0/377947734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03a01075a0 0x7f03a0199a50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f039800bcd0 tx=0x7f039800bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:59.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.494+0000 7f0396ffd700 1 -- 172.21.15.119:0/377947734 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039800e040 con 0x7f03a01075a0 2023-12-07T10:36:59.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.494+0000 7f0396ffd700 1 -- 172.21.15.119:0/377947734 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0398005070 con 0x7f03a01075a0 2023-12-07T10:36:59.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.494+0000 7f03a7dff700 1 -- 172.21.15.119:0/377947734 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a019a660 con 0x7f03a01075a0 2023-12-07T10:36:59.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.494+0000 7f0396ffd700 1 -- 172.21.15.119:0/377947734 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039801b6a0 con 0x7f03a01075a0 2023-12-07T10:36:59.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.494+0000 7f03a7dff700 1 -- 172.21.15.119:0/377947734 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a01a6910 con 0x7f03a01075a0 2023-12-07T10:36:59.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.495+0000 7f03a7dff700 1 -- 172.21.15.119:0/377947734 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a0102c00 con 0x7f03a01075a0 2023-12-07T10:36:59.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.497+0000 7f0396ffd700 1 -- 172.21.15.119:0/377947734 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f03980051e0 con 0x7f03a01075a0 2023-12-07T10:36:59.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.498+0000 7f0396ffd700 1 --2- 172.21.15.119:0/377947734 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f038c0a1d00 0x7f038c0a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:59.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.498+0000 7f0396ffd700 1 -- 172.21.15.119:0/377947734 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9461+0+0 (secure 0 0 0) 0x7f03980ce790 con 0x7f03a01075a0 2023-12-07T10:36:59.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.498+0000 7f03a5b9b700 1 --2- 172.21.15.119:0/377947734 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f038c0a1d00 0x7f038c0a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:59.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.499+0000 7f03a5b9b700 1 --2- 172.21.15.119:0/377947734 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f038c0a1d00 0x7f038c0a41c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f039c009c60 tx=0x7f039c009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:59.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.502+0000 7f0396ffd700 1 -- 172.21.15.119:0/377947734 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f03980974f0 con 0x7f03a01075a0 2023-12-07T10:36:59.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:59 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3833815927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:36:59.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:36:59 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2029424549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:36:59.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:59 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3833815927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:36:59.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:36:59 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2029424549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:36:59.727 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:59 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3833815927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:36:59.727 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:59 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2029424549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-07T10:36:59.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.750+0000 7f03a7dff700 1 -- 172.21.15.119:0/377947734 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f03a0068350 con 0x7f03a01075a0 2023-12-07T10:36:59.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.753+0000 7f0396ffd700 1 -- 172.21.15.119:0/377947734 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v81) v1 ==== 161+0+0 (secure 0 0 0) 0x7f039809ab10 con 0x7f03a01075a0 2023-12-07T10:36:59.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.756+0000 7f03a7dff700 1 -- 172.21.15.119:0/377947734 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f038c0a1d00 msgr2=0x7f038c0a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:59.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.756+0000 7f03a7dff700 1 --2- 172.21.15.119:0/377947734 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f038c0a1d00 0x7f038c0a41c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f039c009c60 tx=0x7f039c009260 comp rx=0 tx=0).stop 2023-12-07T10:36:59.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.756+0000 7f03a7dff700 1 -- 172.21.15.119:0/377947734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03a01075a0 msgr2=0x7f03a0199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:59.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.756+0000 7f03a7dff700 1 --2- 172.21.15.119:0/377947734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03a01075a0 0x7f03a0199a50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f039800bcd0 tx=0x7f039800bd00 comp rx=0 tx=0).stop 2023-12-07T10:36:59.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.756+0000 7f03a7dff700 1 -- 172.21.15.119:0/377947734 shutdown_connections 2023-12-07T10:36:59.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.756+0000 7f03a7dff700 1 --2- 172.21.15.119:0/377947734 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f038c0a1d00 0x7f038c0a41c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:59.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.756+0000 7f03a7dff700 1 --2- 172.21.15.119:0/377947734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03a01075a0 0x7f03a0199a50 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:59.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.756+0000 7f03a7dff700 1 --2- 172.21.15.119:0/377947734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a01014e0 0x7f03a019f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:59.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.756+0000 7f03a7dff700 1 --2- 172.21.15.119:0/377947734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03a0101ea0 0x7f03a019fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:59.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.756+0000 7f03a7dff700 1 -- 172.21.15.119:0/377947734 >> 172.21.15.119:0/377947734 conn(0x7f03a00fcfb0 msgr2=0x7f03a00fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:59.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.756+0000 7f03a7dff700 1 -- 172.21.15.119:0/377947734 shutdown_connections 2023-12-07T10:36:59.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.756+0000 7f03a7dff700 1 -- 172.21.15.119:0/377947734 wait complete. 2023-12-07T10:36:59.759 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-07T10:36:59.778 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-07T10:36:59.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.993+0000 7fa11e9ff700 1 -- 172.21.15.119:0/4005926868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa118107620 msgr2=0x7fa118107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:59.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.993+0000 7fa11e9ff700 1 --2- 172.21.15.119:0/4005926868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa118107620 0x7fa118107a00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa1080099c0 tx=0x7fa108009cd0 comp rx=0 tx=0).stop 2023-12-07T10:36:59.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.993+0000 7fa11e9ff700 1 -- 172.21.15.119:0/4005926868 shutdown_connections 2023-12-07T10:36:59.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.993+0000 7fa11e9ff700 1 --2- 172.21.15.119:0/4005926868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa118101f20 0x7fa11810e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:59.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.993+0000 7fa11e9ff700 1 --2- 172.21.15.119:0/4005926868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa118107620 0x7fa118107a00 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:59.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.993+0000 7fa11e9ff700 1 --2- 172.21.15.119:0/4005926868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa118101560 0x7fa1181019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:59.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.993+0000 7fa11e9ff700 1 -- 172.21.15.119:0/4005926868 >> 172.21.15.119:0/4005926868 conn(0x7fa1180fcfd0 msgr2=0x7fa1180ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:36:59.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.993+0000 7fa11e9ff700 1 -- 172.21.15.119:0/4005926868 shutdown_connections 2023-12-07T10:36:59.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.993+0000 7fa11e9ff700 1 -- 172.21.15.119:0/4005926868 wait complete. 2023-12-07T10:36:59.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.994+0000 7fa11e9ff700 1 Processor -- start 2023-12-07T10:36:59.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.995+0000 7fa11e9ff700 1 -- start start 2023-12-07T10:36:59.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.995+0000 7fa11e9ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa118101560 0x7fa118073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:59.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.995+0000 7fa11e9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa118101f20 0x7fa118073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:59.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.995+0000 7fa11e9ff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa118107620 0x7fa118074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:36:59.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.995+0000 7fa11e9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1181a5e50 con 0x7fa118101f20 2023-12-07T10:36:59.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.995+0000 7fa11e9ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1181a5fc0 con 0x7fa118101560 2023-12-07T10:36:59.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.995+0000 7fa11e9ff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1181a6180 con 0x7fa118107620 2023-12-07T10:36:59.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.995+0000 7fa1177fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa118101f20 0x7fa118073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:59.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.995+0000 7fa117fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa118101560 0x7fa118073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:59.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.995+0000 7fa117fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa118101560 0x7fa118073910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.119:58876/0 (socket says 172.21.15.119:58876) 2023-12-07T10:36:59.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.995+0000 7fa117fff700 1 -- 172.21.15.119:0/1306689095 learned_addr learned my addr 172.21.15.119:0/1306689095 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:36:59.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.996+0000 7fa11cf9c700 1 --2- 172.21.15.119:0/1306689095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa118107620 0x7fa118074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:36:59.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.996+0000 7fa117fff700 1 -- 172.21.15.119:0/1306689095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa118107620 msgr2=0x7fa118074390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:59.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.996+0000 7fa117fff700 1 --2- 172.21.15.119:0/1306689095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa118107620 0x7fa118074390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:59.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.996+0000 7fa117fff700 1 -- 172.21.15.119:0/1306689095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa118101f20 msgr2=0x7fa118073e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:36:59.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.996+0000 7fa117fff700 1 --2- 172.21.15.119:0/1306689095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa118101f20 0x7fa118073e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:36:59.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.996+0000 7fa117fff700 1 -- 172.21.15.119:0/1306689095 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1080095f0 con 0x7fa118101560 2023-12-07T10:36:59.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.996+0000 7fa117fff700 1 --2- 172.21.15.119:0/1306689095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa118101560 0x7fa118073910 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa108005b40 tx=0x7fa10800ba30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:36:59.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.996+0000 7fa1157fa700 1 -- 172.21.15.119:0/1306689095 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10801d070 con 0x7fa118101560 2023-12-07T10:36:59.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.996+0000 7fa1157fa700 1 -- 172.21.15.119:0/1306689095 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa10800be70 con 0x7fa118101560 2023-12-07T10:36:59.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.996+0000 7fa11e9ff700 1 -- 172.21.15.119:0/1306689095 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1181a6400 con 0x7fa118101560 2023-12-07T10:36:59.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.996+0000 7fa1157fa700 1 -- 172.21.15.119:0/1306689095 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10800fb70 con 0x7fa118101560 2023-12-07T10:36:59.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.996+0000 7fa11e9ff700 1 -- 172.21.15.119:0/1306689095 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1181a6880 con 0x7fa118101560 2023-12-07T10:36:59.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.997+0000 7fa11e9ff700 1 -- 172.21.15.119:0/1306689095 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa118109aa0 con 0x7fa118101560 2023-12-07T10:37:00.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:36:59.999+0000 7fa1157fa700 1 -- 172.21.15.119:0/1306689095 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7fa108003680 con 0x7fa118101560 2023-12-07T10:37:00.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.001+0000 7fa1157fa700 1 --2- 172.21.15.119:0/1306689095 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa1000a1d00 0x7fa1000a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:37:00.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.001+0000 7fa1157fa700 1 -- 172.21.15.119:0/1306689095 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9803+0+0 (secure 0 0 0) 0x7fa1080ceff0 con 0x7fa118101560 2023-12-07T10:37:00.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.001+0000 7fa1177fe700 1 --2- 172.21.15.119:0/1306689095 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa1000a1d00 0x7fa1000a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:37:00.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.001+0000 7fa1177fe700 1 --2- 172.21.15.119:0/1306689095 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa1000a1d00 0x7fa1000a41c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa118075040 tx=0x7fa10c00b410 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:37:00.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.003+0000 7fa1157fa700 1 -- 172.21.15.119:0/1306689095 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa108097c20 con 0x7fa118101560 2023-12-07T10:37:00.227 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:36:59 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mon-a[97884]: 2023-12-07T10:36:59.751+0000 7fc36531c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-07T10:37:00.227 INFO:journalctl@ceph.mds.a.smithi119.stdout:Dec 07 10:36:59 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-a[146736]: ignoring --setuser ceph since I am not root 2023-12-07T10:37:00.227 INFO:journalctl@ceph.mds.a.smithi119.stdout:Dec 07 10:36:59 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-a[146736]: ignoring --setgroup ceph since I am not root 2023-12-07T10:37:00.227 INFO:journalctl@ceph.mds.a.smithi119.stdout:Dec 07 10:36:59 smithi119 ceph-mds[146760]: -- 172.21.15.119:0/902228965 <== mon.2 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d028f881a0 con 0x55d028f60c00 2023-12-07T10:37:00.228 INFO:journalctl@ceph.mds.a.smithi119.stdout:Dec 07 10:36:59 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-a[146736]: starting mds.a at 2023-12-07T10:37:00.228 INFO:journalctl@ceph.mds.g.smithi119.stdout:Dec 07 10:36:59 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-g[146351]: ignoring --setuser ceph since I am not root 2023-12-07T10:37:00.228 INFO:journalctl@ceph.mds.g.smithi119.stdout:Dec 07 10:36:59 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-g[146351]: ignoring --setgroup ceph since I am not root 2023-12-07T10:37:00.228 INFO:journalctl@ceph.mds.g.smithi119.stdout:Dec 07 10:36:59 smithi119 ceph-mds[146375]: -- 172.21.15.119:0/1380997539 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e5a72821a0 con 0x55e5a725a800 2023-12-07T10:37:00.228 INFO:journalctl@ceph.mds.g.smithi119.stdout:Dec 07 10:36:59 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-g[146351]: starting mds.g at 2023-12-07T10:37:00.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.252+0000 7fa11e9ff700 1 -- 172.21.15.119:0/1306689095 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa118102c80 con 0x7fa118101560 2023-12-07T10:37:00.260 INFO:journalctl@ceph.mds.c.smithi159.stdout:Dec 07 10:36:59 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-c[172250]: ignoring --setuser ceph since I am not root 2023-12-07T10:37:00.260 INFO:journalctl@ceph.mds.c.smithi159.stdout:Dec 07 10:36:59 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-c[172250]: ignoring --setgroup ceph since I am not root 2023-12-07T10:37:00.261 INFO:journalctl@ceph.mds.c.smithi159.stdout:Dec 07 10:36:59 smithi159 ceph-mds[172274]: -- 172.21.15.159:0/2170158103 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558eda3f61a0 con 0x558eda3ccc00 2023-12-07T10:37:00.261 INFO:journalctl@ceph.mds.c.smithi159.stdout:Dec 07 10:36:59 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-c[172250]: starting mds.c at 2023-12-07T10:37:00.261 INFO:journalctl@ceph.mds.i.smithi159.stdout:Dec 07 10:36:59 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-i[172585]: ignoring --setuser ceph since I am not root 2023-12-07T10:37:00.261 INFO:journalctl@ceph.mds.i.smithi159.stdout:Dec 07 10:36:59 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-i[172585]: ignoring --setgroup ceph since I am not root 2023-12-07T10:37:00.261 INFO:journalctl@ceph.mds.i.smithi159.stdout:Dec 07 10:36:59 smithi159 ceph-mds[172609]: -- 172.21.15.159:0/2232212001 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556f6885e1a0 con 0x556f68835000 2023-12-07T10:37:00.261 INFO:journalctl@ceph.mds.i.smithi159.stdout:Dec 07 10:36:59 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-i[172585]: starting mds.i at 2023-12-07T10:37:00.262 INFO:journalctl@ceph.mds.f.smithi159.stdout:Dec 07 10:36:59 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-f[171910]: ignoring --setuser ceph since I am not root 2023-12-07T10:37:00.262 INFO:journalctl@ceph.mds.f.smithi159.stdout:Dec 07 10:36:59 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-f[171910]: ignoring --setgroup ceph since I am not root 2023-12-07T10:37:00.262 INFO:journalctl@ceph.mds.f.smithi159.stdout:Dec 07 10:36:59 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-f[171910]: starting mds.f at 2023-12-07T10:37:00.262 INFO:journalctl@ceph.mds.l.smithi159.stdout:Dec 07 10:36:59 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-l[171572]: ignoring --setuser ceph since I am not root 2023-12-07T10:37:00.262 INFO:journalctl@ceph.mds.l.smithi159.stdout:Dec 07 10:36:59 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-l[171572]: ignoring --setgroup ceph since I am not root 2023-12-07T10:37:00.263 INFO:journalctl@ceph.mds.l.smithi159.stdout:Dec 07 10:36:59 smithi159 ceph-mds[171596]: -- 172.21.15.159:0/2836381862 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55656b0e01a0 con 0x55656b0b6c00 2023-12-07T10:37:00.263 INFO:journalctl@ceph.mds.l.smithi159.stdout:Dec 07 10:36:59 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-l[171572]: starting mds.l at 2023-12-07T10:37:00.263 INFO:journalctl@ceph.mds.e.smithi153.stdout:Dec 07 10:36:59 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-e[169759]: ignoring --setuser ceph since I am not root 2023-12-07T10:37:00.263 INFO:journalctl@ceph.mds.e.smithi153.stdout:Dec 07 10:36:59 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-e[169759]: ignoring --setgroup ceph since I am not root 2023-12-07T10:37:00.263 INFO:journalctl@ceph.mds.e.smithi153.stdout:Dec 07 10:36:59 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-e[169759]: starting mds.e at 2023-12-07T10:37:00.264 INFO:journalctl@ceph.mds.k.smithi153.stdout:Dec 07 10:36:59 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-k[169421]: ignoring --setuser ceph since I am not root 2023-12-07T10:37:00.264 INFO:journalctl@ceph.mds.k.smithi153.stdout:Dec 07 10:36:59 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-k[169421]: ignoring --setgroup ceph since I am not root 2023-12-07T10:37:00.264 INFO:journalctl@ceph.mds.k.smithi153.stdout:Dec 07 10:36:59 smithi153 ceph-mds[169445]: -- 172.21.15.153:0/3587554552 <== mon.2 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e1dd95a1a0 con 0x55e1dd930800 2023-12-07T10:37:00.264 INFO:journalctl@ceph.mds.k.smithi153.stdout:Dec 07 10:36:59 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-k[169421]: starting mds.k at 2023-12-07T10:37:00.264 INFO:journalctl@ceph.mds.h.smithi153.stdout:Dec 07 10:36:59 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-h[170103]: ignoring --setuser ceph since I am not root 2023-12-07T10:37:00.265 INFO:journalctl@ceph.mds.h.smithi153.stdout:Dec 07 10:36:59 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-h[170103]: ignoring --setgroup ceph since I am not root 2023-12-07T10:37:00.265 INFO:journalctl@ceph.mds.h.smithi153.stdout:Dec 07 10:36:59 smithi153 ceph-mds[170127]: -- 172.21.15.153:0/3576135025 <== mon.2 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562310c9a1a0 con 0x562310c70c00 2023-12-07T10:37:00.265 INFO:journalctl@ceph.mds.h.smithi153.stdout:Dec 07 10:36:59 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-h[170103]: starting mds.h at 2023-12-07T10:37:00.265 INFO:journalctl@ceph.mds.b.smithi153.stdout:Dec 07 10:36:59 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-b[170442]: ignoring --setuser ceph since I am not root 2023-12-07T10:37:00.265 INFO:journalctl@ceph.mds.b.smithi153.stdout:Dec 07 10:36:59 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-b[170442]: ignoring --setgroup ceph since I am not root 2023-12-07T10:37:00.265 INFO:journalctl@ceph.mds.b.smithi153.stdout:Dec 07 10:36:59 smithi153 ceph-mds[170466]: -- 172.21.15.153:0/1168817791 <== mon.2 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563d208901a0 con 0x563d20866800 2023-12-07T10:37:00.266 INFO:journalctl@ceph.mds.b.smithi153.stdout:Dec 07 10:36:59 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-b[170442]: starting mds.b at 2023-12-07T10:37:00.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:00 smithi159 ceph-mon[134726]: pgmap v520: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 64 KiB/s wr, 15 op/s 2023-12-07T10:37:00.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:00 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2673635540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:37:00.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:00 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/377947734' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-12-07T10:37:00.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:00 smithi159 ceph-mon[134726]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-12-07T10:37:00.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:00 smithi159 ceph-mon[134726]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-07T10:37:00.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:00 smithi159 ceph-mon[134726]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-12-07T10:37:00.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:00 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/377947734' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-12-07T10:37:00.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:00 smithi159 ceph-mon[134726]: osdmap e115: 12 total, 12 up, 12 in 2023-12-07T10:37:00.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:00 smithi159 ceph-mon[134726]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-12-07T10:37:00.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:00 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/1306689095' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-12-07T10:37:00.512 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:00 smithi159 ceph-mon[134726]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-12-07T10:37:00.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:00 smithi153 ceph-mon[133625]: pgmap v520: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 64 KiB/s wr, 15 op/s 2023-12-07T10:37:00.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:00 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2673635540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:37:00.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:00 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/377947734' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-12-07T10:37:00.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:00 smithi153 ceph-mon[133625]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-12-07T10:37:00.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:00 smithi153 ceph-mon[133625]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-07T10:37:00.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:00 smithi153 ceph-mon[133625]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-12-07T10:37:00.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:00 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/377947734' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-12-07T10:37:00.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:00 smithi153 ceph-mon[133625]: osdmap e115: 12 total, 12 up, 12 in 2023-12-07T10:37:00.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:00 smithi153 ceph-mon[133625]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-12-07T10:37:00.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:00 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/1306689095' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-12-07T10:37:00.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:00 smithi153 ceph-mon[133625]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-12-07T10:37:00.727 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:00 smithi119 ceph-mon[97908]: pgmap v520: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 64 KiB/s wr, 15 op/s 2023-12-07T10:37:00.727 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:00 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2673635540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-07T10:37:00.727 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:00 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/377947734' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-12-07T10:37:00.727 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:00 smithi119 ceph-mon[97908]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-12-07T10:37:00.728 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:00 smithi119 ceph-mon[97908]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-07T10:37:00.728 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:00 smithi119 ceph-mon[97908]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-12-07T10:37:00.728 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:00 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/377947734' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-12-07T10:37:00.728 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:00 smithi119 ceph-mon[97908]: osdmap e115: 12 total, 12 up, 12 in 2023-12-07T10:37:00.728 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:00 smithi119 ceph-mon[97908]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-12-07T10:37:00.728 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:00 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/1306689095' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-12-07T10:37:00.728 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:00 smithi119 ceph-mon[97908]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-12-07T10:37:00.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.760+0000 7fa1157fa700 1 -- 172.21.15.119:0/1306689095 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v82) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa10809b240 con 0x7fa118101560 2023-12-07T10:37:00.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.764+0000 7fa11e9ff700 1 -- 172.21.15.119:0/1306689095 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa1000a1d00 msgr2=0x7fa1000a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:37:00.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.764+0000 7fa11e9ff700 1 --2- 172.21.15.119:0/1306689095 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa1000a1d00 0x7fa1000a41c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa118075040 tx=0x7fa10c00b410 comp rx=0 tx=0).stop 2023-12-07T10:37:00.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.764+0000 7fa11e9ff700 1 -- 172.21.15.119:0/1306689095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa118101560 msgr2=0x7fa118073910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:37:00.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.764+0000 7fa11e9ff700 1 --2- 172.21.15.119:0/1306689095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa118101560 0x7fa118073910 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa108005b40 tx=0x7fa10800ba30 comp rx=0 tx=0).stop 2023-12-07T10:37:00.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.765+0000 7fa11e9ff700 1 -- 172.21.15.119:0/1306689095 shutdown_connections 2023-12-07T10:37:00.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.765+0000 7fa11e9ff700 1 --2- 172.21.15.119:0/1306689095 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7fa1000a1d00 0x7fa1000a41c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:00.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.765+0000 7fa11e9ff700 1 --2- 172.21.15.119:0/1306689095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa118107620 0x7fa118074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:00.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.765+0000 7fa11e9ff700 1 --2- 172.21.15.119:0/1306689095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa118101560 0x7fa118073910 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:00.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.765+0000 7fa11e9ff700 1 --2- 172.21.15.119:0/1306689095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa118101f20 0x7fa118073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:00.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.765+0000 7fa11e9ff700 1 -- 172.21.15.119:0/1306689095 >> 172.21.15.119:0/1306689095 conn(0x7fa1180fcfd0 msgr2=0x7fa1180fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:37:00.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.766+0000 7fa11e9ff700 1 -- 172.21.15.119:0/1306689095 shutdown_connections 2023-12-07T10:37:00.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.766+0000 7fa11e9ff700 1 -- 172.21.15.119:0/1306689095 wait complete. 2023-12-07T10:37:00.786 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-07T10:37:00.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.992+0000 7f4f23dff700 1 -- 172.21.15.119:0/1061528226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f1c107620 msgr2=0x7f4f1c107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:37:00.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.992+0000 7f4f23dff700 1 --2- 172.21.15.119:0/1061528226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f1c107620 0x7f4f1c107a00 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4f180099c0 tx=0x7f4f18009cd0 comp rx=0 tx=0).stop 2023-12-07T10:37:00.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.992+0000 7f4f23dff700 1 -- 172.21.15.119:0/1061528226 shutdown_connections 2023-12-07T10:37:00.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.992+0000 7f4f23dff700 1 --2- 172.21.15.119:0/1061528226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f1c101f20 0x7f4f1c10e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:00.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.992+0000 7f4f23dff700 1 --2- 172.21.15.119:0/1061528226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f1c107620 0x7f4f1c107a00 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:00.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.992+0000 7f4f23dff700 1 --2- 172.21.15.119:0/1061528226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c101560 0x7f4f1c1019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:00.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.992+0000 7f4f23dff700 1 -- 172.21.15.119:0/1061528226 >> 172.21.15.119:0/1061528226 conn(0x7f4f1c0fcfd0 msgr2=0x7f4f1c0ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:37:00.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.992+0000 7f4f23dff700 1 -- 172.21.15.119:0/1061528226 shutdown_connections 2023-12-07T10:37:00.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.992+0000 7f4f23dff700 1 -- 172.21.15.119:0/1061528226 wait complete. 2023-12-07T10:37:00.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.994+0000 7f4f23dff700 1 Processor -- start 2023-12-07T10:37:00.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.994+0000 7f4f23dff700 1 -- start start 2023-12-07T10:37:00.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.994+0000 7f4f23dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c101560 0x7f4f1c073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:37:00.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.995+0000 7f4f23dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f1c101f20 0x7f4f1c073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:37:00.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.995+0000 7f4f23dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f1c107620 0x7f4f1c074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:37:00.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.995+0000 7f4f23dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c1a5d10 con 0x7f4f1c101f20 2023-12-07T10:37:00.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.995+0000 7f4f23dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c1a5e80 con 0x7f4f1c101560 2023-12-07T10:37:00.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.995+0000 7f4f23dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c1a6040 con 0x7f4f1c107620 2023-12-07T10:37:00.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.995+0000 7f4f2139a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f1c101f20 0x7f4f1c073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:37:00.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.995+0000 7f4f2139a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f1c101f20 0x7f4f1c073e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58300/0 (socket says 172.21.15.119:58300) 2023-12-07T10:37:00.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.995+0000 7f4f2139a700 1 -- 172.21.15.119:0/3012882762 learned_addr learned my addr 172.21.15.119:0/3012882762 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:37:00.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.995+0000 7f4f21b9b700 1 --2- 172.21.15.119:0/3012882762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c101560 0x7f4f1c073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:37:00.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.995+0000 7f4f2239c700 1 --2- 172.21.15.119:0/3012882762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f1c107620 0x7f4f1c074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:37:00.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.995+0000 7f4f2139a700 1 -- 172.21.15.119:0/3012882762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f1c107620 msgr2=0x7f4f1c074390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:37:00.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.995+0000 7f4f2139a700 1 --2- 172.21.15.119:0/3012882762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f1c107620 0x7f4f1c074390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:00.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.995+0000 7f4f2139a700 1 -- 172.21.15.119:0/3012882762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c101560 msgr2=0x7f4f1c073910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:37:00.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.995+0000 7f4f2139a700 1 --2- 172.21.15.119:0/3012882762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c101560 0x7f4f1c073910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:00.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.995+0000 7f4f2139a700 1 -- 172.21.15.119:0/3012882762 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f180095f0 con 0x7f4f1c101f20 2023-12-07T10:37:00.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.995+0000 7f4f21b9b700 1 --2- 172.21.15.119:0/3012882762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c101560 0x7f4f1c073910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:37:00.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.995+0000 7f4f2239c700 1 --2- 172.21.15.119:0/3012882762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f1c107620 0x7f4f1c074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-07T10:37:00.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.995+0000 7f4f2139a700 1 --2- 172.21.15.119:0/3012882762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f1c101f20 0x7f4f1c073e50 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00e9a0 tx=0x7f4f0c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:37:00.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.996+0000 7f4f12ffd700 1 -- 172.21.15.119:0/3012882762 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c009790 con 0x7f4f1c101f20 2023-12-07T10:37:00.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.996+0000 7f4f12ffd700 1 -- 172.21.15.119:0/3012882762 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f0c00cbd0 con 0x7f4f1c101f20 2023-12-07T10:37:00.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.996+0000 7f4f23dff700 1 -- 172.21.15.119:0/3012882762 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f1c1a6320 con 0x7f4f1c101f20 2023-12-07T10:37:00.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.996+0000 7f4f12ffd700 1 -- 172.21.15.119:0/3012882762 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c018680 con 0x7f4f1c101f20 2023-12-07T10:37:00.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.996+0000 7f4f23dff700 1 -- 172.21.15.119:0/3012882762 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1c1a6880 con 0x7f4f1c101f20 2023-12-07T10:37:00.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.997+0000 7f4f23dff700 1 -- 172.21.15.119:0/3012882762 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f1c109ac0 con 0x7f4f1c101f20 2023-12-07T10:37:00.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:00.999+0000 7f4f12ffd700 1 -- 172.21.15.119:0/3012882762 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f4f0c00cd40 con 0x7f4f1c101f20 2023-12-07T10:37:01.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:01.001+0000 7f4f12ffd700 1 --2- 172.21.15.119:0/3012882762 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4f080a1d00 0x7f4f080a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:37:01.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:01.001+0000 7f4f12ffd700 1 -- 172.21.15.119:0/3012882762 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9803+0+0 (secure 0 0 0) 0x7f4f0c014070 con 0x7f4f1c101f20 2023-12-07T10:37:01.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:01.001+0000 7f4f21b9b700 1 --2- 172.21.15.119:0/3012882762 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4f080a1d00 0x7f4f080a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:37:01.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:01.002+0000 7f4f21b9b700 1 --2- 172.21.15.119:0/3012882762 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4f080a1d00 0x7f4f080a41c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4f18009990 tx=0x7f4f18005ab0 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:37:01.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:01.005+0000 7f4f12ffd700 1 -- 172.21.15.119:0/3012882762 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4f0c0522f0 con 0x7f4f1c101f20 2023-12-07T10:37:01.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:01.210+0000 7f4f23dff700 1 -- 172.21.15.119:0/3012882762 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4f1c102c80 con 0x7f4f1c101f20 2023-12-07T10:37:01.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:01 smithi153 ceph-mon[133625]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-12-07T10:37:01.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:01 smithi153 ceph-mon[133625]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-12-07T10:37:01.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:01 smithi153 ceph-mon[133625]: Cluster is now healthy 2023-12-07T10:37:01.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:01 smithi153 ceph-mon[133625]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-12-07T10:37:01.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:01 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.159:6833/415425626,v1:172.21.15.159:6837/415425626] up:boot 2023-12-07T10:37:01.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:01 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.159:6835/3336272883,v1:172.21.15.159:6839/3336272883] up:boot 2023-12-07T10:37:01.529 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:01 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.159:6834/2801133664,v1:172.21.15.159:6838/2801133664] up:boot 2023-12-07T10:37:01.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:01 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.159:6832/2658134851,v1:172.21.15.159:6836/2658134851] up:boot 2023-12-07T10:37:01.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:01 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.153:6837/2576107856,v1:172.21.15.153:6839/2576107856] up:boot 2023-12-07T10:37:01.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:01 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.153:6836/239177097,v1:172.21.15.153:6838/239177097] up:boot 2023-12-07T10:37:01.530 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:01 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.153:6833/716962199,v1:172.21.15.153:6835/716962199] up:boot 2023-12-07T10:37:01.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:01 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.153:6832/1670345847,v1:172.21.15.153:6834/1670345847] up:boot 2023-12-07T10:37:01.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:01 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.119:6838/2655284377,v1:172.21.15.119:6839/2655284377] up:boot 2023-12-07T10:37:01.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:01 smithi153 ceph-mon[133625]: mds.? [v2:172.21.15.119:6840/3497426997,v1:172.21.15.119:6841/3497426997] up:boot 2023-12-07T10:37:01.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:01 smithi153 ceph-mon[133625]: fsmap 12 up:standby 2023-12-07T10:37:01.531 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:01 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-07T10:37:01.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:01 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-07T10:37:01.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:01 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-07T10:37:01.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:01 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-07T10:37:01.532 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:01 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-07T10:37:01.533 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:01 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-07T10:37:01.533 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:01 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-07T10:37:01.533 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:01 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-07T10:37:01.533 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:01 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-07T10:37:01.533 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:01 smithi153 ceph-mon[133625]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-07T10:37:01.534 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:01 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3012882762' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-07T10:37:01.727 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:01 smithi119 ceph-mon[97908]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-12-07T10:37:01.728 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:01 smithi119 ceph-mon[97908]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-12-07T10:37:01.728 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:01 smithi119 ceph-mon[97908]: Cluster is now healthy 2023-12-07T10:37:01.728 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:01 smithi119 ceph-mon[97908]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-12-07T10:37:01.728 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:01 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.159:6833/415425626,v1:172.21.15.159:6837/415425626] up:boot 2023-12-07T10:37:01.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:01 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.159:6835/3336272883,v1:172.21.15.159:6839/3336272883] up:boot 2023-12-07T10:37:01.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:01 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.159:6834/2801133664,v1:172.21.15.159:6838/2801133664] up:boot 2023-12-07T10:37:01.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:01 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.159:6832/2658134851,v1:172.21.15.159:6836/2658134851] up:boot 2023-12-07T10:37:01.729 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:01 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.153:6837/2576107856,v1:172.21.15.153:6839/2576107856] up:boot 2023-12-07T10:37:01.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:01 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.153:6836/239177097,v1:172.21.15.153:6838/239177097] up:boot 2023-12-07T10:37:01.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:01 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.153:6833/716962199,v1:172.21.15.153:6835/716962199] up:boot 2023-12-07T10:37:01.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:01 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.153:6832/1670345847,v1:172.21.15.153:6834/1670345847] up:boot 2023-12-07T10:37:01.730 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:01 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.119:6838/2655284377,v1:172.21.15.119:6839/2655284377] up:boot 2023-12-07T10:37:01.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:01 smithi119 ceph-mon[97908]: mds.? [v2:172.21.15.119:6840/3497426997,v1:172.21.15.119:6841/3497426997] up:boot 2023-12-07T10:37:01.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:01 smithi119 ceph-mon[97908]: fsmap 12 up:standby 2023-12-07T10:37:01.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:01 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-07T10:37:01.731 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:01 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-07T10:37:01.732 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:01 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-07T10:37:01.732 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:01 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-07T10:37:01.732 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:01 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-07T10:37:01.732 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:01 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-07T10:37:01.733 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:01 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-07T10:37:01.733 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:01 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-07T10:37:01.733 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:01 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-07T10:37:01.733 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:01 smithi119 ceph-mon[97908]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-07T10:37:01.734 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:01 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3012882762' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-07T10:37:01.761 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:01 smithi159 ceph-mon[134726]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-12-07T10:37:01.762 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:01 smithi159 ceph-mon[134726]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-12-07T10:37:01.762 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:01 smithi159 ceph-mon[134726]: Cluster is now healthy 2023-12-07T10:37:01.762 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:01 smithi159 ceph-mon[134726]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-12-07T10:37:01.762 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:01 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.159:6833/415425626,v1:172.21.15.159:6837/415425626] up:boot 2023-12-07T10:37:01.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:01 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.159:6835/3336272883,v1:172.21.15.159:6839/3336272883] up:boot 2023-12-07T10:37:01.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:01 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.159:6834/2801133664,v1:172.21.15.159:6838/2801133664] up:boot 2023-12-07T10:37:01.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:01 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.159:6832/2658134851,v1:172.21.15.159:6836/2658134851] up:boot 2023-12-07T10:37:01.763 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:01 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.153:6837/2576107856,v1:172.21.15.153:6839/2576107856] up:boot 2023-12-07T10:37:01.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:01 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.153:6836/239177097,v1:172.21.15.153:6838/239177097] up:boot 2023-12-07T10:37:01.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:01 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.153:6833/716962199,v1:172.21.15.153:6835/716962199] up:boot 2023-12-07T10:37:01.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:01 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.153:6832/1670345847,v1:172.21.15.153:6834/1670345847] up:boot 2023-12-07T10:37:01.764 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:01 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.119:6838/2655284377,v1:172.21.15.119:6839/2655284377] up:boot 2023-12-07T10:37:01.765 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:01 smithi159 ceph-mon[134726]: mds.? [v2:172.21.15.119:6840/3497426997,v1:172.21.15.119:6841/3497426997] up:boot 2023-12-07T10:37:01.765 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:01 smithi159 ceph-mon[134726]: fsmap 12 up:standby 2023-12-07T10:37:01.765 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:01 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-07T10:37:01.765 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:01 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-07T10:37:01.766 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:01 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-07T10:37:01.766 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:01 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-07T10:37:01.766 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:01 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-07T10:37:01.766 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:01 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-07T10:37:01.767 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:01 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-07T10:37:01.767 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:01 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-07T10:37:01.767 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:01 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-07T10:37:01.767 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:01 smithi159 ceph-mon[134726]: from='mgr.15612 172.21.15.119:0/3345301548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-07T10:37:01.767 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:01 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3012882762' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-07T10:37:02.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.078+0000 7f4f12ffd700 1 -- 172.21.15.119:0/3012882762 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v116) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4f0c09a800 con 0x7f4f1c101f20 2023-12-07T10:37:02.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.082+0000 7f4f23dff700 1 -- 172.21.15.119:0/3012882762 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4f080a1d00 msgr2=0x7f4f080a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:37:02.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.083+0000 7f4f23dff700 1 --2- 172.21.15.119:0/3012882762 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4f080a1d00 0x7f4f080a41c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4f18009990 tx=0x7f4f18005ab0 comp rx=0 tx=0).stop 2023-12-07T10:37:02.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.083+0000 7f4f23dff700 1 -- 172.21.15.119:0/3012882762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f1c101f20 msgr2=0x7f4f1c073e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:37:02.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.083+0000 7f4f23dff700 1 --2- 172.21.15.119:0/3012882762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f1c101f20 0x7f4f1c073e50 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00e9a0 tx=0x7f4f0c00ed60 comp rx=0 tx=0).stop 2023-12-07T10:37:02.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.083+0000 7f4f23dff700 1 -- 172.21.15.119:0/3012882762 shutdown_connections 2023-12-07T10:37:02.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.083+0000 7f4f23dff700 1 --2- 172.21.15.119:0/3012882762 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4f080a1d00 0x7f4f080a41c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:02.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.083+0000 7f4f23dff700 1 --2- 172.21.15.119:0/3012882762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f1c107620 0x7f4f1c074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:02.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.083+0000 7f4f23dff700 1 --2- 172.21.15.119:0/3012882762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c101560 0x7f4f1c073910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:02.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.083+0000 7f4f23dff700 1 --2- 172.21.15.119:0/3012882762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f1c101f20 0x7f4f1c073e50 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:02.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.083+0000 7f4f23dff700 1 -- 172.21.15.119:0/3012882762 >> 172.21.15.119:0/3012882762 conn(0x7f4f1c0fcfd0 msgr2=0x7f4f1c0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:37:02.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.083+0000 7f4f23dff700 1 -- 172.21.15.119:0/3012882762 shutdown_connections 2023-12-07T10:37:02.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.084+0000 7f4f23dff700 1 -- 172.21.15.119:0/3012882762 wait complete. 2023-12-07T10:37:02.089 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2023-12-07T10:37:02.119 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-07T10:37:02.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.366+0000 7f6c9cdff700 1 -- 172.21.15.119:0/584932399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c98101f20 msgr2=0x7f6c9810e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:37:02.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.366+0000 7f6c9cdff700 1 --2- 172.21.15.119:0/584932399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c98101f20 0x7f6c9810e3e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000b0d0 tx=0x7f6c9000b3e0 comp rx=0 tx=0).stop 2023-12-07T10:37:02.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.366+0000 7f6c9cdff700 1 -- 172.21.15.119:0/584932399 shutdown_connections 2023-12-07T10:37:02.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.366+0000 7f6c9cdff700 1 --2- 172.21.15.119:0/584932399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c98101f20 0x7f6c9810e3e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:02.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.366+0000 7f6c9cdff700 1 --2- 172.21.15.119:0/584932399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c98107620 0x7f6c98107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:02.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.366+0000 7f6c9cdff700 1 --2- 172.21.15.119:0/584932399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c98101560 0x7f6c981019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:02.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.366+0000 7f6c9cdff700 1 -- 172.21.15.119:0/584932399 >> 172.21.15.119:0/584932399 conn(0x7f6c980fcfd0 msgr2=0x7f6c980ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:37:02.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.366+0000 7f6c9cdff700 1 -- 172.21.15.119:0/584932399 shutdown_connections 2023-12-07T10:37:02.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.366+0000 7f6c9cdff700 1 -- 172.21.15.119:0/584932399 wait complete. 2023-12-07T10:37:02.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.367+0000 7f6c9cdff700 1 Processor -- start 2023-12-07T10:37:02.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.368+0000 7f6c9cdff700 1 -- start start 2023-12-07T10:37:02.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.368+0000 7f6c9cdff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c98101560 0x7f6c9819d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:37:02.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.368+0000 7f6c9cdff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c98101f20 0x7f6c9819dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:37:02.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.368+0000 7f6c9cdff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c98107620 0x7f6c981978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:37:02.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.368+0000 7f6c9cdff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c98197f30 con 0x7f6c98107620 2023-12-07T10:37:02.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.368+0000 7f6c9cdff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c981980a0 con 0x7f6c98101560 2023-12-07T10:37:02.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.368+0000 7f6c9cdff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c98198260 con 0x7f6c98101f20 2023-12-07T10:37:02.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.368+0000 7f6c96d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c98107620 0x7f6c981978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:37:02.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.368+0000 7f6c96d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c98107620 0x7f6c981978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58324/0 (socket says 172.21.15.119:58324) 2023-12-07T10:37:02.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.368+0000 7f6c96d9d700 1 -- 172.21.15.119:0/3311189501 learned_addr learned my addr 172.21.15.119:0/3311189501 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:37:02.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.368+0000 7f6c95d9b700 1 --2- 172.21.15.119:0/3311189501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c98101f20 0x7f6c9819dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:37:02.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.368+0000 7f6c9659c700 1 --2- 172.21.15.119:0/3311189501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c98101560 0x7f6c9819d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:37:02.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.368+0000 7f6c96d9d700 1 -- 172.21.15.119:0/3311189501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c98101f20 msgr2=0x7f6c9819dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:37:02.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.368+0000 7f6c96d9d700 1 --2- 172.21.15.119:0/3311189501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c98101f20 0x7f6c9819dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:02.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.368+0000 7f6c96d9d700 1 -- 172.21.15.119:0/3311189501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c98101560 msgr2=0x7f6c9819d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:37:02.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.368+0000 7f6c96d9d700 1 --2- 172.21.15.119:0/3311189501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c98101560 0x7f6c9819d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:02.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.368+0000 7f6c96d9d700 1 -- 172.21.15.119:0/3311189501 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c88009520 con 0x7f6c98107620 2023-12-07T10:37:02.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.369+0000 7f6c96d9d700 1 --2- 172.21.15.119:0/3311189501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c98107620 0x7f6c981978d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000bcd0 tx=0x7f6c9000bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:37:02.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.369+0000 7f6c877fe700 1 -- 172.21.15.119:0/3311189501 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9000e040 con 0x7f6c98107620 2023-12-07T10:37:02.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.369+0000 7f6c877fe700 1 -- 172.21.15.119:0/3311189501 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c90005070 con 0x7f6c98107620 2023-12-07T10:37:02.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.369+0000 7f6c877fe700 1 -- 172.21.15.119:0/3311189501 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9001b6a0 con 0x7f6c98107620 2023-12-07T10:37:02.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.369+0000 7f6c9cdff700 1 -- 172.21.15.119:0/3311189501 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c90009d00 con 0x7f6c98107620 2023-12-07T10:37:02.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.369+0000 7f6c9cdff700 1 -- 172.21.15.119:0/3311189501 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c98198720 con 0x7f6c98107620 2023-12-07T10:37:02.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.370+0000 7f6c9cdff700 1 -- 172.21.15.119:0/3311189501 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c98109ac0 con 0x7f6c98107620 2023-12-07T10:37:02.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.372+0000 7f6c877fe700 1 -- 172.21.15.119:0/3311189501 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f6c900051e0 con 0x7f6c98107620 2023-12-07T10:37:02.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.373+0000 7f6c877fe700 1 --2- 172.21.15.119:0/3311189501 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f6c800a1c80 0x7f6c800a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:37:02.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.373+0000 7f6c877fe700 1 -- 172.21.15.119:0/3311189501 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9249+0+0 (secure 0 0 0) 0x7f6c900ce760 con 0x7f6c98107620 2023-12-07T10:37:02.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.374+0000 7f6c9659c700 1 --2- 172.21.15.119:0/3311189501 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f6c800a1c80 0x7f6c800a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:37:02.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.374+0000 7f6c9659c700 1 --2- 172.21.15.119:0/3311189501 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f6c800a1c80 0x7f6c800a4140 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6c88009df0 tx=0x7f6c88009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:37:02.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.378+0000 7f6c877fe700 1 -- 172.21.15.119:0/3311189501 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f6c90097590 con 0x7f6c98107620 2023-12-07T10:37:02.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:02 smithi119 ceph-mon[97908]: pgmap v522: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 92 KiB/s wr, 15 op/s 2023-12-07T10:37:02.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:02 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3012882762' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-12-07T10:37:02.477 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:02 smithi119 ceph-mon[97908]: osdmap e116: 12 total, 12 up, 12 in 2023-12-07T10:37:02.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:02 smithi153 ceph-mon[133625]: pgmap v522: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 92 KiB/s wr, 15 op/s 2023-12-07T10:37:02.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:02 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3012882762' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-12-07T10:37:02.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:02 smithi153 ceph-mon[133625]: osdmap e116: 12 total, 12 up, 12 in 2023-12-07T10:37:02.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:02.582+0000 7f6c9cdff700 1 -- 172.21.15.119:0/3311189501 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6c98102c80 con 0x7f6c98107620 2023-12-07T10:37:02.761 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:02 smithi159 ceph-mon[134726]: pgmap v522: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 92 KiB/s wr, 15 op/s 2023-12-07T10:37:02.761 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:02 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3012882762' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-12-07T10:37:02.761 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:02 smithi159 ceph-mon[134726]: osdmap e116: 12 total, 12 up, 12 in 2023-12-07T10:37:03.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.270+0000 7f6c877fe700 1 -- 172.21.15.119:0/3311189501 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v117) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6c90095e60 con 0x7f6c98107620 2023-12-07T10:37:03.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.274+0000 7f6c9cdff700 1 -- 172.21.15.119:0/3311189501 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f6c800a1c80 msgr2=0x7f6c800a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:37:03.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.274+0000 7f6c9cdff700 1 --2- 172.21.15.119:0/3311189501 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f6c800a1c80 0x7f6c800a4140 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6c88009df0 tx=0x7f6c88009260 comp rx=0 tx=0).stop 2023-12-07T10:37:03.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.274+0000 7f6c9cdff700 1 -- 172.21.15.119:0/3311189501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c98107620 msgr2=0x7f6c981978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:37:03.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.274+0000 7f6c9cdff700 1 --2- 172.21.15.119:0/3311189501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c98107620 0x7f6c981978d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000bcd0 tx=0x7f6c9000bd00 comp rx=0 tx=0).stop 2023-12-07T10:37:03.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.275+0000 7f6c9cdff700 1 -- 172.21.15.119:0/3311189501 shutdown_connections 2023-12-07T10:37:03.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.275+0000 7f6c9cdff700 1 --2- 172.21.15.119:0/3311189501 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f6c800a1c80 0x7f6c800a4140 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:03.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.275+0000 7f6c9cdff700 1 --2- 172.21.15.119:0/3311189501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c98107620 0x7f6c981978d0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:03.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.275+0000 7f6c9cdff700 1 --2- 172.21.15.119:0/3311189501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c98101560 0x7f6c9819d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:03.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.275+0000 7f6c9cdff700 1 --2- 172.21.15.119:0/3311189501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c98101f20 0x7f6c9819dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:03.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.275+0000 7f6c9cdff700 1 -- 172.21.15.119:0/3311189501 >> 172.21.15.119:0/3311189501 conn(0x7f6c980fcfd0 msgr2=0x7f6c980fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:37:03.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.276+0000 7f6c9cdff700 1 -- 172.21.15.119:0/3311189501 shutdown_connections 2023-12-07T10:37:03.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.276+0000 7f6c9cdff700 1 -- 172.21.15.119:0/3311189501 wait complete. 2023-12-07T10:37:03.277 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2023-12-07T10:37:03.300 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-12-07T10:37:03.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:03 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3311189501' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-07T10:37:03.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.542+0000 7f4ae2dff700 1 -- 172.21.15.119:0/971859586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4adc100460 msgr2=0x7f4adc1008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:37:03.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.542+0000 7f4ae2dff700 1 --2- 172.21.15.119:0/971859586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4adc100460 0x7f4adc1008e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8009990 tx=0x7f4ac8009ca0 comp rx=0 tx=0).stop 2023-12-07T10:37:03.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.542+0000 7f4ae2dff700 1 -- 172.21.15.119:0/971859586 shutdown_connections 2023-12-07T10:37:03.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.542+0000 7f4ae2dff700 1 --2- 172.21.15.119:0/971859586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4adc100e20 0x7f4adc10d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:03.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.542+0000 7f4ae2dff700 1 --2- 172.21.15.119:0/971859586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4adc1053d0 0x7f4adc1057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:03.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.542+0000 7f4ae2dff700 1 --2- 172.21.15.119:0/971859586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4adc100460 0x7f4adc1008e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:03.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.542+0000 7f4ae2dff700 1 -- 172.21.15.119:0/971859586 >> 172.21.15.119:0/971859586 conn(0x7f4adc077610 msgr2=0x7f4adc077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:37:03.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.542+0000 7f4ae2dff700 1 -- 172.21.15.119:0/971859586 shutdown_connections 2023-12-07T10:37:03.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.542+0000 7f4ae2dff700 1 -- 172.21.15.119:0/971859586 wait complete. 2023-12-07T10:37:03.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.543+0000 7f4ae2dff700 1 Processor -- start 2023-12-07T10:37:03.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.543+0000 7f4ae2dff700 1 -- start start 2023-12-07T10:37:03.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.544+0000 7f4ae2dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4adc100460 0x7f4adc10dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:37:03.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.544+0000 7f4ae2dff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4adc100e20 0x7f4adc10e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:37:03.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.544+0000 7f4ae2dff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4adc1053d0 0x7f4adc10e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:37:03.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.544+0000 7f4ae2dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc1a5dc0 con 0x7f4adc100460 2023-12-07T10:37:03.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.544+0000 7f4ae2dff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc1a5f30 con 0x7f4adc100e20 2023-12-07T10:37:03.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.544+0000 7f4ae2dff700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc1a60f0 con 0x7f4adc1053d0 2023-12-07T10:37:03.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.544+0000 7f4ae0b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4adc100460 0x7f4adc10dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:37:03.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.544+0000 7f4ae0b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4adc100460 0x7f4adc10dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39594/0 (socket says 172.21.15.119:39594) 2023-12-07T10:37:03.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.544+0000 7f4ae0b9b700 1 -- 172.21.15.119:0/2306890653 learned_addr learned my addr 172.21.15.119:0/2306890653 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-12-07T10:37:03.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.544+0000 7f4adbfff700 1 --2- 172.21.15.119:0/2306890653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4adc100e20 0x7f4adc10e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:37:03.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.544+0000 7f4ae139c700 1 --2- 172.21.15.119:0/2306890653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4adc1053d0 0x7f4adc10e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:37:03.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.544+0000 7f4adbfff700 1 -- 172.21.15.119:0/2306890653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4adc1053d0 msgr2=0x7f4adc10e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:37:03.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.544+0000 7f4adbfff700 1 --2- 172.21.15.119:0/2306890653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4adc1053d0 0x7f4adc10e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:03.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.544+0000 7f4adbfff700 1 -- 172.21.15.119:0/2306890653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4adc100460 msgr2=0x7f4adc10dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:37:03.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.544+0000 7f4adbfff700 1 --2- 172.21.15.119:0/2306890653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4adc100460 0x7f4adc10dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:03.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.544+0000 7f4adbfff700 1 -- 172.21.15.119:0/2306890653 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac80095f0 con 0x7f4adc100e20 2023-12-07T10:37:03.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.544+0000 7f4ae0b9b700 1 --2- 172.21.15.119:0/2306890653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4adc100460 0x7f4adc10dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-07T10:37:03.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.545+0000 7f4adbfff700 1 --2- 172.21.15.119:0/2306890653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4adc100e20 0x7f4adc10e1c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8000c00 tx=0x7f4ac8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:37:03.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.545+0000 7f4ad9ffb700 1 -- 172.21.15.119:0/2306890653 <== mon.2 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac801d070 con 0x7f4adc100e20 2023-12-07T10:37:03.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.545+0000 7f4ae2dff700 1 -- 172.21.15.119:0/2306890653 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4adc1a6370 con 0x7f4adc100e20 2023-12-07T10:37:03.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.545+0000 7f4ad9ffb700 1 -- 172.21.15.119:0/2306890653 <== mon.2 v2:172.21.15.153:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ac8021410 con 0x7f4adc100e20 2023-12-07T10:37:03.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.545+0000 7f4ad9ffb700 1 -- 172.21.15.119:0/2306890653 <== mon.2 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8017650 con 0x7f4adc100e20 2023-12-07T10:37:03.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.545+0000 7f4ae2dff700 1 -- 172.21.15.119:0/2306890653 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4adc1a6870 con 0x7f4adc100e20 2023-12-07T10:37:03.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.546+0000 7f4ae2dff700 1 -- 172.21.15.119:0/2306890653 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4adc1089a0 con 0x7f4adc100e20 2023-12-07T10:37:03.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.548+0000 7f4ad9ffb700 1 -- 172.21.15.119:0/2306890653 <== mon.2 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 26) v1 ==== 136615+0+0 (secure 0 0 0) 0x7f4ac8021580 con 0x7f4adc100e20 2023-12-07T10:37:03.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.549+0000 7f4ad9ffb700 1 --2- 172.21.15.119:0/2306890653 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4acc0a1c80 0x7f4acc0a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-07T10:37:03.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.549+0000 7f4ad9ffb700 1 -- 172.21.15.119:0/2306890653 <== mon.2 v2:172.21.15.153:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8682+0+0 (secure 0 0 0) 0x7f4ac80cec70 con 0x7f4adc100e20 2023-12-07T10:37:03.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.549+0000 7f4ae0b9b700 1 --2- 172.21.15.119:0/2306890653 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4acc0a1c80 0x7f4acc0a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-07T10:37:03.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.550+0000 7f4ae0b9b700 1 --2- 172.21.15.119:0/2306890653 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4acc0a1c80 0x7f4acc0a4140 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0011550 tx=0x7f4ad0009260 comp rx=0 tx=0).ready entity=mgr.15612 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-07T10:37:03.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.553+0000 7f4ad9ffb700 1 -- 172.21.15.119:0/2306890653 <== mon.2 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4ac8097ce0 con 0x7f4adc100e20 2023-12-07T10:37:03.727 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:03 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3311189501' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-07T10:37:03.761 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:03 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3311189501' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-07T10:37:03.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:03.762+0000 7f4ae2dff700 1 -- 172.21.15.119:0/2306890653 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4adc10f160 con 0x7f4adc100e20 2023-12-07T10:37:04.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:04.278+0000 7f4ad9ffb700 1 -- 172.21.15.119:0/2306890653 <== mon.2 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v118) v1 ==== 178+0+0 (secure 0 0 0) 0x7f4ac809b300 con 0x7f4adc100e20 2023-12-07T10:37:04.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:04.283+0000 7f4ae2dff700 1 -- 172.21.15.119:0/2306890653 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4acc0a1c80 msgr2=0x7f4acc0a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:37:04.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:04.283+0000 7f4ae2dff700 1 --2- 172.21.15.119:0/2306890653 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4acc0a1c80 0x7f4acc0a4140 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0011550 tx=0x7f4ad0009260 comp rx=0 tx=0).stop 2023-12-07T10:37:04.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:04.283+0000 7f4ae2dff700 1 -- 172.21.15.119:0/2306890653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4adc100e20 msgr2=0x7f4adc10e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-07T10:37:04.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:04.283+0000 7f4ae2dff700 1 --2- 172.21.15.119:0/2306890653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4adc100e20 0x7f4adc10e1c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8000c00 tx=0x7f4ac8004730 comp rx=0 tx=0).stop 2023-12-07T10:37:04.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:04.284+0000 7f4ae2dff700 1 -- 172.21.15.119:0/2306890653 shutdown_connections 2023-12-07T10:37:04.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:04.284+0000 7f4ae2dff700 1 --2- 172.21.15.119:0/2306890653 >> [v2:172.21.15.119:6800/3763427845,v1:172.21.15.119:6801/3763427845] conn(0x7f4acc0a1c80 0x7f4acc0a4140 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:04.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:04.284+0000 7f4ae2dff700 1 --2- 172.21.15.119:0/2306890653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4adc1053d0 0x7f4adc10e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:04.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:04.284+0000 7f4ae2dff700 1 --2- 172.21.15.119:0/2306890653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4adc100460 0x7f4adc10dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:04.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:04.284+0000 7f4ae2dff700 1 --2- 172.21.15.119:0/2306890653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4adc100e20 0x7f4adc10e1c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-07T10:37:04.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:04.284+0000 7f4ae2dff700 1 -- 172.21.15.119:0/2306890653 >> 172.21.15.119:0/2306890653 conn(0x7f4adc077610 msgr2=0x7f4adc0fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-07T10:37:04.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:04.285+0000 7f4ae2dff700 1 -- 172.21.15.119:0/2306890653 shutdown_connections 2023-12-07T10:37:04.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-12-07T10:37:04.285+0000 7f4ae2dff700 1 -- 172.21.15.119:0/2306890653 wait complete. 2023-12-07T10:37:04.286 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data_ec' removed 2023-12-07T10:37:04.311 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-07T10:37:04.390 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-07T10:37:04.421 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-07T10:37:04.457 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-12-07T10:37:04.458 DEBUG:teuthology.orchestra.run.smithi119:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-07T10:37:04.503 DEBUG:teuthology.orchestra.run.smithi153:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-07T10:37:04.520 DEBUG:teuthology.orchestra.run.smithi159:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-07T10:37:04.527 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:04 smithi153 ceph-mon[133625]: pgmap v524: 97 pgs: 97 active+clean; 310 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:37:04.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:04 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/3311189501' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-12-07T10:37:04.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:04 smithi153 ceph-mon[133625]: osdmap e117: 12 total, 12 up, 12 in 2023-12-07T10:37:04.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:04 smithi153 ceph-mon[133625]: from='client.? 172.21.15.119:0/2306890653' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-07T10:37:04.528 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:04 smithi153 ceph-mon[133625]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-07T10:37:04.537 INFO:tasks.cephadm:Stopping all daemons... 2023-12-07T10:37:04.538 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-12-07T10:37:04.538 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mon.a 2023-12-07T10:37:04.604 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:04 smithi119 ceph-mon[97908]: pgmap v524: 97 pgs: 97 active+clean; 310 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:37:04.605 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:04 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/3311189501' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-12-07T10:37:04.605 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:04 smithi119 ceph-mon[97908]: osdmap e117: 12 total, 12 up, 12 in 2023-12-07T10:37:04.605 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:04 smithi119 ceph-mon[97908]: from='client.? 172.21.15.119:0/2306890653' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-07T10:37:04.605 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:04 smithi119 ceph-mon[97908]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-07T10:37:04.761 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:04 smithi159 ceph-mon[134726]: pgmap v524: 97 pgs: 97 active+clean; 310 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail 2023-12-07T10:37:04.761 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:04 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/3311189501' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-12-07T10:37:04.761 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:04 smithi159 ceph-mon[134726]: osdmap e117: 12 total, 12 up, 12 in 2023-12-07T10:37:04.761 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:04 smithi159 ceph-mon[134726]: from='client.? 172.21.15.119:0/2306890653' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-07T10:37:04.761 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:04 smithi159 ceph-mon[134726]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-07T10:37:04.977 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:04 smithi119 systemd[1]: Stopping Ceph mon.a for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:37:05.326 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:05 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mon-a[97884]: 2023-12-07T10:37:05.061+0000 7fc36ab27700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:37:05.327 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:05 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mon-a[97884]: 2023-12-07T10:37:05.061+0000 7fc36ab27700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-12-07T10:37:05.727 INFO:journalctl@ceph.mon.a.smithi119.stdout:Dec 07 10:37:05 smithi119 bash[180670]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mon-a 2023-12-07T10:37:05.909 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mon.a.service' 2023-12-07T10:37:06.029 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:37:06.029 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-12-07T10:37:06.030 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-12-07T10:37:06.030 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mon.b 2023-12-07T10:37:06.415 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:06 smithi153 systemd[1]: Stopping Ceph mon.b for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:37:06.707 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:06 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mon-b[133601]: 2023-12-07T10:37:06.414+0000 7f603c038700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:37:06.708 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:06 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mon-b[133601]: 2023-12-07T10:37:06.414+0000 7f603c038700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-12-07T10:37:07.223 INFO:journalctl@ceph.mon.b.smithi153.stdout:Dec 07 10:37:06 smithi153 bash[178946]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mon-b 2023-12-07T10:37:07.230 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mon.b.service' 2023-12-07T10:37:07.279 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:37:07.280 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-12-07T10:37:07.280 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-12-07T10:37:07.280 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mon.c 2023-12-07T10:37:07.648 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:07 smithi159 systemd[1]: Stopping Ceph mon.c for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:37:07.969 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:07 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mon-c[134702]: 2023-12-07T10:37:07.648+0000 7fba991c3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:37:07.969 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:07 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mon-c[134702]: 2023-12-07T10:37:07.648+0000 7fba991c3700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-12-07T10:37:08.261 INFO:journalctl@ceph.mon.c.smithi159.stdout:Dec 07 10:37:08 smithi159 bash[180943]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mon-c 2023-12-07T10:37:08.315 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mon.c.service' 2023-12-07T10:37:08.387 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:37:08.387 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-12-07T10:37:08.388 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-12-07T10:37:08.388 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mgr.x 2023-12-07T10:37:08.727 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:37:08 smithi119 systemd[1]: Stopping Ceph mgr.x for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:37:09.176 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:37:08 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:37:08.924+0000 7fb690557700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:37:09.176 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:37:08 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x[98159]: 2023-12-07T10:37:08.924+0000 7fb690557700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-12-07T10:37:09.477 INFO:journalctl@ceph.mgr.x.smithi119.stdout:Dec 07 10:37:09 smithi119 bash[180814]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-x 2023-12-07T10:37:09.728 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mgr.x.service' 2023-12-07T10:37:09.812 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:37:09.813 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-12-07T10:37:09.813 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-12-07T10:37:09.813 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mgr.y 2023-12-07T10:37:10.278 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:37:09 smithi153 systemd[1]: Stopping Ceph mgr.y for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:37:10.559 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:37:10 smithi153 bash[179086]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-y 2023-12-07T10:37:10.739 INFO:journalctl@ceph.mgr.y.smithi153.stdout:Dec 07 10:37:10 smithi153 systemd[1]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-12-07T10:37:10.743 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mgr.y.service' 2023-12-07T10:37:10.780 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:37:10.780 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-12-07T10:37:10.781 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-12-07T10:37:10.781 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mgr.z 2023-12-07T10:37:11.209 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:37:10 smithi159 systemd[1]: Stopping Ceph mgr.z for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:37:11.500 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:37:11 smithi159 bash[181100]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mgr-z 2023-12-07T10:37:11.500 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Dec 07 10:37:11 smithi159 systemd[1]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-12-07T10:37:11.685 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mgr.z.service' 2023-12-07T10:37:11.721 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:37:11.721 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-12-07T10:37:11.721 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-12-07T10:37:11.721 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.0 2023-12-07T10:37:12.227 INFO:journalctl@ceph.osd.0.smithi119.stdout:Dec 07 10:37:11 smithi119 systemd[1]: Stopping Ceph osd.0 for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:37:12.727 INFO:journalctl@ceph.osd.0.smithi119.stdout:Dec 07 10:37:12 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-0[113022]: 2023-12-07T10:37:12.268+0000 7fbbddb04700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:37:12.727 INFO:journalctl@ceph.osd.0.smithi119.stdout:Dec 07 10:37:12 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-0[113022]: 2023-12-07T10:37:12.268+0000 7fbbddb04700 -1 osd.0 118 *** Got signal Terminated *** 2023-12-07T10:37:12.727 INFO:journalctl@ceph.osd.0.smithi119.stdout:Dec 07 10:37:12 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-0[113022]: 2023-12-07T10:37:12.268+0000 7fbbddb04700 -1 osd.0 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-07T10:37:18.477 INFO:journalctl@ceph.osd.0.smithi119.stdout:Dec 07 10:37:18 smithi119 bash[180960]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-0 2023-12-07T10:37:20.362 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.0.service' 2023-12-07T10:37:20.451 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:37:20.452 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-12-07T10:37:20.452 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-12-07T10:37:20.452 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.1 2023-12-07T10:37:20.778 INFO:journalctl@ceph.osd.1.smithi153.stdout:Dec 07 10:37:20 smithi153 systemd[1]: Stopping Ceph osd.1 for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:37:21.278 INFO:journalctl@ceph.osd.1.smithi153.stdout:Dec 07 10:37:20 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-1[138700]: 2023-12-07T10:37:20.841+0000 7f9fc0c82700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:37:21.278 INFO:journalctl@ceph.osd.1.smithi153.stdout:Dec 07 10:37:20 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-1[138700]: 2023-12-07T10:37:20.841+0000 7f9fc0c82700 -1 osd.1 118 *** Got signal Terminated *** 2023-12-07T10:37:21.278 INFO:journalctl@ceph.osd.1.smithi153.stdout:Dec 07 10:37:20 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-1[138700]: 2023-12-07T10:37:20.841+0000 7f9fc0c82700 -1 osd.1 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-07T10:37:26.691 INFO:journalctl@ceph.osd.1.smithi153.stdout:Dec 07 10:37:26 smithi153 bash[179234]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-1 2023-12-07T10:37:28.057 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.1.service' 2023-12-07T10:37:28.097 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:37:28.097 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-12-07T10:37:28.097 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-12-07T10:37:28.097 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.2 2023-12-07T10:37:28.465 INFO:journalctl@ceph.osd.2.smithi159.stdout:Dec 07 10:37:28 smithi159 systemd[1]: Stopping Ceph osd.2 for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:37:28.761 INFO:journalctl@ceph.osd.2.smithi159.stdout:Dec 07 10:37:28 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-2[140541]: 2023-12-07T10:37:28.465+0000 7f3108c7b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:37:28.761 INFO:journalctl@ceph.osd.2.smithi159.stdout:Dec 07 10:37:28 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-2[140541]: 2023-12-07T10:37:28.465+0000 7f3108c7b700 -1 osd.2 118 *** Got signal Terminated *** 2023-12-07T10:37:28.761 INFO:journalctl@ceph.osd.2.smithi159.stdout:Dec 07 10:37:28 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-2[140541]: 2023-12-07T10:37:28.465+0000 7f3108c7b700 -1 osd.2 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-07T10:37:34.511 INFO:journalctl@ceph.osd.2.smithi159.stdout:Dec 07 10:37:34 smithi159 bash[181248]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-2 2023-12-07T10:37:35.496 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.2.service' 2023-12-07T10:37:35.536 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:37:35.536 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-12-07T10:37:35.537 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-12-07T10:37:35.537 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.3 2023-12-07T10:37:35.977 INFO:journalctl@ceph.osd.3.smithi119.stdout:Dec 07 10:37:35 smithi119 systemd[1]: Stopping Ceph osd.3 for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:37:36.477 INFO:journalctl@ceph.osd.3.smithi119.stdout:Dec 07 10:37:36 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-3[121584]: 2023-12-07T10:37:36.074+0000 7f06db13a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:37:36.477 INFO:journalctl@ceph.osd.3.smithi119.stdout:Dec 07 10:37:36 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-3[121584]: 2023-12-07T10:37:36.074+0000 7f06db13a700 -1 osd.3 118 *** Got signal Terminated *** 2023-12-07T10:37:36.477 INFO:journalctl@ceph.osd.3.smithi119.stdout:Dec 07 10:37:36 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-3[121584]: 2023-12-07T10:37:36.074+0000 7f06db13a700 -1 osd.3 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-07T10:37:38.761 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:38 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:38.444+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:13.956208+0000 front 2023-12-07T10:37:13.956221+0000 (oldest deadline 2023-12-07T10:37:38.056093+0000) 2023-12-07T10:37:38.977 INFO:journalctl@ceph.osd.3.smithi119.stdout:Dec 07 10:37:38 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-3[121584]: 2023-12-07T10:37:38.574+0000 7f06d3f2e700 -1 osd.3 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:13.165629+0000 front 2023-12-07T10:37:13.165491+0000 (oldest deadline 2023-12-07T10:37:38.465621+0000) 2023-12-07T10:37:39.727 INFO:journalctl@ceph.osd.3.smithi119.stdout:Dec 07 10:37:39 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-3[121584]: 2023-12-07T10:37:39.569+0000 7f06d3f2e700 -1 osd.3 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:13.165629+0000 front 2023-12-07T10:37:13.165491+0000 (oldest deadline 2023-12-07T10:37:38.465621+0000) 2023-12-07T10:37:39.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:39 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:39.416+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:14.001791+0000 front 2023-12-07T10:37:14.001815+0000 (oldest deadline 2023-12-07T10:37:39.301932+0000) 2023-12-07T10:37:39.761 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:39 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:39.489+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:13.956208+0000 front 2023-12-07T10:37:13.956221+0000 (oldest deadline 2023-12-07T10:37:38.056093+0000) 2023-12-07T10:37:40.727 INFO:journalctl@ceph.osd.3.smithi119.stdout:Dec 07 10:37:40 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-3[121584]: 2023-12-07T10:37:40.534+0000 7f06d3f2e700 -1 osd.3 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:13.165629+0000 front 2023-12-07T10:37:13.165491+0000 (oldest deadline 2023-12-07T10:37:38.465621+0000) 2023-12-07T10:37:40.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:40 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:40.384+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:14.001791+0000 front 2023-12-07T10:37:14.001815+0000 (oldest deadline 2023-12-07T10:37:39.301932+0000) 2023-12-07T10:37:40.761 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:40 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:40.450+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:13.956208+0000 front 2023-12-07T10:37:13.956221+0000 (oldest deadline 2023-12-07T10:37:38.056093+0000) 2023-12-07T10:37:41.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:41 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:41.348+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:14.001791+0000 front 2023-12-07T10:37:14.001815+0000 (oldest deadline 2023-12-07T10:37:39.301932+0000) 2023-12-07T10:37:41.761 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:41 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:41.405+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:13.956208+0000 front 2023-12-07T10:37:13.956221+0000 (oldest deadline 2023-12-07T10:37:38.056093+0000) 2023-12-07T10:37:41.777 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:41 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:41.334+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:37:42.227 INFO:journalctl@ceph.osd.3.smithi119.stdout:Dec 07 10:37:42 smithi119 bash[181240]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-3 2023-12-07T10:37:42.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:42 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:42.342+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:14.001791+0000 front 2023-12-07T10:37:14.001815+0000 (oldest deadline 2023-12-07T10:37:39.301932+0000) 2023-12-07T10:37:42.761 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:42 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:42.386+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:13.956208+0000 front 2023-12-07T10:37:13.956221+0000 (oldest deadline 2023-12-07T10:37:38.056093+0000) 2023-12-07T10:37:42.777 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:42 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:42.375+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:37:43.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:42 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:42.934+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:37:43.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:43 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:43.360+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:14.001791+0000 front 2023-12-07T10:37:14.001815+0000 (oldest deadline 2023-12-07T10:37:39.301932+0000) 2023-12-07T10:37:43.761 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:43 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:43.337+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:13.956208+0000 front 2023-12-07T10:37:13.956221+0000 (oldest deadline 2023-12-07T10:37:38.056093+0000) 2023-12-07T10:37:43.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:43 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:43.422+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:37:43.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:43 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:43.416+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:37:44.061 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:43 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:43.960+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:37:44.085 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.3.service' 2023-12-07T10:37:44.174 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:37:44.174 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-12-07T10:37:44.175 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-12-07T10:37:44.175 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.4 2023-12-07T10:37:44.528 INFO:journalctl@ceph.osd.4.smithi153.stdout:Dec 07 10:37:44 smithi153 systemd[1]: Stopping Ceph osd.4 for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:37:44.528 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:44 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:44.370+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:37:44.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:44 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:44.335+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:14.001791+0000 front 2023-12-07T10:37:14.001815+0000 (oldest deadline 2023-12-07T10:37:39.301932+0000) 2023-12-07T10:37:44.761 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:44 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:44.299+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:13.956208+0000 front 2023-12-07T10:37:13.956221+0000 (oldest deadline 2023-12-07T10:37:38.056093+0000) 2023-12-07T10:37:44.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:44 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:44.400+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:37:45.028 INFO:journalctl@ceph.osd.4.smithi153.stdout:Dec 07 10:37:44 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-4[147230]: 2023-12-07T10:37:44.537+0000 7f16bb1e2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:37:45.028 INFO:journalctl@ceph.osd.4.smithi153.stdout:Dec 07 10:37:44 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-4[147230]: 2023-12-07T10:37:44.537+0000 7f16bb1e2700 -1 osd.4 118 *** Got signal Terminated *** 2023-12-07T10:37:45.028 INFO:journalctl@ceph.osd.4.smithi153.stdout:Dec 07 10:37:44 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-4[147230]: 2023-12-07T10:37:44.537+0000 7f16bb1e2700 -1 osd.4 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-07T10:37:45.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:44 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:44.974+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:37:45.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:45 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:45.342+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:14.001791+0000 front 2023-12-07T10:37:14.001815+0000 (oldest deadline 2023-12-07T10:37:39.301932+0000) 2023-12-07T10:37:45.761 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:45 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:45.314+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:13.956208+0000 front 2023-12-07T10:37:13.956221+0000 (oldest deadline 2023-12-07T10:37:38.056093+0000) 2023-12-07T10:37:45.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:45 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:45.429+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:37:45.777 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:45 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:45.371+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:37:46.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:45 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:45.951+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:37:46.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:46 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:46.347+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:14.001791+0000 front 2023-12-07T10:37:14.001815+0000 (oldest deadline 2023-12-07T10:37:39.301932+0000) 2023-12-07T10:37:46.761 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:46 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:46.323+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:13.956208+0000 front 2023-12-07T10:37:13.956221+0000 (oldest deadline 2023-12-07T10:37:38.056093+0000) 2023-12-07T10:37:46.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:46 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:46.455+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:37:46.777 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:46 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:46.324+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:37:47.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:46 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:46.925+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:37:47.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:47 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:47.386+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:14.001791+0000 front 2023-12-07T10:37:14.001815+0000 (oldest deadline 2023-12-07T10:37:39.301932+0000) 2023-12-07T10:37:47.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:47 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:47.465+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:37:47.761 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:47 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:47.319+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:13.956208+0000 front 2023-12-07T10:37:13.956221+0000 (oldest deadline 2023-12-07T10:37:38.056093+0000) 2023-12-07T10:37:47.761 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:47 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:47.319+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.557260+0000 front 2023-12-07T10:37:21.557225+0000 (oldest deadline 2023-12-07T10:37:46.857275+0000) 2023-12-07T10:37:47.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:47 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:47.282+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:37:47.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:47 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:47.282+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:37:48.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:47 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:47.912+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:37:48.471 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:37:48 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:37:48.172+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:37:48.471 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:48 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:48.321+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:13.956208+0000 front 2023-12-07T10:37:13.956221+0000 (oldest deadline 2023-12-07T10:37:38.056093+0000) 2023-12-07T10:37:48.472 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:48 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:48.321+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.557260+0000 front 2023-12-07T10:37:21.557225+0000 (oldest deadline 2023-12-07T10:37:46.857275+0000) 2023-12-07T10:37:48.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:48 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:48.383+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:14.001791+0000 front 2023-12-07T10:37:14.001815+0000 (oldest deadline 2023-12-07T10:37:39.301932+0000) 2023-12-07T10:37:48.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:48 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:48.472+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:37:48.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:37:48 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:37:48.391+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.206897+0000 front 2023-12-07T10:37:22.206862+0000 (oldest deadline 2023-12-07T10:37:48.107185+0000) 2023-12-07T10:37:48.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:48 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:48.304+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:37:48.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:48 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:48.304+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:37:49.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:48 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:48.922+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:37:49.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:48 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:48.922+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:37:49.511 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:37:49 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:37:49.128+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:37:49.511 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:49 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:49.336+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:13.956208+0000 front 2023-12-07T10:37:13.956221+0000 (oldest deadline 2023-12-07T10:37:38.056093+0000) 2023-12-07T10:37:49.512 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:49 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:49.336+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.557260+0000 front 2023-12-07T10:37:21.557225+0000 (oldest deadline 2023-12-07T10:37:46.857275+0000) 2023-12-07T10:37:49.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:49 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:49.341+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:14.001791+0000 front 2023-12-07T10:37:14.001815+0000 (oldest deadline 2023-12-07T10:37:39.301932+0000) 2023-12-07T10:37:49.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:37:49 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:37:49.380+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.206897+0000 front 2023-12-07T10:37:22.206862+0000 (oldest deadline 2023-12-07T10:37:48.107185+0000) 2023-12-07T10:37:49.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:49 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:49.320+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:37:49.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:49 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:49.320+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:37:50.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:49 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:49.512+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:37:50.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:49 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:49.971+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:37:50.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:49 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:49.971+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:37:50.415 INFO:journalctl@ceph.osd.4.smithi153.stdout:Dec 07 10:37:50 smithi153 bash[179512]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-4 2023-12-07T10:37:50.415 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:37:50 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:37:50.413+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.206897+0000 front 2023-12-07T10:37:22.206862+0000 (oldest deadline 2023-12-07T10:37:48.107185+0000) 2023-12-07T10:37:50.415 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:50 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:50.341+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:37:50.416 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:50 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:50.341+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:37:50.470 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:37:50 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:37:50.120+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:37:50.470 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:50 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:50.341+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:13.956208+0000 front 2023-12-07T10:37:13.956221+0000 (oldest deadline 2023-12-07T10:37:38.056093+0000) 2023-12-07T10:37:50.470 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:50 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:50.341+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.557260+0000 front 2023-12-07T10:37:21.557225+0000 (oldest deadline 2023-12-07T10:37:46.857275+0000) 2023-12-07T10:37:50.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:50 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:50.348+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:14.001791+0000 front 2023-12-07T10:37:14.001815+0000 (oldest deadline 2023-12-07T10:37:39.301932+0000) 2023-12-07T10:37:50.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:50 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:50.471+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:37:50.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:50 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:50.471+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:37:51.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:50 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:50.946+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:37:51.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:50 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:50.946+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:37:51.459 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:37:51 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:37:51.120+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:37:51.459 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:51 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:51.350+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:13.956208+0000 front 2023-12-07T10:37:13.956221+0000 (oldest deadline 2023-12-07T10:37:38.056093+0000) 2023-12-07T10:37:51.459 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:51 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:51.350+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.557260+0000 front 2023-12-07T10:37:21.557225+0000 (oldest deadline 2023-12-07T10:37:46.857275+0000) 2023-12-07T10:37:51.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:51 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:51.330+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:14.001791+0000 front 2023-12-07T10:37:14.001815+0000 (oldest deadline 2023-12-07T10:37:39.301932+0000) 2023-12-07T10:37:51.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:51 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:51.459+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:37:51.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:51 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:51.459+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:37:51.769 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:37:51 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:37:51.373+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.206897+0000 front 2023-12-07T10:37:22.206862+0000 (oldest deadline 2023-12-07T10:37:48.107185+0000) 2023-12-07T10:37:51.769 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:51 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:51.353+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:37:51.770 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:51 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:51.353+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:37:51.777 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.4.service' 2023-12-07T10:37:51.817 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:37:51.817 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-12-07T10:37:51.817 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-12-07T10:37:51.817 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.5 2023-12-07T10:37:52.099 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:51 smithi159 systemd[1]: Stopping Ceph osd.5 for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:37:52.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:51 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:51.897+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:37:52.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:51 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:51.897+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:37:52.379 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:37:52 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:37:52.099+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:37:52.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:52 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:52.304+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:14.001791+0000 front 2023-12-07T10:37:14.001815+0000 (oldest deadline 2023-12-07T10:37:39.301932+0000) 2023-12-07T10:37:52.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:52 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:52.304+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.505951+0000 front 2023-12-07T10:37:25.506048+0000 (oldest deadline 2023-12-07T10:37:51.406070+0000) 2023-12-07T10:37:52.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:52 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:52.497+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:37:52.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:52 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:52.497+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:37:52.761 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:52 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:52.379+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:13.956208+0000 front 2023-12-07T10:37:13.956221+0000 (oldest deadline 2023-12-07T10:37:38.056093+0000) 2023-12-07T10:37:52.761 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:52 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:52.379+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.557260+0000 front 2023-12-07T10:37:21.557225+0000 (oldest deadline 2023-12-07T10:37:46.857275+0000) 2023-12-07T10:37:52.762 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:52 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:52.525+0000 7f930bb9a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:37:52.762 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:52 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:52.525+0000 7f930bb9a700 -1 osd.5 118 *** Got signal Terminated *** 2023-12-07T10:37:52.762 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:52 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:52.525+0000 7f930bb9a700 -1 osd.5 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-07T10:37:52.777 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:37:52 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:37:52.325+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.206897+0000 front 2023-12-07T10:37:22.206862+0000 (oldest deadline 2023-12-07T10:37:48.107185+0000) 2023-12-07T10:37:52.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:52 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:52.327+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:37:52.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:52 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:52.327+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:37:53.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:52 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:52.947+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:37:53.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:52 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:52.947+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:37:53.388 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:37:53 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:37:53.053+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:37:53.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:53 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:53.318+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:14.001791+0000 front 2023-12-07T10:37:14.001815+0000 (oldest deadline 2023-12-07T10:37:39.301932+0000) 2023-12-07T10:37:53.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:53 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:53.318+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.505951+0000 front 2023-12-07T10:37:25.506048+0000 (oldest deadline 2023-12-07T10:37:51.406070+0000) 2023-12-07T10:37:53.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:53 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:53.494+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:37:53.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:53 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:53.494+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:37:53.761 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:53 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:53.389+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:13.956208+0000 front 2023-12-07T10:37:13.956221+0000 (oldest deadline 2023-12-07T10:37:38.056093+0000) 2023-12-07T10:37:53.761 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:53 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:53.389+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.557260+0000 front 2023-12-07T10:37:21.557225+0000 (oldest deadline 2023-12-07T10:37:46.857275+0000) 2023-12-07T10:37:53.777 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:37:53 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:37:53.318+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.206897+0000 front 2023-12-07T10:37:22.206862+0000 (oldest deadline 2023-12-07T10:37:48.107185+0000) 2023-12-07T10:37:53.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:53 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:53.309+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:37:53.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:53 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:53.309+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:37:54.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:53 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:53.950+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:37:54.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:53 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:53.950+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:37:54.379 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:37:54 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:37:54.083+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:37:54.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:54 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:54.283+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:14.001791+0000 front 2023-12-07T10:37:14.001815+0000 (oldest deadline 2023-12-07T10:37:39.301932+0000) 2023-12-07T10:37:54.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:54 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:54.283+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.505951+0000 front 2023-12-07T10:37:25.506048+0000 (oldest deadline 2023-12-07T10:37:51.406070+0000) 2023-12-07T10:37:54.761 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:54 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:54.379+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:13.956208+0000 front 2023-12-07T10:37:13.956221+0000 (oldest deadline 2023-12-07T10:37:38.056093+0000) 2023-12-07T10:37:54.761 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:54 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:54.379+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.557260+0000 front 2023-12-07T10:37:21.557225+0000 (oldest deadline 2023-12-07T10:37:46.857275+0000) 2023-12-07T10:37:54.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:54 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:54.471+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:37:54.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:54 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:54.471+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:37:54.777 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:37:54 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:37:54.365+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.206897+0000 front 2023-12-07T10:37:22.206862+0000 (oldest deadline 2023-12-07T10:37:48.107185+0000) 2023-12-07T10:37:54.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:54 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:54.297+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:37:54.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:54 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:54.297+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:37:55.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:54 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:54.960+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:37:55.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:54 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:54.960+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:37:55.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:54 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:54.960+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:37:55.370 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:37:55 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:37:55.037+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:37:55.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:55 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:55.289+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:14.001791+0000 front 2023-12-07T10:37:14.001815+0000 (oldest deadline 2023-12-07T10:37:39.301932+0000) 2023-12-07T10:37:55.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:55 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:55.289+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.505951+0000 front 2023-12-07T10:37:25.506048+0000 (oldest deadline 2023-12-07T10:37:51.406070+0000) 2023-12-07T10:37:55.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:55 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:55.289+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.406944+0000 front 2023-12-07T10:37:31.407001+0000 (oldest deadline 2023-12-07T10:37:54.907059+0000) 2023-12-07T10:37:55.761 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:55 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:55.371+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:13.956208+0000 front 2023-12-07T10:37:13.956221+0000 (oldest deadline 2023-12-07T10:37:38.056093+0000) 2023-12-07T10:37:55.761 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:55 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:55.371+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.557260+0000 front 2023-12-07T10:37:21.557225+0000 (oldest deadline 2023-12-07T10:37:46.857275+0000) 2023-12-07T10:37:55.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:55 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:55.513+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:37:55.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:55 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:55.513+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:37:55.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:55 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:55.513+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:37:55.777 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:37:55 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:37:55.384+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.206897+0000 front 2023-12-07T10:37:22.206862+0000 (oldest deadline 2023-12-07T10:37:48.107185+0000) 2023-12-07T10:37:55.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:55 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:55.303+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:37:55.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:55 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:55.303+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:37:56.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:55 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:55.934+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:37:56.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:55 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:55.934+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:37:56.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:55 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:55.934+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:37:56.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:37:55 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:37:55.992+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:37:56.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:37:55 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:37:55.992+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.767435+0000 front 2023-12-07T10:37:31.767410+0000 (oldest deadline 2023-12-07T10:37:55.267563+0000) 2023-12-07T10:37:56.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:56 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:56.307+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:14.001791+0000 front 2023-12-07T10:37:14.001815+0000 (oldest deadline 2023-12-07T10:37:39.301932+0000) 2023-12-07T10:37:56.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:56 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:56.307+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.505951+0000 front 2023-12-07T10:37:25.506048+0000 (oldest deadline 2023-12-07T10:37:51.406070+0000) 2023-12-07T10:37:56.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:56 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:56.307+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.406944+0000 front 2023-12-07T10:37:31.407001+0000 (oldest deadline 2023-12-07T10:37:54.907059+0000) 2023-12-07T10:37:56.761 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:56 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:56.329+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:13.956208+0000 front 2023-12-07T10:37:13.956221+0000 (oldest deadline 2023-12-07T10:37:38.056093+0000) 2023-12-07T10:37:56.761 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:56 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:56.329+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.557260+0000 front 2023-12-07T10:37:21.557225+0000 (oldest deadline 2023-12-07T10:37:46.857275+0000) 2023-12-07T10:37:56.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:56 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:56.563+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:37:56.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:56 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:56.563+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:37:56.762 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:56 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:56.563+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:37:56.777 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:37:56 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:37:56.392+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.206897+0000 front 2023-12-07T10:37:22.206862+0000 (oldest deadline 2023-12-07T10:37:48.107185+0000) 2023-12-07T10:37:56.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:37:56 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:37:56.392+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.308926+0000 front 2023-12-07T10:37:30.309131+0000 (oldest deadline 2023-12-07T10:37:56.208937+0000) 2023-12-07T10:37:56.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:56 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:56.333+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:37:56.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:56 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:56.333+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:37:57.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:56 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:56.954+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:37:57.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:56 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:56.954+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:37:57.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:56 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:56.954+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:37:57.343 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:37:57 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:37:57.026+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:37:57.343 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:37:57 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:37:57.026+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.767435+0000 front 2023-12-07T10:37:31.767410+0000 (oldest deadline 2023-12-07T10:37:55.267563+0000) 2023-12-07T10:37:57.596 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:57 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:57.343+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:13.956208+0000 front 2023-12-07T10:37:13.956221+0000 (oldest deadline 2023-12-07T10:37:38.056093+0000) 2023-12-07T10:37:57.597 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:57 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5[149250]: 2023-12-07T10:37:57.343+0000 7f9306992700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.557260+0000 front 2023-12-07T10:37:21.557225+0000 (oldest deadline 2023-12-07T10:37:46.857275+0000) 2023-12-07T10:37:57.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:57 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:57.276+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:14.001791+0000 front 2023-12-07T10:37:14.001815+0000 (oldest deadline 2023-12-07T10:37:39.301932+0000) 2023-12-07T10:37:57.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:57 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:57.276+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.505951+0000 front 2023-12-07T10:37:25.506048+0000 (oldest deadline 2023-12-07T10:37:51.406070+0000) 2023-12-07T10:37:57.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:57 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:57.276+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.406944+0000 front 2023-12-07T10:37:31.407001+0000 (oldest deadline 2023-12-07T10:37:54.907059+0000) 2023-12-07T10:37:57.777 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:37:57 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:37:57.345+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.206897+0000 front 2023-12-07T10:37:22.206862+0000 (oldest deadline 2023-12-07T10:37:48.107185+0000) 2023-12-07T10:37:57.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:37:57 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:37:57.345+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.308926+0000 front 2023-12-07T10:37:30.309131+0000 (oldest deadline 2023-12-07T10:37:56.208937+0000) 2023-12-07T10:37:57.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:57 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:57.302+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:37:57.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:57 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:57.302+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:37:58.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:57 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:57.596+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:37:58.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:57 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:57.596+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:37:58.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:57 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:57.596+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:37:58.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:57 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:57.935+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:37:58.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:57 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:57.935+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:37:58.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:57 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:57.935+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:37:58.511 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:58 smithi159 bash[181517]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-5 2023-12-07T10:37:58.540 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:37:58 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:37:58.031+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:37:58.541 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:37:58 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:37:58.031+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.767435+0000 front 2023-12-07T10:37:31.767410+0000 (oldest deadline 2023-12-07T10:37:55.267563+0000) 2023-12-07T10:37:58.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:58 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:58.315+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:14.001791+0000 front 2023-12-07T10:37:14.001815+0000 (oldest deadline 2023-12-07T10:37:39.301932+0000) 2023-12-07T10:37:58.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:58 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:58.315+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.505951+0000 front 2023-12-07T10:37:25.506048+0000 (oldest deadline 2023-12-07T10:37:51.406070+0000) 2023-12-07T10:37:58.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:58 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:58.315+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.406944+0000 front 2023-12-07T10:37:31.407001+0000 (oldest deadline 2023-12-07T10:37:54.907059+0000) 2023-12-07T10:37:58.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:37:58 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:37:58.306+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.206897+0000 front 2023-12-07T10:37:22.206862+0000 (oldest deadline 2023-12-07T10:37:48.107185+0000) 2023-12-07T10:37:58.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:37:58 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:37:58.306+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.308926+0000 front 2023-12-07T10:37:30.309131+0000 (oldest deadline 2023-12-07T10:37:56.208937+0000) 2023-12-07T10:37:58.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:58 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:58.315+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:37:58.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:58 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:58.315+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:37:59.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:58 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:58.593+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:37:59.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:58 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:58.593+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:37:59.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:58 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:58.593+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:37:59.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:58 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:58.922+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:37:59.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:58 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:58.922+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:37:59.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:58 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:58.922+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:37:59.285 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:37:59 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:37:59.025+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:37:59.285 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:37:59 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:37:59.025+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.767435+0000 front 2023-12-07T10:37:31.767410+0000 (oldest deadline 2023-12-07T10:37:55.267563+0000) 2023-12-07T10:37:59.531 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.5.service' 2023-12-07T10:37:59.557 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:59 smithi159 systemd[1]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.5.service: Succeeded. 2023-12-07T10:37:59.557 INFO:journalctl@ceph.osd.5.smithi159.stdout:Dec 07 10:37:59 smithi159 systemd[1]: Stopped Ceph osd.5 for a4c642c6-94e8-11ee-95a2-87774f69a715. 2023-12-07T10:37:59.571 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:37:59.572 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-12-07T10:37:59.572 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-12-07T10:37:59.572 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.6 2023-12-07T10:37:59.638 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:59 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:59.276+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:14.001791+0000 front 2023-12-07T10:37:14.001815+0000 (oldest deadline 2023-12-07T10:37:39.301932+0000) 2023-12-07T10:37:59.638 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:59 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:59.276+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.505951+0000 front 2023-12-07T10:37:25.506048+0000 (oldest deadline 2023-12-07T10:37:51.406070+0000) 2023-12-07T10:37:59.638 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:59 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:37:59.276+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.406944+0000 front 2023-12-07T10:37:31.407001+0000 (oldest deadline 2023-12-07T10:37:54.907059+0000) 2023-12-07T10:37:59.777 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:37:59 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:37:59.334+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.206897+0000 front 2023-12-07T10:37:22.206862+0000 (oldest deadline 2023-12-07T10:37:48.107185+0000) 2023-12-07T10:37:59.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:37:59 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:37:59.334+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.308926+0000 front 2023-12-07T10:37:30.309131+0000 (oldest deadline 2023-12-07T10:37:56.208937+0000) 2023-12-07T10:37:59.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:59 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:59.322+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:37:59.781 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:59 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:59.322+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:37:59.781 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:37:59 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:37:59.322+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:37:59.950 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:37:59 smithi119 systemd[1]: Stopping Ceph osd.6 for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:38:00.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:59 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:59.616+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:00.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:59 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:59.616+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:00.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:37:59 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:37:59.616+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:00.227 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:38:00 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:38:00.101+0000 7f2ced670700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:38:00.227 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:38:00 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:38:00.101+0000 7f2ced670700 -1 osd.6 118 *** Got signal Terminated *** 2023-12-07T10:38:00.227 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:38:00 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:38:00.101+0000 7f2ced670700 -1 osd.6 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-07T10:38:00.228 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:59 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:59.950+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:00.228 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:59 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:59.950+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:00.228 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:37:59 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:37:59.950+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:00.511 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:00 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:00.046+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:38:00.511 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:00 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:00.046+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.767435+0000 front 2023-12-07T10:37:31.767410+0000 (oldest deadline 2023-12-07T10:37:55.267563+0000) 2023-12-07T10:38:00.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:38:00 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:38:00.259+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:14.001791+0000 front 2023-12-07T10:37:14.001815+0000 (oldest deadline 2023-12-07T10:37:39.301932+0000) 2023-12-07T10:38:00.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:38:00 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:38:00.259+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.505951+0000 front 2023-12-07T10:37:25.506048+0000 (oldest deadline 2023-12-07T10:37:51.406070+0000) 2023-12-07T10:38:00.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:38:00 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:38:00.259+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.406944+0000 front 2023-12-07T10:37:31.407001+0000 (oldest deadline 2023-12-07T10:37:54.907059+0000) 2023-12-07T10:38:00.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:00 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:00.382+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.206897+0000 front 2023-12-07T10:37:22.206862+0000 (oldest deadline 2023-12-07T10:37:48.107185+0000) 2023-12-07T10:38:00.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:00 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:00.382+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.308926+0000 front 2023-12-07T10:37:30.309131+0000 (oldest deadline 2023-12-07T10:37:56.208937+0000) 2023-12-07T10:38:00.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:00 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:00.297+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:00.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:00 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:00.297+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:00.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:00 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:00.297+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:01.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:00 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:00.643+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:01.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:00 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:00.643+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:01.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:00 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:00.643+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:01.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:00 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:00.905+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:01.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:00 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:00.905+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:01.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:00 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:00.905+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:01.511 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:01 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:01.042+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:38:01.511 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:01 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:01.042+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.767435+0000 front 2023-12-07T10:37:31.767410+0000 (oldest deadline 2023-12-07T10:37:55.267563+0000) 2023-12-07T10:38:01.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:38:01 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:38:01.267+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:14.001791+0000 front 2023-12-07T10:37:14.001815+0000 (oldest deadline 2023-12-07T10:37:39.301932+0000) 2023-12-07T10:38:01.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:38:01 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:38:01.267+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.505951+0000 front 2023-12-07T10:37:25.506048+0000 (oldest deadline 2023-12-07T10:37:51.406070+0000) 2023-12-07T10:38:01.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:38:01 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:38:01.267+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.406944+0000 front 2023-12-07T10:37:31.407001+0000 (oldest deadline 2023-12-07T10:37:54.907059+0000) 2023-12-07T10:38:01.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:01 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:01.379+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.206897+0000 front 2023-12-07T10:37:22.206862+0000 (oldest deadline 2023-12-07T10:37:48.107185+0000) 2023-12-07T10:38:01.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:01 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:01.379+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.308926+0000 front 2023-12-07T10:37:30.309131+0000 (oldest deadline 2023-12-07T10:37:56.208937+0000) 2023-12-07T10:38:01.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:01 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:01.340+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:01.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:01 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:01.340+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:01.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:01 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:01.340+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:02.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:01 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:01.633+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:02.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:01 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:01.633+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:02.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:01 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:01.633+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:02.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:01 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:01.877+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:02.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:01 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:01.877+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:02.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:01 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:01.877+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:02.511 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:02 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:02.062+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:38:02.511 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:02 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:02.062+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.767435+0000 front 2023-12-07T10:37:31.767410+0000 (oldest deadline 2023-12-07T10:37:55.267563+0000) 2023-12-07T10:38:02.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:38:02 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:38:02.244+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:14.001791+0000 front 2023-12-07T10:37:14.001815+0000 (oldest deadline 2023-12-07T10:37:39.301932+0000) 2023-12-07T10:38:02.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:38:02 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:38:02.244+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.505951+0000 front 2023-12-07T10:37:25.506048+0000 (oldest deadline 2023-12-07T10:37:51.406070+0000) 2023-12-07T10:38:02.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:38:02 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:38:02.244+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.406944+0000 front 2023-12-07T10:37:31.407001+0000 (oldest deadline 2023-12-07T10:37:54.907059+0000) 2023-12-07T10:38:02.777 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:02 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:02.353+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.206897+0000 front 2023-12-07T10:37:22.206862+0000 (oldest deadline 2023-12-07T10:37:48.107185+0000) 2023-12-07T10:38:02.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:02 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:02.353+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.308926+0000 front 2023-12-07T10:37:30.309131+0000 (oldest deadline 2023-12-07T10:37:56.208937+0000) 2023-12-07T10:38:02.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:02 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:02.343+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:02.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:02 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:02.343+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:02.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:02 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:02.343+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:02.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:02 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:02.343+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:03.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:02 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:02.660+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:03.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:02 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:02.661+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:03.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:02 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:02.661+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:03.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:02 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:02.860+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:03.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:02 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:02.860+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:03.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:02 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:02.860+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:03.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:02 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:02.860+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.523947+0000 front 2023-12-07T10:37:40.523979+0000 (oldest deadline 2023-12-07T10:38:02.224192+0000) 2023-12-07T10:38:03.511 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:03 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:03.029+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:38:03.511 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:03 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:03.029+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.767435+0000 front 2023-12-07T10:37:31.767410+0000 (oldest deadline 2023-12-07T10:37:55.267563+0000) 2023-12-07T10:38:03.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:38:03 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:38:03.268+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:14.001791+0000 front 2023-12-07T10:37:14.001815+0000 (oldest deadline 2023-12-07T10:37:39.301932+0000) 2023-12-07T10:38:03.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:38:03 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:38:03.268+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.505951+0000 front 2023-12-07T10:37:25.506048+0000 (oldest deadline 2023-12-07T10:37:51.406070+0000) 2023-12-07T10:38:03.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:38:03 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:38:03.268+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.406944+0000 front 2023-12-07T10:37:31.407001+0000 (oldest deadline 2023-12-07T10:37:54.907059+0000) 2023-12-07T10:38:03.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:03 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:03.331+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.206897+0000 front 2023-12-07T10:37:22.206862+0000 (oldest deadline 2023-12-07T10:37:48.107185+0000) 2023-12-07T10:38:03.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:03 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:03.331+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.308926+0000 front 2023-12-07T10:37:30.309131+0000 (oldest deadline 2023-12-07T10:37:56.208937+0000) 2023-12-07T10:38:03.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:03 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:03.338+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:03.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:03 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:03.338+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:03.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:03 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:03.338+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:03.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:03 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:03.338+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:04.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:03 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:03.648+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:04.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:03 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:03.648+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:04.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:03 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:03.648+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:04.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:03 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:03.887+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:04.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:03 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:03.887+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:04.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:03 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:03.887+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:04.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:03 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:03.887+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.523947+0000 front 2023-12-07T10:37:40.523979+0000 (oldest deadline 2023-12-07T10:38:02.224192+0000) 2023-12-07T10:38:04.511 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:04 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:04.046+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:38:04.511 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:04 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:04.046+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.767435+0000 front 2023-12-07T10:37:31.767410+0000 (oldest deadline 2023-12-07T10:37:55.267563+0000) 2023-12-07T10:38:04.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:38:04 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:38:04.230+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:14.001791+0000 front 2023-12-07T10:37:14.001815+0000 (oldest deadline 2023-12-07T10:37:39.301932+0000) 2023-12-07T10:38:04.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:38:04 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:38:04.230+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.505951+0000 front 2023-12-07T10:37:25.506048+0000 (oldest deadline 2023-12-07T10:37:51.406070+0000) 2023-12-07T10:38:04.727 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:38:04 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6[130260]: 2023-12-07T10:38:04.230+0000 7f2ce7c67700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.406944+0000 front 2023-12-07T10:37:31.407001+0000 (oldest deadline 2023-12-07T10:37:54.907059+0000) 2023-12-07T10:38:04.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:04 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:04.302+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.206897+0000 front 2023-12-07T10:37:22.206862+0000 (oldest deadline 2023-12-07T10:37:48.107185+0000) 2023-12-07T10:38:04.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:04 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:04.302+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.308926+0000 front 2023-12-07T10:37:30.309131+0000 (oldest deadline 2023-12-07T10:37:56.208937+0000) 2023-12-07T10:38:04.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:04 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:04.388+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:04.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:04 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:04.388+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:04.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:04 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:04.388+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:04.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:04 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:04.388+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:04.999 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:04 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:04.639+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:04.999 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:04 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:04.639+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:05.000 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:04 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:04.639+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:05.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:04 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:04.920+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:05.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:04 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:04.920+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:05.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:04 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:04.920+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:05.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:04 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:04.920+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.523947+0000 front 2023-12-07T10:37:40.523979+0000 (oldest deadline 2023-12-07T10:38:02.224192+0000) 2023-12-07T10:38:05.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:05 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:04.999+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:38:05.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:05 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:04.999+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.767435+0000 front 2023-12-07T10:37:31.767410+0000 (oldest deadline 2023-12-07T10:37:55.267563+0000) 2023-12-07T10:38:05.528 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:05 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:05.254+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.206897+0000 front 2023-12-07T10:37:22.206862+0000 (oldest deadline 2023-12-07T10:37:48.107185+0000) 2023-12-07T10:38:05.528 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:05 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:05.254+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.308926+0000 front 2023-12-07T10:37:30.309131+0000 (oldest deadline 2023-12-07T10:37:56.208937+0000) 2023-12-07T10:38:05.528 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:05 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:05.366+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:05.528 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:05 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:05.366+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:05.528 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:05 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:05.366+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:05.528 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:05 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:05.366+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:05.998 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:05 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:05.610+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:05.998 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:05 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:05.610+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:05.999 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:05 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:05.610+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:06.227 INFO:journalctl@ceph.osd.6.smithi119.stdout:Dec 07 10:38:05 smithi119 bash[181511]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-6 2023-12-07T10:38:06.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:05 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:05.887+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:06.228 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:05 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:05.887+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:06.228 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:05 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:05.887+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:06.228 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:05 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:05.887+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.523947+0000 front 2023-12-07T10:37:40.523979+0000 (oldest deadline 2023-12-07T10:38:02.224192+0000) 2023-12-07T10:38:06.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:05 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:05.998+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:38:06.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:05 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:05.998+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.767435+0000 front 2023-12-07T10:37:31.767410+0000 (oldest deadline 2023-12-07T10:37:55.267563+0000) 2023-12-07T10:38:06.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:05 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:05.998+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:41.069771+0000 front 2023-12-07T10:37:41.069673+0000 (oldest deadline 2023-12-07T10:38:05.769602+0000) 2023-12-07T10:38:06.528 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:06 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:06.268+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.206897+0000 front 2023-12-07T10:37:22.206862+0000 (oldest deadline 2023-12-07T10:37:48.107185+0000) 2023-12-07T10:38:06.528 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:06 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:06.268+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.308926+0000 front 2023-12-07T10:37:30.309131+0000 (oldest deadline 2023-12-07T10:37:56.208937+0000) 2023-12-07T10:38:06.528 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:06 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:06.268+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.910122+0000 front 2023-12-07T10:37:40.910141+0000 (oldest deadline 2023-12-07T10:38:05.610016+0000) 2023-12-07T10:38:06.528 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:06 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:06.347+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:06.529 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:06 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:06.347+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:06.529 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:06 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:06.347+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:06.529 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:06 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:06.347+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:06.966 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:06 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:06.593+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:06.967 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:06 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:06.593+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:06.967 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:06 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:06.593+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:07.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:06 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:06.857+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:07.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:06 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:06.857+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:07.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:06 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:06.857+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:07.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:06 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:06.857+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.523947+0000 front 2023-12-07T10:37:40.523979+0000 (oldest deadline 2023-12-07T10:38:02.224192+0000) 2023-12-07T10:38:07.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:06 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:06.966+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:38:07.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:06 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:06.966+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.767435+0000 front 2023-12-07T10:37:31.767410+0000 (oldest deadline 2023-12-07T10:37:55.267563+0000) 2023-12-07T10:38:07.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:06 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:06.966+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:41.069771+0000 front 2023-12-07T10:37:41.069673+0000 (oldest deadline 2023-12-07T10:38:05.769602+0000) 2023-12-07T10:38:07.528 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:07 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:07.228+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.206897+0000 front 2023-12-07T10:37:22.206862+0000 (oldest deadline 2023-12-07T10:37:48.107185+0000) 2023-12-07T10:38:07.528 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:07 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:07.228+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.308926+0000 front 2023-12-07T10:37:30.309131+0000 (oldest deadline 2023-12-07T10:37:56.208937+0000) 2023-12-07T10:38:07.528 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:07 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:07.228+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.910122+0000 front 2023-12-07T10:37:40.910141+0000 (oldest deadline 2023-12-07T10:38:05.610016+0000) 2023-12-07T10:38:07.528 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:07 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:07.329+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:07.528 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:07 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:07.329+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:07.529 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:07 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:07.329+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:07.529 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:07 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:07.329+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:07.943 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.6.service' 2023-12-07T10:38:07.981 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:07 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:07.606+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:07.981 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:07 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:07.606+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:07.982 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:07 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:07.606+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:08.032 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:38:08.032 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-12-07T10:38:08.033 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-12-07T10:38:08.033 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.7 2023-12-07T10:38:08.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:07 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:07.899+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:08.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:07 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:07.899+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:08.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:07 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:07.899+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:08.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:07 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:07.899+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.523947+0000 front 2023-12-07T10:37:40.523979+0000 (oldest deadline 2023-12-07T10:38:02.224192+0000) 2023-12-07T10:38:08.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:07 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:07.981+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:38:08.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:07 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:07.981+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.767435+0000 front 2023-12-07T10:37:31.767410+0000 (oldest deadline 2023-12-07T10:37:55.267563+0000) 2023-12-07T10:38:08.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:07 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:07.981+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:41.069771+0000 front 2023-12-07T10:37:41.069673+0000 (oldest deadline 2023-12-07T10:38:05.769602+0000) 2023-12-07T10:38:08.365 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:08 smithi153 systemd[1]: Stopping Ceph osd.7 for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:38:08.366 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:08 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:08.277+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.206897+0000 front 2023-12-07T10:37:22.206862+0000 (oldest deadline 2023-12-07T10:37:48.107185+0000) 2023-12-07T10:38:08.366 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:08 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:08.277+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.308926+0000 front 2023-12-07T10:37:30.309131+0000 (oldest deadline 2023-12-07T10:37:56.208937+0000) 2023-12-07T10:38:08.366 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:08 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:08.277+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.910122+0000 front 2023-12-07T10:37:40.910141+0000 (oldest deadline 2023-12-07T10:38:05.610016+0000) 2023-12-07T10:38:08.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:08 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:08.391+0000 7f6787595700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:38:08.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:08 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:08.391+0000 7f6787595700 -1 osd.7 118 *** Got signal Terminated *** 2023-12-07T10:38:08.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:08 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:08.391+0000 7f6787595700 -1 osd.7 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-07T10:38:08.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:08 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:08.365+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:08.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:08 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:08.365+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:08.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:08 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:08.365+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:08.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:08 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:08.365+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:08.934 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:08 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:08.651+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:08.934 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:08 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:08.651+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:08.934 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:08 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:08.651+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:09.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:08 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:08.903+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:09.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:08 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:08.903+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:09.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:08 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:08.903+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:09.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:08 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:08.903+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.523947+0000 front 2023-12-07T10:37:40.523979+0000 (oldest deadline 2023-12-07T10:38:02.224192+0000) 2023-12-07T10:38:09.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:08 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:08.934+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:38:09.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:08 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:08.934+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.767435+0000 front 2023-12-07T10:37:31.767410+0000 (oldest deadline 2023-12-07T10:37:55.267563+0000) 2023-12-07T10:38:09.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:08 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:08.934+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:41.069771+0000 front 2023-12-07T10:37:41.069673+0000 (oldest deadline 2023-12-07T10:38:05.769602+0000) 2023-12-07T10:38:09.528 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:09 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:09.264+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.206897+0000 front 2023-12-07T10:37:22.206862+0000 (oldest deadline 2023-12-07T10:37:48.107185+0000) 2023-12-07T10:38:09.528 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:09 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:09.264+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.308926+0000 front 2023-12-07T10:37:30.309131+0000 (oldest deadline 2023-12-07T10:37:56.208937+0000) 2023-12-07T10:38:09.528 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:09 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:09.264+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.910122+0000 front 2023-12-07T10:37:40.910141+0000 (oldest deadline 2023-12-07T10:38:05.610016+0000) 2023-12-07T10:38:09.528 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:09 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:09.317+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:09.529 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:09 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:09.317+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:09.529 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:09 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:09.317+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:09.529 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:09 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:09.317+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:10.011 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:09 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:09.893+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:38:10.011 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:09 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:09.893+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.767435+0000 front 2023-12-07T10:37:31.767410+0000 (oldest deadline 2023-12-07T10:37:55.267563+0000) 2023-12-07T10:38:10.011 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:09 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:09.893+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:41.069771+0000 front 2023-12-07T10:37:41.069673+0000 (oldest deadline 2023-12-07T10:38:05.769602+0000) 2023-12-07T10:38:10.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:09 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:09.645+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:10.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:09 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:09.645+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:10.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:09 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:09.645+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:10.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:09 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:09.911+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:10.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:09 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:09.911+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:10.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:09 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:09.911+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:10.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:09 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:09.911+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.523947+0000 front 2023-12-07T10:37:40.523979+0000 (oldest deadline 2023-12-07T10:38:02.224192+0000) 2023-12-07T10:38:10.528 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:10 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:10.222+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.206897+0000 front 2023-12-07T10:37:22.206862+0000 (oldest deadline 2023-12-07T10:37:48.107185+0000) 2023-12-07T10:38:10.528 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:10 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:10.222+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.308926+0000 front 2023-12-07T10:37:30.309131+0000 (oldest deadline 2023-12-07T10:37:56.208937+0000) 2023-12-07T10:38:10.528 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:10 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:10.222+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.910122+0000 front 2023-12-07T10:37:40.910141+0000 (oldest deadline 2023-12-07T10:38:05.610016+0000) 2023-12-07T10:38:10.528 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:10 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:10.339+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:10.528 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:10 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:10.339+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:10.529 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:10 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:10.339+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:10.529 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:10 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:10.339+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:10.887 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:10 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:10.597+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:10.888 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:10 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:10.597+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:10.888 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:10 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:10.597+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:11.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:10 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:10.936+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:11.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:10 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:10.936+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:11.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:10 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:10.936+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:11.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:10 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:10.936+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.523947+0000 front 2023-12-07T10:37:40.523979+0000 (oldest deadline 2023-12-07T10:38:02.224192+0000) 2023-12-07T10:38:11.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:10 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:10.887+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:38:11.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:10 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:10.887+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.767435+0000 front 2023-12-07T10:37:31.767410+0000 (oldest deadline 2023-12-07T10:37:55.267563+0000) 2023-12-07T10:38:11.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:10 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:10.887+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:41.069771+0000 front 2023-12-07T10:37:41.069673+0000 (oldest deadline 2023-12-07T10:38:05.769602+0000) 2023-12-07T10:38:11.262 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:10 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:10.887+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:49.270777+0000 front 2023-12-07T10:37:49.270812+0000 (oldest deadline 2023-12-07T10:38:10.370821+0000) 2023-12-07T10:38:11.528 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:11 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:11.236+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.206897+0000 front 2023-12-07T10:37:22.206862+0000 (oldest deadline 2023-12-07T10:37:48.107185+0000) 2023-12-07T10:38:11.528 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:11 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:11.236+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.308926+0000 front 2023-12-07T10:37:30.309131+0000 (oldest deadline 2023-12-07T10:37:56.208937+0000) 2023-12-07T10:38:11.528 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:11 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:11.236+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.910122+0000 front 2023-12-07T10:37:40.910141+0000 (oldest deadline 2023-12-07T10:38:05.610016+0000) 2023-12-07T10:38:11.528 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:11 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:11.236+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:45.610342+0000 front 2023-12-07T10:37:45.610389+0000 (oldest deadline 2023-12-07T10:38:10.910636+0000) 2023-12-07T10:38:11.529 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:11 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:11.338+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:11.529 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:11 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:11.338+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:11.529 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:11 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:11.338+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:11.529 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:11 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:11.338+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:11.871 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:11 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:11.574+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:11.872 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:11 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:11.574+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:11.872 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:11 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:11.574+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:12.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:11 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:11.915+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:12.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:11 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:11.915+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:12.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:11 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:11.915+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:12.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:11 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:11.915+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.523947+0000 front 2023-12-07T10:37:40.523979+0000 (oldest deadline 2023-12-07T10:38:02.224192+0000) 2023-12-07T10:38:12.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:11 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:11.871+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:38:12.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:11 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:11.871+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.767435+0000 front 2023-12-07T10:37:31.767410+0000 (oldest deadline 2023-12-07T10:37:55.267563+0000) 2023-12-07T10:38:12.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:11 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:11.871+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:41.069771+0000 front 2023-12-07T10:37:41.069673+0000 (oldest deadline 2023-12-07T10:38:05.769602+0000) 2023-12-07T10:38:12.262 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:11 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:11.871+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:49.270777+0000 front 2023-12-07T10:37:49.270812+0000 (oldest deadline 2023-12-07T10:38:10.370821+0000) 2023-12-07T10:38:12.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:12 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:12.285+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.206897+0000 front 2023-12-07T10:37:22.206862+0000 (oldest deadline 2023-12-07T10:37:48.107185+0000) 2023-12-07T10:38:12.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:12 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:12.285+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.308926+0000 front 2023-12-07T10:37:30.309131+0000 (oldest deadline 2023-12-07T10:37:56.208937+0000) 2023-12-07T10:38:12.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:12 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:12.285+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.910122+0000 front 2023-12-07T10:37:40.910141+0000 (oldest deadline 2023-12-07T10:38:05.610016+0000) 2023-12-07T10:38:12.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:12 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:12.285+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:45.610342+0000 front 2023-12-07T10:37:45.610389+0000 (oldest deadline 2023-12-07T10:38:10.910636+0000) 2023-12-07T10:38:12.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:12 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:12.350+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:12.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:12 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:12.350+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:12.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:12 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:12.350+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:12.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:12 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:12.350+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:12.840 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:12 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:12.536+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:12.840 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:12 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:12.536+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:12.840 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:12 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:12.536+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:12.840 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:12 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:12.536+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:13.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:12 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:12.959+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:13.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:12 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:12.959+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:13.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:12 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:12.959+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:13.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:12 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:12.959+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.523947+0000 front 2023-12-07T10:37:40.523979+0000 (oldest deadline 2023-12-07T10:38:02.224192+0000) 2023-12-07T10:38:13.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:12 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:12.839+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:38:13.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:12 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:12.839+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.767435+0000 front 2023-12-07T10:37:31.767410+0000 (oldest deadline 2023-12-07T10:37:55.267563+0000) 2023-12-07T10:38:13.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:12 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:12.839+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:41.069771+0000 front 2023-12-07T10:37:41.069673+0000 (oldest deadline 2023-12-07T10:38:05.769602+0000) 2023-12-07T10:38:13.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:12 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:12.839+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:49.270777+0000 front 2023-12-07T10:37:49.270812+0000 (oldest deadline 2023-12-07T10:38:10.370821+0000) 2023-12-07T10:38:13.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:13 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:13.327+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.206897+0000 front 2023-12-07T10:37:22.206862+0000 (oldest deadline 2023-12-07T10:37:48.107185+0000) 2023-12-07T10:38:13.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:13 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:13.327+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.308926+0000 front 2023-12-07T10:37:30.309131+0000 (oldest deadline 2023-12-07T10:37:56.208937+0000) 2023-12-07T10:38:13.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:13 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:13.327+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.910122+0000 front 2023-12-07T10:37:40.910141+0000 (oldest deadline 2023-12-07T10:38:05.610016+0000) 2023-12-07T10:38:13.778 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:13 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7[155992]: 2023-12-07T10:38:13.327+0000 7f6781b8c700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:45.610342+0000 front 2023-12-07T10:37:45.610389+0000 (oldest deadline 2023-12-07T10:38:10.910636+0000) 2023-12-07T10:38:13.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:13 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:13.325+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:13.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:13 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:13.325+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:13.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:13 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:13.325+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:13.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:13 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:13.325+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:13.833 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:13 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:13.555+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:13.833 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:13 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:13.555+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:13.833 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:13 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:13.555+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:13.834 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:13 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:13.555+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:14.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:13 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:13.920+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:14.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:13 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:13.920+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:14.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:13 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:13.920+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:14.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:13 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:13.920+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.523947+0000 front 2023-12-07T10:37:40.523979+0000 (oldest deadline 2023-12-07T10:38:02.224192+0000) 2023-12-07T10:38:14.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:13 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:13.920+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:47.926800+0000 front 2023-12-07T10:37:47.926682+0000 (oldest deadline 2023-12-07T10:38:13.226741+0000) 2023-12-07T10:38:14.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:13 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:13.833+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:38:14.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:13 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:13.833+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.767435+0000 front 2023-12-07T10:37:31.767410+0000 (oldest deadline 2023-12-07T10:37:55.267563+0000) 2023-12-07T10:38:14.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:13 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:13.833+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:41.069771+0000 front 2023-12-07T10:37:41.069673+0000 (oldest deadline 2023-12-07T10:38:05.769602+0000) 2023-12-07T10:38:14.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:13 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:13.833+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:49.270777+0000 front 2023-12-07T10:37:49.270812+0000 (oldest deadline 2023-12-07T10:38:10.370821+0000) 2023-12-07T10:38:14.528 INFO:journalctl@ceph.osd.7.smithi153.stdout:Dec 07 10:38:14 smithi153 bash[179776]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-7 2023-12-07T10:38:14.528 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:14 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:14.371+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:14.528 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:14 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:14.371+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:14.528 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:14 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:14.371+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:14.529 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:14 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:14.371+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:14.529 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:14 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:14.371+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.627146+0000 front 2023-12-07T10:37:48.627171+0000 (oldest deadline 2023-12-07T10:38:13.327348+0000) 2023-12-07T10:38:14.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:14 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:14.507+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:14.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:14 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:14.507+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:14.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:14 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:14.507+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:14.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:14 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:14.507+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:15.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:14 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:14.968+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:15.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:14 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:14.968+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:15.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:14 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:14.968+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:15.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:14 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:14.968+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.523947+0000 front 2023-12-07T10:37:40.523979+0000 (oldest deadline 2023-12-07T10:38:02.224192+0000) 2023-12-07T10:38:15.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:14 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:14.968+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:47.926800+0000 front 2023-12-07T10:37:47.926682+0000 (oldest deadline 2023-12-07T10:38:13.226741+0000) 2023-12-07T10:38:15.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:14 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:14.825+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:38:15.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:14 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:14.825+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.767435+0000 front 2023-12-07T10:37:31.767410+0000 (oldest deadline 2023-12-07T10:37:55.267563+0000) 2023-12-07T10:38:15.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:14 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:14.825+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:41.069771+0000 front 2023-12-07T10:37:41.069673+0000 (oldest deadline 2023-12-07T10:38:05.769602+0000) 2023-12-07T10:38:15.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:14 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:14.825+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:49.270777+0000 front 2023-12-07T10:37:49.270812+0000 (oldest deadline 2023-12-07T10:38:10.370821+0000) 2023-12-07T10:38:15.528 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:15 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:15.337+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:15.528 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:15 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:15.337+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:15.528 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:15 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:15.337+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:15.528 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:15 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:15.337+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:15.528 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:15 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:15.337+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.627146+0000 front 2023-12-07T10:37:48.627171+0000 (oldest deadline 2023-12-07T10:38:13.327348+0000) 2023-12-07T10:38:15.568 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.7.service' 2023-12-07T10:38:15.609 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:38:15.609 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-12-07T10:38:15.609 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-12-07T10:38:15.610 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.8 2023-12-07T10:38:15.966 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:15 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:15.554+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:15.966 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:15 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:15.554+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:15.967 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:15 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:15.554+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:15.967 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:15 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:15.554+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:15.967 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:15 smithi159 systemd[1]: Stopping Ceph osd.8 for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:38:15.967 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:15 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:15.792+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:38:15.967 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:15 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:15.792+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.767435+0000 front 2023-12-07T10:37:31.767410+0000 (oldest deadline 2023-12-07T10:37:55.267563+0000) 2023-12-07T10:38:15.967 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:15 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:15.792+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:41.069771+0000 front 2023-12-07T10:37:41.069673+0000 (oldest deadline 2023-12-07T10:38:05.769602+0000) 2023-12-07T10:38:15.967 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:15 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:15.792+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:49.270777+0000 front 2023-12-07T10:37:49.270812+0000 (oldest deadline 2023-12-07T10:38:10.370821+0000) 2023-12-07T10:38:15.967 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:15 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:15.966+0000 7fdd4878f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:38:16.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:15 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:15.931+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:16.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:15 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:15.931+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:16.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:15 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:15.931+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:16.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:15 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:15.931+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.523947+0000 front 2023-12-07T10:37:40.523979+0000 (oldest deadline 2023-12-07T10:38:02.224192+0000) 2023-12-07T10:38:16.228 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:15 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:15.931+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:47.926800+0000 front 2023-12-07T10:37:47.926682+0000 (oldest deadline 2023-12-07T10:38:13.226741+0000) 2023-12-07T10:38:16.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:15 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:15.966+0000 7fdd4878f700 -1 osd.8 118 *** Got signal Terminated *** 2023-12-07T10:38:16.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:15 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:15.966+0000 7fdd4878f700 -1 osd.8 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-07T10:38:16.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:16 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:16.372+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:16.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:16 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:16.372+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:16.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:16 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:16.372+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:16.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:16 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:16.372+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:16.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:16 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:16.372+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.627146+0000 front 2023-12-07T10:37:48.627171+0000 (oldest deadline 2023-12-07T10:38:13.327348+0000) 2023-12-07T10:38:17.011 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:16 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:16.760+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:38:17.011 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:16 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:16.760+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.767435+0000 front 2023-12-07T10:37:31.767410+0000 (oldest deadline 2023-12-07T10:37:55.267563+0000) 2023-12-07T10:38:17.011 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:16 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:16.760+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:41.069771+0000 front 2023-12-07T10:37:41.069673+0000 (oldest deadline 2023-12-07T10:38:05.769602+0000) 2023-12-07T10:38:17.011 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:16 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:16.760+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:49.270777+0000 front 2023-12-07T10:37:49.270812+0000 (oldest deadline 2023-12-07T10:38:10.370821+0000) 2023-12-07T10:38:17.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:16 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:16.556+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:17.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:16 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:16.556+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:17.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:16 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:16.556+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:17.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:16 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:16.556+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:17.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:16 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:16.900+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:17.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:16 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:16.900+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:17.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:16 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:16.900+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:17.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:16 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:16.900+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.523947+0000 front 2023-12-07T10:37:40.523979+0000 (oldest deadline 2023-12-07T10:38:02.224192+0000) 2023-12-07T10:38:17.228 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:16 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:16.900+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:47.926800+0000 front 2023-12-07T10:37:47.926682+0000 (oldest deadline 2023-12-07T10:38:13.226741+0000) 2023-12-07T10:38:17.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:17 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:17.332+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:17.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:17 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:17.332+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:17.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:17 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:17.332+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:17.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:17 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:17.332+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:17.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:17 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:17.332+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.627146+0000 front 2023-12-07T10:37:48.627171+0000 (oldest deadline 2023-12-07T10:38:13.327348+0000) 2023-12-07T10:38:18.011 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:17 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:17.734+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:38:18.012 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:17 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:17.734+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.767435+0000 front 2023-12-07T10:37:31.767410+0000 (oldest deadline 2023-12-07T10:37:55.267563+0000) 2023-12-07T10:38:18.012 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:17 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:17.734+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:41.069771+0000 front 2023-12-07T10:37:41.069673+0000 (oldest deadline 2023-12-07T10:38:05.769602+0000) 2023-12-07T10:38:18.012 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:17 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:17.734+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:49.270777+0000 front 2023-12-07T10:37:49.270812+0000 (oldest deadline 2023-12-07T10:38:10.370821+0000) 2023-12-07T10:38:18.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:17 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:17.522+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:18.013 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:17 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:17.522+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:18.013 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:17 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:17.522+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:18.013 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:17 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:17.522+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:18.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:17 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:17.884+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:18.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:17 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:17.884+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:18.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:17 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:17.884+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:18.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:17 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:17.884+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.523947+0000 front 2023-12-07T10:37:40.523979+0000 (oldest deadline 2023-12-07T10:38:02.224192+0000) 2023-12-07T10:38:18.228 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:17 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:17.884+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:47.926800+0000 front 2023-12-07T10:37:47.926682+0000 (oldest deadline 2023-12-07T10:38:13.226741+0000) 2023-12-07T10:38:18.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:18 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:18.353+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:18.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:18 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:18.353+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:18.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:18 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:18.353+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:18.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:18 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:18.353+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:18.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:18 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:18.353+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.627146+0000 front 2023-12-07T10:37:48.627171+0000 (oldest deadline 2023-12-07T10:38:13.327348+0000) 2023-12-07T10:38:18.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:18 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:18.353+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.628466+0000 front 2023-12-07T10:37:55.628343+0000 (oldest deadline 2023-12-07T10:38:17.928359+0000) 2023-12-07T10:38:19.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:18 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:18.546+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:19.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:18 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:18.546+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:19.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:18 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:18.546+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:19.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:18 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:18.546+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:19.012 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:18 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:18.777+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:38:19.012 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:18 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:18.777+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.767435+0000 front 2023-12-07T10:37:31.767410+0000 (oldest deadline 2023-12-07T10:37:55.267563+0000) 2023-12-07T10:38:19.012 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:18 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:18.777+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:41.069771+0000 front 2023-12-07T10:37:41.069673+0000 (oldest deadline 2023-12-07T10:38:05.769602+0000) 2023-12-07T10:38:19.012 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:18 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:18.777+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:49.270777+0000 front 2023-12-07T10:37:49.270812+0000 (oldest deadline 2023-12-07T10:38:10.370821+0000) 2023-12-07T10:38:19.012 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:18 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:18.777+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:52.671833+0000 front 2023-12-07T10:37:52.671853+0000 (oldest deadline 2023-12-07T10:38:18.571905+0000) 2023-12-07T10:38:19.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:18 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:18.868+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:19.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:18 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:18.868+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:19.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:18 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:18.868+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:19.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:18 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:18.868+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.523947+0000 front 2023-12-07T10:37:40.523979+0000 (oldest deadline 2023-12-07T10:38:02.224192+0000) 2023-12-07T10:38:19.228 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:18 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:18.868+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:47.926800+0000 front 2023-12-07T10:37:47.926682+0000 (oldest deadline 2023-12-07T10:38:13.226741+0000) 2023-12-07T10:38:19.228 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:18 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:18.868+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:53.227705+0000 front 2023-12-07T10:37:53.227785+0000 (oldest deadline 2023-12-07T10:38:18.527199+0000) 2023-12-07T10:38:19.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:19 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:19.357+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:19.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:19 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:19.357+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:19.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:19 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:19.357+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:19.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:19 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:19.357+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:19.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:19 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:19.357+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.627146+0000 front 2023-12-07T10:37:48.627171+0000 (oldest deadline 2023-12-07T10:38:13.327348+0000) 2023-12-07T10:38:19.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:19 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:19.357+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.628466+0000 front 2023-12-07T10:37:55.628343+0000 (oldest deadline 2023-12-07T10:38:17.928359+0000) 2023-12-07T10:38:20.011 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:19 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:19.790+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:38:20.011 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:19 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:19.790+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.767435+0000 front 2023-12-07T10:37:31.767410+0000 (oldest deadline 2023-12-07T10:37:55.267563+0000) 2023-12-07T10:38:20.011 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:19 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:19.790+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:41.069771+0000 front 2023-12-07T10:37:41.069673+0000 (oldest deadline 2023-12-07T10:38:05.769602+0000) 2023-12-07T10:38:20.011 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:19 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:19.790+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:49.270777+0000 front 2023-12-07T10:37:49.270812+0000 (oldest deadline 2023-12-07T10:38:10.370821+0000) 2023-12-07T10:38:20.012 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:19 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:19.790+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:52.671833+0000 front 2023-12-07T10:37:52.671853+0000 (oldest deadline 2023-12-07T10:38:18.571905+0000) 2023-12-07T10:38:20.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:19 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:19.548+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:20.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:19 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:19.548+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:20.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:19 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:19.548+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:20.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:19 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:19.548+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:20.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:19 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:19.548+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.716425+0000 front 2023-12-07T10:37:55.716362+0000 (oldest deadline 2023-12-07T10:38:18.616525+0000) 2023-12-07T10:38:20.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:19 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:19.871+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:20.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:19 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:19.871+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:20.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:19 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:19.871+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:20.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:19 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:19.871+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.523947+0000 front 2023-12-07T10:37:40.523979+0000 (oldest deadline 2023-12-07T10:38:02.224192+0000) 2023-12-07T10:38:20.228 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:19 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:19.871+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:47.926800+0000 front 2023-12-07T10:37:47.926682+0000 (oldest deadline 2023-12-07T10:38:13.226741+0000) 2023-12-07T10:38:20.228 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:19 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:19.871+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:53.227705+0000 front 2023-12-07T10:37:53.227785+0000 (oldest deadline 2023-12-07T10:38:18.527199+0000) 2023-12-07T10:38:20.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:20 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:20.353+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:20.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:20 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:20.353+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:20.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:20 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:20.353+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:20.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:20 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:20.353+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:20.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:20 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:20.353+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.627146+0000 front 2023-12-07T10:37:48.627171+0000 (oldest deadline 2023-12-07T10:38:13.327348+0000) 2023-12-07T10:38:20.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:20 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:20.353+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.628466+0000 front 2023-12-07T10:37:55.628343+0000 (oldest deadline 2023-12-07T10:38:17.928359+0000) 2023-12-07T10:38:20.792 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:20 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:20.530+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:20.793 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:20 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:20.530+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:20.793 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:20 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:20.530+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:20.793 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:20 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:20.530+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:20.793 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:20 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:20.530+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.716425+0000 front 2023-12-07T10:37:55.716362+0000 (oldest deadline 2023-12-07T10:38:18.616525+0000) 2023-12-07T10:38:21.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:20 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:20.824+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:21.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:20 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:20.824+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:21.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:20 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:20.824+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:21.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:20 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:20.824+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.523947+0000 front 2023-12-07T10:37:40.523979+0000 (oldest deadline 2023-12-07T10:38:02.224192+0000) 2023-12-07T10:38:21.228 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:20 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:20.824+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:47.926800+0000 front 2023-12-07T10:37:47.926682+0000 (oldest deadline 2023-12-07T10:38:13.226741+0000) 2023-12-07T10:38:21.228 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:20 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:20.824+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:53.227705+0000 front 2023-12-07T10:37:53.227785+0000 (oldest deadline 2023-12-07T10:38:18.527199+0000) 2023-12-07T10:38:21.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:20 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:20.792+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.966033+0000 front 2023-12-07T10:37:22.966061+0000 (oldest deadline 2023-12-07T10:37:47.666240+0000) 2023-12-07T10:38:21.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:20 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:20.792+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:31.767435+0000 front 2023-12-07T10:37:31.767410+0000 (oldest deadline 2023-12-07T10:37:55.267563+0000) 2023-12-07T10:38:21.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:20 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:20.792+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:41.069771+0000 front 2023-12-07T10:37:41.069673+0000 (oldest deadline 2023-12-07T10:38:05.769602+0000) 2023-12-07T10:38:21.261 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:20 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:20.792+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:49.270777+0000 front 2023-12-07T10:37:49.270812+0000 (oldest deadline 2023-12-07T10:38:10.370821+0000) 2023-12-07T10:38:21.262 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:20 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8[157402]: 2023-12-07T10:38:20.792+0000 7fdd40d82700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:52.671833+0000 front 2023-12-07T10:37:52.671853+0000 (oldest deadline 2023-12-07T10:38:18.571905+0000) 2023-12-07T10:38:21.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:21 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:21.362+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:21.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:21 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:21.362+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:21.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:21 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:21.362+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:21.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:21 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:21.362+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:21.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:21 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:21.362+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.627146+0000 front 2023-12-07T10:37:48.627171+0000 (oldest deadline 2023-12-07T10:38:13.327348+0000) 2023-12-07T10:38:21.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:21 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:21.362+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.628466+0000 front 2023-12-07T10:37:55.628343+0000 (oldest deadline 2023-12-07T10:38:17.928359+0000) 2023-12-07T10:38:22.011 INFO:journalctl@ceph.osd.8.smithi159.stdout:Dec 07 10:38:21 smithi159 bash[181787]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-8 2023-12-07T10:38:22.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:21 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:21.575+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:22.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:21 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:21.575+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:22.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:21 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:21.575+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:22.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:21 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:21.575+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:22.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:21 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:21.575+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.716425+0000 front 2023-12-07T10:37:55.716362+0000 (oldest deadline 2023-12-07T10:38:18.616525+0000) 2023-12-07T10:38:22.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:21 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:21.829+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:22.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:21 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:21.829+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:22.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:21 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:21.829+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:22.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:21 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:21.829+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.523947+0000 front 2023-12-07T10:37:40.523979+0000 (oldest deadline 2023-12-07T10:38:02.224192+0000) 2023-12-07T10:38:22.228 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:21 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:21.829+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:47.926800+0000 front 2023-12-07T10:37:47.926682+0000 (oldest deadline 2023-12-07T10:38:13.226741+0000) 2023-12-07T10:38:22.228 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:21 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:21.829+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:53.227705+0000 front 2023-12-07T10:37:53.227785+0000 (oldest deadline 2023-12-07T10:38:18.527199+0000) 2023-12-07T10:38:22.762 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:22 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:22.546+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:22.762 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:22 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:22.546+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:22.762 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:22 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:22.546+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:22.763 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:22 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:22.546+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:22.763 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:22 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:22.546+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.716425+0000 front 2023-12-07T10:37:55.716362+0000 (oldest deadline 2023-12-07T10:38:18.616525+0000) 2023-12-07T10:38:22.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:22 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:22.376+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:22.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:22 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:22.376+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:22.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:22 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:22.376+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:22.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:22 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:22.376+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:22.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:22 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:22.376+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.627146+0000 front 2023-12-07T10:37:48.627171+0000 (oldest deadline 2023-12-07T10:38:13.327348+0000) 2023-12-07T10:38:22.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:22 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:22.376+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.628466+0000 front 2023-12-07T10:37:55.628343+0000 (oldest deadline 2023-12-07T10:38:17.928359+0000) 2023-12-07T10:38:23.107 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.8.service' 2023-12-07T10:38:23.147 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:38:23.147 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-12-07T10:38:23.147 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-12-07T10:38:23.147 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.9 2023-12-07T10:38:23.213 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:22 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:22.832+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:23.214 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:22 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:22.832+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:23.214 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:22 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:22.832+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:23.214 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:22 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:22.832+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.523947+0000 front 2023-12-07T10:37:40.523979+0000 (oldest deadline 2023-12-07T10:38:02.224192+0000) 2023-12-07T10:38:23.214 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:22 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:22.832+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:47.926800+0000 front 2023-12-07T10:37:47.926682+0000 (oldest deadline 2023-12-07T10:38:13.226741+0000) 2023-12-07T10:38:23.214 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:22 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:22.832+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:53.227705+0000 front 2023-12-07T10:37:53.227785+0000 (oldest deadline 2023-12-07T10:38:18.527199+0000) 2023-12-07T10:38:23.477 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:23 smithi119 systemd[1]: Stopping Ceph osd.9 for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:38:23.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:23 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:23.363+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:23.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:23 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:23.363+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:23.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:23 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:23.363+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:23.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:23 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:23.363+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:23.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:23 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:23.363+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.627146+0000 front 2023-12-07T10:37:48.627171+0000 (oldest deadline 2023-12-07T10:38:13.327348+0000) 2023-12-07T10:38:23.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:23 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:23.363+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.628466+0000 front 2023-12-07T10:37:55.628343+0000 (oldest deadline 2023-12-07T10:38:17.928359+0000) 2023-12-07T10:38:23.977 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:23 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:23.665+0000 7f6e23f46700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:38:23.977 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:23 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:23.665+0000 7f6e23f46700 -1 osd.9 118 *** Got signal Terminated *** 2023-12-07T10:38:23.977 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:23 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:23.665+0000 7f6e23f46700 -1 osd.9 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-07T10:38:23.977 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:23 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:23.818+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:23.977 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:23 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:23.818+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:23.978 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:23 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:23.818+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:23.978 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:23 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:23.818+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.523947+0000 front 2023-12-07T10:37:40.523979+0000 (oldest deadline 2023-12-07T10:38:02.224192+0000) 2023-12-07T10:38:23.978 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:23 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:23.818+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:47.926800+0000 front 2023-12-07T10:37:47.926682+0000 (oldest deadline 2023-12-07T10:38:13.226741+0000) 2023-12-07T10:38:23.978 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:23 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:23.818+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:53.227705+0000 front 2023-12-07T10:37:53.227785+0000 (oldest deadline 2023-12-07T10:38:18.527199+0000) 2023-12-07T10:38:24.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:23 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:23.587+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:24.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:23 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:23.587+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:24.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:23 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:23.587+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:24.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:23 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:23.587+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:24.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:23 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:23.587+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.716425+0000 front 2023-12-07T10:37:55.716362+0000 (oldest deadline 2023-12-07T10:38:18.616525+0000) 2023-12-07T10:38:24.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:24 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:24.338+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:24.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:24 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:24.338+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:24.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:24 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:24.338+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:24.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:24 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:24.338+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:24.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:24 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:24.338+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.627146+0000 front 2023-12-07T10:37:48.627171+0000 (oldest deadline 2023-12-07T10:38:13.327348+0000) 2023-12-07T10:38:24.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:24 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:24.338+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.628466+0000 front 2023-12-07T10:37:55.628343+0000 (oldest deadline 2023-12-07T10:38:17.928359+0000) 2023-12-07T10:38:25.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:24 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:24.626+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:25.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:24 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:24.626+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:25.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:24 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:24.626+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:25.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:24 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:24.626+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:25.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:24 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:24.626+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.716425+0000 front 2023-12-07T10:37:55.716362+0000 (oldest deadline 2023-12-07T10:38:18.616525+0000) 2023-12-07T10:38:25.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:24 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:24.834+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:25.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:24 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:24.834+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:25.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:24 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:24.834+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:25.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:24 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:24.834+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.523947+0000 front 2023-12-07T10:37:40.523979+0000 (oldest deadline 2023-12-07T10:38:02.224192+0000) 2023-12-07T10:38:25.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:24 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:24.834+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:47.926800+0000 front 2023-12-07T10:37:47.926682+0000 (oldest deadline 2023-12-07T10:38:13.226741+0000) 2023-12-07T10:38:25.228 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:24 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:24.834+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:53.227705+0000 front 2023-12-07T10:37:53.227785+0000 (oldest deadline 2023-12-07T10:38:18.527199+0000) 2023-12-07T10:38:25.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:25 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:25.350+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:25.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:25 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:25.350+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:25.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:25 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:25.350+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:25.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:25 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:25.350+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:25.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:25 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:25.350+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.627146+0000 front 2023-12-07T10:37:48.627171+0000 (oldest deadline 2023-12-07T10:38:13.327348+0000) 2023-12-07T10:38:25.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:25 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:25.350+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.628466+0000 front 2023-12-07T10:37:55.628343+0000 (oldest deadline 2023-12-07T10:38:17.928359+0000) 2023-12-07T10:38:26.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:25 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:25.659+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:26.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:25 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:25.659+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:26.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:25 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:25.659+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:26.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:25 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:25.659+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:26.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:25 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:25.659+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.716425+0000 front 2023-12-07T10:37:55.716362+0000 (oldest deadline 2023-12-07T10:38:18.616525+0000) 2023-12-07T10:38:26.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:25 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:25.822+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:26.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:25 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:25.822+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:26.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:25 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:25.822+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:26.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:25 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:25.822+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.523947+0000 front 2023-12-07T10:37:40.523979+0000 (oldest deadline 2023-12-07T10:38:02.224192+0000) 2023-12-07T10:38:26.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:25 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:25.822+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:47.926800+0000 front 2023-12-07T10:37:47.926682+0000 (oldest deadline 2023-12-07T10:38:13.226741+0000) 2023-12-07T10:38:26.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:25 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:25.822+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:53.227705+0000 front 2023-12-07T10:37:53.227785+0000 (oldest deadline 2023-12-07T10:38:18.527199+0000) 2023-12-07T10:38:26.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:26 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:26.388+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:26.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:26 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:26.388+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:26.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:26 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:26.388+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:26.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:26 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:26.388+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:26.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:26 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:26.388+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.627146+0000 front 2023-12-07T10:37:48.627171+0000 (oldest deadline 2023-12-07T10:38:13.327348+0000) 2023-12-07T10:38:26.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:26 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:26.388+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.628466+0000 front 2023-12-07T10:37:55.628343+0000 (oldest deadline 2023-12-07T10:38:17.928359+0000) 2023-12-07T10:38:26.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:26 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:26.388+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2023-12-07T10:38:03.129973+0000 front 2023-12-07T10:38:03.130042+0000 (oldest deadline 2023-12-07T10:38:25.930273+0000) 2023-12-07T10:38:27.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:26 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:26.617+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:27.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:26 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:26.617+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:27.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:26 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:26.617+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:27.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:26 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:26.617+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:27.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:26 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:26.617+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.716425+0000 front 2023-12-07T10:37:55.716362+0000 (oldest deadline 2023-12-07T10:38:18.616525+0000) 2023-12-07T10:38:27.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:26 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:26.838+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:27.228 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:26 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:26.838+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:27.228 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:26 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:26.838+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:27.228 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:26 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:26.838+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.523947+0000 front 2023-12-07T10:37:40.523979+0000 (oldest deadline 2023-12-07T10:38:02.224192+0000) 2023-12-07T10:38:27.228 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:26 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:26.838+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:47.926800+0000 front 2023-12-07T10:37:47.926682+0000 (oldest deadline 2023-12-07T10:38:13.226741+0000) 2023-12-07T10:38:27.228 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:26 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:26.838+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:53.227705+0000 front 2023-12-07T10:37:53.227785+0000 (oldest deadline 2023-12-07T10:38:18.527199+0000) 2023-12-07T10:38:27.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:27 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:27.421+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:27.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:27 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:27.421+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:27.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:27 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:27.421+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:27.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:27 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:27.421+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:27.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:27 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:27.421+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.627146+0000 front 2023-12-07T10:37:48.627171+0000 (oldest deadline 2023-12-07T10:38:13.327348+0000) 2023-12-07T10:38:27.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:27 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:27.421+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.628466+0000 front 2023-12-07T10:37:55.628343+0000 (oldest deadline 2023-12-07T10:38:17.928359+0000) 2023-12-07T10:38:27.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:27 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:27.421+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2023-12-07T10:38:03.129973+0000 front 2023-12-07T10:38:03.130042+0000 (oldest deadline 2023-12-07T10:38:25.930273+0000) 2023-12-07T10:38:28.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:27 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:27.594+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:28.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:27 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:27.594+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:28.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:27 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:27.594+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:28.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:27 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:27.594+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:28.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:27 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:27.594+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.716425+0000 front 2023-12-07T10:37:55.716362+0000 (oldest deadline 2023-12-07T10:38:18.616525+0000) 2023-12-07T10:38:28.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:27 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:27.841+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:15.617583+0000 front 2023-12-07T10:37:15.617742+0000 (oldest deadline 2023-12-07T10:37:42.618094+0000) 2023-12-07T10:38:28.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:27 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:27.841+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:22.618675+0000 front 2023-12-07T10:37:22.618783+0000 (oldest deadline 2023-12-07T10:37:48.518603+0000) 2023-12-07T10:38:28.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:27 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:27.841+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:32.019904+0000 front 2023-12-07T10:37:32.020066+0000 (oldest deadline 2023-12-07T10:37:54.319951+0000) 2023-12-07T10:38:28.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:27 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:27.841+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:40.523947+0000 front 2023-12-07T10:37:40.523979+0000 (oldest deadline 2023-12-07T10:38:02.224192+0000) 2023-12-07T10:38:28.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:27 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:27.841+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:47.926800+0000 front 2023-12-07T10:37:47.926682+0000 (oldest deadline 2023-12-07T10:38:13.226741+0000) 2023-12-07T10:38:28.227 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:27 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9[138937]: 2023-12-07T10:38:27.841+0000 7f6e1f53f700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:53.227705+0000 front 2023-12-07T10:37:53.227785+0000 (oldest deadline 2023-12-07T10:38:18.527199+0000) 2023-12-07T10:38:28.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:28 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:28.469+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:28.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:28 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:28.469+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:28.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:28 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:28.469+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:28.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:28 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:28.469+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:28.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:28 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:28.469+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.627146+0000 front 2023-12-07T10:37:48.627171+0000 (oldest deadline 2023-12-07T10:38:13.327348+0000) 2023-12-07T10:38:28.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:28 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:28.469+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.628466+0000 front 2023-12-07T10:37:55.628343+0000 (oldest deadline 2023-12-07T10:38:17.928359+0000) 2023-12-07T10:38:28.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:28 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:28.469+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2023-12-07T10:38:03.129973+0000 front 2023-12-07T10:38:03.130042+0000 (oldest deadline 2023-12-07T10:38:25.930273+0000) 2023-12-07T10:38:29.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:28 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:28.587+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:29.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:28 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:28.587+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:29.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:28 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:28.587+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:29.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:28 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:28.587+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:29.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:28 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:28.587+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.716425+0000 front 2023-12-07T10:37:55.716362+0000 (oldest deadline 2023-12-07T10:38:18.616525+0000) 2023-12-07T10:38:29.727 INFO:journalctl@ceph.osd.9.smithi119.stdout:Dec 07 10:38:29 smithi119 bash[181781]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-9 2023-12-07T10:38:29.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:29 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:29.430+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:29.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:29 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:29.430+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:29.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:29 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:29.430+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:29.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:29 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:29.430+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:29.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:29 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:29.430+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.627146+0000 front 2023-12-07T10:37:48.627171+0000 (oldest deadline 2023-12-07T10:38:13.327348+0000) 2023-12-07T10:38:29.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:29 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:29.430+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.628466+0000 front 2023-12-07T10:37:55.628343+0000 (oldest deadline 2023-12-07T10:38:17.928359+0000) 2023-12-07T10:38:29.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:29 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:29.430+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2023-12-07T10:38:03.129973+0000 front 2023-12-07T10:38:03.130042+0000 (oldest deadline 2023-12-07T10:38:25.930273+0000) 2023-12-07T10:38:30.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:29 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:29.618+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:30.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:29 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:29.618+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:30.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:29 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:29.618+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:30.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:29 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:29.618+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:30.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:29 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:29.618+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.716425+0000 front 2023-12-07T10:37:55.716362+0000 (oldest deadline 2023-12-07T10:38:18.616525+0000) 2023-12-07T10:38:30.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:30 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:30.471+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:30.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:30 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:30.471+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:30.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:30 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:30.471+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:30.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:30 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:30.471+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:30.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:30 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:30.471+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.627146+0000 front 2023-12-07T10:37:48.627171+0000 (oldest deadline 2023-12-07T10:38:13.327348+0000) 2023-12-07T10:38:30.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:30 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:30.471+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.628466+0000 front 2023-12-07T10:37:55.628343+0000 (oldest deadline 2023-12-07T10:38:17.928359+0000) 2023-12-07T10:38:30.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:30 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:30.471+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2023-12-07T10:38:03.129973+0000 front 2023-12-07T10:38:03.130042+0000 (oldest deadline 2023-12-07T10:38:25.930273+0000) 2023-12-07T10:38:31.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:30 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:30.614+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:31.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:30 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:30.614+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:31.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:30 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:30.614+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:31.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:30 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:30.614+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:31.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:30 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:30.614+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.716425+0000 front 2023-12-07T10:37:55.716362+0000 (oldest deadline 2023-12-07T10:38:18.616525+0000) 2023-12-07T10:38:31.474 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.9.service' 2023-12-07T10:38:31.562 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:38:31.562 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-12-07T10:38:31.562 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-12-07T10:38:31.563 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.10 2023-12-07T10:38:31.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:31 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:31.460+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:31.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:31 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:31.460+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:31.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:31 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:31.460+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:31.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:31 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:31.460+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:31.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:31 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:31.460+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.627146+0000 front 2023-12-07T10:37:48.627171+0000 (oldest deadline 2023-12-07T10:38:13.327348+0000) 2023-12-07T10:38:31.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:31 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:31.460+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.628466+0000 front 2023-12-07T10:37:55.628343+0000 (oldest deadline 2023-12-07T10:38:17.928359+0000) 2023-12-07T10:38:31.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:31 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:31.460+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2023-12-07T10:38:03.129973+0000 front 2023-12-07T10:38:03.130042+0000 (oldest deadline 2023-12-07T10:38:25.930273+0000) 2023-12-07T10:38:31.780 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:31 smithi153 systemd[1]: Stopping Ceph osd.10 for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:38:32.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:31 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:31.594+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:32.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:31 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:31.594+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:32.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:31 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:31.594+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:32.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:31 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:31.594+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:32.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:31 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:31.594+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.716425+0000 front 2023-12-07T10:37:55.716362+0000 (oldest deadline 2023-12-07T10:38:18.616525+0000) 2023-12-07T10:38:32.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:31 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:31.594+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2023-12-07T10:38:05.018031+0000 front 2023-12-07T10:38:05.018089+0000 (oldest deadline 2023-12-07T10:38:30.918091+0000) 2023-12-07T10:38:32.277 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:31 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:31.912+0000 7fb8f8f59700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:38:32.278 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:31 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:31.912+0000 7fb8f8f59700 -1 osd.10 118 *** Got signal Terminated *** 2023-12-07T10:38:32.278 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:31 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:31.912+0000 7fb8f8f59700 -1 osd.10 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-07T10:38:32.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:32 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:32.458+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:32.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:32 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:32.458+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:32.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:32 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:32.458+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:32.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:32 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:32.458+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:32.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:32 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:32.458+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.627146+0000 front 2023-12-07T10:37:48.627171+0000 (oldest deadline 2023-12-07T10:38:13.327348+0000) 2023-12-07T10:38:32.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:32 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:32.458+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.628466+0000 front 2023-12-07T10:37:55.628343+0000 (oldest deadline 2023-12-07T10:38:17.928359+0000) 2023-12-07T10:38:32.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:32 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:32.458+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2023-12-07T10:38:03.129973+0000 front 2023-12-07T10:38:03.130042+0000 (oldest deadline 2023-12-07T10:38:25.930273+0000) 2023-12-07T10:38:33.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:32 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:32.602+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:33.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:32 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:32.602+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:33.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:32 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:32.602+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:33.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:32 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:32.602+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:33.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:32 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:32.602+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.716425+0000 front 2023-12-07T10:37:55.716362+0000 (oldest deadline 2023-12-07T10:38:18.616525+0000) 2023-12-07T10:38:33.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:32 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:32.602+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2023-12-07T10:38:05.018031+0000 front 2023-12-07T10:38:05.018089+0000 (oldest deadline 2023-12-07T10:38:30.918091+0000) 2023-12-07T10:38:33.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:33 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:33.460+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:33.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:33 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:33.460+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:33.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:33 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:33.460+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:33.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:33 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:33.460+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:33.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:33 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:33.460+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.627146+0000 front 2023-12-07T10:37:48.627171+0000 (oldest deadline 2023-12-07T10:38:13.327348+0000) 2023-12-07T10:38:33.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:33 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:33.460+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.628466+0000 front 2023-12-07T10:37:55.628343+0000 (oldest deadline 2023-12-07T10:38:17.928359+0000) 2023-12-07T10:38:33.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:33 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:33.460+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2023-12-07T10:38:03.129973+0000 front 2023-12-07T10:38:03.130042+0000 (oldest deadline 2023-12-07T10:38:25.930273+0000) 2023-12-07T10:38:34.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:33 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:33.572+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:34.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:33 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:33.572+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:34.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:33 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:33.572+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:34.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:33 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:33.572+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:34.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:33 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:33.572+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.716425+0000 front 2023-12-07T10:37:55.716362+0000 (oldest deadline 2023-12-07T10:38:18.616525+0000) 2023-12-07T10:38:34.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:33 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:33.572+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2023-12-07T10:38:05.018031+0000 front 2023-12-07T10:38:05.018089+0000 (oldest deadline 2023-12-07T10:38:30.918091+0000) 2023-12-07T10:38:34.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:34 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:34.488+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:34.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:34 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:34.488+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:34.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:34 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:34.488+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:34.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:34 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:34.488+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:34.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:34 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:34.488+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.627146+0000 front 2023-12-07T10:37:48.627171+0000 (oldest deadline 2023-12-07T10:38:13.327348+0000) 2023-12-07T10:38:34.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:34 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:34.488+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.628466+0000 front 2023-12-07T10:37:55.628343+0000 (oldest deadline 2023-12-07T10:38:17.928359+0000) 2023-12-07T10:38:34.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:34 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:34.488+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2023-12-07T10:38:03.129973+0000 front 2023-12-07T10:38:03.130042+0000 (oldest deadline 2023-12-07T10:38:25.930273+0000) 2023-12-07T10:38:35.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:34 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:34.548+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:35.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:34 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:34.548+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:35.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:34 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:34.548+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:35.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:34 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:34.548+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:35.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:34 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:34.548+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.716425+0000 front 2023-12-07T10:37:55.716362+0000 (oldest deadline 2023-12-07T10:38:18.616525+0000) 2023-12-07T10:38:35.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:34 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:34.548+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2023-12-07T10:38:05.018031+0000 front 2023-12-07T10:38:05.018089+0000 (oldest deadline 2023-12-07T10:38:30.918091+0000) 2023-12-07T10:38:35.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:35 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:35.473+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:35.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:35 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:35.473+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:35.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:35 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:35.473+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:35.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:35 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:35.473+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:35.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:35 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:35.473+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.627146+0000 front 2023-12-07T10:37:48.627171+0000 (oldest deadline 2023-12-07T10:38:13.327348+0000) 2023-12-07T10:38:35.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:35 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:35.473+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.628466+0000 front 2023-12-07T10:37:55.628343+0000 (oldest deadline 2023-12-07T10:38:17.928359+0000) 2023-12-07T10:38:35.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:35 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:35.473+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2023-12-07T10:38:03.129973+0000 front 2023-12-07T10:38:03.130042+0000 (oldest deadline 2023-12-07T10:38:25.930273+0000) 2023-12-07T10:38:36.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:35 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:35.521+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:36.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:35 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:35.521+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:36.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:35 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:35.521+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:36.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:35 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:35.521+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:36.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:35 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:35.521+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.716425+0000 front 2023-12-07T10:37:55.716362+0000 (oldest deadline 2023-12-07T10:38:18.616525+0000) 2023-12-07T10:38:36.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:35 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:35.521+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2023-12-07T10:38:05.018031+0000 front 2023-12-07T10:38:05.018089+0000 (oldest deadline 2023-12-07T10:38:30.918091+0000) 2023-12-07T10:38:36.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:36 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:36.483+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:36.762 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:36 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:36.483+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:36.762 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:36 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:36.483+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:36.762 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:36 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:36.483+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:36.762 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:36 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:36.483+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.716425+0000 front 2023-12-07T10:37:55.716362+0000 (oldest deadline 2023-12-07T10:38:18.616525+0000) 2023-12-07T10:38:36.762 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:36 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:36.483+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2023-12-07T10:38:05.018031+0000 front 2023-12-07T10:38:05.018089+0000 (oldest deadline 2023-12-07T10:38:30.918091+0000) 2023-12-07T10:38:36.762 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:36 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:36.483+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6820 osd.7 since back 2023-12-07T10:38:10.918723+0000 front 2023-12-07T10:38:10.918703+0000 (oldest deadline 2023-12-07T10:38:35.618682+0000) 2023-12-07T10:38:36.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:36 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:36.426+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:16.522244+0000 front 2023-12-07T10:37:16.522288+0000 (oldest deadline 2023-12-07T10:37:41.221941+0000) 2023-12-07T10:38:36.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:36 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:36.426+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:21.222310+0000 front 2023-12-07T10:37:21.222268+0000 (oldest deadline 2023-12-07T10:37:47.122566+0000) 2023-12-07T10:38:36.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:36 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:36.426+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:29.423586+0000 front 2023-12-07T10:37:29.423565+0000 (oldest deadline 2023-12-07T10:37:58.824404+0000) 2023-12-07T10:38:36.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:36 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:36.426+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6814 osd.3 since back 2023-12-07T10:37:38.824952+0000 front 2023-12-07T10:37:38.824982+0000 (oldest deadline 2023-12-07T10:38:02.325077+0000) 2023-12-07T10:38:36.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:36 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:36.426+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.627146+0000 front 2023-12-07T10:37:48.627171+0000 (oldest deadline 2023-12-07T10:38:13.327348+0000) 2023-12-07T10:38:36.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:36 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:36.426+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.628466+0000 front 2023-12-07T10:37:55.628343+0000 (oldest deadline 2023-12-07T10:38:17.928359+0000) 2023-12-07T10:38:36.779 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:36 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10[164054]: 2023-12-07T10:38:36.426+0000 7fb8f2d4f700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2023-12-07T10:38:03.129973+0000 front 2023-12-07T10:38:03.130042+0000 (oldest deadline 2023-12-07T10:38:25.930273+0000) 2023-12-07T10:38:37.778 INFO:journalctl@ceph.osd.10.smithi153.stdout:Dec 07 10:38:37 smithi153 bash[180054]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-10 2023-12-07T10:38:38.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:37 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:37.516+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:38.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:37 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:37.516+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:38.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:37 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:37.516+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:38.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:37 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:37.516+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:38.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:37 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:37.516+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.716425+0000 front 2023-12-07T10:37:55.716362+0000 (oldest deadline 2023-12-07T10:38:18.616525+0000) 2023-12-07T10:38:38.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:37 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:37.516+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2023-12-07T10:38:05.018031+0000 front 2023-12-07T10:38:05.018089+0000 (oldest deadline 2023-12-07T10:38:30.918091+0000) 2023-12-07T10:38:38.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:37 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:37.516+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6820 osd.7 since back 2023-12-07T10:38:10.918723+0000 front 2023-12-07T10:38:10.918703+0000 (oldest deadline 2023-12-07T10:38:35.618682+0000) 2023-12-07T10:38:39.004 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.10.service' 2023-12-07T10:38:39.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:38 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:38.524+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:39.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:38 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:38.524+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:39.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:38 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:38.524+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:39.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:38 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:38.524+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:39.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:38 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:38.524+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.716425+0000 front 2023-12-07T10:37:55.716362+0000 (oldest deadline 2023-12-07T10:38:18.616525+0000) 2023-12-07T10:38:39.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:38 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:38.524+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2023-12-07T10:38:05.018031+0000 front 2023-12-07T10:38:05.018089+0000 (oldest deadline 2023-12-07T10:38:30.918091+0000) 2023-12-07T10:38:39.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:38 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:38.524+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6820 osd.7 since back 2023-12-07T10:38:10.918723+0000 front 2023-12-07T10:38:10.918703+0000 (oldest deadline 2023-12-07T10:38:35.618682+0000) 2023-12-07T10:38:39.044 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:38:39.044 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-12-07T10:38:39.044 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-12-07T10:38:39.044 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.11 2023-12-07T10:38:39.404 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:39 smithi159 systemd[1]: Stopping Ceph osd.11 for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:38:39.404 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:39 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:39.404+0000 7f345f9bc700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:38:39.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:39 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:39.404+0000 7f345f9bc700 -1 osd.11 118 *** Got signal Terminated *** 2023-12-07T10:38:39.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:39 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:39.404+0000 7f345f9bc700 -1 osd.11 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-07T10:38:39.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:39 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:39.542+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:39.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:39 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:39.542+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:39.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:39 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:39.542+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:39.762 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:39 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:39.542+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:39.762 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:39 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:39.542+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.716425+0000 front 2023-12-07T10:37:55.716362+0000 (oldest deadline 2023-12-07T10:38:18.616525+0000) 2023-12-07T10:38:39.762 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:39 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:39.542+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2023-12-07T10:38:05.018031+0000 front 2023-12-07T10:38:05.018089+0000 (oldest deadline 2023-12-07T10:38:30.918091+0000) 2023-12-07T10:38:39.762 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:39 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:39.542+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6820 osd.7 since back 2023-12-07T10:38:10.918723+0000 front 2023-12-07T10:38:10.918703+0000 (oldest deadline 2023-12-07T10:38:35.618682+0000) 2023-12-07T10:38:41.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:40 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:40.583+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:41.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:40 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:40.583+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:41.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:40 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:40.583+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:41.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:40 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:40.583+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:41.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:40 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:40.583+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.716425+0000 front 2023-12-07T10:37:55.716362+0000 (oldest deadline 2023-12-07T10:38:18.616525+0000) 2023-12-07T10:38:41.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:40 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:40.583+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2023-12-07T10:38:05.018031+0000 front 2023-12-07T10:38:05.018089+0000 (oldest deadline 2023-12-07T10:38:30.918091+0000) 2023-12-07T10:38:41.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:40 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:40.583+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6820 osd.7 since back 2023-12-07T10:38:10.918723+0000 front 2023-12-07T10:38:10.918703+0000 (oldest deadline 2023-12-07T10:38:35.618682+0000) 2023-12-07T10:38:42.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:41 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:41.550+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:42.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:41 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:41.550+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:42.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:41 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:41.550+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:42.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:41 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:41.550+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:42.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:41 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:41.550+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.716425+0000 front 2023-12-07T10:37:55.716362+0000 (oldest deadline 2023-12-07T10:38:18.616525+0000) 2023-12-07T10:38:42.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:41 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:41.550+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2023-12-07T10:38:05.018031+0000 front 2023-12-07T10:38:05.018089+0000 (oldest deadline 2023-12-07T10:38:30.918091+0000) 2023-12-07T10:38:42.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:41 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:41.550+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6820 osd.7 since back 2023-12-07T10:38:10.918723+0000 front 2023-12-07T10:38:10.918703+0000 (oldest deadline 2023-12-07T10:38:35.618682+0000) 2023-12-07T10:38:43.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:42 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:42.566+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:43.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:42 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:42.566+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:43.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:42 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:42.566+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:43.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:42 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:42.566+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:43.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:42 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:42.566+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.716425+0000 front 2023-12-07T10:37:55.716362+0000 (oldest deadline 2023-12-07T10:38:18.616525+0000) 2023-12-07T10:38:43.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:42 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:42.566+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2023-12-07T10:38:05.018031+0000 front 2023-12-07T10:38:05.018089+0000 (oldest deadline 2023-12-07T10:38:30.918091+0000) 2023-12-07T10:38:43.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:42 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:42.566+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6820 osd.7 since back 2023-12-07T10:38:10.918723+0000 front 2023-12-07T10:38:10.918703+0000 (oldest deadline 2023-12-07T10:38:35.618682+0000) 2023-12-07T10:38:44.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:43 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:43.573+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2023-12-07T10:37:11.508146+0000 front 2023-12-07T10:37:11.508132+0000 (oldest deadline 2023-12-07T10:37:43.308720+0000) 2023-12-07T10:38:44.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:43 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:43.573+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6804 osd.1 since back 2023-12-07T10:37:25.610049+0000 front 2023-12-07T10:37:25.609915+0000 (oldest deadline 2023-12-07T10:37:49.709957+0000) 2023-12-07T10:38:44.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:43 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:43.573+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2023-12-07T10:37:30.210858+0000 front 2023-12-07T10:37:30.210824+0000 (oldest deadline 2023-12-07T10:37:55.510940+0000) 2023-12-07T10:38:44.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:43 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:43.573+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6812 osd.4 since back 2023-12-07T10:37:48.214931+0000 front 2023-12-07T10:37:48.215106+0000 (oldest deadline 2023-12-07T10:38:12.215486+0000) 2023-12-07T10:38:44.011 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:43 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:43.573+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2023-12-07T10:37:55.716425+0000 front 2023-12-07T10:37:55.716362+0000 (oldest deadline 2023-12-07T10:38:18.616525+0000) 2023-12-07T10:38:44.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:43 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:43.573+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.119:6822 osd.6 since back 2023-12-07T10:38:05.018031+0000 front 2023-12-07T10:38:05.018089+0000 (oldest deadline 2023-12-07T10:38:30.918091+0000) 2023-12-07T10:38:44.012 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:43 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11[166069]: 2023-12-07T10:38:43.573+0000 7f3458fb1700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.153:6820 osd.7 since back 2023-12-07T10:38:10.918723+0000 front 2023-12-07T10:38:10.918703+0000 (oldest deadline 2023-12-07T10:38:35.618682+0000) 2023-12-07T10:38:45.511 INFO:journalctl@ceph.osd.11.smithi159.stdout:Dec 07 10:38:45 smithi159 bash[182064]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-osd-11 2023-12-07T10:38:46.384 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@osd.11.service' 2023-12-07T10:38:46.424 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:38:46.425 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-12-07T10:38:46.425 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-12-07T10:38:46.425 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.a 2023-12-07T10:38:46.945 INFO:journalctl@ceph.mds.a.smithi119.stdout:Dec 07 10:38:46 smithi119 systemd[1]: Stopping Ceph mds.a for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:38:47.226 INFO:journalctl@ceph.mds.a.smithi119.stdout:Dec 07 10:38:46 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-a[146736]: 2023-12-07T10:38:46.945+0000 7f21b96ab700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:38:47.227 INFO:journalctl@ceph.mds.a.smithi119.stdout:Dec 07 10:38:46 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-a[146736]: 2023-12-07T10:38:46.945+0000 7f21b96ab700 -1 mds.a *** got signal Terminated *** 2023-12-07T10:38:51.700 INFO:journalctl@ceph.mds.a.smithi119.stdout:Dec 07 10:38:51 smithi119 bash[182060]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-a 2023-12-07T10:38:51.734 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.a.service' 2023-12-07T10:38:51.818 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:38:51.819 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-12-07T10:38:51.819 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-12-07T10:38:51.819 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.d 2023-12-07T10:38:51.977 INFO:journalctl@ceph.mds.d.smithi119.stdout:Dec 07 10:38:51 smithi119 systemd[1]: Stopping Ceph mds.d for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:38:52.727 INFO:journalctl@ceph.mds.d.smithi119.stdout:Dec 07 10:38:52 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-d[145734]: 2023-12-07T10:38:52.327+0000 7f10cfbfa700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:38:52.727 INFO:journalctl@ceph.mds.d.smithi119.stdout:Dec 07 10:38:52 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-d[145734]: 2023-12-07T10:38:52.327+0000 7f10cfbfa700 -1 mds.d *** got signal Terminated *** 2023-12-07T10:38:53.976 INFO:journalctl@ceph.mds.d.smithi119.stdout:Dec 07 10:38:53 smithi119 bash[182198]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-d 2023-12-07T10:38:54.164 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.d.service' 2023-12-07T10:38:54.245 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:38:54.245 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-12-07T10:38:54.246 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-12-07T10:38:54.246 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.g 2023-12-07T10:38:54.477 INFO:journalctl@ceph.mds.g.smithi119.stdout:Dec 07 10:38:54 smithi119 systemd[1]: Stopping Ceph mds.g for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:38:55.227 INFO:journalctl@ceph.mds.g.smithi119.stdout:Dec 07 10:38:54 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-g[146351]: 2023-12-07T10:38:54.799+0000 7f12e28a9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:38:55.227 INFO:journalctl@ceph.mds.g.smithi119.stdout:Dec 07 10:38:54 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-g[146351]: 2023-12-07T10:38:54.799+0000 7f12e28a9700 -1 mds.g *** got signal Terminated *** 2023-12-07T10:38:56.476 INFO:journalctl@ceph.mds.g.smithi119.stdout:Dec 07 10:38:56 smithi119 bash[182346]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-g 2023-12-07T10:38:56.674 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.g.service' 2023-12-07T10:38:56.755 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:38:56.755 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-12-07T10:38:56.755 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-12-07T10:38:56.756 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.j 2023-12-07T10:38:56.977 INFO:journalctl@ceph.mds.j.smithi119.stdout:Dec 07 10:38:56 smithi119 systemd[1]: Stopping Ceph mds.j for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:38:57.727 INFO:journalctl@ceph.mds.j.smithi119.stdout:Dec 07 10:38:57 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-j[145350]: 2023-12-07T10:38:57.259+0000 7f91da928700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:38:57.727 INFO:journalctl@ceph.mds.j.smithi119.stdout:Dec 07 10:38:57 smithi119 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-j[145350]: 2023-12-07T10:38:57.259+0000 7f91da928700 -1 mds.j *** got signal Terminated *** 2023-12-07T10:38:58.722 INFO:journalctl@ceph.mds.j.smithi119.stdout:Dec 07 10:38:58 smithi119 bash[182490]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-j 2023-12-07T10:38:58.944 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.j.service' 2023-12-07T10:38:58.976 INFO:journalctl@ceph.mds.j.smithi119.stdout:Dec 07 10:38:58 smithi119 systemd[1]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.j.service: Succeeded. 2023-12-07T10:38:58.977 INFO:journalctl@ceph.mds.j.smithi119.stdout:Dec 07 10:38:58 smithi119 systemd[1]: Stopped Ceph mds.j for a4c642c6-94e8-11ee-95a2-87774f69a715. 2023-12-07T10:38:59.026 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:38:59.026 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-12-07T10:38:59.026 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-12-07T10:38:59.027 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.b 2023-12-07T10:38:59.383 INFO:journalctl@ceph.mds.b.smithi153.stdout:Dec 07 10:38:59 smithi153 systemd[1]: Stopping Ceph mds.b for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:38:59.777 INFO:journalctl@ceph.mds.b.smithi153.stdout:Dec 07 10:38:59 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-b[170442]: 2023-12-07T10:38:59.382+0000 7ffb99da6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:38:59.778 INFO:journalctl@ceph.mds.b.smithi153.stdout:Dec 07 10:38:59 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-b[170442]: 2023-12-07T10:38:59.382+0000 7ffb99da6700 -1 mds.b *** got signal Terminated *** 2023-12-07T10:39:03.754 INFO:journalctl@ceph.mds.b.smithi153.stdout:Dec 07 10:39:03 smithi153 bash[180322]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-b 2023-12-07T10:39:03.943 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.b.service' 2023-12-07T10:39:03.979 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:39:03.979 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-12-07T10:39:03.979 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-12-07T10:39:03.979 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.e 2023-12-07T10:39:04.278 INFO:journalctl@ceph.mds.e.smithi153.stdout:Dec 07 10:39:04 smithi153 systemd[1]: Stopping Ceph mds.e for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:39:04.777 INFO:journalctl@ceph.mds.e.smithi153.stdout:Dec 07 10:39:04 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-e[169759]: 2023-12-07T10:39:04.355+0000 7f8d366a3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:39:04.778 INFO:journalctl@ceph.mds.e.smithi153.stdout:Dec 07 10:39:04 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-e[169759]: 2023-12-07T10:39:04.355+0000 7f8d366a3700 -1 mds.e *** got signal Terminated *** 2023-12-07T10:39:08.127 INFO:journalctl@ceph.mds.e.smithi153.stdout:Dec 07 10:39:08 smithi153 bash[180466]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-e 2023-12-07T10:39:08.313 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.e.service' 2023-12-07T10:39:08.348 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:39:08.348 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-12-07T10:39:08.348 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-12-07T10:39:08.348 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.h 2023-12-07T10:39:08.732 INFO:journalctl@ceph.mds.h.smithi153.stdout:Dec 07 10:39:08 smithi153 systemd[1]: Stopping Ceph mds.h for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:39:09.027 INFO:journalctl@ceph.mds.h.smithi153.stdout:Dec 07 10:39:08 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-h[170103]: 2023-12-07T10:39:08.731+0000 7f9ba81e7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:39:09.028 INFO:journalctl@ceph.mds.h.smithi153.stdout:Dec 07 10:39:08 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-h[170103]: 2023-12-07T10:39:08.731+0000 7f9ba81e7700 -1 mds.h *** got signal Terminated *** 2023-12-07T10:39:12.277 INFO:journalctl@ceph.mds.h.smithi153.stdout:Dec 07 10:39:12 smithi153 bash[180603]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-h 2023-12-07T10:39:12.398 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.h.service' 2023-12-07T10:39:12.431 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:39:12.431 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-12-07T10:39:12.432 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-12-07T10:39:12.432 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.k 2023-12-07T10:39:12.777 INFO:journalctl@ceph.mds.k.smithi153.stdout:Dec 07 10:39:12 smithi153 systemd[1]: Stopping Ceph mds.k for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:39:13.277 INFO:journalctl@ceph.mds.k.smithi153.stdout:Dec 07 10:39:12 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-k[169421]: 2023-12-07T10:39:12.810+0000 7fd4212d2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:39:13.278 INFO:journalctl@ceph.mds.k.smithi153.stdout:Dec 07 10:39:12 smithi153 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-k[169421]: 2023-12-07T10:39:12.810+0000 7fd4212d2700 -1 mds.k *** got signal Terminated *** 2023-12-07T10:39:16.177 INFO:journalctl@ceph.mds.k.smithi153.stdout:Dec 07 10:39:16 smithi153 bash[180750]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-k 2023-12-07T10:39:16.378 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.k.service' 2023-12-07T10:39:16.412 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:39:16.413 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-12-07T10:39:16.413 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-12-07T10:39:16.413 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.c 2023-12-07T10:39:16.761 INFO:journalctl@ceph.mds.c.smithi159.stdout:Dec 07 10:39:16 smithi159 systemd[1]: Stopping Ceph mds.c for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:39:17.261 INFO:journalctl@ceph.mds.c.smithi159.stdout:Dec 07 10:39:16 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-c[172250]: 2023-12-07T10:39:16.823+0000 7f393a894700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:39:17.261 INFO:journalctl@ceph.mds.c.smithi159.stdout:Dec 07 10:39:16 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-c[172250]: 2023-12-07T10:39:16.823+0000 7f393a894700 -1 mds.c *** got signal Terminated *** 2023-12-07T10:39:20.140 INFO:journalctl@ceph.mds.c.smithi159.stdout:Dec 07 10:39:20 smithi159 bash[182335]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-c 2023-12-07T10:39:20.324 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.c.service' 2023-12-07T10:39:20.359 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:39:20.359 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-12-07T10:39:20.360 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-12-07T10:39:20.360 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.f 2023-12-07T10:39:20.739 INFO:journalctl@ceph.mds.f.smithi159.stdout:Dec 07 10:39:20 smithi159 systemd[1]: Stopping Ceph mds.f for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:39:21.011 INFO:journalctl@ceph.mds.f.smithi159.stdout:Dec 07 10:39:20 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-f[171910]: 2023-12-07T10:39:20.739+0000 7f7d92e44700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:39:21.011 INFO:journalctl@ceph.mds.f.smithi159.stdout:Dec 07 10:39:20 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-f[171910]: 2023-12-07T10:39:20.739+0000 7f7d92e44700 -1 mds.f *** got signal Terminated *** 2023-12-07T10:39:24.634 INFO:journalctl@ceph.mds.f.smithi159.stdout:Dec 07 10:39:24 smithi159 bash[182475]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-f 2023-12-07T10:39:24.641 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.f.service' 2023-12-07T10:39:24.676 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:39:24.676 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-12-07T10:39:24.677 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-12-07T10:39:24.677 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.i 2023-12-07T10:39:25.011 INFO:journalctl@ceph.mds.i.smithi159.stdout:Dec 07 10:39:24 smithi159 systemd[1]: Stopping Ceph mds.i for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:39:25.511 INFO:journalctl@ceph.mds.i.smithi159.stdout:Dec 07 10:39:25 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-i[172585]: 2023-12-07T10:39:25.079+0000 7f02d16f6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:39:25.511 INFO:journalctl@ceph.mds.i.smithi159.stdout:Dec 07 10:39:25 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-i[172585]: 2023-12-07T10:39:25.079+0000 7f02d16f6700 -1 mds.i *** got signal Terminated *** 2023-12-07T10:39:28.261 INFO:journalctl@ceph.mds.i.smithi159.stdout:Dec 07 10:39:28 smithi159 bash[182625]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-i 2023-12-07T10:39:28.312 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.i.service' 2023-12-07T10:39:28.345 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:39:28.345 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-12-07T10:39:28.346 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-12-07T10:39:28.346 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.l 2023-12-07T10:39:28.761 INFO:journalctl@ceph.mds.l.smithi159.stdout:Dec 07 10:39:28 smithi159 systemd[1]: Stopping Ceph mds.l for a4c642c6-94e8-11ee-95a2-87774f69a715... 2023-12-07T10:39:29.261 INFO:journalctl@ceph.mds.l.smithi159.stdout:Dec 07 10:39:28 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-l[171572]: 2023-12-07T10:39:28.770+0000 7f2e2e15d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-07T10:39:29.261 INFO:journalctl@ceph.mds.l.smithi159.stdout:Dec 07 10:39:28 smithi159 ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-l[171572]: 2023-12-07T10:39:28.770+0000 7f2e2e15d700 -1 mds.l *** got signal Terminated *** 2023-12-07T10:39:32.261 INFO:journalctl@ceph.mds.l.smithi159.stdout:Dec 07 10:39:32 smithi159 bash[182764]: ceph-a4c642c6-94e8-11ee-95a2-87774f69a715-mds-l 2023-12-07T10:39:32.297 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a4c642c6-94e8-11ee-95a2-87774f69a715@mds.l.service' 2023-12-07T10:39:32.332 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-07T10:39:32.332 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-12-07T10:39:32.333 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 --force --keep-logs 2023-12-07T10:39:32.770 INFO:teuthology.orchestra.run.smithi119.stdout:Deleting cluster with fsid: a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:39:36.091 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 --force --keep-logs 2023-12-07T10:39:36.433 INFO:teuthology.orchestra.run.smithi153.stdout:Deleting cluster with fsid: a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:39:37.986 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 --force --keep-logs 2023-12-07T10:39:38.326 INFO:teuthology.orchestra.run.smithi159.stdout:Deleting cluster with fsid: a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:39:39.904 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-07T10:39:39.982 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-07T10:39:40.018 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-07T10:39:40.047 INFO:tasks.cephadm:Archiving crash dumps... 2023-12-07T10:39:40.050 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/crash to /home/teuthworker/archive/vshankar-2023-12-07_09:28:49-fs-wip-vshankar-testing-20231127.102654-r2-distro-default-smithi/7481878/remote/smithi119/crash 2023-12-07T10:39:40.050 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/crash -- . 2023-12-07T10:39:40.130 INFO:teuthology.orchestra.run.smithi119.stderr:tar: /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/crash: Cannot open: No such file or directory 2023-12-07T10:39:40.130 INFO:teuthology.orchestra.run.smithi119.stderr:tar: Error is not recoverable: exiting now 2023-12-07T10:39:40.133 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/crash to /home/teuthworker/archive/vshankar-2023-12-07_09:28:49-fs-wip-vshankar-testing-20231127.102654-r2-distro-default-smithi/7481878/remote/smithi153/crash 2023-12-07T10:39:40.134 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/crash -- . 2023-12-07T10:39:40.185 INFO:teuthology.orchestra.run.smithi153.stderr:tar: /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/crash: Cannot open: No such file or directory 2023-12-07T10:39:40.185 INFO:teuthology.orchestra.run.smithi153.stderr:tar: Error is not recoverable: exiting now 2023-12-07T10:39:40.187 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/crash to /home/teuthworker/archive/vshankar-2023-12-07_09:28:49-fs-wip-vshankar-testing-20231127.102654-r2-distro-default-smithi/7481878/remote/smithi159/crash 2023-12-07T10:39:40.188 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/crash -- . 2023-12-07T10:39:40.222 INFO:teuthology.orchestra.run.smithi159.stderr:tar: /var/lib/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/crash: Cannot open: No such file or directory 2023-12-07T10:39:40.222 INFO:teuthology.orchestra.run.smithi159.stderr:tar: Error is not recoverable: exiting now 2023-12-07T10:39:40.223 INFO:tasks.cephadm:Checking cluster log for badness... 2023-12-07T10:39:40.223 DEBUG:teuthology.orchestra.run.smithi119:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-12-07T10:39:40.301 INFO:teuthology.orchestra.run.smithi119.stderr:grep: /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph.log: No such file or directory 2023-12-07T10:39:40.313 INFO:tasks.cephadm:Compressing logs... 2023-12-07T10:39:40.314 DEBUG:teuthology.orchestra.run.smithi119:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-07T10:39:40.318 DEBUG:teuthology.orchestra.run.smithi153:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-07T10:39:40.322 DEBUG:teuthology.orchestra.run.smithi159:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-07T10:39:40.359 INFO:teuthology.orchestra.run.smithi159.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-12-07T10:39:40.360 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-12-07T10:39:40.363 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.5.log 2023-12-07T10:39:40.363 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.2.log 2023-12-07T10:39:40.364 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.11.log 2023-12-07T10:39:40.364 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mgr.z.log 2023-12-07T10:39:40.364 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.8.log 2023-12-07T10:39:40.366 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.f.log 2023-12-07T10:39:40.366 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.c.log 2023-12-07T10:39:40.366 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.i.log 2023-12-07T10:39:40.367 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.f.log: /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mon.c.log 2023-12-07T10:39:40.367 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-volume.log 2023-12-07T10:39:40.367 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.l.log 2023-12-07T10:39:40.367 INFO:teuthology.orchestra.run.smithi153.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-12-07T10:39:40.367 INFO:teuthology.orchestra.run.smithi153.stderr:: No such file or directory 2023-12-07T10:39:40.371 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mon.c.log: /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-volume.log: /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.l.log: 92.2% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mgr.z.log.gz 2023-12-07T10:39:40.389 INFO:teuthology.orchestra.run.smithi119.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-12-07T10:39:40.389 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-12-07T10:39:40.389 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.10.log 2023-12-07T10:39:40.390 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.h.log 2023-12-07T10:39:40.391 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.e.log 2023-12-07T10:39:40.391 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.10.log: /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.4.log 2023-12-07T10:39:40.391 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.1.log 2023-12-07T10:39:40.391 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.k.log 2023-12-07T10:39:40.391 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-volume.log 2023-12-07T10:39:40.393 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.7.log 2023-12-07T10:39:40.393 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mon.b.log 2023-12-07T10:39:40.393 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.b.log 2023-12-07T10:39:40.393 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mgr.y.log 2023-12-07T10:39:40.393 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.0.log 2023-12-07T10:39:40.394 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.g.log 2023-12-07T10:39:40.395 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.a.log 2023-12-07T10:39:40.396 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.6.log 2023-12-07T10:39:40.397 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.3.log 2023-12-07T10:39:40.399 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.d.log 2023-12-07T10:39:40.399 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.b.log: /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mgr.y.log: 92.2% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mgr.y.log.gz 2023-12-07T10:39:40.400 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mgr.x.log 2023-12-07T10:39:40.400 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.9.log 2023-12-07T10:39:40.401 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-volume.log 2023-12-07T10:39:40.402 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.j.log 2023-12-07T10:39:40.403 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mon.a.log 2023-12-07T10:39:40.405 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2023-12-07T10:39:40.405 INFO:teuthology.orchestra.run.smithi119.stderr: 94.6% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.d.log.gz 2023-12-07T10:39:40.408 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.log: 93.9% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2023-12-07T10:39:40.409 INFO:teuthology.orchestra.run.smithi119.stderr: 94.5% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.j.log.gz 2023-12-07T10:39:40.462 INFO:teuthology.orchestra.run.smithi119.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-12-07T10:39:40.642 INFO:teuthology.orchestra.run.smithi153.stderr: 91.2% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-volume.log.gz 2023-12-07T10:39:40.697 INFO:teuthology.orchestra.run.smithi153.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-12-07T10:39:41.043 INFO:teuthology.orchestra.run.smithi159.stderr: 91.9% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mon.c.log.gz 2023-12-07T10:39:41.070 INFO:teuthology.orchestra.run.smithi119.stderr: 91.1% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-volume.log.gz 2023-12-07T10:39:41.364 INFO:teuthology.orchestra.run.smithi119.stderr: 89.7% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mgr.x.log.gz 2023-12-07T10:39:41.410 INFO:teuthology.orchestra.run.smithi159.stderr: 91.2% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-volume.log.gz 2023-12-07T10:39:41.932 INFO:teuthology.orchestra.run.smithi159.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-12-07T10:39:42.564 INFO:teuthology.orchestra.run.smithi153.stderr: 93.1% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mon.b.log.gz 2023-12-07T10:39:42.689 INFO:teuthology.orchestra.run.smithi119.stderr: 90.5% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mon.a.log.gz 2023-12-07T10:39:44.925 INFO:teuthology.orchestra.run.smithi159.stderr: 95.3% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.i.log.gz 2023-12-07T10:39:44.984 INFO:teuthology.orchestra.run.smithi159.stderr: 95.4% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.l.log.gz 2023-12-07T10:39:45.704 INFO:teuthology.orchestra.run.smithi153.stderr: 95.1% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.k.log.gz 2023-12-07T10:39:46.534 INFO:teuthology.orchestra.run.smithi119.stderr: 95.4% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.a.log.gz 2023-12-07T10:39:47.876 INFO:teuthology.orchestra.run.smithi153.stderr: 95.7% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.h.log.gz 2023-12-07T10:40:12.670 INFO:teuthology.orchestra.run.smithi159.stderr: 91.4% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.11.log.gz 2023-12-07T10:40:17.398 INFO:teuthology.orchestra.run.smithi153.stderr: 91.4% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.10.log.gz 2023-12-07T10:40:19.354 INFO:teuthology.orchestra.run.smithi153.stderr: 91.6% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.1.log.gz 2023-12-07T10:40:21.825 INFO:teuthology.orchestra.run.smithi119.stderr: 91.5% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.0.log.gz 2023-12-07T10:40:26.986 INFO:teuthology.orchestra.run.smithi119.stderr: 91.2% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.6.log.gz 2023-12-07T10:40:32.401 INFO:teuthology.orchestra.run.smithi119.stderr: 91.0% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.9.log.gz 2023-12-07T10:40:35.947 INFO:teuthology.orchestra.run.smithi159.stderr: 91.2% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.2.log.gz 2023-12-07T10:40:36.232 INFO:teuthology.orchestra.run.smithi119.stderr: 91.2% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.3.log.gz 2023-12-07T10:40:36.270 INFO:teuthology.orchestra.run.smithi159.stderr: 91.1% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.5.log.gz 2023-12-07T10:40:40.544 INFO:teuthology.orchestra.run.smithi159.stderr: 91.1% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.8.log.gz 2023-12-07T10:40:41.264 INFO:teuthology.orchestra.run.smithi153.stderr: 91.2% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.4.log.gz 2023-12-07T10:40:42.701 INFO:teuthology.orchestra.run.smithi153.stderr: 91.3% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-osd.7.log.gz 2023-12-07T10:40:52.096 INFO:teuthology.orchestra.run.smithi159.stderr: 93.2% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.c.log.gz 2023-12-07T10:40:55.065 INFO:teuthology.orchestra.run.smithi159.stderr: 93.4% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.f.log.gz 2023-12-07T10:40:55.067 INFO:teuthology.orchestra.run.smithi159.stderr: 2023-12-07T10:40:55.067 INFO:teuthology.orchestra.run.smithi159.stderr:real 1m14.729s 2023-12-07T10:40:55.067 INFO:teuthology.orchestra.run.smithi159.stderr:user 4m32.303s 2023-12-07T10:40:55.068 INFO:teuthology.orchestra.run.smithi159.stderr:sys 0m8.892s 2023-12-07T10:41:08.814 INFO:teuthology.orchestra.run.smithi153.stderr: 93.6% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.e.log.gz 2023-12-07T10:41:10.898 INFO:teuthology.orchestra.run.smithi153.stderr: 93.3% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.b.log.gz 2023-12-07T10:41:10.931 INFO:teuthology.orchestra.run.smithi153.stderr: 2023-12-07T10:41:10.932 INFO:teuthology.orchestra.run.smithi153.stderr:real 1m30.594s 2023-12-07T10:41:10.932 INFO:teuthology.orchestra.run.smithi153.stderr:user 4m19.568s 2023-12-07T10:41:10.932 INFO:teuthology.orchestra.run.smithi153.stderr:sys 0m9.364s 2023-12-07T10:41:55.853 INFO:teuthology.orchestra.run.smithi119.stderr: 93.4% -- replaced with /var/log/ceph/a4c642c6-94e8-11ee-95a2-87774f69a715/ceph-mds.g.log.gz 2023-12-07T10:41:55.857 INFO:teuthology.orchestra.run.smithi119.stderr: 2023-12-07T10:41:55.857 INFO:teuthology.orchestra.run.smithi119.stderr:real 2m15.502s 2023-12-07T10:41:55.857 INFO:teuthology.orchestra.run.smithi119.stderr:user 4m7.443s 2023-12-07T10:41:55.857 INFO:teuthology.orchestra.run.smithi119.stderr:sys 0m22.064s 2023-12-07T10:41:55.859 INFO:tasks.cephadm:Archiving logs... 2023-12-07T10:41:55.859 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-12-07_09:28:49-fs-wip-vshankar-testing-20231127.102654-r2-distro-default-smithi/7481878/remote/smithi119/log 2023-12-07T10:41:55.860 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-07T10:42:34.166 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-12-07_09:28:49-fs-wip-vshankar-testing-20231127.102654-r2-distro-default-smithi/7481878/remote/smithi153/log 2023-12-07T10:42:34.168 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-07T10:43:14.002 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-12-07_09:28:49-fs-wip-vshankar-testing-20231127.102654-r2-distro-default-smithi/7481878/remote/smithi159/log 2023-12-07T10:43:14.003 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-07T10:43:51.738 INFO:tasks.cephadm:Removing cluster... 2023-12-07T10:43:51.739 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 --force 2023-12-07T10:43:52.198 INFO:teuthology.orchestra.run.smithi119.stdout:Deleting cluster with fsid: a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:43:53.764 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 --force 2023-12-07T10:43:54.475 INFO:teuthology.orchestra.run.smithi153.stdout:Deleting cluster with fsid: a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:43:55.314 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a4c642c6-94e8-11ee-95a2-87774f69a715 --force 2023-12-07T10:43:55.663 INFO:teuthology.orchestra.run.smithi159.stdout:Deleting cluster with fsid: a4c642c6-94e8-11ee-95a2-87774f69a715 2023-12-07T10:43:56.477 INFO:tasks.cephadm:Removing cephadm ... 2023-12-07T10:43:56.477 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-07T10:43:56.526 DEBUG:teuthology.orchestra.run.smithi153:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-07T10:43:56.578 DEBUG:teuthology.orchestra.run.smithi159:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-07T10:43:56.595 INFO:tasks.cephadm:Teardown complete 2023-12-07T10:43:56.596 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-12-07T10:43:56.608 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-12-07T10:43:56.608 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-07T10:43:56.613 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-07T10:43:56.615 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-07T10:43:56.907 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-12-07T10:43:56.917 INFO:teuthology.task.clock:Checking final clock skew... 2023-12-07T10:43:56.917 DEBUG:teuthology.orchestra.run.smithi119:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-07T10:43:56.922 DEBUG:teuthology.orchestra.run.smithi153:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-07T10:43:56.925 DEBUG:teuthology.orchestra.run.smithi159:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-07T10:43:56.941 INFO:teuthology.orchestra.run.smithi159.stderr:bash: ntpq: command not found 2023-12-07T10:43:56.941 INFO:teuthology.orchestra.run.smithi153.stderr:bash: ntpq: command not found 2023-12-07T10:43:56.960 INFO:teuthology.orchestra.run.smithi119.stderr:bash: ntpq: command not found 2023-12-07T10:43:56.970 INFO:teuthology.orchestra.run.smithi153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-07T10:43:56.970 INFO:teuthology.orchestra.run.smithi153.stdout:=============================================================================== 2023-12-07T10:43:56.971 INFO:teuthology.orchestra.run.smithi153.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 1 +1284us[+1284us] +/- 76ms 2023-12-07T10:43:56.971 INFO:teuthology.orchestra.run.smithi153.stdout:^* hv02.front.sepia.ceph.com 3 6 377 3 -444us[ -435us] +/- 58ms 2023-12-07T10:43:56.971 INFO:teuthology.orchestra.run.smithi153.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 -638us[ -629us] +/- 69ms 2023-12-07T10:43:56.971 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-12-07T10:43:56.975 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-07T10:43:56.975 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2023-12-07T10:43:56.975 INFO:teuthology.orchestra.run.smithi159.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 5 +1347us[+1374us] +/- 76ms 2023-12-07T10:43:56.975 INFO:teuthology.orchestra.run.smithi159.stdout:^* hv02.front.sepia.ceph.com 3 7 377 0 -409us[ -382us] +/- 58ms 2023-12-07T10:43:56.975 INFO:teuthology.orchestra.run.smithi159.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 6 -609us[ -582us] +/- 69ms 2023-12-07T10:43:56.975 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-12-07T10:43:56.995 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-07T10:43:56.995 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2023-12-07T10:43:56.996 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 3 +1299us[+1299us] +/- 76ms 2023-12-07T10:43:56.996 INFO:teuthology.orchestra.run.smithi119.stdout:^* hv02.front.sepia.ceph.com 3 6 377 4 -445us[ -423us] +/- 58ms 2023-12-07T10:43:56.996 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 1 -601us[ -601us] +/- 69ms 2023-12-07T10:43:56.996 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-12-07T10:43:56.997 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-12-07T10:43:57.008 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-12-07T10:43:57.008 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-12-07T10:43:57.018 DEBUG:teuthology.orchestra.run.smithi119:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-07T10:43:57.306 DEBUG:teuthology.orchestra.run.smithi153:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-07T10:43:57.452 DEBUG:teuthology.orchestra.run.smithi159:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-07T10:43:57.554 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-07T10:43:57.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-07T10:43:57.631 DEBUG:teuthology.orchestra.run.smithi153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-07T10:43:57.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-07T10:43:57.668 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-07T10:43:57.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-07T10:43:57.700 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-12-07T10:43:57.711 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-12-07T10:43:57.721 INFO:teuthology.task.internal:Duration was 3022.680084 seconds 2023-12-07T10:43:57.722 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-12-07T10:43:57.732 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-12-07T10:43:57.732 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-07T10:43:57.737 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-07T10:43:57.739 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-07T10:43:57.803 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-07T10:43:57.843 INFO:teuthology.orchestra.run.smithi153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-07T10:43:57.881 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-07T10:43:58.301 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-12-07T10:43:58.302 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi119.front.sepia.ceph.com 2023-12-07T10:43:58.302 DEBUG:teuthology.orchestra.run.smithi119:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-07T10:43:58.361 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi153.front.sepia.ceph.com 2023-12-07T10:43:58.362 DEBUG:teuthology.orchestra.run.smithi153:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-07T10:43:58.407 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi159.front.sepia.ceph.com 2023-12-07T10:43:58.408 DEBUG:teuthology.orchestra.run.smithi159:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-07T10:43:58.442 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-12-07T10:43:58.443 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-07T10:43:58.447 DEBUG:teuthology.orchestra.run.smithi153:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-07T10:43:58.450 DEBUG:teuthology.orchestra.run.smithi159:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-07T10:43:58.548 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-12-07T10:43:58.548 DEBUG:teuthology.orchestra.run.smithi119:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-07T10:43:58.553 DEBUG:teuthology.orchestra.run.smithi153:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-07T10:43:58.555 DEBUG:teuthology.orchestra.run.smithi159:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-07T10:43:58.716 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-12-07T10:43:58.727 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-12-07T10:43:58.728 DEBUG:teuthology.orchestra.run.smithi119:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-07T10:43:58.798 DEBUG:teuthology.orchestra.run.smithi153:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-07T10:43:58.835 DEBUG:teuthology.orchestra.run.smithi159:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-07T10:43:58.863 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-12-07T10:43:58.875 DEBUG:teuthology.orchestra.run.smithi119:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-07T10:43:58.880 DEBUG:teuthology.orchestra.run.smithi153:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-07T10:43:58.883 DEBUG:teuthology.orchestra.run.smithi159:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-07T10:43:58.929 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = core 2023-12-07T10:43:58.936 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = core 2023-12-07T10:43:58.952 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = core 2023-12-07T10:43:58.988 DEBUG:teuthology.orchestra.run.smithi119:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-07T10:43:59.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-07T10:43:59.030 DEBUG:teuthology.orchestra.run.smithi153:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-07T10:43:59.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-07T10:43:59.046 DEBUG:teuthology.orchestra.run.smithi159:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-07T10:43:59.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-07T10:43:59.062 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-12-07T10:43:59.074 INFO:teuthology.task.internal:Transferring archived files... 2023-12-07T10:43:59.075 DEBUG:teuthology.misc:Transferring archived files from smithi119:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-12-07_09:28:49-fs-wip-vshankar-testing-20231127.102654-r2-distro-default-smithi/7481878/remote/smithi119 2023-12-07T10:43:59.075 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-07T10:43:59.159 DEBUG:teuthology.misc:Transferring archived files from smithi153:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-12-07_09:28:49-fs-wip-vshankar-testing-20231127.102654-r2-distro-default-smithi/7481878/remote/smithi153 2023-12-07T10:43:59.159 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-07T10:43:59.198 DEBUG:teuthology.misc:Transferring archived files from smithi159:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-12-07_09:28:49-fs-wip-vshankar-testing-20231127.102654-r2-distro-default-smithi/7481878/remote/smithi159 2023-12-07T10:43:59.198 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-07T10:43:59.238 INFO:teuthology.task.internal:Removing archive directory... 2023-12-07T10:43:59.238 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-07T10:43:59.243 DEBUG:teuthology.orchestra.run.smithi153:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-07T10:43:59.245 DEBUG:teuthology.orchestra.run.smithi159:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-07T10:43:59.288 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-12-07T10:43:59.302 INFO:teuthology.task.internal:Not uploading archives. 2023-12-07T10:43:59.302 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-12-07T10:43:59.313 INFO:teuthology.task.internal:Tidying up after the test... 2023-12-07T10:43:59.313 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-07T10:43:59.326 DEBUG:teuthology.orchestra.run.smithi153:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-07T10:43:59.329 DEBUG:teuthology.orchestra.run.smithi159:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-07T10:43:59.343 INFO:teuthology.orchestra.run.smithi153.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 7 10:43 /home/ubuntu/cephtest 2023-12-07T10:43:59.344 INFO:teuthology.orchestra.run.smithi159.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 7 10:43 /home/ubuntu/cephtest 2023-12-07T10:43:59.365 INFO:teuthology.orchestra.run.smithi119.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 7 10:43 /home/ubuntu/cephtest 2023-12-07T10:43:59.369 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-12-07T10:43:59.381 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-12-07T10:43:59.412 INFO:teuthology.nuke:Checking targets against current locks 2023-12-07T10:43:59.440 DEBUG:teuthology.nuke:shortname: smithi119 2023-12-07T10:43:59.440 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-07T10:43:59.464 DEBUG:teuthology.nuke:shortname: smithi153 2023-12-07T10:43:59.464 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-07T10:43:59.469 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-07_09:28:49-fs-wip-vshankar-testing-20231127.102654-r2-distro-default-smithi/7481878', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-07 09:44:39.115502', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-07T10:43:59.489 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi153.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-07_09:28:49-fs-wip-vshankar-testing-20231127.102654-r2-distro-default-smithi/7481878', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-07 09:44:39.117617', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-07T10:43:59.492 DEBUG:teuthology.nuke:shortname: smithi159 2023-12-07T10:43:59.493 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-07T10:43:59.515 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-07_09:28:49-fs-wip-vshankar-testing-20231127.102654-r2-distro-default-smithi/7481878', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-07 09:44:39.116665', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-07T10:43:59.524 INFO:teuthology.orchestra.console.smithi119:Power off 2023-12-07T10:43:59.525 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-07T10:43:59.546 DEBUG:teuthology.orchestra.console.smithi119:power off output: Chassis Power Control: Down/Off 2023-12-07T10:43:59.547 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-07T10:43:59.563 INFO:teuthology.orchestra.console.smithi153:Power off 2023-12-07T10:43:59.563 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-07T10:43:59.577 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2023-12-07T10:43:59.586 DEBUG:teuthology.orchestra.console.smithi153:power off output: Chassis Power Control: Down/Off 2023-12-07T10:43:59.586 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-07T10:43:59.605 INFO:teuthology.orchestra.console.smithi159:Power off 2023-12-07T10:43:59.606 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-07T10:43:59.616 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on 2023-12-07T10:43:59.631 DEBUG:teuthology.orchestra.console.smithi159:power off output: Chassis Power Control: Down/Off 2023-12-07T10:43:59.631 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-07T10:43:59.655 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on 2023-12-07T10:44:03.578 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-07T10:44:03.617 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-07T10:44:03.656 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-07T10:44:03.691 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2023-12-07T10:44:03.728 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on 2023-12-07T10:44:03.767 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on 2023-12-07T10:44:07.696 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-07T10:44:07.730 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-07T10:44:07.768 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-07T10:44:07.808 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is off 2023-12-07T10:44:07.840 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is off 2023-12-07T10:44:07.879 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off 2023-12-07T10:44:07.910 INFO:teuthology.orchestra.console.smithi119:Power off completed 2023-12-07T10:44:07.942 INFO:teuthology.orchestra.console.smithi153:Power off completed 2023-12-07T10:44:07.980 INFO:teuthology.orchestra.console.smithi159:Power off completed 2023-12-07T10:44:08.087 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 9471594ae3a4c6992c4327df9a789df8f2b6aeae description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} duration: 3022.68008351326 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi119 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3bd0bfa458b4399e7266c56d8215a2cf3a6091a0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_vshankar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=6c49b911ab864ff8ad53779d28ab3fed status: fail success: false 2023-12-07T10:44:08.087 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-07T10:44:08.166 INFO:teuthology.run:FAIL